[PATCH v2 0/5] KVM: selftests: arm64 exception handling and debug test

[PATCH v2 0/5] KVM: selftests: arm64 exception handling and debug test

[Ricardo Koller]

Hi,

These patches add a debug exception test in aarch64 KVM selftests while
also adding basic exception handling support.

The structure of the exception handling is based on its x86 counterpart.
Tests use the same calls to initialize exception handling and both
architectures allow tests to override the handler for a particular
vector, or (vector, ec) for synchronous exceptions in the arm64 case.

The debug test is similar to x86_64/debug_regs, except that the x86 one
controls the debugging from outside the VM. This proposed arm64 test
controls and handles debug exceptions from the inside.

Thanks,
Ricardo

v1 -> v2:

Addressed comments from Andrew and Marc (thank you very much):
- rename vm_handle_exception in all tests.
- introduce UCALL_UNHANDLED in x86 first.
- move GUEST_ASSERT_EQ to common utils header.
- handle sync and other exceptions separately: use two tables (like
  kvm-unit-tests).
- add two separate functions for installing sync versus other exceptions
- changes in handlers.S: use the same layout as user_pt_regs, treat the
  EL1t vectors as invalid, refactor the vector table creation to not use
  manual numbering, add comments, remove LR from the stored registers.
- changes in debug-exceptions.c: remove unused headers, use the common
  GUEST_ASSERT_EQ, use vcpu_run instead of _vcpu_run.
- changes in processor.h: write_sysreg with support for xzr, replace EL1
  with current in macro names, define ESR_EC_MASK as ESR_EC_NUM-1.

Ricardo Koller (5):
  KVM: selftests: Rename vm_handle_exception
  KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector
    reporting
  KVM: selftests: Move GUEST_ASSERT_EQ to utils header
  KVM: selftests: Add exception handling support for aarch64
  KVM: selftests: Add aarch64/debug-exceptions test

 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   3 +-
 .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
 .../selftests/kvm/include/aarch64/processor.h |  90 ++++++-
 .../testing/selftests/kvm/include/kvm_util.h  |  10 +
 .../selftests/kvm/include/x86_64/processor.h  |   4 +-
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++
 .../selftests/kvm/lib/x86_64/processor.c      |  19 +-
 .../selftests/kvm/x86_64/kvm_pv_test.c        |   2 +-
 .../selftests/kvm/x86_64/tsc_msrs_test.c      |   9 -
 .../kvm/x86_64/userspace_msr_exit_test.c      |   8 +-
 .../selftests/kvm/x86_64/xapic_ipi_test.c     |   2 +-
 13 files changed, 611 insertions(+), 35 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S

-- 
2.31.1.527.g47e6f16901-goog

[PATCH v2 1/5] KVM: selftests: Rename vm_handle_exception

[Ricardo Koller]

Rename the vm_handle_exception function to a name that indicates more
clearly that it installs something: vm_install_vector_handler.

Suggested-by: Marc Zyngier <maz@kernel.org>
Suggested-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Ricardo Koller <ricarkol@google.com>
---
 tools/testing/selftests/kvm/include/x86_64/processor.h    | 2 +-
 tools/testing/selftests/kvm/lib/x86_64/processor.c        | 4 ++--
 tools/testing/selftests/kvm/x86_64/kvm_pv_test.c          | 2 +-
 .../selftests/kvm/x86_64/userspace_msr_exit_test.c        | 8 ++++----
 tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c       | 2 +-
 5 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
index 0b30b4e15c38..12889d3e8948 100644
--- a/tools/testing/selftests/kvm/include/x86_64/processor.h
+++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
@@ -391,7 +391,7 @@ struct ex_regs {
 
 void vm_init_descriptor_tables(struct kvm_vm *vm);
 void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
-void vm_handle_exception(struct kvm_vm *vm, int vector,
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
 			void (*handler)(struct ex_regs *));
 
 /*
diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c
index a8906e60a108..e156061263a6 100644
--- a/tools/testing/selftests/kvm/lib/x86_64/processor.c
+++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c
@@ -1250,8 +1250,8 @@ void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
 	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
 }
 
-void vm_handle_exception(struct kvm_vm *vm, int vector,
-			 void (*handler)(struct ex_regs *))
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			       void (*handler)(struct ex_regs *))
 {
 	vm_vaddr_t *handlers = (vm_vaddr_t *)addr_gva2hva(vm, vm->handlers);
 
diff --git a/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c b/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c
index 732b244d6956..5ae5f748723a 100644
--- a/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c
+++ b/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c
@@ -227,7 +227,7 @@ int main(void)
 
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, VCPU_ID);
-	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
+	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
 
 	enter_guest(vm);
 	kvm_vm_free(vm);
diff --git a/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c b/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c
index 72c0d0797522..20c373e2d329 100644
--- a/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c
+++ b/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c
@@ -574,7 +574,7 @@ static void test_msr_filter_allow(void) {
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, VCPU_ID);
 
-	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
+	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
 
 	/* Process guest code userspace exits. */
 	run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
@@ -588,12 +588,12 @@ static void test_msr_filter_allow(void) {
 	run_guest_then_process_wrmsr(vm, MSR_NON_EXISTENT);
 	run_guest_then_process_rdmsr(vm, MSR_NON_EXISTENT);
 
-	vm_handle_exception(vm, UD_VECTOR, guest_ud_handler);
+	vm_install_vector_handler(vm, UD_VECTOR, guest_ud_handler);
 	run_guest(vm);
-	vm_handle_exception(vm, UD_VECTOR, NULL);
+	vm_install_vector_handler(vm, UD_VECTOR, NULL);
 
 	if (process_ucall(vm) != UCALL_DONE) {
-		vm_handle_exception(vm, GP_VECTOR, guest_fep_gp_handler);
+		vm_install_vector_handler(vm, GP_VECTOR, guest_fep_gp_handler);
 
 		/* Process emulated rdmsr and wrmsr instructions. */
 		run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
diff --git a/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c b/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c
index 2f964cdc273c..ded70ff465d5 100644
--- a/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c
+++ b/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c
@@ -462,7 +462,7 @@ int main(int argc, char *argv[])
 
 	vm_init_descriptor_tables(vm);
 	vcpu_init_descriptor_tables(vm, HALTER_VCPU_ID);
-	vm_handle_exception(vm, IPI_VECTOR, guest_ipi_handler);
+	vm_install_vector_handler(vm, IPI_VECTOR, guest_ipi_handler);
 
 	virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA, 0);
 
-- 
2.31.1.527.g47e6f16901-goog

Re: [PATCH v2 1/5] KVM: selftests: Rename vm_handle_exception

[Andrew Jones]

On Fri, Apr 30, 2021 at 04:24:03PM -0700, Ricardo Koller wrote:
> Rename the vm_handle_exception function to a name that indicates more
> clearly that it installs something: vm_install_vector_handler.
> 
> Suggested-by: Marc Zyngier <maz@kernel.org>
> Suggested-by: Andrew Jones <drjones@redhat.com>
> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> ---
>  tools/testing/selftests/kvm/include/x86_64/processor.h    | 2 +-
>  tools/testing/selftests/kvm/lib/x86_64/processor.c        | 4 ++--
>  tools/testing/selftests/kvm/x86_64/kvm_pv_test.c          | 2 +-
>  .../selftests/kvm/x86_64/userspace_msr_exit_test.c        | 8 ++++----
>  tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c       | 2 +-
>  5 files changed, 9 insertions(+), 9 deletions(-)
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

Re: [PATCH v2 1/5] KVM: selftests: Rename vm_handle_exception

[Auger Eric]

Hi Ricardo,

On 5/1/21 1:24 AM, Ricardo Koller wrote:
> Rename the vm_handle_exception function to a name that indicates more
> clearly that it installs something: vm_install_vector_handler.
> 
> Suggested-by: Marc Zyngier <maz@kernel.org>
> Suggested-by: Andrew Jones <drjones@redhat.com>
> Signed-off-by: Ricardo Koller <ricarkol@google.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Thanks

Eric

> ---
>  tools/testing/selftests/kvm/include/x86_64/processor.h    | 2 +-
>  tools/testing/selftests/kvm/lib/x86_64/processor.c        | 4 ++--
>  tools/testing/selftests/kvm/x86_64/kvm_pv_test.c          | 2 +-
>  .../selftests/kvm/x86_64/userspace_msr_exit_test.c        | 8 ++++----
>  tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c       | 2 +-
>  5 files changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
> index 0b30b4e15c38..12889d3e8948 100644
> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
> @@ -391,7 +391,7 @@ struct ex_regs {
>  
>  void vm_init_descriptor_tables(struct kvm_vm *vm);
>  void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
> -void vm_handle_exception(struct kvm_vm *vm, int vector,
> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
>  			void (*handler)(struct ex_regs *));
>  
>  /*
> diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c
> index a8906e60a108..e156061263a6 100644
> --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c
> +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c
> @@ -1250,8 +1250,8 @@ void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
>  	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
>  }
>  
> -void vm_handle_exception(struct kvm_vm *vm, int vector,
> -			 void (*handler)(struct ex_regs *))
> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
> +			       void (*handler)(struct ex_regs *))
>  {
>  	vm_vaddr_t *handlers = (vm_vaddr_t *)addr_gva2hva(vm, vm->handlers);
>  
> diff --git a/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c b/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c
> index 732b244d6956..5ae5f748723a 100644
> --- a/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c
> +++ b/tools/testing/selftests/kvm/x86_64/kvm_pv_test.c
> @@ -227,7 +227,7 @@ int main(void)
>  
>  	vm_init_descriptor_tables(vm);
>  	vcpu_init_descriptor_tables(vm, VCPU_ID);
> -	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
> +	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
>  
>  	enter_guest(vm);
>  	kvm_vm_free(vm);
> diff --git a/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c b/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c
> index 72c0d0797522..20c373e2d329 100644
> --- a/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c
> +++ b/tools/testing/selftests/kvm/x86_64/userspace_msr_exit_test.c
> @@ -574,7 +574,7 @@ static void test_msr_filter_allow(void) {
>  	vm_init_descriptor_tables(vm);
>  	vcpu_init_descriptor_tables(vm, VCPU_ID);
>  
> -	vm_handle_exception(vm, GP_VECTOR, guest_gp_handler);
> +	vm_install_vector_handler(vm, GP_VECTOR, guest_gp_handler);
>  
>  	/* Process guest code userspace exits. */
>  	run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
> @@ -588,12 +588,12 @@ static void test_msr_filter_allow(void) {
>  	run_guest_then_process_wrmsr(vm, MSR_NON_EXISTENT);
>  	run_guest_then_process_rdmsr(vm, MSR_NON_EXISTENT);
>  
> -	vm_handle_exception(vm, UD_VECTOR, guest_ud_handler);
> +	vm_install_vector_handler(vm, UD_VECTOR, guest_ud_handler);
>  	run_guest(vm);
> -	vm_handle_exception(vm, UD_VECTOR, NULL);
> +	vm_install_vector_handler(vm, UD_VECTOR, NULL);
>  
>  	if (process_ucall(vm) != UCALL_DONE) {
> -		vm_handle_exception(vm, GP_VECTOR, guest_fep_gp_handler);
> +		vm_install_vector_handler(vm, GP_VECTOR, guest_fep_gp_handler);
>  
>  		/* Process emulated rdmsr and wrmsr instructions. */
>  		run_guest_then_process_rdmsr(vm, MSR_IA32_XSS);
> diff --git a/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c b/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c
> index 2f964cdc273c..ded70ff465d5 100644
> --- a/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c
> +++ b/tools/testing/selftests/kvm/x86_64/xapic_ipi_test.c
> @@ -462,7 +462,7 @@ int main(int argc, char *argv[])
>  
>  	vm_init_descriptor_tables(vm);
>  	vcpu_init_descriptor_tables(vm, HALTER_VCPU_ID);
> -	vm_handle_exception(vm, IPI_VECTOR, guest_ipi_handler);
> +	vm_install_vector_handler(vm, IPI_VECTOR, guest_ipi_handler);
>  
>  	virt_pg_map(vm, APIC_DEFAULT_GPA, APIC_DEFAULT_GPA, 0);
>  
> 

[PATCH v2 2/5] KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector reporting

[Ricardo Koller]

x86, the only arch implementing exception handling, reports unhandled
vectors using port IO at a specific port number. This replicates what
ucall already does.

Introduce a new ucall type, UCALL_UNHANDLED, for guests to report
unhandled exceptions. Then replace the x86 unhandled vector exception
reporting to use it instead of port IO.  This new ucall type will be
used in the next commits by arm64 to report unhandled vectors as well.

Tested: Forcing a page fault in the ./x86_64/xapic_ipi_test
	halter_guest_code() shows this:

	$ ./x86_64/xapic_ipi_test
	...
	  Unexpected vectored event in guest (vector:0xe)

Signed-off-by: Ricardo Koller <ricarkol@google.com>
---
 tools/testing/selftests/kvm/include/kvm_util.h    |  1 +
 .../selftests/kvm/include/x86_64/processor.h      |  2 --
 .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++---------
 3 files changed, 7 insertions(+), 11 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index bea4644d645d..7880929ea548 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -347,6 +347,7 @@ enum {
 	UCALL_SYNC,
 	UCALL_ABORT,
 	UCALL_DONE,
+	UCALL_UNHANDLED,
 };
 
 #define UCALL_MAX_ARGS 6
diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
index 12889d3e8948..ff4da2f95b13 100644
--- a/tools/testing/selftests/kvm/include/x86_64/processor.h
+++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
@@ -53,8 +53,6 @@
 #define CPUID_PKU		(1ul << 3)
 #define CPUID_LA57		(1ul << 16)
 
-#define UNEXPECTED_VECTOR_PORT 0xfff0u
-
 /* General Registers in 64-Bit Mode */
 struct gpr64_regs {
 	u64 rax;
diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c
index e156061263a6..96e2bd9d66eb 100644
--- a/tools/testing/selftests/kvm/lib/x86_64/processor.c
+++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c
@@ -1207,7 +1207,7 @@ static void set_idt_entry(struct kvm_vm *vm, int vector, unsigned long addr,
 
 void kvm_exit_unexpected_vector(uint32_t value)
 {
-	outl(UNEXPECTED_VECTOR_PORT, value);
+	ucall(UCALL_UNHANDLED, 1, value);
 }
 
 void route_exception(struct ex_regs *regs)
@@ -1260,16 +1260,13 @@ void vm_install_vector_handler(struct kvm_vm *vm, int vector,
 
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
-	if (vcpu_state(vm, vcpuid)->exit_reason == KVM_EXIT_IO
-		&& vcpu_state(vm, vcpuid)->io.port == UNEXPECTED_VECTOR_PORT
-		&& vcpu_state(vm, vcpuid)->io.size == 4) {
-		/* Grab pointer to io data */
-		uint32_t *data = (void *)vcpu_state(vm, vcpuid)
-			+ vcpu_state(vm, vcpuid)->io.data_offset;
+	struct ucall uc;
 
+	if (get_ucall(vm, vcpuid, &uc) == UCALL_UNHANDLED) {
+		uint64_t vector = uc.args[0];
 		TEST_ASSERT(false,
-			    "Unexpected vectored event in guest (vector:0x%x)",
-			    *data);
+			    "Unexpected vectored event in guest (vector:0x%lx)",
+			    vector);
 	}
 }
 
-- 
2.31.1.527.g47e6f16901-goog

Re: [PATCH v2 2/5] KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector reporting

[Andrew Jones]

On Fri, Apr 30, 2021 at 04:24:04PM -0700, Ricardo Koller wrote:
> x86, the only arch implementing exception handling, reports unhandled
> vectors using port IO at a specific port number. This replicates what
> ucall already does.
> 
> Introduce a new ucall type, UCALL_UNHANDLED, for guests to report
> unhandled exceptions. Then replace the x86 unhandled vector exception
> reporting to use it instead of port IO.  This new ucall type will be
> used in the next commits by arm64 to report unhandled vectors as well.
> 
> Tested: Forcing a page fault in the ./x86_64/xapic_ipi_test
> 	halter_guest_code() shows this:
> 
> 	$ ./x86_64/xapic_ipi_test
> 	...
> 	  Unexpected vectored event in guest (vector:0xe)
> 
> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> ---
>  tools/testing/selftests/kvm/include/kvm_util.h    |  1 +
>  .../selftests/kvm/include/x86_64/processor.h      |  2 --
>  .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++---------
>  3 files changed, 7 insertions(+), 11 deletions(-)
> 
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index bea4644d645d..7880929ea548 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -347,6 +347,7 @@ enum {
>  	UCALL_SYNC,
>  	UCALL_ABORT,
>  	UCALL_DONE,
> +	UCALL_UNHANDLED,
>  };
>  
>  #define UCALL_MAX_ARGS 6
> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
> index 12889d3e8948..ff4da2f95b13 100644
> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
> @@ -53,8 +53,6 @@
>  #define CPUID_PKU		(1ul << 3)
>  #define CPUID_LA57		(1ul << 16)
>  
> -#define UNEXPECTED_VECTOR_PORT 0xfff0u
> -
>  /* General Registers in 64-Bit Mode */
>  struct gpr64_regs {
>  	u64 rax;
> diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c
> index e156061263a6..96e2bd9d66eb 100644
> --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c
> +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c
> @@ -1207,7 +1207,7 @@ static void set_idt_entry(struct kvm_vm *vm, int vector, unsigned long addr,
>  
>  void kvm_exit_unexpected_vector(uint32_t value)
>  {
> -	outl(UNEXPECTED_VECTOR_PORT, value);
> +	ucall(UCALL_UNHANDLED, 1, value);
>  }
>  
>  void route_exception(struct ex_regs *regs)
> @@ -1260,16 +1260,13 @@ void vm_install_vector_handler(struct kvm_vm *vm, int vector,
>  
>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
>  {
> -	if (vcpu_state(vm, vcpuid)->exit_reason == KVM_EXIT_IO
> -		&& vcpu_state(vm, vcpuid)->io.port == UNEXPECTED_VECTOR_PORT
> -		&& vcpu_state(vm, vcpuid)->io.size == 4) {
> -		/* Grab pointer to io data */
> -		uint32_t *data = (void *)vcpu_state(vm, vcpuid)
> -			+ vcpu_state(vm, vcpuid)->io.data_offset;
> +	struct ucall uc;
>  
> +	if (get_ucall(vm, vcpuid, &uc) == UCALL_UNHANDLED) {
> +		uint64_t vector = uc.args[0];
>  		TEST_ASSERT(false,
> -			    "Unexpected vectored event in guest (vector:0x%x)",
> -			    *data);
> +			    "Unexpected vectored event in guest (vector:0x%lx)",
> +			    vector);

nit: Could have changed this TEST_ASSERT(false, ...) to TEST_FAIL while
touching it.

>  	}
>  }
>  
> -- 
> 2.31.1.527.g47e6f16901-goog
>

Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks,
drew

Re: [PATCH v2 2/5] KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector reporting

[Auger Eric]

Hi Ricardo,

On 5/3/21 1:09 PM, Andrew Jones wrote:
> On Fri, Apr 30, 2021 at 04:24:04PM -0700, Ricardo Koller wrote:
>> x86, the only arch implementing exception handling, reports unhandled
>> vectors using port IO at a specific port number. This replicates what
>> ucall already does.
>>
>> Introduce a new ucall type, UCALL_UNHANDLED, for guests to report
>> unhandled exceptions. Then replace the x86 unhandled vector exception
>> reporting to use it instead of port IO.  This new ucall type will be
>> used in the next commits by arm64 to report unhandled vectors as well.
>>
>> Tested: Forcing a page fault in the ./x86_64/xapic_ipi_test
>> 	halter_guest_code() shows this:
>>
>> 	$ ./x86_64/xapic_ipi_test
>> 	...
>> 	  Unexpected vectored event in guest (vector:0xe)
>>
>> Signed-off-by: Ricardo Koller <ricarkol@google.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>

Eric

>> ---
>>  tools/testing/selftests/kvm/include/kvm_util.h    |  1 +
>>  .../selftests/kvm/include/x86_64/processor.h      |  2 --
>>  .../testing/selftests/kvm/lib/x86_64/processor.c  | 15 ++++++---------
>>  3 files changed, 7 insertions(+), 11 deletions(-)
>>
>> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
>> index bea4644d645d..7880929ea548 100644
>> --- a/tools/testing/selftests/kvm/include/kvm_util.h
>> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
>> @@ -347,6 +347,7 @@ enum {
>>  	UCALL_SYNC,
>>  	UCALL_ABORT,
>>  	UCALL_DONE,
>> +	UCALL_UNHANDLED,
>>  };
>>  
>>  #define UCALL_MAX_ARGS 6
>> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
>> index 12889d3e8948..ff4da2f95b13 100644
>> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
>> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
>> @@ -53,8 +53,6 @@
>>  #define CPUID_PKU		(1ul << 3)
>>  #define CPUID_LA57		(1ul << 16)
>>  
>> -#define UNEXPECTED_VECTOR_PORT 0xfff0u
>> -
>>  /* General Registers in 64-Bit Mode */
>>  struct gpr64_regs {
>>  	u64 rax;
>> diff --git a/tools/testing/selftests/kvm/lib/x86_64/processor.c b/tools/testing/selftests/kvm/lib/x86_64/processor.c
>> index e156061263a6..96e2bd9d66eb 100644
>> --- a/tools/testing/selftests/kvm/lib/x86_64/processor.c
>> +++ b/tools/testing/selftests/kvm/lib/x86_64/processor.c
>> @@ -1207,7 +1207,7 @@ static void set_idt_entry(struct kvm_vm *vm, int vector, unsigned long addr,
>>  
>>  void kvm_exit_unexpected_vector(uint32_t value)
>>  {
>> -	outl(UNEXPECTED_VECTOR_PORT, value);
>> +	ucall(UCALL_UNHANDLED, 1, value);
>>  }
>>  
>>  void route_exception(struct ex_regs *regs)
>> @@ -1260,16 +1260,13 @@ void vm_install_vector_handler(struct kvm_vm *vm, int vector,
>>  
>>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
>>  {
>> -	if (vcpu_state(vm, vcpuid)->exit_reason == KVM_EXIT_IO
>> -		&& vcpu_state(vm, vcpuid)->io.port == UNEXPECTED_VECTOR_PORT
>> -		&& vcpu_state(vm, vcpuid)->io.size == 4) {
>> -		/* Grab pointer to io data */
>> -		uint32_t *data = (void *)vcpu_state(vm, vcpuid)
>> -			+ vcpu_state(vm, vcpuid)->io.data_offset;
>> +	struct ucall uc;
>>  
>> +	if (get_ucall(vm, vcpuid, &uc) == UCALL_UNHANDLED) {
>> +		uint64_t vector = uc.args[0];
>>  		TEST_ASSERT(false,
>> -			    "Unexpected vectored event in guest (vector:0x%x)",
>> -			    *data);
>> +			    "Unexpected vectored event in guest (vector:0x%lx)",
>> +			    vector);
> 
> nit: Could have changed this TEST_ASSERT(false, ...) to TEST_FAIL while
> touching it.
> 
>>  	}
>>  }
>>  
>> -- 
>> 2.31.1.527.g47e6f16901-goog
>>
> 
> Reviewed-by: Andrew Jones <drjones@redhat.com>
> 
> Thanks,
> drew
> 

[PATCH v2 3/5] KVM: selftests: Move GUEST_ASSERT_EQ to utils header

[Ricardo Koller]

Move GUEST_ASSERT_EQ to a common header, kvm_util.h, for other
architectures and tests to use.

Signed-off-by: Ricardo Koller <ricarkol@google.com>
---
 tools/testing/selftests/kvm/include/kvm_util.h     | 9 +++++++++
 tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c | 9 ---------
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index 7880929ea548..bd26dd93ab56 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -388,4 +388,13 @@ uint64_t get_ucall(struct kvm_vm *vm, uint32_t vcpu_id, struct ucall *uc);
 #define GUEST_ASSERT_4(_condition, arg1, arg2, arg3, arg4) \
 	__GUEST_ASSERT((_condition), 4, (arg1), (arg2), (arg3), (arg4))
 
+#define GUEST_ASSERT_EQ(a, b) do {				\
+	__typeof(a) _a = (a);					\
+	__typeof(b) _b = (b);					\
+	if (_a != _b)						\
+		ucall(UCALL_ABORT, 4,				\
+			"Failed guest assert: "			\
+			#a " == " #b, __LINE__, _a, _b);	\
+} while(0)
+
 #endif /* SELFTEST_KVM_UTIL_H */
diff --git a/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c b/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c
index e357d8e222d4..5a6a662f2e59 100644
--- a/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c
+++ b/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c
@@ -18,15 +18,6 @@
 #define rounded_rdmsr(x)       ROUND(rdmsr(x))
 #define rounded_host_rdmsr(x)  ROUND(vcpu_get_msr(vm, 0, x))
 
-#define GUEST_ASSERT_EQ(a, b) do {				\
-	__typeof(a) _a = (a);					\
-	__typeof(b) _b = (b);					\
-	if (_a != _b)						\
-                ucall(UCALL_ABORT, 4,				\
-                        "Failed guest assert: "			\
-                        #a " == " #b, __LINE__, _a, _b);	\
-  } while(0)
-
 static void guest_code(void)
 {
 	u64 val = 0;
-- 
2.31.1.527.g47e6f16901-goog

Re: [PATCH v2 3/5] KVM: selftests: Move GUEST_ASSERT_EQ to utils header

[Andrew Jones]

On Fri, Apr 30, 2021 at 04:24:05PM -0700, Ricardo Koller wrote:
> Move GUEST_ASSERT_EQ to a common header, kvm_util.h, for other
> architectures and tests to use.
> 
> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> ---
>  tools/testing/selftests/kvm/include/kvm_util.h     | 9 +++++++++
>  tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c | 9 ---------
>  2 files changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
> index 7880929ea548..bd26dd93ab56 100644
> --- a/tools/testing/selftests/kvm/include/kvm_util.h
> +++ b/tools/testing/selftests/kvm/include/kvm_util.h
> @@ -388,4 +388,13 @@ uint64_t get_ucall(struct kvm_vm *vm, uint32_t vcpu_id, struct ucall *uc);
>  #define GUEST_ASSERT_4(_condition, arg1, arg2, arg3, arg4) \
>  	__GUEST_ASSERT((_condition), 4, (arg1), (arg2), (arg3), (arg4))
>  
> +#define GUEST_ASSERT_EQ(a, b) do {				\
> +	__typeof(a) _a = (a);					\
> +	__typeof(b) _b = (b);					\
> +	if (_a != _b)						\
> +		ucall(UCALL_ABORT, 4,				\
> +			"Failed guest assert: "			\
> +			#a " == " #b, __LINE__, _a, _b);	\
> +} while(0)
> +
>  #endif /* SELFTEST_KVM_UTIL_H */
> diff --git a/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c b/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c
> index e357d8e222d4..5a6a662f2e59 100644
> --- a/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c
> +++ b/tools/testing/selftests/kvm/x86_64/tsc_msrs_test.c
> @@ -18,15 +18,6 @@
>  #define rounded_rdmsr(x)       ROUND(rdmsr(x))
>  #define rounded_host_rdmsr(x)  ROUND(vcpu_get_msr(vm, 0, x))
>  
> -#define GUEST_ASSERT_EQ(a, b) do {				\
> -	__typeof(a) _a = (a);					\
> -	__typeof(b) _b = (b);					\
> -	if (_a != _b)						\
> -                ucall(UCALL_ABORT, 4,				\
> -                        "Failed guest assert: "			\
> -                        #a " == " #b, __LINE__, _a, _b);	\
> -  } while(0)
> -
>  static void guest_code(void)
>  {
>  	u64 val = 0;
> -- 
> 2.31.1.527.g47e6f16901-goog
>

How about modify __GUEST_ASSERT so we can reuse it instead, like below?
(I also took the opportunity to remove the unnecessary () within the comma
separated statements.)

Thanks,
drew


-#define __GUEST_ASSERT(_condition, _nargs, _args...) do {      \
-       if (!(_condition))                                      \
-               ucall(UCALL_ABORT, 2 + _nargs,                  \
-                       "Failed guest assert: "                 \
-                       #_condition, __LINE__, _args);          \
+#define __GUEST_ASSERT(_condition, _condstr, _nargs, _args...) do {    \
+       if (!(_condition))                                              \
+               ucall(UCALL_ABORT, 2 + _nargs,                          \
+                       "Failed guest assert: "                         \
+                       _condstr, __LINE__, _args);                     \
 } while (0)
 
 #define GUEST_ASSERT(_condition) \
-       __GUEST_ASSERT((_condition), 0, 0)
+       __GUEST_ASSERT(_condition, #_condition, 0, 0)
 
 #define GUEST_ASSERT_1(_condition, arg1) \
-       __GUEST_ASSERT((_condition), 1, (arg1))
+       __GUEST_ASSERT(_condition, #_condition, 1, arg1)
 
 #define GUEST_ASSERT_2(_condition, arg1, arg2) \
-       __GUEST_ASSERT((_condition), 2, (arg1), (arg2))
+       __GUEST_ASSERT(_condition, #_condition, 2, arg1, arg2)
 
 #define GUEST_ASSERT_3(_condition, arg1, arg2, arg3) \
-       __GUEST_ASSERT((_condition), 3, (arg1), (arg2), (arg3))
+       __GUEST_ASSERT(_condition, #_condition, 3, arg1, arg2, arg3)
 
 #define GUEST_ASSERT_4(_condition, arg1, arg2, arg3, arg4) \
-       __GUEST_ASSERT((_condition), 4, (arg1), (arg2), (arg3), (arg4))
-
-#define GUEST_ASSERT_EQ(a, b) do {                             \
-       __typeof(a) _a = (a);                                   \
-       __typeof(b) _b = (b);                                   \
-       if (_a != _b)                                           \
-               ucall(UCALL_ABORT, 4,                           \
-                       "Failed guest assert: "                 \
-                       #a " == " #b, __LINE__, _a, _b);        \
-} while(0)
+       __GUEST_ASSERT(_condition, #_condition, 4, arg1, arg2, arg3, arg4)
+
+#define GUEST_ASSERT_EQ(a, b) __GUEST_ASSERT((a) == (b), #a " == " #b, 2, a, b)

[PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

Add the infrastructure needed to enable exception handling in aarch64
selftests. The exception handling defaults to an unhandled-exception
handler which aborts the test, just like x86. These handlers can be
overridden by calling vm_install_vector_handler(vector) or
vm_install_exception_handler(vector, ec). The unhandled exception
reporting from the guest is done using the ucall type introduced in a
previous commit, UCALL_UNHANDLED.

The exception handling code is heavily inspired on kvm-unit-tests.

Signed-off-by: Ricardo Koller <ricarkol@google.com>
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
 .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
 .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
 4 files changed, 333 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S

diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 4e548d7ab0ab..618c5903f478 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -35,7 +35,7 @@ endif
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
 LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
-LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
+LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
 
 TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index b7fa0c8551db..40aae31b4afc 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -8,6 +8,7 @@
 #define SELFTEST_KVM_PROCESSOR_H
 
 #include "kvm_util.h"
+#include <linux/stringify.h>
 
 
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
@@ -18,6 +19,7 @@
 #define MAIR_EL1	3, 0, 10, 2, 0
 #define TTBR0_EL1	3, 0,  2, 0, 0
 #define SCTLR_EL1	3, 0,  1, 0, 0
+#define VBAR_EL1	3, 0, 12, 0, 0
 
 /*
  * Default MAIR
@@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
 void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
 			      struct kvm_vcpu_init *init, void *guest_code);
 
+struct ex_regs {
+	u64 regs[31];
+	u64 sp;
+	u64 pc;
+	u64 pstate;
+};
+
+#define VECTOR_NUM	16
+
+enum {
+	VECTOR_SYNC_CURRENT_SP0,
+	VECTOR_IRQ_CURRENT_SP0,
+	VECTOR_FIQ_CURRENT_SP0,
+	VECTOR_ERROR_CURRENT_SP0,
+
+	VECTOR_SYNC_CURRENT,
+	VECTOR_IRQ_CURRENT,
+	VECTOR_FIQ_CURRENT,
+	VECTOR_ERROR_CURRENT,
+
+	VECTOR_SYNC_LOWER_64,
+	VECTOR_IRQ_LOWER_64,
+	VECTOR_FIQ_LOWER_64,
+	VECTOR_ERROR_LOWER_64,
+
+	VECTOR_SYNC_LOWER_32,
+	VECTOR_IRQ_LOWER_32,
+	VECTOR_FIQ_LOWER_32,
+	VECTOR_ERROR_LOWER_32,
+};
+
+#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
+			   (v) == VECTOR_SYNC_CURRENT     || \
+			   (v) == VECTOR_SYNC_LOWER_64    || \
+			   (v) == VECTOR_SYNC_LOWER_32)
+
+/* Some common EC (Exception classes) */
+#define ESR_EC_ILLEGAL_INS	0x0e
+#define ESR_EC_SVC64		0x15
+#define ESR_EC_IABORT_CURRENT	0x21
+#define ESR_EC_DABORT_CURRENT	0x25
+#define ESR_EC_SERROR		0x2f
+#define ESR_EC_HW_BP_CURRENT	0x31
+#define ESR_EC_SSTEP_CURRENT	0x33
+#define ESR_EC_WP_CURRENT	0x35
+#define ESR_EC_BRK_INS		0x3C
+
+#define ESR_EC_NUM		64
+
+#define ESR_EC_SHIFT		26
+#define ESR_EC_MASK		(ESR_EC_NUM - 1)
+
+void vm_init_descriptor_tables(struct kvm_vm *vm);
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
+
+typedef void(*handler_fn)(struct ex_regs *);
+void vm_install_exception_handler(struct kvm_vm *vm,
+		int vector, int ec, handler_fn handler);
+void vm_install_vector_handler(struct kvm_vm *vm,
+		int vector, handler_fn handler);
+
+#define SPSR_D          (1 << 9)
+#define SPSR_SS         (1 << 21)
+
+#define write_sysreg(reg, val)						  \
+({									  \
+	u64 __val = (u64)(val);						  \
+	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
+})
+
+#define read_sysreg(reg)						  \
+({	u64 val;							  \
+	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
+	val;								  \
+})
+
 #endif /* SELFTEST_KVM_PROCESSOR_H */
diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
new file mode 100644
index 000000000000..8a560021892b
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
@@ -0,0 +1,130 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+.macro save_registers, vector
+	add	sp, sp, #-16 * 17
+
+	stp	x0, x1, [sp, #16 * 0]
+	stp	x2, x3, [sp, #16 * 1]
+	stp	x4, x5, [sp, #16 * 2]
+	stp	x6, x7, [sp, #16 * 3]
+	stp	x8, x9, [sp, #16 * 4]
+	stp	x10, x11, [sp, #16 * 5]
+	stp	x12, x13, [sp, #16 * 6]
+	stp	x14, x15, [sp, #16 * 7]
+	stp	x16, x17, [sp, #16 * 8]
+	stp	x18, x19, [sp, #16 * 9]
+	stp	x20, x21, [sp, #16 * 10]
+	stp	x22, x23, [sp, #16 * 11]
+	stp	x24, x25, [sp, #16 * 12]
+	stp	x26, x27, [sp, #16 * 13]
+	stp	x28, x29, [sp, #16 * 14]
+
+	.if \vector >= 8
+	mrs	x1, sp_el0
+	.else
+	/*
+	 * This stores sp_el1 into ex_regs.sp so exception handlers can
+	 * "look" at it. It will _not_ be used to restore the sp_el1 on
+	 * return from the exception so handlers can not update it.
+	 */
+	mov	x1, sp
+	.endif
+	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
+
+	mrs	x1, elr_el1
+	mrs	x2, spsr_el1
+	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+.endm
+
+.macro restore_registers, vector
+	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
+	msr	elr_el1, x1
+	msr	spsr_el1, x2
+
+	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
+	.if \vector >= 8
+	msr	sp_el0, x1
+	.endif
+
+	ldp	x28, x29, [sp, #16 * 14]
+	ldp	x26, x27, [sp, #16 * 13]
+	ldp	x24, x25, [sp, #16 * 12]
+	ldp	x22, x23, [sp, #16 * 11]
+	ldp	x20, x21, [sp, #16 * 10]
+	ldp	x18, x19, [sp, #16 * 9]
+	ldp	x16, x17, [sp, #16 * 8]
+	ldp	x14, x15, [sp, #16 * 7]
+	ldp	x12, x13, [sp, #16 * 6]
+	ldp	x10, x11, [sp, #16 * 5]
+	ldp	x8, x9, [sp, #16 * 4]
+	ldp	x6, x7, [sp, #16 * 3]
+	ldp	x4, x5, [sp, #16 * 2]
+	ldp	x2, x3, [sp, #16 * 1]
+	ldp	x0, x1, [sp, #16 * 0]
+
+	add	sp, sp, #16 * 17
+
+	eret
+.endm
+
+.pushsection ".entry.text", "ax"
+.balign 0x800
+.global vectors
+vectors:
+.popsection
+
+.set	vector, 0
+
+/*
+ * Build an exception handler for vector and append a jump to it into
+ * vectors (while making sure that it's 0x80 aligned).
+ */
+.macro HANDLER, label
+handler_\()\label:
+	save_registers vector
+	mov	x0, sp
+	mov	x1, #vector
+	bl	route_exception
+	restore_registers vector
+
+.pushsection ".entry.text", "ax"
+.balign 0x80
+	b	handler_\()\label
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+.macro HANDLER_INVALID
+.pushsection ".entry.text", "ax"
+.balign 0x80
+/* This will abort so no need to save and restore registers. */
+	mov	x0, #vector
+	b	kvm_exit_unexpected_vector
+.popsection
+
+.set	vector, vector + 1
+.endm
+
+/*
+ * Caution: be sure to not add anything between the declaration of vectors
+ * above and these macro calls that will build the vectors table below it.
+ */
+	HANDLER_INVALID                         // Synchronous EL1t
+	HANDLER_INVALID                         // IRQ EL1t
+	HANDLER_INVALID                         // FIQ EL1t
+	HANDLER_INVALID                         // Error EL1t
+
+	HANDLER	el1h_sync                       // Synchronous EL1h
+	HANDLER	el1h_irq                        // IRQ EL1h
+	HANDLER el1h_fiq                        // FIQ EL1h
+	HANDLER	el1h_error                      // Error EL1h
+
+	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
+	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
+	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
+	HANDLER	el0_error_64                    // Error 64-bit EL0
+
+	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
+	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
+	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
+	HANDLER	el0_error_32                    // Error 32-bit EL0
diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
index cee92d477dc0..25be71ec88be 100644
--- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
+++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
@@ -6,6 +6,7 @@
  */
 
 #include <linux/compiler.h>
+#include <assert.h>
 
 #include "kvm_util.h"
 #include "../kvm_util_internal.h"
@@ -14,6 +15,8 @@
 #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
 #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
 
+vm_vaddr_t exception_handlers;
+
 static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
 {
 	return (v + vm->page_size) & ~(vm->page_size - 1);
@@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
 	va_end(ap);
 }
 
+void kvm_exit_unexpected_vector(int vector)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
+}
+
+void kvm_exit_unexpected_exception(int vector, uint64_t ec)
+{
+	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
+}
+
 void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
 {
+	struct ucall uc;
+
+	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
+		return;
+
+	if (uc.args[2]) /* valid_ec */ {
+		assert(VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
+			uc.args[0], uc.args[1]);
+	} else {
+		assert(!VECTOR_IS_SYNC(uc.args[0]));
+		TEST_ASSERT(false,
+			"Unexpected exception (vector:0x%lx)",
+			uc.args[0]);
+	}
+}
+
+/*
+ * This exception handling code was heavily inspired on kvm-unit-tests. There
+ * is a set of default vector handlers stored in vector_handlers. These default
+ * vector handlers call user-installed handlers stored in exception_handlers.
+ * Synchronous handlers are indexed by (vector, ec), and irq handlers by
+ * (vector, ec=0).
+ */
+
+typedef void(*vector_fn)(struct ex_regs *, int vector);
+
+struct handlers {
+	vector_fn vector_handlers[VECTOR_NUM];
+	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
+};
+
+void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
+{
+	extern char vectors;
+
+	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
+}
+
+void default_sync_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+	uint64_t esr = read_sysreg(esr_el1);
+	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
+
+	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][ec])
+		handlers->exception_handlers[vector][ec](regs);
+	else
+		kvm_exit_unexpected_exception(vector, ec);
+}
+
+void default_irq_handler(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
+
+	if (handlers && handlers->exception_handlers[vector][0])
+		handlers->exception_handlers[vector][0](regs);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void route_exception(struct ex_regs *regs, int vector)
+{
+	struct handlers *handlers = (struct handlers *)exception_handlers;
+
+	if (handlers && handlers->vector_handlers[vector])
+		handlers->vector_handlers[vector](regs, vector);
+	else
+		kvm_exit_unexpected_vector(vector);
+}
+
+void vm_init_descriptor_tables(struct kvm_vm *vm)
+{
+	struct handlers *handlers;
+
+	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
+			vm->page_size, 0, 0);
+
+	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
+	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
+	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
+
+	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
+}
+
+void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	assert(ec < ESR_EC_NUM);
+	handlers->exception_handlers[vector][ec] = handler;
+}
+
+void vm_install_vector_handler(struct kvm_vm *vm, int vector,
+			 void (*handler)(struct ex_regs *))
+{
+	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
+
+	assert(!VECTOR_IS_SYNC(vector));
+	assert(vector < VECTOR_NUM);
+	handlers->exception_handlers[vector][0] = handler;
 }
-- 
2.31.1.527.g47e6f16901-goog

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Marc Zyngier]

On Sat, 01 May 2021 00:24:06 +0100,
Ricardo Koller <ricarkol@google.com> wrote:
> 
> Add the infrastructure needed to enable exception handling in aarch64
> selftests. The exception handling defaults to an unhandled-exception
> handler which aborts the test, just like x86. These handlers can be
> overridden by calling vm_install_vector_handler(vector) or
> vm_install_exception_handler(vector, ec). The unhandled exception
> reporting from the guest is done using the ucall type introduced in a
> previous commit, UCALL_UNHANDLED.
> 
> The exception handling code is heavily inspired on kvm-unit-tests.
> 
> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> ---
>  tools/testing/selftests/kvm/Makefile          |   2 +-
>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
>  4 files changed, 333 insertions(+), 1 deletion(-)
>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
> 
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 4e548d7ab0ab..618c5903f478 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -35,7 +35,7 @@ endif
>  
>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
>  
>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
> index b7fa0c8551db..40aae31b4afc 100644
> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
> @@ -8,6 +8,7 @@
>  #define SELFTEST_KVM_PROCESSOR_H
>  
>  #include "kvm_util.h"
> +#include <linux/stringify.h>
>  
>  
>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> @@ -18,6 +19,7 @@
>  #define MAIR_EL1	3, 0, 10, 2, 0
>  #define TTBR0_EL1	3, 0,  2, 0, 0
>  #define SCTLR_EL1	3, 0,  1, 0, 0
> +#define VBAR_EL1	3, 0, 12, 0, 0
>  
>  /*
>   * Default MAIR
> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
>  			      struct kvm_vcpu_init *init, void *guest_code);
>  
> +struct ex_regs {
> +	u64 regs[31];
> +	u64 sp;
> +	u64 pc;
> +	u64 pstate;
> +};
> +
> +#define VECTOR_NUM	16
> +
> +enum {
> +	VECTOR_SYNC_CURRENT_SP0,
> +	VECTOR_IRQ_CURRENT_SP0,
> +	VECTOR_FIQ_CURRENT_SP0,
> +	VECTOR_ERROR_CURRENT_SP0,
> +
> +	VECTOR_SYNC_CURRENT,
> +	VECTOR_IRQ_CURRENT,
> +	VECTOR_FIQ_CURRENT,
> +	VECTOR_ERROR_CURRENT,
> +
> +	VECTOR_SYNC_LOWER_64,
> +	VECTOR_IRQ_LOWER_64,
> +	VECTOR_FIQ_LOWER_64,
> +	VECTOR_ERROR_LOWER_64,
> +
> +	VECTOR_SYNC_LOWER_32,
> +	VECTOR_IRQ_LOWER_32,
> +	VECTOR_FIQ_LOWER_32,
> +	VECTOR_ERROR_LOWER_32,
> +};
> +
> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
> +			   (v) == VECTOR_SYNC_CURRENT     || \
> +			   (v) == VECTOR_SYNC_LOWER_64    || \
> +			   (v) == VECTOR_SYNC_LOWER_32)
> +
> +/* Some common EC (Exception classes) */
> +#define ESR_EC_ILLEGAL_INS	0x0e
> +#define ESR_EC_SVC64		0x15
> +#define ESR_EC_IABORT_CURRENT	0x21
> +#define ESR_EC_DABORT_CURRENT	0x25
> +#define ESR_EC_SERROR		0x2f
> +#define ESR_EC_HW_BP_CURRENT	0x31
> +#define ESR_EC_SSTEP_CURRENT	0x33
> +#define ESR_EC_WP_CURRENT	0x35
> +#define ESR_EC_BRK_INS		0x3C
> +
> +#define ESR_EC_NUM		64
> +
> +#define ESR_EC_SHIFT		26
> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
> +
> +void vm_init_descriptor_tables(struct kvm_vm *vm);
> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
> +
> +typedef void(*handler_fn)(struct ex_regs *);
> +void vm_install_exception_handler(struct kvm_vm *vm,
> +		int vector, int ec, handler_fn handler);
> +void vm_install_vector_handler(struct kvm_vm *vm,
> +		int vector, handler_fn handler);
> +
> +#define SPSR_D          (1 << 9)
> +#define SPSR_SS         (1 << 21)
> +
> +#define write_sysreg(reg, val)						  \
> +({									  \
> +	u64 __val = (u64)(val);						  \
> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
> +})
> +
> +#define read_sysreg(reg)						  \
> +({	u64 val;							  \
> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
> +	val;								  \
> +})
> +
>  #endif /* SELFTEST_KVM_PROCESSOR_H */
> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> new file mode 100644
> index 000000000000..8a560021892b
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> @@ -0,0 +1,130 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +.macro save_registers, vector
> +	add	sp, sp, #-16 * 17
> +
> +	stp	x0, x1, [sp, #16 * 0]
> +	stp	x2, x3, [sp, #16 * 1]
> +	stp	x4, x5, [sp, #16 * 2]
> +	stp	x6, x7, [sp, #16 * 3]
> +	stp	x8, x9, [sp, #16 * 4]
> +	stp	x10, x11, [sp, #16 * 5]
> +	stp	x12, x13, [sp, #16 * 6]
> +	stp	x14, x15, [sp, #16 * 7]
> +	stp	x16, x17, [sp, #16 * 8]
> +	stp	x18, x19, [sp, #16 * 9]
> +	stp	x20, x21, [sp, #16 * 10]
> +	stp	x22, x23, [sp, #16 * 11]
> +	stp	x24, x25, [sp, #16 * 12]
> +	stp	x26, x27, [sp, #16 * 13]
> +	stp	x28, x29, [sp, #16 * 14]
> +
> +	.if \vector >= 8
> +	mrs	x1, sp_el0

I'm still a bit perplexed by this. SP_EL0 is never changed, since you
always run in handler mode. Therefore, saving/restoring it is only
overhead. If an exception handler wants to introspect it, it is
already available in the relevant system register.

Or did you have something else in mind for it?

> +	.else
> +	/*
> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
> +	 * return from the exception so handlers can not update it.
> +	 */
> +	mov	x1, sp
> +	.endif
> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
> +
> +	mrs	x1, elr_el1
> +	mrs	x2, spsr_el1
> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> +.endm
> +
> +.macro restore_registers, vector
> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> +	msr	elr_el1, x1
> +	msr	spsr_el1, x2
> +
> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
> +	.if \vector >= 8
> +	msr	sp_el0, x1
> +	.endif
> +
> +	ldp	x28, x29, [sp, #16 * 14]
> +	ldp	x26, x27, [sp, #16 * 13]
> +	ldp	x24, x25, [sp, #16 * 12]
> +	ldp	x22, x23, [sp, #16 * 11]
> +	ldp	x20, x21, [sp, #16 * 10]
> +	ldp	x18, x19, [sp, #16 * 9]
> +	ldp	x16, x17, [sp, #16 * 8]
> +	ldp	x14, x15, [sp, #16 * 7]
> +	ldp	x12, x13, [sp, #16 * 6]
> +	ldp	x10, x11, [sp, #16 * 5]
> +	ldp	x8, x9, [sp, #16 * 4]
> +	ldp	x6, x7, [sp, #16 * 3]
> +	ldp	x4, x5, [sp, #16 * 2]
> +	ldp	x2, x3, [sp, #16 * 1]
> +	ldp	x0, x1, [sp, #16 * 0]
> +
> +	add	sp, sp, #16 * 17
> +
> +	eret
> +.endm
> +
> +.pushsection ".entry.text", "ax"
> +.balign 0x800
> +.global vectors
> +vectors:
> +.popsection
> +
> +.set	vector, 0
> +
> +/*
> + * Build an exception handler for vector and append a jump to it into
> + * vectors (while making sure that it's 0x80 aligned).
> + */
> +.macro HANDLER, label
> +handler_\()\label:
> +	save_registers vector
> +	mov	x0, sp
> +	mov	x1, #vector
> +	bl	route_exception
> +	restore_registers vector
> +
> +.pushsection ".entry.text", "ax"
> +.balign 0x80
> +	b	handler_\()\label
> +.popsection
> +
> +.set	vector, vector + 1
> +.endm
> +
> +.macro HANDLER_INVALID
> +.pushsection ".entry.text", "ax"
> +.balign 0x80
> +/* This will abort so no need to save and restore registers. */
> +	mov	x0, #vector
> +	b	kvm_exit_unexpected_vector
> +.popsection
> +
> +.set	vector, vector + 1
> +.endm
> +
> +/*
> + * Caution: be sure to not add anything between the declaration of vectors
> + * above and these macro calls that will build the vectors table below it.
> + */
> +	HANDLER_INVALID                         // Synchronous EL1t
> +	HANDLER_INVALID                         // IRQ EL1t
> +	HANDLER_INVALID                         // FIQ EL1t
> +	HANDLER_INVALID                         // Error EL1t
> +
> +	HANDLER	el1h_sync                       // Synchronous EL1h
> +	HANDLER	el1h_irq                        // IRQ EL1h
> +	HANDLER el1h_fiq                        // FIQ EL1h
> +	HANDLER	el1h_error                      // Error EL1h
> +
> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
> +	HANDLER	el0_error_64                    // Error 64-bit EL0
> +
> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
> +	HANDLER	el0_error_32                    // Error 32-bit EL0
> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> index cee92d477dc0..25be71ec88be 100644
> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> @@ -6,6 +6,7 @@
>   */
>  
>  #include <linux/compiler.h>
> +#include <assert.h>
>  
>  #include "kvm_util.h"
>  #include "../kvm_util_internal.h"
> @@ -14,6 +15,8 @@
>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
>  
> +vm_vaddr_t exception_handlers;
> +
>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
>  {
>  	return (v + vm->page_size) & ~(vm->page_size - 1);
> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
>  	va_end(ap);
>  }
>  
> +void kvm_exit_unexpected_vector(int vector)
> +{
> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
> +}
> +
> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
> +{
> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
> +}
> +
>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
>  {
> +	struct ucall uc;
> +
> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
> +		return;
> +
> +	if (uc.args[2]) /* valid_ec */ {
> +		assert(VECTOR_IS_SYNC(uc.args[0]));
> +		TEST_ASSERT(false,
> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
> +			uc.args[0], uc.args[1]);
> +	} else {
> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
> +		TEST_ASSERT(false,
> +			"Unexpected exception (vector:0x%lx)",
> +			uc.args[0]);
> +	}
> +}
> +
> +/*
> + * This exception handling code was heavily inspired on kvm-unit-tests. There
> + * is a set of default vector handlers stored in vector_handlers. These default
> + * vector handlers call user-installed handlers stored in exception_handlers.
> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
> + * (vector, ec=0).
> + */
> +
> +typedef void(*vector_fn)(struct ex_regs *, int vector);
> +
> +struct handlers {
> +	vector_fn vector_handlers[VECTOR_NUM];
> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
> +};
> +
> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
> +{
> +	extern char vectors;
> +
> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
> +}
> +
> +void default_sync_handler(struct ex_regs *regs, int vector)
> +{
> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> +	uint64_t esr = read_sysreg(esr_el1);
> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
> +
> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
> +
> +	if (handlers && handlers->exception_handlers[vector][ec])
> +		handlers->exception_handlers[vector][ec](regs);
> +	else
> +		kvm_exit_unexpected_exception(vector, ec);
> +}
> +
> +void default_irq_handler(struct ex_regs *regs, int vector)
> +{
> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> +
> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
> +
> +	if (handlers && handlers->exception_handlers[vector][0])
> +		handlers->exception_handlers[vector][0](regs);
> +	else
> +		kvm_exit_unexpected_vector(vector);
> +}
> +
> +void route_exception(struct ex_regs *regs, int vector)
> +{
> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> +
> +	if (handlers && handlers->vector_handlers[vector])
> +		handlers->vector_handlers[vector](regs, vector);
> +	else
> +		kvm_exit_unexpected_vector(vector);
> +}
> +
> +void vm_init_descriptor_tables(struct kvm_vm *vm)
> +{
> +	struct handlers *handlers;
> +
> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
> +			vm->page_size, 0, 0);
> +
> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;

How about FIQ, Error? Although they are unlikely, they are valid
exceptions.

> +
> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
> +}
> +
> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
> +			 void (*handler)(struct ex_regs *))
> +{
> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> +
> +	assert(VECTOR_IS_SYNC(vector));
> +	assert(vector < VECTOR_NUM);
> +	assert(ec < ESR_EC_NUM);
> +	handlers->exception_handlers[vector][ec] = handler;
> +}
> +
> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
> +			 void (*handler)(struct ex_regs *))
> +{
> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> +
> +	assert(!VECTOR_IS_SYNC(vector));
> +	assert(vector < VECTOR_NUM);
> +	handlers->exception_handlers[vector][0] = handler;
>  }

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
> On Sat, 01 May 2021 00:24:06 +0100,
> Ricardo Koller <ricarkol@google.com> wrote:
> > 
> > Add the infrastructure needed to enable exception handling in aarch64
> > selftests. The exception handling defaults to an unhandled-exception
> > handler which aborts the test, just like x86. These handlers can be
> > overridden by calling vm_install_vector_handler(vector) or
> > vm_install_exception_handler(vector, ec). The unhandled exception
> > reporting from the guest is done using the ucall type introduced in a
> > previous commit, UCALL_UNHANDLED.
> > 
> > The exception handling code is heavily inspired on kvm-unit-tests.
> > 
> > Signed-off-by: Ricardo Koller <ricarkol@google.com>
> > ---
> >  tools/testing/selftests/kvm/Makefile          |   2 +-
> >  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
> >  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
> >  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
> >  4 files changed, 333 insertions(+), 1 deletion(-)
> >  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
> > 
> > diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> > index 4e548d7ab0ab..618c5903f478 100644
> > --- a/tools/testing/selftests/kvm/Makefile
> > +++ b/tools/testing/selftests/kvm/Makefile
> > @@ -35,7 +35,7 @@ endif
> >  
> >  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
> >  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
> > -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
> > +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
> >  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
> >  
> >  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
> > diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
> > index b7fa0c8551db..40aae31b4afc 100644
> > --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
> > +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
> > @@ -8,6 +8,7 @@
> >  #define SELFTEST_KVM_PROCESSOR_H
> >  
> >  #include "kvm_util.h"
> > +#include <linux/stringify.h>
> >  
> >  
> >  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> > @@ -18,6 +19,7 @@
> >  #define MAIR_EL1	3, 0, 10, 2, 0
> >  #define TTBR0_EL1	3, 0,  2, 0, 0
> >  #define SCTLR_EL1	3, 0,  1, 0, 0
> > +#define VBAR_EL1	3, 0, 12, 0, 0
> >  
> >  /*
> >   * Default MAIR
> > @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
> >  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
> >  			      struct kvm_vcpu_init *init, void *guest_code);
> >  
> > +struct ex_regs {
> > +	u64 regs[31];
> > +	u64 sp;
> > +	u64 pc;
> > +	u64 pstate;
> > +};
> > +
> > +#define VECTOR_NUM	16
> > +
> > +enum {
> > +	VECTOR_SYNC_CURRENT_SP0,
> > +	VECTOR_IRQ_CURRENT_SP0,
> > +	VECTOR_FIQ_CURRENT_SP0,
> > +	VECTOR_ERROR_CURRENT_SP0,
> > +
> > +	VECTOR_SYNC_CURRENT,
> > +	VECTOR_IRQ_CURRENT,
> > +	VECTOR_FIQ_CURRENT,
> > +	VECTOR_ERROR_CURRENT,
> > +
> > +	VECTOR_SYNC_LOWER_64,
> > +	VECTOR_IRQ_LOWER_64,
> > +	VECTOR_FIQ_LOWER_64,
> > +	VECTOR_ERROR_LOWER_64,
> > +
> > +	VECTOR_SYNC_LOWER_32,
> > +	VECTOR_IRQ_LOWER_32,
> > +	VECTOR_FIQ_LOWER_32,
> > +	VECTOR_ERROR_LOWER_32,
> > +};
> > +
> > +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
> > +			   (v) == VECTOR_SYNC_CURRENT     || \
> > +			   (v) == VECTOR_SYNC_LOWER_64    || \
> > +			   (v) == VECTOR_SYNC_LOWER_32)
> > +
> > +/* Some common EC (Exception classes) */
> > +#define ESR_EC_ILLEGAL_INS	0x0e
> > +#define ESR_EC_SVC64		0x15
> > +#define ESR_EC_IABORT_CURRENT	0x21
> > +#define ESR_EC_DABORT_CURRENT	0x25
> > +#define ESR_EC_SERROR		0x2f
> > +#define ESR_EC_HW_BP_CURRENT	0x31
> > +#define ESR_EC_SSTEP_CURRENT	0x33
> > +#define ESR_EC_WP_CURRENT	0x35
> > +#define ESR_EC_BRK_INS		0x3C
> > +
> > +#define ESR_EC_NUM		64
> > +
> > +#define ESR_EC_SHIFT		26
> > +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
> > +
> > +void vm_init_descriptor_tables(struct kvm_vm *vm);
> > +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
> > +
> > +typedef void(*handler_fn)(struct ex_regs *);
> > +void vm_install_exception_handler(struct kvm_vm *vm,
> > +		int vector, int ec, handler_fn handler);
> > +void vm_install_vector_handler(struct kvm_vm *vm,
> > +		int vector, handler_fn handler);
> > +
> > +#define SPSR_D          (1 << 9)
> > +#define SPSR_SS         (1 << 21)
> > +
> > +#define write_sysreg(reg, val)						  \
> > +({									  \
> > +	u64 __val = (u64)(val);						  \
> > +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
> > +})
> > +
> > +#define read_sysreg(reg)						  \
> > +({	u64 val;							  \
> > +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
> > +	val;								  \
> > +})
> > +
> >  #endif /* SELFTEST_KVM_PROCESSOR_H */
> > diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> > new file mode 100644
> > index 000000000000..8a560021892b
> > --- /dev/null
> > +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> > @@ -0,0 +1,130 @@
> > +/* SPDX-License-Identifier: GPL-2.0 */
> > +.macro save_registers, vector
> > +	add	sp, sp, #-16 * 17
> > +
> > +	stp	x0, x1, [sp, #16 * 0]
> > +	stp	x2, x3, [sp, #16 * 1]
> > +	stp	x4, x5, [sp, #16 * 2]
> > +	stp	x6, x7, [sp, #16 * 3]
> > +	stp	x8, x9, [sp, #16 * 4]
> > +	stp	x10, x11, [sp, #16 * 5]
> > +	stp	x12, x13, [sp, #16 * 6]
> > +	stp	x14, x15, [sp, #16 * 7]
> > +	stp	x16, x17, [sp, #16 * 8]
> > +	stp	x18, x19, [sp, #16 * 9]
> > +	stp	x20, x21, [sp, #16 * 10]
> > +	stp	x22, x23, [sp, #16 * 11]
> > +	stp	x24, x25, [sp, #16 * 12]
> > +	stp	x26, x27, [sp, #16 * 13]
> > +	stp	x28, x29, [sp, #16 * 14]
> > +
> > +	.if \vector >= 8
> > +	mrs	x1, sp_el0
> 
> I'm still a bit perplexed by this. SP_EL0 is never changed, since you
> always run in handler mode. Therefore, saving/restoring it is only
> overhead. If an exception handler wants to introspect it, it is
> already available in the relevant system register.
> 
> Or did you have something else in mind for it?
> 

Not really. The reason for saving sp_el0 in there was just for
consistency, so that handlers for both el0 and el1 exceptions could get
the sp at regs->sp.

Restoring sp_el0 might be too much. So, what do you think of this v3: we
keep the saving of sp_el0 into regs->sp (to keep things the same between
el0 and el1) and delete the restoring of sp_el0?

Thanks,
Ricardo

> > +	.else
> > +	/*
> > +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
> > +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
> > +	 * return from the exception so handlers can not update it.
> > +	 */
> > +	mov	x1, sp
> > +	.endif
> > +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
> > +
> > +	mrs	x1, elr_el1
> > +	mrs	x2, spsr_el1
> > +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> > +.endm
> > +
> > +.macro restore_registers, vector
> > +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> > +	msr	elr_el1, x1
> > +	msr	spsr_el1, x2
> > +
> > +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
> > +	.if \vector >= 8
> > +	msr	sp_el0, x1
> > +	.endif
> > +
> > +	ldp	x28, x29, [sp, #16 * 14]
> > +	ldp	x26, x27, [sp, #16 * 13]
> > +	ldp	x24, x25, [sp, #16 * 12]
> > +	ldp	x22, x23, [sp, #16 * 11]
> > +	ldp	x20, x21, [sp, #16 * 10]
> > +	ldp	x18, x19, [sp, #16 * 9]
> > +	ldp	x16, x17, [sp, #16 * 8]
> > +	ldp	x14, x15, [sp, #16 * 7]
> > +	ldp	x12, x13, [sp, #16 * 6]
> > +	ldp	x10, x11, [sp, #16 * 5]
> > +	ldp	x8, x9, [sp, #16 * 4]
> > +	ldp	x6, x7, [sp, #16 * 3]
> > +	ldp	x4, x5, [sp, #16 * 2]
> > +	ldp	x2, x3, [sp, #16 * 1]
> > +	ldp	x0, x1, [sp, #16 * 0]
> > +
> > +	add	sp, sp, #16 * 17
> > +
> > +	eret
> > +.endm
> > +
> > +.pushsection ".entry.text", "ax"
> > +.balign 0x800
> > +.global vectors
> > +vectors:
> > +.popsection
> > +
> > +.set	vector, 0
> > +
> > +/*
> > + * Build an exception handler for vector and append a jump to it into
> > + * vectors (while making sure that it's 0x80 aligned).
> > + */
> > +.macro HANDLER, label
> > +handler_\()\label:
> > +	save_registers vector
> > +	mov	x0, sp
> > +	mov	x1, #vector
> > +	bl	route_exception
> > +	restore_registers vector
> > +
> > +.pushsection ".entry.text", "ax"
> > +.balign 0x80
> > +	b	handler_\()\label
> > +.popsection
> > +
> > +.set	vector, vector + 1
> > +.endm
> > +
> > +.macro HANDLER_INVALID
> > +.pushsection ".entry.text", "ax"
> > +.balign 0x80
> > +/* This will abort so no need to save and restore registers. */
> > +	mov	x0, #vector
> > +	b	kvm_exit_unexpected_vector
> > +.popsection
> > +
> > +.set	vector, vector + 1
> > +.endm
> > +
> > +/*
> > + * Caution: be sure to not add anything between the declaration of vectors
> > + * above and these macro calls that will build the vectors table below it.
> > + */
> > +	HANDLER_INVALID                         // Synchronous EL1t
> > +	HANDLER_INVALID                         // IRQ EL1t
> > +	HANDLER_INVALID                         // FIQ EL1t
> > +	HANDLER_INVALID                         // Error EL1t
> > +
> > +	HANDLER	el1h_sync                       // Synchronous EL1h
> > +	HANDLER	el1h_irq                        // IRQ EL1h
> > +	HANDLER el1h_fiq                        // FIQ EL1h
> > +	HANDLER	el1h_error                      // Error EL1h
> > +
> > +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
> > +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
> > +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
> > +	HANDLER	el0_error_64                    // Error 64-bit EL0
> > +
> > +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
> > +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
> > +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
> > +	HANDLER	el0_error_32                    // Error 32-bit EL0
> > diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> > index cee92d477dc0..25be71ec88be 100644
> > --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
> > +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> > @@ -6,6 +6,7 @@
> >   */
> >  
> >  #include <linux/compiler.h>
> > +#include <assert.h>
> >  
> >  #include "kvm_util.h"
> >  #include "../kvm_util_internal.h"
> > @@ -14,6 +15,8 @@
> >  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
> >  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
> >  
> > +vm_vaddr_t exception_handlers;
> > +
> >  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
> >  {
> >  	return (v + vm->page_size) & ~(vm->page_size - 1);
> > @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
> >  	va_end(ap);
> >  }
> >  
> > +void kvm_exit_unexpected_vector(int vector)
> > +{
> > +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
> > +}
> > +
> > +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
> > +{
> > +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
> > +}
> > +
> >  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
> >  {
> > +	struct ucall uc;
> > +
> > +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
> > +		return;
> > +
> > +	if (uc.args[2]) /* valid_ec */ {
> > +		assert(VECTOR_IS_SYNC(uc.args[0]));
> > +		TEST_ASSERT(false,
> > +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
> > +			uc.args[0], uc.args[1]);
> > +	} else {
> > +		assert(!VECTOR_IS_SYNC(uc.args[0]));
> > +		TEST_ASSERT(false,
> > +			"Unexpected exception (vector:0x%lx)",
> > +			uc.args[0]);
> > +	}
> > +}
> > +
> > +/*
> > + * This exception handling code was heavily inspired on kvm-unit-tests. There
> > + * is a set of default vector handlers stored in vector_handlers. These default
> > + * vector handlers call user-installed handlers stored in exception_handlers.
> > + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
> > + * (vector, ec=0).
> > + */
> > +
> > +typedef void(*vector_fn)(struct ex_regs *, int vector);
> > +
> > +struct handlers {
> > +	vector_fn vector_handlers[VECTOR_NUM];
> > +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
> > +};
> > +
> > +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
> > +{
> > +	extern char vectors;
> > +
> > +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
> > +}
> > +
> > +void default_sync_handler(struct ex_regs *regs, int vector)
> > +{
> > +	struct handlers *handlers = (struct handlers *)exception_handlers;
> > +	uint64_t esr = read_sysreg(esr_el1);
> > +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
> > +
> > +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
> > +
> > +	if (handlers && handlers->exception_handlers[vector][ec])
> > +		handlers->exception_handlers[vector][ec](regs);
> > +	else
> > +		kvm_exit_unexpected_exception(vector, ec);
> > +}
> > +
> > +void default_irq_handler(struct ex_regs *regs, int vector)
> > +{
> > +	struct handlers *handlers = (struct handlers *)exception_handlers;
> > +
> > +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
> > +
> > +	if (handlers && handlers->exception_handlers[vector][0])
> > +		handlers->exception_handlers[vector][0](regs);
> > +	else
> > +		kvm_exit_unexpected_vector(vector);
> > +}
> > +
> > +void route_exception(struct ex_regs *regs, int vector)
> > +{
> > +	struct handlers *handlers = (struct handlers *)exception_handlers;
> > +
> > +	if (handlers && handlers->vector_handlers[vector])
> > +		handlers->vector_handlers[vector](regs, vector);
> > +	else
> > +		kvm_exit_unexpected_vector(vector);
> > +}
> > +
> > +void vm_init_descriptor_tables(struct kvm_vm *vm)
> > +{
> > +	struct handlers *handlers;
> > +
> > +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
> > +			vm->page_size, 0, 0);
> > +
> > +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> > +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
> > +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
> > +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
> > +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
> 
> How about FIQ, Error? Although they are unlikely, they are valid
> exceptions.
> 
> > +
> > +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
> > +}
> > +
> > +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
> > +			 void (*handler)(struct ex_regs *))
> > +{
> > +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> > +
> > +	assert(VECTOR_IS_SYNC(vector));
> > +	assert(vector < VECTOR_NUM);
> > +	assert(ec < ESR_EC_NUM);
> > +	handlers->exception_handlers[vector][ec] = handler;
> > +}
> > +
> > +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
> > +			 void (*handler)(struct ex_regs *))
> > +{
> > +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> > +
> > +	assert(!VECTOR_IS_SYNC(vector));
> > +	assert(vector < VECTOR_NUM);
> > +	handlers->exception_handlers[vector][0] = handler;
> >  }
> 
> Thanks,
> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Auger Eric]

Hi Ricardo,

On 5/3/21 9:12 PM, Ricardo Koller wrote:
> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
>> On Sat, 01 May 2021 00:24:06 +0100,
>> Ricardo Koller <ricarkol@google.com> wrote:
>>>
>>> Add the infrastructure needed to enable exception handling in aarch64
>>> selftests. The exception handling defaults to an unhandled-exception
>>> handler which aborts the test, just like x86. These handlers can be
>>> overridden by calling vm_install_vector_handler(vector) or
>>> vm_install_exception_handler(vector, ec). The unhandled exception
>>> reporting from the guest is done using the ucall type introduced in a
>>> previous commit, UCALL_UNHANDLED.
>>>
>>> The exception handling code is heavily inspired on kvm-unit-tests.

running the test on 5.12 I get

==== Test Assertion Failure ====
  aarch64/debug-exceptions.c:232: false
  pid=6477 tid=6477 errno=4 - Interrupted system call
     1	0x000000000040147b: main at debug-exceptions.c:230
     2	0x000003ff8aa60de3: ?? ??:0
     3	0x0000000000401517: _start at :?
  Failed guest assert: hw_bp_addr == PC(hw_bp) at
aarch64/debug-exceptions.c:105
	values: 0, 0x401794


I guess it is not an expected result. Any known bug waiting on the list?


Thanks

Eric

>>>
>>> Signed-off-by: Ricardo Koller <ricarkol@google.com>
>>> ---
>>>  tools/testing/selftests/kvm/Makefile          |   2 +-
>>>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
>>>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
>>>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
>>>  4 files changed, 333 insertions(+), 1 deletion(-)
>>>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>>
>>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
>>> index 4e548d7ab0ab..618c5903f478 100644
>>> --- a/tools/testing/selftests/kvm/Makefile
>>> +++ b/tools/testing/selftests/kvm/Makefile
>>> @@ -35,7 +35,7 @@ endif
>>>  
>>>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
>>>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
>>> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
>>> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
>>>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
>>>  
>>>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
>>> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
>>> index b7fa0c8551db..40aae31b4afc 100644
>>> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
>>> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
>>> @@ -8,6 +8,7 @@
>>>  #define SELFTEST_KVM_PROCESSOR_H
>>>  
>>>  #include "kvm_util.h"
>>> +#include <linux/stringify.h>
>>>  
>>>  
>>>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
>>> @@ -18,6 +19,7 @@
>>>  #define MAIR_EL1	3, 0, 10, 2, 0
>>>  #define TTBR0_EL1	3, 0,  2, 0, 0
>>>  #define SCTLR_EL1	3, 0,  1, 0, 0
>>> +#define VBAR_EL1	3, 0, 12, 0, 0
>>>  
>>>  /*
>>>   * Default MAIR
>>> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
>>>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
>>>  			      struct kvm_vcpu_init *init, void *guest_code);
>>>  
>>> +struct ex_regs {
>>> +	u64 regs[31];
>>> +	u64 sp;
>>> +	u64 pc;
>>> +	u64 pstate;
>>> +};
>>> +
>>> +#define VECTOR_NUM	16
>>> +
>>> +enum {
>>> +	VECTOR_SYNC_CURRENT_SP0,
>>> +	VECTOR_IRQ_CURRENT_SP0,
>>> +	VECTOR_FIQ_CURRENT_SP0,
>>> +	VECTOR_ERROR_CURRENT_SP0,
>>> +
>>> +	VECTOR_SYNC_CURRENT,
>>> +	VECTOR_IRQ_CURRENT,
>>> +	VECTOR_FIQ_CURRENT,
>>> +	VECTOR_ERROR_CURRENT,
>>> +
>>> +	VECTOR_SYNC_LOWER_64,
>>> +	VECTOR_IRQ_LOWER_64,
>>> +	VECTOR_FIQ_LOWER_64,
>>> +	VECTOR_ERROR_LOWER_64,
>>> +
>>> +	VECTOR_SYNC_LOWER_32,
>>> +	VECTOR_IRQ_LOWER_32,
>>> +	VECTOR_FIQ_LOWER_32,
>>> +	VECTOR_ERROR_LOWER_32,
>>> +};
>>> +
>>> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
>>> +			   (v) == VECTOR_SYNC_CURRENT     || \
>>> +			   (v) == VECTOR_SYNC_LOWER_64    || \
>>> +			   (v) == VECTOR_SYNC_LOWER_32)
>>> +
>>> +/* Some common EC (Exception classes) */
>>> +#define ESR_EC_ILLEGAL_INS	0x0e
>>> +#define ESR_EC_SVC64		0x15
>>> +#define ESR_EC_IABORT_CURRENT	0x21
>>> +#define ESR_EC_DABORT_CURRENT	0x25
>>> +#define ESR_EC_SERROR		0x2f
>>> +#define ESR_EC_HW_BP_CURRENT	0x31
>>> +#define ESR_EC_SSTEP_CURRENT	0x33
>>> +#define ESR_EC_WP_CURRENT	0x35
>>> +#define ESR_EC_BRK_INS		0x3C
>>> +
>>> +#define ESR_EC_NUM		64
>>> +
>>> +#define ESR_EC_SHIFT		26
>>> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
>>> +
>>> +void vm_init_descriptor_tables(struct kvm_vm *vm);
>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
>>> +
>>> +typedef void(*handler_fn)(struct ex_regs *);
>>> +void vm_install_exception_handler(struct kvm_vm *vm,
>>> +		int vector, int ec, handler_fn handler);
>>> +void vm_install_vector_handler(struct kvm_vm *vm,
>>> +		int vector, handler_fn handler);
>>> +
>>> +#define SPSR_D          (1 << 9)
>>> +#define SPSR_SS         (1 << 21)
>>> +
>>> +#define write_sysreg(reg, val)						  \
>>> +({									  \
>>> +	u64 __val = (u64)(val);						  \
>>> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
>>> +})
>>> +
>>> +#define read_sysreg(reg)						  \
>>> +({	u64 val;							  \
>>> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
>>> +	val;								  \
>>> +})
>>> +
>>>  #endif /* SELFTEST_KVM_PROCESSOR_H */
>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>> new file mode 100644
>>> index 000000000000..8a560021892b
>>> --- /dev/null
>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>> @@ -0,0 +1,130 @@
>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>> +.macro save_registers, vector
>>> +	add	sp, sp, #-16 * 17
>>> +
>>> +	stp	x0, x1, [sp, #16 * 0]
>>> +	stp	x2, x3, [sp, #16 * 1]
>>> +	stp	x4, x5, [sp, #16 * 2]
>>> +	stp	x6, x7, [sp, #16 * 3]
>>> +	stp	x8, x9, [sp, #16 * 4]
>>> +	stp	x10, x11, [sp, #16 * 5]
>>> +	stp	x12, x13, [sp, #16 * 6]
>>> +	stp	x14, x15, [sp, #16 * 7]
>>> +	stp	x16, x17, [sp, #16 * 8]
>>> +	stp	x18, x19, [sp, #16 * 9]
>>> +	stp	x20, x21, [sp, #16 * 10]
>>> +	stp	x22, x23, [sp, #16 * 11]
>>> +	stp	x24, x25, [sp, #16 * 12]
>>> +	stp	x26, x27, [sp, #16 * 13]
>>> +	stp	x28, x29, [sp, #16 * 14]
>>> +
>>> +	.if \vector >= 8
>>> +	mrs	x1, sp_el0
>>
>> I'm still a bit perplexed by this. SP_EL0 is never changed, since you
>> always run in handler mode. Therefore, saving/restoring it is only
>> overhead. If an exception handler wants to introspect it, it is
>> already available in the relevant system register.
>>
>> Or did you have something else in mind for it?
>>
> 
> Not really. The reason for saving sp_el0 in there was just for
> consistency, so that handlers for both el0 and el1 exceptions could get
> the sp at regs->sp.
> 
> Restoring sp_el0 might be too much. So, what do you think of this v3: we
> keep the saving of sp_el0 into regs->sp (to keep things the same between
> el0 and el1) and delete the restoring of sp_el0?
> 
> Thanks,
> Ricardo
> 
>>> +	.else
>>> +	/*
>>> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
>>> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
>>> +	 * return from the exception so handlers can not update it.
>>> +	 */
>>> +	mov	x1, sp
>>> +	.endif
>>> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
>>> +
>>> +	mrs	x1, elr_el1
>>> +	mrs	x2, spsr_el1
>>> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
>>> +.endm
>>> +
>>> +.macro restore_registers, vector
>>> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
>>> +	msr	elr_el1, x1
>>> +	msr	spsr_el1, x2
>>> +
>>> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
>>> +	.if \vector >= 8
>>> +	msr	sp_el0, x1
>>> +	.endif
>>> +
>>> +	ldp	x28, x29, [sp, #16 * 14]
>>> +	ldp	x26, x27, [sp, #16 * 13]
>>> +	ldp	x24, x25, [sp, #16 * 12]
>>> +	ldp	x22, x23, [sp, #16 * 11]
>>> +	ldp	x20, x21, [sp, #16 * 10]
>>> +	ldp	x18, x19, [sp, #16 * 9]
>>> +	ldp	x16, x17, [sp, #16 * 8]
>>> +	ldp	x14, x15, [sp, #16 * 7]
>>> +	ldp	x12, x13, [sp, #16 * 6]
>>> +	ldp	x10, x11, [sp, #16 * 5]
>>> +	ldp	x8, x9, [sp, #16 * 4]
>>> +	ldp	x6, x7, [sp, #16 * 3]
>>> +	ldp	x4, x5, [sp, #16 * 2]
>>> +	ldp	x2, x3, [sp, #16 * 1]
>>> +	ldp	x0, x1, [sp, #16 * 0]
>>> +
>>> +	add	sp, sp, #16 * 17
>>> +
>>> +	eret
>>> +.endm
>>> +
>>> +.pushsection ".entry.text", "ax"
>>> +.balign 0x800
>>> +.global vectors
>>> +vectors:
>>> +.popsection
>>> +
>>> +.set	vector, 0
>>> +
>>> +/*
>>> + * Build an exception handler for vector and append a jump to it into
>>> + * vectors (while making sure that it's 0x80 aligned).
>>> + */
>>> +.macro HANDLER, label
>>> +handler_\()\label:
>>> +	save_registers vector
>>> +	mov	x0, sp
>>> +	mov	x1, #vector
>>> +	bl	route_exception
>>> +	restore_registers vector
>>> +
>>> +.pushsection ".entry.text", "ax"
>>> +.balign 0x80
>>> +	b	handler_\()\label
>>> +.popsection
>>> +
>>> +.set	vector, vector + 1
>>> +.endm
>>> +
>>> +.macro HANDLER_INVALID
>>> +.pushsection ".entry.text", "ax"
>>> +.balign 0x80
>>> +/* This will abort so no need to save and restore registers. */
>>> +	mov	x0, #vector
>>> +	b	kvm_exit_unexpected_vector
>>> +.popsection
>>> +
>>> +.set	vector, vector + 1
>>> +.endm
>>> +
>>> +/*
>>> + * Caution: be sure to not add anything between the declaration of vectors
>>> + * above and these macro calls that will build the vectors table below it.
>>> + */
>>> +	HANDLER_INVALID                         // Synchronous EL1t
>>> +	HANDLER_INVALID                         // IRQ EL1t
>>> +	HANDLER_INVALID                         // FIQ EL1t
>>> +	HANDLER_INVALID                         // Error EL1t
>>> +
>>> +	HANDLER	el1h_sync                       // Synchronous EL1h
>>> +	HANDLER	el1h_irq                        // IRQ EL1h
>>> +	HANDLER el1h_fiq                        // FIQ EL1h
>>> +	HANDLER	el1h_error                      // Error EL1h
>>> +
>>> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
>>> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
>>> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
>>> +	HANDLER	el0_error_64                    // Error 64-bit EL0
>>> +
>>> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
>>> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
>>> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
>>> +	HANDLER	el0_error_32                    // Error 32-bit EL0
>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>> index cee92d477dc0..25be71ec88be 100644
>>> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>> @@ -6,6 +6,7 @@
>>>   */
>>>  
>>>  #include <linux/compiler.h>
>>> +#include <assert.h>
>>>  
>>>  #include "kvm_util.h"
>>>  #include "../kvm_util_internal.h"
>>> @@ -14,6 +15,8 @@
>>>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
>>>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
>>>  
>>> +vm_vaddr_t exception_handlers;
>>> +
>>>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
>>>  {
>>>  	return (v + vm->page_size) & ~(vm->page_size - 1);
>>> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
>>>  	va_end(ap);
>>>  }
>>>  
>>> +void kvm_exit_unexpected_vector(int vector)
>>> +{
>>> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
>>> +}
>>> +
>>> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
>>> +{
>>> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
>>> +}
>>> +
>>>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
>>>  {
>>> +	struct ucall uc;
>>> +
>>> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
>>> +		return;
>>> +
>>> +	if (uc.args[2]) /* valid_ec */ {
>>> +		assert(VECTOR_IS_SYNC(uc.args[0]));
>>> +		TEST_ASSERT(false,
>>> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
>>> +			uc.args[0], uc.args[1]);
>>> +	} else {
>>> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
>>> +		TEST_ASSERT(false,
>>> +			"Unexpected exception (vector:0x%lx)",
>>> +			uc.args[0]);
>>> +	}
>>> +}
>>> +
>>> +/*
>>> + * This exception handling code was heavily inspired on kvm-unit-tests. There
>>> + * is a set of default vector handlers stored in vector_handlers. These default
>>> + * vector handlers call user-installed handlers stored in exception_handlers.
>>> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
>>> + * (vector, ec=0).
>>> + */
>>> +
>>> +typedef void(*vector_fn)(struct ex_regs *, int vector);
>>> +
>>> +struct handlers {
>>> +	vector_fn vector_handlers[VECTOR_NUM];
>>> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
>>> +};
>>> +
>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
>>> +{
>>> +	extern char vectors;
>>> +
>>> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
>>> +}
>>> +
>>> +void default_sync_handler(struct ex_regs *regs, int vector)
>>> +{
>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>> +	uint64_t esr = read_sysreg(esr_el1);
>>> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
>>> +
>>> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
>>> +
>>> +	if (handlers && handlers->exception_handlers[vector][ec])
>>> +		handlers->exception_handlers[vector][ec](regs);
>>> +	else
>>> +		kvm_exit_unexpected_exception(vector, ec);
>>> +}
>>> +
>>> +void default_irq_handler(struct ex_regs *regs, int vector)
>>> +{
>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>> +
>>> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
>>> +
>>> +	if (handlers && handlers->exception_handlers[vector][0])
>>> +		handlers->exception_handlers[vector][0](regs);
>>> +	else
>>> +		kvm_exit_unexpected_vector(vector);
>>> +}
>>> +
>>> +void route_exception(struct ex_regs *regs, int vector)
>>> +{
>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>> +
>>> +	if (handlers && handlers->vector_handlers[vector])
>>> +		handlers->vector_handlers[vector](regs, vector);
>>> +	else
>>> +		kvm_exit_unexpected_vector(vector);
>>> +}
>>> +
>>> +void vm_init_descriptor_tables(struct kvm_vm *vm)
>>> +{
>>> +	struct handlers *handlers;
>>> +
>>> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
>>> +			vm->page_size, 0, 0);
>>> +
>>> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
>>> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
>>> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
>>> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
>>
>> How about FIQ, Error? Although they are unlikely, they are valid
>> exceptions.
>>
>>> +
>>> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
>>> +}
>>> +
>>> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
>>> +			 void (*handler)(struct ex_regs *))
>>> +{
>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>> +
>>> +	assert(VECTOR_IS_SYNC(vector));
>>> +	assert(vector < VECTOR_NUM);
>>> +	assert(ec < ESR_EC_NUM);
>>> +	handlers->exception_handlers[vector][ec] = handler;
>>> +}
>>> +
>>> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
>>> +			 void (*handler)(struct ex_regs *))
>>> +{
>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>> +
>>> +	assert(!VECTOR_IS_SYNC(vector));
>>> +	assert(vector < VECTOR_NUM);
>>> +	handlers->exception_handlers[vector][0] = handler;
>>>  }
>>
>> Thanks,
>>
>> 	M.
>>
>> -- 
>> Without deviation from the norm, progress is not possible.
> 

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
> Hi Ricardo,
> 

Hi Eric,

Thank you very much for the test.

> On 5/3/21 9:12 PM, Ricardo Koller wrote:
> > On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
> >> On Sat, 01 May 2021 00:24:06 +0100,
> >> Ricardo Koller <ricarkol@google.com> wrote:
> >>>
> >>> Add the infrastructure needed to enable exception handling in aarch64
> >>> selftests. The exception handling defaults to an unhandled-exception
> >>> handler which aborts the test, just like x86. These handlers can be
> >>> overridden by calling vm_install_vector_handler(vector) or
> >>> vm_install_exception_handler(vector, ec). The unhandled exception
> >>> reporting from the guest is done using the ucall type introduced in a
> >>> previous commit, UCALL_UNHANDLED.
> >>>
> >>> The exception handling code is heavily inspired on kvm-unit-tests.
> 
> running the test on 5.12 I get
> 
> ==== Test Assertion Failure ====
>   aarch64/debug-exceptions.c:232: false
>   pid=6477 tid=6477 errno=4 - Interrupted system call
>      1	0x000000000040147b: main at debug-exceptions.c:230
>      2	0x000003ff8aa60de3: ?? ??:0
>      3	0x0000000000401517: _start at :?
>   Failed guest assert: hw_bp_addr == PC(hw_bp) at
> aarch64/debug-exceptions.c:105
> 	values: 0, 0x401794
> 
> 
> I guess it is not an expected result. Any known bug waiting on the list?
>

Not expected. That should work, or at least abort early because there is
no HW breakpoints support.

I'm trying to reproduce the failure; can you help me with some
questions, please?

- does your setup have support for hardware breakpoints? Can you try a
  'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
  found ...'. If there is no such line it's very likely that the check
  for "debug_ver >= 6" is not enough and the test should check for
  "num_breakpoints > 0".
- does it fail consistently (every single attempt)?

Thanks!
Ricardo

> 
> Thanks
> 
> Eric


> >>>
> >>> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> >>> ---
> >>>  tools/testing/selftests/kvm/Makefile          |   2 +-
> >>>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
> >>>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
> >>>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
> >>>  4 files changed, 333 insertions(+), 1 deletion(-)
> >>>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>>
> >>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> >>> index 4e548d7ab0ab..618c5903f478 100644
> >>> --- a/tools/testing/selftests/kvm/Makefile
> >>> +++ b/tools/testing/selftests/kvm/Makefile
> >>> @@ -35,7 +35,7 @@ endif
> >>>  
> >>>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
> >>>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
> >>> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
> >>> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
> >>>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
> >>>  
> >>>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
> >>> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>> index b7fa0c8551db..40aae31b4afc 100644
> >>> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>> @@ -8,6 +8,7 @@
> >>>  #define SELFTEST_KVM_PROCESSOR_H
> >>>  
> >>>  #include "kvm_util.h"
> >>> +#include <linux/stringify.h>
> >>>  
> >>>  
> >>>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> >>> @@ -18,6 +19,7 @@
> >>>  #define MAIR_EL1	3, 0, 10, 2, 0
> >>>  #define TTBR0_EL1	3, 0,  2, 0, 0
> >>>  #define SCTLR_EL1	3, 0,  1, 0, 0
> >>> +#define VBAR_EL1	3, 0, 12, 0, 0
> >>>  
> >>>  /*
> >>>   * Default MAIR
> >>> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
> >>>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
> >>>  			      struct kvm_vcpu_init *init, void *guest_code);
> >>>  
> >>> +struct ex_regs {
> >>> +	u64 regs[31];
> >>> +	u64 sp;
> >>> +	u64 pc;
> >>> +	u64 pstate;
> >>> +};
> >>> +
> >>> +#define VECTOR_NUM	16
> >>> +
> >>> +enum {
> >>> +	VECTOR_SYNC_CURRENT_SP0,
> >>> +	VECTOR_IRQ_CURRENT_SP0,
> >>> +	VECTOR_FIQ_CURRENT_SP0,
> >>> +	VECTOR_ERROR_CURRENT_SP0,
> >>> +
> >>> +	VECTOR_SYNC_CURRENT,
> >>> +	VECTOR_IRQ_CURRENT,
> >>> +	VECTOR_FIQ_CURRENT,
> >>> +	VECTOR_ERROR_CURRENT,
> >>> +
> >>> +	VECTOR_SYNC_LOWER_64,
> >>> +	VECTOR_IRQ_LOWER_64,
> >>> +	VECTOR_FIQ_LOWER_64,
> >>> +	VECTOR_ERROR_LOWER_64,
> >>> +
> >>> +	VECTOR_SYNC_LOWER_32,
> >>> +	VECTOR_IRQ_LOWER_32,
> >>> +	VECTOR_FIQ_LOWER_32,
> >>> +	VECTOR_ERROR_LOWER_32,
> >>> +};
> >>> +
> >>> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
> >>> +			   (v) == VECTOR_SYNC_CURRENT     || \
> >>> +			   (v) == VECTOR_SYNC_LOWER_64    || \
> >>> +			   (v) == VECTOR_SYNC_LOWER_32)
> >>> +
> >>> +/* Some common EC (Exception classes) */
> >>> +#define ESR_EC_ILLEGAL_INS	0x0e
> >>> +#define ESR_EC_SVC64		0x15
> >>> +#define ESR_EC_IABORT_CURRENT	0x21
> >>> +#define ESR_EC_DABORT_CURRENT	0x25
> >>> +#define ESR_EC_SERROR		0x2f
> >>> +#define ESR_EC_HW_BP_CURRENT	0x31
> >>> +#define ESR_EC_SSTEP_CURRENT	0x33
> >>> +#define ESR_EC_WP_CURRENT	0x35
> >>> +#define ESR_EC_BRK_INS		0x3C
> >>> +
> >>> +#define ESR_EC_NUM		64
> >>> +
> >>> +#define ESR_EC_SHIFT		26
> >>> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
> >>> +
> >>> +void vm_init_descriptor_tables(struct kvm_vm *vm);
> >>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
> >>> +
> >>> +typedef void(*handler_fn)(struct ex_regs *);
> >>> +void vm_install_exception_handler(struct kvm_vm *vm,
> >>> +		int vector, int ec, handler_fn handler);
> >>> +void vm_install_vector_handler(struct kvm_vm *vm,
> >>> +		int vector, handler_fn handler);
> >>> +
> >>> +#define SPSR_D          (1 << 9)
> >>> +#define SPSR_SS         (1 << 21)
> >>> +
> >>> +#define write_sysreg(reg, val)						  \
> >>> +({									  \
> >>> +	u64 __val = (u64)(val);						  \
> >>> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
> >>> +})
> >>> +
> >>> +#define read_sysreg(reg)						  \
> >>> +({	u64 val;							  \
> >>> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
> >>> +	val;								  \
> >>> +})
> >>> +
> >>>  #endif /* SELFTEST_KVM_PROCESSOR_H */
> >>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>> new file mode 100644
> >>> index 000000000000..8a560021892b
> >>> --- /dev/null
> >>> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>> @@ -0,0 +1,130 @@
> >>> +/* SPDX-License-Identifier: GPL-2.0 */
> >>> +.macro save_registers, vector
> >>> +	add	sp, sp, #-16 * 17
> >>> +
> >>> +	stp	x0, x1, [sp, #16 * 0]
> >>> +	stp	x2, x3, [sp, #16 * 1]
> >>> +	stp	x4, x5, [sp, #16 * 2]
> >>> +	stp	x6, x7, [sp, #16 * 3]
> >>> +	stp	x8, x9, [sp, #16 * 4]
> >>> +	stp	x10, x11, [sp, #16 * 5]
> >>> +	stp	x12, x13, [sp, #16 * 6]
> >>> +	stp	x14, x15, [sp, #16 * 7]
> >>> +	stp	x16, x17, [sp, #16 * 8]
> >>> +	stp	x18, x19, [sp, #16 * 9]
> >>> +	stp	x20, x21, [sp, #16 * 10]
> >>> +	stp	x22, x23, [sp, #16 * 11]
> >>> +	stp	x24, x25, [sp, #16 * 12]
> >>> +	stp	x26, x27, [sp, #16 * 13]
> >>> +	stp	x28, x29, [sp, #16 * 14]
> >>> +
> >>> +	.if \vector >= 8
> >>> +	mrs	x1, sp_el0
> >>
> >> I'm still a bit perplexed by this. SP_EL0 is never changed, since you
> >> always run in handler mode. Therefore, saving/restoring it is only
> >> overhead. If an exception handler wants to introspect it, it is
> >> already available in the relevant system register.
> >>
> >> Or did you have something else in mind for it?
> >>
> > 
> > Not really. The reason for saving sp_el0 in there was just for
> > consistency, so that handlers for both el0 and el1 exceptions could get
> > the sp at regs->sp.
> > 
> > Restoring sp_el0 might be too much. So, what do you think of this v3: we
> > keep the saving of sp_el0 into regs->sp (to keep things the same between
> > el0 and el1) and delete the restoring of sp_el0?
> > 
> > Thanks,
> > Ricardo
> > 
> >>> +	.else
> >>> +	/*
> >>> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
> >>> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
> >>> +	 * return from the exception so handlers can not update it.
> >>> +	 */
> >>> +	mov	x1, sp
> >>> +	.endif
> >>> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
> >>> +
> >>> +	mrs	x1, elr_el1
> >>> +	mrs	x2, spsr_el1
> >>> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> >>> +.endm
> >>> +
> >>> +.macro restore_registers, vector
> >>> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> >>> +	msr	elr_el1, x1
> >>> +	msr	spsr_el1, x2
> >>> +
> >>> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
> >>> +	.if \vector >= 8
> >>> +	msr	sp_el0, x1
> >>> +	.endif
> >>> +
> >>> +	ldp	x28, x29, [sp, #16 * 14]
> >>> +	ldp	x26, x27, [sp, #16 * 13]
> >>> +	ldp	x24, x25, [sp, #16 * 12]
> >>> +	ldp	x22, x23, [sp, #16 * 11]
> >>> +	ldp	x20, x21, [sp, #16 * 10]
> >>> +	ldp	x18, x19, [sp, #16 * 9]
> >>> +	ldp	x16, x17, [sp, #16 * 8]
> >>> +	ldp	x14, x15, [sp, #16 * 7]
> >>> +	ldp	x12, x13, [sp, #16 * 6]
> >>> +	ldp	x10, x11, [sp, #16 * 5]
> >>> +	ldp	x8, x9, [sp, #16 * 4]
> >>> +	ldp	x6, x7, [sp, #16 * 3]
> >>> +	ldp	x4, x5, [sp, #16 * 2]
> >>> +	ldp	x2, x3, [sp, #16 * 1]
> >>> +	ldp	x0, x1, [sp, #16 * 0]
> >>> +
> >>> +	add	sp, sp, #16 * 17
> >>> +
> >>> +	eret
> >>> +.endm
> >>> +
> >>> +.pushsection ".entry.text", "ax"
> >>> +.balign 0x800
> >>> +.global vectors
> >>> +vectors:
> >>> +.popsection
> >>> +
> >>> +.set	vector, 0
> >>> +
> >>> +/*
> >>> + * Build an exception handler for vector and append a jump to it into
> >>> + * vectors (while making sure that it's 0x80 aligned).
> >>> + */
> >>> +.macro HANDLER, label
> >>> +handler_\()\label:
> >>> +	save_registers vector
> >>> +	mov	x0, sp
> >>> +	mov	x1, #vector
> >>> +	bl	route_exception
> >>> +	restore_registers vector
> >>> +
> >>> +.pushsection ".entry.text", "ax"
> >>> +.balign 0x80
> >>> +	b	handler_\()\label
> >>> +.popsection
> >>> +
> >>> +.set	vector, vector + 1
> >>> +.endm
> >>> +
> >>> +.macro HANDLER_INVALID
> >>> +.pushsection ".entry.text", "ax"
> >>> +.balign 0x80
> >>> +/* This will abort so no need to save and restore registers. */
> >>> +	mov	x0, #vector
> >>> +	b	kvm_exit_unexpected_vector
> >>> +.popsection
> >>> +
> >>> +.set	vector, vector + 1
> >>> +.endm
> >>> +
> >>> +/*
> >>> + * Caution: be sure to not add anything between the declaration of vectors
> >>> + * above and these macro calls that will build the vectors table below it.
> >>> + */
> >>> +	HANDLER_INVALID                         // Synchronous EL1t
> >>> +	HANDLER_INVALID                         // IRQ EL1t
> >>> +	HANDLER_INVALID                         // FIQ EL1t
> >>> +	HANDLER_INVALID                         // Error EL1t
> >>> +
> >>> +	HANDLER	el1h_sync                       // Synchronous EL1h
> >>> +	HANDLER	el1h_irq                        // IRQ EL1h
> >>> +	HANDLER el1h_fiq                        // FIQ EL1h
> >>> +	HANDLER	el1h_error                      // Error EL1h
> >>> +
> >>> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
> >>> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
> >>> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
> >>> +	HANDLER	el0_error_64                    // Error 64-bit EL0
> >>> +
> >>> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
> >>> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
> >>> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
> >>> +	HANDLER	el0_error_32                    // Error 32-bit EL0
> >>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>> index cee92d477dc0..25be71ec88be 100644
> >>> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>> @@ -6,6 +6,7 @@
> >>>   */
> >>>  
> >>>  #include <linux/compiler.h>
> >>> +#include <assert.h>
> >>>  
> >>>  #include "kvm_util.h"
> >>>  #include "../kvm_util_internal.h"
> >>> @@ -14,6 +15,8 @@
> >>>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
> >>>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
> >>>  
> >>> +vm_vaddr_t exception_handlers;
> >>> +
> >>>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
> >>>  {
> >>>  	return (v + vm->page_size) & ~(vm->page_size - 1);
> >>> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
> >>>  	va_end(ap);
> >>>  }
> >>>  
> >>> +void kvm_exit_unexpected_vector(int vector)
> >>> +{
> >>> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
> >>> +}
> >>> +
> >>> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
> >>> +{
> >>> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
> >>> +}
> >>> +
> >>>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
> >>>  {
> >>> +	struct ucall uc;
> >>> +
> >>> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
> >>> +		return;
> >>> +
> >>> +	if (uc.args[2]) /* valid_ec */ {
> >>> +		assert(VECTOR_IS_SYNC(uc.args[0]));
> >>> +		TEST_ASSERT(false,
> >>> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
> >>> +			uc.args[0], uc.args[1]);
> >>> +	} else {
> >>> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
> >>> +		TEST_ASSERT(false,
> >>> +			"Unexpected exception (vector:0x%lx)",
> >>> +			uc.args[0]);
> >>> +	}
> >>> +}
> >>> +
> >>> +/*
> >>> + * This exception handling code was heavily inspired on kvm-unit-tests. There
> >>> + * is a set of default vector handlers stored in vector_handlers. These default
> >>> + * vector handlers call user-installed handlers stored in exception_handlers.
> >>> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
> >>> + * (vector, ec=0).
> >>> + */
> >>> +
> >>> +typedef void(*vector_fn)(struct ex_regs *, int vector);
> >>> +
> >>> +struct handlers {
> >>> +	vector_fn vector_handlers[VECTOR_NUM];
> >>> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
> >>> +};
> >>> +
> >>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
> >>> +{
> >>> +	extern char vectors;
> >>> +
> >>> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
> >>> +}
> >>> +
> >>> +void default_sync_handler(struct ex_regs *regs, int vector)
> >>> +{
> >>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>> +	uint64_t esr = read_sysreg(esr_el1);
> >>> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
> >>> +
> >>> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
> >>> +
> >>> +	if (handlers && handlers->exception_handlers[vector][ec])
> >>> +		handlers->exception_handlers[vector][ec](regs);
> >>> +	else
> >>> +		kvm_exit_unexpected_exception(vector, ec);
> >>> +}
> >>> +
> >>> +void default_irq_handler(struct ex_regs *regs, int vector)
> >>> +{
> >>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>> +
> >>> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
> >>> +
> >>> +	if (handlers && handlers->exception_handlers[vector][0])
> >>> +		handlers->exception_handlers[vector][0](regs);
> >>> +	else
> >>> +		kvm_exit_unexpected_vector(vector);
> >>> +}
> >>> +
> >>> +void route_exception(struct ex_regs *regs, int vector)
> >>> +{
> >>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>> +
> >>> +	if (handlers && handlers->vector_handlers[vector])
> >>> +		handlers->vector_handlers[vector](regs, vector);
> >>> +	else
> >>> +		kvm_exit_unexpected_vector(vector);
> >>> +}
> >>> +
> >>> +void vm_init_descriptor_tables(struct kvm_vm *vm)
> >>> +{
> >>> +	struct handlers *handlers;
> >>> +
> >>> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
> >>> +			vm->page_size, 0, 0);
> >>> +
> >>> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
> >>> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
> >>> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
> >>> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
> >>
> >> How about FIQ, Error? Although they are unlikely, they are valid
> >> exceptions.
> >>
> >>> +
> >>> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
> >>> +}
> >>> +
> >>> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
> >>> +			 void (*handler)(struct ex_regs *))
> >>> +{
> >>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>> +
> >>> +	assert(VECTOR_IS_SYNC(vector));
> >>> +	assert(vector < VECTOR_NUM);
> >>> +	assert(ec < ESR_EC_NUM);
> >>> +	handlers->exception_handlers[vector][ec] = handler;
> >>> +}
> >>> +
> >>> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
> >>> +			 void (*handler)(struct ex_regs *))
> >>> +{
> >>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>> +
> >>> +	assert(!VECTOR_IS_SYNC(vector));
> >>> +	assert(vector < VECTOR_NUM);
> >>> +	handlers->exception_handlers[vector][0] = handler;
> >>>  }
> >>
> >> Thanks,
> >>
> >> 	M.
> >>
> >> -- 
> >> Without deviation from the norm, progress is not possible.
> > 
> 

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Auger Eric]

Hi Ricardo,

On 5/6/21 9:14 PM, Ricardo Koller wrote:
> On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
>> Hi Ricardo,
>>
> 
> Hi Eric,
> 
> Thank you very much for the test.
> 
>> On 5/3/21 9:12 PM, Ricardo Koller wrote:
>>> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
>>>> On Sat, 01 May 2021 00:24:06 +0100,
>>>> Ricardo Koller <ricarkol@google.com> wrote:
>>>>>
>>>>> Add the infrastructure needed to enable exception handling in aarch64
>>>>> selftests. The exception handling defaults to an unhandled-exception
>>>>> handler which aborts the test, just like x86. These handlers can be
>>>>> overridden by calling vm_install_vector_handler(vector) or
>>>>> vm_install_exception_handler(vector, ec). The unhandled exception
>>>>> reporting from the guest is done using the ucall type introduced in a
>>>>> previous commit, UCALL_UNHANDLED.
>>>>>
>>>>> The exception handling code is heavily inspired on kvm-unit-tests.
>>
>> running the test on 5.12 I get
>>
>> ==== Test Assertion Failure ====
>>   aarch64/debug-exceptions.c:232: false
>>   pid=6477 tid=6477 errno=4 - Interrupted system call
>>      1	0x000000000040147b: main at debug-exceptions.c:230
>>      2	0x000003ff8aa60de3: ?? ??:0
>>      3	0x0000000000401517: _start at :?
>>   Failed guest assert: hw_bp_addr == PC(hw_bp) at
>> aarch64/debug-exceptions.c:105
>> 	values: 0, 0x401794
>>
>>
>> I guess it is not an expected result. Any known bug waiting on the list?
>>
> 
> Not expected. That should work, or at least abort early because there is
> no HW breakpoints support.
> 
> I'm trying to reproduce the failure; can you help me with some
> questions, please?
sure, please find the answers below.
> 
> - does your setup have support for hardware breakpoints? Can you try a
>   'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
>   found ...'. If there is no such line it's very likely that the check
>   for "debug_ver >= 6" is not enough and the test should check for
>   "num_breakpoints > 0".
[   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
> - does it fail consistently (every single attempt)?
yes it does.

I will try to find some time to investigate too

Thanks

Eric
> 
> Thanks!
> Ricardo
> 
>>
>> Thanks
>>
>> Eric
> 
> 
>>>>>
>>>>> Signed-off-by: Ricardo Koller <ricarkol@google.com>
>>>>> ---
>>>>>  tools/testing/selftests/kvm/Makefile          |   2 +-
>>>>>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
>>>>>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
>>>>>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
>>>>>  4 files changed, 333 insertions(+), 1 deletion(-)
>>>>>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>>>>
>>>>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
>>>>> index 4e548d7ab0ab..618c5903f478 100644
>>>>> --- a/tools/testing/selftests/kvm/Makefile
>>>>> +++ b/tools/testing/selftests/kvm/Makefile
>>>>> @@ -35,7 +35,7 @@ endif
>>>>>  
>>>>>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
>>>>>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
>>>>> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
>>>>> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
>>>>>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
>>>>>  
>>>>>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
>>>>> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
>>>>> index b7fa0c8551db..40aae31b4afc 100644
>>>>> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
>>>>> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
>>>>> @@ -8,6 +8,7 @@
>>>>>  #define SELFTEST_KVM_PROCESSOR_H
>>>>>  
>>>>>  #include "kvm_util.h"
>>>>> +#include <linux/stringify.h>
>>>>>  
>>>>>  
>>>>>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
>>>>> @@ -18,6 +19,7 @@
>>>>>  #define MAIR_EL1	3, 0, 10, 2, 0
>>>>>  #define TTBR0_EL1	3, 0,  2, 0, 0
>>>>>  #define SCTLR_EL1	3, 0,  1, 0, 0
>>>>> +#define VBAR_EL1	3, 0, 12, 0, 0
>>>>>  
>>>>>  /*
>>>>>   * Default MAIR
>>>>> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
>>>>>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
>>>>>  			      struct kvm_vcpu_init *init, void *guest_code);
>>>>>  
>>>>> +struct ex_regs {
>>>>> +	u64 regs[31];
>>>>> +	u64 sp;
>>>>> +	u64 pc;
>>>>> +	u64 pstate;
>>>>> +};
>>>>> +
>>>>> +#define VECTOR_NUM	16
>>>>> +
>>>>> +enum {
>>>>> +	VECTOR_SYNC_CURRENT_SP0,
>>>>> +	VECTOR_IRQ_CURRENT_SP0,
>>>>> +	VECTOR_FIQ_CURRENT_SP0,
>>>>> +	VECTOR_ERROR_CURRENT_SP0,
>>>>> +
>>>>> +	VECTOR_SYNC_CURRENT,
>>>>> +	VECTOR_IRQ_CURRENT,
>>>>> +	VECTOR_FIQ_CURRENT,
>>>>> +	VECTOR_ERROR_CURRENT,
>>>>> +
>>>>> +	VECTOR_SYNC_LOWER_64,
>>>>> +	VECTOR_IRQ_LOWER_64,
>>>>> +	VECTOR_FIQ_LOWER_64,
>>>>> +	VECTOR_ERROR_LOWER_64,
>>>>> +
>>>>> +	VECTOR_SYNC_LOWER_32,
>>>>> +	VECTOR_IRQ_LOWER_32,
>>>>> +	VECTOR_FIQ_LOWER_32,
>>>>> +	VECTOR_ERROR_LOWER_32,
>>>>> +};
>>>>> +
>>>>> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
>>>>> +			   (v) == VECTOR_SYNC_CURRENT     || \
>>>>> +			   (v) == VECTOR_SYNC_LOWER_64    || \
>>>>> +			   (v) == VECTOR_SYNC_LOWER_32)
>>>>> +
>>>>> +/* Some common EC (Exception classes) */
>>>>> +#define ESR_EC_ILLEGAL_INS	0x0e
>>>>> +#define ESR_EC_SVC64		0x15
>>>>> +#define ESR_EC_IABORT_CURRENT	0x21
>>>>> +#define ESR_EC_DABORT_CURRENT	0x25
>>>>> +#define ESR_EC_SERROR		0x2f
>>>>> +#define ESR_EC_HW_BP_CURRENT	0x31
>>>>> +#define ESR_EC_SSTEP_CURRENT	0x33
>>>>> +#define ESR_EC_WP_CURRENT	0x35
>>>>> +#define ESR_EC_BRK_INS		0x3C
>>>>> +
>>>>> +#define ESR_EC_NUM		64
>>>>> +
>>>>> +#define ESR_EC_SHIFT		26
>>>>> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
>>>>> +
>>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm);
>>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
>>>>> +
>>>>> +typedef void(*handler_fn)(struct ex_regs *);
>>>>> +void vm_install_exception_handler(struct kvm_vm *vm,
>>>>> +		int vector, int ec, handler_fn handler);
>>>>> +void vm_install_vector_handler(struct kvm_vm *vm,
>>>>> +		int vector, handler_fn handler);
>>>>> +
>>>>> +#define SPSR_D          (1 << 9)
>>>>> +#define SPSR_SS         (1 << 21)
>>>>> +
>>>>> +#define write_sysreg(reg, val)						  \
>>>>> +({									  \
>>>>> +	u64 __val = (u64)(val);						  \
>>>>> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
>>>>> +})
>>>>> +
>>>>> +#define read_sysreg(reg)						  \
>>>>> +({	u64 val;							  \
>>>>> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
>>>>> +	val;								  \
>>>>> +})
>>>>> +
>>>>>  #endif /* SELFTEST_KVM_PROCESSOR_H */
>>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>>>> new file mode 100644
>>>>> index 000000000000..8a560021892b
>>>>> --- /dev/null
>>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>>>> @@ -0,0 +1,130 @@
>>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>>> +.macro save_registers, vector
>>>>> +	add	sp, sp, #-16 * 17
>>>>> +
>>>>> +	stp	x0, x1, [sp, #16 * 0]
>>>>> +	stp	x2, x3, [sp, #16 * 1]
>>>>> +	stp	x4, x5, [sp, #16 * 2]
>>>>> +	stp	x6, x7, [sp, #16 * 3]
>>>>> +	stp	x8, x9, [sp, #16 * 4]
>>>>> +	stp	x10, x11, [sp, #16 * 5]
>>>>> +	stp	x12, x13, [sp, #16 * 6]
>>>>> +	stp	x14, x15, [sp, #16 * 7]
>>>>> +	stp	x16, x17, [sp, #16 * 8]
>>>>> +	stp	x18, x19, [sp, #16 * 9]
>>>>> +	stp	x20, x21, [sp, #16 * 10]
>>>>> +	stp	x22, x23, [sp, #16 * 11]
>>>>> +	stp	x24, x25, [sp, #16 * 12]
>>>>> +	stp	x26, x27, [sp, #16 * 13]
>>>>> +	stp	x28, x29, [sp, #16 * 14]
>>>>> +
>>>>> +	.if \vector >= 8
>>>>> +	mrs	x1, sp_el0
>>>>
>>>> I'm still a bit perplexed by this. SP_EL0 is never changed, since you
>>>> always run in handler mode. Therefore, saving/restoring it is only
>>>> overhead. If an exception handler wants to introspect it, it is
>>>> already available in the relevant system register.
>>>>
>>>> Or did you have something else in mind for it?
>>>>
>>>
>>> Not really. The reason for saving sp_el0 in there was just for
>>> consistency, so that handlers for both el0 and el1 exceptions could get
>>> the sp at regs->sp.
>>>
>>> Restoring sp_el0 might be too much. So, what do you think of this v3: we
>>> keep the saving of sp_el0 into regs->sp (to keep things the same between
>>> el0 and el1) and delete the restoring of sp_el0?
>>>
>>> Thanks,
>>> Ricardo
>>>
>>>>> +	.else
>>>>> +	/*
>>>>> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
>>>>> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
>>>>> +	 * return from the exception so handlers can not update it.
>>>>> +	 */
>>>>> +	mov	x1, sp
>>>>> +	.endif
>>>>> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
>>>>> +
>>>>> +	mrs	x1, elr_el1
>>>>> +	mrs	x2, spsr_el1
>>>>> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
>>>>> +.endm
>>>>> +
>>>>> +.macro restore_registers, vector
>>>>> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
>>>>> +	msr	elr_el1, x1
>>>>> +	msr	spsr_el1, x2
>>>>> +
>>>>> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
>>>>> +	.if \vector >= 8
>>>>> +	msr	sp_el0, x1
>>>>> +	.endif
>>>>> +
>>>>> +	ldp	x28, x29, [sp, #16 * 14]
>>>>> +	ldp	x26, x27, [sp, #16 * 13]
>>>>> +	ldp	x24, x25, [sp, #16 * 12]
>>>>> +	ldp	x22, x23, [sp, #16 * 11]
>>>>> +	ldp	x20, x21, [sp, #16 * 10]
>>>>> +	ldp	x18, x19, [sp, #16 * 9]
>>>>> +	ldp	x16, x17, [sp, #16 * 8]
>>>>> +	ldp	x14, x15, [sp, #16 * 7]
>>>>> +	ldp	x12, x13, [sp, #16 * 6]
>>>>> +	ldp	x10, x11, [sp, #16 * 5]
>>>>> +	ldp	x8, x9, [sp, #16 * 4]
>>>>> +	ldp	x6, x7, [sp, #16 * 3]
>>>>> +	ldp	x4, x5, [sp, #16 * 2]
>>>>> +	ldp	x2, x3, [sp, #16 * 1]
>>>>> +	ldp	x0, x1, [sp, #16 * 0]
>>>>> +
>>>>> +	add	sp, sp, #16 * 17
>>>>> +
>>>>> +	eret
>>>>> +.endm
>>>>> +
>>>>> +.pushsection ".entry.text", "ax"
>>>>> +.balign 0x800
>>>>> +.global vectors
>>>>> +vectors:
>>>>> +.popsection
>>>>> +
>>>>> +.set	vector, 0
>>>>> +
>>>>> +/*
>>>>> + * Build an exception handler for vector and append a jump to it into
>>>>> + * vectors (while making sure that it's 0x80 aligned).
>>>>> + */
>>>>> +.macro HANDLER, label
>>>>> +handler_\()\label:
>>>>> +	save_registers vector
>>>>> +	mov	x0, sp
>>>>> +	mov	x1, #vector
>>>>> +	bl	route_exception
>>>>> +	restore_registers vector
>>>>> +
>>>>> +.pushsection ".entry.text", "ax"
>>>>> +.balign 0x80
>>>>> +	b	handler_\()\label
>>>>> +.popsection
>>>>> +
>>>>> +.set	vector, vector + 1
>>>>> +.endm
>>>>> +
>>>>> +.macro HANDLER_INVALID
>>>>> +.pushsection ".entry.text", "ax"
>>>>> +.balign 0x80
>>>>> +/* This will abort so no need to save and restore registers. */
>>>>> +	mov	x0, #vector
>>>>> +	b	kvm_exit_unexpected_vector
>>>>> +.popsection
>>>>> +
>>>>> +.set	vector, vector + 1
>>>>> +.endm
>>>>> +
>>>>> +/*
>>>>> + * Caution: be sure to not add anything between the declaration of vectors
>>>>> + * above and these macro calls that will build the vectors table below it.
>>>>> + */
>>>>> +	HANDLER_INVALID                         // Synchronous EL1t
>>>>> +	HANDLER_INVALID                         // IRQ EL1t
>>>>> +	HANDLER_INVALID                         // FIQ EL1t
>>>>> +	HANDLER_INVALID                         // Error EL1t
>>>>> +
>>>>> +	HANDLER	el1h_sync                       // Synchronous EL1h
>>>>> +	HANDLER	el1h_irq                        // IRQ EL1h
>>>>> +	HANDLER el1h_fiq                        // FIQ EL1h
>>>>> +	HANDLER	el1h_error                      // Error EL1h
>>>>> +
>>>>> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
>>>>> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
>>>>> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
>>>>> +	HANDLER	el0_error_64                    // Error 64-bit EL0
>>>>> +
>>>>> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
>>>>> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
>>>>> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
>>>>> +	HANDLER	el0_error_32                    // Error 32-bit EL0
>>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>>>> index cee92d477dc0..25be71ec88be 100644
>>>>> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>>>> @@ -6,6 +6,7 @@
>>>>>   */
>>>>>  
>>>>>  #include <linux/compiler.h>
>>>>> +#include <assert.h>
>>>>>  
>>>>>  #include "kvm_util.h"
>>>>>  #include "../kvm_util_internal.h"
>>>>> @@ -14,6 +15,8 @@
>>>>>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
>>>>>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
>>>>>  
>>>>> +vm_vaddr_t exception_handlers;
>>>>> +
>>>>>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
>>>>>  {
>>>>>  	return (v + vm->page_size) & ~(vm->page_size - 1);
>>>>> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
>>>>>  	va_end(ap);
>>>>>  }
>>>>>  
>>>>> +void kvm_exit_unexpected_vector(int vector)
>>>>> +{
>>>>> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
>>>>> +}
>>>>> +
>>>>> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
>>>>> +{
>>>>> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
>>>>> +}
>>>>> +
>>>>>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
>>>>>  {
>>>>> +	struct ucall uc;
>>>>> +
>>>>> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
>>>>> +		return;
>>>>> +
>>>>> +	if (uc.args[2]) /* valid_ec */ {
>>>>> +		assert(VECTOR_IS_SYNC(uc.args[0]));
>>>>> +		TEST_ASSERT(false,
>>>>> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
>>>>> +			uc.args[0], uc.args[1]);
>>>>> +	} else {
>>>>> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
>>>>> +		TEST_ASSERT(false,
>>>>> +			"Unexpected exception (vector:0x%lx)",
>>>>> +			uc.args[0]);
>>>>> +	}
>>>>> +}
>>>>> +
>>>>> +/*
>>>>> + * This exception handling code was heavily inspired on kvm-unit-tests. There
>>>>> + * is a set of default vector handlers stored in vector_handlers. These default
>>>>> + * vector handlers call user-installed handlers stored in exception_handlers.
>>>>> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
>>>>> + * (vector, ec=0).
>>>>> + */
>>>>> +
>>>>> +typedef void(*vector_fn)(struct ex_regs *, int vector);
>>>>> +
>>>>> +struct handlers {
>>>>> +	vector_fn vector_handlers[VECTOR_NUM];
>>>>> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
>>>>> +};
>>>>> +
>>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
>>>>> +{
>>>>> +	extern char vectors;
>>>>> +
>>>>> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
>>>>> +}
>>>>> +
>>>>> +void default_sync_handler(struct ex_regs *regs, int vector)
>>>>> +{
>>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>>>> +	uint64_t esr = read_sysreg(esr_el1);
>>>>> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
>>>>> +
>>>>> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
>>>>> +
>>>>> +	if (handlers && handlers->exception_handlers[vector][ec])
>>>>> +		handlers->exception_handlers[vector][ec](regs);
>>>>> +	else
>>>>> +		kvm_exit_unexpected_exception(vector, ec);
>>>>> +}
>>>>> +
>>>>> +void default_irq_handler(struct ex_regs *regs, int vector)
>>>>> +{
>>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>>>> +
>>>>> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
>>>>> +
>>>>> +	if (handlers && handlers->exception_handlers[vector][0])
>>>>> +		handlers->exception_handlers[vector][0](regs);
>>>>> +	else
>>>>> +		kvm_exit_unexpected_vector(vector);
>>>>> +}
>>>>> +
>>>>> +void route_exception(struct ex_regs *regs, int vector)
>>>>> +{
>>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>>>> +
>>>>> +	if (handlers && handlers->vector_handlers[vector])
>>>>> +		handlers->vector_handlers[vector](regs, vector);
>>>>> +	else
>>>>> +		kvm_exit_unexpected_vector(vector);
>>>>> +}
>>>>> +
>>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm)
>>>>> +{
>>>>> +	struct handlers *handlers;
>>>>> +
>>>>> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
>>>>> +			vm->page_size, 0, 0);
>>>>> +
>>>>> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>>>> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
>>>>> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
>>>>> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
>>>>> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
>>>>
>>>> How about FIQ, Error? Although they are unlikely, they are valid
>>>> exceptions.
>>>>
>>>>> +
>>>>> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
>>>>> +}
>>>>> +
>>>>> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
>>>>> +			 void (*handler)(struct ex_regs *))
>>>>> +{
>>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>>>> +
>>>>> +	assert(VECTOR_IS_SYNC(vector));
>>>>> +	assert(vector < VECTOR_NUM);
>>>>> +	assert(ec < ESR_EC_NUM);
>>>>> +	handlers->exception_handlers[vector][ec] = handler;
>>>>> +}
>>>>> +
>>>>> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
>>>>> +			 void (*handler)(struct ex_regs *))
>>>>> +{
>>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>>>> +
>>>>> +	assert(!VECTOR_IS_SYNC(vector));
>>>>> +	assert(vector < VECTOR_NUM);
>>>>> +	handlers->exception_handlers[vector][0] = handler;
>>>>>  }
>>>>
>>>> Thanks,
>>>>
>>>> 	M.
>>>>
>>>> -- 
>>>> Without deviation from the norm, progress is not possible.
>>>
>>
> 

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Fri, May 07, 2021 at 04:08:07PM +0200, Auger Eric wrote:
> Hi Ricardo,
> 
> On 5/6/21 9:14 PM, Ricardo Koller wrote:
> > On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
> >> Hi Ricardo,
> >>
> > 
> > Hi Eric,
> > 
> > Thank you very much for the test.
> > 
> >> On 5/3/21 9:12 PM, Ricardo Koller wrote:
> >>> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
> >>>> On Sat, 01 May 2021 00:24:06 +0100,
> >>>> Ricardo Koller <ricarkol@google.com> wrote:
> >>>>>
> >>>>> Add the infrastructure needed to enable exception handling in aarch64
> >>>>> selftests. The exception handling defaults to an unhandled-exception
> >>>>> handler which aborts the test, just like x86. These handlers can be
> >>>>> overridden by calling vm_install_vector_handler(vector) or
> >>>>> vm_install_exception_handler(vector, ec). The unhandled exception
> >>>>> reporting from the guest is done using the ucall type introduced in a
> >>>>> previous commit, UCALL_UNHANDLED.
> >>>>>
> >>>>> The exception handling code is heavily inspired on kvm-unit-tests.
> >>
> >> running the test on 5.12 I get
> >>
> >> ==== Test Assertion Failure ====
> >>   aarch64/debug-exceptions.c:232: false
> >>   pid=6477 tid=6477 errno=4 - Interrupted system call
> >>      1	0x000000000040147b: main at debug-exceptions.c:230
> >>      2	0x000003ff8aa60de3: ?? ??:0
> >>      3	0x0000000000401517: _start at :?
> >>   Failed guest assert: hw_bp_addr == PC(hw_bp) at
> >> aarch64/debug-exceptions.c:105
> >> 	values: 0, 0x401794
> >>
> >>
> >> I guess it is not an expected result. Any known bug waiting on the list?
> >>
> > 
> > Not expected. That should work, or at least abort early because there is
> > no HW breakpoints support.
> > 
> > I'm trying to reproduce the failure; can you help me with some
> > questions, please?
> sure, please find the answers below.
> > 
> > - does your setup have support for hardware breakpoints? Can you try a
> >   'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
> >   found ...'. If there is no such line it's very likely that the check
> >   for "debug_ver >= 6" is not enough and the test should check for
> >   "num_breakpoints > 0".
> [   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
> > - does it fail consistently (every single attempt)?
> yes it does.
> 
> I will try to find some time to investigate too

Thank you very much Eric.

> 
> Thanks
> 
> Eric
> > 
> > Thanks!
> > Ricardo
> > 
> >>
> >> Thanks
> >>
> >> Eric
> > 
> > 
> >>>>>
> >>>>> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> >>>>> ---
> >>>>>  tools/testing/selftests/kvm/Makefile          |   2 +-
> >>>>>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
> >>>>>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
> >>>>>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
> >>>>>  4 files changed, 333 insertions(+), 1 deletion(-)
> >>>>>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>>>>
> >>>>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> >>>>> index 4e548d7ab0ab..618c5903f478 100644
> >>>>> --- a/tools/testing/selftests/kvm/Makefile
> >>>>> +++ b/tools/testing/selftests/kvm/Makefile
> >>>>> @@ -35,7 +35,7 @@ endif
> >>>>>  
> >>>>>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
> >>>>>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
> >>>>> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
> >>>>> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
> >>>>>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
> >>>>>  
> >>>>>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
> >>>>> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>>>> index b7fa0c8551db..40aae31b4afc 100644
> >>>>> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>>>> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>>>> @@ -8,6 +8,7 @@
> >>>>>  #define SELFTEST_KVM_PROCESSOR_H
> >>>>>  
> >>>>>  #include "kvm_util.h"
> >>>>> +#include <linux/stringify.h>
> >>>>>  
> >>>>>  
> >>>>>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> >>>>> @@ -18,6 +19,7 @@
> >>>>>  #define MAIR_EL1	3, 0, 10, 2, 0
> >>>>>  #define TTBR0_EL1	3, 0,  2, 0, 0
> >>>>>  #define SCTLR_EL1	3, 0,  1, 0, 0
> >>>>> +#define VBAR_EL1	3, 0, 12, 0, 0
> >>>>>  
> >>>>>  /*
> >>>>>   * Default MAIR
> >>>>> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
> >>>>>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
> >>>>>  			      struct kvm_vcpu_init *init, void *guest_code);
> >>>>>  
> >>>>> +struct ex_regs {
> >>>>> +	u64 regs[31];
> >>>>> +	u64 sp;
> >>>>> +	u64 pc;
> >>>>> +	u64 pstate;
> >>>>> +};
> >>>>> +
> >>>>> +#define VECTOR_NUM	16
> >>>>> +
> >>>>> +enum {
> >>>>> +	VECTOR_SYNC_CURRENT_SP0,
> >>>>> +	VECTOR_IRQ_CURRENT_SP0,
> >>>>> +	VECTOR_FIQ_CURRENT_SP0,
> >>>>> +	VECTOR_ERROR_CURRENT_SP0,
> >>>>> +
> >>>>> +	VECTOR_SYNC_CURRENT,
> >>>>> +	VECTOR_IRQ_CURRENT,
> >>>>> +	VECTOR_FIQ_CURRENT,
> >>>>> +	VECTOR_ERROR_CURRENT,
> >>>>> +
> >>>>> +	VECTOR_SYNC_LOWER_64,
> >>>>> +	VECTOR_IRQ_LOWER_64,
> >>>>> +	VECTOR_FIQ_LOWER_64,
> >>>>> +	VECTOR_ERROR_LOWER_64,
> >>>>> +
> >>>>> +	VECTOR_SYNC_LOWER_32,
> >>>>> +	VECTOR_IRQ_LOWER_32,
> >>>>> +	VECTOR_FIQ_LOWER_32,
> >>>>> +	VECTOR_ERROR_LOWER_32,
> >>>>> +};
> >>>>> +
> >>>>> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
> >>>>> +			   (v) == VECTOR_SYNC_CURRENT     || \
> >>>>> +			   (v) == VECTOR_SYNC_LOWER_64    || \
> >>>>> +			   (v) == VECTOR_SYNC_LOWER_32)
> >>>>> +
> >>>>> +/* Some common EC (Exception classes) */
> >>>>> +#define ESR_EC_ILLEGAL_INS	0x0e
> >>>>> +#define ESR_EC_SVC64		0x15
> >>>>> +#define ESR_EC_IABORT_CURRENT	0x21
> >>>>> +#define ESR_EC_DABORT_CURRENT	0x25
> >>>>> +#define ESR_EC_SERROR		0x2f
> >>>>> +#define ESR_EC_HW_BP_CURRENT	0x31
> >>>>> +#define ESR_EC_SSTEP_CURRENT	0x33
> >>>>> +#define ESR_EC_WP_CURRENT	0x35
> >>>>> +#define ESR_EC_BRK_INS		0x3C
> >>>>> +
> >>>>> +#define ESR_EC_NUM		64
> >>>>> +
> >>>>> +#define ESR_EC_SHIFT		26
> >>>>> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
> >>>>> +
> >>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm);
> >>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
> >>>>> +
> >>>>> +typedef void(*handler_fn)(struct ex_regs *);
> >>>>> +void vm_install_exception_handler(struct kvm_vm *vm,
> >>>>> +		int vector, int ec, handler_fn handler);
> >>>>> +void vm_install_vector_handler(struct kvm_vm *vm,
> >>>>> +		int vector, handler_fn handler);
> >>>>> +
> >>>>> +#define SPSR_D          (1 << 9)
> >>>>> +#define SPSR_SS         (1 << 21)
> >>>>> +
> >>>>> +#define write_sysreg(reg, val)						  \
> >>>>> +({									  \
> >>>>> +	u64 __val = (u64)(val);						  \
> >>>>> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
> >>>>> +})
> >>>>> +
> >>>>> +#define read_sysreg(reg)						  \
> >>>>> +({	u64 val;							  \
> >>>>> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
> >>>>> +	val;								  \
> >>>>> +})
> >>>>> +
> >>>>>  #endif /* SELFTEST_KVM_PROCESSOR_H */
> >>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>>>> new file mode 100644
> >>>>> index 000000000000..8a560021892b
> >>>>> --- /dev/null
> >>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>>>> @@ -0,0 +1,130 @@
> >>>>> +/* SPDX-License-Identifier: GPL-2.0 */
> >>>>> +.macro save_registers, vector
> >>>>> +	add	sp, sp, #-16 * 17
> >>>>> +
> >>>>> +	stp	x0, x1, [sp, #16 * 0]
> >>>>> +	stp	x2, x3, [sp, #16 * 1]
> >>>>> +	stp	x4, x5, [sp, #16 * 2]
> >>>>> +	stp	x6, x7, [sp, #16 * 3]
> >>>>> +	stp	x8, x9, [sp, #16 * 4]
> >>>>> +	stp	x10, x11, [sp, #16 * 5]
> >>>>> +	stp	x12, x13, [sp, #16 * 6]
> >>>>> +	stp	x14, x15, [sp, #16 * 7]
> >>>>> +	stp	x16, x17, [sp, #16 * 8]
> >>>>> +	stp	x18, x19, [sp, #16 * 9]
> >>>>> +	stp	x20, x21, [sp, #16 * 10]
> >>>>> +	stp	x22, x23, [sp, #16 * 11]
> >>>>> +	stp	x24, x25, [sp, #16 * 12]
> >>>>> +	stp	x26, x27, [sp, #16 * 13]
> >>>>> +	stp	x28, x29, [sp, #16 * 14]
> >>>>> +
> >>>>> +	.if \vector >= 8
> >>>>> +	mrs	x1, sp_el0
> >>>>
> >>>> I'm still a bit perplexed by this. SP_EL0 is never changed, since you
> >>>> always run in handler mode. Therefore, saving/restoring it is only
> >>>> overhead. If an exception handler wants to introspect it, it is
> >>>> already available in the relevant system register.
> >>>>
> >>>> Or did you have something else in mind for it?
> >>>>
> >>>
> >>> Not really. The reason for saving sp_el0 in there was just for
> >>> consistency, so that handlers for both el0 and el1 exceptions could get
> >>> the sp at regs->sp.
> >>>
> >>> Restoring sp_el0 might be too much. So, what do you think of this v3: we
> >>> keep the saving of sp_el0 into regs->sp (to keep things the same between
> >>> el0 and el1) and delete the restoring of sp_el0?
> >>>
> >>> Thanks,
> >>> Ricardo
> >>>
> >>>>> +	.else
> >>>>> +	/*
> >>>>> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
> >>>>> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
> >>>>> +	 * return from the exception so handlers can not update it.
> >>>>> +	 */
> >>>>> +	mov	x1, sp
> >>>>> +	.endif
> >>>>> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
> >>>>> +
> >>>>> +	mrs	x1, elr_el1
> >>>>> +	mrs	x2, spsr_el1
> >>>>> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> >>>>> +.endm
> >>>>> +
> >>>>> +.macro restore_registers, vector
> >>>>> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> >>>>> +	msr	elr_el1, x1
> >>>>> +	msr	spsr_el1, x2
> >>>>> +
> >>>>> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
> >>>>> +	.if \vector >= 8
> >>>>> +	msr	sp_el0, x1
> >>>>> +	.endif
> >>>>> +
> >>>>> +	ldp	x28, x29, [sp, #16 * 14]
> >>>>> +	ldp	x26, x27, [sp, #16 * 13]
> >>>>> +	ldp	x24, x25, [sp, #16 * 12]
> >>>>> +	ldp	x22, x23, [sp, #16 * 11]
> >>>>> +	ldp	x20, x21, [sp, #16 * 10]
> >>>>> +	ldp	x18, x19, [sp, #16 * 9]
> >>>>> +	ldp	x16, x17, [sp, #16 * 8]
> >>>>> +	ldp	x14, x15, [sp, #16 * 7]
> >>>>> +	ldp	x12, x13, [sp, #16 * 6]
> >>>>> +	ldp	x10, x11, [sp, #16 * 5]
> >>>>> +	ldp	x8, x9, [sp, #16 * 4]
> >>>>> +	ldp	x6, x7, [sp, #16 * 3]
> >>>>> +	ldp	x4, x5, [sp, #16 * 2]
> >>>>> +	ldp	x2, x3, [sp, #16 * 1]
> >>>>> +	ldp	x0, x1, [sp, #16 * 0]
> >>>>> +
> >>>>> +	add	sp, sp, #16 * 17
> >>>>> +
> >>>>> +	eret
> >>>>> +.endm
> >>>>> +
> >>>>> +.pushsection ".entry.text", "ax"
> >>>>> +.balign 0x800
> >>>>> +.global vectors
> >>>>> +vectors:
> >>>>> +.popsection
> >>>>> +
> >>>>> +.set	vector, 0
> >>>>> +
> >>>>> +/*
> >>>>> + * Build an exception handler for vector and append a jump to it into
> >>>>> + * vectors (while making sure that it's 0x80 aligned).
> >>>>> + */
> >>>>> +.macro HANDLER, label
> >>>>> +handler_\()\label:
> >>>>> +	save_registers vector
> >>>>> +	mov	x0, sp
> >>>>> +	mov	x1, #vector
> >>>>> +	bl	route_exception
> >>>>> +	restore_registers vector
> >>>>> +
> >>>>> +.pushsection ".entry.text", "ax"
> >>>>> +.balign 0x80
> >>>>> +	b	handler_\()\label
> >>>>> +.popsection
> >>>>> +
> >>>>> +.set	vector, vector + 1
> >>>>> +.endm
> >>>>> +
> >>>>> +.macro HANDLER_INVALID
> >>>>> +.pushsection ".entry.text", "ax"
> >>>>> +.balign 0x80
> >>>>> +/* This will abort so no need to save and restore registers. */
> >>>>> +	mov	x0, #vector
> >>>>> +	b	kvm_exit_unexpected_vector
> >>>>> +.popsection
> >>>>> +
> >>>>> +.set	vector, vector + 1
> >>>>> +.endm
> >>>>> +
> >>>>> +/*
> >>>>> + * Caution: be sure to not add anything between the declaration of vectors
> >>>>> + * above and these macro calls that will build the vectors table below it.
> >>>>> + */
> >>>>> +	HANDLER_INVALID                         // Synchronous EL1t
> >>>>> +	HANDLER_INVALID                         // IRQ EL1t
> >>>>> +	HANDLER_INVALID                         // FIQ EL1t
> >>>>> +	HANDLER_INVALID                         // Error EL1t
> >>>>> +
> >>>>> +	HANDLER	el1h_sync                       // Synchronous EL1h
> >>>>> +	HANDLER	el1h_irq                        // IRQ EL1h
> >>>>> +	HANDLER el1h_fiq                        // FIQ EL1h
> >>>>> +	HANDLER	el1h_error                      // Error EL1h
> >>>>> +
> >>>>> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
> >>>>> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
> >>>>> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
> >>>>> +	HANDLER	el0_error_64                    // Error 64-bit EL0
> >>>>> +
> >>>>> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
> >>>>> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
> >>>>> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
> >>>>> +	HANDLER	el0_error_32                    // Error 32-bit EL0
> >>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>>>> index cee92d477dc0..25be71ec88be 100644
> >>>>> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>>>> @@ -6,6 +6,7 @@
> >>>>>   */
> >>>>>  
> >>>>>  #include <linux/compiler.h>
> >>>>> +#include <assert.h>
> >>>>>  
> >>>>>  #include "kvm_util.h"
> >>>>>  #include "../kvm_util_internal.h"
> >>>>> @@ -14,6 +15,8 @@
> >>>>>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
> >>>>>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
> >>>>>  
> >>>>> +vm_vaddr_t exception_handlers;
> >>>>> +
> >>>>>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
> >>>>>  {
> >>>>>  	return (v + vm->page_size) & ~(vm->page_size - 1);
> >>>>> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
> >>>>>  	va_end(ap);
> >>>>>  }
> >>>>>  
> >>>>> +void kvm_exit_unexpected_vector(int vector)
> >>>>> +{
> >>>>> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
> >>>>> +}
> >>>>> +
> >>>>> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
> >>>>> +{
> >>>>> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
> >>>>> +}
> >>>>> +
> >>>>>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
> >>>>>  {
> >>>>> +	struct ucall uc;
> >>>>> +
> >>>>> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
> >>>>> +		return;
> >>>>> +
> >>>>> +	if (uc.args[2]) /* valid_ec */ {
> >>>>> +		assert(VECTOR_IS_SYNC(uc.args[0]));
> >>>>> +		TEST_ASSERT(false,
> >>>>> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
> >>>>> +			uc.args[0], uc.args[1]);
> >>>>> +	} else {
> >>>>> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
> >>>>> +		TEST_ASSERT(false,
> >>>>> +			"Unexpected exception (vector:0x%lx)",
> >>>>> +			uc.args[0]);
> >>>>> +	}
> >>>>> +}
> >>>>> +
> >>>>> +/*
> >>>>> + * This exception handling code was heavily inspired on kvm-unit-tests. There
> >>>>> + * is a set of default vector handlers stored in vector_handlers. These default
> >>>>> + * vector handlers call user-installed handlers stored in exception_handlers.
> >>>>> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
> >>>>> + * (vector, ec=0).
> >>>>> + */
> >>>>> +
> >>>>> +typedef void(*vector_fn)(struct ex_regs *, int vector);
> >>>>> +
> >>>>> +struct handlers {
> >>>>> +	vector_fn vector_handlers[VECTOR_NUM];
> >>>>> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
> >>>>> +};
> >>>>> +
> >>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
> >>>>> +{
> >>>>> +	extern char vectors;
> >>>>> +
> >>>>> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
> >>>>> +}
> >>>>> +
> >>>>> +void default_sync_handler(struct ex_regs *regs, int vector)
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>>>> +	uint64_t esr = read_sysreg(esr_el1);
> >>>>> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
> >>>>> +
> >>>>> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
> >>>>> +
> >>>>> +	if (handlers && handlers->exception_handlers[vector][ec])
> >>>>> +		handlers->exception_handlers[vector][ec](regs);
> >>>>> +	else
> >>>>> +		kvm_exit_unexpected_exception(vector, ec);
> >>>>> +}
> >>>>> +
> >>>>> +void default_irq_handler(struct ex_regs *regs, int vector)
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>>>> +
> >>>>> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
> >>>>> +
> >>>>> +	if (handlers && handlers->exception_handlers[vector][0])
> >>>>> +		handlers->exception_handlers[vector][0](regs);
> >>>>> +	else
> >>>>> +		kvm_exit_unexpected_vector(vector);
> >>>>> +}
> >>>>> +
> >>>>> +void route_exception(struct ex_regs *regs, int vector)
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>>>> +
> >>>>> +	if (handlers && handlers->vector_handlers[vector])
> >>>>> +		handlers->vector_handlers[vector](regs, vector);
> >>>>> +	else
> >>>>> +		kvm_exit_unexpected_vector(vector);
> >>>>> +}
> >>>>> +
> >>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm)
> >>>>> +{
> >>>>> +	struct handlers *handlers;
> >>>>> +
> >>>>> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
> >>>>> +			vm->page_size, 0, 0);
> >>>>> +
> >>>>> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>>>> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
> >>>>> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
> >>>>> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
> >>>>> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
> >>>>
> >>>> How about FIQ, Error? Although they are unlikely, they are valid
> >>>> exceptions.
> >>>>
> >>>>> +
> >>>>> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
> >>>>> +}
> >>>>> +
> >>>>> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
> >>>>> +			 void (*handler)(struct ex_regs *))
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>>>> +
> >>>>> +	assert(VECTOR_IS_SYNC(vector));
> >>>>> +	assert(vector < VECTOR_NUM);
> >>>>> +	assert(ec < ESR_EC_NUM);
> >>>>> +	handlers->exception_handlers[vector][ec] = handler;
> >>>>> +}
> >>>>> +
> >>>>> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
> >>>>> +			 void (*handler)(struct ex_regs *))
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>>>> +
> >>>>> +	assert(!VECTOR_IS_SYNC(vector));
> >>>>> +	assert(vector < VECTOR_NUM);
> >>>>> +	handlers->exception_handlers[vector][0] = handler;
> >>>>>  }
> >>>>
> >>>> Thanks,
> >>>>
> >>>> 	M.
> >>>>
> >>>> -- 
> >>>> Without deviation from the norm, progress is not possible.
> >>>
> >>
> > 
> 

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Fri, May 07, 2021 at 04:08:07PM +0200, Auger Eric wrote:
> Hi Ricardo,
> 
> On 5/6/21 9:14 PM, Ricardo Koller wrote:
> > On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
> >> Hi Ricardo,
> >>
> > 
> > Hi Eric,
> > 
> > Thank you very much for the test.
> > 
> >> On 5/3/21 9:12 PM, Ricardo Koller wrote:
> >>> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
> >>>> On Sat, 01 May 2021 00:24:06 +0100,
> >>>> Ricardo Koller <ricarkol@google.com> wrote:
> >>>>>
> >>>>> Add the infrastructure needed to enable exception handling in aarch64
> >>>>> selftests. The exception handling defaults to an unhandled-exception
> >>>>> handler which aborts the test, just like x86. These handlers can be
> >>>>> overridden by calling vm_install_vector_handler(vector) or
> >>>>> vm_install_exception_handler(vector, ec). The unhandled exception
> >>>>> reporting from the guest is done using the ucall type introduced in a
> >>>>> previous commit, UCALL_UNHANDLED.
> >>>>>
> >>>>> The exception handling code is heavily inspired on kvm-unit-tests.
> >>
> >> running the test on 5.12 I get
> >>

Hi Eric,

I'm able to reproduce the failure you are seeing on 5.6, specifically
with kernels older than this commit:

  4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last vcpu_load on VHE

but not yet on v5.12. Could you share the commit of the kernel you are
testing, please?

Thanks!
Ricardo

> >> ==== Test Assertion Failure ====
> >>   aarch64/debug-exceptions.c:232: false
> >>   pid=6477 tid=6477 errno=4 - Interrupted system call
> >>      1	0x000000000040147b: main at debug-exceptions.c:230
> >>      2	0x000003ff8aa60de3: ?? ??:0
> >>      3	0x0000000000401517: _start at :?
> >>   Failed guest assert: hw_bp_addr == PC(hw_bp) at
> >> aarch64/debug-exceptions.c:105
> >> 	values: 0, 0x401794
> >>
> >>
> >> I guess it is not an expected result. Any known bug waiting on the list?
> >>
> > 
> > Not expected. That should work, or at least abort early because there is
> > no HW breakpoints support.
> > 
> > I'm trying to reproduce the failure; can you help me with some
> > questions, please?
> sure, please find the answers below.
> > 
> > - does your setup have support for hardware breakpoints? Can you try a
> >   'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
> >   found ...'. If there is no such line it's very likely that the check
> >   for "debug_ver >= 6" is not enough and the test should check for
> >   "num_breakpoints > 0".
> [   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
> > - does it fail consistently (every single attempt)?
> yes it does.
> 
> I will try to find some time to investigate too
> 
> Thanks
> 
> Eric
> > 
> > Thanks!
> > Ricardo
> > 
> >>
> >> Thanks
> >>
> >> Eric
> > 
> > 
> >>>>>
> >>>>> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> >>>>> ---
> >>>>>  tools/testing/selftests/kvm/Makefile          |   2 +-
> >>>>>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
> >>>>>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
> >>>>>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
> >>>>>  4 files changed, 333 insertions(+), 1 deletion(-)
> >>>>>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>>>>
> >>>>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> >>>>> index 4e548d7ab0ab..618c5903f478 100644
> >>>>> --- a/tools/testing/selftests/kvm/Makefile
> >>>>> +++ b/tools/testing/selftests/kvm/Makefile
> >>>>> @@ -35,7 +35,7 @@ endif
> >>>>>  
> >>>>>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
> >>>>>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
> >>>>> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
> >>>>> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
> >>>>>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
> >>>>>  
> >>>>>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
> >>>>> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>>>> index b7fa0c8551db..40aae31b4afc 100644
> >>>>> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>>>> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
> >>>>> @@ -8,6 +8,7 @@
> >>>>>  #define SELFTEST_KVM_PROCESSOR_H
> >>>>>  
> >>>>>  #include "kvm_util.h"
> >>>>> +#include <linux/stringify.h>
> >>>>>  
> >>>>>  
> >>>>>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> >>>>> @@ -18,6 +19,7 @@
> >>>>>  #define MAIR_EL1	3, 0, 10, 2, 0
> >>>>>  #define TTBR0_EL1	3, 0,  2, 0, 0
> >>>>>  #define SCTLR_EL1	3, 0,  1, 0, 0
> >>>>> +#define VBAR_EL1	3, 0, 12, 0, 0
> >>>>>  
> >>>>>  /*
> >>>>>   * Default MAIR
> >>>>> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
> >>>>>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
> >>>>>  			      struct kvm_vcpu_init *init, void *guest_code);
> >>>>>  
> >>>>> +struct ex_regs {
> >>>>> +	u64 regs[31];
> >>>>> +	u64 sp;
> >>>>> +	u64 pc;
> >>>>> +	u64 pstate;
> >>>>> +};
> >>>>> +
> >>>>> +#define VECTOR_NUM	16
> >>>>> +
> >>>>> +enum {
> >>>>> +	VECTOR_SYNC_CURRENT_SP0,
> >>>>> +	VECTOR_IRQ_CURRENT_SP0,
> >>>>> +	VECTOR_FIQ_CURRENT_SP0,
> >>>>> +	VECTOR_ERROR_CURRENT_SP0,
> >>>>> +
> >>>>> +	VECTOR_SYNC_CURRENT,
> >>>>> +	VECTOR_IRQ_CURRENT,
> >>>>> +	VECTOR_FIQ_CURRENT,
> >>>>> +	VECTOR_ERROR_CURRENT,
> >>>>> +
> >>>>> +	VECTOR_SYNC_LOWER_64,
> >>>>> +	VECTOR_IRQ_LOWER_64,
> >>>>> +	VECTOR_FIQ_LOWER_64,
> >>>>> +	VECTOR_ERROR_LOWER_64,
> >>>>> +
> >>>>> +	VECTOR_SYNC_LOWER_32,
> >>>>> +	VECTOR_IRQ_LOWER_32,
> >>>>> +	VECTOR_FIQ_LOWER_32,
> >>>>> +	VECTOR_ERROR_LOWER_32,
> >>>>> +};
> >>>>> +
> >>>>> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
> >>>>> +			   (v) == VECTOR_SYNC_CURRENT     || \
> >>>>> +			   (v) == VECTOR_SYNC_LOWER_64    || \
> >>>>> +			   (v) == VECTOR_SYNC_LOWER_32)
> >>>>> +
> >>>>> +/* Some common EC (Exception classes) */
> >>>>> +#define ESR_EC_ILLEGAL_INS	0x0e
> >>>>> +#define ESR_EC_SVC64		0x15
> >>>>> +#define ESR_EC_IABORT_CURRENT	0x21
> >>>>> +#define ESR_EC_DABORT_CURRENT	0x25
> >>>>> +#define ESR_EC_SERROR		0x2f
> >>>>> +#define ESR_EC_HW_BP_CURRENT	0x31
> >>>>> +#define ESR_EC_SSTEP_CURRENT	0x33
> >>>>> +#define ESR_EC_WP_CURRENT	0x35
> >>>>> +#define ESR_EC_BRK_INS		0x3C
> >>>>> +
> >>>>> +#define ESR_EC_NUM		64
> >>>>> +
> >>>>> +#define ESR_EC_SHIFT		26
> >>>>> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
> >>>>> +
> >>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm);
> >>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
> >>>>> +
> >>>>> +typedef void(*handler_fn)(struct ex_regs *);
> >>>>> +void vm_install_exception_handler(struct kvm_vm *vm,
> >>>>> +		int vector, int ec, handler_fn handler);
> >>>>> +void vm_install_vector_handler(struct kvm_vm *vm,
> >>>>> +		int vector, handler_fn handler);
> >>>>> +
> >>>>> +#define SPSR_D          (1 << 9)
> >>>>> +#define SPSR_SS         (1 << 21)
> >>>>> +
> >>>>> +#define write_sysreg(reg, val)						  \
> >>>>> +({									  \
> >>>>> +	u64 __val = (u64)(val);						  \
> >>>>> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
> >>>>> +})
> >>>>> +
> >>>>> +#define read_sysreg(reg)						  \
> >>>>> +({	u64 val;							  \
> >>>>> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
> >>>>> +	val;								  \
> >>>>> +})
> >>>>> +
> >>>>>  #endif /* SELFTEST_KVM_PROCESSOR_H */
> >>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>>>> new file mode 100644
> >>>>> index 000000000000..8a560021892b
> >>>>> --- /dev/null
> >>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> >>>>> @@ -0,0 +1,130 @@
> >>>>> +/* SPDX-License-Identifier: GPL-2.0 */
> >>>>> +.macro save_registers, vector
> >>>>> +	add	sp, sp, #-16 * 17
> >>>>> +
> >>>>> +	stp	x0, x1, [sp, #16 * 0]
> >>>>> +	stp	x2, x3, [sp, #16 * 1]
> >>>>> +	stp	x4, x5, [sp, #16 * 2]
> >>>>> +	stp	x6, x7, [sp, #16 * 3]
> >>>>> +	stp	x8, x9, [sp, #16 * 4]
> >>>>> +	stp	x10, x11, [sp, #16 * 5]
> >>>>> +	stp	x12, x13, [sp, #16 * 6]
> >>>>> +	stp	x14, x15, [sp, #16 * 7]
> >>>>> +	stp	x16, x17, [sp, #16 * 8]
> >>>>> +	stp	x18, x19, [sp, #16 * 9]
> >>>>> +	stp	x20, x21, [sp, #16 * 10]
> >>>>> +	stp	x22, x23, [sp, #16 * 11]
> >>>>> +	stp	x24, x25, [sp, #16 * 12]
> >>>>> +	stp	x26, x27, [sp, #16 * 13]
> >>>>> +	stp	x28, x29, [sp, #16 * 14]
> >>>>> +
> >>>>> +	.if \vector >= 8
> >>>>> +	mrs	x1, sp_el0
> >>>>
> >>>> I'm still a bit perplexed by this. SP_EL0 is never changed, since you
> >>>> always run in handler mode. Therefore, saving/restoring it is only
> >>>> overhead. If an exception handler wants to introspect it, it is
> >>>> already available in the relevant system register.
> >>>>
> >>>> Or did you have something else in mind for it?
> >>>>
> >>>
> >>> Not really. The reason for saving sp_el0 in there was just for
> >>> consistency, so that handlers for both el0 and el1 exceptions could get
> >>> the sp at regs->sp.
> >>>
> >>> Restoring sp_el0 might be too much. So, what do you think of this v3: we
> >>> keep the saving of sp_el0 into regs->sp (to keep things the same between
> >>> el0 and el1) and delete the restoring of sp_el0?
> >>>
> >>> Thanks,
> >>> Ricardo
> >>>
> >>>>> +	.else
> >>>>> +	/*
> >>>>> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
> >>>>> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
> >>>>> +	 * return from the exception so handlers can not update it.
> >>>>> +	 */
> >>>>> +	mov	x1, sp
> >>>>> +	.endif
> >>>>> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
> >>>>> +
> >>>>> +	mrs	x1, elr_el1
> >>>>> +	mrs	x2, spsr_el1
> >>>>> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> >>>>> +.endm
> >>>>> +
> >>>>> +.macro restore_registers, vector
> >>>>> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> >>>>> +	msr	elr_el1, x1
> >>>>> +	msr	spsr_el1, x2
> >>>>> +
> >>>>> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
> >>>>> +	.if \vector >= 8
> >>>>> +	msr	sp_el0, x1
> >>>>> +	.endif
> >>>>> +
> >>>>> +	ldp	x28, x29, [sp, #16 * 14]
> >>>>> +	ldp	x26, x27, [sp, #16 * 13]
> >>>>> +	ldp	x24, x25, [sp, #16 * 12]
> >>>>> +	ldp	x22, x23, [sp, #16 * 11]
> >>>>> +	ldp	x20, x21, [sp, #16 * 10]
> >>>>> +	ldp	x18, x19, [sp, #16 * 9]
> >>>>> +	ldp	x16, x17, [sp, #16 * 8]
> >>>>> +	ldp	x14, x15, [sp, #16 * 7]
> >>>>> +	ldp	x12, x13, [sp, #16 * 6]
> >>>>> +	ldp	x10, x11, [sp, #16 * 5]
> >>>>> +	ldp	x8, x9, [sp, #16 * 4]
> >>>>> +	ldp	x6, x7, [sp, #16 * 3]
> >>>>> +	ldp	x4, x5, [sp, #16 * 2]
> >>>>> +	ldp	x2, x3, [sp, #16 * 1]
> >>>>> +	ldp	x0, x1, [sp, #16 * 0]
> >>>>> +
> >>>>> +	add	sp, sp, #16 * 17
> >>>>> +
> >>>>> +	eret
> >>>>> +.endm
> >>>>> +
> >>>>> +.pushsection ".entry.text", "ax"
> >>>>> +.balign 0x800
> >>>>> +.global vectors
> >>>>> +vectors:
> >>>>> +.popsection
> >>>>> +
> >>>>> +.set	vector, 0
> >>>>> +
> >>>>> +/*
> >>>>> + * Build an exception handler for vector and append a jump to it into
> >>>>> + * vectors (while making sure that it's 0x80 aligned).
> >>>>> + */
> >>>>> +.macro HANDLER, label
> >>>>> +handler_\()\label:
> >>>>> +	save_registers vector
> >>>>> +	mov	x0, sp
> >>>>> +	mov	x1, #vector
> >>>>> +	bl	route_exception
> >>>>> +	restore_registers vector
> >>>>> +
> >>>>> +.pushsection ".entry.text", "ax"
> >>>>> +.balign 0x80
> >>>>> +	b	handler_\()\label
> >>>>> +.popsection
> >>>>> +
> >>>>> +.set	vector, vector + 1
> >>>>> +.endm
> >>>>> +
> >>>>> +.macro HANDLER_INVALID
> >>>>> +.pushsection ".entry.text", "ax"
> >>>>> +.balign 0x80
> >>>>> +/* This will abort so no need to save and restore registers. */
> >>>>> +	mov	x0, #vector
> >>>>> +	b	kvm_exit_unexpected_vector
> >>>>> +.popsection
> >>>>> +
> >>>>> +.set	vector, vector + 1
> >>>>> +.endm
> >>>>> +
> >>>>> +/*
> >>>>> + * Caution: be sure to not add anything between the declaration of vectors
> >>>>> + * above and these macro calls that will build the vectors table below it.
> >>>>> + */
> >>>>> +	HANDLER_INVALID                         // Synchronous EL1t
> >>>>> +	HANDLER_INVALID                         // IRQ EL1t
> >>>>> +	HANDLER_INVALID                         // FIQ EL1t
> >>>>> +	HANDLER_INVALID                         // Error EL1t
> >>>>> +
> >>>>> +	HANDLER	el1h_sync                       // Synchronous EL1h
> >>>>> +	HANDLER	el1h_irq                        // IRQ EL1h
> >>>>> +	HANDLER el1h_fiq                        // FIQ EL1h
> >>>>> +	HANDLER	el1h_error                      // Error EL1h
> >>>>> +
> >>>>> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
> >>>>> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
> >>>>> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
> >>>>> +	HANDLER	el0_error_64                    // Error 64-bit EL0
> >>>>> +
> >>>>> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
> >>>>> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
> >>>>> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
> >>>>> +	HANDLER	el0_error_32                    // Error 32-bit EL0
> >>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>>>> index cee92d477dc0..25be71ec88be 100644
> >>>>> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> >>>>> @@ -6,6 +6,7 @@
> >>>>>   */
> >>>>>  
> >>>>>  #include <linux/compiler.h>
> >>>>> +#include <assert.h>
> >>>>>  
> >>>>>  #include "kvm_util.h"
> >>>>>  #include "../kvm_util_internal.h"
> >>>>> @@ -14,6 +15,8 @@
> >>>>>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
> >>>>>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
> >>>>>  
> >>>>> +vm_vaddr_t exception_handlers;
> >>>>> +
> >>>>>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
> >>>>>  {
> >>>>>  	return (v + vm->page_size) & ~(vm->page_size - 1);
> >>>>> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
> >>>>>  	va_end(ap);
> >>>>>  }
> >>>>>  
> >>>>> +void kvm_exit_unexpected_vector(int vector)
> >>>>> +{
> >>>>> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
> >>>>> +}
> >>>>> +
> >>>>> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
> >>>>> +{
> >>>>> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
> >>>>> +}
> >>>>> +
> >>>>>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
> >>>>>  {
> >>>>> +	struct ucall uc;
> >>>>> +
> >>>>> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
> >>>>> +		return;
> >>>>> +
> >>>>> +	if (uc.args[2]) /* valid_ec */ {
> >>>>> +		assert(VECTOR_IS_SYNC(uc.args[0]));
> >>>>> +		TEST_ASSERT(false,
> >>>>> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
> >>>>> +			uc.args[0], uc.args[1]);
> >>>>> +	} else {
> >>>>> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
> >>>>> +		TEST_ASSERT(false,
> >>>>> +			"Unexpected exception (vector:0x%lx)",
> >>>>> +			uc.args[0]);
> >>>>> +	}
> >>>>> +}
> >>>>> +
> >>>>> +/*
> >>>>> + * This exception handling code was heavily inspired on kvm-unit-tests. There
> >>>>> + * is a set of default vector handlers stored in vector_handlers. These default
> >>>>> + * vector handlers call user-installed handlers stored in exception_handlers.
> >>>>> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
> >>>>> + * (vector, ec=0).
> >>>>> + */
> >>>>> +
> >>>>> +typedef void(*vector_fn)(struct ex_regs *, int vector);
> >>>>> +
> >>>>> +struct handlers {
> >>>>> +	vector_fn vector_handlers[VECTOR_NUM];
> >>>>> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
> >>>>> +};
> >>>>> +
> >>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
> >>>>> +{
> >>>>> +	extern char vectors;
> >>>>> +
> >>>>> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
> >>>>> +}
> >>>>> +
> >>>>> +void default_sync_handler(struct ex_regs *regs, int vector)
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>>>> +	uint64_t esr = read_sysreg(esr_el1);
> >>>>> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
> >>>>> +
> >>>>> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
> >>>>> +
> >>>>> +	if (handlers && handlers->exception_handlers[vector][ec])
> >>>>> +		handlers->exception_handlers[vector][ec](regs);
> >>>>> +	else
> >>>>> +		kvm_exit_unexpected_exception(vector, ec);
> >>>>> +}
> >>>>> +
> >>>>> +void default_irq_handler(struct ex_regs *regs, int vector)
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>>>> +
> >>>>> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
> >>>>> +
> >>>>> +	if (handlers && handlers->exception_handlers[vector][0])
> >>>>> +		handlers->exception_handlers[vector][0](regs);
> >>>>> +	else
> >>>>> +		kvm_exit_unexpected_vector(vector);
> >>>>> +}
> >>>>> +
> >>>>> +void route_exception(struct ex_regs *regs, int vector)
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> >>>>> +
> >>>>> +	if (handlers && handlers->vector_handlers[vector])
> >>>>> +		handlers->vector_handlers[vector](regs, vector);
> >>>>> +	else
> >>>>> +		kvm_exit_unexpected_vector(vector);
> >>>>> +}
> >>>>> +
> >>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm)
> >>>>> +{
> >>>>> +	struct handlers *handlers;
> >>>>> +
> >>>>> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
> >>>>> +			vm->page_size, 0, 0);
> >>>>> +
> >>>>> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>>>> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
> >>>>> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
> >>>>> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
> >>>>> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
> >>>>
> >>>> How about FIQ, Error? Although they are unlikely, they are valid
> >>>> exceptions.
> >>>>
> >>>>> +
> >>>>> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
> >>>>> +}
> >>>>> +
> >>>>> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
> >>>>> +			 void (*handler)(struct ex_regs *))
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>>>> +
> >>>>> +	assert(VECTOR_IS_SYNC(vector));
> >>>>> +	assert(vector < VECTOR_NUM);
> >>>>> +	assert(ec < ESR_EC_NUM);
> >>>>> +	handlers->exception_handlers[vector][ec] = handler;
> >>>>> +}
> >>>>> +
> >>>>> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
> >>>>> +			 void (*handler)(struct ex_regs *))
> >>>>> +{
> >>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> >>>>> +
> >>>>> +	assert(!VECTOR_IS_SYNC(vector));
> >>>>> +	assert(vector < VECTOR_NUM);
> >>>>> +	handlers->exception_handlers[vector][0] = handler;
> >>>>>  }
> >>>>
> >>>> Thanks,
> >>>>
> >>>> 	M.
> >>>>
> >>>> -- 
> >>>> Without deviation from the norm, progress is not possible.
> >>>
> >>
> > 
> 

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Auger Eric]

Hi Ricardo,

On 5/12/21 9:27 AM, Ricardo Koller wrote:
> On Fri, May 07, 2021 at 04:08:07PM +0200, Auger Eric wrote:
>> Hi Ricardo,
>>
>> On 5/6/21 9:14 PM, Ricardo Koller wrote:
>>> On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
>>>> Hi Ricardo,
>>>>
>>>
>>> Hi Eric,
>>>
>>> Thank you very much for the test.
>>>
>>>> On 5/3/21 9:12 PM, Ricardo Koller wrote:
>>>>> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
>>>>>> On Sat, 01 May 2021 00:24:06 +0100,
>>>>>> Ricardo Koller <ricarkol@google.com> wrote:
>>>>>>>
>>>>>>> Add the infrastructure needed to enable exception handling in aarch64
>>>>>>> selftests. The exception handling defaults to an unhandled-exception
>>>>>>> handler which aborts the test, just like x86. These handlers can be
>>>>>>> overridden by calling vm_install_vector_handler(vector) or
>>>>>>> vm_install_exception_handler(vector, ec). The unhandled exception
>>>>>>> reporting from the guest is done using the ucall type introduced in a
>>>>>>> previous commit, UCALL_UNHANDLED.
>>>>>>>
>>>>>>> The exception handling code is heavily inspired on kvm-unit-tests.
>>>>
>>>> running the test on 5.12 I get
>>>>
> 
> Hi Eric,
> 
> I'm able to reproduce the failure you are seeing on 5.6, specifically
> with kernels older than this commit:
> 
>   4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last vcpu_load on VHE
> 
> but not yet on v5.12. Could you share the commit of the kernel you are
> testing, please?

my host is a 5.12 kernel (8404c9fbc84b)

Thanks

Eric
> 
> Thanks!
> Ricardo
> 
>>>> ==== Test Assertion Failure ====
>>>>   aarch64/debug-exceptions.c:232: false
>>>>   pid=6477 tid=6477 errno=4 - Interrupted system call
>>>>      1	0x000000000040147b: main at debug-exceptions.c:230
>>>>      2	0x000003ff8aa60de3: ?? ??:0
>>>>      3	0x0000000000401517: _start at :?
>>>>   Failed guest assert: hw_bp_addr == PC(hw_bp) at
>>>> aarch64/debug-exceptions.c:105
>>>> 	values: 0, 0x401794
>>>>
>>>>
>>>> I guess it is not an expected result. Any known bug waiting on the list?
>>>>
>>>
>>> Not expected. That should work, or at least abort early because there is
>>> no HW breakpoints support.
>>>
>>> I'm trying to reproduce the failure; can you help me with some
>>> questions, please?
>> sure, please find the answers below.
>>>
>>> - does your setup have support for hardware breakpoints? Can you try a
>>>   'dmesg | grep break'? I'm looking for something like 'hw-breakpoint:
>>>   found ...'. If there is no such line it's very likely that the check
>>>   for "debug_ver >= 6" is not enough and the test should check for
>>>   "num_breakpoints > 0".
>> [   25.640418] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
>>> - does it fail consistently (every single attempt)?
>> yes it does.
>>
>> I will try to find some time to investigate too
>>
>> Thanks
>>
>> Eric
>>>
>>> Thanks!
>>> Ricardo
>>>
>>>>
>>>> Thanks
>>>>
>>>> Eric
>>>
>>>
>>>>>>>
>>>>>>> Signed-off-by: Ricardo Koller <ricarkol@google.com>
>>>>>>> ---
>>>>>>>  tools/testing/selftests/kvm/Makefile          |   2 +-
>>>>>>>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
>>>>>>>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
>>>>>>>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
>>>>>>>  4 files changed, 333 insertions(+), 1 deletion(-)
>>>>>>>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>>>>>>
>>>>>>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
>>>>>>> index 4e548d7ab0ab..618c5903f478 100644
>>>>>>> --- a/tools/testing/selftests/kvm/Makefile
>>>>>>> +++ b/tools/testing/selftests/kvm/Makefile
>>>>>>> @@ -35,7 +35,7 @@ endif
>>>>>>>  
>>>>>>>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
>>>>>>>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
>>>>>>> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
>>>>>>> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
>>>>>>>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
>>>>>>>  
>>>>>>>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
>>>>>>> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
>>>>>>> index b7fa0c8551db..40aae31b4afc 100644
>>>>>>> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
>>>>>>> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
>>>>>>> @@ -8,6 +8,7 @@
>>>>>>>  #define SELFTEST_KVM_PROCESSOR_H
>>>>>>>  
>>>>>>>  #include "kvm_util.h"
>>>>>>> +#include <linux/stringify.h>
>>>>>>>  
>>>>>>>  
>>>>>>>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
>>>>>>> @@ -18,6 +19,7 @@
>>>>>>>  #define MAIR_EL1	3, 0, 10, 2, 0
>>>>>>>  #define TTBR0_EL1	3, 0,  2, 0, 0
>>>>>>>  #define SCTLR_EL1	3, 0,  1, 0, 0
>>>>>>> +#define VBAR_EL1	3, 0, 12, 0, 0
>>>>>>>  
>>>>>>>  /*
>>>>>>>   * Default MAIR
>>>>>>> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
>>>>>>>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
>>>>>>>  			      struct kvm_vcpu_init *init, void *guest_code);
>>>>>>>  
>>>>>>> +struct ex_regs {
>>>>>>> +	u64 regs[31];
>>>>>>> +	u64 sp;
>>>>>>> +	u64 pc;
>>>>>>> +	u64 pstate;
>>>>>>> +};
>>>>>>> +
>>>>>>> +#define VECTOR_NUM	16
>>>>>>> +
>>>>>>> +enum {
>>>>>>> +	VECTOR_SYNC_CURRENT_SP0,
>>>>>>> +	VECTOR_IRQ_CURRENT_SP0,
>>>>>>> +	VECTOR_FIQ_CURRENT_SP0,
>>>>>>> +	VECTOR_ERROR_CURRENT_SP0,
>>>>>>> +
>>>>>>> +	VECTOR_SYNC_CURRENT,
>>>>>>> +	VECTOR_IRQ_CURRENT,
>>>>>>> +	VECTOR_FIQ_CURRENT,
>>>>>>> +	VECTOR_ERROR_CURRENT,
>>>>>>> +
>>>>>>> +	VECTOR_SYNC_LOWER_64,
>>>>>>> +	VECTOR_IRQ_LOWER_64,
>>>>>>> +	VECTOR_FIQ_LOWER_64,
>>>>>>> +	VECTOR_ERROR_LOWER_64,
>>>>>>> +
>>>>>>> +	VECTOR_SYNC_LOWER_32,
>>>>>>> +	VECTOR_IRQ_LOWER_32,
>>>>>>> +	VECTOR_FIQ_LOWER_32,
>>>>>>> +	VECTOR_ERROR_LOWER_32,
>>>>>>> +};
>>>>>>> +
>>>>>>> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
>>>>>>> +			   (v) == VECTOR_SYNC_CURRENT     || \
>>>>>>> +			   (v) == VECTOR_SYNC_LOWER_64    || \
>>>>>>> +			   (v) == VECTOR_SYNC_LOWER_32)
>>>>>>> +
>>>>>>> +/* Some common EC (Exception classes) */
>>>>>>> +#define ESR_EC_ILLEGAL_INS	0x0e
>>>>>>> +#define ESR_EC_SVC64		0x15
>>>>>>> +#define ESR_EC_IABORT_CURRENT	0x21
>>>>>>> +#define ESR_EC_DABORT_CURRENT	0x25
>>>>>>> +#define ESR_EC_SERROR		0x2f
>>>>>>> +#define ESR_EC_HW_BP_CURRENT	0x31
>>>>>>> +#define ESR_EC_SSTEP_CURRENT	0x33
>>>>>>> +#define ESR_EC_WP_CURRENT	0x35
>>>>>>> +#define ESR_EC_BRK_INS		0x3C
>>>>>>> +
>>>>>>> +#define ESR_EC_NUM		64
>>>>>>> +
>>>>>>> +#define ESR_EC_SHIFT		26
>>>>>>> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
>>>>>>> +
>>>>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm);
>>>>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
>>>>>>> +
>>>>>>> +typedef void(*handler_fn)(struct ex_regs *);
>>>>>>> +void vm_install_exception_handler(struct kvm_vm *vm,
>>>>>>> +		int vector, int ec, handler_fn handler);
>>>>>>> +void vm_install_vector_handler(struct kvm_vm *vm,
>>>>>>> +		int vector, handler_fn handler);
>>>>>>> +
>>>>>>> +#define SPSR_D          (1 << 9)
>>>>>>> +#define SPSR_SS         (1 << 21)
>>>>>>> +
>>>>>>> +#define write_sysreg(reg, val)						  \
>>>>>>> +({									  \
>>>>>>> +	u64 __val = (u64)(val);						  \
>>>>>>> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
>>>>>>> +})
>>>>>>> +
>>>>>>> +#define read_sysreg(reg)						  \
>>>>>>> +({	u64 val;							  \
>>>>>>> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
>>>>>>> +	val;								  \
>>>>>>> +})
>>>>>>> +
>>>>>>>  #endif /* SELFTEST_KVM_PROCESSOR_H */
>>>>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>>>>>> new file mode 100644
>>>>>>> index 000000000000..8a560021892b
>>>>>>> --- /dev/null
>>>>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
>>>>>>> @@ -0,0 +1,130 @@
>>>>>>> +/* SPDX-License-Identifier: GPL-2.0 */
>>>>>>> +.macro save_registers, vector
>>>>>>> +	add	sp, sp, #-16 * 17
>>>>>>> +
>>>>>>> +	stp	x0, x1, [sp, #16 * 0]
>>>>>>> +	stp	x2, x3, [sp, #16 * 1]
>>>>>>> +	stp	x4, x5, [sp, #16 * 2]
>>>>>>> +	stp	x6, x7, [sp, #16 * 3]
>>>>>>> +	stp	x8, x9, [sp, #16 * 4]
>>>>>>> +	stp	x10, x11, [sp, #16 * 5]
>>>>>>> +	stp	x12, x13, [sp, #16 * 6]
>>>>>>> +	stp	x14, x15, [sp, #16 * 7]
>>>>>>> +	stp	x16, x17, [sp, #16 * 8]
>>>>>>> +	stp	x18, x19, [sp, #16 * 9]
>>>>>>> +	stp	x20, x21, [sp, #16 * 10]
>>>>>>> +	stp	x22, x23, [sp, #16 * 11]
>>>>>>> +	stp	x24, x25, [sp, #16 * 12]
>>>>>>> +	stp	x26, x27, [sp, #16 * 13]
>>>>>>> +	stp	x28, x29, [sp, #16 * 14]
>>>>>>> +
>>>>>>> +	.if \vector >= 8
>>>>>>> +	mrs	x1, sp_el0
>>>>>>
>>>>>> I'm still a bit perplexed by this. SP_EL0 is never changed, since you
>>>>>> always run in handler mode. Therefore, saving/restoring it is only
>>>>>> overhead. If an exception handler wants to introspect it, it is
>>>>>> already available in the relevant system register.
>>>>>>
>>>>>> Or did you have something else in mind for it?
>>>>>>
>>>>>
>>>>> Not really. The reason for saving sp_el0 in there was just for
>>>>> consistency, so that handlers for both el0 and el1 exceptions could get
>>>>> the sp at regs->sp.
>>>>>
>>>>> Restoring sp_el0 might be too much. So, what do you think of this v3: we
>>>>> keep the saving of sp_el0 into regs->sp (to keep things the same between
>>>>> el0 and el1) and delete the restoring of sp_el0?
>>>>>
>>>>> Thanks,
>>>>> Ricardo
>>>>>
>>>>>>> +	.else
>>>>>>> +	/*
>>>>>>> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
>>>>>>> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
>>>>>>> +	 * return from the exception so handlers can not update it.
>>>>>>> +	 */
>>>>>>> +	mov	x1, sp
>>>>>>> +	.endif
>>>>>>> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
>>>>>>> +
>>>>>>> +	mrs	x1, elr_el1
>>>>>>> +	mrs	x2, spsr_el1
>>>>>>> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
>>>>>>> +.endm
>>>>>>> +
>>>>>>> +.macro restore_registers, vector
>>>>>>> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
>>>>>>> +	msr	elr_el1, x1
>>>>>>> +	msr	spsr_el1, x2
>>>>>>> +
>>>>>>> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
>>>>>>> +	.if \vector >= 8
>>>>>>> +	msr	sp_el0, x1
>>>>>>> +	.endif
>>>>>>> +
>>>>>>> +	ldp	x28, x29, [sp, #16 * 14]
>>>>>>> +	ldp	x26, x27, [sp, #16 * 13]
>>>>>>> +	ldp	x24, x25, [sp, #16 * 12]
>>>>>>> +	ldp	x22, x23, [sp, #16 * 11]
>>>>>>> +	ldp	x20, x21, [sp, #16 * 10]
>>>>>>> +	ldp	x18, x19, [sp, #16 * 9]
>>>>>>> +	ldp	x16, x17, [sp, #16 * 8]
>>>>>>> +	ldp	x14, x15, [sp, #16 * 7]
>>>>>>> +	ldp	x12, x13, [sp, #16 * 6]
>>>>>>> +	ldp	x10, x11, [sp, #16 * 5]
>>>>>>> +	ldp	x8, x9, [sp, #16 * 4]
>>>>>>> +	ldp	x6, x7, [sp, #16 * 3]
>>>>>>> +	ldp	x4, x5, [sp, #16 * 2]
>>>>>>> +	ldp	x2, x3, [sp, #16 * 1]
>>>>>>> +	ldp	x0, x1, [sp, #16 * 0]
>>>>>>> +
>>>>>>> +	add	sp, sp, #16 * 17
>>>>>>> +
>>>>>>> +	eret
>>>>>>> +.endm
>>>>>>> +
>>>>>>> +.pushsection ".entry.text", "ax"
>>>>>>> +.balign 0x800
>>>>>>> +.global vectors
>>>>>>> +vectors:
>>>>>>> +.popsection
>>>>>>> +
>>>>>>> +.set	vector, 0
>>>>>>> +
>>>>>>> +/*
>>>>>>> + * Build an exception handler for vector and append a jump to it into
>>>>>>> + * vectors (while making sure that it's 0x80 aligned).
>>>>>>> + */
>>>>>>> +.macro HANDLER, label
>>>>>>> +handler_\()\label:
>>>>>>> +	save_registers vector
>>>>>>> +	mov	x0, sp
>>>>>>> +	mov	x1, #vector
>>>>>>> +	bl	route_exception
>>>>>>> +	restore_registers vector
>>>>>>> +
>>>>>>> +.pushsection ".entry.text", "ax"
>>>>>>> +.balign 0x80
>>>>>>> +	b	handler_\()\label
>>>>>>> +.popsection
>>>>>>> +
>>>>>>> +.set	vector, vector + 1
>>>>>>> +.endm
>>>>>>> +
>>>>>>> +.macro HANDLER_INVALID
>>>>>>> +.pushsection ".entry.text", "ax"
>>>>>>> +.balign 0x80
>>>>>>> +/* This will abort so no need to save and restore registers. */
>>>>>>> +	mov	x0, #vector
>>>>>>> +	b	kvm_exit_unexpected_vector
>>>>>>> +.popsection
>>>>>>> +
>>>>>>> +.set	vector, vector + 1
>>>>>>> +.endm
>>>>>>> +
>>>>>>> +/*
>>>>>>> + * Caution: be sure to not add anything between the declaration of vectors
>>>>>>> + * above and these macro calls that will build the vectors table below it.
>>>>>>> + */
>>>>>>> +	HANDLER_INVALID                         // Synchronous EL1t
>>>>>>> +	HANDLER_INVALID                         // IRQ EL1t
>>>>>>> +	HANDLER_INVALID                         // FIQ EL1t
>>>>>>> +	HANDLER_INVALID                         // Error EL1t
>>>>>>> +
>>>>>>> +	HANDLER	el1h_sync                       // Synchronous EL1h
>>>>>>> +	HANDLER	el1h_irq                        // IRQ EL1h
>>>>>>> +	HANDLER el1h_fiq                        // FIQ EL1h
>>>>>>> +	HANDLER	el1h_error                      // Error EL1h
>>>>>>> +
>>>>>>> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
>>>>>>> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
>>>>>>> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
>>>>>>> +	HANDLER	el0_error_64                    // Error 64-bit EL0
>>>>>>> +
>>>>>>> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
>>>>>>> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
>>>>>>> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
>>>>>>> +	HANDLER	el0_error_32                    // Error 32-bit EL0
>>>>>>> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>>>>>> index cee92d477dc0..25be71ec88be 100644
>>>>>>> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>>>>>> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
>>>>>>> @@ -6,6 +6,7 @@
>>>>>>>   */
>>>>>>>  
>>>>>>>  #include <linux/compiler.h>
>>>>>>> +#include <assert.h>
>>>>>>>  
>>>>>>>  #include "kvm_util.h"
>>>>>>>  #include "../kvm_util_internal.h"
>>>>>>> @@ -14,6 +15,8 @@
>>>>>>>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
>>>>>>>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
>>>>>>>  
>>>>>>> +vm_vaddr_t exception_handlers;
>>>>>>> +
>>>>>>>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
>>>>>>>  {
>>>>>>>  	return (v + vm->page_size) & ~(vm->page_size - 1);
>>>>>>> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
>>>>>>>  	va_end(ap);
>>>>>>>  }
>>>>>>>  
>>>>>>> +void kvm_exit_unexpected_vector(int vector)
>>>>>>> +{
>>>>>>> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);
>>>>>>> +}
>>>>>>> +
>>>>>>> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
>>>>>>> +{
>>>>>>> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);
>>>>>>> +}
>>>>>>> +
>>>>>>>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
>>>>>>>  {
>>>>>>> +	struct ucall uc;
>>>>>>> +
>>>>>>> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
>>>>>>> +		return;
>>>>>>> +
>>>>>>> +	if (uc.args[2]) /* valid_ec */ {
>>>>>>> +		assert(VECTOR_IS_SYNC(uc.args[0]));
>>>>>>> +		TEST_ASSERT(false,
>>>>>>> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
>>>>>>> +			uc.args[0], uc.args[1]);
>>>>>>> +	} else {
>>>>>>> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
>>>>>>> +		TEST_ASSERT(false,
>>>>>>> +			"Unexpected exception (vector:0x%lx)",
>>>>>>> +			uc.args[0]);
>>>>>>> +	}
>>>>>>> +}
>>>>>>> +
>>>>>>> +/*
>>>>>>> + * This exception handling code was heavily inspired on kvm-unit-tests. There
>>>>>>> + * is a set of default vector handlers stored in vector_handlers. These default
>>>>>>> + * vector handlers call user-installed handlers stored in exception_handlers.
>>>>>>> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
>>>>>>> + * (vector, ec=0).
>>>>>>> + */
>>>>>>> +
>>>>>>> +typedef void(*vector_fn)(struct ex_regs *, int vector);
>>>>>>> +
>>>>>>> +struct handlers {
>>>>>>> +	vector_fn vector_handlers[VECTOR_NUM];
>>>>>>> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
>>>>>>> +};
>>>>>>> +
>>>>>>> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
>>>>>>> +{
>>>>>>> +	extern char vectors;
>>>>>>> +
>>>>>>> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
>>>>>>> +}
>>>>>>> +
>>>>>>> +void default_sync_handler(struct ex_regs *regs, int vector)
>>>>>>> +{
>>>>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>>>>>> +	uint64_t esr = read_sysreg(esr_el1);
>>>>>>> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
>>>>>>> +
>>>>>>> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
>>>>>>> +
>>>>>>> +	if (handlers && handlers->exception_handlers[vector][ec])
>>>>>>> +		handlers->exception_handlers[vector][ec](regs);
>>>>>>> +	else
>>>>>>> +		kvm_exit_unexpected_exception(vector, ec);
>>>>>>> +}
>>>>>>> +
>>>>>>> +void default_irq_handler(struct ex_regs *regs, int vector)
>>>>>>> +{
>>>>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>>>>>> +
>>>>>>> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
>>>>>>> +
>>>>>>> +	if (handlers && handlers->exception_handlers[vector][0])
>>>>>>> +		handlers->exception_handlers[vector][0](regs);
>>>>>>> +	else
>>>>>>> +		kvm_exit_unexpected_vector(vector);
>>>>>>> +}
>>>>>>> +
>>>>>>> +void route_exception(struct ex_regs *regs, int vector)
>>>>>>> +{
>>>>>>> +	struct handlers *handlers = (struct handlers *)exception_handlers;
>>>>>>> +
>>>>>>> +	if (handlers && handlers->vector_handlers[vector])
>>>>>>> +		handlers->vector_handlers[vector](regs, vector);
>>>>>>> +	else
>>>>>>> +		kvm_exit_unexpected_vector(vector);
>>>>>>> +}
>>>>>>> +
>>>>>>> +void vm_init_descriptor_tables(struct kvm_vm *vm)
>>>>>>> +{
>>>>>>> +	struct handlers *handlers;
>>>>>>> +
>>>>>>> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
>>>>>>> +			vm->page_size, 0, 0);
>>>>>>> +
>>>>>>> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>>>>>> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
>>>>>>> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
>>>>>>> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
>>>>>>> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
>>>>>>
>>>>>> How about FIQ, Error? Although they are unlikely, they are valid
>>>>>> exceptions.
>>>>>>
>>>>>>> +
>>>>>>> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;
>>>>>>> +}
>>>>>>> +
>>>>>>> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
>>>>>>> +			 void (*handler)(struct ex_regs *))
>>>>>>> +{
>>>>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>>>>>> +
>>>>>>> +	assert(VECTOR_IS_SYNC(vector));
>>>>>>> +	assert(vector < VECTOR_NUM);
>>>>>>> +	assert(ec < ESR_EC_NUM);
>>>>>>> +	handlers->exception_handlers[vector][ec] = handler;
>>>>>>> +}
>>>>>>> +
>>>>>>> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
>>>>>>> +			 void (*handler)(struct ex_regs *))
>>>>>>> +{
>>>>>>> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
>>>>>>> +
>>>>>>> +	assert(!VECTOR_IS_SYNC(vector));
>>>>>>> +	assert(vector < VECTOR_NUM);
>>>>>>> +	handlers->exception_handlers[vector][0] = handler;
>>>>>>>  }
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> 	M.
>>>>>>
>>>>>> -- 
>>>>>> Without deviation from the norm, progress is not possible.
>>>>>
>>>>
>>>
>>
> 

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Marc Zyngier]

On 2021-05-12 09:19, Auger Eric wrote:
> Hi Ricardo,
> 
> On 5/12/21 9:27 AM, Ricardo Koller wrote:
>> On Fri, May 07, 2021 at 04:08:07PM +0200, Auger Eric wrote:
>>> Hi Ricardo,
>>> 
>>> On 5/6/21 9:14 PM, Ricardo Koller wrote:
>>>> On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
>>>>> Hi Ricardo,
>>>>> 
>>>> 
>>>> Hi Eric,
>>>> 
>>>> Thank you very much for the test.
>>>> 
>>>>> On 5/3/21 9:12 PM, Ricardo Koller wrote:
>>>>>> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
>>>>>>> On Sat, 01 May 2021 00:24:06 +0100,
>>>>>>> Ricardo Koller <ricarkol@google.com> wrote:
>>>>>>>> 
>>>>>>>> Add the infrastructure needed to enable exception handling in 
>>>>>>>> aarch64
>>>>>>>> selftests. The exception handling defaults to an 
>>>>>>>> unhandled-exception
>>>>>>>> handler which aborts the test, just like x86. These handlers can 
>>>>>>>> be
>>>>>>>> overridden by calling vm_install_vector_handler(vector) or
>>>>>>>> vm_install_exception_handler(vector, ec). The unhandled 
>>>>>>>> exception
>>>>>>>> reporting from the guest is done using the ucall type introduced 
>>>>>>>> in a
>>>>>>>> previous commit, UCALL_UNHANDLED.
>>>>>>>> 
>>>>>>>> The exception handling code is heavily inspired on 
>>>>>>>> kvm-unit-tests.
>>>>> 
>>>>> running the test on 5.12 I get
>>>>> 
>> 
>> Hi Eric,
>> 
>> I'm able to reproduce the failure you are seeing on 5.6, specifically
>> with kernels older than this commit:
>> 
>>   4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last 
>> vcpu_load on VHE
>> 
>> but not yet on v5.12. Could you share the commit of the kernel you are
>> testing, please?
> 
> my host is a 5.12 kernel (8404c9fbc84b)

Time to compare notes then. What HW are you using? Running VHE or not?

Thanks,

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Auger Eric]

Hi,

On 5/12/21 10:33 AM, Marc Zyngier wrote:
> On 2021-05-12 09:19, Auger Eric wrote:
>> Hi Ricardo,
>>
>> On 5/12/21 9:27 AM, Ricardo Koller wrote:
>>> On Fri, May 07, 2021 at 04:08:07PM +0200, Auger Eric wrote:
>>>> Hi Ricardo,
>>>>
>>>> On 5/6/21 9:14 PM, Ricardo Koller wrote:
>>>>> On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
>>>>>> Hi Ricardo,
>>>>>>
>>>>>
>>>>> Hi Eric,
>>>>>
>>>>> Thank you very much for the test.
>>>>>
>>>>>> On 5/3/21 9:12 PM, Ricardo Koller wrote:
>>>>>>> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
>>>>>>>> On Sat, 01 May 2021 00:24:06 +0100,
>>>>>>>> Ricardo Koller <ricarkol@google.com> wrote:
>>>>>>>>>
>>>>>>>>> Add the infrastructure needed to enable exception handling in
>>>>>>>>> aarch64
>>>>>>>>> selftests. The exception handling defaults to an
>>>>>>>>> unhandled-exception
>>>>>>>>> handler which aborts the test, just like x86. These handlers
>>>>>>>>> can be
>>>>>>>>> overridden by calling vm_install_vector_handler(vector) or
>>>>>>>>> vm_install_exception_handler(vector, ec). The unhandled exception
>>>>>>>>> reporting from the guest is done using the ucall type
>>>>>>>>> introduced in a
>>>>>>>>> previous commit, UCALL_UNHANDLED.
>>>>>>>>>
>>>>>>>>> The exception handling code is heavily inspired on kvm-unit-tests.
>>>>>>
>>>>>> running the test on 5.12 I get
>>>>>>
>>>
>>> Hi Eric,
>>>
>>> I'm able to reproduce the failure you are seeing on 5.6, specifically
>>> with kernels older than this commit:
>>>
>>>   4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last
>>> vcpu_load on VHE
>>>
>>> but not yet on v5.12. Could you share the commit of the kernel you are
>>> testing, please?
>>
>> my host is a 5.12 kernel (8404c9fbc84b)
> 
> Time to compare notes then. What HW are you using? Running VHE or not?
VHE yes. Cavium Sabre system.

Thanks

Eric
> 
> Thanks,
> 
>         M.

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Wed, May 12, 2021 at 10:52:09AM +0200, Auger Eric wrote:
> Hi,
> 
> On 5/12/21 10:33 AM, Marc Zyngier wrote:
> > On 2021-05-12 09:19, Auger Eric wrote:
> >> Hi Ricardo,
> >>
> >> On 5/12/21 9:27 AM, Ricardo Koller wrote:
> >>> On Fri, May 07, 2021 at 04:08:07PM +0200, Auger Eric wrote:
> >>>> Hi Ricardo,
> >>>>
> >>>> On 5/6/21 9:14 PM, Ricardo Koller wrote:
> >>>>> On Thu, May 06, 2021 at 02:30:17PM +0200, Auger Eric wrote:
> >>>>>> Hi Ricardo,
> >>>>>>
> >>>>>
> >>>>> Hi Eric,
> >>>>>
> >>>>> Thank you very much for the test.
> >>>>>
> >>>>>> On 5/3/21 9:12 PM, Ricardo Koller wrote:
> >>>>>>> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
> >>>>>>>> On Sat, 01 May 2021 00:24:06 +0100,
> >>>>>>>> Ricardo Koller <ricarkol@google.com> wrote:
> >>>>>>>>>
> >>>>>>>>> Add the infrastructure needed to enable exception handling in
> >>>>>>>>> aarch64
> >>>>>>>>> selftests. The exception handling defaults to an
> >>>>>>>>> unhandled-exception
> >>>>>>>>> handler which aborts the test, just like x86. These handlers
> >>>>>>>>> can be
> >>>>>>>>> overridden by calling vm_install_vector_handler(vector) or
> >>>>>>>>> vm_install_exception_handler(vector, ec). The unhandled exception
> >>>>>>>>> reporting from the guest is done using the ucall type
> >>>>>>>>> introduced in a
> >>>>>>>>> previous commit, UCALL_UNHANDLED.
> >>>>>>>>>
> >>>>>>>>> The exception handling code is heavily inspired on kvm-unit-tests.
> >>>>>>
> >>>>>> running the test on 5.12 I get
> >>>>>>
> >>>
> >>> Hi Eric,
> >>>
> >>> I'm able to reproduce the failure you are seeing on 5.6, specifically
> >>> with kernels older than this commit:
> >>>
> >>>   4942dc6638b0 KVM: arm64: Write arch.mdcr_el2 changes since last
> >>> vcpu_load on VHE
> >>>
> >>> but not yet on v5.12. Could you share the commit of the kernel you are
> >>> testing, please?
> >>
> >> my host is a 5.12 kernel (8404c9fbc84b)
> > 
> > Time to compare notes then. What HW are you using? Running VHE or not?
> VHE yes. Cavium Sabre system.
> 

On my side. VHE (v5.12) on both QEMU (5.2.0 cpu=max) and Ampere Altra.

> Thanks
> 
> Eric
> > 
> > Thanks,
> > 
> >         M.
> 

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Zenghui Yu]

Hi Eric,

On 2021/5/6 20:30, Auger Eric wrote:
> running the test on 5.12 I get
> 
> ==== Test Assertion Failure ====
>   aarch64/debug-exceptions.c:232: false
>   pid=6477 tid=6477 errno=4 - Interrupted system call
>      1	0x000000000040147b: main at debug-exceptions.c:230
>      2	0x000003ff8aa60de3: ?? ??:0
>      3	0x0000000000401517: _start at :?
>   Failed guest assert: hw_bp_addr == PC(hw_bp) at
> aarch64/debug-exceptions.c:105
> 	values: 0, 0x401794

FYI I can also reproduce it on my VHE box. And Drew's suggestion [*]
seemed to work for me. Is the ISB a requirement of architecture?

[*] https://lore.kernel.org/kvm/20210503124925.wxdcyzharpyzeu4v@gator.home/


Thanks,
Zenghui

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Marc Zyngier]

On 2021-05-12 13:59, Zenghui Yu wrote:
> Hi Eric,
> 
> On 2021/5/6 20:30, Auger Eric wrote:
>> running the test on 5.12 I get
>> 
>> ==== Test Assertion Failure ====
>>   aarch64/debug-exceptions.c:232: false
>>   pid=6477 tid=6477 errno=4 - Interrupted system call
>>      1	0x000000000040147b: main at debug-exceptions.c:230
>>      2	0x000003ff8aa60de3: ?? ??:0
>>      3	0x0000000000401517: _start at :?
>>   Failed guest assert: hw_bp_addr == PC(hw_bp) at
>> aarch64/debug-exceptions.c:105
>> 	values: 0, 0x401794
> 
> FYI I can also reproduce it on my VHE box. And Drew's suggestion [*]
> seemed to work for me. Is the ISB a requirement of architecture?

Very much so. Given that there is no context synchronisation (such as
ERET or an interrupt) in this code, the CPU is perfectly allowed to
delay the system register effect as long as it can.

         M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Wed, May 12, 2021 at 02:43:28PM +0100, Marc Zyngier wrote:
> On 2021-05-12 13:59, Zenghui Yu wrote:
> > Hi Eric,
> > 
> > On 2021/5/6 20:30, Auger Eric wrote:
> > > running the test on 5.12 I get
> > > 
> > > ==== Test Assertion Failure ====
> > >   aarch64/debug-exceptions.c:232: false
> > >   pid=6477 tid=6477 errno=4 - Interrupted system call
> > >      1	0x000000000040147b: main at debug-exceptions.c:230
> > >      2	0x000003ff8aa60de3: ?? ??:0
> > >      3	0x0000000000401517: _start at :?
> > >   Failed guest assert: hw_bp_addr == PC(hw_bp) at
> > > aarch64/debug-exceptions.c:105
> > > 	values: 0, 0x401794
> > 
> > FYI I can also reproduce it on my VHE box. And Drew's suggestion [*]
> > seemed to work for me. Is the ISB a requirement of architecture?
> 
> Very much so. Given that there is no context synchronisation (such as
> ERET or an interrupt) in this code, the CPU is perfectly allowed to
> delay the system register effect as long as it can.
> 
>         M.
> -- 
> Jazz is not dead. It just smells funny...

Thank you very much Eric, Zenghui, Marc, and Andrew (for the ISB
suggestion)!

As per Zenghui test, will send a V3 that includes the missing ISBs.
Hopefully that will fix the issue for Eric as well. It's very
interesting that the CPU seems to _always_ reorder those instructions.

Thanks!
Ricardo

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Marc Zyngier]

On 2021-05-12 17:03, Ricardo Koller wrote:
> On Wed, May 12, 2021 at 02:43:28PM +0100, Marc Zyngier wrote:
>> On 2021-05-12 13:59, Zenghui Yu wrote:
>> > Hi Eric,
>> >
>> > On 2021/5/6 20:30, Auger Eric wrote:
>> > > running the test on 5.12 I get
>> > >
>> > > ==== Test Assertion Failure ====
>> > >   aarch64/debug-exceptions.c:232: false
>> > >   pid=6477 tid=6477 errno=4 - Interrupted system call
>> > >      1	0x000000000040147b: main at debug-exceptions.c:230
>> > >      2	0x000003ff8aa60de3: ?? ??:0
>> > >      3	0x0000000000401517: _start at :?
>> > >   Failed guest assert: hw_bp_addr == PC(hw_bp) at
>> > > aarch64/debug-exceptions.c:105
>> > > 	values: 0, 0x401794
>> >
>> > FYI I can also reproduce it on my VHE box. And Drew's suggestion [*]
>> > seemed to work for me. Is the ISB a requirement of architecture?
>> 
>> Very much so. Given that there is no context synchronisation (such as
>> ERET or an interrupt) in this code, the CPU is perfectly allowed to
>> delay the system register effect as long as it can.
>> 
>>         M.
>> --
>> Jazz is not dead. It just smells funny...
> 
> Thank you very much Eric, Zenghui, Marc, and Andrew (for the ISB
> suggestion)!
> 
> As per Zenghui test, will send a V3 that includes the missing ISBs.
> Hopefully that will fix the issue for Eric as well. It's very
> interesting that the CPU seems to _always_ reorder those instructions.

I suspect that because hitting the debug registers can be a costly
operation (it mobilises a lot of resources in the CPU), there is
a strong incentive to let it slide until there is an actual mandate
to commit the resource.

It also means that SW can issue a bunch of these without too much
overhead, and only pay the cost *once*.

Your N1 CPU seems to be less aggressive on this. Implement choice,
I'd say (it probably is more aggressive than TX2 on other things).
Also, QEMU will almost always hide these problems, due to the nature
of TCG.

Thanks,

          M.
-- 
Jazz is not dead. It just smells funny...

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Wed, May 12, 2021 at 05:18:42PM +0100, Marc Zyngier wrote:
> On 2021-05-12 17:03, Ricardo Koller wrote:
> > On Wed, May 12, 2021 at 02:43:28PM +0100, Marc Zyngier wrote:
> > > On 2021-05-12 13:59, Zenghui Yu wrote:
> > > > Hi Eric,
> > > >
> > > > On 2021/5/6 20:30, Auger Eric wrote:
> > > > > running the test on 5.12 I get
> > > > >
> > > > > ==== Test Assertion Failure ====
> > > > >   aarch64/debug-exceptions.c:232: false
> > > > >   pid=6477 tid=6477 errno=4 - Interrupted system call
> > > > >      1	0x000000000040147b: main at debug-exceptions.c:230
> > > > >      2	0x000003ff8aa60de3: ?? ??:0
> > > > >      3	0x0000000000401517: _start at :?
> > > > >   Failed guest assert: hw_bp_addr == PC(hw_bp) at
> > > > > aarch64/debug-exceptions.c:105
> > > > > 	values: 0, 0x401794
> > > >
> > > > FYI I can also reproduce it on my VHE box. And Drew's suggestion [*]
> > > > seemed to work for me. Is the ISB a requirement of architecture?
> > > 
> > > Very much so. Given that there is no context synchronisation (such as
> > > ERET or an interrupt) in this code, the CPU is perfectly allowed to
> > > delay the system register effect as long as it can.
> > > 
> > >         M.
> > > --
> > > Jazz is not dead. It just smells funny...
> > 
> > Thank you very much Eric, Zenghui, Marc, and Andrew (for the ISB
> > suggestion)!
> > 
> > As per Zenghui test, will send a V3 that includes the missing ISBs.
> > Hopefully that will fix the issue for Eric as well. It's very
> > interesting that the CPU seems to _always_ reorder those instructions.
> 
> I suspect that because hitting the debug registers can be a costly
> operation (it mobilises a lot of resources in the CPU), there is
> a strong incentive to let it slide until there is an actual mandate
> to commit the resource.
> 
> It also means that SW can issue a bunch of these without too much
> overhead, and only pay the cost *once*.
> 
> Your N1 CPU seems to be less aggressive on this. Implement choice,
> I'd say (it probably is more aggressive than TX2 on other things).
> Also, QEMU will almost always hide these problems, due to the nature
> of TCG.
> 
> Thanks,
> 
>          M.
> -- 
> Jazz is not dead. It just smells funny...

Thank you, this is very informative.

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Marc Zyngier]

On Mon, 03 May 2021 20:12:21 +0100,
Ricardo Koller <ricarkol@google.com> wrote:
> 
> On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
> > On Sat, 01 May 2021 00:24:06 +0100,
> > Ricardo Koller <ricarkol@google.com> wrote:

[...]

> > > +	.if \vector >= 8
> > > +	mrs	x1, sp_el0
> > 
> > I'm still a bit perplexed by this. SP_EL0 is never changed, since you
> > always run in handler mode. Therefore, saving/restoring it is only
> > overhead. If an exception handler wants to introspect it, it is
> > already available in the relevant system register.
> > 
> > Or did you have something else in mind for it?
> > 
> 
> Not really. The reason for saving sp_el0 in there was just for
> consistency, so that handlers for both el0 and el1 exceptions could
> get the sp at regs->sp.

We already have sp_el0 consistency by virtue of having it stored in in
a sysreg.

> Restoring sp_el0 might be too much. So, what do you think of this
> v3: we keep the saving of sp_el0 into regs->sp (to keep things the
> same between el0 and el1) and delete the restoring of sp_el0?

To me, the whole purpose of saving some some context is to allow the
exception handling code to run C code and introspect the interrupted
state. But saving things that are not affected by the context change
seems a bit pointless.

One thing I'd like to see though is to save sp_el1 as it was at the
point of the exception (because that is meaningful to get the
exception context -- think of an unaligned EL1 stack for example),
which means correcting the value that gets saved.

So I would suggest to *only* save sp_el1, to always save it
(irrespective of the exception coming from EL0 or EL1), and to save a
retro-corrected value so that the handler can directly know where the
previous stack pointer was.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Ricardo Koller]

On Fri, May 07, 2021 at 03:31:56PM +0100, Marc Zyngier wrote:
> On Mon, 03 May 2021 20:12:21 +0100,
> Ricardo Koller <ricarkol@google.com> wrote:
> > 
> > On Mon, May 03, 2021 at 11:32:39AM +0100, Marc Zyngier wrote:
> > > On Sat, 01 May 2021 00:24:06 +0100,
> > > Ricardo Koller <ricarkol@google.com> wrote:
> 
> [...]
> 
> > > > +	.if \vector >= 8
> > > > +	mrs	x1, sp_el0
> > > 
> > > I'm still a bit perplexed by this. SP_EL0 is never changed, since you
> > > always run in handler mode. Therefore, saving/restoring it is only
> > > overhead. If an exception handler wants to introspect it, it is
> > > already available in the relevant system register.
> > > 
> > > Or did you have something else in mind for it?
> > > 
> > 
> > Not really. The reason for saving sp_el0 in there was just for
> > consistency, so that handlers for both el0 and el1 exceptions could
> > get the sp at regs->sp.
> 
> We already have sp_el0 consistency by virtue of having it stored in in
> a sysreg.
> 
> > Restoring sp_el0 might be too much. So, what do you think of this
> > v3: we keep the saving of sp_el0 into regs->sp (to keep things the
> > same between el0 and el1) and delete the restoring of sp_el0?
> 
> To me, the whole purpose of saving some some context is to allow the
> exception handling code to run C code and introspect the interrupted
> state. But saving things that are not affected by the context change
> seems a bit pointless.
> 
> One thing I'd like to see though is to save sp_el1 as it was at the
> point of the exception (because that is meaningful to get the
> exception context -- think of an unaligned EL1 stack for example),
> which means correcting the value that gets saved.

Got it. Replacing:
	mov     x1, sp
with:
	add     x1, sp, #16 * 17

> 
> So I would suggest to *only* save sp_el1, to always save it
> (irrespective of the exception coming from EL0 or EL1), and to save a
> retro-corrected value so that the handler can directly know where the
> previous stack pointer was.

Sounds good, will send a V3 accordingly.

Thanks!
Ricardo

> 
> Thanks,
> 
> 	M.
> 
> -- 
> Without deviation from the norm, progress is not possible.

Re: [PATCH v2 4/5] KVM: selftests: Add exception handling support for aarch64

[Andrew Jones]

On Fri, Apr 30, 2021 at 04:24:06PM -0700, Ricardo Koller wrote:
> Add the infrastructure needed to enable exception handling in aarch64
> selftests. The exception handling defaults to an unhandled-exception
> handler which aborts the test, just like x86. These handlers can be
> overridden by calling vm_install_vector_handler(vector) or
> vm_install_exception_handler(vector, ec). The unhandled exception
> reporting from the guest is done using the ucall type introduced in a
> previous commit, UCALL_UNHANDLED.
> 
> The exception handling code is heavily inspired on kvm-unit-tests.
> 
> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> ---
>  tools/testing/selftests/kvm/Makefile          |   2 +-
>  .../selftests/kvm/include/aarch64/processor.h |  78 +++++++++++
>  .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++++++++++
>  .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++++++++++
>  4 files changed, 333 insertions(+), 1 deletion(-)
>  create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
> 
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 4e548d7ab0ab..618c5903f478 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -35,7 +35,7 @@ endif
>  
>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c lib/test_util.c lib/guest_modes.c lib/perf_test_util.c
>  LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c lib/x86_64/handlers.S
> -LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
> +LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c lib/aarch64/handlers.S
>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c lib/s390x/diag318_test_handler.c
>  
>  TEST_GEN_PROGS_x86_64 = x86_64/cr4_cpuid_sync_test
> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
> index b7fa0c8551db..40aae31b4afc 100644
> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
> @@ -8,6 +8,7 @@
>  #define SELFTEST_KVM_PROCESSOR_H
>  
>  #include "kvm_util.h"
> +#include <linux/stringify.h>
>  
>  
>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
> @@ -18,6 +19,7 @@
>  #define MAIR_EL1	3, 0, 10, 2, 0
>  #define TTBR0_EL1	3, 0,  2, 0, 0
>  #define SCTLR_EL1	3, 0,  1, 0, 0
> +#define VBAR_EL1	3, 0, 12, 0, 0
>  
>  /*
>   * Default MAIR
> @@ -56,4 +58,80 @@ void aarch64_vcpu_setup(struct kvm_vm *vm, int vcpuid, struct kvm_vcpu_init *ini
>  void aarch64_vcpu_add_default(struct kvm_vm *vm, uint32_t vcpuid,
>  			      struct kvm_vcpu_init *init, void *guest_code);
>  
> +struct ex_regs {
> +	u64 regs[31];
> +	u64 sp;
> +	u64 pc;
> +	u64 pstate;
> +};
> +
> +#define VECTOR_NUM	16
> +
> +enum {
> +	VECTOR_SYNC_CURRENT_SP0,
> +	VECTOR_IRQ_CURRENT_SP0,
> +	VECTOR_FIQ_CURRENT_SP0,
> +	VECTOR_ERROR_CURRENT_SP0,
> +
> +	VECTOR_SYNC_CURRENT,
> +	VECTOR_IRQ_CURRENT,
> +	VECTOR_FIQ_CURRENT,
> +	VECTOR_ERROR_CURRENT,
> +
> +	VECTOR_SYNC_LOWER_64,
> +	VECTOR_IRQ_LOWER_64,
> +	VECTOR_FIQ_LOWER_64,
> +	VECTOR_ERROR_LOWER_64,
> +
> +	VECTOR_SYNC_LOWER_32,
> +	VECTOR_IRQ_LOWER_32,
> +	VECTOR_FIQ_LOWER_32,
> +	VECTOR_ERROR_LOWER_32,
> +};
> +
> +#define VECTOR_IS_SYNC(v) ((v) == VECTOR_SYNC_CURRENT_SP0 || \
> +			   (v) == VECTOR_SYNC_CURRENT     || \
> +			   (v) == VECTOR_SYNC_LOWER_64    || \
> +			   (v) == VECTOR_SYNC_LOWER_32)
> +
> +/* Some common EC (Exception classes) */
> +#define ESR_EC_ILLEGAL_INS	0x0e
> +#define ESR_EC_SVC64		0x15
> +#define ESR_EC_IABORT_CURRENT	0x21
> +#define ESR_EC_DABORT_CURRENT	0x25
> +#define ESR_EC_SERROR		0x2f
> +#define ESR_EC_HW_BP_CURRENT	0x31
> +#define ESR_EC_SSTEP_CURRENT	0x33
> +#define ESR_EC_WP_CURRENT	0x35
> +#define ESR_EC_BRK_INS		0x3C

Let's leave it to the specific unit tests to define the above EC's as they
are needed. The unit tests can also then decide if the defines belong in
the common header or in the unit test itself.

> +
> +#define ESR_EC_NUM		64
> +
> +#define ESR_EC_SHIFT		26
> +#define ESR_EC_MASK		(ESR_EC_NUM - 1)
> +
> +void vm_init_descriptor_tables(struct kvm_vm *vm);
> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid);
> +
> +typedef void(*handler_fn)(struct ex_regs *);
> +void vm_install_exception_handler(struct kvm_vm *vm,
> +		int vector, int ec, handler_fn handler);
> +void vm_install_vector_handler(struct kvm_vm *vm,
> +		int vector, handler_fn handler);
> +
> +#define SPSR_D          (1 << 9)
> +#define SPSR_SS         (1 << 21)

I think these two SPSR defines belong directly in the debug unit test,
rather than the common header.

> +
> +#define write_sysreg(reg, val)						  \
> +({									  \
> +	u64 __val = (u64)(val);						  \
> +	asm volatile("msr " __stringify(reg) ", %x0" : : "rZ" (__val));	  \
> +})
> +
> +#define read_sysreg(reg)						  \
> +({	u64 val;							  \
> +	asm volatile("mrs %0, "__stringify(reg) : "=r"(val) : : "memory");\
> +	val;								  \
> +})
> +
>  #endif /* SELFTEST_KVM_PROCESSOR_H */
> diff --git a/tools/testing/selftests/kvm/lib/aarch64/handlers.S b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> new file mode 100644
> index 000000000000..8a560021892b
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/lib/aarch64/handlers.S
> @@ -0,0 +1,130 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +.macro save_registers, vector
> +	add	sp, sp, #-16 * 17
> +
> +	stp	x0, x1, [sp, #16 * 0]
> +	stp	x2, x3, [sp, #16 * 1]
> +	stp	x4, x5, [sp, #16 * 2]
> +	stp	x6, x7, [sp, #16 * 3]
> +	stp	x8, x9, [sp, #16 * 4]
> +	stp	x10, x11, [sp, #16 * 5]
> +	stp	x12, x13, [sp, #16 * 6]
> +	stp	x14, x15, [sp, #16 * 7]
> +	stp	x16, x17, [sp, #16 * 8]
> +	stp	x18, x19, [sp, #16 * 9]
> +	stp	x20, x21, [sp, #16 * 10]
> +	stp	x22, x23, [sp, #16 * 11]
> +	stp	x24, x25, [sp, #16 * 12]
> +	stp	x26, x27, [sp, #16 * 13]
> +	stp	x28, x29, [sp, #16 * 14]
> +
> +	.if \vector >= 8
> +	mrs	x1, sp_el0
> +	.else
> +	/*
> +	 * This stores sp_el1 into ex_regs.sp so exception handlers can
> +	 * "look" at it. It will _not_ be used to restore the sp_el1 on
> +	 * return from the exception so handlers can not update it.
> +	 */
> +	mov	x1, sp
> +	.endif
> +	stp	x30, x1, [sp, #16 * 15] /* x30, SP */
> +
> +	mrs	x1, elr_el1
> +	mrs	x2, spsr_el1
> +	stp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> +.endm
> +
> +.macro restore_registers, vector
> +	ldp	x1, x2, [sp, #16 * 16] /* PC, PSTATE */
> +	msr	elr_el1, x1
> +	msr	spsr_el1, x2
> +
> +	ldp	x30, x1, [sp, #16 * 15] /* x30, SP */
> +	.if \vector >= 8
> +	msr	sp_el0, x1
> +	.endif
> +
> +	ldp	x28, x29, [sp, #16 * 14]
> +	ldp	x26, x27, [sp, #16 * 13]
> +	ldp	x24, x25, [sp, #16 * 12]
> +	ldp	x22, x23, [sp, #16 * 11]
> +	ldp	x20, x21, [sp, #16 * 10]
> +	ldp	x18, x19, [sp, #16 * 9]
> +	ldp	x16, x17, [sp, #16 * 8]
> +	ldp	x14, x15, [sp, #16 * 7]
> +	ldp	x12, x13, [sp, #16 * 6]
> +	ldp	x10, x11, [sp, #16 * 5]
> +	ldp	x8, x9, [sp, #16 * 4]
> +	ldp	x6, x7, [sp, #16 * 3]
> +	ldp	x4, x5, [sp, #16 * 2]
> +	ldp	x2, x3, [sp, #16 * 1]
> +	ldp	x0, x1, [sp, #16 * 0]
> +
> +	add	sp, sp, #16 * 17
> +
> +	eret
> +.endm
> +
> +.pushsection ".entry.text", "ax"
> +.balign 0x800
> +.global vectors
> +vectors:
> +.popsection
> +
> +.set	vector, 0
> +
> +/*
> + * Build an exception handler for vector and append a jump to it into
> + * vectors (while making sure that it's 0x80 aligned).
> + */
> +.macro HANDLER, label
> +handler_\()\label:

I don't think we need the \(). Based on [1] it may be necessary *after* a
macro argument to separate it from the following text. I don't think it
serves any purpose before the argument though.

[1] https://sourceware.org/binutils/docs/as/Macro.html


> +	save_registers vector
> +	mov	x0, sp
> +	mov	x1, #vector
> +	bl	route_exception
> +	restore_registers vector
> +
> +.pushsection ".entry.text", "ax"
> +.balign 0x80
> +	b	handler_\()\label
> +.popsection
> +
> +.set	vector, vector + 1
> +.endm
> +
> +.macro HANDLER_INVALID
> +.pushsection ".entry.text", "ax"
> +.balign 0x80
> +/* This will abort so no need to save and restore registers. */
> +	mov	x0, #vector
> +	b	kvm_exit_unexpected_vector

I'd put a 'b .' here out of paranoia.

> +.popsection
> +
> +.set	vector, vector + 1
> +.endm
> +
> +/*
> + * Caution: be sure to not add anything between the declaration of vectors
> + * above and these macro calls that will build the vectors table below it.
> + */
> +	HANDLER_INVALID                         // Synchronous EL1t
> +	HANDLER_INVALID                         // IRQ EL1t
> +	HANDLER_INVALID                         // FIQ EL1t
> +	HANDLER_INVALID                         // Error EL1t
> +
> +	HANDLER	el1h_sync                       // Synchronous EL1h
> +	HANDLER	el1h_irq                        // IRQ EL1h
> +	HANDLER el1h_fiq                        // FIQ EL1h
> +	HANDLER	el1h_error                      // Error EL1h
> +
> +	HANDLER	el0_sync_64                     // Synchronous 64-bit EL0
> +	HANDLER	el0_irq_64                      // IRQ 64-bit EL0
> +	HANDLER	el0_fiq_64                      // FIQ 64-bit EL0
> +	HANDLER	el0_error_64                    // Error 64-bit EL0
> +
> +	HANDLER	el0_sync_32                     // Synchronous 32-bit EL0
> +	HANDLER	el0_irq_32                      // IRQ 32-bit EL0
> +	HANDLER	el0_fiq_32                      // FIQ 32-bit EL0
> +	HANDLER	el0_error_32                    // Error 32-bit EL0
> diff --git a/tools/testing/selftests/kvm/lib/aarch64/processor.c b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> index cee92d477dc0..25be71ec88be 100644
> --- a/tools/testing/selftests/kvm/lib/aarch64/processor.c
> +++ b/tools/testing/selftests/kvm/lib/aarch64/processor.c
> @@ -6,6 +6,7 @@
>   */
>  
>  #include <linux/compiler.h>
> +#include <assert.h>
>  
>  #include "kvm_util.h"
>  #include "../kvm_util_internal.h"
> @@ -14,6 +15,8 @@
>  #define KVM_GUEST_PAGE_TABLE_MIN_PADDR		0x180000
>  #define DEFAULT_ARM64_GUEST_STACK_VADDR_MIN	0xac0000
>  
> +vm_vaddr_t exception_handlers;

static? Also some functions below could maybe be static.

> +
>  static uint64_t page_align(struct kvm_vm *vm, uint64_t v)
>  {
>  	return (v + vm->page_size) & ~(vm->page_size - 1);
> @@ -334,6 +337,127 @@ void vcpu_args_set(struct kvm_vm *vm, uint32_t vcpuid, unsigned int num, ...)
>  	va_end(ap);
>  }
>  
> +void kvm_exit_unexpected_vector(int vector)
> +{
> +	ucall(UCALL_UNHANDLED, 3, vector, 0, false /* !valid_ec */);

Instead of the 'b . ' suggested above, there could be a 'while (1) ;' here.


> +}
> +
> +void kvm_exit_unexpected_exception(int vector, uint64_t ec)
> +{
> +	ucall(UCALL_UNHANDLED, 3, vector, ec, true /* valid_ec */);

And also a 'while (1) ;' here.

> +}
> +
>  void assert_on_unhandled_exception(struct kvm_vm *vm, uint32_t vcpuid)
>  {
> +	struct ucall uc;
> +
> +	if (get_ucall(vm, vcpuid, &uc) != UCALL_UNHANDLED)
> +		return;
> +
> +	if (uc.args[2]) /* valid_ec */ {
> +		assert(VECTOR_IS_SYNC(uc.args[0]));
> +		TEST_ASSERT(false,
> +			"Unexpected exception (vector:0x%lx, ec:0x%lx)",
> +			uc.args[0], uc.args[1]);
> +	} else {
> +		assert(!VECTOR_IS_SYNC(uc.args[0]));
> +		TEST_ASSERT(false,
> +			"Unexpected exception (vector:0x%lx)",
> +			uc.args[0]);
> +	}

Can use TEST_FAIL for the above to TEST_ASSERT(false, ...)

> +}
> +
> +/*
> + * This exception handling code was heavily inspired on kvm-unit-tests. There
> + * is a set of default vector handlers stored in vector_handlers. These default
> + * vector handlers call user-installed handlers stored in exception_handlers.
> + * Synchronous handlers are indexed by (vector, ec), and irq handlers by
> + * (vector, ec=0).
> + */
> +
> +typedef void(*vector_fn)(struct ex_regs *, int vector);
> +
> +struct handlers {
> +	vector_fn vector_handlers[VECTOR_NUM];
> +	handler_fn exception_handlers[VECTOR_NUM][ESR_EC_NUM];
> +};
> +
> +void vcpu_init_descriptor_tables(struct kvm_vm *vm, uint32_t vcpuid)
> +{
> +	extern char vectors;
> +
> +	set_reg(vm, vcpuid, ARM64_SYS_REG(VBAR_EL1), (uint64_t)&vectors);
> +}
> +
> +void default_sync_handler(struct ex_regs *regs, int vector)
> +{
> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> +	uint64_t esr = read_sysreg(esr_el1);
> +	uint64_t ec = (esr >> ESR_EC_SHIFT) & ESR_EC_MASK;
> +
> +	GUEST_ASSERT(VECTOR_IS_SYNC(vector));
> +
> +	if (handlers && handlers->exception_handlers[vector][ec])
> +		handlers->exception_handlers[vector][ec](regs);
> +	else
> +		kvm_exit_unexpected_exception(vector, ec);
> +}
> +
> +void default_irq_handler(struct ex_regs *regs, int vector)
> +{
> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> +
> +	GUEST_ASSERT(!VECTOR_IS_SYNC(vector));
> +
> +	if (handlers && handlers->exception_handlers[vector][0])
> +		handlers->exception_handlers[vector][0](regs);
> +	else
> +		kvm_exit_unexpected_vector(vector);
> +}
> +
> +void route_exception(struct ex_regs *regs, int vector)
> +{
> +	struct handlers *handlers = (struct handlers *)exception_handlers;
> +
> +	if (handlers && handlers->vector_handlers[vector])
> +		handlers->vector_handlers[vector](regs, vector);
> +	else
> +		kvm_exit_unexpected_vector(vector);
> +}
> +
> +void vm_init_descriptor_tables(struct kvm_vm *vm)
> +{
> +	struct handlers *handlers;
> +
> +	vm->handlers = vm_vaddr_alloc(vm, sizeof(struct handlers),
> +			vm->page_size, 0, 0);
> +
> +	handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> +	handlers->vector_handlers[VECTOR_SYNC_CURRENT] = default_sync_handler;
> +	handlers->vector_handlers[VECTOR_IRQ_CURRENT] = default_irq_handler;
> +	handlers->vector_handlers[VECTOR_SYNC_LOWER_64] = default_sync_handler;
> +	handlers->vector_handlers[VECTOR_IRQ_LOWER_64] = default_irq_handler;
> +
> +	*(vm_vaddr_t *)addr_gva2hva(vm, (vm_vaddr_t)(&exception_handlers)) = vm->handlers;

I see this is derived from the x86 code, so I guess it's OK for now, but
having the handlers be VM state instead of VCPU state means the VCPUs will
have to always share the same handlers. I think we should make them
per-vcpu instead.

> +}
> +
> +void vm_install_exception_handler(struct kvm_vm *vm, int vector, int ec,
> +			 void (*handler)(struct ex_regs *))
> +{
> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> +
> +	assert(VECTOR_IS_SYNC(vector));
> +	assert(vector < VECTOR_NUM);
> +	assert(ec < ESR_EC_NUM);
> +	handlers->exception_handlers[vector][ec] = handler;
> +}
> +
> +void vm_install_vector_handler(struct kvm_vm *vm, int vector,
> +			 void (*handler)(struct ex_regs *))
> +{
> +	struct handlers *handlers = (struct handlers *)addr_gva2hva(vm, vm->handlers);
> +
> +	assert(!VECTOR_IS_SYNC(vector));
> +	assert(vector < VECTOR_NUM);
> +	handlers->exception_handlers[vector][0] = handler;
>  }
> -- 
> 2.31.1.527.g47e6f16901-goog
>

Thanks,
drew

[PATCH v2 5/5] KVM: selftests: Add aarch64/debug-exceptions test

[Ricardo Koller]

Covers fundamental tests for debug exceptions. The guest installs and
handle its debug exceptions itself, without KVM_SET_GUEST_DEBUG.

Signed-off-by: Ricardo Koller <ricarkol@google.com>
---
 tools/testing/selftests/kvm/.gitignore        |   1 +
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
 .../selftests/kvm/include/aarch64/processor.h |  14 +-
 4 files changed, 254 insertions(+), 6 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c

diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
index e65d5572aefc..f09ed908422b 100644
--- a/tools/testing/selftests/kvm/.gitignore
+++ b/tools/testing/selftests/kvm/.gitignore
@@ -1,4 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
+/aarch64/debug-exceptions
 /aarch64/get-reg-list
 /aarch64/get-reg-list-sve
 /aarch64/vgic_init
diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 618c5903f478..2f92442c0cc9 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -73,6 +73,7 @@ TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test
 TEST_GEN_PROGS_x86_64 += set_memory_region_test
 TEST_GEN_PROGS_x86_64 += steal_time
 
+TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
 TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve
 TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
diff --git a/tools/testing/selftests/kvm/aarch64/debug-exceptions.c b/tools/testing/selftests/kvm/aarch64/debug-exceptions.c
new file mode 100644
index 000000000000..87352ee09211
--- /dev/null
+++ b/tools/testing/selftests/kvm/aarch64/debug-exceptions.c
@@ -0,0 +1,244 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_util.h>
+#include <kvm_util.h>
+#include <processor.h>
+
+#define VCPU_ID 0
+
+#define MDSCR_KDE	(1 << 13)
+#define MDSCR_MDE	(1 << 15)
+#define MDSCR_SS	(1 << 0)
+
+#define DBGBCR_LEN8	(0xff << 5)
+#define DBGBCR_EXEC	(0x0 << 3)
+#define DBGBCR_EL1	(0x1 << 1)
+#define DBGBCR_E	(0x1 << 0)
+
+#define DBGWCR_LEN8	(0xff << 5)
+#define DBGWCR_RD	(0x1 << 3)
+#define DBGWCR_WR	(0x2 << 3)
+#define DBGWCR_EL1	(0x1 << 1)
+#define DBGWCR_E	(0x1 << 0)
+
+extern unsigned char sw_bp, hw_bp, bp_svc, bp_brk, hw_wp, ss_start;
+static volatile uint64_t sw_bp_addr, hw_bp_addr;
+static volatile uint64_t wp_addr, wp_data_addr;
+static volatile uint64_t svc_addr;
+static volatile uint64_t ss_addr[4], ss_idx;
+#define  PC(v)  ((uint64_t)&(v))
+
+static void reset_debug_state(void)
+{
+	asm volatile("msr daifset, #8");
+
+	write_sysreg(osdlr_el1, 0);
+	write_sysreg(oslar_el1, 0);
+	asm volatile("isb" : : : "memory");
+
+	write_sysreg(mdscr_el1, 0);
+	/* This test only uses the first bp and wp slot. */
+	write_sysreg(dbgbvr0_el1, 0);
+	write_sysreg(dbgbcr0_el1, 0);
+	write_sysreg(dbgwcr0_el1, 0);
+	write_sysreg(dbgwvr0_el1, 0);
+	asm volatile("isb" : : : "memory");
+}
+
+static void install_wp(uint64_t addr)
+{
+	uint32_t wcr;
+	uint32_t mdscr;
+
+	wcr = DBGWCR_LEN8 | DBGWCR_RD | DBGWCR_WR | DBGWCR_EL1 | DBGWCR_E;
+	write_sysreg(dbgwcr0_el1, wcr);
+	write_sysreg(dbgwvr0_el1, addr);
+	asm volatile("isb" : : : "memory");
+
+	asm volatile("msr daifclr, #8");
+
+	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_MDE;
+	write_sysreg(mdscr_el1, mdscr);
+}
+
+static void install_hw_bp(uint64_t addr)
+{
+	uint32_t bcr;
+	uint32_t mdscr;
+
+	bcr = DBGBCR_LEN8 | DBGBCR_EXEC | DBGBCR_EL1 | DBGBCR_E;
+	write_sysreg(dbgbcr0_el1, bcr);
+	write_sysreg(dbgbvr0_el1, addr);
+	asm volatile("isb" : : : "memory");
+
+	asm volatile("msr daifclr, #8");
+
+	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_MDE;
+	write_sysreg(mdscr_el1, mdscr);
+}
+
+static void install_ss(void)
+{
+	uint32_t mdscr;
+
+	asm volatile("msr daifclr, #8");
+
+	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_SS;
+	write_sysreg(mdscr_el1, mdscr);
+}
+
+static volatile char write_data;
+
+static void guest_code(void)
+{
+	GUEST_SYNC(0);
+
+	/* Software-breakpoint */
+	asm volatile("sw_bp: brk #0");
+	GUEST_ASSERT_EQ(sw_bp_addr, PC(sw_bp));
+
+	GUEST_SYNC(1);
+
+	/* Hardware-breakpoint */
+	reset_debug_state();
+	install_hw_bp(PC(hw_bp));
+	asm volatile("hw_bp: nop");
+	GUEST_ASSERT_EQ(hw_bp_addr, PC(hw_bp));
+
+	GUEST_SYNC(2);
+
+	/* Hardware-breakpoint + svc */
+	reset_debug_state();
+	install_hw_bp(PC(bp_svc));
+	asm volatile("bp_svc: svc #0");
+	GUEST_ASSERT_EQ(hw_bp_addr, PC(bp_svc));
+	GUEST_ASSERT_EQ(svc_addr, PC(bp_svc) + 4);
+
+	GUEST_SYNC(3);
+
+	/* Hardware-breakpoint + software-breakpoint */
+	reset_debug_state();
+	install_hw_bp(PC(bp_brk));
+	asm volatile("bp_brk: brk #0");
+	GUEST_ASSERT_EQ(sw_bp_addr, PC(bp_brk));
+	GUEST_ASSERT_EQ(hw_bp_addr, PC(bp_brk));
+
+	GUEST_SYNC(4);
+
+	/* Watchpoint */
+	reset_debug_state();
+	install_wp(PC(write_data));
+	write_data = 'x';
+	GUEST_ASSERT_EQ(write_data, 'x');
+	GUEST_ASSERT_EQ(wp_data_addr, PC(write_data));
+
+	GUEST_SYNC(5);
+
+	/* Single-step */
+	reset_debug_state();
+	install_ss();
+	ss_idx = 0;
+	asm volatile("ss_start:\n"
+		     "mrs x0, esr_el1\n"
+		     "add x0, x0, #1\n"
+		     "msr daifset, #8\n"
+		     : : : "x0");
+	GUEST_ASSERT_EQ(ss_addr[0], PC(ss_start));
+	GUEST_ASSERT_EQ(ss_addr[1], PC(ss_start) + 4);
+	GUEST_ASSERT_EQ(ss_addr[2], PC(ss_start) + 8);
+
+	GUEST_DONE();
+}
+
+static void guest_sw_bp_handler(struct ex_regs *regs)
+{
+	sw_bp_addr = regs->pc;
+	regs->pc += 4;
+}
+
+static void guest_hw_bp_handler(struct ex_regs *regs)
+{
+	hw_bp_addr = regs->pc;
+	regs->pstate |= SPSR_D;
+}
+
+static void guest_wp_handler(struct ex_regs *regs)
+{
+	wp_data_addr = read_sysreg(far_el1);
+	wp_addr = regs->pc;
+	regs->pstate |= SPSR_D;
+}
+
+static void guest_ss_handler(struct ex_regs *regs)
+{
+	GUEST_ASSERT_1(ss_idx < 4, ss_idx);
+	ss_addr[ss_idx++] = regs->pc;
+	regs->pstate |= SPSR_SS;
+}
+
+static void guest_svc_handler(struct ex_regs *regs)
+{
+	svc_addr = regs->pc;
+}
+
+static int debug_version(struct kvm_vm *vm)
+{
+	uint64_t id_aa64dfr0;
+
+	get_reg(vm, VCPU_ID, ARM64_SYS_REG(ID_AA64DFR0_EL1), &id_aa64dfr0);
+	return id_aa64dfr0 & 0xf;
+}
+
+int main(int argc, char *argv[])
+{
+	struct kvm_vm *vm;
+	struct ucall uc;
+	int stage;
+
+	vm = vm_create_default(VCPU_ID, 0, guest_code);
+	ucall_init(vm, NULL);
+
+	vm_init_descriptor_tables(vm);
+	vcpu_init_descriptor_tables(vm, VCPU_ID);
+
+	if (debug_version(vm) < 6) {
+		print_skip("Armv8 debug architecture not supported.");
+		kvm_vm_free(vm);
+		exit(KSFT_SKIP);
+	}
+
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_BRK_INS, guest_sw_bp_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_HW_BP_CURRENT, guest_hw_bp_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_WP_CURRENT, guest_wp_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_SSTEP_CURRENT, guest_ss_handler);
+	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
+				ESR_EC_SVC64, guest_svc_handler);
+
+	for (stage = 0; stage < 7; stage++) {
+		vcpu_run(vm, VCPU_ID);
+
+		switch (get_ucall(vm, VCPU_ID, &uc)) {
+		case UCALL_SYNC:
+			TEST_ASSERT(uc.args[1] == stage,
+				"Stage %d: Unexpected sync ucall, got %lx",
+				stage, (ulong)uc.args[1]);
+			break;
+		case UCALL_ABORT:
+			TEST_FAIL("%s at %s:%ld\n\tvalues: %#lx, %#lx",
+				(const char *)uc.args[0],
+				__FILE__, uc.args[1], uc.args[2], uc.args[3]);
+			break;
+		case UCALL_DONE:
+			goto done;
+		default:
+			TEST_FAIL("Unknown ucall %lu", uc.cmd);
+		}
+	}
+
+done:
+	kvm_vm_free(vm);
+	return 0;
+}
diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
index 40aae31b4afc..a3ebef8e88c7 100644
--- a/tools/testing/selftests/kvm/include/aarch64/processor.h
+++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
@@ -14,12 +14,14 @@
 #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
 			   KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
 
-#define CPACR_EL1	3, 0,  1, 0, 2
-#define TCR_EL1		3, 0,  2, 0, 2
-#define MAIR_EL1	3, 0, 10, 2, 0
-#define TTBR0_EL1	3, 0,  2, 0, 0
-#define SCTLR_EL1	3, 0,  1, 0, 0
-#define VBAR_EL1	3, 0, 12, 0, 0
+#define CPACR_EL1               3, 0,  1, 0, 2
+#define TCR_EL1                 3, 0,  2, 0, 2
+#define MAIR_EL1                3, 0, 10, 2, 0
+#define TTBR0_EL1               3, 0,  2, 0, 0
+#define SCTLR_EL1               3, 0,  1, 0, 0
+#define VBAR_EL1                3, 0, 12, 0, 0
+
+#define ID_AA64DFR0_EL1         3, 0,  0, 5, 0
 
 /*
  * Default MAIR
-- 
2.31.1.527.g47e6f16901-goog

Re: [PATCH v2 5/5] KVM: selftests: Add aarch64/debug-exceptions test

[Andrew Jones]

On Fri, Apr 30, 2021 at 04:24:07PM -0700, Ricardo Koller wrote:
> Covers fundamental tests for debug exceptions. The guest installs and
> handle its debug exceptions itself, without KVM_SET_GUEST_DEBUG.
> 
> Signed-off-by: Ricardo Koller <ricarkol@google.com>
> ---
>  tools/testing/selftests/kvm/.gitignore        |   1 +
>  tools/testing/selftests/kvm/Makefile          |   1 +
>  .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
>  .../selftests/kvm/include/aarch64/processor.h |  14 +-
>  4 files changed, 254 insertions(+), 6 deletions(-)
>  create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c
> 
> diff --git a/tools/testing/selftests/kvm/.gitignore b/tools/testing/selftests/kvm/.gitignore
> index e65d5572aefc..f09ed908422b 100644
> --- a/tools/testing/selftests/kvm/.gitignore
> +++ b/tools/testing/selftests/kvm/.gitignore
> @@ -1,4 +1,5 @@
>  # SPDX-License-Identifier: GPL-2.0-only
> +/aarch64/debug-exceptions
>  /aarch64/get-reg-list
>  /aarch64/get-reg-list-sve
>  /aarch64/vgic_init
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 618c5903f478..2f92442c0cc9 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -73,6 +73,7 @@ TEST_GEN_PROGS_x86_64 += memslot_modification_stress_test
>  TEST_GEN_PROGS_x86_64 += set_memory_region_test
>  TEST_GEN_PROGS_x86_64 += steal_time
>  
> +TEST_GEN_PROGS_aarch64 += aarch64/debug-exceptions
>  TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list
>  TEST_GEN_PROGS_aarch64 += aarch64/get-reg-list-sve
>  TEST_GEN_PROGS_aarch64 += aarch64/vgic_init
> diff --git a/tools/testing/selftests/kvm/aarch64/debug-exceptions.c b/tools/testing/selftests/kvm/aarch64/debug-exceptions.c
> new file mode 100644
> index 000000000000..87352ee09211
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/aarch64/debug-exceptions.c
> @@ -0,0 +1,244 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <test_util.h>
> +#include <kvm_util.h>
> +#include <processor.h>
> +
> +#define VCPU_ID 0
> +
> +#define MDSCR_KDE	(1 << 13)
> +#define MDSCR_MDE	(1 << 15)
> +#define MDSCR_SS	(1 << 0)
> +
> +#define DBGBCR_LEN8	(0xff << 5)
> +#define DBGBCR_EXEC	(0x0 << 3)
> +#define DBGBCR_EL1	(0x1 << 1)
> +#define DBGBCR_E	(0x1 << 0)
> +
> +#define DBGWCR_LEN8	(0xff << 5)
> +#define DBGWCR_RD	(0x1 << 3)
> +#define DBGWCR_WR	(0x2 << 3)
> +#define DBGWCR_EL1	(0x1 << 1)
> +#define DBGWCR_E	(0x1 << 0)
> +
> +extern unsigned char sw_bp, hw_bp, bp_svc, bp_brk, hw_wp, ss_start;
> +static volatile uint64_t sw_bp_addr, hw_bp_addr;
> +static volatile uint64_t wp_addr, wp_data_addr;
> +static volatile uint64_t svc_addr;
> +static volatile uint64_t ss_addr[4], ss_idx;
> +#define  PC(v)  ((uint64_t)&(v))
> +
> +static void reset_debug_state(void)
> +{
> +	asm volatile("msr daifset, #8");
> +
> +	write_sysreg(osdlr_el1, 0);
> +	write_sysreg(oslar_el1, 0);
> +	asm volatile("isb" : : : "memory");

We may want to create an isb() macro in include/aarch64/processor.h

> +
> +	write_sysreg(mdscr_el1, 0);
> +	/* This test only uses the first bp and wp slot. */
> +	write_sysreg(dbgbvr0_el1, 0);
> +	write_sysreg(dbgbcr0_el1, 0);
> +	write_sysreg(dbgwcr0_el1, 0);
> +	write_sysreg(dbgwvr0_el1, 0);
> +	asm volatile("isb" : : : "memory");
> +}
> +
> +static void install_wp(uint64_t addr)
> +{
> +	uint32_t wcr;
> +	uint32_t mdscr;
> +
> +	wcr = DBGWCR_LEN8 | DBGWCR_RD | DBGWCR_WR | DBGWCR_EL1 | DBGWCR_E;
> +	write_sysreg(dbgwcr0_el1, wcr);
> +	write_sysreg(dbgwvr0_el1, addr);
> +	asm volatile("isb" : : : "memory");
> +
> +	asm volatile("msr daifclr, #8");
> +
> +	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_MDE;
> +	write_sysreg(mdscr_el1, mdscr);

no isb?

> +}
> +
> +static void install_hw_bp(uint64_t addr)
> +{
> +	uint32_t bcr;
> +	uint32_t mdscr;
> +
> +	bcr = DBGBCR_LEN8 | DBGBCR_EXEC | DBGBCR_EL1 | DBGBCR_E;
> +	write_sysreg(dbgbcr0_el1, bcr);
> +	write_sysreg(dbgbvr0_el1, addr);
> +	asm volatile("isb" : : : "memory");
> +
> +	asm volatile("msr daifclr, #8");
> +
> +	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_MDE;
> +	write_sysreg(mdscr_el1, mdscr);

no isb?

> +}
> +
> +static void install_ss(void)
> +{
> +	uint32_t mdscr;
> +
> +	asm volatile("msr daifclr, #8");
> +
> +	mdscr = read_sysreg(mdscr_el1) | MDSCR_KDE | MDSCR_SS;
> +	write_sysreg(mdscr_el1, mdscr);

no isb?

> +}
> +
> +static volatile char write_data;
> +
> +static void guest_code(void)
> +{
> +	GUEST_SYNC(0);
> +
> +	/* Software-breakpoint */
> +	asm volatile("sw_bp: brk #0");
> +	GUEST_ASSERT_EQ(sw_bp_addr, PC(sw_bp));
> +
> +	GUEST_SYNC(1);
> +
> +	/* Hardware-breakpoint */
> +	reset_debug_state();
> +	install_hw_bp(PC(hw_bp));
> +	asm volatile("hw_bp: nop");
> +	GUEST_ASSERT_EQ(hw_bp_addr, PC(hw_bp));
> +
> +	GUEST_SYNC(2);
> +
> +	/* Hardware-breakpoint + svc */
> +	reset_debug_state();
> +	install_hw_bp(PC(bp_svc));
> +	asm volatile("bp_svc: svc #0");
> +	GUEST_ASSERT_EQ(hw_bp_addr, PC(bp_svc));
> +	GUEST_ASSERT_EQ(svc_addr, PC(bp_svc) + 4);
> +
> +	GUEST_SYNC(3);
> +
> +	/* Hardware-breakpoint + software-breakpoint */
> +	reset_debug_state();
> +	install_hw_bp(PC(bp_brk));
> +	asm volatile("bp_brk: brk #0");
> +	GUEST_ASSERT_EQ(sw_bp_addr, PC(bp_brk));
> +	GUEST_ASSERT_EQ(hw_bp_addr, PC(bp_brk));
> +
> +	GUEST_SYNC(4);
> +
> +	/* Watchpoint */
> +	reset_debug_state();
> +	install_wp(PC(write_data));
> +	write_data = 'x';
> +	GUEST_ASSERT_EQ(write_data, 'x');
> +	GUEST_ASSERT_EQ(wp_data_addr, PC(write_data));
> +
> +	GUEST_SYNC(5);
> +
> +	/* Single-step */
> +	reset_debug_state();
> +	install_ss();
> +	ss_idx = 0;
> +	asm volatile("ss_start:\n"
> +		     "mrs x0, esr_el1\n"
> +		     "add x0, x0, #1\n"
> +		     "msr daifset, #8\n"
> +		     : : : "x0");
> +	GUEST_ASSERT_EQ(ss_addr[0], PC(ss_start));
> +	GUEST_ASSERT_EQ(ss_addr[1], PC(ss_start) + 4);
> +	GUEST_ASSERT_EQ(ss_addr[2], PC(ss_start) + 8);
> +
> +	GUEST_DONE();
> +}
> +
> +static void guest_sw_bp_handler(struct ex_regs *regs)
> +{
> +	sw_bp_addr = regs->pc;
> +	regs->pc += 4;
> +}
> +
> +static void guest_hw_bp_handler(struct ex_regs *regs)
> +{
> +	hw_bp_addr = regs->pc;
> +	regs->pstate |= SPSR_D;
> +}
> +
> +static void guest_wp_handler(struct ex_regs *regs)
> +{
> +	wp_data_addr = read_sysreg(far_el1);
> +	wp_addr = regs->pc;
> +	regs->pstate |= SPSR_D;
> +}
> +
> +static void guest_ss_handler(struct ex_regs *regs)
> +{
> +	GUEST_ASSERT_1(ss_idx < 4, ss_idx);
> +	ss_addr[ss_idx++] = regs->pc;
> +	regs->pstate |= SPSR_SS;
> +}
> +
> +static void guest_svc_handler(struct ex_regs *regs)
> +{
> +	svc_addr = regs->pc;
> +}
> +
> +static int debug_version(struct kvm_vm *vm)
> +{
> +	uint64_t id_aa64dfr0;
> +
> +	get_reg(vm, VCPU_ID, ARM64_SYS_REG(ID_AA64DFR0_EL1), &id_aa64dfr0);
> +	return id_aa64dfr0 & 0xf;
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	struct kvm_vm *vm;
> +	struct ucall uc;
> +	int stage;
> +
> +	vm = vm_create_default(VCPU_ID, 0, guest_code);
> +	ucall_init(vm, NULL);
> +
> +	vm_init_descriptor_tables(vm);
> +	vcpu_init_descriptor_tables(vm, VCPU_ID);
> +
> +	if (debug_version(vm) < 6) {
> +		print_skip("Armv8 debug architecture not supported.");
> +		kvm_vm_free(vm);
> +		exit(KSFT_SKIP);
> +	}
> +
> +	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
> +				ESR_EC_BRK_INS, guest_sw_bp_handler);
> +	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
> +				ESR_EC_HW_BP_CURRENT, guest_hw_bp_handler);
> +	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
> +				ESR_EC_WP_CURRENT, guest_wp_handler);
> +	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
> +				ESR_EC_SSTEP_CURRENT, guest_ss_handler);
> +	vm_install_exception_handler(vm, VECTOR_SYNC_CURRENT,
> +				ESR_EC_SVC64, guest_svc_handler);
> +
> +	for (stage = 0; stage < 7; stage++) {
> +		vcpu_run(vm, VCPU_ID);
> +
> +		switch (get_ucall(vm, VCPU_ID, &uc)) {
> +		case UCALL_SYNC:
> +			TEST_ASSERT(uc.args[1] == stage,
> +				"Stage %d: Unexpected sync ucall, got %lx",
> +				stage, (ulong)uc.args[1]);
> +			break;
> +		case UCALL_ABORT:
> +			TEST_FAIL("%s at %s:%ld\n\tvalues: %#lx, %#lx",
> +				(const char *)uc.args[0],
> +				__FILE__, uc.args[1], uc.args[2], uc.args[3]);
> +			break;
> +		case UCALL_DONE:
> +			goto done;
> +		default:
> +			TEST_FAIL("Unknown ucall %lu", uc.cmd);
> +		}
> +	}
> +
> +done:
> +	kvm_vm_free(vm);
> +	return 0;
> +}
> diff --git a/tools/testing/selftests/kvm/include/aarch64/processor.h b/tools/testing/selftests/kvm/include/aarch64/processor.h
> index 40aae31b4afc..a3ebef8e88c7 100644
> --- a/tools/testing/selftests/kvm/include/aarch64/processor.h
> +++ b/tools/testing/selftests/kvm/include/aarch64/processor.h
> @@ -14,12 +14,14 @@
>  #define ARM64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \
>  			   KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x))
>  
> -#define CPACR_EL1	3, 0,  1, 0, 2
> -#define TCR_EL1		3, 0,  2, 0, 2
> -#define MAIR_EL1	3, 0, 10, 2, 0
> -#define TTBR0_EL1	3, 0,  2, 0, 0
> -#define SCTLR_EL1	3, 0,  1, 0, 0
> -#define VBAR_EL1	3, 0, 12, 0, 0
> +#define CPACR_EL1               3, 0,  1, 0, 2
> +#define TCR_EL1                 3, 0,  2, 0, 2
> +#define MAIR_EL1                3, 0, 10, 2, 0
> +#define TTBR0_EL1               3, 0,  2, 0, 0
> +#define SCTLR_EL1               3, 0,  1, 0, 0
> +#define VBAR_EL1                3, 0, 12, 0, 0
> +
> +#define ID_AA64DFR0_EL1         3, 0,  0, 5, 0
>  
>  /*
>   * Default MAIR
> -- 
> 2.31.1.527.g47e6f16901-goog
>

Other than the potentially missing isb's it looks good to me.

Reviewed-by: Andrew Jones <drjones@redhat.com>

Thanks,
drew

Re: [PATCH v2 0/5] KVM: selftests: arm64 exception handling and debug test

[Paolo Bonzini]

On 01/05/21 01:24, Ricardo Koller wrote:
> Hi,
> 
> These patches add a debug exception test in aarch64 KVM selftests while
> also adding basic exception handling support.
> 
> The structure of the exception handling is based on its x86 counterpart.
> Tests use the same calls to initialize exception handling and both
> architectures allow tests to override the handler for a particular
> vector, or (vector, ec) for synchronous exceptions in the arm64 case.
> 
> The debug test is similar to x86_64/debug_regs, except that the x86 one
> controls the debugging from outside the VM. This proposed arm64 test
> controls and handles debug exceptions from the inside.
> 
> Thanks,
> Ricardo

Marc, are you going to queue this in your tree?

Thanks,

Paolo

> v1 -> v2:
> 
> Addressed comments from Andrew and Marc (thank you very much):
> - rename vm_handle_exception in all tests.
> - introduce UCALL_UNHANDLED in x86 first.
> - move GUEST_ASSERT_EQ to common utils header.
> - handle sync and other exceptions separately: use two tables (like
>    kvm-unit-tests).
> - add two separate functions for installing sync versus other exceptions
> - changes in handlers.S: use the same layout as user_pt_regs, treat the
>    EL1t vectors as invalid, refactor the vector table creation to not use
>    manual numbering, add comments, remove LR from the stored registers.
> - changes in debug-exceptions.c: remove unused headers, use the common
>    GUEST_ASSERT_EQ, use vcpu_run instead of _vcpu_run.
> - changes in processor.h: write_sysreg with support for xzr, replace EL1
>    with current in macro names, define ESR_EC_MASK as ESR_EC_NUM-1.
> 
> Ricardo Koller (5):
>    KVM: selftests: Rename vm_handle_exception
>    KVM: selftests: Introduce UCALL_UNHANDLED for unhandled vector
>      reporting
>    KVM: selftests: Move GUEST_ASSERT_EQ to utils header
>    KVM: selftests: Add exception handling support for aarch64
>    KVM: selftests: Add aarch64/debug-exceptions test
> 
>   tools/testing/selftests/kvm/.gitignore        |   1 +
>   tools/testing/selftests/kvm/Makefile          |   3 +-
>   .../selftests/kvm/aarch64/debug-exceptions.c  | 244 ++++++++++++++++++
>   .../selftests/kvm/include/aarch64/processor.h |  90 ++++++-
>   .../testing/selftests/kvm/include/kvm_util.h  |  10 +
>   .../selftests/kvm/include/x86_64/processor.h  |   4 +-
>   .../selftests/kvm/lib/aarch64/handlers.S      | 130 ++++++++++
>   .../selftests/kvm/lib/aarch64/processor.c     | 124 +++++++++
>   .../selftests/kvm/lib/x86_64/processor.c      |  19 +-
>   .../selftests/kvm/x86_64/kvm_pv_test.c        |   2 +-
>   .../selftests/kvm/x86_64/tsc_msrs_test.c      |   9 -
>   .../kvm/x86_64/userspace_msr_exit_test.c      |   8 +-
>   .../selftests/kvm/x86_64/xapic_ipi_test.c     |   2 +-
>   13 files changed, 611 insertions(+), 35 deletions(-)
>   create mode 100644 tools/testing/selftests/kvm/aarch64/debug-exceptions.c
>   create mode 100644 tools/testing/selftests/kvm/lib/aarch64/handlers.S
> 

Re: [PATCH v2 0/5] KVM: selftests: arm64 exception handling and debug test

[Marc Zyngier]

On Mon, 24 May 2021 13:14:52 +0100,
Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
> On 01/05/21 01:24, Ricardo Koller wrote:
> > Hi,
> > 
> > These patches add a debug exception test in aarch64 KVM selftests while
> > also adding basic exception handling support.
> > 
> > The structure of the exception handling is based on its x86 counterpart.
> > Tests use the same calls to initialize exception handling and both
> > architectures allow tests to override the handler for a particular
> > vector, or (vector, ec) for synchronous exceptions in the arm64 case.
> > 
> > The debug test is similar to x86_64/debug_regs, except that the x86 one
> > controls the debugging from outside the VM. This proposed arm64 test
> > controls and handles debug exceptions from the inside.
> > 
> > Thanks,
> > Ricardo
> 
> Marc, are you going to queue this in your tree?

That is my plan. I'm waiting for the fixes branch to be merged in
Linus' tree to start queuing stuff on top.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.