[PATCH v4 0/3] selftests: KVM: AMD Nested SVM test infrastructure

[PATCH v4 0/3] selftests: KVM: AMD Nested SVM test infrastructure

[Eric Auger]

Add the basic infrastructure needed to test AMD nested SVM.
Also add a first basic vmcall test.

Best regards

Eric

This series can be found at:
https://github.com/eauger/linux/tree/v5.5-amd-svm-v4

History:
v3 -> v4:
- gpr64_regs struct just contains 64b mode GPRs ordered
  as in x86_register
- cleanup in run_guest (vm* instructions) and reduce
  clubber list.
- add some comments

v2 -> v3:
- Took into account Vitaly's comment:
  - added "selftests: KVM: Replace get_gdt/idt_base() by
    get_gdt/idt()"
  - svm.h now is a copy of arch/x86/include/asm/svm.h
  - avoid duplicates

v1 -> v2:
- split into 2 patches
- remove the infrastructure to run low-level sub-tests and only
  keep vmmcall's one.
- move struct regs into processor.h
- force vmcb_gpa into rax in run_guest()

Eric Auger (3):
  selftests: KVM: Replace get_gdt/idt_base() by get_gdt/idt()
  selftests: KVM: AMD Nested test infrastructure
  selftests: KVM: SVM: Add vmcall test

 tools/testing/selftests/kvm/Makefile          |   3 +-
 .../selftests/kvm/include/x86_64/processor.h  |  28 +-
 .../selftests/kvm/include/x86_64/svm.h        | 297 ++++++++++++++++++
 .../selftests/kvm/include/x86_64/svm_util.h   |  38 +++
 tools/testing/selftests/kvm/lib/x86_64/svm.c  | 161 ++++++++++
 tools/testing/selftests/kvm/lib/x86_64/vmx.c  |   6 +-
 .../selftests/kvm/x86_64/svm_vmcall_test.c    |  79 +++++
 7 files changed, 604 insertions(+), 8 deletions(-)
 create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm.h
 create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm_util.h
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/svm.c
 create mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c

-- 
2.20.1

[PATCH v4 1/3] selftests: KVM: Replace get_gdt/idt_base() by get_gdt/idt()

[Eric Auger]

get_gdt_base() and get_idt_base() only return the base address
of the descriptor tables. Soon we will need to get the size as well.
Change the prototype of those functions so that they return
the whole desc_ptr struct instead of the address field.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>

---

v3 -> v4:
- Collected R-b's
---
 tools/testing/selftests/kvm/include/x86_64/processor.h | 8 ++++----
 tools/testing/selftests/kvm/lib/x86_64/vmx.c           | 6 +++---
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
index aa6451b3f740..6f7fffaea2e8 100644
--- a/tools/testing/selftests/kvm/include/x86_64/processor.h
+++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
@@ -220,20 +220,20 @@ static inline void set_cr4(uint64_t val)
 	__asm__ __volatile__("mov %0, %%cr4" : : "r" (val) : "memory");
 }
 
-static inline uint64_t get_gdt_base(void)
+static inline struct desc_ptr get_gdt(void)
 {
 	struct desc_ptr gdt;
 	__asm__ __volatile__("sgdt %[gdt]"
 			     : /* output */ [gdt]"=m"(gdt));
-	return gdt.address;
+	return gdt;
 }
 
-static inline uint64_t get_idt_base(void)
+static inline struct desc_ptr get_idt(void)
 {
 	struct desc_ptr idt;
 	__asm__ __volatile__("sidt %[idt]"
 			     : /* output */ [idt]"=m"(idt));
-	return idt.address;
+	return idt;
 }
 
 #define SET_XMM(__var, __xmm) \
diff --git a/tools/testing/selftests/kvm/lib/x86_64/vmx.c b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
index 85064baf5e97..7aaa99ca4dbc 100644
--- a/tools/testing/selftests/kvm/lib/x86_64/vmx.c
+++ b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
@@ -288,9 +288,9 @@ static inline void init_vmcs_host_state(void)
 	vmwrite(HOST_FS_BASE, rdmsr(MSR_FS_BASE));
 	vmwrite(HOST_GS_BASE, rdmsr(MSR_GS_BASE));
 	vmwrite(HOST_TR_BASE,
-		get_desc64_base((struct desc64 *)(get_gdt_base() + get_tr())));
-	vmwrite(HOST_GDTR_BASE, get_gdt_base());
-	vmwrite(HOST_IDTR_BASE, get_idt_base());
+		get_desc64_base((struct desc64 *)(get_gdt().address + get_tr())));
+	vmwrite(HOST_GDTR_BASE, get_gdt().address);
+	vmwrite(HOST_IDTR_BASE, get_idt().address);
 	vmwrite(HOST_IA32_SYSENTER_ESP, rdmsr(MSR_IA32_SYSENTER_ESP));
 	vmwrite(HOST_IA32_SYSENTER_EIP, rdmsr(MSR_IA32_SYSENTER_EIP));
 }
-- 
2.20.1

[PATCH v4 2/3] selftests: KVM: AMD Nested test infrastructure

[Eric Auger]

Add the basic infrastructure needed to test AMD nested SVM.
This is largely copied from the KVM unit test infrastructure.

Signed-off-by: Eric Auger <eric.auger@redhat.com>

---

v3 -> v4:
- just keep the 16 GPRs in gpr64_regs struct
- vm* instructions do not take any param
- add comments

v2 -> v3:
- s/regs/gp_regs64
- Split the header into 2 parts: svm.h is a copy of
  arch/x86/include/asm/svm.h whereas svm_util.h contains
  testing add-ons
- use get_gdt/dt() and remove sgdt/sidt
- use get_es/ss/ds/cs
- fix clobber for dr6 & dr7
- use u64 instead of ulong
---
 tools/testing/selftests/kvm/Makefile          |   2 +-
 .../selftests/kvm/include/x86_64/processor.h  |  20 ++
 .../selftests/kvm/include/x86_64/svm.h        | 297 ++++++++++++++++++
 .../selftests/kvm/include/x86_64/svm_util.h   |  38 +++
 tools/testing/selftests/kvm/lib/x86_64/svm.c  | 161 ++++++++++
 5 files changed, 517 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm.h
 create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm_util.h
 create mode 100644 tools/testing/selftests/kvm/lib/x86_64/svm.c

diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 608fa835c764..2e770f554cae 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -8,7 +8,7 @@ KSFT_KHDR_INSTALL := 1
 UNAME_M := $(shell uname -m)
 
 LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c
-LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/ucall.c
+LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c
 LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
 LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c
 
diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
index 6f7fffaea2e8..12475047869f 100644
--- a/tools/testing/selftests/kvm/include/x86_64/processor.h
+++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
@@ -56,6 +56,26 @@ enum x86_register {
 	R15,
 };
 
+/* General Registers in 64-Bit Mode */
+struct gpr64_regs {
+	u64 rax;
+	u64 rcx;
+	u64 rdx;
+	u64 rbx;
+	u64 rsp;
+	u64 rbp;
+	u64 rsi;
+	u64 rdi;
+	u64 r8;
+	u64 r9;
+	u64 r10;
+	u64 r11;
+	u64 r12;
+	u64 r13;
+	u64 r14;
+	u64 r15;
+};
+
 struct desc64 {
 	uint16_t limit0;
 	uint16_t base0;
diff --git a/tools/testing/selftests/kvm/include/x86_64/svm.h b/tools/testing/selftests/kvm/include/x86_64/svm.h
new file mode 100644
index 000000000000..f4ea2355dbc2
--- /dev/null
+++ b/tools/testing/selftests/kvm/include/x86_64/svm.h
@@ -0,0 +1,297 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * tools/testing/selftests/kvm/include/x86_64/svm.h
+ * This is a copy of arch/x86/include/asm/svm.h
+ *
+ */
+
+#ifndef SELFTEST_KVM_SVM_H
+#define SELFTEST_KVM_SVM_H
+
+enum {
+	INTERCEPT_INTR,
+	INTERCEPT_NMI,
+	INTERCEPT_SMI,
+	INTERCEPT_INIT,
+	INTERCEPT_VINTR,
+	INTERCEPT_SELECTIVE_CR0,
+	INTERCEPT_STORE_IDTR,
+	INTERCEPT_STORE_GDTR,
+	INTERCEPT_STORE_LDTR,
+	INTERCEPT_STORE_TR,
+	INTERCEPT_LOAD_IDTR,
+	INTERCEPT_LOAD_GDTR,
+	INTERCEPT_LOAD_LDTR,
+	INTERCEPT_LOAD_TR,
+	INTERCEPT_RDTSC,
+	INTERCEPT_RDPMC,
+	INTERCEPT_PUSHF,
+	INTERCEPT_POPF,
+	INTERCEPT_CPUID,
+	INTERCEPT_RSM,
+	INTERCEPT_IRET,
+	INTERCEPT_INTn,
+	INTERCEPT_INVD,
+	INTERCEPT_PAUSE,
+	INTERCEPT_HLT,
+	INTERCEPT_INVLPG,
+	INTERCEPT_INVLPGA,
+	INTERCEPT_IOIO_PROT,
+	INTERCEPT_MSR_PROT,
+	INTERCEPT_TASK_SWITCH,
+	INTERCEPT_FERR_FREEZE,
+	INTERCEPT_SHUTDOWN,
+	INTERCEPT_VMRUN,
+	INTERCEPT_VMMCALL,
+	INTERCEPT_VMLOAD,
+	INTERCEPT_VMSAVE,
+	INTERCEPT_STGI,
+	INTERCEPT_CLGI,
+	INTERCEPT_SKINIT,
+	INTERCEPT_RDTSCP,
+	INTERCEPT_ICEBP,
+	INTERCEPT_WBINVD,
+	INTERCEPT_MONITOR,
+	INTERCEPT_MWAIT,
+	INTERCEPT_MWAIT_COND,
+	INTERCEPT_XSETBV,
+	INTERCEPT_RDPRU,
+};
+
+
+struct __attribute__ ((__packed__)) vmcb_control_area {
+	u32 intercept_cr;
+	u32 intercept_dr;
+	u32 intercept_exceptions;
+	u64 intercept;
+	u8 reserved_1[40];
+	u16 pause_filter_thresh;
+	u16 pause_filter_count;
+	u64 iopm_base_pa;
+	u64 msrpm_base_pa;
+	u64 tsc_offset;
+	u32 asid;
+	u8 tlb_ctl;
+	u8 reserved_2[3];
+	u32 int_ctl;
+	u32 int_vector;
+	u32 int_state;
+	u8 reserved_3[4];
+	u32 exit_code;
+	u32 exit_code_hi;
+	u64 exit_info_1;
+	u64 exit_info_2;
+	u32 exit_int_info;
+	u32 exit_int_info_err;
+	u64 nested_ctl;
+	u64 avic_vapic_bar;
+	u8 reserved_4[8];
+	u32 event_inj;
+	u32 event_inj_err;
+	u64 nested_cr3;
+	u64 virt_ext;
+	u32 clean;
+	u32 reserved_5;
+	u64 next_rip;
+	u8 insn_len;
+	u8 insn_bytes[15];
+	u64 avic_backing_page;	/* Offset 0xe0 */
+	u8 reserved_6[8];	/* Offset 0xe8 */
+	u64 avic_logical_id;	/* Offset 0xf0 */
+	u64 avic_physical_id;	/* Offset 0xf8 */
+	u8 reserved_7[768];
+};
+
+
+#define TLB_CONTROL_DO_NOTHING 0
+#define TLB_CONTROL_FLUSH_ALL_ASID 1
+#define TLB_CONTROL_FLUSH_ASID 3
+#define TLB_CONTROL_FLUSH_ASID_LOCAL 7
+
+#define V_TPR_MASK 0x0f
+
+#define V_IRQ_SHIFT 8
+#define V_IRQ_MASK (1 << V_IRQ_SHIFT)
+
+#define V_GIF_SHIFT 9
+#define V_GIF_MASK (1 << V_GIF_SHIFT)
+
+#define V_INTR_PRIO_SHIFT 16
+#define V_INTR_PRIO_MASK (0x0f << V_INTR_PRIO_SHIFT)
+
+#define V_IGN_TPR_SHIFT 20
+#define V_IGN_TPR_MASK (1 << V_IGN_TPR_SHIFT)
+
+#define V_INTR_MASKING_SHIFT 24
+#define V_INTR_MASKING_MASK (1 << V_INTR_MASKING_SHIFT)
+
+#define V_GIF_ENABLE_SHIFT 25
+#define V_GIF_ENABLE_MASK (1 << V_GIF_ENABLE_SHIFT)
+
+#define AVIC_ENABLE_SHIFT 31
+#define AVIC_ENABLE_MASK (1 << AVIC_ENABLE_SHIFT)
+
+#define LBR_CTL_ENABLE_MASK BIT_ULL(0)
+#define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1)
+
+#define SVM_INTERRUPT_SHADOW_MASK 1
+
+#define SVM_IOIO_STR_SHIFT 2
+#define SVM_IOIO_REP_SHIFT 3
+#define SVM_IOIO_SIZE_SHIFT 4
+#define SVM_IOIO_ASIZE_SHIFT 7
+
+#define SVM_IOIO_TYPE_MASK 1
+#define SVM_IOIO_STR_MASK (1 << SVM_IOIO_STR_SHIFT)
+#define SVM_IOIO_REP_MASK (1 << SVM_IOIO_REP_SHIFT)
+#define SVM_IOIO_SIZE_MASK (7 << SVM_IOIO_SIZE_SHIFT)
+#define SVM_IOIO_ASIZE_MASK (7 << SVM_IOIO_ASIZE_SHIFT)
+
+#define SVM_VM_CR_VALID_MASK	0x001fULL
+#define SVM_VM_CR_SVM_LOCK_MASK 0x0008ULL
+#define SVM_VM_CR_SVM_DIS_MASK  0x0010ULL
+
+#define SVM_NESTED_CTL_NP_ENABLE	BIT(0)
+#define SVM_NESTED_CTL_SEV_ENABLE	BIT(1)
+
+struct __attribute__ ((__packed__)) vmcb_seg {
+	u16 selector;
+	u16 attrib;
+	u32 limit;
+	u64 base;
+};
+
+struct __attribute__ ((__packed__)) vmcb_save_area {
+	struct vmcb_seg es;
+	struct vmcb_seg cs;
+	struct vmcb_seg ss;
+	struct vmcb_seg ds;
+	struct vmcb_seg fs;
+	struct vmcb_seg gs;
+	struct vmcb_seg gdtr;
+	struct vmcb_seg ldtr;
+	struct vmcb_seg idtr;
+	struct vmcb_seg tr;
+	u8 reserved_1[43];
+	u8 cpl;
+	u8 reserved_2[4];
+	u64 efer;
+	u8 reserved_3[112];
+	u64 cr4;
+	u64 cr3;
+	u64 cr0;
+	u64 dr7;
+	u64 dr6;
+	u64 rflags;
+	u64 rip;
+	u8 reserved_4[88];
+	u64 rsp;
+	u8 reserved_5[24];
+	u64 rax;
+	u64 star;
+	u64 lstar;
+	u64 cstar;
+	u64 sfmask;
+	u64 kernel_gs_base;
+	u64 sysenter_cs;
+	u64 sysenter_esp;
+	u64 sysenter_eip;
+	u64 cr2;
+	u8 reserved_6[32];
+	u64 g_pat;
+	u64 dbgctl;
+	u64 br_from;
+	u64 br_to;
+	u64 last_excp_from;
+	u64 last_excp_to;
+};
+
+struct __attribute__ ((__packed__)) vmcb {
+	struct vmcb_control_area control;
+	struct vmcb_save_area save;
+};
+
+#define SVM_CPUID_FUNC 0x8000000a
+
+#define SVM_VM_CR_SVM_DISABLE 4
+
+#define SVM_SELECTOR_S_SHIFT 4
+#define SVM_SELECTOR_DPL_SHIFT 5
+#define SVM_SELECTOR_P_SHIFT 7
+#define SVM_SELECTOR_AVL_SHIFT 8
+#define SVM_SELECTOR_L_SHIFT 9
+#define SVM_SELECTOR_DB_SHIFT 10
+#define SVM_SELECTOR_G_SHIFT 11
+
+#define SVM_SELECTOR_TYPE_MASK (0xf)
+#define SVM_SELECTOR_S_MASK (1 << SVM_SELECTOR_S_SHIFT)
+#define SVM_SELECTOR_DPL_MASK (3 << SVM_SELECTOR_DPL_SHIFT)
+#define SVM_SELECTOR_P_MASK (1 << SVM_SELECTOR_P_SHIFT)
+#define SVM_SELECTOR_AVL_MASK (1 << SVM_SELECTOR_AVL_SHIFT)
+#define SVM_SELECTOR_L_MASK (1 << SVM_SELECTOR_L_SHIFT)
+#define SVM_SELECTOR_DB_MASK (1 << SVM_SELECTOR_DB_SHIFT)
+#define SVM_SELECTOR_G_MASK (1 << SVM_SELECTOR_G_SHIFT)
+
+#define SVM_SELECTOR_WRITE_MASK (1 << 1)
+#define SVM_SELECTOR_READ_MASK SVM_SELECTOR_WRITE_MASK
+#define SVM_SELECTOR_CODE_MASK (1 << 3)
+
+#define INTERCEPT_CR0_READ	0
+#define INTERCEPT_CR3_READ	3
+#define INTERCEPT_CR4_READ	4
+#define INTERCEPT_CR8_READ	8
+#define INTERCEPT_CR0_WRITE	(16 + 0)
+#define INTERCEPT_CR3_WRITE	(16 + 3)
+#define INTERCEPT_CR4_WRITE	(16 + 4)
+#define INTERCEPT_CR8_WRITE	(16 + 8)
+
+#define INTERCEPT_DR0_READ	0
+#define INTERCEPT_DR1_READ	1
+#define INTERCEPT_DR2_READ	2
+#define INTERCEPT_DR3_READ	3
+#define INTERCEPT_DR4_READ	4
+#define INTERCEPT_DR5_READ	5
+#define INTERCEPT_DR6_READ	6
+#define INTERCEPT_DR7_READ	7
+#define INTERCEPT_DR0_WRITE	(16 + 0)
+#define INTERCEPT_DR1_WRITE	(16 + 1)
+#define INTERCEPT_DR2_WRITE	(16 + 2)
+#define INTERCEPT_DR3_WRITE	(16 + 3)
+#define INTERCEPT_DR4_WRITE	(16 + 4)
+#define INTERCEPT_DR5_WRITE	(16 + 5)
+#define INTERCEPT_DR6_WRITE	(16 + 6)
+#define INTERCEPT_DR7_WRITE	(16 + 7)
+
+#define SVM_EVTINJ_VEC_MASK 0xff
+
+#define SVM_EVTINJ_TYPE_SHIFT 8
+#define SVM_EVTINJ_TYPE_MASK (7 << SVM_EVTINJ_TYPE_SHIFT)
+
+#define SVM_EVTINJ_TYPE_INTR (0 << SVM_EVTINJ_TYPE_SHIFT)
+#define SVM_EVTINJ_TYPE_NMI (2 << SVM_EVTINJ_TYPE_SHIFT)
+#define SVM_EVTINJ_TYPE_EXEPT (3 << SVM_EVTINJ_TYPE_SHIFT)
+#define SVM_EVTINJ_TYPE_SOFT (4 << SVM_EVTINJ_TYPE_SHIFT)
+
+#define SVM_EVTINJ_VALID (1 << 31)
+#define SVM_EVTINJ_VALID_ERR (1 << 11)
+
+#define SVM_EXITINTINFO_VEC_MASK SVM_EVTINJ_VEC_MASK
+#define SVM_EXITINTINFO_TYPE_MASK SVM_EVTINJ_TYPE_MASK
+
+#define	SVM_EXITINTINFO_TYPE_INTR SVM_EVTINJ_TYPE_INTR
+#define	SVM_EXITINTINFO_TYPE_NMI SVM_EVTINJ_TYPE_NMI
+#define	SVM_EXITINTINFO_TYPE_EXEPT SVM_EVTINJ_TYPE_EXEPT
+#define	SVM_EXITINTINFO_TYPE_SOFT SVM_EVTINJ_TYPE_SOFT
+
+#define SVM_EXITINTINFO_VALID SVM_EVTINJ_VALID
+#define SVM_EXITINTINFO_VALID_ERR SVM_EVTINJ_VALID_ERR
+
+#define SVM_EXITINFOSHIFT_TS_REASON_IRET 36
+#define SVM_EXITINFOSHIFT_TS_REASON_JMP 38
+#define SVM_EXITINFOSHIFT_TS_HAS_ERROR_CODE 44
+
+#define SVM_EXITINFO_REG_MASK 0x0F
+
+#define SVM_CR0_SELECTIVE_MASK (X86_CR0_TS | X86_CR0_MP)
+
+#endif /* SELFTEST_KVM_SVM_H */
diff --git a/tools/testing/selftests/kvm/include/x86_64/svm_util.h b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
new file mode 100644
index 000000000000..cd037917fece
--- /dev/null
+++ b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
@@ -0,0 +1,38 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * tools/testing/selftests/kvm/include/x86_64/svm_utils.h
+ * Header for nested SVM testing
+ *
+ * Copyright (C) 2020, Red Hat, Inc.
+ */
+
+#ifndef SELFTEST_KVM_SVM_UTILS_H
+#define SELFTEST_KVM_SVM_UTILS_H
+
+#include <stdint.h>
+#include "svm.h"
+#include "processor.h"
+
+#define CPUID_SVM_BIT		2
+#define CPUID_SVM		BIT_ULL(CPUID_SVM_BIT)
+
+#define SVM_EXIT_VMMCALL	0x081
+
+struct svm_test_data {
+	/* VMCB */
+	struct vmcb *vmcb; /* gva */
+	void *vmcb_hva;
+	uint64_t vmcb_gpa;
+
+	/* host state-save area */
+	struct vmcb_save_area *save_area; /* gva */
+	void *save_area_hva;
+	uint64_t save_area_gpa;
+};
+
+struct svm_test_data *vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva);
+void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp);
+void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa);
+void nested_svm_check_supported(void);
+
+#endif /* SELFTEST_KVM_SVM_UTILS_H */
diff --git a/tools/testing/selftests/kvm/lib/x86_64/svm.c b/tools/testing/selftests/kvm/lib/x86_64/svm.c
new file mode 100644
index 000000000000..6e05a8fc3fe0
--- /dev/null
+++ b/tools/testing/selftests/kvm/lib/x86_64/svm.c
@@ -0,0 +1,161 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * tools/testing/selftests/kvm/lib/x86_64/svm.c
+ * Helpers used for nested SVM testing
+ * Largely inspired from KVM unit test svm.c
+ *
+ * Copyright (C) 2020, Red Hat, Inc.
+ */
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "../kvm_util_internal.h"
+#include "processor.h"
+#include "svm_util.h"
+
+struct gpr64_regs guest_regs;
+u64 rflags;
+
+/* Allocate memory regions for nested SVM tests.
+ *
+ * Input Args:
+ *   vm - The VM to allocate guest-virtual addresses in.
+ *
+ * Output Args:
+ *   p_svm_gva - The guest virtual address for the struct svm_test_data.
+ *
+ * Return:
+ *   Pointer to structure with the addresses of the SVM areas.
+ */
+struct svm_test_data *
+vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva)
+{
+	vm_vaddr_t svm_gva = vm_vaddr_alloc(vm, getpagesize(),
+					    0x10000, 0, 0);
+	struct svm_test_data *svm = addr_gva2hva(vm, svm_gva);
+
+	svm->vmcb = (void *)vm_vaddr_alloc(vm, getpagesize(),
+					   0x10000, 0, 0);
+	svm->vmcb_hva = addr_gva2hva(vm, (uintptr_t)svm->vmcb);
+	svm->vmcb_gpa = addr_gva2gpa(vm, (uintptr_t)svm->vmcb);
+
+	svm->save_area = (void *)vm_vaddr_alloc(vm, getpagesize(),
+						0x10000, 0, 0);
+	svm->save_area_hva = addr_gva2hva(vm, (uintptr_t)svm->save_area);
+	svm->save_area_gpa = addr_gva2gpa(vm, (uintptr_t)svm->save_area);
+
+	*p_svm_gva = svm_gva;
+	return svm;
+}
+
+static void vmcb_set_seg(struct vmcb_seg *seg, u16 selector,
+			 u64 base, u32 limit, u32 attr)
+{
+	seg->selector = selector;
+	seg->attrib = attr;
+	seg->limit = limit;
+	seg->base = base;
+}
+
+void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp)
+{
+	struct vmcb *vmcb = svm->vmcb;
+	uint64_t vmcb_gpa = svm->vmcb_gpa;
+	struct vmcb_save_area *save = &vmcb->save;
+	struct vmcb_control_area *ctrl = &vmcb->control;
+	u32 data_seg_attr = 3 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
+	      | SVM_SELECTOR_DB_MASK | SVM_SELECTOR_G_MASK;
+	u32 code_seg_attr = 9 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
+		| SVM_SELECTOR_L_MASK | SVM_SELECTOR_G_MASK;
+	uint64_t efer;
+
+	efer = rdmsr(MSR_EFER);
+	wrmsr(MSR_EFER, efer | EFER_SVME);
+	wrmsr(MSR_VM_HSAVE_PA, svm->save_area_gpa);
+
+	memset(vmcb, 0, sizeof(*vmcb));
+	asm volatile ("vmsave\n\t" : : "a" (vmcb_gpa) : "memory");
+	vmcb_set_seg(&save->es, get_es(), 0, -1U, data_seg_attr);
+	vmcb_set_seg(&save->cs, get_cs(), 0, -1U, code_seg_attr);
+	vmcb_set_seg(&save->ss, get_ss(), 0, -1U, data_seg_attr);
+	vmcb_set_seg(&save->ds, get_ds(), 0, -1U, data_seg_attr);
+	vmcb_set_seg(&save->gdtr, 0, get_gdt().address, get_gdt().size, 0);
+	vmcb_set_seg(&save->idtr, 0, get_idt().address, get_idt().size, 0);
+
+	ctrl->asid = 1;
+	save->cpl = 0;
+	save->efer = rdmsr(MSR_EFER);
+	asm volatile ("mov %%cr4, %0" : "=r"(save->cr4) : : "memory");
+	asm volatile ("mov %%cr3, %0" : "=r"(save->cr3) : : "memory");
+	asm volatile ("mov %%cr0, %0" : "=r"(save->cr0) : : "memory");
+	asm volatile ("mov %%dr7, %0" : "=r"(save->dr7) : : "memory");
+	asm volatile ("mov %%dr6, %0" : "=r"(save->dr6) : : "memory");
+	asm volatile ("mov %%cr2, %0" : "=r"(save->cr2) : : "memory");
+	save->g_pat = rdmsr(MSR_IA32_CR_PAT);
+	save->dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
+	ctrl->intercept = (1ULL << INTERCEPT_VMRUN) |
+				(1ULL << INTERCEPT_VMMCALL);
+
+	vmcb->save.rip = (u64)guest_rip;
+	vmcb->save.rsp = (u64)guest_rsp;
+	guest_regs.rdi = (u64)svm;
+}
+
+/*
+ * save/restore 64-bit general registers except rax, rip, rsp
+ * which are directly handed through the VMCB guest processor state
+ */
+#define SAVE_GPR_C				\
+	"xchg %%rbx, guest_regs+0x20\n\t"	\
+	"xchg %%rcx, guest_regs+0x10\n\t"	\
+	"xchg %%rdx, guest_regs+0x18\n\t"	\
+	"xchg %%rbp, guest_regs+0x30\n\t"	\
+	"xchg %%rsi, guest_regs+0x38\n\t"	\
+	"xchg %%rdi, guest_regs+0x40\n\t"	\
+	"xchg %%r8,  guest_regs+0x48\n\t"	\
+	"xchg %%r9,  guest_regs+0x50\n\t"	\
+	"xchg %%r10, guest_regs+0x58\n\t"	\
+	"xchg %%r11, guest_regs+0x60\n\t"	\
+	"xchg %%r12, guest_regs+0x68\n\t"	\
+	"xchg %%r13, guest_regs+0x70\n\t"	\
+	"xchg %%r14, guest_regs+0x78\n\t"	\
+	"xchg %%r15, guest_regs+0x80\n\t"
+
+#define LOAD_GPR_C      SAVE_GPR_C
+
+/*
+ * selftests do not use interrupts so we dropped clgi/sti/cli/stgi
+ * for now. registers involved in LOAD/SAVE_GPR_C are eventually
+ * unmodified so they do not need to be in the clobber list.
+ */
+void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa)
+{
+	asm volatile (
+		"vmload\n\t"
+		"mov rflags, %%r15\n\t"	// rflags
+		"mov %%r15, 0x170(%[vmcb])\n\t"
+		"mov guest_regs, %%r15\n\t"	// rax
+		"mov %%r15, 0x1f8(%[vmcb])\n\t"
+		LOAD_GPR_C
+		"vmrun\n\t"
+		SAVE_GPR_C
+		"mov 0x170(%[vmcb]), %%r15\n\t"	// rflags
+		"mov %%r15, rflags\n\t"
+		"mov 0x1f8(%[vmcb]), %%r15\n\t"	// rax
+		"mov %%r15, guest_regs\n\t"
+		"vmsave\n\t"
+		: : [vmcb] "r" (vmcb), [vmcb_gpa] "a" (vmcb_gpa)
+		: "r15", "memory");
+}
+
+void nested_svm_check_supported(void)
+{
+	struct kvm_cpuid_entry2 *entry =
+		kvm_get_supported_cpuid_entry(0x80000001);
+
+	if (!(entry->ecx & CPUID_SVM)) {
+		fprintf(stderr, "nested SVM not enabled, skipping test\n");
+		exit(KSFT_SKIP);
+	}
+}
+
-- 
2.20.1

[PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Eric Auger]

L2 guest calls vmcall and L1 checks the exit status does
correspond.

Signed-off-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>

---

v3 -> v4:
- remove useless includes
- collected Lin's R-b

v2 -> v3:
- remove useless comment and add Vitaly's R-b
---
 tools/testing/selftests/kvm/Makefile          |  1 +
 .../selftests/kvm/x86_64/svm_vmcall_test.c    | 79 +++++++++++++++++++
 2 files changed, 80 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c

diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
index 2e770f554cae..b529d3b42c02 100644
--- a/tools/testing/selftests/kvm/Makefile
+++ b/tools/testing/selftests/kvm/Makefile
@@ -26,6 +26,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test
 TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test
 TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
 TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test
+TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test
 TEST_GEN_PROGS_x86_64 += clear_dirty_log_test
 TEST_GEN_PROGS_x86_64 += dirty_log_test
 TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus
diff --git a/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
new file mode 100644
index 000000000000..6d3565aab94e
--- /dev/null
+++ b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
@@ -0,0 +1,79 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * svm_vmcall_test
+ *
+ * Copyright (C) 2020, Red Hat, Inc.
+ *
+ * Nested SVM testing: VMCALL
+ */
+
+#include "test_util.h"
+#include "kvm_util.h"
+#include "processor.h"
+#include "svm_util.h"
+
+#define VCPU_ID		5
+
+static struct kvm_vm *vm;
+
+static inline void l2_vmcall(struct svm_test_data *svm)
+{
+	__asm__ __volatile__("vmcall");
+}
+
+static void l1_guest_code(struct svm_test_data *svm)
+{
+	#define L2_GUEST_STACK_SIZE 64
+	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
+	struct vmcb *vmcb = svm->vmcb;
+
+	/* Prepare for L2 execution. */
+	generic_svm_setup(svm, l2_vmcall,
+			  &l2_guest_stack[L2_GUEST_STACK_SIZE]);
+
+	run_guest(vmcb, svm->vmcb_gpa);
+
+	GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
+	GUEST_DONE();
+}
+
+int main(int argc, char *argv[])
+{
+	vm_vaddr_t svm_gva;
+
+	nested_svm_check_supported();
+
+	vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code);
+	vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
+
+	vcpu_alloc_svm(vm, &svm_gva);
+	vcpu_args_set(vm, VCPU_ID, 1, svm_gva);
+
+	for (;;) {
+		volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
+		struct ucall uc;
+
+		vcpu_run(vm, VCPU_ID);
+		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
+			    "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
+			    run->exit_reason,
+			    exit_reason_str(run->exit_reason));
+
+		switch (get_ucall(vm, VCPU_ID, &uc)) {
+		case UCALL_ABORT:
+			TEST_ASSERT(false, "%s",
+				    (const char *)uc.args[0]);
+			/* NOT REACHED */
+		case UCALL_SYNC:
+			break;
+		case UCALL_DONE:
+			goto done;
+		default:
+			TEST_ASSERT(false,
+				    "Unknown ucall 0x%x.", uc.cmd);
+		}
+	}
+done:
+	kvm_vm_free(vm);
+	return 0;
+}
-- 
2.20.1

Re: [PATCH v4 2/3] selftests: KVM: AMD Nested test infrastructure

[Vitaly Kuznetsov]

Eric Auger <eric.auger@redhat.com> writes:

> Add the basic infrastructure needed to test AMD nested SVM.
> This is largely copied from the KVM unit test infrastructure.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v3 -> v4:
> - just keep the 16 GPRs in gpr64_regs struct
> - vm* instructions do not take any param
> - add comments
>
> v2 -> v3:
> - s/regs/gp_regs64
> - Split the header into 2 parts: svm.h is a copy of
>   arch/x86/include/asm/svm.h whereas svm_util.h contains
>   testing add-ons
> - use get_gdt/dt() and remove sgdt/sidt
> - use get_es/ss/ds/cs
> - fix clobber for dr6 & dr7
> - use u64 instead of ulong
> ---
>  tools/testing/selftests/kvm/Makefile          |   2 +-
>  .../selftests/kvm/include/x86_64/processor.h  |  20 ++
>  .../selftests/kvm/include/x86_64/svm.h        | 297 ++++++++++++++++++
>  .../selftests/kvm/include/x86_64/svm_util.h   |  38 +++
>  tools/testing/selftests/kvm/lib/x86_64/svm.c  | 161 ++++++++++
>  5 files changed, 517 insertions(+), 1 deletion(-)
>  create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm.h
>  create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm_util.h
>  create mode 100644 tools/testing/selftests/kvm/lib/x86_64/svm.c
>
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 608fa835c764..2e770f554cae 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -8,7 +8,7 @@ KSFT_KHDR_INSTALL := 1
>  UNAME_M := $(shell uname -m)
>  
>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c
> -LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/ucall.c
> +LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c
>  LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c
>  
> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
> index 6f7fffaea2e8..12475047869f 100644
> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
> @@ -56,6 +56,26 @@ enum x86_register {
>  	R15,
>  };
>  
> +/* General Registers in 64-Bit Mode */
> +struct gpr64_regs {
> +	u64 rax;
> +	u64 rcx;
> +	u64 rdx;
> +	u64 rbx;
> +	u64 rsp;
> +	u64 rbp;
> +	u64 rsi;
> +	u64 rdi;
> +	u64 r8;
> +	u64 r9;
> +	u64 r10;
> +	u64 r11;
> +	u64 r12;
> +	u64 r13;
> +	u64 r14;
> +	u64 r15;
> +};

It's nice that this struct now matches 'enum x86_register' specified
above, however, we don't seem to use it anywhere (I checked and it goes
back to when kvm selftests framework was introduces).

And, if we decide to drop 'enum x86_register' I'd suggest we order regs
alphabetically in the newly introduced structure :-)

Just a nitpick, can be done in a follow-up patch of course.

> +
>  struct desc64 {
>  	uint16_t limit0;
>  	uint16_t base0;
> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm.h b/tools/testing/selftests/kvm/include/x86_64/svm.h
> new file mode 100644
> index 000000000000..f4ea2355dbc2
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/include/x86_64/svm.h
> @@ -0,0 +1,297 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * tools/testing/selftests/kvm/include/x86_64/svm.h
> + * This is a copy of arch/x86/include/asm/svm.h
> + *
> + */
> +
> +#ifndef SELFTEST_KVM_SVM_H
> +#define SELFTEST_KVM_SVM_H
> +
> +enum {
> +	INTERCEPT_INTR,
> +	INTERCEPT_NMI,
> +	INTERCEPT_SMI,
> +	INTERCEPT_INIT,
> +	INTERCEPT_VINTR,
> +	INTERCEPT_SELECTIVE_CR0,
> +	INTERCEPT_STORE_IDTR,
> +	INTERCEPT_STORE_GDTR,
> +	INTERCEPT_STORE_LDTR,
> +	INTERCEPT_STORE_TR,
> +	INTERCEPT_LOAD_IDTR,
> +	INTERCEPT_LOAD_GDTR,
> +	INTERCEPT_LOAD_LDTR,
> +	INTERCEPT_LOAD_TR,
> +	INTERCEPT_RDTSC,
> +	INTERCEPT_RDPMC,
> +	INTERCEPT_PUSHF,
> +	INTERCEPT_POPF,
> +	INTERCEPT_CPUID,
> +	INTERCEPT_RSM,
> +	INTERCEPT_IRET,
> +	INTERCEPT_INTn,
> +	INTERCEPT_INVD,
> +	INTERCEPT_PAUSE,
> +	INTERCEPT_HLT,
> +	INTERCEPT_INVLPG,
> +	INTERCEPT_INVLPGA,
> +	INTERCEPT_IOIO_PROT,
> +	INTERCEPT_MSR_PROT,
> +	INTERCEPT_TASK_SWITCH,
> +	INTERCEPT_FERR_FREEZE,
> +	INTERCEPT_SHUTDOWN,
> +	INTERCEPT_VMRUN,
> +	INTERCEPT_VMMCALL,
> +	INTERCEPT_VMLOAD,
> +	INTERCEPT_VMSAVE,
> +	INTERCEPT_STGI,
> +	INTERCEPT_CLGI,
> +	INTERCEPT_SKINIT,
> +	INTERCEPT_RDTSCP,
> +	INTERCEPT_ICEBP,
> +	INTERCEPT_WBINVD,
> +	INTERCEPT_MONITOR,
> +	INTERCEPT_MWAIT,
> +	INTERCEPT_MWAIT_COND,
> +	INTERCEPT_XSETBV,
> +	INTERCEPT_RDPRU,
> +};
> +
> +
> +struct __attribute__ ((__packed__)) vmcb_control_area {
> +	u32 intercept_cr;
> +	u32 intercept_dr;
> +	u32 intercept_exceptions;
> +	u64 intercept;
> +	u8 reserved_1[40];
> +	u16 pause_filter_thresh;
> +	u16 pause_filter_count;
> +	u64 iopm_base_pa;
> +	u64 msrpm_base_pa;
> +	u64 tsc_offset;
> +	u32 asid;
> +	u8 tlb_ctl;
> +	u8 reserved_2[3];
> +	u32 int_ctl;
> +	u32 int_vector;
> +	u32 int_state;
> +	u8 reserved_3[4];
> +	u32 exit_code;
> +	u32 exit_code_hi;
> +	u64 exit_info_1;
> +	u64 exit_info_2;
> +	u32 exit_int_info;
> +	u32 exit_int_info_err;
> +	u64 nested_ctl;
> +	u64 avic_vapic_bar;
> +	u8 reserved_4[8];
> +	u32 event_inj;
> +	u32 event_inj_err;
> +	u64 nested_cr3;
> +	u64 virt_ext;
> +	u32 clean;
> +	u32 reserved_5;
> +	u64 next_rip;
> +	u8 insn_len;
> +	u8 insn_bytes[15];
> +	u64 avic_backing_page;	/* Offset 0xe0 */
> +	u8 reserved_6[8];	/* Offset 0xe8 */
> +	u64 avic_logical_id;	/* Offset 0xf0 */
> +	u64 avic_physical_id;	/* Offset 0xf8 */
> +	u8 reserved_7[768];
> +};
> +
> +
> +#define TLB_CONTROL_DO_NOTHING 0
> +#define TLB_CONTROL_FLUSH_ALL_ASID 1
> +#define TLB_CONTROL_FLUSH_ASID 3
> +#define TLB_CONTROL_FLUSH_ASID_LOCAL 7
> +
> +#define V_TPR_MASK 0x0f
> +
> +#define V_IRQ_SHIFT 8
> +#define V_IRQ_MASK (1 << V_IRQ_SHIFT)
> +
> +#define V_GIF_SHIFT 9
> +#define V_GIF_MASK (1 << V_GIF_SHIFT)
> +
> +#define V_INTR_PRIO_SHIFT 16
> +#define V_INTR_PRIO_MASK (0x0f << V_INTR_PRIO_SHIFT)
> +
> +#define V_IGN_TPR_SHIFT 20
> +#define V_IGN_TPR_MASK (1 << V_IGN_TPR_SHIFT)
> +
> +#define V_INTR_MASKING_SHIFT 24
> +#define V_INTR_MASKING_MASK (1 << V_INTR_MASKING_SHIFT)
> +
> +#define V_GIF_ENABLE_SHIFT 25
> +#define V_GIF_ENABLE_MASK (1 << V_GIF_ENABLE_SHIFT)
> +
> +#define AVIC_ENABLE_SHIFT 31
> +#define AVIC_ENABLE_MASK (1 << AVIC_ENABLE_SHIFT)
> +
> +#define LBR_CTL_ENABLE_MASK BIT_ULL(0)
> +#define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1)
> +
> +#define SVM_INTERRUPT_SHADOW_MASK 1
> +
> +#define SVM_IOIO_STR_SHIFT 2
> +#define SVM_IOIO_REP_SHIFT 3
> +#define SVM_IOIO_SIZE_SHIFT 4
> +#define SVM_IOIO_ASIZE_SHIFT 7
> +
> +#define SVM_IOIO_TYPE_MASK 1
> +#define SVM_IOIO_STR_MASK (1 << SVM_IOIO_STR_SHIFT)
> +#define SVM_IOIO_REP_MASK (1 << SVM_IOIO_REP_SHIFT)
> +#define SVM_IOIO_SIZE_MASK (7 << SVM_IOIO_SIZE_SHIFT)
> +#define SVM_IOIO_ASIZE_MASK (7 << SVM_IOIO_ASIZE_SHIFT)
> +
> +#define SVM_VM_CR_VALID_MASK	0x001fULL
> +#define SVM_VM_CR_SVM_LOCK_MASK 0x0008ULL
> +#define SVM_VM_CR_SVM_DIS_MASK  0x0010ULL
> +
> +#define SVM_NESTED_CTL_NP_ENABLE	BIT(0)
> +#define SVM_NESTED_CTL_SEV_ENABLE	BIT(1)
> +
> +struct __attribute__ ((__packed__)) vmcb_seg {
> +	u16 selector;
> +	u16 attrib;
> +	u32 limit;
> +	u64 base;
> +};
> +
> +struct __attribute__ ((__packed__)) vmcb_save_area {
> +	struct vmcb_seg es;
> +	struct vmcb_seg cs;
> +	struct vmcb_seg ss;
> +	struct vmcb_seg ds;
> +	struct vmcb_seg fs;
> +	struct vmcb_seg gs;
> +	struct vmcb_seg gdtr;
> +	struct vmcb_seg ldtr;
> +	struct vmcb_seg idtr;
> +	struct vmcb_seg tr;
> +	u8 reserved_1[43];
> +	u8 cpl;
> +	u8 reserved_2[4];
> +	u64 efer;
> +	u8 reserved_3[112];
> +	u64 cr4;
> +	u64 cr3;
> +	u64 cr0;
> +	u64 dr7;
> +	u64 dr6;
> +	u64 rflags;
> +	u64 rip;
> +	u8 reserved_4[88];
> +	u64 rsp;
> +	u8 reserved_5[24];
> +	u64 rax;
> +	u64 star;
> +	u64 lstar;
> +	u64 cstar;
> +	u64 sfmask;
> +	u64 kernel_gs_base;
> +	u64 sysenter_cs;
> +	u64 sysenter_esp;
> +	u64 sysenter_eip;
> +	u64 cr2;
> +	u8 reserved_6[32];
> +	u64 g_pat;
> +	u64 dbgctl;
> +	u64 br_from;
> +	u64 br_to;
> +	u64 last_excp_from;
> +	u64 last_excp_to;
> +};
> +
> +struct __attribute__ ((__packed__)) vmcb {
> +	struct vmcb_control_area control;
> +	struct vmcb_save_area save;
> +};
> +
> +#define SVM_CPUID_FUNC 0x8000000a
> +
> +#define SVM_VM_CR_SVM_DISABLE 4
> +
> +#define SVM_SELECTOR_S_SHIFT 4
> +#define SVM_SELECTOR_DPL_SHIFT 5
> +#define SVM_SELECTOR_P_SHIFT 7
> +#define SVM_SELECTOR_AVL_SHIFT 8
> +#define SVM_SELECTOR_L_SHIFT 9
> +#define SVM_SELECTOR_DB_SHIFT 10
> +#define SVM_SELECTOR_G_SHIFT 11
> +
> +#define SVM_SELECTOR_TYPE_MASK (0xf)
> +#define SVM_SELECTOR_S_MASK (1 << SVM_SELECTOR_S_SHIFT)
> +#define SVM_SELECTOR_DPL_MASK (3 << SVM_SELECTOR_DPL_SHIFT)
> +#define SVM_SELECTOR_P_MASK (1 << SVM_SELECTOR_P_SHIFT)
> +#define SVM_SELECTOR_AVL_MASK (1 << SVM_SELECTOR_AVL_SHIFT)
> +#define SVM_SELECTOR_L_MASK (1 << SVM_SELECTOR_L_SHIFT)
> +#define SVM_SELECTOR_DB_MASK (1 << SVM_SELECTOR_DB_SHIFT)
> +#define SVM_SELECTOR_G_MASK (1 << SVM_SELECTOR_G_SHIFT)
> +
> +#define SVM_SELECTOR_WRITE_MASK (1 << 1)
> +#define SVM_SELECTOR_READ_MASK SVM_SELECTOR_WRITE_MASK
> +#define SVM_SELECTOR_CODE_MASK (1 << 3)
> +
> +#define INTERCEPT_CR0_READ	0
> +#define INTERCEPT_CR3_READ	3
> +#define INTERCEPT_CR4_READ	4
> +#define INTERCEPT_CR8_READ	8
> +#define INTERCEPT_CR0_WRITE	(16 + 0)
> +#define INTERCEPT_CR3_WRITE	(16 + 3)
> +#define INTERCEPT_CR4_WRITE	(16 + 4)
> +#define INTERCEPT_CR8_WRITE	(16 + 8)
> +
> +#define INTERCEPT_DR0_READ	0
> +#define INTERCEPT_DR1_READ	1
> +#define INTERCEPT_DR2_READ	2
> +#define INTERCEPT_DR3_READ	3
> +#define INTERCEPT_DR4_READ	4
> +#define INTERCEPT_DR5_READ	5
> +#define INTERCEPT_DR6_READ	6
> +#define INTERCEPT_DR7_READ	7
> +#define INTERCEPT_DR0_WRITE	(16 + 0)
> +#define INTERCEPT_DR1_WRITE	(16 + 1)
> +#define INTERCEPT_DR2_WRITE	(16 + 2)
> +#define INTERCEPT_DR3_WRITE	(16 + 3)
> +#define INTERCEPT_DR4_WRITE	(16 + 4)
> +#define INTERCEPT_DR5_WRITE	(16 + 5)
> +#define INTERCEPT_DR6_WRITE	(16 + 6)
> +#define INTERCEPT_DR7_WRITE	(16 + 7)
> +
> +#define SVM_EVTINJ_VEC_MASK 0xff
> +
> +#define SVM_EVTINJ_TYPE_SHIFT 8
> +#define SVM_EVTINJ_TYPE_MASK (7 << SVM_EVTINJ_TYPE_SHIFT)
> +
> +#define SVM_EVTINJ_TYPE_INTR (0 << SVM_EVTINJ_TYPE_SHIFT)
> +#define SVM_EVTINJ_TYPE_NMI (2 << SVM_EVTINJ_TYPE_SHIFT)
> +#define SVM_EVTINJ_TYPE_EXEPT (3 << SVM_EVTINJ_TYPE_SHIFT)
> +#define SVM_EVTINJ_TYPE_SOFT (4 << SVM_EVTINJ_TYPE_SHIFT)
> +
> +#define SVM_EVTINJ_VALID (1 << 31)
> +#define SVM_EVTINJ_VALID_ERR (1 << 11)
> +
> +#define SVM_EXITINTINFO_VEC_MASK SVM_EVTINJ_VEC_MASK
> +#define SVM_EXITINTINFO_TYPE_MASK SVM_EVTINJ_TYPE_MASK
> +
> +#define	SVM_EXITINTINFO_TYPE_INTR SVM_EVTINJ_TYPE_INTR
> +#define	SVM_EXITINTINFO_TYPE_NMI SVM_EVTINJ_TYPE_NMI
> +#define	SVM_EXITINTINFO_TYPE_EXEPT SVM_EVTINJ_TYPE_EXEPT
> +#define	SVM_EXITINTINFO_TYPE_SOFT SVM_EVTINJ_TYPE_SOFT
> +
> +#define SVM_EXITINTINFO_VALID SVM_EVTINJ_VALID
> +#define SVM_EXITINTINFO_VALID_ERR SVM_EVTINJ_VALID_ERR
> +
> +#define SVM_EXITINFOSHIFT_TS_REASON_IRET 36
> +#define SVM_EXITINFOSHIFT_TS_REASON_JMP 38
> +#define SVM_EXITINFOSHIFT_TS_HAS_ERROR_CODE 44
> +
> +#define SVM_EXITINFO_REG_MASK 0x0F
> +
> +#define SVM_CR0_SELECTIVE_MASK (X86_CR0_TS | X86_CR0_MP)
> +
> +#endif /* SELFTEST_KVM_SVM_H */
> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm_util.h b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
> new file mode 100644
> index 000000000000..cd037917fece
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
> @@ -0,0 +1,38 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * tools/testing/selftests/kvm/include/x86_64/svm_utils.h
> + * Header for nested SVM testing
> + *
> + * Copyright (C) 2020, Red Hat, Inc.
> + */
> +
> +#ifndef SELFTEST_KVM_SVM_UTILS_H
> +#define SELFTEST_KVM_SVM_UTILS_H
> +
> +#include <stdint.h>
> +#include "svm.h"
> +#include "processor.h"
> +
> +#define CPUID_SVM_BIT		2
> +#define CPUID_SVM		BIT_ULL(CPUID_SVM_BIT)
> +
> +#define SVM_EXIT_VMMCALL	0x081
> +
> +struct svm_test_data {
> +	/* VMCB */
> +	struct vmcb *vmcb; /* gva */
> +	void *vmcb_hva;
> +	uint64_t vmcb_gpa;
> +
> +	/* host state-save area */
> +	struct vmcb_save_area *save_area; /* gva */
> +	void *save_area_hva;
> +	uint64_t save_area_gpa;
> +};
> +
> +struct svm_test_data *vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva);
> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp);
> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa);
> +void nested_svm_check_supported(void);
> +
> +#endif /* SELFTEST_KVM_SVM_UTILS_H */
> diff --git a/tools/testing/selftests/kvm/lib/x86_64/svm.c b/tools/testing/selftests/kvm/lib/x86_64/svm.c
> new file mode 100644
> index 000000000000..6e05a8fc3fe0
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/lib/x86_64/svm.c
> @@ -0,0 +1,161 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * tools/testing/selftests/kvm/lib/x86_64/svm.c
> + * Helpers used for nested SVM testing
> + * Largely inspired from KVM unit test svm.c
> + *
> + * Copyright (C) 2020, Red Hat, Inc.
> + */
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "../kvm_util_internal.h"
> +#include "processor.h"
> +#include "svm_util.h"
> +
> +struct gpr64_regs guest_regs;
> +u64 rflags;
> +
> +/* Allocate memory regions for nested SVM tests.
> + *
> + * Input Args:
> + *   vm - The VM to allocate guest-virtual addresses in.
> + *
> + * Output Args:
> + *   p_svm_gva - The guest virtual address for the struct svm_test_data.
> + *
> + * Return:
> + *   Pointer to structure with the addresses of the SVM areas.
> + */
> +struct svm_test_data *
> +vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva)
> +{
> +	vm_vaddr_t svm_gva = vm_vaddr_alloc(vm, getpagesize(),
> +					    0x10000, 0, 0);
> +	struct svm_test_data *svm = addr_gva2hva(vm, svm_gva);
> +
> +	svm->vmcb = (void *)vm_vaddr_alloc(vm, getpagesize(),
> +					   0x10000, 0, 0);
> +	svm->vmcb_hva = addr_gva2hva(vm, (uintptr_t)svm->vmcb);
> +	svm->vmcb_gpa = addr_gva2gpa(vm, (uintptr_t)svm->vmcb);
> +
> +	svm->save_area = (void *)vm_vaddr_alloc(vm, getpagesize(),
> +						0x10000, 0, 0);
> +	svm->save_area_hva = addr_gva2hva(vm, (uintptr_t)svm->save_area);
> +	svm->save_area_gpa = addr_gva2gpa(vm, (uintptr_t)svm->save_area);
> +
> +	*p_svm_gva = svm_gva;
> +	return svm;
> +}
> +
> +static void vmcb_set_seg(struct vmcb_seg *seg, u16 selector,
> +			 u64 base, u32 limit, u32 attr)
> +{
> +	seg->selector = selector;
> +	seg->attrib = attr;
> +	seg->limit = limit;
> +	seg->base = base;
> +}
> +
> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp)
> +{
> +	struct vmcb *vmcb = svm->vmcb;
> +	uint64_t vmcb_gpa = svm->vmcb_gpa;
> +	struct vmcb_save_area *save = &vmcb->save;
> +	struct vmcb_control_area *ctrl = &vmcb->control;
> +	u32 data_seg_attr = 3 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
> +	      | SVM_SELECTOR_DB_MASK | SVM_SELECTOR_G_MASK;
> +	u32 code_seg_attr = 9 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
> +		| SVM_SELECTOR_L_MASK | SVM_SELECTOR_G_MASK;
> +	uint64_t efer;
> +
> +	efer = rdmsr(MSR_EFER);
> +	wrmsr(MSR_EFER, efer | EFER_SVME);
> +	wrmsr(MSR_VM_HSAVE_PA, svm->save_area_gpa);
> +
> +	memset(vmcb, 0, sizeof(*vmcb));
> +	asm volatile ("vmsave\n\t" : : "a" (vmcb_gpa) : "memory");
> +	vmcb_set_seg(&save->es, get_es(), 0, -1U, data_seg_attr);
> +	vmcb_set_seg(&save->cs, get_cs(), 0, -1U, code_seg_attr);
> +	vmcb_set_seg(&save->ss, get_ss(), 0, -1U, data_seg_attr);
> +	vmcb_set_seg(&save->ds, get_ds(), 0, -1U, data_seg_attr);
> +	vmcb_set_seg(&save->gdtr, 0, get_gdt().address, get_gdt().size, 0);
> +	vmcb_set_seg(&save->idtr, 0, get_idt().address, get_idt().size, 0);
> +
> +	ctrl->asid = 1;
> +	save->cpl = 0;
> +	save->efer = rdmsr(MSR_EFER);
> +	asm volatile ("mov %%cr4, %0" : "=r"(save->cr4) : : "memory");
> +	asm volatile ("mov %%cr3, %0" : "=r"(save->cr3) : : "memory");
> +	asm volatile ("mov %%cr0, %0" : "=r"(save->cr0) : : "memory");
> +	asm volatile ("mov %%dr7, %0" : "=r"(save->dr7) : : "memory");
> +	asm volatile ("mov %%dr6, %0" : "=r"(save->dr6) : : "memory");
> +	asm volatile ("mov %%cr2, %0" : "=r"(save->cr2) : : "memory");
> +	save->g_pat = rdmsr(MSR_IA32_CR_PAT);
> +	save->dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
> +	ctrl->intercept = (1ULL << INTERCEPT_VMRUN) |
> +				(1ULL << INTERCEPT_VMMCALL);
> +
> +	vmcb->save.rip = (u64)guest_rip;
> +	vmcb->save.rsp = (u64)guest_rsp;
> +	guest_regs.rdi = (u64)svm;
> +}
> +
> +/*
> + * save/restore 64-bit general registers except rax, rip, rsp
> + * which are directly handed through the VMCB guest processor state
> + */
> +#define SAVE_GPR_C				\
> +	"xchg %%rbx, guest_regs+0x20\n\t"	\
> +	"xchg %%rcx, guest_regs+0x10\n\t"	\
> +	"xchg %%rdx, guest_regs+0x18\n\t"	\
> +	"xchg %%rbp, guest_regs+0x30\n\t"	\
> +	"xchg %%rsi, guest_regs+0x38\n\t"	\
> +	"xchg %%rdi, guest_regs+0x40\n\t"	\
> +	"xchg %%r8,  guest_regs+0x48\n\t"	\
> +	"xchg %%r9,  guest_regs+0x50\n\t"	\
> +	"xchg %%r10, guest_regs+0x58\n\t"	\
> +	"xchg %%r11, guest_regs+0x60\n\t"	\
> +	"xchg %%r12, guest_regs+0x68\n\t"	\
> +	"xchg %%r13, guest_regs+0x70\n\t"	\
> +	"xchg %%r14, guest_regs+0x78\n\t"	\
> +	"xchg %%r15, guest_regs+0x80\n\t"
> +
> +#define LOAD_GPR_C      SAVE_GPR_C
> +
> +/*
> + * selftests do not use interrupts so we dropped clgi/sti/cli/stgi
> + * for now. registers involved in LOAD/SAVE_GPR_C are eventually
> + * unmodified so they do not need to be in the clobber list.
> + */
> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa)
> +{
> +	asm volatile (
> +		"vmload\n\t"
> +		"mov rflags, %%r15\n\t"	// rflags
> +		"mov %%r15, 0x170(%[vmcb])\n\t"
> +		"mov guest_regs, %%r15\n\t"	// rax
> +		"mov %%r15, 0x1f8(%[vmcb])\n\t"
> +		LOAD_GPR_C
> +		"vmrun\n\t"
> +		SAVE_GPR_C
> +		"mov 0x170(%[vmcb]), %%r15\n\t"	// rflags
> +		"mov %%r15, rflags\n\t"
> +		"mov 0x1f8(%[vmcb]), %%r15\n\t"	// rax
> +		"mov %%r15, guest_regs\n\t"
> +		"vmsave\n\t"
> +		: : [vmcb] "r" (vmcb), [vmcb_gpa] "a" (vmcb_gpa)

'vmcb_gpa' doesn't need to be named I guess as we don't use it (by
name), we just need it to be in RAX.

> +		: "r15", "memory");
> +}
> +
> +void nested_svm_check_supported(void)
> +{
> +	struct kvm_cpuid_entry2 *entry =
> +		kvm_get_supported_cpuid_entry(0x80000001);
> +
> +	if (!(entry->ecx & CPUID_SVM)) {
> +		fprintf(stderr, "nested SVM not enabled, skipping test\n");
> +		exit(KSFT_SKIP);
> +	}
> +}
> +

I think this is good enough for selftests, we can always improve this
later.

Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>

-- 
Vitaly

Re: [PATCH v4 2/3] selftests: KVM: AMD Nested test infrastructure

[Auger Eric]

Hi Vitaly,

On 2/6/20 1:20 PM, Vitaly Kuznetsov wrote:
> Eric Auger <eric.auger@redhat.com> writes:
> 
>> Add the basic infrastructure needed to test AMD nested SVM.
>> This is largely copied from the KVM unit test infrastructure.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v3 -> v4:
>> - just keep the 16 GPRs in gpr64_regs struct
>> - vm* instructions do not take any param
>> - add comments
>>
>> v2 -> v3:
>> - s/regs/gp_regs64
>> - Split the header into 2 parts: svm.h is a copy of
>>   arch/x86/include/asm/svm.h whereas svm_util.h contains
>>   testing add-ons
>> - use get_gdt/dt() and remove sgdt/sidt
>> - use get_es/ss/ds/cs
>> - fix clobber for dr6 & dr7
>> - use u64 instead of ulong
>> ---
>>  tools/testing/selftests/kvm/Makefile          |   2 +-
>>  .../selftests/kvm/include/x86_64/processor.h  |  20 ++
>>  .../selftests/kvm/include/x86_64/svm.h        | 297 ++++++++++++++++++
>>  .../selftests/kvm/include/x86_64/svm_util.h   |  38 +++
>>  tools/testing/selftests/kvm/lib/x86_64/svm.c  | 161 ++++++++++
>>  5 files changed, 517 insertions(+), 1 deletion(-)
>>  create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm.h
>>  create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm_util.h
>>  create mode 100644 tools/testing/selftests/kvm/lib/x86_64/svm.c
>>
>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
>> index 608fa835c764..2e770f554cae 100644
>> --- a/tools/testing/selftests/kvm/Makefile
>> +++ b/tools/testing/selftests/kvm/Makefile
>> @@ -8,7 +8,7 @@ KSFT_KHDR_INSTALL := 1
>>  UNAME_M := $(shell uname -m)
>>  
>>  LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c
>> -LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/ucall.c
>> +LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c
>>  LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
>>  LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c
>>  
>> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
>> index 6f7fffaea2e8..12475047869f 100644
>> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
>> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
>> @@ -56,6 +56,26 @@ enum x86_register {
>>  	R15,
>>  };
>>  
>> +/* General Registers in 64-Bit Mode */
>> +struct gpr64_regs {
>> +	u64 rax;
>> +	u64 rcx;
>> +	u64 rdx;
>> +	u64 rbx;
>> +	u64 rsp;
>> +	u64 rbp;
>> +	u64 rsi;
>> +	u64 rdi;
>> +	u64 r8;
>> +	u64 r9;
>> +	u64 r10;
>> +	u64 r11;
>> +	u64 r12;
>> +	u64 r13;
>> +	u64 r14;
>> +	u64 r15;
>> +};
> 
> It's nice that this struct now matches 'enum x86_register' specified
> above, however, we don't seem to use it anywhere (I checked and it goes
> back to when kvm selftests framework was introduces).
> 
> And, if we decide to drop 'enum x86_register' I'd suggest we order regs
> alphabetically in the newly introduced structure :-)
> 
> Just a nitpick, can be done in a follow-up patch of course.

Hum OK. So then it may be better to fix that right now as it also
changes the SAVE_GPR_C macro
> 
>> +
>>  struct desc64 {
>>  	uint16_t limit0;
>>  	uint16_t base0;
>> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm.h b/tools/testing/selftests/kvm/include/x86_64/svm.h
>> new file mode 100644
>> index 000000000000..f4ea2355dbc2
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/include/x86_64/svm.h
>> @@ -0,0 +1,297 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * tools/testing/selftests/kvm/include/x86_64/svm.h
>> + * This is a copy of arch/x86/include/asm/svm.h
>> + *
>> + */
>> +
>> +#ifndef SELFTEST_KVM_SVM_H
>> +#define SELFTEST_KVM_SVM_H
>> +
>> +enum {
>> +	INTERCEPT_INTR,
>> +	INTERCEPT_NMI,
>> +	INTERCEPT_SMI,
>> +	INTERCEPT_INIT,
>> +	INTERCEPT_VINTR,
>> +	INTERCEPT_SELECTIVE_CR0,
>> +	INTERCEPT_STORE_IDTR,
>> +	INTERCEPT_STORE_GDTR,
>> +	INTERCEPT_STORE_LDTR,
>> +	INTERCEPT_STORE_TR,
>> +	INTERCEPT_LOAD_IDTR,
>> +	INTERCEPT_LOAD_GDTR,
>> +	INTERCEPT_LOAD_LDTR,
>> +	INTERCEPT_LOAD_TR,
>> +	INTERCEPT_RDTSC,
>> +	INTERCEPT_RDPMC,
>> +	INTERCEPT_PUSHF,
>> +	INTERCEPT_POPF,
>> +	INTERCEPT_CPUID,
>> +	INTERCEPT_RSM,
>> +	INTERCEPT_IRET,
>> +	INTERCEPT_INTn,
>> +	INTERCEPT_INVD,
>> +	INTERCEPT_PAUSE,
>> +	INTERCEPT_HLT,
>> +	INTERCEPT_INVLPG,
>> +	INTERCEPT_INVLPGA,
>> +	INTERCEPT_IOIO_PROT,
>> +	INTERCEPT_MSR_PROT,
>> +	INTERCEPT_TASK_SWITCH,
>> +	INTERCEPT_FERR_FREEZE,
>> +	INTERCEPT_SHUTDOWN,
>> +	INTERCEPT_VMRUN,
>> +	INTERCEPT_VMMCALL,
>> +	INTERCEPT_VMLOAD,
>> +	INTERCEPT_VMSAVE,
>> +	INTERCEPT_STGI,
>> +	INTERCEPT_CLGI,
>> +	INTERCEPT_SKINIT,
>> +	INTERCEPT_RDTSCP,
>> +	INTERCEPT_ICEBP,
>> +	INTERCEPT_WBINVD,
>> +	INTERCEPT_MONITOR,
>> +	INTERCEPT_MWAIT,
>> +	INTERCEPT_MWAIT_COND,
>> +	INTERCEPT_XSETBV,
>> +	INTERCEPT_RDPRU,
>> +};
>> +
>> +
>> +struct __attribute__ ((__packed__)) vmcb_control_area {
>> +	u32 intercept_cr;
>> +	u32 intercept_dr;
>> +	u32 intercept_exceptions;
>> +	u64 intercept;
>> +	u8 reserved_1[40];
>> +	u16 pause_filter_thresh;
>> +	u16 pause_filter_count;
>> +	u64 iopm_base_pa;
>> +	u64 msrpm_base_pa;
>> +	u64 tsc_offset;
>> +	u32 asid;
>> +	u8 tlb_ctl;
>> +	u8 reserved_2[3];
>> +	u32 int_ctl;
>> +	u32 int_vector;
>> +	u32 int_state;
>> +	u8 reserved_3[4];
>> +	u32 exit_code;
>> +	u32 exit_code_hi;
>> +	u64 exit_info_1;
>> +	u64 exit_info_2;
>> +	u32 exit_int_info;
>> +	u32 exit_int_info_err;
>> +	u64 nested_ctl;
>> +	u64 avic_vapic_bar;
>> +	u8 reserved_4[8];
>> +	u32 event_inj;
>> +	u32 event_inj_err;
>> +	u64 nested_cr3;
>> +	u64 virt_ext;
>> +	u32 clean;
>> +	u32 reserved_5;
>> +	u64 next_rip;
>> +	u8 insn_len;
>> +	u8 insn_bytes[15];
>> +	u64 avic_backing_page;	/* Offset 0xe0 */
>> +	u8 reserved_6[8];	/* Offset 0xe8 */
>> +	u64 avic_logical_id;	/* Offset 0xf0 */
>> +	u64 avic_physical_id;	/* Offset 0xf8 */
>> +	u8 reserved_7[768];
>> +};
>> +
>> +
>> +#define TLB_CONTROL_DO_NOTHING 0
>> +#define TLB_CONTROL_FLUSH_ALL_ASID 1
>> +#define TLB_CONTROL_FLUSH_ASID 3
>> +#define TLB_CONTROL_FLUSH_ASID_LOCAL 7
>> +
>> +#define V_TPR_MASK 0x0f
>> +
>> +#define V_IRQ_SHIFT 8
>> +#define V_IRQ_MASK (1 << V_IRQ_SHIFT)
>> +
>> +#define V_GIF_SHIFT 9
>> +#define V_GIF_MASK (1 << V_GIF_SHIFT)
>> +
>> +#define V_INTR_PRIO_SHIFT 16
>> +#define V_INTR_PRIO_MASK (0x0f << V_INTR_PRIO_SHIFT)
>> +
>> +#define V_IGN_TPR_SHIFT 20
>> +#define V_IGN_TPR_MASK (1 << V_IGN_TPR_SHIFT)
>> +
>> +#define V_INTR_MASKING_SHIFT 24
>> +#define V_INTR_MASKING_MASK (1 << V_INTR_MASKING_SHIFT)
>> +
>> +#define V_GIF_ENABLE_SHIFT 25
>> +#define V_GIF_ENABLE_MASK (1 << V_GIF_ENABLE_SHIFT)
>> +
>> +#define AVIC_ENABLE_SHIFT 31
>> +#define AVIC_ENABLE_MASK (1 << AVIC_ENABLE_SHIFT)
>> +
>> +#define LBR_CTL_ENABLE_MASK BIT_ULL(0)
>> +#define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1)
>> +
>> +#define SVM_INTERRUPT_SHADOW_MASK 1
>> +
>> +#define SVM_IOIO_STR_SHIFT 2
>> +#define SVM_IOIO_REP_SHIFT 3
>> +#define SVM_IOIO_SIZE_SHIFT 4
>> +#define SVM_IOIO_ASIZE_SHIFT 7
>> +
>> +#define SVM_IOIO_TYPE_MASK 1
>> +#define SVM_IOIO_STR_MASK (1 << SVM_IOIO_STR_SHIFT)
>> +#define SVM_IOIO_REP_MASK (1 << SVM_IOIO_REP_SHIFT)
>> +#define SVM_IOIO_SIZE_MASK (7 << SVM_IOIO_SIZE_SHIFT)
>> +#define SVM_IOIO_ASIZE_MASK (7 << SVM_IOIO_ASIZE_SHIFT)
>> +
>> +#define SVM_VM_CR_VALID_MASK	0x001fULL
>> +#define SVM_VM_CR_SVM_LOCK_MASK 0x0008ULL
>> +#define SVM_VM_CR_SVM_DIS_MASK  0x0010ULL
>> +
>> +#define SVM_NESTED_CTL_NP_ENABLE	BIT(0)
>> +#define SVM_NESTED_CTL_SEV_ENABLE	BIT(1)
>> +
>> +struct __attribute__ ((__packed__)) vmcb_seg {
>> +	u16 selector;
>> +	u16 attrib;
>> +	u32 limit;
>> +	u64 base;
>> +};
>> +
>> +struct __attribute__ ((__packed__)) vmcb_save_area {
>> +	struct vmcb_seg es;
>> +	struct vmcb_seg cs;
>> +	struct vmcb_seg ss;
>> +	struct vmcb_seg ds;
>> +	struct vmcb_seg fs;
>> +	struct vmcb_seg gs;
>> +	struct vmcb_seg gdtr;
>> +	struct vmcb_seg ldtr;
>> +	struct vmcb_seg idtr;
>> +	struct vmcb_seg tr;
>> +	u8 reserved_1[43];
>> +	u8 cpl;
>> +	u8 reserved_2[4];
>> +	u64 efer;
>> +	u8 reserved_3[112];
>> +	u64 cr4;
>> +	u64 cr3;
>> +	u64 cr0;
>> +	u64 dr7;
>> +	u64 dr6;
>> +	u64 rflags;
>> +	u64 rip;
>> +	u8 reserved_4[88];
>> +	u64 rsp;
>> +	u8 reserved_5[24];
>> +	u64 rax;
>> +	u64 star;
>> +	u64 lstar;
>> +	u64 cstar;
>> +	u64 sfmask;
>> +	u64 kernel_gs_base;
>> +	u64 sysenter_cs;
>> +	u64 sysenter_esp;
>> +	u64 sysenter_eip;
>> +	u64 cr2;
>> +	u8 reserved_6[32];
>> +	u64 g_pat;
>> +	u64 dbgctl;
>> +	u64 br_from;
>> +	u64 br_to;
>> +	u64 last_excp_from;
>> +	u64 last_excp_to;
>> +};
>> +
>> +struct __attribute__ ((__packed__)) vmcb {
>> +	struct vmcb_control_area control;
>> +	struct vmcb_save_area save;
>> +};
>> +
>> +#define SVM_CPUID_FUNC 0x8000000a
>> +
>> +#define SVM_VM_CR_SVM_DISABLE 4
>> +
>> +#define SVM_SELECTOR_S_SHIFT 4
>> +#define SVM_SELECTOR_DPL_SHIFT 5
>> +#define SVM_SELECTOR_P_SHIFT 7
>> +#define SVM_SELECTOR_AVL_SHIFT 8
>> +#define SVM_SELECTOR_L_SHIFT 9
>> +#define SVM_SELECTOR_DB_SHIFT 10
>> +#define SVM_SELECTOR_G_SHIFT 11
>> +
>> +#define SVM_SELECTOR_TYPE_MASK (0xf)
>> +#define SVM_SELECTOR_S_MASK (1 << SVM_SELECTOR_S_SHIFT)
>> +#define SVM_SELECTOR_DPL_MASK (3 << SVM_SELECTOR_DPL_SHIFT)
>> +#define SVM_SELECTOR_P_MASK (1 << SVM_SELECTOR_P_SHIFT)
>> +#define SVM_SELECTOR_AVL_MASK (1 << SVM_SELECTOR_AVL_SHIFT)
>> +#define SVM_SELECTOR_L_MASK (1 << SVM_SELECTOR_L_SHIFT)
>> +#define SVM_SELECTOR_DB_MASK (1 << SVM_SELECTOR_DB_SHIFT)
>> +#define SVM_SELECTOR_G_MASK (1 << SVM_SELECTOR_G_SHIFT)
>> +
>> +#define SVM_SELECTOR_WRITE_MASK (1 << 1)
>> +#define SVM_SELECTOR_READ_MASK SVM_SELECTOR_WRITE_MASK
>> +#define SVM_SELECTOR_CODE_MASK (1 << 3)
>> +
>> +#define INTERCEPT_CR0_READ	0
>> +#define INTERCEPT_CR3_READ	3
>> +#define INTERCEPT_CR4_READ	4
>> +#define INTERCEPT_CR8_READ	8
>> +#define INTERCEPT_CR0_WRITE	(16 + 0)
>> +#define INTERCEPT_CR3_WRITE	(16 + 3)
>> +#define INTERCEPT_CR4_WRITE	(16 + 4)
>> +#define INTERCEPT_CR8_WRITE	(16 + 8)
>> +
>> +#define INTERCEPT_DR0_READ	0
>> +#define INTERCEPT_DR1_READ	1
>> +#define INTERCEPT_DR2_READ	2
>> +#define INTERCEPT_DR3_READ	3
>> +#define INTERCEPT_DR4_READ	4
>> +#define INTERCEPT_DR5_READ	5
>> +#define INTERCEPT_DR6_READ	6
>> +#define INTERCEPT_DR7_READ	7
>> +#define INTERCEPT_DR0_WRITE	(16 + 0)
>> +#define INTERCEPT_DR1_WRITE	(16 + 1)
>> +#define INTERCEPT_DR2_WRITE	(16 + 2)
>> +#define INTERCEPT_DR3_WRITE	(16 + 3)
>> +#define INTERCEPT_DR4_WRITE	(16 + 4)
>> +#define INTERCEPT_DR5_WRITE	(16 + 5)
>> +#define INTERCEPT_DR6_WRITE	(16 + 6)
>> +#define INTERCEPT_DR7_WRITE	(16 + 7)
>> +
>> +#define SVM_EVTINJ_VEC_MASK 0xff
>> +
>> +#define SVM_EVTINJ_TYPE_SHIFT 8
>> +#define SVM_EVTINJ_TYPE_MASK (7 << SVM_EVTINJ_TYPE_SHIFT)
>> +
>> +#define SVM_EVTINJ_TYPE_INTR (0 << SVM_EVTINJ_TYPE_SHIFT)
>> +#define SVM_EVTINJ_TYPE_NMI (2 << SVM_EVTINJ_TYPE_SHIFT)
>> +#define SVM_EVTINJ_TYPE_EXEPT (3 << SVM_EVTINJ_TYPE_SHIFT)
>> +#define SVM_EVTINJ_TYPE_SOFT (4 << SVM_EVTINJ_TYPE_SHIFT)
>> +
>> +#define SVM_EVTINJ_VALID (1 << 31)
>> +#define SVM_EVTINJ_VALID_ERR (1 << 11)
>> +
>> +#define SVM_EXITINTINFO_VEC_MASK SVM_EVTINJ_VEC_MASK
>> +#define SVM_EXITINTINFO_TYPE_MASK SVM_EVTINJ_TYPE_MASK
>> +
>> +#define	SVM_EXITINTINFO_TYPE_INTR SVM_EVTINJ_TYPE_INTR
>> +#define	SVM_EXITINTINFO_TYPE_NMI SVM_EVTINJ_TYPE_NMI
>> +#define	SVM_EXITINTINFO_TYPE_EXEPT SVM_EVTINJ_TYPE_EXEPT
>> +#define	SVM_EXITINTINFO_TYPE_SOFT SVM_EVTINJ_TYPE_SOFT
>> +
>> +#define SVM_EXITINTINFO_VALID SVM_EVTINJ_VALID
>> +#define SVM_EXITINTINFO_VALID_ERR SVM_EVTINJ_VALID_ERR
>> +
>> +#define SVM_EXITINFOSHIFT_TS_REASON_IRET 36
>> +#define SVM_EXITINFOSHIFT_TS_REASON_JMP 38
>> +#define SVM_EXITINFOSHIFT_TS_HAS_ERROR_CODE 44
>> +
>> +#define SVM_EXITINFO_REG_MASK 0x0F
>> +
>> +#define SVM_CR0_SELECTIVE_MASK (X86_CR0_TS | X86_CR0_MP)
>> +
>> +#endif /* SELFTEST_KVM_SVM_H */
>> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm_util.h b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
>> new file mode 100644
>> index 000000000000..cd037917fece
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
>> @@ -0,0 +1,38 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * tools/testing/selftests/kvm/include/x86_64/svm_utils.h
>> + * Header for nested SVM testing
>> + *
>> + * Copyright (C) 2020, Red Hat, Inc.
>> + */
>> +
>> +#ifndef SELFTEST_KVM_SVM_UTILS_H
>> +#define SELFTEST_KVM_SVM_UTILS_H
>> +
>> +#include <stdint.h>
>> +#include "svm.h"
>> +#include "processor.h"
>> +
>> +#define CPUID_SVM_BIT		2
>> +#define CPUID_SVM		BIT_ULL(CPUID_SVM_BIT)
>> +
>> +#define SVM_EXIT_VMMCALL	0x081
>> +
>> +struct svm_test_data {
>> +	/* VMCB */
>> +	struct vmcb *vmcb; /* gva */
>> +	void *vmcb_hva;
>> +	uint64_t vmcb_gpa;
>> +
>> +	/* host state-save area */
>> +	struct vmcb_save_area *save_area; /* gva */
>> +	void *save_area_hva;
>> +	uint64_t save_area_gpa;
>> +};
>> +
>> +struct svm_test_data *vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva);
>> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp);
>> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa);
>> +void nested_svm_check_supported(void);
>> +
>> +#endif /* SELFTEST_KVM_SVM_UTILS_H */
>> diff --git a/tools/testing/selftests/kvm/lib/x86_64/svm.c b/tools/testing/selftests/kvm/lib/x86_64/svm.c
>> new file mode 100644
>> index 000000000000..6e05a8fc3fe0
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/lib/x86_64/svm.c
>> @@ -0,0 +1,161 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * tools/testing/selftests/kvm/lib/x86_64/svm.c
>> + * Helpers used for nested SVM testing
>> + * Largely inspired from KVM unit test svm.c
>> + *
>> + * Copyright (C) 2020, Red Hat, Inc.
>> + */
>> +
>> +#include "test_util.h"
>> +#include "kvm_util.h"
>> +#include "../kvm_util_internal.h"
>> +#include "processor.h"
>> +#include "svm_util.h"
>> +
>> +struct gpr64_regs guest_regs;
>> +u64 rflags;
>> +
>> +/* Allocate memory regions for nested SVM tests.
>> + *
>> + * Input Args:
>> + *   vm - The VM to allocate guest-virtual addresses in.
>> + *
>> + * Output Args:
>> + *   p_svm_gva - The guest virtual address for the struct svm_test_data.
>> + *
>> + * Return:
>> + *   Pointer to structure with the addresses of the SVM areas.
>> + */
>> +struct svm_test_data *
>> +vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva)
>> +{
>> +	vm_vaddr_t svm_gva = vm_vaddr_alloc(vm, getpagesize(),
>> +					    0x10000, 0, 0);
>> +	struct svm_test_data *svm = addr_gva2hva(vm, svm_gva);
>> +
>> +	svm->vmcb = (void *)vm_vaddr_alloc(vm, getpagesize(),
>> +					   0x10000, 0, 0);
>> +	svm->vmcb_hva = addr_gva2hva(vm, (uintptr_t)svm->vmcb);
>> +	svm->vmcb_gpa = addr_gva2gpa(vm, (uintptr_t)svm->vmcb);
>> +
>> +	svm->save_area = (void *)vm_vaddr_alloc(vm, getpagesize(),
>> +						0x10000, 0, 0);
>> +	svm->save_area_hva = addr_gva2hva(vm, (uintptr_t)svm->save_area);
>> +	svm->save_area_gpa = addr_gva2gpa(vm, (uintptr_t)svm->save_area);
>> +
>> +	*p_svm_gva = svm_gva;
>> +	return svm;
>> +}
>> +
>> +static void vmcb_set_seg(struct vmcb_seg *seg, u16 selector,
>> +			 u64 base, u32 limit, u32 attr)
>> +{
>> +	seg->selector = selector;
>> +	seg->attrib = attr;
>> +	seg->limit = limit;
>> +	seg->base = base;
>> +}
>> +
>> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp)
>> +{
>> +	struct vmcb *vmcb = svm->vmcb;
>> +	uint64_t vmcb_gpa = svm->vmcb_gpa;
>> +	struct vmcb_save_area *save = &vmcb->save;
>> +	struct vmcb_control_area *ctrl = &vmcb->control;
>> +	u32 data_seg_attr = 3 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
>> +	      | SVM_SELECTOR_DB_MASK | SVM_SELECTOR_G_MASK;
>> +	u32 code_seg_attr = 9 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
>> +		| SVM_SELECTOR_L_MASK | SVM_SELECTOR_G_MASK;
>> +	uint64_t efer;
>> +
>> +	efer = rdmsr(MSR_EFER);
>> +	wrmsr(MSR_EFER, efer | EFER_SVME);
>> +	wrmsr(MSR_VM_HSAVE_PA, svm->save_area_gpa);
>> +
>> +	memset(vmcb, 0, sizeof(*vmcb));
>> +	asm volatile ("vmsave\n\t" : : "a" (vmcb_gpa) : "memory");
>> +	vmcb_set_seg(&save->es, get_es(), 0, -1U, data_seg_attr);
>> +	vmcb_set_seg(&save->cs, get_cs(), 0, -1U, code_seg_attr);
>> +	vmcb_set_seg(&save->ss, get_ss(), 0, -1U, data_seg_attr);
>> +	vmcb_set_seg(&save->ds, get_ds(), 0, -1U, data_seg_attr);
>> +	vmcb_set_seg(&save->gdtr, 0, get_gdt().address, get_gdt().size, 0);
>> +	vmcb_set_seg(&save->idtr, 0, get_idt().address, get_idt().size, 0);
>> +
>> +	ctrl->asid = 1;
>> +	save->cpl = 0;
>> +	save->efer = rdmsr(MSR_EFER);
>> +	asm volatile ("mov %%cr4, %0" : "=r"(save->cr4) : : "memory");
>> +	asm volatile ("mov %%cr3, %0" : "=r"(save->cr3) : : "memory");
>> +	asm volatile ("mov %%cr0, %0" : "=r"(save->cr0) : : "memory");
>> +	asm volatile ("mov %%dr7, %0" : "=r"(save->dr7) : : "memory");
>> +	asm volatile ("mov %%dr6, %0" : "=r"(save->dr6) : : "memory");
>> +	asm volatile ("mov %%cr2, %0" : "=r"(save->cr2) : : "memory");
>> +	save->g_pat = rdmsr(MSR_IA32_CR_PAT);
>> +	save->dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
>> +	ctrl->intercept = (1ULL << INTERCEPT_VMRUN) |
>> +				(1ULL << INTERCEPT_VMMCALL);
>> +
>> +	vmcb->save.rip = (u64)guest_rip;
>> +	vmcb->save.rsp = (u64)guest_rsp;
>> +	guest_regs.rdi = (u64)svm;
>> +}
>> +
>> +/*
>> + * save/restore 64-bit general registers except rax, rip, rsp
>> + * which are directly handed through the VMCB guest processor state
>> + */
>> +#define SAVE_GPR_C				\
>> +	"xchg %%rbx, guest_regs+0x20\n\t"	\
>> +	"xchg %%rcx, guest_regs+0x10\n\t"	\
>> +	"xchg %%rdx, guest_regs+0x18\n\t"	\
>> +	"xchg %%rbp, guest_regs+0x30\n\t"	\
>> +	"xchg %%rsi, guest_regs+0x38\n\t"	\
>> +	"xchg %%rdi, guest_regs+0x40\n\t"	\
>> +	"xchg %%r8,  guest_regs+0x48\n\t"	\
>> +	"xchg %%r9,  guest_regs+0x50\n\t"	\
>> +	"xchg %%r10, guest_regs+0x58\n\t"	\
>> +	"xchg %%r11, guest_regs+0x60\n\t"	\
>> +	"xchg %%r12, guest_regs+0x68\n\t"	\
>> +	"xchg %%r13, guest_regs+0x70\n\t"	\
>> +	"xchg %%r14, guest_regs+0x78\n\t"	\
>> +	"xchg %%r15, guest_regs+0x80\n\t"
>> +
>> +#define LOAD_GPR_C      SAVE_GPR_C
>> +
>> +/*
>> + * selftests do not use interrupts so we dropped clgi/sti/cli/stgi
>> + * for now. registers involved in LOAD/SAVE_GPR_C are eventually
>> + * unmodified so they do not need to be in the clobber list.
>> + */
>> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa)
>> +{
>> +	asm volatile (
>> +		"vmload\n\t"
>> +		"mov rflags, %%r15\n\t"	// rflags
>> +		"mov %%r15, 0x170(%[vmcb])\n\t"
>> +		"mov guest_regs, %%r15\n\t"	// rax
>> +		"mov %%r15, 0x1f8(%[vmcb])\n\t"
>> +		LOAD_GPR_C
>> +		"vmrun\n\t"
>> +		SAVE_GPR_C
>> +		"mov 0x170(%[vmcb]), %%r15\n\t"	// rflags
>> +		"mov %%r15, rflags\n\t"
>> +		"mov 0x1f8(%[vmcb]), %%r15\n\t"	// rax
>> +		"mov %%r15, guest_regs\n\t"
>> +		"vmsave\n\t"
>> +		: : [vmcb] "r" (vmcb), [vmcb_gpa] "a" (vmcb_gpa)
> 
> 'vmcb_gpa' doesn't need to be named I guess as we don't use it (by
> name), we just need it to be in RAX.
OK
> 
>> +		: "r15", "memory");
>> +}
>> +
>> +void nested_svm_check_supported(void)
>> +{
>> +	struct kvm_cpuid_entry2 *entry =
>> +		kvm_get_supported_cpuid_entry(0x80000001);
>> +
>> +	if (!(entry->ecx & CPUID_SVM)) {
>> +		fprintf(stderr, "nested SVM not enabled, skipping test\n");
>> +		exit(KSFT_SKIP);
>> +	}
>> +}
>> +
> 
> I think this is good enough for selftests, we can always improve this
> later.
> 
> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>

Thank you!

I will however send a v5 to fix above nits ;-)

Eric

Re: [PATCH v4 1/3] selftests: KVM: Replace get_gdt/idt_base() by get_gdt/idt()

[Wei Huang]

On 02/06 11:47, Eric Auger wrote:
> get_gdt_base() and get_idt_base() only return the base address
> of the descriptor tables. Soon we will need to get the size as well.
> Change the prototype of those functions so that they return
> the whole desc_ptr struct instead of the address field.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> 
> ---
> 
> v3 -> v4:
> - Collected R-b's
> ---
>  tools/testing/selftests/kvm/include/x86_64/processor.h | 8 ++++----
>  tools/testing/selftests/kvm/lib/x86_64/vmx.c           | 6 +++---
>  2 files changed, 7 insertions(+), 7 deletions(-)
> 
> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
> index aa6451b3f740..6f7fffaea2e8 100644
> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
> @@ -220,20 +220,20 @@ static inline void set_cr4(uint64_t val)
>  	__asm__ __volatile__("mov %0, %%cr4" : : "r" (val) : "memory");
>  }
>  
> -static inline uint64_t get_gdt_base(void)
> +static inline struct desc_ptr get_gdt(void)
>  {
>  	struct desc_ptr gdt;
>  	__asm__ __volatile__("sgdt %[gdt]"
>  			     : /* output */ [gdt]"=m"(gdt));
> -	return gdt.address;
> +	return gdt;
>  }
>  
> -static inline uint64_t get_idt_base(void)
> +static inline struct desc_ptr get_idt(void)
>  {
>  	struct desc_ptr idt;
>  	__asm__ __volatile__("sidt %[idt]"
>  			     : /* output */ [idt]"=m"(idt));
> -	return idt.address;
> +	return idt;
>  }
>  
>  #define SET_XMM(__var, __xmm) \
> diff --git a/tools/testing/selftests/kvm/lib/x86_64/vmx.c b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
> index 85064baf5e97..7aaa99ca4dbc 100644
> --- a/tools/testing/selftests/kvm/lib/x86_64/vmx.c
> +++ b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
> @@ -288,9 +288,9 @@ static inline void init_vmcs_host_state(void)
>  	vmwrite(HOST_FS_BASE, rdmsr(MSR_FS_BASE));
>  	vmwrite(HOST_GS_BASE, rdmsr(MSR_GS_BASE));
>  	vmwrite(HOST_TR_BASE,
> -		get_desc64_base((struct desc64 *)(get_gdt_base() + get_tr())));
> -	vmwrite(HOST_GDTR_BASE, get_gdt_base());
> -	vmwrite(HOST_IDTR_BASE, get_idt_base());
> +		get_desc64_base((struct desc64 *)(get_gdt().address + get_tr())));
> +	vmwrite(HOST_GDTR_BASE, get_gdt().address);
> +	vmwrite(HOST_IDTR_BASE, get_idt().address);
>  	vmwrite(HOST_IA32_SYSENTER_ESP, rdmsr(MSR_IA32_SYSENTER_ESP));
>  	vmwrite(HOST_IA32_SYSENTER_EIP, rdmsr(MSR_IA32_SYSENTER_EIP));
>  }

Reviewed-by: Wei Huang <wei.huang2@amd.com>

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Wei Huang]

On 02/06 11:47, Eric Auger wrote:
> L2 guest calls vmcall and L1 checks the exit status does
> correspond.
> 
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>

I verified this patch with my AMD box, both with nested=1 and nested=0. I
also intentionally changed the assertion of exit_code to a different
value (0x082) and the test complained about it. So the test is good.

# selftests: kvm: svm_vmcall_test
# ==== Test Assertion Failure ====
#   x86_64/svm_vmcall_test.c:64: false
#   pid=2485656 tid=2485656 - Interrupted system call
#      1        0x0000000000401387: main at svm_vmcall_test.c:72
#      2        0x00007fd0978d71a2: ?? ??:0
#      3        0x00000000004013ed: _start at ??:?
#   Failed guest assert: vmcb->control.exit_code == SVM_EXIT_VMMCALL
# Testing guest mode: PA-bits:ANY, VA-bits:48,  4K pages
# Guest physical address width detected: 48
not ok 15 selftests: kvm: svm_vmcall_test # exit=254

> 
> ---
> 
> v3 -> v4:
> - remove useless includes
> - collected Lin's R-b
> 
> v2 -> v3:
> - remove useless comment and add Vitaly's R-b
> ---
>  tools/testing/selftests/kvm/Makefile          |  1 +
>  .../selftests/kvm/x86_64/svm_vmcall_test.c    | 79 +++++++++++++++++++
>  2 files changed, 80 insertions(+)
>  create mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
> 
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 2e770f554cae..b529d3b42c02 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -26,6 +26,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test
>  TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test
>  TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
>  TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test
> +TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test
>  TEST_GEN_PROGS_x86_64 += clear_dirty_log_test
>  TEST_GEN_PROGS_x86_64 += dirty_log_test
>  TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus
> diff --git a/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
> new file mode 100644
> index 000000000000..6d3565aab94e
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c

Probably rename the file to svm_nested_vmcall_test.c. This matches with
the naming convention of VMX's nested tests. Otherwise people might not know
it is a nested one.

Everything else looks good.

> @@ -0,0 +1,79 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * svm_vmcall_test
> + *
> + * Copyright (C) 2020, Red Hat, Inc.
> + *
> + * Nested SVM testing: VMCALL
> + */
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +#include "svm_util.h"
> +
> +#define VCPU_ID		5
> +
> +static struct kvm_vm *vm;
> +
> +static inline void l2_vmcall(struct svm_test_data *svm)
> +{
> +	__asm__ __volatile__("vmcall");
> +}
> +
> +static void l1_guest_code(struct svm_test_data *svm)
> +{
> +	#define L2_GUEST_STACK_SIZE 64
> +	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
> +	struct vmcb *vmcb = svm->vmcb;
> +
> +	/* Prepare for L2 execution. */
> +	generic_svm_setup(svm, l2_vmcall,
> +			  &l2_guest_stack[L2_GUEST_STACK_SIZE]);
> +
> +	run_guest(vmcb, svm->vmcb_gpa);
> +
> +	GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
> +	GUEST_DONE();
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	vm_vaddr_t svm_gva;
> +
> +	nested_svm_check_supported();
> +
> +	vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code);
> +	vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
> +
> +	vcpu_alloc_svm(vm, &svm_gva);
> +	vcpu_args_set(vm, VCPU_ID, 1, svm_gva);
> +
> +	for (;;) {
> +		volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
> +		struct ucall uc;
> +
> +		vcpu_run(vm, VCPU_ID);
> +		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
> +			    "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
> +			    run->exit_reason,
> +			    exit_reason_str(run->exit_reason));
> +
> +		switch (get_ucall(vm, VCPU_ID, &uc)) {
> +		case UCALL_ABORT:
> +			TEST_ASSERT(false, "%s",
> +				    (const char *)uc.args[0]);
> +			/* NOT REACHED */
> +		case UCALL_SYNC:
> +			break;
> +		case UCALL_DONE:
> +			goto done;
> +		default:
> +			TEST_ASSERT(false,
> +				    "Unknown ucall 0x%x.", uc.cmd);
> +		}
> +	}
> +done:
> +	kvm_vm_free(vm);
> +	return 0;
> +}

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Krish Sadhukhan]

On 2/6/20 9:39 AM, Wei Huang wrote:
> On 02/06 11:47, Eric Auger wrote:
>> L2 guest calls vmcall and L1 checks the exit status does
>> correspond.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> I verified this patch with my AMD box, both with nested=1 and nested=0. I
> also intentionally changed the assertion of exit_code to a different
> value (0x082) and the test complained about it. So the test is good.
>
> # selftests: kvm: svm_vmcall_test
> # ==== Test Assertion Failure ====
> #   x86_64/svm_vmcall_test.c:64: false
> #   pid=2485656 tid=2485656 - Interrupted system call
> #      1        0x0000000000401387: main at svm_vmcall_test.c:72
> #      2        0x00007fd0978d71a2: ?? ??:0
> #      3        0x00000000004013ed: _start at ??:?
> #   Failed guest assert: vmcb->control.exit_code == SVM_EXIT_VMMCALL
> # Testing guest mode: PA-bits:ANY, VA-bits:48,  4K pages
> # Guest physical address width detected: 48
> not ok 15 selftests: kvm: svm_vmcall_test # exit=254
>
>> ---
>>
>> v3 -> v4:
>> - remove useless includes
>> - collected Lin's R-b
>>
>> v2 -> v3:
>> - remove useless comment and add Vitaly's R-b
>> ---
>>   tools/testing/selftests/kvm/Makefile          |  1 +
>>   .../selftests/kvm/x86_64/svm_vmcall_test.c    | 79 +++++++++++++++++++
>>   2 files changed, 80 insertions(+)
>>   create mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>>
>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
>> index 2e770f554cae..b529d3b42c02 100644
>> --- a/tools/testing/selftests/kvm/Makefile
>> +++ b/tools/testing/selftests/kvm/Makefile
>> @@ -26,6 +26,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test
>>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test
>>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
>>   TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test
>> +TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test
>>   TEST_GEN_PROGS_x86_64 += clear_dirty_log_test
>>   TEST_GEN_PROGS_x86_64 += dirty_log_test
>>   TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus
>> diff --git a/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>> new file mode 100644
>> index 000000000000..6d3565aab94e
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
> Probably rename the file to svm_nested_vmcall_test.c. This matches with
> the naming convention of VMX's nested tests. Otherwise people might not know
> it is a nested one.

Is it better to give this file a generic name, say, nsvm_tests or 
something like that, and place all future nested SVM tests in it, rather 
than creating a separate file for each nested test ?
>
> Everything else looks good.
>
>> @@ -0,0 +1,79 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * svm_vmcall_test
>> + *
>> + * Copyright (C) 2020, Red Hat, Inc.
>> + *
>> + * Nested SVM testing: VMCALL
>> + */
>> +
>> +#include "test_util.h"
>> +#include "kvm_util.h"
>> +#include "processor.h"
>> +#include "svm_util.h"
>> +
>> +#define VCPU_ID		5
>> +
>> +static struct kvm_vm *vm;
>> +
>> +static inline void l2_vmcall(struct svm_test_data *svm)
>> +{
>> +	__asm__ __volatile__("vmcall");
>> +}
>> +
>> +static void l1_guest_code(struct svm_test_data *svm)
>> +{
>> +	#define L2_GUEST_STACK_SIZE 64
>> +	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
>> +	struct vmcb *vmcb = svm->vmcb;
>> +
>> +	/* Prepare for L2 execution. */
>> +	generic_svm_setup(svm, l2_vmcall,
>> +			  &l2_guest_stack[L2_GUEST_STACK_SIZE]);
>> +
>> +	run_guest(vmcb, svm->vmcb_gpa);
>> +
>> +	GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
>> +	GUEST_DONE();
>> +}
>> +
>> +int main(int argc, char *argv[])
>> +{
>> +	vm_vaddr_t svm_gva;
>> +
>> +	nested_svm_check_supported();
>> +
>> +	vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code);
>> +	vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
>> +
>> +	vcpu_alloc_svm(vm, &svm_gva);
>> +	vcpu_args_set(vm, VCPU_ID, 1, svm_gva);
>> +
>> +	for (;;) {
>> +		volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
>> +		struct ucall uc;
>> +
>> +		vcpu_run(vm, VCPU_ID);
>> +		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
>> +			    "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
>> +			    run->exit_reason,
>> +			    exit_reason_str(run->exit_reason));
>> +
>> +		switch (get_ucall(vm, VCPU_ID, &uc)) {
>> +		case UCALL_ABORT:
>> +			TEST_ASSERT(false, "%s",
>> +				    (const char *)uc.args[0]);
>> +			/* NOT REACHED */
>> +		case UCALL_SYNC:
>> +			break;
>> +		case UCALL_DONE:
>> +			goto done;
>> +		default:
>> +			TEST_ASSERT(false,
>> +				    "Unknown ucall 0x%x.", uc.cmd);
>> +		}
>> +	}
>> +done:
>> +	kvm_vm_free(vm);
>> +	return 0;
>> +}
>

Re: [PATCH v4 1/3] selftests: KVM: Replace get_gdt/idt_base() by get_gdt/idt()

[Krish Sadhukhan]

On 2/6/20 2:47 AM, Eric Auger wrote:
> get_gdt_base() and get_idt_base() only return the base address
> of the descriptor tables. Soon we will need to get the size as well.
> Change the prototype of those functions so that they return
> the whole desc_ptr struct instead of the address field.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
>
> ---
>
> v3 -> v4:
> - Collected R-b's
> ---
>   tools/testing/selftests/kvm/include/x86_64/processor.h | 8 ++++----
>   tools/testing/selftests/kvm/lib/x86_64/vmx.c           | 6 +++---
>   2 files changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
> index aa6451b3f740..6f7fffaea2e8 100644
> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
> @@ -220,20 +220,20 @@ static inline void set_cr4(uint64_t val)
>   	__asm__ __volatile__("mov %0, %%cr4" : : "r" (val) : "memory");
>   }
>   
> -static inline uint64_t get_gdt_base(void)
> +static inline struct desc_ptr get_gdt(void)
>   {
>   	struct desc_ptr gdt;
>   	__asm__ __volatile__("sgdt %[gdt]"
>   			     : /* output */ [gdt]"=m"(gdt));
> -	return gdt.address;
> +	return gdt;
>   }
>   
> -static inline uint64_t get_idt_base(void)
> +static inline struct desc_ptr get_idt(void)
>   {
>   	struct desc_ptr idt;
>   	__asm__ __volatile__("sidt %[idt]"
>   			     : /* output */ [idt]"=m"(idt));
> -	return idt.address;
> +	return idt;
>   }
>   
>   #define SET_XMM(__var, __xmm) \
> diff --git a/tools/testing/selftests/kvm/lib/x86_64/vmx.c b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
> index 85064baf5e97..7aaa99ca4dbc 100644
> --- a/tools/testing/selftests/kvm/lib/x86_64/vmx.c
> +++ b/tools/testing/selftests/kvm/lib/x86_64/vmx.c
> @@ -288,9 +288,9 @@ static inline void init_vmcs_host_state(void)
>   	vmwrite(HOST_FS_BASE, rdmsr(MSR_FS_BASE));
>   	vmwrite(HOST_GS_BASE, rdmsr(MSR_GS_BASE));
>   	vmwrite(HOST_TR_BASE,
> -		get_desc64_base((struct desc64 *)(get_gdt_base() + get_tr())));
> -	vmwrite(HOST_GDTR_BASE, get_gdt_base());
> -	vmwrite(HOST_IDTR_BASE, get_idt_base());
> +		get_desc64_base((struct desc64 *)(get_gdt().address + get_tr())));
> +	vmwrite(HOST_GDTR_BASE, get_gdt().address);
> +	vmwrite(HOST_IDTR_BASE, get_idt().address);
>   	vmwrite(HOST_IA32_SYSENTER_ESP, rdmsr(MSR_IA32_SYSENTER_ESP));
>   	vmwrite(HOST_IA32_SYSENTER_EIP, rdmsr(MSR_IA32_SYSENTER_EIP));
>   }

Nit: The commit message header can be made better to reflect the correct 
function names. For example,

     Replace get_[gdt | idt]_base() with get_[gdt | idt]()


With that,

     Reviewed-by: Krish Sadhukhan <krish.sadhukhan@oracle.com>

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Krish Sadhukhan]

On 02/06/2020 02:47 AM, Eric Auger wrote:
> L2 guest calls vmcall and L1 checks the exit status does
> correspond.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
>
> ---
>
> v3 -> v4:
> - remove useless includes
> - collected Lin's R-b
>
> v2 -> v3:
> - remove useless comment and add Vitaly's R-b
> ---
>   tools/testing/selftests/kvm/Makefile          |  1 +
>   .../selftests/kvm/x86_64/svm_vmcall_test.c    | 79 +++++++++++++++++++
>   2 files changed, 80 insertions(+)
>   create mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 2e770f554cae..b529d3b42c02 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -26,6 +26,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test
>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test
>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
>   TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test
> +TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test
>   TEST_GEN_PROGS_x86_64 += clear_dirty_log_test
>   TEST_GEN_PROGS_x86_64 += dirty_log_test
>   TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus
> diff --git a/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
> new file mode 100644
> index 000000000000..6d3565aab94e
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
> @@ -0,0 +1,79 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * svm_vmcall_test
> + *
> + * Copyright (C) 2020, Red Hat, Inc.
> + *
> + * Nested SVM testing: VMCALL
> + */
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "processor.h"
> +#include "svm_util.h"
> +
> +#define VCPU_ID		5
> +
> +static struct kvm_vm *vm;
> +
> +static inline void l2_vmcall(struct svm_test_data *svm)
> +{
> +	__asm__ __volatile__("vmcall");
Is it possible to re-use the existing vmcall() function ?
Also, we should probably re-name the function to 'l2_guest_code' which 
is used in the existing code and also it matches with 'l1_guest_code' 
naming.
> +}
> +
> +static void l1_guest_code(struct svm_test_data *svm)
> +{
> +	#define L2_GUEST_STACK_SIZE 64
> +	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
> +	struct vmcb *vmcb = svm->vmcb;
> +
> +	/* Prepare for L2 execution. */
> +	generic_svm_setup(svm, l2_vmcall,
> +			  &l2_guest_stack[L2_GUEST_STACK_SIZE]);
> +
> +	run_guest(vmcb, svm->vmcb_gpa);
> +
> +	GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
> +	GUEST_DONE();
> +}
> +
> +int main(int argc, char *argv[])
> +{
> +	vm_vaddr_t svm_gva;
> +
> +	nested_svm_check_supported();
> +
> +	vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code);
> +	vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
> +
> +	vcpu_alloc_svm(vm, &svm_gva);
> +	vcpu_args_set(vm, VCPU_ID, 1, svm_gva);
> +
> +	for (;;) {
> +		volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
> +		struct ucall uc;
> +
> +		vcpu_run(vm, VCPU_ID);
> +		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
> +			    "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
> +			    run->exit_reason,
> +			    exit_reason_str(run->exit_reason));
> +
> +		switch (get_ucall(vm, VCPU_ID, &uc)) {
> +		case UCALL_ABORT:
> +			TEST_ASSERT(false, "%s",
> +				    (const char *)uc.args[0]);
> +			/* NOT REACHED */
> +		case UCALL_SYNC:
> +			break;
> +		case UCALL_DONE:
> +			goto done;
> +		default:
> +			TEST_ASSERT(false,
> +				    "Unknown ucall 0x%x.", uc.cmd);
> +		}
> +	}
> +done:
> +	kvm_vm_free(vm);
> +	return 0;
> +}

Re: [PATCH v4 2/3] selftests: KVM: AMD Nested test infrastructure

[Krish Sadhukhan]

On 02/06/2020 02:47 AM, Eric Auger wrote:
> Add the basic infrastructure needed to test AMD nested SVM.
> This is largely copied from the KVM unit test infrastructure.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v3 -> v4:
> - just keep the 16 GPRs in gpr64_regs struct
> - vm* instructions do not take any param
> - add comments
>
> v2 -> v3:
> - s/regs/gp_regs64
> - Split the header into 2 parts: svm.h is a copy of
>    arch/x86/include/asm/svm.h whereas svm_util.h contains
>    testing add-ons
> - use get_gdt/dt() and remove sgdt/sidt
> - use get_es/ss/ds/cs
> - fix clobber for dr6 & dr7
> - use u64 instead of ulong
> ---
>   tools/testing/selftests/kvm/Makefile          |   2 +-
>   .../selftests/kvm/include/x86_64/processor.h  |  20 ++
>   .../selftests/kvm/include/x86_64/svm.h        | 297 ++++++++++++++++++
>   .../selftests/kvm/include/x86_64/svm_util.h   |  38 +++
>   tools/testing/selftests/kvm/lib/x86_64/svm.c  | 161 ++++++++++
>   5 files changed, 517 insertions(+), 1 deletion(-)
>   create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm.h
>   create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm_util.h
>   create mode 100644 tools/testing/selftests/kvm/lib/x86_64/svm.c
>
> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
> index 608fa835c764..2e770f554cae 100644
> --- a/tools/testing/selftests/kvm/Makefile
> +++ b/tools/testing/selftests/kvm/Makefile
> @@ -8,7 +8,7 @@ KSFT_KHDR_INSTALL := 1
>   UNAME_M := $(shell uname -m)
>   
>   LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c lib/sparsebit.c
> -LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/ucall.c
> +LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c lib/x86_64/svm.c lib/x86_64/ucall.c
>   LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
>   LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c
>   
> diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h b/tools/testing/selftests/kvm/include/x86_64/processor.h
> index 6f7fffaea2e8..12475047869f 100644
> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
> @@ -56,6 +56,26 @@ enum x86_register {
>   	R15,
>   };
>   
> +/* General Registers in 64-Bit Mode */
> +struct gpr64_regs {
> +	u64 rax;
> +	u64 rcx;
> +	u64 rdx;
> +	u64 rbx;
> +	u64 rsp;
> +	u64 rbp;
> +	u64 rsi;
> +	u64 rdi;
> +	u64 r8;
> +	u64 r9;
> +	u64 r10;
> +	u64 r11;
> +	u64 r12;
> +	u64 r13;
> +	u64 r14;
> +	u64 r15;
> +};
> +
>   struct desc64 {
>   	uint16_t limit0;
>   	uint16_t base0;
> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm.h b/tools/testing/selftests/kvm/include/x86_64/svm.h
> new file mode 100644
> index 000000000000..f4ea2355dbc2
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/include/x86_64/svm.h
> @@ -0,0 +1,297 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * tools/testing/selftests/kvm/include/x86_64/svm.h
> + * This is a copy of arch/x86/include/asm/svm.h
> + *
> + */
> +
> +#ifndef SELFTEST_KVM_SVM_H
> +#define SELFTEST_KVM_SVM_H
> +
> +enum {
> +	INTERCEPT_INTR,
> +	INTERCEPT_NMI,
> +	INTERCEPT_SMI,
> +	INTERCEPT_INIT,
> +	INTERCEPT_VINTR,
> +	INTERCEPT_SELECTIVE_CR0,
> +	INTERCEPT_STORE_IDTR,
> +	INTERCEPT_STORE_GDTR,
> +	INTERCEPT_STORE_LDTR,
> +	INTERCEPT_STORE_TR,
> +	INTERCEPT_LOAD_IDTR,
> +	INTERCEPT_LOAD_GDTR,
> +	INTERCEPT_LOAD_LDTR,
> +	INTERCEPT_LOAD_TR,
> +	INTERCEPT_RDTSC,
> +	INTERCEPT_RDPMC,
> +	INTERCEPT_PUSHF,
> +	INTERCEPT_POPF,
> +	INTERCEPT_CPUID,
> +	INTERCEPT_RSM,
> +	INTERCEPT_IRET,
> +	INTERCEPT_INTn,
> +	INTERCEPT_INVD,
> +	INTERCEPT_PAUSE,
> +	INTERCEPT_HLT,
> +	INTERCEPT_INVLPG,
> +	INTERCEPT_INVLPGA,
> +	INTERCEPT_IOIO_PROT,
> +	INTERCEPT_MSR_PROT,
> +	INTERCEPT_TASK_SWITCH,
> +	INTERCEPT_FERR_FREEZE,
> +	INTERCEPT_SHUTDOWN,
> +	INTERCEPT_VMRUN,
> +	INTERCEPT_VMMCALL,
> +	INTERCEPT_VMLOAD,
> +	INTERCEPT_VMSAVE,
> +	INTERCEPT_STGI,
> +	INTERCEPT_CLGI,
> +	INTERCEPT_SKINIT,
> +	INTERCEPT_RDTSCP,
> +	INTERCEPT_ICEBP,
> +	INTERCEPT_WBINVD,
> +	INTERCEPT_MONITOR,
> +	INTERCEPT_MWAIT,
> +	INTERCEPT_MWAIT_COND,
> +	INTERCEPT_XSETBV,
> +	INTERCEPT_RDPRU,
> +};
> +
> +
> +struct __attribute__ ((__packed__)) vmcb_control_area {
> +	u32 intercept_cr;
> +	u32 intercept_dr;
> +	u32 intercept_exceptions;
> +	u64 intercept;
> +	u8 reserved_1[40];
> +	u16 pause_filter_thresh;
> +	u16 pause_filter_count;
> +	u64 iopm_base_pa;
> +	u64 msrpm_base_pa;
> +	u64 tsc_offset;
> +	u32 asid;
> +	u8 tlb_ctl;
> +	u8 reserved_2[3];
> +	u32 int_ctl;
> +	u32 int_vector;
> +	u32 int_state;
> +	u8 reserved_3[4];
> +	u32 exit_code;
> +	u32 exit_code_hi;
> +	u64 exit_info_1;
> +	u64 exit_info_2;
> +	u32 exit_int_info;
> +	u32 exit_int_info_err;
> +	u64 nested_ctl;
> +	u64 avic_vapic_bar;
> +	u8 reserved_4[8];
> +	u32 event_inj;
> +	u32 event_inj_err;
> +	u64 nested_cr3;
> +	u64 virt_ext;
> +	u32 clean;
> +	u32 reserved_5;
> +	u64 next_rip;
> +	u8 insn_len;
> +	u8 insn_bytes[15];
> +	u64 avic_backing_page;	/* Offset 0xe0 */
> +	u8 reserved_6[8];	/* Offset 0xe8 */
> +	u64 avic_logical_id;	/* Offset 0xf0 */
> +	u64 avic_physical_id;	/* Offset 0xf8 */
> +	u8 reserved_7[768];
> +};
> +
> +
> +#define TLB_CONTROL_DO_NOTHING 0
> +#define TLB_CONTROL_FLUSH_ALL_ASID 1
> +#define TLB_CONTROL_FLUSH_ASID 3
> +#define TLB_CONTROL_FLUSH_ASID_LOCAL 7
> +
> +#define V_TPR_MASK 0x0f
> +
> +#define V_IRQ_SHIFT 8
> +#define V_IRQ_MASK (1 << V_IRQ_SHIFT)
> +
> +#define V_GIF_SHIFT 9
> +#define V_GIF_MASK (1 << V_GIF_SHIFT)
> +
> +#define V_INTR_PRIO_SHIFT 16
> +#define V_INTR_PRIO_MASK (0x0f << V_INTR_PRIO_SHIFT)
> +
> +#define V_IGN_TPR_SHIFT 20
> +#define V_IGN_TPR_MASK (1 << V_IGN_TPR_SHIFT)
> +
> +#define V_INTR_MASKING_SHIFT 24
> +#define V_INTR_MASKING_MASK (1 << V_INTR_MASKING_SHIFT)
> +
> +#define V_GIF_ENABLE_SHIFT 25
> +#define V_GIF_ENABLE_MASK (1 << V_GIF_ENABLE_SHIFT)
> +
> +#define AVIC_ENABLE_SHIFT 31
> +#define AVIC_ENABLE_MASK (1 << AVIC_ENABLE_SHIFT)
> +
> +#define LBR_CTL_ENABLE_MASK BIT_ULL(0)
> +#define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1)
> +
> +#define SVM_INTERRUPT_SHADOW_MASK 1
> +
> +#define SVM_IOIO_STR_SHIFT 2
> +#define SVM_IOIO_REP_SHIFT 3
> +#define SVM_IOIO_SIZE_SHIFT 4
> +#define SVM_IOIO_ASIZE_SHIFT 7
> +
> +#define SVM_IOIO_TYPE_MASK 1
> +#define SVM_IOIO_STR_MASK (1 << SVM_IOIO_STR_SHIFT)
> +#define SVM_IOIO_REP_MASK (1 << SVM_IOIO_REP_SHIFT)
> +#define SVM_IOIO_SIZE_MASK (7 << SVM_IOIO_SIZE_SHIFT)
> +#define SVM_IOIO_ASIZE_MASK (7 << SVM_IOIO_ASIZE_SHIFT)
> +
> +#define SVM_VM_CR_VALID_MASK	0x001fULL
> +#define SVM_VM_CR_SVM_LOCK_MASK 0x0008ULL
> +#define SVM_VM_CR_SVM_DIS_MASK  0x0010ULL
> +
> +#define SVM_NESTED_CTL_NP_ENABLE	BIT(0)
> +#define SVM_NESTED_CTL_SEV_ENABLE	BIT(1)
> +
> +struct __attribute__ ((__packed__)) vmcb_seg {
> +	u16 selector;
> +	u16 attrib;
> +	u32 limit;
> +	u64 base;
> +};
> +
> +struct __attribute__ ((__packed__)) vmcb_save_area {
> +	struct vmcb_seg es;
> +	struct vmcb_seg cs;
> +	struct vmcb_seg ss;
> +	struct vmcb_seg ds;
> +	struct vmcb_seg fs;
> +	struct vmcb_seg gs;
> +	struct vmcb_seg gdtr;
> +	struct vmcb_seg ldtr;
> +	struct vmcb_seg idtr;
> +	struct vmcb_seg tr;
> +	u8 reserved_1[43];
> +	u8 cpl;
> +	u8 reserved_2[4];
> +	u64 efer;
> +	u8 reserved_3[112];
> +	u64 cr4;
> +	u64 cr3;
> +	u64 cr0;
> +	u64 dr7;
> +	u64 dr6;
> +	u64 rflags;
> +	u64 rip;
> +	u8 reserved_4[88];
> +	u64 rsp;
> +	u8 reserved_5[24];
> +	u64 rax;
> +	u64 star;
> +	u64 lstar;
> +	u64 cstar;
> +	u64 sfmask;
> +	u64 kernel_gs_base;
> +	u64 sysenter_cs;
> +	u64 sysenter_esp;
> +	u64 sysenter_eip;
> +	u64 cr2;
> +	u8 reserved_6[32];
> +	u64 g_pat;
> +	u64 dbgctl;
> +	u64 br_from;
> +	u64 br_to;
> +	u64 last_excp_from;
> +	u64 last_excp_to;
> +};
> +
> +struct __attribute__ ((__packed__)) vmcb {
> +	struct vmcb_control_area control;
> +	struct vmcb_save_area save;
> +};
> +
> +#define SVM_CPUID_FUNC 0x8000000a
> +
> +#define SVM_VM_CR_SVM_DISABLE 4
> +
> +#define SVM_SELECTOR_S_SHIFT 4
> +#define SVM_SELECTOR_DPL_SHIFT 5
> +#define SVM_SELECTOR_P_SHIFT 7
> +#define SVM_SELECTOR_AVL_SHIFT 8
> +#define SVM_SELECTOR_L_SHIFT 9
> +#define SVM_SELECTOR_DB_SHIFT 10
> +#define SVM_SELECTOR_G_SHIFT 11
> +
> +#define SVM_SELECTOR_TYPE_MASK (0xf)
> +#define SVM_SELECTOR_S_MASK (1 << SVM_SELECTOR_S_SHIFT)
> +#define SVM_SELECTOR_DPL_MASK (3 << SVM_SELECTOR_DPL_SHIFT)
> +#define SVM_SELECTOR_P_MASK (1 << SVM_SELECTOR_P_SHIFT)
> +#define SVM_SELECTOR_AVL_MASK (1 << SVM_SELECTOR_AVL_SHIFT)
> +#define SVM_SELECTOR_L_MASK (1 << SVM_SELECTOR_L_SHIFT)
> +#define SVM_SELECTOR_DB_MASK (1 << SVM_SELECTOR_DB_SHIFT)
> +#define SVM_SELECTOR_G_MASK (1 << SVM_SELECTOR_G_SHIFT)
> +
> +#define SVM_SELECTOR_WRITE_MASK (1 << 1)
> +#define SVM_SELECTOR_READ_MASK SVM_SELECTOR_WRITE_MASK
> +#define SVM_SELECTOR_CODE_MASK (1 << 3)
> +
> +#define INTERCEPT_CR0_READ	0
> +#define INTERCEPT_CR3_READ	3
> +#define INTERCEPT_CR4_READ	4
> +#define INTERCEPT_CR8_READ	8
> +#define INTERCEPT_CR0_WRITE	(16 + 0)
> +#define INTERCEPT_CR3_WRITE	(16 + 3)
> +#define INTERCEPT_CR4_WRITE	(16 + 4)
> +#define INTERCEPT_CR8_WRITE	(16 + 8)
> +
> +#define INTERCEPT_DR0_READ	0
> +#define INTERCEPT_DR1_READ	1
> +#define INTERCEPT_DR2_READ	2
> +#define INTERCEPT_DR3_READ	3
> +#define INTERCEPT_DR4_READ	4
> +#define INTERCEPT_DR5_READ	5
> +#define INTERCEPT_DR6_READ	6
> +#define INTERCEPT_DR7_READ	7
> +#define INTERCEPT_DR0_WRITE	(16 + 0)
> +#define INTERCEPT_DR1_WRITE	(16 + 1)
> +#define INTERCEPT_DR2_WRITE	(16 + 2)
> +#define INTERCEPT_DR3_WRITE	(16 + 3)
> +#define INTERCEPT_DR4_WRITE	(16 + 4)
> +#define INTERCEPT_DR5_WRITE	(16 + 5)
> +#define INTERCEPT_DR6_WRITE	(16 + 6)
> +#define INTERCEPT_DR7_WRITE	(16 + 7)
> +
> +#define SVM_EVTINJ_VEC_MASK 0xff
> +
> +#define SVM_EVTINJ_TYPE_SHIFT 8
> +#define SVM_EVTINJ_TYPE_MASK (7 << SVM_EVTINJ_TYPE_SHIFT)
> +
> +#define SVM_EVTINJ_TYPE_INTR (0 << SVM_EVTINJ_TYPE_SHIFT)
> +#define SVM_EVTINJ_TYPE_NMI (2 << SVM_EVTINJ_TYPE_SHIFT)
> +#define SVM_EVTINJ_TYPE_EXEPT (3 << SVM_EVTINJ_TYPE_SHIFT)
> +#define SVM_EVTINJ_TYPE_SOFT (4 << SVM_EVTINJ_TYPE_SHIFT)
> +
> +#define SVM_EVTINJ_VALID (1 << 31)
> +#define SVM_EVTINJ_VALID_ERR (1 << 11)
> +
> +#define SVM_EXITINTINFO_VEC_MASK SVM_EVTINJ_VEC_MASK
> +#define SVM_EXITINTINFO_TYPE_MASK SVM_EVTINJ_TYPE_MASK
> +
> +#define	SVM_EXITINTINFO_TYPE_INTR SVM_EVTINJ_TYPE_INTR
> +#define	SVM_EXITINTINFO_TYPE_NMI SVM_EVTINJ_TYPE_NMI
> +#define	SVM_EXITINTINFO_TYPE_EXEPT SVM_EVTINJ_TYPE_EXEPT
> +#define	SVM_EXITINTINFO_TYPE_SOFT SVM_EVTINJ_TYPE_SOFT
> +
> +#define SVM_EXITINTINFO_VALID SVM_EVTINJ_VALID
> +#define SVM_EXITINTINFO_VALID_ERR SVM_EVTINJ_VALID_ERR
> +
> +#define SVM_EXITINFOSHIFT_TS_REASON_IRET 36
> +#define SVM_EXITINFOSHIFT_TS_REASON_JMP 38
> +#define SVM_EXITINFOSHIFT_TS_HAS_ERROR_CODE 44
> +
> +#define SVM_EXITINFO_REG_MASK 0x0F
> +
> +#define SVM_CR0_SELECTIVE_MASK (X86_CR0_TS | X86_CR0_MP)
> +
> +#endif /* SELFTEST_KVM_SVM_H */
> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm_util.h b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
> new file mode 100644
> index 000000000000..cd037917fece
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
> @@ -0,0 +1,38 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +/*
> + * tools/testing/selftests/kvm/include/x86_64/svm_utils.h
> + * Header for nested SVM testing
> + *
> + * Copyright (C) 2020, Red Hat, Inc.
> + */
> +
> +#ifndef SELFTEST_KVM_SVM_UTILS_H
> +#define SELFTEST_KVM_SVM_UTILS_H
> +
> +#include <stdint.h>
> +#include "svm.h"
> +#include "processor.h"
> +
> +#define CPUID_SVM_BIT		2
> +#define CPUID_SVM		BIT_ULL(CPUID_SVM_BIT)
> +
> +#define SVM_EXIT_VMMCALL	0x081
> +
> +struct svm_test_data {
> +	/* VMCB */
> +	struct vmcb *vmcb; /* gva */
> +	void *vmcb_hva;
> +	uint64_t vmcb_gpa;
> +
> +	/* host state-save area */
> +	struct vmcb_save_area *save_area; /* gva */
> +	void *save_area_hva;
> +	uint64_t save_area_gpa;
> +};
Looks like vmcb_hva and save_area_hva haven't been used anywhere. Do we 
need them ?
> +
> +struct svm_test_data *vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva);
> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp);
> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa);
> +void nested_svm_check_supported(void);
> +
> +#endif /* SELFTEST_KVM_SVM_UTILS_H */
> diff --git a/tools/testing/selftests/kvm/lib/x86_64/svm.c b/tools/testing/selftests/kvm/lib/x86_64/svm.c
> new file mode 100644
> index 000000000000..6e05a8fc3fe0
> --- /dev/null
> +++ b/tools/testing/selftests/kvm/lib/x86_64/svm.c
> @@ -0,0 +1,161 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +/*
> + * tools/testing/selftests/kvm/lib/x86_64/svm.c
> + * Helpers used for nested SVM testing
> + * Largely inspired from KVM unit test svm.c
> + *
> + * Copyright (C) 2020, Red Hat, Inc.
> + */
> +
> +#include "test_util.h"
> +#include "kvm_util.h"
> +#include "../kvm_util_internal.h"
> +#include "processor.h"
> +#include "svm_util.h"
> +
> +struct gpr64_regs guest_regs;
> +u64 rflags;
> +
> +/* Allocate memory regions for nested SVM tests.
> + *
> + * Input Args:
> + *   vm - The VM to allocate guest-virtual addresses in.
> + *
> + * Output Args:
> + *   p_svm_gva - The guest virtual address for the struct svm_test_data.
> + *
> + * Return:
> + *   Pointer to structure with the addresses of the SVM areas.
> + */
> +struct svm_test_data *
> +vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva)
> +{
> +	vm_vaddr_t svm_gva = vm_vaddr_alloc(vm, getpagesize(),
> +					    0x10000, 0, 0);
> +	struct svm_test_data *svm = addr_gva2hva(vm, svm_gva);
> +
> +	svm->vmcb = (void *)vm_vaddr_alloc(vm, getpagesize(),
> +					   0x10000, 0, 0);
> +	svm->vmcb_hva = addr_gva2hva(vm, (uintptr_t)svm->vmcb);
> +	svm->vmcb_gpa = addr_gva2gpa(vm, (uintptr_t)svm->vmcb);
> +
> +	svm->save_area = (void *)vm_vaddr_alloc(vm, getpagesize(),
> +						0x10000, 0, 0);
> +	svm->save_area_hva = addr_gva2hva(vm, (uintptr_t)svm->save_area);
> +	svm->save_area_gpa = addr_gva2gpa(vm, (uintptr_t)svm->save_area);
> +
> +	*p_svm_gva = svm_gva;
> +	return svm;
> +}
> +
> +static void vmcb_set_seg(struct vmcb_seg *seg, u16 selector,
> +			 u64 base, u32 limit, u32 attr)
> +{
> +	seg->selector = selector;
> +	seg->attrib = attr;
> +	seg->limit = limit;
> +	seg->base = base;
> +}
> +
> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip, void *guest_rsp)
> +{
> +	struct vmcb *vmcb = svm->vmcb;
> +	uint64_t vmcb_gpa = svm->vmcb_gpa;
> +	struct vmcb_save_area *save = &vmcb->save;
> +	struct vmcb_control_area *ctrl = &vmcb->control;
> +	u32 data_seg_attr = 3 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
> +	      | SVM_SELECTOR_DB_MASK | SVM_SELECTOR_G_MASK;
> +	u32 code_seg_attr = 9 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
> +		| SVM_SELECTOR_L_MASK | SVM_SELECTOR_G_MASK;
> +	uint64_t efer;
> +
> +	efer = rdmsr(MSR_EFER);
> +	wrmsr(MSR_EFER, efer | EFER_SVME);
> +	wrmsr(MSR_VM_HSAVE_PA, svm->save_area_gpa);
> +
> +	memset(vmcb, 0, sizeof(*vmcb));
> +	asm volatile ("vmsave\n\t" : : "a" (vmcb_gpa) : "memory");
> +	vmcb_set_seg(&save->es, get_es(), 0, -1U, data_seg_attr);
> +	vmcb_set_seg(&save->cs, get_cs(), 0, -1U, code_seg_attr);
> +	vmcb_set_seg(&save->ss, get_ss(), 0, -1U, data_seg_attr);
> +	vmcb_set_seg(&save->ds, get_ds(), 0, -1U, data_seg_attr);
> +	vmcb_set_seg(&save->gdtr, 0, get_gdt().address, get_gdt().size, 0);
> +	vmcb_set_seg(&save->idtr, 0, get_idt().address, get_idt().size, 0);
> +
> +	ctrl->asid = 1;
> +	save->cpl = 0;
> +	save->efer = rdmsr(MSR_EFER);
> +	asm volatile ("mov %%cr4, %0" : "=r"(save->cr4) : : "memory");
> +	asm volatile ("mov %%cr3, %0" : "=r"(save->cr3) : : "memory");
> +	asm volatile ("mov %%cr0, %0" : "=r"(save->cr0) : : "memory");
> +	asm volatile ("mov %%dr7, %0" : "=r"(save->dr7) : : "memory");
> +	asm volatile ("mov %%dr6, %0" : "=r"(save->dr6) : : "memory");
> +	asm volatile ("mov %%cr2, %0" : "=r"(save->cr2) : : "memory");
> +	save->g_pat = rdmsr(MSR_IA32_CR_PAT);
> +	save->dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
> +	ctrl->intercept = (1ULL << INTERCEPT_VMRUN) |
> +				(1ULL << INTERCEPT_VMMCALL);
> +
> +	vmcb->save.rip = (u64)guest_rip;
> +	vmcb->save.rsp = (u64)guest_rsp;
> +	guest_regs.rdi = (u64)svm;
> +}
> +
> +/*
> + * save/restore 64-bit general registers except rax, rip, rsp
> + * which are directly handed through the VMCB guest processor state
> + */
> +#define SAVE_GPR_C				\
> +	"xchg %%rbx, guest_regs+0x20\n\t"	\
> +	"xchg %%rcx, guest_regs+0x10\n\t"	\
> +	"xchg %%rdx, guest_regs+0x18\n\t"	\
> +	"xchg %%rbp, guest_regs+0x30\n\t"	\
> +	"xchg %%rsi, guest_regs+0x38\n\t"	\
> +	"xchg %%rdi, guest_regs+0x40\n\t"	\
> +	"xchg %%r8,  guest_regs+0x48\n\t"	\
> +	"xchg %%r9,  guest_regs+0x50\n\t"	\
> +	"xchg %%r10, guest_regs+0x58\n\t"	\
> +	"xchg %%r11, guest_regs+0x60\n\t"	\
> +	"xchg %%r12, guest_regs+0x68\n\t"	\
> +	"xchg %%r13, guest_regs+0x70\n\t"	\
> +	"xchg %%r14, guest_regs+0x78\n\t"	\
> +	"xchg %%r15, guest_regs+0x80\n\t"
> +
> +#define LOAD_GPR_C      SAVE_GPR_C
> +
> +/*
> + * selftests do not use interrupts so we dropped clgi/sti/cli/stgi
> + * for now. registers involved in LOAD/SAVE_GPR_C are eventually
> + * unmodified so they do not need to be in the clobber list.
> + */
> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa)
> +{
> +	asm volatile (
> +		"vmload\n\t"
Don't we need to set %rax before calling vmload ?

             "mov %[vmcb_gpa], %%rax \n\t"
             "vmload %%rax\n\t"

> +		"mov rflags, %%r15\n\t"	// rflags
> +		"mov %%r15, 0x170(%[vmcb])\n\t"
> +		"mov guest_regs, %%r15\n\t"	// rax
> +		"mov %%r15, 0x1f8(%[vmcb])\n\t"
> +		LOAD_GPR_C
> +		"vmrun\n\t"
> +		SAVE_GPR_C
> +		"mov 0x170(%[vmcb]), %%r15\n\t"	// rflags
> +		"mov %%r15, rflags\n\t"
> +		"mov 0x1f8(%[vmcb]), %%r15\n\t"	// rax
> +		"mov %%r15, guest_regs\n\t"
> +		"vmsave\n\t"
> +		: : [vmcb] "r" (vmcb), [vmcb_gpa] "a" (vmcb_gpa)
> +		: "r15", "memory");
> +}
> +
> +void nested_svm_check_supported(void)
> +{
> +	struct kvm_cpuid_entry2 *entry =
> +		kvm_get_supported_cpuid_entry(0x80000001);
> +
> +	if (!(entry->ecx & CPUID_SVM)) {
> +		fprintf(stderr, "nested SVM not enabled, skipping test\n");
I think a better message would be:

     "nested SVM not supported on this CPU, skipping test\n"

Also, the function should ideally return a boolean and let the callers 
print whatever they want.

> +		exit(KSFT_SKIP);
> +	}
> +}
> +

Re: [PATCH v4 2/3] selftests: KVM: AMD Nested test infrastructure

[Vitaly Kuznetsov]

Krish Sadhukhan <krish.sadhukhan@oracle.com> writes:

...
> +	asm volatile (
>> +		"vmload\n\t"
> Don't we need to set %rax before calling vmload ?
>

No, because it is already there

...
>> +		: : [vmcb] "r" (vmcb), [vmcb_gpa] "a" (vmcb_gpa)

"a" constraint in input operands does the job.

-- 
Vitaly

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Auger Eric]

Hi Krish,
On 2/6/20 8:08 PM, Krish Sadhukhan wrote:
> 
> On 2/6/20 9:39 AM, Wei Huang wrote:
>> On 02/06 11:47, Eric Auger wrote:
>>> L2 guest calls vmcall and L1 checks the exit status does
>>> correspond.
>>>
>>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>>> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
>> I verified this patch with my AMD box, both with nested=1 and nested=0. I
>> also intentionally changed the assertion of exit_code to a different
>> value (0x082) and the test complained about it. So the test is good.
>>
>> # selftests: kvm: svm_vmcall_test
>> # ==== Test Assertion Failure ====
>> #   x86_64/svm_vmcall_test.c:64: false
>> #   pid=2485656 tid=2485656 - Interrupted system call
>> #      1        0x0000000000401387: main at svm_vmcall_test.c:72
>> #      2        0x00007fd0978d71a2: ?? ??:0
>> #      3        0x00000000004013ed: _start at ??:?
>> #   Failed guest assert: vmcb->control.exit_code == SVM_EXIT_VMMCALL
>> # Testing guest mode: PA-bits:ANY, VA-bits:48,  4K pages
>> # Guest physical address width detected: 48
>> not ok 15 selftests: kvm: svm_vmcall_test # exit=254
>>
>>> ---
>>>
>>> v3 -> v4:
>>> - remove useless includes
>>> - collected Lin's R-b
>>>
>>> v2 -> v3:
>>> - remove useless comment and add Vitaly's R-b
>>> ---
>>>   tools/testing/selftests/kvm/Makefile          |  1 +
>>>   .../selftests/kvm/x86_64/svm_vmcall_test.c    | 79 +++++++++++++++++++
>>>   2 files changed, 80 insertions(+)
>>>   create mode 100644
>>> tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>>>
>>> diff --git a/tools/testing/selftests/kvm/Makefile
>>> b/tools/testing/selftests/kvm/Makefile
>>> index 2e770f554cae..b529d3b42c02 100644
>>> --- a/tools/testing/selftests/kvm/Makefile
>>> +++ b/tools/testing/selftests/kvm/Makefile
>>> @@ -26,6 +26,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test
>>>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test
>>>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
>>>   TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test
>>> +TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test
>>>   TEST_GEN_PROGS_x86_64 += clear_dirty_log_test
>>>   TEST_GEN_PROGS_x86_64 += dirty_log_test
>>>   TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus
>>> diff --git a/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>>> b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>>> new file mode 100644
>>> index 000000000000..6d3565aab94e
>>> --- /dev/null
>>> +++ b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>> Probably rename the file to svm_nested_vmcall_test.c. This matches with
>> the naming convention of VMX's nested tests. Otherwise people might
>> not know
>> it is a nested one.
> 
> Is it better to give this file a generic name, say, nsvm_tests or
> something like that, and place all future nested SVM tests in it, rather
> than creating a separate file for each nested test ?
We had this discussion earlier. See https://lkml.org/lkml/2020/1/21/429

In v1 I proposed a similar framework as kut with sub-tests but it looks
we do not target such kind of tests in kselftests. vmcall test is just a
first dummy test that paves the way for more involved API tests.

Thanks

Eric
>>
>> Everything else looks good.
>>
>>> @@ -0,0 +1,79 @@
>>> +// SPDX-License-Identifier: GPL-2.0-only
>>> +/*
>>> + * svm_vmcall_test
>>> + *
>>> + * Copyright (C) 2020, Red Hat, Inc.
>>> + *
>>> + * Nested SVM testing: VMCALL
>>> + */
>>> +
>>> +#include "test_util.h"
>>> +#include "kvm_util.h"
>>> +#include "processor.h"
>>> +#include "svm_util.h"
>>> +
>>> +#define VCPU_ID        5
>>> +
>>> +static struct kvm_vm *vm;
>>> +
>>> +static inline void l2_vmcall(struct svm_test_data *svm)
>>> +{
>>> +    __asm__ __volatile__("vmcall");
>>> +}
>>> +
>>> +static void l1_guest_code(struct svm_test_data *svm)
>>> +{
>>> +    #define L2_GUEST_STACK_SIZE 64
>>> +    unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
>>> +    struct vmcb *vmcb = svm->vmcb;
>>> +
>>> +    /* Prepare for L2 execution. */
>>> +    generic_svm_setup(svm, l2_vmcall,
>>> +              &l2_guest_stack[L2_GUEST_STACK_SIZE]);
>>> +
>>> +    run_guest(vmcb, svm->vmcb_gpa);
>>> +
>>> +    GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
>>> +    GUEST_DONE();
>>> +}
>>> +
>>> +int main(int argc, char *argv[])
>>> +{
>>> +    vm_vaddr_t svm_gva;
>>> +
>>> +    nested_svm_check_supported();
>>> +
>>> +    vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code);
>>> +    vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
>>> +
>>> +    vcpu_alloc_svm(vm, &svm_gva);
>>> +    vcpu_args_set(vm, VCPU_ID, 1, svm_gva);
>>> +
>>> +    for (;;) {
>>> +        volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
>>> +        struct ucall uc;
>>> +
>>> +        vcpu_run(vm, VCPU_ID);
>>> +        TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
>>> +                "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
>>> +                run->exit_reason,
>>> +                exit_reason_str(run->exit_reason));
>>> +
>>> +        switch (get_ucall(vm, VCPU_ID, &uc)) {
>>> +        case UCALL_ABORT:
>>> +            TEST_ASSERT(false, "%s",
>>> +                    (const char *)uc.args[0]);
>>> +            /* NOT REACHED */
>>> +        case UCALL_SYNC:
>>> +            break;
>>> +        case UCALL_DONE:
>>> +            goto done;
>>> +        default:
>>> +            TEST_ASSERT(false,
>>> +                    "Unknown ucall 0x%x.", uc.cmd);
>>> +        }
>>> +    }
>>> +done:
>>> +    kvm_vm_free(vm);
>>> +    return 0;
>>> +}
>>
> 

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Auger Eric]

Hi Wei,

On 2/6/20 6:39 PM, Wei Huang wrote:
> On 02/06 11:47, Eric Auger wrote:
>> L2 guest calls vmcall and L1 checks the exit status does
>> correspond.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
> 
> I verified this patch with my AMD box, both with nested=1 and nested=0. I
> also intentionally changed the assertion of exit_code to a different
> value (0x082) and the test complained about it. So the test is good.
> 
> # selftests: kvm: svm_vmcall_test
> # ==== Test Assertion Failure ====
> #   x86_64/svm_vmcall_test.c:64: false
> #   pid=2485656 tid=2485656 - Interrupted system call
> #      1        0x0000000000401387: main at svm_vmcall_test.c:72
> #      2        0x00007fd0978d71a2: ?? ??:0
> #      3        0x00000000004013ed: _start at ??:?
> #   Failed guest assert: vmcb->control.exit_code == SVM_EXIT_VMMCALL
> # Testing guest mode: PA-bits:ANY, VA-bits:48,  4K pages
> # Guest physical address width detected: 48
> not ok 15 selftests: kvm: svm_vmcall_test # exit=254

thank you for testing! May I include your T-b then?
> 
>>
>> ---
>>
>> v3 -> v4:
>> - remove useless includes
>> - collected Lin's R-b
>>
>> v2 -> v3:
>> - remove useless comment and add Vitaly's R-b
>> ---
>>  tools/testing/selftests/kvm/Makefile          |  1 +
>>  .../selftests/kvm/x86_64/svm_vmcall_test.c    | 79 +++++++++++++++++++
>>  2 files changed, 80 insertions(+)
>>  create mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>>
>> diff --git a/tools/testing/selftests/kvm/Makefile b/tools/testing/selftests/kvm/Makefile
>> index 2e770f554cae..b529d3b42c02 100644
>> --- a/tools/testing/selftests/kvm/Makefile
>> +++ b/tools/testing/selftests/kvm/Makefile
>> @@ -26,6 +26,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test
>>  TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test
>>  TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
>>  TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test
>> +TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test
>>  TEST_GEN_PROGS_x86_64 += clear_dirty_log_test
>>  TEST_GEN_PROGS_x86_64 += dirty_log_test
>>  TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus
>> diff --git a/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>> new file mode 100644
>> index 000000000000..6d3565aab94e
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
> 
> Probably rename the file to svm_nested_vmcall_test.c. This matches with
> the naming convention of VMX's nested tests. Otherwise people might not know
> it is a nested one.

From what I understand, all the vmx_* (including vmx_tsc_adjust_test for
instance) are related to nested. So I'd rather leave svm_ prefix for
nested SVM.

Thanks

Eric
> 
> Everything else looks good.
> 
>> @@ -0,0 +1,79 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * svm_vmcall_test
>> + *
>> + * Copyright (C) 2020, Red Hat, Inc.
>> + *
>> + * Nested SVM testing: VMCALL
>> + */
>> +
>> +#include "test_util.h"
>> +#include "kvm_util.h"
>> +#include "processor.h"
>> +#include "svm_util.h"
>> +
>> +#define VCPU_ID		5
>> +
>> +static struct kvm_vm *vm;
>> +
>> +static inline void l2_vmcall(struct svm_test_data *svm)
>> +{
>> +	__asm__ __volatile__("vmcall");
>> +}
>> +
>> +static void l1_guest_code(struct svm_test_data *svm)
>> +{
>> +	#define L2_GUEST_STACK_SIZE 64
>> +	unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
>> +	struct vmcb *vmcb = svm->vmcb;
>> +
>> +	/* Prepare for L2 execution. */
>> +	generic_svm_setup(svm, l2_vmcall,
>> +			  &l2_guest_stack[L2_GUEST_STACK_SIZE]);
>> +
>> +	run_guest(vmcb, svm->vmcb_gpa);
>> +
>> +	GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
>> +	GUEST_DONE();
>> +}
>> +
>> +int main(int argc, char *argv[])
>> +{
>> +	vm_vaddr_t svm_gva;
>> +
>> +	nested_svm_check_supported();
>> +
>> +	vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code);
>> +	vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
>> +
>> +	vcpu_alloc_svm(vm, &svm_gva);
>> +	vcpu_args_set(vm, VCPU_ID, 1, svm_gva);
>> +
>> +	for (;;) {
>> +		volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
>> +		struct ucall uc;
>> +
>> +		vcpu_run(vm, VCPU_ID);
>> +		TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
>> +			    "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
>> +			    run->exit_reason,
>> +			    exit_reason_str(run->exit_reason));
>> +
>> +		switch (get_ucall(vm, VCPU_ID, &uc)) {
>> +		case UCALL_ABORT:
>> +			TEST_ASSERT(false, "%s",
>> +				    (const char *)uc.args[0]);
>> +			/* NOT REACHED */
>> +		case UCALL_SYNC:
>> +			break;
>> +		case UCALL_DONE:
>> +			goto done;
>> +		default:
>> +			TEST_ASSERT(false,
>> +				    "Unknown ucall 0x%x.", uc.cmd);
>> +		}
>> +	}
>> +done:
>> +	kvm_vm_free(vm);
>> +	return 0;
>> +}
> 
> 

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Auger Eric]

Hi Krish,

On 2/6/20 11:46 PM, Krish Sadhukhan wrote:
> 
> 
> On 02/06/2020 02:47 AM, Eric Auger wrote:
>> L2 guest calls vmcall and L1 checks the exit status does
>> correspond.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>> Reviewed-by: Vitaly Kuznetsov <vkuznets@redhat.com>
>> Reviewed-by: Miaohe Lin <linmiaohe@huawei.com>
>>
>> ---
>>
>> v3 -> v4:
>> - remove useless includes
>> - collected Lin's R-b
>>
>> v2 -> v3:
>> - remove useless comment and add Vitaly's R-b
>> ---
>>   tools/testing/selftests/kvm/Makefile          |  1 +
>>   .../selftests/kvm/x86_64/svm_vmcall_test.c    | 79 +++++++++++++++++++
>>   2 files changed, 80 insertions(+)
>>   create mode 100644 tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>>
>> diff --git a/tools/testing/selftests/kvm/Makefile
>> b/tools/testing/selftests/kvm/Makefile
>> index 2e770f554cae..b529d3b42c02 100644
>> --- a/tools/testing/selftests/kvm/Makefile
>> +++ b/tools/testing/selftests/kvm/Makefile
>> @@ -26,6 +26,7 @@ TEST_GEN_PROGS_x86_64 += x86_64/vmx_dirty_log_test
>>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_set_nested_state_test
>>   TEST_GEN_PROGS_x86_64 += x86_64/vmx_tsc_adjust_test
>>   TEST_GEN_PROGS_x86_64 += x86_64/xss_msr_test
>> +TEST_GEN_PROGS_x86_64 += x86_64/svm_vmcall_test
>>   TEST_GEN_PROGS_x86_64 += clear_dirty_log_test
>>   TEST_GEN_PROGS_x86_64 += dirty_log_test
>>   TEST_GEN_PROGS_x86_64 += kvm_create_max_vcpus
>> diff --git a/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>> b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>> new file mode 100644
>> index 000000000000..6d3565aab94e
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/x86_64/svm_vmcall_test.c
>> @@ -0,0 +1,79 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * svm_vmcall_test
>> + *
>> + * Copyright (C) 2020, Red Hat, Inc.
>> + *
>> + * Nested SVM testing: VMCALL
>> + */
>> +
>> +#include "test_util.h"
>> +#include "kvm_util.h"
>> +#include "processor.h"
>> +#include "svm_util.h"
>> +
>> +#define VCPU_ID        5
>> +
>> +static struct kvm_vm *vm;
>> +
>> +static inline void l2_vmcall(struct svm_test_data *svm)
>> +{
>> +    __asm__ __volatile__("vmcall");
> Is it possible to re-use the existing vmcall() function ?
well the function is declared in vmx header. Also vmx_tsc_adjust_test
does not use it for instance. For this test the above is simple and does
the job.

> Also, we should probably re-name the function to 'l2_guest_code' which
> is used in the existing code and also it matches with 'l1_guest_code'
> naming.
OK
>> +}
>> +
>> +static void l1_guest_code(struct svm_test_data *svm)
>> +{
>> +    #define L2_GUEST_STACK_SIZE 64
>> +    unsigned long l2_guest_stack[L2_GUEST_STACK_SIZE];
>> +    struct vmcb *vmcb = svm->vmcb;
>> +
>> +    /* Prepare for L2 execution. */
>> +    generic_svm_setup(svm, l2_vmcall,
>> +              &l2_guest_stack[L2_GUEST_STACK_SIZE]);
>> +
>> +    run_guest(vmcb, svm->vmcb_gpa);
>> +
>> +    GUEST_ASSERT(vmcb->control.exit_code == SVM_EXIT_VMMCALL);
>> +    GUEST_DONE();
>> +}
>> +
>> +int main(int argc, char *argv[])
>> +{
>> +    vm_vaddr_t svm_gva;
>> +
>> +    nested_svm_check_supported();
>> +
>> +    vm = vm_create_default(VCPU_ID, 0, (void *) l1_guest_code);
>> +    vcpu_set_cpuid(vm, VCPU_ID, kvm_get_supported_cpuid());
>> +
>> +    vcpu_alloc_svm(vm, &svm_gva);
>> +    vcpu_args_set(vm, VCPU_ID, 1, svm_gva);
>> +
>> +    for (;;) {
>> +        volatile struct kvm_run *run = vcpu_state(vm, VCPU_ID);
>> +        struct ucall uc;
>> +
>> +        vcpu_run(vm, VCPU_ID);
>> +        TEST_ASSERT(run->exit_reason == KVM_EXIT_IO,
>> +                "Got exit_reason other than KVM_EXIT_IO: %u (%s)\n",
>> +                run->exit_reason,
>> +                exit_reason_str(run->exit_reason));
>> +
>> +        switch (get_ucall(vm, VCPU_ID, &uc)) {
>> +        case UCALL_ABORT:
>> +            TEST_ASSERT(false, "%s",
>> +                    (const char *)uc.args[0]);
>> +            /* NOT REACHED */
>> +        case UCALL_SYNC:
>> +            break;
>> +        case UCALL_DONE:
>> +            goto done;
>> +        default:
>> +            TEST_ASSERT(false,
>> +                    "Unknown ucall 0x%x.", uc.cmd);
>> +        }
>> +    }
>> +done:
>> +    kvm_vm_free(vm);
>> +    return 0;
>> +}
> 
Thanks

Eric

Re: [PATCH v4 2/3] selftests: KVM: AMD Nested test infrastructure

[Auger Eric]

Hi Krish,

On 2/6/20 11:57 PM, Krish Sadhukhan wrote:
> 
> 
> On 02/06/2020 02:47 AM, Eric Auger wrote:
>> Add the basic infrastructure needed to test AMD nested SVM.
>> This is largely copied from the KVM unit test infrastructure.
>>
>> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>>
>> ---
>>
>> v3 -> v4:
>> - just keep the 16 GPRs in gpr64_regs struct
>> - vm* instructions do not take any param
>> - add comments
>>
>> v2 -> v3:
>> - s/regs/gp_regs64
>> - Split the header into 2 parts: svm.h is a copy of
>>    arch/x86/include/asm/svm.h whereas svm_util.h contains
>>    testing add-ons
>> - use get_gdt/dt() and remove sgdt/sidt
>> - use get_es/ss/ds/cs
>> - fix clobber for dr6 & dr7
>> - use u64 instead of ulong
>> ---
>>   tools/testing/selftests/kvm/Makefile          |   2 +-
>>   .../selftests/kvm/include/x86_64/processor.h  |  20 ++
>>   .../selftests/kvm/include/x86_64/svm.h        | 297 ++++++++++++++++++
>>   .../selftests/kvm/include/x86_64/svm_util.h   |  38 +++
>>   tools/testing/selftests/kvm/lib/x86_64/svm.c  | 161 ++++++++++
>>   5 files changed, 517 insertions(+), 1 deletion(-)
>>   create mode 100644 tools/testing/selftests/kvm/include/x86_64/svm.h
>>   create mode 100644
>> tools/testing/selftests/kvm/include/x86_64/svm_util.h
>>   create mode 100644 tools/testing/selftests/kvm/lib/x86_64/svm.c
>>
>> diff --git a/tools/testing/selftests/kvm/Makefile
>> b/tools/testing/selftests/kvm/Makefile
>> index 608fa835c764..2e770f554cae 100644
>> --- a/tools/testing/selftests/kvm/Makefile
>> +++ b/tools/testing/selftests/kvm/Makefile
>> @@ -8,7 +8,7 @@ KSFT_KHDR_INSTALL := 1
>>   UNAME_M := $(shell uname -m)
>>     LIBKVM = lib/assert.c lib/elf.c lib/io.c lib/kvm_util.c
>> lib/sparsebit.c
>> -LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c
>> lib/x86_64/ucall.c
>> +LIBKVM_x86_64 = lib/x86_64/processor.c lib/x86_64/vmx.c
>> lib/x86_64/svm.c lib/x86_64/ucall.c
>>   LIBKVM_aarch64 = lib/aarch64/processor.c lib/aarch64/ucall.c
>>   LIBKVM_s390x = lib/s390x/processor.c lib/s390x/ucall.c
>>   diff --git a/tools/testing/selftests/kvm/include/x86_64/processor.h
>> b/tools/testing/selftests/kvm/include/x86_64/processor.h
>> index 6f7fffaea2e8..12475047869f 100644
>> --- a/tools/testing/selftests/kvm/include/x86_64/processor.h
>> +++ b/tools/testing/selftests/kvm/include/x86_64/processor.h
>> @@ -56,6 +56,26 @@ enum x86_register {
>>       R15,
>>   };
>>   +/* General Registers in 64-Bit Mode */
>> +struct gpr64_regs {
>> +    u64 rax;
>> +    u64 rcx;
>> +    u64 rdx;
>> +    u64 rbx;
>> +    u64 rsp;
>> +    u64 rbp;
>> +    u64 rsi;
>> +    u64 rdi;
>> +    u64 r8;
>> +    u64 r9;
>> +    u64 r10;
>> +    u64 r11;
>> +    u64 r12;
>> +    u64 r13;
>> +    u64 r14;
>> +    u64 r15;
>> +};
>> +
>>   struct desc64 {
>>       uint16_t limit0;
>>       uint16_t base0;
>> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm.h
>> b/tools/testing/selftests/kvm/include/x86_64/svm.h
>> new file mode 100644
>> index 000000000000..f4ea2355dbc2
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/include/x86_64/svm.h
>> @@ -0,0 +1,297 @@
>> +/* SPDX-License-Identifier: GPL-2.0 */
>> +/*
>> + * tools/testing/selftests/kvm/include/x86_64/svm.h
>> + * This is a copy of arch/x86/include/asm/svm.h
>> + *
>> + */
>> +
>> +#ifndef SELFTEST_KVM_SVM_H
>> +#define SELFTEST_KVM_SVM_H
>> +
>> +enum {
>> +    INTERCEPT_INTR,
>> +    INTERCEPT_NMI,
>> +    INTERCEPT_SMI,
>> +    INTERCEPT_INIT,
>> +    INTERCEPT_VINTR,
>> +    INTERCEPT_SELECTIVE_CR0,
>> +    INTERCEPT_STORE_IDTR,
>> +    INTERCEPT_STORE_GDTR,
>> +    INTERCEPT_STORE_LDTR,
>> +    INTERCEPT_STORE_TR,
>> +    INTERCEPT_LOAD_IDTR,
>> +    INTERCEPT_LOAD_GDTR,
>> +    INTERCEPT_LOAD_LDTR,
>> +    INTERCEPT_LOAD_TR,
>> +    INTERCEPT_RDTSC,
>> +    INTERCEPT_RDPMC,
>> +    INTERCEPT_PUSHF,
>> +    INTERCEPT_POPF,
>> +    INTERCEPT_CPUID,
>> +    INTERCEPT_RSM,
>> +    INTERCEPT_IRET,
>> +    INTERCEPT_INTn,
>> +    INTERCEPT_INVD,
>> +    INTERCEPT_PAUSE,
>> +    INTERCEPT_HLT,
>> +    INTERCEPT_INVLPG,
>> +    INTERCEPT_INVLPGA,
>> +    INTERCEPT_IOIO_PROT,
>> +    INTERCEPT_MSR_PROT,
>> +    INTERCEPT_TASK_SWITCH,
>> +    INTERCEPT_FERR_FREEZE,
>> +    INTERCEPT_SHUTDOWN,
>> +    INTERCEPT_VMRUN,
>> +    INTERCEPT_VMMCALL,
>> +    INTERCEPT_VMLOAD,
>> +    INTERCEPT_VMSAVE,
>> +    INTERCEPT_STGI,
>> +    INTERCEPT_CLGI,
>> +    INTERCEPT_SKINIT,
>> +    INTERCEPT_RDTSCP,
>> +    INTERCEPT_ICEBP,
>> +    INTERCEPT_WBINVD,
>> +    INTERCEPT_MONITOR,
>> +    INTERCEPT_MWAIT,
>> +    INTERCEPT_MWAIT_COND,
>> +    INTERCEPT_XSETBV,
>> +    INTERCEPT_RDPRU,
>> +};
>> +
>> +
>> +struct __attribute__ ((__packed__)) vmcb_control_area {
>> +    u32 intercept_cr;
>> +    u32 intercept_dr;
>> +    u32 intercept_exceptions;
>> +    u64 intercept;
>> +    u8 reserved_1[40];
>> +    u16 pause_filter_thresh;
>> +    u16 pause_filter_count;
>> +    u64 iopm_base_pa;
>> +    u64 msrpm_base_pa;
>> +    u64 tsc_offset;
>> +    u32 asid;
>> +    u8 tlb_ctl;
>> +    u8 reserved_2[3];
>> +    u32 int_ctl;
>> +    u32 int_vector;
>> +    u32 int_state;
>> +    u8 reserved_3[4];
>> +    u32 exit_code;
>> +    u32 exit_code_hi;
>> +    u64 exit_info_1;
>> +    u64 exit_info_2;
>> +    u32 exit_int_info;
>> +    u32 exit_int_info_err;
>> +    u64 nested_ctl;
>> +    u64 avic_vapic_bar;
>> +    u8 reserved_4[8];
>> +    u32 event_inj;
>> +    u32 event_inj_err;
>> +    u64 nested_cr3;
>> +    u64 virt_ext;
>> +    u32 clean;
>> +    u32 reserved_5;
>> +    u64 next_rip;
>> +    u8 insn_len;
>> +    u8 insn_bytes[15];
>> +    u64 avic_backing_page;    /* Offset 0xe0 */
>> +    u8 reserved_6[8];    /* Offset 0xe8 */
>> +    u64 avic_logical_id;    /* Offset 0xf0 */
>> +    u64 avic_physical_id;    /* Offset 0xf8 */
>> +    u8 reserved_7[768];
>> +};
>> +
>> +
>> +#define TLB_CONTROL_DO_NOTHING 0
>> +#define TLB_CONTROL_FLUSH_ALL_ASID 1
>> +#define TLB_CONTROL_FLUSH_ASID 3
>> +#define TLB_CONTROL_FLUSH_ASID_LOCAL 7
>> +
>> +#define V_TPR_MASK 0x0f
>> +
>> +#define V_IRQ_SHIFT 8
>> +#define V_IRQ_MASK (1 << V_IRQ_SHIFT)
>> +
>> +#define V_GIF_SHIFT 9
>> +#define V_GIF_MASK (1 << V_GIF_SHIFT)
>> +
>> +#define V_INTR_PRIO_SHIFT 16
>> +#define V_INTR_PRIO_MASK (0x0f << V_INTR_PRIO_SHIFT)
>> +
>> +#define V_IGN_TPR_SHIFT 20
>> +#define V_IGN_TPR_MASK (1 << V_IGN_TPR_SHIFT)
>> +
>> +#define V_INTR_MASKING_SHIFT 24
>> +#define V_INTR_MASKING_MASK (1 << V_INTR_MASKING_SHIFT)
>> +
>> +#define V_GIF_ENABLE_SHIFT 25
>> +#define V_GIF_ENABLE_MASK (1 << V_GIF_ENABLE_SHIFT)
>> +
>> +#define AVIC_ENABLE_SHIFT 31
>> +#define AVIC_ENABLE_MASK (1 << AVIC_ENABLE_SHIFT)
>> +
>> +#define LBR_CTL_ENABLE_MASK BIT_ULL(0)
>> +#define VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK BIT_ULL(1)
>> +
>> +#define SVM_INTERRUPT_SHADOW_MASK 1
>> +
>> +#define SVM_IOIO_STR_SHIFT 2
>> +#define SVM_IOIO_REP_SHIFT 3
>> +#define SVM_IOIO_SIZE_SHIFT 4
>> +#define SVM_IOIO_ASIZE_SHIFT 7
>> +
>> +#define SVM_IOIO_TYPE_MASK 1
>> +#define SVM_IOIO_STR_MASK (1 << SVM_IOIO_STR_SHIFT)
>> +#define SVM_IOIO_REP_MASK (1 << SVM_IOIO_REP_SHIFT)
>> +#define SVM_IOIO_SIZE_MASK (7 << SVM_IOIO_SIZE_SHIFT)
>> +#define SVM_IOIO_ASIZE_MASK (7 << SVM_IOIO_ASIZE_SHIFT)
>> +
>> +#define SVM_VM_CR_VALID_MASK    0x001fULL
>> +#define SVM_VM_CR_SVM_LOCK_MASK 0x0008ULL
>> +#define SVM_VM_CR_SVM_DIS_MASK  0x0010ULL
>> +
>> +#define SVM_NESTED_CTL_NP_ENABLE    BIT(0)
>> +#define SVM_NESTED_CTL_SEV_ENABLE    BIT(1)
>> +
>> +struct __attribute__ ((__packed__)) vmcb_seg {
>> +    u16 selector;
>> +    u16 attrib;
>> +    u32 limit;
>> +    u64 base;
>> +};
>> +
>> +struct __attribute__ ((__packed__)) vmcb_save_area {
>> +    struct vmcb_seg es;
>> +    struct vmcb_seg cs;
>> +    struct vmcb_seg ss;
>> +    struct vmcb_seg ds;
>> +    struct vmcb_seg fs;
>> +    struct vmcb_seg gs;
>> +    struct vmcb_seg gdtr;
>> +    struct vmcb_seg ldtr;
>> +    struct vmcb_seg idtr;
>> +    struct vmcb_seg tr;
>> +    u8 reserved_1[43];
>> +    u8 cpl;
>> +    u8 reserved_2[4];
>> +    u64 efer;
>> +    u8 reserved_3[112];
>> +    u64 cr4;
>> +    u64 cr3;
>> +    u64 cr0;
>> +    u64 dr7;
>> +    u64 dr6;
>> +    u64 rflags;
>> +    u64 rip;
>> +    u8 reserved_4[88];
>> +    u64 rsp;
>> +    u8 reserved_5[24];
>> +    u64 rax;
>> +    u64 star;
>> +    u64 lstar;
>> +    u64 cstar;
>> +    u64 sfmask;
>> +    u64 kernel_gs_base;
>> +    u64 sysenter_cs;
>> +    u64 sysenter_esp;
>> +    u64 sysenter_eip;
>> +    u64 cr2;
>> +    u8 reserved_6[32];
>> +    u64 g_pat;
>> +    u64 dbgctl;
>> +    u64 br_from;
>> +    u64 br_to;
>> +    u64 last_excp_from;
>> +    u64 last_excp_to;
>> +};
>> +
>> +struct __attribute__ ((__packed__)) vmcb {
>> +    struct vmcb_control_area control;
>> +    struct vmcb_save_area save;
>> +};
>> +
>> +#define SVM_CPUID_FUNC 0x8000000a
>> +
>> +#define SVM_VM_CR_SVM_DISABLE 4
>> +
>> +#define SVM_SELECTOR_S_SHIFT 4
>> +#define SVM_SELECTOR_DPL_SHIFT 5
>> +#define SVM_SELECTOR_P_SHIFT 7
>> +#define SVM_SELECTOR_AVL_SHIFT 8
>> +#define SVM_SELECTOR_L_SHIFT 9
>> +#define SVM_SELECTOR_DB_SHIFT 10
>> +#define SVM_SELECTOR_G_SHIFT 11
>> +
>> +#define SVM_SELECTOR_TYPE_MASK (0xf)
>> +#define SVM_SELECTOR_S_MASK (1 << SVM_SELECTOR_S_SHIFT)
>> +#define SVM_SELECTOR_DPL_MASK (3 << SVM_SELECTOR_DPL_SHIFT)
>> +#define SVM_SELECTOR_P_MASK (1 << SVM_SELECTOR_P_SHIFT)
>> +#define SVM_SELECTOR_AVL_MASK (1 << SVM_SELECTOR_AVL_SHIFT)
>> +#define SVM_SELECTOR_L_MASK (1 << SVM_SELECTOR_L_SHIFT)
>> +#define SVM_SELECTOR_DB_MASK (1 << SVM_SELECTOR_DB_SHIFT)
>> +#define SVM_SELECTOR_G_MASK (1 << SVM_SELECTOR_G_SHIFT)
>> +
>> +#define SVM_SELECTOR_WRITE_MASK (1 << 1)
>> +#define SVM_SELECTOR_READ_MASK SVM_SELECTOR_WRITE_MASK
>> +#define SVM_SELECTOR_CODE_MASK (1 << 3)
>> +
>> +#define INTERCEPT_CR0_READ    0
>> +#define INTERCEPT_CR3_READ    3
>> +#define INTERCEPT_CR4_READ    4
>> +#define INTERCEPT_CR8_READ    8
>> +#define INTERCEPT_CR0_WRITE    (16 + 0)
>> +#define INTERCEPT_CR3_WRITE    (16 + 3)
>> +#define INTERCEPT_CR4_WRITE    (16 + 4)
>> +#define INTERCEPT_CR8_WRITE    (16 + 8)
>> +
>> +#define INTERCEPT_DR0_READ    0
>> +#define INTERCEPT_DR1_READ    1
>> +#define INTERCEPT_DR2_READ    2
>> +#define INTERCEPT_DR3_READ    3
>> +#define INTERCEPT_DR4_READ    4
>> +#define INTERCEPT_DR5_READ    5
>> +#define INTERCEPT_DR6_READ    6
>> +#define INTERCEPT_DR7_READ    7
>> +#define INTERCEPT_DR0_WRITE    (16 + 0)
>> +#define INTERCEPT_DR1_WRITE    (16 + 1)
>> +#define INTERCEPT_DR2_WRITE    (16 + 2)
>> +#define INTERCEPT_DR3_WRITE    (16 + 3)
>> +#define INTERCEPT_DR4_WRITE    (16 + 4)
>> +#define INTERCEPT_DR5_WRITE    (16 + 5)
>> +#define INTERCEPT_DR6_WRITE    (16 + 6)
>> +#define INTERCEPT_DR7_WRITE    (16 + 7)
>> +
>> +#define SVM_EVTINJ_VEC_MASK 0xff
>> +
>> +#define SVM_EVTINJ_TYPE_SHIFT 8
>> +#define SVM_EVTINJ_TYPE_MASK (7 << SVM_EVTINJ_TYPE_SHIFT)
>> +
>> +#define SVM_EVTINJ_TYPE_INTR (0 << SVM_EVTINJ_TYPE_SHIFT)
>> +#define SVM_EVTINJ_TYPE_NMI (2 << SVM_EVTINJ_TYPE_SHIFT)
>> +#define SVM_EVTINJ_TYPE_EXEPT (3 << SVM_EVTINJ_TYPE_SHIFT)
>> +#define SVM_EVTINJ_TYPE_SOFT (4 << SVM_EVTINJ_TYPE_SHIFT)
>> +
>> +#define SVM_EVTINJ_VALID (1 << 31)
>> +#define SVM_EVTINJ_VALID_ERR (1 << 11)
>> +
>> +#define SVM_EXITINTINFO_VEC_MASK SVM_EVTINJ_VEC_MASK
>> +#define SVM_EXITINTINFO_TYPE_MASK SVM_EVTINJ_TYPE_MASK
>> +
>> +#define    SVM_EXITINTINFO_TYPE_INTR SVM_EVTINJ_TYPE_INTR
>> +#define    SVM_EXITINTINFO_TYPE_NMI SVM_EVTINJ_TYPE_NMI
>> +#define    SVM_EXITINTINFO_TYPE_EXEPT SVM_EVTINJ_TYPE_EXEPT
>> +#define    SVM_EXITINTINFO_TYPE_SOFT SVM_EVTINJ_TYPE_SOFT
>> +
>> +#define SVM_EXITINTINFO_VALID SVM_EVTINJ_VALID
>> +#define SVM_EXITINTINFO_VALID_ERR SVM_EVTINJ_VALID_ERR
>> +
>> +#define SVM_EXITINFOSHIFT_TS_REASON_IRET 36
>> +#define SVM_EXITINFOSHIFT_TS_REASON_JMP 38
>> +#define SVM_EXITINFOSHIFT_TS_HAS_ERROR_CODE 44
>> +
>> +#define SVM_EXITINFO_REG_MASK 0x0F
>> +
>> +#define SVM_CR0_SELECTIVE_MASK (X86_CR0_TS | X86_CR0_MP)
>> +
>> +#endif /* SELFTEST_KVM_SVM_H */
>> diff --git a/tools/testing/selftests/kvm/include/x86_64/svm_util.h
>> b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
>> new file mode 100644
>> index 000000000000..cd037917fece
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/include/x86_64/svm_util.h
>> @@ -0,0 +1,38 @@
>> +/* SPDX-License-Identifier: GPL-2.0-only */
>> +/*
>> + * tools/testing/selftests/kvm/include/x86_64/svm_utils.h
>> + * Header for nested SVM testing
>> + *
>> + * Copyright (C) 2020, Red Hat, Inc.
>> + */
>> +
>> +#ifndef SELFTEST_KVM_SVM_UTILS_H
>> +#define SELFTEST_KVM_SVM_UTILS_H
>> +
>> +#include <stdint.h>
>> +#include "svm.h"
>> +#include "processor.h"
>> +
>> +#define CPUID_SVM_BIT        2
>> +#define CPUID_SVM        BIT_ULL(CPUID_SVM_BIT)
>> +
>> +#define SVM_EXIT_VMMCALL    0x081
>> +
>> +struct svm_test_data {
>> +    /* VMCB */
>> +    struct vmcb *vmcb; /* gva */
>> +    void *vmcb_hva;
>> +    uint64_t vmcb_gpa;
>> +
>> +    /* host state-save area */
>> +    struct vmcb_save_area *save_area; /* gva */
>> +    void *save_area_hva;
>> +    uint64_t save_area_gpa;
>> +};
> Looks like vmcb_hva and save_area_hva haven't been used anywhere. Do we
> need them ?
Yes I can remove them for now.
>> +
>> +struct svm_test_data *vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t
>> *p_svm_gva);
>> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip,
>> void *guest_rsp);
>> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa);
>> +void nested_svm_check_supported(void);
>> +
>> +#endif /* SELFTEST_KVM_SVM_UTILS_H */
>> diff --git a/tools/testing/selftests/kvm/lib/x86_64/svm.c
>> b/tools/testing/selftests/kvm/lib/x86_64/svm.c
>> new file mode 100644
>> index 000000000000..6e05a8fc3fe0
>> --- /dev/null
>> +++ b/tools/testing/selftests/kvm/lib/x86_64/svm.c
>> @@ -0,0 +1,161 @@
>> +// SPDX-License-Identifier: GPL-2.0-only
>> +/*
>> + * tools/testing/selftests/kvm/lib/x86_64/svm.c
>> + * Helpers used for nested SVM testing
>> + * Largely inspired from KVM unit test svm.c
>> + *
>> + * Copyright (C) 2020, Red Hat, Inc.
>> + */
>> +
>> +#include "test_util.h"
>> +#include "kvm_util.h"
>> +#include "../kvm_util_internal.h"
>> +#include "processor.h"
>> +#include "svm_util.h"
>> +
>> +struct gpr64_regs guest_regs;
>> +u64 rflags;
>> +
>> +/* Allocate memory regions for nested SVM tests.
>> + *
>> + * Input Args:
>> + *   vm - The VM to allocate guest-virtual addresses in.
>> + *
>> + * Output Args:
>> + *   p_svm_gva - The guest virtual address for the struct svm_test_data.
>> + *
>> + * Return:
>> + *   Pointer to structure with the addresses of the SVM areas.
>> + */
>> +struct svm_test_data *
>> +vcpu_alloc_svm(struct kvm_vm *vm, vm_vaddr_t *p_svm_gva)
>> +{
>> +    vm_vaddr_t svm_gva = vm_vaddr_alloc(vm, getpagesize(),
>> +                        0x10000, 0, 0);
>> +    struct svm_test_data *svm = addr_gva2hva(vm, svm_gva);
>> +
>> +    svm->vmcb = (void *)vm_vaddr_alloc(vm, getpagesize(),
>> +                       0x10000, 0, 0);
>> +    svm->vmcb_hva = addr_gva2hva(vm, (uintptr_t)svm->vmcb);
>> +    svm->vmcb_gpa = addr_gva2gpa(vm, (uintptr_t)svm->vmcb);
>> +
>> +    svm->save_area = (void *)vm_vaddr_alloc(vm, getpagesize(),
>> +                        0x10000, 0, 0);
>> +    svm->save_area_hva = addr_gva2hva(vm, (uintptr_t)svm->save_area);
>> +    svm->save_area_gpa = addr_gva2gpa(vm, (uintptr_t)svm->save_area);
>> +
>> +    *p_svm_gva = svm_gva;
>> +    return svm;
>> +}
>> +
>> +static void vmcb_set_seg(struct vmcb_seg *seg, u16 selector,
>> +             u64 base, u32 limit, u32 attr)
>> +{
>> +    seg->selector = selector;
>> +    seg->attrib = attr;
>> +    seg->limit = limit;
>> +    seg->base = base;
>> +}
>> +
>> +void generic_svm_setup(struct svm_test_data *svm, void *guest_rip,
>> void *guest_rsp)
>> +{
>> +    struct vmcb *vmcb = svm->vmcb;
>> +    uint64_t vmcb_gpa = svm->vmcb_gpa;
>> +    struct vmcb_save_area *save = &vmcb->save;
>> +    struct vmcb_control_area *ctrl = &vmcb->control;
>> +    u32 data_seg_attr = 3 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
>> +          | SVM_SELECTOR_DB_MASK | SVM_SELECTOR_G_MASK;
>> +    u32 code_seg_attr = 9 | SVM_SELECTOR_S_MASK | SVM_SELECTOR_P_MASK
>> +        | SVM_SELECTOR_L_MASK | SVM_SELECTOR_G_MASK;
>> +    uint64_t efer;
>> +
>> +    efer = rdmsr(MSR_EFER);
>> +    wrmsr(MSR_EFER, efer | EFER_SVME);
>> +    wrmsr(MSR_VM_HSAVE_PA, svm->save_area_gpa);
>> +
>> +    memset(vmcb, 0, sizeof(*vmcb));
>> +    asm volatile ("vmsave\n\t" : : "a" (vmcb_gpa) : "memory");
>> +    vmcb_set_seg(&save->es, get_es(), 0, -1U, data_seg_attr);
>> +    vmcb_set_seg(&save->cs, get_cs(), 0, -1U, code_seg_attr);
>> +    vmcb_set_seg(&save->ss, get_ss(), 0, -1U, data_seg_attr);
>> +    vmcb_set_seg(&save->ds, get_ds(), 0, -1U, data_seg_attr);
>> +    vmcb_set_seg(&save->gdtr, 0, get_gdt().address, get_gdt().size, 0);
>> +    vmcb_set_seg(&save->idtr, 0, get_idt().address, get_idt().size, 0);
>> +
>> +    ctrl->asid = 1;
>> +    save->cpl = 0;
>> +    save->efer = rdmsr(MSR_EFER);
>> +    asm volatile ("mov %%cr4, %0" : "=r"(save->cr4) : : "memory");
>> +    asm volatile ("mov %%cr3, %0" : "=r"(save->cr3) : : "memory");
>> +    asm volatile ("mov %%cr0, %0" : "=r"(save->cr0) : : "memory");
>> +    asm volatile ("mov %%dr7, %0" : "=r"(save->dr7) : : "memory");
>> +    asm volatile ("mov %%dr6, %0" : "=r"(save->dr6) : : "memory");
>> +    asm volatile ("mov %%cr2, %0" : "=r"(save->cr2) : : "memory");
>> +    save->g_pat = rdmsr(MSR_IA32_CR_PAT);
>> +    save->dbgctl = rdmsr(MSR_IA32_DEBUGCTLMSR);
>> +    ctrl->intercept = (1ULL << INTERCEPT_VMRUN) |
>> +                (1ULL << INTERCEPT_VMMCALL);
>> +
>> +    vmcb->save.rip = (u64)guest_rip;
>> +    vmcb->save.rsp = (u64)guest_rsp;
>> +    guest_regs.rdi = (u64)svm;
>> +}
>> +
>> +/*
>> + * save/restore 64-bit general registers except rax, rip, rsp
>> + * which are directly handed through the VMCB guest processor state
>> + */
>> +#define SAVE_GPR_C                \
>> +    "xchg %%rbx, guest_regs+0x20\n\t"    \
>> +    "xchg %%rcx, guest_regs+0x10\n\t"    \
>> +    "xchg %%rdx, guest_regs+0x18\n\t"    \
>> +    "xchg %%rbp, guest_regs+0x30\n\t"    \
>> +    "xchg %%rsi, guest_regs+0x38\n\t"    \
>> +    "xchg %%rdi, guest_regs+0x40\n\t"    \
>> +    "xchg %%r8,  guest_regs+0x48\n\t"    \
>> +    "xchg %%r9,  guest_regs+0x50\n\t"    \
>> +    "xchg %%r10, guest_regs+0x58\n\t"    \
>> +    "xchg %%r11, guest_regs+0x60\n\t"    \
>> +    "xchg %%r12, guest_regs+0x68\n\t"    \
>> +    "xchg %%r13, guest_regs+0x70\n\t"    \
>> +    "xchg %%r14, guest_regs+0x78\n\t"    \
>> +    "xchg %%r15, guest_regs+0x80\n\t"
>> +
>> +#define LOAD_GPR_C      SAVE_GPR_C
>> +
>> +/*
>> + * selftests do not use interrupts so we dropped clgi/sti/cli/stgi
>> + * for now. registers involved in LOAD/SAVE_GPR_C are eventually
>> + * unmodified so they do not need to be in the clobber list.
>> + */
>> +void run_guest(struct vmcb *vmcb, uint64_t vmcb_gpa)
>> +{
>> +    asm volatile (
>> +        "vmload\n\t"
> Don't we need to set %rax before calling vmload ?
> 
>             "mov %[vmcb_gpa], %%rax \n\t"
>             "vmload %%rax\n\t"
> 
>> +        "mov rflags, %%r15\n\t"    // rflags
>> +        "mov %%r15, 0x170(%[vmcb])\n\t"
>> +        "mov guest_regs, %%r15\n\t"    // rax
>> +        "mov %%r15, 0x1f8(%[vmcb])\n\t"
>> +        LOAD_GPR_C
>> +        "vmrun\n\t"
>> +        SAVE_GPR_C
>> +        "mov 0x170(%[vmcb]), %%r15\n\t"    // rflags
>> +        "mov %%r15, rflags\n\t"
>> +        "mov 0x1f8(%[vmcb]), %%r15\n\t"    // rax
>> +        "mov %%r15, guest_regs\n\t"
>> +        "vmsave\n\t"
>> +        : : [vmcb] "r" (vmcb), [vmcb_gpa] "a" (vmcb_gpa)
as explained by Vitaly "a" (vmcb_gpa) does the job
>> +        : "r15", "memory");
>> +}
>> +
>> +void nested_svm_check_supported(void)
>> +{
>> +    struct kvm_cpuid_entry2 *entry =
>> +        kvm_get_supported_cpuid_entry(0x80000001);
>> +
>> +    if (!(entry->ecx & CPUID_SVM)) {
>> +        fprintf(stderr, "nested SVM not enabled, skipping test\n");
> I think a better message would be:
> 
>     "nested SVM not supported on this CPU, skipping test\n"
> 
> Also, the function should ideally return a boolean and let the callers
> print whatever they want.
This is inspired from nested_vmx_check_supported(). I think this can be
done later on and if we do we can change both messages/proto at the same
time.

Thank you for the review!

Best Regards

Eric
> 
>> +        exit(KSFT_SKIP);
>> +    }
>> +}
>> +
> 

Re: [PATCH v4 2/3] selftests: KVM: AMD Nested test infrastructure

[Paolo Bonzini]

On 06/02/20 23:57, Krish Sadhukhan wrote:
>>
>> +
>> +void nested_svm_check_supported(void)
>> +{
>> +    struct kvm_cpuid_entry2 *entry =
>> +        kvm_get_supported_cpuid_entry(0x80000001);
>> +
>> +    if (!(entry->ecx & CPUID_SVM)) {
>> +        fprintf(stderr, "nested SVM not enabled, skipping test\n");
> I think a better message would be:
> 
>     "nested SVM not supported on this CPU, skipping test\n"
> 
> Also, the function should ideally return a boolean and let the callers
> print whatever they want.

It would be "not supported by KVM", which is equivalent to "not enabled"
for all purposes.

Thanks,

Paolo

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Paolo Bonzini]

On 07/02/20 11:15, Auger Eric wrote:
>> Probably rename the file to svm_nested_vmcall_test.c. This matches with
>> the naming convention of VMX's nested tests. Otherwise people might not know
>> it is a nested one.
> From what I understand, all the vmx_* (including vmx_tsc_adjust_test for
> instance) are related to nested. So I'd rather leave svm_ prefix for
> nested SVM.

That is not strictly necessary, as there could be tests for Intel or
AMD-specific bugs or features.  But in practice you are right, "vmx_"
right now means it's testing nested.  We can rename all of them to
"nvmx_*" and "nsvm_*", but in the meanwhile your patch does not
introduce any inconsistency.

Queued, thanks!

Paolo

Re: [PATCH v4 3/3] selftests: KVM: SVM: Add vmcall test

[Paolo Bonzini]

On 06/02/20 23:46, Krish Sadhukhan wrote:
>>
>> +
>> +static inline void l2_vmcall(struct svm_test_data *svm)
>> +{
>> +    __asm__ __volatile__("vmcall");
> Is it possible to re-use the existing vmcall() function ?

Technically the AMD opcode is "vmmcall".  Using vmcall() still makes
sense as it tests KVM's emulation of the Intel opcode.

> Also, we should probably re-name the function to 'l2_guest_code' which
> is used in the existing code and also it matches with 'l1_guest_code'
> naming.

Ok.  I also removed the "inline" which is really not used since we take
the address of the function.

Paolo