* [PATCH v2 00/21] arm64: KVM: world switch in C
@ 2015-11-27 18:49 Marc Zyngier
2015-11-27 18:49 ` [PATCH v2 01/21] arm64: Add macros to read/write system registers Marc Zyngier
` (21 more replies)
0 siblings, 22 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:49 UTC (permalink / raw)
To: linux-arm-kernel
Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
and mean piece of hand-crafted assembly code. Over time, features have
crept in, the code has become harder to maintain, and the smallest
change is a pain to introduce. The VHE patches are a prime example of
why this doesn't work anymore.
This series rewrites most of the existing assembly code in C, but keeps
the existing code structure in place (most function names will look
familiar to the reader). The biggest change is that we don't have to
deal with a static register allocation (the compiler does it for us),
we can easily follow structure and pointers, and only the lowest level
is still in assembly code. Oh, and a negative diffstat.
There is still a healthy dose of inline assembly (system register
accessors, runtime code patching), but I've tried not to make it too
invasive. The generated code, while not exactly brilliant, doesn't
look too shaby. I do expect a small performance degradation, but I
believe this is something we can improve over time (my initial
measurements don't show any obvious regression though).
Eventually (and assuming people are happy with the general approach
taken here), it should be possible to make the 32bit converge with
this and reuse some parts of the code.
Patches are against 4.4-rc2 (mostly), and I've pushed a branch out
(kvm-arm64/wsinc). This has been tested on Juno, Seattle and the FVP
model. I also have pushed out kvm-arm64/vhe-wsinc that implements VHE
on top of these patches.
* From v1:
- A number of bugs have been squashed: vgic, FP/SIMD, debug, panic
(thanks to Mark, Steve, Ard and Alex for their reviews)
- Rebased on 4.4-rc2 plus KVM fixes aimed at -rc3 plus a couple
of gic-v3 fixes
Marc Zyngier (20):
arm64: KVM: Add a HYP-specific header file
arm64: KVM: Implement vgic-v2 save/restore
arm64: KVM: Implement vgic-v3 save/restore
arm64: KVM: Implement timer save/restore
arm64: KVM: Implement system register save/restore
arm64: KVM: Implement 32bit system register save/restore
arm64: KVM: Implement debug save/restore
arm64: KVM: Implement guest entry
arm64: KVM: Add patchable function selector
arm64: KVM: Implement the core world switch
arm64: KVM: Implement fpsimd save/restore
arm64: KVM: Implement TLB handling
arm64: KVM: HYP mode entry points
arm64: KVM: Add panic handling
arm64: KVM: Add compatibility aliases
arm64: KVM: Map the kernel RO section into HYP
arm64: KVM: Move away from the assembly version of the world switch
arm64: KVM: Turn system register numbers to an enum
arm64: KVM: Cleanup asm-offset.c
arm64: KVM: Remove weak attributes
Mark Rutland (1):
arm64: Add macros to read/write system registers
arch/arm/kvm/arm.c | 7 +
arch/arm64/include/asm/kvm_asm.h | 76 ---
arch/arm64/include/asm/kvm_emulate.h | 1 -
arch/arm64/include/asm/kvm_host.h | 81 ++-
arch/arm64/include/asm/kvm_mmio.h | 1 -
arch/arm64/include/asm/sysreg.h | 17 +
arch/arm64/kernel/asm-offsets.c | 40 +-
arch/arm64/kvm/Makefile | 3 +-
arch/arm64/kvm/guest.c | 1 -
arch/arm64/kvm/handle_exit.c | 1 +
arch/arm64/kvm/hyp.S | 1081 +---------------------------------
arch/arm64/kvm/hyp/Makefile | 14 +
arch/arm64/kvm/hyp/debug-sr.c | 133 +++++
arch/arm64/kvm/hyp/entry.S | 184 ++++++
arch/arm64/kvm/hyp/fpsimd.S | 33 ++
arch/arm64/kvm/hyp/hyp-entry.S | 207 +++++++
arch/arm64/kvm/hyp/hyp.h | 80 +++
arch/arm64/kvm/hyp/switch.c | 174 ++++++
arch/arm64/kvm/hyp/sysreg-sr.c | 132 +++++
arch/arm64/kvm/hyp/timer-sr.c | 71 +++
arch/arm64/kvm/hyp/tlb.c | 79 +++
arch/arm64/kvm/hyp/vgic-v2-sr.c | 89 +++
arch/arm64/kvm/hyp/vgic-v3-sr.c | 224 +++++++
arch/arm64/kvm/sys_regs.c | 1 +
arch/arm64/kvm/vgic-v2-switch.S | 134 -----
arch/arm64/kvm/vgic-v3-switch.S | 269 ---------
include/clocksource/arm_arch_timer.h | 6 +
virt/kvm/arm/vgic-v3.c | 1 +
28 files changed, 1536 insertions(+), 1604 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/Makefile
create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
create mode 100644 arch/arm64/kvm/hyp/entry.S
create mode 100644 arch/arm64/kvm/hyp/fpsimd.S
create mode 100644 arch/arm64/kvm/hyp/hyp-entry.S
create mode 100644 arch/arm64/kvm/hyp/hyp.h
create mode 100644 arch/arm64/kvm/hyp/switch.c
create mode 100644 arch/arm64/kvm/hyp/sysreg-sr.c
create mode 100644 arch/arm64/kvm/hyp/timer-sr.c
create mode 100644 arch/arm64/kvm/hyp/tlb.c
create mode 100644 arch/arm64/kvm/hyp/vgic-v2-sr.c
create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
delete mode 100644 arch/arm64/kvm/vgic-v2-switch.S
delete mode 100644 arch/arm64/kvm/vgic-v3-switch.S
--
2.1.4
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 01/21] arm64: Add macros to read/write system registers
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
@ 2015-11-27 18:49 ` Marc Zyngier
2015-11-30 20:00 ` Christoffer Dall
2015-11-27 18:49 ` [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file Marc Zyngier
` (20 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:49 UTC (permalink / raw)
To: linux-arm-kernel
From: Mark Rutland <mark.rutland@arm.com>
Rather than crafting custom macros for reading/writing each system
register provide generics accessors, read_sysreg and write_sysreg, for
this purpose.
Unlike read_cpuid, calls to read_exception_reg are never expected
to be optimized away or replaced with synthetic values.
Signed-off-by: Mark Rutland <mark.rutland@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Suzuki Poulose <suzuki.poulose@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/include/asm/sysreg.h | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index d48ab5b..c9c283a 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -20,6 +20,8 @@
#ifndef __ASM_SYSREG_H
#define __ASM_SYSREG_H
+#include <linux/stringify.h>
+
#include <asm/opcodes.h>
/*
@@ -208,6 +210,8 @@
#else
+#include <linux/types.h>
+
asm(
" .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
" .equ __reg_num_x\\num, \\num\n"
@@ -232,6 +236,19 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
val |= set;
asm volatile("msr sctlr_el1, %0" : : "r" (val));
}
+
+#define read_sysreg(r) ({ \
+ u64 __val; \
+ asm volatile("mrs %0, " __stringify(r) : "=r" (__val)); \
+ __val; \
+})
+
+#define write_sysreg(v, r) do { \
+ u64 __val = (u64)v; \
+ asm volatile("msr " __stringify(r) ", %0" \
+ : : "r" (__val)); \
+} while (0)
+
#endif
#endif /* __ASM_SYSREG_H */
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
2015-11-27 18:49 ` [PATCH v2 01/21] arm64: Add macros to read/write system registers Marc Zyngier
@ 2015-11-27 18:49 ` Marc Zyngier
2015-11-30 20:00 ` Christoffer Dall
2015-11-27 18:49 ` [PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore Marc Zyngier
` (19 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:49 UTC (permalink / raw)
To: linux-arm-kernel
In order to expose the various EL2 services that are private to
the hypervisor, add a new hyp.h file.
So far, it only contains mundane things such as section annotation
and VA manipulation.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/hyp.h | 31 +++++++++++++++++++++++++++++++
1 file changed, 31 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/hyp.h
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
new file mode 100644
index 0000000..dac843e
--- /dev/null
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ARM64_KVM_HYP_H__
+#define __ARM64_KVM_HYP_H__
+
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+#include <asm/kvm_mmu.h>
+#include <asm/sysreg.h>
+
+#define __hyp_text __section(.hyp.text) notrace
+
+#define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
+
+#endif /* __ARM64_KVM_HYP_H__ */
+
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
2015-11-27 18:49 ` [PATCH v2 01/21] arm64: Add macros to read/write system registers Marc Zyngier
2015-11-27 18:49 ` [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file Marc Zyngier
@ 2015-11-27 18:49 ` Marc Zyngier
2015-11-30 20:00 ` Christoffer Dall
2015-11-27 18:49 ` [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore Marc Zyngier
` (18 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:49 UTC (permalink / raw)
To: linux-arm-kernel
Implement the vgic-v2 save restore (mostly) as a direct translation
of the assembly code version.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/Makefile | 1 +
arch/arm64/kvm/hyp/Makefile | 5 +++
arch/arm64/kvm/hyp/hyp.h | 3 ++
arch/arm64/kvm/hyp/vgic-v2-sr.c | 89 +++++++++++++++++++++++++++++++++++++++++
4 files changed, 98 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/Makefile
create mode 100644 arch/arm64/kvm/hyp/vgic-v2-sr.c
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index 1949fe5..d31e4e5 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -10,6 +10,7 @@ KVM=../../../virt/kvm
ARM=../../../arch/arm/kvm
obj-$(CONFIG_KVM_ARM_HOST) += kvm.o
+obj-$(CONFIG_KVM_ARM_HOST) += hyp/
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o $(KVM)/vfio.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/arm.o $(ARM)/mmu.o $(ARM)/mmio.o
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
new file mode 100644
index 0000000..d8d5968
--- /dev/null
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -0,0 +1,5 @@
+#
+# Makefile for Kernel-based Virtual Machine module, HYP part
+#
+
+obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index dac843e..78f25c4 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -27,5 +27,8 @@
#define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
+void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
+
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
new file mode 100644
index 0000000..29a5c1d
--- /dev/null
+++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
@@ -0,0 +1,89 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/compiler.h>
+#include <linux/irqchip/arm-gic.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_mmu.h>
+
+#include "hyp.h"
+
+/* vcpu is already in the HYP VA space */
+void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+ void __iomem *base = kern_hyp_va(vgic->vctrl_base);
+ u32 __iomem *lr_base;
+ u32 eisr0, eisr1, elrsr0, elrsr1;
+ int i = 0, nr_lr;
+
+ if (!base)
+ return;
+
+ nr_lr = vcpu->arch.vgic_cpu.nr_lr;
+ cpu_if->vgic_vmcr = readl_relaxed(base + GICH_VMCR);
+ cpu_if->vgic_misr = readl_relaxed(base + GICH_MISR);
+ eisr0 = readl_relaxed(base + GICH_EISR0);
+ elrsr0 = readl_relaxed(base + GICH_ELRSR0);
+ if (unlikely(nr_lr > 32)) {
+ eisr1 = readl_relaxed(base + GICH_EISR1);
+ elrsr1 = readl_relaxed(base + GICH_ELRSR1);
+ } else {
+ eisr1 = elrsr1 = 0;
+ }
+#ifdef CONFIG_CPU_BIG_ENDIAN
+ cpu_if->vgic_eisr = ((u64)eisr0 << 32) | eisr1;
+ cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
+#else
+ cpu_if->vgic_eisr = ((u64)eisr1 << 32) | eisr0;
+ cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
+#endif
+ cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
+
+ writel_relaxed(0, base + GICH_HCR);
+
+ lr_base = base + GICH_LR0;
+ do {
+ cpu_if->vgic_lr[i++] = readl_relaxed(lr_base++);
+ } while (--nr_lr);
+}
+
+void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+ struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
+ struct vgic_dist *vgic = &kvm->arch.vgic;
+ void __iomem *base = kern_hyp_va(vgic->vctrl_base);
+ u32 __iomem *lr_base;
+ unsigned int i = 0, nr_lr;
+
+ if (!base)
+ return;
+
+ writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
+ writel_relaxed(cpu_if->vgic_vmcr, base + GICH_VMCR);
+ writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
+
+ lr_base = base + GICH_LR0;
+ nr_lr = vcpu->arch.vgic_cpu.nr_lr;
+ do {
+ writel_relaxed(cpu_if->vgic_lr[i++], lr_base++);
+ } while (--nr_lr);
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (2 preceding siblings ...)
2015-11-27 18:49 ` [PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore Marc Zyngier
@ 2015-11-27 18:49 ` Marc Zyngier
2015-11-30 9:59 ` Alex Bennée
2015-11-30 19:50 ` Christoffer Dall
2015-11-27 18:49 ` [PATCH v2 05/21] arm64: KVM: Implement timer save/restore Marc Zyngier
` (17 subsequent siblings)
21 siblings, 2 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:49 UTC (permalink / raw)
To: linux-arm-kernel
Implement the vgic-v3 save restore as a direct translation of
the assembly code version.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/hyp.h | 3 +
arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 226 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index d8d5968..d1e38ce 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -3,3 +3,4 @@
#
obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 78f25c4..a31cb6e 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -30,5 +30,8 @@
void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
+void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
new file mode 100644
index 0000000..b490db5
--- /dev/null
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -0,0 +1,222 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/compiler.h>
+#include <linux/irqchip/arm-gic-v3.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_mmu.h>
+
+#include "hyp.h"
+
+/*
+ * We store LRs in reverse order to let the CPU deal with streaming
+ * access. Use this macro to make it look saner...
+ */
+#define LR_OFFSET(n) (15 - n)
+
+#define read_gicreg(r) \
+ ({ \
+ u64 reg; \
+ asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
+ reg; \
+ })
+
+#define write_gicreg(v,r) \
+ do { \
+ u64 __val = (v); \
+ asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
+ } while (0)
+
+/* vcpu is already in the HYP VA space */
+void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ u64 val;
+ u32 nr_lr, nr_pri;
+
+ /*
+ * Make sure stores to the GIC via the memory mapped interface
+ * are now visible to the system register interface.
+ */
+ dsb(st);
+
+ cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
+ cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
+ cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
+ cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
+
+ write_gicreg(0, ICH_HCR_EL2);
+ val = read_gicreg(ICH_VTR_EL2);
+ nr_lr = val & 0xf;
+ nr_pri = ((u32)val >> 29) + 1;
+
+ switch (nr_lr) {
+ case 15:
+ cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
+ case 14:
+ cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
+ case 13:
+ cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
+ case 12:
+ cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
+ case 11:
+ cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
+ case 10:
+ cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
+ case 9:
+ cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
+ case 8:
+ cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
+ case 7:
+ cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
+ case 6:
+ cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
+ case 5:
+ cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
+ case 4:
+ cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
+ case 3:
+ cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
+ case 2:
+ cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
+ case 1:
+ cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
+ case 0:
+ cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
+ }
+
+ switch (nr_pri) {
+ case 7:
+ cpu_if->vgic_ap0r[3] = read_gicreg(ICH_AP0R3_EL2);
+ cpu_if->vgic_ap0r[2] = read_gicreg(ICH_AP0R2_EL2);
+ case 6:
+ cpu_if->vgic_ap0r[1] = read_gicreg(ICH_AP0R1_EL2);
+ default:
+ cpu_if->vgic_ap0r[0] = read_gicreg(ICH_AP0R0_EL2);
+ }
+
+ switch (nr_pri) {
+ case 7:
+ cpu_if->vgic_ap1r[3] = read_gicreg(ICH_AP1R3_EL2);
+ cpu_if->vgic_ap1r[2] = read_gicreg(ICH_AP1R2_EL2);
+ case 6:
+ cpu_if->vgic_ap1r[1] = read_gicreg(ICH_AP1R1_EL2);
+ default:
+ cpu_if->vgic_ap1r[0] = read_gicreg(ICH_AP1R0_EL2);
+ }
+
+ write_gicreg(read_gicreg(ICC_SRE_EL2) | ICC_SRE_EL2_ENABLE,
+ ICC_SRE_EL2);
+ isb();
+ write_gicreg(1, ICC_SRE_EL1);
+}
+
+void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
+{
+ struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
+ u64 val;
+ u32 nr_lr, nr_pri;
+
+ /* Make sure SRE is valid before writing the other registers */
+ write_gicreg(cpu_if->vgic_sre, ICC_SRE_EL1);
+ isb();
+
+ write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
+ write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
+
+ val = read_gicreg(ICH_VTR_EL2);
+ nr_lr = val & 0xf;
+ nr_pri = ((u32)val >> 29) + 1;
+
+ switch (nr_pri) {
+ case 7:
+ write_gicreg(cpu_if->vgic_ap1r[3], ICH_AP1R3_EL2);
+ write_gicreg(cpu_if->vgic_ap1r[2], ICH_AP1R2_EL2);
+ case 6:
+ write_gicreg(cpu_if->vgic_ap1r[1], ICH_AP1R1_EL2);
+ default:
+ write_gicreg(cpu_if->vgic_ap1r[0], ICH_AP1R0_EL2);
+ }
+
+ switch (nr_pri) {
+ case 7:
+ write_gicreg(cpu_if->vgic_ap0r[3], ICH_AP0R3_EL2);
+ write_gicreg(cpu_if->vgic_ap0r[2], ICH_AP0R2_EL2);
+ case 6:
+ write_gicreg(cpu_if->vgic_ap0r[1], ICH_AP0R1_EL2);
+ default:
+ write_gicreg(cpu_if->vgic_ap0r[0], ICH_AP0R0_EL2);
+ }
+
+ switch (nr_lr) {
+ case 15:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(15)], ICH_LR15_EL2);
+ case 14:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(14)], ICH_LR14_EL2);
+ case 13:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(13)], ICH_LR13_EL2);
+ case 12:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(12)], ICH_LR12_EL2);
+ case 11:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(11)], ICH_LR11_EL2);
+ case 10:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(10)], ICH_LR10_EL2);
+ case 9:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(9)], ICH_LR9_EL2);
+ case 8:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(8)], ICH_LR8_EL2);
+ case 7:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(7)], ICH_LR7_EL2);
+ case 6:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(6)], ICH_LR6_EL2);
+ case 5:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(5)], ICH_LR5_EL2);
+ case 4:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(4)], ICH_LR4_EL2);
+ case 3:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(3)], ICH_LR3_EL2);
+ case 2:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(2)], ICH_LR2_EL2);
+ case 1:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(1)], ICH_LR1_EL2);
+ case 0:
+ write_gicreg(cpu_if->vgic_lr[LR_OFFSET(0)], ICH_LR0_EL2);
+ }
+
+ /*
+ * Ensure that the above will have reached the
+ * (re)distributors. This ensure the guest will read the
+ * correct values from the memory-mapped interface.
+ */
+ isb();
+ dsb(sy);
+
+ /*
+ * Prevent the guest from touching the GIC system registers if
+ * SRE isn't enabled for GICv3 emulation.
+ */
+ if (!cpu_if->vgic_sre) {
+ write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
+ ICC_SRE_EL2);
+ }
+}
+
+u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
+{
+ return read_gicreg(ICH_VTR_EL2);
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 05/21] arm64: KVM: Implement timer save/restore
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (3 preceding siblings ...)
2015-11-27 18:49 ` [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore Marc Zyngier
@ 2015-11-27 18:49 ` Marc Zyngier
2015-11-30 19:59 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 06/21] arm64: KVM: Implement system register save/restore Marc Zyngier
` (16 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:49 UTC (permalink / raw)
To: linux-arm-kernel
Implement the timer save restore as a direct translation of
the assembly code version.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/hyp.h | 3 ++
arch/arm64/kvm/hyp/timer-sr.c | 71 ++++++++++++++++++++++++++++++++++++
include/clocksource/arm_arch_timer.h | 6 +++
4 files changed, 81 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/timer-sr.c
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index d1e38ce..455dc0a 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -4,3 +4,4 @@
obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index a31cb6e..86aa5a2 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -33,5 +33,8 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
+void __timer_save_state(struct kvm_vcpu *vcpu);
+void __timer_restore_state(struct kvm_vcpu *vcpu);
+
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/timer-sr.c b/arch/arm64/kvm/hyp/timer-sr.c
new file mode 100644
index 0000000..8e2209c
--- /dev/null
+++ b/arch/arm64/kvm/hyp/timer-sr.c
@@ -0,0 +1,71 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <clocksource/arm_arch_timer.h>
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_mmu.h>
+
+#include "hyp.h"
+
+/* vcpu is already in the HYP VA space */
+void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+ struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+
+ if (kvm->arch.timer.enabled) {
+ timer->cntv_ctl = read_sysreg(cntv_ctl_el0);
+ isb();
+ timer->cntv_cval = read_sysreg(cntv_cval_el0);
+ }
+
+ /* Disable the virtual timer */
+ write_sysreg(0, cntv_ctl_el0);
+
+ /* Allow physical timer/counter access for the host */
+ write_sysreg((read_sysreg(cnthctl_el2) | CNTHCTL_EL1PCTEN |
+ CNTHCTL_EL1PCEN),
+ cnthctl_el2);
+
+ /* Clear cntvoff for the host */
+ write_sysreg(0, cntvoff_el2);
+}
+
+void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+ struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+ u64 val;
+
+ /*
+ * Disallow physical timer access for the guest
+ * Physical counter access is allowed
+ */
+ val = read_sysreg(cnthctl_el2);
+ val &= ~CNTHCTL_EL1PCEN;
+ val |= CNTHCTL_EL1PCTEN;
+ write_sysreg(val, cnthctl_el2);
+
+ if (kvm->arch.timer.enabled) {
+ write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
+ write_sysreg(timer->cntv_cval, cntv_cval_el0);
+ isb();
+ write_sysreg(timer->cntv_ctl, cntv_ctl_el0);
+ }
+}
diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm_arch_timer.h
index 9916d0e..25d0914 100644
--- a/include/clocksource/arm_arch_timer.h
+++ b/include/clocksource/arm_arch_timer.h
@@ -23,6 +23,12 @@
#define ARCH_TIMER_CTRL_IT_MASK (1 << 1)
#define ARCH_TIMER_CTRL_IT_STAT (1 << 2)
+#define CNTHCTL_EL1PCTEN (1 << 0)
+#define CNTHCTL_EL1PCEN (1 << 1)
+#define CNTHCTL_EVNTEN (1 << 2)
+#define CNTHCTL_EVNTDIR (1 << 3)
+#define CNTHCTL_EVNTI (0xF << 4)
+
enum arch_timer_reg {
ARCH_TIMER_REG_CTRL,
ARCH_TIMER_REG_TVAL,
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 06/21] arm64: KVM: Implement system register save/restore
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (4 preceding siblings ...)
2015-11-27 18:49 ` [PATCH v2 05/21] arm64: KVM: Implement timer save/restore Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-01 15:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 07/21] arm64: KVM: Implement 32bit " Marc Zyngier
` (15 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Implement the system registe save restore as a direct translation of
the assembly code version.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/hyp.h | 3 ++
arch/arm64/kvm/hyp/sysreg-sr.c | 90 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 94 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/sysreg-sr.c
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 455dc0a..ec94200 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -5,3 +5,4 @@
obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 86aa5a2..087d3a5 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -36,5 +36,8 @@ void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
void __timer_save_state(struct kvm_vcpu *vcpu);
void __timer_restore_state(struct kvm_vcpu *vcpu);
+void __sysreg_save_state(struct kvm_cpu_context *ctxt);
+void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
+
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
new file mode 100644
index 0000000..add8fcb
--- /dev/null
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -0,0 +1,90 @@
+/*
+ * Copyright (C) 2012-2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_mmu.h>
+
+#include "hyp.h"
+
+/* ctxt is already in the HYP VA space */
+void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
+{
+ ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
+ ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
+ ctxt->sys_regs[SCTLR_EL1] = read_sysreg(sctlr_el1);
+ ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
+ ctxt->sys_regs[CPACR_EL1] = read_sysreg(cpacr_el1);
+ ctxt->sys_regs[TTBR0_EL1] = read_sysreg(ttbr0_el1);
+ ctxt->sys_regs[TTBR1_EL1] = read_sysreg(ttbr1_el1);
+ ctxt->sys_regs[TCR_EL1] = read_sysreg(tcr_el1);
+ ctxt->sys_regs[ESR_EL1] = read_sysreg(esr_el1);
+ ctxt->sys_regs[AFSR0_EL1] = read_sysreg(afsr0_el1);
+ ctxt->sys_regs[AFSR1_EL1] = read_sysreg(afsr1_el1);
+ ctxt->sys_regs[FAR_EL1] = read_sysreg(far_el1);
+ ctxt->sys_regs[MAIR_EL1] = read_sysreg(mair_el1);
+ ctxt->sys_regs[VBAR_EL1] = read_sysreg(vbar_el1);
+ ctxt->sys_regs[CONTEXTIDR_EL1] = read_sysreg(contextidr_el1);
+ ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
+ ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
+ ctxt->sys_regs[TPIDR_EL1] = read_sysreg(tpidr_el1);
+ ctxt->sys_regs[AMAIR_EL1] = read_sysreg(amair_el1);
+ ctxt->sys_regs[CNTKCTL_EL1] = read_sysreg(cntkctl_el1);
+ ctxt->sys_regs[PAR_EL1] = read_sysreg(par_el1);
+ ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
+
+ ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
+ ctxt->gp_regs.regs.pc = read_sysreg(elr_el2);
+ ctxt->gp_regs.regs.pstate = read_sysreg(spsr_el2);
+ ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
+ ctxt->gp_regs.elr_el1 = read_sysreg(elr_el1);
+ ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg(spsr_el1);
+}
+
+void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
+{
+ write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
+ write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
+ write_sysreg(ctxt->sys_regs[SCTLR_EL1], sctlr_el1);
+ write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
+ write_sysreg(ctxt->sys_regs[CPACR_EL1], cpacr_el1);
+ write_sysreg(ctxt->sys_regs[TTBR0_EL1], ttbr0_el1);
+ write_sysreg(ctxt->sys_regs[TTBR1_EL1], ttbr1_el1);
+ write_sysreg(ctxt->sys_regs[TCR_EL1], tcr_el1);
+ write_sysreg(ctxt->sys_regs[ESR_EL1], esr_el1);
+ write_sysreg(ctxt->sys_regs[AFSR0_EL1], afsr0_el1);
+ write_sysreg(ctxt->sys_regs[AFSR1_EL1], afsr1_el1);
+ write_sysreg(ctxt->sys_regs[FAR_EL1], far_el1);
+ write_sysreg(ctxt->sys_regs[MAIR_EL1], mair_el1);
+ write_sysreg(ctxt->sys_regs[VBAR_EL1], vbar_el1);
+ write_sysreg(ctxt->sys_regs[CONTEXTIDR_EL1], contextidr_el1);
+ write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
+ write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
+ write_sysreg(ctxt->sys_regs[TPIDR_EL1], tpidr_el1);
+ write_sysreg(ctxt->sys_regs[AMAIR_EL1], amair_el1);
+ write_sysreg(ctxt->sys_regs[CNTKCTL_EL1], cntkctl_el1);
+ write_sysreg(ctxt->sys_regs[PAR_EL1], par_el1);
+ write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
+
+ write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
+ write_sysreg(ctxt->gp_regs.regs.pc, elr_el2);
+ write_sysreg(ctxt->gp_regs.regs.pstate, spsr_el2);
+ write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
+ write_sysreg(ctxt->gp_regs.elr_el1, elr_el1);
+ write_sysreg(ctxt->gp_regs.spsr[KVM_SPSR_EL1], spsr_el1);
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 07/21] arm64: KVM: Implement 32bit system register save/restore
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (5 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 06/21] arm64: KVM: Implement system register save/restore Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-01 15:52 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 08/21] arm64: KVM: Implement debug save/restore Marc Zyngier
` (14 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Implement the 32bit system register save restore as a direct
translation of the assembly code version.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/hyp.h | 2 ++
arch/arm64/kvm/hyp/sysreg-sr.c | 41 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+)
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 087d3a5..4639330 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -38,6 +38,8 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
void __sysreg_save_state(struct kvm_cpu_context *ctxt);
void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
+void __sysreg32_save_state(struct kvm_vcpu *vcpu);
+void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index add8fcb..3f81a4d 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -88,3 +88,44 @@ void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
write_sysreg(ctxt->gp_regs.elr_el1, elr_el1);
write_sysreg(ctxt->gp_regs.spsr[KVM_SPSR_EL1], spsr_el1);
}
+
+void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
+{
+ if (!(read_sysreg(hcr_el2) & HCR_RW)) {
+ u64 *spsr = vcpu->arch.ctxt.gp_regs.spsr;
+ u64 *sysreg = vcpu->arch.ctxt.sys_regs;
+
+ spsr[KVM_SPSR_ABT] = read_sysreg(spsr_abt);
+ spsr[KVM_SPSR_UND] = read_sysreg(spsr_und);
+ spsr[KVM_SPSR_IRQ] = read_sysreg(spsr_irq);
+ spsr[KVM_SPSR_FIQ] = read_sysreg(spsr_fiq);
+
+ sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
+ sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
+
+ if (!(read_sysreg(cptr_el2) & CPTR_EL2_TFP))
+ sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
+
+ if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
+ sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
+ }
+}
+
+void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
+{
+ if (!(read_sysreg(hcr_el2) & HCR_RW)) {
+ u64 *spsr = vcpu->arch.ctxt.gp_regs.spsr;
+ u64 *sysreg = vcpu->arch.ctxt.sys_regs;
+
+ write_sysreg(spsr[KVM_SPSR_ABT], spsr_abt);
+ write_sysreg(spsr[KVM_SPSR_UND], spsr_und);
+ write_sysreg(spsr[KVM_SPSR_IRQ], spsr_irq);
+ write_sysreg(spsr[KVM_SPSR_FIQ], spsr_fiq);
+
+ write_sysreg(sysreg[DACR32_EL2], dacr32_el2);
+ write_sysreg(sysreg[IFSR32_EL2], ifsr32_el2);
+
+ if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
+ write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
+ }
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (6 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 07/21] arm64: KVM: Implement 32bit " Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-11-30 12:00 ` Alex Bennée
2015-12-01 12:56 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 09/21] arm64: KVM: Implement guest entry Marc Zyngier
` (13 subsequent siblings)
21 siblings, 2 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Implement the debug save restore as a direct translation of
the assembly code version.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/hyp.h | 9 +++
3 files changed, 140 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index ec94200..ec14cac 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
new file mode 100644
index 0000000..a0b2b99
--- /dev/null
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -0,0 +1,130 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/compiler.h>
+#include <linux/kvm_host.h>
+
+#include <asm/kvm_mmu.h>
+
+#include "hyp.h"
+
+#define read_debug(r,n) read_sysreg(r##n##_el1)
+#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
+
+#define save_debug(ptr,reg,nr) \
+ switch (nr) { \
+ case 15: ptr[15] = read_debug(reg, 15); \
+ case 14: ptr[14] = read_debug(reg, 14); \
+ case 13: ptr[13] = read_debug(reg, 13); \
+ case 12: ptr[12] = read_debug(reg, 12); \
+ case 11: ptr[11] = read_debug(reg, 11); \
+ case 10: ptr[10] = read_debug(reg, 10); \
+ case 9: ptr[9] = read_debug(reg, 9); \
+ case 8: ptr[8] = read_debug(reg, 8); \
+ case 7: ptr[7] = read_debug(reg, 7); \
+ case 6: ptr[6] = read_debug(reg, 6); \
+ case 5: ptr[5] = read_debug(reg, 5); \
+ case 4: ptr[4] = read_debug(reg, 4); \
+ case 3: ptr[3] = read_debug(reg, 3); \
+ case 2: ptr[2] = read_debug(reg, 2); \
+ case 1: ptr[1] = read_debug(reg, 1); \
+ default: ptr[0] = read_debug(reg, 0); \
+ }
+
+#define restore_debug(ptr,reg,nr) \
+ switch (nr) { \
+ case 15: write_debug(ptr[15], reg, 15); \
+ case 14: write_debug(ptr[14], reg, 14); \
+ case 13: write_debug(ptr[13], reg, 13); \
+ case 12: write_debug(ptr[12], reg, 12); \
+ case 11: write_debug(ptr[11], reg, 11); \
+ case 10: write_debug(ptr[10], reg, 10); \
+ case 9: write_debug(ptr[9], reg, 9); \
+ case 8: write_debug(ptr[8], reg, 8); \
+ case 7: write_debug(ptr[7], reg, 7); \
+ case 6: write_debug(ptr[6], reg, 6); \
+ case 5: write_debug(ptr[5], reg, 5); \
+ case 4: write_debug(ptr[4], reg, 4); \
+ case 3: write_debug(ptr[3], reg, 3); \
+ case 2: write_debug(ptr[2], reg, 2); \
+ case 1: write_debug(ptr[1], reg, 1); \
+ default: write_debug(ptr[0], reg, 0); \
+ }
+
+void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt)
+{
+ if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
+ u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
+ int brps, wrps;
+
+ brps = (aa64dfr0 >> 12) & 0xf;
+ wrps = (aa64dfr0 >> 20) & 0xf;
+
+ save_debug(dbg->dbg_bcr, dbgbcr, brps);
+ save_debug(dbg->dbg_bvr, dbgbvr, brps);
+ save_debug(dbg->dbg_wcr, dbgwcr, wrps);
+ save_debug(dbg->dbg_wvr, dbgwvr, wrps);
+
+ ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
+ }
+}
+
+void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt)
+{
+ if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
+ u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
+ int brps, wrps;
+
+ brps = (aa64dfr0 >> 12) & 0xf;
+ wrps = (aa64dfr0 >> 20) & 0xf;
+
+ restore_debug(dbg->dbg_bcr, dbgbcr, brps);
+ restore_debug(dbg->dbg_bvr, dbgbvr, brps);
+ restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
+ restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
+
+ write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
+ }
+}
+
+void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
+{
+ if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
+ (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
+ vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
+
+ __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
+ kern_hyp_va(vcpu->arch.host_cpu_context));
+}
+
+void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
+{
+ if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
+ __debug_restore_state(vcpu, &vcpu->arch.host_debug_state,
+ kern_hyp_va(vcpu->arch.host_cpu_context));
+ vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
+ }
+}
+
+u32 __hyp_text __debug_read_mdcr_el2(void)
+{
+ return read_sysreg(mdcr_el2);
+}
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 4639330..2581232 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -41,5 +41,14 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
void __sysreg32_save_state(struct kvm_vcpu *vcpu);
void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
+void __debug_save_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt);
+void __debug_restore_state(struct kvm_vcpu *vcpu,
+ struct kvm_guest_debug_arch *dbg,
+ struct kvm_cpu_context *ctxt);
+void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
+void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
+
#endif /* __ARM64_KVM_HYP_H__ */
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 09/21] arm64: KVM: Implement guest entry
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (7 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 08/21] arm64: KVM: Implement debug save/restore Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-01 15:29 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 10/21] arm64: KVM: Add patchable function selector Marc Zyngier
` (12 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Contrary to the previous patch, the guest entry is fairly different
from its assembly counterpart, mostly because it is only concerned
with saving/restoring the GP registers, and nothing else.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/entry.S | 155 ++++++++++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/hyp.h | 2 +
3 files changed, 158 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/entry.S
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index ec14cac..1e1ff06 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -7,3 +7,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
+obj-$(CONFIG_KVM_ARM_HOST) += entry.o
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
new file mode 100644
index 0000000..2c4449a
--- /dev/null
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -0,0 +1,155 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/asm-offsets.h>
+#include <asm/assembler.h>
+#include <asm/fpsimdmacros.h>
+#include <asm/kvm.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_asm.h>
+#include <asm/kvm_mmu.h>
+
+#define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
+#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
+
+ .text
+ .pushsection .hyp.text, "ax"
+
+.macro save_common_regs ctxt
+ stp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
+ stp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
+ stp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
+ stp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
+ stp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
+ stp x29, lr, [\ctxt, #CPU_XREG_OFFSET(29)]
+.endm
+
+.macro restore_common_regs ctxt
+ ldp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
+ ldp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
+ ldp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
+ ldp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
+ ldp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
+ ldp x29, lr, [\ctxt, #CPU_XREG_OFFSET(29)]
+.endm
+
+.macro save_host_regs reg
+ save_common_regs \reg
+.endm
+
+.macro restore_host_regs reg
+ restore_common_regs \reg
+.endm
+
+.macro save_guest_regs
+ // x0 is the vcpu address
+ // x1 is the return code, do not corrupt!
+ // x2 is the cpu context
+ // x3 is a tmp register
+ // Guest's x0-x3 are on the stack
+
+ add x2, x0, #VCPU_CONTEXT
+
+ // Compute base to save registers
+ stp x4, x5, [x2, #CPU_XREG_OFFSET(4)]
+ stp x6, x7, [x2, #CPU_XREG_OFFSET(6)]
+ stp x8, x9, [x2, #CPU_XREG_OFFSET(8)]
+ stp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
+ stp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
+ stp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
+ stp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
+ str x18, [x2, #CPU_XREG_OFFSET(18)]
+
+ pop x6, x7 // x2, x3
+ pop x4, x5 // x0, x1
+
+ stp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
+ stp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
+
+ save_common_regs x2
+.endm
+
+.macro restore_guest_regs
+ // Assume vcpu in x0, clobbers everything else
+
+ add x2, x0, #VCPU_CONTEXT
+
+ // Prepare x0-x3 for later restore
+ ldp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
+ ldp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
+ push x4, x5 // Push x0-x3 on the stack
+ push x6, x7
+
+ // x4-x18
+ ldp x4, x5, [x2, #CPU_XREG_OFFSET(4)]
+ ldp x6, x7, [x2, #CPU_XREG_OFFSET(6)]
+ ldp x8, x9, [x2, #CPU_XREG_OFFSET(8)]
+ ldp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
+ ldp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
+ ldp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
+ ldp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
+ ldr x18, [x2, #CPU_XREG_OFFSET(18)]
+
+ // x19-x29, lr
+ restore_common_regs x2
+
+ // Last bits of the 64bit state
+ pop x2, x3
+ pop x0, x1
+
+ // Do not touch any register after this!
+.endm
+
+/*
+ * u64 __guest_enter(struct kvm_vcpu *vcpu,
+ * struct kvm_cpu_context *host_ctxt);
+ */
+ENTRY(__guest_enter)
+ // x0: vcpu
+ // x1: host_ctxt
+ // x2, x3: parameter registers
+ // x4-x18: clobbered by macros
+
+ save_host_regs x1
+
+ // Preserve vcpu & host_ctxt for use at exit time
+ stp x0, x1, [sp, #-16]!
+
+ restore_guest_regs
+ eret
+ENDPROC(__guest_enter)
+
+ENTRY(__guest_exit)
+ // x0: vcpu
+ // x1: return code
+ // x2-x3: free
+ // x4-x29,lr: vcpu regs
+ // vcpu x0-x3 on the stack
+ save_guest_regs
+
+ // Restore vcpu & host_ctxt from the stack
+ // (preserving return code in x1)
+ ldp x0, x2, [sp], #16
+ restore_host_regs x2
+
+ mov x0, x1
+ ret
+ENDPROC(__guest_exit)
+
+ /* Insert fault handling here */
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 2581232..7ac8e11 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -50,5 +50,7 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
+u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
+
#endif /* __ARM64_KVM_HYP_H__ */
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (8 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 09/21] arm64: KVM: Implement guest entry Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-01 15:39 ` Christoffer Dall
2015-12-02 22:34 ` Andrew Jones
2015-11-27 18:50 ` [PATCH v2 11/21] arm64: KVM: Implement the core world switch Marc Zyngier
` (11 subsequent siblings)
21 siblings, 2 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
KVM so far relies on code patching, and is likely to use it more
in the future. The main issue is that our alternative system works
at the instruction level, while we'd like to have alternatives at
the function level.
In order to cope with this, add the "hyp_alternate_select" macro that
outputs a brief sequence of code that in turn can be patched, allowing
al alternative function to be selected.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 7ac8e11..f0427ee 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -27,6 +27,22 @@
#define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
+/*
+ * Generates patchable code sequences that are used to switch between
+ * two implementations of a function, depending on the availability of
+ * a feature.
+ */
+#define hyp_alternate_select(fname, orig, alt, cond) \
+typeof(orig) * __hyp_text fname(void) \
+{ \
+ typeof(alt) *val = orig; \
+ asm volatile(ALTERNATIVE("nop \n", \
+ "mov %0, %1 \n", \
+ cond) \
+ : "+r" (val) : "r" (alt)); \
+ return val; \
+}
+
void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 11/21] arm64: KVM: Implement the core world switch
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (9 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 10/21] arm64: KVM: Add patchable function selector Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-01 15:55 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore Marc Zyngier
` (10 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Implement the core of the world switch in C. Not everything is there
yet, and there is nothing to re-enter the world switch either.
But this already outlines the code structure well enough.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/switch.c | 134 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 135 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/switch.c
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 1e1ff06..9c11b0f 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -8,3 +8,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += entry.o
+obj-$(CONFIG_KVM_ARM_HOST) += switch.o
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
new file mode 100644
index 0000000..d67ed9e
--- /dev/null
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -0,0 +1,134 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "hyp.h"
+
+static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
+{
+ u64 val;
+
+ /*
+ * We are about to set CPTR_EL2.TFP to trap all floating point
+ * register accesses to EL2, however, the ARM ARM clearly states that
+ * traps are only taken to EL2 if the operation would not otherwise
+ * trap to EL1. Therefore, always make sure that for 32-bit guests,
+ * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
+ */
+ val = vcpu->arch.hcr_el2;
+ if (!(val & HCR_RW)) {
+ write_sysreg(1 << 30, fpexc32_el2);
+ isb();
+ }
+ write_sysreg(val, hcr_el2);
+ write_sysreg(1 << 15, hstr_el2);
+ write_sysreg(CPTR_EL2_TTA | CPTR_EL2_TFP, cptr_el2);
+ write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
+}
+
+static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
+{
+ write_sysreg(HCR_RW, hcr_el2);
+ write_sysreg(0, hstr_el2);
+ write_sysreg(read_sysreg(mdcr_el2) & MDCR_EL2_HPMN_MASK, mdcr_el2);
+ write_sysreg(0, cptr_el2);
+}
+
+static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
+{
+ struct kvm *kvm = kern_hyp_va(vcpu->kvm);
+ write_sysreg(kvm->arch.vttbr, vttbr_el2);
+}
+
+static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
+{
+ write_sysreg(0, vttbr_el2);
+}
+
+static hyp_alternate_select(__vgic_call_save_state,
+ __vgic_v2_save_state, __vgic_v3_save_state,
+ ARM64_HAS_SYSREG_GIC_CPUIF);
+
+static hyp_alternate_select(__vgic_call_restore_state,
+ __vgic_v2_restore_state, __vgic_v3_restore_state,
+ ARM64_HAS_SYSREG_GIC_CPUIF);
+
+static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
+{
+ __vgic_call_save_state()(vcpu);
+ write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
+}
+
+static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
+{
+ u64 val;
+
+ val = read_sysreg(hcr_el2);
+ val |= HCR_INT_OVERRIDE;
+ val |= vcpu->arch.irq_lines;
+ write_sysreg(val, hcr_el2);
+
+ __vgic_call_restore_state()(vcpu);
+}
+
+int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpu_context *host_ctxt;
+ struct kvm_cpu_context *guest_ctxt;
+ u64 exit_code;
+
+ vcpu = kern_hyp_va(vcpu);
+ write_sysreg(vcpu, tpidr_el2);
+
+ host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ guest_ctxt = &vcpu->arch.ctxt;
+
+ __sysreg_save_state(host_ctxt);
+ __debug_cond_save_host_state(vcpu);
+
+ __activate_traps(vcpu);
+ __activate_vm(vcpu);
+
+ __vgic_restore_state(vcpu);
+ __timer_restore_state(vcpu);
+
+ /*
+ * We must restore the 32-bit state before the sysregs, thanks
+ * to Cortex-A57 erratum #852523.
+ */
+ __sysreg32_restore_state(vcpu);
+ __sysreg_restore_state(guest_ctxt);
+ __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
+
+ /* Jump in the fire! */
+ exit_code = __guest_enter(vcpu, host_ctxt);
+ /* And we're baaack! */
+
+ __sysreg_save_state(guest_ctxt);
+ __sysreg32_save_state(vcpu);
+ __timer_save_state(vcpu);
+ __vgic_save_state(vcpu);
+
+ __deactivate_traps(vcpu);
+ __deactivate_vm(vcpu);
+
+ __sysreg_restore_state(host_ctxt);
+
+ __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
+ __debug_cond_restore_host_state(vcpu);
+
+ return exit_code;
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (10 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 11/21] arm64: KVM: Implement the core world switch Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 13/21] arm64: KVM: Implement TLB handling Marc Zyngier
` (9 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Implement the fpsimd save restore, keeping the lazy part in
assembler (as returning to C would be overkill).
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/entry.S | 32 +++++++++++++++++++++++++++++++-
arch/arm64/kvm/hyp/fpsimd.S | 33 +++++++++++++++++++++++++++++++++
arch/arm64/kvm/hyp/hyp.h | 7 +++++++
arch/arm64/kvm/hyp/switch.c | 8 ++++++++
arch/arm64/kvm/hyp/sysreg-sr.c | 2 +-
6 files changed, 81 insertions(+), 2 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/fpsimd.S
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 9c11b0f..56238d0 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += entry.o
obj-$(CONFIG_KVM_ARM_HOST) += switch.o
+obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 2c4449a..7552922 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -27,6 +27,7 @@
#define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
+#define CPU_SYSREG_OFFSET(x) (CPU_SYSREGS + 8*x)
.text
.pushsection .hyp.text, "ax"
@@ -152,4 +153,33 @@ ENTRY(__guest_exit)
ret
ENDPROC(__guest_exit)
- /* Insert fault handling here */
+ENTRY(__fpsimd_guest_restore)
+ push x4, lr
+
+ mrs x2, cptr_el2
+ bic x2, x2, #CPTR_EL2_TFP
+ msr cptr_el2, x2
+ isb
+
+ mrs x3, tpidr_el2
+
+ ldr x0, [x3, #VCPU_HOST_CONTEXT]
+ kern_hyp_va x0
+ add x0, x0, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
+ bl __fpsimd_save_state
+
+ add x2, x3, #VCPU_CONTEXT
+ add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
+ bl __fpsimd_restore_state
+
+ mrs x1, hcr_el2
+ tbnz x1, #HCR_RW_SHIFT, 1f
+ ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
+ msr fpexc32_el2, x4
+1:
+ pop x4, lr
+ pop x2, x3
+ pop x0, x1
+
+ eret
+ENDPROC(__fpsimd_guest_restore)
diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S
new file mode 100644
index 0000000..da3f22c
--- /dev/null
+++ b/arch/arm64/kvm/hyp/fpsimd.S
@@ -0,0 +1,33 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/fpsimdmacros.h>
+
+ .text
+ .pushsection .hyp.text, "ax"
+
+ENTRY(__fpsimd_save_state)
+ fpsimd_save x0, 1
+ ret
+ENDPROC(__fpsimd_save_state)
+
+ENTRY(__fpsimd_restore_state)
+ fpsimd_restore x0, 1
+ ret
+ENDPROC(__fpsimd_restore_state)
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index f0427ee..18365dd 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -66,6 +66,13 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
+void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
+void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
+static inline bool __fpsimd_enabled(void)
+{
+ return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
+}
+
u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index d67ed9e..8affc19 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -88,6 +88,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
+ bool fp_enabled;
u64 exit_code;
vcpu = kern_hyp_va(vcpu);
@@ -117,6 +118,8 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
exit_code = __guest_enter(vcpu, host_ctxt);
/* And we're baaack! */
+ fp_enabled = __fpsimd_enabled();
+
__sysreg_save_state(guest_ctxt);
__sysreg32_save_state(vcpu);
__timer_save_state(vcpu);
@@ -127,6 +130,11 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
__sysreg_restore_state(host_ctxt);
+ if (fp_enabled) {
+ __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
+ __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
+ }
+
__debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
__debug_cond_restore_host_state(vcpu);
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 3f81a4d..41b9d30 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -103,7 +103,7 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
- if (!(read_sysreg(cptr_el2) & CPTR_EL2_TFP))
+ if (__fpsimd_enabled())
sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 13/21] arm64: KVM: Implement TLB handling
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (11 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 14/21] arm64: KVM: HYP mode entry points Marc Zyngier
` (8 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Implement the TLB handling as a direct translation of the assembly
code version.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/tlb.c | 72 +++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 73 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/tlb.c
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 56238d0..1a529f5 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -10,3 +10,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
obj-$(CONFIG_KVM_ARM_HOST) += entry.o
obj-$(CONFIG_KVM_ARM_HOST) += switch.o
obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
+obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
new file mode 100644
index 0000000..d4a07d0
--- /dev/null
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -0,0 +1,72 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "hyp.h"
+
+void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
+{
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ kvm = kern_hyp_va(kvm);
+ write_sysreg(kvm->arch.vttbr, vttbr_el2);
+ isb();
+
+ /*
+ * We could do so much better if we had the VA as well.
+ * Instead, we invalidate Stage-2 for this IPA, and the
+ * whole of Stage-1. Weep...
+ */
+ ipa >>= 12;
+ asm volatile("tlbi ipas2e1is, %0" : : "r" (ipa));
+ dsb(ish);
+ /*
+ * We have to ensure completion of the invalidation at Stage-2,
+ * since a table walk on another CPU could refill a TLB with a
+ * complete (S1 + S2) walk based on the old Stage-2 mapping if
+ * the Stage-1 invalidation happened first.
+ */
+ asm volatile("tlbi vmalle1is" : : );
+ dsb(ish);
+ isb();
+
+ write_sysreg(0, vttbr_el2);
+}
+
+void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
+{
+ dsb(ishst);
+
+ /* Switch to requested VMID */
+ kvm = kern_hyp_va(kvm);
+ write_sysreg(kvm->arch.vttbr, vttbr_el2);
+ isb();
+
+ asm volatile("tlbi vmalls12e1is" : : );
+ dsb(ish);
+ isb();
+
+ write_sysreg(0, vttbr_el2);
+}
+
+void __hyp_text __tlb_flush_vm_context(void)
+{
+ dsb(ishst);
+ asm volatile("tlbi alle1is \n"
+ "ic ialluis ": : );
+ dsb(ish);
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 14/21] arm64: KVM: HYP mode entry points
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (12 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 13/21] arm64: KVM: Implement TLB handling Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 15/21] arm64: KVM: Add panic handling Marc Zyngier
` (7 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Add the entry points for HYP mode (both for hypercalls and
exception handling).
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/Makefile | 1 +
arch/arm64/kvm/hyp/hyp-entry.S | 198 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 199 insertions(+)
create mode 100644 arch/arm64/kvm/hyp/hyp-entry.S
diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
index 1a529f5..826032b 100644
--- a/arch/arm64/kvm/hyp/Makefile
+++ b/arch/arm64/kvm/hyp/Makefile
@@ -11,3 +11,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += entry.o
obj-$(CONFIG_KVM_ARM_HOST) += switch.o
obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
+obj-$(CONFIG_KVM_ARM_HOST) += hyp-entry.o
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
new file mode 100644
index 0000000..8334407
--- /dev/null
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -0,0 +1,198 @@
+/*
+ * Copyright (C) 2015 - ARM Ltd
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include <linux/linkage.h>
+
+#include <asm/alternative.h>
+#include <asm/assembler.h>
+#include <asm/asm-offsets.h>
+#include <asm/cpufeature.h>
+#include <asm/kvm_arm.h>
+#include <asm/kvm_asm.h>
+#include <asm/kvm_mmu.h>
+
+ .text
+ .pushsection .hyp.text, "ax"
+
+el1_sync: // Guest trapped into EL2
+ push x0, x1
+ push x2, x3
+
+ mrs x1, esr_el2
+ lsr x2, x1, #ESR_ELx_EC_SHIFT
+
+ cmp x2, #ESR_ELx_EC_HVC64
+ b.ne el1_trap
+
+ mrs x3, vttbr_el2 // If vttbr is valid, the 64bit guest
+ cbnz x3, el1_trap // called HVC
+
+ /* Here, we're pretty sure the host called HVC. */
+ pop x2, x3
+ pop x0, x1
+
+ /* Check for __hyp_get_vectors */
+ cbnz x0, 1f
+ mrs x0, vbar_el2
+ b 2f
+
+1: push lr, xzr
+
+ /*
+ * Compute the function address in EL2, and shuffle the parameters.
+ */
+ kern_hyp_va x0
+ mov lr, x0
+ mov x0, x1
+ mov x1, x2
+ mov x2, x3
+ blr lr
+
+ pop lr, xzr
+2: eret
+
+el1_trap:
+ /*
+ * x1: ESR
+ * x2: ESR_EC
+ */
+
+ /* Guest accessed VFP/SIMD registers, save host, restore Guest */
+ cmp x2, #ESR_ELx_EC_FP_ASIMD
+ b.eq __fpsimd_guest_restore
+
+ cmp x2, #ESR_ELx_EC_DABT_LOW
+ mov x0, #ESR_ELx_EC_IABT_LOW
+ ccmp x2, x0, #4, ne
+ b.ne 1f // Not an abort we care about
+
+ /* This is an abort. Check for permission fault */
+alternative_if_not ARM64_WORKAROUND_834220
+ and x2, x1, #ESR_ELx_FSC_TYPE
+ cmp x2, #FSC_PERM
+ b.ne 1f // Not a permission fault
+alternative_else
+ nop // Use the permission fault path to
+ nop // check for a valid S1 translation,
+ nop // regardless of the ESR value.
+alternative_endif
+
+
+ /*
+ * Check for Stage-1 page table walk, which is guaranteed
+ * to give a valid HPFAR_EL2.
+ */
+ tbnz x1, #7, 1f // S1PTW is set
+
+ /* Preserve PAR_EL1 */
+ mrs x3, par_el1
+ push x3, xzr
+
+ /*
+ * Permission fault, HPFAR_EL2 is invalid.
+ * Resolve the IPA the hard way using the guest VA.
+ * Stage-1 translation already validated the memory access rights.
+ * As such, we can use the EL1 translation regime, and don't have
+ * to distinguish between EL0 and EL1 access.
+ */
+ mrs x2, far_el2
+ at s1e1r, x2
+ isb
+
+ /* Read result */
+ mrs x3, par_el1
+ pop x0, xzr // Restore PAR_EL1 from the stack
+ msr par_el1, x0
+ tbnz x3, #0, 3f // Bail out if we failed the translation
+ ubfx x3, x3, #12, #36 // Extract IPA
+ lsl x3, x3, #4 // and present it like HPFAR
+ b 2f
+
+1: mrs x3, hpfar_el2
+ mrs x2, far_el2
+
+2: mrs x0, tpidr_el2
+ str w1, [x0, #VCPU_ESR_EL2]
+ str x2, [x0, #VCPU_FAR_EL2]
+ str x3, [x0, #VCPU_HPFAR_EL2]
+
+ mov x1, #ARM_EXCEPTION_TRAP
+ b __guest_exit
+
+ /*
+ * Translation failed. Just return to the guest and
+ * let it fault again. Another CPU is probably playing
+ * behind our back.
+ */
+3: pop x2, x3
+ pop x0, x1
+
+ eret
+
+el1_irq:
+ push x0, x1
+ push x2, x3
+ mrs x0, tpidr_el2
+ mov x1, #ARM_EXCEPTION_IRQ
+ b __guest_exit
+
+.macro invalid_vector label, target = __kvm_hyp_panic
+ .align 2
+\label:
+ b \target
+ENDPROC(\label)
+.endm
+
+ /* None of these should ever happen */
+ invalid_vector el2t_sync_invalid
+ invalid_vector el2t_irq_invalid
+ invalid_vector el2t_fiq_invalid
+ invalid_vector el2t_error_invalid
+ invalid_vector el2h_sync_invalid
+ invalid_vector el2h_irq_invalid
+ invalid_vector el2h_fiq_invalid
+ invalid_vector el2h_error_invalid
+ invalid_vector el1_sync_invalid
+ invalid_vector el1_irq_invalid
+ invalid_vector el1_fiq_invalid
+ invalid_vector el1_error_invalid
+
+ .ltorg
+
+ .align 11
+
+ENTRY(__hyp_vector)
+ ventry el2t_sync_invalid // Synchronous EL2t
+ ventry el2t_irq_invalid // IRQ EL2t
+ ventry el2t_fiq_invalid // FIQ EL2t
+ ventry el2t_error_invalid // Error EL2t
+
+ ventry el2h_sync_invalid // Synchronous EL2h
+ ventry el2h_irq_invalid // IRQ EL2h
+ ventry el2h_fiq_invalid // FIQ EL2h
+ ventry el2h_error_invalid // Error EL2h
+
+ ventry el1_sync // Synchronous 64-bit EL1
+ ventry el1_irq // IRQ 64-bit EL1
+ ventry el1_fiq_invalid // FIQ 64-bit EL1
+ ventry el1_error_invalid // Error 64-bit EL1
+
+ ventry el1_sync // Synchronous 32-bit EL1
+ ventry el1_irq // IRQ 32-bit EL1
+ ventry el1_fiq_invalid // FIQ 32-bit EL1
+ ventry el1_error_invalid // Error 32-bit EL1
+ENDPROC(__hyp_vector)
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 15/21] arm64: KVM: Add panic handling
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (13 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 14/21] arm64: KVM: HYP mode entry points Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 16/21] arm64: KVM: Add compatibility aliases Marc Zyngier
` (6 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Add the panic handler, together with the small bits of assembly
code to call the kernel's panic implementation.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/hyp-entry.S | 11 ++++++++++-
arch/arm64/kvm/hyp/hyp.h | 1 +
arch/arm64/kvm/hyp/switch.c | 30 ++++++++++++++++++++++++++++++
3 files changed, 41 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 8334407..39d6935 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -150,7 +150,16 @@ el1_irq:
mov x1, #ARM_EXCEPTION_IRQ
b __guest_exit
-.macro invalid_vector label, target = __kvm_hyp_panic
+ENTRY(__hyp_do_panic)
+ mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\
+ PSR_MODE_EL1h)
+ msr spsr_el2, lr
+ ldr lr, =panic
+ msr elr_el2, lr
+ eret
+ENDPROC(__hyp_do_panic)
+
+.macro invalid_vector label, target = __hyp_panic
.align 2
\label:
b \target
diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
index 18365dd..87f16fa 100644
--- a/arch/arm64/kvm/hyp/hyp.h
+++ b/arch/arm64/kvm/hyp/hyp.h
@@ -74,6 +74,7 @@ static inline bool __fpsimd_enabled(void)
}
u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
+void __noreturn __hyp_do_panic(unsigned long, ...);
#endif /* __ARM64_KVM_HYP_H__ */
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 8affc19..7b81089 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -140,3 +140,33 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
return exit_code;
}
+
+static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
+
+void __hyp_text __noreturn __hyp_panic(void)
+{
+ unsigned long str_va = (unsigned long)__hyp_panic_string;
+ u64 spsr = read_sysreg(spsr_el2);
+ u64 elr = read_sysreg(elr_el2);
+ u64 par = read_sysreg(par_el1);
+
+ if (read_sysreg(vttbr_el2)) {
+ struct kvm_vcpu *vcpu;
+ struct kvm_cpu_context *host_ctxt;
+
+ vcpu = (struct kvm_vcpu *)read_sysreg(tpidr_el2);
+ host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
+ __deactivate_traps(vcpu);
+ __deactivate_vm(vcpu);
+ __sysreg_restore_state(host_ctxt);
+ }
+
+ /* Call panic for real */
+ __hyp_do_panic(str_va - HYP_PAGE_OFFSET + PAGE_OFFSET,
+ spsr, elr,
+ read_sysreg(esr_el2), read_sysreg(far_el2),
+ read_sysreg(hpfar_el2), par,
+ (void *)read_sysreg(tpidr_el2));
+
+ unreachable();
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 16/21] arm64: KVM: Add compatibility aliases
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (14 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 15/21] arm64: KVM: Add panic handling Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:49 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 17/21] arm64: KVM: Map the kernel RO section into HYP Marc Zyngier
` (5 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
So far, we've implemented the new world switch with a completely
different namespace, so that we could have both implementation
compiled in.
Let's take things one step further by adding weak aliases that
have the same names as the original implementation. The weak
attributes allows the new implementation to be overriden by the
old one, and everything still work.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/debug-sr.c | 3 +++
arch/arm64/kvm/hyp/hyp-entry.S | 3 +++
arch/arm64/kvm/hyp/switch.c | 3 +++
arch/arm64/kvm/hyp/tlb.c | 9 +++++++++
arch/arm64/kvm/hyp/vgic-v3-sr.c | 3 +++
5 files changed, 21 insertions(+)
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index a0b2b99..afd0a53 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -128,3 +128,6 @@ u32 __hyp_text __debug_read_mdcr_el2(void)
{
return read_sysreg(mdcr_el2);
}
+
+__alias(__debug_read_mdcr_el2)
+u32 __weak __kvm_get_mdcr_el2(void);
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index 39d6935..ace919b 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -184,6 +184,8 @@ ENDPROC(\label)
.align 11
+ .weak __kvm_hyp_vector
+ENTRY(__kvm_hyp_vector)
ENTRY(__hyp_vector)
ventry el2t_sync_invalid // Synchronous EL2t
ventry el2t_irq_invalid // IRQ EL2t
@@ -205,3 +207,4 @@ ENTRY(__hyp_vector)
ventry el1_fiq_invalid // FIQ 32-bit EL1
ventry el1_error_invalid // Error 32-bit EL1
ENDPROC(__hyp_vector)
+ENDPROC(__kvm_hyp_vector)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 7b81089..c8ba370 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -141,6 +141,9 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
return exit_code;
}
+__alias(__guest_run)
+int __weak __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+
static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
void __hyp_text __noreturn __hyp_panic(void)
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
index d4a07d0..2c279a8 100644
--- a/arch/arm64/kvm/hyp/tlb.c
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -47,6 +47,9 @@ void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
write_sysreg(0, vttbr_el2);
}
+__alias(__tlb_flush_vmid_ipa)
+void __weak __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
+
void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
{
dsb(ishst);
@@ -63,6 +66,9 @@ void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
write_sysreg(0, vttbr_el2);
}
+__alias(__tlb_flush_vmid)
+void __weak __kvm_tlb_flush_vmid(struct kvm *kvm);
+
void __hyp_text __tlb_flush_vm_context(void)
{
dsb(ishst);
@@ -70,3 +76,6 @@ void __hyp_text __tlb_flush_vm_context(void)
"ic ialluis ": : );
dsb(ish);
}
+
+__alias(__tlb_flush_vm_context)
+void __weak __kvm_flush_vm_context(void);
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index b490db5..1b0eedb 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -220,3 +220,6 @@ u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
{
return read_gicreg(ICH_VTR_EL2);
}
+
+__alias(__vgic_v3_read_ich_vtr_el2)
+u64 __weak __vgic_v3_get_ich_vtr_el2(void);
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 17/21] arm64: KVM: Map the kernel RO section into HYP
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (15 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 16/21] arm64: KVM: Add compatibility aliases Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:49 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 18/21] arm64: KVM: Move away from the assembly version of the world switch Marc Zyngier
` (4 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
In order to run C code in HYP, we must make sure that the kernel's
RO section in mapped into HYP (otherwise things break badly).
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm/kvm/arm.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index e06fd29..d80aefe 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -44,6 +44,7 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_coproc.h>
#include <asm/kvm_psci.h>
+#include <asm/sections.h>
#ifdef REQUIRES_VIRT
__asm__(".arch_extension virt");
@@ -1067,6 +1068,12 @@ static int init_hyp_mode(void)
goto out_free_mappings;
}
+ err = create_hyp_mappings(__start_rodata, __end_rodata);
+ if (err) {
+ kvm_err("Cannot map rodata section\n");
+ goto out_free_mappings;
+ }
+
/*
* Map the Hyp stack pages
*/
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 18/21] arm64: KVM: Move away from the assembly version of the world switch
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (16 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 17/21] arm64: KVM: Map the kernel RO section into HYP Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:49 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum Marc Zyngier
` (3 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
This is it. We remove all of the code that has now been rewritten.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/Makefile | 2 -
arch/arm64/kvm/hyp.S | 1081 +--------------------------------------
arch/arm64/kvm/vgic-v2-switch.S | 134 -----
arch/arm64/kvm/vgic-v3-switch.S | 269 ----------
4 files changed, 1 insertion(+), 1485 deletions(-)
delete mode 100644 arch/arm64/kvm/vgic-v2-switch.S
delete mode 100644 arch/arm64/kvm/vgic-v3-switch.S
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index d31e4e5..caee9ee 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -23,8 +23,6 @@ kvm-$(CONFIG_KVM_ARM_HOST) += guest.o debug.o reset.o sys_regs.o sys_regs_generi
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v2.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v2-emul.o
-kvm-$(CONFIG_KVM_ARM_HOST) += vgic-v2-switch.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v3.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v3-emul.o
-kvm-$(CONFIG_KVM_ARM_HOST) += vgic-v3-switch.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
diff --git a/arch/arm64/kvm/hyp.S b/arch/arm64/kvm/hyp.S
index 86c2898..0ccdcbb 100644
--- a/arch/arm64/kvm/hyp.S
+++ b/arch/arm64/kvm/hyp.S
@@ -17,910 +17,7 @@
#include <linux/linkage.h>
-#include <asm/alternative.h>
-#include <asm/asm-offsets.h>
#include <asm/assembler.h>
-#include <asm/cpufeature.h>
-#include <asm/debug-monitors.h>
-#include <asm/esr.h>
-#include <asm/fpsimdmacros.h>
-#include <asm/kvm.h>
-#include <asm/kvm_arm.h>
-#include <asm/kvm_asm.h>
-#include <asm/kvm_mmu.h>
-#include <asm/memory.h>
-
-#define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
-#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
-#define CPU_SPSR_OFFSET(x) CPU_GP_REG_OFFSET(CPU_SPSR + 8*x)
-#define CPU_SYSREG_OFFSET(x) (CPU_SYSREGS + 8*x)
-
- .text
- .pushsection .hyp.text, "ax"
- .align PAGE_SHIFT
-
-.macro save_common_regs
- // x2: base address for cpu context
- // x3: tmp register
-
- add x3, x2, #CPU_XREG_OFFSET(19)
- stp x19, x20, [x3]
- stp x21, x22, [x3, #16]
- stp x23, x24, [x3, #32]
- stp x25, x26, [x3, #48]
- stp x27, x28, [x3, #64]
- stp x29, lr, [x3, #80]
-
- mrs x19, sp_el0
- mrs x20, elr_el2 // pc before entering el2
- mrs x21, spsr_el2 // pstate before entering el2
-
- stp x19, x20, [x3, #96]
- str x21, [x3, #112]
-
- mrs x22, sp_el1
- mrs x23, elr_el1
- mrs x24, spsr_el1
-
- str x22, [x2, #CPU_GP_REG_OFFSET(CPU_SP_EL1)]
- str x23, [x2, #CPU_GP_REG_OFFSET(CPU_ELR_EL1)]
- str x24, [x2, #CPU_SPSR_OFFSET(KVM_SPSR_EL1)]
-.endm
-
-.macro restore_common_regs
- // x2: base address for cpu context
- // x3: tmp register
-
- ldr x22, [x2, #CPU_GP_REG_OFFSET(CPU_SP_EL1)]
- ldr x23, [x2, #CPU_GP_REG_OFFSET(CPU_ELR_EL1)]
- ldr x24, [x2, #CPU_SPSR_OFFSET(KVM_SPSR_EL1)]
-
- msr sp_el1, x22
- msr elr_el1, x23
- msr spsr_el1, x24
-
- add x3, x2, #CPU_XREG_OFFSET(31) // SP_EL0
- ldp x19, x20, [x3]
- ldr x21, [x3, #16]
-
- msr sp_el0, x19
- msr elr_el2, x20 // pc on return from el2
- msr spsr_el2, x21 // pstate on return from el2
-
- add x3, x2, #CPU_XREG_OFFSET(19)
- ldp x19, x20, [x3]
- ldp x21, x22, [x3, #16]
- ldp x23, x24, [x3, #32]
- ldp x25, x26, [x3, #48]
- ldp x27, x28, [x3, #64]
- ldp x29, lr, [x3, #80]
-.endm
-
-.macro save_host_regs
- save_common_regs
-.endm
-
-.macro restore_host_regs
- restore_common_regs
-.endm
-
-.macro save_fpsimd
- // x2: cpu context address
- // x3, x4: tmp regs
- add x3, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
- fpsimd_save x3, 4
-.endm
-
-.macro restore_fpsimd
- // x2: cpu context address
- // x3, x4: tmp regs
- add x3, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
- fpsimd_restore x3, 4
-.endm
-
-.macro save_guest_regs
- // x0 is the vcpu address
- // x1 is the return code, do not corrupt!
- // x2 is the cpu context
- // x3 is a tmp register
- // Guest's x0-x3 are on the stack
-
- // Compute base to save registers
- add x3, x2, #CPU_XREG_OFFSET(4)
- stp x4, x5, [x3]
- stp x6, x7, [x3, #16]
- stp x8, x9, [x3, #32]
- stp x10, x11, [x3, #48]
- stp x12, x13, [x3, #64]
- stp x14, x15, [x3, #80]
- stp x16, x17, [x3, #96]
- str x18, [x3, #112]
-
- pop x6, x7 // x2, x3
- pop x4, x5 // x0, x1
-
- add x3, x2, #CPU_XREG_OFFSET(0)
- stp x4, x5, [x3]
- stp x6, x7, [x3, #16]
-
- save_common_regs
-.endm
-
-.macro restore_guest_regs
- // x0 is the vcpu address.
- // x2 is the cpu context
- // x3 is a tmp register
-
- // Prepare x0-x3 for later restore
- add x3, x2, #CPU_XREG_OFFSET(0)
- ldp x4, x5, [x3]
- ldp x6, x7, [x3, #16]
- push x4, x5 // Push x0-x3 on the stack
- push x6, x7
-
- // x4-x18
- ldp x4, x5, [x3, #32]
- ldp x6, x7, [x3, #48]
- ldp x8, x9, [x3, #64]
- ldp x10, x11, [x3, #80]
- ldp x12, x13, [x3, #96]
- ldp x14, x15, [x3, #112]
- ldp x16, x17, [x3, #128]
- ldr x18, [x3, #144]
-
- // x19-x29, lr, sp*, elr*, spsr*
- restore_common_regs
-
- // Last bits of the 64bit state
- pop x2, x3
- pop x0, x1
-
- // Do not touch any register after this!
-.endm
-
-/*
- * Macros to perform system register save/restore.
- *
- * Ordering here is absolutely critical, and must be kept consistent
- * in {save,restore}_sysregs, {save,restore}_guest_32bit_state,
- * and in kvm_asm.h.
- *
- * In other words, don't touch any of these unless you know what
- * you are doing.
- */
-.macro save_sysregs
- // x2: base address for cpu context
- // x3: tmp register
-
- add x3, x2, #CPU_SYSREG_OFFSET(MPIDR_EL1)
-
- mrs x4, vmpidr_el2
- mrs x5, csselr_el1
- mrs x6, sctlr_el1
- mrs x7, actlr_el1
- mrs x8, cpacr_el1
- mrs x9, ttbr0_el1
- mrs x10, ttbr1_el1
- mrs x11, tcr_el1
- mrs x12, esr_el1
- mrs x13, afsr0_el1
- mrs x14, afsr1_el1
- mrs x15, far_el1
- mrs x16, mair_el1
- mrs x17, vbar_el1
- mrs x18, contextidr_el1
- mrs x19, tpidr_el0
- mrs x20, tpidrro_el0
- mrs x21, tpidr_el1
- mrs x22, amair_el1
- mrs x23, cntkctl_el1
- mrs x24, par_el1
- mrs x25, mdscr_el1
-
- stp x4, x5, [x3]
- stp x6, x7, [x3, #16]
- stp x8, x9, [x3, #32]
- stp x10, x11, [x3, #48]
- stp x12, x13, [x3, #64]
- stp x14, x15, [x3, #80]
- stp x16, x17, [x3, #96]
- stp x18, x19, [x3, #112]
- stp x20, x21, [x3, #128]
- stp x22, x23, [x3, #144]
- stp x24, x25, [x3, #160]
-.endm
-
-.macro save_debug type
- // x4: pointer to register set
- // x5: number of registers to skip
- // x6..x22 trashed
-
- adr x22, 1f
- add x22, x22, x5, lsl #2
- br x22
-1:
- mrs x21, \type\()15_el1
- mrs x20, \type\()14_el1
- mrs x19, \type\()13_el1
- mrs x18, \type\()12_el1
- mrs x17, \type\()11_el1
- mrs x16, \type\()10_el1
- mrs x15, \type\()9_el1
- mrs x14, \type\()8_el1
- mrs x13, \type\()7_el1
- mrs x12, \type\()6_el1
- mrs x11, \type\()5_el1
- mrs x10, \type\()4_el1
- mrs x9, \type\()3_el1
- mrs x8, \type\()2_el1
- mrs x7, \type\()1_el1
- mrs x6, \type\()0_el1
-
- adr x22, 1f
- add x22, x22, x5, lsl #2
- br x22
-1:
- str x21, [x4, #(15 * 8)]
- str x20, [x4, #(14 * 8)]
- str x19, [x4, #(13 * 8)]
- str x18, [x4, #(12 * 8)]
- str x17, [x4, #(11 * 8)]
- str x16, [x4, #(10 * 8)]
- str x15, [x4, #(9 * 8)]
- str x14, [x4, #(8 * 8)]
- str x13, [x4, #(7 * 8)]
- str x12, [x4, #(6 * 8)]
- str x11, [x4, #(5 * 8)]
- str x10, [x4, #(4 * 8)]
- str x9, [x4, #(3 * 8)]
- str x8, [x4, #(2 * 8)]
- str x7, [x4, #(1 * 8)]
- str x6, [x4, #(0 * 8)]
-.endm
-
-.macro restore_sysregs
- // x2: base address for cpu context
- // x3: tmp register
-
- add x3, x2, #CPU_SYSREG_OFFSET(MPIDR_EL1)
-
- ldp x4, x5, [x3]
- ldp x6, x7, [x3, #16]
- ldp x8, x9, [x3, #32]
- ldp x10, x11, [x3, #48]
- ldp x12, x13, [x3, #64]
- ldp x14, x15, [x3, #80]
- ldp x16, x17, [x3, #96]
- ldp x18, x19, [x3, #112]
- ldp x20, x21, [x3, #128]
- ldp x22, x23, [x3, #144]
- ldp x24, x25, [x3, #160]
-
- msr vmpidr_el2, x4
- msr csselr_el1, x5
- msr sctlr_el1, x6
- msr actlr_el1, x7
- msr cpacr_el1, x8
- msr ttbr0_el1, x9
- msr ttbr1_el1, x10
- msr tcr_el1, x11
- msr esr_el1, x12
- msr afsr0_el1, x13
- msr afsr1_el1, x14
- msr far_el1, x15
- msr mair_el1, x16
- msr vbar_el1, x17
- msr contextidr_el1, x18
- msr tpidr_el0, x19
- msr tpidrro_el0, x20
- msr tpidr_el1, x21
- msr amair_el1, x22
- msr cntkctl_el1, x23
- msr par_el1, x24
- msr mdscr_el1, x25
-.endm
-
-.macro restore_debug type
- // x4: pointer to register set
- // x5: number of registers to skip
- // x6..x22 trashed
-
- adr x22, 1f
- add x22, x22, x5, lsl #2
- br x22
-1:
- ldr x21, [x4, #(15 * 8)]
- ldr x20, [x4, #(14 * 8)]
- ldr x19, [x4, #(13 * 8)]
- ldr x18, [x4, #(12 * 8)]
- ldr x17, [x4, #(11 * 8)]
- ldr x16, [x4, #(10 * 8)]
- ldr x15, [x4, #(9 * 8)]
- ldr x14, [x4, #(8 * 8)]
- ldr x13, [x4, #(7 * 8)]
- ldr x12, [x4, #(6 * 8)]
- ldr x11, [x4, #(5 * 8)]
- ldr x10, [x4, #(4 * 8)]
- ldr x9, [x4, #(3 * 8)]
- ldr x8, [x4, #(2 * 8)]
- ldr x7, [x4, #(1 * 8)]
- ldr x6, [x4, #(0 * 8)]
-
- adr x22, 1f
- add x22, x22, x5, lsl #2
- br x22
-1:
- msr \type\()15_el1, x21
- msr \type\()14_el1, x20
- msr \type\()13_el1, x19
- msr \type\()12_el1, x18
- msr \type\()11_el1, x17
- msr \type\()10_el1, x16
- msr \type\()9_el1, x15
- msr \type\()8_el1, x14
- msr \type\()7_el1, x13
- msr \type\()6_el1, x12
- msr \type\()5_el1, x11
- msr \type\()4_el1, x10
- msr \type\()3_el1, x9
- msr \type\()2_el1, x8
- msr \type\()1_el1, x7
- msr \type\()0_el1, x6
-.endm
-
-.macro skip_32bit_state tmp, target
- // Skip 32bit state if not needed
- mrs \tmp, hcr_el2
- tbnz \tmp, #HCR_RW_SHIFT, \target
-.endm
-
-.macro skip_tee_state tmp, target
- // Skip ThumbEE state if not needed
- mrs \tmp, id_pfr0_el1
- tbz \tmp, #12, \target
-.endm
-
-.macro skip_debug_state tmp, target
- ldr \tmp, [x0, #VCPU_DEBUG_FLAGS]
- tbz \tmp, #KVM_ARM64_DEBUG_DIRTY_SHIFT, \target
-.endm
-
-/*
- * Branch to target if CPTR_EL2.TFP bit is set (VFP/SIMD trapping enabled)
- */
-.macro skip_fpsimd_state tmp, target
- mrs \tmp, cptr_el2
- tbnz \tmp, #CPTR_EL2_TFP_SHIFT, \target
-.endm
-
-.macro compute_debug_state target
- // Compute debug state: If any of KDE, MDE or KVM_ARM64_DEBUG_DIRTY
- // is set, we do a full save/restore cycle and disable trapping.
- add x25, x0, #VCPU_CONTEXT
-
- // Check the state of MDSCR_EL1
- ldr x25, [x25, #CPU_SYSREG_OFFSET(MDSCR_EL1)]
- and x26, x25, #DBG_MDSCR_KDE
- and x25, x25, #DBG_MDSCR_MDE
- adds xzr, x25, x26
- b.eq 9998f // Nothing to see there
-
- // If any interesting bits was set, we must set the flag
- mov x26, #KVM_ARM64_DEBUG_DIRTY
- str x26, [x0, #VCPU_DEBUG_FLAGS]
- b 9999f // Don't skip restore
-
-9998:
- // Otherwise load the flags from memory in case we recently
- // trapped
- skip_debug_state x25, \target
-9999:
-.endm
-
-.macro save_guest_32bit_state
- skip_32bit_state x3, 1f
-
- add x3, x2, #CPU_SPSR_OFFSET(KVM_SPSR_ABT)
- mrs x4, spsr_abt
- mrs x5, spsr_und
- mrs x6, spsr_irq
- mrs x7, spsr_fiq
- stp x4, x5, [x3]
- stp x6, x7, [x3, #16]
-
- add x3, x2, #CPU_SYSREG_OFFSET(DACR32_EL2)
- mrs x4, dacr32_el2
- mrs x5, ifsr32_el2
- stp x4, x5, [x3]
-
- skip_fpsimd_state x8, 2f
- mrs x6, fpexc32_el2
- str x6, [x3, #16]
-2:
- skip_debug_state x8, 1f
- mrs x7, dbgvcr32_el2
- str x7, [x3, #24]
-1:
-.endm
-
-.macro restore_guest_32bit_state
- skip_32bit_state x3, 1f
-
- add x3, x2, #CPU_SPSR_OFFSET(KVM_SPSR_ABT)
- ldp x4, x5, [x3]
- ldp x6, x7, [x3, #16]
- msr spsr_abt, x4
- msr spsr_und, x5
- msr spsr_irq, x6
- msr spsr_fiq, x7
-
- add x3, x2, #CPU_SYSREG_OFFSET(DACR32_EL2)
- ldp x4, x5, [x3]
- msr dacr32_el2, x4
- msr ifsr32_el2, x5
-
- skip_debug_state x8, 1f
- ldr x7, [x3, #24]
- msr dbgvcr32_el2, x7
-1:
-.endm
-
-.macro activate_traps
- ldr x2, [x0, #VCPU_HCR_EL2]
-
- /*
- * We are about to set CPTR_EL2.TFP to trap all floating point
- * register accesses to EL2, however, the ARM ARM clearly states that
- * traps are only taken to EL2 if the operation would not otherwise
- * trap to EL1. Therefore, always make sure that for 32-bit guests,
- * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
- */
- tbnz x2, #HCR_RW_SHIFT, 99f // open code skip_32bit_state
- mov x3, #(1 << 30)
- msr fpexc32_el2, x3
- isb
-99:
- msr hcr_el2, x2
- mov x2, #CPTR_EL2_TTA
- orr x2, x2, #CPTR_EL2_TFP
- msr cptr_el2, x2
-
- mov x2, #(1 << 15) // Trap CP15 Cr=15
- msr hstr_el2, x2
-
- // Monitor Debug Config - see kvm_arm_setup_debug()
- ldr x2, [x0, #VCPU_MDCR_EL2]
- msr mdcr_el2, x2
-.endm
-
-.macro deactivate_traps
- mov x2, #HCR_RW
- msr hcr_el2, x2
- msr hstr_el2, xzr
-
- mrs x2, mdcr_el2
- and x2, x2, #MDCR_EL2_HPMN_MASK
- msr mdcr_el2, x2
-.endm
-
-.macro activate_vm
- ldr x1, [x0, #VCPU_KVM]
- kern_hyp_va x1
- ldr x2, [x1, #KVM_VTTBR]
- msr vttbr_el2, x2
-.endm
-
-.macro deactivate_vm
- msr vttbr_el2, xzr
-.endm
-
-/*
- * Call into the vgic backend for state saving
- */
-.macro save_vgic_state
-alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF
- bl __save_vgic_v2_state
-alternative_else
- bl __save_vgic_v3_state
-alternative_endif
- mrs x24, hcr_el2
- mov x25, #HCR_INT_OVERRIDE
- neg x25, x25
- and x24, x24, x25
- msr hcr_el2, x24
-.endm
-
-/*
- * Call into the vgic backend for state restoring
- */
-.macro restore_vgic_state
- mrs x24, hcr_el2
- ldr x25, [x0, #VCPU_IRQ_LINES]
- orr x24, x24, #HCR_INT_OVERRIDE
- orr x24, x24, x25
- msr hcr_el2, x24
-alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF
- bl __restore_vgic_v2_state
-alternative_else
- bl __restore_vgic_v3_state
-alternative_endif
-.endm
-
-.macro save_timer_state
- // x0: vcpu pointer
- ldr x2, [x0, #VCPU_KVM]
- kern_hyp_va x2
- ldr w3, [x2, #KVM_TIMER_ENABLED]
- cbz w3, 1f
-
- mrs x3, cntv_ctl_el0
- and x3, x3, #3
- str w3, [x0, #VCPU_TIMER_CNTV_CTL]
-
- isb
-
- mrs x3, cntv_cval_el0
- str x3, [x0, #VCPU_TIMER_CNTV_CVAL]
-
-1:
- // Disable the virtual timer
- msr cntv_ctl_el0, xzr
-
- // Allow physical timer/counter access for the host
- mrs x2, cnthctl_el2
- orr x2, x2, #3
- msr cnthctl_el2, x2
-
- // Clear cntvoff for the host
- msr cntvoff_el2, xzr
-.endm
-
-.macro restore_timer_state
- // x0: vcpu pointer
- // Disallow physical timer access for the guest
- // Physical counter access is allowed
- mrs x2, cnthctl_el2
- orr x2, x2, #1
- bic x2, x2, #2
- msr cnthctl_el2, x2
-
- ldr x2, [x0, #VCPU_KVM]
- kern_hyp_va x2
- ldr w3, [x2, #KVM_TIMER_ENABLED]
- cbz w3, 1f
-
- ldr x3, [x2, #KVM_TIMER_CNTVOFF]
- msr cntvoff_el2, x3
- ldr x2, [x0, #VCPU_TIMER_CNTV_CVAL]
- msr cntv_cval_el0, x2
- isb
-
- ldr w2, [x0, #VCPU_TIMER_CNTV_CTL]
- and x2, x2, #3
- msr cntv_ctl_el0, x2
-1:
-.endm
-
-__save_sysregs:
- save_sysregs
- ret
-
-__restore_sysregs:
- restore_sysregs
- ret
-
-/* Save debug state */
-__save_debug:
- // x2: ptr to CPU context
- // x3: ptr to debug reg struct
- // x4/x5/x6-22/x24-26: trashed
-
- mrs x26, id_aa64dfr0_el1
- ubfx x24, x26, #12, #4 // Extract BRPs
- ubfx x25, x26, #20, #4 // Extract WRPs
- mov w26, #15
- sub w24, w26, w24 // How many BPs to skip
- sub w25, w26, w25 // How many WPs to skip
-
- mov x5, x24
- add x4, x3, #DEBUG_BCR
- save_debug dbgbcr
- add x4, x3, #DEBUG_BVR
- save_debug dbgbvr
-
- mov x5, x25
- add x4, x3, #DEBUG_WCR
- save_debug dbgwcr
- add x4, x3, #DEBUG_WVR
- save_debug dbgwvr
-
- mrs x21, mdccint_el1
- str x21, [x2, #CPU_SYSREG_OFFSET(MDCCINT_EL1)]
- ret
-
-/* Restore debug state */
-__restore_debug:
- // x2: ptr to CPU context
- // x3: ptr to debug reg struct
- // x4/x5/x6-22/x24-26: trashed
-
- mrs x26, id_aa64dfr0_el1
- ubfx x24, x26, #12, #4 // Extract BRPs
- ubfx x25, x26, #20, #4 // Extract WRPs
- mov w26, #15
- sub w24, w26, w24 // How many BPs to skip
- sub w25, w26, w25 // How many WPs to skip
-
- mov x5, x24
- add x4, x3, #DEBUG_BCR
- restore_debug dbgbcr
- add x4, x3, #DEBUG_BVR
- restore_debug dbgbvr
-
- mov x5, x25
- add x4, x3, #DEBUG_WCR
- restore_debug dbgwcr
- add x4, x3, #DEBUG_WVR
- restore_debug dbgwvr
-
- ldr x21, [x2, #CPU_SYSREG_OFFSET(MDCCINT_EL1)]
- msr mdccint_el1, x21
-
- ret
-
-__save_fpsimd:
- skip_fpsimd_state x3, 1f
- save_fpsimd
-1: ret
-
-__restore_fpsimd:
- skip_fpsimd_state x3, 1f
- restore_fpsimd
-1: ret
-
-switch_to_guest_fpsimd:
- push x4, lr
-
- mrs x2, cptr_el2
- bic x2, x2, #CPTR_EL2_TFP
- msr cptr_el2, x2
- isb
-
- mrs x0, tpidr_el2
-
- ldr x2, [x0, #VCPU_HOST_CONTEXT]
- kern_hyp_va x2
- bl __save_fpsimd
-
- add x2, x0, #VCPU_CONTEXT
- bl __restore_fpsimd
-
- skip_32bit_state x3, 1f
- ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
- msr fpexc32_el2, x4
-1:
- pop x4, lr
- pop x2, x3
- pop x0, x1
-
- eret
-
-/*
- * u64 __kvm_vcpu_run(struct kvm_vcpu *vcpu);
- *
- * This is the world switch. The first half of the function
- * deals with entering the guest, and anything from __kvm_vcpu_return
- * to the end of the function deals with reentering the host.
- * On the enter path, only x0 (vcpu pointer) must be preserved until
- * the last moment. On the exit path, x0 (vcpu pointer) and x1 (exception
- * code) must both be preserved until the epilogue.
- * In both cases, x2 points to the CPU context we're saving/restoring from/to.
- */
-ENTRY(__kvm_vcpu_run)
- kern_hyp_va x0
- msr tpidr_el2, x0 // Save the vcpu register
-
- // Host context
- ldr x2, [x0, #VCPU_HOST_CONTEXT]
- kern_hyp_va x2
-
- save_host_regs
- bl __save_sysregs
-
- compute_debug_state 1f
- add x3, x0, #VCPU_HOST_DEBUG_STATE
- bl __save_debug
-1:
- activate_traps
- activate_vm
-
- restore_vgic_state
- restore_timer_state
-
- // Guest context
- add x2, x0, #VCPU_CONTEXT
-
- // We must restore the 32-bit state before the sysregs, thanks
- // to Cortex-A57 erratum #852523.
- restore_guest_32bit_state
- bl __restore_sysregs
-
- skip_debug_state x3, 1f
- ldr x3, [x0, #VCPU_DEBUG_PTR]
- kern_hyp_va x3
- bl __restore_debug
-1:
- restore_guest_regs
-
- // That's it, no more messing around.
- eret
-
-__kvm_vcpu_return:
- // Assume x0 is the vcpu pointer, x1 the return code
- // Guest's x0-x3 are on the stack
-
- // Guest context
- add x2, x0, #VCPU_CONTEXT
-
- save_guest_regs
- bl __save_fpsimd
- bl __save_sysregs
-
- skip_debug_state x3, 1f
- ldr x3, [x0, #VCPU_DEBUG_PTR]
- kern_hyp_va x3
- bl __save_debug
-1:
- save_guest_32bit_state
-
- save_timer_state
- save_vgic_state
-
- deactivate_traps
- deactivate_vm
-
- // Host context
- ldr x2, [x0, #VCPU_HOST_CONTEXT]
- kern_hyp_va x2
-
- bl __restore_sysregs
- bl __restore_fpsimd
- /* Clear FPSIMD and Trace trapping */
- msr cptr_el2, xzr
-
- skip_debug_state x3, 1f
- // Clear the dirty flag for the next run, as all the state has
- // already been saved. Note that we nuke the whole 64bit word.
- // If we ever add more flags, we'll have to be more careful...
- str xzr, [x0, #VCPU_DEBUG_FLAGS]
- add x3, x0, #VCPU_HOST_DEBUG_STATE
- bl __restore_debug
-1:
- restore_host_regs
-
- mov x0, x1
- ret
-END(__kvm_vcpu_run)
-
-// void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
-ENTRY(__kvm_tlb_flush_vmid_ipa)
- dsb ishst
-
- kern_hyp_va x0
- ldr x2, [x0, #KVM_VTTBR]
- msr vttbr_el2, x2
- isb
-
- /*
- * We could do so much better if we had the VA as well.
- * Instead, we invalidate Stage-2 for this IPA, and the
- * whole of Stage-1. Weep...
- */
- lsr x1, x1, #12
- tlbi ipas2e1is, x1
- /*
- * We have to ensure completion of the invalidation at Stage-2,
- * since a table walk on another CPU could refill a TLB with a
- * complete (S1 + S2) walk based on the old Stage-2 mapping if
- * the Stage-1 invalidation happened first.
- */
- dsb ish
- tlbi vmalle1is
- dsb ish
- isb
-
- msr vttbr_el2, xzr
- ret
-ENDPROC(__kvm_tlb_flush_vmid_ipa)
-
-/**
- * void __kvm_tlb_flush_vmid(struct kvm *kvm) - Flush per-VMID TLBs
- * @struct kvm *kvm - pointer to kvm structure
- *
- * Invalidates all Stage 1 and 2 TLB entries for current VMID.
- */
-ENTRY(__kvm_tlb_flush_vmid)
- dsb ishst
-
- kern_hyp_va x0
- ldr x2, [x0, #KVM_VTTBR]
- msr vttbr_el2, x2
- isb
-
- tlbi vmalls12e1is
- dsb ish
- isb
-
- msr vttbr_el2, xzr
- ret
-ENDPROC(__kvm_tlb_flush_vmid)
-
-ENTRY(__kvm_flush_vm_context)
- dsb ishst
- tlbi alle1is
- ic ialluis
- dsb ish
- ret
-ENDPROC(__kvm_flush_vm_context)
-
-__kvm_hyp_panic:
- // Stash PAR_EL1 before corrupting it in __restore_sysregs
- mrs x0, par_el1
- push x0, xzr
-
- // Guess the context by looking at VTTBR:
- // If zero, then we're already a host.
- // Otherwise restore a minimal host context before panicing.
- mrs x0, vttbr_el2
- cbz x0, 1f
-
- mrs x0, tpidr_el2
-
- deactivate_traps
- deactivate_vm
-
- ldr x2, [x0, #VCPU_HOST_CONTEXT]
- kern_hyp_va x2
-
- bl __restore_sysregs
-
- /*
- * Make sure we have a valid host stack, and don't leave junk in the
- * frame pointer that will give us a misleading host stack unwinding.
- */
- ldr x22, [x2, #CPU_GP_REG_OFFSET(CPU_SP_EL1)]
- msr sp_el1, x22
- mov x29, xzr
-
-1: adr x0, __hyp_panic_str
- adr x1, 2f
- ldp x2, x3, [x1]
- sub x0, x0, x2
- add x0, x0, x3
- mrs x1, spsr_el2
- mrs x2, elr_el2
- mrs x3, esr_el2
- mrs x4, far_el2
- mrs x5, hpfar_el2
- pop x6, xzr // active context PAR_EL1
- mrs x7, tpidr_el2
-
- mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\
- PSR_MODE_EL1h)
- msr spsr_el2, lr
- ldr lr, =panic
- msr elr_el2, lr
- eret
-
- .align 3
-2: .quad HYP_PAGE_OFFSET
- .quad PAGE_OFFSET
-ENDPROC(__kvm_hyp_panic)
-
-__hyp_panic_str:
- .ascii "HYP panic:\nPS:%08x PC:%016x ESR:%08x\nFAR:%016x HPFAR:%016x PAR:%016x\nVCPU:%p\n\0"
-
- .align 2
/*
* u64 kvm_call_hyp(void *hypfn, ...);
@@ -934,7 +31,7 @@ __hyp_panic_str:
* passed as x0, x1, and x2 (a maximum of 3 arguments in addition to the
* function pointer can be passed). The function being called must be mapped
* in Hyp mode (see init_hyp_mode in arch/arm/kvm/arm.c). Return values are
- * passed in r0 and r1.
+ * passed in x0.
*
* A function pointer with a value of 0 has a special meaning, and is
* used to implement __hyp_get_vectors in the same way as in
@@ -944,179 +41,3 @@ ENTRY(kvm_call_hyp)
hvc #0
ret
ENDPROC(kvm_call_hyp)
-
-.macro invalid_vector label, target
- .align 2
-\label:
- b \target
-ENDPROC(\label)
-.endm
-
- /* None of these should ever happen */
- invalid_vector el2t_sync_invalid, __kvm_hyp_panic
- invalid_vector el2t_irq_invalid, __kvm_hyp_panic
- invalid_vector el2t_fiq_invalid, __kvm_hyp_panic
- invalid_vector el2t_error_invalid, __kvm_hyp_panic
- invalid_vector el2h_sync_invalid, __kvm_hyp_panic
- invalid_vector el2h_irq_invalid, __kvm_hyp_panic
- invalid_vector el2h_fiq_invalid, __kvm_hyp_panic
- invalid_vector el2h_error_invalid, __kvm_hyp_panic
- invalid_vector el1_sync_invalid, __kvm_hyp_panic
- invalid_vector el1_irq_invalid, __kvm_hyp_panic
- invalid_vector el1_fiq_invalid, __kvm_hyp_panic
- invalid_vector el1_error_invalid, __kvm_hyp_panic
-
-el1_sync: // Guest trapped into EL2
- push x0, x1
- push x2, x3
-
- mrs x1, esr_el2
- lsr x2, x1, #ESR_ELx_EC_SHIFT
-
- cmp x2, #ESR_ELx_EC_HVC64
- b.ne el1_trap
-
- mrs x3, vttbr_el2 // If vttbr is valid, the 64bit guest
- cbnz x3, el1_trap // called HVC
-
- /* Here, we're pretty sure the host called HVC. */
- pop x2, x3
- pop x0, x1
-
- /* Check for __hyp_get_vectors */
- cbnz x0, 1f
- mrs x0, vbar_el2
- b 2f
-
-1: push lr, xzr
-
- /*
- * Compute the function address in EL2, and shuffle the parameters.
- */
- kern_hyp_va x0
- mov lr, x0
- mov x0, x1
- mov x1, x2
- mov x2, x3
- blr lr
-
- pop lr, xzr
-2: eret
-
-el1_trap:
- /*
- * x1: ESR
- * x2: ESR_EC
- */
-
- /* Guest accessed VFP/SIMD registers, save host, restore Guest */
- cmp x2, #ESR_ELx_EC_FP_ASIMD
- b.eq switch_to_guest_fpsimd
-
- cmp x2, #ESR_ELx_EC_DABT_LOW
- mov x0, #ESR_ELx_EC_IABT_LOW
- ccmp x2, x0, #4, ne
- b.ne 1f // Not an abort we care about
-
- /* This is an abort. Check for permission fault */
-alternative_if_not ARM64_WORKAROUND_834220
- and x2, x1, #ESR_ELx_FSC_TYPE
- cmp x2, #FSC_PERM
- b.ne 1f // Not a permission fault
-alternative_else
- nop // Use the permission fault path to
- nop // check for a valid S1 translation,
- nop // regardless of the ESR value.
-alternative_endif
-
- /*
- * Check for Stage-1 page table walk, which is guaranteed
- * to give a valid HPFAR_EL2.
- */
- tbnz x1, #7, 1f // S1PTW is set
-
- /* Preserve PAR_EL1 */
- mrs x3, par_el1
- push x3, xzr
-
- /*
- * Permission fault, HPFAR_EL2 is invalid.
- * Resolve the IPA the hard way using the guest VA.
- * Stage-1 translation already validated the memory access rights.
- * As such, we can use the EL1 translation regime, and don't have
- * to distinguish between EL0 and EL1 access.
- */
- mrs x2, far_el2
- at s1e1r, x2
- isb
-
- /* Read result */
- mrs x3, par_el1
- pop x0, xzr // Restore PAR_EL1 from the stack
- msr par_el1, x0
- tbnz x3, #0, 3f // Bail out if we failed the translation
- ubfx x3, x3, #12, #36 // Extract IPA
- lsl x3, x3, #4 // and present it like HPFAR
- b 2f
-
-1: mrs x3, hpfar_el2
- mrs x2, far_el2
-
-2: mrs x0, tpidr_el2
- str w1, [x0, #VCPU_ESR_EL2]
- str x2, [x0, #VCPU_FAR_EL2]
- str x3, [x0, #VCPU_HPFAR_EL2]
-
- mov x1, #ARM_EXCEPTION_TRAP
- b __kvm_vcpu_return
-
- /*
- * Translation failed. Just return to the guest and
- * let it fault again. Another CPU is probably playing
- * behind our back.
- */
-3: pop x2, x3
- pop x0, x1
-
- eret
-
-el1_irq:
- push x0, x1
- push x2, x3
- mrs x0, tpidr_el2
- mov x1, #ARM_EXCEPTION_IRQ
- b __kvm_vcpu_return
-
- .ltorg
-
- .align 11
-
-ENTRY(__kvm_hyp_vector)
- ventry el2t_sync_invalid // Synchronous EL2t
- ventry el2t_irq_invalid // IRQ EL2t
- ventry el2t_fiq_invalid // FIQ EL2t
- ventry el2t_error_invalid // Error EL2t
-
- ventry el2h_sync_invalid // Synchronous EL2h
- ventry el2h_irq_invalid // IRQ EL2h
- ventry el2h_fiq_invalid // FIQ EL2h
- ventry el2h_error_invalid // Error EL2h
-
- ventry el1_sync // Synchronous 64-bit EL1
- ventry el1_irq // IRQ 64-bit EL1
- ventry el1_fiq_invalid // FIQ 64-bit EL1
- ventry el1_error_invalid // Error 64-bit EL1
-
- ventry el1_sync // Synchronous 32-bit EL1
- ventry el1_irq // IRQ 32-bit EL1
- ventry el1_fiq_invalid // FIQ 32-bit EL1
- ventry el1_error_invalid // Error 32-bit EL1
-ENDPROC(__kvm_hyp_vector)
-
-
-ENTRY(__kvm_get_mdcr_el2)
- mrs x0, mdcr_el2
- ret
-ENDPROC(__kvm_get_mdcr_el2)
-
- .popsection
diff --git a/arch/arm64/kvm/vgic-v2-switch.S b/arch/arm64/kvm/vgic-v2-switch.S
deleted file mode 100644
index 3f00071..0000000
--- a/arch/arm64/kvm/vgic-v2-switch.S
+++ /dev/null
@@ -1,134 +0,0 @@
-/*
- * Copyright (C) 2012,2013 - ARM Ltd
- * Author: Marc Zyngier <marc.zyngier@arm.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <linux/linkage.h>
-#include <linux/irqchip/arm-gic.h>
-
-#include <asm/assembler.h>
-#include <asm/memory.h>
-#include <asm/asm-offsets.h>
-#include <asm/kvm.h>
-#include <asm/kvm_asm.h>
-#include <asm/kvm_arm.h>
-#include <asm/kvm_mmu.h>
-
- .text
- .pushsection .hyp.text, "ax"
-
-/*
- * Save the VGIC CPU state into memory
- * x0: Register pointing to VCPU struct
- * Do not corrupt x1!!!
- */
-ENTRY(__save_vgic_v2_state)
-__save_vgic_v2_state:
- /* Get VGIC VCTRL base into x2 */
- ldr x2, [x0, #VCPU_KVM]
- kern_hyp_va x2
- ldr x2, [x2, #KVM_VGIC_VCTRL]
- kern_hyp_va x2
- cbz x2, 2f // disabled
-
- /* Compute the address of struct vgic_cpu */
- add x3, x0, #VCPU_VGIC_CPU
-
- /* Save all interesting registers */
- ldr w5, [x2, #GICH_VMCR]
- ldr w6, [x2, #GICH_MISR]
- ldr w7, [x2, #GICH_EISR0]
- ldr w8, [x2, #GICH_EISR1]
- ldr w9, [x2, #GICH_ELRSR0]
- ldr w10, [x2, #GICH_ELRSR1]
- ldr w11, [x2, #GICH_APR]
-CPU_BE( rev w5, w5 )
-CPU_BE( rev w6, w6 )
-CPU_BE( rev w7, w7 )
-CPU_BE( rev w8, w8 )
-CPU_BE( rev w9, w9 )
-CPU_BE( rev w10, w10 )
-CPU_BE( rev w11, w11 )
-
- str w5, [x3, #VGIC_V2_CPU_VMCR]
- str w6, [x3, #VGIC_V2_CPU_MISR]
-CPU_LE( str w7, [x3, #VGIC_V2_CPU_EISR] )
-CPU_LE( str w8, [x3, #(VGIC_V2_CPU_EISR + 4)] )
-CPU_LE( str w9, [x3, #VGIC_V2_CPU_ELRSR] )
-CPU_LE( str w10, [x3, #(VGIC_V2_CPU_ELRSR + 4)] )
-CPU_BE( str w7, [x3, #(VGIC_V2_CPU_EISR + 4)] )
-CPU_BE( str w8, [x3, #VGIC_V2_CPU_EISR] )
-CPU_BE( str w9, [x3, #(VGIC_V2_CPU_ELRSR + 4)] )
-CPU_BE( str w10, [x3, #VGIC_V2_CPU_ELRSR] )
- str w11, [x3, #VGIC_V2_CPU_APR]
-
- /* Clear GICH_HCR */
- str wzr, [x2, #GICH_HCR]
-
- /* Save list registers */
- add x2, x2, #GICH_LR0
- ldr w4, [x3, #VGIC_CPU_NR_LR]
- add x3, x3, #VGIC_V2_CPU_LR
-1: ldr w5, [x2], #4
-CPU_BE( rev w5, w5 )
- str w5, [x3], #4
- sub w4, w4, #1
- cbnz w4, 1b
-2:
- ret
-ENDPROC(__save_vgic_v2_state)
-
-/*
- * Restore the VGIC CPU state from memory
- * x0: Register pointing to VCPU struct
- */
-ENTRY(__restore_vgic_v2_state)
-__restore_vgic_v2_state:
- /* Get VGIC VCTRL base into x2 */
- ldr x2, [x0, #VCPU_KVM]
- kern_hyp_va x2
- ldr x2, [x2, #KVM_VGIC_VCTRL]
- kern_hyp_va x2
- cbz x2, 2f // disabled
-
- /* Compute the address of struct vgic_cpu */
- add x3, x0, #VCPU_VGIC_CPU
-
- /* We only restore a minimal set of registers */
- ldr w4, [x3, #VGIC_V2_CPU_HCR]
- ldr w5, [x3, #VGIC_V2_CPU_VMCR]
- ldr w6, [x3, #VGIC_V2_CPU_APR]
-CPU_BE( rev w4, w4 )
-CPU_BE( rev w5, w5 )
-CPU_BE( rev w6, w6 )
-
- str w4, [x2, #GICH_HCR]
- str w5, [x2, #GICH_VMCR]
- str w6, [x2, #GICH_APR]
-
- /* Restore list registers */
- add x2, x2, #GICH_LR0
- ldr w4, [x3, #VGIC_CPU_NR_LR]
- add x3, x3, #VGIC_V2_CPU_LR
-1: ldr w5, [x3], #4
-CPU_BE( rev w5, w5 )
- str w5, [x2], #4
- sub w4, w4, #1
- cbnz w4, 1b
-2:
- ret
-ENDPROC(__restore_vgic_v2_state)
-
- .popsection
diff --git a/arch/arm64/kvm/vgic-v3-switch.S b/arch/arm64/kvm/vgic-v3-switch.S
deleted file mode 100644
index 3c20730..0000000
--- a/arch/arm64/kvm/vgic-v3-switch.S
+++ /dev/null
@@ -1,269 +0,0 @@
-/*
- * Copyright (C) 2012,2013 - ARM Ltd
- * Author: Marc Zyngier <marc.zyngier@arm.com>
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-
-#include <linux/linkage.h>
-#include <linux/irqchip/arm-gic-v3.h>
-
-#include <asm/assembler.h>
-#include <asm/memory.h>
-#include <asm/asm-offsets.h>
-#include <asm/kvm.h>
-#include <asm/kvm_asm.h>
-#include <asm/kvm_arm.h>
-
- .text
- .pushsection .hyp.text, "ax"
-
-/*
- * We store LRs in reverse order to let the CPU deal with streaming
- * access. Use this macro to make it look saner...
- */
-#define LR_OFFSET(n) (VGIC_V3_CPU_LR + (15 - n) * 8)
-
-/*
- * Save the VGIC CPU state into memory
- * x0: Register pointing to VCPU struct
- * Do not corrupt x1!!!
- */
-.macro save_vgic_v3_state
- // Compute the address of struct vgic_cpu
- add x3, x0, #VCPU_VGIC_CPU
-
- // Make sure stores to the GIC via the memory mapped interface
- // are now visible to the system register interface
- dsb st
-
- // Save all interesting registers
- mrs_s x5, ICH_VMCR_EL2
- mrs_s x6, ICH_MISR_EL2
- mrs_s x7, ICH_EISR_EL2
- mrs_s x8, ICH_ELSR_EL2
-
- str w5, [x3, #VGIC_V3_CPU_VMCR]
- str w6, [x3, #VGIC_V3_CPU_MISR]
- str w7, [x3, #VGIC_V3_CPU_EISR]
- str w8, [x3, #VGIC_V3_CPU_ELRSR]
-
- msr_s ICH_HCR_EL2, xzr
-
- mrs_s x21, ICH_VTR_EL2
- mvn w22, w21
- ubfiz w23, w22, 2, 4 // w23 = (15 - ListRegs) * 4
-
- adr x24, 1f
- add x24, x24, x23
- br x24
-
-1:
- mrs_s x20, ICH_LR15_EL2
- mrs_s x19, ICH_LR14_EL2
- mrs_s x18, ICH_LR13_EL2
- mrs_s x17, ICH_LR12_EL2
- mrs_s x16, ICH_LR11_EL2
- mrs_s x15, ICH_LR10_EL2
- mrs_s x14, ICH_LR9_EL2
- mrs_s x13, ICH_LR8_EL2
- mrs_s x12, ICH_LR7_EL2
- mrs_s x11, ICH_LR6_EL2
- mrs_s x10, ICH_LR5_EL2
- mrs_s x9, ICH_LR4_EL2
- mrs_s x8, ICH_LR3_EL2
- mrs_s x7, ICH_LR2_EL2
- mrs_s x6, ICH_LR1_EL2
- mrs_s x5, ICH_LR0_EL2
-
- adr x24, 1f
- add x24, x24, x23
- br x24
-
-1:
- str x20, [x3, #LR_OFFSET(15)]
- str x19, [x3, #LR_OFFSET(14)]
- str x18, [x3, #LR_OFFSET(13)]
- str x17, [x3, #LR_OFFSET(12)]
- str x16, [x3, #LR_OFFSET(11)]
- str x15, [x3, #LR_OFFSET(10)]
- str x14, [x3, #LR_OFFSET(9)]
- str x13, [x3, #LR_OFFSET(8)]
- str x12, [x3, #LR_OFFSET(7)]
- str x11, [x3, #LR_OFFSET(6)]
- str x10, [x3, #LR_OFFSET(5)]
- str x9, [x3, #LR_OFFSET(4)]
- str x8, [x3, #LR_OFFSET(3)]
- str x7, [x3, #LR_OFFSET(2)]
- str x6, [x3, #LR_OFFSET(1)]
- str x5, [x3, #LR_OFFSET(0)]
-
- tbnz w21, #29, 6f // 6 bits
- tbz w21, #30, 5f // 5 bits
- // 7 bits
- mrs_s x20, ICH_AP0R3_EL2
- str w20, [x3, #(VGIC_V3_CPU_AP0R + 3*4)]
- mrs_s x19, ICH_AP0R2_EL2
- str w19, [x3, #(VGIC_V3_CPU_AP0R + 2*4)]
-6: mrs_s x18, ICH_AP0R1_EL2
- str w18, [x3, #(VGIC_V3_CPU_AP0R + 1*4)]
-5: mrs_s x17, ICH_AP0R0_EL2
- str w17, [x3, #VGIC_V3_CPU_AP0R]
-
- tbnz w21, #29, 6f // 6 bits
- tbz w21, #30, 5f // 5 bits
- // 7 bits
- mrs_s x20, ICH_AP1R3_EL2
- str w20, [x3, #(VGIC_V3_CPU_AP1R + 3*4)]
- mrs_s x19, ICH_AP1R2_EL2
- str w19, [x3, #(VGIC_V3_CPU_AP1R + 2*4)]
-6: mrs_s x18, ICH_AP1R1_EL2
- str w18, [x3, #(VGIC_V3_CPU_AP1R + 1*4)]
-5: mrs_s x17, ICH_AP1R0_EL2
- str w17, [x3, #VGIC_V3_CPU_AP1R]
-
- // Restore SRE_EL1 access and re-enable SRE at EL1.
- mrs_s x5, ICC_SRE_EL2
- orr x5, x5, #ICC_SRE_EL2_ENABLE
- msr_s ICC_SRE_EL2, x5
- isb
- mov x5, #1
- msr_s ICC_SRE_EL1, x5
-.endm
-
-/*
- * Restore the VGIC CPU state from memory
- * x0: Register pointing to VCPU struct
- */
-.macro restore_vgic_v3_state
- // Compute the address of struct vgic_cpu
- add x3, x0, #VCPU_VGIC_CPU
-
- // Restore all interesting registers
- ldr w4, [x3, #VGIC_V3_CPU_HCR]
- ldr w5, [x3, #VGIC_V3_CPU_VMCR]
- ldr w25, [x3, #VGIC_V3_CPU_SRE]
-
- msr_s ICC_SRE_EL1, x25
-
- // make sure SRE is valid before writing the other registers
- isb
-
- msr_s ICH_HCR_EL2, x4
- msr_s ICH_VMCR_EL2, x5
-
- mrs_s x21, ICH_VTR_EL2
-
- tbnz w21, #29, 6f // 6 bits
- tbz w21, #30, 5f // 5 bits
- // 7 bits
- ldr w20, [x3, #(VGIC_V3_CPU_AP1R + 3*4)]
- msr_s ICH_AP1R3_EL2, x20
- ldr w19, [x3, #(VGIC_V3_CPU_AP1R + 2*4)]
- msr_s ICH_AP1R2_EL2, x19
-6: ldr w18, [x3, #(VGIC_V3_CPU_AP1R + 1*4)]
- msr_s ICH_AP1R1_EL2, x18
-5: ldr w17, [x3, #VGIC_V3_CPU_AP1R]
- msr_s ICH_AP1R0_EL2, x17
-
- tbnz w21, #29, 6f // 6 bits
- tbz w21, #30, 5f // 5 bits
- // 7 bits
- ldr w20, [x3, #(VGIC_V3_CPU_AP0R + 3*4)]
- msr_s ICH_AP0R3_EL2, x20
- ldr w19, [x3, #(VGIC_V3_CPU_AP0R + 2*4)]
- msr_s ICH_AP0R2_EL2, x19
-6: ldr w18, [x3, #(VGIC_V3_CPU_AP0R + 1*4)]
- msr_s ICH_AP0R1_EL2, x18
-5: ldr w17, [x3, #VGIC_V3_CPU_AP0R]
- msr_s ICH_AP0R0_EL2, x17
-
- and w22, w21, #0xf
- mvn w22, w21
- ubfiz w23, w22, 2, 4 // w23 = (15 - ListRegs) * 4
-
- adr x24, 1f
- add x24, x24, x23
- br x24
-
-1:
- ldr x20, [x3, #LR_OFFSET(15)]
- ldr x19, [x3, #LR_OFFSET(14)]
- ldr x18, [x3, #LR_OFFSET(13)]
- ldr x17, [x3, #LR_OFFSET(12)]
- ldr x16, [x3, #LR_OFFSET(11)]
- ldr x15, [x3, #LR_OFFSET(10)]
- ldr x14, [x3, #LR_OFFSET(9)]
- ldr x13, [x3, #LR_OFFSET(8)]
- ldr x12, [x3, #LR_OFFSET(7)]
- ldr x11, [x3, #LR_OFFSET(6)]
- ldr x10, [x3, #LR_OFFSET(5)]
- ldr x9, [x3, #LR_OFFSET(4)]
- ldr x8, [x3, #LR_OFFSET(3)]
- ldr x7, [x3, #LR_OFFSET(2)]
- ldr x6, [x3, #LR_OFFSET(1)]
- ldr x5, [x3, #LR_OFFSET(0)]
-
- adr x24, 1f
- add x24, x24, x23
- br x24
-
-1:
- msr_s ICH_LR15_EL2, x20
- msr_s ICH_LR14_EL2, x19
- msr_s ICH_LR13_EL2, x18
- msr_s ICH_LR12_EL2, x17
- msr_s ICH_LR11_EL2, x16
- msr_s ICH_LR10_EL2, x15
- msr_s ICH_LR9_EL2, x14
- msr_s ICH_LR8_EL2, x13
- msr_s ICH_LR7_EL2, x12
- msr_s ICH_LR6_EL2, x11
- msr_s ICH_LR5_EL2, x10
- msr_s ICH_LR4_EL2, x9
- msr_s ICH_LR3_EL2, x8
- msr_s ICH_LR2_EL2, x7
- msr_s ICH_LR1_EL2, x6
- msr_s ICH_LR0_EL2, x5
-
- // Ensure that the above will have reached the
- // (re)distributors. This ensure the guest will read
- // the correct values from the memory-mapped interface.
- isb
- dsb sy
-
- // Prevent the guest from touching the GIC system registers
- // if SRE isn't enabled for GICv3 emulation
- cbnz x25, 1f
- mrs_s x5, ICC_SRE_EL2
- and x5, x5, #~ICC_SRE_EL2_ENABLE
- msr_s ICC_SRE_EL2, x5
-1:
-.endm
-
-ENTRY(__save_vgic_v3_state)
- save_vgic_v3_state
- ret
-ENDPROC(__save_vgic_v3_state)
-
-ENTRY(__restore_vgic_v3_state)
- restore_vgic_v3_state
- ret
-ENDPROC(__restore_vgic_v3_state)
-
-ENTRY(__vgic_v3_get_ich_vtr_el2)
- mrs_s x0, ICH_VTR_EL2
- ret
-ENDPROC(__vgic_v3_get_ich_vtr_el2)
-
- .popsection
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (17 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 18/21] arm64: KVM: Move away from the assembly version of the world switch Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:51 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 20/21] arm64: KVM: Cleanup asm-offset.c Marc Zyngier
` (2 subsequent siblings)
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
Having the system register numbers as #defines has been a pain
since day one, as the ordering is pretty fragile, and moving
things around leads to renumbering and epic conflict resolutions.
Now that we're mostly acessing the sysreg file in C, an enum is
a much better type to use, and we can clean things up a bit.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/include/asm/kvm_asm.h | 76 ---------------------------------
arch/arm64/include/asm/kvm_emulate.h | 1 -
arch/arm64/include/asm/kvm_host.h | 81 +++++++++++++++++++++++++++++++++++-
arch/arm64/include/asm/kvm_mmio.h | 1 -
arch/arm64/kernel/asm-offsets.c | 1 +
arch/arm64/kvm/guest.c | 1 -
arch/arm64/kvm/handle_exit.c | 1 +
arch/arm64/kvm/hyp/debug-sr.c | 1 +
arch/arm64/kvm/hyp/entry.S | 3 +-
arch/arm64/kvm/hyp/sysreg-sr.c | 1 +
arch/arm64/kvm/sys_regs.c | 1 +
virt/kvm/arm/vgic-v3.c | 1 +
12 files changed, 87 insertions(+), 82 deletions(-)
diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
index 5e37710..52b777b 100644
--- a/arch/arm64/include/asm/kvm_asm.h
+++ b/arch/arm64/include/asm/kvm_asm.h
@@ -20,82 +20,6 @@
#include <asm/virt.h>
-/*
- * 0 is reserved as an invalid value.
- * Order *must* be kept in sync with the hyp switch code.
- */
-#define MPIDR_EL1 1 /* MultiProcessor Affinity Register */
-#define CSSELR_EL1 2 /* Cache Size Selection Register */
-#define SCTLR_EL1 3 /* System Control Register */
-#define ACTLR_EL1 4 /* Auxiliary Control Register */
-#define CPACR_EL1 5 /* Coprocessor Access Control */
-#define TTBR0_EL1 6 /* Translation Table Base Register 0 */
-#define TTBR1_EL1 7 /* Translation Table Base Register 1 */
-#define TCR_EL1 8 /* Translation Control Register */
-#define ESR_EL1 9 /* Exception Syndrome Register */
-#define AFSR0_EL1 10 /* Auxilary Fault Status Register 0 */
-#define AFSR1_EL1 11 /* Auxilary Fault Status Register 1 */
-#define FAR_EL1 12 /* Fault Address Register */
-#define MAIR_EL1 13 /* Memory Attribute Indirection Register */
-#define VBAR_EL1 14 /* Vector Base Address Register */
-#define CONTEXTIDR_EL1 15 /* Context ID Register */
-#define TPIDR_EL0 16 /* Thread ID, User R/W */
-#define TPIDRRO_EL0 17 /* Thread ID, User R/O */
-#define TPIDR_EL1 18 /* Thread ID, Privileged */
-#define AMAIR_EL1 19 /* Aux Memory Attribute Indirection Register */
-#define CNTKCTL_EL1 20 /* Timer Control Register (EL1) */
-#define PAR_EL1 21 /* Physical Address Register */
-#define MDSCR_EL1 22 /* Monitor Debug System Control Register */
-#define MDCCINT_EL1 23 /* Monitor Debug Comms Channel Interrupt Enable Reg */
-
-/* 32bit specific registers. Keep them at the end of the range */
-#define DACR32_EL2 24 /* Domain Access Control Register */
-#define IFSR32_EL2 25 /* Instruction Fault Status Register */
-#define FPEXC32_EL2 26 /* Floating-Point Exception Control Register */
-#define DBGVCR32_EL2 27 /* Debug Vector Catch Register */
-#define NR_SYS_REGS 28
-
-/* 32bit mapping */
-#define c0_MPIDR (MPIDR_EL1 * 2) /* MultiProcessor ID Register */
-#define c0_CSSELR (CSSELR_EL1 * 2)/* Cache Size Selection Register */
-#define c1_SCTLR (SCTLR_EL1 * 2) /* System Control Register */
-#define c1_ACTLR (ACTLR_EL1 * 2) /* Auxiliary Control Register */
-#define c1_CPACR (CPACR_EL1 * 2) /* Coprocessor Access Control */
-#define c2_TTBR0 (TTBR0_EL1 * 2) /* Translation Table Base Register 0 */
-#define c2_TTBR0_high (c2_TTBR0 + 1) /* TTBR0 top 32 bits */
-#define c2_TTBR1 (TTBR1_EL1 * 2) /* Translation Table Base Register 1 */
-#define c2_TTBR1_high (c2_TTBR1 + 1) /* TTBR1 top 32 bits */
-#define c2_TTBCR (TCR_EL1 * 2) /* Translation Table Base Control R. */
-#define c3_DACR (DACR32_EL2 * 2)/* Domain Access Control Register */
-#define c5_DFSR (ESR_EL1 * 2) /* Data Fault Status Register */
-#define c5_IFSR (IFSR32_EL2 * 2)/* Instruction Fault Status Register */
-#define c5_ADFSR (AFSR0_EL1 * 2) /* Auxiliary Data Fault Status R */
-#define c5_AIFSR (AFSR1_EL1 * 2) /* Auxiliary Instr Fault Status R */
-#define c6_DFAR (FAR_EL1 * 2) /* Data Fault Address Register */
-#define c6_IFAR (c6_DFAR + 1) /* Instruction Fault Address Register */
-#define c7_PAR (PAR_EL1 * 2) /* Physical Address Register */
-#define c7_PAR_high (c7_PAR + 1) /* PAR top 32 bits */
-#define c10_PRRR (MAIR_EL1 * 2) /* Primary Region Remap Register */
-#define c10_NMRR (c10_PRRR + 1) /* Normal Memory Remap Register */
-#define c12_VBAR (VBAR_EL1 * 2) /* Vector Base Address Register */
-#define c13_CID (CONTEXTIDR_EL1 * 2) /* Context ID Register */
-#define c13_TID_URW (TPIDR_EL0 * 2) /* Thread ID, User R/W */
-#define c13_TID_URO (TPIDRRO_EL0 * 2)/* Thread ID, User R/O */
-#define c13_TID_PRIV (TPIDR_EL1 * 2) /* Thread ID, Privileged */
-#define c10_AMAIR0 (AMAIR_EL1 * 2) /* Aux Memory Attr Indirection Reg */
-#define c10_AMAIR1 (c10_AMAIR0 + 1)/* Aux Memory Attr Indirection Reg */
-#define c14_CNTKCTL (CNTKCTL_EL1 * 2) /* Timer Control Register (PL1) */
-
-#define cp14_DBGDSCRext (MDSCR_EL1 * 2)
-#define cp14_DBGBCR0 (DBGBCR0_EL1 * 2)
-#define cp14_DBGBVR0 (DBGBVR0_EL1 * 2)
-#define cp14_DBGBXVR0 (cp14_DBGBVR0 + 1)
-#define cp14_DBGWCR0 (DBGWCR0_EL1 * 2)
-#define cp14_DBGWVR0 (DBGWVR0_EL1 * 2)
-#define cp14_DBGDCCINT (MDCCINT_EL1 * 2)
-
-#define NR_COPRO_REGS (NR_SYS_REGS * 2)
-
#define ARM_EXCEPTION_IRQ 0
#define ARM_EXCEPTION_TRAP 1
diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
index 3ca894e..170e17d 100644
--- a/arch/arm64/include/asm/kvm_emulate.h
+++ b/arch/arm64/include/asm/kvm_emulate.h
@@ -26,7 +26,6 @@
#include <asm/esr.h>
#include <asm/kvm_arm.h>
-#include <asm/kvm_asm.h>
#include <asm/kvm_mmio.h>
#include <asm/ptrace.h>
#include <asm/cputype.h>
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index a35ce72..2fae2d4 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -25,7 +25,6 @@
#include <linux/types.h>
#include <linux/kvm_types.h>
#include <asm/kvm.h>
-#include <asm/kvm_asm.h>
#include <asm/kvm_mmio.h>
#define __KVM_HAVE_ARCH_INTC_INITIALIZED
@@ -85,6 +84,86 @@ struct kvm_vcpu_fault_info {
u64 hpfar_el2; /* Hyp IPA Fault Address Register */
};
+/*
+ * 0 is reserved as an invalid value.
+ * Order *must* be kept in sync with the hyp switch code.
+ */
+enum vcpu_sysreg {
+ __INVALID_SYSREG__,
+ MPIDR_EL1, /* MultiProcessor Affinity Register */
+ CSSELR_EL1, /* Cache Size Selection Register */
+ SCTLR_EL1, /* System Control Register */
+ ACTLR_EL1, /* Auxiliary Control Register */
+ CPACR_EL1, /* Coprocessor Access Control */
+ TTBR0_EL1, /* Translation Table Base Register 0 */
+ TTBR1_EL1, /* Translation Table Base Register 1 */
+ TCR_EL1, /* Translation Control Register */
+ ESR_EL1, /* Exception Syndrome Register */
+ AFSR0_EL1, /* Auxilary Fault Status Register 0 */
+ AFSR1_EL1, /* Auxilary Fault Status Register 1 */
+ FAR_EL1, /* Fault Address Register */
+ MAIR_EL1, /* Memory Attribute Indirection Register */
+ VBAR_EL1, /* Vector Base Address Register */
+ CONTEXTIDR_EL1, /* Context ID Register */
+ TPIDR_EL0, /* Thread ID, User R/W */
+ TPIDRRO_EL0, /* Thread ID, User R/O */
+ TPIDR_EL1, /* Thread ID, Privileged */
+ AMAIR_EL1, /* Aux Memory Attribute Indirection Register */
+ CNTKCTL_EL1, /* Timer Control Register (EL1) */
+ PAR_EL1, /* Physical Address Register */
+ MDSCR_EL1, /* Monitor Debug System Control Register */
+ MDCCINT_EL1, /* Monitor Debug Comms Channel Interrupt Enable Reg */
+
+ /* 32bit specific registers. Keep them at the end of the range */
+ DACR32_EL2, /* Domain Access Control Register */
+ IFSR32_EL2, /* Instruction Fault Status Register */
+ FPEXC32_EL2, /* Floating-Point Exception Control Register */
+ DBGVCR32_EL2, /* Debug Vector Catch Register */
+
+ NR_SYS_REGS /* Nothing after this line! */
+};
+
+/* 32bit mapping */
+#define c0_MPIDR (MPIDR_EL1 * 2) /* MultiProcessor ID Register */
+#define c0_CSSELR (CSSELR_EL1 * 2)/* Cache Size Selection Register */
+#define c1_SCTLR (SCTLR_EL1 * 2) /* System Control Register */
+#define c1_ACTLR (ACTLR_EL1 * 2) /* Auxiliary Control Register */
+#define c1_CPACR (CPACR_EL1 * 2) /* Coprocessor Access Control */
+#define c2_TTBR0 (TTBR0_EL1 * 2) /* Translation Table Base Register 0 */
+#define c2_TTBR0_high (c2_TTBR0 + 1) /* TTBR0 top 32 bits */
+#define c2_TTBR1 (TTBR1_EL1 * 2) /* Translation Table Base Register 1 */
+#define c2_TTBR1_high (c2_TTBR1 + 1) /* TTBR1 top 32 bits */
+#define c2_TTBCR (TCR_EL1 * 2) /* Translation Table Base Control R. */
+#define c3_DACR (DACR32_EL2 * 2)/* Domain Access Control Register */
+#define c5_DFSR (ESR_EL1 * 2) /* Data Fault Status Register */
+#define c5_IFSR (IFSR32_EL2 * 2)/* Instruction Fault Status Register */
+#define c5_ADFSR (AFSR0_EL1 * 2) /* Auxiliary Data Fault Status R */
+#define c5_AIFSR (AFSR1_EL1 * 2) /* Auxiliary Instr Fault Status R */
+#define c6_DFAR (FAR_EL1 * 2) /* Data Fault Address Register */
+#define c6_IFAR (c6_DFAR + 1) /* Instruction Fault Address Register */
+#define c7_PAR (PAR_EL1 * 2) /* Physical Address Register */
+#define c7_PAR_high (c7_PAR + 1) /* PAR top 32 bits */
+#define c10_PRRR (MAIR_EL1 * 2) /* Primary Region Remap Register */
+#define c10_NMRR (c10_PRRR + 1) /* Normal Memory Remap Register */
+#define c12_VBAR (VBAR_EL1 * 2) /* Vector Base Address Register */
+#define c13_CID (CONTEXTIDR_EL1 * 2) /* Context ID Register */
+#define c13_TID_URW (TPIDR_EL0 * 2) /* Thread ID, User R/W */
+#define c13_TID_URO (TPIDRRO_EL0 * 2)/* Thread ID, User R/O */
+#define c13_TID_PRIV (TPIDR_EL1 * 2) /* Thread ID, Privileged */
+#define c10_AMAIR0 (AMAIR_EL1 * 2) /* Aux Memory Attr Indirection Reg */
+#define c10_AMAIR1 (c10_AMAIR0 + 1)/* Aux Memory Attr Indirection Reg */
+#define c14_CNTKCTL (CNTKCTL_EL1 * 2) /* Timer Control Register (PL1) */
+
+#define cp14_DBGDSCRext (MDSCR_EL1 * 2)
+#define cp14_DBGBCR0 (DBGBCR0_EL1 * 2)
+#define cp14_DBGBVR0 (DBGBVR0_EL1 * 2)
+#define cp14_DBGBXVR0 (cp14_DBGBVR0 + 1)
+#define cp14_DBGWCR0 (DBGWCR0_EL1 * 2)
+#define cp14_DBGWVR0 (DBGWVR0_EL1 * 2)
+#define cp14_DBGDCCINT (MDCCINT_EL1 * 2)
+
+#define NR_COPRO_REGS (NR_SYS_REGS * 2)
+
struct kvm_cpu_context {
struct kvm_regs gp_regs;
union {
diff --git a/arch/arm64/include/asm/kvm_mmio.h b/arch/arm64/include/asm/kvm_mmio.h
index 889c908..fe612a9 100644
--- a/arch/arm64/include/asm/kvm_mmio.h
+++ b/arch/arm64/include/asm/kvm_mmio.h
@@ -19,7 +19,6 @@
#define __ARM64_KVM_MMIO_H__
#include <linux/kvm_host.h>
-#include <asm/kvm_asm.h>
#include <asm/kvm_arm.h>
/*
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 25de8b2..4b72231 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -112,6 +112,7 @@ int main(void)
DEFINE(CPU_ELR_EL1, offsetof(struct kvm_regs, elr_el1));
DEFINE(CPU_SPSR, offsetof(struct kvm_regs, spsr));
DEFINE(CPU_SYSREGS, offsetof(struct kvm_cpu_context, sys_regs));
+ DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
DEFINE(VCPU_ESR_EL2, offsetof(struct kvm_vcpu, arch.fault.esr_el2));
DEFINE(VCPU_FAR_EL2, offsetof(struct kvm_vcpu, arch.fault.far_el2));
DEFINE(VCPU_HPFAR_EL2, offsetof(struct kvm_vcpu, arch.fault.hpfar_el2));
diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
index d250160..88e59f2 100644
--- a/arch/arm64/kvm/guest.c
+++ b/arch/arm64/kvm/guest.c
@@ -28,7 +28,6 @@
#include <asm/cputype.h>
#include <asm/uaccess.h>
#include <asm/kvm.h>
-#include <asm/kvm_asm.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_coproc.h>
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 68a0759..4323627 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -23,6 +23,7 @@
#include <linux/kvm_host.h>
#include <asm/esr.h>
+#include <asm/kvm_asm.h>
#include <asm/kvm_coproc.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_mmu.h>
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index afd0a53..774a3f69 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -18,6 +18,7 @@
#include <linux/compiler.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_asm.h>
#include <asm/kvm_mmu.h>
#include "hyp.h"
diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
index 7552922..599911c 100644
--- a/arch/arm64/kvm/hyp/entry.S
+++ b/arch/arm64/kvm/hyp/entry.S
@@ -27,7 +27,6 @@
#define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
-#define CPU_SYSREG_OFFSET(x) (CPU_SYSREGS + 8*x)
.text
.pushsection .hyp.text, "ax"
@@ -174,7 +173,7 @@ ENTRY(__fpsimd_guest_restore)
mrs x1, hcr_el2
tbnz x1, #HCR_RW_SHIFT, 1f
- ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
+ ldr x4, [x3, #VCPU_FPEXC32_EL2]
msr fpexc32_el2, x4
1:
pop x4, lr
diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
index 41b9d30..b893c45 100644
--- a/arch/arm64/kvm/hyp/sysreg-sr.c
+++ b/arch/arm64/kvm/hyp/sysreg-sr.c
@@ -18,6 +18,7 @@
#include <linux/compiler.h>
#include <linux/kvm_host.h>
+#include <asm/kvm_asm.h>
#include <asm/kvm_mmu.h>
#include "hyp.h"
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 87a64e8..0db5311 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -29,6 +29,7 @@
#include <asm/debug-monitors.h>
#include <asm/esr.h>
#include <asm/kvm_arm.h>
+#include <asm/kvm_asm.h>
#include <asm/kvm_coproc.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_host.h>
diff --git a/virt/kvm/arm/vgic-v3.c b/virt/kvm/arm/vgic-v3.c
index 487d635..c8506a2 100644
--- a/virt/kvm/arm/vgic-v3.c
+++ b/virt/kvm/arm/vgic-v3.c
@@ -28,6 +28,7 @@
#include <asm/kvm_emulate.h>
#include <asm/kvm_arm.h>
+#include <asm/kvm_asm.h>
#include <asm/kvm_mmu.h>
/* These are for GICv2 emulation only */
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 20/21] arm64: KVM: Cleanup asm-offset.c
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (18 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:51 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 21/21] arm64: KVM: Remove weak attributes Marc Zyngier
2015-11-30 20:33 ` [PATCH v2 00/21] arm64: KVM: world switch in C Christoffer Dall
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
As we've now rewritten most of our code-base in C, most of the
KVM-specific code in asm-offset.c is useless. Delete-time again!
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kernel/asm-offsets.c | 39 ---------------------------------------
1 file changed, 39 deletions(-)
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 4b72231..94090a6 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -108,50 +108,11 @@ int main(void)
DEFINE(CPU_GP_REGS, offsetof(struct kvm_cpu_context, gp_regs));
DEFINE(CPU_USER_PT_REGS, offsetof(struct kvm_regs, regs));
DEFINE(CPU_FP_REGS, offsetof(struct kvm_regs, fp_regs));
- DEFINE(CPU_SP_EL1, offsetof(struct kvm_regs, sp_el1));
- DEFINE(CPU_ELR_EL1, offsetof(struct kvm_regs, elr_el1));
- DEFINE(CPU_SPSR, offsetof(struct kvm_regs, spsr));
- DEFINE(CPU_SYSREGS, offsetof(struct kvm_cpu_context, sys_regs));
DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
DEFINE(VCPU_ESR_EL2, offsetof(struct kvm_vcpu, arch.fault.esr_el2));
DEFINE(VCPU_FAR_EL2, offsetof(struct kvm_vcpu, arch.fault.far_el2));
DEFINE(VCPU_HPFAR_EL2, offsetof(struct kvm_vcpu, arch.fault.hpfar_el2));
- DEFINE(VCPU_DEBUG_FLAGS, offsetof(struct kvm_vcpu, arch.debug_flags));
- DEFINE(VCPU_DEBUG_PTR, offsetof(struct kvm_vcpu, arch.debug_ptr));
- DEFINE(DEBUG_BCR, offsetof(struct kvm_guest_debug_arch, dbg_bcr));
- DEFINE(DEBUG_BVR, offsetof(struct kvm_guest_debug_arch, dbg_bvr));
- DEFINE(DEBUG_WCR, offsetof(struct kvm_guest_debug_arch, dbg_wcr));
- DEFINE(DEBUG_WVR, offsetof(struct kvm_guest_debug_arch, dbg_wvr));
- DEFINE(VCPU_HCR_EL2, offsetof(struct kvm_vcpu, arch.hcr_el2));
- DEFINE(VCPU_MDCR_EL2, offsetof(struct kvm_vcpu, arch.mdcr_el2));
- DEFINE(VCPU_IRQ_LINES, offsetof(struct kvm_vcpu, arch.irq_lines));
DEFINE(VCPU_HOST_CONTEXT, offsetof(struct kvm_vcpu, arch.host_cpu_context));
- DEFINE(VCPU_HOST_DEBUG_STATE, offsetof(struct kvm_vcpu, arch.host_debug_state));
- DEFINE(VCPU_TIMER_CNTV_CTL, offsetof(struct kvm_vcpu, arch.timer_cpu.cntv_ctl));
- DEFINE(VCPU_TIMER_CNTV_CVAL, offsetof(struct kvm_vcpu, arch.timer_cpu.cntv_cval));
- DEFINE(KVM_TIMER_CNTVOFF, offsetof(struct kvm, arch.timer.cntvoff));
- DEFINE(KVM_TIMER_ENABLED, offsetof(struct kvm, arch.timer.enabled));
- DEFINE(VCPU_KVM, offsetof(struct kvm_vcpu, kvm));
- DEFINE(VCPU_VGIC_CPU, offsetof(struct kvm_vcpu, arch.vgic_cpu));
- DEFINE(VGIC_V2_CPU_HCR, offsetof(struct vgic_cpu, vgic_v2.vgic_hcr));
- DEFINE(VGIC_V2_CPU_VMCR, offsetof(struct vgic_cpu, vgic_v2.vgic_vmcr));
- DEFINE(VGIC_V2_CPU_MISR, offsetof(struct vgic_cpu, vgic_v2.vgic_misr));
- DEFINE(VGIC_V2_CPU_EISR, offsetof(struct vgic_cpu, vgic_v2.vgic_eisr));
- DEFINE(VGIC_V2_CPU_ELRSR, offsetof(struct vgic_cpu, vgic_v2.vgic_elrsr));
- DEFINE(VGIC_V2_CPU_APR, offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
- DEFINE(VGIC_V2_CPU_LR, offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
- DEFINE(VGIC_V3_CPU_SRE, offsetof(struct vgic_cpu, vgic_v3.vgic_sre));
- DEFINE(VGIC_V3_CPU_HCR, offsetof(struct vgic_cpu, vgic_v3.vgic_hcr));
- DEFINE(VGIC_V3_CPU_VMCR, offsetof(struct vgic_cpu, vgic_v3.vgic_vmcr));
- DEFINE(VGIC_V3_CPU_MISR, offsetof(struct vgic_cpu, vgic_v3.vgic_misr));
- DEFINE(VGIC_V3_CPU_EISR, offsetof(struct vgic_cpu, vgic_v3.vgic_eisr));
- DEFINE(VGIC_V3_CPU_ELRSR, offsetof(struct vgic_cpu, vgic_v3.vgic_elrsr));
- DEFINE(VGIC_V3_CPU_AP0R, offsetof(struct vgic_cpu, vgic_v3.vgic_ap0r));
- DEFINE(VGIC_V3_CPU_AP1R, offsetof(struct vgic_cpu, vgic_v3.vgic_ap1r));
- DEFINE(VGIC_V3_CPU_LR, offsetof(struct vgic_cpu, vgic_v3.vgic_lr));
- DEFINE(VGIC_CPU_NR_LR, offsetof(struct vgic_cpu, nr_lr));
- DEFINE(KVM_VTTBR, offsetof(struct kvm, arch.vttbr));
- DEFINE(KVM_VGIC_VCTRL, offsetof(struct kvm, arch.vgic.vctrl_base));
#endif
#ifdef CONFIG_CPU_PM
DEFINE(CPU_SUSPEND_SZ, sizeof(struct cpu_suspend_ctx));
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 21/21] arm64: KVM: Remove weak attributes
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (19 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 20/21] arm64: KVM: Cleanup asm-offset.c Marc Zyngier
@ 2015-11-27 18:50 ` Marc Zyngier
2015-12-02 11:47 ` Christoffer Dall
2015-11-30 20:33 ` [PATCH v2 00/21] arm64: KVM: world switch in C Christoffer Dall
21 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-11-27 18:50 UTC (permalink / raw)
To: linux-arm-kernel
As we've now switched to the new world switch implementation,
remove the weak attributes, as nobody is supposed to override
it anymore.
Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
---
arch/arm64/kvm/hyp/debug-sr.c | 5 ++---
arch/arm64/kvm/hyp/hyp-entry.S | 3 ---
arch/arm64/kvm/hyp/switch.c | 5 ++---
arch/arm64/kvm/hyp/tlb.c | 16 +++++++---------
arch/arm64/kvm/hyp/vgic-v3-sr.c | 5 ++---
5 files changed, 13 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
index 774a3f69..747546b 100644
--- a/arch/arm64/kvm/hyp/debug-sr.c
+++ b/arch/arm64/kvm/hyp/debug-sr.c
@@ -125,10 +125,9 @@ void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
}
}
-u32 __hyp_text __debug_read_mdcr_el2(void)
+static u32 __hyp_text __debug_read_mdcr_el2(void)
{
return read_sysreg(mdcr_el2);
}
-__alias(__debug_read_mdcr_el2)
-u32 __weak __kvm_get_mdcr_el2(void);
+__alias(__debug_read_mdcr_el2) u32 __kvm_get_mdcr_el2(void);
diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
index ace919b..bbc0be1 100644
--- a/arch/arm64/kvm/hyp/hyp-entry.S
+++ b/arch/arm64/kvm/hyp/hyp-entry.S
@@ -184,9 +184,7 @@ ENDPROC(\label)
.align 11
- .weak __kvm_hyp_vector
ENTRY(__kvm_hyp_vector)
-ENTRY(__hyp_vector)
ventry el2t_sync_invalid // Synchronous EL2t
ventry el2t_irq_invalid // IRQ EL2t
ventry el2t_fiq_invalid // FIQ EL2t
@@ -206,5 +204,4 @@ ENTRY(__hyp_vector)
ventry el1_irq // IRQ 32-bit EL1
ventry el1_fiq_invalid // FIQ 32-bit EL1
ventry el1_error_invalid // Error 32-bit EL1
-ENDPROC(__hyp_vector)
ENDPROC(__kvm_hyp_vector)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index c8ba370..1154d66 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -84,7 +84,7 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
__vgic_call_restore_state()(vcpu);
}
-int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
+static int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
{
struct kvm_cpu_context *host_ctxt;
struct kvm_cpu_context *guest_ctxt;
@@ -141,8 +141,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
return exit_code;
}
-__alias(__guest_run)
-int __weak __kvm_vcpu_run(struct kvm_vcpu *vcpu);
+__alias(__guest_run) int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
index 2c279a8..250e06c 100644
--- a/arch/arm64/kvm/hyp/tlb.c
+++ b/arch/arm64/kvm/hyp/tlb.c
@@ -17,7 +17,7 @@
#include "hyp.h"
-void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
+static void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
{
dsb(ishst);
@@ -47,10 +47,10 @@ void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
write_sysreg(0, vttbr_el2);
}
-__alias(__tlb_flush_vmid_ipa)
-void __weak __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
+__alias(__tlb_flush_vmid_ipa) void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm,
+ phys_addr_t ipa);
-void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
+static void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
{
dsb(ishst);
@@ -66,10 +66,9 @@ void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
write_sysreg(0, vttbr_el2);
}
-__alias(__tlb_flush_vmid)
-void __weak __kvm_tlb_flush_vmid(struct kvm *kvm);
+__alias(__tlb_flush_vmid) void __kvm_tlb_flush_vmid(struct kvm *kvm);
-void __hyp_text __tlb_flush_vm_context(void)
+static void __hyp_text __tlb_flush_vm_context(void)
{
dsb(ishst);
asm volatile("tlbi alle1is \n"
@@ -77,5 +76,4 @@ void __hyp_text __tlb_flush_vm_context(void)
dsb(ish);
}
-__alias(__tlb_flush_vm_context)
-void __weak __kvm_flush_vm_context(void);
+__alias(__tlb_flush_vm_context) void __kvm_flush_vm_context(void);
diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
index 1b0eedb..82a4f4b 100644
--- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
+++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
@@ -216,10 +216,9 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
}
}
-u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
+static u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
{
return read_gicreg(ICH_VTR_EL2);
}
-__alias(__vgic_v3_read_ich_vtr_el2)
-u64 __weak __vgic_v3_get_ich_vtr_el2(void);
+__alias(__vgic_v3_read_ich_vtr_el2) u64 __vgic_v3_get_ich_vtr_el2(void);
--
2.1.4
^ permalink raw reply related [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-11-27 18:49 ` [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore Marc Zyngier
@ 2015-11-30 9:59 ` Alex Bennée
2015-11-30 10:43 ` Marc Zyngier
2015-11-30 19:50 ` Christoffer Dall
1 sibling, 1 reply; 88+ messages in thread
From: Alex Bennée @ 2015-11-30 9:59 UTC (permalink / raw)
To: linux-arm-kernel
Marc Zyngier <marc.zyngier@arm.com> writes:
> Implement the vgic-v3 save restore as a direct translation of
> the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/hyp.h | 3 +
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 226 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index d8d5968..d1e38ce 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -3,3 +3,4 @@
> #
>
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 78f25c4..a31cb6e 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -30,5 +30,8 @@
> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>
> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HYP_H__ */
>
> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> new file mode 100644
> index 0000000..b490db5
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> @@ -0,0 +1,222 @@
> +/*
> + * Copyright (C) 2012-2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/compiler.h>
> +#include <linux/irqchip/arm-gic-v3.h>
This include starts spitting out compiler warnings due to use of
undefined barrier primitives. I'm not sure where the best place to:
#include <asm/barrier.h>
is. I added it to:
arch/arm64/include/asm/arch_gicv3.h
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_mmu.h>
> +
> +#include "hyp.h"
> +
> +/*
> + * We store LRs in reverse order to let the CPU deal with streaming
> + * access. Use this macro to make it look saner...
> + */
> +#define LR_OFFSET(n) (15 - n)
> +
> +#define read_gicreg(r) \
> + ({ \
> + u64 reg; \
> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
> + reg; \
> + })
> +
> +#define write_gicreg(v,r) \
> + do { \
> + u64 __val = (v); \
> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
> + } while (0)
> +
> +/* vcpu is already in the HYP VA space */
> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> + u64 val;
> + u32 nr_lr, nr_pri;
> +
> + /*
> + * Make sure stores to the GIC via the memory mapped interface
> + * are now visible to the system register interface.
> + */
> + dsb(st);
> +
> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> +
> + write_gicreg(0, ICH_HCR_EL2);
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_lr = val & 0xf;
> + nr_pri = ((u32)val >> 29) + 1;
> +
> + switch (nr_lr) {
> + case 15:
> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
> + case 14:
> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
> + case 13:
> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
> + case 12:
> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
> + case 11:
> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
> + case 10:
> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
> + case 9:
> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
> + case 8:
> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
> + case 7:
> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
> + case 6:
> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
> + case 5:
> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
> + case 4:
> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
> + case 3:
> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
> + case 2:
> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
> + case 1:
> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
> + case 0:
> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
> + }
> +
> + switch (nr_pri) {
> + case 7:
> + cpu_if->vgic_ap0r[3] = read_gicreg(ICH_AP0R3_EL2);
> + cpu_if->vgic_ap0r[2] = read_gicreg(ICH_AP0R2_EL2);
> + case 6:
> + cpu_if->vgic_ap0r[1] = read_gicreg(ICH_AP0R1_EL2);
> + default:
> + cpu_if->vgic_ap0r[0] = read_gicreg(ICH_AP0R0_EL2);
> + }
> +
> + switch (nr_pri) {
> + case 7:
> + cpu_if->vgic_ap1r[3] = read_gicreg(ICH_AP1R3_EL2);
> + cpu_if->vgic_ap1r[2] = read_gicreg(ICH_AP1R2_EL2);
> + case 6:
> + cpu_if->vgic_ap1r[1] = read_gicreg(ICH_AP1R1_EL2);
> + default:
> + cpu_if->vgic_ap1r[0] = read_gicreg(ICH_AP1R0_EL2);
> + }
> +
> + write_gicreg(read_gicreg(ICC_SRE_EL2) | ICC_SRE_EL2_ENABLE,
> + ICC_SRE_EL2);
> + isb();
> + write_gicreg(1, ICC_SRE_EL1);
> +}
> +
> +void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> + u64 val;
> + u32 nr_lr, nr_pri;
> +
> + /* Make sure SRE is valid before writing the other registers */
> + write_gicreg(cpu_if->vgic_sre, ICC_SRE_EL1);
> + isb();
> +
> + write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
> + write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
> +
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_lr = val & 0xf;
> + nr_pri = ((u32)val >> 29) + 1;
> +
> + switch (nr_pri) {
> + case 7:
> + write_gicreg(cpu_if->vgic_ap1r[3], ICH_AP1R3_EL2);
> + write_gicreg(cpu_if->vgic_ap1r[2], ICH_AP1R2_EL2);
> + case 6:
> + write_gicreg(cpu_if->vgic_ap1r[1], ICH_AP1R1_EL2);
> + default:
> + write_gicreg(cpu_if->vgic_ap1r[0], ICH_AP1R0_EL2);
> + }
> +
> + switch (nr_pri) {
> + case 7:
> + write_gicreg(cpu_if->vgic_ap0r[3], ICH_AP0R3_EL2);
> + write_gicreg(cpu_if->vgic_ap0r[2], ICH_AP0R2_EL2);
> + case 6:
> + write_gicreg(cpu_if->vgic_ap0r[1], ICH_AP0R1_EL2);
> + default:
> + write_gicreg(cpu_if->vgic_ap0r[0], ICH_AP0R0_EL2);
> + }
> +
> + switch (nr_lr) {
> + case 15:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(15)], ICH_LR15_EL2);
> + case 14:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(14)], ICH_LR14_EL2);
> + case 13:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(13)], ICH_LR13_EL2);
> + case 12:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(12)], ICH_LR12_EL2);
> + case 11:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(11)], ICH_LR11_EL2);
> + case 10:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(10)], ICH_LR10_EL2);
> + case 9:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(9)], ICH_LR9_EL2);
> + case 8:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(8)], ICH_LR8_EL2);
> + case 7:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(7)], ICH_LR7_EL2);
> + case 6:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(6)], ICH_LR6_EL2);
> + case 5:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(5)], ICH_LR5_EL2);
> + case 4:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(4)], ICH_LR4_EL2);
> + case 3:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(3)], ICH_LR3_EL2);
> + case 2:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(2)], ICH_LR2_EL2);
> + case 1:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(1)], ICH_LR1_EL2);
> + case 0:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(0)], ICH_LR0_EL2);
> + }
> +
> + /*
> + * Ensure that the above will have reached the
> + * (re)distributors. This ensure the guest will read the
> + * correct values from the memory-mapped interface.
> + */
> + isb();
> + dsb(sy);
> +
> + /*
> + * Prevent the guest from touching the GIC system registers if
> + * SRE isn't enabled for GICv3 emulation.
> + */
> + if (!cpu_if->vgic_sre) {
> + write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
> + ICC_SRE_EL2);
> + }
> +}
> +
> +u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
> +{
> + return read_gicreg(ICH_VTR_EL2);
> +}
--
Alex Benn?e
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-11-30 9:59 ` Alex Bennée
@ 2015-11-30 10:43 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-11-30 10:43 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, 30 Nov 2015 09:59:32 +0000
Alex Benn?e <alex.bennee@linaro.org> wrote:
>
> Marc Zyngier <marc.zyngier@arm.com> writes:
>
> > Implement the vgic-v3 save restore as a direct translation of
> > the assembly code version.
> >
> > Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> > ---
> > arch/arm64/kvm/hyp/Makefile | 1 +
> > arch/arm64/kvm/hyp/hyp.h | 3 +
> > arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
> > 3 files changed, 226 insertions(+)
> > create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
> >
> > diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> > index d8d5968..d1e38ce 100644
> > --- a/arch/arm64/kvm/hyp/Makefile
> > +++ b/arch/arm64/kvm/hyp/Makefile
> > @@ -3,3 +3,4 @@
> > #
> >
> > obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> > +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> > diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> > index 78f25c4..a31cb6e 100644
> > --- a/arch/arm64/kvm/hyp/hyp.h
> > +++ b/arch/arm64/kvm/hyp/hyp.h
> > @@ -30,5 +30,8 @@
> > void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> > void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >
> > +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> > +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> > +
> > #endif /* __ARM64_KVM_HYP_H__ */
> >
> > diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> > new file mode 100644
> > index 0000000..b490db5
> > --- /dev/null
> > +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> > @@ -0,0 +1,222 @@
> > +/*
> > + * Copyright (C) 2012-2015 - ARM Ltd
> > + * Author: Marc Zyngier <marc.zyngier@arm.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU General Public License
> > + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> > + */
> > +
> > +#include <linux/compiler.h>
> > +#include <linux/irqchip/arm-gic-v3.h>
>
> This include starts spitting out compiler warnings due to use of
> undefined barrier primitives. I'm not sure where the best place to:
>
> #include <asm/barrier.h>
>
> is. I added it to:
>
> arch/arm64/include/asm/arch_gicv3.h
I already have a couple of fixes queued to that effect in my tree,
hopefully heading for 4.4-rc4. If you pull the branch I have on korg,
you'll get the whole thing that should compile without warning.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-11-27 18:50 ` [PATCH v2 08/21] arm64: KVM: Implement debug save/restore Marc Zyngier
@ 2015-11-30 12:00 ` Alex Bennée
2015-11-30 12:24 ` Marc Zyngier
2015-12-01 12:56 ` Christoffer Dall
1 sibling, 1 reply; 88+ messages in thread
From: Alex Bennée @ 2015-11-30 12:00 UTC (permalink / raw)
To: linux-arm-kernel
Marc Zyngier <marc.zyngier@arm.com> writes:
> Implement the debug save restore as a direct translation of
> the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/hyp.h | 9 +++
> 3 files changed, 140 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index ec94200..ec14cac 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> new file mode 100644
> index 0000000..a0b2b99
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -0,0 +1,130 @@
> +/*
> + * Copyright (C) 2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/compiler.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_mmu.h>
> +
> +#include "hyp.h"
> +
> +#define read_debug(r,n) read_sysreg(r##n##_el1)
> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
> +
> +#define save_debug(ptr,reg,nr) \
> + switch (nr) { \
> + case 15: ptr[15] = read_debug(reg, 15); \
> + case 14: ptr[14] = read_debug(reg, 14); \
> + case 13: ptr[13] = read_debug(reg, 13); \
> + case 12: ptr[12] = read_debug(reg, 12); \
> + case 11: ptr[11] = read_debug(reg, 11); \
> + case 10: ptr[10] = read_debug(reg, 10); \
> + case 9: ptr[9] = read_debug(reg, 9); \
> + case 8: ptr[8] = read_debug(reg, 8); \
> + case 7: ptr[7] = read_debug(reg, 7); \
> + case 6: ptr[6] = read_debug(reg, 6); \
> + case 5: ptr[5] = read_debug(reg, 5); \
> + case 4: ptr[4] = read_debug(reg, 4); \
> + case 3: ptr[3] = read_debug(reg, 3); \
> + case 2: ptr[2] = read_debug(reg, 2); \
> + case 1: ptr[1] = read_debug(reg, 1); \
> + default: ptr[0] = read_debug(reg, 0); \
> + }
> +
> +#define restore_debug(ptr,reg,nr) \
> + switch (nr) { \
> + case 15: write_debug(ptr[15], reg, 15); \
> + case 14: write_debug(ptr[14], reg, 14); \
> + case 13: write_debug(ptr[13], reg, 13); \
> + case 12: write_debug(ptr[12], reg, 12); \
> + case 11: write_debug(ptr[11], reg, 11); \
> + case 10: write_debug(ptr[10], reg, 10); \
> + case 9: write_debug(ptr[9], reg, 9); \
> + case 8: write_debug(ptr[8], reg, 8); \
> + case 7: write_debug(ptr[7], reg, 7); \
> + case 6: write_debug(ptr[6], reg, 6); \
> + case 5: write_debug(ptr[5], reg, 5); \
> + case 4: write_debug(ptr[4], reg, 4); \
> + case 3: write_debug(ptr[3], reg, 3); \
> + case 2: write_debug(ptr[2], reg, 2); \
> + case 1: write_debug(ptr[1], reg, 1); \
> + default: write_debug(ptr[0], reg, 0); \
> + }
> +
> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> +{
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> + int brps, wrps;
> +
> + brps = (aa64dfr0 >> 12) & 0xf;
> + wrps = (aa64dfr0 >> 20) & 0xf;
I'm sure we have that defined - get_num_brps/wrps() - but I don't know
if the compiler would do a better job with it.
> +
> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
> +
> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> + }
> +}
> +
> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> +{
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> + int brps, wrps;
> +
> + brps = (aa64dfr0 >> 12) & 0xf;
> + wrps = (aa64dfr0 >> 20) & 0xf;
> +
> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
> +
> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> + }
> +}
> +
> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> +{
> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> +
> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
> + kern_hyp_va(vcpu->arch.host_cpu_context));
> +}
> +
> +void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> + __debug_restore_state(vcpu, &vcpu->arch.host_debug_state,
> + kern_hyp_va(vcpu->arch.host_cpu_context));
> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> + }
> +}
> +
> +u32 __hyp_text __debug_read_mdcr_el2(void)
> +{
> + return read_sysreg(mdcr_el2);
> +}
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 4639330..2581232 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -41,5 +41,14 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
> void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
>
> +void __debug_save_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt);
> +void __debug_restore_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt);
> +void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> +void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HYP_H__ */
I spun it up against my KVM debug tests and everything was fine. Have a:
Tested-by: Alex Benn?e <alex.bennee@linaro.org>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
--
Alex Benn?e
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-11-30 12:00 ` Alex Bennée
@ 2015-11-30 12:24 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-11-30 12:24 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, 30 Nov 2015 12:00:24 +0000
Alex Benn?e <alex.bennee@linaro.org> wrote:
>
> Marc Zyngier <marc.zyngier@arm.com> writes:
>
> > Implement the debug save restore as a direct translation of
> > the assembly code version.
> >
> > Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> > ---
> > arch/arm64/kvm/hyp/Makefile | 1 +
> > arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
> > arch/arm64/kvm/hyp/hyp.h | 9 +++
> > 3 files changed, 140 insertions(+)
> > create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
> >
> > diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> > index ec94200..ec14cac 100644
> > --- a/arch/arm64/kvm/hyp/Makefile
> > +++ b/arch/arm64/kvm/hyp/Makefile
> > @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> > obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> > obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> > obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> > +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> > diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > new file mode 100644
> > index 0000000..a0b2b99
> > --- /dev/null
> > +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > @@ -0,0 +1,130 @@
> > +/*
> > + * Copyright (C) 2015 - ARM Ltd
> > + * Author: Marc Zyngier <marc.zyngier@arm.com>
> > + *
> > + * This program is free software; you can redistribute it and/or modify
> > + * it under the terms of the GNU General Public License version 2 as
> > + * published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it will be useful,
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > + * GNU General Public License for more details.
> > + *
> > + * You should have received a copy of the GNU General Public License
> > + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> > + */
> > +
> > +#include <linux/compiler.h>
> > +#include <linux/kvm_host.h>
> > +
> > +#include <asm/kvm_mmu.h>
> > +
> > +#include "hyp.h"
> > +
> > +#define read_debug(r,n) read_sysreg(r##n##_el1)
> > +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
> > +
> > +#define save_debug(ptr,reg,nr) \
> > + switch (nr) { \
> > + case 15: ptr[15] = read_debug(reg, 15); \
> > + case 14: ptr[14] = read_debug(reg, 14); \
> > + case 13: ptr[13] = read_debug(reg, 13); \
> > + case 12: ptr[12] = read_debug(reg, 12); \
> > + case 11: ptr[11] = read_debug(reg, 11); \
> > + case 10: ptr[10] = read_debug(reg, 10); \
> > + case 9: ptr[9] = read_debug(reg, 9); \
> > + case 8: ptr[8] = read_debug(reg, 8); \
> > + case 7: ptr[7] = read_debug(reg, 7); \
> > + case 6: ptr[6] = read_debug(reg, 6); \
> > + case 5: ptr[5] = read_debug(reg, 5); \
> > + case 4: ptr[4] = read_debug(reg, 4); \
> > + case 3: ptr[3] = read_debug(reg, 3); \
> > + case 2: ptr[2] = read_debug(reg, 2); \
> > + case 1: ptr[1] = read_debug(reg, 1); \
> > + default: ptr[0] = read_debug(reg, 0); \
> > + }
> > +
> > +#define restore_debug(ptr,reg,nr) \
> > + switch (nr) { \
> > + case 15: write_debug(ptr[15], reg, 15); \
> > + case 14: write_debug(ptr[14], reg, 14); \
> > + case 13: write_debug(ptr[13], reg, 13); \
> > + case 12: write_debug(ptr[12], reg, 12); \
> > + case 11: write_debug(ptr[11], reg, 11); \
> > + case 10: write_debug(ptr[10], reg, 10); \
> > + case 9: write_debug(ptr[9], reg, 9); \
> > + case 8: write_debug(ptr[8], reg, 8); \
> > + case 7: write_debug(ptr[7], reg, 7); \
> > + case 6: write_debug(ptr[6], reg, 6); \
> > + case 5: write_debug(ptr[5], reg, 5); \
> > + case 4: write_debug(ptr[4], reg, 4); \
> > + case 3: write_debug(ptr[3], reg, 3); \
> > + case 2: write_debug(ptr[2], reg, 2); \
> > + case 1: write_debug(ptr[1], reg, 1); \
> > + default: write_debug(ptr[0], reg, 0); \
> > + }
> > +
> > +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt)
> > +{
> > + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> > + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > + int brps, wrps;
> > +
> > + brps = (aa64dfr0 >> 12) & 0xf;
> > + wrps = (aa64dfr0 >> 20) & 0xf;
>
> I'm sure we have that defined - get_num_brps/wrps() - but I don't know
> if the compiler would do a better job with it.
The problem is that it lives in the main kernel text, which makes it
unavailable to the hyp code.
> > +
> > + save_debug(dbg->dbg_bcr, dbgbcr, brps);
> > + save_debug(dbg->dbg_bvr, dbgbvr, brps);
> > + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
> > + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
> > +
> > + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > + }
> > +}
> > +
> > +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt)
> > +{
> > + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> > + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > + int brps, wrps;
> > +
> > + brps = (aa64dfr0 >> 12) & 0xf;
> > + wrps = (aa64dfr0 >> 20) & 0xf;
> > +
> > + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
> > + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
> > + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
> > + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
> > +
> > + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > + }
> > +}
> > +
> > +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > +{
> > + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> > + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
> > + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > +
> > + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
> > + kern_hyp_va(vcpu->arch.host_cpu_context));
> > +}
> > +
> > +void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> > +{
> > + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> > + __debug_restore_state(vcpu, &vcpu->arch.host_debug_state,
> > + kern_hyp_va(vcpu->arch.host_cpu_context));
> > + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> > + }
> > +}
> > +
> > +u32 __hyp_text __debug_read_mdcr_el2(void)
> > +{
> > + return read_sysreg(mdcr_el2);
> > +}
> > diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> > index 4639330..2581232 100644
> > --- a/arch/arm64/kvm/hyp/hyp.h
> > +++ b/arch/arm64/kvm/hyp/hyp.h
> > @@ -41,5 +41,14 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
> > void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> > void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
> >
> > +void __debug_save_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt);
> > +void __debug_restore_state(struct kvm_vcpu *vcpu,
> > + struct kvm_guest_debug_arch *dbg,
> > + struct kvm_cpu_context *ctxt);
> > +void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> > +void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> > +
> > #endif /* __ARM64_KVM_HYP_H__ */
>
> I spun it up against my KVM debug tests and everything was fine. Have a:
>
> Tested-by: Alex Benn?e <alex.bennee@linaro.org>
> Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
Awesome. Thanks Alex!
M.
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-11-27 18:49 ` [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore Marc Zyngier
2015-11-30 9:59 ` Alex Bennée
@ 2015-11-30 19:50 ` Christoffer Dall
2015-12-01 11:32 ` Marc Zyngier
1 sibling, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-11-30 19:50 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
> Implement the vgic-v3 save restore as a direct translation of
> the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/hyp.h | 3 +
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 226 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index d8d5968..d1e38ce 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -3,3 +3,4 @@
> #
>
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 78f25c4..a31cb6e 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -30,5 +30,8 @@
> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>
> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HYP_H__ */
>
> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> new file mode 100644
> index 0000000..b490db5
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> @@ -0,0 +1,222 @@
> +/*
> + * Copyright (C) 2012-2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/compiler.h>
> +#include <linux/irqchip/arm-gic-v3.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_mmu.h>
> +
> +#include "hyp.h"
> +
> +/*
> + * We store LRs in reverse order to let the CPU deal with streaming
> + * access. Use this macro to make it look saner...
> + */
> +#define LR_OFFSET(n) (15 - n)
> +
> +#define read_gicreg(r) \
> + ({ \
> + u64 reg; \
> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
> + reg; \
> + })
> +
> +#define write_gicreg(v,r) \
> + do { \
> + u64 __val = (v); \
> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
> + } while (0)
remind me what the msr_s and mrs_s do compared to msr and mrs?
are these the reason why we need separate macros to access the gic
registers compared to 'normal' sysregs?
> +
> +/* vcpu is already in the HYP VA space */
> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> + u64 val;
> + u32 nr_lr, nr_pri;
> +
> + /*
> + * Make sure stores to the GIC via the memory mapped interface
> + * are now visible to the system register interface.
> + */
> + dsb(st);
> +
> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> +
> + write_gicreg(0, ICH_HCR_EL2);
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_lr = val & 0xf;
this is not technically nr_lr, it's max_lr or max_lr_idx or something
like that.
> + nr_pri = ((u32)val >> 29) + 1;
nit: nr_pri_bits
> +
> + switch (nr_lr) {
> + case 15:
> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
> + case 14:
> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
> + case 13:
> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
> + case 12:
> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
> + case 11:
> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
> + case 10:
> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
> + case 9:
> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
> + case 8:
> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
> + case 7:
> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
> + case 6:
> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
> + case 5:
> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
> + case 4:
> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
> + case 3:
> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
> + case 2:
> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
> + case 1:
> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
> + case 0:
> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
> + }
> +
> + switch (nr_pri) {
> + case 7:
> + cpu_if->vgic_ap0r[3] = read_gicreg(ICH_AP0R3_EL2);
> + cpu_if->vgic_ap0r[2] = read_gicreg(ICH_AP0R2_EL2);
> + case 6:
> + cpu_if->vgic_ap0r[1] = read_gicreg(ICH_AP0R1_EL2);
> + default:
> + cpu_if->vgic_ap0r[0] = read_gicreg(ICH_AP0R0_EL2);
> + }
> +
> + switch (nr_pri) {
> + case 7:
> + cpu_if->vgic_ap1r[3] = read_gicreg(ICH_AP1R3_EL2);
> + cpu_if->vgic_ap1r[2] = read_gicreg(ICH_AP1R2_EL2);
> + case 6:
> + cpu_if->vgic_ap1r[1] = read_gicreg(ICH_AP1R1_EL2);
> + default:
> + cpu_if->vgic_ap1r[0] = read_gicreg(ICH_AP1R0_EL2);
> + }
> +
> + write_gicreg(read_gicreg(ICC_SRE_EL2) | ICC_SRE_EL2_ENABLE,
> + ICC_SRE_EL2);
nit: reading this out in a variable probably looks nicer.
> + isb();
nit: should we comment on why the isb is required?
> + write_gicreg(1, ICC_SRE_EL1);
> +}
> +
> +void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> +{
> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> + u64 val;
> + u32 nr_lr, nr_pri;
> +
> + /* Make sure SRE is valid before writing the other registers */
I know that I've reviewed this code before (the asm version) but coming
back to it I have no idea what the above really means...
> + write_gicreg(cpu_if->vgic_sre, ICC_SRE_EL1);
> + isb();
> +
> + write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
> + write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
> +
> + val = read_gicreg(ICH_VTR_EL2);
> + nr_lr = val & 0xf;
> + nr_pri = ((u32)val >> 29) + 1;
you could have a define for this shift now that you use it more than
once.
> +
> + switch (nr_pri) {
> + case 7:
> + write_gicreg(cpu_if->vgic_ap1r[3], ICH_AP1R3_EL2);
> + write_gicreg(cpu_if->vgic_ap1r[2], ICH_AP1R2_EL2);
> + case 6:
> + write_gicreg(cpu_if->vgic_ap1r[1], ICH_AP1R1_EL2);
> + default:
> + write_gicreg(cpu_if->vgic_ap1r[0], ICH_AP1R0_EL2);
> + }
> +
> + switch (nr_pri) {
> + case 7:
> + write_gicreg(cpu_if->vgic_ap0r[3], ICH_AP0R3_EL2);
> + write_gicreg(cpu_if->vgic_ap0r[2], ICH_AP0R2_EL2);
> + case 6:
> + write_gicreg(cpu_if->vgic_ap0r[1], ICH_AP0R1_EL2);
> + default:
> + write_gicreg(cpu_if->vgic_ap0r[0], ICH_AP0R0_EL2);
> + }
> +
> + switch (nr_lr) {
> + case 15:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(15)], ICH_LR15_EL2);
> + case 14:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(14)], ICH_LR14_EL2);
> + case 13:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(13)], ICH_LR13_EL2);
> + case 12:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(12)], ICH_LR12_EL2);
> + case 11:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(11)], ICH_LR11_EL2);
> + case 10:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(10)], ICH_LR10_EL2);
> + case 9:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(9)], ICH_LR9_EL2);
> + case 8:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(8)], ICH_LR8_EL2);
> + case 7:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(7)], ICH_LR7_EL2);
> + case 6:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(6)], ICH_LR6_EL2);
> + case 5:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(5)], ICH_LR5_EL2);
> + case 4:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(4)], ICH_LR4_EL2);
> + case 3:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(3)], ICH_LR3_EL2);
> + case 2:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(2)], ICH_LR2_EL2);
> + case 1:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(1)], ICH_LR1_EL2);
> + case 0:
> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(0)], ICH_LR0_EL2);
same question as above.
> + }
loads of trailing whitespace all over above
> +
> + /*
> + * Ensure that the above will have reached the
> + * (re)distributors. This ensure the guest will read the
s/ensure/ensures/
> + * correct values from the memory-mapped interface.
> + */
> + isb();
> + dsb(sy);
> +
> + /*
> + * Prevent the guest from touching the GIC system registers if
> + * SRE isn't enabled for GICv3 emulation.
so can we emulate a GICv3 to the guest without system register access?
I.e. an MMIO only GICv3 interface?
> + */
> + if (!cpu_if->vgic_sre) {
> + write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
> + ICC_SRE_EL2);
> + }
> +}
> +
> +u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
> +{
> + return read_gicreg(ICH_VTR_EL2);
> +}
> --
> 2.1.4
>
As for translating the assembly code to C, this patch looks correct.
Nevertheless, I felt obliged to ask into the details above, now when
you're touching all this code.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 05/21] arm64: KVM: Implement timer save/restore
2015-11-27 18:49 ` [PATCH v2 05/21] arm64: KVM: Implement timer save/restore Marc Zyngier
@ 2015-11-30 19:59 ` Christoffer Dall
2015-12-01 11:34 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-11-30 19:59 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:49:59PM +0000, Marc Zyngier wrote:
> Implement the timer save restore as a direct translation of
> the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/hyp.h | 3 ++
> arch/arm64/kvm/hyp/timer-sr.c | 71 ++++++++++++++++++++++++++++++++++++
> include/clocksource/arm_arch_timer.h | 6 +++
> 4 files changed, 81 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/timer-sr.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index d1e38ce..455dc0a 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -4,3 +4,4 @@
>
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index a31cb6e..86aa5a2 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -33,5 +33,8 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>
> +void __timer_save_state(struct kvm_vcpu *vcpu);
> +void __timer_restore_state(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HYP_H__ */
>
> diff --git a/arch/arm64/kvm/hyp/timer-sr.c b/arch/arm64/kvm/hyp/timer-sr.c
> new file mode 100644
> index 0000000..8e2209c
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/timer-sr.c
> @@ -0,0 +1,71 @@
> +/*
> + * Copyright (C) 2012-2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <clocksource/arm_arch_timer.h>
> +#include <linux/compiler.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_mmu.h>
> +
> +#include "hyp.h"
> +
> +/* vcpu is already in the HYP VA space */
> +void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> + struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
> +
> + if (kvm->arch.timer.enabled) {
> + timer->cntv_ctl = read_sysreg(cntv_ctl_el0);
> + isb();
> + timer->cntv_cval = read_sysreg(cntv_cval_el0);
> + }
> +
> + /* Disable the virtual timer */
> + write_sysreg(0, cntv_ctl_el0);
> +
> + /* Allow physical timer/counter access for the host */
> + write_sysreg((read_sysreg(cnthctl_el2) | CNTHCTL_EL1PCTEN |
> + CNTHCTL_EL1PCEN),
> + cnthctl_el2);
nit: again I probably prefer reading cnthctl_el2 into a variable, modify
the bits and write it back, but it's no big deal.
> +
> + /* Clear cntvoff for the host */
> + write_sysreg(0, cntvoff_el2);
why do we do this when we've just disabled the timer?
> +}
> +
> +void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> + struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
> + u64 val;
> +
> + /*
> + * Disallow physical timer access for the guest
> + * Physical counter access is allowed
> + */
> + val = read_sysreg(cnthctl_el2);
> + val &= ~CNTHCTL_EL1PCEN;
> + val |= CNTHCTL_EL1PCTEN;
> + write_sysreg(val, cnthctl_el2);
> +
> + if (kvm->arch.timer.enabled) {
> + write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
> + write_sysreg(timer->cntv_cval, cntv_cval_el0);
> + isb();
> + write_sysreg(timer->cntv_ctl, cntv_ctl_el0);
> + }
> +}
> diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm_arch_timer.h
> index 9916d0e..25d0914 100644
> --- a/include/clocksource/arm_arch_timer.h
> +++ b/include/clocksource/arm_arch_timer.h
> @@ -23,6 +23,12 @@
> #define ARCH_TIMER_CTRL_IT_MASK (1 << 1)
> #define ARCH_TIMER_CTRL_IT_STAT (1 << 2)
>
> +#define CNTHCTL_EL1PCTEN (1 << 0)
> +#define CNTHCTL_EL1PCEN (1 << 1)
> +#define CNTHCTL_EVNTEN (1 << 2)
> +#define CNTHCTL_EVNTDIR (1 << 3)
> +#define CNTHCTL_EVNTI (0xF << 4)
> +
> enum arch_timer_reg {
> ARCH_TIMER_REG_CTRL,
> ARCH_TIMER_REG_TVAL,
> --
> 2.1.4
>
Otherwise this looks good.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore
2015-11-27 18:49 ` [PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore Marc Zyngier
@ 2015-11-30 20:00 ` Christoffer Dall
2015-12-01 11:39 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-11-30 20:00 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:49:57PM +0000, Marc Zyngier wrote:
> Implement the vgic-v2 save restore (mostly) as a direct translation
> of the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/Makefile | 1 +
> arch/arm64/kvm/hyp/Makefile | 5 +++
> arch/arm64/kvm/hyp/hyp.h | 3 ++
> arch/arm64/kvm/hyp/vgic-v2-sr.c | 89 +++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 98 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/Makefile
> create mode 100644 arch/arm64/kvm/hyp/vgic-v2-sr.c
>
> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
> index 1949fe5..d31e4e5 100644
> --- a/arch/arm64/kvm/Makefile
> +++ b/arch/arm64/kvm/Makefile
> @@ -10,6 +10,7 @@ KVM=../../../virt/kvm
> ARM=../../../arch/arm/kvm
>
> obj-$(CONFIG_KVM_ARM_HOST) += kvm.o
> +obj-$(CONFIG_KVM_ARM_HOST) += hyp/
>
> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o $(KVM)/vfio.o
> kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/arm.o $(ARM)/mmu.o $(ARM)/mmio.o
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> new file mode 100644
> index 0000000..d8d5968
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -0,0 +1,5 @@
> +#
> +# Makefile for Kernel-based Virtual Machine module, HYP part
> +#
> +
> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index dac843e..78f25c4 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -27,5 +27,8 @@
>
> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
>
> +void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> +void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
should we call these flush/sync here now ?
> +
> #endif /* __ARM64_KVM_HYP_H__ */
>
> diff --git a/arch/arm64/kvm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
> new file mode 100644
> index 0000000..29a5c1d
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
> @@ -0,0 +1,89 @@
> +/*
> + * Copyright (C) 2012-2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/compiler.h>
> +#include <linux/irqchip/arm-gic.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_mmu.h>
> +
> +#include "hyp.h"
> +
> +/* vcpu is already in the HYP VA space */
should we annotate hyp pointers similarly to __user or will that be
confusing when VHE enters the scene ?
> +void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + struct vgic_dist *vgic = &kvm->arch.vgic;
> + void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> + u32 __iomem *lr_base;
> + u32 eisr0, eisr1, elrsr0, elrsr1;
> + int i = 0, nr_lr;
> +
> + if (!base)
> + return;
> +
> + nr_lr = vcpu->arch.vgic_cpu.nr_lr;
> + cpu_if->vgic_vmcr = readl_relaxed(base + GICH_VMCR);
> + cpu_if->vgic_misr = readl_relaxed(base + GICH_MISR);
> + eisr0 = readl_relaxed(base + GICH_EISR0);
> + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
> + if (unlikely(nr_lr > 32)) {
> + eisr1 = readl_relaxed(base + GICH_EISR1);
> + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
> + } else {
> + eisr1 = elrsr1 = 0;
> + }
> +#ifdef CONFIG_CPU_BIG_ENDIAN
> + cpu_if->vgic_eisr = ((u64)eisr0 << 32) | eisr1;
> + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
> +#else
> + cpu_if->vgic_eisr = ((u64)eisr1 << 32) | eisr0;
> + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
> +#endif
> + cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
> +
> + writel_relaxed(0, base + GICH_HCR);
> +
> + lr_base = base + GICH_LR0;
> + do {
> + cpu_if->vgic_lr[i++] = readl_relaxed(lr_base++);
> + } while (--nr_lr);
why not a simple for-loop?
> +}
> +
copy the vcpu HYP VA comment down here.
> +void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
> + struct vgic_dist *vgic = &kvm->arch.vgic;
> + void __iomem *base = kern_hyp_va(vgic->vctrl_base);
> + u32 __iomem *lr_base;
> + unsigned int i = 0, nr_lr;
> +
> + if (!base)
> + return;
> +
> + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
> + writel_relaxed(cpu_if->vgic_vmcr, base + GICH_VMCR);
> + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
> +
> + lr_base = base + GICH_LR0;
> + nr_lr = vcpu->arch.vgic_cpu.nr_lr;
> + do {
> + writel_relaxed(cpu_if->vgic_lr[i++], lr_base++);
> + } while (--nr_lr);
same question as above.
> +}
> --
> 2.1.4
>
Otherwise looks good.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file
2015-11-27 18:49 ` [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file Marc Zyngier
@ 2015-11-30 20:00 ` Christoffer Dall
2015-12-01 11:41 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-11-30 20:00 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:49:56PM +0000, Marc Zyngier wrote:
> In order to expose the various EL2 services that are private to
> the hypervisor, add a new hyp.h file.
>
> So far, it only contains mundane things such as section annotation
> and VA manipulation.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/hyp.h | 31 +++++++++++++++++++++++++++++++
> 1 file changed, 31 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/hyp.h
>
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> new file mode 100644
> index 0000000..dac843e
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -0,0 +1,31 @@
> +/*
> + * Copyright (C) 2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#ifndef __ARM64_KVM_HYP_H__
> +#define __ARM64_KVM_HYP_H__
> +
> +#include <linux/compiler.h>
> +#include <linux/kvm_host.h>
> +#include <asm/kvm_mmu.h>
> +#include <asm/sysreg.h>
> +
> +#define __hyp_text __section(.hyp.text) notrace
why notrace?
> +
> +#define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
should you have parenthesis around 'v' ?
> +
> +#endif /* __ARM64_KVM_HYP_H__ */
> +
> --
> 2.1.4
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 01/21] arm64: Add macros to read/write system registers
2015-11-27 18:49 ` [PATCH v2 01/21] arm64: Add macros to read/write system registers Marc Zyngier
@ 2015-11-30 20:00 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-11-30 20:00 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:49:55PM +0000, Marc Zyngier wrote:
> From: Mark Rutland <mark.rutland@arm.com>
>
> Rather than crafting custom macros for reading/writing each system
> register provide generics accessors, read_sysreg and write_sysreg, for
> this purpose.
>
> Unlike read_cpuid, calls to read_exception_reg are never expected
> to be optimized away or replaced with synthetic values.
how does this comment about read_exception_reg relate to this patch?
>
> Signed-off-by: Mark Rutland <mark.rutland@arm.com>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Suzuki Poulose <suzuki.poulose@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/include/asm/sysreg.h | 17 +++++++++++++++++
> 1 file changed, 17 insertions(+)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index d48ab5b..c9c283a 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -20,6 +20,8 @@
> #ifndef __ASM_SYSREG_H
> #define __ASM_SYSREG_H
>
> +#include <linux/stringify.h>
> +
> #include <asm/opcodes.h>
>
> /*
> @@ -208,6 +210,8 @@
>
> #else
>
> +#include <linux/types.h>
> +
> asm(
> " .irp num,0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30\n"
> " .equ __reg_num_x\\num, \\num\n"
> @@ -232,6 +236,19 @@ static inline void config_sctlr_el1(u32 clear, u32 set)
> val |= set;
> asm volatile("msr sctlr_el1, %0" : : "r" (val));
> }
> +
> +#define read_sysreg(r) ({ \
> + u64 __val; \
> + asm volatile("mrs %0, " __stringify(r) : "=r" (__val)); \
> + __val; \
> +})
> +
> +#define write_sysreg(v, r) do { \
> + u64 __val = (u64)v; \
> + asm volatile("msr " __stringify(r) ", %0" \
> + : : "r" (__val)); \
> +} while (0)
> +
> #endif
>
> #endif /* __ASM_SYSREG_H */
> --
> 2.1.4
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 00/21] arm64: KVM: world switch in C
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
` (20 preceding siblings ...)
2015-11-27 18:50 ` [PATCH v2 21/21] arm64: KVM: Remove weak attributes Marc Zyngier
@ 2015-11-30 20:33 ` Christoffer Dall
2015-12-01 3:19 ` Mario Smarduch
2015-12-01 9:58 ` Marc Zyngier
21 siblings, 2 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-11-30 20:33 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:49:54PM +0000, Marc Zyngier wrote:
> Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
> and mean piece of hand-crafted assembly code. Over time, features have
> crept in, the code has become harder to maintain, and the smallest
> change is a pain to introduce. The VHE patches are a prime example of
> why this doesn't work anymore.
>
> This series rewrites most of the existing assembly code in C, but keeps
> the existing code structure in place (most function names will look
> familiar to the reader). The biggest change is that we don't have to
> deal with a static register allocation (the compiler does it for us),
> we can easily follow structure and pointers, and only the lowest level
> is still in assembly code. Oh, and a negative diffstat.
>
> There is still a healthy dose of inline assembly (system register
> accessors, runtime code patching), but I've tried not to make it too
> invasive. The generated code, while not exactly brilliant, doesn't
> look too shaby. I do expect a small performance degradation, but I
> believe this is something we can improve over time (my initial
> measurements don't show any obvious regression though).
I ran this through my experimental setup on m400 and got this:
BM v4.4-rc2 v4.4-rc2-wsinc overhead
-- -------- -------------- --------
Apache 5297.11 5243.77 101.02%
fio rand read 4354.33 4294.50 101.39%
fio rand write 2465.33 2231.33 110.49%
hackbench 17.48 19.78 113.16%
memcached 96442.69 101274.04 95.23%
TCP_MAERTS 5966.89 6029.72 98.96%
TCP_STREAM 6284.60 6351.74 98.94%
TCP_RR 15044.71 14324.03 105.03%
pbzip2 c 18.13 17.89 98.68%
pbzip2 d 11.42 11.45 100.26%
kernbench 50.13 50.28 100.30%
mysql 1 152.84 154.01 100.77%
mysql 2 98.12 98.94 100.84%
mysql 4 51.32 51.17 99.71%
mysql 8 27.31 27.70 101.42%
mysql 20 16.80 17.21 102.47%
mysql 100 13.71 14.11 102.92%
mysql 200 15.20 15.20 100.00%
mysql 400 17.16 17.16 100.00%
(you want to see this with a viewer that renders clear-text and tabs
properly)
What this tells me is that we do take a noticable hit on the
world-switch path, which shows up in the TCP_RR and hackbench workloads,
which have a high precision in their output.
Note that the memcached number is well within its variability between
individual benchmark runs, where it varies to 12% of its average in over
80% of the executions.
I don't think this is a showstopper thought, but we could consider
looking more closely at a breakdown of the world-switch path and verify
if/where we are really taking a hit.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 00/21] arm64: KVM: world switch in C
2015-11-30 20:33 ` [PATCH v2 00/21] arm64: KVM: world switch in C Christoffer Dall
@ 2015-12-01 3:19 ` Mario Smarduch
2015-12-01 9:58 ` Marc Zyngier
1 sibling, 0 replies; 88+ messages in thread
From: Mario Smarduch @ 2015-12-01 3:19 UTC (permalink / raw)
To: linux-arm-kernel
On 11/30/2015 12:33 PM, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:49:54PM +0000, Marc Zyngier wrote:
>> Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
>> and mean piece of hand-crafted assembly code. Over time, features have
>> crept in, the code has become harder to maintain, and the smallest
>> change is a pain to introduce. The VHE patches are a prime example of
>> why this doesn't work anymore.
>>
>> This series rewrites most of the existing assembly code in C, but keeps
>> the existing code structure in place (most function names will look
>> familiar to the reader). The biggest change is that we don't have to
>> deal with a static register allocation (the compiler does it for us),
>> we can easily follow structure and pointers, and only the lowest level
>> is still in assembly code. Oh, and a negative diffstat.
>>
>> There is still a healthy dose of inline assembly (system register
>> accessors, runtime code patching), but I've tried not to make it too
>> invasive. The generated code, while not exactly brilliant, doesn't
>> look too shaby. I do expect a small performance degradation, but I
>> believe this is something we can improve over time (my initial
>> measurements don't show any obvious regression though).
>
> I ran this through my experimental setup on m400 and got this:
>
> BM v4.4-rc2 v4.4-rc2-wsinc overhead
> -- -------- -------------- --------
> Apache 5297.11 5243.77 101.02%
> fio rand read 4354.33 4294.50 101.39%
> fio rand write 2465.33 2231.33 110.49%
> hackbench 17.48 19.78 113.16%
> memcached 96442.69 101274.04 95.23%
> TCP_MAERTS 5966.89 6029.72 98.96%
> TCP_STREAM 6284.60 6351.74 98.94%
> TCP_RR 15044.71 14324.03 105.03%
> pbzip2 c 18.13 17.89 98.68%
> pbzip2 d 11.42 11.45 100.26%
> kernbench 50.13 50.28 100.30%
> mysql 1 152.84 154.01 100.77%
> mysql 2 98.12 98.94 100.84%
> mysql 4 51.32 51.17 99.71%
> mysql 8 27.31 27.70 101.42%
> mysql 20 16.80 17.21 102.47%
> mysql 100 13.71 14.11 102.92%
> mysql 200 15.20 15.20 100.00%
> mysql 400 17.16 17.16 100.00%
>
> (you want to see this with a viewer that renders clear-text and tabs
> properly)
>
> What this tells me is that we do take a noticable hit on the
> world-switch path, which shows up in the TCP_RR and hackbench workloads,
> which have a high precision in their output.
>
> Note that the memcached number is well within its variability between
> individual benchmark runs, where it varies to 12% of its average in over
> 80% of the executions.
>
> I don't think this is a showstopper thought, but we could consider
> looking more closely at a breakdown of the world-switch path and verify
> if/where we are really taking a hit.
>
> -Christoffer
> _______________________________________________
> kvmarm mailing list
> kvmarm at lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
I ran some of the lmbench 'micro benchmarks' - currently
the usleep one consistently stands out by about .4% or extra 300ns
per sleep. Few other ones have some outliers, I will look at these
closer. Tests were ran on Juno.
- Mario
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 00/21] arm64: KVM: world switch in C
2015-11-30 20:33 ` [PATCH v2 00/21] arm64: KVM: world switch in C Christoffer Dall
2015-12-01 3:19 ` Mario Smarduch
@ 2015-12-01 9:58 ` Marc Zyngier
2015-12-01 12:00 ` Christoffer Dall
1 sibling, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 9:58 UTC (permalink / raw)
To: linux-arm-kernel
On 30/11/15 20:33, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:49:54PM +0000, Marc Zyngier wrote:
>> Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
>> and mean piece of hand-crafted assembly code. Over time, features have
>> crept in, the code has become harder to maintain, and the smallest
>> change is a pain to introduce. The VHE patches are a prime example of
>> why this doesn't work anymore.
>>
>> This series rewrites most of the existing assembly code in C, but keeps
>> the existing code structure in place (most function names will look
>> familiar to the reader). The biggest change is that we don't have to
>> deal with a static register allocation (the compiler does it for us),
>> we can easily follow structure and pointers, and only the lowest level
>> is still in assembly code. Oh, and a negative diffstat.
>>
>> There is still a healthy dose of inline assembly (system register
>> accessors, runtime code patching), but I've tried not to make it too
>> invasive. The generated code, while not exactly brilliant, doesn't
>> look too shaby. I do expect a small performance degradation, but I
>> believe this is something we can improve over time (my initial
>> measurements don't show any obvious regression though).
>
> I ran this through my experimental setup on m400 and got this:
[...]
> What this tells me is that we do take a noticable hit on the
> world-switch path, which shows up in the TCP_RR and hackbench workloads,
> which have a high precision in their output.
>
> Note that the memcached number is well within its variability between
> individual benchmark runs, where it varies to 12% of its average in over
> 80% of the executions.
>
> I don't think this is a showstopper thought, but we could consider
> looking more closely at a breakdown of the world-switch path and verify
> if/where we are really taking a hit.
Thanks for doing so, very interesting. As a data point, what compiler
are you using? I'd expect some variability based on the compiler version...
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-11-30 19:50 ` Christoffer Dall
@ 2015-12-01 11:32 ` Marc Zyngier
2015-12-01 11:44 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 11:32 UTC (permalink / raw)
To: linux-arm-kernel
On 30/11/15 19:50, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
>> Implement the vgic-v3 save restore as a direct translation of
>> the assembly code version.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/hyp/Makefile | 1 +
>> arch/arm64/kvm/hyp/hyp.h | 3 +
>> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 226 insertions(+)
>> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
>>
>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>> index d8d5968..d1e38ce 100644
>> --- a/arch/arm64/kvm/hyp/Makefile
>> +++ b/arch/arm64/kvm/hyp/Makefile
>> @@ -3,3 +3,4 @@
>> #
>>
>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>> index 78f25c4..a31cb6e 100644
>> --- a/arch/arm64/kvm/hyp/hyp.h
>> +++ b/arch/arm64/kvm/hyp/hyp.h
>> @@ -30,5 +30,8 @@
>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>
>> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
>> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>> +
>> #endif /* __ARM64_KVM_HYP_H__ */
>>
>> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>> new file mode 100644
>> index 0000000..b490db5
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>> @@ -0,0 +1,222 @@
>> +/*
>> + * Copyright (C) 2012-2015 - ARM Ltd
>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <linux/compiler.h>
>> +#include <linux/irqchip/arm-gic-v3.h>
>> +#include <linux/kvm_host.h>
>> +
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "hyp.h"
>> +
>> +/*
>> + * We store LRs in reverse order to let the CPU deal with streaming
>> + * access. Use this macro to make it look saner...
>> + */
>> +#define LR_OFFSET(n) (15 - n)
>> +
>> +#define read_gicreg(r) \
>> + ({ \
>> + u64 reg; \
>> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
>> + reg; \
>> + })
>> +
>> +#define write_gicreg(v,r) \
>> + do { \
>> + u64 __val = (v); \
>> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
>> + } while (0)
>
> remind me what the msr_s and mrs_s do compared to msr and mrs?
They do the same job, only for the system registers which are not in the
original ARMv8 architecture spec, and most likely not implemented by
old(er) compilers.
> are these the reason why we need separate macros to access the gic
> registers compared to 'normal' sysregs?
Indeed.
>> +
>> +/* vcpu is already in the HYP VA space */
>> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>> +{
>> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
>> + u64 val;
>> + u32 nr_lr, nr_pri;
>> +
>> + /*
>> + * Make sure stores to the GIC via the memory mapped interface
>> + * are now visible to the system register interface.
>> + */
>> + dsb(st);
>> +
>> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
>> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
>> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
>> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
>> +
>> + write_gicreg(0, ICH_HCR_EL2);
>> + val = read_gicreg(ICH_VTR_EL2);
>> + nr_lr = val & 0xf;
>
> this is not technically nr_lr, it's max_lr or max_lr_idx or something
> like that.
Let's go for max_lr_idx then.
>> + nr_pri = ((u32)val >> 29) + 1;
>
> nit: nr_pri_bits
>
>> +
>> + switch (nr_lr) {
>> + case 15:
>> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
>> + case 14:
>> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
>> + case 13:
>> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
>> + case 12:
>> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
>> + case 11:
>> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
>> + case 10:
>> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
>> + case 9:
>> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
>> + case 8:
>> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
>> + case 7:
>> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
>> + case 6:
>> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
>> + case 5:
>> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
>> + case 4:
>> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
>> + case 3:
>> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
>> + case 2:
>> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
>> + case 1:
>> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
>> + case 0:
>> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
>
> I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
>
> cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
Just like in the assembly version. We store the LRs in the order we read
them so that we don't confuse the CPU by writing backward (believe it or
not, CPUs do get horribly confused if you do that).
>> + }
>> +
>> + switch (nr_pri) {
>> + case 7:
>> + cpu_if->vgic_ap0r[3] = read_gicreg(ICH_AP0R3_EL2);
>> + cpu_if->vgic_ap0r[2] = read_gicreg(ICH_AP0R2_EL2);
>> + case 6:
>> + cpu_if->vgic_ap0r[1] = read_gicreg(ICH_AP0R1_EL2);
>> + default:
>> + cpu_if->vgic_ap0r[0] = read_gicreg(ICH_AP0R0_EL2);
>> + }
>> +
>> + switch (nr_pri) {
>> + case 7:
>> + cpu_if->vgic_ap1r[3] = read_gicreg(ICH_AP1R3_EL2);
>> + cpu_if->vgic_ap1r[2] = read_gicreg(ICH_AP1R2_EL2);
>> + case 6:
>> + cpu_if->vgic_ap1r[1] = read_gicreg(ICH_AP1R1_EL2);
>> + default:
>> + cpu_if->vgic_ap1r[0] = read_gicreg(ICH_AP1R0_EL2);
>> + }
>> +
>> + write_gicreg(read_gicreg(ICC_SRE_EL2) | ICC_SRE_EL2_ENABLE,
>> + ICC_SRE_EL2);
>
> nit: reading this out in a variable probably looks nicer.
>
>> + isb();
>
> nit: should we comment on why the isb is required?
>
>> + write_gicreg(1, ICC_SRE_EL1);
>> +}
>> +
>> +void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
>> +{
>> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
>> + u64 val;
>> + u32 nr_lr, nr_pri;
>> +
>> + /* Make sure SRE is valid before writing the other registers */
>
> I know that I've reviewed this code before (the asm version) but coming
> back to it I have no idea what the above really means...
The meaning of some of the bits in ICH_VMCR_EL2 are conditioned by the
configuration set in ICC_SRE_EL1. For example, VFIQEn is RES1 if
ICC_SRE_EL1.SRE is 1. This causes a Group0 interrupt (as generated in
GICv2 mode) to be delivered as a FIQ to the guest, with potentially
fatal consequences.
So we must make sure that ICC_SRE_EL1 has been actually programmed with
the value we want before starting to mess with the rest of the GIC.
>> + write_gicreg(cpu_if->vgic_sre, ICC_SRE_EL1);
>> + isb();
>> +
>> + write_gicreg(cpu_if->vgic_hcr, ICH_HCR_EL2);
>> + write_gicreg(cpu_if->vgic_vmcr, ICH_VMCR_EL2);
>> +
>> + val = read_gicreg(ICH_VTR_EL2);
>> + nr_lr = val & 0xf;
>> + nr_pri = ((u32)val >> 29) + 1;
>
> you could have a define for this shift now that you use it more than
> once.
Sure.
>> +
>> + switch (nr_pri) {
>> + case 7:
>> + write_gicreg(cpu_if->vgic_ap1r[3], ICH_AP1R3_EL2);
>> + write_gicreg(cpu_if->vgic_ap1r[2], ICH_AP1R2_EL2);
>> + case 6:
>> + write_gicreg(cpu_if->vgic_ap1r[1], ICH_AP1R1_EL2);
>> + default:
>> + write_gicreg(cpu_if->vgic_ap1r[0], ICH_AP1R0_EL2);
>> + }
>> +
>> + switch (nr_pri) {
>> + case 7:
>> + write_gicreg(cpu_if->vgic_ap0r[3], ICH_AP0R3_EL2);
>> + write_gicreg(cpu_if->vgic_ap0r[2], ICH_AP0R2_EL2);
>> + case 6:
>> + write_gicreg(cpu_if->vgic_ap0r[1], ICH_AP0R1_EL2);
>> + default:
>> + write_gicreg(cpu_if->vgic_ap0r[0], ICH_AP0R0_EL2);
>> + }
>> +
>> + switch (nr_lr) {
>> + case 15:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(15)], ICH_LR15_EL2);
>> + case 14:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(14)], ICH_LR14_EL2);
>> + case 13:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(13)], ICH_LR13_EL2);
>> + case 12:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(12)], ICH_LR12_EL2);
>> + case 11:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(11)], ICH_LR11_EL2);
>> + case 10:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(10)], ICH_LR10_EL2);
>> + case 9:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(9)], ICH_LR9_EL2);
>> + case 8:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(8)], ICH_LR8_EL2);
>> + case 7:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(7)], ICH_LR7_EL2);
>> + case 6:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(6)], ICH_LR6_EL2);
>> + case 5:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(5)], ICH_LR5_EL2);
>> + case 4:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(4)], ICH_LR4_EL2);
>> + case 3:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(3)], ICH_LR3_EL2);
>> + case 2:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(2)], ICH_LR2_EL2);
>> + case 1:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(1)], ICH_LR1_EL2);
>> + case 0:
>> + write_gicreg(cpu_if->vgic_lr[LR_OFFSET(0)], ICH_LR0_EL2);
>
> same question as above.
>
>> + }
>
> loads of trailing whitespace all over above
>
>> +
>> + /*
>> + * Ensure that the above will have reached the
>> + * (re)distributors. This ensure the guest will read the
>
> s/ensure/ensures/
>
>> + * correct values from the memory-mapped interface.
>> + */
>> + isb();
>> + dsb(sy);
>> +
>> + /*
>> + * Prevent the guest from touching the GIC system registers if
>> + * SRE isn't enabled for GICv3 emulation.
>
> so can we emulate a GICv3 to the guest without system register access?
> I.e. an MMIO only GICv3 interface?
There is no such thing as a MMIO-based GICv3 CPU interface. The only
purpose of this is to ensure that a guest presented with a GICv2 doesn't
see the system registers as being available, because that both confusing
and borderline illegal.
>> + */
>> + if (!cpu_if->vgic_sre) {
>> + write_gicreg(read_gicreg(ICC_SRE_EL2) & ~ICC_SRE_EL2_ENABLE,
>> + ICC_SRE_EL2);
>> + }
>> +}
>> +
>> +u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
>> +{
>> + return read_gicreg(ICH_VTR_EL2);
>> +}
>> --
>> 2.1.4
>>
>
> As for translating the assembly code to C, this patch looks correct.
>
> Nevertheless, I felt obliged to ask into the details above, now when
> you're touching all this code.
No problem. Thanks for reviewing it.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 05/21] arm64: KVM: Implement timer save/restore
2015-11-30 19:59 ` Christoffer Dall
@ 2015-12-01 11:34 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 11:34 UTC (permalink / raw)
To: linux-arm-kernel
On 30/11/15 19:59, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:49:59PM +0000, Marc Zyngier wrote:
>> Implement the timer save restore as a direct translation of
>> the assembly code version.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/hyp/Makefile | 1 +
>> arch/arm64/kvm/hyp/hyp.h | 3 ++
>> arch/arm64/kvm/hyp/timer-sr.c | 71 ++++++++++++++++++++++++++++++++++++
>> include/clocksource/arm_arch_timer.h | 6 +++
>> 4 files changed, 81 insertions(+)
>> create mode 100644 arch/arm64/kvm/hyp/timer-sr.c
>>
>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>> index d1e38ce..455dc0a 100644
>> --- a/arch/arm64/kvm/hyp/Makefile
>> +++ b/arch/arm64/kvm/hyp/Makefile
>> @@ -4,3 +4,4 @@
>>
>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>> index a31cb6e..86aa5a2 100644
>> --- a/arch/arm64/kvm/hyp/hyp.h
>> +++ b/arch/arm64/kvm/hyp/hyp.h
>> @@ -33,5 +33,8 @@ void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>> void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
>> void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>>
>> +void __timer_save_state(struct kvm_vcpu *vcpu);
>> +void __timer_restore_state(struct kvm_vcpu *vcpu);
>> +
>> #endif /* __ARM64_KVM_HYP_H__ */
>>
>> diff --git a/arch/arm64/kvm/hyp/timer-sr.c b/arch/arm64/kvm/hyp/timer-sr.c
>> new file mode 100644
>> index 0000000..8e2209c
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/timer-sr.c
>> @@ -0,0 +1,71 @@
>> +/*
>> + * Copyright (C) 2012-2015 - ARM Ltd
>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <clocksource/arm_arch_timer.h>
>> +#include <linux/compiler.h>
>> +#include <linux/kvm_host.h>
>> +
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "hyp.h"
>> +
>> +/* vcpu is already in the HYP VA space */
>> +void __hyp_text __timer_save_state(struct kvm_vcpu *vcpu)
>> +{
>> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
>> + struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
>> +
>> + if (kvm->arch.timer.enabled) {
>> + timer->cntv_ctl = read_sysreg(cntv_ctl_el0);
>> + isb();
>> + timer->cntv_cval = read_sysreg(cntv_cval_el0);
>> + }
>> +
>> + /* Disable the virtual timer */
>> + write_sysreg(0, cntv_ctl_el0);
>> +
>> + /* Allow physical timer/counter access for the host */
>> + write_sysreg((read_sysreg(cnthctl_el2) | CNTHCTL_EL1PCTEN |
>> + CNTHCTL_EL1PCEN),
>> + cnthctl_el2);
>
> nit: again I probably prefer reading cnthctl_el2 into a variable, modify
> the bits and write it back, but it's no big deal.
Sure.
>> +
>> + /* Clear cntvoff for the host */
>> + write_sysreg(0, cntvoff_el2);
>
> why do we do this when we've just disabled the timer?
Because the host does use CNTVCT_EL0 (see the VDSO code), and you don't
want time to go backward over there...
>> +}
>> +
>> +void __hyp_text __timer_restore_state(struct kvm_vcpu *vcpu)
>> +{
>> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
>> + struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
>> + u64 val;
>> +
>> + /*
>> + * Disallow physical timer access for the guest
>> + * Physical counter access is allowed
>> + */
>> + val = read_sysreg(cnthctl_el2);
>> + val &= ~CNTHCTL_EL1PCEN;
>> + val |= CNTHCTL_EL1PCTEN;
>> + write_sysreg(val, cnthctl_el2);
>> +
>> + if (kvm->arch.timer.enabled) {
>> + write_sysreg(kvm->arch.timer.cntvoff, cntvoff_el2);
>> + write_sysreg(timer->cntv_cval, cntv_cval_el0);
>> + isb();
>> + write_sysreg(timer->cntv_ctl, cntv_ctl_el0);
>> + }
>> +}
>> diff --git a/include/clocksource/arm_arch_timer.h b/include/clocksource/arm_arch_timer.h
>> index 9916d0e..25d0914 100644
>> --- a/include/clocksource/arm_arch_timer.h
>> +++ b/include/clocksource/arm_arch_timer.h
>> @@ -23,6 +23,12 @@
>> #define ARCH_TIMER_CTRL_IT_MASK (1 << 1)
>> #define ARCH_TIMER_CTRL_IT_STAT (1 << 2)
>>
>> +#define CNTHCTL_EL1PCTEN (1 << 0)
>> +#define CNTHCTL_EL1PCEN (1 << 1)
>> +#define CNTHCTL_EVNTEN (1 << 2)
>> +#define CNTHCTL_EVNTDIR (1 << 3)
>> +#define CNTHCTL_EVNTI (0xF << 4)
>> +
>> enum arch_timer_reg {
>> ARCH_TIMER_REG_CTRL,
>> ARCH_TIMER_REG_TVAL,
>> --
>> 2.1.4
>>
>
> Otherwise this looks good.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore
2015-11-30 20:00 ` Christoffer Dall
@ 2015-12-01 11:39 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 11:39 UTC (permalink / raw)
To: linux-arm-kernel
On 30/11/15 20:00, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:49:57PM +0000, Marc Zyngier wrote:
>> Implement the vgic-v2 save restore (mostly) as a direct translation
>> of the assembly code version.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/Makefile | 1 +
>> arch/arm64/kvm/hyp/Makefile | 5 +++
>> arch/arm64/kvm/hyp/hyp.h | 3 ++
>> arch/arm64/kvm/hyp/vgic-v2-sr.c | 89 +++++++++++++++++++++++++++++++++++++++++
>> 4 files changed, 98 insertions(+)
>> create mode 100644 arch/arm64/kvm/hyp/Makefile
>> create mode 100644 arch/arm64/kvm/hyp/vgic-v2-sr.c
>>
>> diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
>> index 1949fe5..d31e4e5 100644
>> --- a/arch/arm64/kvm/Makefile
>> +++ b/arch/arm64/kvm/Makefile
>> @@ -10,6 +10,7 @@ KVM=../../../virt/kvm
>> ARM=../../../arch/arm/kvm
>>
>> obj-$(CONFIG_KVM_ARM_HOST) += kvm.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += hyp/
>>
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o $(KVM)/vfio.o
>> kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/arm.o $(ARM)/mmu.o $(ARM)/mmio.o
>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>> new file mode 100644
>> index 0000000..d8d5968
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/Makefile
>> @@ -0,0 +1,5 @@
>> +#
>> +# Makefile for Kernel-based Virtual Machine module, HYP part
>> +#
>> +
>> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>> index dac843e..78f25c4 100644
>> --- a/arch/arm64/kvm/hyp/hyp.h
>> +++ b/arch/arm64/kvm/hyp/hyp.h
>> @@ -27,5 +27,8 @@
>>
>> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
>>
>> +void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>> +void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>
> should we call these flush/sync here now ?
>
>> +
>> #endif /* __ARM64_KVM_HYP_H__ */
>>
>> diff --git a/arch/arm64/kvm/hyp/vgic-v2-sr.c b/arch/arm64/kvm/hyp/vgic-v2-sr.c
>> new file mode 100644
>> index 0000000..29a5c1d
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/vgic-v2-sr.c
>> @@ -0,0 +1,89 @@
>> +/*
>> + * Copyright (C) 2012-2015 - ARM Ltd
>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <linux/compiler.h>
>> +#include <linux/irqchip/arm-gic.h>
>> +#include <linux/kvm_host.h>
>> +
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "hyp.h"
>> +
>> +/* vcpu is already in the HYP VA space */
>
> should we annotate hyp pointers similarly to __user or will that be
> confusing when VHE enters the scene ?
I looked at doing that. That's a possibility, and I don't think that
would be too bad as long as we have kern_hyp_va() doing the (potentially
NOP) conversion. The only issue is that this is only enforced with
sparse, not by a usual compilation.
Still, this is a valid use case, and I'll try to invest some time doing
that.
>
>> +void __hyp_text __vgic_v2_save_state(struct kvm_vcpu *vcpu)
>> +{
>> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
>> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
>> + struct vgic_dist *vgic = &kvm->arch.vgic;
>> + void __iomem *base = kern_hyp_va(vgic->vctrl_base);
>> + u32 __iomem *lr_base;
>> + u32 eisr0, eisr1, elrsr0, elrsr1;
>> + int i = 0, nr_lr;
>> +
>> + if (!base)
>> + return;
>> +
>> + nr_lr = vcpu->arch.vgic_cpu.nr_lr;
>> + cpu_if->vgic_vmcr = readl_relaxed(base + GICH_VMCR);
>> + cpu_if->vgic_misr = readl_relaxed(base + GICH_MISR);
>> + eisr0 = readl_relaxed(base + GICH_EISR0);
>> + elrsr0 = readl_relaxed(base + GICH_ELRSR0);
>> + if (unlikely(nr_lr > 32)) {
>> + eisr1 = readl_relaxed(base + GICH_EISR1);
>> + elrsr1 = readl_relaxed(base + GICH_ELRSR1);
>> + } else {
>> + eisr1 = elrsr1 = 0;
>> + }
>> +#ifdef CONFIG_CPU_BIG_ENDIAN
>> + cpu_if->vgic_eisr = ((u64)eisr0 << 32) | eisr1;
>> + cpu_if->vgic_elrsr = ((u64)elrsr0 << 32) | elrsr1;
>> +#else
>> + cpu_if->vgic_eisr = ((u64)eisr1 << 32) | eisr0;
>> + cpu_if->vgic_elrsr = ((u64)elrsr1 << 32) | elrsr0;
>> +#endif
>> + cpu_if->vgic_apr = readl_relaxed(base + GICH_APR);
>> +
>> + writel_relaxed(0, base + GICH_HCR);
>> +
>> + lr_base = base + GICH_LR0;
>> + do {
>> + cpu_if->vgic_lr[i++] = readl_relaxed(lr_base++);
>> + } while (--nr_lr);
>
> why not a simple for-loop?
Good question. I blame coding from 30000 feet.
>> +}
>> +
>
> copy the vcpu HYP VA comment down here.
>
>> +void __hyp_text __vgic_v2_restore_state(struct kvm_vcpu *vcpu)
>> +{
>> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
>> + struct vgic_v2_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v2;
>> + struct vgic_dist *vgic = &kvm->arch.vgic;
>> + void __iomem *base = kern_hyp_va(vgic->vctrl_base);
>> + u32 __iomem *lr_base;
>> + unsigned int i = 0, nr_lr;
>> +
>> + if (!base)
>> + return;
>> +
>> + writel_relaxed(cpu_if->vgic_hcr, base + GICH_HCR);
>> + writel_relaxed(cpu_if->vgic_vmcr, base + GICH_VMCR);
>> + writel_relaxed(cpu_if->vgic_apr, base + GICH_APR);
>> +
>> + lr_base = base + GICH_LR0;
>> + nr_lr = vcpu->arch.vgic_cpu.nr_lr;
>> + do {
>> + writel_relaxed(cpu_if->vgic_lr[i++], lr_base++);
>> + } while (--nr_lr);
>
> same question as above.
>
>> +}
>> --
>> 2.1.4
>>
> Otherwise looks good.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file
2015-11-30 20:00 ` Christoffer Dall
@ 2015-12-01 11:41 ` Marc Zyngier
2015-12-01 11:47 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 11:41 UTC (permalink / raw)
To: linux-arm-kernel
On 30/11/15 20:00, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:49:56PM +0000, Marc Zyngier wrote:
>> In order to expose the various EL2 services that are private to
>> the hypervisor, add a new hyp.h file.
>>
>> So far, it only contains mundane things such as section annotation
>> and VA manipulation.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/hyp/hyp.h | 31 +++++++++++++++++++++++++++++++
>> 1 file changed, 31 insertions(+)
>> create mode 100644 arch/arm64/kvm/hyp/hyp.h
>>
>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>> new file mode 100644
>> index 0000000..dac843e
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/hyp.h
>> @@ -0,0 +1,31 @@
>> +/*
>> + * Copyright (C) 2015 - ARM Ltd
>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#ifndef __ARM64_KVM_HYP_H__
>> +#define __ARM64_KVM_HYP_H__
>> +
>> +#include <linux/compiler.h>
>> +#include <linux/kvm_host.h>
>> +#include <asm/kvm_mmu.h>
>> +#include <asm/sysreg.h>
>> +
>> +#define __hyp_text __section(.hyp.text) notrace
>
> why notrace?
Because you'd end with calls to mcount in each function prologue, and
that doesn't really well for stuff that is not executed in the kernel
address space.
>> +
>> +#define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
>
> should you have parenthesis around 'v' ?
Yup.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-12-01 11:32 ` Marc Zyngier
@ 2015-12-01 11:44 ` Christoffer Dall
2015-12-01 11:50 ` Christoffer Dall
2015-12-01 11:54 ` Marc Zyngier
0 siblings, 2 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 11:44 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 11:32:20AM +0000, Marc Zyngier wrote:
> On 30/11/15 19:50, Christoffer Dall wrote:
> > On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
> >> Implement the vgic-v3 save restore as a direct translation of
> >> the assembly code version.
> >>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >> arch/arm64/kvm/hyp/Makefile | 1 +
> >> arch/arm64/kvm/hyp/hyp.h | 3 +
> >> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
> >> 3 files changed, 226 insertions(+)
> >> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
> >>
> >> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> >> index d8d5968..d1e38ce 100644
> >> --- a/arch/arm64/kvm/hyp/Makefile
> >> +++ b/arch/arm64/kvm/hyp/Makefile
> >> @@ -3,3 +3,4 @@
> >> #
> >>
> >> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> >> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> >> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> >> index 78f25c4..a31cb6e 100644
> >> --- a/arch/arm64/kvm/hyp/hyp.h
> >> +++ b/arch/arm64/kvm/hyp/hyp.h
> >> @@ -30,5 +30,8 @@
> >> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> >> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >>
> >> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> >> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> >> +
> >> #endif /* __ARM64_KVM_HYP_H__ */
> >>
> >> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> >> new file mode 100644
> >> index 0000000..b490db5
> >> --- /dev/null
> >> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> >> @@ -0,0 +1,222 @@
> >> +/*
> >> + * Copyright (C) 2012-2015 - ARM Ltd
> >> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> >> + *
> >> + * This program is free software; you can redistribute it and/or modify
> >> + * it under the terms of the GNU General Public License version 2 as
> >> + * published by the Free Software Foundation.
> >> + *
> >> + * This program is distributed in the hope that it will be useful,
> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> + * GNU General Public License for more details.
> >> + *
> >> + * You should have received a copy of the GNU General Public License
> >> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> >> + */
> >> +
> >> +#include <linux/compiler.h>
> >> +#include <linux/irqchip/arm-gic-v3.h>
> >> +#include <linux/kvm_host.h>
> >> +
> >> +#include <asm/kvm_mmu.h>
> >> +
> >> +#include "hyp.h"
> >> +
> >> +/*
> >> + * We store LRs in reverse order to let the CPU deal with streaming
> >> + * access. Use this macro to make it look saner...
> >> + */
> >> +#define LR_OFFSET(n) (15 - n)
> >> +
> >> +#define read_gicreg(r) \
> >> + ({ \
> >> + u64 reg; \
> >> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
> >> + reg; \
> >> + })
> >> +
> >> +#define write_gicreg(v,r) \
> >> + do { \
> >> + u64 __val = (v); \
> >> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
> >> + } while (0)
> >
> > remind me what the msr_s and mrs_s do compared to msr and mrs?
>
> They do the same job, only for the system registers which are not in the
> original ARMv8 architecture spec, and most likely not implemented by
> old(er) compilers.
>
> > are these the reason why we need separate macros to access the gic
> > registers compared to 'normal' sysregs?
>
> Indeed.
>
> >> +
> >> +/* vcpu is already in the HYP VA space */
> >> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> >> +{
> >> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> >> + u64 val;
> >> + u32 nr_lr, nr_pri;
> >> +
> >> + /*
> >> + * Make sure stores to the GIC via the memory mapped interface
> >> + * are now visible to the system register interface.
> >> + */
> >> + dsb(st);
> >> +
> >> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> >> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
> >> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
> >> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> >> +
> >> + write_gicreg(0, ICH_HCR_EL2);
> >> + val = read_gicreg(ICH_VTR_EL2);
> >> + nr_lr = val & 0xf;
> >
> > this is not technically nr_lr, it's max_lr or max_lr_idx or something
> > like that.
>
> Let's go for max_lr_idx then.
>
> >> + nr_pri = ((u32)val >> 29) + 1;
> >
> > nit: nr_pri_bits
> >
> >> +
> >> + switch (nr_lr) {
> >> + case 15:
> >> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
> >> + case 14:
> >> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
> >> + case 13:
> >> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
> >> + case 12:
> >> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
> >> + case 11:
> >> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
> >> + case 10:
> >> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
> >> + case 9:
> >> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
> >> + case 8:
> >> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
> >> + case 7:
> >> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
> >> + case 6:
> >> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
> >> + case 5:
> >> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
> >> + case 4:
> >> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
> >> + case 3:
> >> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
> >> + case 2:
> >> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
> >> + case 1:
> >> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
> >> + case 0:
> >> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
> >
> > I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
> >
> > cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
>
> Just like in the assembly version. We store the LRs in the order we read
> them so that we don't confuse the CPU by writing backward (believe it or
> not, CPUs do get horribly confused if you do that).
but aren't we storing the wrong register to the wrong index in the
array?
Do we really access cpu_if->vgic_lr[15..12] in the C-code if the system
only has 4 LRs?
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file
2015-12-01 11:41 ` Marc Zyngier
@ 2015-12-01 11:47 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 11:47 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 11:41:08AM +0000, Marc Zyngier wrote:
> On 30/11/15 20:00, Christoffer Dall wrote:
> > On Fri, Nov 27, 2015 at 06:49:56PM +0000, Marc Zyngier wrote:
> >> In order to expose the various EL2 services that are private to
> >> the hypervisor, add a new hyp.h file.
> >>
> >> So far, it only contains mundane things such as section annotation
> >> and VA manipulation.
> >>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >> arch/arm64/kvm/hyp/hyp.h | 31 +++++++++++++++++++++++++++++++
> >> 1 file changed, 31 insertions(+)
> >> create mode 100644 arch/arm64/kvm/hyp/hyp.h
> >>
> >> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> >> new file mode 100644
> >> index 0000000..dac843e
> >> --- /dev/null
> >> +++ b/arch/arm64/kvm/hyp/hyp.h
> >> @@ -0,0 +1,31 @@
> >> +/*
> >> + * Copyright (C) 2015 - ARM Ltd
> >> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> >> + *
> >> + * This program is free software; you can redistribute it and/or modify
> >> + * it under the terms of the GNU General Public License version 2 as
> >> + * published by the Free Software Foundation.
> >> + *
> >> + * This program is distributed in the hope that it will be useful,
> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> + * GNU General Public License for more details.
> >> + *
> >> + * You should have received a copy of the GNU General Public License
> >> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> >> + */
> >> +
> >> +#ifndef __ARM64_KVM_HYP_H__
> >> +#define __ARM64_KVM_HYP_H__
> >> +
> >> +#include <linux/compiler.h>
> >> +#include <linux/kvm_host.h>
> >> +#include <asm/kvm_mmu.h>
> >> +#include <asm/sysreg.h>
> >> +
> >> +#define __hyp_text __section(.hyp.text) notrace
> >
> > why notrace?
>
> Because you'd end with calls to mcount in each function prologue, and
> that doesn't really well for stuff that is not executed in the kernel
> address space.
right, makes good sense.
>
> >> +
> >> +#define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
> >
> > should you have parenthesis around 'v' ?
>
> Yup.
>
> Thanks,
>
> M.
> --
> Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-12-01 11:44 ` Christoffer Dall
@ 2015-12-01 11:50 ` Christoffer Dall
2015-12-01 11:57 ` Marc Zyngier
2015-12-01 11:54 ` Marc Zyngier
1 sibling, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 11:50 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 12:44:26PM +0100, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 11:32:20AM +0000, Marc Zyngier wrote:
> > On 30/11/15 19:50, Christoffer Dall wrote:
> > > On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
> > >> Implement the vgic-v3 save restore as a direct translation of
> > >> the assembly code version.
> > >>
> > >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> > >> ---
> > >> arch/arm64/kvm/hyp/Makefile | 1 +
> > >> arch/arm64/kvm/hyp/hyp.h | 3 +
> > >> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
> > >> 3 files changed, 226 insertions(+)
> > >> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
> > >>
> > >> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> > >> index d8d5968..d1e38ce 100644
> > >> --- a/arch/arm64/kvm/hyp/Makefile
> > >> +++ b/arch/arm64/kvm/hyp/Makefile
> > >> @@ -3,3 +3,4 @@
> > >> #
> > >>
> > >> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> > >> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> > >> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> > >> index 78f25c4..a31cb6e 100644
> > >> --- a/arch/arm64/kvm/hyp/hyp.h
> > >> +++ b/arch/arm64/kvm/hyp/hyp.h
> > >> @@ -30,5 +30,8 @@
> > >> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> > >> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> > >>
> > >> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> > >> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> > >> +
> > >> #endif /* __ARM64_KVM_HYP_H__ */
> > >>
> > >> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> > >> new file mode 100644
> > >> index 0000000..b490db5
> > >> --- /dev/null
> > >> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> > >> @@ -0,0 +1,222 @@
> > >> +/*
> > >> + * Copyright (C) 2012-2015 - ARM Ltd
> > >> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> > >> + *
> > >> + * This program is free software; you can redistribute it and/or modify
> > >> + * it under the terms of the GNU General Public License version 2 as
> > >> + * published by the Free Software Foundation.
> > >> + *
> > >> + * This program is distributed in the hope that it will be useful,
> > >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > >> + * GNU General Public License for more details.
> > >> + *
> > >> + * You should have received a copy of the GNU General Public License
> > >> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> > >> + */
> > >> +
> > >> +#include <linux/compiler.h>
> > >> +#include <linux/irqchip/arm-gic-v3.h>
> > >> +#include <linux/kvm_host.h>
> > >> +
> > >> +#include <asm/kvm_mmu.h>
> > >> +
> > >> +#include "hyp.h"
> > >> +
> > >> +/*
> > >> + * We store LRs in reverse order to let the CPU deal with streaming
> > >> + * access. Use this macro to make it look saner...
> > >> + */
> > >> +#define LR_OFFSET(n) (15 - n)
> > >> +
> > >> +#define read_gicreg(r) \
> > >> + ({ \
> > >> + u64 reg; \
> > >> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
> > >> + reg; \
> > >> + })
> > >> +
> > >> +#define write_gicreg(v,r) \
> > >> + do { \
> > >> + u64 __val = (v); \
> > >> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
> > >> + } while (0)
> > >
> > > remind me what the msr_s and mrs_s do compared to msr and mrs?
> >
> > They do the same job, only for the system registers which are not in the
> > original ARMv8 architecture spec, and most likely not implemented by
> > old(er) compilers.
> >
> > > are these the reason why we need separate macros to access the gic
> > > registers compared to 'normal' sysregs?
> >
> > Indeed.
> >
> > >> +
> > >> +/* vcpu is already in the HYP VA space */
> > >> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> > >> +{
> > >> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> > >> + u64 val;
> > >> + u32 nr_lr, nr_pri;
> > >> +
> > >> + /*
> > >> + * Make sure stores to the GIC via the memory mapped interface
> > >> + * are now visible to the system register interface.
> > >> + */
> > >> + dsb(st);
> > >> +
> > >> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> > >> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
> > >> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
> > >> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> > >> +
> > >> + write_gicreg(0, ICH_HCR_EL2);
> > >> + val = read_gicreg(ICH_VTR_EL2);
> > >> + nr_lr = val & 0xf;
> > >
> > > this is not technically nr_lr, it's max_lr or max_lr_idx or something
> > > like that.
> >
> > Let's go for max_lr_idx then.
> >
> > >> + nr_pri = ((u32)val >> 29) + 1;
> > >
> > > nit: nr_pri_bits
> > >
> > >> +
> > >> + switch (nr_lr) {
> > >> + case 15:
> > >> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
> > >> + case 14:
> > >> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
> > >> + case 13:
> > >> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
> > >> + case 12:
> > >> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
> > >> + case 11:
> > >> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
> > >> + case 10:
> > >> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
> > >> + case 9:
> > >> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
> > >> + case 8:
> > >> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
> > >> + case 7:
> > >> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
> > >> + case 6:
> > >> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
> > >> + case 5:
> > >> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
> > >> + case 4:
> > >> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
> > >> + case 3:
> > >> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
> > >> + case 2:
> > >> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
> > >> + case 1:
> > >> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
> > >> + case 0:
> > >> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
> > >
> > > I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
> > >
> > > cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
> >
> > Just like in the assembly version. We store the LRs in the order we read
> > them so that we don't confuse the CPU by writing backward (believe it or
> > not, CPUs do get horribly confused if you do that).
>
> but aren't we storing the wrong register to the wrong index in the
> array?
>
> Do we really access cpu_if->vgic_lr[15..12] in the C-code if the system
> only has 4 LRs?
>
ok, I looked at the code myself (not sure why I didn't do that in the
first place) and indeed we use a different but with similar results
macro to access the array from the C code.
This is just insane to me, and we don't have a comment on the data
structure saying "this is not stored the way you'd think it is".
Why can't we just do:
cpu_if->vgic_lr[3] = read_gicreg(ICH_LR3_EL2);
cpu_if->vgic_lr[2] = read_gicreg(ICH_LR2_EL2);
cpu_if->vgic_lr[1] = read_gicreg(ICH_LR1_EL2);
cpu_if->vgic_lr[0] = read_gicreg(ICH_LR0_EL2);
?
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-12-01 11:44 ` Christoffer Dall
2015-12-01 11:50 ` Christoffer Dall
@ 2015-12-01 11:54 ` Marc Zyngier
1 sibling, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 11:54 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 11:44, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 11:32:20AM +0000, Marc Zyngier wrote:
>> On 30/11/15 19:50, Christoffer Dall wrote:
>>> On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
>>>> Implement the vgic-v3 save restore as a direct translation of
>>>> the assembly code version.
>>>>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>> arch/arm64/kvm/hyp/Makefile | 1 +
>>>> arch/arm64/kvm/hyp/hyp.h | 3 +
>>>> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
>>>> 3 files changed, 226 insertions(+)
>>>> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>>
>>>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>>>> index d8d5968..d1e38ce 100644
>>>> --- a/arch/arm64/kvm/hyp/Makefile
>>>> +++ b/arch/arm64/kvm/hyp/Makefile
>>>> @@ -3,3 +3,4 @@
>>>> #
>>>>
>>>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>>>> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>>>> index 78f25c4..a31cb6e 100644
>>>> --- a/arch/arm64/kvm/hyp/hyp.h
>>>> +++ b/arch/arm64/kvm/hyp/hyp.h
>>>> @@ -30,5 +30,8 @@
>>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>>>
>>>> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
>>>> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>>>> +
>>>> #endif /* __ARM64_KVM_HYP_H__ */
>>>>
>>>> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>> new file mode 100644
>>>> index 0000000..b490db5
>>>> --- /dev/null
>>>> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>> @@ -0,0 +1,222 @@
>>>> +/*
>>>> + * Copyright (C) 2012-2015 - ARM Ltd
>>>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>>>> + *
>>>> + * This program is free software; you can redistribute it and/or modify
>>>> + * it under the terms of the GNU General Public License version 2 as
>>>> + * published by the Free Software Foundation.
>>>> + *
>>>> + * This program is distributed in the hope that it will be useful,
>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>> + * GNU General Public License for more details.
>>>> + *
>>>> + * You should have received a copy of the GNU General Public License
>>>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>> + */
>>>> +
>>>> +#include <linux/compiler.h>
>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>> +#include <linux/kvm_host.h>
>>>> +
>>>> +#include <asm/kvm_mmu.h>
>>>> +
>>>> +#include "hyp.h"
>>>> +
>>>> +/*
>>>> + * We store LRs in reverse order to let the CPU deal with streaming
>>>> + * access. Use this macro to make it look saner...
>>>> + */
>>>> +#define LR_OFFSET(n) (15 - n)
>>>> +
>>>> +#define read_gicreg(r) \
>>>> + ({ \
>>>> + u64 reg; \
>>>> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
>>>> + reg; \
>>>> + })
>>>> +
>>>> +#define write_gicreg(v,r) \
>>>> + do { \
>>>> + u64 __val = (v); \
>>>> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
>>>> + } while (0)
>>>
>>> remind me what the msr_s and mrs_s do compared to msr and mrs?
>>
>> They do the same job, only for the system registers which are not in the
>> original ARMv8 architecture spec, and most likely not implemented by
>> old(er) compilers.
>>
>>> are these the reason why we need separate macros to access the gic
>>> registers compared to 'normal' sysregs?
>>
>> Indeed.
>>
>>>> +
>>>> +/* vcpu is already in the HYP VA space */
>>>> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>>>> +{
>>>> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
>>>> + u64 val;
>>>> + u32 nr_lr, nr_pri;
>>>> +
>>>> + /*
>>>> + * Make sure stores to the GIC via the memory mapped interface
>>>> + * are now visible to the system register interface.
>>>> + */
>>>> + dsb(st);
>>>> +
>>>> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
>>>> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
>>>> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
>>>> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
>>>> +
>>>> + write_gicreg(0, ICH_HCR_EL2);
>>>> + val = read_gicreg(ICH_VTR_EL2);
>>>> + nr_lr = val & 0xf;
>>>
>>> this is not technically nr_lr, it's max_lr or max_lr_idx or something
>>> like that.
>>
>> Let's go for max_lr_idx then.
>>
>>>> + nr_pri = ((u32)val >> 29) + 1;
>>>
>>> nit: nr_pri_bits
>>>
>>>> +
>>>> + switch (nr_lr) {
>>>> + case 15:
>>>> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
>>>> + case 14:
>>>> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
>>>> + case 13:
>>>> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
>>>> + case 12:
>>>> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
>>>> + case 11:
>>>> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
>>>> + case 10:
>>>> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
>>>> + case 9:
>>>> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
>>>> + case 8:
>>>> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
>>>> + case 7:
>>>> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
>>>> + case 6:
>>>> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
>>>> + case 5:
>>>> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
>>>> + case 4:
>>>> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
>>>> + case 3:
>>>> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
>>>> + case 2:
>>>> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
>>>> + case 1:
>>>> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
>>>> + case 0:
>>>> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
>>>
>>> I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
>>>
>>> cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
>>
>> Just like in the assembly version. We store the LRs in the order we read
>> them so that we don't confuse the CPU by writing backward (believe it or
>> not, CPUs do get horribly confused if you do that).
>
> but aren't we storing the wrong register to the wrong index in the
> array?
I don't think so. If we did, GICv3 support wouldn't get off the ground
at all...
> Do we really access cpu_if->vgic_lr[15..12] in the C-code if the system
> only has 4 LRs?
Yes we do. See how the vgic_v3_{get,set}_lr functions are using the
exact same macro to get to the right LR.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-12-01 11:50 ` Christoffer Dall
@ 2015-12-01 11:57 ` Marc Zyngier
2015-12-01 12:24 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 11:57 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 11:50, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 12:44:26PM +0100, Christoffer Dall wrote:
>> On Tue, Dec 01, 2015 at 11:32:20AM +0000, Marc Zyngier wrote:
>>> On 30/11/15 19:50, Christoffer Dall wrote:
>>>> On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
>>>>> Implement the vgic-v3 save restore as a direct translation of
>>>>> the assembly code version.
>>>>>
>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>>> ---
>>>>> arch/arm64/kvm/hyp/Makefile | 1 +
>>>>> arch/arm64/kvm/hyp/hyp.h | 3 +
>>>>> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
>>>>> 3 files changed, 226 insertions(+)
>>>>> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>>>
>>>>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>>>>> index d8d5968..d1e38ce 100644
>>>>> --- a/arch/arm64/kvm/hyp/Makefile
>>>>> +++ b/arch/arm64/kvm/hyp/Makefile
>>>>> @@ -3,3 +3,4 @@
>>>>> #
>>>>>
>>>>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>>>>> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>>>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>>>>> index 78f25c4..a31cb6e 100644
>>>>> --- a/arch/arm64/kvm/hyp/hyp.h
>>>>> +++ b/arch/arm64/kvm/hyp/hyp.h
>>>>> @@ -30,5 +30,8 @@
>>>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>>>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>>>>
>>>>> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
>>>>> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>>>>> +
>>>>> #endif /* __ARM64_KVM_HYP_H__ */
>>>>>
>>>>> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>>> new file mode 100644
>>>>> index 0000000..b490db5
>>>>> --- /dev/null
>>>>> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>>> @@ -0,0 +1,222 @@
>>>>> +/*
>>>>> + * Copyright (C) 2012-2015 - ARM Ltd
>>>>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>>>>> + *
>>>>> + * This program is free software; you can redistribute it and/or modify
>>>>> + * it under the terms of the GNU General Public License version 2 as
>>>>> + * published by the Free Software Foundation.
>>>>> + *
>>>>> + * This program is distributed in the hope that it will be useful,
>>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>> + * GNU General Public License for more details.
>>>>> + *
>>>>> + * You should have received a copy of the GNU General Public License
>>>>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>> + */
>>>>> +
>>>>> +#include <linux/compiler.h>
>>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>> +#include <linux/kvm_host.h>
>>>>> +
>>>>> +#include <asm/kvm_mmu.h>
>>>>> +
>>>>> +#include "hyp.h"
>>>>> +
>>>>> +/*
>>>>> + * We store LRs in reverse order to let the CPU deal with streaming
>>>>> + * access. Use this macro to make it look saner...
>>>>> + */
>>>>> +#define LR_OFFSET(n) (15 - n)
>>>>> +
>>>>> +#define read_gicreg(r) \
>>>>> + ({ \
>>>>> + u64 reg; \
>>>>> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
>>>>> + reg; \
>>>>> + })
>>>>> +
>>>>> +#define write_gicreg(v,r) \
>>>>> + do { \
>>>>> + u64 __val = (v); \
>>>>> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
>>>>> + } while (0)
>>>>
>>>> remind me what the msr_s and mrs_s do compared to msr and mrs?
>>>
>>> They do the same job, only for the system registers which are not in the
>>> original ARMv8 architecture spec, and most likely not implemented by
>>> old(er) compilers.
>>>
>>>> are these the reason why we need separate macros to access the gic
>>>> registers compared to 'normal' sysregs?
>>>
>>> Indeed.
>>>
>>>>> +
>>>>> +/* vcpu is already in the HYP VA space */
>>>>> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>>>>> +{
>>>>> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
>>>>> + u64 val;
>>>>> + u32 nr_lr, nr_pri;
>>>>> +
>>>>> + /*
>>>>> + * Make sure stores to the GIC via the memory mapped interface
>>>>> + * are now visible to the system register interface.
>>>>> + */
>>>>> + dsb(st);
>>>>> +
>>>>> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
>>>>> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
>>>>> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
>>>>> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
>>>>> +
>>>>> + write_gicreg(0, ICH_HCR_EL2);
>>>>> + val = read_gicreg(ICH_VTR_EL2);
>>>>> + nr_lr = val & 0xf;
>>>>
>>>> this is not technically nr_lr, it's max_lr or max_lr_idx or something
>>>> like that.
>>>
>>> Let's go for max_lr_idx then.
>>>
>>>>> + nr_pri = ((u32)val >> 29) + 1;
>>>>
>>>> nit: nr_pri_bits
>>>>
>>>>> +
>>>>> + switch (nr_lr) {
>>>>> + case 15:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
>>>>> + case 14:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
>>>>> + case 13:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
>>>>> + case 12:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
>>>>> + case 11:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
>>>>> + case 10:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
>>>>> + case 9:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
>>>>> + case 8:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
>>>>> + case 7:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
>>>>> + case 6:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
>>>>> + case 5:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
>>>>> + case 4:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
>>>>> + case 3:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
>>>>> + case 2:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
>>>>> + case 1:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
>>>>> + case 0:
>>>>> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
>>>>
>>>> I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
>>>>
>>>> cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
>>>
>>> Just like in the assembly version. We store the LRs in the order we read
>>> them so that we don't confuse the CPU by writing backward (believe it or
>>> not, CPUs do get horribly confused if you do that).
>>
>> but aren't we storing the wrong register to the wrong index in the
>> array?
>>
>> Do we really access cpu_if->vgic_lr[15..12] in the C-code if the system
>> only has 4 LRs?
>>
> ok, I looked at the code myself (not sure why I didn't do that in the
> first place) and indeed we use a different but with similar results
> macro to access the array from the C code.
>
> This is just insane to me, and we don't have a comment on the data
> structure saying "this is not stored the way you'd think it is".
>
> Why can't we just do:
>
> cpu_if->vgic_lr[3] = read_gicreg(ICH_LR3_EL2);
> cpu_if->vgic_lr[2] = read_gicreg(ICH_LR2_EL2);
> cpu_if->vgic_lr[1] = read_gicreg(ICH_LR1_EL2);
> cpu_if->vgic_lr[0] = read_gicreg(ICH_LR0_EL2);
>
> ?
Because you're *really* killing performance by doing what are
essentially streaming read/writes in the opposite direction. CPU
prefetchers only work in one direction (incrementing the address). Doing
it backwards breaks it.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 00/21] arm64: KVM: world switch in C
2015-12-01 9:58 ` Marc Zyngier
@ 2015-12-01 12:00 ` Christoffer Dall
2015-12-01 17:51 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 12:00 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 09:58:23AM +0000, Marc Zyngier wrote:
> On 30/11/15 20:33, Christoffer Dall wrote:
> > On Fri, Nov 27, 2015 at 06:49:54PM +0000, Marc Zyngier wrote:
> >> Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
> >> and mean piece of hand-crafted assembly code. Over time, features have
> >> crept in, the code has become harder to maintain, and the smallest
> >> change is a pain to introduce. The VHE patches are a prime example of
> >> why this doesn't work anymore.
> >>
> >> This series rewrites most of the existing assembly code in C, but keeps
> >> the existing code structure in place (most function names will look
> >> familiar to the reader). The biggest change is that we don't have to
> >> deal with a static register allocation (the compiler does it for us),
> >> we can easily follow structure and pointers, and only the lowest level
> >> is still in assembly code. Oh, and a negative diffstat.
> >>
> >> There is still a healthy dose of inline assembly (system register
> >> accessors, runtime code patching), but I've tried not to make it too
> >> invasive. The generated code, while not exactly brilliant, doesn't
> >> look too shaby. I do expect a small performance degradation, but I
> >> believe this is something we can improve over time (my initial
> >> measurements don't show any obvious regression though).
> >
> > I ran this through my experimental setup on m400 and got this:
>
> [...]
>
> > What this tells me is that we do take a noticable hit on the
> > world-switch path, which shows up in the TCP_RR and hackbench workloads,
> > which have a high precision in their output.
> >
> > Note that the memcached number is well within its variability between
> > individual benchmark runs, where it varies to 12% of its average in over
> > 80% of the executions.
> >
> > I don't think this is a showstopper thought, but we could consider
> > looking more closely at a breakdown of the world-switch path and verify
> > if/where we are really taking a hit.
>
> Thanks for doing so, very interesting. As a data point, what compiler
> are you using? I'd expect some variability based on the compiler version...
>
I used the following (compiling natively on the m400):
gcc version 4.8.2 (Ubuntu/Linaro 4.8.2-19ubuntu1)
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-12-01 11:57 ` Marc Zyngier
@ 2015-12-01 12:24 ` Christoffer Dall
2015-12-01 12:49 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 12:24 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 11:57:16AM +0000, Marc Zyngier wrote:
> On 01/12/15 11:50, Christoffer Dall wrote:
> > On Tue, Dec 01, 2015 at 12:44:26PM +0100, Christoffer Dall wrote:
> >> On Tue, Dec 01, 2015 at 11:32:20AM +0000, Marc Zyngier wrote:
> >>> On 30/11/15 19:50, Christoffer Dall wrote:
> >>>> On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
> >>>>> Implement the vgic-v3 save restore as a direct translation of
> >>>>> the assembly code version.
> >>>>>
> >>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >>>>> ---
> >>>>> arch/arm64/kvm/hyp/Makefile | 1 +
> >>>>> arch/arm64/kvm/hyp/hyp.h | 3 +
> >>>>> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
> >>>>> 3 files changed, 226 insertions(+)
> >>>>> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
> >>>>>
> >>>>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> >>>>> index d8d5968..d1e38ce 100644
> >>>>> --- a/arch/arm64/kvm/hyp/Makefile
> >>>>> +++ b/arch/arm64/kvm/hyp/Makefile
> >>>>> @@ -3,3 +3,4 @@
> >>>>> #
> >>>>>
> >>>>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> >>>>> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> >>>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> >>>>> index 78f25c4..a31cb6e 100644
> >>>>> --- a/arch/arm64/kvm/hyp/hyp.h
> >>>>> +++ b/arch/arm64/kvm/hyp/hyp.h
> >>>>> @@ -30,5 +30,8 @@
> >>>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> >>>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >>>>>
> >>>>> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
> >>>>> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> >>>>> +
> >>>>> #endif /* __ARM64_KVM_HYP_H__ */
> >>>>>
> >>>>> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> >>>>> new file mode 100644
> >>>>> index 0000000..b490db5
> >>>>> --- /dev/null
> >>>>> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> >>>>> @@ -0,0 +1,222 @@
> >>>>> +/*
> >>>>> + * Copyright (C) 2012-2015 - ARM Ltd
> >>>>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> >>>>> + *
> >>>>> + * This program is free software; you can redistribute it and/or modify
> >>>>> + * it under the terms of the GNU General Public License version 2 as
> >>>>> + * published by the Free Software Foundation.
> >>>>> + *
> >>>>> + * This program is distributed in the hope that it will be useful,
> >>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >>>>> + * GNU General Public License for more details.
> >>>>> + *
> >>>>> + * You should have received a copy of the GNU General Public License
> >>>>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> >>>>> + */
> >>>>> +
> >>>>> +#include <linux/compiler.h>
> >>>>> +#include <linux/irqchip/arm-gic-v3.h>
> >>>>> +#include <linux/kvm_host.h>
> >>>>> +
> >>>>> +#include <asm/kvm_mmu.h>
> >>>>> +
> >>>>> +#include "hyp.h"
> >>>>> +
> >>>>> +/*
> >>>>> + * We store LRs in reverse order to let the CPU deal with streaming
> >>>>> + * access. Use this macro to make it look saner...
> >>>>> + */
> >>>>> +#define LR_OFFSET(n) (15 - n)
> >>>>> +
> >>>>> +#define read_gicreg(r) \
> >>>>> + ({ \
> >>>>> + u64 reg; \
> >>>>> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
> >>>>> + reg; \
> >>>>> + })
> >>>>> +
> >>>>> +#define write_gicreg(v,r) \
> >>>>> + do { \
> >>>>> + u64 __val = (v); \
> >>>>> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
> >>>>> + } while (0)
> >>>>
> >>>> remind me what the msr_s and mrs_s do compared to msr and mrs?
> >>>
> >>> They do the same job, only for the system registers which are not in the
> >>> original ARMv8 architecture spec, and most likely not implemented by
> >>> old(er) compilers.
> >>>
> >>>> are these the reason why we need separate macros to access the gic
> >>>> registers compared to 'normal' sysregs?
> >>>
> >>> Indeed.
> >>>
> >>>>> +
> >>>>> +/* vcpu is already in the HYP VA space */
> >>>>> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
> >>>>> +{
> >>>>> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
> >>>>> + u64 val;
> >>>>> + u32 nr_lr, nr_pri;
> >>>>> +
> >>>>> + /*
> >>>>> + * Make sure stores to the GIC via the memory mapped interface
> >>>>> + * are now visible to the system register interface.
> >>>>> + */
> >>>>> + dsb(st);
> >>>>> +
> >>>>> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
> >>>>> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
> >>>>> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
> >>>>> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
> >>>>> +
> >>>>> + write_gicreg(0, ICH_HCR_EL2);
> >>>>> + val = read_gicreg(ICH_VTR_EL2);
> >>>>> + nr_lr = val & 0xf;
> >>>>
> >>>> this is not technically nr_lr, it's max_lr or max_lr_idx or something
> >>>> like that.
> >>>
> >>> Let's go for max_lr_idx then.
> >>>
> >>>>> + nr_pri = ((u32)val >> 29) + 1;
> >>>>
> >>>> nit: nr_pri_bits
> >>>>
> >>>>> +
> >>>>> + switch (nr_lr) {
> >>>>> + case 15:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
> >>>>> + case 14:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
> >>>>> + case 13:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
> >>>>> + case 12:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
> >>>>> + case 11:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
> >>>>> + case 10:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
> >>>>> + case 9:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
> >>>>> + case 8:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
> >>>>> + case 7:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
> >>>>> + case 6:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
> >>>>> + case 5:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
> >>>>> + case 4:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
> >>>>> + case 3:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
> >>>>> + case 2:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
> >>>>> + case 1:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
> >>>>> + case 0:
> >>>>> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
> >>>>
> >>>> I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
> >>>>
> >>>> cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
> >>>
> >>> Just like in the assembly version. We store the LRs in the order we read
> >>> them so that we don't confuse the CPU by writing backward (believe it or
> >>> not, CPUs do get horribly confused if you do that).
> >>
> >> but aren't we storing the wrong register to the wrong index in the
> >> array?
> >>
> >> Do we really access cpu_if->vgic_lr[15..12] in the C-code if the system
> >> only has 4 LRs?
> >>
> > ok, I looked at the code myself (not sure why I didn't do that in the
> > first place) and indeed we use a different but with similar results
> > macro to access the array from the C code.
> >
> > This is just insane to me, and we don't have a comment on the data
> > structure saying "this is not stored the way you'd think it is".
> >
> > Why can't we just do:
> >
> > cpu_if->vgic_lr[3] = read_gicreg(ICH_LR3_EL2);
> > cpu_if->vgic_lr[2] = read_gicreg(ICH_LR2_EL2);
> > cpu_if->vgic_lr[1] = read_gicreg(ICH_LR1_EL2);
> > cpu_if->vgic_lr[0] = read_gicreg(ICH_LR0_EL2);
> >
> > ?
>
> Because you're *really* killing performance by doing what are
> essentially streaming read/writes in the opposite direction. CPU
> prefetchers only work in one direction (incrementing the address). Doing
> it backwards breaks it.
>
hmm, and what are prefetching with the stores here?
Did anyone actually measure this or is it theoretically really slow?
Anyway, for the purposes of rewriting the world-switch in C, this looks
fine. I hope we can come up with something less convoluted some time,
perhaps at least using the same macro for LR_INDEX and making the
comments more clear.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore
2015-12-01 12:24 ` Christoffer Dall
@ 2015-12-01 12:49 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 12:49 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 12:24, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 11:57:16AM +0000, Marc Zyngier wrote:
>> On 01/12/15 11:50, Christoffer Dall wrote:
>>> On Tue, Dec 01, 2015 at 12:44:26PM +0100, Christoffer Dall wrote:
>>>> On Tue, Dec 01, 2015 at 11:32:20AM +0000, Marc Zyngier wrote:
>>>>> On 30/11/15 19:50, Christoffer Dall wrote:
>>>>>> On Fri, Nov 27, 2015 at 06:49:58PM +0000, Marc Zyngier wrote:
>>>>>>> Implement the vgic-v3 save restore as a direct translation of
>>>>>>> the assembly code version.
>>>>>>>
>>>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>>>>> ---
>>>>>>> arch/arm64/kvm/hyp/Makefile | 1 +
>>>>>>> arch/arm64/kvm/hyp/hyp.h | 3 +
>>>>>>> arch/arm64/kvm/hyp/vgic-v3-sr.c | 222 ++++++++++++++++++++++++++++++++++++++++
>>>>>>> 3 files changed, 226 insertions(+)
>>>>>>> create mode 100644 arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>>>>>
>>>>>>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>>>>>>> index d8d5968..d1e38ce 100644
>>>>>>> --- a/arch/arm64/kvm/hyp/Makefile
>>>>>>> +++ b/arch/arm64/kvm/hyp/Makefile
>>>>>>> @@ -3,3 +3,4 @@
>>>>>>> #
>>>>>>>
>>>>>>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>>>>>>> +obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>>>>>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>>>>>>> index 78f25c4..a31cb6e 100644
>>>>>>> --- a/arch/arm64/kvm/hyp/hyp.h
>>>>>>> +++ b/arch/arm64/kvm/hyp/hyp.h
>>>>>>> @@ -30,5 +30,8 @@
>>>>>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>>>>>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>>>>>>
>>>>>>> +void __vgic_v3_save_state(struct kvm_vcpu *vcpu);
>>>>>>> +void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
>>>>>>> +
>>>>>>> #endif /* __ARM64_KVM_HYP_H__ */
>>>>>>>
>>>>>>> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>>>>> new file mode 100644
>>>>>>> index 0000000..b490db5
>>>>>>> --- /dev/null
>>>>>>> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
>>>>>>> @@ -0,0 +1,222 @@
>>>>>>> +/*
>>>>>>> + * Copyright (C) 2012-2015 - ARM Ltd
>>>>>>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>>>>>>> + *
>>>>>>> + * This program is free software; you can redistribute it and/or modify
>>>>>>> + * it under the terms of the GNU General Public License version 2 as
>>>>>>> + * published by the Free Software Foundation.
>>>>>>> + *
>>>>>>> + * This program is distributed in the hope that it will be useful,
>>>>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>>>> + * GNU General Public License for more details.
>>>>>>> + *
>>>>>>> + * You should have received a copy of the GNU General Public License
>>>>>>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>>>> + */
>>>>>>> +
>>>>>>> +#include <linux/compiler.h>
>>>>>>> +#include <linux/irqchip/arm-gic-v3.h>
>>>>>>> +#include <linux/kvm_host.h>
>>>>>>> +
>>>>>>> +#include <asm/kvm_mmu.h>
>>>>>>> +
>>>>>>> +#include "hyp.h"
>>>>>>> +
>>>>>>> +/*
>>>>>>> + * We store LRs in reverse order to let the CPU deal with streaming
>>>>>>> + * access. Use this macro to make it look saner...
>>>>>>> + */
>>>>>>> +#define LR_OFFSET(n) (15 - n)
>>>>>>> +
>>>>>>> +#define read_gicreg(r) \
>>>>>>> + ({ \
>>>>>>> + u64 reg; \
>>>>>>> + asm volatile("mrs_s %0, " __stringify(r) : "=r" (reg)); \
>>>>>>> + reg; \
>>>>>>> + })
>>>>>>> +
>>>>>>> +#define write_gicreg(v,r) \
>>>>>>> + do { \
>>>>>>> + u64 __val = (v); \
>>>>>>> + asm volatile("msr_s " __stringify(r) ", %0" : : "r" (__val));\
>>>>>>> + } while (0)
>>>>>>
>>>>>> remind me what the msr_s and mrs_s do compared to msr and mrs?
>>>>>
>>>>> They do the same job, only for the system registers which are not in the
>>>>> original ARMv8 architecture spec, and most likely not implemented by
>>>>> old(er) compilers.
>>>>>
>>>>>> are these the reason why we need separate macros to access the gic
>>>>>> registers compared to 'normal' sysregs?
>>>>>
>>>>> Indeed.
>>>>>
>>>>>>> +
>>>>>>> +/* vcpu is already in the HYP VA space */
>>>>>>> +void __hyp_text __vgic_v3_save_state(struct kvm_vcpu *vcpu)
>>>>>>> +{
>>>>>>> + struct vgic_v3_cpu_if *cpu_if = &vcpu->arch.vgic_cpu.vgic_v3;
>>>>>>> + u64 val;
>>>>>>> + u32 nr_lr, nr_pri;
>>>>>>> +
>>>>>>> + /*
>>>>>>> + * Make sure stores to the GIC via the memory mapped interface
>>>>>>> + * are now visible to the system register interface.
>>>>>>> + */
>>>>>>> + dsb(st);
>>>>>>> +
>>>>>>> + cpu_if->vgic_vmcr = read_gicreg(ICH_VMCR_EL2);
>>>>>>> + cpu_if->vgic_misr = read_gicreg(ICH_MISR_EL2);
>>>>>>> + cpu_if->vgic_eisr = read_gicreg(ICH_EISR_EL2);
>>>>>>> + cpu_if->vgic_elrsr = read_gicreg(ICH_ELSR_EL2);
>>>>>>> +
>>>>>>> + write_gicreg(0, ICH_HCR_EL2);
>>>>>>> + val = read_gicreg(ICH_VTR_EL2);
>>>>>>> + nr_lr = val & 0xf;
>>>>>>
>>>>>> this is not technically nr_lr, it's max_lr or max_lr_idx or something
>>>>>> like that.
>>>>>
>>>>> Let's go for max_lr_idx then.
>>>>>
>>>>>>> + nr_pri = ((u32)val >> 29) + 1;
>>>>>>
>>>>>> nit: nr_pri_bits
>>>>>>
>>>>>>> +
>>>>>>> + switch (nr_lr) {
>>>>>>> + case 15:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(15)] = read_gicreg(ICH_LR15_EL2);
>>>>>>> + case 14:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(14)] = read_gicreg(ICH_LR14_EL2);
>>>>>>> + case 13:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(13)] = read_gicreg(ICH_LR13_EL2);
>>>>>>> + case 12:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(12)] = read_gicreg(ICH_LR12_EL2);
>>>>>>> + case 11:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(11)] = read_gicreg(ICH_LR11_EL2);
>>>>>>> + case 10:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(10)] = read_gicreg(ICH_LR10_EL2);
>>>>>>> + case 9:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(9)] = read_gicreg(ICH_LR9_EL2);
>>>>>>> + case 8:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(8)] = read_gicreg(ICH_LR8_EL2);
>>>>>>> + case 7:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(7)] = read_gicreg(ICH_LR7_EL2);
>>>>>>> + case 6:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(6)] = read_gicreg(ICH_LR6_EL2);
>>>>>>> + case 5:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(5)] = read_gicreg(ICH_LR5_EL2);
>>>>>>> + case 4:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(4)] = read_gicreg(ICH_LR4_EL2);
>>>>>>> + case 3:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(3)] = read_gicreg(ICH_LR3_EL2);
>>>>>>> + case 2:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(2)] = read_gicreg(ICH_LR2_EL2);
>>>>>>> + case 1:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(1)] = read_gicreg(ICH_LR1_EL2);
>>>>>>> + case 0:
>>>>>>> + cpu_if->vgic_lr[LR_OFFSET(0)] = read_gicreg(ICH_LR0_EL2);
>>>>>>
>>>>>> I don't understand this; LR_OFFSET(0) == (15 - 0) == 15, so
>>>>>>
>>>>>> cpu_if->vgic_lr[15] = read_gicreg(ICH_LR0_EL2) ?
>>>>>
>>>>> Just like in the assembly version. We store the LRs in the order we read
>>>>> them so that we don't confuse the CPU by writing backward (believe it or
>>>>> not, CPUs do get horribly confused if you do that).
>>>>
>>>> but aren't we storing the wrong register to the wrong index in the
>>>> array?
>>>>
>>>> Do we really access cpu_if->vgic_lr[15..12] in the C-code if the system
>>>> only has 4 LRs?
>>>>
>>> ok, I looked at the code myself (not sure why I didn't do that in the
>>> first place) and indeed we use a different but with similar results
>>> macro to access the array from the C code.
>>>
>>> This is just insane to me, and we don't have a comment on the data
>>> structure saying "this is not stored the way you'd think it is".
>>>
>>> Why can't we just do:
>>>
>>> cpu_if->vgic_lr[3] = read_gicreg(ICH_LR3_EL2);
>>> cpu_if->vgic_lr[2] = read_gicreg(ICH_LR2_EL2);
>>> cpu_if->vgic_lr[1] = read_gicreg(ICH_LR1_EL2);
>>> cpu_if->vgic_lr[0] = read_gicreg(ICH_LR0_EL2);
>>>
>>> ?
>>
>> Because you're *really* killing performance by doing what are
>> essentially streaming read/writes in the opposite direction. CPU
>> prefetchers only work in one direction (incrementing the address). Doing
>> it backwards breaks it.
>>
> hmm, and what are prefetching with the stores here?
Cache lines. When you increase the address, the core can trigger a
prefetch of the next cache line so that it is ready by the time you get
to start clobbering it.
> Did anyone actually measure this or is it theoretically really slow?
I did some measurements around that in a previous life.
> Anyway, for the purposes of rewriting the world-switch in C, this looks
> fine. I hope we can come up with something less convoluted some time,
> perhaps at least using the same macro for LR_INDEX and making the
> comments more clear.
Sure.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-11-27 18:50 ` [PATCH v2 08/21] arm64: KVM: Implement debug save/restore Marc Zyngier
2015-11-30 12:00 ` Alex Bennée
@ 2015-12-01 12:56 ` Christoffer Dall
2015-12-01 13:06 ` Marc Zyngier
1 sibling, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 12:56 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
> Implement the debug save restore as a direct translation of
> the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/hyp.h | 9 +++
> 3 files changed, 140 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index ec94200..ec14cac 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> new file mode 100644
> index 0000000..a0b2b99
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -0,0 +1,130 @@
> +/*
> + * Copyright (C) 2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/compiler.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_mmu.h>
> +
> +#include "hyp.h"
> +
> +#define read_debug(r,n) read_sysreg(r##n##_el1)
> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
> +
> +#define save_debug(ptr,reg,nr) \
> + switch (nr) { \
> + case 15: ptr[15] = read_debug(reg, 15); \
> + case 14: ptr[14] = read_debug(reg, 14); \
> + case 13: ptr[13] = read_debug(reg, 13); \
> + case 12: ptr[12] = read_debug(reg, 12); \
> + case 11: ptr[11] = read_debug(reg, 11); \
> + case 10: ptr[10] = read_debug(reg, 10); \
> + case 9: ptr[9] = read_debug(reg, 9); \
> + case 8: ptr[8] = read_debug(reg, 8); \
> + case 7: ptr[7] = read_debug(reg, 7); \
> + case 6: ptr[6] = read_debug(reg, 6); \
> + case 5: ptr[5] = read_debug(reg, 5); \
> + case 4: ptr[4] = read_debug(reg, 4); \
> + case 3: ptr[3] = read_debug(reg, 3); \
> + case 2: ptr[2] = read_debug(reg, 2); \
> + case 1: ptr[1] = read_debug(reg, 1); \
> + default: ptr[0] = read_debug(reg, 0); \
> + }
> +
> +#define restore_debug(ptr,reg,nr) \
> + switch (nr) { \
> + case 15: write_debug(ptr[15], reg, 15); \
> + case 14: write_debug(ptr[14], reg, 14); \
> + case 13: write_debug(ptr[13], reg, 13); \
> + case 12: write_debug(ptr[12], reg, 12); \
> + case 11: write_debug(ptr[11], reg, 11); \
> + case 10: write_debug(ptr[10], reg, 10); \
> + case 9: write_debug(ptr[9], reg, 9); \
> + case 8: write_debug(ptr[8], reg, 8); \
> + case 7: write_debug(ptr[7], reg, 7); \
> + case 6: write_debug(ptr[6], reg, 6); \
> + case 5: write_debug(ptr[5], reg, 5); \
> + case 4: write_debug(ptr[4], reg, 4); \
> + case 3: write_debug(ptr[3], reg, 3); \
> + case 2: write_debug(ptr[2], reg, 2); \
> + case 1: write_debug(ptr[1], reg, 1); \
> + default: write_debug(ptr[0], reg, 0); \
> + }
> +
> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> +{
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> + int brps, wrps;
> +
> + brps = (aa64dfr0 >> 12) & 0xf;
> + wrps = (aa64dfr0 >> 20) & 0xf;
> +
> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
> +
> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> + }
> +}
> +
> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt)
> +{
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> + int brps, wrps;
> +
> + brps = (aa64dfr0 >> 12) & 0xf;
> + wrps = (aa64dfr0 >> 20) & 0xf;
> +
> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
> +
> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> + }
> +}
> +
> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> +{
> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> +
> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
> + kern_hyp_va(vcpu->arch.host_cpu_context));
doesn't the assmebly code jump across saving this state neither bits are
set where this always saves the state?
in any case, I feel some context is lost when this is moved away from
assembly and understanding this patch would be easier if the semantics
of these two _cond functions were documented.
-Christoffer
> +}
> +
> +void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> +{
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> + __debug_restore_state(vcpu, &vcpu->arch.host_debug_state,
> + kern_hyp_va(vcpu->arch.host_cpu_context));
> + vcpu->arch.debug_flags &= ~KVM_ARM64_DEBUG_DIRTY;
> + }
> +}
> +
> +u32 __hyp_text __debug_read_mdcr_el2(void)
> +{
> + return read_sysreg(mdcr_el2);
> +}
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 4639330..2581232 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -41,5 +41,14 @@ void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
> void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
>
> +void __debug_save_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt);
> +void __debug_restore_state(struct kvm_vcpu *vcpu,
> + struct kvm_guest_debug_arch *dbg,
> + struct kvm_cpu_context *ctxt);
> +void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> +void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> +
> #endif /* __ARM64_KVM_HYP_H__ */
>
> --
> 2.1.4
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 12:56 ` Christoffer Dall
@ 2015-12-01 13:06 ` Marc Zyngier
2015-12-01 13:19 ` Alex Bennée
2015-12-01 14:47 ` Christoffer Dall
0 siblings, 2 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 13:06 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 12:56, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
>> Implement the debug save restore as a direct translation of
>> the assembly code version.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/hyp/Makefile | 1 +
>> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
>> arch/arm64/kvm/hyp/hyp.h | 9 +++
>> 3 files changed, 140 insertions(+)
>> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
>>
>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>> index ec94200..ec14cac 100644
>> --- a/arch/arm64/kvm/hyp/Makefile
>> +++ b/arch/arm64/kvm/hyp/Makefile
>> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
>> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
>> new file mode 100644
>> index 0000000..a0b2b99
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/debug-sr.c
>> @@ -0,0 +1,130 @@
>> +/*
>> + * Copyright (C) 2015 - ARM Ltd
>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <linux/compiler.h>
>> +#include <linux/kvm_host.h>
>> +
>> +#include <asm/kvm_mmu.h>
>> +
>> +#include "hyp.h"
>> +
>> +#define read_debug(r,n) read_sysreg(r##n##_el1)
>> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
>> +
>> +#define save_debug(ptr,reg,nr) \
>> + switch (nr) { \
>> + case 15: ptr[15] = read_debug(reg, 15); \
>> + case 14: ptr[14] = read_debug(reg, 14); \
>> + case 13: ptr[13] = read_debug(reg, 13); \
>> + case 12: ptr[12] = read_debug(reg, 12); \
>> + case 11: ptr[11] = read_debug(reg, 11); \
>> + case 10: ptr[10] = read_debug(reg, 10); \
>> + case 9: ptr[9] = read_debug(reg, 9); \
>> + case 8: ptr[8] = read_debug(reg, 8); \
>> + case 7: ptr[7] = read_debug(reg, 7); \
>> + case 6: ptr[6] = read_debug(reg, 6); \
>> + case 5: ptr[5] = read_debug(reg, 5); \
>> + case 4: ptr[4] = read_debug(reg, 4); \
>> + case 3: ptr[3] = read_debug(reg, 3); \
>> + case 2: ptr[2] = read_debug(reg, 2); \
>> + case 1: ptr[1] = read_debug(reg, 1); \
>> + default: ptr[0] = read_debug(reg, 0); \
>> + }
>> +
>> +#define restore_debug(ptr,reg,nr) \
>> + switch (nr) { \
>> + case 15: write_debug(ptr[15], reg, 15); \
>> + case 14: write_debug(ptr[14], reg, 14); \
>> + case 13: write_debug(ptr[13], reg, 13); \
>> + case 12: write_debug(ptr[12], reg, 12); \
>> + case 11: write_debug(ptr[11], reg, 11); \
>> + case 10: write_debug(ptr[10], reg, 10); \
>> + case 9: write_debug(ptr[9], reg, 9); \
>> + case 8: write_debug(ptr[8], reg, 8); \
>> + case 7: write_debug(ptr[7], reg, 7); \
>> + case 6: write_debug(ptr[6], reg, 6); \
>> + case 5: write_debug(ptr[5], reg, 5); \
>> + case 4: write_debug(ptr[4], reg, 4); \
>> + case 3: write_debug(ptr[3], reg, 3); \
>> + case 2: write_debug(ptr[2], reg, 2); \
>> + case 1: write_debug(ptr[1], reg, 1); \
>> + default: write_debug(ptr[0], reg, 0); \
>> + }
>> +
>> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
>> + struct kvm_guest_debug_arch *dbg,
>> + struct kvm_cpu_context *ctxt)
>> +{
>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>> + int brps, wrps;
>> +
>> + brps = (aa64dfr0 >> 12) & 0xf;
>> + wrps = (aa64dfr0 >> 20) & 0xf;
>> +
>> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
>> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
>> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
>> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
>> +
>> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
>> + }
>> +}
>> +
>> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
>> + struct kvm_guest_debug_arch *dbg,
>> + struct kvm_cpu_context *ctxt)
>> +{
>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>> + int brps, wrps;
>> +
>> + brps = (aa64dfr0 >> 12) & 0xf;
>> + wrps = (aa64dfr0 >> 20) & 0xf;
>> +
>> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
>> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
>> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
>> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
>> +
>> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
>> + }
>> +}
>> +
>> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
>> +{
>> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
>> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
>> +
>> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
>> + kern_hyp_va(vcpu->arch.host_cpu_context));
>
> doesn't the assmebly code jump across saving this state neither bits are
> set where this always saves the state?
It doesn't. The save/restore functions are guarded by tests on
KVM_ARM64_DEBUG_DIRTY, just like we have skip_debug_state on all actions
involving the save/restore in the assembly version.
> in any case, I feel some context is lost when this is moved away from
> assembly and understanding this patch would be easier if the semantics
> of these two _cond functions were documented.
I can migrate the existing comments if you think that helps.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 13:06 ` Marc Zyngier
@ 2015-12-01 13:19 ` Alex Bennée
2015-12-01 13:34 ` Marc Zyngier
2015-12-01 14:47 ` Christoffer Dall
1 sibling, 1 reply; 88+ messages in thread
From: Alex Bennée @ 2015-12-01 13:19 UTC (permalink / raw)
To: linux-arm-kernel
Marc Zyngier <marc.zyngier@arm.com> writes:
> On 01/12/15 12:56, Christoffer Dall wrote:
>> On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
>>> Implement the debug save restore as a direct translation of
>>> the assembly code version.
>>>
>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>> ---
>>> arch/arm64/kvm/hyp/Makefile | 1 +
>>> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
>>> arch/arm64/kvm/hyp/hyp.h | 9 +++
>>> 3 files changed, 140 insertions(+)
>>> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
<snip>
>>> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
>>> +{
>>> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
>>> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
I've just noticed I'm seeing double here. Did a DBG_MDSCR_MDE can
transliterated here?
>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
>>> +
>>> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
>>> + kern_hyp_va(vcpu->arch.host_cpu_context));
>>
>> doesn't the assmebly code jump across saving this state neither bits are
>> set where this always saves the state?
>
> It doesn't. The save/restore functions are guarded by tests on
> KVM_ARM64_DEBUG_DIRTY, just like we have skip_debug_state on all actions
> involving the save/restore in the assembly version.
>
>> in any case, I feel some context is lost when this is moved away from
>> assembly and understanding this patch would be easier if the semantics
>> of these two _cond functions were documented.
>
> I can migrate the existing comments if you think that helps.
>
> Thanks,
>
> M.
--
Alex Benn?e
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 13:19 ` Alex Bennée
@ 2015-12-01 13:34 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 13:34 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 13:19, Alex Benn?e wrote:
>
> Marc Zyngier <marc.zyngier@arm.com> writes:
>
>> On 01/12/15 12:56, Christoffer Dall wrote:
>>> On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
>>>> Implement the debug save restore as a direct translation of
>>>> the assembly code version.
>>>>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>> arch/arm64/kvm/hyp/Makefile | 1 +
>>>> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
>>>> arch/arm64/kvm/hyp/hyp.h | 9 +++
>>>> 3 files changed, 140 insertions(+)
>>>> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
> <snip>
>>>> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
>>>> +{
>>>> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
>>>> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
>
> I've just noticed I'm seeing double here. Did a DBG_MDSCR_MDE can
> transliterated here?
Quite probably! Guess there is a small hole in your test suite! ;-)
Thanks for noticing this.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 13:06 ` Marc Zyngier
2015-12-01 13:19 ` Alex Bennée
@ 2015-12-01 14:47 ` Christoffer Dall
2015-12-01 14:56 ` Christoffer Dall
2015-12-01 15:01 ` Marc Zyngier
1 sibling, 2 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 14:47 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 01:06:31PM +0000, Marc Zyngier wrote:
> On 01/12/15 12:56, Christoffer Dall wrote:
> > On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
> >> Implement the debug save restore as a direct translation of
> >> the assembly code version.
> >>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >> arch/arm64/kvm/hyp/Makefile | 1 +
> >> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
> >> arch/arm64/kvm/hyp/hyp.h | 9 +++
> >> 3 files changed, 140 insertions(+)
> >> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
> >>
> >> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> >> index ec94200..ec14cac 100644
> >> --- a/arch/arm64/kvm/hyp/Makefile
> >> +++ b/arch/arm64/kvm/hyp/Makefile
> >> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> >> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> >> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> >> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> >> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> >> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> >> new file mode 100644
> >> index 0000000..a0b2b99
> >> --- /dev/null
> >> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> >> @@ -0,0 +1,130 @@
> >> +/*
> >> + * Copyright (C) 2015 - ARM Ltd
> >> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> >> + *
> >> + * This program is free software; you can redistribute it and/or modify
> >> + * it under the terms of the GNU General Public License version 2 as
> >> + * published by the Free Software Foundation.
> >> + *
> >> + * This program is distributed in the hope that it will be useful,
> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> + * GNU General Public License for more details.
> >> + *
> >> + * You should have received a copy of the GNU General Public License
> >> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> >> + */
> >> +
> >> +#include <linux/compiler.h>
> >> +#include <linux/kvm_host.h>
> >> +
> >> +#include <asm/kvm_mmu.h>
> >> +
> >> +#include "hyp.h"
> >> +
> >> +#define read_debug(r,n) read_sysreg(r##n##_el1)
> >> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
> >> +
> >> +#define save_debug(ptr,reg,nr) \
> >> + switch (nr) { \
> >> + case 15: ptr[15] = read_debug(reg, 15); \
> >> + case 14: ptr[14] = read_debug(reg, 14); \
> >> + case 13: ptr[13] = read_debug(reg, 13); \
> >> + case 12: ptr[12] = read_debug(reg, 12); \
> >> + case 11: ptr[11] = read_debug(reg, 11); \
> >> + case 10: ptr[10] = read_debug(reg, 10); \
> >> + case 9: ptr[9] = read_debug(reg, 9); \
> >> + case 8: ptr[8] = read_debug(reg, 8); \
> >> + case 7: ptr[7] = read_debug(reg, 7); \
> >> + case 6: ptr[6] = read_debug(reg, 6); \
> >> + case 5: ptr[5] = read_debug(reg, 5); \
> >> + case 4: ptr[4] = read_debug(reg, 4); \
> >> + case 3: ptr[3] = read_debug(reg, 3); \
> >> + case 2: ptr[2] = read_debug(reg, 2); \
> >> + case 1: ptr[1] = read_debug(reg, 1); \
> >> + default: ptr[0] = read_debug(reg, 0); \
> >> + }
> >> +
> >> +#define restore_debug(ptr,reg,nr) \
> >> + switch (nr) { \
> >> + case 15: write_debug(ptr[15], reg, 15); \
> >> + case 14: write_debug(ptr[14], reg, 14); \
> >> + case 13: write_debug(ptr[13], reg, 13); \
> >> + case 12: write_debug(ptr[12], reg, 12); \
> >> + case 11: write_debug(ptr[11], reg, 11); \
> >> + case 10: write_debug(ptr[10], reg, 10); \
> >> + case 9: write_debug(ptr[9], reg, 9); \
> >> + case 8: write_debug(ptr[8], reg, 8); \
> >> + case 7: write_debug(ptr[7], reg, 7); \
> >> + case 6: write_debug(ptr[6], reg, 6); \
> >> + case 5: write_debug(ptr[5], reg, 5); \
> >> + case 4: write_debug(ptr[4], reg, 4); \
> >> + case 3: write_debug(ptr[3], reg, 3); \
> >> + case 2: write_debug(ptr[2], reg, 2); \
> >> + case 1: write_debug(ptr[1], reg, 1); \
> >> + default: write_debug(ptr[0], reg, 0); \
> >> + }
> >> +
> >> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> >> + struct kvm_guest_debug_arch *dbg,
> >> + struct kvm_cpu_context *ctxt)
> >> +{
> >> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> >> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> >> + int brps, wrps;
> >> +
> >> + brps = (aa64dfr0 >> 12) & 0xf;
> >> + wrps = (aa64dfr0 >> 20) & 0xf;
> >> +
> >> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
> >> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
> >> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
> >> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
> >> +
> >> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> >> + }
> >> +}
> >> +
> >> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> >> + struct kvm_guest_debug_arch *dbg,
> >> + struct kvm_cpu_context *ctxt)
> >> +{
> >> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> >> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> >> + int brps, wrps;
> >> +
> >> + brps = (aa64dfr0 >> 12) & 0xf;
> >> + wrps = (aa64dfr0 >> 20) & 0xf;
> >> +
> >> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
> >> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
> >> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
> >> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
> >> +
> >> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> >> + }
> >> +}
> >> +
> >> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> >> +{
> >> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> >> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
> >> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> >> +
> >> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
> >> + kern_hyp_va(vcpu->arch.host_cpu_context));
> >
> > doesn't the assmebly code jump across saving this state neither bits are
> > set where this always saves the state?
>
> It doesn't. The save/restore functions are guarded by tests on
> KVM_ARM64_DEBUG_DIRTY, just like we have skip_debug_state on all actions
> involving the save/restore in the assembly version.
I think the confusing part is that the save function unconditionally
calls __debug_save_state where the restore function only calls it when
the dirty flag is set. Plus I suck at reading assembly apparently.
>
> > in any case, I feel some context is lost when this is moved away from
> > assembly and understanding this patch would be easier if the semantics
> > of these two _cond functions were documented.
>
> I can migrate the existing comments if you think that helps.
>
It just wasn't not quite clear to me exactly when
__debug_cond_save_host_state is called for example - is this going to be
called unconditionally on every entry - that's how I understand it now
anyway.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 14:47 ` Christoffer Dall
@ 2015-12-01 14:56 ` Christoffer Dall
2015-12-01 15:01 ` Marc Zyngier
1 sibling, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 14:56 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 03:47:37PM +0100, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 01:06:31PM +0000, Marc Zyngier wrote:
> > On 01/12/15 12:56, Christoffer Dall wrote:
> > > On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
> > >> Implement the debug save restore as a direct translation of
> > >> the assembly code version.
> > >>
> > >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> > >> ---
> > >> arch/arm64/kvm/hyp/Makefile | 1 +
> > >> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
> > >> arch/arm64/kvm/hyp/hyp.h | 9 +++
> > >> 3 files changed, 140 insertions(+)
> > >> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
> > >>
> > >> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> > >> index ec94200..ec14cac 100644
> > >> --- a/arch/arm64/kvm/hyp/Makefile
> > >> +++ b/arch/arm64/kvm/hyp/Makefile
> > >> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> > >> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> > >> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> > >> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> > >> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> > >> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> > >> new file mode 100644
> > >> index 0000000..a0b2b99
> > >> --- /dev/null
> > >> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> > >> @@ -0,0 +1,130 @@
> > >> +/*
> > >> + * Copyright (C) 2015 - ARM Ltd
> > >> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> > >> + *
> > >> + * This program is free software; you can redistribute it and/or modify
> > >> + * it under the terms of the GNU General Public License version 2 as
> > >> + * published by the Free Software Foundation.
> > >> + *
> > >> + * This program is distributed in the hope that it will be useful,
> > >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> > >> + * GNU General Public License for more details.
> > >> + *
> > >> + * You should have received a copy of the GNU General Public License
> > >> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> > >> + */
> > >> +
> > >> +#include <linux/compiler.h>
> > >> +#include <linux/kvm_host.h>
> > >> +
> > >> +#include <asm/kvm_mmu.h>
> > >> +
> > >> +#include "hyp.h"
> > >> +
> > >> +#define read_debug(r,n) read_sysreg(r##n##_el1)
> > >> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
> > >> +
> > >> +#define save_debug(ptr,reg,nr) \
> > >> + switch (nr) { \
> > >> + case 15: ptr[15] = read_debug(reg, 15); \
> > >> + case 14: ptr[14] = read_debug(reg, 14); \
> > >> + case 13: ptr[13] = read_debug(reg, 13); \
> > >> + case 12: ptr[12] = read_debug(reg, 12); \
> > >> + case 11: ptr[11] = read_debug(reg, 11); \
> > >> + case 10: ptr[10] = read_debug(reg, 10); \
> > >> + case 9: ptr[9] = read_debug(reg, 9); \
> > >> + case 8: ptr[8] = read_debug(reg, 8); \
> > >> + case 7: ptr[7] = read_debug(reg, 7); \
> > >> + case 6: ptr[6] = read_debug(reg, 6); \
> > >> + case 5: ptr[5] = read_debug(reg, 5); \
> > >> + case 4: ptr[4] = read_debug(reg, 4); \
> > >> + case 3: ptr[3] = read_debug(reg, 3); \
> > >> + case 2: ptr[2] = read_debug(reg, 2); \
> > >> + case 1: ptr[1] = read_debug(reg, 1); \
> > >> + default: ptr[0] = read_debug(reg, 0); \
> > >> + }
> > >> +
> > >> +#define restore_debug(ptr,reg,nr) \
> > >> + switch (nr) { \
> > >> + case 15: write_debug(ptr[15], reg, 15); \
> > >> + case 14: write_debug(ptr[14], reg, 14); \
> > >> + case 13: write_debug(ptr[13], reg, 13); \
> > >> + case 12: write_debug(ptr[12], reg, 12); \
> > >> + case 11: write_debug(ptr[11], reg, 11); \
> > >> + case 10: write_debug(ptr[10], reg, 10); \
> > >> + case 9: write_debug(ptr[9], reg, 9); \
> > >> + case 8: write_debug(ptr[8], reg, 8); \
> > >> + case 7: write_debug(ptr[7], reg, 7); \
> > >> + case 6: write_debug(ptr[6], reg, 6); \
> > >> + case 5: write_debug(ptr[5], reg, 5); \
> > >> + case 4: write_debug(ptr[4], reg, 4); \
> > >> + case 3: write_debug(ptr[3], reg, 3); \
> > >> + case 2: write_debug(ptr[2], reg, 2); \
> > >> + case 1: write_debug(ptr[1], reg, 1); \
> > >> + default: write_debug(ptr[0], reg, 0); \
> > >> + }
> > >> +
> > >> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> > >> + struct kvm_guest_debug_arch *dbg,
> > >> + struct kvm_cpu_context *ctxt)
> > >> +{
> > >> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> > >> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > >> + int brps, wrps;
> > >> +
> > >> + brps = (aa64dfr0 >> 12) & 0xf;
> > >> + wrps = (aa64dfr0 >> 20) & 0xf;
> > >> +
> > >> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
> > >> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
> > >> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
> > >> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
> > >> +
> > >> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> > >> + }
> > >> +}
> > >> +
> > >> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> > >> + struct kvm_guest_debug_arch *dbg,
> > >> + struct kvm_cpu_context *ctxt)
> > >> +{
> > >> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> > >> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> > >> + int brps, wrps;
> > >> +
> > >> + brps = (aa64dfr0 >> 12) & 0xf;
> > >> + wrps = (aa64dfr0 >> 20) & 0xf;
> > >> +
> > >> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
> > >> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
> > >> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
> > >> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
> > >> +
> > >> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> > >> + }
> > >> +}
> > >> +
> > >> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> > >> +{
> > >> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> > >> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
> > >> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> > >> +
> > >> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
> > >> + kern_hyp_va(vcpu->arch.host_cpu_context));
> > >
> > > doesn't the assmebly code jump across saving this state neither bits are
> > > set where this always saves the state?
> >
> > It doesn't. The save/restore functions are guarded by tests on
> > KVM_ARM64_DEBUG_DIRTY, just like we have skip_debug_state on all actions
> > involving the save/restore in the assembly version.
>
> I think the confusing part is that the save function unconditionally
> calls __debug_save_state where the restore function only calls it when
> the dirty flag is set. Plus I suck at reading assembly apparently.
>
> >
> > > in any case, I feel some context is lost when this is moved away from
> > > assembly and understanding this patch would be easier if the semantics
> > > of these two _cond functions were documented.
> >
> > I can migrate the existing comments if you think that helps.
> >
> It just wasn't not quite clear to me exactly when
> __debug_cond_save_host_state is called for example - is this going to be
> called unconditionally on every entry - that's how I understand it now
> anyway.
>
Assuming I got that right:
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 14:47 ` Christoffer Dall
2015-12-01 14:56 ` Christoffer Dall
@ 2015-12-01 15:01 ` Marc Zyngier
2015-12-01 15:41 ` Christoffer Dall
1 sibling, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 15:01 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 14:47, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 01:06:31PM +0000, Marc Zyngier wrote:
>> On 01/12/15 12:56, Christoffer Dall wrote:
>>> On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
>>>> Implement the debug save restore as a direct translation of
>>>> the assembly code version.
>>>>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>> arch/arm64/kvm/hyp/Makefile | 1 +
>>>> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
>>>> arch/arm64/kvm/hyp/hyp.h | 9 +++
>>>> 3 files changed, 140 insertions(+)
>>>> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
>>>>
>>>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>>>> index ec94200..ec14cac 100644
>>>> --- a/arch/arm64/kvm/hyp/Makefile
>>>> +++ b/arch/arm64/kvm/hyp/Makefile
>>>> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>>>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>>>> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
>>>> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
>>>> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
>>>> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
>>>> new file mode 100644
>>>> index 0000000..a0b2b99
>>>> --- /dev/null
>>>> +++ b/arch/arm64/kvm/hyp/debug-sr.c
>>>> @@ -0,0 +1,130 @@
>>>> +/*
>>>> + * Copyright (C) 2015 - ARM Ltd
>>>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>>>> + *
>>>> + * This program is free software; you can redistribute it and/or modify
>>>> + * it under the terms of the GNU General Public License version 2 as
>>>> + * published by the Free Software Foundation.
>>>> + *
>>>> + * This program is distributed in the hope that it will be useful,
>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>> + * GNU General Public License for more details.
>>>> + *
>>>> + * You should have received a copy of the GNU General Public License
>>>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>> + */
>>>> +
>>>> +#include <linux/compiler.h>
>>>> +#include <linux/kvm_host.h>
>>>> +
>>>> +#include <asm/kvm_mmu.h>
>>>> +
>>>> +#include "hyp.h"
>>>> +
>>>> +#define read_debug(r,n) read_sysreg(r##n##_el1)
>>>> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
>>>> +
>>>> +#define save_debug(ptr,reg,nr) \
>>>> + switch (nr) { \
>>>> + case 15: ptr[15] = read_debug(reg, 15); \
>>>> + case 14: ptr[14] = read_debug(reg, 14); \
>>>> + case 13: ptr[13] = read_debug(reg, 13); \
>>>> + case 12: ptr[12] = read_debug(reg, 12); \
>>>> + case 11: ptr[11] = read_debug(reg, 11); \
>>>> + case 10: ptr[10] = read_debug(reg, 10); \
>>>> + case 9: ptr[9] = read_debug(reg, 9); \
>>>> + case 8: ptr[8] = read_debug(reg, 8); \
>>>> + case 7: ptr[7] = read_debug(reg, 7); \
>>>> + case 6: ptr[6] = read_debug(reg, 6); \
>>>> + case 5: ptr[5] = read_debug(reg, 5); \
>>>> + case 4: ptr[4] = read_debug(reg, 4); \
>>>> + case 3: ptr[3] = read_debug(reg, 3); \
>>>> + case 2: ptr[2] = read_debug(reg, 2); \
>>>> + case 1: ptr[1] = read_debug(reg, 1); \
>>>> + default: ptr[0] = read_debug(reg, 0); \
>>>> + }
>>>> +
>>>> +#define restore_debug(ptr,reg,nr) \
>>>> + switch (nr) { \
>>>> + case 15: write_debug(ptr[15], reg, 15); \
>>>> + case 14: write_debug(ptr[14], reg, 14); \
>>>> + case 13: write_debug(ptr[13], reg, 13); \
>>>> + case 12: write_debug(ptr[12], reg, 12); \
>>>> + case 11: write_debug(ptr[11], reg, 11); \
>>>> + case 10: write_debug(ptr[10], reg, 10); \
>>>> + case 9: write_debug(ptr[9], reg, 9); \
>>>> + case 8: write_debug(ptr[8], reg, 8); \
>>>> + case 7: write_debug(ptr[7], reg, 7); \
>>>> + case 6: write_debug(ptr[6], reg, 6); \
>>>> + case 5: write_debug(ptr[5], reg, 5); \
>>>> + case 4: write_debug(ptr[4], reg, 4); \
>>>> + case 3: write_debug(ptr[3], reg, 3); \
>>>> + case 2: write_debug(ptr[2], reg, 2); \
>>>> + case 1: write_debug(ptr[1], reg, 1); \
>>>> + default: write_debug(ptr[0], reg, 0); \
>>>> + }
>>>> +
>>>> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
>>>> + struct kvm_guest_debug_arch *dbg,
>>>> + struct kvm_cpu_context *ctxt)
>>>> +{
>>>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
>>>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>>>> + int brps, wrps;
>>>> +
>>>> + brps = (aa64dfr0 >> 12) & 0xf;
>>>> + wrps = (aa64dfr0 >> 20) & 0xf;
>>>> +
>>>> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
>>>> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
>>>> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
>>>> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
>>>> +
>>>> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
>>>> + }
>>>> +}
>>>> +
>>>> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
>>>> + struct kvm_guest_debug_arch *dbg,
>>>> + struct kvm_cpu_context *ctxt)
>>>> +{
>>>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
>>>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>>>> + int brps, wrps;
>>>> +
>>>> + brps = (aa64dfr0 >> 12) & 0xf;
>>>> + wrps = (aa64dfr0 >> 20) & 0xf;
>>>> +
>>>> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
>>>> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
>>>> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
>>>> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
>>>> +
>>>> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
>>>> + }
>>>> +}
>>>> +
>>>> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
>>>> +{
>>>> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
>>>> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
>>>> +
>>>> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
>>>> + kern_hyp_va(vcpu->arch.host_cpu_context));
>>>
>>> doesn't the assmebly code jump across saving this state neither bits are
>>> set where this always saves the state?
>>
>> It doesn't. The save/restore functions are guarded by tests on
>> KVM_ARM64_DEBUG_DIRTY, just like we have skip_debug_state on all actions
>> involving the save/restore in the assembly version.
>
> I think the confusing part is that the save function unconditionally
> calls __debug_save_state where the restore function only calls it when
> the dirty flag is set. Plus I suck at reading assembly apparently.
So the way I initially wrote it, I had the same 'if' statement as in the
restore function, making them fairly symmetric. But it quickly became
obvious that this double-if was a bit pointless.
And actually, I wonder if I shouldn't drop it from the restore function,
because it only save us a spurious clear of the dirty bit.
>>> in any case, I feel some context is lost when this is moved away from
>>> assembly and understanding this patch would be easier if the semantics
>>> of these two _cond functions were documented.
>>
>> I can migrate the existing comments if you think that helps.
>>
> It just wasn't not quite clear to me exactly when
> __debug_cond_save_host_state is called for example - is this going to be
> called unconditionally on every entry - that's how I understand it now
> anyway.
On every entry, yes. I'm trying to have the guest_run function as simple
as possible, with the various subsystems making their 'own' decisions.
Not optimal (you get to branch for nothing), but clearer. At least for
me, but I may be the odd duck out here. Any idea to make the flow look
clearer?
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 09/21] arm64: KVM: Implement guest entry
2015-11-27 18:50 ` [PATCH v2 09/21] arm64: KVM: Implement guest entry Marc Zyngier
@ 2015-12-01 15:29 ` Christoffer Dall
2015-12-01 18:41 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 15:29 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:03PM +0000, Marc Zyngier wrote:
> Contrary to the previous patch, the guest entry is fairly different
> from its assembly counterpart, mostly because it is only concerned
> with saving/restoring the GP registers, and nothing else.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/entry.S | 155 ++++++++++++++++++++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/hyp.h | 2 +
> 3 files changed, 158 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/entry.S
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index ec14cac..1e1ff06 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -7,3 +7,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += entry.o
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> new file mode 100644
> index 0000000..2c4449a
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -0,0 +1,155 @@
> +/*
> + * Copyright (C) 2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/linkage.h>
> +
> +#include <asm/asm-offsets.h>
> +#include <asm/assembler.h>
> +#include <asm/fpsimdmacros.h>
> +#include <asm/kvm.h>
> +#include <asm/kvm_arm.h>
> +#include <asm/kvm_asm.h>
> +#include <asm/kvm_mmu.h>
> +
> +#define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
> +#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
> +
> + .text
> + .pushsection .hyp.text, "ax"
> +
> +.macro save_common_regs ctxt
> + stp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
> + stp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
> + stp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
> + stp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
> + stp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
> + stp x29, lr, [\ctxt, #CPU_XREG_OFFSET(29)]
> +.endm
> +
> +.macro restore_common_regs ctxt
> + ldp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
> + ldp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
> + ldp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
> + ldp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
> + ldp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
> + ldp x29, lr, [\ctxt, #CPU_XREG_OFFSET(29)]
> +.endm
> +
> +.macro save_host_regs reg
> + save_common_regs \reg
> +.endm
> +
> +.macro restore_host_regs reg
> + restore_common_regs \reg
> +.endm
> +
> +.macro save_guest_regs
> + // x0 is the vcpu address
> + // x1 is the return code, do not corrupt!
> + // x2 is the cpu context
this is confusing because the caller says x2 is free, so are these the
inputs or invariants preserved in the function, or?
note that you'll avoid this kind of confusion by inlining this stuff in
__guest_exit.
> + // x3 is a tmp register
> + // Guest's x0-x3 are on the stack
> +
> + add x2, x0, #VCPU_CONTEXT
> +
> + // Compute base to save registers
misleading comment?
> + stp x4, x5, [x2, #CPU_XREG_OFFSET(4)]
> + stp x6, x7, [x2, #CPU_XREG_OFFSET(6)]
> + stp x8, x9, [x2, #CPU_XREG_OFFSET(8)]
> + stp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
> + stp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
> + stp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
> + stp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
> + str x18, [x2, #CPU_XREG_OFFSET(18)]
> +
> + pop x6, x7 // x2, x3
> + pop x4, x5 // x0, x1
hard to review when I haven't seen the code that calls this, but I'll
assume we store things in register order on the stack.
> +
> + stp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
> + stp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
> +
> + save_common_regs x2
> +.endm
> +
> +.macro restore_guest_regs
> + // Assume vcpu in x0, clobbers everything else
nit: clobbers everything (x0 gets nuked too)
> +
> + add x2, x0, #VCPU_CONTEXT
> +
> + // Prepare x0-x3 for later restore
> + ldp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
> + ldp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
> + push x4, x5 // Push x0-x3 on the stack
> + push x6, x7
why do you need x2 and x3 later? can't you just make do with x0 and x1
and move the cpu context pointer to x1 ?
> +
> + // x4-x18
> + ldp x4, x5, [x2, #CPU_XREG_OFFSET(4)]
> + ldp x6, x7, [x2, #CPU_XREG_OFFSET(6)]
> + ldp x8, x9, [x2, #CPU_XREG_OFFSET(8)]
> + ldp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
> + ldp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
> + ldp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
> + ldp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
> + ldr x18, [x2, #CPU_XREG_OFFSET(18)]
> +
> + // x19-x29, lr
> + restore_common_regs x2
> +
> + // Last bits of the 64bit state
> + pop x2, x3
> + pop x0, x1
> +
> + // Do not touch any register after this!
> +.endm
> +
> +/*
> + * u64 __guest_enter(struct kvm_vcpu *vcpu,
> + * struct kvm_cpu_context *host_ctxt);
> + */
> +ENTRY(__guest_enter)
> + // x0: vcpu
> + // x1: host_ctxt
> + // x2, x3: parameter registers
> + // x4-x18: clobbered by macros
> +
> + save_host_regs x1
> +
> + // Preserve vcpu & host_ctxt for use at exit time
> + stp x0, x1, [sp, #-16]!
why is this not a simple push?
> +
> + restore_guest_regs
do we ever reuse any of the above macros? If not, perhaps it's more
clear to simply inline them here?
> + eret
> +ENDPROC(__guest_enter)
> +
> +ENTRY(__guest_exit)
> + // x0: vcpu
> + // x1: return code
> + // x2-x3: free
> + // x4-x29,lr: vcpu regs
> + // vcpu x0-x3 on the stack
> + save_guest_regs
> +
> + // Restore vcpu & host_ctxt from the stack
> + // (preserving return code in x1)
> + ldp x0, x2, [sp], #16
why is this not a regular pop?
> + restore_host_regs x2
> +
> + mov x0, x1
> + ret
> +ENDPROC(__guest_exit)
> +
> + /* Insert fault handling here */
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 2581232..7ac8e11 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -50,5 +50,7 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
> void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
>
> +u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
> +
> #endif /* __ARM64_KVM_HYP_H__ */
>
> --
> 2.1.4
>
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-11-27 18:50 ` [PATCH v2 10/21] arm64: KVM: Add patchable function selector Marc Zyngier
@ 2015-12-01 15:39 ` Christoffer Dall
2015-12-01 18:51 ` Marc Zyngier
2015-12-02 22:34 ` Andrew Jones
1 sibling, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 15:39 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
> KVM so far relies on code patching, and is likely to use it more
> in the future. The main issue is that our alternative system works
> at the instruction level, while we'd like to have alternatives at
> the function level.
>
> In order to cope with this, add the "hyp_alternate_select" macro that
> outputs a brief sequence of code that in turn can be patched, allowing
> al alternative function to be selected.
s/al/an/ ?
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
> 1 file changed, 16 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 7ac8e11..f0427ee 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -27,6 +27,22 @@
>
> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
>
> +/*
> + * Generates patchable code sequences that are used to switch between
> + * two implementations of a function, depending on the availability of
> + * a feature.
> + */
This looks right to me, but I'm a bit unclear what the types of this is
and how to use it.
Are orig and alt function pointers and cond is a CONFIG_FOO ? fname is
a symbol, which is defined as a prototype somewhere and then implemented
here, or?
Perhaps a Usage: part of the docs would be helpful.
> +#define hyp_alternate_select(fname, orig, alt, cond) \
> +typeof(orig) * __hyp_text fname(void) \
> +{ \
> + typeof(alt) *val = orig; \
> + asm volatile(ALTERNATIVE("nop \n", \
> + "mov %0, %1 \n", \
> + cond) \
> + : "+r" (val) : "r" (alt)); \
> + return val; \
> +}
> +
> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>
> --
> 2.1.4
>
I haven't thought much about how all of this is implemented, but from my
point of views the ideal situation would be something like:
void foo(int a, int b)
{
ALTERNATIVE_IF_NOT CONFIG_BAR
foo_legacy(a, b);
ALTERNATIVE_ELSE
foo_new(a, b);
ALTERNATIVE_END
}
I realize this may be impossible because the C code could implement all
sort of fun stuff around the actual function calls, but would there be
some way to annotate the functions and find the actual branch statement
and change the target?
Apologies if this question is just outright ridiculous.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 15:01 ` Marc Zyngier
@ 2015-12-01 15:41 ` Christoffer Dall
2015-12-01 18:34 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 15:41 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 03:01:16PM +0000, Marc Zyngier wrote:
> On 01/12/15 14:47, Christoffer Dall wrote:
> > On Tue, Dec 01, 2015 at 01:06:31PM +0000, Marc Zyngier wrote:
> >> On 01/12/15 12:56, Christoffer Dall wrote:
> >>> On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
> >>>> Implement the debug save restore as a direct translation of
> >>>> the assembly code version.
> >>>>
> >>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >>>> ---
> >>>> arch/arm64/kvm/hyp/Makefile | 1 +
> >>>> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
> >>>> arch/arm64/kvm/hyp/hyp.h | 9 +++
> >>>> 3 files changed, 140 insertions(+)
> >>>> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
> >>>>
> >>>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> >>>> index ec94200..ec14cac 100644
> >>>> --- a/arch/arm64/kvm/hyp/Makefile
> >>>> +++ b/arch/arm64/kvm/hyp/Makefile
> >>>> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> >>>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> >>>> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> >>>> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> >>>> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> >>>> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> >>>> new file mode 100644
> >>>> index 0000000..a0b2b99
> >>>> --- /dev/null
> >>>> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> >>>> @@ -0,0 +1,130 @@
> >>>> +/*
> >>>> + * Copyright (C) 2015 - ARM Ltd
> >>>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> >>>> + *
> >>>> + * This program is free software; you can redistribute it and/or modify
> >>>> + * it under the terms of the GNU General Public License version 2 as
> >>>> + * published by the Free Software Foundation.
> >>>> + *
> >>>> + * This program is distributed in the hope that it will be useful,
> >>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >>>> + * GNU General Public License for more details.
> >>>> + *
> >>>> + * You should have received a copy of the GNU General Public License
> >>>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> >>>> + */
> >>>> +
> >>>> +#include <linux/compiler.h>
> >>>> +#include <linux/kvm_host.h>
> >>>> +
> >>>> +#include <asm/kvm_mmu.h>
> >>>> +
> >>>> +#include "hyp.h"
> >>>> +
> >>>> +#define read_debug(r,n) read_sysreg(r##n##_el1)
> >>>> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
> >>>> +
> >>>> +#define save_debug(ptr,reg,nr) \
> >>>> + switch (nr) { \
> >>>> + case 15: ptr[15] = read_debug(reg, 15); \
> >>>> + case 14: ptr[14] = read_debug(reg, 14); \
> >>>> + case 13: ptr[13] = read_debug(reg, 13); \
> >>>> + case 12: ptr[12] = read_debug(reg, 12); \
> >>>> + case 11: ptr[11] = read_debug(reg, 11); \
> >>>> + case 10: ptr[10] = read_debug(reg, 10); \
> >>>> + case 9: ptr[9] = read_debug(reg, 9); \
> >>>> + case 8: ptr[8] = read_debug(reg, 8); \
> >>>> + case 7: ptr[7] = read_debug(reg, 7); \
> >>>> + case 6: ptr[6] = read_debug(reg, 6); \
> >>>> + case 5: ptr[5] = read_debug(reg, 5); \
> >>>> + case 4: ptr[4] = read_debug(reg, 4); \
> >>>> + case 3: ptr[3] = read_debug(reg, 3); \
> >>>> + case 2: ptr[2] = read_debug(reg, 2); \
> >>>> + case 1: ptr[1] = read_debug(reg, 1); \
> >>>> + default: ptr[0] = read_debug(reg, 0); \
> >>>> + }
> >>>> +
> >>>> +#define restore_debug(ptr,reg,nr) \
> >>>> + switch (nr) { \
> >>>> + case 15: write_debug(ptr[15], reg, 15); \
> >>>> + case 14: write_debug(ptr[14], reg, 14); \
> >>>> + case 13: write_debug(ptr[13], reg, 13); \
> >>>> + case 12: write_debug(ptr[12], reg, 12); \
> >>>> + case 11: write_debug(ptr[11], reg, 11); \
> >>>> + case 10: write_debug(ptr[10], reg, 10); \
> >>>> + case 9: write_debug(ptr[9], reg, 9); \
> >>>> + case 8: write_debug(ptr[8], reg, 8); \
> >>>> + case 7: write_debug(ptr[7], reg, 7); \
> >>>> + case 6: write_debug(ptr[6], reg, 6); \
> >>>> + case 5: write_debug(ptr[5], reg, 5); \
> >>>> + case 4: write_debug(ptr[4], reg, 4); \
> >>>> + case 3: write_debug(ptr[3], reg, 3); \
> >>>> + case 2: write_debug(ptr[2], reg, 2); \
> >>>> + case 1: write_debug(ptr[1], reg, 1); \
> >>>> + default: write_debug(ptr[0], reg, 0); \
> >>>> + }
> >>>> +
> >>>> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
> >>>> + struct kvm_guest_debug_arch *dbg,
> >>>> + struct kvm_cpu_context *ctxt)
> >>>> +{
> >>>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> >>>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> >>>> + int brps, wrps;
> >>>> +
> >>>> + brps = (aa64dfr0 >> 12) & 0xf;
> >>>> + wrps = (aa64dfr0 >> 20) & 0xf;
> >>>> +
> >>>> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
> >>>> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
> >>>> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
> >>>> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
> >>>> +
> >>>> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
> >>>> + }
> >>>> +}
> >>>> +
> >>>> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
> >>>> + struct kvm_guest_debug_arch *dbg,
> >>>> + struct kvm_cpu_context *ctxt)
> >>>> +{
> >>>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
> >>>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
> >>>> + int brps, wrps;
> >>>> +
> >>>> + brps = (aa64dfr0 >> 12) & 0xf;
> >>>> + wrps = (aa64dfr0 >> 20) & 0xf;
> >>>> +
> >>>> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
> >>>> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
> >>>> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
> >>>> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
> >>>> +
> >>>> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
> >>>> + }
> >>>> +}
> >>>> +
> >>>> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
> >>>> +{
> >>>> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
> >>>> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
> >>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
> >>>> +
> >>>> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
> >>>> + kern_hyp_va(vcpu->arch.host_cpu_context));
> >>>
> >>> doesn't the assmebly code jump across saving this state neither bits are
> >>> set where this always saves the state?
> >>
> >> It doesn't. The save/restore functions are guarded by tests on
> >> KVM_ARM64_DEBUG_DIRTY, just like we have skip_debug_state on all actions
> >> involving the save/restore in the assembly version.
> >
> > I think the confusing part is that the save function unconditionally
> > calls __debug_save_state where the restore function only calls it when
> > the dirty flag is set. Plus I suck at reading assembly apparently.
>
> So the way I initially wrote it, I had the same 'if' statement as in the
> restore function, making them fairly symmetric. But it quickly became
> obvious that this double-if was a bit pointless.
>
> And actually, I wonder if I shouldn't drop it from the restore function,
> because it only save us a spurious clear of the dirty bit.
>
I would just move the __debug_restore_state call above the conditional,
then they look more symmetric. Does that work?
> >>> in any case, I feel some context is lost when this is moved away from
> >>> assembly and understanding this patch would be easier if the semantics
> >>> of these two _cond functions were documented.
> >>
> >> I can migrate the existing comments if you think that helps.
> >>
> > It just wasn't not quite clear to me exactly when
> > __debug_cond_save_host_state is called for example - is this going to be
> > called unconditionally on every entry - that's how I understand it now
> > anyway.
>
> On every entry, yes. I'm trying to have the guest_run function as simple
> as possible, with the various subsystems making their 'own' decisions.
>
> Not optimal (you get to branch for nothing), but clearer. At least for
> me, but I may be the odd duck out here. Any idea to make the flow look
> clearer?
>
For me, if you always make the call unconditionally on both paths and
then change the implementations to do
if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
return;
then I think it's clear enough.
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 07/21] arm64: KVM: Implement 32bit system register save/restore
2015-11-27 18:50 ` [PATCH v2 07/21] arm64: KVM: Implement 32bit " Marc Zyngier
@ 2015-12-01 15:52 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 15:52 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:01PM +0000, Marc Zyngier wrote:
> Implement the 32bit system register save restore as a direct
> translation of the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/hyp.h | 2 ++
> arch/arm64/kvm/hyp/sysreg-sr.c | 41 +++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 43 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 087d3a5..4639330 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -38,6 +38,8 @@ void __timer_restore_state(struct kvm_vcpu *vcpu);
>
> void __sysreg_save_state(struct kvm_cpu_context *ctxt);
> void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
> +void __sysreg32_save_state(struct kvm_vcpu *vcpu);
> +void __sysreg32_restore_state(struct kvm_vcpu *vcpu);
>
> #endif /* __ARM64_KVM_HYP_H__ */
>
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index add8fcb..3f81a4d 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -88,3 +88,44 @@ void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> write_sysreg(ctxt->gp_regs.elr_el1, elr_el1);
> write_sysreg(ctxt->gp_regs.spsr[KVM_SPSR_EL1], spsr_el1);
> }
> +
> +void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> +{
> + if (!(read_sysreg(hcr_el2) & HCR_RW)) {
nit: I would probably invert the check and return early.
> + u64 *spsr = vcpu->arch.ctxt.gp_regs.spsr;
> + u64 *sysreg = vcpu->arch.ctxt.sys_regs;
> +
> + spsr[KVM_SPSR_ABT] = read_sysreg(spsr_abt);
> + spsr[KVM_SPSR_UND] = read_sysreg(spsr_und);
> + spsr[KVM_SPSR_IRQ] = read_sysreg(spsr_irq);
> + spsr[KVM_SPSR_FIQ] = read_sysreg(spsr_fiq);
> +
> + sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> + sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
> +
> + if (!(read_sysreg(cptr_el2) & CPTR_EL2_TFP))
> + sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
> +
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> + sysreg[DBGVCR32_EL2] = read_sysreg(dbgvcr32_el2);
> + }
> +}
> +
> +void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu)
> +{
> + if (!(read_sysreg(hcr_el2) & HCR_RW)) {
same here
> + u64 *spsr = vcpu->arch.ctxt.gp_regs.spsr;
> + u64 *sysreg = vcpu->arch.ctxt.sys_regs;
> +
> + write_sysreg(spsr[KVM_SPSR_ABT], spsr_abt);
> + write_sysreg(spsr[KVM_SPSR_UND], spsr_und);
> + write_sysreg(spsr[KVM_SPSR_IRQ], spsr_irq);
> + write_sysreg(spsr[KVM_SPSR_FIQ], spsr_fiq);
> +
nit: white space
> + write_sysreg(sysreg[DACR32_EL2], dacr32_el2);
> + write_sysreg(sysreg[IFSR32_EL2], ifsr32_el2);
> +
> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> + write_sysreg(sysreg[DBGVCR32_EL2], dbgvcr32_el2);
> + }
> +}
> --
> 2.1.4
>
Otherwise:
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 06/21] arm64: KVM: Implement system register save/restore
2015-11-27 18:50 ` [PATCH v2 06/21] arm64: KVM: Implement system register save/restore Marc Zyngier
@ 2015-12-01 15:53 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 15:53 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:00PM +0000, Marc Zyngier wrote:
> Implement the system registe save restore as a direct translation of
nit: s/registe/register/
> the assembly code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/hyp.h | 3 ++
> arch/arm64/kvm/hyp/sysreg-sr.c | 90 ++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 94 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/sysreg-sr.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index 455dc0a..ec94200 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -5,3 +5,4 @@
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> +obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 86aa5a2..087d3a5 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -36,5 +36,8 @@ void __vgic_v3_restore_state(struct kvm_vcpu *vcpu);
> void __timer_save_state(struct kvm_vcpu *vcpu);
> void __timer_restore_state(struct kvm_vcpu *vcpu);
>
> +void __sysreg_save_state(struct kvm_cpu_context *ctxt);
> +void __sysreg_restore_state(struct kvm_cpu_context *ctxt);
> +
> #endif /* __ARM64_KVM_HYP_H__ */
>
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> new file mode 100644
> index 0000000..add8fcb
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -0,0 +1,90 @@
> +/*
> + * Copyright (C) 2012-2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/compiler.h>
> +#include <linux/kvm_host.h>
> +
> +#include <asm/kvm_mmu.h>
> +
> +#include "hyp.h"
> +
> +/* ctxt is already in the HYP VA space */
> +void __hyp_text __sysreg_save_state(struct kvm_cpu_context *ctxt)
> +{
> + ctxt->sys_regs[MPIDR_EL1] = read_sysreg(vmpidr_el2);
> + ctxt->sys_regs[CSSELR_EL1] = read_sysreg(csselr_el1);
> + ctxt->sys_regs[SCTLR_EL1] = read_sysreg(sctlr_el1);
> + ctxt->sys_regs[ACTLR_EL1] = read_sysreg(actlr_el1);
> + ctxt->sys_regs[CPACR_EL1] = read_sysreg(cpacr_el1);
> + ctxt->sys_regs[TTBR0_EL1] = read_sysreg(ttbr0_el1);
> + ctxt->sys_regs[TTBR1_EL1] = read_sysreg(ttbr1_el1);
> + ctxt->sys_regs[TCR_EL1] = read_sysreg(tcr_el1);
> + ctxt->sys_regs[ESR_EL1] = read_sysreg(esr_el1);
> + ctxt->sys_regs[AFSR0_EL1] = read_sysreg(afsr0_el1);
> + ctxt->sys_regs[AFSR1_EL1] = read_sysreg(afsr1_el1);
> + ctxt->sys_regs[FAR_EL1] = read_sysreg(far_el1);
> + ctxt->sys_regs[MAIR_EL1] = read_sysreg(mair_el1);
> + ctxt->sys_regs[VBAR_EL1] = read_sysreg(vbar_el1);
> + ctxt->sys_regs[CONTEXTIDR_EL1] = read_sysreg(contextidr_el1);
> + ctxt->sys_regs[TPIDR_EL0] = read_sysreg(tpidr_el0);
> + ctxt->sys_regs[TPIDRRO_EL0] = read_sysreg(tpidrro_el0);
> + ctxt->sys_regs[TPIDR_EL1] = read_sysreg(tpidr_el1);
> + ctxt->sys_regs[AMAIR_EL1] = read_sysreg(amair_el1);
> + ctxt->sys_regs[CNTKCTL_EL1] = read_sysreg(cntkctl_el1);
> + ctxt->sys_regs[PAR_EL1] = read_sysreg(par_el1);
> + ctxt->sys_regs[MDSCR_EL1] = read_sysreg(mdscr_el1);
> +
> + ctxt->gp_regs.regs.sp = read_sysreg(sp_el0);
> + ctxt->gp_regs.regs.pc = read_sysreg(elr_el2);
> + ctxt->gp_regs.regs.pstate = read_sysreg(spsr_el2);
> + ctxt->gp_regs.sp_el1 = read_sysreg(sp_el1);
> + ctxt->gp_regs.elr_el1 = read_sysreg(elr_el1);
> + ctxt->gp_regs.spsr[KVM_SPSR_EL1]= read_sysreg(spsr_el1);
> +}
> +
> +void __hyp_text __sysreg_restore_state(struct kvm_cpu_context *ctxt)
> +{
> + write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2);
> + write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1);
> + write_sysreg(ctxt->sys_regs[SCTLR_EL1], sctlr_el1);
> + write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1);
> + write_sysreg(ctxt->sys_regs[CPACR_EL1], cpacr_el1);
> + write_sysreg(ctxt->sys_regs[TTBR0_EL1], ttbr0_el1);
> + write_sysreg(ctxt->sys_regs[TTBR1_EL1], ttbr1_el1);
> + write_sysreg(ctxt->sys_regs[TCR_EL1], tcr_el1);
> + write_sysreg(ctxt->sys_regs[ESR_EL1], esr_el1);
> + write_sysreg(ctxt->sys_regs[AFSR0_EL1], afsr0_el1);
> + write_sysreg(ctxt->sys_regs[AFSR1_EL1], afsr1_el1);
> + write_sysreg(ctxt->sys_regs[FAR_EL1], far_el1);
> + write_sysreg(ctxt->sys_regs[MAIR_EL1], mair_el1);
> + write_sysreg(ctxt->sys_regs[VBAR_EL1], vbar_el1);
> + write_sysreg(ctxt->sys_regs[CONTEXTIDR_EL1], contextidr_el1);
> + write_sysreg(ctxt->sys_regs[TPIDR_EL0], tpidr_el0);
> + write_sysreg(ctxt->sys_regs[TPIDRRO_EL0], tpidrro_el0);
> + write_sysreg(ctxt->sys_regs[TPIDR_EL1], tpidr_el1);
> + write_sysreg(ctxt->sys_regs[AMAIR_EL1], amair_el1);
> + write_sysreg(ctxt->sys_regs[CNTKCTL_EL1], cntkctl_el1);
> + write_sysreg(ctxt->sys_regs[PAR_EL1], par_el1);
> + write_sysreg(ctxt->sys_regs[MDSCR_EL1], mdscr_el1);
> +
> + write_sysreg(ctxt->gp_regs.regs.sp, sp_el0);
> + write_sysreg(ctxt->gp_regs.regs.pc, elr_el2);
> + write_sysreg(ctxt->gp_regs.regs.pstate, spsr_el2);
> + write_sysreg(ctxt->gp_regs.sp_el1, sp_el1);
> + write_sysreg(ctxt->gp_regs.elr_el1, elr_el1);
> + write_sysreg(ctxt->gp_regs.spsr[KVM_SPSR_EL1], spsr_el1);
> +}
> --
> 2.1.4
>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 11/21] arm64: KVM: Implement the core world switch
2015-11-27 18:50 ` [PATCH v2 11/21] arm64: KVM: Implement the core world switch Marc Zyngier
@ 2015-12-01 15:55 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 15:55 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:05PM +0000, Marc Zyngier wrote:
> Implement the core of the world switch in C. Not everything is there
> yet, and there is nothing to re-enter the world switch either.
>
> But this already outlines the code structure well enough.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/switch.c | 134 ++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 135 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/switch.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index 1e1ff06..9c11b0f 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -8,3 +8,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += entry.o
> +obj-$(CONFIG_KVM_ARM_HOST) += switch.o
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> new file mode 100644
> index 0000000..d67ed9e
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -0,0 +1,134 @@
> +/*
> + * Copyright (C) 2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include "hyp.h"
> +
> +static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> +{
> + u64 val;
> +
> + /*
> + * We are about to set CPTR_EL2.TFP to trap all floating point
> + * register accesses to EL2, however, the ARM ARM clearly states that
> + * traps are only taken to EL2 if the operation would not otherwise
> + * trap to EL1. Therefore, always make sure that for 32-bit guests,
> + * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit.
> + */
> + val = vcpu->arch.hcr_el2;
> + if (!(val & HCR_RW)) {
> + write_sysreg(1 << 30, fpexc32_el2);
> + isb();
> + }
> + write_sysreg(val, hcr_el2);
> + write_sysreg(1 << 15, hstr_el2);
can you add a comment on this 1 << 15 stuff or use a define or something
to remind mushy-brained people like myself that this is about trapping
something we care about?
> + write_sysreg(CPTR_EL2_TTA | CPTR_EL2_TFP, cptr_el2);
> + write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2);
> +}
> +
> +static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)
> +{
> + write_sysreg(HCR_RW, hcr_el2);
> + write_sysreg(0, hstr_el2);
> + write_sysreg(read_sysreg(mdcr_el2) & MDCR_EL2_HPMN_MASK, mdcr_el2);
> + write_sysreg(0, cptr_el2);
> +}
> +
> +static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu)
> +{
> + struct kvm *kvm = kern_hyp_va(vcpu->kvm);
> + write_sysreg(kvm->arch.vttbr, vttbr_el2);
> +}
> +
> +static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu)
> +{
> + write_sysreg(0, vttbr_el2);
> +}
> +
> +static hyp_alternate_select(__vgic_call_save_state,
> + __vgic_v2_save_state, __vgic_v3_save_state,
> + ARM64_HAS_SYSREG_GIC_CPUIF);
> +
> +static hyp_alternate_select(__vgic_call_restore_state,
> + __vgic_v2_restore_state, __vgic_v3_restore_state,
> + ARM64_HAS_SYSREG_GIC_CPUIF);
> +
> +static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu)
> +{
> + __vgic_call_save_state()(vcpu);
> + write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2);
> +}
> +
> +static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> +{
> + u64 val;
> +
> + val = read_sysreg(hcr_el2);
> + val |= HCR_INT_OVERRIDE;
> + val |= vcpu->arch.irq_lines;
> + write_sysreg(val, hcr_el2);
> +
> + __vgic_call_restore_state()(vcpu);
> +}
> +
> +int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> +{
> + struct kvm_cpu_context *host_ctxt;
> + struct kvm_cpu_context *guest_ctxt;
> + u64 exit_code;
> +
> + vcpu = kern_hyp_va(vcpu);
> + write_sysreg(vcpu, tpidr_el2);
> +
> + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + guest_ctxt = &vcpu->arch.ctxt;
> +
> + __sysreg_save_state(host_ctxt);
> + __debug_cond_save_host_state(vcpu);
> +
> + __activate_traps(vcpu);
> + __activate_vm(vcpu);
> +
> + __vgic_restore_state(vcpu);
> + __timer_restore_state(vcpu);
> +
> + /*
> + * We must restore the 32-bit state before the sysregs, thanks
> + * to Cortex-A57 erratum #852523.
> + */
> + __sysreg32_restore_state(vcpu);
> + __sysreg_restore_state(guest_ctxt);
> + __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> +
> + /* Jump in the fire! */
> + exit_code = __guest_enter(vcpu, host_ctxt);
> + /* And we're baaack! */
> +
> + __sysreg_save_state(guest_ctxt);
> + __sysreg32_save_state(vcpu);
> + __timer_save_state(vcpu);
> + __vgic_save_state(vcpu);
> +
> + __deactivate_traps(vcpu);
> + __deactivate_vm(vcpu);
> +
> + __sysreg_restore_state(host_ctxt);
> +
> + __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> + __debug_cond_restore_host_state(vcpu);
> +
> + return exit_code;
> +}
> --
> 2.1.4
>
Looks very nice!
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 00/21] arm64: KVM: world switch in C
2015-12-01 12:00 ` Christoffer Dall
@ 2015-12-01 17:51 ` Marc Zyngier
2015-12-01 19:34 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 17:51 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 12:00, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 09:58:23AM +0000, Marc Zyngier wrote:
>> On 30/11/15 20:33, Christoffer Dall wrote:
>>> On Fri, Nov 27, 2015 at 06:49:54PM +0000, Marc Zyngier wrote:
>>>> Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
>>>> and mean piece of hand-crafted assembly code. Over time, features have
>>>> crept in, the code has become harder to maintain, and the smallest
>>>> change is a pain to introduce. The VHE patches are a prime example of
>>>> why this doesn't work anymore.
>>>>
>>>> This series rewrites most of the existing assembly code in C, but keeps
>>>> the existing code structure in place (most function names will look
>>>> familiar to the reader). The biggest change is that we don't have to
>>>> deal with a static register allocation (the compiler does it for us),
>>>> we can easily follow structure and pointers, and only the lowest level
>>>> is still in assembly code. Oh, and a negative diffstat.
>>>>
>>>> There is still a healthy dose of inline assembly (system register
>>>> accessors, runtime code patching), but I've tried not to make it too
>>>> invasive. The generated code, while not exactly brilliant, doesn't
>>>> look too shaby. I do expect a small performance degradation, but I
>>>> believe this is something we can improve over time (my initial
>>>> measurements don't show any obvious regression though).
>>>
>>> I ran this through my experimental setup on m400 and got this:
>>
>> [...]
>>
>>> What this tells me is that we do take a noticable hit on the
>>> world-switch path, which shows up in the TCP_RR and hackbench workloads,
>>> which have a high precision in their output.
>>>
>>> Note that the memcached number is well within its variability between
>>> individual benchmark runs, where it varies to 12% of its average in over
>>> 80% of the executions.
>>>
>>> I don't think this is a showstopper thought, but we could consider
>>> looking more closely at a breakdown of the world-switch path and verify
>>> if/where we are really taking a hit.
>>
>> Thanks for doing so, very interesting. As a data point, what compiler
>> are you using? I'd expect some variability based on the compiler version...
>>
> I used the following (compiling natively on the m400):
>
> gcc version 4.8.2 (Ubuntu/Linaro 4.8.2-19ubuntu1)
For what it is worth, I've ran hackbench on my Seattle B0 (8xA57 2GHz),
with a 4 vcpu VM and got the following results (10 runs per kernel
version, same configuration):
v4.4-rc3-wsinc: Average 31.750
32.459
32.124
32.435
31.940
31.085
31.804
31.862
30.985
31.450
31.359
v4.4-rc3: Average 31.954
31.806
31.598
32.697
31.472
31.410
32.562
31.938
31.932
31.672
32.459
This is with GCC as produced by Linaro:
aarch64-linux-gnu-gcc (Linaro GCC 5.1-2015.08) 5.1.1 20150608
It could well be that your compiler generates worse code than the one I
use, or that the code it outputs is badly tuned for XGene. I guess I
need to unearth my Mustang to find out...
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 08/21] arm64: KVM: Implement debug save/restore
2015-12-01 15:41 ` Christoffer Dall
@ 2015-12-01 18:34 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 18:34 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 15:41, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 03:01:16PM +0000, Marc Zyngier wrote:
>> On 01/12/15 14:47, Christoffer Dall wrote:
>>> On Tue, Dec 01, 2015 at 01:06:31PM +0000, Marc Zyngier wrote:
>>>> On 01/12/15 12:56, Christoffer Dall wrote:
>>>>> On Fri, Nov 27, 2015 at 06:50:02PM +0000, Marc Zyngier wrote:
>>>>>> Implement the debug save restore as a direct translation of
>>>>>> the assembly code version.
>>>>>>
>>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>>>> ---
>>>>>> arch/arm64/kvm/hyp/Makefile | 1 +
>>>>>> arch/arm64/kvm/hyp/debug-sr.c | 130 ++++++++++++++++++++++++++++++++++++++++++
>>>>>> arch/arm64/kvm/hyp/hyp.h | 9 +++
>>>>>> 3 files changed, 140 insertions(+)
>>>>>> create mode 100644 arch/arm64/kvm/hyp/debug-sr.c
>>>>>>
>>>>>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>>>>>> index ec94200..ec14cac 100644
>>>>>> --- a/arch/arm64/kvm/hyp/Makefile
>>>>>> +++ b/arch/arm64/kvm/hyp/Makefile
>>>>>> @@ -6,3 +6,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v2-sr.o
>>>>>> obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>>>>>> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
>>>>>> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
>>>>>> +obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
>>>>>> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
>>>>>> new file mode 100644
>>>>>> index 0000000..a0b2b99
>>>>>> --- /dev/null
>>>>>> +++ b/arch/arm64/kvm/hyp/debug-sr.c
>>>>>> @@ -0,0 +1,130 @@
>>>>>> +/*
>>>>>> + * Copyright (C) 2015 - ARM Ltd
>>>>>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>>>>>> + *
>>>>>> + * This program is free software; you can redistribute it and/or modify
>>>>>> + * it under the terms of the GNU General Public License version 2 as
>>>>>> + * published by the Free Software Foundation.
>>>>>> + *
>>>>>> + * This program is distributed in the hope that it will be useful,
>>>>>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>>>>>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>>>>>> + * GNU General Public License for more details.
>>>>>> + *
>>>>>> + * You should have received a copy of the GNU General Public License
>>>>>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>>>>>> + */
>>>>>> +
>>>>>> +#include <linux/compiler.h>
>>>>>> +#include <linux/kvm_host.h>
>>>>>> +
>>>>>> +#include <asm/kvm_mmu.h>
>>>>>> +
>>>>>> +#include "hyp.h"
>>>>>> +
>>>>>> +#define read_debug(r,n) read_sysreg(r##n##_el1)
>>>>>> +#define write_debug(v,r,n) write_sysreg(v, r##n##_el1)
>>>>>> +
>>>>>> +#define save_debug(ptr,reg,nr) \
>>>>>> + switch (nr) { \
>>>>>> + case 15: ptr[15] = read_debug(reg, 15); \
>>>>>> + case 14: ptr[14] = read_debug(reg, 14); \
>>>>>> + case 13: ptr[13] = read_debug(reg, 13); \
>>>>>> + case 12: ptr[12] = read_debug(reg, 12); \
>>>>>> + case 11: ptr[11] = read_debug(reg, 11); \
>>>>>> + case 10: ptr[10] = read_debug(reg, 10); \
>>>>>> + case 9: ptr[9] = read_debug(reg, 9); \
>>>>>> + case 8: ptr[8] = read_debug(reg, 8); \
>>>>>> + case 7: ptr[7] = read_debug(reg, 7); \
>>>>>> + case 6: ptr[6] = read_debug(reg, 6); \
>>>>>> + case 5: ptr[5] = read_debug(reg, 5); \
>>>>>> + case 4: ptr[4] = read_debug(reg, 4); \
>>>>>> + case 3: ptr[3] = read_debug(reg, 3); \
>>>>>> + case 2: ptr[2] = read_debug(reg, 2); \
>>>>>> + case 1: ptr[1] = read_debug(reg, 1); \
>>>>>> + default: ptr[0] = read_debug(reg, 0); \
>>>>>> + }
>>>>>> +
>>>>>> +#define restore_debug(ptr,reg,nr) \
>>>>>> + switch (nr) { \
>>>>>> + case 15: write_debug(ptr[15], reg, 15); \
>>>>>> + case 14: write_debug(ptr[14], reg, 14); \
>>>>>> + case 13: write_debug(ptr[13], reg, 13); \
>>>>>> + case 12: write_debug(ptr[12], reg, 12); \
>>>>>> + case 11: write_debug(ptr[11], reg, 11); \
>>>>>> + case 10: write_debug(ptr[10], reg, 10); \
>>>>>> + case 9: write_debug(ptr[9], reg, 9); \
>>>>>> + case 8: write_debug(ptr[8], reg, 8); \
>>>>>> + case 7: write_debug(ptr[7], reg, 7); \
>>>>>> + case 6: write_debug(ptr[6], reg, 6); \
>>>>>> + case 5: write_debug(ptr[5], reg, 5); \
>>>>>> + case 4: write_debug(ptr[4], reg, 4); \
>>>>>> + case 3: write_debug(ptr[3], reg, 3); \
>>>>>> + case 2: write_debug(ptr[2], reg, 2); \
>>>>>> + case 1: write_debug(ptr[1], reg, 1); \
>>>>>> + default: write_debug(ptr[0], reg, 0); \
>>>>>> + }
>>>>>> +
>>>>>> +void __hyp_text __debug_save_state(struct kvm_vcpu *vcpu,
>>>>>> + struct kvm_guest_debug_arch *dbg,
>>>>>> + struct kvm_cpu_context *ctxt)
>>>>>> +{
>>>>>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
>>>>>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>>>>>> + int brps, wrps;
>>>>>> +
>>>>>> + brps = (aa64dfr0 >> 12) & 0xf;
>>>>>> + wrps = (aa64dfr0 >> 20) & 0xf;
>>>>>> +
>>>>>> + save_debug(dbg->dbg_bcr, dbgbcr, brps);
>>>>>> + save_debug(dbg->dbg_bvr, dbgbvr, brps);
>>>>>> + save_debug(dbg->dbg_wcr, dbgwcr, wrps);
>>>>>> + save_debug(dbg->dbg_wvr, dbgwvr, wrps);
>>>>>> +
>>>>>> + ctxt->sys_regs[MDCCINT_EL1] = read_sysreg(mdccint_el1);
>>>>>> + }
>>>>>> +}
>>>>>> +
>>>>>> +void __hyp_text __debug_restore_state(struct kvm_vcpu *vcpu,
>>>>>> + struct kvm_guest_debug_arch *dbg,
>>>>>> + struct kvm_cpu_context *ctxt)
>>>>>> +{
>>>>>> + if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY) {
>>>>>> + u64 aa64dfr0 = read_sysreg(id_aa64dfr0_el1);
>>>>>> + int brps, wrps;
>>>>>> +
>>>>>> + brps = (aa64dfr0 >> 12) & 0xf;
>>>>>> + wrps = (aa64dfr0 >> 20) & 0xf;
>>>>>> +
>>>>>> + restore_debug(dbg->dbg_bcr, dbgbcr, brps);
>>>>>> + restore_debug(dbg->dbg_bvr, dbgbvr, brps);
>>>>>> + restore_debug(dbg->dbg_wcr, dbgwcr, wrps);
>>>>>> + restore_debug(dbg->dbg_wvr, dbgwvr, wrps);
>>>>>> +
>>>>>> + write_sysreg(ctxt->sys_regs[MDCCINT_EL1], mdccint_el1);
>>>>>> + }
>>>>>> +}
>>>>>> +
>>>>>> +void __hyp_text __debug_cond_save_host_state(struct kvm_vcpu *vcpu)
>>>>>> +{
>>>>>> + if ((vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE) ||
>>>>>> + (vcpu->arch.ctxt.sys_regs[MDSCR_EL1] & DBG_MDSCR_KDE))
>>>>>> + vcpu->arch.debug_flags |= KVM_ARM64_DEBUG_DIRTY;
>>>>>> +
>>>>>> + __debug_save_state(vcpu, &vcpu->arch.host_debug_state,
>>>>>> + kern_hyp_va(vcpu->arch.host_cpu_context));
>>>>>
>>>>> doesn't the assmebly code jump across saving this state neither bits are
>>>>> set where this always saves the state?
>>>>
>>>> It doesn't. The save/restore functions are guarded by tests on
>>>> KVM_ARM64_DEBUG_DIRTY, just like we have skip_debug_state on all actions
>>>> involving the save/restore in the assembly version.
>>>
>>> I think the confusing part is that the save function unconditionally
>>> calls __debug_save_state where the restore function only calls it when
>>> the dirty flag is set. Plus I suck at reading assembly apparently.
>>
>> So the way I initially wrote it, I had the same 'if' statement as in the
>> restore function, making them fairly symmetric. But it quickly became
>> obvious that this double-if was a bit pointless.
>>
>> And actually, I wonder if I shouldn't drop it from the restore function,
>> because it only save us a spurious clear of the dirty bit.
>>
>
> I would just move the __debug_restore_state call above the conditional,
> then they look more symmetric. Does that work?
>
>>>>> in any case, I feel some context is lost when this is moved away from
>>>>> assembly and understanding this patch would be easier if the semantics
>>>>> of these two _cond functions were documented.
>>>>
>>>> I can migrate the existing comments if you think that helps.
>>>>
>>> It just wasn't not quite clear to me exactly when
>>> __debug_cond_save_host_state is called for example - is this going to be
>>> called unconditionally on every entry - that's how I understand it now
>>> anyway.
>>
>> On every entry, yes. I'm trying to have the guest_run function as simple
>> as possible, with the various subsystems making their 'own' decisions.
>>
>> Not optimal (you get to branch for nothing), but clearer. At least for
>> me, but I may be the odd duck out here. Any idea to make the flow look
>> clearer?
>>
>
> For me, if you always make the call unconditionally on both paths and
> then change the implementations to do
>
> if (!(vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY))
> return;
>
> then I think it's clear enough.
Fair enough. I'll rework it that way.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 09/21] arm64: KVM: Implement guest entry
2015-12-01 15:29 ` Christoffer Dall
@ 2015-12-01 18:41 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 18:41 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 15:29, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:50:03PM +0000, Marc Zyngier wrote:
>> Contrary to the previous patch, the guest entry is fairly different
>> from its assembly counterpart, mostly because it is only concerned
>> with saving/restoring the GP registers, and nothing else.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/hyp/Makefile | 1 +
>> arch/arm64/kvm/hyp/entry.S | 155 ++++++++++++++++++++++++++++++++++++++++++++
>> arch/arm64/kvm/hyp/hyp.h | 2 +
>> 3 files changed, 158 insertions(+)
>> create mode 100644 arch/arm64/kvm/hyp/entry.S
>>
>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>> index ec14cac..1e1ff06 100644
>> --- a/arch/arm64/kvm/hyp/Makefile
>> +++ b/arch/arm64/kvm/hyp/Makefile
>> @@ -7,3 +7,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += vgic-v3-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += entry.o
>> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
>> new file mode 100644
>> index 0000000..2c4449a
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/entry.S
>> @@ -0,0 +1,155 @@
>> +/*
>> + * Copyright (C) 2015 - ARM Ltd
>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <linux/linkage.h>
>> +
>> +#include <asm/asm-offsets.h>
>> +#include <asm/assembler.h>
>> +#include <asm/fpsimdmacros.h>
>> +#include <asm/kvm.h>
>> +#include <asm/kvm_arm.h>
>> +#include <asm/kvm_asm.h>
>> +#include <asm/kvm_mmu.h>
>> +
>> +#define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
>> +#define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
>> +
>> + .text
>> + .pushsection .hyp.text, "ax"
>> +
>> +.macro save_common_regs ctxt
>> + stp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
>> + stp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
>> + stp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
>> + stp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
>> + stp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
>> + stp x29, lr, [\ctxt, #CPU_XREG_OFFSET(29)]
>> +.endm
>> +
>> +.macro restore_common_regs ctxt
>> + ldp x19, x20, [\ctxt, #CPU_XREG_OFFSET(19)]
>> + ldp x21, x22, [\ctxt, #CPU_XREG_OFFSET(21)]
>> + ldp x23, x24, [\ctxt, #CPU_XREG_OFFSET(23)]
>> + ldp x25, x26, [\ctxt, #CPU_XREG_OFFSET(25)]
>> + ldp x27, x28, [\ctxt, #CPU_XREG_OFFSET(27)]
>> + ldp x29, lr, [\ctxt, #CPU_XREG_OFFSET(29)]
>> +.endm
>> +
>> +.macro save_host_regs reg
>> + save_common_regs \reg
>> +.endm
>> +
>> +.macro restore_host_regs reg
>> + restore_common_regs \reg
>> +.endm
>> +
>> +.macro save_guest_regs
>> + // x0 is the vcpu address
>> + // x1 is the return code, do not corrupt!
>> + // x2 is the cpu context
>
> this is confusing because the caller says x2 is free, so are these the
> inputs or invariants preserved in the function, or?
>
> note that you'll avoid this kind of confusion by inlining this stuff in
> __guest_exit.
Indeed. I might just do that.
>> + // x3 is a tmp register
>> + // Guest's x0-x3 are on the stack
>> +
>> + add x2, x0, #VCPU_CONTEXT
>> +
>> + // Compute base to save registers
>
> misleading comment?
Of course. Isn't that the very purpose of a comment? I'm confused... ;-)
>> + stp x4, x5, [x2, #CPU_XREG_OFFSET(4)]
>> + stp x6, x7, [x2, #CPU_XREG_OFFSET(6)]
>> + stp x8, x9, [x2, #CPU_XREG_OFFSET(8)]
>> + stp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
>> + stp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
>> + stp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
>> + stp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
>> + str x18, [x2, #CPU_XREG_OFFSET(18)]
>> +
>> + pop x6, x7 // x2, x3
>> + pop x4, x5 // x0, x1
>
> hard to review when I haven't seen the code that calls this, but I'll
> assume we store things in register order on the stack.
Indeed. I've basically lifted that code from the previous version, so
some things may have stuck...
>> +
>> + stp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
>> + stp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
>> +
>> + save_common_regs x2
>> +.endm
>> +
>> +.macro restore_guest_regs
>> + // Assume vcpu in x0, clobbers everything else
>
> nit: clobbers everything (x0 gets nuked too)
>
>> +
>> + add x2, x0, #VCPU_CONTEXT
>> +
>> + // Prepare x0-x3 for later restore
>> + ldp x4, x5, [x2, #CPU_XREG_OFFSET(0)]
>> + ldp x6, x7, [x2, #CPU_XREG_OFFSET(2)]
>> + push x4, x5 // Push x0-x3 on the stack
>> + push x6, x7
>
> why do you need x2 and x3 later? can't you just make do with x0 and x1
> and move the cpu context pointer to x1 ?
Maybe I can optimize this a bit indeed.
>> +
>> + // x4-x18
>> + ldp x4, x5, [x2, #CPU_XREG_OFFSET(4)]
>> + ldp x6, x7, [x2, #CPU_XREG_OFFSET(6)]
>> + ldp x8, x9, [x2, #CPU_XREG_OFFSET(8)]
>> + ldp x10, x11, [x2, #CPU_XREG_OFFSET(10)]
>> + ldp x12, x13, [x2, #CPU_XREG_OFFSET(12)]
>> + ldp x14, x15, [x2, #CPU_XREG_OFFSET(14)]
>> + ldp x16, x17, [x2, #CPU_XREG_OFFSET(16)]
>> + ldr x18, [x2, #CPU_XREG_OFFSET(18)]
>> +
>> + // x19-x29, lr
>> + restore_common_regs x2
>> +
>> + // Last bits of the 64bit state
>> + pop x2, x3
>> + pop x0, x1
>> +
>> + // Do not touch any register after this!
>> +.endm
>> +
>> +/*
>> + * u64 __guest_enter(struct kvm_vcpu *vcpu,
>> + * struct kvm_cpu_context *host_ctxt);
>> + */
>> +ENTRY(__guest_enter)
>> + // x0: vcpu
>> + // x1: host_ctxt
>> + // x2, x3: parameter registers
>> + // x4-x18: clobbered by macros
>> +
>> + save_host_regs x1
>> +
>> + // Preserve vcpu & host_ctxt for use at exit time
>> + stp x0, x1, [sp, #-16]!
>
> why is this not a simple push?
This *is* a simple push. Catalin has been threatening me to remove
push/pop from the convenience macros, and I've started toying with the
idea. Maybe I'll should bite the bullet and convert them all.
>
>> +
>> + restore_guest_regs
>
> do we ever reuse any of the above macros? If not, perhaps it's more
> clear to simply inline them here?
Yeah, that's clearly better.
>> + eret
>> +ENDPROC(__guest_enter)
>> +
>> +ENTRY(__guest_exit)
>> + // x0: vcpu
>> + // x1: return code
>> + // x2-x3: free
>> + // x4-x29,lr: vcpu regs
>> + // vcpu x0-x3 on the stack
>> + save_guest_regs
>> +
>> + // Restore vcpu & host_ctxt from the stack
>> + // (preserving return code in x1)
>> + ldp x0, x2, [sp], #16
>
> why is this not a regular pop?
See above.
>> + restore_host_regs x2
>> +
>> + mov x0, x1
>> + ret
>> +ENDPROC(__guest_exit)
>> +
>> + /* Insert fault handling here */
>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>> index 2581232..7ac8e11 100644
>> --- a/arch/arm64/kvm/hyp/hyp.h
>> +++ b/arch/arm64/kvm/hyp/hyp.h
>> @@ -50,5 +50,7 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
>> void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
>> void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
>>
>> +u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
>> +
>> #endif /* __ARM64_KVM_HYP_H__ */
>>
>> --
>> 2.1.4
>>
>
> Thanks,
> -Christoffer
>
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-12-01 15:39 ` Christoffer Dall
@ 2015-12-01 18:51 ` Marc Zyngier
2015-12-02 9:27 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-01 18:51 UTC (permalink / raw)
To: linux-arm-kernel
On 01/12/15 15:39, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
>> KVM so far relies on code patching, and is likely to use it more
>> in the future. The main issue is that our alternative system works
>> at the instruction level, while we'd like to have alternatives at
>> the function level.
>>
>> In order to cope with this, add the "hyp_alternate_select" macro that
>> outputs a brief sequence of code that in turn can be patched, allowing
>> al alternative function to be selected.
>
> s/al/an/ ?
>
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
>> 1 file changed, 16 insertions(+)
>>
>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>> index 7ac8e11..f0427ee 100644
>> --- a/arch/arm64/kvm/hyp/hyp.h
>> +++ b/arch/arm64/kvm/hyp/hyp.h
>> @@ -27,6 +27,22 @@
>>
>> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
>>
>> +/*
>> + * Generates patchable code sequences that are used to switch between
>> + * two implementations of a function, depending on the availability of
>> + * a feature.
>> + */
>
> This looks right to me, but I'm a bit unclear what the types of this is
> and how to use it.
>
> Are orig and alt function pointers and cond is a CONFIG_FOO ? fname is
> a symbol, which is defined as a prototype somewhere and then implemented
> here, or?
>
> Perhaps a Usage: part of the docs would be helpful.
How about:
@fname: a symbol name that will be defined as a function returning a
function pointer whose type will match @orig and @alt
@orig: A pointer to the default function, as returned by @fname when
@cond doesn't hold
@alt: A pointer to the alternate function, as returned by @fname when
@cond holds
@cond: a CPU feature (as described in asm/cpufeature.h)
>
>> +#define hyp_alternate_select(fname, orig, alt, cond) \
>> +typeof(orig) * __hyp_text fname(void) \
>> +{ \
>> + typeof(alt) *val = orig; \
>> + asm volatile(ALTERNATIVE("nop \n", \
>> + "mov %0, %1 \n", \
>> + cond) \
>> + : "+r" (val) : "r" (alt)); \
>> + return val; \
>> +}
>> +
>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>
>> --
>> 2.1.4
>>
>
> I haven't thought much about how all of this is implemented, but from my
> point of views the ideal situation would be something like:
>
> void foo(int a, int b)
> {
> ALTERNATIVE_IF_NOT CONFIG_BAR
> foo_legacy(a, b);
> ALTERNATIVE_ELSE
> foo_new(a, b);
> ALTERNATIVE_END
> }
>
> I realize this may be impossible because the C code could implement all
> sort of fun stuff around the actual function calls, but would there be
> some way to annotate the functions and find the actual branch statement
> and change the target?
The main issue is that C doesn't give you any access to the branch
function itself, except for the asm-goto statements. It also makes it
very hard to preserve the return type. For your idea to work, we'd need
some support in the compiler itself. I'm sure that it is doable, just
not by me! ;-)
This is why I've ended up creating something that returns a function
*pointer*, because that's something that exists in the language (no new
concept). I simply made sure I could return it at minimal cost.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 00/21] arm64: KVM: world switch in C
2015-12-01 17:51 ` Marc Zyngier
@ 2015-12-01 19:34 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-01 19:34 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 05:51:46PM +0000, Marc Zyngier wrote:
> On 01/12/15 12:00, Christoffer Dall wrote:
> > On Tue, Dec 01, 2015 at 09:58:23AM +0000, Marc Zyngier wrote:
> >> On 30/11/15 20:33, Christoffer Dall wrote:
> >>> On Fri, Nov 27, 2015 at 06:49:54PM +0000, Marc Zyngier wrote:
> >>>> Once upon a time, the KVM/arm64 world switch was a nice, clean, lean
> >>>> and mean piece of hand-crafted assembly code. Over time, features have
> >>>> crept in, the code has become harder to maintain, and the smallest
> >>>> change is a pain to introduce. The VHE patches are a prime example of
> >>>> why this doesn't work anymore.
> >>>>
> >>>> This series rewrites most of the existing assembly code in C, but keeps
> >>>> the existing code structure in place (most function names will look
> >>>> familiar to the reader). The biggest change is that we don't have to
> >>>> deal with a static register allocation (the compiler does it for us),
> >>>> we can easily follow structure and pointers, and only the lowest level
> >>>> is still in assembly code. Oh, and a negative diffstat.
> >>>>
> >>>> There is still a healthy dose of inline assembly (system register
> >>>> accessors, runtime code patching), but I've tried not to make it too
> >>>> invasive. The generated code, while not exactly brilliant, doesn't
> >>>> look too shaby. I do expect a small performance degradation, but I
> >>>> believe this is something we can improve over time (my initial
> >>>> measurements don't show any obvious regression though).
> >>>
> >>> I ran this through my experimental setup on m400 and got this:
> >>
> >> [...]
> >>
> >>> What this tells me is that we do take a noticable hit on the
> >>> world-switch path, which shows up in the TCP_RR and hackbench workloads,
> >>> which have a high precision in their output.
> >>>
> >>> Note that the memcached number is well within its variability between
> >>> individual benchmark runs, where it varies to 12% of its average in over
> >>> 80% of the executions.
> >>>
> >>> I don't think this is a showstopper thought, but we could consider
> >>> looking more closely at a breakdown of the world-switch path and verify
> >>> if/where we are really taking a hit.
> >>
> >> Thanks for doing so, very interesting. As a data point, what compiler
> >> are you using? I'd expect some variability based on the compiler version...
> >>
> > I used the following (compiling natively on the m400):
> >
> > gcc version 4.8.2 (Ubuntu/Linaro 4.8.2-19ubuntu1)
>
> For what it is worth, I've ran hackbench on my Seattle B0 (8xA57 2GHz),
> with a 4 vcpu VM and got the following results (10 runs per kernel
> version, same configuration):
>
> v4.4-rc3-wsinc: Average 31.750
> 32.459
> 32.124
> 32.435
> 31.940
> 31.085
> 31.804
> 31.862
> 30.985
> 31.450
> 31.359
>
> v4.4-rc3: Average 31.954
> 31.806
> 31.598
> 32.697
> 31.472
> 31.410
> 32.562
> 31.938
> 31.932
> 31.672
> 32.459
>
> This is with GCC as produced by Linaro:
> aarch64-linux-gnu-gcc (Linaro GCC 5.1-2015.08) 5.1.1 20150608
>
> It could well be that your compiler generates worse code than the one I
> use, or that the code it outputs is badly tuned for XGene. I guess I
> need to unearth my Mustang to find out...
>
Worth investigating I suppose. At any rate, the conclusion stays the
same; we should proceed with these patches.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-12-01 18:51 ` Marc Zyngier
@ 2015-12-02 9:27 ` Christoffer Dall
2015-12-02 9:47 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 9:27 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Dec 01, 2015 at 06:51:00PM +0000, Marc Zyngier wrote:
> On 01/12/15 15:39, Christoffer Dall wrote:
> > On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
> >> KVM so far relies on code patching, and is likely to use it more
> >> in the future. The main issue is that our alternative system works
> >> at the instruction level, while we'd like to have alternatives at
> >> the function level.
> >>
> >> In order to cope with this, add the "hyp_alternate_select" macro that
> >> outputs a brief sequence of code that in turn can be patched, allowing
> >> al alternative function to be selected.
> >
> > s/al/an/ ?
> >
> >>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >> arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
> >> 1 file changed, 16 insertions(+)
> >>
> >> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> >> index 7ac8e11..f0427ee 100644
> >> --- a/arch/arm64/kvm/hyp/hyp.h
> >> +++ b/arch/arm64/kvm/hyp/hyp.h
> >> @@ -27,6 +27,22 @@
> >>
> >> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
> >>
> >> +/*
> >> + * Generates patchable code sequences that are used to switch between
> >> + * two implementations of a function, depending on the availability of
> >> + * a feature.
> >> + */
> >
> > This looks right to me, but I'm a bit unclear what the types of this is
> > and how to use it.
> >
> > Are orig and alt function pointers and cond is a CONFIG_FOO ? fname is
> > a symbol, which is defined as a prototype somewhere and then implemented
> > here, or?
> >
> > Perhaps a Usage: part of the docs would be helpful.
>
> How about:
>
> @fname: a symbol name that will be defined as a function returning a
> function pointer whose type will match @orig and @alt
> @orig: A pointer to the default function, as returned by @fname when
> @cond doesn't hold
> @alt: A pointer to the alternate function, as returned by @fname when
> @cond holds
> @cond: a CPU feature (as described in asm/cpufeature.h)
looks good.
>
> >
> >> +#define hyp_alternate_select(fname, orig, alt, cond) \
> >> +typeof(orig) * __hyp_text fname(void) \
> >> +{ \
> >> + typeof(alt) *val = orig; \
> >> + asm volatile(ALTERNATIVE("nop \n", \
> >> + "mov %0, %1 \n", \
> >> + cond) \
> >> + : "+r" (val) : "r" (alt)); \
> >> + return val; \
> >> +}
> >> +
> >> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> >> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >>
> >> --
> >> 2.1.4
> >>
> >
> > I haven't thought much about how all of this is implemented, but from my
> > point of views the ideal situation would be something like:
> >
> > void foo(int a, int b)
> > {
> > ALTERNATIVE_IF_NOT CONFIG_BAR
> > foo_legacy(a, b);
> > ALTERNATIVE_ELSE
> > foo_new(a, b);
> > ALTERNATIVE_END
> > }
> >
> > I realize this may be impossible because the C code could implement all
> > sort of fun stuff around the actual function calls, but would there be
> > some way to annotate the functions and find the actual branch statement
> > and change the target?
>
> The main issue is that C doesn't give you any access to the branch
> function itself, except for the asm-goto statements. It also makes it
> very hard to preserve the return type. For your idea to work, we'd need
> some support in the compiler itself. I'm sure that it is doable, just
> not by me! ;-)
Not by me either, I'm just asking stupid questions - as always.
>
> This is why I've ended up creating something that returns a function
> *pointer*, because that's something that exists in the language (no new
> concept). I simply made sure I could return it at minimal cost.
>
I don't have a problem with this either. I'm curious though, how much
of a performance improvement (and why) we get from doing this as opposed
to a simple if-statement?
Thanks,
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-12-02 9:27 ` Christoffer Dall
@ 2015-12-02 9:47 ` Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-02 9:47 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 09:27, Christoffer Dall wrote:
> On Tue, Dec 01, 2015 at 06:51:00PM +0000, Marc Zyngier wrote:
>> On 01/12/15 15:39, Christoffer Dall wrote:
>>> On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
>>>> KVM so far relies on code patching, and is likely to use it more
>>>> in the future. The main issue is that our alternative system works
>>>> at the instruction level, while we'd like to have alternatives at
>>>> the function level.
>>>>
>>>> In order to cope with this, add the "hyp_alternate_select" macro that
>>>> outputs a brief sequence of code that in turn can be patched, allowing
>>>> al alternative function to be selected.
>>>
>>> s/al/an/ ?
>>>
>>>>
>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>> ---
>>>> arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
>>>> 1 file changed, 16 insertions(+)
>>>>
>>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>>>> index 7ac8e11..f0427ee 100644
>>>> --- a/arch/arm64/kvm/hyp/hyp.h
>>>> +++ b/arch/arm64/kvm/hyp/hyp.h
>>>> @@ -27,6 +27,22 @@
>>>>
>>>> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
>>>>
>>>> +/*
>>>> + * Generates patchable code sequences that are used to switch between
>>>> + * two implementations of a function, depending on the availability of
>>>> + * a feature.
>>>> + */
>>>
>>> This looks right to me, but I'm a bit unclear what the types of this is
>>> and how to use it.
>>>
>>> Are orig and alt function pointers and cond is a CONFIG_FOO ? fname is
>>> a symbol, which is defined as a prototype somewhere and then implemented
>>> here, or?
>>>
>>> Perhaps a Usage: part of the docs would be helpful.
>>
>> How about:
>>
>> @fname: a symbol name that will be defined as a function returning a
>> function pointer whose type will match @orig and @alt
>> @orig: A pointer to the default function, as returned by @fname when
>> @cond doesn't hold
>> @alt: A pointer to the alternate function, as returned by @fname when
>> @cond holds
>> @cond: a CPU feature (as described in asm/cpufeature.h)
>
> looks good.
>
>>
>>>
>>>> +#define hyp_alternate_select(fname, orig, alt, cond) \
>>>> +typeof(orig) * __hyp_text fname(void) \
>>>> +{ \
>>>> + typeof(alt) *val = orig; \
>>>> + asm volatile(ALTERNATIVE("nop \n", \
>>>> + "mov %0, %1 \n", \
>>>> + cond) \
>>>> + : "+r" (val) : "r" (alt)); \
>>>> + return val; \
>>>> +}
>>>> +
>>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>>>
>>>> --
>>>> 2.1.4
>>>>
>>>
>>> I haven't thought much about how all of this is implemented, but from my
>>> point of views the ideal situation would be something like:
>>>
>>> void foo(int a, int b)
>>> {
>>> ALTERNATIVE_IF_NOT CONFIG_BAR
>>> foo_legacy(a, b);
>>> ALTERNATIVE_ELSE
>>> foo_new(a, b);
>>> ALTERNATIVE_END
>>> }
>>>
>>> I realize this may be impossible because the C code could implement all
>>> sort of fun stuff around the actual function calls, but would there be
>>> some way to annotate the functions and find the actual branch statement
>>> and change the target?
>>
>> The main issue is that C doesn't give you any access to the branch
>> function itself, except for the asm-goto statements. It also makes it
>> very hard to preserve the return type. For your idea to work, we'd need
>> some support in the compiler itself. I'm sure that it is doable, just
>> not by me! ;-)
>
> Not by me either, I'm just asking stupid questions - as always.
I don't find that stupid. Asking that kind of stuff is useful to put
things in perspective.
>>
>> This is why I've ended up creating something that returns a function
>> *pointer*, because that's something that exists in the language (no new
>> concept). I simply made sure I could return it at minimal cost.
>>
>
> I don't have a problem with this either. I'm curious though, how much
> of a performance improvement (and why) we get from doing this as opposed
> to a simple if-statement?
An if statement will involve fetching some configuration from memory.
You can do that, but you are going to waste a cache line and memory
bandwidth (both which are scarce resources) for something that never
ever changes over the life of the system. These things tend to accumulate.
There is also a small number of cases where you *have* to patch
instructions (think VHE, for example). And having two different ways to
check for things is just asking for trouble in the long run.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 21/21] arm64: KVM: Remove weak attributes
2015-11-27 18:50 ` [PATCH v2 21/21] arm64: KVM: Remove weak attributes Marc Zyngier
@ 2015-12-02 11:47 ` Christoffer Dall
2015-12-02 15:21 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:47 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:15PM +0000, Marc Zyngier wrote:
> As we've now switched to the new world switch implementation,
> remove the weak attributes, as nobody is supposed to override
> it anymore.
Why not remove the aliases and change the callers?
-Christoffer
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/debug-sr.c | 5 ++---
> arch/arm64/kvm/hyp/hyp-entry.S | 3 ---
> arch/arm64/kvm/hyp/switch.c | 5 ++---
> arch/arm64/kvm/hyp/tlb.c | 16 +++++++---------
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 5 ++---
> 5 files changed, 13 insertions(+), 21 deletions(-)
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index 774a3f69..747546b 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -125,10 +125,9 @@ void __hyp_text __debug_cond_restore_host_state(struct kvm_vcpu *vcpu)
> }
> }
>
> -u32 __hyp_text __debug_read_mdcr_el2(void)
> +static u32 __hyp_text __debug_read_mdcr_el2(void)
> {
> return read_sysreg(mdcr_el2);
> }
>
> -__alias(__debug_read_mdcr_el2)
> -u32 __weak __kvm_get_mdcr_el2(void);
> +__alias(__debug_read_mdcr_el2) u32 __kvm_get_mdcr_el2(void);
> diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> index ace919b..bbc0be1 100644
> --- a/arch/arm64/kvm/hyp/hyp-entry.S
> +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> @@ -184,9 +184,7 @@ ENDPROC(\label)
>
> .align 11
>
> - .weak __kvm_hyp_vector
> ENTRY(__kvm_hyp_vector)
> -ENTRY(__hyp_vector)
> ventry el2t_sync_invalid // Synchronous EL2t
> ventry el2t_irq_invalid // IRQ EL2t
> ventry el2t_fiq_invalid // FIQ EL2t
> @@ -206,5 +204,4 @@ ENTRY(__hyp_vector)
> ventry el1_irq // IRQ 32-bit EL1
> ventry el1_fiq_invalid // FIQ 32-bit EL1
> ventry el1_error_invalid // Error 32-bit EL1
> -ENDPROC(__hyp_vector)
> ENDPROC(__kvm_hyp_vector)
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index c8ba370..1154d66 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -84,7 +84,7 @@ static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu)
> __vgic_call_restore_state()(vcpu);
> }
>
> -int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> +static int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> struct kvm_cpu_context *guest_ctxt;
> @@ -141,8 +141,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> return exit_code;
> }
>
> -__alias(__guest_run)
> -int __weak __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> +__alias(__guest_run) int __kvm_vcpu_run(struct kvm_vcpu *vcpu);
>
> static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
>
> diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
> index 2c279a8..250e06c 100644
> --- a/arch/arm64/kvm/hyp/tlb.c
> +++ b/arch/arm64/kvm/hyp/tlb.c
> @@ -17,7 +17,7 @@
>
> #include "hyp.h"
>
> -void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
> +static void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
> {
> dsb(ishst);
>
> @@ -47,10 +47,10 @@ void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
> write_sysreg(0, vttbr_el2);
> }
>
> -__alias(__tlb_flush_vmid_ipa)
> -void __weak __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
> +__alias(__tlb_flush_vmid_ipa) void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm,
> + phys_addr_t ipa);
>
> -void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
> +static void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
> {
> dsb(ishst);
>
> @@ -66,10 +66,9 @@ void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
> write_sysreg(0, vttbr_el2);
> }
>
> -__alias(__tlb_flush_vmid)
> -void __weak __kvm_tlb_flush_vmid(struct kvm *kvm);
> +__alias(__tlb_flush_vmid) void __kvm_tlb_flush_vmid(struct kvm *kvm);
>
> -void __hyp_text __tlb_flush_vm_context(void)
> +static void __hyp_text __tlb_flush_vm_context(void)
> {
> dsb(ishst);
> asm volatile("tlbi alle1is \n"
> @@ -77,5 +76,4 @@ void __hyp_text __tlb_flush_vm_context(void)
> dsb(ish);
> }
>
> -__alias(__tlb_flush_vm_context)
> -void __weak __kvm_flush_vm_context(void);
> +__alias(__tlb_flush_vm_context) void __kvm_flush_vm_context(void);
> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> index 1b0eedb..82a4f4b 100644
> --- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> @@ -216,10 +216,9 @@ void __hyp_text __vgic_v3_restore_state(struct kvm_vcpu *vcpu)
> }
> }
>
> -u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
> +static u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
> {
> return read_gicreg(ICH_VTR_EL2);
> }
>
> -__alias(__vgic_v3_read_ich_vtr_el2)
> -u64 __weak __vgic_v3_get_ich_vtr_el2(void);
> +__alias(__vgic_v3_read_ich_vtr_el2) u64 __vgic_v3_get_ich_vtr_el2(void);
> --
> 2.1.4
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 16/21] arm64: KVM: Add compatibility aliases
2015-11-27 18:50 ` [PATCH v2 16/21] arm64: KVM: Add compatibility aliases Marc Zyngier
@ 2015-12-02 11:49 ` Christoffer Dall
2015-12-02 15:23 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:49 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:10PM +0000, Marc Zyngier wrote:
> So far, we've implemented the new world switch with a completely
> different namespace, so that we could have both implementation
> compiled in.
>
> Let's take things one step further by adding weak aliases that
> have the same names as the original implementation. The weak
> attributes allows the new implementation to be overriden by the
> old one, and everything still work.
Do I understand correctly that the whole point of this is to keep
everything compiling nicely while at the same time being able to split
the patches so that you can have an isolated "remove old code" patch
that doesn't have to change the callers?
If so, I think explaining this rationale would be helpful in the commit
message in case we have to go back and track these changes in connection
with a regression and don't remember why we did things this way.
Maybe I'm being over-cautious though...
Otherwise:
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/debug-sr.c | 3 +++
> arch/arm64/kvm/hyp/hyp-entry.S | 3 +++
> arch/arm64/kvm/hyp/switch.c | 3 +++
> arch/arm64/kvm/hyp/tlb.c | 9 +++++++++
> arch/arm64/kvm/hyp/vgic-v3-sr.c | 3 +++
> 5 files changed, 21 insertions(+)
>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index a0b2b99..afd0a53 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -128,3 +128,6 @@ u32 __hyp_text __debug_read_mdcr_el2(void)
> {
> return read_sysreg(mdcr_el2);
> }
> +
> +__alias(__debug_read_mdcr_el2)
> +u32 __weak __kvm_get_mdcr_el2(void);
> diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> index 39d6935..ace919b 100644
> --- a/arch/arm64/kvm/hyp/hyp-entry.S
> +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> @@ -184,6 +184,8 @@ ENDPROC(\label)
>
> .align 11
>
> + .weak __kvm_hyp_vector
> +ENTRY(__kvm_hyp_vector)
> ENTRY(__hyp_vector)
> ventry el2t_sync_invalid // Synchronous EL2t
> ventry el2t_irq_invalid // IRQ EL2t
> @@ -205,3 +207,4 @@ ENTRY(__hyp_vector)
> ventry el1_fiq_invalid // FIQ 32-bit EL1
> ventry el1_error_invalid // Error 32-bit EL1
> ENDPROC(__hyp_vector)
> +ENDPROC(__kvm_hyp_vector)
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 7b81089..c8ba370 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -141,6 +141,9 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> return exit_code;
> }
>
> +__alias(__guest_run)
> +int __weak __kvm_vcpu_run(struct kvm_vcpu *vcpu);
> +
> static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
>
> void __hyp_text __noreturn __hyp_panic(void)
> diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
> index d4a07d0..2c279a8 100644
> --- a/arch/arm64/kvm/hyp/tlb.c
> +++ b/arch/arm64/kvm/hyp/tlb.c
> @@ -47,6 +47,9 @@ void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
> write_sysreg(0, vttbr_el2);
> }
>
> +__alias(__tlb_flush_vmid_ipa)
> +void __weak __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa);
> +
> void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
> {
> dsb(ishst);
> @@ -63,6 +66,9 @@ void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
> write_sysreg(0, vttbr_el2);
> }
>
> +__alias(__tlb_flush_vmid)
> +void __weak __kvm_tlb_flush_vmid(struct kvm *kvm);
> +
> void __hyp_text __tlb_flush_vm_context(void)
> {
> dsb(ishst);
> @@ -70,3 +76,6 @@ void __hyp_text __tlb_flush_vm_context(void)
> "ic ialluis ": : );
> dsb(ish);
> }
> +
> +__alias(__tlb_flush_vm_context)
> +void __weak __kvm_flush_vm_context(void);
> diff --git a/arch/arm64/kvm/hyp/vgic-v3-sr.c b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> index b490db5..1b0eedb 100644
> --- a/arch/arm64/kvm/hyp/vgic-v3-sr.c
> +++ b/arch/arm64/kvm/hyp/vgic-v3-sr.c
> @@ -220,3 +220,6 @@ u64 __hyp_text __vgic_v3_read_ich_vtr_el2(void)
> {
> return read_gicreg(ICH_VTR_EL2);
> }
> +
> +__alias(__vgic_v3_read_ich_vtr_el2)
> +u64 __weak __vgic_v3_get_ich_vtr_el2(void);
> --
> 2.1.4
>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 17/21] arm64: KVM: Map the kernel RO section into HYP
2015-11-27 18:50 ` [PATCH v2 17/21] arm64: KVM: Map the kernel RO section into HYP Marc Zyngier
@ 2015-12-02 11:49 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:49 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:11PM +0000, Marc Zyngier wrote:
> In order to run C code in HYP, we must make sure that the kernel's
> RO section in mapped into HYP (otherwise things break badly).
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 18/21] arm64: KVM: Move away from the assembly version of the world switch
2015-11-27 18:50 ` [PATCH v2 18/21] arm64: KVM: Move away from the assembly version of the world switch Marc Zyngier
@ 2015-12-02 11:49 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:49 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:12PM +0000, Marc Zyngier wrote:
> This is it. We remove all of the code that has now been rewritten.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum
2015-11-27 18:50 ` [PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum Marc Zyngier
@ 2015-12-02 11:51 ` Christoffer Dall
2015-12-02 15:26 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:51 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:13PM +0000, Marc Zyngier wrote:
> Having the system register numbers as #defines has been a pain
> since day one, as the ordering is pretty fragile, and moving
> things around leads to renumbering and epic conflict resolutions.
>
> Now that we're mostly acessing the sysreg file in C, an enum is
> a much better type to use, and we can clean things up a bit.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/include/asm/kvm_asm.h | 76 ---------------------------------
> arch/arm64/include/asm/kvm_emulate.h | 1 -
> arch/arm64/include/asm/kvm_host.h | 81 +++++++++++++++++++++++++++++++++++-
> arch/arm64/include/asm/kvm_mmio.h | 1 -
> arch/arm64/kernel/asm-offsets.c | 1 +
> arch/arm64/kvm/guest.c | 1 -
> arch/arm64/kvm/handle_exit.c | 1 +
> arch/arm64/kvm/hyp/debug-sr.c | 1 +
> arch/arm64/kvm/hyp/entry.S | 3 +-
> arch/arm64/kvm/hyp/sysreg-sr.c | 1 +
> arch/arm64/kvm/sys_regs.c | 1 +
> virt/kvm/arm/vgic-v3.c | 1 +
> 12 files changed, 87 insertions(+), 82 deletions(-)
>
> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
> index 5e37710..52b777b 100644
> --- a/arch/arm64/include/asm/kvm_asm.h
> +++ b/arch/arm64/include/asm/kvm_asm.h
> @@ -20,82 +20,6 @@
>
> #include <asm/virt.h>
>
> -/*
> - * 0 is reserved as an invalid value.
> - * Order *must* be kept in sync with the hyp switch code.
> - */
> -#define MPIDR_EL1 1 /* MultiProcessor Affinity Register */
> -#define CSSELR_EL1 2 /* Cache Size Selection Register */
> -#define SCTLR_EL1 3 /* System Control Register */
> -#define ACTLR_EL1 4 /* Auxiliary Control Register */
> -#define CPACR_EL1 5 /* Coprocessor Access Control */
> -#define TTBR0_EL1 6 /* Translation Table Base Register 0 */
> -#define TTBR1_EL1 7 /* Translation Table Base Register 1 */
> -#define TCR_EL1 8 /* Translation Control Register */
> -#define ESR_EL1 9 /* Exception Syndrome Register */
> -#define AFSR0_EL1 10 /* Auxilary Fault Status Register 0 */
> -#define AFSR1_EL1 11 /* Auxilary Fault Status Register 1 */
> -#define FAR_EL1 12 /* Fault Address Register */
> -#define MAIR_EL1 13 /* Memory Attribute Indirection Register */
> -#define VBAR_EL1 14 /* Vector Base Address Register */
> -#define CONTEXTIDR_EL1 15 /* Context ID Register */
> -#define TPIDR_EL0 16 /* Thread ID, User R/W */
> -#define TPIDRRO_EL0 17 /* Thread ID, User R/O */
> -#define TPIDR_EL1 18 /* Thread ID, Privileged */
> -#define AMAIR_EL1 19 /* Aux Memory Attribute Indirection Register */
> -#define CNTKCTL_EL1 20 /* Timer Control Register (EL1) */
> -#define PAR_EL1 21 /* Physical Address Register */
> -#define MDSCR_EL1 22 /* Monitor Debug System Control Register */
> -#define MDCCINT_EL1 23 /* Monitor Debug Comms Channel Interrupt Enable Reg */
> -
> -/* 32bit specific registers. Keep them at the end of the range */
> -#define DACR32_EL2 24 /* Domain Access Control Register */
> -#define IFSR32_EL2 25 /* Instruction Fault Status Register */
> -#define FPEXC32_EL2 26 /* Floating-Point Exception Control Register */
> -#define DBGVCR32_EL2 27 /* Debug Vector Catch Register */
> -#define NR_SYS_REGS 28
> -
> -/* 32bit mapping */
> -#define c0_MPIDR (MPIDR_EL1 * 2) /* MultiProcessor ID Register */
> -#define c0_CSSELR (CSSELR_EL1 * 2)/* Cache Size Selection Register */
> -#define c1_SCTLR (SCTLR_EL1 * 2) /* System Control Register */
> -#define c1_ACTLR (ACTLR_EL1 * 2) /* Auxiliary Control Register */
> -#define c1_CPACR (CPACR_EL1 * 2) /* Coprocessor Access Control */
> -#define c2_TTBR0 (TTBR0_EL1 * 2) /* Translation Table Base Register 0 */
> -#define c2_TTBR0_high (c2_TTBR0 + 1) /* TTBR0 top 32 bits */
> -#define c2_TTBR1 (TTBR1_EL1 * 2) /* Translation Table Base Register 1 */
> -#define c2_TTBR1_high (c2_TTBR1 + 1) /* TTBR1 top 32 bits */
> -#define c2_TTBCR (TCR_EL1 * 2) /* Translation Table Base Control R. */
> -#define c3_DACR (DACR32_EL2 * 2)/* Domain Access Control Register */
> -#define c5_DFSR (ESR_EL1 * 2) /* Data Fault Status Register */
> -#define c5_IFSR (IFSR32_EL2 * 2)/* Instruction Fault Status Register */
> -#define c5_ADFSR (AFSR0_EL1 * 2) /* Auxiliary Data Fault Status R */
> -#define c5_AIFSR (AFSR1_EL1 * 2) /* Auxiliary Instr Fault Status R */
> -#define c6_DFAR (FAR_EL1 * 2) /* Data Fault Address Register */
> -#define c6_IFAR (c6_DFAR + 1) /* Instruction Fault Address Register */
> -#define c7_PAR (PAR_EL1 * 2) /* Physical Address Register */
> -#define c7_PAR_high (c7_PAR + 1) /* PAR top 32 bits */
> -#define c10_PRRR (MAIR_EL1 * 2) /* Primary Region Remap Register */
> -#define c10_NMRR (c10_PRRR + 1) /* Normal Memory Remap Register */
> -#define c12_VBAR (VBAR_EL1 * 2) /* Vector Base Address Register */
> -#define c13_CID (CONTEXTIDR_EL1 * 2) /* Context ID Register */
> -#define c13_TID_URW (TPIDR_EL0 * 2) /* Thread ID, User R/W */
> -#define c13_TID_URO (TPIDRRO_EL0 * 2)/* Thread ID, User R/O */
> -#define c13_TID_PRIV (TPIDR_EL1 * 2) /* Thread ID, Privileged */
> -#define c10_AMAIR0 (AMAIR_EL1 * 2) /* Aux Memory Attr Indirection Reg */
> -#define c10_AMAIR1 (c10_AMAIR0 + 1)/* Aux Memory Attr Indirection Reg */
> -#define c14_CNTKCTL (CNTKCTL_EL1 * 2) /* Timer Control Register (PL1) */
> -
> -#define cp14_DBGDSCRext (MDSCR_EL1 * 2)
> -#define cp14_DBGBCR0 (DBGBCR0_EL1 * 2)
> -#define cp14_DBGBVR0 (DBGBVR0_EL1 * 2)
> -#define cp14_DBGBXVR0 (cp14_DBGBVR0 + 1)
> -#define cp14_DBGWCR0 (DBGWCR0_EL1 * 2)
> -#define cp14_DBGWVR0 (DBGWVR0_EL1 * 2)
> -#define cp14_DBGDCCINT (MDCCINT_EL1 * 2)
> -
> -#define NR_COPRO_REGS (NR_SYS_REGS * 2)
> -
> #define ARM_EXCEPTION_IRQ 0
> #define ARM_EXCEPTION_TRAP 1
>
> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
> index 3ca894e..170e17d 100644
> --- a/arch/arm64/include/asm/kvm_emulate.h
> +++ b/arch/arm64/include/asm/kvm_emulate.h
> @@ -26,7 +26,6 @@
>
> #include <asm/esr.h>
> #include <asm/kvm_arm.h>
> -#include <asm/kvm_asm.h>
> #include <asm/kvm_mmio.h>
> #include <asm/ptrace.h>
> #include <asm/cputype.h>
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index a35ce72..2fae2d4 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -25,7 +25,6 @@
> #include <linux/types.h>
> #include <linux/kvm_types.h>
> #include <asm/kvm.h>
> -#include <asm/kvm_asm.h>
> #include <asm/kvm_mmio.h>
>
> #define __KVM_HAVE_ARCH_INTC_INITIALIZED
> @@ -85,6 +84,86 @@ struct kvm_vcpu_fault_info {
> u64 hpfar_el2; /* Hyp IPA Fault Address Register */
> };
>
> +/*
> + * 0 is reserved as an invalid value.
> + * Order *must* be kept in sync with the hyp switch code.
do we still have such an ordering requirement?
> + */
> +enum vcpu_sysreg {
> + __INVALID_SYSREG__,
> + MPIDR_EL1, /* MultiProcessor Affinity Register */
> + CSSELR_EL1, /* Cache Size Selection Register */
> + SCTLR_EL1, /* System Control Register */
> + ACTLR_EL1, /* Auxiliary Control Register */
> + CPACR_EL1, /* Coprocessor Access Control */
> + TTBR0_EL1, /* Translation Table Base Register 0 */
> + TTBR1_EL1, /* Translation Table Base Register 1 */
> + TCR_EL1, /* Translation Control Register */
> + ESR_EL1, /* Exception Syndrome Register */
> + AFSR0_EL1, /* Auxilary Fault Status Register 0 */
> + AFSR1_EL1, /* Auxilary Fault Status Register 1 */
> + FAR_EL1, /* Fault Address Register */
> + MAIR_EL1, /* Memory Attribute Indirection Register */
> + VBAR_EL1, /* Vector Base Address Register */
> + CONTEXTIDR_EL1, /* Context ID Register */
> + TPIDR_EL0, /* Thread ID, User R/W */
> + TPIDRRO_EL0, /* Thread ID, User R/O */
> + TPIDR_EL1, /* Thread ID, Privileged */
> + AMAIR_EL1, /* Aux Memory Attribute Indirection Register */
> + CNTKCTL_EL1, /* Timer Control Register (EL1) */
> + PAR_EL1, /* Physical Address Register */
> + MDSCR_EL1, /* Monitor Debug System Control Register */
> + MDCCINT_EL1, /* Monitor Debug Comms Channel Interrupt Enable Reg */
> +
> + /* 32bit specific registers. Keep them at the end of the range */
> + DACR32_EL2, /* Domain Access Control Register */
> + IFSR32_EL2, /* Instruction Fault Status Register */
> + FPEXC32_EL2, /* Floating-Point Exception Control Register */
> + DBGVCR32_EL2, /* Debug Vector Catch Register */
> +
> + NR_SYS_REGS /* Nothing after this line! */
> +};
> +
> +/* 32bit mapping */
> +#define c0_MPIDR (MPIDR_EL1 * 2) /* MultiProcessor ID Register */
> +#define c0_CSSELR (CSSELR_EL1 * 2)/* Cache Size Selection Register */
> +#define c1_SCTLR (SCTLR_EL1 * 2) /* System Control Register */
> +#define c1_ACTLR (ACTLR_EL1 * 2) /* Auxiliary Control Register */
> +#define c1_CPACR (CPACR_EL1 * 2) /* Coprocessor Access Control */
> +#define c2_TTBR0 (TTBR0_EL1 * 2) /* Translation Table Base Register 0 */
> +#define c2_TTBR0_high (c2_TTBR0 + 1) /* TTBR0 top 32 bits */
> +#define c2_TTBR1 (TTBR1_EL1 * 2) /* Translation Table Base Register 1 */
> +#define c2_TTBR1_high (c2_TTBR1 + 1) /* TTBR1 top 32 bits */
> +#define c2_TTBCR (TCR_EL1 * 2) /* Translation Table Base Control R. */
> +#define c3_DACR (DACR32_EL2 * 2)/* Domain Access Control Register */
> +#define c5_DFSR (ESR_EL1 * 2) /* Data Fault Status Register */
> +#define c5_IFSR (IFSR32_EL2 * 2)/* Instruction Fault Status Register */
> +#define c5_ADFSR (AFSR0_EL1 * 2) /* Auxiliary Data Fault Status R */
> +#define c5_AIFSR (AFSR1_EL1 * 2) /* Auxiliary Instr Fault Status R */
> +#define c6_DFAR (FAR_EL1 * 2) /* Data Fault Address Register */
> +#define c6_IFAR (c6_DFAR + 1) /* Instruction Fault Address Register */
> +#define c7_PAR (PAR_EL1 * 2) /* Physical Address Register */
> +#define c7_PAR_high (c7_PAR + 1) /* PAR top 32 bits */
> +#define c10_PRRR (MAIR_EL1 * 2) /* Primary Region Remap Register */
> +#define c10_NMRR (c10_PRRR + 1) /* Normal Memory Remap Register */
> +#define c12_VBAR (VBAR_EL1 * 2) /* Vector Base Address Register */
> +#define c13_CID (CONTEXTIDR_EL1 * 2) /* Context ID Register */
> +#define c13_TID_URW (TPIDR_EL0 * 2) /* Thread ID, User R/W */
> +#define c13_TID_URO (TPIDRRO_EL0 * 2)/* Thread ID, User R/O */
> +#define c13_TID_PRIV (TPIDR_EL1 * 2) /* Thread ID, Privileged */
> +#define c10_AMAIR0 (AMAIR_EL1 * 2) /* Aux Memory Attr Indirection Reg */
> +#define c10_AMAIR1 (c10_AMAIR0 + 1)/* Aux Memory Attr Indirection Reg */
> +#define c14_CNTKCTL (CNTKCTL_EL1 * 2) /* Timer Control Register (PL1) */
> +
> +#define cp14_DBGDSCRext (MDSCR_EL1 * 2)
> +#define cp14_DBGBCR0 (DBGBCR0_EL1 * 2)
> +#define cp14_DBGBVR0 (DBGBVR0_EL1 * 2)
> +#define cp14_DBGBXVR0 (cp14_DBGBVR0 + 1)
> +#define cp14_DBGWCR0 (DBGWCR0_EL1 * 2)
> +#define cp14_DBGWVR0 (DBGWVR0_EL1 * 2)
> +#define cp14_DBGDCCINT (MDCCINT_EL1 * 2)
> +
> +#define NR_COPRO_REGS (NR_SYS_REGS * 2)
> +
> struct kvm_cpu_context {
> struct kvm_regs gp_regs;
> union {
> diff --git a/arch/arm64/include/asm/kvm_mmio.h b/arch/arm64/include/asm/kvm_mmio.h
> index 889c908..fe612a9 100644
> --- a/arch/arm64/include/asm/kvm_mmio.h
> +++ b/arch/arm64/include/asm/kvm_mmio.h
> @@ -19,7 +19,6 @@
> #define __ARM64_KVM_MMIO_H__
>
> #include <linux/kvm_host.h>
> -#include <asm/kvm_asm.h>
> #include <asm/kvm_arm.h>
>
> /*
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 25de8b2..4b72231 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -112,6 +112,7 @@ int main(void)
> DEFINE(CPU_ELR_EL1, offsetof(struct kvm_regs, elr_el1));
> DEFINE(CPU_SPSR, offsetof(struct kvm_regs, spsr));
> DEFINE(CPU_SYSREGS, offsetof(struct kvm_cpu_context, sys_regs));
> + DEFINE(VCPU_FPEXC32_EL2, offsetof(struct kvm_vcpu, arch.ctxt.sys_regs[FPEXC32_EL2]));
> DEFINE(VCPU_ESR_EL2, offsetof(struct kvm_vcpu, arch.fault.esr_el2));
> DEFINE(VCPU_FAR_EL2, offsetof(struct kvm_vcpu, arch.fault.far_el2));
> DEFINE(VCPU_HPFAR_EL2, offsetof(struct kvm_vcpu, arch.fault.hpfar_el2));
> diff --git a/arch/arm64/kvm/guest.c b/arch/arm64/kvm/guest.c
> index d250160..88e59f2 100644
> --- a/arch/arm64/kvm/guest.c
> +++ b/arch/arm64/kvm/guest.c
> @@ -28,7 +28,6 @@
> #include <asm/cputype.h>
> #include <asm/uaccess.h>
> #include <asm/kvm.h>
> -#include <asm/kvm_asm.h>
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_coproc.h>
>
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index 68a0759..4323627 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -23,6 +23,7 @@
> #include <linux/kvm_host.h>
>
> #include <asm/esr.h>
> +#include <asm/kvm_asm.h>
> #include <asm/kvm_coproc.h>
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_mmu.h>
> diff --git a/arch/arm64/kvm/hyp/debug-sr.c b/arch/arm64/kvm/hyp/debug-sr.c
> index afd0a53..774a3f69 100644
> --- a/arch/arm64/kvm/hyp/debug-sr.c
> +++ b/arch/arm64/kvm/hyp/debug-sr.c
> @@ -18,6 +18,7 @@
> #include <linux/compiler.h>
> #include <linux/kvm_host.h>
>
> +#include <asm/kvm_asm.h>
> #include <asm/kvm_mmu.h>
>
> #include "hyp.h"
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 7552922..599911c 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -27,7 +27,6 @@
>
> #define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
> #define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
> -#define CPU_SYSREG_OFFSET(x) (CPU_SYSREGS + 8*x)
>
> .text
> .pushsection .hyp.text, "ax"
> @@ -174,7 +173,7 @@ ENTRY(__fpsimd_guest_restore)
>
> mrs x1, hcr_el2
> tbnz x1, #HCR_RW_SHIFT, 1f
> - ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
> + ldr x4, [x3, #VCPU_FPEXC32_EL2]
> msr fpexc32_el2, x4
> 1:
> pop x4, lr
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index 41b9d30..b893c45 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -18,6 +18,7 @@
> #include <linux/compiler.h>
> #include <linux/kvm_host.h>
>
> +#include <asm/kvm_asm.h>
> #include <asm/kvm_mmu.h>
>
> #include "hyp.h"
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 87a64e8..0db5311 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -29,6 +29,7 @@
> #include <asm/debug-monitors.h>
> #include <asm/esr.h>
> #include <asm/kvm_arm.h>
> +#include <asm/kvm_asm.h>
> #include <asm/kvm_coproc.h>
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_host.h>
> diff --git a/virt/kvm/arm/vgic-v3.c b/virt/kvm/arm/vgic-v3.c
> index 487d635..c8506a2 100644
> --- a/virt/kvm/arm/vgic-v3.c
> +++ b/virt/kvm/arm/vgic-v3.c
> @@ -28,6 +28,7 @@
>
> #include <asm/kvm_emulate.h>
> #include <asm/kvm_arm.h>
> +#include <asm/kvm_asm.h>
> #include <asm/kvm_mmu.h>
>
> /* These are for GICv2 emulation only */
> --
> 2.1.4
>
Apart from the comment:
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 20/21] arm64: KVM: Cleanup asm-offset.c
2015-11-27 18:50 ` [PATCH v2 20/21] arm64: KVM: Cleanup asm-offset.c Marc Zyngier
@ 2015-12-02 11:51 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:51 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:14PM +0000, Marc Zyngier wrote:
> As we've now rewritten most of our code-base in C, most of the
> KVM-specific code in asm-offset.c is useless. Delete-time again!
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-12-02 9:47 ` Marc Zyngier
@ 2015-12-02 11:53 ` Christoffer Dall
2015-12-02 13:19 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:53 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Dec 02, 2015 at 09:47:43AM +0000, Marc Zyngier wrote:
> On 02/12/15 09:27, Christoffer Dall wrote:
> > On Tue, Dec 01, 2015 at 06:51:00PM +0000, Marc Zyngier wrote:
> >> On 01/12/15 15:39, Christoffer Dall wrote:
> >>> On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
> >>>> KVM so far relies on code patching, and is likely to use it more
> >>>> in the future. The main issue is that our alternative system works
> >>>> at the instruction level, while we'd like to have alternatives at
> >>>> the function level.
> >>>>
> >>>> In order to cope with this, add the "hyp_alternate_select" macro that
> >>>> outputs a brief sequence of code that in turn can be patched, allowing
> >>>> al alternative function to be selected.
> >>>
> >>> s/al/an/ ?
> >>>
> >>>>
> >>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >>>> ---
> >>>> arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
> >>>> 1 file changed, 16 insertions(+)
> >>>>
> >>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> >>>> index 7ac8e11..f0427ee 100644
> >>>> --- a/arch/arm64/kvm/hyp/hyp.h
> >>>> +++ b/arch/arm64/kvm/hyp/hyp.h
> >>>> @@ -27,6 +27,22 @@
> >>>>
> >>>> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
> >>>>
> >>>> +/*
> >>>> + * Generates patchable code sequences that are used to switch between
> >>>> + * two implementations of a function, depending on the availability of
> >>>> + * a feature.
> >>>> + */
> >>>
> >>> This looks right to me, but I'm a bit unclear what the types of this is
> >>> and how to use it.
> >>>
> >>> Are orig and alt function pointers and cond is a CONFIG_FOO ? fname is
> >>> a symbol, which is defined as a prototype somewhere and then implemented
> >>> here, or?
> >>>
> >>> Perhaps a Usage: part of the docs would be helpful.
> >>
> >> How about:
> >>
> >> @fname: a symbol name that will be defined as a function returning a
> >> function pointer whose type will match @orig and @alt
> >> @orig: A pointer to the default function, as returned by @fname when
> >> @cond doesn't hold
> >> @alt: A pointer to the alternate function, as returned by @fname when
> >> @cond holds
> >> @cond: a CPU feature (as described in asm/cpufeature.h)
> >
> > looks good.
> >
> >>
> >>>
> >>>> +#define hyp_alternate_select(fname, orig, alt, cond) \
> >>>> +typeof(orig) * __hyp_text fname(void) \
> >>>> +{ \
> >>>> + typeof(alt) *val = orig; \
> >>>> + asm volatile(ALTERNATIVE("nop \n", \
> >>>> + "mov %0, %1 \n", \
> >>>> + cond) \
> >>>> + : "+r" (val) : "r" (alt)); \
> >>>> + return val; \
> >>>> +}
> >>>> +
> >>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> >>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >>>>
> >>>> --
> >>>> 2.1.4
> >>>>
> >>>
> >>> I haven't thought much about how all of this is implemented, but from my
> >>> point of views the ideal situation would be something like:
> >>>
> >>> void foo(int a, int b)
> >>> {
> >>> ALTERNATIVE_IF_NOT CONFIG_BAR
> >>> foo_legacy(a, b);
> >>> ALTERNATIVE_ELSE
> >>> foo_new(a, b);
> >>> ALTERNATIVE_END
> >>> }
> >>>
> >>> I realize this may be impossible because the C code could implement all
> >>> sort of fun stuff around the actual function calls, but would there be
> >>> some way to annotate the functions and find the actual branch statement
> >>> and change the target?
> >>
> >> The main issue is that C doesn't give you any access to the branch
> >> function itself, except for the asm-goto statements. It also makes it
> >> very hard to preserve the return type. For your idea to work, we'd need
> >> some support in the compiler itself. I'm sure that it is doable, just
> >> not by me! ;-)
> >
> > Not by me either, I'm just asking stupid questions - as always.
>
> I don't find that stupid. Asking that kind of stuff is useful to put
> things in perspective.
>
Thanks!
> >>
> >> This is why I've ended up creating something that returns a function
> >> *pointer*, because that's something that exists in the language (no new
> >> concept). I simply made sure I could return it at minimal cost.
> >>
> >
> > I don't have a problem with this either. I'm curious though, how much
> > of a performance improvement (and why) we get from doing this as opposed
> > to a simple if-statement?
>
> An if statement will involve fetching some configuration from memory.
> You can do that, but you are going to waste a cache line and memory
> bandwidth (both which are scarce resources) for something that never
> ever changes over the life of the system. These things tend to accumulate.
Sure, but won't you be fetching the function pointer from memory anyway?
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore
2015-11-27 18:50 ` [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore Marc Zyngier
@ 2015-12-02 11:53 ` Christoffer Dall
2015-12-02 15:29 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:53 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:06PM +0000, Marc Zyngier wrote:
> Implement the fpsimd save restore, keeping the lazy part in
> assembler (as returning to C would be overkill).
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/entry.S | 32 +++++++++++++++++++++++++++++++-
> arch/arm64/kvm/hyp/fpsimd.S | 33 +++++++++++++++++++++++++++++++++
> arch/arm64/kvm/hyp/hyp.h | 7 +++++++
> arch/arm64/kvm/hyp/switch.c | 8 ++++++++
> arch/arm64/kvm/hyp/sysreg-sr.c | 2 +-
> 6 files changed, 81 insertions(+), 2 deletions(-)
> create mode 100644 arch/arm64/kvm/hyp/fpsimd.S
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index 9c11b0f..56238d0 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -9,3 +9,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += entry.o
> obj-$(CONFIG_KVM_ARM_HOST) += switch.o
> +obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> index 2c4449a..7552922 100644
> --- a/arch/arm64/kvm/hyp/entry.S
> +++ b/arch/arm64/kvm/hyp/entry.S
> @@ -27,6 +27,7 @@
>
> #define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
> #define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
> +#define CPU_SYSREG_OFFSET(x) (CPU_SYSREGS + 8*x)
>
> .text
> .pushsection .hyp.text, "ax"
> @@ -152,4 +153,33 @@ ENTRY(__guest_exit)
> ret
> ENDPROC(__guest_exit)
>
> - /* Insert fault handling here */
> +ENTRY(__fpsimd_guest_restore)
> + push x4, lr
> +
> + mrs x2, cptr_el2
> + bic x2, x2, #CPTR_EL2_TFP
> + msr cptr_el2, x2
> + isb
> +
> + mrs x3, tpidr_el2
> +
> + ldr x0, [x3, #VCPU_HOST_CONTEXT]
> + kern_hyp_va x0
> + add x0, x0, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> + bl __fpsimd_save_state
> +
> + add x2, x3, #VCPU_CONTEXT
> + add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> + bl __fpsimd_restore_state
> +
> + mrs x1, hcr_el2
> + tbnz x1, #HCR_RW_SHIFT, 1f
nit: Add a comment along the lines of:
// Skip restoring fpexc32 for AArch64 guests
> + ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
> + msr fpexc32_el2, x4
> +1:
> + pop x4, lr
> + pop x2, x3
> + pop x0, x1
> +
> + eret
> +ENDPROC(__fpsimd_guest_restore)
> diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S
> new file mode 100644
> index 0000000..da3f22c
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/fpsimd.S
> @@ -0,0 +1,33 @@
> +/*
> + * Copyright (C) 2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <linux/linkage.h>
> +
> +#include <asm/fpsimdmacros.h>
> +
> + .text
> + .pushsection .hyp.text, "ax"
> +
> +ENTRY(__fpsimd_save_state)
> + fpsimd_save x0, 1
> + ret
> +ENDPROC(__fpsimd_save_state)
> +
> +ENTRY(__fpsimd_restore_state)
> + fpsimd_restore x0, 1
> + ret
> +ENDPROC(__fpsimd_restore_state)
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index f0427ee..18365dd 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -66,6 +66,13 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
> void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
>
> +void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> +void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> +static inline bool __fpsimd_enabled(void)
> +{
> + return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> +}
> +
> u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
>
> #endif /* __ARM64_KVM_HYP_H__ */
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index d67ed9e..8affc19 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -88,6 +88,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> {
> struct kvm_cpu_context *host_ctxt;
> struct kvm_cpu_context *guest_ctxt;
> + bool fp_enabled;
> u64 exit_code;
>
> vcpu = kern_hyp_va(vcpu);
> @@ -117,6 +118,8 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> exit_code = __guest_enter(vcpu, host_ctxt);
> /* And we're baaack! */
>
> + fp_enabled = __fpsimd_enabled();
> +
what does 'enabled' really mean here? Isn't it really
__fpsimd_is_dirty() or __fpsimd_is_guest() or something like that (I
suck at naming too).
> __sysreg_save_state(guest_ctxt);
> __sysreg32_save_state(vcpu);
> __timer_save_state(vcpu);
> @@ -127,6 +130,11 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
>
> __sysreg_restore_state(host_ctxt);
>
> + if (fp_enabled) {
> + __fpsimd_save_state(&guest_ctxt->gp_regs.fp_regs);
> + __fpsimd_restore_state(&host_ctxt->gp_regs.fp_regs);
> + }
> +
> __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt);
> __debug_cond_restore_host_state(vcpu);
>
> diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c
> index 3f81a4d..41b9d30 100644
> --- a/arch/arm64/kvm/hyp/sysreg-sr.c
> +++ b/arch/arm64/kvm/hyp/sysreg-sr.c
> @@ -103,7 +103,7 @@ void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu)
> sysreg[DACR32_EL2] = read_sysreg(dacr32_el2);
> sysreg[IFSR32_EL2] = read_sysreg(ifsr32_el2);
>
> - if (!(read_sysreg(cptr_el2) & CPTR_EL2_TFP))
> + if (__fpsimd_enabled())
> sysreg[FPEXC32_EL2] = read_sysreg(fpexc32_el2);
>
> if (vcpu->arch.debug_flags & KVM_ARM64_DEBUG_DIRTY)
> --
> 2.1.4
>
Otherwise:
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 13/21] arm64: KVM: Implement TLB handling
2015-11-27 18:50 ` [PATCH v2 13/21] arm64: KVM: Implement TLB handling Marc Zyngier
@ 2015-12-02 11:53 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:53 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:07PM +0000, Marc Zyngier wrote:
> Implement the TLB handling as a direct translation of the assembly
> code version.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/Makefile | 1 +
> arch/arm64/kvm/hyp/tlb.c | 72 +++++++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 73 insertions(+)
> create mode 100644 arch/arm64/kvm/hyp/tlb.c
>
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index 56238d0..1a529f5 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -10,3 +10,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> obj-$(CONFIG_KVM_ARM_HOST) += entry.o
> obj-$(CONFIG_KVM_ARM_HOST) += switch.o
> obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
> +obj-$(CONFIG_KVM_ARM_HOST) += tlb.o
> diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c
> new file mode 100644
> index 0000000..d4a07d0
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/tlb.c
> @@ -0,0 +1,72 @@
> +/*
> + * Copyright (C) 2015 - ARM Ltd
> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License version 2 as
> + * published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License
> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> + */
> +
> +#include "hyp.h"
> +
> +void __hyp_text __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)
> +{
> + dsb(ishst);
> +
> + /* Switch to requested VMID */
> + kvm = kern_hyp_va(kvm);
> + write_sysreg(kvm->arch.vttbr, vttbr_el2);
> + isb();
> +
> + /*
> + * We could do so much better if we had the VA as well.
> + * Instead, we invalidate Stage-2 for this IPA, and the
> + * whole of Stage-1. Weep...
> + */
> + ipa >>= 12;
> + asm volatile("tlbi ipas2e1is, %0" : : "r" (ipa));
> + dsb(ish);
nit: missing white space
> + /*
> + * We have to ensure completion of the invalidation at Stage-2,
> + * since a table walk on another CPU could refill a TLB with a
> + * complete (S1 + S2) walk based on the old Stage-2 mapping if
> + * the Stage-1 invalidation happened first.
> + */
nit: isn't that comment targeting the dsb(ish) above as in the asm code
and should be moved above that line?
> + asm volatile("tlbi vmalle1is" : : );
> + dsb(ish);
> + isb();
> +
> + write_sysreg(0, vttbr_el2);
> +}
> +
> +void __hyp_text __tlb_flush_vmid(struct kvm *kvm)
> +{
> + dsb(ishst);
> +
> + /* Switch to requested VMID */
> + kvm = kern_hyp_va(kvm);
> + write_sysreg(kvm->arch.vttbr, vttbr_el2);
> + isb();
> +
> + asm volatile("tlbi vmalls12e1is" : : );
> + dsb(ish);
> + isb();
> +
> + write_sysreg(0, vttbr_el2);
> +}
> +
> +void __hyp_text __tlb_flush_vm_context(void)
> +{
> + dsb(ishst);
> + asm volatile("tlbi alle1is \n"
> + "ic ialluis ": : );
> + dsb(ish);
> +}
> --
> 2.1.4
>
Otherwise:
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 14/21] arm64: KVM: HYP mode entry points
2015-11-27 18:50 ` [PATCH v2 14/21] arm64: KVM: HYP mode entry points Marc Zyngier
@ 2015-12-02 11:53 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:53 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:08PM +0000, Marc Zyngier wrote:
> Add the entry points for HYP mode (both for hypercalls and
> exception handling).
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 15/21] arm64: KVM: Add panic handling
2015-11-27 18:50 ` [PATCH v2 15/21] arm64: KVM: Add panic handling Marc Zyngier
@ 2015-12-02 11:53 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 11:53 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:09PM +0000, Marc Zyngier wrote:
> Add the panic handler, together with the small bits of assembly
> code to call the kernel's panic implementation.
>
> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> ---
> arch/arm64/kvm/hyp/hyp-entry.S | 11 ++++++++++-
> arch/arm64/kvm/hyp/hyp.h | 1 +
> arch/arm64/kvm/hyp/switch.c | 30 ++++++++++++++++++++++++++++++
> 3 files changed, 41 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S
> index 8334407..39d6935 100644
> --- a/arch/arm64/kvm/hyp/hyp-entry.S
> +++ b/arch/arm64/kvm/hyp/hyp-entry.S
> @@ -150,7 +150,16 @@ el1_irq:
> mov x1, #ARM_EXCEPTION_IRQ
> b __guest_exit
>
> -.macro invalid_vector label, target = __kvm_hyp_panic
> +ENTRY(__hyp_do_panic)
> + mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\
> + PSR_MODE_EL1h)
> + msr spsr_el2, lr
> + ldr lr, =panic
> + msr elr_el2, lr
> + eret
> +ENDPROC(__hyp_do_panic)
> +
> +.macro invalid_vector label, target = __hyp_panic
> .align 2
> \label:
> b \target
> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> index 18365dd..87f16fa 100644
> --- a/arch/arm64/kvm/hyp/hyp.h
> +++ b/arch/arm64/kvm/hyp/hyp.h
> @@ -74,6 +74,7 @@ static inline bool __fpsimd_enabled(void)
> }
>
> u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
> +void __noreturn __hyp_do_panic(unsigned long, ...);
>
> #endif /* __ARM64_KVM_HYP_H__ */
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 8affc19..7b81089 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -140,3 +140,33 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
>
> return exit_code;
> }
> +
> +static const char __hyp_panic_string[] = "HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n";
> +
> +void __hyp_text __noreturn __hyp_panic(void)
> +{
> + unsigned long str_va = (unsigned long)__hyp_panic_string;
> + u64 spsr = read_sysreg(spsr_el2);
> + u64 elr = read_sysreg(elr_el2);
> + u64 par = read_sysreg(par_el1);
> +
> + if (read_sysreg(vttbr_el2)) {
> + struct kvm_vcpu *vcpu;
> + struct kvm_cpu_context *host_ctxt;
> +
> + vcpu = (struct kvm_vcpu *)read_sysreg(tpidr_el2);
> + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context);
> + __deactivate_traps(vcpu);
> + __deactivate_vm(vcpu);
> + __sysreg_restore_state(host_ctxt);
> + }
> +
> + /* Call panic for real */
> + __hyp_do_panic(str_va - HYP_PAGE_OFFSET + PAGE_OFFSET,
is the first parameter hyp_kern_va(str_va) ? If so, can you add that
define instead?
> + spsr, elr,
> + read_sysreg(esr_el2), read_sysreg(far_el2),
> + read_sysreg(hpfar_el2), par,
> + (void *)read_sysreg(tpidr_el2));
> +
> + unreachable();
> +}
> --
> 2.1.4
>
Otherwise:
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-12-02 11:53 ` Christoffer Dall
@ 2015-12-02 13:19 ` Marc Zyngier
2015-12-02 16:19 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-02 13:19 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 11:53, Christoffer Dall wrote:
> On Wed, Dec 02, 2015 at 09:47:43AM +0000, Marc Zyngier wrote:
>> On 02/12/15 09:27, Christoffer Dall wrote:
>>> On Tue, Dec 01, 2015 at 06:51:00PM +0000, Marc Zyngier wrote:
>>>> On 01/12/15 15:39, Christoffer Dall wrote:
>>>>> On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
>>>>>> KVM so far relies on code patching, and is likely to use it more
>>>>>> in the future. The main issue is that our alternative system works
>>>>>> at the instruction level, while we'd like to have alternatives at
>>>>>> the function level.
>>>>>>
>>>>>> In order to cope with this, add the "hyp_alternate_select" macro that
>>>>>> outputs a brief sequence of code that in turn can be patched, allowing
>>>>>> al alternative function to be selected.
>>>>>
>>>>> s/al/an/ ?
>>>>>
>>>>>>
>>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>>>>>> ---
>>>>>> arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
>>>>>> 1 file changed, 16 insertions(+)
>>>>>>
>>>>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>>>>>> index 7ac8e11..f0427ee 100644
>>>>>> --- a/arch/arm64/kvm/hyp/hyp.h
>>>>>> +++ b/arch/arm64/kvm/hyp/hyp.h
>>>>>> @@ -27,6 +27,22 @@
>>>>>>
>>>>>> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
>>>>>>
>>>>>> +/*
>>>>>> + * Generates patchable code sequences that are used to switch between
>>>>>> + * two implementations of a function, depending on the availability of
>>>>>> + * a feature.
>>>>>> + */
>>>>>
>>>>> This looks right to me, but I'm a bit unclear what the types of this is
>>>>> and how to use it.
>>>>>
>>>>> Are orig and alt function pointers and cond is a CONFIG_FOO ? fname is
>>>>> a symbol, which is defined as a prototype somewhere and then implemented
>>>>> here, or?
>>>>>
>>>>> Perhaps a Usage: part of the docs would be helpful.
>>>>
>>>> How about:
>>>>
>>>> @fname: a symbol name that will be defined as a function returning a
>>>> function pointer whose type will match @orig and @alt
>>>> @orig: A pointer to the default function, as returned by @fname when
>>>> @cond doesn't hold
>>>> @alt: A pointer to the alternate function, as returned by @fname when
>>>> @cond holds
>>>> @cond: a CPU feature (as described in asm/cpufeature.h)
>>>
>>> looks good.
>>>
>>>>
>>>>>
>>>>>> +#define hyp_alternate_select(fname, orig, alt, cond) \
>>>>>> +typeof(orig) * __hyp_text fname(void) \
>>>>>> +{ \
>>>>>> + typeof(alt) *val = orig; \
>>>>>> + asm volatile(ALTERNATIVE("nop \n", \
>>>>>> + "mov %0, %1 \n", \
>>>>>> + cond) \
>>>>>> + : "+r" (val) : "r" (alt)); \
>>>>>> + return val; \
>>>>>> +}
>>>>>> +
>>>>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
>>>>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
>>>>>>
>>>>>> --
>>>>>> 2.1.4
>>>>>>
>>>>>
>>>>> I haven't thought much about how all of this is implemented, but from my
>>>>> point of views the ideal situation would be something like:
>>>>>
>>>>> void foo(int a, int b)
>>>>> {
>>>>> ALTERNATIVE_IF_NOT CONFIG_BAR
>>>>> foo_legacy(a, b);
>>>>> ALTERNATIVE_ELSE
>>>>> foo_new(a, b);
>>>>> ALTERNATIVE_END
>>>>> }
>>>>>
>>>>> I realize this may be impossible because the C code could implement all
>>>>> sort of fun stuff around the actual function calls, but would there be
>>>>> some way to annotate the functions and find the actual branch statement
>>>>> and change the target?
>>>>
>>>> The main issue is that C doesn't give you any access to the branch
>>>> function itself, except for the asm-goto statements. It also makes it
>>>> very hard to preserve the return type. For your idea to work, we'd need
>>>> some support in the compiler itself. I'm sure that it is doable, just
>>>> not by me! ;-)
>>>
>>> Not by me either, I'm just asking stupid questions - as always.
>>
>> I don't find that stupid. Asking that kind of stuff is useful to put
>> things in perspective.
>>
>
> Thanks!
>
>>>>
>>>> This is why I've ended up creating something that returns a function
>>>> *pointer*, because that's something that exists in the language (no new
>>>> concept). I simply made sure I could return it at minimal cost.
>>>>
>>>
>>> I don't have a problem with this either. I'm curious though, how much
>>> of a performance improvement (and why) we get from doing this as opposed
>>> to a simple if-statement?
>>
>> An if statement will involve fetching some configuration from memory.
>> You can do that, but you are going to waste a cache line and memory
>> bandwidth (both which are scarce resources) for something that never
>> ever changes over the life of the system. These things tend to accumulate.
>
> Sure, but won't you be fetching the function pointer from memory anyway?
No, and that's the whole point of this patch: the function pointers are
loaded into registers as PC-relative constants (adrp+add), the selection
being done by a mov or a nop. For example:
ffffffc0007f1f60: 90000001 adrp x1, ffffffc0007f1000
ffffffc0007f1f64: 90000000 adrp x0, ffffffc0007f1000
ffffffc0007f1f68: 91036021 add x1, x1, #0xd8
ffffffc0007f1f6c: 910b0000 add x0, x0, #0x2c0
ffffffc0007f1f70: d503201f nop
ffffffc0007f1f74: aa1303e0 mov x0, x19
ffffffc0007f1f78: d63f0020 blr x1
For the default condition (the above code), the CPU is likely to discard
the second adrp+add (because of the mov x0, x19). For the alternate, the
nop is replaced by a "mov x1, x0", which makes the first adrp+add
irrelevant (it will be eliminated in the pipeline of any decent CPU).
While this has a cost in terms of instruction footprint, the branch
predictor is quickly going to find out where we're branching. We also
avoid fetching both from the I and D sides, which could kill the branch
predictor if not speculated in time. In the end, we get something that
is a lot more predictable, even for simpler CPU designs.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 21/21] arm64: KVM: Remove weak attributes
2015-12-02 11:47 ` Christoffer Dall
@ 2015-12-02 15:21 ` Marc Zyngier
2015-12-02 16:21 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-02 15:21 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 11:47, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:50:15PM +0000, Marc Zyngier wrote:
>> As we've now switched to the new world switch implementation,
>> remove the weak attributes, as nobody is supposed to override
>> it anymore.
>
> Why not remove the aliases and change the callers?
This is likely to be a bigger patch, and it would affect the 32bit as
well. So far, I'm choosing to keep things the same. Another solution
would be to completely drop the aliases, and just rename the new
function to have the old names.
I don't mind either way.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 16/21] arm64: KVM: Add compatibility aliases
2015-12-02 11:49 ` Christoffer Dall
@ 2015-12-02 15:23 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-02 15:23 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 11:49, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:50:10PM +0000, Marc Zyngier wrote:
>> So far, we've implemented the new world switch with a completely
>> different namespace, so that we could have both implementation
>> compiled in.
>>
>> Let's take things one step further by adding weak aliases that
>> have the same names as the original implementation. The weak
>> attributes allows the new implementation to be overriden by the
>> old one, and everything still work.
>
> Do I understand correctly that the whole point of this is to keep
> everything compiling nicely while at the same time being able to split
> the patches so that you can have an isolated "remove old code" patch
> that doesn't have to change the callers?
Exactly.
> If so, I think explaining this rationale would be helpful in the commit
> message in case we have to go back and track these changes in connection
> with a regression and don't remember why we did things this way.
Fair enough. I'll update the commit message (possibly by stealing a
large part of the above text!).
> Maybe I'm being over-cautious though...
>
> Otherwise:
>
> Acked-by: Christoffer Dall <christoffer.dall@linaro.org>
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum
2015-12-02 11:51 ` Christoffer Dall
@ 2015-12-02 15:26 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-02 15:26 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 11:51, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:50:13PM +0000, Marc Zyngier wrote:
>> Having the system register numbers as #defines has been a pain
>> since day one, as the ordering is pretty fragile, and moving
>> things around leads to renumbering and epic conflict resolutions.
>>
>> Now that we're mostly acessing the sysreg file in C, an enum is
>> a much better type to use, and we can clean things up a bit.
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/include/asm/kvm_asm.h | 76 ---------------------------------
>> arch/arm64/include/asm/kvm_emulate.h | 1 -
>> arch/arm64/include/asm/kvm_host.h | 81 +++++++++++++++++++++++++++++++++++-
>> arch/arm64/include/asm/kvm_mmio.h | 1 -
>> arch/arm64/kernel/asm-offsets.c | 1 +
>> arch/arm64/kvm/guest.c | 1 -
>> arch/arm64/kvm/handle_exit.c | 1 +
>> arch/arm64/kvm/hyp/debug-sr.c | 1 +
>> arch/arm64/kvm/hyp/entry.S | 3 +-
>> arch/arm64/kvm/hyp/sysreg-sr.c | 1 +
>> arch/arm64/kvm/sys_regs.c | 1 +
>> virt/kvm/arm/vgic-v3.c | 1 +
>> 12 files changed, 87 insertions(+), 82 deletions(-)
>>
>> diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_asm.h
>> index 5e37710..52b777b 100644
>> --- a/arch/arm64/include/asm/kvm_asm.h
>> +++ b/arch/arm64/include/asm/kvm_asm.h
>> @@ -20,82 +20,6 @@
>>
>> #include <asm/virt.h>
>>
>> -/*
>> - * 0 is reserved as an invalid value.
>> - * Order *must* be kept in sync with the hyp switch code.
>> - */
>> -#define MPIDR_EL1 1 /* MultiProcessor Affinity Register */
>> -#define CSSELR_EL1 2 /* Cache Size Selection Register */
>> -#define SCTLR_EL1 3 /* System Control Register */
>> -#define ACTLR_EL1 4 /* Auxiliary Control Register */
>> -#define CPACR_EL1 5 /* Coprocessor Access Control */
>> -#define TTBR0_EL1 6 /* Translation Table Base Register 0 */
>> -#define TTBR1_EL1 7 /* Translation Table Base Register 1 */
>> -#define TCR_EL1 8 /* Translation Control Register */
>> -#define ESR_EL1 9 /* Exception Syndrome Register */
>> -#define AFSR0_EL1 10 /* Auxilary Fault Status Register 0 */
>> -#define AFSR1_EL1 11 /* Auxilary Fault Status Register 1 */
>> -#define FAR_EL1 12 /* Fault Address Register */
>> -#define MAIR_EL1 13 /* Memory Attribute Indirection Register */
>> -#define VBAR_EL1 14 /* Vector Base Address Register */
>> -#define CONTEXTIDR_EL1 15 /* Context ID Register */
>> -#define TPIDR_EL0 16 /* Thread ID, User R/W */
>> -#define TPIDRRO_EL0 17 /* Thread ID, User R/O */
>> -#define TPIDR_EL1 18 /* Thread ID, Privileged */
>> -#define AMAIR_EL1 19 /* Aux Memory Attribute Indirection Register */
>> -#define CNTKCTL_EL1 20 /* Timer Control Register (EL1) */
>> -#define PAR_EL1 21 /* Physical Address Register */
>> -#define MDSCR_EL1 22 /* Monitor Debug System Control Register */
>> -#define MDCCINT_EL1 23 /* Monitor Debug Comms Channel Interrupt Enable Reg */
>> -
>> -/* 32bit specific registers. Keep them at the end of the range */
>> -#define DACR32_EL2 24 /* Domain Access Control Register */
>> -#define IFSR32_EL2 25 /* Instruction Fault Status Register */
>> -#define FPEXC32_EL2 26 /* Floating-Point Exception Control Register */
>> -#define DBGVCR32_EL2 27 /* Debug Vector Catch Register */
>> -#define NR_SYS_REGS 28
>> -
>> -/* 32bit mapping */
>> -#define c0_MPIDR (MPIDR_EL1 * 2) /* MultiProcessor ID Register */
>> -#define c0_CSSELR (CSSELR_EL1 * 2)/* Cache Size Selection Register */
>> -#define c1_SCTLR (SCTLR_EL1 * 2) /* System Control Register */
>> -#define c1_ACTLR (ACTLR_EL1 * 2) /* Auxiliary Control Register */
>> -#define c1_CPACR (CPACR_EL1 * 2) /* Coprocessor Access Control */
>> -#define c2_TTBR0 (TTBR0_EL1 * 2) /* Translation Table Base Register 0 */
>> -#define c2_TTBR0_high (c2_TTBR0 + 1) /* TTBR0 top 32 bits */
>> -#define c2_TTBR1 (TTBR1_EL1 * 2) /* Translation Table Base Register 1 */
>> -#define c2_TTBR1_high (c2_TTBR1 + 1) /* TTBR1 top 32 bits */
>> -#define c2_TTBCR (TCR_EL1 * 2) /* Translation Table Base Control R. */
>> -#define c3_DACR (DACR32_EL2 * 2)/* Domain Access Control Register */
>> -#define c5_DFSR (ESR_EL1 * 2) /* Data Fault Status Register */
>> -#define c5_IFSR (IFSR32_EL2 * 2)/* Instruction Fault Status Register */
>> -#define c5_ADFSR (AFSR0_EL1 * 2) /* Auxiliary Data Fault Status R */
>> -#define c5_AIFSR (AFSR1_EL1 * 2) /* Auxiliary Instr Fault Status R */
>> -#define c6_DFAR (FAR_EL1 * 2) /* Data Fault Address Register */
>> -#define c6_IFAR (c6_DFAR + 1) /* Instruction Fault Address Register */
>> -#define c7_PAR (PAR_EL1 * 2) /* Physical Address Register */
>> -#define c7_PAR_high (c7_PAR + 1) /* PAR top 32 bits */
>> -#define c10_PRRR (MAIR_EL1 * 2) /* Primary Region Remap Register */
>> -#define c10_NMRR (c10_PRRR + 1) /* Normal Memory Remap Register */
>> -#define c12_VBAR (VBAR_EL1 * 2) /* Vector Base Address Register */
>> -#define c13_CID (CONTEXTIDR_EL1 * 2) /* Context ID Register */
>> -#define c13_TID_URW (TPIDR_EL0 * 2) /* Thread ID, User R/W */
>> -#define c13_TID_URO (TPIDRRO_EL0 * 2)/* Thread ID, User R/O */
>> -#define c13_TID_PRIV (TPIDR_EL1 * 2) /* Thread ID, Privileged */
>> -#define c10_AMAIR0 (AMAIR_EL1 * 2) /* Aux Memory Attr Indirection Reg */
>> -#define c10_AMAIR1 (c10_AMAIR0 + 1)/* Aux Memory Attr Indirection Reg */
>> -#define c14_CNTKCTL (CNTKCTL_EL1 * 2) /* Timer Control Register (PL1) */
>> -
>> -#define cp14_DBGDSCRext (MDSCR_EL1 * 2)
>> -#define cp14_DBGBCR0 (DBGBCR0_EL1 * 2)
>> -#define cp14_DBGBVR0 (DBGBVR0_EL1 * 2)
>> -#define cp14_DBGBXVR0 (cp14_DBGBVR0 + 1)
>> -#define cp14_DBGWCR0 (DBGWCR0_EL1 * 2)
>> -#define cp14_DBGWVR0 (DBGWVR0_EL1 * 2)
>> -#define cp14_DBGDCCINT (MDCCINT_EL1 * 2)
>> -
>> -#define NR_COPRO_REGS (NR_SYS_REGS * 2)
>> -
>> #define ARM_EXCEPTION_IRQ 0
>> #define ARM_EXCEPTION_TRAP 1
>>
>> diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h
>> index 3ca894e..170e17d 100644
>> --- a/arch/arm64/include/asm/kvm_emulate.h
>> +++ b/arch/arm64/include/asm/kvm_emulate.h
>> @@ -26,7 +26,6 @@
>>
>> #include <asm/esr.h>
>> #include <asm/kvm_arm.h>
>> -#include <asm/kvm_asm.h>
>> #include <asm/kvm_mmio.h>
>> #include <asm/ptrace.h>
>> #include <asm/cputype.h>
>> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
>> index a35ce72..2fae2d4 100644
>> --- a/arch/arm64/include/asm/kvm_host.h
>> +++ b/arch/arm64/include/asm/kvm_host.h
>> @@ -25,7 +25,6 @@
>> #include <linux/types.h>
>> #include <linux/kvm_types.h>
>> #include <asm/kvm.h>
>> -#include <asm/kvm_asm.h>
>> #include <asm/kvm_mmio.h>
>>
>> #define __KVM_HAVE_ARCH_INTC_INITIALIZED
>> @@ -85,6 +84,86 @@ struct kvm_vcpu_fault_info {
>> u64 hpfar_el2; /* Hyp IPA Fault Address Register */
>> };
>>
>> +/*
>> + * 0 is reserved as an invalid value.
>> + * Order *must* be kept in sync with the hyp switch code.
>
> do we still have such an ordering requirement?
Hmmm. Not as strong as it was before. For performance, it is better to
keep it in order (prefetcher, silly things like that), but this is in no
way an absolute requirement.
I'll amend the comment (or even drop it).
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore
2015-12-02 11:53 ` Christoffer Dall
@ 2015-12-02 15:29 ` Marc Zyngier
2015-12-02 16:19 ` Christoffer Dall
0 siblings, 1 reply; 88+ messages in thread
From: Marc Zyngier @ 2015-12-02 15:29 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 11:53, Christoffer Dall wrote:
> On Fri, Nov 27, 2015 at 06:50:06PM +0000, Marc Zyngier wrote:
>> Implement the fpsimd save restore, keeping the lazy part in
>> assembler (as returning to C would be overkill).
>>
>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
>> ---
>> arch/arm64/kvm/hyp/Makefile | 1 +
>> arch/arm64/kvm/hyp/entry.S | 32 +++++++++++++++++++++++++++++++-
>> arch/arm64/kvm/hyp/fpsimd.S | 33 +++++++++++++++++++++++++++++++++
>> arch/arm64/kvm/hyp/hyp.h | 7 +++++++
>> arch/arm64/kvm/hyp/switch.c | 8 ++++++++
>> arch/arm64/kvm/hyp/sysreg-sr.c | 2 +-
>> 6 files changed, 81 insertions(+), 2 deletions(-)
>> create mode 100644 arch/arm64/kvm/hyp/fpsimd.S
>>
>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>> index 9c11b0f..56238d0 100644
>> --- a/arch/arm64/kvm/hyp/Makefile
>> +++ b/arch/arm64/kvm/hyp/Makefile
>> @@ -9,3 +9,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
>> obj-$(CONFIG_KVM_ARM_HOST) += entry.o
>> obj-$(CONFIG_KVM_ARM_HOST) += switch.o
>> +obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
>> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
>> index 2c4449a..7552922 100644
>> --- a/arch/arm64/kvm/hyp/entry.S
>> +++ b/arch/arm64/kvm/hyp/entry.S
>> @@ -27,6 +27,7 @@
>>
>> #define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
>> #define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
>> +#define CPU_SYSREG_OFFSET(x) (CPU_SYSREGS + 8*x)
>>
>> .text
>> .pushsection .hyp.text, "ax"
>> @@ -152,4 +153,33 @@ ENTRY(__guest_exit)
>> ret
>> ENDPROC(__guest_exit)
>>
>> - /* Insert fault handling here */
>> +ENTRY(__fpsimd_guest_restore)
>> + push x4, lr
>> +
>> + mrs x2, cptr_el2
>> + bic x2, x2, #CPTR_EL2_TFP
>> + msr cptr_el2, x2
>> + isb
>> +
>> + mrs x3, tpidr_el2
>> +
>> + ldr x0, [x3, #VCPU_HOST_CONTEXT]
>> + kern_hyp_va x0
>> + add x0, x0, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
>> + bl __fpsimd_save_state
>> +
>> + add x2, x3, #VCPU_CONTEXT
>> + add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
>> + bl __fpsimd_restore_state
>> +
>> + mrs x1, hcr_el2
>> + tbnz x1, #HCR_RW_SHIFT, 1f
>
> nit: Add a comment along the lines of:
> // Skip restoring fpexc32 for AArch64 guests
>
>> + ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
>> + msr fpexc32_el2, x4
>> +1:
>> + pop x4, lr
>> + pop x2, x3
>> + pop x0, x1
>> +
>> + eret
>> +ENDPROC(__fpsimd_guest_restore)
>> diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S
>> new file mode 100644
>> index 0000000..da3f22c
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/fpsimd.S
>> @@ -0,0 +1,33 @@
>> +/*
>> + * Copyright (C) 2015 - ARM Ltd
>> + * Author: Marc Zyngier <marc.zyngier@arm.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU General Public License version 2 as
>> + * published by the Free Software Foundation.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
>> + * GNU General Public License for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License
>> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <linux/linkage.h>
>> +
>> +#include <asm/fpsimdmacros.h>
>> +
>> + .text
>> + .pushsection .hyp.text, "ax"
>> +
>> +ENTRY(__fpsimd_save_state)
>> + fpsimd_save x0, 1
>> + ret
>> +ENDPROC(__fpsimd_save_state)
>> +
>> +ENTRY(__fpsimd_restore_state)
>> + fpsimd_restore x0, 1
>> + ret
>> +ENDPROC(__fpsimd_restore_state)
>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
>> index f0427ee..18365dd 100644
>> --- a/arch/arm64/kvm/hyp/hyp.h
>> +++ b/arch/arm64/kvm/hyp/hyp.h
>> @@ -66,6 +66,13 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
>> void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
>> void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
>>
>> +void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
>> +void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
>> +static inline bool __fpsimd_enabled(void)
>> +{
>> + return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
>> +}
>> +
>> u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
>>
>> #endif /* __ARM64_KVM_HYP_H__ */
>> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
>> index d67ed9e..8affc19 100644
>> --- a/arch/arm64/kvm/hyp/switch.c
>> +++ b/arch/arm64/kvm/hyp/switch.c
>> @@ -88,6 +88,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
>> {
>> struct kvm_cpu_context *host_ctxt;
>> struct kvm_cpu_context *guest_ctxt;
>> + bool fp_enabled;
>> u64 exit_code;
>>
>> vcpu = kern_hyp_va(vcpu);
>> @@ -117,6 +118,8 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
>> exit_code = __guest_enter(vcpu, host_ctxt);
>> /* And we're baaack! */
>>
>> + fp_enabled = __fpsimd_enabled();
>> +
>
> what does 'enabled' really mean here? Isn't it really
> __fpsimd_is_dirty() or __fpsimd_is_guest() or something like that (I
> suck at naming too).
It really means "the FP regs are accessible and won't explode in your
face". It doesn't necessary means dirty (though that's the way we use it
below.
Your call, really.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-12-02 13:19 ` Marc Zyngier
@ 2015-12-02 16:19 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 16:19 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Dec 02, 2015 at 01:19:22PM +0000, Marc Zyngier wrote:
> On 02/12/15 11:53, Christoffer Dall wrote:
> > On Wed, Dec 02, 2015 at 09:47:43AM +0000, Marc Zyngier wrote:
> >> On 02/12/15 09:27, Christoffer Dall wrote:
> >>> On Tue, Dec 01, 2015 at 06:51:00PM +0000, Marc Zyngier wrote:
> >>>> On 01/12/15 15:39, Christoffer Dall wrote:
> >>>>> On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
> >>>>>> KVM so far relies on code patching, and is likely to use it more
> >>>>>> in the future. The main issue is that our alternative system works
> >>>>>> at the instruction level, while we'd like to have alternatives at
> >>>>>> the function level.
> >>>>>>
> >>>>>> In order to cope with this, add the "hyp_alternate_select" macro that
> >>>>>> outputs a brief sequence of code that in turn can be patched, allowing
> >>>>>> al alternative function to be selected.
> >>>>>
> >>>>> s/al/an/ ?
> >>>>>
> >>>>>>
> >>>>>> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >>>>>> ---
> >>>>>> arch/arm64/kvm/hyp/hyp.h | 16 ++++++++++++++++
> >>>>>> 1 file changed, 16 insertions(+)
> >>>>>>
> >>>>>> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> >>>>>> index 7ac8e11..f0427ee 100644
> >>>>>> --- a/arch/arm64/kvm/hyp/hyp.h
> >>>>>> +++ b/arch/arm64/kvm/hyp/hyp.h
> >>>>>> @@ -27,6 +27,22 @@
> >>>>>>
> >>>>>> #define kern_hyp_va(v) (typeof(v))((unsigned long)v & HYP_PAGE_OFFSET_MASK)
> >>>>>>
> >>>>>> +/*
> >>>>>> + * Generates patchable code sequences that are used to switch between
> >>>>>> + * two implementations of a function, depending on the availability of
> >>>>>> + * a feature.
> >>>>>> + */
> >>>>>
> >>>>> This looks right to me, but I'm a bit unclear what the types of this is
> >>>>> and how to use it.
> >>>>>
> >>>>> Are orig and alt function pointers and cond is a CONFIG_FOO ? fname is
> >>>>> a symbol, which is defined as a prototype somewhere and then implemented
> >>>>> here, or?
> >>>>>
> >>>>> Perhaps a Usage: part of the docs would be helpful.
> >>>>
> >>>> How about:
> >>>>
> >>>> @fname: a symbol name that will be defined as a function returning a
> >>>> function pointer whose type will match @orig and @alt
> >>>> @orig: A pointer to the default function, as returned by @fname when
> >>>> @cond doesn't hold
> >>>> @alt: A pointer to the alternate function, as returned by @fname when
> >>>> @cond holds
> >>>> @cond: a CPU feature (as described in asm/cpufeature.h)
> >>>
> >>> looks good.
> >>>
> >>>>
> >>>>>
> >>>>>> +#define hyp_alternate_select(fname, orig, alt, cond) \
> >>>>>> +typeof(orig) * __hyp_text fname(void) \
> >>>>>> +{ \
> >>>>>> + typeof(alt) *val = orig; \
> >>>>>> + asm volatile(ALTERNATIVE("nop \n", \
> >>>>>> + "mov %0, %1 \n", \
> >>>>>> + cond) \
> >>>>>> + : "+r" (val) : "r" (alt)); \
> >>>>>> + return val; \
> >>>>>> +}
> >>>>>> +
> >>>>>> void __vgic_v2_save_state(struct kvm_vcpu *vcpu);
> >>>>>> void __vgic_v2_restore_state(struct kvm_vcpu *vcpu);
> >>>>>>
> >>>>>> --
> >>>>>> 2.1.4
> >>>>>>
> >>>>>
> >>>>> I haven't thought much about how all of this is implemented, but from my
> >>>>> point of views the ideal situation would be something like:
> >>>>>
> >>>>> void foo(int a, int b)
> >>>>> {
> >>>>> ALTERNATIVE_IF_NOT CONFIG_BAR
> >>>>> foo_legacy(a, b);
> >>>>> ALTERNATIVE_ELSE
> >>>>> foo_new(a, b);
> >>>>> ALTERNATIVE_END
> >>>>> }
> >>>>>
> >>>>> I realize this may be impossible because the C code could implement all
> >>>>> sort of fun stuff around the actual function calls, but would there be
> >>>>> some way to annotate the functions and find the actual branch statement
> >>>>> and change the target?
> >>>>
> >>>> The main issue is that C doesn't give you any access to the branch
> >>>> function itself, except for the asm-goto statements. It also makes it
> >>>> very hard to preserve the return type. For your idea to work, we'd need
> >>>> some support in the compiler itself. I'm sure that it is doable, just
> >>>> not by me! ;-)
> >>>
> >>> Not by me either, I'm just asking stupid questions - as always.
> >>
> >> I don't find that stupid. Asking that kind of stuff is useful to put
> >> things in perspective.
> >>
> >
> > Thanks!
> >
> >>>>
> >>>> This is why I've ended up creating something that returns a function
> >>>> *pointer*, because that's something that exists in the language (no new
> >>>> concept). I simply made sure I could return it at minimal cost.
> >>>>
> >>>
> >>> I don't have a problem with this either. I'm curious though, how much
> >>> of a performance improvement (and why) we get from doing this as opposed
> >>> to a simple if-statement?
> >>
> >> An if statement will involve fetching some configuration from memory.
> >> You can do that, but you are going to waste a cache line and memory
> >> bandwidth (both which are scarce resources) for something that never
> >> ever changes over the life of the system. These things tend to accumulate.
> >
> > Sure, but won't you be fetching the function pointer from memory anyway?
>
> No, and that's the whole point of this patch: the function pointers are
> loaded into registers as PC-relative constants (adrp+add), the selection
> being done by a mov or a nop. For example:
>
> ffffffc0007f1f60: 90000001 adrp x1, ffffffc0007f1000
> ffffffc0007f1f64: 90000000 adrp x0, ffffffc0007f1000
> ffffffc0007f1f68: 91036021 add x1, x1, #0xd8
> ffffffc0007f1f6c: 910b0000 add x0, x0, #0x2c0
> ffffffc0007f1f70: d503201f nop
> ffffffc0007f1f74: aa1303e0 mov x0, x19
> ffffffc0007f1f78: d63f0020 blr x1
right, looking at the disassembly was a good idea.
>
> For the default condition (the above code), the CPU is likely to discard
> the second adrp+add (because of the mov x0, x19). For the alternate, the
> nop is replaced by a "mov x1, x0", which makes the first adrp+add
> irrelevant (it will be eliminated in the pipeline of any decent CPU).
>
> While this has a cost in terms of instruction footprint, the branch
> predictor is quickly going to find out where we're branching. We also
> avoid fetching both from the I and D sides, which could kill the branch
> predictor if not speculated in time. In the end, we get something that
> is a lot more predictable, even for simpler CPU designs.
>
consider me convinced. Thanks for the in-depth!
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore
2015-12-02 15:29 ` Marc Zyngier
@ 2015-12-02 16:19 ` Christoffer Dall
0 siblings, 0 replies; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 16:19 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Dec 02, 2015 at 03:29:50PM +0000, Marc Zyngier wrote:
> On 02/12/15 11:53, Christoffer Dall wrote:
> > On Fri, Nov 27, 2015 at 06:50:06PM +0000, Marc Zyngier wrote:
> >> Implement the fpsimd save restore, keeping the lazy part in
> >> assembler (as returning to C would be overkill).
> >>
> >> Signed-off-by: Marc Zyngier <marc.zyngier@arm.com>
> >> ---
> >> arch/arm64/kvm/hyp/Makefile | 1 +
> >> arch/arm64/kvm/hyp/entry.S | 32 +++++++++++++++++++++++++++++++-
> >> arch/arm64/kvm/hyp/fpsimd.S | 33 +++++++++++++++++++++++++++++++++
> >> arch/arm64/kvm/hyp/hyp.h | 7 +++++++
> >> arch/arm64/kvm/hyp/switch.c | 8 ++++++++
> >> arch/arm64/kvm/hyp/sysreg-sr.c | 2 +-
> >> 6 files changed, 81 insertions(+), 2 deletions(-)
> >> create mode 100644 arch/arm64/kvm/hyp/fpsimd.S
> >>
> >> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> >> index 9c11b0f..56238d0 100644
> >> --- a/arch/arm64/kvm/hyp/Makefile
> >> +++ b/arch/arm64/kvm/hyp/Makefile
> >> @@ -9,3 +9,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o
> >> obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o
> >> obj-$(CONFIG_KVM_ARM_HOST) += entry.o
> >> obj-$(CONFIG_KVM_ARM_HOST) += switch.o
> >> +obj-$(CONFIG_KVM_ARM_HOST) += fpsimd.o
> >> diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S
> >> index 2c4449a..7552922 100644
> >> --- a/arch/arm64/kvm/hyp/entry.S
> >> +++ b/arch/arm64/kvm/hyp/entry.S
> >> @@ -27,6 +27,7 @@
> >>
> >> #define CPU_GP_REG_OFFSET(x) (CPU_GP_REGS + x)
> >> #define CPU_XREG_OFFSET(x) CPU_GP_REG_OFFSET(CPU_USER_PT_REGS + 8*x)
> >> +#define CPU_SYSREG_OFFSET(x) (CPU_SYSREGS + 8*x)
> >>
> >> .text
> >> .pushsection .hyp.text, "ax"
> >> @@ -152,4 +153,33 @@ ENTRY(__guest_exit)
> >> ret
> >> ENDPROC(__guest_exit)
> >>
> >> - /* Insert fault handling here */
> >> +ENTRY(__fpsimd_guest_restore)
> >> + push x4, lr
> >> +
> >> + mrs x2, cptr_el2
> >> + bic x2, x2, #CPTR_EL2_TFP
> >> + msr cptr_el2, x2
> >> + isb
> >> +
> >> + mrs x3, tpidr_el2
> >> +
> >> + ldr x0, [x3, #VCPU_HOST_CONTEXT]
> >> + kern_hyp_va x0
> >> + add x0, x0, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> >> + bl __fpsimd_save_state
> >> +
> >> + add x2, x3, #VCPU_CONTEXT
> >> + add x0, x2, #CPU_GP_REG_OFFSET(CPU_FP_REGS)
> >> + bl __fpsimd_restore_state
> >> +
> >> + mrs x1, hcr_el2
> >> + tbnz x1, #HCR_RW_SHIFT, 1f
> >
> > nit: Add a comment along the lines of:
> > // Skip restoring fpexc32 for AArch64 guests
> >
> >> + ldr x4, [x2, #CPU_SYSREG_OFFSET(FPEXC32_EL2)]
> >> + msr fpexc32_el2, x4
> >> +1:
> >> + pop x4, lr
> >> + pop x2, x3
> >> + pop x0, x1
> >> +
> >> + eret
> >> +ENDPROC(__fpsimd_guest_restore)
> >> diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S
> >> new file mode 100644
> >> index 0000000..da3f22c
> >> --- /dev/null
> >> +++ b/arch/arm64/kvm/hyp/fpsimd.S
> >> @@ -0,0 +1,33 @@
> >> +/*
> >> + * Copyright (C) 2015 - ARM Ltd
> >> + * Author: Marc Zyngier <marc.zyngier@arm.com>
> >> + *
> >> + * This program is free software; you can redistribute it and/or modify
> >> + * it under the terms of the GNU General Public License version 2 as
> >> + * published by the Free Software Foundation.
> >> + *
> >> + * This program is distributed in the hope that it will be useful,
> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> >> + * GNU General Public License for more details.
> >> + *
> >> + * You should have received a copy of the GNU General Public License
> >> + * along with this program. If not, see <http://www.gnu.org/licenses/>.
> >> + */
> >> +
> >> +#include <linux/linkage.h>
> >> +
> >> +#include <asm/fpsimdmacros.h>
> >> +
> >> + .text
> >> + .pushsection .hyp.text, "ax"
> >> +
> >> +ENTRY(__fpsimd_save_state)
> >> + fpsimd_save x0, 1
> >> + ret
> >> +ENDPROC(__fpsimd_save_state)
> >> +
> >> +ENTRY(__fpsimd_restore_state)
> >> + fpsimd_restore x0, 1
> >> + ret
> >> +ENDPROC(__fpsimd_restore_state)
> >> diff --git a/arch/arm64/kvm/hyp/hyp.h b/arch/arm64/kvm/hyp/hyp.h
> >> index f0427ee..18365dd 100644
> >> --- a/arch/arm64/kvm/hyp/hyp.h
> >> +++ b/arch/arm64/kvm/hyp/hyp.h
> >> @@ -66,6 +66,13 @@ void __debug_restore_state(struct kvm_vcpu *vcpu,
> >> void __debug_cond_save_host_state(struct kvm_vcpu *vcpu);
> >> void __debug_cond_restore_host_state(struct kvm_vcpu *vcpu);
> >>
> >> +void __fpsimd_save_state(struct user_fpsimd_state *fp_regs);
> >> +void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs);
> >> +static inline bool __fpsimd_enabled(void)
> >> +{
> >> + return !(read_sysreg(cptr_el2) & CPTR_EL2_TFP);
> >> +}
> >> +
> >> u64 __guest_enter(struct kvm_vcpu *vcpu, struct kvm_cpu_context *host_ctxt);
> >>
> >> #endif /* __ARM64_KVM_HYP_H__ */
> >> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> >> index d67ed9e..8affc19 100644
> >> --- a/arch/arm64/kvm/hyp/switch.c
> >> +++ b/arch/arm64/kvm/hyp/switch.c
> >> @@ -88,6 +88,7 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> >> {
> >> struct kvm_cpu_context *host_ctxt;
> >> struct kvm_cpu_context *guest_ctxt;
> >> + bool fp_enabled;
> >> u64 exit_code;
> >>
> >> vcpu = kern_hyp_va(vcpu);
> >> @@ -117,6 +118,8 @@ int __hyp_text __guest_run(struct kvm_vcpu *vcpu)
> >> exit_code = __guest_enter(vcpu, host_ctxt);
> >> /* And we're baaack! */
> >>
> >> + fp_enabled = __fpsimd_enabled();
> >> +
> >
> > what does 'enabled' really mean here? Isn't it really
> > __fpsimd_is_dirty() or __fpsimd_is_guest() or something like that (I
> > suck at naming too).
>
> It really means "the FP regs are accessible and won't explode in your
> face". It doesn't necessary means dirty (though that's the way we use it
> below.
>
> Your call, really.
>
meh, it's fine the way it is. Enabled for the guest I suppose is valid
enough :)
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 21/21] arm64: KVM: Remove weak attributes
2015-12-02 15:21 ` Marc Zyngier
@ 2015-12-02 16:21 ` Christoffer Dall
2015-12-02 17:52 ` Marc Zyngier
0 siblings, 1 reply; 88+ messages in thread
From: Christoffer Dall @ 2015-12-02 16:21 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Dec 02, 2015 at 03:21:49PM +0000, Marc Zyngier wrote:
> On 02/12/15 11:47, Christoffer Dall wrote:
> > On Fri, Nov 27, 2015 at 06:50:15PM +0000, Marc Zyngier wrote:
> >> As we've now switched to the new world switch implementation,
> >> remove the weak attributes, as nobody is supposed to override
> >> it anymore.
> >
> > Why not remove the aliases and change the callers?
>
> This is likely to be a bigger patch, and it would affect the 32bit as
> well. So far, I'm choosing to keep things the same. Another solution
> would be to completely drop the aliases, and just rename the new
> function to have the old names.
>
> I don't mind either way.
>
I didn't think of the 32-bit side. I think eventually we should get rid
of the aliases and just have the funcitons named as they are called, but
there's no rush. We can wait until we've the done the 32-bit side if
you prefer.
-Christoffer
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 21/21] arm64: KVM: Remove weak attributes
2015-12-02 16:21 ` Christoffer Dall
@ 2015-12-02 17:52 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-02 17:52 UTC (permalink / raw)
To: linux-arm-kernel
On 02/12/15 16:21, Christoffer Dall wrote:
> On Wed, Dec 02, 2015 at 03:21:49PM +0000, Marc Zyngier wrote:
>> On 02/12/15 11:47, Christoffer Dall wrote:
>>> On Fri, Nov 27, 2015 at 06:50:15PM +0000, Marc Zyngier wrote:
>>>> As we've now switched to the new world switch implementation,
>>>> remove the weak attributes, as nobody is supposed to override
>>>> it anymore.
>>>
>>> Why not remove the aliases and change the callers?
>>
>> This is likely to be a bigger patch, and it would affect the 32bit as
>> well. So far, I'm choosing to keep things the same. Another solution
>> would be to completely drop the aliases, and just rename the new
>> function to have the old names.
>>
>> I don't mind either way.
>>
> I didn't think of the 32-bit side. I think eventually we should get rid
> of the aliases and just have the funcitons named as they are called, but
> there's no rush. We can wait until we've the done the 32-bit side if
> you prefer.
Probably safest. We're already changing a lot of things. Hopefully for
4.6 I'll have the 32bit side sorted.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-11-27 18:50 ` [PATCH v2 10/21] arm64: KVM: Add patchable function selector Marc Zyngier
2015-12-01 15:39 ` Christoffer Dall
@ 2015-12-02 22:34 ` Andrew Jones
2015-12-03 8:18 ` Marc Zyngier
1 sibling, 1 reply; 88+ messages in thread
From: Andrew Jones @ 2015-12-02 22:34 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
> KVM so far relies on code patching, and is likely to use it more
> in the future. The main issue is that our alternative system works
> at the instruction level, while we'd like to have alternatives at
> the function level.
How about setting static-keys at hyp init time?
Thanks,
drew
^ permalink raw reply [flat|nested] 88+ messages in thread
* [PATCH v2 10/21] arm64: KVM: Add patchable function selector
2015-12-02 22:34 ` Andrew Jones
@ 2015-12-03 8:18 ` Marc Zyngier
0 siblings, 0 replies; 88+ messages in thread
From: Marc Zyngier @ 2015-12-03 8:18 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, 2 Dec 2015 16:34:56 -0600
Andrew Jones <drjones@redhat.com> wrote:
> On Fri, Nov 27, 2015 at 06:50:04PM +0000, Marc Zyngier wrote:
> > KVM so far relies on code patching, and is likely to use it more
> > in the future. The main issue is that our alternative system works
> > at the instruction level, while we'd like to have alternatives at
> > the function level.
>
> How about setting static-keys at hyp init time?
That was an option I looked at. And while static keys would work to some
extent, they also have some nasty side effects:
- They create both a fast and a slow path. We don't want that - both
path should be equally fast, or at least have as little overhead as
possible
- We do need code patching for some assembly code, and using static
keys on top creates a parallel mechanism that makes it hard to
follow/debug/maintain.
You can view this alternative function call as a slightly different
kind of static keys - one that can give you the capability to handle
function calls instead of just jumping over code sequences. Both have
their own merits.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
^ permalink raw reply [flat|nested] 88+ messages in thread
end of thread, other threads:[~2015-12-03 8:18 UTC | newest]
Thread overview: 88+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-27 18:49 [PATCH v2 00/21] arm64: KVM: world switch in C Marc Zyngier
2015-11-27 18:49 ` [PATCH v2 01/21] arm64: Add macros to read/write system registers Marc Zyngier
2015-11-30 20:00 ` Christoffer Dall
2015-11-27 18:49 ` [PATCH v2 02/21] arm64: KVM: Add a HYP-specific header file Marc Zyngier
2015-11-30 20:00 ` Christoffer Dall
2015-12-01 11:41 ` Marc Zyngier
2015-12-01 11:47 ` Christoffer Dall
2015-11-27 18:49 ` [PATCH v2 03/21] arm64: KVM: Implement vgic-v2 save/restore Marc Zyngier
2015-11-30 20:00 ` Christoffer Dall
2015-12-01 11:39 ` Marc Zyngier
2015-11-27 18:49 ` [PATCH v2 04/21] arm64: KVM: Implement vgic-v3 save/restore Marc Zyngier
2015-11-30 9:59 ` Alex Bennée
2015-11-30 10:43 ` Marc Zyngier
2015-11-30 19:50 ` Christoffer Dall
2015-12-01 11:32 ` Marc Zyngier
2015-12-01 11:44 ` Christoffer Dall
2015-12-01 11:50 ` Christoffer Dall
2015-12-01 11:57 ` Marc Zyngier
2015-12-01 12:24 ` Christoffer Dall
2015-12-01 12:49 ` Marc Zyngier
2015-12-01 11:54 ` Marc Zyngier
2015-11-27 18:49 ` [PATCH v2 05/21] arm64: KVM: Implement timer save/restore Marc Zyngier
2015-11-30 19:59 ` Christoffer Dall
2015-12-01 11:34 ` Marc Zyngier
2015-11-27 18:50 ` [PATCH v2 06/21] arm64: KVM: Implement system register save/restore Marc Zyngier
2015-12-01 15:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 07/21] arm64: KVM: Implement 32bit " Marc Zyngier
2015-12-01 15:52 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 08/21] arm64: KVM: Implement debug save/restore Marc Zyngier
2015-11-30 12:00 ` Alex Bennée
2015-11-30 12:24 ` Marc Zyngier
2015-12-01 12:56 ` Christoffer Dall
2015-12-01 13:06 ` Marc Zyngier
2015-12-01 13:19 ` Alex Bennée
2015-12-01 13:34 ` Marc Zyngier
2015-12-01 14:47 ` Christoffer Dall
2015-12-01 14:56 ` Christoffer Dall
2015-12-01 15:01 ` Marc Zyngier
2015-12-01 15:41 ` Christoffer Dall
2015-12-01 18:34 ` Marc Zyngier
2015-11-27 18:50 ` [PATCH v2 09/21] arm64: KVM: Implement guest entry Marc Zyngier
2015-12-01 15:29 ` Christoffer Dall
2015-12-01 18:41 ` Marc Zyngier
2015-11-27 18:50 ` [PATCH v2 10/21] arm64: KVM: Add patchable function selector Marc Zyngier
2015-12-01 15:39 ` Christoffer Dall
2015-12-01 18:51 ` Marc Zyngier
2015-12-02 9:27 ` Christoffer Dall
2015-12-02 9:47 ` Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-12-02 13:19 ` Marc Zyngier
2015-12-02 16:19 ` Christoffer Dall
2015-12-02 22:34 ` Andrew Jones
2015-12-03 8:18 ` Marc Zyngier
2015-11-27 18:50 ` [PATCH v2 11/21] arm64: KVM: Implement the core world switch Marc Zyngier
2015-12-01 15:55 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 12/21] arm64: KVM: Implement fpsimd save/restore Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-12-02 15:29 ` Marc Zyngier
2015-12-02 16:19 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 13/21] arm64: KVM: Implement TLB handling Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 14/21] arm64: KVM: HYP mode entry points Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 15/21] arm64: KVM: Add panic handling Marc Zyngier
2015-12-02 11:53 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 16/21] arm64: KVM: Add compatibility aliases Marc Zyngier
2015-12-02 11:49 ` Christoffer Dall
2015-12-02 15:23 ` Marc Zyngier
2015-11-27 18:50 ` [PATCH v2 17/21] arm64: KVM: Map the kernel RO section into HYP Marc Zyngier
2015-12-02 11:49 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 18/21] arm64: KVM: Move away from the assembly version of the world switch Marc Zyngier
2015-12-02 11:49 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 19/21] arm64: KVM: Turn system register numbers to an enum Marc Zyngier
2015-12-02 11:51 ` Christoffer Dall
2015-12-02 15:26 ` Marc Zyngier
2015-11-27 18:50 ` [PATCH v2 20/21] arm64: KVM: Cleanup asm-offset.c Marc Zyngier
2015-12-02 11:51 ` Christoffer Dall
2015-11-27 18:50 ` [PATCH v2 21/21] arm64: KVM: Remove weak attributes Marc Zyngier
2015-12-02 11:47 ` Christoffer Dall
2015-12-02 15:21 ` Marc Zyngier
2015-12-02 16:21 ` Christoffer Dall
2015-12-02 17:52 ` Marc Zyngier
2015-11-30 20:33 ` [PATCH v2 00/21] arm64: KVM: world switch in C Christoffer Dall
2015-12-01 3:19 ` Mario Smarduch
2015-12-01 9:58 ` Marc Zyngier
2015-12-01 12:00 ` Christoffer Dall
2015-12-01 17:51 ` Marc Zyngier
2015-12-01 19:34 ` Christoffer Dall
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).