From: Marc Zyngier <marc.zyngier@arm.com> To: Paolo Bonzini <pbonzini@redhat.com> Cc: "Christoffer Dall" <christoffer.dall@linaro.org>, "Gleb Natapov" <gleb@kernel.org>, "Alex Bennée" <alex.bennee@linaro.org>, "Amit Singh Tomar" <amittomer25@gmail.com>, "Catalin Marinas" <catalin.marinas@arm.com>, "David Binderman" <dcb314@hotmail.com>, "Fengguang Wu" <fengguang.wu@intel.com>, "Jisheng Zhang" <jszhang@marvell.com>, "Mark Rutland" <mark.rutland@arm.com>, "Pavel Fedin" <p.fedin@samsung.com>, "Vladimir Murzin" <vladimir.murzin@arm.com>, kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: [PATCH 14/31] arm64: KVM: Implement the core world switch Date: Thu, 24 Dec 2015 11:12:22 +0000 [thread overview] Message-ID: <1450955559-15639-15-git-send-email-marc.zyngier@arm.com> (raw) In-Reply-To: <1450955559-15639-1-git-send-email-marc.zyngier@arm.com> Implement the core of the world switch in C. Not everything is there yet, and there is nothing to re-enter the world switch either. But this already outlines the code structure well enough. Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org> --- arch/arm64/kvm/hyp/Makefile | 1 + arch/arm64/kvm/hyp/switch.c | 135 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 136 insertions(+) create mode 100644 arch/arm64/kvm/hyp/switch.c diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile index 1e1ff06..9c11b0f 100644 --- a/arch/arm64/kvm/hyp/Makefile +++ b/arch/arm64/kvm/hyp/Makefile @@ -8,3 +8,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o obj-$(CONFIG_KVM_ARM_HOST) += entry.o +obj-$(CONFIG_KVM_ARM_HOST) += switch.o diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c new file mode 100644 index 0000000..79f59c9 --- /dev/null +++ b/arch/arm64/kvm/hyp/switch.c @@ -0,0 +1,135 @@ +/* + * Copyright (C) 2015 - ARM Ltd + * Author: Marc Zyngier <marc.zyngier@arm.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#include "hyp.h" + +static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu) +{ + u64 val; + + /* + * We are about to set CPTR_EL2.TFP to trap all floating point + * register accesses to EL2, however, the ARM ARM clearly states that + * traps are only taken to EL2 if the operation would not otherwise + * trap to EL1. Therefore, always make sure that for 32-bit guests, + * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit. + */ + val = vcpu->arch.hcr_el2; + if (!(val & HCR_RW)) { + write_sysreg(1 << 30, fpexc32_el2); + isb(); + } + write_sysreg(val, hcr_el2); + /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */ + write_sysreg(1 << 15, hstr_el2); + write_sysreg(CPTR_EL2_TTA | CPTR_EL2_TFP, cptr_el2); + write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2); +} + +static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu) +{ + write_sysreg(HCR_RW, hcr_el2); + write_sysreg(0, hstr_el2); + write_sysreg(read_sysreg(mdcr_el2) & MDCR_EL2_HPMN_MASK, mdcr_el2); + write_sysreg(0, cptr_el2); +} + +static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu) +{ + struct kvm *kvm = kern_hyp_va(vcpu->kvm); + write_sysreg(kvm->arch.vttbr, vttbr_el2); +} + +static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu) +{ + write_sysreg(0, vttbr_el2); +} + +static hyp_alternate_select(__vgic_call_save_state, + __vgic_v2_save_state, __vgic_v3_save_state, + ARM64_HAS_SYSREG_GIC_CPUIF); + +static hyp_alternate_select(__vgic_call_restore_state, + __vgic_v2_restore_state, __vgic_v3_restore_state, + ARM64_HAS_SYSREG_GIC_CPUIF); + +static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu) +{ + __vgic_call_save_state()(vcpu); + write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2); +} + +static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu) +{ + u64 val; + + val = read_sysreg(hcr_el2); + val |= HCR_INT_OVERRIDE; + val |= vcpu->arch.irq_lines; + write_sysreg(val, hcr_el2); + + __vgic_call_restore_state()(vcpu); +} + +int __hyp_text __guest_run(struct kvm_vcpu *vcpu) +{ + struct kvm_cpu_context *host_ctxt; + struct kvm_cpu_context *guest_ctxt; + u64 exit_code; + + vcpu = kern_hyp_va(vcpu); + write_sysreg(vcpu, tpidr_el2); + + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context); + guest_ctxt = &vcpu->arch.ctxt; + + __sysreg_save_state(host_ctxt); + __debug_cond_save_host_state(vcpu); + + __activate_traps(vcpu); + __activate_vm(vcpu); + + __vgic_restore_state(vcpu); + __timer_restore_state(vcpu); + + /* + * We must restore the 32-bit state before the sysregs, thanks + * to Cortex-A57 erratum #852523. + */ + __sysreg32_restore_state(vcpu); + __sysreg_restore_state(guest_ctxt); + __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt); + + /* Jump in the fire! */ + exit_code = __guest_enter(vcpu, host_ctxt); + /* And we're baaack! */ + + __sysreg_save_state(guest_ctxt); + __sysreg32_save_state(vcpu); + __timer_save_state(vcpu); + __vgic_save_state(vcpu); + + __deactivate_traps(vcpu); + __deactivate_vm(vcpu); + + __sysreg_restore_state(host_ctxt); + + __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt); + __debug_cond_restore_host_state(vcpu); + + return exit_code; +} -- 2.1.4
WARNING: multiple messages have this Message-ID (diff)
From: marc.zyngier@arm.com (Marc Zyngier) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH 14/31] arm64: KVM: Implement the core world switch Date: Thu, 24 Dec 2015 11:12:22 +0000 [thread overview] Message-ID: <1450955559-15639-15-git-send-email-marc.zyngier@arm.com> (raw) In-Reply-To: <1450955559-15639-1-git-send-email-marc.zyngier@arm.com> Implement the core of the world switch in C. Not everything is there yet, and there is nothing to re-enter the world switch either. But this already outlines the code structure well enough. Signed-off-by: Marc Zyngier <marc.zyngier@arm.com> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org> --- arch/arm64/kvm/hyp/Makefile | 1 + arch/arm64/kvm/hyp/switch.c | 135 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 136 insertions(+) create mode 100644 arch/arm64/kvm/hyp/switch.c diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile index 1e1ff06..9c11b0f 100644 --- a/arch/arm64/kvm/hyp/Makefile +++ b/arch/arm64/kvm/hyp/Makefile @@ -8,3 +8,4 @@ obj-$(CONFIG_KVM_ARM_HOST) += timer-sr.o obj-$(CONFIG_KVM_ARM_HOST) += sysreg-sr.o obj-$(CONFIG_KVM_ARM_HOST) += debug-sr.o obj-$(CONFIG_KVM_ARM_HOST) += entry.o +obj-$(CONFIG_KVM_ARM_HOST) += switch.o diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c new file mode 100644 index 0000000..79f59c9 --- /dev/null +++ b/arch/arm64/kvm/hyp/switch.c @@ -0,0 +1,135 @@ +/* + * Copyright (C) 2015 - ARM Ltd + * Author: Marc Zyngier <marc.zyngier@arm.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see <http://www.gnu.org/licenses/>. + */ + +#include "hyp.h" + +static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu) +{ + u64 val; + + /* + * We are about to set CPTR_EL2.TFP to trap all floating point + * register accesses to EL2, however, the ARM ARM clearly states that + * traps are only taken to EL2 if the operation would not otherwise + * trap to EL1. Therefore, always make sure that for 32-bit guests, + * we set FPEXC.EN to prevent traps to EL1, when setting the TFP bit. + */ + val = vcpu->arch.hcr_el2; + if (!(val & HCR_RW)) { + write_sysreg(1 << 30, fpexc32_el2); + isb(); + } + write_sysreg(val, hcr_el2); + /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */ + write_sysreg(1 << 15, hstr_el2); + write_sysreg(CPTR_EL2_TTA | CPTR_EL2_TFP, cptr_el2); + write_sysreg(vcpu->arch.mdcr_el2, mdcr_el2); +} + +static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu) +{ + write_sysreg(HCR_RW, hcr_el2); + write_sysreg(0, hstr_el2); + write_sysreg(read_sysreg(mdcr_el2) & MDCR_EL2_HPMN_MASK, mdcr_el2); + write_sysreg(0, cptr_el2); +} + +static void __hyp_text __activate_vm(struct kvm_vcpu *vcpu) +{ + struct kvm *kvm = kern_hyp_va(vcpu->kvm); + write_sysreg(kvm->arch.vttbr, vttbr_el2); +} + +static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu) +{ + write_sysreg(0, vttbr_el2); +} + +static hyp_alternate_select(__vgic_call_save_state, + __vgic_v2_save_state, __vgic_v3_save_state, + ARM64_HAS_SYSREG_GIC_CPUIF); + +static hyp_alternate_select(__vgic_call_restore_state, + __vgic_v2_restore_state, __vgic_v3_restore_state, + ARM64_HAS_SYSREG_GIC_CPUIF); + +static void __hyp_text __vgic_save_state(struct kvm_vcpu *vcpu) +{ + __vgic_call_save_state()(vcpu); + write_sysreg(read_sysreg(hcr_el2) & ~HCR_INT_OVERRIDE, hcr_el2); +} + +static void __hyp_text __vgic_restore_state(struct kvm_vcpu *vcpu) +{ + u64 val; + + val = read_sysreg(hcr_el2); + val |= HCR_INT_OVERRIDE; + val |= vcpu->arch.irq_lines; + write_sysreg(val, hcr_el2); + + __vgic_call_restore_state()(vcpu); +} + +int __hyp_text __guest_run(struct kvm_vcpu *vcpu) +{ + struct kvm_cpu_context *host_ctxt; + struct kvm_cpu_context *guest_ctxt; + u64 exit_code; + + vcpu = kern_hyp_va(vcpu); + write_sysreg(vcpu, tpidr_el2); + + host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context); + guest_ctxt = &vcpu->arch.ctxt; + + __sysreg_save_state(host_ctxt); + __debug_cond_save_host_state(vcpu); + + __activate_traps(vcpu); + __activate_vm(vcpu); + + __vgic_restore_state(vcpu); + __timer_restore_state(vcpu); + + /* + * We must restore the 32-bit state before the sysregs, thanks + * to Cortex-A57 erratum #852523. + */ + __sysreg32_restore_state(vcpu); + __sysreg_restore_state(guest_ctxt); + __debug_restore_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt); + + /* Jump in the fire! */ + exit_code = __guest_enter(vcpu, host_ctxt); + /* And we're baaack! */ + + __sysreg_save_state(guest_ctxt); + __sysreg32_save_state(vcpu); + __timer_save_state(vcpu); + __vgic_save_state(vcpu); + + __deactivate_traps(vcpu); + __deactivate_vm(vcpu); + + __sysreg_restore_state(host_ctxt); + + __debug_save_state(vcpu, kern_hyp_va(vcpu->arch.debug_ptr), guest_ctxt); + __debug_cond_restore_host_state(vcpu); + + return exit_code; +} -- 2.1.4
next prev parent reply other threads:[~2015-12-24 11:14 UTC|newest] Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top 2015-12-24 11:12 [PULL] KVM/ARM updates for 4.5 Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 01/31] KVM: arm/arm64: vgic: make vgic_io_ops static Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 02/31] KVM: arm/arm64: Count guest exit due to various reasons Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 03/31] arm64: Add macros to read/write system registers Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 04/31] arm64: KVM: Add a HYP-specific header file Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 05/31] arm64: KVM: Implement vgic-v2 save/restore Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 06/31] KVM: arm/arm64: vgic-v3: Make the LR indexing macro public Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 07/31] arm64: KVM: Implement vgic-v3 save/restore Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 08/31] arm64: KVM: Implement timer save/restore Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 09/31] arm64: KVM: Implement system register save/restore Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 10/31] arm64: KVM: Implement 32bit " Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 11/31] arm64: KVM: Implement debug save/restore Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 12/31] arm64: KVM: Implement guest entry Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 13/31] arm64: KVM: Add patchable function selector Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier [this message] 2015-12-24 11:12 ` [PATCH 14/31] arm64: KVM: Implement the core world switch Marc Zyngier 2015-12-24 11:12 ` [PATCH 15/31] arm64: KVM: Implement fpsimd save/restore Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 16/31] arm64: KVM: Implement TLB handling Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 17/31] arm64: KVM: HYP mode entry points Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 18/31] arm64: KVM: Add panic handling Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 19/31] arm64: KVM: Add compatibility aliases Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 20/31] arm64: KVM: Map the kernel RO section into HYP Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 21/31] arm64: KVM: Move away from the assembly version of the world switch Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 22/31] arm64: KVM: Turn system register numbers to an enum Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 23/31] arm64: KVM: Cleanup asm-offset.c Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 24/31] arm64: KVM: Remove weak attributes Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 25/31] ARM: KVM: Cleanup exception injection Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 26/31] arm64: KVM: debug: Remove spurious inline attributes Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 27/31] arm/arm64: KVM: Remove unreferenced S2_PGD_ORDER Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 28/31] arm: KVM: Make kvm_arm.h friendly to assembly code Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 29/31] arm64: KVM: Add support for 16-bit VMID Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 30/31] MAINTAINERS: add git URL for KVM/ARM Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2015-12-24 11:12 ` [PATCH 31/31] arm/arm64: KVM: Detect vGIC presence at runtime Marc Zyngier 2015-12-24 11:12 ` Marc Zyngier 2016-01-07 10:07 ` [PULL] KVM/ARM updates for 4.5 Paolo Bonzini 2016-01-07 10:07 ` Paolo Bonzini
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1450955559-15639-15-git-send-email-marc.zyngier@arm.com \ --to=marc.zyngier@arm.com \ --cc=alex.bennee@linaro.org \ --cc=amittomer25@gmail.com \ --cc=catalin.marinas@arm.com \ --cc=christoffer.dall@linaro.org \ --cc=dcb314@hotmail.com \ --cc=fengguang.wu@intel.com \ --cc=gleb@kernel.org \ --cc=jszhang@marvell.com \ --cc=kvm@vger.kernel.org \ --cc=kvmarm@lists.cs.columbia.edu \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=mark.rutland@arm.com \ --cc=p.fedin@samsung.com \ --cc=pbonzini@redhat.com \ --cc=vladimir.murzin@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.