From: Christoffer Dall <christoffer.dall@arm.com> To: Julien Thierry <julien.thierry@arm.com> Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, daniel.thompson@linaro.org, joel@joelfernandes.org, marc.zyngier@arm.com, james.morse@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, mark.rutland@arm.com, kvmarm@lists.cs.columbia.edu Subject: Re: [PATCH v9 10/26] arm64: kvm: Unmask PMR before entering guest Date: Wed, 30 Jan 2019 13:07:39 +0100 [thread overview] Message-ID: <20190130120739.GH13482@e113682-lin.lund.arm.com> (raw) In-Reply-To: <1548084825-8803-11-git-send-email-julien.thierry@arm.com> On Mon, Jan 21, 2019 at 03:33:29PM +0000, Julien Thierry wrote: > Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This > means that hypervisor will not receive masked interrupts while running a > guest. > You could add to the commit description how this works overall, something along the lines of: We need to make sure that all maskable interrupts are masked from the time we call local_irq_disable() in the main run loop, and remain so until we call local_irq_enable() after returning from the guest, and we need to ensure that we see no interrupts at all (including pseudo-NMIs) in the middle of the VM world-switch, while at the same time we need to ensure we exit the guest when there are interrupts for the host. We can accomplish this with pseudo-NMIs enabled by: (1) local_irq_disable: set the priority mask (2) enter guest: set PSTATE.I (3) clear the priority mask (4) eret to guest (5) exit guest: set the priotiy mask clear PSTATE.I (and restore other host PSTATE bits) (6) local_irq_enable: clear the priority mask. Also, took me a while to realize that when we come back from the guest, we call local_daif_restore with DAIF_PROCCTX_NOIRQ, which actually does both of the things in (5). > Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest. > > Signed-off-by: Julien Thierry <julien.thierry@arm.com> > Acked-by: Catalin Marinas <catalin.marinas@arm.com> > Cc: Christoffer Dall <christoffer.dall@arm.com> > Cc: Marc Zyngier <marc.zyngier@arm.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will.deacon@arm.com> > Cc: kvmarm@lists.cs.columbia.edu > --- > arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++ > arch/arm64/kvm/hyp/switch.c | 16 ++++++++++++++++ > 2 files changed, 28 insertions(+) > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index 7732d0b..a1f9f55 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -24,6 +24,7 @@ > > #include <linux/types.h> > #include <linux/kvm_types.h> > +#include <asm/arch_gicv3.h> > #include <asm/cpufeature.h> > #include <asm/daifflags.h> > #include <asm/fpsimd.h> > @@ -474,6 +475,17 @@ static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu) > static inline void kvm_arm_vhe_guest_enter(void) > { > local_daif_mask(); > + > + /* > + * Having IRQs masked via PMR when entering the guest means the GIC > + * will not signal the CPU of interrupts of lower priority, and the > + * only way to get out will be via guest exceptions. > + * Naturally, we want to avoid this. > + */ > + if (system_uses_irq_prio_masking()) { > + gic_write_pmr(GIC_PRIO_IRQON); > + dsb(sy); > + } > } > > static inline void kvm_arm_vhe_guest_exit(void) > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c > index b0b1478..6a4c2d6 100644 > --- a/arch/arm64/kvm/hyp/switch.c > +++ b/arch/arm64/kvm/hyp/switch.c > @@ -22,6 +22,7 @@ > > #include <kvm/arm_psci.h> > > +#include <asm/arch_gicv3.h> > #include <asm/cpufeature.h> > #include <asm/kvm_asm.h> > #include <asm/kvm_emulate.h> > @@ -521,6 +522,17 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) > struct kvm_cpu_context *guest_ctxt; > u64 exit_code; > > + /* > + * Having IRQs masked via PMR when entering the guest means the GIC > + * will not signal the CPU of interrupts of lower priority, and the > + * only way to get out will be via guest exceptions. > + * Naturally, we want to avoid this. > + */ > + if (system_uses_irq_prio_masking()) { > + gic_write_pmr(GIC_PRIO_IRQON); > + dsb(sy); > + } > + > vcpu = kern_hyp_va(vcpu); > > host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context); > @@ -573,6 +585,10 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) > */ > __debug_switch_to_host(vcpu); > > + /* Returning to host will clear PSR.I, remask PMR if needed */ > + if (system_uses_irq_prio_masking()) > + gic_write_pmr(GIC_PRIO_IRQOFF); > + > return exit_code; > } > nit: you could consider moving the non-vhe part into a new kvm_arm_nvhe_guest_enter, for symmetry with the vhe part. Otherwise looks good to me: Reviewed-by: Christoffer Dall <christoffer.dall@arm.com>
WARNING: multiple messages have this Message-ID (diff)
From: Christoffer Dall <christoffer.dall@arm.com> To: Julien Thierry <julien.thierry@arm.com> Cc: mark.rutland@arm.com, daniel.thompson@linaro.org, marc.zyngier@arm.com, catalin.marinas@arm.com, will.deacon@arm.com, linux-kernel@vger.kernel.org, james.morse@arm.com, joel@joelfernandes.org, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org Subject: Re: [PATCH v9 10/26] arm64: kvm: Unmask PMR before entering guest Date: Wed, 30 Jan 2019 13:07:39 +0100 [thread overview] Message-ID: <20190130120739.GH13482@e113682-lin.lund.arm.com> (raw) In-Reply-To: <1548084825-8803-11-git-send-email-julien.thierry@arm.com> On Mon, Jan 21, 2019 at 03:33:29PM +0000, Julien Thierry wrote: > Interrupts masked by ICC_PMR_EL1 will not be signaled to the CPU. This > means that hypervisor will not receive masked interrupts while running a > guest. > You could add to the commit description how this works overall, something along the lines of: We need to make sure that all maskable interrupts are masked from the time we call local_irq_disable() in the main run loop, and remain so until we call local_irq_enable() after returning from the guest, and we need to ensure that we see no interrupts at all (including pseudo-NMIs) in the middle of the VM world-switch, while at the same time we need to ensure we exit the guest when there are interrupts for the host. We can accomplish this with pseudo-NMIs enabled by: (1) local_irq_disable: set the priority mask (2) enter guest: set PSTATE.I (3) clear the priority mask (4) eret to guest (5) exit guest: set the priotiy mask clear PSTATE.I (and restore other host PSTATE bits) (6) local_irq_enable: clear the priority mask. Also, took me a while to realize that when we come back from the guest, we call local_daif_restore with DAIF_PROCCTX_NOIRQ, which actually does both of the things in (5). > Avoid this by making sure ICC_PMR_EL1 is unmasked when we enter a guest. > > Signed-off-by: Julien Thierry <julien.thierry@arm.com> > Acked-by: Catalin Marinas <catalin.marinas@arm.com> > Cc: Christoffer Dall <christoffer.dall@arm.com> > Cc: Marc Zyngier <marc.zyngier@arm.com> > Cc: Catalin Marinas <catalin.marinas@arm.com> > Cc: Will Deacon <will.deacon@arm.com> > Cc: kvmarm@lists.cs.columbia.edu > --- > arch/arm64/include/asm/kvm_host.h | 12 ++++++++++++ > arch/arm64/kvm/hyp/switch.c | 16 ++++++++++++++++ > 2 files changed, 28 insertions(+) > > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index 7732d0b..a1f9f55 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -24,6 +24,7 @@ > > #include <linux/types.h> > #include <linux/kvm_types.h> > +#include <asm/arch_gicv3.h> > #include <asm/cpufeature.h> > #include <asm/daifflags.h> > #include <asm/fpsimd.h> > @@ -474,6 +475,17 @@ static inline int kvm_arch_vcpu_run_pid_change(struct kvm_vcpu *vcpu) > static inline void kvm_arm_vhe_guest_enter(void) > { > local_daif_mask(); > + > + /* > + * Having IRQs masked via PMR when entering the guest means the GIC > + * will not signal the CPU of interrupts of lower priority, and the > + * only way to get out will be via guest exceptions. > + * Naturally, we want to avoid this. > + */ > + if (system_uses_irq_prio_masking()) { > + gic_write_pmr(GIC_PRIO_IRQON); > + dsb(sy); > + } > } > > static inline void kvm_arm_vhe_guest_exit(void) > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c > index b0b1478..6a4c2d6 100644 > --- a/arch/arm64/kvm/hyp/switch.c > +++ b/arch/arm64/kvm/hyp/switch.c > @@ -22,6 +22,7 @@ > > #include <kvm/arm_psci.h> > > +#include <asm/arch_gicv3.h> > #include <asm/cpufeature.h> > #include <asm/kvm_asm.h> > #include <asm/kvm_emulate.h> > @@ -521,6 +522,17 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) > struct kvm_cpu_context *guest_ctxt; > u64 exit_code; > > + /* > + * Having IRQs masked via PMR when entering the guest means the GIC > + * will not signal the CPU of interrupts of lower priority, and the > + * only way to get out will be via guest exceptions. > + * Naturally, we want to avoid this. > + */ > + if (system_uses_irq_prio_masking()) { > + gic_write_pmr(GIC_PRIO_IRQON); > + dsb(sy); > + } > + > vcpu = kern_hyp_va(vcpu); > > host_ctxt = kern_hyp_va(vcpu->arch.host_cpu_context); > @@ -573,6 +585,10 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) > */ > __debug_switch_to_host(vcpu); > > + /* Returning to host will clear PSR.I, remask PMR if needed */ > + if (system_uses_irq_prio_masking()) > + gic_write_pmr(GIC_PRIO_IRQOFF); > + > return exit_code; > } > nit: you could consider moving the non-vhe part into a new kvm_arm_nvhe_guest_enter, for symmetry with the vhe part. Otherwise looks good to me: Reviewed-by: Christoffer Dall <christoffer.dall@arm.com> _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2019-01-30 12:07 UTC|newest] Thread overview: 166+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-01-21 15:33 [PATCH v9 00/26] arm64: provide pseudo NMI with GICv3 Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-21 15:33 ` [PATCH v9 01/26] arm64: Fix HCR.TGE status for NMI contexts Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-23 22:57 ` Sasha Levin 2019-01-23 22:57 ` Sasha Levin 2019-01-28 11:48 ` James Morse 2019-01-28 11:48 ` James Morse 2019-01-28 15:42 ` Julien Thierry 2019-01-28 15:42 ` Julien Thierry 2019-01-31 8:19 ` Christoffer Dall 2019-01-31 8:19 ` Christoffer Dall 2019-01-31 8:56 ` Julien Thierry 2019-01-31 8:56 ` Julien Thierry 2019-01-31 9:27 ` Christoffer Dall 2019-01-31 9:27 ` Christoffer Dall 2019-01-31 9:40 ` Julien Thierry 2019-01-31 9:40 ` Julien Thierry 2019-01-31 9:48 ` Christoffer Dall 2019-01-31 9:48 ` Christoffer Dall 2019-01-31 9:53 ` Marc Zyngier 2019-01-31 9:53 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 02/26] arm64: Remove unused daif related functions/macros Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:21 ` Marc Zyngier 2019-01-28 9:21 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 03/26] arm64: cpufeature: Set SYSREG_GIC_CPUIF as a boot system feature Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:22 ` Marc Zyngier 2019-01-28 9:22 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 04/26] arm64: cpufeature: Add cpufeature for IRQ priority masking Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:24 ` Marc Zyngier 2019-01-28 9:24 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 05/26] arm/arm64: gic-v3: Add PMR and RPR accessors Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:25 ` Marc Zyngier 2019-01-28 9:25 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 06/26] irqchip/gic-v3: Switch to PMR masking before calling IRQ handler Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:30 ` Marc Zyngier 2019-01-28 9:30 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 07/26] arm64: ptrace: Provide definitions for PMR values Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:37 ` Marc Zyngier 2019-01-28 9:37 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 08/26] arm64: Make PMR part of task context Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:42 ` Marc Zyngier 2019-01-28 9:42 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 09/26] arm64: Unmask PMR before going idle Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-22 15:23 ` Catalin Marinas 2019-01-22 15:23 ` Catalin Marinas 2019-01-22 20:18 ` Ard Biesheuvel 2019-01-22 20:18 ` Ard Biesheuvel 2019-01-23 8:56 ` Julien Thierry 2019-01-23 8:56 ` Julien Thierry 2019-01-23 9:38 ` Ard Biesheuvel 2019-01-23 9:38 ` Ard Biesheuvel 2019-01-28 9:44 ` Marc Zyngier 2019-01-28 9:44 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 10/26] arm64: kvm: Unmask PMR before entering guest Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 9:58 ` Marc Zyngier 2019-01-28 9:58 ` Marc Zyngier 2019-01-28 9:58 ` Marc Zyngier 2019-01-30 12:07 ` Christoffer Dall [this message] 2019-01-30 12:07 ` Christoffer Dall 2019-01-30 14:58 ` Julien Thierry 2019-01-30 14:58 ` Julien Thierry 2019-01-21 15:33 ` [PATCH v9 11/26] efi: Let architectures decide the flags that should be saved/restored Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-21 15:42 ` Ard Biesheuvel 2019-01-21 15:42 ` Ard Biesheuvel 2019-01-23 9:04 ` Julien Thierry 2019-01-23 9:04 ` Julien Thierry 2019-01-28 10:00 ` Marc Zyngier 2019-01-28 10:00 ` Marc Zyngier 2019-01-28 10:00 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 12/26] arm64: irqflags: Use ICC_PMR_EL1 for interrupt masking Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-21 15:45 ` Ard Biesheuvel 2019-01-21 15:45 ` Ard Biesheuvel 2019-01-21 18:05 ` Julien Thierry 2019-01-21 18:05 ` Julien Thierry 2019-01-22 15:21 ` Catalin Marinas 2019-01-22 15:21 ` Catalin Marinas 2019-01-23 10:44 ` Julien Thierry 2019-01-23 10:44 ` Julien Thierry 2019-01-30 11:52 ` Julien Thierry 2019-01-30 11:52 ` Julien Thierry 2019-01-21 15:33 ` [PATCH v9 13/26] arm64: daifflags: Include PMR in daifflags restore operations Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 10:37 ` Marc Zyngier 2019-01-28 10:37 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 14/26] arm64: alternative: Allow alternative status checking per cpufeature Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 11:00 ` Marc Zyngier 2019-01-28 11:00 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 15/26] arm64: alternative: Apply alternatives early in boot process Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 11:17 ` Marc Zyngier 2019-01-28 11:17 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 16/26] irqchip/gic-v3: Factor group0 detection into functions Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 11:19 ` Marc Zyngier 2019-01-28 11:19 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 17/26] arm64: Switch to PMR masking when starting CPUs Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 11:21 ` Marc Zyngier 2019-01-28 11:21 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 18/26] arm64: gic-v3: Implement arch support for priority masking Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 11:23 ` Marc Zyngier 2019-01-28 11:23 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 19/26] irqchip/gic-v3: Detect if GIC can support pseudo-NMIs Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 11:39 ` Marc Zyngier 2019-01-28 11:39 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 20/26] irqchip/gic-v3: Handle pseudo-NMIs Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 11:59 ` Marc Zyngier 2019-01-28 11:59 ` Marc Zyngier 2019-01-29 11:33 ` Julien Thierry 2019-01-29 11:33 ` Julien Thierry 2019-01-29 12:31 ` Marc Zyngier 2019-01-29 12:31 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 21/26] irqchip/gic: Add functions to access irq priorities Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 12:04 ` Marc Zyngier 2019-01-28 12:04 ` Marc Zyngier 2019-01-29 11:36 ` Julien Thierry 2019-01-29 11:36 ` Julien Thierry 2019-01-21 15:33 ` [PATCH v9 22/26] irqchip/gic-v3: Allow interrupts to be set as pseudo-NMI Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-26 10:19 ` liwei (GF) 2019-01-26 10:19 ` liwei (GF) 2019-01-26 10:41 ` Marc Zyngier 2019-01-26 10:41 ` Marc Zyngier 2019-01-28 8:57 ` Julien Thierry 2019-01-28 8:57 ` Julien Thierry 2019-01-28 13:59 ` liwei (GF) 2019-01-28 13:59 ` liwei (GF) 2019-01-28 14:49 ` Julien Thierry 2019-01-28 14:49 ` Julien Thierry 2019-01-28 12:08 ` Marc Zyngier 2019-01-28 12:08 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 23/26] arm64: Handle serror in NMI context Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 12:26 ` Marc Zyngier 2019-01-28 12:26 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 24/26] arm64: Skip preemption when exiting an NMI Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 12:34 ` Marc Zyngier 2019-01-28 12:34 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 25/26] arm64: Skip irqflags tracing for NMI in IRQs disabled context Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 12:40 ` Marc Zyngier 2019-01-28 12:40 ` Marc Zyngier 2019-01-21 15:33 ` [PATCH v9 26/26] arm64: Enable the support of pseudo-NMIs Julien Thierry 2019-01-21 15:33 ` Julien Thierry 2019-01-28 12:47 ` Marc Zyngier 2019-01-28 12:47 ` Marc Zyngier 2019-01-30 13:46 ` Julien Thierry 2019-01-30 13:46 ` Julien Thierry
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20190130120739.GH13482@e113682-lin.lund.arm.com \ --to=christoffer.dall@arm.com \ --cc=catalin.marinas@arm.com \ --cc=daniel.thompson@linaro.org \ --cc=james.morse@arm.com \ --cc=joel@joelfernandes.org \ --cc=julien.thierry@arm.com \ --cc=kvmarm@lists.cs.columbia.edu \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=marc.zyngier@arm.com \ --cc=mark.rutland@arm.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.