* [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU
@ 2014-07-31 6:53 Pranavkumar Sawargaonkar
2014-08-04 12:06 ` Christoffer Dall
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Pranavkumar Sawargaonkar @ 2014-07-31 6:53 UTC (permalink / raw)
To: linux-arm-kernel
X-Gene u-boot runs in EL2 mode with MMU enabled hence we might
have stale EL2 tlb enteris when we enable EL2 MMU on each host CPU.
This can happen on any ARM/ARM64 board running bootloader in
Hyp-mode (or EL2-mode) with MMU enabled.
This patch ensures that we flush all Hyp-mode (or EL2-mode) TLBs
on each host CPU before enabling Hyp-mode (or EL2-mode) MMU.
Changelog:
V2:
- Flush Hyp-mode TLBs for both KVM ARM32 and KVM ARM64 at boot time
V1:
- Initial patch with only arm64 change
Signed-off-by: Pranavkumar Sawargaonkar <pranavkumar@linaro.org>
Signed-off-by: Anup Patel <anup.patel@linaro.org>
---
arch/arm/kvm/init.S | 4 ++++
arch/arm64/kvm/hyp-init.S | 4 ++++
2 files changed, 8 insertions(+)
diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
index 1b9844d..ee4f744 100644
--- a/arch/arm/kvm/init.S
+++ b/arch/arm/kvm/init.S
@@ -98,6 +98,10 @@ __do_hyp_init:
mrc p15, 0, r0, c10, c2, 1
mcr p15, 4, r0, c10, c2, 1
+ @ Invalidate the stale TLBs from Bootloader
+ mcr p15, 4, r0, c8, c7, 0 @ TLBIALLH
+ dsb ish
+
@ Set the HSCTLR to:
@ - ARM/THUMB exceptions: Kernel config (Thumb-2 kernel)
@ - Endianness: Kernel config
diff --git a/arch/arm64/kvm/hyp-init.S b/arch/arm64/kvm/hyp-init.S
index d968796..c319116 100644
--- a/arch/arm64/kvm/hyp-init.S
+++ b/arch/arm64/kvm/hyp-init.S
@@ -80,6 +80,10 @@ __do_hyp_init:
msr mair_el2, x4
isb
+ /* Invalidate the stale TLBs from Bootloader */
+ tlbi alle2
+ dsb sy
+
mrs x4, sctlr_el2
and x4, x4, #SCTLR_EL2_EE // preserve endianness of EL2
ldr x5, =SCTLR_EL2_FLAGS
--
1.7.9.5
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU
2014-07-31 6:53 [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU Pranavkumar Sawargaonkar
@ 2014-08-04 12:06 ` Christoffer Dall
2014-08-22 9:03 ` Marc Zyngier
2014-08-04 13:26 ` Marc Zyngier
2014-08-22 10:23 ` Mark Rutland
2 siblings, 1 reply; 6+ messages in thread
From: Christoffer Dall @ 2014-08-04 12:06 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Jul 31, 2014 at 12:23:23PM +0530, Pranavkumar Sawargaonkar wrote:
> X-Gene u-boot runs in EL2 mode with MMU enabled hence we might
> have stale EL2 tlb enteris when we enable EL2 MMU on each host CPU.
>
> This can happen on any ARM/ARM64 board running bootloader in
> Hyp-mode (or EL2-mode) with MMU enabled.
>
> This patch ensures that we flush all Hyp-mode (or EL2-mode) TLBs
> on each host CPU before enabling Hyp-mode (or EL2-mode) MMU.
>
> Changelog:
>
> V2:
> - Flush Hyp-mode TLBs for both KVM ARM32 and KVM ARM64 at boot time
>
> V1:
> - Initial patch with only arm64 change
>
> Signed-off-by: Pranavkumar Sawargaonkar <pranavkumar@linaro.org>
> Signed-off-by: Anup Patel <anup.patel@linaro.org>
Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU
2014-08-04 12:06 ` Christoffer Dall
@ 2014-08-22 9:03 ` Marc Zyngier
0 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2014-08-22 9:03 UTC (permalink / raw)
To: linux-arm-kernel
On 04/08/14 13:06, Christoffer Dall wrote:
> On Thu, Jul 31, 2014 at 12:23:23PM +0530, Pranavkumar Sawargaonkar wrote:
>> X-Gene u-boot runs in EL2 mode with MMU enabled hence we might
>> have stale EL2 tlb enteris when we enable EL2 MMU on each host CPU.
>>
>> This can happen on any ARM/ARM64 board running bootloader in
>> Hyp-mode (or EL2-mode) with MMU enabled.
>>
>> This patch ensures that we flush all Hyp-mode (or EL2-mode) TLBs
>> on each host CPU before enabling Hyp-mode (or EL2-mode) MMU.
>>
>> Changelog:
>>
>> V2:
>> - Flush Hyp-mode TLBs for both KVM ARM32 and KVM ARM64 at boot time
>>
>> V1:
>> - Initial patch with only arm64 change
>>
>> Signed-off-by: Pranavkumar Sawargaonkar <pranavkumar@linaro.org>
>> Signed-off-by: Anup Patel <anup.patel@linaro.org>
>
> Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>
Acked-by: Marc Zyngier <marc.zyngier@arm.com>
This should probably go in quickly as a fix.
Thanks,
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU
2014-07-31 6:53 [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU Pranavkumar Sawargaonkar
2014-08-04 12:06 ` Christoffer Dall
@ 2014-08-04 13:26 ` Marc Zyngier
2014-08-22 10:23 ` Mark Rutland
2 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2014-08-04 13:26 UTC (permalink / raw)
To: linux-arm-kernel
Hi Pranav,
On Thu, Jul 31 2014 at 7:53:23 am BST, Pranavkumar Sawargaonkar <pranavkumar@linaro.org> wrote:
> X-Gene u-boot runs in EL2 mode with MMU enabled hence we might
> have stale EL2 tlb enteris when we enable EL2 MMU on each host CPU.
>
> This can happen on any ARM/ARM64 board running bootloader in
> Hyp-mode (or EL2-mode) with MMU enabled.
>
> This patch ensures that we flush all Hyp-mode (or EL2-mode) TLBs
> on each host CPU before enabling Hyp-mode (or EL2-mode) MMU.
>
> Changelog:
>
> V2:
> - Flush Hyp-mode TLBs for both KVM ARM32 and KVM ARM64 at boot time
>
> V1:
> - Initial patch with only arm64 change
>
> Signed-off-by: Pranavkumar Sawargaonkar <pranavkumar@linaro.org>
> Signed-off-by: Anup Patel <anup.patel@linaro.org>
> ---
> arch/arm/kvm/init.S | 4 ++++
> arch/arm64/kvm/hyp-init.S | 4 ++++
> 2 files changed, 8 insertions(+)
>
> diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
> index 1b9844d..ee4f744 100644
> --- a/arch/arm/kvm/init.S
> +++ b/arch/arm/kvm/init.S
> @@ -98,6 +98,10 @@ __do_hyp_init:
> mrc p15, 0, r0, c10, c2, 1
> mcr p15, 4, r0, c10, c2, 1
>
> + @ Invalidate the stale TLBs from Bootloader
> + mcr p15, 4, r0, c8, c7, 0 @ TLBIALLH
> + dsb ish
> +
> @ Set the HSCTLR to:
> @ - ARM/THUMB exceptions: Kernel config (Thumb-2 kernel)
> @ - Endianness: Kernel config
> diff --git a/arch/arm64/kvm/hyp-init.S b/arch/arm64/kvm/hyp-init.S
> index d968796..c319116 100644
> --- a/arch/arm64/kvm/hyp-init.S
> +++ b/arch/arm64/kvm/hyp-init.S
> @@ -80,6 +80,10 @@ __do_hyp_init:
> msr mair_el2, x4
> isb
>
> + /* Invalidate the stale TLBs from Bootloader */
> + tlbi alle2
> + dsb sy
> +
> mrs x4, sctlr_el2
> and x4, x4, #SCTLR_EL2_EE // preserve endianness of EL2
> ldr x5, =SCTLR_EL2_FLAGS
Reviewed-by: Marc Zyngier <marc.zyngier@arm.com>
M.
--
Jazz is not dead. It just smells funny.
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU
2014-07-31 6:53 [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU Pranavkumar Sawargaonkar
2014-08-04 12:06 ` Christoffer Dall
2014-08-04 13:26 ` Marc Zyngier
@ 2014-08-22 10:23 ` Mark Rutland
2014-08-22 10:41 ` Marc Zyngier
2 siblings, 1 reply; 6+ messages in thread
From: Mark Rutland @ 2014-08-22 10:23 UTC (permalink / raw)
To: linux-arm-kernel
Hi Pranav,
On Thu, Jul 31, 2014 at 07:53:23AM +0100, Pranavkumar Sawargaonkar wrote:
> X-Gene u-boot runs in EL2 mode with MMU enabled hence we might
> have stale EL2 tlb enteris when we enable EL2 MMU on each host CPU.
Nit: entries
> This can happen on any ARM/ARM64 board running bootloader in
> Hyp-mode (or EL2-mode) with MMU enabled.
>
> This patch ensures that we flush all Hyp-mode (or EL2-mode) TLBs
> on each host CPU before enabling Hyp-mode (or EL2-mode) MMU.
I hit a similar issue intermittently on a Juno board when I boot Linux
as an EFI application, where CPU0 would get stuck in a recursive
exception at EL2 while initialising hyp. We don't nuke the TLBs in
efi_stub_entry nor do we do so in el2_setup, so there's the possibility
of stale EL2 TLB entries lying around from UEFI.
With this patch applied I'm unable to reproduce the issue (with ~30
boots so far). So:
Tested-by: Mark Rutland <mark.rutland@arm.com>
It would be nice to see this hit mainline soon.
Thanks,
Mark.
>
> Changelog:
>
> V2:
> - Flush Hyp-mode TLBs for both KVM ARM32 and KVM ARM64 at boot time
>
> V1:
> - Initial patch with only arm64 change
>
> Signed-off-by: Pranavkumar Sawargaonkar <pranavkumar@linaro.org>
> Signed-off-by: Anup Patel <anup.patel@linaro.org>
> ---
> arch/arm/kvm/init.S | 4 ++++
> arch/arm64/kvm/hyp-init.S | 4 ++++
> 2 files changed, 8 insertions(+)
>
> diff --git a/arch/arm/kvm/init.S b/arch/arm/kvm/init.S
> index 1b9844d..ee4f744 100644
> --- a/arch/arm/kvm/init.S
> +++ b/arch/arm/kvm/init.S
> @@ -98,6 +98,10 @@ __do_hyp_init:
> mrc p15, 0, r0, c10, c2, 1
> mcr p15, 4, r0, c10, c2, 1
>
> + @ Invalidate the stale TLBs from Bootloader
> + mcr p15, 4, r0, c8, c7, 0 @ TLBIALLH
> + dsb ish
> +
> @ Set the HSCTLR to:
> @ - ARM/THUMB exceptions: Kernel config (Thumb-2 kernel)
> @ - Endianness: Kernel config
> diff --git a/arch/arm64/kvm/hyp-init.S b/arch/arm64/kvm/hyp-init.S
> index d968796..c319116 100644
> --- a/arch/arm64/kvm/hyp-init.S
> +++ b/arch/arm64/kvm/hyp-init.S
> @@ -80,6 +80,10 @@ __do_hyp_init:
> msr mair_el2, x4
> isb
>
> + /* Invalidate the stale TLBs from Bootloader */
> + tlbi alle2
> + dsb sy
> +
> mrs x4, sctlr_el2
> and x4, x4, #SCTLR_EL2_EE // preserve endianness of EL2
> ldr x5, =SCTLR_EL2_FLAGS
> --
> 1.7.9.5
>
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU
2014-08-22 10:23 ` Mark Rutland
@ 2014-08-22 10:41 ` Marc Zyngier
0 siblings, 0 replies; 6+ messages in thread
From: Marc Zyngier @ 2014-08-22 10:41 UTC (permalink / raw)
To: linux-arm-kernel
On 22/08/14 11:23, Mark Rutland wrote:
> Hi Pranav,
>
> On Thu, Jul 31, 2014 at 07:53:23AM +0100, Pranavkumar Sawargaonkar wrote:
>> X-Gene u-boot runs in EL2 mode with MMU enabled hence we might
>> have stale EL2 tlb enteris when we enable EL2 MMU on each host CPU.
>
> Nit: entries
>
>> This can happen on any ARM/ARM64 board running bootloader in
>> Hyp-mode (or EL2-mode) with MMU enabled.
>>
>> This patch ensures that we flush all Hyp-mode (or EL2-mode) TLBs
>> on each host CPU before enabling Hyp-mode (or EL2-mode) MMU.
>
> I hit a similar issue intermittently on a Juno board when I boot Linux
> as an EFI application, where CPU0 would get stuck in a recursive
> exception at EL2 while initialising hyp. We don't nuke the TLBs in
> efi_stub_entry nor do we do so in el2_setup, so there's the possibility
> of stale EL2 TLB entries lying around from UEFI.
>
> With this patch applied I'm unable to reproduce the issue (with ~30
> boots so far). So:
>
> Tested-by: Mark Rutland <mark.rutland@arm.com>
>
> It would be nice to see this hit mainline soon.
Thanks for testing. I guess this is a candidate for stable as well.
M.
--
Jazz is not dead. It just smells funny...
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-08-22 10:41 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-07-31 6:53 [PATCH V2] ARM/ARM64: KVM: Nuke Hyp-mode tlbs before enabling MMU Pranavkumar Sawargaonkar
2014-08-04 12:06 ` Christoffer Dall
2014-08-22 9:03 ` Marc Zyngier
2014-08-04 13:26 ` Marc Zyngier
2014-08-22 10:23 ` Mark Rutland
2014-08-22 10:41 ` Marc Zyngier
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.