All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-03-29 18:24 ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-03-29 18:24 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: linux-efi, Ard Biesheuvel, Marc Zyngier, linux, leif.lindholm,
	steve.mcintyre

The EFI stub is entered with the caches and MMU enabled by the
firmware, and once the stub is ready to hand over to the decompressor,
we clean and disable the caches.

The cache clean routines use CP15 barrier instructions, which can be
disabled via SCTLR. Normally, when using the provided cache handling
routines to enable the caches and MMU, this bit is enabled as well.
However, but since we entered the stub with the caches already enabled,
this routine is not executed before we call the cache clean routines,
resulting in undefined instruction exceptions if the firmware never
enabled this bit.

So set the bit explicitly in the EFI entry code.

Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/boot/compressed/head.S | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 6c7ccb428c07..62a49356fca3 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
 
 		@ Preserve return value of efi_entry() in r4
 		mov	r4, r0
+
+		@ our cache maintenance code relies on CP15 barrier instructions
+		@ but since we arrived here with the MMU and caches configured
+		@ by UEFI, we must ensure that the use of those instructions is
+		@ enabled in the SCTLR register, since we never executed our own
+		@ cache enable routine, which is normally in charge of this.
+		mrc	p15, 0, r1, c1, c0, 0	@ read SCTLR
+		orr	r1, r1, #(1 << 5)	@ CP15 barrier instructions
+		mcr	p15, 0, r1, c1, c0, 0	@ write SCTLR
+
 		bl	cache_clean_flush
 		bl	cache_off
 
-- 
2.20.1

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-03-29 18:24 ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-03-29 18:24 UTC (permalink / raw)
  To: linux-arm-kernel
  Cc: linux-efi, Ard Biesheuvel, Marc Zyngier, linux, leif.lindholm,
	steve.mcintyre

The EFI stub is entered with the caches and MMU enabled by the
firmware, and once the stub is ready to hand over to the decompressor,
we clean and disable the caches.

The cache clean routines use CP15 barrier instructions, which can be
disabled via SCTLR. Normally, when using the provided cache handling
routines to enable the caches and MMU, this bit is enabled as well.
However, but since we entered the stub with the caches already enabled,
this routine is not executed before we call the cache clean routines,
resulting in undefined instruction exceptions if the firmware never
enabled this bit.

So set the bit explicitly in the EFI entry code.

Cc: Marc Zyngier <marc.zyngier@arm.com>
Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
 arch/arm/boot/compressed/head.S | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
index 6c7ccb428c07..62a49356fca3 100644
--- a/arch/arm/boot/compressed/head.S
+++ b/arch/arm/boot/compressed/head.S
@@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
 
 		@ Preserve return value of efi_entry() in r4
 		mov	r4, r0
+
+		@ our cache maintenance code relies on CP15 barrier instructions
+		@ but since we arrived here with the MMU and caches configured
+		@ by UEFI, we must ensure that the use of those instructions is
+		@ enabled in the SCTLR register, since we never executed our own
+		@ cache enable routine, which is normally in charge of this.
+		mrc	p15, 0, r1, c1, c0, 0	@ read SCTLR
+		orr	r1, r1, #(1 << 5)	@ CP15 barrier instructions
+		mcr	p15, 0, r1, c1, c0, 0	@ write SCTLR
+
 		bl	cache_clean_flush
 		bl	cache_off
 
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
  2019-03-29 18:24 ` Ard Biesheuvel
@ 2019-03-30  9:50   ` Marc Zyngier
  -1 siblings, 0 replies; 20+ messages in thread
From: Marc Zyngier @ 2019-03-30  9:50 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: leif.lindholm, linux-efi, steve.mcintyre, linux, linux-arm-kernel

Hi Ard,

On Fri, 29 Mar 2019 18:24:18 +0000,
Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> The EFI stub is entered with the caches and MMU enabled by the
> firmware, and once the stub is ready to hand over to the decompressor,
> we clean and disable the caches.
> 
> The cache clean routines use CP15 barrier instructions, which can be
> disabled via SCTLR. Normally, when using the provided cache handling
> routines to enable the caches and MMU, this bit is enabled as well.
> However, but since we entered the stub with the caches already enabled,
> this routine is not executed before we call the cache clean routines,
> resulting in undefined instruction exceptions if the firmware never
> enabled this bit.
> 
> So set the bit explicitly in the EFI entry code.
> 
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  arch/arm/boot/compressed/head.S | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> index 6c7ccb428c07..62a49356fca3 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
>  
>  		@ Preserve return value of efi_entry() in r4
>  		mov	r4, r0
> +
> +		@ our cache maintenance code relies on CP15 barrier instructions
> +		@ but since we arrived here with the MMU and caches configured
> +		@ by UEFI, we must ensure that the use of those instructions is
> +		@ enabled in the SCTLR register, since we never executed our own
> +		@ cache enable routine, which is normally in charge of this.
> +		mrc	p15, 0, r1, c1, c0, 0	@ read SCTLR
> +		orr	r1, r1, #(1 << 5)	@ CP15 barrier instructions
> +		mcr	p15, 0, r1, c1, c0, 0	@ write SCTLR
> +

To be on the safe side, you could add an isb here. I'm pretty sure it
is immaterial on any ARMv7 core, but hey, I'm paranoid.

With that:

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-03-30  9:50   ` Marc Zyngier
  0 siblings, 0 replies; 20+ messages in thread
From: Marc Zyngier @ 2019-03-30  9:50 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: leif.lindholm, linux-efi, steve.mcintyre, linux, linux-arm-kernel

Hi Ard,

On Fri, 29 Mar 2019 18:24:18 +0000,
Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> The EFI stub is entered with the caches and MMU enabled by the
> firmware, and once the stub is ready to hand over to the decompressor,
> we clean and disable the caches.
> 
> The cache clean routines use CP15 barrier instructions, which can be
> disabled via SCTLR. Normally, when using the provided cache handling
> routines to enable the caches and MMU, this bit is enabled as well.
> However, but since we entered the stub with the caches already enabled,
> this routine is not executed before we call the cache clean routines,
> resulting in undefined instruction exceptions if the firmware never
> enabled this bit.
> 
> So set the bit explicitly in the EFI entry code.
> 
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> ---
>  arch/arm/boot/compressed/head.S | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> index 6c7ccb428c07..62a49356fca3 100644
> --- a/arch/arm/boot/compressed/head.S
> +++ b/arch/arm/boot/compressed/head.S
> @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
>  
>  		@ Preserve return value of efi_entry() in r4
>  		mov	r4, r0
> +
> +		@ our cache maintenance code relies on CP15 barrier instructions
> +		@ but since we arrived here with the MMU and caches configured
> +		@ by UEFI, we must ensure that the use of those instructions is
> +		@ enabled in the SCTLR register, since we never executed our own
> +		@ cache enable routine, which is normally in charge of this.
> +		mrc	p15, 0, r1, c1, c0, 0	@ read SCTLR
> +		orr	r1, r1, #(1 << 5)	@ CP15 barrier instructions
> +		mcr	p15, 0, r1, c1, c0, 0	@ write SCTLR
> +

To be on the safe side, you could add an isb here. I'm pretty sure it
is immaterial on any ARMv7 core, but hey, I'm paranoid.

With that:

Acked-by: Marc Zyngier <marc.zyngier@arm.com>

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
  2019-03-30  9:50   ` Marc Zyngier
@ 2019-03-30 13:10     ` Ard Biesheuvel
  -1 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-03-30 13:10 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> Hi Ard,
>
> On Fri, 29 Mar 2019 18:24:18 +0000,
> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >
> > The EFI stub is entered with the caches and MMU enabled by the
> > firmware, and once the stub is ready to hand over to the decompressor,
> > we clean and disable the caches.
> >
> > The cache clean routines use CP15 barrier instructions, which can be
> > disabled via SCTLR. Normally, when using the provided cache handling
> > routines to enable the caches and MMU, this bit is enabled as well.
> > However, but since we entered the stub with the caches already enabled,
> > this routine is not executed before we call the cache clean routines,
> > resulting in undefined instruction exceptions if the firmware never
> > enabled this bit.
> >
> > So set the bit explicitly in the EFI entry code.
> >
> > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > ---
> >  arch/arm/boot/compressed/head.S | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> >
> > diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> > index 6c7ccb428c07..62a49356fca3 100644
> > --- a/arch/arm/boot/compressed/head.S
> > +++ b/arch/arm/boot/compressed/head.S
> > @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
> >
> >               @ Preserve return value of efi_entry() in r4
> >               mov     r4, r0
> > +
> > +             @ our cache maintenance code relies on CP15 barrier instructions
> > +             @ but since we arrived here with the MMU and caches configured
> > +             @ by UEFI, we must ensure that the use of those instructions is
> > +             @ enabled in the SCTLR register, since we never executed our own
> > +             @ cache enable routine, which is normally in charge of this.
> > +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
> > +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
> > +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
> > +
>
> To be on the safe side, you could add an isb here. I'm pretty sure it
> is immaterial on any ARMv7 core, but hey, I'm paranoid.
>

Well, this is actually triggering now under KVM, so I wonder if this
is a regression of some kind on the host side. But the EFI stub
shouldn't use CP15 barriers without enabling them, so we need this
patch in any case.

> With that:
>
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
>

Cheers,

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-03-30 13:10     ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-03-30 13:10 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> Hi Ard,
>
> On Fri, 29 Mar 2019 18:24:18 +0000,
> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >
> > The EFI stub is entered with the caches and MMU enabled by the
> > firmware, and once the stub is ready to hand over to the decompressor,
> > we clean and disable the caches.
> >
> > The cache clean routines use CP15 barrier instructions, which can be
> > disabled via SCTLR. Normally, when using the provided cache handling
> > routines to enable the caches and MMU, this bit is enabled as well.
> > However, but since we entered the stub with the caches already enabled,
> > this routine is not executed before we call the cache clean routines,
> > resulting in undefined instruction exceptions if the firmware never
> > enabled this bit.
> >
> > So set the bit explicitly in the EFI entry code.
> >
> > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > ---
> >  arch/arm/boot/compressed/head.S | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> >
> > diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> > index 6c7ccb428c07..62a49356fca3 100644
> > --- a/arch/arm/boot/compressed/head.S
> > +++ b/arch/arm/boot/compressed/head.S
> > @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
> >
> >               @ Preserve return value of efi_entry() in r4
> >               mov     r4, r0
> > +
> > +             @ our cache maintenance code relies on CP15 barrier instructions
> > +             @ but since we arrived here with the MMU and caches configured
> > +             @ by UEFI, we must ensure that the use of those instructions is
> > +             @ enabled in the SCTLR register, since we never executed our own
> > +             @ cache enable routine, which is normally in charge of this.
> > +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
> > +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
> > +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
> > +
>
> To be on the safe side, you could add an isb here. I'm pretty sure it
> is immaterial on any ARMv7 core, but hey, I'm paranoid.
>

Well, this is actually triggering now under KVM, so I wonder if this
is a regression of some kind on the host side. But the EFI stub
shouldn't use CP15 barriers without enabling them, so we need this
patch in any case.

> With that:
>
> Acked-by: Marc Zyngier <marc.zyngier@arm.com>
>

Cheers,

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
  2019-03-30 13:10     ` Ard Biesheuvel
@ 2019-03-31  8:47       ` Marc Zyngier
  -1 siblings, 0 replies; 20+ messages in thread
From: Marc Zyngier @ 2019-03-31  8:47 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On Sat, 30 Mar 2019 13:10:58 +0000,
Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
> >
> > Hi Ard,
> >
> > On Fri, 29 Mar 2019 18:24:18 +0000,
> > Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > >
> > > The EFI stub is entered with the caches and MMU enabled by the
> > > firmware, and once the stub is ready to hand over to the decompressor,
> > > we clean and disable the caches.
> > >
> > > The cache clean routines use CP15 barrier instructions, which can be
> > > disabled via SCTLR. Normally, when using the provided cache handling
> > > routines to enable the caches and MMU, this bit is enabled as well.
> > > However, but since we entered the stub with the caches already enabled,
> > > this routine is not executed before we call the cache clean routines,
> > > resulting in undefined instruction exceptions if the firmware never
> > > enabled this bit.
> > >
> > > So set the bit explicitly in the EFI entry code.
> > >
> > > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > ---
> > >  arch/arm/boot/compressed/head.S | 10 ++++++++++
> > >  1 file changed, 10 insertions(+)
> > >
> > > diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> > > index 6c7ccb428c07..62a49356fca3 100644
> > > --- a/arch/arm/boot/compressed/head.S
> > > +++ b/arch/arm/boot/compressed/head.S
> > > @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
> > >
> > >               @ Preserve return value of efi_entry() in r4
> > >               mov     r4, r0
> > > +
> > > +             @ our cache maintenance code relies on CP15 barrier instructions
> > > +             @ but since we arrived here with the MMU and caches configured
> > > +             @ by UEFI, we must ensure that the use of those instructions is
> > > +             @ enabled in the SCTLR register, since we never executed our own
> > > +             @ cache enable routine, which is normally in charge of this.
> > > +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
> > > +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
> > > +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
> > > +
> >
> > To be on the safe side, you could add an isb here. I'm pretty sure it
> > is immaterial on any ARMv7 core, but hey, I'm paranoid.
> >
> 
> Well, this is actually triggering now under KVM, so I wonder if this
> is a regression of some kind on the host side. But the EFI stub
> shouldn't use CP15 barriers without enabling them, so we need this
> patch in any case.

My remark was only about the lack of an ISB instruction after the
write to SCTLR, and you're perfectly right to enable CP15BEN.

But there is something fishy. Looking at the KVM/arm64 code,
SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
something must have cleared it (or the reset code is busted...).

I've dumped SCTLR from a guest running EFI (using GDB), and I do see
SCTLR.CP15EN being set...

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-03-31  8:47       ` Marc Zyngier
  0 siblings, 0 replies; 20+ messages in thread
From: Marc Zyngier @ 2019-03-31  8:47 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On Sat, 30 Mar 2019 13:10:58 +0000,
Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
> >
> > Hi Ard,
> >
> > On Fri, 29 Mar 2019 18:24:18 +0000,
> > Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > >
> > > The EFI stub is entered with the caches and MMU enabled by the
> > > firmware, and once the stub is ready to hand over to the decompressor,
> > > we clean and disable the caches.
> > >
> > > The cache clean routines use CP15 barrier instructions, which can be
> > > disabled via SCTLR. Normally, when using the provided cache handling
> > > routines to enable the caches and MMU, this bit is enabled as well.
> > > However, but since we entered the stub with the caches already enabled,
> > > this routine is not executed before we call the cache clean routines,
> > > resulting in undefined instruction exceptions if the firmware never
> > > enabled this bit.
> > >
> > > So set the bit explicitly in the EFI entry code.
> > >
> > > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > ---
> > >  arch/arm/boot/compressed/head.S | 10 ++++++++++
> > >  1 file changed, 10 insertions(+)
> > >
> > > diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> > > index 6c7ccb428c07..62a49356fca3 100644
> > > --- a/arch/arm/boot/compressed/head.S
> > > +++ b/arch/arm/boot/compressed/head.S
> > > @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
> > >
> > >               @ Preserve return value of efi_entry() in r4
> > >               mov     r4, r0
> > > +
> > > +             @ our cache maintenance code relies on CP15 barrier instructions
> > > +             @ but since we arrived here with the MMU and caches configured
> > > +             @ by UEFI, we must ensure that the use of those instructions is
> > > +             @ enabled in the SCTLR register, since we never executed our own
> > > +             @ cache enable routine, which is normally in charge of this.
> > > +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
> > > +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
> > > +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
> > > +
> >
> > To be on the safe side, you could add an isb here. I'm pretty sure it
> > is immaterial on any ARMv7 core, but hey, I'm paranoid.
> >
> 
> Well, this is actually triggering now under KVM, so I wonder if this
> is a regression of some kind on the host side. But the EFI stub
> shouldn't use CP15 barriers without enabling them, so we need this
> patch in any case.

My remark was only about the lack of an ISB instruction after the
write to SCTLR, and you're perfectly right to enable CP15BEN.

But there is something fishy. Looking at the KVM/arm64 code,
SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
something must have cleared it (or the reset code is busted...).

I've dumped SCTLR from a guest running EFI (using GDB), and I do see
SCTLR.CP15EN being set...

Thanks,

	M.

-- 
Jazz is not dead, it just smell funny.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-07 18:19         ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-04-07 18:19 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On Sun, 31 Mar 2019 at 10:47, Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> On Sat, 30 Mar 2019 13:10:58 +0000,
> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >
> > On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > >
> > > Hi Ard,
> > >
> > > On Fri, 29 Mar 2019 18:24:18 +0000,
> > > Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > > >
> > > > The EFI stub is entered with the caches and MMU enabled by the
> > > > firmware, and once the stub is ready to hand over to the decompressor,
> > > > we clean and disable the caches.
> > > >
> > > > The cache clean routines use CP15 barrier instructions, which can be
> > > > disabled via SCTLR. Normally, when using the provided cache handling
> > > > routines to enable the caches and MMU, this bit is enabled as well.
> > > > However, but since we entered the stub with the caches already enabled,
> > > > this routine is not executed before we call the cache clean routines,
> > > > resulting in undefined instruction exceptions if the firmware never
> > > > enabled this bit.
> > > >
> > > > So set the bit explicitly in the EFI entry code.
> > > >
> > > > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > > ---
> > > >  arch/arm/boot/compressed/head.S | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> > > > index 6c7ccb428c07..62a49356fca3 100644
> > > > --- a/arch/arm/boot/compressed/head.S
> > > > +++ b/arch/arm/boot/compressed/head.S
> > > > @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
> > > >
> > > >               @ Preserve return value of efi_entry() in r4
> > > >               mov     r4, r0
> > > > +
> > > > +             @ our cache maintenance code relies on CP15 barrier instructions
> > > > +             @ but since we arrived here with the MMU and caches configured
> > > > +             @ by UEFI, we must ensure that the use of those instructions is
> > > > +             @ enabled in the SCTLR register, since we never executed our own
> > > > +             @ cache enable routine, which is normally in charge of this.
> > > > +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
> > > > +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
> > > > +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
> > > > +
> > >
> > > To be on the safe side, you could add an isb here. I'm pretty sure it
> > > is immaterial on any ARMv7 core, but hey, I'm paranoid.
> > >
> >
> > Well, this is actually triggering now under KVM, so I wonder if this
> > is a regression of some kind on the host side. But the EFI stub
> > shouldn't use CP15 barriers without enabling them, so we need this
> > patch in any case.
>
> My remark was only about the lack of an ISB instruction after the
> write to SCTLR, and you're perfectly right to enable CP15BEN.
>

Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
v6. Would reading back SCTLR suffice?

> But there is something fishy. Looking at the KVM/arm64 code,
> SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
> something must have cleared it (or the reset code is busted...).
>
> I've dumped SCTLR from a guest running EFI (using GDB), and I do see
> SCTLR.CP15EN being set...
>

Yeah. I won't be able to dig into this for a while, but in the mean
time, I'd like to get a guest fix in in any case.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-07 18:19         ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-04-07 18:19 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: linux-arm-kernel, linux-efi, Russell King, Leif Lindholm, Steve McIntyre

On Sun, 31 Mar 2019 at 10:47, Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> On Sat, 30 Mar 2019 13:10:58 +0000,
> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >
> > On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > >
> > > Hi Ard,
> > >
> > > On Fri, 29 Mar 2019 18:24:18 +0000,
> > > Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > > >
> > > > The EFI stub is entered with the caches and MMU enabled by the
> > > > firmware, and once the stub is ready to hand over to the decompressor,
> > > > we clean and disable the caches.
> > > >
> > > > The cache clean routines use CP15 barrier instructions, which can be
> > > > disabled via SCTLR. Normally, when using the provided cache handling
> > > > routines to enable the caches and MMU, this bit is enabled as well.
> > > > However, but since we entered the stub with the caches already enabled,
> > > > this routine is not executed before we call the cache clean routines,
> > > > resulting in undefined instruction exceptions if the firmware never
> > > > enabled this bit.
> > > >
> > > > So set the bit explicitly in the EFI entry code.
> > > >
> > > > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > > ---
> > > >  arch/arm/boot/compressed/head.S | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> > > > index 6c7ccb428c07..62a49356fca3 100644
> > > > --- a/arch/arm/boot/compressed/head.S
> > > > +++ b/arch/arm/boot/compressed/head.S
> > > > @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
> > > >
> > > >               @ Preserve return value of efi_entry() in r4
> > > >               mov     r4, r0
> > > > +
> > > > +             @ our cache maintenance code relies on CP15 barrier instructions
> > > > +             @ but since we arrived here with the MMU and caches configured
> > > > +             @ by UEFI, we must ensure that the use of those instructions is
> > > > +             @ enabled in the SCTLR register, since we never executed our own
> > > > +             @ cache enable routine, which is normally in charge of this.
> > > > +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
> > > > +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
> > > > +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
> > > > +
> > >
> > > To be on the safe side, you could add an isb here. I'm pretty sure it
> > > is immaterial on any ARMv7 core, but hey, I'm paranoid.
> > >
> >
> > Well, this is actually triggering now under KVM, so I wonder if this
> > is a regression of some kind on the host side. But the EFI stub
> > shouldn't use CP15 barriers without enabling them, so we need this
> > patch in any case.
>
> My remark was only about the lack of an ISB instruction after the
> write to SCTLR, and you're perfectly right to enable CP15BEN.
>

Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
v6. Would reading back SCTLR suffice?

> But there is something fishy. Looking at the KVM/arm64 code,
> SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
> something must have cleared it (or the reset code is busted...).
>
> I've dumped SCTLR from a guest running EFI (using GDB), and I do see
> SCTLR.CP15EN being set...
>

Yeah. I won't be able to dig into this for a while, but in the mean
time, I'd like to get a guest fix in in any case.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-07 18:19         ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-04-07 18:19 UTC (permalink / raw)
  To: Marc Zyngier
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On Sun, 31 Mar 2019 at 10:47, Marc Zyngier <marc.zyngier@arm.com> wrote:
>
> On Sat, 30 Mar 2019 13:10:58 +0000,
> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >
> > On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
> > >
> > > Hi Ard,
> > >
> > > On Fri, 29 Mar 2019 18:24:18 +0000,
> > > Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> > > >
> > > > The EFI stub is entered with the caches and MMU enabled by the
> > > > firmware, and once the stub is ready to hand over to the decompressor,
> > > > we clean and disable the caches.
> > > >
> > > > The cache clean routines use CP15 barrier instructions, which can be
> > > > disabled via SCTLR. Normally, when using the provided cache handling
> > > > routines to enable the caches and MMU, this bit is enabled as well.
> > > > However, but since we entered the stub with the caches already enabled,
> > > > this routine is not executed before we call the cache clean routines,
> > > > resulting in undefined instruction exceptions if the firmware never
> > > > enabled this bit.
> > > >
> > > > So set the bit explicitly in the EFI entry code.
> > > >
> > > > Cc: Marc Zyngier <marc.zyngier@arm.com>
> > > > Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> > > > ---
> > > >  arch/arm/boot/compressed/head.S | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
> > > > index 6c7ccb428c07..62a49356fca3 100644
> > > > --- a/arch/arm/boot/compressed/head.S
> > > > +++ b/arch/arm/boot/compressed/head.S
> > > > @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
> > > >
> > > >               @ Preserve return value of efi_entry() in r4
> > > >               mov     r4, r0
> > > > +
> > > > +             @ our cache maintenance code relies on CP15 barrier instructions
> > > > +             @ but since we arrived here with the MMU and caches configured
> > > > +             @ by UEFI, we must ensure that the use of those instructions is
> > > > +             @ enabled in the SCTLR register, since we never executed our own
> > > > +             @ cache enable routine, which is normally in charge of this.
> > > > +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
> > > > +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
> > > > +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
> > > > +
> > >
> > > To be on the safe side, you could add an isb here. I'm pretty sure it
> > > is immaterial on any ARMv7 core, but hey, I'm paranoid.
> > >
> >
> > Well, this is actually triggering now under KVM, so I wonder if this
> > is a regression of some kind on the host side. But the EFI stub
> > shouldn't use CP15 barriers without enabling them, so we need this
> > patch in any case.
>
> My remark was only about the lack of an ISB instruction after the
> write to SCTLR, and you're perfectly right to enable CP15BEN.
>

Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
v6. Would reading back SCTLR suffice?

> But there is something fishy. Looking at the KVM/arm64 code,
> SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
> something must have cleared it (or the reset code is busted...).
>
> I've dumped SCTLR from a guest running EFI (using GDB), and I do see
> SCTLR.CP15EN being set...
>

Yeah. I won't be able to dig into this for a while, but in the mean
time, I'd like to get a guest fix in in any case.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08  9:11           ` Marc Zyngier
  0 siblings, 0 replies; 20+ messages in thread
From: Marc Zyngier @ 2019-04-08  9:11 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On 07/04/2019 19:19, Ard Biesheuvel wrote:
> On Sun, 31 Mar 2019 at 10:47, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>
>> On Sat, 30 Mar 2019 13:10:58 +0000,
>> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>>
>>> On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>>>
>>>> Hi Ard,
>>>>
>>>> On Fri, 29 Mar 2019 18:24:18 +0000,
>>>> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>>>>
>>>>> The EFI stub is entered with the caches and MMU enabled by the
>>>>> firmware, and once the stub is ready to hand over to the decompressor,
>>>>> we clean and disable the caches.
>>>>>
>>>>> The cache clean routines use CP15 barrier instructions, which can be
>>>>> disabled via SCTLR. Normally, when using the provided cache handling
>>>>> routines to enable the caches and MMU, this bit is enabled as well.
>>>>> However, but since we entered the stub with the caches already enabled,
>>>>> this routine is not executed before we call the cache clean routines,
>>>>> resulting in undefined instruction exceptions if the firmware never
>>>>> enabled this bit.
>>>>>
>>>>> So set the bit explicitly in the EFI entry code.
>>>>>
>>>>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>>>>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>>> ---
>>>>>  arch/arm/boot/compressed/head.S | 10 ++++++++++
>>>>>  1 file changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
>>>>> index 6c7ccb428c07..62a49356fca3 100644
>>>>> --- a/arch/arm/boot/compressed/head.S
>>>>> +++ b/arch/arm/boot/compressed/head.S
>>>>> @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
>>>>>
>>>>>               @ Preserve return value of efi_entry() in r4
>>>>>               mov     r4, r0
>>>>> +
>>>>> +             @ our cache maintenance code relies on CP15 barrier instructions
>>>>> +             @ but since we arrived here with the MMU and caches configured
>>>>> +             @ by UEFI, we must ensure that the use of those instructions is
>>>>> +             @ enabled in the SCTLR register, since we never executed our own
>>>>> +             @ cache enable routine, which is normally in charge of this.
>>>>> +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
>>>>> +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
>>>>> +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
>>>>> +
>>>>
>>>> To be on the safe side, you could add an isb here. I'm pretty sure it
>>>> is immaterial on any ARMv7 core, but hey, I'm paranoid.
>>>>
>>>
>>> Well, this is actually triggering now under KVM, so I wonder if this
>>> is a regression of some kind on the host side. But the EFI stub
>>> shouldn't use CP15 barriers without enabling them, so we need this
>>> patch in any case.
>>
>> My remark was only about the lack of an ISB instruction after the
>> write to SCTLR, and you're perfectly right to enable CP15BEN.
>>
> 
> Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> v6. Would reading back SCTLR suffice?

I don't think it would, at least from an architectural perspective. But
any CPU that has a non RAO/WI SCTLR.CP15BEN must also support the
architected barriers, otherwise it wouldn't be able to flip them back
on. I believe this is the case on all ARMv6 CPUs.

We could write something like:

	mrc	p15, 0, r1, c1, c0, 0	@ read SCTLR
	tst	r1, r1, #(1 << 5)
	bne	1f
	orr	r1, r1, #(1 << 5)	@ CP15 barrier instructions
	mcr	p15, 0, r1, c1, c0, 0	@ write SCTLR
	isb
1:

What do you think?

> 
>> But there is something fishy. Looking at the KVM/arm64 code,
>> SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
>> something must have cleared it (or the reset code is busted...).
>>
>> I've dumped SCTLR from a guest running EFI (using GDB), and I do see
>> SCTLR.CP15EN being set...
>>
> 
> Yeah. I won't be able to dig into this for a while, but in the mean
> time, I'd like to get a guest fix in in any case.

Absolutely. These are two distinct issues, and the first one definitely
needs addressing.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08  9:11           ` Marc Zyngier
  0 siblings, 0 replies; 20+ messages in thread
From: Marc Zyngier @ 2019-04-08  9:11 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: linux-arm-kernel, linux-efi, Russell King, Leif Lindholm, Steve McIntyre

On 07/04/2019 19:19, Ard Biesheuvel wrote:
> On Sun, 31 Mar 2019 at 10:47, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>
>> On Sat, 30 Mar 2019 13:10:58 +0000,
>> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>>
>>> On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>>>
>>>> Hi Ard,
>>>>
>>>> On Fri, 29 Mar 2019 18:24:18 +0000,
>>>> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>>>>
>>>>> The EFI stub is entered with the caches and MMU enabled by the
>>>>> firmware, and once the stub is ready to hand over to the decompressor,
>>>>> we clean and disable the caches.
>>>>>
>>>>> The cache clean routines use CP15 barrier instructions, which can be
>>>>> disabled via SCTLR. Normally, when using the provided cache handling
>>>>> routines to enable the caches and MMU, this bit is enabled as well.
>>>>> However, but since we entered the stub with the caches already enabled,
>>>>> this routine is not executed before we call the cache clean routines,
>>>>> resulting in undefined instruction exceptions if the firmware never
>>>>> enabled this bit.
>>>>>
>>>>> So set the bit explicitly in the EFI entry code.
>>>>>
>>>>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>>>>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>>> ---
>>>>>  arch/arm/boot/compressed/head.S | 10 ++++++++++
>>>>>  1 file changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
>>>>> index 6c7ccb428c07..62a49356fca3 100644
>>>>> --- a/arch/arm/boot/compressed/head.S
>>>>> +++ b/arch/arm/boot/compressed/head.S
>>>>> @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
>>>>>
>>>>>               @ Preserve return value of efi_entry() in r4
>>>>>               mov     r4, r0
>>>>> +
>>>>> +             @ our cache maintenance code relies on CP15 barrier instructions
>>>>> +             @ but since we arrived here with the MMU and caches configured
>>>>> +             @ by UEFI, we must ensure that the use of those instructions is
>>>>> +             @ enabled in the SCTLR register, since we never executed our own
>>>>> +             @ cache enable routine, which is normally in charge of this.
>>>>> +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
>>>>> +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
>>>>> +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
>>>>> +
>>>>
>>>> To be on the safe side, you could add an isb here. I'm pretty sure it
>>>> is immaterial on any ARMv7 core, but hey, I'm paranoid.
>>>>
>>>
>>> Well, this is actually triggering now under KVM, so I wonder if this
>>> is a regression of some kind on the host side. But the EFI stub
>>> shouldn't use CP15 barriers without enabling them, so we need this
>>> patch in any case.
>>
>> My remark was only about the lack of an ISB instruction after the
>> write to SCTLR, and you're perfectly right to enable CP15BEN.
>>
> 
> Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> v6. Would reading back SCTLR suffice?

I don't think it would, at least from an architectural perspective. But
any CPU that has a non RAO/WI SCTLR.CP15BEN must also support the
architected barriers, otherwise it wouldn't be able to flip them back
on. I believe this is the case on all ARMv6 CPUs.

We could write something like:

	mrc	p15, 0, r1, c1, c0, 0	@ read SCTLR
	tst	r1, r1, #(1 << 5)
	bne	1f
	orr	r1, r1, #(1 << 5)	@ CP15 barrier instructions
	mcr	p15, 0, r1, c1, c0, 0	@ write SCTLR
	isb
1:

What do you think?

> 
>> But there is something fishy. Looking at the KVM/arm64 code,
>> SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
>> something must have cleared it (or the reset code is busted...).
>>
>> I've dumped SCTLR from a guest running EFI (using GDB), and I do see
>> SCTLR.CP15EN being set...
>>
> 
> Yeah. I won't be able to dig into this for a while, but in the mean
> time, I'd like to get a guest fix in in any case.

Absolutely. These are two distinct issues, and the first one definitely
needs addressing.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08  9:11           ` Marc Zyngier
  0 siblings, 0 replies; 20+ messages in thread
From: Marc Zyngier @ 2019-04-08  9:11 UTC (permalink / raw)
  To: Ard Biesheuvel
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On 07/04/2019 19:19, Ard Biesheuvel wrote:
> On Sun, 31 Mar 2019 at 10:47, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>
>> On Sat, 30 Mar 2019 13:10:58 +0000,
>> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>>
>>> On Sat, 30 Mar 2019 at 10:50, Marc Zyngier <marc.zyngier@arm.com> wrote:
>>>>
>>>> Hi Ard,
>>>>
>>>> On Fri, 29 Mar 2019 18:24:18 +0000,
>>>> Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>>>>
>>>>> The EFI stub is entered with the caches and MMU enabled by the
>>>>> firmware, and once the stub is ready to hand over to the decompressor,
>>>>> we clean and disable the caches.
>>>>>
>>>>> The cache clean routines use CP15 barrier instructions, which can be
>>>>> disabled via SCTLR. Normally, when using the provided cache handling
>>>>> routines to enable the caches and MMU, this bit is enabled as well.
>>>>> However, but since we entered the stub with the caches already enabled,
>>>>> this routine is not executed before we call the cache clean routines,
>>>>> resulting in undefined instruction exceptions if the firmware never
>>>>> enabled this bit.
>>>>>
>>>>> So set the bit explicitly in the EFI entry code.
>>>>>
>>>>> Cc: Marc Zyngier <marc.zyngier@arm.com>
>>>>> Signed-off-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
>>>>> ---
>>>>>  arch/arm/boot/compressed/head.S | 10 ++++++++++
>>>>>  1 file changed, 10 insertions(+)
>>>>>
>>>>> diff --git a/arch/arm/boot/compressed/head.S b/arch/arm/boot/compressed/head.S
>>>>> index 6c7ccb428c07..62a49356fca3 100644
>>>>> --- a/arch/arm/boot/compressed/head.S
>>>>> +++ b/arch/arm/boot/compressed/head.S
>>>>> @@ -1438,6 +1438,16 @@ ENTRY(efi_stub_entry)
>>>>>
>>>>>               @ Preserve return value of efi_entry() in r4
>>>>>               mov     r4, r0
>>>>> +
>>>>> +             @ our cache maintenance code relies on CP15 barrier instructions
>>>>> +             @ but since we arrived here with the MMU and caches configured
>>>>> +             @ by UEFI, we must ensure that the use of those instructions is
>>>>> +             @ enabled in the SCTLR register, since we never executed our own
>>>>> +             @ cache enable routine, which is normally in charge of this.
>>>>> +             mrc     p15, 0, r1, c1, c0, 0   @ read SCTLR
>>>>> +             orr     r1, r1, #(1 << 5)       @ CP15 barrier instructions
>>>>> +             mcr     p15, 0, r1, c1, c0, 0   @ write SCTLR
>>>>> +
>>>>
>>>> To be on the safe side, you could add an isb here. I'm pretty sure it
>>>> is immaterial on any ARMv7 core, but hey, I'm paranoid.
>>>>
>>>
>>> Well, this is actually triggering now under KVM, so I wonder if this
>>> is a regression of some kind on the host side. But the EFI stub
>>> shouldn't use CP15 barriers without enabling them, so we need this
>>> patch in any case.
>>
>> My remark was only about the lack of an ISB instruction after the
>> write to SCTLR, and you're perfectly right to enable CP15BEN.
>>
> 
> Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> v6. Would reading back SCTLR suffice?

I don't think it would, at least from an architectural perspective. But
any CPU that has a non RAO/WI SCTLR.CP15BEN must also support the
architected barriers, otherwise it wouldn't be able to flip them back
on. I believe this is the case on all ARMv6 CPUs.

We could write something like:

	mrc	p15, 0, r1, c1, c0, 0	@ read SCTLR
	tst	r1, r1, #(1 << 5)
	bne	1f
	orr	r1, r1, #(1 << 5)	@ CP15 barrier instructions
	mcr	p15, 0, r1, c1, c0, 0	@ write SCTLR
	isb
1:

What do you think?

> 
>> But there is something fishy. Looking at the KVM/arm64 code,
>> SCTLR_EL1.CP15BEN resets to 1. If the barrier undefs in the stubs,
>> something must have cleared it (or the reset code is busted...).
>>
>> I've dumped SCTLR from a guest running EFI (using GDB), and I do see
>> SCTLR.CP15EN being set...
>>
> 
> Yeah. I won't be able to dig into this for a while, but in the mean
> time, I'd like to get a guest fix in in any case.

Absolutely. These are two distinct issues, and the first one definitely
needs addressing.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08  9:20           ` Vladimir Murzin
  0 siblings, 0 replies; 20+ messages in thread
From: Vladimir Murzin @ 2019-04-08  9:20 UTC (permalink / raw)
  To: Ard Biesheuvel, Marc Zyngier
  Cc: linux-arm-kernel, linux-efi, Steve McIntyre, Russell King, Leif Lindholm

On 4/7/19 7:19 PM, Ard Biesheuvel wrote:
> Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> v6. Would reading back SCTLR suffice?


I think instr_sync macro should do the trick.

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08  9:20           ` Vladimir Murzin
  0 siblings, 0 replies; 20+ messages in thread
From: Vladimir Murzin @ 2019-04-08  9:20 UTC (permalink / raw)
  To: Ard Biesheuvel, Marc Zyngier
  Cc: Leif Lindholm, linux-efi, Steve McIntyre, Russell King, linux-arm-kernel

On 4/7/19 7:19 PM, Ard Biesheuvel wrote:
> Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> v6. Would reading back SCTLR suffice?


I think instr_sync macro should do the trick.

Cheers
Vladimir

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08  9:20           ` Vladimir Murzin
  0 siblings, 0 replies; 20+ messages in thread
From: Vladimir Murzin @ 2019-04-08  9:20 UTC (permalink / raw)
  To: Ard Biesheuvel, Marc Zyngier
  Cc: linux-arm-kernel, linux-efi, Steve McIntyre, Russell King, Leif Lindholm

On 4/7/19 7:19 PM, Ard Biesheuvel wrote:
> Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> v6. Would reading back SCTLR suffice?


I think instr_sync macro should do the trick.

Cheers
Vladimir

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08 21:38             ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-04-08 21:38 UTC (permalink / raw)
  To: Vladimir Murzin
  Cc: linux-efi, Marc Zyngier, Russell King, Leif Lindholm,
	Steve McIntyre, linux-arm-kernel

On Mon, 8 Apr 2019 at 11:20, Vladimir Murzin <vladimir.murzin@arm.com> wrote:
>
> On 4/7/19 7:19 PM, Ard Biesheuvel wrote:
> > Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> > v6. Would reading back SCTLR suffice?
>
>
> I think instr_sync macro should do the trick.
>

This code should run on v7 when compiled for v6, and instr_sync uses
the CP15 ISB in that case, which we cannot use since that is what we
are enabling in the first place.

I'll go with Marc's suggestion, which should work in all cases
(although it may require a .arch directive to ensure the isb is
accepted)

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08 21:38             ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-04-08 21:38 UTC (permalink / raw)
  To: Vladimir Murzin
  Cc: Marc Zyngier, Leif Lindholm, linux-efi, Steve McIntyre,
	Russell King, linux-arm-kernel

On Mon, 8 Apr 2019 at 11:20, Vladimir Murzin <vladimir.murzin@arm.com> wrote:
>
> On 4/7/19 7:19 PM, Ard Biesheuvel wrote:
> > Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> > v6. Would reading back SCTLR suffice?
>
>
> I think instr_sync macro should do the trick.
>

This code should run on v7 when compiled for v6, and instr_sync uses
the CP15 ISB in that case, which we cannot use since that is what we
are enabling in the first place.

I'll go with Marc's suggestion, which should work in all cases
(although it may require a .arch directive to ensure the isb is
accepted)

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache
@ 2019-04-08 21:38             ` Ard Biesheuvel
  0 siblings, 0 replies; 20+ messages in thread
From: Ard Biesheuvel @ 2019-04-08 21:38 UTC (permalink / raw)
  To: Vladimir Murzin
  Cc: linux-efi, Marc Zyngier, Russell King, Leif Lindholm,
	Steve McIntyre, linux-arm-kernel

On Mon, 8 Apr 2019 at 11:20, Vladimir Murzin <vladimir.murzin@arm.com> wrote:
>
> On 4/7/19 7:19 PM, Ard Biesheuvel wrote:
> > Actually, the CP15 ISB is not usable here, and using the v7 ISB breaks
> > v6. Would reading back SCTLR suffice?
>
>
> I think instr_sync macro should do the trick.
>

This code should run on v7 when compiled for v6, and instr_sync uses
the CP15 ISB in that case, which we cannot use since that is what we
are enabling in the first place.

I'll go with Marc's suggestion, which should work in all cases
(although it may require a .arch directive to ensure the isb is
accepted)

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2019-04-08 21:39 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-29 18:24 [PATCH] efi/arm: enable CP15 DMB instructions before cleaning the cache Ard Biesheuvel
2019-03-29 18:24 ` Ard Biesheuvel
2019-03-30  9:50 ` Marc Zyngier
2019-03-30  9:50   ` Marc Zyngier
2019-03-30 13:10   ` Ard Biesheuvel
2019-03-30 13:10     ` Ard Biesheuvel
2019-03-31  8:47     ` Marc Zyngier
2019-03-31  8:47       ` Marc Zyngier
2019-04-07 18:19       ` Ard Biesheuvel
2019-04-07 18:19         ` Ard Biesheuvel
2019-04-07 18:19         ` Ard Biesheuvel
2019-04-08  9:11         ` Marc Zyngier
2019-04-08  9:11           ` Marc Zyngier
2019-04-08  9:11           ` Marc Zyngier
2019-04-08  9:20         ` Vladimir Murzin
2019-04-08  9:20           ` Vladimir Murzin
2019-04-08  9:20           ` Vladimir Murzin
2019-04-08 21:38           ` Ard Biesheuvel
2019-04-08 21:38             ` Ard Biesheuvel
2019-04-08 21:38             ` Ard Biesheuvel

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.