[PATCH] x86/efistub: disable paging at mixed mode entry

[PATCH] x86/efistub: disable paging at mixed mode entry

[Ard Biesheuvel]

The EFI mixed mode entry code goes through the ordinary startup_32()
routine before jumping into the kernel's EFI boot code in 64-bit
mode. The 32-bit startup code must be entered with paging disabled,
but this is not documented as a requirement for the EFI handover
protocol, and so we should disable paging explicitly when entering
the kernel from 32-bit EFI firmware.

Cc: <stable@vger.kernel.org>
Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 arch/x86/boot/compressed/head_64.S | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/arch/x86/boot/compressed/head_64.S b/arch/x86/boot/compressed/head_64.S
index 58a512e33d8d..ee60b81944a7 100644
--- a/arch/x86/boot/compressed/head_64.S
+++ b/arch/x86/boot/compressed/head_64.S
@@ -244,6 +244,11 @@ SYM_FUNC_START(efi32_stub_entry)
 	leal	efi32_config(%ebp), %eax
 	movl	%eax, efi_config(%ebp)
 
+	/* Disable paging */
+	movl	%cr0, %eax
+	btrl	$X86_CR0_PG_BIT, %eax
+	movl	%eax, %cr0
+
 	jmp	startup_32
 SYM_FUNC_END(efi32_stub_entry)
 #endif
-- 
2.20.1

Re: [PATCH] x86/efistub: disable paging at mixed mode entry

[Andy Lutomirski]

On Mon, Dec 23, 2019 at 7:23 AM Ard Biesheuvel <ardb@kernel.org> wrote:
>
> The EFI mixed mode entry code goes through the ordinary startup_32()
> routine before jumping into the kernel's EFI boot code in 64-bit
> mode. The 32-bit startup code must be entered with paging disabled,
> but this is not documented as a requirement for the EFI handover
> protocol, and so we should disable paging explicitly when entering
> the kernel from 32-bit EFI firmware.

Does this mean that EFI is allowed to call the kernel with paging on
but the text identity-mapped?  Have you seen this happen in practice?

If the kernel is entered with paging on and the text not
identity-mapped, this is going to blow up badly.

--Andy

Re: [PATCH] x86/efistub: disable paging at mixed mode entry

[Ard Biesheuvel]

On Tue, 24 Dec 2019 at 03:15, Andy Lutomirski <luto@amacapital.net> wrote:
>
> On Mon, Dec 23, 2019 at 7:23 AM Ard Biesheuvel <ardb@kernel.org> wrote:
> >
> > The EFI mixed mode entry code goes through the ordinary startup_32()
> > routine before jumping into the kernel's EFI boot code in 64-bit
> > mode. The 32-bit startup code must be entered with paging disabled,
> > but this is not documented as a requirement for the EFI handover
> > protocol, and so we should disable paging explicitly when entering
> > the kernel from 32-bit EFI firmware.
>
> Does this mean that EFI is allowed to call the kernel with paging on
> but the text identity-mapped?

Yes. This is explicitly mentioned in the spec. Paging may be on or
off, but all memory must be mapped 1:1

>  Have you seen this happen in practice?
>

Yes. GRUB and OVMF both implement the EFI handover protocol, but OVMF
doesn't disable paging before calling the 32-bit entry point, and so
it explodes in startup_32(). GRUB calls the EFI handover entrypoint
with paging disabled, and so then everything works fine.

> If the kernel is entered with paging on and the text not
> identity-mapped, this is going to blow up badly.
>

Not just text: all of system memory is guaranteed to be 1:1 mapped if
paging is on when entering the kernel from EFI, so this should be
safe.
Note that this change only affects mixed mode configurations that use
OVMF instead of GRUB.

We are using OVMF/qemu in kernelCI for test coverage of the EFI stub
for all architectures, and this is the missing puzzle piece to get it
working in x86 mixed mode as well.

Re: [PATCH] x86/efistub: disable paging at mixed mode entry

[Andy Lutomirski]

> On Dec 24, 2019, at 3:38 PM, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> On Tue, 24 Dec 2019 at 03:15, Andy Lutomirski <luto@amacapital.net> wrote:
>> 
>>> On Mon, Dec 23, 2019 at 7:23 AM Ard Biesheuvel <ardb@kernel.org> wrote:
>>> 
>>> The EFI mixed mode entry code goes through the ordinary startup_32()
>>> routine before jumping into the kernel's EFI boot code in 64-bit
>>> mode. The 32-bit startup code must be entered with paging disabled,
>>> but this is not documented as a requirement for the EFI handover
>>> protocol, and so we should disable paging explicitly when entering
>>> the kernel from 32-bit EFI firmware.
>> 
>> Does this mean that EFI is allowed to call the kernel with paging on
>> but the text identity-mapped?
> 
> Yes. This is explicitly mentioned in the spec. Paging may be on or
> off, but all memory must be mapped 1:1
> 
>> Have you seen this happen in practice?
>> 
> 
> Yes. GRUB and OVMF both implement the EFI handover protocol, but OVMF
> doesn't disable paging before calling the 32-bit entry point, and so
> it explodes in startup_32(). GRUB calls the EFI handover entrypoint
> with paging disabled, and so then everything works fine.
> 
>> If the kernel is entered with paging on and the text not
>> identity-mapped, this is going to blow up badly.
>> 
> 
> Not just text: all of system memory is guaranteed to be 1:1 mapped if
> paging is on when entering the kernel from EFI, so this should be
> safe.
> Note that this change only affects mixed mode configurations that use
> OVMF instead of GRUB.
> 
> We are using OVMF/qemu in kernelCI for test coverage of the EFI stub
> for all architectures, and this is the missing puzzle piece to get it
> working in x86 mixed mode as well.

Sounds good to me, then.

I admit to being a bit confused, since I would have sworn that I’ve personally tested mixed mode with OVMF.

Re: [PATCH] x86/efistub: disable paging at mixed mode entry

[Ard Biesheuvel]

On Tue, 24 Dec 2019 at 08:47, Andy Lutomirski <luto@amacapital.net> wrote:
>
>
>
> > On Dec 24, 2019, at 3:38 PM, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> >
> > On Tue, 24 Dec 2019 at 03:15, Andy Lutomirski <luto@amacapital.net> wrote:
> >>
> >>> On Mon, Dec 23, 2019 at 7:23 AM Ard Biesheuvel <ardb@kernel.org> wrote:
> >>>
> >>> The EFI mixed mode entry code goes through the ordinary startup_32()
> >>> routine before jumping into the kernel's EFI boot code in 64-bit
> >>> mode. The 32-bit startup code must be entered with paging disabled,
> >>> but this is not documented as a requirement for the EFI handover
> >>> protocol, and so we should disable paging explicitly when entering
> >>> the kernel from 32-bit EFI firmware.
> >>
> >> Does this mean that EFI is allowed to call the kernel with paging on
> >> but the text identity-mapped?
> >
> > Yes. This is explicitly mentioned in the spec. Paging may be on or
> > off, but all memory must be mapped 1:1
> >
> >> Have you seen this happen in practice?
> >>
> >
> > Yes. GRUB and OVMF both implement the EFI handover protocol, but OVMF
> > doesn't disable paging before calling the 32-bit entry point, and so
> > it explodes in startup_32(). GRUB calls the EFI handover entrypoint
> > with paging disabled, and so then everything works fine.
> >
> >> If the kernel is entered with paging on and the text not
> >> identity-mapped, this is going to blow up badly.
> >>
> >
> > Not just text: all of system memory is guaranteed to be 1:1 mapped if
> > paging is on when entering the kernel from EFI, so this should be
> > safe.
> > Note that this change only affects mixed mode configurations that use
> > OVMF instead of GRUB.
> >
> > We are using OVMF/qemu in kernelCI for test coverage of the EFI stub
> > for all architectures, and this is the missing puzzle piece to get it
> > working in x86 mixed mode as well.
>
> Sounds good to me, then.
>
> I admit to being a bit confused, since I would have sworn that I’ve personally tested mixed mode with OVMF.

Interesting that you should say that. I thought exactly the same, but
after noticing that it didn't work, I went back years testing old
kernels and old builds of OVMF, and no combination that I tried would
actually work without a change like the one in this patch. This is
using both KVM and emulation, so it doesn't seem likely that this is
caused by a change in QEMU.

Re: [PATCH] x86/efistub: disable paging at mixed mode entry

[Andy Lutomirski]

> On Dec 24, 2019, at 3:50 PM, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> 
> On Tue, 24 Dec 2019 at 08:47, Andy Lutomirski <luto@amacapital.net> wrote:
>> 
>> 
>> 
>>>> On Dec 24, 2019, at 3:38 PM, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
>>> 
>>> On Tue, 24 Dec 2019 at 03:15, Andy Lutomirski <luto@amacapital.net> wrote:
>>>> 
>>>>> On Mon, Dec 23, 2019 at 7:23 AM Ard Biesheuvel <ardb@kernel.org> wrote:
>>>>> 
>>>>> The EFI mixed mode entry code goes through the ordinary startup_32()
>>>>> routine before jumping into the kernel's EFI boot code in 64-bit
>>>>> mode. The 32-bit startup code must be entered with paging disabled,
>>>>> but this is not documented as a requirement for the EFI handover
>>>>> protocol, and so we should disable paging explicitly when entering
>>>>> the kernel from 32-bit EFI firmware.
>>>> 
>>>> Does this mean that EFI is allowed to call the kernel with paging on
>>>> but the text identity-mapped?
>>> 
>>> Yes. This is explicitly mentioned in the spec. Paging may be on or
>>> off, but all memory must be mapped 1:1
>>> 
>>>> Have you seen this happen in practice?
>>>> 
>>> 
>>> Yes. GRUB and OVMF both implement the EFI handover protocol, but OVMF
>>> doesn't disable paging before calling the 32-bit entry point, and so
>>> it explodes in startup_32(). GRUB calls the EFI handover entrypoint
>>> with paging disabled, and so then everything works fine.
>>> 
>>>> If the kernel is entered with paging on and the text not
>>>> identity-mapped, this is going to blow up badly.
>>>> 
>>> 
>>> Not just text: all of system memory is guaranteed to be 1:1 mapped if
>>> paging is on when entering the kernel from EFI, so this should be
>>> safe.
>>> Note that this change only affects mixed mode configurations that use
>>> OVMF instead of GRUB.
>>> 
>>> We are using OVMF/qemu in kernelCI for test coverage of the EFI stub
>>> for all architectures, and this is the missing puzzle piece to get it
>>> working in x86 mixed mode as well.
>> 
>> Sounds good to me, then.
>> 
>> I admit to being a bit confused, since I would have sworn that I’ve personally tested mixed mode with OVMF.
> 
> Interesting that you should say that. I thought exactly the same, but
> after noticing that it didn't work, I went back years testing old
> kernels and old builds of OVMF, and no combination that I tried would
> actually work without a change like the one in this patch. This is
> using both KVM and emulation, so it doesn't seem likely that this is
> caused by a change in QEMU.

It would have been a build from Red Hat or Fedora, possibly a prerelease I got from Peter Jones. And I reproduced a severe bug in the kernel that I had found by inspection, so it wasn’t totally my imagination. Maybe I actually had OVMF chaining to GRUB?

The bug in question was that an NMI hitting EFI code would explode back when EFI mixed mode worked by fiddling with EFER to exit long mode. I think I tested by running perf while reading efivars.

I probably passed -kernel to QEMU while also telling QEMU to use OVMF.

Anyway, it’s okay if this mystery doesn’t get solved.