From: Ard Biesheuvel <ardb@kernel.org> To: Will Deacon <will@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com>, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, keescook@chromium.org, Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org> Subject: Re: [PATCH v5 0/7] arm64: efi: leave MMU and caches on at boot Date: Sat, 26 Nov 2022 15:17:15 +0100 [thread overview] Message-ID: <CAMj1kXH+B3vKkrkMg_NcZNsP4zSULuUgQ-xJwiq60UJs2wLP3A@mail.gmail.com> (raw) In-Reply-To: <CAMj1kXFdJHvT-XKaEC0tXpxM_RMfCic_sS-JkMkVYfKZCYkZuA@mail.gmail.com> On Sat, 26 Nov 2022 at 15:16, Ard Biesheuvel <ardb@kernel.org> wrote: > > On Tue, 15 Nov 2022 at 12:31, Will Deacon <will@kernel.org> wrote: > > > > On Tue, Nov 15, 2022 at 12:21:55PM +0100, Ard Biesheuvel wrote: > > > On Tue, 15 Nov 2022 at 12:17, Will Deacon <will@kernel.org> wrote: > > > > > > > > On Fri, Nov 11, 2022 at 05:36:19PM +0000, Mark Rutland wrote: > > > > > On Tue, Nov 08, 2022 at 07:21:57PM +0100, Ard Biesheuvel wrote: > > > > > > The purpose of this series is to remove any explicit cache maintenance > > > > > > for coherency during early boot that becomes unnecessary if we simply > > > > > > retain the cacheable 1:1 mapping of all of system RAM provided by EFI, > > > > > > and use it to populate the ID map page tables. After setting up this > > > > > > preliminary ID map, we disable the MMU, drop to EL1, reprogram the MAIR, > > > > > > TCR and SCTLR registers as before, and proceed as usual, avoiding the > > > > > > need for any manipulations of memory while the MMU and caches are off. > > > > > > > > > > > > The only properties of the firmware provided 1:1 map we rely on is that > > > > > > it does not require any explicit cache maintenance for coherency, and > > > > > > that it covers the entire memory footprint of the image, including the > > > > > > BSS and padding at the end - all else is under control of the kernel > > > > > > itself, as before. > > > > > > > > > > As a high-level thing, I'm still very much not keen on entering the kernel with > > > > > the MMU on. Given that we have to support booting with the MMU off for !EFI > > > > > boot (including kexec when EFI is in use), I think this makes it harder to > > > > > reason about the boot code overall (e.g. due to the conditional maintenance > > > > > added to head.S), and adds more scope for error, even if it simplifies the EFI > > > > > stub itself. > > > > > > > > As discussed offline, two things that would help the current series are: > > > > > > > > (1) Some performance numbers comparing MMU off vs MMU on boot > > > > > > Finally got around to measuring this - I lost access to my TX2 machine > for a couple of days during the past week, > > With the patch below applied to mainline, I measure ~6 ms spent > cleaning the entire image to the PoC (which is the bulk of it) and > subsequently populating the initial ID map and activating it. > > This drops to about 0.6 ms with my changes applied. This is unlikely > to ever matter in practice, perhaps, but I will note that booting a VM > in EFI mode using Tianocore/EDK2 from the point where KVM clears the > counter to the point where we start user space can be done (on the > same machine) in 500-700 ms so it is not entirely insignificant > either. > > I could try and measure it on bare metal as well, but I suppose that > launch times are even less relevant there so I didn't bother. diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 61a87fa1c3055e26..27f59784a1c0be2c 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -22,6 +22,7 @@ SYM_CODE_START(efi_enter_kernel) ldr w2, =primary_entry_offset add x19, x0, x2 // relocated Image entrypoint mov x20, x1 // DTB address + mrs x27, cntvct_el0 /* * Clean the copied Image to the PoC, and ensure it is not shadowed by diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 2196aad7b55bcef0..068a7d111836382b 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -430,6 +430,8 @@ SYM_FUNC_START_LOCAL(__primary_switched) str_l x21, __fdt_pointer, x5 // Save FDT pointer + str_l x27, boot_args + 8, x5 + ldr_l x4, kimage_vaddr // Save the offset between sub x4, x4, x0 // the kernel virtual and str_l x4, kimage_voffset, x5 // physical mappings @@ -797,6 +799,10 @@ SYM_FUNC_START_LOCAL(__primary_switch) adrp x1, reserved_pg_dir adrp x2, init_idmap_pg_dir bl __enable_mmu + + mrs x0, cntvct_el0 + sub x27, x0, x27 + #ifdef CONFIG_RELOCATABLE adrp x23, KERNEL_START and x23, x23, MIN_KIMG_ALIGN - 1
WARNING: multiple messages have this Message-ID (diff)
From: Ard Biesheuvel <ardb@kernel.org> To: Will Deacon <will@kernel.org> Cc: Mark Rutland <mark.rutland@arm.com>, linux-arm-kernel@lists.infradead.org, linux-efi@vger.kernel.org, keescook@chromium.org, Catalin Marinas <catalin.marinas@arm.com>, Marc Zyngier <maz@kernel.org> Subject: Re: [PATCH v5 0/7] arm64: efi: leave MMU and caches on at boot Date: Sat, 26 Nov 2022 15:17:15 +0100 [thread overview] Message-ID: <CAMj1kXH+B3vKkrkMg_NcZNsP4zSULuUgQ-xJwiq60UJs2wLP3A@mail.gmail.com> (raw) In-Reply-To: <CAMj1kXFdJHvT-XKaEC0tXpxM_RMfCic_sS-JkMkVYfKZCYkZuA@mail.gmail.com> On Sat, 26 Nov 2022 at 15:16, Ard Biesheuvel <ardb@kernel.org> wrote: > > On Tue, 15 Nov 2022 at 12:31, Will Deacon <will@kernel.org> wrote: > > > > On Tue, Nov 15, 2022 at 12:21:55PM +0100, Ard Biesheuvel wrote: > > > On Tue, 15 Nov 2022 at 12:17, Will Deacon <will@kernel.org> wrote: > > > > > > > > On Fri, Nov 11, 2022 at 05:36:19PM +0000, Mark Rutland wrote: > > > > > On Tue, Nov 08, 2022 at 07:21:57PM +0100, Ard Biesheuvel wrote: > > > > > > The purpose of this series is to remove any explicit cache maintenance > > > > > > for coherency during early boot that becomes unnecessary if we simply > > > > > > retain the cacheable 1:1 mapping of all of system RAM provided by EFI, > > > > > > and use it to populate the ID map page tables. After setting up this > > > > > > preliminary ID map, we disable the MMU, drop to EL1, reprogram the MAIR, > > > > > > TCR and SCTLR registers as before, and proceed as usual, avoiding the > > > > > > need for any manipulations of memory while the MMU and caches are off. > > > > > > > > > > > > The only properties of the firmware provided 1:1 map we rely on is that > > > > > > it does not require any explicit cache maintenance for coherency, and > > > > > > that it covers the entire memory footprint of the image, including the > > > > > > BSS and padding at the end - all else is under control of the kernel > > > > > > itself, as before. > > > > > > > > > > As a high-level thing, I'm still very much not keen on entering the kernel with > > > > > the MMU on. Given that we have to support booting with the MMU off for !EFI > > > > > boot (including kexec when EFI is in use), I think this makes it harder to > > > > > reason about the boot code overall (e.g. due to the conditional maintenance > > > > > added to head.S), and adds more scope for error, even if it simplifies the EFI > > > > > stub itself. > > > > > > > > As discussed offline, two things that would help the current series are: > > > > > > > > (1) Some performance numbers comparing MMU off vs MMU on boot > > > > > > Finally got around to measuring this - I lost access to my TX2 machine > for a couple of days during the past week, > > With the patch below applied to mainline, I measure ~6 ms spent > cleaning the entire image to the PoC (which is the bulk of it) and > subsequently populating the initial ID map and activating it. > > This drops to about 0.6 ms with my changes applied. This is unlikely > to ever matter in practice, perhaps, but I will note that booting a VM > in EFI mode using Tianocore/EDK2 from the point where KVM clears the > counter to the point where we start user space can be done (on the > same machine) in 500-700 ms so it is not entirely insignificant > either. > > I could try and measure it on bare metal as well, but I suppose that > launch times are even less relevant there so I didn't bother. diff --git a/arch/arm64/kernel/efi-entry.S b/arch/arm64/kernel/efi-entry.S index 61a87fa1c3055e26..27f59784a1c0be2c 100644 --- a/arch/arm64/kernel/efi-entry.S +++ b/arch/arm64/kernel/efi-entry.S @@ -22,6 +22,7 @@ SYM_CODE_START(efi_enter_kernel) ldr w2, =primary_entry_offset add x19, x0, x2 // relocated Image entrypoint mov x20, x1 // DTB address + mrs x27, cntvct_el0 /* * Clean the copied Image to the PoC, and ensure it is not shadowed by diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S index 2196aad7b55bcef0..068a7d111836382b 100644 --- a/arch/arm64/kernel/head.S +++ b/arch/arm64/kernel/head.S @@ -430,6 +430,8 @@ SYM_FUNC_START_LOCAL(__primary_switched) str_l x21, __fdt_pointer, x5 // Save FDT pointer + str_l x27, boot_args + 8, x5 + ldr_l x4, kimage_vaddr // Save the offset between sub x4, x4, x0 // the kernel virtual and str_l x4, kimage_voffset, x5 // physical mappings @@ -797,6 +799,10 @@ SYM_FUNC_START_LOCAL(__primary_switch) adrp x1, reserved_pg_dir adrp x2, init_idmap_pg_dir bl __enable_mmu + + mrs x0, cntvct_el0 + sub x27, x0, x27 + #ifdef CONFIG_RELOCATABLE adrp x23, KERNEL_START and x23, x23, MIN_KIMG_ALIGN - 1 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2022-11-26 14:17 UTC|newest] Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-11-08 18:21 [PATCH v5 0/7] arm64: efi: leave MMU and caches on at boot Ard Biesheuvel 2022-11-08 18:21 ` Ard Biesheuvel 2022-11-08 18:21 ` [PATCH v5 1/7] arm64: head: Move all finalise_el2 calls to after __enable_mmu Ard Biesheuvel 2022-11-08 18:21 ` Ard Biesheuvel 2022-11-08 18:21 ` [PATCH v5 2/7] arm64: kernel: move identity map out of .text mapping Ard Biesheuvel 2022-11-08 18:21 ` Ard Biesheuvel 2022-11-08 18:22 ` [PATCH v5 3/7] arm64: head: record the MMU state at primary entry Ard Biesheuvel 2022-11-08 18:22 ` Ard Biesheuvel 2022-11-08 18:22 ` [PATCH v5 4/7] arm64: head: avoid cache invalidation when entering with the MMU on Ard Biesheuvel 2022-11-08 18:22 ` Ard Biesheuvel 2022-11-08 18:22 ` [PATCH v5 5/7] arm64: head: Clean the ID map and the HYP text to the PoC if needed Ard Biesheuvel 2022-11-08 18:22 ` Ard Biesheuvel 2022-11-08 22:11 ` Ard Biesheuvel 2022-11-08 22:11 ` Ard Biesheuvel 2022-11-08 18:22 ` [PATCH v5 6/7] arm64: lds: reduce effective minimum image alignment to 64k Ard Biesheuvel 2022-11-08 18:22 ` Ard Biesheuvel 2022-11-08 18:22 ` [PATCH v5 7/7] efi: arm64: enter with MMU and caches enabled Ard Biesheuvel 2022-11-08 18:22 ` Ard Biesheuvel 2022-11-11 17:36 ` [PATCH v5 0/7] arm64: efi: leave MMU and caches on at boot Mark Rutland 2022-11-11 17:36 ` Mark Rutland 2022-11-15 11:17 ` Will Deacon 2022-11-15 11:17 ` Will Deacon 2022-11-15 11:21 ` Ard Biesheuvel 2022-11-15 11:21 ` Ard Biesheuvel 2022-11-15 11:31 ` Will Deacon 2022-11-15 11:31 ` Will Deacon 2022-11-26 14:16 ` Ard Biesheuvel 2022-11-26 14:16 ` Ard Biesheuvel 2022-11-26 14:17 ` Ard Biesheuvel [this message] 2022-11-26 14:17 ` Ard Biesheuvel
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CAMj1kXH+B3vKkrkMg_NcZNsP4zSULuUgQ-xJwiq60UJs2wLP3A@mail.gmail.com \ --to=ardb@kernel.org \ --cc=catalin.marinas@arm.com \ --cc=keescook@chromium.org \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-efi@vger.kernel.org \ --cc=mark.rutland@arm.com \ --cc=maz@kernel.org \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.