From: Will Deacon <will@kernel.org>
To: Ard Biesheuvel <ardb@kernel.org>
Cc: linux-arm-kernel@lists.infradead.org,
linux-hardening@vger.kernel.org, Marc Zyngier <maz@kernel.org>,
Mark Rutland <mark.rutland@arm.com>,
Kees Cook <keescook@chromium.org>,
Catalin Marinas <catalin.marinas@arm.com>,
Mark Brown <broonie@kernel.org>,
Anshuman Khandual <anshuman.khandual@arm.com>
Subject: Re: [PATCH v4 17/26] arm64: head: populate kernel page tables with MMU and caches on
Date: Fri, 24 Jun 2022 13:56:31 +0100 [thread overview]
Message-ID: <20220624125631.GD18561@willie-the-truck> (raw)
In-Reply-To: <20220613144550.3760857-18-ardb@kernel.org>
On Mon, Jun 13, 2022 at 04:45:41PM +0200, Ard Biesheuvel wrote:
> Now that we can access the entire kernel image via the ID map, we can
> execute the page table population code with the MMU and caches enabled.
> The only thing we need to ensure is that translations via TTBR1 remain
> disabled while we are updating the page tables the second time around,
> in case KASLR wants them to be randomized.
>
> Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
> ---
> arch/arm64/kernel/head.S | 62 +++++---------------
> 1 file changed, 16 insertions(+), 46 deletions(-)
>
> diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
> index d704d0bd8ffc..583cbea865e1 100644
> --- a/arch/arm64/kernel/head.S
> +++ b/arch/arm64/kernel/head.S
> @@ -85,8 +85,6 @@
> * x21 primary_entry() .. start_kernel() FDT pointer passed at boot in x0
> * x22 create_idmap() .. start_kernel() ID map VA of the DT blob
> * x23 primary_entry() .. start_kernel() physical misalignment/KASLR offset
> - * x28 clear_page_tables() callee preserved temp register
> - * x19/x20 __primary_switch() callee preserved temp registers
> * x24 __primary_switch() .. relocate_kernel() current RELR displacement
> * x28 create_idmap() callee preserved temp register
> */
> @@ -96,9 +94,7 @@ SYM_CODE_START(primary_entry)
> adrp x23, __PHYS_OFFSET
> and x23, x23, MIN_KIMG_ALIGN - 1 // KASLR offset, defaults to 0
> bl set_cpu_boot_mode_flag
> - bl clear_page_tables
> bl create_idmap
> - bl create_kernel_mapping
>
> /*
> * The following calls CPU setup code, see arch/arm64/mm/proc.S for
> @@ -128,32 +124,14 @@ SYM_CODE_START_LOCAL(preserve_boot_args)
> SYM_CODE_END(preserve_boot_args)
>
> SYM_FUNC_START_LOCAL(clear_page_tables)
> - mov x28, lr
> -
> - /*
> - * Invalidate the init page tables to avoid potential dirty cache lines
> - * being evicted. Other page tables are allocated in rodata as part of
> - * the kernel image, and thus are clean to the PoC per the boot
> - * protocol.
> - */
> - adrp x0, init_pg_dir
> - adrp x1, init_pg_end
> - bl dcache_inval_poc
> -
> /*
> * Clear the init page tables.
> */
> adrp x0, init_pg_dir
> adrp x1, init_pg_end
> - sub x1, x1, x0
> -1: stp xzr, xzr, [x0], #16
> - stp xzr, xzr, [x0], #16
> - stp xzr, xzr, [x0], #16
> - stp xzr, xzr, [x0], #16
> - subs x1, x1, #64
> - b.ne 1b
> -
> - ret x28
> + sub x2, x1, x0
> + mov x1, xzr
> + b __pi_memset // tail call
> SYM_FUNC_END(clear_page_tables)
>
> /*
> @@ -399,16 +377,8 @@ SYM_FUNC_START_LOCAL(create_kernel_mapping)
>
> map_memory x0, x1, x5, x6, x7, x3, (VA_BITS - PGDIR_SHIFT), x10, x11, x12, x13, x14
>
> - /*
> - * Since the page tables have been populated with non-cacheable
> - * accesses (MMU disabled), invalidate those tables again to
> - * remove any speculatively loaded cache lines.
> - */
> - dmb sy
> -
> - adrp x0, init_pg_dir
> - adrp x1, init_pg_end
> - b dcache_inval_poc // tail call
> + dsb ishst // sync with page table walker
> + ret
> SYM_FUNC_END(create_kernel_mapping)
>
> /*
> @@ -863,14 +833,15 @@ SYM_FUNC_END(__relocate_kernel)
> #endif
>
> SYM_FUNC_START_LOCAL(__primary_switch)
> -#ifdef CONFIG_RANDOMIZE_BASE
> - mov x19, x0 // preserve new SCTLR_EL1 value
> - mrs x20, sctlr_el1 // preserve old SCTLR_EL1 value
> -#endif
> -
> - adrp x1, init_pg_dir
> + adrp x1, reserved_pg_dir
> adrp x2, init_idmap_pg_dir
> bl __enable_mmu
> +
> + bl clear_page_tables
> + bl create_kernel_mapping
> +
> + adrp x1, init_pg_dir
> + load_ttbr1 x1, x1, x2
> #ifdef CONFIG_RELOCATABLE
> #ifdef CONFIG_RELR
> mov x24, #0 // no RELR displacement yet
> @@ -886,9 +857,8 @@ SYM_FUNC_START_LOCAL(__primary_switch)
> * to take into account by discarding the current kernel mapping and
> * creating a new one.
> */
> - pre_disable_mmu_workaround
> - msr sctlr_el1, x20 // disable the MMU
> - isb
> + adrp x1, reserved_pg_dir // Disable translations via TTBR1
> + load_ttbr1 x1, x1, x2
I'd have thought we'd need some TLB maintenance here... is that not the
case?
Also, it might be a tiny bit easier to clear EPD1 instead of using the
reserved_pg_dir.
Will
next prev parent reply other threads:[~2022-06-24 12:56 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-13 14:45 [PATCH v4 00/26] arm64: refactor boot flow and add support for WXN Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 01/26] arm64: head: move kimage_vaddr variable into C file Ard Biesheuvel
2022-06-14 8:26 ` Anshuman Khandual
2022-06-13 14:45 ` [PATCH v4 02/26] arm64: mm: make vabits_actual a build time constant if possible Ard Biesheuvel
2022-06-14 8:25 ` Anshuman Khandual
2022-06-14 8:34 ` Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 03/26] arm64: head: move assignment of idmap_t0sz to C code Ard Biesheuvel
2022-06-14 9:22 ` Anshuman Khandual
2022-06-14 9:34 ` Ard Biesheuvel
2022-06-24 12:36 ` Will Deacon
2022-06-24 12:57 ` Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 04/26] arm64: head: drop idmap_ptrs_per_pgd Ard Biesheuvel
2022-06-15 4:07 ` Anshuman Khandual
2022-06-13 14:45 ` [PATCH v4 05/26] arm64: head: simplify page table mapping macros (slightly) Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 06/26] arm64: head: switch to map_memory macro for the extended ID map Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 07/26] arm64: head: split off idmap creation code Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 08/26] arm64: kernel: drop unnecessary PoC cache clean+invalidate Ard Biesheuvel
2022-06-15 4:32 ` Anshuman Khandual
2022-06-13 14:45 ` [PATCH v4 09/26] arm64: head: pass ID map root table address to __enable_mmu() Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 10/26] arm64: mm: provide idmap pointer to cpu_replace_ttbr1() Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 11/26] arm64: head: add helper function to remap regions in early page tables Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 12/26] arm64: head: cover entire kernel image in initial ID map Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 13/26] arm64: head: use relative references to the RELA and RELR tables Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 14/26] arm64: head: create a temporary FDT mapping in the initial ID map Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 15/26] arm64: idreg-override: use early FDT mapping in " Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 16/26] arm64: head: factor out TTBR1 assignment into a macro Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 17/26] arm64: head: populate kernel page tables with MMU and caches on Ard Biesheuvel
2022-06-24 12:56 ` Will Deacon [this message]
2022-06-24 13:07 ` Ard Biesheuvel
2022-06-24 13:29 ` Will Deacon
2022-06-24 14:07 ` Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 18/26] arm64: head: record CPU boot mode after enabling the MMU Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 19/26] arm64: kaslr: defer initialization to late initcall where permitted Ard Biesheuvel
2022-06-24 13:08 ` Will Deacon
2022-06-24 13:09 ` Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 20/26] arm64: head: avoid relocating the kernel twice for KASLR Ard Biesheuvel
2022-06-24 13:16 ` Will Deacon
2022-06-24 13:17 ` Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 21/26] arm64: setup: drop early FDT pointer helpers Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 22/26] arm64: mm: move ro_after_init section into the data segment Ard Biesheuvel
2022-06-13 17:00 ` Kees Cook
2022-06-13 17:16 ` Ard Biesheuvel
2022-06-13 23:38 ` Kees Cook
2022-06-16 11:31 ` Ard Biesheuvel
2022-06-16 16:18 ` Kees Cook
2022-06-16 16:31 ` Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 23/26] arm64: head: remap the kernel text/inittext region read-only Ard Biesheuvel
2022-06-13 16:57 ` Kees Cook
2022-06-13 14:45 ` [PATCH v4 24/26] mm: add arch hook to validate mmap() prot flags Ard Biesheuvel
2022-06-13 16:37 ` Kees Cook
2022-06-13 16:44 ` Ard Biesheuvel
2022-06-13 14:45 ` [PATCH v4 25/26] arm64: mm: add support for WXN memory translation attribute Ard Biesheuvel
2022-06-13 16:51 ` Kees Cook
2022-06-13 14:45 ` [PATCH v4 26/26] arm64: kernel: move ID map out of .text mapping Ard Biesheuvel
2022-06-13 16:52 ` Kees Cook
2022-06-24 13:19 ` [PATCH v4 00/26] arm64: refactor boot flow and add support for WXN Will Deacon
2022-06-24 14:40 ` Ard Biesheuvel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220624125631.GD18561@willie-the-truck \
--to=will@kernel.org \
--cc=anshuman.khandual@arm.com \
--cc=ardb@kernel.org \
--cc=broonie@kernel.org \
--cc=catalin.marinas@arm.com \
--cc=keescook@chromium.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-hardening@vger.kernel.org \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).