All of
 help / color / mirror / Atom feed
From: Joerg Roedel <>
Cc: Thomas Gleixner <>,
	Ingo Molnar <>, Borislav Petkov <>,, Dave Hansen <>,
	Andy Lutomirski <>,
	Peter Zijlstra <>,
	Mike Rapoport <>,
	Andrew Morton <>,
	Brijesh Singh <>,, Joerg Roedel <>,
	Joerg Roedel <>,
Subject: [PATCH v4 4/4] x86/64/mm: Map all kernel memory into trampoline_pgd
Date: Thu,  2 Dec 2021 16:32:26 +0100	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

From: Joerg Roedel <>

The trampoline_pgd only maps the 0xfffffff000000000-0xffffffffffffffff
range of kernel memory (with 4-level paging). This range contains the
kernels text+data+bss mappings and the module mapping space, but not the
direct mapping and the vmalloc area.

This is enough to get an application processors out of real-mode, but
for code that switches back to real-mode the trampoline_pgd is missing
important parts of the address space. For example, consider this code
from arch/x86/kernel/reboot.c, function machine_real_restart() for a
64-bit kernel:

	#ifdef CONFIG_X86_32

		/* Exiting long mode will fail if CR4.PCIDE is set. */
		if (boot_cpu_has(X86_FEATURE_PCID))

		/* Jump to the identity-mapped low memory code */
	#ifdef CONFIG_X86_32
		asm volatile("jmpl *%0" : :
			     "rm" (real_mode_header->machine_real_restart_asm),
			     "a" (type));
		asm volatile("ljmpl *%0" : :
			     "m" (real_mode_header->machine_real_restart_asm),
			     "D" (type));

The code switches to the trampoline_pgd, which unmaps the direct mapping
and also the kernel stack. The call to cr4_clear_bits() will find no
stack and crash the machine. The real_mode_header pointer below points
into the direct mapping, and dereferencing it also causes a crash.

The reason this does not crash always is only that kernel mappings are
global and the CR3 switch does not flush those mappings. But if theses
mappings are not in the TLB already, the above code will crash before it
can jump to the real-mode stub.

Extend the trampoline_pgd to contain all kernel mappings to prevent
these crashes and to make code which runs on this page-table more

Signed-off-by: Joerg Roedel <>
 arch/x86/realmode/init.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 6d98609387ba..c5e29db02a46 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -98,6 +98,7 @@ static void __init setup_real_mode(void)
 #ifdef CONFIG_X86_64
 	u64 *trampoline_pgd;
 	u64 efer;
+	int i;
 	base = (unsigned char *)real_mode_header;
@@ -154,8 +155,17 @@ static void __init setup_real_mode(void)
 	trampoline_header->flags = 0;
 	trampoline_pgd = (u64 *) __va(real_mode_header->trampoline_pgd);
+	/* Map the real mode stub as virtual == physical */
 	trampoline_pgd[0] = trampoline_pgd_entry.pgd;
-	trampoline_pgd[511] = init_top_pgt[511].pgd;
+	/*
+	 * Include the entirety of the kernel mapping into the trampoline
+	 * PGD.  This way, all mappings present in the normal kernel page
+	 * tables are usable while running on trampoline_pgd.
+	 */
+	for (i = pgd_index(__PAGE_OFFSET); i < PTRS_PER_PGD; i++)
+		trampoline_pgd[i] = init_top_pgt[i].pgd;

  parent reply	other threads:[~2021-12-02 15:33 UTC|newest]

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-12-02 15:32 [PATCH v4 0/4] x86/mm: Fix some issues with using trampoline_pgd Joerg Roedel
2021-12-02 15:32 ` [PATCH v4 1/4] x86/realmode: Add comment for Global bit usage in trampline_pgd Joerg Roedel
2021-12-06 21:57   ` [tip: x86/mm] x86/realmode: Add comment for Global bit usage in trampoline_pgd tip-bot2 for Joerg Roedel
2021-12-02 15:32 ` [PATCH v4 2/4] x86/mm/64: Flush global TLB on boot and AP bringup Joerg Roedel
2021-12-06 21:57   ` [tip: x86/mm] " tip-bot2 for Joerg Roedel
2021-12-02 15:32 ` [PATCH v4 3/4] x86/mm: Flush global TLB when switching to trampoline page-table Joerg Roedel
2021-12-06 21:57   ` [tip: x86/mm] " tip-bot2 for Joerg Roedel
2021-12-02 15:32 ` Joerg Roedel [this message]
2021-12-03 10:04   ` [tip: x86/urgent] x86/64/mm: Map all kernel memory into trampoline_pgd tip-bot2 for Joerg Roedel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \
    --subject='Re: [PATCH v4 4/4] x86/64/mm: Map all kernel memory into trampoline_pgd' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.