From: Joerg Roedel <joro@8bytes.org>
To: x86@kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
hpa@zytor.com, Dave Hansen <dave.hansen@linux.intel.com>,
Andy Lutomirski <luto@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Joerg Roedel <jroedel@suse.de>, Mike Rapoport <rppt@kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
Brijesh Singh <brijesh.singh@amd.com>,
linux-kernel@vger.kernel.org
Subject: [PATCH v3 3/4] x86/mm: Flush global TLB when switching to trampoline page-table
Date: Fri, 1 Oct 2021 17:48:16 +0200 [thread overview]
Message-ID: <20211001154817.29225-4-joro@8bytes.org> (raw)
In-Reply-To: <20211001154817.29225-1-joro@8bytes.org>
From: Joerg Roedel <jroedel@suse.de>
Move the switching code into a function so that it can be re-used and
add a global TLB flush. This makes sure that usage of memory which is
not mapped in the trampoline page-table is reliably caught.
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
arch/x86/include/asm/realmode.h | 1 +
arch/x86/kernel/reboot.c | 12 ++----------
arch/x86/realmode/init.c | 23 +++++++++++++++++++++++
3 files changed, 26 insertions(+), 10 deletions(-)
diff --git a/arch/x86/include/asm/realmode.h b/arch/x86/include/asm/realmode.h
index 5db5d083c873..331474b150f1 100644
--- a/arch/x86/include/asm/realmode.h
+++ b/arch/x86/include/asm/realmode.h
@@ -89,6 +89,7 @@ static inline void set_real_mode_mem(phys_addr_t mem)
}
void reserve_real_mode(void);
+void load_trampoline_pgtable(void);
#endif /* __ASSEMBLY__ */
diff --git a/arch/x86/kernel/reboot.c b/arch/x86/kernel/reboot.c
index 0a40df66a40d..fa700b46588e 100644
--- a/arch/x86/kernel/reboot.c
+++ b/arch/x86/kernel/reboot.c
@@ -113,17 +113,9 @@ void __noreturn machine_real_restart(unsigned int type)
spin_unlock(&rtc_lock);
/*
- * Switch back to the initial page table.
+ * Switch to the trampoline page table.
*/
-#ifdef CONFIG_X86_32
- load_cr3(initial_page_table);
-#else
- write_cr3(real_mode_header->trampoline_pgd);
-
- /* Exiting long mode will fail if CR4.PCIDE is set. */
- if (boot_cpu_has(X86_FEATURE_PCID))
- cr4_clear_bits(X86_CR4_PCIDE);
-#endif
+ load_trampoline_pgtable();
/* Jump to the identity-mapped low memory code */
#ifdef CONFIG_X86_32
diff --git a/arch/x86/realmode/init.c b/arch/x86/realmode/init.c
index 31b5856010cb..b9802b18f504 100644
--- a/arch/x86/realmode/init.c
+++ b/arch/x86/realmode/init.c
@@ -17,6 +17,29 @@ u32 *trampoline_cr4_features;
/* Hold the pgd entry used on booting additional CPUs */
pgd_t trampoline_pgd_entry;
+void load_trampoline_pgtable(void)
+{
+#ifdef CONFIG_X86_32
+ load_cr3(initial_page_table);
+#else
+ /* Exiting long mode will fail if CR4.PCIDE is set. */
+ if (boot_cpu_has(X86_FEATURE_PCID))
+ cr4_clear_bits(X86_CR4_PCIDE);
+
+ write_cr3(real_mode_header->trampoline_pgd);
+#endif
+
+ /*
+ * The CR3 write above will not flush global TLB entries.
+ * Stale, global entries from previous sets of page tables may
+ * still be present. Flush those stale entries.
+ *
+ * This ensures that memory accessed while running with
+ * trampoline_pgd is *actually* mapped into trampoline_pgd.
+ */
+ __flush_tlb_all();
+}
+
void __init reserve_real_mode(void)
{
phys_addr_t mem;
--
2.33.0
next prev parent reply other threads:[~2021-10-01 15:48 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-01 15:48 [PATCH v3 0/4] x86/mm: Fix some issues with using trampoline_pgd Joerg Roedel
2021-10-01 15:48 ` [PATCH v3 1/4] x86/realmode: Add comment for Global bit usage in trampline_pgd Joerg Roedel
2021-10-01 15:48 ` [PATCH v3 2/4] x86/mm/64: Flush global TLB on boot and AP bringup Joerg Roedel
2021-10-26 9:55 ` Borislav Petkov
2021-10-26 12:58 ` Borislav Petkov
2021-12-02 12:50 ` Joerg Roedel
2021-12-02 18:19 ` Borislav Petkov
2021-12-02 21:17 ` Joerg Roedel
2021-10-01 15:48 ` Joerg Roedel [this message]
2021-10-01 16:13 ` [PATCH v3 3/4] x86/mm: Flush global TLB when switching to trampoline page-table Dave Hansen
2021-10-01 17:57 ` Jörg Rödel
2021-10-27 9:58 ` Borislav Petkov
2021-12-02 12:58 ` Joerg Roedel
2021-12-02 18:26 ` Borislav Petkov
2021-10-01 15:48 ` [PATCH v3 4/4] x86/64/mm: Map all kernel memory into trampoline_pgd Joerg Roedel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211001154817.29225-4-joro@8bytes.org \
--to=joro@8bytes.org \
--cc=akpm@linux-foundation.org \
--cc=bp@alien8.de \
--cc=brijesh.singh@amd.com \
--cc=dave.hansen@linux.intel.com \
--cc=hpa@zytor.com \
--cc=jroedel@suse.de \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rppt@kernel.org \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).