From: Andy Lutomirski <luto@kernel.org>
To: x86@kernel.org
Cc: linux-kernel@vger.kernel.org, Borislav Petkov <bp@alien8.de>,
Brian Gerst <brgerst@gmail.com>,
David Laight <David.Laight@aculab.com>,
Kees Cook <keescook@chromium.org>,
Peter Zijlstra <peterz@infradead.org>,
Andy Lutomirski <luto@kernel.org>
Subject: [PATCH PTI v3 02/10] x86/pti: Vastly simplify pgd synchronization
Date: Tue, 12 Dec 2017 07:56:37 -0800 [thread overview]
Message-ID: <0fb19e5aa26f51c5c7a256de16c09c838f17b47c.1513035461.git.luto@kernel.org> (raw)
In-Reply-To: <24c898b4f44fdf8c22d93703850fb384ef87cfdc.1513035461.git.luto@kernel.org>
In-Reply-To: <cover.1513035461.git.luto@kernel.org>
Back when we would dynamically add mappings to the usermode tables,
we needed to preallocate all the high top-level entries in the
usermode tables. We don't need this in recent versions of PTI, so
get rid of preallocation.
With preallocation gone, the comments in pti_set_user_pgd() make
even less sense. Rearrange the function to make it entirely obvious
what it does and does not do. FWIW, I haven't even tried to wrap my
head around the old logic, since it seemed to be somewhere between
incomprehensible and wrong.
I admit that a bit of the earlier complexity was based on my
suggestions. Mea culpa.
Signed-off-by: Andy Lutomirski <luto@kernel.org>
---
arch/x86/include/asm/pgtable_64.h | 74 +++++++++++++++------------------------
arch/x86/mm/pti.c | 52 ++++++---------------------
2 files changed, 39 insertions(+), 87 deletions(-)
diff --git a/arch/x86/include/asm/pgtable_64.h b/arch/x86/include/asm/pgtable_64.h
index f5adf92091c6..be8d086de927 100644
--- a/arch/x86/include/asm/pgtable_64.h
+++ b/arch/x86/include/asm/pgtable_64.h
@@ -195,14 +195,6 @@ static inline bool pgdp_maps_userspace(void *__ptr)
}
/*
- * Does this PGD allow access from userspace?
- */
-static inline bool pgd_userspace_access(pgd_t pgd)
-{
- return pgd.pgd & _PAGE_USER;
-}
-
-/*
* Take a PGD location (pgdp) and a pgd value that needs to be set there.
* Populates the user and returns the resulting PGD that must be set in
* the kernel copy of the page tables.
@@ -213,50 +205,42 @@ static inline pgd_t pti_set_user_pgd(pgd_t *pgdp, pgd_t pgd)
if (!static_cpu_has_bug(X86_BUG_CPU_SECURE_MODE_PTI))
return pgd;
- if (pgd_userspace_access(pgd)) {
- if (pgdp_maps_userspace(pgdp)) {
- /*
- * The user page tables get the full PGD,
- * accessible from userspace:
- */
- kernel_to_user_pgdp(pgdp)->pgd = pgd.pgd;
- /*
- * For the copy of the pgd that the kernel uses,
- * make it unusable to userspace. This ensures on
- * in case that a return to userspace with the
- * kernel CR3 value, userspace will crash instead
- * of running.
- *
- * Note: NX might be not available or disabled.
- */
- if (__supported_pte_mask & _PAGE_NX)
- pgd.pgd |= _PAGE_NX;
- }
- } else if (pgd_userspace_access(*pgdp)) {
+ if (pgdp_maps_userspace(pgdp)) {
/*
- * We are clearing a _PAGE_USER PGD for which we presumably
- * populated the user PGD. We must now clear the user PGD
- * entry.
+ * The user page tables get the full PGD,
+ * accessible from userspace:
*/
- if (pgdp_maps_userspace(pgdp)) {
- kernel_to_user_pgdp(pgdp)->pgd = pgd.pgd;
- } else {
- /*
- * Attempted to clear a _PAGE_USER PGD which is in
- * the kernel portion of the address space. PGDs
- * are pre-populated and we never clear them.
- */
- WARN_ON_ONCE(1);
- }
+ kernel_to_user_pgdp(pgdp)->pgd = pgd.pgd;
+
+ /*
+ * If this is normal user memory, make it NX in the kernel
+ * pagetables so that, if we somehow screw up and return to
+ * usermode with the kernel CR3 loaded, we'll get a page
+ * fault instead of allowing user code to execute with
+ * the wrong CR3.
+ *
+ * As exceptions, we don't set NX if:
+ * - this is EFI or similar, the kernel may execute from it
+ * - we don't have NX support
+ * - we're clearing the PGD (i.e. pgd.pgd == 0).
+ */
+ if ((pgd.pgd & _PAGE_USER) && (__supported_pte_mask & _PAGE_NX))
+ pgd.pgd |= _PAGE_NX;
} else {
/*
- * _PAGE_USER was not set in either the PGD being set or
- * cleared. All kernel PGDs should be pre-populated so
- * this should never happen after boot.
+ * Changes to the high (kernel) portion of the kernelmode
+ * page tables are not automatically propagated to the
+ * usermode tables.
+ *
+ * Users should keep in mind that, unlike the kernelmode
+ * tables, there is no vmalloc_fault equivalent for the
+ * usermode tables. Top-level entries added to init_mm's
+ * usermode pgd after boot will not be automatically
+ * propagated to other mms.
*/
- WARN_ON_ONCE(system_state == SYSTEM_RUNNING);
}
#endif
+
/* return the copy of the PGD we want the kernel to use: */
return pgd;
}
diff --git a/arch/x86/mm/pti.c b/arch/x86/mm/pti.c
index bd5d042adb3c..f48645d2f3fd 100644
--- a/arch/x86/mm/pti.c
+++ b/arch/x86/mm/pti.c
@@ -83,8 +83,16 @@ static pmd_t *pti_user_pagetable_walk_pmd(unsigned long address)
}
if (pgd_none(*pgd)) {
- WARN_ONCE(1, "All user pgds should have been populated\n");
- return NULL;
+ unsigned long new_p4d_page = __get_free_page(gfp);
+ if (!new_p4d_page)
+ return NULL;
+
+ if (pgd_none(*pgd)) {
+ set_pgd(pgd, __pgd(_KERNPG_TABLE | __pa(new_p4d_page)));
+ new_p4d_page = 0;
+ }
+ if (new_p4d_page)
+ free_page(new_p4d_page);
}
BUILD_BUG_ON(pgd_large(*pgd) != 0);
@@ -193,45 +201,6 @@ static void __init pti_clone_entry_text(void)
}
/*
- * Ensure that the top level of the user page tables are entirely
- * populated. This ensures that all processes that get forked have the
- * same entries. This way, we do not have to ever go set up new entries in
- * older processes.
- *
- * Note: we never free these, so there are no updates to them after this.
- */
-static void __init pti_init_all_pgds(void)
-{
- pgd_t *pgd;
- int i;
-
- pgd = kernel_to_user_pgdp(pgd_offset_k(0UL));
- for (i = PTRS_PER_PGD / 2; i < PTRS_PER_PGD; i++) {
- /*
- * Each PGD entry moves up PGDIR_SIZE bytes through the
- * address space, so get the first virtual address mapped
- * by PGD #i:
- */
- unsigned long addr = i * PGDIR_SIZE;
-#if CONFIG_PGTABLE_LEVELS > 4
- p4d_t *p4d = p4d_alloc_one(&init_mm, addr);
- if (!p4d) {
- WARN_ON(1);
- break;
- }
- set_pgd(pgd + i, __pgd(_KERNPG_TABLE | __pa(p4d)));
-#else /* CONFIG_PGTABLE_LEVELS <= 4 */
- pud_t *pud = pud_alloc_one(&init_mm, addr);
- if (!pud) {
- WARN_ON(1);
- break;
- }
- set_pgd(pgd + i, __pgd(_KERNPG_TABLE | __pa(pud)));
-#endif /* CONFIG_PGTABLE_LEVELS */
- }
-}
-
-/*
* Initialize kernel page table isolation
*/
void __init pti_init(void)
@@ -241,7 +210,6 @@ void __init pti_init(void)
pr_info("enabled\n");
- pti_init_all_pgds();
pti_clone_user_shared();
pti_clone_entry_text();
}
--
2.13.6
next prev parent reply other threads:[~2017-12-12 15:56 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-12-11 23:40 [PATCH PTI v3 00/10] Clean up pgd handling and fix VSYSCALL and LDT Andy Lutomirski
2017-12-12 10:09 ` Ingo Molnar
2017-12-12 15:58 ` Andy Lutomirski
2017-12-12 16:13 ` Borislav Petkov
2017-12-12 16:14 ` Juergen Gross
2017-12-12 16:20 ` Borislav Petkov
2017-12-12 15:56 ` [PATCH PTI v3 01/10] x86/espfix/64: Fix espfix double-fault handling on 5-level systems Andy Lutomirski
2017-12-12 17:18 ` Kirill A. Shutemov
2017-12-15 18:34 ` [tip:x86/urgent] " tip-bot for Andy Lutomirski
2017-12-12 15:56 ` Andy Lutomirski [this message]
2017-12-12 15:56 ` [PATCH PTI v3 03/10] x86/pti/64: Fix ESPFIX64 user mapping Andy Lutomirski
2017-12-13 13:12 ` Kirill A. Shutemov
2017-12-13 17:01 ` Andy Lutomirski
2017-12-14 14:10 ` Kirill A. Shutemov
2017-12-14 16:18 ` Andy Lutomirski
2017-12-12 15:56 ` [PATCH PTI v3 04/10] Revert "x86/mm/pti: Disable native VSYSCALL" Andy Lutomirski
2017-12-12 15:56 ` [PATCH PTI v3 05/10] x86/vsyscall/64: Explicitly set _PAGE_USER in the pagetable hierarchy Andy Lutomirski
2017-12-12 15:56 ` [PATCH PTI v3 06/10] x86/vsyscall/64: Warn and fail vsyscall emulation in NATIVE mode Andy Lutomirski
2017-12-12 15:56 ` [PATCH PTI v3 07/10] x86/pti: Map the vsyscall page if needed Andy Lutomirski
2017-12-12 15:56 ` [PATCH PTI v3 08/10] x86/mm/64: Improve the memory map documentation Andy Lutomirski
2017-12-12 15:56 ` [PATCH PTI v3 09/10] x86/mm/64: Make a full PGD-entry size hole in the memory map Andy Lutomirski
2017-12-13 13:17 ` Kirill A. Shutemov
2017-12-13 17:04 ` Andy Lutomirski
2017-12-12 15:56 ` [PATCH PTI v3 10/10] x86/pti: Put the LDT in its own PGD if PTI is on Andy Lutomirski
2017-12-15 22:54 ` Thomas Gleixner
2017-12-16 0:39 ` Thomas Gleixner
2017-12-16 6:41 ` Andy Lutomirski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=0fb19e5aa26f51c5c7a256de16c09c838f17b47c.1513035461.git.luto@kernel.org \
--to=luto@kernel.org \
--cc=David.Laight@aculab.com \
--cc=bp@alien8.de \
--cc=brgerst@gmail.com \
--cc=keescook@chromium.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).