From: Rick Edgecombe <rick.p.edgecombe@intel.com>
To: x86@kernel.org, "H . Peter Anvin" <hpa@zytor.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>,
linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
linux-mm@kvack.org, linux-arch@vger.kernel.org,
linux-api@vger.kernel.org, Arnd Bergmann <arnd@arndb.de>,
Andy Lutomirski <luto@kernel.org>,
Balbir Singh <bsingharora@gmail.com>,
Borislav Petkov <bp@alien8.de>,
Cyrill Gorcunov <gorcunov@gmail.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Eugene Syromiatnikov <esyr@redhat.com>,
Florian Weimer <fweimer@redhat.com>,
"H . J . Lu" <hjl.tools@gmail.com>, Jann Horn <jannh@google.com>,
Jonathan Corbet <corbet@lwn.net>,
Kees Cook <keescook@chromium.org>,
Mike Kravetz <mike.kravetz@oracle.com>,
Nadav Amit <nadav.amit@gmail.com>,
Oleg Nesterov <oleg@redhat.com>, Pavel Machek <pavel@ucw.cz>,
Peter Zijlstra <peterz@infradead.org>,
Randy Dunlap <rdunlap@infradead.org>,
Weijiang Yang <weijiang.yang@intel.com>,
"Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
John Allen <john.allen@amd.com>,
kcc@google.com, eranian@google.com, rppt@kernel.org,
jamorris@linux.microsoft.com, dethoma@microsoft.com,
akpm@linux-foundation.org, Andrew.Cooper3@citrix.com,
christina.schimpe@intel.com, david@redhat.com,
debug@rivosinc.com
Cc: rick.p.edgecombe@intel.com
Subject: [PATCH v7 20/41] x86/mm: Teach pte_mkwrite() about stack memory
Date: Mon, 27 Feb 2023 14:29:36 -0800 [thread overview]
Message-ID: <20230227222957.24501-21-rick.p.edgecombe@intel.com> (raw)
In-Reply-To: <20230227222957.24501-1-rick.p.edgecombe@intel.com>
If a VMA has the VM_SHADOW_STACK flag, it is shadow stack memory. So
when it is made writable with pte_mkwrite(), it should create shadow
stack memory, not conventionally writable memory. Now that pte_mkwrite()
takes a VMA, and places where shadow stack memory might be created pass
one, pte_mkwrite() can know when it should do this.
So make pte_mkwrite() create shadow stack memory when the VMA has the
VM_SHADOW_STACK flag. Do the same thing for pmd_mkwrite().
This requires referencing VM_SHADOW_STACK in these functions, which are
currently defined in pgtable.h, however mm.h (where VM_SHADOW_STACK is
located) can't be pulled in without causing problems for files that
reference pgtable.h. So also move pte/pmd_mkwrite() into pgtable.c, where
they can safely reference VM_SHADOW_STACK.
Tested-by: Pengfei Xu <pengfei.xu@intel.com>
Tested-by: John Allen <john.allen@amd.com>
Tested-by: Kees Cook <keescook@chromium.org>
Acked-by: Mike Rapoport (IBM) <rppt@kernel.org>
Reviewed-by: Kees Cook <keescook@chromium.org>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
v6:
- New patch
---
arch/x86/include/asm/pgtable.h | 20 ++------------------
arch/x86/mm/pgtable.c | 26 ++++++++++++++++++++++++++
2 files changed, 28 insertions(+), 18 deletions(-)
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 05dfdbdf96b4..d81e7ec27507 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -456,15 +456,7 @@ static inline pte_t pte_mkwrite_kernel(pte_t pte)
struct vm_area_struct;
-static inline pte_t pte_mkwrite(pte_t pte, struct vm_area_struct *vma)
-{
- pte = pte_mkwrite_kernel(pte);
-
- if (pte_dirty(pte))
- pte = pte_clear_saveddirty(pte);
-
- return pte;
-}
+pte_t pte_mkwrite(pte_t pte, struct vm_area_struct *vma);
static inline pte_t pte_mkhuge(pte_t pte)
{
@@ -601,15 +593,7 @@ static inline pmd_t pmd_mkyoung(pmd_t pmd)
return pmd_set_flags(pmd, _PAGE_ACCESSED);
}
-static inline pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma)
-{
- pmd = pmd_set_flags(pmd, _PAGE_RW);
-
- if (pmd_dirty(pmd))
- pmd = pmd_clear_saveddirty(pmd);
-
- return pmd;
-}
+pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);
static inline pud_t pud_set_flags(pud_t pud, pudval_t set)
{
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index e4f499eb0f29..98856bcc8102 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -880,3 +880,29 @@ int pmd_free_pte_page(pmd_t *pmd, unsigned long addr)
#endif /* CONFIG_X86_64 */
#endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */
+
+pte_t pte_mkwrite(pte_t pte, struct vm_area_struct *vma)
+{
+ if (vma->vm_flags & VM_SHADOW_STACK)
+ return pte_mkwrite_shstk(pte);
+
+ pte = pte_mkwrite_kernel(pte);
+
+ if (pte_dirty(pte))
+ pte = pte_clear_saveddirty(pte);
+
+ return pte;
+}
+
+pmd_t pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma)
+{
+ if (vma->vm_flags & VM_SHADOW_STACK)
+ return pmd_mkwrite_shstk(pmd);
+
+ pmd = pmd_set_flags(pmd, _PAGE_RW);
+
+ if (pmd_dirty(pmd))
+ pmd = pmd_clear_saveddirty(pmd);
+
+ return pmd;
+}
--
2.17.1
next prev parent reply other threads:[~2023-02-27 22:33 UTC|newest]
Thread overview: 159+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-27 22:29 [PATCH v7 00/41] Shadow stacks for userspace Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 01/41] Documentation/x86: Add CET shadow stack description Rick Edgecombe
2023-03-01 14:21 ` Szabolcs Nagy
2023-03-01 14:38 ` Szabolcs Nagy
2023-03-01 18:07 ` Edgecombe, Rick P
2023-03-01 18:32 ` Edgecombe, Rick P
2023-03-02 16:34 ` szabolcs.nagy
2023-03-03 22:35 ` Edgecombe, Rick P
2023-03-06 16:20 ` szabolcs.nagy
2023-03-06 16:31 ` Florian Weimer
2023-03-06 18:08 ` Edgecombe, Rick P
2023-03-07 13:03 ` szabolcs.nagy
2023-03-07 14:00 ` Florian Weimer
2023-03-07 16:14 ` Szabolcs Nagy
2023-03-06 18:05 ` Edgecombe, Rick P
2023-03-06 20:31 ` Liang, Kan
2023-03-02 16:14 ` szabolcs.nagy
2023-03-02 21:17 ` Edgecombe, Rick P
2023-03-03 16:30 ` szabolcs.nagy
2023-03-03 16:57 ` H.J. Lu
2023-03-03 17:39 ` szabolcs.nagy
2023-03-03 17:50 ` H.J. Lu
2023-03-03 17:41 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 02/41] x86/shstk: Add Kconfig option for shadow stack Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 03/41] x86/cpufeatures: Add CPU feature flags for shadow stacks Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 04/41] x86/cpufeatures: Enable CET CR4 bit for shadow stack Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 05/41] x86/fpu/xstate: Introduce CET MSR and XSAVES supervisor states Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 06/41] x86/fpu: Add helper for modifying xstate Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 07/41] x86: Move control protection handler to separate file Rick Edgecombe
2023-03-01 15:38 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 08/41] x86/shstk: Add user control-protection fault handler Rick Edgecombe
2023-03-01 18:06 ` Borislav Petkov
2023-03-01 18:14 ` Edgecombe, Rick P
2023-03-01 18:37 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 09/41] x86/mm: Remove _PAGE_DIRTY from kernel RO pages Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 10/41] x86/mm: Move pmd_write(), pud_write() up in the file Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 11/41] mm: Introduce pte_mkwrite_kernel() Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 12/41] s390/mm: Introduce pmd_mkwrite_kernel() Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 13/41] mm: Make pte_mkwrite() take a VMA Rick Edgecombe
2023-03-01 7:03 ` Christophe Leroy
2023-03-01 8:16 ` David Hildenbrand
2023-03-02 12:19 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 14/41] x86/mm: Introduce _PAGE_SAVED_DIRTY Rick Edgecombe
2023-03-02 12:48 ` Borislav Petkov
2023-03-02 17:01 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 15/41] x86/mm: Update ptep/pmdp_set_wrprotect() for _PAGE_SAVED_DIRTY Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 16/41] x86/mm: Start actually marking _PAGE_SAVED_DIRTY Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 17/41] mm: Move VM_UFFD_MINOR_BIT from 37 to 38 Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 18/41] mm: Introduce VM_SHADOW_STACK for shadow stack memory Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 19/41] x86/mm: Check shadow stack page fault errors Rick Edgecombe
2023-03-03 14:00 ` Borislav Petkov
2023-03-03 14:39 ` Dave Hansen
2023-02-27 22:29 ` Rick Edgecombe [this message]
2023-03-03 15:37 ` [PATCH v7 20/41] x86/mm: Teach pte_mkwrite() about stack memory Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 21/41] mm: Add guard pages around a shadow stack Rick Edgecombe
2023-03-06 8:08 ` Borislav Petkov
2023-03-07 1:29 ` Edgecombe, Rick P
2023-03-07 10:32 ` Borislav Petkov
2023-03-07 10:44 ` David Hildenbrand
2023-03-08 22:48 ` Edgecombe, Rick P
2023-03-17 17:09 ` Deepak Gupta
2023-02-27 22:29 ` [PATCH v7 22/41] mm/mmap: Add shadow stack pages to memory accounting Rick Edgecombe
2023-03-06 13:01 ` Borislav Petkov
2023-03-06 18:11 ` Edgecombe, Rick P
2023-03-06 18:16 ` Borislav Petkov
2023-03-07 10:42 ` David Hildenbrand
2023-03-17 17:12 ` Deepak Gupta
2023-03-17 17:16 ` Dave Hansen
2023-03-17 17:28 ` Deepak Gupta
2023-03-17 17:42 ` Edgecombe, Rick P
2023-03-17 19:26 ` Deepak Gupta
2023-02-27 22:29 ` [PATCH v7 23/41] mm: Re-introduce vm_flags to do_mmap() Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 24/41] mm: Don't allow write GUPs to shadow stack memory Rick Edgecombe
2023-03-06 13:10 ` Borislav Petkov
2023-03-06 18:15 ` Andy Lutomirski
2023-03-06 18:33 ` Edgecombe, Rick P
2023-03-06 18:57 ` Andy Lutomirski
2023-03-07 1:47 ` Edgecombe, Rick P
2023-03-17 17:05 ` Deepak Gupta
2023-02-27 22:29 ` [PATCH v7 25/41] x86/mm: Introduce MAP_ABOVE4G Rick Edgecombe
2023-03-06 18:09 ` Borislav Petkov
2023-03-07 1:10 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 26/41] mm: Warn on shadow stack memory in wrong vma Rick Edgecombe
2023-03-08 8:53 ` Borislav Petkov
2023-03-08 23:36 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 27/41] x86/mm: Warn if create Write=0,Dirty=1 with raw prot Rick Edgecombe
2023-02-27 22:54 ` Kees Cook
2023-03-08 9:23 ` Borislav Petkov
2023-03-08 23:35 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 28/41] x86: Introduce userspace API for shadow stack Rick Edgecombe
2023-03-08 10:27 ` Borislav Petkov
2023-03-08 23:32 ` Edgecombe, Rick P
2023-03-09 12:57 ` Borislav Petkov
2023-03-09 16:56 ` Edgecombe, Rick P
2023-03-09 23:51 ` Borislav Petkov
2023-03-10 1:13 ` Edgecombe, Rick P
2023-03-10 2:03 ` H.J. Lu
2023-03-10 20:00 ` H.J. Lu
2023-03-10 20:27 ` Edgecombe, Rick P
2023-03-10 20:43 ` H.J. Lu
2023-03-10 21:01 ` Edgecombe, Rick P
2023-03-10 11:40 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 29/41] x86/shstk: Add user-mode shadow stack support Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 30/41] x86/shstk: Handle thread shadow stack Rick Edgecombe
2023-03-02 17:34 ` Szabolcs Nagy
2023-03-02 21:48 ` Edgecombe, Rick P
2023-03-08 15:26 ` Borislav Petkov
2023-03-08 20:03 ` Edgecombe, Rick P
2023-03-09 14:12 ` Borislav Petkov
2023-03-09 16:59 ` Edgecombe, Rick P
2023-03-09 17:04 ` Borislav Petkov
2023-03-09 20:29 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 31/41] x86/shstk: Introduce routines modifying shstk Rick Edgecombe
2023-03-09 16:48 ` Borislav Petkov
2023-03-09 17:03 ` Edgecombe, Rick P
2023-03-09 17:22 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 32/41] x86/shstk: Handle signals for shadow stack Rick Edgecombe
2023-03-09 17:02 ` Borislav Petkov
2023-03-09 17:16 ` Edgecombe, Rick P
2023-03-09 23:35 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 33/41] x86/shstk: Introduce map_shadow_stack syscall Rick Edgecombe
2023-03-02 17:22 ` Szabolcs Nagy
2023-03-02 21:21 ` Edgecombe, Rick P
2023-03-09 18:55 ` Deepak Gupta
2023-03-09 19:39 ` Edgecombe, Rick P
2023-03-09 21:08 ` Deepak Gupta
2023-03-10 0:14 ` Edgecombe, Rick P
2023-03-10 21:00 ` Deepak Gupta
2023-03-10 21:43 ` Edgecombe, Rick P
2023-03-16 20:07 ` Deepak Gupta
2023-03-14 7:19 ` Mike Rapoport
2023-03-16 19:30 ` Deepak Gupta
2023-03-20 11:35 ` Szabolcs Nagy
2023-03-10 16:11 ` Borislav Petkov
2023-03-10 17:12 ` Edgecombe, Rick P
2023-03-10 20:05 ` Borislav Petkov
2023-03-10 20:19 ` Edgecombe, Rick P
2023-03-10 20:26 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 34/41] x86/shstk: Support WRSS for userspace Rick Edgecombe
2023-03-10 16:44 ` Borislav Petkov
2023-03-10 17:16 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 35/41] x86: Expose thread features in /proc/$PID/status Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 36/41] x86/shstk: Wire in shadow stack interface Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 37/41] selftests/x86: Add shadow stack test Rick Edgecombe
2023-02-27 22:29 ` [PATCH v7 38/41] x86/fpu: Add helper for initing features Rick Edgecombe
2023-03-11 12:54 ` Borislav Petkov
2023-03-13 2:45 ` Edgecombe, Rick P
2023-03-13 11:03 ` Borislav Petkov
2023-03-13 16:10 ` Edgecombe, Rick P
2023-03-13 17:10 ` Borislav Petkov
2023-03-13 23:31 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 39/41] x86: Add PTRACE interface for shadow stack Rick Edgecombe
2023-03-11 15:06 ` Borislav Petkov
2023-03-13 2:53 ` Edgecombe, Rick P
2023-02-27 22:29 ` [PATCH v7 40/41] x86/shstk: Add ARCH_SHSTK_UNLOCK Rick Edgecombe
2023-03-11 15:11 ` Borislav Petkov
2023-03-13 3:04 ` Edgecombe, Rick P
2023-03-13 11:05 ` Borislav Petkov
2023-02-27 22:29 ` [PATCH v7 41/41] x86/shstk: Add ARCH_SHSTK_STATUS Rick Edgecombe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230227222957.24501-21-rick.p.edgecombe@intel.com \
--to=rick.p.edgecombe@intel.com \
--cc=Andrew.Cooper3@citrix.com \
--cc=akpm@linux-foundation.org \
--cc=arnd@arndb.de \
--cc=bp@alien8.de \
--cc=bsingharora@gmail.com \
--cc=christina.schimpe@intel.com \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=david@redhat.com \
--cc=debug@rivosinc.com \
--cc=dethoma@microsoft.com \
--cc=eranian@google.com \
--cc=esyr@redhat.com \
--cc=fweimer@redhat.com \
--cc=gorcunov@gmail.com \
--cc=hjl.tools@gmail.com \
--cc=hpa@zytor.com \
--cc=jamorris@linux.microsoft.com \
--cc=jannh@google.com \
--cc=john.allen@amd.com \
--cc=kcc@google.com \
--cc=keescook@chromium.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
--cc=mike.kravetz@oracle.com \
--cc=mingo@redhat.com \
--cc=nadav.amit@gmail.com \
--cc=oleg@redhat.com \
--cc=pavel@ucw.cz \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rppt@kernel.org \
--cc=tglx@linutronix.de \
--cc=weijiang.yang@intel.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).