Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
From: Yu-cheng Yu <yu-cheng.yu@intel.com>
To: x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org,
	linux-mm@kvack.org, linux-arch@vger.kernel.org,
	linux-api@vger.kernel.org, Arnd Bergmann <arnd@arndb.de>,
	Andy Lutomirski <luto@amacapital.net>,
	Balbir Singh <bsingharora@gmail.com>,
	Borislav Petkov <bp@alien8.de>,
	Cyrill Gorcunov <gorcunov@gmail.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Eugene Syromiatnikov <esyr@redhat.com>,
	Florian Weimer <fweimer@redhat.com>,
	"H.J. Lu" <hjl.tools@gmail.com>, Jann Horn <jannh@google.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Kees Cook <keescook@chromium.org>,
	Mike Kravetz <mike.kravetz@oracle.com>,
	Nadav Amit <nadav.amit@gmail.com>,
	Oleg Nesterov <oleg@redhat.com>, Pavel Machek <pavel@ucw.cz>,
	Peter Zijlstra <peterz@infradead.org>,
	Randy Dunlap <rdunlap@infradead.org>,
	"Ravi V. Shankar" <ravi.v.shankar@intel.com>,
	Vedvyas Shanbhogue <vedvyas.shanbhogue@intel.com>,
	Dave Martin <Dave.Martin@arm.com>
Cc: Yu-cheng Yu <yu-cheng.yu@intel.com>
Subject: [PATCH v8 13/27] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW
Date: Tue, 13 Aug 2019 13:52:11 -0700
Message-ID: <20190813205225.12032-14-yu-cheng.yu@intel.com> (raw)
In-Reply-To: <20190813205225.12032-1-yu-cheng.yu@intel.com>

When Shadow Stack is enabled, the [R/O + PAGE_DIRTY_HW] setting is
reserved only for the Shadow Stack.  Non-Shadow Stack R/O PTEs use
[R/O + PAGE_DIRTY_SW].

When a PTE goes from [R/W + PAGE_DIRTY_HW] to [R/O + PAGE_DIRTY_SW],
it could become a transient Shadow Stack PTE in two cases.

The first case is that some processors can start a write but end up
seeing a read-only PTE by the time they get to the Dirty bit,
creating a transient Shadow Stack PTE.  However, this will not occur
on processors supporting Shadow Stack therefore we don't need a TLB
flush here.

The second case is that when the software, without atomic, tests &
replaces PAGE_DIRTY_HW with PAGE_DIRTY_SW, a transient Shadow Stack
PTE can exist.  This is prevented with cmpxchg.

Dave Hansen, Jann Horn, Andy Lutomirski, and Peter Zijlstra provided
many insights to the issue.  Jann Horn provided the cmpxchg solution.

Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com>
---
 arch/x86/include/asm/pgtable.h | 58 ++++++++++++++++++++++++++++++++++
 1 file changed, 58 insertions(+)

diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index 1448fb38f248..81c8c5ec221e 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -1222,7 +1222,36 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
 static inline void ptep_set_wrprotect(struct mm_struct *mm,
 				      unsigned long addr, pte_t *ptep)
 {
+#ifdef CONFIG_X86_INTEL_SHADOW_STACK_USER
+	pte_t new_pte, pte = READ_ONCE(*ptep);
+
+	/*
+	 * Some processors can start a write, but end up
+	 * seeing a read-only PTE by the time they get
+	 * to the Dirty bit.  In this case, they will
+	 * set the Dirty bit, leaving a read-only, Dirty
+	 * PTE which looks like a Shadow Stack PTE.
+	 *
+	 * However, this behavior has been improved and
+	 * will not occur on processors supporting
+	 * Shadow Stacks.  Without this guarantee, a
+	 * transition to a non-present PTE and flush the
+	 * TLB would be needed.
+	 *
+	 * When changing a writable PTE to read-only and
+	 * if the PTE has _PAGE_DIRTY_HW set, we move
+	 * that bit to _PAGE_DIRTY_SW so that the PTE is
+	 * not a valid Shadow Stack PTE.
+	 */
+	do {
+		new_pte = pte_wrprotect(pte);
+		new_pte.pte |= (new_pte.pte & _PAGE_DIRTY_HW) >>
+				_PAGE_BIT_DIRTY_HW << _PAGE_BIT_DIRTY_SW;
+		new_pte.pte &= ~_PAGE_DIRTY_HW;
+	} while (!try_cmpxchg(ptep, &pte, new_pte));
+#else
 	clear_bit(_PAGE_BIT_RW, (unsigned long *)&ptep->pte);
+#endif
 }
 
 #define flush_tlb_fix_spurious_fault(vma, address) do { } while (0)
@@ -1285,7 +1314,36 @@ static inline pud_t pudp_huge_get_and_clear(struct mm_struct *mm,
 static inline void pmdp_set_wrprotect(struct mm_struct *mm,
 				      unsigned long addr, pmd_t *pmdp)
 {
+#ifdef CONFIG_X86_INTEL_SHADOW_STACK_USER
+	pmd_t new_pmd, pmd = READ_ONCE(*pmdp);
+
+	/*
+	 * Some processors can start a write, but end up
+	 * seeing a read-only PMD by the time they get
+	 * to the Dirty bit.  In this case, they will
+	 * set the Dirty bit, leaving a read-only, Dirty
+	 * PMD which looks like a Shadow Stack PMD.
+	 *
+	 * However, this behavior has been improved and
+	 * will not occur on processors supporting
+	 * Shadow Stacks.  Without this guarantee, a
+	 * transition to a non-present PMD and flush the
+	 * TLB would be needed.
+	 *
+	 * When changing a writable PMD to read-only and
+	 * if the PMD has _PAGE_DIRTY_HW set, we move
+	 * that bit to _PAGE_DIRTY_SW so that the PMD is
+	 * not a valid Shadow Stack PMD.
+	 */
+	do {
+		new_pmd = pmd_wrprotect(pmd);
+		new_pmd.pmd |= (new_pmd.pmd & _PAGE_DIRTY_HW) >>
+				_PAGE_BIT_DIRTY_HW << _PAGE_BIT_DIRTY_SW;
+		new_pmd.pmd &= ~_PAGE_DIRTY_HW;
+	} while (!try_cmpxchg(pmdp, &pmd, new_pmd));
+#else
 	clear_bit(_PAGE_BIT_RW, (unsigned long *)pmdp);
+#endif
 }
 
 #define pud_write pud_write
-- 
2.17.1



  parent reply index

Thread overview: 53+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-08-13 20:51 [PATCH v8 00/27] Control-flow Enforcement: Shadow Stack Yu-cheng Yu
2019-08-13 20:51 ` [PATCH v8 01/27] Documentation/x86: Add CET description Yu-cheng Yu
2019-08-14  8:07   ` Florian Weimer
2019-08-14 15:57     ` Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 02/27] x86/cpufeatures: Add CET CPU feature flags for Control-flow Enforcement Technology (CET) Yu-cheng Yu
2019-08-21 10:20   ` Borislav Petkov
2019-08-21 14:46     ` Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 03/27] x86/fpu/xstate: Change names to separate XSAVES system and user states Yu-cheng Yu
2019-08-21 14:45   ` Borislav Petkov
2019-08-13 20:52 ` [PATCH v8 04/27] x86/fpu/xstate: Introduce XSAVES system states Yu-cheng Yu
2019-08-16 19:56   ` Thomas Gleixner
2019-08-16 19:59     ` Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 05/27] x86/fpu/xstate: Introduce CET MSR " Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 06/27] x86/cet: Add control protection exception handler Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 07/27] x86/cet/shstk: Add Kconfig option for user-mode shadow stack Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 08/27] mm: Introduce VM_SHSTK for shadow stack memory Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 09/27] mm/mmap: Prevent Shadow Stack VMA merges Yu-cheng Yu
2019-08-13 22:34   ` Dave Hansen
2019-08-14 16:20     ` Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 10/27] x86/mm: Change _PAGE_DIRTY to _PAGE_DIRTY_HW Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 11/27] x86/mm: Introduce _PAGE_DIRTY_SW Yu-cheng Yu
2019-08-13 23:02   ` Dave Hansen
2019-08-13 23:49     ` Andy Lutomirski
2019-08-14 16:42     ` Yu-cheng Yu
2019-08-14 16:58       ` Dave Hansen
2019-08-23 14:02   ` Peter Zijlstra
2019-08-27 22:37     ` Yu-cheng Yu
2019-08-28  7:03       ` Peter Zijlstra
2019-08-28 14:57         ` Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 12/27] drm/i915/gvt: Update _PAGE_DIRTY to _PAGE_DIRTY_BITS Yu-cheng Yu
2019-08-13 20:52 ` Yu-cheng Yu [this message]
2019-08-13 20:52 ` [PATCH v8 14/27] x86/mm: Shadow stack page fault error checking Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 15/27] mm: Handle shadow stack page fault Yu-cheng Yu
2019-08-13 22:55   ` Andy Lutomirski
2019-08-14 16:27     ` Yu-cheng Yu
2019-08-14 16:48       ` Dave Hansen
2019-08-14 17:00         ` Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 16/27] mm: Handle THP/HugeTLB " Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 17/27] mm: Update can_follow_write_pte/pmd for shadow stack Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 18/27] mm: Introduce do_mmap_locked() Yu-cheng Yu
2019-08-20  1:02   ` Sean Christopherson
2019-08-20 16:08     ` Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 19/27] x86/cet/shstk: User-mode shadow stack support Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 20/27] x86/cet/shstk: Introduce WRUSS instruction Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 21/27] x86/cet/shstk: Handle signals for shadow stack Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 22/27] binfmt_elf: Extract .note.gnu.property from an ELF file Yu-cheng Yu
2019-08-16 13:55   ` Dave Martin
2019-08-20 10:02   ` Dave Martin
2019-08-13 20:52 ` [PATCH v8 23/27] x86/cet/shstk: ELF header parsing of Shadow Stack Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 24/27] x86/cet/shstk: Handle thread shadow stack Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 25/27] mm/mmap: Add Shadow stack pages to memory accounting Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 26/27] x86/cet/shstk: Add arch_prctl functions for Shadow Stack Yu-cheng Yu
2019-08-13 20:52 ` [PATCH v8 27/27] x86/cet/shstk: Add Shadow Stack instructions to opcode map Yu-cheng Yu

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190813205225.12032-14-yu-cheng.yu@intel.com \
    --to=yu-cheng.yu@intel.com \
    --cc=Dave.Martin@arm.com \
    --cc=arnd@arndb.de \
    --cc=bp@alien8.de \
    --cc=bsingharora@gmail.com \
    --cc=corbet@lwn.net \
    --cc=dave.hansen@linux.intel.com \
    --cc=esyr@redhat.com \
    --cc=fweimer@redhat.com \
    --cc=gorcunov@gmail.com \
    --cc=hjl.tools@gmail.com \
    --cc=hpa@zytor.com \
    --cc=jannh@google.com \
    --cc=keescook@chromium.org \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-arch@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=luto@amacapital.net \
    --cc=mike.kravetz@oracle.com \
    --cc=mingo@redhat.com \
    --cc=nadav.amit@gmail.com \
    --cc=oleg@redhat.com \
    --cc=pavel@ucw.cz \
    --cc=peterz@infradead.org \
    --cc=ravi.v.shankar@intel.com \
    --cc=rdunlap@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=vedvyas.shanbhogue@intel.com \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org linux-mm@archiver.kernel.org
	public-inbox-index linux-mm


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/ public-inbox