From: Yu-cheng Yu <yu-cheng.yu@intel.com> To: x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann <arnd@arndb.de>, Andy Lutomirski <luto@amacapital.net>, Balbir Singh <bsingharora@gmail.com>, Cyrill Gorcunov <gorcunov@gmail.com>, Dave Hansen <dave.hansen@linux.intel.com>, Eugene Syromiatnikov <esyr@redhat.com>, Florian Weimer <fweimer@redhat.com>, "H.J. Lu" <hjl.tools@gmail.com>, Jann Horn <jannh@google.com>, Jonathan Corbet <corbet@lwn.net>, Kees Cook <keescook@chromium.org>, Mike Kravetz <mike.kravetz@oracle.com>, Nadav Amit <nadav.amit@gmail.com>, Oleg Nesterov <oleg@redhat.com>, Pa Cc: Yu-cheng Yu <yu-cheng.yu@intel.com> Subject: [PATCH v5 21/27] x86/cet/shstk: Introduce WRUSS instruction Date: Thu, 11 Oct 2018 08:15:17 -0700 [thread overview] Message-ID: <20181011151523.27101-22-yu-cheng.yu@intel.com> (raw) In-Reply-To: <20181011151523.27101-1-yu-cheng.yu@intel.com> WRUSS is a new kernel-mode instruction but writes directly to user shadow stack memory. This is used to construct a return address on the shadow stack for the signal handler. This instruction can fault if the user shadow stack is invalid shadow stack memory. In that case, the kernel does a fixup. Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com> --- arch/x86/include/asm/special_insns.h | 32 ++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 9 ++++++++ 2 files changed, 41 insertions(+) diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h index 317fc59b512c..37f16269747d 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -237,6 +237,38 @@ static inline void clwb(volatile void *__p) : [pax] "a" (p)); } +#ifdef CONFIG_X86_INTEL_CET +#if defined(CONFIG_IA32_EMULATION) || defined(CONFIG_X86_X32) +static inline int write_user_shstk_32(unsigned long addr, unsigned int val) +{ + asm_volatile_goto("1: wrussd %1, (%0)\n" + _ASM_EXTABLE(1b, %l[fail]) + :: "r" (addr), "r" (val) + :: fail); + return 0; +fail: + return -EPERM; +} +#else +static inline int write_user_shstk_32(unsigned long addr, unsigned int val) +{ + WARN_ONCE(1, "%s used but not supported.\n", __func__); + return -EFAULT; +} +#endif + +static inline int write_user_shstk_64(unsigned long addr, unsigned long val) +{ + asm_volatile_goto("1: wrussq %1, (%0)\n" + _ASM_EXTABLE(1b, %l[fail]) + :: "r" (addr), "r" (val) + :: fail); + return 0; +fail: + return -EPERM; +} +#endif /* CONFIG_X86_INTEL_CET */ + #define nop() asm volatile ("nop") diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 7c3877a982f4..b91fc008f33a 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1305,6 +1305,15 @@ __do_page_fault(struct pt_regs *regs, unsigned long error_code, error_code |= X86_PF_USER; flags |= FAULT_FLAG_USER; } else { + /* + * WRUSS is a kernel instruction and but writes + * to user shadow stack. When a fault occurs, + * both X86_PF_USER and X86_PF_SHSTK are set. + * Clear X86_PF_USER here. + */ + if ((error_code & (X86_PF_USER | X86_PF_SHSTK)) == + (X86_PF_USER | X86_PF_SHSTK)) + error_code &= ~X86_PF_USER; if (regs->flags & X86_EFLAGS_IF) local_irq_enable(); } -- 2.17.1
WARNING: multiple messages have this Message-ID (diff)
From: Yu-cheng Yu <yu-cheng.yu@intel.com> To: x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>, Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@redhat.com>, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann <arnd@arndb.de>, Andy Lutomirski <luto@amacapital.net>, Balbir Singh <bsingharora@gmail.com>, Cyrill Gorcunov <gorcunov@gmail.com>, Dave Hansen <dave.hansen@linux.intel.com>, Eugene Syromiatnikov <esyr@redhat.com>, Florian Weimer <fweimer@redhat.com>, "H.J. Lu" <hjl.tools@gmail.com>, Jann Horn <jannh@google.com>, Jonathan Corbet <corbet@lwn.net>, Kees Cook <keescook@chromium.org>, Mike Kravetz <mike.kravetz@oracle.com>, Nadav Amit <nadav.amit@gmail.com>, Oleg Nesterov <oleg@redhat.com>, Pavel Machek <pavel@ucw.cz>, Peter Zijlstra <peterz@infradead.org>, Randy Dunlap <rdunlap@infradead.org>, "Ravi V. Shankar" <ravi.v.shankar@intel.com>, Vedvyas Shanbhogue <vedvyas.shanbhogue@intel.com> Cc: Yu-cheng Yu <yu-cheng.yu@intel.com> Subject: [PATCH v5 21/27] x86/cet/shstk: Introduce WRUSS instruction Date: Thu, 11 Oct 2018 08:15:17 -0700 [thread overview] Message-ID: <20181011151523.27101-22-yu-cheng.yu@intel.com> (raw) Message-ID: <20181011151517.TKEcsf41ie7syc8qZwe_fA3qkg0cOoqXIrtWFohInOA@z> (raw) In-Reply-To: <20181011151523.27101-1-yu-cheng.yu@intel.com> WRUSS is a new kernel-mode instruction but writes directly to user shadow stack memory. This is used to construct a return address on the shadow stack for the signal handler. This instruction can fault if the user shadow stack is invalid shadow stack memory. In that case, the kernel does a fixup. Signed-off-by: Yu-cheng Yu <yu-cheng.yu@intel.com> --- arch/x86/include/asm/special_insns.h | 32 ++++++++++++++++++++++++++++ arch/x86/mm/fault.c | 9 ++++++++ 2 files changed, 41 insertions(+) diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/special_insns.h index 317fc59b512c..37f16269747d 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -237,6 +237,38 @@ static inline void clwb(volatile void *__p) : [pax] "a" (p)); } +#ifdef CONFIG_X86_INTEL_CET +#if defined(CONFIG_IA32_EMULATION) || defined(CONFIG_X86_X32) +static inline int write_user_shstk_32(unsigned long addr, unsigned int val) +{ + asm_volatile_goto("1: wrussd %1, (%0)\n" + _ASM_EXTABLE(1b, %l[fail]) + :: "r" (addr), "r" (val) + :: fail); + return 0; +fail: + return -EPERM; +} +#else +static inline int write_user_shstk_32(unsigned long addr, unsigned int val) +{ + WARN_ONCE(1, "%s used but not supported.\n", __func__); + return -EFAULT; +} +#endif + +static inline int write_user_shstk_64(unsigned long addr, unsigned long val) +{ + asm_volatile_goto("1: wrussq %1, (%0)\n" + _ASM_EXTABLE(1b, %l[fail]) + :: "r" (addr), "r" (val) + :: fail); + return 0; +fail: + return -EPERM; +} +#endif /* CONFIG_X86_INTEL_CET */ + #define nop() asm volatile ("nop") diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 7c3877a982f4..b91fc008f33a 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1305,6 +1305,15 @@ __do_page_fault(struct pt_regs *regs, unsigned long error_code, error_code |= X86_PF_USER; flags |= FAULT_FLAG_USER; } else { + /* + * WRUSS is a kernel instruction and but writes + * to user shadow stack. When a fault occurs, + * both X86_PF_USER and X86_PF_SHSTK are set. + * Clear X86_PF_USER here. + */ + if ((error_code & (X86_PF_USER | X86_PF_SHSTK)) == + (X86_PF_USER | X86_PF_SHSTK)) + error_code &= ~X86_PF_USER; if (regs->flags & X86_EFLAGS_IF) local_irq_enable(); } -- 2.17.1
next prev parent reply other threads:[~2018-10-11 15:15 UTC|newest] Thread overview: 160+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-10-11 15:14 [PATCH v5 00/27] Control Flow Enforcement: Shadow Stack Yu-cheng Yu 2018-10-11 15:14 ` Yu-cheng Yu 2018-10-11 15:14 ` [PATCH v5 01/27] x86/cpufeatures: Add CPUIDs for Control Flow Enforcement Technology (CET) Yu-cheng Yu 2018-10-11 15:14 ` Yu-cheng Yu 2018-10-11 16:43 ` Borislav Petkov 2018-10-11 16:43 ` Borislav Petkov 2018-10-11 16:45 ` Yu-cheng Yu 2018-10-11 16:45 ` Yu-cheng Yu 2018-10-11 15:14 ` [PATCH v5 02/27] x86/fpu/xstate: Change names to separate XSAVES system and user states Yu-cheng Yu 2018-10-11 15:14 ` Yu-cheng Yu 2018-10-15 17:03 ` Borislav Petkov 2018-10-15 17:03 ` Borislav Petkov 2018-10-11 15:14 ` [PATCH v5 03/27] x86/fpu/xstate: Introduce XSAVES system states Yu-cheng Yu 2018-10-11 15:14 ` Yu-cheng Yu 2018-10-17 10:41 ` Borislav Petkov 2018-10-17 10:41 ` Borislav Petkov 2018-10-17 22:39 ` Randy Dunlap 2018-10-17 22:39 ` Randy Dunlap 2018-10-17 22:58 ` Borislav Petkov 2018-10-17 22:58 ` Borislav Petkov 2018-10-17 23:17 ` Randy Dunlap 2018-10-17 23:17 ` Randy Dunlap 2018-10-18 9:26 ` Borislav Petkov 2018-10-18 9:26 ` Borislav Petkov 2018-10-18 9:31 ` Pavel Machek 2018-10-18 9:31 ` Pavel Machek 2018-10-18 12:10 ` Borislav Petkov 2018-10-18 12:10 ` Borislav Petkov 2018-10-18 18:33 ` Randy Dunlap 2018-10-18 18:33 ` Randy Dunlap 2018-10-18 9:24 ` Pavel Machek 2018-10-18 9:24 ` Pavel Machek 2018-10-11 15:15 ` [PATCH v5 04/27] x86/fpu/xstate: Add XSAVES system states for shadow stack Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-11-08 18:40 ` Borislav Petkov 2018-11-08 18:40 ` Borislav Petkov 2018-11-08 20:40 ` Yu-cheng Yu 2018-11-08 20:40 ` Yu-cheng Yu 2018-11-08 23:52 ` Borislav Petkov 2018-11-08 23:52 ` Borislav Petkov 2018-11-11 11:31 ` Pavel Machek 2018-11-11 11:31 ` Pavel Machek 2018-11-11 11:31 ` Pavel Machek 2018-11-11 11:31 ` Pavel Machek 2018-11-11 14:59 ` Andy Lutomirski 2018-11-11 14:59 ` Andy Lutomirski 2018-11-11 19:02 ` Pavel Machek 2018-11-11 19:02 ` Pavel Machek 2018-11-08 20:46 ` Andy Lutomirski 2018-11-08 20:46 ` Andy Lutomirski 2018-11-08 21:01 ` Yu-cheng Yu 2018-11-08 21:01 ` Yu-cheng Yu 2018-11-08 21:22 ` Andy Lutomirski 2018-11-08 21:22 ` Andy Lutomirski 2018-11-08 21:31 ` Cyrill Gorcunov 2018-11-08 21:31 ` Cyrill Gorcunov 2018-11-08 22:01 ` Andy Lutomirski 2018-11-08 22:01 ` Andy Lutomirski 2018-11-08 22:18 ` Cyrill Gorcunov 2018-11-08 22:18 ` Cyrill Gorcunov 2018-11-08 21:48 ` Dave Hansen 2018-11-08 21:48 ` Dave Hansen 2018-11-08 22:00 ` Matthew Wilcox 2018-11-08 22:00 ` Matthew Wilcox 2018-11-08 23:35 ` Dave Hansen 2018-11-08 23:35 ` Dave Hansen 2018-11-09 0:32 ` Matthew Wilcox 2018-11-09 0:32 ` Matthew Wilcox 2018-11-09 0:45 ` Andy Lutomirski 2018-11-09 0:45 ` Andy Lutomirski 2018-11-09 17:13 ` Dave Hansen 2018-11-09 17:13 ` Dave Hansen 2018-11-09 17:17 ` Matthew Wilcox 2018-11-09 17:17 ` Matthew Wilcox 2018-11-09 17:20 ` Dave Hansen 2018-11-09 17:20 ` Dave Hansen 2018-11-09 17:28 ` Dave Hansen 2018-11-09 17:28 ` Dave Hansen 2018-11-11 11:31 ` Pavel Machek 2018-11-11 11:31 ` Pavel Machek 2018-10-11 15:15 ` [PATCH v5 05/27] Documentation/x86: Add CET description Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-11-13 18:43 ` Borislav Petkov 2018-11-13 18:43 ` Borislav Petkov 2018-11-13 21:02 ` Yu-cheng Yu 2018-11-13 21:02 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 06/27] x86/cet: Control protection exception handler Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-11-14 18:44 ` Borislav Petkov 2018-11-14 18:44 ` Borislav Petkov 2018-11-14 20:19 ` Yu-cheng Yu 2018-11-14 20:19 ` Yu-cheng Yu 2018-11-14 20:28 ` Borislav Petkov 2018-11-14 20:28 ` Borislav Petkov 2018-10-11 15:15 ` [PATCH v5 07/27] mm/mmap: Create a guard area between VMAs Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 20:39 ` Jann Horn 2018-10-11 20:39 ` Jann Horn 2018-10-11 20:49 ` Yu-cheng Yu 2018-10-11 20:49 ` Yu-cheng Yu 2018-10-11 20:55 ` Andy Lutomirski 2018-10-11 20:55 ` Andy Lutomirski 2018-10-12 21:49 ` Yu-cheng Yu 2018-10-12 21:49 ` Yu-cheng Yu 2018-10-12 13:17 ` Matthew Wilcox 2018-10-12 13:17 ` Matthew Wilcox 2018-10-11 20:49 ` Dave Hansen 2018-10-11 20:49 ` Dave Hansen 2018-10-12 10:24 ` Florian Weimer 2018-10-12 10:24 ` Florian Weimer 2018-10-11 15:15 ` [PATCH v5 08/27] x86/cet/shstk: Add Kconfig option for user-mode shadow stack Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 09/27] mm: Introduce VM_SHSTK for shadow stack memory Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 10/27] mm/mmap: Prevent Shadow Stack VMA merges Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 11/27] x86/mm: Change _PAGE_DIRTY to _PAGE_DIRTY_HW Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 12/27] x86/mm: Introduce _PAGE_DIRTY_SW Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 13/27] drm/i915/gvt: Update _PAGE_DIRTY to _PAGE_DIRTY_BITS Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 14/27] x86/mm: Modify ptep_set_wrprotect and pmdp_set_wrprotect for _PAGE_DIRTY_SW Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 15/27] x86/mm: Shadow stack page fault error checking Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 16/27] mm: Handle shadow stack page fault Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 17/27] mm: Handle THP/HugeTLB " Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 18/27] mm: Update can_follow_write_pte/pmd for shadow stack Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 19/27] mm: Introduce do_mmap_locked() Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 20/27] x86/cet/shstk: User-mode shadow stack support Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu [this message] 2018-10-11 15:15 ` [PATCH v5 21/27] x86/cet/shstk: Introduce WRUSS instruction Yu-cheng Yu 2018-11-06 18:43 ` Dave Hansen 2018-11-06 18:43 ` Dave Hansen 2018-11-06 18:55 ` Andy Lutomirski 2018-11-06 18:55 ` Andy Lutomirski 2018-11-06 20:21 ` Yu-cheng Yu 2018-11-06 20:21 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 22/27] x86/cet/shstk: Signal handling for shadow stack Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 23/27] x86/cet/shstk: ELF header parsing of Shadow Stack Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 24/27] x86/cet/shstk: Handle thread shadow stack Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 25/27] mm/mmap: Add Shadow stack pages to memory accounting Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 26/27] x86/cet/shstk: Add arch_prctl functions for Shadow Stack Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 15:15 ` [PATCH v5 27/27] x86/cet/shstk: Add Shadow Stack instructions to opcode map Yu-cheng Yu 2018-10-11 15:15 ` Yu-cheng Yu 2018-10-11 19:21 ` [PATCH v5 00/27] Control Flow Enforcement: Shadow Stack Dave Hansen 2018-10-11 19:21 ` Dave Hansen 2018-10-11 19:29 ` Yu-cheng Yu 2018-10-11 19:29 ` Yu-cheng Yu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20181011151523.27101-22-yu-cheng.yu@intel.com \ --to=yu-cheng.yu@intel.com \ --cc=arnd@arndb.de \ --cc=bsingharora@gmail.com \ --cc=corbet@lwn.net \ --cc=dave.hansen@linux.intel.com \ --cc=esyr@redhat.com \ --cc=fweimer@redhat.com \ --cc=gorcunov@gmail.com \ --cc=hjl.tools@gmail.com \ --cc=hpa@zytor.com \ --cc=jannh@google.com \ --cc=keescook@chromium.org \ --cc=linux-api@vger.kernel.org \ --cc=linux-arch@vger.kernel.org \ --cc=linux-doc@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=luto@amacapital.net \ --cc=mike.kravetz@oracle.com \ --cc=mingo@redhat.com \ --cc=nadav.amit@gmail.com \ --cc=oleg@redhat.com \ --cc=tglx@linutronix.de \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).