From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DB48C47084 for ; Fri, 21 May 2021 22:13:55 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 3BCB3613EC for ; Fri, 21 May 2021 22:13:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BCB3613EC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 6C3958E0063; Fri, 21 May 2021 18:13:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 628458E0061; Fri, 21 May 2021 18:13:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33FDA8E0063; Fri, 21 May 2021 18:13:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0168.hostedemail.com [216.40.44.168]) by kanga.kvack.org (Postfix) with ESMTP id 003478E0061 for ; Fri, 21 May 2021 18:13:22 -0400 (EDT) Received: from smtpin08.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 9BA531801BD6D for ; Fri, 21 May 2021 22:13:22 +0000 (UTC) X-FDA: 78166640244.08.5024B52 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf06.hostedemail.com (Postfix) with ESMTP id 3D619C0042D0 for ; Fri, 21 May 2021 22:13:19 +0000 (UTC) IronPort-SDR: 8KtH355Dyo/1XKKLUZRWWvOfivx1vr/aGRvLAe5XfqRaQ6Rj/piy5ixstpGdcviMJ1gmq18vhn azJq4b/e53Bg== X-IronPort-AV: E=McAfee;i="6200,9189,9991"; a="201636881" X-IronPort-AV: E=Sophos;i="5.82,319,1613462400"; d="scan'208";a="201636881" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2021 15:13:21 -0700 IronPort-SDR: 9grkf+SI7ppJ+5YNiyUxWQdmGXyuFHoYom/p4zvTvihQMWd68GcVfyfkZMWGVmqXfxwHXYHNwC ylQ1fAtfgsAw== X-IronPort-AV: E=Sophos;i="5.82,319,1613462400"; d="scan'208";a="441116233" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 21 May 2021 15:13:21 -0700 From: Yu-cheng Yu To: x86@kernel.org, "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang , Pengfei Xu , Haitao Huang Cc: Yu-cheng Yu Subject: [PATCH v27 25/31] x86/cet/shstk: Introduce shadow stack token setup/verify routines Date: Fri, 21 May 2021 15:12:05 -0700 Message-Id: <20210521221211.29077-26-yu-cheng.yu@intel.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20210521221211.29077-1-yu-cheng.yu@intel.com> References: <20210521221211.29077-1-yu-cheng.yu@intel.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 3D619C0042D0 Authentication-Results: imf06.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf06.hostedemail.com: domain of yu-cheng.yu@intel.com has no SPF policy when checking 134.134.136.65) smtp.mailfrom=yu-cheng.yu@intel.com X-Rspamd-Server: rspam04 X-Stat-Signature: 5kffnw8fy8qd1rhaesewh5dk8q5sgbhq X-HE-Tag: 1621635199-333793 Content-Transfer-Encoding: quoted-printable X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A shadow stack restore token marks a restore point of the shadow stack, a= nd the address in a token must point directly above the token, which is with= in the same shadow stack. This is distinctively different from other pointe= rs on the shadow stack, since those pointers point to executable code area. The restore token can be used as an extra protection for signal handling. To deliver a signal, create a shadow stack restore token and put the toke= n and the signal restorer address on the shadow stack. In sigreturn, verif= y the token and restore from it the shadow stack pointer. Introduce token setup and verify routines. Also introduce WRUSS, which i= s a kernel-mode instruction but writes directly to user shadow stack. It i= s used to construct user signal stack as described above. Signed-off-by: Yu-cheng Yu Cc: Kees Cook --- v27: - For shstk_check_rstor_token(), instead of an input param, use current shadow stack pointer. - In response to comments, fix/simplify a few syntax/format issues. v25: - Update inline assembly syntax, use %[]. - Change token address from (unsigned long) to (u64/u32 __user *). - Change -EPERM to -EFAULT. arch/x86/include/asm/cet.h | 7 ++ arch/x86/include/asm/special_insns.h | 30 ++++++ arch/x86/kernel/shstk.c | 133 +++++++++++++++++++++++++++ 3 files changed, 170 insertions(+) diff --git a/arch/x86/include/asm/cet.h b/arch/x86/include/asm/cet.h index 4314a41ab3c9..aa533700ba31 100644 --- a/arch/x86/include/asm/cet.h +++ b/arch/x86/include/asm/cet.h @@ -21,6 +21,9 @@ int shstk_alloc_thread_stack(struct task_struct *p, uns= igned long clone_flags, unsigned long stack_size); void shstk_free(struct task_struct *p); void shstk_disable(void); +int shstk_setup_rstor_token(bool ia32, unsigned long restorer, + unsigned long *new_ssp); +int shstk_check_rstor_token(bool ia32, unsigned long *new_ssp); #else static inline int shstk_setup(void) { return 0; } static inline int shstk_alloc_thread_stack(struct task_struct *p, @@ -28,6 +31,10 @@ static inline int shstk_alloc_thread_stack(struct task= _struct *p, unsigned long stack_size) { return 0; } static inline void shstk_free(struct task_struct *p) {} static inline void shstk_disable(void) {} +static inline int shstk_setup_rstor_token(bool ia32, unsigned long resto= rer, + unsigned long *new_ssp) { return 0; } +static inline int shstk_check_rstor_token(bool ia32, + unsigned long *new_ssp) { return 0; } #endif =20 #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/= special_insns.h index 2acd6cb62328..5b48c91fa8d4 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -234,6 +234,36 @@ static inline void clwb(volatile void *__p) : [pax] "a" (p)); } =20 +#ifdef CONFIG_X86_SHADOW_STACK +static inline int write_user_shstk_32(u32 __user *addr, u32 val) +{ + if (WARN_ONCE(!IS_ENABLED(CONFIG_IA32_EMULATION) && + !IS_ENABLED(CONFIG_X86_X32), + "%s used but not supported.\n", __func__)) { + return -EFAULT; + } + + asm_volatile_goto("1: wrussd %[val], (%[addr])\n" + _ASM_EXTABLE(1b, %l[fail]) + :: [addr] "r" (addr), [val] "r" (val) + :: fail); + return 0; +fail: + return -EFAULT; +} + +static inline int write_user_shstk_64(u64 __user *addr, u64 val) +{ + asm_volatile_goto("1: wrussq %[val], (%[addr])\n" + _ASM_EXTABLE(1b, %l[fail]) + :: [addr] "r" (addr), [val] "r" (val) + :: fail); + return 0; +fail: + return -EFAULT; +} +#endif /* CONFIG_X86_SHADOW_STACK */ + #define nop() asm volatile ("nop") =20 static inline void serialize(void) diff --git a/arch/x86/kernel/shstk.c b/arch/x86/kernel/shstk.c index 8e5f772181b9..61ec300c1a97 100644 --- a/arch/x86/kernel/shstk.c +++ b/arch/x86/kernel/shstk.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 static void start_update_msrs(void) { @@ -181,3 +182,135 @@ void shstk_disable(void) =20 shstk_free(current); } + +static unsigned long get_user_shstk_addr(void) +{ + struct fpu *fpu =3D ¤t->thread.fpu; + unsigned long ssp =3D 0; + + fpregs_lock(); + + if (fpregs_state_valid(fpu, smp_processor_id())) { + rdmsrl(MSR_IA32_PL3_SSP, ssp); + } else { + struct cet_user_state *p; + + p =3D get_xsave_addr(&fpu->state.xsave, XFEATURE_CET_USER); + if (p) + ssp =3D p->user_ssp; + } + + fpregs_unlock(); + + return ssp; +} + +/* + * Create a restore token on the shadow stack. A token is always 8-byte + * and aligned to 8. + */ +static int create_rstor_token(bool ia32, unsigned long ssp, + unsigned long *token_addr) +{ + unsigned long addr; + + /* Aligned to 8 is aligned to 4, so test 8 first */ + if ((!ia32 && !IS_ALIGNED(ssp, 8)) || !IS_ALIGNED(ssp, 4)) + return -EINVAL; + + addr =3D ALIGN_DOWN(ssp, 8) - 8; + + /* Is the token for 64-bit? */ + if (!ia32) + ssp |=3D BIT(0); + + if (write_user_shstk_64((u64 __user *)addr, (u64)ssp)) + return -EFAULT; + + *token_addr =3D addr; + + return 0; +} + +/* + * Create a restore token on shadow stack, and then push the user-mode + * function return address. + */ +int shstk_setup_rstor_token(bool ia32, unsigned long ret_addr, + unsigned long *new_ssp) +{ + struct thread_shstk *shstk =3D ¤t->thread.shstk; + unsigned long ssp, token_addr; + int err; + + if (!shstk->size) + return 0; + + if (!ret_addr) + return -EINVAL; + + ssp =3D get_user_shstk_addr(); + if (!ssp) + return -EINVAL; + + err =3D create_rstor_token(ia32, ssp, &token_addr); + if (err) + return err; + + if (ia32) { + ssp =3D token_addr - sizeof(u32); + err =3D write_user_shstk_32((u32 __user *)ssp, (u32)ret_addr); + } else { + ssp =3D token_addr - sizeof(u64); + err =3D write_user_shstk_64((u64 __user *)ssp, (u64)ret_addr); + } + + if (!err) + *new_ssp =3D ssp; + + return err; +} + +/* + * Verify token_addr points to a valid token, and then set *new_ssp + * according to the token. + */ +int shstk_check_rstor_token(bool proc32, unsigned long *new_ssp) +{ + unsigned long token_addr; + unsigned long token; + bool shstk32; + + token_addr =3D get_user_shstk_addr(); + + if (get_user(token, (unsigned long __user *)token_addr)) + return -EFAULT; + + /* Is mode flag correct? */ + shstk32 =3D !(token & BIT(0)); + if (proc32 ^ shstk32) + return -EINVAL; + + /* Is busy flag set? */ + if (token & BIT(1)) + return -EINVAL; + + /* Mask out flags */ + token &=3D ~3UL; + + /* + * Restore address aligned? + */ + if ((!proc32 && !IS_ALIGNED(token, 8)) || !IS_ALIGNED(token, 4)) + return -EINVAL; + + /* + * Token placed properly? + */ + if (((ALIGN_DOWN(token, 8) - 8) !=3D token_addr) || token >=3D TASK_SIZ= E_MAX) + return -EINVAL; + + *new_ssp =3D token; + + return 0; +} --=20 2.21.0