From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A5EEC46467 for ; Thu, 19 Jan 2023 21:23:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5784C6B007D; Thu, 19 Jan 2023 16:23:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 4B2976B007E; Thu, 19 Jan 2023 16:23:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2433C6B0080; Thu, 19 Jan 2023 16:23:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 0DAB76B007D for ; Thu, 19 Jan 2023 16:23:34 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id CBEDE807E9 for ; Thu, 19 Jan 2023 21:23:33 +0000 (UTC) X-FDA: 80372825106.19.2B478FD Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf29.hostedemail.com (Postfix) with ESMTP id D5C4A120008 for ; Thu, 19 Jan 2023 21:23:31 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Rux9FGGx; spf=pass (imf29.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674163412; a=rsa-sha256; cv=none; b=WSEmvyXri3UBlyjroUjXeI8PlDokuu6tmxpUtHzZgrZMljOPSIHqzeb+EfCcVzKVZ4CFBu ae8/kE48sonW014Fs8/s1Ly14hc5k9cWDQk6mKz/A7jKEJp31joQWsM0W1aiAFkzO2KPPw /1DWZiMdUEFk9NjVa6ldL6hNyevYkOo= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Rux9FGGx; spf=pass (imf29.hostedemail.com: domain of rick.p.edgecombe@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=rick.p.edgecombe@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674163412; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references:dkim-signature; bh=vxp0lIQdGnMd31RbCo3vuJaZKtAwgMarQuDGlKx2nuY=; b=SkQYxLC6v6pkhP4+YcsMUWduDmybCTEDI/yKEhi0KdvwoLFqdX2P5CYa3bAaGZy7dt4oxT 1uKI80ffRNd0thtzuEFJHi7qCTZ/j8VHJN33Sx4ysHxe6evUqN8jtiC0l+txVU+/9JLllQ q0BfOaTg/396aOwSyrI4i668WAv9TuI= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1674163411; x=1705699411; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=d7P5HLFSxYXgzr8kgfmXPsFqpuUUbCRpcWe/3LmTQFM=; b=Rux9FGGx+4Ascy/Gj71YzZ16od31T3djf10JESqSWbscq+mF+ot2RdNB bKCIQ/PJvTO3vU8u3Vn+zYLMvl5nLZVe1OnYmchaJZPbo4G5KhzfYFuch Abi2UDV9GNUPxWRg8hBmWiWdO731kbipFT7KUvvgqn+m5uAjfq5xBemeY FmxKiG5F0VjJqFadMGu2YAwSSmt1NJZ/nTDF/ROz5vlmEMVDC8eyQg13G DbMoJTkPbPfTUFkbkwFQ/9f+B43XiLCxNFQTONLq4MAHHsuANkkXTYpH8 f0hBznpL65ntHsXVGnN/tyrpm0+SGN2euORp3pYHx77zYjEPGpJaFW2un w==; X-IronPort-AV: E=McAfee;i="6500,9779,10595"; a="323119247" X-IronPort-AV: E=Sophos;i="5.97,230,1669104000"; d="scan'208";a="323119247" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Jan 2023 13:23:31 -0800 X-IronPort-AV: E=McAfee;i="6500,9779,10595"; a="989139002" X-IronPort-AV: E=Sophos;i="5.97,230,1669104000"; d="scan'208";a="989139002" Received: from hossain3-mobl.amr.corp.intel.com (HELO rpedgeco-desk.amr.corp.intel.com) ([10.252.128.187]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Jan 2023 13:23:29 -0800 From: Rick Edgecombe To: x86@kernel.org, "H . Peter Anvin" , Thomas Gleixner , Ingo Molnar , linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, linux-mm@kvack.org, linux-arch@vger.kernel.org, linux-api@vger.kernel.org, Arnd Bergmann , Andy Lutomirski , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H . J . Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , Weijiang Yang , "Kirill A . Shutemov" , John Allen , kcc@google.com, eranian@google.com, rppt@kernel.org, jamorris@linux.microsoft.com, dethoma@microsoft.com, akpm@linux-foundation.org, Andrew.Cooper3@citrix.com, christina.schimpe@intel.com Cc: rick.p.edgecombe@intel.com, Yu-cheng Yu Subject: [PATCH v5 04/39] x86/cpufeatures: Enable CET CR4 bit for shadow stack Date: Thu, 19 Jan 2023 13:22:42 -0800 Message-Id: <20230119212317.8324-5-rick.p.edgecombe@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230119212317.8324-1-rick.p.edgecombe@intel.com> References: <20230119212317.8324-1-rick.p.edgecombe@intel.com> X-Rspam-User: X-Rspamd-Queue-Id: D5C4A120008 X-Rspamd-Server: rspam01 X-Stat-Signature: fzir9hterp5fyujsgw6c4b7rwhtbx7rn X-HE-Tag: 1674163411-249626 X-HE-Meta: U2FsdGVkX1+r9uT23vBufA/xdrr1WTcmquffuSZunOngseOtEG5lK4LBnaAn12sYpXzmRGDv0fExZcuhJl90Q2cDsv/uDU0FD3qRKJY9wk+sAZ4tKhaj6iT+aWiKj0jOSRnzcn74J/y5XgUl03Hq7C8j1O8qMI7Nn1nEM9h4XckarlMQIO+ru5VWQG9I6S8zfLoVWj6OzqCnOgVi+TbJYoZBxBbnGBZXR4+8w6rA5jWw3autTZZjxmU/Vi87RTx5hp3TI1DHpbLr0/Yc1hKEZDMObHqpgsloCq9dlONOxI+TrKaWuGlV9PKjbA+v0Ovj7P7mYOHYZyLOCfb0SNXrZap1AEva6uobuiTH/GvmoP+w2ed9Ho0q2xwaryCTSzBuqzVdwtzLYvi4xXlNzmVEpR995ytfRDGUVLAuCw6tS3DqSM2MXd7WKEWpYQOyItBBJ/vZ23OBm0KBxj3lKek4GKZZzaHfRzxIa8ReUQq7N1qgqZk4g2h5fo92jsQU+K6PTGkCUtUhca0XvGEtHf4hrEeVgTEMbv98fPZ0TTzX6qXc5Y2/QaquczMHLpe3fNklyeEAD4VV1ZKp/sWSczE7gFh3rjytITbfIIEwqhZQzyxspU+WisawSt/m31X942Ij7myyt/4SkSlGinF/a94GlOMUIux9rClb0ZsuqkiLraKJx9Hc+o3EkEOolZ2D0nwflRBoqG2VxI8j+j+mcduC29u/CWpz/JJXhrPyrzxEiKQWAqEHLOkNZ//MkpLBszejP+u22PsGhYuQN28IrEvsoN44k+wlTXeY1cz/E999Q1E9yFsh6sBaupQI5vua5tju5EbfWUm7rWZRe2vHZEVQrrMw5mMmwNbxn5FSn8OdXgJcrncNGuDebRvjwiaCX2TeLYHVeIXcZwni+UZ81XyIxDGpQ5F0H7lvWU3uaKa9qqjC4+M49c6wGPK0EK2edrhcP3v4SwicdvR/oxIwblQ qniNy8EE 05+frYUwzn2e81RLKW9JMJAiYaidrM63hpbWhx4vyVhShdhC/qzpPStSxK4t9snLzTiUyOdHRnL7pzLseILlpQTJjQQSEWyiA1L1X6E0D/3He8ld0AhN9TlNK5ja8dxqnP57zLwVtnNOO/JFyAPxbR7ElMTfjyLh5zRORnHM4hCyM7rYbhFWxGuUOcuSXyaGmDNMIyoZ4ek9h3akOqLp1hvla0e5b/uEm6J/oFxt+ZfllPzpBL44zqC0mY+RwhocaYQ3vj3juRx0ymrXfjG4niQC/eDAZ8kSyAt// X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Yu-cheng Yu Setting CR4.CET is a prerequisite for utilizing any CET features, most of which also require setting MSRs. Kernel IBT already enables the CET CR4 bit when it detects IBT HW support and is configured with kernel IBT. However, future patches that enable userspace shadow stack support will need the bit set as well. So change the logic to enable it in either case. Clear MSR_IA32_U_CET in cet_disable() so that it can't live to see userspace in a new kexec-ed kernel that has CR4.CET set from kernel IBT. Tested-by: Pengfei Xu Tested-by: John Allen Signed-off-by: Yu-cheng Yu Co-developed-by: Rick Edgecombe Signed-off-by: Rick Edgecombe Cc: Kees Cook --- v5: - Remove #ifdeffery (Boris) v4: - Add back dedicated command line disable: "nousershtk" (Boris) v3: - Remove stay new line (Boris) - Simplify commit log (Andrew Cooper) v2: - In the shadow stack case, go back to only setting CR4.CET if the kernel is compiled with user shadow stack support. - Clear MSR_IA32_U_CET as well. (PeterZ) arch/x86/kernel/cpu/common.c | 35 +++++++++++++++++++++++++++-------- 1 file changed, 27 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index cec654e674ff..80507a5ba0ca 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -599,27 +599,43 @@ __noendbr void ibt_restore(u64 save) static __always_inline void setup_cet(struct cpuinfo_x86 *c) { - u64 msr = CET_ENDBR_EN; + bool user_shstk, kernel_ibt; - if (!HAS_KERNEL_IBT || - !cpu_feature_enabled(X86_FEATURE_IBT)) + if (!IS_ENABLED(CONFIG_X86_CET)) return; - wrmsrl(MSR_IA32_S_CET, msr); + kernel_ibt = HAS_KERNEL_IBT && cpu_feature_enabled(X86_FEATURE_IBT); + user_shstk = cpu_feature_enabled(X86_FEATURE_SHSTK) && + IS_ENABLED(CONFIG_X86_USER_SHADOW_STACK); + + if (!kernel_ibt && !user_shstk) + return; + + if (user_shstk) + set_cpu_cap(c, X86_FEATURE_USER_SHSTK); + + if (kernel_ibt) + wrmsrl(MSR_IA32_S_CET, CET_ENDBR_EN); + else + wrmsrl(MSR_IA32_S_CET, 0); + cr4_set_bits(X86_CR4_CET); - if (!ibt_selftest()) { + if (kernel_ibt && !ibt_selftest()) { pr_err("IBT selftest: Failed!\n"); wrmsrl(MSR_IA32_S_CET, 0); setup_clear_cpu_cap(X86_FEATURE_IBT); - return; } } __noendbr void cet_disable(void) { - if (cpu_feature_enabled(X86_FEATURE_IBT)) - wrmsrl(MSR_IA32_S_CET, 0); + if (!(cpu_feature_enabled(X86_FEATURE_IBT) || + cpu_feature_enabled(X86_FEATURE_SHSTK))) + return; + + wrmsrl(MSR_IA32_S_CET, 0); + wrmsrl(MSR_IA32_U_CET, 0); } /* @@ -1476,6 +1492,9 @@ static void __init cpu_parse_early_param(void) if (cmdline_find_option_bool(boot_command_line, "noxsaves")) setup_clear_cpu_cap(X86_FEATURE_XSAVES); + if (cmdline_find_option_bool(boot_command_line, "nousershstk")) + setup_clear_cpu_cap(X86_FEATURE_USER_SHSTK); + arglen = cmdline_find_option(boot_command_line, "clearcpuid", arg, sizeof(arg)); if (arglen <= 0) return; -- 2.17.1