linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, tglx@linutronix.de, bp@alien8.de,
	luto@kernel.org
Cc: hpa@zytor.com, dave.hansen@intel.com, tony.luck@intel.com,
	ak@linux.intel.com, ravi.v.shankar@intel.com,
	chang.seok.bae@intel.com,
	Andrew Cooper <andrew.cooper3@citrix.com>,
	Sasha Levin <sashal@kernel.org>
Subject: [PATCH v10 10/18] x86/fsgsbase/64: Enable FSGSBASE instructions in helper functions
Date: Thu, 23 Apr 2020 19:21:59 -0400	[thread overview]
Message-ID: <20200423232207.5797-11-sashal@kernel.org> (raw)
In-Reply-To: <20200423232207.5797-1-sashal@kernel.org>

From: "Chang S. Bae" <chang.seok.bae@intel.com>

Add CPU feature conditional FS/GS base access to the relevant helper
functions. That allows accelerating certain FS/GS base operations in
subsequent changes.

Note, that while possible, the user space entry/exit GS base operations are
not going to use the new FSGSBASE instructions. The reason is that it would
require additional storage for the user space value which adds more
complexity to the low level code and experiments have shown marginal
benefit. This may be revisited later but for now the SWAPGS based handling
in the entry code is preserved except for the paranoid entry/exit code.

Suggested-by: Tony Luck <tony.luck@intel.com>
Signed-off-by: Chang S. Bae <chang.seok.bae@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Borislav Petkov <bp@alien8.de>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: H. Peter Anvin <hpa@zytor.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Tony Luck <tony.luck@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 arch/x86/include/asm/fsgsbase.h | 27 +++++++--------
 arch/x86/kernel/process_64.c    | 58 +++++++++++++++++++++++++++++++++
 2 files changed, 70 insertions(+), 15 deletions(-)

diff --git a/arch/x86/include/asm/fsgsbase.h b/arch/x86/include/asm/fsgsbase.h
index fdd1177499b40..aefd53767a5d4 100644
--- a/arch/x86/include/asm/fsgsbase.h
+++ b/arch/x86/include/asm/fsgsbase.h
@@ -49,35 +49,32 @@ static __always_inline void wrgsbase(unsigned long gsbase)
 	asm volatile("wrgsbase %0" :: "r" (gsbase) : "memory");
 }
 
+#include <asm/cpufeature.h>
+
 /* Helper functions for reading/writing FS/GS base */
 
 static inline unsigned long x86_fsbase_read_cpu(void)
 {
 	unsigned long fsbase;
 
-	rdmsrl(MSR_FS_BASE, fsbase);
+	if (static_cpu_has(X86_FEATURE_FSGSBASE))
+		fsbase = rdfsbase();
+	else
+		rdmsrl(MSR_FS_BASE, fsbase);
 
 	return fsbase;
 }
 
-static inline unsigned long x86_gsbase_read_cpu_inactive(void)
-{
-	unsigned long gsbase;
-
-	rdmsrl(MSR_KERNEL_GS_BASE, gsbase);
-
-	return gsbase;
-}
-
 static inline void x86_fsbase_write_cpu(unsigned long fsbase)
 {
-	wrmsrl(MSR_FS_BASE, fsbase);
+	if (static_cpu_has(X86_FEATURE_FSGSBASE))
+		wrfsbase(fsbase);
+	else
+		wrmsrl(MSR_FS_BASE, fsbase);
 }
 
-static inline void x86_gsbase_write_cpu_inactive(unsigned long gsbase)
-{
-	wrmsrl(MSR_KERNEL_GS_BASE, gsbase);
-}
+extern unsigned long x86_gsbase_read_cpu_inactive(void);
+extern void x86_gsbase_write_cpu_inactive(unsigned long gsbase);
 
 #endif /* CONFIG_X86_64 */
 
diff --git a/arch/x86/kernel/process_64.c b/arch/x86/kernel/process_64.c
index 5ef9d8f25b0e8..aaa65f284b9b9 100644
--- a/arch/x86/kernel/process_64.c
+++ b/arch/x86/kernel/process_64.c
@@ -328,6 +328,64 @@ static unsigned long x86_fsgsbase_read_task(struct task_struct *task,
 	return base;
 }
 
+unsigned long x86_gsbase_read_cpu_inactive(void)
+{
+	unsigned long gsbase;
+
+	if (static_cpu_has(X86_FEATURE_FSGSBASE)) {
+		bool need_restore = false;
+		unsigned long flags;
+
+		/*
+		 * We read the inactive GS base value by swapping
+		 * to make it the active one. But we cannot allow
+		 * an interrupt while we switch to and from.
+		 */
+		if (!irqs_disabled()) {
+			local_irq_save(flags);
+			need_restore = true;
+		}
+
+		native_swapgs();
+		gsbase = rdgsbase();
+		native_swapgs();
+
+		if (need_restore)
+			local_irq_restore(flags);
+	} else {
+		rdmsrl(MSR_KERNEL_GS_BASE, gsbase);
+	}
+
+	return gsbase;
+}
+
+void x86_gsbase_write_cpu_inactive(unsigned long gsbase)
+{
+	if (static_cpu_has(X86_FEATURE_FSGSBASE)) {
+		bool need_restore = false;
+		unsigned long flags;
+
+		/*
+		 * We write the inactive GS base value by swapping
+		 * to make it the active one. But we cannot allow
+		 * an interrupt while we switch to and from.
+		 */
+		if (!irqs_disabled()) {
+			local_irq_save(flags);
+			need_restore = true;
+		}
+
+		native_swapgs();
+		wrgsbase(gsbase);
+		native_swapgs();
+
+		if (need_restore)
+			local_irq_restore(flags);
+	} else {
+		wrmsrl(MSR_KERNEL_GS_BASE, gsbase);
+	}
+}
+
 unsigned long x86_fsbase_read_task(struct task_struct *task)
 {
 	unsigned long fsbase;
-- 
2.20.1


  parent reply	other threads:[~2020-04-23 23:23 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-23 23:21 [PATCH v10 00/18] Enable FSGSBASE instructions Sasha Levin
2020-04-23 23:21 ` [PATCH v10 01/18] x86/ptrace: Prevent ptrace from clearing the FS/GS selector Sasha Levin
2020-04-25 22:46   ` Andy Lutomirski
2020-04-23 23:21 ` [PATCH v10 02/18] selftests/x86/fsgsbase: Test GS selector on ptracer-induced GS base write Sasha Levin
2020-04-23 23:21 ` [PATCH v10 03/18] x86/cpu: Add 'unsafe_fsgsbase' to enable CR4.FSGSBASE Sasha Levin
2020-04-23 23:21 ` [PATCH v10 04/18] x86/entry/64: Clean up paranoid exit Sasha Levin
2020-04-23 23:21 ` [PATCH v10 05/18] x86/entry/64: Switch CR3 before SWAPGS in paranoid entry Sasha Levin
2020-04-23 23:21 ` [PATCH v10 06/18] x86/entry/64: Introduce the FIND_PERCPU_BASE macro Sasha Levin
2020-04-23 23:21 ` [PATCH v10 07/18] x86/entry/64: Handle FSGSBASE enabled paranoid entry/exit Sasha Levin
2020-04-23 23:21 ` [PATCH v10 08/18] x86/entry/64: Document GSBASE handling in the paranoid path Sasha Levin
2020-04-23 23:21 ` [PATCH v10 09/18] x86/fsgsbase/64: Add intrinsics for FSGSBASE instructions Sasha Levin
2020-04-23 23:21 ` Sasha Levin [this message]
2020-04-23 23:22 ` [PATCH v10 11/18] x86/fsgsbase/64: Use FSGSBASE in switch_to() if available Sasha Levin
2020-04-23 23:22 ` [PATCH v10 12/18] x86/fsgsbase/64: move save_fsgs to header file Sasha Levin
2020-04-23 23:22 ` [PATCH v10 13/18] x86/fsgsbase/64: Use FSGSBASE instructions on thread copy and ptrace Sasha Levin
2020-04-23 23:22 ` [PATCH v10 14/18] x86/speculation/swapgs: Check FSGSBASE in enabling SWAPGS mitigation Sasha Levin
2020-04-23 23:22 ` [PATCH v10 15/18] selftests/x86/fsgsbase: Test ptracer-induced GS base write with FSGSBASE Sasha Levin
2020-04-23 23:22 ` [PATCH v10 16/18] x86/fsgsbase/64: Enable FSGSBASE on 64bit by default and add a chicken bit Sasha Levin
2020-04-23 23:22 ` [PATCH v10 17/18] x86/elf: Enumerate kernel FSGSBASE capability in AT_HWCAP2 Sasha Levin
2020-04-23 23:22 ` [PATCH v10 18/18] Documentation/x86/64: Add documentation for GS/FS addressing mode Sasha Levin
2020-05-10  8:09 ` [PATCH v10 00/18] Enable FSGSBASE instructions Vegard Nossum
2020-05-10  8:29   ` Vegard Nossum
2020-05-10 10:15     ` Thomas Gleixner
2020-05-10 14:17       ` Sasha Levin
2020-05-11  0:48     ` Andi Kleen
2020-05-11  0:50       ` Andi Kleen
2020-05-11  5:03         ` Sasha Levin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200423232207.5797-11-sashal@kernel.org \
    --to=sashal@kernel.org \
    --cc=ak@linux.intel.com \
    --cc=andrew.cooper3@citrix.com \
    --cc=bp@alien8.de \
    --cc=chang.seok.bae@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=hpa@zytor.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=luto@kernel.org \
    --cc=ravi.v.shankar@intel.com \
    --cc=tglx@linutronix.de \
    --cc=tony.luck@intel.com \
    --subject='Re: [PATCH v10 10/18] x86/fsgsbase/64: Enable FSGSBASE instructions in helper functions' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
on how to clone and mirror all data and code used for this inbox