LKML Archive on lore.kernel.org
 help / color / Atom feed
From: tip-bot for Thomas Gleixner <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: aarcange@redhat.com, longman9394@gmail.com, mingo@kernel.org,
	torvalds@linux-foundation.org, dave.hansen@intel.com,
	peterz@infradead.org, tglx@linutronix.de, ak@linux.intel.com,
	luto@kernel.org, thomas.lendacky@amd.com, jcm@redhat.com,
	jpoimboe@redhat.com, hpa@zytor.com, arjan@linux.intel.com,
	asit.k.mallick@intel.com, casey.schaufler@intel.com,
	keescook@chromium.org, jkosina@suse.cz,
	linux-kernel@vger.kernel.org, gregkh@linuxfoundation.org,
	dwmw@amazon.co.uk, tim.c.chen@linux.intel.com,
	david.c.stewart@intel.com
Subject: [tip:x86/pti] x86/speculation: Avoid __switch_to_xtra() calls
Date: Wed, 28 Nov 2018 06:31:22 -0800
Message-ID: <tip-5635d99953f04b550738f6f4c1c532667c3fd872@git.kernel.org> (raw)
In-Reply-To: <20181125185005.374062201@linutronix.de>

Commit-ID:  5635d99953f04b550738f6f4c1c532667c3fd872
Gitweb:     https://git.kernel.org/tip/5635d99953f04b550738f6f4c1c532667c3fd872
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Sun, 25 Nov 2018 19:33:48 +0100
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Wed, 28 Nov 2018 11:57:11 +0100

x86/speculation: Avoid __switch_to_xtra() calls

The TIF_SPEC_IB bit does not need to be evaluated in the decision to invoke
__switch_to_xtra() when:

 - CONFIG_SMP is disabled

 - The conditional STIPB mode is disabled

The TIF_SPEC_IB bit still controls IBPB in both cases so the TIF work mask
checks might invoke __switch_to_xtra() for nothing if TIF_SPEC_IB is the
only set bit in the work masks.

Optimize it out by masking the bit at compile time for CONFIG_SMP=n and at
run time when the static key controlling the conditional STIBP mode is
disabled.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Reviewed-by: Ingo Molnar <mingo@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Andy Lutomirski <luto@kernel.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jiri Kosina <jkosina@suse.cz>
Cc: Tom Lendacky <thomas.lendacky@amd.com>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: David Woodhouse <dwmw@amazon.co.uk>
Cc: Tim Chen <tim.c.chen@linux.intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Casey Schaufler <casey.schaufler@intel.com>
Cc: Asit Mallick <asit.k.mallick@intel.com>
Cc: Arjan van de Ven <arjan@linux.intel.com>
Cc: Jon Masters <jcm@redhat.com>
Cc: Waiman Long <longman9394@gmail.com>
Cc: Greg KH <gregkh@linuxfoundation.org>
Cc: Dave Stewart <david.c.stewart@intel.com>
Cc: Kees Cook <keescook@chromium.org>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/20181125185005.374062201@linutronix.de


---
 arch/x86/include/asm/thread_info.h | 13 +++++++++++--
 arch/x86/kernel/process.h          | 15 +++++++++++++++
 2 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/thread_info.h b/arch/x86/include/asm/thread_info.h
index fa583ec99e3e..6d201699c651 100644
--- a/arch/x86/include/asm/thread_info.h
+++ b/arch/x86/include/asm/thread_info.h
@@ -147,9 +147,18 @@ struct thread_info {
 	 _TIF_FSCHECK)
 
 /* flags to check in __switch_to() */
-#define _TIF_WORK_CTXSW							\
+#define _TIF_WORK_CTXSW_BASE						\
 	(_TIF_IO_BITMAP|_TIF_NOCPUID|_TIF_NOTSC|_TIF_BLOCKSTEP|		\
-	 _TIF_SSBD|_TIF_SPEC_IB)
+	 _TIF_SSBD)
+
+/*
+ * Avoid calls to __switch_to_xtra() on UP as STIBP is not evaluated.
+ */
+#ifdef CONFIG_SMP
+# define _TIF_WORK_CTXSW	(_TIF_WORK_CTXSW_BASE | _TIF_SPEC_IB)
+#else
+# define _TIF_WORK_CTXSW	(_TIF_WORK_CTXSW_BASE)
+#endif
 
 #define _TIF_WORK_CTXSW_PREV (_TIF_WORK_CTXSW|_TIF_USER_RETURN_NOTIFY)
 #define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW)
diff --git a/arch/x86/kernel/process.h b/arch/x86/kernel/process.h
index 020fbfac3a27..898e97cf6629 100644
--- a/arch/x86/kernel/process.h
+++ b/arch/x86/kernel/process.h
@@ -2,6 +2,8 @@
 //
 // Code shared between 32 and 64 bit
 
+#include <asm/spec-ctrl.h>
+
 void __switch_to_xtra(struct task_struct *prev_p, struct task_struct *next_p);
 
 /*
@@ -14,6 +16,19 @@ static inline void switch_to_extra(struct task_struct *prev,
 	unsigned long next_tif = task_thread_info(next)->flags;
 	unsigned long prev_tif = task_thread_info(prev)->flags;
 
+	if (IS_ENABLED(CONFIG_SMP)) {
+		/*
+		 * Avoid __switch_to_xtra() invocation when conditional
+		 * STIPB is disabled and the only different bit is
+		 * TIF_SPEC_IB. For CONFIG_SMP=n TIF_SPEC_IB is not
+		 * in the TIF_WORK_CTXSW masks.
+		 */
+		if (!static_branch_likely(&switch_to_cond_stibp)) {
+			prev_tif &= ~_TIF_SPEC_IB;
+			next_tif &= ~_TIF_SPEC_IB;
+		}
+	}
+
 	/*
 	 * __switch_to_xtra() handles debug registers, i/o bitmaps,
 	 * speculation mitigations etc.

  reply index

Thread overview: 112+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-11-25 18:33 [patch V2 00/28] x86/speculation: Remedy the STIBP/IBPB overhead Thomas Gleixner
2018-11-25 18:33 ` [patch V2 01/28] x86/speculation: Update the TIF_SSBD comment Thomas Gleixner
2018-11-28 14:20   ` [tip:x86/pti] " tip-bot for Tim Chen
2018-11-29 14:27   ` [patch V2 01/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 02/28] x86/speculation: Clean up spectre_v2_parse_cmdline() Thomas Gleixner
2018-11-28 14:20   ` [tip:x86/pti] " tip-bot for Tim Chen
2018-11-29 14:28   ` [patch V2 02/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 03/28] x86/speculation: Remove unnecessary ret variable in cpu_show_common() Thomas Gleixner
2018-11-28 14:21   ` [tip:x86/pti] " tip-bot for Tim Chen
2018-11-29 14:28   ` [patch V2 03/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 04/28] x86/speculation: Reorganize cpu_show_common() Thomas Gleixner
2018-11-26 15:08   ` Borislav Petkov
2018-11-28 14:22   ` [tip:x86/pti] x86/speculation: Move STIPB/IBPB string conditionals out of cpu_show_common() tip-bot for Tim Chen
2018-11-29 14:29   ` [patch V2 04/28] x86/speculation: Reorganize cpu_show_common() Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 05/28] x86/speculation: Disable STIBP when enhanced IBRS is in use Thomas Gleixner
2018-11-28 14:22   ` [tip:x86/pti] " tip-bot for Tim Chen
2018-11-29 14:35   ` [patch V2 05/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 06/28] x86/speculation: Rename SSBD update functions Thomas Gleixner
2018-11-26 15:24   ` Borislav Petkov
2018-11-28 14:23   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-29 14:37   ` [patch V2 06/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 07/28] x86/speculation: Reorganize speculation control MSRs update Thomas Gleixner
2018-11-26 15:47   ` Borislav Petkov
2018-11-28 14:23   ` [tip:x86/pti] " tip-bot for Tim Chen
2018-11-29 14:41   ` [patch V2 07/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 08/28] sched/smt: Make sched_smt_present track topology Thomas Gleixner
2018-11-28 14:24   ` [tip:x86/pti] " tip-bot for Peter Zijlstra (Intel)
2018-11-29 14:42   ` [patch V2 08/28] " Konrad Rzeszutek Wilk
2018-11-29 14:50     ` Konrad Rzeszutek Wilk
2018-11-29 15:48       ` Peter Zijlstra
2018-11-25 18:33 ` [patch V2 09/28] x86/Kconfig: Select SCHED_SMT if SMP enabled Thomas Gleixner
2018-11-28 14:24   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-29 14:44   ` [patch V2 09/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 10/28] sched/smt: Expose sched_smt_present static key Thomas Gleixner
2018-11-28 14:25   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-29 14:44   ` [patch V2 10/28] " Konrad Rzeszutek Wilk
2018-11-25 18:33 ` [patch V2 11/28] x86/speculation: Rework SMT state change Thomas Gleixner
2018-11-28 14:26   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 12/28] x86/l1tf: Show actual SMT state Thomas Gleixner
2018-11-28 14:26   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 13/28] x86/speculation: Reorder the spec_v2 code Thomas Gleixner
2018-11-26 22:21   ` Borislav Petkov
2018-11-28 14:27   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 14/28] x86/speculation: Mark string arrays const correctly Thomas Gleixner
2018-11-28 14:27   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 15/28] x86/speculataion: Mark command line parser data __initdata Thomas Gleixner
2018-11-28 14:28   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 16/28] x86/speculation: Unify conditional spectre v2 print functions Thomas Gleixner
2018-11-28 14:29   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 17/28] x86/speculation: Add command line control for indirect branch speculation Thomas Gleixner
2018-11-28 14:29   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 18/28] x86/speculation: Prepare for per task indirect branch speculation control Thomas Gleixner
2018-11-27 17:25   ` Lendacky, Thomas
2018-11-27 19:51     ` Tim Chen
2018-11-28  9:39       ` Thomas Gleixner
2018-11-27 20:39     ` Thomas Gleixner
2018-11-27 20:42       ` Thomas Gleixner
2018-11-27 21:52         ` Lendacky, Thomas
2018-11-28 14:30   ` [tip:x86/pti] " tip-bot for Tim Chen
2018-11-25 18:33 ` [patch V2 19/28] x86/process: Consolidate and simplify switch_to_xtra() code Thomas Gleixner
2018-11-26 18:30   ` Borislav Petkov
2018-11-28 14:30   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 20/28] x86/speculation: Avoid __switch_to_xtra() calls Thomas Gleixner
2018-11-28 14:31   ` tip-bot for Thomas Gleixner [this message]
2018-11-25 18:33 ` [patch V2 21/28] x86/speculation: Prepare for conditional IBPB in switch_mm() Thomas Gleixner
2018-11-25 19:11   ` Thomas Gleixner
2018-11-25 20:53   ` Andi Kleen
2018-11-25 22:20     ` Thomas Gleixner
2018-11-25 23:04       ` Andy Lutomirski
2018-11-26  7:10         ` Thomas Gleixner
2018-11-26 13:36           ` Ingo Molnar
2018-11-26  3:07       ` Andi Kleen
2018-11-26  6:50         ` Thomas Gleixner
2018-11-28 14:31   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 22/28] ptrace: Remove unused ptrace_may_access_sched() and MODE_IBRS Thomas Gleixner
2018-11-28 14:32   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 23/28] x86/speculation: Split out TIF update Thomas Gleixner
2018-11-28 14:33   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 24/28] x86/speculation: Prepare arch_smt_update() for PRCTL mode Thomas Gleixner
2018-11-27 20:18   ` Lendacky, Thomas
2018-11-27 20:30     ` Thomas Gleixner
2018-11-27 21:20       ` Lendacky, Thomas
2018-11-28 14:34   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 25/28] x86/speculation: Add prctl() control for indirect branch speculation Thomas Gleixner
2018-11-28 14:34   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 26/28] x86/speculation: Enable prctl mode for spectre_v2_user Thomas Gleixner
2018-11-26  7:56   ` Dominik Brodowski
2018-11-28 14:35   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-25 18:33 ` [patch V2 27/28] x86/speculation: Add seccomp Spectre v2 user space protection mode Thomas Gleixner
2018-11-25 19:35   ` Randy Dunlap
2018-11-25 20:40   ` Linus Torvalds
2018-11-25 20:52     ` Jiri Kosina
2018-11-25 22:28     ` Thomas Gleixner
2018-11-26 13:30       ` Ingo Molnar
2018-11-26 20:48       ` Andrea Arcangeli
2018-11-26 20:58         ` Thomas Gleixner
2018-11-26 21:52           ` Lendacky, Thomas
2018-11-27  0:37             ` Tim Chen
2018-12-04  1:38     ` Tim Chen
2018-12-04  8:39       ` Jiri Kosina
2018-12-04  9:43         ` Arjan van de Ven
2018-12-04  9:46         ` Arjan van de Ven
2018-12-04 17:20       ` Linus Torvalds
2018-12-04 18:58         ` Tim Chen
2018-11-28 14:35   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-12-04 18:45   ` [patch V2 27/28] " Dave Hansen
2018-11-25 18:33 ` [patch V2 28/28] x86/speculation: Provide IBPB always command line options Thomas Gleixner
2018-11-28 14:36   ` [tip:x86/pti] " tip-bot for Thomas Gleixner
2018-11-26 13:37 ` [patch V2 00/28] x86/speculation: Remedy the STIBP/IBPB overhead Ingo Molnar
2018-11-28 14:24 ` Thomas Gleixner
2018-11-29 19:02   ` Tim Chen
2018-12-10 23:43 ` Pavel Machek

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=tip-5635d99953f04b550738f6f4c1c532667c3fd872@git.kernel.org \
    --to=tipbot@zytor.com \
    --cc=aarcange@redhat.com \
    --cc=ak@linux.intel.com \
    --cc=arjan@linux.intel.com \
    --cc=asit.k.mallick@intel.com \
    --cc=casey.schaufler@intel.com \
    --cc=dave.hansen@intel.com \
    --cc=david.c.stewart@intel.com \
    --cc=dwmw@amazon.co.uk \
    --cc=gregkh@linuxfoundation.org \
    --cc=hpa@zytor.com \
    --cc=jcm@redhat.com \
    --cc=jkosina@suse.cz \
    --cc=jpoimboe@redhat.com \
    --cc=keescook@chromium.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-tip-commits@vger.kernel.org \
    --cc=longman9394@gmail.com \
    --cc=luto@kernel.org \
    --cc=mingo@kernel.org \
    --cc=peterz@infradead.org \
    --cc=tglx@linutronix.de \
    --cc=thomas.lendacky@amd.com \
    --cc=tim.c.chen@linux.intel.com \
    --cc=torvalds@linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git