[PATCH v2 0/2] TSX update

[PATCH v2 0/2] TSX update

[Pawan Gupta]

v2:
- Added patch to disable TSX development mode (Andrew, Boris)
- Rebased to v5.17-rc7

v1: https://lore.kernel.org/lkml/5bd785a1d6ea0b572250add0c6617b4504bc24d1.1644440311.git.pawan.kumar.gupta@linux.intel.com/

Hi,

After a recent microcode update some Intel processors will always abort
Transactional Synchronization Extensions (TSX) transactions [*]. On
these processors a new CPUID bit,
CPUID.07H.0H.EDX[11](RTM_ALWAYS_ABORT), will be enumerated to indicate
that the loaded microcode is forcing Restricted Transactional Memory
(RTM) abort. If the processor enumerated support for RTM previously, the
CPUID enumeration bits for TSX (CPUID.RTM and CPUID.HLE) continue to be
set by default after the microcode update.

First patch in this series clears CPUID.RTM and CPUID.HLE so that
userspace doesn't enumerate TSX feature. 

Microcode also added support to re-enable TSX for development purpose,
doing so is not recommended for production deployments, because MD_CLEAR
flows for the mitigation of TSX Asynchronous Abort (TAA) may not be
effective on these processors when TSX is enabled with updated
microcode.

Second patch unconditionally disables this TSX development mode, in case
it was enabled by the software running before kernel boot.

Thanks,
Pawan

[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
    https://cdrdv2.intel.com/v1/dl/getContent/643557

Pawan Gupta (2):
  x86/tsx: Use MSR_TSX_CTRL to clear CPUID bits
  x86/tsx: Disable TSX development mode at boot

 arch/x86/include/asm/msr-index.h       |  4 +-
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  5 ++
 arch/x86/kernel/cpu/tsx.c              | 88 ++++++++++++++++++++++++--
 tools/arch/x86/include/asm/msr-index.h |  4 +-
 5 files changed, 91 insertions(+), 11 deletions(-)


base-commit: ffb217a13a2eaf6d5bd974fc83036a53ca69f1e2
-- 
2.25.1

[PATCH v2 1/2] x86/tsx: Use MSR_TSX_CTRL to clear CPUID bits

[Pawan Gupta]

tsx_clear_cpuid() uses MSR_TSX_FORCE_ABORT to clear CPUID.RTM and
CPUID.HLE. Not all CPUs support MSR_TSX_FORCE_ABORT, alternatively use
MSR_IA32_TSX_CTRL when supported.

  [ bp: Document how and why TSX gets disabled. ]

Fixes: 293649307ef9 ("x86/tsx: Clear CPUID bits when TSX always force aborts")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Tested-by: Neelima Krishnan <neelima.krishnan@intel.com>
Cc: <stable@vger.kernel.org>
---
 arch/x86/kernel/cpu/intel.c |  1 +
 arch/x86/kernel/cpu/tsx.c   | 54 ++++++++++++++++++++++++++++++++-----
 2 files changed, 48 insertions(+), 7 deletions(-)

diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8321c43554a1..8abf995677a4 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -722,6 +722,7 @@ static void init_intel(struct cpuinfo_x86 *c)
 	else if (tsx_ctrl_state == TSX_CTRL_DISABLE)
 		tsx_disable();
 	else if (tsx_ctrl_state == TSX_CTRL_RTM_ALWAYS_ABORT)
+		/* See comment over that function for more details. */
 		tsx_clear_cpuid();
 
 	split_lock_init();
diff --git a/arch/x86/kernel/cpu/tsx.c b/arch/x86/kernel/cpu/tsx.c
index 9c7a5f049292..2835fa89fc6f 100644
--- a/arch/x86/kernel/cpu/tsx.c
+++ b/arch/x86/kernel/cpu/tsx.c
@@ -58,7 +58,7 @@ void tsx_enable(void)
 	wrmsrl(MSR_IA32_TSX_CTRL, tsx);
 }
 
-static bool __init tsx_ctrl_is_supported(void)
+static bool tsx_ctrl_is_supported(void)
 {
 	u64 ia32_cap = x86_read_arch_cap_msr();
 
@@ -84,6 +84,44 @@ static enum tsx_ctrl_states x86_get_tsx_auto_mode(void)
 	return TSX_CTRL_ENABLE;
 }
 
+/*
+ * Disabling TSX is not a trivial business.
+ *
+ * First of all, there's a CPUID bit: X86_FEATURE_RTM_ALWAYS_ABORT
+ * which says that TSX is practically disabled (all transactions are
+ * aborted by default). When that bit is set, the kernel unconditionally
+ * disables TSX.
+ *
+ * In order to do that, however, it needs to dance a bit:
+ *
+ * 1. The first method to disable it is through MSR_TSX_FORCE_ABORT and
+ * the MSR is present only when *two* CPUID bits are set:
+ *
+ * - X86_FEATURE_RTM_ALWAYS_ABORT
+ * - X86_FEATURE_TSX_FORCE_ABORT
+ *
+ * 2. The second method is for CPUs which do not have the above-mentioned
+ * MSR: those use a different MSR - MSR_IA32_TSX_CTRL and disable TSX
+ * through that one. Those CPUs can also have the initially mentioned
+ * CPUID bit X86_FEATURE_RTM_ALWAYS_ABORT set and for those the same strategy
+ * applies: TSX gets disabled unconditionally.
+ *
+ * When either of the two methods are present, the kernel disables TSX and
+ * clears the respective RTM and HLE feature flags.
+ *
+ * An additional twist in the whole thing presents late microcode loading
+ * which, when done, may cause for the X86_FEATURE_RTM_ALWAYS_ABORT CPUID
+ * bit to be set after the update.
+ *
+ * A subsequent hotplug operation on any logical CPU except the BSP will
+ * cause for the supported CPUID feature bits to get re-detected and, if
+ * RTM and HLE get cleared all of a sudden, but, userspace did consult
+ * them before the update, then funny explosions will happen. Long story
+ * short: the kernel doesn't modify CPUID feature bits after booting.
+ *
+ * That's why, this function's call in init_intel() doesn't clear the
+ * feature flags.
+ */
 void tsx_clear_cpuid(void)
 {
 	u64 msr;
@@ -97,6 +135,10 @@ void tsx_clear_cpuid(void)
 		rdmsrl(MSR_TSX_FORCE_ABORT, msr);
 		msr |= MSR_TFA_TSX_CPUID_CLEAR;
 		wrmsrl(MSR_TSX_FORCE_ABORT, msr);
+	} else if (tsx_ctrl_is_supported()) {
+		rdmsrl(MSR_IA32_TSX_CTRL, msr);
+		msr |= TSX_CTRL_CPUID_CLEAR;
+		wrmsrl(MSR_IA32_TSX_CTRL, msr);
 	}
 }
 
@@ -106,13 +148,11 @@ void __init tsx_init(void)
 	int ret;
 
 	/*
-	 * Hardware will always abort a TSX transaction if both CPUID bits
-	 * RTM_ALWAYS_ABORT and TSX_FORCE_ABORT are set. In this case, it is
-	 * better not to enumerate CPUID.RTM and CPUID.HLE bits. Clear them
-	 * here.
+	 * Hardware will always abort a TSX transaction when CPUID
+	 * RTM_ALWAYS_ABORT is set. In this case, it is better not to enumerate
+	 * CPUID.RTM and CPUID.HLE bits. Clear them here.
 	 */
-	if (boot_cpu_has(X86_FEATURE_RTM_ALWAYS_ABORT) &&
-	    boot_cpu_has(X86_FEATURE_TSX_FORCE_ABORT)) {
+	if (boot_cpu_has(X86_FEATURE_RTM_ALWAYS_ABORT)) {
 		tsx_ctrl_state = TSX_CTRL_RTM_ALWAYS_ABORT;
 		tsx_clear_cpuid();
 		setup_clear_cpu_cap(X86_FEATURE_RTM);
-- 
2.25.1

[PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

[Pawan Gupta]

A microcode update on some Intel processors causes all TSX transactions
to always abort by default [*]. Microcode also added functionality to
re-enable TSX for development purpose. With this microcode loaded, if
tsx=on was passed on the cmdline, and TSX development mode was already
enabled before the kernel boot, it may make the system vulnerable to TSX
Asynchronous Abort (TAA).

To be on safer side, unconditionally disable TSX development mode at
boot. If needed, a user can enable it using msr-tools.

[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
    https://cdrdv2.intel.com/v1/dl/getContent/643557

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Suggested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Cc: <stable@vger.kernel.org>
---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/cpu.h              |  1 +
 arch/x86/kernel/cpu/intel.c            |  4 +++
 arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 5 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index a4a39c3e0f19..0c2610cde6ea 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175
diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h
index ee6f23f7587d..628d18062372 100644
--- a/arch/x86/kernel/cpu/cpu.h
+++ b/arch/x86/kernel/cpu/cpu.h
@@ -58,6 +58,7 @@ extern void __init tsx_init(void);
 extern void tsx_enable(void);
 extern void tsx_disable(void);
 extern void tsx_clear_cpuid(void);
+extern bool tsx_dev_mode_disable(void);
 #else
 static inline void tsx_init(void) { }
 #endif /* CONFIG_CPU_SUP_INTEL */
diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8abf995677a4..46cb5a18bd97 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -717,6 +717,10 @@ static void init_intel(struct cpuinfo_x86 *c)
 
 	init_intel_misc_features(c);
 
+	/* Boot CPU is handled in tsx_init() */
+	if (c->cpu_index != boot_cpu_data.cpu_index)
+		tsx_dev_mode_disable();
+
 	if (tsx_ctrl_state == TSX_CTRL_ENABLE)
 		tsx_enable();
 	else if (tsx_ctrl_state == TSX_CTRL_DISABLE)
diff --git a/arch/x86/kernel/cpu/tsx.c b/arch/x86/kernel/cpu/tsx.c
index 2835fa89fc6f..513e479bca2e 100644
--- a/arch/x86/kernel/cpu/tsx.c
+++ b/arch/x86/kernel/cpu/tsx.c
@@ -142,11 +142,45 @@ void tsx_clear_cpuid(void)
 	}
 }
 
+/*
+ * Disable TSX development mode
+ *
+ * When the microcode released in Feb 2022 is applied, TSX will be disabled by
+ * default on some processors. MSR 0x122 (TSX_CTRL) and MSR 0x123
+ * (IA32_MCU_OPT_CTRL) can be used to re-enable TSX for development, doing so is
+ * not recommended for production deployments. In particular, applying MD_CLEAR
+ * flows for mitigation of the Intel TSX Asynchronous Abort (TAA) transient
+ * execution attack may not be effective on these processors when Intel TSX is
+ * enabled with updated microcode.
+ */
+bool tsx_dev_mode_disable(void)
+{
+	u64 mcu_opt_ctrl;
+
+	/* Check if RTM_ALLOW exists */
+	if (!boot_cpu_has_bug(X86_BUG_TAA) || !tsx_ctrl_is_supported() ||
+	    !boot_cpu_has(X86_FEATURE_SRBDS_CTRL))
+		return false;
+
+	rdmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_opt_ctrl);
+
+	if (mcu_opt_ctrl & RTM_ALLOW) {
+		mcu_opt_ctrl &= ~RTM_ALLOW;
+		wrmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_opt_ctrl);
+		return true;
+	}
+
+	return false;
+}
+
 void __init tsx_init(void)
 {
 	char arg[5] = {};
 	int ret;
 
+	if (tsx_dev_mode_disable())
+		setup_force_cpu_cap(X86_FEATURE_RTM_ALWAYS_ABORT);
+
 	/*
 	 * Hardware will always abort a TSX transaction when CPUID
 	 * RTM_ALWAYS_ABORT is set. In this case, it is better not to enumerate
diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h
index a4a39c3e0f19..0c2610cde6ea 100644
--- a/tools/arch/x86/include/asm/msr-index.h
+++ b/tools/arch/x86/include/asm/msr-index.h
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175
-- 
2.25.1

Re: [PATCH v2 0/2] TSX update

[Pawan Gupta]

Hi Boris and others,

On Thu, Mar 10, 2022 at 01:59:47PM -0800, Pawan Gupta wrote:
>v2:
>- Added patch to disable TSX development mode (Andrew, Boris)
>- Rebased to v5.17-rc7
>
>v1: https://lore.kernel.org/lkml/5bd785a1d6ea0b572250add0c6617b4504bc24d1.1644440311.git.pawan.kumar.gupta@linux.intel.com/
>
>Hi,
>
>After a recent microcode update some Intel processors will always abort
>Transactional Synchronization Extensions (TSX) transactions [*]. On
>these processors a new CPUID bit,
>CPUID.07H.0H.EDX[11](RTM_ALWAYS_ABORT), will be enumerated to indicate
>that the loaded microcode is forcing Restricted Transactional Memory
>(RTM) abort. If the processor enumerated support for RTM previously, the
>CPUID enumeration bits for TSX (CPUID.RTM and CPUID.HLE) continue to be
>set by default after the microcode update.
>
>First patch in this series clears CPUID.RTM and CPUID.HLE so that
>userspace doesn't enumerate TSX feature.
>
>Microcode also added support to re-enable TSX for development purpose,
>doing so is not recommended for production deployments, because MD_CLEAR
>flows for the mitigation of TSX Asynchronous Abort (TAA) may not be
>effective on these processors when TSX is enabled with updated
>microcode.
>
>Second patch unconditionally disables this TSX development mode, in case
>it was enabled by the software running before kernel boot.
>
>Thanks,
>Pawan
>
>[*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
>    https://cdrdv2.intel.com/v1/dl/getContent/643557
>
>Pawan Gupta (2):
>  x86/tsx: Use MSR_TSX_CTRL to clear CPUID bits
>  x86/tsx: Disable TSX development mode at boot

I am hoping to get some feedback on v2. Are these patches looking okay?

Thanks,
Pawan

Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

[Borislav Petkov]

On Thu, Mar 10, 2022 at 02:02:09PM -0800, Pawan Gupta wrote:
> A microcode update on some Intel processors causes all TSX transactions
> to always abort by default [*]. Microcode also added functionality to
> re-enable TSX for development purpose. With this microcode loaded, if
> tsx=on was passed on the cmdline, and TSX development mode was already
> enabled before the kernel boot, it may make the system vulnerable to TSX
> Asynchronous Abort (TAA).
> 
> To be on safer side, unconditionally disable TSX development mode at
> boot. If needed, a user can enable it using msr-tools.
> 
> [*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
>     https://cdrdv2.intel.com/v1/dl/getContent/643557
> 
> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Suggested-by: Borislav Petkov <bp@alien8.de>
> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
> Cc: <stable@vger.kernel.org>
> ---
>  arch/x86/include/asm/msr-index.h       |  4 +--
>  arch/x86/kernel/cpu/cpu.h              |  1 +
>  arch/x86/kernel/cpu/intel.c            |  4 +++
>  arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
>  tools/arch/x86/include/asm/msr-index.h |  4 +--
>  5 files changed, 43 insertions(+), 4 deletions(-)

Does this a lot more encapsulated version work too?

---
From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Date: Thu, 10 Mar 2022 14:02:09 -0800
Subject: [PATCH] x86/tsx: Disable TSX development mode at boot

A microcode update on some Intel processors causes all TSX transactions
to always abort by default[*]. Microcode also added functionality to
re-enable TSX for development purposes. With this microcode loaded, if
tsx=on was passed on the cmdline, and TSX development mode was already
enabled before the kernel boot, it may make the system vulnerable to TSX
Asynchronous Abort (TAA).

To be on safer side, unconditionally disable TSX development mode during
boot. If a viable use case appears, this can be revisited later.

  [*]: Intel TSX Disable Update for Selected Processors, doc ID: 643557

  [ bp: Drop unstable web link, massage heavily. ]

Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
Suggested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: <stable@vger.kernel.org>
Link: https://lore.kernel.org/r/347bd844da3a333a9793c6687d4e4eb3b2419a3e.1646943780.git.pawan.kumar.gupta@linux.intel.com
---
 arch/x86/include/asm/msr-index.h       |  4 +--
 arch/x86/kernel/cpu/common.c           |  2 ++
 arch/x86/kernel/cpu/cpu.h              |  5 ++-
 arch/x86/kernel/cpu/intel.c            |  8 -----
 arch/x86/kernel/cpu/tsx.c              | 50 ++++++++++++++++++++++++--
 tools/arch/x86/include/asm/msr-index.h |  4 +--
 6 files changed, 55 insertions(+), 18 deletions(-)

diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h
index 9f1741ac4769..adce6a17770b 100644
--- a/arch/x86/include/asm/msr-index.h
+++ b/arch/x86/include/asm/msr-index.h
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175
diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c
index 64deb7727d00..4616753d1510 100644
--- a/arch/x86/kernel/cpu/common.c
+++ b/arch/x86/kernel/cpu/common.c
@@ -1798,6 +1798,8 @@ void identify_secondary_cpu(struct cpuinfo_x86 *c)
 	validate_apic_and_package_id(c);
 	x86_spec_ctrl_setup_ap();
 	update_srbds_msr();
+
+	tsx_ap_init();
 }
 
 static __init int setup_noclflush(char *arg)
diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h
index ee6f23f7587d..2a8e584fc991 100644
--- a/arch/x86/kernel/cpu/cpu.h
+++ b/arch/x86/kernel/cpu/cpu.h
@@ -55,11 +55,10 @@ enum tsx_ctrl_states {
 extern __ro_after_init enum tsx_ctrl_states tsx_ctrl_state;
 
 extern void __init tsx_init(void);
-extern void tsx_enable(void);
-extern void tsx_disable(void);
-extern void tsx_clear_cpuid(void);
+void tsx_ap_init(void);
 #else
 static inline void tsx_init(void) { }
+static inline void tsx_ap_init(void) { }
 #endif /* CONFIG_CPU_SUP_INTEL */
 
 extern void get_cpu_cap(struct cpuinfo_x86 *c);
diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8abf995677a4..f7a5370a9b3b 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -717,14 +717,6 @@ static void init_intel(struct cpuinfo_x86 *c)
 
 	init_intel_misc_features(c);
 
-	if (tsx_ctrl_state == TSX_CTRL_ENABLE)
-		tsx_enable();
-	else if (tsx_ctrl_state == TSX_CTRL_DISABLE)
-		tsx_disable();
-	else if (tsx_ctrl_state == TSX_CTRL_RTM_ALWAYS_ABORT)
-		/* See comment over that function for more details. */
-		tsx_clear_cpuid();
-
 	split_lock_init();
 	bus_lock_init();
 
diff --git a/arch/x86/kernel/cpu/tsx.c b/arch/x86/kernel/cpu/tsx.c
index ec6ff8000920..ec7bbac3a9f2 100644
--- a/arch/x86/kernel/cpu/tsx.c
+++ b/arch/x86/kernel/cpu/tsx.c
@@ -19,7 +19,7 @@
 
 enum tsx_ctrl_states tsx_ctrl_state __ro_after_init = TSX_CTRL_NOT_SUPPORTED;
 
-void tsx_disable(void)
+static void tsx_disable(void)
 {
 	u64 tsx;
 
@@ -39,7 +39,7 @@ void tsx_disable(void)
 	wrmsrl(MSR_IA32_TSX_CTRL, tsx);
 }
 
-void tsx_enable(void)
+static void tsx_enable(void)
 {
 	u64 tsx;
 
@@ -122,7 +122,7 @@ static enum tsx_ctrl_states x86_get_tsx_auto_mode(void)
  * That's why, this function's call in init_intel() doesn't clear the
  * feature flags.
  */
-void tsx_clear_cpuid(void)
+static void tsx_clear_cpuid(void)
 {
 	u64 msr;
 
@@ -142,11 +142,42 @@ void tsx_clear_cpuid(void)
 	}
 }
 
+/*
+ * Disable TSX development mode
+ *
+ * When the microcode released in Feb 2022 is applied, TSX will be disabled by
+ * default on some processors. MSR 0x122 (TSX_CTRL) and MSR 0x123
+ * (IA32_MCU_OPT_CTRL) can be used to re-enable TSX for development, doing so is
+ * not recommended for production deployments. In particular, applying MD_CLEAR
+ * flows for mitigation of the Intel TSX Asynchronous Abort (TAA) transient
+ * execution attack may not be effective on these processors when Intel TSX is
+ * enabled with updated microcode.
+ */
+static void tsx_dev_mode_disable(void)
+{
+	u64 mcu_opt_ctrl;
+
+	/* Check if RTM_ALLOW exists */
+	if (!boot_cpu_has_bug(X86_BUG_TAA) || !tsx_ctrl_is_supported() ||
+	    !cpu_feature_enabled(X86_FEATURE_SRBDS_CTRL))
+		return;
+
+	rdmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_opt_ctrl);
+
+	if (mcu_opt_ctrl & RTM_ALLOW) {
+		mcu_opt_ctrl &= ~RTM_ALLOW;
+		wrmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_opt_ctrl);
+		setup_force_cpu_cap(X86_FEATURE_RTM_ALWAYS_ABORT);
+	}
+}
+
 void __init tsx_init(void)
 {
 	char arg[5] = {};
 	int ret;
 
+	tsx_dev_mode_disable();
+
 	/*
 	 * Hardware will always abort a TSX transaction when the CPUID bit
 	 * RTM_ALWAYS_ABORT is set. In this case, it is better not to enumerate
@@ -215,3 +246,16 @@ void __init tsx_init(void)
 		setup_force_cpu_cap(X86_FEATURE_HLE);
 	}
 }
+
+void tsx_ap_init(void)
+{
+	tsx_dev_mode_disable();
+
+	if (tsx_ctrl_state == TSX_CTRL_ENABLE)
+		tsx_enable();
+	else if (tsx_ctrl_state == TSX_CTRL_DISABLE)
+		tsx_disable();
+	else if (tsx_ctrl_state == TSX_CTRL_RTM_ALWAYS_ABORT)
+		/* See comment over that function for more details. */
+		tsx_clear_cpuid();
+}
diff --git a/tools/arch/x86/include/asm/msr-index.h b/tools/arch/x86/include/asm/msr-index.h
index a4a39c3e0f19..0c2610cde6ea 100644
--- a/tools/arch/x86/include/asm/msr-index.h
+++ b/tools/arch/x86/include/asm/msr-index.h
@@ -128,9 +128,9 @@
 #define TSX_CTRL_RTM_DISABLE		BIT(0)	/* Disable RTM feature */
 #define TSX_CTRL_CPUID_CLEAR		BIT(1)	/* Disable TSX enumeration */
 
-/* SRBDS support */
 #define MSR_IA32_MCU_OPT_CTRL		0x00000123
-#define RNGDS_MITG_DIS			BIT(0)
+#define RNGDS_MITG_DIS			BIT(0)	/* SRBDS support */
+#define RTM_ALLOW			BIT(1)	/* TSX development mode */
 
 #define MSR_IA32_SYSENTER_CS		0x00000174
 #define MSR_IA32_SYSENTER_ESP		0x00000175
-- 
2.35.1

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette

Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

[Pawan Gupta]

On Tue, Mar 29, 2022 at 06:24:03PM +0200, Borislav Petkov wrote:
>On Thu, Mar 10, 2022 at 02:02:09PM -0800, Pawan Gupta wrote:
>> A microcode update on some Intel processors causes all TSX transactions
>> to always abort by default [*]. Microcode also added functionality to
>> re-enable TSX for development purpose. With this microcode loaded, if
>> tsx=on was passed on the cmdline, and TSX development mode was already
>> enabled before the kernel boot, it may make the system vulnerable to TSX
>> Asynchronous Abort (TAA).
>>
>> To be on safer side, unconditionally disable TSX development mode at
>> boot. If needed, a user can enable it using msr-tools.
>>
>> [*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
>>     https://cdrdv2.intel.com/v1/dl/getContent/643557
>>
>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Suggested-by: Borislav Petkov <bp@alien8.de>
>> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
>> Cc: <stable@vger.kernel.org>
>> ---
>>  arch/x86/include/asm/msr-index.h       |  4 +--
>>  arch/x86/kernel/cpu/cpu.h              |  1 +
>>  arch/x86/kernel/cpu/intel.c            |  4 +++
>>  arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
>>  tools/arch/x86/include/asm/msr-index.h |  4 +--
>>  5 files changed, 43 insertions(+), 4 deletions(-)
>
>Does this a lot more encapsulated version work too?

It look good to me.

Thanks,
Pawan

Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

[Pawan Gupta]

On Tue, Mar 29, 2022 at 06:24:03PM +0200, Borislav Petkov wrote:
>On Thu, Mar 10, 2022 at 02:02:09PM -0800, Pawan Gupta wrote:
>> A microcode update on some Intel processors causes all TSX transactions
>> to always abort by default [*]. Microcode also added functionality to
>> re-enable TSX for development purpose. With this microcode loaded, if
>> tsx=on was passed on the cmdline, and TSX development mode was already
>> enabled before the kernel boot, it may make the system vulnerable to TSX
>> Asynchronous Abort (TAA).
>>
>> To be on safer side, unconditionally disable TSX development mode at
>> boot. If needed, a user can enable it using msr-tools.
>>
>> [*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
>>     https://cdrdv2.intel.com/v1/dl/getContent/643557
>>
>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Suggested-by: Borislav Petkov <bp@alien8.de>
>> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
>> Cc: <stable@vger.kernel.org>
>> ---
>>  arch/x86/include/asm/msr-index.h       |  4 +--
>>  arch/x86/kernel/cpu/cpu.h              |  1 +
>>  arch/x86/kernel/cpu/intel.c            |  4 +++
>>  arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
>>  tools/arch/x86/include/asm/msr-index.h |  4 +--
>>  5 files changed, 43 insertions(+), 4 deletions(-)
>
>Does this a lot more encapsulated version work too?

Neelima is testing this patch, she will share the results tomorrow.

Thanks,
Pawan

RE: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

[Krishnan, Neelima]

-----Original Message-----
From: Pawan Gupta <pawan.kumar.gupta@linux.intel.com> 
Sent: Tuesday, March 29, 2022 10:28 PM
To: Borislav Petkov <bp@alien8.de>
Cc: Thomas Gleixner <tglx@linutronix.de>; Ingo Molnar <mingo@redhat.com>; Dave Hansen <dave.hansen@linux.intel.com>; x86@kernel.org; H. Peter Anvin <hpa@zytor.com>; Andi Kleen <ak@linux.intel.com>; Luck, Tony <tony.luck@intel.com>; linux-kernel@vger.kernel.org; antonio.gomez.iglesias@linux.intel.com; Krishnan, Neelima <neelima.krishnan@intel.com>; stable@vger.kernel.org; Cooper, Andrew <andrew.cooper3@citrix.com>; Poimboe, Josh <jpoimboe@redhat.com>
Subject: Re: [PATCH v2 2/2] x86/tsx: Disable TSX development mode at boot

On Tue, Mar 29, 2022 at 06:24:03PM +0200, Borislav Petkov wrote:
>On Thu, Mar 10, 2022 at 02:02:09PM -0800, Pawan Gupta wrote:
>> A microcode update on some Intel processors causes all TSX 
>> transactions to always abort by default [*]. Microcode also added 
>> functionality to re-enable TSX for development purpose. With this 
>> microcode loaded, if tsx=on was passed on the cmdline, and TSX 
>> development mode was already enabled before the kernel boot, it may 
>> make the system vulnerable to TSX Asynchronous Abort (TAA).
>>
>> To be on safer side, unconditionally disable TSX development mode at 
>> boot. If needed, a user can enable it using msr-tools.
>>
>> [*] Intel Transactional Synchronization Extension (Intel TSX) Disable Update for Selected Processors
>>     https://cdrdv2.intel.com/v1/dl/getContent/643557
>>
>> Suggested-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Suggested-by: Borislav Petkov <bp@alien8.de>
>> Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
>> Cc: <stable@vger.kernel.org>
>> ---
>>  arch/x86/include/asm/msr-index.h       |  4 +--
>>  arch/x86/kernel/cpu/cpu.h              |  1 +
>>  arch/x86/kernel/cpu/intel.c            |  4 +++
>>  arch/x86/kernel/cpu/tsx.c              | 34 ++++++++++++++++++++++++++
>>  tools/arch/x86/include/asm/msr-index.h |  4 +--
>>  5 files changed, 43 insertions(+), 4 deletions(-)
>
>Does this a lot more encapsulated version work too?

>Neelima is testing this patch, she will share the results tomorrow.

Following up on this email thread, I did some basic functional validation of the patch[1]. 
Initially I ran into the bug where the mitigation was getting disabled in one CPU after a suspend/resume [2]. 
But after applying the patch [1] on latest upstream, with the fix for restoring speculation related MSRs during s3 resume [2], my tests are passing. 

Quick summary of testcases executed:
Testcase 1:  Verify RTM_ALLOW was getting reset after kexec reboot
Testcase2: Verify TSX_CTRL_MSR is restored after system goes to S3 suspend state

[1] https://lore.kernel.org/lkml/YkMyo2Jw8iYx9wAU@zn.tnic/
[2] https://github.com/torvalds/linux/commit/e2a1256b17b16f9b9adf1b6fea56819e7b68e463

Thanks
Neelima