Re: [patch V10 00/10] Control knobs and Documentation 0

[Thomas Gleixner <tglx@linutronix.de>]

On Thu, 12 Jul 2018, speck for Josh Poimboeuf wrote:
> On Thu, Jul 12, 2018 at 04:19:02PM +0200, speck for Thomas Gleixner wrote:
> > Git bundle follows in separate mail.
> 
> It looks great to me (other than the few minor issues I already pointed
> out).
> 
> Reviewed-by: Josh Poimboeuf <jpoimboe@redhat.com>

Thanks Josh! The delta patch against the V10 lot with the last fixups is
below.

Thanks,

	tglx
8<-------------
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -1993,7 +1993,7 @@
 				Provides all available mitigations for the
 				L1TF vulnerability. Disables SMT and
 				enables all mitigations in the
-				hypervisors.
+				hypervisors, i.e. unconditional L1D flush.
 
 				SMT control and L1D flush control via the
 				sysfs interface is still possible after
@@ -2010,7 +2010,8 @@
 
 			flush
 				Leaves SMT enabled and enables the default
-				hypervisor mitigation.
+				hypervisor mitigation, i.e. conditional
+				L1D flush.
 
 				SMT control and L1D flush control via the
 				sysfs interface is still possible after
@@ -2042,6 +2043,8 @@
 
 			Default is 'flush'.
 
+			For details see: Documentation/admin-guide/l1tf.rst
+
 	l2cr=		[PPC]
 
 	l3cr=		[PPC]
--- a/Documentation/admin-guide/l1tf.rst
+++ b/Documentation/admin-guide/l1tf.rst
@@ -146,13 +146,13 @@ If KVM/VMX is enabled and the processor
 
   - L1D Flush mode:
 
-    ================================  ===================================
+    ================================  ====================================
     'L1D vulnerable'		      L1D flushing is disabled
 
     'L1D conditional cache flushes'   L1D flush is conditionally enabled
 
-    'L1D cache flushes'		      SMT is disabled and L1D flush
-    ================================  ===================================
+    'L1D cache flushes'		      L1D flush is unconditionally enabled
+    ================================  ====================================
 
 The resulting grade of protection is discussed in the following sections.
 
@@ -167,6 +167,8 @@ user space running on the host.
 Guest mitigation mechanisms
 ---------------------------
 
+.. _l1d_flush:
+
 1. L1D flush on VMENTER
 ^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -184,9 +186,25 @@ Guest mitigation mechanisms
    confine the VMEXITs to a bare minimum, but specific configurations and
    application scenarios might still suffer from a high VMEXIT rate.
 
-   The general recommendation is to enable L1D flush on VMENTER.
+   The kernel provides two L1D flush modes:
+    - conditional ('cond')
+    - unconditional ('always')
+
+   The conditional mode avoids L1D flushing after VMEXITs which execute
+   only audited code pathes before the corresponding VMENTER. These code
+   pathes have beed verified that they cannot expose secrets or other
+   interesting data to an attacker, but they can leak information about the
+   address space layout of the hypervisor.
+
+   Unconditional mode flushes L1D on all VMENTER invocations and provides
+   maximum protection. It has a higher overhead than the conditional
+   mode. The overhead cannot be quantified correctly as it depends on the
+   work load scenario and the resulting number of VMEXITs.
+
+   The general recommendation is to enable L1D flush on VMENTER. The kernel
+   defaults to conditional mode on affected processors.
 
-   Note, that L1D flush does not prevent the SMT problem because the
+   **Note**, that L1D flush does not prevent the SMT problem because the
    sibling thread will also bring back its data into the L1D which makes it
    attackable again.
 
@@ -216,10 +234,11 @@ Guest mitigation mechanisms
    declared as non-interesting for an attacker without deep inspection of
    the code.
 
-   Note, that assigning guests to a fixed set of physical cores affects the
-   ability of the scheduler to do load balancing and might have negative
-   effects on CPU utilization depending on the hosting scenario. Disabling
-   SMT might be a viable alternative for particular scenarios.
+   **Note**, that assigning guests to a fixed set of physical cores affects
+   the ability of the scheduler to do load balancing and might have
+   negative effects on CPU utilization depending on the hosting
+   scenario. Disabling SMT might be a viable alternative for particular
+   scenarios.
 
    For further information about confining guests to a single or to a group
    of cores consult the cpusets documentation:
@@ -276,7 +295,7 @@ Guest mitigation mechanisms
 		 boot process. "nosmt" makes sure that from each physical
 		 core only one - the so called primary (hyper) thread is
 		 activated. Due to a design flaw of Intel processors related
-	 	 to Machine Check Exceptions the non primary siblings have
+		 to Machine Check Exceptions the non primary siblings have
 		 to be brought up at least partially and are then shut down
 		 again.  "nosmt" can be undone via the sysfs interface.
 
@@ -350,10 +369,10 @@ Mitigation control on the kernel command
 The kernel command line allows to control the L1TF mitigations at boot
 time with the option "l1tf=". The valid arguments for this option are:
 
-  ============  ===================================================
+  ============  =============================================================
   full		Provides all available mitigations for the L1TF
 		vulnerability. Disables SMT and enables all mitigations in
-		the hypervisors.
+		the hypervisors, i.e. unconditional L1D flushing
 
 		SMT control and L1D flush control via the sysfs interface
 		is still possible after boot.  Hypervisors will issue a
@@ -366,7 +385,7 @@ The kernel command line allows to contro
 		(i.e. sysfs control of SMT is disabled.)
 
   flush		Leaves SMT enabled and enables the default hypervisor
-		mitigation.
+		mitigation, i.e. conditional L1D flushing
 
 		SMT control and L1D flush control via the sysfs interface
 		is still possible after boot.  Hypervisors will issue a
@@ -374,7 +393,8 @@ The kernel command line allows to contro
 		insecure configuration, i.e. SMT enabled or L1D flush
 		disabled.
 
-  flush,nosmt	Disables SMT and enables the default hypervisor mitigation.
+  flush,nosmt	Disables SMT and enables the default hypervisor mitigation,
+		i.e. conditional L1D flushing.
 
 		SMT control and L1D flush control via the sysfs interface
 		is still possible after boot.  Hypervisors will issue a
@@ -387,9 +407,9 @@ The kernel command line allows to contro
 
   off		Disables hypervisor mitigations and doesn't emit any
 		warnings.
-  ============  ===================================================
+  ============  =============================================================
 
-The default is 'flush'.
+The default is 'flush'. For details about L1D flushing see :ref:`l1d_flush`.
 
 
 .. _mitigation_control_kvm:
@@ -418,7 +438,7 @@ The parameter can be provided on the ker
 parameter when loading the modules and at runtime modified via the sysfs
 file:
 
- /sys/module/kvm_intel/parameters/vmentry_l1d_flush
+/sys/module/kvm_intel/parameters/vmentry_l1d_flush
 
 The default is 'cond'. If 'l1tf=full,force' is given on the kernel command
 line, then 'always' is enforced and the kvm-intel.vmentry_l1d_flush
@@ -569,4 +589,3 @@ Default mitigations
     machines and also result in other changes of their overall setup.
     There is no way for the kernel to provide a sensible default for this
     kind of scenarios.
-
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -59,6 +59,12 @@ void __init check_bugs(void)
 {
 	identify_boot_cpu();
 
+	/*
+	 * identify_boot_cpu() initialized SMT support information, let the
+	 * core code know.
+	 */
+	cpu_smt_check_topology();
+
 	if (!IS_ENABLED(CONFIG_SMP)) {
 		pr_info("CPU: ");
 		print_cpu_info(&boot_cpu_data);
@@ -742,11 +748,11 @@ early_param("l1tf", l1tf_cmdline);
 
 #if IS_ENABLED(CONFIG_KVM_INTEL)
 static const char *l1tf_vmx_states[] = {
-	[VMENTER_L1D_FLUSH_AUTO]	= "auto",
-	[VMENTER_L1D_FLUSH_NEVER]	= "vulnerable",
-	[VMENTER_L1D_FLUSH_COND]	= "conditional cache flushes",
-	[VMENTER_L1D_FLUSH_ALWAYS]	= "cache flushes",
-	[VMENTER_L1D_FLUSH_EPT_DISABLED]= "EPT disabled"
+	[VMENTER_L1D_FLUSH_AUTO]		= "auto",
+	[VMENTER_L1D_FLUSH_NEVER]		= "vulnerable",
+	[VMENTER_L1D_FLUSH_COND]		= "conditional cache flushes",
+	[VMENTER_L1D_FLUSH_ALWAYS]		= "cache flushes",
+	[VMENTER_L1D_FLUSH_EPT_DISABLED]	= "EPT disabled",
 };
 
 static ssize_t l1tf_show_state(char *buf)
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -178,9 +178,11 @@ enum cpuhp_smt_control {
 #if defined(CONFIG_SMP) && defined(CONFIG_HOTPLUG_SMT)
 extern enum cpuhp_smt_control cpu_smt_control;
 extern void cpu_smt_disable(bool force);
+extern void cpu_smt_check_topology(void);
 #else
 # define cpu_smt_control		(CPU_SMT_ENABLED)
 static inline void cpu_smt_disable(bool force) { }
+static inline void cpu_smt_check_topology(void) { }
 #endif
 
 #endif /* _LINUX_CPU_H_ */
--- a/kernel/cpu.c
+++ b/kernel/cpu.c
@@ -349,16 +349,28 @@ EXPORT_SYMBOL_GPL(cpu_smt_control);
 
 void __init cpu_smt_disable(bool force)
 {
-	if (cpu_smt_control == CPU_SMT_FORCE_DISABLED)
+	if (cpu_smt_control == CPU_SMT_FORCE_DISABLED ||
+		cpu_smt_control == CPU_SMT_NOT_SUPPORTED)
 		return;
 
-	cpu_smt_control = CPU_SMT_DISABLED;
 	if (force) {
 		pr_info("SMT: Force disabled\n");
 		cpu_smt_control = CPU_SMT_FORCE_DISABLED;
+	} else {
+		cpu_smt_control = CPU_SMT_DISABLED;
 	}
 }
 
+/*
+ * The decision whether SMT is supported can only be done after the full
+ * CPU identification. Called from architecture code.
+ */
+void __init cpu_smt_check_topology(void)
+{
+	if (!topology_smt_supported())
+		cpu_smt_control = CPU_SMT_NOT_SUPPORTED;
+}
+
 static int __init smt_cmdline_disable(char *str)
 {
 	cpu_smt_disable(str && !strcmp(str, "force"));
@@ -2113,9 +2125,6 @@ static const struct attribute_group cpuh
 
 static int __init cpu_smt_state_init(void)
 {
-	if (!topology_smt_supported())
-		cpu_smt_control = CPU_SMT_NOT_SUPPORTED;
-
 	return sysfs_create_group(&cpu_subsys.dev_root->kobj,
 				  &cpuhp_smt_attr_group);
 }
--- a/Documentation/ABI/testing/sysfs-devices-system-cpu
+++ b/Documentation/ABI/testing/sysfs-devices-system-cpu
@@ -476,6 +476,7 @@ What:		/sys/devices/system/cpu/vulnerabi
 		/sys/devices/system/cpu/vulnerabilities/spectre_v1
 		/sys/devices/system/cpu/vulnerabilities/spectre_v2
 		/sys/devices/system/cpu/vulnerabilities/spec_store_bypass
+		/sys/devices/system/cpu/vulnerabilities/l1tf
 Date:		January 2018
 Contact:	Linux kernel mailing list <linux-kernel@vger.kernel.org>
 Description:	Information about CPU vulnerabilities
@@ -488,6 +489,9 @@ Description:	Information about CPU vulne
 		"Vulnerable"	  CPU is affected and no mitigation in effect
 		"Mitigation: $M"  CPU is affected and mitigation $M is in effect
 
+		Details about the l1tf file can be found in
+		Documentation/admin-guide/l1tf.rst
+
 What:		/sys/devices/system/cpu/smt
 		/sys/devices/system/cpu/smt/active
 		/sys/devices/system/cpu/smt/control