From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail.linutronix.de (193.142.43.55:993) by crypto-ml.lab.linutronix.de with IMAP4-SSL for ; 06 Apr 2020 18:34:27 -0000 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by Galois.linutronix.de with esmtps (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.80) (envelope-from ) id 1jLWZm-0007aX-I9 for speck@linutronix.de; Mon, 06 Apr 2020 20:34:26 +0200 Received: by mail-pl1-x62c.google.com with SMTP id x1so170823plm.4 for ; Mon, 06 Apr 2020 11:34:26 -0700 (PDT) Received: from www.outflux.net (smtp.outflux.net. [198.145.64.163]) by smtp.gmail.com with ESMTPSA id 184sm11479978pge.71.2020.04.06.11.34.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 06 Apr 2020 11:34:18 -0700 (PDT) Date: Mon, 6 Apr 2020 11:34:17 -0700 From: Kees Cook Subject: [MODERATED] Re: [PATCH 2/3] V5 more sampling fun 2 Message-ID: <202004061133.92517B7@keescook> References: <5e8b71d8.1c69fb81.64075.43abSMTPIN_ADDED_BROKEN@mx.google.com> MIME-Version: 1.0 In-Reply-To: <5e8b71d8.1c69fb81.64075.43abSMTPIN_ADDED_BROKEN@mx.google.com> Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit To: speck@linutronix.de List-ID: On Thu, Jan 16, 2020 at 02:16:07PM -0800, speck for mark gross wrote: > From: mark gross > Subject: [PATCH 2/3] x86/speculation: Special Register Buffer Data Sampling > (SRBDS) mitigation control. > > SRBDS is an MDS-like speculative side channel that can leak bits from > the RNG across cores and threads. New microcode serializes the processor > access during the execution of RDRAND and RDSEED. This ensures that the > shared buffer is overwritten before it is released for reuse. > > While it is present on all affected CPU models, the microcode mitigation > is not needed on models that enumerate ARCH_CAPABILITIES[MDS_NO] in the > cases where TSX is not supported or has been disabled with TSX_CTRL. > > The mitigation is activated by default on affected processors and it > increases latency for RDRAND and RDSEED instructions. Among other > effects this will reduce throughput from /dev/urandom. > > * enable administrator to configure the mitigation off when desired > using either mitigations=off or srbds=off. > * export vulnerability status via sysfs > * rename file scoped macros to apply for non-whitelist table > initializations. > > Signed-off-by: Mark Gross > Reviewed-by: Tony Luck > Reviewed-by: Pawan Gupta > Tested-by: Neelima Krishnan > --- > .../ABI/testing/sysfs-devices-system-cpu | 1 + > .../admin-guide/kernel-parameters.txt | 20 +++ > arch/x86/include/asm/cpufeatures.h | 2 + > arch/x86/include/asm/msr-index.h | 4 + > arch/x86/kernel/cpu/bugs.c | 123 ++++++++++++++++++ > arch/x86/kernel/cpu/common.c | 61 +++++++-- > arch/x86/kernel/cpu/cpu.h | 3 + > drivers/base/cpu.c | 8 ++ > 8 files changed, 210 insertions(+), 12 deletions(-) > > diff --git a/Documentation/ABI/testing/sysfs-devices-system-cpu b/Documentation/ABI/testing/sysfs-devices-system-cpu > index 2e0e3b45d02a..b39531a3c5bc 100644 > --- a/Documentation/ABI/testing/sysfs-devices-system-cpu > +++ b/Documentation/ABI/testing/sysfs-devices-system-cpu > @@ -492,6 +492,7 @@ What: /sys/devices/system/cpu/vulnerabilities > /sys/devices/system/cpu/vulnerabilities/spec_store_bypass > /sys/devices/system/cpu/vulnerabilities/l1tf > /sys/devices/system/cpu/vulnerabilities/mds > + /sys/devices/system/cpu/vulnerabilities/srbds > /sys/devices/system/cpu/vulnerabilities/tsx_async_abort > /sys/devices/system/cpu/vulnerabilities/itlb_multihit > Date: January 2018 > diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt > index 4d5a4fe22703..017bd682d2b9 100644 > --- a/Documentation/admin-guide/kernel-parameters.txt > +++ b/Documentation/admin-guide/kernel-parameters.txt > @@ -4739,6 +4739,26 @@ > the kernel will oops in either "warn" or "fatal" > mode. > > + srbds= [X86,INTEL] > + Control mitigation for the Special Register > + Buffer Data Sampling (SRBDS) mitigation. > + > + Certain CPUs are vulnerable to an MDS-like > + exploit which can leak bits from the random > + number generator. > + > + By default, this issue is mitigated by > + microcode. However, the microcode fix can cause > + the RDRAND and RDSEED instructions to become > + much slower. Among other effects, this will > + result in reduced throughput from /dev/urandom. This is this true about /dev/urandom? I thought the RDRAND dependency had been removed? -Kees > + > + The microcode mitigation can be disabled with > + the following option: > + > + off: Disable mitigation and remove > + performance impact to RDRAND and RDSEED > + > srcutree.counter_wrap_check [KNL] > Specifies how frequently to check for > grace-period sequence counter wrap for the > diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h > index db189945e9b0..02dabc9e77b0 100644 > --- a/arch/x86/include/asm/cpufeatures.h > +++ b/arch/x86/include/asm/cpufeatures.h > @@ -362,6 +362,7 @@ > #define X86_FEATURE_AVX512_4FMAPS (18*32+ 3) /* AVX-512 Multiply Accumulation Single precision */ > #define X86_FEATURE_FSRM (18*32+ 4) /* Fast Short Rep Mov */ > #define X86_FEATURE_AVX512_VP2INTERSECT (18*32+ 8) /* AVX-512 Intersect for D/Q */ > +#define X86_FEATURE_SRBDS_CTRL (18*32+ 9) /* "" SRBDS mitigation MSR available */ > #define X86_FEATURE_MD_CLEAR (18*32+10) /* VERW clears CPU buffers */ > #define X86_FEATURE_TSX_FORCE_ABORT (18*32+13) /* "" TSX_FORCE_ABORT */ > #define X86_FEATURE_PCONFIG (18*32+18) /* Intel PCONFIG */ > @@ -407,5 +408,6 @@ > #define X86_BUG_SWAPGS X86_BUG(21) /* CPU is affected by speculation through SWAPGS */ > #define X86_BUG_TAA X86_BUG(22) /* CPU is affected by TSX Async Abort(TAA) */ > #define X86_BUG_ITLB_MULTIHIT X86_BUG(23) /* CPU may incur MCE during certain page attribute changes */ > +#define X86_BUG_SRBDS X86_BUG(24) /* CPU may leak RNG bits if not mitigated */ > > #endif /* _ASM_X86_CPUFEATURES_H */ > diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h > index 12c9684d59ba..3efde600a674 100644 > --- a/arch/x86/include/asm/msr-index.h > +++ b/arch/x86/include/asm/msr-index.h > @@ -128,6 +128,10 @@ > #define TSX_CTRL_RTM_DISABLE BIT(0) /* Disable RTM feature */ > #define TSX_CTRL_CPUID_CLEAR BIT(1) /* Disable TSX enumeration */ > > +/* SRBDS support */ > +#define MSR_IA32_MCU_OPT_CTRL 0x00000123 > +#define RNGDS_MITG_DIS BIT(0) > + > #define MSR_IA32_SYSENTER_CS 0x00000174 > #define MSR_IA32_SYSENTER_ESP 0x00000175 > #define MSR_IA32_SYSENTER_EIP 0x00000176 > diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c > index ed54b3b21c39..51cc38d95a8f 100644 > --- a/arch/x86/kernel/cpu/bugs.c > +++ b/arch/x86/kernel/cpu/bugs.c > @@ -41,6 +41,7 @@ static void __init l1tf_select_mitigation(void); > static void __init mds_select_mitigation(void); > static void __init mds_print_mitigation(void); > static void __init taa_select_mitigation(void); > +static void __init srbds_select_mitigation(void); > > /* The base value of the SPEC_CTRL MSR that always has to be preserved. */ > u64 x86_spec_ctrl_base; > @@ -108,6 +109,7 @@ void __init check_bugs(void) > l1tf_select_mitigation(); > mds_select_mitigation(); > taa_select_mitigation(); > + srbds_select_mitigation(); > > /* > * As MDS and TAA mitigations are inter-related, print MDS > @@ -397,6 +399,114 @@ static int __init tsx_async_abort_parse_cmdline(char *str) > } > early_param("tsx_async_abort", tsx_async_abort_parse_cmdline); > > +#undef pr_fmt > +#define pr_fmt(fmt) "SRBDS: " fmt > + > +enum srbds_mitigations { > + SRBDS_MITIGATION_OFF, > + SRBDS_MITIGATION_UCODE_NEEDED, > + SRBDS_MITIGATION_FULL, > + SRBDS_NOT_AFFECTED_TSX_OFF, > + SRBDS_HYPERVISOR, > +}; > + > +enum srbds_mitigations srbds_mitigation __ro_after_init = SRBDS_MITIGATION_FULL; > +static const char * const srbds_strings[] = { > + [SRBDS_MITIGATION_OFF] = "Vulnerable", > + [SRBDS_MITIGATION_UCODE_NEEDED] = "Vulnerable: no microcode", > + [SRBDS_MITIGATION_FULL] = "Mitigated", > + [SRBDS_NOT_AFFECTED_TSX_OFF] = "Not affected (TSX disabled)", > + [SRBDS_HYPERVISOR] = "Unknown", > +}; > + > +static bool srbds_off; > + > +void update_srbds_msr(void) > +{ > + u64 mcu_ctrl; > + > + if (!boot_cpu_has_bug(X86_BUG_SRBDS)) > + return; > + > + if (boot_cpu_has(X86_FEATURE_HYPERVISOR)) > + return; > + > + if (srbds_mitigation == SRBDS_MITIGATION_UCODE_NEEDED) > + return; > + > + rdmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_ctrl); > + > + switch (srbds_mitigation) { > + case SRBDS_MITIGATION_OFF: > + case SRBDS_NOT_AFFECTED_TSX_OFF: > + mcu_ctrl |= RNGDS_MITG_DIS; > + break; > + case SRBDS_MITIGATION_FULL: > + mcu_ctrl &= ~RNGDS_MITG_DIS; > + break; > + default: > + break; > + } > + > + wrmsrl(MSR_IA32_MCU_OPT_CTRL, mcu_ctrl); > +} > + > +static void __init srbds_select_mitigation(void) > +{ > + u64 ia32_cap; > + > + /* > + * This test relies on the CPUID values of vendor, family, model, > + * stepping which might not reflect the real hardware when we are > + * running as a guest. But VMM vendors have asked that we do this > + * before the X86_FEATURE_HYPERVISOR test since this provides better > + * guidance to users in most real situations. > + */ > + > + if (!boot_cpu_has_bug(X86_BUG_SRBDS)) > + return; > + /* > + * Check to see if this is one of the MDS_NO systems supporting > + * TSX that are only exposed to SRBDS when TSX is enabled. > + */ > + ia32_cap = x86_read_arch_cap_msr(); > + if (ia32_cap & ARCH_CAP_MDS_NO) { > + if (!boot_cpu_has(X86_FEATURE_RTM)) > + srbds_mitigation = SRBDS_NOT_AFFECTED_TSX_OFF; > + } > + > + if (boot_cpu_has(X86_FEATURE_HYPERVISOR)) { > + if (srbds_mitigation != SRBDS_NOT_AFFECTED_TSX_OFF) > + srbds_mitigation = SRBDS_HYPERVISOR; > + return; > + } > + > + if (!boot_cpu_has(X86_FEATURE_SRBDS_CTRL)) { > + srbds_mitigation = SRBDS_MITIGATION_UCODE_NEEDED; > + return; > + } > + > + if (cpu_mitigations_off() || srbds_off) { > + if (srbds_mitigation != SRBDS_NOT_AFFECTED_TSX_OFF) > + srbds_mitigation = SRBDS_MITIGATION_OFF; > + } > + > + update_srbds_msr(); > + pr_info("%s\n", srbds_strings[srbds_mitigation]); > +} > + > +static int __init srbds_parse_cmdline(char *str) > +{ > + if (!str) > + return -EINVAL; > + > + if (!strcmp(str, "off")) > + srbds_off = true; > + > + return 0; > +} > +early_param("srbds", srbds_parse_cmdline); > + > #undef pr_fmt > #define pr_fmt(fmt) "Spectre V1 : " fmt > > @@ -1528,6 +1638,11 @@ static char *ibpb_state(void) > return ""; > } > > +static ssize_t srbds_show_state(char *buf) > +{ > + return sprintf(buf, "%s\n", srbds_strings[srbds_mitigation]); > +} > + > static ssize_t cpu_show_common(struct device *dev, struct device_attribute *attr, > char *buf, unsigned int bug) > { > @@ -1572,6 +1687,9 @@ static ssize_t cpu_show_common(struct device *dev, struct device_attribute *attr > case X86_BUG_ITLB_MULTIHIT: > return itlb_multihit_show_state(buf); > > + case X86_BUG_SRBDS: > + return srbds_show_state(buf); > + > default: > break; > } > @@ -1618,4 +1736,9 @@ ssize_t cpu_show_itlb_multihit(struct device *dev, struct device_attribute *attr > { > return cpu_show_common(dev, attr, buf, X86_BUG_ITLB_MULTIHIT); > } > + > +ssize_t cpu_show_srbds(struct device *dev, struct device_attribute *attr, char *buf) > +{ > + return cpu_show_common(dev, attr, buf, X86_BUG_SRBDS); > +} > #endif > diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c > index bed0cb83fe24..db9b9fbf9c0f 100644 > --- a/arch/x86/kernel/cpu/common.c > +++ b/arch/x86/kernel/cpu/common.c > @@ -1075,9 +1075,34 @@ static const __initconst struct x86_cpu_id cpu_vuln_whitelist[] = { > {} > }; > > -static bool __init cpu_matches(unsigned long which) > +#define VULNHW_INTEL_STEPPING(model, steppings, issues) \ > + X86_MATCH_VENDOR_FAM_MODEL_STEPPING_FEATURE(INTEL, 6, \ > + INTEL_FAM6_##model, steppings, \ > + X86_FEATURE_ANY, issues) > + > +#define VULNHW_INTEL(model, issues) X86_MATCH_INTEL_FAM6_MODEL(model, issues) > +#define SRBDS BIT(1) > + > +/* > + * List affected CPU's for issues that cannot be enumerated. > + */ > +static const struct x86_cpu_id affected_cpus[] __initconst = { > + VULNHW_INTEL(IVYBRIDGE, SRBDS), > + VULNHW_INTEL(HASWELL, SRBDS), > + VULNHW_INTEL(HASWELL_L, SRBDS), > + VULNHW_INTEL(HASWELL_G, SRBDS), > + VULNHW_INTEL(BROADWELL_G, SRBDS), > + VULNHW_INTEL(BROADWELL, SRBDS), > + VULNHW_INTEL(SKYLAKE_L, SRBDS), > + VULNHW_INTEL(SKYLAKE, SRBDS), > + VULNHW_INTEL_STEPPING(KABYLAKE_L, GENMASK(0xC, 0), SRBDS), /*06_8E steppings <=C*/ > + VULNHW_INTEL_STEPPING(KABYLAKE, GENMASK(0xD, 0), SRBDS), /*06_9E steppings <=D*/ > + {} > +}; > + > +static bool __init cpu_matches(unsigned long which, const struct x86_cpu_id *table) > { > - const struct x86_cpu_id *m = x86_match_cpu(cpu_vuln_whitelist); > + const struct x86_cpu_id *m = x86_match_cpu(table); > > return m && !!(m->driver_data & which); > } > @@ -1097,33 +1122,44 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c) > u64 ia32_cap = x86_read_arch_cap_msr(); > > /* Set ITLB_MULTIHIT bug if cpu is not in the whitelist and not mitigated */ > - if (!cpu_matches(NO_ITLB_MULTIHIT) && !(ia32_cap & ARCH_CAP_PSCHANGE_MC_NO)) > + if (!cpu_matches(NO_ITLB_MULTIHIT, cpu_vuln_whitelist) && > + !(ia32_cap & ARCH_CAP_PSCHANGE_MC_NO)) > setup_force_cpu_bug(X86_BUG_ITLB_MULTIHIT); > > - if (cpu_matches(NO_SPECULATION)) > + if (cpu_matches(NO_SPECULATION, cpu_vuln_whitelist)) > return; > > setup_force_cpu_bug(X86_BUG_SPECTRE_V1); > > - if (!cpu_matches(NO_SPECTRE_V2)) > + if (!cpu_matches(NO_SPECTRE_V2, cpu_vuln_whitelist)) > setup_force_cpu_bug(X86_BUG_SPECTRE_V2); > > - if (!cpu_matches(NO_SSB) && !(ia32_cap & ARCH_CAP_SSB_NO) && > - !cpu_has(c, X86_FEATURE_AMD_SSB_NO)) > + if (!cpu_matches(NO_SSB, cpu_vuln_whitelist) && > + (ia32_cap & ARCH_CAP_SSB_NO) && > + !cpu_has(c, X86_FEATURE_AMD_SSB_NO)) > setup_force_cpu_bug(X86_BUG_SPEC_STORE_BYPASS); > > if (ia32_cap & ARCH_CAP_IBRS_ALL) > setup_force_cpu_cap(X86_FEATURE_IBRS_ENHANCED); > > - if (!cpu_matches(NO_MDS) && !(ia32_cap & ARCH_CAP_MDS_NO)) { > + if (!cpu_matches(NO_MDS, cpu_vuln_whitelist) && > + !(ia32_cap & ARCH_CAP_MDS_NO)) { > setup_force_cpu_bug(X86_BUG_MDS); > - if (cpu_matches(MSBDS_ONLY)) > + if (cpu_matches(MSBDS_ONLY, cpu_vuln_whitelist)) > setup_force_cpu_bug(X86_BUG_MSBDS_ONLY); > } > > - if (!cpu_matches(NO_SWAPGS)) > + if (!cpu_matches(NO_SWAPGS, cpu_vuln_whitelist)) > setup_force_cpu_bug(X86_BUG_SWAPGS); > > + /* > + * Some low-end SKUs on the affected list do not support > + * RDRAND or RDSEED. Make sure they show as "Not affected". > + */ > + if (cpu_matches(SRBDS, affected_cpus) && > + (cpu_has(c, X86_FEATURE_RDRAND) || cpu_has(c, X86_FEATURE_RDSEED))) > + setup_force_cpu_bug(X86_BUG_SRBDS); > + > /* > * When the CPU is not mitigated for TAA (TAA_NO=0) set TAA bug when: > * - TSX is supported or > @@ -1139,7 +1175,7 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c) > (ia32_cap & ARCH_CAP_TSX_CTRL_MSR))) > setup_force_cpu_bug(X86_BUG_TAA); > > - if (cpu_matches(NO_MELTDOWN)) > + if (cpu_matches(NO_MELTDOWN, cpu_vuln_whitelist)) > return; > > /* Rogue Data Cache Load? No! */ > @@ -1148,7 +1184,7 @@ static void __init cpu_set_bug_bits(struct cpuinfo_x86 *c) > > setup_force_cpu_bug(X86_BUG_CPU_MELTDOWN); > > - if (cpu_matches(NO_L1TF)) > + if (cpu_matches(NO_L1TF, cpu_vuln_whitelist)) > return; > > setup_force_cpu_bug(X86_BUG_L1TF); > @@ -1591,6 +1627,7 @@ void identify_secondary_cpu(struct cpuinfo_x86 *c) > mtrr_ap_init(); > validate_apic_and_package_id(c); > x86_spec_ctrl_setup_ap(); > + update_srbds_msr(); > } > > static __init int setup_noclflush(char *arg) > diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h > index 37fdefd14f28..9188ad5be4ed 100644 > --- a/arch/x86/kernel/cpu/cpu.h > +++ b/arch/x86/kernel/cpu/cpu.h > @@ -44,7 +44,10 @@ struct _tlb_table { > extern const struct cpu_dev *const __x86_cpu_dev_start[], > *const __x86_cpu_dev_end[]; > > +void update_srbds_msr(void); > + > #ifdef CONFIG_CPU_SUP_INTEL > + > enum tsx_ctrl_states { > TSX_CTRL_ENABLE, > TSX_CTRL_DISABLE, > diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c > index 9a1c00fbbaef..d2136ab9b14a 100644 > --- a/drivers/base/cpu.c > +++ b/drivers/base/cpu.c > @@ -562,6 +562,12 @@ ssize_t __weak cpu_show_itlb_multihit(struct device *dev, > return sprintf(buf, "Not affected\n"); > } > > +ssize_t __weak cpu_show_srbds(struct device *dev, > + struct device_attribute *attr, char *buf) > +{ > + return sprintf(buf, "Not affected\n"); > +} > + > static DEVICE_ATTR(meltdown, 0444, cpu_show_meltdown, NULL); > static DEVICE_ATTR(spectre_v1, 0444, cpu_show_spectre_v1, NULL); > static DEVICE_ATTR(spectre_v2, 0444, cpu_show_spectre_v2, NULL); > @@ -570,6 +576,7 @@ static DEVICE_ATTR(l1tf, 0444, cpu_show_l1tf, NULL); > static DEVICE_ATTR(mds, 0444, cpu_show_mds, NULL); > static DEVICE_ATTR(tsx_async_abort, 0444, cpu_show_tsx_async_abort, NULL); > static DEVICE_ATTR(itlb_multihit, 0444, cpu_show_itlb_multihit, NULL); > +static DEVICE_ATTR(srbds, 0444, cpu_show_srbds, NULL); > > static struct attribute *cpu_root_vulnerabilities_attrs[] = { > &dev_attr_meltdown.attr, > @@ -580,6 +587,7 @@ static struct attribute *cpu_root_vulnerabilities_attrs[] = { > &dev_attr_mds.attr, > &dev_attr_tsx_async_abort.attr, > &dev_attr_itlb_multihit.attr, > + &dev_attr_srbds.attr, > NULL > }; > > -- > 2.17.1 -- Kees Cook