Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

[Jinpu Wang]

Resend with plain text, and remove confidential unnecessary signature.
sorry for spam.

Hi Thomas, hi Greg, hi Tony, hi Arjan, hi other expert on the list

I noticed on our Cascade lake with 4.14.120,  the kernel is reporting
vulnerable:
jwang@ps401a-912:~$ head /sys/devices/system/cpu/vulnerabilities/mds
Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable

But according to INTEL,  they have built the mitigation in hardware
for Cascade Lake:
https://www.intel.com/content/www/us/en/architecture-and-technology/engineering-new-protections-into-hardware.html

We are using latest microcode from debian:
https://metadata.ftp-master.debian.org/changelogs//non-free/i/intel-microcode/intel-microcode_3.20190514.1~deb9u1_changelog
lscpu:
jwang@ps401a-912:~$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                96
On-line CPU(s) list:   0-95
Thread(s) per core:    2
Core(s) per socket:    24
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz
Stepping:              5
CPU MHz:               3228.226
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000
BogoMIPS:              5800.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              33792K
NUMA node0 CPU(s):     0-23,48-71
NUMA node1 CPU(s):     24-47,72-95
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep
mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht
tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs
bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq
dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm
pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3
cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow
vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2
erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap
clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec
xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local
dtherm ida arat pln pts pku ospke flush_l1d arch_capabilities

Clearly we want to buy Cascade Lake if the hardware mitigations are in
place, but the information on the internet is quite limited and
misleading.

Could anyone of you could clarify it for us?
Does it mean 4.14.120 is missing patches for detecting Cascade Lake
mitigation, Or maybe only small set of cascade lake has mitigation?
Or the community/intel are not sure about it yet?

Looking forward for your reply.

Thanks,

-- 
Jack Wang
Linux Kernel Developer

Re: Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

[Greg Kroah-Hartman]

On Fri, May 24, 2019 at 03:19:34PM +0200, Jinpu Wang wrote:
> Resend with plain text, and remove confidential unnecessary signature.
> sorry for spam.
> 
> Hi Thomas, hi Greg, hi Tony, hi Arjan, hi other expert on the list
> 
> I noticed on our Cascade lake with 4.14.120,  the kernel is reporting
> vulnerable:
> jwang@ps401a-912:~$ head /sys/devices/system/cpu/vulnerabilities/mds
> Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
> 
> But according to INTEL,  they have built the mitigation in hardware
> for Cascade Lake:
> https://www.intel.com/content/www/us/en/architecture-and-technology/engineering-new-protections-into-hardware.html
> 
> We are using latest microcode from debian:
> https://metadata.ftp-master.debian.org/changelogs//non-free/i/intel-microcode/intel-microcode_3.20190514.1~deb9u1_changelog
> lscpu:
> jwang@ps401a-912:~$ lscpu
> Architecture:          x86_64
> CPU op-mode(s):        32-bit, 64-bit
> Byte Order:            Little Endian
> CPU(s):                96
> On-line CPU(s) list:   0-95
> Thread(s) per core:    2
> Core(s) per socket:    24
> Socket(s):             2
> NUMA node(s):          2
> Vendor ID:             GenuineIntel
> CPU family:            6
> Model:                 85
> Model name:            Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz
> Stepping:              5
> CPU MHz:               3228.226
> CPU max MHz:           3900.0000
> CPU min MHz:           1000.0000
> BogoMIPS:              5800.00
> Virtualization:        VT-x
> L1d cache:             32K
> L1i cache:             32K
> L2 cache:              1024K
> L3 cache:              33792K
> NUMA node0 CPU(s):     0-23,48-71
> NUMA node1 CPU(s):     24-47,72-95
> Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep
> mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht
> tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs
> bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq
> dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm
> pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
> xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3
> cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow
> vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2
> erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap
> clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec
> xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local
> dtherm ida arat pln pts pku ospke flush_l1d arch_capabilities
> 
> Clearly we want to buy Cascade Lake if the hardware mitigations are in
> place, but the information on the internet is quite limited and
> misleading.
> 
> Could anyone of you could clarify it for us?
> Does it mean 4.14.120 is missing patches for detecting Cascade Lake
> mitigation, Or maybe only small set of cascade lake has mitigation?
> Or the community/intel are not sure about it yet?

Did you try 4.19 or 5.1?  Try those and see if anything changes.

If not, odds are you need a microcode update from Intel, please contact
them, nothing we can do about that.

If 4.14 works different from 4.19 or 5.1, please let us know.

thanks,

greg k-h

Re: Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

[Tony Luck]

Stepping 5 is preproduction of cascade lake. MDS mitigation is in production parts (stepping >= 6)

Sent from my iPhone

> On May 24, 2019, at 07:17, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> 
>> On Fri, May 24, 2019 at 03:19:34PM +0200, Jinpu Wang wrote:
>> Resend with plain text, and remove confidential unnecessary signature.
>> sorry for spam.
>> 
>> Hi Thomas, hi Greg, hi Tony, hi Arjan, hi other expert on the list
>> 
>> I noticed on our Cascade lake with 4.14.120,  the kernel is reporting
>> vulnerable:
>> jwang@ps401a-912:~$ head /sys/devices/system/cpu/vulnerabilities/mds
>> Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
>> 
>> But according to INTEL,  they have built the mitigation in hardware
>> for Cascade Lake:
>> https://www.intel.com/content/www/us/en/architecture-and-technology/engineering-new-protections-into-hardware.html
>> 
>> We are using latest microcode from debian:
>> https://metadata.ftp-master.debian.org/changelogs//non-free/i/intel-microcode/intel-microcode_3.20190514.1~deb9u1_changelog
>> lscpu:
>> jwang@ps401a-912:~$ lscpu
>> Architecture:          x86_64
>> CPU op-mode(s):        32-bit, 64-bit
>> Byte Order:            Little Endian
>> CPU(s):                96
>> On-line CPU(s) list:   0-95
>> Thread(s) per core:    2
>> Core(s) per socket:    24
>> Socket(s):             2
>> NUMA node(s):          2
>> Vendor ID:             GenuineIntel
>> CPU family:            6
>> Model:                 85
>> Model name:            Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz
>> Stepping:              5
>> CPU MHz:               3228.226
>> CPU max MHz:           3900.0000
>> CPU min MHz:           1000.0000
>> BogoMIPS:              5800.00
>> Virtualization:        VT-x
>> L1d cache:             32K
>> L1i cache:             32K
>> L2 cache:              1024K
>> L3 cache:              33792K
>> NUMA node0 CPU(s):     0-23,48-71
>> NUMA node1 CPU(s):     24-47,72-95
>> Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep
>> mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht
>> tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs
>> bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq
>> dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm
>> pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
>> xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3
>> cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow
>> vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2
>> erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap
>> clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec
>> xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local
>> dtherm ida arat pln pts pku ospke flush_l1d arch_capabilities
>> 
>> Clearly we want to buy Cascade Lake if the hardware mitigations are in
>> place, but the information on the internet is quite limited and
>> misleading.
>> 
>> Could anyone of you could clarify it for us?
>> Does it mean 4.14.120 is missing patches for detecting Cascade Lake
>> mitigation, Or maybe only small set of cascade lake has mitigation?
>> Or the community/intel are not sure about it yet?
> 
> Did you try 4.19 or 5.1?  Try those and see if anything changes.
> 
> If not, odds are you need a microcode update from Intel, please contact
> them, nothing we can do about that.
> 
> If 4.14 works different from 4.19 or 5.1, please let us know.
> 
> thanks,
> 
> greg k-h

Re: Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

[Jinpu Wang]

On Fri, May 24, 2019 at 4:45 PM Tony Luck <tony.luck@gmail.com> wrote:
>
> Stepping 5 is preproduction of cascade lake. MDS mitigation is in production parts (stepping >= 6)
>
> Sent from my iPhone
>
> > On May 24, 2019, at 07:17, Greg Kroah-Hartman <gregkh@linuxfoundation.org> wrote:
> >
> >> On Fri, May 24, 2019 at 03:19:34PM +0200, Jinpu Wang wrote:
> >> Resend with plain text, and remove confidential unnecessary signature.
> >> sorry for spam.
> >>
> >> Hi Thomas, hi Greg, hi Tony, hi Arjan, hi other expert on the list
> >>
> >> I noticed on our Cascade lake with 4.14.120,  the kernel is reporting
> >> vulnerable:
> >> jwang@ps401a-912:~$ head /sys/devices/system/cpu/vulnerabilities/mds
> >> Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable
> >>
> >> But according to INTEL,  they have built the mitigation in hardware
> >> for Cascade Lake:
> >> https://www.intel.com/content/www/us/en/architecture-and-technology/engineering-new-protections-into-hardware.html
> >>
> >> We are using latest microcode from debian:
> >> https://metadata.ftp-master.debian.org/changelogs//non-free/i/intel-microcode/intel-microcode_3.20190514.1~deb9u1_changelog
> >> lscpu:
> >> jwang@ps401a-912:~$ lscpu
> >> Architecture:          x86_64
> >> CPU op-mode(s):        32-bit, 64-bit
> >> Byte Order:            Little Endian
> >> CPU(s):                96
> >> On-line CPU(s) list:   0-95
> >> Thread(s) per core:    2
> >> Core(s) per socket:    24
> >> Socket(s):             2
> >> NUMA node(s):          2
> >> Vendor ID:             GenuineIntel
> >> CPU family:            6
> >> Model:                 85
> >> Model name:            Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz
> >> Stepping:              5
> >> CPU MHz:               3228.226
> >> CPU max MHz:           3900.0000
> >> CPU min MHz:           1000.0000
> >> BogoMIPS:              5800.00
> >> Virtualization:        VT-x
> >> L1d cache:             32K
> >> L1i cache:             32K
> >> L2 cache:              1024K
> >> L3 cache:              33792K
> >> NUMA node0 CPU(s):     0-23,48-71
> >> NUMA node1 CPU(s):     24-47,72-95
> >> Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep
> >> mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht
> >> tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs
> >> bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq
> >> dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm
> >> pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
> >> xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3
> >> cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow
> >> vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2
> >> erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap
> >> clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec
> >> xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local
> >> dtherm ida arat pln pts pku ospke flush_l1d arch_capabilities
> >>
> >> Clearly we want to buy Cascade Lake if the hardware mitigations are in
> >> place, but the information on the internet is quite limited and
> >> misleading.
> >>
> >> Could anyone of you could clarify it for us?
> >> Does it mean 4.14.120 is missing patches for detecting Cascade Lake
> >> mitigation, Or maybe only small set of cascade lake has mitigation?
> >> Or the community/intel are not sure about it yet?
> >
> > Did you try 4.19 or 5.1?  Try those and see if anything changes.
> >
> > If not, odds are you need a microcode update from Intel, please contact
> > them, nothing we can do about that.
> >
> > If 4.14 works different from 4.19 or 5.1, please let us know.
> >
> > thanks,
> >
> > greg k-h

Thanks, Greg, thanks Tony,

I tried also 4.19.46 kernel, it show the same.

As Tony mentioned, it's due to the stepping difference, our test
machine is a preproduction one.
I would expect the production Cascade Lake supports md-clear feature,
so VM migration works from old generation.
Could some one give me an answer?

Thanks a lot.
-- 
Jack Wang
Linux Kernel Developer @ 1 & 1 Cloud IONOS GmbH

Re: Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

[Greg Kroah-Hartman]

On Fri, May 24, 2019 at 05:14:38PM +0200, Jinpu Wang wrote:
> I would expect the production Cascade Lake supports md-clear feature,
> so VM migration works from old generation.
> Could some one give me an answer?

No idea, try it and see!

good luck!

greg k-h

Re: Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

[Greg Kroah-Hartman]

On Fri, May 24, 2019 at 11:23:40AM +0200, Jinpu Wang wrote:
> This e-mail may contain confidential and/or privileged information. If
> you are not the intended recipient of this e-mail, you are hereby
> notified that saving, distribution or use of the content of this
> e-mail in any way is prohibited. If you have received this e-mail in
> error, please notify the sender and delete the e-mail.

I am not allowed to respond to such emails as they do not mix well with
Linux kernel development requirements, sorry.

Is 2nd Generation Intel(R) Xeon(R) Processors (Formerly Cascade Lake) affected by MDS

[Jinpu Wang]

Resend with plain text, sorry for spam.

Hi Thomas, hi Greg, hi Tony, hi Arjan, hi other expert on the list

I noticed on our Cascade lake with 4.14.120,  the kernel is reporting
vulnerable:
jwang@ps401a-912:~$ head /sys/devices/system/cpu/vulnerabilities/mds
Vulnerable: Clear CPU buffers attempted, no microcode; SMT vulnerable

But according to INTEL,  they have built the mitigation in hardware
for Cascade Lake:
https://www.intel.com/content/www/us/en/architecture-and-technology/engineering-new-protections-into-hardware.html

We are using latest microcode from debian:
https://metadata.ftp-master.debian.org/changelogs//non-free/i/intel-microcode/intel-microcode_3.20190514.1~deb9u1_changelog
lscpu:
jwang@ps401a-912:~$ lscpu
Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                96
On-line CPU(s) list:   0-95
Thread(s) per core:    2
Core(s) per socket:    24
Socket(s):             2
NUMA node(s):          2
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 85
Model name:            Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz
Stepping:              5
CPU MHz:               3228.226
CPU max MHz:           3900.0000
CPU min MHz:           1000.0000
BogoMIPS:              5800.00
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              1024K
L3 cache:              33792K
NUMA node0 CPU(s):     0-23,48-71
NUMA node1 CPU(s):     24-47,72-95
Flags:                 fpu vme de pse tsc msr pae mce cx8 apic sep
mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht
tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs
bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq
dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm
pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes
xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3
cdp_l3 invpcid_single intel_ppin ssbd mba ibrs ibpb stibp tpr_shadow
vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2
erms invpcid rtm cqm mpx rdt_a avx512f avx512dq rdseed adx smap
clflushopt clwb intel_pt avx512cd avx512bw avx512vl xsaveopt xsavec
xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local
dtherm ida arat pln pts pku ospke flush_l1d arch_capabilities

Clearly we want to buy Cascade Lake if the hardware mitigations are in
place, but the information on the internet is quite limited and
misleading.

Could anyone of you could clarify it for us?
Do it mean 4.14.120 is missing patches for detecting Cascade Lake
mitigation, Or maybe only small set of cascade lake has mitigation?
Or the community/intel are not sure about it yet?

Thanks,
-- 
Jack Wang
Linux Kernel Developer

1&1 IONOS Cloud GmbH | Greifswalder Str. 207 | 10405 Berlin | Germany
Phone: +49 30 57700-8042 | Fax: +49 30 57700-8598
E-mail: jinpu.wang@cloud.ionos.com | Web: www.ionos.de


Head Office: Berlin, Germany
District Court Berlin Charlottenburg, Registration number: HRB 125506 B
Executive Management: Christoph Steffens, Matthias Steinberg, Achim Weiss

Member of United Internet

This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient of this e-mail, you are hereby
notified that saving, distribution or use of the content of this
e-mail in any way is prohibited. If you have received this e-mail in
error, please notify the sender and delete the e-mail.

-- 
Jack Wang
Linux Kernel Developer

1&1 IONOS Cloud GmbH | Greifswalder Str. 207 | 10405 Berlin | Germany
Phone: +49 30 57700-8042 | Fax: +49 30 57700-8598
E-mail: jinpu.wang@cloud.ionos.com | Web: www.ionos.de


Head Office: Berlin, Germany
District Court Berlin Charlottenburg, Registration number: HRB 125506 B
Executive Management: Christoph Steffens, Matthias Steinberg, Achim Weiss

Member of United Internet

This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient of this e-mail, you are hereby
notified that saving, distribution or use of the content of this
e-mail in any way is prohibited. If you have received this e-mail in
error, please notify the sender and delete the e-mail.