All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] Parallel microcode update in Linux
@ 2019-08-22 20:43 Mihai Carabas
  2019-08-22 20:43 ` [PATCH] x86/microcode: Update microcode for all cores in parallel Mihai Carabas
  2019-09-01 17:25 ` [PATCH] Parallel microcode update in Linux Pavel Machek
  0 siblings, 2 replies; 24+ messages in thread
From: Mihai Carabas @ 2019-08-22 20:43 UTC (permalink / raw)
  To: linux-kernel
  Cc: bp, ashok.raj, boris.ostrovsky, konrad.wilk, patrick.colp,
	kanth.ghatraju, Jon.Grimm, Thomas.Lendacky, mihai.carabas

This patch enables parallel microcode loading. In order to measure the
improvements of parallel vs serial, we have used the following diff:

diff --git a/arch/x86/kernel/cpu/microcode/core.c b/arch/x86/kernel/cpu/microcode/core.c
index 577b223..1ea08d8 100644
--- a/arch/x86/kernel/cpu/microcode/core.c
+++ b/arch/x86/kernel/cpu/microcode/core.c
@@ -614,17 +614,25 @@ static int __reload_late(void *info)
  */
 static int microcode_reload_late(void)
 {
+       u64 p0, p1;
        int ret;

        atomic_set(&late_cpus_in,  0);
        atomic_set(&late_cpus_out, 0);

+       p0 = rdtsc_ordered();
+
        ret = stop_machine_cpuslocked(__reload_late, NULL, cpu_online_mask);
+
+       p1 = rdtsc_ordered();
+
        if (ret > 0)
                microcode_check();

        pr_info("Reload completed, microcode revision: 0x%x\n", boot_cpu_data.microcode);

+       pr_info("p0: %lld, p1: %lld, diff: %lld\n", p0, p1, p1 - p0);
+
        return ret;
 }

We have used a machine with a broken microcode in BIOS and no microcode in
initramfs (to bypass early loading).

Here are the results for parallel loading (we made two measurements):

[root@ovs108 ~]# uname -a
Linux ovs108 5.3.0-rc5.master.parallel.el7.dev.x86_64 #1 SMP Thu Aug 22 10:17:04 GMT 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@ovs108 ~]# dmesg | grep microcode
[    0.000000] Intel Spectre v2 broken microcode detected; disabling Speculation Control
[    0.197658] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    2.114135] microcode: sig=0x50654, pf=0x80, revision=0x200003a
[    2.117555] microcode: Microcode Update Driver: v2.2.
[   18.197760] microcode: updated to revision 0x200005e, date = 2019-04-02
[   18.201225] x86/CPU: CPU features have changed after loading microcode, but might not take effect.
[   18.201230] microcode: Reload completed, microcode revision: 0x200005e
[   18.201232] microcode: p0: 118138123843052, p1: 118138153732656, diff: 29889604


[root@ovs108 ~]# dmesg | grep microcode
[    0.000000] Intel Spectre v2 broken microcode detected; disabling Speculation Control
[    0.195218] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    2.111882] microcode: sig=0x50654, pf=0x80, revision=0x200003a
[    2.115265] microcode: Microcode Update Driver: v2.2.
[   18.033397] microcode: updated to revision 0x200005e, date = 2019-04-02
[   18.036590] x86/CPU: CPU features have changed after loading microcode, but might not take effect.
[   18.036595] microcode: Reload completed, microcode revision: 0x200005e
[   18.036597] microcode: p0: 118947162428414, p1: 118947191490162, diff: 29061748

Here are the results of serial loading:

[root@ovs108 ~]# uname -a
Linux ovs108 5.3.0-rc5.master.serial.el7.dev.x86_64 #1 SMP Thu Aug 22 12:22:18 GMT 2019 x86_64 x86_64 x86_64 GNU/Linux
[root@ovs108 ~]# dmesg | grep microcode
[    0.000000] Intel Spectre v2 broken microcode detected; disabling Speculation Control
[    0.195158] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    2.111353] microcode: sig=0x50654, pf=0x80, revision=0x200003a
[    2.114834] microcode: Microcode Update Driver: v2.2.
[   17.542518] microcode: updated to revision 0x200005e, date = 2019-04-02
[   17.898365] x86/CPU: CPU features have changed after loading microcode, but might not take effect.
[   17.898370] microcode: Reload completed, microcode revision: 0x200005e
[   17.898372] microcode: p0: 149220216047388, p1: 149221058945422, diff: 842898034

[root@ovs108 ~]# dmesg | grep microcode
[    0.000000] Intel Spectre v2 broken microcode detected; disabling Speculation Control
[    0.197158] MDS: Vulnerable: Clear CPU buffers attempted, no microcode
[    2.114005] microcode: sig=0x50654, pf=0x80, revision=0x200003a
[    2.117451] microcode: Microcode Update Driver: v2.2.
[   17.732026] microcode: updated to revision 0x200005e, date = 2019-04-02
[   18.041398] x86/CPU: CPU features have changed after loading microcode, but might not take effect.
[   18.041404] microcode: Reload completed, microcode revision: 0x200005e
[   18.041407] microcode: p0: 149835792698162, p1: 149836532930286, diff: 740232124


One can see that the difference is an order magnitude.

---

I also tested microcode loading with cpu hotplug.
- I unplugged the last two CPUs  (basically the last core with 2 hyperthreads)
[ 1077.756759] IRQ 324: no longer affine to CPU71
[ 1077.756889] IRQ 619: no longer affine to CPU71
[ 1077.756908] IRQ 645: no longer affine to CPU71
[ 1077.761213] smpboot: CPU 71 is now offline
[ 1082.702759] IRQ 289: no longer affine to CPU70
[ 1082.702771] IRQ 305: no longer affine to CPU70
[ 1082.702827] IRQ 521: no longer affine to CPU70
[ 1082.702860] IRQ 636: no longer affine to CPU70
[ 1082.702876] IRQ 679: no longer affine to CPU70
[ 1082.706897] smpboot: CPU 70 is now offline

- I did the microcode update:
[ 1123.818741] microcode: updated to revision 0x200005e, date = 2019-04-02
[ 1123.821013] x86/CPU: CPU features have changed after loading microcode, but might not take effect.
[ 1123.821014] x86/CPU: Please consider either early loading through initrd/built-in or a potential BIOS update.
[ 1123.821014] microcode: Reload completed, microcode revision: 0x200005e
[ 1123.821015] microcode: p0: 197460831869308, p1: 197460853607904, diff: 21738596

- Than I onlined CPU 70/71
[ 1151.170814] smpboot: Booting Node 1 Processor 70 APIC 0x75
[ 1151.177199] microcode: sig=0x50654, pf=0x80, revision=0x200005e
[ 1182.523811] smpboot: Booting Node 1 Processor 71 APIC 0x77

root@ovs108 ~]# cat /proc/cpuinfo | tr "\t" " " | grep -A 6 "processor : 70" | grep microcode
microcode : 0x200005e

[root@ovs108 ~]# cat /proc/cpuinfo | tr "\t" " " | grep -A 6 "processor : 71" | grep microcode
microcode : 0x200005e


We can see that both CPUs have been updated to the same microcode revision.

Thank you,
Mihai

Ashok Raj (1):
  x86/microcode: Update microcode for all cores in parallel

 arch/x86/kernel/cpu/microcode/core.c  | 44 ++++++++++++++++++++++++-----------
 arch/x86/kernel/cpu/microcode/intel.c | 14 ++++-------
 2 files changed, 36 insertions(+), 22 deletions(-)

-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2019-10-01 17:43 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-08-22 20:43 [PATCH] Parallel microcode update in Linux Mihai Carabas
2019-08-22 20:43 ` [PATCH] x86/microcode: Update microcode for all cores in parallel Mihai Carabas
2019-08-24  8:51   ` Borislav Petkov
2019-08-24  8:53     ` [PATCH 1/2] x86/microcode: Update late microcode " Borislav Petkov
2019-08-26 12:53       ` Boris Ostrovsky
2019-08-26 13:03         ` Borislav Petkov
2019-08-26 20:32         ` Raj, Ashok
2019-08-27 19:43           ` Boris Ostrovsky
2019-08-27 21:24             ` Boris Ostrovsky
2019-08-28 19:16               ` Borislav Petkov
2019-08-26 20:23       ` Raj, Ashok
2019-08-27 12:25         ` Borislav Petkov
2019-08-28  0:56           ` Raj, Ashok
2019-08-28 19:13             ` Borislav Petkov
2019-08-28 19:45               ` Raj, Ashok
2019-08-24  8:53     ` [PATCH 2/2] x86/microcode/intel: Issue the revision updated message only on the BSP Borislav Petkov
2019-08-26 13:34       ` Boris Ostrovsky
2019-08-26 13:45         ` Borislav Petkov
2019-10-01 17:43       ` [tip: x86/microcode] " tip-bot2 for Borislav Petkov
2019-10-01 17:43   ` [tip: x86/microcode] x86/microcode: Update late microcode in parallel tip-bot2 for Ashok Raj
2019-09-01 17:25 ` [PATCH] Parallel microcode update in Linux Pavel Machek
2019-09-02  7:27   ` Mihai Carabas
2019-09-02  7:39     ` Pavel Machek
2019-09-02 11:10       ` Mihai Carabas

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.