* Re: [PATCH 1/1] x86/microcode: Check for offline CPUs before checking for microcode update
[not found] ` <20210319165515.9240-2-otavio.pontes@intel.com>
@ 2021-03-19 18:48 ` Pontes, Otavio
2021-03-20 14:55 ` Borislav Petkov
2021-03-22 21:35 ` [tip: x86/microcode] x86/microcode: Check for offline CPUs before requesting new microcode tip-bot2 for Otavio Pontes
1 sibling, 1 reply; 4+ messages in thread
From: Pontes, Otavio @ 2021-03-19 18:48 UTC (permalink / raw)
To: x86
Cc: Borislav Petkov, Thomas Gleixner, Raj, Ashok, Luck, Tony, linux-kernel
Sorry, I forgot to copy LKML
________________________________________
From: Pontes, Otavio <otavio.pontes@intel.com>
Sent: Friday, March 19, 2021 9:55 AM
To: x86@kernel.org
Cc: Borislav Petkov; Thomas Gleixner; Pontes, Otavio; Raj, Ashok; Luck, Tony
Subject: [PATCH 1/1] x86/microcode: Check for offline CPUs before checking for microcode update
Checking for any offline CPUs to abort microcode update must be done before
kernel caching a new microcode from the filesystem. Otherwise when offlined
CPUs are onlined later, those cores are going to be updated with the new
microcode while CPUs previously onine will continue to run with the older
microcode. If update is aborted before checking for the microcode file,
offline CPUs that are onlined later will get the same microcode as online
CPUs.
Turn off one core (2 threads)
$ echo 0 > /sys/devices/system/cpu/cpu3/online
$ echo 0 > /sys/devices/system/cpu/cpu1/online
Install the ucode fails because there's one core off
$ cp intel-ucode/06-8e-09 /lib/firmware/intel-ucode/
$ echo 1 > /sys/devices/system/cpu/microcode/reload
bash: echo: write error: Invalid argument
Turn the core back on
$ echo 1 > /sys/devices/system/cpu/cpu3/online
$ echo 1 > /sys/devices/system/cpu/cpu1/online
$ cat /proc/cpuinfo |grep microcode
microcode : 0x30
microcode : 0xde
microcode : 0x30
microcode : 0xde
Fixes: 30ec26da9967 ("x86/microcode: Do not upload microcode if CPUs are offline")
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Acked-by: Ashok Raj <ashok.raj@intel.com>
Reviewed-by: Tony Luck <tony.luck@intel.com>
---
arch/x86/kernel/cpu/microcode/core.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kernel/cpu/microcode/core.c b/arch/x86/kernel/cpu/microcode/core.c
index b935e1b5f115..6a6318e9590c 100644
--- a/arch/x86/kernel/cpu/microcode/core.c
+++ b/arch/x86/kernel/cpu/microcode/core.c
@@ -629,16 +629,16 @@ static ssize_t reload_store(struct device *dev,
if (val != 1)
return size;
- tmp_ret = microcode_ops->request_microcode_fw(bsp, µcode_pdev->dev, true);
- if (tmp_ret != UCODE_NEW)
- return size;
-
get_online_cpus();
ret = check_online_cpus();
if (ret)
goto put;
+ tmp_ret = microcode_ops->request_microcode_fw(bsp, µcode_pdev->dev, true);
+ if (tmp_ret != UCODE_NEW)
+ goto put;
+
mutex_lock(µcode_mutex);
ret = microcode_reload_late();
mutex_unlock(µcode_mutex);
--
2.30.2
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] x86/microcode: Check for offline CPUs before checking for microcode update
2021-03-19 18:48 ` [PATCH 1/1] x86/microcode: Check for offline CPUs before checking for microcode update Pontes, Otavio
@ 2021-03-20 14:55 ` Borislav Petkov
2021-03-21 4:56 ` Raj, Ashok
0 siblings, 1 reply; 4+ messages in thread
From: Borislav Petkov @ 2021-03-20 14:55 UTC (permalink / raw)
To: Pontes, Otavio; +Cc: x86, Thomas Gleixner, Raj, Ashok, Luck, Tony, linux-kernel
On Fri, Mar 19, 2021 at 06:48:14PM +0000, Pontes, Otavio wrote:
> Turn off one core (2 threads)
> $ echo 0 > /sys/devices/system/cpu/cpu3/online
> $ echo 0 > /sys/devices/system/cpu/cpu1/online
>
> Install the ucode fails because there's one core off
> $ cp intel-ucode/06-8e-09 /lib/firmware/intel-ucode/
> $ echo 1 > /sys/devices/system/cpu/microcode/reload
> bash: echo: write error: Invalid argument
>
> Turn the core back on
> $ echo 1 > /sys/devices/system/cpu/cpu3/online
> $ echo 1 > /sys/devices/system/cpu/cpu1/online
> $ cat /proc/cpuinfo |grep microcode
> microcode : 0x30
> microcode : 0xde
> microcode : 0x30
> microcode : 0xde
Yeah, I'm looking at that check_online_cpus() thing and wondering why we
even need that:
0. So you have CPUs 1 and 3 offline.
1. We can update on the subset of cores which are online
2. If a core is offline and comes online, we have the hotplug notifier:
cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "x86/microcode:online",
mc_cpu_online, mc_cpu_down_prep);
which takes care of updating the microcode when that CPU comes online.
So unless your microcode folks don't come back with a real requirement
why all CPUs must absolutely be online for a late update, then the
proper fix is to get rid of check_online_cpus() altogether and update
what's online and the rest will get updated when they come online.
I know Ashok did:
commit 30ec26da9967d0d785abc24073129a34c3211777
Author: Ashok Raj <ashok.raj@intel.com>
Date: Wed Feb 28 11:28:43 2018 +0100
x86/microcode: Do not upload microcode if CPUs are offline
Avoid loading microcode if any of the CPUs are offline, and issue a
warning. Having different microcode revisions on the system at any time
is outright dangerous.
but those cores are offlined so they're executing some idle routine...
Thx.
--
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquette
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH 1/1] x86/microcode: Check for offline CPUs before checking for microcode update
2021-03-20 14:55 ` Borislav Petkov
@ 2021-03-21 4:56 ` Raj, Ashok
0 siblings, 0 replies; 4+ messages in thread
From: Raj, Ashok @ 2021-03-21 4:56 UTC (permalink / raw)
To: Borislav Petkov
Cc: Pontes, Otavio, x86, Thomas Gleixner, Luck, Tony, linux-kernel,
Ashok Raj
On Sat, Mar 20, 2021 at 03:55:46PM +0100, Borislav Petkov wrote:
[snip]
> > microcode : 0x30
> > microcode : 0xde
> > microcode : 0x30
> > microcode : 0xde
>
> Yeah, I'm looking at that check_online_cpus() thing and wondering why we
> even need that:
>
> 0. So you have CPUs 1 and 3 offline.
> 1. We can update on the subset of cores which are online
> 2. If a core is offline and comes online, we have the hotplug notifier:
>
> cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "x86/microcode:online",
> mc_cpu_online, mc_cpu_down_prep);
>
> which takes care of updating the microcode when that CPU comes online.
>
> So unless your microcode folks don't come back with a real requirement
> why all CPUs must absolutely be online for a late update, then the
> proper fix is to get rid of check_online_cpus() altogether and update
> what's online and the rest will get updated when they come online.
Its true we update them during the online flow, but the core is still
behind compared to other cores. It still participates when it enters SMM,
or when running MCE for instance. Unless its in WAIT_FOR_SIPI state its
best to not leave a core behind when updating microcode.
--
Cheers,
Ashok
[Forgiveness is the attribute of the STRONG - Gandhi]
^ permalink raw reply [flat|nested] 4+ messages in thread
* [tip: x86/microcode] x86/microcode: Check for offline CPUs before requesting new microcode
[not found] ` <20210319165515.9240-2-otavio.pontes@intel.com>
2021-03-19 18:48 ` [PATCH 1/1] x86/microcode: Check for offline CPUs before checking for microcode update Pontes, Otavio
@ 2021-03-22 21:35 ` tip-bot2 for Otavio Pontes
1 sibling, 0 replies; 4+ messages in thread
From: tip-bot2 for Otavio Pontes @ 2021-03-22 21:35 UTC (permalink / raw)
To: linux-tip-commits
Cc: Otavio Pontes, Borislav Petkov, Tony Luck, Ashok Raj, x86, linux-kernel
The following commit has been merged into the x86/microcode branch of tip:
Commit-ID: 7189b3c11903667808029ec9766a6e96de5012a5
Gitweb: https://git.kernel.org/tip/7189b3c11903667808029ec9766a6e96de5012a5
Author: Otavio Pontes <otavio.pontes@intel.com>
AuthorDate: Fri, 19 Mar 2021 09:55:15 -07:00
Committer: Borislav Petkov <bp@suse.de>
CommitterDate: Mon, 22 Mar 2021 22:29:40 +01:00
x86/microcode: Check for offline CPUs before requesting new microcode
Currently, the late microcode loading mechanism checks whether any CPUs
are offlined, and, in such a case, aborts the load attempt.
However, this must be done before the kernel caches new microcode from
the filesystem. Otherwise, when offlined CPUs are onlined later, those
cores are going to be updated through the CPU hotplug notifier callback
with the new microcode, while CPUs previously onine will continue to run
with the older microcode.
For example:
Turn off one core (2 threads):
echo 0 > /sys/devices/system/cpu/cpu3/online
echo 0 > /sys/devices/system/cpu/cpu1/online
Install the ucode fails because a primary SMT thread is offline:
cp intel-ucode/06-8e-09 /lib/firmware/intel-ucode/
echo 1 > /sys/devices/system/cpu/microcode/reload
bash: echo: write error: Invalid argument
Turn the core back on
echo 1 > /sys/devices/system/cpu/cpu3/online
echo 1 > /sys/devices/system/cpu/cpu1/online
cat /proc/cpuinfo |grep microcode
microcode : 0x30
microcode : 0xde
microcode : 0x30
microcode : 0xde
The rationale for why the update is aborted when at least one primary
thread is offline is because even if that thread is soft-offlined
and idle, it will still have to participate in broadcasted MCE's
synchronization dance or enter SMM, and in both examples it will execute
instructions so it better have the same microcode revision as the other
cores.
[ bp: Heavily edit and extend commit message with the reasoning behind all
this. ]
Fixes: 30ec26da9967 ("x86/microcode: Do not upload microcode if CPUs are offline")
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Signed-off-by: Borislav Petkov <bp@suse.de>
Reviewed-by: Tony Luck <tony.luck@intel.com>
Acked-by: Ashok Raj <ashok.raj@intel.com>
Link: https://lkml.kernel.org/r/20210319165515.9240-2-otavio.pontes@intel.com
---
arch/x86/kernel/cpu/microcode/core.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kernel/cpu/microcode/core.c b/arch/x86/kernel/cpu/microcode/core.c
index b935e1b..6a6318e 100644
--- a/arch/x86/kernel/cpu/microcode/core.c
+++ b/arch/x86/kernel/cpu/microcode/core.c
@@ -629,16 +629,16 @@ static ssize_t reload_store(struct device *dev,
if (val != 1)
return size;
- tmp_ret = microcode_ops->request_microcode_fw(bsp, µcode_pdev->dev, true);
- if (tmp_ret != UCODE_NEW)
- return size;
-
get_online_cpus();
ret = check_online_cpus();
if (ret)
goto put;
+ tmp_ret = microcode_ops->request_microcode_fw(bsp, µcode_pdev->dev, true);
+ if (tmp_ret != UCODE_NEW)
+ goto put;
+
mutex_lock(µcode_mutex);
ret = microcode_reload_late();
mutex_unlock(µcode_mutex);
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-03-22 21:36 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <20210319165515.9240-1-otavio.pontes@intel.com>
[not found] ` <20210319165515.9240-2-otavio.pontes@intel.com>
2021-03-19 18:48 ` [PATCH 1/1] x86/microcode: Check for offline CPUs before checking for microcode update Pontes, Otavio
2021-03-20 14:55 ` Borislav Petkov
2021-03-21 4:56 ` Raj, Ashok
2021-03-22 21:35 ` [tip: x86/microcode] x86/microcode: Check for offline CPUs before requesting new microcode tip-bot2 for Otavio Pontes
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).