* [PATCH v7 00/10] improve late microcode loading
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Boris Ostrovsky, Chao Gao, Brian Woods, Suravee Suthikulpanit,
Roger Pau Monné
Changes in version 7:
- cache one microcode update rather than a list of it. Assuming that all CPUs
(including those will be plugged in later) in the system have the same
signature, one update matches with one CPU should match with others. Thus, one
update is enough for microcode updating during CPU hot-plug and resuming.
- To handle load failure, microcode update is cached after it is applied to
avoid a broken update overriding a validated one. Unvalidated microcode updates
are passed by arguments rather than another global variable, where this series
slightly differs from Roger's suggestion in:
https://lists.xen.org/archives/html/xen-devel/2019-03/msg00776.html
- incorporate Sergey's patch (patch 10) to fix a bug: we maintain a variable
to reflect current microcode revision. But in some cases, this variable isn't
initialized during system boot time, which results in falsely reporting that
processor is susceptible to some known vulnerabilities.
- fix issues reported by Sergey:
https://lists.xenproject.org/archives/html/xen-devel/2019-03/msg00901.html
- Responses to Sergey/Roger/Wei/Ashok's other comments.
Major changes in version 6:
- run wbinvd before updating microcode (patch 10)
- add an userspace tool for late microcode update (patch 1)
- scale time to wait by the number of remaining CPUs to respond
- remove 'cpu' parameters from some related callbacks and functins
- save an ucode patch only if its supported CPU is allowed to mix with
current cpu.
Changes in version 5:
- support parallel microcode updates for all cores (see patch 8)
- Address Roger's comments on the last version.
The intention of this series is to make the late microcode loading
more reliable by rendezvousing all cpus in stop_machine context.
This idea comes from Ashok. I am porting his linux patch to Xen
(see patch 7 and 8 for more details).
This series includes five changes:
1. Patch 1: an userspace tool for late microcode update
2. Patch 2-6: introduce a global microcode cache and some cleanup
3. Patch 7: writeback and invalidate cache before updating microcode
3. Patch 8: synchronize late microcode loading
4. Patch 9: support parallel microcodes update on different cores
5. Patch 10: always read microcode revision at boot time
Currently, late microcode loading does a lot of things including
parsing microcode blob, checking the signature/revision and performing
update. Putting all of them into stop_machine context is a bad idea
because of complexity (one issue I observed is memory allocation
triggered one assertion in stop_machine context). To simplify the
load process, parsing microcode is moved out of the load process.
Remaining parts of load process is put to stop_machine context.
Regarding changes to AMD side, I didn't do any test for them due to
lack of hardware. Sergey, could you help to test this series on an
AMD machine again?
At least, two basic tests are needed:
* do a microcode update after system bootup
* don't bring all pCPUs up at bootup by specifying maxcpus option in xen
command line and then do a microcode update and online all offlined
CPUs via 'xen-hptool'.
Chao Gao (9):
misc/xen-ucode: Upload a microcode blob to the hypervisor
microcode/intel: extend microcode_update_match()
microcode: introduce a global cache of ucode patch
microcode: remove struct ucode_cpu_info
microcode: remove pointless 'cpu' parameter
microcode: split out apply_microcode() from cpu_request_microcode()
microcode/intel: Writeback and invalidate caches before updating
microcode
x86/microcode: Synchronize late microcode loading
microcode: remove microcode_update_lock
Sergey Dyasli (1):
x86/microcode: always collect_cpu_info() during boot
tools/libxc/include/xenctrl.h | 1 +
tools/libxc/xc_misc.c | 23 +++
tools/misc/Makefile | 4 +
tools/misc/xen-ucode.c | 78 ++++++++
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/apic.c | 2 +-
xen/arch/x86/microcode.c | 401 ++++++++++++++++++++++++++++------------
xen/arch/x86/microcode_amd.c | 245 ++++++++++++------------
xen/arch/x86/microcode_intel.c | 202 ++++++++++----------
xen/arch/x86/smpboot.c | 5 +-
xen/arch/x86/spec_ctrl.c | 2 +-
xen/include/asm-x86/microcode.h | 39 ++--
xen/include/asm-x86/processor.h | 3 +-
13 files changed, 639 insertions(+), 368 deletions(-)
create mode 100644 tools/misc/xen-ucode.c
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 00/10] improve late microcode loading
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Boris Ostrovsky, Chao Gao, Brian Woods, Suravee Suthikulpanit,
Roger Pau Monné
Changes in version 7:
- cache one microcode update rather than a list of it. Assuming that all CPUs
(including those will be plugged in later) in the system have the same
signature, one update matches with one CPU should match with others. Thus, one
update is enough for microcode updating during CPU hot-plug and resuming.
- To handle load failure, microcode update is cached after it is applied to
avoid a broken update overriding a validated one. Unvalidated microcode updates
are passed by arguments rather than another global variable, where this series
slightly differs from Roger's suggestion in:
https://lists.xen.org/archives/html/xen-devel/2019-03/msg00776.html
- incorporate Sergey's patch (patch 10) to fix a bug: we maintain a variable
to reflect current microcode revision. But in some cases, this variable isn't
initialized during system boot time, which results in falsely reporting that
processor is susceptible to some known vulnerabilities.
- fix issues reported by Sergey:
https://lists.xenproject.org/archives/html/xen-devel/2019-03/msg00901.html
- Responses to Sergey/Roger/Wei/Ashok's other comments.
Major changes in version 6:
- run wbinvd before updating microcode (patch 10)
- add an userspace tool for late microcode update (patch 1)
- scale time to wait by the number of remaining CPUs to respond
- remove 'cpu' parameters from some related callbacks and functins
- save an ucode patch only if its supported CPU is allowed to mix with
current cpu.
Changes in version 5:
- support parallel microcode updates for all cores (see patch 8)
- Address Roger's comments on the last version.
The intention of this series is to make the late microcode loading
more reliable by rendezvousing all cpus in stop_machine context.
This idea comes from Ashok. I am porting his linux patch to Xen
(see patch 7 and 8 for more details).
This series includes five changes:
1. Patch 1: an userspace tool for late microcode update
2. Patch 2-6: introduce a global microcode cache and some cleanup
3. Patch 7: writeback and invalidate cache before updating microcode
3. Patch 8: synchronize late microcode loading
4. Patch 9: support parallel microcodes update on different cores
5. Patch 10: always read microcode revision at boot time
Currently, late microcode loading does a lot of things including
parsing microcode blob, checking the signature/revision and performing
update. Putting all of them into stop_machine context is a bad idea
because of complexity (one issue I observed is memory allocation
triggered one assertion in stop_machine context). To simplify the
load process, parsing microcode is moved out of the load process.
Remaining parts of load process is put to stop_machine context.
Regarding changes to AMD side, I didn't do any test for them due to
lack of hardware. Sergey, could you help to test this series on an
AMD machine again?
At least, two basic tests are needed:
* do a microcode update after system bootup
* don't bring all pCPUs up at bootup by specifying maxcpus option in xen
command line and then do a microcode update and online all offlined
CPUs via 'xen-hptool'.
Chao Gao (9):
misc/xen-ucode: Upload a microcode blob to the hypervisor
microcode/intel: extend microcode_update_match()
microcode: introduce a global cache of ucode patch
microcode: remove struct ucode_cpu_info
microcode: remove pointless 'cpu' parameter
microcode: split out apply_microcode() from cpu_request_microcode()
microcode/intel: Writeback and invalidate caches before updating
microcode
x86/microcode: Synchronize late microcode loading
microcode: remove microcode_update_lock
Sergey Dyasli (1):
x86/microcode: always collect_cpu_info() during boot
tools/libxc/include/xenctrl.h | 1 +
tools/libxc/xc_misc.c | 23 +++
tools/misc/Makefile | 4 +
tools/misc/xen-ucode.c | 78 ++++++++
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/apic.c | 2 +-
xen/arch/x86/microcode.c | 401 ++++++++++++++++++++++++++++------------
xen/arch/x86/microcode_amd.c | 245 ++++++++++++------------
xen/arch/x86/microcode_intel.c | 202 ++++++++++----------
xen/arch/x86/smpboot.c | 5 +-
xen/arch/x86/spec_ctrl.c | 2 +-
xen/include/asm-x86/microcode.h | 39 ++--
xen/include/asm-x86/processor.h | 3 +-
13 files changed, 639 insertions(+), 368 deletions(-)
create mode 100644 tools/misc/xen-ucode.c
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* [PATCH v7 01/10] misc/xen-ucode: Upload a microcode blob to the hypervisor
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Konrad Rzeszutek Wilk,
Ian Jackson, Chao Gao
This patch provides a tool for late microcode update.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- introduce xc_microcode_update() rather than xc_platform_op()
- avoid creating bounce buffer twice
- rename xenmicrocode to xen-ucode, following naming tradition
of other tools there.
---
tools/libxc/include/xenctrl.h | 1 +
tools/libxc/xc_misc.c | 23 +++++++++++++
tools/misc/Makefile | 4 +++
tools/misc/xen-ucode.c | 78 +++++++++++++++++++++++++++++++++++++++++++
4 files changed, 106 insertions(+)
create mode 100644 tools/misc/xen-ucode.c
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 538007a..6d80ae5 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1244,6 +1244,7 @@ typedef uint32_t xc_node_to_node_dist_t;
int xc_physinfo(xc_interface *xch, xc_physinfo_t *info);
int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
xc_cputopo_t *cputopo);
+int xc_microcode_update(xc_interface *xch, const void *buf, size_t len);
int xc_numainfo(xc_interface *xch, unsigned *max_nodes,
xc_meminfo_t *meminfo, uint32_t *distance);
int xc_pcitopoinfo(xc_interface *xch, unsigned num_devs,
diff --git a/tools/libxc/xc_misc.c b/tools/libxc/xc_misc.c
index 5e6714a..85538e0 100644
--- a/tools/libxc/xc_misc.c
+++ b/tools/libxc/xc_misc.c
@@ -226,6 +226,29 @@ int xc_physinfo(xc_interface *xch,
return 0;
}
+int xc_microcode_update(xc_interface *xch, const void *buf, size_t len)
+{
+ int ret;
+ DECLARE_PLATFORM_OP;
+ DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update, uc);
+
+ uc = xc_hypercall_buffer_alloc(xch, uc, len);
+ if (uc == NULL)
+ return -1;
+
+ memcpy(uc, buf, len);
+
+ platform_op.cmd = XENPF_microcode_update;
+ platform_op.u.microcode.length = len;
+ set_xen_guest_handle(platform_op.u.microcode.data, uc);
+
+ ret = do_platform_op(xch, &platform_op);
+
+ xc_hypercall_buffer_free(xch, uc);
+
+ return ret;
+}
+
int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
xc_cputopo_t *cputopo)
{
diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index d4320dc..63947bf 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -22,6 +22,7 @@ INSTALL_SBIN-$(CONFIG_X86) += xen-hvmcrash
INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx
INSTALL_SBIN-$(CONFIG_X86) += xen-lowmemd
INSTALL_SBIN-$(CONFIG_X86) += xen-mfndump
+INSTALL_SBIN-$(CONFIG_X86) += xen-ucode
INSTALL_SBIN += xencov
INSTALL_SBIN += xenlockprof
INSTALL_SBIN += xenperf
@@ -113,4 +114,7 @@ xen-lowmemd: xen-lowmemd.o
xencov: xencov.o
$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
+xen-ucode: xen-ucode.o
+ $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
+
-include $(DEPS_INCLUDE)
diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
new file mode 100644
index 0000000..da668ca
--- /dev/null
+++ b/tools/misc/xen-ucode.c
@@ -0,0 +1,78 @@
+#define _GNU_SOURCE
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+#include <errno.h>
+#include <string.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <xenctrl.h>
+
+void show_help(void)
+{
+ fprintf(stderr,
+ "xenmicrocode: Xen microcode updating tool\n"
+ "Usage: xenmicrocode <microcode blob>\n");
+}
+
+int main(int argc, char *argv[])
+{
+ int fd, len, ret;
+ char *filename, *buf;
+ struct stat st;
+ xc_interface *xch;
+
+ if (argc < 2)
+ {
+ show_help();
+ return 0;
+ }
+
+ filename = argv[1];
+ fd = open(filename, O_RDONLY);
+ if (fd < 0) {
+ fprintf(stderr, "Could not open %s. (err: %s)\n",
+ filename, strerror(errno));
+ return errno;
+ }
+
+ if (stat(filename, &st) != 0) {
+ fprintf(stderr, "Could not get the size of %s. (err: %s)\n",
+ filename, strerror(errno));
+ return errno;
+ }
+
+ len = st.st_size;
+ buf = mmap(0, len, PROT_READ, MAP_PRIVATE, fd, 0);
+ if (buf == MAP_FAILED) {
+ fprintf(stderr, "mmap failed. (error: %s)\n", strerror(errno));
+ return errno;
+ }
+
+ xch = xc_interface_open(0,0,0);
+ if (xch == NULL)
+ {
+ fprintf(stderr, "Error opening xc interface. (err: %s)\n",
+ strerror(errno));
+ return errno;
+ }
+
+ ret = xc_microcode_update(xch, buf, len);
+ if (ret)
+ fprintf(stderr, "Failed to update microcode. (err: %s)\n",
+ strerror(errno));
+
+ xc_interface_close(xch);
+
+ if (munmap(buf, len)) {
+ printf("Could not unmap: %d(%s)\n", errno, strerror(errno));
+ return errno;
+ }
+ close(fd);
+
+ return 0;
+}
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 01/10] misc/xen-ucode: Upload a microcode blob to the hypervisor
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Konrad Rzeszutek Wilk,
Ian Jackson, Chao Gao
This patch provides a tool for late microcode update.
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- introduce xc_microcode_update() rather than xc_platform_op()
- avoid creating bounce buffer twice
- rename xenmicrocode to xen-ucode, following naming tradition
of other tools there.
---
tools/libxc/include/xenctrl.h | 1 +
tools/libxc/xc_misc.c | 23 +++++++++++++
tools/misc/Makefile | 4 +++
tools/misc/xen-ucode.c | 78 +++++++++++++++++++++++++++++++++++++++++++
4 files changed, 106 insertions(+)
create mode 100644 tools/misc/xen-ucode.c
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 538007a..6d80ae5 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -1244,6 +1244,7 @@ typedef uint32_t xc_node_to_node_dist_t;
int xc_physinfo(xc_interface *xch, xc_physinfo_t *info);
int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
xc_cputopo_t *cputopo);
+int xc_microcode_update(xc_interface *xch, const void *buf, size_t len);
int xc_numainfo(xc_interface *xch, unsigned *max_nodes,
xc_meminfo_t *meminfo, uint32_t *distance);
int xc_pcitopoinfo(xc_interface *xch, unsigned num_devs,
diff --git a/tools/libxc/xc_misc.c b/tools/libxc/xc_misc.c
index 5e6714a..85538e0 100644
--- a/tools/libxc/xc_misc.c
+++ b/tools/libxc/xc_misc.c
@@ -226,6 +226,29 @@ int xc_physinfo(xc_interface *xch,
return 0;
}
+int xc_microcode_update(xc_interface *xch, const void *buf, size_t len)
+{
+ int ret;
+ DECLARE_PLATFORM_OP;
+ DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update, uc);
+
+ uc = xc_hypercall_buffer_alloc(xch, uc, len);
+ if (uc == NULL)
+ return -1;
+
+ memcpy(uc, buf, len);
+
+ platform_op.cmd = XENPF_microcode_update;
+ platform_op.u.microcode.length = len;
+ set_xen_guest_handle(platform_op.u.microcode.data, uc);
+
+ ret = do_platform_op(xch, &platform_op);
+
+ xc_hypercall_buffer_free(xch, uc);
+
+ return ret;
+}
+
int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
xc_cputopo_t *cputopo)
{
diff --git a/tools/misc/Makefile b/tools/misc/Makefile
index d4320dc..63947bf 100644
--- a/tools/misc/Makefile
+++ b/tools/misc/Makefile
@@ -22,6 +22,7 @@ INSTALL_SBIN-$(CONFIG_X86) += xen-hvmcrash
INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx
INSTALL_SBIN-$(CONFIG_X86) += xen-lowmemd
INSTALL_SBIN-$(CONFIG_X86) += xen-mfndump
+INSTALL_SBIN-$(CONFIG_X86) += xen-ucode
INSTALL_SBIN += xencov
INSTALL_SBIN += xenlockprof
INSTALL_SBIN += xenperf
@@ -113,4 +114,7 @@ xen-lowmemd: xen-lowmemd.o
xencov: xencov.o
$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
+xen-ucode: xen-ucode.o
+ $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
+
-include $(DEPS_INCLUDE)
diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
new file mode 100644
index 0000000..da668ca
--- /dev/null
+++ b/tools/misc/xen-ucode.c
@@ -0,0 +1,78 @@
+#define _GNU_SOURCE
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/mman.h>
+#include <errno.h>
+#include <string.h>
+#include <inttypes.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <xenctrl.h>
+
+void show_help(void)
+{
+ fprintf(stderr,
+ "xenmicrocode: Xen microcode updating tool\n"
+ "Usage: xenmicrocode <microcode blob>\n");
+}
+
+int main(int argc, char *argv[])
+{
+ int fd, len, ret;
+ char *filename, *buf;
+ struct stat st;
+ xc_interface *xch;
+
+ if (argc < 2)
+ {
+ show_help();
+ return 0;
+ }
+
+ filename = argv[1];
+ fd = open(filename, O_RDONLY);
+ if (fd < 0) {
+ fprintf(stderr, "Could not open %s. (err: %s)\n",
+ filename, strerror(errno));
+ return errno;
+ }
+
+ if (stat(filename, &st) != 0) {
+ fprintf(stderr, "Could not get the size of %s. (err: %s)\n",
+ filename, strerror(errno));
+ return errno;
+ }
+
+ len = st.st_size;
+ buf = mmap(0, len, PROT_READ, MAP_PRIVATE, fd, 0);
+ if (buf == MAP_FAILED) {
+ fprintf(stderr, "mmap failed. (error: %s)\n", strerror(errno));
+ return errno;
+ }
+
+ xch = xc_interface_open(0,0,0);
+ if (xch == NULL)
+ {
+ fprintf(stderr, "Error opening xc interface. (err: %s)\n",
+ strerror(errno));
+ return errno;
+ }
+
+ ret = xc_microcode_update(xch, buf, len);
+ if (ret)
+ fprintf(stderr, "Failed to update microcode. (err: %s)\n",
+ strerror(errno));
+
+ xc_interface_close(xch);
+
+ if (munmap(buf, len)) {
+ printf("Could not unmap: %d(%s)\n", errno, strerror(errno));
+ return errno;
+ }
+ close(fd);
+
+ return 0;
+}
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 02/10] microcode/intel: extend microcode_update_match()
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
to a more generic function. Then, this function can compare two given
microcodes' signature/revision as well. Comparing two microcodes is
used to update the global microcode cache (introduced by the later
patches in this series) when a new microcode is given.
Note that enum microcode_match_result will be used in common code
(aka microcode.c), it has been placed in the common header.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes in v6:
- eliminate unnecessary type casting in microcode_update_match
- check if a patch has an extend header
Changes in v5:
- constify the extended_signature
- use named enum type for the return value of microcode_update_match
---
xen/arch/x86/microcode_intel.c | 48 +++++++++++++++++++----------------------
xen/include/asm-x86/microcode.h | 6 ++++++
2 files changed, 28 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index 22fdeca..ecec83b 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -134,14 +134,28 @@ static int collect_cpu_info(unsigned int cpu_num, struct cpu_signature *csig)
return 0;
}
-static inline int microcode_update_match(
- unsigned int cpu_num, const struct microcode_header_intel *mc_header,
- int sig, int pf)
+static enum microcode_match_result microcode_update_match(
+ const struct microcode_header_intel *mc_header, unsigned int sig,
+ unsigned int pf, unsigned int rev)
{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
+ const struct extended_sigtable *ext_header;
+ const struct extended_signature *ext_sig;
+ unsigned long data_size = get_datasize(mc_header);
+ unsigned int i;
+
+ if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
+ return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
- return (sigmatch(sig, uci->cpu_sig.sig, pf, uci->cpu_sig.pf) &&
- (mc_header->rev > uci->cpu_sig.rev));
+ if ( get_totalsize(mc_header) == (data_size + MC_HEADER_SIZE) )
+ return MIS_UCODE;
+
+ ext_header = (const void *)(mc_header + 1) + data_size;
+ ext_sig = (const void *)(ext_header + 1);
+ for ( i = 0; i < ext_header->count; i++ )
+ if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
+ return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
+
+ return MIS_UCODE;
}
static int microcode_sanity_check(void *mc)
@@ -243,31 +257,13 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
{
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
const struct microcode_header_intel *mc_header = mc;
- const struct extended_sigtable *ext_header;
unsigned long total_size = get_totalsize(mc_header);
- int ext_sigcount, i;
- struct extended_signature *ext_sig;
void *new_mc;
- if ( microcode_update_match(cpu, mc_header,
- mc_header->sig, mc_header->pf) )
- goto find;
-
- if ( total_size <= (get_datasize(mc_header) + MC_HEADER_SIZE) )
+ if ( microcode_update_match(mc, uci->cpu_sig.sig, uci->cpu_sig.pf,
+ uci->cpu_sig.rev) != NEW_UCODE )
return 0;
- ext_header = mc + get_datasize(mc_header) + MC_HEADER_SIZE;
- ext_sigcount = ext_header->count;
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
- for ( i = 0; i < ext_sigcount; i++ )
- {
- if ( microcode_update_match(cpu, mc_header,
- ext_sig->sig, ext_sig->pf) )
- goto find;
- ext_sig++;
- }
- return 0;
- find:
pr_debug("microcode: CPU%d found a matching microcode update with"
" version %#x (current=%#x)\n",
cpu, mc_header->rev, uci->cpu_sig.rev);
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index 23ea954..73ebe9a 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -3,6 +3,12 @@
#include <xen/percpu.h>
+enum microcode_match_result {
+ OLD_UCODE, /* signature matched, but revision id isn't newer */
+ NEW_UCODE, /* signature matched, but revision id is newer */
+ MIS_UCODE, /* signature mismatched */
+};
+
struct cpu_signature;
struct ucode_cpu_info;
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 02/10] microcode/intel: extend microcode_update_match()
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
to a more generic function. Then, this function can compare two given
microcodes' signature/revision as well. Comparing two microcodes is
used to update the global microcode cache (introduced by the later
patches in this series) when a new microcode is given.
Note that enum microcode_match_result will be used in common code
(aka microcode.c), it has been placed in the common header.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes in v6:
- eliminate unnecessary type casting in microcode_update_match
- check if a patch has an extend header
Changes in v5:
- constify the extended_signature
- use named enum type for the return value of microcode_update_match
---
xen/arch/x86/microcode_intel.c | 48 +++++++++++++++++++----------------------
xen/include/asm-x86/microcode.h | 6 ++++++
2 files changed, 28 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index 22fdeca..ecec83b 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -134,14 +134,28 @@ static int collect_cpu_info(unsigned int cpu_num, struct cpu_signature *csig)
return 0;
}
-static inline int microcode_update_match(
- unsigned int cpu_num, const struct microcode_header_intel *mc_header,
- int sig, int pf)
+static enum microcode_match_result microcode_update_match(
+ const struct microcode_header_intel *mc_header, unsigned int sig,
+ unsigned int pf, unsigned int rev)
{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
+ const struct extended_sigtable *ext_header;
+ const struct extended_signature *ext_sig;
+ unsigned long data_size = get_datasize(mc_header);
+ unsigned int i;
+
+ if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
+ return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
- return (sigmatch(sig, uci->cpu_sig.sig, pf, uci->cpu_sig.pf) &&
- (mc_header->rev > uci->cpu_sig.rev));
+ if ( get_totalsize(mc_header) == (data_size + MC_HEADER_SIZE) )
+ return MIS_UCODE;
+
+ ext_header = (const void *)(mc_header + 1) + data_size;
+ ext_sig = (const void *)(ext_header + 1);
+ for ( i = 0; i < ext_header->count; i++ )
+ if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
+ return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
+
+ return MIS_UCODE;
}
static int microcode_sanity_check(void *mc)
@@ -243,31 +257,13 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
{
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
const struct microcode_header_intel *mc_header = mc;
- const struct extended_sigtable *ext_header;
unsigned long total_size = get_totalsize(mc_header);
- int ext_sigcount, i;
- struct extended_signature *ext_sig;
void *new_mc;
- if ( microcode_update_match(cpu, mc_header,
- mc_header->sig, mc_header->pf) )
- goto find;
-
- if ( total_size <= (get_datasize(mc_header) + MC_HEADER_SIZE) )
+ if ( microcode_update_match(mc, uci->cpu_sig.sig, uci->cpu_sig.pf,
+ uci->cpu_sig.rev) != NEW_UCODE )
return 0;
- ext_header = mc + get_datasize(mc_header) + MC_HEADER_SIZE;
- ext_sigcount = ext_header->count;
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
- for ( i = 0; i < ext_sigcount; i++ )
- {
- if ( microcode_update_match(cpu, mc_header,
- ext_sig->sig, ext_sig->pf) )
- goto find;
- ext_sig++;
- }
- return 0;
- find:
pr_debug("microcode: CPU%d found a matching microcode update with"
" version %#x (current=%#x)\n",
cpu, mc_header->rev, uci->cpu_sig.rev);
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index 23ea954..73ebe9a 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -3,6 +3,12 @@
#include <xen/percpu.h>
+enum microcode_match_result {
+ OLD_UCODE, /* signature matched, but revision id isn't newer */
+ NEW_UCODE, /* signature matched, but revision id is newer */
+ MIS_UCODE, /* signature mismatched */
+};
+
struct cpu_signature;
struct ucode_cpu_info;
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 03/10] microcode: introduce a global cache of ucode patch
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
to replace the current per-cpu cache 'uci->mc'.
With the assumption that all CPUs in the system have the same signature
(family, model, stepping and 'pf'), one microcode update matches with
one cpu should match with others. And Having multiple microcode revisions
on different cpus would cause system unstable and is what should be
avoided. Hence, caching only one microcode update is good enough for all
cases.
Introduce a global variable, microcode_cache, to store the newest
matching microcode update. Whenever we get a new valid microcode update,
its revision id is compared against that of the microcode update to
determine whether the "microcode_cache" needs to be replaced. And now
this global cache is loaded to cpu in apply_microcode().
All operations on the cache is expected to be done with the
'microcode_mutex' hold.
Note that I deliberately avoid touching 'uci->mc' as I am going to
remove it completely in the next patch.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- reworked to cache only one microcode patch rather than a list of
microcode patches.
Changes in v6:
- constify local variables and function parameters if possible
- comment that the global cache is protected by 'microcode_mutex'.
and add assertions to catch violations in microcode_{save/find}_patch()
Changes in v5:
- reword the commit description
- find_patch() and save_patch() are abstracted into common functions
with some hooks for AMD and Intel
---
xen/arch/x86/microcode.c | 36 ++++++++++++++++
xen/arch/x86/microcode_amd.c | 91 +++++++++++++++++++++++++++++++++++++----
xen/arch/x86/microcode_intel.c | 77 +++++++++++++++++++++++++++-------
xen/include/asm-x86/microcode.h | 15 +++++++
4 files changed, 197 insertions(+), 22 deletions(-)
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 4163f50..cff86a9 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -61,6 +61,9 @@ static struct ucode_mod_blob __initdata ucode_blob;
*/
static bool_t __initdata ucode_scan;
+/* Protected by microcode_mutex */
+static struct microcode_patch *microcode_cache;
+
void __init microcode_set_module(unsigned int idx)
{
ucode_mod_idx = idx;
@@ -262,6 +265,39 @@ int microcode_resume_cpu(unsigned int cpu)
return err;
}
+const struct microcode_patch *microcode_get_cache(void)
+{
+ ASSERT(spin_is_locked(µcode_mutex));
+
+ return microcode_cache;
+}
+
+/* Return true if cache gets updated. Otherwise, return false */
+bool microcode_update_cache(struct microcode_patch *patch)
+{
+
+ ASSERT(spin_is_locked(µcode_mutex));
+
+ if ( !microcode_ops->match_cpu(patch) )
+ return false;
+
+ if ( !microcode_cache )
+ microcode_cache = patch;
+ else if ( microcode_ops->compare_patch(patch, microcode_cache) ==
+ NEW_UCODE )
+ {
+ microcode_ops->free_patch(microcode_cache);
+ microcode_cache = patch;
+ }
+ else
+ {
+ microcode_ops->free_patch(patch);
+ return false;
+ }
+
+ return true;
+}
+
static int microcode_update_cpu(const void *buf, size_t size)
{
int err;
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 7a854c0..1f05899 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -190,24 +190,85 @@ static bool_t microcode_fits(const struct microcode_amd *mc_amd,
return 1;
}
+static bool match_cpu(const struct microcode_patch *patch)
+{
+ if ( !patch )
+ return false;
+ return microcode_fits(patch->mc_amd, smp_processor_id());
+}
+
+static struct microcode_patch *alloc_microcode_patch(
+ const struct microcode_amd *mc_amd)
+{
+ struct microcode_patch *microcode_patch = xmalloc(struct microcode_patch);
+ struct microcode_amd *cache = xmalloc(struct microcode_amd);
+ void *mpb = xmalloc_bytes(mc_amd->mpb_size);
+ struct equiv_cpu_entry *equiv_cpu_table =
+ xmalloc_bytes(mc_amd->equiv_cpu_table_size);
+
+ if ( !microcode_patch || !cache || !mpb || !equiv_cpu_table )
+ {
+ xfree(microcode_patch);
+ xfree(cache);
+ xfree(mpb);
+ xfree(equiv_cpu_table);
+ printk(XENLOG_ERR "microcode: Can not allocate memory\n");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ cache->equiv_cpu_table = equiv_cpu_table;
+ cache->mpb = mpb;
+ memcpy(cache->equiv_cpu_table, mc_amd->equiv_cpu_table,
+ mc_amd->equiv_cpu_table_size);
+ memcpy(cache->mpb, mc_amd->mpb, mc_amd->mpb_size);
+ cache->equiv_cpu_table_size = mc_amd->equiv_cpu_table_size;
+ cache->mpb_size = mc_amd->mpb_size;
+ microcode_patch->mc_amd = cache;
+
+ return microcode_patch;
+}
+
+static void free_patch(struct microcode_patch *microcode_patch)
+{
+ struct microcode_amd *mc_amd = microcode_patch->mc_amd;
+
+ xfree(mc_amd->equiv_cpu_table);
+ xfree(mc_amd->mpb);
+ xfree(mc_amd);
+ xfree(microcode_patch);
+}
+
+static enum microcode_match_result compare_patch(
+ const struct microcode_patch *new, const struct microcode_patch *old)
+{
+ const struct microcode_amd *new_mc = new->mc_amd;
+ const struct microcode_header_amd *new_header = new_mc->mpb;
+ const struct microcode_amd *old_mc = old->mc_amd;
+ const struct microcode_header_amd *old_header = old_mc->mpb;
+
+ if ( new_header->processor_rev_id == old_header->processor_rev_id )
+ return (new_header->patch_id > old_header->patch_id) ?
+ NEW_UCODE : OLD_UCODE;
+
+ return MIS_UCODE;
+}
+
static int apply_microcode(unsigned int cpu)
{
unsigned long flags;
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
uint32_t rev;
- struct microcode_amd *mc_amd = uci->mc.mc_amd;
- struct microcode_header_amd *hdr;
int hw_err;
+ const struct microcode_header_amd *hdr;
+ const struct microcode_patch *patch = microcode_get_cache();
/* We should bind the task to the CPU */
BUG_ON(raw_smp_processor_id() != cpu);
- if ( mc_amd == NULL )
+ if ( !match_cpu(patch) )
return -EINVAL;
- hdr = mc_amd->mpb;
- if ( hdr == NULL )
- return -EINVAL;
+ hdr = patch->mc_amd->mpb;
spin_lock_irqsave(µcode_update_lock, flags);
@@ -497,7 +558,20 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
&offset)) == 0 )
{
- if ( microcode_fits(mc_amd, cpu) )
+ struct microcode_patch *new_patch = alloc_microcode_patch(mc_amd);
+
+ if ( IS_ERR(new_patch) )
+ {
+ error = PTR_ERR(new_patch);
+ break;
+ }
+
+ if ( match_cpu(new_patch) )
+ microcode_update_cache(new_patch);
+ else
+ free_patch(new_patch);
+
+ if ( match_cpu(microcode_get_cache()) )
{
error = apply_microcode(cpu);
if ( error )
@@ -639,6 +713,9 @@ static const struct microcode_ops microcode_amd_ops = {
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
.start_update = start_update,
+ .free_patch = free_patch,
+ .compare_patch = compare_patch,
+ .match_cpu = match_cpu,
};
int __init microcode_init_amd(void)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index ecec83b..d3405a0 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -248,6 +248,32 @@ static int microcode_sanity_check(void *mc)
return 0;
}
+static bool match_cpu(const struct microcode_patch *patch)
+{
+ const struct ucode_cpu_info *uci = &this_cpu(ucode_cpu_info);
+
+ if ( !patch )
+ return false;
+
+ return microcode_update_match(&patch->mc_intel->hdr, uci->cpu_sig.sig,
+ uci->cpu_sig.pf, uci->cpu_sig.rev) == NEW_UCODE;
+}
+
+static void free_patch(struct microcode_patch *patch)
+{
+ xfree(patch->mc_intel);
+ xfree(patch);
+}
+
+static enum microcode_match_result compare_patch(
+ const struct microcode_patch *new, const struct microcode_patch *old)
+{
+ const struct microcode_header_intel *old_header = &old->mc_intel->hdr;
+
+ return microcode_update_match(&new->mc_intel->hdr, old_header->sig,
+ old_header->pf, old_header->rev);
+}
+
/*
* return 0 - no update found
* return 1 - found update
@@ -258,10 +284,26 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
- void *new_mc;
+ void *new_mc = xmalloc_bytes(total_size);
+ struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
+
+ if ( !new_patch || !new_mc )
+ {
+ xfree(new_patch);
+ xfree(new_mc);
+ printk(XENLOG_ERR "microcode: Can not allocate memory\n");
+ return -ENOMEM;
+ }
+ memcpy(new_mc, mc, total_size);
+ new_patch->mc_intel = new_mc;
- if ( microcode_update_match(mc, uci->cpu_sig.sig, uci->cpu_sig.pf,
- uci->cpu_sig.rev) != NEW_UCODE )
+ if ( !match_cpu(new_patch) )
+ {
+ free_patch(new_patch);
+ return 0;
+ }
+
+ if ( !microcode_update_cache(new_patch) )
return 0;
pr_debug("microcode: CPU%d found a matching microcode update with"
@@ -277,6 +319,7 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
memcpy(new_mc, mc, total_size);
xfree(uci->mc.mc_intel);
uci->mc.mc_intel = new_mc;
+
return 1;
}
@@ -287,18 +330,22 @@ static int apply_microcode(unsigned int cpu)
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
+ const struct microcode_intel *mc_intel;
+ const struct microcode_patch *patch = microcode_get_cache();
/* We should bind the task to the CPU */
BUG_ON(cpu_num != cpu);
- if ( uci->mc.mc_intel == NULL )
+ if ( !match_cpu(patch) )
return -EINVAL;
+ mc_intel = patch->mc_intel;
+
/* serialize access to the physical write to MSR 0x79 */
spin_lock_irqsave(µcode_update_lock, flags);
/* write microcode via MSR 0x79 */
- wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)uci->mc.mc_intel->bits);
+ wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc_intel->bits);
wrmsrl(MSR_IA32_UCODE_REV, 0x0ULL);
/* As documented in the SDM: Do a CPUID 1 here */
@@ -309,19 +356,19 @@ static int apply_microcode(unsigned int cpu)
val[1] = (uint32_t)(msr_content >> 32);
spin_unlock_irqrestore(µcode_update_lock, flags);
- if ( val[1] != uci->mc.mc_intel->hdr.rev )
+ if ( val[1] != mc_intel->hdr.rev )
{
printk(KERN_ERR "microcode: CPU%d update from revision "
"%#x to %#x failed. Resulting revision is %#x.\n", cpu_num,
- uci->cpu_sig.rev, uci->mc.mc_intel->hdr.rev, val[1]);
+ uci->cpu_sig.rev, mc_intel->hdr.rev, val[1]);
return -EIO;
}
printk(KERN_INFO "microcode: CPU%d updated from revision "
"%#x to %#x, date = %04x-%02x-%02x \n",
cpu_num, uci->cpu_sig.rev, val[1],
- uci->mc.mc_intel->hdr.year,
- uci->mc.mc_intel->hdr.month,
- uci->mc.mc_intel->hdr.day);
+ mc_intel->hdr.year,
+ mc_intel->hdr.month,
+ mc_intel->hdr.day);
uci->cpu_sig.rev = val[1];
return 0;
@@ -361,7 +408,6 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
long offset = 0;
int error = 0;
void *mc;
- unsigned int matching_count = 0;
/* We should bind the task to the CPU */
BUG_ON(cpu != raw_smp_processor_id());
@@ -379,10 +425,8 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
* lets keep searching till the latest version
*/
if ( error == 1 )
- {
- matching_count++;
error = 0;
- }
+
xfree(mc);
}
if ( offset > 0 )
@@ -390,7 +434,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
if ( offset < 0 )
error = offset;
- if ( !error && matching_count )
+ if ( !error && match_cpu(microcode_get_cache()) )
error = apply_microcode(cpu);
return error;
@@ -406,6 +450,9 @@ static const struct microcode_ops microcode_intel_ops = {
.cpu_request_microcode = cpu_request_microcode,
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
+ .free_patch = free_patch,
+ .compare_patch = compare_patch,
+ .match_cpu = match_cpu,
};
int __init microcode_init_intel(void)
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index 73ebe9a..6541c58 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -12,6 +12,13 @@ enum microcode_match_result {
struct cpu_signature;
struct ucode_cpu_info;
+struct microcode_patch {
+ union {
+ struct microcode_intel *mc_intel;
+ struct microcode_amd *mc_amd;
+ };
+};
+
struct microcode_ops {
int (*microcode_resume_match)(unsigned int cpu, const void *mc);
int (*cpu_request_microcode)(unsigned int cpu, const void *buf,
@@ -19,6 +26,11 @@ struct microcode_ops {
int (*collect_cpu_info)(unsigned int cpu, struct cpu_signature *csig);
int (*apply_microcode)(unsigned int cpu);
int (*start_update)(void);
+ void (*free_patch)(struct microcode_patch *patch);
+ bool (*match_cpu)(const struct microcode_patch *patch);
+ enum microcode_match_result (*compare_patch)(
+ const struct microcode_patch *new,
+ const struct microcode_patch *old);
};
struct cpu_signature {
@@ -39,4 +51,7 @@ struct ucode_cpu_info {
DECLARE_PER_CPU(struct ucode_cpu_info, ucode_cpu_info);
extern const struct microcode_ops *microcode_ops;
+const struct microcode_patch *microcode_get_cache(void);
+bool microcode_update_cache(struct microcode_patch *patch);
+
#endif /* ASM_X86__MICROCODE_H */
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 03/10] microcode: introduce a global cache of ucode patch
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
to replace the current per-cpu cache 'uci->mc'.
With the assumption that all CPUs in the system have the same signature
(family, model, stepping and 'pf'), one microcode update matches with
one cpu should match with others. And Having multiple microcode revisions
on different cpus would cause system unstable and is what should be
avoided. Hence, caching only one microcode update is good enough for all
cases.
Introduce a global variable, microcode_cache, to store the newest
matching microcode update. Whenever we get a new valid microcode update,
its revision id is compared against that of the microcode update to
determine whether the "microcode_cache" needs to be replaced. And now
this global cache is loaded to cpu in apply_microcode().
All operations on the cache is expected to be done with the
'microcode_mutex' hold.
Note that I deliberately avoid touching 'uci->mc' as I am going to
remove it completely in the next patch.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- reworked to cache only one microcode patch rather than a list of
microcode patches.
Changes in v6:
- constify local variables and function parameters if possible
- comment that the global cache is protected by 'microcode_mutex'.
and add assertions to catch violations in microcode_{save/find}_patch()
Changes in v5:
- reword the commit description
- find_patch() and save_patch() are abstracted into common functions
with some hooks for AMD and Intel
---
xen/arch/x86/microcode.c | 36 ++++++++++++++++
xen/arch/x86/microcode_amd.c | 91 +++++++++++++++++++++++++++++++++++++----
xen/arch/x86/microcode_intel.c | 77 +++++++++++++++++++++++++++-------
xen/include/asm-x86/microcode.h | 15 +++++++
4 files changed, 197 insertions(+), 22 deletions(-)
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 4163f50..cff86a9 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -61,6 +61,9 @@ static struct ucode_mod_blob __initdata ucode_blob;
*/
static bool_t __initdata ucode_scan;
+/* Protected by microcode_mutex */
+static struct microcode_patch *microcode_cache;
+
void __init microcode_set_module(unsigned int idx)
{
ucode_mod_idx = idx;
@@ -262,6 +265,39 @@ int microcode_resume_cpu(unsigned int cpu)
return err;
}
+const struct microcode_patch *microcode_get_cache(void)
+{
+ ASSERT(spin_is_locked(µcode_mutex));
+
+ return microcode_cache;
+}
+
+/* Return true if cache gets updated. Otherwise, return false */
+bool microcode_update_cache(struct microcode_patch *patch)
+{
+
+ ASSERT(spin_is_locked(µcode_mutex));
+
+ if ( !microcode_ops->match_cpu(patch) )
+ return false;
+
+ if ( !microcode_cache )
+ microcode_cache = patch;
+ else if ( microcode_ops->compare_patch(patch, microcode_cache) ==
+ NEW_UCODE )
+ {
+ microcode_ops->free_patch(microcode_cache);
+ microcode_cache = patch;
+ }
+ else
+ {
+ microcode_ops->free_patch(patch);
+ return false;
+ }
+
+ return true;
+}
+
static int microcode_update_cpu(const void *buf, size_t size)
{
int err;
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 7a854c0..1f05899 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -190,24 +190,85 @@ static bool_t microcode_fits(const struct microcode_amd *mc_amd,
return 1;
}
+static bool match_cpu(const struct microcode_patch *patch)
+{
+ if ( !patch )
+ return false;
+ return microcode_fits(patch->mc_amd, smp_processor_id());
+}
+
+static struct microcode_patch *alloc_microcode_patch(
+ const struct microcode_amd *mc_amd)
+{
+ struct microcode_patch *microcode_patch = xmalloc(struct microcode_patch);
+ struct microcode_amd *cache = xmalloc(struct microcode_amd);
+ void *mpb = xmalloc_bytes(mc_amd->mpb_size);
+ struct equiv_cpu_entry *equiv_cpu_table =
+ xmalloc_bytes(mc_amd->equiv_cpu_table_size);
+
+ if ( !microcode_patch || !cache || !mpb || !equiv_cpu_table )
+ {
+ xfree(microcode_patch);
+ xfree(cache);
+ xfree(mpb);
+ xfree(equiv_cpu_table);
+ printk(XENLOG_ERR "microcode: Can not allocate memory\n");
+ return ERR_PTR(-ENOMEM);
+ }
+
+ cache->equiv_cpu_table = equiv_cpu_table;
+ cache->mpb = mpb;
+ memcpy(cache->equiv_cpu_table, mc_amd->equiv_cpu_table,
+ mc_amd->equiv_cpu_table_size);
+ memcpy(cache->mpb, mc_amd->mpb, mc_amd->mpb_size);
+ cache->equiv_cpu_table_size = mc_amd->equiv_cpu_table_size;
+ cache->mpb_size = mc_amd->mpb_size;
+ microcode_patch->mc_amd = cache;
+
+ return microcode_patch;
+}
+
+static void free_patch(struct microcode_patch *microcode_patch)
+{
+ struct microcode_amd *mc_amd = microcode_patch->mc_amd;
+
+ xfree(mc_amd->equiv_cpu_table);
+ xfree(mc_amd->mpb);
+ xfree(mc_amd);
+ xfree(microcode_patch);
+}
+
+static enum microcode_match_result compare_patch(
+ const struct microcode_patch *new, const struct microcode_patch *old)
+{
+ const struct microcode_amd *new_mc = new->mc_amd;
+ const struct microcode_header_amd *new_header = new_mc->mpb;
+ const struct microcode_amd *old_mc = old->mc_amd;
+ const struct microcode_header_amd *old_header = old_mc->mpb;
+
+ if ( new_header->processor_rev_id == old_header->processor_rev_id )
+ return (new_header->patch_id > old_header->patch_id) ?
+ NEW_UCODE : OLD_UCODE;
+
+ return MIS_UCODE;
+}
+
static int apply_microcode(unsigned int cpu)
{
unsigned long flags;
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
uint32_t rev;
- struct microcode_amd *mc_amd = uci->mc.mc_amd;
- struct microcode_header_amd *hdr;
int hw_err;
+ const struct microcode_header_amd *hdr;
+ const struct microcode_patch *patch = microcode_get_cache();
/* We should bind the task to the CPU */
BUG_ON(raw_smp_processor_id() != cpu);
- if ( mc_amd == NULL )
+ if ( !match_cpu(patch) )
return -EINVAL;
- hdr = mc_amd->mpb;
- if ( hdr == NULL )
- return -EINVAL;
+ hdr = patch->mc_amd->mpb;
spin_lock_irqsave(µcode_update_lock, flags);
@@ -497,7 +558,20 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
&offset)) == 0 )
{
- if ( microcode_fits(mc_amd, cpu) )
+ struct microcode_patch *new_patch = alloc_microcode_patch(mc_amd);
+
+ if ( IS_ERR(new_patch) )
+ {
+ error = PTR_ERR(new_patch);
+ break;
+ }
+
+ if ( match_cpu(new_patch) )
+ microcode_update_cache(new_patch);
+ else
+ free_patch(new_patch);
+
+ if ( match_cpu(microcode_get_cache()) )
{
error = apply_microcode(cpu);
if ( error )
@@ -639,6 +713,9 @@ static const struct microcode_ops microcode_amd_ops = {
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
.start_update = start_update,
+ .free_patch = free_patch,
+ .compare_patch = compare_patch,
+ .match_cpu = match_cpu,
};
int __init microcode_init_amd(void)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index ecec83b..d3405a0 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -248,6 +248,32 @@ static int microcode_sanity_check(void *mc)
return 0;
}
+static bool match_cpu(const struct microcode_patch *patch)
+{
+ const struct ucode_cpu_info *uci = &this_cpu(ucode_cpu_info);
+
+ if ( !patch )
+ return false;
+
+ return microcode_update_match(&patch->mc_intel->hdr, uci->cpu_sig.sig,
+ uci->cpu_sig.pf, uci->cpu_sig.rev) == NEW_UCODE;
+}
+
+static void free_patch(struct microcode_patch *patch)
+{
+ xfree(patch->mc_intel);
+ xfree(patch);
+}
+
+static enum microcode_match_result compare_patch(
+ const struct microcode_patch *new, const struct microcode_patch *old)
+{
+ const struct microcode_header_intel *old_header = &old->mc_intel->hdr;
+
+ return microcode_update_match(&new->mc_intel->hdr, old_header->sig,
+ old_header->pf, old_header->rev);
+}
+
/*
* return 0 - no update found
* return 1 - found update
@@ -258,10 +284,26 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
- void *new_mc;
+ void *new_mc = xmalloc_bytes(total_size);
+ struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
+
+ if ( !new_patch || !new_mc )
+ {
+ xfree(new_patch);
+ xfree(new_mc);
+ printk(XENLOG_ERR "microcode: Can not allocate memory\n");
+ return -ENOMEM;
+ }
+ memcpy(new_mc, mc, total_size);
+ new_patch->mc_intel = new_mc;
- if ( microcode_update_match(mc, uci->cpu_sig.sig, uci->cpu_sig.pf,
- uci->cpu_sig.rev) != NEW_UCODE )
+ if ( !match_cpu(new_patch) )
+ {
+ free_patch(new_patch);
+ return 0;
+ }
+
+ if ( !microcode_update_cache(new_patch) )
return 0;
pr_debug("microcode: CPU%d found a matching microcode update with"
@@ -277,6 +319,7 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
memcpy(new_mc, mc, total_size);
xfree(uci->mc.mc_intel);
uci->mc.mc_intel = new_mc;
+
return 1;
}
@@ -287,18 +330,22 @@ static int apply_microcode(unsigned int cpu)
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
+ const struct microcode_intel *mc_intel;
+ const struct microcode_patch *patch = microcode_get_cache();
/* We should bind the task to the CPU */
BUG_ON(cpu_num != cpu);
- if ( uci->mc.mc_intel == NULL )
+ if ( !match_cpu(patch) )
return -EINVAL;
+ mc_intel = patch->mc_intel;
+
/* serialize access to the physical write to MSR 0x79 */
spin_lock_irqsave(µcode_update_lock, flags);
/* write microcode via MSR 0x79 */
- wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)uci->mc.mc_intel->bits);
+ wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc_intel->bits);
wrmsrl(MSR_IA32_UCODE_REV, 0x0ULL);
/* As documented in the SDM: Do a CPUID 1 here */
@@ -309,19 +356,19 @@ static int apply_microcode(unsigned int cpu)
val[1] = (uint32_t)(msr_content >> 32);
spin_unlock_irqrestore(µcode_update_lock, flags);
- if ( val[1] != uci->mc.mc_intel->hdr.rev )
+ if ( val[1] != mc_intel->hdr.rev )
{
printk(KERN_ERR "microcode: CPU%d update from revision "
"%#x to %#x failed. Resulting revision is %#x.\n", cpu_num,
- uci->cpu_sig.rev, uci->mc.mc_intel->hdr.rev, val[1]);
+ uci->cpu_sig.rev, mc_intel->hdr.rev, val[1]);
return -EIO;
}
printk(KERN_INFO "microcode: CPU%d updated from revision "
"%#x to %#x, date = %04x-%02x-%02x \n",
cpu_num, uci->cpu_sig.rev, val[1],
- uci->mc.mc_intel->hdr.year,
- uci->mc.mc_intel->hdr.month,
- uci->mc.mc_intel->hdr.day);
+ mc_intel->hdr.year,
+ mc_intel->hdr.month,
+ mc_intel->hdr.day);
uci->cpu_sig.rev = val[1];
return 0;
@@ -361,7 +408,6 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
long offset = 0;
int error = 0;
void *mc;
- unsigned int matching_count = 0;
/* We should bind the task to the CPU */
BUG_ON(cpu != raw_smp_processor_id());
@@ -379,10 +425,8 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
* lets keep searching till the latest version
*/
if ( error == 1 )
- {
- matching_count++;
error = 0;
- }
+
xfree(mc);
}
if ( offset > 0 )
@@ -390,7 +434,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
if ( offset < 0 )
error = offset;
- if ( !error && matching_count )
+ if ( !error && match_cpu(microcode_get_cache()) )
error = apply_microcode(cpu);
return error;
@@ -406,6 +450,9 @@ static const struct microcode_ops microcode_intel_ops = {
.cpu_request_microcode = cpu_request_microcode,
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
+ .free_patch = free_patch,
+ .compare_patch = compare_patch,
+ .match_cpu = match_cpu,
};
int __init microcode_init_intel(void)
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index 73ebe9a..6541c58 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -12,6 +12,13 @@ enum microcode_match_result {
struct cpu_signature;
struct ucode_cpu_info;
+struct microcode_patch {
+ union {
+ struct microcode_intel *mc_intel;
+ struct microcode_amd *mc_amd;
+ };
+};
+
struct microcode_ops {
int (*microcode_resume_match)(unsigned int cpu, const void *mc);
int (*cpu_request_microcode)(unsigned int cpu, const void *buf,
@@ -19,6 +26,11 @@ struct microcode_ops {
int (*collect_cpu_info)(unsigned int cpu, struct cpu_signature *csig);
int (*apply_microcode)(unsigned int cpu);
int (*start_update)(void);
+ void (*free_patch)(struct microcode_patch *patch);
+ bool (*match_cpu)(const struct microcode_patch *patch);
+ enum microcode_match_result (*compare_patch)(
+ const struct microcode_patch *new,
+ const struct microcode_patch *old);
};
struct cpu_signature {
@@ -39,4 +51,7 @@ struct ucode_cpu_info {
DECLARE_PER_CPU(struct ucode_cpu_info, ucode_cpu_info);
extern const struct microcode_ops *microcode_ops;
+const struct microcode_patch *microcode_get_cache(void);
+bool microcode_update_cache(struct microcode_patch *patch);
+
#endif /* ASM_X86__MICROCODE_H */
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 04/10] microcode: remove struct ucode_cpu_info
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
We can remove the per-cpu cache field in struct ucode_cpu_info since
it has been replaced by a global cache. It would leads to only one field
remaining in ucode_cpu_info. Then, this struct is removed and the
remaining field (cpu signature) is stored in per-cpu area.
Also remove 'microcode_resume_match' from microcode_ops because the
check is done in find_patch(). The cpu status notifier is also
removed. It was used to free the "mc" field to avoid memory leak.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v6:
- remove the whole struct ucode_cpu_info instead of the per-cpu cache
in it.
---
xen/arch/x86/apic.c | 2 +-
xen/arch/x86/microcode.c | 91 +++---------------------------------
xen/arch/x86/microcode_amd.c | 100 +++++-----------------------------------
xen/arch/x86/microcode_intel.c | 33 ++++---------
xen/arch/x86/spec_ctrl.c | 2 +-
xen/include/asm-x86/microcode.h | 13 +-----
6 files changed, 30 insertions(+), 211 deletions(-)
diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index fafc0bd..d216455 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -1188,7 +1188,7 @@ static void __init check_deadline_errata(void)
else
rev = (unsigned long)m->driver_data;
- if ( this_cpu(ucode_cpu_info).cpu_sig.rev >= rev )
+ if ( this_cpu(cpu_sig).rev >= rev )
return;
setup_clear_cpu_cap(X86_FEATURE_TSC_DEADLINE);
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index cff86a9..0c01dfa 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -187,7 +187,7 @@ const struct microcode_ops *microcode_ops;
static DEFINE_SPINLOCK(microcode_mutex);
-DEFINE_PER_CPU(struct ucode_cpu_info, ucode_cpu_info);
+DEFINE_PER_CPU(struct cpu_signature, cpu_sig);
struct microcode_info {
unsigned int cpu;
@@ -196,70 +196,19 @@ struct microcode_info {
char buffer[1];
};
-static void __microcode_fini_cpu(unsigned int cpu)
-{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
-
- xfree(uci->mc.mc_valid);
- memset(uci, 0, sizeof(*uci));
-}
-
-static void microcode_fini_cpu(unsigned int cpu)
-{
- spin_lock(µcode_mutex);
- __microcode_fini_cpu(cpu);
- spin_unlock(µcode_mutex);
-}
-
int microcode_resume_cpu(unsigned int cpu)
{
int err;
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
- struct cpu_signature nsig;
- unsigned int cpu2;
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
if ( !microcode_ops )
return 0;
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, &uci->cpu_sig);
- if ( err )
- {
- __microcode_fini_cpu(cpu);
- spin_unlock(µcode_mutex);
- return err;
- }
-
- if ( uci->mc.mc_valid )
- {
- err = microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid);
- if ( err >= 0 )
- {
- if ( err )
- err = microcode_ops->apply_microcode(cpu);
- spin_unlock(µcode_mutex);
- return err;
- }
- }
-
- nsig = uci->cpu_sig;
- __microcode_fini_cpu(cpu);
- uci->cpu_sig = nsig;
-
- err = -EIO;
- for_each_online_cpu ( cpu2 )
- {
- uci = &per_cpu(ucode_cpu_info, cpu2);
- if ( uci->mc.mc_valid &&
- microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid) > 0 )
- {
- err = microcode_ops->apply_microcode(cpu);
- break;
- }
- }
-
- __microcode_fini_cpu(cpu);
+ err = microcode_ops->collect_cpu_info(cpu, sig);
+ if ( likely(!err) )
+ err = microcode_ops->apply_microcode(cpu);
spin_unlock(µcode_mutex);
return err;
@@ -302,16 +251,13 @@ static int microcode_update_cpu(const void *buf, size_t size)
{
int err;
unsigned int cpu = smp_processor_id();
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, &uci->cpu_sig);
+ err = microcode_ops->collect_cpu_info(cpu, sig);
if ( likely(!err) )
err = microcode_ops->cpu_request_microcode(cpu, buf, size);
- else
- __microcode_fini_cpu(cpu);
-
spin_unlock(µcode_mutex);
return err;
@@ -398,25 +344,6 @@ static int __init microcode_init(void)
}
__initcall(microcode_init);
-static int microcode_percpu_callback(
- struct notifier_block *nfb, unsigned long action, void *hcpu)
-{
- unsigned int cpu = (unsigned long)hcpu;
-
- switch ( action )
- {
- case CPU_DEAD:
- microcode_fini_cpu(cpu);
- break;
- }
-
- return NOTIFY_DONE;
-}
-
-static struct notifier_block microcode_percpu_nfb = {
- .notifier_call = microcode_percpu_callback,
-};
-
int __init early_microcode_update_cpu(bool start_update)
{
int rc = 0;
@@ -460,12 +387,8 @@ int __init early_microcode_init(void)
return rc;
if ( microcode_ops )
- {
if ( ucode_mod.mod_end || ucode_blob.size )
rc = early_microcode_update_cpu(true);
- register_cpu_notifier(µcode_percpu_nfb);
- }
-
return rc;
}
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 1f05899..93af2c9 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -155,7 +155,7 @@ static bool_t find_equiv_cpu_id(const struct equiv_cpu_entry *equiv_cpu_table,
static bool_t microcode_fits(const struct microcode_amd *mc_amd,
unsigned int cpu)
{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *mc_header = mc_amd->mpb;
const struct equiv_cpu_entry *equiv_cpu_table = mc_amd->equiv_cpu_table;
unsigned int current_cpu_id;
@@ -178,14 +178,14 @@ static bool_t microcode_fits(const struct microcode_amd *mc_amd,
return 0;
}
- if ( mc_header->patch_id <= uci->cpu_sig.rev )
+ if ( mc_header->patch_id <= sig->rev )
{
pr_debug("microcode: patch is already at required level or greater.\n");
return 0;
}
pr_debug("microcode: CPU%d found a matching microcode update with version %#x (current=%#x)\n",
- cpu, mc_header->patch_id, uci->cpu_sig.rev);
+ cpu, mc_header->patch_id, sig->rev);
return 1;
}
@@ -256,9 +256,9 @@ static enum microcode_match_result compare_patch(
static int apply_microcode(unsigned int cpu)
{
unsigned long flags;
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
uint32_t rev;
int hw_err;
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *hdr;
const struct microcode_patch *patch = microcode_get_cache();
@@ -294,9 +294,9 @@ static int apply_microcode(unsigned int cpu)
}
printk(KERN_WARNING "microcode: CPU%d updated from revision %#x to %#x\n",
- cpu, uci->cpu_sig.rev, hdr->patch_id);
+ cpu, sig->rev, hdr->patch_id);
- uci->cpu_sig.rev = rev;
+ sig->rev = rev;
return 0;
}
@@ -442,14 +442,14 @@ static bool_t check_final_patch_levels(unsigned int cpu)
* any of the 'final_levels', then we should not update the microcode
* patch on the cpu as system will hang otherwise.
*/
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
unsigned int i;
if ( boot_cpu_data.x86 != 0x10 )
return 0;
for ( i = 0; i < ARRAY_SIZE(final_levels); i++ )
- if ( uci->cpu_sig.rev == final_levels[i] )
+ if ( sig->rev == final_levels[i] )
return 1;
return 0;
@@ -458,13 +458,12 @@ static bool_t check_final_patch_levels(unsigned int cpu)
static int cpu_request_microcode(unsigned int cpu, const void *buf,
size_t bufsize)
{
- struct microcode_amd *mc_amd, *mc_old;
+ struct microcode_amd *mc_amd;
size_t offset = 0;
- size_t last_offset, applied_offset = 0;
- int error = 0, save_error = 1;
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ int error = 0;
unsigned int current_cpu_id;
unsigned int equiv_cpu_id;
+ const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
/* We should bind the task to the CPU */
BUG_ON(cpu != raw_smp_processor_id());
@@ -533,7 +532,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
{
printk(KERN_ERR "microcode: CPU%d incorrect or corrupt container file\n"
"microcode: Failed to update patch level. "
- "Current lvl:%#x\n", cpu, uci->cpu_sig.rev);
+ "Current lvl:%#x\n", cpu, sig->rev);
break;
}
}
@@ -544,17 +543,12 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
goto out;
}
- mc_old = uci->mc.mc_amd;
- /* implicitely validates uci->mc.mc_valid */
- uci->mc.mc_amd = mc_amd;
-
/*
* It's possible the data file has multiple matching ucode,
* lets keep searching till the latest version
*/
mc_amd->mpb = NULL;
mc_amd->mpb_size = 0;
- last_offset = offset;
while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
&offset)) == 0 )
{
@@ -576,11 +570,8 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
error = apply_microcode(cpu);
if ( error )
break;
- applied_offset = last_offset;
}
- last_offset = offset;
-
if ( offset >= bufsize )
break;
@@ -609,26 +600,6 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
break;
}
- /* On success keep the microcode patch for
- * re-apply on resume.
- */
- if ( applied_offset )
- {
- save_error = get_ucode_from_buffer_amd(
- mc_amd, buf, bufsize, &applied_offset);
-
- if ( save_error )
- error = save_error;
- }
-
- if ( save_error )
- {
- xfree(mc_amd);
- uci->mc.mc_amd = mc_old;
- }
- else
- xfree(mc_old);
-
out:
#if CONFIG_HVM
svm_host_osvw_init();
@@ -643,52 +614,6 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
return error;
}
-static int microcode_resume_match(unsigned int cpu, const void *mc)
-{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
- struct microcode_amd *mc_amd = uci->mc.mc_amd;
- const struct microcode_amd *src = mc;
-
- if ( !microcode_fits(src, cpu) )
- return 0;
-
- if ( src != mc_amd )
- {
- if ( mc_amd )
- {
- xfree(mc_amd->equiv_cpu_table);
- xfree(mc_amd->mpb);
- xfree(mc_amd);
- }
-
- mc_amd = xmalloc(struct microcode_amd);
- uci->mc.mc_amd = mc_amd;
- if ( !mc_amd )
- return -ENOMEM;
- mc_amd->equiv_cpu_table = xmalloc_bytes(src->equiv_cpu_table_size);
- if ( !mc_amd->equiv_cpu_table )
- goto err1;
- mc_amd->mpb = xmalloc_bytes(src->mpb_size);
- if ( !mc_amd->mpb )
- goto err2;
-
- mc_amd->equiv_cpu_table_size = src->equiv_cpu_table_size;
- mc_amd->mpb_size = src->mpb_size;
- memcpy(mc_amd->mpb, src->mpb, src->mpb_size);
- memcpy(mc_amd->equiv_cpu_table, src->equiv_cpu_table,
- src->equiv_cpu_table_size);
- }
-
- return 1;
-
-err2:
- xfree(mc_amd->equiv_cpu_table);
-err1:
- xfree(mc_amd);
- uci->mc.mc_amd = NULL;
- return -ENOMEM;
-}
-
static int start_update(void)
{
#if CONFIG_HVM
@@ -708,7 +633,6 @@ static int start_update(void)
}
static const struct microcode_ops microcode_amd_ops = {
- .microcode_resume_match = microcode_resume_match,
.cpu_request_microcode = cpu_request_microcode,
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index d3405a0..bf6497f 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -250,13 +250,13 @@ static int microcode_sanity_check(void *mc)
static bool match_cpu(const struct microcode_patch *patch)
{
- const struct ucode_cpu_info *uci = &this_cpu(ucode_cpu_info);
+ const struct cpu_signature *sig = &this_cpu(cpu_sig);
if ( !patch )
return false;
- return microcode_update_match(&patch->mc_intel->hdr, uci->cpu_sig.sig,
- uci->cpu_sig.pf, uci->cpu_sig.rev) == NEW_UCODE;
+ return microcode_update_match(&patch->mc_intel->hdr,
+ sig->sig, sig->pf, sig->rev) == NEW_UCODE;
}
static void free_patch(struct microcode_patch *patch)
@@ -281,7 +281,6 @@ static enum microcode_match_result compare_patch(
*/
static int get_matching_microcode(const void *mc, unsigned int cpu)
{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
void *new_mc = xmalloc_bytes(total_size);
@@ -308,17 +307,7 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
pr_debug("microcode: CPU%d found a matching microcode update with"
" version %#x (current=%#x)\n",
- cpu, mc_header->rev, uci->cpu_sig.rev);
- new_mc = xmalloc_bytes(total_size);
- if ( new_mc == NULL )
- {
- printk(KERN_ERR "microcode: error! Can not allocate memory\n");
- return -ENOMEM;
- }
-
- memcpy(new_mc, mc, total_size);
- xfree(uci->mc.mc_intel);
- uci->mc.mc_intel = new_mc;
+ cpu, mc_header->rev, this_cpu(cpu_sig).rev);
return 1;
}
@@ -329,7 +318,7 @@ static int apply_microcode(unsigned int cpu)
uint64_t msr_content;
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_intel *mc_intel;
const struct microcode_patch *patch = microcode_get_cache();
@@ -360,16 +349,16 @@ static int apply_microcode(unsigned int cpu)
{
printk(KERN_ERR "microcode: CPU%d update from revision "
"%#x to %#x failed. Resulting revision is %#x.\n", cpu_num,
- uci->cpu_sig.rev, mc_intel->hdr.rev, val[1]);
+ sig->rev, mc_intel->hdr.rev, val[1]);
return -EIO;
}
printk(KERN_INFO "microcode: CPU%d updated from revision "
"%#x to %#x, date = %04x-%02x-%02x \n",
- cpu_num, uci->cpu_sig.rev, val[1],
+ cpu_num, sig->rev, val[1],
mc_intel->hdr.year,
mc_intel->hdr.month,
mc_intel->hdr.day);
- uci->cpu_sig.rev = val[1];
+ sig->rev = val[1];
return 0;
}
@@ -440,13 +429,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
return error;
}
-static int microcode_resume_match(unsigned int cpu, const void *mc)
-{
- return get_matching_microcode(mc, cpu);
-}
-
static const struct microcode_ops microcode_intel_ops = {
- .microcode_resume_match = microcode_resume_match,
.cpu_request_microcode = cpu_request_microcode,
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 5d98cac..43128c3 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -436,7 +436,7 @@ static bool __init check_smt_enabled(void)
/* Calculate whether Retpoline is known-safe on this CPU. */
static bool __init retpoline_safe(uint64_t caps)
{
- unsigned int ucode_rev = this_cpu(ucode_cpu_info).cpu_sig.rev;
+ unsigned int ucode_rev = this_cpu(cpu_sig).rev;
if ( boot_cpu_data.x86_vendor == X86_VENDOR_AMD )
return true;
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index 6541c58..f2ac509 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -10,7 +10,6 @@ enum microcode_match_result {
};
struct cpu_signature;
-struct ucode_cpu_info;
struct microcode_patch {
union {
@@ -20,7 +19,6 @@ struct microcode_patch {
};
struct microcode_ops {
- int (*microcode_resume_match)(unsigned int cpu, const void *mc);
int (*cpu_request_microcode)(unsigned int cpu, const void *buf,
size_t size);
int (*collect_cpu_info)(unsigned int cpu, struct cpu_signature *csig);
@@ -39,16 +37,7 @@ struct cpu_signature {
unsigned int rev;
};
-struct ucode_cpu_info {
- struct cpu_signature cpu_sig;
- union {
- struct microcode_intel *mc_intel;
- struct microcode_amd *mc_amd;
- void *mc_valid;
- } mc;
-};
-
-DECLARE_PER_CPU(struct ucode_cpu_info, ucode_cpu_info);
+DECLARE_PER_CPU(struct cpu_signature, cpu_sig);
extern const struct microcode_ops *microcode_ops;
const struct microcode_patch *microcode_get_cache(void);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 04/10] microcode: remove struct ucode_cpu_info
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
We can remove the per-cpu cache field in struct ucode_cpu_info since
it has been replaced by a global cache. It would leads to only one field
remaining in ucode_cpu_info. Then, this struct is removed and the
remaining field (cpu signature) is stored in per-cpu area.
Also remove 'microcode_resume_match' from microcode_ops because the
check is done in find_patch(). The cpu status notifier is also
removed. It was used to free the "mc" field to avoid memory leak.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v6:
- remove the whole struct ucode_cpu_info instead of the per-cpu cache
in it.
---
xen/arch/x86/apic.c | 2 +-
xen/arch/x86/microcode.c | 91 +++---------------------------------
xen/arch/x86/microcode_amd.c | 100 +++++-----------------------------------
xen/arch/x86/microcode_intel.c | 33 ++++---------
xen/arch/x86/spec_ctrl.c | 2 +-
xen/include/asm-x86/microcode.h | 13 +-----
6 files changed, 30 insertions(+), 211 deletions(-)
diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index fafc0bd..d216455 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -1188,7 +1188,7 @@ static void __init check_deadline_errata(void)
else
rev = (unsigned long)m->driver_data;
- if ( this_cpu(ucode_cpu_info).cpu_sig.rev >= rev )
+ if ( this_cpu(cpu_sig).rev >= rev )
return;
setup_clear_cpu_cap(X86_FEATURE_TSC_DEADLINE);
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index cff86a9..0c01dfa 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -187,7 +187,7 @@ const struct microcode_ops *microcode_ops;
static DEFINE_SPINLOCK(microcode_mutex);
-DEFINE_PER_CPU(struct ucode_cpu_info, ucode_cpu_info);
+DEFINE_PER_CPU(struct cpu_signature, cpu_sig);
struct microcode_info {
unsigned int cpu;
@@ -196,70 +196,19 @@ struct microcode_info {
char buffer[1];
};
-static void __microcode_fini_cpu(unsigned int cpu)
-{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
-
- xfree(uci->mc.mc_valid);
- memset(uci, 0, sizeof(*uci));
-}
-
-static void microcode_fini_cpu(unsigned int cpu)
-{
- spin_lock(µcode_mutex);
- __microcode_fini_cpu(cpu);
- spin_unlock(µcode_mutex);
-}
-
int microcode_resume_cpu(unsigned int cpu)
{
int err;
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
- struct cpu_signature nsig;
- unsigned int cpu2;
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
if ( !microcode_ops )
return 0;
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, &uci->cpu_sig);
- if ( err )
- {
- __microcode_fini_cpu(cpu);
- spin_unlock(µcode_mutex);
- return err;
- }
-
- if ( uci->mc.mc_valid )
- {
- err = microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid);
- if ( err >= 0 )
- {
- if ( err )
- err = microcode_ops->apply_microcode(cpu);
- spin_unlock(µcode_mutex);
- return err;
- }
- }
-
- nsig = uci->cpu_sig;
- __microcode_fini_cpu(cpu);
- uci->cpu_sig = nsig;
-
- err = -EIO;
- for_each_online_cpu ( cpu2 )
- {
- uci = &per_cpu(ucode_cpu_info, cpu2);
- if ( uci->mc.mc_valid &&
- microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid) > 0 )
- {
- err = microcode_ops->apply_microcode(cpu);
- break;
- }
- }
-
- __microcode_fini_cpu(cpu);
+ err = microcode_ops->collect_cpu_info(cpu, sig);
+ if ( likely(!err) )
+ err = microcode_ops->apply_microcode(cpu);
spin_unlock(µcode_mutex);
return err;
@@ -302,16 +251,13 @@ static int microcode_update_cpu(const void *buf, size_t size)
{
int err;
unsigned int cpu = smp_processor_id();
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, &uci->cpu_sig);
+ err = microcode_ops->collect_cpu_info(cpu, sig);
if ( likely(!err) )
err = microcode_ops->cpu_request_microcode(cpu, buf, size);
- else
- __microcode_fini_cpu(cpu);
-
spin_unlock(µcode_mutex);
return err;
@@ -398,25 +344,6 @@ static int __init microcode_init(void)
}
__initcall(microcode_init);
-static int microcode_percpu_callback(
- struct notifier_block *nfb, unsigned long action, void *hcpu)
-{
- unsigned int cpu = (unsigned long)hcpu;
-
- switch ( action )
- {
- case CPU_DEAD:
- microcode_fini_cpu(cpu);
- break;
- }
-
- return NOTIFY_DONE;
-}
-
-static struct notifier_block microcode_percpu_nfb = {
- .notifier_call = microcode_percpu_callback,
-};
-
int __init early_microcode_update_cpu(bool start_update)
{
int rc = 0;
@@ -460,12 +387,8 @@ int __init early_microcode_init(void)
return rc;
if ( microcode_ops )
- {
if ( ucode_mod.mod_end || ucode_blob.size )
rc = early_microcode_update_cpu(true);
- register_cpu_notifier(µcode_percpu_nfb);
- }
-
return rc;
}
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 1f05899..93af2c9 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -155,7 +155,7 @@ static bool_t find_equiv_cpu_id(const struct equiv_cpu_entry *equiv_cpu_table,
static bool_t microcode_fits(const struct microcode_amd *mc_amd,
unsigned int cpu)
{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *mc_header = mc_amd->mpb;
const struct equiv_cpu_entry *equiv_cpu_table = mc_amd->equiv_cpu_table;
unsigned int current_cpu_id;
@@ -178,14 +178,14 @@ static bool_t microcode_fits(const struct microcode_amd *mc_amd,
return 0;
}
- if ( mc_header->patch_id <= uci->cpu_sig.rev )
+ if ( mc_header->patch_id <= sig->rev )
{
pr_debug("microcode: patch is already at required level or greater.\n");
return 0;
}
pr_debug("microcode: CPU%d found a matching microcode update with version %#x (current=%#x)\n",
- cpu, mc_header->patch_id, uci->cpu_sig.rev);
+ cpu, mc_header->patch_id, sig->rev);
return 1;
}
@@ -256,9 +256,9 @@ static enum microcode_match_result compare_patch(
static int apply_microcode(unsigned int cpu)
{
unsigned long flags;
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
uint32_t rev;
int hw_err;
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *hdr;
const struct microcode_patch *patch = microcode_get_cache();
@@ -294,9 +294,9 @@ static int apply_microcode(unsigned int cpu)
}
printk(KERN_WARNING "microcode: CPU%d updated from revision %#x to %#x\n",
- cpu, uci->cpu_sig.rev, hdr->patch_id);
+ cpu, sig->rev, hdr->patch_id);
- uci->cpu_sig.rev = rev;
+ sig->rev = rev;
return 0;
}
@@ -442,14 +442,14 @@ static bool_t check_final_patch_levels(unsigned int cpu)
* any of the 'final_levels', then we should not update the microcode
* patch on the cpu as system will hang otherwise.
*/
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
unsigned int i;
if ( boot_cpu_data.x86 != 0x10 )
return 0;
for ( i = 0; i < ARRAY_SIZE(final_levels); i++ )
- if ( uci->cpu_sig.rev == final_levels[i] )
+ if ( sig->rev == final_levels[i] )
return 1;
return 0;
@@ -458,13 +458,12 @@ static bool_t check_final_patch_levels(unsigned int cpu)
static int cpu_request_microcode(unsigned int cpu, const void *buf,
size_t bufsize)
{
- struct microcode_amd *mc_amd, *mc_old;
+ struct microcode_amd *mc_amd;
size_t offset = 0;
- size_t last_offset, applied_offset = 0;
- int error = 0, save_error = 1;
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
+ int error = 0;
unsigned int current_cpu_id;
unsigned int equiv_cpu_id;
+ const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
/* We should bind the task to the CPU */
BUG_ON(cpu != raw_smp_processor_id());
@@ -533,7 +532,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
{
printk(KERN_ERR "microcode: CPU%d incorrect or corrupt container file\n"
"microcode: Failed to update patch level. "
- "Current lvl:%#x\n", cpu, uci->cpu_sig.rev);
+ "Current lvl:%#x\n", cpu, sig->rev);
break;
}
}
@@ -544,17 +543,12 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
goto out;
}
- mc_old = uci->mc.mc_amd;
- /* implicitely validates uci->mc.mc_valid */
- uci->mc.mc_amd = mc_amd;
-
/*
* It's possible the data file has multiple matching ucode,
* lets keep searching till the latest version
*/
mc_amd->mpb = NULL;
mc_amd->mpb_size = 0;
- last_offset = offset;
while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
&offset)) == 0 )
{
@@ -576,11 +570,8 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
error = apply_microcode(cpu);
if ( error )
break;
- applied_offset = last_offset;
}
- last_offset = offset;
-
if ( offset >= bufsize )
break;
@@ -609,26 +600,6 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
break;
}
- /* On success keep the microcode patch for
- * re-apply on resume.
- */
- if ( applied_offset )
- {
- save_error = get_ucode_from_buffer_amd(
- mc_amd, buf, bufsize, &applied_offset);
-
- if ( save_error )
- error = save_error;
- }
-
- if ( save_error )
- {
- xfree(mc_amd);
- uci->mc.mc_amd = mc_old;
- }
- else
- xfree(mc_old);
-
out:
#if CONFIG_HVM
svm_host_osvw_init();
@@ -643,52 +614,6 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
return error;
}
-static int microcode_resume_match(unsigned int cpu, const void *mc)
-{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
- struct microcode_amd *mc_amd = uci->mc.mc_amd;
- const struct microcode_amd *src = mc;
-
- if ( !microcode_fits(src, cpu) )
- return 0;
-
- if ( src != mc_amd )
- {
- if ( mc_amd )
- {
- xfree(mc_amd->equiv_cpu_table);
- xfree(mc_amd->mpb);
- xfree(mc_amd);
- }
-
- mc_amd = xmalloc(struct microcode_amd);
- uci->mc.mc_amd = mc_amd;
- if ( !mc_amd )
- return -ENOMEM;
- mc_amd->equiv_cpu_table = xmalloc_bytes(src->equiv_cpu_table_size);
- if ( !mc_amd->equiv_cpu_table )
- goto err1;
- mc_amd->mpb = xmalloc_bytes(src->mpb_size);
- if ( !mc_amd->mpb )
- goto err2;
-
- mc_amd->equiv_cpu_table_size = src->equiv_cpu_table_size;
- mc_amd->mpb_size = src->mpb_size;
- memcpy(mc_amd->mpb, src->mpb, src->mpb_size);
- memcpy(mc_amd->equiv_cpu_table, src->equiv_cpu_table,
- src->equiv_cpu_table_size);
- }
-
- return 1;
-
-err2:
- xfree(mc_amd->equiv_cpu_table);
-err1:
- xfree(mc_amd);
- uci->mc.mc_amd = NULL;
- return -ENOMEM;
-}
-
static int start_update(void)
{
#if CONFIG_HVM
@@ -708,7 +633,6 @@ static int start_update(void)
}
static const struct microcode_ops microcode_amd_ops = {
- .microcode_resume_match = microcode_resume_match,
.cpu_request_microcode = cpu_request_microcode,
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index d3405a0..bf6497f 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -250,13 +250,13 @@ static int microcode_sanity_check(void *mc)
static bool match_cpu(const struct microcode_patch *patch)
{
- const struct ucode_cpu_info *uci = &this_cpu(ucode_cpu_info);
+ const struct cpu_signature *sig = &this_cpu(cpu_sig);
if ( !patch )
return false;
- return microcode_update_match(&patch->mc_intel->hdr, uci->cpu_sig.sig,
- uci->cpu_sig.pf, uci->cpu_sig.rev) == NEW_UCODE;
+ return microcode_update_match(&patch->mc_intel->hdr,
+ sig->sig, sig->pf, sig->rev) == NEW_UCODE;
}
static void free_patch(struct microcode_patch *patch)
@@ -281,7 +281,6 @@ static enum microcode_match_result compare_patch(
*/
static int get_matching_microcode(const void *mc, unsigned int cpu)
{
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
void *new_mc = xmalloc_bytes(total_size);
@@ -308,17 +307,7 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
pr_debug("microcode: CPU%d found a matching microcode update with"
" version %#x (current=%#x)\n",
- cpu, mc_header->rev, uci->cpu_sig.rev);
- new_mc = xmalloc_bytes(total_size);
- if ( new_mc == NULL )
- {
- printk(KERN_ERR "microcode: error! Can not allocate memory\n");
- return -ENOMEM;
- }
-
- memcpy(new_mc, mc, total_size);
- xfree(uci->mc.mc_intel);
- uci->mc.mc_intel = new_mc;
+ cpu, mc_header->rev, this_cpu(cpu_sig).rev);
return 1;
}
@@ -329,7 +318,7 @@ static int apply_microcode(unsigned int cpu)
uint64_t msr_content;
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
- struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
+ struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_intel *mc_intel;
const struct microcode_patch *patch = microcode_get_cache();
@@ -360,16 +349,16 @@ static int apply_microcode(unsigned int cpu)
{
printk(KERN_ERR "microcode: CPU%d update from revision "
"%#x to %#x failed. Resulting revision is %#x.\n", cpu_num,
- uci->cpu_sig.rev, mc_intel->hdr.rev, val[1]);
+ sig->rev, mc_intel->hdr.rev, val[1]);
return -EIO;
}
printk(KERN_INFO "microcode: CPU%d updated from revision "
"%#x to %#x, date = %04x-%02x-%02x \n",
- cpu_num, uci->cpu_sig.rev, val[1],
+ cpu_num, sig->rev, val[1],
mc_intel->hdr.year,
mc_intel->hdr.month,
mc_intel->hdr.day);
- uci->cpu_sig.rev = val[1];
+ sig->rev = val[1];
return 0;
}
@@ -440,13 +429,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
return error;
}
-static int microcode_resume_match(unsigned int cpu, const void *mc)
-{
- return get_matching_microcode(mc, cpu);
-}
-
static const struct microcode_ops microcode_intel_ops = {
- .microcode_resume_match = microcode_resume_match,
.cpu_request_microcode = cpu_request_microcode,
.collect_cpu_info = collect_cpu_info,
.apply_microcode = apply_microcode,
diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c
index 5d98cac..43128c3 100644
--- a/xen/arch/x86/spec_ctrl.c
+++ b/xen/arch/x86/spec_ctrl.c
@@ -436,7 +436,7 @@ static bool __init check_smt_enabled(void)
/* Calculate whether Retpoline is known-safe on this CPU. */
static bool __init retpoline_safe(uint64_t caps)
{
- unsigned int ucode_rev = this_cpu(ucode_cpu_info).cpu_sig.rev;
+ unsigned int ucode_rev = this_cpu(cpu_sig).rev;
if ( boot_cpu_data.x86_vendor == X86_VENDOR_AMD )
return true;
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index 6541c58..f2ac509 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -10,7 +10,6 @@ enum microcode_match_result {
};
struct cpu_signature;
-struct ucode_cpu_info;
struct microcode_patch {
union {
@@ -20,7 +19,6 @@ struct microcode_patch {
};
struct microcode_ops {
- int (*microcode_resume_match)(unsigned int cpu, const void *mc);
int (*cpu_request_microcode)(unsigned int cpu, const void *buf,
size_t size);
int (*collect_cpu_info)(unsigned int cpu, struct cpu_signature *csig);
@@ -39,16 +37,7 @@ struct cpu_signature {
unsigned int rev;
};
-struct ucode_cpu_info {
- struct cpu_signature cpu_sig;
- union {
- struct microcode_intel *mc_intel;
- struct microcode_amd *mc_amd;
- void *mc_valid;
- } mc;
-};
-
-DECLARE_PER_CPU(struct ucode_cpu_info, ucode_cpu_info);
+DECLARE_PER_CPU(struct cpu_signature, cpu_sig);
extern const struct microcode_ops *microcode_ops;
const struct microcode_patch *microcode_get_cache(void);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 05/10] microcode: remove pointless 'cpu' parameter
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
Some callbacks in microcode_ops or related functions take a cpu
id parameter. But at current call sites, the cpu id parameter is
always equal to current cpu id. Some of them even use an assertion
to guarantee this. Remove this redundent 'cpu' parameter.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/microcode.c | 12 ++++++------
xen/arch/x86/microcode_amd.c | 35 +++++++++++++----------------------
xen/arch/x86/microcode_intel.c | 25 +++++++++----------------
xen/arch/x86/smpboot.c | 2 +-
xen/include/asm-x86/microcode.h | 7 +++----
xen/include/asm-x86/processor.h | 2 +-
7 files changed, 34 insertions(+), 51 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index aecc754..4f21903 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -253,7 +253,7 @@ static int enter_state(u32 state)
console_end_sync();
- microcode_resume_cpu(0);
+ microcode_resume_cpu();
if ( !recheck_cpu_features(0) )
panic("Missing previously available feature(s)\n");
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 0c01dfa..16a6d50 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -196,19 +196,19 @@ struct microcode_info {
char buffer[1];
};
-int microcode_resume_cpu(unsigned int cpu)
+int microcode_resume_cpu(void)
{
int err;
- struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
+ struct cpu_signature *sig = &this_cpu(cpu_sig);
if ( !microcode_ops )
return 0;
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, sig);
+ err = microcode_ops->collect_cpu_info(sig);
if ( likely(!err) )
- err = microcode_ops->apply_microcode(cpu);
+ err = microcode_ops->apply_microcode();
spin_unlock(µcode_mutex);
return err;
@@ -255,9 +255,9 @@ static int microcode_update_cpu(const void *buf, size_t size)
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, sig);
+ err = microcode_ops->collect_cpu_info(sig);
if ( likely(!err) )
- err = microcode_ops->cpu_request_microcode(cpu, buf, size);
+ err = microcode_ops->cpu_request_microcode(buf, size);
spin_unlock(µcode_mutex);
return err;
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 93af2c9..0144df1 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -78,8 +78,9 @@ struct mpbhdr {
static DEFINE_SPINLOCK(microcode_update_lock);
/* See comment in start_update() for cases when this routine fails */
-static int collect_cpu_info(unsigned int cpu, struct cpu_signature *csig)
+static int collect_cpu_info(struct cpu_signature *csig)
{
+ unsigned int cpu = smp_processor_id();
struct cpuinfo_x86 *c = &cpu_data[cpu];
memset(csig, 0, sizeof(*csig));
@@ -152,18 +153,15 @@ static bool_t find_equiv_cpu_id(const struct equiv_cpu_entry *equiv_cpu_table,
return 0;
}
-static bool_t microcode_fits(const struct microcode_amd *mc_amd,
- unsigned int cpu)
+static bool microcode_fits(const struct microcode_amd *mc_amd)
{
+ unsigned int cpu = smp_processor_id();
const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *mc_header = mc_amd->mpb;
const struct equiv_cpu_entry *equiv_cpu_table = mc_amd->equiv_cpu_table;
unsigned int current_cpu_id;
unsigned int equiv_cpu_id;
- /* We should bind the task to the CPU */
- BUG_ON(cpu != raw_smp_processor_id());
-
current_cpu_id = cpuid_eax(0x00000001);
if ( !find_equiv_cpu_id(equiv_cpu_table, current_cpu_id, &equiv_cpu_id) )
@@ -192,9 +190,7 @@ static bool_t microcode_fits(const struct microcode_amd *mc_amd,
static bool match_cpu(const struct microcode_patch *patch)
{
- if ( !patch )
- return false;
- return microcode_fits(patch->mc_amd, smp_processor_id());
+ return patch ? microcode_fits(patch->mc_amd) : false;
}
static struct microcode_patch *alloc_microcode_patch(
@@ -253,18 +249,16 @@ static enum microcode_match_result compare_patch(
return MIS_UCODE;
}
-static int apply_microcode(unsigned int cpu)
+static int apply_microcode(void)
{
unsigned long flags;
uint32_t rev;
int hw_err;
+ unsigned int cpu = smp_processor_id();
struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *hdr;
const struct microcode_patch *patch = microcode_get_cache();
- /* We should bind the task to the CPU */
- BUG_ON(raw_smp_processor_id() != cpu);
-
if ( !match_cpu(patch) )
return -EINVAL;
@@ -435,14 +429,14 @@ static const unsigned int final_levels[] = {
0x010000af
};
-static bool_t check_final_patch_levels(unsigned int cpu)
+static bool check_final_patch_levels(void)
{
/*
* Check the current patch levels on the cpu. If they are equal to
* any of the 'final_levels', then we should not update the microcode
* patch on the cpu as system will hang otherwise.
*/
- const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
+ const struct cpu_signature *sig = &this_cpu(cpu_sig);
unsigned int i;
if ( boot_cpu_data.x86 != 0x10 )
@@ -455,19 +449,16 @@ static bool_t check_final_patch_levels(unsigned int cpu)
return 0;
}
-static int cpu_request_microcode(unsigned int cpu, const void *buf,
- size_t bufsize)
+static int cpu_request_microcode(const void *buf, size_t bufsize)
{
struct microcode_amd *mc_amd;
size_t offset = 0;
int error = 0;
unsigned int current_cpu_id;
unsigned int equiv_cpu_id;
+ unsigned int cpu = smp_processor_id();
const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
- /* We should bind the task to the CPU */
- BUG_ON(cpu != raw_smp_processor_id());
-
current_cpu_id = cpuid_eax(0x00000001);
if ( *(const uint32_t *)buf != UCODE_MAGIC )
@@ -477,7 +468,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
goto out;
}
- if ( check_final_patch_levels(cpu) )
+ if ( check_final_patch_levels() )
{
printk(XENLOG_INFO
"microcode: Cannot update microcode patch on the cpu as we hit a final level\n");
@@ -567,7 +558,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
if ( match_cpu(microcode_get_cache()) )
{
- error = apply_microcode(cpu);
+ error = apply_microcode();
if ( error )
break;
}
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index bf6497f..b66844d 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -96,13 +96,12 @@ struct extended_sigtable {
/* serialize access to the physical write to MSR 0x79 */
static DEFINE_SPINLOCK(microcode_update_lock);
-static int collect_cpu_info(unsigned int cpu_num, struct cpu_signature *csig)
+static int collect_cpu_info(struct cpu_signature *csig)
{
+ unsigned int cpu_num = smp_processor_id();
struct cpuinfo_x86 *c = &cpu_data[cpu_num];
uint64_t msr_content;
- BUG_ON(cpu_num != smp_processor_id());
-
memset(csig, 0, sizeof(*csig));
if ( (c->x86_vendor != X86_VENDOR_INTEL) || (c->x86 < 6) )
@@ -279,12 +278,13 @@ static enum microcode_match_result compare_patch(
* return 1 - found update
* return < 0 - error
*/
-static int get_matching_microcode(const void *mc, unsigned int cpu)
+static int get_matching_microcode(const void *mc)
{
const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
void *new_mc = xmalloc_bytes(total_size);
struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
+ unsigned int __maybe_unused cpu = smp_processor_id();
if ( !new_patch || !new_mc )
{
@@ -312,19 +312,16 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
return 1;
}
-static int apply_microcode(unsigned int cpu)
+static int apply_microcode(void)
{
unsigned long flags;
uint64_t msr_content;
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
- struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
+ struct cpu_signature *sig = &this_cpu(cpu_sig);
const struct microcode_intel *mc_intel;
const struct microcode_patch *patch = microcode_get_cache();
- /* We should bind the task to the CPU */
- BUG_ON(cpu_num != cpu);
-
if ( !match_cpu(patch) )
return -EINVAL;
@@ -391,22 +388,18 @@ static long get_next_ucode_from_buffer(void **mc, const u8 *buf,
return offset + total_size;
}
-static int cpu_request_microcode(unsigned int cpu, const void *buf,
- size_t size)
+static int cpu_request_microcode(const void *buf, size_t size)
{
long offset = 0;
int error = 0;
void *mc;
- /* We should bind the task to the CPU */
- BUG_ON(cpu != raw_smp_processor_id());
-
while ( (offset = get_next_ucode_from_buffer(&mc, buf, size, offset)) > 0 )
{
error = microcode_sanity_check(mc);
if ( error )
break;
- error = get_matching_microcode(mc, cpu);
+ error = get_matching_microcode(mc);
if ( error < 0 )
break;
/*
@@ -424,7 +417,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
error = offset;
if ( !error && match_cpu(microcode_get_cache()) )
- error = apply_microcode(cpu);
+ error = apply_microcode();
return error;
}
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 274865a..de19b67 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -366,7 +366,7 @@ void start_secondary(void *unused)
if ( system_state <= SYS_STATE_smp_boot )
early_microcode_update_cpu(false);
else
- microcode_resume_cpu(cpu);
+ microcode_resume_cpu();
/*
* If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index f2ac509..e6842d4 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -19,10 +19,9 @@ struct microcode_patch {
};
struct microcode_ops {
- int (*cpu_request_microcode)(unsigned int cpu, const void *buf,
- size_t size);
- int (*collect_cpu_info)(unsigned int cpu, struct cpu_signature *csig);
- int (*apply_microcode)(unsigned int cpu);
+ int (*cpu_request_microcode)(const void *buf, size_t size);
+ int (*collect_cpu_info)(struct cpu_signature *csig);
+ int (*apply_microcode)(void);
int (*start_update)(void);
void (*free_patch)(struct microcode_patch *patch);
bool (*match_cpu)(const struct microcode_patch *patch);
diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h
index cef3ffb..8b7e484 100644
--- a/xen/include/asm-x86/processor.h
+++ b/xen/include/asm-x86/processor.h
@@ -573,7 +573,7 @@ int guest_wrmsr_xen(struct vcpu *v, uint32_t idx, uint64_t val);
void microcode_set_module(unsigned int);
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void), unsigned long len);
-int microcode_resume_cpu(unsigned int cpu);
+int microcode_resume_cpu(void);
int early_microcode_update_cpu(bool start_update);
int early_microcode_init(void);
int microcode_init_intel(void);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 05/10] microcode: remove pointless 'cpu' parameter
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
Some callbacks in microcode_ops or related functions take a cpu
id parameter. But at current call sites, the cpu id parameter is
always equal to current cpu id. Some of them even use an assertion
to guarantee this. Remove this redundent 'cpu' parameter.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/microcode.c | 12 ++++++------
xen/arch/x86/microcode_amd.c | 35 +++++++++++++----------------------
xen/arch/x86/microcode_intel.c | 25 +++++++++----------------
xen/arch/x86/smpboot.c | 2 +-
xen/include/asm-x86/microcode.h | 7 +++----
xen/include/asm-x86/processor.h | 2 +-
7 files changed, 34 insertions(+), 51 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index aecc754..4f21903 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -253,7 +253,7 @@ static int enter_state(u32 state)
console_end_sync();
- microcode_resume_cpu(0);
+ microcode_resume_cpu();
if ( !recheck_cpu_features(0) )
panic("Missing previously available feature(s)\n");
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 0c01dfa..16a6d50 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -196,19 +196,19 @@ struct microcode_info {
char buffer[1];
};
-int microcode_resume_cpu(unsigned int cpu)
+int microcode_resume_cpu(void)
{
int err;
- struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
+ struct cpu_signature *sig = &this_cpu(cpu_sig);
if ( !microcode_ops )
return 0;
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, sig);
+ err = microcode_ops->collect_cpu_info(sig);
if ( likely(!err) )
- err = microcode_ops->apply_microcode(cpu);
+ err = microcode_ops->apply_microcode();
spin_unlock(µcode_mutex);
return err;
@@ -255,9 +255,9 @@ static int microcode_update_cpu(const void *buf, size_t size)
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(cpu, sig);
+ err = microcode_ops->collect_cpu_info(sig);
if ( likely(!err) )
- err = microcode_ops->cpu_request_microcode(cpu, buf, size);
+ err = microcode_ops->cpu_request_microcode(buf, size);
spin_unlock(µcode_mutex);
return err;
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 93af2c9..0144df1 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -78,8 +78,9 @@ struct mpbhdr {
static DEFINE_SPINLOCK(microcode_update_lock);
/* See comment in start_update() for cases when this routine fails */
-static int collect_cpu_info(unsigned int cpu, struct cpu_signature *csig)
+static int collect_cpu_info(struct cpu_signature *csig)
{
+ unsigned int cpu = smp_processor_id();
struct cpuinfo_x86 *c = &cpu_data[cpu];
memset(csig, 0, sizeof(*csig));
@@ -152,18 +153,15 @@ static bool_t find_equiv_cpu_id(const struct equiv_cpu_entry *equiv_cpu_table,
return 0;
}
-static bool_t microcode_fits(const struct microcode_amd *mc_amd,
- unsigned int cpu)
+static bool microcode_fits(const struct microcode_amd *mc_amd)
{
+ unsigned int cpu = smp_processor_id();
const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *mc_header = mc_amd->mpb;
const struct equiv_cpu_entry *equiv_cpu_table = mc_amd->equiv_cpu_table;
unsigned int current_cpu_id;
unsigned int equiv_cpu_id;
- /* We should bind the task to the CPU */
- BUG_ON(cpu != raw_smp_processor_id());
-
current_cpu_id = cpuid_eax(0x00000001);
if ( !find_equiv_cpu_id(equiv_cpu_table, current_cpu_id, &equiv_cpu_id) )
@@ -192,9 +190,7 @@ static bool_t microcode_fits(const struct microcode_amd *mc_amd,
static bool match_cpu(const struct microcode_patch *patch)
{
- if ( !patch )
- return false;
- return microcode_fits(patch->mc_amd, smp_processor_id());
+ return patch ? microcode_fits(patch->mc_amd) : false;
}
static struct microcode_patch *alloc_microcode_patch(
@@ -253,18 +249,16 @@ static enum microcode_match_result compare_patch(
return MIS_UCODE;
}
-static int apply_microcode(unsigned int cpu)
+static int apply_microcode(void)
{
unsigned long flags;
uint32_t rev;
int hw_err;
+ unsigned int cpu = smp_processor_id();
struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *hdr;
const struct microcode_patch *patch = microcode_get_cache();
- /* We should bind the task to the CPU */
- BUG_ON(raw_smp_processor_id() != cpu);
-
if ( !match_cpu(patch) )
return -EINVAL;
@@ -435,14 +429,14 @@ static const unsigned int final_levels[] = {
0x010000af
};
-static bool_t check_final_patch_levels(unsigned int cpu)
+static bool check_final_patch_levels(void)
{
/*
* Check the current patch levels on the cpu. If they are equal to
* any of the 'final_levels', then we should not update the microcode
* patch on the cpu as system will hang otherwise.
*/
- const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
+ const struct cpu_signature *sig = &this_cpu(cpu_sig);
unsigned int i;
if ( boot_cpu_data.x86 != 0x10 )
@@ -455,19 +449,16 @@ static bool_t check_final_patch_levels(unsigned int cpu)
return 0;
}
-static int cpu_request_microcode(unsigned int cpu, const void *buf,
- size_t bufsize)
+static int cpu_request_microcode(const void *buf, size_t bufsize)
{
struct microcode_amd *mc_amd;
size_t offset = 0;
int error = 0;
unsigned int current_cpu_id;
unsigned int equiv_cpu_id;
+ unsigned int cpu = smp_processor_id();
const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
- /* We should bind the task to the CPU */
- BUG_ON(cpu != raw_smp_processor_id());
-
current_cpu_id = cpuid_eax(0x00000001);
if ( *(const uint32_t *)buf != UCODE_MAGIC )
@@ -477,7 +468,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
goto out;
}
- if ( check_final_patch_levels(cpu) )
+ if ( check_final_patch_levels() )
{
printk(XENLOG_INFO
"microcode: Cannot update microcode patch on the cpu as we hit a final level\n");
@@ -567,7 +558,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
if ( match_cpu(microcode_get_cache()) )
{
- error = apply_microcode(cpu);
+ error = apply_microcode();
if ( error )
break;
}
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index bf6497f..b66844d 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -96,13 +96,12 @@ struct extended_sigtable {
/* serialize access to the physical write to MSR 0x79 */
static DEFINE_SPINLOCK(microcode_update_lock);
-static int collect_cpu_info(unsigned int cpu_num, struct cpu_signature *csig)
+static int collect_cpu_info(struct cpu_signature *csig)
{
+ unsigned int cpu_num = smp_processor_id();
struct cpuinfo_x86 *c = &cpu_data[cpu_num];
uint64_t msr_content;
- BUG_ON(cpu_num != smp_processor_id());
-
memset(csig, 0, sizeof(*csig));
if ( (c->x86_vendor != X86_VENDOR_INTEL) || (c->x86 < 6) )
@@ -279,12 +278,13 @@ static enum microcode_match_result compare_patch(
* return 1 - found update
* return < 0 - error
*/
-static int get_matching_microcode(const void *mc, unsigned int cpu)
+static int get_matching_microcode(const void *mc)
{
const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
void *new_mc = xmalloc_bytes(total_size);
struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
+ unsigned int __maybe_unused cpu = smp_processor_id();
if ( !new_patch || !new_mc )
{
@@ -312,19 +312,16 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
return 1;
}
-static int apply_microcode(unsigned int cpu)
+static int apply_microcode(void)
{
unsigned long flags;
uint64_t msr_content;
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
- struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
+ struct cpu_signature *sig = &this_cpu(cpu_sig);
const struct microcode_intel *mc_intel;
const struct microcode_patch *patch = microcode_get_cache();
- /* We should bind the task to the CPU */
- BUG_ON(cpu_num != cpu);
-
if ( !match_cpu(patch) )
return -EINVAL;
@@ -391,22 +388,18 @@ static long get_next_ucode_from_buffer(void **mc, const u8 *buf,
return offset + total_size;
}
-static int cpu_request_microcode(unsigned int cpu, const void *buf,
- size_t size)
+static int cpu_request_microcode(const void *buf, size_t size)
{
long offset = 0;
int error = 0;
void *mc;
- /* We should bind the task to the CPU */
- BUG_ON(cpu != raw_smp_processor_id());
-
while ( (offset = get_next_ucode_from_buffer(&mc, buf, size, offset)) > 0 )
{
error = microcode_sanity_check(mc);
if ( error )
break;
- error = get_matching_microcode(mc, cpu);
+ error = get_matching_microcode(mc);
if ( error < 0 )
break;
/*
@@ -424,7 +417,7 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
error = offset;
if ( !error && match_cpu(microcode_get_cache()) )
- error = apply_microcode(cpu);
+ error = apply_microcode();
return error;
}
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 274865a..de19b67 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -366,7 +366,7 @@ void start_secondary(void *unused)
if ( system_state <= SYS_STATE_smp_boot )
early_microcode_update_cpu(false);
else
- microcode_resume_cpu(cpu);
+ microcode_resume_cpu();
/*
* If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index f2ac509..e6842d4 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -19,10 +19,9 @@ struct microcode_patch {
};
struct microcode_ops {
- int (*cpu_request_microcode)(unsigned int cpu, const void *buf,
- size_t size);
- int (*collect_cpu_info)(unsigned int cpu, struct cpu_signature *csig);
- int (*apply_microcode)(unsigned int cpu);
+ int (*cpu_request_microcode)(const void *buf, size_t size);
+ int (*collect_cpu_info)(struct cpu_signature *csig);
+ int (*apply_microcode)(void);
int (*start_update)(void);
void (*free_patch)(struct microcode_patch *patch);
bool (*match_cpu)(const struct microcode_patch *patch);
diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h
index cef3ffb..8b7e484 100644
--- a/xen/include/asm-x86/processor.h
+++ b/xen/include/asm-x86/processor.h
@@ -573,7 +573,7 @@ int guest_wrmsr_xen(struct vcpu *v, uint32_t idx, uint64_t val);
void microcode_set_module(unsigned int);
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void), unsigned long len);
-int microcode_resume_cpu(unsigned int cpu);
+int microcode_resume_cpu(void);
int early_microcode_update_cpu(bool start_update);
int early_microcode_init(void);
int microcode_init_intel(void);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode()
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
During late microcode update, apply_microcode() is invoked in
cpu_request_microcode(). To make late microcode update more reliable,
we want to put the apply_microcode() into stop_machine context. So
we split out it from cpu_request_microcode(). As a consequence,
apply_microcode() should be invoked explicitly in the common code.
Previously, apply_microcode() gets the microcode patch to be applied from
the microcode cache. Now, the patch is passed as a function argument and
a patch is cached for cpu-hotplug and cpu resuming, only after it has
been loaded to a cpu without any error. As a consequence, the
'match_cpu' check in microcode_update_cache is removed, which otherwise
would fail.
Assuming that all CPUs have the same signature, one patch matching with
current CPU should match with others. Then parsing microcode only needs
to be done once; cpu_request_microcode() is also moved out of
microcode_update_cpu().
On AMD side, svm_host_osvw_init() is supposed to be called after
microcode update. As apply_micrcode() won't be called by
cpu_request_microcode() now, svm_host_osvw_init() is moved to the
end of apply_microcode().
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- to handle load failure, unvalidated patches won't be cached. They
are passed as function arguments. So if update failed, we needn't
any cleanup to microcode cache.
- microcode_info which passes microcode blob to be parsed to each CPU is
replaced by microcode_patch.
Changes in v6:
- during early microcode update, BSP and APs call different functions.
Thus AP can bypass parsing microcode blob.
---
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/microcode.c | 209 ++++++++++++++++++++++++++--------------
xen/arch/x86/microcode_amd.c | 41 ++++----
xen/arch/x86/microcode_intel.c | 69 ++++++-------
xen/arch/x86/smpboot.c | 5 +-
xen/include/asm-x86/microcode.h | 8 +-
xen/include/asm-x86/processor.h | 3 +-
7 files changed, 193 insertions(+), 144 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 4f21903..9583172 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -253,7 +253,7 @@ static int enter_state(u32 state)
console_end_sync();
- microcode_resume_cpu();
+ early_microcode_update_cpu();
if ( !recheck_cpu_features(0) )
panic("Missing previously available feature(s)\n");
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 16a6d50..23cf550 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -189,36 +189,62 @@ static DEFINE_SPINLOCK(microcode_mutex);
DEFINE_PER_CPU(struct cpu_signature, cpu_sig);
-struct microcode_info {
- unsigned int cpu;
- uint32_t buffer_size;
- int error;
- char buffer[1];
-};
+/*
+ * Return the patch with the highest revision id among all matching
+ * patches in the blob. Return NULL if no suitable patch.
+ */
+static struct microcode_patch *microcode_parse_blob(const char *buf,
+ uint32_t len)
+{
+ if ( likely(!microcode_ops->collect_cpu_info(&this_cpu(cpu_sig))) )
+ return microcode_ops->cpu_request_microcode(buf, len);
-int microcode_resume_cpu(void)
+ return NULL;
+}
+
+/*
+ * Load a microcode update to current CPU.
+ *
+ * If no patch is provided, the cached patch will be loaded. Microcode update
+ * during APs bringup and CPU resuming falls into this case.
+ */
+static int microcode_update_cpu(struct microcode_patch *patch)
{
- int err;
- struct cpu_signature *sig = &this_cpu(cpu_sig);
+ int ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
- if ( !microcode_ops )
- return 0;
+ if ( unlikely(ret) )
+ return ret;
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(sig);
- if ( likely(!err) )
- err = microcode_ops->apply_microcode();
- spin_unlock(µcode_mutex);
+ if ( patch )
+ {
+ /*
+ * If a patch is specified, it should has newer revision than
+ * that of the patch cached.
+ */
+ if ( microcode_cache &&
+ microcode_ops->compare_patch(patch, microcode_cache) != NEW_UCODE )
+ {
+ spin_unlock(µcode_mutex);
+ return -EINVAL;
+ }
- return err;
-}
+ ret = microcode_ops->apply_microcode(patch);
+ }
+ else if ( microcode_cache )
+ {
+ ret = microcode_ops->apply_microcode(microcode_cache);
+ if ( ret == -EIO )
+ printk("Update failed. Reboot needed\n");
+ }
+ else
+ /* No patch to update */
+ ret = -EINVAL;
-const struct microcode_patch *microcode_get_cache(void)
-{
- ASSERT(spin_is_locked(µcode_mutex));
+ spin_unlock(µcode_mutex);
- return microcode_cache;
+ return ret;
}
/* Return true if cache gets updated. Otherwise, return false */
@@ -227,9 +253,6 @@ bool microcode_update_cache(struct microcode_patch *patch)
ASSERT(spin_is_locked(µcode_mutex));
- if ( !microcode_ops->match_cpu(patch) )
- return false;
-
if ( !microcode_cache )
microcode_cache = patch;
else if ( microcode_ops->compare_patch(patch, microcode_cache) ==
@@ -247,46 +270,32 @@ bool microcode_update_cache(struct microcode_patch *patch)
return true;
}
-static int microcode_update_cpu(const void *buf, size_t size)
+static long do_microcode_update(void *patch)
{
- int err;
- unsigned int cpu = smp_processor_id();
- struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
-
- spin_lock(µcode_mutex);
+ int error, cpu;
- err = microcode_ops->collect_cpu_info(sig);
- if ( likely(!err) )
- err = microcode_ops->cpu_request_microcode(buf, size);
- spin_unlock(µcode_mutex);
-
- return err;
-}
-
-static long do_microcode_update(void *_info)
-{
- struct microcode_info *info = _info;
- int error;
+ error = microcode_update_cpu(patch);
+ if ( error )
+ {
+ microcode_ops->free_patch(microcode_cache);
+ return error;
+ }
- BUG_ON(info->cpu != smp_processor_id());
- error = microcode_update_cpu(info->buffer, info->buffer_size);
- if ( error )
- info->error = error;
+ cpu = cpumask_next(smp_processor_id(), &cpu_online_map);
+ if ( cpu < nr_cpu_ids )
+ return continue_hypercall_on_cpu(cpu, do_microcode_update, patch);
- info->cpu = cpumask_next(info->cpu, &cpu_online_map);
- if ( info->cpu < nr_cpu_ids )
- return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
+ microcode_update_cache(patch);
- error = info->error;
- xfree(info);
return error;
}
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
{
int ret;
- struct microcode_info *info;
+ void *buffer;
+ struct microcode_patch *patch;
if ( len != (uint32_t)len )
return -E2BIG;
@@ -294,32 +303,49 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
if ( microcode_ops == NULL )
return -EINVAL;
- info = xmalloc_bytes(sizeof(*info) + len);
- if ( info == NULL )
- return -ENOMEM;
-
- ret = copy_from_guest(info->buffer, buf, len);
- if ( ret != 0 )
+ buffer = xmalloc_bytes(len);
+ if ( !buffer )
{
- xfree(info);
- return ret;
+ ret = -ENOMEM;
+ goto free;
}
- info->buffer_size = len;
- info->error = 0;
- info->cpu = cpumask_first(&cpu_online_map);
+ if ( copy_from_guest(buffer, buf, len) )
+ {
+ ret = -EFAULT;
+ goto free;
+ }
if ( microcode_ops->start_update )
{
ret = microcode_ops->start_update();
if ( ret != 0 )
- {
- xfree(info);
- return ret;
- }
+ goto free;
}
- return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
+ patch = microcode_parse_blob(buffer, len);
+ if ( IS_ERR(patch) )
+ {
+ printk(XENLOG_ERR "Parsing microcode blob error %ld\n", PTR_ERR(patch));
+ ret = PTR_ERR(patch);
+ goto free;
+ }
+
+ if ( !microcode_ops->match_cpu(patch) )
+ {
+ printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
+ if ( patch )
+ microcode_ops->free_patch(patch);
+ ret = -EINVAL;
+ goto free;
+ }
+
+ ret = continue_hypercall_on_cpu(cpumask_first(&cpu_online_map),
+ do_microcode_update, patch);
+
+ free:
+ xfree(buffer);
+ return ret;
}
static int __init microcode_init(void)
@@ -344,7 +370,16 @@ static int __init microcode_init(void)
}
__initcall(microcode_init);
-int __init early_microcode_update_cpu(bool start_update)
+int early_microcode_update_cpu(void)
+{
+ return microcode_ops ? microcode_update_cpu(NULL) : 0;
+}
+
+/*
+ * BSP needs to parse the ucode blob and then apply an update.
+ * APs just apply an update by calling early_microcode_update_cpu().
+ */
+static int __init early_microcode_parse_and_update_cpu(void)
{
int rc = 0;
void *data = NULL;
@@ -362,13 +397,41 @@ int __init early_microcode_update_cpu(bool start_update)
}
if ( data )
{
- if ( start_update && microcode_ops->start_update )
+ struct microcode_patch *patch;
+
+ if ( microcode_ops->start_update )
rc = microcode_ops->start_update();
if ( rc )
return rc;
- return microcode_update_cpu(data, len);
+ patch = microcode_parse_blob(data, len);
+ if ( IS_ERR(patch) )
+ {
+ printk(XENLOG_ERR "Parsing microcode blob error %ld\n",
+ PTR_ERR(patch));
+ return PTR_ERR(patch);
+ }
+
+ if ( !microcode_ops->match_cpu(patch) )
+ {
+ printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
+ if ( patch )
+ microcode_ops->free_patch(patch);
+ return -EINVAL;
+ }
+
+ rc = microcode_update_cpu(patch);
+ if ( !rc )
+ {
+ spin_lock(µcode_mutex);
+ microcode_update_cache(patch);
+ spin_unlock(µcode_mutex);
+ }
+ else
+ microcode_ops->free_patch(patch);
+
+ return rc;
}
else
return -ENOMEM;
@@ -387,8 +450,10 @@ int __init early_microcode_init(void)
return rc;
if ( microcode_ops )
+ {
if ( ucode_mod.mod_end || ucode_blob.size )
- rc = early_microcode_update_cpu(true);
+ rc = early_microcode_parse_and_update_cpu();
+ }
return rc;
}
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 0144df1..c819028 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -249,7 +249,7 @@ static enum microcode_match_result compare_patch(
return MIS_UCODE;
}
-static int apply_microcode(void)
+static int apply_microcode(const struct microcode_patch *patch)
{
unsigned long flags;
uint32_t rev;
@@ -257,7 +257,6 @@ static int apply_microcode(void)
unsigned int cpu = smp_processor_id();
struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *hdr;
- const struct microcode_patch *patch = microcode_get_cache();
if ( !match_cpu(patch) )
return -EINVAL;
@@ -292,6 +291,10 @@ static int apply_microcode(void)
sig->rev = rev;
+#ifdef CONFIG_HVM
+ svm_host_osvw_init();
+#endif
+
return 0;
}
@@ -449,9 +452,11 @@ static bool check_final_patch_levels(void)
return 0;
}
-static int cpu_request_microcode(const void *buf, size_t bufsize)
+static struct microcode_patch *cpu_request_microcode(const void *buf,
+ size_t bufsize)
{
struct microcode_amd *mc_amd;
+ struct microcode_patch *patch = NULL;
size_t offset = 0;
int error = 0;
unsigned int current_cpu_id;
@@ -551,17 +556,16 @@ static int cpu_request_microcode(const void *buf, size_t bufsize)
break;
}
- if ( match_cpu(new_patch) )
- microcode_update_cache(new_patch);
- else
- free_patch(new_patch);
-
- if ( match_cpu(microcode_get_cache()) )
+ /* Compare patches and store the one with higher revision */
+ if ( !patch && match_cpu(new_patch) )
+ patch = new_patch;
+ else if ( patch && (compare_patch(new_patch, patch) == NEW_UCODE) )
{
- error = apply_microcode();
- if ( error )
- break;
+ free_patch(patch);
+ patch = new_patch;
}
+ else
+ free_patch(new_patch);
if ( offset >= bufsize )
break;
@@ -592,17 +596,10 @@ static int cpu_request_microcode(const void *buf, size_t bufsize)
}
out:
-#if CONFIG_HVM
- svm_host_osvw_init();
-#endif
+ if ( error && !patch )
+ patch = ERR_PTR(error);
- /*
- * In some cases we may return an error even if processor's microcode has
- * been updated. For example, the first patch in a container file is loaded
- * successfully but subsequent container file processing encounters a
- * failure.
- */
- return error;
+ return patch;
}
static int start_update(void)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index b66844d..650495d 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -273,46 +273,27 @@ static enum microcode_match_result compare_patch(
old_header->pf, old_header->rev);
}
-/*
- * return 0 - no update found
- * return 1 - found update
- * return < 0 - error
- */
-static int get_matching_microcode(const void *mc)
+static struct microcode_patch *allow_microcode_patch(
+ const struct microcode_header_intel *mc_header)
{
- const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
void *new_mc = xmalloc_bytes(total_size);
struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
- unsigned int __maybe_unused cpu = smp_processor_id();
if ( !new_patch || !new_mc )
{
xfree(new_patch);
xfree(new_mc);
printk(XENLOG_ERR "microcode: Can not allocate memory\n");
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
- memcpy(new_mc, mc, total_size);
+ memcpy(new_mc, mc_header, total_size);
new_patch->mc_intel = new_mc;
- if ( !match_cpu(new_patch) )
- {
- free_patch(new_patch);
- return 0;
- }
-
- if ( !microcode_update_cache(new_patch) )
- return 0;
-
- pr_debug("microcode: CPU%d found a matching microcode update with"
- " version %#x (current=%#x)\n",
- cpu, mc_header->rev, this_cpu(cpu_sig).rev);
-
- return 1;
+ return new_patch;
}
-static int apply_microcode(void)
+static int apply_microcode(const struct microcode_patch *patch)
{
unsigned long flags;
uint64_t msr_content;
@@ -320,7 +301,6 @@ static int apply_microcode(void)
unsigned int cpu_num = raw_smp_processor_id();
struct cpu_signature *sig = &this_cpu(cpu_sig);
const struct microcode_intel *mc_intel;
- const struct microcode_patch *patch = microcode_get_cache();
if ( !match_cpu(patch) )
return -EINVAL;
@@ -388,26 +368,39 @@ static long get_next_ucode_from_buffer(void **mc, const u8 *buf,
return offset + total_size;
}
-static int cpu_request_microcode(const void *buf, size_t size)
+static struct microcode_patch *cpu_request_microcode(const void *buf,
+ size_t size)
{
long offset = 0;
int error = 0;
void *mc;
+ struct microcode_patch *patch = NULL;
while ( (offset = get_next_ucode_from_buffer(&mc, buf, size, offset)) > 0 )
{
+ struct microcode_patch *new_patch;
+
error = microcode_sanity_check(mc);
if ( error )
break;
- error = get_matching_microcode(mc);
- if ( error < 0 )
+
+ new_patch = allow_microcode_patch(mc);
+ if ( IS_ERR(new_patch) )
+ {
+ error = PTR_ERR(new_patch);
break;
- /*
- * It's possible the data file has multiple matching ucode,
- * lets keep searching till the latest version
- */
- if ( error == 1 )
- error = 0;
+ }
+
+ /* Compare patches and store the one with higher revision */
+ if ( !patch && match_cpu(new_patch) )
+ patch = new_patch;
+ else if ( patch && (compare_patch(new_patch, patch) == NEW_UCODE) )
+ {
+ free_patch(patch);
+ patch = new_patch;
+ }
+ else
+ free_patch(new_patch);
xfree(mc);
}
@@ -416,10 +409,10 @@ static int cpu_request_microcode(const void *buf, size_t size)
if ( offset < 0 )
error = offset;
- if ( !error && match_cpu(microcode_get_cache()) )
- error = apply_microcode();
+ if ( error && !patch )
+ patch = ERR_PTR(error);
- return error;
+ return patch;
}
static const struct microcode_ops microcode_intel_ops = {
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index de19b67..cd5f9cc 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -363,10 +363,7 @@ void start_secondary(void *unused)
initialize_cpu_data(cpu);
- if ( system_state <= SYS_STATE_smp_boot )
- early_microcode_update_cpu(false);
- else
- microcode_resume_cpu();
+ early_microcode_update_cpu();
/*
* If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index e6842d4..3fa3acd 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -19,9 +19,10 @@ struct microcode_patch {
};
struct microcode_ops {
- int (*cpu_request_microcode)(const void *buf, size_t size);
+ struct microcode_patch *(*cpu_request_microcode)(const void *buf,
+ size_t size);
int (*collect_cpu_info)(struct cpu_signature *csig);
- int (*apply_microcode)(void);
+ int (*apply_microcode)(const struct microcode_patch *patch);
int (*start_update)(void);
void (*free_patch)(struct microcode_patch *patch);
bool (*match_cpu)(const struct microcode_patch *patch);
@@ -39,7 +40,4 @@ struct cpu_signature {
DECLARE_PER_CPU(struct cpu_signature, cpu_sig);
extern const struct microcode_ops *microcode_ops;
-const struct microcode_patch *microcode_get_cache(void);
-bool microcode_update_cache(struct microcode_patch *patch);
-
#endif /* ASM_X86__MICROCODE_H */
diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h
index 8b7e484..d8668e6 100644
--- a/xen/include/asm-x86/processor.h
+++ b/xen/include/asm-x86/processor.h
@@ -573,8 +573,7 @@ int guest_wrmsr_xen(struct vcpu *v, uint32_t idx, uint64_t val);
void microcode_set_module(unsigned int);
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void), unsigned long len);
-int microcode_resume_cpu(void);
-int early_microcode_update_cpu(bool start_update);
+int early_microcode_update_cpu(void);
int early_microcode_init(void);
int microcode_init_intel(void);
int microcode_init_amd(void);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode()
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
During late microcode update, apply_microcode() is invoked in
cpu_request_microcode(). To make late microcode update more reliable,
we want to put the apply_microcode() into stop_machine context. So
we split out it from cpu_request_microcode(). As a consequence,
apply_microcode() should be invoked explicitly in the common code.
Previously, apply_microcode() gets the microcode patch to be applied from
the microcode cache. Now, the patch is passed as a function argument and
a patch is cached for cpu-hotplug and cpu resuming, only after it has
been loaded to a cpu without any error. As a consequence, the
'match_cpu' check in microcode_update_cache is removed, which otherwise
would fail.
Assuming that all CPUs have the same signature, one patch matching with
current CPU should match with others. Then parsing microcode only needs
to be done once; cpu_request_microcode() is also moved out of
microcode_update_cpu().
On AMD side, svm_host_osvw_init() is supposed to be called after
microcode update. As apply_micrcode() won't be called by
cpu_request_microcode() now, svm_host_osvw_init() is moved to the
end of apply_microcode().
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- to handle load failure, unvalidated patches won't be cached. They
are passed as function arguments. So if update failed, we needn't
any cleanup to microcode cache.
- microcode_info which passes microcode blob to be parsed to each CPU is
replaced by microcode_patch.
Changes in v6:
- during early microcode update, BSP and APs call different functions.
Thus AP can bypass parsing microcode blob.
---
xen/arch/x86/acpi/power.c | 2 +-
xen/arch/x86/microcode.c | 209 ++++++++++++++++++++++++++--------------
xen/arch/x86/microcode_amd.c | 41 ++++----
xen/arch/x86/microcode_intel.c | 69 ++++++-------
xen/arch/x86/smpboot.c | 5 +-
xen/include/asm-x86/microcode.h | 8 +-
xen/include/asm-x86/processor.h | 3 +-
7 files changed, 193 insertions(+), 144 deletions(-)
diff --git a/xen/arch/x86/acpi/power.c b/xen/arch/x86/acpi/power.c
index 4f21903..9583172 100644
--- a/xen/arch/x86/acpi/power.c
+++ b/xen/arch/x86/acpi/power.c
@@ -253,7 +253,7 @@ static int enter_state(u32 state)
console_end_sync();
- microcode_resume_cpu();
+ early_microcode_update_cpu();
if ( !recheck_cpu_features(0) )
panic("Missing previously available feature(s)\n");
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 16a6d50..23cf550 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -189,36 +189,62 @@ static DEFINE_SPINLOCK(microcode_mutex);
DEFINE_PER_CPU(struct cpu_signature, cpu_sig);
-struct microcode_info {
- unsigned int cpu;
- uint32_t buffer_size;
- int error;
- char buffer[1];
-};
+/*
+ * Return the patch with the highest revision id among all matching
+ * patches in the blob. Return NULL if no suitable patch.
+ */
+static struct microcode_patch *microcode_parse_blob(const char *buf,
+ uint32_t len)
+{
+ if ( likely(!microcode_ops->collect_cpu_info(&this_cpu(cpu_sig))) )
+ return microcode_ops->cpu_request_microcode(buf, len);
-int microcode_resume_cpu(void)
+ return NULL;
+}
+
+/*
+ * Load a microcode update to current CPU.
+ *
+ * If no patch is provided, the cached patch will be loaded. Microcode update
+ * during APs bringup and CPU resuming falls into this case.
+ */
+static int microcode_update_cpu(struct microcode_patch *patch)
{
- int err;
- struct cpu_signature *sig = &this_cpu(cpu_sig);
+ int ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
- if ( !microcode_ops )
- return 0;
+ if ( unlikely(ret) )
+ return ret;
spin_lock(µcode_mutex);
- err = microcode_ops->collect_cpu_info(sig);
- if ( likely(!err) )
- err = microcode_ops->apply_microcode();
- spin_unlock(µcode_mutex);
+ if ( patch )
+ {
+ /*
+ * If a patch is specified, it should has newer revision than
+ * that of the patch cached.
+ */
+ if ( microcode_cache &&
+ microcode_ops->compare_patch(patch, microcode_cache) != NEW_UCODE )
+ {
+ spin_unlock(µcode_mutex);
+ return -EINVAL;
+ }
- return err;
-}
+ ret = microcode_ops->apply_microcode(patch);
+ }
+ else if ( microcode_cache )
+ {
+ ret = microcode_ops->apply_microcode(microcode_cache);
+ if ( ret == -EIO )
+ printk("Update failed. Reboot needed\n");
+ }
+ else
+ /* No patch to update */
+ ret = -EINVAL;
-const struct microcode_patch *microcode_get_cache(void)
-{
- ASSERT(spin_is_locked(µcode_mutex));
+ spin_unlock(µcode_mutex);
- return microcode_cache;
+ return ret;
}
/* Return true if cache gets updated. Otherwise, return false */
@@ -227,9 +253,6 @@ bool microcode_update_cache(struct microcode_patch *patch)
ASSERT(spin_is_locked(µcode_mutex));
- if ( !microcode_ops->match_cpu(patch) )
- return false;
-
if ( !microcode_cache )
microcode_cache = patch;
else if ( microcode_ops->compare_patch(patch, microcode_cache) ==
@@ -247,46 +270,32 @@ bool microcode_update_cache(struct microcode_patch *patch)
return true;
}
-static int microcode_update_cpu(const void *buf, size_t size)
+static long do_microcode_update(void *patch)
{
- int err;
- unsigned int cpu = smp_processor_id();
- struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
-
- spin_lock(µcode_mutex);
+ int error, cpu;
- err = microcode_ops->collect_cpu_info(sig);
- if ( likely(!err) )
- err = microcode_ops->cpu_request_microcode(buf, size);
- spin_unlock(µcode_mutex);
-
- return err;
-}
-
-static long do_microcode_update(void *_info)
-{
- struct microcode_info *info = _info;
- int error;
+ error = microcode_update_cpu(patch);
+ if ( error )
+ {
+ microcode_ops->free_patch(microcode_cache);
+ return error;
+ }
- BUG_ON(info->cpu != smp_processor_id());
- error = microcode_update_cpu(info->buffer, info->buffer_size);
- if ( error )
- info->error = error;
+ cpu = cpumask_next(smp_processor_id(), &cpu_online_map);
+ if ( cpu < nr_cpu_ids )
+ return continue_hypercall_on_cpu(cpu, do_microcode_update, patch);
- info->cpu = cpumask_next(info->cpu, &cpu_online_map);
- if ( info->cpu < nr_cpu_ids )
- return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
+ microcode_update_cache(patch);
- error = info->error;
- xfree(info);
return error;
}
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
{
int ret;
- struct microcode_info *info;
+ void *buffer;
+ struct microcode_patch *patch;
if ( len != (uint32_t)len )
return -E2BIG;
@@ -294,32 +303,49 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
if ( microcode_ops == NULL )
return -EINVAL;
- info = xmalloc_bytes(sizeof(*info) + len);
- if ( info == NULL )
- return -ENOMEM;
-
- ret = copy_from_guest(info->buffer, buf, len);
- if ( ret != 0 )
+ buffer = xmalloc_bytes(len);
+ if ( !buffer )
{
- xfree(info);
- return ret;
+ ret = -ENOMEM;
+ goto free;
}
- info->buffer_size = len;
- info->error = 0;
- info->cpu = cpumask_first(&cpu_online_map);
+ if ( copy_from_guest(buffer, buf, len) )
+ {
+ ret = -EFAULT;
+ goto free;
+ }
if ( microcode_ops->start_update )
{
ret = microcode_ops->start_update();
if ( ret != 0 )
- {
- xfree(info);
- return ret;
- }
+ goto free;
}
- return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
+ patch = microcode_parse_blob(buffer, len);
+ if ( IS_ERR(patch) )
+ {
+ printk(XENLOG_ERR "Parsing microcode blob error %ld\n", PTR_ERR(patch));
+ ret = PTR_ERR(patch);
+ goto free;
+ }
+
+ if ( !microcode_ops->match_cpu(patch) )
+ {
+ printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
+ if ( patch )
+ microcode_ops->free_patch(patch);
+ ret = -EINVAL;
+ goto free;
+ }
+
+ ret = continue_hypercall_on_cpu(cpumask_first(&cpu_online_map),
+ do_microcode_update, patch);
+
+ free:
+ xfree(buffer);
+ return ret;
}
static int __init microcode_init(void)
@@ -344,7 +370,16 @@ static int __init microcode_init(void)
}
__initcall(microcode_init);
-int __init early_microcode_update_cpu(bool start_update)
+int early_microcode_update_cpu(void)
+{
+ return microcode_ops ? microcode_update_cpu(NULL) : 0;
+}
+
+/*
+ * BSP needs to parse the ucode blob and then apply an update.
+ * APs just apply an update by calling early_microcode_update_cpu().
+ */
+static int __init early_microcode_parse_and_update_cpu(void)
{
int rc = 0;
void *data = NULL;
@@ -362,13 +397,41 @@ int __init early_microcode_update_cpu(bool start_update)
}
if ( data )
{
- if ( start_update && microcode_ops->start_update )
+ struct microcode_patch *patch;
+
+ if ( microcode_ops->start_update )
rc = microcode_ops->start_update();
if ( rc )
return rc;
- return microcode_update_cpu(data, len);
+ patch = microcode_parse_blob(data, len);
+ if ( IS_ERR(patch) )
+ {
+ printk(XENLOG_ERR "Parsing microcode blob error %ld\n",
+ PTR_ERR(patch));
+ return PTR_ERR(patch);
+ }
+
+ if ( !microcode_ops->match_cpu(patch) )
+ {
+ printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
+ if ( patch )
+ microcode_ops->free_patch(patch);
+ return -EINVAL;
+ }
+
+ rc = microcode_update_cpu(patch);
+ if ( !rc )
+ {
+ spin_lock(µcode_mutex);
+ microcode_update_cache(patch);
+ spin_unlock(µcode_mutex);
+ }
+ else
+ microcode_ops->free_patch(patch);
+
+ return rc;
}
else
return -ENOMEM;
@@ -387,8 +450,10 @@ int __init early_microcode_init(void)
return rc;
if ( microcode_ops )
+ {
if ( ucode_mod.mod_end || ucode_blob.size )
- rc = early_microcode_update_cpu(true);
+ rc = early_microcode_parse_and_update_cpu();
+ }
return rc;
}
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index 0144df1..c819028 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -249,7 +249,7 @@ static enum microcode_match_result compare_patch(
return MIS_UCODE;
}
-static int apply_microcode(void)
+static int apply_microcode(const struct microcode_patch *patch)
{
unsigned long flags;
uint32_t rev;
@@ -257,7 +257,6 @@ static int apply_microcode(void)
unsigned int cpu = smp_processor_id();
struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
const struct microcode_header_amd *hdr;
- const struct microcode_patch *patch = microcode_get_cache();
if ( !match_cpu(patch) )
return -EINVAL;
@@ -292,6 +291,10 @@ static int apply_microcode(void)
sig->rev = rev;
+#ifdef CONFIG_HVM
+ svm_host_osvw_init();
+#endif
+
return 0;
}
@@ -449,9 +452,11 @@ static bool check_final_patch_levels(void)
return 0;
}
-static int cpu_request_microcode(const void *buf, size_t bufsize)
+static struct microcode_patch *cpu_request_microcode(const void *buf,
+ size_t bufsize)
{
struct microcode_amd *mc_amd;
+ struct microcode_patch *patch = NULL;
size_t offset = 0;
int error = 0;
unsigned int current_cpu_id;
@@ -551,17 +556,16 @@ static int cpu_request_microcode(const void *buf, size_t bufsize)
break;
}
- if ( match_cpu(new_patch) )
- microcode_update_cache(new_patch);
- else
- free_patch(new_patch);
-
- if ( match_cpu(microcode_get_cache()) )
+ /* Compare patches and store the one with higher revision */
+ if ( !patch && match_cpu(new_patch) )
+ patch = new_patch;
+ else if ( patch && (compare_patch(new_patch, patch) == NEW_UCODE) )
{
- error = apply_microcode();
- if ( error )
- break;
+ free_patch(patch);
+ patch = new_patch;
}
+ else
+ free_patch(new_patch);
if ( offset >= bufsize )
break;
@@ -592,17 +596,10 @@ static int cpu_request_microcode(const void *buf, size_t bufsize)
}
out:
-#if CONFIG_HVM
- svm_host_osvw_init();
-#endif
+ if ( error && !patch )
+ patch = ERR_PTR(error);
- /*
- * In some cases we may return an error even if processor's microcode has
- * been updated. For example, the first patch in a container file is loaded
- * successfully but subsequent container file processing encounters a
- * failure.
- */
- return error;
+ return patch;
}
static int start_update(void)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index b66844d..650495d 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -273,46 +273,27 @@ static enum microcode_match_result compare_patch(
old_header->pf, old_header->rev);
}
-/*
- * return 0 - no update found
- * return 1 - found update
- * return < 0 - error
- */
-static int get_matching_microcode(const void *mc)
+static struct microcode_patch *allow_microcode_patch(
+ const struct microcode_header_intel *mc_header)
{
- const struct microcode_header_intel *mc_header = mc;
unsigned long total_size = get_totalsize(mc_header);
void *new_mc = xmalloc_bytes(total_size);
struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
- unsigned int __maybe_unused cpu = smp_processor_id();
if ( !new_patch || !new_mc )
{
xfree(new_patch);
xfree(new_mc);
printk(XENLOG_ERR "microcode: Can not allocate memory\n");
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
}
- memcpy(new_mc, mc, total_size);
+ memcpy(new_mc, mc_header, total_size);
new_patch->mc_intel = new_mc;
- if ( !match_cpu(new_patch) )
- {
- free_patch(new_patch);
- return 0;
- }
-
- if ( !microcode_update_cache(new_patch) )
- return 0;
-
- pr_debug("microcode: CPU%d found a matching microcode update with"
- " version %#x (current=%#x)\n",
- cpu, mc_header->rev, this_cpu(cpu_sig).rev);
-
- return 1;
+ return new_patch;
}
-static int apply_microcode(void)
+static int apply_microcode(const struct microcode_patch *patch)
{
unsigned long flags;
uint64_t msr_content;
@@ -320,7 +301,6 @@ static int apply_microcode(void)
unsigned int cpu_num = raw_smp_processor_id();
struct cpu_signature *sig = &this_cpu(cpu_sig);
const struct microcode_intel *mc_intel;
- const struct microcode_patch *patch = microcode_get_cache();
if ( !match_cpu(patch) )
return -EINVAL;
@@ -388,26 +368,39 @@ static long get_next_ucode_from_buffer(void **mc, const u8 *buf,
return offset + total_size;
}
-static int cpu_request_microcode(const void *buf, size_t size)
+static struct microcode_patch *cpu_request_microcode(const void *buf,
+ size_t size)
{
long offset = 0;
int error = 0;
void *mc;
+ struct microcode_patch *patch = NULL;
while ( (offset = get_next_ucode_from_buffer(&mc, buf, size, offset)) > 0 )
{
+ struct microcode_patch *new_patch;
+
error = microcode_sanity_check(mc);
if ( error )
break;
- error = get_matching_microcode(mc);
- if ( error < 0 )
+
+ new_patch = allow_microcode_patch(mc);
+ if ( IS_ERR(new_patch) )
+ {
+ error = PTR_ERR(new_patch);
break;
- /*
- * It's possible the data file has multiple matching ucode,
- * lets keep searching till the latest version
- */
- if ( error == 1 )
- error = 0;
+ }
+
+ /* Compare patches and store the one with higher revision */
+ if ( !patch && match_cpu(new_patch) )
+ patch = new_patch;
+ else if ( patch && (compare_patch(new_patch, patch) == NEW_UCODE) )
+ {
+ free_patch(patch);
+ patch = new_patch;
+ }
+ else
+ free_patch(new_patch);
xfree(mc);
}
@@ -416,10 +409,10 @@ static int cpu_request_microcode(const void *buf, size_t size)
if ( offset < 0 )
error = offset;
- if ( !error && match_cpu(microcode_get_cache()) )
- error = apply_microcode();
+ if ( error && !patch )
+ patch = ERR_PTR(error);
- return error;
+ return patch;
}
static const struct microcode_ops microcode_intel_ops = {
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index de19b67..cd5f9cc 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -363,10 +363,7 @@ void start_secondary(void *unused)
initialize_cpu_data(cpu);
- if ( system_state <= SYS_STATE_smp_boot )
- early_microcode_update_cpu(false);
- else
- microcode_resume_cpu();
+ early_microcode_update_cpu();
/*
* If MSR_SPEC_CTRL is available, apply Xen's default setting and discard
diff --git a/xen/include/asm-x86/microcode.h b/xen/include/asm-x86/microcode.h
index e6842d4..3fa3acd 100644
--- a/xen/include/asm-x86/microcode.h
+++ b/xen/include/asm-x86/microcode.h
@@ -19,9 +19,10 @@ struct microcode_patch {
};
struct microcode_ops {
- int (*cpu_request_microcode)(const void *buf, size_t size);
+ struct microcode_patch *(*cpu_request_microcode)(const void *buf,
+ size_t size);
int (*collect_cpu_info)(struct cpu_signature *csig);
- int (*apply_microcode)(void);
+ int (*apply_microcode)(const struct microcode_patch *patch);
int (*start_update)(void);
void (*free_patch)(struct microcode_patch *patch);
bool (*match_cpu)(const struct microcode_patch *patch);
@@ -39,7 +40,4 @@ struct cpu_signature {
DECLARE_PER_CPU(struct cpu_signature, cpu_sig);
extern const struct microcode_ops *microcode_ops;
-const struct microcode_patch *microcode_get_cache(void);
-bool microcode_update_cache(struct microcode_patch *patch);
-
#endif /* ASM_X86__MICROCODE_H */
diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processor.h
index 8b7e484..d8668e6 100644
--- a/xen/include/asm-x86/processor.h
+++ b/xen/include/asm-x86/processor.h
@@ -573,8 +573,7 @@ int guest_wrmsr_xen(struct vcpu *v, uint32_t idx, uint64_t val);
void microcode_set_module(unsigned int);
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void), unsigned long len);
-int microcode_resume_cpu(void);
-int early_microcode_update_cpu(bool start_update);
+int early_microcode_update_cpu(void);
int early_microcode_init(void);
int microcode_init_intel(void);
int microcode_init_amd(void);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 07/10] microcode/intel: Writeback and invalidate caches before updating microcode
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
Updating microcode is less error prone when caches have been flushed and
depending on what exactly the microcode is updating. For example, some
of the issues around certain Broadwell parts can be addressed by doing a
full cache flush.
With parallel microcode update, the cost of this patch is hardly
noticable. Although only BDX with an old microcode needs this fix, we
would like to avoid future issues in case they come by later due to
other reasons.
[linux commit: 91df9fdf51492aec9fed6b4cbd33160886740f47]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Cc: Ashok Raj <ashok.raj@intel.com>
---
Changes in v7:
- explain why we do 'wbinvd' unconditionally rather than only for BDX
in commit message
Changes in v6:
- new
---
xen/arch/x86/microcode_intel.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index 650495d..bfb48ce 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -310,6 +310,12 @@ static int apply_microcode(const struct microcode_patch *patch)
/* serialize access to the physical write to MSR 0x79 */
spin_lock_irqsave(µcode_update_lock, flags);
+ /*
+ * Writeback and invalidate caches before updating microcode to avoid
+ * internal issues depending on what the microcode is updating.
+ */
+ wbinvd();
+
/* write microcode via MSR 0x79 */
wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc_intel->bits);
wrmsrl(MSR_IA32_UCODE_REV, 0x0ULL);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 07/10] microcode/intel: Writeback and invalidate caches before updating microcode
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
Updating microcode is less error prone when caches have been flushed and
depending on what exactly the microcode is updating. For example, some
of the issues around certain Broadwell parts can be addressed by doing a
full cache flush.
With parallel microcode update, the cost of this patch is hardly
noticable. Although only BDX with an old microcode needs this fix, we
would like to avoid future issues in case they come by later due to
other reasons.
[linux commit: 91df9fdf51492aec9fed6b4cbd33160886740f47]
Signed-off-by: Chao Gao <chao.gao@intel.com>
Cc: Ashok Raj <ashok.raj@intel.com>
---
Changes in v7:
- explain why we do 'wbinvd' unconditionally rather than only for BDX
in commit message
Changes in v6:
- new
---
xen/arch/x86/microcode_intel.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index 650495d..bfb48ce 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -310,6 +310,12 @@ static int apply_microcode(const struct microcode_patch *patch)
/* serialize access to the physical write to MSR 0x79 */
spin_lock_irqsave(µcode_update_lock, flags);
+ /*
+ * Writeback and invalidate caches before updating microcode to avoid
+ * internal issues depending on what the microcode is updating.
+ */
+ wbinvd();
+
/* write microcode via MSR 0x79 */
wrmsrl(MSR_IA32_UCODE_WRITE, (unsigned long)mc_intel->bits);
wrmsrl(MSR_IA32_UCODE_REV, 0x0ULL);
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Kevin Tian, Borislav Petkov, Ashok Raj, Wei Liu,
Jun Nakajima, Andrew Cooper, Jan Beulich, Thomas Gleixner,
Chao Gao, Roger Pau Monné
This patch ports microcode improvement patches from linux kernel.
Before you read any further: the early loading method is still the
preferred one and you should always do that. The following patch is
improving the late loading mechanism for long running jobs and cloud use
cases.
Gather all cores and serialize the microcode update on them by doing it
one-by-one to make the late update process as reliable as possible and
avoid potential issues caused by the microcode update.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Tested-by: Chao Gao <chao.gao@intel.com>
[linux commit: a5321aec6412b20b5ad15db2d6b916c05349dbff]
[linux commit: bb8c13d61a629276a162c1d2b1a20a815cbcfbb7]
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
---
Changes in v7:
- Check whether 'timeout' is 0 rather than "<=0" since it is unsigned int.
- reword the comment above microcode_update_cpu() to clearly state that
one thread per core should do the update.
Changes in v6:
- Use one timeout period for rendezvous stage and another for update stage.
- scale time to wait by the number of remaining cpus to respond.
It helps to find something wrong earlier and thus we can reboot the
system earlier.
---
xen/arch/x86/microcode.c | 171 ++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 155 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 23cf550..f4a417e 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -22,6 +22,7 @@
*/
#include <xen/cpu.h>
+#include <xen/cpumask.h>
#include <xen/lib.h>
#include <xen/kernel.h>
#include <xen/init.h>
@@ -30,15 +31,34 @@
#include <xen/smp.h>
#include <xen/softirq.h>
#include <xen/spinlock.h>
+#include <xen/stop_machine.h>
#include <xen/tasklet.h>
#include <xen/guest_access.h>
#include <xen/earlycpio.h>
+#include <xen/watchdog.h>
+#include <asm/delay.h>
#include <asm/msr.h>
#include <asm/processor.h>
#include <asm/setup.h>
#include <asm/microcode.h>
+/*
+ * Before performing a late microcode update on any thread, we
+ * rendezvous all cpus in stop_machine context. The timeout for
+ * waiting for cpu rendezvous is 30ms. It is the timeout used by
+ * live patching
+ */
+#define MICROCODE_CALLIN_TIMEOUT_US 30000
+
+/*
+ * Timeout for each thread to complete update is set to 1s. It is a
+ * conservative choice considering all possible interference (for
+ * instance, sometimes wbinvd takes relative long time). And a perfect
+ * timeout doesn't help a lot except an early shutdown.
+ */
+#define MICROCODE_UPDATE_TIMEOUT_US 1000000
+
static module_t __initdata ucode_mod;
static signed int __initdata ucode_mod_idx;
static bool_t __initdata ucode_mod_forced;
@@ -190,6 +210,12 @@ static DEFINE_SPINLOCK(microcode_mutex);
DEFINE_PER_CPU(struct cpu_signature, cpu_sig);
/*
+ * Count the CPUs that have entered, exited the rendezvous and succeeded in
+ * microcode update during late microcode update respectively.
+ */
+static atomic_t cpu_in, cpu_out, cpu_updated;
+
+/*
* Return the patch with the highest revision id among all matching
* patches in the blob. Return NULL if no suitable patch.
*/
@@ -270,31 +296,90 @@ bool microcode_update_cache(struct microcode_patch *patch)
return true;
}
-static long do_microcode_update(void *patch)
+/* Wait for CPUs to rendezvous with a timeout (us) */
+static int wait_for_cpus(atomic_t *cnt, unsigned int expect,
+ unsigned int timeout)
{
- int error, cpu;
-
- error = microcode_update_cpu(patch);
- if ( error )
+ while ( atomic_read(cnt) < expect )
{
- microcode_ops->free_patch(microcode_cache);
- return error;
+ if ( !timeout )
+ {
+ printk("CPU%d: Timeout when waiting for CPUs calling in\n",
+ smp_processor_id());
+ return -EBUSY;
+ }
+ udelay(1);
+ timeout--;
}
+ return 0;
+}
- cpu = cpumask_next(smp_processor_id(), &cpu_online_map);
- if ( cpu < nr_cpu_ids )
- return continue_hypercall_on_cpu(cpu, do_microcode_update, patch);
+static int do_microcode_update(void *patch)
+{
+ unsigned int cpu = smp_processor_id();
+ unsigned int cpu_nr = num_online_cpus();
+ unsigned int finished;
+ int ret;
+ static bool error;
- microcode_update_cache(patch);
+ atomic_inc(&cpu_in);
+ ret = wait_for_cpus(&cpu_in, cpu_nr, MICROCODE_CALLIN_TIMEOUT_US);
+ if ( ret )
+ return ret;
- return error;
+ ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
+ /*
+ * Load microcode update on only one logical processor per core.
+ * Here, among logical processors of a core, the one with the
+ * lowest thread id is chosen to perform the loading.
+ */
+ if ( !ret && (cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu))) )
+ {
+ ret = microcode_ops->apply_microcode(patch);
+ if ( !ret )
+ atomic_inc(&cpu_updated);
+ }
+ /*
+ * Increase the wait timeout to a safe value here since we're serializing
+ * the microcode update and that could take a while on a large number of
+ * CPUs. And that is fine as the *actual* timeout will be determined by
+ * the last CPU finished updating and thus cut short
+ */
+ atomic_inc(&cpu_out);
+ finished = atomic_read(&cpu_out);
+ while ( !error && finished != cpu_nr )
+ {
+ /*
+ * During each timeout interval, at least a CPU is expected to
+ * finish its update. Otherwise, something goes wrong.
+ */
+ if ( wait_for_cpus(&cpu_out, finished + 1,
+ MICROCODE_UPDATE_TIMEOUT_US) && !error )
+ {
+ error = true;
+ panic("Timeout when finishing updating microcode (finished %d/%d)",
+ finished, cpu_nr);
+ }
+
+ finished = atomic_read(&cpu_out);
+ }
+
+ /*
+ * Refresh CPU signature (revision) on threads which didn't call
+ * apply_microcode().
+ */
+ if ( cpu != cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
+ ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
+
+ return ret;
}
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
{
int ret;
void *buffer;
+ unsigned int cpu, nr_cores;
struct microcode_patch *patch;
if ( len != (uint32_t)len )
@@ -316,11 +401,18 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
goto free;
}
+ /* cpu_online_map must not change during update */
+ if ( !get_cpu_maps() )
+ {
+ ret = -EBUSY;
+ goto free;
+ }
+
if ( microcode_ops->start_update )
{
ret = microcode_ops->start_update();
if ( ret != 0 )
- goto free;
+ goto put;
}
patch = microcode_parse_blob(buffer, len);
@@ -337,12 +429,59 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
if ( patch )
microcode_ops->free_patch(patch);
ret = -EINVAL;
- goto free;
+ goto put;
}
- ret = continue_hypercall_on_cpu(cpumask_first(&cpu_online_map),
- do_microcode_update, patch);
+ atomic_set(&cpu_in, 0);
+ atomic_set(&cpu_out, 0);
+ atomic_set(&cpu_updated, 0);
+
+ /* Calculate the number of online CPU core */
+ nr_cores = 0;
+ for_each_online_cpu(cpu)
+ if ( cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
+ nr_cores++;
+
+ printk(XENLOG_INFO "%d cores are to update their microcode\n", nr_cores);
+
+ /*
+ * We intend to disable interrupt for long time, which may lead to
+ * watchdog timeout.
+ */
+ watchdog_disable();
+ /*
+ * Late loading dance. Why the heavy-handed stop_machine effort?
+ *
+ * - HT siblings must be idle and not execute other code while the other
+ * sibling is loading microcode in order to avoid any negative
+ * interactions cause by the loading.
+ *
+ * - In addition, microcode update on the cores must be serialized until
+ * this requirement can be relaxed in the future. Right now, this is
+ * conservative and good.
+ */
+ ret = stop_machine_run(do_microcode_update, patch, NR_CPUS);
+ watchdog_enable();
+
+ if ( atomic_read(&cpu_updated) == nr_cores )
+ {
+ spin_lock(µcode_mutex);
+ microcode_update_cache(patch);
+ spin_unlock(µcode_mutex);
+ }
+ else if ( atomic_read(&cpu_updated) == 0 )
+ microcode_ops->free_patch(patch);
+ else
+ {
+ printk("Updating microcode succeeded on part of CPUs and failed on\n"
+ "others due to an unknown reason. A system with different\n"
+ "microcode revisions is considered unstable. Please reboot and\n"
+ "do not load the microcode that triggers this warning\n");
+ microcode_ops->free_patch(patch);
+ }
+ put:
+ put_cpu_maps();
free:
xfree(buffer);
return ret;
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Kevin Tian, Borislav Petkov, Ashok Raj, Wei Liu,
Jun Nakajima, Andrew Cooper, Jan Beulich, Thomas Gleixner,
Chao Gao, Roger Pau Monné
This patch ports microcode improvement patches from linux kernel.
Before you read any further: the early loading method is still the
preferred one and you should always do that. The following patch is
improving the late loading mechanism for long running jobs and cloud use
cases.
Gather all cores and serialize the microcode update on them by doing it
one-by-one to make the late update process as reliable as possible and
avoid potential issues caused by the microcode update.
Signed-off-by: Chao Gao <chao.gao@intel.com>
Tested-by: Chao Gao <chao.gao@intel.com>
[linux commit: a5321aec6412b20b5ad15db2d6b916c05349dbff]
[linux commit: bb8c13d61a629276a162c1d2b1a20a815cbcfbb7]
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>
Cc: Ashok Raj <ashok.raj@intel.com>
Cc: Borislav Petkov <bp@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
---
Changes in v7:
- Check whether 'timeout' is 0 rather than "<=0" since it is unsigned int.
- reword the comment above microcode_update_cpu() to clearly state that
one thread per core should do the update.
Changes in v6:
- Use one timeout period for rendezvous stage and another for update stage.
- scale time to wait by the number of remaining cpus to respond.
It helps to find something wrong earlier and thus we can reboot the
system earlier.
---
xen/arch/x86/microcode.c | 171 ++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 155 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index 23cf550..f4a417e 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -22,6 +22,7 @@
*/
#include <xen/cpu.h>
+#include <xen/cpumask.h>
#include <xen/lib.h>
#include <xen/kernel.h>
#include <xen/init.h>
@@ -30,15 +31,34 @@
#include <xen/smp.h>
#include <xen/softirq.h>
#include <xen/spinlock.h>
+#include <xen/stop_machine.h>
#include <xen/tasklet.h>
#include <xen/guest_access.h>
#include <xen/earlycpio.h>
+#include <xen/watchdog.h>
+#include <asm/delay.h>
#include <asm/msr.h>
#include <asm/processor.h>
#include <asm/setup.h>
#include <asm/microcode.h>
+/*
+ * Before performing a late microcode update on any thread, we
+ * rendezvous all cpus in stop_machine context. The timeout for
+ * waiting for cpu rendezvous is 30ms. It is the timeout used by
+ * live patching
+ */
+#define MICROCODE_CALLIN_TIMEOUT_US 30000
+
+/*
+ * Timeout for each thread to complete update is set to 1s. It is a
+ * conservative choice considering all possible interference (for
+ * instance, sometimes wbinvd takes relative long time). And a perfect
+ * timeout doesn't help a lot except an early shutdown.
+ */
+#define MICROCODE_UPDATE_TIMEOUT_US 1000000
+
static module_t __initdata ucode_mod;
static signed int __initdata ucode_mod_idx;
static bool_t __initdata ucode_mod_forced;
@@ -190,6 +210,12 @@ static DEFINE_SPINLOCK(microcode_mutex);
DEFINE_PER_CPU(struct cpu_signature, cpu_sig);
/*
+ * Count the CPUs that have entered, exited the rendezvous and succeeded in
+ * microcode update during late microcode update respectively.
+ */
+static atomic_t cpu_in, cpu_out, cpu_updated;
+
+/*
* Return the patch with the highest revision id among all matching
* patches in the blob. Return NULL if no suitable patch.
*/
@@ -270,31 +296,90 @@ bool microcode_update_cache(struct microcode_patch *patch)
return true;
}
-static long do_microcode_update(void *patch)
+/* Wait for CPUs to rendezvous with a timeout (us) */
+static int wait_for_cpus(atomic_t *cnt, unsigned int expect,
+ unsigned int timeout)
{
- int error, cpu;
-
- error = microcode_update_cpu(patch);
- if ( error )
+ while ( atomic_read(cnt) < expect )
{
- microcode_ops->free_patch(microcode_cache);
- return error;
+ if ( !timeout )
+ {
+ printk("CPU%d: Timeout when waiting for CPUs calling in\n",
+ smp_processor_id());
+ return -EBUSY;
+ }
+ udelay(1);
+ timeout--;
}
+ return 0;
+}
- cpu = cpumask_next(smp_processor_id(), &cpu_online_map);
- if ( cpu < nr_cpu_ids )
- return continue_hypercall_on_cpu(cpu, do_microcode_update, patch);
+static int do_microcode_update(void *patch)
+{
+ unsigned int cpu = smp_processor_id();
+ unsigned int cpu_nr = num_online_cpus();
+ unsigned int finished;
+ int ret;
+ static bool error;
- microcode_update_cache(patch);
+ atomic_inc(&cpu_in);
+ ret = wait_for_cpus(&cpu_in, cpu_nr, MICROCODE_CALLIN_TIMEOUT_US);
+ if ( ret )
+ return ret;
- return error;
+ ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
+ /*
+ * Load microcode update on only one logical processor per core.
+ * Here, among logical processors of a core, the one with the
+ * lowest thread id is chosen to perform the loading.
+ */
+ if ( !ret && (cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu))) )
+ {
+ ret = microcode_ops->apply_microcode(patch);
+ if ( !ret )
+ atomic_inc(&cpu_updated);
+ }
+ /*
+ * Increase the wait timeout to a safe value here since we're serializing
+ * the microcode update and that could take a while on a large number of
+ * CPUs. And that is fine as the *actual* timeout will be determined by
+ * the last CPU finished updating and thus cut short
+ */
+ atomic_inc(&cpu_out);
+ finished = atomic_read(&cpu_out);
+ while ( !error && finished != cpu_nr )
+ {
+ /*
+ * During each timeout interval, at least a CPU is expected to
+ * finish its update. Otherwise, something goes wrong.
+ */
+ if ( wait_for_cpus(&cpu_out, finished + 1,
+ MICROCODE_UPDATE_TIMEOUT_US) && !error )
+ {
+ error = true;
+ panic("Timeout when finishing updating microcode (finished %d/%d)",
+ finished, cpu_nr);
+ }
+
+ finished = atomic_read(&cpu_out);
+ }
+
+ /*
+ * Refresh CPU signature (revision) on threads which didn't call
+ * apply_microcode().
+ */
+ if ( cpu != cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
+ ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
+
+ return ret;
}
int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
{
int ret;
void *buffer;
+ unsigned int cpu, nr_cores;
struct microcode_patch *patch;
if ( len != (uint32_t)len )
@@ -316,11 +401,18 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
goto free;
}
+ /* cpu_online_map must not change during update */
+ if ( !get_cpu_maps() )
+ {
+ ret = -EBUSY;
+ goto free;
+ }
+
if ( microcode_ops->start_update )
{
ret = microcode_ops->start_update();
if ( ret != 0 )
- goto free;
+ goto put;
}
patch = microcode_parse_blob(buffer, len);
@@ -337,12 +429,59 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
if ( patch )
microcode_ops->free_patch(patch);
ret = -EINVAL;
- goto free;
+ goto put;
}
- ret = continue_hypercall_on_cpu(cpumask_first(&cpu_online_map),
- do_microcode_update, patch);
+ atomic_set(&cpu_in, 0);
+ atomic_set(&cpu_out, 0);
+ atomic_set(&cpu_updated, 0);
+
+ /* Calculate the number of online CPU core */
+ nr_cores = 0;
+ for_each_online_cpu(cpu)
+ if ( cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
+ nr_cores++;
+
+ printk(XENLOG_INFO "%d cores are to update their microcode\n", nr_cores);
+
+ /*
+ * We intend to disable interrupt for long time, which may lead to
+ * watchdog timeout.
+ */
+ watchdog_disable();
+ /*
+ * Late loading dance. Why the heavy-handed stop_machine effort?
+ *
+ * - HT siblings must be idle and not execute other code while the other
+ * sibling is loading microcode in order to avoid any negative
+ * interactions cause by the loading.
+ *
+ * - In addition, microcode update on the cores must be serialized until
+ * this requirement can be relaxed in the future. Right now, this is
+ * conservative and good.
+ */
+ ret = stop_machine_run(do_microcode_update, patch, NR_CPUS);
+ watchdog_enable();
+
+ if ( atomic_read(&cpu_updated) == nr_cores )
+ {
+ spin_lock(µcode_mutex);
+ microcode_update_cache(patch);
+ spin_unlock(µcode_mutex);
+ }
+ else if ( atomic_read(&cpu_updated) == 0 )
+ microcode_ops->free_patch(patch);
+ else
+ {
+ printk("Updating microcode succeeded on part of CPUs and failed on\n"
+ "others due to an unknown reason. A system with different\n"
+ "microcode revisions is considered unstable. Please reboot and\n"
+ "do not load the microcode that triggers this warning\n");
+ microcode_ops->free_patch(patch);
+ }
+ put:
+ put_cpu_maps();
free:
xfree(buffer);
return ret;
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 09/10] microcode: remove microcode_update_lock
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
microcode_update_lock is to prevent logic threads of a same core from
updating microcode at the same time. But due to using a global lock, it
also prevented parallel microcode updating on different cores.
Remove this lock in order to update microcode in parallel. It is safe
because we have already ensured serialization of sibling threads at the
caller side.
1.For late microcode update, do_microcode_update() ensures that only one
sibiling thread of a core can update microcode.
2.For microcode update during system startup or CPU-hotplug,
microcode_mutex() guarantees update serialization of logical threads.
3.get/put_cpu_bitmaps() prevents the concurrency of CPU-hotplug and
late microcode update.
Note that printk in apply_microcode() and svm_host_osvm_init() (for AMD
only) are still processed sequentially.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- reworked. Remove complex lock logics introduced in v5 and v6. The microcode
patch to be applied is passed as an argument without any global variable. Thus
no lock is added to serialize potential readers/writers. Callers of
apply_microcode() will guarantee the correctness: the patch poninted by the
arguments won't be changed by others.
Changes in v6:
- introduce early_ucode_update_lock to serialize early ucode update.
Changes in v5:
- newly add
---
xen/arch/x86/microcode_amd.c | 8 +-------
xen/arch/x86/microcode_intel.c | 8 +-------
2 files changed, 2 insertions(+), 14 deletions(-)
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index c819028..b64a58d 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -74,9 +74,6 @@ struct mpbhdr {
uint8_t data[];
};
-/* serialize access to the physical write */
-static DEFINE_SPINLOCK(microcode_update_lock);
-
/* See comment in start_update() for cases when this routine fails */
static int collect_cpu_info(struct cpu_signature *csig)
{
@@ -251,7 +248,6 @@ static enum microcode_match_result compare_patch(
static int apply_microcode(const struct microcode_patch *patch)
{
- unsigned long flags;
uint32_t rev;
int hw_err;
unsigned int cpu = smp_processor_id();
@@ -263,15 +259,13 @@ static int apply_microcode(const struct microcode_patch *patch)
hdr = patch->mc_amd->mpb;
- spin_lock_irqsave(µcode_update_lock, flags);
+ BUG_ON(local_irq_is_enabled());
hw_err = wrmsr_safe(MSR_AMD_PATCHLOADER, (unsigned long)hdr);
/* get patch id after patching */
rdmsrl(MSR_AMD_PATCHLEVEL, rev);
- spin_unlock_irqrestore(µcode_update_lock, flags);
-
/*
* Some processors leave the ucode blob mapping as UC after the update.
* Flush the mapping to regain normal cacheability.
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index bfb48ce..94a1561 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -93,9 +93,6 @@ struct extended_sigtable {
#define exttable_size(et) ((et)->count * EXT_SIGNATURE_SIZE + EXT_HEADER_SIZE)
-/* serialize access to the physical write to MSR 0x79 */
-static DEFINE_SPINLOCK(microcode_update_lock);
-
static int collect_cpu_info(struct cpu_signature *csig)
{
unsigned int cpu_num = smp_processor_id();
@@ -295,7 +292,6 @@ static struct microcode_patch *allow_microcode_patch(
static int apply_microcode(const struct microcode_patch *patch)
{
- unsigned long flags;
uint64_t msr_content;
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
@@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch *patch)
mc_intel = patch->mc_intel;
- /* serialize access to the physical write to MSR 0x79 */
- spin_lock_irqsave(µcode_update_lock, flags);
+ BUG_ON(local_irq_is_enabled());
/*
* Writeback and invalidate caches before updating microcode to avoid
@@ -327,7 +322,6 @@ static int apply_microcode(const struct microcode_patch *patch)
rdmsrl(MSR_IA32_UCODE_REV, msr_content);
val[1] = (uint32_t)(msr_content >> 32);
- spin_unlock_irqrestore(µcode_update_lock, flags);
if ( val[1] != mc_intel->hdr.rev )
{
printk(KERN_ERR "microcode: CPU%d update from revision "
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
microcode_update_lock is to prevent logic threads of a same core from
updating microcode at the same time. But due to using a global lock, it
also prevented parallel microcode updating on different cores.
Remove this lock in order to update microcode in parallel. It is safe
because we have already ensured serialization of sibling threads at the
caller side.
1.For late microcode update, do_microcode_update() ensures that only one
sibiling thread of a core can update microcode.
2.For microcode update during system startup or CPU-hotplug,
microcode_mutex() guarantees update serialization of logical threads.
3.get/put_cpu_bitmaps() prevents the concurrency of CPU-hotplug and
late microcode update.
Note that printk in apply_microcode() and svm_host_osvm_init() (for AMD
only) are still processed sequentially.
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
Changes in v7:
- reworked. Remove complex lock logics introduced in v5 and v6. The microcode
patch to be applied is passed as an argument without any global variable. Thus
no lock is added to serialize potential readers/writers. Callers of
apply_microcode() will guarantee the correctness: the patch poninted by the
arguments won't be changed by others.
Changes in v6:
- introduce early_ucode_update_lock to serialize early ucode update.
Changes in v5:
- newly add
---
xen/arch/x86/microcode_amd.c | 8 +-------
xen/arch/x86/microcode_intel.c | 8 +-------
2 files changed, 2 insertions(+), 14 deletions(-)
diff --git a/xen/arch/x86/microcode_amd.c b/xen/arch/x86/microcode_amd.c
index c819028..b64a58d 100644
--- a/xen/arch/x86/microcode_amd.c
+++ b/xen/arch/x86/microcode_amd.c
@@ -74,9 +74,6 @@ struct mpbhdr {
uint8_t data[];
};
-/* serialize access to the physical write */
-static DEFINE_SPINLOCK(microcode_update_lock);
-
/* See comment in start_update() for cases when this routine fails */
static int collect_cpu_info(struct cpu_signature *csig)
{
@@ -251,7 +248,6 @@ static enum microcode_match_result compare_patch(
static int apply_microcode(const struct microcode_patch *patch)
{
- unsigned long flags;
uint32_t rev;
int hw_err;
unsigned int cpu = smp_processor_id();
@@ -263,15 +259,13 @@ static int apply_microcode(const struct microcode_patch *patch)
hdr = patch->mc_amd->mpb;
- spin_lock_irqsave(µcode_update_lock, flags);
+ BUG_ON(local_irq_is_enabled());
hw_err = wrmsr_safe(MSR_AMD_PATCHLOADER, (unsigned long)hdr);
/* get patch id after patching */
rdmsrl(MSR_AMD_PATCHLEVEL, rev);
- spin_unlock_irqrestore(µcode_update_lock, flags);
-
/*
* Some processors leave the ucode blob mapping as UC after the update.
* Flush the mapping to regain normal cacheability.
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index bfb48ce..94a1561 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -93,9 +93,6 @@ struct extended_sigtable {
#define exttable_size(et) ((et)->count * EXT_SIGNATURE_SIZE + EXT_HEADER_SIZE)
-/* serialize access to the physical write to MSR 0x79 */
-static DEFINE_SPINLOCK(microcode_update_lock);
-
static int collect_cpu_info(struct cpu_signature *csig)
{
unsigned int cpu_num = smp_processor_id();
@@ -295,7 +292,6 @@ static struct microcode_patch *allow_microcode_patch(
static int apply_microcode(const struct microcode_patch *patch)
{
- unsigned long flags;
uint64_t msr_content;
unsigned int val[2];
unsigned int cpu_num = raw_smp_processor_id();
@@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch *patch)
mc_intel = patch->mc_intel;
- /* serialize access to the physical write to MSR 0x79 */
- spin_lock_irqsave(µcode_update_lock, flags);
+ BUG_ON(local_irq_is_enabled());
/*
* Writeback and invalidate caches before updating microcode to avoid
@@ -327,7 +322,6 @@ static int apply_microcode(const struct microcode_patch *patch)
rdmsrl(MSR_IA32_UCODE_REV, msr_content);
val[1] = (uint32_t)(msr_content >> 32);
- spin_unlock_irqrestore(µcode_update_lock, flags);
if ( val[1] != mc_intel->hdr.rev )
{
printk(KERN_ERR "microcode: CPU%d update from revision "
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [PATCH v7 10/10] x86/microcode: always collect_cpu_info() during boot
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
From: Sergey Dyasli <sergey.dyasli@citrix.com>
Currently cpu_sig struct is not updated during boot when either:
1. ucode_scan is set to false (e.g. no "ucode=scan" in cmdline)
2. initrd does not contain a microcode blob
These will result in cpu_sig.rev being 0 which affects APIC's
check_deadline_errata() and retpoline_safe() functions.
Fix this by getting ucode revision early during boot and SMP bring up.
While at it.
Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
changes in v7:
- rebase on patch 1~9
---
xen/arch/x86/microcode.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index f4a417e..8aeb152 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -590,6 +590,10 @@ int __init early_microcode_init(void)
if ( microcode_ops )
{
+ rc = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
+ if ( rc )
+ return rc;
+
if ( ucode_mod.mod_end || ucode_blob.size )
rc = early_microcode_parse_and_update_cpu();
}
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* [Xen-devel] [PATCH v7 10/10] x86/microcode: always collect_cpu_info() during boot
@ 2019-05-27 8:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-05-27 8:31 UTC (permalink / raw)
To: xen-devel
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich,
Chao Gao, Roger Pau Monné
From: Sergey Dyasli <sergey.dyasli@citrix.com>
Currently cpu_sig struct is not updated during boot when either:
1. ucode_scan is set to false (e.g. no "ucode=scan" in cmdline)
2. initrd does not contain a microcode blob
These will result in cpu_sig.rev being 0 which affects APIC's
check_deadline_errata() and retpoline_safe() functions.
Fix this by getting ucode revision early during boot and SMP bring up.
While at it.
Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
Signed-off-by: Chao Gao <chao.gao@intel.com>
---
changes in v7:
- rebase on patch 1~9
---
xen/arch/x86/microcode.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
index f4a417e..8aeb152 100644
--- a/xen/arch/x86/microcode.c
+++ b/xen/arch/x86/microcode.c
@@ -590,6 +590,10 @@ int __init early_microcode_init(void)
if ( microcode_ops )
{
+ rc = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
+ if ( rc )
+ return rc;
+
if ( ucode_mod.mod_end || ucode_blob.size )
rc = early_microcode_parse_and_update_cpu();
}
--
1.8.3.1
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 02/10] microcode/intel: extend microcode_update_match()
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-04 14:39 ` Jan Beulich
2019-06-05 13:22 ` Roger Pau Monné
2019-06-06 8:26 ` Chao Gao
-1 siblings, 2 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-04 14:39 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> --- a/xen/arch/x86/microcode_intel.c
> +++ b/xen/arch/x86/microcode_intel.c
> @@ -134,14 +134,28 @@ static int collect_cpu_info(unsigned int cpu_num, struct cpu_signature *csig)
> return 0;
> }
>
> -static inline int microcode_update_match(
> - unsigned int cpu_num, const struct microcode_header_intel *mc_header,
> - int sig, int pf)
> +static enum microcode_match_result microcode_update_match(
> + const struct microcode_header_intel *mc_header, unsigned int sig,
> + unsigned int pf, unsigned int rev)
> {
> - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
> + const struct extended_sigtable *ext_header;
> + const struct extended_signature *ext_sig;
> + unsigned long data_size = get_datasize(mc_header);
> + unsigned int i;
> +
> + if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
> + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
As indicated before, I think you would better also provide an "equal"
indication. Iirc I've told you that I have one system where the cores
get handed over from the BIOS in an inconsistent state (only core
has ucode loaded). Hence we'd want to be able to also _store_
ucode matching that found on CPU 0, without actually want to _load_
it there.
> - return (sigmatch(sig, uci->cpu_sig.sig, pf, uci->cpu_sig.pf) &&
> - (mc_header->rev > uci->cpu_sig.rev));
> + if ( get_totalsize(mc_header) == (data_size + MC_HEADER_SIZE) )
> + return MIS_UCODE;
Okay, you're tightening the original <= to == here. But if you're
already tightening things, why don't you make sure you actually
have enough data to ...
> + ext_header = (const void *)(mc_header + 1) + data_size;
... hold an extended header, and then also to hold ...
> + ext_sig = (const void *)(ext_header + 1);
> + for ( i = 0; i < ext_header->count; i++ )
> + if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
> + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
... enough array elements?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 03/10] microcode: introduce a global cache of ucode patch
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-04 15:03 ` Jan Beulich
2019-06-10 5:33 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-04 15:03 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> +bool microcode_update_cache(struct microcode_patch *patch)
> +{
> +
> + ASSERT(spin_is_locked(µcode_mutex));
> +
> + if ( !microcode_ops->match_cpu(patch) )
> + return false;
> +
> + if ( !microcode_cache )
> + microcode_cache = patch;
> + else if ( microcode_ops->compare_patch(patch, microcode_cache) ==
> + NEW_UCODE )
> + {
> + microcode_ops->free_patch(microcode_cache);
> + microcode_cache = patch;
> + }
Hmm, okay, the way you do things here three enumeration values
may indeed be sufficient. "old" may just be a little misleading then.
(As to my respective comment on the previous patch.)
> +static struct microcode_patch *alloc_microcode_patch(
> + const struct microcode_amd *mc_amd)
> +{
> + struct microcode_patch *microcode_patch = xmalloc(struct microcode_patch);
> + struct microcode_amd *cache = xmalloc(struct microcode_amd);
> + void *mpb = xmalloc_bytes(mc_amd->mpb_size);
> + struct equiv_cpu_entry *equiv_cpu_table =
> + xmalloc_bytes(mc_amd->equiv_cpu_table_size);
> +
> + if ( !microcode_patch || !cache || !mpb || !equiv_cpu_table )
> + {
> + xfree(microcode_patch);
> + xfree(cache);
> + xfree(mpb);
> + xfree(equiv_cpu_table);
> + printk(XENLOG_ERR "microcode: Can not allocate memory\n");
I'm not convinced this needs logging.
> + return ERR_PTR(-ENOMEM);
> + }
> +
> + cache->equiv_cpu_table = equiv_cpu_table;
> + cache->mpb = mpb;
> + memcpy(cache->equiv_cpu_table, mc_amd->equiv_cpu_table,
Why not use the local variable here and ...
> + mc_amd->equiv_cpu_table_size);
> + memcpy(cache->mpb, mc_amd->mpb, mc_amd->mpb_size);
here? Less source code and presumably also slightly less binary
code. In fact I wonder if you wouldn't better memcpy() first
anyway, and only then store the values into the fields. It won't
matter much with the global lock held, but it's generally good
practice to do things in an order that won't risk to confuse
hypothetical consumers of the data.
> +static void free_patch(struct microcode_patch *microcode_patch)
> +{
> + struct microcode_amd *mc_amd = microcode_patch->mc_amd;
> +
> + xfree(mc_amd->equiv_cpu_table);
> + xfree(mc_amd->mpb);
> + xfree(mc_amd);
> + xfree(microcode_patch);
I think I said so before: Freeing of the generic wrapper struct
would probably better be placed in generic code.
> @@ -497,7 +558,20 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
> while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
> &offset)) == 0 )
> {
> - if ( microcode_fits(mc_amd, cpu) )
> + struct microcode_patch *new_patch = alloc_microcode_patch(mc_amd);
> +
> + if ( IS_ERR(new_patch) )
> + {
> + error = PTR_ERR(new_patch);
> + break;
> + }
> +
> + if ( match_cpu(new_patch) )
> + microcode_update_cache(new_patch);
> + else
> + free_patch(new_patch);
Why do you re-do what microcode_update_cache() already does?
It calls ->match_cpu() and ->free_patch() all by itself. It looks as
if it would need to gain one more ->free_patch() invocation though.
(These last two comments apply to the respective Intel code as
well then.)
> @@ -277,6 +319,7 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
> memcpy(new_mc, mc, total_size);
> xfree(uci->mc.mc_intel);
> uci->mc.mc_intel = new_mc;
> +
> return 1;
> }
>
Stray cosmetics?
> @@ -309,19 +356,19 @@ static int apply_microcode(unsigned int cpu)
> val[1] = (uint32_t)(msr_content >> 32);
>
> spin_unlock_irqrestore(µcode_update_lock, flags);
> - if ( val[1] != uci->mc.mc_intel->hdr.rev )
> + if ( val[1] != mc_intel->hdr.rev )
> {
> printk(KERN_ERR "microcode: CPU%d update from revision "
> "%#x to %#x failed. Resulting revision is %#x.\n", cpu_num,
> - uci->cpu_sig.rev, uci->mc.mc_intel->hdr.rev, val[1]);
> + uci->cpu_sig.rev, mc_intel->hdr.rev, val[1]);
> return -EIO;
> }
> printk(KERN_INFO "microcode: CPU%d updated from revision "
> "%#x to %#x, date = %04x-%02x-%02x \n",
> cpu_num, uci->cpu_sig.rev, val[1],
> - uci->mc.mc_intel->hdr.year,
> - uci->mc.mc_intel->hdr.month,
> - uci->mc.mc_intel->hdr.day);
> + mc_intel->hdr.year,
> + mc_intel->hdr.month,
> + mc_intel->hdr.day);
The three arguments now look to all fit on a single line.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 04/10] microcode: remove struct ucode_cpu_info
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-04 15:13 ` Jan Beulich
2019-06-10 7:19 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-04 15:13 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> We can remove the per-cpu cache field in struct ucode_cpu_info since
> it has been replaced by a global cache. It would leads to only one field
> remaining in ucode_cpu_info. Then, this struct is removed and the
> remaining field (cpu signature) is stored in per-cpu area.
>
> Also remove 'microcode_resume_match' from microcode_ops because the
> check is done in find_patch(). The cpu status notifier is also
> removed. It was used to free the "mc" field to avoid memory leak.
There's no find_patch() function anymore afaics. And I also think this
should be a separate patch. The above isn't enough imo to justify ...
> int microcode_resume_cpu(unsigned int cpu)
> {
> int err;
> - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
> - struct cpu_signature nsig;
> - unsigned int cpu2;
> + struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
>
> if ( !microcode_ops )
> return 0;
>
> spin_lock(µcode_mutex);
>
> - err = microcode_ops->collect_cpu_info(cpu, &uci->cpu_sig);
> - if ( err )
> - {
> - __microcode_fini_cpu(cpu);
> - spin_unlock(µcode_mutex);
> - return err;
> - }
> -
> - if ( uci->mc.mc_valid )
> - {
> - err = microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid);
> - if ( err >= 0 )
> - {
> - if ( err )
> - err = microcode_ops->apply_microcode(cpu);
> - spin_unlock(µcode_mutex);
> - return err;
> - }
> - }
> -
> - nsig = uci->cpu_sig;
> - __microcode_fini_cpu(cpu);
> - uci->cpu_sig = nsig;
> -
> - err = -EIO;
> - for_each_online_cpu ( cpu2 )
> - {
> - uci = &per_cpu(ucode_cpu_info, cpu2);
> - if ( uci->mc.mc_valid &&
> - microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid) > 0 )
> - {
> - err = microcode_ops->apply_microcode(cpu);
> - break;
> - }
> - }
... in particular the removal of this loop, the more that both the
loop and the code ahead of it also call ->apply_microcode().
> @@ -281,7 +281,6 @@ static enum microcode_match_result compare_patch(
> */
> static int get_matching_microcode(const void *mc, unsigned int cpu)
> {
> - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
Note how this was using "cpu".
> @@ -308,17 +307,7 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
>
> pr_debug("microcode: CPU%d found a matching microcode update with"
> " version %#x (current=%#x)\n",
> - cpu, mc_header->rev, uci->cpu_sig.rev);
> - new_mc = xmalloc_bytes(total_size);
> - if ( new_mc == NULL )
> - {
> - printk(KERN_ERR "microcode: error! Can not allocate memory\n");
> - return -ENOMEM;
> - }
> -
> - memcpy(new_mc, mc, total_size);
> - xfree(uci->mc.mc_intel);
> - uci->mc.mc_intel = new_mc;
> + cpu, mc_header->rev, this_cpu(cpu_sig).rev);
Why "this_cpu()" here?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 05/10] microcode: remove pointless 'cpu' parameter
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-04 15:29 ` Jan Beulich
2019-06-10 7:31 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-04 15:29 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> --- a/xen/arch/x86/microcode_amd.c
> +++ b/xen/arch/x86/microcode_amd.c
> @@ -78,8 +78,9 @@ struct mpbhdr {
> static DEFINE_SPINLOCK(microcode_update_lock);
>
> /* See comment in start_update() for cases when this routine fails */
> -static int collect_cpu_info(unsigned int cpu, struct cpu_signature *csig)
> +static int collect_cpu_info(struct cpu_signature *csig)
> {
> + unsigned int cpu = smp_processor_id();
> struct cpuinfo_x86 *c = &cpu_data[cpu];
I think it would be more clear if you used current_cpu_data here.
The only other use of "cpu" is in a pr_debug(), which by default
expands to nothing anyway, and hence is cheap to change to
use smp_processor_id() instead.
> @@ -435,14 +429,14 @@ static const unsigned int final_levels[] = {
> 0x010000af
> };
>
> -static bool_t check_final_patch_levels(unsigned int cpu)
> +static bool check_final_patch_levels(void)
> {
> /*
> * Check the current patch levels on the cpu. If they are equal to
> * any of the 'final_levels', then we should not update the microcode
> * patch on the cpu as system will hang otherwise.
> */
> - const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
> + const struct cpu_signature *sig = &this_cpu(cpu_sig);
> unsigned int i;
I don't see any dependency of this function upon running on
the subject CPU.
> @@ -279,12 +278,13 @@ static enum microcode_match_result compare_patch(
> * return 1 - found update
> * return < 0 - error
> */
> -static int get_matching_microcode(const void *mc, unsigned int cpu)
> +static int get_matching_microcode(const void *mc)
> {
> const struct microcode_header_intel *mc_header = mc;
> unsigned long total_size = get_totalsize(mc_header);
> void *new_mc = xmalloc_bytes(total_size);
> struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
> + unsigned int __maybe_unused cpu = smp_processor_id();
The __maybe_unused is for the sole use in pr_debug()? Please
instead use smp_processor_id() there, if so.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 01/10] misc/xen-ucode: Upload a microcode blob to the hypervisor
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-04 16:14 ` Andrew Cooper
2019-06-04 16:23 ` Jan Beulich
2019-06-06 2:29 ` Chao Gao
-1 siblings, 2 replies; 64+ messages in thread
From: Andrew Cooper @ 2019-06-04 16:14 UTC (permalink / raw)
To: Chao Gao, xen-devel
Cc: Sergey Dyasli, Ian Jackson, Ashok Raj, Wei Liu, Konrad Rzeszutek Wilk
On 27/05/2019 09:31, Chao Gao wrote:
> This patch provides a tool for late microcode update.
>
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> ---
> Changes in v7:
> - introduce xc_microcode_update() rather than xc_platform_op()
> - avoid creating bounce buffer twice
> - rename xenmicrocode to xen-ucode, following naming tradition
> of other tools there.
>
> ---
> tools/libxc/include/xenctrl.h | 1 +
> tools/libxc/xc_misc.c | 23 +++++++++++++
> tools/misc/Makefile | 4 +++
> tools/misc/xen-ucode.c | 78 +++++++++++++++++++++++++++++++++++++++++++
> 4 files changed, 106 insertions(+)
> create mode 100644 tools/misc/xen-ucode.c
>
> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
> index 538007a..6d80ae5 100644
> --- a/tools/libxc/include/xenctrl.h
> +++ b/tools/libxc/include/xenctrl.h
> @@ -1244,6 +1244,7 @@ typedef uint32_t xc_node_to_node_dist_t;
> int xc_physinfo(xc_interface *xch, xc_physinfo_t *info);
> int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
> xc_cputopo_t *cputopo);
> +int xc_microcode_update(xc_interface *xch, const void *buf, size_t len);
> int xc_numainfo(xc_interface *xch, unsigned *max_nodes,
> xc_meminfo_t *meminfo, uint32_t *distance);
> int xc_pcitopoinfo(xc_interface *xch, unsigned num_devs,
> diff --git a/tools/libxc/xc_misc.c b/tools/libxc/xc_misc.c
> index 5e6714a..85538e0 100644
> --- a/tools/libxc/xc_misc.c
> +++ b/tools/libxc/xc_misc.c
> @@ -226,6 +226,29 @@ int xc_physinfo(xc_interface *xch,
> return 0;
> }
>
> +int xc_microcode_update(xc_interface *xch, const void *buf, size_t len)
> +{
> + int ret;
> + DECLARE_PLATFORM_OP;
> + DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update, uc);
> +
> + uc = xc_hypercall_buffer_alloc(xch, uc, len);
> + if (uc == NULL)
Xen style. Extra space please.
> + return -1;
> +
> + memcpy(uc, buf, len);
> +
> + platform_op.cmd = XENPF_microcode_update;
> + platform_op.u.microcode.length = len;
> + set_xen_guest_handle(platform_op.u.microcode.data, uc);
> +
> + ret = do_platform_op(xch, &platform_op);
> +
> + xc_hypercall_buffer_free(xch, uc);
> +
> + return ret;
> +}
> +
> int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
> xc_cputopo_t *cputopo)
> {
> diff --git a/tools/misc/Makefile b/tools/misc/Makefile
> index d4320dc..63947bf 100644
> --- a/tools/misc/Makefile
> +++ b/tools/misc/Makefile
> @@ -22,6 +22,7 @@ INSTALL_SBIN-$(CONFIG_X86) += xen-hvmcrash
> INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx
> INSTALL_SBIN-$(CONFIG_X86) += xen-lowmemd
> INSTALL_SBIN-$(CONFIG_X86) += xen-mfndump
> +INSTALL_SBIN-$(CONFIG_X86) += xen-ucode
> INSTALL_SBIN += xencov
> INSTALL_SBIN += xenlockprof
> INSTALL_SBIN += xenperf
> @@ -113,4 +114,7 @@ xen-lowmemd: xen-lowmemd.o
> xencov: xencov.o
> $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
>
> +xen-ucode: xen-ucode.o
> + $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
> +
> -include $(DEPS_INCLUDE)
> diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
> new file mode 100644
> index 0000000..da668ca
> --- /dev/null
> +++ b/tools/misc/xen-ucode.c
> @@ -0,0 +1,78 @@
> +#define _GNU_SOURCE
> +
> +#include <stdio.h>
> +#include <stdlib.h>
> +#include <sys/mman.h>
> +#include <errno.h>
> +#include <string.h>
> +#include <inttypes.h>
> +#include <unistd.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +#include <xenctrl.h>
> +
> +void show_help(void)
> +{
> + fprintf(stderr,
> + "xenmicrocode: Xen microcode updating tool\n"
> + "Usage: xenmicrocode <microcode blob>\n");
s/xenmicrocode/xen-ucode/
Both can be fixed on commit
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 01/10] misc/xen-ucode: Upload a microcode blob to the hypervisor
2019-06-04 16:14 ` Andrew Cooper
@ 2019-06-04 16:23 ` Jan Beulich
2019-06-06 2:29 ` Chao Gao
1 sibling, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-04 16:23 UTC (permalink / raw)
To: Andrew Cooper, Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Konrad Rzeszutek Wilk,
Ian Jackson, xen-devel
>>> On 04.06.19 at 18:14, <andrew.cooper3@citrix.com> wrote:
> On 27/05/2019 09:31, Chao Gao wrote:
>> --- /dev/null
>> +++ b/tools/misc/xen-ucode.c
>> @@ -0,0 +1,78 @@
>> +#define _GNU_SOURCE
>> +
>> +#include <stdio.h>
>> +#include <stdlib.h>
>> +#include <sys/mman.h>
>> +#include <errno.h>
>> +#include <string.h>
>> +#include <inttypes.h>
>> +#include <unistd.h>
>> +#include <sys/types.h>
>> +#include <sys/stat.h>
>> +#include <fcntl.h>
>> +#include <xenctrl.h>
>> +
>> +void show_help(void)
>> +{
>> + fprintf(stderr,
>> + "xenmicrocode: Xen microcode updating tool\n"
>> + "Usage: xenmicrocode <microcode blob>\n");
>
> s/xenmicrocode/xen-ucode/
Isn't it common practice to use argv[0] for such?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode()
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-05 12:37 ` Jan Beulich
2019-06-11 3:32 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-05 12:37 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> During late microcode update, apply_microcode() is invoked in
> cpu_request_microcode(). To make late microcode update more reliable,
> we want to put the apply_microcode() into stop_machine context. So
> we split out it from cpu_request_microcode(). As a consequence,
> apply_microcode() should be invoked explicitly in the common code.
>
> Previously, apply_microcode() gets the microcode patch to be applied from
> the microcode cache. Now, the patch is passed as a function argument and
> a patch is cached for cpu-hotplug and cpu resuming, only after it has
> been loaded to a cpu without any error. As a consequence, the
> 'match_cpu' check in microcode_update_cache is removed, which otherwise
> would fail.
The "only after it has been loaded to a cpu without any error" is a
problem, precisely for the case where ucode on the different cores
is not in sync initially. I would actually like to put up this question:
When a core has no ucode loaded at all yet and only strictly older
(than loaded on some other cores) ucode is found to be available,
whether then it wouldn't still be better to apply that ucode to
_at least_ the cores that have none loaded yet.
To get the system into "sane" state it may even be necessary to
downgrade ucode on the cores which did have it loaded already,
in such a situation.
> On AMD side, svm_host_osvw_init() is supposed to be called after
> microcode update. As apply_micrcode() won't be called by
> cpu_request_microcode() now, svm_host_osvw_init() is moved to the
> end of apply_microcode().
I guess this really ought to become a vendor hook as well, but I
wouldn't insist on you doing so here.
> --- a/xen/arch/x86/acpi/power.c
> +++ b/xen/arch/x86/acpi/power.c
> @@ -253,7 +253,7 @@ static int enter_state(u32 state)
>
> console_end_sync();
>
> - microcode_resume_cpu();
> + early_microcode_update_cpu();
The use here, the (changed) use in start_secondary(), and the dropping
of its __init suggest to make an attempt to find a better name for the
function. Maybe microcode_update_one()?
> +/*
> + * Load a microcode update to current CPU.
> + *
> + * If no patch is provided, the cached patch will be loaded. Microcode update
> + * during APs bringup and CPU resuming falls into this case.
> + */
> +static int microcode_update_cpu(struct microcode_patch *patch)
const?
> {
> - int err;
> - struct cpu_signature *sig = &this_cpu(cpu_sig);
> + int ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
>
> - if ( !microcode_ops )
> - return 0;
> + if ( unlikely(ret) )
> + return ret;
>
> spin_lock(µcode_mutex);
>
> - err = microcode_ops->collect_cpu_info(sig);
> - if ( likely(!err) )
> - err = microcode_ops->apply_microcode();
> - spin_unlock(µcode_mutex);
> + if ( patch )
> + {
> + /*
> + * If a patch is specified, it should has newer revision than
> + * that of the patch cached.
> + */
> + if ( microcode_cache &&
> + microcode_ops->compare_patch(patch, microcode_cache) != NEW_UCODE )
> + {
> + spin_unlock(µcode_mutex);
> + return -EINVAL;
> + }
>
> - return err;
> -}
> + ret = microcode_ops->apply_microcode(patch);
There's no printk() here but ...
> + }
> + else if ( microcode_cache )
> + {
> + ret = microcode_ops->apply_microcode(microcode_cache);
> + if ( ret == -EIO )
> + printk("Update failed. Reboot needed\n");
... you emit a log message here. Why the difference? And wouldn't
it be better to have just a single call to ->apply anyway, by simply
assigning "microcode_cache" to "patch" and moving the call a little
further down?
> + }
> + else
> + /* No patch to update */
> + ret = -EINVAL;
-ENOENT?
> @@ -247,46 +270,32 @@ bool microcode_update_cache(struct microcode_patch *patch)
> return true;
> }
>
> -static int microcode_update_cpu(const void *buf, size_t size)
> +static long do_microcode_update(void *patch)
> {
> - int err;
> - unsigned int cpu = smp_processor_id();
> - struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
> -
> - spin_lock(µcode_mutex);
> + int error, cpu;
While "int" is fine for error codes, it almost certainly wants to be
"unsigned int" for "cpu". The more that it had been that was before.
I also don't see why you need to switch from "err" to "error" - oh,
...
> - err = microcode_ops->collect_cpu_info(sig);
> - if ( likely(!err) )
> - err = microcode_ops->cpu_request_microcode(buf, size);
> - spin_unlock(µcode_mutex);
> -
> - return err;
> -}
> -
> -static long do_microcode_update(void *_info)
> -{
> - struct microcode_info *info = _info;
> - int error;
> + error = microcode_update_cpu(patch);
... there was a pre-existing variable of that name here.
> + if ( error )
> + {
> + microcode_ops->free_patch(microcode_cache);
Does this also set "microcode_cache" to NULL? I didn't think so.
It's anyway not really clear why _all_ forms of errors should lead
to clearing of the cache. However - looking at the code further
down, don't you rather mean to free "patch" here anyway?
> + return error;
> + }
>
> - BUG_ON(info->cpu != smp_processor_id());
>
> - error = microcode_update_cpu(info->buffer, info->buffer_size);
> - if ( error )
> - info->error = error;
> + cpu = cpumask_next(smp_processor_id(), &cpu_online_map);
> + if ( cpu < nr_cpu_ids )
> + return continue_hypercall_on_cpu(cpu, do_microcode_update, patch);
>
> - info->cpu = cpumask_next(info->cpu, &cpu_online_map);
> - if ( info->cpu < nr_cpu_ids )
> - return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
> + microcode_update_cache(patch);
Independent of my remarks at the top I would think that updating of
the cache should happen after the first successful loading on a CPU,
not after all CPUs have been updated successfully. There would then
also not be any need to pass "patch" on to continue_hypercall_on_cpu()
a few lines up from here (albeit from a general logic perspective it may
indeed be easier to keep it that way).
> @@ -294,32 +303,49 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
> if ( microcode_ops == NULL )
> return -EINVAL;
>
> - info = xmalloc_bytes(sizeof(*info) + len);
> - if ( info == NULL )
> - return -ENOMEM;
> -
> - ret = copy_from_guest(info->buffer, buf, len);
> - if ( ret != 0 )
> + buffer = xmalloc_bytes(len);
> + if ( !buffer )
> {
> - xfree(info);
> - return ret;
> + ret = -ENOMEM;
> + goto free;
> }
>
> - info->buffer_size = len;
> - info->error = 0;
> - info->cpu = cpumask_first(&cpu_online_map);
> + if ( copy_from_guest(buffer, buf, len) )
> + {
> + ret = -EFAULT;
> + goto free;
> + }
>
> if ( microcode_ops->start_update )
> {
> ret = microcode_ops->start_update();
> if ( ret != 0 )
> - {
> - xfree(info);
> - return ret;
> - }
> + goto free;
> }
>
> - return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
> + patch = microcode_parse_blob(buffer, len);
> + if ( IS_ERR(patch) )
> + {
> + printk(XENLOG_ERR "Parsing microcode blob error %ld\n", PTR_ERR(patch));
> + ret = PTR_ERR(patch);
So I assume we would get here when the system already has the
newest (or even newer) ucode loaded. That's not an error, and
imo no log message suggesting so should be issued. Perhaps the
parsing code could return NULL to indicate so? Although, judging
by the code further down you already expect NULL potentially
coming back, but I can't seem to be able to figure the condition(s).
Also please switch the two lines around and use "ret" in the printk()
invocation.
> + goto free;
> + }
> +
> + if ( !microcode_ops->match_cpu(patch) )
> + {
> + printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
I assume the "matching" here is meant to cover the CPU signature?
The wording is ambiguous this way, because it could also mean there
was no ucode found matching that which is already loaded (which, as
per above, may end up being relevant).
Furthermore - why this check, when microcode_parse_blob() already
looks for something that's newer than what is currently loaded (and
matches the CPU signature)?
> @@ -362,13 +397,41 @@ int __init early_microcode_update_cpu(bool start_update)
> }
> if ( data )
> {
> - if ( start_update && microcode_ops->start_update )
> + struct microcode_patch *patch;
> +
> + if ( microcode_ops->start_update )
> rc = microcode_ops->start_update();
>
> if ( rc )
> return rc;
>
> - return microcode_update_cpu(data, len);
> + patch = microcode_parse_blob(data, len);
> + if ( IS_ERR(patch) )
> + {
> + printk(XENLOG_ERR "Parsing microcode blob error %ld\n",
> + PTR_ERR(patch));
> + return PTR_ERR(patch);
> + }
> +
> + if ( !microcode_ops->match_cpu(patch) )
> + {
> + printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
> + if ( patch )
> + microcode_ops->free_patch(patch);
> + return -EINVAL;
> + }
Same remarks here then.
> @@ -292,6 +291,10 @@ static int apply_microcode(void)
>
> sig->rev = rev;
>
> +#ifdef CONFIG_HVM
> + svm_host_osvw_init();
> +#endif
> +
> return 0;
> }
While this now sits on the success path only, ...
> @@ -592,17 +596,10 @@ static int cpu_request_microcode(const void *buf, size_t bufsize)
> }
>
> out:
> -#if CONFIG_HVM
> - svm_host_osvw_init();
> -#endif
> + if ( error && !patch )
> + patch = ERR_PTR(error);
>
> - /*
> - * In some cases we may return an error even if processor's microcode has
> - * been updated. For example, the first patch in a container file is loaded
> - * successfully but subsequent container file processing encounters a
> - * failure.
> - */
> - return error;
> + return patch;
> }
... previously it has also been invoked in the error case. See the
comment in start_update().
> --- a/xen/arch/x86/microcode_intel.c
> +++ b/xen/arch/x86/microcode_intel.c
> @@ -273,46 +273,27 @@ static enum microcode_match_result compare_patch(
> old_header->pf, old_header->rev);
> }
>
> -/*
> - * return 0 - no update found
> - * return 1 - found update
> - * return < 0 - error
> - */
> -static int get_matching_microcode(const void *mc)
> +static struct microcode_patch *allow_microcode_patch(
Did you perhaps mean this to be alloc_microcode_patch()?
> @@ -388,26 +368,39 @@ static long get_next_ucode_from_buffer(void **mc, const u8 *buf,
> return offset + total_size;
> }
>
> -static int cpu_request_microcode(const void *buf, size_t size)
> +static struct microcode_patch *cpu_request_microcode(const void *buf,
> + size_t size)
> {
> long offset = 0;
> int error = 0;
> void *mc;
> + struct microcode_patch *patch = NULL;
>
> while ( (offset = get_next_ucode_from_buffer(&mc, buf, size, offset)) > 0 )
> {
> + struct microcode_patch *new_patch;
> +
> error = microcode_sanity_check(mc);
> if ( error )
> break;
> - error = get_matching_microcode(mc);
> - if ( error < 0 )
> +
> + new_patch = allow_microcode_patch(mc);
> + if ( IS_ERR(new_patch) )
> + {
> + error = PTR_ERR(new_patch);
> break;
> - /*
> - * It's possible the data file has multiple matching ucode,
> - * lets keep searching till the latest version
> - */
> - if ( error == 1 )
> - error = 0;
> + }
> +
> + /* Compare patches and store the one with higher revision */
> + if ( !patch && match_cpu(new_patch) )
> + patch = new_patch;
> + else if ( patch && (compare_patch(new_patch, patch) == NEW_UCODE) )
At least the appearance of this is misleading, but unless I'm missing
something it's actually wrong: You seem to imply that from
match_cpu(patch1) returning true and compare_patch(patch2, patch1)
returning true it follows that also match_cpu(patch2) would return
true. But I don't think that's the case, because of how the "pf" field
works.
Even in case I've overlooked an implicit match_cpu(new_patch) I'd
like to ask for this to be clarified, either by changing the code
structure, or by attaching a suitable comment.
> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -363,10 +363,7 @@ void start_secondary(void *unused)
>
> initialize_cpu_data(cpu);
>
> - if ( system_state <= SYS_STATE_smp_boot )
> - early_microcode_update_cpu(false);
> - else
> - microcode_resume_cpu();
> + early_microcode_update_cpu();
I'm struggling to understand how you get away without the "false"
argument that was passed here before. You look to now be calling
->start_update() unconditionally (so long as the hook is not NULL),
which I don't think is correct. This should be called only once by
the CPU _leading_ an update (the BSP during boot, and the CPU
the hypercall gets invoked on (or the first CPU an update gets
issued one) for a late update. Am I missing something?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 07/10] microcode/intel: Writeback and invalidate caches before updating microcode
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-05 13:20 ` Jan Beulich
-1 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-05 13:20 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> Updating microcode is less error prone when caches have been flushed and
> depending on what exactly the microcode is updating.
Up to the "and" I understand this sentence, but the rest doesn't really
seem to fit. Taking out the good part it seems to me you're saying
"Updating microcode is less error prone depending on what exactly the
microcode is updating," which - to me at least - doesn't make a hole lot
of sense. Should it perhaps be
"Updating microcode, depending on what exactly the microcode is
updating, may be less error prone when caches have been flushed."
(The same could perhaps also be achieved by replacing the "and" by
a comma.)
> For example, some
> of the issues around certain Broadwell parts can be addressed by doing a
> full cache flush.
>
> With parallel microcode update, the cost of this patch is hardly
> noticable. Although only BDX with an old microcode needs this fix, we
> would like to avoid future issues in case they come by later due to
> other reasons.
I doubt the "hardly noticable" part, and I'm sure you're also aware of
the patch (going on top of your series) to make selecting between
serial or parallel application a runtime option. But I'm not going to
stand in the way if everyone else thinks this is the way to go; it's
just that from previous discussions I didn't get such an impression.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 02/10] microcode/intel: extend microcode_update_match()
2019-06-04 14:39 ` Jan Beulich
@ 2019-06-05 13:22 ` Roger Pau Monné
2019-06-05 14:16 ` Jan Beulich
2019-06-06 8:26 ` Chao Gao
1 sibling, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2019-06-05 13:22 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel, Chao Gao
On Tue, Jun 04, 2019 at 08:39:15AM -0600, Jan Beulich wrote:
> >>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> > --- a/xen/arch/x86/microcode_intel.c
> > +++ b/xen/arch/x86/microcode_intel.c
> > @@ -134,14 +134,28 @@ static int collect_cpu_info(unsigned int cpu_num, struct cpu_signature *csig)
> > return 0;
> > }
> >
> > -static inline int microcode_update_match(
> > - unsigned int cpu_num, const struct microcode_header_intel *mc_header,
> > - int sig, int pf)
> > +static enum microcode_match_result microcode_update_match(
> > + const struct microcode_header_intel *mc_header, unsigned int sig,
> > + unsigned int pf, unsigned int rev)
> > {
> > - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
> > + const struct extended_sigtable *ext_header;
> > + const struct extended_signature *ext_sig;
> > + unsigned long data_size = get_datasize(mc_header);
> > + unsigned int i;
> > +
> > + if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
> > + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
>
> As indicated before, I think you would better also provide an "equal"
> indication. Iirc I've told you that I have one system where the cores
> get handed over from the BIOS in an inconsistent state (only core
> has ucode loaded). Hence we'd want to be able to also _store_
> ucode matching that found on CPU 0, without actually want to _load_
> it there.
Hm, without me being an expert on microcode, isn't such a system utterly
broken?
I'm not against making Xen capable of booting in this scenario where
firmware leaves the CPUs with different microcode versions, but this
is something that should be reported to the vendor in order to get it
fixed IMO?
What happens when you don't load any microcode at all, is the system
capable of operating normally with such mixed microcode?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-05 14:09 ` Jan Beulich
2019-06-11 12:36 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-05 14:09 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Kevin Tian, Ashok Raj, WeiLiu, Andrew Cooper,
Jun Nakajima, xen-devel, tglx, Borislav Petkov, Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> This patch ports microcode improvement patches from linux kernel.
>
> Before you read any further: the early loading method is still the
> preferred one and you should always do that. The following patch is
> improving the late loading mechanism for long running jobs and cloud use
> cases.
>
> Gather all cores and serialize the microcode update on them by doing it
> one-by-one to make the late update process as reliable as possible and
> avoid potential issues caused by the microcode update.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Tested-by: Chao Gao <chao.gao@intel.com>
> [linux commit: a5321aec6412b20b5ad15db2d6b916c05349dbff]
> [linux commit: bb8c13d61a629276a162c1d2b1a20a815cbcfbb7]
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: Jun Nakajima <jun.nakajima@intel.com>
> Cc: Ashok Raj <ashok.raj@intel.com>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> ---
> Changes in v7:
> - Check whether 'timeout' is 0 rather than "<=0" since it is unsigned int.
> - reword the comment above microcode_update_cpu() to clearly state that
> one thread per core should do the update.
>
> Changes in v6:
> - Use one timeout period for rendezvous stage and another for update stage.
> - scale time to wait by the number of remaining cpus to respond.
> It helps to find something wrong earlier and thus we can reboot the
> system earlier.
> ---
> xen/arch/x86/microcode.c | 171 ++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 155 insertions(+), 16 deletions(-)
>
> diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
> index 23cf550..f4a417e 100644
> --- a/xen/arch/x86/microcode.c
> +++ b/xen/arch/x86/microcode.c
> @@ -22,6 +22,7 @@
> */
>
> #include <xen/cpu.h>
> +#include <xen/cpumask.h>
It seems vanishingly unlikely that you would need this explicit #include
here, but it certainly isn't wrong.
> @@ -270,31 +296,90 @@ bool microcode_update_cache(struct microcode_patch *patch)
> return true;
> }
>
> -static long do_microcode_update(void *patch)
> +/* Wait for CPUs to rendezvous with a timeout (us) */
> +static int wait_for_cpus(atomic_t *cnt, unsigned int expect,
> + unsigned int timeout)
> {
> - int error, cpu;
> -
> - error = microcode_update_cpu(patch);
> - if ( error )
> + while ( atomic_read(cnt) < expect )
> {
> - microcode_ops->free_patch(microcode_cache);
> - return error;
> + if ( !timeout )
> + {
> + printk("CPU%d: Timeout when waiting for CPUs calling in\n",
> + smp_processor_id());
> + return -EBUSY;
> + }
> + udelay(1);
> + timeout--;
> }
There's no comment here and nothing in the description: I don't
recall clarification as to whether RDTSC is fine to be issued by a
thread when ucode is being updated by another thread on the
same core.
> +static int do_microcode_update(void *patch)
> +{
> + unsigned int cpu = smp_processor_id();
> + unsigned int cpu_nr = num_online_cpus();
> + unsigned int finished;
> + int ret;
> + static bool error;
>
> - microcode_update_cache(patch);
> + atomic_inc(&cpu_in);
> + ret = wait_for_cpus(&cpu_in, cpu_nr, MICROCODE_CALLIN_TIMEOUT_US);
> + if ( ret )
> + return ret;
>
> - return error;
> + ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
> + /*
> + * Load microcode update on only one logical processor per core.
> + * Here, among logical processors of a core, the one with the
> + * lowest thread id is chosen to perform the loading.
> + */
> + if ( !ret && (cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu))) )
At the very least it's not obvious whether this hyper-threading-centric
view ("logical processor") also applies to AMD's compute unit model
(which reuses cpu_sibling_mask). It does, as the respective MSRs are
per-compute-unit rather than per-core, but I'd appreciate if the
wording could be adjusted to explicitly name both cases (multiple
threads per core and multiple cores per CU).
> + {
> + ret = microcode_ops->apply_microcode(patch);
> + if ( !ret )
> + atomic_inc(&cpu_updated);
> + }
> + /*
> + * Increase the wait timeout to a safe value here since we're serializing
I'm struggling with the "increase": I don't see anything being increased
here. You simply use a larger timeout than above.
> + * the microcode update and that could take a while on a large number of
> + * CPUs. And that is fine as the *actual* timeout will be determined by
> + * the last CPU finished updating and thus cut short
> + */
> + atomic_inc(&cpu_out);
> + finished = atomic_read(&cpu_out);
> + while ( !error && finished != cpu_nr )
> + {
> + /*
> + * During each timeout interval, at least a CPU is expected to
> + * finish its update. Otherwise, something goes wrong.
> + */
> + if ( wait_for_cpus(&cpu_out, finished + 1,
> + MICROCODE_UPDATE_TIMEOUT_US) && !error )
> + {
> + error = true;
> + panic("Timeout when finishing updating microcode (finished %d/%d)",
> + finished, cpu_nr);
Why the setting of "error" when you panic anyway?
And please use format specifiers matching the types of the
further arguments (i.e. twice %u here, but please check other
code as well).
Furthermore (and I'm sure I've given this comment before) if
you really hit the limit, how many panic() invocations are there
going to be? You run this function on all CPUs after all.
On the whole, taking a 256-thread system as example, you
allow the whole process to take over 4 min without calling
panic(). Leaving aside guests, I don't think Xen itself would
survive this in all cases. We've found the need to process
softirqs with far smaller delays, in particular from key handlers
producing lots of output. At the very least there should be a
bold warning logged if the system had been in stop-machine
state for, say, longer than 100ms (value subject to discussion).
> + }
> +
> + finished = atomic_read(&cpu_out);
> + }
> +
> + /*
> + * Refresh CPU signature (revision) on threads which didn't call
> + * apply_microcode().
> + */
> + if ( cpu != cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
> + ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
Another option would be for the CPU doing the update to simply
propagate the new value to all its siblings' cpu_sig values.
> @@ -337,12 +429,59 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
> if ( patch )
> microcode_ops->free_patch(patch);
> ret = -EINVAL;
> - goto free;
> + goto put;
> }
>
> - ret = continue_hypercall_on_cpu(cpumask_first(&cpu_online_map),
> - do_microcode_update, patch);
> + atomic_set(&cpu_in, 0);
> + atomic_set(&cpu_out, 0);
> + atomic_set(&cpu_updated, 0);
> +
> + /* Calculate the number of online CPU core */
> + nr_cores = 0;
> + for_each_online_cpu(cpu)
> + if ( cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
> + nr_cores++;
> +
> + printk(XENLOG_INFO "%d cores are to update their microcode\n", nr_cores);
> +
> + /*
> + * We intend to disable interrupt for long time, which may lead to
> + * watchdog timeout.
> + */
> + watchdog_disable();
> + /*
> + * Late loading dance. Why the heavy-handed stop_machine effort?
> + *
> + * - HT siblings must be idle and not execute other code while the other
> + * sibling is loading microcode in order to avoid any negative
> + * interactions cause by the loading.
> + *
> + * - In addition, microcode update on the cores must be serialized until
> + * this requirement can be relaxed in the future. Right now, this is
> + * conservative and good.
> + */
> + ret = stop_machine_run(do_microcode_update, patch, NR_CPUS);
> + watchdog_enable();
> +
> + if ( atomic_read(&cpu_updated) == nr_cores )
> + {
> + spin_lock(µcode_mutex);
> + microcode_update_cache(patch);
> + spin_unlock(µcode_mutex);
> + }
> + else if ( atomic_read(&cpu_updated) == 0 )
> + microcode_ops->free_patch(patch);
> + else
> + {
> + printk("Updating microcode succeeded on part of CPUs and failed on\n"
> + "others due to an unknown reason. A system with different\n"
> + "microcode revisions is considered unstable. Please reboot and\n"
> + "do not load the microcode that triggers this warning\n");
> + microcode_ops->free_patch(patch);
> + }
As said on an earlier patch, I think the cache can be updated if at
least one CPU loaded the blob successfully. Additionally I'd like to
ask that you log the number of successfully updated cores. And
finally perhaps "differing" instead of "different" and omit "due to
an unknown reason"?
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 02/10] microcode/intel: extend microcode_update_match()
2019-06-05 13:22 ` Roger Pau Monné
@ 2019-06-05 14:16 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-05 14:16 UTC (permalink / raw)
To: Roger Pau Monne
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel, Chao Gao
>>> On 05.06.19 at 15:22, <roger.pau@citrix.com> wrote:
> On Tue, Jun 04, 2019 at 08:39:15AM -0600, Jan Beulich wrote:
>> >>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> > --- a/xen/arch/x86/microcode_intel.c
>> > +++ b/xen/arch/x86/microcode_intel.c
>> > @@ -134,14 +134,28 @@ static int collect_cpu_info(unsigned int cpu_num,
> struct cpu_signature *csig)
>> > return 0;
>> > }
>> >
>> > -static inline int microcode_update_match(
>> > - unsigned int cpu_num, const struct microcode_header_intel *mc_header,
>> > - int sig, int pf)
>> > +static enum microcode_match_result microcode_update_match(
>> > + const struct microcode_header_intel *mc_header, unsigned int sig,
>> > + unsigned int pf, unsigned int rev)
>> > {
>> > - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
>> > + const struct extended_sigtable *ext_header;
>> > + const struct extended_signature *ext_sig;
>> > + unsigned long data_size = get_datasize(mc_header);
>> > + unsigned int i;
>> > +
>> > + if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
>> > + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
>>
>> As indicated before, I think you would better also provide an "equal"
>> indication. Iirc I've told you that I have one system where the cores
>> get handed over from the BIOS in an inconsistent state (only core
>> has ucode loaded). Hence we'd want to be able to also _store_
>> ucode matching that found on CPU 0, without actually want to _load_
>> it there.
>
> Hm, without me being an expert on microcode, isn't such a system utterly
> broken?
It's working fine (from all I can tell). It really depends on what exactly
the ucode update changes.
> I'm not against making Xen capable of booting in this scenario where
> firmware leaves the CPUs with different microcode versions, but this
> is something that should be reported to the vendor in order to get it
> fixed IMO?
I did report it, years ago.
> What happens when you don't load any microcode at all, is the system
> capable of operating normally with such mixed microcode?
Yes. Obviously there are anomalies (like fixed errata of particular
insns showing up or not depending on what core a process executes),
but the system has been stable this way for many years. (I've always
been doing early loading of ucode when running it with Xen, but the
distro that's on it never managed to arrange for early ucode loading
when booting a bare metal kernel).
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
(?)
@ 2019-06-05 14:42 ` Roger Pau Monné
-1 siblings, 0 replies; 64+ messages in thread
From: Roger Pau Monné @ 2019-06-05 14:42 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Kevin Tian, Ashok Raj, Wei Liu, Jun Nakajima,
Andrew Cooper, Jan Beulich, xen-devel, Thomas Gleixner,
Borislav Petkov
On Mon, May 27, 2019 at 04:31:29PM +0800, Chao Gao wrote:
> This patch ports microcode improvement patches from linux kernel.
>
> Before you read any further: the early loading method is still the
> preferred one and you should always do that. The following patch is
> improving the late loading mechanism for long running jobs and cloud use
> cases.
>
> Gather all cores and serialize the microcode update on them by doing it
> one-by-one to make the late update process as reliable as possible and
> avoid potential issues caused by the microcode update.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> Tested-by: Chao Gao <chao.gao@intel.com>
> [linux commit: a5321aec6412b20b5ad15db2d6b916c05349dbff]
> [linux commit: bb8c13d61a629276a162c1d2b1a20a815cbcfbb7]
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: Jun Nakajima <jun.nakajima@intel.com>
> Cc: Ashok Raj <ashok.raj@intel.com>
> Cc: Borislav Petkov <bp@suse.de>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> ---
> Changes in v7:
> - Check whether 'timeout' is 0 rather than "<=0" since it is unsigned int.
> - reword the comment above microcode_update_cpu() to clearly state that
> one thread per core should do the update.
>
> Changes in v6:
> - Use one timeout period for rendezvous stage and another for update stage.
> - scale time to wait by the number of remaining cpus to respond.
> It helps to find something wrong earlier and thus we can reboot the
> system earlier.
> ---
> xen/arch/x86/microcode.c | 171 ++++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 155 insertions(+), 16 deletions(-)
>
> diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
> index 23cf550..f4a417e 100644
> --- a/xen/arch/x86/microcode.c
> +++ b/xen/arch/x86/microcode.c
> @@ -22,6 +22,7 @@
> */
>
> #include <xen/cpu.h>
> +#include <xen/cpumask.h>
> #include <xen/lib.h>
> #include <xen/kernel.h>
> #include <xen/init.h>
> @@ -30,15 +31,34 @@
> #include <xen/smp.h>
> #include <xen/softirq.h>
> #include <xen/spinlock.h>
> +#include <xen/stop_machine.h>
> #include <xen/tasklet.h>
> #include <xen/guest_access.h>
> #include <xen/earlycpio.h>
> +#include <xen/watchdog.h>
>
> +#include <asm/delay.h>
> #include <asm/msr.h>
> #include <asm/processor.h>
> #include <asm/setup.h>
> #include <asm/microcode.h>
>
> +/*
> + * Before performing a late microcode update on any thread, we
> + * rendezvous all cpus in stop_machine context. The timeout for
> + * waiting for cpu rendezvous is 30ms. It is the timeout used by
> + * live patching
> + */
> +#define MICROCODE_CALLIN_TIMEOUT_US 30000
> +
> +/*
> + * Timeout for each thread to complete update is set to 1s. It is a
> + * conservative choice considering all possible interference (for
> + * instance, sometimes wbinvd takes relative long time). And a perfect
> + * timeout doesn't help a lot except an early shutdown.
I would remove the "And a perfect..." sentence. I don't think it makes
much sense to speak about "perfect timeouts".
> + */
> +#define MICROCODE_UPDATE_TIMEOUT_US 1000000
> +
> static module_t __initdata ucode_mod;
> static signed int __initdata ucode_mod_idx;
> static bool_t __initdata ucode_mod_forced;
> @@ -190,6 +210,12 @@ static DEFINE_SPINLOCK(microcode_mutex);
> DEFINE_PER_CPU(struct cpu_signature, cpu_sig);
>
> /*
> + * Count the CPUs that have entered, exited the rendezvous and succeeded in
> + * microcode update during late microcode update respectively.
> + */
> +static atomic_t cpu_in, cpu_out, cpu_updated;
> +
> +/*
> * Return the patch with the highest revision id among all matching
> * patches in the blob. Return NULL if no suitable patch.
> */
> @@ -270,31 +296,90 @@ bool microcode_update_cache(struct microcode_patch *patch)
> return true;
> }
>
> -static long do_microcode_update(void *patch)
> +/* Wait for CPUs to rendezvous with a timeout (us) */
> +static int wait_for_cpus(atomic_t *cnt, unsigned int expect,
> + unsigned int timeout)
> {
> - int error, cpu;
> -
> - error = microcode_update_cpu(patch);
> - if ( error )
> + while ( atomic_read(cnt) < expect )
> {
> - microcode_ops->free_patch(microcode_cache);
> - return error;
> + if ( !timeout )
> + {
> + printk("CPU%d: Timeout when waiting for CPUs calling in\n",
> + smp_processor_id());
> + return -EBUSY;
> + }
> + udelay(1);
> + timeout--;
Nit: you could do the decrement inside the if condition.
> }
>
> + return 0;
> +}
>
> - cpu = cpumask_next(smp_processor_id(), &cpu_online_map);
> - if ( cpu < nr_cpu_ids )
> - return continue_hypercall_on_cpu(cpu, do_microcode_update, patch);
> +static int do_microcode_update(void *patch)
> +{
> + unsigned int cpu = smp_processor_id();
> + unsigned int cpu_nr = num_online_cpus();
> + unsigned int finished;
> + int ret;
> + static bool error;
>
> - microcode_update_cache(patch);
> + atomic_inc(&cpu_in);
> + ret = wait_for_cpus(&cpu_in, cpu_nr, MICROCODE_CALLIN_TIMEOUT_US);
> + if ( ret )
> + return ret;
>
> - return error;
> + ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
> + /*
> + * Load microcode update on only one logical processor per core.
> + * Here, among logical processors of a core, the one with the
> + * lowest thread id is chosen to perform the loading.
> + */
> + if ( !ret && (cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu))) )
> + {
> + ret = microcode_ops->apply_microcode(patch);
> + if ( !ret )
> + atomic_inc(&cpu_updated);
> + }
> + /*
> + * Increase the wait timeout to a safe value here since we're serializing
> + * the microcode update and that could take a while on a large number of
> + * CPUs. And that is fine as the *actual* timeout will be determined by
> + * the last CPU finished updating and thus cut short
It's likely me missing something, but where is this serialization
being done?
I assume it's done by apply_microcode because do_microcode_update
doesn't do any serialization of microcode loading.
> + */
> + atomic_inc(&cpu_out);
> + finished = atomic_read(&cpu_out);
> + while ( !error && finished != cpu_nr )
> + {
> + /*
> + * During each timeout interval, at least a CPU is expected to
> + * finish its update. Otherwise, something goes wrong.
> + */
> + if ( wait_for_cpus(&cpu_out, finished + 1,
> + MICROCODE_UPDATE_TIMEOUT_US) && !error )
> + {
> + error = true;
I'm not sure I see the point of the error variable, you already bring
the system down with panic. If the intention is to prevent multiple
panics from different threads then you need to use some kind of
atomic fetch and set or else the code is racy.
> + panic("Timeout when finishing updating microcode (finished %d/%d)",
Both finished and cpu_nr are unsigned ints, hence you should use %u
instead of %d.
> + finished, cpu_nr);
> + }
This whole loop seems to be designed for serialized microcode
application, which is not the case with the current implementation
where microcode is updated in parallel on all the cores?
IMO you should just wait for MICROCODE_UPDATE_TIMEOUT_US a single
time.
> + finished = atomic_read(&cpu_out);
> + }
> +
> + /*
> + * Refresh CPU signature (revision) on threads which didn't call
> + * apply_microcode().
> + */
> + if ( cpu != cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
> + ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
> +
> + return ret;
> }
>
> int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
> {
> int ret;
> void *buffer;
> + unsigned int cpu, nr_cores;
> struct microcode_patch *patch;
>
> if ( len != (uint32_t)len )
> @@ -316,11 +401,18 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
> goto free;
> }
>
> + /* cpu_online_map must not change during update */
> + if ( !get_cpu_maps() )
> + {
> + ret = -EBUSY;
> + goto free;
> + }
> +
> if ( microcode_ops->start_update )
> {
> ret = microcode_ops->start_update();
> if ( ret != 0 )
> - goto free;
> + goto put;
> }
>
> patch = microcode_parse_blob(buffer, len);
> @@ -337,12 +429,59 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
> if ( patch )
> microcode_ops->free_patch(patch);
> ret = -EINVAL;
> - goto free;
> + goto put;
> }
>
> - ret = continue_hypercall_on_cpu(cpumask_first(&cpu_online_map),
> - do_microcode_update, patch);
> + atomic_set(&cpu_in, 0);
> + atomic_set(&cpu_out, 0);
> + atomic_set(&cpu_updated, 0);
> +
> + /* Calculate the number of online CPU core */
> + nr_cores = 0;
> + for_each_online_cpu(cpu)
> + if ( cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
> + nr_cores++;
> +
> + printk(XENLOG_INFO "%d cores are to update their microcode\n", nr_cores);
Same here, nr_cores is unsigned.
> +
> + /*
> + * We intend to disable interrupt for long time, which may lead to
> + * watchdog timeout.
> + */
> + watchdog_disable();
> + /*
> + * Late loading dance. Why the heavy-handed stop_machine effort?
> + *
> + * - HT siblings must be idle and not execute other code while the other
> + * sibling is loading microcode in order to avoid any negative
> + * interactions cause by the loading.
> + *
> + * - In addition, microcode update on the cores must be serialized until
> + * this requirement can be relaxed in the future. Right now, this is
> + * conservative and good.
> + */
> + ret = stop_machine_run(do_microcode_update, patch, NR_CPUS);
> + watchdog_enable();
> +
> + if ( atomic_read(&cpu_updated) == nr_cores )
> + {
> + spin_lock(µcode_mutex);
> + microcode_update_cache(patch);
> + spin_unlock(µcode_mutex);
> + }
> + else if ( atomic_read(&cpu_updated) == 0 )
> + microcode_ops->free_patch(patch);
> + else
> + {
> + printk("Updating microcode succeeded on part of CPUs and failed on\n"
I would prefix this with XENLOG_ERR and an explicit "ERROR: " prefix
in the format string.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-05 14:52 ` Roger Pau Monné
2019-06-05 15:15 ` Jan Beulich
-1 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2019-06-05 14:52 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich, xen-devel
On Mon, May 27, 2019 at 04:31:30PM +0800, Chao Gao wrote:
> microcode_update_lock is to prevent logic threads of a same core from
> updating microcode at the same time. But due to using a global lock, it
> also prevented parallel microcode updating on different cores.
Oh, OK, so that's what I was missing from patch 8 and what serializes
the updating.
> Remove this lock in order to update microcode in parallel. It is safe
> because we have already ensured serialization of sibling threads at the
> caller side.
Then you certainly need to fix the wait loop in do_microcode_update to
only wait for MICROCODE_UPDATE_TIMEOUT_US regardless of the number of
CPUs in the system?
> 1.For late microcode update, do_microcode_update() ensures that only one
> sibiling thread of a core can update microcode.
> 2.For microcode update during system startup or CPU-hotplug,
> microcode_mutex() guarantees update serialization of logical threads.
> 3.get/put_cpu_bitmaps() prevents the concurrency of CPU-hotplug and
> late microcode update.
>
> Note that printk in apply_microcode() and svm_host_osvm_init() (for AMD
> only) are still processed sequentially.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
Patch LGTM, but it needs to fix the wait loop in do_microcode_update.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
(?)
@ 2019-06-05 14:53 ` Jan Beulich
2019-06-11 12:46 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-05 14:53 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> microcode_update_lock is to prevent logic threads of a same core from
> updating microcode at the same time. But due to using a global lock, it
> also prevented parallel microcode updating on different cores.
>
> Remove this lock in order to update microcode in parallel. It is safe
> because we have already ensured serialization of sibling threads at the
> caller side.
> 1.For late microcode update, do_microcode_update() ensures that only one
> sibiling thread of a core can update microcode.
> 2.For microcode update during system startup or CPU-hotplug,
> microcode_mutex() guarantees update serialization of logical threads.
> 3.get/put_cpu_bitmaps() prevents the concurrency of CPU-hotplug and
> late microcode update.
>
> Note that printk in apply_microcode() and svm_host_osvm_init() (for AMD
> only) are still processed sequentially.
>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
> ---
> Changes in v7:
> - reworked. Remove complex lock logics introduced in v5 and v6. The microcode
> patch to be applied is passed as an argument without any global variable. Thus
> no lock is added to serialize potential readers/writers. Callers of
> apply_microcode() will guarantee the correctness: the patch poninted by the
> arguments won't be changed by others.
Much better this way indeed.
> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch *patch)
>
> mc_intel = patch->mc_intel;
>
> - /* serialize access to the physical write to MSR 0x79 */
> - spin_lock_irqsave(µcode_update_lock, flags);
> + BUG_ON(local_irq_is_enabled());
>
> /*
> * Writeback and invalidate caches before updating microcode to avoid
Thinking about it - what happens if we hit an NMI or #MC here?
watchdog_disable(), a call to which you add in an earlier patch,
doesn't really suppress the generation of NMIs, it only tells the
handler not to look at the accumulated statistics.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 10/10] x86/microcode: always collect_cpu_info() during boot
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
@ 2019-06-05 14:56 ` Roger Pau Monné
2019-06-11 13:02 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Roger Pau Monné @ 2019-06-05 14:56 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich, xen-devel
On Mon, May 27, 2019 at 04:31:31PM +0800, Chao Gao wrote:
> From: Sergey Dyasli <sergey.dyasli@citrix.com>
>
> Currently cpu_sig struct is not updated during boot when either:
>
> 1. ucode_scan is set to false (e.g. no "ucode=scan" in cmdline)
> 2. initrd does not contain a microcode blob
>
> These will result in cpu_sig.rev being 0 which affects APIC's
> check_deadline_errata() and retpoline_safe() functions.
>
> Fix this by getting ucode revision early during boot and SMP bring up.
> While at it.
I don't understand the last "While at it" sentence. Can it be
removed?
Is this an issue with current code? If so this could be merged ahead of
the rest of the series, and should likely be patch 1.
OTOH if the issue this patch is fixing is introduced by this series
please merge the fix with the respective patch that introduced the
bug.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 10/10] x86/microcode: always collect_cpu_info() during boot
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
(?)
(?)
@ 2019-06-05 15:05 ` Jan Beulich
2019-06-11 12:58 ` Chao Gao
-1 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-05 15:05 UTC (permalink / raw)
To: Sergey Dyasli, Chao Gao
Cc: Andrew Cooper, xen-devel, Ashok Raj, WeiLiu, Roger Pau Monne
>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
> From: Sergey Dyasli <sergey.dyasli@citrix.com>
>
> Currently cpu_sig struct is not updated during boot when either:
>
> 1. ucode_scan is set to false (e.g. no "ucode=scan" in cmdline)
> 2. initrd does not contain a microcode blob
I thought we'd already discussed this - "ucode=<number>" is not
covered by this.
> These will result in cpu_sig.rev being 0 which affects APIC's
> check_deadline_errata() and retpoline_safe() functions.
>
> Fix this by getting ucode revision early during boot and SMP bring up.
> While at it.
While at it?
> Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
> Signed-off-by: Chao Gao <chao.gao@intel.com>
> ---
> changes in v7:
> - rebase on patch 1~9
From the looks of it this doesn't depend on any of the earlier changes
(except the ucode_cpu_info -> cpu_sig change), and hence could go
in right away. Am I overlooking something? If not, all that's needed
would be clarifications of the description as per above.
> --- a/xen/arch/x86/microcode.c
> +++ b/xen/arch/x86/microcode.c
> @@ -590,6 +590,10 @@ int __init early_microcode_init(void)
>
> if ( microcode_ops )
> {
> + rc = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
> + if ( rc )
> + return rc;
> +
> if ( ucode_mod.mod_end || ucode_blob.size )
> rc = early_microcode_parse_and_update_cpu();
> }
Do we really need to bail on error here? I don't see anything wrong
with simply continuing. The caller doesn't care about the return
value anyway, so best effort would seem to be good enough.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-05 14:52 ` Roger Pau Monné
@ 2019-06-05 15:15 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-05 15:15 UTC (permalink / raw)
To: Roger Pau Monne
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel, Chao Gao
>>> On 05.06.19 at 16:52, <roger.pau@citrix.com> wrote:
> On Mon, May 27, 2019 at 04:31:30PM +0800, Chao Gao wrote:
>> microcode_update_lock is to prevent logic threads of a same core from
>> updating microcode at the same time. But due to using a global lock, it
>> also prevented parallel microcode updating on different cores.
>
> Oh, OK, so that's what I was missing from patch 8 and what serializes
> the updating.
>
>> Remove this lock in order to update microcode in parallel. It is safe
>> because we have already ensured serialization of sibling threads at the
>> caller side.
>
> Then you certainly need to fix the wait loop in do_microcode_update to
> only wait for MICROCODE_UPDATE_TIMEOUT_US regardless of the number of
> CPUs in the system?
Well, no, not exactly. On huge systems it may indeed still take longer
than on smaller ones. The way the waiting is coded now (expecting
forward progress in every MICROCODE_UPDATE_TIMEOUT_US period, is,
I think, acceptable. Plus leaving that logic alone will avoid touching it
yet again when introducing serial application policy as an option.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 01/10] misc/xen-ucode: Upload a microcode blob to the hypervisor
2019-06-04 16:14 ` Andrew Cooper
2019-06-04 16:23 ` Jan Beulich
@ 2019-06-06 2:29 ` Chao Gao
1 sibling, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-06 2:29 UTC (permalink / raw)
To: Andrew Cooper
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Konrad Rzeszutek Wilk,
Ian Jackson, xen-devel
On Tue, Jun 04, 2019 at 05:14:14PM +0100, Andrew Cooper wrote:
>On 27/05/2019 09:31, Chao Gao wrote:
>> This patch provides a tool for late microcode update.
>>
>> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>> ---
>> Changes in v7:
>> - introduce xc_microcode_update() rather than xc_platform_op()
>> - avoid creating bounce buffer twice
>> - rename xenmicrocode to xen-ucode, following naming tradition
>> of other tools there.
>>
>> ---
>> tools/libxc/include/xenctrl.h | 1 +
>> tools/libxc/xc_misc.c | 23 +++++++++++++
>> tools/misc/Makefile | 4 +++
>> tools/misc/xen-ucode.c | 78 +++++++++++++++++++++++++++++++++++++++++++
>> 4 files changed, 106 insertions(+)
>> create mode 100644 tools/misc/xen-ucode.c
>>
>> diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
>> index 538007a..6d80ae5 100644
>> --- a/tools/libxc/include/xenctrl.h
>> +++ b/tools/libxc/include/xenctrl.h
>> @@ -1244,6 +1244,7 @@ typedef uint32_t xc_node_to_node_dist_t;
>> int xc_physinfo(xc_interface *xch, xc_physinfo_t *info);
>> int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
>> xc_cputopo_t *cputopo);
>> +int xc_microcode_update(xc_interface *xch, const void *buf, size_t len);
>> int xc_numainfo(xc_interface *xch, unsigned *max_nodes,
>> xc_meminfo_t *meminfo, uint32_t *distance);
>> int xc_pcitopoinfo(xc_interface *xch, unsigned num_devs,
>> diff --git a/tools/libxc/xc_misc.c b/tools/libxc/xc_misc.c
>> index 5e6714a..85538e0 100644
>> --- a/tools/libxc/xc_misc.c
>> +++ b/tools/libxc/xc_misc.c
>> @@ -226,6 +226,29 @@ int xc_physinfo(xc_interface *xch,
>> return 0;
>> }
>>
>> +int xc_microcode_update(xc_interface *xch, const void *buf, size_t len)
>> +{
>> + int ret;
>> + DECLARE_PLATFORM_OP;
>> + DECLARE_HYPERCALL_BUFFER(struct xenpf_microcode_update, uc);
>> +
>> + uc = xc_hypercall_buffer_alloc(xch, uc, len);
>> + if (uc == NULL)
>
>Xen style. Extra space please.
>
>> + return -1;
>> +
>> + memcpy(uc, buf, len);
>> +
>> + platform_op.cmd = XENPF_microcode_update;
>> + platform_op.u.microcode.length = len;
>> + set_xen_guest_handle(platform_op.u.microcode.data, uc);
>> +
>> + ret = do_platform_op(xch, &platform_op);
>> +
>> + xc_hypercall_buffer_free(xch, uc);
>> +
>> + return ret;
>> +}
>> +
>> int xc_cputopoinfo(xc_interface *xch, unsigned *max_cpus,
>> xc_cputopo_t *cputopo)
>> {
>> diff --git a/tools/misc/Makefile b/tools/misc/Makefile
>> index d4320dc..63947bf 100644
>> --- a/tools/misc/Makefile
>> +++ b/tools/misc/Makefile
>> @@ -22,6 +22,7 @@ INSTALL_SBIN-$(CONFIG_X86) += xen-hvmcrash
>> INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx
>> INSTALL_SBIN-$(CONFIG_X86) += xen-lowmemd
>> INSTALL_SBIN-$(CONFIG_X86) += xen-mfndump
>> +INSTALL_SBIN-$(CONFIG_X86) += xen-ucode
>> INSTALL_SBIN += xencov
>> INSTALL_SBIN += xenlockprof
>> INSTALL_SBIN += xenperf
>> @@ -113,4 +114,7 @@ xen-lowmemd: xen-lowmemd.o
>> xencov: xencov.o
>> $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
>>
>> +xen-ucode: xen-ucode.o
>> + $(CC) $(LDFLAGS) -o $@ $< $(LDLIBS_libxenctrl) $(APPEND_LDFLAGS)
>> +
>> -include $(DEPS_INCLUDE)
>> diff --git a/tools/misc/xen-ucode.c b/tools/misc/xen-ucode.c
>> new file mode 100644
>> index 0000000..da668ca
>> --- /dev/null
>> +++ b/tools/misc/xen-ucode.c
>> @@ -0,0 +1,78 @@
>> +#define _GNU_SOURCE
>> +
>> +#include <stdio.h>
>> +#include <stdlib.h>
>> +#include <sys/mman.h>
>> +#include <errno.h>
>> +#include <string.h>
>> +#include <inttypes.h>
>> +#include <unistd.h>
>> +#include <sys/types.h>
>> +#include <sys/stat.h>
>> +#include <fcntl.h>
>> +#include <xenctrl.h>
>> +
>> +void show_help(void)
>> +{
>> + fprintf(stderr,
>> + "xenmicrocode: Xen microcode updating tool\n"
>> + "Usage: xenmicrocode <microcode blob>\n");
>
>s/xenmicrocode/xen-ucode/
>
>Both can be fixed on commit
>
>Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Thanks.
As Jan said, it is better to use argv[0] here.
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 02/10] microcode/intel: extend microcode_update_match()
2019-06-04 14:39 ` Jan Beulich
2019-06-05 13:22 ` Roger Pau Monné
@ 2019-06-06 8:26 ` Chao Gao
2019-06-06 9:01 ` Jan Beulich
1 sibling, 1 reply; 64+ messages in thread
From: Chao Gao @ 2019-06-06 8:26 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Tue, Jun 04, 2019 at 08:39:15AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> --- a/xen/arch/x86/microcode_intel.c
>> +++ b/xen/arch/x86/microcode_intel.c
>> @@ -134,14 +134,28 @@ static int collect_cpu_info(unsigned int cpu_num, struct cpu_signature *csig)
>> return 0;
>> }
>>
>> -static inline int microcode_update_match(
>> - unsigned int cpu_num, const struct microcode_header_intel *mc_header,
>> - int sig, int pf)
>> +static enum microcode_match_result microcode_update_match(
>> + const struct microcode_header_intel *mc_header, unsigned int sig,
>> + unsigned int pf, unsigned int rev)
>> {
>> - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
>> + const struct extended_sigtable *ext_header;
>> + const struct extended_signature *ext_sig;
>> + unsigned long data_size = get_datasize(mc_header);
>> + unsigned int i;
>> +
>> + if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
>> + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
>
>As indicated before, I think you would better also provide an "equal"
>indication. Iirc I've told you that I have one system where the cores
>get handed over from the BIOS in an inconsistent state (only core
>has ucode loaded). Hence we'd want to be able to also _store_
>ucode matching that found on CPU 0, without actually want to _load_
>it there.
Will do. What if no microcode update is provided in this case? Shall
we refuse to boot? If we allow different microcode revisions in the
system, it would complicate late microcode loading.
>
>> - return (sigmatch(sig, uci->cpu_sig.sig, pf, uci->cpu_sig.pf) &&
>> - (mc_header->rev > uci->cpu_sig.rev));
>> + if ( get_totalsize(mc_header) == (data_size + MC_HEADER_SIZE) )
>> + return MIS_UCODE;
>
>Okay, you're tightening the original <= to == here. But if you're
>already tightening things, why don't you make sure you actually
>have enough data to ...
>
>> + ext_header = (const void *)(mc_header + 1) + data_size;
>
>... hold an extended header, and then also to hold ...
>
>> + ext_sig = (const void *)(ext_header + 1);
>> + for ( i = 0; i < ext_header->count; i++ )
>> + if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
>> + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
>
>... enough array elements?
Do you think below incremental change is fine?
diff --git a/xen/arch/x86/microcode_intel.c b/xen/arch/x86/microcode_intel.c
index 94a1561..3dcbd28 100644
--- a/xen/arch/x86/microcode_intel.c
+++ b/xen/arch/x86/microcode_intel.c
@@ -138,18 +138,25 @@ static enum microcode_match_result microcode_update_match(
const struct extended_signature *ext_sig;
unsigned long data_size = get_datasize(mc_header);
unsigned int i;
+ const void *end = (const void *)mc_header + get_totalsize(mc_header);
if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
- if ( get_totalsize(mc_header) == (data_size + MC_HEADER_SIZE) )
- return MIS_UCODE;
-
ext_header = (const void *)(mc_header + 1) + data_size;
ext_sig = (const void *)(ext_header + 1);
- for ( i = 0; i < ext_header->count; i++ )
- if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
- return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
+
+ /*
+ * Make sure there is enough space to hold an extended header and enough
+ * array elements.
+ */
+ if ( (end >= (const void *)ext_sig) &&
+ (end >= (const void *)(ext_sig + ext_header->count)) )
+ {
+ for ( i = 0; i < ext_header->count; i++ )
+ if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
+ return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
+ }
return MIS_UCODE;
}
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 02/10] microcode/intel: extend microcode_update_match()
2019-06-06 8:26 ` Chao Gao
@ 2019-06-06 9:01 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-06 9:01 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 06.06.19 at 10:26, <chao.gao@intel.com> wrote:
> On Tue, Jun 04, 2019 at 08:39:15AM -0600, Jan Beulich wrote:
>>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>>> --- a/xen/arch/x86/microcode_intel.c
>>> +++ b/xen/arch/x86/microcode_intel.c
>>> @@ -134,14 +134,28 @@ static int collect_cpu_info(unsigned int cpu_num,
> struct cpu_signature *csig)
>>> return 0;
>>> }
>>>
>>> -static inline int microcode_update_match(
>>> - unsigned int cpu_num, const struct microcode_header_intel *mc_header,
>>> - int sig, int pf)
>>> +static enum microcode_match_result microcode_update_match(
>>> + const struct microcode_header_intel *mc_header, unsigned int sig,
>>> + unsigned int pf, unsigned int rev)
>>> {
>>> - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu_num);
>>> + const struct extended_sigtable *ext_header;
>>> + const struct extended_signature *ext_sig;
>>> + unsigned long data_size = get_datasize(mc_header);
>>> + unsigned int i;
>>> +
>>> + if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
>>> + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
>>
>>As indicated before, I think you would better also provide an "equal"
>>indication. Iirc I've told you that I have one system where the cores
>>get handed over from the BIOS in an inconsistent state (only core
>>has ucode loaded). Hence we'd want to be able to also _store_
>>ucode matching that found on CPU 0, without actually want to _load_
>>it there.
>
> Will do. What if no microcode update is provided in this case? Shall
> we refuse to boot? If we allow different microcode revisions in the
> system, it would complicate late microcode loading.
No, I don't think we should refuse to boot in such a case. We may
want to warn about the situation, but should continue booting in
a best effort manner. And no, I also don't think it'll complicate
late loading meaningfully. Late loading, after all, is then the only
way to get the system into "proper" shape again (other than
rebooting after making available ucode to early boot).
>>> - return (sigmatch(sig, uci->cpu_sig.sig, pf, uci->cpu_sig.pf) &&
>>> - (mc_header->rev > uci->cpu_sig.rev));
>>> + if ( get_totalsize(mc_header) == (data_size + MC_HEADER_SIZE) )
>>> + return MIS_UCODE;
>>
>>Okay, you're tightening the original <= to == here. But if you're
>>already tightening things, why don't you make sure you actually
>>have enough data to ...
>>
>>> + ext_header = (const void *)(mc_header + 1) + data_size;
>>
>>... hold an extended header, and then also to hold ...
>>
>>> + ext_sig = (const void *)(ext_header + 1);
>>> + for ( i = 0; i < ext_header->count; i++ )
>>> + if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
>>> + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
>>
>>... enough array elements?
>
> Do you think below incremental change is fine?
Something along these lines, yes. I don't think ...
> --- a/xen/arch/x86/microcode_intel.c
> +++ b/xen/arch/x86/microcode_intel.c
> @@ -138,18 +138,25 @@ static enum microcode_match_result microcode_update_match(
> const struct extended_signature *ext_sig;
> unsigned long data_size = get_datasize(mc_header);
> unsigned int i;
> + const void *end = (const void *)mc_header + get_totalsize(mc_header);
>
> if ( sigmatch(sig, mc_header->sig, pf, mc_header->pf) )
> return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
>
> - if ( get_totalsize(mc_header) == (data_size + MC_HEADER_SIZE) )
> - return MIS_UCODE;
> -
> ext_header = (const void *)(mc_header + 1) + data_size;
> ext_sig = (const void *)(ext_header + 1);
> - for ( i = 0; i < ext_header->count; i++ )
> - if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
> - return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
> +
> + /*
> + * Make sure there is enough space to hold an extended header and enough
> + * array elements.
> + */
> + if ( (end >= (const void *)ext_sig) &&
> + (end >= (const void *)(ext_sig + ext_header->count)) )
> + {
> + for ( i = 0; i < ext_header->count; i++ )
> + if ( sigmatch(sig, ext_sig[i].sig, pf, ext_sig[i].pf) )
> + return (mc_header->rev > rev) ? NEW_UCODE : OLD_UCODE;
> + }
... this re-indentation of the for() is needed. Just like the function was
previously coded, the if() condition could be inverted and its body
could be "return MIS_UCODE;". But it's a style question, and hence
largely up to you.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 03/10] microcode: introduce a global cache of ucode patch
2019-06-04 15:03 ` Jan Beulich
@ 2019-06-10 5:33 ` Chao Gao
2019-06-11 6:50 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Chao Gao @ 2019-06-10 5:33 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Tue, Jun 04, 2019 at 09:03:20AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> +bool microcode_update_cache(struct microcode_patch *patch)
>> +{
>> +
>> + ASSERT(spin_is_locked(µcode_mutex));
>> +
>> + if ( !microcode_ops->match_cpu(patch) )
>> + return false;
>> +
>> + if ( !microcode_cache )
>> + microcode_cache = patch;
>> + else if ( microcode_ops->compare_patch(patch, microcode_cache) ==
>> + NEW_UCODE )
>> + {
>> + microcode_ops->free_patch(microcode_cache);
>> + microcode_cache = patch;
>> + }
>
>Hmm, okay, the way you do things here three enumeration values
>may indeed be sufficient. "old" may just be a little misleading then.
>(As to my respective comment on the previous patch.)
>
>> +static struct microcode_patch *alloc_microcode_patch(
>> + const struct microcode_amd *mc_amd)
>> +{
>> + struct microcode_patch *microcode_patch = xmalloc(struct microcode_patch);
>> + struct microcode_amd *cache = xmalloc(struct microcode_amd);
>> + void *mpb = xmalloc_bytes(mc_amd->mpb_size);
>> + struct equiv_cpu_entry *equiv_cpu_table =
>> + xmalloc_bytes(mc_amd->equiv_cpu_table_size);
>> +
>> + if ( !microcode_patch || !cache || !mpb || !equiv_cpu_table )
>> + {
>> + xfree(microcode_patch);
>> + xfree(cache);
>> + xfree(mpb);
>> + xfree(equiv_cpu_table);
>> + printk(XENLOG_ERR "microcode: Can not allocate memory\n");
>
>I'm not convinced this needs logging.
>
>> + return ERR_PTR(-ENOMEM);
>> + }
>> +
>> + cache->equiv_cpu_table = equiv_cpu_table;
>> + cache->mpb = mpb;
>> + memcpy(cache->equiv_cpu_table, mc_amd->equiv_cpu_table,
>
>Why not use the local variable here and ...
>
>> + mc_amd->equiv_cpu_table_size);
>> + memcpy(cache->mpb, mc_amd->mpb, mc_amd->mpb_size);
>
>here? Less source code and presumably also slightly less binary
>code. In fact I wonder if you wouldn't better memcpy() first
>anyway, and only then store the values into the fields. It won't
>matter much with the global lock held, but it's generally good
>practice to do things in an order that won't risk to confuse
>hypothetical consumers of the data.
Will do.
>
>> +static void free_patch(struct microcode_patch *microcode_patch)
>> +{
>> + struct microcode_amd *mc_amd = microcode_patch->mc_amd;
>> +
>> + xfree(mc_amd->equiv_cpu_table);
>> + xfree(mc_amd->mpb);
>> + xfree(mc_amd);
>> + xfree(microcode_patch);
>
>I think I said so before: Freeing of the generic wrapper struct
>would probably better be placed in generic code.
Do you mean something as shown below:
/* in generic code */
struct microcode_patch {
union {
struct microcode_intel *mc_intel;
struct microcode_amd *mc_amd;
void *mc;
};
};
void microcode_free_patch(struct microcode_patch *microcode_patch)
{
microcode_ops->free_patch(microcode_patch->mc);
xfree(microcode_patch);
}
/* in vendor-specific (AMD) code */
static void free_patch(void *mc)
{
struct microcode_amd *mc_amd = mc;
xfree(mc_amd->equiv_cpu_table);
xfree(mc_amd->mpb);
xfree(mc_amd);
}
>
>> @@ -497,7 +558,20 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
>> while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
>> &offset)) == 0 )
>> {
>> - if ( microcode_fits(mc_amd, cpu) )
>> + struct microcode_patch *new_patch = alloc_microcode_patch(mc_amd);
>> +
>> + if ( IS_ERR(new_patch) )
>> + {
>> + error = PTR_ERR(new_patch);
>> + break;
>> + }
>> +
>> + if ( match_cpu(new_patch) )
>> + microcode_update_cache(new_patch);
>> + else
>> + free_patch(new_patch);
>
>Why do you re-do what microcode_update_cache() already does?
>It calls ->match_cpu() and ->free_patch() all by itself. It looks as
>if it would need to gain one more ->free_patch() invocation though.
>
Will remove both invocations of match_cpu().
To support the case (the broken bios) you described, a patch which
needs to be stored isn't necessary to be newer than the microcode loaded
to current CPU. As long as the processor's signature is covered by the
patch, we will store the patch regardless the revision number.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 04/10] microcode: remove struct ucode_cpu_info
2019-06-04 15:13 ` Jan Beulich
@ 2019-06-10 7:19 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-10 7:19 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Tue, Jun 04, 2019 at 09:13:46AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> We can remove the per-cpu cache field in struct ucode_cpu_info since
>> it has been replaced by a global cache. It would leads to only one field
>> remaining in ucode_cpu_info. Then, this struct is removed and the
>> remaining field (cpu signature) is stored in per-cpu area.
>>
>> Also remove 'microcode_resume_match' from microcode_ops because the
>> check is done in find_patch(). The cpu status notifier is also
>> removed. It was used to free the "mc" field to avoid memory leak.
>
>There's no find_patch() function anymore afaics. And I also think this
>should be a separate patch. The above isn't enough imo to justify ...
>
>> int microcode_resume_cpu(unsigned int cpu)
>> {
>> int err;
>> - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
>> - struct cpu_signature nsig;
>> - unsigned int cpu2;
>> + struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
>>
>> if ( !microcode_ops )
>> return 0;
>>
>> spin_lock(µcode_mutex);
>>
>> - err = microcode_ops->collect_cpu_info(cpu, &uci->cpu_sig);
>> - if ( err )
>> - {
>> - __microcode_fini_cpu(cpu);
>> - spin_unlock(µcode_mutex);
>> - return err;
>> - }
>> -
>> - if ( uci->mc.mc_valid )
>> - {
>> - err = microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid);
>> - if ( err >= 0 )
>> - {
>> - if ( err )
>> - err = microcode_ops->apply_microcode(cpu);
>> - spin_unlock(µcode_mutex);
>> - return err;
>> - }
>> - }
>> -
>> - nsig = uci->cpu_sig;
>> - __microcode_fini_cpu(cpu);
>> - uci->cpu_sig = nsig;
>> -
>> - err = -EIO;
>> - for_each_online_cpu ( cpu2 )
>> - {
>> - uci = &per_cpu(ucode_cpu_info, cpu2);
>> - if ( uci->mc.mc_valid &&
>> - microcode_ops->microcode_resume_match(cpu, uci->mc.mc_valid) > 0 )
>> - {
>> - err = microcode_ops->apply_microcode(cpu);
>> - break;
>> - }
>> - }
>
>... in particular the removal of this loop, the more that both the
>loop and the code ahead of it also call ->apply_microcode().
Ok. Will split it out from this patch and refine the patch description.
Basically, this function tries best to find a suitable patch from the
per-cpu cache and loads it. Currently, the per-cpu cache is replaced by
the global cache, and ->apply_microcode() loads the global cache rather
then the per-cpu cache. Hence, a simple invocation of ->apply_microcode()
is enough to apply the global cache during CPU hotplug or resuming from
hibernation.
>
>> @@ -281,7 +281,6 @@ static enum microcode_match_result compare_patch(
>> */
>> static int get_matching_microcode(const void *mc, unsigned int cpu)
>> {
>> - struct ucode_cpu_info *uci = &per_cpu(ucode_cpu_info, cpu);
>
>Note how this was using "cpu".
>
>> @@ -308,17 +307,7 @@ static int get_matching_microcode(const void *mc, unsigned int cpu)
>>
>> pr_debug("microcode: CPU%d found a matching microcode update with"
>> " version %#x (current=%#x)\n",
>> - cpu, mc_header->rev, uci->cpu_sig.rev);
>> - new_mc = xmalloc_bytes(total_size);
>> - if ( new_mc == NULL )
>> - {
>> - printk(KERN_ERR "microcode: error! Can not allocate memory\n");
>> - return -ENOMEM;
>> - }
>> -
>> - memcpy(new_mc, mc, total_size);
>> - xfree(uci->mc.mc_intel);
>> - uci->mc.mc_intel = new_mc;
>> + cpu, mc_header->rev, this_cpu(cpu_sig).rev);
>
>Why "this_cpu()" here?
It should be a part of next patch.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 05/10] microcode: remove pointless 'cpu' parameter
2019-06-04 15:29 ` Jan Beulich
@ 2019-06-10 7:31 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-10 7:31 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Tue, Jun 04, 2019 at 09:29:34AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> --- a/xen/arch/x86/microcode_amd.c
>> +++ b/xen/arch/x86/microcode_amd.c
>> @@ -78,8 +78,9 @@ struct mpbhdr {
>> static DEFINE_SPINLOCK(microcode_update_lock);
>>
>> /* See comment in start_update() for cases when this routine fails */
>> -static int collect_cpu_info(unsigned int cpu, struct cpu_signature *csig)
>> +static int collect_cpu_info(struct cpu_signature *csig)
>> {
>> + unsigned int cpu = smp_processor_id();
>> struct cpuinfo_x86 *c = &cpu_data[cpu];
>
>I think it would be more clear if you used current_cpu_data here.
>The only other use of "cpu" is in a pr_debug(), which by default
>expands to nothing anyway, and hence is cheap to change to
>use smp_processor_id() instead.
Will do.
>
>> @@ -435,14 +429,14 @@ static const unsigned int final_levels[] = {
>> 0x010000af
>> };
>>
>> -static bool_t check_final_patch_levels(unsigned int cpu)
>> +static bool check_final_patch_levels(void)
>> {
>> /*
>> * Check the current patch levels on the cpu. If they are equal to
>> * any of the 'final_levels', then we should not update the microcode
>> * patch on the cpu as system will hang otherwise.
>> */
>> - const struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
>> + const struct cpu_signature *sig = &this_cpu(cpu_sig);
>> unsigned int i;
>
>I don't see any dependency of this function upon running on
>the subject CPU.
Ok. I will drop this change.
>
>> @@ -279,12 +278,13 @@ static enum microcode_match_result compare_patch(
>> * return 1 - found update
>> * return < 0 - error
>> */
>> -static int get_matching_microcode(const void *mc, unsigned int cpu)
>> +static int get_matching_microcode(const void *mc)
>> {
>> const struct microcode_header_intel *mc_header = mc;
>> unsigned long total_size = get_totalsize(mc_header);
>> void *new_mc = xmalloc_bytes(total_size);
>> struct microcode_patch *new_patch = xmalloc(struct microcode_patch);
>> + unsigned int __maybe_unused cpu = smp_processor_id();
>
>The __maybe_unused is for the sole use in pr_debug()? Please
>instead use smp_processor_id() there, if so.
Will do.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode()
2019-06-05 12:37 ` Jan Beulich
@ 2019-06-11 3:32 ` Chao Gao
2019-06-11 7:08 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Chao Gao @ 2019-06-11 3:32 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Wed, Jun 05, 2019 at 06:37:27AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> During late microcode update, apply_microcode() is invoked in
>> cpu_request_microcode(). To make late microcode update more reliable,
>> we want to put the apply_microcode() into stop_machine context. So
>> we split out it from cpu_request_microcode(). As a consequence,
>> apply_microcode() should be invoked explicitly in the common code.
>>
>> Previously, apply_microcode() gets the microcode patch to be applied from
>> the microcode cache. Now, the patch is passed as a function argument and
>> a patch is cached for cpu-hotplug and cpu resuming, only after it has
>> been loaded to a cpu without any error. As a consequence, the
>> 'match_cpu' check in microcode_update_cache is removed, which otherwise
>> would fail.
>
>The "only after it has been loaded to a cpu without any error" is a
>problem, precisely for the case where ucode on the different cores
>is not in sync initially. I would actually like to put up this question:
>When a core has no ucode loaded at all yet and only strictly older
>(than loaded on some other cores) ucode is found to be available,
>whether then it wouldn't still be better to apply that ucode to
>_at least_ the cores that have none loaded yet.
Yes, it is better for this special case. And I agree to support this case.
This in v7, a patch is loaded only if its revision is newer than that
loaded to current CPU. And it is stored only if it has been loaded
successfully. But, as you described, a broken bios might puts the system
in an inconsistent state (multiple microcode revision in the system) and
furthermore in this case, if no or an older microcode update is
provided, early loading cannot get the system into sane state. So for
both early and late microcode loading, we could face a situation that
the patch to be loaded has equal or old revision than microcode of some
CPUs.
Changes I plan to make in next version are:
1. For early microcode, a patch would be stored if it covers current CPU's
signature. All CPUs would try to load from the cache.
2. For late microcode, a patch is loaded only if its revision is newer than
*the patch cached*. And it is stored only if has been loaded without an
"EIO" error.
3. Cache replacement remains the same.
But it is a temperary solution, especially for CSPs. A better way
might be getting the newest ucode or upgrading to the newest bios,
even downgrading the bios to an older version which wouldn't put
the system into "insane" state.
>
>To get the system into "sane" state it may even be necessary to
>downgrade ucode on the cores which did have it loaded already,
>in such a situation.
>
>> On AMD side, svm_host_osvw_init() is supposed to be called after
>> microcode update. As apply_micrcode() won't be called by
>> cpu_request_microcode() now, svm_host_osvw_init() is moved to the
>> end of apply_microcode().
>
>I guess this really ought to become a vendor hook as well, but I
>wouldn't insist on you doing so here.
>
>> --- a/xen/arch/x86/acpi/power.c
>> +++ b/xen/arch/x86/acpi/power.c
>> @@ -253,7 +253,7 @@ static int enter_state(u32 state)
>>
>> console_end_sync();
>>
>> - microcode_resume_cpu();
>> + early_microcode_update_cpu();
>
>The use here, the (changed) use in start_secondary(), and the dropping
>of its __init suggest to make an attempt to find a better name for the
>function. Maybe microcode_update_one()?
Will do.
>> +/*
>> + * Load a microcode update to current CPU.
>> + *
>> + * If no patch is provided, the cached patch will be loaded. Microcode update
>> + * during APs bringup and CPU resuming falls into this case.
>> + */
>> +static int microcode_update_cpu(struct microcode_patch *patch)
>
>const?
>
>> {
>> - int err;
>> - struct cpu_signature *sig = &this_cpu(cpu_sig);
>> + int ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
>>
>> - if ( !microcode_ops )
>> - return 0;
>> + if ( unlikely(ret) )
>> + return ret;
>>
>> spin_lock(µcode_mutex);
>>
>> - err = microcode_ops->collect_cpu_info(sig);
>> - if ( likely(!err) )
>> - err = microcode_ops->apply_microcode();
>> - spin_unlock(µcode_mutex);
>> + if ( patch )
>> + {
>> + /*
>> + * If a patch is specified, it should has newer revision than
>> + * that of the patch cached.
>> + */
>> + if ( microcode_cache &&
>> + microcode_ops->compare_patch(patch, microcode_cache) != NEW_UCODE )
>> + {
>> + spin_unlock(µcode_mutex);
>> + return -EINVAL;
>> + }
>>
>> - return err;
>> -}
>> + ret = microcode_ops->apply_microcode(patch);
>
>There's no printk() here but ...
>
>> + }
>> + else if ( microcode_cache )
>> + {
>> + ret = microcode_ops->apply_microcode(microcode_cache);
>> + if ( ret == -EIO )
>> + printk("Update failed. Reboot needed\n");
>
>... you emit a log message here. Why the difference? And wouldn't
>it be better to have just a single call to ->apply anyway, by simply
>assigning "microcode_cache" to "patch" and moving the call a little
>further down?
-EIO means we loaded the patch but the revision didn't change. This
error code indicates an error in the patch. It is unlikely to happen.
And in this v7, a patch is stored after being loading successfully
on a CPU. To simplify handling of load failure and avoid cleanup to the
global cache (if a patch has a success rate, we need to consider which
one to save when we have a new patch with 95% rate and an old one with
100% rate, it would be really complex), we assume that loading the cache
(being loaded successfully on a CPU) shouldn't return EIO. Otherwise,
an error message is prompted.
>
>> + }
>> + else
>> + /* No patch to update */
>> + ret = -EINVAL;
>
>-ENOENT?
>
>> @@ -247,46 +270,32 @@ bool microcode_update_cache(struct microcode_patch *patch)
>> return true;
>> }
>>
>> -static int microcode_update_cpu(const void *buf, size_t size)
>> +static long do_microcode_update(void *patch)
>> {
>> - int err;
>> - unsigned int cpu = smp_processor_id();
>> - struct cpu_signature *sig = &per_cpu(cpu_sig, cpu);
>> -
>> - spin_lock(µcode_mutex);
>> + int error, cpu;
>
>While "int" is fine for error codes, it almost certainly wants to be
>"unsigned int" for "cpu". The more that it had been that was before.
>I also don't see why you need to switch from "err" to "error" - oh,
>...
>
>> - err = microcode_ops->collect_cpu_info(sig);
>> - if ( likely(!err) )
>> - err = microcode_ops->cpu_request_microcode(buf, size);
>> - spin_unlock(µcode_mutex);
>> -
>> - return err;
>> -}
>> -
>> -static long do_microcode_update(void *_info)
>> -{
>> - struct microcode_info *info = _info;
>> - int error;
>> + error = microcode_update_cpu(patch);
>
>... there was a pre-existing variable of that name here.
>
>> + if ( error )
>> + {
>> + microcode_ops->free_patch(microcode_cache);
>
>Does this also set "microcode_cache" to NULL? I didn't think so.
>It's anyway not really clear why _all_ forms of errors should lead
>to clearing of the cache. However - looking at the code further
>down, don't you rather mean to free "patch" here anyway?
Sorry, it should be "patch".
>
>> + return error;
>> + }
>>
>> - BUG_ON(info->cpu != smp_processor_id());
>>
>> - error = microcode_update_cpu(info->buffer, info->buffer_size);
>> - if ( error )
>> - info->error = error;
>> + cpu = cpumask_next(smp_processor_id(), &cpu_online_map);
>> + if ( cpu < nr_cpu_ids )
>> + return continue_hypercall_on_cpu(cpu, do_microcode_update, patch);
>>
>> - info->cpu = cpumask_next(info->cpu, &cpu_online_map);
>> - if ( info->cpu < nr_cpu_ids )
>> - return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
>> + microcode_update_cache(patch);
>
>Independent of my remarks at the top I would think that updating of
>the cache should happen after the first successful loading on a CPU,
>not after all CPUs have been updated successfully. There would then
>also not be any need to pass "patch" on to continue_hypercall_on_cpu()
>a few lines up from here (albeit from a general logic perspective it may
>indeed be easier to keep it that way).
The logic is removed in the next patch. So I prefer to keep it the same.
>
>> @@ -294,32 +303,49 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
>> if ( microcode_ops == NULL )
>> return -EINVAL;
>>
>> - info = xmalloc_bytes(sizeof(*info) + len);
>> - if ( info == NULL )
>> - return -ENOMEM;
>> -
>> - ret = copy_from_guest(info->buffer, buf, len);
>> - if ( ret != 0 )
>> + buffer = xmalloc_bytes(len);
>> + if ( !buffer )
>> {
>> - xfree(info);
>> - return ret;
>> + ret = -ENOMEM;
>> + goto free;
>> }
>>
>> - info->buffer_size = len;
>> - info->error = 0;
>> - info->cpu = cpumask_first(&cpu_online_map);
>> + if ( copy_from_guest(buffer, buf, len) )
>> + {
>> + ret = -EFAULT;
>> + goto free;
>> + }
>>
>> if ( microcode_ops->start_update )
>> {
>> ret = microcode_ops->start_update();
>> if ( ret != 0 )
>> - {
>> - xfree(info);
>> - return ret;
>> - }
>> + goto free;
>> }
>>
>> - return continue_hypercall_on_cpu(info->cpu, do_microcode_update, info);
>> + patch = microcode_parse_blob(buffer, len);
>> + if ( IS_ERR(patch) )
>> + {
>> + printk(XENLOG_ERR "Parsing microcode blob error %ld\n", PTR_ERR(patch));
>> + ret = PTR_ERR(patch);
>
>So I assume we would get here when the system already has the
>newest (or even newer) ucode loaded. That's not an error, and
>imo no log message suggesting so should be issued. Perhaps the
>parsing code could return NULL to indicate so?
Yes. 'patch' is set to indicate an error only if no matching patch has
been found and an error (for example, -ENOMEM) happens during parsing.
If no patch in the blob covers the current cpu signature, the parsing
code will return NULL.
>Although, judging
>by the code further down you already expect NULL potentially
>coming back, but I can't seem to be able to figure the condition(s).
>
>Also please switch the two lines around and use "ret" in the printk()
>invocation.
>
>> + goto free;
>> + }
>> +
>> + if ( !microcode_ops->match_cpu(patch) )
>> + {
>> + printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
>
>I assume the "matching" here is meant to cover the CPU signature?
Yes.
>The wording is ambiguous this way, because it could also mean there
>was no ucode found matching that which is already loaded (which, as
>per above, may end up being relevant).
Then I suppose it is fine to remove the "matching or".
>
>Furthermore - why this check, when microcode_parse_blob() already
>looks for something that's newer than what is currently loaded (and
>matches the CPU signature)?
Yes. It can be replaced with a null check.
>
>> @@ -362,13 +397,41 @@ int __init early_microcode_update_cpu(bool start_update)
>> }
>> if ( data )
>> {
>> - if ( start_update && microcode_ops->start_update )
>> + struct microcode_patch *patch;
>> +
>> + if ( microcode_ops->start_update )
>> rc = microcode_ops->start_update();
>>
>> if ( rc )
>> return rc;
>>
>> - return microcode_update_cpu(data, len);
>> + patch = microcode_parse_blob(data, len);
>> + if ( IS_ERR(patch) )
>> + {
>> + printk(XENLOG_ERR "Parsing microcode blob error %ld\n",
>> + PTR_ERR(patch));
>> + return PTR_ERR(patch);
>> + }
>> +
>> + if ( !microcode_ops->match_cpu(patch) )
>> + {
>> + printk(XENLOG_ERR "No matching or newer ucode found. Update aborted!\n");
>> + if ( patch )
>> + microcode_ops->free_patch(patch);
>> + return -EINVAL;
>> + }
>
>Same remarks here then.
>
>> @@ -292,6 +291,10 @@ static int apply_microcode(void)
>>
>> sig->rev = rev;
>>
>> +#ifdef CONFIG_HVM
>> + svm_host_osvw_init();
>> +#endif
>> +
>> return 0;
>> }
>
>While this now sits on the success path only, ...
>
>> @@ -592,17 +596,10 @@ static int cpu_request_microcode(const void *buf, size_t bufsize)
>> }
>>
>> out:
>> -#if CONFIG_HVM
>> - svm_host_osvw_init();
>> -#endif
>> + if ( error && !patch )
>> + patch = ERR_PTR(error);
>>
>> - /*
>> - * In some cases we may return an error even if processor's microcode has
>> - * been updated. For example, the first patch in a container file is loaded
>> - * successfully but subsequent container file processing encounters a
>> - * failure.
>> - */
>> - return error;
>> + return patch;
>> }
>
>... previously it has also been invoked in the error case. See the
>comment in start_update().
It depends on the scope of related MSRs. But I failed to find such
information in AMD SDM. So I will find a better place for
svm_host_osvw_init(). The last resort is to introduce another callback,
probably ->end_update().
>
>> --- a/xen/arch/x86/microcode_intel.c
>> +++ b/xen/arch/x86/microcode_intel.c
>> @@ -273,46 +273,27 @@ static enum microcode_match_result compare_patch(
>> old_header->pf, old_header->rev);
>> }
>>
>> -/*
>> - * return 0 - no update found
>> - * return 1 - found update
>> - * return < 0 - error
>> - */
>> -static int get_matching_microcode(const void *mc)
>> +static struct microcode_patch *allow_microcode_patch(
>
>Did you perhaps mean this to be alloc_microcode_patch()?
>
>> @@ -388,26 +368,39 @@ static long get_next_ucode_from_buffer(void **mc, const u8 *buf,
>> return offset + total_size;
>> }
>>
>> -static int cpu_request_microcode(const void *buf, size_t size)
>> +static struct microcode_patch *cpu_request_microcode(const void *buf,
>> + size_t size)
>> {
>> long offset = 0;
>> int error = 0;
>> void *mc;
>> + struct microcode_patch *patch = NULL;
>>
>> while ( (offset = get_next_ucode_from_buffer(&mc, buf, size, offset)) > 0 )
>> {
>> + struct microcode_patch *new_patch;
>> +
>> error = microcode_sanity_check(mc);
>> if ( error )
>> break;
>> - error = get_matching_microcode(mc);
>> - if ( error < 0 )
>> +
>> + new_patch = allow_microcode_patch(mc);
>> + if ( IS_ERR(new_patch) )
>> + {
>> + error = PTR_ERR(new_patch);
>> break;
>> - /*
>> - * It's possible the data file has multiple matching ucode,
>> - * lets keep searching till the latest version
>> - */
>> - if ( error == 1 )
>> - error = 0;
>> + }
>> +
>> + /* Compare patches and store the one with higher revision */
>> + if ( !patch && match_cpu(new_patch) )
>> + patch = new_patch;
>> + else if ( patch && (compare_patch(new_patch, patch) == NEW_UCODE) )
>
>At least the appearance of this is misleading, but unless I'm missing
>something it's actually wrong: You seem to imply that from
>match_cpu(patch1) returning true and compare_patch(patch2, patch1)
>returning true it follows that also match_cpu(patch2) would return
>true. But I don't think that's the case, because of how the "pf" field
>works.
>
>Even in case I've overlooked an implicit match_cpu(new_patch) I'd
>like to ask for this to be clarified, either by changing the code
>structure, or by attaching a suitable comment.
I did take it for granted. Checking whether each patch covers current
signature before the comparison can solve this problem.
>
>> --- a/xen/arch/x86/smpboot.c
>> +++ b/xen/arch/x86/smpboot.c
>> @@ -363,10 +363,7 @@ void start_secondary(void *unused)
>>
>> initialize_cpu_data(cpu);
>>
>> - if ( system_state <= SYS_STATE_smp_boot )
>> - early_microcode_update_cpu(false);
>> - else
>> - microcode_resume_cpu();
>> + early_microcode_update_cpu();
>
>I'm struggling to understand how you get away without the "false"
>argument that was passed here before. You look to now be calling
>->start_update() unconditionally (so long as the hook is not NULL),
>which I don't think is correct. This should be called only once by
>the CPU _leading_ an update (the BSP during boot, and the CPU
>the hypercall gets invoked on (or the first CPU an update gets
>issued one) for a late update. Am I missing something?
BSP and APs call different functions, early_microcode_parse_and_update_cpu()
and early_microcode_update_cpu() respectively. The latter won't call
->start_update().
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 03/10] microcode: introduce a global cache of ucode patch
2019-06-10 5:33 ` Chao Gao
@ 2019-06-11 6:50 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-11 6:50 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 10.06.19 at 07:33, <chao.gao@intel.com> wrote:
> On Tue, Jun 04, 2019 at 09:03:20AM -0600, Jan Beulich wrote:
>>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>>> +static void free_patch(struct microcode_patch *microcode_patch)
>>> +{
>>> + struct microcode_amd *mc_amd = microcode_patch->mc_amd;
>>> +
>>> + xfree(mc_amd->equiv_cpu_table);
>>> + xfree(mc_amd->mpb);
>>> + xfree(mc_amd);
>>> + xfree(microcode_patch);
>>
>>I think I said so before: Freeing of the generic wrapper struct
>>would probably better be placed in generic code.
>
> Do you mean something as shown below:
>
> /* in generic code */
>
> struct microcode_patch {
> union {
> struct microcode_intel *mc_intel;
> struct microcode_amd *mc_amd;
> void *mc;
> };
> };
>
> void microcode_free_patch(struct microcode_patch *microcode_patch)
> {
> microcode_ops->free_patch(microcode_patch->mc);
> xfree(microcode_patch);
> }
>
> /* in vendor-specific (AMD) code */
>
> static void free_patch(void *mc)
> {
> struct microcode_amd *mc_amd = mc;
>
> xfree(mc_amd->equiv_cpu_table);
> xfree(mc_amd->mpb);
> xfree(mc_amd);
> }
Something along these lines, yes. Whether you do as above or
whether instead you continue to pass struct microcode_patch *
is secondary (and really up to you). Perhaps the above the
(slightly) better.
>>> @@ -497,7 +558,20 @@ static int cpu_request_microcode(unsigned int cpu, const void *buf,
>>> while ( (error = get_ucode_from_buffer_amd(mc_amd, buf, bufsize,
>>> &offset)) == 0 )
>>> {
>>> - if ( microcode_fits(mc_amd, cpu) )
>>> + struct microcode_patch *new_patch = alloc_microcode_patch(mc_amd);
>>> +
>>> + if ( IS_ERR(new_patch) )
>>> + {
>>> + error = PTR_ERR(new_patch);
>>> + break;
>>> + }
>>> +
>>> + if ( match_cpu(new_patch) )
>>> + microcode_update_cache(new_patch);
>>> + else
>>> + free_patch(new_patch);
>>
>>Why do you re-do what microcode_update_cache() already does?
>>It calls ->match_cpu() and ->free_patch() all by itself. It looks as
>>if it would need to gain one more ->free_patch() invocation though.
>>
>
> Will remove both invocations of match_cpu().
>
> To support the case (the broken bios) you described, a patch which
> needs to be stored isn't necessary to be newer than the microcode loaded
> to current CPU. As long as the processor's signature is covered by the
> patch, we will store the patch regardless the revision number.
Well, if so, then fine. I did get the impression that successful application
is a required pre-condition for storing.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode()
2019-06-11 3:32 ` Chao Gao
@ 2019-06-11 7:08 ` Jan Beulich
2019-06-11 8:53 ` Chao Gao
0 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-11 7:08 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 11.06.19 at 05:32, <chao.gao@intel.com> wrote:
> On Wed, Jun 05, 2019 at 06:37:27AM -0600, Jan Beulich wrote:
>>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>>> During late microcode update, apply_microcode() is invoked in
>>> cpu_request_microcode(). To make late microcode update more reliable,
>>> we want to put the apply_microcode() into stop_machine context. So
>>> we split out it from cpu_request_microcode(). As a consequence,
>>> apply_microcode() should be invoked explicitly in the common code.
>>>
>>> Previously, apply_microcode() gets the microcode patch to be applied from
>>> the microcode cache. Now, the patch is passed as a function argument and
>>> a patch is cached for cpu-hotplug and cpu resuming, only after it has
>>> been loaded to a cpu without any error. As a consequence, the
>>> 'match_cpu' check in microcode_update_cache is removed, which otherwise
>>> would fail.
>>
>>The "only after it has been loaded to a cpu without any error" is a
>>problem, precisely for the case where ucode on the different cores
>>is not in sync initially. I would actually like to put up this question:
>>When a core has no ucode loaded at all yet and only strictly older
>>(than loaded on some other cores) ucode is found to be available,
>>whether then it wouldn't still be better to apply that ucode to
>>_at least_ the cores that have none loaded yet.
>
> Yes, it is better for this special case. And I agree to support this case.
>
> This in v7, a patch is loaded only if its revision is newer than that
> loaded to current CPU. And it is stored only if it has been loaded
> successfully. But, as you described, a broken bios might puts the system
> in an inconsistent state (multiple microcode revision in the system) and
> furthermore in this case, if no or an older microcode update is
> provided, early loading cannot get the system into sane state. So for
> both early and late microcode loading, we could face a situation that
> the patch to be loaded has equal or old revision than microcode of some
> CPUs.
>
> Changes I plan to make in next version are:
> 1. For early microcode, a patch would be stored if it covers current CPU's
> signature. All CPUs would try to load from the cache.
> 2. For late microcode, a patch is loaded only if its revision is newer than
> *the patch cached*. And it is stored only if has been loaded without an
> "EIO" error.
> 3. Cache replacement remains the same.
Why the difference between early and late loading?
> But it is a temperary solution, especially for CSPs. A better way
> might be getting the newest ucode or upgrading to the newest bios,
> even downgrading the bios to an older version which wouldn't put
> the system into "insane" state.
On the quoted system, all BIOS versions I've ever been provided
had the same odd behavior.
>>> +static int microcode_update_cpu(struct microcode_patch *patch)
>>> {
>>> - int err;
>>> - struct cpu_signature *sig = &this_cpu(cpu_sig);
>>> + int ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
>>>
>>> - if ( !microcode_ops )
>>> - return 0;
>>> + if ( unlikely(ret) )
>>> + return ret;
>>>
>>> spin_lock(µcode_mutex);
>>>
>>> - err = microcode_ops->collect_cpu_info(sig);
>>> - if ( likely(!err) )
>>> - err = microcode_ops->apply_microcode();
>>> - spin_unlock(µcode_mutex);
>>> + if ( patch )
>>> + {
>>> + /*
>>> + * If a patch is specified, it should has newer revision than
>>> + * that of the patch cached.
>>> + */
>>> + if ( microcode_cache &&
>>> + microcode_ops->compare_patch(patch, microcode_cache) != NEW_UCODE )
>>> + {
>>> + spin_unlock(µcode_mutex);
>>> + return -EINVAL;
>>> + }
>>>
>>> - return err;
>>> -}
>>> + ret = microcode_ops->apply_microcode(patch);
>>
>>There's no printk() here but ...
>>
>>> + }
>>> + else if ( microcode_cache )
>>> + {
>>> + ret = microcode_ops->apply_microcode(microcode_cache);
>>> + if ( ret == -EIO )
>>> + printk("Update failed. Reboot needed\n");
>>
>>... you emit a log message here. Why the difference? And wouldn't
>>it be better to have just a single call to ->apply anyway, by simply
>>assigning "microcode_cache" to "patch" and moving the call a little
>>further down?
>
> -EIO means we loaded the patch but the revision didn't change. This
> error code indicates an error in the patch. It is unlikely to happen.
> And in this v7, a patch is stored after being loading successfully
> on a CPU. To simplify handling of load failure and avoid cleanup to the
> global cache (if a patch has a success rate, we need to consider which
> one to save when we have a new patch with 95% rate and an old one with
> 100% rate, it would be really complex), we assume that loading the cache
> (being loaded successfully on a CPU) shouldn't return EIO. Otherwise,
> an error message is prompted.
But then the log message itself needs to be more specific. That'll
then either allow to understand why one is wanted here but not
elsewhere, or additionally a comment should be attached.
>>> --- a/xen/arch/x86/smpboot.c
>>> +++ b/xen/arch/x86/smpboot.c
>>> @@ -363,10 +363,7 @@ void start_secondary(void *unused)
>>>
>>> initialize_cpu_data(cpu);
>>>
>>> - if ( system_state <= SYS_STATE_smp_boot )
>>> - early_microcode_update_cpu(false);
>>> - else
>>> - microcode_resume_cpu();
>>> + early_microcode_update_cpu();
>>
>>I'm struggling to understand how you get away without the "false"
>>argument that was passed here before. You look to now be calling
>>->start_update() unconditionally (so long as the hook is not NULL),
>>which I don't think is correct. This should be called only once by
>>the CPU _leading_ an update (the BSP during boot, and the CPU
>>the hypercall gets invoked on (or the first CPU an update gets
>>issued one) for a late update. Am I missing something?
>
> BSP and APs call different functions, early_microcode_parse_and_update_cpu()
> and early_microcode_update_cpu() respectively. The latter won't call
> ->start_update().
Oh, I see - I'm sorry. I've been mislead by the patch context
indicators. But still, by their names there's not supposed to be
such a difference. I wonder whether we wouldn't better stick
to just one function (early_microcode_update_cpu()), having
it take a boolean as is the case now. That would control both
parsing and ->start_update() invocation.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode()
2019-06-11 7:08 ` Jan Beulich
@ 2019-06-11 8:53 ` Chao Gao
2019-06-11 9:15 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Chao Gao @ 2019-06-11 8:53 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Tue, Jun 11, 2019 at 01:08:36AM -0600, Jan Beulich wrote:
>>>> On 11.06.19 at 05:32, <chao.gao@intel.com> wrote:
>> On Wed, Jun 05, 2019 at 06:37:27AM -0600, Jan Beulich wrote:
>>>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>>>> During late microcode update, apply_microcode() is invoked in
>>>> cpu_request_microcode(). To make late microcode update more reliable,
>>>> we want to put the apply_microcode() into stop_machine context. So
>>>> we split out it from cpu_request_microcode(). As a consequence,
>>>> apply_microcode() should be invoked explicitly in the common code.
>>>>
>>>> Previously, apply_microcode() gets the microcode patch to be applied from
>>>> the microcode cache. Now, the patch is passed as a function argument and
>>>> a patch is cached for cpu-hotplug and cpu resuming, only after it has
>>>> been loaded to a cpu without any error. As a consequence, the
>>>> 'match_cpu' check in microcode_update_cache is removed, which otherwise
>>>> would fail.
>>>
>>>The "only after it has been loaded to a cpu without any error" is a
>>>problem, precisely for the case where ucode on the different cores
>>>is not in sync initially. I would actually like to put up this question:
>>>When a core has no ucode loaded at all yet and only strictly older
>>>(than loaded on some other cores) ucode is found to be available,
>>>whether then it wouldn't still be better to apply that ucode to
>>>_at least_ the cores that have none loaded yet.
>>
>> Yes, it is better for this special case. And I agree to support this case.
>>
>> This in v7, a patch is loaded only if its revision is newer than that
>> loaded to current CPU. And it is stored only if it has been loaded
>> successfully. But, as you described, a broken bios might puts the system
>> in an inconsistent state (multiple microcode revision in the system) and
>> furthermore in this case, if no or an older microcode update is
>> provided, early loading cannot get the system into sane state. So for
>> both early and late microcode loading, we could face a situation that
>> the patch to be loaded has equal or old revision than microcode of some
>> CPUs.
>>
>> Changes I plan to make in next version are:
>> 1. For early microcode, a patch would be stored if it covers current CPU's
>> signature. All CPUs would try to load from the cache.
>> 2. For late microcode, a patch is loaded only if its revision is newer than
>> *the patch cached*. And it is stored only if has been loaded without an
>> "EIO" error.
>> 3. Cache replacement remains the same.
>
>Why the difference between early and late loading?
Storing a patch without loading it is problematic. We need complex logics
to restore the old patch if the current patch is proved to be broken.
I really want to avoid going this way. So for late microcode, we still
stick to the rule: storing a patch only after it has been loaded. For
late loading, we can try to load a patch as long as the patch covers
current cpu signature to avoid missing any possible update. But thanks
to early loading, the oldest microcode revision on all online CPUs
shouldn't be older than the cache. So as an optimization, we initiate an
update system-wide only if the patch's revision is newer than the cache.
For early loading, to avoid discarding a potential useful patch, an
exception is made to store the newest matching patch without loading it
and all CPus try to load the patch. One problem is if a broken patch
with very high revision is provided, any subsequent attempt of late
loading would fail. It is unlikely to happen, so I plan to leave it
aside. Otherwise, we can clean up the cache in microcode_init() if no
cpu has loaded this patch (we need a global variable to track the status).
>
>> But it is a temperary solution, especially for CSPs. A better way
>> might be getting the newest ucode or upgrading to the newest bios,
>> even downgrading the bios to an older version which wouldn't put
>> the system into "insane" state.
>
>On the quoted system, all BIOS versions I've ever been provided
>had the same odd behavior.
>
>>>> +static int microcode_update_cpu(struct microcode_patch *patch)
>>>> {
>>>> - int err;
>>>> - struct cpu_signature *sig = &this_cpu(cpu_sig);
>>>> + int ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
>>>>
>>>> - if ( !microcode_ops )
>>>> - return 0;
>>>> + if ( unlikely(ret) )
>>>> + return ret;
>>>>
>>>> spin_lock(µcode_mutex);
>>>>
>>>> - err = microcode_ops->collect_cpu_info(sig);
>>>> - if ( likely(!err) )
>>>> - err = microcode_ops->apply_microcode();
>>>> - spin_unlock(µcode_mutex);
>>>> + if ( patch )
>>>> + {
>>>> + /*
>>>> + * If a patch is specified, it should has newer revision than
>>>> + * that of the patch cached.
>>>> + */
>>>> + if ( microcode_cache &&
>>>> + microcode_ops->compare_patch(patch, microcode_cache) != NEW_UCODE )
>>>> + {
>>>> + spin_unlock(µcode_mutex);
>>>> + return -EINVAL;
>>>> + }
>>>>
>>>> - return err;
>>>> -}
>>>> + ret = microcode_ops->apply_microcode(patch);
>>>
>>>There's no printk() here but ...
>>>
>>>> + }
>>>> + else if ( microcode_cache )
>>>> + {
>>>> + ret = microcode_ops->apply_microcode(microcode_cache);
>>>> + if ( ret == -EIO )
>>>> + printk("Update failed. Reboot needed\n");
>>>
>>>... you emit a log message here. Why the difference? And wouldn't
>>>it be better to have just a single call to ->apply anyway, by simply
>>>assigning "microcode_cache" to "patch" and moving the call a little
>>>further down?
>>
>> -EIO means we loaded the patch but the revision didn't change. This
>> error code indicates an error in the patch. It is unlikely to happen.
>> And in this v7, a patch is stored after being loading successfully
>> on a CPU. To simplify handling of load failure and avoid cleanup to the
>> global cache (if a patch has a success rate, we need to consider which
>> one to save when we have a new patch with 95% rate and an old one with
>> 100% rate, it would be really complex), we assume that loading the cache
>> (being loaded successfully on a CPU) shouldn't return EIO. Otherwise,
>> an error message is prompted.
>
>But then the log message itself needs to be more specific. That'll
>then either allow to understand why one is wanted here but not
>elsewhere, or additionally a comment should be attached.
>
>>>> --- a/xen/arch/x86/smpboot.c
>>>> +++ b/xen/arch/x86/smpboot.c
>>>> @@ -363,10 +363,7 @@ void start_secondary(void *unused)
>>>>
>>>> initialize_cpu_data(cpu);
>>>>
>>>> - if ( system_state <= SYS_STATE_smp_boot )
>>>> - early_microcode_update_cpu(false);
>>>> - else
>>>> - microcode_resume_cpu();
>>>> + early_microcode_update_cpu();
>>>
>>>I'm struggling to understand how you get away without the "false"
>>>argument that was passed here before. You look to now be calling
>>>->start_update() unconditionally (so long as the hook is not NULL),
>>>which I don't think is correct. This should be called only once by
>>>the CPU _leading_ an update (the BSP during boot, and the CPU
>>>the hypercall gets invoked on (or the first CPU an update gets
>>>issued one) for a late update. Am I missing something?
>>
>> BSP and APs call different functions, early_microcode_parse_and_update_cpu()
>> and early_microcode_update_cpu() respectively. The latter won't call
>> ->start_update().
>
>Oh, I see - I'm sorry. I've been mislead by the patch context
>indicators. But still, by their names there's not supposed to be
>such a difference. I wonder whether we wouldn't better stick
>to just one function (early_microcode_update_cpu()), having
>it take a boolean as is the case now. That would control both
>parsing and ->start_update() invocation.
Will do
Thanks
Chao
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode()
2019-06-11 8:53 ` Chao Gao
@ 2019-06-11 9:15 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-11 9:15 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 11.06.19 at 10:53, <chao.gao@intel.com> wrote:
> On Tue, Jun 11, 2019 at 01:08:36AM -0600, Jan Beulich wrote:
>>>>> On 11.06.19 at 05:32, <chao.gao@intel.com> wrote:
>>> On Wed, Jun 05, 2019 at 06:37:27AM -0600, Jan Beulich wrote:
>>>>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>>>>> During late microcode update, apply_microcode() is invoked in
>>>>> cpu_request_microcode(). To make late microcode update more reliable,
>>>>> we want to put the apply_microcode() into stop_machine context. So
>>>>> we split out it from cpu_request_microcode(). As a consequence,
>>>>> apply_microcode() should be invoked explicitly in the common code.
>>>>>
>>>>> Previously, apply_microcode() gets the microcode patch to be applied from
>>>>> the microcode cache. Now, the patch is passed as a function argument and
>>>>> a patch is cached for cpu-hotplug and cpu resuming, only after it has
>>>>> been loaded to a cpu without any error. As a consequence, the
>>>>> 'match_cpu' check in microcode_update_cache is removed, which otherwise
>>>>> would fail.
>>>>
>>>>The "only after it has been loaded to a cpu without any error" is a
>>>>problem, precisely for the case where ucode on the different cores
>>>>is not in sync initially. I would actually like to put up this question:
>>>>When a core has no ucode loaded at all yet and only strictly older
>>>>(than loaded on some other cores) ucode is found to be available,
>>>>whether then it wouldn't still be better to apply that ucode to
>>>>_at least_ the cores that have none loaded yet.
>>>
>>> Yes, it is better for this special case. And I agree to support this case.
>>>
>>> This in v7, a patch is loaded only if its revision is newer than that
>>> loaded to current CPU. And it is stored only if it has been loaded
>>> successfully. But, as you described, a broken bios might puts the system
>>> in an inconsistent state (multiple microcode revision in the system) and
>>> furthermore in this case, if no or an older microcode update is
>>> provided, early loading cannot get the system into sane state. So for
>>> both early and late microcode loading, we could face a situation that
>>> the patch to be loaded has equal or old revision than microcode of some
>>> CPUs.
>>>
>>> Changes I plan to make in next version are:
>>> 1. For early microcode, a patch would be stored if it covers current CPU's
>>> signature. All CPUs would try to load from the cache.
>>> 2. For late microcode, a patch is loaded only if its revision is newer than
>>> *the patch cached*. And it is stored only if has been loaded without an
>>> "EIO" error.
>>> 3. Cache replacement remains the same.
>>
>>Why the difference between early and late loading?
>
> Storing a patch without loading it is problematic. We need complex logics
> to restore the old patch if the current patch is proved to be broken.
> I really want to avoid going this way. So for late microcode, we still
> stick to the rule: storing a patch only after it has been loaded. For
> late loading, we can try to load a patch as long as the patch covers
> current cpu signature to avoid missing any possible update. But thanks
> to early loading, the oldest microcode revision on all online CPUs
> shouldn't be older than the cache. So as an optimization, we initiate an
> update system-wide only if the patch's revision is newer than the cache.
Well, you seem to be considering only one out of two possible cases.
What if no ucode was loaded early at all? There's no "thanks to early
loading" then.
> For early loading, to avoid discarding a potential useful patch, an
> exception is made to store the newest matching patch without loading it
> and all CPus try to load the patch. One problem is if a broken patch
> with very high revision is provided, any subsequent attempt of late
> loading would fail. It is unlikely to happen, so I plan to leave it
> aside. Otherwise, we can clean up the cache in microcode_init() if no
> cpu has loaded this patch (we need a global variable to track the status).
Yes, if a patch that claims to be newer than what any CPU has
already loaded fails to apply on every CPU, then yes, it surely
should be purged from the cache.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading
2019-06-05 14:09 ` Jan Beulich
@ 2019-06-11 12:36 ` Chao Gao
2019-06-11 12:58 ` Jan Beulich
2019-06-11 15:47 ` Raj, Ashok
0 siblings, 2 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-11 12:36 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Kevin Tian, Ashok Raj, WeiLiu, Andrew Cooper,
Jun Nakajima, xen-devel, tglx, Borislav Petkov, Roger Pau Monne
On Wed, Jun 05, 2019 at 08:09:43AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> This patch ports microcode improvement patches from linux kernel.
>>
>> Before you read any further: the early loading method is still the
>> preferred one and you should always do that. The following patch is
>> improving the late loading mechanism for long running jobs and cloud use
>> cases.
>>
>> Gather all cores and serialize the microcode update on them by doing it
>> one-by-one to make the late update process as reliable as possible and
>> avoid potential issues caused by the microcode update.
>>
>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>> Tested-by: Chao Gao <chao.gao@intel.com>
>> [linux commit: a5321aec6412b20b5ad15db2d6b916c05349dbff]
>> [linux commit: bb8c13d61a629276a162c1d2b1a20a815cbcfbb7]
>> Cc: Kevin Tian <kevin.tian@intel.com>
>> Cc: Jun Nakajima <jun.nakajima@intel.com>
>> Cc: Ashok Raj <ashok.raj@intel.com>
>> Cc: Borislav Petkov <bp@suse.de>
>> Cc: Thomas Gleixner <tglx@linutronix.de>
>> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
>> Cc: Jan Beulich <jbeulich@suse.com>
>> ---
>> Changes in v7:
>> - Check whether 'timeout' is 0 rather than "<=0" since it is unsigned int.
>> - reword the comment above microcode_update_cpu() to clearly state that
>> one thread per core should do the update.
>>
>> Changes in v6:
>> - Use one timeout period for rendezvous stage and another for update stage.
>> - scale time to wait by the number of remaining cpus to respond.
>> It helps to find something wrong earlier and thus we can reboot the
>> system earlier.
>> ---
>> xen/arch/x86/microcode.c | 171 ++++++++++++++++++++++++++++++++++++++++++-----
>> 1 file changed, 155 insertions(+), 16 deletions(-)
>>
>> diff --git a/xen/arch/x86/microcode.c b/xen/arch/x86/microcode.c
>> index 23cf550..f4a417e 100644
>> --- a/xen/arch/x86/microcode.c
>> +++ b/xen/arch/x86/microcode.c
>> @@ -22,6 +22,7 @@
>> */
>>
>> #include <xen/cpu.h>
>> +#include <xen/cpumask.h>
>
>It seems vanishingly unlikely that you would need this explicit #include
>here, but it certainly isn't wrong.
>
>> @@ -270,31 +296,90 @@ bool microcode_update_cache(struct microcode_patch *patch)
>> return true;
>> }
>>
>> -static long do_microcode_update(void *patch)
>> +/* Wait for CPUs to rendezvous with a timeout (us) */
>> +static int wait_for_cpus(atomic_t *cnt, unsigned int expect,
>> + unsigned int timeout)
>> {
>> - int error, cpu;
>> -
>> - error = microcode_update_cpu(patch);
>> - if ( error )
>> + while ( atomic_read(cnt) < expect )
>> {
>> - microcode_ops->free_patch(microcode_cache);
>> - return error;
>> + if ( !timeout )
>> + {
>> + printk("CPU%d: Timeout when waiting for CPUs calling in\n",
>> + smp_processor_id());
>> + return -EBUSY;
>> + }
>> + udelay(1);
>> + timeout--;
>> }
>
>There's no comment here and nothing in the description: I don't
>recall clarification as to whether RDTSC is fine to be issued by a
>thread when ucode is being updated by another thread on the
>same core.
Yes. I think it is fine.
Ashok, could you share your opinion on this question?
>
>> +static int do_microcode_update(void *patch)
>> +{
>> + unsigned int cpu = smp_processor_id();
>> + unsigned int cpu_nr = num_online_cpus();
>> + unsigned int finished;
>> + int ret;
>> + static bool error;
>>
>> - microcode_update_cache(patch);
>> + atomic_inc(&cpu_in);
>> + ret = wait_for_cpus(&cpu_in, cpu_nr, MICROCODE_CALLIN_TIMEOUT_US);
>> + if ( ret )
>> + return ret;
>>
>> - return error;
>> + ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
>> + /*
>> + * Load microcode update on only one logical processor per core.
>> + * Here, among logical processors of a core, the one with the
>> + * lowest thread id is chosen to perform the loading.
>> + */
>> + if ( !ret && (cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu))) )
>
>At the very least it's not obvious whether this hyper-threading-centric
>view ("logical processor") also applies to AMD's compute unit model
>(which reuses cpu_sibling_mask). It does, as the respective MSRs are
>per-compute-unit rather than per-core, but I'd appreciate if the
>wording could be adjusted to explicitly name both cases (multiple
>threads per core and multiple cores per CU).
OK. Will do
>
>> + {
>> + ret = microcode_ops->apply_microcode(patch);
>> + if ( !ret )
>> + atomic_inc(&cpu_updated);
>> + }
>> + /*
>> + * Increase the wait timeout to a safe value here since we're serializing
>
>I'm struggling with the "increase": I don't see anything being increased
>here. You simply use a larger timeout than above.
>
>> + * the microcode update and that could take a while on a large number of
>> + * CPUs. And that is fine as the *actual* timeout will be determined by
>> + * the last CPU finished updating and thus cut short
>> + */
>> + atomic_inc(&cpu_out);
>> + finished = atomic_read(&cpu_out);
>> + while ( !error && finished != cpu_nr )
>> + {
>> + /*
>> + * During each timeout interval, at least a CPU is expected to
>> + * finish its update. Otherwise, something goes wrong.
>> + */
>> + if ( wait_for_cpus(&cpu_out, finished + 1,
>> + MICROCODE_UPDATE_TIMEOUT_US) && !error )
>> + {
>> + error = true;
>> + panic("Timeout when finishing updating microcode (finished %d/%d)",
>> + finished, cpu_nr);
>
>Why the setting of "error" when you panic anyway?
>
>And please use format specifiers matching the types of the
>further arguments (i.e. twice %u here, but please check other
>code as well).
>
>Furthermore (and I'm sure I've given this comment before) if
>you really hit the limit, how many panic() invocations are there
>going to be? You run this function on all CPUs after all.
"error" is to avoid calling of panic() on multiple CPUs simultaneously.
Roger is right: atomic primitives should be used here.
>
>On the whole, taking a 256-thread system as example, you
>allow the whole process to take over 4 min without calling
>panic().
>Leaving aside guests, I don't think Xen itself would
>survive this in all cases. We've found the need to process
>softirqs with far smaller delays, in particular from key handlers
>producing lots of output. At the very least there should be a
>bold warning logged if the system had been in stop-machine
>state for, say, longer than 100ms (value subject to discussion).
>
In theory, if you mean 256 cores, yes. Do you think a configurable and
run-time changeable upper bound for the whole process can address your
concern? The default value for this upper bound can be set to a large
value (for example, 1s * the number of online core) and the admin can
ajust/lower the upper bound according to the way (serial or parallel) to
perform the update and other requirements. Once the upper bound is
reached, we would call panic().
>> + }
>> +
>> + finished = atomic_read(&cpu_out);
>> + }
>> +
>> + /*
>> + * Refresh CPU signature (revision) on threads which didn't call
>> + * apply_microcode().
>> + */
>> + if ( cpu != cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
>> + ret = microcode_ops->collect_cpu_info(&this_cpu(cpu_sig));
>
>Another option would be for the CPU doing the update to simply
>propagate the new value to all its siblings' cpu_sig values.
Will do.
>
>> @@ -337,12 +429,59 @@ int microcode_update(XEN_GUEST_HANDLE_PARAM(const_void) buf, unsigned long len)
>> if ( patch )
>> microcode_ops->free_patch(patch);
>> ret = -EINVAL;
>> - goto free;
>> + goto put;
>> }
>>
>> - ret = continue_hypercall_on_cpu(cpumask_first(&cpu_online_map),
>> - do_microcode_update, patch);
>> + atomic_set(&cpu_in, 0);
>> + atomic_set(&cpu_out, 0);
>> + atomic_set(&cpu_updated, 0);
>> +
>> + /* Calculate the number of online CPU core */
>> + nr_cores = 0;
>> + for_each_online_cpu(cpu)
>> + if ( cpu == cpumask_first(per_cpu(cpu_sibling_mask, cpu)) )
>> + nr_cores++;
>> +
>> + printk(XENLOG_INFO "%d cores are to update their microcode\n", nr_cores);
>> +
>> + /*
>> + * We intend to disable interrupt for long time, which may lead to
>> + * watchdog timeout.
>> + */
>> + watchdog_disable();
>> + /*
>> + * Late loading dance. Why the heavy-handed stop_machine effort?
>> + *
>> + * - HT siblings must be idle and not execute other code while the other
>> + * sibling is loading microcode in order to avoid any negative
>> + * interactions cause by the loading.
>> + *
>> + * - In addition, microcode update on the cores must be serialized until
>> + * this requirement can be relaxed in the future. Right now, this is
>> + * conservative and good.
>> + */
>> + ret = stop_machine_run(do_microcode_update, patch, NR_CPUS);
>> + watchdog_enable();
>> +
>> + if ( atomic_read(&cpu_updated) == nr_cores )
>> + {
>> + spin_lock(µcode_mutex);
>> + microcode_update_cache(patch);
>> + spin_unlock(µcode_mutex);
>> + }
>> + else if ( atomic_read(&cpu_updated) == 0 )
>> + microcode_ops->free_patch(patch);
>> + else
>> + {
>> + printk("Updating microcode succeeded on part of CPUs and failed on\n"
>> + "others due to an unknown reason. A system with different\n"
>> + "microcode revisions is considered unstable. Please reboot and\n"
>> + "do not load the microcode that triggers this warning\n");
>> + microcode_ops->free_patch(patch);
>> + }
>
>As said on an earlier patch, I think the cache can be updated if at
>least one CPU loaded the blob successfully. Additionally I'd like to
>ask that you log the number of successfully updated cores. And
>finally perhaps "differing" instead of "different" and omit "due to
>an unknown reason"?
Will do.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-05 14:53 ` Jan Beulich
@ 2019-06-11 12:46 ` Chao Gao
2019-06-11 13:23 ` Jan Beulich
2019-06-11 16:04 ` Raj, Ashok
0 siblings, 2 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-11 12:46 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> microcode_update_lock is to prevent logic threads of a same core from
>> updating microcode at the same time. But due to using a global lock, it
>> also prevented parallel microcode updating on different cores.
>>
>> Remove this lock in order to update microcode in parallel. It is safe
>> because we have already ensured serialization of sibling threads at the
>> caller side.
>> 1.For late microcode update, do_microcode_update() ensures that only one
>> sibiling thread of a core can update microcode.
>> 2.For microcode update during system startup or CPU-hotplug,
>> microcode_mutex() guarantees update serialization of logical threads.
>> 3.get/put_cpu_bitmaps() prevents the concurrency of CPU-hotplug and
>> late microcode update.
>>
>> Note that printk in apply_microcode() and svm_host_osvm_init() (for AMD
>> only) are still processed sequentially.
>>
>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>
>Reviewed-by: Jan Beulich <jbeulich@suse.com>
Thanks.
>
>> ---
>> Changes in v7:
>> - reworked. Remove complex lock logics introduced in v5 and v6. The microcode
>> patch to be applied is passed as an argument without any global variable. Thus
>> no lock is added to serialize potential readers/writers. Callers of
>> apply_microcode() will guarantee the correctness: the patch poninted by the
>> arguments won't be changed by others.
>
>Much better this way indeed.
>
>> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch *patch)
>>
>> mc_intel = patch->mc_intel;
>>
>> - /* serialize access to the physical write to MSR 0x79 */
>> - spin_lock_irqsave(µcode_update_lock, flags);
>> + BUG_ON(local_irq_is_enabled());
>>
>> /*
>> * Writeback and invalidate caches before updating microcode to avoid
>
>Thinking about it - what happens if we hit an NMI or #MC here?
>watchdog_disable(), a call to which you add in an earlier patch,
>doesn't really suppress the generation of NMIs, it only tells the
>handler not to look at the accumulated statistics.
I think they should be suppressed. Ashok, could you confirm it?
I will figure out how to suppress them in Xen.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading
2019-06-11 12:36 ` Chao Gao
@ 2019-06-11 12:58 ` Jan Beulich
2019-06-11 15:47 ` Raj, Ashok
1 sibling, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-11 12:58 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Kevin Tian, Ashok Raj, WeiLiu, Andrew Cooper,
Jun Nakajima, xen-devel, tglx, Borislav Petkov, Roger Pau Monne
>>> On 11.06.19 at 14:36, <chao.gao@intel.com> wrote:
> On Wed, Jun 05, 2019 at 08:09:43AM -0600, Jan Beulich wrote:
>>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>>On the whole, taking a 256-thread system as example, you
>>allow the whole process to take over 4 min without calling
>>panic().
>>Leaving aside guests, I don't think Xen itself would
>>survive this in all cases. We've found the need to process
>>softirqs with far smaller delays, in particular from key handlers
>>producing lots of output. At the very least there should be a
>>bold warning logged if the system had been in stop-machine
>>state for, say, longer than 100ms (value subject to discussion).
>
> In theory, if you mean 256 cores, yes. Do you think a configurable and
> run-time changeable upper bound for the whole process can address your
> concern? The default value for this upper bound can be set to a large
> value (for example, 1s * the number of online core) and the admin can
> ajust/lower the upper bound according to the way (serial or parallel) to
> perform the update and other requirements. Once the upper bound is
> reached, we would call panic().
Well, a command line option to control the total time until
calling panic() may help, but as you've said in the past: If we
panic anyway, it doesn't matter much what the timeout is. My
point was rather to make explicit that the process may have
completed after a (too) long time. Remember you mean this
late loading to happen with guests running. We should avoid
making the system unstable as much as we can. This includes
this taking long and the completing successfully _as well as_
calling panic().
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 10/10] x86/microcode: always collect_cpu_info() during boot
2019-06-05 15:05 ` Jan Beulich
@ 2019-06-11 12:58 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-11 12:58 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Wed, Jun 05, 2019 at 09:05:49AM -0600, Jan Beulich wrote:
>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>> From: Sergey Dyasli <sergey.dyasli@citrix.com>
>>
>> Currently cpu_sig struct is not updated during boot when either:
>>
>> 1. ucode_scan is set to false (e.g. no "ucode=scan" in cmdline)
>> 2. initrd does not contain a microcode blob
>
>I thought we'd already discussed this - "ucode=<number>" is not
>covered by this.
>
>> These will result in cpu_sig.rev being 0 which affects APIC's
>> check_deadline_errata() and retpoline_safe() functions.
>>
>> Fix this by getting ucode revision early during boot and SMP bring up.
>> While at it.
>
>While at it?
>
>> Signed-off-by: Sergey Dyasli <sergey.dyasli@citrix.com>
>> Signed-off-by: Chao Gao <chao.gao@intel.com>
>> ---
>> changes in v7:
>> - rebase on patch 1~9
>
>From the looks of it this doesn't depend on any of the earlier changes
>(except the ucode_cpu_info -> cpu_sig change), and hence could go
>in right away. Am I overlooking something? If not, all that's needed
>would be clarifications of the description as per above.
I think no. Will send this patch separately.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 10/10] x86/microcode: always collect_cpu_info() during boot
2019-06-05 14:56 ` Roger Pau Monné
@ 2019-06-11 13:02 ` Chao Gao
0 siblings, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-11 13:02 UTC (permalink / raw)
To: Roger Pau Monné
Cc: Sergey Dyasli, Ashok Raj, Wei Liu, Andrew Cooper, Jan Beulich, xen-devel
On Wed, Jun 05, 2019 at 04:56:01PM +0200, Roger Pau Monné wrote:
>On Mon, May 27, 2019 at 04:31:31PM +0800, Chao Gao wrote:
>> From: Sergey Dyasli <sergey.dyasli@citrix.com>
>>
>> Currently cpu_sig struct is not updated during boot when either:
>>
>> 1. ucode_scan is set to false (e.g. no "ucode=scan" in cmdline)
>> 2. initrd does not contain a microcode blob
>>
>> These will result in cpu_sig.rev being 0 which affects APIC's
>> check_deadline_errata() and retpoline_safe() functions.
>>
>> Fix this by getting ucode revision early during boot and SMP bring up.
>> While at it.
>
>I don't understand the last "While at it" sentence. Can it be
>removed?
Yes.
>
>Is this an issue with current code? If so this could be merged ahead of
>the rest of the series, and should likely be patch 1.
>
>OTOH if the issue this patch is fixing is introduced by this series
>please merge the fix with the respective patch that introduced the
>bug.
It is the former. Will send it separately.
Really appreciate your other comments.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-11 12:46 ` Chao Gao
@ 2019-06-11 13:23 ` Jan Beulich
2019-06-11 16:04 ` Raj, Ashok
1 sibling, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-11 13:23 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 11.06.19 at 14:46, <chao.gao@intel.com> wrote:
> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
>>>>> On 27.05.19 at 10:31, <chao.gao@intel.com> wrote:
>>> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch *patch)
>>>
>>> mc_intel = patch->mc_intel;
>>>
>>> - /* serialize access to the physical write to MSR 0x79 */
>>> - spin_lock_irqsave(µcode_update_lock, flags);
>>> + BUG_ON(local_irq_is_enabled());
>>>
>>> /*
>>> * Writeback and invalidate caches before updating microcode to avoid
>>
>>Thinking about it - what happens if we hit an NMI or #MC here?
>>watchdog_disable(), a call to which you add in an earlier patch,
>>doesn't really suppress the generation of NMIs, it only tells the
>>handler not to look at the accumulated statistics.
>
> I think they should be suppressed. Ashok, could you confirm it?
>
> I will figure out how to suppress them in Xen.
Well, afaik suppressing #MC is impossible. CR4.MCE clear leads
to immediate shutdown in case of a machine check, unless I'm
mistaken. The watchdog NMI, otoh, could of course be "properly"
disabled.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading
2019-06-11 12:36 ` Chao Gao
2019-06-11 12:58 ` Jan Beulich
@ 2019-06-11 15:47 ` Raj, Ashok
1 sibling, 0 replies; 64+ messages in thread
From: Raj, Ashok @ 2019-06-11 15:47 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Kevin Tian, Jan Beulich, WeiLiu, Andrew Cooper,
Jun Nakajima, Ashok Raj, xen-devel, tglx, Borislav Petkov,
Roger Pau Monne
Hi Gao, Jan
On Tue, Jun 11, 2019 at 08:36:17PM +0800, Chao Gao wrote:
> On Wed, Jun 05, 2019 at 08:09:43AM -0600, Jan Beulich wrote:
> >
> >There's no comment here and nothing in the description: I don't
> >recall clarification as to whether RDTSC is fine to be issued by a
> >thread when ucode is being updated by another thread on the
> >same core.
>
> Yes. I think it is fine.
>
> Ashok, could you share your opinion on this question?
>
Yes, rdtsc should be fine for other threads to execute while waiting for the
microcode update to complete on others. We do the same in Linux as well.
Cheers,
Ashok
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-11 12:46 ` Chao Gao
2019-06-11 13:23 ` Jan Beulich
@ 2019-06-11 16:04 ` Raj, Ashok
2019-06-12 7:38 ` Jan Beulich
1 sibling, 1 reply; 64+ messages in thread
From: Raj, Ashok @ 2019-06-11 16:04 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, Jan Beulich,
xen-devel, Roger Pau Monne
On Tue, Jun 11, 2019 at 08:46:04PM +0800, Chao Gao wrote:
> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
> >
> >> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch *patch)
> >>
> >> mc_intel = patch->mc_intel;
> >>
> >> - /* serialize access to the physical write to MSR 0x79 */
> >> - spin_lock_irqsave(µcode_update_lock, flags);
> >> + BUG_ON(local_irq_is_enabled());
> >>
> >> /*
> >> * Writeback and invalidate caches before updating microcode to avoid
> >
> >Thinking about it - what happens if we hit an NMI or #MC here?
> >watchdog_disable(), a call to which you add in an earlier patch,
> >doesn't really suppress the generation of NMIs, it only tells the
> >handler not to look at the accumulated statistics.
>
> I think they should be suppressed. Ashok, could you confirm it?
I think the only sources would be the watchdog as you pointed out
which we already touch to keep it from expiring. The perf counters
i'm not an expert in, but i'll check. When we are in stop_machine() type
flow, its not clear if any of those would fire. (I might be wrong, but let
me check).
#MC shouldn't fire once you entered the rendezvous, if it does its usually
fatal like a 3strike or something. Machine is going down at that time
so nothing we could do to handle.
Cheers,
Ashok
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-11 16:04 ` Raj, Ashok
@ 2019-06-12 7:38 ` Jan Beulich
2019-06-13 14:05 ` Chao Gao
0 siblings, 1 reply; 64+ messages in thread
From: Jan Beulich @ 2019-06-12 7:38 UTC (permalink / raw)
To: Ashok Raj
Cc: Sergey Dyasli, WeiLiu, Andrew Cooper, xen-devel, Roger Pau Monne,
Chao Gao
>>> On 11.06.19 at 18:04, <ashok.raj@intel.com> wrote:
> On Tue, Jun 11, 2019 at 08:46:04PM +0800, Chao Gao wrote:
>> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
>> >
>> >> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch
> *patch)
>> >>
>> >> mc_intel = patch->mc_intel;
>> >>
>> >> - /* serialize access to the physical write to MSR 0x79 */
>> >> - spin_lock_irqsave(µcode_update_lock, flags);
>> >> + BUG_ON(local_irq_is_enabled());
>> >>
>> >> /*
>> >> * Writeback and invalidate caches before updating microcode to avoid
>> >
>> >Thinking about it - what happens if we hit an NMI or #MC here?
>> >watchdog_disable(), a call to which you add in an earlier patch,
>> >doesn't really suppress the generation of NMIs, it only tells the
>> >handler not to look at the accumulated statistics.
>>
>> I think they should be suppressed. Ashok, could you confirm it?
>
> I think the only sources would be the watchdog as you pointed out
> which we already touch to keep it from expiring. The perf counters
> i'm not an expert in, but i'll check. When we are in stop_machine() type
> flow, its not clear if any of those would fire. (I might be wrong, but let
> me check).
Well, without disarming the watchdog NMI at the LAPIC / IO-APIC,
how would it _not_ potentially fire?
> #MC shouldn't fire once you entered the rendezvous, if it does its usually
> fatal like a 3strike or something. Machine is going down at that time
> so nothing we could do to handle.
Right - as long as we assume that #MC would be fatal anyway,
there's no point in thinking about ways to suppress it. I guess
it is fatal (almost) always right now, but I don't think it ought to
be. It's just that no-one has the time and environment to make
it actually behave better.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-12 7:38 ` Jan Beulich
@ 2019-06-13 14:05 ` Chao Gao
2019-06-13 14:08 ` Jan Beulich
0 siblings, 1 reply; 64+ messages in thread
From: Chao Gao @ 2019-06-13 14:05 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Wed, Jun 12, 2019 at 01:38:31AM -0600, Jan Beulich wrote:
>>>> On 11.06.19 at 18:04, <ashok.raj@intel.com> wrote:
>> On Tue, Jun 11, 2019 at 08:46:04PM +0800, Chao Gao wrote:
>>> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
>>> >
>>> >> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch
>> *patch)
>>> >>
>>> >> mc_intel = patch->mc_intel;
>>> >>
>>> >> - /* serialize access to the physical write to MSR 0x79 */
>>> >> - spin_lock_irqsave(µcode_update_lock, flags);
>>> >> + BUG_ON(local_irq_is_enabled());
>>> >>
>>> >> /*
>>> >> * Writeback and invalidate caches before updating microcode to avoid
>>> >
>>> >Thinking about it - what happens if we hit an NMI or #MC here?
>>> >watchdog_disable(), a call to which you add in an earlier patch,
>>> >doesn't really suppress the generation of NMIs, it only tells the
>>> >handler not to look at the accumulated statistics.
>>>
>>> I think they should be suppressed. Ashok, could you confirm it?
>>
>> I think the only sources would be the watchdog as you pointed out
>> which we already touch to keep it from expiring. The perf counters
>> i'm not an expert in, but i'll check. When we are in stop_machine() type
>> flow, its not clear if any of those would fire. (I might be wrong, but let
>> me check).
>
>Well, without disarming the watchdog NMI at the LAPIC / IO-APIC,
>how would it _not_ potentially fire?
We plan not to prevent NMI being fired. Instead, if one thread of a core
is updating microcode, other threads of this core would stop in the
handler of NMI until the update completion. Is this approach acceptable?
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-13 14:05 ` Chao Gao
@ 2019-06-13 14:08 ` Jan Beulich
2019-06-13 14:58 ` Chao Gao
2019-06-13 17:47 ` Raj, Ashok
0 siblings, 2 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-13 14:08 UTC (permalink / raw)
To: Chao Gao
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
>>> On 13.06.19 at 16:05, <chao.gao@intel.com> wrote:
> On Wed, Jun 12, 2019 at 01:38:31AM -0600, Jan Beulich wrote:
>>>>> On 11.06.19 at 18:04, <ashok.raj@intel.com> wrote:
>>> On Tue, Jun 11, 2019 at 08:46:04PM +0800, Chao Gao wrote:
>>>> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
>>>> >
>>>> >> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch
>>> *patch)
>>>> >>
>>>> >> mc_intel = patch->mc_intel;
>>>> >>
>>>> >> - /* serialize access to the physical write to MSR 0x79 */
>>>> >> - spin_lock_irqsave(µcode_update_lock, flags);
>>>> >> + BUG_ON(local_irq_is_enabled());
>>>> >>
>>>> >> /*
>>>> >> * Writeback and invalidate caches before updating microcode to avoid
>>>> >
>>>> >Thinking about it - what happens if we hit an NMI or #MC here?
>>>> >watchdog_disable(), a call to which you add in an earlier patch,
>>>> >doesn't really suppress the generation of NMIs, it only tells the
>>>> >handler not to look at the accumulated statistics.
>>>>
>>>> I think they should be suppressed. Ashok, could you confirm it?
>>>
>>> I think the only sources would be the watchdog as you pointed out
>>> which we already touch to keep it from expiring. The perf counters
>>> i'm not an expert in, but i'll check. When we are in stop_machine() type
>>> flow, its not clear if any of those would fire. (I might be wrong, but let
>>> me check).
>>
>>Well, without disarming the watchdog NMI at the LAPIC / IO-APIC,
>>how would it _not_ potentially fire?
>
> We plan not to prevent NMI being fired. Instead, if one thread of a core
> is updating microcode, other threads of this core would stop in the
> handler of NMI until the update completion. Is this approach acceptable?
Well, I have to return the question: It is you who knows what is or
is not acceptable while an ucode update is in progress. In particular
it obviously matters how much ucode is involved in the delivery of
an NMI (and in allowing the handler to get to the point where you'd
"stop" it).
If the approach you suggest is fine for the NMI case, I'd then wonder
if it couldn't also be used for the #MC one.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-13 14:08 ` Jan Beulich
@ 2019-06-13 14:58 ` Chao Gao
2019-06-13 17:47 ` Raj, Ashok
1 sibling, 0 replies; 64+ messages in thread
From: Chao Gao @ 2019-06-13 14:58 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Roger Pau Monne
On Thu, Jun 13, 2019 at 08:08:46AM -0600, Jan Beulich wrote:
>>>> On 13.06.19 at 16:05, <chao.gao@intel.com> wrote:
>> On Wed, Jun 12, 2019 at 01:38:31AM -0600, Jan Beulich wrote:
>>>>>> On 11.06.19 at 18:04, <ashok.raj@intel.com> wrote:
>>>> On Tue, Jun 11, 2019 at 08:46:04PM +0800, Chao Gao wrote:
>>>>> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
>>>>> >
>>>>> >> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch
>>>> *patch)
>>>>> >>
>>>>> >> mc_intel = patch->mc_intel;
>>>>> >>
>>>>> >> - /* serialize access to the physical write to MSR 0x79 */
>>>>> >> - spin_lock_irqsave(µcode_update_lock, flags);
>>>>> >> + BUG_ON(local_irq_is_enabled());
>>>>> >>
>>>>> >> /*
>>>>> >> * Writeback and invalidate caches before updating microcode to avoid
>>>>> >
>>>>> >Thinking about it - what happens if we hit an NMI or #MC here?
>>>>> >watchdog_disable(), a call to which you add in an earlier patch,
>>>>> >doesn't really suppress the generation of NMIs, it only tells the
>>>>> >handler not to look at the accumulated statistics.
>>>>>
>>>>> I think they should be suppressed. Ashok, could you confirm it?
>>>>
>>>> I think the only sources would be the watchdog as you pointed out
>>>> which we already touch to keep it from expiring. The perf counters
>>>> i'm not an expert in, but i'll check. When we are in stop_machine() type
>>>> flow, its not clear if any of those would fire. (I might be wrong, but let
>>>> me check).
>>>
>>>Well, without disarming the watchdog NMI at the LAPIC / IO-APIC,
>>>how would it _not_ potentially fire?
>>
>> We plan not to prevent NMI being fired. Instead, if one thread of a core
>> is updating microcode, other threads of this core would stop in the
>> handler of NMI until the update completion. Is this approach acceptable?
>
>Well, I have to return the question: It is you who knows what is or
>is not acceptable while an ucode update is in progress. In particular
>it obviously matters how much ucode is involved in the delivery of
>an NMI (and in allowing the handler to get to the point where you'd
>"stop" it).
>
>If the approach you suggest is fine for the NMI case,
Yes. It is fine. It is a suggestion from Ashok and what he is working
on in linux kernel. I just wanted to make sure you didn't oppose this
approach in Xen (considering disarming watchdog NMI might be an
alternative).
>I'd then wonder if it couldn't also be used for the #MC one.
I think no much pratical value for #MC because we still need to wait for
the callin of all threads. But as you and Ashok said, #MC is usually
fatal and machine goes down anyway.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-13 14:08 ` Jan Beulich
2019-06-13 14:58 ` Chao Gao
@ 2019-06-13 17:47 ` Raj, Ashok
2019-06-14 8:58 ` Jan Beulich
1 sibling, 1 reply; 64+ messages in thread
From: Raj, Ashok @ 2019-06-13 17:47 UTC (permalink / raw)
To: Jan Beulich
Cc: Sergey Dyasli, Ashok Raj, WeiLiu, Andrew Cooper, xen-devel,
Chao Gao, Roger Pau Monne
On Thu, Jun 13, 2019 at 08:08:46AM -0600, Jan Beulich wrote:
> >>> On 13.06.19 at 16:05, <chao.gao@intel.com> wrote:
> > On Wed, Jun 12, 2019 at 01:38:31AM -0600, Jan Beulich wrote:
> >>>>> On 11.06.19 at 18:04, <ashok.raj@intel.com> wrote:
> >>> On Tue, Jun 11, 2019 at 08:46:04PM +0800, Chao Gao wrote:
> >>>> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
> >>>> >
> >>>> >> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch
> >>> *patch)
> >>>> >>
> >>>> >> mc_intel = patch->mc_intel;
> >>>> >>
> >>>> >> - /* serialize access to the physical write to MSR 0x79 */
> >>>> >> - spin_lock_irqsave(µcode_update_lock, flags);
> >>>> >> + BUG_ON(local_irq_is_enabled());
> >>>> >>
> >>>> >> /*
> >>>> >> * Writeback and invalidate caches before updating microcode to avoid
> >>>> >
> >>>> >Thinking about it - what happens if we hit an NMI or #MC here?
> >>>> >watchdog_disable(), a call to which you add in an earlier patch,
> >>>> >doesn't really suppress the generation of NMIs, it only tells the
> >>>> >handler not to look at the accumulated statistics.
> >>>>
> >>>> I think they should be suppressed. Ashok, could you confirm it?
> >>>
> >>> I think the only sources would be the watchdog as you pointed out
> >>> which we already touch to keep it from expiring. The perf counters
> >>> i'm not an expert in, but i'll check. When we are in stop_machine() type
> >>> flow, its not clear if any of those would fire. (I might be wrong, but let
> >>> me check).
> >>
> >>Well, without disarming the watchdog NMI at the LAPIC / IO-APIC,
> >>how would it _not_ potentially fire?
> >
> > We plan not to prevent NMI being fired. Instead, if one thread of a core
> > is updating microcode, other threads of this core would stop in the
> > handler of NMI until the update completion. Is this approach acceptable?
>
> Well, I have to return the question: It is you who knows what is or
> is not acceptable while an ucode update is in progress. In particular
> it obviously matters how much ucode is involved in the delivery of
> an NMI (and in allowing the handler to get to the point where you'd
> "stop" it).
>
> If the approach you suggest is fine for the NMI case, I'd then wonder
> if it couldn't also be used for the #MC one.
Architecturally only one #MC can be active in the system. If a new #MC
condition happens when MCG_STATUS.MCIP is already set, that would cause spontaneous
shutdown.
If another NMI arrives on the CPU doing the wrmsr, it will be pended
in the lapic and delivered after the wrmsr returns. wrmsr flow
can't be interrupted.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
* Re: [Xen-devel] [PATCH v7 09/10] microcode: remove microcode_update_lock
2019-06-13 17:47 ` Raj, Ashok
@ 2019-06-14 8:58 ` Jan Beulich
0 siblings, 0 replies; 64+ messages in thread
From: Jan Beulich @ 2019-06-14 8:58 UTC (permalink / raw)
To: Ashok Raj
Cc: Sergey Dyasli, WeiLiu, Andrew Cooper, xen-devel, Roger Pau Monne,
Chao Gao
>>> On 13.06.19 at 19:47, <ashok.raj@intel.com> wrote:
> On Thu, Jun 13, 2019 at 08:08:46AM -0600, Jan Beulich wrote:
>> >>> On 13.06.19 at 16:05, <chao.gao@intel.com> wrote:
>> > On Wed, Jun 12, 2019 at 01:38:31AM -0600, Jan Beulich wrote:
>> >>>>> On 11.06.19 at 18:04, <ashok.raj@intel.com> wrote:
>> >>> On Tue, Jun 11, 2019 at 08:46:04PM +0800, Chao Gao wrote:
>> >>>> On Wed, Jun 05, 2019 at 08:53:46AM -0600, Jan Beulich wrote:
>> >>>> >
>> >>>> >> @@ -307,8 +303,7 @@ static int apply_microcode(const struct microcode_patch
>
>> >>> *patch)
>> >>>> >>
>> >>>> >> mc_intel = patch->mc_intel;
>> >>>> >>
>> >>>> >> - /* serialize access to the physical write to MSR 0x79 */
>> >>>> >> - spin_lock_irqsave(µcode_update_lock, flags);
>> >>>> >> + BUG_ON(local_irq_is_enabled());
>> >>>> >>
>> >>>> >> /*
>> >>>> >> * Writeback and invalidate caches before updating microcode to avoid
>> >>>> >
>> >>>> >Thinking about it - what happens if we hit an NMI or #MC here?
>> >>>> >watchdog_disable(), a call to which you add in an earlier patch,
>> >>>> >doesn't really suppress the generation of NMIs, it only tells the
>> >>>> >handler not to look at the accumulated statistics.
>> >>>>
>> >>>> I think they should be suppressed. Ashok, could you confirm it?
>> >>>
>> >>> I think the only sources would be the watchdog as you pointed out
>> >>> which we already touch to keep it from expiring. The perf counters
>> >>> i'm not an expert in, but i'll check. When we are in stop_machine() type
>> >>> flow, its not clear if any of those would fire. (I might be wrong, but let
>> >>> me check).
>> >>
>> >>Well, without disarming the watchdog NMI at the LAPIC / IO-APIC,
>> >>how would it _not_ potentially fire?
>> >
>> > We plan not to prevent NMI being fired. Instead, if one thread of a core
>> > is updating microcode, other threads of this core would stop in the
>> > handler of NMI until the update completion. Is this approach acceptable?
>>
>> Well, I have to return the question: It is you who knows what is or
>> is not acceptable while an ucode update is in progress. In particular
>> it obviously matters how much ucode is involved in the delivery of
>> an NMI (and in allowing the handler to get to the point where you'd
>> "stop" it).
>>
>> If the approach you suggest is fine for the NMI case, I'd then wonder
>> if it couldn't also be used for the #MC one.
>
> Architecturally only one #MC can be active in the system. If a new #MC
> condition happens when MCG_STATUS.MCIP is already set, that would cause
> spontaneous
> shutdown.
That's understood.
> If another NMI arrives on the CPU doing the wrmsr, it will be pended
> in the lapic and delivered after the wrmsr returns. wrmsr flow
> can't be interrupted.
Of course.
Neither part of your response is an argument against adding the same
"defense" to the #MC handler, though. While likely #MC will be fatal
to the system anyway, we should try to avoid making things worse
when we can.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 64+ messages in thread
end of thread, other threads:[~2019-06-14 8:59 UTC | newest]
Thread overview: 64+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-05-27 8:31 [PATCH v7 00/10] improve late microcode loading Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-05-27 8:31 ` [PATCH v7 01/10] misc/xen-ucode: Upload a microcode blob to the hypervisor Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-04 16:14 ` Andrew Cooper
2019-06-04 16:23 ` Jan Beulich
2019-06-06 2:29 ` Chao Gao
2019-05-27 8:31 ` [PATCH v7 02/10] microcode/intel: extend microcode_update_match() Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-04 14:39 ` Jan Beulich
2019-06-05 13:22 ` Roger Pau Monné
2019-06-05 14:16 ` Jan Beulich
2019-06-06 8:26 ` Chao Gao
2019-06-06 9:01 ` Jan Beulich
2019-05-27 8:31 ` [PATCH v7 03/10] microcode: introduce a global cache of ucode patch Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-04 15:03 ` Jan Beulich
2019-06-10 5:33 ` Chao Gao
2019-06-11 6:50 ` Jan Beulich
2019-05-27 8:31 ` [PATCH v7 04/10] microcode: remove struct ucode_cpu_info Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-04 15:13 ` Jan Beulich
2019-06-10 7:19 ` Chao Gao
2019-05-27 8:31 ` [PATCH v7 05/10] microcode: remove pointless 'cpu' parameter Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-04 15:29 ` Jan Beulich
2019-06-10 7:31 ` Chao Gao
2019-05-27 8:31 ` [PATCH v7 06/10] microcode: split out apply_microcode() from cpu_request_microcode() Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-05 12:37 ` Jan Beulich
2019-06-11 3:32 ` Chao Gao
2019-06-11 7:08 ` Jan Beulich
2019-06-11 8:53 ` Chao Gao
2019-06-11 9:15 ` Jan Beulich
2019-05-27 8:31 ` [PATCH v7 07/10] microcode/intel: Writeback and invalidate caches before updating microcode Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-05 13:20 ` Jan Beulich
2019-05-27 8:31 ` [PATCH v7 08/10] x86/microcode: Synchronize late microcode loading Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-05 14:09 ` Jan Beulich
2019-06-11 12:36 ` Chao Gao
2019-06-11 12:58 ` Jan Beulich
2019-06-11 15:47 ` Raj, Ashok
2019-06-05 14:42 ` Roger Pau Monné
2019-05-27 8:31 ` [PATCH v7 09/10] microcode: remove microcode_update_lock Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-05 14:52 ` Roger Pau Monné
2019-06-05 15:15 ` Jan Beulich
2019-06-05 14:53 ` Jan Beulich
2019-06-11 12:46 ` Chao Gao
2019-06-11 13:23 ` Jan Beulich
2019-06-11 16:04 ` Raj, Ashok
2019-06-12 7:38 ` Jan Beulich
2019-06-13 14:05 ` Chao Gao
2019-06-13 14:08 ` Jan Beulich
2019-06-13 14:58 ` Chao Gao
2019-06-13 17:47 ` Raj, Ashok
2019-06-14 8:58 ` Jan Beulich
2019-05-27 8:31 ` [PATCH v7 10/10] x86/microcode: always collect_cpu_info() during boot Chao Gao
2019-05-27 8:31 ` [Xen-devel] " Chao Gao
2019-06-05 14:56 ` Roger Pau Monné
2019-06-11 13:02 ` Chao Gao
2019-06-05 15:05 ` Jan Beulich
2019-06-11 12:58 ` Chao Gao
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.