In the ARMv8 platform, the CPU error types are synchronous external abort(SEA) and SError Interrupt (SEI). If exception happens in guest, sometimes it's better for guest to perform the recovery, because host does not know the detailed information of guest. For example, if an exception happens in a user-space application within guest, host does not know which application encounters errors. For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify userspace. After user space gets the notification, it will record the CPER into guest GHES buffer and inject an exception or IRQ into guest. In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we will treat it as a synchronous exception, and notify guest with ARMv8 SEA notification type after recording CPER into guest. This series of patches are based on Qemu 4.2, which include two parts: 1. Generate APEI/GHES table. 2. Handle the SIGBUS signal, record the CPER in runtime and fill it into guest memory, then notify guest according to the type of SIGBUS. The whole solution was suggested by James(james.morse@arm.com); The solution of APEI section was suggested by Laszlo(lersek@redhat.com). Show some discussions in [1]. This series of patches have already been tested on ARM64 platform with RAS feature enabled: Show the APEI part verification result in [2]. Show the BUS_MCEERR_AR SIGBUS handling verification result in [3]. --- Change since v21: 1. Make the user-facing 'ras' option description more clearly to address Peter's comments. 2. Update the doc description in "docs/specs/acpi_hest_ghes.rst" 3. Split HEST/GHES patches to more patches to make the review easily 4. Using source_id to index the location to save the CPER. 5. Optimize and simplify the logic to build HEST/GHES table to address Igor/Michael/Beata comments. 6. make ghes_addr_le a part of GED device. Change since v20: 1. Move some implementation details from acpi_ghes.h to acpi_ghes.c 2. Add the reviewers for the ACPI/APEI/GHES part Change since v19: 1. Fix clang compile error 2. Fix sphinx build error Change since v18: 1. Fix some code-style and typo/grammar problems. 2. Remove no_ras in the VirtMachineClass struct. 3. Convert documentation to rst format. 4. Simplize the code and add comments for some magic value. 5. Move kvm_inject_arm_sea() function into the patch where it's used. 6. Register the reset handler(kvm_unpoison_all()) in the kvm_init() function. Change since v17: 1. Improve some commit messages and comments. 2. Fix some code-style problems. 3. Add a *ras* machine option. 4. Move HEST/GHES related structures and macros into "hw/acpi/acpi_ghes.*". 5. Move HWPoison page functions into "include/sysemu/kvm_int.h". 6. Fix some bugs. 7. Improve the design document. Change since v16: 1. check whether ACPI table is enabled when handling the memory error in the SIGBUS handler. Change since v15: 1. Add a doc-comment in the proper format for 'include/exec/ram_addr.h' 2. Remove write_part_cpustate_to_list() because there is another bug fix patch has been merged "arm: Allow system registers for KVM guests to be changed by QEMU code" 3. Add some comments for kvm_inject_arm_sea() in 'target/arm/kvm64.c' 4. Compare the arm_current_el() return value to 0,1,2,3, not to PSTATE_MODE_* constants. 5. Change the RAS support wasn't introduced before 4.1 QEMU version. 6. Move the no_ras flag patch to begin in this series Change since v14: 1. Remove the BUS_MCEERR_AO handling logic because this asynchronous signal was masked by main thread 2. Address some Igor Mammedov's comments(ACPI part) 1) change the comments for the enum AcpiHestNotifyType definition and remove ditto in patch 1 2) change some patch commit messages and separate "APEI GHES table generation" patch to more patches. 3. Address some peter's comments(arm64 Synchronous External Abort injection) 1) change some code notes 2) using arm_current_el() for current EL 2) use the helper functions for those (syn_data_abort_*). Change since v13: 1. Move the patches that set guest ESR and inject virtual SError out of this series 2. Clean and optimize the APEI part patches 3. Update the commit messages and add some comments for the code Change since v12: 1. Address Paolo's comments to move HWPoisonPage definition to accel/kvm/kvm-all.c 2. Only call kvm_cpu_synchronize_state() when get the BUS_MCEERR_AR signal 3. Only add and enable GPIO-Signal and ARMv8 SEA two hardware error sources 4. Address Michael's comments to not sync SPDX from Linux kernel header file Change since v11: Address James's comments(james.morse@arm.com) 1. Check whether KVM has the capability to to set ESR instead of detecting host CPU RAS capability 2. For SIGBUS_MCEERR_AR SIGBUS, use Synchronous-External-Abort(SEA) notification type for SIGBUS_MCEERR_AO SIGBUS, use GPIO-Signal notification Address Shannon's comments(for ACPI part): 1. Unify hest_ghes.c and hest_ghes.h license declaration 2. Remove unnecessary including "qmp-commands.h" in hest_ghes.c 3. Unconditionally add guest APEI table based on James's comments(james.morse@arm.com) 4. Add a option to virt machine for migration compatibility. On new virt machine it's on by default while off for old ones, we enabled it since 2.12 5. Refer to the ACPI spec version which introduces Hardware Error Notification first time 6. Add ACPI_HEST_NOTIFY_RESERVED notification type Address Igor's comments(for ACPI part): 1. Add doc patch first which will describe how it's supposed to work between QEMU/firmware/guest OS with expected flows. 2. Move APEI diagrams into doc/spec patch 3. Remove redundant g_malloc in ghes_record_cper() 4. Use build_append_int_noprefix() API to compose whole error status block and whole APEI table, and try to get rid of most structures in patch 1, as they will be left unused after that 5. Reuse something like https://github.com/imammedo/qemu/commit/3d2fd6d13a3ea298d2ee814835495ce6241d085c to build GAS 6. Remove much offsetof() in the function 7. Build independent tables first and only then build dependent tables passing to it pointers to previously build table if necessary. 8. Redefine macro GHES_ACPI_HEST_NOTIFY_RESERVED to ACPI_HEST_ERROR_SOURCE_COUNT to avoid confusion Address Peter Maydell's comments 1. linux-headers is done as a patch of their own created using scripts/update-linux-headers.sh run against a mainline kernel tree 2. Tested whether this patchset builds OK on aarch32 3. Abstract Hwpoison page adding code out properly into a cpu-independent source file from target/i386/kvm.c, such as kvm-all.c 4. Add doc-comment formatted documentation comment for new globally-visible function prototype in a header --- [1]: https://lkml.org/lkml/2017/2/27/246 https://patchwork.kernel.org/patch/9633105/ https://patchwork.kernel.org/patch/9925227/ [2]: Note: the UEFI(QEMU_EFI.fd) is needed if guest want to use ACPI table. After guest boot up, dump the APEI table, then can see the initialized table (1) # iasl -p ./HEST -d /sys/firmware/acpi/tables/HEST (2) # cat HEST.dsl /* * Intel ACPI Component Architecture * AML/ASL+ Disassembler version 20170728 (64-bit version) * Copyright (c) 2000 - 2017 Intel Corporation * * Disassembly of /sys/firmware/acpi/tables/HEST, Mon Sep 5 07:59:17 2016 * * ACPI Data Table [HEST] * * Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue */ .................................................................................. [308h 0776 2] Subtable Type : 000A [Generic Hardware Error Source V2] [30Ah 0778 2] Source Id : 0001 [30Ch 0780 2] Related Source Id : FFFF [30Eh 0782 1] Reserved : 00 [30Fh 0783 1] Enabled : 01 [310h 0784 4] Records To Preallocate : 00000001 [314h 0788 4] Max Sections Per Record : 00000001 [318h 0792 4] Max Raw Data Length : 00001000 [31Ch 0796 12] Error Status Address : [Generic Address Structure] [31Ch 0796 1] Space ID : 00 [SystemMemory] [31Dh 0797 1] Bit Width : 40 [31Eh 0798 1] Bit Offset : 00 [31Fh 0799 1] Encoded Access Width : 04 [QWord Access:64] [320h 0800 8] Address : 00000000785D0040 [328h 0808 28] Notify : [Hardware Error Notification Structure] [328h 0808 1] Notify Type : 08 [SEA] [329h 0809 1] Notify Length : 1C [32Ah 0810 2] Configuration Write Enable : 0000 [32Ch 0812 4] PollInterval : 00000000 [330h 0816 4] Vector : 00000000 [334h 0820 4] Polling Threshold Value : 00000000 [338h 0824 4] Polling Threshold Window : 00000000 [33Ch 0828 4] Error Threshold Value : 00000000 [340h 0832 4] Error Threshold Window : 00000000 [344h 0836 4] Error Status Block Length : 00001000 [348h 0840 12] Read Ack Register : [Generic Address Structure] [348h 0840 1] Space ID : 00 [SystemMemory] [349h 0841 1] Bit Width : 40 [34Ah 0842 1] Bit Offset : 00 [34Bh 0843 1] Encoded Access Width : 04 [QWord Access:64] [34Ch 0844 8] Address : 00000000785D0098 [354h 0852 8] Read Ack Preserve : 00000000FFFFFFFE [35Ch 0860 8] Read Ack Write : 0000000000000001 ..................................................................................... (3) After a synchronous external abort(SEA) happen, Qemu receive a SIGBUS and filled the CPER into guest GHES memory. For example, according to above table, the address that contains the physical address of a block of memory that holds the error status data for this abort is 0x00000000785D0040 (4) the address for SEA notification error source is 0x785d80b0 (qemu) xp /1 0x00000000785D0040 00000000785d0040: 0x785d80b0 (5) check the content of generic error status block and generic error data entry (qemu) xp /100x 0x785d80b0 00000000785d80b0: 0x00000001 0x00000000 0x00000000 0x00000098 00000000785d80c0: 0x00000000 0xa5bc1114 0x4ede6f64 0x833e63b8 00000000785d80d0: 0xb1837ced 0x00000000 0x00000300 0x00000050 00000000785d80e0: 0x00000000 0x00000000 0x00000000 0x00000000 00000000785d80f0: 0x00000000 0x00000000 0x00000000 0x00000000 00000000785d8100: 0x00000000 0x00000000 0x00000000 0x00004002 (6) check the OSPM's ACK value(for example SEA) /* Before OSPM acknowledges the error, check the ACK value */ (qemu) xp /1 0x00000000785D0098 00000000785d00f0: 0x00000000 /* After OSPM acknowledges the error, check the ACK value, it change to 1 from 0 */ (qemu) xp /1 0x00000000785D0098 00000000785d00f0: 0x00000001 [3]: KVM deliver "BUS_MCEERR_AR" to Qemu, Qemu record the guest CPER and inject synchronous external abort to notify guest, then guest do the recovery. [ 1552.516170] Synchronous External Abort: synchronous external abort (0x92000410) at 0x000000003751c6b4 [ 1553.074073] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 8 [ 1553.081654] {1}[Hardware Error]: event severity: recoverable [ 1554.034191] {1}[Hardware Error]: Error 0, type: recoverable [ 1554.037934] {1}[Hardware Error]: section_type: memory error [ 1554.513261] {1}[Hardware Error]: physical_address: 0x0000000040fa6000 [ 1554.513944] {1}[Hardware Error]: error_type: 0, unknown [ 1555.041451] Memory failure: 0x40fa6: Killing mca-recover:1296 due to hardware memory corruption [ 1555.373116] Memory failure: 0x40fa6: recovery action for dirty LRU page: Recovered Dongjiu Geng (9): hw/arm/virt: Introduce a RAS machine option docs: APEI GHES generation and CPER record description ACPI: Build related register address fields via hardware error fw_cfg blob ACPI: Build Hardware Error Source Table ACPI: Record the Generic Error Status Block address KVM: Move hwpoison page related functions into kvm-all.c ACPI: Record Generic Error Status Block(GESB) table target-arm: kvm64: handle SIGBUS signal from kernel or KVM MAINTAINERS: Add ACPI/HEST/GHES entries MAINTAINERS | 9 + accel/kvm/kvm-all.c | 36 +++ default-configs/arm-softmmu.mak | 1 + docs/specs/acpi_hest_ghes.rst | 110 ++++++++ docs/specs/index.rst | 1 + hw/acpi/Kconfig | 4 + hw/acpi/Makefile.objs | 1 + hw/acpi/aml-build.c | 2 + hw/acpi/generic_event_device.c | 15 +- hw/acpi/ghes.c | 446 +++++++++++++++++++++++++++++++++ hw/arm/virt-acpi-build.c | 19 ++ hw/arm/virt.c | 23 ++ include/hw/acpi/aml-build.h | 1 + include/hw/acpi/generic_event_device.h | 2 + include/hw/acpi/ghes.h | 75 ++++++ include/hw/arm/virt.h | 1 + include/qemu/uuid.h | 5 + include/sysemu/kvm.h | 3 +- include/sysemu/kvm_int.h | 12 + target/arm/cpu.h | 4 + target/arm/helper.c | 2 +- target/arm/internals.h | 5 +- target/arm/kvm64.c | 66 +++++ target/arm/tlb_helper.c | 2 +- target/i386/cpu.h | 2 + target/i386/kvm.c | 36 --- 26 files changed, 840 insertions(+), 43 deletions(-) create mode 100644 docs/specs/acpi_hest_ghes.rst create mode 100644 hw/acpi/ghes.c create mode 100644 include/hw/acpi/ghes.h -- 1.8.3.1
RAS Virtualization feature is not supported now, so add a RAS machine option and disable it by default. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> Acked-by: Xiang Zheng <zhengxiang9@huawei.com> --- hw/arm/virt.c | 23 +++++++++++++++++++++++ include/hw/arm/virt.h | 1 + 2 files changed, 24 insertions(+) diff --git a/hw/arm/virt.c b/hw/arm/virt.c index 39ab5f4..fe2571f 100644 --- a/hw/arm/virt.c +++ b/hw/arm/virt.c @@ -1818,6 +1818,20 @@ static void virt_set_its(Object *obj, bool value, Error **errp) vms->its = value; } +static bool virt_get_ras(Object *obj, Error **errp) +{ + VirtMachineState *vms = VIRT_MACHINE(obj); + + return vms->ras; +} + +static void virt_set_ras(Object *obj, bool value, Error **errp) +{ + VirtMachineState *vms = VIRT_MACHINE(obj); + + vms->ras = value; +} + static char *virt_get_gic_version(Object *obj, Error **errp) { VirtMachineState *vms = VIRT_MACHINE(obj); @@ -2121,6 +2135,15 @@ static void virt_instance_init(Object *obj) "Valid values are none and smmuv3", NULL); + /* Default disallows RAS instantiation */ + vms->ras = false; + object_property_add_bool(obj, "ras", virt_get_ras, + virt_set_ras, NULL); + object_property_set_description(obj, "ras", + "Set on/off to enable/disable reporting host memory errors " + "to a KVM guest using ACPI and guest external abort exceptions", + NULL); + vms->irqmap = a15irqmap; virt_flash_create(vms); diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h index 38f0c33..15057f7 100644 --- a/include/hw/arm/virt.h +++ b/include/hw/arm/virt.h @@ -122,6 +122,7 @@ typedef struct { bool highmem_ecam; bool its; bool virt; + bool ras; int32_t gic_version; VirtIOMMUType iommu; struct arm_boot_info bootinfo; -- 1.8.3.1
Add APEI/GHES detailed design document Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> --- docs/specs/acpi_hest_ghes.rst | 110 ++++++++++++++++++++++++++++++++++++++++++ docs/specs/index.rst | 1 + 2 files changed, 111 insertions(+) create mode 100644 docs/specs/acpi_hest_ghes.rst diff --git a/docs/specs/acpi_hest_ghes.rst b/docs/specs/acpi_hest_ghes.rst new file mode 100644 index 0000000..7a1aa90 --- /dev/null +++ b/docs/specs/acpi_hest_ghes.rst @@ -0,0 +1,110 @@ +APEI tables generating and CPER record +====================================== + +.. + Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. + + This work is licensed under the terms of the GNU GPL, version 2 or later. + See the COPYING file in the top-level directory. + +Design Details +-------------- + +:: + + etc/acpi/tables etc/hardware_errors + ==================== =============================== + + +--------------------------+ +----------------------------+ + | | HEST | +--------->| error_block_address1 |------+ + | +--------------------------+ | +----------------------------+ | + | | GHES1 | | +------->| error_block_address2 |------+-+ + | +--------------------------+ | | +----------------------------+ | | + | | ................. | | | | .............. | | | + | | error_status_address-----+-+ | -----------------------------+ | | + | | ................. | | +--->| error_block_addressN |------+-+---+ + | | read_ack_register--------+-+ | | +----------------------------+ | | | + | | read_ack_preserve | +-+---+--->| read_ack_register1 | | | | + | | read_ack_write | | | +----------------------------+ | | | + + +--------------------------+ | +-+--->| read_ack_register2 | | | | + | | GHES2 | | | | +----------------------------+ | | | + + +--------------------------+ | | | | ............. | | | | + | | ................. | | | | +----------------------------+ | | | + | | error_status_address-----+---+ | | +->| read_ack_registerN | | | | + | | ................. | | | | +----------------------------+ | | | + | | read_ack_register--------+-----+ | | |Generic Error Status Block 1|<-----+ | | + | | read_ack_preserve | | | |-+------------------------+-+ | | + | | read_ack_write | | | | | CPER | | | | + + +--------------------------| | | | | CPER | | | | + | | ............... | | | | | .... | | | | + + +--------------------------+ | | | | CPER | | | | + | | GHESN | | | |-+------------------------+-| | | + + +--------------------------+ | | |Generic Error Status Block 2|<-------+ | + | | ................. | | | |-+------------------------+-+ | + | | error_status_address-----+-------+ | | | CPER | | | + | | ................. | | | | CPER | | | + | | read_ack_register--------+---------+ | | .... | | | + | | read_ack_preserve | | | CPER | | | + | | read_ack_write | +-+------------------------+-+ | + + +--------------------------+ | .......... | | + |----------------------------+ | + |Generic Error Status Block N |<----------+ + |-+-------------------------+-+ + | | CPER | | + | | CPER | | + | | .... | | + | | CPER | | + +-+-------------------------+-+ + + +(1) QEMU generates the ACPI HEST table. This table goes in the current + "etc/acpi/tables" fw_cfg blob. Each error source has different + notification types. + +(2) A new fw_cfg blob called "etc/hardware_errors" is introduced. QEMU + also needs to populate this blob. The "etc/hardware_errors" fw_cfg blob + contains an address registers table and an Error Status Data Block table. + +(3) The address registers table contains N Error Block Address entries + and N Read Ack Register entries. The size for each entry is 8-byte. + The Error Status Data Block table contains N Error Status Data Block + entries. The size for each entry is 4096(0x1000) bytes. The total size + for the "etc/hardware_errors" fw_cfg blob is (N * 8 * 2 + N * 4096) bytes. + N is the number of the kinds of hardware error sources. + +(4) QEMU generates the ACPI linker/loader script for the firmware. The + firmware pre-allocates memory for "etc/acpi/tables", "etc/hardware_errors" + and copies blob contents there. + +(5) QEMU generates N ADD_POINTER commands, which patch addresses in the + "error_status_address" fields of the HEST table with a pointer to the + corresponding "address registers" in the "etc/hardware_errors" blob. + +(6) QEMU generates N ADD_POINTER commands, which patch addresses in the + "read_ack_register" fields of the HEST table with a pointer to the + corresponding "read_ack_register" within the "etc/hardware_errors" blob. + +(7) QEMU generates N ADD_POINTER commands for the firmware, which patch + addresses in the "error_block_address" fields with a pointer to the + respective "Error Status Data Block" in the "etc/hardware_errors" blob. + +(8) QEMU defines a third and write-only fw_cfg blob which is called + "etc/hardware_errors_addr". Through that blob, the firmware can send back + the guest-side allocation addresses to QEMU. The "etc/hardware_errors_addr" + blob contains a 8-byte entry. QEMU generates a single WRITE_POINTER command + for the firmware. The firmware will write back the start address of + "etc/hardware_errors" blob to the fw_cfg file "etc/hardware_errors_addr". + +(9) When QEMU gets a SIGBUS from the kernel, QEMU writes CPER into corresponding + "Error Status Data Block", guest memory, and then injects platform specific + interrupt (in case of arm/virt machine it's Synchronous External Abort) as a + notification which is necessary for notifying the guest. + +(10) This notification (in virtual hardware) will be handled by the guest + kernel, on receiving notification, guest APEI driver could read the CPER error + and take appropriate action. + +(11) kvm_arch_on_sigbus_vcpu() uses source_id as index in "etc/hardware_errors" to + find out "Error Status Data Block" entry corresponding to error source. So supported + source_id values should be assigned here and not be changed afterwards to make sure + that guest will write error into expected "Error Status Data Block" even if guest was + migrated to a newer QEMU. diff --git a/docs/specs/index.rst b/docs/specs/index.rst index 984ba44..3019b9c 100644 --- a/docs/specs/index.rst +++ b/docs/specs/index.rst @@ -13,3 +13,4 @@ Contents: ppc-xive ppc-spapr-xive acpi_hw_reduced_hotplug + acpi_hest_ghes -- 1.8.3.1
This patch builds error_block_address and read_ack_register fields in hardware errors table , the error_block_address points to Generic Error Status Block(GESB) via bios_linker. The max size for one GESB is 0x1000 in bytes, For more detailed information, please refer to document: docs/specs/acpi_hest_ghes.rst Now we only support one Error source, if necessary, we can extend to support more. Suggested-by: Laszlo Ersek <lersek@redhat.com> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> --- default-configs/arm-softmmu.mak | 1 + hw/acpi/Kconfig | 4 ++ hw/acpi/Makefile.objs | 1 + hw/acpi/aml-build.c | 2 + hw/acpi/ghes.c | 94 +++++++++++++++++++++++++++++++++++++++++ hw/arm/virt-acpi-build.c | 6 +++ include/hw/acpi/aml-build.h | 1 + include/hw/acpi/ghes.h | 26 ++++++++++++ 8 files changed, 135 insertions(+) create mode 100644 hw/acpi/ghes.c create mode 100644 include/hw/acpi/ghes.h diff --git a/default-configs/arm-softmmu.mak b/default-configs/arm-softmmu.mak index 1f2e0e7..5722f31 100644 --- a/default-configs/arm-softmmu.mak +++ b/default-configs/arm-softmmu.mak @@ -40,3 +40,4 @@ CONFIG_FSL_IMX25=y CONFIG_FSL_IMX7=y CONFIG_FSL_IMX6UL=y CONFIG_SEMIHOSTING=y +CONFIG_ACPI_APEI=y diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig index 12e3f1e..ed8c34d 100644 --- a/hw/acpi/Kconfig +++ b/hw/acpi/Kconfig @@ -23,6 +23,10 @@ config ACPI_NVDIMM bool depends on ACPI +config ACPI_APEI + bool + depends on ACPI + config ACPI_PCI bool depends on ACPI && PCI diff --git a/hw/acpi/Makefile.objs b/hw/acpi/Makefile.objs index 9925305..7b5db4b 100644 --- a/hw/acpi/Makefile.objs +++ b/hw/acpi/Makefile.objs @@ -5,6 +5,7 @@ common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu_hotplug.o common-obj-$(CONFIG_ACPI_MEMORY_HOTPLUG) += memory_hotplug.o common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu.o common-obj-$(CONFIG_ACPI_NVDIMM) += nvdimm.o +common-obj-$(CONFIG_ACPI_APEI) += ghes.o common-obj-$(CONFIG_ACPI_VMGENID) += vmgenid.o common-obj-$(CONFIG_ACPI_HW_REDUCED) += generic_event_device.o common-obj-$(call lnot,$(CONFIG_ACPI_X86)) += acpi-stub.o diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c index 2c3702b..3681ec6 100644 --- a/hw/acpi/aml-build.c +++ b/hw/acpi/aml-build.c @@ -1578,6 +1578,7 @@ void acpi_build_tables_init(AcpiBuildTables *tables) tables->table_data = g_array_new(false, true /* clear */, 1); tables->tcpalog = g_array_new(false, true /* clear */, 1); tables->vmgenid = g_array_new(false, true /* clear */, 1); + tables->hardware_errors = g_array_new(false, true /* clear */, 1); tables->linker = bios_linker_loader_init(); } @@ -1588,6 +1589,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre) g_array_free(tables->table_data, true); g_array_free(tables->tcpalog, mfre); g_array_free(tables->vmgenid, mfre); + g_array_free(tables->hardware_errors, mfre); } /* diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c new file mode 100644 index 0000000..b7fdbbb --- /dev/null +++ b/hw/acpi/ghes.c @@ -0,0 +1,94 @@ +/* + * Support for generating APEI tables and recording CPER for Guests + * + * Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. + * + * Author: Dongjiu Geng <gengdongjiu@huawei.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + + * You should have received a copy of the GNU General Public License along + * with this program; if not, see <http://www.gnu.org/licenses/>. + */ + +#include "qemu/osdep.h" +#include "hw/acpi/acpi.h" +#include "hw/acpi/ghes.h" +#include "hw/acpi/aml-build.h" +#include "hw/nvram/fw_cfg.h" +#include "sysemu/sysemu.h" +#include "qemu/error-report.h" + +#include "hw/acpi/bios-linker-loader.h" + +#define ACPI_GHES_ERRORS_FW_CFG_FILE "etc/hardware_errors" +#define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr" + +/* The max size in bytes for one error block */ +#define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 + +/* Now only support ARMv8 SEA notification type error source */ +#define ACPI_GHES_ERROR_SOURCE_COUNT 1 + +/* + * Build table for the hardware error fw_cfg blob. + * Initialize "etc/hardware_errors" and "etc/hardware_errors_addr" fw_cfg blobs. + * See docs/specs/acpi_hest_ghes.rst for blobs format. + */ +void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker) +{ + int i, error_status_block_offset; + + /* Build error_block_address */ + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { + build_append_int_noprefix(hardware_errors, 0, sizeof(uint64_t)); + } + + /* Build read_ack_register */ + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { + /* + * Initialize the value of read_ack_register to 1, so GHES can be + * writeable after (re)boot. + * ACPI 6.2: 18.3.2.8 Generic Hardware Error Source version 2 + * (GHESv2 - Type 10) + */ + build_append_int_noprefix(hardware_errors, 1, sizeof(uint64_t)); + } + + /* Generic Error Status Block offset in the hardware error fw_cfg blob */ + error_status_block_offset = hardware_errors->len; + + /* Reserve space for Error Status Data Block */ + acpi_data_push(hardware_errors, + ACPI_GHES_MAX_RAW_DATA_LENGTH * ACPI_GHES_ERROR_SOURCE_COUNT); + + /* Tell guest firmware to place hardware_errors blob into RAM */ + bios_linker_loader_alloc(linker, ACPI_GHES_ERRORS_FW_CFG_FILE, + hardware_errors, 1, false); + + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { + /* + * Tell firmware to patch error_block_address entries to point to + * corresponding "Generic Error Status Block" + */ + bios_linker_loader_add_pointer(linker, + ACPI_GHES_ERRORS_FW_CFG_FILE, sizeof(uint64_t) * i, + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, + error_status_block_offset + i * ACPI_GHES_MAX_RAW_DATA_LENGTH); + } + + /* + * tell firmware to write hardware_errors GPA into + * hardware_errors_addr fw_cfg, once the former has been initialized. + */ + bios_linker_loader_write_pointer(linker, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, + 0, sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); +} diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index bd5f771..6819fcf 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -48,6 +48,7 @@ #include "sysemu/reset.h" #include "kvm_arm.h" #include "migration/vmstate.h" +#include "hw/acpi/ghes.h" #define ARM_SPI_BASE 32 @@ -830,6 +831,11 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) acpi_add_table(table_offsets, tables_blob); build_spcr(tables_blob, tables->linker, vms); + if (vms->ras) { + acpi_add_table(table_offsets, tables_blob); + build_ghes_error_table(tables->hardware_errors, tables->linker); + } + if (ms->numa_state->num_nodes > 0) { acpi_add_table(table_offsets, tables_blob); build_srat(tables_blob, tables->linker, vms); diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h index de4a406..8f13620 100644 --- a/include/hw/acpi/aml-build.h +++ b/include/hw/acpi/aml-build.h @@ -220,6 +220,7 @@ struct AcpiBuildTables { GArray *rsdp; GArray *tcpalog; GArray *vmgenid; + GArray *hardware_errors; BIOSLinker *linker; } AcpiBuildTables; diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h new file mode 100644 index 0000000..3dbda3f --- /dev/null +++ b/include/hw/acpi/ghes.h @@ -0,0 +1,26 @@ +/* + * Support for generating APEI tables and recording CPER for Guests + * + * Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. + * + * Author: Dongjiu Geng <gengdongjiu@huawei.com> + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + + * You should have received a copy of the GNU General Public License along + * with this program; if not, see <http://www.gnu.org/licenses/>. + */ + +#ifndef ACPI_GHES_H +#define ACPI_GHES_H + +void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); +#endif -- 1.8.3.1
This patch builds Hardware Error Source Table(HEST) via fw_cfg blobs. Now it only supports ARMv8 SEA, a type of Generic Hardware Error Source version 2(GHESv2) error source. Afterwards, we can extend the supported types if needed. For the CPER section, currently it is memory section because kernel mainly wants userspace to handle the memory errors. This patch follows the spec ACPI 6.2 to build the Hardware Error Source table. For more detailed information, please refer to document: docs/specs/acpi_hest_ghes.rst build_append_ghes_notify() will help to add Hardware Error Notification to ACPI tables without using packed C structures and avoid endianness issues as API doesn't need explicit conversion. Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Xiang Zheng <zhengxiang9@huawei.com> --- hw/acpi/ghes.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++- hw/arm/virt-acpi-build.c | 2 + include/hw/acpi/ghes.h | 40 ++++++++++++++++ 3 files changed, 159 insertions(+), 1 deletion(-) diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c index b7fdbbb..9d37798 100644 --- a/hw/acpi/ghes.c +++ b/hw/acpi/ghes.c @@ -34,9 +34,42 @@ /* The max size in bytes for one error block */ #define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 - /* Now only support ARMv8 SEA notification type error source */ #define ACPI_GHES_ERROR_SOURCE_COUNT 1 +/* Generic Hardware Error Source version 2 */ +#define ACPI_GHES_SOURCE_GENERIC_ERROR_V2 10 +/* Address offset in Generic Address Structure(GAS) */ +#define GAS_ADDR_OFFSET 4 + +/* + * Hardware Error Notification + * ACPI 4.0: 17.3.2.7 Hardware Error Notification + * Composes dummy Hardware Error Notification descriptor of specified type + */ +static void build_ghes_hw_error_notification(GArray *table, const uint8_t type) +{ + /* Type */ + build_append_int_noprefix(table, type, 1); + /* + * Length: + * Total length of the structure in bytes + */ + build_append_int_noprefix(table, 28, 1); + /* Configuration Write Enable */ + build_append_int_noprefix(table, 0, 2); + /* Poll Interval */ + build_append_int_noprefix(table, 0, 4); + /* Vector */ + build_append_int_noprefix(table, 0, 4); + /* Switch To Polling Threshold Value */ + build_append_int_noprefix(table, 0, 4); + /* Switch To Polling Threshold Window */ + build_append_int_noprefix(table, 0, 4); + /* Error Threshold Value */ + build_append_int_noprefix(table, 0, 4); + /* Error Threshold Window */ + build_append_int_noprefix(table, 0, 4); +} /* * Build table for the hardware error fw_cfg blob. @@ -92,3 +125,86 @@ void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker) bios_linker_loader_write_pointer(linker, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, 0, sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); } + +/* Build Generic Hardware Error Source version 2 (GHESv2) */ +static void build_ghes_v2(GArray *table_data, int source_id, BIOSLinker *linker) +{ + uint64_t address_offset; + /* + * Type: + * Generic Hardware Error Source version 2(GHESv2 - Type 10) + */ + build_append_int_noprefix(table_data, ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2); + /* Source Id */ + build_append_int_noprefix(table_data, source_id, 2); + /* Related Source Id */ + build_append_int_noprefix(table_data, 0xffff, 2); + /* Flags */ + build_append_int_noprefix(table_data, 0, 1); + /* Enabled */ + build_append_int_noprefix(table_data, 1, 1); + + /* Number of Records To Pre-allocate */ + build_append_int_noprefix(table_data, 1, 4); + /* Max Sections Per Record */ + build_append_int_noprefix(table_data, 1, 4); + /* Max Raw Data Length */ + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); + + address_offset = table_data->len; + /* Error Status Address */ + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, + 4 /* QWord access */, 0); + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, + address_offset + GAS_ADDR_OFFSET, + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); + + /* + * Notification Structure + * Now only enable ARMv8 SEA notification type + */ + build_ghes_hw_error_notification(table_data, ACPI_GHES_NOTIFY_SEA); + + /* Error Status Block Length */ + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); + + /* + * Read Ack Register + * ACPI 6.1: 18.3.2.8 Generic Hardware Error Source + * version 2 (GHESv2 - Type 10) + */ + address_offset = table_data->len; + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, + 4 /* QWord access */, 0); + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, + address_offset + GAS_ADDR_OFFSET, + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t)); + + /* + * Read Ack Preserve + * We only provide the first bit in Read Ack Register to OSPM to write + * while the other bits are preserved. + */ + build_append_int_noprefix(table_data, ~0x1ULL, 8); + /* Read Ack Write */ + build_append_int_noprefix(table_data, 0x1, 8); +} + +/* Build Hardware Error Source Table */ +void acpi_build_hest(GArray *table_data, GArray *hardware_errors, + BIOSLinker *linker) +{ + uint64_t hest_start = table_data->len; + + /* Hardware Error Source Table header*/ + acpi_data_push(table_data, sizeof(AcpiTableHeader)); + + /* Error Source Count */ + build_append_int_noprefix(table_data, ACPI_GHES_ERROR_SOURCE_COUNT, 4); + + build_ghes_v2(table_data, ACPI_HEST_SRC_ID_SEA, linker); + + build_header(linker, table_data, (void *)(table_data->data + hest_start), + "HEST", table_data->len - hest_start, 1, NULL, ""); +} diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 6819fcf..837bbf9 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -834,6 +834,8 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) if (vms->ras) { acpi_add_table(table_offsets, tables_blob); build_ghes_error_table(tables->hardware_errors, tables->linker); + acpi_build_hest(tables_blob, tables->hardware_errors, + tables->linker); } if (ms->numa_state->num_nodes > 0) { diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h index 3dbda3f..09a7f86 100644 --- a/include/hw/acpi/ghes.h +++ b/include/hw/acpi/ghes.h @@ -22,5 +22,45 @@ #ifndef ACPI_GHES_H #define ACPI_GHES_H +/* + * Values for Hardware Error Notification Type field + */ +enum AcpiGhesNotifyType { + /* Polled */ + ACPI_GHES_NOTIFY_POLLED = 0, + /* External Interrupt */ + ACPI_GHES_NOTIFY_EXTERNAL = 1, + /* Local Interrupt */ + ACPI_GHES_NOTIFY_LOCAL = 2, + /* SCI */ + ACPI_GHES_NOTIFY_SCI = 3, + /* NMI */ + ACPI_GHES_NOTIFY_NMI = 4, + /* CMCI, ACPI 5.0: 18.3.2.7, Table 18-290 */ + ACPI_GHES_NOTIFY_CMCI = 5, + /* MCE, ACPI 5.0: 18.3.2.7, Table 18-290 */ + ACPI_GHES_NOTIFY_MCE = 6, + /* GPIO-Signal, ACPI 6.0: 18.3.2.7, Table 18-332 */ + ACPI_GHES_NOTIFY_GPIO = 7, + /* ARMv8 SEA, ACPI 6.1: 18.3.2.9, Table 18-345 */ + ACPI_GHES_NOTIFY_SEA = 8, + /* ARMv8 SEI, ACPI 6.1: 18.3.2.9, Table 18-345 */ + ACPI_GHES_NOTIFY_SEI = 9, + /* External Interrupt - GSIV, ACPI 6.1: 18.3.2.9, Table 18-345 */ + ACPI_GHES_NOTIFY_GSIV = 10, + /* Software Delegated Exception, ACPI 6.2: 18.3.2.9, Table 18-383 */ + ACPI_GHES_NOTIFY_SDEI = 11, + /* 12 and greater are reserved */ + ACPI_GHES_NOTIFY_RESERVED = 12 +}; + +enum { + ACPI_HEST_SRC_ID_SEA = 0, + /* future ids go here */ + ACPI_HEST_SRC_ID_RESERVED, +}; + void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); +void acpi_build_hest(GArray *table_data, GArray *hardware_error, + BIOSLinker *linker); #endif -- 1.8.3.1
Record the GHEB address via fw_cfg file, when recording a error to CPER, it will use this address to find out Generic Error Data Entries and write the error. Make the HEST GHES to a GED device. Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> --- hw/acpi/generic_event_device.c | 15 ++++++++++++++- hw/acpi/ghes.c | 16 ++++++++++++++++ hw/arm/virt-acpi-build.c | 13 ++++++++++++- include/hw/acpi/generic_event_device.h | 2 ++ include/hw/acpi/ghes.h | 6 ++++++ 5 files changed, 50 insertions(+), 2 deletions(-) diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c index 9cee90c..9bf37e4 100644 --- a/hw/acpi/generic_event_device.c +++ b/hw/acpi/generic_event_device.c @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = { } }; +static const VMStateDescription vmstate_ghes_state = { + .name = "acpi-ghes-state", + .version_id = 1, + .minimum_version_id = 1, + .fields = (VMStateField[]) { + VMSTATE_UINT64(ghes_addr_le, AcpiGhesState), + VMSTATE_END_OF_LIST() + } +}; + static const VMStateDescription vmstate_acpi_ged = { .name = "acpi-ged", .version_id = 1, .minimum_version_id = 1, .fields = (VMStateField[]) { - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState), + VMSTATE_STRUCT(ged_state, AcpiGedState, 1, + vmstate_ged_state, GEDState), + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, + vmstate_ghes_state, AcpiGhesState), VMSTATE_END_OF_LIST(), }, .subsections = (const VMStateDescription * []) { diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c index 9d37798..68f4abf 100644 --- a/hw/acpi/ghes.c +++ b/hw/acpi/ghes.c @@ -23,6 +23,7 @@ #include "hw/acpi/acpi.h" #include "hw/acpi/ghes.h" #include "hw/acpi/aml-build.h" +#include "hw/acpi/generic_event_device.h" #include "hw/nvram/fw_cfg.h" #include "sysemu/sysemu.h" #include "qemu/error-report.h" @@ -208,3 +209,18 @@ void acpi_build_hest(GArray *table_data, GArray *hardware_errors, build_header(linker, table_data, (void *)(table_data->data + hest_start), "HEST", table_data->len - hest_start, 1, NULL, ""); } + +void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, + GArray *hardware_error) +{ + size_t size = 2 * sizeof(uint64_t) + ACPI_GHES_MAX_RAW_DATA_LENGTH; + size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size; + + /* Create a read-only fw_cfg file for GHES */ + fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data, + request_block_size); + + /* Create a read-write fw_cfg file for Address */ + fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, + NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); +} diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 837bbf9..c8aa94d 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -797,6 +797,7 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) unsigned dsdt, xsdt; GArray *tables_blob = tables->table_data; MachineState *ms = MACHINE(vms); + AcpiGedState *acpi_ged_state; table_offsets = g_array_new(false, true /* clear */, sizeof(uint32_t)); @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) acpi_add_table(table_offsets, tables_blob); build_spcr(tables_blob, tables->linker, vms); - if (vms->ras) { + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, + NULL)); + if (acpi_ged_state && vms->ras) { acpi_add_table(table_offsets, tables_blob); build_ghes_error_table(tables->hardware_errors, tables->linker); acpi_build_hest(tables_blob, tables->hardware_errors, @@ -925,6 +928,7 @@ void virt_acpi_setup(VirtMachineState *vms) { AcpiBuildTables tables; AcpiBuildState *build_state; + AcpiGedState *acpi_ged_state; if (!vms->fw_cfg) { trace_virt_acpi_setup(); @@ -955,6 +959,13 @@ void virt_acpi_setup(VirtMachineState *vms) fw_cfg_add_file(vms->fw_cfg, ACPI_BUILD_TPMLOG_FILE, tables.tcpalog->data, acpi_data_len(tables.tcpalog)); + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, + NULL)); + if (acpi_ged_state && vms->ras) { + acpi_ghes_add_fw_cfg(&acpi_ged_state->ghes_state, + vms->fw_cfg, tables.hardware_errors); + } + build_state->rsdp_mr = acpi_add_rom_blob(virt_acpi_build_update, build_state, tables.rsdp, ACPI_BUILD_RSDP_FILE, 0); diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h index d157eac..037d2b5 100644 --- a/include/hw/acpi/generic_event_device.h +++ b/include/hw/acpi/generic_event_device.h @@ -61,6 +61,7 @@ #include "hw/sysbus.h" #include "hw/acpi/memory_hotplug.h" +#include "hw/acpi/ghes.h" #define ACPI_POWER_BUTTON_DEVICE "PWRB" @@ -95,6 +96,7 @@ typedef struct AcpiGedState { GEDState ged_state; uint32_t ged_event_bitmap; qemu_irq irq; + AcpiGhesState ghes_state; } AcpiGedState; void build_ged_aml(Aml *table, const char* name, HotplugHandler *hotplug_dev, diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h index 09a7f86..a6761e6 100644 --- a/include/hw/acpi/ghes.h +++ b/include/hw/acpi/ghes.h @@ -60,7 +60,13 @@ enum { ACPI_HEST_SRC_ID_RESERVED, }; +typedef struct AcpiGhesState { + uint64_t ghes_addr_le; +} AcpiGhesState; + void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); void acpi_build_hest(GArray *table_data, GArray *hardware_error, BIOSLinker *linker); +void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, + GArray *hardware_errors); #endif -- 1.8.3.1
kvm_hwpoison_page_add() and kvm_unpoison_all() will both be used by X86 and ARM platforms, so moving them into "accel/kvm/kvm-all.c" to avoid duplicate code. For architectures that don't use the poison-list functionality the reset handler will harmlessly do nothing, so let's register the kvm_unpoison_all() function in the generic kvm_init() function. Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> Acked-by: Xiang Zheng <zhengxiang9@huawei.com> --- accel/kvm/kvm-all.c | 36 ++++++++++++++++++++++++++++++++++++ include/sysemu/kvm_int.h | 12 ++++++++++++ target/i386/kvm.c | 36 ------------------------------------ 3 files changed, 48 insertions(+), 36 deletions(-) diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c index b2f1a5b..404e863 100644 --- a/accel/kvm/kvm-all.c +++ b/accel/kvm/kvm-all.c @@ -44,6 +44,7 @@ #include "qapi/visitor.h" #include "qapi/qapi-types-common.h" #include "qapi/qapi-visit-common.h" +#include "sysemu/reset.h" #include "hw/boards.h" @@ -873,6 +874,39 @@ int kvm_vm_check_extension(KVMState *s, unsigned int extension) return ret; } +typedef struct HWPoisonPage { + ram_addr_t ram_addr; + QLIST_ENTRY(HWPoisonPage) list; +} HWPoisonPage; + +static QLIST_HEAD(, HWPoisonPage) hwpoison_page_list = + QLIST_HEAD_INITIALIZER(hwpoison_page_list); + +static void kvm_unpoison_all(void *param) +{ + HWPoisonPage *page, *next_page; + + QLIST_FOREACH_SAFE(page, &hwpoison_page_list, list, next_page) { + QLIST_REMOVE(page, list); + qemu_ram_remap(page->ram_addr, TARGET_PAGE_SIZE); + g_free(page); + } +} + +void kvm_hwpoison_page_add(ram_addr_t ram_addr) +{ + HWPoisonPage *page; + + QLIST_FOREACH(page, &hwpoison_page_list, list) { + if (page->ram_addr == ram_addr) { + return; + } + } + page = g_new(HWPoisonPage, 1); + page->ram_addr = ram_addr; + QLIST_INSERT_HEAD(&hwpoison_page_list, page, list); +} + static uint32_t adjust_ioeventfd_endianness(uint32_t val, uint32_t size) { #if defined(HOST_WORDS_BIGENDIAN) != defined(TARGET_WORDS_BIGENDIAN) @@ -2070,6 +2104,8 @@ static int kvm_init(MachineState *ms) goto err; } + qemu_register_reset(kvm_unpoison_all, NULL); + if (s->kernel_irqchip_allowed) { kvm_irqchip_create(s); } diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h index ac2d1f8..c660a70 100644 --- a/include/sysemu/kvm_int.h +++ b/include/sysemu/kvm_int.h @@ -42,4 +42,16 @@ void kvm_memory_listener_register(KVMState *s, KVMMemoryListener *kml, AddressSpace *as, int as_id); void kvm_set_max_memslot_size(hwaddr max_slot_size); + +/** + * kvm_hwpoison_page_add: + * + * Parameters: + * @ram_addr: the address in the RAM for the poisoned page + * + * Add a poisoned page to the list + * + * Return: None. + */ +void kvm_hwpoison_page_add(ram_addr_t ram_addr); #endif diff --git a/target/i386/kvm.c b/target/i386/kvm.c index 0b51190..66e7543 100644 --- a/target/i386/kvm.c +++ b/target/i386/kvm.c @@ -24,7 +24,6 @@ #include "sysemu/sysemu.h" #include "sysemu/hw_accel.h" #include "sysemu/kvm_int.h" -#include "sysemu/reset.h" #include "sysemu/runstate.h" #include "kvm_i386.h" #include "hyperv.h" @@ -522,40 +521,6 @@ uint64_t kvm_arch_get_supported_msr_feature(KVMState *s, uint32_t index) } } - -typedef struct HWPoisonPage { - ram_addr_t ram_addr; - QLIST_ENTRY(HWPoisonPage) list; -} HWPoisonPage; - -static QLIST_HEAD(, HWPoisonPage) hwpoison_page_list = - QLIST_HEAD_INITIALIZER(hwpoison_page_list); - -static void kvm_unpoison_all(void *param) -{ - HWPoisonPage *page, *next_page; - - QLIST_FOREACH_SAFE(page, &hwpoison_page_list, list, next_page) { - QLIST_REMOVE(page, list); - qemu_ram_remap(page->ram_addr, TARGET_PAGE_SIZE); - g_free(page); - } -} - -static void kvm_hwpoison_page_add(ram_addr_t ram_addr) -{ - HWPoisonPage *page; - - QLIST_FOREACH(page, &hwpoison_page_list, list) { - if (page->ram_addr == ram_addr) { - return; - } - } - page = g_new(HWPoisonPage, 1); - page->ram_addr = ram_addr; - QLIST_INSERT_HEAD(&hwpoison_page_list, page, list); -} - static int kvm_get_mce_cap_supported(KVMState *s, uint64_t *mce_cap, int *max_banks) { @@ -2161,7 +2126,6 @@ int kvm_arch_init(MachineState *ms, KVMState *s) fprintf(stderr, "e820_add_entry() table is full\n"); return ret; } - qemu_register_reset(kvm_unpoison_all, NULL); shadow_mem = object_property_get_int(OBJECT(s), "kvm-shadow-mem", &error_abort); if (shadow_mem != -1) { -- 1.8.3.1
kvm_arch_on_sigbus_vcpu() error injection uses source_id as index in etc/hardware_errors to find out Error Status Data Block entry corresponding to error source. So supported source_id values should be assigned here and not be changed afterwards to make sure that guest will write error into expected Error Status Data Block even if guest was migrated to a newer QEMU. Before QEMU writes a new error to ACPI table, it will check whether previous error has been acknowledged. Otherwise it will ignore the new error. For the errors section type, QEMU simulate it to memory section error. Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> --- hw/acpi/ghes.c | 224 ++++++++++++++++++++++++++++++++++++++++++++++++- include/hw/acpi/ghes.h | 3 + include/qemu/uuid.h | 5 ++ 3 files changed, 230 insertions(+), 2 deletions(-) diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c index 68f4abf..f2ecffe 100644 --- a/hw/acpi/ghes.c +++ b/hw/acpi/ghes.c @@ -28,21 +28,56 @@ #include "sysemu/sysemu.h" #include "qemu/error-report.h" -#include "hw/acpi/bios-linker-loader.h" - #define ACPI_GHES_ERRORS_FW_CFG_FILE "etc/hardware_errors" #define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr" /* The max size in bytes for one error block */ #define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 + /* Now only support ARMv8 SEA notification type error source */ #define ACPI_GHES_ERROR_SOURCE_COUNT 1 + /* Generic Hardware Error Source version 2 */ #define ACPI_GHES_SOURCE_GENERIC_ERROR_V2 10 + /* Address offset in Generic Address Structure(GAS) */ #define GAS_ADDR_OFFSET 4 /* + * The total size of Generic Error Data Entry + * ACPI 6.1/6.2: 18.3.2.7.1 Generic Error Data, + * Table 18-343 Generic Error Data Entry + */ +#define ACPI_GHES_DATA_LENGTH 72 + +/* The memory section CPER size, UEFI 2.6: N.2.5 Memory Error Section */ +#define ACPI_GHES_MEM_CPER_LENGTH 80 + +/* Masks for block_status flags */ +#define ACPI_GEBS_UNCORRECTABLE 1 + +#define UEFI_CPER_SEC_PLATFORM_MEM \ + UUID_LE(0xA5BC1114, 0x6F64, 0x4EDE, 0xB8, 0x63, 0x3E, 0x83, \ + 0xED, 0x7C, 0x83, 0xB1) + +/* + * Total size for Generic Error Status Block except Generic Error Data Entries + * ACPI 6.2: 18.3.2.7.1 Generic Error Data, + * Table 18-380 Generic Error Status Block + */ +#define ACPI_GHES_GESB_SIZE 20 + +/* + * Values for error_severity field + */ +enum AcpiGenericErrorSeverity { + ACPI_CPER_SEV_RECOVERABLE = 0, + ACPI_CPER_SEV_FATAL = 1, + ACPI_CPER_SEV_CORRECTED = 2, + ACPI_CPER_SEV_NONE = 3, +}; + +/* * Hardware Error Notification * ACPI 4.0: 17.3.2.7 Hardware Error Notification * Composes dummy Hardware Error Notification descriptor of specified type @@ -73,6 +108,127 @@ static void build_ghes_hw_error_notification(GArray *table, const uint8_t type) } /* + * Generic Error Data Entry + * ACPI 6.1: 18.3.2.7.1 Generic Error Data + */ +static void acpi_ghes_generic_error_data(GArray *table, QemuUUID section_type, + uint32_t error_severity, uint8_t validation_bits, uint8_t flags, + uint32_t error_data_length, QemuUUID fru_id, + uint64_t time_stamp) +{ + /* Section Type */ + g_array_append_vals(table, section_type.data, + ARRAY_SIZE(section_type.data)); + + /* Error Severity */ + build_append_int_noprefix(table, error_severity, 4); + /* Revision */ + build_append_int_noprefix(table, 0x300, 2); + /* Validation Bits */ + build_append_int_noprefix(table, validation_bits, 1); + /* Flags */ + build_append_int_noprefix(table, flags, 1); + /* Error Data Length */ + build_append_int_noprefix(table, error_data_length, 4); + + /* FRU Id */ + g_array_append_vals(table, fru_id.data, ARRAY_SIZE(fru_id.data)); + + /* FRU Text */ + build_append_int_noprefix(table, 0, 20); + /* Timestamp */ + build_append_int_noprefix(table, time_stamp, 8); +} + +/* + * Generic Error Status Block + * ACPI 6.1: 18.3.2.7.1 Generic Error Data + */ +static void acpi_ghes_generic_error_status(GArray *table, uint32_t block_status, + uint32_t raw_data_offset, uint32_t raw_data_length, + uint32_t data_length, uint32_t error_severity) +{ + /* Block Status */ + build_append_int_noprefix(table, block_status, 4); + /* Raw Data Offset */ + build_append_int_noprefix(table, raw_data_offset, 4); + /* Raw Data Length */ + build_append_int_noprefix(table, raw_data_length, 4); + /* Data Length */ + build_append_int_noprefix(table, data_length, 4); + /* Error Severity */ + build_append_int_noprefix(table, error_severity, 4); +} + +/* UEFI 2.6: N.2.5 Memory Error Section */ +static void acpi_ghes_build_append_mem_cper(GArray *table, + uint64_t error_physical_addr) +{ + /* + * Memory Error Record + */ + + /* Validation Bits */ + build_append_int_noprefix(table, + (1ULL << 14) | /* Type Valid */ + (1ULL << 1) /* Physical Address Valid */, + 8); + /* Error Status */ + build_append_int_noprefix(table, 0, 8); + /* Physical Address */ + build_append_int_noprefix(table, error_physical_addr, 8); + /* Skip all the detailed information normally found in such a record */ + build_append_int_noprefix(table, 0, 48); + /* Memory Error Type */ + build_append_int_noprefix(table, 0 /* Unknown error */, 1); + /* Skip all the detailed information normally found in such a record */ + build_append_int_noprefix(table, 0, 7); +} + +static int acpi_ghes_record_mem_error(uint64_t error_block_address, + uint64_t error_physical_addr) +{ + GArray *block; + /* Memory Error Section Type */ + QemuUUID mem_section_id_le = UEFI_CPER_SEC_PLATFORM_MEM; + QemuUUID fru_id = {}; + uint32_t data_length; + + block = g_array_new(false, true /* clear */, 1); + + /* This is the length if adding a new generic error data entry*/ + data_length = ACPI_GHES_DATA_LENGTH + ACPI_GHES_MEM_CPER_LENGTH; + + /* + * Check whether it will run out of the preallocated memory if adding a new + * generic error data entry + */ + if ((data_length + ACPI_GHES_GESB_SIZE) > ACPI_GHES_MAX_RAW_DATA_LENGTH) { + error_report("Not enough memory to record new CPER!!!"); + return -1; + } + + /* Build the new generic error status block header */ + acpi_ghes_generic_error_status(block, ACPI_GEBS_UNCORRECTABLE, + 0, 0, data_length, ACPI_CPER_SEV_RECOVERABLE); + + /* Build this new generic error data entry header */ + acpi_ghes_generic_error_data(block, mem_section_id_le, + ACPI_CPER_SEV_RECOVERABLE, 0, 0, + ACPI_GHES_MEM_CPER_LENGTH, fru_id, 0); + + /* Build the memory section CPER for above new generic error data entry */ + acpi_ghes_build_append_mem_cper(block, error_physical_addr); + + /* Write back above this new generic error data entry to guest memory */ + cpu_physical_memory_write(error_block_address, block->data, block->len); + + g_array_free(block, true); + + return 0; +} + +/* * Build table for the hardware error fw_cfg blob. * Initialize "etc/hardware_errors" and "etc/hardware_errors_addr" fw_cfg blobs. * See docs/specs/acpi_hest_ghes.rst for blobs format. @@ -224,3 +380,67 @@ void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); } + +int acpi_ghes_record_errors(uint8_t source_id, uint64_t physical_address) +{ + uint64_t error_block_addr, read_ack_register_addr, read_ack_register = 0; + int loop = 0; + uint64_t start_addr; + bool ret = -1; + AcpiGedState *acpi_ged_state; + AcpiGhesState *ags; + + assert(source_id < ACPI_HEST_SRC_ID_RESERVED); + + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, + NULL)); + if (acpi_ged_state) { + ags = &acpi_ged_state->ghes_state; + } else { + error_report("ACPI GED device not found"); + return -1; + } + + start_addr = le64_to_cpu(ags->ghes_addr_le); + + if (physical_address) { + + if (source_id < ACPI_HEST_SRC_ID_RESERVED) { + start_addr += source_id * sizeof(uint64_t); + } + + cpu_physical_memory_read(start_addr, &error_block_addr, + sizeof(error_block_addr)); + + read_ack_register_addr = start_addr + + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t); +retry: + cpu_physical_memory_read(read_ack_register_addr, + &read_ack_register, sizeof(read_ack_register)); + + /* zero means OSPM does not acknowledge the error */ + if (!read_ack_register) { + if (loop < 3) { + usleep(100 * 1000); + loop++; + goto retry; + } else { + error_report("OSPM does not acknowledge previous error," + " so can not record CPER for current error anymore"); + } + } else if (error_block_addr) { + read_ack_register = cpu_to_le64(0); + /* + * Clear the Read Ack Register, OSPM will write it to 1 when + * acknowledge this error. + */ + cpu_physical_memory_write(read_ack_register_addr, + &read_ack_register, sizeof(uint64_t)); + + ret = acpi_ghes_record_mem_error(error_block_addr, + physical_address); + } + } + + return ret; +} diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h index a6761e6..ab0ae33 100644 --- a/include/hw/acpi/ghes.h +++ b/include/hw/acpi/ghes.h @@ -22,6 +22,8 @@ #ifndef ACPI_GHES_H #define ACPI_GHES_H +#include "hw/acpi/bios-linker-loader.h" + /* * Values for Hardware Error Notification Type field */ @@ -69,4 +71,5 @@ void acpi_build_hest(GArray *table_data, GArray *hardware_error, BIOSLinker *linker); void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, GArray *hardware_errors); +int acpi_ghes_record_errors(uint8_t notify, uint64_t error_physical_addr); #endif diff --git a/include/qemu/uuid.h b/include/qemu/uuid.h index 129c45f..b35e294 100644 --- a/include/qemu/uuid.h +++ b/include/qemu/uuid.h @@ -34,6 +34,11 @@ typedef struct { }; } QemuUUID; +#define UUID_LE(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7) \ + {{{ (a) & 0xff, ((a) >> 8) & 0xff, ((a) >> 16) & 0xff, ((a) >> 24) & 0xff, \ + (b) & 0xff, ((b) >> 8) & 0xff, (c) & 0xff, ((c) >> 8) & 0xff, \ + (d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) } } } + #define UUID_FMT "%02hhx%02hhx%02hhx%02hhx-" \ "%02hhx%02hhx-%02hhx%02hhx-" \ "%02hhx%02hhx-" \ -- 1.8.3.1
Add a SIGBUS signal handler. In this handler, it checks the SIGBUS type, translates the host VA delivered by host to guest PA, then fills this PA to guest APEI GHES memory, then notifies guest according to the SIGBUS type. When guest accesses the poisoned memory, it will generate a Synchronous External Abort(SEA). Then host kernel gets an APEI notification and calls memory_failure() to unmapped the affected page in stage 2, finally returns to guest. Guest continues to access the PG_hwpoison page, it will trap to KVM as stage2 fault, then a SIGBUS_MCEERR_AR synchronous signal is delivered to Qemu, Qemu records this error address into guest APEI GHES memory and notifes guest using Synchronous-External-Abort(SEA). In order to inject a vSEA, we introduce the kvm_inject_arm_sea() function in which we can setup the type of exception and the syndrome information. When switching to guest, the target vcpu will jump to the synchronous external abort vector table entry. The ESR_ELx.DFSC is set to synchronous external abort(0x10), and the ESR_ELx.FnV is set to not valid(0x1), which will tell guest that FAR is not valid and hold an UNKNOWN value. These values will be set to KVM register structures through KVM_SET_ONE_REG IOCTL. Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Xiang Zheng <zhengxiang9@huawei.com> --- include/sysemu/kvm.h | 3 +-- target/arm/cpu.h | 4 +++ target/arm/helper.c | 2 +- target/arm/internals.h | 5 ++-- target/arm/kvm64.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++ target/arm/tlb_helper.c | 2 +- target/i386/cpu.h | 2 ++ 7 files changed, 78 insertions(+), 6 deletions(-) diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h index 141342d..3b22504 100644 --- a/include/sysemu/kvm.h +++ b/include/sysemu/kvm.h @@ -379,8 +379,7 @@ bool kvm_vcpu_id_is_valid(int vcpu_id); /* Returns VCPU ID to be used on KVM_CREATE_VCPU ioctl() */ unsigned long kvm_arch_vcpu_id(CPUState *cpu); -#ifdef TARGET_I386 -#define KVM_HAVE_MCE_INJECTION 1 +#ifdef KVM_HAVE_MCE_INJECTION void kvm_arch_on_sigbus_vcpu(CPUState *cpu, int code, void *addr); #endif diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 5f70e9e..723bdb9 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -28,6 +28,10 @@ /* ARM processors have a weak memory model */ #define TCG_GUEST_DEFAULT_MO (0) +#ifdef TARGET_AARCH64 +#define KVM_HAVE_MCE_INJECTION 1 +#endif + #define EXCP_UDEF 1 /* undefined instruction */ #define EXCP_SWI 2 /* software interrupt */ #define EXCP_PREFETCH_ABORT 3 diff --git a/target/arm/helper.c b/target/arm/helper.c index 5074b5f..05bffd3 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -3045,7 +3045,7 @@ static uint64_t do_ats_write(CPUARMState *env, uint64_t value, * Report exception with ESR indicating a fault due to a * translation table walk for a cache maintenance instruction. */ - syn = syn_data_abort_no_iss(current_el == target_el, + syn = syn_data_abort_no_iss(current_el == target_el, 0, fi.ea, 1, fi.s1ptw, 1, fsc); env->exception.vaddress = value; env->exception.fsr = fsr; diff --git a/target/arm/internals.h b/target/arm/internals.h index f5313dd..28b8451 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -451,13 +451,14 @@ static inline uint32_t syn_insn_abort(int same_el, int ea, int s1ptw, int fsc) | ARM_EL_IL | (ea << 9) | (s1ptw << 7) | fsc; } -static inline uint32_t syn_data_abort_no_iss(int same_el, +static inline uint32_t syn_data_abort_no_iss(int same_el, int fnv, int ea, int cm, int s1ptw, int wnr, int fsc) { return (EC_DATAABORT << ARM_EL_EC_SHIFT) | (same_el << ARM_EL_EC_SHIFT) | ARM_EL_IL - | (ea << 9) | (cm << 8) | (s1ptw << 7) | (wnr << 6) | fsc; + | (fnv << 10) | (ea << 9) | (cm << 8) | (s1ptw << 7) + | (wnr << 6) | fsc; } static inline uint32_t syn_data_abort_with_iss(int same_el, diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index 876184b..f3b05c1 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -28,6 +28,8 @@ #include "kvm_arm.h" #include "hw/boards.h" #include "internals.h" +#include "hw/acpi/acpi.h" +#include "hw/acpi/ghes.h" static bool have_guest_debug; @@ -843,6 +845,30 @@ int kvm_arm_cpreg_level(uint64_t regidx) return KVM_PUT_RUNTIME_STATE; } +/* Callers must hold the iothread mutex lock */ +static void kvm_inject_arm_sea(CPUState *c) +{ + ARMCPU *cpu = ARM_CPU(c); + CPUARMState *env = &cpu->env; + CPUClass *cc = CPU_GET_CLASS(c); + uint32_t esr; + bool same_el; + + c->exception_index = EXCP_DATA_ABORT; + env->exception.target_el = 1; + + /* + * Set the DFSC to synchronous external abort and set FnV to not valid, + * this will tell guest the FAR_ELx is UNKNOWN for this abort. + */ + same_el = arm_current_el(env) == env->exception.target_el; + esr = syn_data_abort_no_iss(same_el, 1, 0, 0, 0, 0, 0x10); + + env->exception.syndrome = esr; + + cc->do_interrupt(c); +} + #define AARCH64_CORE_REG(x) (KVM_REG_ARM64 | KVM_REG_SIZE_U64 | \ KVM_REG_ARM_CORE | KVM_REG_ARM_CORE_REG(x)) @@ -1295,6 +1321,46 @@ int kvm_arch_get_registers(CPUState *cs) return ret; } +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) +{ + ram_addr_t ram_addr; + hwaddr paddr; + + assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO); + + if (acpi_enabled && addr && + object_property_get_bool(qdev_get_machine(), "ras", NULL)) { + ram_addr = qemu_ram_addr_from_host(addr); + if (ram_addr != RAM_ADDR_INVALID && + kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) { + kvm_hwpoison_page_add(ram_addr); + /* + * Asynchronous signal will be masked by main thread, so + * only handle synchronous signal. + */ + if (code == BUS_MCEERR_AR) { + kvm_cpu_synchronize_state(c); + if (!acpi_ghes_record_errors(ACPI_HEST_SRC_ID_SEA, paddr)) { + kvm_inject_arm_sea(c); + } else { + error_report("failed to record the error"); + abort(); + } + } + return; + } + if (code == BUS_MCEERR_AO) { + error_report("Hardware memory error at addr %p for memory used by " + "QEMU itself instead of guest system!", addr); + } + } + + if (code == BUS_MCEERR_AR) { + error_report("Hardware memory error!"); + exit(1); + } +} + /* C6.6.29 BRK instruction */ static const uint32_t brk_insn = 0xd4200000; diff --git a/target/arm/tlb_helper.c b/target/arm/tlb_helper.c index 5feb312..499672e 100644 --- a/target/arm/tlb_helper.c +++ b/target/arm/tlb_helper.c @@ -33,7 +33,7 @@ static inline uint32_t merge_syn_data_abort(uint32_t template_syn, * ISV field. */ if (!(template_syn & ARM_EL_ISV) || target_el != 2 || s1ptw) { - syn = syn_data_abort_no_iss(same_el, + syn = syn_data_abort_no_iss(same_el, 0, ea, 0, s1ptw, is_write, fsc); } else { /* diff --git a/target/i386/cpu.h b/target/i386/cpu.h index af28293..1a0dbc1 100644 --- a/target/i386/cpu.h +++ b/target/i386/cpu.h @@ -29,6 +29,8 @@ /* The x86 has a strong memory model with some store-after-load re-ordering */ #define TCG_GUEST_DEFAULT_MO (TCG_MO_ALL & ~TCG_MO_ST_LD) +#define KVM_HAVE_MCE_INJECTION 1 + /* Maximum instruction code size */ #define TARGET_MAX_INSN_SIZE 16 -- 1.8.3.1
I and Xiang are willing to review the APEI-related patches and volunteer as the reviewers for the HEST/GHES part. Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> --- MAINTAINERS | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 387879a..5af70a5 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c F: tests/acpi-utils.[hc] F: tests/data/acpi/ +ACPI/HEST/GHES +R: Dongjiu Geng <gengdongjiu@huawei.com> +R: Xiang Zheng <zhengxiang9@huawei.com> +L: qemu-arm@nongnu.org +S: Maintained +F: hw/acpi/ghes.c +F: include/hw/acpi/ghes.h +F: docs/specs/acpi_hest_ghes.rst + ppc4xx M: David Gibson <david@gibson.dropbear.id.au> L: qemu-ppc@nongnu.org -- 1.8.3.1
1. How to enable this feature: (a) In KVM mode: ./qemu-system-aarch64 --enable-kvm -cpu host --bios QEMU_EFI.fd_new -machine virt,gic-version=3,ras,kernel-irqchip=on -smp 4 -nographic -kernel Image -append "rdinit=/init console=ttyAMA0 mem=512M root=/dev/ram0" -initrd guestfs_new.cpio.gz (b)In TCG mode: ./qemu-system-aarch64 -cpu cortex-a57 -bios QEMU_EFI.fd_new -machine virt,gic-version=3,ras -smp 4 -nographic -kernel Image -append "rdinit=/init console=ttyAMA0 mem=512M root=/dev/ram0 earlycon=pl011,0x9000000 rw" -initrd guestfs_new.cpio.gz 2. How to quickly test this series patches: (a)hack some code as shown in [1]. (b) build and run the qemu: ./qemu-system-aarch64 -cpu cortex-a57 -bios QEMU_EFI.fd_new -machine virt,gic-version=3,ras -smp 4 -nographic -kernel Image -append "rdinit=/init console=ttyAMA0 mem=512M root=/dev/ram0 earlycon=pl011,0x9000000 rw" -initrd guestfs_new.cpio.gz (c) find the process id(PID) of "qemu-system-aarch64": $ ps -aux | grep "qemu (d) then send SIGBUS to qemu process: kill -s SIGBUS PID [1]: diff --git a/cpus.c b/cpus.c index b472378b70..a0b67f5947 100644 --- a/cpus.c +++ b/cpus.c @@ -1087,6 +1087,8 @@ static void sigbus_reraise(void) static void sigbus_handler(int n, siginfo_t *siginfo, void *ctx) { + siginfo->si_code = BUS_MCEERR_AO; + if (siginfo->si_code != BUS_MCEERR_AO && siginfo->si_code != BUS_MCEERR_AR) { sigbus_reraise(); } diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index f3b05c1977..03e8e20f4a 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -1323,17 +1323,23 @@ int kvm_arch_get_registers(CPUState *cs) void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) { +#if 0 ram_addr_t ram_addr; hwaddr paddr; assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO); - - if (acpi_enabled && addr && +#else + hwaddr paddr = 0x400a1000; + code = BUS_MCEERR_AR; +#endif + if (acpi_enabled && paddr && object_property_get_bool(qdev_get_machine(), "ras", NULL)) { +#if 0 ram_addr = qemu_ram_addr_from_host(addr); if (ram_addr != RAM_ADDR_INVALID && kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) { kvm_hwpoison_page_add(ram_addr); +#endif /* * Asynchronous signal will be masked by main thread, so * only handle synchronous signal. @@ -1348,11 +1354,13 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) } } return; +#if 0 } if (code == BUS_MCEERR_AO) { error_report("Hardware memory error at addr %p for memory used by " "QEMU itself instead of guest system!", addr); } +#endif } if (code == BUS_MCEERR_AR) { On 2020/1/8 19:32, Dongjiu Geng wrote: > In the ARMv8 platform, the CPU error types are synchronous external abort(SEA) > and SError Interrupt (SEI). If exception happens in guest, sometimes it's better > for guest to perform the recovery, because host does not know the detailed > information of guest. For example, if an exception happens in a user-space > application within guest, host does not know which application encounters > errors. > > For the ARMv8 SEA/SEI, KVM or host kernel delivers SIGBUS to notify userspace. > After user space gets the notification, it will record the CPER into guest GHES > buffer and inject an exception or IRQ into guest. > > In the current implementation, if the type of SIGBUS is BUS_MCEERR_AR, we will > treat it as a synchronous exception, and notify guest with ARMv8 SEA > notification type after recording CPER into guest. > > This series of patches are based on Qemu 4.2, which include two parts: > 1. Generate APEI/GHES table. > 2. Handle the SIGBUS signal, record the CPER in runtime and fill it into guest > memory, then notify guest according to the type of SIGBUS. > > The whole solution was suggested by James(james.morse@arm.com); The solution of > APEI section was suggested by Laszlo(lersek@redhat.com). > Show some discussions in [1]. > > This series of patches have already been tested on ARM64 platform with RAS > feature enabled: > Show the APEI part verification result in [2]. > Show the BUS_MCEERR_AR SIGBUS handling verification result in [3]. > > --- > Change since v21: > 1. Make the user-facing 'ras' option description more clearly to address Peter's comments. > 2. Update the doc description in "docs/specs/acpi_hest_ghes.rst" > 3. Split HEST/GHES patches to more patches to make the review easily > 4. Using source_id to index the location to save the CPER. > 5. Optimize and simplify the logic to build HEST/GHES table to address Igor/Michael/Beata comments. > 6. make ghes_addr_le a part of GED device. > > Change since v20: > 1. Move some implementation details from acpi_ghes.h to acpi_ghes.c > 2. Add the reviewers for the ACPI/APEI/GHES part > > Change since v19: > 1. Fix clang compile error > 2. Fix sphinx build error > > Change since v18: > 1. Fix some code-style and typo/grammar problems. > 2. Remove no_ras in the VirtMachineClass struct. > 3. Convert documentation to rst format. > 4. Simplize the code and add comments for some magic value. > 5. Move kvm_inject_arm_sea() function into the patch where it's used. > 6. Register the reset handler(kvm_unpoison_all()) in the kvm_init() function. > > Change since v17: > 1. Improve some commit messages and comments. > 2. Fix some code-style problems. > 3. Add a *ras* machine option. > 4. Move HEST/GHES related structures and macros into "hw/acpi/acpi_ghes.*". > 5. Move HWPoison page functions into "include/sysemu/kvm_int.h". > 6. Fix some bugs. > 7. Improve the design document. > > Change since v16: > 1. check whether ACPI table is enabled when handling the memory error in the SIGBUS handler. > > Change since v15: > 1. Add a doc-comment in the proper format for 'include/exec/ram_addr.h' > 2. Remove write_part_cpustate_to_list() because there is another bug fix patch > has been merged "arm: Allow system registers for KVM guests to be changed by QEMU code" > 3. Add some comments for kvm_inject_arm_sea() in 'target/arm/kvm64.c' > 4. Compare the arm_current_el() return value to 0,1,2,3, not to PSTATE_MODE_* constants. > 5. Change the RAS support wasn't introduced before 4.1 QEMU version. > 6. Move the no_ras flag patch to begin in this series > > Change since v14: > 1. Remove the BUS_MCEERR_AO handling logic because this asynchronous signal was masked by main thread > 2. Address some Igor Mammedov's comments(ACPI part) > 1) change the comments for the enum AcpiHestNotifyType definition and remove ditto in patch 1 > 2) change some patch commit messages and separate "APEI GHES table generation" patch to more patches. > 3. Address some peter's comments(arm64 Synchronous External Abort injection) > 1) change some code notes > 2) using arm_current_el() for current EL > 2) use the helper functions for those (syn_data_abort_*). > > Change since v13: > 1. Move the patches that set guest ESR and inject virtual SError out of this series > 2. Clean and optimize the APEI part patches > 3. Update the commit messages and add some comments for the code > > Change since v12: > 1. Address Paolo's comments to move HWPoisonPage definition to accel/kvm/kvm-all.c > 2. Only call kvm_cpu_synchronize_state() when get the BUS_MCEERR_AR signal > 3. Only add and enable GPIO-Signal and ARMv8 SEA two hardware error sources > 4. Address Michael's comments to not sync SPDX from Linux kernel header file > > Change since v11: > Address James's comments(james.morse@arm.com) > 1. Check whether KVM has the capability to to set ESR instead of detecting host CPU RAS capability > 2. For SIGBUS_MCEERR_AR SIGBUS, use Synchronous-External-Abort(SEA) notification type > for SIGBUS_MCEERR_AO SIGBUS, use GPIO-Signal notification > > > Address Shannon's comments(for ACPI part): > 1. Unify hest_ghes.c and hest_ghes.h license declaration > 2. Remove unnecessary including "qmp-commands.h" in hest_ghes.c > 3. Unconditionally add guest APEI table based on James's comments(james.morse@arm.com) > 4. Add a option to virt machine for migration compatibility. On new virt machine it's on > by default while off for old ones, we enabled it since 2.12 > 5. Refer to the ACPI spec version which introduces Hardware Error Notification first time > 6. Add ACPI_HEST_NOTIFY_RESERVED notification type > > Address Igor's comments(for ACPI part): > 1. Add doc patch first which will describe how it's supposed to work between QEMU/firmware/guest > OS with expected flows. > 2. Move APEI diagrams into doc/spec patch > 3. Remove redundant g_malloc in ghes_record_cper() > 4. Use build_append_int_noprefix() API to compose whole error status block and whole APEI table, > and try to get rid of most structures in patch 1, as they will be left unused after that > 5. Reuse something like https://github.com/imammedo/qemu/commit/3d2fd6d13a3ea298d2ee814835495ce6241d085c > to build GAS > 6. Remove much offsetof() in the function > 7. Build independent tables first and only then build dependent tables passing to it pointers > to previously build table if necessary. > 8. Redefine macro GHES_ACPI_HEST_NOTIFY_RESERVED to ACPI_HEST_ERROR_SOURCE_COUNT to avoid confusion > > > Address Peter Maydell's comments > 1. linux-headers is done as a patch of their own created using scripts/update-linux-headers.sh run against a > mainline kernel tree > 2. Tested whether this patchset builds OK on aarch32 > 3. Abstract Hwpoison page adding code out properly into a cpu-independent source file from target/i386/kvm.c, > such as kvm-all.c > 4. Add doc-comment formatted documentation comment for new globally-visible function prototype in a header > > --- > [1]: > https://lkml.org/lkml/2017/2/27/246 > https://patchwork.kernel.org/patch/9633105/ > https://patchwork.kernel.org/patch/9925227/ > > [2]: > Note: the UEFI(QEMU_EFI.fd) is needed if guest want to use ACPI table. > > After guest boot up, dump the APEI table, then can see the initialized table > (1) # iasl -p ./HEST -d /sys/firmware/acpi/tables/HEST > (2) # cat HEST.dsl > /* > * Intel ACPI Component Architecture > * AML/ASL+ Disassembler version 20170728 (64-bit version) > * Copyright (c) 2000 - 2017 Intel Corporation > * > * Disassembly of /sys/firmware/acpi/tables/HEST, Mon Sep 5 07:59:17 2016 > * > * ACPI Data Table [HEST] > * > * Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue > */ > > .................................................................................. > [308h 0776 2] Subtable Type : 000A [Generic Hardware Error Source V2] > [30Ah 0778 2] Source Id : 0001 > [30Ch 0780 2] Related Source Id : FFFF > [30Eh 0782 1] Reserved : 00 > [30Fh 0783 1] Enabled : 01 > [310h 0784 4] Records To Preallocate : 00000001 > [314h 0788 4] Max Sections Per Record : 00000001 > [318h 0792 4] Max Raw Data Length : 00001000 > > [31Ch 0796 12] Error Status Address : [Generic Address Structure] > [31Ch 0796 1] Space ID : 00 [SystemMemory] > [31Dh 0797 1] Bit Width : 40 > [31Eh 0798 1] Bit Offset : 00 > [31Fh 0799 1] Encoded Access Width : 04 [QWord Access:64] > [320h 0800 8] Address : 00000000785D0040 > > [328h 0808 28] Notify : [Hardware Error Notification Structure] > [328h 0808 1] Notify Type : 08 [SEA] > [329h 0809 1] Notify Length : 1C > [32Ah 0810 2] Configuration Write Enable : 0000 > [32Ch 0812 4] PollInterval : 00000000 > [330h 0816 4] Vector : 00000000 > [334h 0820 4] Polling Threshold Value : 00000000 > [338h 0824 4] Polling Threshold Window : 00000000 > [33Ch 0828 4] Error Threshold Value : 00000000 > [340h 0832 4] Error Threshold Window : 00000000 > > [344h 0836 4] Error Status Block Length : 00001000 > [348h 0840 12] Read Ack Register : [Generic Address Structure] > [348h 0840 1] Space ID : 00 [SystemMemory] > [349h 0841 1] Bit Width : 40 > [34Ah 0842 1] Bit Offset : 00 > [34Bh 0843 1] Encoded Access Width : 04 [QWord Access:64] > [34Ch 0844 8] Address : 00000000785D0098 > > [354h 0852 8] Read Ack Preserve : 00000000FFFFFFFE > [35Ch 0860 8] Read Ack Write : 0000000000000001 > > ..................................................................................... > > (3) After a synchronous external abort(SEA) happen, Qemu receive a SIGBUS and > filled the CPER into guest GHES memory. For example, according to above table, > the address that contains the physical address of a block of memory that holds > the error status data for this abort is 0x00000000785D0040 > (4) the address for SEA notification error source is 0x785d80b0 > (qemu) xp /1 0x00000000785D0040 > 00000000785d0040: 0x785d80b0 > > (5) check the content of generic error status block and generic error data entry > (qemu) xp /100x 0x785d80b0 > 00000000785d80b0: 0x00000001 0x00000000 0x00000000 0x00000098 > 00000000785d80c0: 0x00000000 0xa5bc1114 0x4ede6f64 0x833e63b8 > 00000000785d80d0: 0xb1837ced 0x00000000 0x00000300 0x00000050 > 00000000785d80e0: 0x00000000 0x00000000 0x00000000 0x00000000 > 00000000785d80f0: 0x00000000 0x00000000 0x00000000 0x00000000 > 00000000785d8100: 0x00000000 0x00000000 0x00000000 0x00004002 > (6) check the OSPM's ACK value(for example SEA) > /* Before OSPM acknowledges the error, check the ACK value */ > (qemu) xp /1 0x00000000785D0098 > 00000000785d00f0: 0x00000000 > > /* After OSPM acknowledges the error, check the ACK value, it change to 1 from 0 */ > (qemu) xp /1 0x00000000785D0098 > 00000000785d00f0: 0x00000001 > > [3]: KVM deliver "BUS_MCEERR_AR" to Qemu, Qemu record the guest CPER and inject > synchronous external abort to notify guest, then guest do the recovery. > > [ 1552.516170] Synchronous External Abort: synchronous external abort (0x92000410) at 0x000000003751c6b4 > [ 1553.074073] {1}[Hardware Error]: Hardware error from APEI Generic Hardware Error Source: 8 > [ 1553.081654] {1}[Hardware Error]: event severity: recoverable > [ 1554.034191] {1}[Hardware Error]: Error 0, type: recoverable > [ 1554.037934] {1}[Hardware Error]: section_type: memory error > [ 1554.513261] {1}[Hardware Error]: physical_address: 0x0000000040fa6000 > [ 1554.513944] {1}[Hardware Error]: error_type: 0, unknown > [ 1555.041451] Memory failure: 0x40fa6: Killing mca-recover:1296 due to hardware memory corruption > [ 1555.373116] Memory failure: 0x40fa6: recovery action for dirty LRU page: Recovered > > Dongjiu Geng (9): > hw/arm/virt: Introduce a RAS machine option > docs: APEI GHES generation and CPER record description > ACPI: Build related register address fields via hardware error fw_cfg > blob > ACPI: Build Hardware Error Source Table > ACPI: Record the Generic Error Status Block address > KVM: Move hwpoison page related functions into kvm-all.c > ACPI: Record Generic Error Status Block(GESB) table > target-arm: kvm64: handle SIGBUS signal from kernel or KVM > MAINTAINERS: Add ACPI/HEST/GHES entries > > MAINTAINERS | 9 + > accel/kvm/kvm-all.c | 36 +++ > default-configs/arm-softmmu.mak | 1 + > docs/specs/acpi_hest_ghes.rst | 110 ++++++++ > docs/specs/index.rst | 1 + > hw/acpi/Kconfig | 4 + > hw/acpi/Makefile.objs | 1 + > hw/acpi/aml-build.c | 2 + > hw/acpi/generic_event_device.c | 15 +- > hw/acpi/ghes.c | 446 +++++++++++++++++++++++++++++++++ > hw/arm/virt-acpi-build.c | 19 ++ > hw/arm/virt.c | 23 ++ > include/hw/acpi/aml-build.h | 1 + > include/hw/acpi/generic_event_device.h | 2 + > include/hw/acpi/ghes.h | 75 ++++++ > include/hw/arm/virt.h | 1 + > include/qemu/uuid.h | 5 + > include/sysemu/kvm.h | 3 +- > include/sysemu/kvm_int.h | 12 + > target/arm/cpu.h | 4 + > target/arm/helper.c | 2 +- > target/arm/internals.h | 5 +- > target/arm/kvm64.c | 66 +++++ > target/arm/tlb_helper.c | 2 +- > target/i386/cpu.h | 2 + > target/i386/kvm.c | 36 --- > 26 files changed, 840 insertions(+), 43 deletions(-) > create mode 100644 docs/specs/acpi_hest_ghes.rst > create mode 100644 hw/acpi/ghes.c > create mode 100644 include/hw/acpi/ghes.h >
On Wed, 8 Jan 2020 19:32:16 +0800 Dongjiu Geng <gengdongjiu@huawei.com> wrote: > Add APEI/GHES detailed design document > > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com> Reviewed-by: Igor Mammedov <imammedo@redhat.com> > --- > docs/specs/acpi_hest_ghes.rst | 110 ++++++++++++++++++++++++++++++++++++++++++ > docs/specs/index.rst | 1 + > 2 files changed, 111 insertions(+) > create mode 100644 docs/specs/acpi_hest_ghes.rst > > diff --git a/docs/specs/acpi_hest_ghes.rst b/docs/specs/acpi_hest_ghes.rst > new file mode 100644 > index 0000000..7a1aa90 > --- /dev/null > +++ b/docs/specs/acpi_hest_ghes.rst > @@ -0,0 +1,110 @@ > +APEI tables generating and CPER record > +====================================== > + > +.. > + Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. > + > + This work is licensed under the terms of the GNU GPL, version 2 or later. > + See the COPYING file in the top-level directory. > + > +Design Details > +-------------- > + > +:: > + > + etc/acpi/tables etc/hardware_errors > + ==================== =============================== > + + +--------------------------+ +----------------------------+ > + | | HEST | +--------->| error_block_address1 |------+ > + | +--------------------------+ | +----------------------------+ | > + | | GHES1 | | +------->| error_block_address2 |------+-+ > + | +--------------------------+ | | +----------------------------+ | | > + | | ................. | | | | .............. | | | > + | | error_status_address-----+-+ | -----------------------------+ | | > + | | ................. | | +--->| error_block_addressN |------+-+---+ > + | | read_ack_register--------+-+ | | +----------------------------+ | | | > + | | read_ack_preserve | +-+---+--->| read_ack_register1 | | | | > + | | read_ack_write | | | +----------------------------+ | | | > + + +--------------------------+ | +-+--->| read_ack_register2 | | | | > + | | GHES2 | | | | +----------------------------+ | | | > + + +--------------------------+ | | | | ............. | | | | > + | | ................. | | | | +----------------------------+ | | | > + | | error_status_address-----+---+ | | +->| read_ack_registerN | | | | > + | | ................. | | | | +----------------------------+ | | | > + | | read_ack_register--------+-----+ | | |Generic Error Status Block 1|<-----+ | | > + | | read_ack_preserve | | | |-+------------------------+-+ | | > + | | read_ack_write | | | | | CPER | | | | > + + +--------------------------| | | | | CPER | | | | > + | | ............... | | | | | .... | | | | > + + +--------------------------+ | | | | CPER | | | | > + | | GHESN | | | |-+------------------------+-| | | > + + +--------------------------+ | | |Generic Error Status Block 2|<-------+ | > + | | ................. | | | |-+------------------------+-+ | > + | | error_status_address-----+-------+ | | | CPER | | | > + | | ................. | | | | CPER | | | > + | | read_ack_register--------+---------+ | | .... | | | > + | | read_ack_preserve | | | CPER | | | > + | | read_ack_write | +-+------------------------+-+ | > + + +--------------------------+ | .......... | | > + |----------------------------+ | > + |Generic Error Status Block N |<----------+ > + |-+-------------------------+-+ > + | | CPER | | > + | | CPER | | > + | | .... | | > + | | CPER | | > + +-+-------------------------+-+ > + > + > +(1) QEMU generates the ACPI HEST table. This table goes in the current > + "etc/acpi/tables" fw_cfg blob. Each error source has different > + notification types. > + > +(2) A new fw_cfg blob called "etc/hardware_errors" is introduced. QEMU > + also needs to populate this blob. The "etc/hardware_errors" fw_cfg blob > + contains an address registers table and an Error Status Data Block table. > + > +(3) The address registers table contains N Error Block Address entries > + and N Read Ack Register entries. The size for each entry is 8-byte. > + The Error Status Data Block table contains N Error Status Data Block > + entries. The size for each entry is 4096(0x1000) bytes. The total size > + for the "etc/hardware_errors" fw_cfg blob is (N * 8 * 2 + N * 4096) bytes. > + N is the number of the kinds of hardware error sources. > + > +(4) QEMU generates the ACPI linker/loader script for the firmware. The > + firmware pre-allocates memory for "etc/acpi/tables", "etc/hardware_errors" > + and copies blob contents there. > + > +(5) QEMU generates N ADD_POINTER commands, which patch addresses in the > + "error_status_address" fields of the HEST table with a pointer to the > + corresponding "address registers" in the "etc/hardware_errors" blob. > + > +(6) QEMU generates N ADD_POINTER commands, which patch addresses in the > + "read_ack_register" fields of the HEST table with a pointer to the > + corresponding "read_ack_register" within the "etc/hardware_errors" blob. > + > +(7) QEMU generates N ADD_POINTER commands for the firmware, which patch > + addresses in the "error_block_address" fields with a pointer to the > + respective "Error Status Data Block" in the "etc/hardware_errors" blob. > + > +(8) QEMU defines a third and write-only fw_cfg blob which is called > + "etc/hardware_errors_addr". Through that blob, the firmware can send back > + the guest-side allocation addresses to QEMU. The "etc/hardware_errors_addr" > + blob contains a 8-byte entry. QEMU generates a single WRITE_POINTER command > + for the firmware. The firmware will write back the start address of > + "etc/hardware_errors" blob to the fw_cfg file "etc/hardware_errors_addr". > + > +(9) When QEMU gets a SIGBUS from the kernel, QEMU writes CPER into corresponding > + "Error Status Data Block", guest memory, and then injects platform specific > + interrupt (in case of arm/virt machine it's Synchronous External Abort) as a > + notification which is necessary for notifying the guest. > + > +(10) This notification (in virtual hardware) will be handled by the guest > + kernel, on receiving notification, guest APEI driver could read the CPER error > + and take appropriate action. > + > +(11) kvm_arch_on_sigbus_vcpu() uses source_id as index in "etc/hardware_errors" to > + find out "Error Status Data Block" entry corresponding to error source. So supported > + source_id values should be assigned here and not be changed afterwards to make sure > + that guest will write error into expected "Error Status Data Block" even if guest was > + migrated to a newer QEMU. > diff --git a/docs/specs/index.rst b/docs/specs/index.rst > index 984ba44..3019b9c 100644 > --- a/docs/specs/index.rst > +++ b/docs/specs/index.rst > @@ -13,3 +13,4 @@ Contents: > ppc-xive > ppc-spapr-xive > acpi_hw_reduced_hotplug > + acpi_hest_ghes
On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng <gengdongjiu@huawei.com> wrote: > +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) > +{ > + ram_addr_t ram_addr; > + hwaddr paddr; > + > + assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO); > + > + if (acpi_enabled && addr && > + object_property_get_bool(qdev_get_machine(), "ras", NULL)) { > + ram_addr = qemu_ram_addr_from_host(addr); > + if (ram_addr != RAM_ADDR_INVALID && > + kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) { > + kvm_hwpoison_page_add(ram_addr); > + /* > + * Asynchronous signal will be masked by main thread, so > + * only handle synchronous signal. > + */ I don't understand this comment. (I think we've had discussions about it before, but it's still not clear to me.) This function (kvm_arch_on_sigbus_vcpu()) will be called in two contexts: (1) in the vcpu thread: * the real SIGBUS handler sigbus_handler() sets a flag and arranges for an immediate vcpu exit * the vcpu thread reads the flag on exit from KVM_RUN and calls kvm_arch_on_sigbus_vcpu() directly * the error could be MCEERR_AR or MCEERR_AO (2) MCE errors on other threads: * here SIGBUS is blocked, so MCEERR_AR (action-required) errors will cause the kernel to just kill the QEMU process * MCEERR_AO errors will be handled via the iothread's use of signalfd(), so kvm_on_sigbus() will get called from the main thread, and it will call kvm_arch_on_sigbus_vcpu() * in this case the passed in CPUState will (arbitrarily) be that for the first vCPU For MCEERR_AR, the code here looks correct -- we know this is only going to happen for the relevant vCPU so we can go ahead and do the "record it in the ACPI table and tell the guest" bit. But shouldn't we be doing something with the MCEERR_AO too? That of course will be trickier because we're not necessarily in the vcpu thread, but it would be nice to let the guest know about it. One comment that would work with the current code would be: /* * If this is a BUS_MCEERR_AR, we know we have been called * synchronously from the vCPU thread, so we can easily * synchronize the state and inject an error. * * TODO: we currently don't tell the guest at all about BUS_MCEERR_AO. * In that case we might either be being called synchronously from * the vCPU thread, or a bit later from the main thread, so doing * the injection of the error would be more complicated. */ but I don't know if that's what you meant to say/implement... > + if (code == BUS_MCEERR_AR) { > + kvm_cpu_synchronize_state(c); > + if (!acpi_ghes_record_errors(ACPI_HEST_SRC_ID_SEA, paddr)) { > + kvm_inject_arm_sea(c); > + } else { > + error_report("failed to record the error"); > + abort(); > + } > + } > + return; > + } > + if (code == BUS_MCEERR_AO) { > + error_report("Hardware memory error at addr %p for memory used by " > + "QEMU itself instead of guest system!", addr); > + } > + } > + > + if (code == BUS_MCEERR_AR) { > + error_report("Hardware memory error!"); > + exit(1); > + } > +} > thanks -- PMM
On Thu, 16 Jan 2020 at 16:28, Peter Maydell <peter.maydell@linaro.org> wrote:
> This function (kvm_arch_on_sigbus_vcpu()) will be called in two contexts:
>
> (1) in the vcpu thread:
> * the real SIGBUS handler sigbus_handler() sets a flag and arranges
> for an immediate vcpu exit
> * the vcpu thread reads the flag on exit from KVM_RUN and
> calls kvm_arch_on_sigbus_vcpu() directly
> * the error could be MCEERR_AR or MCEERR_AO
> (2) MCE errors on other threads:
> * here SIGBUS is blocked, so MCEERR_AR (action-required)
> errors will cause the kernel to just kill the QEMU process
> * MCEERR_AO errors will be handled via the iothread's use
> of signalfd(), so kvm_on_sigbus() will get called from
> the main thread, and it will call kvm_arch_on_sigbus_vcpu()
> * in this case the passed in CPUState will (arbitrarily) be that
> for the first vCPU
>
> For MCEERR_AR, the code here looks correct -- we know this is
> only going to happen for the relevant vCPU so we can go ahead
> and do the "record it in the ACPI table and tell the guest" bit.
>
> But shouldn't we be doing something with the MCEERR_AO too?
> That of course will be trickier because we're not necessarily
> in the vcpu thread, but it would be nice to let the guest
> know about it.
An IRC discussion with Paolo came to the conclusion that
the nicest approach here would be for kvm_on_sigbus() to
use run_on_cpu() to call the whole of kvm_arch_on_sigbus_vcpu()
in the vcpu thread for the cpu it gets passed. Then the code
here would not have to worry about the "not on the right thread"
case. This would be a refactoring of the x86 code, which currently
does the run_on_cpu inside its implementation, in
cpu_x86_inject_mce().
thanks
-- PMM
On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
>
> Record the GHEB address via fw_cfg file, when recording
> a error to CPER, it will use this address to find out
> Generic Error Data Entries and write the error.
>
> Make the HEST GHES to a GED device.
>
> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com>
> ---
> hw/acpi/generic_event_device.c | 15 ++++++++++++++-
> hw/acpi/ghes.c | 16 ++++++++++++++++
> hw/arm/virt-acpi-build.c | 13 ++++++++++++-
> include/hw/acpi/generic_event_device.h | 2 ++
> include/hw/acpi/ghes.h | 6 ++++++
> 5 files changed, 50 insertions(+), 2 deletions(-)
>
> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c
> index 9cee90c..9bf37e4 100644
> --- a/hw/acpi/generic_event_device.c
> +++ b/hw/acpi/generic_event_device.c
> @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = {
> }
> };
>
> +static const VMStateDescription vmstate_ghes_state = {
> + .name = "acpi-ghes-state",
> + .version_id = 1,
> + .minimum_version_id = 1,
> + .fields = (VMStateField[]) {
> + VMSTATE_UINT64(ghes_addr_le, AcpiGhesState),
> + VMSTATE_END_OF_LIST()
> + }
> +};
> +
> static const VMStateDescription vmstate_acpi_ged = {
> .name = "acpi-ged",
> .version_id = 1,
> .minimum_version_id = 1,
> .fields = (VMStateField[]) {
> - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState),
> + VMSTATE_STRUCT(ged_state, AcpiGedState, 1,
> + vmstate_ged_state, GEDState),
> + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1,
> + vmstate_ghes_state, AcpiGhesState),
> VMSTATE_END_OF_LIST(),
> },
> .subsections = (const VMStateDescription * []) {
You can't just add fields to an existing VMStateDescription
like this -- it will break migration compatibility. Instead you
need to add a new subsection to this vmstate, with a '.needed'
function which indicates when the subsection should be present.
thanks
-- PMM
On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
>
> I and Xiang are willing to review the APEI-related patches and
> volunteer as the reviewers for the HEST/GHES part.
>
> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com>
> ---
> MAINTAINERS | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 387879a..5af70a5 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c
> F: tests/acpi-utils.[hc]
> F: tests/data/acpi/
>
> +ACPI/HEST/GHES
> +R: Dongjiu Geng <gengdongjiu@huawei.com>
> +R: Xiang Zheng <zhengxiang9@huawei.com>
> +L: qemu-arm@nongnu.org
> +S: Maintained
> +F: hw/acpi/ghes.c
> +F: include/hw/acpi/ghes.h
> +F: docs/specs/acpi_hest_ghes.rst
> +
> ppc4xx
> M: David Gibson <david@gibson.dropbear.id.au>
> L: qemu-ppc@nongnu.org
> --
Michael, Igor: since this new MAINTAINERS section is
moving files out of the 'ACPI/SMBIOS' section that you're
currently responsible for, do you want to provide an
acked-by: that you think this division of files makes sense?
thanks
-- PMM
Hi Peter,
On 1/16/20 5:46 PM, Peter Maydell wrote:
> On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
>>
>> I and Xiang are willing to review the APEI-related patches and
>> volunteer as the reviewers for the HEST/GHES part.
>>
>> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
>> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com>
>> ---
>> MAINTAINERS | 9 +++++++++
>> 1 file changed, 9 insertions(+)
>>
>> diff --git a/MAINTAINERS b/MAINTAINERS
>> index 387879a..5af70a5 100644
>> --- a/MAINTAINERS
>> +++ b/MAINTAINERS
>> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c
>> F: tests/acpi-utils.[hc]
>> F: tests/data/acpi/
>>
>> +ACPI/HEST/GHES
>> +R: Dongjiu Geng <gengdongjiu@huawei.com>
>> +R: Xiang Zheng <zhengxiang9@huawei.com>
>> +L: qemu-arm@nongnu.org
>> +S: Maintained
>> +F: hw/acpi/ghes.c
>> +F: include/hw/acpi/ghes.h
>> +F: docs/specs/acpi_hest_ghes.rst
>> +
>> ppc4xx
>> M: David Gibson <david@gibson.dropbear.id.au>
>> L: qemu-ppc@nongnu.org
>> --
>
> Michael, Igor: since this new MAINTAINERS section is
> moving files out of the 'ACPI/SMBIOS' section that you're
> currently responsible for, do you want to provide an
> acked-by: that you think this division of files makes sense?
The files are not 'moved out', Michael and Igor are still the
maintainers of the supported ACPI/SMBIOS subsystem:
ACPI/SMBIOS
M: Michael S. Tsirkin <mst@redhat.com>
M: Igor Mammedov <imammedo@redhat.com>
S: Supported
F: include/hw/acpi/*
F: hw/acpi/*
Dongjiu and Xiang only add themselves as reviewers to get notified on
changes on these specific files. The more eyes the better :)
The docs/specs/acpi_hest_ghes.rst document has no maintainer, as these
others too:
- docs/specs/acpi_cpu_hotplug.txt
- docs/specs/acpi_hw_reduced_hotplug.rst
- docs/specs/acpi_mem_hotplug.txt
- docs/specs/acpi_nvdimm.txt
The only ACPI file reported as maintained in docs/specs/ is
acpi_pci_hotplug.txt, from this entry:
PCI
M: Michael S. Tsirkin <mst@redhat.com>
M: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
S: Supported
F: docs/specs/*pci*
FWIW:
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
On 1/8/20 12:32 PM, Dongjiu Geng wrote: > Record the GHEB address via fw_cfg file, when recording > a error to CPER, it will use this address to find out > Generic Error Data Entries and write the error. > > Make the HEST GHES to a GED device. > > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > --- > hw/acpi/generic_event_device.c | 15 ++++++++++++++- > hw/acpi/ghes.c | 16 ++++++++++++++++ > hw/arm/virt-acpi-build.c | 13 ++++++++++++- > include/hw/acpi/generic_event_device.h | 2 ++ > include/hw/acpi/ghes.h | 6 ++++++ > 5 files changed, 50 insertions(+), 2 deletions(-) > > diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c > index 9cee90c..9bf37e4 100644 > --- a/hw/acpi/generic_event_device.c > +++ b/hw/acpi/generic_event_device.c > @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = { > } > }; > > +static const VMStateDescription vmstate_ghes_state = { > + .name = "acpi-ghes-state", > + .version_id = 1, > + .minimum_version_id = 1, > + .fields = (VMStateField[]) { > + VMSTATE_UINT64(ghes_addr_le, AcpiGhesState), > + VMSTATE_END_OF_LIST() > + } > +}; > + > static const VMStateDescription vmstate_acpi_ged = { > .name = "acpi-ged", > .version_id = 1, > .minimum_version_id = 1, > .fields = (VMStateField[]) { > - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState), > + VMSTATE_STRUCT(ged_state, AcpiGedState, 1, > + vmstate_ged_state, GEDState), > + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, > + vmstate_ghes_state, AcpiGhesState), > VMSTATE_END_OF_LIST(), > }, > .subsections = (const VMStateDescription * []) { > diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c > index 9d37798..68f4abf 100644 > --- a/hw/acpi/ghes.c > +++ b/hw/acpi/ghes.c > @@ -23,6 +23,7 @@ > #include "hw/acpi/acpi.h" > #include "hw/acpi/ghes.h" > #include "hw/acpi/aml-build.h" > +#include "hw/acpi/generic_event_device.h" > #include "hw/nvram/fw_cfg.h" > #include "sysemu/sysemu.h" > #include "qemu/error-report.h" > @@ -208,3 +209,18 @@ void acpi_build_hest(GArray *table_data, GArray *hardware_errors, > build_header(linker, table_data, (void *)(table_data->data + hest_start), > "HEST", table_data->len - hest_start, 1, NULL, ""); > } > + > +void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, > + GArray *hardware_error) > +{ > + size_t size = 2 * sizeof(uint64_t) + ACPI_GHES_MAX_RAW_DATA_LENGTH; > + size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size; > + > + /* Create a read-only fw_cfg file for GHES */ > + fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data, > + request_block_size); > + > + /* Create a read-write fw_cfg file for Address */ > + fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, > + NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); > +} > diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c > index 837bbf9..c8aa94d 100644 > --- a/hw/arm/virt-acpi-build.c > +++ b/hw/arm/virt-acpi-build.c > @@ -797,6 +797,7 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) > unsigned dsdt, xsdt; > GArray *tables_blob = tables->table_data; > MachineState *ms = MACHINE(vms); > + AcpiGedState *acpi_ged_state; > > table_offsets = g_array_new(false, true /* clear */, > sizeof(uint32_t)); > @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) > acpi_add_table(table_offsets, tables_blob); > build_spcr(tables_blob, tables->linker, vms); > > - if (vms->ras) { > + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, > + NULL)); Testing vms->ras first is cheaper than calling object_resolve_path_type(). Since some people are spending lot of time to reduce VM boot time, it might be worth considering. > + if (acpi_ged_state && vms->ras) { > acpi_add_table(table_offsets, tables_blob); > build_ghes_error_table(tables->hardware_errors, tables->linker); > acpi_build_hest(tables_blob, tables->hardware_errors, > @@ -925,6 +928,7 @@ void virt_acpi_setup(VirtMachineState *vms) > { > AcpiBuildTables tables; > AcpiBuildState *build_state; > + AcpiGedState *acpi_ged_state; > > if (!vms->fw_cfg) { > trace_virt_acpi_setup(); > @@ -955,6 +959,13 @@ void virt_acpi_setup(VirtMachineState *vms) > fw_cfg_add_file(vms->fw_cfg, ACPI_BUILD_TPMLOG_FILE, tables.tcpalog->data, > acpi_data_len(tables.tcpalog)); > > + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, > + NULL)); > + if (acpi_ged_state && vms->ras) { > + acpi_ghes_add_fw_cfg(&acpi_ged_state->ghes_state, > + vms->fw_cfg, tables.hardware_errors); > + } > + > build_state->rsdp_mr = acpi_add_rom_blob(virt_acpi_build_update, > build_state, tables.rsdp, > ACPI_BUILD_RSDP_FILE, 0); > diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h > index d157eac..037d2b5 100644 > --- a/include/hw/acpi/generic_event_device.h > +++ b/include/hw/acpi/generic_event_device.h > @@ -61,6 +61,7 @@ > > #include "hw/sysbus.h" > #include "hw/acpi/memory_hotplug.h" > +#include "hw/acpi/ghes.h" > > #define ACPI_POWER_BUTTON_DEVICE "PWRB" > > @@ -95,6 +96,7 @@ typedef struct AcpiGedState { > GEDState ged_state; > uint32_t ged_event_bitmap; > qemu_irq irq; > + AcpiGhesState ghes_state; > } AcpiGedState; > > void build_ged_aml(Aml *table, const char* name, HotplugHandler *hotplug_dev, > diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h > index 09a7f86..a6761e6 100644 > --- a/include/hw/acpi/ghes.h > +++ b/include/hw/acpi/ghes.h > @@ -60,7 +60,13 @@ enum { > ACPI_HEST_SRC_ID_RESERVED, > }; > > +typedef struct AcpiGhesState { > + uint64_t ghes_addr_le; > +} AcpiGhesState; > + > void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); > void acpi_build_hest(GArray *table_data, GArray *hardware_error, > BIOSLinker *linker); > +void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, > + GArray *hardware_errors); > #endif >
On 2020/1/17 0:28, Peter Maydell wrote: > On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng <gengdongjiu@huawei.com> wrote: > >> +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) >> +{ >> + ram_addr_t ram_addr; >> + hwaddr paddr; >> + >> + assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO); >> + >> + if (acpi_enabled && addr && >> + object_property_get_bool(qdev_get_machine(), "ras", NULL)) { >> + ram_addr = qemu_ram_addr_from_host(addr); >> + if (ram_addr != RAM_ADDR_INVALID && >> + kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) { >> + kvm_hwpoison_page_add(ram_addr); >> + /* >> + * Asynchronous signal will be masked by main thread, so >> + * only handle synchronous signal. >> + */ > > I don't understand this comment. (I think we've had discussions > about it before, but it's still not clear to me.) > > This function (kvm_arch_on_sigbus_vcpu()) will be called in two contexts: > > (1) in the vcpu thread: > * the real SIGBUS handler sigbus_handler() sets a flag and arranges > for an immediate vcpu exit > * the vcpu thread reads the flag on exit from KVM_RUN and > calls kvm_arch_on_sigbus_vcpu() directly > * the error could be MCEERR_AR or MCEERR_AOFor the vcpu thread, the error can be MCEERR_AR or MCEERR_AO, but kernel/KVM usually uses MCEERR_AR(action required) instead of MCEERR_AO, because it needs do action immediately. For MCEERR_AO error, the action is optional and the error can be ignored. At least I do not find Linux kernel/KVM deliver MCEERR_AO in the vcpu threads. > (2) MCE errors on other threads: > * here SIGBUS is blocked, so MCEERR_AR (action-required) > errors will cause the kernel to just kill the QEMU process > * MCEERR_AO errors will be handled via the iothread's use > of signalfd(), so kvm_on_sigbus() will get called from > the main thread, and it will call kvm_arch_on_sigbus_vcpu() > * in this case the passed in CPUState will (arbitrarily) be that > for the first vCPU For the MCE errors on other threads, it can only handle MCEERR_AO. If it is MCEERR_AR, the QEMU will assert and exit[2]. Case1: Other APP indeed can send MCEERR_AO to QEMU, QEMU handle it via the iothread's use of signalfd() through above path. Case2: But if the MCEERR_AO is delivered by kernel, I see QEMU ignore it because SIGBUS is masked in main thread[3], for this case, I do not see QEMU handle it via signalfd() for MCEERR_AO errors from my test. For Case1,I think we should not let guest know it, because it is not triggered by guest. only other APP send SIGBUS to tell QEMU do somethings. For Case2,it does not call call kvm_arch_on_sigbus_vcpu(). [1]: /* Called synchronously (via signalfd) in main thread. */ int kvm_on_sigbus(int code, void *addr) { #ifdef KVM_HAVE_MCE_INJECTION /* Action required MCE kills the process if SIGBUS is blocked. Because * that's what happens in the I/O thread, where we handle MCE via signalfd, * we can only get action optional here. */ [2]: assert(code != BUS_MCEERR_AR); kvm_arch_on_sigbus_vcpu(first_cpu, code, addr); return 0; #else return 1; #endif } [3]: https://lists.gnu.org/archive/html/qemu-devel/2017-11/msg03575.html > > For MCEERR_AR, the code here looks correct -- we know this is > only going to happen for the relevant vCPU so we can go ahead > and do the "record it in the ACPI table and tell the guest" bit. > > But shouldn't we be doing something with the MCEERR_AO too? Above all, from my test, for MCEERR_AO error which is triggered by guest, it not call kvm_arch_on_sigbus_vcpu(). so I think currently we can just report error. If afterwards MCEERR_AO error can call kvm_arch_on_sigbus_vcpu(), we can let the guest know about it. > That of course will be trickier because we're not necessarily > in the vcpu thread, but it would be nice to let the guest > know about it. > > One comment that would work with the current code would be: > > /* > * If this is a BUS_MCEERR_AR, we know we have been called > * synchronously from the vCPU thread, so we can easily > * synchronize the state and inject an error. > * > * TODO: we currently don't tell the guest at all about BUS_MCEERR_AO. > * In that case we might either be being called synchronously from > * the vCPU thread, or a bit later from the main thread, so doing At least I do not find Linux kernel/KVM deliver MCEERR_AO in the vcpu threads. In the main thread, signalfd() is not called when it is BUS_MCEERR_AO which is triggered by guest. > * the injection of the error would be more complicated. > */ > > but I don't know if that's what you meant to say/implement... we can implement MCEERR_AO logic when QEMU can receive MCEERR_AO error which is triggered by guest. > >> + if (code == BUS_MCEERR_AR) { >> + kvm_cpu_synchronize_state(c); >> + if (!acpi_ghes_record_errors(ACPI_HEST_SRC_ID_SEA, paddr)) { >> + kvm_inject_arm_sea(c); >> + } else { >> + error_report("failed to record the error"); >> + abort(); >> + } >> + } >> + return; >> + } >> + if (code == BUS_MCEERR_AO) { >> + error_report("Hardware memory error at addr %p for memory used by " >> + "QEMU itself instead of guest system!", addr); >> + } >> + } >> + >> + if (code == BUS_MCEERR_AR) { >> + error_report("Hardware memory error!"); >> + exit(1); >> + } >> +} >> > > thanks > -- PMM > . >
On 2020/1/17 15:22, Philippe Mathieu-Daudé wrote: > Hi Peter, > > On 1/16/20 5:46 PM, Peter Maydell wrote: >> On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng <gengdongjiu@huawei.com> wrote: >>> >>> I and Xiang are willing to review the APEI-related patches and >>> volunteer as the reviewers for the HEST/GHES part. >>> >>> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >>> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >>> --- >>> MAINTAINERS | 9 +++++++++ >>> 1 file changed, 9 insertions(+) >>> >>> diff --git a/MAINTAINERS b/MAINTAINERS >>> index 387879a..5af70a5 100644 >>> --- a/MAINTAINERS >>> +++ b/MAINTAINERS >>> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c >>> F: tests/acpi-utils.[hc] >>> F: tests/data/acpi/ >>> >>> +ACPI/HEST/GHES >>> +R: Dongjiu Geng <gengdongjiu@huawei.com> >>> +R: Xiang Zheng <zhengxiang9@huawei.com> >>> +L: qemu-arm@nongnu.org >>> +S: Maintained >>> +F: hw/acpi/ghes.c >>> +F: include/hw/acpi/ghes.h >>> +F: docs/specs/acpi_hest_ghes.rst >>> + >>> ppc4xx >>> M: David Gibson <david@gibson.dropbear.id.au> >>> L: qemu-ppc@nongnu.org >>> -- >> >> Michael, Igor: since this new MAINTAINERS section is >> moving files out of the 'ACPI/SMBIOS' section that you're >> currently responsible for, do you want to provide an >> acked-by: that you think this division of files makes sense? > > The files are not 'moved out', Michael and Igor are still the maintainers of the supported ACPI/SMBIOS subsystem: > > ACPI/SMBIOS > M: Michael S. Tsirkin <mst@redhat.com> > M: Igor Mammedov <imammedo@redhat.com> > S: Supported > F: include/hw/acpi/* > F: hw/acpi/* > > Dongjiu and Xiang only add themselves as reviewers to get notified on changes on these specific files. The more eyes the better :) > > The docs/specs/acpi_hest_ghes.rst document has no maintainer, as these others too: If this file has no maintainer, may be it needs a M tag for this file, otherwise when people change this file, and use "./scripts/get_maintainer.pl xxxxx" to get maintainer, it will be empty. > > - docs/specs/acpi_cpu_hotplug.txt > - docs/specs/acpi_hw_reduced_hotplug.rst > - docs/specs/acpi_mem_hotplug.txt > - docs/specs/acpi_nvdimm.txt > > The only ACPI file reported as maintained in docs/specs/ is acpi_pci_hotplug.txt, from this entry: > > PCI > M: Michael S. Tsirkin <mst@redhat.com> > M: Marcel Apfelbaum <marcel.apfelbaum@gmail.com> > S: Supported > F: docs/specs/*pci* > > FWIW: > Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> > > . >
On 2020/1/17 0:44, Peter Maydell wrote: >> .minimum_version_id = 1, >> .fields = (VMStateField[]) { >> - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState), >> + VMSTATE_STRUCT(ged_state, AcpiGedState, 1, >> + vmstate_ged_state, GEDState), >> + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, >> + vmstate_ghes_state, AcpiGhesState), >> VMSTATE_END_OF_LIST(), >> }, >> .subsections = (const VMStateDescription * []) { > You can't just add fields to an existing VMStateDescription > like this -- it will break migration compatibility. Instead you > need to add a new subsection to this vmstate, with a '.needed' > function which indicates when the subsection should be present. Thanks Peter's pointing out. I will change it. > > thanks > -- PMM > . >
On 2020/1/17 15:39, Philippe Mathieu-Daudé wrote: >> table_offsets = g_array_new(false, true /* clear */, >> sizeof(uint32_t)); >> @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) >> acpi_add_table(table_offsets, tables_blob); >> build_spcr(tables_blob, tables->linker, vms); >> - if (vms->ras) { >> + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, >> + NULL)); > > Testing vms->ras first is cheaper than calling object_resolve_path_type(). Since some people are spending lot of time to reduce VM boot time, it might be worth considering. Thanks Philippe's comments. Do you think it should be written to below[1]? right? [1]: if (vms->ras && acpi_ged_state) > >> + if (acpi_ged_state && vms->ras) { >> acpi_add_table(table_offsets, tables_blob); >> build_ghes_error_table(tables->hardware_errors, tables->linker); >> acpi_build_hest(tables_blob, tables->hardware_errors, >> @@ -925,6 +928,7 @@ void virt_acpi_setup(VirtMachineState *vms) >> {
On Fri, 17 Jan 2020 at 07:22, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>
> Hi Peter,
>
> On 1/16/20 5:46 PM, Peter Maydell wrote:
> > On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
> >>
> >> I and Xiang are willing to review the APEI-related patches and
> >> volunteer as the reviewers for the HEST/GHES part.
> >>
> >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
> >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com>
> >> ---
> >> MAINTAINERS | 9 +++++++++
> >> 1 file changed, 9 insertions(+)
> >>
> >> diff --git a/MAINTAINERS b/MAINTAINERS
> >> index 387879a..5af70a5 100644
> >> --- a/MAINTAINERS
> >> +++ b/MAINTAINERS
> >> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c
> >> F: tests/acpi-utils.[hc]
> >> F: tests/data/acpi/
> >>
> >> +ACPI/HEST/GHES
> >> +R: Dongjiu Geng <gengdongjiu@huawei.com>
> >> +R: Xiang Zheng <zhengxiang9@huawei.com>
> >> +L: qemu-arm@nongnu.org
> >> +S: Maintained
> >> +F: hw/acpi/ghes.c
> >> +F: include/hw/acpi/ghes.h
> >> +F: docs/specs/acpi_hest_ghes.rst
> >> +
> >> ppc4xx
> >> M: David Gibson <david@gibson.dropbear.id.au>
> >> L: qemu-ppc@nongnu.org
> >> --
> >
> > Michael, Igor: since this new MAINTAINERS section is
> > moving files out of the 'ACPI/SMBIOS' section that you're
> > currently responsible for, do you want to provide an
> > acked-by: that you think this division of files makes sense?
>
> The files are not 'moved out', Michael and Igor are still the
> maintainers of the supported ACPI/SMBIOS subsystem:
Does get_maintainer.pl print the answers for all matching
sections, rather than just the most specific, then?
In any case, I'd still like an acked-by from them.
thanks
-- PMM
On 1/17/20 11:47 AM, gengdongjiu wrote: > On 2020/1/17 15:39, Philippe Mathieu-Daudé wrote: >>> table_offsets = g_array_new(false, true /* clear */, >>> sizeof(uint32_t)); >>> @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) >>> acpi_add_table(table_offsets, tables_blob); >>> build_spcr(tables_blob, tables->linker, vms); >>> - if (vms->ras) { >>> + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, >>> + NULL)); >> >> Testing vms->ras first is cheaper than calling object_resolve_path_type(). Since some people are spending lot of time to reduce VM boot time, it might be worth considering. > Thanks Philippe's comments. > > Do you think it should be written to below[1]? right? > > [1]: > if (vms->ras && acpi_ged_state) No, as: if (vms->ras) { AcpiGedState *acpi_ged_state; acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, NULL)); if (acpi_ged_state) { acpi_add_table(table_offsets, tables_blob); ... Maybe I'm wrong and there is not much impact in VM boot time here, reviews welcomed :) >> >>> + if (acpi_ged_state && vms->ras) { >>> acpi_add_table(table_offsets, tables_blob); >>> build_ghes_error_table(tables->hardware_errors, tables->linker); >>> acpi_build_hest(tables_blob, tables->hardware_errors, >>> @@ -925,6 +928,7 @@ void virt_acpi_setup(VirtMachineState *vms) >>> { >
On 1/17/20 12:09 PM, Peter Maydell wrote: > On Fri, 17 Jan 2020 at 07:22, Philippe Mathieu-Daudé <philmd@redhat.com> wrote: >> >> Hi Peter, >> >> On 1/16/20 5:46 PM, Peter Maydell wrote: >>> On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng <gengdongjiu@huawei.com> wrote: >>>> >>>> I and Xiang are willing to review the APEI-related patches and >>>> volunteer as the reviewers for the HEST/GHES part. >>>> >>>> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >>>> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >>>> --- >>>> MAINTAINERS | 9 +++++++++ >>>> 1 file changed, 9 insertions(+) >>>> >>>> diff --git a/MAINTAINERS b/MAINTAINERS >>>> index 387879a..5af70a5 100644 >>>> --- a/MAINTAINERS >>>> +++ b/MAINTAINERS >>>> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c >>>> F: tests/acpi-utils.[hc] >>>> F: tests/data/acpi/ >>>> >>>> +ACPI/HEST/GHES >>>> +R: Dongjiu Geng <gengdongjiu@huawei.com> >>>> +R: Xiang Zheng <zhengxiang9@huawei.com> >>>> +L: qemu-arm@nongnu.org >>>> +S: Maintained >>>> +F: hw/acpi/ghes.c >>>> +F: include/hw/acpi/ghes.h >>>> +F: docs/specs/acpi_hest_ghes.rst >>>> + >>>> ppc4xx >>>> M: David Gibson <david@gibson.dropbear.id.au> >>>> L: qemu-ppc@nongnu.org >>>> -- >>> >>> Michael, Igor: since this new MAINTAINERS section is >>> moving files out of the 'ACPI/SMBIOS' section that you're >>> currently responsible for, do you want to provide an >>> acked-by: that you think this division of files makes sense? >> >> The files are not 'moved out', Michael and Igor are still the >> maintainers of the supported ACPI/SMBIOS subsystem: > > Does get_maintainer.pl print the answers for all matching > sections, rather than just the most specific, then? Yes: $ ./scripts/get_maintainer.pl -f hw/acpi/ghes.c Dongjiu Geng <gengdongjiu@huawei.com> (reviewer:ACPI/HEST/GHES) Xiang Zheng <zhengxiang9@huawei.com> (reviewer:ACPI/HEST/GHES) "Michael S. Tsirkin" <mst@redhat.com> (supporter:ACPI/SMBIOS) Igor Mammedov <imammedo@redhat.com> (supporter:ACPI/SMBIOS) qemu-arm@nongnu.org (open list:ACPI/HEST/GHES) qemu-devel@nongnu.org (open list:All patches CC here) > In any case, I'd still like an acked-by from them. Sure :)
On Fri, Jan 17, 2020 at 11:09:07AM +0000, Peter Maydell wrote:
> On Fri, 17 Jan 2020 at 07:22, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> >
> > Hi Peter,
> >
> > On 1/16/20 5:46 PM, Peter Maydell wrote:
> > > On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
> > >>
> > >> I and Xiang are willing to review the APEI-related patches and
> > >> volunteer as the reviewers for the HEST/GHES part.
> > >>
> > >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
> > >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com>
> > >> ---
> > >> MAINTAINERS | 9 +++++++++
> > >> 1 file changed, 9 insertions(+)
> > >>
> > >> diff --git a/MAINTAINERS b/MAINTAINERS
> > >> index 387879a..5af70a5 100644
> > >> --- a/MAINTAINERS
> > >> +++ b/MAINTAINERS
> > >> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c
> > >> F: tests/acpi-utils.[hc]
> > >> F: tests/data/acpi/
> > >>
> > >> +ACPI/HEST/GHES
> > >> +R: Dongjiu Geng <gengdongjiu@huawei.com>
> > >> +R: Xiang Zheng <zhengxiang9@huawei.com>
> > >> +L: qemu-arm@nongnu.org
> > >> +S: Maintained
> > >> +F: hw/acpi/ghes.c
> > >> +F: include/hw/acpi/ghes.h
> > >> +F: docs/specs/acpi_hest_ghes.rst
> > >> +
> > >> ppc4xx
> > >> M: David Gibson <david@gibson.dropbear.id.au>
> > >> L: qemu-ppc@nongnu.org
> > >> --
> > >
> > > Michael, Igor: since this new MAINTAINERS section is
> > > moving files out of the 'ACPI/SMBIOS' section that you're
> > > currently responsible for, do you want to provide an
> > > acked-by: that you think this division of files makes sense?
> >
> > The files are not 'moved out', Michael and Igor are still the
> > maintainers of the supported ACPI/SMBIOS subsystem:
>
> Does get_maintainer.pl print the answers for all matching
> sections, rather than just the most specific, then?
>
> In any case, I'd still like an acked-by from them.
>
> thanks
> -- PMM
Acked-by: Michael S. Tsirkin <mst@redhat.com>
On 2020/1/17 19:22, Philippe Mathieu-Daudé wrote:
> On 1/17/20 12:09 PM, Peter Maydell wrote:
>> On Fri, 17 Jan 2020 at 07:22, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
>>>
>>> Hi Peter,
>>>
>>> On 1/16/20 5:46 PM, Peter Maydell wrote:
>>>> On Wed, 8 Jan 2020 at 11:32, Dongjiu Geng <gengdongjiu@huawei.com> wrote:
>>>>>
>>>>> I and Xiang are willing to review the APEI-related patches and
>>>>> volunteer as the reviewers for the HEST/GHES part.
>>>>>
>>>>> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com>
>>>>> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com>
>>>>> ---
>>>>> MAINTAINERS | 9 +++++++++
>>>>> 1 file changed, 9 insertions(+)
>>>>>
>>>>> diff --git a/MAINTAINERS b/MAINTAINERS
>>>>> index 387879a..5af70a5 100644
>>>>> --- a/MAINTAINERS
>>>>> +++ b/MAINTAINERS
>>>>> @@ -1423,6 +1423,15 @@ F: tests/bios-tables-test.c
>>>>> F: tests/acpi-utils.[hc]
>>>>> F: tests/data/acpi/
>>>>>
>>>>> +ACPI/HEST/GHES
>>>>> +R: Dongjiu Geng <gengdongjiu@huawei.com>
>>>>> +R: Xiang Zheng <zhengxiang9@huawei.com>
>>>>> +L: qemu-arm@nongnu.org
>>>>> +S: Maintained
>>>>> +F: hw/acpi/ghes.c
>>>>> +F: include/hw/acpi/ghes.h
>>>>> +F: docs/specs/acpi_hest_ghes.rst
>>>>> +
>>>>> ppc4xx
>>>>> M: David Gibson <david@gibson.dropbear.id.au>
>>>>> L: qemu-ppc@nongnu.org
>>>>> --
>>>>
>>>> Michael, Igor: since this new MAINTAINERS section is
>>>> moving files out of the 'ACPI/SMBIOS' section that you're
>>>> currently responsible for, do you want to provide an
>>>> acked-by: that you think this division of files makes sense?
>>>
>>> The files are not 'moved out', Michael and Igor are still the
>>> maintainers of the supported ACPI/SMBIOS subsystem:
In fact, I am willing to maintain with others to the new added files , I see some new added files in "hw/acpi" folders are moved out, such as NVDIMM/vmgenid.
NVDIMM
M: Xiao Guangrong <xiaoguangrong.eric@gmail.com>
S: Maintained
F: hw/acpi/nvdimm.c
F: hw/mem/nvdimm.c
F: include/hw/mem/nvdimm.h
F: docs/nvdimm.txt
VM Generation ID
M: Ben Warren <ben@skyportsystems.com>
S: Maintained
F: hw/acpi/vmgenid.c
F: include/hw/acpi/vmgenid.h
F: docs/specs/vmgenid.txt
F: tests/vmgenid-test.c
F: stubs/vmgenid.c
On Fri, 17 Jan 2020 at 10:05, gengdongjiu <gengdongjiu@huawei.com> wrote: > > On 2020/1/17 0:28, Peter Maydell wrote: > > On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng <gengdongjiu@huawei.com> wrote: > > > >> +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) > >> +{ > >> + ram_addr_t ram_addr; > >> + hwaddr paddr; > >> + > >> + assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO); > >> + > >> + if (acpi_enabled && addr && > >> + object_property_get_bool(qdev_get_machine(), "ras", NULL)) { > >> + ram_addr = qemu_ram_addr_from_host(addr); > >> + if (ram_addr != RAM_ADDR_INVALID && > >> + kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) { > >> + kvm_hwpoison_page_add(ram_addr); > >> + /* > >> + * Asynchronous signal will be masked by main thread, so > >> + * only handle synchronous signal. > >> + */ > > > > I don't understand this comment. (I think we've had discussions > > about it before, but it's still not clear to me.) > > > > This function (kvm_arch_on_sigbus_vcpu()) will be called in two contexts: > > > > (1) in the vcpu thread: > > * the real SIGBUS handler sigbus_handler() sets a flag and arranges > > for an immediate vcpu exit > > * the vcpu thread reads the flag on exit from KVM_RUN and > > calls kvm_arch_on_sigbus_vcpu() directly > > * the error could be MCEERR_AR or MCEERR_AOFor the vcpu thread, the error can be MCEERR_AR or MCEERR_AO, > but kernel/KVM usually uses MCEERR_AR(action required) instead of MCEERR_AO, because it needs do action immediately. For MCEERR_AO error, the action is optional and the error can be ignored. > At least I do not find Linux kernel/KVM deliver MCEERR_AO in the vcpu threads. > > > (2) MCE errors on other threads: > > * here SIGBUS is blocked, so MCEERR_AR (action-required) > > errors will cause the kernel to just kill the QEMU process > > * MCEERR_AO errors will be handled via the iothread's use > > of signalfd(), so kvm_on_sigbus() will get called from > > the main thread, and it will call kvm_arch_on_sigbus_vcpu() > > * in this case the passed in CPUState will (arbitrarily) be that > > for the first vCPU > > For the MCE errors on other threads, it can only handle MCEERR_AO. If it is MCEERR_AR, the QEMU will assert and exit[2]. > > Case1: Other APP indeed can send MCEERR_AO to QEMU, QEMU handle it via the iothread's use of signalfd() through above path. > Case2: But if the MCEERR_AO is delivered by kernel, I see QEMU ignore it because SIGBUS is masked in main thread[3], for this case, I do not see QEMU handle it via signalfd() for MCEERR_AO errors from my test. SIGBUS is blocked in the main thread because we use signalfd(). The function sigfd_handler() should be called and it will then manually invoke the correct function for the signal. > For Case1,I think we should not let guest know it, because it is not triggered by guest. only other APP send SIGBUS to tell QEMU do somethings. I don't understand what you mean here by "other app" or "guest" triggering of MCEERR. I thought that an MCEERR meant "the hardware has detected that there is a problem with the RAM". If there's a problem with the RAM and it's the RAM that's being used as guest RAM, we need to tell the guest, surely ? > For Case2,it does not call call kvm_arch_on_sigbus_vcpu(). It should do. The code you quote calls that function for that case: > [1]: > /* Called synchronously (via signalfd) in main thread. */ > int kvm_on_sigbus(int code, void *addr) > { > #ifdef KVM_HAVE_MCE_INJECTION > /* Action required MCE kills the process if SIGBUS is blocked. Because > * that's what happens in the I/O thread, where we handle MCE via signalfd, > * we can only get action optional here. > */ > [2]: assert(code != BUS_MCEERR_AR); > kvm_arch_on_sigbus_vcpu(first_cpu, code, addr); > return 0; > #else > return 1; > #endif > } > Above all, from my test, for MCEERR_AO error which is triggered by guest, it not call kvm_arch_on_sigbus_vcpu(). I'm not sure what you mean by "triggered by guest". I assume that exactly what kind of errors the kernel can report and when will depend to some extent on the underlying hardware/firmware implementation of reporting of memory errors, but in principle the ABI allows the kernel to send SIGBUS_(BUS_MCEERR_AO) to the main thread, the signal should be handled by signalfd, our code for working with multiple fds should mean that the main thread calls sigfd_handler() to deal with reading bytes from the signalfd fd, and that function should then call sigbus_handler(), which calls kvm_on_sigbus(), which calls kvm_arch_on_sigbus_vcpu(). If something in that code path is not working then we need to find out what it is. thanks -- PMM
On 2020/1/20 20:15, Peter Maydell wrote: > On Fri, 17 Jan 2020 at 10:05, gengdongjiu <gengdongjiu@huawei.com> wrote: >> >> On 2020/1/17 0:28, Peter Maydell wrote: >>> On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng <gengdongjiu@huawei.com> wrote: >>> >>>> +void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr) >>>> +{ >>>> + ram_addr_t ram_addr; >>>> + hwaddr paddr; >>>> + >>>> + assert(code == BUS_MCEERR_AR || code == BUS_MCEERR_AO); >>>> + >>>> + if (acpi_enabled && addr && >>>> + object_property_get_bool(qdev_get_machine(), "ras", NULL)) { >>>> + ram_addr = qemu_ram_addr_from_host(addr); >>>> + if (ram_addr != RAM_ADDR_INVALID && >>>> + kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) { >>>> + kvm_hwpoison_page_add(ram_addr); >>>> + /* >>>> + * Asynchronous signal will be masked by main thread, so >>>> + * only handle synchronous signal. >>>> + */ >>> >>> I don't understand this comment. (I think we've had discussions >>> about it before, but it's still not clear to me.) >>> >>> This function (kvm_arch_on_sigbus_vcpu()) will be called in two contexts: >>> >>> (1) in the vcpu thread: >>> * the real SIGBUS handler sigbus_handler() sets a flag and arranges >>> for an immediate vcpu exit >>> * the vcpu thread reads the flag on exit from KVM_RUN and >>> calls kvm_arch_on_sigbus_vcpu() directly >>> * the error could be MCEERR_AR or MCEERR_AOFor the vcpu thread, the error can be MCEERR_AR or MCEERR_AO, >> but kernel/KVM usually uses MCEERR_AR(action required) instead of MCEERR_AO, because it needs do action immediately. For MCEERR_AO error, the action is optional and the error can be ignored. >> At least I do not find Linux kernel/KVM deliver MCEERR_AO in the vcpu threads. >> >>> (2) MCE errors on other threads: >>> * here SIGBUS is blocked, so MCEERR_AR (action-required) >>> errors will cause the kernel to just kill the QEMU process >>> * MCEERR_AO errors will be handled via the iothread's use >>> of signalfd(), so kvm_on_sigbus() will get called from >>> the main thread, and it will call kvm_arch_on_sigbus_vcpu() >>> * in this case the passed in CPUState will (arbitrarily) be that >>> for the first vCPU >> >> For the MCE errors on other threads, it can only handle MCEERR_AO. If it is MCEERR_AR, the QEMU will assert and exit[2]. >> >> Case1: Other APP indeed can send MCEERR_AO to QEMU, QEMU handle it via the iothread's use of signalfd() through above path. >> Case2: But if the MCEERR_AO is delivered by kernel, I see QEMU ignore it because SIGBUS is masked in main thread[3], for this case, I do not see QEMU handle it via signalfd() for MCEERR_AO errors from my test. > > SIGBUS is blocked in the main thread because we use signalfd(). > The function sigfd_handler() should be called and it will then > manually invoke the correct function for the signal. > >> For Case1,I think we should not let guest know it, because it is not triggered by guest. only other APP send SIGBUS to tell QEMU do somethings. > > I don't understand what you mean here by "other app" or > "guest" triggering of MCEERR. I thought that an MCEERR meant > "the hardware has detected that there is a problem with the > RAM". If there's a problem with the RAM and it's the RAM that's > being used as guest RAM, we need to tell the guest, surely ? sure, If the error is guest RAM, we need to test the guest. I mean if the RAM that is being used as QEMU RAM(not guest RAM), we should not tell the guest. OR if another user space manually send SIGBUS to qemu, such as using "kill -s SIGBUS xxx" commands, we should not tell the guest. > >> For Case2,it does not call call kvm_arch_on_sigbus_vcpu(). > > It should do. The code you quote calls that function > for that case: According to our analysis, I also think it should call the function for that case. But from my test, I see kvm_arch_on_sigbus_vcpu() is not called when KVM/Kernel delivers SIGBUS to QEMU main thread. So I am also confused. I haven't even dig into the reason yet. If anyone has done the test or knows the reason, welcome comments. > >> [1]: >> /* Called synchronously (via signalfd) in main thread. */ >> int kvm_on_sigbus(int code, void *addr) >> { >> #ifdef KVM_HAVE_MCE_INJECTION >> /* Action required MCE kills the process if SIGBUS is blocked. Because >> * that's what happens in the I/O thread, where we handle MCE via signalfd, >> * we can only get action optional here. >> */ >> [2]: assert(code != BUS_MCEERR_AR); >> kvm_arch_on_sigbus_vcpu(first_cpu, code, addr); >> return 0; >> #else >> return 1; >> #endif >> } > > >> Above all, from my test, for MCEERR_AO error which is triggered by guest, it not call > kvm_arch_on_sigbus_vcpu(). > > I'm not sure what you mean by "triggered by guest". I assume that > exactly what kind of errors the kernel can report and when will > depend to some extent on the underlying hardware/firmware > implementation of reporting of memory errors, but in principle > the ABI allows the kernel to send SIGBUS_(BUS_MCEERR_AO) to the > main thread, the signal should be handled by signalfd, our code > for working with multiple fds should mean that the main thread > calls sigfd_handler() to deal with reading bytes from the signalfd > fd, and that function should then call sigbus_handler(), which > calls kvm_on_sigbus(), which calls kvm_arch_on_sigbus_vcpu(). > If something in that code path is not working then we need to > find out what it is. I agree with you, we need to check why it does not call sigbus_handler() for the SIGBUS delivered by kernel/KVM. But I think it can put it in another series, this series we only handle the SIGBUS_(BUS_MCEERR_AR), whether do you think it is OK? Of course I will update the comments that you ever mentioned. By the way, If using "kill -s SIGBUS xxx" command to send SIGBUS to QEMU main thread, it indeed will be handled by signalfd. > > thanks > -- PMM > . >
On Wed, 8 Jan 2020 19:32:17 +0800 Dongjiu Geng <gengdongjiu@huawei.com> wrote: > This patch builds error_block_address and read_ack_register fields > in hardware errors table , the error_block_address points to Generic > Error Status Block(GESB) via bios_linker. The max size for one GESB > is 0x1000 in bytes, For more detailed information, please refer to s/0x1000/4Kb/ > document: docs/specs/acpi_hest_ghes.rst > > Now we only support one Error source, if necessary, we can extend to > support more. > > Suggested-by: Laszlo Ersek <lersek@redhat.com> > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com> > --- > default-configs/arm-softmmu.mak | 1 + > hw/acpi/Kconfig | 4 ++ > hw/acpi/Makefile.objs | 1 + > hw/acpi/aml-build.c | 2 + > hw/acpi/ghes.c | 94 +++++++++++++++++++++++++++++++++++++++++ > hw/arm/virt-acpi-build.c | 6 +++ > include/hw/acpi/aml-build.h | 1 + > include/hw/acpi/ghes.h | 26 ++++++++++++ > 8 files changed, 135 insertions(+) > create mode 100644 hw/acpi/ghes.c > create mode 100644 include/hw/acpi/ghes.h > > diff --git a/default-configs/arm-softmmu.mak b/default-configs/arm-softmmu.mak > index 1f2e0e7..5722f31 100644 > --- a/default-configs/arm-softmmu.mak > +++ b/default-configs/arm-softmmu.mak > @@ -40,3 +40,4 @@ CONFIG_FSL_IMX25=y > CONFIG_FSL_IMX7=y > CONFIG_FSL_IMX6UL=y > CONFIG_SEMIHOSTING=y > +CONFIG_ACPI_APEI=y > diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig > index 12e3f1e..ed8c34d 100644 > --- a/hw/acpi/Kconfig > +++ b/hw/acpi/Kconfig > @@ -23,6 +23,10 @@ config ACPI_NVDIMM > bool > depends on ACPI > > +config ACPI_APEI > + bool > + depends on ACPI > + > config ACPI_PCI > bool > depends on ACPI && PCI > diff --git a/hw/acpi/Makefile.objs b/hw/acpi/Makefile.objs > index 9925305..7b5db4b 100644 > --- a/hw/acpi/Makefile.objs > +++ b/hw/acpi/Makefile.objs > @@ -5,6 +5,7 @@ common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu_hotplug.o > common-obj-$(CONFIG_ACPI_MEMORY_HOTPLUG) += memory_hotplug.o > common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu.o > common-obj-$(CONFIG_ACPI_NVDIMM) += nvdimm.o > +common-obj-$(CONFIG_ACPI_APEI) += ghes.o > common-obj-$(CONFIG_ACPI_VMGENID) += vmgenid.o > common-obj-$(CONFIG_ACPI_HW_REDUCED) += generic_event_device.o > common-obj-$(call lnot,$(CONFIG_ACPI_X86)) += acpi-stub.o > diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c > index 2c3702b..3681ec6 100644 > --- a/hw/acpi/aml-build.c > +++ b/hw/acpi/aml-build.c > @@ -1578,6 +1578,7 @@ void acpi_build_tables_init(AcpiBuildTables *tables) > tables->table_data = g_array_new(false, true /* clear */, 1); > tables->tcpalog = g_array_new(false, true /* clear */, 1); > tables->vmgenid = g_array_new(false, true /* clear */, 1); > + tables->hardware_errors = g_array_new(false, true /* clear */, 1); > tables->linker = bios_linker_loader_init(); > } > > @@ -1588,6 +1589,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre) > g_array_free(tables->table_data, true); > g_array_free(tables->tcpalog, mfre); > g_array_free(tables->vmgenid, mfre); > + g_array_free(tables->hardware_errors, mfre); > } > > /* > diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c > new file mode 100644 > index 0000000..b7fdbbb > --- /dev/null > +++ b/hw/acpi/ghes.c > @@ -0,0 +1,94 @@ > +/* > + * Support for generating APEI tables and recording CPER for Guests > + * > + * Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. s/2019/2020/ the same for other places that add this string within this series > + * > + * Author: Dongjiu Geng <gengdongjiu@huawei.com> > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation; either version 2 of the License, or > + * (at your option) any later version. > + > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + > + * You should have received a copy of the GNU General Public License along > + * with this program; if not, see <http://www.gnu.org/licenses/>. > + */ > + > +#include "qemu/osdep.h" > +#include "hw/acpi/acpi.h" > +#include "hw/acpi/ghes.h" > +#include "hw/acpi/aml-build.h" > +#include "hw/nvram/fw_cfg.h" > +#include "sysemu/sysemu.h" > +#include "qemu/error-report.h" > + > +#include "hw/acpi/bios-linker-loader.h" > + > +#define ACPI_GHES_ERRORS_FW_CFG_FILE "etc/hardware_errors" > +#define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr" > + > +/* The max size in bytes for one error block */ > +#define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 using 1 * KiB would make it more readable > + > +/* Now only support ARMv8 SEA notification type error source */ > +#define ACPI_GHES_ERROR_SOURCE_COUNT 1 > + > +/* > + * Build table for the hardware error fw_cfg blob. > + * Initialize "etc/hardware_errors" and "etc/hardware_errors_addr" fw_cfg blobs. > + * See docs/specs/acpi_hest_ghes.rst for blobs format. > + */ > +void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker) > +{ > + int i, error_status_block_offset; > + > + /* Build error_block_address */ > + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { > + build_append_int_noprefix(hardware_errors, 0, sizeof(uint64_t)); > + } > + > + /* Build read_ack_register */ > + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { > + /* > + * Initialize the value of read_ack_register to 1, so GHES can be > + * writeable after (re)boot. > + * ACPI 6.2: 18.3.2.8 Generic Hardware Error Source version 2 > + * (GHESv2 - Type 10) > + */ > + build_append_int_noprefix(hardware_errors, 1, sizeof(uint64_t)); > + } > + > + /* Generic Error Status Block offset in the hardware error fw_cfg blob */ > + error_status_block_offset = hardware_errors->len; > + > + /* Reserve space for Error Status Data Block */ > + acpi_data_push(hardware_errors, > + ACPI_GHES_MAX_RAW_DATA_LENGTH * ACPI_GHES_ERROR_SOURCE_COUNT); > + > + /* Tell guest firmware to place hardware_errors blob into RAM */ > + bios_linker_loader_alloc(linker, ACPI_GHES_ERRORS_FW_CFG_FILE, > + hardware_errors, 1, false); ^^ /* * bios_linker_loader_alloc: ask guest to load file into guest memory. [...] * @alloc_align: required minimal alignment in bytes. Must be a power of 2. Maybe use 8 here to make 64bit addresses naturally aligned? > + > + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { > + /* > + * Tell firmware to patch error_block_address entries to point to > + * corresponding "Generic Error Status Block" > + */ > + bios_linker_loader_add_pointer(linker, > + ACPI_GHES_ERRORS_FW_CFG_FILE, sizeof(uint64_t) * i, > + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, > + error_status_block_offset + i * ACPI_GHES_MAX_RAW_DATA_LENGTH); > + } > + > + /* > + * tell firmware to write hardware_errors GPA into > + * hardware_errors_addr fw_cfg, once the former has been initialized. > + */ > + bios_linker_loader_write_pointer(linker, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, > + 0, sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); > +} > diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c > index bd5f771..6819fcf 100644 > --- a/hw/arm/virt-acpi-build.c > +++ b/hw/arm/virt-acpi-build.c > @@ -48,6 +48,7 @@ > #include "sysemu/reset.h" > #include "kvm_arm.h" > #include "migration/vmstate.h" > +#include "hw/acpi/ghes.h" > > #define ARM_SPI_BASE 32 > > @@ -830,6 +831,11 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) > acpi_add_table(table_offsets, tables_blob); > build_spcr(tables_blob, tables->linker, vms); > > + if (vms->ras) { > + acpi_add_table(table_offsets, tables_blob); > + build_ghes_error_table(tables->hardware_errors, tables->linker); > + } > + > if (ms->numa_state->num_nodes > 0) { > acpi_add_table(table_offsets, tables_blob); > build_srat(tables_blob, tables->linker, vms); > diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h > index de4a406..8f13620 100644 > --- a/include/hw/acpi/aml-build.h > +++ b/include/hw/acpi/aml-build.h > @@ -220,6 +220,7 @@ struct AcpiBuildTables { > GArray *rsdp; > GArray *tcpalog; > GArray *vmgenid; > + GArray *hardware_errors; > BIOSLinker *linker; > } AcpiBuildTables; > > diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h > new file mode 100644 > index 0000000..3dbda3f > --- /dev/null > +++ b/include/hw/acpi/ghes.h > @@ -0,0 +1,26 @@ > +/* > + * Support for generating APEI tables and recording CPER for Guests > + * > + * Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. > + * > + * Author: Dongjiu Geng <gengdongjiu@huawei.com> > + * > + * This program is free software; you can redistribute it and/or modify > + * it under the terms of the GNU General Public License as published by > + * the Free Software Foundation; either version 2 of the License, or > + * (at your option) any later version. > + > + * This program is distributed in the hope that it will be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > + * GNU General Public License for more details. > + > + * You should have received a copy of the GNU General Public License along > + * with this program; if not, see <http://www.gnu.org/licenses/>. > + */ > + > +#ifndef ACPI_GHES_H > +#define ACPI_GHES_H > + > +void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); > +#endif
On Wed, 8 Jan 2020 19:32:18 +0800 Dongjiu Geng <gengdongjiu@huawei.com> wrote: > This patch builds Hardware Error Source Table(HEST) via fw_cfg blobs. > Now it only supports ARMv8 SEA, a type of Generic Hardware Error > Source version 2(GHESv2) error source. Afterwards, we can extend > the supported types if needed. For the CPER section, currently it > is memory section because kernel mainly wants userspace to handle > the memory errors. > > This patch follows the spec ACPI 6.2 to build the Hardware Error > Source table. For more detailed information, please refer to > document: docs/specs/acpi_hest_ghes.rst > > build_append_ghes_notify() will help to add Hardware Error Notification > to ACPI tables without using packed C structures and avoid endianness > issues as API doesn't need explicit conversion. > > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com> > Acked-by: Xiang Zheng <zhengxiang9@huawei.com> Overall it looks fine to me, see couple nits below > --- > hw/acpi/ghes.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++- > hw/arm/virt-acpi-build.c | 2 + > include/hw/acpi/ghes.h | 40 ++++++++++++++++ > 3 files changed, 159 insertions(+), 1 deletion(-) > > diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c > index b7fdbbb..9d37798 100644 > --- a/hw/acpi/ghes.c > +++ b/hw/acpi/ghes.c > @@ -34,9 +34,42 @@ > > /* The max size in bytes for one error block */ > #define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 > - > /* Now only support ARMv8 SEA notification type error source */ > #define ACPI_GHES_ERROR_SOURCE_COUNT 1 > +/* Generic Hardware Error Source version 2 */ > +#define ACPI_GHES_SOURCE_GENERIC_ERROR_V2 10 > +/* Address offset in Generic Address Structure(GAS) */ > +#define GAS_ADDR_OFFSET 4 > + > +/* > + * Hardware Error Notification > + * ACPI 4.0: 17.3.2.7 Hardware Error Notification > + * Composes dummy Hardware Error Notification descriptor of specified type > + */ > +static void build_ghes_hw_error_notification(GArray *table, const uint8_t type) > +{ > + /* Type */ > + build_append_int_noprefix(table, type, 1); > + /* > + * Length: > + * Total length of the structure in bytes > + */ > + build_append_int_noprefix(table, 28, 1); > + /* Configuration Write Enable */ > + build_append_int_noprefix(table, 0, 2); > + /* Poll Interval */ > + build_append_int_noprefix(table, 0, 4); > + /* Vector */ > + build_append_int_noprefix(table, 0, 4); > + /* Switch To Polling Threshold Value */ > + build_append_int_noprefix(table, 0, 4); > + /* Switch To Polling Threshold Window */ > + build_append_int_noprefix(table, 0, 4); > + /* Error Threshold Value */ > + build_append_int_noprefix(table, 0, 4); > + /* Error Threshold Window */ > + build_append_int_noprefix(table, 0, 4); > +} > > /* > * Build table for the hardware error fw_cfg blob. > @@ -92,3 +125,86 @@ void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker) > bios_linker_loader_write_pointer(linker, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, > 0, sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); > } > + > +/* Build Generic Hardware Error Source version 2 (GHESv2) */ > +static void build_ghes_v2(GArray *table_data, int source_id, BIOSLinker *linker) > +{ > + uint64_t address_offset; > + /* > + * Type: > + * Generic Hardware Error Source version 2(GHESv2 - Type 10) > + */ > + build_append_int_noprefix(table_data, ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2); > + /* Source Id */ > + build_append_int_noprefix(table_data, source_id, 2); > + /* Related Source Id */ > + build_append_int_noprefix(table_data, 0xffff, 2); > + /* Flags */ > + build_append_int_noprefix(table_data, 0, 1); > + /* Enabled */ > + build_append_int_noprefix(table_data, 1, 1); > + > + /* Number of Records To Pre-allocate */ > + build_append_int_noprefix(table_data, 1, 4); > + /* Max Sections Per Record */ > + build_append_int_noprefix(table_data, 1, 4); > + /* Max Raw Data Length */ > + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); > + > + address_offset = table_data->len; > + /* Error Status Address */ > + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, > + 4 /* QWord access */, 0); > + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, > + address_offset + GAS_ADDR_OFFSET, > + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); > + > + /* > + * Notification Structure > + * Now only enable ARMv8 SEA notification type > + */ > + build_ghes_hw_error_notification(table_data, ACPI_GHES_NOTIFY_SEA); > + > + /* Error Status Block Length */ > + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); > + > + /* > + * Read Ack Register > + * ACPI 6.1: 18.3.2.8 Generic Hardware Error Source > + * version 2 (GHESv2 - Type 10) > + */ > + address_offset = table_data->len; > + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, > + 4 /* QWord access */, 0); > + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, > + address_offset + GAS_ADDR_OFFSET, > + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, > + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t)); > + > + /* > + * Read Ack Preserve > + * We only provide the first bit in Read Ack Register to OSPM to write > + * while the other bits are preserved. > + */ > + build_append_int_noprefix(table_data, ~0x1ULL, 8); > + /* Read Ack Write */ > + build_append_int_noprefix(table_data, 0x1, 8); > +} > + > +/* Build Hardware Error Source Table */ > +void acpi_build_hest(GArray *table_data, GArray *hardware_errors, ^^^^^^^^ it seems to be unused, so why it's here? > + BIOSLinker *linker) > +{ > + uint64_t hest_start = table_data->len; > + > + /* Hardware Error Source Table header*/ > + acpi_data_push(table_data, sizeof(AcpiTableHeader)); > + > + /* Error Source Count */ > + build_append_int_noprefix(table_data, ACPI_GHES_ERROR_SOURCE_COUNT, 4); > + > + build_ghes_v2(table_data, ACPI_HEST_SRC_ID_SEA, linker); > + > + build_header(linker, table_data, (void *)(table_data->data + hest_start), > + "HEST", table_data->len - hest_start, 1, NULL, ""); > +} > diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c > index 6819fcf..837bbf9 100644 > --- a/hw/arm/virt-acpi-build.c > +++ b/hw/arm/virt-acpi-build.c > @@ -834,6 +834,8 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) > if (vms->ras) { > acpi_add_table(table_offsets, tables_blob); > build_ghes_error_table(tables->hardware_errors, tables->linker); > + acpi_build_hest(tables_blob, tables->hardware_errors, > + tables->linker); not aligned properly you can use ./scripts/checkpatch.pl to see if there is style errors > } > > if (ms->numa_state->num_nodes > 0) { > diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h > index 3dbda3f..09a7f86 100644 > --- a/include/hw/acpi/ghes.h > +++ b/include/hw/acpi/ghes.h > @@ -22,5 +22,45 @@ > #ifndef ACPI_GHES_H > #define ACPI_GHES_H > > +/* > + * Values for Hardware Error Notification Type field > + */ > +enum AcpiGhesNotifyType { > + /* Polled */ > + ACPI_GHES_NOTIFY_POLLED = 0, > + /* External Interrupt */ > + ACPI_GHES_NOTIFY_EXTERNAL = 1, > + /* Local Interrupt */ > + ACPI_GHES_NOTIFY_LOCAL = 2, > + /* SCI */ > + ACPI_GHES_NOTIFY_SCI = 3, > + /* NMI */ > + ACPI_GHES_NOTIFY_NMI = 4, > + /* CMCI, ACPI 5.0: 18.3.2.7, Table 18-290 */ > + ACPI_GHES_NOTIFY_CMCI = 5, > + /* MCE, ACPI 5.0: 18.3.2.7, Table 18-290 */ > + ACPI_GHES_NOTIFY_MCE = 6, > + /* GPIO-Signal, ACPI 6.0: 18.3.2.7, Table 18-332 */ > + ACPI_GHES_NOTIFY_GPIO = 7, > + /* ARMv8 SEA, ACPI 6.1: 18.3.2.9, Table 18-345 */ > + ACPI_GHES_NOTIFY_SEA = 8, > + /* ARMv8 SEI, ACPI 6.1: 18.3.2.9, Table 18-345 */ > + ACPI_GHES_NOTIFY_SEI = 9, > + /* External Interrupt - GSIV, ACPI 6.1: 18.3.2.9, Table 18-345 */ > + ACPI_GHES_NOTIFY_GSIV = 10, > + /* Software Delegated Exception, ACPI 6.2: 18.3.2.9, Table 18-383 */ > + ACPI_GHES_NOTIFY_SDEI = 11, > + /* 12 and greater are reserved */ > + ACPI_GHES_NOTIFY_RESERVED = 12 > +}; > + > +enum { > + ACPI_HEST_SRC_ID_SEA = 0, > + /* future ids go here */ > + ACPI_HEST_SRC_ID_RESERVED, > +}; > + > void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); > +void acpi_build_hest(GArray *table_data, GArray *hardware_error, > + BIOSLinker *linker); > #endif
On Wed, 8 Jan 2020 19:32:19 +0800 Dongjiu Geng <gengdongjiu@huawei.com> wrote: in addition to comments of others: > Record the GHEB address via fw_cfg file, when recording > a error to CPER, it will use this address to find out > Generic Error Data Entries and write the error. > > Make the HEST GHES to a GED device. It's hard to parse this even kno Pls rephrase/make commit message more verbose, so it would describe why and what patch is supposed to do > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > --- > hw/acpi/generic_event_device.c | 15 ++++++++++++++- > hw/acpi/ghes.c | 16 ++++++++++++++++ > hw/arm/virt-acpi-build.c | 13 ++++++++++++- > include/hw/acpi/generic_event_device.h | 2 ++ > include/hw/acpi/ghes.h | 6 ++++++ > 5 files changed, 50 insertions(+), 2 deletions(-) > > diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c > index 9cee90c..9bf37e4 100644 > --- a/hw/acpi/generic_event_device.c > +++ b/hw/acpi/generic_event_device.c > @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = { > } > }; > > +static const VMStateDescription vmstate_ghes_state = { > + .name = "acpi-ghes-state", > + .version_id = 1, > + .minimum_version_id = 1, > + .fields = (VMStateField[]) { > + VMSTATE_UINT64(ghes_addr_le, AcpiGhesState), > + VMSTATE_END_OF_LIST() > + } > +}; > + > static const VMStateDescription vmstate_acpi_ged = { > .name = "acpi-ged", > .version_id = 1, > .minimum_version_id = 1, > .fields = (VMStateField[]) { > - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState), > + VMSTATE_STRUCT(ged_state, AcpiGedState, 1, > + vmstate_ged_state, GEDState), > + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, > + vmstate_ghes_state, AcpiGhesState), > VMSTATE_END_OF_LIST(), > }, > .subsections = (const VMStateDescription * []) { > diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c > index 9d37798..68f4abf 100644 > --- a/hw/acpi/ghes.c > +++ b/hw/acpi/ghes.c > @@ -23,6 +23,7 @@ > #include "hw/acpi/acpi.h" > #include "hw/acpi/ghes.h" > #include "hw/acpi/aml-build.h" > +#include "hw/acpi/generic_event_device.h" > #include "hw/nvram/fw_cfg.h" > #include "sysemu/sysemu.h" > #include "qemu/error-report.h" > @@ -208,3 +209,18 @@ void acpi_build_hest(GArray *table_data, GArray *hardware_errors, > build_header(linker, table_data, (void *)(table_data->data + hest_start), > "HEST", table_data->len - hest_start, 1, NULL, ""); > } > + > +void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, > + GArray *hardware_error) not aligned properly > +{ > + size_t size = 2 * sizeof(uint64_t) + ACPI_GHES_MAX_RAW_DATA_LENGTH; > + size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size; > + > + /* Create a read-only fw_cfg file for GHES */ > + fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data, > + request_block_size); > + > + /* Create a read-write fw_cfg file for Address */ > + fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, > + NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); > +} > diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c > index 837bbf9..c8aa94d 100644 > --- a/hw/arm/virt-acpi-build.c > +++ b/hw/arm/virt-acpi-build.c > @@ -797,6 +797,7 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) > unsigned dsdt, xsdt; > GArray *tables_blob = tables->table_data; > MachineState *ms = MACHINE(vms); > + AcpiGedState *acpi_ged_state; > > table_offsets = g_array_new(false, true /* clear */, > sizeof(uint32_t)); > @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) > acpi_add_table(table_offsets, tables_blob); > build_spcr(tables_blob, tables->linker, vms); > > - if (vms->ras) { > + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, > + NULL)); > + if (acpi_ged_state && vms->ras) { there is vms->acpi_dev which is GED, so you don't need to look it up suggest: if (ras) { assert(ged) do other fun stuff ... } > acpi_add_table(table_offsets, tables_blob); > build_ghes_error_table(tables->hardware_errors, tables->linker); > acpi_build_hest(tables_blob, tables->hardware_errors, > @@ -925,6 +928,7 @@ void virt_acpi_setup(VirtMachineState *vms) > { > AcpiBuildTables tables; > AcpiBuildState *build_state; > + AcpiGedState *acpi_ged_state; > > if (!vms->fw_cfg) { > trace_virt_acpi_setup(); > @@ -955,6 +959,13 @@ void virt_acpi_setup(VirtMachineState *vms) > fw_cfg_add_file(vms->fw_cfg, ACPI_BUILD_TPMLOG_FILE, tables.tcpalog->data, > acpi_data_len(tables.tcpalog)); > > + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, > + NULL)); > + if (acpi_ged_state && vms->ras) { ditto > + acpi_ghes_add_fw_cfg(&acpi_ged_state->ghes_state, > + vms->fw_cfg, tables.hardware_errors); > + } > + > build_state->rsdp_mr = acpi_add_rom_blob(virt_acpi_build_update, > build_state, tables.rsdp, > ACPI_BUILD_RSDP_FILE, 0); > diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h > index d157eac..037d2b5 100644 > --- a/include/hw/acpi/generic_event_device.h > +++ b/include/hw/acpi/generic_event_device.h > @@ -61,6 +61,7 @@ > > #include "hw/sysbus.h" > #include "hw/acpi/memory_hotplug.h" > +#include "hw/acpi/ghes.h" > > #define ACPI_POWER_BUTTON_DEVICE "PWRB" > > @@ -95,6 +96,7 @@ typedef struct AcpiGedState { > GEDState ged_state; > uint32_t ged_event_bitmap; > qemu_irq irq; > + AcpiGhesState ghes_state; > } AcpiGedState; > > void build_ged_aml(Aml *table, const char* name, HotplugHandler *hotplug_dev, > diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h > index 09a7f86..a6761e6 100644 > --- a/include/hw/acpi/ghes.h > +++ b/include/hw/acpi/ghes.h > @@ -60,7 +60,13 @@ enum { > ACPI_HEST_SRC_ID_RESERVED, > }; > > +typedef struct AcpiGhesState { > + uint64_t ghes_addr_le; > +} AcpiGhesState; > + > void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); > void acpi_build_hest(GArray *table_data, GArray *hardware_error, > BIOSLinker *linker); > +void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, > + GArray *hardware_errors); > #endif
On Wed, 8 Jan 2020 19:32:21 +0800 Dongjiu Geng <gengdongjiu@huawei.com> wrote: > kvm_arch_on_sigbus_vcpu() error injection uses source_id as > index in etc/hardware_errors to find out Error Status Data > Block entry corresponding to error source. So supported source_id > values should be assigned here and not be changed afterwards to > make sure that guest will write error into expected Error Status > Data Block even if guest was migrated to a newer QEMU. > > Before QEMU writes a new error to ACPI table, it will check whether > previous error has been acknowledged. Otherwise it will ignore the new > error. For the errors section type, QEMU simulate it to memory section > error. > > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com> > --- > hw/acpi/ghes.c | 224 ++++++++++++++++++++++++++++++++++++++++++++++++- > include/hw/acpi/ghes.h | 3 + > include/qemu/uuid.h | 5 ++ > 3 files changed, 230 insertions(+), 2 deletions(-) > > diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c > index 68f4abf..f2ecffe 100644 > --- a/hw/acpi/ghes.c > +++ b/hw/acpi/ghes.c > @@ -28,21 +28,56 @@ > #include "sysemu/sysemu.h" > #include "qemu/error-report.h" > > -#include "hw/acpi/bios-linker-loader.h" why it's moved to header? > - > #define ACPI_GHES_ERRORS_FW_CFG_FILE "etc/hardware_errors" > #define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr" > > /* The max size in bytes for one error block */ > #define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 > + > /* Now only support ARMv8 SEA notification type error source */ > #define ACPI_GHES_ERROR_SOURCE_COUNT 1 > + > /* Generic Hardware Error Source version 2 */ > #define ACPI_GHES_SOURCE_GENERIC_ERROR_V2 10 > + > /* Address offset in Generic Address Structure(GAS) */ > #define GAS_ADDR_OFFSET 4 > > /* > + * The total size of Generic Error Data Entry > + * ACPI 6.1/6.2: 18.3.2.7.1 Generic Error Data, > + * Table 18-343 Generic Error Data Entry > + */ > +#define ACPI_GHES_DATA_LENGTH 72 > + > +/* The memory section CPER size, UEFI 2.6: N.2.5 Memory Error Section */ > +#define ACPI_GHES_MEM_CPER_LENGTH 80 > + > +/* Masks for block_status flags */ > +#define ACPI_GEBS_UNCORRECTABLE 1 > + > +#define UEFI_CPER_SEC_PLATFORM_MEM \ > + UUID_LE(0xA5BC1114, 0x6F64, 0x4EDE, 0xB8, 0x63, 0x3E, 0x83, \ > + 0xED, 0x7C, 0x83, 0xB1) > + > +/* > + * Total size for Generic Error Status Block except Generic Error Data Entries > + * ACPI 6.2: 18.3.2.7.1 Generic Error Data, > + * Table 18-380 Generic Error Status Block > + */ > +#define ACPI_GHES_GESB_SIZE 20 > + > +/* > + * Values for error_severity field > + */ > +enum AcpiGenericErrorSeverity { > + ACPI_CPER_SEV_RECOVERABLE = 0, > + ACPI_CPER_SEV_FATAL = 1, > + ACPI_CPER_SEV_CORRECTED = 2, > + ACPI_CPER_SEV_NONE = 3, > +}; > + > +/* > * Hardware Error Notification > * ACPI 4.0: 17.3.2.7 Hardware Error Notification > * Composes dummy Hardware Error Notification descriptor of specified type > @@ -73,6 +108,127 @@ static void build_ghes_hw_error_notification(GArray *table, const uint8_t type) > } > > /* > + * Generic Error Data Entry > + * ACPI 6.1: 18.3.2.7.1 Generic Error Data > + */ > +static void acpi_ghes_generic_error_data(GArray *table, QemuUUID section_type, > + uint32_t error_severity, uint8_t validation_bits, uint8_t flags, > + uint32_t error_data_length, QemuUUID fru_id, > + uint64_t time_stamp) > +{ > + /* Section Type */ > + g_array_append_vals(table, section_type.data, > + ARRAY_SIZE(section_type.data)); > + > + /* Error Severity */ > + build_append_int_noprefix(table, error_severity, 4); > + /* Revision */ > + build_append_int_noprefix(table, 0x300, 2); > + /* Validation Bits */ > + build_append_int_noprefix(table, validation_bits, 1); > + /* Flags */ > + build_append_int_noprefix(table, flags, 1); > + /* Error Data Length */ > + build_append_int_noprefix(table, error_data_length, 4); > + > + /* FRU Id */ > + g_array_append_vals(table, fru_id.data, ARRAY_SIZE(fru_id.data)); > + > + /* FRU Text */ > + build_append_int_noprefix(table, 0, 20); > + /* Timestamp */ > + build_append_int_noprefix(table, time_stamp, 8); > +} > + > +/* > + * Generic Error Status Block > + * ACPI 6.1: 18.3.2.7.1 Generic Error Data > + */ > +static void acpi_ghes_generic_error_status(GArray *table, uint32_t block_status, > + uint32_t raw_data_offset, uint32_t raw_data_length, > + uint32_t data_length, uint32_t error_severity) > +{ > + /* Block Status */ > + build_append_int_noprefix(table, block_status, 4); > + /* Raw Data Offset */ > + build_append_int_noprefix(table, raw_data_offset, 4); > + /* Raw Data Length */ > + build_append_int_noprefix(table, raw_data_length, 4); > + /* Data Length */ > + build_append_int_noprefix(table, data_length, 4); > + /* Error Severity */ > + build_append_int_noprefix(table, error_severity, 4); > +} > + > +/* UEFI 2.6: N.2.5 Memory Error Section */ > +static void acpi_ghes_build_append_mem_cper(GArray *table, > + uint64_t error_physical_addr) > +{ > + /* > + * Memory Error Record > + */ > + > + /* Validation Bits */ > + build_append_int_noprefix(table, > + (1ULL << 14) | /* Type Valid */ > + (1ULL << 1) /* Physical Address Valid */, > + 8); > + /* Error Status */ > + build_append_int_noprefix(table, 0, 8); > + /* Physical Address */ > + build_append_int_noprefix(table, error_physical_addr, 8); > + /* Skip all the detailed information normally found in such a record */ > + build_append_int_noprefix(table, 0, 48); > + /* Memory Error Type */ > + build_append_int_noprefix(table, 0 /* Unknown error */, 1); > + /* Skip all the detailed information normally found in such a record */ > + build_append_int_noprefix(table, 0, 7); > +} > + > +static int acpi_ghes_record_mem_error(uint64_t error_block_address, > + uint64_t error_physical_addr) > +{ > + GArray *block; > + /* Memory Error Section Type */ > + QemuUUID mem_section_id_le = UEFI_CPER_SEC_PLATFORM_MEM; > + QemuUUID fru_id = {}; add comment /* invalid fru id: UEFI 2.6: ..., table ... > + uint32_t data_length; > + > + block = g_array_new(false, true /* clear */, 1); > + > + /* This is the length if adding a new generic error data entry*/ > + data_length = ACPI_GHES_DATA_LENGTH + ACPI_GHES_MEM_CPER_LENGTH; > + > + /* > + * Check whether it will run out of the preallocated memory if adding a new > + * generic error data entry > + */ > + if ((data_length + ACPI_GHES_GESB_SIZE) > ACPI_GHES_MAX_RAW_DATA_LENGTH) { > + error_report("Not enough memory to record new CPER!!!"); > + return -1; > + } > + > + /* Build the new generic error status block header */ > + acpi_ghes_generic_error_status(block, ACPI_GEBS_UNCORRECTABLE, > + 0, 0, data_length, ACPI_CPER_SEV_RECOVERABLE); > + > + /* Build this new generic error data entry header */ > + acpi_ghes_generic_error_data(block, mem_section_id_le, > + ACPI_CPER_SEV_RECOVERABLE, 0, 0, > + ACPI_GHES_MEM_CPER_LENGTH, fru_id, 0); > + > + /* Build the memory section CPER for above new generic error data entry */ > + acpi_ghes_build_append_mem_cper(block, error_physical_addr); > + > + /* Write back above this new generic error data entry to guest memory */ /* Write the generic error data entry into guest memory */ > + cpu_physical_memory_write(error_block_address, block->data, block->len); > + > + g_array_free(block, true); > + > + return 0; > +} > + > +/* > * Build table for the hardware error fw_cfg blob. > * Initialize "etc/hardware_errors" and "etc/hardware_errors_addr" fw_cfg blobs. > * See docs/specs/acpi_hest_ghes.rst for blobs format. > @@ -224,3 +380,67 @@ void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, > fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, > NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); > } > + > +int acpi_ghes_record_errors(uint8_t source_id, uint64_t physical_address) > +{ > + uint64_t error_block_addr, read_ack_register_addr, read_ack_register = 0; > + int loop = 0; > + uint64_t start_addr; > + bool ret = -1; > + AcpiGedState *acpi_ged_state; > + AcpiGhesState *ags; > + > + assert(source_id < ACPI_HEST_SRC_ID_RESERVED); > + > + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, > + NULL)); > + if (acpi_ged_state) { > + ags = &acpi_ged_state->ghes_state; > + } else { > + error_report("ACPI GED device not found"); > + return -1; > + } > + > + start_addr = le64_to_cpu(ags->ghes_addr_le); > + > + if (physical_address) { > + > + if (source_id < ACPI_HEST_SRC_ID_RESERVED) { > + start_addr += source_id * sizeof(uint64_t); > + } > + > + cpu_physical_memory_read(start_addr, &error_block_addr, > + sizeof(error_block_addr)); > + > + read_ack_register_addr = start_addr + > + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t); > +retry: > + cpu_physical_memory_read(read_ack_register_addr, > + &read_ack_register, sizeof(read_ack_register)); > + > + /* zero means OSPM does not acknowledge the error */ > + if (!read_ack_register) { > + if (loop < 3) { > + usleep(100 * 1000); > + loop++; > + goto retry; it's not doing what commit message says if message is supposed to be dropped, just drop it without any delay loops > + } else { > + error_report("OSPM does not acknowledge previous error," > + " so can not record CPER for current error anymore"); > + } > + } else if (error_block_addr) { > + read_ack_register = cpu_to_le64(0); ^^^^ pointless "= 0" will do just fine > + /* > + * Clear the Read Ack Register, OSPM will write it to 1 when > + * acknowledge this error. s/acknowledge/it acknowledges/ > + */ > + cpu_physical_memory_write(read_ack_register_addr, > + &read_ack_register, sizeof(uint64_t)); > + > + ret = acpi_ghes_record_mem_error(error_block_addr, > + physical_address); > + } > + } > + > + return ret; > +} > diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h > index a6761e6..ab0ae33 100644 > --- a/include/hw/acpi/ghes.h > +++ b/include/hw/acpi/ghes.h > @@ -22,6 +22,8 @@ > #ifndef ACPI_GHES_H > #define ACPI_GHES_H > > +#include "hw/acpi/bios-linker-loader.h" > + > /* > * Values for Hardware Error Notification Type field > */ > @@ -69,4 +71,5 @@ void acpi_build_hest(GArray *table_data, GArray *hardware_error, > BIOSLinker *linker); > void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, > GArray *hardware_errors); > +int acpi_ghes_record_errors(uint8_t notify, uint64_t error_physical_addr); > #endif > diff --git a/include/qemu/uuid.h b/include/qemu/uuid.h > index 129c45f..b35e294 100644 > --- a/include/qemu/uuid.h > +++ b/include/qemu/uuid.h > @@ -34,6 +34,11 @@ typedef struct { > }; > } QemuUUID; > > +#define UUID_LE(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7) \ > + {{{ (a) & 0xff, ((a) >> 8) & 0xff, ((a) >> 16) & 0xff, ((a) >> 24) & 0xff, \ > + (b) & 0xff, ((b) >> 8) & 0xff, (c) & 0xff, ((c) >> 8) & 0xff, \ > + (d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) } } } since you are adding generalizing macro, take of NVDIMM_UUID_LE which served as model > #define UUID_FMT "%02hhx%02hhx%02hhx%02hhx-" \ > "%02hhx%02hhx-%02hhx%02hhx-" \ > "%02hhx%02hhx-" \
On Tue, 28 Jan 2020 15:41:10 +0100 Igor Mammedov <imammedo@redhat.com> wrote: > On Wed, 8 Jan 2020 19:32:19 +0800 > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > > in addition to comments of others: > > > Record the GHEB address via fw_cfg file, when recording > > a error to CPER, it will use this address to find out > > Generic Error Data Entries and write the error. > > > > Make the HEST GHES to a GED device. > > It's hard to parse this even kno mine comments are not readable either, let me try again: What I've lost at the end and wanted to say is that even when one knows context it's hard to read commit message and it will definitely be unreadable when one forgets context. > Pls rephrase/make commit message more verbose, > so it would describe why and what patch is supposed to do [...]
sorry for the late response due to Chinese new year On 2020/1/28 22:41, Igor Mammedov wrote: > On Wed, 8 Jan 2020 19:32:19 +0800 > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > > in addition to comments of others: > >> Record the GHEB address via fw_cfg file, when recording >> a error to CPER, it will use this address to find out >> Generic Error Data Entries and write the error. >> >> Make the HEST GHES to a GED device. > > It's hard to parse this even kno > Pls rephrase/make commit message more verbose, > so it would describe why and what patch is supposed to do Ok, thanks for the comments. > > >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >> --- >> hw/acpi/generic_event_device.c | 15 ++++++++++++++- >> hw/acpi/ghes.c | 16 ++++++++++++++++ >> hw/arm/virt-acpi-build.c | 13 ++++++++++++- >> include/hw/acpi/generic_event_device.h | 2 ++ >> include/hw/acpi/ghes.h | 6 ++++++ >> 5 files changed, 50 insertions(+), 2 deletions(-) >> >> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c >> index 9cee90c..9bf37e4 100644 >> --- a/hw/acpi/generic_event_device.c >> +++ b/hw/acpi/generic_event_device.c >> @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = { >> } >> }; >> >> +static const VMStateDescription vmstate_ghes_state = { >> + .name = "acpi-ghes-state", >> + .version_id = 1, >> + .minimum_version_id = 1, >> + .fields = (VMStateField[]) { >> + VMSTATE_UINT64(ghes_addr_le, AcpiGhesState), >> + VMSTATE_END_OF_LIST() >> + } >> +}; >> + >> static const VMStateDescription vmstate_acpi_ged = { >> .name = "acpi-ged", >> .version_id = 1, >> .minimum_version_id = 1, >> .fields = (VMStateField[]) { >> - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState), >> + VMSTATE_STRUCT(ged_state, AcpiGedState, 1, >> + vmstate_ged_state, GEDState), >> + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, >> + vmstate_ghes_state, AcpiGhesState), >> VMSTATE_END_OF_LIST(), >> }, >> .subsections = (const VMStateDescription * []) { >> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c >> index 9d37798..68f4abf 100644 >> --- a/hw/acpi/ghes.c >> +++ b/hw/acpi/ghes.c >> @@ -23,6 +23,7 @@ >> #include "hw/acpi/acpi.h" >> #include "hw/acpi/ghes.h" >> #include "hw/acpi/aml-build.h" >> +#include "hw/acpi/generic_event_device.h" >> #include "hw/nvram/fw_cfg.h" >> #include "sysemu/sysemu.h" >> #include "qemu/error-report.h" >> @@ -208,3 +209,18 @@ void acpi_build_hest(GArray *table_data, GArray *hardware_errors, >> build_header(linker, table_data, (void *)(table_data->data + hest_start), >> "HEST", table_data->len - hest_start, 1, NULL, ""); >> } >> + >> +void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, >> + GArray *hardware_error) > > not aligned properly will modify it. > >> +{ >> + size_t size = 2 * sizeof(uint64_t) + ACPI_GHES_MAX_RAW_DATA_LENGTH; >> + size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size; >> + >> + /* Create a read-only fw_cfg file for GHES */ >> + fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data, >> + request_block_size); >> + >> + /* Create a read-write fw_cfg file for Address */ >> + fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, >> + NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); >> +} >> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c >> index 837bbf9..c8aa94d 100644 >> --- a/hw/arm/virt-acpi-build.c >> +++ b/hw/arm/virt-acpi-build.c >> @@ -797,6 +797,7 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) >> unsigned dsdt, xsdt; >> GArray *tables_blob = tables->table_data; >> MachineState *ms = MACHINE(vms); >> + AcpiGedState *acpi_ged_state; >> >> table_offsets = g_array_new(false, true /* clear */, >> sizeof(uint32_t)); >> @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) >> acpi_add_table(table_offsets, tables_blob); >> build_spcr(tables_blob, tables->linker, vms); >> >> - if (vms->ras) { >> + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, >> + NULL)); >> + if (acpi_ged_state && vms->ras) { > > there is vms->acpi_dev which is GED, so you don't need to look it up > > suggest: Thanks for the suggestion. > if (ras) { > assert(ged) assert(vms->acpi_dev), right? > do other fun stuff ... > } > >> acpi_add_table(table_offsets, tables_blob); >> build_ghes_error_table(tables->hardware_errors, tables->linker); >> acpi_build_hest(tables_blob, tables->hardware_errors, >> @@ -925,6 +928,7 @@ void virt_acpi_setup(VirtMachineState *vms) >> { >> AcpiBuildTables tables; >> AcpiBuildState *build_state; >> + AcpiGedState *acpi_ged_state; >> >> if (!vms->fw_cfg) { >> trace_virt_acpi_setup(); >> @@ -955,6 +959,13 @@ void virt_acpi_setup(VirtMachineState *vms) >> fw_cfg_add_file(vms->fw_cfg, ACPI_BUILD_TPMLOG_FILE, tables.tcpalog->data, >> acpi_data_len(tables.tcpalog)); >> >> + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, >> + NULL)); >> + if (acpi_ged_state && vms->ras) { > > ditto Ok. > >> + acpi_ghes_add_fw_cfg(&acpi_ged_state->ghes_state, >> + vms->fw_cfg, tables.hardware_errors); >> + } >> + >> build_state->rsdp_mr = acpi_add_rom_blob(virt_acpi_build_update, >> build_state, tables.rsdp, >> ACPI_BUILD_RSDP_FILE, 0); >> diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h >> index d157eac..037d2b5 100644 >> --- a/include/hw/acpi/generic_event_device.h >> +++ b/include/hw/acpi/generic_event_device.h >> @@ -61,6 +61,7 @@ >> >> #include "hw/sysbus.h" >> #include "hw/acpi/memory_hotplug.h" >> +#include "hw/acpi/ghes.h" >> >> #define ACPI_POWER_BUTTON_DEVICE "PWRB" >> >> @@ -95,6 +96,7 @@ typedef struct AcpiGedState { >> GEDState ged_state; >> uint32_t ged_event_bitmap; >> qemu_irq irq; >> + AcpiGhesState ghes_state; >> } AcpiGedState; >> >> void build_ged_aml(Aml *table, const char* name, HotplugHandler *hotplug_dev, >> diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h >> index 09a7f86..a6761e6 100644 >> --- a/include/hw/acpi/ghes.h >> +++ b/include/hw/acpi/ghes.h >> @@ -60,7 +60,13 @@ enum { >> ACPI_HEST_SRC_ID_RESERVED, >> }; >> >> +typedef struct AcpiGhesState { >> + uint64_t ghes_addr_le; >> +} AcpiGhesState; >> + >> void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); >> void acpi_build_hest(GArray *table_data, GArray *hardware_error, >> BIOSLinker *linker); >> +void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, >> + GArray *hardware_errors); >> #endif > > . >
On 2020/1/28 23:29, Igor Mammedov wrote: > On Wed, 8 Jan 2020 19:32:21 +0800 > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > >> kvm_arch_on_sigbus_vcpu() error injection uses source_id as >> index in etc/hardware_errors to find out Error Status Data >> Block entry corresponding to error source. So supported source_id >> values should be assigned here and not be changed afterwards to >> make sure that guest will write error into expected Error Status >> Data Block even if guest was migrated to a newer QEMU. >> >> Before QEMU writes a new error to ACPI table, it will check whether >> previous error has been acknowledged. Otherwise it will ignore the new >> error. For the errors section type, QEMU simulate it to memory section >> error. >> >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> >> --- >> hw/acpi/ghes.c | 224 ++++++++++++++++++++++++++++++++++++++++++++++++- >> include/hw/acpi/ghes.h | 3 + >> include/qemu/uuid.h | 5 ++ >> 3 files changed, 230 insertions(+), 2 deletions(-) >> >> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c >> index 68f4abf..f2ecffe 100644 >> --- a/hw/acpi/ghes.c >> +++ b/hw/acpi/ghes.c >> @@ -28,21 +28,56 @@ >> #include "sysemu/sysemu.h" >> #include "qemu/error-report.h" >> >> -#include "hw/acpi/bios-linker-loader.h" > why it's moved to header? Because the function declaration "void acpi_build_hest(GArray *table_data, GArray *hardware_error, BIOSLinker *linker);" needs it in the header file, Otherwise The compilation will be failed. Anyway I will move this change to patch "[v22,4/9] ACPI: Build Hardware Error Source Table", because that patch needs it . > >> - >> #define ACPI_GHES_ERRORS_FW_CFG_FILE "etc/hardware_errors" >> #define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr" >> >> /* The max size in bytes for one error block */ >> #define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 >> + >> /* Now only support ARMv8 SEA notification type error source */ >> #define ACPI_GHES_ERROR_SOURCE_COUNT 1 >> + >> /* Generic Hardware Error Source version 2 */ >> #define ACPI_GHES_SOURCE_GENERIC_ERROR_V2 10 >> + >> /* Address offset in Generic Address Structure(GAS) */ >> #define GAS_ADDR_OFFSET 4 >> >> /* >> + * The total size of Generic Error Data Entry >> + * ACPI 6.1/6.2: 18.3.2.7.1 Generic Error Data, >> + * Table 18-343 Generic Error Data Entry >> + */ >> +#define ACPI_GHES_DATA_LENGTH 72 >> + >> +/* The memory section CPER size, UEFI 2.6: N.2.5 Memory Error Section */ >> +#define ACPI_GHES_MEM_CPER_LENGTH 80 >> + >> +/* Masks for block_status flags */ >> +#define ACPI_GEBS_UNCORRECTABLE 1 >> + >> +#define UEFI_CPER_SEC_PLATFORM_MEM \ >> + UUID_LE(0xA5BC1114, 0x6F64, 0x4EDE, 0xB8, 0x63, 0x3E, 0x83, \ >> + 0xED, 0x7C, 0x83, 0xB1) >> + >> +/* >> + * Total size for Generic Error Status Block except Generic Error Data Entries >> + * ACPI 6.2: 18.3.2.7.1 Generic Error Data, >> + * Table 18-380 Generic Error Status Block >> + */ >> +#define ACPI_GHES_GESB_SIZE 20 >> + >> +/* >> + * Values for error_severity field >> + */ >> +enum AcpiGenericErrorSeverity { >> + ACPI_CPER_SEV_RECOVERABLE = 0, >> + ACPI_CPER_SEV_FATAL = 1, >> + ACPI_CPER_SEV_CORRECTED = 2, >> + ACPI_CPER_SEV_NONE = 3, >> +}; >> + >> +/* >> * Hardware Error Notification >> * ACPI 4.0: 17.3.2.7 Hardware Error Notification >> * Composes dummy Hardware Error Notification descriptor of specified type >> @@ -73,6 +108,127 @@ static void build_ghes_hw_error_notification(GArray *table, const uint8_t type) >> } >> >> /* >> + * Generic Error Data Entry >> + * ACPI 6.1: 18.3.2.7.1 Generic Error Data >> + */ >> +static void acpi_ghes_generic_error_data(GArray *table, QemuUUID section_type, >> + uint32_t error_severity, uint8_t validation_bits, uint8_t flags, >> + uint32_t error_data_length, QemuUUID fru_id, >> + uint64_t time_stamp) >> +{ >> + /* Section Type */ >> + g_array_append_vals(table, section_type.data, >> + ARRAY_SIZE(section_type.data)); >> + >> + /* Error Severity */ >> + build_append_int_noprefix(table, error_severity, 4); >> + /* Revision */ >> + build_append_int_noprefix(table, 0x300, 2); >> + /* Validation Bits */ >> + build_append_int_noprefix(table, validation_bits, 1); >> + /* Flags */ >> + build_append_int_noprefix(table, flags, 1); >> + /* Error Data Length */ >> + build_append_int_noprefix(table, error_data_length, 4); >> + >> + /* FRU Id */ >> + g_array_append_vals(table, fru_id.data, ARRAY_SIZE(fru_id.data)); >> + >> + /* FRU Text */ >> + build_append_int_noprefix(table, 0, 20); >> + /* Timestamp */ >> + build_append_int_noprefix(table, time_stamp, 8); >> +} >> + >> +/* >> + * Generic Error Status Block >> + * ACPI 6.1: 18.3.2.7.1 Generic Error Data >> + */ >> +static void acpi_ghes_generic_error_status(GArray *table, uint32_t block_status, >> + uint32_t raw_data_offset, uint32_t raw_data_length, >> + uint32_t data_length, uint32_t error_severity) >> +{ >> + /* Block Status */ >> + build_append_int_noprefix(table, block_status, 4); >> + /* Raw Data Offset */ >> + build_append_int_noprefix(table, raw_data_offset, 4); >> + /* Raw Data Length */ >> + build_append_int_noprefix(table, raw_data_length, 4); >> + /* Data Length */ >> + build_append_int_noprefix(table, data_length, 4); >> + /* Error Severity */ >> + build_append_int_noprefix(table, error_severity, 4); >> +} >> + >> +/* UEFI 2.6: N.2.5 Memory Error Section */ >> +static void acpi_ghes_build_append_mem_cper(GArray *table, >> + uint64_t error_physical_addr) >> +{ >> + /* >> + * Memory Error Record >> + */ >> + >> + /* Validation Bits */ >> + build_append_int_noprefix(table, >> + (1ULL << 14) | /* Type Valid */ >> + (1ULL << 1) /* Physical Address Valid */, >> + 8); >> + /* Error Status */ >> + build_append_int_noprefix(table, 0, 8); >> + /* Physical Address */ >> + build_append_int_noprefix(table, error_physical_addr, 8); >> + /* Skip all the detailed information normally found in such a record */ >> + build_append_int_noprefix(table, 0, 48); >> + /* Memory Error Type */ >> + build_append_int_noprefix(table, 0 /* Unknown error */, 1); >> + /* Skip all the detailed information normally found in such a record */ >> + build_append_int_noprefix(table, 0, 7); >> +} >> + >> +static int acpi_ghes_record_mem_error(uint64_t error_block_address, >> + uint64_t error_physical_addr) >> +{ >> + GArray *block; >> + /* Memory Error Section Type */ >> + QemuUUID mem_section_id_le = UEFI_CPER_SEC_PLATFORM_MEM; >> + QemuUUID fru_id = {}; > add comment /* invalid fru id: UEFI 2.6: ..., table ... ok. > > >> + uint32_t data_length; >> + >> + block = g_array_new(false, true /* clear */, 1); >> + >> + /* This is the length if adding a new generic error data entry*/ >> + data_length = ACPI_GHES_DATA_LENGTH + ACPI_GHES_MEM_CPER_LENGTH; >> + >> + /* >> + * Check whether it will run out of the preallocated memory if adding a new >> + * generic error data entry >> + */ >> + if ((data_length + ACPI_GHES_GESB_SIZE) > ACPI_GHES_MAX_RAW_DATA_LENGTH) { >> + error_report("Not enough memory to record new CPER!!!"); >> + return -1; >> + } >> + >> + /* Build the new generic error status block header */ >> + acpi_ghes_generic_error_status(block, ACPI_GEBS_UNCORRECTABLE, >> + 0, 0, data_length, ACPI_CPER_SEV_RECOVERABLE); >> + >> + /* Build this new generic error data entry header */ >> + acpi_ghes_generic_error_data(block, mem_section_id_le, >> + ACPI_CPER_SEV_RECOVERABLE, 0, 0, >> + ACPI_GHES_MEM_CPER_LENGTH, fru_id, 0); >> + >> + /* Build the memory section CPER for above new generic error data entry */ >> + acpi_ghes_build_append_mem_cper(block, error_physical_addr); >> + >> + /* Write back above this new generic error data entry to guest memory */ > > /* Write the generic error data entry into guest memory */ >> + cpu_physical_memory_write(error_block_address, block->data, block->len); >> + >> + g_array_free(block, true); >> + >> + return 0; >> +} >> + >> +/* >> * Build table for the hardware error fw_cfg blob. >> * Initialize "etc/hardware_errors" and "etc/hardware_errors_addr" fw_cfg blobs. >> * See docs/specs/acpi_hest_ghes.rst for blobs format. >> @@ -224,3 +380,67 @@ void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, >> fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, >> NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); >> } >> + >> +int acpi_ghes_record_errors(uint8_t source_id, uint64_t physical_address) >> +{ >> + uint64_t error_block_addr, read_ack_register_addr, read_ack_register = 0; >> + int loop = 0; >> + uint64_t start_addr; >> + bool ret = -1; >> + AcpiGedState *acpi_ged_state; >> + AcpiGhesState *ags; >> + >> + assert(source_id < ACPI_HEST_SRC_ID_RESERVED); >> + >> + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, >> + NULL)); >> + if (acpi_ged_state) { >> + ags = &acpi_ged_state->ghes_state; >> + } else { >> + error_report("ACPI GED device not found"); >> + return -1; >> + } >> + >> + start_addr = le64_to_cpu(ags->ghes_addr_le); >> + >> + if (physical_address) { >> + >> + if (source_id < ACPI_HEST_SRC_ID_RESERVED) { >> + start_addr += source_id * sizeof(uint64_t); >> + } >> + >> + cpu_physical_memory_read(start_addr, &error_block_addr, >> + sizeof(error_block_addr)); >> + >> + read_ack_register_addr = start_addr + >> + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t); >> +retry: >> + cpu_physical_memory_read(read_ack_register_addr, >> + &read_ack_register, sizeof(read_ack_register)); >> + >> + /* zero means OSPM does not acknowledge the error */ >> + if (!read_ack_register) { >> + if (loop < 3) { >> + usleep(100 * 1000); >> + loop++; >> + goto retry; > it's not doing what commit message says > > if message is supposed to be dropped, just drop it without any delay loops. ok, I will drop it. > >> + } else { >> + error_report("OSPM does not acknowledge previous error," >> + " so can not record CPER for current error anymore"); >> + } >> + } else if (error_block_addr) { >> + read_ack_register = cpu_to_le64(0); > ^^^^ pointless "= 0" will do just fine > >> + /* >> + * Clear the Read Ack Register, OSPM will write it to 1 when >> + * acknowledge this error. > s/acknowledge/it acknowledges/ Thanks for this pointing out. > >> + */ >> + cpu_physical_memory_write(read_ack_register_addr, >> + &read_ack_register, sizeof(uint64_t)); >> + >> + ret = acpi_ghes_record_mem_error(error_block_addr, >> + physical_address); >> + } >> + } >> + >> + return ret; >> +} >> diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h >> index a6761e6..ab0ae33 100644 >> --- a/include/hw/acpi/ghes.h >> +++ b/include/hw/acpi/ghes.h >> @@ -22,6 +22,8 @@ >> #ifndef ACPI_GHES_H >> #define ACPI_GHES_H >> >> +#include "hw/acpi/bios-linker-loader.h" >> + >> /* >> * Values for Hardware Error Notification Type field >> */ >> @@ -69,4 +71,5 @@ void acpi_build_hest(GArray *table_data, GArray *hardware_error, >> BIOSLinker *linker); >> void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, >> GArray *hardware_errors); >> +int acpi_ghes_record_errors(uint8_t notify, uint64_t error_physical_addr); >> #endif >> diff --git a/include/qemu/uuid.h b/include/qemu/uuid.h >> index 129c45f..b35e294 100644 >> --- a/include/qemu/uuid.h >> +++ b/include/qemu/uuid.h >> @@ -34,6 +34,11 @@ typedef struct { >> }; >> } QemuUUID; >> >> +#define UUID_LE(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7) \ >> + {{{ (a) & 0xff, ((a) >> 8) & 0xff, ((a) >> 16) & 0xff, ((a) >> 24) & 0xff, \ >> + (b) & 0xff, ((b) >> 8) & 0xff, (c) & 0xff, ((c) >> 8) & 0xff, \ >> + (d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) } } } > > since you are adding generalizing macro, take of NVDIMM_UUID_LE which served as model do you mean use this generalizing macro to replace NVDIMM_UUID_LE, right? > > >> #define UUID_FMT "%02hhx%02hhx%02hhx%02hhx-" \ >> "%02hhx%02hhx-%02hhx%02hhx-" \ >> "%02hhx%02hhx-" \ > > . >
On 2020/1/23 23:14, Igor Mammedov wrote: > On Wed, 8 Jan 2020 19:32:17 +0800 > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > >> This patch builds error_block_address and read_ack_register fields >> in hardware errors table , the error_block_address points to Generic >> Error Status Block(GESB) via bios_linker. The max size for one GESB >> is 0x1000 in bytes, For more detailed information, please refer to > > s/0x1000/4Kb/ will update it. > >> document: docs/specs/acpi_hest_ghes.rst >> >> Now we only support one Error source, if necessary, we can extend to >> support more. >> >> Suggested-by: Laszlo Ersek <lersek@redhat.com> >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> >> --- >> default-configs/arm-softmmu.mak | 1 + >> hw/acpi/Kconfig | 4 ++ >> hw/acpi/Makefile.objs | 1 + >> hw/acpi/aml-build.c | 2 + >> hw/acpi/ghes.c | 94 +++++++++++++++++++++++++++++++++++++++++ >> hw/arm/virt-acpi-build.c | 6 +++ >> include/hw/acpi/aml-build.h | 1 + >> include/hw/acpi/ghes.h | 26 ++++++++++++ >> 8 files changed, 135 insertions(+) >> create mode 100644 hw/acpi/ghes.c >> create mode 100644 include/hw/acpi/ghes.h >> >> diff --git a/default-configs/arm-softmmu.mak b/default-configs/arm-softmmu.mak >> index 1f2e0e7..5722f31 100644 >> --- a/default-configs/arm-softmmu.mak >> +++ b/default-configs/arm-softmmu.mak >> @@ -40,3 +40,4 @@ CONFIG_FSL_IMX25=y >> CONFIG_FSL_IMX7=y >> CONFIG_FSL_IMX6UL=y >> CONFIG_SEMIHOSTING=y >> +CONFIG_ACPI_APEI=y >> diff --git a/hw/acpi/Kconfig b/hw/acpi/Kconfig >> index 12e3f1e..ed8c34d 100644 >> --- a/hw/acpi/Kconfig >> +++ b/hw/acpi/Kconfig >> @@ -23,6 +23,10 @@ config ACPI_NVDIMM >> bool >> depends on ACPI >> >> +config ACPI_APEI >> + bool >> + depends on ACPI >> + >> config ACPI_PCI >> bool >> depends on ACPI && PCI >> diff --git a/hw/acpi/Makefile.objs b/hw/acpi/Makefile.objs >> index 9925305..7b5db4b 100644 >> --- a/hw/acpi/Makefile.objs >> +++ b/hw/acpi/Makefile.objs >> @@ -5,6 +5,7 @@ common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu_hotplug.o >> common-obj-$(CONFIG_ACPI_MEMORY_HOTPLUG) += memory_hotplug.o >> common-obj-$(CONFIG_ACPI_CPU_HOTPLUG) += cpu.o >> common-obj-$(CONFIG_ACPI_NVDIMM) += nvdimm.o >> +common-obj-$(CONFIG_ACPI_APEI) += ghes.o >> common-obj-$(CONFIG_ACPI_VMGENID) += vmgenid.o >> common-obj-$(CONFIG_ACPI_HW_REDUCED) += generic_event_device.o >> common-obj-$(call lnot,$(CONFIG_ACPI_X86)) += acpi-stub.o >> diff --git a/hw/acpi/aml-build.c b/hw/acpi/aml-build.c >> index 2c3702b..3681ec6 100644 >> --- a/hw/acpi/aml-build.c >> +++ b/hw/acpi/aml-build.c >> @@ -1578,6 +1578,7 @@ void acpi_build_tables_init(AcpiBuildTables *tables) >> tables->table_data = g_array_new(false, true /* clear */, 1); >> tables->tcpalog = g_array_new(false, true /* clear */, 1); >> tables->vmgenid = g_array_new(false, true /* clear */, 1); >> + tables->hardware_errors = g_array_new(false, true /* clear */, 1); >> tables->linker = bios_linker_loader_init(); >> } >> >> @@ -1588,6 +1589,7 @@ void acpi_build_tables_cleanup(AcpiBuildTables *tables, bool mfre) >> g_array_free(tables->table_data, true); >> g_array_free(tables->tcpalog, mfre); >> g_array_free(tables->vmgenid, mfre); >> + g_array_free(tables->hardware_errors, mfre); >> } >> >> /* >> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c >> new file mode 100644 >> index 0000000..b7fdbbb >> --- /dev/null >> +++ b/hw/acpi/ghes.c >> @@ -0,0 +1,94 @@ >> +/* >> + * Support for generating APEI tables and recording CPER for Guests >> + * >> + * Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. > s/2019/2020/ > > the same for other places that add this string within this series sure, thanks for the pointing out. > >> + * >> + * Author: Dongjiu Geng <gengdongjiu@huawei.com> >> + * >> + * This program is free software; you can redistribute it and/or modify >> + * it under the terms of the GNU General Public License as published by >> + * the Free Software Foundation; either version 2 of the License, or >> + * (at your option) any later version. >> + >> + * This program is distributed in the hope that it will be useful, >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> + * GNU General Public License for more details. >> + >> + * You should have received a copy of the GNU General Public License along >> + * with this program; if not, see <http://www.gnu.org/licenses/>. >> + */ >> + >> +#include "qemu/osdep.h" >> +#include "hw/acpi/acpi.h" >> +#include "hw/acpi/ghes.h" >> +#include "hw/acpi/aml-build.h" >> +#include "hw/nvram/fw_cfg.h" >> +#include "sysemu/sysemu.h" >> +#include "qemu/error-report.h" >> + >> +#include "hw/acpi/bios-linker-loader.h" >> + >> +#define ACPI_GHES_ERRORS_FW_CFG_FILE "etc/hardware_errors" >> +#define ACPI_GHES_DATA_ADDR_FW_CFG_FILE "etc/hardware_errors_addr" >> + >> +/* The max size in bytes for one error block */ >> +#define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 > > using 1 * KiB would make it more readable sure > > >> + >> +/* Now only support ARMv8 SEA notification type error source */ >> +#define ACPI_GHES_ERROR_SOURCE_COUNT 1 >> + >> +/* >> + * Build table for the hardware error fw_cfg blob. >> + * Initialize "etc/hardware_errors" and "etc/hardware_errors_addr" fw_cfg blobs. >> + * See docs/specs/acpi_hest_ghes.rst for blobs format. >> + */ >> +void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker) >> +{ >> + int i, error_status_block_offset; >> + >> + /* Build error_block_address */ >> + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { >> + build_append_int_noprefix(hardware_errors, 0, sizeof(uint64_t)); >> + } >> + >> + /* Build read_ack_register */ >> + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { >> + /* >> + * Initialize the value of read_ack_register to 1, so GHES can be >> + * writeable after (re)boot. >> + * ACPI 6.2: 18.3.2.8 Generic Hardware Error Source version 2 >> + * (GHESv2 - Type 10) >> + */ >> + build_append_int_noprefix(hardware_errors, 1, sizeof(uint64_t)); >> + } >> + >> + /* Generic Error Status Block offset in the hardware error fw_cfg blob */ >> + error_status_block_offset = hardware_errors->len; >> + >> + /* Reserve space for Error Status Data Block */ >> + acpi_data_push(hardware_errors, >> + ACPI_GHES_MAX_RAW_DATA_LENGTH * ACPI_GHES_ERROR_SOURCE_COUNT); >> + >> + /* Tell guest firmware to place hardware_errors blob into RAM */ >> + bios_linker_loader_alloc(linker, ACPI_GHES_ERRORS_FW_CFG_FILE, >> + hardware_errors, 1, false); > ^^ > /* > * bios_linker_loader_alloc: ask guest to load file into guest memory. > [...] > * @alloc_align: required minimal alignment in bytes. Must be a power of 2. > Maybe use 8 here to make 64bit addresses naturally aligned? I think we can use "sizeof(uint64_t)" to make it aligned. > >> + >> + for (i = 0; i < ACPI_GHES_ERROR_SOURCE_COUNT; i++) { >> + /* >> + * Tell firmware to patch error_block_address entries to point to >> + * corresponding "Generic Error Status Block" >> + */ >> + bios_linker_loader_add_pointer(linker, >> + ACPI_GHES_ERRORS_FW_CFG_FILE, sizeof(uint64_t) * i, >> + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, >> + error_status_block_offset + i * ACPI_GHES_MAX_RAW_DATA_LENGTH); >> + } >> + >> + /* >> + * tell firmware to write hardware_errors GPA into >> + * hardware_errors_addr fw_cfg, once the former has been initialized. >> + */ >> + bios_linker_loader_write_pointer(linker, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, >> + 0, sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); >> +} >> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c >> index bd5f771..6819fcf 100644 >> --- a/hw/arm/virt-acpi-build.c >> +++ b/hw/arm/virt-acpi-build.c >> @@ -48,6 +48,7 @@ >> #include "sysemu/reset.h" >> #include "kvm_arm.h" >> #include "migration/vmstate.h" >> +#include "hw/acpi/ghes.h" >> >> #define ARM_SPI_BASE 32 >> >> @@ -830,6 +831,11 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) >> acpi_add_table(table_offsets, tables_blob); >> build_spcr(tables_blob, tables->linker, vms); >> >> + if (vms->ras) { >> + acpi_add_table(table_offsets, tables_blob); >> + build_ghes_error_table(tables->hardware_errors, tables->linker); >> + } >> + >> if (ms->numa_state->num_nodes > 0) { >> acpi_add_table(table_offsets, tables_blob); >> build_srat(tables_blob, tables->linker, vms); >> diff --git a/include/hw/acpi/aml-build.h b/include/hw/acpi/aml-build.h >> index de4a406..8f13620 100644 >> --- a/include/hw/acpi/aml-build.h >> +++ b/include/hw/acpi/aml-build.h >> @@ -220,6 +220,7 @@ struct AcpiBuildTables { >> GArray *rsdp; >> GArray *tcpalog; >> GArray *vmgenid; >> + GArray *hardware_errors; >> BIOSLinker *linker; >> } AcpiBuildTables; >> >> diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h >> new file mode 100644 >> index 0000000..3dbda3f >> --- /dev/null >> +++ b/include/hw/acpi/ghes.h >> @@ -0,0 +1,26 @@ >> +/* >> + * Support for generating APEI tables and recording CPER for Guests >> + * >> + * Copyright (c) 2019 HUAWEI TECHNOLOGIES CO., LTD. >> + * >> + * Author: Dongjiu Geng <gengdongjiu@huawei.com> >> + * >> + * This program is free software; you can redistribute it and/or modify >> + * it under the terms of the GNU General Public License as published by >> + * the Free Software Foundation; either version 2 of the License, or >> + * (at your option) any later version. >> + >> + * This program is distributed in the hope that it will be useful, >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the >> + * GNU General Public License for more details. >> + >> + * You should have received a copy of the GNU General Public License along >> + * with this program; if not, see <http://www.gnu.org/licenses/>. >> + */ >> + >> +#ifndef ACPI_GHES_H >> +#define ACPI_GHES_H >> + >> +void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); >> +#endif > > . >
On 2020/1/23 23:48, Igor Mammedov wrote: > On Wed, 8 Jan 2020 19:32:18 +0800 > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > >> This patch builds Hardware Error Source Table(HEST) via fw_cfg blobs. >> Now it only supports ARMv8 SEA, a type of Generic Hardware Error >> Source version 2(GHESv2) error source. Afterwards, we can extend >> the supported types if needed. For the CPER section, currently it >> is memory section because kernel mainly wants userspace to handle >> the memory errors. >> >> This patch follows the spec ACPI 6.2 to build the Hardware Error >> Source table. For more detailed information, please refer to >> document: docs/specs/acpi_hest_ghes.rst >> >> build_append_ghes_notify() will help to add Hardware Error Notification >> to ACPI tables without using packed C structures and avoid endianness >> issues as API doesn't need explicit conversion. >> >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> >> Acked-by: Xiang Zheng <zhengxiang9@huawei.com> > > > Overall it looks fine to me, see couple nits below > > >> --- >> hw/acpi/ghes.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++- >> hw/arm/virt-acpi-build.c | 2 + >> include/hw/acpi/ghes.h | 40 ++++++++++++++++ >> 3 files changed, 159 insertions(+), 1 deletion(-) >> >> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c >> index b7fdbbb..9d37798 100644 >> --- a/hw/acpi/ghes.c >> +++ b/hw/acpi/ghes.c >> @@ -34,9 +34,42 @@ >> >> /* The max size in bytes for one error block */ >> #define ACPI_GHES_MAX_RAW_DATA_LENGTH 0x400 >> - >> /* Now only support ARMv8 SEA notification type error source */ >> #define ACPI_GHES_ERROR_SOURCE_COUNT 1 >> +/* Generic Hardware Error Source version 2 */ >> +#define ACPI_GHES_SOURCE_GENERIC_ERROR_V2 10 >> +/* Address offset in Generic Address Structure(GAS) */ >> +#define GAS_ADDR_OFFSET 4 >> + >> +/* >> + * Hardware Error Notification >> + * ACPI 4.0: 17.3.2.7 Hardware Error Notification >> + * Composes dummy Hardware Error Notification descriptor of specified type >> + */ >> +static void build_ghes_hw_error_notification(GArray *table, const uint8_t type) >> +{ >> + /* Type */ >> + build_append_int_noprefix(table, type, 1); >> + /* >> + * Length: >> + * Total length of the structure in bytes >> + */ >> + build_append_int_noprefix(table, 28, 1); >> + /* Configuration Write Enable */ >> + build_append_int_noprefix(table, 0, 2); >> + /* Poll Interval */ >> + build_append_int_noprefix(table, 0, 4); >> + /* Vector */ >> + build_append_int_noprefix(table, 0, 4); >> + /* Switch To Polling Threshold Value */ >> + build_append_int_noprefix(table, 0, 4); >> + /* Switch To Polling Threshold Window */ >> + build_append_int_noprefix(table, 0, 4); >> + /* Error Threshold Value */ >> + build_append_int_noprefix(table, 0, 4); >> + /* Error Threshold Window */ >> + build_append_int_noprefix(table, 0, 4); >> +} >> >> /* >> * Build table for the hardware error fw_cfg blob. >> @@ -92,3 +125,86 @@ void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker) >> bios_linker_loader_write_pointer(linker, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, >> 0, sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); >> } >> + >> +/* Build Generic Hardware Error Source version 2 (GHESv2) */ >> +static void build_ghes_v2(GArray *table_data, int source_id, BIOSLinker *linker) >> +{ >> + uint64_t address_offset; >> + /* >> + * Type: >> + * Generic Hardware Error Source version 2(GHESv2 - Type 10) >> + */ >> + build_append_int_noprefix(table_data, ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2); >> + /* Source Id */ >> + build_append_int_noprefix(table_data, source_id, 2); >> + /* Related Source Id */ >> + build_append_int_noprefix(table_data, 0xffff, 2); >> + /* Flags */ >> + build_append_int_noprefix(table_data, 0, 1); >> + /* Enabled */ >> + build_append_int_noprefix(table_data, 1, 1); >> + >> + /* Number of Records To Pre-allocate */ >> + build_append_int_noprefix(table_data, 1, 4); >> + /* Max Sections Per Record */ >> + build_append_int_noprefix(table_data, 1, 4); >> + /* Max Raw Data Length */ >> + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); >> + >> + address_offset = table_data->len; >> + /* Error Status Address */ >> + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, >> + 4 /* QWord access */, 0); >> + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, >> + address_offset + GAS_ADDR_OFFSET, >> + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); >> + >> + /* >> + * Notification Structure >> + * Now only enable ARMv8 SEA notification type >> + */ >> + build_ghes_hw_error_notification(table_data, ACPI_GHES_NOTIFY_SEA); >> + >> + /* Error Status Block Length */ >> + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); >> + >> + /* >> + * Read Ack Register >> + * ACPI 6.1: 18.3.2.8 Generic Hardware Error Source >> + * version 2 (GHESv2 - Type 10) >> + */ >> + address_offset = table_data->len; >> + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, >> + 4 /* QWord access */, 0); >> + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, >> + address_offset + GAS_ADDR_OFFSET, >> + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, >> + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t)); >> + >> + /* >> + * Read Ack Preserve >> + * We only provide the first bit in Read Ack Register to OSPM to write >> + * while the other bits are preserved. >> + */ >> + build_append_int_noprefix(table_data, ~0x1ULL, 8); >> + /* Read Ack Write */ >> + build_append_int_noprefix(table_data, 0x1, 8); >> +} >> + >> +/* Build Hardware Error Source Table */ >> +void acpi_build_hest(GArray *table_data, GArray *hardware_errors, > ^^^^^^^^ it seems to be unused, so why it's here? Thanks, I will remove this parameter. > >> + BIOSLinker *linker) >> +{ >> + uint64_t hest_start = table_data->len; >> + >> + /* Hardware Error Source Table header*/ >> + acpi_data_push(table_data, sizeof(AcpiTableHeader)); >> + >> + /* Error Source Count */ >> + build_append_int_noprefix(table_data, ACPI_GHES_ERROR_SOURCE_COUNT, 4); >> + >> + build_ghes_v2(table_data, ACPI_HEST_SRC_ID_SEA, linker); >> + >> + build_header(linker, table_data, (void *)(table_data->data + hest_start), >> + "HEST", table_data->len - hest_start, 1, NULL, ""); >> +} >> diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c >> index 6819fcf..837bbf9 100644 >> --- a/hw/arm/virt-acpi-build.c >> +++ b/hw/arm/virt-acpi-build.c >> @@ -834,6 +834,8 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) >> if (vms->ras) { >> acpi_add_table(table_offsets, tables_blob); >> build_ghes_error_table(tables->hardwarerrors, tables->linker); >> + acpi_build_hest(tables_blob, tables->hardware_errors, >> + tables->linker); > > not aligned properly > > you can use ./scripts/checkpatch.pl to see if there is style errors It will not report style errors for this alignment, anyway I will do alignment to it. > > >> } >> >> if (ms->numa_state->num_nodes > 0) { >> diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h >> index 3dbda3f..09a7f86 100644 >> --- a/include/hw/acpi/ghes.h >> +++ b/include/hw/acpi/ghes.h >> @@ -22,5 +22,45 @@ >> #ifndef ACPI_GHES_H >> #define ACPI_GHES_H >> >> +/* >> + * Values for Hardware Error Notification Type field >> + */ >> +enum AcpiGhesNotifyType { >> + /* Polled */ >> + ACPI_GHES_NOTIFY_POLLED = 0, >> + /* External Interrupt */ >> + ACPI_GHES_NOTIFY_EXTERNAL = 1, >> + /* Local Interrupt */ >> + ACPI_GHES_NOTIFY_LOCAL = 2, >> + /* SCI */ >> + ACPI_GHES_NOTIFY_SCI = 3, >> + /* NMI */ >> + ACPI_GHES_NOTIFY_NMI = 4, >> + /* CMCI, ACPI 5.0: 18.3.2.7, Table 18-290 */ >> + ACPI_GHES_NOTIFY_CMCI = 5, >> + /* MCE, ACPI 5.0: 18.3.2.7, Table 18-290 */ >> + ACPI_GHES_NOTIFY_MCE = 6, >> + /* GPIO-Signal, ACPI 6.0: 18.3.2.7, Table 18-332 */ >> + ACPI_GHES_NOTIFY_GPIO = 7, >> + /* ARMv8 SEA, ACPI 6.1: 18.3.2.9, Table 18-345 */ >> + ACPI_GHES_NOTIFY_SEA = 8, >> + /* ARMv8 SEI, ACPI 6.1: 18.3.2.9, Table 18-345 */ >> + ACPI_GHES_NOTIFY_SEI = 9, >> + /* External Interrupt - GSIV, ACPI 6.1: 18.3.2.9, Table 18-345 */ >> + ACPI_GHES_NOTIFY_GSIV = 10, >> + /* Software Delegated Exception, ACPI 6.2: 18.3.2.9, Table 18-383 */ >> + ACPI_GHES_NOTIFY_SDEI = 11, >> + /* 12 and greater are reserved */ >> + ACPI_GHES_NOTIFY_RESERVED = 12 >> +}; >> + >> +enum { >> + ACPI_HEST_SRC_ID_SEA = 0, >> + /* future ids go here */ >> + ACPI_HEST_SRC_ID_RESERVED, >> +}; >> + >> void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); >> +void acpi_build_hest(GArray *table_data, GArray *hardware_error, >> + BIOSLinker *linker); >> #endif > > . >
On Sun, 2 Feb 2020 20:44:35 +0800 gengdongjiu <gengdongjiu@huawei.com> wrote: > sorry for the late response due to Chinese new year > > On 2020/1/28 22:41, Igor Mammedov wrote: > > On Wed, 8 Jan 2020 19:32:19 +0800 > > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > > > > in addition to comments of others: > > > >> Record the GHEB address via fw_cfg file, when recording > >> a error to CPER, it will use this address to find out > >> Generic Error Data Entries and write the error. > >> > >> Make the HEST GHES to a GED device. [...] > >> @@ -831,7 +832,9 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) > >> acpi_add_table(table_offsets, tables_blob); > >> build_spcr(tables_blob, tables->linker, vms); > >> > >> - if (vms->ras) { > >> + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, > >> + NULL)); > >> + if (acpi_ged_state && vms->ras) { > > > > there is vms->acpi_dev which is GED, so you don't need to look it up > > > > suggest: > Thanks for the suggestion. > > > if (ras) { > > assert(ged) > assert(vms->acpi_dev), right? yes, something like this. > > do other fun stuff ... > > } > > > > >> acpi_add_table(table_offsets, tables_blob); > >> build_ghes_error_table(tables->hardware_errors, tables->linker); > >> acpi_build_hest(tables_blob, tables->hardware_errors, [...]
On Sun, 2 Feb 2020 21:42:23 +0800 gengdongjiu <gengdongjiu@huawei.com> wrote: > On 2020/1/28 23:29, Igor Mammedov wrote: > > On Wed, 8 Jan 2020 19:32:21 +0800 > > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > > > >> kvm_arch_on_sigbus_vcpu() error injection uses source_id as > >> index in etc/hardware_errors to find out Error Status Data > >> Block entry corresponding to error source. So supported source_id > >> values should be assigned here and not be changed afterwards to > >> make sure that guest will write error into expected Error Status > >> Data Block even if guest was migrated to a newer QEMU. > >> > >> Before QEMU writes a new error to ACPI table, it will check whether > >> previous error has been acknowledged. Otherwise it will ignore the new > >> error. For the errors section type, QEMU simulate it to memory section > >> error. > >> > >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > >> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> btw: when you are changing patch and it's not a trivial change, you are supposed to drop Reviewed-by/Acked-by tags. [...] > >> diff --git a/include/qemu/uuid.h b/include/qemu/uuid.h > >> index 129c45f..b35e294 100644 > >> --- a/include/qemu/uuid.h > >> +++ b/include/qemu/uuid.h > >> @@ -34,6 +34,11 @@ typedef struct { > >> }; > >> } QemuUUID; > >> > >> +#define UUID_LE(a, b, c, d0, d1, d2, d3, d4, d5, d6, d7) \ > >> + {{{ (a) & 0xff, ((a) >> 8) & 0xff, ((a) >> 16) & 0xff, ((a) >> 24) & 0xff, \ > >> + (b) & 0xff, ((b) >> 8) & 0xff, (c) & 0xff, ((c) >> 8) & 0xff, \ > >> + (d0), (d1), (d2), (d3), (d4), (d5), (d6), (d7) } } } > > > > since you are adding generalizing macro, take of NVDIMM_UUID_LE which served as model > > do you mean use this generalizing macro to replace NVDIMM_UUID_LE, right? yes, and preferably do that in a separate patch > > > > > > >> #define UUID_FMT "%02hhx%02hhx%02hhx%02hhx-" \ > >> "%02hhx%02hhx-%02hhx%02hhx-" \ > >> "%02hhx%02hhx-" \ > > > > . > > >
On Wed, 8 Jan 2020 19:32:18 +0800 Dongjiu Geng <gengdongjiu@huawei.com> wrote: > This patch builds Hardware Error Source Table(HEST) via fw_cfg blobs. > Now it only supports ARMv8 SEA, a type of Generic Hardware Error > Source version 2(GHESv2) error source. Afterwards, we can extend > the supported types if needed. For the CPER section, currently it > is memory section because kernel mainly wants userspace to handle > the memory errors. > > This patch follows the spec ACPI 6.2 to build the Hardware Error > Source table. For more detailed information, please refer to > document: docs/specs/acpi_hest_ghes.rst > > build_append_ghes_notify() will help to add Hardware Error Notification > to ACPI tables without using packed C structures and avoid endianness > issues as API doesn't need explicit conversion. > > Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> > Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> > Reviewed-by: Michael S. Tsirkin <mst@redhat.com> > Acked-by: Xiang Zheng <zhengxiang9@huawei.com> Hi. I was forwards porting my old series adding CCIX error injection support and came across a place this could 'possibly' be improved. I say possibly because it's really about enabling more flexibility in how this code is reused than actually 'fixing' anything here. If you don't make the change here, I'll just add a precursor patch to my series. Just seems nice to tidy it up at source. The rest of the parts of this series I am using seems to work great. Thanks! Jonathan > --- > hw/acpi/ghes.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++- > hw/arm/virt-acpi-build.c | 2 + > include/hw/acpi/ghes.h | 40 ++++++++++++++++ > 3 files changed, 159 insertions(+), 1 deletion(-) > > diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c > index b7fdbbb..9d37798 100644 > --- a/hw/acpi/ghes.c > +++ b/hw/acpi/ghes.c > @@ -34,9 +34,42 @@ > + ... > +/* Build Generic Hardware Error Source version 2 (GHESv2) */ > +static void build_ghes_v2(GArray *table_data, int source_id, BIOSLinker *linker) This function takes source ID, which uses the enum of all sources registered. However, it doesn't use it to locate the actual physical addresses. Currently the code effectively assumes the value is 0. > +{ > + uint64_t address_offset; > + /* > + * Type: > + * Generic Hardware Error Source version 2(GHESv2 - Type 10) > + */ > + build_append_int_noprefix(table_data, ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2); > + /* Source Id */ > + build_append_int_noprefix(table_data, source_id, 2); > + /* Related Source Id */ > + build_append_int_noprefix(table_data, 0xffff, 2); > + /* Flags */ > + build_append_int_noprefix(table_data, 0, 1); > + /* Enabled */ > + build_append_int_noprefix(table_data, 1, 1); > + > + /* Number of Records To Pre-allocate */ > + build_append_int_noprefix(table_data, 1, 4); > + /* Max Sections Per Record */ > + build_append_int_noprefix(table_data, 1, 4); > + /* Max Raw Data Length */ > + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); > + > + address_offset = table_data->len; > + /* Error Status Address */ > + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, > + 4 /* QWord access */, 0); > + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, > + address_offset + GAS_ADDR_OFFSET, > + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); The offset here would need to be source_id * sizeof(uint64_t) I think > + > + /* > + * Notification Structure > + * Now only enable ARMv8 SEA notification type > + */ > + build_ghes_hw_error_notification(table_data, ACPI_GHES_NOTIFY_SEA); Perhaps a switch for this to allow for other options later. switch (source_id) { case ACPI_HEST_SRC_ID_SEA: ... break; default: //print some error message. } > + > + /* Error Status Block Length */ > + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); > + > + /* > + * Read Ack Register > + * ACPI 6.1: 18.3.2.8 Generic Hardware Error Source > + * version 2 (GHESv2 - Type 10) > + */ > + address_offset = table_data->len; > + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, > + 4 /* QWord access */, 0); > + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, > + address_offset + GAS_ADDR_OFFSET, > + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, > + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t)); Offset of (ACPI_GHES_ERROR_SOURCE_COUNT + source_id) * sizeof(uint64_t) > + > + /* > + * Read Ack Preserve > + * We only provide the first bit in Read Ack Register to OSPM to write > + * while the other bits are preserved. > + */ > + build_append_int_noprefix(table_data, ~0x1ULL, 8); > + /* Read Ack Write */ > + build_append_int_noprefix(table_data, 0x1, 8); > +}
On 2020/2/6 0:43, Jonathan Cameron wrote: > On Wed, 8 Jan 2020 19:32:18 +0800 > Dongjiu Geng <gengdongjiu@huawei.com> wrote: > >> This patch builds Hardware Error Source Table(HEST) via fw_cfg blobs. >> Now it only supports ARMv8 SEA, a type of Generic Hardware Error >> Source version 2(GHESv2) error source. Afterwards, we can extend >> the supported types if needed. For the CPER section, currently it >> is memory section because kernel mainly wants userspace to handle >> the memory errors. >> >> This patch follows the spec ACPI 6.2 to build the Hardware Error >> Source table. For more detailed information, please refer to >> document: docs/specs/acpi_hest_ghes.rst >> >> build_append_ghes_notify() will help to add Hardware Error Notification >> to ACPI tables without using packed C structures and avoid endianness >> issues as API doesn't need explicit conversion. >> >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >> Reviewed-by: Michael S. Tsirkin <mst@redhat.com> >> Acked-by: Xiang Zheng <zhengxiang9@huawei.com> > > Hi. > > I was forwards porting my old series adding CCIX error injection support > and came across a place this could 'possibly' be improved. Jonathan, It is great that you add CCIX error injection support based on this series. thanks for using it. > > I say possibly because it's really about enabling more flexibility > in how this code is reused than actually 'fixing' anything here. > > If you don't make the change here, I'll just add a precursor patch to my > series. Just seems nice to tidy it up at source. sure, I make a change to make your patch work well. > > The rest of the partMake your patch very good work.s of this series I am using seems to work great. > > Thanks! > > Jonathan > >> --- >> hw/acpi/ghes.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++- >> hw/arm/virt-acpi-build.c | 2 + >> include/hw/acpi/ghes.h | 40 ++++++++++++++++ >> 3 files changed, 159 insertions(+), 1 deletion(-) >> >> diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c >> index b7fdbbb..9d37798 100644 >> --- a/hw/acpi/ghes.c >> +++ b/hw/acpi/ghes.c >> @@ -34,9 +34,42 @@ >> + > ... >> +/* Build Generic Hardware Error Source version 2 (GHESv2) */ >> +static void build_ghes_v2(GArray *table_data, int source_id, BIOSLinker *linker) > This function takes source ID, which uses the enum of all sources registered. > However, it doesn't use it to locate the actual physical addresses. > > Currently the code effectively assumes the value is 0. yes, because there is only one source, so the value is 0. > >> +{ >> + uint64_t address_offset; >> + /* >> + * Type: >> + * Generic Hardware Error Source version 2(GHESv2 - Type 10) >> + */ >> + build_append_int_noprefix(table_data, ACPI_GHES_SOURCE_GENERIC_ERROR_V2, 2); >> + /* Source Id */ >> + build_append_int_noprefix(table_data, source_id, 2); >> + /* Related Source Id */ >> + build_append_int_noprefix(table_data, 0xffff, 2); >> + /* Flags */ >> + build_append_int_noprefix(table_data, 0, 1); >> + /* Enabled */ >> + build_append_int_noprefix(table_data, 1, 1); >> + >> + /* Number of Records To Pre-allocate */ >> + build_append_int_noprefix(table_data, 1, 4); >> + /* Max Sections Per Record */ >> + build_append_int_noprefix(table_data, 1, 4); >> + /* Max Raw Data Length */ >> + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); >> + >> + address_offset = table_data->len; >> + /* Error Status Address */ >> + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, >> + 4 /* QWord access */, 0); >> + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, >> + address_offset + GAS_ADDR_OFFSET, >> + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, 0); > > The offset here would need to be source_id * sizeof(uint64_t) I think > >> + >> + /* >> + * Notification Structure >> + * Now only enable ARMv8 SEA notification type >> + */ >> + build_ghes_hw_error_notification(table_data, ACPI_GHES_NOTIFY_SEA); > Perhaps a switch for this to allow for other options later. OK, I will make this change in order to easily support more hardware error source. > > switch (source_id) { > case ACPI_HEST_SRC_ID_SEA: > ... > break; > default: > //print some error message. > > } ok >> + >> + /* Error Status Block Length */ >> + build_append_int_noprefix(table_data, ACPI_GHES_MAX_RAW_DATA_LENGTH, 4); >> + >> + /* >> + * Read Ack Register >> + * ACPI 6.1: 18.3.2.8 Generic Hardware Error Source >> + * version 2 (GHESv2 - Type 10) >> + */ >> + address_offset = table_data->len; >> + build_append_gas(table_data, AML_AS_SYSTEM_MEMORY, 0x40, 0, >> + 4 /* QWord access */, 0); >> + bios_linker_loader_add_pointer(linker, ACPI_BUILD_TABLE_FILE, >> + address_offset + GAS_ADDR_OFFSET, >> + sizeof(uint64_t), ACPI_GHES_ERRORS_FW_CFG_FILE, >> + ACPI_GHES_ERROR_SOURCE_COUNT * sizeof(uint64_t)); > > Offset of (ACPI_GHES_ERROR_SOURCE_COUNT + source_id) * sizeof(uint64_t) yes, It due to I only support one source, the source_id is zero and not use it. In order to easily extend, I will add this change. > >> + >> + /* >> + * Read Ack Preserve >> + * We only provide the first bit in Read Ack Register to OSPM to write >> + * while the other bits are preserved. >> + */ >> + build_append_int_noprefix(table_data, ~0x1ULL, 8); >> + /* Read Ack Write */ >> + build_append_int_noprefix(table_data, 0x1, 8); >> +} > > > > . >
On 2020/1/17 0:44, Peter Maydell wrote: > On Wed, 8 Jan 2020 at 11:33, Dongjiu Geng <gengdongjiu@huawei.com> wrote: >> >> Record the GHEB address via fw_cfg file, when recording >> a error to CPER, it will use this address to find out >> Generic Error Data Entries and write the error. >> >> Make the HEST GHES to a GED device. >> >> Signed-off-by: Dongjiu Geng <gengdongjiu@huawei.com> >> Signed-off-by: Xiang Zheng <zhengxiang9@huawei.com> >> --- >> hw/acpi/generic_event_device.c | 15 ++++++++++++++- >> hw/acpi/ghes.c | 16 ++++++++++++++++ >> hw/arm/virt-acpi-build.c | 13 ++++++++++++- >> include/hw/acpi/generic_event_device.h | 2 ++ >> include/hw/acpi/ghes.h | 6 ++++++ >> 5 files changed, 50 insertions(+), 2 deletions(-) >> >> diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c >> index 9cee90c..9bf37e4 100644 >> --- a/hw/acpi/generic_event_device.c >> +++ b/hw/acpi/generic_event_device.c >> @@ -234,12 +234,25 @@ static const VMStateDescription vmstate_ged_state = { >> } >> }; >> >> +static const VMStateDescription vmstate_ghes_state = { >> + .name = "acpi-ghes-state", >> + .version_id = 1, >> + .minimum_version_id = 1, >> + .fields = (VMStateField[]) { >> + VMSTATE_UINT64(ghes_addr_le, AcpiGhesState), >> + VMSTATE_END_OF_LIST() >> + } >> +}; >> + >> static const VMStateDescription vmstate_acpi_ged = { >> .name = "acpi-ged", >> .version_id = 1, >> .minimum_version_id = 1, >> .fields = (VMStateField[]) { >> - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState), >> + VMSTATE_STRUCT(ged_state, AcpiGedState, 1, >> + vmstate_ged_state, GEDState), >> + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, >> + vmstate_ghes_state, AcpiGhesState), >> VMSTATE_END_OF_LIST(), >> }, >> .subsections = (const VMStateDescription * []) { > > You can't just add fields to an existing VMStateDescription > like this -- it will break migration compatibility. Instead you > need to add a new subsection to this vmstate, with a '.needed' > function which indicates when the subsection should be present. Hi Peter/Igor In order to avoid migration failure, do you think whether below change is Ok to make error table address(AcpiGhesState) to a part of GED device? thanks a lot in advance. --------------------------------------------------------------------------------------------- diff --git a/hw/acpi/generic_event_device.c b/hw/acpi/generic_event_device.c index 021ed2b..264154d 100644 --- a/hw/acpi/generic_event_device.c +++ b/hw/acpi/generic_event_device.c @@ -234,16 +234,34 @@ static const VMStateDescription vmstate_ged_state = { } }; +static bool ghes_needed(void *opaque) +{ + return object_property_get_bool(qdev_get_machine(), "ras", NULL); +} + +static const VMStateDescription vmstate_ghes_state = { + .name = "acpi-ged/ghes", + .version_id = 1, + .minimum_version_id = 1, + .needed = ghes_needed, + .fields = (VMStateField[]) { + VMSTATE_STRUCT(ghes_state, AcpiGedState, 1, vmstate_ghes_state, AcpiGhesState), + VMSTATE_END_OF_LIST() + } +}; + static const VMStateDescription vmstate_acpi_ged = { .name = "acpi-ged", .version_id = 1, .minimum_version_id = 1, .fields = (VMStateField[]) { - VMSTATE_STRUCT(ged_state, AcpiGedState, 1, vmstate_ged_state, GEDState), + VMSTATE_STRUCT(ged_state, AcpiGedState, 1, + vmstate_ged_state, GEDState), VMSTATE_END_OF_LIST(), }, .subsections = (const VMStateDescription * []) { &vmstate_memhp_state, + &vmstate_ghes_state, NULL } }; diff --git a/hw/acpi/ghes.c b/hw/acpi/ghes.c index a67b1de..3bf32ec 100644 --- a/hw/acpi/ghes.c +++ b/hw/acpi/ghes.c @@ -24,6 +24,7 @@ #include "hw/acpi/acpi.h" #include "hw/acpi/ghes.h" #include "hw/acpi/aml-build.h" +#include "hw/acpi/generic_event_device.h" #include "hw/nvram/fw_cfg.h" #include "sysemu/sysemu.h" #include "qemu/error-report.h" @@ -216,3 +217,18 @@ void acpi_build_hest(GArray *table_data, BIOSLinker *linker) build_header(linker, table_data, (void *)(table_data->data + hest_start), "HEST", table_data->len - hest_start, 1, NULL, ""); } + +void acpi_ghes_add_fw_cfg(AcpiGhesState *ags, FWCfgState *s, + GArray *hardware_error) +{ + size_t size = 2 * sizeof(uint64_t) + ACPI_GHES_MAX_RAW_DATA_LENGTH; + size_t request_block_size = ACPI_GHES_ERROR_SOURCE_COUNT * size; + + /* Create a read-only fw_cfg file for GHES */ + fw_cfg_add_file(s, ACPI_GHES_ERRORS_FW_CFG_FILE, hardware_error->data, + request_block_size); + + /* Create a read-write fw_cfg file for Address */ + fw_cfg_add_file_callback(s, ACPI_GHES_DATA_ADDR_FW_CFG_FILE, NULL, NULL, + NULL, &(ags->ghes_addr_le), sizeof(ags->ghes_addr_le), false); +} diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c index 12a9a78..d6e7521 100644 --- a/hw/arm/virt-acpi-build.c +++ b/hw/arm/virt-acpi-build.c @@ -832,6 +832,7 @@ void virt_acpi_build(VirtMachineState *vms, AcpiBuildTables *tables) build_spcr(tables_blob, tables->linker, vms); if (vms->ras) { + assert(vms->acpi_dev); acpi_add_table(table_offsets, tables_blob); build_ghes_error_table(tables->hardware_errors, tables->linker); acpi_build_hest(tables_blob, tables->linker); @@ -924,6 +925,7 @@ void virt_acpi_setup(VirtMachineState *vms) { AcpiBuildTables tables; AcpiBuildState *build_state; + AcpiGedState *acpi_ged_state; if (!vms->fw_cfg) { trace_virt_acpi_setup(); @@ -954,6 +956,14 @@ void virt_acpi_setup(VirtMachineState *vms) fw_cfg_add_file(vms->fw_cfg, ACPI_BUILD_TPMLOG_FILE, tables.tcpalog->data, acpi_data_len(tables.tcpalog)); + if (vms->ras) { + assert(vms->acpi_dev); + acpi_ged_state = ACPI_GED(object_resolve_path_type("", TYPE_ACPI_GED, + NULL)); + acpi_ghes_add_fw_cfg(&acpi_ged_state->ghes_state, + vms->fw_cfg, tables.hardware_errors); + } + build_state->rsdp_mr = acpi_add_rom_blob(virt_acpi_build_update, build_state, tables.rsdp, ACPI_BUILD_RSDP_FILE, 0); diff --git a/include/hw/acpi/generic_event_device.h b/include/hw/acpi/generic_event_device.h index d157eac..037d2b5 100644 --- a/include/hw/acpi/generic_event_device.h +++ b/include/hw/acpi/generic_event_device.h @@ -61,6 +61,7 @@ #include "hw/sysbus.h" #include "hw/acpi/memory_hotplug.h" +#include "hw/acpi/ghes.h" #define ACPI_POWER_BUTTON_DEVICE "PWRB" @@ -95,6 +96,7 @@ typedef struct AcpiGedState { GEDState ged_state; uint32_t ged_event_bitmap; qemu_irq irq; + AcpiGhesState ghes_state; } AcpiGedState; void build_ged_aml(Aml *table, const char* name, HotplugHandler *hotplug_dev, diff --git a/include/hw/acpi/ghes.h b/include/hw/acpi/ghes.h index 18debd8..a3420fc 100644 --- a/include/hw/acpi/ghes.h +++ b/include/hw/acpi/ghes.h @@ -62,6 +62,12 @@ enum { ACPI_HEST_SRC_ID_RESERVED, }; +typedef struct AcpiGhesState { + uint64_t ghes_addr_le; +} AcpiGhesState; + void build_ghes_error_table(GArray *hardware_errors, BIOSLinker *linker); void acpi_build_hest(GArray *table_data, BIOSLinker *linker); +void acpi_ghes_add_fw_cfg(AcpiGhesState *vms, FWCfgState *s, + GArray *hardware_errors); #endif > > thanks > -- PMM > . >