From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756107AbdDFSzz (ORCPT ); Thu, 6 Apr 2017 14:55:55 -0400 Received: from mx1.redhat.com ([209.132.183.28]:49310 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753472AbdDFSzp (ORCPT ); Thu, 6 Apr 2017 14:55:45 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com D368FC04B92B Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx07.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=lersek@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com D368FC04B92B Subject: Re: [PATCH] kvm: pass the virtual SEI syndrome to guest OS To: gengdongjiu , Achin Gupta References: <76795e20-2f20-1e54-cfa5-7444f28b18ee@huawei.com> <20170321113428.GC15920@cbox> <58D17AF0.2010802@arm.com> <20170321193933.GB31111@cbox> <58DA3F68.6090901@arm.com> <20170328112328.GA31156@cbox> <20170328115413.GJ23682@e104320-lin> <58DA67BA.8070404@arm.com> <5b7352f4-4965-3ed5-3879-db871797be47@huawei.com> <20170329103658.GQ23682@e104320-lin> <2a427164-9b37-6711-3a56-906634ba7f12@redhat.com> <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> Cc: ard.biesheuvel@linaro.org, edk2-devel@ml01.01.org, qemu-devel@nongnu.org, zhaoshenglong@huawei.com, James Morse , Christoffer Dall , xiexiuqi@huawei.com, Marc Zyngier , catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@linaro.org, rkrcmar@redhat.com, suzuki.poulose@arm.com, andre.przywara@arm.com, mark.rutland@arm.com, vladimir.murzin@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, wangxiongfeng2@huawei.com, wuquanming@huawei.com, huangshaoyu@huawei.com, Leif.Lindholm@linaro.com, nd@arm.com, Michael Tsirkin , Igor Mammedov From: Laszlo Ersek Message-ID: <6ac1597a-2ed5-36b2-848d-5fd048b16d66@redhat.com> Date: Thu, 6 Apr 2017 20:55:34 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]); Thu, 06 Apr 2017 18:55:44 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 04/06/17 14:35, gengdongjiu wrote: > Dear, Laszlo > Thanks for your detailed explanation. > > On 2017/3/29 19:58, Laszlo Ersek wrote: >> (This ought to be one of the longest address lists I've ever seen :) >> Thanks for the CC. I'm glad Shannon is already on the CC list. For good >> measure, I'm adding MST and Igor.) >> >> On 03/29/17 12:36, Achin Gupta wrote: >>> Hi gengdongjiu, >>> >>> On Wed, Mar 29, 2017 at 05:36:37PM +0800, gengdongjiu wrote: >>>> >>>> Hi Laszlo/Biesheuvel/Qemu developer, >>>> >>>> Now I encounter a issue and want to consult with you in ARM64 platform， as described below: >>>> >>>> when guest OS happen synchronous or asynchronous abort, kvm needs >>>> to send the error address to Qemu or UEFI through sigbus to >>>> dynamically generate APEI table. from my investigation, there are >>>> two ways: >>>> >>>> (1) Qemu get the error address, and generate the APEI table, then >>>> notify UEFI to know this generation, then inject abort error to >>>> guest OS, guest OS read the APEI table. >>>> (2) Qemu get the error address, and let UEFI to generate the APEI >>>> table, then inject abort error to guest OS, guest OS read the APEI >>>> table. >>> >>> Just being pedantic! I don't think we are talking about creating the APEI table >>> dynamically here. The issue is: Once KVM has received an error that is destined >>> for a guest it will raise a SIGBUS to Qemu. Now before Qemu can inject the error >>> into the guest OS, a CPER (Common Platform Error Record) has to be generated >>> corresponding to the error source (GHES corresponding to memory subsystem, >>> processor etc) to allow the guest OS to do anything meaningful with the >>> error. So who should create the CPER is the question. >>> >>> At the EL3/EL2 interface (Secure Firmware and OS/Hypervisor), an error arrives >>> at EL3 and secure firmware (at EL3 or a lower secure exception level) is >>> responsible for creating the CPER. ARM is experimenting with using a Standalone >>> MM EDK2 image in the secure world to do the CPER creation. This will avoid >>> adding the same code in ARM TF in EL3 (better for security). The error will then >>> be injected into the OS/Hypervisor (through SEA/SEI/SDEI) through ARM Trusted >>> Firmware. >>> >>> Qemu is essentially fulfilling the role of secure firmware at the EL2/EL1 >>> interface (as discussed with Christoffer below). So it should generate the CPER >>> before injecting the error. >>> >>> This is corresponds to (1) above apart from notifying UEFI (I am assuming you >>> mean guest UEFI). At this time, the guest OS already knows where to pick up the >>> CPER from through the HEST. Qemu has to create the CPER and populate its address >>> at the address exported in the HEST. Guest UEFI should not be involved in this >>> flow. Its job was to create the HEST at boot and that has been done by this >>> stage. >>> >>> Qemu folk will be able to add but it looks like support for CPER generation will >>> need to be added to Qemu. We need to resolve this. >>> >>> Do shout if I am missing anything above. >> >> After reading this email, the use case looks *very* similar to what >> we've just done with VMGENID for QEMU 2.9. >> >> We have a facility between QEMU and the guest firmware, called "ACPI >> linker/loader", with which QEMU instructs the firmware to >> >> - allocate and download blobs into guest RAM (AcpiNVS type memory) -- >> ALLOCATE command, >> >> - relocate pointers in those blobs, to fields in other (or the same) >> blobs -- ADD_POINTER command, >> >> - set ACPI table checksums -- ADD_CHECKSUM command, >> >> - and send GPAs of fields within such blobs back to QEMU -- >> WRITE_POINTER command. >> >> This is how I imagine we can map the facility to the current use case >> (note that this is the first time I read about HEST / GHES / CPER): >> >> etc/acpi/tables etc/hardware_errors >> ================ ========================================== >> +-----------+ >> +--------------+ | address | +-> +--------------+ >> | HEST + | registers | | | Error Status | >> + +------------+ | +---------+ | | Data Block 1 | >> | | GHES | --> | | address | --------+ | +------------+ >> | | GHES | --> | | address | ------+ | | CPER | >> | | GHES | --> | | address | ----+ | | | CPER | >> | | GHES | --> | | address | -+ | | | | CPER | >> +-+------------+ +-+---------+ | | | +-+------------+ >> | | | >> | | +---> +--------------+ >> | | | Error Status | >> | | | Data Block 2 | >> | | | +------------+ >> | | | | CPER | >> | | | | CPER | >> | | +-+------------+ >> | | >> | +-----> +--------------+ >> | | Error Status | >> | | Data Block 3 | >> | | +------------+ >> | | | CPER | >> | +-+------------+ >> | >> +--------> +--------------+ >> | Error Status | >> | Data Block 4 | >> | +------------+ >> | | CPER | >> | | CPER | >> | | CPER | >> +-+------------+ >> >> (1) QEMU generates the HEST ACPI table. This table goes in the current >> "etc/acpi/tables" fw_cfg blob. Given N error sources, there will be N >> GHES objects in the HEST. >> >> (2) We introduce a new fw_cfg blob called "etc/hardware_errors". QEMU >> also populates this blob. >> >> (2a) Given N error sources, the (unnamed) table of address registers >> will contain N address registers. >> >> (2b) Given N error sources, the "etc/hardwre_errors" fw_cfg blob will >> also contain N Error Status Data Blocks. >> >> I don't know about the sizing (number of CPERs) each Error Status Data >> Block has to contain, but I understand it is all pre-allocated as far as >> the OS is concerned, which matches our capabilities well. > here I have a question. as you comment: " 'etc/hardwre_errors' fw_cfg blob will also contain N Error Status Data Blocks", > Because the CPER numbers is not fixed, how to assign each "Error Status Data Block" size using one "etc/hardwre_errors" fw_cfg blob. > when use one etc/hardwre_errors, will the N Error Status Data Block use one continuous buffer? as shown below. if so, maybe it not convenient for each data block size extension. > I see the bios_linker_loader_alloc will allocate one continuous buffer for a blob(such as VMGENID_GUID_FW_CFG_FILE) > > /* Allocate guest memory for the Data fw_cfg blob */ > bios_linker_loader_alloc(linker, VMGENID_GUID_FW_CFG_FILE, guid, 4096, > false /* page boundary, high memory */); > > > > -> +--------------+ > | HEST + | registers | | Error Status | > + +------------+ | +---------+ | Data Block | > | | GHES | --> | | address | --------+-->| +------------+ > | | GHES | --> | | address | ------+ | | CPER | > | | GHES | --> | | address | ----+ | | | CPER | > | | GHES | --> | | address | -+ | | | | CPER | > +-+------------+ +-+---------+ | | +---> +--------------+ > | | | | CPER | > | | | | CPER | > | +-----> +--------------+ > | | | CPER | > +--------> +--------------+ > | | CPER | > | | CPER | > | | CPER | > +-+------------+ > > > > so how about we use separate etc/hardwre_errorsN for each Error Status status Block? then > > etc/hardwre_errors0 > etc/hardwre_errors1 > ................... > etc/hardwre_errors10 > (the max N is 10) > > > the N can be one of below values, according to ACPI spec "Table 18-345 Hardware Error Notification Structure" > 0 – Polled > 1 – External Interrupt > 2 – Local Interrupt > 3 – SCI > 4 – NMI > 5 - CMCI > 6 - MCE > 7 - GPIO-Signal > 8 - ARMv8 SEA > 9 - ARMv8 SEI > 10 - External Interrupt - GSIV I'm unsure if, by "not fixed", you are saying the number of CPER entries that fits in Error Status Data Block N is not *uniform* across 0 <= N <= 10 [1] or the number of CPER entries that fits in Error Status Data Block N is not *known* in advance, for all of 0 <= N <= 10 [2] Which one is your point? If [1], that's no problem; you can simply sum the individual error status data block sizes in advance, and allocate "etc/hardware_errors" accordingly, using the total size. (Allocating one shared fw_cfg blob for all status data blocks is more memory efficient, as each ALLOCATE command will allocate whole pages (rounded up from the actual blob size).) If your point is [2], then splitting the error status data blocks to separate fw_cfg blobs makes no difference: regardless of whether we try to place all the error status data blocks in a single fw_cfg blob, or in separate fw_cfg blobs, the individual data block cannot be resized at OS runtime, so there's no way to make it work. Thanks, Laszlo > > > > >> >> (3) QEMU generates the ACPI linker/loader script for the firmware, as >> always. >> >> (3a) The HEST table is part of "etc/acpi/tables", which the firmware >> already allocates memory for, and downloads (because QEMU already >> generates an ALLOCATE linker/loader command for it already). >> >> (3b) QEMU will have to create another ALLOCATE command for the >> "etc/hardware_errors" blob. The firmware allocates memory for this blob, >> and downloads it. >> >> (4) QEMU generates, in the ACPI linker/loader script for the firwmare, N >> ADD_POINTER commands, which point the GHES."Error Status >> Address" fields in the HEST table, to the corresponding address >> registers in the downloaded "etc/hardware_errors" blob. >> >> (5) QEMU generates an ADD_CHECKSUM command for the firmware, so that the >> HEST table is correctly checksummed after executing the N ADD_POINTER >> commands from (4). >> >> (6) QEMU generates N ADD_POINTER commands for the firmware, pointing the >> address registers (located in guest memory, in the downloaded >> "etc/hardware_errors" blob) to the respective Error Status Data Blocks. >> >> (7) (This is the trick.) For this step, we need a third, write-only >> fw_cfg blob, called "etc/hardware_errors_addr". Through that blob, the >> firmware can send back the guest-side allocation addresses to QEMU. >> >> Namely, the "etc/hardware_errors_addr" blob contains N 8-byte entries. >> QEMU generates N WRITE_POINTER commands for the firmware. >> >> For error source K (0 <= K < N), QEMU instructs the firmware to >> calculate the guest address of Error Status Data Block K, from the >> QEMU-dictated offset within "etc/hardware_errors", and from the >> guest-determined allocation base address for "etc/hardware_errors". The >> firmware then writes the calculated address back to fw_cfg file >> "etc/hardware_errors_addr", at offset K*8, according to the >> WRITE_POINTER command. >> >> This way QEMU will know the GPA of each Error Status Data Block. >> >> (In fact this can be simplified to a single WRITE_POINTER command: the >> address of the "address register table" can be sent back to QEMU as >> well, which already contains all Error Status Data Block addresses.) >> >> (8) When QEMU gets SIGBUS from the kernel -- I hope that's going to come >> through a signalfd -- QEMU can format the CPER right into guest memory, >> and then inject whatever interrupt (or assert whatever GPIO line) is >> necessary for notifying the guest. >> >> (9) This notification (in virtual hardware) can either be handled by the >> guest kernel stand-alone, or else the guest kernel can invoke an ACPI >> event handler method with it (which would be in the DSDT or one of the >> SSDTs, also generated by QEMU). The ACPI event handler method could >> invoke the specific guest kernel driver for errror handling via a >> Notify() operation. >> >> I'm attracted to the above design because: >> - it would leave the firmware alone after OS boot, and >> - it would leave the firmware blissfully ignorant about HEST, GHES, >> CPER, and the like. (That's why QEMU's ACPI linker/loader was invented >> in the first place.) >> >> Thanks >> Laszlo >> >>>> Do you think which modules generates the APEI table is better? UEFI or Qemu? >>>> >>>> >>>> >>>> >>>> On 2017/3/28 21:40, James Morse wrote: >>>>> Hi gengdongjiu, >>>>> >>>>> On 28/03/17 13:16, gengdongjiu wrote: >>>>>> On 2017/3/28 19:54, Achin Gupta wrote: >>>>>>> On Tue, Mar 28, 2017 at 01:23:28PM +0200, Christoffer Dall wrote: >>>>>>>> On Tue, Mar 28, 2017 at 11:48:08AM +0100, James Morse wrote: >>>>>>>>> On the host, part of UEFI is involved to generate the CPER records. >>>>>>>>> In a guest?, I don't know. >>>>>>>>> Qemu could generate the records, or drive some other component to do it. >>>>>>>> >>>>>>>> I think I am beginning to understand this a bit. Since the guet UEFI >>>>>>>> instance is specifically built for the machine it runs on, QEMU's virt >>>>>>>> machine in this case, they could simply agree (by some contract) to >>>>>>>> place the records at some specific location in memory, and if the guest >>>>>>>> kernel asks its guest UEFI for that location, things should just work by >>>>>>>> having logic in QEMU to process error reports and populate guest memory. >>>>>>>> >>>>>>>> Is this how others see the world too? >>>>>>> >>>>>>> I think so! >>>>>>> >>>>>>> AFAIU, the memory where CPERs will reside should be specified in a GHES entry in >>>>>>> the HEST. Is this not the case with a guest kernel i.e. the guest UEFI creates a >>>>>>> HEST for the guest Kernel? >>>>>>> >>>>>>> If so, then the question is how the guest UEFI finds out where QEMU (acting as >>>>>>> EL3 firmware) will populate the CPERs. This could either be a contract between >>>>>>> the two or a guest DXE driver uses the MM_COMMUNICATE call (see [1]) to ask QEMU >>>>>>> where the memory is. >>>>>> >>>>>> whether invoke the guest UEFI will be complex? not see the advantage. it seems x86 Qemu >>>>>> directly generate the ACPI table, but I am not sure, we are checking the qemu >>>>> logical. >>>>>> let Qemu generate CPER record may be clear. >>>>> >>>>> At boot UEFI in the guest will need to make sure the areas of memory that may be >>>>> used for CPER records are reserved. Whether UEFI or Qemu decides where these are >>>>> needs deciding, (but probably not here)... >>>>> >>>>> At runtime, when an error has occurred, I agree it would be simpler (fewer >>>>> components involved) if Qemu generates the CPER records. But if UEFI made the >>>>> memory choice above they need to interact and it gets complicated again. The >>>>> CPER records are defined in the UEFI spec, so I would expect UEFI to contain >>>>> code to generate/parse them. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> James >>>>> >>>>> >>>>> . >>>>> >>>> >> >> >> . >> > From mboxrd@z Thu Jan 1 00:00:00 1970 From: Laszlo Ersek Subject: Re: [edk2] [PATCH] kvm: pass the virtual SEI syndrome to guest OS Date: Thu, 6 Apr 2017 20:55:34 +0200 Message-ID: <6ac1597a-2ed5-36b2-848d-5fd048b16d66@redhat.com> References: <76795e20-2f20-1e54-cfa5-7444f28b18ee@huawei.com> <20170321113428.GC15920@cbox> <58D17AF0.2010802@arm.com> <20170321193933.GB31111@cbox> <58DA3F68.6090901@arm.com> <20170328112328.GA31156@cbox> <20170328115413.GJ23682@e104320-lin> <58DA67BA.8070404@arm.com> <5b7352f4-4965-3ed5-3879-db871797be47@huawei.com> <20170329103658.GQ23682@e104320-lin> <2a427164-9b37-6711-3a56-906634ba7f12@redhat.com> <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: Michael Tsirkin , kvm@vger.kernel.org, rkrcmar@redhat.com, catalin.marinas@arm.com, will.deacon@arm.com, qemu-devel@nongnu.org, wuquanming@huawei.com, wangxiongfeng2@huawei.com, Christoffer Dall , suzuki.poulose@arm.com, kvmarm@lists.cs.columbia.edu, Leif.Lindholm@linaro.com, huangshaoyu@huawei.com, vladimir.murzin@arm.com, xiexiuqi@huawei.com, Marc Zyngier , andre.przywara@arm.com, edk2-devel@lists.01.org, nd@arm.com, linux-arm-kernel@lists.infradead.org, ard.biesheuvel@linaro.org, linux-kernel@vger.kernel.org, James Morse , christoffer.dall@linaro.org To: gengdongjiu , Achin Gupta Return-path: In-Reply-To: <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: edk2-devel-bounces@lists.01.org Sender: "edk2-devel" List-Id: kvm.vger.kernel.org T24gMDQvMDYvMTcgMTQ6MzUsIGdlbmdkb25naml1IHdyb3RlOgo+IERlYXIsIExhc3psbwo+ICAg IFRoYW5rcyBmb3IgeW91ciBkZXRhaWxlZCBleHBsYW5hdGlvbi4KPiAKPiBPbiAyMDE3LzMvMjkg MTk6NTgsIExhc3psbyBFcnNlayB3cm90ZToKPj4gKFRoaXMgb3VnaHQgdG8gYmUgb25lIG9mIHRo ZSBsb25nZXN0IGFkZHJlc3MgbGlzdHMgSSd2ZSBldmVyIHNlZW4gOikKPj4gVGhhbmtzIGZvciB0 aGUgQ0MuIEknbSBnbGFkIFNoYW5ub24gaXMgYWxyZWFkeSBvbiB0aGUgQ0MgbGlzdC4gRm9yIGdv b2QKPj4gbWVhc3VyZSwgSSdtIGFkZGluZyBNU1QgYW5kIElnb3IuKQo+Pgo+PiBPbiAwMy8yOS8x NyAxMjozNiwgQWNoaW4gR3VwdGEgd3JvdGU6Cj4+PiBIaSBnZW5nZG9uZ2ppdSwKPj4+Cj4+PiBP biBXZWQsIE1hciAyOSwgMjAxNyBhdCAwNTozNjozN1BNICswODAwLCBnZW5nZG9uZ2ppdSB3cm90 ZToKPj4+Pgo+Pj4+IEhpIExhc3psby9CaWVzaGV1dmVsL1FlbXUgZGV2ZWxvcGVyLAo+Pj4+Cj4+ Pj4gICAgTm93IEkgZW5jb3VudGVyIGEgaXNzdWUgYW5kIHdhbnQgdG8gY29uc3VsdCB3aXRoIHlv dSBpbiBBUk02NCBwbGF0Zm9ybe+8jCBhcyBkZXNjcmliZWQgYmVsb3c6Cj4+Pj4KPj4+PiB3aGVu IGd1ZXN0IE9TIGhhcHBlbiBzeW5jaHJvbm91cyBvciBhc3luY2hyb25vdXMgYWJvcnQsIGt2bSBu ZWVkcwo+Pj4+IHRvIHNlbmQgdGhlIGVycm9yIGFkZHJlc3MgdG8gUWVtdSBvciBVRUZJIHRocm91 Z2ggc2lnYnVzIHRvCj4+Pj4gZHluYW1pY2FsbHkgZ2VuZXJhdGUgQVBFSSB0YWJsZS4gZnJvbSBt eSBpbnZlc3RpZ2F0aW9uLCB0aGVyZSBhcmUKPj4+PiB0d28gd2F5czoKPj4+Pgo+Pj4+ICgxKSBR ZW11IGdldCB0aGUgZXJyb3IgYWRkcmVzcywgYW5kIGdlbmVyYXRlIHRoZSBBUEVJIHRhYmxlLCB0 aGVuCj4+Pj4gbm90aWZ5IFVFRkkgdG8ga25vdyB0aGlzIGdlbmVyYXRpb24sIHRoZW4gaW5qZWN0 IGFib3J0IGVycm9yIHRvCj4+Pj4gZ3Vlc3QgT1MsIGd1ZXN0IE9TIHJlYWQgdGhlIEFQRUkgdGFi bGUuCj4+Pj4gKDIpIFFlbXUgZ2V0IHRoZSBlcnJvciBhZGRyZXNzLCBhbmQgbGV0IFVFRkkgdG8g Z2VuZXJhdGUgdGhlIEFQRUkKPj4+PiB0YWJsZSwgdGhlbiBpbmplY3QgYWJvcnQgZXJyb3IgdG8g Z3Vlc3QgT1MsIGd1ZXN0IE9TIHJlYWQgdGhlIEFQRUkKPj4+PiB0YWJsZS4KPj4+Cj4+PiBKdXN0 IGJlaW5nIHBlZGFudGljISBJIGRvbid0IHRoaW5rIHdlIGFyZSB0YWxraW5nIGFib3V0IGNyZWF0 aW5nIHRoZSBBUEVJIHRhYmxlCj4+PiBkeW5hbWljYWxseSBoZXJlLiBUaGUgaXNzdWUgaXM6IE9u Y2UgS1ZNIGhhcyByZWNlaXZlZCBhbiBlcnJvciB0aGF0IGlzIGRlc3RpbmVkCj4+PiBmb3IgYSBn dWVzdCBpdCB3aWxsIHJhaXNlIGEgU0lHQlVTIHRvIFFlbXUuIE5vdyBiZWZvcmUgUWVtdSBjYW4g aW5qZWN0IHRoZSBlcnJvcgo+Pj4gaW50byB0aGUgZ3Vlc3QgT1MsIGEgQ1BFUiAoQ29tbW9uIFBs YXRmb3JtIEVycm9yIFJlY29yZCkgaGFzIHRvIGJlIGdlbmVyYXRlZAo+Pj4gY29ycmVzcG9uZGlu ZyB0byB0aGUgZXJyb3Igc291cmNlIChHSEVTIGNvcnJlc3BvbmRpbmcgdG8gbWVtb3J5IHN1YnN5 c3RlbSwKPj4+IHByb2Nlc3NvciBldGMpIHRvIGFsbG93IHRoZSBndWVzdCBPUyB0byBkbyBhbnl0 aGluZyBtZWFuaW5nZnVsIHdpdGggdGhlCj4+PiBlcnJvci4gU28gd2hvIHNob3VsZCBjcmVhdGUg dGhlIENQRVIgaXMgdGhlIHF1ZXN0aW9uLgo+Pj4KPj4+IEF0IHRoZSBFTDMvRUwyIGludGVyZmFj ZSAoU2VjdXJlIEZpcm13YXJlIGFuZCBPUy9IeXBlcnZpc29yKSwgYW4gZXJyb3IgYXJyaXZlcwo+ Pj4gYXQgRUwzIGFuZCBzZWN1cmUgZmlybXdhcmUgKGF0IEVMMyBvciBhIGxvd2VyIHNlY3VyZSBl eGNlcHRpb24gbGV2ZWwpIGlzCj4+PiByZXNwb25zaWJsZSBmb3IgY3JlYXRpbmcgdGhlIENQRVIu IEFSTSBpcyBleHBlcmltZW50aW5nIHdpdGggdXNpbmcgYSBTdGFuZGFsb25lCj4+PiBNTSBFREsy IGltYWdlIGluIHRoZSBzZWN1cmUgd29ybGQgdG8gZG8gdGhlIENQRVIgY3JlYXRpb24uIFRoaXMg d2lsbCBhdm9pZAo+Pj4gYWRkaW5nIHRoZSBzYW1lIGNvZGUgaW4gQVJNIFRGIGluIEVMMyAoYmV0 dGVyIGZvciBzZWN1cml0eSkuIFRoZSBlcnJvciB3aWxsIHRoZW4KPj4+IGJlIGluamVjdGVkIGlu dG8gdGhlIE9TL0h5cGVydmlzb3IgKHRocm91Z2ggU0VBL1NFSS9TREVJKSB0aHJvdWdoIEFSTSBU cnVzdGVkCj4+PiBGaXJtd2FyZS4KPj4+Cj4+PiBRZW11IGlzIGVzc2VudGlhbGx5IGZ1bGZpbGxp bmcgdGhlIHJvbGUgb2Ygc2VjdXJlIGZpcm13YXJlIGF0IHRoZSBFTDIvRUwxCj4+PiBpbnRlcmZh Y2UgKGFzIGRpc2N1c3NlZCB3aXRoIENocmlzdG9mZmVyIGJlbG93KS4gU28gaXQgc2hvdWxkIGdl bmVyYXRlIHRoZSBDUEVSCj4+PiBiZWZvcmUgaW5qZWN0aW5nIHRoZSBlcnJvci4KPj4+Cj4+PiBU aGlzIGlzIGNvcnJlc3BvbmRzIHRvICgxKSBhYm92ZSBhcGFydCBmcm9tIG5vdGlmeWluZyBVRUZJ IChJIGFtIGFzc3VtaW5nIHlvdQo+Pj4gbWVhbiBndWVzdCBVRUZJKS4gQXQgdGhpcyB0aW1lLCB0 aGUgZ3Vlc3QgT1MgYWxyZWFkeSBrbm93cyB3aGVyZSB0byBwaWNrIHVwIHRoZQo+Pj4gQ1BFUiBm cm9tIHRocm91Z2ggdGhlIEhFU1QuIFFlbXUgaGFzIHRvIGNyZWF0ZSB0aGUgQ1BFUiBhbmQgcG9w dWxhdGUgaXRzIGFkZHJlc3MKPj4+IGF0IHRoZSBhZGRyZXNzIGV4cG9ydGVkIGluIHRoZSBIRVNU LiBHdWVzdCBVRUZJIHNob3VsZCBub3QgYmUgaW52b2x2ZWQgaW4gdGhpcwo+Pj4gZmxvdy4gSXRz IGpvYiB3YXMgdG8gY3JlYXRlIHRoZSBIRVNUIGF0IGJvb3QgYW5kIHRoYXQgaGFzIGJlZW4gZG9u ZSBieSB0aGlzCj4+PiBzdGFnZS4KPj4+Cj4+PiBRZW11IGZvbGsgd2lsbCBiZSBhYmxlIHRvIGFk ZCBidXQgaXQgbG9va3MgbGlrZSBzdXBwb3J0IGZvciBDUEVSIGdlbmVyYXRpb24gd2lsbAo+Pj4g bmVlZCB0byBiZSBhZGRlZCB0byBRZW11LiBXZSBuZWVkIHRvIHJlc29sdmUgdGhpcy4KPj4+Cj4+ PiBEbyBzaG91dCBpZiBJIGFtIG1pc3NpbmcgYW55dGhpbmcgYWJvdmUuCj4+Cj4+IEFmdGVyIHJl YWRpbmcgdGhpcyBlbWFpbCwgdGhlIHVzZSBjYXNlIGxvb2tzICp2ZXJ5KiBzaW1pbGFyIHRvIHdo YXQKPj4gd2UndmUganVzdCBkb25lIHdpdGggVk1HRU5JRCBmb3IgUUVNVSAyLjkuCj4+Cj4+IFdl IGhhdmUgYSBmYWNpbGl0eSBiZXR3ZWVuIFFFTVUgYW5kIHRoZSBndWVzdCBmaXJtd2FyZSwgY2Fs bGVkICJBQ1BJCj4+IGxpbmtlci9sb2FkZXIiLCB3aXRoIHdoaWNoIFFFTVUgaW5zdHJ1Y3RzIHRo ZSBmaXJtd2FyZSB0bwo+Pgo+PiAtIGFsbG9jYXRlIGFuZCBkb3dubG9hZCBibG9icyBpbnRvIGd1 ZXN0IFJBTSAoQWNwaU5WUyB0eXBlIG1lbW9yeSkgLS0KPj4gQUxMT0NBVEUgY29tbWFuZCwKPj4K Pj4gLSByZWxvY2F0ZSBwb2ludGVycyBpbiB0aG9zZSBibG9icywgdG8gZmllbGRzIGluIG90aGVy IChvciB0aGUgc2FtZSkKPj4gYmxvYnMgLS0gQUREX1BPSU5URVIgY29tbWFuZCwKPj4KPj4gLSBz ZXQgQUNQSSB0YWJsZSBjaGVja3N1bXMgLS0gQUREX0NIRUNLU1VNIGNvbW1hbmQsCj4+Cj4+IC0g YW5kIHNlbmQgR1BBcyBvZiBmaWVsZHMgd2l0aGluIHN1Y2ggYmxvYnMgYmFjayB0byBRRU1VIC0t Cj4+IFdSSVRFX1BPSU5URVIgY29tbWFuZC4KPj4KPj4gVGhpcyBpcyBob3cgSSBpbWFnaW5lIHdl IGNhbiBtYXAgdGhlIGZhY2lsaXR5IHRvIHRoZSBjdXJyZW50IHVzZSBjYXNlCj4+IChub3RlIHRo YXQgdGhpcyBpcyB0aGUgZmlyc3QgdGltZSBJIHJlYWQgYWJvdXQgSEVTVCAvIEdIRVMgLyBDUEVS KToKPj4KPj4gICAgIGV0Yy9hY3BpL3RhYmxlcyAgICAgICAgICAgICAgICAgZXRjL2hhcmR3YXJl X2Vycm9ycwo+PiAgICAgPT09PT09PT09PT09PT09PSAgICAgPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgICArLS0tLS0t LS0tLS0rCj4+ICAgICArLS0tLS0tLS0tLS0tLS0rICAgICB8IGFkZHJlc3MgICB8ICAgICAgICAg Ky0+ICstLS0tLS0tLS0tLS0tLSsKPj4gICAgIHwgICAgSEVTVCAgICAgICsgICAgIHwgcmVnaXN0 ZXJzIHwgICAgICAgICB8ICAgfCBFcnJvciBTdGF0dXMgfAo+PiAgICAgKyArLS0tLS0tLS0tLS0t KyAgICAgfCArLS0tLS0tLS0tKyAgICAgICAgIHwgICB8IERhdGEgQmxvY2sgMSB8Cj4+ICAgICB8 IHwgR0hFUyAgICAgICB8IC0tPiB8IHwgYWRkcmVzcyB8IC0tLS0tLS0tKyAgIHwgKy0tLS0tLS0t LS0tLSsKPj4gICAgIHwgfCBHSEVTICAgICAgIHwgLS0+IHwgfCBhZGRyZXNzIHwgLS0tLS0tKyAg ICAgfCB8ICBDUEVSICAgICAgfAo+PiAgICAgfCB8IEdIRVMgICAgICAgfCAtLT4gfCB8IGFkZHJl c3MgfCAtLS0tKyB8ICAgICB8IHwgIENQRVIgICAgICB8Cj4+ICAgICB8IHwgR0hFUyAgICAgICB8 IC0tPiB8IHwgYWRkcmVzcyB8IC0rICB8IHwgICAgIHwgfCAgQ1BFUiAgICAgIHwKPj4gICAgICst Ky0tLS0tLS0tLS0tLSsgICAgICstKy0tLS0tLS0tLSsgIHwgIHwgfCAgICAgKy0rLS0tLS0tLS0t LS0tKwo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgfCB8Cj4+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICB8ICstLS0+ICstLS0t LS0tLS0tLS0tLSsKPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwg IHwgICAgICAgfCBFcnJvciBTdGF0dXMgfAo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgfCAgfCAgICAgICB8IERhdGEgQmxvY2sgMiB8Cj4+ICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICB8ICB8ICAgICAgIHwgKy0tLS0tLS0tLS0tLSsKPj4g ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgIHwgICAgICAgfCB8ICBD UEVSICAgICAgfAo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAg fCAgICAgICB8IHwgIENQRVIgICAgICB8Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICB8ICB8ICAgICAgICstKy0tLS0tLS0tLS0tLSsKPj4gICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgIHwgIHwKPj4gICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHwgICstLS0tLT4gKy0tLS0tLS0tLS0tLS0tKwo+PiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgICAgICAgICB8IEVycm9yIFN0YXR1cyB8 Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAgICAgICAgIHwg RGF0YSBCbG9jayAzIHwKPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg IHwgICAgICAgICAgfCArLS0tLS0tLS0tLS0tKwo+PiAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgfCAgICAgICAgICB8IHwgIENQRVIgICAgICB8Cj4+ICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAgICAgICAgICstKy0tLS0tLS0tLS0tLSsK Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwKPj4gICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICstLS0tLS0tLT4gKy0tLS0tLS0tLS0tLS0t Kwo+PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8 IEVycm9yIFN0YXR1cyB8Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgIHwgRGF0YSBCbG9jayA0IHwKPj4gICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgfCArLS0tLS0tLS0tLS0tKwo+PiAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8IHwgIENQRVIgICAgICB8 Cj4+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwg fCAgQ1BFUiAgICAgIHwKPj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgfCB8ICBDUEVSICAgICAgfAo+PiAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICArLSstLS0tLS0tLS0tLS0rCj4+Cj4+ICgxKSBRRU1V IGdlbmVyYXRlcyB0aGUgSEVTVCBBQ1BJIHRhYmxlLiBUaGlzIHRhYmxlIGdvZXMgaW4gdGhlIGN1 cnJlbnQKPj4gImV0Yy9hY3BpL3RhYmxlcyIgZndfY2ZnIGJsb2IuIEdpdmVuIE4gZXJyb3Igc291 cmNlcywgdGhlcmUgd2lsbCBiZSBOCj4+IEdIRVMgb2JqZWN0cyBpbiB0aGUgSEVTVC4KPj4KPj4g KDIpIFdlIGludHJvZHVjZSBhIG5ldyBmd19jZmcgYmxvYiBjYWxsZWQgImV0Yy9oYXJkd2FyZV9l cnJvcnMiLiBRRU1VCj4+IGFsc28gcG9wdWxhdGVzIHRoaXMgYmxvYi4KPj4KPj4gKDJhKSBHaXZl biBOIGVycm9yIHNvdXJjZXMsIHRoZSAodW5uYW1lZCkgdGFibGUgb2YgYWRkcmVzcyByZWdpc3Rl cnMKPj4gd2lsbCBjb250YWluIE4gYWRkcmVzcyByZWdpc3RlcnMuCj4+Cj4+ICgyYikgR2l2ZW4g TiBlcnJvciBzb3VyY2VzLCB0aGUgImV0Yy9oYXJkd3JlX2Vycm9ycyIgZndfY2ZnIGJsb2Igd2ls bAo+PiBhbHNvIGNvbnRhaW4gTiBFcnJvciBTdGF0dXMgRGF0YSBCbG9ja3MuCj4+Cj4+IEkgZG9u J3Qga25vdyBhYm91dCB0aGUgc2l6aW5nIChudW1iZXIgb2YgQ1BFUnMpIGVhY2ggRXJyb3IgU3Rh dHVzIERhdGEKPj4gQmxvY2sgaGFzIHRvIGNvbnRhaW4sIGJ1dCBJIHVuZGVyc3RhbmQgaXQgaXMg YWxsIHByZS1hbGxvY2F0ZWQgYXMgZmFyIGFzCj4+IHRoZSBPUyBpcyBjb25jZXJuZWQsIHdoaWNo IG1hdGNoZXMgb3VyIGNhcGFiaWxpdGllcyB3ZWxsLgo+IGhlcmUgSSBoYXZlIGEgcXVlc3Rpb24u IGFzIHlvdSBjb21tZW50OiAiICdldGMvaGFyZHdyZV9lcnJvcnMnIGZ3X2NmZyBibG9iIHdpbGwg YWxzbyBjb250YWluIE4gRXJyb3IgU3RhdHVzIERhdGEgQmxvY2tzIiwKPiBCZWNhdXNlIHRoZSBD UEVSIG51bWJlcnMgaXMgbm90IGZpeGVkLCBob3cgdG8gYXNzaWduIGVhY2ggIkVycm9yIFN0YXR1 cyBEYXRhIEJsb2NrIiBzaXplIHVzaW5nIG9uZSAiZXRjL2hhcmR3cmVfZXJyb3JzIiBmd19jZmcg YmxvYi4KPiB3aGVuIHVzZSBvbmUgZXRjL2hhcmR3cmVfZXJyb3JzLCB3aWxsIHRoZSBOIEVycm9y IFN0YXR1cyBEYXRhIEJsb2NrIHVzZSBvbmUgY29udGludW91cyBidWZmZXI/IGFzIHNob3duIGJl bG93LiBpZiBzbywgbWF5YmUgaXQgbm90IGNvbnZlbmllbnQgZm9yIGVhY2ggZGF0YSBibG9jayBz aXplIGV4dGVuc2lvbi4KPiBJIHNlZSB0aGUgYmlvc19saW5rZXJfbG9hZGVyX2FsbG9jIHdpbGwg YWxsb2NhdGUgb25lIGNvbnRpbnVvdXMgYnVmZmVyIGZvciBhIGJsb2Ioc3VjaCBhcyBWTUdFTklE X0dVSURfRldfQ0ZHX0ZJTEUpCj4gCj4gICAgIC8qIEFsbG9jYXRlIGd1ZXN0IG1lbW9yeSBmb3Ig dGhlIERhdGEgZndfY2ZnIGJsb2IgKi8KPiAgICAgYmlvc19saW5rZXJfbG9hZGVyX2FsbG9jKGxp bmtlciwgVk1HRU5JRF9HVUlEX0ZXX0NGR19GSUxFLCBndWlkLCA0MDk2LAo+ICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgZmFsc2UgLyogcGFnZSBib3VuZGFyeSwgaGlnaCBtZW1vcnkgKi8p Owo+IAo+IAo+IAo+IC0+ICstLS0tLS0tLS0tLS0tLSsKPiAgICAgIHwgICAgSEVTVCAgICAgICsg ICAgIHwgcmVnaXN0ZXJzIHwgICAgICAgICAgICAgfCBFcnJvciBTdGF0dXMgfAo+ICAgICAgKyAr LS0tLS0tLS0tLS0tKyAgICAgfCArLS0tLS0tLS0tKyAgICAgICAgICAgICB8IERhdGEgQmxvY2sg IHwKPiAgICAgIHwgfCBHSEVTICAgICAgIHwgLS0+IHwgfCBhZGRyZXNzIHwgLS0tLS0tLS0rLS0+ fCArLS0tLS0tLS0tLS0tKwo+ICAgICAgfCB8IEdIRVMgICAgICAgfCAtLT4gfCB8IGFkZHJlc3Mg fCAtLS0tLS0rICAgICB8IHwgIENQRVIgICAgICB8Cj4gICAgICB8IHwgR0hFUyAgICAgICB8IC0t PiB8IHwgYWRkcmVzcyB8IC0tLS0rIHwgICAgIHwgfCAgQ1BFUiAgICAgIHwKPiAgICAgIHwgfCBH SEVTICAgICAgIHwgLS0+IHwgfCBhZGRyZXNzIHwgLSsgIHwgfCAgICAgfCB8ICBDUEVSICAgICAg fAo+ICAgICAgKy0rLS0tLS0tLS0tLS0tKyAgICAgKy0rLS0tLS0tLS0tKyAgfCAgfCArLS0tPiAr LS0tLS0tLS0tLS0tLS0rCj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICB8ICB8ICAgICAgIHwgfCAgQ1BFUiAgICAgIHwKPiAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHwgIHwgICAgICAgfCB8ICBDUEVSICAgICAgfAo+ICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCAgKy0tLS0tPiArLS0tLS0tLS0tLS0tLS0r Cj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB8ICAgICAgICAgIHwg fCAgQ1BFUiAgICAgIHwKPiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICstLS0tLS0tLT4gKy0tLS0tLS0tLS0tLS0tKwo+ICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICB8IHwgIENQRVIgICAgICB8Cj4gICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgfCAgQ1BFUiAgICAgIHwK PiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgfCB8 ICBDUEVSICAgICAgfAo+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICArLSstLS0tLS0tLS0tLS0rCj4gCj4gCj4gCj4gc28gaG93IGFib3V0IHdlIHVz ZSBzZXBhcmF0ZSBldGMvaGFyZHdyZV9lcnJvcnNOIGZvciBlYWNoIEVycm9yIFN0YXR1cyBzdGF0 dXMgQmxvY2s/IHRoZW4KPiAKPiBldGMvaGFyZHdyZV9lcnJvcnMwCj4gZXRjL2hhcmR3cmVfZXJy b3JzMQo+IC4uLi4uLi4uLi4uLi4uLi4uLi4KPiBldGMvaGFyZHdyZV9lcnJvcnMxMAo+ICh0aGUg bWF4IE4gaXMgMTApCj4gCj4gCj4gdGhlIE4gY2FuIGJlIG9uZSBvZiBiZWxvdyB2YWx1ZXMsIGFj Y29yZGluZyB0byBBQ1BJIHNwZWMgIlRhYmxlIDE4LTM0NSBIYXJkd2FyZSBFcnJvciBOb3RpZmlj YXRpb24gU3RydWN0dXJlIgo+IDAg4oCTIFBvbGxlZAo+IDEg4oCTIEV4dGVybmFsIEludGVycnVw dAo+IDIg4oCTIExvY2FsIEludGVycnVwdAo+IDMg4oCTIFNDSQo+IDQg4oCTIE5NSQo+IDUgLSBD TUNJCj4gNiAtIE1DRQo+IDcgLSBHUElPLVNpZ25hbAo+IDggLSBBUk12OCBTRUEKPiA5IC0gQVJN djggU0VJCj4gMTAgLSBFeHRlcm5hbCBJbnRlcnJ1cHQgLSBHU0lWCgpJJ20gdW5zdXJlIGlmLCBi eSAibm90IGZpeGVkIiwgeW91IGFyZSBzYXlpbmcKCiAgdGhlIG51bWJlciBvZiBDUEVSIGVudHJp ZXMgdGhhdCBmaXRzIGluIEVycm9yIFN0YXR1cyBEYXRhIEJsb2NrIE4gaXMKICBub3QgKnVuaWZv cm0qIGFjcm9zcyAwIDw9IE4gPD0gMTAgWzFdCgpvcgoKICB0aGUgbnVtYmVyIG9mIENQRVIgZW50 cmllcyB0aGF0IGZpdHMgaW4gRXJyb3IgU3RhdHVzIERhdGEgQmxvY2sgTiBpcwogIG5vdCAqa25v d24qIGluIGFkdmFuY2UsIGZvciBhbGwgb2YgMCA8PSBOIDw9IDEwIFsyXQoKV2hpY2ggb25lIGlz IHlvdXIgcG9pbnQ/CgpJZiBbMV0sIHRoYXQncyBubyBwcm9ibGVtOyB5b3UgY2FuIHNpbXBseSBz dW0gdGhlIGluZGl2aWR1YWwgZXJyb3IKc3RhdHVzIGRhdGEgYmxvY2sgc2l6ZXMgaW4gYWR2YW5j ZSwgYW5kIGFsbG9jYXRlICJldGMvaGFyZHdhcmVfZXJyb3JzIgphY2NvcmRpbmdseSwgdXNpbmcg dGhlIHRvdGFsIHNpemUuCgooQWxsb2NhdGluZyBvbmUgc2hhcmVkIGZ3X2NmZyBibG9iIGZvciBh bGwgc3RhdHVzIGRhdGEgYmxvY2tzIGlzIG1vcmUKbWVtb3J5IGVmZmljaWVudCwgYXMgZWFjaCBB TExPQ0FURSBjb21tYW5kIHdpbGwgYWxsb2NhdGUgd2hvbGUgcGFnZXMKKHJvdW5kZWQgdXAgZnJv bSB0aGUgYWN0dWFsIGJsb2Igc2l6ZSkuKQoKSWYgeW91ciBwb2ludCBpcyBbMl0sIHRoZW4gc3Bs aXR0aW5nIHRoZSBlcnJvciBzdGF0dXMgZGF0YSBibG9ja3MgdG8Kc2VwYXJhdGUgZndfY2ZnIGJs b2JzIG1ha2VzIG5vIGRpZmZlcmVuY2U6IHJlZ2FyZGxlc3Mgb2Ygd2hldGhlciB3ZSB0cnkKdG8g cGxhY2UgYWxsIHRoZSBlcnJvciBzdGF0dXMgZGF0YSBibG9ja3MgaW4gYSBzaW5nbGUgZndfY2Zn IGJsb2IsIG9yIGluCnNlcGFyYXRlIGZ3X2NmZyBibG9icywgdGhlIGluZGl2aWR1YWwgZGF0YSBi bG9jayBjYW5ub3QgYmUgcmVzaXplZCBhdCBPUwpydW50aW1lLCBzbyB0aGVyZSdzIG5vIHdheSB0 byBtYWtlIGl0IHdvcmsuCgpUaGFua3MsCkxhc3psbwoKPiAKPiAKPiAKPiAKPj4KPj4gKDMpIFFF TVUgZ2VuZXJhdGVzIHRoZSBBQ1BJIGxpbmtlci9sb2FkZXIgc2NyaXB0IGZvciB0aGUgZmlybXdh cmUsIGFzCj4+IGFsd2F5cy4KPj4KPj4gKDNhKSBUaGUgSEVTVCB0YWJsZSBpcyBwYXJ0IG9mICJl dGMvYWNwaS90YWJsZXMiLCB3aGljaCB0aGUgZmlybXdhcmUKPj4gYWxyZWFkeSBhbGxvY2F0ZXMg bWVtb3J5IGZvciwgYW5kIGRvd25sb2FkcyAoYmVjYXVzZSBRRU1VIGFscmVhZHkKPj4gZ2VuZXJh dGVzIGFuIEFMTE9DQVRFIGxpbmtlci9sb2FkZXIgY29tbWFuZCBmb3IgaXQgYWxyZWFkeSkuCj4+ Cj4+ICgzYikgUUVNVSB3aWxsIGhhdmUgdG8gY3JlYXRlIGFub3RoZXIgQUxMT0NBVEUgY29tbWFu ZCBmb3IgdGhlCj4+ICJldGMvaGFyZHdhcmVfZXJyb3JzIiBibG9iLiBUaGUgZmlybXdhcmUgYWxs b2NhdGVzIG1lbW9yeSBmb3IgdGhpcyBibG9iLAo+PiBhbmQgZG93bmxvYWRzIGl0Lgo+Pgo+PiAo NCkgUUVNVSBnZW5lcmF0ZXMsIGluIHRoZSBBQ1BJIGxpbmtlci9sb2FkZXIgc2NyaXB0IGZvciB0 aGUgZmlyd21hcmUsIE4KPj4gQUREX1BPSU5URVIgY29tbWFuZHMsIHdoaWNoIHBvaW50IHRoZSBH SEVTLiJFcnJvciBTdGF0dXMKPj4gQWRkcmVzcyIgZmllbGRzIGluIHRoZSBIRVNUIHRhYmxlLCB0 byB0aGUgY29ycmVzcG9uZGluZyBhZGRyZXNzCj4+IHJlZ2lzdGVycyBpbiB0aGUgZG93bmxvYWRl ZCAiZXRjL2hhcmR3YXJlX2Vycm9ycyIgYmxvYi4KPj4KPj4gKDUpIFFFTVUgZ2VuZXJhdGVzIGFu IEFERF9DSEVDS1NVTSBjb21tYW5kIGZvciB0aGUgZmlybXdhcmUsIHNvIHRoYXQgdGhlCj4+IEhF U1QgdGFibGUgaXMgY29ycmVjdGx5IGNoZWNrc3VtbWVkIGFmdGVyIGV4ZWN1dGluZyB0aGUgTiBB RERfUE9JTlRFUgo+PiBjb21tYW5kcyBmcm9tICg0KS4KPj4KPj4gKDYpIFFFTVUgZ2VuZXJhdGVz IE4gQUREX1BPSU5URVIgY29tbWFuZHMgZm9yIHRoZSBmaXJtd2FyZSwgcG9pbnRpbmcgdGhlCj4+ IGFkZHJlc3MgcmVnaXN0ZXJzIChsb2NhdGVkIGluIGd1ZXN0IG1lbW9yeSwgaW4gdGhlIGRvd25s b2FkZWQKPj4gImV0Yy9oYXJkd2FyZV9lcnJvcnMiIGJsb2IpIHRvIHRoZSByZXNwZWN0aXZlIEVy cm9yIFN0YXR1cyBEYXRhIEJsb2Nrcy4KPj4KPj4gKDcpIChUaGlzIGlzIHRoZSB0cmljay4pIEZv ciB0aGlzIHN0ZXAsIHdlIG5lZWQgYSB0aGlyZCwgd3JpdGUtb25seQo+PiBmd19jZmcgYmxvYiwg Y2FsbGVkICJldGMvaGFyZHdhcmVfZXJyb3JzX2FkZHIiLiBUaHJvdWdoIHRoYXQgYmxvYiwgdGhl Cj4+IGZpcm13YXJlIGNhbiBzZW5kIGJhY2sgdGhlIGd1ZXN0LXNpZGUgYWxsb2NhdGlvbiBhZGRy ZXNzZXMgdG8gUUVNVS4KPj4KPj4gTmFtZWx5LCB0aGUgImV0Yy9oYXJkd2FyZV9lcnJvcnNfYWRk ciIgYmxvYiBjb250YWlucyBOIDgtYnl0ZSBlbnRyaWVzLgo+PiBRRU1VIGdlbmVyYXRlcyBOIFdS SVRFX1BPSU5URVIgY29tbWFuZHMgZm9yIHRoZSBmaXJtd2FyZS4KPj4KPj4gRm9yIGVycm9yIHNv dXJjZSBLICgwIDw9IEsgPCBOKSwgUUVNVSBpbnN0cnVjdHMgdGhlIGZpcm13YXJlIHRvCj4+IGNh bGN1bGF0ZSB0aGUgZ3Vlc3QgYWRkcmVzcyBvZiBFcnJvciBTdGF0dXMgRGF0YSBCbG9jayBLLCBm cm9tIHRoZQo+PiBRRU1VLWRpY3RhdGVkIG9mZnNldCB3aXRoaW4gImV0Yy9oYXJkd2FyZV9lcnJv cnMiLCBhbmQgZnJvbSB0aGUKPj4gZ3Vlc3QtZGV0ZXJtaW5lZCBhbGxvY2F0aW9uIGJhc2UgYWRk cmVzcyBmb3IgImV0Yy9oYXJkd2FyZV9lcnJvcnMiLiBUaGUKPj4gZmlybXdhcmUgdGhlbiB3cml0 ZXMgdGhlIGNhbGN1bGF0ZWQgYWRkcmVzcyBiYWNrIHRvIGZ3X2NmZyBmaWxlCj4+ICJldGMvaGFy ZHdhcmVfZXJyb3JzX2FkZHIiLCBhdCBvZmZzZXQgSyo4LCBhY2NvcmRpbmcgdG8gdGhlCj4+IFdS SVRFX1BPSU5URVIgY29tbWFuZC4KPj4KPj4gVGhpcyB3YXkgUUVNVSB3aWxsIGtub3cgdGhlIEdQ QSBvZiBlYWNoIEVycm9yIFN0YXR1cyBEYXRhIEJsb2NrLgo+Pgo+PiAoSW4gZmFjdCB0aGlzIGNh biBiZSBzaW1wbGlmaWVkIHRvIGEgc2luZ2xlIFdSSVRFX1BPSU5URVIgY29tbWFuZDogdGhlCj4+ IGFkZHJlc3Mgb2YgdGhlICJhZGRyZXNzIHJlZ2lzdGVyIHRhYmxlIiBjYW4gYmUgc2VudCBiYWNr IHRvIFFFTVUgYXMKPj4gd2VsbCwgd2hpY2ggYWxyZWFkeSBjb250YWlucyBhbGwgRXJyb3IgU3Rh dHVzIERhdGEgQmxvY2sgYWRkcmVzc2VzLikKPj4KPj4gKDgpIFdoZW4gUUVNVSBnZXRzIFNJR0JV UyBmcm9tIHRoZSBrZXJuZWwgLS0gSSBob3BlIHRoYXQncyBnb2luZyB0byBjb21lCj4+IHRocm91 Z2ggYSBzaWduYWxmZCAtLSBRRU1VIGNhbiBmb3JtYXQgdGhlIENQRVIgcmlnaHQgaW50byBndWVz dCBtZW1vcnksCj4+IGFuZCB0aGVuIGluamVjdCB3aGF0ZXZlciBpbnRlcnJ1cHQgKG9yIGFzc2Vy dCB3aGF0ZXZlciBHUElPIGxpbmUpIGlzCj4+IG5lY2Vzc2FyeSBmb3Igbm90aWZ5aW5nIHRoZSBn dWVzdC4KPj4KPj4gKDkpIFRoaXMgbm90aWZpY2F0aW9uIChpbiB2aXJ0dWFsIGhhcmR3YXJlKSBj YW4gZWl0aGVyIGJlIGhhbmRsZWQgYnkgdGhlCj4+IGd1ZXN0IGtlcm5lbCBzdGFuZC1hbG9uZSwg b3IgZWxzZSB0aGUgZ3Vlc3Qga2VybmVsIGNhbiBpbnZva2UgYW4gQUNQSQo+PiBldmVudCBoYW5k bGVyIG1ldGhvZCB3aXRoIGl0ICh3aGljaCB3b3VsZCBiZSBpbiB0aGUgRFNEVCBvciBvbmUgb2Yg dGhlCj4+IFNTRFRzLCBhbHNvIGdlbmVyYXRlZCBieSBRRU1VKS4gVGhlIEFDUEkgZXZlbnQgaGFu ZGxlciBtZXRob2QgY291bGQKPj4gaW52b2tlIHRoZSBzcGVjaWZpYyBndWVzdCBrZXJuZWwgZHJp dmVyIGZvciBlcnJyb3IgaGFuZGxpbmcgdmlhIGEKPj4gTm90aWZ5KCkgb3BlcmF0aW9uLgo+Pgo+ PiBJJ20gYXR0cmFjdGVkIHRvIHRoZSBhYm92ZSBkZXNpZ24gYmVjYXVzZToKPj4gLSBpdCB3b3Vs ZCBsZWF2ZSB0aGUgZmlybXdhcmUgYWxvbmUgYWZ0ZXIgT1MgYm9vdCwgYW5kCj4+IC0gaXQgd291 bGQgbGVhdmUgdGhlIGZpcm13YXJlIGJsaXNzZnVsbHkgaWdub3JhbnQgYWJvdXQgSEVTVCwgR0hF UywKPj4gQ1BFUiwgYW5kIHRoZSBsaWtlLiAoVGhhdCdzIHdoeSBRRU1VJ3MgQUNQSSBsaW5rZXIv bG9hZGVyIHdhcyBpbnZlbnRlZAo+PiBpbiB0aGUgZmlyc3QgcGxhY2UuKQo+Pgo+PiBUaGFua3MK Pj4gTGFzemxvCj4+Cj4+Pj4gICAgRG8geW91IHRoaW5rIHdoaWNoIG1vZHVsZXMgZ2VuZXJhdGVz IHRoZSBBUEVJIHRhYmxlIGlzIGJldHRlcj8gVUVGSSBvciBRZW11Pwo+Pj4+Cj4+Pj4KPj4+Pgo+ Pj4+Cj4+Pj4gT24gMjAxNy8zLzI4IDIxOjQwLCBKYW1lcyBNb3JzZSB3cm90ZToKPj4+Pj4gSGkg Z2VuZ2RvbmdqaXUsCj4+Pj4+Cj4+Pj4+IE9uIDI4LzAzLzE3IDEzOjE2LCBnZW5nZG9uZ2ppdSB3 cm90ZToKPj4+Pj4+IE9uIDIwMTcvMy8yOCAxOTo1NCwgQWNoaW4gR3VwdGEgd3JvdGU6Cj4+Pj4+ Pj4gT24gVHVlLCBNYXIgMjgsIDIwMTcgYXQgMDE6MjM6MjhQTSArMDIwMCwgQ2hyaXN0b2ZmZXIg RGFsbCB3cm90ZToKPj4+Pj4+Pj4gT24gVHVlLCBNYXIgMjgsIDIwMTcgYXQgMTE6NDg6MDhBTSAr MDEwMCwgSmFtZXMgTW9yc2Ugd3JvdGU6Cj4+Pj4+Pj4+PiBPbiB0aGUgaG9zdCwgcGFydCBvZiBV RUZJIGlzIGludm9sdmVkIHRvIGdlbmVyYXRlIHRoZSBDUEVSIHJlY29yZHMuCj4+Pj4+Pj4+PiBJ biBhIGd1ZXN0PywgSSBkb24ndCBrbm93Lgo+Pj4+Pj4+Pj4gUWVtdSBjb3VsZCBnZW5lcmF0ZSB0 aGUgcmVjb3Jkcywgb3IgZHJpdmUgc29tZSBvdGhlciBjb21wb25lbnQgdG8gZG8gaXQuCj4+Pj4+ Pj4+Cj4+Pj4+Pj4+IEkgdGhpbmsgSSBhbSBiZWdpbm5pbmcgdG8gdW5kZXJzdGFuZCB0aGlzIGEg Yml0LiAgU2luY2UgdGhlIGd1ZXQgVUVGSQo+Pj4+Pj4+PiBpbnN0YW5jZSBpcyBzcGVjaWZpY2Fs bHkgYnVpbHQgZm9yIHRoZSBtYWNoaW5lIGl0IHJ1bnMgb24sIFFFTVUncyB2aXJ0Cj4+Pj4+Pj4+ IG1hY2hpbmUgaW4gdGhpcyBjYXNlLCB0aGV5IGNvdWxkIHNpbXBseSBhZ3JlZSAoYnkgc29tZSBj b250cmFjdCkgdG8KPj4+Pj4+Pj4gcGxhY2UgdGhlIHJlY29yZHMgYXQgc29tZSBzcGVjaWZpYyBs b2NhdGlvbiBpbiBtZW1vcnksIGFuZCBpZiB0aGUgZ3Vlc3QKPj4+Pj4+Pj4ga2VybmVsIGFza3Mg aXRzIGd1ZXN0IFVFRkkgZm9yIHRoYXQgbG9jYXRpb24sIHRoaW5ncyBzaG91bGQganVzdCB3b3Jr IGJ5Cj4+Pj4+Pj4+IGhhdmluZyBsb2dpYyBpbiBRRU1VIHRvIHByb2Nlc3MgZXJyb3IgcmVwb3J0 cyBhbmQgcG9wdWxhdGUgZ3Vlc3QgbWVtb3J5Lgo+Pj4+Pj4+Pgo+Pj4+Pj4+PiBJcyB0aGlzIGhv dyBvdGhlcnMgc2VlIHRoZSB3b3JsZCB0b28/Cj4+Pj4+Pj4KPj4+Pj4+PiBJIHRoaW5rIHNvIQo+ Pj4+Pj4+Cj4+Pj4+Pj4gQUZBSVUsIHRoZSBtZW1vcnkgd2hlcmUgQ1BFUnMgd2lsbCByZXNpZGUg c2hvdWxkIGJlIHNwZWNpZmllZCBpbiBhIEdIRVMgZW50cnkgaW4KPj4+Pj4+PiB0aGUgSEVTVC4g SXMgdGhpcyBub3QgdGhlIGNhc2Ugd2l0aCBhIGd1ZXN0IGtlcm5lbCBpLmUuIHRoZSBndWVzdCBV RUZJIGNyZWF0ZXMgYQo+Pj4+Pj4+IEhFU1QgZm9yIHRoZSBndWVzdCBLZXJuZWw/Cj4+Pj4+Pj4K Pj4+Pj4+PiBJZiBzbywgdGhlbiB0aGUgcXVlc3Rpb24gaXMgaG93IHRoZSBndWVzdCBVRUZJIGZp bmRzIG91dCB3aGVyZSBRRU1VIChhY3RpbmcgYXMKPj4+Pj4+PiBFTDMgZmlybXdhcmUpIHdpbGwg cG9wdWxhdGUgdGhlIENQRVJzLiBUaGlzIGNvdWxkIGVpdGhlciBiZSBhIGNvbnRyYWN0IGJldHdl ZW4KPj4+Pj4+PiB0aGUgdHdvIG9yIGEgZ3Vlc3QgRFhFIGRyaXZlciB1c2VzIHRoZSBNTV9DT01N VU5JQ0FURSBjYWxsIChzZWUgWzFdKSB0byBhc2sgUUVNVQo+Pj4+Pj4+IHdoZXJlIHRoZSBtZW1v cnkgaXMuCj4+Pj4+Pgo+Pj4+Pj4gd2hldGhlciBpbnZva2UgdGhlIGd1ZXN0IFVFRkkgd2lsbCBi ZSBjb21wbGV4PyBub3Qgc2VlIHRoZSBhZHZhbnRhZ2UuIGl0IHNlZW1zIHg4NiBRZW11Cj4+Pj4+ PiBkaXJlY3RseSBnZW5lcmF0ZSB0aGUgQUNQSSB0YWJsZSwgYnV0IEkgYW0gbm90IHN1cmUsIHdl IGFyZSBjaGVja2luZyB0aGUgcWVtdQo+Pj4+PiBsb2dpY2FsLgo+Pj4+Pj4gbGV0IFFlbXUgZ2Vu ZXJhdGUgQ1BFUiByZWNvcmQgbWF5IGJlIGNsZWFyLgo+Pj4+Pgo+Pj4+PiBBdCBib290IFVFRkkg aW4gdGhlIGd1ZXN0IHdpbGwgbmVlZCB0byBtYWtlIHN1cmUgdGhlIGFyZWFzIG9mIG1lbW9yeSB0 aGF0IG1heSBiZQo+Pj4+PiB1c2VkIGZvciBDUEVSIHJlY29yZHMgYXJlIHJlc2VydmVkLiBXaGV0 aGVyIFVFRkkgb3IgUWVtdSBkZWNpZGVzIHdoZXJlIHRoZXNlIGFyZQo+Pj4+PiBuZWVkcyBkZWNp ZGluZywgKGJ1dCBwcm9iYWJseSBub3QgaGVyZSkuLi4KPj4+Pj4KPj4+Pj4gQXQgcnVudGltZSwg d2hlbiBhbiBlcnJvciBoYXMgb2NjdXJyZWQsIEkgYWdyZWUgaXQgd291bGQgYmUgc2ltcGxlciAo ZmV3ZXIKPj4+Pj4gY29tcG9uZW50cyBpbnZvbHZlZCkgaWYgUWVtdSBnZW5lcmF0ZXMgdGhlIENQ RVIgcmVjb3Jkcy4gQnV0IGlmIFVFRkkgbWFkZSB0aGUKPj4+Pj4gbWVtb3J5IGNob2ljZSBhYm92 ZSB0aGV5IG5lZWQgdG8gaW50ZXJhY3QgYW5kIGl0IGdldHMgY29tcGxpY2F0ZWQgYWdhaW4uIFRo ZQo+Pj4+PiBDUEVSIHJlY29yZHMgYXJlIGRlZmluZWQgaW4gdGhlIFVFRkkgc3BlYywgc28gSSB3 b3VsZCBleHBlY3QgVUVGSSB0byBjb250YWluCj4+Pj4+IGNvZGUgdG8gZ2VuZXJhdGUvcGFyc2Ug dGhlbS4KPj4+Pj4KPj4+Pj4KPj4+Pj4gVGhhbmtzLAo+Pj4+Pgo+Pj4+PiBKYW1lcwo+Pj4+Pgo+ Pj4+Pgo+Pj4+PiAuCj4+Pj4+Cj4+Pj4KPj4KPj4KPj4gLgo+Pgo+IAoKX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZWRrMi1kZXZlbCBtYWlsaW5nIGxpc3QK ZWRrMi1kZXZlbEBsaXN0cy4wMS5vcmcKaHR0cHM6Ly9saXN0cy4wMS5vcmcvbWFpbG1hbi9saXN0 aW5mby9lZGsyLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:45004) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cwCZT-0001bB-EV for qemu-devel@nongnu.org; Thu, 06 Apr 2017 14:55:54 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cwCZO-0001jj-FV for qemu-devel@nongnu.org; Thu, 06 Apr 2017 14:55:51 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37272) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1cwCZO-0001iy-5L for qemu-devel@nongnu.org; Thu, 06 Apr 2017 14:55:46 -0400 References: <76795e20-2f20-1e54-cfa5-7444f28b18ee@huawei.com> <20170321113428.GC15920@cbox> <58D17AF0.2010802@arm.com> <20170321193933.GB31111@cbox> <58DA3F68.6090901@arm.com> <20170328112328.GA31156@cbox> <20170328115413.GJ23682@e104320-lin> <58DA67BA.8070404@arm.com> <5b7352f4-4965-3ed5-3879-db871797be47@huawei.com> <20170329103658.GQ23682@e104320-lin> <2a427164-9b37-6711-3a56-906634ba7f12@redhat.com> <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> From: Laszlo Ersek Message-ID: <6ac1597a-2ed5-36b2-848d-5fd048b16d66@redhat.com> Date: Thu, 6 Apr 2017 20:55:34 +0200 MIME-Version: 1.0 In-Reply-To: <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: [Qemu-devel] [PATCH] kvm: pass the virtual SEI syndrome to guest OS List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: gengdongjiu , Achin Gupta Cc: ard.biesheuvel@linaro.org, edk2-devel@lists.01.org, qemu-devel@nongnu.org, zhaoshenglong@huawei.com, James Morse , Christoffer Dall , xiexiuqi@huawei.com, Marc Zyngier , catalin.marinas@arm.com, will.deacon@arm.com, christoffer.dall@linaro.org, rkrcmar@redhat.com, suzuki.poulose@arm.com, andre.przywara@arm.com, mark.rutland@arm.com, vladimir.murzin@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, wangxiongfeng2@huawei.com, wuquanming@huawei.com, huangshaoyu@huawei.com, Leif.Lindholm@linaro.comnd@arm.com, Michael Tsirkin , Igor Mammedov On 04/06/17 14:35, gengdongjiu wrote: > Dear, Laszlo > Thanks for your detailed explanation. >=20 > On 2017/3/29 19:58, Laszlo Ersek wrote: >> (This ought to be one of the longest address lists I've ever seen :) >> Thanks for the CC. I'm glad Shannon is already on the CC list. For goo= d >> measure, I'm adding MST and Igor.) >> >> On 03/29/17 12:36, Achin Gupta wrote: >>> Hi gengdongjiu, >>> >>> On Wed, Mar 29, 2017 at 05:36:37PM +0800, gengdongjiu wrote: >>>> >>>> Hi Laszlo/Biesheuvel/Qemu developer, >>>> >>>> Now I encounter a issue and want to consult with you in ARM64 pla= tform=EF=BC=8C as described below: >>>> >>>> when guest OS happen synchronous or asynchronous abort, kvm needs >>>> to send the error address to Qemu or UEFI through sigbus to >>>> dynamically generate APEI table. from my investigation, there are >>>> two ways: >>>> >>>> (1) Qemu get the error address, and generate the APEI table, then >>>> notify UEFI to know this generation, then inject abort error to >>>> guest OS, guest OS read the APEI table. >>>> (2) Qemu get the error address, and let UEFI to generate the APEI >>>> table, then inject abort error to guest OS, guest OS read the APEI >>>> table. >>> >>> Just being pedantic! I don't think we are talking about creating the = APEI table >>> dynamically here. The issue is: Once KVM has received an error that i= s destined >>> for a guest it will raise a SIGBUS to Qemu. Now before Qemu can injec= t the error >>> into the guest OS, a CPER (Common Platform Error Record) has to be ge= nerated >>> corresponding to the error source (GHES corresponding to memory subsy= stem, >>> processor etc) to allow the guest OS to do anything meaningful with t= he >>> error. So who should create the CPER is the question. >>> >>> At the EL3/EL2 interface (Secure Firmware and OS/Hypervisor), an erro= r arrives >>> at EL3 and secure firmware (at EL3 or a lower secure exception level)= is >>> responsible for creating the CPER. ARM is experimenting with using a = Standalone >>> MM EDK2 image in the secure world to do the CPER creation. This will = avoid >>> adding the same code in ARM TF in EL3 (better for security). The erro= r will then >>> be injected into the OS/Hypervisor (through SEA/SEI/SDEI) through ARM= Trusted >>> Firmware. >>> >>> Qemu is essentially fulfilling the role of secure firmware at the EL2= /EL1 >>> interface (as discussed with Christoffer below). So it should generat= e the CPER >>> before injecting the error. >>> >>> This is corresponds to (1) above apart from notifying UEFI (I am assu= ming you >>> mean guest UEFI). At this time, the guest OS already knows where to p= ick up the >>> CPER from through the HEST. Qemu has to create the CPER and populate = its address >>> at the address exported in the HEST. Guest UEFI should not be involve= d in this >>> flow. Its job was to create the HEST at boot and that has been done b= y this >>> stage. >>> >>> Qemu folk will be able to add but it looks like support for CPER gene= ration will >>> need to be added to Qemu. We need to resolve this. >>> >>> Do shout if I am missing anything above. >> >> After reading this email, the use case looks *very* similar to what >> we've just done with VMGENID for QEMU 2.9. >> >> We have a facility between QEMU and the guest firmware, called "ACPI >> linker/loader", with which QEMU instructs the firmware to >> >> - allocate and download blobs into guest RAM (AcpiNVS type memory) -- >> ALLOCATE command, >> >> - relocate pointers in those blobs, to fields in other (or the same) >> blobs -- ADD_POINTER command, >> >> - set ACPI table checksums -- ADD_CHECKSUM command, >> >> - and send GPAs of fields within such blobs back to QEMU -- >> WRITE_POINTER command. >> >> This is how I imagine we can map the facility to the current use case >> (note that this is the first time I read about HEST / GHES / CPER): >> >> etc/acpi/tables etc/hardware_errors >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >> +-----------+ >> +--------------+ | address | +-> +--------------+ >> | HEST + | registers | | | Error Status | >> + +------------+ | +---------+ | | Data Block 1 | >> | | GHES | --> | | address | --------+ | +------------+ >> | | GHES | --> | | address | ------+ | | CPER | >> | | GHES | --> | | address | ----+ | | | CPER | >> | | GHES | --> | | address | -+ | | | | CPER | >> +-+------------+ +-+---------+ | | | +-+------------+ >> | | | >> | | +---> +--------------+ >> | | | Error Status | >> | | | Data Block 2 | >> | | | +------------+ >> | | | | CPER | >> | | | | CPER | >> | | +-+------------+ >> | | >> | +-----> +--------------+ >> | | Error Status | >> | | Data Block 3 | >> | | +------------+ >> | | | CPER | >> | +-+------------+ >> | >> +--------> +--------------+ >> | Error Status | >> | Data Block 4 | >> | +------------+ >> | | CPER | >> | | CPER | >> | | CPER | >> +-+------------+ >> >> (1) QEMU generates the HEST ACPI table. This table goes in the current >> "etc/acpi/tables" fw_cfg blob. Given N error sources, there will be N >> GHES objects in the HEST. >> >> (2) We introduce a new fw_cfg blob called "etc/hardware_errors". QEMU >> also populates this blob. >> >> (2a) Given N error sources, the (unnamed) table of address registers >> will contain N address registers. >> >> (2b) Given N error sources, the "etc/hardwre_errors" fw_cfg blob will >> also contain N Error Status Data Blocks. >> >> I don't know about the sizing (number of CPERs) each Error Status Data >> Block has to contain, but I understand it is all pre-allocated as far = as >> the OS is concerned, which matches our capabilities well. > here I have a question. as you comment: " 'etc/hardwre_errors' fw_cfg b= lob will also contain N Error Status Data Blocks", > Because the CPER numbers is not fixed, how to assign each "Error Status= Data Block" size using one "etc/hardwre_errors" fw_cfg blob. > when use one etc/hardwre_errors, will the N Error Status Data Block use= one continuous buffer? as shown below. if so, maybe it not convenient fo= r each data block size extension. > I see the bios_linker_loader_alloc will allocate one continuous buffer = for a blob(such as VMGENID_GUID_FW_CFG_FILE) >=20 > /* Allocate guest memory for the Data fw_cfg blob */ > bios_linker_loader_alloc(linker, VMGENID_GUID_FW_CFG_FILE, guid, 40= 96, > false /* page boundary, high memory */); >=20 >=20 >=20 > -> +--------------+ > | HEST + | registers | | Error Status | > + +------------+ | +---------+ | Data Block | > | | GHES | --> | | address | --------+-->| +------------+ > | | GHES | --> | | address | ------+ | | CPER | > | | GHES | --> | | address | ----+ | | | CPER | > | | GHES | --> | | address | -+ | | | | CPER | > +-+------------+ +-+---------+ | | +---> +--------------+ > | | | | CPER | > | | | | CPER | > | +-----> +--------------+ > | | | CPER | > +--------> +--------------+ > | | CPER | > | | CPER | > | | CPER | > +-+------------+ >=20 >=20 >=20 > so how about we use separate etc/hardwre_errorsN for each Error Status = status Block? then >=20 > etc/hardwre_errors0 > etc/hardwre_errors1 > ................... > etc/hardwre_errors10 > (the max N is 10) >=20 >=20 > the N can be one of below values, according to ACPI spec "Table 18-345 = Hardware Error Notification Structure" > 0 =E2=80=93 Polled > 1 =E2=80=93 External Interrupt > 2 =E2=80=93 Local Interrupt > 3 =E2=80=93 SCI > 4 =E2=80=93 NMI > 5 - CMCI > 6 - MCE > 7 - GPIO-Signal > 8 - ARMv8 SEA > 9 - ARMv8 SEI > 10 - External Interrupt - GSIV I'm unsure if, by "not fixed", you are saying the number of CPER entries that fits in Error Status Data Block N is not *uniform* across 0 <=3D N <=3D 10 [1] or the number of CPER entries that fits in Error Status Data Block N is not *known* in advance, for all of 0 <=3D N <=3D 10 [2] Which one is your point? If [1], that's no problem; you can simply sum the individual error status data block sizes in advance, and allocate "etc/hardware_errors" accordingly, using the total size. (Allocating one shared fw_cfg blob for all status data blocks is more memory efficient, as each ALLOCATE command will allocate whole pages (rounded up from the actual blob size).) If your point is [2], then splitting the error status data blocks to separate fw_cfg blobs makes no difference: regardless of whether we try to place all the error status data blocks in a single fw_cfg blob, or in separate fw_cfg blobs, the individual data block cannot be resized at OS runtime, so there's no way to make it work. Thanks, Laszlo >=20 >=20 >=20 >=20 >> >> (3) QEMU generates the ACPI linker/loader script for the firmware, as >> always. >> >> (3a) The HEST table is part of "etc/acpi/tables", which the firmware >> already allocates memory for, and downloads (because QEMU already >> generates an ALLOCATE linker/loader command for it already). >> >> (3b) QEMU will have to create another ALLOCATE command for the >> "etc/hardware_errors" blob. The firmware allocates memory for this blo= b, >> and downloads it. >> >> (4) QEMU generates, in the ACPI linker/loader script for the firwmare,= N >> ADD_POINTER commands, which point the GHES."Error Status >> Address" fields in the HEST table, to the corresponding address >> registers in the downloaded "etc/hardware_errors" blob. >> >> (5) QEMU generates an ADD_CHECKSUM command for the firmware, so that t= he >> HEST table is correctly checksummed after executing the N ADD_POINTER >> commands from (4). >> >> (6) QEMU generates N ADD_POINTER commands for the firmware, pointing t= he >> address registers (located in guest memory, in the downloaded >> "etc/hardware_errors" blob) to the respective Error Status Data Blocks= . >> >> (7) (This is the trick.) For this step, we need a third, write-only >> fw_cfg blob, called "etc/hardware_errors_addr". Through that blob, the >> firmware can send back the guest-side allocation addresses to QEMU. >> >> Namely, the "etc/hardware_errors_addr" blob contains N 8-byte entries. >> QEMU generates N WRITE_POINTER commands for the firmware. >> >> For error source K (0 <=3D K < N), QEMU instructs the firmware to >> calculate the guest address of Error Status Data Block K, from the >> QEMU-dictated offset within "etc/hardware_errors", and from the >> guest-determined allocation base address for "etc/hardware_errors". Th= e >> firmware then writes the calculated address back to fw_cfg file >> "etc/hardware_errors_addr", at offset K*8, according to the >> WRITE_POINTER command. >> >> This way QEMU will know the GPA of each Error Status Data Block. >> >> (In fact this can be simplified to a single WRITE_POINTER command: the >> address of the "address register table" can be sent back to QEMU as >> well, which already contains all Error Status Data Block addresses.) >> >> (8) When QEMU gets SIGBUS from the kernel -- I hope that's going to co= me >> through a signalfd -- QEMU can format the CPER right into guest memory= , >> and then inject whatever interrupt (or assert whatever GPIO line) is >> necessary for notifying the guest. >> >> (9) This notification (in virtual hardware) can either be handled by t= he >> guest kernel stand-alone, or else the guest kernel can invoke an ACPI >> event handler method with it (which would be in the DSDT or one of the >> SSDTs, also generated by QEMU). The ACPI event handler method could >> invoke the specific guest kernel driver for errror handling via a >> Notify() operation. >> >> I'm attracted to the above design because: >> - it would leave the firmware alone after OS boot, and >> - it would leave the firmware blissfully ignorant about HEST, GHES, >> CPER, and the like. (That's why QEMU's ACPI linker/loader was invented >> in the first place.) >> >> Thanks >> Laszlo >> >>>> Do you think which modules generates the APEI table is better? UE= FI or Qemu? >>>> >>>> >>>> >>>> >>>> On 2017/3/28 21:40, James Morse wrote: >>>>> Hi gengdongjiu, >>>>> >>>>> On 28/03/17 13:16, gengdongjiu wrote: >>>>>> On 2017/3/28 19:54, Achin Gupta wrote: >>>>>>> On Tue, Mar 28, 2017 at 01:23:28PM +0200, Christoffer Dall wrote: >>>>>>>> On Tue, Mar 28, 2017 at 11:48:08AM +0100, James Morse wrote: >>>>>>>>> On the host, part of UEFI is involved to generate the CPER reco= rds. >>>>>>>>> In a guest?, I don't know. >>>>>>>>> Qemu could generate the records, or drive some other component = to do it. >>>>>>>> >>>>>>>> I think I am beginning to understand this a bit. Since the guet= UEFI >>>>>>>> instance is specifically built for the machine it runs on, QEMU'= s virt >>>>>>>> machine in this case, they could simply agree (by some contract)= to >>>>>>>> place the records at some specific location in memory, and if th= e guest >>>>>>>> kernel asks its guest UEFI for that location, things should just= work by >>>>>>>> having logic in QEMU to process error reports and populate guest= memory. >>>>>>>> >>>>>>>> Is this how others see the world too? >>>>>>> >>>>>>> I think so! >>>>>>> >>>>>>> AFAIU, the memory where CPERs will reside should be specified in = a GHES entry in >>>>>>> the HEST. Is this not the case with a guest kernel i.e. the guest= UEFI creates a >>>>>>> HEST for the guest Kernel? >>>>>>> >>>>>>> If so, then the question is how the guest UEFI finds out where QE= MU (acting as >>>>>>> EL3 firmware) will populate the CPERs. This could either be a con= tract between >>>>>>> the two or a guest DXE driver uses the MM_COMMUNICATE call (see [= 1]) to ask QEMU >>>>>>> where the memory is. >>>>>> >>>>>> whether invoke the guest UEFI will be complex? not see the advanta= ge. it seems x86 Qemu >>>>>> directly generate the ACPI table, but I am not sure, we are checki= ng the qemu >>>>> logical. >>>>>> let Qemu generate CPER record may be clear. >>>>> >>>>> At boot UEFI in the guest will need to make sure the areas of memor= y that may be >>>>> used for CPER records are reserved. Whether UEFI or Qemu decides wh= ere these are >>>>> needs deciding, (but probably not here)... >>>>> >>>>> At runtime, when an error has occurred, I agree it would be simpler= (fewer >>>>> components involved) if Qemu generates the CPER records. But if UEF= I made the >>>>> memory choice above they need to interact and it gets complicated a= gain. The >>>>> CPER records are defined in the UEFI spec, so I would expect UEFI t= o contain >>>>> code to generate/parse them. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> James >>>>> >>>>> >>>>> . >>>>> >>>> >> >> >> . >> >=20 From mboxrd@z Thu Jan 1 00:00:00 1970 From: lersek@redhat.com (Laszlo Ersek) Date: Thu, 6 Apr 2017 20:55:34 +0200 Subject: [PATCH] kvm: pass the virtual SEI syndrome to guest OS In-Reply-To: <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> References: <76795e20-2f20-1e54-cfa5-7444f28b18ee@huawei.com> <20170321113428.GC15920@cbox> <58D17AF0.2010802@arm.com> <20170321193933.GB31111@cbox> <58DA3F68.6090901@arm.com> <20170328112328.GA31156@cbox> <20170328115413.GJ23682@e104320-lin> <58DA67BA.8070404@arm.com> <5b7352f4-4965-3ed5-3879-db871797be47@huawei.com> <20170329103658.GQ23682@e104320-lin> <2a427164-9b37-6711-3a56-906634ba7f12@redhat.com> <7c5c8ab7-8fcc-1c98-0bc1-cccb66c4c84d@huawei.com> Message-ID: <6ac1597a-2ed5-36b2-848d-5fd048b16d66@redhat.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 04/06/17 14:35, gengdongjiu wrote: > Dear, Laszlo > Thanks for your detailed explanation. > > On 2017/3/29 19:58, Laszlo Ersek wrote: >> (This ought to be one of the longest address lists I've ever seen :) >> Thanks for the CC. I'm glad Shannon is already on the CC list. For good >> measure, I'm adding MST and Igor.) >> >> On 03/29/17 12:36, Achin Gupta wrote: >>> Hi gengdongjiu, >>> >>> On Wed, Mar 29, 2017 at 05:36:37PM +0800, gengdongjiu wrote: >>>> >>>> Hi Laszlo/Biesheuvel/Qemu developer, >>>> >>>> Now I encounter a issue and want to consult with you in ARM64 platform? as described below: >>>> >>>> when guest OS happen synchronous or asynchronous abort, kvm needs >>>> to send the error address to Qemu or UEFI through sigbus to >>>> dynamically generate APEI table. from my investigation, there are >>>> two ways: >>>> >>>> (1) Qemu get the error address, and generate the APEI table, then >>>> notify UEFI to know this generation, then inject abort error to >>>> guest OS, guest OS read the APEI table. >>>> (2) Qemu get the error address, and let UEFI to generate the APEI >>>> table, then inject abort error to guest OS, guest OS read the APEI >>>> table. >>> >>> Just being pedantic! I don't think we are talking about creating the APEI table >>> dynamically here. The issue is: Once KVM has received an error that is destined >>> for a guest it will raise a SIGBUS to Qemu. Now before Qemu can inject the error >>> into the guest OS, a CPER (Common Platform Error Record) has to be generated >>> corresponding to the error source (GHES corresponding to memory subsystem, >>> processor etc) to allow the guest OS to do anything meaningful with the >>> error. So who should create the CPER is the question. >>> >>> At the EL3/EL2 interface (Secure Firmware and OS/Hypervisor), an error arrives >>> at EL3 and secure firmware (at EL3 or a lower secure exception level) is >>> responsible for creating the CPER. ARM is experimenting with using a Standalone >>> MM EDK2 image in the secure world to do the CPER creation. This will avoid >>> adding the same code in ARM TF in EL3 (better for security). The error will then >>> be injected into the OS/Hypervisor (through SEA/SEI/SDEI) through ARM Trusted >>> Firmware. >>> >>> Qemu is essentially fulfilling the role of secure firmware at the EL2/EL1 >>> interface (as discussed with Christoffer below). So it should generate the CPER >>> before injecting the error. >>> >>> This is corresponds to (1) above apart from notifying UEFI (I am assuming you >>> mean guest UEFI). At this time, the guest OS already knows where to pick up the >>> CPER from through the HEST. Qemu has to create the CPER and populate its address >>> at the address exported in the HEST. Guest UEFI should not be involved in this >>> flow. Its job was to create the HEST at boot and that has been done by this >>> stage. >>> >>> Qemu folk will be able to add but it looks like support for CPER generation will >>> need to be added to Qemu. We need to resolve this. >>> >>> Do shout if I am missing anything above. >> >> After reading this email, the use case looks *very* similar to what >> we've just done with VMGENID for QEMU 2.9. >> >> We have a facility between QEMU and the guest firmware, called "ACPI >> linker/loader", with which QEMU instructs the firmware to >> >> - allocate and download blobs into guest RAM (AcpiNVS type memory) -- >> ALLOCATE command, >> >> - relocate pointers in those blobs, to fields in other (or the same) >> blobs -- ADD_POINTER command, >> >> - set ACPI table checksums -- ADD_CHECKSUM command, >> >> - and send GPAs of fields within such blobs back to QEMU -- >> WRITE_POINTER command. >> >> This is how I imagine we can map the facility to the current use case >> (note that this is the first time I read about HEST / GHES / CPER): >> >> etc/acpi/tables etc/hardware_errors >> ================ ========================================== >> +-----------+ >> +--------------+ | address | +-> +--------------+ >> | HEST + | registers | | | Error Status | >> + +------------+ | +---------+ | | Data Block 1 | >> | | GHES | --> | | address | --------+ | +------------+ >> | | GHES | --> | | address | ------+ | | CPER | >> | | GHES | --> | | address | ----+ | | | CPER | >> | | GHES | --> | | address | -+ | | | | CPER | >> +-+------------+ +-+---------+ | | | +-+------------+ >> | | | >> | | +---> +--------------+ >> | | | Error Status | >> | | | Data Block 2 | >> | | | +------------+ >> | | | | CPER | >> | | | | CPER | >> | | +-+------------+ >> | | >> | +-----> +--------------+ >> | | Error Status | >> | | Data Block 3 | >> | | +------------+ >> | | | CPER | >> | +-+------------+ >> | >> +--------> +--------------+ >> | Error Status | >> | Data Block 4 | >> | +------------+ >> | | CPER | >> | | CPER | >> | | CPER | >> +-+------------+ >> >> (1) QEMU generates the HEST ACPI table. This table goes in the current >> "etc/acpi/tables" fw_cfg blob. Given N error sources, there will be N >> GHES objects in the HEST. >> >> (2) We introduce a new fw_cfg blob called "etc/hardware_errors". QEMU >> also populates this blob. >> >> (2a) Given N error sources, the (unnamed) table of address registers >> will contain N address registers. >> >> (2b) Given N error sources, the "etc/hardwre_errors" fw_cfg blob will >> also contain N Error Status Data Blocks. >> >> I don't know about the sizing (number of CPERs) each Error Status Data >> Block has to contain, but I understand it is all pre-allocated as far as >> the OS is concerned, which matches our capabilities well. > here I have a question. as you comment: " 'etc/hardwre_errors' fw_cfg blob will also contain N Error Status Data Blocks", > Because the CPER numbers is not fixed, how to assign each "Error Status Data Block" size using one "etc/hardwre_errors" fw_cfg blob. > when use one etc/hardwre_errors, will the N Error Status Data Block use one continuous buffer? as shown below. if so, maybe it not convenient for each data block size extension. > I see the bios_linker_loader_alloc will allocate one continuous buffer for a blob(such as VMGENID_GUID_FW_CFG_FILE) > > /* Allocate guest memory for the Data fw_cfg blob */ > bios_linker_loader_alloc(linker, VMGENID_GUID_FW_CFG_FILE, guid, 4096, > false /* page boundary, high memory */); > > > > -> +--------------+ > | HEST + | registers | | Error Status | > + +------------+ | +---------+ | Data Block | > | | GHES | --> | | address | --------+-->| +------------+ > | | GHES | --> | | address | ------+ | | CPER | > | | GHES | --> | | address | ----+ | | | CPER | > | | GHES | --> | | address | -+ | | | | CPER | > +-+------------+ +-+---------+ | | +---> +--------------+ > | | | | CPER | > | | | | CPER | > | +-----> +--------------+ > | | | CPER | > +--------> +--------------+ > | | CPER | > | | CPER | > | | CPER | > +-+------------+ > > > > so how about we use separate etc/hardwre_errorsN for each Error Status status Block? then > > etc/hardwre_errors0 > etc/hardwre_errors1 > ................... > etc/hardwre_errors10 > (the max N is 10) > > > the N can be one of below values, according to ACPI spec "Table 18-345 Hardware Error Notification Structure" > 0 ? Polled > 1 ? External Interrupt > 2 ? Local Interrupt > 3 ? SCI > 4 ? NMI > 5 - CMCI > 6 - MCE > 7 - GPIO-Signal > 8 - ARMv8 SEA > 9 - ARMv8 SEI > 10 - External Interrupt - GSIV I'm unsure if, by "not fixed", you are saying the number of CPER entries that fits in Error Status Data Block N is not *uniform* across 0 <= N <= 10 [1] or the number of CPER entries that fits in Error Status Data Block N is not *known* in advance, for all of 0 <= N <= 10 [2] Which one is your point? If [1], that's no problem; you can simply sum the individual error status data block sizes in advance, and allocate "etc/hardware_errors" accordingly, using the total size. (Allocating one shared fw_cfg blob for all status data blocks is more memory efficient, as each ALLOCATE command will allocate whole pages (rounded up from the actual blob size).) If your point is [2], then splitting the error status data blocks to separate fw_cfg blobs makes no difference: regardless of whether we try to place all the error status data blocks in a single fw_cfg blob, or in separate fw_cfg blobs, the individual data block cannot be resized@OS runtime, so there's no way to make it work. Thanks, Laszlo > > > > >> >> (3) QEMU generates the ACPI linker/loader script for the firmware, as >> always. >> >> (3a) The HEST table is part of "etc/acpi/tables", which the firmware >> already allocates memory for, and downloads (because QEMU already >> generates an ALLOCATE linker/loader command for it already). >> >> (3b) QEMU will have to create another ALLOCATE command for the >> "etc/hardware_errors" blob. The firmware allocates memory for this blob, >> and downloads it. >> >> (4) QEMU generates, in the ACPI linker/loader script for the firwmare, N >> ADD_POINTER commands, which point the GHES."Error Status >> Address" fields in the HEST table, to the corresponding address >> registers in the downloaded "etc/hardware_errors" blob. >> >> (5) QEMU generates an ADD_CHECKSUM command for the firmware, so that the >> HEST table is correctly checksummed after executing the N ADD_POINTER >> commands from (4). >> >> (6) QEMU generates N ADD_POINTER commands for the firmware, pointing the >> address registers (located in guest memory, in the downloaded >> "etc/hardware_errors" blob) to the respective Error Status Data Blocks. >> >> (7) (This is the trick.) For this step, we need a third, write-only >> fw_cfg blob, called "etc/hardware_errors_addr". Through that blob, the >> firmware can send back the guest-side allocation addresses to QEMU. >> >> Namely, the "etc/hardware_errors_addr" blob contains N 8-byte entries. >> QEMU generates N WRITE_POINTER commands for the firmware. >> >> For error source K (0 <= K < N), QEMU instructs the firmware to >> calculate the guest address of Error Status Data Block K, from the >> QEMU-dictated offset within "etc/hardware_errors", and from the >> guest-determined allocation base address for "etc/hardware_errors". The >> firmware then writes the calculated address back to fw_cfg file >> "etc/hardware_errors_addr", at offset K*8, according to the >> WRITE_POINTER command. >> >> This way QEMU will know the GPA of each Error Status Data Block. >> >> (In fact this can be simplified to a single WRITE_POINTER command: the >> address of the "address register table" can be sent back to QEMU as >> well, which already contains all Error Status Data Block addresses.) >> >> (8) When QEMU gets SIGBUS from the kernel -- I hope that's going to come >> through a signalfd -- QEMU can format the CPER right into guest memory, >> and then inject whatever interrupt (or assert whatever GPIO line) is >> necessary for notifying the guest. >> >> (9) This notification (in virtual hardware) can either be handled by the >> guest kernel stand-alone, or else the guest kernel can invoke an ACPI >> event handler method with it (which would be in the DSDT or one of the >> SSDTs, also generated by QEMU). The ACPI event handler method could >> invoke the specific guest kernel driver for errror handling via a >> Notify() operation. >> >> I'm attracted to the above design because: >> - it would leave the firmware alone after OS boot, and >> - it would leave the firmware blissfully ignorant about HEST, GHES, >> CPER, and the like. (That's why QEMU's ACPI linker/loader was invented >> in the first place.) >> >> Thanks >> Laszlo >> >>>> Do you think which modules generates the APEI table is better? UEFI or Qemu? >>>> >>>> >>>> >>>> >>>> On 2017/3/28 21:40, James Morse wrote: >>>>> Hi gengdongjiu, >>>>> >>>>> On 28/03/17 13:16, gengdongjiu wrote: >>>>>> On 2017/3/28 19:54, Achin Gupta wrote: >>>>>>> On Tue, Mar 28, 2017 at 01:23:28PM +0200, Christoffer Dall wrote: >>>>>>>> On Tue, Mar 28, 2017 at 11:48:08AM +0100, James Morse wrote: >>>>>>>>> On the host, part of UEFI is involved to generate the CPER records. >>>>>>>>> In a guest?, I don't know. >>>>>>>>> Qemu could generate the records, or drive some other component to do it. >>>>>>>> >>>>>>>> I think I am beginning to understand this a bit. Since the guet UEFI >>>>>>>> instance is specifically built for the machine it runs on, QEMU's virt >>>>>>>> machine in this case, they could simply agree (by some contract) to >>>>>>>> place the records at some specific location in memory, and if the guest >>>>>>>> kernel asks its guest UEFI for that location, things should just work by >>>>>>>> having logic in QEMU to process error reports and populate guest memory. >>>>>>>> >>>>>>>> Is this how others see the world too? >>>>>>> >>>>>>> I think so! >>>>>>> >>>>>>> AFAIU, the memory where CPERs will reside should be specified in a GHES entry in >>>>>>> the HEST. Is this not the case with a guest kernel i.e. the guest UEFI creates a >>>>>>> HEST for the guest Kernel? >>>>>>> >>>>>>> If so, then the question is how the guest UEFI finds out where QEMU (acting as >>>>>>> EL3 firmware) will populate the CPERs. This could either be a contract between >>>>>>> the two or a guest DXE driver uses the MM_COMMUNICATE call (see [1]) to ask QEMU >>>>>>> where the memory is. >>>>>> >>>>>> whether invoke the guest UEFI will be complex? not see the advantage. it seems x86 Qemu >>>>>> directly generate the ACPI table, but I am not sure, we are checking the qemu >>>>> logical. >>>>>> let Qemu generate CPER record may be clear. >>>>> >>>>> At boot UEFI in the guest will need to make sure the areas of memory that may be >>>>> used for CPER records are reserved. Whether UEFI or Qemu decides where these are >>>>> needs deciding, (but probably not here)... >>>>> >>>>> At runtime, when an error has occurred, I agree it would be simpler (fewer >>>>> components involved) if Qemu generates the CPER records. But if UEFI made the >>>>> memory choice above they need to interact and it gets complicated again. The >>>>> CPER records are defined in the UEFI spec, so I would expect UEFI to contain >>>>> code to generate/parse them. >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> James >>>>> >>>>> >>>>> . >>>>> >>>> >> >> >> . >> >