From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22AD8C433F5 for ; Thu, 23 Sep 2021 15:11:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 050B261248 for ; Thu, 23 Sep 2021 15:11:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241931AbhIWPMi (ORCPT ); Thu, 23 Sep 2021 11:12:38 -0400 Received: from foss.arm.com ([217.140.110.172]:36254 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S241897AbhIWPMa (ORCPT ); Thu, 23 Sep 2021 11:12:30 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 51AABD6E; Thu, 23 Sep 2021 08:10:58 -0700 (PDT) Received: from [192.168.0.110] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27EAE3F718; Thu, 23 Sep 2021 08:10:57 -0700 (PDT) Message-ID: <62a2f16c-3aa0-3d7e-4b35-3a5bad701586@arm.com> Date: Thu, 23 Sep 2021 16:12:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.1 Subject: Re: [RFC PATCH v4 00/39] KVM: arm64: Add Statistical Profiling Extension (SPE) support Content-Language: en-US To: Suzuki K Poulose , maz@kernel.org, james.morse@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, will@kernel.org, linux-kernel@vger.kernel.org References: <20210825161815.266051-1-alexandru.elisei@arm.com> <963f68c8-b109-7ebb-751d-14ce46e3cdde@arm.com> From: Alexandru Elisei In-Reply-To: <963f68c8-b109-7ebb-751d-14ce46e3cdde@arm.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Suzuki, Thank you for having a look! On 9/22/21 11:11, Suzuki K Poulose wrote: > On 25/08/2021 17:17, Alexandru Elisei wrote: >> This is v4 of the SPE series posted at [1]. v2 can be found at [2], and the >> original series at [3]. >> >> Statistical Profiling Extension (SPE) is an optional feature added in >> ARMv8.2. It allows sampling at regular intervals of the operations executed >> by the PE and storing a record of each operation in a memory buffer. A high >> level overview of the extension is presented in an article on arm.com [4]. >> >> This is another complete rewrite of the series, and nothing is set in >> stone. If you think of a better way to do things, please suggest it. >> >> >> Features added >> ============== >> >> The rewrite enabled me to add support for several features not >> present in the previous iteration: >> >> - Support for heterogeneous systems, where only some of the CPUs support SPE. >>    This is accomplished via the KVM_ARM_VCPU_SUPPORTED_CPUS VCPU ioctl. >> >> - Support for VM migration with the KVM_ARM_VCPU_SPE_CTRL(KVM_ARM_VCPU_SPE_STOP) >>    VCPU ioctl. >> >> - The requirement for userspace to mlock() the guest memory has been removed, >>    and now userspace can make changes to memory contents after the memory is >>    mapped at stage 2. >> >> - Better debugging of guest memory pinning by printing a warning when we >>    get an unexpected read or write fault. This helped me catch several bugs >>    during development, it has already proven very useful. Many thanks to >>    James who suggested when reviewing v3. >> >> >> Missing features >> ================ >> >> I've tried to keep the series as small as possible to make it easier to review, >> while implementing the core functionality needed for the SPE emulation. As such, >> I've chosen to not implement several features: >> >> - Host profiling a guest which has the SPE feature bit set (see open >>    questions). >> >> - No errata workarounds have been implemented yet, and there are quite a few of >>    them for Neoverse N1 and Neoverse V1. >> >> - Disabling CONFIG_NUMA_BALANCING is a hack to get KVM SPE to work and I am >>    investigating other ways to get around automatic numa balancing, like >>    requiring userspace to disable it via set_mempolicy(). I am also going to >>    look at how VFIO gets around it. Suggestions welcome. >> >> - There's plenty of room for optimization. Off the top of my head, using >>    block mappings at stage 2, batch pinning of pages (similar to what VFIO >>    does), optimize the way KVM keeps track of pinned pages (using a linked >>    list triples the memory usage), context-switch the SPE registers on >>    vcpu_load/vcpu_put on VHE if the host is not profiling, locking >>    optimizations, etc, etc. >> >> - ...and others. I'm sure I'm missing at least a few things which are >>    important for someone. >> >> >> Known issues >> ============ >> >> This is an RFC, so keep in mind that almost definitely there will be scary >> bugs. For example, below is a list of known issues which don't affect the >> correctness of the emulation, and which I'm planning to fix in a future >> iteration: >> >> - With CONFIG_PROVE_LOCKING=y, lockdep complains about lock contention when >>    the VCPU executes the dcache clean pending ops. >> >> - With CONFIG_PROVE_LOCKING=y, KVM will hit a BUG at >>    kvm_lock_all_vcpus()->mutex_trylock(&vcpu->mutex) with more than 48 >>    VCPUs. >> >> This BUG statement can also be triggered with mainline. To reproduce it, >> compile kvmtool from this branch [5] and follow the instruction in the >> kvmtool commit message. >> >> One workaround could be to stop trying to lock all VCPUs when locking a >> memslot and document the fact that it is required that no VCPUs are run >> before the ioctl completes, otherwise bad things might happen to the VM. >> >> >> Open questions >> ============== >> >> 1. Implementing support for host profiling a guest with the SPE feature >> means setting the profiling buffer owning regime to EL2. While that is in >> effect,  PMBIDR_EL1.P will equal 1. This has two consequences: if the guest >> probes SPE during this time, the driver will fail; and the guest will be >> able to determine when it is profiled. I see two options here: > > This doesn't mean the EL2 is owning the SPE. It only tells you that a > higher level EL is owning the SPE. It could as well be EL3. (e.g, MDCR_EL3.NSPB > == 0 or 1). So I think this is architecturally correct, > as long as we trap the guest access to other SPE registers and inject > and UNDEF. You are right, I was wrong about the part about the guest being able to determine when it is profiled, I forgot that PMBIDR_EL1.P can also be 1 if EL3 is owning the buffer. But I don't understand why you are suggesting that KVM injects UNDEF in this case. I was thinking that when the host is profiling the guest, KVM can store guest writes to the SPE registers to the shadow copy of the registers. On a world switch to guest, KVM will not restore the registers onto the hardware as to not interfere with the profiling operation performed by the host. When profiling ends, KVM can then let the guest use SPE again by restoring the guest register values onto the hardware at every world switch and setting the buffer owning regime to EL1 again. I put this as an open question because if the guest sees PMBIDR_EL1.P set when the guest SPE driver probes, the driver will likely decide that SPE is not available. But the VM has been created with SPE enabled because the host userspace wants the guest to have access to SPE, and the host userspace might not realize that the act of profiling a guest can make SPE unusable by that guest. I am inclined now to let the host's userspace do whatever it wishes and allow it to profile a guest if it so desires, and mention this possible side effect in the KVM documentation, as it might be surprising for someone who isn't familiar with the inner workings of KVM's SPE emulation. Thanks, Alex > > > Thanks > Suzuki From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE075C433EF for ; Thu, 23 Sep 2021 15:11:04 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 719D761248 for ; Thu, 23 Sep 2021 15:11:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 719D761248 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id E8F8E4B0BE; Thu, 23 Sep 2021 11:11:03 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id r6UKzVojGIsx; Thu, 23 Sep 2021 11:11:01 -0400 (EDT) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 26C6A4B0E1; Thu, 23 Sep 2021 11:11:01 -0400 (EDT) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 9B8844B0D0 for ; Thu, 23 Sep 2021 11:11:00 -0400 (EDT) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 5S9UdIv+ra01 for ; Thu, 23 Sep 2021 11:10:59 -0400 (EDT) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 0B2894B0BE for ; Thu, 23 Sep 2021 11:10:59 -0400 (EDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 51AABD6E; Thu, 23 Sep 2021 08:10:58 -0700 (PDT) Received: from [192.168.0.110] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27EAE3F718; Thu, 23 Sep 2021 08:10:57 -0700 (PDT) Message-ID: <62a2f16c-3aa0-3d7e-4b35-3a5bad701586@arm.com> Date: Thu, 23 Sep 2021 16:12:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.1 Subject: Re: [RFC PATCH v4 00/39] KVM: arm64: Add Statistical Profiling Extension (SPE) support Content-Language: en-US To: Suzuki K Poulose , maz@kernel.org, james.morse@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, will@kernel.org, linux-kernel@vger.kernel.org References: <20210825161815.266051-1-alexandru.elisei@arm.com> <963f68c8-b109-7ebb-751d-14ce46e3cdde@arm.com> From: Alexandru Elisei In-Reply-To: <963f68c8-b109-7ebb-751d-14ce46e3cdde@arm.com> X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu SGkgU3V6dWtpLAoKVGhhbmsgeW91IGZvciBoYXZpbmcgYSBsb29rIQoKT24gOS8yMi8yMSAxMTox MSwgU3V6dWtpIEsgUG91bG9zZSB3cm90ZToKPiBPbiAyNS8wOC8yMDIxIDE3OjE3LCBBbGV4YW5k cnUgRWxpc2VpIHdyb3RlOgo+PiBUaGlzIGlzIHY0IG9mIHRoZSBTUEUgc2VyaWVzIHBvc3RlZCBh dCBbMV0uIHYyIGNhbiBiZSBmb3VuZCBhdCBbMl0sIGFuZCB0aGUKPj4gb3JpZ2luYWwgc2VyaWVz IGF0IFszXS4KPj4KPj4gU3RhdGlzdGljYWwgUHJvZmlsaW5nIEV4dGVuc2lvbiAoU1BFKSBpcyBh biBvcHRpb25hbCBmZWF0dXJlIGFkZGVkIGluCj4+IEFSTXY4LjIuIEl0IGFsbG93cyBzYW1wbGlu ZyBhdCByZWd1bGFyIGludGVydmFscyBvZiB0aGUgb3BlcmF0aW9ucyBleGVjdXRlZAo+PiBieSB0 aGUgUEUgYW5kIHN0b3JpbmcgYSByZWNvcmQgb2YgZWFjaCBvcGVyYXRpb24gaW4gYSBtZW1vcnkg YnVmZmVyLiBBIGhpZ2gKPj4gbGV2ZWwgb3ZlcnZpZXcgb2YgdGhlIGV4dGVuc2lvbiBpcyBwcmVz ZW50ZWQgaW4gYW4gYXJ0aWNsZSBvbiBhcm0uY29tIFs0XS4KPj4KPj4gVGhpcyBpcyBhbm90aGVy IGNvbXBsZXRlIHJld3JpdGUgb2YgdGhlIHNlcmllcywgYW5kIG5vdGhpbmcgaXMgc2V0IGluCj4+ IHN0b25lLiBJZiB5b3UgdGhpbmsgb2YgYSBiZXR0ZXIgd2F5IHRvIGRvIHRoaW5ncywgcGxlYXNl IHN1Z2dlc3QgaXQuCj4+Cj4+Cj4+IEZlYXR1cmVzIGFkZGVkCj4+ID09PT09PT09PT09PT09Cj4+ Cj4+IFRoZSByZXdyaXRlIGVuYWJsZWQgbWUgdG8gYWRkIHN1cHBvcnQgZm9yIHNldmVyYWwgZmVh dHVyZXMgbm90Cj4+IHByZXNlbnQgaW4gdGhlIHByZXZpb3VzIGl0ZXJhdGlvbjoKPj4KPj4gLSBT dXBwb3J0IGZvciBoZXRlcm9nZW5lb3VzIHN5c3RlbXMsIHdoZXJlIG9ubHkgc29tZSBvZiB0aGUg Q1BVcyBzdXBwb3J0IFNQRS4KPj4gwqDCoCBUaGlzIGlzIGFjY29tcGxpc2hlZCB2aWEgdGhlIEtW TV9BUk1fVkNQVV9TVVBQT1JURURfQ1BVUyBWQ1BVIGlvY3RsLgo+Pgo+PiAtIFN1cHBvcnQgZm9y IFZNIG1pZ3JhdGlvbiB3aXRoIHRoZSBLVk1fQVJNX1ZDUFVfU1BFX0NUUkwoS1ZNX0FSTV9WQ1BV X1NQRV9TVE9QKQo+PiDCoMKgIFZDUFUgaW9jdGwuCj4+Cj4+IC0gVGhlIHJlcXVpcmVtZW50IGZv ciB1c2Vyc3BhY2UgdG8gbWxvY2soKSB0aGUgZ3Vlc3QgbWVtb3J5IGhhcyBiZWVuIHJlbW92ZWQs Cj4+IMKgwqAgYW5kIG5vdyB1c2Vyc3BhY2UgY2FuIG1ha2UgY2hhbmdlcyB0byBtZW1vcnkgY29u dGVudHMgYWZ0ZXIgdGhlIG1lbW9yeSBpcwo+PiDCoMKgIG1hcHBlZCBhdCBzdGFnZSAyLgo+Pgo+ PiAtIEJldHRlciBkZWJ1Z2dpbmcgb2YgZ3Vlc3QgbWVtb3J5IHBpbm5pbmcgYnkgcHJpbnRpbmcg YSB3YXJuaW5nIHdoZW4gd2UKPj4gwqDCoCBnZXQgYW4gdW5leHBlY3RlZCByZWFkIG9yIHdyaXRl IGZhdWx0LiBUaGlzIGhlbHBlZCBtZSBjYXRjaCBzZXZlcmFsIGJ1Z3MKPj4gwqDCoCBkdXJpbmcg ZGV2ZWxvcG1lbnQsIGl0IGhhcyBhbHJlYWR5IHByb3ZlbiB2ZXJ5IHVzZWZ1bC4gTWFueSB0aGFu a3MgdG8KPj4gwqDCoCBKYW1lcyB3aG8gc3VnZ2VzdGVkIHdoZW4gcmV2aWV3aW5nIHYzLgo+Pgo+ Pgo+PiBNaXNzaW5nIGZlYXR1cmVzCj4+ID09PT09PT09PT09PT09PT0KPj4KPj4gSSd2ZSB0cmll ZCB0byBrZWVwIHRoZSBzZXJpZXMgYXMgc21hbGwgYXMgcG9zc2libGUgdG8gbWFrZSBpdCBlYXNp ZXIgdG8gcmV2aWV3LAo+PiB3aGlsZSBpbXBsZW1lbnRpbmcgdGhlIGNvcmUgZnVuY3Rpb25hbGl0 eSBuZWVkZWQgZm9yIHRoZSBTUEUgZW11bGF0aW9uLiBBcyBzdWNoLAo+PiBJJ3ZlIGNob3NlbiB0 byBub3QgaW1wbGVtZW50IHNldmVyYWwgZmVhdHVyZXM6Cj4+Cj4+IC0gSG9zdCBwcm9maWxpbmcg YSBndWVzdCB3aGljaCBoYXMgdGhlIFNQRSBmZWF0dXJlIGJpdCBzZXQgKHNlZSBvcGVuCj4+IMKg wqAgcXVlc3Rpb25zKS4KPj4KPj4gLSBObyBlcnJhdGEgd29ya2Fyb3VuZHMgaGF2ZSBiZWVuIGlt cGxlbWVudGVkIHlldCwgYW5kIHRoZXJlIGFyZSBxdWl0ZSBhIGZldyBvZgo+PiDCoMKgIHRoZW0g Zm9yIE5lb3ZlcnNlIE4xIGFuZCBOZW92ZXJzZSBWMS4KPj4KPj4gLSBEaXNhYmxpbmcgQ09ORklH X05VTUFfQkFMQU5DSU5HIGlzIGEgaGFjayB0byBnZXQgS1ZNIFNQRSB0byB3b3JrIGFuZCBJIGFt Cj4+IMKgwqAgaW52ZXN0aWdhdGluZyBvdGhlciB3YXlzIHRvIGdldCBhcm91bmQgYXV0b21hdGlj IG51bWEgYmFsYW5jaW5nLCBsaWtlCj4+IMKgwqAgcmVxdWlyaW5nIHVzZXJzcGFjZSB0byBkaXNh YmxlIGl0IHZpYSBzZXRfbWVtcG9saWN5KCkuIEkgYW0gYWxzbyBnb2luZyB0bwo+PiDCoMKgIGxv b2sgYXQgaG93IFZGSU8gZ2V0cyBhcm91bmQgaXQuIFN1Z2dlc3Rpb25zIHdlbGNvbWUuCj4+Cj4+ IC0gVGhlcmUncyBwbGVudHkgb2Ygcm9vbSBmb3Igb3B0aW1pemF0aW9uLiBPZmYgdGhlIHRvcCBv ZiBteSBoZWFkLCB1c2luZwo+PiDCoMKgIGJsb2NrIG1hcHBpbmdzIGF0IHN0YWdlIDIsIGJhdGNo IHBpbm5pbmcgb2YgcGFnZXMgKHNpbWlsYXIgdG8gd2hhdCBWRklPCj4+IMKgwqAgZG9lcyksIG9w dGltaXplIHRoZSB3YXkgS1ZNIGtlZXBzIHRyYWNrIG9mIHBpbm5lZCBwYWdlcyAodXNpbmcgYSBs aW5rZWQKPj4gwqDCoCBsaXN0IHRyaXBsZXMgdGhlIG1lbW9yeSB1c2FnZSksIGNvbnRleHQtc3dp dGNoIHRoZSBTUEUgcmVnaXN0ZXJzIG9uCj4+IMKgwqAgdmNwdV9sb2FkL3ZjcHVfcHV0IG9uIFZI RSBpZiB0aGUgaG9zdCBpcyBub3QgcHJvZmlsaW5nLCBsb2NraW5nCj4+IMKgwqAgb3B0aW1pemF0 aW9ucywgZXRjLCBldGMuCj4+Cj4+IC0gLi4uYW5kIG90aGVycy4gSSdtIHN1cmUgSSdtIG1pc3Np bmcgYXQgbGVhc3QgYSBmZXcgdGhpbmdzIHdoaWNoIGFyZQo+PiDCoMKgIGltcG9ydGFudCBmb3Ig c29tZW9uZS4KPj4KPj4KPj4gS25vd24gaXNzdWVzCj4+ID09PT09PT09PT09PQo+Pgo+PiBUaGlz IGlzIGFuIFJGQywgc28ga2VlcCBpbiBtaW5kIHRoYXQgYWxtb3N0IGRlZmluaXRlbHkgdGhlcmUg d2lsbCBiZSBzY2FyeQo+PiBidWdzLiBGb3IgZXhhbXBsZSwgYmVsb3cgaXMgYSBsaXN0IG9mIGtu b3duIGlzc3VlcyB3aGljaCBkb24ndCBhZmZlY3QgdGhlCj4+IGNvcnJlY3RuZXNzIG9mIHRoZSBl bXVsYXRpb24sIGFuZCB3aGljaCBJJ20gcGxhbm5pbmcgdG8gZml4IGluIGEgZnV0dXJlCj4+IGl0 ZXJhdGlvbjoKPj4KPj4gLSBXaXRoIENPTkZJR19QUk9WRV9MT0NLSU5HPXksIGxvY2tkZXAgY29t cGxhaW5zIGFib3V0IGxvY2sgY29udGVudGlvbiB3aGVuCj4+IMKgwqAgdGhlIFZDUFUgZXhlY3V0 ZXMgdGhlIGRjYWNoZSBjbGVhbiBwZW5kaW5nIG9wcy4KPj4KPj4gLSBXaXRoIENPTkZJR19QUk9W RV9MT0NLSU5HPXksIEtWTSB3aWxsIGhpdCBhIEJVRyBhdAo+PiDCoMKgIGt2bV9sb2NrX2FsbF92 Y3B1cygpLT5tdXRleF90cnlsb2NrKCZ2Y3B1LT5tdXRleCkgd2l0aCBtb3JlIHRoYW4gNDgKPj4g wqDCoCBWQ1BVcy4KPj4KPj4gVGhpcyBCVUcgc3RhdGVtZW50IGNhbiBhbHNvIGJlIHRyaWdnZXJl ZCB3aXRoIG1haW5saW5lLiBUbyByZXByb2R1Y2UgaXQsCj4+IGNvbXBpbGUga3ZtdG9vbCBmcm9t IHRoaXMgYnJhbmNoIFs1XSBhbmQgZm9sbG93IHRoZSBpbnN0cnVjdGlvbiBpbiB0aGUKPj4ga3Zt dG9vbCBjb21taXQgbWVzc2FnZS4KPj4KPj4gT25lIHdvcmthcm91bmQgY291bGQgYmUgdG8gc3Rv cCB0cnlpbmcgdG8gbG9jayBhbGwgVkNQVXMgd2hlbiBsb2NraW5nIGEKPj4gbWVtc2xvdCBhbmQg ZG9jdW1lbnQgdGhlIGZhY3QgdGhhdCBpdCBpcyByZXF1aXJlZCB0aGF0IG5vIFZDUFVzIGFyZSBy dW4KPj4gYmVmb3JlIHRoZSBpb2N0bCBjb21wbGV0ZXMsIG90aGVyd2lzZSBiYWQgdGhpbmdzIG1p Z2h0IGhhcHBlbiB0byB0aGUgVk0uCj4+Cj4+Cj4+IE9wZW4gcXVlc3Rpb25zCj4+ID09PT09PT09 PT09PT09Cj4+Cj4+IDEuIEltcGxlbWVudGluZyBzdXBwb3J0IGZvciBob3N0IHByb2ZpbGluZyBh IGd1ZXN0IHdpdGggdGhlIFNQRSBmZWF0dXJlCj4+IG1lYW5zIHNldHRpbmcgdGhlIHByb2ZpbGlu ZyBidWZmZXIgb3duaW5nIHJlZ2ltZSB0byBFTDIuIFdoaWxlIHRoYXQgaXMgaW4KPj4gZWZmZWN0 LMKgIFBNQklEUl9FTDEuUCB3aWxsIGVxdWFsIDEuIFRoaXMgaGFzIHR3byBjb25zZXF1ZW5jZXM6 IGlmIHRoZSBndWVzdAo+PiBwcm9iZXMgU1BFIGR1cmluZyB0aGlzIHRpbWUsIHRoZSBkcml2ZXIg d2lsbCBmYWlsOyBhbmQgdGhlIGd1ZXN0IHdpbGwgYmUKPj4gYWJsZSB0byBkZXRlcm1pbmUgd2hl biBpdCBpcyBwcm9maWxlZC4gSSBzZWUgdHdvIG9wdGlvbnMgaGVyZToKPgo+IFRoaXMgZG9lc24n dCBtZWFuIHRoZSBFTDIgaXMgb3duaW5nIHRoZSBTUEUuIEl0IG9ubHkgdGVsbHMgeW91IHRoYXQg YQo+IGhpZ2hlciBsZXZlbCBFTCBpcyBvd25pbmcgdGhlIFNQRS4gSXQgY291bGQgYXMgd2VsbCBi ZSBFTDMuIChlLmcsIE1EQ1JfRUwzLk5TUEIKPiA9PSAwIG9yIDEpLiBTbyBJIHRoaW5rIHRoaXMg aXMgYXJjaGl0ZWN0dXJhbGx5IGNvcnJlY3QsCj4gYXMgbG9uZyBhcyB3ZSB0cmFwIHRoZSBndWVz dCBhY2Nlc3MgdG8gb3RoZXIgU1BFIHJlZ2lzdGVycyBhbmQgaW5qZWN0Cj4gYW5kIFVOREVGLgoK WW91IGFyZSByaWdodCwgSSB3YXMgd3JvbmcgYWJvdXQgdGhlIHBhcnQgYWJvdXQgdGhlIGd1ZXN0 IGJlaW5nIGFibGUgdG8gZGV0ZXJtaW5lCndoZW4gaXQgaXMgcHJvZmlsZWQsIEkgZm9yZ290IHRo YXQgUE1CSURSX0VMMS5QIGNhbiBhbHNvIGJlIDEgaWYgRUwzIGlzIG93bmluZyB0aGUKYnVmZmVy LgoKQnV0IEkgZG9uJ3QgdW5kZXJzdGFuZCB3aHkgeW91IGFyZSBzdWdnZXN0aW5nIHRoYXQgS1ZN IGluamVjdHMgVU5ERUYgaW4gdGhpcyBjYXNlLgpJIHdhcyB0aGlua2luZyB0aGF0IHdoZW4gdGhl IGhvc3QgaXMgcHJvZmlsaW5nIHRoZSBndWVzdCwgS1ZNIGNhbiBzdG9yZSBndWVzdAp3cml0ZXMg dG8gdGhlIFNQRSByZWdpc3RlcnMgdG8gdGhlIHNoYWRvdyBjb3B5IG9mIHRoZSByZWdpc3RlcnMu IE9uIGEgd29ybGQgc3dpdGNoCnRvIGd1ZXN0LCBLVk0gd2lsbCBub3QgcmVzdG9yZSB0aGUgcmVn aXN0ZXJzIG9udG8gdGhlIGhhcmR3YXJlIGFzIHRvIG5vdCBpbnRlcmZlcmUKd2l0aCB0aGUgcHJv ZmlsaW5nIG9wZXJhdGlvbiBwZXJmb3JtZWQgYnkgdGhlIGhvc3QuIFdoZW4gcHJvZmlsaW5nIGVu ZHMsIEtWTSBjYW4KdGhlbiBsZXQgdGhlIGd1ZXN0IHVzZSBTUEUgYWdhaW4gYnkgcmVzdG9yaW5n IHRoZSBndWVzdCByZWdpc3RlciB2YWx1ZXMgb250byB0aGUKaGFyZHdhcmUgYXQgZXZlcnkgd29y bGQgc3dpdGNoIGFuZCBzZXR0aW5nIHRoZSBidWZmZXIgb3duaW5nIHJlZ2ltZSB0byBFTDEgYWdh aW4uCgpJIHB1dCB0aGlzIGFzIGFuIG9wZW4gcXVlc3Rpb24gYmVjYXVzZSBpZiB0aGUgZ3Vlc3Qg c2VlcyBQTUJJRFJfRUwxLlAgc2V0IHdoZW4gdGhlCmd1ZXN0IFNQRSBkcml2ZXIgcHJvYmVzLCB0 aGUgZHJpdmVyIHdpbGwgbGlrZWx5IGRlY2lkZSB0aGF0IFNQRSBpcyBub3QgYXZhaWxhYmxlLgpC dXQgdGhlIFZNIGhhcyBiZWVuIGNyZWF0ZWQgd2l0aCBTUEUgZW5hYmxlZCBiZWNhdXNlIHRoZSBo b3N0IHVzZXJzcGFjZSB3YW50cyB0aGUKZ3Vlc3QgdG8gaGF2ZSBhY2Nlc3MgdG8gU1BFLCBhbmQg dGhlIGhvc3QgdXNlcnNwYWNlIG1pZ2h0IG5vdCByZWFsaXplIHRoYXQgdGhlIGFjdApvZiBwcm9m aWxpbmcgYSBndWVzdCBjYW4gbWFrZSBTUEUgdW51c2FibGUgYnkgdGhhdCBndWVzdC4gSSBhbSBp bmNsaW5lZCBub3cgdG8gbGV0CnRoZSBob3N0J3MgdXNlcnNwYWNlIGRvIHdoYXRldmVyIGl0IHdp c2hlcyBhbmQgYWxsb3cgaXQgdG8gcHJvZmlsZSBhIGd1ZXN0IGlmIGl0CnNvIGRlc2lyZXMsIGFu ZCBtZW50aW9uIHRoaXMgcG9zc2libGUgc2lkZSBlZmZlY3QgaW4gdGhlIEtWTSBkb2N1bWVudGF0 aW9uLCBhcyBpdAptaWdodCBiZSBzdXJwcmlzaW5nIGZvciBzb21lb25lIHdobyBpc24ndCBmYW1p bGlhciB3aXRoIHRoZSBpbm5lciB3b3JraW5ncyBvZgpLVk0ncyBTUEUgZW11bGF0aW9uLgoKVGhh bmtzLAoKQWxleAoKPgo+Cj4gVGhhbmtzCj4gU3V6dWtpCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmt2bWFybSBtYWlsaW5nIGxpc3QKa3ZtYXJtQGxpc3Rz LmNzLmNvbHVtYmlhLmVkdQpodHRwczovL2xpc3RzLmNzLmNvbHVtYmlhLmVkdS9tYWlsbWFuL2xp c3RpbmZvL2t2bWFybQo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 74004C433F5 for ; Thu, 23 Sep 2021 15:13:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3799C61029 for ; Thu, 23 Sep 2021 15:13:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 3799C61029 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:From:References:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Cc:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=L7YZH59gs9LySajagzXe/1L6kLlMLYY6E2Fxuz5Ehys=; b=W6l+6fo0kFPkkv OeloL+m7IluHERkwKg+OGaIW022p1rdd3UJa7UMItkjFLGKtfNre4HLYyB1RLb00g4CoHWQIKVMvs cE6xHY2TVFGLzlE77sjfVyoMxmkM0pGhKLSzK4uIdnKEYfuY+0/di1gZR17OhF+jiymV6VHt3dIQo FYoayAz494VX0MZBO1O1NZQurzUKUxY6UciX3fG0NRf7vsFzbcJsb1Tj6zmZXrcw6JXughh3bbZS0 fwQuSgsvhB5NzK0pa/q7WQhqq4oOW4u1ufhGe1EfVIkiRFf99qanQoUpoKCsJEXnbaTpMXQI101Ad zBUWz290mdzbnDY4cFuw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTQNO-00C1sA-3u; Thu, 23 Sep 2021 15:11:06 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mTQNI-00C1qT-ST for linux-arm-kernel@lists.infradead.org; Thu, 23 Sep 2021 15:11:02 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 51AABD6E; Thu, 23 Sep 2021 08:10:58 -0700 (PDT) Received: from [192.168.0.110] (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 27EAE3F718; Thu, 23 Sep 2021 08:10:57 -0700 (PDT) Message-ID: <62a2f16c-3aa0-3d7e-4b35-3a5bad701586@arm.com> Date: Thu, 23 Sep 2021 16:12:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.1 Subject: Re: [RFC PATCH v4 00/39] KVM: arm64: Add Statistical Profiling Extension (SPE) support Content-Language: en-US To: Suzuki K Poulose , maz@kernel.org, james.morse@arm.com, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, will@kernel.org, linux-kernel@vger.kernel.org References: <20210825161815.266051-1-alexandru.elisei@arm.com> <963f68c8-b109-7ebb-751d-14ce46e3cdde@arm.com> From: Alexandru Elisei In-Reply-To: <963f68c8-b109-7ebb-751d-14ce46e3cdde@arm.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210923_081101_065319_050CB1D1 X-CRM114-Status: GOOD ( 38.04 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgU3V6dWtpLAoKVGhhbmsgeW91IGZvciBoYXZpbmcgYSBsb29rIQoKT24gOS8yMi8yMSAxMTox MSwgU3V6dWtpIEsgUG91bG9zZSB3cm90ZToKPiBPbiAyNS8wOC8yMDIxIDE3OjE3LCBBbGV4YW5k cnUgRWxpc2VpIHdyb3RlOgo+PiBUaGlzIGlzIHY0IG9mIHRoZSBTUEUgc2VyaWVzIHBvc3RlZCBh dCBbMV0uIHYyIGNhbiBiZSBmb3VuZCBhdCBbMl0sIGFuZCB0aGUKPj4gb3JpZ2luYWwgc2VyaWVz IGF0IFszXS4KPj4KPj4gU3RhdGlzdGljYWwgUHJvZmlsaW5nIEV4dGVuc2lvbiAoU1BFKSBpcyBh biBvcHRpb25hbCBmZWF0dXJlIGFkZGVkIGluCj4+IEFSTXY4LjIuIEl0IGFsbG93cyBzYW1wbGlu ZyBhdCByZWd1bGFyIGludGVydmFscyBvZiB0aGUgb3BlcmF0aW9ucyBleGVjdXRlZAo+PiBieSB0 aGUgUEUgYW5kIHN0b3JpbmcgYSByZWNvcmQgb2YgZWFjaCBvcGVyYXRpb24gaW4gYSBtZW1vcnkg YnVmZmVyLiBBIGhpZ2gKPj4gbGV2ZWwgb3ZlcnZpZXcgb2YgdGhlIGV4dGVuc2lvbiBpcyBwcmVz ZW50ZWQgaW4gYW4gYXJ0aWNsZSBvbiBhcm0uY29tIFs0XS4KPj4KPj4gVGhpcyBpcyBhbm90aGVy IGNvbXBsZXRlIHJld3JpdGUgb2YgdGhlIHNlcmllcywgYW5kIG5vdGhpbmcgaXMgc2V0IGluCj4+ IHN0b25lLiBJZiB5b3UgdGhpbmsgb2YgYSBiZXR0ZXIgd2F5IHRvIGRvIHRoaW5ncywgcGxlYXNl IHN1Z2dlc3QgaXQuCj4+Cj4+Cj4+IEZlYXR1cmVzIGFkZGVkCj4+ID09PT09PT09PT09PT09Cj4+ Cj4+IFRoZSByZXdyaXRlIGVuYWJsZWQgbWUgdG8gYWRkIHN1cHBvcnQgZm9yIHNldmVyYWwgZmVh dHVyZXMgbm90Cj4+IHByZXNlbnQgaW4gdGhlIHByZXZpb3VzIGl0ZXJhdGlvbjoKPj4KPj4gLSBT dXBwb3J0IGZvciBoZXRlcm9nZW5lb3VzIHN5c3RlbXMsIHdoZXJlIG9ubHkgc29tZSBvZiB0aGUg Q1BVcyBzdXBwb3J0IFNQRS4KPj4gwqDCoCBUaGlzIGlzIGFjY29tcGxpc2hlZCB2aWEgdGhlIEtW TV9BUk1fVkNQVV9TVVBQT1JURURfQ1BVUyBWQ1BVIGlvY3RsLgo+Pgo+PiAtIFN1cHBvcnQgZm9y IFZNIG1pZ3JhdGlvbiB3aXRoIHRoZSBLVk1fQVJNX1ZDUFVfU1BFX0NUUkwoS1ZNX0FSTV9WQ1BV X1NQRV9TVE9QKQo+PiDCoMKgIFZDUFUgaW9jdGwuCj4+Cj4+IC0gVGhlIHJlcXVpcmVtZW50IGZv ciB1c2Vyc3BhY2UgdG8gbWxvY2soKSB0aGUgZ3Vlc3QgbWVtb3J5IGhhcyBiZWVuIHJlbW92ZWQs Cj4+IMKgwqAgYW5kIG5vdyB1c2Vyc3BhY2UgY2FuIG1ha2UgY2hhbmdlcyB0byBtZW1vcnkgY29u dGVudHMgYWZ0ZXIgdGhlIG1lbW9yeSBpcwo+PiDCoMKgIG1hcHBlZCBhdCBzdGFnZSAyLgo+Pgo+ PiAtIEJldHRlciBkZWJ1Z2dpbmcgb2YgZ3Vlc3QgbWVtb3J5IHBpbm5pbmcgYnkgcHJpbnRpbmcg YSB3YXJuaW5nIHdoZW4gd2UKPj4gwqDCoCBnZXQgYW4gdW5leHBlY3RlZCByZWFkIG9yIHdyaXRl IGZhdWx0LiBUaGlzIGhlbHBlZCBtZSBjYXRjaCBzZXZlcmFsIGJ1Z3MKPj4gwqDCoCBkdXJpbmcg ZGV2ZWxvcG1lbnQsIGl0IGhhcyBhbHJlYWR5IHByb3ZlbiB2ZXJ5IHVzZWZ1bC4gTWFueSB0aGFu a3MgdG8KPj4gwqDCoCBKYW1lcyB3aG8gc3VnZ2VzdGVkIHdoZW4gcmV2aWV3aW5nIHYzLgo+Pgo+ Pgo+PiBNaXNzaW5nIGZlYXR1cmVzCj4+ID09PT09PT09PT09PT09PT0KPj4KPj4gSSd2ZSB0cmll ZCB0byBrZWVwIHRoZSBzZXJpZXMgYXMgc21hbGwgYXMgcG9zc2libGUgdG8gbWFrZSBpdCBlYXNp ZXIgdG8gcmV2aWV3LAo+PiB3aGlsZSBpbXBsZW1lbnRpbmcgdGhlIGNvcmUgZnVuY3Rpb25hbGl0 eSBuZWVkZWQgZm9yIHRoZSBTUEUgZW11bGF0aW9uLiBBcyBzdWNoLAo+PiBJJ3ZlIGNob3NlbiB0 byBub3QgaW1wbGVtZW50IHNldmVyYWwgZmVhdHVyZXM6Cj4+Cj4+IC0gSG9zdCBwcm9maWxpbmcg YSBndWVzdCB3aGljaCBoYXMgdGhlIFNQRSBmZWF0dXJlIGJpdCBzZXQgKHNlZSBvcGVuCj4+IMKg wqAgcXVlc3Rpb25zKS4KPj4KPj4gLSBObyBlcnJhdGEgd29ya2Fyb3VuZHMgaGF2ZSBiZWVuIGlt cGxlbWVudGVkIHlldCwgYW5kIHRoZXJlIGFyZSBxdWl0ZSBhIGZldyBvZgo+PiDCoMKgIHRoZW0g Zm9yIE5lb3ZlcnNlIE4xIGFuZCBOZW92ZXJzZSBWMS4KPj4KPj4gLSBEaXNhYmxpbmcgQ09ORklH X05VTUFfQkFMQU5DSU5HIGlzIGEgaGFjayB0byBnZXQgS1ZNIFNQRSB0byB3b3JrIGFuZCBJIGFt Cj4+IMKgwqAgaW52ZXN0aWdhdGluZyBvdGhlciB3YXlzIHRvIGdldCBhcm91bmQgYXV0b21hdGlj IG51bWEgYmFsYW5jaW5nLCBsaWtlCj4+IMKgwqAgcmVxdWlyaW5nIHVzZXJzcGFjZSB0byBkaXNh YmxlIGl0IHZpYSBzZXRfbWVtcG9saWN5KCkuIEkgYW0gYWxzbyBnb2luZyB0bwo+PiDCoMKgIGxv b2sgYXQgaG93IFZGSU8gZ2V0cyBhcm91bmQgaXQuIFN1Z2dlc3Rpb25zIHdlbGNvbWUuCj4+Cj4+ IC0gVGhlcmUncyBwbGVudHkgb2Ygcm9vbSBmb3Igb3B0aW1pemF0aW9uLiBPZmYgdGhlIHRvcCBv ZiBteSBoZWFkLCB1c2luZwo+PiDCoMKgIGJsb2NrIG1hcHBpbmdzIGF0IHN0YWdlIDIsIGJhdGNo IHBpbm5pbmcgb2YgcGFnZXMgKHNpbWlsYXIgdG8gd2hhdCBWRklPCj4+IMKgwqAgZG9lcyksIG9w dGltaXplIHRoZSB3YXkgS1ZNIGtlZXBzIHRyYWNrIG9mIHBpbm5lZCBwYWdlcyAodXNpbmcgYSBs aW5rZWQKPj4gwqDCoCBsaXN0IHRyaXBsZXMgdGhlIG1lbW9yeSB1c2FnZSksIGNvbnRleHQtc3dp dGNoIHRoZSBTUEUgcmVnaXN0ZXJzIG9uCj4+IMKgwqAgdmNwdV9sb2FkL3ZjcHVfcHV0IG9uIFZI RSBpZiB0aGUgaG9zdCBpcyBub3QgcHJvZmlsaW5nLCBsb2NraW5nCj4+IMKgwqAgb3B0aW1pemF0 aW9ucywgZXRjLCBldGMuCj4+Cj4+IC0gLi4uYW5kIG90aGVycy4gSSdtIHN1cmUgSSdtIG1pc3Np bmcgYXQgbGVhc3QgYSBmZXcgdGhpbmdzIHdoaWNoIGFyZQo+PiDCoMKgIGltcG9ydGFudCBmb3Ig c29tZW9uZS4KPj4KPj4KPj4gS25vd24gaXNzdWVzCj4+ID09PT09PT09PT09PQo+Pgo+PiBUaGlz IGlzIGFuIFJGQywgc28ga2VlcCBpbiBtaW5kIHRoYXQgYWxtb3N0IGRlZmluaXRlbHkgdGhlcmUg d2lsbCBiZSBzY2FyeQo+PiBidWdzLiBGb3IgZXhhbXBsZSwgYmVsb3cgaXMgYSBsaXN0IG9mIGtu b3duIGlzc3VlcyB3aGljaCBkb24ndCBhZmZlY3QgdGhlCj4+IGNvcnJlY3RuZXNzIG9mIHRoZSBl bXVsYXRpb24sIGFuZCB3aGljaCBJJ20gcGxhbm5pbmcgdG8gZml4IGluIGEgZnV0dXJlCj4+IGl0 ZXJhdGlvbjoKPj4KPj4gLSBXaXRoIENPTkZJR19QUk9WRV9MT0NLSU5HPXksIGxvY2tkZXAgY29t cGxhaW5zIGFib3V0IGxvY2sgY29udGVudGlvbiB3aGVuCj4+IMKgwqAgdGhlIFZDUFUgZXhlY3V0 ZXMgdGhlIGRjYWNoZSBjbGVhbiBwZW5kaW5nIG9wcy4KPj4KPj4gLSBXaXRoIENPTkZJR19QUk9W RV9MT0NLSU5HPXksIEtWTSB3aWxsIGhpdCBhIEJVRyBhdAo+PiDCoMKgIGt2bV9sb2NrX2FsbF92 Y3B1cygpLT5tdXRleF90cnlsb2NrKCZ2Y3B1LT5tdXRleCkgd2l0aCBtb3JlIHRoYW4gNDgKPj4g wqDCoCBWQ1BVcy4KPj4KPj4gVGhpcyBCVUcgc3RhdGVtZW50IGNhbiBhbHNvIGJlIHRyaWdnZXJl ZCB3aXRoIG1haW5saW5lLiBUbyByZXByb2R1Y2UgaXQsCj4+IGNvbXBpbGUga3ZtdG9vbCBmcm9t IHRoaXMgYnJhbmNoIFs1XSBhbmQgZm9sbG93IHRoZSBpbnN0cnVjdGlvbiBpbiB0aGUKPj4ga3Zt dG9vbCBjb21taXQgbWVzc2FnZS4KPj4KPj4gT25lIHdvcmthcm91bmQgY291bGQgYmUgdG8gc3Rv cCB0cnlpbmcgdG8gbG9jayBhbGwgVkNQVXMgd2hlbiBsb2NraW5nIGEKPj4gbWVtc2xvdCBhbmQg ZG9jdW1lbnQgdGhlIGZhY3QgdGhhdCBpdCBpcyByZXF1aXJlZCB0aGF0IG5vIFZDUFVzIGFyZSBy dW4KPj4gYmVmb3JlIHRoZSBpb2N0bCBjb21wbGV0ZXMsIG90aGVyd2lzZSBiYWQgdGhpbmdzIG1p Z2h0IGhhcHBlbiB0byB0aGUgVk0uCj4+Cj4+Cj4+IE9wZW4gcXVlc3Rpb25zCj4+ID09PT09PT09 PT09PT09Cj4+Cj4+IDEuIEltcGxlbWVudGluZyBzdXBwb3J0IGZvciBob3N0IHByb2ZpbGluZyBh IGd1ZXN0IHdpdGggdGhlIFNQRSBmZWF0dXJlCj4+IG1lYW5zIHNldHRpbmcgdGhlIHByb2ZpbGlu ZyBidWZmZXIgb3duaW5nIHJlZ2ltZSB0byBFTDIuIFdoaWxlIHRoYXQgaXMgaW4KPj4gZWZmZWN0 LMKgIFBNQklEUl9FTDEuUCB3aWxsIGVxdWFsIDEuIFRoaXMgaGFzIHR3byBjb25zZXF1ZW5jZXM6 IGlmIHRoZSBndWVzdAo+PiBwcm9iZXMgU1BFIGR1cmluZyB0aGlzIHRpbWUsIHRoZSBkcml2ZXIg d2lsbCBmYWlsOyBhbmQgdGhlIGd1ZXN0IHdpbGwgYmUKPj4gYWJsZSB0byBkZXRlcm1pbmUgd2hl biBpdCBpcyBwcm9maWxlZC4gSSBzZWUgdHdvIG9wdGlvbnMgaGVyZToKPgo+IFRoaXMgZG9lc24n dCBtZWFuIHRoZSBFTDIgaXMgb3duaW5nIHRoZSBTUEUuIEl0IG9ubHkgdGVsbHMgeW91IHRoYXQg YQo+IGhpZ2hlciBsZXZlbCBFTCBpcyBvd25pbmcgdGhlIFNQRS4gSXQgY291bGQgYXMgd2VsbCBi ZSBFTDMuIChlLmcsIE1EQ1JfRUwzLk5TUEIKPiA9PSAwIG9yIDEpLiBTbyBJIHRoaW5rIHRoaXMg aXMgYXJjaGl0ZWN0dXJhbGx5IGNvcnJlY3QsCj4gYXMgbG9uZyBhcyB3ZSB0cmFwIHRoZSBndWVz dCBhY2Nlc3MgdG8gb3RoZXIgU1BFIHJlZ2lzdGVycyBhbmQgaW5qZWN0Cj4gYW5kIFVOREVGLgoK WW91IGFyZSByaWdodCwgSSB3YXMgd3JvbmcgYWJvdXQgdGhlIHBhcnQgYWJvdXQgdGhlIGd1ZXN0 IGJlaW5nIGFibGUgdG8gZGV0ZXJtaW5lCndoZW4gaXQgaXMgcHJvZmlsZWQsIEkgZm9yZ290IHRo YXQgUE1CSURSX0VMMS5QIGNhbiBhbHNvIGJlIDEgaWYgRUwzIGlzIG93bmluZyB0aGUKYnVmZmVy LgoKQnV0IEkgZG9uJ3QgdW5kZXJzdGFuZCB3aHkgeW91IGFyZSBzdWdnZXN0aW5nIHRoYXQgS1ZN IGluamVjdHMgVU5ERUYgaW4gdGhpcyBjYXNlLgpJIHdhcyB0aGlua2luZyB0aGF0IHdoZW4gdGhl IGhvc3QgaXMgcHJvZmlsaW5nIHRoZSBndWVzdCwgS1ZNIGNhbiBzdG9yZSBndWVzdAp3cml0ZXMg dG8gdGhlIFNQRSByZWdpc3RlcnMgdG8gdGhlIHNoYWRvdyBjb3B5IG9mIHRoZSByZWdpc3RlcnMu IE9uIGEgd29ybGQgc3dpdGNoCnRvIGd1ZXN0LCBLVk0gd2lsbCBub3QgcmVzdG9yZSB0aGUgcmVn aXN0ZXJzIG9udG8gdGhlIGhhcmR3YXJlIGFzIHRvIG5vdCBpbnRlcmZlcmUKd2l0aCB0aGUgcHJv ZmlsaW5nIG9wZXJhdGlvbiBwZXJmb3JtZWQgYnkgdGhlIGhvc3QuIFdoZW4gcHJvZmlsaW5nIGVu ZHMsIEtWTSBjYW4KdGhlbiBsZXQgdGhlIGd1ZXN0IHVzZSBTUEUgYWdhaW4gYnkgcmVzdG9yaW5n IHRoZSBndWVzdCByZWdpc3RlciB2YWx1ZXMgb250byB0aGUKaGFyZHdhcmUgYXQgZXZlcnkgd29y bGQgc3dpdGNoIGFuZCBzZXR0aW5nIHRoZSBidWZmZXIgb3duaW5nIHJlZ2ltZSB0byBFTDEgYWdh aW4uCgpJIHB1dCB0aGlzIGFzIGFuIG9wZW4gcXVlc3Rpb24gYmVjYXVzZSBpZiB0aGUgZ3Vlc3Qg c2VlcyBQTUJJRFJfRUwxLlAgc2V0IHdoZW4gdGhlCmd1ZXN0IFNQRSBkcml2ZXIgcHJvYmVzLCB0 aGUgZHJpdmVyIHdpbGwgbGlrZWx5IGRlY2lkZSB0aGF0IFNQRSBpcyBub3QgYXZhaWxhYmxlLgpC dXQgdGhlIFZNIGhhcyBiZWVuIGNyZWF0ZWQgd2l0aCBTUEUgZW5hYmxlZCBiZWNhdXNlIHRoZSBo b3N0IHVzZXJzcGFjZSB3YW50cyB0aGUKZ3Vlc3QgdG8gaGF2ZSBhY2Nlc3MgdG8gU1BFLCBhbmQg dGhlIGhvc3QgdXNlcnNwYWNlIG1pZ2h0IG5vdCByZWFsaXplIHRoYXQgdGhlIGFjdApvZiBwcm9m aWxpbmcgYSBndWVzdCBjYW4gbWFrZSBTUEUgdW51c2FibGUgYnkgdGhhdCBndWVzdC4gSSBhbSBp bmNsaW5lZCBub3cgdG8gbGV0CnRoZSBob3N0J3MgdXNlcnNwYWNlIGRvIHdoYXRldmVyIGl0IHdp c2hlcyBhbmQgYWxsb3cgaXQgdG8gcHJvZmlsZSBhIGd1ZXN0IGlmIGl0CnNvIGRlc2lyZXMsIGFu ZCBtZW50aW9uIHRoaXMgcG9zc2libGUgc2lkZSBlZmZlY3QgaW4gdGhlIEtWTSBkb2N1bWVudGF0 aW9uLCBhcyBpdAptaWdodCBiZSBzdXJwcmlzaW5nIGZvciBzb21lb25lIHdobyBpc24ndCBmYW1p bGlhciB3aXRoIHRoZSBpbm5lciB3b3JraW5ncyBvZgpLVk0ncyBTUEUgZW11bGF0aW9uLgoKVGhh bmtzLAoKQWxleAoKPgo+Cj4gVGhhbmtzCj4gU3V6dWtpCgpfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdAps aW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVh ZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0ta2VybmVsCg==