From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7572FC64E7B for ; Tue, 1 Dec 2020 15:09:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 186EC2076C for ; Tue, 1 Dec 2020 15:09:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391781AbgLAPJJ (ORCPT ); Tue, 1 Dec 2020 10:09:09 -0500 Received: from foss.arm.com ([217.140.110.172]:44684 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2389268AbgLAPJJ (ORCPT ); Tue, 1 Dec 2020 10:09:09 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 882FF30E; Tue, 1 Dec 2020 07:08:23 -0800 (PST) Received: from [10.37.12.21] (unknown [10.37.12.21]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9EBE73F575; Tue, 1 Dec 2020 07:08:22 -0800 (PST) Subject: Re: [kvm-unit-tests PATCH 10/10] arm64: gic: Use IPI test checking for the LPI tests From: Alexandru Elisei To: Zenghui Yu , kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, drjones@redhat.com Cc: andre.przywara@arm.com References: <20201125155113.192079-1-alexandru.elisei@arm.com> <20201125155113.192079-11-alexandru.elisei@arm.com> <49be46a8-2c29-b805-366e-7c955d395874@huawei.com> Message-ID: <1dc913fd-80f5-4c8b-9c4d-52a1f0eee400@arm.com> Date: Tue, 1 Dec 2020 15:09:44 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Hi, On 11/30/20 2:19 PM, Alexandru Elisei wrote: > Hi Zenghui, > > On 11/30/20 1:59 PM, Zenghui Yu wrote: >> Hi Alex, >> >> On 2020/11/27 22:50, Alexandru Elisei wrote: >>> Hi Zhenghui, >>> >>> Thank you for having a look at this! >>> >>> On 11/26/20 9:30 AM, Zenghui Yu wrote: >>>> On 2020/11/25 23:51, Alexandru Elisei wrote: >>>>> The reason for the failure is that the test "dev2/eventid=20 now triggers >>>>> an LPI" triggers 2 LPIs, not one. This behavior was present before this >>>>> patch, but it was ignored because check_lpi_stats() wasn't looking at the >>>>> acked array. >>>>> >>>>> I'm not familiar with the ITS so I'm not sure if this is expected, if the >>>>> test is incorrect or if there is something wrong with KVM emulation. >>>> I think this is expected, or not. >>>> >>>> Before INVALL, the LPI-8195 was already pending but disabled. On >>>> receiving INVALL, VGIC will reload configuration for all LPIs targeting >>>> collection-3 and deliver the now enabled LPI-8195. We'll therefore see >>>> and handle it before sending the following INT (which will set the >>>> LPI-8195 pending again). >>>> >>>>> Did some more testing on an Ampere eMAG (fast out-of-order cores) using >>>>> qemu and kvmtool and Linux v5.8, here's what I found: >>>>> >>>>> - Using qemu and gic.flat built from*master*: error encountered 864 times >>>>>     out of 1088 runs. >>>>> - Using qemu: error encountered 852 times out of 1027 runs. >>>>> - Using kvmtool: error encountered 8164 times out of 10602 runs. >>>> If vcpu-3 hadn't seen and handled LPI-8195 as quickly as possible (e.g., >>>> vcpu-3 hadn't been scheduled), the following INT will set the already >>>> pending LPI-8195 pending again and we'll receive it *once* on vcpu-3. >>>> And we won't see the mentioned failure. >>>> >>>> I think we can just drop the (meaningless and confusing?) INT. >>> I think I understand your explanation, the VCPU takes the interrupt immediately >>> after the INVALL and before the INT, and the second interrupt that I am seeing is >>> the one caused by the INT command. >> Yes. >> >>> I tried modifying the test like this: >>> >>> diff --git a/arm/gic.c b/arm/gic.c >>> index 6e93da80fe0d..0ef8c12ea234 100644 >>> --- a/arm/gic.c >>> +++ b/arm/gic.c >>> @@ -761,10 +761,17 @@ static void test_its_trigger(void) >>>          wmb(); >>>          cpumask_clear(&mask); >>>          cpumask_set_cpu(3, &mask); >>> -       its_send_int(dev2, 20); >> Shouldn't its_send_invall(col3) be moved down here? See below. >> >>>          wait_for_interrupts(&mask); >>>          report(check_acked(&mask, 0, 8195), >>> -                       "dev2/eventid=20 now triggers an LPI"); >>> +                       "dev2/eventid=20 pending LPI is received"); >>> + >>> +       stats_reset(); >>> +       wmb(); >>> +       cpumask_clear(&mask); >>> +       cpumask_set_cpu(3, &mask); >>> +       its_send_int(dev2, 20); >>> +       wait_for_interrupts(&mask); >>> +       report(check_acked(&mask, 0, 8195), "dev2/eventid=20 triggers an LPI"); >>>            report_prefix_pop(); >>>   I removed the INT from the initial test, and added a separate one to check that >>> the INT command still works. That looks to me that preserves the spirit of the >>> original test. After doing stress testing this is what I got: >>> >>> - with kvmtool, 47,709 iterations, 27 times the test timed out when waiting for >>> the interrupt after INVALL. >>> - with qemu, 15,511 iterations, 258 times the test timed out when waiting for the >>> interrupt after INVALL, just like with kvmtool. >> I guess the reason of failure is that the LPI is taken *immediately* >> after the INVALL? >> >>     /* Now call the invall and check the LPI hits */ >>     its_send_invall(col3); >>         <- LPI is taken, acked[]++ >>     stats_reset(); >>         <- acked[] is cleared unexpectedly >>     wmb(); >>     cpumask_clear(&mask); >>     cpumask_set_cpu(3, &mask); >>     wait_for_interrupts(&mask); >>         <- we'll hit timed-out since acked[] is 0 > Yes, of course, you're right, I didn't realize that I was resetting the stats > *after* the interrupt was enabled. This also explains why I was still seeing > timeouts even when the timeout duration was set to 50 seconds. I'll retest with > the fix: > > diff --git a/arm/gic.c b/arm/gic.c > index 6e93da80fe0d..c4240f5aba39 100644 > --- a/arm/gic.c > +++ b/arm/gic.c > @@ -756,15 +756,22 @@ static void test_its_trigger(void) >                         "dev2/eventid=20 still does not trigger any LPI"); >   >         /* Now call the invall and check the LPI hits */ > +       stats_reset(); > +       wmb(); > +       cpumask_clear(&mask); > +       cpumask_set_cpu(3, &mask); >         its_send_invall(col3); > +       wait_for_interrupts(&mask); > +       report(check_acked(&mask, 0, 8195), > +                       "dev2/eventid=20 pending LPI is received"); > + >         stats_reset(); >         wmb(); >         cpumask_clear(&mask); >         cpumask_set_cpu(3, &mask); >         its_send_int(dev2, 20); >         wait_for_interrupts(&mask); > -       report(check_acked(&mask, 0, 8195), > -                       "dev2/eventid=20 now triggers an LPI"); > +       report(check_acked(&mask, 0, 8195), "dev2/eventid20 triggers an LPI"); >   >         report_prefix_pop(); >   > I also pushed a branch at [1]. > > Thank you so much for spotting this! You've saved me (and probably others) a lot > of time debugging. > > [1] https://gitlab.arm.com/linux-arm/kvm-unit-tests-ae/-/tree/fixes1-v2 I have been testing the branch, no failures after 17,996 runs with qemu and 58,669 runs with kvmtool. This looks fine to me, I'll send a v2 with the fix. Thanks, Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.2 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FB71C64E90 for ; Tue, 1 Dec 2020 15:08:30 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 79FAE20758 for ; Tue, 1 Dec 2020 15:08:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79FAE20758 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 7B7D64C288; Tue, 1 Dec 2020 10:08:28 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id W1fleghikeKY; Tue, 1 Dec 2020 10:08:27 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 1C6964C261; Tue, 1 Dec 2020 10:08:27 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 99F764C253 for ; Tue, 1 Dec 2020 10:08:25 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id rrDN4yuiu8HL for ; Tue, 1 Dec 2020 10:08:24 -0500 (EST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 026D34C24C for ; Tue, 1 Dec 2020 10:08:23 -0500 (EST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 882FF30E; Tue, 1 Dec 2020 07:08:23 -0800 (PST) Received: from [10.37.12.21] (unknown [10.37.12.21]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 9EBE73F575; Tue, 1 Dec 2020 07:08:22 -0800 (PST) Subject: Re: [kvm-unit-tests PATCH 10/10] arm64: gic: Use IPI test checking for the LPI tests From: Alexandru Elisei To: Zenghui Yu , kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, drjones@redhat.com References: <20201125155113.192079-1-alexandru.elisei@arm.com> <20201125155113.192079-11-alexandru.elisei@arm.com> <49be46a8-2c29-b805-366e-7c955d395874@huawei.com> Message-ID: <1dc913fd-80f5-4c8b-9c4d-52a1f0eee400@arm.com> Date: Tue, 1 Dec 2020 15:09:44 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Cc: andre.przywara@arm.com X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu SGksCgpPbiAxMS8zMC8yMCAyOjE5IFBNLCBBbGV4YW5kcnUgRWxpc2VpIHdyb3RlOgo+IEhpIFpl bmdodWksCj4KPiBPbiAxMS8zMC8yMCAxOjU5IFBNLCBaZW5naHVpIFl1IHdyb3RlOgo+PiBIaSBB bGV4LAo+Pgo+PiBPbiAyMDIwLzExLzI3IDIyOjUwLCBBbGV4YW5kcnUgRWxpc2VpIHdyb3RlOgo+ Pj4gSGkgWmhlbmdodWksCj4+Pgo+Pj4gVGhhbmsgeW91IGZvciBoYXZpbmcgYSBsb29rIGF0IHRo aXMhCj4+Pgo+Pj4gT24gMTEvMjYvMjAgOTozMCBBTSwgWmVuZ2h1aSBZdSB3cm90ZToKPj4+PiBP biAyMDIwLzExLzI1IDIzOjUxLCBBbGV4YW5kcnUgRWxpc2VpIHdyb3RlOgo+Pj4+PiBUaGUgcmVh c29uIGZvciB0aGUgZmFpbHVyZSBpcyB0aGF0IHRoZSB0ZXN0ICJkZXYyL2V2ZW50aWQ9MjAgbm93 IHRyaWdnZXJzCj4+Pj4+IGFuIExQSSIgdHJpZ2dlcnMgMiBMUElzLCBub3Qgb25lLiBUaGlzIGJl aGF2aW9yIHdhcyBwcmVzZW50IGJlZm9yZSB0aGlzCj4+Pj4+IHBhdGNoLCBidXQgaXQgd2FzIGln bm9yZWQgYmVjYXVzZSBjaGVja19scGlfc3RhdHMoKSB3YXNuJ3QgbG9va2luZyBhdCB0aGUKPj4+ Pj4gYWNrZWQgYXJyYXkuCj4+Pj4+Cj4+Pj4+IEknbSBub3QgZmFtaWxpYXIgd2l0aCB0aGUgSVRT IHNvIEknbSBub3Qgc3VyZSBpZiB0aGlzIGlzIGV4cGVjdGVkLCBpZiB0aGUKPj4+Pj4gdGVzdCBp cyBpbmNvcnJlY3Qgb3IgaWYgdGhlcmUgaXMgc29tZXRoaW5nIHdyb25nIHdpdGggS1ZNIGVtdWxh dGlvbi4KPj4+PiBJIHRoaW5rIHRoaXMgaXMgZXhwZWN0ZWQsIG9yIG5vdC4KPj4+Pgo+Pj4+IEJl Zm9yZSBJTlZBTEwsIHRoZSBMUEktODE5NSB3YXMgYWxyZWFkeSBwZW5kaW5nIGJ1dCBkaXNhYmxl ZC4gT24KPj4+PiByZWNlaXZpbmcgSU5WQUxMLCBWR0lDIHdpbGwgcmVsb2FkIGNvbmZpZ3VyYXRp b24gZm9yIGFsbCBMUElzIHRhcmdldGluZwo+Pj4+IGNvbGxlY3Rpb24tMyBhbmQgZGVsaXZlciB0 aGUgbm93IGVuYWJsZWQgTFBJLTgxOTUuIFdlJ2xsIHRoZXJlZm9yZSBzZWUKPj4+PiBhbmQgaGFu ZGxlIGl0IGJlZm9yZSBzZW5kaW5nIHRoZSBmb2xsb3dpbmcgSU5UICh3aGljaCB3aWxsIHNldCB0 aGUKPj4+PiBMUEktODE5NSBwZW5kaW5nIGFnYWluKS4KPj4+Pgo+Pj4+PiBEaWQgc29tZSBtb3Jl IHRlc3Rpbmcgb24gYW4gQW1wZXJlIGVNQUcgKGZhc3Qgb3V0LW9mLW9yZGVyIGNvcmVzKSB1c2lu Zwo+Pj4+PiBxZW11IGFuZCBrdm10b29sIGFuZCBMaW51eCB2NS44LCBoZXJlJ3Mgd2hhdCBJIGZv dW5kOgo+Pj4+Pgo+Pj4+PiAtIFVzaW5nIHFlbXUgYW5kIGdpYy5mbGF0IGJ1aWx0IGZyb20qbWFz dGVyKjogZXJyb3IgZW5jb3VudGVyZWQgODY0IHRpbWVzCj4+Pj4+IMKgwqDCoCBvdXQgb2YgMTA4 OCBydW5zLgo+Pj4+PiAtIFVzaW5nIHFlbXU6IGVycm9yIGVuY291bnRlcmVkIDg1MiB0aW1lcyBv dXQgb2YgMTAyNyBydW5zLgo+Pj4+PiAtIFVzaW5nIGt2bXRvb2w6IGVycm9yIGVuY291bnRlcmVk IDgxNjQgdGltZXMgb3V0IG9mIDEwNjAyIHJ1bnMuCj4+Pj4gSWYgdmNwdS0zIGhhZG4ndCBzZWVu IGFuZCBoYW5kbGVkIExQSS04MTk1IGFzIHF1aWNrbHkgYXMgcG9zc2libGUgKGUuZy4sCj4+Pj4g dmNwdS0zIGhhZG4ndCBiZWVuIHNjaGVkdWxlZCksIHRoZSBmb2xsb3dpbmcgSU5UIHdpbGwgc2V0 IHRoZSBhbHJlYWR5Cj4+Pj4gcGVuZGluZyBMUEktODE5NSBwZW5kaW5nIGFnYWluIGFuZCB3ZSds bCByZWNlaXZlIGl0ICpvbmNlKiBvbiB2Y3B1LTMuCj4+Pj4gQW5kIHdlIHdvbid0IHNlZSB0aGUg bWVudGlvbmVkIGZhaWx1cmUuCj4+Pj4KPj4+PiBJIHRoaW5rIHdlIGNhbiBqdXN0IGRyb3AgdGhl IChtZWFuaW5nbGVzcyBhbmQgY29uZnVzaW5nPykgSU5ULgo+Pj4gSSB0aGluayBJIHVuZGVyc3Rh bmQgeW91ciBleHBsYW5hdGlvbiwgdGhlIFZDUFUgdGFrZXMgdGhlIGludGVycnVwdCBpbW1lZGlh dGVseQo+Pj4gYWZ0ZXIgdGhlIElOVkFMTCBhbmQgYmVmb3JlIHRoZSBJTlQsIGFuZCB0aGUgc2Vj b25kIGludGVycnVwdCB0aGF0IEkgYW0gc2VlaW5nIGlzCj4+PiB0aGUgb25lIGNhdXNlZCBieSB0 aGUgSU5UIGNvbW1hbmQuCj4+IFllcy4KPj4KPj4+IEkgdHJpZWQgbW9kaWZ5aW5nIHRoZSB0ZXN0 IGxpa2UgdGhpczoKPj4+Cj4+PiBkaWZmIC0tZ2l0IGEvYXJtL2dpYy5jIGIvYXJtL2dpYy5jCj4+ PiBpbmRleCA2ZTkzZGE4MGZlMGQuLjBlZjhjMTJlYTIzNCAxMDA2NDQKPj4+IC0tLSBhL2FybS9n aWMuYwo+Pj4gKysrIGIvYXJtL2dpYy5jCj4+PiBAQCAtNzYxLDEwICs3NjEsMTcgQEAgc3RhdGlj IHZvaWQgdGVzdF9pdHNfdHJpZ2dlcih2b2lkKQo+Pj4gwqDCoMKgwqDCoMKgwqDCoCB3bWIoKTsK Pj4+IMKgwqDCoMKgwqDCoMKgwqAgY3B1bWFza19jbGVhcigmbWFzayk7Cj4+PiDCoMKgwqDCoMKg wqDCoMKgIGNwdW1hc2tfc2V0X2NwdSgzLCAmbWFzayk7Cj4+PiAtwqDCoMKgwqDCoMKgIGl0c19z ZW5kX2ludChkZXYyLCAyMCk7Cj4+IFNob3VsZG4ndCBpdHNfc2VuZF9pbnZhbGwoY29sMykgYmUg bW92ZWQgZG93biBoZXJlPyBTZWUgYmVsb3cuCj4+Cj4+PiDCoMKgwqDCoMKgwqDCoMKgIHdhaXRf Zm9yX2ludGVycnVwdHMoJm1hc2spOwo+Pj4gwqDCoMKgwqDCoMKgwqDCoCByZXBvcnQoY2hlY2tf YWNrZWQoJm1hc2ssIDAsIDgxOTUpLAo+Pj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgICJkZXYyL2V2ZW50aWQ9MjAgbm93IHRyaWdnZXJzIGFuIExQSSIpOwo+ Pj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICJkZXYyL2V2 ZW50aWQ9MjAgcGVuZGluZyBMUEkgaXMgcmVjZWl2ZWQiKTsKPj4+ICsKPj4+ICvCoMKgwqDCoMKg wqAgc3RhdHNfcmVzZXQoKTsKPj4+ICvCoMKgwqDCoMKgwqAgd21iKCk7Cj4+PiArwqDCoMKgwqDC oMKgIGNwdW1hc2tfY2xlYXIoJm1hc2spOwo+Pj4gK8KgwqDCoMKgwqDCoCBjcHVtYXNrX3NldF9j cHUoMywgJm1hc2spOwo+Pj4gK8KgwqDCoMKgwqDCoCBpdHNfc2VuZF9pbnQoZGV2MiwgMjApOwo+ Pj4gK8KgwqDCoMKgwqDCoCB3YWl0X2Zvcl9pbnRlcnJ1cHRzKCZtYXNrKTsKPj4+ICvCoMKgwqDC oMKgwqAgcmVwb3J0KGNoZWNrX2Fja2VkKCZtYXNrLCAwLCA4MTk1KSwgImRldjIvZXZlbnRpZD0y MCB0cmlnZ2VycyBhbiBMUEkiKTsKPj4+IMKgIMKgwqDCoMKgwqDCoMKgwqAgcmVwb3J0X3ByZWZp eF9wb3AoKTsKPj4+IMKgIEkgcmVtb3ZlZCB0aGUgSU5UIGZyb20gdGhlIGluaXRpYWwgdGVzdCwg YW5kIGFkZGVkIGEgc2VwYXJhdGUgb25lIHRvIGNoZWNrIHRoYXQKPj4+IHRoZSBJTlQgY29tbWFu ZCBzdGlsbCB3b3Jrcy4gVGhhdCBsb29rcyB0byBtZSB0aGF0IHByZXNlcnZlcyB0aGUgc3Bpcml0 IG9mIHRoZQo+Pj4gb3JpZ2luYWwgdGVzdC4gQWZ0ZXIgZG9pbmcgc3RyZXNzIHRlc3RpbmcgdGhp cyBpcyB3aGF0IEkgZ290Ogo+Pj4KPj4+IC0gd2l0aCBrdm10b29sLCA0Nyw3MDkgaXRlcmF0aW9u cywgMjcgdGltZXMgdGhlIHRlc3QgdGltZWQgb3V0IHdoZW4gd2FpdGluZyBmb3IKPj4+IHRoZSBp bnRlcnJ1cHQgYWZ0ZXIgSU5WQUxMLgo+Pj4gLSB3aXRoIHFlbXUsIDE1LDUxMSBpdGVyYXRpb25z LCAyNTggdGltZXMgdGhlIHRlc3QgdGltZWQgb3V0IHdoZW4gd2FpdGluZyBmb3IgdGhlCj4+PiBp bnRlcnJ1cHQgYWZ0ZXIgSU5WQUxMLCBqdXN0IGxpa2Ugd2l0aCBrdm10b29sLgo+PiBJIGd1ZXNz IHRoZSByZWFzb24gb2YgZmFpbHVyZSBpcyB0aGF0IHRoZSBMUEkgaXMgdGFrZW4gKmltbWVkaWF0 ZWx5Kgo+PiBhZnRlciB0aGUgSU5WQUxMPwo+Pgo+PiDCoMKgwqDCoC8qIE5vdyBjYWxsIHRoZSBp bnZhbGwgYW5kIGNoZWNrIHRoZSBMUEkgaGl0cyAqLwo+PiDCoMKgwqDCoGl0c19zZW5kX2ludmFs bChjb2wzKTsKPj4gwqDCoMKgwqDCoMKgwqAgPC0gTFBJIGlzIHRha2VuLCBhY2tlZFtdKysKPj4g wqDCoMKgwqBzdGF0c19yZXNldCgpOwo+PiDCoMKgwqDCoMKgwqDCoCA8LSBhY2tlZFtdIGlzIGNs ZWFyZWQgdW5leHBlY3RlZGx5Cj4+IMKgwqDCoMKgd21iKCk7Cj4+IMKgwqDCoMKgY3B1bWFza19j bGVhcigmbWFzayk7Cj4+IMKgwqDCoMKgY3B1bWFza19zZXRfY3B1KDMsICZtYXNrKTsKPj4gwqDC oMKgwqB3YWl0X2Zvcl9pbnRlcnJ1cHRzKCZtYXNrKTsKPj4gwqDCoMKgwqDCoMKgwqAgPC0gd2Un bGwgaGl0IHRpbWVkLW91dCBzaW5jZSBhY2tlZFtdIGlzIDAKPiBZZXMsIG9mIGNvdXJzZSwgeW91 J3JlIHJpZ2h0LCBJIGRpZG4ndCByZWFsaXplIHRoYXQgSSB3YXMgcmVzZXR0aW5nIHRoZSBzdGF0 cwo+ICphZnRlciogdGhlIGludGVycnVwdCB3YXMgZW5hYmxlZC4gVGhpcyBhbHNvIGV4cGxhaW5z IHdoeSBJIHdhcyBzdGlsbCBzZWVpbmcKPiB0aW1lb3V0cyBldmVuIHdoZW4gdGhlIHRpbWVvdXQg ZHVyYXRpb24gd2FzIHNldCB0byA1MCBzZWNvbmRzLiBJJ2xsIHJldGVzdCB3aXRoCj4gdGhlIGZp eDoKPgo+IGRpZmYgLS1naXQgYS9hcm0vZ2ljLmMgYi9hcm0vZ2ljLmMKPiBpbmRleCA2ZTkzZGE4 MGZlMGQuLmM0MjQwZjVhYmEzOSAxMDA2NDQKPiAtLS0gYS9hcm0vZ2ljLmMKPiArKysgYi9hcm0v Z2ljLmMKPiBAQCAtNzU2LDE1ICs3NTYsMjIgQEAgc3RhdGljIHZvaWQgdGVzdF9pdHNfdHJpZ2dl cih2b2lkKQo+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAg ImRldjIvZXZlbnRpZD0yMCBzdGlsbCBkb2VzIG5vdCB0cmlnZ2VyIGFueSBMUEkiKTsKPiDCoAo+ IMKgwqDCoMKgwqDCoMKgIC8qIE5vdyBjYWxsIHRoZSBpbnZhbGwgYW5kIGNoZWNrIHRoZSBMUEkg aGl0cyAqLwo+ICvCoMKgwqDCoMKgwqAgc3RhdHNfcmVzZXQoKTsKPiArwqDCoMKgwqDCoMKgIHdt YigpOwo+ICvCoMKgwqDCoMKgwqAgY3B1bWFza19jbGVhcigmbWFzayk7Cj4gK8KgwqDCoMKgwqDC oCBjcHVtYXNrX3NldF9jcHUoMywgJm1hc2spOwo+IMKgwqDCoMKgwqDCoMKgIGl0c19zZW5kX2lu dmFsbChjb2wzKTsKPiArwqDCoMKgwqDCoMKgIHdhaXRfZm9yX2ludGVycnVwdHMoJm1hc2spOwo+ ICvCoMKgwqDCoMKgwqAgcmVwb3J0KGNoZWNrX2Fja2VkKCZtYXNrLCAwLCA4MTk1KSwKPiArwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgImRldjIvZXZlbnRpZD0y MCBwZW5kaW5nIExQSSBpcyByZWNlaXZlZCIpOwo+ICsKPiDCoMKgwqDCoMKgwqDCoCBzdGF0c19y ZXNldCgpOwo+IMKgwqDCoMKgwqDCoMKgIHdtYigpOwo+IMKgwqDCoMKgwqDCoMKgIGNwdW1hc2tf Y2xlYXIoJm1hc2spOwo+IMKgwqDCoMKgwqDCoMKgIGNwdW1hc2tfc2V0X2NwdSgzLCAmbWFzayk7 Cj4gwqDCoMKgwqDCoMKgwqAgaXRzX3NlbmRfaW50KGRldjIsIDIwKTsKPiDCoMKgwqDCoMKgwqDC oCB3YWl0X2Zvcl9pbnRlcnJ1cHRzKCZtYXNrKTsKPiAtwqDCoMKgwqDCoMKgIHJlcG9ydChjaGVj a19hY2tlZCgmbWFzaywgMCwgODE5NSksCj4gLcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgICJkZXYyL2V2ZW50aWQ9MjAgbm93IHRyaWdnZXJzIGFuIExQSSIpOwo+ ICvCoMKgwqDCoMKgwqAgcmVwb3J0KGNoZWNrX2Fja2VkKCZtYXNrLCAwLCA4MTk1KSwgImRldjIv ZXZlbnRpZDIwIHRyaWdnZXJzIGFuIExQSSIpOwo+IMKgCj4gwqDCoMKgwqDCoMKgwqAgcmVwb3J0 X3ByZWZpeF9wb3AoKTsKPiDCoAo+IEkgYWxzbyBwdXNoZWQgYSBicmFuY2ggYXQgWzFdLgo+Cj4g VGhhbmsgeW91IHNvIG11Y2ggZm9yIHNwb3R0aW5nIHRoaXMhIFlvdSd2ZSBzYXZlZCBtZSAoYW5k IHByb2JhYmx5IG90aGVycykgYSBsb3QKPiBvZiB0aW1lIGRlYnVnZ2luZy4KPgo+IFsxXSBodHRw czovL2dpdGxhYi5hcm0uY29tL2xpbnV4LWFybS9rdm0tdW5pdC10ZXN0cy1hZS8tL3RyZWUvZml4 ZXMxLXYyCgpJIGhhdmUgYmVlbiB0ZXN0aW5nIHRoZSBicmFuY2gsIG5vIGZhaWx1cmVzIGFmdGVy IDE3LDk5NiBydW5zIHdpdGggcWVtdSBhbmQgNTgsNjY5CnJ1bnMgd2l0aCBrdm10b29sLiBUaGlz IGxvb2tzIGZpbmUgdG8gbWUsIEknbGwgc2VuZCBhIHYyIHdpdGggdGhlIGZpeC4KClRoYW5rcywK QWxleApfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwprdm1h cm0gbWFpbGluZyBsaXN0Cmt2bWFybUBsaXN0cy5jcy5jb2x1bWJpYS5lZHUKaHR0cHM6Ly9saXN0 cy5jcy5jb2x1bWJpYS5lZHUvbWFpbG1hbi9saXN0aW5mby9rdm1hcm0K