From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-wr1-f45.google.com (mail-wr1-f45.google.com [209.85.221.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9E4447C for ; Mon, 23 May 2022 11:01:05 +0000 (UTC) Received: by mail-wr1-f45.google.com with SMTP id h5so19660558wrb.11 for ; Mon, 23 May 2022 04:01:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=message-id:date:mime-version:user-agent:subject:to:cc:references :from:in-reply-to:content-transfer-encoding; bh=7Gj0myDG7Upk1Lm/yIITVfKpkM1xaCwFPb15m2KrOlA=; b=SJc2DPtddEaRZtuiQIVwI9U6g2Kqmyg3NynK+fD7QTldXAL63VcMRo+aMpJExYDpea U61wuSKnmZudN9/aPDkW+8q2i6sI42HdU+daRbOBydwTH/40cNUSgVlse0Y2aAyMf2c5 M84vKj+sPSwuuOPh+WMeLIkLYg5u0mh0YUqnX4r/H0MM7ztqP4pL9mX9GZMSDnCcC/Eb JGCq9NAZ7xkWCQurfiXYTq3ZBPfihEm7fptSmT/Td81FKIlPYQCJRIcieZ7zpDRZQWaW wJJlN3j/UIGTboshXaa5yTkQil2BreC2vL3dzzRVBrCLzj0GOELRnczy6unvHZKSluZg mP2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :to:cc:references:from:in-reply-to:content-transfer-encoding; bh=7Gj0myDG7Upk1Lm/yIITVfKpkM1xaCwFPb15m2KrOlA=; b=lRs6MSeCidb4iAuw5FX4lKoyS4ecmE5Mxi3esOk96NUDn/IHYSO6awMSyAfbK0V0WE AUO2n8SmCUdzzWwEPBJ7bsgx9fwXRnInZEwUtys802gfSPSP81hKDRg2DGjlxRJGciGj 6islOT02ZLC0QGLL4KO89IXm6ObMtV6XxsXRCbZ7aeKP5Luslyk7QU/agUc2mOHMflLZ YY7dRjD3bi876FzW5pq4ydYYdEMzqOGo0T5A9pIQ85uLuKxz3Ls7x1qfR9+4cRAOqexC +xTQLOISBIEkYZcNeQk60gsEBJrQU95dcU3hYK4nBidB0k6Ar+/jR8HAo2qBRHvCzP6i VllA== X-Gm-Message-State: AOAM533bOreMyc9E92fQlIg9xutSE70zr2CoV/XqDewYX9eWGnm8pXm2 doPM6vbKbf9PtrawzPv2+yEPpg== X-Google-Smtp-Source: ABdhPJx6MdtFMgPpDDHtWkV8+ZFaWG7x/MF24zzfaJQeetYugRp7nIBEH/k2SC8JHmNUjXnyYkNFsw== X-Received: by 2002:a5d:6406:0:b0:20e:5bf9:8e31 with SMTP id z6-20020a5d6406000000b0020e5bf98e31mr18094381wru.432.1653303663852; Mon, 23 May 2022 04:01:03 -0700 (PDT) Received: from ?IPV6:2a00:1098:3142:14:3110:d736:2a7:6aff? ([2a00:1098:3142:14:3110:d736:2a7:6aff]) by smtp.gmail.com with ESMTPSA id g10-20020adfa58a000000b0020e60baffd1sm6477332wrc.52.2022.05.23.04.01.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 23 May 2022 04:01:03 -0700 (PDT) Message-ID: Date: Mon, 23 May 2022 12:01:03 +0100 Precedence: bulk X-Mailing-List: regressions@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: vchiq: Performance regression since 5.18-rc1 To: Stefan Wahren , paulmck@kernel.org Cc: Marcelo Tosatti , Andrew Morton , Nicolas Saenz Julienne , Borislav Petkov , Minchan Kim , Mel Gorman , Juri Lelli , Thomas Gleixner , Sebastian Andrzej Siewior , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linux ARM , regressions@lists.linux.dev, riel@surriel.com, viro@zeniv.linux.org.uk References: <77d6d498-7dd9-03eb-60f2-d7e682bb1b20@i2se.com> <20220521234616.GO1790663@paulmck-ThinkPad-P17-Gen-1> <20220523044818.GS1790663@paulmck-ThinkPad-P17-Gen-1> <58cb7fbb-d317-83e6-0427-d3f3944b24b8@raspberrypi.com> From: Phil Elwell In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi Stefan, On 23/05/2022 11:48, Stefan Wahren wrote: > Hi Phil, > > Am 23.05.22 um 11:29 schrieb Phil Elwell: >> Hi Stefan, >> >> On 23/05/2022 07:19, Stefan Wahren wrote: >>> Hi Paul, >>> >>> Am 23.05.22 um 06:48 schrieb Paul E. McKenney: >>>> On Sun, May 22, 2022 at 05:11:36PM +0200, Stefan Wahren wrote: >>>>> Hi Paul, >>>>> >>>>> Am 22.05.22 um 01:46 schrieb Paul E. McKenney: >>>>>> On Sun, May 22, 2022 at 01:22:00AM +0200, Stefan Wahren wrote: >>>>>>> Hi, >>>>>>> >>>>>>> while testing the staging/vc04_services/interface/vchiq_arm driver with my >>>>>>> Raspberry Pi 3 B+ (multi_v7_defconfig) i noticed a huge performance >>>>>>> regression since [ff042f4a9b050895a42cae893cc01fa2ca81b95c] mm: >>>>>>> lru_cache_disable: replace work queue synchronization with synchronize_rcu >>>>>>> >>>>>>> Usually i run "vchiq_test -f 1" to see the driver is still working [1]. >>>>>>> >>>>>>> Before commit: >>>>>>> >>>>>>> real    0m1,500s >>>>>>> user    0m0,068s >>>>>>> sys    0m0,846s >>>>>>> >>>>>>> After commit: >>>>>>> >>>>>>> real    7m11,449s >>>>>>> user    0m2,049s >>>>>>> sys    0m0,023s >>>>>>> >>>>>>> Best regards >>>>>>> >>>>>>> [1] - https://github.com/raspberrypi/userland >>>>>> Please feel free to try the patch shown below.  Or the pair of patches >>>>>> from Rik here: >>>>>> >>>>>> https://lore.kernel.org/lkml/20220218183114.2867528-2-riel@surriel.com/ >>>>>> https://lore.kernel.org/lkml/20220218183114.2867528-3-riel@surriel.com/ >>>>> I tried your patch and Rik's patches but in both cases vchiq_test runs 7 >>>>> minutes instead of ~ 1 second. >>>> That is surprising.  Do you boot with rcupdate.rcu_normal=1? >>> No, not explicit. >>>>    That would >>>> nullify my patch, but I would expect that Rik's patch would still provide >>>> increased performance even in that case. >>> I will retest with a fresh SD card image. >>>> >>>> Could you please characterize where the slowdown is occurring? >>> >>> Unfortunately i don't have a deep insight into driver and vchiq_test tool. >>> Just a user view. >>> >>> Do you think an strace would be a good starting point? >>> >>> @Phil Any advices to analyse this issue? >> >> Sending many small control packets: >> >>    vchiq_test -c 1 10000 >> >> essentially tests interrupt latency. Using a small number of large bulk >> transfers: >> >>    vchiq_test -b 10000 1 >> >> becomes a test of how long it takes to lock down pages. It also tests DMA >> transfer speeds, but since the DMA is run by the firmware (which you aren't >> changing), I think you can rule that. > Thanks i will try. >> >> You may also find it helpful to include "force_turbo=1" in config.txt for more >> predictable results. >> >> By the way, running our 5.18-rc7-based branch on a 3B+ I'm not seeing any >> performance problems: > I assume you are using arm/bcm2709_defconfig and not arm/multi_v7_defconfig as me? That's correct. Simply switching to multi_v7_defconfig breaks vchiq completely, presumably because it doesn't define CONFIG_BCM2835_VCHIQ. Phil >> >> pi@raspberrypi:~$ time vchiq_test -f 1 >> Functional test - iters:1 >> ======== iteration 1 ======== >> Testing bulk transfer for alignment. >> Testing bulk transfer at PAGE_SIZE. >> >> real    0m0.512s >> user    0m0.042s >> sys     0m0.165s >> >> Phil From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BB4F9C433F5 for ; Mon, 23 May 2022 11:47:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=33zrI2m/iXB4da1R0zPYBuxeGW8xarfdWusS4MH9HR0=; b=3Jhnxm0aX2/QBM pZYjIk3buiSLIR9dq6AAzg9yWA/fDg/wdYtxO3N0hGIAi492owNaZxi+sco3uS9QQSIkF16tDIRNe jRPPusGtuBAu0RDKUsfke/Ayc8n6qe0lm9ZjNf36QBKgEr1JcrjdfW5NegZfhTp+STaoP25zF50Fg YTRkuaj+nr9hOgazZ5AXKmFs78WnJ83ZhWZQR5ae99kSN9K+k0uVnRD2ove8jM9Ucr8MgrWntfPRF +4dR4fLAMAzUrX/bn9Y0jQ4id0r3WZfpaAsFB9n2195WF7F4r4QQVx3B9RMpL8twxPMz+VAw6PaVl sZ04nChf8jmkLZqTB0YA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nt6VW-003pDV-U0; Mon, 23 May 2022 11:45:55 +0000 Received: from mail-wr1-x42e.google.com ([2a00:1450:4864:20::42e]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nt5oA-003Ut9-L8 for linux-arm-kernel@lists.infradead.org; Mon, 23 May 2022 11:01:08 +0000 Received: by mail-wr1-x42e.google.com with SMTP id k30so20781294wrd.5 for ; Mon, 23 May 2022 04:01:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=raspberrypi.com; s=google; h=message-id:date:mime-version:user-agent:subject:to:cc:references :from:in-reply-to:content-transfer-encoding; bh=7Gj0myDG7Upk1Lm/yIITVfKpkM1xaCwFPb15m2KrOlA=; b=SJc2DPtddEaRZtuiQIVwI9U6g2Kqmyg3NynK+fD7QTldXAL63VcMRo+aMpJExYDpea U61wuSKnmZudN9/aPDkW+8q2i6sI42HdU+daRbOBydwTH/40cNUSgVlse0Y2aAyMf2c5 M84vKj+sPSwuuOPh+WMeLIkLYg5u0mh0YUqnX4r/H0MM7ztqP4pL9mX9GZMSDnCcC/Eb JGCq9NAZ7xkWCQurfiXYTq3ZBPfihEm7fptSmT/Td81FKIlPYQCJRIcieZ7zpDRZQWaW wJJlN3j/UIGTboshXaa5yTkQil2BreC2vL3dzzRVBrCLzj0GOELRnczy6unvHZKSluZg mP2A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :to:cc:references:from:in-reply-to:content-transfer-encoding; bh=7Gj0myDG7Upk1Lm/yIITVfKpkM1xaCwFPb15m2KrOlA=; b=SpDGojkLSNiXudgPmiqKTMzZtRc+b5URkgMH9lH4X1COlWN+2gOVsQ/7/hFRX+pO7z YeXSl9ZSWlpUx5HA6bM7vT5c9CtOCMBVtP8TO+hkBAO/Xw151Ze4AP74cENsQ6fAECER ZaiRqiN04rVhj/H380/pjjpZT13oCV5yI5p9YIMQbTZixIRE8n89NmTSjiyJ5x64ZCPJ JS7qyY4R14PJiZwW+36t0ufehF1YLkVATRzTltvQpEc07qT4DQ8tlIXtVPsYL3Ys9xBx ghFkFUrw4yvyxwtplux5hJINHpQlQzZUM9iuDAvWg8Ob2Ynm7nuee4VFLhTIA1NT2+RE 2+VQ== X-Gm-Message-State: AOAM5335JXMbUQzDbMVSkLAaMnDkWgxHTmsHY9iQbj/KCmBUUuxW9KA2 qMlMFVgRgzWo5XHp5Bf9nvpjSQ== X-Google-Smtp-Source: ABdhPJx6MdtFMgPpDDHtWkV8+ZFaWG7x/MF24zzfaJQeetYugRp7nIBEH/k2SC8JHmNUjXnyYkNFsw== X-Received: by 2002:a5d:6406:0:b0:20e:5bf9:8e31 with SMTP id z6-20020a5d6406000000b0020e5bf98e31mr18094381wru.432.1653303663852; Mon, 23 May 2022 04:01:03 -0700 (PDT) Received: from ?IPV6:2a00:1098:3142:14:3110:d736:2a7:6aff? ([2a00:1098:3142:14:3110:d736:2a7:6aff]) by smtp.gmail.com with ESMTPSA id g10-20020adfa58a000000b0020e60baffd1sm6477332wrc.52.2022.05.23.04.01.03 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 23 May 2022 04:01:03 -0700 (PDT) Message-ID: Date: Mon, 23 May 2022 12:01:03 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101 Thunderbird/91.9.1 Subject: Re: vchiq: Performance regression since 5.18-rc1 To: Stefan Wahren , paulmck@kernel.org Cc: Marcelo Tosatti , Andrew Morton , Nicolas Saenz Julienne , Borislav Petkov , Minchan Kim , Mel Gorman , Juri Lelli , Thomas Gleixner , Sebastian Andrzej Siewior , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linux ARM , regressions@lists.linux.dev, riel@surriel.com, viro@zeniv.linux.org.uk References: <77d6d498-7dd9-03eb-60f2-d7e682bb1b20@i2se.com> <20220521234616.GO1790663@paulmck-ThinkPad-P17-Gen-1> <20220523044818.GS1790663@paulmck-ThinkPad-P17-Gen-1> <58cb7fbb-d317-83e6-0427-d3f3944b24b8@raspberrypi.com> From: Phil Elwell In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220523_040106_830242_0BD92E1B X-CRM114-Status: GOOD ( 18.06 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org SGkgU3RlZmFuLAoKT24gMjMvMDUvMjAyMiAxMTo0OCwgU3RlZmFuIFdhaHJlbiB3cm90ZToKPiBI aSBQaGlsLAo+IAo+IEFtIDIzLjA1LjIyIHVtIDExOjI5IHNjaHJpZWIgUGhpbCBFbHdlbGw6Cj4+ IEhpIFN0ZWZhbiwKPj4KPj4gT24gMjMvMDUvMjAyMiAwNzoxOSwgU3RlZmFuIFdhaHJlbiB3cm90 ZToKPj4+IEhpIFBhdWwsCj4+Pgo+Pj4gQW0gMjMuMDUuMjIgdW0gMDY6NDggc2NocmllYiBQYXVs IEUuIE1jS2VubmV5Ogo+Pj4+IE9uIFN1biwgTWF5IDIyLCAyMDIyIGF0IDA1OjExOjM2UE0gKzAy MDAsIFN0ZWZhbiBXYWhyZW4gd3JvdGU6Cj4+Pj4+IEhpIFBhdWwsCj4+Pj4+Cj4+Pj4+IEFtIDIy LjA1LjIyIHVtIDAxOjQ2IHNjaHJpZWIgUGF1bCBFLiBNY0tlbm5leToKPj4+Pj4+IE9uIFN1biwg TWF5IDIyLCAyMDIyIGF0IDAxOjIyOjAwQU0gKzAyMDAsIFN0ZWZhbiBXYWhyZW4gd3JvdGU6Cj4+ Pj4+Pj4gSGksCj4+Pj4+Pj4KPj4+Pj4+PiB3aGlsZSB0ZXN0aW5nIHRoZSBzdGFnaW5nL3ZjMDRf c2VydmljZXMvaW50ZXJmYWNlL3ZjaGlxX2FybSBkcml2ZXIgd2l0aCBteQo+Pj4+Pj4+IFJhc3Bi ZXJyeSBQaSAzIEIrIChtdWx0aV92N19kZWZjb25maWcpIGkgbm90aWNlZCBhIGh1Z2UgcGVyZm9y bWFuY2UKPj4+Pj4+PiByZWdyZXNzaW9uIHNpbmNlIFtmZjA0MmY0YTliMDUwODk1YTQyY2FlODkz Y2MwMWZhMmNhODFiOTVjXSBtbToKPj4+Pj4+PiBscnVfY2FjaGVfZGlzYWJsZTogcmVwbGFjZSB3 b3JrIHF1ZXVlIHN5bmNocm9uaXphdGlvbiB3aXRoIHN5bmNocm9uaXplX3JjdQo+Pj4+Pj4+Cj4+ Pj4+Pj4gVXN1YWxseSBpIHJ1biAidmNoaXFfdGVzdCAtZiAxIiB0byBzZWUgdGhlIGRyaXZlciBp cyBzdGlsbCB3b3JraW5nIFsxXS4KPj4+Pj4+Pgo+Pj4+Pj4+IEJlZm9yZSBjb21taXQ6Cj4+Pj4+ Pj4KPj4+Pj4+PiByZWFswqDCoMKgIDBtMSw1MDBzCj4+Pj4+Pj4gdXNlcsKgwqDCoCAwbTAsMDY4 cwo+Pj4+Pj4+IHN5c8KgwqDCoCAwbTAsODQ2cwo+Pj4+Pj4+Cj4+Pj4+Pj4gQWZ0ZXIgY29tbWl0 Ogo+Pj4+Pj4+Cj4+Pj4+Pj4gcmVhbMKgwqDCoCA3bTExLDQ0OXMKPj4+Pj4+PiB1c2VywqDCoMKg IDBtMiwwNDlzCj4+Pj4+Pj4gc3lzwqDCoMKgIDBtMCwwMjNzCj4+Pj4+Pj4KPj4+Pj4+PiBCZXN0 IHJlZ2FyZHMKPj4+Pj4+Pgo+Pj4+Pj4+IFsxXSAtIGh0dHBzOi8vZ2l0aHViLmNvbS9yYXNwYmVy cnlwaS91c2VybGFuZAo+Pj4+Pj4gUGxlYXNlIGZlZWwgZnJlZSB0byB0cnkgdGhlIHBhdGNoIHNo b3duIGJlbG93LsKgIE9yIHRoZSBwYWlyIG9mIHBhdGNoZXMKPj4+Pj4+IGZyb20gUmlrIGhlcmU6 Cj4+Pj4+Pgo+Pj4+Pj4gaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvbGttbC8yMDIyMDIxODE4MzEx NC4yODY3NTI4LTItcmllbEBzdXJyaWVsLmNvbS8KPj4+Pj4+IGh0dHBzOi8vbG9yZS5rZXJuZWwu b3JnL2xrbWwvMjAyMjAyMTgxODMxMTQuMjg2NzUyOC0zLXJpZWxAc3VycmllbC5jb20vCj4+Pj4+ IEkgdHJpZWQgeW91ciBwYXRjaCBhbmQgUmlrJ3MgcGF0Y2hlcyBidXQgaW4gYm90aCBjYXNlcyB2 Y2hpcV90ZXN0IHJ1bnMgNwo+Pj4+PiBtaW51dGVzIGluc3RlYWQgb2YgfiAxIHNlY29uZC4KPj4+ PiBUaGF0IGlzIHN1cnByaXNpbmcuwqAgRG8geW91IGJvb3Qgd2l0aCByY3VwZGF0ZS5yY3Vfbm9y bWFsPTE/Cj4+PiBObywgbm90IGV4cGxpY2l0Lgo+Pj4+IMKgwqAgVGhhdCB3b3VsZAo+Pj4+IG51 bGxpZnkgbXkgcGF0Y2gsIGJ1dCBJIHdvdWxkIGV4cGVjdCB0aGF0IFJpaydzIHBhdGNoIHdvdWxk IHN0aWxsIHByb3ZpZGUKPj4+PiBpbmNyZWFzZWQgcGVyZm9ybWFuY2UgZXZlbiBpbiB0aGF0IGNh c2UuCj4+PiBJIHdpbGwgcmV0ZXN0IHdpdGggYSBmcmVzaCBTRCBjYXJkIGltYWdlLgo+Pj4+Cj4+ Pj4gQ291bGQgeW91IHBsZWFzZSBjaGFyYWN0ZXJpemUgd2hlcmUgdGhlIHNsb3dkb3duIGlzIG9j Y3VycmluZz8KPj4+Cj4+PiBVbmZvcnR1bmF0ZWx5IGkgZG9uJ3QgaGF2ZSBhIGRlZXAgaW5zaWdo dCBpbnRvIGRyaXZlciBhbmQgdmNoaXFfdGVzdCB0b29sLiAKPj4+IEp1c3QgYSB1c2VyIHZpZXcu Cj4+Pgo+Pj4gRG8geW91IHRoaW5rIGFuIHN0cmFjZSB3b3VsZCBiZSBhIGdvb2Qgc3RhcnRpbmcg cG9pbnQ/Cj4+Pgo+Pj4gQFBoaWwgQW55IGFkdmljZXMgdG8gYW5hbHlzZSB0aGlzIGlzc3VlPwo+ Pgo+PiBTZW5kaW5nIG1hbnkgc21hbGwgY29udHJvbCBwYWNrZXRzOgo+Pgo+PiDCoMKgIHZjaGlx X3Rlc3QgLWMgMSAxMDAwMAo+Pgo+PiBlc3NlbnRpYWxseSB0ZXN0cyBpbnRlcnJ1cHQgbGF0ZW5j eS4gVXNpbmcgYSBzbWFsbCBudW1iZXIgb2YgbGFyZ2UgYnVsayAKPj4gdHJhbnNmZXJzOgo+Pgo+ PiDCoMKgIHZjaGlxX3Rlc3QgLWIgMTAwMDAgMQo+Pgo+PiBiZWNvbWVzIGEgdGVzdCBvZiBob3cg bG9uZyBpdCB0YWtlcyB0byBsb2NrIGRvd24gcGFnZXMuIEl0IGFsc28gdGVzdHMgRE1BIAo+PiB0 cmFuc2ZlciBzcGVlZHMsIGJ1dCBzaW5jZSB0aGUgRE1BIGlzIHJ1biBieSB0aGUgZmlybXdhcmUg KHdoaWNoIHlvdSBhcmVuJ3QgCj4+IGNoYW5naW5nKSwgSSB0aGluayB5b3UgY2FuIHJ1bGUgdGhh dC4KPiBUaGFua3MgaSB3aWxsIHRyeS4KPj4KPj4gWW91IG1heSBhbHNvIGZpbmQgaXQgaGVscGZ1 bCB0byBpbmNsdWRlICJmb3JjZV90dXJibz0xIiBpbiBjb25maWcudHh0IGZvciBtb3JlIAo+PiBw cmVkaWN0YWJsZSByZXN1bHRzLgo+Pgo+PiBCeSB0aGUgd2F5LCBydW5uaW5nIG91ciA1LjE4LXJj Ny1iYXNlZCBicmFuY2ggb24gYSAzQisgSSdtIG5vdCBzZWVpbmcgYW55IAo+PiBwZXJmb3JtYW5j ZSBwcm9ibGVtczoKPiBJIGFzc3VtZSB5b3UgYXJlIHVzaW5nIGFybS9iY20yNzA5X2RlZmNvbmZp ZyBhbmQgbm90IGFybS9tdWx0aV92N19kZWZjb25maWcgYXMgbWU/CgpUaGF0J3MgY29ycmVjdC4g U2ltcGx5IHN3aXRjaGluZyB0byBtdWx0aV92N19kZWZjb25maWcgYnJlYWtzIHZjaGlxIGNvbXBs ZXRlbHksIApwcmVzdW1hYmx5IGJlY2F1c2UgaXQgZG9lc24ndCBkZWZpbmUgQ09ORklHX0JDTTI4 MzVfVkNISVEuCgpQaGlsCgo+Pgo+PiBwaUByYXNwYmVycnlwaTp+JCB0aW1lIHZjaGlxX3Rlc3Qg LWYgMQo+PiBGdW5jdGlvbmFsIHRlc3QgLSBpdGVyczoxCj4+ID09PT09PT09IGl0ZXJhdGlvbiAx ID09PT09PT09Cj4+IFRlc3RpbmcgYnVsayB0cmFuc2ZlciBmb3IgYWxpZ25tZW50Lgo+PiBUZXN0 aW5nIGJ1bGsgdHJhbnNmZXIgYXQgUEFHRV9TSVpFLgo+Pgo+PiByZWFswqDCoMKgIDBtMC41MTJz Cj4+IHVzZXLCoMKgwqAgMG0wLjA0MnMKPj4gc3lzwqDCoMKgwqAgMG0wLjE2NXMKPj4KPj4gUGhp bAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgt YXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQu b3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJt LWtlcm5lbAo=