From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx3.molgen.mpg.de (mx3.molgen.mpg.de [141.14.17.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51FE83D9F for ; Sun, 22 Jan 2023 20:28:41 +0000 (UTC) Received: from [192.168.0.2] (ip5f5ae93c.dynamic.kabel-deutschland.de [95.90.233.60]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) (Authenticated sender: pmenzel) by mx.molgen.mpg.de (Postfix) with ESMTPSA id C3AB261CC40F9; Sun, 22 Jan 2023 21:28:32 +0100 (CET) Message-ID: <05d381af-5ccb-0d87-97d3-e2fc4ce870fc@molgen.mpg.de> Date: Sun, 22 Jan 2023 21:28:32 +0100 Precedence: bulk X-Mailing-List: regressions@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Subject: Re: [Intel-wired-lan] Supermicro AOC-STGN-I1S (Intel 82599EN based 10G adapter) - poor network perfomance after moving to Debian 11.5 Content-Language: en-US To: Bartek Kois Cc: intel-wired-lan@osuosl.org, regressions@lists.linux.dev References: <652bf236-d97e-832c-e0f3-24927a46d7ad@molgen.mpg.de> <744de70c-782d-5d36-87fc-e6b92ac84190@gmail.com> <30de7b89-6a4f-8dab-d671-027140bbb52b@gmail.com> <3b957674-a559-ac1e-27b8-b81e6eeffe75@gmail.com> From: Paul Menzel In-Reply-To: <3b957674-a559-ac1e-27b8-b81e6eeffe75@gmail.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Dear Bartek, Am 19.01.23 um 18:17 schrieb Bartek Kois: > W dniu 19.01.2023 o 18:09, Paul Menzel pisze: >> Am 19.01.23 um 17:58 schrieb Bartek Kois: >>> W dniu 19.01.2023 o 13:24, Bartek Kois pisze: >>>> >>>> W dniu 19.01.2023 o 11:17, Paul Menzel pisze: >>>>> >>>>> #regzbot ^introduced: 4.9.88..5.10.149 >> >>>>> Am 14.01.23 um 11:23 schrieb Bartek Kois: >>>>> >>>>>> After moving from Debian 9.7 to 11.5 as soon as I perform "ip link >>>>>> set enp1s0 up" for my 10G adapter (AOC-STGN-I1S - Intel 82599EN >>>>>> based 10G adapter) I am experiencing high cpu load (even if no >>>>>> traffic is passing through the adapter) and network performance is >>>>>> low (when network is connected). >>>>> >>>>> How do you test the network performance? Please give exact numbers >>>>> for comparison. >>>>> >>>> I am using this server as a router for my subscribers with iptables >>>> (for NAT and firewall) and hfsc (for QoS). First I encountered this >>>> problem while migrating form Debian 9.7 to 11.5. Routers based  on >>>> Supermicro X11SSL-F (Intel® C232 chipset) works with no problems >>>> after that migration, but routers based on Supermicro X9SCL (Intel >>>> C202 PCH) and Supermicro X10SLL+-F (Intel C222 Express PCH) starts >>>> behaving strangely with high cpu load (0.5-0.8 while before it was >>>> around 0.0-0.1) and subscribers not being able to utilize their >>>> plans. I tried to strip down the problem and ends up with clean >>>> system with no iptables or hfsc rules behaving the same (higher >>>> load) right after setting the 10G link upeven if no traffic is >>>> passing by. >>>> >>>>>> The cpu load is oscillating between 0.1 and 0.3 on vanilla system >>>>>> with no network attached. The problem can be observed on the >>>>>> following platforms: Supermicro X9SCL (Intel C202 PCH) and >>>>>> Supermicro X10SLL+-F (Intel C222 Express PCH), but for the Supermicro >>>>>> X11SSL-F (Intel® C232 chipset) everything is working well. >>>>>> >>>>>> Tested environments: >>>>>> Debian 9.7 - Linux 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 >>>>>> (2018-05-07) x86_64 GNU/Linux [all platforms working well with no >>>>>> problems: Supermicro X9SCL (Intel C202 PCH), Supermicro X10SLL+-F >>>>>> (Intel C222 Express PCH), Supermicro X11SSL-F (Intel® C232 chipset)] >>>>> >>>>>> Debian 11.5 - Linux 5.10.0-19-amd64 #1 SMP Debian 5.10.149-2 >>>>>> (2022-10-21) x86_64 GNU/Linux [older platforms: Supermicro X9SCL >>>>>> (Intel C202 PCH), Supermicro X10SLL+-F (Intel C222 Express PCH) >>>>>> behave problematic as described above | newer platform: Supermicro >>>>>> X11SSL-F (Intel® C232 chipset) working well with no problems] >>>>> >>>>> Maybe create a bug at the Linux kernel bug tracker [1], where you >>>>> can attach all the logs (`dmesg`, `lspci -nnk -s …`, …). >>>>> >>>> I`ve already reported that to the Debian team >>>> ttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1024763, but so far >>>> nobody took care of this issue so far. >>>> >>>>>> So far to solve the problem I was trying to upgrade system to the >>>>>> newest stable version, upgrade kernel to version 6.x, upgrade >>>>>> ixgbe driver to the newest version but with no luck. >>>>> >>>>> Thank you for checking that. Too bad it’s still present. To rule >>>>> out some user space problem, could you test Debian 9.7 with a >>>>> stable Linux release, currently 6.1.7? >>>>> >>>>> What does `sudo perf top --sort comm,dso` show, where the time is >>>>> spent? >>>> >>>> During my first test in real enviroment with subscribers I gether >>>> the following data through the perf: >>>> >>>>   27.83%  [kernel]                   [k] strncpy >>>>   14.80%  [kernel]                   [k] nft_do_chain >>>>    7.61%  [kernel]                   [k] memcmp >>>>    5.63%  [kernel]                   [k] nft_meta_get_eval >>>>    3.14%  [kernel]                   [k] nft_cmp_eval >>>>    2.79%  [kernel]                   [k] asm_exc_nmi >>>>    1.07%  [kernel]                   [k] module_get_kallsym >>>>    0.92%  [kernel]                   [k] kallsyms_expand_symbol.constprop.0 >>>>    0.85%  [kernel]                   [k] ixgbe_poll >>>>    0.75%  [kernel]                   [k] format_decode >>>>    0.61%  [kernel]                   [k] number >>>>    0.56%  [kernel]                   [k] menu_select >>>>    0.54%  [kernel]                   [k] clflush_cache_range >>>>    0.52%  [kernel]                   [k] cpuidle_enter_state >>>>    0.51%  [kernel]                   [k] vsnprintf >>>>    0.50%  [kernel]                   [k] u32_classify >>>>    0.49%  [kernel]                   [k] fib_table_lookup >>>>    0.40%  [kernel]                   [k] dma_pte_clear_level >>>>    0.39%  [kernel]                   [k] domain_mapping >>>>    0.36%  [kernel]                   [k] ixgbe_xmit_fram >>>> >>>> >>>>     PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM TIME+ COMMAND >>>>      18 root      20   0       0      0      0 S  28.2   0.0 7:06.27 ksoftirqd/1 >>>>      12 root      20   0       0      0      0 R  12.0   0.0 4:10.88 ksoftirqd/0 >> >> […] >> >> Do you see different behavior in `/proc/interrupts`? >> > This is how it looks like for Debian 11.5 - Linux 5.10.0-19-amd64 #1 SMP > Debian 5.10.149-2 (2022-10-21) x86_64 GNU/Linux on Supermicro X10SLL+-F > (Intel C222 Express PCH): > >       1 root      20   0  163948  10288   7696 S   0.0   0.1 0:39.58 systemd […] The content of `/proc/interrupts` has a different format on my system. ``` $ head -3 /proc/interrupts CPU0 CPU1 CPU2 CPU3 1: 55560 0 113 0 IR-IO-APIC 1-edge i8042 8: 0 0 0 0 IR-IO-APIC 8-edge rtc0 ``` […] > and for Debian 9.7 - Linux 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 > on Supermicro X10SLL+-F (Intel C222 Express PCH) > > 31659 root      20   0       0      0      0 S   0.3  0.0 0:00.92 > kworker/7:0 >     1 root      20   0   57032   6736   5256 S   0.0  0.1 2:28.14 systemd […] >>>>>> Supermicro support suggested as follows: >>>>>> it might be kernel related debian 11.5 has kernel 5.10 which is a >>>>>> recent kernel it might not properly support the chipsets for X9 >>>>>> therefore i suggest to use RHEL or CentOS as they use much older >>>>>> kernel versions. I expect that with ubuntu 20.04 you see the same >>>>>> problem it uses kernel 5.4 >>>>> >>> Testing another GNU/Linux distribution for another data point, >>>>> might >>>>> be a good idea. >>>>> >>>>> As nobody has responded yet, bisecting the issue is probably the >>>>> fastest way to get to the bottom of this. Luckily the problem seems >>>>> reproducible and you seem to be able to build a Linux kernel >>>>> yourself, so that should work. (For testing purposes you could also >>>>> test with Ubuntu, as they provide Linux kernel builds for (almost) >>>>> all releases in their Linux kernel mainline PPA [2].) >>>>> >>>> Of course  I can try Ubuntu and report how it is working. >>>> >>> Ubuntu (5.15.0-43-generic) seems to be working in the same way >>> generating higher load after executing "ip link set enp1s0 up". >> >> That is good to know. (Is this Ubuntu 22.04?) What about Ubuntu 20.04 >> with Linux 5.4, and Ubuntu 18.04 with 4.15? >> >> Anyway, I think, you won’t come around bisecting. Another hint, make >> sure that you can build a 4.9 Linux kernel yourself, that does not >> exhibit that issue. >> > That`s right, it is 22.04. I don`t have to build it. Standard kernel > Linux 4.9.0-6-amd64 from Debian 9.7 worked without problems for past 4 > years. If nobody of the developers/maintainers is going to step up, you are on your own. Again, as you can reproduce this easily, the fastest way is to bisect the issue, which you can do on your own. Kind regards, Paul >>>>> [1]: https://bugzilla.kernel.org/ >>>>> [2]: https://kernel.ubuntu.com/~kernel-ppa/mainline/ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C000EC25B4E for ; Sun, 22 Jan 2023 20:28:41 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 51D4481551; Sun, 22 Jan 2023 20:28:41 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 51D4481551 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=osuosl.org; s=default; t=1674419321; bh=oV2RjCPcNYdf+BZo1JPRyH9wpZrtQFhatqZYaGoxc+0=; h=Date:To:References:From:In-Reply-To:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: Cc:From; b=tJQ61Ypo26rdnLv5VOry/3qALmDSEHG9hXhgVORXEFXKgmWlzTlhLJCafEWkMuFEO aJqaQt5jx4NtZ78CvBY+uZLHv6LUVcY5arfYwoz12F63m60aLRm8KW22Mx3mzBgA/n /aYpHevXSkoQS1Epyu5rnzhewev23Tm9xjzxXMX20CO2hTAmGsJsj2KtFrLoEf358Q WFDGASy9t/DASNVGiynONM4sBMW9p06Z0rOlH0SyDem9E9HboB3d/EeeDglz5P/4HZ I+lRGTVtY9hx9vhivQFjex1CxXQ9xyjSVIBeJUd+q4D9dJ5pdyCsqaThM+2JhEhwzt X4//vg1+82Vgg== X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id JDhIr3dR7BG2; Sun, 22 Jan 2023 20:28:40 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp1.osuosl.org (Postfix) with ESMTP id 16AE08149B; Sun, 22 Jan 2023 20:28:40 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org 16AE08149B Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id E40451BF31B for ; Sun, 22 Jan 2023 20:28:37 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id BA17B40382 for ; Sun, 22 Jan 2023 20:28:37 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org BA17B40382 X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Yumc0L-wyyb5 for ; Sun, 22 Jan 2023 20:28:35 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 458A540187 Received: from mx3.molgen.mpg.de (mx3.molgen.mpg.de [141.14.17.11]) by smtp2.osuosl.org (Postfix) with ESMTPS id 458A540187 for ; Sun, 22 Jan 2023 20:28:35 +0000 (UTC) Received: from [192.168.0.2] (ip5f5ae93c.dynamic.kabel-deutschland.de [95.90.233.60]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) (Authenticated sender: pmenzel) by mx.molgen.mpg.de (Postfix) with ESMTPSA id C3AB261CC40F9; Sun, 22 Jan 2023 21:28:32 +0100 (CET) Message-ID: <05d381af-5ccb-0d87-97d3-e2fc4ce870fc@molgen.mpg.de> Date: Sun, 22 Jan 2023 21:28:32 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Content-Language: en-US To: Bartek Kois References: <652bf236-d97e-832c-e0f3-24927a46d7ad@molgen.mpg.de> <744de70c-782d-5d36-87fc-e6b92ac84190@gmail.com> <30de7b89-6a4f-8dab-d671-027140bbb52b@gmail.com> <3b957674-a559-ac1e-27b8-b81e6eeffe75@gmail.com> From: Paul Menzel In-Reply-To: <3b957674-a559-ac1e-27b8-b81e6eeffe75@gmail.com> Subject: Re: [Intel-wired-lan] Supermicro AOC-STGN-I1S (Intel 82599EN based 10G adapter) - poor network perfomance after moving to Debian 11.5 X-BeenThere: intel-wired-lan@osuosl.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel Wired Ethernet Linux Kernel Driver Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: intel-wired-lan@osuosl.org, regressions@lists.linux.dev Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: intel-wired-lan-bounces@osuosl.org Sender: "Intel-wired-lan" RGVhciBCYXJ0ZWssCgoKQW0gMTkuMDEuMjMgdW0gMTg6MTcgc2NocmllYiBCYXJ0ZWsgS29pczoK PiBXIGRuaXUgMTkuMDEuMjAyMyBvwqAxODowOSwgUGF1bCBNZW56ZWwgcGlzemU6Cgo+PiBBbSAx OS4wMS4yMyB1bSAxNzo1OCBzY2hyaWViIEJhcnRlayBLb2lzOgo+Pj4gVyBkbml1IDE5LjAxLjIw MjMgb8KgMTM6MjQsIEJhcnRlayBLb2lzIHBpc3plOgo+Pj4+Cj4+Pj4gVyBkbml1IDE5LjAxLjIw MjMgb8KgMTE6MTcsIFBhdWwgTWVuemVsIHBpc3plOgo+Pj4+Pgo+Pj4+PiAjcmVnemJvdCBeaW50 cm9kdWNlZDogNC45Ljg4Li41LjEwLjE0OQo+Pgo+Pj4+PiBBbSAxNC4wMS4yMyB1bSAxMToyMyBz Y2hyaWViIEJhcnRlayBLb2lzOgo+Pj4+Pgo+Pj4+Pj4gQWZ0ZXIgbW92aW5nIGZyb20gRGViaWFu IDkuNyB0byAxMS41IGFzIHNvb24gYXMgSSBwZXJmb3JtICJpcCBsaW5rIAo+Pj4+Pj4gc2V0IGVu cDFzMCB1cCIgZm9yIG15IDEwRyBhZGFwdGVyIChBT0MtU1RHTi1JMVMgLSBJbnRlbCA4MjU5OUVO IAo+Pj4+Pj4gYmFzZWQgMTBHIGFkYXB0ZXIpIEkgYW0gZXhwZXJpZW5jaW5nIGhpZ2ggY3B1IGxv YWQgKGV2ZW4gaWYgbm8gCj4+Pj4+PiB0cmFmZmljIGlzIHBhc3NpbmcgdGhyb3VnaCB0aGUgYWRh cHRlcikgYW5kIG5ldHdvcmsgcGVyZm9ybWFuY2UgaXMgCj4+Pj4+PiBsb3cgKHdoZW4gbmV0d29y ayBpcyBjb25uZWN0ZWQpLgo+Pj4+Pgo+Pj4+PiBIb3cgZG8geW91IHRlc3QgdGhlIG5ldHdvcmsg cGVyZm9ybWFuY2U/IFBsZWFzZSBnaXZlIGV4YWN0IG51bWJlcnMgCj4+Pj4+IGZvciBjb21wYXJp c29uLgo+Pj4+Pgo+Pj4+IEkgYW0gdXNpbmcgdGhpcyBzZXJ2ZXIgYXMgYSByb3V0ZXIgZm9yIG15 IHN1YnNjcmliZXJzIHdpdGggaXB0YWJsZXMgCj4+Pj4gKGZvciBOQVQgYW5kIGZpcmV3YWxsKSBh bmQgaGZzYyAoZm9yIFFvUykuIEZpcnN0IEkgZW5jb3VudGVyZWQgdGhpcyAKPj4+PiBwcm9ibGVt IHdoaWxlIG1pZ3JhdGluZyBmb3JtIERlYmlhbiA5LjcgdG8gMTEuNS4gUm91dGVycyBiYXNlZMKg IG9uIAo+Pj4+IFN1cGVybWljcm8gWDExU1NMLUYgKEludGVswq4gQzIzMiBjaGlwc2V0KSB3b3Jr cyB3aXRoIG5vIHByb2JsZW1zIAo+Pj4+IGFmdGVyIHRoYXQgbWlncmF0aW9uLCBidXQgcm91dGVy cyBiYXNlZCBvbiBTdXBlcm1pY3JvIFg5U0NMIChJbnRlbCAKPj4+PiBDMjAyIFBDSCkgYW5kIFN1 cGVybWljcm8gWDEwU0xMKy1GIChJbnRlbCBDMjIyIEV4cHJlc3MgUENIKSBzdGFydHMgCj4+Pj4g YmVoYXZpbmcgc3RyYW5nZWx5IHdpdGggaGlnaCBjcHUgbG9hZCAoMC41LTAuOCB3aGlsZSBiZWZv cmUgaXQgd2FzIAo+Pj4+IGFyb3VuZCAwLjAtMC4xKSBhbmQgc3Vic2NyaWJlcnMgbm90IGJlaW5n IGFibGUgdG8gdXRpbGl6ZSB0aGVpciAKPj4+PiBwbGFucy4gSSB0cmllZCB0byBzdHJpcCBkb3du IHRoZSBwcm9ibGVtIGFuZCBlbmRzIHVwIHdpdGggY2xlYW4gCj4+Pj4gc3lzdGVtIHdpdGggbm8g aXB0YWJsZXMgb3IgaGZzYyBydWxlcyBiZWhhdmluZyB0aGUgc2FtZSAoaGlnaGVyIAo+Pj4+IGxv YWQpIHJpZ2h0IGFmdGVyIHNldHRpbmcgdGhlIDEwRyBsaW5rIHVwZXZlbiBpZiBubyB0cmFmZmlj IGlzIAo+Pj4+IHBhc3NpbmcgYnkuCj4+Pj4KPj4+Pj4+IFRoZSBjcHUgbG9hZCBpcyBvc2NpbGxh dGluZyBiZXR3ZWVuIDAuMSBhbmQgMC4zIG9uIHZhbmlsbGEgc3lzdGVtCj4+Pj4+PiB3aXRoIG5v IG5ldHdvcmsgYXR0YWNoZWQuIFRoZSBwcm9ibGVtIGNhbiBiZSBvYnNlcnZlZCBvbiB0aGUgCj4+ Pj4+PiBmb2xsb3dpbmcgcGxhdGZvcm1zOiBTdXBlcm1pY3JvIFg5U0NMIChJbnRlbCBDMjAyIFBD SCkgYW5kCj4+Pj4+PiBTdXBlcm1pY3JvIFgxMFNMTCstRiAoSW50ZWwgQzIyMiBFeHByZXNzIFBD SCksIGJ1dCBmb3IgdGhlIFN1cGVybWljcm8KPj4+Pj4+IFgxMVNTTC1GIChJbnRlbMKuIEMyMzIg Y2hpcHNldCkgZXZlcnl0aGluZyBpcyB3b3JraW5nIHdlbGwuCj4+Pj4+Pgo+Pj4+Pj4gVGVzdGVk IGVudmlyb25tZW50czoKPj4+Pj4+IERlYmlhbiA5LjcgLSBMaW51eCA0LjkuMC02LWFtZDY0ICMx IFNNUCBEZWJpYW4gNC45Ljg4LTErZGViOXUxIAo+Pj4+Pj4gKDIwMTgtMDUtMDcpIHg4Nl82NCBH TlUvTGludXggW2FsbCBwbGF0Zm9ybXMgd29ya2luZyB3ZWxsIHdpdGggbm8gCj4+Pj4+PiBwcm9i bGVtczogU3VwZXJtaWNybyBYOVNDTCAoSW50ZWwgQzIwMiBQQ0gpLCBTdXBlcm1pY3JvIFgxMFNM TCstRiAKPj4+Pj4+IChJbnRlbCBDMjIyIEV4cHJlc3MgUENIKSwgU3VwZXJtaWNybyBYMTFTU0wt RiAoSW50ZWzCriBDMjMyIGNoaXBzZXQpXQo+Pj4+Pgo+Pj4+Pj4gRGViaWFuIDExLjUgLSBMaW51 eCA1LjEwLjAtMTktYW1kNjQgIzEgU01QIERlYmlhbiA1LjEwLjE0OS0yIAo+Pj4+Pj4gKDIwMjIt MTAtMjEpIHg4Nl82NCBHTlUvTGludXggW29sZGVyIHBsYXRmb3JtczogU3VwZXJtaWNybyBYOVND TCAKPj4+Pj4+IChJbnRlbCBDMjAyIFBDSCksIFN1cGVybWljcm8gWDEwU0xMKy1GIChJbnRlbCBD MjIyIEV4cHJlc3MgUENIKSAKPj4+Pj4+IGJlaGF2ZSBwcm9ibGVtYXRpYyBhcyBkZXNjcmliZWQg YWJvdmUgfCBuZXdlciBwbGF0Zm9ybTogU3VwZXJtaWNybyAKPj4+Pj4+IFgxMVNTTC1GIChJbnRl bMKuIEMyMzIgY2hpcHNldCkgd29ya2luZyB3ZWxsIHdpdGggbm8gcHJvYmxlbXNdCj4+Pj4+Cj4+ Pj4+IE1heWJlIGNyZWF0ZSBhIGJ1ZyBhdCB0aGUgTGludXgga2VybmVsIGJ1ZyB0cmFja2VyIFsx XSwgd2hlcmUgeW91IAo+Pj4+PiBjYW4gYXR0YWNoIGFsbCB0aGUgbG9ncyAoYGRtZXNnYCwgYGxz cGNpIC1ubmsgLXMg4oCmYCwg4oCmKS4KPj4+Pj4KPj4+PiBJYHZlIGFscmVhZHkgcmVwb3J0ZWQg dGhhdCB0byB0aGUgRGViaWFuIHRlYW0gCj4+Pj4gdHRwczovL2J1Z3MuZGViaWFuLm9yZy9jZ2kt YmluL2J1Z3JlcG9ydC5jZ2k/YnVnPTEwMjQ3NjMsIGJ1dCBzbyBmYXIgCj4+Pj4gbm9ib2R5IHRv b2sgY2FyZSBvZiB0aGlzIGlzc3VlIHNvIGZhci4KPj4+Pgo+Pj4+Pj4gU28gZmFyIHRvIHNvbHZl IHRoZSBwcm9ibGVtIEkgd2FzIHRyeWluZyB0byB1cGdyYWRlIHN5c3RlbSB0byB0aGUgCj4+Pj4+ PiBuZXdlc3Qgc3RhYmxlIHZlcnNpb24sIHVwZ3JhZGUga2VybmVsIHRvIHZlcnNpb24gNi54LCB1 cGdyYWRlIAo+Pj4+Pj4gaXhnYmUgZHJpdmVyIHRvIHRoZSBuZXdlc3QgdmVyc2lvbiBidXQgd2l0 aCBubyBsdWNrLgo+Pj4+Pgo+Pj4+PiBUaGFuayB5b3UgZm9yIGNoZWNraW5nIHRoYXQuIFRvbyBi YWQgaXTigJlzIHN0aWxsIHByZXNlbnQuIFRvIHJ1bGUgCj4+Pj4+IG91dCBzb21lIHVzZXIgc3Bh Y2UgcHJvYmxlbSwgY291bGQgeW91IHRlc3QgRGViaWFuIDkuNyB3aXRoIGEgCj4+Pj4+IHN0YWJs ZSBMaW51eCByZWxlYXNlLCBjdXJyZW50bHkgNi4xLjc/Cj4+Pj4+Cj4+Pj4+IFdoYXQgZG9lcyBg c3VkbyBwZXJmIHRvcCAtLXNvcnQgY29tbSxkc29gIHNob3csIHdoZXJlIHRoZSB0aW1lIGlzIAo+ Pj4+PiBzcGVudD8KPj4+Pgo+Pj4+IER1cmluZyBteSBmaXJzdCB0ZXN0IGluIHJlYWwgZW52aXJv bWVudCB3aXRoIHN1YnNjcmliZXJzIEkgZ2V0aGVyIAo+Pj4+IHRoZSBmb2xsb3dpbmcgZGF0YSB0 aHJvdWdoIHRoZSBwZXJmOgo+Pj4+Cj4+Pj4gwqAgMjcuODMlwqAgW2tlcm5lbF3CoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgW2tdIHN0cm5jcHkKPj4+PiDCoCAxNC44MCXCoCBb a2VybmVsXcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBba10gbmZ0X2RvX2No YWluCj4+Pj4gwqDCoCA3LjYxJcKgIFtrZXJuZWxdwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIFtrXSBtZW1jbXAKPj4+PiDCoMKgIDUuNjMlwqAgW2tlcm5lbF3CoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgW2tdIG5mdF9tZXRhX2dldF9ldmFsCj4+Pj4gwqDC oCAzLjE0JcKgIFtrZXJuZWxdwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFtr XSBuZnRfY21wX2V2YWwKPj4+PiDCoMKgIDIuNzklwqAgW2tlcm5lbF3CoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqAgW2tdIGFzbV9leGNfbm1pCj4+Pj4gwqDCoCAxLjA3JcKgIFtr ZXJuZWxdwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFtrXSBtb2R1bGVfZ2V0 X2thbGxzeW0KPj4+PiDCoMKgIDAuOTIlwqAgW2tlcm5lbF3CoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgW2tdIGthbGxzeW1zX2V4cGFuZF9zeW1ib2wuY29uc3Rwcm9wLjAKPj4+ PiDCoMKgIDAuODUlwqAgW2tlcm5lbF3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgW2tdIGl4Z2JlX3BvbGwKPj4+PiDCoMKgIDAuNzUlwqAgW2tlcm5lbF3CoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgW2tdIGZvcm1hdF9kZWNvZGUKPj4+PiDCoMKgIDAuNjEl wqAgW2tlcm5lbF3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgW2tdIG51bWJl cgo+Pj4+IMKgwqAgMC41NiXCoCBba2VybmVsXcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCBba10gbWVudV9zZWxlY3QKPj4+PiDCoMKgIDAuNTQlwqAgW2tlcm5lbF3CoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgW2tdIGNsZmx1c2hfY2FjaGVfcmFuZ2UKPj4+ PiDCoMKgIDAuNTIlwqAgW2tlcm5lbF3CoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgW2tdIGNwdWlkbGVfZW50ZXJfc3RhdGUKPj4+PiDCoMKgIDAuNTElwqAgW2tlcm5lbF3CoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgW2tdIHZzbnByaW50Zgo+Pj4+IMKgwqAg MC41MCXCoCBba2VybmVsXcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBba10g dTMyX2NsYXNzaWZ5Cj4+Pj4gwqDCoCAwLjQ5JcKgIFtrZXJuZWxdwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIFtrXSBmaWJfdGFibGVfbG9va3VwCj4+Pj4gwqDCoCAwLjQwJcKg IFtrZXJuZWxdwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIFtrXSBkbWFfcHRl X2NsZWFyX2xldmVsCj4+Pj4gwqDCoCAwLjM5JcKgIFtrZXJuZWxdwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIFtrXSBkb21haW5fbWFwcGluZwo+Pj4+IMKgwqAgMC4zNiXCoCBb a2VybmVsXcKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBba10gaXhnYmVfeG1p dF9mcmFtCj4+Pj4KPj4+Pgo+Pj4+IMKgwqDCoCBQSUQgVVNFUsKgwqDCoMKgwqAgUFLCoCBOScKg wqDCoCBWSVJUwqDCoMKgIFJFU8KgwqDCoCBTSFIgU8KgICVDUFXCoCAlTUVNIFRJTUUrIENPTU1B TkQKPj4+PiDCoMKgwqDCoCAxOCByb290wqDCoMKgwqDCoCAyMMKgwqAgMMKgwqDCoMKgwqDCoCAw wqDCoMKgwqDCoCAwwqDCoMKgwqDCoCAwIFPCoCAyOC4ywqDCoCAwLjAgNzowNi4yNyBrc29mdGly cWQvMQo+Pj4+IMKgwqDCoMKgIDEyIHJvb3TCoMKgwqDCoMKgIDIwwqDCoCAwwqDCoMKgwqDCoMKg IDDCoMKgwqDCoMKgIDDCoMKgwqDCoMKgIDAgUsKgIDEyLjDCoMKgIDAuMCA0OjEwLjg4IGtzb2Z0 aXJxZC8wCj4+Cj4+IFvigKZdCj4+Cj4+IERvIHlvdSBzZWUgZGlmZmVyZW50IGJlaGF2aW9yIGlu IGAvcHJvYy9pbnRlcnJ1cHRzYD8KPj4KPiBUaGlzIGlzIGhvdyBpdCBsb29rcyBsaWtlIGZvciBE ZWJpYW4gMTEuNSAtIExpbnV4IDUuMTAuMC0xOS1hbWQ2NCAjMSBTTVAgCj4gRGViaWFuIDUuMTAu MTQ5LTIgKDIwMjItMTAtMjEpIHg4Nl82NCBHTlUvTGludXggb24gU3VwZXJtaWNybyBYMTBTTEwr LUYgCj4gKEludGVsIEMyMjIgRXhwcmVzcyBQQ0gpOgo+IAo+ICDCoMKgwqDCoMKgIDEgcm9vdMKg wqDCoMKgwqAgMjDCoMKgIDDCoCAxNjM5NDjCoCAxMDI4OMKgwqAgNzY5NiBTwqDCoCAwLjDCoMKg IDAuMSAwOjM5LjU4IHN5c3RlbWQKClvigKZdCgpUaGUgY29udGVudCBvZiBgL3Byb2MvaW50ZXJy dXB0c2AgaGFzIGEgZGlmZmVyZW50IGZvcm1hdCBvbiBteSBzeXN0ZW0uCgpgYGAKJCBoZWFkIC0z IC9wcm9jL2ludGVycnVwdHMKICAgICAgICAgICAgQ1BVMCAgICAgICBDUFUxICAgICAgIENQVTIg ICAgICAgQ1BVMwogICAxOiAgICAgIDU1NTYwICAgICAgICAgIDAgICAgICAgIDExMyAgICAgICAg ICAwICBJUi1JTy1BUElDICAgMS1lZGdlIAogICAgaTgwNDIKICAgODogICAgICAgICAgMCAgICAg ICAgICAwICAgICAgICAgIDAgICAgICAgICAgMCAgSVItSU8tQVBJQyAgIDgtZWRnZSAKICAgIHJ0 YzAKYGBgClvigKZdCgo+IGFuZCBmb3IgRGViaWFuIDkuNyAtIExpbnV4IDQuOS4wLTYtYW1kNjQg IzEgU01QIERlYmlhbiA0LjkuODgtMStkZWI5dTEgCj4gb24gU3VwZXJtaWNybyBYMTBTTEwrLUYg KEludGVsIEMyMjIgRXhwcmVzcyBQQ0gpCj4gCj4gMzE2NTkgcm9vdMKgwqDCoMKgwqAgMjDCoMKg IDDCoMKgwqDCoMKgwqAgMMKgwqDCoMKgwqAgMMKgwqDCoMKgwqAgMCBTwqDCoCAwLjPCoCAwLjAg MDowMC45MiAKPiBrd29ya2VyLzc6MAo+ICDCoMKgwqAgMSByb290wqDCoMKgwqDCoCAyMMKgwqAg MMKgwqAgNTcwMzLCoMKgIDY3MzbCoMKgIDUyNTYgU8KgwqAgMC4wwqAgMC4xIDI6MjguMTQgc3lz dGVtZAoKW+KApl0KPj4+Pj4+IFN1cGVybWljcm8gc3VwcG9ydCBzdWdnZXN0ZWQgYXMgZm9sbG93 czoKPj4+Pj4+IGl0IG1pZ2h0IGJlIGtlcm5lbCByZWxhdGVkIGRlYmlhbiAxMS41IGhhcyBrZXJu ZWwgNS4xMCB3aGljaCBpcyBhIAo+Pj4+Pj4gcmVjZW50IGtlcm5lbCBpdCBtaWdodCBub3QgcHJv cGVybHkgc3VwcG9ydCB0aGUgY2hpcHNldHMgZm9yIFg5IAo+Pj4+Pj4gdGhlcmVmb3JlIGkgc3Vn Z2VzdCB0byB1c2UgUkhFTCBvciBDZW50T1MgYXMgdGhleSB1c2UgbXVjaCBvbGRlciAKPj4+Pj4+ IGtlcm5lbCB2ZXJzaW9ucy4gSSBleHBlY3QgdGhhdCB3aXRoIHVidW50dSAyMC4wNCB5b3Ugc2Vl IHRoZSBzYW1lIAo+Pj4+Pj4gcHJvYmxlbSBpdCB1c2VzIGtlcm5lbCA1LjQKPj4+Pj4gPj4+IFRl c3RpbmcgYW5vdGhlciBHTlUvTGludXggZGlzdHJpYnV0aW9uIGZvciBhbm90aGVyIGRhdGEgcG9p bnQsIAo+Pj4+PiBtaWdodAo+Pj4+PiBiZSBhIGdvb2QgaWRlYS4KPj4+Pj4KPj4+Pj4gQXMgbm9i b2R5IGhhcyByZXNwb25kZWQgeWV0LCBiaXNlY3RpbmcgdGhlIGlzc3VlIGlzIHByb2JhYmx5IHRo ZSAKPj4+Pj4gZmFzdGVzdCB3YXkgdG8gZ2V0IHRvIHRoZSBib3R0b20gb2YgdGhpcy4gTHVja2ls eSB0aGUgcHJvYmxlbSBzZWVtcyAKPj4+Pj4gcmVwcm9kdWNpYmxlIGFuZCB5b3Ugc2VlbSB0byBi ZSBhYmxlIHRvIGJ1aWxkIGEgTGludXgga2VybmVsIAo+Pj4+PiB5b3Vyc2VsZiwgc28gdGhhdCBz aG91bGQgd29yay4gKEZvciB0ZXN0aW5nIHB1cnBvc2VzIHlvdSBjb3VsZCBhbHNvIAo+Pj4+PiB0 ZXN0IHdpdGggVWJ1bnR1LCBhcyB0aGV5IHByb3ZpZGUgTGludXgga2VybmVsIGJ1aWxkcyBmb3Ig KGFsbW9zdCkgCj4+Pj4+IGFsbCByZWxlYXNlcyBpbiB0aGVpciBMaW51eCBrZXJuZWwgbWFpbmxp bmUgUFBBIFsyXS4pCj4+Pj4+Cj4+Pj4gT2YgY291cnNlwqAgSSBjYW4gdHJ5IFVidW50dSBhbmQg cmVwb3J0IGhvdyBpdCBpcyB3b3JraW5nLgo+Pj4+Cj4+PiBVYnVudHUgKDUuMTUuMC00My1nZW5l cmljKSBzZWVtcyB0byBiZSB3b3JraW5nIGluIHRoZSBzYW1lIHdheSAKPj4+IGdlbmVyYXRpbmcg aGlnaGVyIGxvYWQgYWZ0ZXIgZXhlY3V0aW5nICJpcCBsaW5rIHNldCBlbnAxczAgdXAiLgo+Pgo+ PiBUaGF0IGlzIGdvb2QgdG8ga25vdy4gKElzIHRoaXMgVWJ1bnR1IDIyLjA0PykgV2hhdCBhYm91 dCBVYnVudHUgMjAuMDQgCj4+IHdpdGggTGludXggNS40LCBhbmQgVWJ1bnR1IDE4LjA0IHdpdGgg NC4xNT8KPj4KPj4gQW55d2F5LCBJIHRoaW5rLCB5b3Ugd29u4oCZdCBjb21lIGFyb3VuZCBiaXNl Y3RpbmcuIEFub3RoZXIgaGludCwgbWFrZSAKPj4gc3VyZSB0aGF0IHlvdSBjYW4gYnVpbGQgYSA0 LjkgTGludXgga2VybmVsIHlvdXJzZWxmLCB0aGF0IGRvZXMgbm90IAo+PiBleGhpYml0IHRoYXQg aXNzdWUuCj4+Cj4gVGhhdGBzIHJpZ2h0LCBpdCBpcyAyMi4wNC4gSSBkb25gdCBoYXZlIHRvIGJ1 aWxkIGl0LiBTdGFuZGFyZCBrZXJuZWwgCj4gTGludXggNC45LjAtNi1hbWQ2NCBmcm9tIERlYmlh biA5Ljcgd29ya2VkIHdpdGhvdXQgcHJvYmxlbXMgZm9yIHBhc3QgNCAKPiB5ZWFycy4KCklmIG5v Ym9keSBvZiB0aGUgZGV2ZWxvcGVycy9tYWludGFpbmVycyBpcyBnb2luZyB0byBzdGVwIHVwLCB5 b3UgYXJlIG9uIAp5b3VyIG93bi4gQWdhaW4sIGFzIHlvdSBjYW4gcmVwcm9kdWNlIHRoaXMgZWFz aWx5LCB0aGUgZmFzdGVzdCB3YXkgaXMgdG8gCmJpc2VjdCB0aGUgaXNzdWUsIHdoaWNoIHlvdSBj YW4gZG8gb24geW91ciBvd24uCgoKS2luZCByZWdhcmRzLAoKUGF1bAoKCj4+Pj4+IFsxXTogaHR0 cHM6Ly9idWd6aWxsYS5rZXJuZWwub3JnLwo+Pj4+PiBbMl06IGh0dHBzOi8va2VybmVsLnVidW50 dS5jb20vfmtlcm5lbC1wcGEvbWFpbmxpbmUvCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCkludGVsLXdpcmVkLWxhbiBtYWlsaW5nIGxpc3QKSW50ZWwtd2ly ZWQtbGFuQG9zdW9zbC5vcmcKaHR0cHM6Ly9saXN0cy5vc3Vvc2wub3JnL21haWxtYW4vbGlzdGlu Zm8vaW50ZWwtd2lyZWQtbGFuCg==