From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dave Taht Subject: Re: OpenWRT wrong adjustment of fq_codel defaults (Was: [Codel] fq_codel_drop vs a udp flood) Date: Mon, 16 May 2016 09:04:01 -0700 Message-ID: References: <1462125592.5535.194.camel@edumazet-glaptop3.roam.corp.google.com> <865DA393-262D-40B6-A9D3-1B978CD5F6C6@gmail.com> <1462128385.5535.200.camel@edumazet-glaptop3.roam.corp.google.com> <1462136140.5535.219.camel@edumazet-glaptop3.roam.corp.google.com> <1462201620.5535.250.camel@edumazet-glaptop3.roam.corp.google.com> <1462205669.5535.254.camel@edumazet-glaptop3.roam.corp.google.com> <1462464776.13075.18.camel@edumazet-glaptop3.roam.corp.google.com> <1462476207.13075.20.camel@edumazet-glaptop3.roam.corp.google.com> <20160506114243.4eb4f95e@redhat.com> <20160506144740.210901f5@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: Rajkumar Manoharan , Michal Kazior , Jesper Dangaard Brouer , Felix Fietkau , Jonathan Morton , "codel@lists.bufferbloat.net" , ath10k , make-wifi-fast@lists.bufferbloat.net, =?UTF-8?B?UmFmYcWCIE1pxYJlY2tp?= , "netdev@vger.kernel.org" , OpenWrt Development List To: Roman Yeryomin Return-path: Received: from mail-oi0-f47.google.com ([209.85.218.47]:34710 "EHLO mail-oi0-f47.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753654AbcEPQEE convert rfc822-to-8bit (ORCPT ); Mon, 16 May 2016 12:04:04 -0400 Received: by mail-oi0-f47.google.com with SMTP id k142so275146414oib.1 for ; Mon, 16 May 2016 09:04:03 -0700 (PDT) In-Reply-To: Sender: netdev-owner@vger.kernel.org List-ID: On Mon, May 16, 2016 at 1:14 AM, Roman Yeryomin = wrote: > On 16 May 2016 at 01:34, Roman Yeryomin wrote= : >> On 6 May 2016 at 22:43, Dave Taht wrote: >>> On Fri, May 6, 2016 at 11:56 AM, Roman Yeryomin wrote: >>>> On 6 May 2016 at 21:43, Roman Yeryomin wro= te: >>>>> On 6 May 2016 at 15:47, Jesper Dangaard Brouer wrote: >>>>>> >>>>>> I've created a OpenWRT ticket[1] on this issue, as it seems that= someone[2] >>>>>> closed Felix'es OpenWRT email account (bad choice! emails bounci= ng). >>>>>> Sounds like OpenWRT and the LEDE https://www.lede-project.org/ p= roject >>>>>> is in some kind of conflict. >>>>>> >>>>>> OpenWRT ticket [1] https://dev.openwrt.org/ticket/22349 >>>>>> >>>>>> [2] http://thread.gmane.org/gmane.comp.embedded.openwrt.devel/40= 298/focus=3D40335 >>>>> >>>>> OK, so, after porting the patch to 4.1 openwrt kernel and playing= a >>>>> bit with fq_codel limits I was able to get 420Mbps UDP like this: >>>>> tc qdisc replace dev wlan0 parent :1 fq_codel flows 16 limit 256 >>>> >>>> Forgot to mention, I've reduced drop_batch_size down to 32 >>> >>> 0) Not clear to me if that's the right line, there are 4 wifi queue= s, >>> and the third one >>> is the BE queue. >> >> That was an example, sorry, should have stated that. I've applied sa= me >> settings to all 4 queues. >> >>> That is too low a limit, also, for normal use. And: >>> for the purpose of this particular UDP test, flows 16 is ok, but no= t >>> ideal. >> >> I played with different combinations, it doesn't make any >> (significant) difference: 20-30Mbps, not more. >> What numbers would you propose? >> >>> 1) What's the tcp number (with a simultaneous ping) with this lates= t patchset? >>> (I care about tcp performance a lot more than udp floods - survivin= g a >>> udp flood yes, performance, no) >> >> During the test (both TCP and UDP) it's roughly 5ms in average, not >> running tests ~2ms. Actually I'm now wondering if target is working = at >> all, because I had same result with target 80ms.. >> So, yes, latency is good, but performance is poor. >> >>> before/after? >>> >>> tc -s qdisc show dev wlan0 during/after results? >> >> during the test: >> >> qdisc mq 0: root >> Sent 1600496000 bytes 1057194 pkt (dropped 1421568, overlimits 0 re= queues 17) >> backlog 1545794b 1021p requeues 17 >> qdisc fq_codel 8001: parent :1 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >> backlog 0b 0p requeues 0 >> maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 >> new_flows_len 0 old_flows_len 0 >> qdisc fq_codel 8002: parent :2 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >> backlog 0b 0p requeues 0 >> maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 >> new_flows_len 0 old_flows_len 0 >> qdisc fq_codel 8003: parent :3 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 1601271168 bytes 1057706 pkt (dropped 1422304, overlimits 0 re= queues 17) >> backlog 1541252b 1018p requeues 17 >> maxpacket 1514 drop_overlimit 1422304 new_flow_count 35 ecn_mark 0 >> new_flows_len 0 old_flows_len 1 >> qdisc fq_codel 8004: parent :4 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >> backlog 0b 0p requeues 0 >> maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 >> new_flows_len 0 old_flows_len 0 >> >> >> after the test (60sec): >> >> qdisc mq 0: root >> Sent 3084996052 bytes 2037744 pkt (dropped 2770176, overlimits 0 re= queues 28) >> backlog 0b 0p requeues 28 >> qdisc fq_codel 8001: parent :1 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >> backlog 0b 0p requeues 0 >> maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 >> new_flows_len 0 old_flows_len 0 >> qdisc fq_codel 8002: parent :2 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >> backlog 0b 0p requeues 0 >> maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 >> new_flows_len 0 old_flows_len 0 >> qdisc fq_codel 8003: parent :3 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 3084996052 bytes 2037744 pkt (dropped 2770176, overlimits 0 re= queues 28) >> backlog 0b 0p requeues 28 >> maxpacket 1514 drop_overlimit 2770176 new_flow_count 64 ecn_mark 0 >> new_flows_len 0 old_flows_len 1 >> qdisc fq_codel 8004: parent :4 limit 1024p flows 16 quantum 1514 >> target 80.0ms ce_threshold 32us interval 100.0ms ecn >> Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) >> backlog 0b 0p requeues 0 >> maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 >> new_flows_len 0 old_flows_len 0 >> >> >>> IF you are doing builds for the archer c7v2, I can join in on this.= =2E. (?) >> >> I'm not but I have c7 somewhere, so I can do a build for it and also >> test, so we are on the same page. >> >>> I did do a test of the ath10k "before", fq_codel *never engaged*, a= nd >>> tcp induced latencies under load, e at 100mbit, cracked 600ms, whil= e >>> staying flat (20ms) at 100mbit. (not the same patches you are testi= ng) >>> on x86. I have got tcp 300Mbit out of an osx box, similar latency, >>> have yet to get anything more on anything I currently have >>> before/after patchsets. >>> >>> I'll go add flooding to the tests, I just finished a series compari= ng >>> two different speed stations and life was good on that. >>> >>> "before" - fq_codel never engages, we see seconds of latency under = load. >>> >>> root@apu2:~# tc -s qdisc show dev wlp4s0 >>> qdisc mq 0: root >>> Sent 8570563893 bytes 6326983 pkt (dropped 0, overlimits 0 requeue= s 0) >>> backlog 0b 0p requeues 0 >>> qdisc fq_codel 0: parent :1 limit 10240p flows 1024 quantum 1514 >>> target 5.0ms interval 100.0ms ecn >>> Sent 2262 bytes 17 pkt (dropped 0, overlimits 0 requeues 0) >>> backlog 0b 0p requeues 0 >>> maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0 >>> new_flows_len 0 old_flows_len 0 >>> qdisc fq_codel 0: parent :2 limit 10240p flows 1024 quantum 1514 >>> target 5.0ms interval 100.0ms ecn >>> Sent 220486569 bytes 152058 pkt (dropped 0, overlimits 0 requeues = 0) >>> backlog 0b 0p requeues 0 >>> maxpacket 18168 drop_overlimit 0 new_flow_count 1 ecn_mark 0 >>> new_flows_len 0 old_flows_len 1 >>> qdisc fq_codel 0: parent :3 limit 10240p flows 1024 quantum 1514 >>> target 5.0ms interval 100.0ms ecn >>> Sent 8340546509 bytes 6163431 pkt (dropped 0, overlimits 0 requeue= s 0) >>> backlog 0b 0p requeues 0 >>> maxpacket 68130 drop_overlimit 0 new_flow_count 120050 ecn_mark 0 >>> new_flows_len 1 old_flows_len 3 >>> qdisc fq_codel 0: parent :4 limit 10240p flows 1024 quantum 1514 >>> target 5.0ms interval 100.0ms ecn >>> Sent 9528553 bytes 11477 pkt (dropped 0, overlimits 0 requeues 0) >>> backlog 0b 0p requeues 0 >>> maxpacket 66 drop_overlimit 0 new_flow_count 1 ecn_mark 0 >>> new_flows_len 1 old_flows_len 0 >>> ``` >>> >>> >>>>> This is certainly better than 30Mbps but still more than two time= s >>>>> less than before (900). >>> >>> The number that I still am not sure we got is that you were sending >>> 900mbit udp and recieving 900mbit on the prior tests? >> >> 900 was sending, AP POV (wifi client is downloading) >> >>>>> TCP also improved a little (550 to ~590). >>> >>> The limit is probably a bit low, also. You might want to try targe= t >>> 20ms as well. >> >> I've tried limit up to 1024 and target up to 80ms >> >>>>> >>>>> Felix, others, do you want to see the ported patch, maybe I did s= omething wrong? >>>>> Doesn't look like it will save ath10k from performance regression= =2E >>> >>> what was tcp "before"? (I'm sorry, such a long thread) >> >> 750Mbps > > Michal, after retesting with your patch (sorry, it was late yesterday= , > confused compat-wireless archives) I saw the difference. > So the progress looks like this (all with fq_codel flows 16 limit 102= 4 > target 20ms): > no patches: 380Mbps UDP, 550 TCP > Eric's (fq_codel drop) patch: 420Mbps UDP, 590 TCP (+40Mbps), latency > 5-6ms during test > Michal's (improve tx scheduling) patch: 580Mbps UDP, 660 TCP, latency > up to 30-40ms during test > after Rajkumar's proposal to "try without registering wake_tx_queue > callback": 820Mbps UDP, 690 TCP. And the simultaneous ping on the last test was? > So, very close to "as before": 900Mbps UDP, 750 TCP. > But still, I was expecting performance improvements from latest ath10= k > code, not regressions. > I know that hw is capable of 800Mbps TCP, which I'm targeting. > > Regards, > Roman > > p.s. sorry for confusion --=20 Dave T=C3=A4ht Let's go make home routers and wifi faster! With better software! http://blog.cerowrt.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from mail-oi0-x234.google.com ([2607:f8b0:4003:c06::234]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1b2L0J-0000J4-Ot for ath10k@lists.infradead.org; Mon, 16 May 2016 16:04:25 +0000 Received: by mail-oi0-x234.google.com with SMTP id v145so274466348oie.0 for ; Mon, 16 May 2016 09:04:03 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: References: <1462125592.5535.194.camel@edumazet-glaptop3.roam.corp.google.com> <865DA393-262D-40B6-A9D3-1B978CD5F6C6@gmail.com> <1462128385.5535.200.camel@edumazet-glaptop3.roam.corp.google.com> <1462136140.5535.219.camel@edumazet-glaptop3.roam.corp.google.com> <1462201620.5535.250.camel@edumazet-glaptop3.roam.corp.google.com> <1462205669.5535.254.camel@edumazet-glaptop3.roam.corp.google.com> <1462464776.13075.18.camel@edumazet-glaptop3.roam.corp.google.com> <1462476207.13075.20.camel@edumazet-glaptop3.roam.corp.google.com> <20160506114243.4eb4f95e@redhat.com> <20160506144740.210901f5@redhat.com> Date: Mon, 16 May 2016 09:04:01 -0700 Message-ID: Subject: Re: OpenWRT wrong adjustment of fq_codel defaults (Was: [Codel] fq_codel_drop vs a udp flood) From: Dave Taht List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "ath10k" Errors-To: ath10k-bounces+kvalo=adurom.com@lists.infradead.org To: Roman Yeryomin Cc: Rajkumar Manoharan , make-wifi-fast@lists.bufferbloat.net, =?UTF-8?B?UmFmYcWCIE1pxYJlY2tp?= , ath10k , "netdev@vger.kernel.org" , "codel@lists.bufferbloat.net" , Michal Kazior , Jesper Dangaard Brouer , Jonathan Morton , OpenWrt Development List , Felix Fietkau T24gTW9uLCBNYXkgMTYsIDIwMTYgYXQgMToxNCBBTSwgUm9tYW4gWWVyeW9taW4gPGxlcm9pLmxp c3RzQGdtYWlsLmNvbT4gd3JvdGU6Cj4gT24gMTYgTWF5IDIwMTYgYXQgMDE6MzQsIFJvbWFuIFll cnlvbWluIDxsZXJvaS5saXN0c0BnbWFpbC5jb20+IHdyb3RlOgo+PiBPbiA2IE1heSAyMDE2IGF0 IDIyOjQzLCBEYXZlIFRhaHQgPGRhdmUudGFodEBnbWFpbC5jb20+IHdyb3RlOgo+Pj4gT24gRnJp LCBNYXkgNiwgMjAxNiBhdCAxMTo1NiBBTSwgUm9tYW4gWWVyeW9taW4gPGxlcm9pLmxpc3RzQGdt YWlsLmNvbT4gd3JvdGU6Cj4+Pj4gT24gNiBNYXkgMjAxNiBhdCAyMTo0MywgUm9tYW4gWWVyeW9t aW4gPGxlcm9pLmxpc3RzQGdtYWlsLmNvbT4gd3JvdGU6Cj4+Pj4+IE9uIDYgTWF5IDIwMTYgYXQg MTU6NDcsIEplc3BlciBEYW5nYWFyZCBCcm91ZXIgPGJyb3VlckByZWRoYXQuY29tPiB3cm90ZToK Pj4+Pj4+Cj4+Pj4+PiBJJ3ZlIGNyZWF0ZWQgYSBPcGVuV1JUIHRpY2tldFsxXSBvbiB0aGlzIGlz c3VlLCBhcyBpdCBzZWVtcyB0aGF0IHNvbWVvbmVbMl0KPj4+Pj4+IGNsb3NlZCBGZWxpeCdlcyBP cGVuV1JUIGVtYWlsIGFjY291bnQgKGJhZCBjaG9pY2UhIGVtYWlscyBib3VuY2luZykuCj4+Pj4+ PiBTb3VuZHMgbGlrZSBPcGVuV1JUIGFuZCB0aGUgTEVERSBodHRwczovL3d3dy5sZWRlLXByb2pl Y3Qub3JnLyBwcm9qZWN0Cj4+Pj4+PiBpcyBpbiBzb21lIGtpbmQgb2YgY29uZmxpY3QuCj4+Pj4+ Pgo+Pj4+Pj4gT3BlbldSVCB0aWNrZXQgWzFdIGh0dHBzOi8vZGV2Lm9wZW53cnQub3JnL3RpY2tl dC8yMjM0OQo+Pj4+Pj4KPj4+Pj4+IFsyXSBodHRwOi8vdGhyZWFkLmdtYW5lLm9yZy9nbWFuZS5j b21wLmVtYmVkZGVkLm9wZW53cnQuZGV2ZWwvNDAyOTgvZm9jdXM9NDAzMzUKPj4+Pj4KPj4+Pj4g T0ssIHNvLCBhZnRlciBwb3J0aW5nIHRoZSBwYXRjaCB0byA0LjEgb3BlbndydCBrZXJuZWwgYW5k IHBsYXlpbmcgYQo+Pj4+PiBiaXQgd2l0aCBmcV9jb2RlbCBsaW1pdHMgSSB3YXMgYWJsZSB0byBn ZXQgNDIwTWJwcyBVRFAgbGlrZSB0aGlzOgo+Pj4+PiB0YyBxZGlzYyByZXBsYWNlIGRldiB3bGFu MCBwYXJlbnQgOjEgZnFfY29kZWwgZmxvd3MgMTYgbGltaXQgMjU2Cj4+Pj4KPj4+PiBGb3Jnb3Qg dG8gbWVudGlvbiwgSSd2ZSByZWR1Y2VkIGRyb3BfYmF0Y2hfc2l6ZSBkb3duIHRvIDMyCj4+Pgo+ Pj4gMCkgTm90IGNsZWFyIHRvIG1lIGlmIHRoYXQncyB0aGUgcmlnaHQgbGluZSwgdGhlcmUgYXJl IDQgd2lmaSBxdWV1ZXMsCj4+PiBhbmQgdGhlIHRoaXJkIG9uZQo+Pj4gaXMgdGhlIEJFIHF1ZXVl Lgo+Pgo+PiBUaGF0IHdhcyBhbiBleGFtcGxlLCBzb3JyeSwgc2hvdWxkIGhhdmUgc3RhdGVkIHRo YXQuIEkndmUgYXBwbGllZCBzYW1lCj4+IHNldHRpbmdzIHRvIGFsbCA0IHF1ZXVlcy4KPj4KPj4+ IFRoYXQgaXMgdG9vIGxvdyBhIGxpbWl0LCBhbHNvLCBmb3Igbm9ybWFsIHVzZS4gQW5kOgo+Pj4g Zm9yIHRoZSBwdXJwb3NlIG9mIHRoaXMgcGFydGljdWxhciBVRFAgdGVzdCwgZmxvd3MgMTYgaXMg b2ssIGJ1dCBub3QKPj4+IGlkZWFsLgo+Pgo+PiBJIHBsYXllZCB3aXRoIGRpZmZlcmVudCBjb21i aW5hdGlvbnMsIGl0IGRvZXNuJ3QgbWFrZSBhbnkKPj4gKHNpZ25pZmljYW50KSBkaWZmZXJlbmNl OiAyMC0zME1icHMsIG5vdCBtb3JlLgo+PiBXaGF0IG51bWJlcnMgd291bGQgeW91IHByb3Bvc2U/ Cj4+Cj4+PiAxKSBXaGF0J3MgdGhlIHRjcCBudW1iZXIgKHdpdGggYSBzaW11bHRhbmVvdXMgcGlu Zykgd2l0aCB0aGlzIGxhdGVzdCBwYXRjaHNldD8KPj4+IChJIGNhcmUgYWJvdXQgdGNwIHBlcmZv cm1hbmNlIGEgbG90IG1vcmUgdGhhbiB1ZHAgZmxvb2RzIC0gc3Vydml2aW5nIGEKPj4+IHVkcCBm bG9vZCB5ZXMsIHBlcmZvcm1hbmNlLCBubykKPj4KPj4gRHVyaW5nIHRoZSB0ZXN0IChib3RoIFRD UCBhbmQgVURQKSBpdCdzIHJvdWdobHkgNW1zIGluIGF2ZXJhZ2UsIG5vdAo+PiBydW5uaW5nIHRl c3RzIH4ybXMuIEFjdHVhbGx5IEknbSBub3cgd29uZGVyaW5nIGlmIHRhcmdldCBpcyB3b3JraW5n IGF0Cj4+IGFsbCwgYmVjYXVzZSBJIGhhZCBzYW1lIHJlc3VsdCB3aXRoIHRhcmdldCA4MG1zLi4K Pj4gU28sIHllcywgbGF0ZW5jeSBpcyBnb29kLCBidXQgcGVyZm9ybWFuY2UgaXMgcG9vci4KPj4K Pj4+IGJlZm9yZS9hZnRlcj8KPj4+Cj4+PiB0YyAtcyBxZGlzYyBzaG93IGRldiB3bGFuMCBkdXJp bmcvYWZ0ZXIgcmVzdWx0cz8KPj4KPj4gZHVyaW5nIHRoZSB0ZXN0Ogo+Pgo+PiBxZGlzYyBtcSAw OiByb290Cj4+ICBTZW50IDE2MDA0OTYwMDAgYnl0ZXMgMTA1NzE5NCBwa3QgKGRyb3BwZWQgMTQy MTU2OCwgb3ZlcmxpbWl0cyAwIHJlcXVldWVzIDE3KQo+PiAgYmFja2xvZyAxNTQ1Nzk0YiAxMDIx cCByZXF1ZXVlcyAxNwo+PiBxZGlzYyBmcV9jb2RlbCA4MDAxOiBwYXJlbnQgOjEgbGltaXQgMTAy NHAgZmxvd3MgMTYgcXVhbnR1bSAxNTE0Cj4+IHRhcmdldCA4MC4wbXMgY2VfdGhyZXNob2xkIDMy dXMgaW50ZXJ2YWwgMTAwLjBtcyBlY24KPj4gIFNlbnQgMCBieXRlcyAwIHBrdCAoZHJvcHBlZCAw LCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMCkKPj4gIGJhY2tsb2cgMGIgMHAgcmVxdWV1ZXMgMAo+ PiAgIG1heHBhY2tldCAwIGRyb3Bfb3ZlcmxpbWl0IDAgbmV3X2Zsb3dfY291bnQgMCBlY25fbWFy ayAwCj4+ICAgbmV3X2Zsb3dzX2xlbiAwIG9sZF9mbG93c19sZW4gMAo+PiBxZGlzYyBmcV9jb2Rl bCA4MDAyOiBwYXJlbnQgOjIgbGltaXQgMTAyNHAgZmxvd3MgMTYgcXVhbnR1bSAxNTE0Cj4+IHRh cmdldCA4MC4wbXMgY2VfdGhyZXNob2xkIDMydXMgaW50ZXJ2YWwgMTAwLjBtcyBlY24KPj4gIFNl bnQgMCBieXRlcyAwIHBrdCAoZHJvcHBlZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMCkKPj4g IGJhY2tsb2cgMGIgMHAgcmVxdWV1ZXMgMAo+PiAgIG1heHBhY2tldCAwIGRyb3Bfb3ZlcmxpbWl0 IDAgbmV3X2Zsb3dfY291bnQgMCBlY25fbWFyayAwCj4+ICAgbmV3X2Zsb3dzX2xlbiAwIG9sZF9m bG93c19sZW4gMAo+PiBxZGlzYyBmcV9jb2RlbCA4MDAzOiBwYXJlbnQgOjMgbGltaXQgMTAyNHAg Zmxvd3MgMTYgcXVhbnR1bSAxNTE0Cj4+IHRhcmdldCA4MC4wbXMgY2VfdGhyZXNob2xkIDMydXMg aW50ZXJ2YWwgMTAwLjBtcyBlY24KPj4gIFNlbnQgMTYwMTI3MTE2OCBieXRlcyAxMDU3NzA2IHBr dCAoZHJvcHBlZCAxNDIyMzA0LCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMTcpCj4+ICBiYWNrbG9n IDE1NDEyNTJiIDEwMThwIHJlcXVldWVzIDE3Cj4+ICAgbWF4cGFja2V0IDE1MTQgZHJvcF9vdmVy bGltaXQgMTQyMjMwNCBuZXdfZmxvd19jb3VudCAzNSBlY25fbWFyayAwCj4+ICAgbmV3X2Zsb3dz X2xlbiAwIG9sZF9mbG93c19sZW4gMQo+PiBxZGlzYyBmcV9jb2RlbCA4MDA0OiBwYXJlbnQgOjQg bGltaXQgMTAyNHAgZmxvd3MgMTYgcXVhbnR1bSAxNTE0Cj4+IHRhcmdldCA4MC4wbXMgY2VfdGhy ZXNob2xkIDMydXMgaW50ZXJ2YWwgMTAwLjBtcyBlY24KPj4gIFNlbnQgMCBieXRlcyAwIHBrdCAo ZHJvcHBlZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMCkKPj4gIGJhY2tsb2cgMGIgMHAgcmVx dWV1ZXMgMAo+PiAgIG1heHBhY2tldCAwIGRyb3Bfb3ZlcmxpbWl0IDAgbmV3X2Zsb3dfY291bnQg MCBlY25fbWFyayAwCj4+ICAgbmV3X2Zsb3dzX2xlbiAwIG9sZF9mbG93c19sZW4gMAo+Pgo+Pgo+ PiBhZnRlciB0aGUgdGVzdCAoNjBzZWMpOgo+Pgo+PiBxZGlzYyBtcSAwOiByb290Cj4+ICBTZW50 IDMwODQ5OTYwNTIgYnl0ZXMgMjAzNzc0NCBwa3QgKGRyb3BwZWQgMjc3MDE3Niwgb3ZlcmxpbWl0 cyAwIHJlcXVldWVzIDI4KQo+PiAgYmFja2xvZyAwYiAwcCByZXF1ZXVlcyAyOAo+PiBxZGlzYyBm cV9jb2RlbCA4MDAxOiBwYXJlbnQgOjEgbGltaXQgMTAyNHAgZmxvd3MgMTYgcXVhbnR1bSAxNTE0 Cj4+IHRhcmdldCA4MC4wbXMgY2VfdGhyZXNob2xkIDMydXMgaW50ZXJ2YWwgMTAwLjBtcyBlY24K Pj4gIFNlbnQgMCBieXRlcyAwIHBrdCAoZHJvcHBlZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1ZXMg MCkKPj4gIGJhY2tsb2cgMGIgMHAgcmVxdWV1ZXMgMAo+PiAgIG1heHBhY2tldCAwIGRyb3Bfb3Zl cmxpbWl0IDAgbmV3X2Zsb3dfY291bnQgMCBlY25fbWFyayAwCj4+ICAgbmV3X2Zsb3dzX2xlbiAw IG9sZF9mbG93c19sZW4gMAo+PiBxZGlzYyBmcV9jb2RlbCA4MDAyOiBwYXJlbnQgOjIgbGltaXQg MTAyNHAgZmxvd3MgMTYgcXVhbnR1bSAxNTE0Cj4+IHRhcmdldCA4MC4wbXMgY2VfdGhyZXNob2xk IDMydXMgaW50ZXJ2YWwgMTAwLjBtcyBlY24KPj4gIFNlbnQgMCBieXRlcyAwIHBrdCAoZHJvcHBl ZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMCkKPj4gIGJhY2tsb2cgMGIgMHAgcmVxdWV1ZXMg MAo+PiAgIG1heHBhY2tldCAwIGRyb3Bfb3ZlcmxpbWl0IDAgbmV3X2Zsb3dfY291bnQgMCBlY25f bWFyayAwCj4+ICAgbmV3X2Zsb3dzX2xlbiAwIG9sZF9mbG93c19sZW4gMAo+PiBxZGlzYyBmcV9j b2RlbCA4MDAzOiBwYXJlbnQgOjMgbGltaXQgMTAyNHAgZmxvd3MgMTYgcXVhbnR1bSAxNTE0Cj4+ IHRhcmdldCA4MC4wbXMgY2VfdGhyZXNob2xkIDMydXMgaW50ZXJ2YWwgMTAwLjBtcyBlY24KPj4g IFNlbnQgMzA4NDk5NjA1MiBieXRlcyAyMDM3NzQ0IHBrdCAoZHJvcHBlZCAyNzcwMTc2LCBvdmVy bGltaXRzIDAgcmVxdWV1ZXMgMjgpCj4+ICBiYWNrbG9nIDBiIDBwIHJlcXVldWVzIDI4Cj4+ICAg bWF4cGFja2V0IDE1MTQgZHJvcF9vdmVybGltaXQgMjc3MDE3NiBuZXdfZmxvd19jb3VudCA2NCBl Y25fbWFyayAwCj4+ICAgbmV3X2Zsb3dzX2xlbiAwIG9sZF9mbG93c19sZW4gMQo+PiBxZGlzYyBm cV9jb2RlbCA4MDA0OiBwYXJlbnQgOjQgbGltaXQgMTAyNHAgZmxvd3MgMTYgcXVhbnR1bSAxNTE0 Cj4+IHRhcmdldCA4MC4wbXMgY2VfdGhyZXNob2xkIDMydXMgaW50ZXJ2YWwgMTAwLjBtcyBlY24K Pj4gIFNlbnQgMCBieXRlcyAwIHBrdCAoZHJvcHBlZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1ZXMg MCkKPj4gIGJhY2tsb2cgMGIgMHAgcmVxdWV1ZXMgMAo+PiAgIG1heHBhY2tldCAwIGRyb3Bfb3Zl cmxpbWl0IDAgbmV3X2Zsb3dfY291bnQgMCBlY25fbWFyayAwCj4+ICAgbmV3X2Zsb3dzX2xlbiAw IG9sZF9mbG93c19sZW4gMAo+Pgo+Pgo+Pj4gSUYgeW91IGFyZSBkb2luZyBidWlsZHMgZm9yIHRo ZSBhcmNoZXIgYzd2MiwgSSBjYW4gam9pbiBpbiBvbiB0aGlzLi4uICg/KQo+Pgo+PiBJJ20gbm90 IGJ1dCBJIGhhdmUgYzcgc29tZXdoZXJlLCBzbyBJIGNhbiBkbyBhIGJ1aWxkIGZvciBpdCBhbmQg YWxzbwo+PiB0ZXN0LCBzbyB3ZSBhcmUgb24gdGhlIHNhbWUgcGFnZS4KPj4KPj4+IEkgZGlkIGRv IGEgdGVzdCBvZiB0aGUgYXRoMTBrICJiZWZvcmUiLCBmcV9jb2RlbCAqbmV2ZXIgZW5nYWdlZCos IGFuZAo+Pj4gdGNwIGluZHVjZWQgbGF0ZW5jaWVzIHVuZGVyIGxvYWQsIGUgYXQgMTAwbWJpdCwg Y3JhY2tlZCA2MDBtcywgd2hpbGUKPj4+IHN0YXlpbmcgZmxhdCAoMjBtcykgYXQgMTAwbWJpdC4g KG5vdCB0aGUgc2FtZSBwYXRjaGVzIHlvdSBhcmUgdGVzdGluZykKPj4+IG9uIHg4Ni4gSSBoYXZl IGdvdCB0Y3AgMzAwTWJpdCBvdXQgb2YgYW4gb3N4IGJveCwgc2ltaWxhciBsYXRlbmN5LAo+Pj4g aGF2ZSB5ZXQgdG8gZ2V0IGFueXRoaW5nIG1vcmUgb24gYW55dGhpbmcgSSBjdXJyZW50bHkgaGF2 ZQo+Pj4gYmVmb3JlL2FmdGVyIHBhdGNoc2V0cy4KPj4+Cj4+PiBJJ2xsIGdvIGFkZCBmbG9vZGlu ZyB0byB0aGUgdGVzdHMsIEkganVzdCBmaW5pc2hlZCBhIHNlcmllcyBjb21wYXJpbmcKPj4+IHR3 byBkaWZmZXJlbnQgc3BlZWQgc3RhdGlvbnMgYW5kIGxpZmUgd2FzIGdvb2Qgb24gdGhhdC4KPj4+ Cj4+PiAiYmVmb3JlIiAtIGZxX2NvZGVsIG5ldmVyIGVuZ2FnZXMsIHdlIHNlZSBzZWNvbmRzIG9m IGxhdGVuY3kgdW5kZXIgbG9hZC4KPj4+Cj4+PiByb290QGFwdTI6fiMgdGMgLXMgcWRpc2Mgc2hv dyBkZXYgd2xwNHMwCj4+PiBxZGlzYyBtcSAwOiByb290Cj4+PiAgU2VudCA4NTcwNTYzODkzIGJ5 dGVzIDYzMjY5ODMgcGt0IChkcm9wcGVkIDAsIG92ZXJsaW1pdHMgMCByZXF1ZXVlcyAwKQo+Pj4g IGJhY2tsb2cgMGIgMHAgcmVxdWV1ZXMgMAo+Pj4gcWRpc2MgZnFfY29kZWwgMDogcGFyZW50IDox IGxpbWl0IDEwMjQwcCBmbG93cyAxMDI0IHF1YW50dW0gMTUxNAo+Pj4gdGFyZ2V0IDUuMG1zIGlu dGVydmFsIDEwMC4wbXMgZWNuCj4+PiAgU2VudCAyMjYyIGJ5dGVzIDE3IHBrdCAoZHJvcHBlZCAw LCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMCkKPj4+ICBiYWNrbG9nIDBiIDBwIHJlcXVldWVzIDAK Pj4+ICAgbWF4cGFja2V0IDAgZHJvcF9vdmVybGltaXQgMCBuZXdfZmxvd19jb3VudCAwIGVjbl9t YXJrIDAKPj4+ICAgbmV3X2Zsb3dzX2xlbiAwIG9sZF9mbG93c19sZW4gMAo+Pj4gcWRpc2MgZnFf Y29kZWwgMDogcGFyZW50IDoyIGxpbWl0IDEwMjQwcCBmbG93cyAxMDI0IHF1YW50dW0gMTUxNAo+ Pj4gdGFyZ2V0IDUuMG1zIGludGVydmFsIDEwMC4wbXMgZWNuCj4+PiAgU2VudCAyMjA0ODY1Njkg Ynl0ZXMgMTUyMDU4IHBrdCAoZHJvcHBlZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMCkKPj4+ ICBiYWNrbG9nIDBiIDBwIHJlcXVldWVzIDAKPj4+ICAgbWF4cGFja2V0IDE4MTY4IGRyb3Bfb3Zl cmxpbWl0IDAgbmV3X2Zsb3dfY291bnQgMSBlY25fbWFyayAwCj4+PiAgIG5ld19mbG93c19sZW4g MCBvbGRfZmxvd3NfbGVuIDEKPj4+IHFkaXNjIGZxX2NvZGVsIDA6IHBhcmVudCA6MyBsaW1pdCAx MDI0MHAgZmxvd3MgMTAyNCBxdWFudHVtIDE1MTQKPj4+IHRhcmdldCA1LjBtcyBpbnRlcnZhbCAx MDAuMG1zIGVjbgo+Pj4gIFNlbnQgODM0MDU0NjUwOSBieXRlcyA2MTYzNDMxIHBrdCAoZHJvcHBl ZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1ZXMgMCkKPj4+ICBiYWNrbG9nIDBiIDBwIHJlcXVldWVz IDAKPj4+ICAgbWF4cGFja2V0IDY4MTMwIGRyb3Bfb3ZlcmxpbWl0IDAgbmV3X2Zsb3dfY291bnQg MTIwMDUwIGVjbl9tYXJrIDAKPj4+ICAgbmV3X2Zsb3dzX2xlbiAxIG9sZF9mbG93c19sZW4gMwo+ Pj4gcWRpc2MgZnFfY29kZWwgMDogcGFyZW50IDo0IGxpbWl0IDEwMjQwcCBmbG93cyAxMDI0IHF1 YW50dW0gMTUxNAo+Pj4gdGFyZ2V0IDUuMG1zIGludGVydmFsIDEwMC4wbXMgZWNuCj4+PiAgU2Vu dCA5NTI4NTUzIGJ5dGVzIDExNDc3IHBrdCAoZHJvcHBlZCAwLCBvdmVybGltaXRzIDAgcmVxdWV1 ZXMgMCkKPj4+ICBiYWNrbG9nIDBiIDBwIHJlcXVldWVzIDAKPj4+ICAgbWF4cGFja2V0IDY2IGRy b3Bfb3ZlcmxpbWl0IDAgbmV3X2Zsb3dfY291bnQgMSBlY25fbWFyayAwCj4+PiAgIG5ld19mbG93 c19sZW4gMSBvbGRfZmxvd3NfbGVuIDAKPj4+ICAgYGBgCj4+Pgo+Pj4KPj4+Pj4gVGhpcyBpcyBj ZXJ0YWlubHkgYmV0dGVyIHRoYW4gMzBNYnBzIGJ1dCBzdGlsbCBtb3JlIHRoYW4gdHdvIHRpbWVz Cj4+Pj4+IGxlc3MgdGhhbiBiZWZvcmUgKDkwMCkuCj4+Pgo+Pj4gVGhlIG51bWJlciB0aGF0IEkg c3RpbGwgYW0gbm90IHN1cmUgd2UgZ290IGlzIHRoYXQgeW91IHdlcmUgc2VuZGluZwo+Pj4gOTAw bWJpdCB1ZHAgYW5kIHJlY2lldmluZyA5MDBtYml0IG9uIHRoZSBwcmlvciB0ZXN0cz8KPj4KPj4g OTAwIHdhcyBzZW5kaW5nLCBBUCBQT1YgKHdpZmkgY2xpZW50IGlzIGRvd25sb2FkaW5nKQo+Pgo+ Pj4+PiBUQ1AgYWxzbyBpbXByb3ZlZCBhIGxpdHRsZSAoNTUwIHRvIH41OTApLgo+Pj4KPj4+IFRo ZSBsaW1pdCBpcyBwcm9iYWJseSBhIGJpdCBsb3csIGFsc28uICBZb3UgbWlnaHQgd2FudCB0byB0 cnkgdGFyZ2V0Cj4+PiAyMG1zIGFzIHdlbGwuCj4+Cj4+IEkndmUgdHJpZWQgbGltaXQgdXAgdG8g MTAyNCBhbmQgdGFyZ2V0IHVwIHRvIDgwbXMKPj4KPj4+Pj4KPj4+Pj4gRmVsaXgsIG90aGVycywg ZG8geW91IHdhbnQgdG8gc2VlIHRoZSBwb3J0ZWQgcGF0Y2gsIG1heWJlIEkgZGlkIHNvbWV0aGlu ZyB3cm9uZz8KPj4+Pj4gRG9lc24ndCBsb29rIGxpa2UgaXQgd2lsbCBzYXZlIGF0aDEwayBmcm9t IHBlcmZvcm1hbmNlIHJlZ3Jlc3Npb24uCj4+Pgo+Pj4gd2hhdCB3YXMgdGNwICJiZWZvcmUiPyAo SSdtIHNvcnJ5LCBzdWNoIGEgbG9uZyB0aHJlYWQpCj4+Cj4+IDc1ME1icHMKPgo+IE1pY2hhbCwg YWZ0ZXIgcmV0ZXN0aW5nIHdpdGggeW91ciBwYXRjaCAoc29ycnksIGl0IHdhcyBsYXRlIHllc3Rl cmRheSwKPiBjb25mdXNlZCBjb21wYXQtd2lyZWxlc3MgYXJjaGl2ZXMpIEkgc2F3IHRoZSBkaWZm ZXJlbmNlLgo+IFNvIHRoZSBwcm9ncmVzcyBsb29rcyBsaWtlIHRoaXMgKGFsbCB3aXRoIGZxX2Nv ZGVsIGZsb3dzIDE2IGxpbWl0IDEwMjQKPiB0YXJnZXQgMjBtcyk6Cj4gbm8gcGF0Y2hlczogMzgw TWJwcyBVRFAsIDU1MCBUQ1AKPiBFcmljJ3MgKGZxX2NvZGVsIGRyb3ApIHBhdGNoOiA0MjBNYnBz IFVEUCwgNTkwIFRDUCAoKzQwTWJwcyksIGxhdGVuY3kKPiA1LTZtcyBkdXJpbmcgdGVzdAo+IE1p Y2hhbCdzIChpbXByb3ZlIHR4IHNjaGVkdWxpbmcpIHBhdGNoOiA1ODBNYnBzIFVEUCwgNjYwIFRD UCwgbGF0ZW5jeQo+IHVwIHRvIDMwLTQwbXMgZHVyaW5nIHRlc3QKPiBhZnRlciBSYWprdW1hcidz IHByb3Bvc2FsIHRvICJ0cnkgd2l0aG91dCByZWdpc3RlcmluZyB3YWtlX3R4X3F1ZXVlCj4gY2Fs bGJhY2siOiA4MjBNYnBzIFVEUCwgNjkwIFRDUC4KCkFuZCB0aGUgc2ltdWx0YW5lb3VzIHBpbmcg b24gdGhlIGxhc3QgdGVzdCB3YXM/Cgo+IFNvLCB2ZXJ5IGNsb3NlIHRvICJhcyBiZWZvcmUiOiA5 MDBNYnBzIFVEUCwgNzUwIFRDUC4KPiBCdXQgc3RpbGwsIEkgd2FzIGV4cGVjdGluZyBwZXJmb3Jt YW5jZSBpbXByb3ZlbWVudHMgZnJvbSBsYXRlc3QgYXRoMTBrCj4gY29kZSwgbm90IHJlZ3Jlc3Np b25zLgo+IEkga25vdyB0aGF0IGh3IGlzIGNhcGFibGUgb2YgODAwTWJwcyBUQ1AsIHdoaWNoIEkn bSB0YXJnZXRpbmcuCj4KPiBSZWdhcmRzLAo+IFJvbWFuCj4KPiBwLnMuIHNvcnJ5IGZvciBjb25m dXNpb24KCgoKLS0gCkRhdmUgVMOkaHQKTGV0J3MgZ28gbWFrZSBob21lIHJvdXRlcnMgYW5kIHdp ZmkgZmFzdGVyISBXaXRoIGJldHRlciBzb2Z0d2FyZSEKaHR0cDovL2Jsb2cuY2Vyb3dydC5vcmcK Cl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmF0aDEwayBt YWlsaW5nIGxpc3QKYXRoMTBrQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJh ZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9hdGgxMGsK