From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.3 required=3.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,HTML_MESSAGE, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BED2AC2D0EC for ; Fri, 10 Apr 2020 19:14:07 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 59E4020732 for ; Fri, 10 Apr 2020 19:14:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="MlH2hsDY" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59E4020732 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id E88878E0053; Fri, 10 Apr 2020 15:14:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E39018E004D; Fri, 10 Apr 2020 15:14:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D284E8E0053; Fri, 10 Apr 2020 15:14:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0222.hostedemail.com [216.40.44.222]) by kanga.kvack.org (Postfix) with ESMTP id B5C818E004D for ; Fri, 10 Apr 2020 15:14:06 -0400 (EDT) Received: from smtpin06.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id 74106181AEF21 for ; Fri, 10 Apr 2020 19:14:06 +0000 (UTC) X-FDA: 76692895692.06.key71_6b1539cadb1e X-HE-Tag: key71_6b1539cadb1e X-Filterd-Recvd-Size: 26384 Received: from mail-wr1-f66.google.com (mail-wr1-f66.google.com [209.85.221.66]) by imf24.hostedemail.com (Postfix) with ESMTP for ; Fri, 10 Apr 2020 19:14:05 +0000 (UTC) Received: by mail-wr1-f66.google.com with SMTP id w10so3344963wrm.4 for ; Fri, 10 Apr 2020 12:14:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=fsAkEOu/47a/c+5s9FtL7cN2iO8As3ssUFecYEMnSR0=; b=MlH2hsDY8O+8uUxgQSt3y4IgykyRlvQojITj0IPSppt5ZqzHpqSHT0+C8AaDM4+e6R BNvV2lavVldKjaYDN0hQM1+Jm7tVADJZs7yHEScqHBzv72qLrzdTULaAAvI7MH+BH0gD QhGmoos3GdY7dpftSXirIMCOm0lf/k162B/CzESIhMoQTL3cWZufKwlv/0Rz4d8ntSlK TnUIHepVGb6AiZffmrhTa5TncqVaHbaYEk/Uvm3gUioDdwxVbG9ufdT8W+DVqkt8CehJ 1zix8e6DNMuJik5PCis7+fFN/xKWfaYi991hkVkR+B1evdNPvZI5WHqiZ4+juc18BmIQ b0IA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=fsAkEOu/47a/c+5s9FtL7cN2iO8As3ssUFecYEMnSR0=; b=BmjRjtNpnvupDxVko2BQMUVJrdgyUCncn/9RIX3XSA75J6xH63lJCztSQkTvrz/kVx C8ZJamLkBLs05lMUoOdfI9oPrSKz+Aa0c6TnnMbUv29nR1GNmz4oUnQeNKcI4wmsaT9C Bnw5JNuPMsv+nBp8Pg4NsHEIHA8OXPt4O/qYYhBc9SKJbaLXtwvY1W4CMeqy45uyUcHp cdnr22rWAcYS3R91rG3IPLlmc70bK+/Ow6gpaYaZf4v6MIgALRM/BMG+iUJXtA7fpnnh TUlYMzs7g7f09af6ipHgo8ZTwyMYIQjWC3MTk9Ior7lNS5AlBuQaEnV6f9b8oD97lMsf l99w== X-Gm-Message-State: AGi0PuZra+vDx4wYSLJ4PjZX+UagRCMWhhp8PQbdg2BkF2Ktv0fh+jlz 2QYeqn9hCz7JvFm9GoD7y1Lu8icX1Rg59mEMJmY0Fg== X-Google-Smtp-Source: APiQypJBMqar2N8cEBat31+ASqnG2doIH6bHz0OkIQpypEreFdZc8Egb1hbTMEhEJ6N/jm6TG4Oz9meNDT41M2NN4VI= X-Received: by 2002:adf:8543:: with SMTP id 61mr5655151wrh.243.1586546044373; Fri, 10 Apr 2020 12:14:04 -0700 (PDT) MIME-Version: 1.0 References: <20200128025958.43490-1-arjunroy.kdev@gmail.com> <20200128025958.43490-3-arjunroy.kdev@gmail.com> <20200212185605.d89c820903b7aa9fbbc060b2@linux-foundation.org> <20200410120443.ad7856db13e158fbd441f3ae@linux-foundation.org> In-Reply-To: <20200410120443.ad7856db13e158fbd441f3ae@linux-foundation.org> From: Arjun Roy Date: Fri, 10 Apr 2020 12:13:53 -0700 Message-ID: Subject: Re: [PATCH resend mm,net-next 3/3] net-zerocopy: Use vm_insert_pages() for tcp rcv zerocopy. To: Andrew Morton Cc: Arjun Roy , David Miller , netdev , linux-mm@kvack.org, Eric Dumazet , Soheil Hassas Yeganeh Content-Type: multipart/alternative; boundary="000000000000656a6505a2f48966" X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: --000000000000656a6505a2f48966 Content-Type: text/plain; charset="UTF-8" On Fri, Apr 10, 2020 at 12:04 PM Andrew Morton wrote: > On Fri, 21 Feb 2020 13:21:41 -0800 Arjun Roy wrote: > > > I remain a bit concerned regarding the merge process for this specific > > patch (0003, the net/ipv4/tcp.c change) since I have other in-flight > > changes for TCP receive zerocopy that I'd like to upstream for > > net-next - and would like to avoid weird merge issues. > > > > So perhaps the following could work: > > > > 1. Andrew, perhaps we could remove this particular patch (0003, the > > net/ipv4/tcp.c change) from mm-next; that way we merge > > vm_insert_pages() but not the call-site within TCP, for now. > > 2. net-next will eventually pick vm_insert_pages() up. > > 3. I can modify the zerocopy code to use it at that point? > > > > Else I'm concerned a complicated merge situation may result. > > The merge situation is quite clean. > > I guess I'll hold off on > net-zerocopy-use-vm_insert_pages-for-tcp-rcv-zerocopy.patch (below) and > shall send it to davem after Linus has merged the prerequisites. > > > Acknowledged, thank you! -Arjun > From: Arjun Roy > Subject: net-zerocopy: use vm_insert_pages() for tcp rcv zerocopy > > Use vm_insert_pages() for tcp receive zerocopy. Spin lock cycles (as > reported by perf) drop from a couple of percentage points to a fraction of > a percent. This results in a roughly 6% increase in efficiency, measured > roughly as zerocopy receive count divided by CPU utilization. > > The intention of this patchset is to reduce atomic ops for tcp zerocopy > receives, which normally hits the same spinlock multiple times > consecutively. > > [akpm@linux-foundation.org: suppress gcc-7.2.0 warning] > Link: > http://lkml.kernel.org/r/20200128025958.43490-3-arjunroy.kdev@gmail.com > Signed-off-by: Arjun Roy > Signed-off-by: Eric Dumazet > Signed-off-by: Soheil Hassas Yeganeh > Cc: David Miller > Cc: Matthew Wilcox > Cc: Jason Gunthorpe > Cc: Stephen Rothwell > Signed-off-by: Andrew Morton > --- > > net/ipv4/tcp.c | 70 ++++++++++++++++++++++++++++++++++++++++++----- > 1 file changed, 63 insertions(+), 7 deletions(-) > > --- a/net/ipv4/tcp.c~net-zerocopy-use-vm_insert_pages-for-tcp-rcv-zerocopy > +++ a/net/ipv4/tcp.c > @@ -1734,14 +1734,48 @@ int tcp_mmap(struct file *file, struct s > } > EXPORT_SYMBOL(tcp_mmap); > > +static int tcp_zerocopy_vm_insert_batch(struct vm_area_struct *vma, > + struct page **pages, > + unsigned long pages_to_map, > + unsigned long *insert_addr, > + u32 *length_with_pending, > + u32 *seq, > + struct tcp_zerocopy_receive *zc) > +{ > + unsigned long pages_remaining = pages_to_map; > + int bytes_mapped; > + int ret; > + > + ret = vm_insert_pages(vma, *insert_addr, pages, &pages_remaining); > + bytes_mapped = PAGE_SIZE * (pages_to_map - pages_remaining); > + /* Even if vm_insert_pages fails, it may have partially succeeded > in > + * mapping (some but not all of the pages). > + */ > + *seq += bytes_mapped; > + *insert_addr += bytes_mapped; > + if (ret) { > + /* But if vm_insert_pages did fail, we have to unroll some > state > + * we speculatively touched before. > + */ > + const int bytes_not_mapped = PAGE_SIZE * pages_remaining; > + *length_with_pending -= bytes_not_mapped; > + zc->recv_skip_hint += bytes_not_mapped; > + } > + return ret; > +} > + > static int tcp_zerocopy_receive(struct sock *sk, > struct tcp_zerocopy_receive *zc) > { > unsigned long address = (unsigned long)zc->address; > u32 length = 0, seq, offset, zap_len; > + #define PAGE_BATCH_SIZE 8 > + struct page *pages[PAGE_BATCH_SIZE]; > const skb_frag_t *frags = NULL; > struct vm_area_struct *vma; > struct sk_buff *skb = NULL; > + unsigned long pg_idx = 0; > + unsigned long curr_addr; > struct tcp_sock *tp; > int inq; > int ret; > @@ -1754,6 +1788,8 @@ static int tcp_zerocopy_receive(struct s > > sock_rps_record_flow(sk); > > + tp = tcp_sk(sk); > + > down_read(¤t->mm->mmap_sem); > > ret = -EINVAL; > @@ -1762,7 +1798,6 @@ static int tcp_zerocopy_receive(struct s > goto out; > zc->length = min_t(unsigned long, zc->length, vma->vm_end - > address); > > - tp = tcp_sk(sk); > seq = tp->copied_seq; > inq = tcp_inq(sk); > zc->length = min_t(u32, zc->length, inq); > @@ -1774,8 +1809,20 @@ static int tcp_zerocopy_receive(struct s > zc->recv_skip_hint = zc->length; > } > ret = 0; > + curr_addr = address; > while (length + PAGE_SIZE <= zc->length) { > if (zc->recv_skip_hint < PAGE_SIZE) { > + /* If we're here, finish the current batch. */ > + if (pg_idx) { > + ret = tcp_zerocopy_vm_insert_batch(vma, > pages, > + pg_idx, > + > &curr_addr, > + &length, > + &seq, > zc); > + if (ret) > + goto out; > + pg_idx = 0; > + } > if (skb) { > if (zc->recv_skip_hint > 0) > break; > @@ -1784,7 +1831,6 @@ static int tcp_zerocopy_receive(struct s > } else { > skb = tcp_recv_skb(sk, seq, &offset); > } > - > zc->recv_skip_hint = skb->len - offset; > offset -= skb_headlen(skb); > if ((int)offset < 0 || skb_has_frag_list(skb)) > @@ -1808,14 +1854,24 @@ static int tcp_zerocopy_receive(struct s > zc->recv_skip_hint -= remaining; > break; > } > - ret = vm_insert_page(vma, address + length, > - skb_frag_page(frags)); > - if (ret) > - break; > + pages[pg_idx] = skb_frag_page(frags); > + pg_idx++; > length += PAGE_SIZE; > - seq += PAGE_SIZE; > zc->recv_skip_hint -= PAGE_SIZE; > frags++; > + if (pg_idx == PAGE_BATCH_SIZE) { > + ret = tcp_zerocopy_vm_insert_batch(vma, pages, > pg_idx, > + &curr_addr, > &length, > + &seq, zc); > + if (ret) > + goto out; > + pg_idx = 0; > + } > + } > + if (pg_idx) { > + ret = tcp_zerocopy_vm_insert_batch(vma, pages, pg_idx, > + &curr_addr, &length, > &seq, > + zc); > } > out: > up_read(¤t->mm->mmap_sem); > _ > > --000000000000656a6505a2f48966 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: base64 PGRpdiBkaXI9Imx0ciI+PGRpdiBkaXI9Imx0ciI+PC9kaXY+PGJyPjxkaXYgY2xhc3M9ImdtYWls X3F1b3RlIj48ZGl2IGRpcj0ibHRyIiBjbGFzcz0iZ21haWxfYXR0ciI+T24gRnJpLCBBcHIgMTAs IDIwMjAgYXQgMTI6MDQgUE0gQW5kcmV3IE1vcnRvbiAmbHQ7PGEgaHJlZj0ibWFpbHRvOmFrcG1A bGludXgtZm91bmRhdGlvbi5vcmciPmFrcG1AbGludXgtZm91bmRhdGlvbi5vcmc8L2E+Jmd0OyB3 cm90ZTo8YnI+PC9kaXY+PGJsb2NrcXVvdGUgY2xhc3M9ImdtYWlsX3F1b3RlIiBzdHlsZT0ibWFy Z2luOjBweCAwcHggMHB4IDAuOGV4O2JvcmRlci1sZWZ0OjFweCBzb2xpZCByZ2IoMjA0LDIwNCwy MDQpO3BhZGRpbmctbGVmdDoxZXgiPk9uIEZyaSwgMjEgRmViIDIwMjAgMTM6MjE6NDEgLTA4MDAg QXJqdW4gUm95ICZsdDs8YSBocmVmPSJtYWlsdG86YXJqdW5yb3lAZ29vZ2xlLmNvbSIgdGFyZ2V0 PSJfYmxhbmsiPmFyanVucm95QGdvb2dsZS5jb208L2E+Jmd0OyB3cm90ZTo8YnI+DQo8YnI+DQom Z3Q7IEkgcmVtYWluIGEgYml0IGNvbmNlcm5lZCByZWdhcmRpbmcgdGhlIG1lcmdlIHByb2Nlc3Mg Zm9yIHRoaXMgc3BlY2lmaWM8YnI+DQomZ3Q7IHBhdGNoICgwMDAzLCB0aGUgbmV0L2lwdjQvdGNw LmMgY2hhbmdlKSBzaW5jZSBJIGhhdmUgb3RoZXIgaW4tZmxpZ2h0PGJyPg0KJmd0OyBjaGFuZ2Vz IGZvciBUQ1AgcmVjZWl2ZSB6ZXJvY29weSB0aGF0IEkmIzM5O2QgbGlrZSB0byB1cHN0cmVhbSBm b3I8YnI+DQomZ3Q7IG5ldC1uZXh0IC0gYW5kIHdvdWxkIGxpa2UgdG8gYXZvaWQgd2VpcmQgbWVy Z2UgaXNzdWVzLjxicj4NCiZndDsgPGJyPg0KJmd0OyBTbyBwZXJoYXBzIHRoZSBmb2xsb3dpbmcg Y291bGQgd29yazo8YnI+DQomZ3Q7IDxicj4NCiZndDsgMS4gQW5kcmV3LCBwZXJoYXBzIHdlIGNv dWxkIHJlbW92ZSB0aGlzIHBhcnRpY3VsYXIgcGF0Y2ggKDAwMDMsIHRoZTxicj4NCiZndDsgbmV0 L2lwdjQvdGNwLmMgY2hhbmdlKSBmcm9tIG1tLW5leHQ7IHRoYXQgd2F5IHdlIG1lcmdlPGJyPg0K Jmd0OyB2bV9pbnNlcnRfcGFnZXMoKSBidXQgbm90IHRoZSBjYWxsLXNpdGUgd2l0aGluIFRDUCwg Zm9yIG5vdy48YnI+DQomZ3Q7IDIuIG5ldC1uZXh0IHdpbGwgZXZlbnR1YWxseSBwaWNrIHZtX2lu c2VydF9wYWdlcygpIHVwLjxicj4NCiZndDsgMy4gSSBjYW4gbW9kaWZ5IHRoZSB6ZXJvY29weSBj b2RlIHRvIHVzZSBpdCBhdCB0aGF0IHBvaW50Pzxicj4NCiZndDsgPGJyPg0KJmd0OyBFbHNlIEkm IzM5O20gY29uY2VybmVkIGEgY29tcGxpY2F0ZWQgbWVyZ2Ugc2l0dWF0aW9uIG1heSByZXN1bHQu PGJyPg0KPGJyPg0KVGhlIG1lcmdlIHNpdHVhdGlvbiBpcyBxdWl0ZSBjbGVhbi48YnI+DQo8YnI+ DQpJIGd1ZXNzIEkmIzM5O2xsIGhvbGQgb2ZmIG9uPGJyPg0KbmV0LXplcm9jb3B5LXVzZS12bV9p bnNlcnRfcGFnZXMtZm9yLXRjcC1yY3YtemVyb2NvcHkucGF0Y2ggKGJlbG93KSBhbmQ8YnI+DQpz aGFsbCBzZW5kIGl0IHRvIGRhdmVtIGFmdGVyIExpbnVzIGhhcyBtZXJnZWQgdGhlIHByZXJlcXVp c2l0ZXMuPGJyPg0KPGJyPg0KPGJyPjwvYmxvY2txdW90ZT48ZGl2Pjxicj48L2Rpdj48ZGl2PkFj a25vd2xlZGdlZCwgdGhhbmsgeW91ITwvZGl2PjxkaXY+PGJyPjwvZGl2PjxkaXY+LUFyanVuPC9k aXY+PGRpdj48YnI+PC9kaXY+PGRpdj7CoDwvZGl2PjxibG9ja3F1b3RlIGNsYXNzPSJnbWFpbF9x dW90ZSIgc3R5bGU9Im1hcmdpbjowcHggMHB4IDBweCAwLjhleDtib3JkZXItbGVmdDoxcHggc29s aWQgcmdiKDIwNCwyMDQsMjA0KTtwYWRkaW5nLWxlZnQ6MWV4Ij4NCkZyb206IEFyanVuIFJveSAm bHQ7PGEgaHJlZj0ibWFpbHRvOmFyanVucm95QGdvb2dsZS5jb20iIHRhcmdldD0iX2JsYW5rIj5h cmp1bnJveUBnb29nbGUuY29tPC9hPiZndDs8YnI+DQpTdWJqZWN0OiBuZXQtemVyb2NvcHk6IHVz ZSB2bV9pbnNlcnRfcGFnZXMoKSBmb3IgdGNwIHJjdiB6ZXJvY29weTxicj4NCjxicj4NClVzZSB2 bV9pbnNlcnRfcGFnZXMoKSBmb3IgdGNwIHJlY2VpdmUgemVyb2NvcHkuwqAgU3BpbiBsb2NrIGN5 Y2xlcyAoYXM8YnI+DQpyZXBvcnRlZCBieSBwZXJmKSBkcm9wIGZyb20gYSBjb3VwbGUgb2YgcGVy Y2VudGFnZSBwb2ludHMgdG8gYSBmcmFjdGlvbiBvZjxicj4NCmEgcGVyY2VudC7CoCBUaGlzIHJl c3VsdHMgaW4gYSByb3VnaGx5IDYlIGluY3JlYXNlIGluIGVmZmljaWVuY3ksIG1lYXN1cmVkPGJy Pg0Kcm91Z2hseSBhcyB6ZXJvY29weSByZWNlaXZlIGNvdW50IGRpdmlkZWQgYnkgQ1BVIHV0aWxp emF0aW9uLjxicj4NCjxicj4NClRoZSBpbnRlbnRpb24gb2YgdGhpcyBwYXRjaHNldCBpcyB0byBy ZWR1Y2UgYXRvbWljIG9wcyBmb3IgdGNwIHplcm9jb3B5PGJyPg0KcmVjZWl2ZXMsIHdoaWNoIG5v cm1hbGx5IGhpdHMgdGhlIHNhbWUgc3BpbmxvY2sgbXVsdGlwbGUgdGltZXM8YnI+DQpjb25zZWN1 dGl2ZWx5Ljxicj4NCjxicj4NCls8YSBocmVmPSJtYWlsdG86YWtwbUBsaW51eC1mb3VuZGF0aW9u Lm9yZyIgdGFyZ2V0PSJfYmxhbmsiPmFrcG1AbGludXgtZm91bmRhdGlvbi5vcmc8L2E+OiBzdXBw cmVzcyBnY2MtNy4yLjAgd2FybmluZ108YnI+DQpMaW5rOiA8YSBocmVmPSJodHRwOi8vbGttbC5r ZXJuZWwub3JnL3IvMjAyMDAxMjgwMjU5NTguNDM0OTAtMy1hcmp1bnJveS5rZGV2QGdtYWlsLmNv bSIgcmVsPSJub3JlZmVycmVyIiB0YXJnZXQ9Il9ibGFuayI+aHR0cDovL2xrbWwua2VybmVsLm9y Zy9yLzIwMjAwMTI4MDI1OTU4LjQzNDkwLTMtYXJqdW5yb3kua2RldkBnbWFpbC5jb208L2E+PGJy Pg0KU2lnbmVkLW9mZi1ieTogQXJqdW4gUm95ICZsdDs8YSBocmVmPSJtYWlsdG86YXJqdW5yb3lA Z29vZ2xlLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmFyanVucm95QGdvb2dsZS5jb208L2E+Jmd0Ozxi cj4NClNpZ25lZC1vZmYtYnk6IEVyaWMgRHVtYXpldCAmbHQ7PGEgaHJlZj0ibWFpbHRvOmVkdW1h emV0QGdvb2dsZS5jb20iIHRhcmdldD0iX2JsYW5rIj5lZHVtYXpldEBnb29nbGUuY29tPC9hPiZn dDs8YnI+DQpTaWduZWQtb2ZmLWJ5OiBTb2hlaWwgSGFzc2FzIFllZ2FuZWggJmx0OzxhIGhyZWY9 Im1haWx0bzpzb2hlaWxAZ29vZ2xlLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPnNvaGVpbEBnb29nbGUu Y29tPC9hPiZndDs8YnI+DQpDYzogRGF2aWQgTWlsbGVyICZsdDs8YSBocmVmPSJtYWlsdG86ZGF2 ZW1AZGF2ZW1sb2Z0Lm5ldCIgdGFyZ2V0PSJfYmxhbmsiPmRhdmVtQGRhdmVtbG9mdC5uZXQ8L2E+ Jmd0Ozxicj4NCkNjOiBNYXR0aGV3IFdpbGNveCAmbHQ7PGEgaHJlZj0ibWFpbHRvOndpbGx5QGlu ZnJhZGVhZC5vcmciIHRhcmdldD0iX2JsYW5rIj53aWxseUBpbmZyYWRlYWQub3JnPC9hPiZndDs8 YnI+DQpDYzogSmFzb24gR3VudGhvcnBlICZsdDs8YSBocmVmPSJtYWlsdG86amdnQHppZXBlLmNh IiB0YXJnZXQ9Il9ibGFuayI+amdnQHppZXBlLmNhPC9hPiZndDs8YnI+DQpDYzogU3RlcGhlbiBS b3Rod2VsbCAmbHQ7PGEgaHJlZj0ibWFpbHRvOnNmckBjYW5iLmF1dWcub3JnLmF1IiB0YXJnZXQ9 Il9ibGFuayI+c2ZyQGNhbmIuYXV1Zy5vcmcuYXU8L2E+Jmd0Ozxicj4NClNpZ25lZC1vZmYtYnk6 IEFuZHJldyBNb3J0b24gJmx0OzxhIGhyZWY9Im1haWx0bzpha3BtQGxpbnV4LWZvdW5kYXRpb24u b3JnIiB0YXJnZXQ9Il9ibGFuayI+YWtwbUBsaW51eC1mb3VuZGF0aW9uLm9yZzwvYT4mZ3Q7PGJy Pg0KLS0tPGJyPg0KPGJyPg0KwqBuZXQvaXB2NC90Y3AuYyB8wqAgwqA3MCArKysrKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKystLS0tLTxicj4NCsKgMSBmaWxlIGNoYW5nZWQs IDYzIGluc2VydGlvbnMoKyksIDcgZGVsZXRpb25zKC0pPGJyPg0KPGJyPg0KLS0tIGEvbmV0L2lw djQvdGNwLmN+bmV0LXplcm9jb3B5LXVzZS12bV9pbnNlcnRfcGFnZXMtZm9yLXRjcC1yY3YtemVy b2NvcHk8YnI+DQorKysgYS9uZXQvaXB2NC90Y3AuYzxicj4NCkBAIC0xNzM0LDE0ICsxNzM0LDQ4 IEBAIGludCB0Y3BfbW1hcChzdHJ1Y3QgZmlsZSAqZmlsZSwgc3RydWN0IHM8YnI+DQrCoH08YnI+ DQrCoEVYUE9SVF9TWU1CT0wodGNwX21tYXApOzxicj4NCjxicj4NCitzdGF0aWMgaW50IHRjcF96 ZXJvY29weV92bV9pbnNlcnRfYmF0Y2goc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEsPGJyPg0K K8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgc3RydWN0IHBhZ2UgKipwYWdlcyw8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqB1bnNpZ25lZCBsb25nIHBhZ2VzX3RvX21h cCw8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqB1bnNpZ25lZCBsb25nICppbnNlcnRfYWRkciw8YnI+DQorwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqB1MzIgKmxlbmd0 aF93aXRoX3BlbmRpbmcsPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgdTMyICpzZXEsPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgc3RydWN0IHRjcF96ZXJv Y29weV9yZWNlaXZlICp6Yyk8YnI+DQorezxicj4NCivCoCDCoCDCoCDCoHVuc2lnbmVkIGxvbmcg cGFnZXNfcmVtYWluaW5nID0gcGFnZXNfdG9fbWFwOzxicj4NCivCoCDCoCDCoCDCoGludCBieXRl c19tYXBwZWQ7PGJyPg0KK8KgIMKgIMKgIMKgaW50IHJldDs8YnI+DQorPGJyPg0KK8KgIMKgIMKg IMKgcmV0ID0gdm1faW5zZXJ0X3BhZ2VzKHZtYSwgKmluc2VydF9hZGRyLCBwYWdlcywgJmFtcDtw YWdlc19yZW1haW5pbmcpOzxicj4NCivCoCDCoCDCoCDCoGJ5dGVzX21hcHBlZCA9IFBBR0VfU0la RSAqIChwYWdlc190b19tYXAgLSBwYWdlc19yZW1haW5pbmcpOzxicj4NCivCoCDCoCDCoCDCoC8q IEV2ZW4gaWYgdm1faW5zZXJ0X3BhZ2VzIGZhaWxzLCBpdCBtYXkgaGF2ZSBwYXJ0aWFsbHkgc3Vj Y2VlZGVkIGluPGJyPg0KK8KgIMKgIMKgIMKgICogbWFwcGluZyAoc29tZSBidXQgbm90IGFsbCBv ZiB0aGUgcGFnZXMpLjxicj4NCivCoCDCoCDCoCDCoCAqLzxicj4NCivCoCDCoCDCoCDCoCpzZXEg Kz0gYnl0ZXNfbWFwcGVkOzxicj4NCivCoCDCoCDCoCDCoCppbnNlcnRfYWRkciArPSBieXRlc19t YXBwZWQ7PGJyPg0KK8KgIMKgIMKgIMKgaWYgKHJldCkgezxicj4NCivCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoC8qIEJ1dCBpZiB2bV9pbnNlcnRfcGFnZXMgZGlkIGZhaWwsIHdlIGhhdmUgdG8gdW5y b2xsIHNvbWUgc3RhdGU8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgKiB3ZSBzcGVjdWxh dGl2ZWx5IHRvdWNoZWQgYmVmb3JlLjxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAqLzxi cj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoGNvbnN0IGludCBieXRlc19ub3RfbWFwcGVkID0g UEFHRV9TSVpFICogcGFnZXNfcmVtYWluaW5nOzxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCpsZW5ndGhfd2l0aF9wZW5kaW5nIC09IGJ5dGVzX25vdF9tYXBwZWQ7PGJyPg0KK8KgIMKgIMKg IMKgIMKgIMKgIMKgIMKgemMtJmd0O3JlY3Zfc2tpcF9oaW50ICs9IGJ5dGVzX25vdF9tYXBwZWQ7 PGJyPg0KK8KgIMKgIMKgIMKgfTxicj4NCivCoCDCoCDCoCDCoHJldHVybiByZXQ7PGJyPg0KK308 YnI+DQorPGJyPg0KwqBzdGF0aWMgaW50IHRjcF96ZXJvY29weV9yZWNlaXZlKHN0cnVjdCBzb2Nr ICpzayw8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCBzdHJ1Y3QgdGNwX3plcm9jb3B5X3JlY2VpdmUgKnpjKTxicj4NCsKgezxicj4NCsKgIMKgIMKg IMKgIHVuc2lnbmVkIGxvbmcgYWRkcmVzcyA9ICh1bnNpZ25lZCBsb25nKXpjLSZndDthZGRyZXNz Ozxicj4NCsKgIMKgIMKgIMKgIHUzMiBsZW5ndGggPSAwLCBzZXEsIG9mZnNldCwgemFwX2xlbjs8 YnI+DQorwqAgwqAgwqAgwqAjZGVmaW5lIFBBR0VfQkFUQ0hfU0laRSA4PGJyPg0KK8KgIMKgIMKg IMKgc3RydWN0IHBhZ2UgKnBhZ2VzW1BBR0VfQkFUQ0hfU0laRV07PGJyPg0KwqAgwqAgwqAgwqAg Y29uc3Qgc2tiX2ZyYWdfdCAqZnJhZ3MgPSBOVUxMOzxicj4NCsKgIMKgIMKgIMKgIHN0cnVjdCB2 bV9hcmVhX3N0cnVjdCAqdm1hOzxicj4NCsKgIMKgIMKgIMKgIHN0cnVjdCBza19idWZmICpza2Ig PSBOVUxMOzxicj4NCivCoCDCoCDCoCDCoHVuc2lnbmVkIGxvbmcgcGdfaWR4ID0gMDs8YnI+DQor wqAgwqAgwqAgwqB1bnNpZ25lZCBsb25nIGN1cnJfYWRkcjs8YnI+DQrCoCDCoCDCoCDCoCBzdHJ1 Y3QgdGNwX3NvY2sgKnRwOzxicj4NCsKgIMKgIMKgIMKgIGludCBpbnE7PGJyPg0KwqAgwqAgwqAg wqAgaW50IHJldDs8YnI+DQpAQCAtMTc1NCw2ICsxNzg4LDggQEAgc3RhdGljIGludCB0Y3BfemVy b2NvcHlfcmVjZWl2ZShzdHJ1Y3Qgczxicj4NCjxicj4NCsKgIMKgIMKgIMKgIHNvY2tfcnBzX3Jl Y29yZF9mbG93KHNrKTs8YnI+DQo8YnI+DQorwqAgwqAgwqAgwqB0cCA9IHRjcF9zayhzayk7PGJy Pg0KKzxicj4NCsKgIMKgIMKgIMKgIGRvd25fcmVhZCgmYW1wO2N1cnJlbnQtJmd0O21tLSZndDtt bWFwX3NlbSk7PGJyPg0KPGJyPg0KwqAgwqAgwqAgwqAgcmV0ID0gLUVJTlZBTDs8YnI+DQpAQCAt MTc2Miw3ICsxNzk4LDYgQEAgc3RhdGljIGludCB0Y3BfemVyb2NvcHlfcmVjZWl2ZShzdHJ1Y3Qg czxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGdvdG8gb3V0Ozxicj4NCsKgIMKgIMKgIMKg IHpjLSZndDtsZW5ndGggPSBtaW5fdCh1bnNpZ25lZCBsb25nLCB6Yy0mZ3Q7bGVuZ3RoLCB2bWEt Jmd0O3ZtX2VuZCAtIGFkZHJlc3MpOzxicj4NCjxicj4NCi3CoCDCoCDCoCDCoHRwID0gdGNwX3Nr KHNrKTs8YnI+DQrCoCDCoCDCoCDCoCBzZXEgPSB0cC0mZ3Q7Y29waWVkX3NlcTs8YnI+DQrCoCDC oCDCoCDCoCBpbnEgPSB0Y3BfaW5xKHNrKTs8YnI+DQrCoCDCoCDCoCDCoCB6Yy0mZ3Q7bGVuZ3Ro ID0gbWluX3QodTMyLCB6Yy0mZ3Q7bGVuZ3RoLCBpbnEpOzxicj4NCkBAIC0xNzc0LDggKzE4MDks MjAgQEAgc3RhdGljIGludCB0Y3BfemVyb2NvcHlfcmVjZWl2ZShzdHJ1Y3Qgczxicj4NCsKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIHpjLSZndDtyZWN2X3NraXBfaGludCA9IHpjLSZndDtsZW5ndGg7 PGJyPg0KwqAgwqAgwqAgwqAgfTxicj4NCsKgIMKgIMKgIMKgIHJldCA9IDA7PGJyPg0KK8KgIMKg IMKgIMKgY3Vycl9hZGRyID0gYWRkcmVzczs8YnI+DQrCoCDCoCDCoCDCoCB3aGlsZSAobGVuZ3Ro ICsgUEFHRV9TSVpFICZsdDs9IHpjLSZndDtsZW5ndGgpIHs8YnI+DQrCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCBpZiAoemMtJmd0O3JlY3Zfc2tpcF9oaW50ICZsdDsgUEFHRV9TSVpFKSB7PGJyPg0K K8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgLyogSWYgd2UmIzM5O3JlIGhlcmUs IGZpbmlzaCB0aGUgY3VycmVudCBiYXRjaC4gKi88YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqBpZiAocGdfaWR4KSB7PGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgcmV0ID0gdGNwX3plcm9jb3B5X3ZtX2luc2VydF9iYXRj aCh2bWEsIHBhZ2VzLDxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCBwZ19pZHgsPGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgICZhbXA7Y3Vycl9hZGRyLDxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCAmYW1wO2xlbmd0aCw8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgJmFtcDtzZXEsIHpjKTs8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpZiAocmV0KTxicj4NCivCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoGdvdG8gb3V0Ozxicj4N CivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoHBnX2lkeCA9 IDA7PGJyPg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgfTxicj4NCsKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGlmIChza2IpIHs8YnI+DQrCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBpZiAoemMtJmd0O3JlY3Zfc2tp cF9oaW50ICZndDsgMCk8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBicmVhazs8YnI+DQpAQCAtMTc4NCw3ICsxODMxLDYgQEAg c3RhdGljIGludCB0Y3BfemVyb2NvcHlfcmVjZWl2ZShzdHJ1Y3Qgczxicj4NCsKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIH0gZWxzZSB7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgc2tiID0gdGNwX3JlY3Zfc2tiKHNrLCBzZXEs ICZhbXA7b2Zmc2V0KTs8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCB9 PGJyPg0KLTxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIHpjLSZndDty ZWN2X3NraXBfaGludCA9IHNrYi0mZ3Q7bGVuIC0gb2Zmc2V0Ozxicj4NCsKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIG9mZnNldCAtPSBza2JfaGVhZGxlbihza2IpOzxicj4NCsKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIGlmICgoaW50KW9mZnNldCAmbHQ7IDAg fHwgc2tiX2hhc19mcmFnX2xpc3Qoc2tiKSk8YnI+DQpAQCAtMTgwOCwxNCArMTg1NCwyNCBAQCBz dGF0aWMgaW50IHRjcF96ZXJvY29weV9yZWNlaXZlKHN0cnVjdCBzPGJyPg0KwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgemMtJmd0O3JlY3Zfc2tpcF9oaW50IC09IHJlbWFpbmlu Zzs8YnI+DQrCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBicmVhazs8YnI+DQrC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCB9PGJyPg0KLcKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgcmV0 ID0gdm1faW5zZXJ0X3BhZ2Uodm1hLCBhZGRyZXNzICsgbGVuZ3RoLDxicj4NCi3CoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCBza2JfZnJhZ19wYWdl KGZyYWdzKSk7PGJyPg0KLcKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgaWYgKHJldCk8YnI+DQotwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBicmVhazs8YnI+DQorwqAgwqAgwqAgwqAg wqAgwqAgwqAgwqBwYWdlc1twZ19pZHhdID0gc2tiX2ZyYWdfcGFnZShmcmFncyk7PGJyPg0KK8Kg IMKgIMKgIMKgIMKgIMKgIMKgIMKgcGdfaWR4Kys7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAg wqAgbGVuZ3RoICs9IFBBR0VfU0laRTs8YnI+DQotwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBzZXEg Kz0gUEFHRV9TSVpFOzxicj4NCsKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIHpjLSZndDtyZWN2X3Nr aXBfaGludCAtPSBQQUdFX1NJWkU7PGJyPg0KwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgZnJhZ3Mr Kzs8YnI+DQorwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBpZiAocGdfaWR4ID09IFBBR0VfQkFUQ0hf U0laRSkgezxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoHJldCA9IHRj cF96ZXJvY29weV92bV9pbnNlcnRfYmF0Y2godm1hLCBwYWdlcywgcGdfaWR4LDxicj4NCivCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCAmYW1wO2N1cnJfYWRkciwgJmFtcDtsZW5ndGgsPGJy Pg0KK8KgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgICZhbXA7c2VxLCB6Yyk7PGJyPg0KK8Kg IMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgIMKgaWYgKHJldCk8YnI+DQorwqAgwqAgwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBnb3RvIG91dDs8YnI+DQorwqAg wqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqAgwqBwZ19pZHggPSAwOzxicj4NCivCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoH08YnI+DQorwqAgwqAgwqAgwqB9PGJyPg0KK8KgIMKgIMKgIMKgaWYg KHBnX2lkeCkgezxicj4NCivCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoHJldCA9IHRjcF96ZXJvY29w eV92bV9pbnNlcnRfYmF0Y2godm1hLCBwYWdlcywgcGdfaWR4LDxicj4NCivCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCAmYW1wO2N1cnJfYWRkciwgJmFtcDtsZW5ndGgsICZhbXA7c2VxLDxicj4NCivCoCDCoCDC oCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDCoCDC oCDCoCDCoCDCoCB6Yyk7PGJyPg0KwqAgwqAgwqAgwqAgfTxicj4NCsKgb3V0Ojxicj4NCsKgIMKg IMKgIMKgIHVwX3JlYWQoJmFtcDtjdXJyZW50LSZndDttbS0mZ3Q7bW1hcF9zZW0pOzxicj4NCl88 YnI+DQo8YnI+DQo8L2Jsb2NrcXVvdGU+PC9kaXY+PC9kaXY+DQo= --000000000000656a6505a2f48966--