From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB31CC282C3 for ; Thu, 24 Jan 2019 04:53:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9A7CC2184C for ; Thu, 24 Jan 2019 04:53:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727363AbfAXExt (ORCPT ); Wed, 23 Jan 2019 23:53:49 -0500 Received: from mx1.redhat.com ([209.132.183.28]:55468 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726249AbfAXExt (ORCPT ); Wed, 23 Jan 2019 23:53:49 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9DD5B811A9; Thu, 24 Jan 2019 04:53:48 +0000 (UTC) Received: from redhat.com (ovpn-121-100.rdu2.redhat.com [10.10.121.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A17042657F; Thu, 24 Jan 2019 04:53:47 +0000 (UTC) Date: Wed, 23 Jan 2019 23:53:47 -0500 From: "Michael S. Tsirkin" To: Jason Wang Cc: virtualization@lists.linux-foundation.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH net-next V4 5/5] vhost: access vq metadata through kernel virtual address Message-ID: <20190123235219-mutt-send-email-mst@kernel.org> References: <20190123095557.30168-1-jasowang@redhat.com> <20190123095557.30168-6-jasowang@redhat.com> <20190123085821-mutt-send-email-mst@kernel.org> <335ba55b-087f-4b35-6311-540070b9647f@redhat.com> <4521d3d8-561e-53f5-98e1-bf7ace003701@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <4521d3d8-561e-53f5-98e1-bf7ace003701@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Thu, 24 Jan 2019 04:53:48 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Jan 24, 2019 at 12:11:28PM +0800, Jason Wang wrote: > > On 2019/1/24 下午12:07, Jason Wang wrote: > > > > On 2019/1/23 下午10:08, Michael S. Tsirkin wrote: > > > On Wed, Jan 23, 2019 at 05:55:57PM +0800, Jason Wang wrote: > > > > It was noticed that the copy_user() friends that was used to access > > > > virtqueue metdata tends to be very expensive for dataplane > > > > implementation like vhost since it involves lots of software checks, > > > > speculation barrier, hardware feature toggling (e.g SMAP). The > > > > extra cost will be more obvious when transferring small packets since > > > > the time spent on metadata accessing become more significant. > > > > > > > > This patch tries to eliminate those overheads by accessing them > > > > through kernel virtual address by vmap(). To make the pages can be > > > > migrated, instead of pinning them through GUP, we use MMU notifiers to > > > > invalidate vmaps and re-establish vmaps during each round of metadata > > > > prefetching if necessary. For devices that doesn't use metadata > > > > prefetching, the memory accessors fallback to normal copy_user() > > > > implementation gracefully. The invalidation was synchronized with > > > > datapath through vq mutex, and in order to avoid hold vq mutex during > > > > range checking, MMU notifier was teared down when trying to modify vq > > > > metadata. > > > > > > > > Another thing is kernel lacks efficient solution for tracking dirty > > > > pages by vmap(), this will lead issues if vhost is using file backed > > > > memory which needs care of writeback. This patch solves this issue by > > > > just skipping the vma that is file backed and fallback to normal > > > > copy_user() friends. This might introduce some overheads for file > > > > backed users but consider this use case is rare we could do > > > > optimizations on top. > > > > > > > > Note that this was only done when device IOTLB is not enabled. We > > > > could use similar method to optimize it in the future. > > > > > > > > Tests shows at most about 22% improvement on TX PPS when using > > > > virtio-user + vhost_net + xdp1 + TAP on 2.6GHz Broadwell: > > > > > > > >          SMAP on | SMAP off > > > > Before: 5.0Mpps | 6.6Mpps > > > > After:  6.1Mpps | 7.4Mpps > > > > > > > > Signed-off-by: Jason Wang > > > > > > So this is the bulk of the change. > > > Threee things that I need to look into > > > - Are there any security issues with bypassing the speculation barrier > > >    that is normally present after access_ok? > > > > > > If we can make sure the bypassing was only used in a kthread (vhost), it > > should be fine I think. > > > > > > > - How hard does the special handling for > > >    file backed storage make testing? > > > > > > It's as simple as un-commenting vhost_can_vmap()? Or I can try to hack > > qemu or dpdk to test this. > > > > > > >    On the one hand we could add a module parameter to > > >    force copy to/from user. on the other that's > > >    another configuration we need to support. > > > > > > That sounds sub-optimal since it leave the choice to users. > > > > > > >    But iotlb is not using vmap, so maybe that's enough > > >    for testing. > > > - How hard is it to figure out which mode uses which code. > > > It's as simple as tracing __get_user() usage in vhost process? > > Thanks Well there are now mtu notifiers etc etc. It's hardly as well contained as that. > > > > > > > > > > > > > Meanwhile, could you pls post data comparing this last patch with the > > > below?  This removes the speculation barrier replacing it with a > > > (useless but at least more lightweight) data dependency. > > > > > > SMAP off > > > > Your patch: 7.2MPPs > > > > vmap: 7.4Mpps > > > > I don't test SMAP on, since it will be much slow for sure. > > > > Thanks > > > > > > > > > > Thanks! > > > > > > > > > diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c > > > index bac939af8dbb..352ee7e14476 100644 > > > --- a/drivers/vhost/vhost.c > > > +++ b/drivers/vhost/vhost.c > > > @@ -739,7 +739,7 @@ static int vhost_copy_to_user(struct > > > vhost_virtqueue *vq, void __user *to, > > >       int ret; > > >         if (!vq->iotlb) > > > -        return __copy_to_user(to, from, size); > > > +        return copy_to_user(to, from, size); > > >       else { > > >           /* This function should be called after iotlb > > >            * prefetch, which means we're sure that all vq > > > @@ -752,7 +752,7 @@ static int vhost_copy_to_user(struct > > > vhost_virtqueue *vq, void __user *to, > > >                        VHOST_ADDR_USED); > > >             if (uaddr) > > > -            return __copy_to_user(uaddr, from, size); > > > +            return copy_to_user(uaddr, from, size); > > >             ret = translate_desc(vq, (u64)(uintptr_t)to, size, > > > vq->iotlb_iov, > > >                        ARRAY_SIZE(vq->iotlb_iov), > > > @@ -774,7 +774,7 @@ static int vhost_copy_from_user(struct > > > vhost_virtqueue *vq, void *to, > > >       int ret; > > >         if (!vq->iotlb) > > > -        return __copy_from_user(to, from, size); > > > +        return copy_from_user(to, from, size); > > >       else { > > >           /* This function should be called after iotlb > > >            * prefetch, which means we're sure that vq > > > @@ -787,7 +787,7 @@ static int vhost_copy_from_user(struct > > > vhost_virtqueue *vq, void *to, > > >           struct iov_iter f; > > >             if (uaddr) > > > -            return __copy_from_user(to, uaddr, size); > > > +            return copy_from_user(to, uaddr, size); > > >             ret = translate_desc(vq, (u64)(uintptr_t)from, size, > > > vq->iotlb_iov, > > >                        ARRAY_SIZE(vq->iotlb_iov), > > > @@ -855,13 +855,13 @@ static inline void __user > > > *__vhost_get_user(struct vhost_virtqueue *vq, > > >   ({ \ > > >       int ret = -EFAULT; \ > > >       if (!vq->iotlb) { \ > > > -        ret = __put_user(x, ptr); \ > > > +        ret = put_user(x, ptr); \ > > >       } else { \ > > >           __typeof__(ptr) to = \ > > >               (__typeof__(ptr)) __vhost_get_user(vq, ptr,    \ > > >                         sizeof(*ptr), VHOST_ADDR_USED); \ > > >           if (to != NULL) \ > > > -            ret = __put_user(x, to); \ > > > +            ret = put_user(x, to); \ > > >           else \ > > >               ret = -EFAULT;    \ > > >       } \ > > > @@ -872,14 +872,14 @@ static inline void __user > > > *__vhost_get_user(struct vhost_virtqueue *vq, > > >   ({ \ > > >       int ret; \ > > >       if (!vq->iotlb) { \ > > > -        ret = __get_user(x, ptr); \ > > > +        ret = get_user(x, ptr); \ > > >       } else { \ > > >           __typeof__(ptr) from = \ > > >               (__typeof__(ptr)) __vhost_get_user(vq, ptr, \ > > >                                  sizeof(*ptr), \ > > >                                  type); \ > > >           if (from != NULL) \ > > > -            ret = __get_user(x, from); \ > > > +            ret = get_user(x, from); \ > > >           else \ > > >               ret = -EFAULT; \ > > >       } \ From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Michael S. Tsirkin" Subject: Re: [PATCH net-next V4 5/5] vhost: access vq metadata through kernel virtual address Date: Wed, 23 Jan 2019 23:53:47 -0500 Message-ID: <20190123235219-mutt-send-email-mst@kernel.org> References: <20190123095557.30168-1-jasowang@redhat.com> <20190123095557.30168-6-jasowang@redhat.com> <20190123085821-mutt-send-email-mst@kernel.org> <335ba55b-087f-4b35-6311-540070b9647f@redhat.com> <4521d3d8-561e-53f5-98e1-bf7ace003701@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org, virtualization@lists.linux-foundation.org To: Jason Wang Return-path: Content-Disposition: inline In-Reply-To: <4521d3d8-561e-53f5-98e1-bf7ace003701@redhat.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: virtualization-bounces@lists.linux-foundation.org Errors-To: virtualization-bounces@lists.linux-foundation.org List-Id: kvm.vger.kernel.org T24gVGh1LCBKYW4gMjQsIDIwMTkgYXQgMTI6MTE6MjhQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90 ZToKPiAKPiBPbiAyMDE5LzEvMjQg5LiL5Y2IMTI6MDcsIEphc29uIFdhbmcgd3JvdGU6Cj4gPiAK PiA+IE9uIDIwMTkvMS8yMyDkuIvljYgxMDowOCwgTWljaGFlbCBTLiBUc2lya2luIHdyb3RlOgo+ ID4gPiBPbiBXZWQsIEphbiAyMywgMjAxOSBhdCAwNTo1NTo1N1BNICswODAwLCBKYXNvbiBXYW5n IHdyb3RlOgo+ID4gPiA+IEl0IHdhcyBub3RpY2VkIHRoYXQgdGhlIGNvcHlfdXNlcigpIGZyaWVu ZHMgdGhhdCB3YXMgdXNlZCB0byBhY2Nlc3MKPiA+ID4gPiB2aXJ0cXVldWUgbWV0ZGF0YSB0ZW5k cyB0byBiZSB2ZXJ5IGV4cGVuc2l2ZSBmb3IgZGF0YXBsYW5lCj4gPiA+ID4gaW1wbGVtZW50YXRp b24gbGlrZSB2aG9zdCBzaW5jZSBpdCBpbnZvbHZlcyBsb3RzIG9mIHNvZnR3YXJlIGNoZWNrcywK PiA+ID4gPiBzcGVjdWxhdGlvbiBiYXJyaWVyLCBoYXJkd2FyZSBmZWF0dXJlIHRvZ2dsaW5nIChl LmcgU01BUCkuIFRoZQo+ID4gPiA+IGV4dHJhIGNvc3Qgd2lsbCBiZSBtb3JlIG9idmlvdXMgd2hl biB0cmFuc2ZlcnJpbmcgc21hbGwgcGFja2V0cyBzaW5jZQo+ID4gPiA+IHRoZSB0aW1lIHNwZW50 IG9uIG1ldGFkYXRhIGFjY2Vzc2luZyBiZWNvbWUgbW9yZSBzaWduaWZpY2FudC4KPiA+ID4gPiAK PiA+ID4gPiBUaGlzIHBhdGNoIHRyaWVzIHRvIGVsaW1pbmF0ZSB0aG9zZSBvdmVyaGVhZHMgYnkg YWNjZXNzaW5nIHRoZW0KPiA+ID4gPiB0aHJvdWdoIGtlcm5lbCB2aXJ0dWFsIGFkZHJlc3MgYnkg dm1hcCgpLiBUbyBtYWtlIHRoZSBwYWdlcyBjYW4gYmUKPiA+ID4gPiBtaWdyYXRlZCwgaW5zdGVh ZCBvZiBwaW5uaW5nIHRoZW0gdGhyb3VnaCBHVVAsIHdlIHVzZSBNTVUgbm90aWZpZXJzIHRvCj4g PiA+ID4gaW52YWxpZGF0ZSB2bWFwcyBhbmQgcmUtZXN0YWJsaXNoIHZtYXBzIGR1cmluZyBlYWNo IHJvdW5kIG9mIG1ldGFkYXRhCj4gPiA+ID4gcHJlZmV0Y2hpbmcgaWYgbmVjZXNzYXJ5LiBGb3Ig ZGV2aWNlcyB0aGF0IGRvZXNuJ3QgdXNlIG1ldGFkYXRhCj4gPiA+ID4gcHJlZmV0Y2hpbmcsIHRo ZSBtZW1vcnkgYWNjZXNzb3JzIGZhbGxiYWNrIHRvIG5vcm1hbCBjb3B5X3VzZXIoKQo+ID4gPiA+ IGltcGxlbWVudGF0aW9uIGdyYWNlZnVsbHkuIFRoZSBpbnZhbGlkYXRpb24gd2FzIHN5bmNocm9u aXplZCB3aXRoCj4gPiA+ID4gZGF0YXBhdGggdGhyb3VnaCB2cSBtdXRleCwgYW5kIGluIG9yZGVy IHRvIGF2b2lkIGhvbGQgdnEgbXV0ZXggZHVyaW5nCj4gPiA+ID4gcmFuZ2UgY2hlY2tpbmcsIE1N VSBub3RpZmllciB3YXMgdGVhcmVkIGRvd24gd2hlbiB0cnlpbmcgdG8gbW9kaWZ5IHZxCj4gPiA+ ID4gbWV0YWRhdGEuCj4gPiA+ID4gCj4gPiA+ID4gQW5vdGhlciB0aGluZyBpcyBrZXJuZWwgbGFj a3MgZWZmaWNpZW50IHNvbHV0aW9uIGZvciB0cmFja2luZyBkaXJ0eQo+ID4gPiA+IHBhZ2VzIGJ5 IHZtYXAoKSwgdGhpcyB3aWxsIGxlYWQgaXNzdWVzIGlmIHZob3N0IGlzIHVzaW5nIGZpbGUgYmFj a2VkCj4gPiA+ID4gbWVtb3J5IHdoaWNoIG5lZWRzIGNhcmUgb2Ygd3JpdGViYWNrLiBUaGlzIHBh dGNoIHNvbHZlcyB0aGlzIGlzc3VlIGJ5Cj4gPiA+ID4ganVzdCBza2lwcGluZyB0aGUgdm1hIHRo YXQgaXMgZmlsZSBiYWNrZWQgYW5kIGZhbGxiYWNrIHRvIG5vcm1hbAo+ID4gPiA+IGNvcHlfdXNl cigpIGZyaWVuZHMuIFRoaXMgbWlnaHQgaW50cm9kdWNlIHNvbWUgb3ZlcmhlYWRzIGZvciBmaWxl Cj4gPiA+ID4gYmFja2VkIHVzZXJzIGJ1dCBjb25zaWRlciB0aGlzIHVzZSBjYXNlIGlzIHJhcmUg d2UgY291bGQgZG8KPiA+ID4gPiBvcHRpbWl6YXRpb25zIG9uIHRvcC4KPiA+ID4gPiAKPiA+ID4g PiBOb3RlIHRoYXQgdGhpcyB3YXMgb25seSBkb25lIHdoZW4gZGV2aWNlIElPVExCIGlzIG5vdCBl bmFibGVkLiBXZQo+ID4gPiA+IGNvdWxkIHVzZSBzaW1pbGFyIG1ldGhvZCB0byBvcHRpbWl6ZSBp dCBpbiB0aGUgZnV0dXJlLgo+ID4gPiA+IAo+ID4gPiA+IFRlc3RzIHNob3dzIGF0IG1vc3QgYWJv dXQgMjIlIGltcHJvdmVtZW50IG9uIFRYIFBQUyB3aGVuIHVzaW5nCj4gPiA+ID4gdmlydGlvLXVz ZXIgKyB2aG9zdF9uZXQgKyB4ZHAxICsgVEFQIG9uIDIuNkdIeiBCcm9hZHdlbGw6Cj4gPiA+ID4g Cj4gPiA+ID4gwqDCoMKgwqDCoMKgwqDCoCBTTUFQIG9uIHwgU01BUCBvZmYKPiA+ID4gPiBCZWZv cmU6IDUuME1wcHMgfCA2LjZNcHBzCj4gPiA+ID4gQWZ0ZXI6wqAgNi4xTXBwcyB8IDcuNE1wcHMK PiA+ID4gPiAKPiA+ID4gPiBTaWduZWQtb2ZmLWJ5OiBKYXNvbiBXYW5nIDxqYXNvd2FuZ0ByZWRo YXQuY29tPgo+ID4gPiAKPiA+ID4gU28gdGhpcyBpcyB0aGUgYnVsayBvZiB0aGUgY2hhbmdlLgo+ ID4gPiBUaHJlZWUgdGhpbmdzIHRoYXQgSSBuZWVkIHRvIGxvb2sgaW50bwo+ID4gPiAtIEFyZSB0 aGVyZSBhbnkgc2VjdXJpdHkgaXNzdWVzIHdpdGggYnlwYXNzaW5nIHRoZSBzcGVjdWxhdGlvbiBi YXJyaWVyCj4gPiA+IMKgwqAgdGhhdCBpcyBub3JtYWxseSBwcmVzZW50IGFmdGVyIGFjY2Vzc19v az8KPiA+IAo+ID4gCj4gPiBJZiB3ZSBjYW4gbWFrZSBzdXJlIHRoZSBieXBhc3Npbmcgd2FzIG9u bHkgdXNlZCBpbiBhIGt0aHJlYWQgKHZob3N0KSwgaXQKPiA+IHNob3VsZCBiZSBmaW5lIEkgdGhp bmsuCj4gPiAKPiA+IAo+ID4gPiAtIEhvdyBoYXJkIGRvZXMgdGhlIHNwZWNpYWwgaGFuZGxpbmcg Zm9yCj4gPiA+IMKgwqAgZmlsZSBiYWNrZWQgc3RvcmFnZSBtYWtlIHRlc3Rpbmc/Cj4gPiAKPiA+ IAo+ID4gSXQncyBhcyBzaW1wbGUgYXMgdW4tY29tbWVudGluZyB2aG9zdF9jYW5fdm1hcCgpPyBP ciBJIGNhbiB0cnkgdG8gaGFjawo+ID4gcWVtdSBvciBkcGRrIHRvIHRlc3QgdGhpcy4KPiA+IAo+ ID4gCj4gPiA+IMKgwqAgT24gdGhlIG9uZSBoYW5kIHdlIGNvdWxkIGFkZCBhIG1vZHVsZSBwYXJh bWV0ZXIgdG8KPiA+ID4gwqDCoCBmb3JjZSBjb3B5IHRvL2Zyb20gdXNlci4gb24gdGhlIG90aGVy IHRoYXQncwo+ID4gPiDCoMKgIGFub3RoZXIgY29uZmlndXJhdGlvbiB3ZSBuZWVkIHRvIHN1cHBv cnQuCj4gPiAKPiA+IAo+ID4gVGhhdCBzb3VuZHMgc3ViLW9wdGltYWwgc2luY2UgaXQgbGVhdmUg dGhlIGNob2ljZSB0byB1c2Vycy4KPiA+IAo+ID4gCj4gPiA+IMKgwqAgQnV0IGlvdGxiIGlzIG5v dCB1c2luZyB2bWFwLCBzbyBtYXliZSB0aGF0J3MgZW5vdWdoCj4gPiA+IMKgwqAgZm9yIHRlc3Rp bmcuCj4gPiA+IC0gSG93IGhhcmQgaXMgaXQgdG8gZmlndXJlIG91dCB3aGljaCBtb2RlIHVzZXMg d2hpY2ggY29kZS4KPiAKPiAKPiBJdCdzIGFzIHNpbXBsZSBhcyB0cmFjaW5nIF9fZ2V0X3VzZXIo KSB1c2FnZSBpbiB2aG9zdCBwcm9jZXNzPwo+IAo+IFRoYW5rcwoKV2VsbCB0aGVyZSBhcmUgbm93 IG10dSBub3RpZmllcnMgZXRjIGV0Yy4gSXQncyBoYXJkbHkgYXMgd2VsbApjb250YWluZWQgYXMg dGhhdC4KCgo+IAo+ID4gPiAKPiA+ID4gCj4gPiA+IAo+ID4gPiBNZWFud2hpbGUsIGNvdWxkIHlv dSBwbHMgcG9zdCBkYXRhIGNvbXBhcmluZyB0aGlzIGxhc3QgcGF0Y2ggd2l0aCB0aGUKPiA+ID4g YmVsb3c/wqAgVGhpcyByZW1vdmVzIHRoZSBzcGVjdWxhdGlvbiBiYXJyaWVyIHJlcGxhY2luZyBp dCB3aXRoIGEKPiA+ID4gKHVzZWxlc3MgYnV0IGF0IGxlYXN0IG1vcmUgbGlnaHR3ZWlnaHQpIGRh dGEgZGVwZW5kZW5jeS4KPiA+IAo+ID4gCj4gPiBTTUFQIG9mZgo+ID4gCj4gPiBZb3VyIHBhdGNo OiA3LjJNUFBzCj4gPiAKPiA+IHZtYXA6IDcuNE1wcHMKPiA+IAo+ID4gSSBkb24ndCB0ZXN0IFNN QVAgb24sIHNpbmNlIGl0IHdpbGwgYmUgbXVjaCBzbG93IGZvciBzdXJlLgo+ID4gCj4gPiBUaGFu a3MKPiA+IAo+ID4gCj4gPiA+IAo+ID4gPiBUaGFua3MhCj4gPiA+IAo+ID4gPiAKPiA+ID4gZGlm ZiAtLWdpdCBhL2RyaXZlcnMvdmhvc3Qvdmhvc3QuYyBiL2RyaXZlcnMvdmhvc3Qvdmhvc3QuYwo+ ID4gPiBpbmRleCBiYWM5MzlhZjhkYmIuLjM1MmVlN2UxNDQ3NiAxMDA2NDQKPiA+ID4gLS0tIGEv ZHJpdmVycy92aG9zdC92aG9zdC5jCj4gPiA+ICsrKyBiL2RyaXZlcnMvdmhvc3Qvdmhvc3QuYwo+ ID4gPiBAQCAtNzM5LDcgKzczOSw3IEBAIHN0YXRpYyBpbnQgdmhvc3RfY29weV90b191c2VyKHN0 cnVjdAo+ID4gPiB2aG9zdF92aXJ0cXVldWUgKnZxLCB2b2lkIF9fdXNlciAqdG8sCj4gPiA+IMKg wqDCoMKgwqAgaW50IHJldDsKPiA+ID4gwqAgwqDCoMKgwqDCoCBpZiAoIXZxLT5pb3RsYikKPiA+ ID4gLcKgwqDCoMKgwqDCoMKgIHJldHVybiBfX2NvcHlfdG9fdXNlcih0bywgZnJvbSwgc2l6ZSk7 Cj4gPiA+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4gY29weV90b191c2VyKHRvLCBmcm9tLCBzaXpl KTsKPiA+ID4gwqDCoMKgwqDCoCBlbHNlIHsKPiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKgIC8qIFRo aXMgZnVuY3Rpb24gc2hvdWxkIGJlIGNhbGxlZCBhZnRlciBpb3RsYgo+ID4gPiDCoMKgwqDCoMKg wqDCoMKgwqDCoCAqIHByZWZldGNoLCB3aGljaCBtZWFucyB3ZSdyZSBzdXJlIHRoYXQgYWxsIHZx Cj4gPiA+IEBAIC03NTIsNyArNzUyLDcgQEAgc3RhdGljIGludCB2aG9zdF9jb3B5X3RvX3VzZXIo c3RydWN0Cj4gPiA+IHZob3N0X3ZpcnRxdWV1ZSAqdnEsIHZvaWQgX191c2VyICp0bywKPiA+ID4g wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgVkhPU1RfQUREUl9V U0VEKTsKPiA+ID4gwqAgwqDCoMKgwqDCoMKgwqDCoMKgIGlmICh1YWRkcikKPiA+ID4gLcKgwqDC oMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIF9fY29weV90b191c2VyKHVhZGRyLCBmcm9tLCBzaXpl KTsKPiA+ID4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0dXJuIGNvcHlfdG9fdXNlcih1YWRk ciwgZnJvbSwgc2l6ZSk7Cj4gPiA+IMKgIMKgwqDCoMKgwqDCoMKgwqDCoCByZXQgPSB0cmFuc2xh dGVfZGVzYyh2cSwgKHU2NCkodWludHB0cl90KXRvLCBzaXplLAo+ID4gPiB2cS0+aW90bGJfaW92 LAo+ID4gPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBBUlJB WV9TSVpFKHZxLT5pb3RsYl9pb3YpLAo+ID4gPiBAQCAtNzc0LDcgKzc3NCw3IEBAIHN0YXRpYyBp bnQgdmhvc3RfY29weV9mcm9tX3VzZXIoc3RydWN0Cj4gPiA+IHZob3N0X3ZpcnRxdWV1ZSAqdnEs IHZvaWQgKnRvLAo+ID4gPiDCoMKgwqDCoMKgIGludCByZXQ7Cj4gPiA+IMKgIMKgwqDCoMKgwqAg aWYgKCF2cS0+aW90bGIpCj4gPiA+IC3CoMKgwqDCoMKgwqDCoCByZXR1cm4gX19jb3B5X2Zyb21f dXNlcih0bywgZnJvbSwgc2l6ZSk7Cj4gPiA+ICvCoMKgwqDCoMKgwqDCoCByZXR1cm4gY29weV9m cm9tX3VzZXIodG8sIGZyb20sIHNpemUpOwo+ID4gPiDCoMKgwqDCoMKgIGVsc2Ugewo+ID4gPiDC oMKgwqDCoMKgwqDCoMKgwqAgLyogVGhpcyBmdW5jdGlvbiBzaG91bGQgYmUgY2FsbGVkIGFmdGVy IGlvdGxiCj4gPiA+IMKgwqDCoMKgwqDCoMKgwqDCoMKgICogcHJlZmV0Y2gsIHdoaWNoIG1lYW5z IHdlJ3JlIHN1cmUgdGhhdCB2cQo+ID4gPiBAQCAtNzg3LDcgKzc4Nyw3IEBAIHN0YXRpYyBpbnQg dmhvc3RfY29weV9mcm9tX3VzZXIoc3RydWN0Cj4gPiA+IHZob3N0X3ZpcnRxdWV1ZSAqdnEsIHZv aWQgKnRvLAo+ID4gPiDCoMKgwqDCoMKgwqDCoMKgwqAgc3RydWN0IGlvdl9pdGVyIGY7Cj4gPiA+ IMKgIMKgwqDCoMKgwqDCoMKgwqDCoCBpZiAodWFkZHIpCj4gPiA+IC3CoMKgwqDCoMKgwqDCoMKg wqDCoMKgIHJldHVybiBfX2NvcHlfZnJvbV91c2VyKHRvLCB1YWRkciwgc2l6ZSk7Cj4gPiA+ICvC oMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldHVybiBjb3B5X2Zyb21fdXNlcih0bywgdWFkZHIsIHNp emUpOwo+ID4gPiDCoCDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0ID0gdHJhbnNsYXRlX2Rlc2ModnEs ICh1NjQpKHVpbnRwdHJfdClmcm9tLCBzaXplLAo+ID4gPiB2cS0+aW90bGJfaW92LAo+ID4gPiDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBBUlJBWV9TSVpFKHZx LT5pb3RsYl9pb3YpLAo+ID4gPiBAQCAtODU1LDEzICs4NTUsMTMgQEAgc3RhdGljIGlubGluZSB2 b2lkIF9fdXNlcgo+ID4gPiAqX192aG9zdF9nZXRfdXNlcihzdHJ1Y3Qgdmhvc3RfdmlydHF1ZXVl ICp2cSwKPiA+ID4gwqAgKHsgXAo+ID4gPiDCoMKgwqDCoMKgIGludCByZXQgPSAtRUZBVUxUOyBc Cj4gPiA+IMKgwqDCoMKgwqAgaWYgKCF2cS0+aW90bGIpIHsgXAo+ID4gPiAtwqDCoMKgwqDCoMKg wqAgcmV0ID0gX19wdXRfdXNlcih4LCBwdHIpOyBcCj4gPiA+ICvCoMKgwqDCoMKgwqDCoCByZXQg PSBwdXRfdXNlcih4LCBwdHIpOyBcCj4gPiA+IMKgwqDCoMKgwqAgfSBlbHNlIHsgXAo+ID4gPiDC oMKgwqDCoMKgwqDCoMKgwqAgX190eXBlb2ZfXyhwdHIpIHRvID0gXAo+ID4gPiDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCAoX190eXBlb2ZfXyhwdHIpKSBfX3Zob3N0X2dldF91c2VyKHZxLCBw dHIswqDCoMKgIFwKPiA+ID4gwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCBzaXplb2YoKnB0ciksIFZIT1NUX0FERFJfVVNFRCk7IFwKPiA+ID4gwqDCoMKgwqDC oMKgwqDCoMKgIGlmICh0byAhPSBOVUxMKSBcCj4gPiA+IC3CoMKgwqDCoMKgwqDCoMKgwqDCoMKg IHJldCA9IF9fcHV0X3VzZXIoeCwgdG8pOyBcCj4gPiA+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKg IHJldCA9IHB1dF91c2VyKHgsIHRvKTsgXAo+ID4gPiDCoMKgwqDCoMKgwqDCoMKgwqAgZWxzZSBc Cj4gPiA+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldCA9IC1FRkFVTFQ7wqDCoMKgIFwK PiA+ID4gwqDCoMKgwqDCoCB9IFwKPiA+ID4gQEAgLTg3MiwxNCArODcyLDE0IEBAIHN0YXRpYyBp bmxpbmUgdm9pZCBfX3VzZXIKPiA+ID4gKl9fdmhvc3RfZ2V0X3VzZXIoc3RydWN0IHZob3N0X3Zp cnRxdWV1ZSAqdnEsCj4gPiA+IMKgICh7IFwKPiA+ID4gwqDCoMKgwqDCoCBpbnQgcmV0OyBcCj4g PiA+IMKgwqDCoMKgwqAgaWYgKCF2cS0+aW90bGIpIHsgXAo+ID4gPiAtwqDCoMKgwqDCoMKgwqAg cmV0ID0gX19nZXRfdXNlcih4LCBwdHIpOyBcCj4gPiA+ICvCoMKgwqDCoMKgwqDCoCByZXQgPSBn ZXRfdXNlcih4LCBwdHIpOyBcCj4gPiA+IMKgwqDCoMKgwqAgfSBlbHNlIHsgXAo+ID4gPiDCoMKg wqDCoMKgwqDCoMKgwqAgX190eXBlb2ZfXyhwdHIpIGZyb20gPSBcCj4gPiA+IMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIChfX3R5cGVvZl9fKHB0cikpIF9fdmhvc3RfZ2V0X3VzZXIodnEsIHB0 ciwgXAo+ID4gPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIHNpemVvZigqcHRyKSwgXAo+ID4gPiDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHR5cGUpOyBc Cj4gPiA+IMKgwqDCoMKgwqDCoMKgwqDCoCBpZiAoZnJvbSAhPSBOVUxMKSBcCj4gPiA+IC3CoMKg wqDCoMKgwqDCoMKgwqDCoMKgIHJldCA9IF9fZ2V0X3VzZXIoeCwgZnJvbSk7IFwKPiA+ID4gK8Kg wqDCoMKgwqDCoMKgwqDCoMKgwqAgcmV0ID0gZ2V0X3VzZXIoeCwgZnJvbSk7IFwKPiA+ID4gwqDC oMKgwqDCoMKgwqDCoMKgIGVsc2UgXAo+ID4gPiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBy ZXQgPSAtRUZBVUxUOyBcCj4gPiA+IMKgwqDCoMKgwqAgfSBcCl9fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fClZpcnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApW aXJ0dWFsaXphdGlvbkBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxp bnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXphdGlvbg==