From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04314C432C0 for ; Mon, 18 Nov 2019 11:58:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CD50220862 for ; Mon, 18 Nov 2019 11:58:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726956AbfKRL6o (ORCPT ); Mon, 18 Nov 2019 06:58:44 -0500 Received: from mx2.suse.de ([195.135.220.15]:36158 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726506AbfKRL6m (ORCPT ); Mon, 18 Nov 2019 06:58:42 -0500 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 67C34AE68; Mon, 18 Nov 2019 11:58:37 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id 912131E4B0C; Mon, 18 Nov 2019 12:58:29 +0100 (CET) Date: Mon, 18 Nov 2019 12:58:29 +0100 From: Jan Kara To: John Hubbard Cc: Andrew Morton , Al Viro , Alex Williamson , Benjamin Herrenschmidt , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , Christoph Hellwig , Dan Williams , Daniel Vetter , Dave Chinner , David Airlie , "David S . Miller" , Ira Weiny , Jan Kara , Jason Gunthorpe , Jens Axboe , Jonathan Corbet , =?iso-8859-1?B?Suly9G1l?= Glisse , Magnus Karlsson , Mauro Carvalho Chehab , Michael Ellerman , Michal Hocko , Mike Kravetz , Paul Mackerras , Shuah Khan , Vlastimil Babka , bpf@vger.kernel.org, dri-devel@lists.freedesktop.org, kvm@vger.kernel.org, linux-block@vger.kernel.org, linux-doc@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-media@vger.kernel.org, linux-rdma@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, netdev@vger.kernel.org, linux-mm@kvack.org, LKML Subject: Re: [PATCH v5 17/24] mm/gup: track FOLL_PIN pages Message-ID: <20191118115829.GJ17319@quack2.suse.cz> References: <20191115055340.1825745-1-jhubbard@nvidia.com> <20191115055340.1825745-18-jhubbard@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20191115055340.1825745-18-jhubbard@nvidia.com> User-Agent: Mutt/1.10.1 (2018-07-13) Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On Thu 14-11-19 21:53:33, John Hubbard wrote: > Add tracking of pages that were pinned via FOLL_PIN. > > As mentioned in the FOLL_PIN documentation, callers who effectively set > FOLL_PIN are required to ultimately free such pages via put_user_page(). > The effect is similar to FOLL_GET, and may be thought of as "FOLL_GET > for DIO and/or RDMA use". > > Pages that have been pinned via FOLL_PIN are identifiable via a > new function call: > > bool page_dma_pinned(struct page *page); > > What to do in response to encountering such a page, is left to later > patchsets. There is discussion about this in [1]. ^^ missing this reference in the changelog... > This also changes a BUG_ON(), to a WARN_ON(), in follow_page_mask(). > > Suggested-by: Jan Kara > Suggested-by: Jérôme Glisse > Signed-off-by: John Hubbard > --- > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 6588d2e02628..db872766480f 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1054,6 +1054,8 @@ static inline __must_check bool try_get_page(struct page *page) > return true; > } > > +__must_check bool user_page_ref_inc(struct page *page); > + > static inline void put_page(struct page *page) > { > page = compound_head(page); > @@ -1071,29 +1073,70 @@ static inline void put_page(struct page *page) > __put_page(page); > } > > -/** > - * put_user_page() - release a gup-pinned page > - * @page: pointer to page to be released > +/* > + * GUP_PIN_COUNTING_BIAS, and the associated functions that use it, overload > + * the page's refcount so that two separate items are tracked: the original page > + * reference count, and also a new count of how many get_user_pages() calls were ^^ pin_user_pages() > + * made against the page. ("gup-pinned" is another term for the latter). > + * > + * With this scheme, get_user_pages() becomes special: such pages are marked ^^^ pin_user_pages() > + * as distinct from normal pages. As such, the put_user_page() call (and its > + * variants) must be used in order to release gup-pinned pages. > + * > + * Choice of value: > * > - * Pages that were pinned via pin_user_pages*() must be released via either > - * put_user_page(), or one of the put_user_pages*() routines. This is so that > - * eventually such pages can be separately tracked and uniquely handled. In > - * particular, interactions with RDMA and filesystems need special handling. > + * By making GUP_PIN_COUNTING_BIAS a power of two, debugging of page reference > + * counts with respect to get_user_pages() and put_user_page() becomes simpler, ^^^ pin_user_pages() > + * due to the fact that adding an even power of two to the page refcount has > + * the effect of using only the upper N bits, for the code that counts up using > + * the bias value. This means that the lower bits are left for the exclusive > + * use of the original code that increments and decrements by one (or at least, > + * by much smaller values than the bias value). > * > - * put_user_page() and put_page() are not interchangeable, despite this early > - * implementation that makes them look the same. put_user_page() calls must > - * be perfectly matched up with pin*() calls. > + * Of course, once the lower bits overflow into the upper bits (and this is > + * OK, because subtraction recovers the original values), then visual inspection > + * no longer suffices to directly view the separate counts. However, for normal > + * applications that don't have huge page reference counts, this won't be an > + * issue. > + * > + * Locking: the lockless algorithm described in page_cache_get_speculative() > + * and page_cache_gup_pin_speculative() provides safe operation for > + * get_user_pages and page_mkclean and other calls that race to set up page > + * table entries. > */ ... > @@ -2070,9 +2191,16 @@ static int gup_hugepte(pte_t *ptep, unsigned long sz, unsigned long addr, > page = head + ((addr & (sz-1)) >> PAGE_SHIFT); > refs = __record_subpages(page, addr, end, pages + *nr); > > - head = try_get_compound_head(head, refs); > - if (!head) > - return 0; > + if (flags & FOLL_PIN) { > + head = page; > + if (unlikely(!user_page_ref_inc(head))) > + return 0; > + head = page; Why do you assign 'head' twice? Also the refcounting logic is repeated several times so perhaps you can factor it out in to a helper function or even move it to __record_subpages()? > + } else { > + head = try_get_compound_head(head, refs); > + if (!head) > + return 0; > + } > > if (unlikely(pte_val(pte) != pte_val(*ptep))) { > put_compound_head(head, refs); So this will do the wrong thing for FOLL_PIN. We took just one "pin" reference there but here we'll release 'refs' normal references AFAICT. Also the fact that you take just one pin reference for each huge page substantially changes how GUP refcounting works in the huge page case. Currently, FOLL_GET users can be completely agnostic of huge pages. So you can e.g. GUP whole 2 MB page, submit it as 2 different bios and then drop page references from each bio completion function. With your new FOLL_PIN behavior you cannot do that and I believe it will be a problem for some users. So I think you have to maintain the behavior that you increase the head->_refcount by (refs * GUP_PIN_COUNTING_BIAS) here. Honza -- Jan Kara SUSE Labs, CR From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 265C6C432C0 for ; Mon, 18 Nov 2019 13:52:18 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5AA82071C for ; Mon, 18 Nov 2019 13:52:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5AA82071C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 47Gr3z1Vx5zDqNB for ; Tue, 19 Nov 2019 00:52:15 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=suse.cz (client-ip=195.135.220.15; helo=mx1.suse.de; envelope-from=jack@suse.cz; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=suse.cz Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 47GnXz1XRzzDqSQ for ; Mon, 18 Nov 2019 22:58:43 +1100 (AEDT) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 67C34AE68; Mon, 18 Nov 2019 11:58:37 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id 912131E4B0C; Mon, 18 Nov 2019 12:58:29 +0100 (CET) Date: Mon, 18 Nov 2019 12:58:29 +0100 From: Jan Kara To: John Hubbard Subject: Re: [PATCH v5 17/24] mm/gup: track FOLL_PIN pages Message-ID: <20191118115829.GJ17319@quack2.suse.cz> References: <20191115055340.1825745-1-jhubbard@nvidia.com> <20191115055340.1825745-18-jhubbard@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20191115055340.1825745-18-jhubbard@nvidia.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, LKML , linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Jonathan Corbet , linux-rdma@vger.kernel.org, Christoph Hellwig , Jason Gunthorpe , Vlastimil Babka , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , linux-media@vger.kernel.org, Shuah Khan , linux-block@vger.kernel.org, =?iso-8859-1?B?Suly9G1l?= Glisse , Al Viro , Dan Williams , Mauro Carvalho Chehab , bpf@vger.kernel.org, Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, Alex Williamson , Daniel Vetter , linux-fsdevel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Thu 14-11-19 21:53:33, John Hubbard wrote: > Add tracking of pages that were pinned via FOLL_PIN. > > As mentioned in the FOLL_PIN documentation, callers who effectively set > FOLL_PIN are required to ultimately free such pages via put_user_page(). > The effect is similar to FOLL_GET, and may be thought of as "FOLL_GET > for DIO and/or RDMA use". > > Pages that have been pinned via FOLL_PIN are identifiable via a > new function call: > > bool page_dma_pinned(struct page *page); > > What to do in response to encountering such a page, is left to later > patchsets. There is discussion about this in [1]. ^^ missing this reference in the changelog... > This also changes a BUG_ON(), to a WARN_ON(), in follow_page_mask(). > > Suggested-by: Jan Kara > Suggested-by: Jérôme Glisse > Signed-off-by: John Hubbard > --- > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 6588d2e02628..db872766480f 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -1054,6 +1054,8 @@ static inline __must_check bool try_get_page(struct page *page) > return true; > } > > +__must_check bool user_page_ref_inc(struct page *page); > + > static inline void put_page(struct page *page) > { > page = compound_head(page); > @@ -1071,29 +1073,70 @@ static inline void put_page(struct page *page) > __put_page(page); > } > > -/** > - * put_user_page() - release a gup-pinned page > - * @page: pointer to page to be released > +/* > + * GUP_PIN_COUNTING_BIAS, and the associated functions that use it, overload > + * the page's refcount so that two separate items are tracked: the original page > + * reference count, and also a new count of how many get_user_pages() calls were ^^ pin_user_pages() > + * made against the page. ("gup-pinned" is another term for the latter). > + * > + * With this scheme, get_user_pages() becomes special: such pages are marked ^^^ pin_user_pages() > + * as distinct from normal pages. As such, the put_user_page() call (and its > + * variants) must be used in order to release gup-pinned pages. > + * > + * Choice of value: > * > - * Pages that were pinned via pin_user_pages*() must be released via either > - * put_user_page(), or one of the put_user_pages*() routines. This is so that > - * eventually such pages can be separately tracked and uniquely handled. In > - * particular, interactions with RDMA and filesystems need special handling. > + * By making GUP_PIN_COUNTING_BIAS a power of two, debugging of page reference > + * counts with respect to get_user_pages() and put_user_page() becomes simpler, ^^^ pin_user_pages() > + * due to the fact that adding an even power of two to the page refcount has > + * the effect of using only the upper N bits, for the code that counts up using > + * the bias value. This means that the lower bits are left for the exclusive > + * use of the original code that increments and decrements by one (or at least, > + * by much smaller values than the bias value). > * > - * put_user_page() and put_page() are not interchangeable, despite this early > - * implementation that makes them look the same. put_user_page() calls must > - * be perfectly matched up with pin*() calls. > + * Of course, once the lower bits overflow into the upper bits (and this is > + * OK, because subtraction recovers the original values), then visual inspection > + * no longer suffices to directly view the separate counts. However, for normal > + * applications that don't have huge page reference counts, this won't be an > + * issue. > + * > + * Locking: the lockless algorithm described in page_cache_get_speculative() > + * and page_cache_gup_pin_speculative() provides safe operation for > + * get_user_pages and page_mkclean and other calls that race to set up page > + * table entries. > */ ... > @@ -2070,9 +2191,16 @@ static int gup_hugepte(pte_t *ptep, unsigned long sz, unsigned long addr, > page = head + ((addr & (sz-1)) >> PAGE_SHIFT); > refs = __record_subpages(page, addr, end, pages + *nr); > > - head = try_get_compound_head(head, refs); > - if (!head) > - return 0; > + if (flags & FOLL_PIN) { > + head = page; > + if (unlikely(!user_page_ref_inc(head))) > + return 0; > + head = page; Why do you assign 'head' twice? Also the refcounting logic is repeated several times so perhaps you can factor it out in to a helper function or even move it to __record_subpages()? > + } else { > + head = try_get_compound_head(head, refs); > + if (!head) > + return 0; > + } > > if (unlikely(pte_val(pte) != pte_val(*ptep))) { > put_compound_head(head, refs); So this will do the wrong thing for FOLL_PIN. We took just one "pin" reference there but here we'll release 'refs' normal references AFAICT. Also the fact that you take just one pin reference for each huge page substantially changes how GUP refcounting works in the huge page case. Currently, FOLL_GET users can be completely agnostic of huge pages. So you can e.g. GUP whole 2 MB page, submit it as 2 different bios and then drop page references from each bio completion function. With your new FOLL_PIN behavior you cannot do that and I believe it will be a problem for some users. So I think you have to maintain the behavior that you increase the head->_refcount by (refs * GUP_PIN_COUNTING_BIAS) here. Honza -- Jan Kara SUSE Labs, CR From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jan Kara Subject: Re: [PATCH v5 17/24] mm/gup: track FOLL_PIN pages Date: Mon, 18 Nov 2019 12:58:29 +0100 Message-ID: <20191118115829.GJ17319@quack2.suse.cz> References: <20191115055340.1825745-1-jhubbard@nvidia.com> <20191115055340.1825745-18-jhubbard@nvidia.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by gabe.freedesktop.org (Postfix) with ESMTPS id D17E16E48F for ; Mon, 18 Nov 2019 11:58:39 +0000 (UTC) Content-Disposition: inline In-Reply-To: <20191115055340.1825745-18-jhubbard@nvidia.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: John Hubbard Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, LKML , linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Jonathan Corbet , linux-rdma@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Jason Gunthorpe , Vlastimil Babka , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , linux-media@vger.kernel.org, Shuah Khan , linux-block@vger.kernel.org, =?iso-8859-1?B?Suly9G1l?= Glisse , Al Viro , Dan Williams , Mauro Carvalho Chehab List-Id: dri-devel@lists.freedesktop.org T24gVGh1IDE0LTExLTE5IDIxOjUzOjMzLCBKb2huIEh1YmJhcmQgd3JvdGU6Cj4gQWRkIHRyYWNr aW5nIG9mIHBhZ2VzIHRoYXQgd2VyZSBwaW5uZWQgdmlhIEZPTExfUElOLgo+IAo+IEFzIG1lbnRp b25lZCBpbiB0aGUgRk9MTF9QSU4gZG9jdW1lbnRhdGlvbiwgY2FsbGVycyB3aG8gZWZmZWN0aXZl bHkgc2V0Cj4gRk9MTF9QSU4gYXJlIHJlcXVpcmVkIHRvIHVsdGltYXRlbHkgZnJlZSBzdWNoIHBh Z2VzIHZpYSBwdXRfdXNlcl9wYWdlKCkuCj4gVGhlIGVmZmVjdCBpcyBzaW1pbGFyIHRvIEZPTExf R0VULCBhbmQgbWF5IGJlIHRob3VnaHQgb2YgYXMgIkZPTExfR0VUCj4gZm9yIERJTyBhbmQvb3Ig UkRNQSB1c2UiLgo+IAo+IFBhZ2VzIHRoYXQgaGF2ZSBiZWVuIHBpbm5lZCB2aWEgRk9MTF9QSU4g YXJlIGlkZW50aWZpYWJsZSB2aWEgYQo+IG5ldyBmdW5jdGlvbiBjYWxsOgo+IAo+ICAgIGJvb2wg cGFnZV9kbWFfcGlubmVkKHN0cnVjdCBwYWdlICpwYWdlKTsKPiAKPiBXaGF0IHRvIGRvIGluIHJl c3BvbnNlIHRvIGVuY291bnRlcmluZyBzdWNoIGEgcGFnZSwgaXMgbGVmdCB0byBsYXRlcgo+IHBh dGNoc2V0cy4gVGhlcmUgaXMgZGlzY3Vzc2lvbiBhYm91dCB0aGlzIGluIFsxXS4KCQkJCQkJXl4g bWlzc2luZyB0aGlzIHJlZmVyZW5jZQppbiB0aGUgY2hhbmdlbG9nLi4uCgo+IFRoaXMgYWxzbyBj aGFuZ2VzIGEgQlVHX09OKCksIHRvIGEgV0FSTl9PTigpLCBpbiBmb2xsb3dfcGFnZV9tYXNrKCku Cj4gCj4gU3VnZ2VzdGVkLWJ5OiBKYW4gS2FyYSA8amFja0BzdXNlLmN6Pgo+IFN1Z2dlc3RlZC1i eTogSsOpcsO0bWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+Cj4gU2lnbmVkLW9mZi1ieTog Sm9obiBIdWJiYXJkIDxqaHViYmFyZEBudmlkaWEuY29tPgo+IC0tLQo+IGRpZmYgLS1naXQgYS9p bmNsdWRlL2xpbnV4L21tLmggYi9pbmNsdWRlL2xpbnV4L21tLmgKPiBpbmRleCA2NTg4ZDJlMDI2 MjguLmRiODcyNzY2NDgwZiAxMDA2NDQKPiAtLS0gYS9pbmNsdWRlL2xpbnV4L21tLmgKPiArKysg Yi9pbmNsdWRlL2xpbnV4L21tLmgKPiBAQCAtMTA1NCw2ICsxMDU0LDggQEAgc3RhdGljIGlubGlu ZSBfX211c3RfY2hlY2sgYm9vbCB0cnlfZ2V0X3BhZ2Uoc3RydWN0IHBhZ2UgKnBhZ2UpCj4gIAly ZXR1cm4gdHJ1ZTsKPiAgfQo+ICAKPiArX19tdXN0X2NoZWNrIGJvb2wgdXNlcl9wYWdlX3JlZl9p bmMoc3RydWN0IHBhZ2UgKnBhZ2UpOwo+ICsKPiAgc3RhdGljIGlubGluZSB2b2lkIHB1dF9wYWdl KHN0cnVjdCBwYWdlICpwYWdlKQo+ICB7Cj4gIAlwYWdlID0gY29tcG91bmRfaGVhZChwYWdlKTsK PiBAQCAtMTA3MSwyOSArMTA3Myw3MCBAQCBzdGF0aWMgaW5saW5lIHZvaWQgcHV0X3BhZ2Uoc3Ry dWN0IHBhZ2UgKnBhZ2UpCj4gIAkJX19wdXRfcGFnZShwYWdlKTsKPiAgfQo+ICAKPiAtLyoqCj4g LSAqIHB1dF91c2VyX3BhZ2UoKSAtIHJlbGVhc2UgYSBndXAtcGlubmVkIHBhZ2UKPiAtICogQHBh Z2U6ICAgICAgICAgICAgcG9pbnRlciB0byBwYWdlIHRvIGJlIHJlbGVhc2VkCj4gKy8qCj4gKyAq IEdVUF9QSU5fQ09VTlRJTkdfQklBUywgYW5kIHRoZSBhc3NvY2lhdGVkIGZ1bmN0aW9ucyB0aGF0 IHVzZSBpdCwgb3ZlcmxvYWQKPiArICogdGhlIHBhZ2UncyByZWZjb3VudCBzbyB0aGF0IHR3byBz ZXBhcmF0ZSBpdGVtcyBhcmUgdHJhY2tlZDogdGhlIG9yaWdpbmFsIHBhZ2UKPiArICogcmVmZXJl bmNlIGNvdW50LCBhbmQgYWxzbyBhIG5ldyBjb3VudCBvZiBob3cgbWFueSBnZXRfdXNlcl9wYWdl cygpIGNhbGxzIHdlcmUKCQkJCQkJCV5eIHBpbl91c2VyX3BhZ2VzKCkKCj4gKyAqIG1hZGUgYWdh aW5zdCB0aGUgcGFnZS4gKCJndXAtcGlubmVkIiBpcyBhbm90aGVyIHRlcm0gZm9yIHRoZSBsYXR0 ZXIpLgo+ICsgKgo+ICsgKiBXaXRoIHRoaXMgc2NoZW1lLCBnZXRfdXNlcl9wYWdlcygpIGJlY29t ZXMgc3BlY2lhbDogc3VjaCBwYWdlcyBhcmUgbWFya2VkCgkJCV5eXiBwaW5fdXNlcl9wYWdlcygp Cgo+ICsgKiBhcyBkaXN0aW5jdCBmcm9tIG5vcm1hbCBwYWdlcy4gQXMgc3VjaCwgdGhlIHB1dF91 c2VyX3BhZ2UoKSBjYWxsIChhbmQgaXRzCj4gKyAqIHZhcmlhbnRzKSBtdXN0IGJlIHVzZWQgaW4g b3JkZXIgdG8gcmVsZWFzZSBndXAtcGlubmVkIHBhZ2VzLgo+ICsgKgo+ICsgKiBDaG9pY2Ugb2Yg dmFsdWU6Cj4gICAqCj4gLSAqIFBhZ2VzIHRoYXQgd2VyZSBwaW5uZWQgdmlhIHBpbl91c2VyX3Bh Z2VzKigpIG11c3QgYmUgcmVsZWFzZWQgdmlhIGVpdGhlcgo+IC0gKiBwdXRfdXNlcl9wYWdlKCks IG9yIG9uZSBvZiB0aGUgcHV0X3VzZXJfcGFnZXMqKCkgcm91dGluZXMuIFRoaXMgaXMgc28gdGhh dAo+IC0gKiBldmVudHVhbGx5IHN1Y2ggcGFnZXMgY2FuIGJlIHNlcGFyYXRlbHkgdHJhY2tlZCBh bmQgdW5pcXVlbHkgaGFuZGxlZC4gSW4KPiAtICogcGFydGljdWxhciwgaW50ZXJhY3Rpb25zIHdp dGggUkRNQSBhbmQgZmlsZXN5c3RlbXMgbmVlZCBzcGVjaWFsIGhhbmRsaW5nLgo+ICsgKiBCeSBt YWtpbmcgR1VQX1BJTl9DT1VOVElOR19CSUFTIGEgcG93ZXIgb2YgdHdvLCBkZWJ1Z2dpbmcgb2Yg cGFnZSByZWZlcmVuY2UKPiArICogY291bnRzIHdpdGggcmVzcGVjdCB0byBnZXRfdXNlcl9wYWdl cygpIGFuZCBwdXRfdXNlcl9wYWdlKCkgYmVjb21lcyBzaW1wbGVyLAoJCQkJXl5eIHBpbl91c2Vy X3BhZ2VzKCkKCj4gKyAqIGR1ZSB0byB0aGUgZmFjdCB0aGF0IGFkZGluZyBhbiBldmVuIHBvd2Vy IG9mIHR3byB0byB0aGUgcGFnZSByZWZjb3VudCBoYXMKPiArICogdGhlIGVmZmVjdCBvZiB1c2lu ZyBvbmx5IHRoZSB1cHBlciBOIGJpdHMsIGZvciB0aGUgY29kZSB0aGF0IGNvdW50cyB1cCB1c2lu Zwo+ICsgKiB0aGUgYmlhcyB2YWx1ZS4gVGhpcyBtZWFucyB0aGF0IHRoZSBsb3dlciBiaXRzIGFy ZSBsZWZ0IGZvciB0aGUgZXhjbHVzaXZlCj4gKyAqIHVzZSBvZiB0aGUgb3JpZ2luYWwgY29kZSB0 aGF0IGluY3JlbWVudHMgYW5kIGRlY3JlbWVudHMgYnkgb25lIChvciBhdCBsZWFzdCwKPiArICog YnkgbXVjaCBzbWFsbGVyIHZhbHVlcyB0aGFuIHRoZSBiaWFzIHZhbHVlKS4KPiAgICoKPiAtICog cHV0X3VzZXJfcGFnZSgpIGFuZCBwdXRfcGFnZSgpIGFyZSBub3QgaW50ZXJjaGFuZ2VhYmxlLCBk ZXNwaXRlIHRoaXMgZWFybHkKPiAtICogaW1wbGVtZW50YXRpb24gdGhhdCBtYWtlcyB0aGVtIGxv b2sgdGhlIHNhbWUuIHB1dF91c2VyX3BhZ2UoKSBjYWxscyBtdXN0Cj4gLSAqIGJlIHBlcmZlY3Rs eSBtYXRjaGVkIHVwIHdpdGggcGluKigpIGNhbGxzLgo+ICsgKiBPZiBjb3Vyc2UsIG9uY2UgdGhl IGxvd2VyIGJpdHMgb3ZlcmZsb3cgaW50byB0aGUgdXBwZXIgYml0cyAoYW5kIHRoaXMgaXMKPiAr ICogT0ssIGJlY2F1c2Ugc3VidHJhY3Rpb24gcmVjb3ZlcnMgdGhlIG9yaWdpbmFsIHZhbHVlcyks IHRoZW4gdmlzdWFsIGluc3BlY3Rpb24KPiArICogbm8gbG9uZ2VyIHN1ZmZpY2VzIHRvIGRpcmVj dGx5IHZpZXcgdGhlIHNlcGFyYXRlIGNvdW50cy4gSG93ZXZlciwgZm9yIG5vcm1hbAo+ICsgKiBh cHBsaWNhdGlvbnMgdGhhdCBkb24ndCBoYXZlIGh1Z2UgcGFnZSByZWZlcmVuY2UgY291bnRzLCB0 aGlzIHdvbid0IGJlIGFuCj4gKyAqIGlzc3VlLgo+ICsgKgo+ICsgKiBMb2NraW5nOiB0aGUgbG9j a2xlc3MgYWxnb3JpdGhtIGRlc2NyaWJlZCBpbiBwYWdlX2NhY2hlX2dldF9zcGVjdWxhdGl2ZSgp Cj4gKyAqIGFuZCBwYWdlX2NhY2hlX2d1cF9waW5fc3BlY3VsYXRpdmUoKSBwcm92aWRlcyBzYWZl IG9wZXJhdGlvbiBmb3IKPiArICogZ2V0X3VzZXJfcGFnZXMgYW5kIHBhZ2VfbWtjbGVhbiBhbmQg b3RoZXIgY2FsbHMgdGhhdCByYWNlIHRvIHNldCB1cCBwYWdlCj4gKyAqIHRhYmxlIGVudHJpZXMu Cj4gICAqLwouLi4KPiBAQCAtMjA3MCw5ICsyMTkxLDE2IEBAIHN0YXRpYyBpbnQgZ3VwX2h1Z2Vw dGUocHRlX3QgKnB0ZXAsIHVuc2lnbmVkIGxvbmcgc3osIHVuc2lnbmVkIGxvbmcgYWRkciwKPiAg CXBhZ2UgPSBoZWFkICsgKChhZGRyICYgKHN6LTEpKSA+PiBQQUdFX1NISUZUKTsKPiAgCXJlZnMg PSBfX3JlY29yZF9zdWJwYWdlcyhwYWdlLCBhZGRyLCBlbmQsIHBhZ2VzICsgKm5yKTsKPiAgCj4g LQloZWFkID0gdHJ5X2dldF9jb21wb3VuZF9oZWFkKGhlYWQsIHJlZnMpOwo+IC0JaWYgKCFoZWFk KQo+IC0JCXJldHVybiAwOwo+ICsJaWYgKGZsYWdzICYgRk9MTF9QSU4pIHsKPiArCQloZWFkID0g cGFnZTsKPiArCQlpZiAodW5saWtlbHkoIXVzZXJfcGFnZV9yZWZfaW5jKGhlYWQpKSkKPiArCQkJ cmV0dXJuIDA7Cj4gKwkJaGVhZCA9IHBhZ2U7CgpXaHkgZG8geW91IGFzc2lnbiAnaGVhZCcgdHdp Y2U/IEFsc28gdGhlIHJlZmNvdW50aW5nIGxvZ2ljIGlzIHJlcGVhdGVkCnNldmVyYWwgdGltZXMg c28gcGVyaGFwcyB5b3UgY2FuIGZhY3RvciBpdCBvdXQgaW4gdG8gYSBoZWxwZXIgZnVuY3Rpb24g b3IKZXZlbiBtb3ZlIGl0IHRvIF9fcmVjb3JkX3N1YnBhZ2VzKCk/Cgo+ICsJfSBlbHNlIHsKPiAr CQloZWFkID0gdHJ5X2dldF9jb21wb3VuZF9oZWFkKGhlYWQsIHJlZnMpOwo+ICsJCWlmICghaGVh ZCkKPiArCQkJcmV0dXJuIDA7Cj4gKwl9Cj4gIAo+ICAJaWYgKHVubGlrZWx5KHB0ZV92YWwocHRl KSAhPSBwdGVfdmFsKCpwdGVwKSkpIHsKPiAgCQlwdXRfY29tcG91bmRfaGVhZChoZWFkLCByZWZz KTsKClNvIHRoaXMgd2lsbCBkbyB0aGUgd3JvbmcgdGhpbmcgZm9yIEZPTExfUElOLiBXZSB0b29r IGp1c3Qgb25lICJwaW4iCnJlZmVyZW5jZSB0aGVyZSBidXQgaGVyZSB3ZSdsbCByZWxlYXNlICdy ZWZzJyBub3JtYWwgcmVmZXJlbmNlcyBBRkFJQ1QuCkFsc28gdGhlIGZhY3QgdGhhdCB5b3UgdGFr ZSBqdXN0IG9uZSBwaW4gcmVmZXJlbmNlIGZvciBlYWNoIGh1Z2UgcGFnZQpzdWJzdGFudGlhbGx5 IGNoYW5nZXMgaG93IEdVUCByZWZjb3VudGluZyB3b3JrcyBpbiB0aGUgaHVnZSBwYWdlIGNhc2Uu CkN1cnJlbnRseSwgRk9MTF9HRVQgdXNlcnMgY2FuIGJlIGNvbXBsZXRlbHkgYWdub3N0aWMgb2Yg aHVnZSBwYWdlcy4gU28geW91CmNhbiBlLmcuIEdVUCB3aG9sZSAyIE1CIHBhZ2UsIHN1Ym1pdCBp dCBhcyAyIGRpZmZlcmVudCBiaW9zIGFuZCB0aGVuCmRyb3AgcGFnZSByZWZlcmVuY2VzIGZyb20g ZWFjaCBiaW8gY29tcGxldGlvbiBmdW5jdGlvbi4gV2l0aCB5b3VyIG5ldwpGT0xMX1BJTiBiZWhh dmlvciB5b3UgY2Fubm90IGRvIHRoYXQgYW5kIEkgYmVsaWV2ZSBpdCB3aWxsIGJlIGEgcHJvYmxl bSBmb3IKc29tZSB1c2Vycy4gU28gSSB0aGluayB5b3UgaGF2ZSB0byBtYWludGFpbiB0aGUgYmVo YXZpb3IgdGhhdCB5b3UgaW5jcmVhc2UKdGhlIGhlYWQtPl9yZWZjb3VudCBieSAocmVmcyAqIEdV UF9QSU5fQ09VTlRJTkdfQklBUykgaGVyZS4KCgkJCQkJCQkJSG9uemEKLS0gCkphbiBLYXJhIDxq YWNrQHN1c2UuY29tPgpTVVNFIExhYnMsIENSCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3Rz LmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2RyaS1kZXZlbA== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8730C43215 for ; Mon, 18 Nov 2019 11:58:42 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B9A892068D for ; Mon, 18 Nov 2019 11:58:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B9A892068D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 159AA6E48F; Mon, 18 Nov 2019 11:58:42 +0000 (UTC) Received: from mx1.suse.de (mx2.suse.de [195.135.220.15]) by gabe.freedesktop.org (Postfix) with ESMTPS id D17E16E48F for ; Mon, 18 Nov 2019 11:58:39 +0000 (UTC) X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id 67C34AE68; Mon, 18 Nov 2019 11:58:37 +0000 (UTC) Received: by quack2.suse.cz (Postfix, from userid 1000) id 912131E4B0C; Mon, 18 Nov 2019 12:58:29 +0100 (CET) Date: Mon, 18 Nov 2019 12:58:29 +0100 From: Jan Kara To: John Hubbard Subject: Re: [PATCH v5 17/24] mm/gup: track FOLL_PIN pages Message-ID: <20191118115829.GJ17319@quack2.suse.cz> References: <20191115055340.1825745-1-jhubbard@nvidia.com> <20191115055340.1825745-18-jhubbard@nvidia.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20191115055340.1825745-18-jhubbard@nvidia.com> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Michal Hocko , Jan Kara , kvm@vger.kernel.org, linux-doc@vger.kernel.org, David Airlie , Dave Chinner , dri-devel@lists.freedesktop.org, LKML , linux-mm@kvack.org, Paul Mackerras , linux-kselftest@vger.kernel.org, Ira Weiny , Jonathan Corbet , linux-rdma@vger.kernel.org, Michael Ellerman , Christoph Hellwig , Jason Gunthorpe , Vlastimil Babka , =?iso-8859-1?Q?Bj=F6rn_T=F6pel?= , linux-media@vger.kernel.org, Shuah Khan , linux-block@vger.kernel.org, =?iso-8859-1?B?Suly9G1l?= Glisse , Al Viro , Dan Williams , Mauro Carvalho Chehab , bpf@vger.kernel.org, Magnus Karlsson , Jens Axboe , netdev@vger.kernel.org, Alex Williamson , linux-fsdevel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, "David S . Miller" , Mike Kravetz Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Message-ID: <20191118115829.hlZXBcdKjVwmttKgtNqaheaoG-kXLt5UcJO7o34YOlA@z> T24gVGh1IDE0LTExLTE5IDIxOjUzOjMzLCBKb2huIEh1YmJhcmQgd3JvdGU6Cj4gQWRkIHRyYWNr aW5nIG9mIHBhZ2VzIHRoYXQgd2VyZSBwaW5uZWQgdmlhIEZPTExfUElOLgo+IAo+IEFzIG1lbnRp b25lZCBpbiB0aGUgRk9MTF9QSU4gZG9jdW1lbnRhdGlvbiwgY2FsbGVycyB3aG8gZWZmZWN0aXZl bHkgc2V0Cj4gRk9MTF9QSU4gYXJlIHJlcXVpcmVkIHRvIHVsdGltYXRlbHkgZnJlZSBzdWNoIHBh Z2VzIHZpYSBwdXRfdXNlcl9wYWdlKCkuCj4gVGhlIGVmZmVjdCBpcyBzaW1pbGFyIHRvIEZPTExf R0VULCBhbmQgbWF5IGJlIHRob3VnaHQgb2YgYXMgIkZPTExfR0VUCj4gZm9yIERJTyBhbmQvb3Ig UkRNQSB1c2UiLgo+IAo+IFBhZ2VzIHRoYXQgaGF2ZSBiZWVuIHBpbm5lZCB2aWEgRk9MTF9QSU4g YXJlIGlkZW50aWZpYWJsZSB2aWEgYQo+IG5ldyBmdW5jdGlvbiBjYWxsOgo+IAo+ICAgIGJvb2wg cGFnZV9kbWFfcGlubmVkKHN0cnVjdCBwYWdlICpwYWdlKTsKPiAKPiBXaGF0IHRvIGRvIGluIHJl c3BvbnNlIHRvIGVuY291bnRlcmluZyBzdWNoIGEgcGFnZSwgaXMgbGVmdCB0byBsYXRlcgo+IHBh dGNoc2V0cy4gVGhlcmUgaXMgZGlzY3Vzc2lvbiBhYm91dCB0aGlzIGluIFsxXS4KCQkJCQkJXl4g bWlzc2luZyB0aGlzIHJlZmVyZW5jZQppbiB0aGUgY2hhbmdlbG9nLi4uCgo+IFRoaXMgYWxzbyBj aGFuZ2VzIGEgQlVHX09OKCksIHRvIGEgV0FSTl9PTigpLCBpbiBmb2xsb3dfcGFnZV9tYXNrKCku Cj4gCj4gU3VnZ2VzdGVkLWJ5OiBKYW4gS2FyYSA8amFja0BzdXNlLmN6Pgo+IFN1Z2dlc3RlZC1i eTogSsOpcsO0bWUgR2xpc3NlIDxqZ2xpc3NlQHJlZGhhdC5jb20+Cj4gU2lnbmVkLW9mZi1ieTog Sm9obiBIdWJiYXJkIDxqaHViYmFyZEBudmlkaWEuY29tPgo+IC0tLQo+IGRpZmYgLS1naXQgYS9p bmNsdWRlL2xpbnV4L21tLmggYi9pbmNsdWRlL2xpbnV4L21tLmgKPiBpbmRleCA2NTg4ZDJlMDI2 MjguLmRiODcyNzY2NDgwZiAxMDA2NDQKPiAtLS0gYS9pbmNsdWRlL2xpbnV4L21tLmgKPiArKysg Yi9pbmNsdWRlL2xpbnV4L21tLmgKPiBAQCAtMTA1NCw2ICsxMDU0LDggQEAgc3RhdGljIGlubGlu ZSBfX211c3RfY2hlY2sgYm9vbCB0cnlfZ2V0X3BhZ2Uoc3RydWN0IHBhZ2UgKnBhZ2UpCj4gIAly ZXR1cm4gdHJ1ZTsKPiAgfQo+ICAKPiArX19tdXN0X2NoZWNrIGJvb2wgdXNlcl9wYWdlX3JlZl9p bmMoc3RydWN0IHBhZ2UgKnBhZ2UpOwo+ICsKPiAgc3RhdGljIGlubGluZSB2b2lkIHB1dF9wYWdl KHN0cnVjdCBwYWdlICpwYWdlKQo+ICB7Cj4gIAlwYWdlID0gY29tcG91bmRfaGVhZChwYWdlKTsK PiBAQCAtMTA3MSwyOSArMTA3Myw3MCBAQCBzdGF0aWMgaW5saW5lIHZvaWQgcHV0X3BhZ2Uoc3Ry dWN0IHBhZ2UgKnBhZ2UpCj4gIAkJX19wdXRfcGFnZShwYWdlKTsKPiAgfQo+ICAKPiAtLyoqCj4g LSAqIHB1dF91c2VyX3BhZ2UoKSAtIHJlbGVhc2UgYSBndXAtcGlubmVkIHBhZ2UKPiAtICogQHBh Z2U6ICAgICAgICAgICAgcG9pbnRlciB0byBwYWdlIHRvIGJlIHJlbGVhc2VkCj4gKy8qCj4gKyAq IEdVUF9QSU5fQ09VTlRJTkdfQklBUywgYW5kIHRoZSBhc3NvY2lhdGVkIGZ1bmN0aW9ucyB0aGF0 IHVzZSBpdCwgb3ZlcmxvYWQKPiArICogdGhlIHBhZ2UncyByZWZjb3VudCBzbyB0aGF0IHR3byBz ZXBhcmF0ZSBpdGVtcyBhcmUgdHJhY2tlZDogdGhlIG9yaWdpbmFsIHBhZ2UKPiArICogcmVmZXJl bmNlIGNvdW50LCBhbmQgYWxzbyBhIG5ldyBjb3VudCBvZiBob3cgbWFueSBnZXRfdXNlcl9wYWdl cygpIGNhbGxzIHdlcmUKCQkJCQkJCV5eIHBpbl91c2VyX3BhZ2VzKCkKCj4gKyAqIG1hZGUgYWdh aW5zdCB0aGUgcGFnZS4gKCJndXAtcGlubmVkIiBpcyBhbm90aGVyIHRlcm0gZm9yIHRoZSBsYXR0 ZXIpLgo+ICsgKgo+ICsgKiBXaXRoIHRoaXMgc2NoZW1lLCBnZXRfdXNlcl9wYWdlcygpIGJlY29t ZXMgc3BlY2lhbDogc3VjaCBwYWdlcyBhcmUgbWFya2VkCgkJCV5eXiBwaW5fdXNlcl9wYWdlcygp Cgo+ICsgKiBhcyBkaXN0aW5jdCBmcm9tIG5vcm1hbCBwYWdlcy4gQXMgc3VjaCwgdGhlIHB1dF91 c2VyX3BhZ2UoKSBjYWxsIChhbmQgaXRzCj4gKyAqIHZhcmlhbnRzKSBtdXN0IGJlIHVzZWQgaW4g b3JkZXIgdG8gcmVsZWFzZSBndXAtcGlubmVkIHBhZ2VzLgo+ICsgKgo+ICsgKiBDaG9pY2Ugb2Yg dmFsdWU6Cj4gICAqCj4gLSAqIFBhZ2VzIHRoYXQgd2VyZSBwaW5uZWQgdmlhIHBpbl91c2VyX3Bh Z2VzKigpIG11c3QgYmUgcmVsZWFzZWQgdmlhIGVpdGhlcgo+IC0gKiBwdXRfdXNlcl9wYWdlKCks IG9yIG9uZSBvZiB0aGUgcHV0X3VzZXJfcGFnZXMqKCkgcm91dGluZXMuIFRoaXMgaXMgc28gdGhh dAo+IC0gKiBldmVudHVhbGx5IHN1Y2ggcGFnZXMgY2FuIGJlIHNlcGFyYXRlbHkgdHJhY2tlZCBh bmQgdW5pcXVlbHkgaGFuZGxlZC4gSW4KPiAtICogcGFydGljdWxhciwgaW50ZXJhY3Rpb25zIHdp dGggUkRNQSBhbmQgZmlsZXN5c3RlbXMgbmVlZCBzcGVjaWFsIGhhbmRsaW5nLgo+ICsgKiBCeSBt YWtpbmcgR1VQX1BJTl9DT1VOVElOR19CSUFTIGEgcG93ZXIgb2YgdHdvLCBkZWJ1Z2dpbmcgb2Yg cGFnZSByZWZlcmVuY2UKPiArICogY291bnRzIHdpdGggcmVzcGVjdCB0byBnZXRfdXNlcl9wYWdl cygpIGFuZCBwdXRfdXNlcl9wYWdlKCkgYmVjb21lcyBzaW1wbGVyLAoJCQkJXl5eIHBpbl91c2Vy X3BhZ2VzKCkKCj4gKyAqIGR1ZSB0byB0aGUgZmFjdCB0aGF0IGFkZGluZyBhbiBldmVuIHBvd2Vy IG9mIHR3byB0byB0aGUgcGFnZSByZWZjb3VudCBoYXMKPiArICogdGhlIGVmZmVjdCBvZiB1c2lu ZyBvbmx5IHRoZSB1cHBlciBOIGJpdHMsIGZvciB0aGUgY29kZSB0aGF0IGNvdW50cyB1cCB1c2lu Zwo+ICsgKiB0aGUgYmlhcyB2YWx1ZS4gVGhpcyBtZWFucyB0aGF0IHRoZSBsb3dlciBiaXRzIGFy ZSBsZWZ0IGZvciB0aGUgZXhjbHVzaXZlCj4gKyAqIHVzZSBvZiB0aGUgb3JpZ2luYWwgY29kZSB0 aGF0IGluY3JlbWVudHMgYW5kIGRlY3JlbWVudHMgYnkgb25lIChvciBhdCBsZWFzdCwKPiArICog YnkgbXVjaCBzbWFsbGVyIHZhbHVlcyB0aGFuIHRoZSBiaWFzIHZhbHVlKS4KPiAgICoKPiAtICog cHV0X3VzZXJfcGFnZSgpIGFuZCBwdXRfcGFnZSgpIGFyZSBub3QgaW50ZXJjaGFuZ2VhYmxlLCBk ZXNwaXRlIHRoaXMgZWFybHkKPiAtICogaW1wbGVtZW50YXRpb24gdGhhdCBtYWtlcyB0aGVtIGxv b2sgdGhlIHNhbWUuIHB1dF91c2VyX3BhZ2UoKSBjYWxscyBtdXN0Cj4gLSAqIGJlIHBlcmZlY3Rs eSBtYXRjaGVkIHVwIHdpdGggcGluKigpIGNhbGxzLgo+ICsgKiBPZiBjb3Vyc2UsIG9uY2UgdGhl IGxvd2VyIGJpdHMgb3ZlcmZsb3cgaW50byB0aGUgdXBwZXIgYml0cyAoYW5kIHRoaXMgaXMKPiAr ICogT0ssIGJlY2F1c2Ugc3VidHJhY3Rpb24gcmVjb3ZlcnMgdGhlIG9yaWdpbmFsIHZhbHVlcyks IHRoZW4gdmlzdWFsIGluc3BlY3Rpb24KPiArICogbm8gbG9uZ2VyIHN1ZmZpY2VzIHRvIGRpcmVj dGx5IHZpZXcgdGhlIHNlcGFyYXRlIGNvdW50cy4gSG93ZXZlciwgZm9yIG5vcm1hbAo+ICsgKiBh cHBsaWNhdGlvbnMgdGhhdCBkb24ndCBoYXZlIGh1Z2UgcGFnZSByZWZlcmVuY2UgY291bnRzLCB0 aGlzIHdvbid0IGJlIGFuCj4gKyAqIGlzc3VlLgo+ICsgKgo+ICsgKiBMb2NraW5nOiB0aGUgbG9j a2xlc3MgYWxnb3JpdGhtIGRlc2NyaWJlZCBpbiBwYWdlX2NhY2hlX2dldF9zcGVjdWxhdGl2ZSgp Cj4gKyAqIGFuZCBwYWdlX2NhY2hlX2d1cF9waW5fc3BlY3VsYXRpdmUoKSBwcm92aWRlcyBzYWZl IG9wZXJhdGlvbiBmb3IKPiArICogZ2V0X3VzZXJfcGFnZXMgYW5kIHBhZ2VfbWtjbGVhbiBhbmQg b3RoZXIgY2FsbHMgdGhhdCByYWNlIHRvIHNldCB1cCBwYWdlCj4gKyAqIHRhYmxlIGVudHJpZXMu Cj4gICAqLwouLi4KPiBAQCAtMjA3MCw5ICsyMTkxLDE2IEBAIHN0YXRpYyBpbnQgZ3VwX2h1Z2Vw dGUocHRlX3QgKnB0ZXAsIHVuc2lnbmVkIGxvbmcgc3osIHVuc2lnbmVkIGxvbmcgYWRkciwKPiAg CXBhZ2UgPSBoZWFkICsgKChhZGRyICYgKHN6LTEpKSA+PiBQQUdFX1NISUZUKTsKPiAgCXJlZnMg PSBfX3JlY29yZF9zdWJwYWdlcyhwYWdlLCBhZGRyLCBlbmQsIHBhZ2VzICsgKm5yKTsKPiAgCj4g LQloZWFkID0gdHJ5X2dldF9jb21wb3VuZF9oZWFkKGhlYWQsIHJlZnMpOwo+IC0JaWYgKCFoZWFk KQo+IC0JCXJldHVybiAwOwo+ICsJaWYgKGZsYWdzICYgRk9MTF9QSU4pIHsKPiArCQloZWFkID0g cGFnZTsKPiArCQlpZiAodW5saWtlbHkoIXVzZXJfcGFnZV9yZWZfaW5jKGhlYWQpKSkKPiArCQkJ cmV0dXJuIDA7Cj4gKwkJaGVhZCA9IHBhZ2U7CgpXaHkgZG8geW91IGFzc2lnbiAnaGVhZCcgdHdp Y2U/IEFsc28gdGhlIHJlZmNvdW50aW5nIGxvZ2ljIGlzIHJlcGVhdGVkCnNldmVyYWwgdGltZXMg c28gcGVyaGFwcyB5b3UgY2FuIGZhY3RvciBpdCBvdXQgaW4gdG8gYSBoZWxwZXIgZnVuY3Rpb24g b3IKZXZlbiBtb3ZlIGl0IHRvIF9fcmVjb3JkX3N1YnBhZ2VzKCk/Cgo+ICsJfSBlbHNlIHsKPiAr CQloZWFkID0gdHJ5X2dldF9jb21wb3VuZF9oZWFkKGhlYWQsIHJlZnMpOwo+ICsJCWlmICghaGVh ZCkKPiArCQkJcmV0dXJuIDA7Cj4gKwl9Cj4gIAo+ICAJaWYgKHVubGlrZWx5KHB0ZV92YWwocHRl KSAhPSBwdGVfdmFsKCpwdGVwKSkpIHsKPiAgCQlwdXRfY29tcG91bmRfaGVhZChoZWFkLCByZWZz KTsKClNvIHRoaXMgd2lsbCBkbyB0aGUgd3JvbmcgdGhpbmcgZm9yIEZPTExfUElOLiBXZSB0b29r IGp1c3Qgb25lICJwaW4iCnJlZmVyZW5jZSB0aGVyZSBidXQgaGVyZSB3ZSdsbCByZWxlYXNlICdy ZWZzJyBub3JtYWwgcmVmZXJlbmNlcyBBRkFJQ1QuCkFsc28gdGhlIGZhY3QgdGhhdCB5b3UgdGFr ZSBqdXN0IG9uZSBwaW4gcmVmZXJlbmNlIGZvciBlYWNoIGh1Z2UgcGFnZQpzdWJzdGFudGlhbGx5 IGNoYW5nZXMgaG93IEdVUCByZWZjb3VudGluZyB3b3JrcyBpbiB0aGUgaHVnZSBwYWdlIGNhc2Uu CkN1cnJlbnRseSwgRk9MTF9HRVQgdXNlcnMgY2FuIGJlIGNvbXBsZXRlbHkgYWdub3N0aWMgb2Yg aHVnZSBwYWdlcy4gU28geW91CmNhbiBlLmcuIEdVUCB3aG9sZSAyIE1CIHBhZ2UsIHN1Ym1pdCBp dCBhcyAyIGRpZmZlcmVudCBiaW9zIGFuZCB0aGVuCmRyb3AgcGFnZSByZWZlcmVuY2VzIGZyb20g ZWFjaCBiaW8gY29tcGxldGlvbiBmdW5jdGlvbi4gV2l0aCB5b3VyIG5ldwpGT0xMX1BJTiBiZWhh dmlvciB5b3UgY2Fubm90IGRvIHRoYXQgYW5kIEkgYmVsaWV2ZSBpdCB3aWxsIGJlIGEgcHJvYmxl bSBmb3IKc29tZSB1c2Vycy4gU28gSSB0aGluayB5b3UgaGF2ZSB0byBtYWludGFpbiB0aGUgYmVo YXZpb3IgdGhhdCB5b3UgaW5jcmVhc2UKdGhlIGhlYWQtPl9yZWZjb3VudCBieSAocmVmcyAqIEdV UF9QSU5fQ09VTlRJTkdfQklBUykgaGVyZS4KCgkJCQkJCQkJSG9uemEKLS0gCkphbiBLYXJhIDxq YWNrQHN1c2UuY29tPgpTVVNFIExhYnMsIENSCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVsQGxpc3Rz LmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xp c3RpbmZvL2RyaS1kZXZlbA==