From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-0.6 required=3.0 tests=DKIM_INVALID,DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4DB4C46477 for ; Tue, 11 Jun 2019 21:20:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 793CF2086A for ; Tue, 11 Jun 2019 21:20:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=vmwopensource.org header.i=@vmwopensource.org header.b="GUhgBcpd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2407623AbfFKVU2 (ORCPT ); Tue, 11 Jun 2019 17:20:28 -0400 Received: from pio-pvt-msa3.bahnhof.se ([79.136.2.42]:59856 "EHLO pio-pvt-msa3.bahnhof.se" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2407612AbfFKVU2 (ORCPT ); Tue, 11 Jun 2019 17:20:28 -0400 Received: from localhost (localhost [127.0.0.1]) by pio-pvt-msa3.bahnhof.se (Postfix) with ESMTP id 482C13F5A0; Tue, 11 Jun 2019 23:20:17 +0200 (CEST) Authentication-Results: pio-pvt-msa3.bahnhof.se; dkim=pass (1024-bit key; unprotected) header.d=vmwopensource.org header.i=@vmwopensource.org header.b=GUhgBcpd; dkim-atps=neutral X-Virus-Scanned: Debian amavisd-new at bahnhof.se Received: from pio-pvt-msa3.bahnhof.se ([127.0.0.1]) by localhost (pio-pvt-msa3.bahnhof.se [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KGMqz9ZSUwae; Tue, 11 Jun 2019 23:20:07 +0200 (CEST) Received: from mail1.shipmail.org (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) (Authenticated sender: mb878879) by pio-pvt-msa3.bahnhof.se (Postfix) with ESMTPA id 924623F534; Tue, 11 Jun 2019 23:20:05 +0200 (CEST) Received: from localhost.localdomain (h-205-35.A357.priv.bahnhof.se [155.4.205.35]) by mail1.shipmail.org (Postfix) with ESMTPSA id 1B349361A96; Tue, 11 Jun 2019 23:20:05 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=vmwopensource.org; s=mail; t=1560288005; bh=V2aF3yRF7Uz8lqdX5QeHaQ5nkyndWzo6SqgyvKNG+to=; h=Subject:To:Cc:References:From:Date:In-Reply-To:From; b=GUhgBcpdyqxFQkx+JvWZ7LQeyQwlIAnuV5Z8+ZvBMTsKOHb+mF3Rdw1w82AzuEVsQ XFWJ7EIeJMIS21RGmfijZao9K8tnZx7R15qWQsueBLeOBwcoalhItNc0BgVQ1Sbtxr 1ViNT6kAbRX0XK5JKaBcVr5K/OCv4jBkUFNhBZsA= Subject: Re: [PATCH v4 3/9] mm: Add write-protect and clean utilities for address space ranges To: Nadav Amit Cc: dri-devel@lists.freedesktop.org, linux-graphics-maintainer@vmware.com, "VMware, Inc." , LKML , Thomas Hellstrom , Andrew Morton , Matthew Wilcox , Will Deacon , Peter Zijlstra , Rik van Riel , Minchan Kim , Michal Hocko , Huang Ying , Souptick Joarder , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Linux-MM , Ralph Campbell , Dave Hansen References: <20190611122454.3075-1-thellstrom@vmwopensource.org> <20190611122454.3075-4-thellstrom@vmwopensource.org> <1CDAE797-4686-4041-938F-DE0456FFF451@gmail.com> <39CC6294-52B5-4ED7-852E-A644132DEA18@gmail.com> From: =?UTF-8?Q?Thomas_Hellstr=c3=b6m_=28VMware=29?= Organization: VMware Inc. Message-ID: Date: Tue, 11 Jun 2019 23:20:04 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <39CC6294-52B5-4ED7-852E-A644132DEA18@gmail.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 6/11/19 9:10 PM, Nadav Amit wrote: >> On Jun 11, 2019, at 11:26 AM, Thomas Hellström (VMware) wrote: >> >> Hi, Nadav, >> >> On 6/11/19 7:21 PM, Nadav Amit wrote: >>>> On Jun 11, 2019, at 5:24 AM, Thomas Hellström (VMware) wrote: >>>> >>>> From: Thomas Hellstrom >>> [ snip ] >>> >>>> +/** >>>> + * apply_pt_wrprotect - Leaf pte callback to write-protect a pte >>>> + * @pte: Pointer to the pte >>>> + * @token: Page table token, see apply_to_pfn_range() >>>> + * @addr: The virtual page address >>>> + * @closure: Pointer to a struct pfn_range_apply embedded in a >>>> + * struct apply_as >>>> + * >>>> + * The function write-protects a pte and records the range in >>>> + * virtual address space of touched ptes for efficient range TLB flushes. >>>> + * >>>> + * Return: Always zero. >>>> + */ >>>> +static int apply_pt_wrprotect(pte_t *pte, pgtable_t token, >>>> + unsigned long addr, >>>> + struct pfn_range_apply *closure) >>>> +{ >>>> + struct apply_as *aas = container_of(closure, typeof(*aas), base); >>>> + pte_t ptent = *pte; >>>> + >>>> + if (pte_write(ptent)) { >>>> + pte_t old_pte = ptep_modify_prot_start(aas->vma, addr, pte); >>>> + >>>> + ptent = pte_wrprotect(old_pte); >>>> + ptep_modify_prot_commit(aas->vma, addr, pte, old_pte, ptent); >>>> + aas->total++; >>>> + aas->start = min(aas->start, addr); >>>> + aas->end = max(aas->end, addr + PAGE_SIZE); >>>> + } >>>> + >>>> + return 0; >>>> +} >>>> + >>>> +/** >>>> + * struct apply_as_clean - Closure structure for apply_as_clean >>>> + * @base: struct apply_as we derive from >>>> + * @bitmap_pgoff: Address_space Page offset of the first bit in @bitmap >>>> + * @bitmap: Bitmap with one bit for each page offset in the address_space range >>>> + * covered. >>>> + * @start: Address_space page offset of first modified pte relative >>>> + * to @bitmap_pgoff >>>> + * @end: Address_space page offset of last modified pte relative >>>> + * to @bitmap_pgoff >>>> + */ >>>> +struct apply_as_clean { >>>> + struct apply_as base; >>>> + pgoff_t bitmap_pgoff; >>>> + unsigned long *bitmap; >>>> + pgoff_t start; >>>> + pgoff_t end; >>>> +}; >>>> + >>>> +/** >>>> + * apply_pt_clean - Leaf pte callback to clean a pte >>>> + * @pte: Pointer to the pte >>>> + * @token: Page table token, see apply_to_pfn_range() >>>> + * @addr: The virtual page address >>>> + * @closure: Pointer to a struct pfn_range_apply embedded in a >>>> + * struct apply_as_clean >>>> + * >>>> + * The function cleans a pte and records the range in >>>> + * virtual address space of touched ptes for efficient TLB flushes. >>>> + * It also records dirty ptes in a bitmap representing page offsets >>>> + * in the address_space, as well as the first and last of the bits >>>> + * touched. >>>> + * >>>> + * Return: Always zero. >>>> + */ >>>> +static int apply_pt_clean(pte_t *pte, pgtable_t token, >>>> + unsigned long addr, >>>> + struct pfn_range_apply *closure) >>>> +{ >>>> + struct apply_as *aas = container_of(closure, typeof(*aas), base); >>>> + struct apply_as_clean *clean = container_of(aas, typeof(*clean), base); >>>> + pte_t ptent = *pte; >>>> + >>>> + if (pte_dirty(ptent)) { >>>> + pgoff_t pgoff = ((addr - aas->vma->vm_start) >> PAGE_SHIFT) + >>>> + aas->vma->vm_pgoff - clean->bitmap_pgoff; >>>> + pte_t old_pte = ptep_modify_prot_start(aas->vma, addr, pte); >>>> + >>>> + ptent = pte_mkclean(old_pte); >>>> + ptep_modify_prot_commit(aas->vma, addr, pte, old_pte, ptent); >>>> + >>>> + aas->total++; >>>> + aas->start = min(aas->start, addr); >>>> + aas->end = max(aas->end, addr + PAGE_SIZE); >>>> + >>>> + __set_bit(pgoff, clean->bitmap); >>>> + clean->start = min(clean->start, pgoff); >>>> + clean->end = max(clean->end, pgoff + 1); >>>> + } >>>> + >>>> + return 0; >>> Usually, when a PTE is write-protected, or when a dirty-bit is cleared, the >>> TLB flush must be done while the page-table lock for that specific table is >>> taken (i.e., within apply_pt_clean() and apply_pt_wrprotect() in this case). >>> >>> Otherwise, in the case of apply_pt_clean() for example, another core might >>> shortly after (before the TLB flush) write to the same page whose PTE was >>> changed. The dirty-bit in such case might not be set, and the change get >>> lost. >> Hmm. Let's assume that was the case, we have two possible situations: >> >> A: pt_clean >> >> 1. That core's TLB entry is invalid. It will set the PTE dirty bit and continue. The dirty bit will probably remain set after the TLB flush. > I guess you mean the PTE is not cached in the TLB. Yes. > >> 2. That core's TLB entry is valid. It will just continue. The dirty bit will remain clear after the TLB flush. >> >> But I fail to see how having the TLB flush within the page table lock would help in this case. Since the writing core will never attempt to take it? In any case, if such a race occurs, the corresponding bit in the bitmap would have been set and we've recorded that the page is dirty. > I don’t understand. What do you mean “recorded that the page is dirty”? > IIUC, the PTE is clear in this case - you mean PG_dirty is set? All PTEs we touch and clean are noted in the bitmap. > > To clarify, this code actually may work correctly on Intel CPUs, based on a > recent discussion with Dave Hansen. Apparently, most Intel CPUs set the > dirty bit in memory atomically when a page is first written. > > But this is a generic code and not arch-specific. My concern is that a > certain page might be written to, but would not be marked as dirty in either > the bitmap or the PTE. Regardless of arch, we have four cases: 1. Writes occuring before we scan (and possibly modify) the PTE. Should be handled correctly. 2. Writes occurning after the TLB flush. Should be handled correctly, unless after a TLB flush the TLB cached entry and the PTE differs on the dirty bit. Then we could in theory go on writing without marking the PTE dirty. But that would probably be an arch code bug: I mean anything using tlb_gather_mmu() would flush TLB outside of the page table lock, and if, for example, unmap_mapping_range() left the TLB entries and the PTES in an inconsistent state, that wouldn't be good. 3. Writes occuring after the PTE scan, but before the TLB flush without us modifying the PTE: That would be the same as a spurious TLB flush. It should be harmless. The write will not be picked up in the bitmap, but the PTE dirty bit will be set. 4. Writes occuring after us clearing the dirty bit and before the TLB flush: We will detect the write, since the bitmap bit is already set. If the write continues after the TLB flush, we go to 2. Note, for archs doing software PTE_DIRTY, that would be very similar to softdirty, which is also doing batched TLB flushes... > > The practice of flushing cleaned/write-protected PTEs while hold the > page-table lock related (sorry for my confusion). > >> B: wrprotect situation, the situation is a bit different: >> >> 1. That core's TLB entry is invalid. It will read the PTE, cause a fault and block in mkwrite() on an external address space lock which is held over this operation. (Is it this situation that is your main concern?) >> 2. That core's TLB entry is valid. It will just continue regardless of any locks. >> >> For both mkwrite() and dirty() if we act on the recorded pages *after* the TLB flush, we're OK. The difference is that just after the TLB flush there should be no write-enabled PTEs in the write-protect case, but there may be dirty PTEs in the pt_clean case. Something that is mentioned in the docs already. > The wrprotect might work correctly, I guess. It does work to mprotect() > (again, sorry for confusing). > >>> Does this function regards a certain use-case in which deferring the TLB >>> flushes is fine? If so, assertions and documentation of the related >>> assumption would be useful. >> If I understand your comment correctly, the page table lock is sometimes used as the lock in B1, blocking a possible software fault until the TLB flush has happened. Here we assume an external address space lock taken both around the wrprotect operation and in mkwrite(). Would it be OK if I add comments about the necessity of an external lock to the doc? Ok with a follow-up patch? > I think the patch should explain itself. I think the comment: > >> + * WARNING: This function should only be used for address spaces that >> + * completely own the pages / memory the page table points to. Typically a >> + * device file. > ... should be more concrete (define address spaces that completely own > memory), and possibly backed by an (debug) assertion to ensure that it is > only used correctly. Agreed. To clarify we should only run this on VM_IO vmas without (trans)huge pages, for which there are already checks. I'll update the doc. /Thomas From mboxrd@z Thu Jan 1 00:00:00 1970 From: =?UTF-8?Q?Thomas_Hellstr=c3=b6m_=28VMware=29?= Subject: Re: [PATCH v4 3/9] mm: Add write-protect and clean utilities for address space ranges Date: Tue, 11 Jun 2019 23:20:04 +0200 Message-ID: References: <20190611122454.3075-1-thellstrom@vmwopensource.org> <20190611122454.3075-4-thellstrom@vmwopensource.org> <1CDAE797-4686-4041-938F-DE0456FFF451@gmail.com> <39CC6294-52B5-4ED7-852E-A644132DEA18@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8"; Format="flowed" Content-Transfer-Encoding: base64 Return-path: Received: from pio-pvt-msa3.bahnhof.se (pio-pvt-msa3.bahnhof.se [79.136.2.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7C05D891E2 for ; Tue, 11 Jun 2019 21:20:26 +0000 (UTC) In-Reply-To: <39CC6294-52B5-4ED7-852E-A644132DEA18@gmail.com> Content-Language: en-US List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To: Nadav Amit Cc: Dave Hansen , Thomas Hellstrom , Michal Hocko , Rik van Riel , "VMware, Inc." , Minchan Kim , Will Deacon , LKML , dri-devel@lists.freedesktop.org, Linux-MM , Peter Zijlstra , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , linux-graphics-maintainer@vmware.com, Matthew Wilcox , Huang Ying , Andrew Morton , Ralph Campbell , Souptick Joarder List-Id: dri-devel@lists.freedesktop.org T24gNi8xMS8xOSA5OjEwIFBNLCBOYWRhdiBBbWl0IHdyb3RlOgo+PiBPbiBKdW4gMTEsIDIwMTks IGF0IDExOjI2IEFNLCBUaG9tYXMgSGVsbHN0csO2bSAoVk13YXJlKSA8dGhlbGxzdHJvbUB2bXdv cGVuc291cmNlLm9yZz4gd3JvdGU6Cj4+Cj4+IEhpLCBOYWRhdiwKPj4KPj4gT24gNi8xMS8xOSA3 OjIxIFBNLCBOYWRhdiBBbWl0IHdyb3RlOgo+Pj4+IE9uIEp1biAxMSwgMjAxOSwgYXQgNToyNCBB TSwgVGhvbWFzIEhlbGxzdHLDtm0gKFZNd2FyZSkgPHRoZWxsc3Ryb21Adm13b3BlbnNvdXJjZS5v cmc+IHdyb3RlOgo+Pj4+Cj4+Pj4gRnJvbTogVGhvbWFzIEhlbGxzdHJvbSA8dGhlbGxzdHJvbUB2 bXdhcmUuY29tPgo+Pj4gWyBzbmlwIF0KPj4+Cj4+Pj4gKy8qKgo+Pj4+ICsgKiBhcHBseV9wdF93 cnByb3RlY3QgLSBMZWFmIHB0ZSBjYWxsYmFjayB0byB3cml0ZS1wcm90ZWN0IGEgcHRlCj4+Pj4g KyAqIEBwdGU6IFBvaW50ZXIgdG8gdGhlIHB0ZQo+Pj4+ICsgKiBAdG9rZW46IFBhZ2UgdGFibGUg dG9rZW4sIHNlZSBhcHBseV90b19wZm5fcmFuZ2UoKQo+Pj4+ICsgKiBAYWRkcjogVGhlIHZpcnR1 YWwgcGFnZSBhZGRyZXNzCj4+Pj4gKyAqIEBjbG9zdXJlOiBQb2ludGVyIHRvIGEgc3RydWN0IHBm bl9yYW5nZV9hcHBseSBlbWJlZGRlZCBpbiBhCj4+Pj4gKyAqIHN0cnVjdCBhcHBseV9hcwo+Pj4+ ICsgKgo+Pj4+ICsgKiBUaGUgZnVuY3Rpb24gd3JpdGUtcHJvdGVjdHMgYSBwdGUgYW5kIHJlY29y ZHMgdGhlIHJhbmdlIGluCj4+Pj4gKyAqIHZpcnR1YWwgYWRkcmVzcyBzcGFjZSBvZiB0b3VjaGVk IHB0ZXMgZm9yIGVmZmljaWVudCByYW5nZSBUTEIgZmx1c2hlcy4KPj4+PiArICoKPj4+PiArICog UmV0dXJuOiBBbHdheXMgemVyby4KPj4+PiArICovCj4+Pj4gK3N0YXRpYyBpbnQgYXBwbHlfcHRf d3Jwcm90ZWN0KHB0ZV90ICpwdGUsIHBndGFibGVfdCB0b2tlbiwKPj4+PiArCQkJICAgICAgdW5z aWduZWQgbG9uZyBhZGRyLAo+Pj4+ICsJCQkgICAgICBzdHJ1Y3QgcGZuX3JhbmdlX2FwcGx5ICpj bG9zdXJlKQo+Pj4+ICt7Cj4+Pj4gKwlzdHJ1Y3QgYXBwbHlfYXMgKmFhcyA9IGNvbnRhaW5lcl9v ZihjbG9zdXJlLCB0eXBlb2YoKmFhcyksIGJhc2UpOwo+Pj4+ICsJcHRlX3QgcHRlbnQgPSAqcHRl Owo+Pj4+ICsKPj4+PiArCWlmIChwdGVfd3JpdGUocHRlbnQpKSB7Cj4+Pj4gKwkJcHRlX3Qgb2xk X3B0ZSA9IHB0ZXBfbW9kaWZ5X3Byb3Rfc3RhcnQoYWFzLT52bWEsIGFkZHIsIHB0ZSk7Cj4+Pj4g Kwo+Pj4+ICsJCXB0ZW50ID0gcHRlX3dycHJvdGVjdChvbGRfcHRlKTsKPj4+PiArCQlwdGVwX21v ZGlmeV9wcm90X2NvbW1pdChhYXMtPnZtYSwgYWRkciwgcHRlLCBvbGRfcHRlLCBwdGVudCk7Cj4+ Pj4gKwkJYWFzLT50b3RhbCsrOwo+Pj4+ICsJCWFhcy0+c3RhcnQgPSBtaW4oYWFzLT5zdGFydCwg YWRkcik7Cj4+Pj4gKwkJYWFzLT5lbmQgPSBtYXgoYWFzLT5lbmQsIGFkZHIgKyBQQUdFX1NJWkUp Owo+Pj4+ICsJfQo+Pj4+ICsKPj4+PiArCXJldHVybiAwOwo+Pj4+ICt9Cj4+Pj4gKwo+Pj4+ICsv KioKPj4+PiArICogc3RydWN0IGFwcGx5X2FzX2NsZWFuIC0gQ2xvc3VyZSBzdHJ1Y3R1cmUgZm9y IGFwcGx5X2FzX2NsZWFuCj4+Pj4gKyAqIEBiYXNlOiBzdHJ1Y3QgYXBwbHlfYXMgd2UgZGVyaXZl IGZyb20KPj4+PiArICogQGJpdG1hcF9wZ29mZjogQWRkcmVzc19zcGFjZSBQYWdlIG9mZnNldCBv ZiB0aGUgZmlyc3QgYml0IGluIEBiaXRtYXAKPj4+PiArICogQGJpdG1hcDogQml0bWFwIHdpdGgg b25lIGJpdCBmb3IgZWFjaCBwYWdlIG9mZnNldCBpbiB0aGUgYWRkcmVzc19zcGFjZSByYW5nZQo+ Pj4+ICsgKiBjb3ZlcmVkLgo+Pj4+ICsgKiBAc3RhcnQ6IEFkZHJlc3Nfc3BhY2UgcGFnZSBvZmZz ZXQgb2YgZmlyc3QgbW9kaWZpZWQgcHRlIHJlbGF0aXZlCj4+Pj4gKyAqIHRvIEBiaXRtYXBfcGdv ZmYKPj4+PiArICogQGVuZDogQWRkcmVzc19zcGFjZSBwYWdlIG9mZnNldCBvZiBsYXN0IG1vZGlm aWVkIHB0ZSByZWxhdGl2ZQo+Pj4+ICsgKiB0byBAYml0bWFwX3Bnb2ZmCj4+Pj4gKyAqLwo+Pj4+ ICtzdHJ1Y3QgYXBwbHlfYXNfY2xlYW4gewo+Pj4+ICsJc3RydWN0IGFwcGx5X2FzIGJhc2U7Cj4+ Pj4gKwlwZ29mZl90IGJpdG1hcF9wZ29mZjsKPj4+PiArCXVuc2lnbmVkIGxvbmcgKmJpdG1hcDsK Pj4+PiArCXBnb2ZmX3Qgc3RhcnQ7Cj4+Pj4gKwlwZ29mZl90IGVuZDsKPj4+PiArfTsKPj4+PiAr Cj4+Pj4gKy8qKgo+Pj4+ICsgKiBhcHBseV9wdF9jbGVhbiAtIExlYWYgcHRlIGNhbGxiYWNrIHRv IGNsZWFuIGEgcHRlCj4+Pj4gKyAqIEBwdGU6IFBvaW50ZXIgdG8gdGhlIHB0ZQo+Pj4+ICsgKiBA dG9rZW46IFBhZ2UgdGFibGUgdG9rZW4sIHNlZSBhcHBseV90b19wZm5fcmFuZ2UoKQo+Pj4+ICsg KiBAYWRkcjogVGhlIHZpcnR1YWwgcGFnZSBhZGRyZXNzCj4+Pj4gKyAqIEBjbG9zdXJlOiBQb2lu dGVyIHRvIGEgc3RydWN0IHBmbl9yYW5nZV9hcHBseSBlbWJlZGRlZCBpbiBhCj4+Pj4gKyAqIHN0 cnVjdCBhcHBseV9hc19jbGVhbgo+Pj4+ICsgKgo+Pj4+ICsgKiBUaGUgZnVuY3Rpb24gY2xlYW5z IGEgcHRlIGFuZCByZWNvcmRzIHRoZSByYW5nZSBpbgo+Pj4+ICsgKiB2aXJ0dWFsIGFkZHJlc3Mg c3BhY2Ugb2YgdG91Y2hlZCBwdGVzIGZvciBlZmZpY2llbnQgVExCIGZsdXNoZXMuCj4+Pj4gKyAq IEl0IGFsc28gcmVjb3JkcyBkaXJ0eSBwdGVzIGluIGEgYml0bWFwIHJlcHJlc2VudGluZyBwYWdl IG9mZnNldHMKPj4+PiArICogaW4gdGhlIGFkZHJlc3Nfc3BhY2UsIGFzIHdlbGwgYXMgdGhlIGZp cnN0IGFuZCBsYXN0IG9mIHRoZSBiaXRzCj4+Pj4gKyAqIHRvdWNoZWQuCj4+Pj4gKyAqCj4+Pj4g KyAqIFJldHVybjogQWx3YXlzIHplcm8uCj4+Pj4gKyAqLwo+Pj4+ICtzdGF0aWMgaW50IGFwcGx5 X3B0X2NsZWFuKHB0ZV90ICpwdGUsIHBndGFibGVfdCB0b2tlbiwKPj4+PiArCQkJICB1bnNpZ25l ZCBsb25nIGFkZHIsCj4+Pj4gKwkJCSAgc3RydWN0IHBmbl9yYW5nZV9hcHBseSAqY2xvc3VyZSkK Pj4+PiArewo+Pj4+ICsJc3RydWN0IGFwcGx5X2FzICphYXMgPSBjb250YWluZXJfb2YoY2xvc3Vy ZSwgdHlwZW9mKCphYXMpLCBiYXNlKTsKPj4+PiArCXN0cnVjdCBhcHBseV9hc19jbGVhbiAqY2xl YW4gPSBjb250YWluZXJfb2YoYWFzLCB0eXBlb2YoKmNsZWFuKSwgYmFzZSk7Cj4+Pj4gKwlwdGVf dCBwdGVudCA9ICpwdGU7Cj4+Pj4gKwo+Pj4+ICsJaWYgKHB0ZV9kaXJ0eShwdGVudCkpIHsKPj4+ PiArCQlwZ29mZl90IHBnb2ZmID0gKChhZGRyIC0gYWFzLT52bWEtPnZtX3N0YXJ0KSA+PiBQQUdF X1NISUZUKSArCj4+Pj4gKwkJCWFhcy0+dm1hLT52bV9wZ29mZiAtIGNsZWFuLT5iaXRtYXBfcGdv ZmY7Cj4+Pj4gKwkJcHRlX3Qgb2xkX3B0ZSA9IHB0ZXBfbW9kaWZ5X3Byb3Rfc3RhcnQoYWFzLT52 bWEsIGFkZHIsIHB0ZSk7Cj4+Pj4gKwo+Pj4+ICsJCXB0ZW50ID0gcHRlX21rY2xlYW4ob2xkX3B0 ZSk7Cj4+Pj4gKwkJcHRlcF9tb2RpZnlfcHJvdF9jb21taXQoYWFzLT52bWEsIGFkZHIsIHB0ZSwg b2xkX3B0ZSwgcHRlbnQpOwo+Pj4+ICsKPj4+PiArCQlhYXMtPnRvdGFsKys7Cj4+Pj4gKwkJYWFz LT5zdGFydCA9IG1pbihhYXMtPnN0YXJ0LCBhZGRyKTsKPj4+PiArCQlhYXMtPmVuZCA9IG1heChh YXMtPmVuZCwgYWRkciArIFBBR0VfU0laRSk7Cj4+Pj4gKwo+Pj4+ICsJCV9fc2V0X2JpdChwZ29m ZiwgY2xlYW4tPmJpdG1hcCk7Cj4+Pj4gKwkJY2xlYW4tPnN0YXJ0ID0gbWluKGNsZWFuLT5zdGFy dCwgcGdvZmYpOwo+Pj4+ICsJCWNsZWFuLT5lbmQgPSBtYXgoY2xlYW4tPmVuZCwgcGdvZmYgKyAx KTsKPj4+PiArCX0KPj4+PiArCj4+Pj4gKwlyZXR1cm4gMDsKPj4+IFVzdWFsbHksIHdoZW4gYSBQ VEUgaXMgd3JpdGUtcHJvdGVjdGVkLCBvciB3aGVuIGEgZGlydHktYml0IGlzIGNsZWFyZWQsIHRo ZQo+Pj4gVExCIGZsdXNoIG11c3QgYmUgZG9uZSB3aGlsZSB0aGUgcGFnZS10YWJsZSBsb2NrIGZv ciB0aGF0IHNwZWNpZmljIHRhYmxlIGlzCj4+PiB0YWtlbiAoaS5lLiwgd2l0aGluIGFwcGx5X3B0 X2NsZWFuKCkgYW5kIGFwcGx5X3B0X3dycHJvdGVjdCgpIGluIHRoaXMgY2FzZSkuCj4+Pgo+Pj4g T3RoZXJ3aXNlLCBpbiB0aGUgY2FzZSBvZiBhcHBseV9wdF9jbGVhbigpIGZvciBleGFtcGxlLCBh bm90aGVyIGNvcmUgbWlnaHQKPj4+IHNob3J0bHkgYWZ0ZXIgKGJlZm9yZSB0aGUgVExCIGZsdXNo KSB3cml0ZSB0byB0aGUgc2FtZSBwYWdlIHdob3NlIFBURSB3YXMKPj4+IGNoYW5nZWQuIFRoZSBk aXJ0eS1iaXQgaW4gc3VjaCBjYXNlIG1pZ2h0IG5vdCBiZSBzZXQsIGFuZCB0aGUgY2hhbmdlIGdl dAo+Pj4gbG9zdC4KPj4gSG1tLiBMZXQncyBhc3N1bWUgdGhhdCB3YXMgdGhlIGNhc2UsIHdlIGhh dmUgdHdvIHBvc3NpYmxlIHNpdHVhdGlvbnM6Cj4+Cj4+IEE6IHB0X2NsZWFuCj4+Cj4+IDEuIFRo YXQgY29yZSdzIFRMQiBlbnRyeSBpcyBpbnZhbGlkLiBJdCB3aWxsIHNldCB0aGUgUFRFIGRpcnR5 IGJpdCBhbmQgY29udGludWUuIFRoZSBkaXJ0eSBiaXQgd2lsbCBwcm9iYWJseSByZW1haW4gc2V0 IGFmdGVyIHRoZSBUTEIgZmx1c2guCj4gSSBndWVzcyB5b3UgbWVhbiB0aGUgUFRFIGlzIG5vdCBj YWNoZWQgaW4gdGhlIFRMQi4KWWVzLgo+Cj4+IDIuIFRoYXQgY29yZSdzIFRMQiBlbnRyeSBpcyB2 YWxpZC4gSXQgd2lsbCBqdXN0IGNvbnRpbnVlLiBUaGUgZGlydHkgYml0IHdpbGwgcmVtYWluIGNs ZWFyIGFmdGVyIHRoZSBUTEIgZmx1c2guCj4+Cj4+IEJ1dCBJIGZhaWwgdG8gc2VlIGhvdyBoYXZp bmcgdGhlIFRMQiBmbHVzaCB3aXRoaW4gdGhlIHBhZ2UgdGFibGUgbG9jayB3b3VsZCBoZWxwIGlu IHRoaXMgY2FzZS4gU2luY2UgdGhlIHdyaXRpbmcgY29yZSB3aWxsIG5ldmVyIGF0dGVtcHQgdG8g dGFrZSBpdD8gSW4gYW55IGNhc2UsIGlmIHN1Y2ggYSByYWNlIG9jY3VycywgdGhlIGNvcnJlc3Bv bmRpbmcgYml0IGluIHRoZSBiaXRtYXAgd291bGQgaGF2ZSBiZWVuIHNldCBhbmQgd2UndmUgcmVj b3JkZWQgdGhhdCB0aGUgcGFnZSBpcyBkaXJ0eS4KPiBJIGRvbuKAmXQgdW5kZXJzdGFuZC4gV2hh dCBkbyB5b3UgbWVhbiDigJxyZWNvcmRlZCB0aGF0IHRoZSBwYWdlIGlzIGRpcnR54oCdPwo+IElJ VUMsIHRoZSBQVEUgaXMgY2xlYXIgaW4gdGhpcyBjYXNlIC0geW91IG1lYW4gUEdfZGlydHkgaXMg c2V0PwoKQWxsIFBURXMgd2UgdG91Y2ggYW5kIGNsZWFuIGFyZSBub3RlZCBpbiB0aGUgYml0bWFw LgoKPgo+IFRvIGNsYXJpZnksIHRoaXMgY29kZSBhY3R1YWxseSBtYXkgd29yayBjb3JyZWN0bHkg b24gSW50ZWwgQ1BVcywgYmFzZWQgb24gYQo+IHJlY2VudCBkaXNjdXNzaW9uIHdpdGggRGF2ZSBI YW5zZW4uIEFwcGFyZW50bHksIG1vc3QgSW50ZWwgQ1BVcyBzZXQgdGhlCj4gZGlydHkgYml0IGlu IG1lbW9yeSBhdG9taWNhbGx5IHdoZW4gYSBwYWdlIGlzIGZpcnN0IHdyaXR0ZW4uCj4KPiBCdXQg dGhpcyBpcyBhIGdlbmVyaWMgY29kZSBhbmQgbm90IGFyY2gtc3BlY2lmaWMuIE15IGNvbmNlcm4g aXMgdGhhdCBhCj4gY2VydGFpbiBwYWdlIG1pZ2h0IGJlIHdyaXR0ZW4gdG8sIGJ1dCB3b3VsZCBu b3QgYmUgbWFya2VkIGFzIGRpcnR5IGluIGVpdGhlcgo+IHRoZSBiaXRtYXAgb3IgdGhlIFBURS4K ClJlZ2FyZGxlc3Mgb2YgYXJjaCwgd2UgaGF2ZSBmb3VyIGNhc2VzOgoxLiBXcml0ZXMgb2NjdXJp bmcgYmVmb3JlIHdlIHNjYW4gKGFuZCBwb3NzaWJseSBtb2RpZnkpIHRoZSBQVEUuIFNob3VsZCAK YmUgaGFuZGxlZCBjb3JyZWN0bHkuCjIuIFdyaXRlcyBvY2N1cm5pbmcgYWZ0ZXIgdGhlIFRMQiBm bHVzaC4gU2hvdWxkIGJlIGhhbmRsZWQgY29ycmVjdGx5LCAKdW5sZXNzIGFmdGVyIGEgVExCIGZs dXNoIHRoZSBUTEIgY2FjaGVkIGVudHJ5IGFuZCB0aGUgUFRFIGRpZmZlcnMgb24gdGhlIApkaXJ0 eSBiaXQuIFRoZW4gd2UgY291bGQgaW4gdGhlb3J5IGdvIG9uIHdyaXRpbmcgd2l0aG91dCBtYXJr aW5nIHRoZSBQVEUgCmRpcnR5LiBCdXQgdGhhdCB3b3VsZCBwcm9iYWJseSBiZSBhbiBhcmNoIGNv ZGUgYnVnOiBJIG1lYW4gYW55dGhpbmcgCnVzaW5nIHRsYl9nYXRoZXJfbW11KCkgd291bGQgZmx1 c2ggVExCIG91dHNpZGUgb2YgdGhlIHBhZ2UgdGFibGUgbG9jaywgCmFuZCBpZiwgZm9yIGV4YW1w bGUsIHVubWFwX21hcHBpbmdfcmFuZ2UoKSBsZWZ0IHRoZSBUTEIgZW50cmllcyBhbmQgdGhlIApQ VEVTIGluIGFuIGluY29uc2lzdGVudCBzdGF0ZSwgdGhhdCB3b3VsZG4ndCBiZSBnb29kLgozLiBX cml0ZXMgb2NjdXJpbmcgYWZ0ZXIgdGhlIFBURSBzY2FuLCBidXQgYmVmb3JlIHRoZSBUTEIgZmx1 c2ggd2l0aG91dCAKdXMgbW9kaWZ5aW5nIHRoZSBQVEU6IFRoYXQgd291bGQgYmUgdGhlIHNhbWUg YXMgYSBzcHVyaW91cyBUTEIgZmx1c2guIEl0IApzaG91bGQgYmUgaGFybWxlc3MuIFRoZSB3cml0 ZSB3aWxsIG5vdCBiZSBwaWNrZWQgdXAgaW4gdGhlIGJpdG1hcCwgYnV0IAp0aGUgUFRFIGRpcnR5 IGJpdCB3aWxsIGJlIHNldC4KNC4gV3JpdGVzIG9jY3VyaW5nIGFmdGVyIHVzIGNsZWFyaW5nIHRo ZSBkaXJ0eSBiaXQgYW5kIGJlZm9yZSB0aGUgVExCIApmbHVzaDogV2Ugd2lsbCBkZXRlY3QgdGhl IHdyaXRlLCBzaW5jZSB0aGUgYml0bWFwIGJpdCBpcyBhbHJlYWR5IHNldC4gSWYgCnRoZSB3cml0 ZSBjb250aW51ZXMgYWZ0ZXIgdGhlIFRMQiBmbHVzaCwgd2UgZ28gdG8gMi4KCk5vdGUsIGZvciBh cmNocyBkb2luZyBzb2Z0d2FyZSBQVEVfRElSVFksIHRoYXQgd291bGQgYmUgdmVyeSBzaW1pbGFy IHRvIApzb2Z0ZGlydHksIHdoaWNoIGlzIGFsc28gZG9pbmcgYmF0Y2hlZCBUTEIgZmx1c2hlcy4u LgoKPgo+IFRoZSBwcmFjdGljZSBvZiBmbHVzaGluZyBjbGVhbmVkL3dyaXRlLXByb3RlY3RlZCBQ VEVzIHdoaWxlIGhvbGQgdGhlCj4gcGFnZS10YWJsZSBsb2NrIHJlbGF0ZWQgKHNvcnJ5IGZvciBt eSBjb25mdXNpb24pLgo+Cj4+IEI6IHdycHJvdGVjdCBzaXR1YXRpb24sIHRoZSBzaXR1YXRpb24g aXMgYSBiaXQgZGlmZmVyZW50Ogo+Pgo+PiAxLiBUaGF0IGNvcmUncyBUTEIgZW50cnkgaXMgaW52 YWxpZC4gSXQgd2lsbCByZWFkIHRoZSBQVEUsIGNhdXNlIGEgZmF1bHQgYW5kIGJsb2NrIGluIG1r d3JpdGUoKSBvbiBhbiBleHRlcm5hbCBhZGRyZXNzIHNwYWNlIGxvY2sgd2hpY2ggaXMgaGVsZCBv dmVyIHRoaXMgb3BlcmF0aW9uLiAoSXMgaXQgdGhpcyBzaXR1YXRpb24gdGhhdCBpcyB5b3VyIG1h aW4gY29uY2Vybj8pCj4+IDIuIFRoYXQgY29yZSdzIFRMQiBlbnRyeSBpcyB2YWxpZC4gSXQgd2ls bCBqdXN0IGNvbnRpbnVlIHJlZ2FyZGxlc3Mgb2YgYW55IGxvY2tzLgo+Pgo+PiBGb3IgYm90aCBt a3dyaXRlKCkgYW5kIGRpcnR5KCkgaWYgd2UgYWN0IG9uIHRoZSByZWNvcmRlZCBwYWdlcyAqYWZ0 ZXIqIHRoZSBUTEIgZmx1c2gsIHdlJ3JlIE9LLiBUaGUgZGlmZmVyZW5jZSBpcyB0aGF0IGp1c3Qg YWZ0ZXIgdGhlIFRMQiBmbHVzaCB0aGVyZSBzaG91bGQgYmUgbm8gd3JpdGUtZW5hYmxlZCBQVEVz IGluIHRoZSB3cml0ZS1wcm90ZWN0IGNhc2UsIGJ1dCB0aGVyZSBtYXkgYmUgZGlydHkgUFRFcyBp biB0aGUgcHRfY2xlYW4gY2FzZS4gU29tZXRoaW5nIHRoYXQgaXMgbWVudGlvbmVkIGluIHRoZSBk b2NzIGFscmVhZHkuCj4gVGhlIHdycHJvdGVjdCBtaWdodCB3b3JrIGNvcnJlY3RseSwgSSBndWVz cy4gSXQgZG9lcyB3b3JrIHRvIG1wcm90ZWN0KCkKPiAoYWdhaW4sIHNvcnJ5IGZvciBjb25mdXNp bmcpLgo+Cj4+PiBEb2VzIHRoaXMgZnVuY3Rpb24gcmVnYXJkcyBhIGNlcnRhaW4gdXNlLWNhc2Ug aW4gd2hpY2ggZGVmZXJyaW5nIHRoZSBUTEIKPj4+IGZsdXNoZXMgaXMgZmluZT8gSWYgc28sIGFz c2VydGlvbnMgYW5kIGRvY3VtZW50YXRpb24gb2YgdGhlIHJlbGF0ZWQKPj4+IGFzc3VtcHRpb24g d291bGQgYmUgdXNlZnVsLgo+PiBJZiBJIHVuZGVyc3RhbmQgeW91ciBjb21tZW50IGNvcnJlY3Rs eSwgdGhlIHBhZ2UgdGFibGUgbG9jayBpcyBzb21ldGltZXMgdXNlZCBhcyB0aGUgbG9jayBpbiBC MSwgYmxvY2tpbmcgYSBwb3NzaWJsZSBzb2Z0d2FyZSBmYXVsdCB1bnRpbCB0aGUgVExCIGZsdXNo IGhhcyBoYXBwZW5lZC4gIEhlcmUgd2UgYXNzdW1lIGFuIGV4dGVybmFsIGFkZHJlc3Mgc3BhY2Ug bG9jayB0YWtlbiBib3RoIGFyb3VuZCB0aGUgd3Jwcm90ZWN0IG9wZXJhdGlvbiBhbmQgaW4gbWt3 cml0ZSgpLiBXb3VsZCBpdCBiZSBPSyBpZiBJIGFkZCBjb21tZW50cyBhYm91dCB0aGUgbmVjZXNz aXR5IG9mIGFuIGV4dGVybmFsIGxvY2sgdG8gdGhlIGRvYz8gT2sgd2l0aCBhIGZvbGxvdy11cCBw YXRjaD8KPiBJIHRoaW5rIHRoZSBwYXRjaCBzaG91bGQgZXhwbGFpbiBpdHNlbGYuIEkgdGhpbmsg dGhlIGNvbW1lbnQ6Cj4KPj4gKyAqIFdBUk5JTkc6IFRoaXMgZnVuY3Rpb24gc2hvdWxkIG9ubHkg YmUgdXNlZCBmb3IgYWRkcmVzcyBzcGFjZXMgdGhhdAo+PiArICogY29tcGxldGVseSBvd24gdGhl IHBhZ2VzIC8gbWVtb3J5IHRoZSBwYWdlIHRhYmxlIHBvaW50cyB0by4gVHlwaWNhbGx5IGEKPj4g KyAqIGRldmljZSBmaWxlLgo+IC4uLiBzaG91bGQgYmUgbW9yZSBjb25jcmV0ZSAoZGVmaW5lIGFk ZHJlc3Mgc3BhY2VzIHRoYXQgY29tcGxldGVseSBvd24KPiBtZW1vcnkpLCBhbmQgcG9zc2libHkg YmFja2VkIGJ5IGFuIChkZWJ1ZykgYXNzZXJ0aW9uIHRvIGVuc3VyZSB0aGF0IGl0IGlzCj4gb25s eSB1c2VkIGNvcnJlY3RseS4KCkFncmVlZC4gVG8gY2xhcmlmeSB3ZSBzaG91bGQgb25seSBydW4g dGhpcyBvbiBWTV9JTyB2bWFzIHdpdGhvdXQgCih0cmFucylodWdlIHBhZ2VzLCBmb3Igd2hpY2gg dGhlcmUgYXJlIGFscmVhZHkgY2hlY2tzLiBJJ2xsIHVwZGF0ZSB0aGUgZG9jLgoKL1Rob21hcwoK CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpkcmktZGV2 ZWwgbWFpbGluZyBsaXN0CmRyaS1kZXZlbEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9s aXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9kcmktZGV2ZWw=