From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA591C433DB for ; Sun, 27 Dec 2020 20:33:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A9FF8221F5 for ; Sun, 27 Dec 2020 20:33:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726198AbgL0UdY (ORCPT ); Sun, 27 Dec 2020 15:33:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726067AbgL0UdX (ORCPT ); Sun, 27 Dec 2020 15:33:23 -0500 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2AAA9C061794 for ; Sun, 27 Dec 2020 12:32:43 -0800 (PST) Received: by mail-ej1-x629.google.com with SMTP id g20so11992872ejb.1 for ; Sun, 27 Dec 2020 12:32:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tometzki.de; s=google; h=date:from:to:cc:subject:message-id:reply-to:mail-followup-to :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=SLf/bQHuPIk4kCUSXjBvZosjTCtFoLnAczMk9hzDGw0=; b=hOaOqRDsDWRR7Zd5L3bK6xnnsbQq6NWMNf1omt0sLcofEbO9bn6BPJpOg8r+4+DFYD RrCcWyzlA8tB6JJVOzdubKhdy+TXT7kqPEdk2iCHAktZIpaVhRKRvJePE1sjZOLhXYaf W09CzGpbNkw3Z3PCUC4uxcatMuK9L+E9AFB89CUOvJzoeWwwD9qkMK88s8oiVSxYKm35 kt45u+PmDiGFBZf8d/smMWnYt5iHuXKKwaqt/h7/RK3OlXPRuU4k6ZJWR/fP49HfeV+J dszCwJEARt07zWBfJ7EmiPJEmJk9j+itfrsf7dJyT1IXSgjTgG3tjeO2COBL+XPAnazo mHkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:reply-to :mail-followup-to:references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=SLf/bQHuPIk4kCUSXjBvZosjTCtFoLnAczMk9hzDGw0=; b=enED4odpfdO5QoL5J5vj7g/kZas2oVBhmpfT7Sx0KfhdtLtCTZWJVFuTSfLtTCJCe5 XOvOHlUkfSCOpWBBi7mIuqFvyk8Sy+SHvQzmz9DXhxeE6w7Zqkrru93KSiFlwwgn+MPw KxbnvRJGIHuxI/XdilbTFASAi1lW/EZ/iISVH3egnQ6LfqGtEINtRMS6EWc4MLcox7Yu DFgXsP7WIXf4Y+a7wfuit2PgJKWZrQKj48o/IhNc6FvXOB1lkEzmCDG9LSaKidBoHUhE J5a/dEcSWwcociWgSoMq/0qzyWHj+yreEwWfBB31bjqdAgA84AYjPIIdz7NM2WaWmMNX RG5A== X-Gm-Message-State: AOAM5303uqTvX9g54XsawXLi6UCYj/uNIJ+YgPgZvtRNi7klKMqHlLML ZyQnIfvdcUxYUENSkwqr0Ee4hA== X-Google-Smtp-Source: ABdhPJz9lND6jenE4yJViyO/k+i8ByeuJD/UHH11fv/AlJUIleGM5XqAEYjb5/bodoHBab6R4PqOfg== X-Received: by 2002:a17:906:378f:: with SMTP id n15mr38567548ejc.263.1609101161687; Sun, 27 Dec 2020 12:32:41 -0800 (PST) Received: from fedora.tometzki.de (p200300e68f334e009f56fbcb8d40a599.dip0.t-ipconnect.de. [2003:e6:8f33:4e00:9f56:fbcb:8d40:a599]) by smtp.gmail.com with ESMTPSA id gl23sm11446551ejb.87.2020.12.27.12.32.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Dec 2020 12:32:41 -0800 (PST) Date: Sun, 27 Dec 2020 21:32:37 +0100 From: Damian Tometzki To: Linus Torvalds Cc: Hugh Dickins , "Kirill A. Shutemov" , Matthew Wilcox , "Kirill A. Shutemov" , Will Deacon , Linux Kernel Mailing List , Linux-MM , Linux ARM , Catalin Marinas , Jan Kara , Minchan Kim , Andrew Morton , Vinayak Menon , Android Kernel Team Subject: Re: [PATCH 1/2] mm: Allow architectures to request 'old' entries when prefaulting Message-ID: Reply-To: Damian Tometzki Mail-Followup-To: Linus Torvalds , Hugh Dickins , "Kirill A. Shutemov" , Matthew Wilcox , "Kirill A. Shutemov" , Will Deacon , Linux Kernel Mailing List , Linux-MM , Linux ARM , Catalin Marinas , Jan Kara , Minchan Kim , Andrew Morton , Vinayak Menon , Android Kernel Team References: <20201225113157.e7hmluffh56fszfc@box> <20201226204335.dikqkrkezqet6oqf@box> <20201226224016.dxjmordcfj75xgte@box> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, 27. Dec 11:38, Linus Torvalds wrote: > On Sat, Dec 26, 2020 at 6:38 PM Hugh Dickins wrote: > > > > This patch (like its antecedents) moves the pte_unmap_unlock() from > > after do_fault_around()'s "check if the page fault is solved" into > > filemap_map_pages() itself (which apparently does not NULLify vmf->pte > > after unmapping it, which is poor, but good for revealing this issue). > > That looks cleaner, but of course there was a very good reason for its > > original positioning. > > Good catch. > > > Maybe you want to change the ->map_pages prototype, to pass down the > > requested address too, so that it can report whether the requested > > address was resolved or not. Or it could be left to __do_fault(), > > or even to a repeated fault; but those would be less efficient. > > Let's keep the old really odd "let's unlock in the caller" for now, > and minimize the changes. > > Adding a big big comment at the end of filemap_map_pages() to note the > odd delayed page table unlocking. > > Here's an updated patch that combines Kirill's original patch, his > additional incremental patch, and the fix for the pte lock oddity into > one thing. > > Does this finally pass your testing? > > Linus Hello together, when i try to build this patch, i got the following error: CC arch/x86/kernel/cpu/mce/threshold.o mm/memory.c:3716:19: error: static declaration of ‘do_set_pmd’ follows non-static declaration static vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) ^~~~~~~~~~ In file included from mm/memory.c:43: ./include/linux/mm.h:984:12: note: previous declaration of ‘do_set_pmd’ was here vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page); ^~~~~~~~~~ make[3]: *** [scripts/Makefile.build:279: mm/memory.o] Error 1 make[2]: *** [Makefile:1805: mm] Error 2 make[2]: *** Waiting for unfinished jobs.... CC arch/x86/kernel/cpu/mce/therm_throt.o Best regards Damian > From 4d221d934d112aa40c3f4978460be098fc9ce831 Mon Sep 17 00:00:00 2001 > From: "Kirill A. Shutemov" > Date: Sat, 19 Dec 2020 15:19:23 +0300 > Subject: [PATCH] mm: Cleanup faultaround and finish_fault() codepaths > > alloc_set_pte() has two users with different requirements: in the > faultaround code, it called from an atomic context and PTE page table > has to be preallocated. finish_fault() can sleep and allocate page table > as needed. > > PTL locking rules are also strange, hard to follow and overkill for > finish_fault(). > > Let's untangle the mess. alloc_set_pte() has gone now. All locking is > explicit. > > The price is some code duplication to handle huge pages in faultaround > path, but it should be fine, having overall improvement in readability. > > Signed-off-by: Kirill A. Shutemov > Signed-off-by: Linus Torvalds > --- > include/linux/mm.h | 8 +- > include/linux/pgtable.h | 11 +++ > mm/filemap.c | 168 ++++++++++++++++++++++++++++++---------- > mm/memory.c | 161 +++++++++++--------------------------- > 4 files changed, 192 insertions(+), 156 deletions(-) > > diff --git a/include/linux/mm.h b/include/linux/mm.h > index 5299b90a6c40..c0643a0ad5ff 100644 > --- a/include/linux/mm.h > +++ b/include/linux/mm.h > @@ -535,8 +535,8 @@ struct vm_fault { > * is not NULL, otherwise pmd. > */ > pgtable_t prealloc_pte; /* Pre-allocated pte page table. > - * vm_ops->map_pages() calls > - * alloc_set_pte() from atomic context. > + * vm_ops->map_pages() sets up a page > + * table from from atomic context. > * do_fault_around() pre-allocates > * page table to avoid allocation from > * atomic context. > @@ -981,7 +981,9 @@ static inline pte_t maybe_mkwrite(pte_t pte, struct vm_area_struct *vma) > return pte; > } > > -vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct page *page); > +vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page); > +void do_set_pte(struct vm_fault *vmf, struct page *page); > + > vm_fault_t finish_fault(struct vm_fault *vmf); > vm_fault_t finish_mkwrite_fault(struct vm_fault *vmf); > #endif > diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h > index 8fcdfa52eb4b..36eb748f3c97 100644 > --- a/include/linux/pgtable.h > +++ b/include/linux/pgtable.h > @@ -1314,6 +1314,17 @@ static inline int pmd_trans_unstable(pmd_t *pmd) > #endif > } > > +/* > + * the ordering of these checks is important for pmds with _page_devmap set. > + * if we check pmd_trans_unstable() first we will trip the bad_pmd() check > + * inside of pmd_none_or_trans_huge_or_clear_bad(). this will end up correctly > + * returning 1 but not before it spams dmesg with the pmd_clear_bad() output. > + */ > +static inline int pmd_devmap_trans_unstable(pmd_t *pmd) > +{ > + return pmd_devmap(*pmd) || pmd_trans_unstable(pmd); > +} > + > #ifndef CONFIG_NUMA_BALANCING > /* > * Technically a PTE can be PROTNONE even when not doing NUMA balancing but > diff --git a/mm/filemap.c b/mm/filemap.c > index 5c9d564317a5..dbc2eda92a53 100644 > --- a/mm/filemap.c > +++ b/mm/filemap.c > @@ -42,6 +42,7 @@ > #include > #include > #include > +#include > #include "internal.h" > > #define CREATE_TRACE_POINTS > @@ -2911,50 +2912,133 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) > } > EXPORT_SYMBOL(filemap_fault); > > +static bool filemap_map_pmd(struct vm_fault *vmf, struct page *page) > +{ > + struct mm_struct *mm = vmf->vma->vm_mm; > + > + /* Huge page is mapped? No need to proceed. */ > + if (pmd_trans_huge(*vmf->pmd)) { > + unlock_page(page); > + put_page(page); > + return true; > + } > + > + if (pmd_none(*vmf->pmd) && PageTransHuge(page)) { > + vm_fault_t ret = do_set_pmd(vmf, page); > + if (!ret) { > + /* The page is mapped successfully, reference consumed. */ > + unlock_page(page); > + return true; > + } > + } > + > + if (pmd_none(*vmf->pmd)) { > + vmf->ptl = pmd_lock(mm, vmf->pmd); > + if (likely(pmd_none(*vmf->pmd))) { > + mm_inc_nr_ptes(mm); > + pmd_populate(mm, vmf->pmd, vmf->prealloc_pte); > + vmf->prealloc_pte = NULL; > + } > + spin_unlock(vmf->ptl); > + } > + > + /* See comment in handle_pte_fault() */ > + if (pmd_devmap_trans_unstable(vmf->pmd)) { > + unlock_page(page); > + put_page(page); > + return true; > + } > + > + return false; > +} > + > +static struct page *next_uptodate_page(struct page *page, struct vm_fault *vmf, > + struct xa_state *xas, pgoff_t end_pgoff) > +{ > + struct address_space *mapping = vmf->vma->vm_file->f_mapping; > + unsigned long max_idx; > + > + do { > + if (!page) > + return NULL; > + if (xas_retry(xas, page)) > + continue; > + if (xa_is_value(page)) > + continue; > + if (PageLocked(page)) > + continue; > + if (!page_cache_get_speculative(page)) > + continue; > + /* Has the page moved or been split? */ > + if (unlikely(page != xas_reload(xas))) > + goto skip; > + if (!PageUptodate(page) || PageReadahead(page)) > + goto skip; > + if (PageHWPoison(page)) > + goto skip; > + if (!trylock_page(page)) > + goto skip; > + if (page->mapping != mapping) > + goto unlock; > + if (!PageUptodate(page)) > + goto unlock; > + max_idx = DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); > + if (xas->xa_index >= max_idx) > + goto unlock; > + return page; > +unlock: > + unlock_page(page); > +skip: > + put_page(page); > + } while ((page = xas_next_entry(xas, end_pgoff)) != NULL); > + > + return NULL; > +} > + > +static inline struct page *first_map_page(struct vm_fault *vmf, > + struct xa_state *xas, > + pgoff_t end_pgoff) > +{ > + return next_uptodate_page(xas_find(xas, end_pgoff), > + vmf, xas, end_pgoff); > +} > + > +static inline struct page *next_map_page(struct vm_fault *vmf, > + struct xa_state *xas, > + pgoff_t end_pgoff) > +{ > + return next_uptodate_page(xas_next_entry(xas, end_pgoff), > + vmf, xas, end_pgoff); > +} > + > void filemap_map_pages(struct vm_fault *vmf, > pgoff_t start_pgoff, pgoff_t end_pgoff) > { > - struct file *file = vmf->vma->vm_file; > + struct vm_area_struct *vma = vmf->vma; > + struct file *file = vma->vm_file; > struct address_space *mapping = file->f_mapping; > pgoff_t last_pgoff = start_pgoff; > - unsigned long max_idx; > XA_STATE(xas, &mapping->i_pages, start_pgoff); > struct page *head, *page; > unsigned int mmap_miss = READ_ONCE(file->f_ra.mmap_miss); > > rcu_read_lock(); > - xas_for_each(&xas, head, end_pgoff) { > - if (xas_retry(&xas, head)) > - continue; > - if (xa_is_value(head)) > - goto next; > + head = first_map_page(vmf, &xas, end_pgoff); > + if (!head) { > + rcu_read_unlock(); > + return; > + } > > - /* > - * Check for a locked page first, as a speculative > - * reference may adversely influence page migration. > - */ > - if (PageLocked(head)) > - goto next; > - if (!page_cache_get_speculative(head)) > - goto next; > + if (filemap_map_pmd(vmf, head)) { > + rcu_read_unlock(); > + return; > + } > > - /* Has the page moved or been split? */ > - if (unlikely(head != xas_reload(&xas))) > - goto skip; > + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, > + vmf->address, &vmf->ptl); > + do { > page = find_subpage(head, xas.xa_index); > - > - if (!PageUptodate(head) || > - PageReadahead(page) || > - PageHWPoison(page)) > - goto skip; > - if (!trylock_page(head)) > - goto skip; > - > - if (head->mapping != mapping || !PageUptodate(head)) > - goto unlock; > - > - max_idx = DIV_ROUND_UP(i_size_read(mapping->host), PAGE_SIZE); > - if (xas.xa_index >= max_idx) > + if (PageHWPoison(page)) > goto unlock; > > if (mmap_miss > 0) > @@ -2964,19 +3048,25 @@ void filemap_map_pages(struct vm_fault *vmf, > if (vmf->pte) > vmf->pte += xas.xa_index - last_pgoff; > last_pgoff = xas.xa_index; > - if (alloc_set_pte(vmf, page)) > + > + if (!pte_none(*vmf->pte)) > goto unlock; > + > + do_set_pte(vmf, page); > + /* no need to invalidate: a not-present page won't be cached */ > + update_mmu_cache(vma, vmf->address, vmf->pte); > unlock_page(head); > - goto next; > + continue; > unlock: > unlock_page(head); > -skip: > put_page(head); > -next: > - /* Huge page is mapped? No need to proceed. */ > - if (pmd_trans_huge(*vmf->pmd)) > - break; > - } > + } while ((head = next_map_page(vmf, &xas, end_pgoff)) != NULL); > + > + /* > + * NOTE! We return with the pte still locked! It is unlocked > + * by do_fault_around() after it has tested whether the target > + * address got filled in. > + */ > rcu_read_unlock(); > WRITE_ONCE(file->f_ra.mmap_miss, mmap_miss); > } > diff --git a/mm/memory.c b/mm/memory.c > index 7d608765932b..07a408c7d38b 100644 > --- a/mm/memory.c > +++ b/mm/memory.c > @@ -3501,7 +3501,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf) > if (pte_alloc(vma->vm_mm, vmf->pmd)) > return VM_FAULT_OOM; > > - /* See the comment in pte_alloc_one_map() */ > + /* See comment in handle_pte_fault() */ > if (unlikely(pmd_trans_unstable(vmf->pmd))) > return 0; > > @@ -3641,66 +3641,6 @@ static vm_fault_t __do_fault(struct vm_fault *vmf) > return ret; > } > > -/* > - * The ordering of these checks is important for pmds with _PAGE_DEVMAP set. > - * If we check pmd_trans_unstable() first we will trip the bad_pmd() check > - * inside of pmd_none_or_trans_huge_or_clear_bad(). This will end up correctly > - * returning 1 but not before it spams dmesg with the pmd_clear_bad() output. > - */ > -static int pmd_devmap_trans_unstable(pmd_t *pmd) > -{ > - return pmd_devmap(*pmd) || pmd_trans_unstable(pmd); > -} > - > -static vm_fault_t pte_alloc_one_map(struct vm_fault *vmf) > -{ > - struct vm_area_struct *vma = vmf->vma; > - > - if (!pmd_none(*vmf->pmd)) > - goto map_pte; > - if (vmf->prealloc_pte) { > - vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd); > - if (unlikely(!pmd_none(*vmf->pmd))) { > - spin_unlock(vmf->ptl); > - goto map_pte; > - } > - > - mm_inc_nr_ptes(vma->vm_mm); > - pmd_populate(vma->vm_mm, vmf->pmd, vmf->prealloc_pte); > - spin_unlock(vmf->ptl); > - vmf->prealloc_pte = NULL; > - } else if (unlikely(pte_alloc(vma->vm_mm, vmf->pmd))) { > - return VM_FAULT_OOM; > - } > -map_pte: > - /* > - * If a huge pmd materialized under us just retry later. Use > - * pmd_trans_unstable() via pmd_devmap_trans_unstable() instead of > - * pmd_trans_huge() to ensure the pmd didn't become pmd_trans_huge > - * under us and then back to pmd_none, as a result of MADV_DONTNEED > - * running immediately after a huge pmd fault in a different thread of > - * this mm, in turn leading to a misleading pmd_trans_huge() retval. > - * All we have to ensure is that it is a regular pmd that we can walk > - * with pte_offset_map() and we can do that through an atomic read in > - * C, which is what pmd_trans_unstable() provides. > - */ > - if (pmd_devmap_trans_unstable(vmf->pmd)) > - return VM_FAULT_NOPAGE; > - > - /* > - * At this point we know that our vmf->pmd points to a page of ptes > - * and it cannot become pmd_none(), pmd_devmap() or pmd_trans_huge() > - * for the duration of the fault. If a racing MADV_DONTNEED runs and > - * we zap the ptes pointed to by our vmf->pmd, the vmf->ptl will still > - * be valid and we will re-check to make sure the vmf->pte isn't > - * pte_none() under vmf->ptl protection when we return to > - * alloc_set_pte(). > - */ > - vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, vmf->address, > - &vmf->ptl); > - return 0; > -} > - > #ifdef CONFIG_TRANSPARENT_HUGEPAGE > static void deposit_prealloc_pte(struct vm_fault *vmf) > { > @@ -3715,7 +3655,7 @@ static void deposit_prealloc_pte(struct vm_fault *vmf) > vmf->prealloc_pte = NULL; > } > > -static vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) > +vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) > { > struct vm_area_struct *vma = vmf->vma; > bool write = vmf->flags & FAULT_FLAG_WRITE; > @@ -3780,45 +3720,11 @@ static vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) > } > #endif > > -/** > - * alloc_set_pte - setup new PTE entry for given page and add reverse page > - * mapping. If needed, the function allocates page table or use pre-allocated. > - * > - * @vmf: fault environment > - * @page: page to map > - * > - * Caller must take care of unlocking vmf->ptl, if vmf->pte is non-NULL on > - * return. > - * > - * Target users are page handler itself and implementations of > - * vm_ops->map_pages. > - * > - * Return: %0 on success, %VM_FAULT_ code in case of error. > - */ > -vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct page *page) > +void do_set_pte(struct vm_fault *vmf, struct page *page) > { > struct vm_area_struct *vma = vmf->vma; > bool write = vmf->flags & FAULT_FLAG_WRITE; > pte_t entry; > - vm_fault_t ret; > - > - if (pmd_none(*vmf->pmd) && PageTransCompound(page)) { > - ret = do_set_pmd(vmf, page); > - if (ret != VM_FAULT_FALLBACK) > - return ret; > - } > - > - if (!vmf->pte) { > - ret = pte_alloc_one_map(vmf); > - if (ret) > - return ret; > - } > - > - /* Re-check under ptl */ > - if (unlikely(!pte_none(*vmf->pte))) { > - update_mmu_tlb(vma, vmf->address, vmf->pte); > - return VM_FAULT_NOPAGE; > - } > > flush_icache_page(vma, page); > entry = mk_pte(page, vma->vm_page_prot); > @@ -3835,14 +3741,8 @@ vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct page *page) > page_add_file_rmap(page, false); > } > set_pte_at(vma->vm_mm, vmf->address, vmf->pte, entry); > - > - /* no need to invalidate: a not-present page won't be cached */ > - update_mmu_cache(vma, vmf->address, vmf->pte); > - > - return 0; > } > > - > /** > * finish_fault - finish page fault once we have prepared the page to fault > * > @@ -3860,12 +3760,12 @@ vm_fault_t alloc_set_pte(struct vm_fault *vmf, struct page *page) > */ > vm_fault_t finish_fault(struct vm_fault *vmf) > { > + struct vm_area_struct *vma = vmf->vma; > struct page *page; > - vm_fault_t ret = 0; > + vm_fault_t ret; > > /* Did we COW the page? */ > - if ((vmf->flags & FAULT_FLAG_WRITE) && > - !(vmf->vma->vm_flags & VM_SHARED)) > + if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) > page = vmf->cow_page; > else > page = vmf->page; > @@ -3874,13 +3774,35 @@ vm_fault_t finish_fault(struct vm_fault *vmf) > * check even for read faults because we might have lost our CoWed > * page > */ > - if (!(vmf->vma->vm_flags & VM_SHARED)) > - ret = check_stable_address_space(vmf->vma->vm_mm); > - if (!ret) > - ret = alloc_set_pte(vmf, page); > - if (vmf->pte) > - pte_unmap_unlock(vmf->pte, vmf->ptl); > - return ret; > + if (!(vma->vm_flags & VM_SHARED)) > + ret = check_stable_address_space(vma->vm_mm); > + if (ret) > + return ret; > + > + if (pmd_none(*vmf->pmd)) { > + if (PageTransCompound(page)) { > + ret = do_set_pmd(vmf, page); > + if (ret != VM_FAULT_FALLBACK) > + return ret; > + } > + > + if (unlikely(pte_alloc(vma->vm_mm, vmf->pmd))) > + return VM_FAULT_OOM; > + } > + > + /* See comment in handle_pte_fault() */ > + if (pmd_devmap_trans_unstable(vmf->pmd)) > + return 0; > + > + vmf->pte = pte_offset_map_lock(vma->vm_mm, vmf->pmd, > + vmf->address, &vmf->ptl); > + /* Re-check under ptl */ > + if (likely(pte_none(*vmf->pte))) > + do_set_pte(vmf, page); > + > + update_mmu_tlb(vma, vmf->address, vmf->pte); > + pte_unmap_unlock(vmf->pte, vmf->ptl); > + return 0; > } > > static unsigned long fault_around_bytes __read_mostly = > @@ -4351,7 +4273,18 @@ static vm_fault_t handle_pte_fault(struct vm_fault *vmf) > */ > vmf->pte = NULL; > } else { > - /* See comment in pte_alloc_one_map() */ > + /* > + * If a huge pmd materialized under us just retry later. Use > + * pmd_trans_unstable() via pmd_devmap_trans_unstable() instead > + * of pmd_trans_huge() to ensure the pmd didn't become > + * pmd_trans_huge under us and then back to pmd_none, as a > + * result of MADV_DONTNEED running immediately after a huge pmd > + * fault in a different thread of this mm, in turn leading to a > + * misleading pmd_trans_huge() retval. All we have to ensure is > + * that it is a regular pmd that we can walk with > + * pte_offset_map() and we can do that through an atomic read > + * in C, which is what pmd_trans_unstable() provides. > + */ > if (pmd_devmap_trans_unstable(vmf->pmd)) > return 0; > /* > -- > 2.29.2.157.g1d47791a39 > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 646A7C433E0 for ; Sun, 27 Dec 2020 20:35:08 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0B790221F5 for ; Sun, 27 Dec 2020 20:35:08 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0B790221F5 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=tometzki.de Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:Reply-To:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mGlsp4vo7aza7YSzOpeOctjnkooPG72pM6ADCeYxuoo=; b=p68pEXZLCBIf2C1AshkBX9eN1b eNOFbW6/lao8xchoLBy79wzPQjMSl+Aux4GU3oK/QRfTjAzFOO69OjhdWwzGtynwZSMJom/mD5B/v XZG3N++pk8+v0ldiV3IJ39jgYGLRTHghVjAp7+V0uXXETiI5PFS38od33bKA33ZfEOaUQYnY+u1bl GZUYfyzo1h5PO7lJn4zGmol7FEzaQRFR9g2sEJ4C64+lkiwGVA6SWkrOqXx6EhULn5NEGofr0sCfb JNem39gq0/WtRTNIJ3w5FX6Fdsn9Z5Z8nCy90QVVrDVG1WfKYfgG7roh40mRSvblJO4b/PNGu6PbD nZIPyIZA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ktcie-00082S-AZ; Sun, 27 Dec 2020 20:32:48 +0000 Received: from mail-ej1-x62e.google.com ([2a00:1450:4864:20::62e]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ktcia-00081V-T9 for linux-arm-kernel@lists.infradead.org; Sun, 27 Dec 2020 20:32:47 +0000 Received: by mail-ej1-x62e.google.com with SMTP id qw4so11899470ejb.12 for ; Sun, 27 Dec 2020 12:32:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tometzki.de; s=google; h=date:from:to:cc:subject:message-id:reply-to:mail-followup-to :references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=SLf/bQHuPIk4kCUSXjBvZosjTCtFoLnAczMk9hzDGw0=; b=hOaOqRDsDWRR7Zd5L3bK6xnnsbQq6NWMNf1omt0sLcofEbO9bn6BPJpOg8r+4+DFYD RrCcWyzlA8tB6JJVOzdubKhdy+TXT7kqPEdk2iCHAktZIpaVhRKRvJePE1sjZOLhXYaf W09CzGpbNkw3Z3PCUC4uxcatMuK9L+E9AFB89CUOvJzoeWwwD9qkMK88s8oiVSxYKm35 kt45u+PmDiGFBZf8d/smMWnYt5iHuXKKwaqt/h7/RK3OlXPRuU4k6ZJWR/fP49HfeV+J dszCwJEARt07zWBfJ7EmiPJEmJk9j+itfrsf7dJyT1IXSgjTgG3tjeO2COBL+XPAnazo mHkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:reply-to :mail-followup-to:references:mime-version:content-disposition :content-transfer-encoding:in-reply-to:user-agent; bh=SLf/bQHuPIk4kCUSXjBvZosjTCtFoLnAczMk9hzDGw0=; b=L08w6biCRmr2uLY0RFDz3Oh6kBaMFdLOVAVjZhNBRME9IRivSliKlK7W1ve5AIRWR2 3PWK+1GdypLPWeCsNfnNGr1JWX2xPdylBHUjreAH7cfCmX5rUEMIaiA+aVGK/3h3Sxrj KsXVL2zurYuVLSKsakOo+q1CogORoB5pPGyobst4AWCT9sCxpkXIAOUqHUQLIrDSpfTt KeSxxXjeWX/+dKNCI72Q33GeWKc743HovEdo+qU2MxGX5p1/AMd0eFJqWz8qpAG21Rkn 9kyraTs3SKSG6rJc6Fp/YWHL07+FMPSv5tv8EomvosV2RzaI9e9iHne76nMNmkxQ/wLx HubA== X-Gm-Message-State: AOAM5333uxSqjAZTTXZoXvAh2sQFtz8maPGHv6P/Uk8G0q648aHCdSsF +UOsPXgrS/ZlD05LU+zCZgQwVg== X-Google-Smtp-Source: ABdhPJz9lND6jenE4yJViyO/k+i8ByeuJD/UHH11fv/AlJUIleGM5XqAEYjb5/bodoHBab6R4PqOfg== X-Received: by 2002:a17:906:378f:: with SMTP id n15mr38567548ejc.263.1609101161687; Sun, 27 Dec 2020 12:32:41 -0800 (PST) Received: from fedora.tometzki.de (p200300e68f334e009f56fbcb8d40a599.dip0.t-ipconnect.de. [2003:e6:8f33:4e00:9f56:fbcb:8d40:a599]) by smtp.gmail.com with ESMTPSA id gl23sm11446551ejb.87.2020.12.27.12.32.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 27 Dec 2020 12:32:41 -0800 (PST) Date: Sun, 27 Dec 2020 21:32:37 +0100 From: Damian Tometzki To: Linus Torvalds Subject: Re: [PATCH 1/2] mm: Allow architectures to request 'old' entries when prefaulting Message-ID: Mail-Followup-To: Linus Torvalds , Hugh Dickins , "Kirill A. Shutemov" , Matthew Wilcox , "Kirill A. Shutemov" , Will Deacon , Linux Kernel Mailing List , Linux-MM , Linux ARM , Catalin Marinas , Jan Kara , Minchan Kim , Andrew Morton , Vinayak Menon , Android Kernel Team References: <20201225113157.e7hmluffh56fszfc@box> <20201226204335.dikqkrkezqet6oqf@box> <20201226224016.dxjmordcfj75xgte@box> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201227_153245_273204_86D9D3D7 X-CRM114-Status: GOOD ( 50.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Damian Tometzki Cc: Android Kernel Team , Jan Kara , Minchan Kim , Catalin Marinas , Hugh Dickins , Linux Kernel Mailing List , Matthew Wilcox , Linux-MM , Vinayak Menon , Linux ARM , "Kirill A. Shutemov" , Andrew Morton , Will Deacon , "Kirill A. Shutemov" Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gU3VuLCAyNy4gRGVjIDExOjM4LCBMaW51cyBUb3J2YWxkcyB3cm90ZToKPiBPbiBTYXQsIERl YyAyNiwgMjAyMCBhdCA2OjM4IFBNIEh1Z2ggRGlja2lucyA8aHVnaGRAZ29vZ2xlLmNvbT4gd3Jv dGU6Cj4gPgo+ID4gVGhpcyBwYXRjaCAobGlrZSBpdHMgYW50ZWNlZGVudHMpIG1vdmVzIHRoZSBw dGVfdW5tYXBfdW5sb2NrKCkgZnJvbQo+ID4gYWZ0ZXIgZG9fZmF1bHRfYXJvdW5kKCkncyAiY2hl Y2sgaWYgdGhlIHBhZ2UgZmF1bHQgaXMgc29sdmVkIiBpbnRvCj4gPiBmaWxlbWFwX21hcF9wYWdl cygpIGl0c2VsZiAod2hpY2ggYXBwYXJlbnRseSBkb2VzIG5vdCBOVUxMaWZ5IHZtZi0+cHRlCj4g PiBhZnRlciB1bm1hcHBpbmcgaXQsIHdoaWNoIGlzIHBvb3IsIGJ1dCBnb29kIGZvciByZXZlYWxp bmcgdGhpcyBpc3N1ZSkuCj4gPiBUaGF0IGxvb2tzIGNsZWFuZXIsIGJ1dCBvZiBjb3Vyc2UgdGhl cmUgd2FzIGEgdmVyeSBnb29kIHJlYXNvbiBmb3IgaXRzCj4gPiBvcmlnaW5hbCBwb3NpdGlvbmlu Zy4KPiAKPiBHb29kIGNhdGNoLgo+IAo+ID4gTWF5YmUgeW91IHdhbnQgdG8gY2hhbmdlIHRoZSAt Pm1hcF9wYWdlcyBwcm90b3R5cGUsIHRvIHBhc3MgZG93biB0aGUKPiA+IHJlcXVlc3RlZCBhZGRy ZXNzIHRvbywgc28gdGhhdCBpdCBjYW4gcmVwb3J0IHdoZXRoZXIgdGhlIHJlcXVlc3RlZAo+ID4g YWRkcmVzcyB3YXMgcmVzb2x2ZWQgb3Igbm90LiAgT3IgaXQgY291bGQgYmUgbGVmdCB0byBfX2Rv X2ZhdWx0KCksCj4gPiBvciBldmVuIHRvIGEgcmVwZWF0ZWQgZmF1bHQ7IGJ1dCB0aG9zZSB3b3Vs ZCBiZSBsZXNzIGVmZmljaWVudC4KPiAKPiBMZXQncyBrZWVwIHRoZSBvbGQgcmVhbGx5IG9kZCAi bGV0J3MgdW5sb2NrIGluIHRoZSBjYWxsZXIiIGZvciBub3csCj4gYW5kIG1pbmltaXplIHRoZSBj aGFuZ2VzLgo+IAo+IEFkZGluZyBhIGJpZyBiaWcgY29tbWVudCBhdCB0aGUgZW5kIG9mIGZpbGVt YXBfbWFwX3BhZ2VzKCkgdG8gbm90ZSB0aGUKPiBvZGQgZGVsYXllZCBwYWdlIHRhYmxlIHVubG9j a2luZy4KPiAKPiBIZXJlJ3MgYW4gdXBkYXRlZCBwYXRjaCB0aGF0IGNvbWJpbmVzIEtpcmlsbCdz IG9yaWdpbmFsIHBhdGNoLCBoaXMKPiBhZGRpdGlvbmFsIGluY3JlbWVudGFsIHBhdGNoLCBhbmQg dGhlIGZpeCBmb3IgdGhlIHB0ZSBsb2NrIG9kZGl0eSBpbnRvCj4gb25lIHRoaW5nLgo+IAo+IERv ZXMgdGhpcyBmaW5hbGx5IHBhc3MgeW91ciB0ZXN0aW5nPwo+IAo+ICAgICAgICAgICAgICAgIExp bnVzCkhlbGxvIHRvZ2V0aGVyLAoKd2hlbiBpIHRyeSB0byBidWlsZCB0aGlzIHBhdGNoLCBpIGdv dCB0aGUgZm9sbG93aW5nIGVycm9yOgoKIENDICAgICAgYXJjaC94ODYva2VybmVsL2NwdS9tY2Uv dGhyZXNob2xkLm8KbW0vbWVtb3J5LmM6MzcxNjoxOTogZXJyb3I6IHN0YXRpYyBkZWNsYXJhdGlv biBvZiDigJhkb19zZXRfcG1k4oCZIGZvbGxvd3Mgbm9uLXN0YXRpYyBkZWNsYXJhdGlvbgogc3Rh dGljIHZtX2ZhdWx0X3QgZG9fc2V0X3BtZChzdHJ1Y3Qgdm1fZmF1bHQgKnZtZiwgc3RydWN0IHBh Z2UgKnBhZ2UpCiAgICAgICAgICAgICAgICAgICBefn5+fn5+fn5+CkluIGZpbGUgaW5jbHVkZWQg ZnJvbSBtbS9tZW1vcnkuYzo0MzoKLi9pbmNsdWRlL2xpbnV4L21tLmg6OTg0OjEyOiBub3RlOiBw cmV2aW91cyBkZWNsYXJhdGlvbiBvZiDigJhkb19zZXRfcG1k4oCZIHdhcyBoZXJlCiB2bV9mYXVs dF90IGRvX3NldF9wbWQoc3RydWN0IHZtX2ZhdWx0ICp2bWYsIHN0cnVjdCBwYWdlICpwYWdlKTsK ICAgICAgICAgICAgXn5+fn5+fn5+fgptYWtlWzNdOiAqKiogW3NjcmlwdHMvTWFrZWZpbGUuYnVp bGQ6Mjc5OiBtbS9tZW1vcnkub10gRXJyb3IgMQptYWtlWzJdOiAqKiogW01ha2VmaWxlOjE4MDU6 IG1tXSBFcnJvciAyCm1ha2VbMl06ICoqKiBXYWl0aW5nIGZvciB1bmZpbmlzaGVkIGpvYnMuLi4u CiAgQ0MgICAgICBhcmNoL3g4Ni9rZXJuZWwvY3B1L21jZS90aGVybV90aHJvdC5vCgpCZXN0IHJl Z2FyZHMKRGFtaWFuCgo+IEZyb20gNGQyMjFkOTM0ZDExMmFhNDBjM2Y0OTc4NDYwYmUwOThmYzlj ZTgzMSBNb24gU2VwIDE3IDAwOjAwOjAwIDIwMDEKPiBGcm9tOiAiS2lyaWxsIEEuIFNodXRlbW92 IiA8a2lyaWxsLnNodXRlbW92QGxpbnV4LmludGVsLmNvbT4KPiBEYXRlOiBTYXQsIDE5IERlYyAy MDIwIDE1OjE5OjIzICswMzAwCj4gU3ViamVjdDogW1BBVENIXSBtbTogQ2xlYW51cCBmYXVsdGFy b3VuZCBhbmQgZmluaXNoX2ZhdWx0KCkgY29kZXBhdGhzCj4gCj4gYWxsb2Nfc2V0X3B0ZSgpIGhh cyB0d28gdXNlcnMgd2l0aCBkaWZmZXJlbnQgcmVxdWlyZW1lbnRzOiBpbiB0aGUKPiBmYXVsdGFy b3VuZCBjb2RlLCBpdCBjYWxsZWQgZnJvbSBhbiBhdG9taWMgY29udGV4dCBhbmQgUFRFIHBhZ2Ug dGFibGUKPiBoYXMgdG8gYmUgcHJlYWxsb2NhdGVkLiBmaW5pc2hfZmF1bHQoKSBjYW4gc2xlZXAg YW5kIGFsbG9jYXRlIHBhZ2UgdGFibGUKPiBhcyBuZWVkZWQuCj4gCj4gUFRMIGxvY2tpbmcgcnVs ZXMgYXJlIGFsc28gc3RyYW5nZSwgaGFyZCB0byBmb2xsb3cgYW5kIG92ZXJraWxsIGZvcgo+IGZp bmlzaF9mYXVsdCgpLgo+IAo+IExldCdzIHVudGFuZ2xlIHRoZSBtZXNzLiBhbGxvY19zZXRfcHRl KCkgaGFzIGdvbmUgbm93LiBBbGwgbG9ja2luZyBpcwo+IGV4cGxpY2l0Lgo+IAo+IFRoZSBwcmlj ZSBpcyBzb21lIGNvZGUgZHVwbGljYXRpb24gdG8gaGFuZGxlIGh1Z2UgcGFnZXMgaW4gZmF1bHRh cm91bmQKPiBwYXRoLCBidXQgaXQgc2hvdWxkIGJlIGZpbmUsIGhhdmluZyBvdmVyYWxsIGltcHJv dmVtZW50IGluIHJlYWRhYmlsaXR5Lgo+IAo+IFNpZ25lZC1vZmYtYnk6IEtpcmlsbCBBLiBTaHV0 ZW1vdiA8a2lyaWxsLnNodXRlbW92QGxpbnV4LmludGVsLmNvbT4KPiBTaWduZWQtb2ZmLWJ5OiBM aW51cyBUb3J2YWxkcyA8dG9ydmFsZHNAbGludXgtZm91bmRhdGlvbi5vcmc+Cj4gLS0tCj4gIGlu Y2x1ZGUvbGludXgvbW0uaCAgICAgIHwgICA4ICstCj4gIGluY2x1ZGUvbGludXgvcGd0YWJsZS5o IHwgIDExICsrKwo+ICBtbS9maWxlbWFwLmMgICAgICAgICAgICB8IDE2OCArKysrKysrKysrKysr KysrKysrKysrKysrKysrKystLS0tLS0tLS0tCj4gIG1tL21lbW9yeS5jICAgICAgICAgICAgIHwg MTYxICsrKysrKysrKysrLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tCj4gIDQgZmlsZXMgY2hh bmdlZCwgMTkyIGluc2VydGlvbnMoKyksIDE1NiBkZWxldGlvbnMoLSkKPiAKPiBkaWZmIC0tZ2l0 IGEvaW5jbHVkZS9saW51eC9tbS5oIGIvaW5jbHVkZS9saW51eC9tbS5oCj4gaW5kZXggNTI5OWI5 MGE2YzQwLi5jMDY0M2EwYWQ1ZmYgMTAwNjQ0Cj4gLS0tIGEvaW5jbHVkZS9saW51eC9tbS5oCj4g KysrIGIvaW5jbHVkZS9saW51eC9tbS5oCj4gQEAgLTUzNSw4ICs1MzUsOCBAQCBzdHJ1Y3Qgdm1f ZmF1bHQgewo+ICAJCQkJCSAqIGlzIG5vdCBOVUxMLCBvdGhlcndpc2UgcG1kLgo+ICAJCQkJCSAq Lwo+ICAJcGd0YWJsZV90IHByZWFsbG9jX3B0ZTsJCS8qIFByZS1hbGxvY2F0ZWQgcHRlIHBhZ2Ug dGFibGUuCj4gLQkJCQkJICogdm1fb3BzLT5tYXBfcGFnZXMoKSBjYWxscwo+IC0JCQkJCSAqIGFs bG9jX3NldF9wdGUoKSBmcm9tIGF0b21pYyBjb250ZXh0Lgo+ICsJCQkJCSAqIHZtX29wcy0+bWFw X3BhZ2VzKCkgc2V0cyB1cCBhIHBhZ2UKPiArCQkJCQkgKiB0YWJsZSBmcm9tIGZyb20gYXRvbWlj IGNvbnRleHQuCj4gIAkJCQkJICogZG9fZmF1bHRfYXJvdW5kKCkgcHJlLWFsbG9jYXRlcwo+ICAJ CQkJCSAqIHBhZ2UgdGFibGUgdG8gYXZvaWQgYWxsb2NhdGlvbiBmcm9tCj4gIAkJCQkJICogYXRv bWljIGNvbnRleHQuCj4gQEAgLTk4MSw3ICs5ODEsOSBAQCBzdGF0aWMgaW5saW5lIHB0ZV90IG1h eWJlX21rd3JpdGUocHRlX3QgcHRlLCBzdHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSkKPiAgCXJl dHVybiBwdGU7Cj4gIH0KPiAgCj4gLXZtX2ZhdWx0X3QgYWxsb2Nfc2V0X3B0ZShzdHJ1Y3Qgdm1f ZmF1bHQgKnZtZiwgc3RydWN0IHBhZ2UgKnBhZ2UpOwo+ICt2bV9mYXVsdF90IGRvX3NldF9wbWQo c3RydWN0IHZtX2ZhdWx0ICp2bWYsIHN0cnVjdCBwYWdlICpwYWdlKTsKPiArdm9pZCBkb19zZXRf cHRlKHN0cnVjdCB2bV9mYXVsdCAqdm1mLCBzdHJ1Y3QgcGFnZSAqcGFnZSk7Cj4gKwo+ICB2bV9m YXVsdF90IGZpbmlzaF9mYXVsdChzdHJ1Y3Qgdm1fZmF1bHQgKnZtZik7Cj4gIHZtX2ZhdWx0X3Qg ZmluaXNoX21rd3JpdGVfZmF1bHQoc3RydWN0IHZtX2ZhdWx0ICp2bWYpOwo+ICAjZW5kaWYKPiBk aWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9wZ3RhYmxlLmggYi9pbmNsdWRlL2xpbnV4L3BndGFi bGUuaAo+IGluZGV4IDhmY2RmYTUyZWI0Yi4uMzZlYjc0OGYzYzk3IDEwMDY0NAo+IC0tLSBhL2lu Y2x1ZGUvbGludXgvcGd0YWJsZS5oCj4gKysrIGIvaW5jbHVkZS9saW51eC9wZ3RhYmxlLmgKPiBA QCAtMTMxNCw2ICsxMzE0LDE3IEBAIHN0YXRpYyBpbmxpbmUgaW50IHBtZF90cmFuc191bnN0YWJs ZShwbWRfdCAqcG1kKQo+ICAjZW5kaWYKPiAgfQo+ICAKPiArLyoKPiArICogdGhlIG9yZGVyaW5n IG9mIHRoZXNlIGNoZWNrcyBpcyBpbXBvcnRhbnQgZm9yIHBtZHMgd2l0aCBfcGFnZV9kZXZtYXAg c2V0Lgo+ICsgKiBpZiB3ZSBjaGVjayBwbWRfdHJhbnNfdW5zdGFibGUoKSBmaXJzdCB3ZSB3aWxs IHRyaXAgdGhlIGJhZF9wbWQoKSBjaGVjawo+ICsgKiBpbnNpZGUgb2YgcG1kX25vbmVfb3JfdHJh bnNfaHVnZV9vcl9jbGVhcl9iYWQoKS4gdGhpcyB3aWxsIGVuZCB1cCBjb3JyZWN0bHkKPiArICog cmV0dXJuaW5nIDEgYnV0IG5vdCBiZWZvcmUgaXQgc3BhbXMgZG1lc2cgd2l0aCB0aGUgcG1kX2Ns ZWFyX2JhZCgpIG91dHB1dC4KPiArICovCj4gK3N0YXRpYyBpbmxpbmUgaW50IHBtZF9kZXZtYXBf dHJhbnNfdW5zdGFibGUocG1kX3QgKnBtZCkKPiArewo+ICsJcmV0dXJuIHBtZF9kZXZtYXAoKnBt ZCkgfHwgcG1kX3RyYW5zX3Vuc3RhYmxlKHBtZCk7Cj4gK30KPiArCj4gICNpZm5kZWYgQ09ORklH X05VTUFfQkFMQU5DSU5HCj4gIC8qCj4gICAqIFRlY2huaWNhbGx5IGEgUFRFIGNhbiBiZSBQUk9U Tk9ORSBldmVuIHdoZW4gbm90IGRvaW5nIE5VTUEgYmFsYW5jaW5nIGJ1dAo+IGRpZmYgLS1naXQg YS9tbS9maWxlbWFwLmMgYi9tbS9maWxlbWFwLmMKPiBpbmRleCA1YzlkNTY0MzE3YTUuLmRiYzJl ZGE5MmE1MyAxMDA2NDQKPiAtLS0gYS9tbS9maWxlbWFwLmMKPiArKysgYi9tbS9maWxlbWFwLmMK PiBAQCAtNDIsNiArNDIsNyBAQAo+ICAjaW5jbHVkZSA8bGludXgvcHNpLmg+Cj4gICNpbmNsdWRl IDxsaW51eC9yYW1mcy5oPgo+ICAjaW5jbHVkZSA8bGludXgvcGFnZV9pZGxlLmg+Cj4gKyNpbmNs dWRlIDxhc20vcGdhbGxvYy5oPgo+ICAjaW5jbHVkZSAiaW50ZXJuYWwuaCIKPiAgCj4gICNkZWZp bmUgQ1JFQVRFX1RSQUNFX1BPSU5UUwo+IEBAIC0yOTExLDUwICsyOTEyLDEzMyBAQCB2bV9mYXVs dF90IGZpbGVtYXBfZmF1bHQoc3RydWN0IHZtX2ZhdWx0ICp2bWYpCj4gIH0KPiAgRVhQT1JUX1NZ TUJPTChmaWxlbWFwX2ZhdWx0KTsKPiAgCj4gK3N0YXRpYyBib29sIGZpbGVtYXBfbWFwX3BtZChz dHJ1Y3Qgdm1fZmF1bHQgKnZtZiwgc3RydWN0IHBhZ2UgKnBhZ2UpCj4gK3sKPiArCXN0cnVjdCBt bV9zdHJ1Y3QgKm1tID0gdm1mLT52bWEtPnZtX21tOwo+ICsKPiArCS8qIEh1Z2UgcGFnZSBpcyBt YXBwZWQ/IE5vIG5lZWQgdG8gcHJvY2VlZC4gKi8KPiArCWlmIChwbWRfdHJhbnNfaHVnZSgqdm1m LT5wbWQpKSB7Cj4gKwkJdW5sb2NrX3BhZ2UocGFnZSk7Cj4gKwkJcHV0X3BhZ2UocGFnZSk7Cj4g KwkJcmV0dXJuIHRydWU7Cj4gKwl9Cj4gKwo+ICsJaWYgKHBtZF9ub25lKCp2bWYtPnBtZCkgJiYg UGFnZVRyYW5zSHVnZShwYWdlKSkgewo+ICsJICAgIHZtX2ZhdWx0X3QgcmV0ID0gZG9fc2V0X3Bt ZCh2bWYsIHBhZ2UpOwo+ICsJICAgIGlmICghcmV0KSB7Cj4gKwkJICAgIC8qIFRoZSBwYWdlIGlz IG1hcHBlZCBzdWNjZXNzZnVsbHksIHJlZmVyZW5jZSBjb25zdW1lZC4gKi8KPiArCQkgICAgdW5s b2NrX3BhZ2UocGFnZSk7Cj4gKwkJICAgIHJldHVybiB0cnVlOwo+ICsJICAgIH0KPiArCX0KPiAr Cj4gKwlpZiAocG1kX25vbmUoKnZtZi0+cG1kKSkgewo+ICsJCXZtZi0+cHRsID0gcG1kX2xvY2so bW0sIHZtZi0+cG1kKTsKPiArCQlpZiAobGlrZWx5KHBtZF9ub25lKCp2bWYtPnBtZCkpKSB7Cj4g KwkJCW1tX2luY19ucl9wdGVzKG1tKTsKPiArCQkJcG1kX3BvcHVsYXRlKG1tLCB2bWYtPnBtZCwg dm1mLT5wcmVhbGxvY19wdGUpOwo+ICsJCQl2bWYtPnByZWFsbG9jX3B0ZSA9IE5VTEw7Cj4gKwkJ fQo+ICsJCXNwaW5fdW5sb2NrKHZtZi0+cHRsKTsKPiArCX0KPiArCj4gKwkvKiBTZWUgY29tbWVu dCBpbiBoYW5kbGVfcHRlX2ZhdWx0KCkgKi8KPiArCWlmIChwbWRfZGV2bWFwX3RyYW5zX3Vuc3Rh YmxlKHZtZi0+cG1kKSkgewo+ICsJCXVubG9ja19wYWdlKHBhZ2UpOwo+ICsJCXB1dF9wYWdlKHBh Z2UpOwo+ICsJCXJldHVybiB0cnVlOwo+ICsJfQo+ICsKPiArCXJldHVybiBmYWxzZTsKPiArfQo+ ICsKPiArc3RhdGljIHN0cnVjdCBwYWdlICpuZXh0X3VwdG9kYXRlX3BhZ2Uoc3RydWN0IHBhZ2Ug KnBhZ2UsIHN0cnVjdCB2bV9mYXVsdCAqdm1mLAo+ICsJCQkJICAgICBzdHJ1Y3QgeGFfc3RhdGUg KnhhcywgcGdvZmZfdCBlbmRfcGdvZmYpCj4gK3sKPiArCXN0cnVjdCBhZGRyZXNzX3NwYWNlICpt YXBwaW5nID0gdm1mLT52bWEtPnZtX2ZpbGUtPmZfbWFwcGluZzsKPiArCXVuc2lnbmVkIGxvbmcg bWF4X2lkeDsKPiArCj4gKwlkbyB7Cj4gKwkJaWYgKCFwYWdlKQo+ICsJCQlyZXR1cm4gTlVMTDsK PiArCQlpZiAoeGFzX3JldHJ5KHhhcywgcGFnZSkpCj4gKwkJCWNvbnRpbnVlOwo+ICsJCWlmICh4 YV9pc192YWx1ZShwYWdlKSkKPiArCQkJY29udGludWU7Cj4gKwkJaWYgKFBhZ2VMb2NrZWQocGFn ZSkpCj4gKwkJCWNvbnRpbnVlOwo+ICsJCWlmICghcGFnZV9jYWNoZV9nZXRfc3BlY3VsYXRpdmUo cGFnZSkpCj4gKwkJCWNvbnRpbnVlOwo+ICsJCS8qIEhhcyB0aGUgcGFnZSBtb3ZlZCBvciBiZWVu IHNwbGl0PyAqLwo+ICsJCWlmICh1bmxpa2VseShwYWdlICE9IHhhc19yZWxvYWQoeGFzKSkpCj4g KwkJCWdvdG8gc2tpcDsKPiArCQlpZiAoIVBhZ2VVcHRvZGF0ZShwYWdlKSB8fCBQYWdlUmVhZGFo ZWFkKHBhZ2UpKQo+ICsJCQlnb3RvIHNraXA7Cj4gKwkJaWYgKFBhZ2VIV1BvaXNvbihwYWdlKSkK PiArCQkJZ290byBza2lwOwo+ICsJCWlmICghdHJ5bG9ja19wYWdlKHBhZ2UpKQo+ICsJCQlnb3Rv IHNraXA7Cj4gKwkJaWYgKHBhZ2UtPm1hcHBpbmcgIT0gbWFwcGluZykKPiArCQkJZ290byB1bmxv Y2s7Cj4gKwkJaWYgKCFQYWdlVXB0b2RhdGUocGFnZSkpCj4gKwkJCWdvdG8gdW5sb2NrOwo+ICsJ CW1heF9pZHggPSBESVZfUk9VTkRfVVAoaV9zaXplX3JlYWQobWFwcGluZy0+aG9zdCksIFBBR0Vf U0laRSk7Cj4gKwkJaWYgKHhhcy0+eGFfaW5kZXggPj0gbWF4X2lkeCkKPiArCQkJZ290byB1bmxv Y2s7Cj4gKwkJcmV0dXJuIHBhZ2U7Cj4gK3VubG9jazoKPiArCQl1bmxvY2tfcGFnZShwYWdlKTsK PiArc2tpcDoKPiArCQlwdXRfcGFnZShwYWdlKTsKPiArCX0gd2hpbGUgKChwYWdlID0geGFzX25l eHRfZW50cnkoeGFzLCBlbmRfcGdvZmYpKSAhPSBOVUxMKTsKPiArCj4gKwlyZXR1cm4gTlVMTDsK PiArfQo+ICsKPiArc3RhdGljIGlubGluZSBzdHJ1Y3QgcGFnZSAqZmlyc3RfbWFwX3BhZ2Uoc3Ry dWN0IHZtX2ZhdWx0ICp2bWYsCj4gKwkJCQkJICBzdHJ1Y3QgeGFfc3RhdGUgKnhhcywKPiArCQkJ CQkgIHBnb2ZmX3QgZW5kX3Bnb2ZmKQo+ICt7Cj4gKwlyZXR1cm4gbmV4dF91cHRvZGF0ZV9wYWdl KHhhc19maW5kKHhhcywgZW5kX3Bnb2ZmKSwKPiArCQkJCSAgdm1mLCB4YXMsIGVuZF9wZ29mZik7 Cj4gK30KPiArCj4gK3N0YXRpYyBpbmxpbmUgc3RydWN0IHBhZ2UgKm5leHRfbWFwX3BhZ2Uoc3Ry dWN0IHZtX2ZhdWx0ICp2bWYsCj4gKwkJCQkJICBzdHJ1Y3QgeGFfc3RhdGUgKnhhcywKPiArCQkJ CQkgIHBnb2ZmX3QgZW5kX3Bnb2ZmKQo+ICt7Cj4gKwlyZXR1cm4gbmV4dF91cHRvZGF0ZV9wYWdl KHhhc19uZXh0X2VudHJ5KHhhcywgZW5kX3Bnb2ZmKSwKPiArCQkJCSAgdm1mLCB4YXMsIGVuZF9w Z29mZik7Cj4gK30KPiArCj4gIHZvaWQgZmlsZW1hcF9tYXBfcGFnZXMoc3RydWN0IHZtX2ZhdWx0 ICp2bWYsCj4gIAkJcGdvZmZfdCBzdGFydF9wZ29mZiwgcGdvZmZfdCBlbmRfcGdvZmYpCj4gIHsK PiAtCXN0cnVjdCBmaWxlICpmaWxlID0gdm1mLT52bWEtPnZtX2ZpbGU7Cj4gKwlzdHJ1Y3Qgdm1f YXJlYV9zdHJ1Y3QgKnZtYSA9IHZtZi0+dm1hOwo+ICsJc3RydWN0IGZpbGUgKmZpbGUgPSB2bWEt PnZtX2ZpbGU7Cj4gIAlzdHJ1Y3QgYWRkcmVzc19zcGFjZSAqbWFwcGluZyA9IGZpbGUtPmZfbWFw cGluZzsKPiAgCXBnb2ZmX3QgbGFzdF9wZ29mZiA9IHN0YXJ0X3Bnb2ZmOwo+IC0JdW5zaWduZWQg bG9uZyBtYXhfaWR4Owo+ICAJWEFfU1RBVEUoeGFzLCAmbWFwcGluZy0+aV9wYWdlcywgc3RhcnRf cGdvZmYpOwo+ICAJc3RydWN0IHBhZ2UgKmhlYWQsICpwYWdlOwo+ICAJdW5zaWduZWQgaW50IG1t YXBfbWlzcyA9IFJFQURfT05DRShmaWxlLT5mX3JhLm1tYXBfbWlzcyk7Cj4gIAo+ICAJcmN1X3Jl YWRfbG9jaygpOwo+IC0JeGFzX2Zvcl9lYWNoKCZ4YXMsIGhlYWQsIGVuZF9wZ29mZikgewo+IC0J CWlmICh4YXNfcmV0cnkoJnhhcywgaGVhZCkpCj4gLQkJCWNvbnRpbnVlOwo+IC0JCWlmICh4YV9p c192YWx1ZShoZWFkKSkKPiAtCQkJZ290byBuZXh0Owo+ICsJaGVhZCA9IGZpcnN0X21hcF9wYWdl KHZtZiwgJnhhcywgZW5kX3Bnb2ZmKTsKPiArCWlmICghaGVhZCkgewo+ICsJCXJjdV9yZWFkX3Vu bG9jaygpOwo+ICsJCXJldHVybjsKPiArCX0KPiAgCj4gLQkJLyoKPiAtCQkgKiBDaGVjayBmb3Ig YSBsb2NrZWQgcGFnZSBmaXJzdCwgYXMgYSBzcGVjdWxhdGl2ZQo+IC0JCSAqIHJlZmVyZW5jZSBt YXkgYWR2ZXJzZWx5IGluZmx1ZW5jZSBwYWdlIG1pZ3JhdGlvbi4KPiAtCQkgKi8KPiAtCQlpZiAo UGFnZUxvY2tlZChoZWFkKSkKPiAtCQkJZ290byBuZXh0Owo+IC0JCWlmICghcGFnZV9jYWNoZV9n ZXRfc3BlY3VsYXRpdmUoaGVhZCkpCj4gLQkJCWdvdG8gbmV4dDsKPiArCWlmIChmaWxlbWFwX21h cF9wbWQodm1mLCBoZWFkKSkgewo+ICsJCXJjdV9yZWFkX3VubG9jaygpOwo+ICsJCXJldHVybjsK PiArCX0KPiAgCj4gLQkJLyogSGFzIHRoZSBwYWdlIG1vdmVkIG9yIGJlZW4gc3BsaXQ/ICovCj4g LQkJaWYgKHVubGlrZWx5KGhlYWQgIT0geGFzX3JlbG9hZCgmeGFzKSkpCj4gLQkJCWdvdG8gc2tp cDsKPiArCXZtZi0+cHRlID0gcHRlX29mZnNldF9tYXBfbG9jayh2bWEtPnZtX21tLCB2bWYtPnBt ZCwKPiArCQkJCSAgICAgICB2bWYtPmFkZHJlc3MsICZ2bWYtPnB0bCk7Cj4gKwlkbyB7Cj4gIAkJ cGFnZSA9IGZpbmRfc3VicGFnZShoZWFkLCB4YXMueGFfaW5kZXgpOwo+IC0KPiAtCQlpZiAoIVBh Z2VVcHRvZGF0ZShoZWFkKSB8fAo+IC0JCQkJUGFnZVJlYWRhaGVhZChwYWdlKSB8fAo+IC0JCQkJ UGFnZUhXUG9pc29uKHBhZ2UpKQo+IC0JCQlnb3RvIHNraXA7Cj4gLQkJaWYgKCF0cnlsb2NrX3Bh Z2UoaGVhZCkpCj4gLQkJCWdvdG8gc2tpcDsKPiAtCj4gLQkJaWYgKGhlYWQtPm1hcHBpbmcgIT0g bWFwcGluZyB8fCAhUGFnZVVwdG9kYXRlKGhlYWQpKQo+IC0JCQlnb3RvIHVubG9jazsKPiAtCj4g LQkJbWF4X2lkeCA9IERJVl9ST1VORF9VUChpX3NpemVfcmVhZChtYXBwaW5nLT5ob3N0KSwgUEFH RV9TSVpFKTsKPiAtCQlpZiAoeGFzLnhhX2luZGV4ID49IG1heF9pZHgpCj4gKwkJaWYgKFBhZ2VI V1BvaXNvbihwYWdlKSkKPiAgCQkJZ290byB1bmxvY2s7Cj4gIAo+ICAJCWlmIChtbWFwX21pc3Mg PiAwKQo+IEBAIC0yOTY0LDE5ICszMDQ4LDI1IEBAIHZvaWQgZmlsZW1hcF9tYXBfcGFnZXMoc3Ry dWN0IHZtX2ZhdWx0ICp2bWYsCj4gIAkJaWYgKHZtZi0+cHRlKQo+ICAJCQl2bWYtPnB0ZSArPSB4 YXMueGFfaW5kZXggLSBsYXN0X3Bnb2ZmOwo+ICAJCWxhc3RfcGdvZmYgPSB4YXMueGFfaW5kZXg7 Cj4gLQkJaWYgKGFsbG9jX3NldF9wdGUodm1mLCBwYWdlKSkKPiArCj4gKwkJaWYgKCFwdGVfbm9u ZSgqdm1mLT5wdGUpKQo+ICAJCQlnb3RvIHVubG9jazsKPiArCj4gKwkJZG9fc2V0X3B0ZSh2bWYs IHBhZ2UpOwo+ICsJCS8qIG5vIG5lZWQgdG8gaW52YWxpZGF0ZTogYSBub3QtcHJlc2VudCBwYWdl IHdvbid0IGJlIGNhY2hlZCAqLwo+ICsJCXVwZGF0ZV9tbXVfY2FjaGUodm1hLCB2bWYtPmFkZHJl c3MsIHZtZi0+cHRlKTsKPiAgCQl1bmxvY2tfcGFnZShoZWFkKTsKPiAtCQlnb3RvIG5leHQ7Cj4g KwkJY29udGludWU7Cj4gIHVubG9jazoKPiAgCQl1bmxvY2tfcGFnZShoZWFkKTsKPiAtc2tpcDoK PiAgCQlwdXRfcGFnZShoZWFkKTsKPiAtbmV4dDoKPiAtCQkvKiBIdWdlIHBhZ2UgaXMgbWFwcGVk PyBObyBuZWVkIHRvIHByb2NlZWQuICovCj4gLQkJaWYgKHBtZF90cmFuc19odWdlKCp2bWYtPnBt ZCkpCj4gLQkJCWJyZWFrOwo+IC0JfQo+ICsJfSB3aGlsZSAoKGhlYWQgPSBuZXh0X21hcF9wYWdl KHZtZiwgJnhhcywgZW5kX3Bnb2ZmKSkgIT0gTlVMTCk7Cj4gKwo+ICsJLyoKPiArCSAqIE5PVEUh IFdlIHJldHVybiB3aXRoIHRoZSBwdGUgc3RpbGwgbG9ja2VkISBJdCBpcyB1bmxvY2tlZAo+ICsJ ICogYnkgZG9fZmF1bHRfYXJvdW5kKCkgYWZ0ZXIgaXQgaGFzIHRlc3RlZCB3aGV0aGVyIHRoZSB0 YXJnZXQKPiArCSAqIGFkZHJlc3MgZ290IGZpbGxlZCBpbi4KPiArCSAqLwo+ICAJcmN1X3JlYWRf dW5sb2NrKCk7Cj4gIAlXUklURV9PTkNFKGZpbGUtPmZfcmEubW1hcF9taXNzLCBtbWFwX21pc3Mp Owo+ICB9Cj4gZGlmZiAtLWdpdCBhL21tL21lbW9yeS5jIGIvbW0vbWVtb3J5LmMKPiBpbmRleCA3 ZDYwODc2NTkzMmIuLjA3YTQwOGM3ZDM4YiAxMDA2NDQKPiAtLS0gYS9tbS9tZW1vcnkuYwo+ICsr KyBiL21tL21lbW9yeS5jCj4gQEAgLTM1MDEsNyArMzUwMSw3IEBAIHN0YXRpYyB2bV9mYXVsdF90 IGRvX2Fub255bW91c19wYWdlKHN0cnVjdCB2bV9mYXVsdCAqdm1mKQo+ICAJaWYgKHB0ZV9hbGxv Yyh2bWEtPnZtX21tLCB2bWYtPnBtZCkpCj4gIAkJcmV0dXJuIFZNX0ZBVUxUX09PTTsKPiAgCj4g LQkvKiBTZWUgdGhlIGNvbW1lbnQgaW4gcHRlX2FsbG9jX29uZV9tYXAoKSAqLwo+ICsJLyogU2Vl IGNvbW1lbnQgaW4gaGFuZGxlX3B0ZV9mYXVsdCgpICovCj4gIAlpZiAodW5saWtlbHkocG1kX3Ry YW5zX3Vuc3RhYmxlKHZtZi0+cG1kKSkpCj4gIAkJcmV0dXJuIDA7Cj4gIAo+IEBAIC0zNjQxLDY2 ICszNjQxLDYgQEAgc3RhdGljIHZtX2ZhdWx0X3QgX19kb19mYXVsdChzdHJ1Y3Qgdm1fZmF1bHQg KnZtZikKPiAgCXJldHVybiByZXQ7Cj4gIH0KPiAgCj4gLS8qCj4gLSAqIFRoZSBvcmRlcmluZyBv ZiB0aGVzZSBjaGVja3MgaXMgaW1wb3J0YW50IGZvciBwbWRzIHdpdGggX1BBR0VfREVWTUFQIHNl dC4KPiAtICogSWYgd2UgY2hlY2sgcG1kX3RyYW5zX3Vuc3RhYmxlKCkgZmlyc3Qgd2Ugd2lsbCB0 cmlwIHRoZSBiYWRfcG1kKCkgY2hlY2sKPiAtICogaW5zaWRlIG9mIHBtZF9ub25lX29yX3RyYW5z X2h1Z2Vfb3JfY2xlYXJfYmFkKCkuIFRoaXMgd2lsbCBlbmQgdXAgY29ycmVjdGx5Cj4gLSAqIHJl dHVybmluZyAxIGJ1dCBub3QgYmVmb3JlIGl0IHNwYW1zIGRtZXNnIHdpdGggdGhlIHBtZF9jbGVh cl9iYWQoKSBvdXRwdXQuCj4gLSAqLwo+IC1zdGF0aWMgaW50IHBtZF9kZXZtYXBfdHJhbnNfdW5z dGFibGUocG1kX3QgKnBtZCkKPiAtewo+IC0JcmV0dXJuIHBtZF9kZXZtYXAoKnBtZCkgfHwgcG1k X3RyYW5zX3Vuc3RhYmxlKHBtZCk7Cj4gLX0KPiAtCj4gLXN0YXRpYyB2bV9mYXVsdF90IHB0ZV9h bGxvY19vbmVfbWFwKHN0cnVjdCB2bV9mYXVsdCAqdm1mKQo+IC17Cj4gLQlzdHJ1Y3Qgdm1fYXJl YV9zdHJ1Y3QgKnZtYSA9IHZtZi0+dm1hOwo+IC0KPiAtCWlmICghcG1kX25vbmUoKnZtZi0+cG1k KSkKPiAtCQlnb3RvIG1hcF9wdGU7Cj4gLQlpZiAodm1mLT5wcmVhbGxvY19wdGUpIHsKPiAtCQl2 bWYtPnB0bCA9IHBtZF9sb2NrKHZtYS0+dm1fbW0sIHZtZi0+cG1kKTsKPiAtCQlpZiAodW5saWtl bHkoIXBtZF9ub25lKCp2bWYtPnBtZCkpKSB7Cj4gLQkJCXNwaW5fdW5sb2NrKHZtZi0+cHRsKTsK PiAtCQkJZ290byBtYXBfcHRlOwo+IC0JCX0KPiAtCj4gLQkJbW1faW5jX25yX3B0ZXModm1hLT52 bV9tbSk7Cj4gLQkJcG1kX3BvcHVsYXRlKHZtYS0+dm1fbW0sIHZtZi0+cG1kLCB2bWYtPnByZWFs bG9jX3B0ZSk7Cj4gLQkJc3Bpbl91bmxvY2sodm1mLT5wdGwpOwo+IC0JCXZtZi0+cHJlYWxsb2Nf cHRlID0gTlVMTDsKPiAtCX0gZWxzZSBpZiAodW5saWtlbHkocHRlX2FsbG9jKHZtYS0+dm1fbW0s IHZtZi0+cG1kKSkpIHsKPiAtCQlyZXR1cm4gVk1fRkFVTFRfT09NOwo+IC0JfQo+IC1tYXBfcHRl Ogo+IC0JLyoKPiAtCSAqIElmIGEgaHVnZSBwbWQgbWF0ZXJpYWxpemVkIHVuZGVyIHVzIGp1c3Qg cmV0cnkgbGF0ZXIuICBVc2UKPiAtCSAqIHBtZF90cmFuc191bnN0YWJsZSgpIHZpYSBwbWRfZGV2 bWFwX3RyYW5zX3Vuc3RhYmxlKCkgaW5zdGVhZCBvZgo+IC0JICogcG1kX3RyYW5zX2h1Z2UoKSB0 byBlbnN1cmUgdGhlIHBtZCBkaWRuJ3QgYmVjb21lIHBtZF90cmFuc19odWdlCj4gLQkgKiB1bmRl ciB1cyBhbmQgdGhlbiBiYWNrIHRvIHBtZF9ub25lLCBhcyBhIHJlc3VsdCBvZiBNQURWX0RPTlRO RUVECj4gLQkgKiBydW5uaW5nIGltbWVkaWF0ZWx5IGFmdGVyIGEgaHVnZSBwbWQgZmF1bHQgaW4g YSBkaWZmZXJlbnQgdGhyZWFkIG9mCj4gLQkgKiB0aGlzIG1tLCBpbiB0dXJuIGxlYWRpbmcgdG8g YSBtaXNsZWFkaW5nIHBtZF90cmFuc19odWdlKCkgcmV0dmFsLgo+IC0JICogQWxsIHdlIGhhdmUg dG8gZW5zdXJlIGlzIHRoYXQgaXQgaXMgYSByZWd1bGFyIHBtZCB0aGF0IHdlIGNhbiB3YWxrCj4g LQkgKiB3aXRoIHB0ZV9vZmZzZXRfbWFwKCkgYW5kIHdlIGNhbiBkbyB0aGF0IHRocm91Z2ggYW4g YXRvbWljIHJlYWQgaW4KPiAtCSAqIEMsIHdoaWNoIGlzIHdoYXQgcG1kX3RyYW5zX3Vuc3RhYmxl KCkgcHJvdmlkZXMuCj4gLQkgKi8KPiAtCWlmIChwbWRfZGV2bWFwX3RyYW5zX3Vuc3RhYmxlKHZt Zi0+cG1kKSkKPiAtCQlyZXR1cm4gVk1fRkFVTFRfTk9QQUdFOwo+IC0KPiAtCS8qCj4gLQkgKiBB dCB0aGlzIHBvaW50IHdlIGtub3cgdGhhdCBvdXIgdm1mLT5wbWQgcG9pbnRzIHRvIGEgcGFnZSBv ZiBwdGVzCj4gLQkgKiBhbmQgaXQgY2Fubm90IGJlY29tZSBwbWRfbm9uZSgpLCBwbWRfZGV2bWFw KCkgb3IgcG1kX3RyYW5zX2h1Z2UoKQo+IC0JICogZm9yIHRoZSBkdXJhdGlvbiBvZiB0aGUgZmF1 bHQuICBJZiBhIHJhY2luZyBNQURWX0RPTlRORUVEIHJ1bnMgYW5kCj4gLQkgKiB3ZSB6YXAgdGhl IHB0ZXMgcG9pbnRlZCB0byBieSBvdXIgdm1mLT5wbWQsIHRoZSB2bWYtPnB0bCB3aWxsIHN0aWxs Cj4gLQkgKiBiZSB2YWxpZCBhbmQgd2Ugd2lsbCByZS1jaGVjayB0byBtYWtlIHN1cmUgdGhlIHZt Zi0+cHRlIGlzbid0Cj4gLQkgKiBwdGVfbm9uZSgpIHVuZGVyIHZtZi0+cHRsIHByb3RlY3Rpb24g d2hlbiB3ZSByZXR1cm4gdG8KPiAtCSAqIGFsbG9jX3NldF9wdGUoKS4KPiAtCSAqLwo+IC0Jdm1m LT5wdGUgPSBwdGVfb2Zmc2V0X21hcF9sb2NrKHZtYS0+dm1fbW0sIHZtZi0+cG1kLCB2bWYtPmFk ZHJlc3MsCj4gLQkJCSZ2bWYtPnB0bCk7Cj4gLQlyZXR1cm4gMDsKPiAtfQo+IC0KPiAgI2lmZGVm IENPTkZJR19UUkFOU1BBUkVOVF9IVUdFUEFHRQo+ICBzdGF0aWMgdm9pZCBkZXBvc2l0X3ByZWFs bG9jX3B0ZShzdHJ1Y3Qgdm1fZmF1bHQgKnZtZikKPiAgewo+IEBAIC0zNzE1LDcgKzM2NTUsNyBA QCBzdGF0aWMgdm9pZCBkZXBvc2l0X3ByZWFsbG9jX3B0ZShzdHJ1Y3Qgdm1fZmF1bHQgKnZtZikK PiAgCXZtZi0+cHJlYWxsb2NfcHRlID0gTlVMTDsKPiAgfQo+ICAKPiAtc3RhdGljIHZtX2ZhdWx0 X3QgZG9fc2V0X3BtZChzdHJ1Y3Qgdm1fZmF1bHQgKnZtZiwgc3RydWN0IHBhZ2UgKnBhZ2UpCj4g K3ZtX2ZhdWx0X3QgZG9fc2V0X3BtZChzdHJ1Y3Qgdm1fZmF1bHQgKnZtZiwgc3RydWN0IHBhZ2Ug KnBhZ2UpCj4gIHsKPiAgCXN0cnVjdCB2bV9hcmVhX3N0cnVjdCAqdm1hID0gdm1mLT52bWE7Cj4g IAlib29sIHdyaXRlID0gdm1mLT5mbGFncyAmIEZBVUxUX0ZMQUdfV1JJVEU7Cj4gQEAgLTM3ODAs NDUgKzM3MjAsMTEgQEAgc3RhdGljIHZtX2ZhdWx0X3QgZG9fc2V0X3BtZChzdHJ1Y3Qgdm1fZmF1 bHQgKnZtZiwgc3RydWN0IHBhZ2UgKnBhZ2UpCj4gIH0KPiAgI2VuZGlmCj4gIAo+IC0vKioKPiAt ICogYWxsb2Nfc2V0X3B0ZSAtIHNldHVwIG5ldyBQVEUgZW50cnkgZm9yIGdpdmVuIHBhZ2UgYW5k IGFkZCByZXZlcnNlIHBhZ2UKPiAtICogbWFwcGluZy4gSWYgbmVlZGVkLCB0aGUgZnVuY3Rpb24g YWxsb2NhdGVzIHBhZ2UgdGFibGUgb3IgdXNlIHByZS1hbGxvY2F0ZWQuCj4gLSAqCj4gLSAqIEB2 bWY6IGZhdWx0IGVudmlyb25tZW50Cj4gLSAqIEBwYWdlOiBwYWdlIHRvIG1hcAo+IC0gKgo+IC0g KiBDYWxsZXIgbXVzdCB0YWtlIGNhcmUgb2YgdW5sb2NraW5nIHZtZi0+cHRsLCBpZiB2bWYtPnB0 ZSBpcyBub24tTlVMTCBvbgo+IC0gKiByZXR1cm4uCj4gLSAqCj4gLSAqIFRhcmdldCB1c2VycyBh cmUgcGFnZSBoYW5kbGVyIGl0c2VsZiBhbmQgaW1wbGVtZW50YXRpb25zIG9mCj4gLSAqIHZtX29w cy0+bWFwX3BhZ2VzLgo+IC0gKgo+IC0gKiBSZXR1cm46ICUwIG9uIHN1Y2Nlc3MsICVWTV9GQVVM VF8gY29kZSBpbiBjYXNlIG9mIGVycm9yLgo+IC0gKi8KPiAtdm1fZmF1bHRfdCBhbGxvY19zZXRf cHRlKHN0cnVjdCB2bV9mYXVsdCAqdm1mLCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiArdm9pZCBkb19z ZXRfcHRlKHN0cnVjdCB2bV9mYXVsdCAqdm1mLCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiAgewo+ICAJ c3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEgPSB2bWYtPnZtYTsKPiAgCWJvb2wgd3JpdGUgPSB2 bWYtPmZsYWdzICYgRkFVTFRfRkxBR19XUklURTsKPiAgCXB0ZV90IGVudHJ5Owo+IC0Jdm1fZmF1 bHRfdCByZXQ7Cj4gLQo+IC0JaWYgKHBtZF9ub25lKCp2bWYtPnBtZCkgJiYgUGFnZVRyYW5zQ29t cG91bmQocGFnZSkpIHsKPiAtCQlyZXQgPSBkb19zZXRfcG1kKHZtZiwgcGFnZSk7Cj4gLQkJaWYg KHJldCAhPSBWTV9GQVVMVF9GQUxMQkFDSykKPiAtCQkJcmV0dXJuIHJldDsKPiAtCX0KPiAtCj4g LQlpZiAoIXZtZi0+cHRlKSB7Cj4gLQkJcmV0ID0gcHRlX2FsbG9jX29uZV9tYXAodm1mKTsKPiAt CQlpZiAocmV0KQo+IC0JCQlyZXR1cm4gcmV0Owo+IC0JfQo+IC0KPiAtCS8qIFJlLWNoZWNrIHVu ZGVyIHB0bCAqLwo+IC0JaWYgKHVubGlrZWx5KCFwdGVfbm9uZSgqdm1mLT5wdGUpKSkgewo+IC0J CXVwZGF0ZV9tbXVfdGxiKHZtYSwgdm1mLT5hZGRyZXNzLCB2bWYtPnB0ZSk7Cj4gLQkJcmV0dXJu IFZNX0ZBVUxUX05PUEFHRTsKPiAtCX0KPiAgCj4gIAlmbHVzaF9pY2FjaGVfcGFnZSh2bWEsIHBh Z2UpOwo+ICAJZW50cnkgPSBta19wdGUocGFnZSwgdm1hLT52bV9wYWdlX3Byb3QpOwo+IEBAIC0z ODM1LDE0ICszNzQxLDggQEAgdm1fZmF1bHRfdCBhbGxvY19zZXRfcHRlKHN0cnVjdCB2bV9mYXVs dCAqdm1mLCBzdHJ1Y3QgcGFnZSAqcGFnZSkKPiAgCQlwYWdlX2FkZF9maWxlX3JtYXAocGFnZSwg ZmFsc2UpOwo+ICAJfQo+ICAJc2V0X3B0ZV9hdCh2bWEtPnZtX21tLCB2bWYtPmFkZHJlc3MsIHZt Zi0+cHRlLCBlbnRyeSk7Cj4gLQo+IC0JLyogbm8gbmVlZCB0byBpbnZhbGlkYXRlOiBhIG5vdC1w cmVzZW50IHBhZ2Ugd29uJ3QgYmUgY2FjaGVkICovCj4gLQl1cGRhdGVfbW11X2NhY2hlKHZtYSwg dm1mLT5hZGRyZXNzLCB2bWYtPnB0ZSk7Cj4gLQo+IC0JcmV0dXJuIDA7Cj4gIH0KPiAgCj4gLQo+ ICAvKioKPiAgICogZmluaXNoX2ZhdWx0IC0gZmluaXNoIHBhZ2UgZmF1bHQgb25jZSB3ZSBoYXZl IHByZXBhcmVkIHRoZSBwYWdlIHRvIGZhdWx0Cj4gICAqCj4gQEAgLTM4NjAsMTIgKzM3NjAsMTIg QEAgdm1fZmF1bHRfdCBhbGxvY19zZXRfcHRlKHN0cnVjdCB2bV9mYXVsdCAqdm1mLCBzdHJ1Y3Qg cGFnZSAqcGFnZSkKPiAgICovCj4gIHZtX2ZhdWx0X3QgZmluaXNoX2ZhdWx0KHN0cnVjdCB2bV9m YXVsdCAqdm1mKQo+ICB7Cj4gKwlzdHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSA9IHZtZi0+dm1h Owo+ICAJc3RydWN0IHBhZ2UgKnBhZ2U7Cj4gLQl2bV9mYXVsdF90IHJldCA9IDA7Cj4gKwl2bV9m YXVsdF90IHJldDsKPiAgCj4gIAkvKiBEaWQgd2UgQ09XIHRoZSBwYWdlPyAqLwo+IC0JaWYgKCh2 bWYtPmZsYWdzICYgRkFVTFRfRkxBR19XUklURSkgJiYKPiAtCSAgICAhKHZtZi0+dm1hLT52bV9m bGFncyAmIFZNX1NIQVJFRCkpCj4gKwlpZiAoKHZtZi0+ZmxhZ3MgJiBGQVVMVF9GTEFHX1dSSVRF KSAmJiAhKHZtYS0+dm1fZmxhZ3MgJiBWTV9TSEFSRUQpKQo+ICAJCXBhZ2UgPSB2bWYtPmNvd19w YWdlOwo+ICAJZWxzZQo+ICAJCXBhZ2UgPSB2bWYtPnBhZ2U7Cj4gQEAgLTM4NzQsMTMgKzM3NzQs MzUgQEAgdm1fZmF1bHRfdCBmaW5pc2hfZmF1bHQoc3RydWN0IHZtX2ZhdWx0ICp2bWYpCj4gIAkg KiBjaGVjayBldmVuIGZvciByZWFkIGZhdWx0cyBiZWNhdXNlIHdlIG1pZ2h0IGhhdmUgbG9zdCBv dXIgQ29XZWQKPiAgCSAqIHBhZ2UKPiAgCSAqLwo+IC0JaWYgKCEodm1mLT52bWEtPnZtX2ZsYWdz ICYgVk1fU0hBUkVEKSkKPiAtCQlyZXQgPSBjaGVja19zdGFibGVfYWRkcmVzc19zcGFjZSh2bWYt PnZtYS0+dm1fbW0pOwo+IC0JaWYgKCFyZXQpCj4gLQkJcmV0ID0gYWxsb2Nfc2V0X3B0ZSh2bWYs IHBhZ2UpOwo+IC0JaWYgKHZtZi0+cHRlKQo+IC0JCXB0ZV91bm1hcF91bmxvY2sodm1mLT5wdGUs IHZtZi0+cHRsKTsKPiAtCXJldHVybiByZXQ7Cj4gKwlpZiAoISh2bWEtPnZtX2ZsYWdzICYgVk1f U0hBUkVEKSkKPiArCQlyZXQgPSBjaGVja19zdGFibGVfYWRkcmVzc19zcGFjZSh2bWEtPnZtX21t KTsKPiArCWlmIChyZXQpCj4gKwkJcmV0dXJuIHJldDsKPiArCj4gKwlpZiAocG1kX25vbmUoKnZt Zi0+cG1kKSkgewo+ICsJCWlmIChQYWdlVHJhbnNDb21wb3VuZChwYWdlKSkgewo+ICsJCQlyZXQg PSBkb19zZXRfcG1kKHZtZiwgcGFnZSk7Cj4gKwkJCWlmIChyZXQgIT0gVk1fRkFVTFRfRkFMTEJB Q0spCj4gKwkJCQlyZXR1cm4gcmV0Owo+ICsJCX0KPiArCj4gKwkJaWYgKHVubGlrZWx5KHB0ZV9h bGxvYyh2bWEtPnZtX21tLCB2bWYtPnBtZCkpKQo+ICsJCQlyZXR1cm4gVk1fRkFVTFRfT09NOwo+ ICsJfQo+ICsKPiArCS8qIFNlZSBjb21tZW50IGluIGhhbmRsZV9wdGVfZmF1bHQoKSAqLwo+ICsJ aWYgKHBtZF9kZXZtYXBfdHJhbnNfdW5zdGFibGUodm1mLT5wbWQpKQo+ICsJCXJldHVybiAwOwo+ ICsKPiArCXZtZi0+cHRlID0gcHRlX29mZnNldF9tYXBfbG9jayh2bWEtPnZtX21tLCB2bWYtPnBt ZCwKPiArCQkJCSAgICAgIHZtZi0+YWRkcmVzcywgJnZtZi0+cHRsKTsKPiArCS8qIFJlLWNoZWNr IHVuZGVyIHB0bCAqLwo+ICsJaWYgKGxpa2VseShwdGVfbm9uZSgqdm1mLT5wdGUpKSkKPiArCQlk b19zZXRfcHRlKHZtZiwgcGFnZSk7Cj4gKwo+ICsJdXBkYXRlX21tdV90bGIodm1hLCB2bWYtPmFk ZHJlc3MsIHZtZi0+cHRlKTsKPiArCXB0ZV91bm1hcF91bmxvY2sodm1mLT5wdGUsIHZtZi0+cHRs KTsKPiArCXJldHVybiAwOwo+ICB9Cj4gIAo+ICBzdGF0aWMgdW5zaWduZWQgbG9uZyBmYXVsdF9h cm91bmRfYnl0ZXMgX19yZWFkX21vc3RseSA9Cj4gQEAgLTQzNTEsNyArNDI3MywxOCBAQCBzdGF0 aWMgdm1fZmF1bHRfdCBoYW5kbGVfcHRlX2ZhdWx0KHN0cnVjdCB2bV9mYXVsdCAqdm1mKQo+ICAJ CSAqLwo+ICAJCXZtZi0+cHRlID0gTlVMTDsKPiAgCX0gZWxzZSB7Cj4gLQkJLyogU2VlIGNvbW1l bnQgaW4gcHRlX2FsbG9jX29uZV9tYXAoKSAqLwo+ICsJCS8qCj4gKwkJICogSWYgYSBodWdlIHBt ZCBtYXRlcmlhbGl6ZWQgdW5kZXIgdXMganVzdCByZXRyeSBsYXRlci4gIFVzZQo+ICsJCSAqIHBt ZF90cmFuc191bnN0YWJsZSgpIHZpYSBwbWRfZGV2bWFwX3RyYW5zX3Vuc3RhYmxlKCkgaW5zdGVh ZAo+ICsJCSAqIG9mIHBtZF90cmFuc19odWdlKCkgdG8gZW5zdXJlIHRoZSBwbWQgZGlkbid0IGJl Y29tZQo+ICsJCSAqIHBtZF90cmFuc19odWdlIHVuZGVyIHVzIGFuZCB0aGVuIGJhY2sgdG8gcG1k X25vbmUsIGFzIGEKPiArCQkgKiByZXN1bHQgb2YgTUFEVl9ET05UTkVFRCBydW5uaW5nIGltbWVk aWF0ZWx5IGFmdGVyIGEgaHVnZSBwbWQKPiArCQkgKiBmYXVsdCBpbiBhIGRpZmZlcmVudCB0aHJl YWQgb2YgdGhpcyBtbSwgaW4gdHVybiBsZWFkaW5nIHRvIGEKPiArCQkgKiBtaXNsZWFkaW5nIHBt ZF90cmFuc19odWdlKCkgcmV0dmFsLiBBbGwgd2UgaGF2ZSB0byBlbnN1cmUgaXMKPiArCQkgKiB0 aGF0IGl0IGlzIGEgcmVndWxhciBwbWQgdGhhdCB3ZSBjYW4gd2FsayB3aXRoCj4gKwkJICogcHRl X29mZnNldF9tYXAoKSBhbmQgd2UgY2FuIGRvIHRoYXQgdGhyb3VnaCBhbiBhdG9taWMgcmVhZAo+ ICsJCSAqIGluIEMsIHdoaWNoIGlzIHdoYXQgcG1kX3RyYW5zX3Vuc3RhYmxlKCkgcHJvdmlkZXMu Cj4gKwkJICovCj4gIAkJaWYgKHBtZF9kZXZtYXBfdHJhbnNfdW5zdGFibGUodm1mLT5wbWQpKQo+ ICAJCQlyZXR1cm4gMDsKPiAgCQkvKgo+IC0tIAo+IDIuMjkuMi4xNTcuZzFkNDc3OTFhMzkKPiAK CgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpsaW51eC1h cm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3RzLmluZnJhZGVhZC5v cmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1hcm0t a2VybmVsCg==