From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AD40C433F5 for ; Wed, 9 Mar 2022 01:14:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230413AbiCIBPA (ORCPT ); Tue, 8 Mar 2022 20:15:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35558 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230285AbiCIBMV (ORCPT ); Tue, 8 Mar 2022 20:12:21 -0500 Received: from mail-il1-x14a.google.com (mail-il1-x14a.google.com [IPv6:2607:f8b0:4864:20::14a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E26EB155C03 for ; Tue, 8 Mar 2022 16:55:00 -0800 (PST) Received: by mail-il1-x14a.google.com with SMTP id y7-20020a056e02128700b002c62013aaa6so401668ilq.3 for ; Tue, 08 Mar 2022 16:55:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=Wo03oKTcfK74rQt9oHw4aIG6Rrb59PZv9cikIDl2s60=; b=cOi8fI8wS5MOKJz0oB5e92Fq16eKMJaMZEbT6oJj7VuEVMDTFpxU7tDshwbwCQrIFC mRx5Ep0fSwdewQaZfshqtz4b7leR87S0W2aqq/apOPkv5yO6z4IEmSoGiGPPm+dDMLcr N5TXaW1sZoJpJwmFW2v1OgdBAQwpQzBlj1Oe2ZtWHzuuYc6Jy3wiRM294OJQ7JNIw+Va uL3/h6NpujuoLeESMm0TRux4cKhQ6q53LdjuEVyKI2+sOqd+qHd8J6h/7HE/mOqABdpB 03B9kpX6KQ9Hg0nB28j+GsMAbvTP3KemvOHabR4LrMCz3w3t9W7NxN3boV50sjZPPIEf 03lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=Wo03oKTcfK74rQt9oHw4aIG6Rrb59PZv9cikIDl2s60=; b=VNS4mQwrLrrhZrtmT0SpyflQOTfV5RoG8S2R7gUk0Ecy0vtIICyPbLytSgwiUkGAy8 XnZadeNTb2ABXDuLc+BIfVtHJNoCPalLW440I6Hwo+iBkEzR1FTmWIYGgh19vWizL6bg PaB4JLdDwJVWN6A+MPCwyQ06tXBIkVsPjOsXzVKakczjoIH24MFu0XGEarOFXIjzyZBU OrFbYXIrvBDeD349amNVGTnnuxSyViStwbENL4OhtxzP/k0YZuDy8qtas7aVpW6RcK9T Ya2YfvNTE4VGxHxLadViozolaTIEjQwHMbaWM8i3oudzIQ/7kqIUXXROm8ph4XBmnGPA c8Qg== X-Gm-Message-State: AOAM532fAaxQB7YBJksdp/gSknvNNjQQP/E8HRhCAK4jAjn/As7YF8o2 0m6E5dbCC8mRmgoWAJ/BT8oVGWz9nrI= X-Google-Smtp-Source: ABdhPJw6nL8am7HX/xOXwwTuij2Z633NwxtO32bcjAeH18GCoGshRZ+5c4wn1ddhFNWIYlIBT1fIJQwB0FQ= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:57a6:54a6:aad1:c0a8]) (user=yuzhao job=sendgmr) by 2002:a5e:990b:0:b0:645:e9e5:3a9b with SMTP id t11-20020a5e990b000000b00645e9e53a9bmr6590718ioj.164.1646783313709; Tue, 08 Mar 2022 15:48:33 -0800 (PST) Date: Tue, 8 Mar 2022 16:47:17 -0700 In-Reply-To: <20220308234723.3834941-1-yuzhao@google.com> Message-Id: <20220308234723.3834941-8-yuzhao@google.com> Mime-Version: 1.0 References: <20220308234723.3834941-1-yuzhao@google.com> X-Mailer: git-send-email 2.35.1.616.g0bdcbb4464-goog Subject: [PATCH v8 07/14] mm: multi-gen LRU: exploit locality in rmap From: Yu Zhao To: Andrew Morton Cc: Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, page-reclaim@google.com, x86@kernel.org, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Searching the rmap for PTEs mapping each page on an LRU list (to test and clear the accessed bit) can be expensive because pages from different VMAs (PA space) are not cache friendly to the rmap (VA space). For workloads mostly using mapped pages, the rmap has a high CPU cost in the reclaim path. This patch exploits spatial locality to reduce the trips into the rmap. When shrink_page_list() walks the rmap and finds a young PTE, a new function lru_gen_look_around() scans at most BITS_PER_LONG-1 adjacent PTEs. On finding another young PTE, it clears the accessed bit and updates the gen counter of the page mapped by this PTE to (max_seq%MAX_NR_GENS)+1. Server benchmark results: Single workload: fio (buffered I/O): no change Single workload: memcached (anon): +[3.5, 5.5]% Ops/sec KB/sec patch1-5: 972526.07 37826.95 patch1-6: 1015292.83 39490.38 Configurations: no change Client benchmark results: kswapd profiles: patch1-5 39.73% lzo1x_1_do_compress (real work) 14.96% page_vma_mapped_walk 6.97% _raw_spin_unlock_irq 3.07% do_raw_spin_lock 2.53% anon_vma_interval_tree_iter_first 2.04% ptep_clear_flush 1.82% __zram_bvec_write 1.76% __anon_vma_interval_tree_subtree_search 1.57% memmove 1.45% free_unref_page_list patch1-6 45.49% lzo1x_1_do_compress (real work) 7.38% page_vma_mapped_walk 7.24% _raw_spin_unlock_irq 2.64% ptep_clear_flush 2.31% __zram_bvec_write 2.13% do_raw_spin_lock 2.09% lru_gen_look_around 1.89% free_unref_page_list 1.85% memmove 1.74% obj_malloc Configurations: no change Signed-off-by: Yu Zhao Acked-by: Brian Geffon Acked-by: Jan Alexander Steffens (heftig) Acked-by: Oleksandr Natalenko Acked-by: Steven Barrett Acked-by: Suleiman Souhlal Tested-by: Daniel Byrne Tested-by: Donald Carr Tested-by: Holger Hoffst=C3=A4tte Tested-by: Konstantin Kharlamov Tested-by: Shuang Zhai Tested-by: Sofia Trinh Tested-by: Vaibhav Jain --- include/linux/memcontrol.h | 31 ++++++++ include/linux/mm.h | 5 ++ include/linux/mmzone.h | 6 ++ include/linux/swap.h | 1 + mm/memcontrol.c | 1 + mm/rmap.c | 7 ++ mm/swap.c | 4 +- mm/vmscan.c | 155 +++++++++++++++++++++++++++++++++++++ 8 files changed, 208 insertions(+), 2 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 0abbd685703b..c8ce74577290 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -437,6 +437,7 @@ static inline struct obj_cgroup *__folio_objcg(struct f= olio *folio) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() * * For a kmem folio a caller should hold an rcu read lock to protect memcg * associated with a kmem folio from being released. @@ -498,6 +499,7 @@ static inline struct mem_cgroup *folio_memcg_rcu(struct= folio *folio) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() * * For a kmem page a caller should hold an rcu read lock to protect memcg * associated with a kmem page from being released. @@ -935,6 +937,23 @@ void unlock_page_memcg(struct page *page); =20 void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val); =20 +/* try to stablize folio_memcg() for all the pages in a memcg */ +static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) +{ + rcu_read_lock(); + + if (mem_cgroup_disabled() || !atomic_read(&memcg->moving_account)) + return true; + + rcu_read_unlock(); + return false; +} + +static inline void mem_cgroup_unlock_pages(void) +{ + rcu_read_unlock(); +} + /* idx can be of type enum memcg_stat_item or node_stat_item */ static inline void mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) @@ -1372,6 +1391,18 @@ static inline void folio_memcg_unlock(struct folio *= folio) { } =20 +static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) +{ + /* to match folio_memcg_rcu() */ + rcu_read_lock(); + return true; +} + +static inline void mem_cgroup_unlock_pages(void) +{ + rcu_read_unlock(); +} + static inline void mem_cgroup_handle_over_high(void) { } diff --git a/include/linux/mm.h b/include/linux/mm.h index 1e3e6dd90c0f..1f3695e95942 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1588,6 +1588,11 @@ static inline unsigned long folio_pfn(struct folio *= folio) return page_to_pfn(&folio->page); } =20 +static inline struct folio *pfn_folio(unsigned long pfn) +{ + return page_folio(pfn_to_page(pfn)); +} + /* MIGRATE_CMA and ZONE_MOVABLE do not allow pin pages */ #ifdef CONFIG_MIGRATION static inline bool is_pinnable_page(struct page *page) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index b92fc0ef8762..2a8bae14de74 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -304,6 +304,7 @@ enum lruvec_flags { }; =20 struct lruvec; +struct page_vma_mapped_walk; =20 #define LRU_GEN_MASK ((BIT(LRU_GEN_WIDTH) - 1) << LRU_GEN_PGOFF) #define LRU_REFS_MASK ((BIT(LRU_REFS_WIDTH) - 1) << LRU_REFS_PGOFF) @@ -399,6 +400,7 @@ struct lru_gen_struct { }; =20 void lru_gen_init_lruvec(struct lruvec *lruvec); +void lru_gen_look_around(struct page_vma_mapped_walk *pvmw); =20 #ifdef CONFIG_MEMCG void lru_gen_init_memcg(struct mem_cgroup *memcg); @@ -411,6 +413,10 @@ static inline void lru_gen_init_lruvec(struct lruvec *= lruvec) { } =20 +static inline void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) +{ +} + #ifdef CONFIG_MEMCG static inline void lru_gen_init_memcg(struct mem_cgroup *memcg) { diff --git a/include/linux/swap.h b/include/linux/swap.h index 1d38d9475c4d..b37520d3ff1d 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -372,6 +372,7 @@ extern void lru_add_drain(void); extern void lru_add_drain_cpu(int cpu); extern void lru_add_drain_cpu_zone(struct zone *zone); extern void lru_add_drain_all(void); +extern void folio_activate(struct folio *folio); extern void deactivate_file_page(struct page *page); extern void deactivate_page(struct page *page); extern void mark_page_lazyfree(struct page *page); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 3fcbfeda259b..e4c30950aa3c 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2744,6 +2744,7 @@ static void commit_charge(struct folio *folio, struct= mem_cgroup *memcg) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() */ folio->memcg_data =3D (unsigned long)memcg; } diff --git a/mm/rmap.c b/mm/rmap.c index 6a1e8c7f6213..112e77dc62f4 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -73,6 +73,7 @@ #include #include #include +#include =20 #include =20 @@ -819,6 +820,12 @@ static bool page_referenced_one(struct page *page, str= uct vm_area_struct *vma, } =20 if (pvmw.pte) { + if (lru_gen_enabled() && pte_young(*pvmw.pte) && + !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ))) { + lru_gen_look_around(&pvmw); + referenced++; + } + if (ptep_clear_flush_young_notify(vma, address, pvmw.pte)) { /* diff --git a/mm/swap.c b/mm/swap.c index f5c0bcac8dcd..e65e7520bebf 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -344,7 +344,7 @@ static bool need_activate_page_drain(int cpu) return pagevec_count(&per_cpu(lru_pvecs.activate_page, cpu)) !=3D 0; } =20 -static void folio_activate(struct folio *folio) +void folio_activate(struct folio *folio) { if (folio_test_lru(folio) && !folio_test_active(folio) && !folio_test_unevictable(folio)) { @@ -364,7 +364,7 @@ static inline void activate_page_drain(int cpu) { } =20 -static void folio_activate(struct folio *folio) +void folio_activate(struct folio *folio) { struct lruvec *lruvec; =20 diff --git a/mm/vmscan.c b/mm/vmscan.c index 91a827ff665d..2b685aa0379c 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1558,6 +1558,11 @@ static unsigned int shrink_page_list(struct list_hea= d *page_list, if (!sc->may_unmap && page_mapped(page)) goto keep_locked; =20 + /* folio_update_gen() tried to promote this page? */ + if (lru_gen_enabled() && !ignore_references && + page_mapped(page) && PageReferenced(page)) + goto keep_locked; + may_enter_fs =3D (sc->gfp_mask & __GFP_FS) || (PageSwapCache(page) && (sc->gfp_mask & __GFP_IO)); =20 @@ -3225,6 +3230,31 @@ static bool positive_ctrl_err(struct ctrl_pos *sp, s= truct ctrl_pos *pv) * the aging *************************************************************************= *****/ =20 +static int folio_update_gen(struct folio *folio, int gen) +{ + unsigned long old_flags, new_flags; + + VM_BUG_ON(gen >=3D MAX_NR_GENS); + VM_BUG_ON(!rcu_read_lock_held()); + + do { + new_flags =3D old_flags =3D READ_ONCE(folio->flags); + + /* for shrink_page_list() */ + if (!(new_flags & LRU_GEN_MASK)) { + new_flags |=3D BIT(PG_referenced); + continue; + } + + new_flags &=3D ~LRU_GEN_MASK; + new_flags |=3D (gen + 1UL) << LRU_GEN_PGOFF; + new_flags &=3D ~(LRU_REFS_MASK | LRU_REFS_FLAGS); + } while (new_flags !=3D old_flags && + cmpxchg(&folio->flags, old_flags, new_flags) !=3D old_flags); + + return ((old_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1; +} + static int folio_inc_gen(struct lruvec *lruvec, struct folio *folio, bool = reclaiming) { unsigned long old_flags, new_flags; @@ -3237,6 +3267,10 @@ static int folio_inc_gen(struct lruvec *lruvec, stru= ct folio *folio, bool reclai VM_BUG_ON_FOLIO(!(new_flags & LRU_GEN_MASK), folio); =20 new_gen =3D ((new_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1; + /* folio_update_gen() has promoted this page? */ + if (new_gen >=3D 0 && new_gen !=3D old_gen) + return new_gen; + new_gen =3D (old_gen + 1) % MAX_NR_GENS; =20 new_flags &=3D ~LRU_GEN_MASK; @@ -3438,6 +3472,122 @@ static void lru_gen_age_node(struct pglist_data *pg= dat, struct scan_control *sc) } while ((memcg =3D mem_cgroup_iter(NULL, memcg, NULL))); } =20 +/* + * This function exploits spatial locality when shrink_page_list() walks t= he + * rmap. It scans the adjacent PTEs of a young PTE and promotes hot pages. + */ +void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) +{ + int i; + pte_t *pte; + unsigned long start; + unsigned long end; + unsigned long addr; + struct folio *folio; + unsigned long bitmap[BITS_TO_LONGS(MIN_LRU_BATCH)] =3D {}; + struct mem_cgroup *memcg =3D page_memcg(pvmw->page); + struct pglist_data *pgdat =3D page_pgdat(pvmw->page); + struct lruvec *lruvec =3D mem_cgroup_lruvec(memcg, pgdat); + DEFINE_MAX_SEQ(lruvec); + int old_gen, new_gen =3D lru_gen_from_seq(max_seq); + + lockdep_assert_held(pvmw->ptl); + VM_BUG_ON_PAGE(PageLRU(pvmw->page), pvmw->page); + + start =3D max(pvmw->address & PMD_MASK, pvmw->vma->vm_start); + end =3D pmd_addr_end(pvmw->address, pvmw->vma->vm_end); + + if (end - start > MIN_LRU_BATCH * PAGE_SIZE) { + if (pvmw->address - start < MIN_LRU_BATCH * PAGE_SIZE / 2) + end =3D start + MIN_LRU_BATCH * PAGE_SIZE; + else if (end - pvmw->address < MIN_LRU_BATCH * PAGE_SIZE / 2) + start =3D end - MIN_LRU_BATCH * PAGE_SIZE; + else { + start =3D pvmw->address - MIN_LRU_BATCH * PAGE_SIZE / 2; + end =3D pvmw->address + MIN_LRU_BATCH * PAGE_SIZE / 2; + } + } + + pte =3D pvmw->pte - (pvmw->address - start) / PAGE_SIZE; + + rcu_read_lock(); + arch_enter_lazy_mmu_mode(); + + for (i =3D 0, addr =3D start; addr !=3D end; i++, addr +=3D PAGE_SIZE) { + unsigned long pfn =3D pte_pfn(pte[i]); + + VM_BUG_ON(addr < pvmw->vma->vm_start || addr >=3D pvmw->vma->vm_end); + + if (!pte_present(pte[i]) || is_zero_pfn(pfn)) + continue; + + if (WARN_ON_ONCE(pte_devmap(pte[i]) || pte_special(pte[i]))) + continue; + + if (!pte_young(pte[i])) + continue; + + VM_BUG_ON(!pfn_valid(pfn)); + if (pfn < pgdat->node_start_pfn || pfn >=3D pgdat_end_pfn(pgdat)) + continue; + + folio =3D pfn_folio(pfn); + if (folio_nid(folio) !=3D pgdat->node_id) + continue; + + if (folio_memcg_rcu(folio) !=3D memcg) + continue; + + if (!ptep_test_and_clear_young(pvmw->vma, addr, pte + i)) + continue; + + if (pte_dirty(pte[i]) && !folio_test_dirty(folio) && + !(folio_test_anon(folio) && folio_test_swapbacked(folio) && + !folio_test_swapcache(folio))) + folio_mark_dirty(folio); + + old_gen =3D folio_lru_gen(folio); + if (old_gen < 0) + folio_set_referenced(folio); + else if (old_gen !=3D new_gen) + __set_bit(i, bitmap); + } + + arch_leave_lazy_mmu_mode(); + rcu_read_unlock(); + + if (bitmap_weight(bitmap, MIN_LRU_BATCH) < PAGEVEC_SIZE) { + for_each_set_bit(i, bitmap, MIN_LRU_BATCH) { + folio =3D page_folio(pte_page(pte[i])); + folio_activate(folio); + } + return; + } + + /* folio_update_gen() requires stable folio_memcg() */ + if (!mem_cgroup_trylock_pages(memcg)) + return; + + spin_lock_irq(&lruvec->lru_lock); + new_gen =3D lru_gen_from_seq(lruvec->lrugen.max_seq); + + for_each_set_bit(i, bitmap, MIN_LRU_BATCH) { + folio =3D page_folio(pte_page(pte[i])); + if (folio_memcg_rcu(folio) !=3D memcg) + continue; + + old_gen =3D folio_update_gen(folio, new_gen); + if (old_gen < 0 || old_gen =3D=3D new_gen) + continue; + + lru_gen_update_size(lruvec, folio, old_gen, new_gen); + } + + spin_unlock_irq(&lruvec->lru_lock); + + mem_cgroup_unlock_pages(); +} + /*************************************************************************= ***** * the eviction *************************************************************************= *****/ @@ -3471,6 +3621,11 @@ static bool sort_folio(struct lruvec *lruvec, struct= folio *folio, int tier_idx) return true; } =20 + if (gen !=3D lru_gen_from_seq(lrugen->min_seq[type])) { + list_move(&folio->lru, &lrugen->lists[gen][type][zone]); + return true; + } + if (tier > tier_idx) { int hist =3D lru_hist_from_seq(lrugen->min_seq[type]); =20 --=20 2.35.1.616.g0bdcbb4464-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2E82C433EF for ; Tue, 8 Mar 2022 23:51:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=THMmkm2jYvfL9F8ldpxLh1qqgqAtoeoYnFSL5tqUjXo=; b=JKCVCmQTvr5zyaW87/4qoxb0jB RasPVr4AvNQ1UbH2O0DgUn/cEaP4XzAQHUKdBHx7baEORkKE7FjEbAxa518glfOp1Z8gYFeIKoua7 mF0jgegVBSeuiq1tkjX1bDrmjoPCc8hDIGkCJyF67o0W089xbPYm5KYY0roHxci0kYkyi5UoCgxU8 Y524l1Pweq15pVCxj+f2cuL4/4IMLe2RHYJoI57IoSUAAwtaV+abIOfSIJhoJPjHwcRialuZ5CHSu TJoum6Hy5dzOL4jLV90Vvmkk5ZdPpLL0W+3eY/IS90m+if/ShH+pvJxwWW9COb7lY69ZxbsoLzgPu y0L3lLWw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nRjab-006atB-9x; Tue, 08 Mar 2022 23:50:01 +0000 Received: from mail-io1-xd49.google.com ([2607:f8b0:4864:20::d49]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nRjZD-006aMy-DS for linux-arm-kernel@lists.infradead.org; Tue, 08 Mar 2022 23:48:42 +0000 Received: by mail-io1-xd49.google.com with SMTP id z10-20020a056602080a00b00645b9fdc630so595412iow.5 for ; Tue, 08 Mar 2022 15:48:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=Wo03oKTcfK74rQt9oHw4aIG6Rrb59PZv9cikIDl2s60=; b=cOi8fI8wS5MOKJz0oB5e92Fq16eKMJaMZEbT6oJj7VuEVMDTFpxU7tDshwbwCQrIFC mRx5Ep0fSwdewQaZfshqtz4b7leR87S0W2aqq/apOPkv5yO6z4IEmSoGiGPPm+dDMLcr N5TXaW1sZoJpJwmFW2v1OgdBAQwpQzBlj1Oe2ZtWHzuuYc6Jy3wiRM294OJQ7JNIw+Va uL3/h6NpujuoLeESMm0TRux4cKhQ6q53LdjuEVyKI2+sOqd+qHd8J6h/7HE/mOqABdpB 03B9kpX6KQ9Hg0nB28j+GsMAbvTP3KemvOHabR4LrMCz3w3t9W7NxN3boV50sjZPPIEf 03lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=Wo03oKTcfK74rQt9oHw4aIG6Rrb59PZv9cikIDl2s60=; b=P1mITBL66UBVeIFgZYvYNBM8QMB9yC/7LvH2txQtVZVnETB7dCe06wub49/Ld0rpe1 93Fey2c8T2QDffmMODYqQaB85cWVsnp4D4yq0ljoRgQAw3OiBBQf8/XzfBtc65W/oGvC QruoVDcpLddhW7XS4rJsK8bnVNKmGsjjjaNGx93TI5+Sx+tHaKSL8ilaG+U0fIeY1yiv TlwdvHrpKTftBmpy+c82BPOBbzcJu1Hk+flwczTyqzOCw9cvRzEa5a+sZ/Nvgi1o5KF6 qdbRPIcA426QnsOxURg5eYNSd+cwEyASGFet6rxzzs1qBeuxeidj273iAOKENdQqBKXU RVZQ== X-Gm-Message-State: AOAM530QrDeCjPHV2KLW0dyyWD84GyQoNwOmPCWsYemKEgZJE114u86v 0jFBUyBomv3RYKeKxA+e4J5ZVXxnyLc= X-Google-Smtp-Source: ABdhPJw6nL8am7HX/xOXwwTuij2Z633NwxtO32bcjAeH18GCoGshRZ+5c4wn1ddhFNWIYlIBT1fIJQwB0FQ= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:57a6:54a6:aad1:c0a8]) (user=yuzhao job=sendgmr) by 2002:a5e:990b:0:b0:645:e9e5:3a9b with SMTP id t11-20020a5e990b000000b00645e9e53a9bmr6590718ioj.164.1646783313709; Tue, 08 Mar 2022 15:48:33 -0800 (PST) Date: Tue, 8 Mar 2022 16:47:17 -0700 In-Reply-To: <20220308234723.3834941-1-yuzhao@google.com> Message-Id: <20220308234723.3834941-8-yuzhao@google.com> Mime-Version: 1.0 References: <20220308234723.3834941-1-yuzhao@google.com> X-Mailer: git-send-email 2.35.1.616.g0bdcbb4464-goog Subject: [PATCH v8 07/14] mm: multi-gen LRU: exploit locality in rmap From: Yu Zhao To: Andrew Morton Cc: Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, page-reclaim@google.com, x86@kernel.org, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220308_154835_534601_03096D08 X-CRM114-Status: GOOD ( 25.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org U2VhcmNoaW5nIHRoZSBybWFwIGZvciBQVEVzIG1hcHBpbmcgZWFjaCBwYWdlIG9uIGFuIExSVSBs aXN0ICh0byB0ZXN0CmFuZCBjbGVhciB0aGUgYWNjZXNzZWQgYml0KSBjYW4gYmUgZXhwZW5zaXZl IGJlY2F1c2UgcGFnZXMgZnJvbQpkaWZmZXJlbnQgVk1BcyAoUEEgc3BhY2UpIGFyZSBub3QgY2Fj aGUgZnJpZW5kbHkgdG8gdGhlIHJtYXAgKFZBCnNwYWNlKS4gRm9yIHdvcmtsb2FkcyBtb3N0bHkg dXNpbmcgbWFwcGVkIHBhZ2VzLCB0aGUgcm1hcCBoYXMgYSBoaWdoCkNQVSBjb3N0IGluIHRoZSBy ZWNsYWltIHBhdGguCgpUaGlzIHBhdGNoIGV4cGxvaXRzIHNwYXRpYWwgbG9jYWxpdHkgdG8gcmVk dWNlIHRoZSB0cmlwcyBpbnRvIHRoZQpybWFwLiBXaGVuIHNocmlua19wYWdlX2xpc3QoKSB3YWxr cyB0aGUgcm1hcCBhbmQgZmluZHMgYSB5b3VuZyBQVEUsIGEKbmV3IGZ1bmN0aW9uIGxydV9nZW5f bG9va19hcm91bmQoKSBzY2FucyBhdCBtb3N0IEJJVFNfUEVSX0xPTkctMQphZGphY2VudCBQVEVz LiBPbiBmaW5kaW5nIGFub3RoZXIgeW91bmcgUFRFLCBpdCBjbGVhcnMgdGhlIGFjY2Vzc2VkCmJp dCBhbmQgdXBkYXRlcyB0aGUgZ2VuIGNvdW50ZXIgb2YgdGhlIHBhZ2UgbWFwcGVkIGJ5IHRoaXMg UFRFIHRvCihtYXhfc2VxJU1BWF9OUl9HRU5TKSsxLgoKU2VydmVyIGJlbmNobWFyayByZXN1bHRz OgogIFNpbmdsZSB3b3JrbG9hZDoKICAgIGZpbyAoYnVmZmVyZWQgSS9PKTogbm8gY2hhbmdlCgog IFNpbmdsZSB3b3JrbG9hZDoKICAgIG1lbWNhY2hlZCAoYW5vbik6ICtbMy41LCA1LjVdJQogICAg ICAgICAgICAgICAgT3BzL3NlYyAgICAgIEtCL3NlYwogICAgICBwYXRjaDEtNTogOTcyNTI2LjA3 ICAgIDM3ODI2Ljk1CiAgICAgIHBhdGNoMS02OiAxMDE1MjkyLjgzICAgMzk0OTAuMzgKCiAgQ29u ZmlndXJhdGlvbnM6CiAgICBubyBjaGFuZ2UKCkNsaWVudCBiZW5jaG1hcmsgcmVzdWx0czoKICBr c3dhcGQgcHJvZmlsZXM6CiAgICBwYXRjaDEtNQogICAgICAzOS43MyUgIGx6bzF4XzFfZG9fY29t cHJlc3MgKHJlYWwgd29yaykKICAgICAgMTQuOTYlICBwYWdlX3ZtYV9tYXBwZWRfd2FsawogICAg ICAgNi45NyUgIF9yYXdfc3Bpbl91bmxvY2tfaXJxCiAgICAgICAzLjA3JSAgZG9fcmF3X3NwaW5f bG9jawogICAgICAgMi41MyUgIGFub25fdm1hX2ludGVydmFsX3RyZWVfaXRlcl9maXJzdAogICAg ICAgMi4wNCUgIHB0ZXBfY2xlYXJfZmx1c2gKICAgICAgIDEuODIlICBfX3pyYW1fYnZlY193cml0 ZQogICAgICAgMS43NiUgIF9fYW5vbl92bWFfaW50ZXJ2YWxfdHJlZV9zdWJ0cmVlX3NlYXJjaAog ICAgICAgMS41NyUgIG1lbW1vdmUKICAgICAgIDEuNDUlICBmcmVlX3VucmVmX3BhZ2VfbGlzdAoK ICAgIHBhdGNoMS02CiAgICAgIDQ1LjQ5JSAgbHpvMXhfMV9kb19jb21wcmVzcyAocmVhbCB3b3Jr KQogICAgICAgNy4zOCUgIHBhZ2Vfdm1hX21hcHBlZF93YWxrCiAgICAgICA3LjI0JSAgX3Jhd19z cGluX3VubG9ja19pcnEKICAgICAgIDIuNjQlICBwdGVwX2NsZWFyX2ZsdXNoCiAgICAgICAyLjMx JSAgX196cmFtX2J2ZWNfd3JpdGUKICAgICAgIDIuMTMlICBkb19yYXdfc3Bpbl9sb2NrCiAgICAg ICAyLjA5JSAgbHJ1X2dlbl9sb29rX2Fyb3VuZAogICAgICAgMS44OSUgIGZyZWVfdW5yZWZfcGFn ZV9saXN0CiAgICAgICAxLjg1JSAgbWVtbW92ZQogICAgICAgMS43NCUgIG9ial9tYWxsb2MKCiAg Q29uZmlndXJhdGlvbnM6CiAgICBubyBjaGFuZ2UKClNpZ25lZC1vZmYtYnk6IFl1IFpoYW8gPHl1 emhhb0Bnb29nbGUuY29tPgpBY2tlZC1ieTogQnJpYW4gR2VmZm9uIDxiZ2VmZm9uQGdvb2dsZS5j b20+CkFja2VkLWJ5OiBKYW4gQWxleGFuZGVyIFN0ZWZmZW5zIChoZWZ0aWcpIDxoZWZ0aWdAYXJj aGxpbnV4Lm9yZz4KQWNrZWQtYnk6IE9sZWtzYW5kciBOYXRhbGVua28gPG9sZWtzYW5kckBuYXRh bGVua28ubmFtZT4KQWNrZWQtYnk6IFN0ZXZlbiBCYXJyZXR0IDxzdGV2ZW5AbGlxdW9yaXgubmV0 PgpBY2tlZC1ieTogU3VsZWltYW4gU291aGxhbCA8c3VsZWltYW5AZ29vZ2xlLmNvbT4KVGVzdGVk LWJ5OiBEYW5pZWwgQnlybmUgPGRqYnlybmVAbXR1LmVkdT4KVGVzdGVkLWJ5OiBEb25hbGQgQ2Fy ciA8ZEBjaGFvcy1yZWlucy5jb20+ClRlc3RlZC1ieTogSG9sZ2VyIEhvZmZzdMOkdHRlIDxob2xn ZXJAYXBwbGllZC1hc3luY2hyb255LmNvbT4KVGVzdGVkLWJ5OiBLb25zdGFudGluIEtoYXJsYW1v diA8SGktQW5nZWxAeWFuZGV4LnJ1PgpUZXN0ZWQtYnk6IFNodWFuZyBaaGFpIDxzemhhaTJAY3Mu cm9jaGVzdGVyLmVkdT4KVGVzdGVkLWJ5OiBTb2ZpYSBUcmluaCA8c29maWEudHJpbmhAZWRpLndv cmtzPgpUZXN0ZWQtYnk6IFZhaWJoYXYgSmFpbiA8dmFpYmhhdkBsaW51eC5pYm0uY29tPgotLS0K IGluY2x1ZGUvbGludXgvbWVtY29udHJvbC5oIHwgIDMxICsrKysrKysrCiBpbmNsdWRlL2xpbnV4 L21tLmggICAgICAgICB8ICAgNSArKwogaW5jbHVkZS9saW51eC9tbXpvbmUuaCAgICAgfCAgIDYg KysKIGluY2x1ZGUvbGludXgvc3dhcC5oICAgICAgIHwgICAxICsKIG1tL21lbWNvbnRyb2wuYyAg ICAgICAgICAgIHwgICAxICsKIG1tL3JtYXAuYyAgICAgICAgICAgICAgICAgIHwgICA3ICsrCiBt bS9zd2FwLmMgICAgICAgICAgICAgICAgICB8ICAgNCArLQogbW0vdm1zY2FuLmMgICAgICAgICAg ICAgICAgfCAxNTUgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKwogOCBmaWxl cyBjaGFuZ2VkLCAyMDggaW5zZXJ0aW9ucygrKSwgMiBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQg YS9pbmNsdWRlL2xpbnV4L21lbWNvbnRyb2wuaCBiL2luY2x1ZGUvbGludXgvbWVtY29udHJvbC5o CmluZGV4IDBhYmJkNjg1NzAzYi4uYzhjZTc0NTc3MjkwIDEwMDY0NAotLS0gYS9pbmNsdWRlL2xp bnV4L21lbWNvbnRyb2wuaAorKysgYi9pbmNsdWRlL2xpbnV4L21lbWNvbnRyb2wuaApAQCAtNDM3 LDYgKzQzNyw3IEBAIHN0YXRpYyBpbmxpbmUgc3RydWN0IG9ial9jZ3JvdXAgKl9fZm9saW9fb2Jq Y2coc3RydWN0IGZvbGlvICpmb2xpbykKICAqIC0gTFJVIGlzb2xhdGlvbgogICogLSBsb2NrX3Bh Z2VfbWVtY2coKQogICogLSBleGNsdXNpdmUgcmVmZXJlbmNlCisgKiAtIG1lbV9jZ3JvdXBfdHJ5 bG9ja19wYWdlcygpCiAgKgogICogRm9yIGEga21lbSBmb2xpbyBhIGNhbGxlciBzaG91bGQgaG9s ZCBhbiByY3UgcmVhZCBsb2NrIHRvIHByb3RlY3QgbWVtY2cKICAqIGFzc29jaWF0ZWQgd2l0aCBh IGttZW0gZm9saW8gZnJvbSBiZWluZyByZWxlYXNlZC4KQEAgLTQ5OCw2ICs0OTksNyBAQCBzdGF0 aWMgaW5saW5lIHN0cnVjdCBtZW1fY2dyb3VwICpmb2xpb19tZW1jZ19yY3Uoc3RydWN0IGZvbGlv ICpmb2xpbykKICAqIC0gTFJVIGlzb2xhdGlvbgogICogLSBsb2NrX3BhZ2VfbWVtY2coKQogICog LSBleGNsdXNpdmUgcmVmZXJlbmNlCisgKiAtIG1lbV9jZ3JvdXBfdHJ5bG9ja19wYWdlcygpCiAg KgogICogRm9yIGEga21lbSBwYWdlIGEgY2FsbGVyIHNob3VsZCBob2xkIGFuIHJjdSByZWFkIGxv Y2sgdG8gcHJvdGVjdCBtZW1jZwogICogYXNzb2NpYXRlZCB3aXRoIGEga21lbSBwYWdlIGZyb20g YmVpbmcgcmVsZWFzZWQuCkBAIC05MzUsNiArOTM3LDIzIEBAIHZvaWQgdW5sb2NrX3BhZ2VfbWVt Y2coc3RydWN0IHBhZ2UgKnBhZ2UpOwogCiB2b2lkIF9fbW9kX21lbWNnX3N0YXRlKHN0cnVjdCBt ZW1fY2dyb3VwICptZW1jZywgaW50IGlkeCwgaW50IHZhbCk7CiAKKy8qIHRyeSB0byBzdGFibGl6 ZSBmb2xpb19tZW1jZygpIGZvciBhbGwgdGhlIHBhZ2VzIGluIGEgbWVtY2cgKi8KK3N0YXRpYyBp bmxpbmUgYm9vbCBtZW1fY2dyb3VwX3RyeWxvY2tfcGFnZXMoc3RydWN0IG1lbV9jZ3JvdXAgKm1l bWNnKQoreworCXJjdV9yZWFkX2xvY2soKTsKKworCWlmIChtZW1fY2dyb3VwX2Rpc2FibGVkKCkg fHwgIWF0b21pY19yZWFkKCZtZW1jZy0+bW92aW5nX2FjY291bnQpKQorCQlyZXR1cm4gdHJ1ZTsK KworCXJjdV9yZWFkX3VubG9jaygpOworCXJldHVybiBmYWxzZTsKK30KKworc3RhdGljIGlubGlu ZSB2b2lkIG1lbV9jZ3JvdXBfdW5sb2NrX3BhZ2VzKHZvaWQpCit7CisJcmN1X3JlYWRfdW5sb2Nr KCk7Cit9CisKIC8qIGlkeCBjYW4gYmUgb2YgdHlwZSBlbnVtIG1lbWNnX3N0YXRfaXRlbSBvciBu b2RlX3N0YXRfaXRlbSAqLwogc3RhdGljIGlubGluZSB2b2lkIG1vZF9tZW1jZ19zdGF0ZShzdHJ1 Y3QgbWVtX2Nncm91cCAqbWVtY2csCiAJCQkJICAgaW50IGlkeCwgaW50IHZhbCkKQEAgLTEzNzIs NiArMTM5MSwxOCBAQCBzdGF0aWMgaW5saW5lIHZvaWQgZm9saW9fbWVtY2dfdW5sb2NrKHN0cnVj dCBmb2xpbyAqZm9saW8pCiB7CiB9CiAKK3N0YXRpYyBpbmxpbmUgYm9vbCBtZW1fY2dyb3VwX3Ry eWxvY2tfcGFnZXMoc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnKQoreworCS8qIHRvIG1hdGNoIGZv bGlvX21lbWNnX3JjdSgpICovCisJcmN1X3JlYWRfbG9jaygpOworCXJldHVybiB0cnVlOworfQor CitzdGF0aWMgaW5saW5lIHZvaWQgbWVtX2Nncm91cF91bmxvY2tfcGFnZXModm9pZCkKK3sKKwly Y3VfcmVhZF91bmxvY2soKTsKK30KKwogc3RhdGljIGlubGluZSB2b2lkIG1lbV9jZ3JvdXBfaGFu ZGxlX292ZXJfaGlnaCh2b2lkKQogewogfQpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tbS5o IGIvaW5jbHVkZS9saW51eC9tbS5oCmluZGV4IDFlM2U2ZGQ5MGMwZi4uMWYzNjk1ZTk1OTQyIDEw MDY0NAotLS0gYS9pbmNsdWRlL2xpbnV4L21tLmgKKysrIGIvaW5jbHVkZS9saW51eC9tbS5oCkBA IC0xNTg4LDYgKzE1ODgsMTEgQEAgc3RhdGljIGlubGluZSB1bnNpZ25lZCBsb25nIGZvbGlvX3Bm bihzdHJ1Y3QgZm9saW8gKmZvbGlvKQogCXJldHVybiBwYWdlX3RvX3BmbigmZm9saW8tPnBhZ2Up OwogfQogCitzdGF0aWMgaW5saW5lIHN0cnVjdCBmb2xpbyAqcGZuX2ZvbGlvKHVuc2lnbmVkIGxv bmcgcGZuKQoreworCXJldHVybiBwYWdlX2ZvbGlvKHBmbl90b19wYWdlKHBmbikpOworfQorCiAv KiBNSUdSQVRFX0NNQSBhbmQgWk9ORV9NT1ZBQkxFIGRvIG5vdCBhbGxvdyBwaW4gcGFnZXMgKi8K ICNpZmRlZiBDT05GSUdfTUlHUkFUSU9OCiBzdGF0aWMgaW5saW5lIGJvb2wgaXNfcGlubmFibGVf cGFnZShzdHJ1Y3QgcGFnZSAqcGFnZSkKZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvbW16b25l LmggYi9pbmNsdWRlL2xpbnV4L21tem9uZS5oCmluZGV4IGI5MmZjMGVmODc2Mi4uMmE4YmFlMTRk ZTc0IDEwMDY0NAotLS0gYS9pbmNsdWRlL2xpbnV4L21tem9uZS5oCisrKyBiL2luY2x1ZGUvbGlu dXgvbW16b25lLmgKQEAgLTMwNCw2ICszMDQsNyBAQCBlbnVtIGxydXZlY19mbGFncyB7CiB9Owog CiBzdHJ1Y3QgbHJ1dmVjOworc3RydWN0IHBhZ2Vfdm1hX21hcHBlZF93YWxrOwogCiAjZGVmaW5l IExSVV9HRU5fTUFTSwkJKChCSVQoTFJVX0dFTl9XSURUSCkgLSAxKSA8PCBMUlVfR0VOX1BHT0ZG KQogI2RlZmluZSBMUlVfUkVGU19NQVNLCQkoKEJJVChMUlVfUkVGU19XSURUSCkgLSAxKSA8PCBM UlVfUkVGU19QR09GRikKQEAgLTM5OSw2ICs0MDAsNyBAQCBzdHJ1Y3QgbHJ1X2dlbl9zdHJ1Y3Qg ewogfTsKIAogdm9pZCBscnVfZ2VuX2luaXRfbHJ1dmVjKHN0cnVjdCBscnV2ZWMgKmxydXZlYyk7 Cit2b2lkIGxydV9nZW5fbG9va19hcm91bmQoc3RydWN0IHBhZ2Vfdm1hX21hcHBlZF93YWxrICpw dm13KTsKIAogI2lmZGVmIENPTkZJR19NRU1DRwogdm9pZCBscnVfZ2VuX2luaXRfbWVtY2coc3Ry dWN0IG1lbV9jZ3JvdXAgKm1lbWNnKTsKQEAgLTQxMSw2ICs0MTMsMTAgQEAgc3RhdGljIGlubGlu ZSB2b2lkIGxydV9nZW5faW5pdF9scnV2ZWMoc3RydWN0IGxydXZlYyAqbHJ1dmVjKQogewogfQog CitzdGF0aWMgaW5saW5lIHZvaWQgbHJ1X2dlbl9sb29rX2Fyb3VuZChzdHJ1Y3QgcGFnZV92bWFf bWFwcGVkX3dhbGsgKnB2bXcpCit7Cit9CisKICNpZmRlZiBDT05GSUdfTUVNQ0cKIHN0YXRpYyBp bmxpbmUgdm9pZCBscnVfZ2VuX2luaXRfbWVtY2coc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnKQog ewpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9zd2FwLmggYi9pbmNsdWRlL2xpbnV4L3N3YXAu aAppbmRleCAxZDM4ZDk0NzVjNGQuLmIzNzUyMGQzZmYxZCAxMDA2NDQKLS0tIGEvaW5jbHVkZS9s aW51eC9zd2FwLmgKKysrIGIvaW5jbHVkZS9saW51eC9zd2FwLmgKQEAgLTM3Miw2ICszNzIsNyBA QCBleHRlcm4gdm9pZCBscnVfYWRkX2RyYWluKHZvaWQpOwogZXh0ZXJuIHZvaWQgbHJ1X2FkZF9k cmFpbl9jcHUoaW50IGNwdSk7CiBleHRlcm4gdm9pZCBscnVfYWRkX2RyYWluX2NwdV96b25lKHN0 cnVjdCB6b25lICp6b25lKTsKIGV4dGVybiB2b2lkIGxydV9hZGRfZHJhaW5fYWxsKHZvaWQpOwor ZXh0ZXJuIHZvaWQgZm9saW9fYWN0aXZhdGUoc3RydWN0IGZvbGlvICpmb2xpbyk7CiBleHRlcm4g dm9pZCBkZWFjdGl2YXRlX2ZpbGVfcGFnZShzdHJ1Y3QgcGFnZSAqcGFnZSk7CiBleHRlcm4gdm9p ZCBkZWFjdGl2YXRlX3BhZ2Uoc3RydWN0IHBhZ2UgKnBhZ2UpOwogZXh0ZXJuIHZvaWQgbWFya19w YWdlX2xhenlmcmVlKHN0cnVjdCBwYWdlICpwYWdlKTsKZGlmZiAtLWdpdCBhL21tL21lbWNvbnRy b2wuYyBiL21tL21lbWNvbnRyb2wuYwppbmRleCAzZmNiZmVkYTI1OWIuLmU0YzMwOTUwYWEzYyAx MDA2NDQKLS0tIGEvbW0vbWVtY29udHJvbC5jCisrKyBiL21tL21lbWNvbnRyb2wuYwpAQCAtMjc0 NCw2ICsyNzQ0LDcgQEAgc3RhdGljIHZvaWQgY29tbWl0X2NoYXJnZShzdHJ1Y3QgZm9saW8gKmZv bGlvLCBzdHJ1Y3QgbWVtX2Nncm91cCAqbWVtY2cpCiAJICogLSBMUlUgaXNvbGF0aW9uCiAJICog LSBsb2NrX3BhZ2VfbWVtY2coKQogCSAqIC0gZXhjbHVzaXZlIHJlZmVyZW5jZQorCSAqIC0gbWVt X2Nncm91cF90cnlsb2NrX3BhZ2VzKCkKIAkgKi8KIAlmb2xpby0+bWVtY2dfZGF0YSA9ICh1bnNp Z25lZCBsb25nKW1lbWNnOwogfQpkaWZmIC0tZ2l0IGEvbW0vcm1hcC5jIGIvbW0vcm1hcC5jCmlu ZGV4IDZhMWU4YzdmNjIxMy4uMTEyZTc3ZGM2MmY0IDEwMDY0NAotLS0gYS9tbS9ybWFwLmMKKysr IGIvbW0vcm1hcC5jCkBAIC03Myw2ICs3Myw3IEBACiAjaW5jbHVkZSA8bGludXgvcGFnZV9pZGxl Lmg+CiAjaW5jbHVkZSA8bGludXgvbWVtcmVtYXAuaD4KICNpbmNsdWRlIDxsaW51eC91c2VyZmF1 bHRmZF9rLmg+CisjaW5jbHVkZSA8bGludXgvbW1faW5saW5lLmg+CiAKICNpbmNsdWRlIDxhc20v dGxiZmx1c2guaD4KIApAQCAtODE5LDYgKzgyMCwxMiBAQCBzdGF0aWMgYm9vbCBwYWdlX3JlZmVy ZW5jZWRfb25lKHN0cnVjdCBwYWdlICpwYWdlLCBzdHJ1Y3Qgdm1fYXJlYV9zdHJ1Y3QgKnZtYSwK IAkJfQogCiAJCWlmIChwdm13LnB0ZSkgeworCQkJaWYgKGxydV9nZW5fZW5hYmxlZCgpICYmIHB0 ZV95b3VuZygqcHZtdy5wdGUpICYmCisJCQkgICAgISh2bWEtPnZtX2ZsYWdzICYgKFZNX1NFUV9S RUFEIHwgVk1fUkFORF9SRUFEKSkpIHsKKwkJCQlscnVfZ2VuX2xvb2tfYXJvdW5kKCZwdm13KTsK KwkJCQlyZWZlcmVuY2VkKys7CisJCQl9CisKIAkJCWlmIChwdGVwX2NsZWFyX2ZsdXNoX3lvdW5n X25vdGlmeSh2bWEsIGFkZHJlc3MsCiAJCQkJCQlwdm13LnB0ZSkpIHsKIAkJCQkvKgpkaWZmIC0t Z2l0IGEvbW0vc3dhcC5jIGIvbW0vc3dhcC5jCmluZGV4IGY1YzBiY2FjOGRjZC4uZTY1ZTc1MjBi ZWJmIDEwMDY0NAotLS0gYS9tbS9zd2FwLmMKKysrIGIvbW0vc3dhcC5jCkBAIC0zNDQsNyArMzQ0 LDcgQEAgc3RhdGljIGJvb2wgbmVlZF9hY3RpdmF0ZV9wYWdlX2RyYWluKGludCBjcHUpCiAJcmV0 dXJuIHBhZ2V2ZWNfY291bnQoJnBlcl9jcHUobHJ1X3B2ZWNzLmFjdGl2YXRlX3BhZ2UsIGNwdSkp ICE9IDA7CiB9CiAKLXN0YXRpYyB2b2lkIGZvbGlvX2FjdGl2YXRlKHN0cnVjdCBmb2xpbyAqZm9s aW8pCit2b2lkIGZvbGlvX2FjdGl2YXRlKHN0cnVjdCBmb2xpbyAqZm9saW8pCiB7CiAJaWYgKGZv bGlvX3Rlc3RfbHJ1KGZvbGlvKSAmJiAhZm9saW9fdGVzdF9hY3RpdmUoZm9saW8pICYmCiAJICAg ICFmb2xpb190ZXN0X3VuZXZpY3RhYmxlKGZvbGlvKSkgewpAQCAtMzY0LDcgKzM2NCw3IEBAIHN0 YXRpYyBpbmxpbmUgdm9pZCBhY3RpdmF0ZV9wYWdlX2RyYWluKGludCBjcHUpCiB7CiB9CiAKLXN0 YXRpYyB2b2lkIGZvbGlvX2FjdGl2YXRlKHN0cnVjdCBmb2xpbyAqZm9saW8pCit2b2lkIGZvbGlv X2FjdGl2YXRlKHN0cnVjdCBmb2xpbyAqZm9saW8pCiB7CiAJc3RydWN0IGxydXZlYyAqbHJ1dmVj OwogCmRpZmYgLS1naXQgYS9tbS92bXNjYW4uYyBiL21tL3Ztc2Nhbi5jCmluZGV4IDkxYTgyN2Zm NjY1ZC4uMmI2ODVhYTAzNzljIDEwMDY0NAotLS0gYS9tbS92bXNjYW4uYworKysgYi9tbS92bXNj YW4uYwpAQCAtMTU1OCw2ICsxNTU4LDExIEBAIHN0YXRpYyB1bnNpZ25lZCBpbnQgc2hyaW5rX3Bh Z2VfbGlzdChzdHJ1Y3QgbGlzdF9oZWFkICpwYWdlX2xpc3QsCiAJCWlmICghc2MtPm1heV91bm1h cCAmJiBwYWdlX21hcHBlZChwYWdlKSkKIAkJCWdvdG8ga2VlcF9sb2NrZWQ7CiAKKwkJLyogZm9s aW9fdXBkYXRlX2dlbigpIHRyaWVkIHRvIHByb21vdGUgdGhpcyBwYWdlPyAqLworCQlpZiAobHJ1 X2dlbl9lbmFibGVkKCkgJiYgIWlnbm9yZV9yZWZlcmVuY2VzICYmCisJCSAgICBwYWdlX21hcHBl ZChwYWdlKSAmJiBQYWdlUmVmZXJlbmNlZChwYWdlKSkKKwkJCWdvdG8ga2VlcF9sb2NrZWQ7CisK IAkJbWF5X2VudGVyX2ZzID0gKHNjLT5nZnBfbWFzayAmIF9fR0ZQX0ZTKSB8fAogCQkJKFBhZ2VT d2FwQ2FjaGUocGFnZSkgJiYgKHNjLT5nZnBfbWFzayAmIF9fR0ZQX0lPKSk7CiAKQEAgLTMyMjUs NiArMzIzMCwzMSBAQCBzdGF0aWMgYm9vbCBwb3NpdGl2ZV9jdHJsX2VycihzdHJ1Y3QgY3RybF9w b3MgKnNwLCBzdHJ1Y3QgY3RybF9wb3MgKnB2KQogICogICAgICAgICAgICAgICAgICAgICAgICAg IHRoZSBhZ2luZwogICoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKi8KIAorc3RhdGljIGludCBmb2xpb191 cGRhdGVfZ2VuKHN0cnVjdCBmb2xpbyAqZm9saW8sIGludCBnZW4pCit7CisJdW5zaWduZWQgbG9u ZyBvbGRfZmxhZ3MsIG5ld19mbGFnczsKKworCVZNX0JVR19PTihnZW4gPj0gTUFYX05SX0dFTlMp OworCVZNX0JVR19PTighcmN1X3JlYWRfbG9ja19oZWxkKCkpOworCisJZG8geworCQluZXdfZmxh Z3MgPSBvbGRfZmxhZ3MgPSBSRUFEX09OQ0UoZm9saW8tPmZsYWdzKTsKKworCQkvKiBmb3Igc2hy aW5rX3BhZ2VfbGlzdCgpICovCisJCWlmICghKG5ld19mbGFncyAmIExSVV9HRU5fTUFTSykpIHsK KwkJCW5ld19mbGFncyB8PSBCSVQoUEdfcmVmZXJlbmNlZCk7CisJCQljb250aW51ZTsKKwkJfQor CisJCW5ld19mbGFncyAmPSB+TFJVX0dFTl9NQVNLOworCQluZXdfZmxhZ3MgfD0gKGdlbiArIDFV TCkgPDwgTFJVX0dFTl9QR09GRjsKKwkJbmV3X2ZsYWdzICY9IH4oTFJVX1JFRlNfTUFTSyB8IExS VV9SRUZTX0ZMQUdTKTsKKwl9IHdoaWxlIChuZXdfZmxhZ3MgIT0gb2xkX2ZsYWdzICYmCisJCSBj bXB4Y2hnKCZmb2xpby0+ZmxhZ3MsIG9sZF9mbGFncywgbmV3X2ZsYWdzKSAhPSBvbGRfZmxhZ3Mp OworCisJcmV0dXJuICgob2xkX2ZsYWdzICYgTFJVX0dFTl9NQVNLKSA+PiBMUlVfR0VOX1BHT0ZG KSAtIDE7Cit9CisKIHN0YXRpYyBpbnQgZm9saW9faW5jX2dlbihzdHJ1Y3QgbHJ1dmVjICpscnV2 ZWMsIHN0cnVjdCBmb2xpbyAqZm9saW8sIGJvb2wgcmVjbGFpbWluZykKIHsKIAl1bnNpZ25lZCBs b25nIG9sZF9mbGFncywgbmV3X2ZsYWdzOwpAQCAtMzIzNyw2ICszMjY3LDEwIEBAIHN0YXRpYyBp bnQgZm9saW9faW5jX2dlbihzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIHN0cnVjdCBmb2xpbyAqZm9s aW8sIGJvb2wgcmVjbGFpCiAJCVZNX0JVR19PTl9GT0xJTyghKG5ld19mbGFncyAmIExSVV9HRU5f TUFTSyksIGZvbGlvKTsKIAogCQluZXdfZ2VuID0gKChuZXdfZmxhZ3MgJiBMUlVfR0VOX01BU0sp ID4+IExSVV9HRU5fUEdPRkYpIC0gMTsKKwkJLyogZm9saW9fdXBkYXRlX2dlbigpIGhhcyBwcm9t b3RlZCB0aGlzIHBhZ2U/ICovCisJCWlmIChuZXdfZ2VuID49IDAgJiYgbmV3X2dlbiAhPSBvbGRf Z2VuKQorCQkJcmV0dXJuIG5ld19nZW47CisKIAkJbmV3X2dlbiA9IChvbGRfZ2VuICsgMSkgJSBN QVhfTlJfR0VOUzsKIAogCQluZXdfZmxhZ3MgJj0gfkxSVV9HRU5fTUFTSzsKQEAgLTM0MzgsNiAr MzQ3MiwxMjIgQEAgc3RhdGljIHZvaWQgbHJ1X2dlbl9hZ2Vfbm9kZShzdHJ1Y3QgcGdsaXN0X2Rh dGEgKnBnZGF0LCBzdHJ1Y3Qgc2Nhbl9jb250cm9sICpzYykKIAl9IHdoaWxlICgobWVtY2cgPSBt ZW1fY2dyb3VwX2l0ZXIoTlVMTCwgbWVtY2csIE5VTEwpKSk7CiB9CiAKKy8qCisgKiBUaGlzIGZ1 bmN0aW9uIGV4cGxvaXRzIHNwYXRpYWwgbG9jYWxpdHkgd2hlbiBzaHJpbmtfcGFnZV9saXN0KCkg d2Fsa3MgdGhlCisgKiBybWFwLiBJdCBzY2FucyB0aGUgYWRqYWNlbnQgUFRFcyBvZiBhIHlvdW5n IFBURSBhbmQgcHJvbW90ZXMgaG90IHBhZ2VzLgorICovCit2b2lkIGxydV9nZW5fbG9va19hcm91 bmQoc3RydWN0IHBhZ2Vfdm1hX21hcHBlZF93YWxrICpwdm13KQoreworCWludCBpOworCXB0ZV90 ICpwdGU7CisJdW5zaWduZWQgbG9uZyBzdGFydDsKKwl1bnNpZ25lZCBsb25nIGVuZDsKKwl1bnNp Z25lZCBsb25nIGFkZHI7CisJc3RydWN0IGZvbGlvICpmb2xpbzsKKwl1bnNpZ25lZCBsb25nIGJp dG1hcFtCSVRTX1RPX0xPTkdTKE1JTl9MUlVfQkFUQ0gpXSA9IHt9OworCXN0cnVjdCBtZW1fY2dy b3VwICptZW1jZyA9IHBhZ2VfbWVtY2cocHZtdy0+cGFnZSk7CisJc3RydWN0IHBnbGlzdF9kYXRh ICpwZ2RhdCA9IHBhZ2VfcGdkYXQocHZtdy0+cGFnZSk7CisJc3RydWN0IGxydXZlYyAqbHJ1dmVj ID0gbWVtX2Nncm91cF9scnV2ZWMobWVtY2csIHBnZGF0KTsKKwlERUZJTkVfTUFYX1NFUShscnV2 ZWMpOworCWludCBvbGRfZ2VuLCBuZXdfZ2VuID0gbHJ1X2dlbl9mcm9tX3NlcShtYXhfc2VxKTsK KworCWxvY2tkZXBfYXNzZXJ0X2hlbGQocHZtdy0+cHRsKTsKKwlWTV9CVUdfT05fUEFHRShQYWdl TFJVKHB2bXctPnBhZ2UpLCBwdm13LT5wYWdlKTsKKworCXN0YXJ0ID0gbWF4KHB2bXctPmFkZHJl c3MgJiBQTURfTUFTSywgcHZtdy0+dm1hLT52bV9zdGFydCk7CisJZW5kID0gcG1kX2FkZHJfZW5k KHB2bXctPmFkZHJlc3MsIHB2bXctPnZtYS0+dm1fZW5kKTsKKworCWlmIChlbmQgLSBzdGFydCA+ IE1JTl9MUlVfQkFUQ0ggKiBQQUdFX1NJWkUpIHsKKwkJaWYgKHB2bXctPmFkZHJlc3MgLSBzdGFy dCA8IE1JTl9MUlVfQkFUQ0ggKiBQQUdFX1NJWkUgLyAyKQorCQkJZW5kID0gc3RhcnQgKyBNSU5f TFJVX0JBVENIICogUEFHRV9TSVpFOworCQllbHNlIGlmIChlbmQgLSBwdm13LT5hZGRyZXNzIDwg TUlOX0xSVV9CQVRDSCAqIFBBR0VfU0laRSAvIDIpCisJCQlzdGFydCA9IGVuZCAtIE1JTl9MUlVf QkFUQ0ggKiBQQUdFX1NJWkU7CisJCWVsc2UgeworCQkJc3RhcnQgPSBwdm13LT5hZGRyZXNzIC0g TUlOX0xSVV9CQVRDSCAqIFBBR0VfU0laRSAvIDI7CisJCQllbmQgPSBwdm13LT5hZGRyZXNzICsg TUlOX0xSVV9CQVRDSCAqIFBBR0VfU0laRSAvIDI7CisJCX0KKwl9CisKKwlwdGUgPSBwdm13LT5w dGUgLSAocHZtdy0+YWRkcmVzcyAtIHN0YXJ0KSAvIFBBR0VfU0laRTsKKworCXJjdV9yZWFkX2xv Y2soKTsKKwlhcmNoX2VudGVyX2xhenlfbW11X21vZGUoKTsKKworCWZvciAoaSA9IDAsIGFkZHIg PSBzdGFydDsgYWRkciAhPSBlbmQ7IGkrKywgYWRkciArPSBQQUdFX1NJWkUpIHsKKwkJdW5zaWdu ZWQgbG9uZyBwZm4gPSBwdGVfcGZuKHB0ZVtpXSk7CisKKwkJVk1fQlVHX09OKGFkZHIgPCBwdm13 LT52bWEtPnZtX3N0YXJ0IHx8IGFkZHIgPj0gcHZtdy0+dm1hLT52bV9lbmQpOworCisJCWlmICgh cHRlX3ByZXNlbnQocHRlW2ldKSB8fCBpc196ZXJvX3BmbihwZm4pKQorCQkJY29udGludWU7CisK KwkJaWYgKFdBUk5fT05fT05DRShwdGVfZGV2bWFwKHB0ZVtpXSkgfHwgcHRlX3NwZWNpYWwocHRl W2ldKSkpCisJCQljb250aW51ZTsKKworCQlpZiAoIXB0ZV95b3VuZyhwdGVbaV0pKQorCQkJY29u dGludWU7CisKKwkJVk1fQlVHX09OKCFwZm5fdmFsaWQocGZuKSk7CisJCWlmIChwZm4gPCBwZ2Rh dC0+bm9kZV9zdGFydF9wZm4gfHwgcGZuID49IHBnZGF0X2VuZF9wZm4ocGdkYXQpKQorCQkJY29u dGludWU7CisKKwkJZm9saW8gPSBwZm5fZm9saW8ocGZuKTsKKwkJaWYgKGZvbGlvX25pZChmb2xp bykgIT0gcGdkYXQtPm5vZGVfaWQpCisJCQljb250aW51ZTsKKworCQlpZiAoZm9saW9fbWVtY2df cmN1KGZvbGlvKSAhPSBtZW1jZykKKwkJCWNvbnRpbnVlOworCisJCWlmICghcHRlcF90ZXN0X2Fu ZF9jbGVhcl95b3VuZyhwdm13LT52bWEsIGFkZHIsIHB0ZSArIGkpKQorCQkJY29udGludWU7CisK KwkJaWYgKHB0ZV9kaXJ0eShwdGVbaV0pICYmICFmb2xpb190ZXN0X2RpcnR5KGZvbGlvKSAmJgor CQkgICAgIShmb2xpb190ZXN0X2Fub24oZm9saW8pICYmIGZvbGlvX3Rlc3Rfc3dhcGJhY2tlZChm b2xpbykgJiYKKwkJICAgICAgIWZvbGlvX3Rlc3Rfc3dhcGNhY2hlKGZvbGlvKSkpCisJCQlmb2xp b19tYXJrX2RpcnR5KGZvbGlvKTsKKworCQlvbGRfZ2VuID0gZm9saW9fbHJ1X2dlbihmb2xpbyk7 CisJCWlmIChvbGRfZ2VuIDwgMCkKKwkJCWZvbGlvX3NldF9yZWZlcmVuY2VkKGZvbGlvKTsKKwkJ ZWxzZSBpZiAob2xkX2dlbiAhPSBuZXdfZ2VuKQorCQkJX19zZXRfYml0KGksIGJpdG1hcCk7CisJ fQorCisJYXJjaF9sZWF2ZV9sYXp5X21tdV9tb2RlKCk7CisJcmN1X3JlYWRfdW5sb2NrKCk7CisK KwlpZiAoYml0bWFwX3dlaWdodChiaXRtYXAsIE1JTl9MUlVfQkFUQ0gpIDwgUEFHRVZFQ19TSVpF KSB7CisJCWZvcl9lYWNoX3NldF9iaXQoaSwgYml0bWFwLCBNSU5fTFJVX0JBVENIKSB7CisJCQlm b2xpbyA9IHBhZ2VfZm9saW8ocHRlX3BhZ2UocHRlW2ldKSk7CisJCQlmb2xpb19hY3RpdmF0ZShm b2xpbyk7CisJCX0KKwkJcmV0dXJuOworCX0KKworCS8qIGZvbGlvX3VwZGF0ZV9nZW4oKSByZXF1 aXJlcyBzdGFibGUgZm9saW9fbWVtY2coKSAqLworCWlmICghbWVtX2Nncm91cF90cnlsb2NrX3Bh Z2VzKG1lbWNnKSkKKwkJcmV0dXJuOworCisJc3Bpbl9sb2NrX2lycSgmbHJ1dmVjLT5scnVfbG9j ayk7CisJbmV3X2dlbiA9IGxydV9nZW5fZnJvbV9zZXEobHJ1dmVjLT5scnVnZW4ubWF4X3NlcSk7 CisKKwlmb3JfZWFjaF9zZXRfYml0KGksIGJpdG1hcCwgTUlOX0xSVV9CQVRDSCkgeworCQlmb2xp byA9IHBhZ2VfZm9saW8ocHRlX3BhZ2UocHRlW2ldKSk7CisJCWlmIChmb2xpb19tZW1jZ19yY3Uo Zm9saW8pICE9IG1lbWNnKQorCQkJY29udGludWU7CisKKwkJb2xkX2dlbiA9IGZvbGlvX3VwZGF0 ZV9nZW4oZm9saW8sIG5ld19nZW4pOworCQlpZiAob2xkX2dlbiA8IDAgfHwgb2xkX2dlbiA9PSBu ZXdfZ2VuKQorCQkJY29udGludWU7CisKKwkJbHJ1X2dlbl91cGRhdGVfc2l6ZShscnV2ZWMsIGZv bGlvLCBvbGRfZ2VuLCBuZXdfZ2VuKTsKKwl9CisKKwlzcGluX3VubG9ja19pcnEoJmxydXZlYy0+ bHJ1X2xvY2spOworCisJbWVtX2Nncm91cF91bmxvY2tfcGFnZXMoKTsKK30KKwogLyoqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKgogICogICAgICAgICAgICAgICAgICAgICAgICAgIHRoZSBldmljdGlvbgog ICoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKi8KQEAgLTM0NzEsNiArMzYyMSwxMSBAQCBzdGF0aWMgYm9v bCBzb3J0X2ZvbGlvKHN0cnVjdCBscnV2ZWMgKmxydXZlYywgc3RydWN0IGZvbGlvICpmb2xpbywg aW50IHRpZXJfaWR4KQogCQlyZXR1cm4gdHJ1ZTsKIAl9CiAKKwlpZiAoZ2VuICE9IGxydV9nZW5f ZnJvbV9zZXEobHJ1Z2VuLT5taW5fc2VxW3R5cGVdKSkgeworCQlsaXN0X21vdmUoJmZvbGlvLT5s cnUsICZscnVnZW4tPmxpc3RzW2dlbl1bdHlwZV1bem9uZV0pOworCQlyZXR1cm4gdHJ1ZTsKKwl9 CisKIAlpZiAodGllciA+IHRpZXJfaWR4KSB7CiAJCWludCBoaXN0ID0gbHJ1X2hpc3RfZnJvbV9z ZXEobHJ1Z2VuLT5taW5fc2VxW3R5cGVdKTsKIAotLSAKMi4zNS4xLjYxNi5nMGJkY2JiNDQ2NC1n b29nCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGlu dXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRl YWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgt YXJtLWtlcm5lbAo=