From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47FE5CCA47C for ; Wed, 6 Jul 2022 22:01:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234699AbiGFWBd (ORCPT ); Wed, 6 Jul 2022 18:01:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44174 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234549AbiGFWBF (ORCPT ); Wed, 6 Jul 2022 18:01:05 -0400 Received: from mail-io1-xd4a.google.com (mail-io1-xd4a.google.com [IPv6:2607:f8b0:4864:20::d4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5C6EE2A941 for ; Wed, 6 Jul 2022 15:01:02 -0700 (PDT) Received: by mail-io1-xd4a.google.com with SMTP id r18-20020a6bd912000000b00675a6c915c9so8830791ioc.10 for ; Wed, 06 Jul 2022 15:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=XKxfVy7bUFxJEpcf3Ov7jSHxjMmYVb5gb0hRKtMPzZA=; b=KBFBaieE6U899pZedfVW186dkzrS93jrjdIku8VfT3EELG4tmwSu4pbA8t8KgFrkX7 PNLYIcjPRwCltpKZ41cpDa72lID2PMQjd0C5UzA1EP6Fozv39FS8efLoCNj0H5GROfCg QMeGKWc2c6xuBh73e/hz1kG0ddQk8uDEqQzdd1hwg6GKOeAe0e98I4co7JiaxOzZQyVa H3rcYT5ECNNWjJIqW6rJYkUeALUPQkQ6SiSCuxFVIHVt/LqtAYlBu4IfaEL80m1SvJmZ XAzuOW4B/+BDlzSPXhxDXR3iWNFF0evXZaEn2Xyp6i9pgpXVhqsXUcbDrh/yv+aznbGp vfZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=XKxfVy7bUFxJEpcf3Ov7jSHxjMmYVb5gb0hRKtMPzZA=; b=SKNTG+WcQJsvIGThYiL+oymyLTNvGt0kwQByQO7/p8gB2nEKVMF26rT/2kUT7JrZ7O G+Wr5X3Sc8fhA4DJAExp/XCJ9pqkSqGiwcZMG4wUIcUk1ZED2cTd0WBH/6DmLrg2ZhDW xdsH/kxsXMQhiqOnLQugyndN7Ikrt2eT6rF1KTfvZz8z2hUAGOeoZ4jQUf9C6B6h6JdD Y28ZUCFp6GIXbDclkIrLvnvLp9ukHHUE7enEm3vL01+9qfN7JxPtO2irgcV42IxKAXDF pmPDBFscKn4lHJAaHwKE5OfcL+HA/FDpwnXQP3LJQvG0fHB1FGreEtUFIdpF6G4ku3XI yQNw== X-Gm-Message-State: AJIora+/1iqtpK/P33+DOqdJYdx6N0rvjGf+jXLCvjOFrBs2PVn5jpWN fVf4WBVPKGlR10/fvXz6rHMOfHmQISw= X-Google-Smtp-Source: AGRyM1sRwdKObor4YUQFmQ9ta0XlmIxSt4ZUo1xITqK6sTS54jUHQ2ZrB8LtbtvMYYrpomN4w49bZKlSJRk= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:b89c:e10a:466e:cf7d]) (user=yuzhao job=sendgmr) by 2002:a05:6e02:148c:b0:2dc:38ae:5c6a with SMTP id n12-20020a056e02148c00b002dc38ae5c6amr2363805ilk.115.1657144861728; Wed, 06 Jul 2022 15:01:01 -0700 (PDT) Date: Wed, 6 Jul 2022 16:00:16 -0600 In-Reply-To: <20220706220022.968789-1-yuzhao@google.com> Message-Id: <20220706220022.968789-8-yuzhao@google.com> Mime-Version: 1.0 References: <20220706220022.968789-1-yuzhao@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v13 07/14] mm: multi-gen LRU: exploit locality in rmap From: Yu Zhao To: Andrew Morton Cc: Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Peter Zijlstra , Tejun Heo , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, page-reclaim@google.com, Yu Zhao , Barry Song , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Searching the rmap for PTEs mapping each page on an LRU list (to test and clear the accessed bit) can be expensive because pages from different VMAs (PA space) are not cache friendly to the rmap (VA space). For workloads mostly using mapped pages, searching the rmap can incur the highest CPU cost in the reclaim path. This patch exploits spatial locality to reduce the trips into the rmap. When shrink_page_list() walks the rmap and finds a young PTE, a new function lru_gen_look_around() scans at most BITS_PER_LONG-1 adjacent PTEs. On finding another young PTE, it clears the accessed bit and updates the gen counter of the page mapped by this PTE to (max_seq%MAX_NR_GENS)+1. Server benchmark results: Single workload: fio (buffered I/O): no change Single workload: memcached (anon): +[3, 5]% Ops/sec KB/sec patch1-6: 1106168.46 43025.04 patch1-7: 1147696.57 44640.29 Configurations: no change Client benchmark results: kswapd profiles: patch1-6 39.03% lzo1x_1_do_compress (real work) 18.47% page_vma_mapped_walk (overhead) 6.74% _raw_spin_unlock_irq 3.97% do_raw_spin_lock 2.49% ptep_clear_flush 2.48% anon_vma_interval_tree_iter_first 1.92% folio_referenced_one 1.88% __zram_bvec_write 1.48% memmove 1.31% vma_interval_tree_iter_next patch1-7 48.16% lzo1x_1_do_compress (real work) 8.20% page_vma_mapped_walk (overhead) 7.06% _raw_spin_unlock_irq 2.92% ptep_clear_flush 2.53% __zram_bvec_write 2.11% do_raw_spin_lock 2.02% memmove 1.93% lru_gen_look_around 1.56% free_unref_page_list 1.40% memset Configurations: no change Signed-off-by: Yu Zhao Acked-by: Barry Song Acked-by: Brian Geffon Acked-by: Jan Alexander Steffens (heftig) Acked-by: Oleksandr Natalenko Acked-by: Steven Barrett Acked-by: Suleiman Souhlal Tested-by: Daniel Byrne Tested-by: Donald Carr Tested-by: Holger Hoffst=C3=A4tte Tested-by: Konstantin Kharlamov Tested-by: Shuang Zhai Tested-by: Sofia Trinh Tested-by: Vaibhav Jain --- include/linux/memcontrol.h | 31 +++++++ include/linux/mm.h | 5 + include/linux/mmzone.h | 6 ++ mm/internal.h | 1 + mm/memcontrol.c | 1 + mm/rmap.c | 6 ++ mm/swap.c | 4 +- mm/vmscan.c | 184 +++++++++++++++++++++++++++++++++++++ 8 files changed, 236 insertions(+), 2 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index 9ecead1042b9..9d0fea17f9ef 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -444,6 +444,7 @@ static inline struct obj_cgroup *__folio_objcg(struct f= olio *folio) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() * * For a kmem folio a caller should hold an rcu read lock to protect memcg * associated with a kmem folio from being released. @@ -505,6 +506,7 @@ static inline struct mem_cgroup *folio_memcg_rcu(struct= folio *folio) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() * * For a kmem page a caller should hold an rcu read lock to protect memcg * associated with a kmem page from being released. @@ -950,6 +952,23 @@ void unlock_page_memcg(struct page *page); =20 void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val); =20 +/* try to stablize folio_memcg() for all the pages in a memcg */ +static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) +{ + rcu_read_lock(); + + if (mem_cgroup_disabled() || !atomic_read(&memcg->moving_account)) + return true; + + rcu_read_unlock(); + return false; +} + +static inline void mem_cgroup_unlock_pages(void) +{ + rcu_read_unlock(); +} + /* idx can be of type enum memcg_stat_item or node_stat_item */ static inline void mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) @@ -1401,6 +1420,18 @@ static inline void folio_memcg_unlock(struct folio *= folio) { } =20 +static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) +{ + /* to match folio_memcg_rcu() */ + rcu_read_lock(); + return true; +} + +static inline void mem_cgroup_unlock_pages(void) +{ + rcu_read_unlock(); +} + static inline void mem_cgroup_handle_over_high(void) { } diff --git a/include/linux/mm.h b/include/linux/mm.h index ed5393e5930d..981b2e447936 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1523,6 +1523,11 @@ static inline unsigned long folio_pfn(struct folio *= folio) return page_to_pfn(&folio->page); } =20 +static inline struct folio *pfn_folio(unsigned long pfn) +{ + return page_folio(pfn_to_page(pfn)); +} + static inline atomic_t *folio_pincount_ptr(struct folio *folio) { return &folio_page(folio, 1)->compound_pincount; diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 0d76222501ed..4fd7fc16eeb4 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -372,6 +372,7 @@ enum lruvec_flags { #ifndef __GENERATING_BOUNDS_H =20 struct lruvec; +struct page_vma_mapped_walk; =20 #define LRU_GEN_MASK ((BIT(LRU_GEN_WIDTH) - 1) << LRU_GEN_PGOFF) #define LRU_REFS_MASK ((BIT(LRU_REFS_WIDTH) - 1) << LRU_REFS_PGOFF) @@ -427,6 +428,7 @@ struct lru_gen_struct { }; =20 void lru_gen_init_lruvec(struct lruvec *lruvec); +void lru_gen_look_around(struct page_vma_mapped_walk *pvmw); =20 #ifdef CONFIG_MEMCG void lru_gen_init_memcg(struct mem_cgroup *memcg); @@ -439,6 +441,10 @@ static inline void lru_gen_init_lruvec(struct lruvec *= lruvec) { } =20 +static inline void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) +{ +} + #ifdef CONFIG_MEMCG static inline void lru_gen_init_memcg(struct mem_cgroup *memcg) { diff --git a/mm/internal.h b/mm/internal.h index c0f8fbe0445b..3d070582052e 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -83,6 +83,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf); void folio_rotate_reclaimable(struct folio *folio); bool __folio_end_writeback(struct folio *folio); void deactivate_file_folio(struct folio *folio); +void folio_activate(struct folio *folio); =20 void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *start_vm= a, unsigned long floor, unsigned long ceiling); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 7d58e8a73ece..743f8513f1c3 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2777,6 +2777,7 @@ static void commit_charge(struct folio *folio, struct= mem_cgroup *memcg) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() */ folio->memcg_data =3D (unsigned long)memcg; } diff --git a/mm/rmap.c b/mm/rmap.c index 5bcb334cd6f2..dce1a56b02f8 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -830,6 +830,12 @@ static bool folio_referenced_one(struct folio *folio, } =20 if (pvmw.pte) { + if (lru_gen_enabled() && pte_young(*pvmw.pte) && + !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ))) { + lru_gen_look_around(&pvmw); + referenced++; + } + if (ptep_clear_flush_young_notify(vma, address, pvmw.pte)) { /* diff --git a/mm/swap.c b/mm/swap.c index 67e7962fbacc..131fc76242a3 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -342,7 +342,7 @@ static bool need_activate_page_drain(int cpu) return pagevec_count(&per_cpu(lru_pvecs.activate_page, cpu)) !=3D 0; } =20 -static void folio_activate(struct folio *folio) +void folio_activate(struct folio *folio) { if (folio_test_lru(folio) && !folio_test_active(folio) && !folio_test_unevictable(folio)) { @@ -362,7 +362,7 @@ static inline void activate_page_drain(int cpu) { } =20 -static void folio_activate(struct folio *folio) +void folio_activate(struct folio *folio) { struct lruvec *lruvec; =20 diff --git a/mm/vmscan.c b/mm/vmscan.c index f768d61e7b85..ec786fc556a7 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1574,6 +1574,11 @@ static unsigned int shrink_page_list(struct list_hea= d *page_list, if (!sc->may_unmap && folio_mapped(folio)) goto keep_locked; =20 + /* folio_update_gen() tried to promote this page? */ + if (lru_gen_enabled() && !ignore_references && + folio_mapped(folio) && folio_test_referenced(folio)) + goto keep_locked; + /* * The number of dirty pages determines if a node is marked * reclaim_congested. kswapd will stall and start writing @@ -3161,6 +3166,29 @@ static bool positive_ctrl_err(struct ctrl_pos *sp, s= truct ctrl_pos *pv) * the aging *************************************************************************= *****/ =20 +/* promote pages accessed through page tables */ +static int folio_update_gen(struct folio *folio, int gen) +{ + unsigned long new_flags, old_flags =3D READ_ONCE(folio->flags); + + VM_WARN_ON_ONCE(gen >=3D MAX_NR_GENS); + VM_WARN_ON_ONCE(!rcu_read_lock_held()); + + do { + /* lru_gen_del_folio() has isolated this page? */ + if (!(old_flags & LRU_GEN_MASK)) { + /* for shrink_page_list() */ + new_flags =3D old_flags | BIT(PG_referenced); + continue; + } + + new_flags =3D old_flags & ~(LRU_GEN_MASK | LRU_REFS_MASK | LRU_REFS_FLAG= S); + new_flags |=3D (gen + 1UL) << LRU_GEN_PGOFF; + } while (!try_cmpxchg(&folio->flags, &old_flags, new_flags)); + + return ((old_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1; +} + /* protect pages accessed multiple times through file descriptors */ static int folio_inc_gen(struct lruvec *lruvec, struct folio *folio, bool = reclaiming) { @@ -3172,6 +3200,11 @@ static int folio_inc_gen(struct lruvec *lruvec, stru= ct folio *folio, bool reclai VM_WARN_ON_ONCE_FOLIO(!(old_flags & LRU_GEN_MASK), folio); =20 do { + new_gen =3D ((old_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1; + /* folio_update_gen() has promoted this page? */ + if (new_gen >=3D 0 && new_gen !=3D old_gen) + return new_gen; + new_gen =3D (old_gen + 1) % MAX_NR_GENS; =20 new_flags =3D old_flags & ~(LRU_GEN_MASK | LRU_REFS_MASK | LRU_REFS_FLAG= S); @@ -3186,6 +3219,43 @@ static int folio_inc_gen(struct lruvec *lruvec, stru= ct folio *folio, bool reclai return new_gen; } =20 +static unsigned long get_pte_pfn(pte_t pte, struct vm_area_struct *vma, un= signed long addr) +{ + unsigned long pfn =3D pte_pfn(pte); + + VM_WARN_ON_ONCE(addr < vma->vm_start || addr >=3D vma->vm_end); + + if (!pte_present(pte) || is_zero_pfn(pfn)) + return -1; + + if (WARN_ON_ONCE(pte_devmap(pte) || pte_special(pte))) + return -1; + + if (WARN_ON_ONCE(!pfn_valid(pfn))) + return -1; + + return pfn; +} + +static struct folio *get_pfn_folio(unsigned long pfn, struct mem_cgroup *m= emcg, + struct pglist_data *pgdat) +{ + struct folio *folio; + + /* try to avoid unnecessary memory loads */ + if (pfn < pgdat->node_start_pfn || pfn >=3D pgdat_end_pfn(pgdat)) + return NULL; + + folio =3D pfn_folio(pfn); + if (folio_nid(folio) !=3D pgdat->node_id) + return NULL; + + if (folio_memcg_rcu(folio) !=3D memcg) + return NULL; + + return folio; +} + static void inc_min_seq(struct lruvec *lruvec, int type) { struct lru_gen_struct *lrugen =3D &lruvec->lrugen; @@ -3387,6 +3457,114 @@ static void lru_gen_age_node(struct pglist_data *pg= dat, struct scan_control *sc) } while ((memcg =3D mem_cgroup_iter(NULL, memcg, NULL))); } =20 +/* + * This function exploits spatial locality when shrink_page_list() walks t= he + * rmap. It scans the adjacent PTEs of a young PTE and promotes hot pages. + */ +void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) +{ + int i; + pte_t *pte; + unsigned long start; + unsigned long end; + unsigned long addr; + unsigned long bitmap[BITS_TO_LONGS(MIN_LRU_BATCH)] =3D {}; + struct folio *folio =3D pfn_folio(pvmw->pfn); + struct mem_cgroup *memcg =3D folio_memcg(folio); + struct pglist_data *pgdat =3D folio_pgdat(folio); + struct lruvec *lruvec =3D mem_cgroup_lruvec(memcg, pgdat); + DEFINE_MAX_SEQ(lruvec); + int old_gen, new_gen =3D lru_gen_from_seq(max_seq); + + lockdep_assert_held(pvmw->ptl); + VM_WARN_ON_ONCE_FOLIO(folio_test_lru(folio), folio); + + if (spin_is_contended(pvmw->ptl)) + return; + + start =3D max(pvmw->address & PMD_MASK, pvmw->vma->vm_start); + end =3D min(pvmw->address | ~PMD_MASK, pvmw->vma->vm_end - 1) + 1; + + if (end - start > MIN_LRU_BATCH * PAGE_SIZE) { + if (pvmw->address - start < MIN_LRU_BATCH * PAGE_SIZE / 2) + end =3D start + MIN_LRU_BATCH * PAGE_SIZE; + else if (end - pvmw->address < MIN_LRU_BATCH * PAGE_SIZE / 2) + start =3D end - MIN_LRU_BATCH * PAGE_SIZE; + else { + start =3D pvmw->address - MIN_LRU_BATCH * PAGE_SIZE / 2; + end =3D pvmw->address + MIN_LRU_BATCH * PAGE_SIZE / 2; + } + } + + pte =3D pvmw->pte - (pvmw->address - start) / PAGE_SIZE; + + rcu_read_lock(); + arch_enter_lazy_mmu_mode(); + + for (i =3D 0, addr =3D start; addr !=3D end; i++, addr +=3D PAGE_SIZE) { + unsigned long pfn; + + pfn =3D get_pte_pfn(pte[i], pvmw->vma, addr); + if (pfn =3D=3D -1) + continue; + + if (!pte_young(pte[i])) + continue; + + folio =3D get_pfn_folio(pfn, memcg, pgdat); + if (!folio) + continue; + + if (!ptep_test_and_clear_young(pvmw->vma, addr, pte + i)) + continue; + + if (pte_dirty(pte[i]) && !folio_test_dirty(folio) && + !(folio_test_anon(folio) && folio_test_swapbacked(folio) && + !folio_test_swapcache(folio))) + folio_mark_dirty(folio); + + old_gen =3D folio_lru_gen(folio); + if (old_gen < 0) + folio_set_referenced(folio); + else if (old_gen !=3D new_gen) + __set_bit(i, bitmap); + } + + arch_leave_lazy_mmu_mode(); + rcu_read_unlock(); + + if (bitmap_weight(bitmap, MIN_LRU_BATCH) < PAGEVEC_SIZE) { + for_each_set_bit(i, bitmap, MIN_LRU_BATCH) { + folio =3D pfn_folio(pte_pfn(pte[i])); + folio_activate(folio); + } + return; + } + + /* folio_update_gen() requires stable folio_memcg() */ + if (!mem_cgroup_trylock_pages(memcg)) + return; + + spin_lock_irq(&lruvec->lru_lock); + new_gen =3D lru_gen_from_seq(lruvec->lrugen.max_seq); + + for_each_set_bit(i, bitmap, MIN_LRU_BATCH) { + folio =3D pfn_folio(pte_pfn(pte[i])); + if (folio_memcg_rcu(folio) !=3D memcg) + continue; + + old_gen =3D folio_update_gen(folio, new_gen); + if (old_gen < 0 || old_gen =3D=3D new_gen) + continue; + + lru_gen_update_size(lruvec, folio, old_gen, new_gen); + } + + spin_unlock_irq(&lruvec->lru_lock); + + mem_cgroup_unlock_pages(); +} + /*************************************************************************= ***** * the eviction *************************************************************************= *****/ @@ -3423,6 +3601,12 @@ static bool sort_folio(struct lruvec *lruvec, struct= folio *folio, int tier_idx) return true; } =20 + /* promoted */ + if (gen !=3D lru_gen_from_seq(lrugen->min_seq[type])) { + list_move(&folio->lru, &lrugen->lists[gen][type][zone]); + return true; + } + /* protected */ if (tier > tier_idx) { int hist =3D lru_hist_from_seq(lrugen->min_seq[type]); --=20 2.37.0.rc0.161.g10f37bed90-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3726AC43334 for ; Wed, 6 Jul 2022 23:01:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=q12f+gGQWAnMtw+Lyu1qgELU75/Y0ma6Awj60IjVh6I=; b=yL/hQcJqkgbbmKhXfCKwWc+wkX Lbi9Vltb9wHCU5GLfFWNukaPxqvqSHVJodGMilzF1fHW1t+GVa3R8XgTAL1XxReTulSFwHZens3b4 yfIF+Xcen0KARWB4cev6oJENhyj54gGCtCJrN6Q8kskf5sVgT6imyYmsKSGWuKQDLNOpbdbyaUHNV Cnp0fFsLYklAVUMUxt8Tw+dFn70X24Zmnm56Nv4m35an3OJ8/Jp67WE3wakNCdBbXcp5NmrdU0d38 wOHRpNoUyY4M7Ap2uHDhNKkUmUo04KYiJhh5aqD/oChdMOMXDYjWFsHAdq4C8nWDCY0Cw6OrlNoma SrVgMpjQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9Dzl-00CfS6-F9; Wed, 06 Jul 2022 22:59:45 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9Dzj-00CfQV-BX for linux-arm-kernel@bombadil.infradead.org; Wed, 06 Jul 2022 22:59:43 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :Cc:To:From:Subject:References:Mime-Version:Message-Id:In-Reply-To:Date: Sender:Reply-To:Content-ID:Content-Description; bh=XKxfVy7bUFxJEpcf3Ov7jSHxjMmYVb5gb0hRKtMPzZA=; b=qUyQNgyIO7raiJsqxDq1q2CR5O Dz6zfmZwVfmORVftZSA0mUzDPKJNKgBl8kNqHRo1U9wlFSPj9sKQt41K2WAWf46aQLzBD6dsDutgD Rn8Fd4SMkfLCF1SdBXUbVgorBdgrN+gK/Ew10nxlpQOlX1gZ90ZJEUSdqdtKF1HovI8VT3VZed5o9 7kPL5VR4DIx5+FEQLULOL4NkfpFO1lkLPtjATPOL8BuUebqXnP68bYsO0W1rWokGNUZ56uwW5wmXf TmoQ4m0FZrUV2PrYFoMbAGYbCnT7UukhjyGjgfXFy05sjYZw2mKZrLWwTgQ4S8bemdt7ROmKz/JO4 QjRCXF7g==; Received: from mail-io1-xd49.google.com ([2607:f8b0:4864:20::d49]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1o9D53-000zVw-34 for linux-arm-kernel@lists.infradead.org; Wed, 06 Jul 2022 22:01:14 +0000 Received: by mail-io1-xd49.google.com with SMTP id y22-20020a056602215600b00673b11a9cd5so8687679ioy.7 for ; Wed, 06 Jul 2022 15:01:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=XKxfVy7bUFxJEpcf3Ov7jSHxjMmYVb5gb0hRKtMPzZA=; b=KBFBaieE6U899pZedfVW186dkzrS93jrjdIku8VfT3EELG4tmwSu4pbA8t8KgFrkX7 PNLYIcjPRwCltpKZ41cpDa72lID2PMQjd0C5UzA1EP6Fozv39FS8efLoCNj0H5GROfCg QMeGKWc2c6xuBh73e/hz1kG0ddQk8uDEqQzdd1hwg6GKOeAe0e98I4co7JiaxOzZQyVa H3rcYT5ECNNWjJIqW6rJYkUeALUPQkQ6SiSCuxFVIHVt/LqtAYlBu4IfaEL80m1SvJmZ XAzuOW4B/+BDlzSPXhxDXR3iWNFF0evXZaEn2Xyp6i9pgpXVhqsXUcbDrh/yv+aznbGp vfZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=XKxfVy7bUFxJEpcf3Ov7jSHxjMmYVb5gb0hRKtMPzZA=; b=IVZFaQ0FquifFuwKin4zR4BMNO67t15pxWJIeU29OI8hQrYJVkZ6/WYwpRPnh6bRgN dl5U4PFjmnh3CZ1dhi+zTjPL0mTn3yfKrc5A0zV+ixW3Oa4R2QFZiffDTijy0aHf92v8 ey1/nfFk4fo2dti7iZYLL3rvkfM1NJUQNMzDqqFjFd+jzkOXtZ7w8wklMObiFOgqaDTp zqRYQgGQJlwM3OLGqy4MOWFzu3nAAhJEs9jYtCnmCWttiZ+riAWHGfmT1uD+Q6kGw+ls 3rvfBb5q1CGpQvN4UZ2j1qAI8tRuuKW6ZmV9y6vejAXMlwhZc8Cq50R/nAB0531Q29Qd MGUQ== X-Gm-Message-State: AJIora9efjt70LKhCww5JxSXwgbMQ1NrRof/lSeq5o1+KYHs7qnsBYdB 1ojyvw1jP3WnJgEKqz+FVakMKO97ins= X-Google-Smtp-Source: AGRyM1sRwdKObor4YUQFmQ9ta0XlmIxSt4ZUo1xITqK6sTS54jUHQ2ZrB8LtbtvMYYrpomN4w49bZKlSJRk= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:b89c:e10a:466e:cf7d]) (user=yuzhao job=sendgmr) by 2002:a05:6e02:148c:b0:2dc:38ae:5c6a with SMTP id n12-20020a056e02148c00b002dc38ae5c6amr2363805ilk.115.1657144861728; Wed, 06 Jul 2022 15:01:01 -0700 (PDT) Date: Wed, 6 Jul 2022 16:00:16 -0600 In-Reply-To: <20220706220022.968789-1-yuzhao@google.com> Message-Id: <20220706220022.968789-8-yuzhao@google.com> Mime-Version: 1.0 References: <20220706220022.968789-1-yuzhao@google.com> X-Mailer: git-send-email 2.37.0.rc0.161.g10f37bed90-goog Subject: [PATCH v13 07/14] mm: multi-gen LRU: exploit locality in rmap From: Yu Zhao To: Andrew Morton Cc: Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Peter Zijlstra , Tejun Heo , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, page-reclaim@google.com, Yu Zhao , Barry Song , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220706_230111_925434_DC4481F9 X-CRM114-Status: GOOD ( 24.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org U2VhcmNoaW5nIHRoZSBybWFwIGZvciBQVEVzIG1hcHBpbmcgZWFjaCBwYWdlIG9uIGFuIExSVSBs aXN0ICh0byB0ZXN0CmFuZCBjbGVhciB0aGUgYWNjZXNzZWQgYml0KSBjYW4gYmUgZXhwZW5zaXZl IGJlY2F1c2UgcGFnZXMgZnJvbQpkaWZmZXJlbnQgVk1BcyAoUEEgc3BhY2UpIGFyZSBub3QgY2Fj aGUgZnJpZW5kbHkgdG8gdGhlIHJtYXAgKFZBCnNwYWNlKS4gRm9yIHdvcmtsb2FkcyBtb3N0bHkg dXNpbmcgbWFwcGVkIHBhZ2VzLCBzZWFyY2hpbmcgdGhlIHJtYXAKY2FuIGluY3VyIHRoZSBoaWdo ZXN0IENQVSBjb3N0IGluIHRoZSByZWNsYWltIHBhdGguCgpUaGlzIHBhdGNoIGV4cGxvaXRzIHNw YXRpYWwgbG9jYWxpdHkgdG8gcmVkdWNlIHRoZSB0cmlwcyBpbnRvIHRoZQpybWFwLiBXaGVuIHNo cmlua19wYWdlX2xpc3QoKSB3YWxrcyB0aGUgcm1hcCBhbmQgZmluZHMgYSB5b3VuZyBQVEUsIGEK bmV3IGZ1bmN0aW9uIGxydV9nZW5fbG9va19hcm91bmQoKSBzY2FucyBhdCBtb3N0IEJJVFNfUEVS X0xPTkctMQphZGphY2VudCBQVEVzLiBPbiBmaW5kaW5nIGFub3RoZXIgeW91bmcgUFRFLCBpdCBj bGVhcnMgdGhlIGFjY2Vzc2VkCmJpdCBhbmQgdXBkYXRlcyB0aGUgZ2VuIGNvdW50ZXIgb2YgdGhl IHBhZ2UgbWFwcGVkIGJ5IHRoaXMgUFRFIHRvCihtYXhfc2VxJU1BWF9OUl9HRU5TKSsxLgoKU2Vy dmVyIGJlbmNobWFyayByZXN1bHRzOgogIFNpbmdsZSB3b3JrbG9hZDoKICAgIGZpbyAoYnVmZmVy ZWQgSS9PKTogbm8gY2hhbmdlCgogIFNpbmdsZSB3b3JrbG9hZDoKICAgIG1lbWNhY2hlZCAoYW5v bik6ICtbMywgNV0lCiAgICAgICAgICAgICAgICBPcHMvc2VjICAgICAgS0Ivc2VjCiAgICAgIHBh dGNoMS02OiAxMTA2MTY4LjQ2ICAgNDMwMjUuMDQKICAgICAgcGF0Y2gxLTc6IDExNDc2OTYuNTcg ICA0NDY0MC4yOQoKICBDb25maWd1cmF0aW9uczoKICAgIG5vIGNoYW5nZQoKQ2xpZW50IGJlbmNo bWFyayByZXN1bHRzOgogIGtzd2FwZCBwcm9maWxlczoKICAgIHBhdGNoMS02CiAgICAgIDM5LjAz JSAgbHpvMXhfMV9kb19jb21wcmVzcyAocmVhbCB3b3JrKQogICAgICAxOC40NyUgIHBhZ2Vfdm1h X21hcHBlZF93YWxrIChvdmVyaGVhZCkKICAgICAgIDYuNzQlICBfcmF3X3NwaW5fdW5sb2NrX2ly cQogICAgICAgMy45NyUgIGRvX3Jhd19zcGluX2xvY2sKICAgICAgIDIuNDklICBwdGVwX2NsZWFy X2ZsdXNoCiAgICAgICAyLjQ4JSAgYW5vbl92bWFfaW50ZXJ2YWxfdHJlZV9pdGVyX2ZpcnN0CiAg ICAgICAxLjkyJSAgZm9saW9fcmVmZXJlbmNlZF9vbmUKICAgICAgIDEuODglICBfX3pyYW1fYnZl Y193cml0ZQogICAgICAgMS40OCUgIG1lbW1vdmUKICAgICAgIDEuMzElICB2bWFfaW50ZXJ2YWxf dHJlZV9pdGVyX25leHQKCiAgICBwYXRjaDEtNwogICAgICA0OC4xNiUgIGx6bzF4XzFfZG9fY29t cHJlc3MgKHJlYWwgd29yaykKICAgICAgIDguMjAlICBwYWdlX3ZtYV9tYXBwZWRfd2FsayAob3Zl cmhlYWQpCiAgICAgICA3LjA2JSAgX3Jhd19zcGluX3VubG9ja19pcnEKICAgICAgIDIuOTIlICBw dGVwX2NsZWFyX2ZsdXNoCiAgICAgICAyLjUzJSAgX196cmFtX2J2ZWNfd3JpdGUKICAgICAgIDIu MTElICBkb19yYXdfc3Bpbl9sb2NrCiAgICAgICAyLjAyJSAgbWVtbW92ZQogICAgICAgMS45MyUg IGxydV9nZW5fbG9va19hcm91bmQKICAgICAgIDEuNTYlICBmcmVlX3VucmVmX3BhZ2VfbGlzdAog ICAgICAgMS40MCUgIG1lbXNldAoKICBDb25maWd1cmF0aW9uczoKICAgIG5vIGNoYW5nZQoKU2ln bmVkLW9mZi1ieTogWXUgWmhhbyA8eXV6aGFvQGdvb2dsZS5jb20+CkFja2VkLWJ5OiBCYXJyeSBT b25nIDxiYW9odWFAa2VybmVsLm9yZz4KQWNrZWQtYnk6IEJyaWFuIEdlZmZvbiA8YmdlZmZvbkBn b29nbGUuY29tPgpBY2tlZC1ieTogSmFuIEFsZXhhbmRlciBTdGVmZmVucyAoaGVmdGlnKSA8aGVm dGlnQGFyY2hsaW51eC5vcmc+CkFja2VkLWJ5OiBPbGVrc2FuZHIgTmF0YWxlbmtvIDxvbGVrc2Fu ZHJAbmF0YWxlbmtvLm5hbWU+CkFja2VkLWJ5OiBTdGV2ZW4gQmFycmV0dCA8c3RldmVuQGxpcXVv cml4Lm5ldD4KQWNrZWQtYnk6IFN1bGVpbWFuIFNvdWhsYWwgPHN1bGVpbWFuQGdvb2dsZS5jb20+ ClRlc3RlZC1ieTogRGFuaWVsIEJ5cm5lIDxkamJ5cm5lQG10dS5lZHU+ClRlc3RlZC1ieTogRG9u YWxkIENhcnIgPGRAY2hhb3MtcmVpbnMuY29tPgpUZXN0ZWQtYnk6IEhvbGdlciBIb2Zmc3TDpHR0 ZSA8aG9sZ2VyQGFwcGxpZWQtYXN5bmNocm9ueS5jb20+ClRlc3RlZC1ieTogS29uc3RhbnRpbiBL aGFybGFtb3YgPEhpLUFuZ2VsQHlhbmRleC5ydT4KVGVzdGVkLWJ5OiBTaHVhbmcgWmhhaSA8c3po YWkyQGNzLnJvY2hlc3Rlci5lZHU+ClRlc3RlZC1ieTogU29maWEgVHJpbmggPHNvZmlhLnRyaW5o QGVkaS53b3Jrcz4KVGVzdGVkLWJ5OiBWYWliaGF2IEphaW4gPHZhaWJoYXZAbGludXguaWJtLmNv bT4KLS0tCiBpbmNsdWRlL2xpbnV4L21lbWNvbnRyb2wuaCB8ICAzMSArKysrKysrCiBpbmNsdWRl L2xpbnV4L21tLmggICAgICAgICB8ICAgNSArCiBpbmNsdWRlL2xpbnV4L21tem9uZS5oICAgICB8 ICAgNiArKwogbW0vaW50ZXJuYWwuaCAgICAgICAgICAgICAgfCAgIDEgKwogbW0vbWVtY29udHJv bC5jICAgICAgICAgICAgfCAgIDEgKwogbW0vcm1hcC5jICAgICAgICAgICAgICAgICAgfCAgIDYg KysKIG1tL3N3YXAuYyAgICAgICAgICAgICAgICAgIHwgICA0ICstCiBtbS92bXNjYW4uYyAgICAg ICAgICAgICAgICB8IDE4NCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrCiA4 IGZpbGVzIGNoYW5nZWQsIDIzNiBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQoKZGlmZiAt LWdpdCBhL2luY2x1ZGUvbGludXgvbWVtY29udHJvbC5oIGIvaW5jbHVkZS9saW51eC9tZW1jb250 cm9sLmgKaW5kZXggOWVjZWFkMTA0MmI5Li45ZDBmZWExN2Y5ZWYgMTAwNjQ0Ci0tLSBhL2luY2x1 ZGUvbGludXgvbWVtY29udHJvbC5oCisrKyBiL2luY2x1ZGUvbGludXgvbWVtY29udHJvbC5oCkBA IC00NDQsNiArNDQ0LDcgQEAgc3RhdGljIGlubGluZSBzdHJ1Y3Qgb2JqX2Nncm91cCAqX19mb2xp b19vYmpjZyhzdHJ1Y3QgZm9saW8gKmZvbGlvKQogICogLSBMUlUgaXNvbGF0aW9uCiAgKiAtIGxv Y2tfcGFnZV9tZW1jZygpCiAgKiAtIGV4Y2x1c2l2ZSByZWZlcmVuY2UKKyAqIC0gbWVtX2Nncm91 cF90cnlsb2NrX3BhZ2VzKCkKICAqCiAgKiBGb3IgYSBrbWVtIGZvbGlvIGEgY2FsbGVyIHNob3Vs ZCBob2xkIGFuIHJjdSByZWFkIGxvY2sgdG8gcHJvdGVjdCBtZW1jZwogICogYXNzb2NpYXRlZCB3 aXRoIGEga21lbSBmb2xpbyBmcm9tIGJlaW5nIHJlbGVhc2VkLgpAQCAtNTA1LDYgKzUwNiw3IEBA IHN0YXRpYyBpbmxpbmUgc3RydWN0IG1lbV9jZ3JvdXAgKmZvbGlvX21lbWNnX3JjdShzdHJ1Y3Qg Zm9saW8gKmZvbGlvKQogICogLSBMUlUgaXNvbGF0aW9uCiAgKiAtIGxvY2tfcGFnZV9tZW1jZygp CiAgKiAtIGV4Y2x1c2l2ZSByZWZlcmVuY2UKKyAqIC0gbWVtX2Nncm91cF90cnlsb2NrX3BhZ2Vz KCkKICAqCiAgKiBGb3IgYSBrbWVtIHBhZ2UgYSBjYWxsZXIgc2hvdWxkIGhvbGQgYW4gcmN1IHJl YWQgbG9jayB0byBwcm90ZWN0IG1lbWNnCiAgKiBhc3NvY2lhdGVkIHdpdGggYSBrbWVtIHBhZ2Ug ZnJvbSBiZWluZyByZWxlYXNlZC4KQEAgLTk1MCw2ICs5NTIsMjMgQEAgdm9pZCB1bmxvY2tfcGFn ZV9tZW1jZyhzdHJ1Y3QgcGFnZSAqcGFnZSk7CiAKIHZvaWQgX19tb2RfbWVtY2dfc3RhdGUoc3Ry dWN0IG1lbV9jZ3JvdXAgKm1lbWNnLCBpbnQgaWR4LCBpbnQgdmFsKTsKIAorLyogdHJ5IHRvIHN0 YWJsaXplIGZvbGlvX21lbWNnKCkgZm9yIGFsbCB0aGUgcGFnZXMgaW4gYSBtZW1jZyAqLworc3Rh dGljIGlubGluZSBib29sIG1lbV9jZ3JvdXBfdHJ5bG9ja19wYWdlcyhzdHJ1Y3QgbWVtX2Nncm91 cCAqbWVtY2cpCit7CisJcmN1X3JlYWRfbG9jaygpOworCisJaWYgKG1lbV9jZ3JvdXBfZGlzYWJs ZWQoKSB8fCAhYXRvbWljX3JlYWQoJm1lbWNnLT5tb3ZpbmdfYWNjb3VudCkpCisJCXJldHVybiB0 cnVlOworCisJcmN1X3JlYWRfdW5sb2NrKCk7CisJcmV0dXJuIGZhbHNlOworfQorCitzdGF0aWMg aW5saW5lIHZvaWQgbWVtX2Nncm91cF91bmxvY2tfcGFnZXModm9pZCkKK3sKKwlyY3VfcmVhZF91 bmxvY2soKTsKK30KKwogLyogaWR4IGNhbiBiZSBvZiB0eXBlIGVudW0gbWVtY2dfc3RhdF9pdGVt IG9yIG5vZGVfc3RhdF9pdGVtICovCiBzdGF0aWMgaW5saW5lIHZvaWQgbW9kX21lbWNnX3N0YXRl KHN0cnVjdCBtZW1fY2dyb3VwICptZW1jZywKIAkJCQkgICBpbnQgaWR4LCBpbnQgdmFsKQpAQCAt MTQwMSw2ICsxNDIwLDE4IEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBmb2xpb19tZW1jZ191bmxvY2so c3RydWN0IGZvbGlvICpmb2xpbykKIHsKIH0KIAorc3RhdGljIGlubGluZSBib29sIG1lbV9jZ3Jv dXBfdHJ5bG9ja19wYWdlcyhzdHJ1Y3QgbWVtX2Nncm91cCAqbWVtY2cpCit7CisJLyogdG8gbWF0 Y2ggZm9saW9fbWVtY2dfcmN1KCkgKi8KKwlyY3VfcmVhZF9sb2NrKCk7CisJcmV0dXJuIHRydWU7 Cit9CisKK3N0YXRpYyBpbmxpbmUgdm9pZCBtZW1fY2dyb3VwX3VubG9ja19wYWdlcyh2b2lkKQor eworCXJjdV9yZWFkX3VubG9jaygpOworfQorCiBzdGF0aWMgaW5saW5lIHZvaWQgbWVtX2Nncm91 cF9oYW5kbGVfb3Zlcl9oaWdoKHZvaWQpCiB7CiB9CmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4 L21tLmggYi9pbmNsdWRlL2xpbnV4L21tLmgKaW5kZXggZWQ1MzkzZTU5MzBkLi45ODFiMmU0NDc5 MzYgMTAwNjQ0Ci0tLSBhL2luY2x1ZGUvbGludXgvbW0uaAorKysgYi9pbmNsdWRlL2xpbnV4L21t LmgKQEAgLTE1MjMsNiArMTUyMywxMSBAQCBzdGF0aWMgaW5saW5lIHVuc2lnbmVkIGxvbmcgZm9s aW9fcGZuKHN0cnVjdCBmb2xpbyAqZm9saW8pCiAJcmV0dXJuIHBhZ2VfdG9fcGZuKCZmb2xpby0+ cGFnZSk7CiB9CiAKK3N0YXRpYyBpbmxpbmUgc3RydWN0IGZvbGlvICpwZm5fZm9saW8odW5zaWdu ZWQgbG9uZyBwZm4pCit7CisJcmV0dXJuIHBhZ2VfZm9saW8ocGZuX3RvX3BhZ2UocGZuKSk7Cit9 CisKIHN0YXRpYyBpbmxpbmUgYXRvbWljX3QgKmZvbGlvX3BpbmNvdW50X3B0cihzdHJ1Y3QgZm9s aW8gKmZvbGlvKQogewogCXJldHVybiAmZm9saW9fcGFnZShmb2xpbywgMSktPmNvbXBvdW5kX3Bp bmNvdW50OwpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tbXpvbmUuaCBiL2luY2x1ZGUvbGlu dXgvbW16b25lLmgKaW5kZXggMGQ3NjIyMjUwMWVkLi40ZmQ3ZmMxNmVlYjQgMTAwNjQ0Ci0tLSBh L2luY2x1ZGUvbGludXgvbW16b25lLmgKKysrIGIvaW5jbHVkZS9saW51eC9tbXpvbmUuaApAQCAt MzcyLDYgKzM3Miw3IEBAIGVudW0gbHJ1dmVjX2ZsYWdzIHsKICNpZm5kZWYgX19HRU5FUkFUSU5H X0JPVU5EU19ICiAKIHN0cnVjdCBscnV2ZWM7CitzdHJ1Y3QgcGFnZV92bWFfbWFwcGVkX3dhbGs7 CiAKICNkZWZpbmUgTFJVX0dFTl9NQVNLCQkoKEJJVChMUlVfR0VOX1dJRFRIKSAtIDEpIDw8IExS VV9HRU5fUEdPRkYpCiAjZGVmaW5lIExSVV9SRUZTX01BU0sJCSgoQklUKExSVV9SRUZTX1dJRFRI KSAtIDEpIDw8IExSVV9SRUZTX1BHT0ZGKQpAQCAtNDI3LDYgKzQyOCw3IEBAIHN0cnVjdCBscnVf Z2VuX3N0cnVjdCB7CiB9OwogCiB2b2lkIGxydV9nZW5faW5pdF9scnV2ZWMoc3RydWN0IGxydXZl YyAqbHJ1dmVjKTsKK3ZvaWQgbHJ1X2dlbl9sb29rX2Fyb3VuZChzdHJ1Y3QgcGFnZV92bWFfbWFw cGVkX3dhbGsgKnB2bXcpOwogCiAjaWZkZWYgQ09ORklHX01FTUNHCiB2b2lkIGxydV9nZW5faW5p dF9tZW1jZyhzdHJ1Y3QgbWVtX2Nncm91cCAqbWVtY2cpOwpAQCAtNDM5LDYgKzQ0MSwxMCBAQCBz dGF0aWMgaW5saW5lIHZvaWQgbHJ1X2dlbl9pbml0X2xydXZlYyhzdHJ1Y3QgbHJ1dmVjICpscnV2 ZWMpCiB7CiB9CiAKK3N0YXRpYyBpbmxpbmUgdm9pZCBscnVfZ2VuX2xvb2tfYXJvdW5kKHN0cnVj dCBwYWdlX3ZtYV9tYXBwZWRfd2FsayAqcHZtdykKK3sKK30KKwogI2lmZGVmIENPTkZJR19NRU1D Rwogc3RhdGljIGlubGluZSB2b2lkIGxydV9nZW5faW5pdF9tZW1jZyhzdHJ1Y3QgbWVtX2Nncm91 cCAqbWVtY2cpCiB7CmRpZmYgLS1naXQgYS9tbS9pbnRlcm5hbC5oIGIvbW0vaW50ZXJuYWwuaApp bmRleCBjMGY4ZmJlMDQ0NWIuLjNkMDcwNTgyMDUyZSAxMDA2NDQKLS0tIGEvbW0vaW50ZXJuYWwu aAorKysgYi9tbS9pbnRlcm5hbC5oCkBAIC04Myw2ICs4Myw3IEBAIHZtX2ZhdWx0X3QgZG9fc3dh cF9wYWdlKHN0cnVjdCB2bV9mYXVsdCAqdm1mKTsKIHZvaWQgZm9saW9fcm90YXRlX3JlY2xhaW1h YmxlKHN0cnVjdCBmb2xpbyAqZm9saW8pOwogYm9vbCBfX2ZvbGlvX2VuZF93cml0ZWJhY2soc3Ry dWN0IGZvbGlvICpmb2xpbyk7CiB2b2lkIGRlYWN0aXZhdGVfZmlsZV9mb2xpbyhzdHJ1Y3QgZm9s aW8gKmZvbGlvKTsKK3ZvaWQgZm9saW9fYWN0aXZhdGUoc3RydWN0IGZvbGlvICpmb2xpbyk7CiAK IHZvaWQgZnJlZV9wZ3RhYmxlcyhzdHJ1Y3QgbW11X2dhdGhlciAqdGxiLCBzdHJ1Y3Qgdm1fYXJl YV9zdHJ1Y3QgKnN0YXJ0X3ZtYSwKIAkJdW5zaWduZWQgbG9uZyBmbG9vciwgdW5zaWduZWQgbG9u ZyBjZWlsaW5nKTsKZGlmZiAtLWdpdCBhL21tL21lbWNvbnRyb2wuYyBiL21tL21lbWNvbnRyb2wu YwppbmRleCA3ZDU4ZThhNzNlY2UuLjc0M2Y4NTEzZjFjMyAxMDA2NDQKLS0tIGEvbW0vbWVtY29u dHJvbC5jCisrKyBiL21tL21lbWNvbnRyb2wuYwpAQCAtMjc3Nyw2ICsyNzc3LDcgQEAgc3RhdGlj IHZvaWQgY29tbWl0X2NoYXJnZShzdHJ1Y3QgZm9saW8gKmZvbGlvLCBzdHJ1Y3QgbWVtX2Nncm91 cCAqbWVtY2cpCiAJICogLSBMUlUgaXNvbGF0aW9uCiAJICogLSBsb2NrX3BhZ2VfbWVtY2coKQog CSAqIC0gZXhjbHVzaXZlIHJlZmVyZW5jZQorCSAqIC0gbWVtX2Nncm91cF90cnlsb2NrX3BhZ2Vz KCkKIAkgKi8KIAlmb2xpby0+bWVtY2dfZGF0YSA9ICh1bnNpZ25lZCBsb25nKW1lbWNnOwogfQpk aWZmIC0tZ2l0IGEvbW0vcm1hcC5jIGIvbW0vcm1hcC5jCmluZGV4IDViY2IzMzRjZDZmMi4uZGNl MWE1NmIwMmY4IDEwMDY0NAotLS0gYS9tbS9ybWFwLmMKKysrIGIvbW0vcm1hcC5jCkBAIC04MzAs NiArODMwLDEyIEBAIHN0YXRpYyBib29sIGZvbGlvX3JlZmVyZW5jZWRfb25lKHN0cnVjdCBmb2xp byAqZm9saW8sCiAJCX0KIAogCQlpZiAocHZtdy5wdGUpIHsKKwkJCWlmIChscnVfZ2VuX2VuYWJs ZWQoKSAmJiBwdGVfeW91bmcoKnB2bXcucHRlKSAmJgorCQkJICAgICEodm1hLT52bV9mbGFncyAm IChWTV9TRVFfUkVBRCB8IFZNX1JBTkRfUkVBRCkpKSB7CisJCQkJbHJ1X2dlbl9sb29rX2Fyb3Vu ZCgmcHZtdyk7CisJCQkJcmVmZXJlbmNlZCsrOworCQkJfQorCiAJCQlpZiAocHRlcF9jbGVhcl9m bHVzaF95b3VuZ19ub3RpZnkodm1hLCBhZGRyZXNzLAogCQkJCQkJcHZtdy5wdGUpKSB7CiAJCQkJ LyoKZGlmZiAtLWdpdCBhL21tL3N3YXAuYyBiL21tL3N3YXAuYwppbmRleCA2N2U3OTYyZmJhY2Mu LjEzMWZjNzYyNDJhMyAxMDA2NDQKLS0tIGEvbW0vc3dhcC5jCisrKyBiL21tL3N3YXAuYwpAQCAt MzQyLDcgKzM0Miw3IEBAIHN0YXRpYyBib29sIG5lZWRfYWN0aXZhdGVfcGFnZV9kcmFpbihpbnQg Y3B1KQogCXJldHVybiBwYWdldmVjX2NvdW50KCZwZXJfY3B1KGxydV9wdmVjcy5hY3RpdmF0ZV9w YWdlLCBjcHUpKSAhPSAwOwogfQogCi1zdGF0aWMgdm9pZCBmb2xpb19hY3RpdmF0ZShzdHJ1Y3Qg Zm9saW8gKmZvbGlvKQordm9pZCBmb2xpb19hY3RpdmF0ZShzdHJ1Y3QgZm9saW8gKmZvbGlvKQog ewogCWlmIChmb2xpb190ZXN0X2xydShmb2xpbykgJiYgIWZvbGlvX3Rlc3RfYWN0aXZlKGZvbGlv KSAmJgogCSAgICAhZm9saW9fdGVzdF91bmV2aWN0YWJsZShmb2xpbykpIHsKQEAgLTM2Miw3ICsz NjIsNyBAQCBzdGF0aWMgaW5saW5lIHZvaWQgYWN0aXZhdGVfcGFnZV9kcmFpbihpbnQgY3B1KQog ewogfQogCi1zdGF0aWMgdm9pZCBmb2xpb19hY3RpdmF0ZShzdHJ1Y3QgZm9saW8gKmZvbGlvKQor dm9pZCBmb2xpb19hY3RpdmF0ZShzdHJ1Y3QgZm9saW8gKmZvbGlvKQogewogCXN0cnVjdCBscnV2 ZWMgKmxydXZlYzsKIApkaWZmIC0tZ2l0IGEvbW0vdm1zY2FuLmMgYi9tbS92bXNjYW4uYwppbmRl eCBmNzY4ZDYxZTdiODUuLmVjNzg2ZmM1NTZhNyAxMDA2NDQKLS0tIGEvbW0vdm1zY2FuLmMKKysr IGIvbW0vdm1zY2FuLmMKQEAgLTE1NzQsNiArMTU3NCwxMSBAQCBzdGF0aWMgdW5zaWduZWQgaW50 IHNocmlua19wYWdlX2xpc3Qoc3RydWN0IGxpc3RfaGVhZCAqcGFnZV9saXN0LAogCQlpZiAoIXNj LT5tYXlfdW5tYXAgJiYgZm9saW9fbWFwcGVkKGZvbGlvKSkKIAkJCWdvdG8ga2VlcF9sb2NrZWQ7 CiAKKwkJLyogZm9saW9fdXBkYXRlX2dlbigpIHRyaWVkIHRvIHByb21vdGUgdGhpcyBwYWdlPyAq LworCQlpZiAobHJ1X2dlbl9lbmFibGVkKCkgJiYgIWlnbm9yZV9yZWZlcmVuY2VzICYmCisJCSAg ICBmb2xpb19tYXBwZWQoZm9saW8pICYmIGZvbGlvX3Rlc3RfcmVmZXJlbmNlZChmb2xpbykpCisJ CQlnb3RvIGtlZXBfbG9ja2VkOworCiAJCS8qCiAJCSAqIFRoZSBudW1iZXIgb2YgZGlydHkgcGFn ZXMgZGV0ZXJtaW5lcyBpZiBhIG5vZGUgaXMgbWFya2VkCiAJCSAqIHJlY2xhaW1fY29uZ2VzdGVk LiBrc3dhcGQgd2lsbCBzdGFsbCBhbmQgc3RhcnQgd3JpdGluZwpAQCAtMzE2MSw2ICszMTY2LDI5 IEBAIHN0YXRpYyBib29sIHBvc2l0aXZlX2N0cmxfZXJyKHN0cnVjdCBjdHJsX3BvcyAqc3AsIHN0 cnVjdCBjdHJsX3BvcyAqcHYpCiAgKiAgICAgICAgICAgICAgICAgICAgICAgICAgdGhlIGFnaW5n CiAgKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqLwogCisvKiBwcm9tb3RlIHBhZ2VzIGFjY2Vzc2VkIHRo cm91Z2ggcGFnZSB0YWJsZXMgKi8KK3N0YXRpYyBpbnQgZm9saW9fdXBkYXRlX2dlbihzdHJ1Y3Qg Zm9saW8gKmZvbGlvLCBpbnQgZ2VuKQoreworCXVuc2lnbmVkIGxvbmcgbmV3X2ZsYWdzLCBvbGRf ZmxhZ3MgPSBSRUFEX09OQ0UoZm9saW8tPmZsYWdzKTsKKworCVZNX1dBUk5fT05fT05DRShnZW4g Pj0gTUFYX05SX0dFTlMpOworCVZNX1dBUk5fT05fT05DRSghcmN1X3JlYWRfbG9ja19oZWxkKCkp OworCisJZG8geworCQkvKiBscnVfZ2VuX2RlbF9mb2xpbygpIGhhcyBpc29sYXRlZCB0aGlzIHBh Z2U/ICovCisJCWlmICghKG9sZF9mbGFncyAmIExSVV9HRU5fTUFTSykpIHsKKwkJCS8qIGZvciBz aHJpbmtfcGFnZV9saXN0KCkgKi8KKwkJCW5ld19mbGFncyA9IG9sZF9mbGFncyB8IEJJVChQR19y ZWZlcmVuY2VkKTsKKwkJCWNvbnRpbnVlOworCQl9CisKKwkJbmV3X2ZsYWdzID0gb2xkX2ZsYWdz ICYgfihMUlVfR0VOX01BU0sgfCBMUlVfUkVGU19NQVNLIHwgTFJVX1JFRlNfRkxBR1MpOworCQlu ZXdfZmxhZ3MgfD0gKGdlbiArIDFVTCkgPDwgTFJVX0dFTl9QR09GRjsKKwl9IHdoaWxlICghdHJ5 X2NtcHhjaGcoJmZvbGlvLT5mbGFncywgJm9sZF9mbGFncywgbmV3X2ZsYWdzKSk7CisKKwlyZXR1 cm4gKChvbGRfZmxhZ3MgJiBMUlVfR0VOX01BU0spID4+IExSVV9HRU5fUEdPRkYpIC0gMTsKK30K KwogLyogcHJvdGVjdCBwYWdlcyBhY2Nlc3NlZCBtdWx0aXBsZSB0aW1lcyB0aHJvdWdoIGZpbGUg ZGVzY3JpcHRvcnMgKi8KIHN0YXRpYyBpbnQgZm9saW9faW5jX2dlbihzdHJ1Y3QgbHJ1dmVjICps cnV2ZWMsIHN0cnVjdCBmb2xpbyAqZm9saW8sIGJvb2wgcmVjbGFpbWluZykKIHsKQEAgLTMxNzIs NiArMzIwMCwxMSBAQCBzdGF0aWMgaW50IGZvbGlvX2luY19nZW4oc3RydWN0IGxydXZlYyAqbHJ1 dmVjLCBzdHJ1Y3QgZm9saW8gKmZvbGlvLCBib29sIHJlY2xhaQogCVZNX1dBUk5fT05fT05DRV9G T0xJTyghKG9sZF9mbGFncyAmIExSVV9HRU5fTUFTSyksIGZvbGlvKTsKIAogCWRvIHsKKwkJbmV3 X2dlbiA9ICgob2xkX2ZsYWdzICYgTFJVX0dFTl9NQVNLKSA+PiBMUlVfR0VOX1BHT0ZGKSAtIDE7 CisJCS8qIGZvbGlvX3VwZGF0ZV9nZW4oKSBoYXMgcHJvbW90ZWQgdGhpcyBwYWdlPyAqLworCQlp ZiAobmV3X2dlbiA+PSAwICYmIG5ld19nZW4gIT0gb2xkX2dlbikKKwkJCXJldHVybiBuZXdfZ2Vu OworCiAJCW5ld19nZW4gPSAob2xkX2dlbiArIDEpICUgTUFYX05SX0dFTlM7CiAKIAkJbmV3X2Zs YWdzID0gb2xkX2ZsYWdzICYgfihMUlVfR0VOX01BU0sgfCBMUlVfUkVGU19NQVNLIHwgTFJVX1JF RlNfRkxBR1MpOwpAQCAtMzE4Niw2ICszMjE5LDQzIEBAIHN0YXRpYyBpbnQgZm9saW9faW5jX2dl bihzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIHN0cnVjdCBmb2xpbyAqZm9saW8sIGJvb2wgcmVjbGFp CiAJcmV0dXJuIG5ld19nZW47CiB9CiAKK3N0YXRpYyB1bnNpZ25lZCBsb25nIGdldF9wdGVfcGZu KHB0ZV90IHB0ZSwgc3RydWN0IHZtX2FyZWFfc3RydWN0ICp2bWEsIHVuc2lnbmVkIGxvbmcgYWRk cikKK3sKKwl1bnNpZ25lZCBsb25nIHBmbiA9IHB0ZV9wZm4ocHRlKTsKKworCVZNX1dBUk5fT05f T05DRShhZGRyIDwgdm1hLT52bV9zdGFydCB8fCBhZGRyID49IHZtYS0+dm1fZW5kKTsKKworCWlm ICghcHRlX3ByZXNlbnQocHRlKSB8fCBpc196ZXJvX3BmbihwZm4pKQorCQlyZXR1cm4gLTE7CisK KwlpZiAoV0FSTl9PTl9PTkNFKHB0ZV9kZXZtYXAocHRlKSB8fCBwdGVfc3BlY2lhbChwdGUpKSkK KwkJcmV0dXJuIC0xOworCisJaWYgKFdBUk5fT05fT05DRSghcGZuX3ZhbGlkKHBmbikpKQorCQly ZXR1cm4gLTE7CisKKwlyZXR1cm4gcGZuOworfQorCitzdGF0aWMgc3RydWN0IGZvbGlvICpnZXRf cGZuX2ZvbGlvKHVuc2lnbmVkIGxvbmcgcGZuLCBzdHJ1Y3QgbWVtX2Nncm91cCAqbWVtY2csCisJ CQkJICAgc3RydWN0IHBnbGlzdF9kYXRhICpwZ2RhdCkKK3sKKwlzdHJ1Y3QgZm9saW8gKmZvbGlv OworCisJLyogdHJ5IHRvIGF2b2lkIHVubmVjZXNzYXJ5IG1lbW9yeSBsb2FkcyAqLworCWlmIChw Zm4gPCBwZ2RhdC0+bm9kZV9zdGFydF9wZm4gfHwgcGZuID49IHBnZGF0X2VuZF9wZm4ocGdkYXQp KQorCQlyZXR1cm4gTlVMTDsKKworCWZvbGlvID0gcGZuX2ZvbGlvKHBmbik7CisJaWYgKGZvbGlv X25pZChmb2xpbykgIT0gcGdkYXQtPm5vZGVfaWQpCisJCXJldHVybiBOVUxMOworCisJaWYgKGZv bGlvX21lbWNnX3JjdShmb2xpbykgIT0gbWVtY2cpCisJCXJldHVybiBOVUxMOworCisJcmV0dXJu IGZvbGlvOworfQorCiBzdGF0aWMgdm9pZCBpbmNfbWluX3NlcShzdHJ1Y3QgbHJ1dmVjICpscnV2 ZWMsIGludCB0eXBlKQogewogCXN0cnVjdCBscnVfZ2VuX3N0cnVjdCAqbHJ1Z2VuID0gJmxydXZl Yy0+bHJ1Z2VuOwpAQCAtMzM4Nyw2ICszNDU3LDExNCBAQCBzdGF0aWMgdm9pZCBscnVfZ2VuX2Fn ZV9ub2RlKHN0cnVjdCBwZ2xpc3RfZGF0YSAqcGdkYXQsIHN0cnVjdCBzY2FuX2NvbnRyb2wgKnNj KQogCX0gd2hpbGUgKChtZW1jZyA9IG1lbV9jZ3JvdXBfaXRlcihOVUxMLCBtZW1jZywgTlVMTCkp KTsKIH0KIAorLyoKKyAqIFRoaXMgZnVuY3Rpb24gZXhwbG9pdHMgc3BhdGlhbCBsb2NhbGl0eSB3 aGVuIHNocmlua19wYWdlX2xpc3QoKSB3YWxrcyB0aGUKKyAqIHJtYXAuIEl0IHNjYW5zIHRoZSBh ZGphY2VudCBQVEVzIG9mIGEgeW91bmcgUFRFIGFuZCBwcm9tb3RlcyBob3QgcGFnZXMuCisgKi8K K3ZvaWQgbHJ1X2dlbl9sb29rX2Fyb3VuZChzdHJ1Y3QgcGFnZV92bWFfbWFwcGVkX3dhbGsgKnB2 bXcpCit7CisJaW50IGk7CisJcHRlX3QgKnB0ZTsKKwl1bnNpZ25lZCBsb25nIHN0YXJ0OworCXVu c2lnbmVkIGxvbmcgZW5kOworCXVuc2lnbmVkIGxvbmcgYWRkcjsKKwl1bnNpZ25lZCBsb25nIGJp dG1hcFtCSVRTX1RPX0xPTkdTKE1JTl9MUlVfQkFUQ0gpXSA9IHt9OworCXN0cnVjdCBmb2xpbyAq Zm9saW8gPSBwZm5fZm9saW8ocHZtdy0+cGZuKTsKKwlzdHJ1Y3QgbWVtX2Nncm91cCAqbWVtY2cg PSBmb2xpb19tZW1jZyhmb2xpbyk7CisJc3RydWN0IHBnbGlzdF9kYXRhICpwZ2RhdCA9IGZvbGlv X3BnZGF0KGZvbGlvKTsKKwlzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMgPSBtZW1fY2dyb3VwX2xydXZl YyhtZW1jZywgcGdkYXQpOworCURFRklORV9NQVhfU0VRKGxydXZlYyk7CisJaW50IG9sZF9nZW4s IG5ld19nZW4gPSBscnVfZ2VuX2Zyb21fc2VxKG1heF9zZXEpOworCisJbG9ja2RlcF9hc3NlcnRf aGVsZChwdm13LT5wdGwpOworCVZNX1dBUk5fT05fT05DRV9GT0xJTyhmb2xpb190ZXN0X2xydShm b2xpbyksIGZvbGlvKTsKKworCWlmIChzcGluX2lzX2NvbnRlbmRlZChwdm13LT5wdGwpKQorCQly ZXR1cm47CisKKwlzdGFydCA9IG1heChwdm13LT5hZGRyZXNzICYgUE1EX01BU0ssIHB2bXctPnZt YS0+dm1fc3RhcnQpOworCWVuZCA9IG1pbihwdm13LT5hZGRyZXNzIHwgflBNRF9NQVNLLCBwdm13 LT52bWEtPnZtX2VuZCAtIDEpICsgMTsKKworCWlmIChlbmQgLSBzdGFydCA+IE1JTl9MUlVfQkFU Q0ggKiBQQUdFX1NJWkUpIHsKKwkJaWYgKHB2bXctPmFkZHJlc3MgLSBzdGFydCA8IE1JTl9MUlVf QkFUQ0ggKiBQQUdFX1NJWkUgLyAyKQorCQkJZW5kID0gc3RhcnQgKyBNSU5fTFJVX0JBVENIICog UEFHRV9TSVpFOworCQllbHNlIGlmIChlbmQgLSBwdm13LT5hZGRyZXNzIDwgTUlOX0xSVV9CQVRD SCAqIFBBR0VfU0laRSAvIDIpCisJCQlzdGFydCA9IGVuZCAtIE1JTl9MUlVfQkFUQ0ggKiBQQUdF X1NJWkU7CisJCWVsc2UgeworCQkJc3RhcnQgPSBwdm13LT5hZGRyZXNzIC0gTUlOX0xSVV9CQVRD SCAqIFBBR0VfU0laRSAvIDI7CisJCQllbmQgPSBwdm13LT5hZGRyZXNzICsgTUlOX0xSVV9CQVRD SCAqIFBBR0VfU0laRSAvIDI7CisJCX0KKwl9CisKKwlwdGUgPSBwdm13LT5wdGUgLSAocHZtdy0+ YWRkcmVzcyAtIHN0YXJ0KSAvIFBBR0VfU0laRTsKKworCXJjdV9yZWFkX2xvY2soKTsKKwlhcmNo X2VudGVyX2xhenlfbW11X21vZGUoKTsKKworCWZvciAoaSA9IDAsIGFkZHIgPSBzdGFydDsgYWRk ciAhPSBlbmQ7IGkrKywgYWRkciArPSBQQUdFX1NJWkUpIHsKKwkJdW5zaWduZWQgbG9uZyBwZm47 CisKKwkJcGZuID0gZ2V0X3B0ZV9wZm4ocHRlW2ldLCBwdm13LT52bWEsIGFkZHIpOworCQlpZiAo cGZuID09IC0xKQorCQkJY29udGludWU7CisKKwkJaWYgKCFwdGVfeW91bmcocHRlW2ldKSkKKwkJ CWNvbnRpbnVlOworCisJCWZvbGlvID0gZ2V0X3Bmbl9mb2xpbyhwZm4sIG1lbWNnLCBwZ2RhdCk7 CisJCWlmICghZm9saW8pCisJCQljb250aW51ZTsKKworCQlpZiAoIXB0ZXBfdGVzdF9hbmRfY2xl YXJfeW91bmcocHZtdy0+dm1hLCBhZGRyLCBwdGUgKyBpKSkKKwkJCWNvbnRpbnVlOworCisJCWlm IChwdGVfZGlydHkocHRlW2ldKSAmJiAhZm9saW9fdGVzdF9kaXJ0eShmb2xpbykgJiYKKwkJICAg ICEoZm9saW9fdGVzdF9hbm9uKGZvbGlvKSAmJiBmb2xpb190ZXN0X3N3YXBiYWNrZWQoZm9saW8p ICYmCisJCSAgICAgICFmb2xpb190ZXN0X3N3YXBjYWNoZShmb2xpbykpKQorCQkJZm9saW9fbWFy a19kaXJ0eShmb2xpbyk7CisKKwkJb2xkX2dlbiA9IGZvbGlvX2xydV9nZW4oZm9saW8pOworCQlp ZiAob2xkX2dlbiA8IDApCisJCQlmb2xpb19zZXRfcmVmZXJlbmNlZChmb2xpbyk7CisJCWVsc2Ug aWYgKG9sZF9nZW4gIT0gbmV3X2dlbikKKwkJCV9fc2V0X2JpdChpLCBiaXRtYXApOworCX0KKwor CWFyY2hfbGVhdmVfbGF6eV9tbXVfbW9kZSgpOworCXJjdV9yZWFkX3VubG9jaygpOworCisJaWYg KGJpdG1hcF93ZWlnaHQoYml0bWFwLCBNSU5fTFJVX0JBVENIKSA8IFBBR0VWRUNfU0laRSkgewor CQlmb3JfZWFjaF9zZXRfYml0KGksIGJpdG1hcCwgTUlOX0xSVV9CQVRDSCkgeworCQkJZm9saW8g PSBwZm5fZm9saW8ocHRlX3BmbihwdGVbaV0pKTsKKwkJCWZvbGlvX2FjdGl2YXRlKGZvbGlvKTsK KwkJfQorCQlyZXR1cm47CisJfQorCisJLyogZm9saW9fdXBkYXRlX2dlbigpIHJlcXVpcmVzIHN0 YWJsZSBmb2xpb19tZW1jZygpICovCisJaWYgKCFtZW1fY2dyb3VwX3RyeWxvY2tfcGFnZXMobWVt Y2cpKQorCQlyZXR1cm47CisKKwlzcGluX2xvY2tfaXJxKCZscnV2ZWMtPmxydV9sb2NrKTsKKwlu ZXdfZ2VuID0gbHJ1X2dlbl9mcm9tX3NlcShscnV2ZWMtPmxydWdlbi5tYXhfc2VxKTsKKworCWZv cl9lYWNoX3NldF9iaXQoaSwgYml0bWFwLCBNSU5fTFJVX0JBVENIKSB7CisJCWZvbGlvID0gcGZu X2ZvbGlvKHB0ZV9wZm4ocHRlW2ldKSk7CisJCWlmIChmb2xpb19tZW1jZ19yY3UoZm9saW8pICE9 IG1lbWNnKQorCQkJY29udGludWU7CisKKwkJb2xkX2dlbiA9IGZvbGlvX3VwZGF0ZV9nZW4oZm9s aW8sIG5ld19nZW4pOworCQlpZiAob2xkX2dlbiA8IDAgfHwgb2xkX2dlbiA9PSBuZXdfZ2VuKQor CQkJY29udGludWU7CisKKwkJbHJ1X2dlbl91cGRhdGVfc2l6ZShscnV2ZWMsIGZvbGlvLCBvbGRf Z2VuLCBuZXdfZ2VuKTsKKwl9CisKKwlzcGluX3VubG9ja19pcnEoJmxydXZlYy0+bHJ1X2xvY2sp OworCisJbWVtX2Nncm91cF91bmxvY2tfcGFnZXMoKTsKK30KKwogLyoqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKgogICogICAgICAgICAgICAgICAgICAgICAgICAgIHRoZSBldmljdGlvbgogICoqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKi8KQEAgLTM0MjMsNiArMzYwMSwxMiBAQCBzdGF0aWMgYm9vbCBzb3J0X2Zv bGlvKHN0cnVjdCBscnV2ZWMgKmxydXZlYywgc3RydWN0IGZvbGlvICpmb2xpbywgaW50IHRpZXJf aWR4KQogCQlyZXR1cm4gdHJ1ZTsKIAl9CiAKKwkvKiBwcm9tb3RlZCAqLworCWlmIChnZW4gIT0g bHJ1X2dlbl9mcm9tX3NlcShscnVnZW4tPm1pbl9zZXFbdHlwZV0pKSB7CisJCWxpc3RfbW92ZSgm Zm9saW8tPmxydSwgJmxydWdlbi0+bGlzdHNbZ2VuXVt0eXBlXVt6b25lXSk7CisJCXJldHVybiB0 cnVlOworCX0KKwogCS8qIHByb3RlY3RlZCAqLwogCWlmICh0aWVyID4gdGllcl9pZHgpIHsKIAkJ aW50IGhpc3QgPSBscnVfaGlzdF9mcm9tX3NlcShscnVnZW4tPm1pbl9zZXFbdHlwZV0pOwotLSAK Mi4zNy4wLnJjMC4xNjEuZzEwZjM3YmVkOTAtZ29vZwoKCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0Cmxp bnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFk Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK