From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B66EAC433F5 for ; Thu, 7 Apr 2022 03:17:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239082AbiDGDT1 (ORCPT ); Wed, 6 Apr 2022 23:19:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239969AbiDGDS0 (ORCPT ); Wed, 6 Apr 2022 23:18:26 -0400 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 254AAC31EE for ; Wed, 6 Apr 2022 20:16:08 -0700 (PDT) Received: by mail-yb1-xb4a.google.com with SMTP id h16-20020a056902009000b00628a70584b2so3258086ybs.6 for ; Wed, 06 Apr 2022 20:16:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=FUNr8Ptg01wK4u0fsqnOqW+junsbsxXOavZffzEGIdk=; b=Rx3U8i8G5m1+UlTrPUIZxIw83wXM0lrDLcawlwrVFaHMnx37cFe4VdsevvjzAdE8wj 9y0u5Oj1ztA6unJS2wyPI8KU4Xtd4gfW7slmWyRTUGaZ8XGHSmhFWsfyWQnZRFNT1M0C 6n2y7uJNPEhypsC+Rc3XPbG4YNqAcJ7jAjYK1WZQ3H581xldni32WRP1LwO2He32hwdL q08J1hwbimizN9PK8igKSRMPO9DAWXRe/Pw3NqF+l1qDAyfMSXgXMvBe9x4/bnbjFRqc wkVwB5rqPiBwkcAcjlM+9P/f2euR+mcTJFlt9Ot1vFLXtZQ2EHryxNLHpeG13wP1WseV xqzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=FUNr8Ptg01wK4u0fsqnOqW+junsbsxXOavZffzEGIdk=; b=s0x9uCvfunWdyV/wqUFwMuTdWxZB4r/irQBAuiMYh8LEU/bRitgJURf1GEzwDW9Aa2 TGI+2QOm9SlKLad59GIpN7xuJXhblPwaRsN3AdBJNu3k59FSJngJscBux4kqFyTpfCW0 J6CB1/B7pF2w05ZlPOmbT7wkflY12DOr4UU/kUoFH+6RAOk9oqqN192xW1seaOTdYwzk sO0zNCuR9XoVV6yLy7AdStwh19NkBTx0+pX3NmtuCJ5HAre8KKl2yv2IB/6uZnW95n4t Uezdu09KAQYoiLPjr1y/jvC1LQTMf/qNvUpJEvMRUGREI7Fjg/CTnPLo8vy4TM4oGe4Y VyLw== X-Gm-Message-State: AOAM532LO3hJrTQ7GfeMrtV4LlWcrm9mlr7vylzB5QSQaz1Sj0S2F6JB yqJs87ECqfTWVNN1b1gCyjSpkyPagxM= X-Google-Smtp-Source: ABdhPJxpRdZ1hhusuNPUwusUSddl03mI4T4fiRDjyemLdicEamLwq/kGPYBCOvcLEA5luSocVTNwZy6y2zE= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:9ea2:c755:ae22:6862]) (user=yuzhao job=sendgmr) by 2002:a0d:d382:0:b0:2e5:9d15:3cfa with SMTP id v124-20020a0dd382000000b002e59d153cfamr9564591ywd.239.1649301367259; Wed, 06 Apr 2022 20:16:07 -0700 (PDT) Date: Wed, 6 Apr 2022 21:15:19 -0600 In-Reply-To: <20220407031525.2368067-1-yuzhao@google.com> Message-Id: <20220407031525.2368067-8-yuzhao@google.com> Mime-Version: 1.0 References: <20220407031525.2368067-1-yuzhao@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v10 07/14] mm: multi-gen LRU: exploit locality in rmap From: Yu Zhao To: Stephen Rothwell , linux-mm@kvack.org Cc: Andi Kleen , Andrew Morton , Aneesh Kumar , Barry Song <21cnbao@gmail.com>, Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, page-reclaim@google.com, x86@kernel.org, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Searching the rmap for PTEs mapping each page on an LRU list (to test and clear the accessed bit) can be expensive because pages from different VMAs (PA space) are not cache friendly to the rmap (VA space). For workloads mostly using mapped pages, the rmap has a high CPU cost in the reclaim path. This patch exploits spatial locality to reduce the trips into the rmap. When shrink_page_list() walks the rmap and finds a young PTE, a new function lru_gen_look_around() scans at most BITS_PER_LONG-1 adjacent PTEs. On finding another young PTE, it clears the accessed bit and updates the gen counter of the page mapped by this PTE to (max_seq%MAX_NR_GENS)+1. Server benchmark results: Single workload: fio (buffered I/O): no change Single workload: memcached (anon): +[5.5, 7.5]% Ops/sec KB/sec patch1-6: 1120643.70 43588.06 patch1-7: 1193918.93 46438.15 Configurations: no change Client benchmark results: kswapd profiles: patch1-6 35.99% lzo1x_1_do_compress (real work) 19.40% page_vma_mapped_walk 6.31% _raw_spin_unlock_irq 3.95% do_raw_spin_lock 2.39% anon_vma_interval_tree_iter_first 2.25% ptep_clear_flush 1.92% __anon_vma_interval_tree_subtree_search 1.70% folio_referenced_one 1.68% __zram_bvec_write 1.43% anon_vma_interval_tree_iter_next patch1-7 45.90% lzo1x_1_do_compress (real work) 9.14% page_vma_mapped_walk 6.81% _raw_spin_unlock_irq 2.80% ptep_clear_flush 2.34% __zram_bvec_write 2.29% do_raw_spin_lock 1.84% lru_gen_look_around 1.78% memmove 1.74% obj_malloc 1.50% free_unref_page_list Configurations: no change Signed-off-by: Yu Zhao Acked-by: Brian Geffon Acked-by: Jan Alexander Steffens (heftig) Acked-by: Oleksandr Natalenko Acked-by: Steven Barrett Acked-by: Suleiman Souhlal Tested-by: Daniel Byrne Tested-by: Donald Carr Tested-by: Holger Hoffst=C3=A4tte Tested-by: Konstantin Kharlamov Tested-by: Shuang Zhai Tested-by: Sofia Trinh Tested-by: Vaibhav Jain --- include/linux/memcontrol.h | 31 ++++++++ include/linux/mm.h | 5 ++ include/linux/mmzone.h | 6 ++ mm/internal.h | 1 + mm/memcontrol.c | 1 + mm/rmap.c | 7 ++ mm/swap.c | 4 +- mm/vmscan.c | 159 +++++++++++++++++++++++++++++++++++++ 8 files changed, 212 insertions(+), 2 deletions(-) diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h index a68dce3873fc..463168fa1670 100644 --- a/include/linux/memcontrol.h +++ b/include/linux/memcontrol.h @@ -438,6 +438,7 @@ static inline struct obj_cgroup *__folio_objcg(struct f= olio *folio) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() * * For a kmem folio a caller should hold an rcu read lock to protect memcg * associated with a kmem folio from being released. @@ -499,6 +500,7 @@ static inline struct mem_cgroup *folio_memcg_rcu(struct= folio *folio) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() * * For a kmem page a caller should hold an rcu read lock to protect memcg * associated with a kmem page from being released. @@ -948,6 +950,23 @@ void unlock_page_memcg(struct page *page); =20 void __mod_memcg_state(struct mem_cgroup *memcg, int idx, int val); =20 +/* try to stablize folio_memcg() for all the pages in a memcg */ +static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) +{ + rcu_read_lock(); + + if (mem_cgroup_disabled() || !atomic_read(&memcg->moving_account)) + return true; + + rcu_read_unlock(); + return false; +} + +static inline void mem_cgroup_unlock_pages(void) +{ + rcu_read_unlock(); +} + /* idx can be of type enum memcg_stat_item or node_stat_item */ static inline void mod_memcg_state(struct mem_cgroup *memcg, int idx, int val) @@ -1385,6 +1404,18 @@ static inline void folio_memcg_unlock(struct folio *= folio) { } =20 +static inline bool mem_cgroup_trylock_pages(struct mem_cgroup *memcg) +{ + /* to match folio_memcg_rcu() */ + rcu_read_lock(); + return true; +} + +static inline void mem_cgroup_unlock_pages(void) +{ + rcu_read_unlock(); +} + static inline void mem_cgroup_handle_over_high(void) { } diff --git a/include/linux/mm.h b/include/linux/mm.h index 980f568204a3..bb907088777e 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1523,6 +1523,11 @@ static inline unsigned long folio_pfn(struct folio *= folio) return page_to_pfn(&folio->page); } =20 +static inline struct folio *pfn_folio(unsigned long pfn) +{ + return page_folio(pfn_to_page(pfn)); +} + static inline atomic_t *folio_pincount_ptr(struct folio *folio) { return &folio_page(folio, 1)->compound_pincount; diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index c8a7ceee7a0a..c69589ad2b05 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -371,6 +371,7 @@ enum lruvec_flags { #ifndef __GENERATING_BOUNDS_H =20 struct lruvec; +struct page_vma_mapped_walk; =20 #define LRU_GEN_MASK ((BIT(LRU_GEN_WIDTH) - 1) << LRU_GEN_PGOFF) #define LRU_REFS_MASK ((BIT(LRU_REFS_WIDTH) - 1) << LRU_REFS_PGOFF) @@ -424,6 +425,7 @@ struct lru_gen_struct { }; =20 void lru_gen_init_lruvec(struct lruvec *lruvec); +void lru_gen_look_around(struct page_vma_mapped_walk *pvmw); =20 #ifdef CONFIG_MEMCG void lru_gen_init_memcg(struct mem_cgroup *memcg); @@ -436,6 +438,10 @@ static inline void lru_gen_init_lruvec(struct lruvec *= lruvec) { } =20 +static inline void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) +{ +} + #ifdef CONFIG_MEMCG static inline void lru_gen_init_memcg(struct mem_cgroup *memcg) { diff --git a/mm/internal.h b/mm/internal.h index cf16280ce132..59d2422b647d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -68,6 +68,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf); void folio_rotate_reclaimable(struct folio *folio); bool __folio_end_writeback(struct folio *folio); void deactivate_file_folio(struct folio *folio); +void folio_activate(struct folio *folio); =20 void free_pgtables(struct mmu_gather *tlb, struct vm_area_struct *start_vm= a, unsigned long floor, unsigned long ceiling); diff --git a/mm/memcontrol.c b/mm/memcontrol.c index f5de8be80c13..8069b58f2422 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2759,6 +2759,7 @@ static void commit_charge(struct folio *folio, struct= mem_cgroup *memcg) * - LRU isolation * - lock_page_memcg() * - exclusive reference + * - mem_cgroup_trylock_pages() */ folio->memcg_data =3D (unsigned long)memcg; } diff --git a/mm/rmap.c b/mm/rmap.c index fedb82371efe..7cb7ef29088a 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -73,6 +73,7 @@ #include #include #include +#include =20 #include =20 @@ -821,6 +822,12 @@ static bool folio_referenced_one(struct folio *folio, } =20 if (pvmw.pte) { + if (lru_gen_enabled() && pte_young(*pvmw.pte) && + !(vma->vm_flags & (VM_SEQ_READ | VM_RAND_READ))) { + lru_gen_look_around(&pvmw); + referenced++; + } + if (ptep_clear_flush_young_notify(vma, address, pvmw.pte)) { /* diff --git a/mm/swap.c b/mm/swap.c index 6a5203f18b0a..c0e4060bbbe0 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -342,7 +342,7 @@ static bool need_activate_page_drain(int cpu) return pagevec_count(&per_cpu(lru_pvecs.activate_page, cpu)) !=3D 0; } =20 -static void folio_activate(struct folio *folio) +void folio_activate(struct folio *folio) { if (folio_test_lru(folio) && !folio_test_active(folio) && !folio_test_unevictable(folio)) { @@ -362,7 +362,7 @@ static inline void activate_page_drain(int cpu) { } =20 -static void folio_activate(struct folio *folio) +void folio_activate(struct folio *folio) { struct lruvec *lruvec; =20 diff --git a/mm/vmscan.c b/mm/vmscan.c index bb3d705c5282..f4dd3c3c589b 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1554,6 +1554,11 @@ static unsigned int shrink_page_list(struct list_hea= d *page_list, if (!sc->may_unmap && page_mapped(page)) goto keep_locked; =20 + /* folio_update_gen() tried to promote this page? */ + if (lru_gen_enabled() && !ignore_references && + page_mapped(page) && PageReferenced(page)) + goto keep_locked; + may_enter_fs =3D (sc->gfp_mask & __GFP_FS) || (PageSwapCache(page) && (sc->gfp_mask & __GFP_IO)); =20 @@ -3153,6 +3158,31 @@ static bool positive_ctrl_err(struct ctrl_pos *sp, s= truct ctrl_pos *pv) * the aging *************************************************************************= *****/ =20 +static int folio_update_gen(struct folio *folio, int gen) +{ + unsigned long old_flags, new_flags; + + VM_BUG_ON(gen >=3D MAX_NR_GENS); + VM_BUG_ON(!rcu_read_lock_held()); + + do { + new_flags =3D old_flags =3D READ_ONCE(folio->flags); + + /* for shrink_page_list() */ + if (!(new_flags & LRU_GEN_MASK)) { + new_flags |=3D BIT(PG_referenced); + continue; + } + + new_flags &=3D ~LRU_GEN_MASK; + new_flags |=3D (gen + 1UL) << LRU_GEN_PGOFF; + new_flags &=3D ~(LRU_REFS_MASK | LRU_REFS_FLAGS); + } while (new_flags !=3D old_flags && + cmpxchg(&folio->flags, old_flags, new_flags) !=3D old_flags); + + return ((old_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1; +} + static int folio_inc_gen(struct lruvec *lruvec, struct folio *folio, bool = reclaiming) { unsigned long old_flags, new_flags; @@ -3164,6 +3194,11 @@ static int folio_inc_gen(struct lruvec *lruvec, stru= ct folio *folio, bool reclai new_flags =3D old_flags =3D READ_ONCE(folio->flags); VM_BUG_ON_FOLIO(!(new_flags & LRU_GEN_MASK), folio); =20 + new_gen =3D ((new_flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1; + /* folio_update_gen() has promoted this page? */ + if (new_gen >=3D 0 && new_gen !=3D old_gen) + return new_gen; + new_gen =3D (old_gen + 1) % MAX_NR_GENS; =20 new_flags &=3D ~LRU_GEN_MASK; @@ -3385,6 +3420,125 @@ static void lru_gen_age_node(struct pglist_data *pg= dat, struct scan_control *sc) } while ((memcg =3D mem_cgroup_iter(NULL, memcg, NULL))); } =20 +/* + * This function exploits spatial locality when shrink_page_list() walks t= he + * rmap. It scans the adjacent PTEs of a young PTE and promotes hot pages. + */ +void lru_gen_look_around(struct page_vma_mapped_walk *pvmw) +{ + int i; + pte_t *pte; + unsigned long start; + unsigned long end; + unsigned long addr; + unsigned long bitmap[BITS_TO_LONGS(MIN_LRU_BATCH)] =3D {}; + struct folio *folio =3D pfn_folio(pvmw->pfn); + struct mem_cgroup *memcg =3D folio_memcg(folio); + struct pglist_data *pgdat =3D folio_pgdat(folio); + struct lruvec *lruvec =3D mem_cgroup_lruvec(memcg, pgdat); + DEFINE_MAX_SEQ(lruvec); + int old_gen, new_gen =3D lru_gen_from_seq(max_seq); + + lockdep_assert_held(pvmw->ptl); + VM_BUG_ON_FOLIO(folio_test_lru(folio), folio); + + if (spin_is_contended(pvmw->ptl)) + return; + + start =3D max(pvmw->address & PMD_MASK, pvmw->vma->vm_start); + end =3D pmd_addr_end(pvmw->address, pvmw->vma->vm_end); + + if (end - start > MIN_LRU_BATCH * PAGE_SIZE) { + if (pvmw->address - start < MIN_LRU_BATCH * PAGE_SIZE / 2) + end =3D start + MIN_LRU_BATCH * PAGE_SIZE; + else if (end - pvmw->address < MIN_LRU_BATCH * PAGE_SIZE / 2) + start =3D end - MIN_LRU_BATCH * PAGE_SIZE; + else { + start =3D pvmw->address - MIN_LRU_BATCH * PAGE_SIZE / 2; + end =3D pvmw->address + MIN_LRU_BATCH * PAGE_SIZE / 2; + } + } + + pte =3D pvmw->pte - (pvmw->address - start) / PAGE_SIZE; + + rcu_read_lock(); + arch_enter_lazy_mmu_mode(); + + for (i =3D 0, addr =3D start; addr !=3D end; i++, addr +=3D PAGE_SIZE) { + unsigned long pfn =3D pte_pfn(pte[i]); + + VM_BUG_ON(addr < pvmw->vma->vm_start || addr >=3D pvmw->vma->vm_end); + + if (!pte_present(pte[i]) || is_zero_pfn(pfn)) + continue; + + if (WARN_ON_ONCE(pte_devmap(pte[i]) || pte_special(pte[i]))) + continue; + + if (!pte_young(pte[i])) + continue; + + VM_BUG_ON(!pfn_valid(pfn)); + if (pfn < pgdat->node_start_pfn || pfn >=3D pgdat_end_pfn(pgdat)) + continue; + + folio =3D pfn_folio(pfn); + if (folio_nid(folio) !=3D pgdat->node_id) + continue; + + if (folio_memcg_rcu(folio) !=3D memcg) + continue; + + if (!ptep_test_and_clear_young(pvmw->vma, addr, pte + i)) + continue; + + if (pte_dirty(pte[i]) && !folio_test_dirty(folio) && + !(folio_test_anon(folio) && folio_test_swapbacked(folio) && + !folio_test_swapcache(folio))) + folio_mark_dirty(folio); + + old_gen =3D folio_lru_gen(folio); + if (old_gen < 0) + folio_set_referenced(folio); + else if (old_gen !=3D new_gen) + __set_bit(i, bitmap); + } + + arch_leave_lazy_mmu_mode(); + rcu_read_unlock(); + + if (bitmap_weight(bitmap, MIN_LRU_BATCH) < PAGEVEC_SIZE) { + for_each_set_bit(i, bitmap, MIN_LRU_BATCH) { + folio =3D pfn_folio(pte_pfn(pte[i])); + folio_activate(folio); + } + return; + } + + /* folio_update_gen() requires stable folio_memcg() */ + if (!mem_cgroup_trylock_pages(memcg)) + return; + + spin_lock_irq(&lruvec->lru_lock); + new_gen =3D lru_gen_from_seq(lruvec->lrugen.max_seq); + + for_each_set_bit(i, bitmap, MIN_LRU_BATCH) { + folio =3D pfn_folio(pte_pfn(pte[i])); + if (folio_memcg_rcu(folio) !=3D memcg) + continue; + + old_gen =3D folio_update_gen(folio, new_gen); + if (old_gen < 0 || old_gen =3D=3D new_gen) + continue; + + lru_gen_update_size(lruvec, folio, old_gen, new_gen); + } + + spin_unlock_irq(&lruvec->lru_lock); + + mem_cgroup_unlock_pages(); +} + /*************************************************************************= ***** * the eviction *************************************************************************= *****/ @@ -3418,6 +3572,11 @@ static bool sort_folio(struct lruvec *lruvec, struct= folio *folio, int tier_idx) return true; } =20 + if (gen !=3D lru_gen_from_seq(lrugen->min_seq[type])) { + list_move(&folio->lru, &lrugen->lists[gen][type][zone]); + return true; + } + if (tier > tier_idx) { int hist =3D lru_hist_from_seq(lrugen->min_seq[type]); =20 --=20 2.35.1.1094.g7c7d902a7c-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ED3FFC433EF for ; Thu, 7 Apr 2022 03:31:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=gLw9Bm5hBxGX3q5nDOdVccAWZjhY1sm87s+fdL+JkpM=; b=frlZqcOcxQramBMyZpLw927brd NVBkII+6gw1CzCpdH04uBJhVaOCde8vSkRfPGGw9seT0s13nEEkTjGzGpn3ORC6lZgUVONrU2tk1w zBtNOll2/fv0/oD2spryrAQPUTTmvApcerFjJpRutRBtiJg+bkhT9gxx7aF/CkEygrcZt1CFyO0VQ T8wMUnWny4iJr9YSWEWqTnmvkxS/z0ughPGexW9RnXy8huOM3AvG+tIx+2Fz1mqrj2gMjdFUpMts2 Dz3qnnWNJSXLM287TbmHGgiTbGUCgOBrnBusKmBOBwsTnPetqDoxOWIv+/5UtG/pW0NUKq2dCwQWe Qf73ZiJA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncIqD-0096mY-F8; Thu, 07 Apr 2022 03:29:51 +0000 Received: from desiato.infradead.org ([90.155.92.199]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncId9-0091i8-TU for linux-arm-kernel@bombadil.infradead.org; Thu, 07 Apr 2022 03:16:20 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :Cc:To:From:Subject:References:Mime-Version:Message-Id:In-Reply-To:Date: Sender:Reply-To:Content-ID:Content-Description; bh=FUNr8Ptg01wK4u0fsqnOqW+junsbsxXOavZffzEGIdk=; b=RmUcUwJlMtFPCceEmTUo/T6RGB wTxOTvHkJiRlX6CFfzpojIRk7EC+LmllRTVLTzzxOf1CNvrJ4SRYjIEJcBI1zxvkrsERe3P/tjN1W YoBx1sQ5vukRFANUBXCtewB3Orq+R0xidNS5VIunbY+83is2MxzlhB8Rg4+g7ePDBIWefPoo5pvf9 tXdSJkvHaPkAcBTMfpHmuv9qPrpSjsoEJTsTBGLKvQy4Siq2UxMgu5vXutDk0ZH3KVcNwXsj+nXeR ztXrkV61+vM0HAW+xCUvJAnJLFmiTjMVXzW+I7x1USAvv7xF0cu2OD/oqxYIvPA9X8kXCBK7lln4W wwyZhRvg==; Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1ncId0-002Mjc-Ni for linux-arm-kernel@lists.infradead.org; Thu, 07 Apr 2022 03:16:14 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id z15-20020a25bb0f000000b00613388c7d99so3286026ybg.8 for ; Wed, 06 Apr 2022 20:16:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=FUNr8Ptg01wK4u0fsqnOqW+junsbsxXOavZffzEGIdk=; b=Rx3U8i8G5m1+UlTrPUIZxIw83wXM0lrDLcawlwrVFaHMnx37cFe4VdsevvjzAdE8wj 9y0u5Oj1ztA6unJS2wyPI8KU4Xtd4gfW7slmWyRTUGaZ8XGHSmhFWsfyWQnZRFNT1M0C 6n2y7uJNPEhypsC+Rc3XPbG4YNqAcJ7jAjYK1WZQ3H581xldni32WRP1LwO2He32hwdL q08J1hwbimizN9PK8igKSRMPO9DAWXRe/Pw3NqF+l1qDAyfMSXgXMvBe9x4/bnbjFRqc wkVwB5rqPiBwkcAcjlM+9P/f2euR+mcTJFlt9Ot1vFLXtZQ2EHryxNLHpeG13wP1WseV xqzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=FUNr8Ptg01wK4u0fsqnOqW+junsbsxXOavZffzEGIdk=; b=I1KKhKUlPGCJwuSrnmNClRXvng5pPZU4pWabCUVECnq1zTftr972H+r0/SZHG5weuR 0q7X/yFMmsl+KqrY6pq+Oy2FyyjXpQSnlburq2D2tRJ6mOYx7D9qG5sDq2QepLBO9kUu jeR0Pw+I8UWLm1MGBh5Ub2iUHFHk3FixCn2SfTQNPlqTYbqDQAMYCppSnRPy/zL++Fea jZFCrHVyHQeY2zACFk7OAN5TKEYZGPdsjmeC2N/4pOWHzN1Uu8t+rTSB/Ue6JvxvaMQ9 UtZbogVafdD/3YJc8/ZPxPLt1rfBXG76h9D/uR8PjgvO78rojixcRXjuDZ3ZvcjzwQ0t LI0A== X-Gm-Message-State: AOAM533ldQmjxftiZTvmEEYKLkH5cDV5wX4DrUslLj51AriDrywvsV8n Tb3DjJaB6AIcWnBEJBjxO2InvePAMmk= X-Google-Smtp-Source: ABdhPJxpRdZ1hhusuNPUwusUSddl03mI4T4fiRDjyemLdicEamLwq/kGPYBCOvcLEA5luSocVTNwZy6y2zE= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:9ea2:c755:ae22:6862]) (user=yuzhao job=sendgmr) by 2002:a0d:d382:0:b0:2e5:9d15:3cfa with SMTP id v124-20020a0dd382000000b002e59d153cfamr9564591ywd.239.1649301367259; Wed, 06 Apr 2022 20:16:07 -0700 (PDT) Date: Wed, 6 Apr 2022 21:15:19 -0600 In-Reply-To: <20220407031525.2368067-1-yuzhao@google.com> Message-Id: <20220407031525.2368067-8-yuzhao@google.com> Mime-Version: 1.0 References: <20220407031525.2368067-1-yuzhao@google.com> X-Mailer: git-send-email 2.35.1.1094.g7c7d902a7c-goog Subject: [PATCH v10 07/14] mm: multi-gen LRU: exploit locality in rmap From: Yu Zhao To: Stephen Rothwell , linux-mm@kvack.org Cc: Andi Kleen , Andrew Morton , Aneesh Kumar , Barry Song <21cnbao@gmail.com>, Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Jesse Barnes , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Rik van Riel , Vlastimil Babka , Will Deacon , Ying Huang , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, page-reclaim@google.com, x86@kernel.org, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220407_041611_773246_B733D03B X-CRM114-Status: GOOD ( 23.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org U2VhcmNoaW5nIHRoZSBybWFwIGZvciBQVEVzIG1hcHBpbmcgZWFjaCBwYWdlIG9uIGFuIExSVSBs aXN0ICh0byB0ZXN0CmFuZCBjbGVhciB0aGUgYWNjZXNzZWQgYml0KSBjYW4gYmUgZXhwZW5zaXZl IGJlY2F1c2UgcGFnZXMgZnJvbQpkaWZmZXJlbnQgVk1BcyAoUEEgc3BhY2UpIGFyZSBub3QgY2Fj aGUgZnJpZW5kbHkgdG8gdGhlIHJtYXAgKFZBCnNwYWNlKS4gRm9yIHdvcmtsb2FkcyBtb3N0bHkg dXNpbmcgbWFwcGVkIHBhZ2VzLCB0aGUgcm1hcCBoYXMgYSBoaWdoCkNQVSBjb3N0IGluIHRoZSBy ZWNsYWltIHBhdGguCgpUaGlzIHBhdGNoIGV4cGxvaXRzIHNwYXRpYWwgbG9jYWxpdHkgdG8gcmVk dWNlIHRoZSB0cmlwcyBpbnRvIHRoZQpybWFwLiBXaGVuIHNocmlua19wYWdlX2xpc3QoKSB3YWxr cyB0aGUgcm1hcCBhbmQgZmluZHMgYSB5b3VuZyBQVEUsIGEKbmV3IGZ1bmN0aW9uIGxydV9nZW5f bG9va19hcm91bmQoKSBzY2FucyBhdCBtb3N0IEJJVFNfUEVSX0xPTkctMQphZGphY2VudCBQVEVz LiBPbiBmaW5kaW5nIGFub3RoZXIgeW91bmcgUFRFLCBpdCBjbGVhcnMgdGhlIGFjY2Vzc2VkCmJp dCBhbmQgdXBkYXRlcyB0aGUgZ2VuIGNvdW50ZXIgb2YgdGhlIHBhZ2UgbWFwcGVkIGJ5IHRoaXMg UFRFIHRvCihtYXhfc2VxJU1BWF9OUl9HRU5TKSsxLgoKU2VydmVyIGJlbmNobWFyayByZXN1bHRz OgogIFNpbmdsZSB3b3JrbG9hZDoKICAgIGZpbyAoYnVmZmVyZWQgSS9PKTogbm8gY2hhbmdlCgog IFNpbmdsZSB3b3JrbG9hZDoKICAgIG1lbWNhY2hlZCAoYW5vbik6ICtbNS41LCA3LjVdJQogICAg ICAgICAgICAgICAgT3BzL3NlYyAgICAgIEtCL3NlYwogICAgICBwYXRjaDEtNjogMTEyMDY0My43 MCAgIDQzNTg4LjA2CiAgICAgIHBhdGNoMS03OiAxMTkzOTE4LjkzICAgNDY0MzguMTUKCiAgQ29u ZmlndXJhdGlvbnM6CiAgICBubyBjaGFuZ2UKCkNsaWVudCBiZW5jaG1hcmsgcmVzdWx0czoKICBr c3dhcGQgcHJvZmlsZXM6CiAgICBwYXRjaDEtNgogICAgICAzNS45OSUgIGx6bzF4XzFfZG9fY29t cHJlc3MgKHJlYWwgd29yaykKICAgICAgMTkuNDAlICBwYWdlX3ZtYV9tYXBwZWRfd2FsawogICAg ICAgNi4zMSUgIF9yYXdfc3Bpbl91bmxvY2tfaXJxCiAgICAgICAzLjk1JSAgZG9fcmF3X3NwaW5f bG9jawogICAgICAgMi4zOSUgIGFub25fdm1hX2ludGVydmFsX3RyZWVfaXRlcl9maXJzdAogICAg ICAgMi4yNSUgIHB0ZXBfY2xlYXJfZmx1c2gKICAgICAgIDEuOTIlICBfX2Fub25fdm1hX2ludGVy dmFsX3RyZWVfc3VidHJlZV9zZWFyY2gKICAgICAgIDEuNzAlICBmb2xpb19yZWZlcmVuY2VkX29u ZQogICAgICAgMS42OCUgIF9fenJhbV9idmVjX3dyaXRlCiAgICAgICAxLjQzJSAgYW5vbl92bWFf aW50ZXJ2YWxfdHJlZV9pdGVyX25leHQKCiAgICBwYXRjaDEtNwogICAgICA0NS45MCUgIGx6bzF4 XzFfZG9fY29tcHJlc3MgKHJlYWwgd29yaykKICAgICAgIDkuMTQlICBwYWdlX3ZtYV9tYXBwZWRf d2FsawogICAgICAgNi44MSUgIF9yYXdfc3Bpbl91bmxvY2tfaXJxCiAgICAgICAyLjgwJSAgcHRl cF9jbGVhcl9mbHVzaAogICAgICAgMi4zNCUgIF9fenJhbV9idmVjX3dyaXRlCiAgICAgICAyLjI5 JSAgZG9fcmF3X3NwaW5fbG9jawogICAgICAgMS44NCUgIGxydV9nZW5fbG9va19hcm91bmQKICAg ICAgIDEuNzglICBtZW1tb3ZlCiAgICAgICAxLjc0JSAgb2JqX21hbGxvYwogICAgICAgMS41MCUg IGZyZWVfdW5yZWZfcGFnZV9saXN0CgogIENvbmZpZ3VyYXRpb25zOgogICAgbm8gY2hhbmdlCgpT aWduZWQtb2ZmLWJ5OiBZdSBaaGFvIDx5dXpoYW9AZ29vZ2xlLmNvbT4KQWNrZWQtYnk6IEJyaWFu IEdlZmZvbiA8YmdlZmZvbkBnb29nbGUuY29tPgpBY2tlZC1ieTogSmFuIEFsZXhhbmRlciBTdGVm ZmVucyAoaGVmdGlnKSA8aGVmdGlnQGFyY2hsaW51eC5vcmc+CkFja2VkLWJ5OiBPbGVrc2FuZHIg TmF0YWxlbmtvIDxvbGVrc2FuZHJAbmF0YWxlbmtvLm5hbWU+CkFja2VkLWJ5OiBTdGV2ZW4gQmFy cmV0dCA8c3RldmVuQGxpcXVvcml4Lm5ldD4KQWNrZWQtYnk6IFN1bGVpbWFuIFNvdWhsYWwgPHN1 bGVpbWFuQGdvb2dsZS5jb20+ClRlc3RlZC1ieTogRGFuaWVsIEJ5cm5lIDxkamJ5cm5lQG10dS5l ZHU+ClRlc3RlZC1ieTogRG9uYWxkIENhcnIgPGRAY2hhb3MtcmVpbnMuY29tPgpUZXN0ZWQtYnk6 IEhvbGdlciBIb2Zmc3TDpHR0ZSA8aG9sZ2VyQGFwcGxpZWQtYXN5bmNocm9ueS5jb20+ClRlc3Rl ZC1ieTogS29uc3RhbnRpbiBLaGFybGFtb3YgPEhpLUFuZ2VsQHlhbmRleC5ydT4KVGVzdGVkLWJ5 OiBTaHVhbmcgWmhhaSA8c3poYWkyQGNzLnJvY2hlc3Rlci5lZHU+ClRlc3RlZC1ieTogU29maWEg VHJpbmggPHNvZmlhLnRyaW5oQGVkaS53b3Jrcz4KVGVzdGVkLWJ5OiBWYWliaGF2IEphaW4gPHZh aWJoYXZAbGludXguaWJtLmNvbT4KLS0tCiBpbmNsdWRlL2xpbnV4L21lbWNvbnRyb2wuaCB8ICAz MSArKysrKysrKwogaW5jbHVkZS9saW51eC9tbS5oICAgICAgICAgfCAgIDUgKysKIGluY2x1ZGUv bGludXgvbW16b25lLmggICAgIHwgICA2ICsrCiBtbS9pbnRlcm5hbC5oICAgICAgICAgICAgICB8 ICAgMSArCiBtbS9tZW1jb250cm9sLmMgICAgICAgICAgICB8ICAgMSArCiBtbS9ybWFwLmMgICAg ICAgICAgICAgICAgICB8ICAgNyArKwogbW0vc3dhcC5jICAgICAgICAgICAgICAgICAgfCAgIDQg Ky0KIG1tL3Ztc2Nhbi5jICAgICAgICAgICAgICAgIHwgMTU5ICsrKysrKysrKysrKysrKysrKysr KysrKysrKysrKysrKysrKysKIDggZmlsZXMgY2hhbmdlZCwgMjEyIGluc2VydGlvbnMoKyksIDIg ZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9tZW1jb250cm9sLmggYi9p bmNsdWRlL2xpbnV4L21lbWNvbnRyb2wuaAppbmRleCBhNjhkY2UzODczZmMuLjQ2MzE2OGZhMTY3 MCAxMDA2NDQKLS0tIGEvaW5jbHVkZS9saW51eC9tZW1jb250cm9sLmgKKysrIGIvaW5jbHVkZS9s aW51eC9tZW1jb250cm9sLmgKQEAgLTQzOCw2ICs0MzgsNyBAQCBzdGF0aWMgaW5saW5lIHN0cnVj dCBvYmpfY2dyb3VwICpfX2ZvbGlvX29iamNnKHN0cnVjdCBmb2xpbyAqZm9saW8pCiAgKiAtIExS VSBpc29sYXRpb24KICAqIC0gbG9ja19wYWdlX21lbWNnKCkKICAqIC0gZXhjbHVzaXZlIHJlZmVy ZW5jZQorICogLSBtZW1fY2dyb3VwX3RyeWxvY2tfcGFnZXMoKQogICoKICAqIEZvciBhIGttZW0g Zm9saW8gYSBjYWxsZXIgc2hvdWxkIGhvbGQgYW4gcmN1IHJlYWQgbG9jayB0byBwcm90ZWN0IG1l bWNnCiAgKiBhc3NvY2lhdGVkIHdpdGggYSBrbWVtIGZvbGlvIGZyb20gYmVpbmcgcmVsZWFzZWQu CkBAIC00OTksNiArNTAwLDcgQEAgc3RhdGljIGlubGluZSBzdHJ1Y3QgbWVtX2Nncm91cCAqZm9s aW9fbWVtY2dfcmN1KHN0cnVjdCBmb2xpbyAqZm9saW8pCiAgKiAtIExSVSBpc29sYXRpb24KICAq IC0gbG9ja19wYWdlX21lbWNnKCkKICAqIC0gZXhjbHVzaXZlIHJlZmVyZW5jZQorICogLSBtZW1f Y2dyb3VwX3RyeWxvY2tfcGFnZXMoKQogICoKICAqIEZvciBhIGttZW0gcGFnZSBhIGNhbGxlciBz aG91bGQgaG9sZCBhbiByY3UgcmVhZCBsb2NrIHRvIHByb3RlY3QgbWVtY2cKICAqIGFzc29jaWF0 ZWQgd2l0aCBhIGttZW0gcGFnZSBmcm9tIGJlaW5nIHJlbGVhc2VkLgpAQCAtOTQ4LDYgKzk1MCwy MyBAQCB2b2lkIHVubG9ja19wYWdlX21lbWNnKHN0cnVjdCBwYWdlICpwYWdlKTsKIAogdm9pZCBf X21vZF9tZW1jZ19zdGF0ZShzdHJ1Y3QgbWVtX2Nncm91cCAqbWVtY2csIGludCBpZHgsIGludCB2 YWwpOwogCisvKiB0cnkgdG8gc3RhYmxpemUgZm9saW9fbWVtY2coKSBmb3IgYWxsIHRoZSBwYWdl cyBpbiBhIG1lbWNnICovCitzdGF0aWMgaW5saW5lIGJvb2wgbWVtX2Nncm91cF90cnlsb2NrX3Bh Z2VzKHN0cnVjdCBtZW1fY2dyb3VwICptZW1jZykKK3sKKwlyY3VfcmVhZF9sb2NrKCk7CisKKwlp ZiAobWVtX2Nncm91cF9kaXNhYmxlZCgpIHx8ICFhdG9taWNfcmVhZCgmbWVtY2ctPm1vdmluZ19h Y2NvdW50KSkKKwkJcmV0dXJuIHRydWU7CisKKwlyY3VfcmVhZF91bmxvY2soKTsKKwlyZXR1cm4g ZmFsc2U7Cit9CisKK3N0YXRpYyBpbmxpbmUgdm9pZCBtZW1fY2dyb3VwX3VubG9ja19wYWdlcyh2 b2lkKQoreworCXJjdV9yZWFkX3VubG9jaygpOworfQorCiAvKiBpZHggY2FuIGJlIG9mIHR5cGUg ZW51bSBtZW1jZ19zdGF0X2l0ZW0gb3Igbm9kZV9zdGF0X2l0ZW0gKi8KIHN0YXRpYyBpbmxpbmUg dm9pZCBtb2RfbWVtY2dfc3RhdGUoc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnLAogCQkJCSAgIGlu dCBpZHgsIGludCB2YWwpCkBAIC0xMzg1LDYgKzE0MDQsMTggQEAgc3RhdGljIGlubGluZSB2b2lk IGZvbGlvX21lbWNnX3VubG9jayhzdHJ1Y3QgZm9saW8gKmZvbGlvKQogewogfQogCitzdGF0aWMg aW5saW5lIGJvb2wgbWVtX2Nncm91cF90cnlsb2NrX3BhZ2VzKHN0cnVjdCBtZW1fY2dyb3VwICpt ZW1jZykKK3sKKwkvKiB0byBtYXRjaCBmb2xpb19tZW1jZ19yY3UoKSAqLworCXJjdV9yZWFkX2xv Y2soKTsKKwlyZXR1cm4gdHJ1ZTsKK30KKworc3RhdGljIGlubGluZSB2b2lkIG1lbV9jZ3JvdXBf dW5sb2NrX3BhZ2VzKHZvaWQpCit7CisJcmN1X3JlYWRfdW5sb2NrKCk7Cit9CisKIHN0YXRpYyBp bmxpbmUgdm9pZCBtZW1fY2dyb3VwX2hhbmRsZV9vdmVyX2hpZ2godm9pZCkKIHsKIH0KZGlmZiAt LWdpdCBhL2luY2x1ZGUvbGludXgvbW0uaCBiL2luY2x1ZGUvbGludXgvbW0uaAppbmRleCA5ODBm NTY4MjA0YTMuLmJiOTA3MDg4Nzc3ZSAxMDA2NDQKLS0tIGEvaW5jbHVkZS9saW51eC9tbS5oCisr KyBiL2luY2x1ZGUvbGludXgvbW0uaApAQCAtMTUyMyw2ICsxNTIzLDExIEBAIHN0YXRpYyBpbmxp bmUgdW5zaWduZWQgbG9uZyBmb2xpb19wZm4oc3RydWN0IGZvbGlvICpmb2xpbykKIAlyZXR1cm4g cGFnZV90b19wZm4oJmZvbGlvLT5wYWdlKTsKIH0KIAorc3RhdGljIGlubGluZSBzdHJ1Y3QgZm9s aW8gKnBmbl9mb2xpbyh1bnNpZ25lZCBsb25nIHBmbikKK3sKKwlyZXR1cm4gcGFnZV9mb2xpbyhw Zm5fdG9fcGFnZShwZm4pKTsKK30KKwogc3RhdGljIGlubGluZSBhdG9taWNfdCAqZm9saW9fcGlu Y291bnRfcHRyKHN0cnVjdCBmb2xpbyAqZm9saW8pCiB7CiAJcmV0dXJuICZmb2xpb19wYWdlKGZv bGlvLCAxKS0+Y29tcG91bmRfcGluY291bnQ7CmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4L21t em9uZS5oIGIvaW5jbHVkZS9saW51eC9tbXpvbmUuaAppbmRleCBjOGE3Y2VlZTdhMGEuLmM2OTU4 OWFkMmIwNSAxMDA2NDQKLS0tIGEvaW5jbHVkZS9saW51eC9tbXpvbmUuaAorKysgYi9pbmNsdWRl L2xpbnV4L21tem9uZS5oCkBAIC0zNzEsNiArMzcxLDcgQEAgZW51bSBscnV2ZWNfZmxhZ3Mgewog I2lmbmRlZiBfX0dFTkVSQVRJTkdfQk9VTkRTX0gKIAogc3RydWN0IGxydXZlYzsKK3N0cnVjdCBw YWdlX3ZtYV9tYXBwZWRfd2FsazsKIAogI2RlZmluZSBMUlVfR0VOX01BU0sJCSgoQklUKExSVV9H RU5fV0lEVEgpIC0gMSkgPDwgTFJVX0dFTl9QR09GRikKICNkZWZpbmUgTFJVX1JFRlNfTUFTSwkJ KChCSVQoTFJVX1JFRlNfV0lEVEgpIC0gMSkgPDwgTFJVX1JFRlNfUEdPRkYpCkBAIC00MjQsNiAr NDI1LDcgQEAgc3RydWN0IGxydV9nZW5fc3RydWN0IHsKIH07CiAKIHZvaWQgbHJ1X2dlbl9pbml0 X2xydXZlYyhzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMpOwordm9pZCBscnVfZ2VuX2xvb2tfYXJvdW5k KHN0cnVjdCBwYWdlX3ZtYV9tYXBwZWRfd2FsayAqcHZtdyk7CiAKICNpZmRlZiBDT05GSUdfTUVN Q0cKIHZvaWQgbHJ1X2dlbl9pbml0X21lbWNnKHN0cnVjdCBtZW1fY2dyb3VwICptZW1jZyk7CkBA IC00MzYsNiArNDM4LDEwIEBAIHN0YXRpYyBpbmxpbmUgdm9pZCBscnVfZ2VuX2luaXRfbHJ1dmVj KHN0cnVjdCBscnV2ZWMgKmxydXZlYykKIHsKIH0KIAorc3RhdGljIGlubGluZSB2b2lkIGxydV9n ZW5fbG9va19hcm91bmQoc3RydWN0IHBhZ2Vfdm1hX21hcHBlZF93YWxrICpwdm13KQoreworfQor CiAjaWZkZWYgQ09ORklHX01FTUNHCiBzdGF0aWMgaW5saW5lIHZvaWQgbHJ1X2dlbl9pbml0X21l bWNnKHN0cnVjdCBtZW1fY2dyb3VwICptZW1jZykKIHsKZGlmZiAtLWdpdCBhL21tL2ludGVybmFs LmggYi9tbS9pbnRlcm5hbC5oCmluZGV4IGNmMTYyODBjZTEzMi4uNTlkMjQyMmI2NDdkIDEwMDY0 NAotLS0gYS9tbS9pbnRlcm5hbC5oCisrKyBiL21tL2ludGVybmFsLmgKQEAgLTY4LDYgKzY4LDcg QEAgdm1fZmF1bHRfdCBkb19zd2FwX3BhZ2Uoc3RydWN0IHZtX2ZhdWx0ICp2bWYpOwogdm9pZCBm b2xpb19yb3RhdGVfcmVjbGFpbWFibGUoc3RydWN0IGZvbGlvICpmb2xpbyk7CiBib29sIF9fZm9s aW9fZW5kX3dyaXRlYmFjayhzdHJ1Y3QgZm9saW8gKmZvbGlvKTsKIHZvaWQgZGVhY3RpdmF0ZV9m aWxlX2ZvbGlvKHN0cnVjdCBmb2xpbyAqZm9saW8pOwordm9pZCBmb2xpb19hY3RpdmF0ZShzdHJ1 Y3QgZm9saW8gKmZvbGlvKTsKIAogdm9pZCBmcmVlX3BndGFibGVzKHN0cnVjdCBtbXVfZ2F0aGVy ICp0bGIsIHN0cnVjdCB2bV9hcmVhX3N0cnVjdCAqc3RhcnRfdm1hLAogCQl1bnNpZ25lZCBsb25n IGZsb29yLCB1bnNpZ25lZCBsb25nIGNlaWxpbmcpOwpkaWZmIC0tZ2l0IGEvbW0vbWVtY29udHJv bC5jIGIvbW0vbWVtY29udHJvbC5jCmluZGV4IGY1ZGU4YmU4MGMxMy4uODA2OWI1OGYyNDIyIDEw MDY0NAotLS0gYS9tbS9tZW1jb250cm9sLmMKKysrIGIvbW0vbWVtY29udHJvbC5jCkBAIC0yNzU5 LDYgKzI3NTksNyBAQCBzdGF0aWMgdm9pZCBjb21taXRfY2hhcmdlKHN0cnVjdCBmb2xpbyAqZm9s aW8sIHN0cnVjdCBtZW1fY2dyb3VwICptZW1jZykKIAkgKiAtIExSVSBpc29sYXRpb24KIAkgKiAt IGxvY2tfcGFnZV9tZW1jZygpCiAJICogLSBleGNsdXNpdmUgcmVmZXJlbmNlCisJICogLSBtZW1f Y2dyb3VwX3RyeWxvY2tfcGFnZXMoKQogCSAqLwogCWZvbGlvLT5tZW1jZ19kYXRhID0gKHVuc2ln bmVkIGxvbmcpbWVtY2c7CiB9CmRpZmYgLS1naXQgYS9tbS9ybWFwLmMgYi9tbS9ybWFwLmMKaW5k ZXggZmVkYjgyMzcxZWZlLi43Y2I3ZWYyOTA4OGEgMTAwNjQ0Ci0tLSBhL21tL3JtYXAuYworKysg Yi9tbS9ybWFwLmMKQEAgLTczLDYgKzczLDcgQEAKICNpbmNsdWRlIDxsaW51eC9wYWdlX2lkbGUu aD4KICNpbmNsdWRlIDxsaW51eC9tZW1yZW1hcC5oPgogI2luY2x1ZGUgPGxpbnV4L3VzZXJmYXVs dGZkX2suaD4KKyNpbmNsdWRlIDxsaW51eC9tbV9pbmxpbmUuaD4KIAogI2luY2x1ZGUgPGFzbS90 bGJmbHVzaC5oPgogCkBAIC04MjEsNiArODIyLDEyIEBAIHN0YXRpYyBib29sIGZvbGlvX3JlZmVy ZW5jZWRfb25lKHN0cnVjdCBmb2xpbyAqZm9saW8sCiAJCX0KIAogCQlpZiAocHZtdy5wdGUpIHsK KwkJCWlmIChscnVfZ2VuX2VuYWJsZWQoKSAmJiBwdGVfeW91bmcoKnB2bXcucHRlKSAmJgorCQkJ ICAgICEodm1hLT52bV9mbGFncyAmIChWTV9TRVFfUkVBRCB8IFZNX1JBTkRfUkVBRCkpKSB7CisJ CQkJbHJ1X2dlbl9sb29rX2Fyb3VuZCgmcHZtdyk7CisJCQkJcmVmZXJlbmNlZCsrOworCQkJfQor CiAJCQlpZiAocHRlcF9jbGVhcl9mbHVzaF95b3VuZ19ub3RpZnkodm1hLCBhZGRyZXNzLAogCQkJ CQkJcHZtdy5wdGUpKSB7CiAJCQkJLyoKZGlmZiAtLWdpdCBhL21tL3N3YXAuYyBiL21tL3N3YXAu YwppbmRleCA2YTUyMDNmMThiMGEuLmMwZTQwNjBiYmJlMCAxMDA2NDQKLS0tIGEvbW0vc3dhcC5j CisrKyBiL21tL3N3YXAuYwpAQCAtMzQyLDcgKzM0Miw3IEBAIHN0YXRpYyBib29sIG5lZWRfYWN0 aXZhdGVfcGFnZV9kcmFpbihpbnQgY3B1KQogCXJldHVybiBwYWdldmVjX2NvdW50KCZwZXJfY3B1 KGxydV9wdmVjcy5hY3RpdmF0ZV9wYWdlLCBjcHUpKSAhPSAwOwogfQogCi1zdGF0aWMgdm9pZCBm b2xpb19hY3RpdmF0ZShzdHJ1Y3QgZm9saW8gKmZvbGlvKQordm9pZCBmb2xpb19hY3RpdmF0ZShz dHJ1Y3QgZm9saW8gKmZvbGlvKQogewogCWlmIChmb2xpb190ZXN0X2xydShmb2xpbykgJiYgIWZv bGlvX3Rlc3RfYWN0aXZlKGZvbGlvKSAmJgogCSAgICAhZm9saW9fdGVzdF91bmV2aWN0YWJsZShm b2xpbykpIHsKQEAgLTM2Miw3ICszNjIsNyBAQCBzdGF0aWMgaW5saW5lIHZvaWQgYWN0aXZhdGVf cGFnZV9kcmFpbihpbnQgY3B1KQogewogfQogCi1zdGF0aWMgdm9pZCBmb2xpb19hY3RpdmF0ZShz dHJ1Y3QgZm9saW8gKmZvbGlvKQordm9pZCBmb2xpb19hY3RpdmF0ZShzdHJ1Y3QgZm9saW8gKmZv bGlvKQogewogCXN0cnVjdCBscnV2ZWMgKmxydXZlYzsKIApkaWZmIC0tZ2l0IGEvbW0vdm1zY2Fu LmMgYi9tbS92bXNjYW4uYwppbmRleCBiYjNkNzA1YzUyODIuLmY0ZGQzYzNjNTg5YiAxMDA2NDQK LS0tIGEvbW0vdm1zY2FuLmMKKysrIGIvbW0vdm1zY2FuLmMKQEAgLTE1NTQsNiArMTU1NCwxMSBA QCBzdGF0aWMgdW5zaWduZWQgaW50IHNocmlua19wYWdlX2xpc3Qoc3RydWN0IGxpc3RfaGVhZCAq cGFnZV9saXN0LAogCQlpZiAoIXNjLT5tYXlfdW5tYXAgJiYgcGFnZV9tYXBwZWQocGFnZSkpCiAJ CQlnb3RvIGtlZXBfbG9ja2VkOwogCisJCS8qIGZvbGlvX3VwZGF0ZV9nZW4oKSB0cmllZCB0byBw cm9tb3RlIHRoaXMgcGFnZT8gKi8KKwkJaWYgKGxydV9nZW5fZW5hYmxlZCgpICYmICFpZ25vcmVf cmVmZXJlbmNlcyAmJgorCQkgICAgcGFnZV9tYXBwZWQocGFnZSkgJiYgUGFnZVJlZmVyZW5jZWQo cGFnZSkpCisJCQlnb3RvIGtlZXBfbG9ja2VkOworCiAJCW1heV9lbnRlcl9mcyA9IChzYy0+Z2Zw X21hc2sgJiBfX0dGUF9GUykgfHwKIAkJCShQYWdlU3dhcENhY2hlKHBhZ2UpICYmIChzYy0+Z2Zw X21hc2sgJiBfX0dGUF9JTykpOwogCkBAIC0zMTUzLDYgKzMxNTgsMzEgQEAgc3RhdGljIGJvb2wg cG9zaXRpdmVfY3RybF9lcnIoc3RydWN0IGN0cmxfcG9zICpzcCwgc3RydWN0IGN0cmxfcG9zICpw dikKICAqICAgICAgICAgICAgICAgICAgICAgICAgICB0aGUgYWdpbmcKICAqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKiovCiAKK3N0YXRpYyBpbnQgZm9saW9fdXBkYXRlX2dlbihzdHJ1Y3QgZm9saW8gKmZv bGlvLCBpbnQgZ2VuKQoreworCXVuc2lnbmVkIGxvbmcgb2xkX2ZsYWdzLCBuZXdfZmxhZ3M7CisK KwlWTV9CVUdfT04oZ2VuID49IE1BWF9OUl9HRU5TKTsKKwlWTV9CVUdfT04oIXJjdV9yZWFkX2xv Y2tfaGVsZCgpKTsKKworCWRvIHsKKwkJbmV3X2ZsYWdzID0gb2xkX2ZsYWdzID0gUkVBRF9PTkNF KGZvbGlvLT5mbGFncyk7CisKKwkJLyogZm9yIHNocmlua19wYWdlX2xpc3QoKSAqLworCQlpZiAo IShuZXdfZmxhZ3MgJiBMUlVfR0VOX01BU0spKSB7CisJCQluZXdfZmxhZ3MgfD0gQklUKFBHX3Jl ZmVyZW5jZWQpOworCQkJY29udGludWU7CisJCX0KKworCQluZXdfZmxhZ3MgJj0gfkxSVV9HRU5f TUFTSzsKKwkJbmV3X2ZsYWdzIHw9IChnZW4gKyAxVUwpIDw8IExSVV9HRU5fUEdPRkY7CisJCW5l d19mbGFncyAmPSB+KExSVV9SRUZTX01BU0sgfCBMUlVfUkVGU19GTEFHUyk7CisJfSB3aGlsZSAo bmV3X2ZsYWdzICE9IG9sZF9mbGFncyAmJgorCQkgY21weGNoZygmZm9saW8tPmZsYWdzLCBvbGRf ZmxhZ3MsIG5ld19mbGFncykgIT0gb2xkX2ZsYWdzKTsKKworCXJldHVybiAoKG9sZF9mbGFncyAm IExSVV9HRU5fTUFTSykgPj4gTFJVX0dFTl9QR09GRikgLSAxOworfQorCiBzdGF0aWMgaW50IGZv bGlvX2luY19nZW4oc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3QgZm9saW8gKmZvbGlvLCBi b29sIHJlY2xhaW1pbmcpCiB7CiAJdW5zaWduZWQgbG9uZyBvbGRfZmxhZ3MsIG5ld19mbGFnczsK QEAgLTMxNjQsNiArMzE5NCwxMSBAQCBzdGF0aWMgaW50IGZvbGlvX2luY19nZW4oc3RydWN0IGxy dXZlYyAqbHJ1dmVjLCBzdHJ1Y3QgZm9saW8gKmZvbGlvLCBib29sIHJlY2xhaQogCQluZXdfZmxh Z3MgPSBvbGRfZmxhZ3MgPSBSRUFEX09OQ0UoZm9saW8tPmZsYWdzKTsKIAkJVk1fQlVHX09OX0ZP TElPKCEobmV3X2ZsYWdzICYgTFJVX0dFTl9NQVNLKSwgZm9saW8pOwogCisJCW5ld19nZW4gPSAo KG5ld19mbGFncyAmIExSVV9HRU5fTUFTSykgPj4gTFJVX0dFTl9QR09GRikgLSAxOworCQkvKiBm b2xpb191cGRhdGVfZ2VuKCkgaGFzIHByb21vdGVkIHRoaXMgcGFnZT8gKi8KKwkJaWYgKG5ld19n ZW4gPj0gMCAmJiBuZXdfZ2VuICE9IG9sZF9nZW4pCisJCQlyZXR1cm4gbmV3X2dlbjsKKwogCQlu ZXdfZ2VuID0gKG9sZF9nZW4gKyAxKSAlIE1BWF9OUl9HRU5TOwogCiAJCW5ld19mbGFncyAmPSB+ TFJVX0dFTl9NQVNLOwpAQCAtMzM4NSw2ICszNDIwLDEyNSBAQCBzdGF0aWMgdm9pZCBscnVfZ2Vu X2FnZV9ub2RlKHN0cnVjdCBwZ2xpc3RfZGF0YSAqcGdkYXQsIHN0cnVjdCBzY2FuX2NvbnRyb2wg KnNjKQogCX0gd2hpbGUgKChtZW1jZyA9IG1lbV9jZ3JvdXBfaXRlcihOVUxMLCBtZW1jZywgTlVM TCkpKTsKIH0KIAorLyoKKyAqIFRoaXMgZnVuY3Rpb24gZXhwbG9pdHMgc3BhdGlhbCBsb2NhbGl0 eSB3aGVuIHNocmlua19wYWdlX2xpc3QoKSB3YWxrcyB0aGUKKyAqIHJtYXAuIEl0IHNjYW5zIHRo ZSBhZGphY2VudCBQVEVzIG9mIGEgeW91bmcgUFRFIGFuZCBwcm9tb3RlcyBob3QgcGFnZXMuCisg Ki8KK3ZvaWQgbHJ1X2dlbl9sb29rX2Fyb3VuZChzdHJ1Y3QgcGFnZV92bWFfbWFwcGVkX3dhbGsg KnB2bXcpCit7CisJaW50IGk7CisJcHRlX3QgKnB0ZTsKKwl1bnNpZ25lZCBsb25nIHN0YXJ0Owor CXVuc2lnbmVkIGxvbmcgZW5kOworCXVuc2lnbmVkIGxvbmcgYWRkcjsKKwl1bnNpZ25lZCBsb25n IGJpdG1hcFtCSVRTX1RPX0xPTkdTKE1JTl9MUlVfQkFUQ0gpXSA9IHt9OworCXN0cnVjdCBmb2xp byAqZm9saW8gPSBwZm5fZm9saW8ocHZtdy0+cGZuKTsKKwlzdHJ1Y3QgbWVtX2Nncm91cCAqbWVt Y2cgPSBmb2xpb19tZW1jZyhmb2xpbyk7CisJc3RydWN0IHBnbGlzdF9kYXRhICpwZ2RhdCA9IGZv bGlvX3BnZGF0KGZvbGlvKTsKKwlzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMgPSBtZW1fY2dyb3VwX2xy dXZlYyhtZW1jZywgcGdkYXQpOworCURFRklORV9NQVhfU0VRKGxydXZlYyk7CisJaW50IG9sZF9n ZW4sIG5ld19nZW4gPSBscnVfZ2VuX2Zyb21fc2VxKG1heF9zZXEpOworCisJbG9ja2RlcF9hc3Nl cnRfaGVsZChwdm13LT5wdGwpOworCVZNX0JVR19PTl9GT0xJTyhmb2xpb190ZXN0X2xydShmb2xp byksIGZvbGlvKTsKKworCWlmIChzcGluX2lzX2NvbnRlbmRlZChwdm13LT5wdGwpKQorCQlyZXR1 cm47CisKKwlzdGFydCA9IG1heChwdm13LT5hZGRyZXNzICYgUE1EX01BU0ssIHB2bXctPnZtYS0+ dm1fc3RhcnQpOworCWVuZCA9IHBtZF9hZGRyX2VuZChwdm13LT5hZGRyZXNzLCBwdm13LT52bWEt PnZtX2VuZCk7CisKKwlpZiAoZW5kIC0gc3RhcnQgPiBNSU5fTFJVX0JBVENIICogUEFHRV9TSVpF KSB7CisJCWlmIChwdm13LT5hZGRyZXNzIC0gc3RhcnQgPCBNSU5fTFJVX0JBVENIICogUEFHRV9T SVpFIC8gMikKKwkJCWVuZCA9IHN0YXJ0ICsgTUlOX0xSVV9CQVRDSCAqIFBBR0VfU0laRTsKKwkJ ZWxzZSBpZiAoZW5kIC0gcHZtdy0+YWRkcmVzcyA8IE1JTl9MUlVfQkFUQ0ggKiBQQUdFX1NJWkUg LyAyKQorCQkJc3RhcnQgPSBlbmQgLSBNSU5fTFJVX0JBVENIICogUEFHRV9TSVpFOworCQllbHNl IHsKKwkJCXN0YXJ0ID0gcHZtdy0+YWRkcmVzcyAtIE1JTl9MUlVfQkFUQ0ggKiBQQUdFX1NJWkUg LyAyOworCQkJZW5kID0gcHZtdy0+YWRkcmVzcyArIE1JTl9MUlVfQkFUQ0ggKiBQQUdFX1NJWkUg LyAyOworCQl9CisJfQorCisJcHRlID0gcHZtdy0+cHRlIC0gKHB2bXctPmFkZHJlc3MgLSBzdGFy dCkgLyBQQUdFX1NJWkU7CisKKwlyY3VfcmVhZF9sb2NrKCk7CisJYXJjaF9lbnRlcl9sYXp5X21t dV9tb2RlKCk7CisKKwlmb3IgKGkgPSAwLCBhZGRyID0gc3RhcnQ7IGFkZHIgIT0gZW5kOyBpKyss IGFkZHIgKz0gUEFHRV9TSVpFKSB7CisJCXVuc2lnbmVkIGxvbmcgcGZuID0gcHRlX3BmbihwdGVb aV0pOworCisJCVZNX0JVR19PTihhZGRyIDwgcHZtdy0+dm1hLT52bV9zdGFydCB8fCBhZGRyID49 IHB2bXctPnZtYS0+dm1fZW5kKTsKKworCQlpZiAoIXB0ZV9wcmVzZW50KHB0ZVtpXSkgfHwgaXNf emVyb19wZm4ocGZuKSkKKwkJCWNvbnRpbnVlOworCisJCWlmIChXQVJOX09OX09OQ0UocHRlX2Rl dm1hcChwdGVbaV0pIHx8IHB0ZV9zcGVjaWFsKHB0ZVtpXSkpKQorCQkJY29udGludWU7CisKKwkJ aWYgKCFwdGVfeW91bmcocHRlW2ldKSkKKwkJCWNvbnRpbnVlOworCisJCVZNX0JVR19PTighcGZu X3ZhbGlkKHBmbikpOworCQlpZiAocGZuIDwgcGdkYXQtPm5vZGVfc3RhcnRfcGZuIHx8IHBmbiA+ PSBwZ2RhdF9lbmRfcGZuKHBnZGF0KSkKKwkJCWNvbnRpbnVlOworCisJCWZvbGlvID0gcGZuX2Zv bGlvKHBmbik7CisJCWlmIChmb2xpb19uaWQoZm9saW8pICE9IHBnZGF0LT5ub2RlX2lkKQorCQkJ Y29udGludWU7CisKKwkJaWYgKGZvbGlvX21lbWNnX3JjdShmb2xpbykgIT0gbWVtY2cpCisJCQlj b250aW51ZTsKKworCQlpZiAoIXB0ZXBfdGVzdF9hbmRfY2xlYXJfeW91bmcocHZtdy0+dm1hLCBh ZGRyLCBwdGUgKyBpKSkKKwkJCWNvbnRpbnVlOworCisJCWlmIChwdGVfZGlydHkocHRlW2ldKSAm JiAhZm9saW9fdGVzdF9kaXJ0eShmb2xpbykgJiYKKwkJICAgICEoZm9saW9fdGVzdF9hbm9uKGZv bGlvKSAmJiBmb2xpb190ZXN0X3N3YXBiYWNrZWQoZm9saW8pICYmCisJCSAgICAgICFmb2xpb190 ZXN0X3N3YXBjYWNoZShmb2xpbykpKQorCQkJZm9saW9fbWFya19kaXJ0eShmb2xpbyk7CisKKwkJ b2xkX2dlbiA9IGZvbGlvX2xydV9nZW4oZm9saW8pOworCQlpZiAob2xkX2dlbiA8IDApCisJCQlm b2xpb19zZXRfcmVmZXJlbmNlZChmb2xpbyk7CisJCWVsc2UgaWYgKG9sZF9nZW4gIT0gbmV3X2dl bikKKwkJCV9fc2V0X2JpdChpLCBiaXRtYXApOworCX0KKworCWFyY2hfbGVhdmVfbGF6eV9tbXVf bW9kZSgpOworCXJjdV9yZWFkX3VubG9jaygpOworCisJaWYgKGJpdG1hcF93ZWlnaHQoYml0bWFw LCBNSU5fTFJVX0JBVENIKSA8IFBBR0VWRUNfU0laRSkgeworCQlmb3JfZWFjaF9zZXRfYml0KGks IGJpdG1hcCwgTUlOX0xSVV9CQVRDSCkgeworCQkJZm9saW8gPSBwZm5fZm9saW8ocHRlX3Bmbihw dGVbaV0pKTsKKwkJCWZvbGlvX2FjdGl2YXRlKGZvbGlvKTsKKwkJfQorCQlyZXR1cm47CisJfQor CisJLyogZm9saW9fdXBkYXRlX2dlbigpIHJlcXVpcmVzIHN0YWJsZSBmb2xpb19tZW1jZygpICov CisJaWYgKCFtZW1fY2dyb3VwX3RyeWxvY2tfcGFnZXMobWVtY2cpKQorCQlyZXR1cm47CisKKwlz cGluX2xvY2tfaXJxKCZscnV2ZWMtPmxydV9sb2NrKTsKKwluZXdfZ2VuID0gbHJ1X2dlbl9mcm9t X3NlcShscnV2ZWMtPmxydWdlbi5tYXhfc2VxKTsKKworCWZvcl9lYWNoX3NldF9iaXQoaSwgYml0 bWFwLCBNSU5fTFJVX0JBVENIKSB7CisJCWZvbGlvID0gcGZuX2ZvbGlvKHB0ZV9wZm4ocHRlW2ld KSk7CisJCWlmIChmb2xpb19tZW1jZ19yY3UoZm9saW8pICE9IG1lbWNnKQorCQkJY29udGludWU7 CisKKwkJb2xkX2dlbiA9IGZvbGlvX3VwZGF0ZV9nZW4oZm9saW8sIG5ld19nZW4pOworCQlpZiAo b2xkX2dlbiA8IDAgfHwgb2xkX2dlbiA9PSBuZXdfZ2VuKQorCQkJY29udGludWU7CisKKwkJbHJ1 X2dlbl91cGRhdGVfc2l6ZShscnV2ZWMsIGZvbGlvLCBvbGRfZ2VuLCBuZXdfZ2VuKTsKKwl9CisK KwlzcGluX3VubG9ja19pcnEoJmxydXZlYy0+bHJ1X2xvY2spOworCisJbWVtX2Nncm91cF91bmxv Y2tfcGFnZXMoKTsKK30KKwogLyoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKgogICogICAgICAgICAgICAg ICAgICAgICAgICAgIHRoZSBldmljdGlvbgogICoqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKi8KQEAgLTM0 MTgsNiArMzU3MiwxMSBAQCBzdGF0aWMgYm9vbCBzb3J0X2ZvbGlvKHN0cnVjdCBscnV2ZWMgKmxy dXZlYywgc3RydWN0IGZvbGlvICpmb2xpbywgaW50IHRpZXJfaWR4KQogCQlyZXR1cm4gdHJ1ZTsK IAl9CiAKKwlpZiAoZ2VuICE9IGxydV9nZW5fZnJvbV9zZXEobHJ1Z2VuLT5taW5fc2VxW3R5cGVd KSkgeworCQlsaXN0X21vdmUoJmZvbGlvLT5scnUsICZscnVnZW4tPmxpc3RzW2dlbl1bdHlwZV1b em9uZV0pOworCQlyZXR1cm4gdHJ1ZTsKKwl9CisKIAlpZiAodGllciA+IHRpZXJfaWR4KSB7CiAJ CWludCBoaXN0ID0gbHJ1X2hpc3RfZnJvbV9zZXEobHJ1Z2VuLT5taW5fc2VxW3R5cGVdKTsKIAot LSAKMi4zNS4xLjEwOTQuZzdjN2Q5MDJhN2MtZ29vZwoKCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0Cmxp bnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFk Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK