From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BE78C3F6B0 for ; Mon, 15 Aug 2022 07:15:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S241504AbiHOHPI (ORCPT ); Mon, 15 Aug 2022 03:15:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46780 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229527AbiHOHPD (ORCPT ); Mon, 15 Aug 2022 03:15:03 -0400 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2B4001C925 for ; Mon, 15 Aug 2022 00:14:23 -0700 (PDT) Received: by mail-yb1-xb49.google.com with SMTP id 207-20020a2505d8000000b0067709d8d3eeso5406847ybf.18 for ; Mon, 15 Aug 2022 00:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:from:to:cc; bh=OG21aV95NWHOo7NMaXGtZDeDVd1PZFs7RFAu3qrJU9s=; b=WHe/o94b6sSWCNtVsvdXInF7Bs9OcbJU1dCj5DMXBNlfz6RhO+0CKOBLCNAPaeawPG psfFyfr3nAdZuKV9pWqSj24SWqUznqL4Tav55NsT9qgTWqeNVABTbOJnaZzcPELXEcB2 GKWQqthHvAhQn69blnDjEgYv96hlJasH5D45Wq24AF+9WLmynB45huVm2am+fMtg5dZ2 lhYGstyBsOHK7LAOQ1wnldaY0xWxOrEWRQQgjq9DlQ+T+z5wqRXFVhXMtMEO8ZHbrlVV AY5uxXmoml/qiGfZT1IhTHRzXxE/3uNBknjngm1EE+mdcivTWPERJ3sQpHNvGdadB3BW v4Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:x-gm-message-state:from:to :cc; bh=OG21aV95NWHOo7NMaXGtZDeDVd1PZFs7RFAu3qrJU9s=; b=5RDpEvirxLFoYC8ibiPLTq7iCQ5anzKHTVKpamLerp5uvRgV20/eLOceMTDNTPfpbe Kx7niUVFDsDKj3lXExjPNBbyAoIarlhRhpG4q6WfhJ4sL+L0i0TrKEp+kU9+njw+CxO0 TDewM87J+JGN7PEQEp22M8YFOEA2q1XXCkosoByfCy2ishVJvPaVF4lV4J3s4c7Hci3U G7cTsd1Mnr+ySG+im1wmsFbEksWPgb/X3oOXbDpceDGQQXwcFsutX+oXXTOUKsXbdbVY +l9Oq8ILKtmmvQ/NcyXV+xOTyXEAFA95rZYdW+DTpZ4MELt9gW0MWQA3bMTMvIYO8RDw A8pA== X-Gm-Message-State: ACgBeo3TMkIZ72cf671nNhomHTuJKYDUAhgOGYhZKjAs2hI0mGZ+2Rm7 GqpVQsBnLDF7OBcjDSpSr/vDVGWli08= X-Google-Smtp-Source: AA6agR7AzqnhxIvkLjTuH7X/p83y5zjJA+3i7KoHjwOuPi8aLZoBUFdTyHtN9BRaL1hIyv7TAKS5Egbks70= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:d91:5887:ac93:ddf0]) (user=yuzhao job=sendgmr) by 2002:a25:424b:0:b0:687:5558:4e7b with SMTP id p72-20020a25424b000000b0068755584e7bmr4905422yba.434.1660547662335; Mon, 15 Aug 2022 00:14:22 -0700 (PDT) Date: Mon, 15 Aug 2022 01:13:25 -0600 In-Reply-To: <20220815071332.627393-1-yuzhao@google.com> Message-Id: <20220815071332.627393-7-yuzhao@google.com> Mime-Version: 1.0 References: <20220815071332.627393-1-yuzhao@google.com> X-Mailer: git-send-email 2.37.1.595.g718a3a8f04-goog Subject: [PATCH v14 06/14] mm: multi-gen LRU: minimal implementation From: Yu Zhao To: Andrew Morton Cc: Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Peter Zijlstra , Tejun Heo , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, page-reclaim@google.com, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To avoid confusion, the terms "promotion" and "demotion" will be applied to the multi-gen LRU, as a new convention; the terms "activation" and "deactivation" will be applied to the active/inactive LRU, as usual. The aging produces young generations. Given an lruvec, it increments max_seq when max_seq-min_seq+1 approaches MIN_NR_GENS. The aging promotes hot pages to the youngest generation when it finds them accessed through page tables; the demotion of cold pages happens consequently when it increments max_seq. Promotion in the aging path does not involve any LRU list operations, only the updates of the gen counter and lrugen->nr_pages[]; demotion, unless as the result of the increment of max_seq, requires LRU list operations, e.g., lru_deactivate_fn(). The aging has the complexity O(nr_hot_pages), since it is only interested in hot pages. The eviction consumes old generations. Given an lruvec, it increments min_seq when lrugen->lists[] indexed by min_seq%MAX_NR_GENS becomes empty. A feedback loop modeled after the PID controller monitors refaults over anon and file types and decides which type to evict when both types are available from the same generation. The protection of pages accessed multiple times through file descriptors takes place in the eviction path. Each generation is divided into multiple tiers. A page accessed N times through file descriptors is in tier order_base_2(N). Tiers do not have dedicated lrugen->lists[], only bits in folio->flags. The aforementioned feedback loop also monitors refaults over all tiers and decides when to protect pages in which tiers (N>1), using the first tier (N=3D0,1) as a baseline. The first tier contains single-use unmapped clean pages, which are most likely the best choices. In contrast to promotion in the aging path, the protection of a page in the eviction path is achieved by moving this page to the next generation, i.e., min_seq+1, if the feedback loop decides so. This approach has the following advantages: 1. It removes the cost of activation in the buffered access path by inferring whether pages accessed multiple times through file descriptors are statistically hot and thus worth protecting in the eviction path. 2. It takes pages accessed through page tables into account and avoids overprotecting pages accessed multiple times through file descriptors. (Pages accessed through page tables are in the first tier, since N=3D0.) 3. More tiers provide better protection for pages accessed more than twice through file descriptors, when under heavy buffered I/O workloads. Server benchmark results: Single workload: fio (buffered I/O): +[30, 32]% IOPS BW 5.19-rc1: 2673k 10.2GiB/s patch1-6: 3491k 13.3GiB/s Single workload: memcached (anon): -[4, 6]% Ops/sec KB/sec 5.19-rc1: 1161501.04 45177.25 patch1-6: 1106168.46 43025.04 Configurations: CPU: two Xeon 6154 Mem: total 256G Node 1 was only used as a ram disk to reduce the variance in the results. patch drivers/block/brd.c < gfp_flags =3D GFP_NOIO | __GFP_ZERO | __GFP_HIGHMEM | __GFP_THISNODE= ; > page =3D alloc_pages_node(1, gfp_flags, 0); EOF cat >>/etc/systemd/system.conf <>/etc/memcached.conf </sys/fs/cgroup/user.slice/test/memory.max echo $$ >/sys/fs/cgroup/user.slice/test/cgroup.procs fio -name=3Dmglru --numjobs=3D72 --directory=3D/mnt --size=3D1408m \ --buffered=3D1 --ioengine=3Dio_uring --iodepth=3D128 \ --iodepth_batch_submit=3D32 --iodepth_batch_complete=3D32 \ --rw=3Drandread --random_distribution=3Drandom --norandommap \ --time_based --ramp_time=3D10m --runtime=3D5m --group_reporting cat memcached.sh modprobe brd rd_nr=3D1 rd_size=3D113246208 swapoff -a mkswap /dev/ram0 swapon /dev/ram0 memtier_benchmark -S /var/run/memcached/memcached.sock \ -P memcache_binary -n allkeys --key-minimum=3D1 \ --key-maximum=3D65000000 --key-pattern=3DP:P -c 1 -t 36 \ --ratio 1:0 --pipeline 8 -d 2000 memtier_benchmark -S /var/run/memcached/memcached.sock \ -P memcache_binary -n allkeys --key-minimum=3D1 \ --key-maximum=3D65000000 --key-pattern=3DR:R -c 1 -t 36 \ --ratio 0:1 --pipeline 8 --randomize --distinct-client-seed Client benchmark results: kswapd profiles: 5.19-rc1 40.33% page_vma_mapped_walk (overhead) 21.80% lzo1x_1_do_compress (real work) 7.53% do_raw_spin_lock 3.95% _raw_spin_unlock_irq 2.52% vma_interval_tree_iter_next 2.37% folio_referenced_one 2.28% vma_interval_tree_subtree_search 1.97% anon_vma_interval_tree_iter_first 1.60% ptep_clear_flush 1.06% __zram_bvec_write patch1-6 39.03% lzo1x_1_do_compress (real work) 18.47% page_vma_mapped_walk (overhead) 6.74% _raw_spin_unlock_irq 3.97% do_raw_spin_lock 2.49% ptep_clear_flush 2.48% anon_vma_interval_tree_iter_first 1.92% folio_referenced_one 1.88% __zram_bvec_write 1.48% memmove 1.31% vma_interval_tree_iter_next Configurations: CPU: single Snapdragon 7c Mem: total 4G Chrome OS MemoryPressure [1] [1] https://chromium.googlesource.com/chromiumos/platform/tast-tests/ Signed-off-by: Yu Zhao Acked-by: Brian Geffon Acked-by: Jan Alexander Steffens (heftig) Acked-by: Oleksandr Natalenko Acked-by: Steven Barrett Acked-by: Suleiman Souhlal Tested-by: Daniel Byrne Tested-by: Donald Carr Tested-by: Holger Hoffst=C3=A4tte Tested-by: Konstantin Kharlamov Tested-by: Shuang Zhai Tested-by: Sofia Trinh Tested-by: Vaibhav Jain --- include/linux/mm_inline.h | 36 ++ include/linux/mmzone.h | 41 ++ include/linux/page-flags-layout.h | 5 +- kernel/bounds.c | 2 + mm/Kconfig | 11 + mm/swap.c | 39 ++ mm/vmscan.c | 815 +++++++++++++++++++++++++++++- mm/workingset.c | 110 +++- 8 files changed, 1049 insertions(+), 10 deletions(-) diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h index 2ff703900fd0..f2b2296a42f9 100644 --- a/include/linux/mm_inline.h +++ b/include/linux/mm_inline.h @@ -121,6 +121,33 @@ static inline int lru_gen_from_seq(unsigned long seq) return seq % MAX_NR_GENS; } =20 +static inline int lru_hist_from_seq(unsigned long seq) +{ + return seq % NR_HIST_GENS; +} + +static inline int lru_tier_from_refs(int refs) +{ + VM_WARN_ON_ONCE(refs > BIT(LRU_REFS_WIDTH)); + + /* see the comment in folio_lru_refs() */ + return order_base_2(refs + 1); +} + +static inline int folio_lru_refs(struct folio *folio) +{ + unsigned long flags =3D READ_ONCE(folio->flags); + bool workingset =3D flags & BIT(PG_workingset); + + /* + * Return the number of accesses beyond PG_referenced, i.e., N-1 if the + * total number of accesses is N>1, since N=3D0,1 both map to the first + * tier. lru_tier_from_refs() will account for this off-by-one. Also see + * the comment on MAX_NR_TIERS. + */ + return ((flags & LRU_REFS_MASK) >> LRU_REFS_PGOFF) + workingset; +} + static inline int folio_lru_gen(struct folio *folio) { unsigned long flags =3D READ_ONCE(folio->flags); @@ -173,6 +200,15 @@ static inline void lru_gen_update_size(struct lruvec *= lruvec, struct folio *foli __update_lru_size(lruvec, lru, zone, -delta); return; } + + /* promotion */ + if (!lru_gen_is_active(lruvec, old_gen) && lru_gen_is_active(lruvec, new_= gen)) { + __update_lru_size(lruvec, lru, zone, -delta); + __update_lru_size(lruvec, lru + LRU_ACTIVE, zone, delta); + } + + /* demotion requires isolation, e.g., lru_deactivate_fn() */ + VM_WARN_ON_ONCE(lru_gen_is_active(lruvec, old_gen) && !lru_gen_is_active(= lruvec, new_gen)); } =20 static inline bool lru_gen_add_folio(struct lruvec *lruvec, struct folio *= folio, bool reclaiming) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 86147f04bf76..019d7c8ee834 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -350,6 +350,28 @@ enum lruvec_flags { #define MIN_NR_GENS 2U #define MAX_NR_GENS 4U =20 +/* + * Each generation is divided into multiple tiers. A page accessed N times + * through file descriptors is in tier order_base_2(N). A page in the firs= t tier + * (N=3D0,1) is marked by PG_referenced unless it was faulted in through p= age + * tables or read ahead. A page in any other tier (N>1) is marked by + * PG_referenced and PG_workingset. This implies a minimum of two tiers is + * supported without using additional bits in folio->flags. + * + * In contrast to moving across generations which requires the LRU lock, m= oving + * across tiers only involves atomic operations on folio->flags and theref= ore + * has a negligible cost in the buffered access path. In the eviction path= , + * comparisons of refaulted/(evicted+protected) from the first tier and th= e + * rest infer whether pages accessed multiple times through file descripto= rs + * are statistically hot and thus worth protecting. + * + * MAX_NR_TIERS is set to 4 so that the multi-gen LRU can support twice th= e + * number of categories of the active/inactive LRU when keeping track of + * accesses through file descriptors. This uses MAX_NR_TIERS-2 spare bits = in + * folio->flags. + */ +#define MAX_NR_TIERS 4U + #ifndef __GENERATING_BOUNDS_H =20 struct lruvec; @@ -364,6 +386,16 @@ enum { LRU_GEN_FILE, }; =20 +#define MIN_LRU_BATCH BITS_PER_LONG +#define MAX_LRU_BATCH (MIN_LRU_BATCH * 128) + +/* whether to keep historical stats from evicted generations */ +#ifdef CONFIG_LRU_GEN_STATS +#define NR_HIST_GENS MAX_NR_GENS +#else +#define NR_HIST_GENS 1U +#endif + /* * The youngest generation number is stored in max_seq for both anon and f= ile * types as they are aged on an equal footing. The oldest generation numbe= rs are @@ -386,6 +418,15 @@ struct lru_gen_struct { struct list_head lists[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; /* the multi-gen LRU sizes, eventually consistent */ long nr_pages[MAX_NR_GENS][ANON_AND_FILE][MAX_NR_ZONES]; + /* the exponential moving average of refaulted */ + unsigned long avg_refaulted[ANON_AND_FILE][MAX_NR_TIERS]; + /* the exponential moving average of evicted+protected */ + unsigned long avg_total[ANON_AND_FILE][MAX_NR_TIERS]; + /* the first tier doesn't need protection, hence the minus one */ + unsigned long protected[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS - 1]; + /* can be modified without holding the LRU lock */ + atomic_long_t evicted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS]; + atomic_long_t refaulted[NR_HIST_GENS][ANON_AND_FILE][MAX_NR_TIERS]; }; =20 void lru_gen_init_lruvec(struct lruvec *lruvec); diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-l= ayout.h index 240905407a18..7d79818dc065 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -106,7 +106,10 @@ #error "Not enough bits in page flags" #endif =20 -#define LRU_REFS_WIDTH 0 +/* see the comment on MAX_NR_TIERS */ +#define LRU_REFS_WIDTH min(__LRU_REFS_WIDTH, BITS_PER_LONG - NR_PAGEFLAGS = - \ + ZONES_WIDTH - LRU_GEN_WIDTH - SECTIONS_WIDTH - \ + NODES_WIDTH - KASAN_TAG_WIDTH - LAST_CPUPID_WIDTH) =20 #endif #endif /* _LINUX_PAGE_FLAGS_LAYOUT */ diff --git a/kernel/bounds.c b/kernel/bounds.c index 5ee60777d8e4..b529182e8b04 100644 --- a/kernel/bounds.c +++ b/kernel/bounds.c @@ -24,8 +24,10 @@ int main(void) DEFINE(SPINLOCK_SIZE, sizeof(spinlock_t)); #ifdef CONFIG_LRU_GEN DEFINE(LRU_GEN_WIDTH, order_base_2(MAX_NR_GENS + 1)); + DEFINE(__LRU_REFS_WIDTH, MAX_NR_TIERS - 2); #else DEFINE(LRU_GEN_WIDTH, 0); + DEFINE(__LRU_REFS_WIDTH, 0); #endif /* End of constants */ =20 diff --git a/mm/Kconfig b/mm/Kconfig index d95f07cd6dcf..5101dca8f21c 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -1124,6 +1124,7 @@ config PTE_MARKER_UFFD_WP purposes. It is required to enable userfaultfd write protection on file-backed memory types like shmem and hugetlbfs. =20 +# multi-gen LRU { config LRU_GEN bool "Multi-Gen LRU" depends on MMU @@ -1132,6 +1133,16 @@ config LRU_GEN help A high performance LRU implementation to overcommit memory. =20 +config LRU_GEN_STATS + bool "Full stats for debugging" + depends on LRU_GEN + help + Do not enable this option unless you plan to look at historical stats + from evicted generations for debugging purpose. + + This option has a per-memcg and per-node memory overhead. +# } + source "mm/damon/Kconfig" =20 endmenu diff --git a/mm/swap.c b/mm/swap.c index 0e423b7d458b..f74fd51fa9e1 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -428,6 +428,40 @@ static void __lru_cache_activate_folio(struct folio *f= olio) local_unlock(&cpu_fbatches.lock); } =20 +#ifdef CONFIG_LRU_GEN +static void folio_inc_refs(struct folio *folio) +{ + unsigned long new_flags, old_flags =3D READ_ONCE(folio->flags); + + if (folio_test_unevictable(folio)) + return; + + if (!folio_test_referenced(folio)) { + folio_set_referenced(folio); + return; + } + + if (!folio_test_workingset(folio)) { + folio_set_workingset(folio); + return; + } + + /* see the comment on MAX_NR_TIERS */ + do { + new_flags =3D old_flags & LRU_REFS_MASK; + if (new_flags =3D=3D LRU_REFS_MASK) + break; + + new_flags +=3D BIT(LRU_REFS_PGOFF); + new_flags |=3D old_flags & ~LRU_REFS_MASK; + } while (!try_cmpxchg(&folio->flags, &old_flags, new_flags)); +} +#else +static void folio_inc_refs(struct folio *folio) +{ +} +#endif /* CONFIG_LRU_GEN */ + /* * Mark a page as having seen activity. * @@ -440,6 +474,11 @@ static void __lru_cache_activate_folio(struct folio *f= olio) */ void folio_mark_accessed(struct folio *folio) { + if (lru_gen_enabled()) { + folio_inc_refs(folio); + return; + } + if (!folio_test_referenced(folio)) { folio_set_referenced(folio); } else if (folio_test_unevictable(folio)) { diff --git a/mm/vmscan.c b/mm/vmscan.c index ed9e149b13c3..4c57fb749a74 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1334,9 +1334,11 @@ static int __remove_mapping(struct address_space *ma= pping, struct folio *folio, =20 if (folio_test_swapcache(folio)) { swp_entry_t swap =3D folio_swap_entry(folio); - mem_cgroup_swapout(folio, swap); + + /* get a shadow entry before mem_cgroup_swapout() clears folio_memcg() *= / if (reclaimed && !mapping_exiting(mapping)) shadow =3D workingset_eviction(folio, target_memcg); + mem_cgroup_swapout(folio, swap); __delete_from_swap_cache(folio, swap, shadow); xa_unlock_irq(&mapping->i_pages); put_swap_page(&folio->page, swap); @@ -2733,6 +2735,9 @@ static void prepare_scan_count(pg_data_t *pgdat, stru= ct scan_control *sc) unsigned long file; struct lruvec *target_lruvec; =20 + if (lru_gen_enabled()) + return; + target_lruvec =3D mem_cgroup_lruvec(sc->target_mem_cgroup, pgdat); =20 /* @@ -3056,6 +3061,17 @@ static bool can_age_anon_pages(struct pglist_data *p= gdat, * shorthand helpers *************************************************************************= *****/ =20 +#define LRU_REFS_FLAGS (BIT(PG_referenced) | BIT(PG_workingset)) + +#define DEFINE_MAX_SEQ(lruvec) \ + unsigned long max_seq =3D READ_ONCE((lruvec)->lrugen.max_seq) + +#define DEFINE_MIN_SEQ(lruvec) \ + unsigned long min_seq[ANON_AND_FILE] =3D { \ + READ_ONCE((lruvec)->lrugen.min_seq[LRU_GEN_ANON]), \ + READ_ONCE((lruvec)->lrugen.min_seq[LRU_GEN_FILE]), \ + } + #define for_each_gen_type_zone(gen, type, zone) \ for ((gen) =3D 0; (gen) < MAX_NR_GENS; (gen)++) \ for ((type) =3D 0; (type) < ANON_AND_FILE; (type)++) \ @@ -3081,6 +3097,769 @@ static struct lruvec __maybe_unused *get_lruvec(str= uct mem_cgroup *memcg, int ni return pgdat ? &pgdat->__lruvec : NULL; } =20 +static int get_swappiness(struct lruvec *lruvec, struct scan_control *sc) +{ + struct mem_cgroup *memcg =3D lruvec_memcg(lruvec); + struct pglist_data *pgdat =3D lruvec_pgdat(lruvec); + + if (!can_demote(pgdat->node_id, sc) && + mem_cgroup_get_nr_swap_pages(memcg) < MIN_LRU_BATCH) + return 0; + + return mem_cgroup_swappiness(memcg); +} + +static int get_nr_gens(struct lruvec *lruvec, int type) +{ + return lruvec->lrugen.max_seq - lruvec->lrugen.min_seq[type] + 1; +} + +static bool __maybe_unused seq_is_valid(struct lruvec *lruvec) +{ + /* see the comment on lru_gen_struct */ + return get_nr_gens(lruvec, LRU_GEN_FILE) >=3D MIN_NR_GENS && + get_nr_gens(lruvec, LRU_GEN_FILE) <=3D get_nr_gens(lruvec, LRU_GEN= _ANON) && + get_nr_gens(lruvec, LRU_GEN_ANON) <=3D MAX_NR_GENS; +} + +/*************************************************************************= ***** + * refault feedback loop + *************************************************************************= *****/ + +/* + * A feedback loop based on Proportional-Integral-Derivative (PID) control= ler. + * + * The P term is refaulted/(evicted+protected) from a tier in the generati= on + * currently being evicted; the I term is the exponential moving average o= f the + * P term over the generations previously evicted, using the smoothing fac= tor + * 1/2; the D term isn't supported. + * + * The setpoint (SP) is always the first tier of one type; the process var= iable + * (PV) is either any tier of the other type or any other tier of the same + * type. + * + * The error is the difference between the SP and the PV; the correction i= s to + * turn off protection when SP>PV or turn on protection when SPlrugen; + int hist =3D lru_hist_from_seq(lrugen->min_seq[type]); + + pos->refaulted =3D lrugen->avg_refaulted[type][tier] + + atomic_long_read(&lrugen->refaulted[hist][type][tier]); + pos->total =3D lrugen->avg_total[type][tier] + + atomic_long_read(&lrugen->evicted[hist][type][tier]); + if (tier) + pos->total +=3D lrugen->protected[hist][type][tier - 1]; + pos->gain =3D gain; +} + +static void reset_ctrl_pos(struct lruvec *lruvec, int type, bool carryover= ) +{ + int hist, tier; + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + bool clear =3D carryover ? NR_HIST_GENS =3D=3D 1 : NR_HIST_GENS > 1; + unsigned long seq =3D carryover ? lrugen->min_seq[type] : lrugen->max_seq= + 1; + + lockdep_assert_held(&lruvec->lru_lock); + + if (!carryover && !clear) + return; + + hist =3D lru_hist_from_seq(seq); + + for (tier =3D 0; tier < MAX_NR_TIERS; tier++) { + if (carryover) { + unsigned long sum; + + sum =3D lrugen->avg_refaulted[type][tier] + + atomic_long_read(&lrugen->refaulted[hist][type][tier]); + WRITE_ONCE(lrugen->avg_refaulted[type][tier], sum / 2); + + sum =3D lrugen->avg_total[type][tier] + + atomic_long_read(&lrugen->evicted[hist][type][tier]); + if (tier) + sum +=3D lrugen->protected[hist][type][tier - 1]; + WRITE_ONCE(lrugen->avg_total[type][tier], sum / 2); + } + + if (clear) { + atomic_long_set(&lrugen->refaulted[hist][type][tier], 0); + atomic_long_set(&lrugen->evicted[hist][type][tier], 0); + if (tier) + WRITE_ONCE(lrugen->protected[hist][type][tier - 1], 0); + } + } +} + +static bool positive_ctrl_err(struct ctrl_pos *sp, struct ctrl_pos *pv) +{ + /* + * Return true if the PV has a limited number of refaults or a lower + * refaulted/total than the SP. + */ + return pv->refaulted < MIN_LRU_BATCH || + pv->refaulted * (sp->total + MIN_LRU_BATCH) * sp->gain <=3D + (sp->refaulted + 1) * pv->total * pv->gain; +} + +/*************************************************************************= ***** + * the aging + *************************************************************************= *****/ + +/* protect pages accessed multiple times through file descriptors */ +static int folio_inc_gen(struct lruvec *lruvec, struct folio *folio, bool = reclaiming) +{ + int type =3D folio_is_file_lru(folio); + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + int new_gen, old_gen =3D lru_gen_from_seq(lrugen->min_seq[type]); + unsigned long new_flags, old_flags =3D READ_ONCE(folio->flags); + + VM_WARN_ON_ONCE_FOLIO(!(old_flags & LRU_GEN_MASK), folio); + + do { + new_gen =3D (old_gen + 1) % MAX_NR_GENS; + + new_flags =3D old_flags & ~(LRU_GEN_MASK | LRU_REFS_MASK | LRU_REFS_FLAG= S); + new_flags |=3D (new_gen + 1UL) << LRU_GEN_PGOFF; + /* for folio_end_writeback() */ + if (reclaiming) + new_flags |=3D BIT(PG_reclaim); + } while (!try_cmpxchg(&folio->flags, &old_flags, new_flags)); + + lru_gen_update_size(lruvec, folio, old_gen, new_gen); + + return new_gen; +} + +static void inc_min_seq(struct lruvec *lruvec, int type) +{ + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + + reset_ctrl_pos(lruvec, type, true); + WRITE_ONCE(lrugen->min_seq[type], lrugen->min_seq[type] + 1); +} + +static bool try_to_inc_min_seq(struct lruvec *lruvec, bool can_swap) +{ + int gen, type, zone; + bool success =3D false; + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + DEFINE_MIN_SEQ(lruvec); + + VM_WARN_ON_ONCE(!seq_is_valid(lruvec)); + + /* find the oldest populated generation */ + for (type =3D !can_swap; type < ANON_AND_FILE; type++) { + while (min_seq[type] + MIN_NR_GENS <=3D lrugen->max_seq) { + gen =3D lru_gen_from_seq(min_seq[type]); + + for (zone =3D 0; zone < MAX_NR_ZONES; zone++) { + if (!list_empty(&lrugen->lists[gen][type][zone])) + goto next; + } + + min_seq[type]++; + } +next: + ; + } + + /* see the comment on lru_gen_struct */ + if (can_swap) { + min_seq[LRU_GEN_ANON] =3D min(min_seq[LRU_GEN_ANON], min_seq[LRU_GEN_FIL= E]); + min_seq[LRU_GEN_FILE] =3D max(min_seq[LRU_GEN_ANON], lrugen->min_seq[LRU= _GEN_FILE]); + } + + for (type =3D !can_swap; type < ANON_AND_FILE; type++) { + if (min_seq[type] =3D=3D lrugen->min_seq[type]) + continue; + + reset_ctrl_pos(lruvec, type, true); + WRITE_ONCE(lrugen->min_seq[type], min_seq[type]); + success =3D true; + } + + return success; +} + +static void inc_max_seq(struct lruvec *lruvec, unsigned long max_seq, bool= can_swap) +{ + int prev, next; + int type, zone; + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + + spin_lock_irq(&lruvec->lru_lock); + + VM_WARN_ON_ONCE(!seq_is_valid(lruvec)); + + if (max_seq !=3D lrugen->max_seq) + goto unlock; + + for (type =3D 0; type < ANON_AND_FILE; type++) { + if (get_nr_gens(lruvec, type) !=3D MAX_NR_GENS) + continue; + + VM_WARN_ON_ONCE(type =3D=3D LRU_GEN_FILE || can_swap); + + inc_min_seq(lruvec, type); + } + + /* + * Update the active/inactive LRU sizes for compatibility. Both sides of + * the current max_seq need to be covered, since max_seq+1 can overlap + * with min_seq[LRU_GEN_ANON] if swapping is constrained. And if they do + * overlap, cold/hot inversion happens. + */ + prev =3D lru_gen_from_seq(lrugen->max_seq - 1); + next =3D lru_gen_from_seq(lrugen->max_seq + 1); + + for (type =3D 0; type < ANON_AND_FILE; type++) { + for (zone =3D 0; zone < MAX_NR_ZONES; zone++) { + enum lru_list lru =3D type * LRU_INACTIVE_FILE; + long delta =3D lrugen->nr_pages[prev][type][zone] - + lrugen->nr_pages[next][type][zone]; + + if (!delta) + continue; + + __update_lru_size(lruvec, lru, zone, delta); + __update_lru_size(lruvec, lru + LRU_ACTIVE, zone, -delta); + } + } + + for (type =3D 0; type < ANON_AND_FILE; type++) + reset_ctrl_pos(lruvec, type, false); + + /* make sure preceding modifications appear */ + smp_store_release(&lrugen->max_seq, lrugen->max_seq + 1); +unlock: + spin_unlock_irq(&lruvec->lru_lock); +} + +static unsigned long get_nr_evictable(struct lruvec *lruvec, unsigned long= max_seq, + unsigned long *min_seq, bool can_swap, bool *need_aging) +{ + int gen, type, zone; + unsigned long old =3D 0; + unsigned long young =3D 0; + unsigned long total =3D 0; + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + + for (type =3D !can_swap; type < ANON_AND_FILE; type++) { + unsigned long seq; + + for (seq =3D min_seq[type]; seq <=3D max_seq; seq++) { + unsigned long size =3D 0; + + gen =3D lru_gen_from_seq(seq); + + for (zone =3D 0; zone < MAX_NR_ZONES; zone++) + size +=3D max(READ_ONCE(lrugen->nr_pages[gen][type][zone]), 0L); + + total +=3D size; + if (seq =3D=3D max_seq) + young +=3D size; + if (seq + MIN_NR_GENS =3D=3D max_seq) + old +=3D size; + } + } + + /* + * The aging tries to be lazy to reduce the overhead. On the other hand, + * the eviction stalls when the number of generations reaches + * MIN_NR_GENS. So ideally, there should be MIN_NR_GENS+1 generations, + * hence the first two if's. + * + * Also it's ideal to spread pages out evenly, meaning 1/(MIN_NR_GENS+1) + * of the total number of pages for each generation. A reasonable range + * for this average portion is [1/MIN_NR_GENS, 1/(MIN_NR_GENS+2)]. The + * eviction cares about the lower bound of cold pages, whereas the aging + * cares about the upper bound of hot pages. + */ + if (min_seq[!can_swap] + MIN_NR_GENS > max_seq) + *need_aging =3D true; + else if (min_seq[!can_swap] + MIN_NR_GENS < max_seq) + *need_aging =3D false; + else if (young * MIN_NR_GENS > total) + *need_aging =3D true; + else if (old * (MIN_NR_GENS + 2) < total) + *need_aging =3D true; + else + *need_aging =3D false; + + return total; +} + +static void age_lruvec(struct lruvec *lruvec, struct scan_control *sc) +{ + bool need_aging; + unsigned long nr_to_scan; + int swappiness =3D get_swappiness(lruvec, sc); + struct mem_cgroup *memcg =3D lruvec_memcg(lruvec); + DEFINE_MAX_SEQ(lruvec); + DEFINE_MIN_SEQ(lruvec); + + VM_WARN_ON_ONCE(sc->memcg_low_reclaim); + + mem_cgroup_calculate_protection(NULL, memcg); + + if (mem_cgroup_below_min(memcg)) + return; + + nr_to_scan =3D get_nr_evictable(lruvec, max_seq, min_seq, swappiness, &ne= ed_aging); + if (!nr_to_scan) + return; + + nr_to_scan >>=3D mem_cgroup_online(memcg) ? sc->priority : 0; + + if (nr_to_scan && need_aging) + inc_max_seq(lruvec, max_seq, swappiness); +} + +static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_contro= l *sc) +{ + struct mem_cgroup *memcg; + + VM_WARN_ON_ONCE(!current_is_kswapd()); + + memcg =3D mem_cgroup_iter(NULL, NULL, NULL); + do { + struct lruvec *lruvec =3D mem_cgroup_lruvec(memcg, pgdat); + + age_lruvec(lruvec, sc); + + cond_resched(); + } while ((memcg =3D mem_cgroup_iter(NULL, memcg, NULL))); +} + +/*************************************************************************= ***** + * the eviction + *************************************************************************= *****/ + +static bool sort_folio(struct lruvec *lruvec, struct folio *folio, int tie= r_idx) +{ + bool success; + int gen =3D folio_lru_gen(folio); + int type =3D folio_is_file_lru(folio); + int zone =3D folio_zonenum(folio); + int delta =3D folio_nr_pages(folio); + int refs =3D folio_lru_refs(folio); + int tier =3D lru_tier_from_refs(refs); + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + + VM_WARN_ON_ONCE_FOLIO(gen >=3D MAX_NR_GENS, folio); + + /* unevictable */ + if (!folio_evictable(folio)) { + success =3D lru_gen_del_folio(lruvec, folio, true); + VM_WARN_ON_ONCE_FOLIO(!success, folio); + folio_set_unevictable(folio); + lruvec_add_folio(lruvec, folio); + __count_vm_events(UNEVICTABLE_PGCULLED, delta); + return true; + } + + /* dirty lazyfree */ + if (type =3D=3D LRU_GEN_FILE && folio_test_anon(folio) && folio_test_dirt= y(folio)) { + success =3D lru_gen_del_folio(lruvec, folio, true); + VM_WARN_ON_ONCE_FOLIO(!success, folio); + folio_set_swapbacked(folio); + lruvec_add_folio_tail(lruvec, folio); + return true; + } + + /* protected */ + if (tier > tier_idx) { + int hist =3D lru_hist_from_seq(lrugen->min_seq[type]); + + gen =3D folio_inc_gen(lruvec, folio, false); + list_move_tail(&folio->lru, &lrugen->lists[gen][type][zone]); + + WRITE_ONCE(lrugen->protected[hist][type][tier - 1], + lrugen->protected[hist][type][tier - 1] + delta); + __mod_lruvec_state(lruvec, WORKINGSET_ACTIVATE_BASE + type, delta); + return true; + } + + /* waiting for writeback */ + if (folio_test_locked(folio) || folio_test_writeback(folio) || + (type =3D=3D LRU_GEN_FILE && folio_test_dirty(folio))) { + gen =3D folio_inc_gen(lruvec, folio, true); + list_move(&folio->lru, &lrugen->lists[gen][type][zone]); + return true; + } + + return false; +} + +static bool isolate_folio(struct lruvec *lruvec, struct folio *folio, stru= ct scan_control *sc) +{ + bool success; + + /* unmapping inhibited */ + if (!sc->may_unmap && folio_mapped(folio)) + return false; + + /* swapping inhibited */ + if (!(sc->may_writepage && (sc->gfp_mask & __GFP_IO)) && + (folio_test_dirty(folio) || + (folio_test_anon(folio) && !folio_test_swapcache(folio)))) + return false; + + /* raced with release_pages() */ + if (!folio_try_get(folio)) + return false; + + /* raced with another isolation */ + if (!folio_test_clear_lru(folio)) { + folio_put(folio); + return false; + } + + /* see the comment on MAX_NR_TIERS */ + if (!folio_test_referenced(folio)) + set_mask_bits(&folio->flags, LRU_REFS_MASK | LRU_REFS_FLAGS, 0); + + /* for shrink_page_list() */ + folio_clear_reclaim(folio); + folio_clear_referenced(folio); + + success =3D lru_gen_del_folio(lruvec, folio, true); + VM_WARN_ON_ONCE_FOLIO(!success, folio); + + return true; +} + +static int scan_folios(struct lruvec *lruvec, struct scan_control *sc, + int type, int tier, struct list_head *list) +{ + int gen, zone; + enum vm_event_item item; + int sorted =3D 0; + int scanned =3D 0; + int isolated =3D 0; + int remaining =3D MAX_LRU_BATCH; + struct lru_gen_struct *lrugen =3D &lruvec->lrugen; + struct mem_cgroup *memcg =3D lruvec_memcg(lruvec); + + VM_WARN_ON_ONCE(!list_empty(list)); + + if (get_nr_gens(lruvec, type) =3D=3D MIN_NR_GENS) + return 0; + + gen =3D lru_gen_from_seq(lrugen->min_seq[type]); + + for (zone =3D sc->reclaim_idx; zone >=3D 0; zone--) { + LIST_HEAD(moved); + int skipped =3D 0; + struct list_head *head =3D &lrugen->lists[gen][type][zone]; + + while (!list_empty(head)) { + struct folio *folio =3D lru_to_folio(head); + int delta =3D folio_nr_pages(folio); + + VM_WARN_ON_ONCE_FOLIO(folio_test_unevictable(folio), folio); + VM_WARN_ON_ONCE_FOLIO(folio_test_active(folio), folio); + VM_WARN_ON_ONCE_FOLIO(folio_is_file_lru(folio) !=3D type, folio); + VM_WARN_ON_ONCE_FOLIO(folio_zonenum(folio) !=3D zone, folio); + + scanned +=3D delta; + + if (sort_folio(lruvec, folio, tier)) + sorted +=3D delta; + else if (isolate_folio(lruvec, folio, sc)) { + list_add(&folio->lru, list); + isolated +=3D delta; + } else { + list_move(&folio->lru, &moved); + skipped +=3D delta; + } + + if (!--remaining || max(isolated, skipped) >=3D MIN_LRU_BATCH) + break; + } + + if (skipped) { + list_splice(&moved, head); + __count_zid_vm_events(PGSCAN_SKIP, zone, skipped); + } + + if (!remaining || isolated >=3D MIN_LRU_BATCH) + break; + } + + item =3D current_is_kswapd() ? PGSCAN_KSWAPD : PGSCAN_DIRECT; + if (!cgroup_reclaim(sc)) { + __count_vm_events(item, isolated); + __count_vm_events(PGREFILL, sorted); + } + __count_memcg_events(memcg, item, isolated); + __count_memcg_events(memcg, PGREFILL, sorted); + __count_vm_events(PGSCAN_ANON + type, isolated); + + /* + * There might not be eligible pages due to reclaim_idx, may_unmap and + * may_writepage. Check the remaining to prevent livelock if it's not + * making progress. + */ + return isolated || !remaining ? scanned : 0; +} + +static int get_tier_idx(struct lruvec *lruvec, int type) +{ + int tier; + struct ctrl_pos sp, pv; + + /* + * To leave a margin for fluctuations, use a larger gain factor (1:2). + * This value is chosen because any other tier would have at least twice + * as many refaults as the first tier. + */ + read_ctrl_pos(lruvec, type, 0, 1, &sp); + for (tier =3D 1; tier < MAX_NR_TIERS; tier++) { + read_ctrl_pos(lruvec, type, tier, 2, &pv); + if (!positive_ctrl_err(&sp, &pv)) + break; + } + + return tier - 1; +} + +static int get_type_to_scan(struct lruvec *lruvec, int swappiness, int *ti= er_idx) +{ + int type, tier; + struct ctrl_pos sp, pv; + int gain[ANON_AND_FILE] =3D { swappiness, 200 - swappiness }; + + /* + * Compare the first tier of anon with that of file to determine which + * type to scan. Also need to compare other tiers of the selected type + * with the first tier of the other type to determine the last tier (of + * the selected type) to evict. + */ + read_ctrl_pos(lruvec, LRU_GEN_ANON, 0, gain[LRU_GEN_ANON], &sp); + read_ctrl_pos(lruvec, LRU_GEN_FILE, 0, gain[LRU_GEN_FILE], &pv); + type =3D positive_ctrl_err(&sp, &pv); + + read_ctrl_pos(lruvec, !type, 0, gain[!type], &sp); + for (tier =3D 1; tier < MAX_NR_TIERS; tier++) { + read_ctrl_pos(lruvec, type, tier, gain[type], &pv); + if (!positive_ctrl_err(&sp, &pv)) + break; + } + + *tier_idx =3D tier - 1; + + return type; +} + +static int isolate_folios(struct lruvec *lruvec, struct scan_control *sc, = int swappiness, + int *type_scanned, struct list_head *list) +{ + int i; + int type; + int scanned; + int tier =3D -1; + DEFINE_MIN_SEQ(lruvec); + + /* + * Try to make the obvious choice first. When anon and file are both + * available from the same generation, interpret swappiness 1 as file + * first and 200 as anon first. + */ + if (!swappiness) + type =3D LRU_GEN_FILE; + else if (min_seq[LRU_GEN_ANON] < min_seq[LRU_GEN_FILE]) + type =3D LRU_GEN_ANON; + else if (swappiness =3D=3D 1) + type =3D LRU_GEN_FILE; + else if (swappiness =3D=3D 200) + type =3D LRU_GEN_ANON; + else + type =3D get_type_to_scan(lruvec, swappiness, &tier); + + for (i =3D !swappiness; i < ANON_AND_FILE; i++) { + if (tier < 0) + tier =3D get_tier_idx(lruvec, type); + + scanned =3D scan_folios(lruvec, sc, type, tier, list); + if (scanned) + break; + + type =3D !type; + tier =3D -1; + } + + *type_scanned =3D type; + + return scanned; +} + +static int evict_folios(struct lruvec *lruvec, struct scan_control *sc, in= t swappiness) +{ + int type; + int scanned; + int reclaimed; + LIST_HEAD(list); + struct folio *folio; + enum vm_event_item item; + struct reclaim_stat stat; + struct mem_cgroup *memcg =3D lruvec_memcg(lruvec); + struct pglist_data *pgdat =3D lruvec_pgdat(lruvec); + + spin_lock_irq(&lruvec->lru_lock); + + scanned =3D isolate_folios(lruvec, sc, swappiness, &type, &list); + + scanned +=3D try_to_inc_min_seq(lruvec, swappiness); + + if (get_nr_gens(lruvec, !swappiness) =3D=3D MIN_NR_GENS) + scanned =3D 0; + + spin_unlock_irq(&lruvec->lru_lock); + + if (list_empty(&list)) + return scanned; + + reclaimed =3D shrink_page_list(&list, pgdat, sc, &stat, false); + + list_for_each_entry(folio, &list, lru) { + /* restore LRU_REFS_FLAGS cleared by isolate_folio() */ + if (folio_test_workingset(folio)) + folio_set_referenced(folio); + + /* don't add rejected pages to the oldest generation */ + if (folio_test_reclaim(folio) && + (folio_test_dirty(folio) || folio_test_writeback(folio))) + folio_clear_active(folio); + else + folio_set_active(folio); + } + + spin_lock_irq(&lruvec->lru_lock); + + move_pages_to_lru(lruvec, &list); + + item =3D current_is_kswapd() ? PGSTEAL_KSWAPD : PGSTEAL_DIRECT; + if (!cgroup_reclaim(sc)) + __count_vm_events(item, reclaimed); + __count_memcg_events(memcg, item, reclaimed); + __count_vm_events(PGSTEAL_ANON + type, reclaimed); + + spin_unlock_irq(&lruvec->lru_lock); + + mem_cgroup_uncharge_list(&list); + free_unref_page_list(&list); + + sc->nr_reclaimed +=3D reclaimed; + + return scanned; +} + +static unsigned long get_nr_to_scan(struct lruvec *lruvec, struct scan_con= trol *sc, + bool can_swap, unsigned long reclaimed) +{ + int priority; + bool need_aging; + unsigned long nr_to_scan; + struct mem_cgroup *memcg =3D lruvec_memcg(lruvec); + DEFINE_MAX_SEQ(lruvec); + DEFINE_MIN_SEQ(lruvec); + + if (fatal_signal_pending(current)) { + sc->nr_reclaimed +=3D MIN_LRU_BATCH; + return 0; + } + + if (mem_cgroup_below_min(memcg) || + (mem_cgroup_below_low(memcg) && !sc->memcg_low_reclaim)) + return 0; + + nr_to_scan =3D get_nr_evictable(lruvec, max_seq, min_seq, can_swap, &need= _aging); + if (!nr_to_scan) + return 0; + + /* adjust priority if memcg is offline or the target is met */ + if (!mem_cgroup_online(memcg)) + priority =3D 0; + else if (sc->nr_reclaimed - reclaimed >=3D sc->nr_to_reclaim) + priority =3D DEF_PRIORITY; + else + priority =3D sc->priority; + + nr_to_scan >>=3D priority; + if (!nr_to_scan) + return 0; + + if (!need_aging) + return nr_to_scan; + + /* skip the aging path at the default priority */ + if (priority =3D=3D DEF_PRIORITY) + goto done; + + /* leave the work to lru_gen_age_node() */ + if (current_is_kswapd()) + return 0; + + inc_max_seq(lruvec, max_seq, can_swap); +done: + return min_seq[!can_swap] + MIN_NR_GENS <=3D max_seq ? nr_to_scan : 0; +} + +static void lru_gen_shrink_lruvec(struct lruvec *lruvec, struct scan_contr= ol *sc) +{ + struct blk_plug plug; + unsigned long scanned =3D 0; + unsigned long reclaimed =3D sc->nr_reclaimed; + + lru_add_drain(); + + blk_start_plug(&plug); + + while (true) { + int delta; + int swappiness; + unsigned long nr_to_scan; + + if (sc->may_swap) + swappiness =3D get_swappiness(lruvec, sc); + else if (!cgroup_reclaim(sc) && get_swappiness(lruvec, sc)) + swappiness =3D 1; + else + swappiness =3D 0; + + nr_to_scan =3D get_nr_to_scan(lruvec, sc, swappiness, reclaimed); + if (!nr_to_scan) + break; + + delta =3D evict_folios(lruvec, sc, swappiness); + if (!delta) + break; + + scanned +=3D delta; + if (scanned >=3D nr_to_scan) + break; + + cond_resched(); + } + + blk_finish_plug(&plug); +} + /*************************************************************************= ***** * initialization *************************************************************************= *****/ @@ -3123,6 +3902,16 @@ static int __init init_lru_gen(void) }; late_initcall(init_lru_gen); =20 +#else /* !CONFIG_LRU_GEN */ + +static void lru_gen_age_node(struct pglist_data *pgdat, struct scan_contro= l *sc) +{ +} + +static void lru_gen_shrink_lruvec(struct lruvec *lruvec, struct scan_contr= ol *sc) +{ +} + #endif /* CONFIG_LRU_GEN */ =20 static void shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc) @@ -3136,6 +3925,11 @@ static void shrink_lruvec(struct lruvec *lruvec, str= uct scan_control *sc) bool proportional_reclaim; struct blk_plug plug; =20 + if (lru_gen_enabled()) { + lru_gen_shrink_lruvec(lruvec, sc); + return; + } + get_scan_count(lruvec, sc, nr); =20 /* Record the original scan target for proportional adjustments later */ @@ -3640,6 +4434,9 @@ static void snapshot_refaults(struct mem_cgroup *targ= et_memcg, pg_data_t *pgdat) struct lruvec *target_lruvec; unsigned long refaults; =20 + if (lru_gen_enabled()) + return; + target_lruvec =3D mem_cgroup_lruvec(target_memcg, pgdat); refaults =3D lruvec_page_state(target_lruvec, WORKINGSET_ACTIVATE_ANON); target_lruvec->refaults[WORKINGSET_ANON] =3D refaults; @@ -4006,12 +4803,17 @@ unsigned long try_to_free_mem_cgroup_pages(struct m= em_cgroup *memcg, } #endif =20 -static void age_active_anon(struct pglist_data *pgdat, +static void kswapd_age_node(struct pglist_data *pgdat, struct scan_control *sc) { struct mem_cgroup *memcg; struct lruvec *lruvec; =20 + if (lru_gen_enabled()) { + lru_gen_age_node(pgdat, sc); + return; + } + if (!can_age_anon_pages(pgdat, sc)) return; =20 @@ -4331,12 +5133,11 @@ static int balance_pgdat(pg_data_t *pgdat, int orde= r, int highest_zoneidx) sc.may_swap =3D !nr_boost_reclaim; =20 /* - * Do some background aging of the anon list, to give - * pages a chance to be referenced before reclaiming. All - * pages are rotated regardless of classzone as this is - * about consistent aging. + * Do some background aging, to give pages a chance to be + * referenced before reclaiming. All pages are rotated + * regardless of classzone as this is about consistent aging. */ - age_active_anon(pgdat, &sc); + kswapd_age_node(pgdat, &sc); =20 /* * If we're getting trouble reclaiming, start doing writepage diff --git a/mm/workingset.c b/mm/workingset.c index a5e84862fc86..ae7e984b23c6 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -187,7 +187,6 @@ static unsigned int bucket_order __read_mostly; static void *pack_shadow(int memcgid, pg_data_t *pgdat, unsigned long evic= tion, bool workingset) { - eviction >>=3D bucket_order; eviction &=3D EVICTION_MASK; eviction =3D (eviction << MEM_CGROUP_ID_SHIFT) | memcgid; eviction =3D (eviction << NODES_SHIFT) | pgdat->node_id; @@ -212,10 +211,107 @@ static void unpack_shadow(void *shadow, int *memcgid= p, pg_data_t **pgdat, =20 *memcgidp =3D memcgid; *pgdat =3D NODE_DATA(nid); - *evictionp =3D entry << bucket_order; + *evictionp =3D entry; *workingsetp =3D workingset; } =20 +#ifdef CONFIG_LRU_GEN + +static void *lru_gen_eviction(struct folio *folio) +{ + int hist; + unsigned long token; + unsigned long min_seq; + struct lruvec *lruvec; + struct lru_gen_struct *lrugen; + int type =3D folio_is_file_lru(folio); + int delta =3D folio_nr_pages(folio); + int refs =3D folio_lru_refs(folio); + int tier =3D lru_tier_from_refs(refs); + struct mem_cgroup *memcg =3D folio_memcg(folio); + struct pglist_data *pgdat =3D folio_pgdat(folio); + + BUILD_BUG_ON(LRU_GEN_WIDTH + LRU_REFS_WIDTH > BITS_PER_LONG - EVICTION_SH= IFT); + + lruvec =3D mem_cgroup_lruvec(memcg, pgdat); + lrugen =3D &lruvec->lrugen; + min_seq =3D READ_ONCE(lrugen->min_seq[type]); + token =3D (min_seq << LRU_REFS_WIDTH) | max(refs - 1, 0); + + hist =3D lru_hist_from_seq(min_seq); + atomic_long_add(delta, &lrugen->evicted[hist][type][tier]); + + return pack_shadow(mem_cgroup_id(memcg), pgdat, token, refs); +} + +static void lru_gen_refault(struct folio *folio, void *shadow) +{ + int hist, tier, refs; + int memcg_id; + bool workingset; + unsigned long token; + unsigned long min_seq; + struct lruvec *lruvec; + struct lru_gen_struct *lrugen; + struct mem_cgroup *memcg; + struct pglist_data *pgdat; + int type =3D folio_is_file_lru(folio); + int delta =3D folio_nr_pages(folio); + + unpack_shadow(shadow, &memcg_id, &pgdat, &token, &workingset); + + if (pgdat !=3D folio_pgdat(folio)) + return; + + rcu_read_lock(); + + memcg =3D folio_memcg_rcu(folio); + if (memcg_id !=3D mem_cgroup_id(memcg)) + goto unlock; + + lruvec =3D mem_cgroup_lruvec(memcg, pgdat); + lrugen =3D &lruvec->lrugen; + + min_seq =3D READ_ONCE(lrugen->min_seq[type]); + if ((token >> LRU_REFS_WIDTH) !=3D (min_seq & (EVICTION_MASK >> LRU_REFS_= WIDTH))) + goto unlock; + + hist =3D lru_hist_from_seq(min_seq); + /* see the comment in folio_lru_refs() */ + refs =3D (token & (BIT(LRU_REFS_WIDTH) - 1)) + workingset; + tier =3D lru_tier_from_refs(refs); + + atomic_long_add(delta, &lrugen->refaulted[hist][type][tier]); + mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + type, delta); + + /* + * Count the following two cases as stalls: + * 1. For pages accessed through page tables, hotter pages pushed out + * hot pages which refaulted immediately. + * 2. For pages accessed multiple times through file descriptors, + * numbers of accesses might have been out of the range. + */ + if (lru_gen_in_fault() || refs =3D=3D BIT(LRU_REFS_WIDTH)) { + folio_set_workingset(folio); + mod_lruvec_state(lruvec, WORKINGSET_RESTORE_BASE + type, delta); + } +unlock: + rcu_read_unlock(); +} + +#else /* !CONFIG_LRU_GEN */ + +static void *lru_gen_eviction(struct folio *folio) +{ + return NULL; +} + +static void lru_gen_refault(struct folio *folio, void *shadow) +{ +} + +#endif /* CONFIG_LRU_GEN */ + /** * workingset_age_nonresident - age non-resident entries as LRU ages * @lruvec: the lruvec that was aged @@ -264,10 +360,14 @@ void *workingset_eviction(struct folio *folio, struct= mem_cgroup *target_memcg) VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); =20 + if (lru_gen_enabled()) + return lru_gen_eviction(folio); + lruvec =3D mem_cgroup_lruvec(target_memcg, pgdat); /* XXX: target_memcg can be NULL, go through lruvec */ memcgid =3D mem_cgroup_id(lruvec_memcg(lruvec)); eviction =3D atomic_long_read(&lruvec->nonresident_age); + eviction >>=3D bucket_order; workingset_age_nonresident(lruvec, folio_nr_pages(folio)); return pack_shadow(memcgid, pgdat, eviction, folio_test_workingset(folio)); @@ -298,7 +398,13 @@ void workingset_refault(struct folio *folio, void *sha= dow) int memcgid; long nr; =20 + if (lru_gen_enabled()) { + lru_gen_refault(folio, shadow); + return; + } + unpack_shadow(shadow, &memcgid, &pgdat, &eviction, &workingset); + eviction <<=3D bucket_order; =20 rcu_read_lock(); /* --=20 2.37.1.595.g718a3a8f04-goog From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F274C25B0D for ; Mon, 15 Aug 2022 07:51:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:References: Mime-Version:Message-Id:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=qSRxcedQA+0mC0wFTGZvv18oK+giu2E9BJdHl9o7UrA=; b=knw0gTPE7u36eoJkViPCfFOSj+ yTO4VBKB9wx8vAxIw67BTAE4sq43QSvlLiikBicgOuyIf9Cv0kmEMZFxMeTP9d8p+z3IDpBnWNuni hXJ/D0EZ6G6Fcq+i5aHKcvZlsYQh2IzjAzQHUPbW9dPKZynujgW/1bjy0YteQqvydYK7yBVy296w+ XcKbf344MxDN4JeE0WdOygIDLoum775YUS0siHEhdjAKCxtVbri5hOP+PT29dwkziV/vRzBty1VeS N2hoTj0OHNQZSczLUXoTXa3STcFoBXd0WzqvRAdl6b+4ujLn3orS2zi9s6X3O9ZzWB54RkNbt5Qp9 OJZ1F0HQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1oNUr0-00Clm0-Ke; Mon, 15 Aug 2022 07:49:43 +0000 Received: from desiato.infradead.org ([90.155.92.199]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oNUpE-00CkVD-68 for linux-arm-kernel@bombadil.infradead.org; Mon, 15 Aug 2022 07:47:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :Cc:To:From:Subject:References:Mime-Version:Message-Id:In-Reply-To:Date: Sender:Reply-To:Content-ID:Content-Description; bh=OG21aV95NWHOo7NMaXGtZDeDVd1PZFs7RFAu3qrJU9s=; b=OCDAFSHPQZJyuWctJIhX3LCqRB G6GJzI3vRPMcakJtvWCMTWuMDMkVJDJq8GHiA3yN4ieq9uoDQEV1IJHyyxhj3WfVseDujoKvQMxyU F7VUbpQ+oX1VQNhF8sxXuDvSmidEOqaK1A794sK/i/wya+N8SlCCCCqwmp8cXL00W7WE7G9Zo4dqA e5aDOvRT8zYVl/uwq1G+1Lt9ha1ufdAO7IhKyTplix/B21mzXaYhkyscaIwb9drJKsMTvC6Sawz5J iR6bgSWAmZFGom33yjHNCFnoPjgBKP3np6zArd+ByrsB2YgSq8dCScjRUV1UKrnaHXM8hZygYoVNw 9F/9SsBQ==; Received: from mail-yw1-x1149.google.com ([2607:f8b0:4864:20::1149]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1oNUIp-002bdD-UI for linux-arm-kernel@lists.infradead.org; Mon, 15 Aug 2022 07:14:32 +0000 Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-31f4b76446aso51128107b3.7 for ; Mon, 15 Aug 2022 00:14:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:from:to:cc; bh=OG21aV95NWHOo7NMaXGtZDeDVd1PZFs7RFAu3qrJU9s=; b=WHe/o94b6sSWCNtVsvdXInF7Bs9OcbJU1dCj5DMXBNlfz6RhO+0CKOBLCNAPaeawPG psfFyfr3nAdZuKV9pWqSj24SWqUznqL4Tav55NsT9qgTWqeNVABTbOJnaZzcPELXEcB2 GKWQqthHvAhQn69blnDjEgYv96hlJasH5D45Wq24AF+9WLmynB45huVm2am+fMtg5dZ2 lhYGstyBsOHK7LAOQ1wnldaY0xWxOrEWRQQgjq9DlQ+T+z5wqRXFVhXMtMEO8ZHbrlVV AY5uxXmoml/qiGfZT1IhTHRzXxE/3uNBknjngm1EE+mdcivTWPERJ3sQpHNvGdadB3BW v4Xg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:x-gm-message-state:from:to :cc; bh=OG21aV95NWHOo7NMaXGtZDeDVd1PZFs7RFAu3qrJU9s=; b=A5vkTBSSce0iwniLtLSpGdMXvkruWwI8chOEM6Aru3DdPC/Gt+NRsdFSUZ4o6ncRkc xpaj6m2Ch7+ZkRYjCKOQkg9UzC2qU5QOfahNJZKJx079EQxyGaZRnSy56c4IwyNMIH24 uSxmxezhp4kQb43lBk0CF38ItSb9+7dxtqpGdXtETpJMY9psizRIZZd06TlNJ/0TuDNM imcs7kZhBLnufb8tBTMmODOHUWcwr+Ykpus2aQ3+Z2h/Eb5YrH9/FlRd6K3T8LkoRQie 2jcenFMem1C730MNj8QeHQhtZVEF1K0VXbaVU7bvgRvoE+eXKiL87aNXe521lR79lRa7 nx5w== X-Gm-Message-State: ACgBeo1lpZj5i/Lk50L0riLj3c9zaRNwkGKZtDyfHdsJb7fmqw9hM5YK TDhEit+W/BOwdqv69gnlwfTi6uNZ0FE= X-Google-Smtp-Source: AA6agR7AzqnhxIvkLjTuH7X/p83y5zjJA+3i7KoHjwOuPi8aLZoBUFdTyHtN9BRaL1hIyv7TAKS5Egbks70= X-Received: from yuzhao.bld.corp.google.com ([2620:15c:183:200:d91:5887:ac93:ddf0]) (user=yuzhao job=sendgmr) by 2002:a25:424b:0:b0:687:5558:4e7b with SMTP id p72-20020a25424b000000b0068755584e7bmr4905422yba.434.1660547662335; Mon, 15 Aug 2022 00:14:22 -0700 (PDT) Date: Mon, 15 Aug 2022 01:13:25 -0600 In-Reply-To: <20220815071332.627393-1-yuzhao@google.com> Message-Id: <20220815071332.627393-7-yuzhao@google.com> Mime-Version: 1.0 References: <20220815071332.627393-1-yuzhao@google.com> X-Mailer: git-send-email 2.37.1.595.g718a3a8f04-goog Subject: [PATCH v14 06/14] mm: multi-gen LRU: minimal implementation From: Yu Zhao To: Andrew Morton Cc: Andi Kleen , Aneesh Kumar , Catalin Marinas , Dave Hansen , Hillf Danton , Jens Axboe , Johannes Weiner , Jonathan Corbet , Linus Torvalds , Matthew Wilcox , Mel Gorman , Michael Larabel , Michal Hocko , Mike Rapoport , Peter Zijlstra , Tejun Heo , Vlastimil Babka , Will Deacon , linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org, page-reclaim@google.com, Yu Zhao , Brian Geffon , Jan Alexander Steffens , Oleksandr Natalenko , Steven Barrett , Suleiman Souhlal , Daniel Byrne , Donald Carr , "=?UTF-8?q?Holger=20Hoffst=C3=A4tte?=" , Konstantin Kharlamov , Shuang Zhai , Sofia Trinh , Vaibhav Jain X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220815_081424_694089_2E195FDF X-CRM114-Status: GOOD ( 26.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org VG8gYXZvaWQgY29uZnVzaW9uLCB0aGUgdGVybXMgInByb21vdGlvbiIgYW5kICJkZW1vdGlvbiIg d2lsbCBiZQphcHBsaWVkIHRvIHRoZSBtdWx0aS1nZW4gTFJVLCBhcyBhIG5ldyBjb252ZW50aW9u OyB0aGUgdGVybXMKImFjdGl2YXRpb24iIGFuZCAiZGVhY3RpdmF0aW9uIiB3aWxsIGJlIGFwcGxp ZWQgdG8gdGhlIGFjdGl2ZS9pbmFjdGl2ZQpMUlUsIGFzIHVzdWFsLgoKVGhlIGFnaW5nIHByb2R1 Y2VzIHlvdW5nIGdlbmVyYXRpb25zLiBHaXZlbiBhbiBscnV2ZWMsIGl0IGluY3JlbWVudHMKbWF4 X3NlcSB3aGVuIG1heF9zZXEtbWluX3NlcSsxIGFwcHJvYWNoZXMgTUlOX05SX0dFTlMuIFRoZSBh Z2luZwpwcm9tb3RlcyBob3QgcGFnZXMgdG8gdGhlIHlvdW5nZXN0IGdlbmVyYXRpb24gd2hlbiBp dCBmaW5kcyB0aGVtCmFjY2Vzc2VkIHRocm91Z2ggcGFnZSB0YWJsZXM7IHRoZSBkZW1vdGlvbiBv ZiBjb2xkIHBhZ2VzIGhhcHBlbnMKY29uc2VxdWVudGx5IHdoZW4gaXQgaW5jcmVtZW50cyBtYXhf c2VxLiBQcm9tb3Rpb24gaW4gdGhlIGFnaW5nIHBhdGgKZG9lcyBub3QgaW52b2x2ZSBhbnkgTFJV IGxpc3Qgb3BlcmF0aW9ucywgb25seSB0aGUgdXBkYXRlcyBvZiB0aGUgZ2VuCmNvdW50ZXIgYW5k IGxydWdlbi0+bnJfcGFnZXNbXTsgZGVtb3Rpb24sIHVubGVzcyBhcyB0aGUgcmVzdWx0IG9mIHRo ZQppbmNyZW1lbnQgb2YgbWF4X3NlcSwgcmVxdWlyZXMgTFJVIGxpc3Qgb3BlcmF0aW9ucywgZS5n LiwKbHJ1X2RlYWN0aXZhdGVfZm4oKS4gVGhlIGFnaW5nIGhhcyB0aGUgY29tcGxleGl0eSBPKG5y X2hvdF9wYWdlcyksCnNpbmNlIGl0IGlzIG9ubHkgaW50ZXJlc3RlZCBpbiBob3QgcGFnZXMuCgpU aGUgZXZpY3Rpb24gY29uc3VtZXMgb2xkIGdlbmVyYXRpb25zLiBHaXZlbiBhbiBscnV2ZWMsIGl0 IGluY3JlbWVudHMKbWluX3NlcSB3aGVuIGxydWdlbi0+bGlzdHNbXSBpbmRleGVkIGJ5IG1pbl9z ZXElTUFYX05SX0dFTlMgYmVjb21lcwplbXB0eS4gQSBmZWVkYmFjayBsb29wIG1vZGVsZWQgYWZ0 ZXIgdGhlIFBJRCBjb250cm9sbGVyIG1vbml0b3JzCnJlZmF1bHRzIG92ZXIgYW5vbiBhbmQgZmls ZSB0eXBlcyBhbmQgZGVjaWRlcyB3aGljaCB0eXBlIHRvIGV2aWN0IHdoZW4KYm90aCB0eXBlcyBh cmUgYXZhaWxhYmxlIGZyb20gdGhlIHNhbWUgZ2VuZXJhdGlvbi4KClRoZSBwcm90ZWN0aW9uIG9m IHBhZ2VzIGFjY2Vzc2VkIG11bHRpcGxlIHRpbWVzIHRocm91Z2ggZmlsZQpkZXNjcmlwdG9ycyB0 YWtlcyBwbGFjZSBpbiB0aGUgZXZpY3Rpb24gcGF0aC4gRWFjaCBnZW5lcmF0aW9uIGlzCmRpdmlk ZWQgaW50byBtdWx0aXBsZSB0aWVycy4gQSBwYWdlIGFjY2Vzc2VkIE4gdGltZXMgdGhyb3VnaCBm aWxlCmRlc2NyaXB0b3JzIGlzIGluIHRpZXIgb3JkZXJfYmFzZV8yKE4pLiBUaWVycyBkbyBub3Qg aGF2ZSBkZWRpY2F0ZWQKbHJ1Z2VuLT5saXN0c1tdLCBvbmx5IGJpdHMgaW4gZm9saW8tPmZsYWdz LiBUaGUgYWZvcmVtZW50aW9uZWQKZmVlZGJhY2sgbG9vcCBhbHNvIG1vbml0b3JzIHJlZmF1bHRz IG92ZXIgYWxsIHRpZXJzIGFuZCBkZWNpZGVzIHdoZW4KdG8gcHJvdGVjdCBwYWdlcyBpbiB3aGlj aCB0aWVycyAoTj4xKSwgdXNpbmcgdGhlIGZpcnN0IHRpZXIgKE49MCwxKSBhcwphIGJhc2VsaW5l LiBUaGUgZmlyc3QgdGllciBjb250YWlucyBzaW5nbGUtdXNlIHVubWFwcGVkIGNsZWFuIHBhZ2Vz LAp3aGljaCBhcmUgbW9zdCBsaWtlbHkgdGhlIGJlc3QgY2hvaWNlcy4gSW4gY29udHJhc3QgdG8g cHJvbW90aW9uIGluCnRoZSBhZ2luZyBwYXRoLCB0aGUgcHJvdGVjdGlvbiBvZiBhIHBhZ2UgaW4g dGhlIGV2aWN0aW9uIHBhdGggaXMKYWNoaWV2ZWQgYnkgbW92aW5nIHRoaXMgcGFnZSB0byB0aGUg bmV4dCBnZW5lcmF0aW9uLCBpLmUuLCBtaW5fc2VxKzEsCmlmIHRoZSBmZWVkYmFjayBsb29wIGRl Y2lkZXMgc28uIFRoaXMgYXBwcm9hY2ggaGFzIHRoZSBmb2xsb3dpbmcKYWR2YW50YWdlczoKMS4g SXQgcmVtb3ZlcyB0aGUgY29zdCBvZiBhY3RpdmF0aW9uIGluIHRoZSBidWZmZXJlZCBhY2Nlc3Mg cGF0aCBieQogICBpbmZlcnJpbmcgd2hldGhlciBwYWdlcyBhY2Nlc3NlZCBtdWx0aXBsZSB0aW1l cyB0aHJvdWdoIGZpbGUKICAgZGVzY3JpcHRvcnMgYXJlIHN0YXRpc3RpY2FsbHkgaG90IGFuZCB0 aHVzIHdvcnRoIHByb3RlY3RpbmcgaW4gdGhlCiAgIGV2aWN0aW9uIHBhdGguCjIuIEl0IHRha2Vz IHBhZ2VzIGFjY2Vzc2VkIHRocm91Z2ggcGFnZSB0YWJsZXMgaW50byBhY2NvdW50IGFuZCBhdm9p ZHMKICAgb3ZlcnByb3RlY3RpbmcgcGFnZXMgYWNjZXNzZWQgbXVsdGlwbGUgdGltZXMgdGhyb3Vn aCBmaWxlCiAgIGRlc2NyaXB0b3JzLiAoUGFnZXMgYWNjZXNzZWQgdGhyb3VnaCBwYWdlIHRhYmxl cyBhcmUgaW4gdGhlIGZpcnN0CiAgIHRpZXIsIHNpbmNlIE49MC4pCjMuIE1vcmUgdGllcnMgcHJv dmlkZSBiZXR0ZXIgcHJvdGVjdGlvbiBmb3IgcGFnZXMgYWNjZXNzZWQgbW9yZSB0aGFuCiAgIHR3 aWNlIHRocm91Z2ggZmlsZSBkZXNjcmlwdG9ycywgd2hlbiB1bmRlciBoZWF2eSBidWZmZXJlZCBJ L08KICAgd29ya2xvYWRzLgoKU2VydmVyIGJlbmNobWFyayByZXN1bHRzOgogIFNpbmdsZSB3b3Jr bG9hZDoKICAgIGZpbyAoYnVmZmVyZWQgSS9PKTogK1szMCwgMzJdJQogICAgICAgICAgICAgICAg SU9QUyAgICAgICAgIEJXCiAgICAgIDUuMTktcmMxOiAyNjczayAgICAgICAgMTAuMkdpQi9zCiAg ICAgIHBhdGNoMS02OiAzNDkxayAgICAgICAgMTMuM0dpQi9zCgogIFNpbmdsZSB3b3JrbG9hZDoK ICAgIG1lbWNhY2hlZCAoYW5vbik6IC1bNCwgNl0lCiAgICAgICAgICAgICAgICBPcHMvc2VjICAg ICAgS0Ivc2VjCiAgICAgIDUuMTktcmMxOiAxMTYxNTAxLjA0ICAgNDUxNzcuMjUKICAgICAgcGF0 Y2gxLTY6IDExMDYxNjguNDYgICA0MzAyNS4wNAoKICBDb25maWd1cmF0aW9uczoKICAgIENQVTog dHdvIFhlb24gNjE1NAogICAgTWVtOiB0b3RhbCAyNTZHCgogICAgTm9kZSAxIHdhcyBvbmx5IHVz ZWQgYXMgYSByYW0gZGlzayB0byByZWR1Y2UgdGhlIHZhcmlhbmNlIGluIHRoZQogICAgcmVzdWx0 cy4KCiAgICBwYXRjaCBkcml2ZXJzL2Jsb2NrL2JyZC5jIDw8RU9GCiAgICA5OSwxMDBjOTksMTAw CiAgICA8IAlnZnBfZmxhZ3MgPSBHRlBfTk9JTyB8IF9fR0ZQX1pFUk8gfCBfX0dGUF9ISUdITUVN OwogICAgPCAJcGFnZSA9IGFsbG9jX3BhZ2UoZ2ZwX2ZsYWdzKTsKICAgIC0tLQogICAgPiAJZ2Zw X2ZsYWdzID0gR0ZQX05PSU8gfCBfX0dGUF9aRVJPIHwgX19HRlBfSElHSE1FTSB8IF9fR0ZQX1RI SVNOT0RFOwogICAgPiAJcGFnZSA9IGFsbG9jX3BhZ2VzX25vZGUoMSwgZ2ZwX2ZsYWdzLCAwKTsK ICAgIEVPRgoKICAgIGNhdCA+Pi9ldGMvc3lzdGVtZC9zeXN0ZW0uY29uZiA8PEVPRgogICAgQ1BV QWZmaW5pdHk9bnVtYQogICAgTlVNQVBvbGljeT1iaW5kCiAgICBOVU1BTWFzaz0wCiAgICBFT0YK CiAgICBjYXQgPj4vZXRjL21lbWNhY2hlZC5jb25mIDw8RU9GCiAgICAtbSAxODQzMjAKICAgIC1z IC92YXIvcnVuL21lbWNhY2hlZC9tZW1jYWNoZWQuc29jawogICAgLWEgMDc2NgogICAgLXQgMzYK ICAgIC1CIGJpbmFyeQogICAgRU9GCgogICAgY2F0IGZpby5zaAogICAgbW9kcHJvYmUgYnJkIHJk X25yPTEgcmRfc2l6ZT0xMTMyNDYyMDgKICAgIHN3YXBvZmYgLWEKICAgIG1rZnMuZXh0NCAvZGV2 L3JhbTAKICAgIG1vdW50IC10IGV4dDQgL2Rldi9yYW0wIC9tbnQKCiAgICBta2RpciAvc3lzL2Zz L2Nncm91cC91c2VyLnNsaWNlL3Rlc3QKICAgIGVjaG8gMzg2NTQ3MDU2NjQgPi9zeXMvZnMvY2dy b3VwL3VzZXIuc2xpY2UvdGVzdC9tZW1vcnkubWF4CiAgICBlY2hvICQkID4vc3lzL2ZzL2Nncm91 cC91c2VyLnNsaWNlL3Rlc3QvY2dyb3VwLnByb2NzCiAgICBmaW8gLW5hbWU9bWdscnUgLS1udW1q b2JzPTcyIC0tZGlyZWN0b3J5PS9tbnQgLS1zaXplPTE0MDhtIFwKICAgICAgLS1idWZmZXJlZD0x IC0taW9lbmdpbmU9aW9fdXJpbmcgLS1pb2RlcHRoPTEyOCBcCiAgICAgIC0taW9kZXB0aF9iYXRj aF9zdWJtaXQ9MzIgLS1pb2RlcHRoX2JhdGNoX2NvbXBsZXRlPTMyIFwKICAgICAgLS1ydz1yYW5k cmVhZCAtLXJhbmRvbV9kaXN0cmlidXRpb249cmFuZG9tIC0tbm9yYW5kb21tYXAgXAogICAgICAt LXRpbWVfYmFzZWQgLS1yYW1wX3RpbWU9MTBtIC0tcnVudGltZT01bSAtLWdyb3VwX3JlcG9ydGlu ZwoKICAgIGNhdCBtZW1jYWNoZWQuc2gKICAgIG1vZHByb2JlIGJyZCByZF9ucj0xIHJkX3NpemU9 MTEzMjQ2MjA4CiAgICBzd2Fwb2ZmIC1hCiAgICBta3N3YXAgL2Rldi9yYW0wCiAgICBzd2Fwb24g L2Rldi9yYW0wCgogICAgbWVtdGllcl9iZW5jaG1hcmsgLVMgL3Zhci9ydW4vbWVtY2FjaGVkL21l bWNhY2hlZC5zb2NrIFwKICAgICAgLVAgbWVtY2FjaGVfYmluYXJ5IC1uIGFsbGtleXMgLS1rZXkt bWluaW11bT0xIFwKICAgICAgLS1rZXktbWF4aW11bT02NTAwMDAwMCAtLWtleS1wYXR0ZXJuPVA6 UCAtYyAxIC10IDM2IFwKICAgICAgLS1yYXRpbyAxOjAgLS1waXBlbGluZSA4IC1kIDIwMDAKCiAg ICBtZW10aWVyX2JlbmNobWFyayAtUyAvdmFyL3J1bi9tZW1jYWNoZWQvbWVtY2FjaGVkLnNvY2sg XAogICAgICAtUCBtZW1jYWNoZV9iaW5hcnkgLW4gYWxsa2V5cyAtLWtleS1taW5pbXVtPTEgXAog ICAgICAtLWtleS1tYXhpbXVtPTY1MDAwMDAwIC0ta2V5LXBhdHRlcm49UjpSIC1jIDEgLXQgMzYg XAogICAgICAtLXJhdGlvIDA6MSAtLXBpcGVsaW5lIDggLS1yYW5kb21pemUgLS1kaXN0aW5jdC1j bGllbnQtc2VlZAoKQ2xpZW50IGJlbmNobWFyayByZXN1bHRzOgogIGtzd2FwZCBwcm9maWxlczoK ICAgIDUuMTktcmMxCiAgICAgIDQwLjMzJSAgcGFnZV92bWFfbWFwcGVkX3dhbGsgKG92ZXJoZWFk KQogICAgICAyMS44MCUgIGx6bzF4XzFfZG9fY29tcHJlc3MgKHJlYWwgd29yaykKICAgICAgIDcu NTMlICBkb19yYXdfc3Bpbl9sb2NrCiAgICAgICAzLjk1JSAgX3Jhd19zcGluX3VubG9ja19pcnEK ICAgICAgIDIuNTIlICB2bWFfaW50ZXJ2YWxfdHJlZV9pdGVyX25leHQKICAgICAgIDIuMzclICBm b2xpb19yZWZlcmVuY2VkX29uZQogICAgICAgMi4yOCUgIHZtYV9pbnRlcnZhbF90cmVlX3N1YnRy ZWVfc2VhcmNoCiAgICAgICAxLjk3JSAgYW5vbl92bWFfaW50ZXJ2YWxfdHJlZV9pdGVyX2ZpcnN0 CiAgICAgICAxLjYwJSAgcHRlcF9jbGVhcl9mbHVzaAogICAgICAgMS4wNiUgIF9fenJhbV9idmVj X3dyaXRlCgogICAgcGF0Y2gxLTYKICAgICAgMzkuMDMlICBsem8xeF8xX2RvX2NvbXByZXNzIChy ZWFsIHdvcmspCiAgICAgIDE4LjQ3JSAgcGFnZV92bWFfbWFwcGVkX3dhbGsgKG92ZXJoZWFkKQog ICAgICAgNi43NCUgIF9yYXdfc3Bpbl91bmxvY2tfaXJxCiAgICAgICAzLjk3JSAgZG9fcmF3X3Nw aW5fbG9jawogICAgICAgMi40OSUgIHB0ZXBfY2xlYXJfZmx1c2gKICAgICAgIDIuNDglICBhbm9u X3ZtYV9pbnRlcnZhbF90cmVlX2l0ZXJfZmlyc3QKICAgICAgIDEuOTIlICBmb2xpb19yZWZlcmVu Y2VkX29uZQogICAgICAgMS44OCUgIF9fenJhbV9idmVjX3dyaXRlCiAgICAgICAxLjQ4JSAgbWVt bW92ZQogICAgICAgMS4zMSUgIHZtYV9pbnRlcnZhbF90cmVlX2l0ZXJfbmV4dAoKICBDb25maWd1 cmF0aW9uczoKICAgIENQVTogc2luZ2xlIFNuYXBkcmFnb24gN2MKICAgIE1lbTogdG90YWwgNEcK CiAgICBDaHJvbWUgT1MgTWVtb3J5UHJlc3N1cmUgWzFdCgpbMV0gaHR0cHM6Ly9jaHJvbWl1bS5n b29nbGVzb3VyY2UuY29tL2Nocm9taXVtb3MvcGxhdGZvcm0vdGFzdC10ZXN0cy8KClNpZ25lZC1v ZmYtYnk6IFl1IFpoYW8gPHl1emhhb0Bnb29nbGUuY29tPgpBY2tlZC1ieTogQnJpYW4gR2VmZm9u IDxiZ2VmZm9uQGdvb2dsZS5jb20+CkFja2VkLWJ5OiBKYW4gQWxleGFuZGVyIFN0ZWZmZW5zICho ZWZ0aWcpIDxoZWZ0aWdAYXJjaGxpbnV4Lm9yZz4KQWNrZWQtYnk6IE9sZWtzYW5kciBOYXRhbGVu a28gPG9sZWtzYW5kckBuYXRhbGVua28ubmFtZT4KQWNrZWQtYnk6IFN0ZXZlbiBCYXJyZXR0IDxz dGV2ZW5AbGlxdW9yaXgubmV0PgpBY2tlZC1ieTogU3VsZWltYW4gU291aGxhbCA8c3VsZWltYW5A Z29vZ2xlLmNvbT4KVGVzdGVkLWJ5OiBEYW5pZWwgQnlybmUgPGRqYnlybmVAbXR1LmVkdT4KVGVz dGVkLWJ5OiBEb25hbGQgQ2FyciA8ZEBjaGFvcy1yZWlucy5jb20+ClRlc3RlZC1ieTogSG9sZ2Vy IEhvZmZzdMOkdHRlIDxob2xnZXJAYXBwbGllZC1hc3luY2hyb255LmNvbT4KVGVzdGVkLWJ5OiBL b25zdGFudGluIEtoYXJsYW1vdiA8SGktQW5nZWxAeWFuZGV4LnJ1PgpUZXN0ZWQtYnk6IFNodWFu ZyBaaGFpIDxzemhhaTJAY3Mucm9jaGVzdGVyLmVkdT4KVGVzdGVkLWJ5OiBTb2ZpYSBUcmluaCA8 c29maWEudHJpbmhAZWRpLndvcmtzPgpUZXN0ZWQtYnk6IFZhaWJoYXYgSmFpbiA8dmFpYmhhdkBs aW51eC5pYm0uY29tPgotLS0KIGluY2x1ZGUvbGludXgvbW1faW5saW5lLmggICAgICAgICB8ICAz NiArKwogaW5jbHVkZS9saW51eC9tbXpvbmUuaCAgICAgICAgICAgIHwgIDQxICsrCiBpbmNsdWRl L2xpbnV4L3BhZ2UtZmxhZ3MtbGF5b3V0LmggfCAgIDUgKy0KIGtlcm5lbC9ib3VuZHMuYyAgICAg ICAgICAgICAgICAgICB8ICAgMiArCiBtbS9LY29uZmlnICAgICAgICAgICAgICAgICAgICAgICAg fCAgMTEgKwogbW0vc3dhcC5jICAgICAgICAgICAgICAgICAgICAgICAgIHwgIDM5ICsrCiBtbS92 bXNjYW4uYyAgICAgICAgICAgICAgICAgICAgICAgfCA4MTUgKysrKysrKysrKysrKysrKysrKysr KysrKysrKystCiBtbS93b3JraW5nc2V0LmMgICAgICAgICAgICAgICAgICAgfCAxMTAgKysrLQog OCBmaWxlcyBjaGFuZ2VkLCAxMDQ5IGluc2VydGlvbnMoKyksIDEwIGRlbGV0aW9ucygtKQoKZGlm ZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvbW1faW5saW5lLmggYi9pbmNsdWRlL2xpbnV4L21tX2lu bGluZS5oCmluZGV4IDJmZjcwMzkwMGZkMC4uZjJiMjI5NmE0MmY5IDEwMDY0NAotLS0gYS9pbmNs dWRlL2xpbnV4L21tX2lubGluZS5oCisrKyBiL2luY2x1ZGUvbGludXgvbW1faW5saW5lLmgKQEAg LTEyMSw2ICsxMjEsMzMgQEAgc3RhdGljIGlubGluZSBpbnQgbHJ1X2dlbl9mcm9tX3NlcSh1bnNp Z25lZCBsb25nIHNlcSkKIAlyZXR1cm4gc2VxICUgTUFYX05SX0dFTlM7CiB9CiAKK3N0YXRpYyBp bmxpbmUgaW50IGxydV9oaXN0X2Zyb21fc2VxKHVuc2lnbmVkIGxvbmcgc2VxKQoreworCXJldHVy biBzZXEgJSBOUl9ISVNUX0dFTlM7Cit9CisKK3N0YXRpYyBpbmxpbmUgaW50IGxydV90aWVyX2Zy b21fcmVmcyhpbnQgcmVmcykKK3sKKwlWTV9XQVJOX09OX09OQ0UocmVmcyA+IEJJVChMUlVfUkVG U19XSURUSCkpOworCisJLyogc2VlIHRoZSBjb21tZW50IGluIGZvbGlvX2xydV9yZWZzKCkgKi8K KwlyZXR1cm4gb3JkZXJfYmFzZV8yKHJlZnMgKyAxKTsKK30KKworc3RhdGljIGlubGluZSBpbnQg Zm9saW9fbHJ1X3JlZnMoc3RydWN0IGZvbGlvICpmb2xpbykKK3sKKwl1bnNpZ25lZCBsb25nIGZs YWdzID0gUkVBRF9PTkNFKGZvbGlvLT5mbGFncyk7CisJYm9vbCB3b3JraW5nc2V0ID0gZmxhZ3Mg JiBCSVQoUEdfd29ya2luZ3NldCk7CisKKwkvKgorCSAqIFJldHVybiB0aGUgbnVtYmVyIG9mIGFj Y2Vzc2VzIGJleW9uZCBQR19yZWZlcmVuY2VkLCBpLmUuLCBOLTEgaWYgdGhlCisJICogdG90YWwg bnVtYmVyIG9mIGFjY2Vzc2VzIGlzIE4+MSwgc2luY2UgTj0wLDEgYm90aCBtYXAgdG8gdGhlIGZp cnN0CisJICogdGllci4gbHJ1X3RpZXJfZnJvbV9yZWZzKCkgd2lsbCBhY2NvdW50IGZvciB0aGlz IG9mZi1ieS1vbmUuIEFsc28gc2VlCisJICogdGhlIGNvbW1lbnQgb24gTUFYX05SX1RJRVJTLgor CSAqLworCXJldHVybiAoKGZsYWdzICYgTFJVX1JFRlNfTUFTSykgPj4gTFJVX1JFRlNfUEdPRkYp ICsgd29ya2luZ3NldDsKK30KKwogc3RhdGljIGlubGluZSBpbnQgZm9saW9fbHJ1X2dlbihzdHJ1 Y3QgZm9saW8gKmZvbGlvKQogewogCXVuc2lnbmVkIGxvbmcgZmxhZ3MgPSBSRUFEX09OQ0UoZm9s aW8tPmZsYWdzKTsKQEAgLTE3Myw2ICsyMDAsMTUgQEAgc3RhdGljIGlubGluZSB2b2lkIGxydV9n ZW5fdXBkYXRlX3NpemUoc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3QgZm9saW8gKmZvbGkK IAkJX191cGRhdGVfbHJ1X3NpemUobHJ1dmVjLCBscnUsIHpvbmUsIC1kZWx0YSk7CiAJCXJldHVy bjsKIAl9CisKKwkvKiBwcm9tb3Rpb24gKi8KKwlpZiAoIWxydV9nZW5faXNfYWN0aXZlKGxydXZl Yywgb2xkX2dlbikgJiYgbHJ1X2dlbl9pc19hY3RpdmUobHJ1dmVjLCBuZXdfZ2VuKSkgeworCQlf X3VwZGF0ZV9scnVfc2l6ZShscnV2ZWMsIGxydSwgem9uZSwgLWRlbHRhKTsKKwkJX191cGRhdGVf bHJ1X3NpemUobHJ1dmVjLCBscnUgKyBMUlVfQUNUSVZFLCB6b25lLCBkZWx0YSk7CisJfQorCisJ LyogZGVtb3Rpb24gcmVxdWlyZXMgaXNvbGF0aW9uLCBlLmcuLCBscnVfZGVhY3RpdmF0ZV9mbigp ICovCisJVk1fV0FSTl9PTl9PTkNFKGxydV9nZW5faXNfYWN0aXZlKGxydXZlYywgb2xkX2dlbikg JiYgIWxydV9nZW5faXNfYWN0aXZlKGxydXZlYywgbmV3X2dlbikpOwogfQogCiBzdGF0aWMgaW5s aW5lIGJvb2wgbHJ1X2dlbl9hZGRfZm9saW8oc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3Qg Zm9saW8gKmZvbGlvLCBib29sIHJlY2xhaW1pbmcpCmRpZmYgLS1naXQgYS9pbmNsdWRlL2xpbnV4 L21tem9uZS5oIGIvaW5jbHVkZS9saW51eC9tbXpvbmUuaAppbmRleCA4NjE0N2YwNGJmNzYuLjAx OWQ3YzhlZTgzNCAxMDA2NDQKLS0tIGEvaW5jbHVkZS9saW51eC9tbXpvbmUuaAorKysgYi9pbmNs dWRlL2xpbnV4L21tem9uZS5oCkBAIC0zNTAsNiArMzUwLDI4IEBAIGVudW0gbHJ1dmVjX2ZsYWdz IHsKICNkZWZpbmUgTUlOX05SX0dFTlMJCTJVCiAjZGVmaW5lIE1BWF9OUl9HRU5TCQk0VQogCisv KgorICogRWFjaCBnZW5lcmF0aW9uIGlzIGRpdmlkZWQgaW50byBtdWx0aXBsZSB0aWVycy4gQSBw YWdlIGFjY2Vzc2VkIE4gdGltZXMKKyAqIHRocm91Z2ggZmlsZSBkZXNjcmlwdG9ycyBpcyBpbiB0 aWVyIG9yZGVyX2Jhc2VfMihOKS4gQSBwYWdlIGluIHRoZSBmaXJzdCB0aWVyCisgKiAoTj0wLDEp IGlzIG1hcmtlZCBieSBQR19yZWZlcmVuY2VkIHVubGVzcyBpdCB3YXMgZmF1bHRlZCBpbiB0aHJv dWdoIHBhZ2UKKyAqIHRhYmxlcyBvciByZWFkIGFoZWFkLiBBIHBhZ2UgaW4gYW55IG90aGVyIHRp ZXIgKE4+MSkgaXMgbWFya2VkIGJ5CisgKiBQR19yZWZlcmVuY2VkIGFuZCBQR193b3JraW5nc2V0 LiBUaGlzIGltcGxpZXMgYSBtaW5pbXVtIG9mIHR3byB0aWVycyBpcworICogc3VwcG9ydGVkIHdp dGhvdXQgdXNpbmcgYWRkaXRpb25hbCBiaXRzIGluIGZvbGlvLT5mbGFncy4KKyAqCisgKiBJbiBj b250cmFzdCB0byBtb3ZpbmcgYWNyb3NzIGdlbmVyYXRpb25zIHdoaWNoIHJlcXVpcmVzIHRoZSBM UlUgbG9jaywgbW92aW5nCisgKiBhY3Jvc3MgdGllcnMgb25seSBpbnZvbHZlcyBhdG9taWMgb3Bl cmF0aW9ucyBvbiBmb2xpby0+ZmxhZ3MgYW5kIHRoZXJlZm9yZQorICogaGFzIGEgbmVnbGlnaWJs ZSBjb3N0IGluIHRoZSBidWZmZXJlZCBhY2Nlc3MgcGF0aC4gSW4gdGhlIGV2aWN0aW9uIHBhdGgs CisgKiBjb21wYXJpc29ucyBvZiByZWZhdWx0ZWQvKGV2aWN0ZWQrcHJvdGVjdGVkKSBmcm9tIHRo ZSBmaXJzdCB0aWVyIGFuZCB0aGUKKyAqIHJlc3QgaW5mZXIgd2hldGhlciBwYWdlcyBhY2Nlc3Nl ZCBtdWx0aXBsZSB0aW1lcyB0aHJvdWdoIGZpbGUgZGVzY3JpcHRvcnMKKyAqIGFyZSBzdGF0aXN0 aWNhbGx5IGhvdCBhbmQgdGh1cyB3b3J0aCBwcm90ZWN0aW5nLgorICoKKyAqIE1BWF9OUl9USUVS UyBpcyBzZXQgdG8gNCBzbyB0aGF0IHRoZSBtdWx0aS1nZW4gTFJVIGNhbiBzdXBwb3J0IHR3aWNl IHRoZQorICogbnVtYmVyIG9mIGNhdGVnb3JpZXMgb2YgdGhlIGFjdGl2ZS9pbmFjdGl2ZSBMUlUg d2hlbiBrZWVwaW5nIHRyYWNrIG9mCisgKiBhY2Nlc3NlcyB0aHJvdWdoIGZpbGUgZGVzY3JpcHRv cnMuIFRoaXMgdXNlcyBNQVhfTlJfVElFUlMtMiBzcGFyZSBiaXRzIGluCisgKiBmb2xpby0+Zmxh Z3MuCisgKi8KKyNkZWZpbmUgTUFYX05SX1RJRVJTCQk0VQorCiAjaWZuZGVmIF9fR0VORVJBVElO R19CT1VORFNfSAogCiBzdHJ1Y3QgbHJ1dmVjOwpAQCAtMzY0LDYgKzM4NiwxNiBAQCBlbnVtIHsK IAlMUlVfR0VOX0ZJTEUsCiB9OwogCisjZGVmaW5lIE1JTl9MUlVfQkFUQ0gJCUJJVFNfUEVSX0xP TkcKKyNkZWZpbmUgTUFYX0xSVV9CQVRDSAkJKE1JTl9MUlVfQkFUQ0ggKiAxMjgpCisKKy8qIHdo ZXRoZXIgdG8ga2VlcCBoaXN0b3JpY2FsIHN0YXRzIGZyb20gZXZpY3RlZCBnZW5lcmF0aW9ucyAq LworI2lmZGVmIENPTkZJR19MUlVfR0VOX1NUQVRTCisjZGVmaW5lIE5SX0hJU1RfR0VOUwkJTUFY X05SX0dFTlMKKyNlbHNlCisjZGVmaW5lIE5SX0hJU1RfR0VOUwkJMVUKKyNlbmRpZgorCiAvKgog ICogVGhlIHlvdW5nZXN0IGdlbmVyYXRpb24gbnVtYmVyIGlzIHN0b3JlZCBpbiBtYXhfc2VxIGZv ciBib3RoIGFub24gYW5kIGZpbGUKICAqIHR5cGVzIGFzIHRoZXkgYXJlIGFnZWQgb24gYW4gZXF1 YWwgZm9vdGluZy4gVGhlIG9sZGVzdCBnZW5lcmF0aW9uIG51bWJlcnMgYXJlCkBAIC0zODYsNiAr NDE4LDE1IEBAIHN0cnVjdCBscnVfZ2VuX3N0cnVjdCB7CiAJc3RydWN0IGxpc3RfaGVhZCBsaXN0 c1tNQVhfTlJfR0VOU11bQU5PTl9BTkRfRklMRV1bTUFYX05SX1pPTkVTXTsKIAkvKiB0aGUgbXVs dGktZ2VuIExSVSBzaXplcywgZXZlbnR1YWxseSBjb25zaXN0ZW50ICovCiAJbG9uZyBucl9wYWdl c1tNQVhfTlJfR0VOU11bQU5PTl9BTkRfRklMRV1bTUFYX05SX1pPTkVTXTsKKwkvKiB0aGUgZXhw b25lbnRpYWwgbW92aW5nIGF2ZXJhZ2Ugb2YgcmVmYXVsdGVkICovCisJdW5zaWduZWQgbG9uZyBh dmdfcmVmYXVsdGVkW0FOT05fQU5EX0ZJTEVdW01BWF9OUl9USUVSU107CisJLyogdGhlIGV4cG9u ZW50aWFsIG1vdmluZyBhdmVyYWdlIG9mIGV2aWN0ZWQrcHJvdGVjdGVkICovCisJdW5zaWduZWQg bG9uZyBhdmdfdG90YWxbQU5PTl9BTkRfRklMRV1bTUFYX05SX1RJRVJTXTsKKwkvKiB0aGUgZmly c3QgdGllciBkb2Vzbid0IG5lZWQgcHJvdGVjdGlvbiwgaGVuY2UgdGhlIG1pbnVzIG9uZSAqLwor CXVuc2lnbmVkIGxvbmcgcHJvdGVjdGVkW05SX0hJU1RfR0VOU11bQU5PTl9BTkRfRklMRV1bTUFY X05SX1RJRVJTIC0gMV07CisJLyogY2FuIGJlIG1vZGlmaWVkIHdpdGhvdXQgaG9sZGluZyB0aGUg TFJVIGxvY2sgKi8KKwlhdG9taWNfbG9uZ190IGV2aWN0ZWRbTlJfSElTVF9HRU5TXVtBTk9OX0FO RF9GSUxFXVtNQVhfTlJfVElFUlNdOworCWF0b21pY19sb25nX3QgcmVmYXVsdGVkW05SX0hJU1Rf R0VOU11bQU5PTl9BTkRfRklMRV1bTUFYX05SX1RJRVJTXTsKIH07CiAKIHZvaWQgbHJ1X2dlbl9p bml0X2xydXZlYyhzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMpOwpkaWZmIC0tZ2l0IGEvaW5jbHVkZS9s aW51eC9wYWdlLWZsYWdzLWxheW91dC5oIGIvaW5jbHVkZS9saW51eC9wYWdlLWZsYWdzLWxheW91 dC5oCmluZGV4IDI0MDkwNTQwN2ExOC4uN2Q3OTgxOGRjMDY1IDEwMDY0NAotLS0gYS9pbmNsdWRl L2xpbnV4L3BhZ2UtZmxhZ3MtbGF5b3V0LmgKKysrIGIvaW5jbHVkZS9saW51eC9wYWdlLWZsYWdz LWxheW91dC5oCkBAIC0xMDYsNyArMTA2LDEwIEBACiAjZXJyb3IgIk5vdCBlbm91Z2ggYml0cyBp biBwYWdlIGZsYWdzIgogI2VuZGlmCiAKLSNkZWZpbmUgTFJVX1JFRlNfV0lEVEgJMAorLyogc2Vl IHRoZSBjb21tZW50IG9uIE1BWF9OUl9USUVSUyAqLworI2RlZmluZSBMUlVfUkVGU19XSURUSAlt aW4oX19MUlVfUkVGU19XSURUSCwgQklUU19QRVJfTE9ORyAtIE5SX1BBR0VGTEFHUyAtIFwKKwkJ CSAgICBaT05FU19XSURUSCAtIExSVV9HRU5fV0lEVEggLSBTRUNUSU9OU19XSURUSCAtIFwKKwkJ CSAgICBOT0RFU19XSURUSCAtIEtBU0FOX1RBR19XSURUSCAtIExBU1RfQ1BVUElEX1dJRFRIKQog CiAjZW5kaWYKICNlbmRpZiAvKiBfTElOVVhfUEFHRV9GTEFHU19MQVlPVVQgKi8KZGlmZiAtLWdp dCBhL2tlcm5lbC9ib3VuZHMuYyBiL2tlcm5lbC9ib3VuZHMuYwppbmRleCA1ZWU2MDc3N2Q4ZTQu LmI1MjkxODJlOGIwNCAxMDA2NDQKLS0tIGEva2VybmVsL2JvdW5kcy5jCisrKyBiL2tlcm5lbC9i b3VuZHMuYwpAQCAtMjQsOCArMjQsMTAgQEAgaW50IG1haW4odm9pZCkKIAlERUZJTkUoU1BJTkxP Q0tfU0laRSwgc2l6ZW9mKHNwaW5sb2NrX3QpKTsKICNpZmRlZiBDT05GSUdfTFJVX0dFTgogCURF RklORShMUlVfR0VOX1dJRFRILCBvcmRlcl9iYXNlXzIoTUFYX05SX0dFTlMgKyAxKSk7CisJREVG SU5FKF9fTFJVX1JFRlNfV0lEVEgsIE1BWF9OUl9USUVSUyAtIDIpOwogI2Vsc2UKIAlERUZJTkUo TFJVX0dFTl9XSURUSCwgMCk7CisJREVGSU5FKF9fTFJVX1JFRlNfV0lEVEgsIDApOwogI2VuZGlm CiAJLyogRW5kIG9mIGNvbnN0YW50cyAqLwogCmRpZmYgLS1naXQgYS9tbS9LY29uZmlnIGIvbW0v S2NvbmZpZwppbmRleCBkOTVmMDdjZDZkY2YuLjUxMDFkY2E4ZjIxYyAxMDA2NDQKLS0tIGEvbW0v S2NvbmZpZworKysgYi9tbS9LY29uZmlnCkBAIC0xMTI0LDYgKzExMjQsNyBAQCBjb25maWcgUFRF X01BUktFUl9VRkZEX1dQCiAJICBwdXJwb3Nlcy4gIEl0IGlzIHJlcXVpcmVkIHRvIGVuYWJsZSB1 c2VyZmF1bHRmZCB3cml0ZSBwcm90ZWN0aW9uIG9uCiAJICBmaWxlLWJhY2tlZCBtZW1vcnkgdHlw ZXMgbGlrZSBzaG1lbSBhbmQgaHVnZXRsYmZzLgogCisjIG11bHRpLWdlbiBMUlUgewogY29uZmln IExSVV9HRU4KIAlib29sICJNdWx0aS1HZW4gTFJVIgogCWRlcGVuZHMgb24gTU1VCkBAIC0xMTMy LDYgKzExMzMsMTYgQEAgY29uZmlnIExSVV9HRU4KIAloZWxwCiAJICBBIGhpZ2ggcGVyZm9ybWFu Y2UgTFJVIGltcGxlbWVudGF0aW9uIHRvIG92ZXJjb21taXQgbWVtb3J5LgogCitjb25maWcgTFJV X0dFTl9TVEFUUworCWJvb2wgIkZ1bGwgc3RhdHMgZm9yIGRlYnVnZ2luZyIKKwlkZXBlbmRzIG9u IExSVV9HRU4KKwloZWxwCisJICBEbyBub3QgZW5hYmxlIHRoaXMgb3B0aW9uIHVubGVzcyB5b3Ug cGxhbiB0byBsb29rIGF0IGhpc3RvcmljYWwgc3RhdHMKKwkgIGZyb20gZXZpY3RlZCBnZW5lcmF0 aW9ucyBmb3IgZGVidWdnaW5nIHB1cnBvc2UuCisKKwkgIFRoaXMgb3B0aW9uIGhhcyBhIHBlci1t ZW1jZyBhbmQgcGVyLW5vZGUgbWVtb3J5IG92ZXJoZWFkLgorIyB9CisKIHNvdXJjZSAibW0vZGFt b24vS2NvbmZpZyIKIAogZW5kbWVudQpkaWZmIC0tZ2l0IGEvbW0vc3dhcC5jIGIvbW0vc3dhcC5j CmluZGV4IDBlNDIzYjdkNDU4Yi4uZjc0ZmQ1MWZhOWUxIDEwMDY0NAotLS0gYS9tbS9zd2FwLmMK KysrIGIvbW0vc3dhcC5jCkBAIC00MjgsNiArNDI4LDQwIEBAIHN0YXRpYyB2b2lkIF9fbHJ1X2Nh Y2hlX2FjdGl2YXRlX2ZvbGlvKHN0cnVjdCBmb2xpbyAqZm9saW8pCiAJbG9jYWxfdW5sb2NrKCZj cHVfZmJhdGNoZXMubG9jayk7CiB9CiAKKyNpZmRlZiBDT05GSUdfTFJVX0dFTgorc3RhdGljIHZv aWQgZm9saW9faW5jX3JlZnMoc3RydWN0IGZvbGlvICpmb2xpbykKK3sKKwl1bnNpZ25lZCBsb25n IG5ld19mbGFncywgb2xkX2ZsYWdzID0gUkVBRF9PTkNFKGZvbGlvLT5mbGFncyk7CisKKwlpZiAo Zm9saW9fdGVzdF91bmV2aWN0YWJsZShmb2xpbykpCisJCXJldHVybjsKKworCWlmICghZm9saW9f dGVzdF9yZWZlcmVuY2VkKGZvbGlvKSkgeworCQlmb2xpb19zZXRfcmVmZXJlbmNlZChmb2xpbyk7 CisJCXJldHVybjsKKwl9CisKKwlpZiAoIWZvbGlvX3Rlc3Rfd29ya2luZ3NldChmb2xpbykpIHsK KwkJZm9saW9fc2V0X3dvcmtpbmdzZXQoZm9saW8pOworCQlyZXR1cm47CisJfQorCisJLyogc2Vl IHRoZSBjb21tZW50IG9uIE1BWF9OUl9USUVSUyAqLworCWRvIHsKKwkJbmV3X2ZsYWdzID0gb2xk X2ZsYWdzICYgTFJVX1JFRlNfTUFTSzsKKwkJaWYgKG5ld19mbGFncyA9PSBMUlVfUkVGU19NQVNL KQorCQkJYnJlYWs7CisKKwkJbmV3X2ZsYWdzICs9IEJJVChMUlVfUkVGU19QR09GRik7CisJCW5l d19mbGFncyB8PSBvbGRfZmxhZ3MgJiB+TFJVX1JFRlNfTUFTSzsKKwl9IHdoaWxlICghdHJ5X2Nt cHhjaGcoJmZvbGlvLT5mbGFncywgJm9sZF9mbGFncywgbmV3X2ZsYWdzKSk7Cit9CisjZWxzZQor c3RhdGljIHZvaWQgZm9saW9faW5jX3JlZnMoc3RydWN0IGZvbGlvICpmb2xpbykKK3sKK30KKyNl bmRpZiAvKiBDT05GSUdfTFJVX0dFTiAqLworCiAvKgogICogTWFyayBhIHBhZ2UgYXMgaGF2aW5n IHNlZW4gYWN0aXZpdHkuCiAgKgpAQCAtNDQwLDYgKzQ3NCwxMSBAQCBzdGF0aWMgdm9pZCBfX2xy dV9jYWNoZV9hY3RpdmF0ZV9mb2xpbyhzdHJ1Y3QgZm9saW8gKmZvbGlvKQogICovCiB2b2lkIGZv bGlvX21hcmtfYWNjZXNzZWQoc3RydWN0IGZvbGlvICpmb2xpbykKIHsKKwlpZiAobHJ1X2dlbl9l bmFibGVkKCkpIHsKKwkJZm9saW9faW5jX3JlZnMoZm9saW8pOworCQlyZXR1cm47CisJfQorCiAJ aWYgKCFmb2xpb190ZXN0X3JlZmVyZW5jZWQoZm9saW8pKSB7CiAJCWZvbGlvX3NldF9yZWZlcmVu Y2VkKGZvbGlvKTsKIAl9IGVsc2UgaWYgKGZvbGlvX3Rlc3RfdW5ldmljdGFibGUoZm9saW8pKSB7 CmRpZmYgLS1naXQgYS9tbS92bXNjYW4uYyBiL21tL3Ztc2Nhbi5jCmluZGV4IGVkOWUxNDliMTNj My4uNGM1N2ZiNzQ5YTc0IDEwMDY0NAotLS0gYS9tbS92bXNjYW4uYworKysgYi9tbS92bXNjYW4u YwpAQCAtMTMzNCw5ICsxMzM0LDExIEBAIHN0YXRpYyBpbnQgX19yZW1vdmVfbWFwcGluZyhzdHJ1 Y3QgYWRkcmVzc19zcGFjZSAqbWFwcGluZywgc3RydWN0IGZvbGlvICpmb2xpbywKIAogCWlmIChm b2xpb190ZXN0X3N3YXBjYWNoZShmb2xpbykpIHsKIAkJc3dwX2VudHJ5X3Qgc3dhcCA9IGZvbGlv X3N3YXBfZW50cnkoZm9saW8pOwotCQltZW1fY2dyb3VwX3N3YXBvdXQoZm9saW8sIHN3YXApOwor CisJCS8qIGdldCBhIHNoYWRvdyBlbnRyeSBiZWZvcmUgbWVtX2Nncm91cF9zd2Fwb3V0KCkgY2xl YXJzIGZvbGlvX21lbWNnKCkgKi8KIAkJaWYgKHJlY2xhaW1lZCAmJiAhbWFwcGluZ19leGl0aW5n KG1hcHBpbmcpKQogCQkJc2hhZG93ID0gd29ya2luZ3NldF9ldmljdGlvbihmb2xpbywgdGFyZ2V0 X21lbWNnKTsKKwkJbWVtX2Nncm91cF9zd2Fwb3V0KGZvbGlvLCBzd2FwKTsKIAkJX19kZWxldGVf ZnJvbV9zd2FwX2NhY2hlKGZvbGlvLCBzd2FwLCBzaGFkb3cpOwogCQl4YV91bmxvY2tfaXJxKCZt YXBwaW5nLT5pX3BhZ2VzKTsKIAkJcHV0X3N3YXBfcGFnZSgmZm9saW8tPnBhZ2UsIHN3YXApOwpA QCAtMjczMyw2ICsyNzM1LDkgQEAgc3RhdGljIHZvaWQgcHJlcGFyZV9zY2FuX2NvdW50KHBnX2Rh dGFfdCAqcGdkYXQsIHN0cnVjdCBzY2FuX2NvbnRyb2wgKnNjKQogCXVuc2lnbmVkIGxvbmcgZmls ZTsKIAlzdHJ1Y3QgbHJ1dmVjICp0YXJnZXRfbHJ1dmVjOwogCisJaWYgKGxydV9nZW5fZW5hYmxl ZCgpKQorCQlyZXR1cm47CisKIAl0YXJnZXRfbHJ1dmVjID0gbWVtX2Nncm91cF9scnV2ZWMoc2Mt PnRhcmdldF9tZW1fY2dyb3VwLCBwZ2RhdCk7CiAKIAkvKgpAQCAtMzA1Niw2ICszMDYxLDE3IEBA IHN0YXRpYyBib29sIGNhbl9hZ2VfYW5vbl9wYWdlcyhzdHJ1Y3QgcGdsaXN0X2RhdGEgKnBnZGF0 LAogICogICAgICAgICAgICAgICAgICAgICAgICAgIHNob3J0aGFuZCBoZWxwZXJzCiAgKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqLwogCisjZGVmaW5lIExSVV9SRUZTX0ZMQUdTCShCSVQoUEdfcmVmZXJl bmNlZCkgfCBCSVQoUEdfd29ya2luZ3NldCkpCisKKyNkZWZpbmUgREVGSU5FX01BWF9TRVEobHJ1 dmVjKQkJCQkJCVwKKwl1bnNpZ25lZCBsb25nIG1heF9zZXEgPSBSRUFEX09OQ0UoKGxydXZlYykt PmxydWdlbi5tYXhfc2VxKQorCisjZGVmaW5lIERFRklORV9NSU5fU0VRKGxydXZlYykJCQkJCQlc CisJdW5zaWduZWQgbG9uZyBtaW5fc2VxW0FOT05fQU5EX0ZJTEVdID0gewkJCVwKKwkJUkVBRF9P TkNFKChscnV2ZWMpLT5scnVnZW4ubWluX3NlcVtMUlVfR0VOX0FOT05dKSwJXAorCQlSRUFEX09O Q0UoKGxydXZlYyktPmxydWdlbi5taW5fc2VxW0xSVV9HRU5fRklMRV0pLAlcCisJfQorCiAjZGVm aW5lIGZvcl9lYWNoX2dlbl90eXBlX3pvbmUoZ2VuLCB0eXBlLCB6b25lKQkJCQlcCiAJZm9yICgo Z2VuKSA9IDA7IChnZW4pIDwgTUFYX05SX0dFTlM7IChnZW4pKyspCQkJXAogCQlmb3IgKCh0eXBl KSA9IDA7ICh0eXBlKSA8IEFOT05fQU5EX0ZJTEU7ICh0eXBlKSsrKQlcCkBAIC0zMDgxLDYgKzMw OTcsNzY5IEBAIHN0YXRpYyBzdHJ1Y3QgbHJ1dmVjIF9fbWF5YmVfdW51c2VkICpnZXRfbHJ1dmVj KHN0cnVjdCBtZW1fY2dyb3VwICptZW1jZywgaW50IG5pCiAJcmV0dXJuIHBnZGF0ID8gJnBnZGF0 LT5fX2xydXZlYyA6IE5VTEw7CiB9CiAKK3N0YXRpYyBpbnQgZ2V0X3N3YXBwaW5lc3Moc3RydWN0 IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3Qgc2Nhbl9jb250cm9sICpzYykKK3sKKwlzdHJ1Y3QgbWVt X2Nncm91cCAqbWVtY2cgPSBscnV2ZWNfbWVtY2cobHJ1dmVjKTsKKwlzdHJ1Y3QgcGdsaXN0X2Rh dGEgKnBnZGF0ID0gbHJ1dmVjX3BnZGF0KGxydXZlYyk7CisKKwlpZiAoIWNhbl9kZW1vdGUocGdk YXQtPm5vZGVfaWQsIHNjKSAmJgorCSAgICBtZW1fY2dyb3VwX2dldF9ucl9zd2FwX3BhZ2VzKG1l bWNnKSA8IE1JTl9MUlVfQkFUQ0gpCisJCXJldHVybiAwOworCisJcmV0dXJuIG1lbV9jZ3JvdXBf c3dhcHBpbmVzcyhtZW1jZyk7Cit9CisKK3N0YXRpYyBpbnQgZ2V0X25yX2dlbnMoc3RydWN0IGxy dXZlYyAqbHJ1dmVjLCBpbnQgdHlwZSkKK3sKKwlyZXR1cm4gbHJ1dmVjLT5scnVnZW4ubWF4X3Nl cSAtIGxydXZlYy0+bHJ1Z2VuLm1pbl9zZXFbdHlwZV0gKyAxOworfQorCitzdGF0aWMgYm9vbCBf X21heWJlX3VudXNlZCBzZXFfaXNfdmFsaWQoc3RydWN0IGxydXZlYyAqbHJ1dmVjKQoreworCS8q IHNlZSB0aGUgY29tbWVudCBvbiBscnVfZ2VuX3N0cnVjdCAqLworCXJldHVybiBnZXRfbnJfZ2Vu cyhscnV2ZWMsIExSVV9HRU5fRklMRSkgPj0gTUlOX05SX0dFTlMgJiYKKwkgICAgICAgZ2V0X25y X2dlbnMobHJ1dmVjLCBMUlVfR0VOX0ZJTEUpIDw9IGdldF9ucl9nZW5zKGxydXZlYywgTFJVX0dF Tl9BTk9OKSAmJgorCSAgICAgICBnZXRfbnJfZ2VucyhscnV2ZWMsIExSVV9HRU5fQU5PTikgPD0g TUFYX05SX0dFTlM7Cit9CisKKy8qKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioKKyAqICAgICAgICAgICAg ICAgICAgICAgICAgICByZWZhdWx0IGZlZWRiYWNrIGxvb3AKKyAqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KiovCisKKy8qCisgKiBBIGZlZWRiYWNrIGxvb3AgYmFzZWQgb24gUHJvcG9ydGlvbmFsLUludGVn cmFsLURlcml2YXRpdmUgKFBJRCkgY29udHJvbGxlci4KKyAqCisgKiBUaGUgUCB0ZXJtIGlzIHJl ZmF1bHRlZC8oZXZpY3RlZCtwcm90ZWN0ZWQpIGZyb20gYSB0aWVyIGluIHRoZSBnZW5lcmF0aW9u CisgKiBjdXJyZW50bHkgYmVpbmcgZXZpY3RlZDsgdGhlIEkgdGVybSBpcyB0aGUgZXhwb25lbnRp YWwgbW92aW5nIGF2ZXJhZ2Ugb2YgdGhlCisgKiBQIHRlcm0gb3ZlciB0aGUgZ2VuZXJhdGlvbnMg cHJldmlvdXNseSBldmljdGVkLCB1c2luZyB0aGUgc21vb3RoaW5nIGZhY3RvcgorICogMS8yOyB0 aGUgRCB0ZXJtIGlzbid0IHN1cHBvcnRlZC4KKyAqCisgKiBUaGUgc2V0cG9pbnQgKFNQKSBpcyBh bHdheXMgdGhlIGZpcnN0IHRpZXIgb2Ygb25lIHR5cGU7IHRoZSBwcm9jZXNzIHZhcmlhYmxlCisg KiAoUFYpIGlzIGVpdGhlciBhbnkgdGllciBvZiB0aGUgb3RoZXIgdHlwZSBvciBhbnkgb3RoZXIg dGllciBvZiB0aGUgc2FtZQorICogdHlwZS4KKyAqCisgKiBUaGUgZXJyb3IgaXMgdGhlIGRpZmZl cmVuY2UgYmV0d2VlbiB0aGUgU1AgYW5kIHRoZSBQVjsgdGhlIGNvcnJlY3Rpb24gaXMgdG8KKyAq IHR1cm4gb2ZmIHByb3RlY3Rpb24gd2hlbiBTUD5QViBvciB0dXJuIG9uIHByb3RlY3Rpb24gd2hl biBTUDxQVi4KKyAqCisgKiBGb3IgZnV0dXJlIG9wdGltaXphdGlvbnM6CisgKiAxLiBUaGUgRCB0 ZXJtIG1heSBkaXNjb3VudCB0aGUgb3RoZXIgdHdvIHRlcm1zIG92ZXIgdGltZSBzbyB0aGF0IGxv bmctbGl2ZWQKKyAqICAgIGdlbmVyYXRpb25zIGNhbiByZXNpc3Qgc3RhbGUgaW5mb3JtYXRpb24u CisgKi8KK3N0cnVjdCBjdHJsX3BvcyB7CisJdW5zaWduZWQgbG9uZyByZWZhdWx0ZWQ7CisJdW5z aWduZWQgbG9uZyB0b3RhbDsKKwlpbnQgZ2FpbjsKK307CisKK3N0YXRpYyB2b2lkIHJlYWRfY3Ry bF9wb3Moc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBpbnQgdHlwZSwgaW50IHRpZXIsIGludCBnYWlu LAorCQkJICBzdHJ1Y3QgY3RybF9wb3MgKnBvcykKK3sKKwlzdHJ1Y3QgbHJ1X2dlbl9zdHJ1Y3Qg KmxydWdlbiA9ICZscnV2ZWMtPmxydWdlbjsKKwlpbnQgaGlzdCA9IGxydV9oaXN0X2Zyb21fc2Vx KGxydWdlbi0+bWluX3NlcVt0eXBlXSk7CisKKwlwb3MtPnJlZmF1bHRlZCA9IGxydWdlbi0+YXZn X3JlZmF1bHRlZFt0eXBlXVt0aWVyXSArCisJCQkgYXRvbWljX2xvbmdfcmVhZCgmbHJ1Z2VuLT5y ZWZhdWx0ZWRbaGlzdF1bdHlwZV1bdGllcl0pOworCXBvcy0+dG90YWwgPSBscnVnZW4tPmF2Z190 b3RhbFt0eXBlXVt0aWVyXSArCisJCSAgICAgYXRvbWljX2xvbmdfcmVhZCgmbHJ1Z2VuLT5ldmlj dGVkW2hpc3RdW3R5cGVdW3RpZXJdKTsKKwlpZiAodGllcikKKwkJcG9zLT50b3RhbCArPSBscnVn ZW4tPnByb3RlY3RlZFtoaXN0XVt0eXBlXVt0aWVyIC0gMV07CisJcG9zLT5nYWluID0gZ2FpbjsK K30KKworc3RhdGljIHZvaWQgcmVzZXRfY3RybF9wb3Moc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBp bnQgdHlwZSwgYm9vbCBjYXJyeW92ZXIpCit7CisJaW50IGhpc3QsIHRpZXI7CisJc3RydWN0IGxy dV9nZW5fc3RydWN0ICpscnVnZW4gPSAmbHJ1dmVjLT5scnVnZW47CisJYm9vbCBjbGVhciA9IGNh cnJ5b3ZlciA/IE5SX0hJU1RfR0VOUyA9PSAxIDogTlJfSElTVF9HRU5TID4gMTsKKwl1bnNpZ25l ZCBsb25nIHNlcSA9IGNhcnJ5b3ZlciA/IGxydWdlbi0+bWluX3NlcVt0eXBlXSA6IGxydWdlbi0+ bWF4X3NlcSArIDE7CisKKwlsb2NrZGVwX2Fzc2VydF9oZWxkKCZscnV2ZWMtPmxydV9sb2NrKTsK KworCWlmICghY2FycnlvdmVyICYmICFjbGVhcikKKwkJcmV0dXJuOworCisJaGlzdCA9IGxydV9o aXN0X2Zyb21fc2VxKHNlcSk7CisKKwlmb3IgKHRpZXIgPSAwOyB0aWVyIDwgTUFYX05SX1RJRVJT OyB0aWVyKyspIHsKKwkJaWYgKGNhcnJ5b3ZlcikgeworCQkJdW5zaWduZWQgbG9uZyBzdW07CisK KwkJCXN1bSA9IGxydWdlbi0+YXZnX3JlZmF1bHRlZFt0eXBlXVt0aWVyXSArCisJCQkgICAgICBh dG9taWNfbG9uZ19yZWFkKCZscnVnZW4tPnJlZmF1bHRlZFtoaXN0XVt0eXBlXVt0aWVyXSk7CisJ CQlXUklURV9PTkNFKGxydWdlbi0+YXZnX3JlZmF1bHRlZFt0eXBlXVt0aWVyXSwgc3VtIC8gMik7 CisKKwkJCXN1bSA9IGxydWdlbi0+YXZnX3RvdGFsW3R5cGVdW3RpZXJdICsKKwkJCSAgICAgIGF0 b21pY19sb25nX3JlYWQoJmxydWdlbi0+ZXZpY3RlZFtoaXN0XVt0eXBlXVt0aWVyXSk7CisJCQlp ZiAodGllcikKKwkJCQlzdW0gKz0gbHJ1Z2VuLT5wcm90ZWN0ZWRbaGlzdF1bdHlwZV1bdGllciAt IDFdOworCQkJV1JJVEVfT05DRShscnVnZW4tPmF2Z190b3RhbFt0eXBlXVt0aWVyXSwgc3VtIC8g Mik7CisJCX0KKworCQlpZiAoY2xlYXIpIHsKKwkJCWF0b21pY19sb25nX3NldCgmbHJ1Z2VuLT5y ZWZhdWx0ZWRbaGlzdF1bdHlwZV1bdGllcl0sIDApOworCQkJYXRvbWljX2xvbmdfc2V0KCZscnVn ZW4tPmV2aWN0ZWRbaGlzdF1bdHlwZV1bdGllcl0sIDApOworCQkJaWYgKHRpZXIpCisJCQkJV1JJ VEVfT05DRShscnVnZW4tPnByb3RlY3RlZFtoaXN0XVt0eXBlXVt0aWVyIC0gMV0sIDApOworCQl9 CisJfQorfQorCitzdGF0aWMgYm9vbCBwb3NpdGl2ZV9jdHJsX2VycihzdHJ1Y3QgY3RybF9wb3Mg KnNwLCBzdHJ1Y3QgY3RybF9wb3MgKnB2KQoreworCS8qCisJICogUmV0dXJuIHRydWUgaWYgdGhl IFBWIGhhcyBhIGxpbWl0ZWQgbnVtYmVyIG9mIHJlZmF1bHRzIG9yIGEgbG93ZXIKKwkgKiByZWZh dWx0ZWQvdG90YWwgdGhhbiB0aGUgU1AuCisJICovCisJcmV0dXJuIHB2LT5yZWZhdWx0ZWQgPCBN SU5fTFJVX0JBVENIIHx8CisJICAgICAgIHB2LT5yZWZhdWx0ZWQgKiAoc3AtPnRvdGFsICsgTUlO X0xSVV9CQVRDSCkgKiBzcC0+Z2FpbiA8PQorCSAgICAgICAoc3AtPnJlZmF1bHRlZCArIDEpICog cHYtPnRvdGFsICogcHYtPmdhaW47Cit9CisKKy8qKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioKKyAqICAg ICAgICAgICAgICAgICAgICAgICAgICB0aGUgYWdpbmcKKyAqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKiov CisKKy8qIHByb3RlY3QgcGFnZXMgYWNjZXNzZWQgbXVsdGlwbGUgdGltZXMgdGhyb3VnaCBmaWxl IGRlc2NyaXB0b3JzICovCitzdGF0aWMgaW50IGZvbGlvX2luY19nZW4oc3RydWN0IGxydXZlYyAq bHJ1dmVjLCBzdHJ1Y3QgZm9saW8gKmZvbGlvLCBib29sIHJlY2xhaW1pbmcpCit7CisJaW50IHR5 cGUgPSBmb2xpb19pc19maWxlX2xydShmb2xpbyk7CisJc3RydWN0IGxydV9nZW5fc3RydWN0ICps cnVnZW4gPSAmbHJ1dmVjLT5scnVnZW47CisJaW50IG5ld19nZW4sIG9sZF9nZW4gPSBscnVfZ2Vu X2Zyb21fc2VxKGxydWdlbi0+bWluX3NlcVt0eXBlXSk7CisJdW5zaWduZWQgbG9uZyBuZXdfZmxh Z3MsIG9sZF9mbGFncyA9IFJFQURfT05DRShmb2xpby0+ZmxhZ3MpOworCisJVk1fV0FSTl9PTl9P TkNFX0ZPTElPKCEob2xkX2ZsYWdzICYgTFJVX0dFTl9NQVNLKSwgZm9saW8pOworCisJZG8gewor CQluZXdfZ2VuID0gKG9sZF9nZW4gKyAxKSAlIE1BWF9OUl9HRU5TOworCisJCW5ld19mbGFncyA9 IG9sZF9mbGFncyAmIH4oTFJVX0dFTl9NQVNLIHwgTFJVX1JFRlNfTUFTSyB8IExSVV9SRUZTX0ZM QUdTKTsKKwkJbmV3X2ZsYWdzIHw9IChuZXdfZ2VuICsgMVVMKSA8PCBMUlVfR0VOX1BHT0ZGOwor CQkvKiBmb3IgZm9saW9fZW5kX3dyaXRlYmFjaygpICovCisJCWlmIChyZWNsYWltaW5nKQorCQkJ bmV3X2ZsYWdzIHw9IEJJVChQR19yZWNsYWltKTsKKwl9IHdoaWxlICghdHJ5X2NtcHhjaGcoJmZv bGlvLT5mbGFncywgJm9sZF9mbGFncywgbmV3X2ZsYWdzKSk7CisKKwlscnVfZ2VuX3VwZGF0ZV9z aXplKGxydXZlYywgZm9saW8sIG9sZF9nZW4sIG5ld19nZW4pOworCisJcmV0dXJuIG5ld19nZW47 Cit9CisKK3N0YXRpYyB2b2lkIGluY19taW5fc2VxKHN0cnVjdCBscnV2ZWMgKmxydXZlYywgaW50 IHR5cGUpCit7CisJc3RydWN0IGxydV9nZW5fc3RydWN0ICpscnVnZW4gPSAmbHJ1dmVjLT5scnVn ZW47CisKKwlyZXNldF9jdHJsX3BvcyhscnV2ZWMsIHR5cGUsIHRydWUpOworCVdSSVRFX09OQ0Uo bHJ1Z2VuLT5taW5fc2VxW3R5cGVdLCBscnVnZW4tPm1pbl9zZXFbdHlwZV0gKyAxKTsKK30KKwor c3RhdGljIGJvb2wgdHJ5X3RvX2luY19taW5fc2VxKHN0cnVjdCBscnV2ZWMgKmxydXZlYywgYm9v bCBjYW5fc3dhcCkKK3sKKwlpbnQgZ2VuLCB0eXBlLCB6b25lOworCWJvb2wgc3VjY2VzcyA9IGZh bHNlOworCXN0cnVjdCBscnVfZ2VuX3N0cnVjdCAqbHJ1Z2VuID0gJmxydXZlYy0+bHJ1Z2VuOwor CURFRklORV9NSU5fU0VRKGxydXZlYyk7CisKKwlWTV9XQVJOX09OX09OQ0UoIXNlcV9pc192YWxp ZChscnV2ZWMpKTsKKworCS8qIGZpbmQgdGhlIG9sZGVzdCBwb3B1bGF0ZWQgZ2VuZXJhdGlvbiAq LworCWZvciAodHlwZSA9ICFjYW5fc3dhcDsgdHlwZSA8IEFOT05fQU5EX0ZJTEU7IHR5cGUrKykg eworCQl3aGlsZSAobWluX3NlcVt0eXBlXSArIE1JTl9OUl9HRU5TIDw9IGxydWdlbi0+bWF4X3Nl cSkgeworCQkJZ2VuID0gbHJ1X2dlbl9mcm9tX3NlcShtaW5fc2VxW3R5cGVdKTsKKworCQkJZm9y ICh6b25lID0gMDsgem9uZSA8IE1BWF9OUl9aT05FUzsgem9uZSsrKSB7CisJCQkJaWYgKCFsaXN0 X2VtcHR5KCZscnVnZW4tPmxpc3RzW2dlbl1bdHlwZV1bem9uZV0pKQorCQkJCQlnb3RvIG5leHQ7 CisJCQl9CisKKwkJCW1pbl9zZXFbdHlwZV0rKzsKKwkJfQorbmV4dDoKKwkJOworCX0KKworCS8q IHNlZSB0aGUgY29tbWVudCBvbiBscnVfZ2VuX3N0cnVjdCAqLworCWlmIChjYW5fc3dhcCkgewor CQltaW5fc2VxW0xSVV9HRU5fQU5PTl0gPSBtaW4obWluX3NlcVtMUlVfR0VOX0FOT05dLCBtaW5f c2VxW0xSVV9HRU5fRklMRV0pOworCQltaW5fc2VxW0xSVV9HRU5fRklMRV0gPSBtYXgobWluX3Nl cVtMUlVfR0VOX0FOT05dLCBscnVnZW4tPm1pbl9zZXFbTFJVX0dFTl9GSUxFXSk7CisJfQorCisJ Zm9yICh0eXBlID0gIWNhbl9zd2FwOyB0eXBlIDwgQU5PTl9BTkRfRklMRTsgdHlwZSsrKSB7CisJ CWlmIChtaW5fc2VxW3R5cGVdID09IGxydWdlbi0+bWluX3NlcVt0eXBlXSkKKwkJCWNvbnRpbnVl OworCisJCXJlc2V0X2N0cmxfcG9zKGxydXZlYywgdHlwZSwgdHJ1ZSk7CisJCVdSSVRFX09OQ0Uo bHJ1Z2VuLT5taW5fc2VxW3R5cGVdLCBtaW5fc2VxW3R5cGVdKTsKKwkJc3VjY2VzcyA9IHRydWU7 CisJfQorCisJcmV0dXJuIHN1Y2Nlc3M7Cit9CisKK3N0YXRpYyB2b2lkIGluY19tYXhfc2VxKHN0 cnVjdCBscnV2ZWMgKmxydXZlYywgdW5zaWduZWQgbG9uZyBtYXhfc2VxLCBib29sIGNhbl9zd2Fw KQoreworCWludCBwcmV2LCBuZXh0OworCWludCB0eXBlLCB6b25lOworCXN0cnVjdCBscnVfZ2Vu X3N0cnVjdCAqbHJ1Z2VuID0gJmxydXZlYy0+bHJ1Z2VuOworCisJc3Bpbl9sb2NrX2lycSgmbHJ1 dmVjLT5scnVfbG9jayk7CisKKwlWTV9XQVJOX09OX09OQ0UoIXNlcV9pc192YWxpZChscnV2ZWMp KTsKKworCWlmIChtYXhfc2VxICE9IGxydWdlbi0+bWF4X3NlcSkKKwkJZ290byB1bmxvY2s7CisK Kwlmb3IgKHR5cGUgPSAwOyB0eXBlIDwgQU5PTl9BTkRfRklMRTsgdHlwZSsrKSB7CisJCWlmIChn ZXRfbnJfZ2VucyhscnV2ZWMsIHR5cGUpICE9IE1BWF9OUl9HRU5TKQorCQkJY29udGludWU7CisK KwkJVk1fV0FSTl9PTl9PTkNFKHR5cGUgPT0gTFJVX0dFTl9GSUxFIHx8IGNhbl9zd2FwKTsKKwor CQlpbmNfbWluX3NlcShscnV2ZWMsIHR5cGUpOworCX0KKworCS8qCisJICogVXBkYXRlIHRoZSBh Y3RpdmUvaW5hY3RpdmUgTFJVIHNpemVzIGZvciBjb21wYXRpYmlsaXR5LiBCb3RoIHNpZGVzIG9m CisJICogdGhlIGN1cnJlbnQgbWF4X3NlcSBuZWVkIHRvIGJlIGNvdmVyZWQsIHNpbmNlIG1heF9z ZXErMSBjYW4gb3ZlcmxhcAorCSAqIHdpdGggbWluX3NlcVtMUlVfR0VOX0FOT05dIGlmIHN3YXBw aW5nIGlzIGNvbnN0cmFpbmVkLiBBbmQgaWYgdGhleSBkbworCSAqIG92ZXJsYXAsIGNvbGQvaG90 IGludmVyc2lvbiBoYXBwZW5zLgorCSAqLworCXByZXYgPSBscnVfZ2VuX2Zyb21fc2VxKGxydWdl bi0+bWF4X3NlcSAtIDEpOworCW5leHQgPSBscnVfZ2VuX2Zyb21fc2VxKGxydWdlbi0+bWF4X3Nl cSArIDEpOworCisJZm9yICh0eXBlID0gMDsgdHlwZSA8IEFOT05fQU5EX0ZJTEU7IHR5cGUrKykg eworCQlmb3IgKHpvbmUgPSAwOyB6b25lIDwgTUFYX05SX1pPTkVTOyB6b25lKyspIHsKKwkJCWVu dW0gbHJ1X2xpc3QgbHJ1ID0gdHlwZSAqIExSVV9JTkFDVElWRV9GSUxFOworCQkJbG9uZyBkZWx0 YSA9IGxydWdlbi0+bnJfcGFnZXNbcHJldl1bdHlwZV1bem9uZV0gLQorCQkJCSAgICAgbHJ1Z2Vu LT5ucl9wYWdlc1tuZXh0XVt0eXBlXVt6b25lXTsKKworCQkJaWYgKCFkZWx0YSkKKwkJCQljb250 aW51ZTsKKworCQkJX191cGRhdGVfbHJ1X3NpemUobHJ1dmVjLCBscnUsIHpvbmUsIGRlbHRhKTsK KwkJCV9fdXBkYXRlX2xydV9zaXplKGxydXZlYywgbHJ1ICsgTFJVX0FDVElWRSwgem9uZSwgLWRl bHRhKTsKKwkJfQorCX0KKworCWZvciAodHlwZSA9IDA7IHR5cGUgPCBBTk9OX0FORF9GSUxFOyB0 eXBlKyspCisJCXJlc2V0X2N0cmxfcG9zKGxydXZlYywgdHlwZSwgZmFsc2UpOworCisJLyogbWFr ZSBzdXJlIHByZWNlZGluZyBtb2RpZmljYXRpb25zIGFwcGVhciAqLworCXNtcF9zdG9yZV9yZWxl YXNlKCZscnVnZW4tPm1heF9zZXEsIGxydWdlbi0+bWF4X3NlcSArIDEpOwordW5sb2NrOgorCXNw aW5fdW5sb2NrX2lycSgmbHJ1dmVjLT5scnVfbG9jayk7Cit9CisKK3N0YXRpYyB1bnNpZ25lZCBs b25nIGdldF9ucl9ldmljdGFibGUoc3RydWN0IGxydXZlYyAqbHJ1dmVjLCB1bnNpZ25lZCBsb25n IG1heF9zZXEsCisJCQkJICAgICAgdW5zaWduZWQgbG9uZyAqbWluX3NlcSwgYm9vbCBjYW5fc3dh cCwgYm9vbCAqbmVlZF9hZ2luZykKK3sKKwlpbnQgZ2VuLCB0eXBlLCB6b25lOworCXVuc2lnbmVk IGxvbmcgb2xkID0gMDsKKwl1bnNpZ25lZCBsb25nIHlvdW5nID0gMDsKKwl1bnNpZ25lZCBsb25n IHRvdGFsID0gMDsKKwlzdHJ1Y3QgbHJ1X2dlbl9zdHJ1Y3QgKmxydWdlbiA9ICZscnV2ZWMtPmxy dWdlbjsKKworCWZvciAodHlwZSA9ICFjYW5fc3dhcDsgdHlwZSA8IEFOT05fQU5EX0ZJTEU7IHR5 cGUrKykgeworCQl1bnNpZ25lZCBsb25nIHNlcTsKKworCQlmb3IgKHNlcSA9IG1pbl9zZXFbdHlw ZV07IHNlcSA8PSBtYXhfc2VxOyBzZXErKykgeworCQkJdW5zaWduZWQgbG9uZyBzaXplID0gMDsK KworCQkJZ2VuID0gbHJ1X2dlbl9mcm9tX3NlcShzZXEpOworCisJCQlmb3IgKHpvbmUgPSAwOyB6 b25lIDwgTUFYX05SX1pPTkVTOyB6b25lKyspCisJCQkJc2l6ZSArPSBtYXgoUkVBRF9PTkNFKGxy dWdlbi0+bnJfcGFnZXNbZ2VuXVt0eXBlXVt6b25lXSksIDBMKTsKKworCQkJdG90YWwgKz0gc2l6 ZTsKKwkJCWlmIChzZXEgPT0gbWF4X3NlcSkKKwkJCQl5b3VuZyArPSBzaXplOworCQkJaWYgKHNl cSArIE1JTl9OUl9HRU5TID09IG1heF9zZXEpCisJCQkJb2xkICs9IHNpemU7CisJCX0KKwl9CisK KwkvKgorCSAqIFRoZSBhZ2luZyB0cmllcyB0byBiZSBsYXp5IHRvIHJlZHVjZSB0aGUgb3Zlcmhl YWQuIE9uIHRoZSBvdGhlciBoYW5kLAorCSAqIHRoZSBldmljdGlvbiBzdGFsbHMgd2hlbiB0aGUg bnVtYmVyIG9mIGdlbmVyYXRpb25zIHJlYWNoZXMKKwkgKiBNSU5fTlJfR0VOUy4gU28gaWRlYWxs eSwgdGhlcmUgc2hvdWxkIGJlIE1JTl9OUl9HRU5TKzEgZ2VuZXJhdGlvbnMsCisJICogaGVuY2Ug dGhlIGZpcnN0IHR3byBpZidzLgorCSAqCisJICogQWxzbyBpdCdzIGlkZWFsIHRvIHNwcmVhZCBw YWdlcyBvdXQgZXZlbmx5LCBtZWFuaW5nIDEvKE1JTl9OUl9HRU5TKzEpCisJICogb2YgdGhlIHRv dGFsIG51bWJlciBvZiBwYWdlcyBmb3IgZWFjaCBnZW5lcmF0aW9uLiBBIHJlYXNvbmFibGUgcmFu Z2UKKwkgKiBmb3IgdGhpcyBhdmVyYWdlIHBvcnRpb24gaXMgWzEvTUlOX05SX0dFTlMsIDEvKE1J Tl9OUl9HRU5TKzIpXS4gVGhlCisJICogZXZpY3Rpb24gY2FyZXMgYWJvdXQgdGhlIGxvd2VyIGJv dW5kIG9mIGNvbGQgcGFnZXMsIHdoZXJlYXMgdGhlIGFnaW5nCisJICogY2FyZXMgYWJvdXQgdGhl IHVwcGVyIGJvdW5kIG9mIGhvdCBwYWdlcy4KKwkgKi8KKwlpZiAobWluX3NlcVshY2FuX3N3YXBd ICsgTUlOX05SX0dFTlMgPiBtYXhfc2VxKQorCQkqbmVlZF9hZ2luZyA9IHRydWU7CisJZWxzZSBp ZiAobWluX3NlcVshY2FuX3N3YXBdICsgTUlOX05SX0dFTlMgPCBtYXhfc2VxKQorCQkqbmVlZF9h Z2luZyA9IGZhbHNlOworCWVsc2UgaWYgKHlvdW5nICogTUlOX05SX0dFTlMgPiB0b3RhbCkKKwkJ Km5lZWRfYWdpbmcgPSB0cnVlOworCWVsc2UgaWYgKG9sZCAqIChNSU5fTlJfR0VOUyArIDIpIDwg dG90YWwpCisJCSpuZWVkX2FnaW5nID0gdHJ1ZTsKKwllbHNlCisJCSpuZWVkX2FnaW5nID0gZmFs c2U7CisKKwlyZXR1cm4gdG90YWw7Cit9CisKK3N0YXRpYyB2b2lkIGFnZV9scnV2ZWMoc3RydWN0 IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3Qgc2Nhbl9jb250cm9sICpzYykKK3sKKwlib29sIG5lZWRf YWdpbmc7CisJdW5zaWduZWQgbG9uZyBucl90b19zY2FuOworCWludCBzd2FwcGluZXNzID0gZ2V0 X3N3YXBwaW5lc3MobHJ1dmVjLCBzYyk7CisJc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnID0gbHJ1 dmVjX21lbWNnKGxydXZlYyk7CisJREVGSU5FX01BWF9TRVEobHJ1dmVjKTsKKwlERUZJTkVfTUlO X1NFUShscnV2ZWMpOworCisJVk1fV0FSTl9PTl9PTkNFKHNjLT5tZW1jZ19sb3dfcmVjbGFpbSk7 CisKKwltZW1fY2dyb3VwX2NhbGN1bGF0ZV9wcm90ZWN0aW9uKE5VTEwsIG1lbWNnKTsKKworCWlm IChtZW1fY2dyb3VwX2JlbG93X21pbihtZW1jZykpCisJCXJldHVybjsKKworCW5yX3RvX3NjYW4g PSBnZXRfbnJfZXZpY3RhYmxlKGxydXZlYywgbWF4X3NlcSwgbWluX3NlcSwgc3dhcHBpbmVzcywg Jm5lZWRfYWdpbmcpOworCWlmICghbnJfdG9fc2NhbikKKwkJcmV0dXJuOworCisJbnJfdG9fc2Nh biA+Pj0gbWVtX2Nncm91cF9vbmxpbmUobWVtY2cpID8gc2MtPnByaW9yaXR5IDogMDsKKworCWlm IChucl90b19zY2FuICYmIG5lZWRfYWdpbmcpCisJCWluY19tYXhfc2VxKGxydXZlYywgbWF4X3Nl cSwgc3dhcHBpbmVzcyk7Cit9CisKK3N0YXRpYyB2b2lkIGxydV9nZW5fYWdlX25vZGUoc3RydWN0 IHBnbGlzdF9kYXRhICpwZ2RhdCwgc3RydWN0IHNjYW5fY29udHJvbCAqc2MpCit7CisJc3RydWN0 IG1lbV9jZ3JvdXAgKm1lbWNnOworCisJVk1fV0FSTl9PTl9PTkNFKCFjdXJyZW50X2lzX2tzd2Fw ZCgpKTsKKworCW1lbWNnID0gbWVtX2Nncm91cF9pdGVyKE5VTEwsIE5VTEwsIE5VTEwpOworCWRv IHsKKwkJc3RydWN0IGxydXZlYyAqbHJ1dmVjID0gbWVtX2Nncm91cF9scnV2ZWMobWVtY2csIHBn ZGF0KTsKKworCQlhZ2VfbHJ1dmVjKGxydXZlYywgc2MpOworCisJCWNvbmRfcmVzY2hlZCgpOwor CX0gd2hpbGUgKChtZW1jZyA9IG1lbV9jZ3JvdXBfaXRlcihOVUxMLCBtZW1jZywgTlVMTCkpKTsK K30KKworLyoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKgorICogICAgICAgICAgICAgICAgICAgICAgICAg IHRoZSBldmljdGlvbgorICoqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKi8KKworc3RhdGljIGJvb2wgc29y dF9mb2xpbyhzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIHN0cnVjdCBmb2xpbyAqZm9saW8sIGludCB0 aWVyX2lkeCkKK3sKKwlib29sIHN1Y2Nlc3M7CisJaW50IGdlbiA9IGZvbGlvX2xydV9nZW4oZm9s aW8pOworCWludCB0eXBlID0gZm9saW9faXNfZmlsZV9scnUoZm9saW8pOworCWludCB6b25lID0g Zm9saW9fem9uZW51bShmb2xpbyk7CisJaW50IGRlbHRhID0gZm9saW9fbnJfcGFnZXMoZm9saW8p OworCWludCByZWZzID0gZm9saW9fbHJ1X3JlZnMoZm9saW8pOworCWludCB0aWVyID0gbHJ1X3Rp ZXJfZnJvbV9yZWZzKHJlZnMpOworCXN0cnVjdCBscnVfZ2VuX3N0cnVjdCAqbHJ1Z2VuID0gJmxy dXZlYy0+bHJ1Z2VuOworCisJVk1fV0FSTl9PTl9PTkNFX0ZPTElPKGdlbiA+PSBNQVhfTlJfR0VO UywgZm9saW8pOworCisJLyogdW5ldmljdGFibGUgKi8KKwlpZiAoIWZvbGlvX2V2aWN0YWJsZShm b2xpbykpIHsKKwkJc3VjY2VzcyA9IGxydV9nZW5fZGVsX2ZvbGlvKGxydXZlYywgZm9saW8sIHRy dWUpOworCQlWTV9XQVJOX09OX09OQ0VfRk9MSU8oIXN1Y2Nlc3MsIGZvbGlvKTsKKwkJZm9saW9f c2V0X3VuZXZpY3RhYmxlKGZvbGlvKTsKKwkJbHJ1dmVjX2FkZF9mb2xpbyhscnV2ZWMsIGZvbGlv KTsKKwkJX19jb3VudF92bV9ldmVudHMoVU5FVklDVEFCTEVfUEdDVUxMRUQsIGRlbHRhKTsKKwkJ cmV0dXJuIHRydWU7CisJfQorCisJLyogZGlydHkgbGF6eWZyZWUgKi8KKwlpZiAodHlwZSA9PSBM UlVfR0VOX0ZJTEUgJiYgZm9saW9fdGVzdF9hbm9uKGZvbGlvKSAmJiBmb2xpb190ZXN0X2RpcnR5 KGZvbGlvKSkgeworCQlzdWNjZXNzID0gbHJ1X2dlbl9kZWxfZm9saW8obHJ1dmVjLCBmb2xpbywg dHJ1ZSk7CisJCVZNX1dBUk5fT05fT05DRV9GT0xJTyghc3VjY2VzcywgZm9saW8pOworCQlmb2xp b19zZXRfc3dhcGJhY2tlZChmb2xpbyk7CisJCWxydXZlY19hZGRfZm9saW9fdGFpbChscnV2ZWMs IGZvbGlvKTsKKwkJcmV0dXJuIHRydWU7CisJfQorCisJLyogcHJvdGVjdGVkICovCisJaWYgKHRp ZXIgPiB0aWVyX2lkeCkgeworCQlpbnQgaGlzdCA9IGxydV9oaXN0X2Zyb21fc2VxKGxydWdlbi0+ bWluX3NlcVt0eXBlXSk7CisKKwkJZ2VuID0gZm9saW9faW5jX2dlbihscnV2ZWMsIGZvbGlvLCBm YWxzZSk7CisJCWxpc3RfbW92ZV90YWlsKCZmb2xpby0+bHJ1LCAmbHJ1Z2VuLT5saXN0c1tnZW5d W3R5cGVdW3pvbmVdKTsKKworCQlXUklURV9PTkNFKGxydWdlbi0+cHJvdGVjdGVkW2hpc3RdW3R5 cGVdW3RpZXIgLSAxXSwKKwkJCSAgIGxydWdlbi0+cHJvdGVjdGVkW2hpc3RdW3R5cGVdW3RpZXIg LSAxXSArIGRlbHRhKTsKKwkJX19tb2RfbHJ1dmVjX3N0YXRlKGxydXZlYywgV09SS0lOR1NFVF9B Q1RJVkFURV9CQVNFICsgdHlwZSwgZGVsdGEpOworCQlyZXR1cm4gdHJ1ZTsKKwl9CisKKwkvKiB3 YWl0aW5nIGZvciB3cml0ZWJhY2sgKi8KKwlpZiAoZm9saW9fdGVzdF9sb2NrZWQoZm9saW8pIHx8 IGZvbGlvX3Rlc3Rfd3JpdGViYWNrKGZvbGlvKSB8fAorCSAgICAodHlwZSA9PSBMUlVfR0VOX0ZJ TEUgJiYgZm9saW9fdGVzdF9kaXJ0eShmb2xpbykpKSB7CisJCWdlbiA9IGZvbGlvX2luY19nZW4o bHJ1dmVjLCBmb2xpbywgdHJ1ZSk7CisJCWxpc3RfbW92ZSgmZm9saW8tPmxydSwgJmxydWdlbi0+ bGlzdHNbZ2VuXVt0eXBlXVt6b25lXSk7CisJCXJldHVybiB0cnVlOworCX0KKworCXJldHVybiBm YWxzZTsKK30KKworc3RhdGljIGJvb2wgaXNvbGF0ZV9mb2xpbyhzdHJ1Y3QgbHJ1dmVjICpscnV2 ZWMsIHN0cnVjdCBmb2xpbyAqZm9saW8sIHN0cnVjdCBzY2FuX2NvbnRyb2wgKnNjKQoreworCWJv b2wgc3VjY2VzczsKKworCS8qIHVubWFwcGluZyBpbmhpYml0ZWQgKi8KKwlpZiAoIXNjLT5tYXlf dW5tYXAgJiYgZm9saW9fbWFwcGVkKGZvbGlvKSkKKwkJcmV0dXJuIGZhbHNlOworCisJLyogc3dh cHBpbmcgaW5oaWJpdGVkICovCisJaWYgKCEoc2MtPm1heV93cml0ZXBhZ2UgJiYgKHNjLT5nZnBf bWFzayAmIF9fR0ZQX0lPKSkgJiYKKwkgICAgKGZvbGlvX3Rlc3RfZGlydHkoZm9saW8pIHx8CisJ ICAgICAoZm9saW9fdGVzdF9hbm9uKGZvbGlvKSAmJiAhZm9saW9fdGVzdF9zd2FwY2FjaGUoZm9s aW8pKSkpCisJCXJldHVybiBmYWxzZTsKKworCS8qIHJhY2VkIHdpdGggcmVsZWFzZV9wYWdlcygp ICovCisJaWYgKCFmb2xpb190cnlfZ2V0KGZvbGlvKSkKKwkJcmV0dXJuIGZhbHNlOworCisJLyog cmFjZWQgd2l0aCBhbm90aGVyIGlzb2xhdGlvbiAqLworCWlmICghZm9saW9fdGVzdF9jbGVhcl9s cnUoZm9saW8pKSB7CisJCWZvbGlvX3B1dChmb2xpbyk7CisJCXJldHVybiBmYWxzZTsKKwl9CisK KwkvKiBzZWUgdGhlIGNvbW1lbnQgb24gTUFYX05SX1RJRVJTICovCisJaWYgKCFmb2xpb190ZXN0 X3JlZmVyZW5jZWQoZm9saW8pKQorCQlzZXRfbWFza19iaXRzKCZmb2xpby0+ZmxhZ3MsIExSVV9S RUZTX01BU0sgfCBMUlVfUkVGU19GTEFHUywgMCk7CisKKwkvKiBmb3Igc2hyaW5rX3BhZ2VfbGlz dCgpICovCisJZm9saW9fY2xlYXJfcmVjbGFpbShmb2xpbyk7CisJZm9saW9fY2xlYXJfcmVmZXJl bmNlZChmb2xpbyk7CisKKwlzdWNjZXNzID0gbHJ1X2dlbl9kZWxfZm9saW8obHJ1dmVjLCBmb2xp bywgdHJ1ZSk7CisJVk1fV0FSTl9PTl9PTkNFX0ZPTElPKCFzdWNjZXNzLCBmb2xpbyk7CisKKwly ZXR1cm4gdHJ1ZTsKK30KKworc3RhdGljIGludCBzY2FuX2ZvbGlvcyhzdHJ1Y3QgbHJ1dmVjICps cnV2ZWMsIHN0cnVjdCBzY2FuX2NvbnRyb2wgKnNjLAorCQkgICAgICAgaW50IHR5cGUsIGludCB0 aWVyLCBzdHJ1Y3QgbGlzdF9oZWFkICpsaXN0KQoreworCWludCBnZW4sIHpvbmU7CisJZW51bSB2 bV9ldmVudF9pdGVtIGl0ZW07CisJaW50IHNvcnRlZCA9IDA7CisJaW50IHNjYW5uZWQgPSAwOwor CWludCBpc29sYXRlZCA9IDA7CisJaW50IHJlbWFpbmluZyA9IE1BWF9MUlVfQkFUQ0g7CisJc3Ry dWN0IGxydV9nZW5fc3RydWN0ICpscnVnZW4gPSAmbHJ1dmVjLT5scnVnZW47CisJc3RydWN0IG1l bV9jZ3JvdXAgKm1lbWNnID0gbHJ1dmVjX21lbWNnKGxydXZlYyk7CisKKwlWTV9XQVJOX09OX09O Q0UoIWxpc3RfZW1wdHkobGlzdCkpOworCisJaWYgKGdldF9ucl9nZW5zKGxydXZlYywgdHlwZSkg PT0gTUlOX05SX0dFTlMpCisJCXJldHVybiAwOworCisJZ2VuID0gbHJ1X2dlbl9mcm9tX3NlcShs cnVnZW4tPm1pbl9zZXFbdHlwZV0pOworCisJZm9yICh6b25lID0gc2MtPnJlY2xhaW1faWR4OyB6 b25lID49IDA7IHpvbmUtLSkgeworCQlMSVNUX0hFQUQobW92ZWQpOworCQlpbnQgc2tpcHBlZCA9 IDA7CisJCXN0cnVjdCBsaXN0X2hlYWQgKmhlYWQgPSAmbHJ1Z2VuLT5saXN0c1tnZW5dW3R5cGVd W3pvbmVdOworCisJCXdoaWxlICghbGlzdF9lbXB0eShoZWFkKSkgeworCQkJc3RydWN0IGZvbGlv ICpmb2xpbyA9IGxydV90b19mb2xpbyhoZWFkKTsKKwkJCWludCBkZWx0YSA9IGZvbGlvX25yX3Bh Z2VzKGZvbGlvKTsKKworCQkJVk1fV0FSTl9PTl9PTkNFX0ZPTElPKGZvbGlvX3Rlc3RfdW5ldmlj dGFibGUoZm9saW8pLCBmb2xpbyk7CisJCQlWTV9XQVJOX09OX09OQ0VfRk9MSU8oZm9saW9fdGVz dF9hY3RpdmUoZm9saW8pLCBmb2xpbyk7CisJCQlWTV9XQVJOX09OX09OQ0VfRk9MSU8oZm9saW9f aXNfZmlsZV9scnUoZm9saW8pICE9IHR5cGUsIGZvbGlvKTsKKwkJCVZNX1dBUk5fT05fT05DRV9G T0xJTyhmb2xpb196b25lbnVtKGZvbGlvKSAhPSB6b25lLCBmb2xpbyk7CisKKwkJCXNjYW5uZWQg Kz0gZGVsdGE7CisKKwkJCWlmIChzb3J0X2ZvbGlvKGxydXZlYywgZm9saW8sIHRpZXIpKQorCQkJ CXNvcnRlZCArPSBkZWx0YTsKKwkJCWVsc2UgaWYgKGlzb2xhdGVfZm9saW8obHJ1dmVjLCBmb2xp bywgc2MpKSB7CisJCQkJbGlzdF9hZGQoJmZvbGlvLT5scnUsIGxpc3QpOworCQkJCWlzb2xhdGVk ICs9IGRlbHRhOworCQkJfSBlbHNlIHsKKwkJCQlsaXN0X21vdmUoJmZvbGlvLT5scnUsICZtb3Zl ZCk7CisJCQkJc2tpcHBlZCArPSBkZWx0YTsKKwkJCX0KKworCQkJaWYgKCEtLXJlbWFpbmluZyB8 fCBtYXgoaXNvbGF0ZWQsIHNraXBwZWQpID49IE1JTl9MUlVfQkFUQ0gpCisJCQkJYnJlYWs7CisJ CX0KKworCQlpZiAoc2tpcHBlZCkgeworCQkJbGlzdF9zcGxpY2UoJm1vdmVkLCBoZWFkKTsKKwkJ CV9fY291bnRfemlkX3ZtX2V2ZW50cyhQR1NDQU5fU0tJUCwgem9uZSwgc2tpcHBlZCk7CisJCX0K KworCQlpZiAoIXJlbWFpbmluZyB8fCBpc29sYXRlZCA+PSBNSU5fTFJVX0JBVENIKQorCQkJYnJl YWs7CisJfQorCisJaXRlbSA9IGN1cnJlbnRfaXNfa3N3YXBkKCkgPyBQR1NDQU5fS1NXQVBEIDog UEdTQ0FOX0RJUkVDVDsKKwlpZiAoIWNncm91cF9yZWNsYWltKHNjKSkgeworCQlfX2NvdW50X3Zt X2V2ZW50cyhpdGVtLCBpc29sYXRlZCk7CisJCV9fY291bnRfdm1fZXZlbnRzKFBHUkVGSUxMLCBz b3J0ZWQpOworCX0KKwlfX2NvdW50X21lbWNnX2V2ZW50cyhtZW1jZywgaXRlbSwgaXNvbGF0ZWQp OworCV9fY291bnRfbWVtY2dfZXZlbnRzKG1lbWNnLCBQR1JFRklMTCwgc29ydGVkKTsKKwlfX2Nv dW50X3ZtX2V2ZW50cyhQR1NDQU5fQU5PTiArIHR5cGUsIGlzb2xhdGVkKTsKKworCS8qCisJICog VGhlcmUgbWlnaHQgbm90IGJlIGVsaWdpYmxlIHBhZ2VzIGR1ZSB0byByZWNsYWltX2lkeCwgbWF5 X3VubWFwIGFuZAorCSAqIG1heV93cml0ZXBhZ2UuIENoZWNrIHRoZSByZW1haW5pbmcgdG8gcHJl dmVudCBsaXZlbG9jayBpZiBpdCdzIG5vdAorCSAqIG1ha2luZyBwcm9ncmVzcy4KKwkgKi8KKwly ZXR1cm4gaXNvbGF0ZWQgfHwgIXJlbWFpbmluZyA/IHNjYW5uZWQgOiAwOworfQorCitzdGF0aWMg aW50IGdldF90aWVyX2lkeChzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIGludCB0eXBlKQoreworCWlu dCB0aWVyOworCXN0cnVjdCBjdHJsX3BvcyBzcCwgcHY7CisKKwkvKgorCSAqIFRvIGxlYXZlIGEg bWFyZ2luIGZvciBmbHVjdHVhdGlvbnMsIHVzZSBhIGxhcmdlciBnYWluIGZhY3RvciAoMToyKS4K KwkgKiBUaGlzIHZhbHVlIGlzIGNob3NlbiBiZWNhdXNlIGFueSBvdGhlciB0aWVyIHdvdWxkIGhh dmUgYXQgbGVhc3QgdHdpY2UKKwkgKiBhcyBtYW55IHJlZmF1bHRzIGFzIHRoZSBmaXJzdCB0aWVy LgorCSAqLworCXJlYWRfY3RybF9wb3MobHJ1dmVjLCB0eXBlLCAwLCAxLCAmc3ApOworCWZvciAo dGllciA9IDE7IHRpZXIgPCBNQVhfTlJfVElFUlM7IHRpZXIrKykgeworCQlyZWFkX2N0cmxfcG9z KGxydXZlYywgdHlwZSwgdGllciwgMiwgJnB2KTsKKwkJaWYgKCFwb3NpdGl2ZV9jdHJsX2Vycigm c3AsICZwdikpCisJCQlicmVhazsKKwl9CisKKwlyZXR1cm4gdGllciAtIDE7Cit9CisKK3N0YXRp YyBpbnQgZ2V0X3R5cGVfdG9fc2NhbihzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIGludCBzd2FwcGlu ZXNzLCBpbnQgKnRpZXJfaWR4KQoreworCWludCB0eXBlLCB0aWVyOworCXN0cnVjdCBjdHJsX3Bv cyBzcCwgcHY7CisJaW50IGdhaW5bQU5PTl9BTkRfRklMRV0gPSB7IHN3YXBwaW5lc3MsIDIwMCAt IHN3YXBwaW5lc3MgfTsKKworCS8qCisJICogQ29tcGFyZSB0aGUgZmlyc3QgdGllciBvZiBhbm9u IHdpdGggdGhhdCBvZiBmaWxlIHRvIGRldGVybWluZSB3aGljaAorCSAqIHR5cGUgdG8gc2Nhbi4g QWxzbyBuZWVkIHRvIGNvbXBhcmUgb3RoZXIgdGllcnMgb2YgdGhlIHNlbGVjdGVkIHR5cGUKKwkg KiB3aXRoIHRoZSBmaXJzdCB0aWVyIG9mIHRoZSBvdGhlciB0eXBlIHRvIGRldGVybWluZSB0aGUg bGFzdCB0aWVyIChvZgorCSAqIHRoZSBzZWxlY3RlZCB0eXBlKSB0byBldmljdC4KKwkgKi8KKwly ZWFkX2N0cmxfcG9zKGxydXZlYywgTFJVX0dFTl9BTk9OLCAwLCBnYWluW0xSVV9HRU5fQU5PTl0s ICZzcCk7CisJcmVhZF9jdHJsX3BvcyhscnV2ZWMsIExSVV9HRU5fRklMRSwgMCwgZ2FpbltMUlVf R0VOX0ZJTEVdLCAmcHYpOworCXR5cGUgPSBwb3NpdGl2ZV9jdHJsX2Vycigmc3AsICZwdik7CisK KwlyZWFkX2N0cmxfcG9zKGxydXZlYywgIXR5cGUsIDAsIGdhaW5bIXR5cGVdLCAmc3ApOworCWZv ciAodGllciA9IDE7IHRpZXIgPCBNQVhfTlJfVElFUlM7IHRpZXIrKykgeworCQlyZWFkX2N0cmxf cG9zKGxydXZlYywgdHlwZSwgdGllciwgZ2Fpblt0eXBlXSwgJnB2KTsKKwkJaWYgKCFwb3NpdGl2 ZV9jdHJsX2Vycigmc3AsICZwdikpCisJCQlicmVhazsKKwl9CisKKwkqdGllcl9pZHggPSB0aWVy IC0gMTsKKworCXJldHVybiB0eXBlOworfQorCitzdGF0aWMgaW50IGlzb2xhdGVfZm9saW9zKHN0 cnVjdCBscnV2ZWMgKmxydXZlYywgc3RydWN0IHNjYW5fY29udHJvbCAqc2MsIGludCBzd2FwcGlu ZXNzLAorCQkJICBpbnQgKnR5cGVfc2Nhbm5lZCwgc3RydWN0IGxpc3RfaGVhZCAqbGlzdCkKK3sK KwlpbnQgaTsKKwlpbnQgdHlwZTsKKwlpbnQgc2Nhbm5lZDsKKwlpbnQgdGllciA9IC0xOworCURF RklORV9NSU5fU0VRKGxydXZlYyk7CisKKwkvKgorCSAqIFRyeSB0byBtYWtlIHRoZSBvYnZpb3Vz IGNob2ljZSBmaXJzdC4gV2hlbiBhbm9uIGFuZCBmaWxlIGFyZSBib3RoCisJICogYXZhaWxhYmxl IGZyb20gdGhlIHNhbWUgZ2VuZXJhdGlvbiwgaW50ZXJwcmV0IHN3YXBwaW5lc3MgMSBhcyBmaWxl CisJICogZmlyc3QgYW5kIDIwMCBhcyBhbm9uIGZpcnN0LgorCSAqLworCWlmICghc3dhcHBpbmVz cykKKwkJdHlwZSA9IExSVV9HRU5fRklMRTsKKwllbHNlIGlmIChtaW5fc2VxW0xSVV9HRU5fQU5P Tl0gPCBtaW5fc2VxW0xSVV9HRU5fRklMRV0pCisJCXR5cGUgPSBMUlVfR0VOX0FOT047CisJZWxz ZSBpZiAoc3dhcHBpbmVzcyA9PSAxKQorCQl0eXBlID0gTFJVX0dFTl9GSUxFOworCWVsc2UgaWYg KHN3YXBwaW5lc3MgPT0gMjAwKQorCQl0eXBlID0gTFJVX0dFTl9BTk9OOworCWVsc2UKKwkJdHlw ZSA9IGdldF90eXBlX3RvX3NjYW4obHJ1dmVjLCBzd2FwcGluZXNzLCAmdGllcik7CisKKwlmb3Ig KGkgPSAhc3dhcHBpbmVzczsgaSA8IEFOT05fQU5EX0ZJTEU7IGkrKykgeworCQlpZiAodGllciA8 IDApCisJCQl0aWVyID0gZ2V0X3RpZXJfaWR4KGxydXZlYywgdHlwZSk7CisKKwkJc2Nhbm5lZCA9 IHNjYW5fZm9saW9zKGxydXZlYywgc2MsIHR5cGUsIHRpZXIsIGxpc3QpOworCQlpZiAoc2Nhbm5l ZCkKKwkJCWJyZWFrOworCisJCXR5cGUgPSAhdHlwZTsKKwkJdGllciA9IC0xOworCX0KKworCSp0 eXBlX3NjYW5uZWQgPSB0eXBlOworCisJcmV0dXJuIHNjYW5uZWQ7Cit9CisKK3N0YXRpYyBpbnQg ZXZpY3RfZm9saW9zKHN0cnVjdCBscnV2ZWMgKmxydXZlYywgc3RydWN0IHNjYW5fY29udHJvbCAq c2MsIGludCBzd2FwcGluZXNzKQoreworCWludCB0eXBlOworCWludCBzY2FubmVkOworCWludCBy ZWNsYWltZWQ7CisJTElTVF9IRUFEKGxpc3QpOworCXN0cnVjdCBmb2xpbyAqZm9saW87CisJZW51 bSB2bV9ldmVudF9pdGVtIGl0ZW07CisJc3RydWN0IHJlY2xhaW1fc3RhdCBzdGF0OworCXN0cnVj dCBtZW1fY2dyb3VwICptZW1jZyA9IGxydXZlY19tZW1jZyhscnV2ZWMpOworCXN0cnVjdCBwZ2xp c3RfZGF0YSAqcGdkYXQgPSBscnV2ZWNfcGdkYXQobHJ1dmVjKTsKKworCXNwaW5fbG9ja19pcnEo JmxydXZlYy0+bHJ1X2xvY2spOworCisJc2Nhbm5lZCA9IGlzb2xhdGVfZm9saW9zKGxydXZlYywg c2MsIHN3YXBwaW5lc3MsICZ0eXBlLCAmbGlzdCk7CisKKwlzY2FubmVkICs9IHRyeV90b19pbmNf bWluX3NlcShscnV2ZWMsIHN3YXBwaW5lc3MpOworCisJaWYgKGdldF9ucl9nZW5zKGxydXZlYywg IXN3YXBwaW5lc3MpID09IE1JTl9OUl9HRU5TKQorCQlzY2FubmVkID0gMDsKKworCXNwaW5fdW5s b2NrX2lycSgmbHJ1dmVjLT5scnVfbG9jayk7CisKKwlpZiAobGlzdF9lbXB0eSgmbGlzdCkpCisJ CXJldHVybiBzY2FubmVkOworCisJcmVjbGFpbWVkID0gc2hyaW5rX3BhZ2VfbGlzdCgmbGlzdCwg cGdkYXQsIHNjLCAmc3RhdCwgZmFsc2UpOworCisJbGlzdF9mb3JfZWFjaF9lbnRyeShmb2xpbywg Jmxpc3QsIGxydSkgeworCQkvKiByZXN0b3JlIExSVV9SRUZTX0ZMQUdTIGNsZWFyZWQgYnkgaXNv bGF0ZV9mb2xpbygpICovCisJCWlmIChmb2xpb190ZXN0X3dvcmtpbmdzZXQoZm9saW8pKQorCQkJ Zm9saW9fc2V0X3JlZmVyZW5jZWQoZm9saW8pOworCisJCS8qIGRvbid0IGFkZCByZWplY3RlZCBw YWdlcyB0byB0aGUgb2xkZXN0IGdlbmVyYXRpb24gKi8KKwkJaWYgKGZvbGlvX3Rlc3RfcmVjbGFp bShmb2xpbykgJiYKKwkJICAgIChmb2xpb190ZXN0X2RpcnR5KGZvbGlvKSB8fCBmb2xpb190ZXN0 X3dyaXRlYmFjayhmb2xpbykpKQorCQkJZm9saW9fY2xlYXJfYWN0aXZlKGZvbGlvKTsKKwkJZWxz ZQorCQkJZm9saW9fc2V0X2FjdGl2ZShmb2xpbyk7CisJfQorCisJc3Bpbl9sb2NrX2lycSgmbHJ1 dmVjLT5scnVfbG9jayk7CisKKwltb3ZlX3BhZ2VzX3RvX2xydShscnV2ZWMsICZsaXN0KTsKKwor CWl0ZW0gPSBjdXJyZW50X2lzX2tzd2FwZCgpID8gUEdTVEVBTF9LU1dBUEQgOiBQR1NURUFMX0RJ UkVDVDsKKwlpZiAoIWNncm91cF9yZWNsYWltKHNjKSkKKwkJX19jb3VudF92bV9ldmVudHMoaXRl bSwgcmVjbGFpbWVkKTsKKwlfX2NvdW50X21lbWNnX2V2ZW50cyhtZW1jZywgaXRlbSwgcmVjbGFp bWVkKTsKKwlfX2NvdW50X3ZtX2V2ZW50cyhQR1NURUFMX0FOT04gKyB0eXBlLCByZWNsYWltZWQp OworCisJc3Bpbl91bmxvY2tfaXJxKCZscnV2ZWMtPmxydV9sb2NrKTsKKworCW1lbV9jZ3JvdXBf dW5jaGFyZ2VfbGlzdCgmbGlzdCk7CisJZnJlZV91bnJlZl9wYWdlX2xpc3QoJmxpc3QpOworCisJ c2MtPm5yX3JlY2xhaW1lZCArPSByZWNsYWltZWQ7CisKKwlyZXR1cm4gc2Nhbm5lZDsKK30KKwor c3RhdGljIHVuc2lnbmVkIGxvbmcgZ2V0X25yX3RvX3NjYW4oc3RydWN0IGxydXZlYyAqbHJ1dmVj LCBzdHJ1Y3Qgc2Nhbl9jb250cm9sICpzYywKKwkJCQkgICAgYm9vbCBjYW5fc3dhcCwgdW5zaWdu ZWQgbG9uZyByZWNsYWltZWQpCit7CisJaW50IHByaW9yaXR5OworCWJvb2wgbmVlZF9hZ2luZzsK Kwl1bnNpZ25lZCBsb25nIG5yX3RvX3NjYW47CisJc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnID0g bHJ1dmVjX21lbWNnKGxydXZlYyk7CisJREVGSU5FX01BWF9TRVEobHJ1dmVjKTsKKwlERUZJTkVf TUlOX1NFUShscnV2ZWMpOworCisJaWYgKGZhdGFsX3NpZ25hbF9wZW5kaW5nKGN1cnJlbnQpKSB7 CisJCXNjLT5ucl9yZWNsYWltZWQgKz0gTUlOX0xSVV9CQVRDSDsKKwkJcmV0dXJuIDA7CisJfQor CisJaWYgKG1lbV9jZ3JvdXBfYmVsb3dfbWluKG1lbWNnKSB8fAorCSAgICAobWVtX2Nncm91cF9i ZWxvd19sb3cobWVtY2cpICYmICFzYy0+bWVtY2dfbG93X3JlY2xhaW0pKQorCQlyZXR1cm4gMDsK KworCW5yX3RvX3NjYW4gPSBnZXRfbnJfZXZpY3RhYmxlKGxydXZlYywgbWF4X3NlcSwgbWluX3Nl cSwgY2FuX3N3YXAsICZuZWVkX2FnaW5nKTsKKwlpZiAoIW5yX3RvX3NjYW4pCisJCXJldHVybiAw OworCisJLyogYWRqdXN0IHByaW9yaXR5IGlmIG1lbWNnIGlzIG9mZmxpbmUgb3IgdGhlIHRhcmdl dCBpcyBtZXQgKi8KKwlpZiAoIW1lbV9jZ3JvdXBfb25saW5lKG1lbWNnKSkKKwkJcHJpb3JpdHkg PSAwOworCWVsc2UgaWYgKHNjLT5ucl9yZWNsYWltZWQgLSByZWNsYWltZWQgPj0gc2MtPm5yX3Rv X3JlY2xhaW0pCisJCXByaW9yaXR5ID0gREVGX1BSSU9SSVRZOworCWVsc2UKKwkJcHJpb3JpdHkg PSBzYy0+cHJpb3JpdHk7CisKKwlucl90b19zY2FuID4+PSBwcmlvcml0eTsKKwlpZiAoIW5yX3Rv X3NjYW4pCisJCXJldHVybiAwOworCisJaWYgKCFuZWVkX2FnaW5nKQorCQlyZXR1cm4gbnJfdG9f c2NhbjsKKworCS8qIHNraXAgdGhlIGFnaW5nIHBhdGggYXQgdGhlIGRlZmF1bHQgcHJpb3JpdHkg Ki8KKwlpZiAocHJpb3JpdHkgPT0gREVGX1BSSU9SSVRZKQorCQlnb3RvIGRvbmU7CisKKwkvKiBs ZWF2ZSB0aGUgd29yayB0byBscnVfZ2VuX2FnZV9ub2RlKCkgKi8KKwlpZiAoY3VycmVudF9pc19r c3dhcGQoKSkKKwkJcmV0dXJuIDA7CisKKwlpbmNfbWF4X3NlcShscnV2ZWMsIG1heF9zZXEsIGNh bl9zd2FwKTsKK2RvbmU6CisJcmV0dXJuIG1pbl9zZXFbIWNhbl9zd2FwXSArIE1JTl9OUl9HRU5T IDw9IG1heF9zZXEgPyBucl90b19zY2FuIDogMDsKK30KKworc3RhdGljIHZvaWQgbHJ1X2dlbl9z aHJpbmtfbHJ1dmVjKHN0cnVjdCBscnV2ZWMgKmxydXZlYywgc3RydWN0IHNjYW5fY29udHJvbCAq c2MpCit7CisJc3RydWN0IGJsa19wbHVnIHBsdWc7CisJdW5zaWduZWQgbG9uZyBzY2FubmVkID0g MDsKKwl1bnNpZ25lZCBsb25nIHJlY2xhaW1lZCA9IHNjLT5ucl9yZWNsYWltZWQ7CisKKwlscnVf YWRkX2RyYWluKCk7CisKKwlibGtfc3RhcnRfcGx1ZygmcGx1Zyk7CisKKwl3aGlsZSAodHJ1ZSkg eworCQlpbnQgZGVsdGE7CisJCWludCBzd2FwcGluZXNzOworCQl1bnNpZ25lZCBsb25nIG5yX3Rv X3NjYW47CisKKwkJaWYgKHNjLT5tYXlfc3dhcCkKKwkJCXN3YXBwaW5lc3MgPSBnZXRfc3dhcHBp bmVzcyhscnV2ZWMsIHNjKTsKKwkJZWxzZSBpZiAoIWNncm91cF9yZWNsYWltKHNjKSAmJiBnZXRf c3dhcHBpbmVzcyhscnV2ZWMsIHNjKSkKKwkJCXN3YXBwaW5lc3MgPSAxOworCQllbHNlCisJCQlz d2FwcGluZXNzID0gMDsKKworCQlucl90b19zY2FuID0gZ2V0X25yX3RvX3NjYW4obHJ1dmVjLCBz Yywgc3dhcHBpbmVzcywgcmVjbGFpbWVkKTsKKwkJaWYgKCFucl90b19zY2FuKQorCQkJYnJlYWs7 CisKKwkJZGVsdGEgPSBldmljdF9mb2xpb3MobHJ1dmVjLCBzYywgc3dhcHBpbmVzcyk7CisJCWlm ICghZGVsdGEpCisJCQlicmVhazsKKworCQlzY2FubmVkICs9IGRlbHRhOworCQlpZiAoc2Nhbm5l ZCA+PSBucl90b19zY2FuKQorCQkJYnJlYWs7CisKKwkJY29uZF9yZXNjaGVkKCk7CisJfQorCisJ YmxrX2ZpbmlzaF9wbHVnKCZwbHVnKTsKK30KKwogLyoqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKgogICog ICAgICAgICAgICAgICAgICAgICAgICAgIGluaXRpYWxpemF0aW9uCiAgKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqLwpAQCAtMzEyMyw2ICszOTAyLDE2IEBAIHN0YXRpYyBpbnQgX19pbml0IGluaXRfbHJ1 X2dlbih2b2lkKQogfTsKIGxhdGVfaW5pdGNhbGwoaW5pdF9scnVfZ2VuKTsKIAorI2Vsc2UgLyog IUNPTkZJR19MUlVfR0VOICovCisKK3N0YXRpYyB2b2lkIGxydV9nZW5fYWdlX25vZGUoc3RydWN0 IHBnbGlzdF9kYXRhICpwZ2RhdCwgc3RydWN0IHNjYW5fY29udHJvbCAqc2MpCit7Cit9CisKK3N0 YXRpYyB2b2lkIGxydV9nZW5fc2hyaW5rX2xydXZlYyhzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIHN0 cnVjdCBzY2FuX2NvbnRyb2wgKnNjKQoreworfQorCiAjZW5kaWYgLyogQ09ORklHX0xSVV9HRU4g Ki8KIAogc3RhdGljIHZvaWQgc2hyaW5rX2xydXZlYyhzdHJ1Y3QgbHJ1dmVjICpscnV2ZWMsIHN0 cnVjdCBzY2FuX2NvbnRyb2wgKnNjKQpAQCAtMzEzNiw2ICszOTI1LDExIEBAIHN0YXRpYyB2b2lk IHNocmlua19scnV2ZWMoc3RydWN0IGxydXZlYyAqbHJ1dmVjLCBzdHJ1Y3Qgc2Nhbl9jb250cm9s ICpzYykKIAlib29sIHByb3BvcnRpb25hbF9yZWNsYWltOwogCXN0cnVjdCBibGtfcGx1ZyBwbHVn OwogCisJaWYgKGxydV9nZW5fZW5hYmxlZCgpKSB7CisJCWxydV9nZW5fc2hyaW5rX2xydXZlYyhs cnV2ZWMsIHNjKTsKKwkJcmV0dXJuOworCX0KKwogCWdldF9zY2FuX2NvdW50KGxydXZlYywgc2Ms IG5yKTsKIAogCS8qIFJlY29yZCB0aGUgb3JpZ2luYWwgc2NhbiB0YXJnZXQgZm9yIHByb3BvcnRp b25hbCBhZGp1c3RtZW50cyBsYXRlciAqLwpAQCAtMzY0MCw2ICs0NDM0LDkgQEAgc3RhdGljIHZv aWQgc25hcHNob3RfcmVmYXVsdHMoc3RydWN0IG1lbV9jZ3JvdXAgKnRhcmdldF9tZW1jZywgcGdf ZGF0YV90ICpwZ2RhdCkKIAlzdHJ1Y3QgbHJ1dmVjICp0YXJnZXRfbHJ1dmVjOwogCXVuc2lnbmVk IGxvbmcgcmVmYXVsdHM7CiAKKwlpZiAobHJ1X2dlbl9lbmFibGVkKCkpCisJCXJldHVybjsKKwog CXRhcmdldF9scnV2ZWMgPSBtZW1fY2dyb3VwX2xydXZlYyh0YXJnZXRfbWVtY2csIHBnZGF0KTsK IAlyZWZhdWx0cyA9IGxydXZlY19wYWdlX3N0YXRlKHRhcmdldF9scnV2ZWMsIFdPUktJTkdTRVRf QUNUSVZBVEVfQU5PTik7CiAJdGFyZ2V0X2xydXZlYy0+cmVmYXVsdHNbV09SS0lOR1NFVF9BTk9O XSA9IHJlZmF1bHRzOwpAQCAtNDAwNiwxMiArNDgwMywxNyBAQCB1bnNpZ25lZCBsb25nIHRyeV90 b19mcmVlX21lbV9jZ3JvdXBfcGFnZXMoc3RydWN0IG1lbV9jZ3JvdXAgKm1lbWNnLAogfQogI2Vu ZGlmCiAKLXN0YXRpYyB2b2lkIGFnZV9hY3RpdmVfYW5vbihzdHJ1Y3QgcGdsaXN0X2RhdGEgKnBn ZGF0LAorc3RhdGljIHZvaWQga3N3YXBkX2FnZV9ub2RlKHN0cnVjdCBwZ2xpc3RfZGF0YSAqcGdk YXQsCiAJCQkJc3RydWN0IHNjYW5fY29udHJvbCAqc2MpCiB7CiAJc3RydWN0IG1lbV9jZ3JvdXAg Km1lbWNnOwogCXN0cnVjdCBscnV2ZWMgKmxydXZlYzsKIAorCWlmIChscnVfZ2VuX2VuYWJsZWQo KSkgeworCQlscnVfZ2VuX2FnZV9ub2RlKHBnZGF0LCBzYyk7CisJCXJldHVybjsKKwl9CisKIAlp ZiAoIWNhbl9hZ2VfYW5vbl9wYWdlcyhwZ2RhdCwgc2MpKQogCQlyZXR1cm47CiAKQEAgLTQzMzEs MTIgKzUxMzMsMTEgQEAgc3RhdGljIGludCBiYWxhbmNlX3BnZGF0KHBnX2RhdGFfdCAqcGdkYXQs IGludCBvcmRlciwgaW50IGhpZ2hlc3Rfem9uZWlkeCkKIAkJc2MubWF5X3N3YXAgPSAhbnJfYm9v c3RfcmVjbGFpbTsKIAogCQkvKgotCQkgKiBEbyBzb21lIGJhY2tncm91bmQgYWdpbmcgb2YgdGhl IGFub24gbGlzdCwgdG8gZ2l2ZQotCQkgKiBwYWdlcyBhIGNoYW5jZSB0byBiZSByZWZlcmVuY2Vk IGJlZm9yZSByZWNsYWltaW5nLiBBbGwKLQkJICogcGFnZXMgYXJlIHJvdGF0ZWQgcmVnYXJkbGVz cyBvZiBjbGFzc3pvbmUgYXMgdGhpcyBpcwotCQkgKiBhYm91dCBjb25zaXN0ZW50IGFnaW5nLgor CQkgKiBEbyBzb21lIGJhY2tncm91bmQgYWdpbmcsIHRvIGdpdmUgcGFnZXMgYSBjaGFuY2UgdG8g YmUKKwkJICogcmVmZXJlbmNlZCBiZWZvcmUgcmVjbGFpbWluZy4gQWxsIHBhZ2VzIGFyZSByb3Rh dGVkCisJCSAqIHJlZ2FyZGxlc3Mgb2YgY2xhc3N6b25lIGFzIHRoaXMgaXMgYWJvdXQgY29uc2lz dGVudCBhZ2luZy4KIAkJICovCi0JCWFnZV9hY3RpdmVfYW5vbihwZ2RhdCwgJnNjKTsKKwkJa3N3 YXBkX2FnZV9ub2RlKHBnZGF0LCAmc2MpOwogCiAJCS8qCiAJCSAqIElmIHdlJ3JlIGdldHRpbmcg dHJvdWJsZSByZWNsYWltaW5nLCBzdGFydCBkb2luZyB3cml0ZXBhZ2UKZGlmZiAtLWdpdCBhL21t L3dvcmtpbmdzZXQuYyBiL21tL3dvcmtpbmdzZXQuYwppbmRleCBhNWU4NDg2MmZjODYuLmFlN2U5 ODRiMjNjNiAxMDA2NDQKLS0tIGEvbW0vd29ya2luZ3NldC5jCisrKyBiL21tL3dvcmtpbmdzZXQu YwpAQCAtMTg3LDcgKzE4Nyw2IEBAIHN0YXRpYyB1bnNpZ25lZCBpbnQgYnVja2V0X29yZGVyIF9f cmVhZF9tb3N0bHk7CiBzdGF0aWMgdm9pZCAqcGFja19zaGFkb3coaW50IG1lbWNnaWQsIHBnX2Rh dGFfdCAqcGdkYXQsIHVuc2lnbmVkIGxvbmcgZXZpY3Rpb24sCiAJCQkgYm9vbCB3b3JraW5nc2V0 KQogewotCWV2aWN0aW9uID4+PSBidWNrZXRfb3JkZXI7CiAJZXZpY3Rpb24gJj0gRVZJQ1RJT05f TUFTSzsKIAlldmljdGlvbiA9IChldmljdGlvbiA8PCBNRU1fQ0dST1VQX0lEX1NISUZUKSB8IG1l bWNnaWQ7CiAJZXZpY3Rpb24gPSAoZXZpY3Rpb24gPDwgTk9ERVNfU0hJRlQpIHwgcGdkYXQtPm5v ZGVfaWQ7CkBAIC0yMTIsMTAgKzIxMSwxMDcgQEAgc3RhdGljIHZvaWQgdW5wYWNrX3NoYWRvdyh2 b2lkICpzaGFkb3csIGludCAqbWVtY2dpZHAsIHBnX2RhdGFfdCAqKnBnZGF0LAogCiAJKm1lbWNn aWRwID0gbWVtY2dpZDsKIAkqcGdkYXQgPSBOT0RFX0RBVEEobmlkKTsKLQkqZXZpY3Rpb25wID0g ZW50cnkgPDwgYnVja2V0X29yZGVyOworCSpldmljdGlvbnAgPSBlbnRyeTsKIAkqd29ya2luZ3Nl dHAgPSB3b3JraW5nc2V0OwogfQogCisjaWZkZWYgQ09ORklHX0xSVV9HRU4KKworc3RhdGljIHZv aWQgKmxydV9nZW5fZXZpY3Rpb24oc3RydWN0IGZvbGlvICpmb2xpbykKK3sKKwlpbnQgaGlzdDsK Kwl1bnNpZ25lZCBsb25nIHRva2VuOworCXVuc2lnbmVkIGxvbmcgbWluX3NlcTsKKwlzdHJ1Y3Qg bHJ1dmVjICpscnV2ZWM7CisJc3RydWN0IGxydV9nZW5fc3RydWN0ICpscnVnZW47CisJaW50IHR5 cGUgPSBmb2xpb19pc19maWxlX2xydShmb2xpbyk7CisJaW50IGRlbHRhID0gZm9saW9fbnJfcGFn ZXMoZm9saW8pOworCWludCByZWZzID0gZm9saW9fbHJ1X3JlZnMoZm9saW8pOworCWludCB0aWVy ID0gbHJ1X3RpZXJfZnJvbV9yZWZzKHJlZnMpOworCXN0cnVjdCBtZW1fY2dyb3VwICptZW1jZyA9 IGZvbGlvX21lbWNnKGZvbGlvKTsKKwlzdHJ1Y3QgcGdsaXN0X2RhdGEgKnBnZGF0ID0gZm9saW9f cGdkYXQoZm9saW8pOworCisJQlVJTERfQlVHX09OKExSVV9HRU5fV0lEVEggKyBMUlVfUkVGU19X SURUSCA+IEJJVFNfUEVSX0xPTkcgLSBFVklDVElPTl9TSElGVCk7CisKKwlscnV2ZWMgPSBtZW1f Y2dyb3VwX2xydXZlYyhtZW1jZywgcGdkYXQpOworCWxydWdlbiA9ICZscnV2ZWMtPmxydWdlbjsK KwltaW5fc2VxID0gUkVBRF9PTkNFKGxydWdlbi0+bWluX3NlcVt0eXBlXSk7CisJdG9rZW4gPSAo bWluX3NlcSA8PCBMUlVfUkVGU19XSURUSCkgfCBtYXgocmVmcyAtIDEsIDApOworCisJaGlzdCA9 IGxydV9oaXN0X2Zyb21fc2VxKG1pbl9zZXEpOworCWF0b21pY19sb25nX2FkZChkZWx0YSwgJmxy dWdlbi0+ZXZpY3RlZFtoaXN0XVt0eXBlXVt0aWVyXSk7CisKKwlyZXR1cm4gcGFja19zaGFkb3co bWVtX2Nncm91cF9pZChtZW1jZyksIHBnZGF0LCB0b2tlbiwgcmVmcyk7Cit9CisKK3N0YXRpYyB2 b2lkIGxydV9nZW5fcmVmYXVsdChzdHJ1Y3QgZm9saW8gKmZvbGlvLCB2b2lkICpzaGFkb3cpCit7 CisJaW50IGhpc3QsIHRpZXIsIHJlZnM7CisJaW50IG1lbWNnX2lkOworCWJvb2wgd29ya2luZ3Nl dDsKKwl1bnNpZ25lZCBsb25nIHRva2VuOworCXVuc2lnbmVkIGxvbmcgbWluX3NlcTsKKwlzdHJ1 Y3QgbHJ1dmVjICpscnV2ZWM7CisJc3RydWN0IGxydV9nZW5fc3RydWN0ICpscnVnZW47CisJc3Ry dWN0IG1lbV9jZ3JvdXAgKm1lbWNnOworCXN0cnVjdCBwZ2xpc3RfZGF0YSAqcGdkYXQ7CisJaW50 IHR5cGUgPSBmb2xpb19pc19maWxlX2xydShmb2xpbyk7CisJaW50IGRlbHRhID0gZm9saW9fbnJf cGFnZXMoZm9saW8pOworCisJdW5wYWNrX3NoYWRvdyhzaGFkb3csICZtZW1jZ19pZCwgJnBnZGF0 LCAmdG9rZW4sICZ3b3JraW5nc2V0KTsKKworCWlmIChwZ2RhdCAhPSBmb2xpb19wZ2RhdChmb2xp bykpCisJCXJldHVybjsKKworCXJjdV9yZWFkX2xvY2soKTsKKworCW1lbWNnID0gZm9saW9fbWVt Y2dfcmN1KGZvbGlvKTsKKwlpZiAobWVtY2dfaWQgIT0gbWVtX2Nncm91cF9pZChtZW1jZykpCisJ CWdvdG8gdW5sb2NrOworCisJbHJ1dmVjID0gbWVtX2Nncm91cF9scnV2ZWMobWVtY2csIHBnZGF0 KTsKKwlscnVnZW4gPSAmbHJ1dmVjLT5scnVnZW47CisKKwltaW5fc2VxID0gUkVBRF9PTkNFKGxy dWdlbi0+bWluX3NlcVt0eXBlXSk7CisJaWYgKCh0b2tlbiA+PiBMUlVfUkVGU19XSURUSCkgIT0g KG1pbl9zZXEgJiAoRVZJQ1RJT05fTUFTSyA+PiBMUlVfUkVGU19XSURUSCkpKQorCQlnb3RvIHVu bG9jazsKKworCWhpc3QgPSBscnVfaGlzdF9mcm9tX3NlcShtaW5fc2VxKTsKKwkvKiBzZWUgdGhl IGNvbW1lbnQgaW4gZm9saW9fbHJ1X3JlZnMoKSAqLworCXJlZnMgPSAodG9rZW4gJiAoQklUKExS VV9SRUZTX1dJRFRIKSAtIDEpKSArIHdvcmtpbmdzZXQ7CisJdGllciA9IGxydV90aWVyX2Zyb21f cmVmcyhyZWZzKTsKKworCWF0b21pY19sb25nX2FkZChkZWx0YSwgJmxydWdlbi0+cmVmYXVsdGVk W2hpc3RdW3R5cGVdW3RpZXJdKTsKKwltb2RfbHJ1dmVjX3N0YXRlKGxydXZlYywgV09SS0lOR1NF VF9SRUZBVUxUX0JBU0UgKyB0eXBlLCBkZWx0YSk7CisKKwkvKgorCSAqIENvdW50IHRoZSBmb2xs b3dpbmcgdHdvIGNhc2VzIGFzIHN0YWxsczoKKwkgKiAxLiBGb3IgcGFnZXMgYWNjZXNzZWQgdGhy b3VnaCBwYWdlIHRhYmxlcywgaG90dGVyIHBhZ2VzIHB1c2hlZCBvdXQKKwkgKiAgICBob3QgcGFn ZXMgd2hpY2ggcmVmYXVsdGVkIGltbWVkaWF0ZWx5LgorCSAqIDIuIEZvciBwYWdlcyBhY2Nlc3Nl ZCBtdWx0aXBsZSB0aW1lcyB0aHJvdWdoIGZpbGUgZGVzY3JpcHRvcnMsCisJICogICAgbnVtYmVy cyBvZiBhY2Nlc3NlcyBtaWdodCBoYXZlIGJlZW4gb3V0IG9mIHRoZSByYW5nZS4KKwkgKi8KKwlp ZiAobHJ1X2dlbl9pbl9mYXVsdCgpIHx8IHJlZnMgPT0gQklUKExSVV9SRUZTX1dJRFRIKSkgewor CQlmb2xpb19zZXRfd29ya2luZ3NldChmb2xpbyk7CisJCW1vZF9scnV2ZWNfc3RhdGUobHJ1dmVj LCBXT1JLSU5HU0VUX1JFU1RPUkVfQkFTRSArIHR5cGUsIGRlbHRhKTsKKwl9Cit1bmxvY2s6CisJ cmN1X3JlYWRfdW5sb2NrKCk7Cit9CisKKyNlbHNlIC8qICFDT05GSUdfTFJVX0dFTiAqLworCitz dGF0aWMgdm9pZCAqbHJ1X2dlbl9ldmljdGlvbihzdHJ1Y3QgZm9saW8gKmZvbGlvKQoreworCXJl dHVybiBOVUxMOworfQorCitzdGF0aWMgdm9pZCBscnVfZ2VuX3JlZmF1bHQoc3RydWN0IGZvbGlv ICpmb2xpbywgdm9pZCAqc2hhZG93KQoreworfQorCisjZW5kaWYgLyogQ09ORklHX0xSVV9HRU4g Ki8KKwogLyoqCiAgKiB3b3JraW5nc2V0X2FnZV9ub25yZXNpZGVudCAtIGFnZSBub24tcmVzaWRl bnQgZW50cmllcyBhcyBMUlUgYWdlcwogICogQGxydXZlYzogdGhlIGxydXZlYyB0aGF0IHdhcyBh Z2VkCkBAIC0yNjQsMTAgKzM2MCwxNCBAQCB2b2lkICp3b3JraW5nc2V0X2V2aWN0aW9uKHN0cnVj dCBmb2xpbyAqZm9saW8sIHN0cnVjdCBtZW1fY2dyb3VwICp0YXJnZXRfbWVtY2cpCiAJVk1fQlVH X09OX0ZPTElPKGZvbGlvX3JlZl9jb3VudChmb2xpbyksIGZvbGlvKTsKIAlWTV9CVUdfT05fRk9M SU8oIWZvbGlvX3Rlc3RfbG9ja2VkKGZvbGlvKSwgZm9saW8pOwogCisJaWYgKGxydV9nZW5fZW5h YmxlZCgpKQorCQlyZXR1cm4gbHJ1X2dlbl9ldmljdGlvbihmb2xpbyk7CisKIAlscnV2ZWMgPSBt ZW1fY2dyb3VwX2xydXZlYyh0YXJnZXRfbWVtY2csIHBnZGF0KTsKIAkvKiBYWFg6IHRhcmdldF9t ZW1jZyBjYW4gYmUgTlVMTCwgZ28gdGhyb3VnaCBscnV2ZWMgKi8KIAltZW1jZ2lkID0gbWVtX2Nn cm91cF9pZChscnV2ZWNfbWVtY2cobHJ1dmVjKSk7CiAJZXZpY3Rpb24gPSBhdG9taWNfbG9uZ19y ZWFkKCZscnV2ZWMtPm5vbnJlc2lkZW50X2FnZSk7CisJZXZpY3Rpb24gPj49IGJ1Y2tldF9vcmRl cjsKIAl3b3JraW5nc2V0X2FnZV9ub25yZXNpZGVudChscnV2ZWMsIGZvbGlvX25yX3BhZ2VzKGZv bGlvKSk7CiAJcmV0dXJuIHBhY2tfc2hhZG93KG1lbWNnaWQsIHBnZGF0LCBldmljdGlvbiwKIAkJ CQlmb2xpb190ZXN0X3dvcmtpbmdzZXQoZm9saW8pKTsKQEAgLTI5OCw3ICszOTgsMTMgQEAgdm9p ZCB3b3JraW5nc2V0X3JlZmF1bHQoc3RydWN0IGZvbGlvICpmb2xpbywgdm9pZCAqc2hhZG93KQog CWludCBtZW1jZ2lkOwogCWxvbmcgbnI7CiAKKwlpZiAobHJ1X2dlbl9lbmFibGVkKCkpIHsKKwkJ bHJ1X2dlbl9yZWZhdWx0KGZvbGlvLCBzaGFkb3cpOworCQlyZXR1cm47CisJfQorCiAJdW5wYWNr X3NoYWRvdyhzaGFkb3csICZtZW1jZ2lkLCAmcGdkYXQsICZldmljdGlvbiwgJndvcmtpbmdzZXQp OworCWV2aWN0aW9uIDw8PSBidWNrZXRfb3JkZXI7CiAKIAlyY3VfcmVhZF9sb2NrKCk7CiAJLyoK LS0gCjIuMzcuMS41OTUuZzcxOGEzYThmMDQtZ29vZwoKCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0Cmxp bnV4LWFybS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFk Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK