From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E805FECE59E for ; Tue, 15 Oct 2019 18:17:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9FA122245A for ; Tue, 15 Oct 2019 18:17:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=ziepe.ca header.i=@ziepe.ca header.b="kqRSyJAZ" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731784AbfJOSQc (ORCPT ); Tue, 15 Oct 2019 14:16:32 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:44774 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726597AbfJOSQb (ORCPT ); Tue, 15 Oct 2019 14:16:31 -0400 Received: by mail-pg1-f194.google.com with SMTP id e10so8632242pgd.11 for ; Tue, 15 Oct 2019 11:16:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ziepe.ca; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LhsrwTYF5Hs5+xaYqnrF1aDXJkncmUn1XCWn3eOwVlo=; b=kqRSyJAZYW9qV+JxDhuSt7/D+h3Kx+X6hbiT608Rj6KMXtesWZK7tSx6vs13+HijdW 89z4EckwqO1ck2klHSb9KeJq6PJ/PdJTUdgWmH09gVEjv8zM1CvwSh8JMRzUXntKjYJW qqStCcRwIYi2r4tXbYJD++OZz7jWr4lQ0vyM68oT9LY7r/rPSf+6g2BL7PUeQ+DfZ5P7 cJi+RHhkbIG9Sa7xoauRM7eCISLP8vsJsc+8NNRjs2h46ukn2Pg/U5bbSLI5RN23kbvc HKl7IQVQCEPVncP9EVh7h0uJR4C5WGSRgNzCNbDJkJ614FMVQMeDeyNdWoPOrJ9eJlAy bO7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LhsrwTYF5Hs5+xaYqnrF1aDXJkncmUn1XCWn3eOwVlo=; b=Mxrk6Bv21T5NetXnI6JDS+5WCdqkbIh8ITzdLsBERkR5vK4xY57LCl1PoKtAE3pdcS EIFuQLnV16kRSVvRiMM5ZR+F9vM4HqGH0axp9oznd8oJE/74CZrN6wzN2Rv1i5WrWFMP NjY2qnlBbNSgut14QcBOkpHh3PEpBPbF+JHNiPV8sWYZrNUA07xbxO3luG2L8FgEFmVs u8fyHXafwunBv1K8PU6XuPg1x2zmRsywsBUhXKGU7z+CvAaD1r8Q9Ibr6u1Ezq0A7XgY phBmjJLGLsRfo4lirJ6Ty5FDcCZhorBEkjEDEYpoXV4xvr+m087NRqNzoHlFYFe7Ii2Q z0Ag== X-Gm-Message-State: APjAAAVcbTEnCAVSRBorRth8BcIOJMegooi6+9CS4MCwdnO427lAM0TL AMu7PuEtG5U6vcpzJ0+W68XZFA== X-Google-Smtp-Source: APXvYqwTNt9Eonm7aD/OgnNnlRng5e5yjyqmkIorHuTKZRGz+GNBZAM9s/KJsVySlPCxPVzZ7SqvGA== X-Received: by 2002:a62:5ec6:: with SMTP id s189mr39679424pfb.169.1571163388735; Tue, 15 Oct 2019 11:16:28 -0700 (PDT) Received: from ziepe.ca ([24.114.26.129]) by smtp.gmail.com with ESMTPSA id w12sm8438765pfq.138.2019.10.15.11.16.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 15 Oct 2019 11:16:28 -0700 (PDT) Received: from jgg by jggl.ziepe.ca with local (Exim 4.90_1) (envelope-from ) id 1iKRJT-0002Br-9R; Tue, 15 Oct 2019 15:12:51 -0300 From: Jason Gunthorpe To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling@amd.com Cc: linux-rdma@vger.kernel.org, linux-mm@kvack.org, Andrea Arcangeli , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, Ben Skeggs , Jason Gunthorpe , Michal Hocko Subject: [PATCH hmm 02/15] mm/mmu_notifier: add an interval tree notifier Date: Tue, 15 Oct 2019 15:12:29 -0300 Message-Id: <20191015181242.8343-3-jgg@ziepe.ca> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191015181242.8343-1-jgg@ziepe.ca> References: <20191015181242.8343-1-jgg@ziepe.ca> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org From: Jason Gunthorpe Of the 13 users of mmu_notifiers, 8 of them use only invalidate_range_start/end() and immediately intersect the mmu_notifier_range with some kind of internal list of VAs. 4 use an interval tree (i915_gem, radeon_mn, umem_odp, hfi1). 4 use a linked list of some kind (scif_dma, vhost, gntdev, hmm) And the remaining 5 either don't use invalidate_range_start() or do some special thing with it. It turns out that building a correct scheme with an interval tree is pretty complicated, particularly if the use case is synchronizing against another thread doing get_user_pages(). Many of these implementations have various subtle and difficult to fix races. This approach puts the interval tree as common code at the top of the mmu notifier call tree and implements a shareable locking scheme. It includes: - An interval tree tracking VA ranges, with per-range callbacks - A read/write locking scheme for the interval tree that avoids sleeping in the notifier path (for OOM killer) - A sequence counter based collision-retry locking scheme to tell device page fault that a VA range is being concurrently invalidated. This is based on various ideas: - hmm accumulates invalidated VA ranges and releases them when all invalidates are done, via active_invalidate_ranges count. This approach avoids having to intersect the interval tree twice (as umem_odp does) at the potential cost of a longer device page fault. - kvm/umem_odp use a sequence counter to drive the collision retry, via invalidate_seq - a deferred work todo list on unlock scheme like RTNL, via deferred_list. This makes adding/removing interval tree members more deterministic - seqlock, except this version makes the seqlock idea multi-holder on the write side by protecting it with active_invalidate_ranges and a spinlock To minimize MM overhead when only the interval tree is being used, the entire SRCU and hlist overheads are dropped using some simple branches. Similarly the interval tree overhead is dropped when in hlist mode. The overhead from the mandatory spinlock is broadly the same as most of existing users which already had a lock (or two) of some sort on the invalidation path. Cc: Andrea Arcangeli Cc: Michal Hocko Signed-off-by: Jason Gunthorpe --- include/linux/mmu_notifier.h | 78 ++++++ mm/Kconfig | 1 + mm/mmu_notifier.c | 529 +++++++++++++++++++++++++++++++++-- 3 files changed, 583 insertions(+), 25 deletions(-) diff --git a/include/linux/mmu_notifier.h b/include/linux/mmu_notifier.h index 12bd603d318ce7..bc2b12483de127 100644 --- a/include/linux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -6,10 +6,12 @@ #include #include #include +#include struct mmu_notifier_mm; struct mmu_notifier; struct mmu_notifier_range; +struct mmu_range_notifier; /** * enum mmu_notifier_event - reason for the mmu notifier callback @@ -32,6 +34,9 @@ struct mmu_notifier_range; * access flags). User should soft dirty the page in the end callback to make * sure that anyone relying on soft dirtyness catch pages that might be written * through non CPU mappings. + * + * @MMU_NOTIFY_RELEASE: used during mmu_range_notifier invalidate to signal that + * the mm refcount is zero and the range is no longer accessible. */ enum mmu_notifier_event { MMU_NOTIFY_UNMAP = 0, @@ -39,6 +44,7 @@ enum mmu_notifier_event { MMU_NOTIFY_PROTECTION_VMA, MMU_NOTIFY_PROTECTION_PAGE, MMU_NOTIFY_SOFT_DIRTY, + MMU_NOTIFY_RELEASE, }; #define MMU_NOTIFIER_RANGE_BLOCKABLE (1 << 0) @@ -222,6 +228,25 @@ struct mmu_notifier { unsigned int users; }; +/** + * struct mmu_range_notifier_ops + * @invalidate: Upon return the caller must stop using any SPTEs within this + * range, this function can sleep. Return false if blocking was + * required but range is non-blocking + */ +struct mmu_range_notifier_ops { + bool (*invalidate)(struct mmu_range_notifier *mrn, + const struct mmu_notifier_range *range); +}; + +struct mmu_range_notifier { + struct interval_tree_node interval_tree; + const struct mmu_range_notifier_ops *ops; + struct hlist_node deferred_item; + unsigned long invalidate_seq; + struct mm_struct *mm; +}; + #ifdef CONFIG_MMU_NOTIFIER #ifdef CONFIG_LOCKDEP @@ -263,6 +288,59 @@ extern int __mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm); extern void mmu_notifier_unregister(struct mmu_notifier *mn, struct mm_struct *mm); + +unsigned long mmu_range_read_begin(struct mmu_range_notifier *mrn); +int mmu_range_notifier_insert(struct mmu_range_notifier *mrn, + unsigned long start, unsigned long length, + struct mm_struct *mm); +int mmu_range_notifier_insert_locked(struct mmu_range_notifier *mrn, + unsigned long start, unsigned long length, + struct mm_struct *mm); +void mmu_range_notifier_remove(struct mmu_range_notifier *mrn); + +/** + * mmu_range_read_retry - End a read side critical section against a VA range + * mrn: The range under lock + * seq: The return of the paired mmu_range_read_begin() + * + * This MUST be called under a user provided lock that is also held + * unconditionally by op->invalidate(). That lock provides the required SMP + * barrier for handling invalidate_seq. + * + * Each call should be paired with a single mmu_range_read_begin() and + * should be used to conclude the read side. + * + * Returns true if an invalidation collided with this critical section, and + * the caller should retry. + */ +static inline bool mmu_range_read_retry(struct mmu_range_notifier *mrn, + unsigned long seq) +{ + return READ_ONCE(mrn->invalidate_seq) != seq; +} + +/** + * mmu_range_check_retry - Test if a collision has occurred + * mrn: The range under lock + * seq: The return of the matching mmu_range_read_begin() + * + * This can be used in the critical section between mmu_range_read_begin() and + * mmu_range_read_retry(). A return of true indicates an invalidation has + * collided with this lock and a future mmu_range_read_retry() will return + * true. + * + * False is not reliable and only suggests a collision has not happened. It + * can be called many times and does not have to hold the user provided lock. + * + * This call can be used as part of loops and other expensive operations to + * expedite a retry. + */ +static inline bool mmu_range_check_retry(struct mmu_range_notifier *mrn, + unsigned long seq) +{ + return READ_ONCE(mrn->invalidate_seq) != seq; +} + extern void __mmu_notifier_mm_destroy(struct mm_struct *mm); extern void __mmu_notifier_release(struct mm_struct *mm); extern int __mmu_notifier_clear_flush_young(struct mm_struct *mm, diff --git a/mm/Kconfig b/mm/Kconfig index a5dae9a7eb510a..d0b5046d9aeffd 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -284,6 +284,7 @@ config VIRT_TO_BUS config MMU_NOTIFIER bool select SRCU + select INTERVAL_TREE config KSM bool "Enable KSM for page merging" diff --git a/mm/mmu_notifier.c b/mm/mmu_notifier.c index 367670cfd02b7b..5e5e75ebcde4af 100644 --- a/mm/mmu_notifier.c +++ b/mm/mmu_notifier.c @@ -12,6 +12,7 @@ #include #include #include +#include #include #include #include @@ -36,10 +37,243 @@ struct lockdep_map __mmu_notifier_invalidate_range_start_map = { struct mmu_notifier_mm { /* all mmu notifiers registered in this mm are queued in this list */ struct hlist_head list; + bool has_interval; /* to serialize the list modifications and hlist_unhashed */ spinlock_t lock; + unsigned long invalidate_seq; + unsigned long active_invalidate_ranges; + struct rb_root_cached itree; + wait_queue_head_t wq; + struct hlist_head deferred_list; }; +/* + * This is a collision-retry read-side/write-side 'lock', a lot like a + * seqcount, however this allows multiple write-sides to hold it at + * once. Conceptually the write side is protecting the values of the PTEs in + * this mm, such that PTES cannot be read into SPTEs while any writer exists. + * + * Note that the core mm creates nested invalidate_range_start()/end() regions + * within the same thread, and runs invalidate_range_start()/end() in parallel + * on multiple CPUs. This is designed to not reduce concurrency or block + * progress on the mm side. + * + * As a secondary function, holding the full write side also serves to prevent + * writers for the itree, this is an optimization to avoid extra locking + * during invalidate_range_start/end notifiers. + * + * The write side has two states, fully excluded: + * - mm->active_invalidate_ranges != 0 + * - mnn->invalidate_seq & 1 == True + * - some range on the mm_struct is being invalidated + * - the itree is not allowed to change + * + * And partially excluded: + * - mm->active_invalidate_ranges != 0 + * - some range on the mm_struct is being invalidated + * - the itree is allowed to change + * + * The later state avoids some expensive work on inv_end in the common case of + * no mrn monitoring the VA. + */ +static bool mn_itree_is_invalidating(struct mmu_notifier_mm *mmn_mm) +{ + lockdep_assert_held(&mmn_mm->lock); + return mmn_mm->invalidate_seq & 1; +} + +static struct mmu_range_notifier * +mn_itree_inv_start_range(struct mmu_notifier_mm *mmn_mm, + const struct mmu_notifier_range *range, + unsigned long *seq) +{ + struct interval_tree_node *node; + struct mmu_range_notifier *res = NULL; + + spin_lock(&mmn_mm->lock); + mmn_mm->active_invalidate_ranges++; + node = interval_tree_iter_first(&mmn_mm->itree, range->start, + range->end - 1); + if (node) { + mmn_mm->invalidate_seq |= 1; + res = container_of(node, struct mmu_range_notifier, + interval_tree); + } + + *seq = mmn_mm->invalidate_seq; + spin_unlock(&mmn_mm->lock); + return res; +} + +static struct mmu_range_notifier * +mn_itree_inv_next(struct mmu_range_notifier *mrn, + const struct mmu_notifier_range *range) +{ + struct interval_tree_node *node; + + node = interval_tree_iter_next(&mrn->interval_tree, range->start, + range->end - 1); + if (!node) + return NULL; + return container_of(node, struct mmu_range_notifier, interval_tree); +} + +static void mn_itree_inv_end(struct mmu_notifier_mm *mmn_mm) +{ + struct mmu_range_notifier *mrn; + struct hlist_node *next; + bool need_wake = false; + + spin_lock(&mmn_mm->lock); + if (--mmn_mm->active_invalidate_ranges || + !mn_itree_is_invalidating(mmn_mm)) { + spin_unlock(&mmn_mm->lock); + return; + } + + mmn_mm->invalidate_seq++; + need_wake = true; + + /* + * The inv_end incorporates a deferred mechanism like rtnl. Adds and + * removes are queued until the final inv_end happens then they are + * progressed. This arrangement for tree updates is used to avoid + * using a blocking lock during invalidate_range_start. + */ + hlist_for_each_entry_safe(mrn, next, &mmn_mm->deferred_list, + deferred_item) { + if (RB_EMPTY_NODE(&mrn->interval_tree.rb)) + interval_tree_insert(&mrn->interval_tree, + &mmn_mm->itree); + else + interval_tree_remove(&mrn->interval_tree, + &mmn_mm->itree); + hlist_del(&mrn->deferred_item); + } + spin_unlock(&mmn_mm->lock); + + /* + * TODO: Since we already have a spinlock above, this would be faster + * as wake_up_q + */ + if (need_wake) + wake_up_all(&mmn_mm->wq); +} + +/** + * mmu_range_read_begin - Begin a read side critical section against a VA range + * mrn: The range to lock + * + * mmu_range_read_begin()/mmu_range_read_retry() implement a collision-retry + * locking scheme similar to seqcount for the VA range under mrn. If the mm + * invokes invalidation during the critical section then + * mmu_range_read_retry() will return true. + * + * This is useful to obtain shadow PTEs where teardown or setup of the SPTEs + * require a blocking context. The critical region formed by this lock can + * sleep, and the required 'user_lock' can also be a sleeping lock. + * + * The caller is required to provide a 'user_lock' to serialize both teardown + * and setup. + * + * The return value should be passed to mmu_range_read_retry(). + */ +unsigned long mmu_range_read_begin(struct mmu_range_notifier *mrn) +{ + struct mmu_notifier_mm *mmn_mm = mrn->mm->mmu_notifier_mm; + unsigned long seq; + bool is_invalidating; + + /* + * If the mrn has a different seq value under the user_lock than we + * started with then it has collided. + * + * If the mrn currently has the same seq value as the mmn_mm seq, then + * it is currently between invalidate_start/end and is colliding. + * + * The locking looks broadly like this: + * mn_tree_invalidate_start(): mmu_range_read_begin(): + * spin_lock + * seq = READ_ONCE(mrn->invalidate_seq); + * seq == mmn_mm->invalidate_seq + * spin_unlock + * spin_lock + * seq = ++mmn_mm->invalidate_seq + * spin_unlock + * mrn->invalidate_seq = seq + * op->invalidate_range(): + * user_lock + * user_unlock + * + * [Required: mmu_range_read_retry() == true] + * + * mn_itree_inv_end(): + * spin_lock + * seq = ++mmn_mm->invalidate_seq + * spin_unlock + * + * user_lock + * mmu_range_read_retry(): + * READ_ONCE(mrn->invalidate_seq) != seq + * user_unlock + * + * Logically mrn->invalidate_seq is locked under the user provided + * lock, however the write is placed before that lock due to the way + * the API is layered. + * + * Barriers are not needed as any races here are closed by an eventual + * mmu_range_read_retry(), which provides a barrier via the user_lock. + */ + spin_lock(&mmn_mm->lock); + seq = READ_ONCE(mrn->invalidate_seq); + is_invalidating = seq == mmn_mm->invalidate_seq; + spin_unlock(&mmn_mm->lock); + + /* + * mrn->invalidate_seq is always set to an odd value. This ensures + * that if seq does wrap we will always clear the below sleep in some + * reasonable time as mmn_mm->invalidate_seq is even in the idle + * state. + */ + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); + lock_map_release(&__mmu_notifier_invalidate_range_start_map); + if (is_invalidating) + wait_event(mmn_mm->wq, + READ_ONCE(mmn_mm->invalidate_seq) != seq); + + /* + * Notice that mmu_range_read_retry() can already be true at this + * point, avoiding loops here allows the user of this lock to provide + * a global time bound. + */ + + return seq; +} +EXPORT_SYMBOL_GPL(mmu_range_read_begin); + +static void mn_itree_release(struct mmu_notifier_mm *mmn_mm, + struct mm_struct *mm) +{ + struct mmu_notifier_range range = { + .flags = MMU_NOTIFIER_RANGE_BLOCKABLE, + .event = MMU_NOTIFY_RELEASE, + .mm = mm, + .start = 0, + .end = ULONG_MAX, + }; + struct mmu_range_notifier *mrn; + unsigned long cur_seq; + bool ret; + + for (mrn = mn_itree_inv_start_range(mmn_mm, &range, &cur_seq); mrn; + mrn = mn_itree_inv_next(mrn, &range)) { + ret = mrn->ops->invalidate(mrn, &range); + WARN_ON(ret); + } + + mn_itree_inv_end(mmn_mm); +} + /* * This function can't run concurrently against mmu_notifier_register * because mm->mm_users > 0 during mmu_notifier_register and exit_mmap @@ -52,17 +286,24 @@ struct mmu_notifier_mm { * can't go away from under us as exit_mmap holds an mm_count pin * itself. */ -void __mmu_notifier_release(struct mm_struct *mm) +static void mn_hlist_release(struct mmu_notifier_mm *mmn_mm, + struct mm_struct *mm) { struct mmu_notifier *mn; int id; + if (mmn_mm->has_interval) + mn_itree_release(mmn_mm, mm); + + if (hlist_empty(&mmn_mm->list)) + return; + /* * SRCU here will block mmu_notifier_unregister until * ->release returns. */ id = srcu_read_lock(&srcu); - hlist_for_each_entry_rcu(mn, &mm->mmu_notifier_mm->list, hlist) + hlist_for_each_entry_rcu(mn, &mmn_mm->list, hlist) /* * If ->release runs before mmu_notifier_unregister it must be * handled, as it's the only way for the driver to flush all @@ -72,9 +313,9 @@ void __mmu_notifier_release(struct mm_struct *mm) if (mn->ops->release) mn->ops->release(mn, mm); - spin_lock(&mm->mmu_notifier_mm->lock); - while (unlikely(!hlist_empty(&mm->mmu_notifier_mm->list))) { - mn = hlist_entry(mm->mmu_notifier_mm->list.first, + spin_lock(&mmn_mm->lock); + while (unlikely(!hlist_empty(&mmn_mm->list))) { + mn = hlist_entry(mmn_mm->list.first, struct mmu_notifier, hlist); /* @@ -85,7 +326,7 @@ void __mmu_notifier_release(struct mm_struct *mm) */ hlist_del_init_rcu(&mn->hlist); } - spin_unlock(&mm->mmu_notifier_mm->lock); + spin_unlock(&mmn_mm->lock); srcu_read_unlock(&srcu, id); /* @@ -100,6 +341,17 @@ void __mmu_notifier_release(struct mm_struct *mm) synchronize_srcu(&srcu); } +void __mmu_notifier_release(struct mm_struct *mm) +{ + struct mmu_notifier_mm *mmn_mm = mm->mmu_notifier_mm; + + if (mmn_mm->has_interval) + mn_itree_release(mmn_mm, mm); + + if (!hlist_empty(&mmn_mm->list)) + mn_hlist_release(mmn_mm, mm); +} + /* * If no young bitflag is supported by the hardware, ->clear_flush_young can * unmap the address and return 1 or 0 depending if the mapping previously @@ -172,14 +424,41 @@ void __mmu_notifier_change_pte(struct mm_struct *mm, unsigned long address, srcu_read_unlock(&srcu, id); } -int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) +static int mn_itree_invalidate(struct mmu_notifier_mm *mmn_mm, + const struct mmu_notifier_range *range) +{ + struct mmu_range_notifier *mrn; + unsigned long cur_seq; + + for (mrn = mn_itree_inv_start_range(mmn_mm, range, &cur_seq); mrn; + mrn = mn_itree_inv_next(mrn, range)) { + bool ret; + + WRITE_ONCE(mrn->invalidate_seq, cur_seq); + ret = mrn->ops->invalidate(mrn, range); + if (!ret && !WARN_ON(mmu_notifier_range_blockable(range))) + goto out_would_block; + } + return 0; + +out_would_block: + /* + * On -EAGAIN the non-blocking caller is not allowed to call + * invalidate_range_end() + */ + mn_itree_inv_end(mmn_mm); + return -EAGAIN; +} + +static int mn_hlist_invalidate_range_start(struct mmu_notifier_mm *mmn_mm, + struct mmu_notifier_range *range) { struct mmu_notifier *mn; int ret = 0; int id; id = srcu_read_lock(&srcu); - hlist_for_each_entry_rcu(mn, &range->mm->mmu_notifier_mm->list, hlist) { + hlist_for_each_entry_rcu(mn, &mmn_mm->list, hlist) { if (mn->ops->invalidate_range_start) { int _ret; @@ -203,15 +482,30 @@ int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) return ret; } -void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *range, - bool only_end) +int __mmu_notifier_invalidate_range_start(struct mmu_notifier_range *range) +{ + struct mmu_notifier_mm *mmn_mm = range->mm->mmu_notifier_mm; + int ret = 0; + + if (mmn_mm->has_interval) { + ret = mn_itree_invalidate(mmn_mm, range); + if (ret) + return ret; + } + if (!hlist_empty(&mmn_mm->list)) + return mn_hlist_invalidate_range_start(mmn_mm, range); + return 0; +} + +static void mn_hlist_invalidate_end(struct mmu_notifier_mm *mmn_mm, + struct mmu_notifier_range *range, + bool only_end) { struct mmu_notifier *mn; int id; - lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); id = srcu_read_lock(&srcu); - hlist_for_each_entry_rcu(mn, &range->mm->mmu_notifier_mm->list, hlist) { + hlist_for_each_entry_rcu(mn, &mmn_mm->list, hlist) { /* * Call invalidate_range here too to avoid the need for the * subsystem of having to register an invalidate_range_end @@ -238,6 +532,19 @@ void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *range, } } srcu_read_unlock(&srcu, id); +} + +void __mmu_notifier_invalidate_range_end(struct mmu_notifier_range *range, + bool only_end) +{ + struct mmu_notifier_mm *mmn_mm = range->mm->mmu_notifier_mm; + + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); + if (mmn_mm->has_interval) + mn_itree_inv_end(mmn_mm); + + if (!hlist_empty(&mmn_mm->list)) + mn_hlist_invalidate_end(mmn_mm, range, only_end); lock_map_release(&__mmu_notifier_invalidate_range_start_map); } @@ -256,8 +563,9 @@ void __mmu_notifier_invalidate_range(struct mm_struct *mm, } /* - * Same as mmu_notifier_register but here the caller must hold the - * mmap_sem in write mode. + * Same as mmu_notifier_register but here the caller must hold the mmap_sem in + * write mode. A NULL mn signals the notifier is being registered for itree + * mode. */ int __mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) { @@ -274,9 +582,6 @@ int __mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) fs_reclaim_release(GFP_KERNEL); } - mn->mm = mm; - mn->users = 1; - if (!mm->mmu_notifier_mm) { /* * kmalloc cannot be called under mm_take_all_locks(), but we @@ -284,21 +589,22 @@ int __mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) * the write side of the mmap_sem. */ mmu_notifier_mm = - kmalloc(sizeof(struct mmu_notifier_mm), GFP_KERNEL); + kzalloc(sizeof(struct mmu_notifier_mm), GFP_KERNEL); if (!mmu_notifier_mm) return -ENOMEM; INIT_HLIST_HEAD(&mmu_notifier_mm->list); spin_lock_init(&mmu_notifier_mm->lock); + mmu_notifier_mm->invalidate_seq = 2; + mmu_notifier_mm->itree = RB_ROOT_CACHED; + init_waitqueue_head(&mmu_notifier_mm->wq); + INIT_HLIST_HEAD(&mmu_notifier_mm->deferred_list); } ret = mm_take_all_locks(mm); if (unlikely(ret)) goto out_clean; - /* Pairs with the mmdrop in mmu_notifier_unregister_* */ - mmgrab(mm); - /* * Serialize the update against mmu_notifier_unregister. A * side note: mmu_notifier_release can't run concurrently with @@ -306,13 +612,26 @@ int __mmu_notifier_register(struct mmu_notifier *mn, struct mm_struct *mm) * current->mm or explicitly with get_task_mm() or similar). * We can't race against any other mmu notifier method either * thanks to mm_take_all_locks(). + * + * release semantics are provided for users not inside a lock covered + * by mm_take_all_locks(). acquire can only be used while holding the + * mmgrab or mmget, and is safe because once created the + * mmu_notififer_mm is not freed until the mm is destroyed. */ if (mmu_notifier_mm) - mm->mmu_notifier_mm = mmu_notifier_mm; + smp_store_release(&mm->mmu_notifier_mm, mmu_notifier_mm); - spin_lock(&mm->mmu_notifier_mm->lock); - hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_mm->list); - spin_unlock(&mm->mmu_notifier_mm->lock); + if (mn) { + /* Pairs with the mmdrop in mmu_notifier_unregister_* */ + mmgrab(mm); + mn->mm = mm; + mn->users = 1; + + spin_lock(&mm->mmu_notifier_mm->lock); + hlist_add_head_rcu(&mn->hlist, &mm->mmu_notifier_mm->list); + spin_unlock(&mm->mmu_notifier_mm->lock); + } else + mm->mmu_notifier_mm->has_interval = true; mm_drop_all_locks(mm); BUG_ON(atomic_read(&mm->mm_users) <= 0); @@ -529,6 +848,166 @@ void mmu_notifier_put(struct mmu_notifier *mn) } EXPORT_SYMBOL_GPL(mmu_notifier_put); +static int __mmu_range_notifier_insert(struct mmu_range_notifier *mrn, + unsigned long start, + unsigned long length, + struct mmu_notifier_mm *mmn_mm, + struct mm_struct *mm) +{ + mrn->mm = mm; + RB_CLEAR_NODE(&mrn->interval_tree.rb); + mrn->interval_tree.start = start; + /* + * Note that the representation of the intervals in the interval tree + * considers the ending point as contained in the interval. + */ + if (length == 0 || + check_add_overflow(start, length - 1, &mrn->interval_tree.last)) + return -EOVERFLOW; + + /* pairs with mmdrop in mmu_range_notifier_remove() */ + mmgrab(mm); + + /* + * If some invalidate_range_start/end region is going on in parallel + * we don't know what VA ranges are affected, so we must assume this + * new range is included. + * + * If the itree is invalidating then we are not allowed to change + * it. Retrying until invalidation is done is tricky due to the + * possibility for live lock, instead defer the add to the unlock so + * this algorithm is deterministic. + * + * In all cases the value for the mrn->mr_invalidate_seq should be + * odd, see mmu_range_read_begin() + */ + spin_lock(&mmn_mm->lock); + if (mmn_mm->active_invalidate_ranges) { + if (mn_itree_is_invalidating(mmn_mm)) + hlist_add_head(&mrn->deferred_item, + &mmn_mm->deferred_list); + else { + mmn_mm->invalidate_seq |= 1; + interval_tree_insert(&mrn->interval_tree, + &mmn_mm->itree); + } + mrn->invalidate_seq = mmn_mm->invalidate_seq; + } else { + WARN_ON(mn_itree_is_invalidating(mmn_mm)); + mrn->invalidate_seq = mmn_mm->invalidate_seq - 1; + interval_tree_insert(&mrn->interval_tree, &mmn_mm->itree); + } + spin_unlock(&mmn_mm->lock); + return 0; +} + +/** + * mmu_range_notifier_insert - Insert a range notifier + * @mrn: Range notifier to register + * @start: Starting virtual address to monitor + * @length: Length of the range to monitor + * @mm : mm_struct to attach to + * + * This function subscribes the range notifier for notifications from the mm. + * Upon return the ops related to mmu_range_notifier will be called whenever + * an event that intersects with the given range occurs. + * + * Upon return the range_notifier may not be present in the interval tree yet. + * The caller must use the normal range notifier locking flow via + * mmu_range_read_begin() to establish SPTEs for this range. + */ +int mmu_range_notifier_insert(struct mmu_range_notifier *mrn, + unsigned long start, unsigned long length, + struct mm_struct *mm) +{ + struct mmu_notifier_mm *mmn_mm; + int ret; + + might_lock(&mm->mmap_sem); + + mmn_mm = smp_load_acquire(&mm->mmu_notifier_mm); + if (!mmn_mm || !mmn_mm->has_interval) { + ret = mmu_notifier_register(NULL, mm); + if (ret) + return ret; + mmn_mm = mm->mmu_notifier_mm; + } + return __mmu_range_notifier_insert(mrn, start, length, mmn_mm, mm); +} +EXPORT_SYMBOL_GPL(mmu_range_notifier_insert); + +int mmu_range_notifier_insert_locked(struct mmu_range_notifier *mrn, + unsigned long start, unsigned long length, + struct mm_struct *mm) +{ + struct mmu_notifier_mm *mmn_mm; + int ret; + + lockdep_assert_held_write(&mm->mmap_sem); + + mmn_mm = mm->mmu_notifier_mm; + if (!mmn_mm || !mmn_mm->has_interval) { + ret = __mmu_notifier_register(NULL, mm); + if (ret) + return ret; + mmn_mm = mm->mmu_notifier_mm; + } + return __mmu_range_notifier_insert(mrn, start, length, mmn_mm, mm); +} +EXPORT_SYMBOL_GPL(mmu_range_notifier_insert_locked); + +/** + * mmu_range_notifier_remove - Remove a range notifier + * @mrn: Range notifier to unregister + * + * This function must be paired with mmu_range_notifier_insert(). It cannot be + * called from any ops callback. + * + * Once this returns ops callbacks are no longer running on other CPUs and + * will not be called in future. + */ +void mmu_range_notifier_remove(struct mmu_range_notifier *mrn) +{ + struct mm_struct *mm = mrn->mm; + struct mmu_notifier_mm *mmn_mm = mm->mmu_notifier_mm; + unsigned long seq = 0; + + might_sleep(); + + spin_lock(&mmn_mm->lock); + if (mn_itree_is_invalidating(mmn_mm)) { + /* + * remove is being called after insert put this on the + * deferred list, but before the deferred list was processed. + */ + if (RB_EMPTY_NODE(&mrn->interval_tree.rb)) { + hlist_del(&mrn->deferred_item); + } else { + hlist_add_head(&mrn->deferred_item, + &mmn_mm->deferred_list); + seq = mmn_mm->invalidate_seq; + } + } else { + WARN_ON(RB_EMPTY_NODE(&mrn->interval_tree.rb)); + interval_tree_remove(&mrn->interval_tree, &mmn_mm->itree); + } + spin_unlock(&mmn_mm->lock); + + /* + * The possible sleep on progress in the invalidation requires the + * caller not hold any locks held by invalidation callbacks. + */ + lock_map_acquire(&__mmu_notifier_invalidate_range_start_map); + lock_map_release(&__mmu_notifier_invalidate_range_start_map); + if (seq) + wait_event(mmn_mm->wq, + READ_ONCE(mmn_mm->invalidate_seq) != seq); + + /* pairs with mmgrab in mmu_range_notifier_insert() */ + mmdrop(mm); +} +EXPORT_SYMBOL_GPL(mmu_range_notifier_remove); + /** * mmu_notifier_synchronize - Ensure all mmu_notifiers are freed * -- 2.23.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Subject: [PATCH hmm 02/15] mm/mmu_notifier: add an interval tree notifier Date: Tue, 15 Oct 2019 15:12:29 -0300 Message-ID: <20191015181242.8343-3-jgg@ziepe.ca> References: <20191015181242.8343-1-jgg@ziepe.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: In-Reply-To: <20191015181242.8343-1-jgg-uk2M96/98Pc@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: amd-gfx-bounces-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org Sender: "amd-gfx" To: Jerome Glisse , Ralph Campbell , John Hubbard , Felix.Kuehling-5C7GfCeVMHo@public.gmane.org Cc: Andrea Arcangeli , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, amd-gfx-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, Michal Hocko , linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, Jason Gunthorpe , dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW@public.gmane.org, Ben Skeggs List-Id: dri-devel@lists.freedesktop.org RnJvbTogSmFzb24gR3VudGhvcnBlIDxqZ2dAbWVsbGFub3guY29tPgoKT2YgdGhlIDEzIHVzZXJz IG9mIG1tdV9ub3RpZmllcnMsIDggb2YgdGhlbSB1c2Ugb25seQppbnZhbGlkYXRlX3JhbmdlX3N0 YXJ0L2VuZCgpIGFuZCBpbW1lZGlhdGVseSBpbnRlcnNlY3QgdGhlCm1tdV9ub3RpZmllcl9yYW5n ZSB3aXRoIHNvbWUga2luZCBvZiBpbnRlcm5hbCBsaXN0IG9mIFZBcy4gIDQgdXNlIGFuCmludGVy dmFsIHRyZWUgKGk5MTVfZ2VtLCByYWRlb25fbW4sIHVtZW1fb2RwLCBoZmkxKS4gNCB1c2UgYSBs aW5rZWQgbGlzdApvZiBzb21lIGtpbmQgKHNjaWZfZG1hLCB2aG9zdCwgZ250ZGV2LCBobW0pCgpB bmQgdGhlIHJlbWFpbmluZyA1IGVpdGhlciBkb24ndCB1c2UgaW52YWxpZGF0ZV9yYW5nZV9zdGFy dCgpIG9yIGRvIHNvbWUKc3BlY2lhbCB0aGluZyB3aXRoIGl0LgoKSXQgdHVybnMgb3V0IHRoYXQg YnVpbGRpbmcgYSBjb3JyZWN0IHNjaGVtZSB3aXRoIGFuIGludGVydmFsIHRyZWUgaXMKcHJldHR5 IGNvbXBsaWNhdGVkLCBwYXJ0aWN1bGFybHkgaWYgdGhlIHVzZSBjYXNlIGlzIHN5bmNocm9uaXpp bmcgYWdhaW5zdAphbm90aGVyIHRocmVhZCBkb2luZyBnZXRfdXNlcl9wYWdlcygpLiAgTWFueSBv ZiB0aGVzZSBpbXBsZW1lbnRhdGlvbnMgaGF2ZQp2YXJpb3VzIHN1YnRsZSBhbmQgZGlmZmljdWx0 IHRvIGZpeCByYWNlcy4KClRoaXMgYXBwcm9hY2ggcHV0cyB0aGUgaW50ZXJ2YWwgdHJlZSBhcyBj b21tb24gY29kZSBhdCB0aGUgdG9wIG9mIHRoZSBtbXUKbm90aWZpZXIgY2FsbCB0cmVlIGFuZCBp bXBsZW1lbnRzIGEgc2hhcmVhYmxlIGxvY2tpbmcgc2NoZW1lLgoKSXQgaW5jbHVkZXM6CiAtIEFu IGludGVydmFsIHRyZWUgdHJhY2tpbmcgVkEgcmFuZ2VzLCB3aXRoIHBlci1yYW5nZSBjYWxsYmFj a3MKIC0gQSByZWFkL3dyaXRlIGxvY2tpbmcgc2NoZW1lIGZvciB0aGUgaW50ZXJ2YWwgdHJlZSB0 aGF0IGF2b2lkcwogICBzbGVlcGluZyBpbiB0aGUgbm90aWZpZXIgcGF0aCAoZm9yIE9PTSBraWxs ZXIpCiAtIEEgc2VxdWVuY2UgY291bnRlciBiYXNlZCBjb2xsaXNpb24tcmV0cnkgbG9ja2luZyBz Y2hlbWUgdG8gdGVsbAogICBkZXZpY2UgcGFnZSBmYXVsdCB0aGF0IGEgVkEgcmFuZ2UgaXMgYmVp bmcgY29uY3VycmVudGx5IGludmFsaWRhdGVkLgoKVGhpcyBpcyBiYXNlZCBvbiB2YXJpb3VzIGlk ZWFzOgotIGhtbSBhY2N1bXVsYXRlcyBpbnZhbGlkYXRlZCBWQSByYW5nZXMgYW5kIHJlbGVhc2Vz IHRoZW0gd2hlbiBhbGwKICBpbnZhbGlkYXRlcyBhcmUgZG9uZSwgdmlhIGFjdGl2ZV9pbnZhbGlk YXRlX3JhbmdlcyBjb3VudC4KICBUaGlzIGFwcHJvYWNoIGF2b2lkcyBoYXZpbmcgdG8gaW50ZXJz ZWN0IHRoZSBpbnRlcnZhbCB0cmVlIHR3aWNlIChhcwogIHVtZW1fb2RwIGRvZXMpIGF0IHRoZSBw b3RlbnRpYWwgY29zdCBvZiBhIGxvbmdlciBkZXZpY2UgcGFnZSBmYXVsdC4KCi0ga3ZtL3VtZW1f b2RwIHVzZSBhIHNlcXVlbmNlIGNvdW50ZXIgdG8gZHJpdmUgdGhlIGNvbGxpc2lvbiByZXRyeSwK ICB2aWEgaW52YWxpZGF0ZV9zZXEKCi0gYSBkZWZlcnJlZCB3b3JrIHRvZG8gbGlzdCBvbiB1bmxv Y2sgc2NoZW1lIGxpa2UgUlROTCwgdmlhIGRlZmVycmVkX2xpc3QuCiAgVGhpcyBtYWtlcyBhZGRp bmcvcmVtb3ZpbmcgaW50ZXJ2YWwgdHJlZSBtZW1iZXJzIG1vcmUgZGV0ZXJtaW5pc3RpYwoKLSBz ZXFsb2NrLCBleGNlcHQgdGhpcyB2ZXJzaW9uIG1ha2VzIHRoZSBzZXFsb2NrIGlkZWEgbXVsdGkt aG9sZGVyIG9uIHRoZQogIHdyaXRlIHNpZGUgYnkgcHJvdGVjdGluZyBpdCB3aXRoIGFjdGl2ZV9p bnZhbGlkYXRlX3JhbmdlcyBhbmQgYSBzcGlubG9jawoKVG8gbWluaW1pemUgTU0gb3ZlcmhlYWQg d2hlbiBvbmx5IHRoZSBpbnRlcnZhbCB0cmVlIGlzIGJlaW5nIHVzZWQsIHRoZQplbnRpcmUgU1JD VSBhbmQgaGxpc3Qgb3ZlcmhlYWRzIGFyZSBkcm9wcGVkIHVzaW5nIHNvbWUgc2ltcGxlCmJyYW5j aGVzLiBTaW1pbGFybHkgdGhlIGludGVydmFsIHRyZWUgb3ZlcmhlYWQgaXMgZHJvcHBlZCB3aGVu IGluIGhsaXN0Cm1vZGUuCgpUaGUgb3ZlcmhlYWQgZnJvbSB0aGUgbWFuZGF0b3J5IHNwaW5sb2Nr IGlzIGJyb2FkbHkgdGhlIHNhbWUgYXMgbW9zdCBvZgpleGlzdGluZyB1c2VycyB3aGljaCBhbHJl YWR5IGhhZCBhIGxvY2sgKG9yIHR3bykgb2Ygc29tZSBzb3J0IG9uIHRoZQppbnZhbGlkYXRpb24g cGF0aC4KCkNjOiBBbmRyZWEgQXJjYW5nZWxpIDxhYXJjYW5nZUByZWRoYXQuY29tPgpDYzogTWlj aGFsIEhvY2tvIDxtaG9ja29Aa2VybmVsLm9yZz4KU2lnbmVkLW9mZi1ieTogSmFzb24gR3VudGhv cnBlIDxqZ2dAbWVsbGFub3guY29tPgotLS0KIGluY2x1ZGUvbGludXgvbW11X25vdGlmaWVyLmgg fCAgNzggKysrKysrCiBtbS9LY29uZmlnICAgICAgICAgICAgICAgICAgIHwgICAxICsKIG1tL21t dV9ub3RpZmllci5jICAgICAgICAgICAgfCA1MjkgKysrKysrKysrKysrKysrKysrKysrKysrKysr KysrKysrLS0KIDMgZmlsZXMgY2hhbmdlZCwgNTgzIGluc2VydGlvbnMoKyksIDI1IGRlbGV0aW9u cygtKQoKZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvbW11X25vdGlmaWVyLmggYi9pbmNsdWRl L2xpbnV4L21tdV9ub3RpZmllci5oCmluZGV4IDEyYmQ2MDNkMzE4Y2U3Li5iYzJiMTI0ODNkZTEy NyAxMDA2NDQKLS0tIGEvaW5jbHVkZS9saW51eC9tbXVfbm90aWZpZXIuaAorKysgYi9pbmNsdWRl L2xpbnV4L21tdV9ub3RpZmllci5oCkBAIC02LDEwICs2LDEyIEBACiAjaW5jbHVkZSA8bGludXgv c3BpbmxvY2suaD4KICNpbmNsdWRlIDxsaW51eC9tbV90eXBlcy5oPgogI2luY2x1ZGUgPGxpbnV4 L3NyY3UuaD4KKyNpbmNsdWRlIDxsaW51eC9pbnRlcnZhbF90cmVlLmg+CiAKIHN0cnVjdCBtbXVf bm90aWZpZXJfbW07CiBzdHJ1Y3QgbW11X25vdGlmaWVyOwogc3RydWN0IG1tdV9ub3RpZmllcl9y YW5nZTsKK3N0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXI7CiAKIC8qKgogICogZW51bSBtbXVfbm90 aWZpZXJfZXZlbnQgLSByZWFzb24gZm9yIHRoZSBtbXUgbm90aWZpZXIgY2FsbGJhY2sKQEAgLTMy LDYgKzM0LDkgQEAgc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZTsKICAqIGFjY2VzcyBmbGFncyku IFVzZXIgc2hvdWxkIHNvZnQgZGlydHkgdGhlIHBhZ2UgaW4gdGhlIGVuZCBjYWxsYmFjayB0byBt YWtlCiAgKiBzdXJlIHRoYXQgYW55b25lIHJlbHlpbmcgb24gc29mdCBkaXJ0eW5lc3MgY2F0Y2gg cGFnZXMgdGhhdCBtaWdodCBiZSB3cml0dGVuCiAgKiB0aHJvdWdoIG5vbiBDUFUgbWFwcGluZ3Mu CisgKgorICogQE1NVV9OT1RJRllfUkVMRUFTRTogdXNlZCBkdXJpbmcgbW11X3JhbmdlX25vdGlm aWVyIGludmFsaWRhdGUgdG8gc2lnbmFsIHRoYXQKKyAqIHRoZSBtbSByZWZjb3VudCBpcyB6ZXJv IGFuZCB0aGUgcmFuZ2UgaXMgbm8gbG9uZ2VyIGFjY2Vzc2libGUuCiAgKi8KIGVudW0gbW11X25v dGlmaWVyX2V2ZW50IHsKIAlNTVVfTk9USUZZX1VOTUFQID0gMCwKQEAgLTM5LDYgKzQ0LDcgQEAg ZW51bSBtbXVfbm90aWZpZXJfZXZlbnQgewogCU1NVV9OT1RJRllfUFJPVEVDVElPTl9WTUEsCiAJ TU1VX05PVElGWV9QUk9URUNUSU9OX1BBR0UsCiAJTU1VX05PVElGWV9TT0ZUX0RJUlRZLAorCU1N VV9OT1RJRllfUkVMRUFTRSwKIH07CiAKICNkZWZpbmUgTU1VX05PVElGSUVSX1JBTkdFX0JMT0NL QUJMRSAoMSA8PCAwKQpAQCAtMjIyLDYgKzIyOCwyNSBAQCBzdHJ1Y3QgbW11X25vdGlmaWVyIHsK IAl1bnNpZ25lZCBpbnQgdXNlcnM7CiB9OwogCisvKioKKyAqIHN0cnVjdCBtbXVfcmFuZ2Vfbm90 aWZpZXJfb3BzCisgKiBAaW52YWxpZGF0ZTogVXBvbiByZXR1cm4gdGhlIGNhbGxlciBtdXN0IHN0 b3AgdXNpbmcgYW55IFNQVEVzIHdpdGhpbiB0aGlzCisgKiAgICAgICAgICAgICAgcmFuZ2UsIHRo aXMgZnVuY3Rpb24gY2FuIHNsZWVwLiBSZXR1cm4gZmFsc2UgaWYgYmxvY2tpbmcgd2FzCisgKiAg ICAgICAgICAgICAgcmVxdWlyZWQgYnV0IHJhbmdlIGlzIG5vbi1ibG9ja2luZworICovCitzdHJ1 Y3QgbW11X3JhbmdlX25vdGlmaWVyX29wcyB7CisJYm9vbCAoKmludmFsaWRhdGUpKHN0cnVjdCBt bXVfcmFuZ2Vfbm90aWZpZXIgKm1ybiwKKwkJCSAgIGNvbnN0IHN0cnVjdCBtbXVfbm90aWZpZXJf cmFuZ2UgKnJhbmdlKTsKK307CisKK3N0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgeworCXN0cnVj dCBpbnRlcnZhbF90cmVlX25vZGUgaW50ZXJ2YWxfdHJlZTsKKwljb25zdCBzdHJ1Y3QgbW11X3Jh bmdlX25vdGlmaWVyX29wcyAqb3BzOworCXN0cnVjdCBobGlzdF9ub2RlIGRlZmVycmVkX2l0ZW07 CisJdW5zaWduZWQgbG9uZyBpbnZhbGlkYXRlX3NlcTsKKwlzdHJ1Y3QgbW1fc3RydWN0ICptbTsK K307CisKICNpZmRlZiBDT05GSUdfTU1VX05PVElGSUVSCiAKICNpZmRlZiBDT05GSUdfTE9DS0RF UApAQCAtMjYzLDYgKzI4OCw1OSBAQCBleHRlcm4gaW50IF9fbW11X25vdGlmaWVyX3JlZ2lzdGVy KHN0cnVjdCBtbXVfbm90aWZpZXIgKm1uLAogCQkJCSAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKTsK IGV4dGVybiB2b2lkIG1tdV9ub3RpZmllcl91bnJlZ2lzdGVyKHN0cnVjdCBtbXVfbm90aWZpZXIg Km1uLAogCQkJCSAgICBzdHJ1Y3QgbW1fc3RydWN0ICptbSk7CisKK3Vuc2lnbmVkIGxvbmcgbW11 X3JhbmdlX3JlYWRfYmVnaW4oc3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciAqbXJuKTsKK2ludCBt bXVfcmFuZ2Vfbm90aWZpZXJfaW5zZXJ0KHN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybiwK KwkJCSAgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLAorCQkJ ICAgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0pOworaW50IG1tdV9yYW5nZV9ub3RpZmllcl9pbnNl cnRfbG9ja2VkKHN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybiwKKwkJCQkgICAgIHVuc2ln bmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLAorCQkJCSAgICAgc3RydWN0IG1t X3N0cnVjdCAqbW0pOwordm9pZCBtbXVfcmFuZ2Vfbm90aWZpZXJfcmVtb3ZlKHN0cnVjdCBtbXVf cmFuZ2Vfbm90aWZpZXIgKm1ybik7CisKKy8qKgorICogbW11X3JhbmdlX3JlYWRfcmV0cnkgLSBF bmQgYSByZWFkIHNpZGUgY3JpdGljYWwgc2VjdGlvbiBhZ2FpbnN0IGEgVkEgcmFuZ2UKKyAqIG1y bjogVGhlIHJhbmdlIHVuZGVyIGxvY2sKKyAqIHNlcTogVGhlIHJldHVybiBvZiB0aGUgcGFpcmVk IG1tdV9yYW5nZV9yZWFkX2JlZ2luKCkKKyAqCisgKiBUaGlzIE1VU1QgYmUgY2FsbGVkIHVuZGVy IGEgdXNlciBwcm92aWRlZCBsb2NrIHRoYXQgaXMgYWxzbyBoZWxkCisgKiB1bmNvbmRpdGlvbmFs bHkgYnkgb3AtPmludmFsaWRhdGUoKS4gVGhhdCBsb2NrIHByb3ZpZGVzIHRoZSByZXF1aXJlZCBT TVAKKyAqIGJhcnJpZXIgZm9yIGhhbmRsaW5nIGludmFsaWRhdGVfc2VxLgorICoKKyAqIEVhY2gg Y2FsbCBzaG91bGQgYmUgcGFpcmVkIHdpdGggYSBzaW5nbGUgbW11X3JhbmdlX3JlYWRfYmVnaW4o KSBhbmQKKyAqIHNob3VsZCBiZSB1c2VkIHRvIGNvbmNsdWRlIHRoZSByZWFkIHNpZGUuCisgKgor ICogUmV0dXJucyB0cnVlIGlmIGFuIGludmFsaWRhdGlvbiBjb2xsaWRlZCB3aXRoIHRoaXMgY3Jp dGljYWwgc2VjdGlvbiwgYW5kCisgKiB0aGUgY2FsbGVyIHNob3VsZCByZXRyeS4KKyAqLworc3Rh dGljIGlubGluZSBib29sIG1tdV9yYW5nZV9yZWFkX3JldHJ5KHN0cnVjdCBtbXVfcmFuZ2Vfbm90 aWZpZXIgKm1ybiwKKwkJCQkJdW5zaWduZWQgbG9uZyBzZXEpCit7CisJcmV0dXJuIFJFQURfT05D RShtcm4tPmludmFsaWRhdGVfc2VxKSAhPSBzZXE7Cit9CisKKy8qKgorICogbW11X3JhbmdlX2No ZWNrX3JldHJ5IC0gVGVzdCBpZiBhIGNvbGxpc2lvbiBoYXMgb2NjdXJyZWQKKyAqIG1ybjogVGhl IHJhbmdlIHVuZGVyIGxvY2sKKyAqIHNlcTogVGhlIHJldHVybiBvZiB0aGUgbWF0Y2hpbmcgbW11 X3JhbmdlX3JlYWRfYmVnaW4oKQorICoKKyAqIFRoaXMgY2FuIGJlIHVzZWQgaW4gdGhlIGNyaXRp Y2FsIHNlY3Rpb24gYmV0d2VlbiBtbXVfcmFuZ2VfcmVhZF9iZWdpbigpIGFuZAorICogbW11X3Jh bmdlX3JlYWRfcmV0cnkoKS4gIEEgcmV0dXJuIG9mIHRydWUgaW5kaWNhdGVzIGFuIGludmFsaWRh dGlvbiBoYXMKKyAqIGNvbGxpZGVkIHdpdGggdGhpcyBsb2NrIGFuZCBhIGZ1dHVyZSBtbXVfcmFu Z2VfcmVhZF9yZXRyeSgpIHdpbGwgcmV0dXJuCisgKiB0cnVlLgorICoKKyAqIEZhbHNlIGlzIG5v dCByZWxpYWJsZSBhbmQgb25seSBzdWdnZXN0cyBhIGNvbGxpc2lvbiBoYXMgbm90IGhhcHBlbmVk LiBJdAorICogY2FuIGJlIGNhbGxlZCBtYW55IHRpbWVzIGFuZCBkb2VzIG5vdCBoYXZlIHRvIGhv bGQgdGhlIHVzZXIgcHJvdmlkZWQgbG9jay4KKyAqCisgKiBUaGlzIGNhbGwgY2FuIGJlIHVzZWQg YXMgcGFydCBvZiBsb29wcyBhbmQgb3RoZXIgZXhwZW5zaXZlIG9wZXJhdGlvbnMgdG8KKyAqIGV4 cGVkaXRlIGEgcmV0cnkuCisgKi8KK3N0YXRpYyBpbmxpbmUgYm9vbCBtbXVfcmFuZ2VfY2hlY2tf cmV0cnkoc3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciAqbXJuLAorCQkJCQkgdW5zaWduZWQgbG9u ZyBzZXEpCit7CisJcmV0dXJuIFJFQURfT05DRShtcm4tPmludmFsaWRhdGVfc2VxKSAhPSBzZXE7 Cit9CisKIGV4dGVybiB2b2lkIF9fbW11X25vdGlmaWVyX21tX2Rlc3Ryb3koc3RydWN0IG1tX3N0 cnVjdCAqbW0pOwogZXh0ZXJuIHZvaWQgX19tbXVfbm90aWZpZXJfcmVsZWFzZShzdHJ1Y3QgbW1f c3RydWN0ICptbSk7CiBleHRlcm4gaW50IF9fbW11X25vdGlmaWVyX2NsZWFyX2ZsdXNoX3lvdW5n KHN0cnVjdCBtbV9zdHJ1Y3QgKm1tLApkaWZmIC0tZ2l0IGEvbW0vS2NvbmZpZyBiL21tL0tjb25m aWcKaW5kZXggYTVkYWU5YTdlYjUxMGEuLmQwYjUwNDZkOWFlZmZkIDEwMDY0NAotLS0gYS9tbS9L Y29uZmlnCisrKyBiL21tL0tjb25maWcKQEAgLTI4NCw2ICsyODQsNyBAQCBjb25maWcgVklSVF9U T19CVVMKIGNvbmZpZyBNTVVfTk9USUZJRVIKIAlib29sCiAJc2VsZWN0IFNSQ1UKKwlzZWxlY3Qg SU5URVJWQUxfVFJFRQogCiBjb25maWcgS1NNCiAJYm9vbCAiRW5hYmxlIEtTTSBmb3IgcGFnZSBt ZXJnaW5nIgpkaWZmIC0tZ2l0IGEvbW0vbW11X25vdGlmaWVyLmMgYi9tbS9tbXVfbm90aWZpZXIu YwppbmRleCAzNjc2NzBjZmQwMmI3Yi4uNWU1ZTc1ZWJjZGU0YWYgMTAwNjQ0Ci0tLSBhL21tL21t dV9ub3RpZmllci5jCisrKyBiL21tL21tdV9ub3RpZmllci5jCkBAIC0xMiw2ICsxMiw3IEBACiAj aW5jbHVkZSA8bGludXgvZXhwb3J0Lmg+CiAjaW5jbHVkZSA8bGludXgvbW0uaD4KICNpbmNsdWRl IDxsaW51eC9lcnIuaD4KKyNpbmNsdWRlIDxsaW51eC9pbnRlcnZhbF90cmVlLmg+CiAjaW5jbHVk ZSA8bGludXgvc3JjdS5oPgogI2luY2x1ZGUgPGxpbnV4L3JjdXBkYXRlLmg+CiAjaW5jbHVkZSA8 bGludXgvc2NoZWQuaD4KQEAgLTM2LDEwICszNywyNDMgQEAgc3RydWN0IGxvY2tkZXBfbWFwIF9f bW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbWFwID0gewogc3RydWN0IG1tdV9u b3RpZmllcl9tbSB7CiAJLyogYWxsIG1tdSBub3RpZmllcnMgcmVnaXN0ZXJlZCBpbiB0aGlzIG1t IGFyZSBxdWV1ZWQgaW4gdGhpcyBsaXN0ICovCiAJc3RydWN0IGhsaXN0X2hlYWQgbGlzdDsKKwli b29sIGhhc19pbnRlcnZhbDsKIAkvKiB0byBzZXJpYWxpemUgdGhlIGxpc3QgbW9kaWZpY2F0aW9u cyBhbmQgaGxpc3RfdW5oYXNoZWQgKi8KIAlzcGlubG9ja190IGxvY2s7CisJdW5zaWduZWQgbG9u ZyBpbnZhbGlkYXRlX3NlcTsKKwl1bnNpZ25lZCBsb25nIGFjdGl2ZV9pbnZhbGlkYXRlX3Jhbmdl czsKKwlzdHJ1Y3QgcmJfcm9vdF9jYWNoZWQgaXRyZWU7CisJd2FpdF9xdWV1ZV9oZWFkX3Qgd3E7 CisJc3RydWN0IGhsaXN0X2hlYWQgZGVmZXJyZWRfbGlzdDsKIH07CiAKKy8qCisgKiBUaGlzIGlz IGEgY29sbGlzaW9uLXJldHJ5IHJlYWQtc2lkZS93cml0ZS1zaWRlICdsb2NrJywgYSBsb3QgbGlr ZSBhCisgKiBzZXFjb3VudCwgaG93ZXZlciB0aGlzIGFsbG93cyBtdWx0aXBsZSB3cml0ZS1zaWRl cyB0byBob2xkIGl0IGF0CisgKiBvbmNlLiBDb25jZXB0dWFsbHkgdGhlIHdyaXRlIHNpZGUgaXMg cHJvdGVjdGluZyB0aGUgdmFsdWVzIG9mIHRoZSBQVEVzIGluCisgKiB0aGlzIG1tLCBzdWNoIHRo YXQgUFRFUyBjYW5ub3QgYmUgcmVhZCBpbnRvIFNQVEVzIHdoaWxlIGFueSB3cml0ZXIgZXhpc3Rz LgorICoKKyAqIE5vdGUgdGhhdCB0aGUgY29yZSBtbSBjcmVhdGVzIG5lc3RlZCBpbnZhbGlkYXRl X3JhbmdlX3N0YXJ0KCkvZW5kKCkgcmVnaW9ucworICogd2l0aGluIHRoZSBzYW1lIHRocmVhZCwg YW5kIHJ1bnMgaW52YWxpZGF0ZV9yYW5nZV9zdGFydCgpL2VuZCgpIGluIHBhcmFsbGVsCisgKiBv biBtdWx0aXBsZSBDUFVzLiBUaGlzIGlzIGRlc2lnbmVkIHRvIG5vdCByZWR1Y2UgY29uY3VycmVu Y3kgb3IgYmxvY2sKKyAqIHByb2dyZXNzIG9uIHRoZSBtbSBzaWRlLgorICoKKyAqIEFzIGEgc2Vj b25kYXJ5IGZ1bmN0aW9uLCBob2xkaW5nIHRoZSBmdWxsIHdyaXRlIHNpZGUgYWxzbyBzZXJ2ZXMg dG8gcHJldmVudAorICogd3JpdGVycyBmb3IgdGhlIGl0cmVlLCB0aGlzIGlzIGFuIG9wdGltaXph dGlvbiB0byBhdm9pZCBleHRyYSBsb2NraW5nCisgKiBkdXJpbmcgaW52YWxpZGF0ZV9yYW5nZV9z dGFydC9lbmQgbm90aWZpZXJzLgorICoKKyAqIFRoZSB3cml0ZSBzaWRlIGhhcyB0d28gc3RhdGVz LCBmdWxseSBleGNsdWRlZDoKKyAqICAtIG1tLT5hY3RpdmVfaW52YWxpZGF0ZV9yYW5nZXMgIT0g MAorICogIC0gbW5uLT5pbnZhbGlkYXRlX3NlcSAmIDEgPT0gVHJ1ZQorICogIC0gc29tZSByYW5n ZSBvbiB0aGUgbW1fc3RydWN0IGlzIGJlaW5nIGludmFsaWRhdGVkCisgKiAgLSB0aGUgaXRyZWUg aXMgbm90IGFsbG93ZWQgdG8gY2hhbmdlCisgKgorICogQW5kIHBhcnRpYWxseSBleGNsdWRlZDoK KyAqICAtIG1tLT5hY3RpdmVfaW52YWxpZGF0ZV9yYW5nZXMgIT0gMAorICogIC0gc29tZSByYW5n ZSBvbiB0aGUgbW1fc3RydWN0IGlzIGJlaW5nIGludmFsaWRhdGVkCisgKiAgLSB0aGUgaXRyZWUg aXMgYWxsb3dlZCB0byBjaGFuZ2UKKyAqCisgKiBUaGUgbGF0ZXIgc3RhdGUgYXZvaWRzIHNvbWUg ZXhwZW5zaXZlIHdvcmsgb24gaW52X2VuZCBpbiB0aGUgY29tbW9uIGNhc2Ugb2YKKyAqIG5vIG1y biBtb25pdG9yaW5nIHRoZSBWQS4KKyAqLworc3RhdGljIGJvb2wgbW5faXRyZWVfaXNfaW52YWxp ZGF0aW5nKHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSkKK3sKKwlsb2NrZGVwX2Fzc2Vy dF9oZWxkKCZtbW5fbW0tPmxvY2spOworCXJldHVybiBtbW5fbW0tPmludmFsaWRhdGVfc2VxICYg MTsKK30KKworc3RhdGljIHN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgKgorbW5faXRyZWVfaW52 X3N0YXJ0X3JhbmdlKHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSwKKwkJCSBjb25zdCBz dHJ1Y3QgbW11X25vdGlmaWVyX3JhbmdlICpyYW5nZSwKKwkJCSB1bnNpZ25lZCBsb25nICpzZXEp Cit7CisJc3RydWN0IGludGVydmFsX3RyZWVfbm9kZSAqbm9kZTsKKwlzdHJ1Y3QgbW11X3Jhbmdl X25vdGlmaWVyICpyZXMgPSBOVUxMOworCisJc3Bpbl9sb2NrKCZtbW5fbW0tPmxvY2spOworCW1t bl9tbS0+YWN0aXZlX2ludmFsaWRhdGVfcmFuZ2VzKys7CisJbm9kZSA9IGludGVydmFsX3RyZWVf aXRlcl9maXJzdCgmbW1uX21tLT5pdHJlZSwgcmFuZ2UtPnN0YXJ0LAorCQkJCQlyYW5nZS0+ZW5k IC0gMSk7CisJaWYgKG5vZGUpIHsKKwkJbW1uX21tLT5pbnZhbGlkYXRlX3NlcSB8PSAxOworCQly ZXMgPSBjb250YWluZXJfb2Yobm9kZSwgc3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciwKKwkJCQkg ICBpbnRlcnZhbF90cmVlKTsKKwl9CisKKwkqc2VxID0gbW1uX21tLT5pbnZhbGlkYXRlX3NlcTsK KwlzcGluX3VubG9jaygmbW1uX21tLT5sb2NrKTsKKwlyZXR1cm4gcmVzOworfQorCitzdGF0aWMg c3RydWN0IG1tdV9yYW5nZV9ub3RpZmllciAqCittbl9pdHJlZV9pbnZfbmV4dChzdHJ1Y3QgbW11 X3JhbmdlX25vdGlmaWVyICptcm4sCisJCSAgY29uc3Qgc3RydWN0IG1tdV9ub3RpZmllcl9yYW5n ZSAqcmFuZ2UpCit7CisJc3RydWN0IGludGVydmFsX3RyZWVfbm9kZSAqbm9kZTsKKworCW5vZGUg PSBpbnRlcnZhbF90cmVlX2l0ZXJfbmV4dCgmbXJuLT5pbnRlcnZhbF90cmVlLCByYW5nZS0+c3Rh cnQsCisJCQkJICAgICAgIHJhbmdlLT5lbmQgLSAxKTsKKwlpZiAoIW5vZGUpCisJCXJldHVybiBO VUxMOworCXJldHVybiBjb250YWluZXJfb2Yobm9kZSwgc3RydWN0IG1tdV9yYW5nZV9ub3RpZmll ciwgaW50ZXJ2YWxfdHJlZSk7Cit9CisKK3N0YXRpYyB2b2lkIG1uX2l0cmVlX2ludl9lbmQoc3Ry dWN0IG1tdV9ub3RpZmllcl9tbSAqbW1uX21tKQoreworCXN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZp ZXIgKm1ybjsKKwlzdHJ1Y3QgaGxpc3Rfbm9kZSAqbmV4dDsKKwlib29sIG5lZWRfd2FrZSA9IGZh bHNlOworCisJc3Bpbl9sb2NrKCZtbW5fbW0tPmxvY2spOworCWlmICgtLW1tbl9tbS0+YWN0aXZl X2ludmFsaWRhdGVfcmFuZ2VzIHx8CisJICAgICFtbl9pdHJlZV9pc19pbnZhbGlkYXRpbmcobW1u X21tKSkgeworCQlzcGluX3VubG9jaygmbW1uX21tLT5sb2NrKTsKKwkJcmV0dXJuOworCX0KKwor CW1tbl9tbS0+aW52YWxpZGF0ZV9zZXErKzsKKwluZWVkX3dha2UgPSB0cnVlOworCisJLyoKKwkg KiBUaGUgaW52X2VuZCBpbmNvcnBvcmF0ZXMgYSBkZWZlcnJlZCBtZWNoYW5pc20gbGlrZSBydG5s LiBBZGRzIGFuZAorCSAqIHJlbW92ZXMgYXJlIHF1ZXVlZCB1bnRpbCB0aGUgZmluYWwgaW52X2Vu ZCBoYXBwZW5zIHRoZW4gdGhleSBhcmUKKwkgKiBwcm9ncmVzc2VkLiBUaGlzIGFycmFuZ2VtZW50 IGZvciB0cmVlIHVwZGF0ZXMgaXMgdXNlZCB0byBhdm9pZAorCSAqIHVzaW5nIGEgYmxvY2tpbmcg bG9jayBkdXJpbmcgaW52YWxpZGF0ZV9yYW5nZV9zdGFydC4KKwkgKi8KKwlobGlzdF9mb3JfZWFj aF9lbnRyeV9zYWZlKG1ybiwgbmV4dCwgJm1tbl9tbS0+ZGVmZXJyZWRfbGlzdCwKKwkJCQkgIGRl ZmVycmVkX2l0ZW0pIHsKKwkJaWYgKFJCX0VNUFRZX05PREUoJm1ybi0+aW50ZXJ2YWxfdHJlZS5y YikpCisJCQlpbnRlcnZhbF90cmVlX2luc2VydCgmbXJuLT5pbnRlcnZhbF90cmVlLAorCQkJCQkg ICAgICZtbW5fbW0tPml0cmVlKTsKKwkJZWxzZQorCQkJaW50ZXJ2YWxfdHJlZV9yZW1vdmUoJm1y bi0+aW50ZXJ2YWxfdHJlZSwKKwkJCQkJICAgICAmbW1uX21tLT5pdHJlZSk7CisJCWhsaXN0X2Rl bCgmbXJuLT5kZWZlcnJlZF9pdGVtKTsKKwl9CisJc3Bpbl91bmxvY2soJm1tbl9tbS0+bG9jayk7 CisKKwkvKgorCSAqIFRPRE86IFNpbmNlIHdlIGFscmVhZHkgaGF2ZSBhIHNwaW5sb2NrIGFib3Zl LCB0aGlzIHdvdWxkIGJlIGZhc3RlcgorCSAqIGFzIHdha2VfdXBfcQorCSAqLworCWlmIChuZWVk X3dha2UpCisJCXdha2VfdXBfYWxsKCZtbW5fbW0tPndxKTsKK30KKworLyoqCisgKiBtbXVfcmFu Z2VfcmVhZF9iZWdpbiAtIEJlZ2luIGEgcmVhZCBzaWRlIGNyaXRpY2FsIHNlY3Rpb24gYWdhaW5z dCBhIFZBIHJhbmdlCisgKiBtcm46IFRoZSByYW5nZSB0byBsb2NrCisgKgorICogbW11X3Jhbmdl X3JlYWRfYmVnaW4oKS9tbXVfcmFuZ2VfcmVhZF9yZXRyeSgpIGltcGxlbWVudCBhIGNvbGxpc2lv bi1yZXRyeQorICogbG9ja2luZyBzY2hlbWUgc2ltaWxhciB0byBzZXFjb3VudCBmb3IgdGhlIFZB IHJhbmdlIHVuZGVyIG1ybi4gSWYgdGhlIG1tCisgKiBpbnZva2VzIGludmFsaWRhdGlvbiBkdXJp bmcgdGhlIGNyaXRpY2FsIHNlY3Rpb24gdGhlbgorICogbW11X3JhbmdlX3JlYWRfcmV0cnkoKSB3 aWxsIHJldHVybiB0cnVlLgorICoKKyAqIFRoaXMgaXMgdXNlZnVsIHRvIG9idGFpbiBzaGFkb3cg UFRFcyB3aGVyZSB0ZWFyZG93biBvciBzZXR1cCBvZiB0aGUgU1BURXMKKyAqIHJlcXVpcmUgYSBi bG9ja2luZyBjb250ZXh0LiAgVGhlIGNyaXRpY2FsIHJlZ2lvbiBmb3JtZWQgYnkgdGhpcyBsb2Nr IGNhbgorICogc2xlZXAsIGFuZCB0aGUgcmVxdWlyZWQgJ3VzZXJfbG9jaycgY2FuIGFsc28gYmUg YSBzbGVlcGluZyBsb2NrLgorICoKKyAqIFRoZSBjYWxsZXIgaXMgcmVxdWlyZWQgdG8gcHJvdmlk ZSBhICd1c2VyX2xvY2snIHRvIHNlcmlhbGl6ZSBib3RoIHRlYXJkb3duCisgKiBhbmQgc2V0dXAu CisgKgorICogVGhlIHJldHVybiB2YWx1ZSBzaG91bGQgYmUgcGFzc2VkIHRvIG1tdV9yYW5nZV9y ZWFkX3JldHJ5KCkuCisgKi8KK3Vuc2lnbmVkIGxvbmcgbW11X3JhbmdlX3JlYWRfYmVnaW4oc3Ry dWN0IG1tdV9yYW5nZV9ub3RpZmllciAqbXJuKQoreworCXN0cnVjdCBtbXVfbm90aWZpZXJfbW0g Km1tbl9tbSA9IG1ybi0+bW0tPm1tdV9ub3RpZmllcl9tbTsKKwl1bnNpZ25lZCBsb25nIHNlcTsK Kwlib29sIGlzX2ludmFsaWRhdGluZzsKKworCS8qCisJICogSWYgdGhlIG1ybiBoYXMgYSBkaWZm ZXJlbnQgc2VxIHZhbHVlIHVuZGVyIHRoZSB1c2VyX2xvY2sgdGhhbiB3ZQorCSAqIHN0YXJ0ZWQg d2l0aCB0aGVuIGl0IGhhcyBjb2xsaWRlZC4KKwkgKgorCSAqIElmIHRoZSBtcm4gY3VycmVudGx5 IGhhcyB0aGUgc2FtZSBzZXEgdmFsdWUgYXMgdGhlIG1tbl9tbSBzZXEsIHRoZW4KKwkgKiBpdCBp cyBjdXJyZW50bHkgYmV0d2VlbiBpbnZhbGlkYXRlX3N0YXJ0L2VuZCBhbmQgaXMgY29sbGlkaW5n LgorCSAqCisJICogVGhlIGxvY2tpbmcgbG9va3MgYnJvYWRseSBsaWtlIHRoaXM6CisJICogICBt bl90cmVlX2ludmFsaWRhdGVfc3RhcnQoKTogICAgICAgICAgbW11X3JhbmdlX3JlYWRfYmVnaW4o KToKKwkgKiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgc3Bpbl9sb2Nr CisJICogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzZXEgPSBSRUFE X09OQ0UobXJuLT5pbnZhbGlkYXRlX3NlcSk7CisJICogICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICBzZXEgPT0gbW1uX21tLT5pbnZhbGlkYXRlX3NlcQorCSAqICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBzcGluX3VubG9jaworCSAqICAgIHNw aW5fbG9jaworCSAqICAgICBzZXEgPSArK21tbl9tbS0+aW52YWxpZGF0ZV9zZXEKKwkgKiAgICBz cGluX3VubG9jaworCSAqICAgIG1ybi0+aW52YWxpZGF0ZV9zZXEgPSBzZXEKKwkgKiAgICAgb3At PmludmFsaWRhdGVfcmFuZ2UoKToKKwkgKiAgICAgICB1c2VyX2xvY2sKKwkgKiAgICAgICB1c2Vy X3VubG9jaworCSAqCisJICogICAgICAgICAgICAgICAgICAgICAgICAgIFtSZXF1aXJlZDogbW11 X3JhbmdlX3JlYWRfcmV0cnkoKSA9PSB0cnVlXQorCSAqCisJICogICBtbl9pdHJlZV9pbnZfZW5k KCk6CisJICogICAgc3Bpbl9sb2NrCisJICogICAgIHNlcSA9ICsrbW1uX21tLT5pbnZhbGlkYXRl X3NlcQorCSAqICAgIHNwaW5fdW5sb2NrCisJICoKKwkgKiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICB1c2VyX2xvY2sKKwkgKiAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgbW11X3JhbmdlX3JlYWRfcmV0cnkoKToKKwkgKiAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgICAgICAgICAgICAgIFJFQURfT05DRShtcm4tPmludmFsaWRhdGVfc2Vx KSAhPSBzZXEKKwkgKiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICB1c2Vy X3VubG9jaworCSAqCisJICogTG9naWNhbGx5IG1ybi0+aW52YWxpZGF0ZV9zZXEgaXMgbG9ja2Vk IHVuZGVyIHRoZSB1c2VyIHByb3ZpZGVkCisJICogbG9jaywgaG93ZXZlciB0aGUgd3JpdGUgaXMg cGxhY2VkIGJlZm9yZSB0aGF0IGxvY2sgZHVlIHRvIHRoZSB3YXkKKwkgKiB0aGUgQVBJIGlzIGxh eWVyZWQuCisJICoKKwkgKiBCYXJyaWVycyBhcmUgbm90IG5lZWRlZCBhcyBhbnkgcmFjZXMgaGVy ZSBhcmUgY2xvc2VkIGJ5IGFuIGV2ZW50dWFsCisJICogbW11X3JhbmdlX3JlYWRfcmV0cnkoKSwg d2hpY2ggcHJvdmlkZXMgYSBiYXJyaWVyIHZpYSB0aGUgdXNlcl9sb2NrLgorCSAqLworCXNwaW5f bG9jaygmbW1uX21tLT5sb2NrKTsKKwlzZXEgPSBSRUFEX09OQ0UobXJuLT5pbnZhbGlkYXRlX3Nl cSk7CisJaXNfaW52YWxpZGF0aW5nID0gc2VxID09IG1tbl9tbS0+aW52YWxpZGF0ZV9zZXE7CisJ c3Bpbl91bmxvY2soJm1tbl9tbS0+bG9jayk7CisKKwkvKgorCSAqIG1ybi0+aW52YWxpZGF0ZV9z ZXEgaXMgYWx3YXlzIHNldCB0byBhbiBvZGQgdmFsdWUuIFRoaXMgZW5zdXJlcworCSAqIHRoYXQg aWYgc2VxIGRvZXMgd3JhcCB3ZSB3aWxsIGFsd2F5cyBjbGVhciB0aGUgYmVsb3cgc2xlZXAgaW4g c29tZQorCSAqIHJlYXNvbmFibGUgdGltZSBhcyBtbW5fbW0tPmludmFsaWRhdGVfc2VxIGlzIGV2 ZW4gaW4gdGhlIGlkbGUKKwkgKiBzdGF0ZS4KKwkgKi8KKwlsb2NrX21hcF9hY3F1aXJlKCZfX21t dV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X21hcCk7CisJbG9ja19tYXBfcmVsZWFz ZSgmX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydF9tYXApOworCWlmIChpc19p bnZhbGlkYXRpbmcpCisJCXdhaXRfZXZlbnQobW1uX21tLT53cSwKKwkJCSAgIFJFQURfT05DRSht bW5fbW0tPmludmFsaWRhdGVfc2VxKSAhPSBzZXEpOworCisJLyoKKwkgKiBOb3RpY2UgdGhhdCBt bXVfcmFuZ2VfcmVhZF9yZXRyeSgpIGNhbiBhbHJlYWR5IGJlIHRydWUgYXQgdGhpcworCSAqIHBv aW50LCBhdm9pZGluZyBsb29wcyBoZXJlIGFsbG93cyB0aGUgdXNlciBvZiB0aGlzIGxvY2sgdG8g cHJvdmlkZQorCSAqIGEgZ2xvYmFsIHRpbWUgYm91bmQuCisJICovCisKKwlyZXR1cm4gc2VxOwor fQorRVhQT1JUX1NZTUJPTF9HUEwobW11X3JhbmdlX3JlYWRfYmVnaW4pOworCitzdGF0aWMgdm9p ZCBtbl9pdHJlZV9yZWxlYXNlKHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSwKKwkJCSAg ICAgc3RydWN0IG1tX3N0cnVjdCAqbW0pCit7CisJc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSBy YW5nZSA9IHsKKwkJLmZsYWdzID0gTU1VX05PVElGSUVSX1JBTkdFX0JMT0NLQUJMRSwKKwkJLmV2 ZW50ID0gTU1VX05PVElGWV9SRUxFQVNFLAorCQkubW0gPSBtbSwKKwkJLnN0YXJ0ID0gMCwKKwkJ LmVuZCA9IFVMT05HX01BWCwKKwl9OworCXN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybjsK Kwl1bnNpZ25lZCBsb25nIGN1cl9zZXE7CisJYm9vbCByZXQ7CisKKwlmb3IgKG1ybiA9IG1uX2l0 cmVlX2ludl9zdGFydF9yYW5nZShtbW5fbW0sICZyYW5nZSwgJmN1cl9zZXEpOyBtcm47CisJICAg ICBtcm4gPSBtbl9pdHJlZV9pbnZfbmV4dChtcm4sICZyYW5nZSkpIHsKKwkJcmV0ID0gbXJuLT5v cHMtPmludmFsaWRhdGUobXJuLCAmcmFuZ2UpOworCQlXQVJOX09OKHJldCk7CisJfQorCisJbW5f aXRyZWVfaW52X2VuZChtbW5fbW0pOworfQorCiAvKgogICogVGhpcyBmdW5jdGlvbiBjYW4ndCBy dW4gY29uY3VycmVudGx5IGFnYWluc3QgbW11X25vdGlmaWVyX3JlZ2lzdGVyCiAgKiBiZWNhdXNl IG1tLT5tbV91c2VycyA+IDAgZHVyaW5nIG1tdV9ub3RpZmllcl9yZWdpc3RlciBhbmQgZXhpdF9t bWFwCkBAIC01MiwxNyArMjg2LDI0IEBAIHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gewogICogY2Fu J3QgZ28gYXdheSBmcm9tIHVuZGVyIHVzIGFzIGV4aXRfbW1hcCBob2xkcyBhbiBtbV9jb3VudCBw aW4KICAqIGl0c2VsZi4KICAqLwotdm9pZCBfX21tdV9ub3RpZmllcl9yZWxlYXNlKHN0cnVjdCBt bV9zdHJ1Y3QgKm1tKQorc3RhdGljIHZvaWQgbW5faGxpc3RfcmVsZWFzZShzdHJ1Y3QgbW11X25v dGlmaWVyX21tICptbW5fbW0sCisJCQkgICAgIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQogewogCXN0 cnVjdCBtbXVfbm90aWZpZXIgKm1uOwogCWludCBpZDsKIAorCWlmIChtbW5fbW0tPmhhc19pbnRl cnZhbCkKKwkJbW5faXRyZWVfcmVsZWFzZShtbW5fbW0sIG1tKTsKKworCWlmIChobGlzdF9lbXB0 eSgmbW1uX21tLT5saXN0KSkKKwkJcmV0dXJuOworCiAJLyoKIAkgKiBTUkNVIGhlcmUgd2lsbCBi bG9jayBtbXVfbm90aWZpZXJfdW5yZWdpc3RlciB1bnRpbAogCSAqIC0+cmVsZWFzZSByZXR1cm5z LgogCSAqLwogCWlkID0gc3JjdV9yZWFkX2xvY2soJnNyY3UpOwotCWhsaXN0X2Zvcl9lYWNoX2Vu dHJ5X3JjdShtbiwgJm1tLT5tbXVfbm90aWZpZXJfbW0tPmxpc3QsIGhsaXN0KQorCWhsaXN0X2Zv cl9lYWNoX2VudHJ5X3JjdShtbiwgJm1tbl9tbS0+bGlzdCwgaGxpc3QpCiAJCS8qCiAJCSAqIElm IC0+cmVsZWFzZSBydW5zIGJlZm9yZSBtbXVfbm90aWZpZXJfdW5yZWdpc3RlciBpdCBtdXN0IGJl CiAJCSAqIGhhbmRsZWQsIGFzIGl0J3MgdGhlIG9ubHkgd2F5IGZvciB0aGUgZHJpdmVyIHRvIGZs dXNoIGFsbApAQCAtNzIsOSArMzEzLDkgQEAgdm9pZCBfX21tdV9ub3RpZmllcl9yZWxlYXNlKHN0 cnVjdCBtbV9zdHJ1Y3QgKm1tKQogCQlpZiAobW4tPm9wcy0+cmVsZWFzZSkKIAkJCW1uLT5vcHMt PnJlbGVhc2UobW4sIG1tKTsKIAotCXNwaW5fbG9jaygmbW0tPm1tdV9ub3RpZmllcl9tbS0+bG9j ayk7Ci0Jd2hpbGUgKHVubGlrZWx5KCFobGlzdF9lbXB0eSgmbW0tPm1tdV9ub3RpZmllcl9tbS0+ bGlzdCkpKSB7Ci0JCW1uID0gaGxpc3RfZW50cnkobW0tPm1tdV9ub3RpZmllcl9tbS0+bGlzdC5m aXJzdCwKKwlzcGluX2xvY2soJm1tbl9tbS0+bG9jayk7CisJd2hpbGUgKHVubGlrZWx5KCFobGlz dF9lbXB0eSgmbW1uX21tLT5saXN0KSkpIHsKKwkJbW4gPSBobGlzdF9lbnRyeShtbW5fbW0tPmxp c3QuZmlyc3QsCiAJCQkJIHN0cnVjdCBtbXVfbm90aWZpZXIsCiAJCQkJIGhsaXN0KTsKIAkJLyoK QEAgLTg1LDcgKzMyNiw3IEBAIHZvaWQgX19tbXVfbm90aWZpZXJfcmVsZWFzZShzdHJ1Y3QgbW1f c3RydWN0ICptbSkKIAkJICovCiAJCWhsaXN0X2RlbF9pbml0X3JjdSgmbW4tPmhsaXN0KTsKIAl9 Ci0Jc3Bpbl91bmxvY2soJm1tLT5tbXVfbm90aWZpZXJfbW0tPmxvY2spOworCXNwaW5fdW5sb2Nr KCZtbW5fbW0tPmxvY2spOwogCXNyY3VfcmVhZF91bmxvY2soJnNyY3UsIGlkKTsKIAogCS8qCkBA IC0xMDAsNiArMzQxLDE3IEBAIHZvaWQgX19tbXVfbm90aWZpZXJfcmVsZWFzZShzdHJ1Y3QgbW1f c3RydWN0ICptbSkKIAlzeW5jaHJvbml6ZV9zcmN1KCZzcmN1KTsKIH0KIAordm9pZCBfX21tdV9u b3RpZmllcl9yZWxlYXNlKHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQoreworCXN0cnVjdCBtbXVfbm90 aWZpZXJfbW0gKm1tbl9tbSA9IG1tLT5tbXVfbm90aWZpZXJfbW07CisKKwlpZiAobW1uX21tLT5o YXNfaW50ZXJ2YWwpCisJCW1uX2l0cmVlX3JlbGVhc2UobW1uX21tLCBtbSk7CisKKwlpZiAoIWhs aXN0X2VtcHR5KCZtbW5fbW0tPmxpc3QpKQorCQltbl9obGlzdF9yZWxlYXNlKG1tbl9tbSwgbW0p OworfQorCiAvKgogICogSWYgbm8geW91bmcgYml0ZmxhZyBpcyBzdXBwb3J0ZWQgYnkgdGhlIGhh cmR3YXJlLCAtPmNsZWFyX2ZsdXNoX3lvdW5nIGNhbgogICogdW5tYXAgdGhlIGFkZHJlc3MgYW5k IHJldHVybiAxIG9yIDAgZGVwZW5kaW5nIGlmIHRoZSBtYXBwaW5nIHByZXZpb3VzbHkKQEAgLTE3 MiwxNCArNDI0LDQxIEBAIHZvaWQgX19tbXVfbm90aWZpZXJfY2hhbmdlX3B0ZShzdHJ1Y3QgbW1f c3RydWN0ICptbSwgdW5zaWduZWQgbG9uZyBhZGRyZXNzLAogCXNyY3VfcmVhZF91bmxvY2soJnNy Y3UsIGlkKTsKIH0KIAotaW50IF9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQo c3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UpCitzdGF0aWMgaW50IG1uX2l0cmVlX2lu dmFsaWRhdGUoc3RydWN0IG1tdV9ub3RpZmllcl9tbSAqbW1uX21tLAorCQkJCSAgICAgY29uc3Qg c3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UpCit7CisJc3RydWN0IG1tdV9yYW5nZV9u b3RpZmllciAqbXJuOworCXVuc2lnbmVkIGxvbmcgY3VyX3NlcTsKKworCWZvciAobXJuID0gbW5f aXRyZWVfaW52X3N0YXJ0X3JhbmdlKG1tbl9tbSwgcmFuZ2UsICZjdXJfc2VxKTsgbXJuOworCSAg ICAgbXJuID0gbW5faXRyZWVfaW52X25leHQobXJuLCByYW5nZSkpIHsKKwkJYm9vbCByZXQ7CisK KwkJV1JJVEVfT05DRShtcm4tPmludmFsaWRhdGVfc2VxLCBjdXJfc2VxKTsKKwkJcmV0ID0gbXJu LT5vcHMtPmludmFsaWRhdGUobXJuLCByYW5nZSk7CisJCWlmICghcmV0ICYmICFXQVJOX09OKG1t dV9ub3RpZmllcl9yYW5nZV9ibG9ja2FibGUocmFuZ2UpKSkKKwkJCWdvdG8gb3V0X3dvdWxkX2Js b2NrOworCX0KKwlyZXR1cm4gMDsKKworb3V0X3dvdWxkX2Jsb2NrOgorCS8qCisJICogT24gLUVB R0FJTiB0aGUgbm9uLWJsb2NraW5nIGNhbGxlciBpcyBub3QgYWxsb3dlZCB0byBjYWxsCisJICog aW52YWxpZGF0ZV9yYW5nZV9lbmQoKQorCSAqLworCW1uX2l0cmVlX2ludl9lbmQobW1uX21tKTsK KwlyZXR1cm4gLUVBR0FJTjsKK30KKworc3RhdGljIGludCBtbl9obGlzdF9pbnZhbGlkYXRlX3Jh bmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSwKKwkJCQkJICAgc3RydWN0 IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UpCiB7CiAJc3RydWN0IG1tdV9ub3RpZmllciAqbW47 CiAJaW50IHJldCA9IDA7CiAJaW50IGlkOwogCiAJaWQgPSBzcmN1X3JlYWRfbG9jaygmc3JjdSk7 Ci0JaGxpc3RfZm9yX2VhY2hfZW50cnlfcmN1KG1uLCAmcmFuZ2UtPm1tLT5tbXVfbm90aWZpZXJf bW0tPmxpc3QsIGhsaXN0KSB7CisJaGxpc3RfZm9yX2VhY2hfZW50cnlfcmN1KG1uLCAmbW1uX21t LT5saXN0LCBobGlzdCkgewogCQlpZiAobW4tPm9wcy0+aW52YWxpZGF0ZV9yYW5nZV9zdGFydCkg ewogCQkJaW50IF9yZXQ7CiAKQEAgLTIwMywxNSArNDgyLDMwIEBAIGludCBfX21tdV9ub3RpZmll cl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0KHN0cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2UgKnJhbmdl KQogCXJldHVybiByZXQ7CiB9CiAKLXZvaWQgX19tbXVfbm90aWZpZXJfaW52YWxpZGF0ZV9yYW5n ZV9lbmQoc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UsCi0JCQkJCSBib29sIG9ubHlf ZW5kKQoraW50IF9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnQoc3RydWN0IG1t dV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UpCit7CisJc3RydWN0IG1tdV9ub3RpZmllcl9tbSAqbW1u X21tID0gcmFuZ2UtPm1tLT5tbXVfbm90aWZpZXJfbW07CisJaW50IHJldCA9IDA7CisKKwlpZiAo bW1uX21tLT5oYXNfaW50ZXJ2YWwpIHsKKwkJcmV0ID0gbW5faXRyZWVfaW52YWxpZGF0ZShtbW5f bW0sIHJhbmdlKTsKKwkJaWYgKHJldCkKKwkJCXJldHVybiByZXQ7CisJfQorCWlmICghaGxpc3Rf ZW1wdHkoJm1tbl9tbS0+bGlzdCkpCisJCXJldHVybiBtbl9obGlzdF9pbnZhbGlkYXRlX3Jhbmdl X3N0YXJ0KG1tbl9tbSwgcmFuZ2UpOworCXJldHVybiAwOworfQorCitzdGF0aWMgdm9pZCBtbl9o bGlzdF9pbnZhbGlkYXRlX2VuZChzdHJ1Y3QgbW11X25vdGlmaWVyX21tICptbW5fbW0sCisJCQkJ ICAgIHN0cnVjdCBtbXVfbm90aWZpZXJfcmFuZ2UgKnJhbmdlLAorCQkJCSAgICBib29sIG9ubHlf ZW5kKQogewogCXN0cnVjdCBtbXVfbm90aWZpZXIgKm1uOwogCWludCBpZDsKIAotCWxvY2tfbWFw X2FjcXVpcmUoJl9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbWFwKTsKIAlp ZCA9IHNyY3VfcmVhZF9sb2NrKCZzcmN1KTsKLQlobGlzdF9mb3JfZWFjaF9lbnRyeV9yY3UobW4s ICZyYW5nZS0+bW0tPm1tdV9ub3RpZmllcl9tbS0+bGlzdCwgaGxpc3QpIHsKKwlobGlzdF9mb3Jf ZWFjaF9lbnRyeV9yY3UobW4sICZtbW5fbW0tPmxpc3QsIGhsaXN0KSB7CiAJCS8qCiAJCSAqIENh bGwgaW52YWxpZGF0ZV9yYW5nZSBoZXJlIHRvbyB0byBhdm9pZCB0aGUgbmVlZCBmb3IgdGhlCiAJ CSAqIHN1YnN5c3RlbSBvZiBoYXZpbmcgdG8gcmVnaXN0ZXIgYW4gaW52YWxpZGF0ZV9yYW5nZV9l bmQKQEAgLTIzOCw2ICs1MzIsMTkgQEAgdm9pZCBfX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3Jh bmdlX2VuZChzdHJ1Y3QgbW11X25vdGlmaWVyX3JhbmdlICpyYW5nZSwKIAkJfQogCX0KIAlzcmN1 X3JlYWRfdW5sb2NrKCZzcmN1LCBpZCk7Cit9CisKK3ZvaWQgX19tbXVfbm90aWZpZXJfaW52YWxp ZGF0ZV9yYW5nZV9lbmQoc3RydWN0IG1tdV9ub3RpZmllcl9yYW5nZSAqcmFuZ2UsCisJCQkJCSBi b29sIG9ubHlfZW5kKQoreworCXN0cnVjdCBtbXVfbm90aWZpZXJfbW0gKm1tbl9tbSA9IHJhbmdl LT5tbS0+bW11X25vdGlmaWVyX21tOworCisJbG9ja19tYXBfYWNxdWlyZSgmX19tbXVfbm90aWZp ZXJfaW52YWxpZGF0ZV9yYW5nZV9zdGFydF9tYXApOworCWlmIChtbW5fbW0tPmhhc19pbnRlcnZh bCkKKwkJbW5faXRyZWVfaW52X2VuZChtbW5fbW0pOworCisJaWYgKCFobGlzdF9lbXB0eSgmbW1u X21tLT5saXN0KSkKKwkJbW5faGxpc3RfaW52YWxpZGF0ZV9lbmQobW1uX21tLCByYW5nZSwgb25s eV9lbmQpOwogCWxvY2tfbWFwX3JlbGVhc2UoJl9fbW11X25vdGlmaWVyX2ludmFsaWRhdGVfcmFu Z2Vfc3RhcnRfbWFwKTsKIH0KIApAQCAtMjU2LDggKzU2Myw5IEBAIHZvaWQgX19tbXVfbm90aWZp ZXJfaW52YWxpZGF0ZV9yYW5nZShzdHJ1Y3QgbW1fc3RydWN0ICptbSwKIH0KIAogLyoKLSAqIFNh bWUgYXMgbW11X25vdGlmaWVyX3JlZ2lzdGVyIGJ1dCBoZXJlIHRoZSBjYWxsZXIgbXVzdCBob2xk IHRoZQotICogbW1hcF9zZW0gaW4gd3JpdGUgbW9kZS4KKyAqIFNhbWUgYXMgbW11X25vdGlmaWVy X3JlZ2lzdGVyIGJ1dCBoZXJlIHRoZSBjYWxsZXIgbXVzdCBob2xkIHRoZSBtbWFwX3NlbSBpbgor ICogd3JpdGUgbW9kZS4gQSBOVUxMIG1uIHNpZ25hbHMgdGhlIG5vdGlmaWVyIGlzIGJlaW5nIHJl Z2lzdGVyZWQgZm9yIGl0cmVlCisgKiBtb2RlLgogICovCiBpbnQgX19tbXVfbm90aWZpZXJfcmVn aXN0ZXIoc3RydWN0IG1tdV9ub3RpZmllciAqbW4sIHN0cnVjdCBtbV9zdHJ1Y3QgKm1tKQogewpA QCAtMjc0LDkgKzU4Miw2IEBAIGludCBfX21tdV9ub3RpZmllcl9yZWdpc3RlcihzdHJ1Y3QgbW11 X25vdGlmaWVyICptbiwgc3RydWN0IG1tX3N0cnVjdCAqbW0pCiAJCWZzX3JlY2xhaW1fcmVsZWFz ZShHRlBfS0VSTkVMKTsKIAl9CiAKLQltbi0+bW0gPSBtbTsKLQltbi0+dXNlcnMgPSAxOwotCiAJ aWYgKCFtbS0+bW11X25vdGlmaWVyX21tKSB7CiAJCS8qCiAJCSAqIGttYWxsb2MgY2Fubm90IGJl IGNhbGxlZCB1bmRlciBtbV90YWtlX2FsbF9sb2NrcygpLCBidXQgd2UKQEAgLTI4NCwyMSArNTg5 LDIyIEBAIGludCBfX21tdV9ub3RpZmllcl9yZWdpc3RlcihzdHJ1Y3QgbW11X25vdGlmaWVyICpt biwgc3RydWN0IG1tX3N0cnVjdCAqbW0pCiAJCSAqIHRoZSB3cml0ZSBzaWRlIG9mIHRoZSBtbWFw X3NlbS4KIAkJICovCiAJCW1tdV9ub3RpZmllcl9tbSA9Ci0JCQlrbWFsbG9jKHNpemVvZihzdHJ1 Y3QgbW11X25vdGlmaWVyX21tKSwgR0ZQX0tFUk5FTCk7CisJCQlremFsbG9jKHNpemVvZihzdHJ1 Y3QgbW11X25vdGlmaWVyX21tKSwgR0ZQX0tFUk5FTCk7CiAJCWlmICghbW11X25vdGlmaWVyX21t KQogCQkJcmV0dXJuIC1FTk9NRU07CiAKIAkJSU5JVF9ITElTVF9IRUFEKCZtbXVfbm90aWZpZXJf bW0tPmxpc3QpOwogCQlzcGluX2xvY2tfaW5pdCgmbW11X25vdGlmaWVyX21tLT5sb2NrKTsKKwkJ bW11X25vdGlmaWVyX21tLT5pbnZhbGlkYXRlX3NlcSA9IDI7CisJCW1tdV9ub3RpZmllcl9tbS0+ aXRyZWUgPSBSQl9ST09UX0NBQ0hFRDsKKwkJaW5pdF93YWl0cXVldWVfaGVhZCgmbW11X25vdGlm aWVyX21tLT53cSk7CisJCUlOSVRfSExJU1RfSEVBRCgmbW11X25vdGlmaWVyX21tLT5kZWZlcnJl ZF9saXN0KTsKIAl9CiAKIAlyZXQgPSBtbV90YWtlX2FsbF9sb2NrcyhtbSk7CiAJaWYgKHVubGlr ZWx5KHJldCkpCiAJCWdvdG8gb3V0X2NsZWFuOwogCi0JLyogUGFpcnMgd2l0aCB0aGUgbW1kcm9w IGluIG1tdV9ub3RpZmllcl91bnJlZ2lzdGVyXyogKi8KLQltbWdyYWIobW0pOwotCiAJLyoKIAkg KiBTZXJpYWxpemUgdGhlIHVwZGF0ZSBhZ2FpbnN0IG1tdV9ub3RpZmllcl91bnJlZ2lzdGVyLiBB CiAJICogc2lkZSBub3RlOiBtbXVfbm90aWZpZXJfcmVsZWFzZSBjYW4ndCBydW4gY29uY3VycmVu dGx5IHdpdGgKQEAgLTMwNiwxMyArNjEyLDI2IEBAIGludCBfX21tdV9ub3RpZmllcl9yZWdpc3Rl cihzdHJ1Y3QgbW11X25vdGlmaWVyICptbiwgc3RydWN0IG1tX3N0cnVjdCAqbW0pCiAJICogY3Vy cmVudC0+bW0gb3IgZXhwbGljaXRseSB3aXRoIGdldF90YXNrX21tKCkgb3Igc2ltaWxhcikuCiAJ ICogV2UgY2FuJ3QgcmFjZSBhZ2FpbnN0IGFueSBvdGhlciBtbXUgbm90aWZpZXIgbWV0aG9kIGVp dGhlcgogCSAqIHRoYW5rcyB0byBtbV90YWtlX2FsbF9sb2NrcygpLgorCSAqCisJICogcmVsZWFz ZSBzZW1hbnRpY3MgYXJlIHByb3ZpZGVkIGZvciB1c2VycyBub3QgaW5zaWRlIGEgbG9jayBjb3Zl cmVkCisJICogYnkgbW1fdGFrZV9hbGxfbG9ja3MoKS4gYWNxdWlyZSBjYW4gb25seSBiZSB1c2Vk IHdoaWxlIGhvbGRpbmcgdGhlCisJICogbW1ncmFiIG9yIG1tZ2V0LCBhbmQgaXMgc2FmZSBiZWNh dXNlIG9uY2UgY3JlYXRlZCB0aGUKKwkgKiBtbXVfbm90aWZpZmVyX21tIGlzIG5vdCBmcmVlZCB1 bnRpbCB0aGUgbW0gaXMgZGVzdHJveWVkLgogCSAqLwogCWlmIChtbXVfbm90aWZpZXJfbW0pCi0J CW1tLT5tbXVfbm90aWZpZXJfbW0gPSBtbXVfbm90aWZpZXJfbW07CisJCXNtcF9zdG9yZV9yZWxl YXNlKCZtbS0+bW11X25vdGlmaWVyX21tLCBtbXVfbm90aWZpZXJfbW0pOwogCi0Jc3Bpbl9sb2Nr KCZtbS0+bW11X25vdGlmaWVyX21tLT5sb2NrKTsKLQlobGlzdF9hZGRfaGVhZF9yY3UoJm1uLT5o bGlzdCwgJm1tLT5tbXVfbm90aWZpZXJfbW0tPmxpc3QpOwotCXNwaW5fdW5sb2NrKCZtbS0+bW11 X25vdGlmaWVyX21tLT5sb2NrKTsKKwlpZiAobW4pIHsKKwkJLyogUGFpcnMgd2l0aCB0aGUgbW1k cm9wIGluIG1tdV9ub3RpZmllcl91bnJlZ2lzdGVyXyogKi8KKwkJbW1ncmFiKG1tKTsKKwkJbW4t Pm1tID0gbW07CisJCW1uLT51c2VycyA9IDE7CisKKwkJc3Bpbl9sb2NrKCZtbS0+bW11X25vdGlm aWVyX21tLT5sb2NrKTsKKwkJaGxpc3RfYWRkX2hlYWRfcmN1KCZtbi0+aGxpc3QsICZtbS0+bW11 X25vdGlmaWVyX21tLT5saXN0KTsKKwkJc3Bpbl91bmxvY2soJm1tLT5tbXVfbm90aWZpZXJfbW0t PmxvY2spOworCX0gZWxzZQorCQltbS0+bW11X25vdGlmaWVyX21tLT5oYXNfaW50ZXJ2YWwgPSB0 cnVlOwogCiAJbW1fZHJvcF9hbGxfbG9ja3MobW0pOwogCUJVR19PTihhdG9taWNfcmVhZCgmbW0t Pm1tX3VzZXJzKSA8PSAwKTsKQEAgLTUyOSw2ICs4NDgsMTY2IEBAIHZvaWQgbW11X25vdGlmaWVy X3B1dChzdHJ1Y3QgbW11X25vdGlmaWVyICptbikKIH0KIEVYUE9SVF9TWU1CT0xfR1BMKG1tdV9u b3RpZmllcl9wdXQpOwogCitzdGF0aWMgaW50IF9fbW11X3JhbmdlX25vdGlmaWVyX2luc2VydChz dHJ1Y3QgbW11X3JhbmdlX25vdGlmaWVyICptcm4sCisJCQkJICAgICAgIHVuc2lnbmVkIGxvbmcg c3RhcnQsCisJCQkJICAgICAgIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLAorCQkJCSAgICAgICBzdHJ1 Y3QgbW11X25vdGlmaWVyX21tICptbW5fbW0sCisJCQkJICAgICAgIHN0cnVjdCBtbV9zdHJ1Y3Qg Km1tKQoreworCW1ybi0+bW0gPSBtbTsKKwlSQl9DTEVBUl9OT0RFKCZtcm4tPmludGVydmFsX3Ry ZWUucmIpOworCW1ybi0+aW50ZXJ2YWxfdHJlZS5zdGFydCA9IHN0YXJ0OworCS8qCisJICogTm90 ZSB0aGF0IHRoZSByZXByZXNlbnRhdGlvbiBvZiB0aGUgaW50ZXJ2YWxzIGluIHRoZSBpbnRlcnZh bCB0cmVlCisJICogY29uc2lkZXJzIHRoZSBlbmRpbmcgcG9pbnQgYXMgY29udGFpbmVkIGluIHRo ZSBpbnRlcnZhbC4KKwkgKi8KKwlpZiAobGVuZ3RoID09IDAgfHwKKwkgICAgY2hlY2tfYWRkX292 ZXJmbG93KHN0YXJ0LCBsZW5ndGggLSAxLCAmbXJuLT5pbnRlcnZhbF90cmVlLmxhc3QpKQorCQly ZXR1cm4gLUVPVkVSRkxPVzsKKworCS8qIHBhaXJzIHdpdGggbW1kcm9wIGluIG1tdV9yYW5nZV9u b3RpZmllcl9yZW1vdmUoKSAqLworCW1tZ3JhYihtbSk7CisKKwkvKgorCSAqIElmIHNvbWUgaW52 YWxpZGF0ZV9yYW5nZV9zdGFydC9lbmQgcmVnaW9uIGlzIGdvaW5nIG9uIGluIHBhcmFsbGVsCisJ ICogd2UgZG9uJ3Qga25vdyB3aGF0IFZBIHJhbmdlcyBhcmUgYWZmZWN0ZWQsIHNvIHdlIG11c3Qg YXNzdW1lIHRoaXMKKwkgKiBuZXcgcmFuZ2UgaXMgaW5jbHVkZWQuCisJICoKKwkgKiBJZiB0aGUg aXRyZWUgaXMgaW52YWxpZGF0aW5nIHRoZW4gd2UgYXJlIG5vdCBhbGxvd2VkIHRvIGNoYW5nZQor CSAqIGl0LiBSZXRyeWluZyB1bnRpbCBpbnZhbGlkYXRpb24gaXMgZG9uZSBpcyB0cmlja3kgZHVl IHRvIHRoZQorCSAqIHBvc3NpYmlsaXR5IGZvciBsaXZlIGxvY2ssIGluc3RlYWQgZGVmZXIgdGhl IGFkZCB0byB0aGUgdW5sb2NrIHNvCisJICogdGhpcyBhbGdvcml0aG0gaXMgZGV0ZXJtaW5pc3Rp Yy4KKwkgKgorCSAqIEluIGFsbCBjYXNlcyB0aGUgdmFsdWUgZm9yIHRoZSBtcm4tPm1yX2ludmFs aWRhdGVfc2VxIHNob3VsZCBiZQorCSAqIG9kZCwgc2VlIG1tdV9yYW5nZV9yZWFkX2JlZ2luKCkK KwkgKi8KKwlzcGluX2xvY2soJm1tbl9tbS0+bG9jayk7CisJaWYgKG1tbl9tbS0+YWN0aXZlX2lu dmFsaWRhdGVfcmFuZ2VzKSB7CisJCWlmIChtbl9pdHJlZV9pc19pbnZhbGlkYXRpbmcobW1uX21t KSkKKwkJCWhsaXN0X2FkZF9oZWFkKCZtcm4tPmRlZmVycmVkX2l0ZW0sCisJCQkJICAgICAgICZt bW5fbW0tPmRlZmVycmVkX2xpc3QpOworCQllbHNlIHsKKwkJCW1tbl9tbS0+aW52YWxpZGF0ZV9z ZXEgfD0gMTsKKwkJCWludGVydmFsX3RyZWVfaW5zZXJ0KCZtcm4tPmludGVydmFsX3RyZWUsCisJ CQkJCSAgICAgJm1tbl9tbS0+aXRyZWUpOworCQl9CisJCW1ybi0+aW52YWxpZGF0ZV9zZXEgPSBt bW5fbW0tPmludmFsaWRhdGVfc2VxOworCX0gZWxzZSB7CisJCVdBUk5fT04obW5faXRyZWVfaXNf aW52YWxpZGF0aW5nKG1tbl9tbSkpOworCQltcm4tPmludmFsaWRhdGVfc2VxID0gbW1uX21tLT5p bnZhbGlkYXRlX3NlcSAtIDE7CisJCWludGVydmFsX3RyZWVfaW5zZXJ0KCZtcm4tPmludGVydmFs X3RyZWUsICZtbW5fbW0tPml0cmVlKTsKKwl9CisJc3Bpbl91bmxvY2soJm1tbl9tbS0+bG9jayk7 CisJcmV0dXJuIDA7Cit9CisKKy8qKgorICogbW11X3JhbmdlX25vdGlmaWVyX2luc2VydCAtIElu c2VydCBhIHJhbmdlIG5vdGlmaWVyCisgKiBAbXJuOiBSYW5nZSBub3RpZmllciB0byByZWdpc3Rl cgorICogQHN0YXJ0OiBTdGFydGluZyB2aXJ0dWFsIGFkZHJlc3MgdG8gbW9uaXRvcgorICogQGxl bmd0aDogTGVuZ3RoIG9mIHRoZSByYW5nZSB0byBtb25pdG9yCisgKiBAbW0gOiBtbV9zdHJ1Y3Qg dG8gYXR0YWNoIHRvCisgKgorICogVGhpcyBmdW5jdGlvbiBzdWJzY3JpYmVzIHRoZSByYW5nZSBu b3RpZmllciBmb3Igbm90aWZpY2F0aW9ucyBmcm9tIHRoZSBtbS4KKyAqIFVwb24gcmV0dXJuIHRo ZSBvcHMgcmVsYXRlZCB0byBtbXVfcmFuZ2Vfbm90aWZpZXIgd2lsbCBiZSBjYWxsZWQgd2hlbmV2 ZXIKKyAqIGFuIGV2ZW50IHRoYXQgaW50ZXJzZWN0cyB3aXRoIHRoZSBnaXZlbiByYW5nZSBvY2N1 cnMuCisgKgorICogVXBvbiByZXR1cm4gdGhlIHJhbmdlX25vdGlmaWVyIG1heSBub3QgYmUgcHJl c2VudCBpbiB0aGUgaW50ZXJ2YWwgdHJlZSB5ZXQuCisgKiBUaGUgY2FsbGVyIG11c3QgdXNlIHRo ZSBub3JtYWwgcmFuZ2Ugbm90aWZpZXIgbG9ja2luZyBmbG93IHZpYQorICogbW11X3JhbmdlX3Jl YWRfYmVnaW4oKSB0byBlc3RhYmxpc2ggU1BURXMgZm9yIHRoaXMgcmFuZ2UuCisgKi8KK2ludCBt bXVfcmFuZ2Vfbm90aWZpZXJfaW5zZXJ0KHN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybiwK KwkJCSAgICAgIHVuc2lnbmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLAorCQkJ ICAgICAgc3RydWN0IG1tX3N0cnVjdCAqbW0pCit7CisJc3RydWN0IG1tdV9ub3RpZmllcl9tbSAq bW1uX21tOworCWludCByZXQ7CisKKwltaWdodF9sb2NrKCZtbS0+bW1hcF9zZW0pOworCisJbW1u X21tID0gc21wX2xvYWRfYWNxdWlyZSgmbW0tPm1tdV9ub3RpZmllcl9tbSk7CisJaWYgKCFtbW5f bW0gfHwgIW1tbl9tbS0+aGFzX2ludGVydmFsKSB7CisJCXJldCA9IG1tdV9ub3RpZmllcl9yZWdp c3RlcihOVUxMLCBtbSk7CisJCWlmIChyZXQpCisJCQlyZXR1cm4gcmV0OworCQltbW5fbW0gPSBt bS0+bW11X25vdGlmaWVyX21tOworCX0KKwlyZXR1cm4gX19tbXVfcmFuZ2Vfbm90aWZpZXJfaW5z ZXJ0KG1ybiwgc3RhcnQsIGxlbmd0aCwgbW1uX21tLCBtbSk7Cit9CitFWFBPUlRfU1lNQk9MX0dQ TChtbXVfcmFuZ2Vfbm90aWZpZXJfaW5zZXJ0KTsKKworaW50IG1tdV9yYW5nZV9ub3RpZmllcl9p bnNlcnRfbG9ja2VkKHN0cnVjdCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybiwKKwkJCQkgICAgIHVu c2lnbmVkIGxvbmcgc3RhcnQsIHVuc2lnbmVkIGxvbmcgbGVuZ3RoLAorCQkJCSAgICAgc3RydWN0 IG1tX3N0cnVjdCAqbW0pCit7CisJc3RydWN0IG1tdV9ub3RpZmllcl9tbSAqbW1uX21tOworCWlu dCByZXQ7CisKKwlsb2NrZGVwX2Fzc2VydF9oZWxkX3dyaXRlKCZtbS0+bW1hcF9zZW0pOworCisJ bW1uX21tID0gbW0tPm1tdV9ub3RpZmllcl9tbTsKKwlpZiAoIW1tbl9tbSB8fCAhbW1uX21tLT5o YXNfaW50ZXJ2YWwpIHsKKwkJcmV0ID0gX19tbXVfbm90aWZpZXJfcmVnaXN0ZXIoTlVMTCwgbW0p OworCQlpZiAocmV0KQorCQkJcmV0dXJuIHJldDsKKwkJbW1uX21tID0gbW0tPm1tdV9ub3RpZmll cl9tbTsKKwl9CisJcmV0dXJuIF9fbW11X3JhbmdlX25vdGlmaWVyX2luc2VydChtcm4sIHN0YXJ0 LCBsZW5ndGgsIG1tbl9tbSwgbW0pOworfQorRVhQT1JUX1NZTUJPTF9HUEwobW11X3JhbmdlX25v dGlmaWVyX2luc2VydF9sb2NrZWQpOworCisvKioKKyAqIG1tdV9yYW5nZV9ub3RpZmllcl9yZW1v dmUgLSBSZW1vdmUgYSByYW5nZSBub3RpZmllcgorICogQG1ybjogUmFuZ2Ugbm90aWZpZXIgdG8g dW5yZWdpc3RlcgorICoKKyAqIFRoaXMgZnVuY3Rpb24gbXVzdCBiZSBwYWlyZWQgd2l0aCBtbXVf cmFuZ2Vfbm90aWZpZXJfaW5zZXJ0KCkuIEl0IGNhbm5vdCBiZQorICogY2FsbGVkIGZyb20gYW55 IG9wcyBjYWxsYmFjay4KKyAqCisgKiBPbmNlIHRoaXMgcmV0dXJucyBvcHMgY2FsbGJhY2tzIGFy ZSBubyBsb25nZXIgcnVubmluZyBvbiBvdGhlciBDUFVzIGFuZAorICogd2lsbCBub3QgYmUgY2Fs bGVkIGluIGZ1dHVyZS4KKyAqLwordm9pZCBtbXVfcmFuZ2Vfbm90aWZpZXJfcmVtb3ZlKHN0cnVj dCBtbXVfcmFuZ2Vfbm90aWZpZXIgKm1ybikKK3sKKwlzdHJ1Y3QgbW1fc3RydWN0ICptbSA9IG1y bi0+bW07CisJc3RydWN0IG1tdV9ub3RpZmllcl9tbSAqbW1uX21tID0gbW0tPm1tdV9ub3RpZmll cl9tbTsKKwl1bnNpZ25lZCBsb25nIHNlcSA9IDA7CisKKwltaWdodF9zbGVlcCgpOworCisJc3Bp bl9sb2NrKCZtbW5fbW0tPmxvY2spOworCWlmIChtbl9pdHJlZV9pc19pbnZhbGlkYXRpbmcobW1u X21tKSkgeworCQkvKgorCQkgKiByZW1vdmUgaXMgYmVpbmcgY2FsbGVkIGFmdGVyIGluc2VydCBw dXQgdGhpcyBvbiB0aGUKKwkJICogZGVmZXJyZWQgbGlzdCwgYnV0IGJlZm9yZSB0aGUgZGVmZXJy ZWQgbGlzdCB3YXMgcHJvY2Vzc2VkLgorCQkgKi8KKwkJaWYgKFJCX0VNUFRZX05PREUoJm1ybi0+ aW50ZXJ2YWxfdHJlZS5yYikpIHsKKwkJCWhsaXN0X2RlbCgmbXJuLT5kZWZlcnJlZF9pdGVtKTsK KwkJfSBlbHNlIHsKKwkJCWhsaXN0X2FkZF9oZWFkKCZtcm4tPmRlZmVycmVkX2l0ZW0sCisJCQkJ ICAgICAgICZtbW5fbW0tPmRlZmVycmVkX2xpc3QpOworCQkJc2VxID0gbW1uX21tLT5pbnZhbGlk YXRlX3NlcTsKKwkJfQorCX0gZWxzZSB7CisJCVdBUk5fT04oUkJfRU1QVFlfTk9ERSgmbXJuLT5p bnRlcnZhbF90cmVlLnJiKSk7CisJCWludGVydmFsX3RyZWVfcmVtb3ZlKCZtcm4tPmludGVydmFs X3RyZWUsICZtbW5fbW0tPml0cmVlKTsKKwl9CisJc3Bpbl91bmxvY2soJm1tbl9tbS0+bG9jayk7 CisKKwkvKgorCSAqIFRoZSBwb3NzaWJsZSBzbGVlcCBvbiBwcm9ncmVzcyBpbiB0aGUgaW52YWxp ZGF0aW9uIHJlcXVpcmVzIHRoZQorCSAqIGNhbGxlciBub3QgaG9sZCBhbnkgbG9ja3MgaGVsZCBi eSBpbnZhbGlkYXRpb24gY2FsbGJhY2tzLgorCSAqLworCWxvY2tfbWFwX2FjcXVpcmUoJl9fbW11 X25vdGlmaWVyX2ludmFsaWRhdGVfcmFuZ2Vfc3RhcnRfbWFwKTsKKwlsb2NrX21hcF9yZWxlYXNl KCZfX21tdV9ub3RpZmllcl9pbnZhbGlkYXRlX3JhbmdlX3N0YXJ0X21hcCk7CisJaWYgKHNlcSkK KwkJd2FpdF9ldmVudChtbW5fbW0tPndxLAorCQkJICAgUkVBRF9PTkNFKG1tbl9tbS0+aW52YWxp ZGF0ZV9zZXEpICE9IHNlcSk7CisKKwkvKiBwYWlycyB3aXRoIG1tZ3JhYiBpbiBtbXVfcmFuZ2Vf bm90aWZpZXJfaW5zZXJ0KCkgKi8KKwltbWRyb3AobW0pOworfQorRVhQT1JUX1NZTUJPTF9HUEwo bW11X3JhbmdlX25vdGlmaWVyX3JlbW92ZSk7CisKIC8qKgogICogbW11X25vdGlmaWVyX3N5bmNo cm9uaXplIC0gRW5zdXJlIGFsbCBtbXVfbm90aWZpZXJzIGFyZSBmcmVlZAogICoKLS0gCjIuMjMu MAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KYW1kLWdm eCBtYWlsaW5nIGxpc3QKYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0 cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4