From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3722BC47094 for ; Thu, 10 Jun 2021 09:18:20 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 089B0613ED for ; Thu, 10 Jun 2021 09:18:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 089B0613ED Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7A25B6EC97; Thu, 10 Jun 2021 09:18:09 +0000 (UTC) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 708946EC91; Thu, 10 Jun 2021 09:18:06 +0000 (UTC) Received: by mail-wm1-x32c.google.com with SMTP id f16-20020a05600c1550b02901b00c1be4abso6078138wmg.2; Thu, 10 Jun 2021 02:18:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BjeLzdv5CJdZ+kjl5N5cB4i0YWorPBc0kCEKEtj36hk=; b=Sa0V5bOX2duti/B0xp/yQlouSvyehGMybsJv64VJbDXSP65dmbnRyBeUk9rLYRRSk5 iHCXkHaDMOiPw6cV09034/j4+RKyZOVY/q2M9BivE8lqrdPBwfCmJCuvJZ1l94H3FKhz o5bfgOaqhPJUVdFD/Gr+ngT4C12kmZ7TKzHtl+VOc+K59LyrE54GbOlOLILykjZqeKAR b/j061/L+rcqEMjBcK/nPSZuP56MuC5hUORRtYNKv8MPhsKFn2IVwaa2FnKMSB16a998 Fm4XOb+v01liBGJeHGwqBOaLmkMHkDetFU5moO2jHD6NFNfQEe0go7gXutxFkZY68HxO 23hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BjeLzdv5CJdZ+kjl5N5cB4i0YWorPBc0kCEKEtj36hk=; b=QPhVIeWjGGbJscaps+M9AP1lTNWWuzVsnCiK1it+QyJrAeZxwd4P2dX2VfsfX3SKpI SONlm9Ed6UoroPc71+ufgHhQkThPxQCITYiQgW6YWFdNdLLHeTI6l/C4GiSfyoTBgA9G x/BuDp2op2Y7lb8C7ad8GZ/PwAPpD2h3txcJKEIzaL5i2Z3rwkv2C69HCP0nqCeK8njN IN1Q+/1IKDCa9Raam4AmPyc3B2Qiq/mJaLrTpvgUFgywhOI/anPm1aOc8TPH8wRpZPWn B89MTZ7qjW8tvA/QaCQVChs74PaSrHxz4L1ES68u4IDMvS9oFJ2s8fxWG3B3sqIUbCm5 uGUw== X-Gm-Message-State: AOAM530+5D/YI0avsQ4IaPwQmzFX2YIbXnuIDzgerijKVVShYkxO67Dm vuWGC6jMlNWO91dQcOSAj4lvodDEOfc= X-Google-Smtp-Source: ABdhPJzJom7o/I8KWbXJK9cMCt6ovyYn3xmDoLbM4QHhtCauCbjwYekHC0PhGdvlWORi9WNqh7h4vQ== X-Received: by 2002:a1c:282:: with SMTP id 124mr14406988wmc.82.1623316685075; Thu, 10 Jun 2021 02:18:05 -0700 (PDT) Received: from abel.fritz.box ([2a02:908:1252:fb60:561f:e43a:edf5:8f95]) by smtp.gmail.com with ESMTPSA id v8sm3087445wrc.29.2021.06.10.02.18.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 02:18:04 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?UTF-8?q?Christian=20K=C3=B6nig?= To: daniel@ffwll.ch, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 4/7] dma-buf: add dma_fence_chain_garbage_collect Date: Thu, 10 Jun 2021 11:17:57 +0200 Message-Id: <20210610091800.1833-5-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210610091800.1833-1-christian.koenig@amd.com> References: <20210610091800.1833-1-christian.koenig@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add some rather sophisticated lockless garbage collection for dma_fence_chain objects. For this keep all initialized dma_fence_chain nodes an a queue and trigger garbage collection before a new one is allocated. Signed-off-by: Christian König --- drivers/dma-buf/dma-fence-chain.c | 160 +++++++++++++++++++++++++----- include/linux/dma-fence-chain.h | 5 + 2 files changed, 142 insertions(+), 23 deletions(-) diff --git a/drivers/dma-buf/dma-fence-chain.c b/drivers/dma-buf/dma-fence-chain.c index 1b4cb3e5cec9..c2f0b69eabb7 100644 --- a/drivers/dma-buf/dma-fence-chain.c +++ b/drivers/dma-buf/dma-fence-chain.c @@ -9,8 +9,53 @@ #include +static struct dma_fence_chain __rcu *fifo_front; +static struct dma_fence_chain __rcu **fifo_back = &fifo_front; + static bool dma_fence_chain_enable_signaling(struct dma_fence *fence); +/** + * dma_fence_chain_enqueue - enqeue a chain node for garbage collection + * @chain: the chain node to enqueue + * + * Add the chain node to the end of the gc fifo. + */ +static void dma_fence_chain_enqueue(struct dma_fence_chain *chain) +{ + struct dma_fence_chain __rcu **tmp; + + RCU_INIT_POINTER(chain->next, NULL); + tmp = xchg((struct dma_fence_chain __force ***)&fifo_back, + &chain->next); + + /* This is intentionally unordered since we only need the fifo for gc */ + rcu_assign_pointer(*tmp, chain); +} + +/** + * dma_fence_chain_dequeue - deqeueue a chain node for garbage collection + * + * Remove the first chain node from the gc fifo while making sure that always + * keep at least one node on the fifo for lockless fifo implementation. + * Returns the dequeued chain node or NULL. + */ +static struct dma_fence_chain *dma_fence_chain_dequeue(void) +{ + struct dma_fence_chain *chain, *tmp; + + rcu_read_lock(); + chain = rcu_dereference(fifo_front); + /* Never dequeue the last chain node for lockless fifo */ + if (unlikely(!chain || !rcu_access_pointer(chain->next))) { + rcu_read_unlock(); + return NULL; + } + tmp = cmpxchg((struct dma_fence_chain __force **)&fifo_front, + chain, rcu_access_pointer(chain->next)); + rcu_read_unlock(); + return tmp == chain ? chain : NULL; +} + /** * dma_fence_chain_get_prev - use RCU to get a reference to the previous fence * @chain: chain node to get the previous node from @@ -28,6 +73,43 @@ static struct dma_fence *dma_fence_chain_get_prev(struct dma_fence_chain *chain) return prev; } +/** + * dma_fence_chain_try_replace - try to replace the prev node + * @chain: Chain node we try to replace prev. + * @prev: the old prev node + * + * Try to replace the previous chain node when it or its containing fence is + * signaled. Returns true if we tried, false if we need to wait. + */ +static bool dma_fence_chain_try_replace(struct dma_fence_chain *chain, + struct dma_fence *prev) +{ + struct dma_fence *replacement, *tmp; + struct dma_fence_chain *prev_chain; + + prev_chain = to_dma_fence_chain(prev); + if (prev_chain) { + if (!dma_fence_is_signaled(prev_chain->fence)) + return false; + + replacement = dma_fence_chain_get_prev(prev_chain); + } else { + if (!dma_fence_is_signaled(prev)) + return false; + + replacement = NULL; + } + + tmp = cmpxchg((struct dma_fence __force **)&chain->prev, prev, + replacement); + if (tmp == prev) + dma_fence_put(tmp); + else + dma_fence_put(replacement); + + return true; +} + /** * dma_fence_chain_walk - chain walking function * @fence: current chain node @@ -38,8 +120,8 @@ static struct dma_fence *dma_fence_chain_get_prev(struct dma_fence_chain *chain) */ struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence) { - struct dma_fence_chain *chain, *prev_chain; - struct dma_fence *prev, *replacement, *tmp; + struct dma_fence_chain *chain; + struct dma_fence *prev; chain = to_dma_fence_chain(fence); if (!chain) { @@ -48,26 +130,8 @@ struct dma_fence *dma_fence_chain_walk(struct dma_fence *fence) } while ((prev = dma_fence_chain_get_prev(chain))) { - - prev_chain = to_dma_fence_chain(prev); - if (prev_chain) { - if (!dma_fence_is_signaled(prev_chain->fence)) - break; - - replacement = dma_fence_chain_get_prev(prev_chain); - } else { - if (!dma_fence_is_signaled(prev)) - break; - - replacement = NULL; - } - - tmp = cmpxchg((struct dma_fence __force **)&chain->prev, - prev, replacement); - if (tmp == prev) - dma_fence_put(tmp); - else - dma_fence_put(replacement); + if (!dma_fence_chain_try_replace(chain, prev)) + break; dma_fence_put(prev); } @@ -205,7 +269,21 @@ static void dma_fence_chain_release(struct dma_fence *fence) dma_fence_put(prev); dma_fence_put(chain->fence); - dma_fence_free(fence); + WRITE_ONCE(chain->fence, NULL); + + /* + * Don't garbage collect here to avoid recursion and potential stack + * overflow. + */ + chain = dma_fence_chain_dequeue(); + if (!chain) + return; + + if (kref_read(&chain->base.refcount) || + READ_ONCE(chain->fence)) + dma_fence_chain_enqueue(chain); + else + dma_fence_free(&chain->base); } const struct dma_fence_ops dma_fence_chain_ops = { @@ -218,6 +296,40 @@ const struct dma_fence_ops dma_fence_chain_ops = { }; EXPORT_SYMBOL(dma_fence_chain_ops); +/** + * dma_fence_chain_garbage_collect - cleanup chain nodes + * + * Do some garbage collection and try to release chain nodes. + */ +void dma_fence_chain_garbage_collect(void) +{ + struct dma_fence_chain *chain = dma_fence_chain_dequeue(); + + if (!chain) + return; + + if (!dma_fence_get_rcu(&chain->base)) { + /* Unused, check if it's also clean */ + if (likely(!READ_ONCE(chain->fence))) { + dma_fence_free(&chain->base); + return; + } + + } else { + struct dma_fence *prev; + + /* Used, do some chain walk */ + prev = dma_fence_chain_get_prev(chain); + if (prev) { + dma_fence_chain_try_replace(chain, prev); + dma_fence_put(prev); + } + dma_fence_put(&chain->base); + } + dma_fence_chain_enqueue(chain); +} +EXPORT_SYMBOL(dma_fence_chain_garbage_collect); + /** * dma_fence_chain_init - initialize a fence chain * @chain: the chain node to initialize @@ -254,5 +366,7 @@ void dma_fence_chain_init(struct dma_fence_chain *chain, dma_fence_init(&chain->base, &dma_fence_chain_ops, &chain->lock, context, seqno); + dma_fence_chain_enqueue(chain); } + EXPORT_SYMBOL(dma_fence_chain_init); diff --git a/include/linux/dma-fence-chain.h b/include/linux/dma-fence-chain.h index 5f45689a6d2e..b412b5396434 100644 --- a/include/linux/dma-fence-chain.h +++ b/include/linux/dma-fence-chain.h @@ -19,6 +19,7 @@ * @base: fence base class * @lock: spinlock for fence handling * @prev: previous fence of the chain + * @next: next chain node for garbage collection * @prev_seqno: original previous seqno before garbage collection * @fence: encapsulated fence * @cb: callback structure for signaling @@ -27,6 +28,7 @@ struct dma_fence_chain { struct dma_fence base; struct dma_fence __rcu *prev; + struct dma_fence_chain __rcu *next; u64 prev_seqno; struct dma_fence *fence; union { @@ -38,6 +40,8 @@ struct dma_fence_chain { extern const struct dma_fence_ops dma_fence_chain_ops; +void dma_fence_chain_garbage_collect(void); + /** * to_dma_fence_chain - cast a fence to a dma_fence_chain * @fence: fence to cast to a dma_fence_array @@ -61,6 +65,7 @@ to_dma_fence_chain(struct dma_fence *fence) */ static inline struct dma_fence_chain *dma_fence_chain_alloc(void) { + dma_fence_chain_garbage_collect(); return kmalloc(sizeof(struct dma_fence_chain), GFP_KERNEL); }; -- 2.25.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63CA8C47094 for ; Thu, 10 Jun 2021 09:18:16 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 301ED61403 for ; Thu, 10 Jun 2021 09:18:16 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 301ED61403 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 16C4D6EC92; Thu, 10 Jun 2021 09:18:08 +0000 (UTC) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 708946EC91; Thu, 10 Jun 2021 09:18:06 +0000 (UTC) Received: by mail-wm1-x32c.google.com with SMTP id f16-20020a05600c1550b02901b00c1be4abso6078138wmg.2; Thu, 10 Jun 2021 02:18:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=BjeLzdv5CJdZ+kjl5N5cB4i0YWorPBc0kCEKEtj36hk=; b=Sa0V5bOX2duti/B0xp/yQlouSvyehGMybsJv64VJbDXSP65dmbnRyBeUk9rLYRRSk5 iHCXkHaDMOiPw6cV09034/j4+RKyZOVY/q2M9BivE8lqrdPBwfCmJCuvJZ1l94H3FKhz o5bfgOaqhPJUVdFD/Gr+ngT4C12kmZ7TKzHtl+VOc+K59LyrE54GbOlOLILykjZqeKAR b/j061/L+rcqEMjBcK/nPSZuP56MuC5hUORRtYNKv8MPhsKFn2IVwaa2FnKMSB16a998 Fm4XOb+v01liBGJeHGwqBOaLmkMHkDetFU5moO2jHD6NFNfQEe0go7gXutxFkZY68HxO 23hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BjeLzdv5CJdZ+kjl5N5cB4i0YWorPBc0kCEKEtj36hk=; b=QPhVIeWjGGbJscaps+M9AP1lTNWWuzVsnCiK1it+QyJrAeZxwd4P2dX2VfsfX3SKpI SONlm9Ed6UoroPc71+ufgHhQkThPxQCITYiQgW6YWFdNdLLHeTI6l/C4GiSfyoTBgA9G x/BuDp2op2Y7lb8C7ad8GZ/PwAPpD2h3txcJKEIzaL5i2Z3rwkv2C69HCP0nqCeK8njN IN1Q+/1IKDCa9Raam4AmPyc3B2Qiq/mJaLrTpvgUFgywhOI/anPm1aOc8TPH8wRpZPWn B89MTZ7qjW8tvA/QaCQVChs74PaSrHxz4L1ES68u4IDMvS9oFJ2s8fxWG3B3sqIUbCm5 uGUw== X-Gm-Message-State: AOAM530+5D/YI0avsQ4IaPwQmzFX2YIbXnuIDzgerijKVVShYkxO67Dm vuWGC6jMlNWO91dQcOSAj4lvodDEOfc= X-Google-Smtp-Source: ABdhPJzJom7o/I8KWbXJK9cMCt6ovyYn3xmDoLbM4QHhtCauCbjwYekHC0PhGdvlWORi9WNqh7h4vQ== X-Received: by 2002:a1c:282:: with SMTP id 124mr14406988wmc.82.1623316685075; Thu, 10 Jun 2021 02:18:05 -0700 (PDT) Received: from abel.fritz.box ([2a02:908:1252:fb60:561f:e43a:edf5:8f95]) by smtp.gmail.com with ESMTPSA id v8sm3087445wrc.29.2021.06.10.02.18.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Jun 2021 02:18:04 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?UTF-8?q?Christian=20K=C3=B6nig?= To: daniel@ffwll.ch, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 4/7] dma-buf: add dma_fence_chain_garbage_collect Date: Thu, 10 Jun 2021 11:17:57 +0200 Message-Id: <20210610091800.1833-5-christian.koenig@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210610091800.1833-1-christian.koenig@amd.com> References: <20210610091800.1833-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" QWRkIHNvbWUgcmF0aGVyIHNvcGhpc3RpY2F0ZWQgbG9ja2xlc3MgZ2FyYmFnZSBjb2xsZWN0aW9u CmZvciBkbWFfZmVuY2VfY2hhaW4gb2JqZWN0cy4KCkZvciB0aGlzIGtlZXAgYWxsIGluaXRpYWxp emVkIGRtYV9mZW5jZV9jaGFpbiBub2RlcyBhbiBhCnF1ZXVlIGFuZCB0cmlnZ2VyIGdhcmJhZ2Ug Y29sbGVjdGlvbiBiZWZvcmUgYSBuZXcgb25lIGlzCmFsbG9jYXRlZC4KClNpZ25lZC1vZmYtYnk6 IENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlhbi5rb2VuaWdAYW1kLmNvbT4KLS0tCiBkcml2ZXJz L2RtYS1idWYvZG1hLWZlbmNlLWNoYWluLmMgfCAxNjAgKysrKysrKysrKysrKysrKysrKysrKysr Ky0tLS0tCiBpbmNsdWRlL2xpbnV4L2RtYS1mZW5jZS1jaGFpbi5oICAgfCAgIDUgKwogMiBmaWxl cyBjaGFuZ2VkLCAxNDIgaW5zZXJ0aW9ucygrKSwgMjMgZGVsZXRpb25zKC0pCgpkaWZmIC0tZ2l0 IGEvZHJpdmVycy9kbWEtYnVmL2RtYS1mZW5jZS1jaGFpbi5jIGIvZHJpdmVycy9kbWEtYnVmL2Rt YS1mZW5jZS1jaGFpbi5jCmluZGV4IDFiNGNiM2U1Y2VjOS4uYzJmMGI2OWVhYmI3IDEwMDY0NAot LS0gYS9kcml2ZXJzL2RtYS1idWYvZG1hLWZlbmNlLWNoYWluLmMKKysrIGIvZHJpdmVycy9kbWEt YnVmL2RtYS1mZW5jZS1jaGFpbi5jCkBAIC05LDggKzksNTMgQEAKIAogI2luY2x1ZGUgPGxpbnV4 L2RtYS1mZW5jZS1jaGFpbi5oPgogCitzdGF0aWMgc3RydWN0IGRtYV9mZW5jZV9jaGFpbiBfX3Jj dSAqZmlmb19mcm9udDsKK3N0YXRpYyBzdHJ1Y3QgZG1hX2ZlbmNlX2NoYWluIF9fcmN1ICoqZmlm b19iYWNrID0gJmZpZm9fZnJvbnQ7CisKIHN0YXRpYyBib29sIGRtYV9mZW5jZV9jaGFpbl9lbmFi bGVfc2lnbmFsaW5nKHN0cnVjdCBkbWFfZmVuY2UgKmZlbmNlKTsKIAorLyoqCisgKiBkbWFfZmVu Y2VfY2hhaW5fZW5xdWV1ZSAtIGVucWV1ZSBhIGNoYWluIG5vZGUgZm9yIGdhcmJhZ2UgY29sbGVj dGlvbgorICogQGNoYWluOiB0aGUgY2hhaW4gbm9kZSB0byBlbnF1ZXVlCisgKgorICogQWRkIHRo ZSBjaGFpbiBub2RlIHRvIHRoZSBlbmQgb2YgdGhlIGdjIGZpZm8uCisgKi8KK3N0YXRpYyB2b2lk IGRtYV9mZW5jZV9jaGFpbl9lbnF1ZXVlKHN0cnVjdCBkbWFfZmVuY2VfY2hhaW4gKmNoYWluKQor eworCXN0cnVjdCBkbWFfZmVuY2VfY2hhaW4gX19yY3UgKip0bXA7CisKKwlSQ1VfSU5JVF9QT0lO VEVSKGNoYWluLT5uZXh0LCBOVUxMKTsKKwl0bXAgPSB4Y2hnKChzdHJ1Y3QgZG1hX2ZlbmNlX2No YWluIF9fZm9yY2UgKioqKSZmaWZvX2JhY2ssCisJCSAgICZjaGFpbi0+bmV4dCk7CisKKwkvKiBU aGlzIGlzIGludGVudGlvbmFsbHkgdW5vcmRlcmVkIHNpbmNlIHdlIG9ubHkgbmVlZCB0aGUgZmlm byBmb3IgZ2MgKi8KKwlyY3VfYXNzaWduX3BvaW50ZXIoKnRtcCwgY2hhaW4pOworfQorCisvKioK KyAqIGRtYV9mZW5jZV9jaGFpbl9kZXF1ZXVlIC0gZGVxZXVldWUgYSBjaGFpbiBub2RlIGZvciBn YXJiYWdlIGNvbGxlY3Rpb24KKyAqCisgKiBSZW1vdmUgdGhlIGZpcnN0IGNoYWluIG5vZGUgZnJv bSB0aGUgZ2MgZmlmbyB3aGlsZSBtYWtpbmcgc3VyZSB0aGF0IGFsd2F5cworICoga2VlcCBhdCBs ZWFzdCBvbmUgbm9kZSBvbiB0aGUgZmlmbyBmb3IgbG9ja2xlc3MgZmlmbyBpbXBsZW1lbnRhdGlv bi4KKyAqIFJldHVybnMgdGhlIGRlcXVldWVkIGNoYWluIG5vZGUgb3IgTlVMTC4KKyAqLworc3Rh dGljIHN0cnVjdCBkbWFfZmVuY2VfY2hhaW4gKmRtYV9mZW5jZV9jaGFpbl9kZXF1ZXVlKHZvaWQp Cit7CisJc3RydWN0IGRtYV9mZW5jZV9jaGFpbiAqY2hhaW4sICp0bXA7CisKKwlyY3VfcmVhZF9s b2NrKCk7CisJY2hhaW4gPSByY3VfZGVyZWZlcmVuY2UoZmlmb19mcm9udCk7CisJLyogTmV2ZXIg ZGVxdWV1ZSB0aGUgbGFzdCBjaGFpbiBub2RlIGZvciBsb2NrbGVzcyBmaWZvICovCisJaWYgKHVu bGlrZWx5KCFjaGFpbiB8fCAhcmN1X2FjY2Vzc19wb2ludGVyKGNoYWluLT5uZXh0KSkpIHsKKwkJ cmN1X3JlYWRfdW5sb2NrKCk7CisJCXJldHVybiBOVUxMOworCX0KKwl0bXAgPSBjbXB4Y2hnKChz dHJ1Y3QgZG1hX2ZlbmNlX2NoYWluIF9fZm9yY2UgKiopJmZpZm9fZnJvbnQsCisJCSAgICAgIGNo YWluLCByY3VfYWNjZXNzX3BvaW50ZXIoY2hhaW4tPm5leHQpKTsKKwlyY3VfcmVhZF91bmxvY2so KTsKKwlyZXR1cm4gdG1wID09IGNoYWluID8gY2hhaW4gOiBOVUxMOworfQorCiAvKioKICAqIGRt YV9mZW5jZV9jaGFpbl9nZXRfcHJldiAtIHVzZSBSQ1UgdG8gZ2V0IGEgcmVmZXJlbmNlIHRvIHRo ZSBwcmV2aW91cyBmZW5jZQogICogQGNoYWluOiBjaGFpbiBub2RlIHRvIGdldCB0aGUgcHJldmlv dXMgbm9kZSBmcm9tCkBAIC0yOCw2ICs3Myw0MyBAQCBzdGF0aWMgc3RydWN0IGRtYV9mZW5jZSAq ZG1hX2ZlbmNlX2NoYWluX2dldF9wcmV2KHN0cnVjdCBkbWFfZmVuY2VfY2hhaW4gKmNoYWluKQog CXJldHVybiBwcmV2OwogfQogCisvKioKKyAqIGRtYV9mZW5jZV9jaGFpbl90cnlfcmVwbGFjZSAt IHRyeSB0byByZXBsYWNlIHRoZSBwcmV2IG5vZGUKKyAqIEBjaGFpbjogQ2hhaW4gbm9kZSB3ZSB0 cnkgdG8gcmVwbGFjZSBwcmV2LgorICogQHByZXY6IHRoZSBvbGQgcHJldiBub2RlCisgKgorICog VHJ5IHRvIHJlcGxhY2UgdGhlIHByZXZpb3VzIGNoYWluIG5vZGUgd2hlbiBpdCBvciBpdHMgY29u dGFpbmluZyBmZW5jZSBpcworICogc2lnbmFsZWQuIFJldHVybnMgdHJ1ZSBpZiB3ZSB0cmllZCwg ZmFsc2UgaWYgd2UgbmVlZCB0byB3YWl0LgorICovCitzdGF0aWMgYm9vbCBkbWFfZmVuY2VfY2hh aW5fdHJ5X3JlcGxhY2Uoc3RydWN0IGRtYV9mZW5jZV9jaGFpbiAqY2hhaW4sCisJCQkJCXN0cnVj dCBkbWFfZmVuY2UgKnByZXYpCit7CisJc3RydWN0IGRtYV9mZW5jZSAqcmVwbGFjZW1lbnQsICp0 bXA7CisJc3RydWN0IGRtYV9mZW5jZV9jaGFpbiAqcHJldl9jaGFpbjsKKworCXByZXZfY2hhaW4g PSB0b19kbWFfZmVuY2VfY2hhaW4ocHJldik7CisJaWYgKHByZXZfY2hhaW4pIHsKKwkJaWYgKCFk bWFfZmVuY2VfaXNfc2lnbmFsZWQocHJldl9jaGFpbi0+ZmVuY2UpKQorCQkJcmV0dXJuIGZhbHNl OworCisJCXJlcGxhY2VtZW50ID0gZG1hX2ZlbmNlX2NoYWluX2dldF9wcmV2KHByZXZfY2hhaW4p OworCX0gZWxzZSB7CisJCWlmICghZG1hX2ZlbmNlX2lzX3NpZ25hbGVkKHByZXYpKQorCQkJcmV0 dXJuIGZhbHNlOworCisJCXJlcGxhY2VtZW50ID0gTlVMTDsKKwl9CisKKwl0bXAgPSBjbXB4Y2hn KChzdHJ1Y3QgZG1hX2ZlbmNlIF9fZm9yY2UgKiopJmNoYWluLT5wcmV2LCBwcmV2LAorCQkgICAg ICByZXBsYWNlbWVudCk7CisJaWYgKHRtcCA9PSBwcmV2KQorCQlkbWFfZmVuY2VfcHV0KHRtcCk7 CisJZWxzZQorCQlkbWFfZmVuY2VfcHV0KHJlcGxhY2VtZW50KTsKKworCXJldHVybiB0cnVlOwor fQorCiAvKioKICAqIGRtYV9mZW5jZV9jaGFpbl93YWxrIC0gY2hhaW4gd2Fsa2luZyBmdW5jdGlv bgogICogQGZlbmNlOiBjdXJyZW50IGNoYWluIG5vZGUKQEAgLTM4LDggKzEyMCw4IEBAIHN0YXRp YyBzdHJ1Y3QgZG1hX2ZlbmNlICpkbWFfZmVuY2VfY2hhaW5fZ2V0X3ByZXYoc3RydWN0IGRtYV9m ZW5jZV9jaGFpbiAqY2hhaW4pCiAgKi8KIHN0cnVjdCBkbWFfZmVuY2UgKmRtYV9mZW5jZV9jaGFp bl93YWxrKHN0cnVjdCBkbWFfZmVuY2UgKmZlbmNlKQogewotCXN0cnVjdCBkbWFfZmVuY2VfY2hh aW4gKmNoYWluLCAqcHJldl9jaGFpbjsKLQlzdHJ1Y3QgZG1hX2ZlbmNlICpwcmV2LCAqcmVwbGFj ZW1lbnQsICp0bXA7CisJc3RydWN0IGRtYV9mZW5jZV9jaGFpbiAqY2hhaW47CisJc3RydWN0IGRt YV9mZW5jZSAqcHJldjsKIAogCWNoYWluID0gdG9fZG1hX2ZlbmNlX2NoYWluKGZlbmNlKTsKIAlp ZiAoIWNoYWluKSB7CkBAIC00OCwyNiArMTMwLDggQEAgc3RydWN0IGRtYV9mZW5jZSAqZG1hX2Zl bmNlX2NoYWluX3dhbGsoc3RydWN0IGRtYV9mZW5jZSAqZmVuY2UpCiAJfQogCiAJd2hpbGUgKChw cmV2ID0gZG1hX2ZlbmNlX2NoYWluX2dldF9wcmV2KGNoYWluKSkpIHsKLQotCQlwcmV2X2NoYWlu ID0gdG9fZG1hX2ZlbmNlX2NoYWluKHByZXYpOwotCQlpZiAocHJldl9jaGFpbikgewotCQkJaWYg KCFkbWFfZmVuY2VfaXNfc2lnbmFsZWQocHJldl9jaGFpbi0+ZmVuY2UpKQotCQkJCWJyZWFrOwot Ci0JCQlyZXBsYWNlbWVudCA9IGRtYV9mZW5jZV9jaGFpbl9nZXRfcHJldihwcmV2X2NoYWluKTsK LQkJfSBlbHNlIHsKLQkJCWlmICghZG1hX2ZlbmNlX2lzX3NpZ25hbGVkKHByZXYpKQotCQkJCWJy ZWFrOwotCi0JCQlyZXBsYWNlbWVudCA9IE5VTEw7Ci0JCX0KLQotCQl0bXAgPSBjbXB4Y2hnKChz dHJ1Y3QgZG1hX2ZlbmNlIF9fZm9yY2UgKiopJmNoYWluLT5wcmV2LAotCQkJICAgICAgcHJldiwg cmVwbGFjZW1lbnQpOwotCQlpZiAodG1wID09IHByZXYpCi0JCQlkbWFfZmVuY2VfcHV0KHRtcCk7 Ci0JCWVsc2UKLQkJCWRtYV9mZW5jZV9wdXQocmVwbGFjZW1lbnQpOworCQlpZiAoIWRtYV9mZW5j ZV9jaGFpbl90cnlfcmVwbGFjZShjaGFpbiwgcHJldikpCisJCQlicmVhazsKIAkJZG1hX2ZlbmNl X3B1dChwcmV2KTsKIAl9CiAKQEAgLTIwNSw3ICsyNjksMjEgQEAgc3RhdGljIHZvaWQgZG1hX2Zl bmNlX2NoYWluX3JlbGVhc2Uoc3RydWN0IGRtYV9mZW5jZSAqZmVuY2UpCiAJZG1hX2ZlbmNlX3B1 dChwcmV2KTsKIAogCWRtYV9mZW5jZV9wdXQoY2hhaW4tPmZlbmNlKTsKLQlkbWFfZmVuY2VfZnJl ZShmZW5jZSk7CisJV1JJVEVfT05DRShjaGFpbi0+ZmVuY2UsIE5VTEwpOworCisJLyoKKwkgKiBE b24ndCBnYXJiYWdlIGNvbGxlY3QgaGVyZSB0byBhdm9pZCByZWN1cnNpb24gYW5kIHBvdGVudGlh bCBzdGFjaworCSAqIG92ZXJmbG93LgorCSAqLworCWNoYWluID0gZG1hX2ZlbmNlX2NoYWluX2Rl cXVldWUoKTsKKwlpZiAoIWNoYWluKQorCQlyZXR1cm47CisKKwlpZiAoa3JlZl9yZWFkKCZjaGFp bi0+YmFzZS5yZWZjb3VudCkgfHwKKwkgICAgUkVBRF9PTkNFKGNoYWluLT5mZW5jZSkpCisJCWRt YV9mZW5jZV9jaGFpbl9lbnF1ZXVlKGNoYWluKTsKKwllbHNlCisJCWRtYV9mZW5jZV9mcmVlKCZj aGFpbi0+YmFzZSk7CiB9CiAKIGNvbnN0IHN0cnVjdCBkbWFfZmVuY2Vfb3BzIGRtYV9mZW5jZV9j aGFpbl9vcHMgPSB7CkBAIC0yMTgsNiArMjk2LDQwIEBAIGNvbnN0IHN0cnVjdCBkbWFfZmVuY2Vf b3BzIGRtYV9mZW5jZV9jaGFpbl9vcHMgPSB7CiB9OwogRVhQT1JUX1NZTUJPTChkbWFfZmVuY2Vf Y2hhaW5fb3BzKTsKIAorLyoqCisgKiBkbWFfZmVuY2VfY2hhaW5fZ2FyYmFnZV9jb2xsZWN0IC0g Y2xlYW51cCBjaGFpbiBub2RlcworICoKKyAqIERvIHNvbWUgZ2FyYmFnZSBjb2xsZWN0aW9uIGFu ZCB0cnkgdG8gcmVsZWFzZSBjaGFpbiBub2Rlcy4KKyAqLwordm9pZCBkbWFfZmVuY2VfY2hhaW5f Z2FyYmFnZV9jb2xsZWN0KHZvaWQpCit7CisJc3RydWN0IGRtYV9mZW5jZV9jaGFpbiAqY2hhaW4g PSBkbWFfZmVuY2VfY2hhaW5fZGVxdWV1ZSgpOworCisJaWYgKCFjaGFpbikKKwkJcmV0dXJuOwor CisJaWYgKCFkbWFfZmVuY2VfZ2V0X3JjdSgmY2hhaW4tPmJhc2UpKSB7CisJCS8qIFVudXNlZCwg Y2hlY2sgaWYgaXQncyBhbHNvIGNsZWFuICovCisJCWlmIChsaWtlbHkoIVJFQURfT05DRShjaGFp bi0+ZmVuY2UpKSkgeworCQkJZG1hX2ZlbmNlX2ZyZWUoJmNoYWluLT5iYXNlKTsKKwkJCXJldHVy bjsKKwkJfQorCisJfSBlbHNlIHsKKwkJc3RydWN0IGRtYV9mZW5jZSAqcHJldjsKKworCQkvKiBV c2VkLCBkbyBzb21lIGNoYWluIHdhbGsgKi8KKwkJcHJldiA9IGRtYV9mZW5jZV9jaGFpbl9nZXRf cHJldihjaGFpbik7CisJCWlmIChwcmV2KSB7CisJCQlkbWFfZmVuY2VfY2hhaW5fdHJ5X3JlcGxh Y2UoY2hhaW4sIHByZXYpOworCQkJZG1hX2ZlbmNlX3B1dChwcmV2KTsKKwkJfQorCQlkbWFfZmVu Y2VfcHV0KCZjaGFpbi0+YmFzZSk7CisJfQorCWRtYV9mZW5jZV9jaGFpbl9lbnF1ZXVlKGNoYWlu KTsKK30KK0VYUE9SVF9TWU1CT0woZG1hX2ZlbmNlX2NoYWluX2dhcmJhZ2VfY29sbGVjdCk7CisK IC8qKgogICogZG1hX2ZlbmNlX2NoYWluX2luaXQgLSBpbml0aWFsaXplIGEgZmVuY2UgY2hhaW4K ICAqIEBjaGFpbjogdGhlIGNoYWluIG5vZGUgdG8gaW5pdGlhbGl6ZQpAQCAtMjU0LDUgKzM2Niw3 IEBAIHZvaWQgZG1hX2ZlbmNlX2NoYWluX2luaXQoc3RydWN0IGRtYV9mZW5jZV9jaGFpbiAqY2hh aW4sCiAKIAlkbWFfZmVuY2VfaW5pdCgmY2hhaW4tPmJhc2UsICZkbWFfZmVuY2VfY2hhaW5fb3Bz LAogCQkgICAgICAgJmNoYWluLT5sb2NrLCBjb250ZXh0LCBzZXFubyk7CisJZG1hX2ZlbmNlX2No YWluX2VucXVldWUoY2hhaW4pOwogfQorCiBFWFBPUlRfU1lNQk9MKGRtYV9mZW5jZV9jaGFpbl9p bml0KTsKZGlmZiAtLWdpdCBhL2luY2x1ZGUvbGludXgvZG1hLWZlbmNlLWNoYWluLmggYi9pbmNs dWRlL2xpbnV4L2RtYS1mZW5jZS1jaGFpbi5oCmluZGV4IDVmNDU2ODlhNmQyZS4uYjQxMmI1Mzk2 NDM0IDEwMDY0NAotLS0gYS9pbmNsdWRlL2xpbnV4L2RtYS1mZW5jZS1jaGFpbi5oCisrKyBiL2lu Y2x1ZGUvbGludXgvZG1hLWZlbmNlLWNoYWluLmgKQEAgLTE5LDYgKzE5LDcgQEAKICAqIEBiYXNl OiBmZW5jZSBiYXNlIGNsYXNzCiAgKiBAbG9jazogc3BpbmxvY2sgZm9yIGZlbmNlIGhhbmRsaW5n CiAgKiBAcHJldjogcHJldmlvdXMgZmVuY2Ugb2YgdGhlIGNoYWluCisgKiBAbmV4dDogbmV4dCBj aGFpbiBub2RlIGZvciBnYXJiYWdlIGNvbGxlY3Rpb24KICAqIEBwcmV2X3NlcW5vOiBvcmlnaW5h bCBwcmV2aW91cyBzZXFubyBiZWZvcmUgZ2FyYmFnZSBjb2xsZWN0aW9uCiAgKiBAZmVuY2U6IGVu Y2Fwc3VsYXRlZCBmZW5jZQogICogQGNiOiBjYWxsYmFjayBzdHJ1Y3R1cmUgZm9yIHNpZ25hbGlu ZwpAQCAtMjcsNiArMjgsNyBAQAogc3RydWN0IGRtYV9mZW5jZV9jaGFpbiB7CiAJc3RydWN0IGRt YV9mZW5jZSBiYXNlOwogCXN0cnVjdCBkbWFfZmVuY2UgX19yY3UgKnByZXY7CisJc3RydWN0IGRt YV9mZW5jZV9jaGFpbiBfX3JjdSAqbmV4dDsKIAl1NjQgcHJldl9zZXFubzsKIAlzdHJ1Y3QgZG1h X2ZlbmNlICpmZW5jZTsKIAl1bmlvbiB7CkBAIC0zOCw2ICs0MCw4IEBAIHN0cnVjdCBkbWFfZmVu Y2VfY2hhaW4gewogCiBleHRlcm4gY29uc3Qgc3RydWN0IGRtYV9mZW5jZV9vcHMgZG1hX2ZlbmNl X2NoYWluX29wczsKIAordm9pZCBkbWFfZmVuY2VfY2hhaW5fZ2FyYmFnZV9jb2xsZWN0KHZvaWQp OworCiAvKioKICAqIHRvX2RtYV9mZW5jZV9jaGFpbiAtIGNhc3QgYSBmZW5jZSB0byBhIGRtYV9m ZW5jZV9jaGFpbgogICogQGZlbmNlOiBmZW5jZSB0byBjYXN0IHRvIGEgZG1hX2ZlbmNlX2FycmF5 CkBAIC02MSw2ICs2NSw3IEBAIHRvX2RtYV9mZW5jZV9jaGFpbihzdHJ1Y3QgZG1hX2ZlbmNlICpm ZW5jZSkKICAqLwogc3RhdGljIGlubGluZSBzdHJ1Y3QgZG1hX2ZlbmNlX2NoYWluICpkbWFfZmVu Y2VfY2hhaW5fYWxsb2Modm9pZCkKIHsKKwlkbWFfZmVuY2VfY2hhaW5fZ2FyYmFnZV9jb2xsZWN0 KCk7CiAJcmV0dXJuIGttYWxsb2Moc2l6ZW9mKHN0cnVjdCBkbWFfZmVuY2VfY2hhaW4pLCBHRlBf S0VSTkVMKTsKIH07CiAKLS0gCjIuMjUuMQoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX18KYW1kLWdmeCBtYWlsaW5nIGxpc3QKYW1kLWdmeEBsaXN0cy5mcmVl ZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m by9hbWQtZ2Z4Cg==