From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6845DC433E0 for ; Wed, 3 Feb 2021 08:50:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 128A264F68 for ; Wed, 3 Feb 2021 08:50:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232991AbhBCItr (ORCPT ); Wed, 3 Feb 2021 03:49:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38136 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232637AbhBCItb (ORCPT ); Wed, 3 Feb 2021 03:49:31 -0500 Received: from mail-ot1-x32d.google.com (mail-ot1-x32d.google.com [IPv6:2607:f8b0:4864:20::32d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4354EC061573 for ; Wed, 3 Feb 2021 00:48:51 -0800 (PST) Received: by mail-ot1-x32d.google.com with SMTP id d7so22659236otf.3 for ; Wed, 03 Feb 2021 00:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=VwNp0a42H2HJdUXf2U+NyyGMLQI597ulmFnoEoIfRg8=; b=bp4qJgQbbTLWXuU91TsSZAOI41pytifc2W9dIiVN6dXquO6uybAKvlcCPaPH1NpkHd P0Mq76vN/q5u7f/lrfgPGRqu2/rWYgb0PAnpfpAeEQ4C/t0wMWRE7aOx1JX0e82Xot46 WQy0xA2TsNhyd9sdod7biEaozcsh5EHAWKcec= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=VwNp0a42H2HJdUXf2U+NyyGMLQI597ulmFnoEoIfRg8=; b=GKE5b/3ac2Nc4AW/M6+JxtwwPvf6zR5WUKGHeqZmHKf+VXVDtfGkXr4KgBBaKCPn1z l1AqIwAYdEkG3Aj/NlD4SmTfYNxqjrZ4IA0YNpGx/ISNTLgoHctIKSBHFqaXccNWR0xs eOV3CTsRY9AM9f8LFNhGIpP9VPY5Tfk7eADfWrGDijSDbTZxBtCvL66ojsO7ug58wC5i RHMAqcD10HN779gWfAva4bguZSi6+cG5IMXgtoKMJ42Zd+BVhZdoepqPRCvmi0x+CX2s 2tkn6ZeFianj2GBTh4+yQ+0ohXjCdlwenKkM4uZ1jPtsHv1WehHTJouO2IBhjJh8iw33 HmUA== X-Gm-Message-State: AOAM531HeQt49sjrgQ3LzA7rVwjsKk9O/15OXN6KF8Yu+BVtu89hkGPQ 1aRLlY9711dJPgpWHzpIA1pZ/tdgI2clq5dyXGoe4w== X-Google-Smtp-Source: ABdhPJwFJFhQYQVJPg8AcL5xpSmXQT4nU/vhOSOoGYxm48EBQ+vEBz006Z/ZUAjAOMfXzIp+7+twjGV9i1z1MNucNfM= X-Received: by 2002:a9d:6c96:: with SMTP id c22mr1296084otr.303.1612342130615; Wed, 03 Feb 2021 00:48:50 -0800 (PST) MIME-Version: 1.0 References: <58e41b62-b8e0-b036-c87d-a84d53f5a26e@amd.com> In-Reply-To: <58e41b62-b8e0-b036-c87d-a84d53f5a26e@amd.com> From: Daniel Vetter Date: Wed, 3 Feb 2021 09:48:39 +0100 Message-ID: Subject: Re: [amdgpu] deadlock To: =?UTF-8?Q?Christian_K=C3=B6nig?= Cc: Daniel Gomez , amd-gfx list , dri-devel , Alex Deucher , Linux Kernel Mailing List Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Feb 3, 2021 at 9:36 AM Christian K=C3=B6nig wrote: > > Hi Daniel, > > this is not a deadlock, but rather a hardware lockup. Are you sure? Ime getting stuck in dma_fence_wait has generally good chance of being a dma_fence deadlock. GPU hang should never result in a forever stuck dma_fence. Daniel, can you pls re-hang your machine and then dump backtraces of all tasks into dmesg with sysrq-t, and then attach that? Without all the backtraces it's tricky to construct the full dependency chain of what's going on. Also is this plain -rc6, not some more patches on top? -Daniel > Which OpenCl stack are you using? > > Regards, > Christian. > > Am 03.02.21 um 09:33 schrieb Daniel Gomez: > > Hi all, > > > > I have a deadlock with the amdgpu mainline driver when running in paral= lel two > > OpenCL applications. So far, we've been able to replicate it easily by = executing > > clinfo and MatrixMultiplication (from AMD opencl-samples). It's quite o= ld the > > opencl-samples so, if you have any other suggestion for testing I'd be = very > > happy to test it as well. > > > > How to replicate the issue: > > > > # while true; do /usr/bin/MatrixMultiplication --device gpu \ > > --deviceId 0 -x 1000 -y 1000 -z 1000 -q -t -i 50; done > > # while true; do clinfo; done > > > > Output: > > > > After a minute or less (sometimes could be more) I can see that > > MatrixMultiplication and clinfo hang. In addition, with radeontop you c= an see > > how the Graphics pipe goes from ~50% to 100%. Also the shader clocks > > goes up from ~35% to ~96%. > > > > clinfo keeps printing: > > ioctl(7, DRM_IOCTL_SYNCOBJ_WAIT, 0x7ffe46e5f950) =3D -1 ETIME (Timer ex= pired) > > > > And MatrixMultiplication prints the following (strace) if you try to > > kill the process: > > > > sched_yield() =3D 0 > > futex(0x557e945343b8, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0= , > > NULL, FUTEX_BITSET_MATCH_ANYstrace: Process 651 detached > > > > > > After this, the gpu is not functional at all and you'd need a power cyc= le reset > > to restore the system. > > > > Hardware info: > > CPU: AMD Ryzen Embedded V1605B with Radeon Vega Gfx (8) @ 2.000GHz > > GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series > > > > 03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. > > [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] > > (rev 83) > > DeviceName: Broadcom 5762 > > Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge > > [Radeon Vega Series / Radeon Vega Mobile Series] > > Kernel driver in use: amdgpu > > Kernel modules: amdgpu > > > > Linux kernel info: > > > > root@qt5222:~# uname -a > > Linux qt5222 5.11.0-rc6-qtec-standard #2 SMP Tue Feb 2 09:41:46 UTC > > 2021 x86_64 x86_64 x86_64 GNU/Linux > > > > By enabling the kernel locks stats I could see the MatrixMultiplication= is > > hanged in the amdgpu_mn_invalidate_gfx function: > > > > [ 738.359202] 1 lock held by MatrixMultiplic/653: > > [ 738.359206] #0: ffff88810e364fe0 > > (&adev->notifier_lock){+.+.}-{3:3}, at: > > amdgpu_mn_invalidate_gfx+0x34/0xa0 [amdgpu] > > > > I can see in the the amdgpu_mn_invalidate_gfx function: the > > dma_resv_wait_timeout_rcu uses wait_all (fences) and MAX_SCHEDULE_TIMEO= UT so, I > > guess the code gets stuck there waiting forever. According to the > > documentation: "When somebody tries to invalidate the page tables we bl= ock the > > update until all operations on the pages in question are completed, the= n those > > pages are marked as accessed and also dirty if it wasn=E2=80=99t a rea= d only access." > > Looks like the fences are deadlocked and therefore, it never returns. C= ould it > > be possible? any hint to where can I look to fix this? > > > > Thank you in advance. > > > > Here the full dmesg output: > > > > [ 738.337726] INFO: task MatrixMultiplic:653 blocked for more than 122= seconds. > > [ 738.344937] Not tainted 5.11.0-rc6-qtec-standard #2 > > [ 738.350384] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > > disables this message. > > [ 738.358240] task:MatrixMultiplic state:D stack: 0 pid: 653 > > ppid: 1 flags:0x00004000 > > [ 738.358254] Call Trace: > > [ 738.358261] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358276] __schedule+0x370/0x960 > > [ 738.358291] ? dma_fence_default_wait+0x117/0x230 > > [ 738.358297] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358305] schedule+0x51/0xc0 > > [ 738.358312] schedule_timeout+0x275/0x380 > > [ 738.358324] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358332] ? mark_held_locks+0x4f/0x70 > > [ 738.358341] ? dma_fence_default_wait+0x117/0x230 > > [ 738.358347] ? lockdep_hardirqs_on_prepare+0xd4/0x180 > > [ 738.358353] ? _raw_spin_unlock_irqrestore+0x39/0x40 > > [ 738.358362] ? dma_fence_default_wait+0x117/0x230 > > [ 738.358370] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358375] dma_fence_default_wait+0x214/0x230 > > [ 738.358384] ? dma_fence_release+0x1a0/0x1a0 > > [ 738.358396] dma_fence_wait_timeout+0x105/0x200 > > [ 738.358405] dma_resv_wait_timeout_rcu+0x1aa/0x5e0 > > [ 738.358421] amdgpu_mn_invalidate_gfx+0x55/0xa0 [amdgpu] > > [ 738.358688] __mmu_notifier_release+0x1bb/0x210 > > [ 738.358710] exit_mmap+0x2f/0x1e0 > > [ 738.358723] ? find_held_lock+0x34/0xa0 > > [ 738.358746] mmput+0x39/0xe0 > > [ 738.358756] do_exit+0x5c3/0xc00 > > [ 738.358763] ? find_held_lock+0x34/0xa0 > > [ 738.358780] do_group_exit+0x47/0xb0 > > [ 738.358791] get_signal+0x15b/0xc50 > > [ 738.358807] arch_do_signal_or_restart+0xaf/0x710 > > [ 738.358816] ? lockdep_hardirqs_on_prepare+0xd4/0x180 > > [ 738.358822] ? _raw_spin_unlock_irqrestore+0x39/0x40 > > [ 738.358831] ? ktime_get_mono_fast_ns+0x50/0xa0 > > [ 738.358844] ? amdgpu_drm_ioctl+0x6b/0x80 [amdgpu] > > [ 738.359044] exit_to_user_mode_prepare+0xf2/0x1b0 > > [ 738.359054] syscall_exit_to_user_mode+0x19/0x60 > > [ 738.359062] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 738.359069] RIP: 0033:0x7f6b89a51887 > > [ 738.359076] RSP: 002b:00007f6b82b54b18 EFLAGS: 00000246 ORIG_RAX: > > 0000000000000010 > > [ 738.359086] RAX: fffffffffffffe00 RBX: 00007f6b82b54b50 RCX: 00007f6= b89a51887 > > [ 738.359091] RDX: 00007f6b82b54b50 RSI: 00000000c02064c3 RDI: 0000000= 000000007 > > [ 738.359096] RBP: 00000000c02064c3 R08: 0000000000000003 R09: 00007f6= b82b54bbc > > [ 738.359101] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000= 165a0bc00 > > [ 738.359106] R13: 0000000000000007 R14: 0000000000000001 R15: 0000000= 000000000 > > [ 738.359129] > > Showing all locks held in the system: > > [ 738.359141] 1 lock held by khungtaskd/54: > > [ 738.359148] #0: ffffffff829f6840 (rcu_read_lock){....}-{1:2}, at: > > debug_show_all_locks+0x15/0x183 > > [ 738.359187] 1 lock held by systemd-journal/174: > > [ 738.359202] 1 lock held by MatrixMultiplic/653: > > [ 738.359206] #0: ffff88810e364fe0 > > (&adev->notifier_lock){+.+.}-{3:3}, at: > > amdgpu_mn_invalidate_gfx+0x34/0xa0 [amdgpu] > > > > Daniel > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel --=20 Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4A523C433DB for ; Wed, 3 Feb 2021 08:48:54 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8489264F6C for ; Wed, 3 Feb 2021 08:48:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8489264F6C Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A55CE6E9DC; Wed, 3 Feb 2021 08:48:52 +0000 (UTC) Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5C4646E9DC for ; Wed, 3 Feb 2021 08:48:51 +0000 (UTC) Received: by mail-ot1-x331.google.com with SMTP id d1so22608946otl.13 for ; Wed, 03 Feb 2021 00:48:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=VwNp0a42H2HJdUXf2U+NyyGMLQI597ulmFnoEoIfRg8=; b=bp4qJgQbbTLWXuU91TsSZAOI41pytifc2W9dIiVN6dXquO6uybAKvlcCPaPH1NpkHd P0Mq76vN/q5u7f/lrfgPGRqu2/rWYgb0PAnpfpAeEQ4C/t0wMWRE7aOx1JX0e82Xot46 WQy0xA2TsNhyd9sdod7biEaozcsh5EHAWKcec= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=VwNp0a42H2HJdUXf2U+NyyGMLQI597ulmFnoEoIfRg8=; b=RKs9QMm+Yzqnl3pypMA1XroYbCSEoAnP1/XEmexTrRottEGIuKMGOrCQEdoLFTU2Hq lPXxEUeFrMss+N+H1bCXs3Wn57+nG35avr9u7X3DObflldaEzeY7eJ1Q1O4AeJyWpy7b Msw2ghd8ds6vWdrudzJ8fgNulxhExoRmRPDriwbr4Vvl93p6fjEkyBCGkx1B6a6eUp3T CJHC7zdF1NWkS+O7Qo+8OyaJ3WQZ2VpwK0uWRyjl7bFIHWjw7bl2Bx4KwAPiSrVtuhcl DQXfKgzzH+d2wX0cI/1Pu2psgV1FpMb9xttQKHCVgR8bzwPRhkFLwzpHDAwPEheXd6wc 8VSQ== X-Gm-Message-State: AOAM531XCLuRH3tnAnhcnEm0ciIh37CQhbz6T5xiMRzrIYTYUQZhWORW WtIUKXUuG/zFKV/nXZ44O+6KPIW0ReFKam/yiqNt5A== X-Google-Smtp-Source: ABdhPJwFJFhQYQVJPg8AcL5xpSmXQT4nU/vhOSOoGYxm48EBQ+vEBz006Z/ZUAjAOMfXzIp+7+twjGV9i1z1MNucNfM= X-Received: by 2002:a9d:6c96:: with SMTP id c22mr1296084otr.303.1612342130615; Wed, 03 Feb 2021 00:48:50 -0800 (PST) MIME-Version: 1.0 References: <58e41b62-b8e0-b036-c87d-a84d53f5a26e@amd.com> In-Reply-To: <58e41b62-b8e0-b036-c87d-a84d53f5a26e@amd.com> From: Daniel Vetter Date: Wed, 3 Feb 2021 09:48:39 +0100 Message-ID: Subject: Re: [amdgpu] deadlock To: =?UTF-8?Q?Christian_K=C3=B6nig?= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , Daniel Gomez , dri-devel , amd-gfx list , Linux Kernel Mailing List Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" T24gV2VkLCBGZWIgMywgMjAyMSBhdCA5OjM2IEFNIENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlh bi5rb2VuaWdAYW1kLmNvbT4gd3JvdGU6Cj4KPiBIaSBEYW5pZWwsCj4KPiB0aGlzIGlzIG5vdCBh IGRlYWRsb2NrLCBidXQgcmF0aGVyIGEgaGFyZHdhcmUgbG9ja3VwLgoKQXJlIHlvdSBzdXJlPyBJ bWUgZ2V0dGluZyBzdHVjayBpbiBkbWFfZmVuY2Vfd2FpdCBoYXMgZ2VuZXJhbGx5IGdvb2QKY2hh bmNlIG9mIGJlaW5nIGEgZG1hX2ZlbmNlIGRlYWRsb2NrLiBHUFUgaGFuZyBzaG91bGQgbmV2ZXIg cmVzdWx0IGluCmEgZm9yZXZlciBzdHVjayBkbWFfZmVuY2UuCgpEYW5pZWwsIGNhbiB5b3UgcGxz IHJlLWhhbmcgeW91ciBtYWNoaW5lIGFuZCB0aGVuIGR1bXAgYmFja3RyYWNlcyBvZgphbGwgdGFz a3MgaW50byBkbWVzZyB3aXRoIHN5c3JxLXQsIGFuZCB0aGVuIGF0dGFjaCB0aGF0PyBXaXRob3V0 IGFsbAp0aGUgYmFja3RyYWNlcyBpdCdzIHRyaWNreSB0byBjb25zdHJ1Y3QgdGhlIGZ1bGwgZGVw ZW5kZW5jeSBjaGFpbiBvZgp3aGF0J3MgZ29pbmcgb24uIEFsc28gaXMgdGhpcyBwbGFpbiAtcmM2 LCBub3Qgc29tZSBtb3JlIHBhdGNoZXMgb24KdG9wPwotRGFuaWVsCgo+IFdoaWNoIE9wZW5DbCBz dGFjayBhcmUgeW91IHVzaW5nPwo+Cj4gUmVnYXJkcywKPiBDaHJpc3RpYW4uCj4KPiBBbSAwMy4w Mi4yMSB1bSAwOTozMyBzY2hyaWViIERhbmllbCBHb21lejoKPiA+IEhpIGFsbCwKPiA+Cj4gPiBJ IGhhdmUgYSBkZWFkbG9jayB3aXRoIHRoZSBhbWRncHUgbWFpbmxpbmUgZHJpdmVyIHdoZW4gcnVu bmluZyBpbiBwYXJhbGxlbCB0d28KPiA+IE9wZW5DTCBhcHBsaWNhdGlvbnMuIFNvIGZhciwgd2Un dmUgYmVlbiBhYmxlIHRvIHJlcGxpY2F0ZSBpdCBlYXNpbHkgYnkgZXhlY3V0aW5nCj4gPiBjbGlu Zm8gYW5kIE1hdHJpeE11bHRpcGxpY2F0aW9uIChmcm9tIEFNRCBvcGVuY2wtc2FtcGxlcykuIEl0 J3MgcXVpdGUgb2xkIHRoZQo+ID4gb3BlbmNsLXNhbXBsZXMgc28sIGlmIHlvdSBoYXZlIGFueSBv dGhlciBzdWdnZXN0aW9uIGZvciB0ZXN0aW5nIEknZCBiZSB2ZXJ5Cj4gPiBoYXBweSB0byB0ZXN0 IGl0IGFzIHdlbGwuCj4gPgo+ID4gSG93IHRvIHJlcGxpY2F0ZSB0aGUgaXNzdWU6Cj4gPgo+ID4g IyB3aGlsZSB0cnVlOyBkbyAvdXNyL2Jpbi9NYXRyaXhNdWx0aXBsaWNhdGlvbiAtLWRldmljZSBn cHUgXAo+ID4gICAgICAtLWRldmljZUlkIDAgLXggMTAwMCAteSAxMDAwIC16IDEwMDAgLXEgLXQg LWkgNTA7IGRvbmUKPiA+ICMgd2hpbGUgdHJ1ZTsgZG8gY2xpbmZvOyBkb25lCj4gPgo+ID4gT3V0 cHV0Ogo+ID4KPiA+IEFmdGVyIGEgbWludXRlIG9yIGxlc3MgKHNvbWV0aW1lcyBjb3VsZCBiZSBt b3JlKSBJIGNhbiBzZWUgdGhhdAo+ID4gTWF0cml4TXVsdGlwbGljYXRpb24gYW5kIGNsaW5mbyBo YW5nLiBJbiBhZGRpdGlvbiwgd2l0aCByYWRlb250b3AgeW91IGNhbiBzZWUKPiA+IGhvdyB0aGUg R3JhcGhpY3MgcGlwZSBnb2VzIGZyb20gfjUwJSB0byAxMDAlLiBBbHNvIHRoZSBzaGFkZXIgY2xv Y2tzCj4gPiBnb2VzIHVwIGZyb20gfjM1JSB0byB+OTYlLgo+ID4KPiA+IGNsaW5mbyBrZWVwcyBw cmludGluZzoKPiA+IGlvY3RsKDcsIERSTV9JT0NUTF9TWU5DT0JKX1dBSVQsIDB4N2ZmZTQ2ZTVm OTUwKSA9IC0xIEVUSU1FIChUaW1lciBleHBpcmVkKQo+ID4KPiA+IEFuZCBNYXRyaXhNdWx0aXBs aWNhdGlvbiBwcmludHMgdGhlIGZvbGxvd2luZyAoc3RyYWNlKSBpZiB5b3UgdHJ5IHRvCj4gPiBr aWxsIHRoZSBwcm9jZXNzOgo+ID4KPiA+IHNjaGVkX3lpZWxkKCkgICAgICAgICAgICAgICAgICAg ICAgICAgICA9IDAKPiA+IGZ1dGV4KDB4NTU3ZTk0NTM0M2I4LCBGVVRFWF9XQUlUX0JJVFNFVF9Q UklWQVRFfEZVVEVYX0NMT0NLX1JFQUxUSU1FLCAwLAo+ID4gTlVMTCwgRlVURVhfQklUU0VUX01B VENIX0FOWXN0cmFjZTogUHJvY2VzcyA2NTEgZGV0YWNoZWQKPiA+ICAgPGRldGFjaGVkIC4uLj4K PiA+Cj4gPiBBZnRlciB0aGlzLCB0aGUgZ3B1IGlzIG5vdCBmdW5jdGlvbmFsIGF0IGFsbCBhbmQg eW91J2QgbmVlZCBhIHBvd2VyIGN5Y2xlIHJlc2V0Cj4gPiB0byByZXN0b3JlIHRoZSBzeXN0ZW0u Cj4gPgo+ID4gSGFyZHdhcmUgaW5mbzoKPiA+IENQVTogQU1EIFJ5emVuIEVtYmVkZGVkIFYxNjA1 QiB3aXRoIFJhZGVvbiBWZWdhIEdmeCAoOCkgQCAyLjAwMEdIego+ID4gR1BVOiBBTUQgQVRJIFJh ZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVvbiBWZWdhIE1vYmlsZSBTZXJpZXMKPiA+Cj4gPiAwMzow MC4wIFZHQSBjb21wYXRpYmxlIGNvbnRyb2xsZXI6IEFkdmFuY2VkIE1pY3JvIERldmljZXMsIElu Yy4KPiA+IFtBTUQvQVRJXSBSYXZlbiBSaWRnZSBbUmFkZW9uIFZlZ2EgU2VyaWVzIC8gUmFkZW9u IFZlZ2EgTW9iaWxlIFNlcmllc10KPiA+IChyZXYgODMpCj4gPiAgICAgIERldmljZU5hbWU6IEJy b2FkY29tIDU3NjIKPiA+ICAgICAgU3Vic3lzdGVtOiBBZHZhbmNlZCBNaWNybyBEZXZpY2VzLCBJ bmMuIFtBTUQvQVRJXSBSYXZlbiBSaWRnZQo+ID4gW1JhZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVv biBWZWdhIE1vYmlsZSBTZXJpZXNdCj4gPiAgICAgIEtlcm5lbCBkcml2ZXIgaW4gdXNlOiBhbWRn cHUKPiA+ICAgICAgS2VybmVsIG1vZHVsZXM6IGFtZGdwdQo+ID4KPiA+IExpbnV4IGtlcm5lbCBp bmZvOgo+ID4KPiA+IHJvb3RAcXQ1MjIyOn4jIHVuYW1lIC1hCj4gPiBMaW51eCBxdDUyMjIgNS4x MS4wLXJjNi1xdGVjLXN0YW5kYXJkICMyIFNNUCBUdWUgRmViIDIgMDk6NDE6NDYgVVRDCj4gPiAy MDIxIHg4Nl82NCB4ODZfNjQgeDg2XzY0IEdOVS9MaW51eAo+ID4KPiA+IEJ5IGVuYWJsaW5nIHRo ZSBrZXJuZWwgbG9ja3Mgc3RhdHMgSSBjb3VsZCBzZWUgdGhlIE1hdHJpeE11bHRpcGxpY2F0aW9u IGlzCj4gPiBoYW5nZWQgaW4gdGhlIGFtZGdwdV9tbl9pbnZhbGlkYXRlX2dmeCBmdW5jdGlvbjoK PiA+Cj4gPiBbICA3MzguMzU5MjAyXSAxIGxvY2sgaGVsZCBieSBNYXRyaXhNdWx0aXBsaWMvNjUz Ogo+ID4gWyAgNzM4LjM1OTIwNl0gICMwOiBmZmZmODg4MTBlMzY0ZmUwCj4gPiAoJmFkZXYtPm5v dGlmaWVyX2xvY2speysuKy59LXszOjN9LCBhdDoKPiA+IGFtZGdwdV9tbl9pbnZhbGlkYXRlX2dm eCsweDM0LzB4YTAgW2FtZGdwdV0KPiA+Cj4gPiBJIGNhbiBzZWUgaW4gdGhlIHRoZSBhbWRncHVf bW5faW52YWxpZGF0ZV9nZnggZnVuY3Rpb246IHRoZQo+ID4gZG1hX3Jlc3Zfd2FpdF90aW1lb3V0 X3JjdSB1c2VzIHdhaXRfYWxsIChmZW5jZXMpIGFuZCBNQVhfU0NIRURVTEVfVElNRU9VVCBzbywg SQo+ID4gZ3Vlc3MgdGhlIGNvZGUgZ2V0cyBzdHVjayB0aGVyZSB3YWl0aW5nIGZvcmV2ZXIuIEFj Y29yZGluZyB0byB0aGUKPiA+IGRvY3VtZW50YXRpb246ICJXaGVuIHNvbWVib2R5IHRyaWVzIHRv IGludmFsaWRhdGUgdGhlIHBhZ2UgdGFibGVzIHdlIGJsb2NrIHRoZQo+ID4gdXBkYXRlIHVudGls IGFsbCBvcGVyYXRpb25zIG9uIHRoZSBwYWdlcyBpbiBxdWVzdGlvbiBhcmUgY29tcGxldGVkLCB0 aGVuIHRob3NlCj4gPiBwYWdlcyBhcmUgbWFya2VkICBhcyBhY2Nlc3NlZCBhbmQgYWxzbyBkaXJ0 eSBpZiBpdCB3YXNu4oCZdCBhIHJlYWQgb25seSBhY2Nlc3MuIgo+ID4gTG9va3MgbGlrZSB0aGUg ZmVuY2VzIGFyZSBkZWFkbG9ja2VkIGFuZCB0aGVyZWZvcmUsIGl0IG5ldmVyIHJldHVybnMuIENv dWxkIGl0Cj4gPiBiZSBwb3NzaWJsZT8gYW55IGhpbnQgdG8gd2hlcmUgY2FuIEkgbG9vayB0byBm aXggdGhpcz8KPiA+Cj4gPiBUaGFuayB5b3UgIGluIGFkdmFuY2UuCj4gPgo+ID4gSGVyZSB0aGUg ZnVsbCBkbWVzZyBvdXRwdXQ6Cj4gPgo+ID4gWyAgNzM4LjMzNzcyNl0gSU5GTzogdGFzayBNYXRy aXhNdWx0aXBsaWM6NjUzIGJsb2NrZWQgZm9yIG1vcmUgdGhhbiAxMjIgc2Vjb25kcy4KPiA+IFsg IDczOC4zNDQ5MzddICAgICAgIE5vdCB0YWludGVkIDUuMTEuMC1yYzYtcXRlYy1zdGFuZGFyZCAj Mgo+ID4gWyAgNzM4LjM1MDM4NF0gImVjaG8gMCA+IC9wcm9jL3N5cy9rZXJuZWwvaHVuZ190YXNr X3RpbWVvdXRfc2VjcyIKPiA+IGRpc2FibGVzIHRoaXMgbWVzc2FnZS4KPiA+IFsgIDczOC4zNTgy NDBdIHRhc2s6TWF0cml4TXVsdGlwbGljIHN0YXRlOkQgc3RhY2s6ICAgIDAgcGlkOiAgNjUzCj4g PiBwcGlkOiAgICAgMSBmbGFnczoweDAwMDA0MDAwCj4gPiBbICA3MzguMzU4MjU0XSBDYWxsIFRy YWNlOgo+ID4gWyAgNzM4LjM1ODI2MV0gID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDFlYi8w eDIzMAo+ID4gWyAgNzM4LjM1ODI3Nl0gIF9fc2NoZWR1bGUrMHgzNzAvMHg5NjAKPiA+IFsgIDcz OC4zNTgyOTFdICA/IGRtYV9mZW5jZV9kZWZhdWx0X3dhaXQrMHgxMTcvMHgyMzAKPiA+IFsgIDcz OC4zNTgyOTddICA/IGRtYV9mZW5jZV9kZWZhdWx0X3dhaXQrMHgxZWIvMHgyMzAKPiA+IFsgIDcz OC4zNTgzMDVdICBzY2hlZHVsZSsweDUxLzB4YzAKPiA+IFsgIDczOC4zNTgzMTJdICBzY2hlZHVs ZV90aW1lb3V0KzB4Mjc1LzB4MzgwCj4gPiBbICA3MzguMzU4MzI0XSAgPyBkbWFfZmVuY2VfZGVm YXVsdF93YWl0KzB4MWViLzB4MjMwCj4gPiBbICA3MzguMzU4MzMyXSAgPyBtYXJrX2hlbGRfbG9j a3MrMHg0Zi8weDcwCj4gPiBbICA3MzguMzU4MzQxXSAgPyBkbWFfZmVuY2VfZGVmYXVsdF93YWl0 KzB4MTE3LzB4MjMwCj4gPiBbICA3MzguMzU4MzQ3XSAgPyBsb2NrZGVwX2hhcmRpcnFzX29uX3By ZXBhcmUrMHhkNC8weDE4MAo+ID4gWyAgNzM4LjM1ODM1M10gID8gX3Jhd19zcGluX3VubG9ja19p cnFyZXN0b3JlKzB4MzkvMHg0MAo+ID4gWyAgNzM4LjM1ODM2Ml0gID8gZG1hX2ZlbmNlX2RlZmF1 bHRfd2FpdCsweDExNy8weDIzMAo+ID4gWyAgNzM4LjM1ODM3MF0gID8gZG1hX2ZlbmNlX2RlZmF1 bHRfd2FpdCsweDFlYi8weDIzMAo+ID4gWyAgNzM4LjM1ODM3NV0gIGRtYV9mZW5jZV9kZWZhdWx0 X3dhaXQrMHgyMTQvMHgyMzAKPiA+IFsgIDczOC4zNTgzODRdICA/IGRtYV9mZW5jZV9yZWxlYXNl KzB4MWEwLzB4MWEwCj4gPiBbICA3MzguMzU4Mzk2XSAgZG1hX2ZlbmNlX3dhaXRfdGltZW91dCsw eDEwNS8weDIwMAo+ID4gWyAgNzM4LjM1ODQwNV0gIGRtYV9yZXN2X3dhaXRfdGltZW91dF9yY3Ur MHgxYWEvMHg1ZTAKPiA+IFsgIDczOC4zNTg0MjFdICBhbWRncHVfbW5faW52YWxpZGF0ZV9nZngr MHg1NS8weGEwIFthbWRncHVdCj4gPiBbICA3MzguMzU4Njg4XSAgX19tbXVfbm90aWZpZXJfcmVs ZWFzZSsweDFiYi8weDIxMAo+ID4gWyAgNzM4LjM1ODcxMF0gIGV4aXRfbW1hcCsweDJmLzB4MWUw Cj4gPiBbICA3MzguMzU4NzIzXSAgPyBmaW5kX2hlbGRfbG9jaysweDM0LzB4YTAKPiA+IFsgIDcz OC4zNTg3NDZdICBtbXB1dCsweDM5LzB4ZTAKPiA+IFsgIDczOC4zNTg3NTZdICBkb19leGl0KzB4 NWMzLzB4YzAwCj4gPiBbICA3MzguMzU4NzYzXSAgPyBmaW5kX2hlbGRfbG9jaysweDM0LzB4YTAK PiA+IFsgIDczOC4zNTg3ODBdICBkb19ncm91cF9leGl0KzB4NDcvMHhiMAo+ID4gWyAgNzM4LjM1 ODc5MV0gIGdldF9zaWduYWwrMHgxNWIvMHhjNTAKPiA+IFsgIDczOC4zNTg4MDddICBhcmNoX2Rv X3NpZ25hbF9vcl9yZXN0YXJ0KzB4YWYvMHg3MTAKPiA+IFsgIDczOC4zNTg4MTZdICA/IGxvY2tk ZXBfaGFyZGlycXNfb25fcHJlcGFyZSsweGQ0LzB4MTgwCj4gPiBbICA3MzguMzU4ODIyXSAgPyBf cmF3X3NwaW5fdW5sb2NrX2lycXJlc3RvcmUrMHgzOS8weDQwCj4gPiBbICA3MzguMzU4ODMxXSAg PyBrdGltZV9nZXRfbW9ub19mYXN0X25zKzB4NTAvMHhhMAo+ID4gWyAgNzM4LjM1ODg0NF0gID8g YW1kZ3B1X2RybV9pb2N0bCsweDZiLzB4ODAgW2FtZGdwdV0KPiA+IFsgIDczOC4zNTkwNDRdICBl eGl0X3RvX3VzZXJfbW9kZV9wcmVwYXJlKzB4ZjIvMHgxYjAKPiA+IFsgIDczOC4zNTkwNTRdICBz eXNjYWxsX2V4aXRfdG9fdXNlcl9tb2RlKzB4MTkvMHg2MAo+ID4gWyAgNzM4LjM1OTA2Ml0gIGVu dHJ5X1NZU0NBTExfNjRfYWZ0ZXJfaHdmcmFtZSsweDQ0LzB4YTkKPiA+IFsgIDczOC4zNTkwNjld IFJJUDogMDAzMzoweDdmNmI4OWE1MTg4Nwo+ID4gWyAgNzM4LjM1OTA3Nl0gUlNQOiAwMDJiOjAw MDA3ZjZiODJiNTRiMTggRUZMQUdTOiAwMDAwMDI0NiBPUklHX1JBWDoKPiA+IDAwMDAwMDAwMDAw MDAwMTAKPiA+IFsgIDczOC4zNTkwODZdIFJBWDogZmZmZmZmZmZmZmZmZmUwMCBSQlg6IDAwMDA3 ZjZiODJiNTRiNTAgUkNYOiAwMDAwN2Y2Yjg5YTUxODg3Cj4gPiBbICA3MzguMzU5MDkxXSBSRFg6 IDAwMDA3ZjZiODJiNTRiNTAgUlNJOiAwMDAwMDAwMGMwMjA2NGMzIFJESTogMDAwMDAwMDAwMDAw MDAwNwo+ID4gWyAgNzM4LjM1OTA5Nl0gUkJQOiAwMDAwMDAwMGMwMjA2NGMzIFIwODogMDAwMDAw MDAwMDAwMDAwMyBSMDk6IDAwMDA3ZjZiODJiNTRiYmMKPiA+IFsgIDczOC4zNTkxMDFdIFIxMDog MDAwMDAwMDAwMDAwMDAwMSBSMTE6IDAwMDAwMDAwMDAwMDAyNDYgUjEyOiAwMDAwMDAwMTY1YTBi YzAwCj4gPiBbICA3MzguMzU5MTA2XSBSMTM6IDAwMDAwMDAwMDAwMDAwMDcgUjE0OiAwMDAwMDAw MDAwMDAwMDAxIFIxNTogMDAwMDAwMDAwMDAwMDAwMAo+ID4gWyAgNzM4LjM1OTEyOV0KPiA+ICAg ICAgICAgICAgICAgICBTaG93aW5nIGFsbCBsb2NrcyBoZWxkIGluIHRoZSBzeXN0ZW06Cj4gPiBb ICA3MzguMzU5MTQxXSAxIGxvY2sgaGVsZCBieSBraHVuZ3Rhc2tkLzU0Ogo+ID4gWyAgNzM4LjM1 OTE0OF0gICMwOiBmZmZmZmZmZjgyOWY2ODQwIChyY3VfcmVhZF9sb2NrKXsuLi4ufS17MToyfSwg YXQ6Cj4gPiBkZWJ1Z19zaG93X2FsbF9sb2NrcysweDE1LzB4MTgzCj4gPiBbICA3MzguMzU5MTg3 XSAxIGxvY2sgaGVsZCBieSBzeXN0ZW1kLWpvdXJuYWwvMTc0Ogo+ID4gWyAgNzM4LjM1OTIwMl0g MSBsb2NrIGhlbGQgYnkgTWF0cml4TXVsdGlwbGljLzY1MzoKPiA+IFsgIDczOC4zNTkyMDZdICAj MDogZmZmZjg4ODEwZTM2NGZlMAo+ID4gKCZhZGV2LT5ub3RpZmllcl9sb2NrKXsrLisufS17Mzoz fSwgYXQ6Cj4gPiBhbWRncHVfbW5faW52YWxpZGF0ZV9nZngrMHgzNC8weGEwIFthbWRncHVdCj4g Pgo+ID4gRGFuaWVsCj4KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXwo+IGRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKPiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRl c2t0b3Aub3JnCj4gaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m by9kcmktZGV2ZWwKCgoKLS0gCkRhbmllbCBWZXR0ZXIKU29mdHdhcmUgRW5naW5lZXIsIEludGVs IENvcnBvcmF0aW9uCmh0dHA6Ly9ibG9nLmZmd2xsLmNoCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKZHJpLWRldmVs QGxpc3RzLmZyZWVkZXNrdG9wLm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWls bWFuL2xpc3RpbmZvL2RyaS1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AB0CCC433E0 for ; Wed, 3 Feb 2021 08:48:58 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AFCF264F72 for ; Wed, 3 Feb 2021 08:48:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AFCF264F72 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id ABC906EA1A; Wed, 3 Feb 2021 08:48:53 +0000 (UTC) Received: from mail-ot1-x331.google.com (mail-ot1-x331.google.com [IPv6:2607:f8b0:4864:20::331]) by gabe.freedesktop.org (Postfix) with ESMTPS id 555ED6E9DC for ; Wed, 3 Feb 2021 08:48:52 +0000 (UTC) Received: by mail-ot1-x331.google.com with SMTP id d7so22659283otf.3 for ; Wed, 03 Feb 2021 00:48:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=VwNp0a42H2HJdUXf2U+NyyGMLQI597ulmFnoEoIfRg8=; b=bp4qJgQbbTLWXuU91TsSZAOI41pytifc2W9dIiVN6dXquO6uybAKvlcCPaPH1NpkHd P0Mq76vN/q5u7f/lrfgPGRqu2/rWYgb0PAnpfpAeEQ4C/t0wMWRE7aOx1JX0e82Xot46 WQy0xA2TsNhyd9sdod7biEaozcsh5EHAWKcec= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=VwNp0a42H2HJdUXf2U+NyyGMLQI597ulmFnoEoIfRg8=; b=dGZrzcMTYSXg9lfBg3TeW6PwygGc1v27OANbBXQA1ivgPvNAlnSZhd+xzQ2jhzthSg xqCERYSZDQg4G7UC9LpAjKi5OELZ++1TLxZg4JjtLi+uQ5CKeIZcBxwqglkNBe1IWPQP yI3nOQSqg0HkZMc1HM+J8W1eF4Cwii0IlEF55IL6eEtpkLwN5TmqA298XoZ46t+650Rd 4WZSCjq/wFDwsrhENAenE1z9lTLRh7zI82BDQkVTy+2MpmprNzT4ygUnJnN1m429wzHe 8W52p7ClWLd77Sv4sgFZjadaxdjKP1gjsg5Eq9HKnh6cU/WqOZZ6M65uuSZdPa1GAbVx E4Ug== X-Gm-Message-State: AOAM530kXAOHHwhSwBSBSdqpbAvGYSIRLyu019AOMvj6NDmDtp6DIGKd T94sfCPas7Ewd+qVluZFB+Gnm1JteH85DW3vMVtMrQ== X-Google-Smtp-Source: ABdhPJwFJFhQYQVJPg8AcL5xpSmXQT4nU/vhOSOoGYxm48EBQ+vEBz006Z/ZUAjAOMfXzIp+7+twjGV9i1z1MNucNfM= X-Received: by 2002:a9d:6c96:: with SMTP id c22mr1296084otr.303.1612342130615; Wed, 03 Feb 2021 00:48:50 -0800 (PST) MIME-Version: 1.0 References: <58e41b62-b8e0-b036-c87d-a84d53f5a26e@amd.com> In-Reply-To: <58e41b62-b8e0-b036-c87d-a84d53f5a26e@amd.com> From: Daniel Vetter Date: Wed, 3 Feb 2021 09:48:39 +0100 Message-ID: Subject: Re: [amdgpu] deadlock To: =?UTF-8?Q?Christian_K=C3=B6nig?= X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , Daniel Gomez , dri-devel , amd-gfx list , Linux Kernel Mailing List Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" T24gV2VkLCBGZWIgMywgMjAyMSBhdCA5OjM2IEFNIENocmlzdGlhbiBLw7ZuaWcgPGNocmlzdGlh bi5rb2VuaWdAYW1kLmNvbT4gd3JvdGU6Cj4KPiBIaSBEYW5pZWwsCj4KPiB0aGlzIGlzIG5vdCBh IGRlYWRsb2NrLCBidXQgcmF0aGVyIGEgaGFyZHdhcmUgbG9ja3VwLgoKQXJlIHlvdSBzdXJlPyBJ bWUgZ2V0dGluZyBzdHVjayBpbiBkbWFfZmVuY2Vfd2FpdCBoYXMgZ2VuZXJhbGx5IGdvb2QKY2hh bmNlIG9mIGJlaW5nIGEgZG1hX2ZlbmNlIGRlYWRsb2NrLiBHUFUgaGFuZyBzaG91bGQgbmV2ZXIg cmVzdWx0IGluCmEgZm9yZXZlciBzdHVjayBkbWFfZmVuY2UuCgpEYW5pZWwsIGNhbiB5b3UgcGxz IHJlLWhhbmcgeW91ciBtYWNoaW5lIGFuZCB0aGVuIGR1bXAgYmFja3RyYWNlcyBvZgphbGwgdGFz a3MgaW50byBkbWVzZyB3aXRoIHN5c3JxLXQsIGFuZCB0aGVuIGF0dGFjaCB0aGF0PyBXaXRob3V0 IGFsbAp0aGUgYmFja3RyYWNlcyBpdCdzIHRyaWNreSB0byBjb25zdHJ1Y3QgdGhlIGZ1bGwgZGVw ZW5kZW5jeSBjaGFpbiBvZgp3aGF0J3MgZ29pbmcgb24uIEFsc28gaXMgdGhpcyBwbGFpbiAtcmM2 LCBub3Qgc29tZSBtb3JlIHBhdGNoZXMgb24KdG9wPwotRGFuaWVsCgo+IFdoaWNoIE9wZW5DbCBz dGFjayBhcmUgeW91IHVzaW5nPwo+Cj4gUmVnYXJkcywKPiBDaHJpc3RpYW4uCj4KPiBBbSAwMy4w Mi4yMSB1bSAwOTozMyBzY2hyaWViIERhbmllbCBHb21lejoKPiA+IEhpIGFsbCwKPiA+Cj4gPiBJ IGhhdmUgYSBkZWFkbG9jayB3aXRoIHRoZSBhbWRncHUgbWFpbmxpbmUgZHJpdmVyIHdoZW4gcnVu bmluZyBpbiBwYXJhbGxlbCB0d28KPiA+IE9wZW5DTCBhcHBsaWNhdGlvbnMuIFNvIGZhciwgd2Un dmUgYmVlbiBhYmxlIHRvIHJlcGxpY2F0ZSBpdCBlYXNpbHkgYnkgZXhlY3V0aW5nCj4gPiBjbGlu Zm8gYW5kIE1hdHJpeE11bHRpcGxpY2F0aW9uIChmcm9tIEFNRCBvcGVuY2wtc2FtcGxlcykuIEl0 J3MgcXVpdGUgb2xkIHRoZQo+ID4gb3BlbmNsLXNhbXBsZXMgc28sIGlmIHlvdSBoYXZlIGFueSBv dGhlciBzdWdnZXN0aW9uIGZvciB0ZXN0aW5nIEknZCBiZSB2ZXJ5Cj4gPiBoYXBweSB0byB0ZXN0 IGl0IGFzIHdlbGwuCj4gPgo+ID4gSG93IHRvIHJlcGxpY2F0ZSB0aGUgaXNzdWU6Cj4gPgo+ID4g IyB3aGlsZSB0cnVlOyBkbyAvdXNyL2Jpbi9NYXRyaXhNdWx0aXBsaWNhdGlvbiAtLWRldmljZSBn cHUgXAo+ID4gICAgICAtLWRldmljZUlkIDAgLXggMTAwMCAteSAxMDAwIC16IDEwMDAgLXEgLXQg LWkgNTA7IGRvbmUKPiA+ICMgd2hpbGUgdHJ1ZTsgZG8gY2xpbmZvOyBkb25lCj4gPgo+ID4gT3V0 cHV0Ogo+ID4KPiA+IEFmdGVyIGEgbWludXRlIG9yIGxlc3MgKHNvbWV0aW1lcyBjb3VsZCBiZSBt b3JlKSBJIGNhbiBzZWUgdGhhdAo+ID4gTWF0cml4TXVsdGlwbGljYXRpb24gYW5kIGNsaW5mbyBo YW5nLiBJbiBhZGRpdGlvbiwgd2l0aCByYWRlb250b3AgeW91IGNhbiBzZWUKPiA+IGhvdyB0aGUg R3JhcGhpY3MgcGlwZSBnb2VzIGZyb20gfjUwJSB0byAxMDAlLiBBbHNvIHRoZSBzaGFkZXIgY2xv Y2tzCj4gPiBnb2VzIHVwIGZyb20gfjM1JSB0byB+OTYlLgo+ID4KPiA+IGNsaW5mbyBrZWVwcyBw cmludGluZzoKPiA+IGlvY3RsKDcsIERSTV9JT0NUTF9TWU5DT0JKX1dBSVQsIDB4N2ZmZTQ2ZTVm OTUwKSA9IC0xIEVUSU1FIChUaW1lciBleHBpcmVkKQo+ID4KPiA+IEFuZCBNYXRyaXhNdWx0aXBs aWNhdGlvbiBwcmludHMgdGhlIGZvbGxvd2luZyAoc3RyYWNlKSBpZiB5b3UgdHJ5IHRvCj4gPiBr aWxsIHRoZSBwcm9jZXNzOgo+ID4KPiA+IHNjaGVkX3lpZWxkKCkgICAgICAgICAgICAgICAgICAg ICAgICAgICA9IDAKPiA+IGZ1dGV4KDB4NTU3ZTk0NTM0M2I4LCBGVVRFWF9XQUlUX0JJVFNFVF9Q UklWQVRFfEZVVEVYX0NMT0NLX1JFQUxUSU1FLCAwLAo+ID4gTlVMTCwgRlVURVhfQklUU0VUX01B VENIX0FOWXN0cmFjZTogUHJvY2VzcyA2NTEgZGV0YWNoZWQKPiA+ICAgPGRldGFjaGVkIC4uLj4K PiA+Cj4gPiBBZnRlciB0aGlzLCB0aGUgZ3B1IGlzIG5vdCBmdW5jdGlvbmFsIGF0IGFsbCBhbmQg eW91J2QgbmVlZCBhIHBvd2VyIGN5Y2xlIHJlc2V0Cj4gPiB0byByZXN0b3JlIHRoZSBzeXN0ZW0u Cj4gPgo+ID4gSGFyZHdhcmUgaW5mbzoKPiA+IENQVTogQU1EIFJ5emVuIEVtYmVkZGVkIFYxNjA1 QiB3aXRoIFJhZGVvbiBWZWdhIEdmeCAoOCkgQCAyLjAwMEdIego+ID4gR1BVOiBBTUQgQVRJIFJh ZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVvbiBWZWdhIE1vYmlsZSBTZXJpZXMKPiA+Cj4gPiAwMzow MC4wIFZHQSBjb21wYXRpYmxlIGNvbnRyb2xsZXI6IEFkdmFuY2VkIE1pY3JvIERldmljZXMsIElu Yy4KPiA+IFtBTUQvQVRJXSBSYXZlbiBSaWRnZSBbUmFkZW9uIFZlZ2EgU2VyaWVzIC8gUmFkZW9u IFZlZ2EgTW9iaWxlIFNlcmllc10KPiA+IChyZXYgODMpCj4gPiAgICAgIERldmljZU5hbWU6IEJy b2FkY29tIDU3NjIKPiA+ICAgICAgU3Vic3lzdGVtOiBBZHZhbmNlZCBNaWNybyBEZXZpY2VzLCBJ bmMuIFtBTUQvQVRJXSBSYXZlbiBSaWRnZQo+ID4gW1JhZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVv biBWZWdhIE1vYmlsZSBTZXJpZXNdCj4gPiAgICAgIEtlcm5lbCBkcml2ZXIgaW4gdXNlOiBhbWRn cHUKPiA+ICAgICAgS2VybmVsIG1vZHVsZXM6IGFtZGdwdQo+ID4KPiA+IExpbnV4IGtlcm5lbCBp bmZvOgo+ID4KPiA+IHJvb3RAcXQ1MjIyOn4jIHVuYW1lIC1hCj4gPiBMaW51eCBxdDUyMjIgNS4x MS4wLXJjNi1xdGVjLXN0YW5kYXJkICMyIFNNUCBUdWUgRmViIDIgMDk6NDE6NDYgVVRDCj4gPiAy MDIxIHg4Nl82NCB4ODZfNjQgeDg2XzY0IEdOVS9MaW51eAo+ID4KPiA+IEJ5IGVuYWJsaW5nIHRo ZSBrZXJuZWwgbG9ja3Mgc3RhdHMgSSBjb3VsZCBzZWUgdGhlIE1hdHJpeE11bHRpcGxpY2F0aW9u IGlzCj4gPiBoYW5nZWQgaW4gdGhlIGFtZGdwdV9tbl9pbnZhbGlkYXRlX2dmeCBmdW5jdGlvbjoK PiA+Cj4gPiBbICA3MzguMzU5MjAyXSAxIGxvY2sgaGVsZCBieSBNYXRyaXhNdWx0aXBsaWMvNjUz Ogo+ID4gWyAgNzM4LjM1OTIwNl0gICMwOiBmZmZmODg4MTBlMzY0ZmUwCj4gPiAoJmFkZXYtPm5v dGlmaWVyX2xvY2speysuKy59LXszOjN9LCBhdDoKPiA+IGFtZGdwdV9tbl9pbnZhbGlkYXRlX2dm eCsweDM0LzB4YTAgW2FtZGdwdV0KPiA+Cj4gPiBJIGNhbiBzZWUgaW4gdGhlIHRoZSBhbWRncHVf bW5faW52YWxpZGF0ZV9nZnggZnVuY3Rpb246IHRoZQo+ID4gZG1hX3Jlc3Zfd2FpdF90aW1lb3V0 X3JjdSB1c2VzIHdhaXRfYWxsIChmZW5jZXMpIGFuZCBNQVhfU0NIRURVTEVfVElNRU9VVCBzbywg SQo+ID4gZ3Vlc3MgdGhlIGNvZGUgZ2V0cyBzdHVjayB0aGVyZSB3YWl0aW5nIGZvcmV2ZXIuIEFj Y29yZGluZyB0byB0aGUKPiA+IGRvY3VtZW50YXRpb246ICJXaGVuIHNvbWVib2R5IHRyaWVzIHRv IGludmFsaWRhdGUgdGhlIHBhZ2UgdGFibGVzIHdlIGJsb2NrIHRoZQo+ID4gdXBkYXRlIHVudGls IGFsbCBvcGVyYXRpb25zIG9uIHRoZSBwYWdlcyBpbiBxdWVzdGlvbiBhcmUgY29tcGxldGVkLCB0 aGVuIHRob3NlCj4gPiBwYWdlcyBhcmUgbWFya2VkICBhcyBhY2Nlc3NlZCBhbmQgYWxzbyBkaXJ0 eSBpZiBpdCB3YXNu4oCZdCBhIHJlYWQgb25seSBhY2Nlc3MuIgo+ID4gTG9va3MgbGlrZSB0aGUg ZmVuY2VzIGFyZSBkZWFkbG9ja2VkIGFuZCB0aGVyZWZvcmUsIGl0IG5ldmVyIHJldHVybnMuIENv dWxkIGl0Cj4gPiBiZSBwb3NzaWJsZT8gYW55IGhpbnQgdG8gd2hlcmUgY2FuIEkgbG9vayB0byBm aXggdGhpcz8KPiA+Cj4gPiBUaGFuayB5b3UgIGluIGFkdmFuY2UuCj4gPgo+ID4gSGVyZSB0aGUg ZnVsbCBkbWVzZyBvdXRwdXQ6Cj4gPgo+ID4gWyAgNzM4LjMzNzcyNl0gSU5GTzogdGFzayBNYXRy aXhNdWx0aXBsaWM6NjUzIGJsb2NrZWQgZm9yIG1vcmUgdGhhbiAxMjIgc2Vjb25kcy4KPiA+IFsg IDczOC4zNDQ5MzddICAgICAgIE5vdCB0YWludGVkIDUuMTEuMC1yYzYtcXRlYy1zdGFuZGFyZCAj Mgo+ID4gWyAgNzM4LjM1MDM4NF0gImVjaG8gMCA+IC9wcm9jL3N5cy9rZXJuZWwvaHVuZ190YXNr X3RpbWVvdXRfc2VjcyIKPiA+IGRpc2FibGVzIHRoaXMgbWVzc2FnZS4KPiA+IFsgIDczOC4zNTgy NDBdIHRhc2s6TWF0cml4TXVsdGlwbGljIHN0YXRlOkQgc3RhY2s6ICAgIDAgcGlkOiAgNjUzCj4g PiBwcGlkOiAgICAgMSBmbGFnczoweDAwMDA0MDAwCj4gPiBbICA3MzguMzU4MjU0XSBDYWxsIFRy YWNlOgo+ID4gWyAgNzM4LjM1ODI2MV0gID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDFlYi8w eDIzMAo+ID4gWyAgNzM4LjM1ODI3Nl0gIF9fc2NoZWR1bGUrMHgzNzAvMHg5NjAKPiA+IFsgIDcz OC4zNTgyOTFdICA/IGRtYV9mZW5jZV9kZWZhdWx0X3dhaXQrMHgxMTcvMHgyMzAKPiA+IFsgIDcz OC4zNTgyOTddICA/IGRtYV9mZW5jZV9kZWZhdWx0X3dhaXQrMHgxZWIvMHgyMzAKPiA+IFsgIDcz OC4zNTgzMDVdICBzY2hlZHVsZSsweDUxLzB4YzAKPiA+IFsgIDczOC4zNTgzMTJdICBzY2hlZHVs ZV90aW1lb3V0KzB4Mjc1LzB4MzgwCj4gPiBbICA3MzguMzU4MzI0XSAgPyBkbWFfZmVuY2VfZGVm YXVsdF93YWl0KzB4MWViLzB4MjMwCj4gPiBbICA3MzguMzU4MzMyXSAgPyBtYXJrX2hlbGRfbG9j a3MrMHg0Zi8weDcwCj4gPiBbICA3MzguMzU4MzQxXSAgPyBkbWFfZmVuY2VfZGVmYXVsdF93YWl0 KzB4MTE3LzB4MjMwCj4gPiBbICA3MzguMzU4MzQ3XSAgPyBsb2NrZGVwX2hhcmRpcnFzX29uX3By ZXBhcmUrMHhkNC8weDE4MAo+ID4gWyAgNzM4LjM1ODM1M10gID8gX3Jhd19zcGluX3VubG9ja19p cnFyZXN0b3JlKzB4MzkvMHg0MAo+ID4gWyAgNzM4LjM1ODM2Ml0gID8gZG1hX2ZlbmNlX2RlZmF1 bHRfd2FpdCsweDExNy8weDIzMAo+ID4gWyAgNzM4LjM1ODM3MF0gID8gZG1hX2ZlbmNlX2RlZmF1 bHRfd2FpdCsweDFlYi8weDIzMAo+ID4gWyAgNzM4LjM1ODM3NV0gIGRtYV9mZW5jZV9kZWZhdWx0 X3dhaXQrMHgyMTQvMHgyMzAKPiA+IFsgIDczOC4zNTgzODRdICA/IGRtYV9mZW5jZV9yZWxlYXNl KzB4MWEwLzB4MWEwCj4gPiBbICA3MzguMzU4Mzk2XSAgZG1hX2ZlbmNlX3dhaXRfdGltZW91dCsw eDEwNS8weDIwMAo+ID4gWyAgNzM4LjM1ODQwNV0gIGRtYV9yZXN2X3dhaXRfdGltZW91dF9yY3Ur MHgxYWEvMHg1ZTAKPiA+IFsgIDczOC4zNTg0MjFdICBhbWRncHVfbW5faW52YWxpZGF0ZV9nZngr MHg1NS8weGEwIFthbWRncHVdCj4gPiBbICA3MzguMzU4Njg4XSAgX19tbXVfbm90aWZpZXJfcmVs ZWFzZSsweDFiYi8weDIxMAo+ID4gWyAgNzM4LjM1ODcxMF0gIGV4aXRfbW1hcCsweDJmLzB4MWUw Cj4gPiBbICA3MzguMzU4NzIzXSAgPyBmaW5kX2hlbGRfbG9jaysweDM0LzB4YTAKPiA+IFsgIDcz OC4zNTg3NDZdICBtbXB1dCsweDM5LzB4ZTAKPiA+IFsgIDczOC4zNTg3NTZdICBkb19leGl0KzB4 NWMzLzB4YzAwCj4gPiBbICA3MzguMzU4NzYzXSAgPyBmaW5kX2hlbGRfbG9jaysweDM0LzB4YTAK PiA+IFsgIDczOC4zNTg3ODBdICBkb19ncm91cF9leGl0KzB4NDcvMHhiMAo+ID4gWyAgNzM4LjM1 ODc5MV0gIGdldF9zaWduYWwrMHgxNWIvMHhjNTAKPiA+IFsgIDczOC4zNTg4MDddICBhcmNoX2Rv X3NpZ25hbF9vcl9yZXN0YXJ0KzB4YWYvMHg3MTAKPiA+IFsgIDczOC4zNTg4MTZdICA/IGxvY2tk ZXBfaGFyZGlycXNfb25fcHJlcGFyZSsweGQ0LzB4MTgwCj4gPiBbICA3MzguMzU4ODIyXSAgPyBf cmF3X3NwaW5fdW5sb2NrX2lycXJlc3RvcmUrMHgzOS8weDQwCj4gPiBbICA3MzguMzU4ODMxXSAg PyBrdGltZV9nZXRfbW9ub19mYXN0X25zKzB4NTAvMHhhMAo+ID4gWyAgNzM4LjM1ODg0NF0gID8g YW1kZ3B1X2RybV9pb2N0bCsweDZiLzB4ODAgW2FtZGdwdV0KPiA+IFsgIDczOC4zNTkwNDRdICBl eGl0X3RvX3VzZXJfbW9kZV9wcmVwYXJlKzB4ZjIvMHgxYjAKPiA+IFsgIDczOC4zNTkwNTRdICBz eXNjYWxsX2V4aXRfdG9fdXNlcl9tb2RlKzB4MTkvMHg2MAo+ID4gWyAgNzM4LjM1OTA2Ml0gIGVu dHJ5X1NZU0NBTExfNjRfYWZ0ZXJfaHdmcmFtZSsweDQ0LzB4YTkKPiA+IFsgIDczOC4zNTkwNjld IFJJUDogMDAzMzoweDdmNmI4OWE1MTg4Nwo+ID4gWyAgNzM4LjM1OTA3Nl0gUlNQOiAwMDJiOjAw MDA3ZjZiODJiNTRiMTggRUZMQUdTOiAwMDAwMDI0NiBPUklHX1JBWDoKPiA+IDAwMDAwMDAwMDAw MDAwMTAKPiA+IFsgIDczOC4zNTkwODZdIFJBWDogZmZmZmZmZmZmZmZmZmUwMCBSQlg6IDAwMDA3 ZjZiODJiNTRiNTAgUkNYOiAwMDAwN2Y2Yjg5YTUxODg3Cj4gPiBbICA3MzguMzU5MDkxXSBSRFg6 IDAwMDA3ZjZiODJiNTRiNTAgUlNJOiAwMDAwMDAwMGMwMjA2NGMzIFJESTogMDAwMDAwMDAwMDAw MDAwNwo+ID4gWyAgNzM4LjM1OTA5Nl0gUkJQOiAwMDAwMDAwMGMwMjA2NGMzIFIwODogMDAwMDAw MDAwMDAwMDAwMyBSMDk6IDAwMDA3ZjZiODJiNTRiYmMKPiA+IFsgIDczOC4zNTkxMDFdIFIxMDog MDAwMDAwMDAwMDAwMDAwMSBSMTE6IDAwMDAwMDAwMDAwMDAyNDYgUjEyOiAwMDAwMDAwMTY1YTBi YzAwCj4gPiBbICA3MzguMzU5MTA2XSBSMTM6IDAwMDAwMDAwMDAwMDAwMDcgUjE0OiAwMDAwMDAw MDAwMDAwMDAxIFIxNTogMDAwMDAwMDAwMDAwMDAwMAo+ID4gWyAgNzM4LjM1OTEyOV0KPiA+ICAg ICAgICAgICAgICAgICBTaG93aW5nIGFsbCBsb2NrcyBoZWxkIGluIHRoZSBzeXN0ZW06Cj4gPiBb ICA3MzguMzU5MTQxXSAxIGxvY2sgaGVsZCBieSBraHVuZ3Rhc2tkLzU0Ogo+ID4gWyAgNzM4LjM1 OTE0OF0gICMwOiBmZmZmZmZmZjgyOWY2ODQwIChyY3VfcmVhZF9sb2NrKXsuLi4ufS17MToyfSwg YXQ6Cj4gPiBkZWJ1Z19zaG93X2FsbF9sb2NrcysweDE1LzB4MTgzCj4gPiBbICA3MzguMzU5MTg3 XSAxIGxvY2sgaGVsZCBieSBzeXN0ZW1kLWpvdXJuYWwvMTc0Ogo+ID4gWyAgNzM4LjM1OTIwMl0g MSBsb2NrIGhlbGQgYnkgTWF0cml4TXVsdGlwbGljLzY1MzoKPiA+IFsgIDczOC4zNTkyMDZdICAj MDogZmZmZjg4ODEwZTM2NGZlMAo+ID4gKCZhZGV2LT5ub3RpZmllcl9sb2NrKXsrLisufS17Mzoz fSwgYXQ6Cj4gPiBhbWRncHVfbW5faW52YWxpZGF0ZV9nZngrMHgzNC8weGEwIFthbWRncHVdCj4g Pgo+ID4gRGFuaWVsCj4KPiBfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fXwo+IGRyaS1kZXZlbCBtYWlsaW5nIGxpc3QKPiBkcmktZGV2ZWxAbGlzdHMuZnJlZWRl c2t0b3Aub3JnCj4gaHR0cHM6Ly9saXN0cy5mcmVlZGVza3RvcC5vcmcvbWFpbG1hbi9saXN0aW5m by9kcmktZGV2ZWwKCgoKLS0gCkRhbmllbCBWZXR0ZXIKU29mdHdhcmUgRW5naW5lZXIsIEludGVs IENvcnBvcmF0aW9uCmh0dHA6Ly9ibG9nLmZmd2xsLmNoCl9fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fCmFtZC1nZnggbWFpbGluZyBsaXN0CmFtZC1nZnhAbGlz dHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4v bGlzdGluZm8vYW1kLWdmeAo=