From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1456C433E0 for ; Wed, 3 Feb 2021 14:55:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B41664F48 for ; Wed, 3 Feb 2021 14:55:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233139AbhBCOze (ORCPT ); Wed, 3 Feb 2021 09:55:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60750 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232847AbhBCOz3 (ORCPT ); Wed, 3 Feb 2021 09:55:29 -0500 Received: from mail-lf1-x133.google.com (mail-lf1-x133.google.com [IPv6:2a00:1450:4864:20::133]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D77D4C061573 for ; Wed, 3 Feb 2021 06:54:48 -0800 (PST) Received: by mail-lf1-x133.google.com with SMTP id p21so33728629lfu.11 for ; Wed, 03 Feb 2021 06:54:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qtec.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=5wJXA1HJZyVo+PhV0nbxE7xxzJossHm5qQWx4jq7Crc=; b=bBGSQOnXGXfMnytcdOMt81OXgjvBv6qrkAqmO/JaxnYO14p+pg629EWnOAd9cJXF0c HW8Dbq9fwgsjYo1P9JElWarNGktzNuh4Pyv1huIcK27T6Qh25l3qc8cWuVnpv+PM8d6o J0oosa7kNX6wTgqc5TW1Nj5zr8YJFmNuEm/4tbXJlBQLTAQ8zrMXRGgq+II2IjIQp6Jn T549VJgAp5TBa4N7VoqVpiS4bNWi+0B5f5GaZx4DlRwpY99urz27VCFi0gOnrYwBOok6 lges+uaQ222fuJnxQFAseX/ukWs5KpoEry+eJT7nXItAdxkQm0Q1cdKXcaCUxDZeWdee bJiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5wJXA1HJZyVo+PhV0nbxE7xxzJossHm5qQWx4jq7Crc=; b=ZWL6xKmanfphQgMk9eTBQ5WE7rIC/1LkzRarsb0OOx3SkRHXQ0A26tMU4p7ns27dVp RdTjRJN/+slJ+V6I3Cb3SxxkTKtXacmLe7YdHfHheV24uB2xxg9kQl/0CllQLyFJUY0P pj29G0Sr7B1T2sMNhbFIMBZ+rwHhJZ6edn2xiExsFRKH+U0jN4OwzMdWc9wlIhxohIyl zDYw1v+H3Its2otNpod/7OW57sYrj+TZTjBQFYYxl0Z/oHJaQbghoD3eWMVoJKBj4EXN ibFpciI2uqqwiFTfvBGjqQLpiD3BcY3gm8sshoXZTXseFliBva6sKxvbATfMNc36XzP/ Leeg== X-Gm-Message-State: AOAM532WSqTd2DmINzuo5iY57CF3IzmL1p1CNQesPEkjQvo5hsVPwnc5 07hOCUiOAeu/XlmPOKIgqBriVtbK2NBYb1EYJXgdnQ== X-Google-Smtp-Source: ABdhPJw9ybjbBjCIBrkrj8loyUZJeejScm+VzMN5pMRxewJEYUQWPnOJl+ZLGh7hUNzxuHiU3VIUaGCS9ITAqQHULmo= X-Received: by 2002:a05:6512:376f:: with SMTP id z15mr1960636lft.59.1612364087238; Wed, 03 Feb 2021 06:54:47 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Daniel Gomez Date: Wed, 3 Feb 2021 15:54:36 +0100 Message-ID: Subject: Re: [amdgpu] deadlock To: =?UTF-8?Q?Christian_K=C3=B6nig?= Cc: amd-gfx list , dri-devel , Linux Kernel Mailing List , Alex Deucher Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 3 Feb 2021 at 15:37, Christian K=C3=B6nig wrote: > > Hi Daniel, > > I've talked a bit with our internal team. > > The problem is that the 20.20 release still uses the older OpenCL stack > which obviously has a bug here and causes a hang. > > The best approach I can give you is to switch to the ROCm stack instead. Thanks Christian. I'll try with the ROCm stack then. As far as I understood= , it should work because the part of the code where it now hangs is not actua= lly used by the ROCm stack, is that correct? However, the hang/bug will still be there even though it is not used in that stack. Anyway, I'll keep you guys posted with this change. > > Regards, > Christian. > > Am 03.02.21 um 09:33 schrieb Daniel Gomez: > > Hi all, > > > > I have a deadlock with the amdgpu mainline driver when running in paral= lel two > > OpenCL applications. So far, we've been able to replicate it easily by = executing > > clinfo and MatrixMultiplication (from AMD opencl-samples). It's quite o= ld the > > opencl-samples so, if you have any other suggestion for testing I'd be = very > > happy to test it as well. > > > > How to replicate the issue: > > > > # while true; do /usr/bin/MatrixMultiplication --device gpu \ > > --deviceId 0 -x 1000 -y 1000 -z 1000 -q -t -i 50; done > > # while true; do clinfo; done > > > > Output: > > > > After a minute or less (sometimes could be more) I can see that > > MatrixMultiplication and clinfo hang. In addition, with radeontop you c= an see > > how the Graphics pipe goes from ~50% to 100%. Also the shader clocks > > goes up from ~35% to ~96%. > > > > clinfo keeps printing: > > ioctl(7, DRM_IOCTL_SYNCOBJ_WAIT, 0x7ffe46e5f950) =3D -1 ETIME (Timer ex= pired) > > > > And MatrixMultiplication prints the following (strace) if you try to > > kill the process: > > > > sched_yield() =3D 0 > > futex(0x557e945343b8, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0= , > > NULL, FUTEX_BITSET_MATCH_ANYstrace: Process 651 detached > > > > > > After this, the gpu is not functional at all and you'd need a power cyc= le reset > > to restore the system. > > > > Hardware info: > > CPU: AMD Ryzen Embedded V1605B with Radeon Vega Gfx (8) @ 2.000GHz > > GPU: AMD ATI Radeon Vega Series / Radeon Vega Mobile Series > > > > 03:00.0 VGA compatible controller: Advanced Micro Devices, Inc. > > [AMD/ATI] Raven Ridge [Radeon Vega Series / Radeon Vega Mobile Series] > > (rev 83) > > DeviceName: Broadcom 5762 > > Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Raven Ridge > > [Radeon Vega Series / Radeon Vega Mobile Series] > > Kernel driver in use: amdgpu > > Kernel modules: amdgpu > > > > Linux kernel info: > > > > root@qt5222:~# uname -a > > Linux qt5222 5.11.0-rc6-qtec-standard #2 SMP Tue Feb 2 09:41:46 UTC > > 2021 x86_64 x86_64 x86_64 GNU/Linux > > > > By enabling the kernel locks stats I could see the MatrixMultiplication= is > > hanged in the amdgpu_mn_invalidate_gfx function: > > > > [ 738.359202] 1 lock held by MatrixMultiplic/653: > > [ 738.359206] #0: ffff88810e364fe0 > > (&adev->notifier_lock){+.+.}-{3:3}, at: > > amdgpu_mn_invalidate_gfx+0x34/0xa0 [amdgpu] > > > > I can see in the the amdgpu_mn_invalidate_gfx function: the > > dma_resv_wait_timeout_rcu uses wait_all (fences) and MAX_SCHEDULE_TIMEO= UT so, I > > guess the code gets stuck there waiting forever. According to the > > documentation: "When somebody tries to invalidate the page tables we bl= ock the > > update until all operations on the pages in question are completed, the= n those > > pages are marked as accessed and also dirty if it wasn=E2=80=99t a rea= d only access." > > Looks like the fences are deadlocked and therefore, it never returns. C= ould it > > be possible? any hint to where can I look to fix this? > > > > Thank you in advance. > > > > Here the full dmesg output: > > > > [ 738.337726] INFO: task MatrixMultiplic:653 blocked for more than 122= seconds. > > [ 738.344937] Not tainted 5.11.0-rc6-qtec-standard #2 > > [ 738.350384] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > > disables this message. > > [ 738.358240] task:MatrixMultiplic state:D stack: 0 pid: 653 > > ppid: 1 flags:0x00004000 > > [ 738.358254] Call Trace: > > [ 738.358261] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358276] __schedule+0x370/0x960 > > [ 738.358291] ? dma_fence_default_wait+0x117/0x230 > > [ 738.358297] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358305] schedule+0x51/0xc0 > > [ 738.358312] schedule_timeout+0x275/0x380 > > [ 738.358324] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358332] ? mark_held_locks+0x4f/0x70 > > [ 738.358341] ? dma_fence_default_wait+0x117/0x230 > > [ 738.358347] ? lockdep_hardirqs_on_prepare+0xd4/0x180 > > [ 738.358353] ? _raw_spin_unlock_irqrestore+0x39/0x40 > > [ 738.358362] ? dma_fence_default_wait+0x117/0x230 > > [ 738.358370] ? dma_fence_default_wait+0x1eb/0x230 > > [ 738.358375] dma_fence_default_wait+0x214/0x230 > > [ 738.358384] ? dma_fence_release+0x1a0/0x1a0 > > [ 738.358396] dma_fence_wait_timeout+0x105/0x200 > > [ 738.358405] dma_resv_wait_timeout_rcu+0x1aa/0x5e0 > > [ 738.358421] amdgpu_mn_invalidate_gfx+0x55/0xa0 [amdgpu] > > [ 738.358688] __mmu_notifier_release+0x1bb/0x210 > > [ 738.358710] exit_mmap+0x2f/0x1e0 > > [ 738.358723] ? find_held_lock+0x34/0xa0 > > [ 738.358746] mmput+0x39/0xe0 > > [ 738.358756] do_exit+0x5c3/0xc00 > > [ 738.358763] ? find_held_lock+0x34/0xa0 > > [ 738.358780] do_group_exit+0x47/0xb0 > > [ 738.358791] get_signal+0x15b/0xc50 > > [ 738.358807] arch_do_signal_or_restart+0xaf/0x710 > > [ 738.358816] ? lockdep_hardirqs_on_prepare+0xd4/0x180 > > [ 738.358822] ? _raw_spin_unlock_irqrestore+0x39/0x40 > > [ 738.358831] ? ktime_get_mono_fast_ns+0x50/0xa0 > > [ 738.358844] ? amdgpu_drm_ioctl+0x6b/0x80 [amdgpu] > > [ 738.359044] exit_to_user_mode_prepare+0xf2/0x1b0 > > [ 738.359054] syscall_exit_to_user_mode+0x19/0x60 > > [ 738.359062] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > > [ 738.359069] RIP: 0033:0x7f6b89a51887 > > [ 738.359076] RSP: 002b:00007f6b82b54b18 EFLAGS: 00000246 ORIG_RAX: > > 0000000000000010 > > [ 738.359086] RAX: fffffffffffffe00 RBX: 00007f6b82b54b50 RCX: 00007f6= b89a51887 > > [ 738.359091] RDX: 00007f6b82b54b50 RSI: 00000000c02064c3 RDI: 0000000= 000000007 > > [ 738.359096] RBP: 00000000c02064c3 R08: 0000000000000003 R09: 00007f6= b82b54bbc > > [ 738.359101] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000= 165a0bc00 > > [ 738.359106] R13: 0000000000000007 R14: 0000000000000001 R15: 0000000= 000000000 > > [ 738.359129] > > Showing all locks held in the system: > > [ 738.359141] 1 lock held by khungtaskd/54: > > [ 738.359148] #0: ffffffff829f6840 (rcu_read_lock){....}-{1:2}, at: > > debug_show_all_locks+0x15/0x183 > > [ 738.359187] 1 lock held by systemd-journal/174: > > [ 738.359202] 1 lock held by MatrixMultiplic/653: > > [ 738.359206] #0: ffff88810e364fe0 > > (&adev->notifier_lock){+.+.}-{3:3}, at: > > amdgpu_mn_invalidate_gfx+0x34/0xa0 [amdgpu] > > > > Daniel > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 184F4C433E0 for ; Wed, 3 Feb 2021 14:54:53 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 86EA364E2E for ; Wed, 3 Feb 2021 14:54:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 86EA364E2E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=qtec.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7F96E6EB0F; Wed, 3 Feb 2021 14:54:50 +0000 (UTC) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) by gabe.freedesktop.org (Postfix) with ESMTPS id 19CEB6EB12 for ; Wed, 3 Feb 2021 14:54:49 +0000 (UTC) Received: by mail-lf1-x134.google.com with SMTP id a12so33772695lfb.1 for ; Wed, 03 Feb 2021 06:54:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qtec.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=5wJXA1HJZyVo+PhV0nbxE7xxzJossHm5qQWx4jq7Crc=; b=bBGSQOnXGXfMnytcdOMt81OXgjvBv6qrkAqmO/JaxnYO14p+pg629EWnOAd9cJXF0c HW8Dbq9fwgsjYo1P9JElWarNGktzNuh4Pyv1huIcK27T6Qh25l3qc8cWuVnpv+PM8d6o J0oosa7kNX6wTgqc5TW1Nj5zr8YJFmNuEm/4tbXJlBQLTAQ8zrMXRGgq+II2IjIQp6Jn T549VJgAp5TBa4N7VoqVpiS4bNWi+0B5f5GaZx4DlRwpY99urz27VCFi0gOnrYwBOok6 lges+uaQ222fuJnxQFAseX/ukWs5KpoEry+eJT7nXItAdxkQm0Q1cdKXcaCUxDZeWdee bJiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5wJXA1HJZyVo+PhV0nbxE7xxzJossHm5qQWx4jq7Crc=; b=IblE+NoEMuH5V/7AUiXCcclOxJ3LWhc+9+2HAzGXLHmB+Nnv1sJ6iaAhDeV6Fcopgk QRGC8JQU7313dh+nBApvPPL1Fdi4TTeqwvHIMigXE9jnDlJUmIw+o27mZEeuhXv8E0eZ e3gopGqV577wsp3rkIp9eiZlT2vAsWa5d5MXq6IKA63JIR0ryVlC78cDlrPDMWr5kQrt rRVMZA8CsgqVUpuNkx3dhTCS9HnI4aVGuboHVSOIlplgt9EEyStx1dWz0QE1T6DA48Gw J9vX7xMez9IMcWhadw8pC2LIaKfTMEpBAhfZAbUGGzIiqawHO2u81xCjdpBPKyIn4YLj j4mQ== X-Gm-Message-State: AOAM532EcZd9/T+uVg96c+NSN0HEE3u1QjvoFGPGKrzg5sIei0hNRyEU FnX6pEU8eVf2h5+7xw/OMn3raDcxHV40+lIs1cwTqw== X-Google-Smtp-Source: ABdhPJw9ybjbBjCIBrkrj8loyUZJeejScm+VzMN5pMRxewJEYUQWPnOJl+ZLGh7hUNzxuHiU3VIUaGCS9ITAqQHULmo= X-Received: by 2002:a05:6512:376f:: with SMTP id z15mr1960636lft.59.1612364087238; Wed, 03 Feb 2021 06:54:47 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Daniel Gomez Date: Wed, 3 Feb 2021 15:54:36 +0100 Message-ID: Subject: Re: [amdgpu] deadlock To: =?UTF-8?Q?Christian_K=C3=B6nig?= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , dri-devel , amd-gfx list , Linux Kernel Mailing List Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" T24gV2VkLCAzIEZlYiAyMDIxIGF0IDE1OjM3LCBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4u a29lbmlnQGFtZC5jb20+IHdyb3RlOgo+Cj4gSGkgRGFuaWVsLAo+Cj4gSSd2ZSB0YWxrZWQgYSBi aXQgd2l0aCBvdXIgaW50ZXJuYWwgdGVhbS4KPgo+IFRoZSBwcm9ibGVtIGlzIHRoYXQgdGhlIDIw LjIwIHJlbGVhc2Ugc3RpbGwgdXNlcyB0aGUgb2xkZXIgT3BlbkNMIHN0YWNrCj4gd2hpY2ggb2J2 aW91c2x5IGhhcyBhIGJ1ZyBoZXJlIGFuZCBjYXVzZXMgYSBoYW5nLgo+Cj4gVGhlIGJlc3QgYXBw cm9hY2ggSSBjYW4gZ2l2ZSB5b3UgaXMgdG8gc3dpdGNoIHRvIHRoZSBST0NtIHN0YWNrIGluc3Rl YWQuClRoYW5rcyBDaHJpc3RpYW4uIEknbGwgdHJ5IHdpdGggdGhlIFJPQ20gc3RhY2sgdGhlbi4g QXMgZmFyIGFzIEkgdW5kZXJzdG9vZCwKaXQgc2hvdWxkIHdvcmsgYmVjYXVzZSB0aGUgcGFydCBv ZiB0aGUgY29kZSB3aGVyZSBpdCBub3cgaGFuZ3MgaXMgbm90IGFjdHVhbGx5CnVzZWQgYnkgdGhl IFJPQ20gc3RhY2ssIGlzIHRoYXQgY29ycmVjdD8gSG93ZXZlciwgdGhlIGhhbmcvYnVnIHdpbGwK c3RpbGwgYmUgdGhlcmUKZXZlbiB0aG91Z2ggaXQgaXMgbm90IHVzZWQgaW4gdGhhdCBzdGFjay4K QW55d2F5LCBJJ2xsIGtlZXAgeW91IGd1eXMgcG9zdGVkIHdpdGggdGhpcyBjaGFuZ2UuCgo+Cj4g UmVnYXJkcywKPiBDaHJpc3RpYW4uCj4KPiBBbSAwMy4wMi4yMSB1bSAwOTozMyBzY2hyaWViIERh bmllbCBHb21lejoKPiA+IEhpIGFsbCwKPiA+Cj4gPiBJIGhhdmUgYSBkZWFkbG9jayB3aXRoIHRo ZSBhbWRncHUgbWFpbmxpbmUgZHJpdmVyIHdoZW4gcnVubmluZyBpbiBwYXJhbGxlbCB0d28KPiA+ IE9wZW5DTCBhcHBsaWNhdGlvbnMuIFNvIGZhciwgd2UndmUgYmVlbiBhYmxlIHRvIHJlcGxpY2F0 ZSBpdCBlYXNpbHkgYnkgZXhlY3V0aW5nCj4gPiBjbGluZm8gYW5kIE1hdHJpeE11bHRpcGxpY2F0 aW9uIChmcm9tIEFNRCBvcGVuY2wtc2FtcGxlcykuIEl0J3MgcXVpdGUgb2xkIHRoZQo+ID4gb3Bl bmNsLXNhbXBsZXMgc28sIGlmIHlvdSBoYXZlIGFueSBvdGhlciBzdWdnZXN0aW9uIGZvciB0ZXN0 aW5nIEknZCBiZSB2ZXJ5Cj4gPiBoYXBweSB0byB0ZXN0IGl0IGFzIHdlbGwuCj4gPgo+ID4gSG93 IHRvIHJlcGxpY2F0ZSB0aGUgaXNzdWU6Cj4gPgo+ID4gIyB3aGlsZSB0cnVlOyBkbyAvdXNyL2Jp bi9NYXRyaXhNdWx0aXBsaWNhdGlvbiAtLWRldmljZSBncHUgXAo+ID4gICAgICAtLWRldmljZUlk IDAgLXggMTAwMCAteSAxMDAwIC16IDEwMDAgLXEgLXQgLWkgNTA7IGRvbmUKPiA+ICMgd2hpbGUg dHJ1ZTsgZG8gY2xpbmZvOyBkb25lCj4gPgo+ID4gT3V0cHV0Ogo+ID4KPiA+IEFmdGVyIGEgbWlu dXRlIG9yIGxlc3MgKHNvbWV0aW1lcyBjb3VsZCBiZSBtb3JlKSBJIGNhbiBzZWUgdGhhdAo+ID4g TWF0cml4TXVsdGlwbGljYXRpb24gYW5kIGNsaW5mbyBoYW5nLiBJbiBhZGRpdGlvbiwgd2l0aCBy YWRlb250b3AgeW91IGNhbiBzZWUKPiA+IGhvdyB0aGUgR3JhcGhpY3MgcGlwZSBnb2VzIGZyb20g fjUwJSB0byAxMDAlLiBBbHNvIHRoZSBzaGFkZXIgY2xvY2tzCj4gPiBnb2VzIHVwIGZyb20gfjM1 JSB0byB+OTYlLgo+ID4KPiA+IGNsaW5mbyBrZWVwcyBwcmludGluZzoKPiA+IGlvY3RsKDcsIERS TV9JT0NUTF9TWU5DT0JKX1dBSVQsIDB4N2ZmZTQ2ZTVmOTUwKSA9IC0xIEVUSU1FIChUaW1lciBl eHBpcmVkKQo+ID4KPiA+IEFuZCBNYXRyaXhNdWx0aXBsaWNhdGlvbiBwcmludHMgdGhlIGZvbGxv d2luZyAoc3RyYWNlKSBpZiB5b3UgdHJ5IHRvCj4gPiBraWxsIHRoZSBwcm9jZXNzOgo+ID4KPiA+ IHNjaGVkX3lpZWxkKCkgICAgICAgICAgICAgICAgICAgICAgICAgICA9IDAKPiA+IGZ1dGV4KDB4 NTU3ZTk0NTM0M2I4LCBGVVRFWF9XQUlUX0JJVFNFVF9QUklWQVRFfEZVVEVYX0NMT0NLX1JFQUxU SU1FLCAwLAo+ID4gTlVMTCwgRlVURVhfQklUU0VUX01BVENIX0FOWXN0cmFjZTogUHJvY2VzcyA2 NTEgZGV0YWNoZWQKPiA+ICAgPGRldGFjaGVkIC4uLj4KPiA+Cj4gPiBBZnRlciB0aGlzLCB0aGUg Z3B1IGlzIG5vdCBmdW5jdGlvbmFsIGF0IGFsbCBhbmQgeW91J2QgbmVlZCBhIHBvd2VyIGN5Y2xl IHJlc2V0Cj4gPiB0byByZXN0b3JlIHRoZSBzeXN0ZW0uCj4gPgo+ID4gSGFyZHdhcmUgaW5mbzoK PiA+IENQVTogQU1EIFJ5emVuIEVtYmVkZGVkIFYxNjA1QiB3aXRoIFJhZGVvbiBWZWdhIEdmeCAo OCkgQCAyLjAwMEdIego+ID4gR1BVOiBBTUQgQVRJIFJhZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVv biBWZWdhIE1vYmlsZSBTZXJpZXMKPiA+Cj4gPiAwMzowMC4wIFZHQSBjb21wYXRpYmxlIGNvbnRy b2xsZXI6IEFkdmFuY2VkIE1pY3JvIERldmljZXMsIEluYy4KPiA+IFtBTUQvQVRJXSBSYXZlbiBS aWRnZSBbUmFkZW9uIFZlZ2EgU2VyaWVzIC8gUmFkZW9uIFZlZ2EgTW9iaWxlIFNlcmllc10KPiA+ IChyZXYgODMpCj4gPiAgICAgIERldmljZU5hbWU6IEJyb2FkY29tIDU3NjIKPiA+ICAgICAgU3Vi c3lzdGVtOiBBZHZhbmNlZCBNaWNybyBEZXZpY2VzLCBJbmMuIFtBTUQvQVRJXSBSYXZlbiBSaWRn ZQo+ID4gW1JhZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVvbiBWZWdhIE1vYmlsZSBTZXJpZXNdCj4g PiAgICAgIEtlcm5lbCBkcml2ZXIgaW4gdXNlOiBhbWRncHUKPiA+ICAgICAgS2VybmVsIG1vZHVs ZXM6IGFtZGdwdQo+ID4KPiA+IExpbnV4IGtlcm5lbCBpbmZvOgo+ID4KPiA+IHJvb3RAcXQ1MjIy On4jIHVuYW1lIC1hCj4gPiBMaW51eCBxdDUyMjIgNS4xMS4wLXJjNi1xdGVjLXN0YW5kYXJkICMy IFNNUCBUdWUgRmViIDIgMDk6NDE6NDYgVVRDCj4gPiAyMDIxIHg4Nl82NCB4ODZfNjQgeDg2XzY0 IEdOVS9MaW51eAo+ID4KPiA+IEJ5IGVuYWJsaW5nIHRoZSBrZXJuZWwgbG9ja3Mgc3RhdHMgSSBj b3VsZCBzZWUgdGhlIE1hdHJpeE11bHRpcGxpY2F0aW9uIGlzCj4gPiBoYW5nZWQgaW4gdGhlIGFt ZGdwdV9tbl9pbnZhbGlkYXRlX2dmeCBmdW5jdGlvbjoKPiA+Cj4gPiBbICA3MzguMzU5MjAyXSAx IGxvY2sgaGVsZCBieSBNYXRyaXhNdWx0aXBsaWMvNjUzOgo+ID4gWyAgNzM4LjM1OTIwNl0gICMw OiBmZmZmODg4MTBlMzY0ZmUwCj4gPiAoJmFkZXYtPm5vdGlmaWVyX2xvY2speysuKy59LXszOjN9 LCBhdDoKPiA+IGFtZGdwdV9tbl9pbnZhbGlkYXRlX2dmeCsweDM0LzB4YTAgW2FtZGdwdV0KPiA+ Cj4gPiBJIGNhbiBzZWUgaW4gdGhlIHRoZSBhbWRncHVfbW5faW52YWxpZGF0ZV9nZnggZnVuY3Rp b246IHRoZQo+ID4gZG1hX3Jlc3Zfd2FpdF90aW1lb3V0X3JjdSB1c2VzIHdhaXRfYWxsIChmZW5j ZXMpIGFuZCBNQVhfU0NIRURVTEVfVElNRU9VVCBzbywgSQo+ID4gZ3Vlc3MgdGhlIGNvZGUgZ2V0 cyBzdHVjayB0aGVyZSB3YWl0aW5nIGZvcmV2ZXIuIEFjY29yZGluZyB0byB0aGUKPiA+IGRvY3Vt ZW50YXRpb246ICJXaGVuIHNvbWVib2R5IHRyaWVzIHRvIGludmFsaWRhdGUgdGhlIHBhZ2UgdGFi bGVzIHdlIGJsb2NrIHRoZQo+ID4gdXBkYXRlIHVudGlsIGFsbCBvcGVyYXRpb25zIG9uIHRoZSBw YWdlcyBpbiBxdWVzdGlvbiBhcmUgY29tcGxldGVkLCB0aGVuIHRob3NlCj4gPiBwYWdlcyBhcmUg bWFya2VkICBhcyBhY2Nlc3NlZCBhbmQgYWxzbyBkaXJ0eSBpZiBpdCB3YXNu4oCZdCBhIHJlYWQg b25seSBhY2Nlc3MuIgo+ID4gTG9va3MgbGlrZSB0aGUgZmVuY2VzIGFyZSBkZWFkbG9ja2VkIGFu ZCB0aGVyZWZvcmUsIGl0IG5ldmVyIHJldHVybnMuIENvdWxkIGl0Cj4gPiBiZSBwb3NzaWJsZT8g YW55IGhpbnQgdG8gd2hlcmUgY2FuIEkgbG9vayB0byBmaXggdGhpcz8KPiA+Cj4gPiBUaGFuayB5 b3UgIGluIGFkdmFuY2UuCj4gPgo+ID4gSGVyZSB0aGUgZnVsbCBkbWVzZyBvdXRwdXQ6Cj4gPgo+ ID4gWyAgNzM4LjMzNzcyNl0gSU5GTzogdGFzayBNYXRyaXhNdWx0aXBsaWM6NjUzIGJsb2NrZWQg Zm9yIG1vcmUgdGhhbiAxMjIgc2Vjb25kcy4KPiA+IFsgIDczOC4zNDQ5MzddICAgICAgIE5vdCB0 YWludGVkIDUuMTEuMC1yYzYtcXRlYy1zdGFuZGFyZCAjMgo+ID4gWyAgNzM4LjM1MDM4NF0gImVj aG8gMCA+IC9wcm9jL3N5cy9rZXJuZWwvaHVuZ190YXNrX3RpbWVvdXRfc2VjcyIKPiA+IGRpc2Fi bGVzIHRoaXMgbWVzc2FnZS4KPiA+IFsgIDczOC4zNTgyNDBdIHRhc2s6TWF0cml4TXVsdGlwbGlj IHN0YXRlOkQgc3RhY2s6ICAgIDAgcGlkOiAgNjUzCj4gPiBwcGlkOiAgICAgMSBmbGFnczoweDAw MDA0MDAwCj4gPiBbICA3MzguMzU4MjU0XSBDYWxsIFRyYWNlOgo+ID4gWyAgNzM4LjM1ODI2MV0g ID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDFlYi8weDIzMAo+ID4gWyAgNzM4LjM1ODI3Nl0g IF9fc2NoZWR1bGUrMHgzNzAvMHg5NjAKPiA+IFsgIDczOC4zNTgyOTFdICA/IGRtYV9mZW5jZV9k ZWZhdWx0X3dhaXQrMHgxMTcvMHgyMzAKPiA+IFsgIDczOC4zNTgyOTddICA/IGRtYV9mZW5jZV9k ZWZhdWx0X3dhaXQrMHgxZWIvMHgyMzAKPiA+IFsgIDczOC4zNTgzMDVdICBzY2hlZHVsZSsweDUx LzB4YzAKPiA+IFsgIDczOC4zNTgzMTJdICBzY2hlZHVsZV90aW1lb3V0KzB4Mjc1LzB4MzgwCj4g PiBbICA3MzguMzU4MzI0XSAgPyBkbWFfZmVuY2VfZGVmYXVsdF93YWl0KzB4MWViLzB4MjMwCj4g PiBbICA3MzguMzU4MzMyXSAgPyBtYXJrX2hlbGRfbG9ja3MrMHg0Zi8weDcwCj4gPiBbICA3Mzgu MzU4MzQxXSAgPyBkbWFfZmVuY2VfZGVmYXVsdF93YWl0KzB4MTE3LzB4MjMwCj4gPiBbICA3Mzgu MzU4MzQ3XSAgPyBsb2NrZGVwX2hhcmRpcnFzX29uX3ByZXBhcmUrMHhkNC8weDE4MAo+ID4gWyAg NzM4LjM1ODM1M10gID8gX3Jhd19zcGluX3VubG9ja19pcnFyZXN0b3JlKzB4MzkvMHg0MAo+ID4g WyAgNzM4LjM1ODM2Ml0gID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDExNy8weDIzMAo+ID4g WyAgNzM4LjM1ODM3MF0gID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDFlYi8weDIzMAo+ID4g WyAgNzM4LjM1ODM3NV0gIGRtYV9mZW5jZV9kZWZhdWx0X3dhaXQrMHgyMTQvMHgyMzAKPiA+IFsg IDczOC4zNTgzODRdICA/IGRtYV9mZW5jZV9yZWxlYXNlKzB4MWEwLzB4MWEwCj4gPiBbICA3Mzgu MzU4Mzk2XSAgZG1hX2ZlbmNlX3dhaXRfdGltZW91dCsweDEwNS8weDIwMAo+ID4gWyAgNzM4LjM1 ODQwNV0gIGRtYV9yZXN2X3dhaXRfdGltZW91dF9yY3UrMHgxYWEvMHg1ZTAKPiA+IFsgIDczOC4z NTg0MjFdICBhbWRncHVfbW5faW52YWxpZGF0ZV9nZngrMHg1NS8weGEwIFthbWRncHVdCj4gPiBb ICA3MzguMzU4Njg4XSAgX19tbXVfbm90aWZpZXJfcmVsZWFzZSsweDFiYi8weDIxMAo+ID4gWyAg NzM4LjM1ODcxMF0gIGV4aXRfbW1hcCsweDJmLzB4MWUwCj4gPiBbICA3MzguMzU4NzIzXSAgPyBm aW5kX2hlbGRfbG9jaysweDM0LzB4YTAKPiA+IFsgIDczOC4zNTg3NDZdICBtbXB1dCsweDM5LzB4 ZTAKPiA+IFsgIDczOC4zNTg3NTZdICBkb19leGl0KzB4NWMzLzB4YzAwCj4gPiBbICA3MzguMzU4 NzYzXSAgPyBmaW5kX2hlbGRfbG9jaysweDM0LzB4YTAKPiA+IFsgIDczOC4zNTg3ODBdICBkb19n cm91cF9leGl0KzB4NDcvMHhiMAo+ID4gWyAgNzM4LjM1ODc5MV0gIGdldF9zaWduYWwrMHgxNWIv MHhjNTAKPiA+IFsgIDczOC4zNTg4MDddICBhcmNoX2RvX3NpZ25hbF9vcl9yZXN0YXJ0KzB4YWYv MHg3MTAKPiA+IFsgIDczOC4zNTg4MTZdICA/IGxvY2tkZXBfaGFyZGlycXNfb25fcHJlcGFyZSsw eGQ0LzB4MTgwCj4gPiBbICA3MzguMzU4ODIyXSAgPyBfcmF3X3NwaW5fdW5sb2NrX2lycXJlc3Rv cmUrMHgzOS8weDQwCj4gPiBbICA3MzguMzU4ODMxXSAgPyBrdGltZV9nZXRfbW9ub19mYXN0X25z KzB4NTAvMHhhMAo+ID4gWyAgNzM4LjM1ODg0NF0gID8gYW1kZ3B1X2RybV9pb2N0bCsweDZiLzB4 ODAgW2FtZGdwdV0KPiA+IFsgIDczOC4zNTkwNDRdICBleGl0X3RvX3VzZXJfbW9kZV9wcmVwYXJl KzB4ZjIvMHgxYjAKPiA+IFsgIDczOC4zNTkwNTRdICBzeXNjYWxsX2V4aXRfdG9fdXNlcl9tb2Rl KzB4MTkvMHg2MAo+ID4gWyAgNzM4LjM1OTA2Ml0gIGVudHJ5X1NZU0NBTExfNjRfYWZ0ZXJfaHdm cmFtZSsweDQ0LzB4YTkKPiA+IFsgIDczOC4zNTkwNjldIFJJUDogMDAzMzoweDdmNmI4OWE1MTg4 Nwo+ID4gWyAgNzM4LjM1OTA3Nl0gUlNQOiAwMDJiOjAwMDA3ZjZiODJiNTRiMTggRUZMQUdTOiAw MDAwMDI0NiBPUklHX1JBWDoKPiA+IDAwMDAwMDAwMDAwMDAwMTAKPiA+IFsgIDczOC4zNTkwODZd IFJBWDogZmZmZmZmZmZmZmZmZmUwMCBSQlg6IDAwMDA3ZjZiODJiNTRiNTAgUkNYOiAwMDAwN2Y2 Yjg5YTUxODg3Cj4gPiBbICA3MzguMzU5MDkxXSBSRFg6IDAwMDA3ZjZiODJiNTRiNTAgUlNJOiAw MDAwMDAwMGMwMjA2NGMzIFJESTogMDAwMDAwMDAwMDAwMDAwNwo+ID4gWyAgNzM4LjM1OTA5Nl0g UkJQOiAwMDAwMDAwMGMwMjA2NGMzIFIwODogMDAwMDAwMDAwMDAwMDAwMyBSMDk6IDAwMDA3ZjZi ODJiNTRiYmMKPiA+IFsgIDczOC4zNTkxMDFdIFIxMDogMDAwMDAwMDAwMDAwMDAwMSBSMTE6IDAw MDAwMDAwMDAwMDAyNDYgUjEyOiAwMDAwMDAwMTY1YTBiYzAwCj4gPiBbICA3MzguMzU5MTA2XSBS MTM6IDAwMDAwMDAwMDAwMDAwMDcgUjE0OiAwMDAwMDAwMDAwMDAwMDAxIFIxNTogMDAwMDAwMDAw MDAwMDAwMAo+ID4gWyAgNzM4LjM1OTEyOV0KPiA+ICAgICAgICAgICAgICAgICBTaG93aW5nIGFs bCBsb2NrcyBoZWxkIGluIHRoZSBzeXN0ZW06Cj4gPiBbICA3MzguMzU5MTQxXSAxIGxvY2sgaGVs ZCBieSBraHVuZ3Rhc2tkLzU0Ogo+ID4gWyAgNzM4LjM1OTE0OF0gICMwOiBmZmZmZmZmZjgyOWY2 ODQwIChyY3VfcmVhZF9sb2NrKXsuLi4ufS17MToyfSwgYXQ6Cj4gPiBkZWJ1Z19zaG93X2FsbF9s b2NrcysweDE1LzB4MTgzCj4gPiBbICA3MzguMzU5MTg3XSAxIGxvY2sgaGVsZCBieSBzeXN0ZW1k LWpvdXJuYWwvMTc0Ogo+ID4gWyAgNzM4LjM1OTIwMl0gMSBsb2NrIGhlbGQgYnkgTWF0cml4TXVs dGlwbGljLzY1MzoKPiA+IFsgIDczOC4zNTkyMDZdICAjMDogZmZmZjg4ODEwZTM2NGZlMAo+ID4g KCZhZGV2LT5ub3RpZmllcl9sb2NrKXsrLisufS17MzozfSwgYXQ6Cj4gPiBhbWRncHVfbW5faW52 YWxpZGF0ZV9nZngrMHgzNC8weGEwIFthbWRncHVdCj4gPgo+ID4gRGFuaWVsCj4KX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KZHJpLWRldmVsIG1haWxpbmcg bGlzdApkcmktZGV2ZWxAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0dHBzOi8vbGlzdHMuZnJlZWRl c2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vZHJpLWRldmVsCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59C5DC433DB for ; Wed, 3 Feb 2021 14:54:51 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B8B7A64E31 for ; Wed, 3 Feb 2021 14:54:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B8B7A64E31 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=qtec.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=amd-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 41D366EB0E; Wed, 3 Feb 2021 14:54:50 +0000 (UTC) Received: from mail-lf1-x134.google.com (mail-lf1-x134.google.com [IPv6:2a00:1450:4864:20::134]) by gabe.freedesktop.org (Postfix) with ESMTPS id 319A16EB34 for ; Wed, 3 Feb 2021 14:54:49 +0000 (UTC) Received: by mail-lf1-x134.google.com with SMTP id d3so7775987lfg.10 for ; Wed, 03 Feb 2021 06:54:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=qtec.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=5wJXA1HJZyVo+PhV0nbxE7xxzJossHm5qQWx4jq7Crc=; b=bBGSQOnXGXfMnytcdOMt81OXgjvBv6qrkAqmO/JaxnYO14p+pg629EWnOAd9cJXF0c HW8Dbq9fwgsjYo1P9JElWarNGktzNuh4Pyv1huIcK27T6Qh25l3qc8cWuVnpv+PM8d6o J0oosa7kNX6wTgqc5TW1Nj5zr8YJFmNuEm/4tbXJlBQLTAQ8zrMXRGgq+II2IjIQp6Jn T549VJgAp5TBa4N7VoqVpiS4bNWi+0B5f5GaZx4DlRwpY99urz27VCFi0gOnrYwBOok6 lges+uaQ222fuJnxQFAseX/ukWs5KpoEry+eJT7nXItAdxkQm0Q1cdKXcaCUxDZeWdee bJiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=5wJXA1HJZyVo+PhV0nbxE7xxzJossHm5qQWx4jq7Crc=; b=RBCoIuHO1zHxZT5OcX+VzdJ9T6KFHbdDL+xJ8LRlXAnQ3TsfSfXM0cqankndYX7tYG t3PDVTtYRSxDobAOemFiNSq0WsCdO87b6I41ahEaCu57VqQ+deogoCnekim192TpQ8xR /34Uk697O2NVYV/Nk/0gLBDAooQDaLV5me4GHjpgy3fdgnpwf/uq5rSclySKStb60sP1 q7CxVpfkdzx5egxJXuAfkv6EwaJnQ/WOMUo37sqWVUT4KYZiL4bpIMkWqGx8GvjkJzwz GAb/tIgfwe2f7yr7fR/GWSrkMCmQQ4j+e9zGyxUNFYm+bBtLjrgLIT7uXZT9SVEqk95Q KTIw== X-Gm-Message-State: AOAM5318H65wHkngIf/eJHr4g7frViKtxyV7XqvOPr5dY6e8G8CVrTrX 2e/Ci1gscDcUFThh+0CeNBAAhIGpmDI9izG4krVn/A== X-Google-Smtp-Source: ABdhPJw9ybjbBjCIBrkrj8loyUZJeejScm+VzMN5pMRxewJEYUQWPnOJl+ZLGh7hUNzxuHiU3VIUaGCS9ITAqQHULmo= X-Received: by 2002:a05:6512:376f:: with SMTP id z15mr1960636lft.59.1612364087238; Wed, 03 Feb 2021 06:54:47 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Daniel Gomez Date: Wed, 3 Feb 2021 15:54:36 +0100 Message-ID: Subject: Re: [amdgpu] deadlock To: =?UTF-8?Q?Christian_K=C3=B6nig?= X-BeenThere: amd-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion list for AMD gfx List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alex Deucher , dri-devel , amd-gfx list , Linux Kernel Mailing List Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: amd-gfx-bounces@lists.freedesktop.org Sender: "amd-gfx" T24gV2VkLCAzIEZlYiAyMDIxIGF0IDE1OjM3LCBDaHJpc3RpYW4gS8O2bmlnIDxjaHJpc3RpYW4u a29lbmlnQGFtZC5jb20+IHdyb3RlOgo+Cj4gSGkgRGFuaWVsLAo+Cj4gSSd2ZSB0YWxrZWQgYSBi aXQgd2l0aCBvdXIgaW50ZXJuYWwgdGVhbS4KPgo+IFRoZSBwcm9ibGVtIGlzIHRoYXQgdGhlIDIw LjIwIHJlbGVhc2Ugc3RpbGwgdXNlcyB0aGUgb2xkZXIgT3BlbkNMIHN0YWNrCj4gd2hpY2ggb2J2 aW91c2x5IGhhcyBhIGJ1ZyBoZXJlIGFuZCBjYXVzZXMgYSBoYW5nLgo+Cj4gVGhlIGJlc3QgYXBw cm9hY2ggSSBjYW4gZ2l2ZSB5b3UgaXMgdG8gc3dpdGNoIHRvIHRoZSBST0NtIHN0YWNrIGluc3Rl YWQuClRoYW5rcyBDaHJpc3RpYW4uIEknbGwgdHJ5IHdpdGggdGhlIFJPQ20gc3RhY2sgdGhlbi4g QXMgZmFyIGFzIEkgdW5kZXJzdG9vZCwKaXQgc2hvdWxkIHdvcmsgYmVjYXVzZSB0aGUgcGFydCBv ZiB0aGUgY29kZSB3aGVyZSBpdCBub3cgaGFuZ3MgaXMgbm90IGFjdHVhbGx5CnVzZWQgYnkgdGhl IFJPQ20gc3RhY2ssIGlzIHRoYXQgY29ycmVjdD8gSG93ZXZlciwgdGhlIGhhbmcvYnVnIHdpbGwK c3RpbGwgYmUgdGhlcmUKZXZlbiB0aG91Z2ggaXQgaXMgbm90IHVzZWQgaW4gdGhhdCBzdGFjay4K QW55d2F5LCBJJ2xsIGtlZXAgeW91IGd1eXMgcG9zdGVkIHdpdGggdGhpcyBjaGFuZ2UuCgo+Cj4g UmVnYXJkcywKPiBDaHJpc3RpYW4uCj4KPiBBbSAwMy4wMi4yMSB1bSAwOTozMyBzY2hyaWViIERh bmllbCBHb21lejoKPiA+IEhpIGFsbCwKPiA+Cj4gPiBJIGhhdmUgYSBkZWFkbG9jayB3aXRoIHRo ZSBhbWRncHUgbWFpbmxpbmUgZHJpdmVyIHdoZW4gcnVubmluZyBpbiBwYXJhbGxlbCB0d28KPiA+ IE9wZW5DTCBhcHBsaWNhdGlvbnMuIFNvIGZhciwgd2UndmUgYmVlbiBhYmxlIHRvIHJlcGxpY2F0 ZSBpdCBlYXNpbHkgYnkgZXhlY3V0aW5nCj4gPiBjbGluZm8gYW5kIE1hdHJpeE11bHRpcGxpY2F0 aW9uIChmcm9tIEFNRCBvcGVuY2wtc2FtcGxlcykuIEl0J3MgcXVpdGUgb2xkIHRoZQo+ID4gb3Bl bmNsLXNhbXBsZXMgc28sIGlmIHlvdSBoYXZlIGFueSBvdGhlciBzdWdnZXN0aW9uIGZvciB0ZXN0 aW5nIEknZCBiZSB2ZXJ5Cj4gPiBoYXBweSB0byB0ZXN0IGl0IGFzIHdlbGwuCj4gPgo+ID4gSG93 IHRvIHJlcGxpY2F0ZSB0aGUgaXNzdWU6Cj4gPgo+ID4gIyB3aGlsZSB0cnVlOyBkbyAvdXNyL2Jp bi9NYXRyaXhNdWx0aXBsaWNhdGlvbiAtLWRldmljZSBncHUgXAo+ID4gICAgICAtLWRldmljZUlk IDAgLXggMTAwMCAteSAxMDAwIC16IDEwMDAgLXEgLXQgLWkgNTA7IGRvbmUKPiA+ICMgd2hpbGUg dHJ1ZTsgZG8gY2xpbmZvOyBkb25lCj4gPgo+ID4gT3V0cHV0Ogo+ID4KPiA+IEFmdGVyIGEgbWlu dXRlIG9yIGxlc3MgKHNvbWV0aW1lcyBjb3VsZCBiZSBtb3JlKSBJIGNhbiBzZWUgdGhhdAo+ID4g TWF0cml4TXVsdGlwbGljYXRpb24gYW5kIGNsaW5mbyBoYW5nLiBJbiBhZGRpdGlvbiwgd2l0aCBy YWRlb250b3AgeW91IGNhbiBzZWUKPiA+IGhvdyB0aGUgR3JhcGhpY3MgcGlwZSBnb2VzIGZyb20g fjUwJSB0byAxMDAlLiBBbHNvIHRoZSBzaGFkZXIgY2xvY2tzCj4gPiBnb2VzIHVwIGZyb20gfjM1 JSB0byB+OTYlLgo+ID4KPiA+IGNsaW5mbyBrZWVwcyBwcmludGluZzoKPiA+IGlvY3RsKDcsIERS TV9JT0NUTF9TWU5DT0JKX1dBSVQsIDB4N2ZmZTQ2ZTVmOTUwKSA9IC0xIEVUSU1FIChUaW1lciBl eHBpcmVkKQo+ID4KPiA+IEFuZCBNYXRyaXhNdWx0aXBsaWNhdGlvbiBwcmludHMgdGhlIGZvbGxv d2luZyAoc3RyYWNlKSBpZiB5b3UgdHJ5IHRvCj4gPiBraWxsIHRoZSBwcm9jZXNzOgo+ID4KPiA+ IHNjaGVkX3lpZWxkKCkgICAgICAgICAgICAgICAgICAgICAgICAgICA9IDAKPiA+IGZ1dGV4KDB4 NTU3ZTk0NTM0M2I4LCBGVVRFWF9XQUlUX0JJVFNFVF9QUklWQVRFfEZVVEVYX0NMT0NLX1JFQUxU SU1FLCAwLAo+ID4gTlVMTCwgRlVURVhfQklUU0VUX01BVENIX0FOWXN0cmFjZTogUHJvY2VzcyA2 NTEgZGV0YWNoZWQKPiA+ICAgPGRldGFjaGVkIC4uLj4KPiA+Cj4gPiBBZnRlciB0aGlzLCB0aGUg Z3B1IGlzIG5vdCBmdW5jdGlvbmFsIGF0IGFsbCBhbmQgeW91J2QgbmVlZCBhIHBvd2VyIGN5Y2xl IHJlc2V0Cj4gPiB0byByZXN0b3JlIHRoZSBzeXN0ZW0uCj4gPgo+ID4gSGFyZHdhcmUgaW5mbzoK PiA+IENQVTogQU1EIFJ5emVuIEVtYmVkZGVkIFYxNjA1QiB3aXRoIFJhZGVvbiBWZWdhIEdmeCAo OCkgQCAyLjAwMEdIego+ID4gR1BVOiBBTUQgQVRJIFJhZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVv biBWZWdhIE1vYmlsZSBTZXJpZXMKPiA+Cj4gPiAwMzowMC4wIFZHQSBjb21wYXRpYmxlIGNvbnRy b2xsZXI6IEFkdmFuY2VkIE1pY3JvIERldmljZXMsIEluYy4KPiA+IFtBTUQvQVRJXSBSYXZlbiBS aWRnZSBbUmFkZW9uIFZlZ2EgU2VyaWVzIC8gUmFkZW9uIFZlZ2EgTW9iaWxlIFNlcmllc10KPiA+ IChyZXYgODMpCj4gPiAgICAgIERldmljZU5hbWU6IEJyb2FkY29tIDU3NjIKPiA+ICAgICAgU3Vi c3lzdGVtOiBBZHZhbmNlZCBNaWNybyBEZXZpY2VzLCBJbmMuIFtBTUQvQVRJXSBSYXZlbiBSaWRn ZQo+ID4gW1JhZGVvbiBWZWdhIFNlcmllcyAvIFJhZGVvbiBWZWdhIE1vYmlsZSBTZXJpZXNdCj4g PiAgICAgIEtlcm5lbCBkcml2ZXIgaW4gdXNlOiBhbWRncHUKPiA+ICAgICAgS2VybmVsIG1vZHVs ZXM6IGFtZGdwdQo+ID4KPiA+IExpbnV4IGtlcm5lbCBpbmZvOgo+ID4KPiA+IHJvb3RAcXQ1MjIy On4jIHVuYW1lIC1hCj4gPiBMaW51eCBxdDUyMjIgNS4xMS4wLXJjNi1xdGVjLXN0YW5kYXJkICMy IFNNUCBUdWUgRmViIDIgMDk6NDE6NDYgVVRDCj4gPiAyMDIxIHg4Nl82NCB4ODZfNjQgeDg2XzY0 IEdOVS9MaW51eAo+ID4KPiA+IEJ5IGVuYWJsaW5nIHRoZSBrZXJuZWwgbG9ja3Mgc3RhdHMgSSBj b3VsZCBzZWUgdGhlIE1hdHJpeE11bHRpcGxpY2F0aW9uIGlzCj4gPiBoYW5nZWQgaW4gdGhlIGFt ZGdwdV9tbl9pbnZhbGlkYXRlX2dmeCBmdW5jdGlvbjoKPiA+Cj4gPiBbICA3MzguMzU5MjAyXSAx IGxvY2sgaGVsZCBieSBNYXRyaXhNdWx0aXBsaWMvNjUzOgo+ID4gWyAgNzM4LjM1OTIwNl0gICMw OiBmZmZmODg4MTBlMzY0ZmUwCj4gPiAoJmFkZXYtPm5vdGlmaWVyX2xvY2speysuKy59LXszOjN9 LCBhdDoKPiA+IGFtZGdwdV9tbl9pbnZhbGlkYXRlX2dmeCsweDM0LzB4YTAgW2FtZGdwdV0KPiA+ Cj4gPiBJIGNhbiBzZWUgaW4gdGhlIHRoZSBhbWRncHVfbW5faW52YWxpZGF0ZV9nZnggZnVuY3Rp b246IHRoZQo+ID4gZG1hX3Jlc3Zfd2FpdF90aW1lb3V0X3JjdSB1c2VzIHdhaXRfYWxsIChmZW5j ZXMpIGFuZCBNQVhfU0NIRURVTEVfVElNRU9VVCBzbywgSQo+ID4gZ3Vlc3MgdGhlIGNvZGUgZ2V0 cyBzdHVjayB0aGVyZSB3YWl0aW5nIGZvcmV2ZXIuIEFjY29yZGluZyB0byB0aGUKPiA+IGRvY3Vt ZW50YXRpb246ICJXaGVuIHNvbWVib2R5IHRyaWVzIHRvIGludmFsaWRhdGUgdGhlIHBhZ2UgdGFi bGVzIHdlIGJsb2NrIHRoZQo+ID4gdXBkYXRlIHVudGlsIGFsbCBvcGVyYXRpb25zIG9uIHRoZSBw YWdlcyBpbiBxdWVzdGlvbiBhcmUgY29tcGxldGVkLCB0aGVuIHRob3NlCj4gPiBwYWdlcyBhcmUg bWFya2VkICBhcyBhY2Nlc3NlZCBhbmQgYWxzbyBkaXJ0eSBpZiBpdCB3YXNu4oCZdCBhIHJlYWQg b25seSBhY2Nlc3MuIgo+ID4gTG9va3MgbGlrZSB0aGUgZmVuY2VzIGFyZSBkZWFkbG9ja2VkIGFu ZCB0aGVyZWZvcmUsIGl0IG5ldmVyIHJldHVybnMuIENvdWxkIGl0Cj4gPiBiZSBwb3NzaWJsZT8g YW55IGhpbnQgdG8gd2hlcmUgY2FuIEkgbG9vayB0byBmaXggdGhpcz8KPiA+Cj4gPiBUaGFuayB5 b3UgIGluIGFkdmFuY2UuCj4gPgo+ID4gSGVyZSB0aGUgZnVsbCBkbWVzZyBvdXRwdXQ6Cj4gPgo+ ID4gWyAgNzM4LjMzNzcyNl0gSU5GTzogdGFzayBNYXRyaXhNdWx0aXBsaWM6NjUzIGJsb2NrZWQg Zm9yIG1vcmUgdGhhbiAxMjIgc2Vjb25kcy4KPiA+IFsgIDczOC4zNDQ5MzddICAgICAgIE5vdCB0 YWludGVkIDUuMTEuMC1yYzYtcXRlYy1zdGFuZGFyZCAjMgo+ID4gWyAgNzM4LjM1MDM4NF0gImVj aG8gMCA+IC9wcm9jL3N5cy9rZXJuZWwvaHVuZ190YXNrX3RpbWVvdXRfc2VjcyIKPiA+IGRpc2Fi bGVzIHRoaXMgbWVzc2FnZS4KPiA+IFsgIDczOC4zNTgyNDBdIHRhc2s6TWF0cml4TXVsdGlwbGlj IHN0YXRlOkQgc3RhY2s6ICAgIDAgcGlkOiAgNjUzCj4gPiBwcGlkOiAgICAgMSBmbGFnczoweDAw MDA0MDAwCj4gPiBbICA3MzguMzU4MjU0XSBDYWxsIFRyYWNlOgo+ID4gWyAgNzM4LjM1ODI2MV0g ID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDFlYi8weDIzMAo+ID4gWyAgNzM4LjM1ODI3Nl0g IF9fc2NoZWR1bGUrMHgzNzAvMHg5NjAKPiA+IFsgIDczOC4zNTgyOTFdICA/IGRtYV9mZW5jZV9k ZWZhdWx0X3dhaXQrMHgxMTcvMHgyMzAKPiA+IFsgIDczOC4zNTgyOTddICA/IGRtYV9mZW5jZV9k ZWZhdWx0X3dhaXQrMHgxZWIvMHgyMzAKPiA+IFsgIDczOC4zNTgzMDVdICBzY2hlZHVsZSsweDUx LzB4YzAKPiA+IFsgIDczOC4zNTgzMTJdICBzY2hlZHVsZV90aW1lb3V0KzB4Mjc1LzB4MzgwCj4g PiBbICA3MzguMzU4MzI0XSAgPyBkbWFfZmVuY2VfZGVmYXVsdF93YWl0KzB4MWViLzB4MjMwCj4g PiBbICA3MzguMzU4MzMyXSAgPyBtYXJrX2hlbGRfbG9ja3MrMHg0Zi8weDcwCj4gPiBbICA3Mzgu MzU4MzQxXSAgPyBkbWFfZmVuY2VfZGVmYXVsdF93YWl0KzB4MTE3LzB4MjMwCj4gPiBbICA3Mzgu MzU4MzQ3XSAgPyBsb2NrZGVwX2hhcmRpcnFzX29uX3ByZXBhcmUrMHhkNC8weDE4MAo+ID4gWyAg NzM4LjM1ODM1M10gID8gX3Jhd19zcGluX3VubG9ja19pcnFyZXN0b3JlKzB4MzkvMHg0MAo+ID4g WyAgNzM4LjM1ODM2Ml0gID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDExNy8weDIzMAo+ID4g WyAgNzM4LjM1ODM3MF0gID8gZG1hX2ZlbmNlX2RlZmF1bHRfd2FpdCsweDFlYi8weDIzMAo+ID4g WyAgNzM4LjM1ODM3NV0gIGRtYV9mZW5jZV9kZWZhdWx0X3dhaXQrMHgyMTQvMHgyMzAKPiA+IFsg IDczOC4zNTgzODRdICA/IGRtYV9mZW5jZV9yZWxlYXNlKzB4MWEwLzB4MWEwCj4gPiBbICA3Mzgu MzU4Mzk2XSAgZG1hX2ZlbmNlX3dhaXRfdGltZW91dCsweDEwNS8weDIwMAo+ID4gWyAgNzM4LjM1 ODQwNV0gIGRtYV9yZXN2X3dhaXRfdGltZW91dF9yY3UrMHgxYWEvMHg1ZTAKPiA+IFsgIDczOC4z NTg0MjFdICBhbWRncHVfbW5faW52YWxpZGF0ZV9nZngrMHg1NS8weGEwIFthbWRncHVdCj4gPiBb ICA3MzguMzU4Njg4XSAgX19tbXVfbm90aWZpZXJfcmVsZWFzZSsweDFiYi8weDIxMAo+ID4gWyAg NzM4LjM1ODcxMF0gIGV4aXRfbW1hcCsweDJmLzB4MWUwCj4gPiBbICA3MzguMzU4NzIzXSAgPyBm aW5kX2hlbGRfbG9jaysweDM0LzB4YTAKPiA+IFsgIDczOC4zNTg3NDZdICBtbXB1dCsweDM5LzB4 ZTAKPiA+IFsgIDczOC4zNTg3NTZdICBkb19leGl0KzB4NWMzLzB4YzAwCj4gPiBbICA3MzguMzU4 NzYzXSAgPyBmaW5kX2hlbGRfbG9jaysweDM0LzB4YTAKPiA+IFsgIDczOC4zNTg3ODBdICBkb19n cm91cF9leGl0KzB4NDcvMHhiMAo+ID4gWyAgNzM4LjM1ODc5MV0gIGdldF9zaWduYWwrMHgxNWIv MHhjNTAKPiA+IFsgIDczOC4zNTg4MDddICBhcmNoX2RvX3NpZ25hbF9vcl9yZXN0YXJ0KzB4YWYv MHg3MTAKPiA+IFsgIDczOC4zNTg4MTZdICA/IGxvY2tkZXBfaGFyZGlycXNfb25fcHJlcGFyZSsw eGQ0LzB4MTgwCj4gPiBbICA3MzguMzU4ODIyXSAgPyBfcmF3X3NwaW5fdW5sb2NrX2lycXJlc3Rv cmUrMHgzOS8weDQwCj4gPiBbICA3MzguMzU4ODMxXSAgPyBrdGltZV9nZXRfbW9ub19mYXN0X25z KzB4NTAvMHhhMAo+ID4gWyAgNzM4LjM1ODg0NF0gID8gYW1kZ3B1X2RybV9pb2N0bCsweDZiLzB4 ODAgW2FtZGdwdV0KPiA+IFsgIDczOC4zNTkwNDRdICBleGl0X3RvX3VzZXJfbW9kZV9wcmVwYXJl KzB4ZjIvMHgxYjAKPiA+IFsgIDczOC4zNTkwNTRdICBzeXNjYWxsX2V4aXRfdG9fdXNlcl9tb2Rl KzB4MTkvMHg2MAo+ID4gWyAgNzM4LjM1OTA2Ml0gIGVudHJ5X1NZU0NBTExfNjRfYWZ0ZXJfaHdm cmFtZSsweDQ0LzB4YTkKPiA+IFsgIDczOC4zNTkwNjldIFJJUDogMDAzMzoweDdmNmI4OWE1MTg4 Nwo+ID4gWyAgNzM4LjM1OTA3Nl0gUlNQOiAwMDJiOjAwMDA3ZjZiODJiNTRiMTggRUZMQUdTOiAw MDAwMDI0NiBPUklHX1JBWDoKPiA+IDAwMDAwMDAwMDAwMDAwMTAKPiA+IFsgIDczOC4zNTkwODZd IFJBWDogZmZmZmZmZmZmZmZmZmUwMCBSQlg6IDAwMDA3ZjZiODJiNTRiNTAgUkNYOiAwMDAwN2Y2 Yjg5YTUxODg3Cj4gPiBbICA3MzguMzU5MDkxXSBSRFg6IDAwMDA3ZjZiODJiNTRiNTAgUlNJOiAw MDAwMDAwMGMwMjA2NGMzIFJESTogMDAwMDAwMDAwMDAwMDAwNwo+ID4gWyAgNzM4LjM1OTA5Nl0g UkJQOiAwMDAwMDAwMGMwMjA2NGMzIFIwODogMDAwMDAwMDAwMDAwMDAwMyBSMDk6IDAwMDA3ZjZi ODJiNTRiYmMKPiA+IFsgIDczOC4zNTkxMDFdIFIxMDogMDAwMDAwMDAwMDAwMDAwMSBSMTE6IDAw MDAwMDAwMDAwMDAyNDYgUjEyOiAwMDAwMDAwMTY1YTBiYzAwCj4gPiBbICA3MzguMzU5MTA2XSBS MTM6IDAwMDAwMDAwMDAwMDAwMDcgUjE0OiAwMDAwMDAwMDAwMDAwMDAxIFIxNTogMDAwMDAwMDAw MDAwMDAwMAo+ID4gWyAgNzM4LjM1OTEyOV0KPiA+ICAgICAgICAgICAgICAgICBTaG93aW5nIGFs bCBsb2NrcyBoZWxkIGluIHRoZSBzeXN0ZW06Cj4gPiBbICA3MzguMzU5MTQxXSAxIGxvY2sgaGVs ZCBieSBraHVuZ3Rhc2tkLzU0Ogo+ID4gWyAgNzM4LjM1OTE0OF0gICMwOiBmZmZmZmZmZjgyOWY2 ODQwIChyY3VfcmVhZF9sb2NrKXsuLi4ufS17MToyfSwgYXQ6Cj4gPiBkZWJ1Z19zaG93X2FsbF9s b2NrcysweDE1LzB4MTgzCj4gPiBbICA3MzguMzU5MTg3XSAxIGxvY2sgaGVsZCBieSBzeXN0ZW1k LWpvdXJuYWwvMTc0Ogo+ID4gWyAgNzM4LjM1OTIwMl0gMSBsb2NrIGhlbGQgYnkgTWF0cml4TXVs dGlwbGljLzY1MzoKPiA+IFsgIDczOC4zNTkyMDZdICAjMDogZmZmZjg4ODEwZTM2NGZlMAo+ID4g KCZhZGV2LT5ub3RpZmllcl9sb2NrKXsrLisufS17MzozfSwgYXQ6Cj4gPiBhbWRncHVfbW5faW52 YWxpZGF0ZV9nZngrMHgzNC8weGEwIFthbWRncHVdCj4gPgo+ID4gRGFuaWVsCj4KX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KYW1kLWdmeCBtYWlsaW5nIGxp c3QKYW1kLWdmeEBsaXN0cy5mcmVlZGVza3RvcC5vcmcKaHR0cHM6Ly9saXN0cy5mcmVlZGVza3Rv cC5vcmcvbWFpbG1hbi9saXN0aW5mby9hbWQtZ2Z4Cg==