From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 488FEC43603 for ; Sat, 14 Dec 2019 02:14:31 +0000 (UTC) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 30A7821655 for ; Sat, 14 Dec 2019 02:14:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 30A7821655 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Received: from localhost ([::1]:55290 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ifwww-0006YW-AJ for qemu-devel@archiver.kernel.org; Fri, 13 Dec 2019 21:14:30 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:53887) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ifvOL-0004Sz-9Z for qemu-devel@nongnu.org; Fri, 13 Dec 2019 19:34:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ifvOJ-0003uJ-OD for qemu-devel@nongnu.org; Fri, 13 Dec 2019 19:34:41 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:53128) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ifvOJ-0003ty-Hy for qemu-devel@nongnu.org; Fri, 13 Dec 2019 19:34:39 -0500 Received: by mail-wm1-f65.google.com with SMTP id p9so625471wmc.2 for ; Fri, 13 Dec 2019 16:34:39 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=ZH/tB32VwDWBcIO+jyMZHCZ/gY+cgypzth0Gb94tIZ8=; b=c/UV8R3HVtiBvGoaI8PdSRuJB9D5GdSTBnLG70Cl5ThpRd8f5dm4jjfBarji6yWFfO yOH3PfwK3zCquTQl9okhl3P+52IBrJF89fZhc2Nn0jx0PFGbGo4Q1jpSrSkqUaaLHb0k 7yUF2dvRMELZg1/f05roVAJfXp/xnt5JrWkrmwHQoXw5Dko5pnvwzOC2ndPBJd8/4Vgr hIIIs3ZHZS1sYX1U9r5t19co5mqZF2d6yeXfaq+ERKaEp0GYBfTHrjnzVwHjenKvQjq3 aTdJd+jWMG0RqSUR2N42qwEEqZYTHgSYlOqKmbUlJi/y9N8KUIr/ogtLoauaU9o11ndc ukYw== X-Gm-Message-State: APjAAAXYrBW+GREU0LwXle1XUUMzjutYf/z6X9Sv5dty8Al6XgRfaXoD F4ZoZhgpcsx/qnHlAXZR/S4Ztx1kW7w= X-Google-Smtp-Source: APXvYqxoFm3nJNtpmA00EH9GTcs0FElV3akt62kTgHGRqlzZXagNWnXTXEYZ2wkS2cKce3EboBdh5g== X-Received: by 2002:a05:600c:24d1:: with SMTP id 17mr16533141wmu.151.1576283677969; Fri, 13 Dec 2019 16:34:37 -0800 (PST) Received: from a483e7b01a66.ant.amazon.com (54-240-197-230.amazon.com. [54.240.197.230]) by smtp.gmail.com with ESMTPSA id a184sm12388953wmf.29.2019.12.13.16.34.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Dec 2019 16:34:37 -0800 (PST) Subject: xen-block: race condition when stopping the device (WAS: Re: [Xen-devel] [xen-4.13-testing test] 144736: regressions - FAIL) To: "Durrant, Paul" , Ian Jackson References: <6ea2af3c-b277-1118-7c83-ebcb0809d449@xen.org> <24051.30893.31444.319978@mariner.uk.xensource.com> <7a0ef296-eb50-fbda-63e2-8d890fad5111@xen.org> From: Julien Grall Message-ID: <65df8a75-a658-1a14-6780-66c8706bcc80@xen.org> Date: Sat, 14 Dec 2019 00:34:36 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.85.128.65 X-Mailman-Approved-At: Fri, 13 Dec 2019 21:13:09 -0500 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= , Stefano Stabellini , qemu-devel@nongnu.org, osstest service owner , Anthony Perard , "xen-devel@lists.xenproject.org" Errors-To: qemu-devel-bounces+qemu-devel=archiver.kernel.org@nongnu.org Sender: "Qemu-devel" Hi Paul, On 13/12/2019 15:55, Durrant, Paul wrote: >> -----Original Message----- >> From: Xen-devel On Behalf Of >> Julien Grall >> Sent: 13 December 2019 15:37 >> To: Ian Jackson >> Cc: Jürgen Groß ; xen-devel@lists.xenproject.org; Stefano >> Stabellini ; osstest service owner > admin@xenproject.org>; Anthony Perard >> Subject: Re: [Xen-devel] [xen-4.13-testing test] 144736: regressions - >> FAIL >> >> +Anthony >> >> On 13/12/2019 11:40, Ian Jackson wrote: >>> Julien Grall writes ("Re: [Xen-devel] [xen-4.13-testing test] 144736: >> regressions - FAIL"): >>>> AMD Seattle boards (laxton*) are known to fail booting time to time >>>> because of PCI training issue. We have workaround for it (involving >>>> longer power cycle) but this is not 100% reliable. >>> >>> This wasn't a power cycle. It was a software-initiated reboot. It >>> does appear to hang in the firmware somewhere. Do we expect the pci >>> training issue to occur in this case ? >> >> The PCI training happens at every reset (including software). So I may >> have confused the workaround for firmware corruption with the PCI >> training. We definitely have a workfround for the former. >> >> For the latter, I can't remember if we did use a new firmware or just >> hope it does not happen often. >> >> I think we had a thread on infra@ about the workaround some times last >> year. Sadly this was sent on my Arm e-mail address and I didn't archive >> it before leaving :(. Can you have a look if you can find the thread? >> >>> >>>>>>   test-armhf-armhf-xl-vhd      18 leak-check/check         fail >> REGR. >>>>>> vs. 144673 >>>>> >>>>> That one is strange. A qemu process seems to have have died producing >>>>> a core file, but I couldn't find any log containing any other >> indication >>>>> of a crashed program. >>>> >>>> I haven't found anything interesting in the log. @Ian could you set up >>>> a repro for this? >>> >>> There is some heisenbug where qemu crashes with very low probability. >>> (I forget whether only on arm or on x86 too). This has been around >>> for a little while. I doubt this particular failure will be >>> reproducible. >> >> I can't remember such bug been reported on Arm before. Anyway, I managed >> to get the stack trace from gdb: >> >> Core was generated by `/usr/local/lib/xen/bin/qemu-system-i386 >> -xen-domid 1 -chardev socket,id=libxl-c'. >> Program terminated with signal SIGSEGV, Segmentation fault. >> #0 0x006342be in xen_block_handle_requests (dataplane=0x108e600) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen- >> dir/hw/block/dataplane/xen-block.c:531 >> 531 >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen- >> dir/hw/block/dataplane/xen-block.c: >> No such file or directory. >> [Current thread is 1 (LWP 1987)] >> (gdb) bt >> #0 0x006342be in xen_block_handle_requests (dataplane=0x108e600) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen- >> dir/hw/block/dataplane/xen-block.c:531 >> #1 0x0063447c in xen_block_dataplane_event (opaque=0x108e600) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen- >> dir/hw/block/dataplane/xen-block.c:626 >> #2 0x008d005c in xen_device_poll (opaque=0x107a3b0) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen-dir/hw/xen/xen- >> bus.c:1077 >> #3 0x00a4175c in run_poll_handlers_once (ctx=0x1079708, >> timeout=0xb1ba17f8) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen-dir/util/aio- >> posix.c:520 >> #4 0x00a41826 in run_poll_handlers (ctx=0x1079708, max_ns=8000, >> timeout=0xb1ba17f8) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen-dir/util/aio- >> posix.c:562 >> #5 0x00a41956 in try_poll_mode (ctx=0x1079708, timeout=0xb1ba17f8) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen-dir/util/aio- >> posix.c:597 >> #6 0x00a41a2c in aio_poll (ctx=0x1079708, blocking=true) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen-dir/util/aio- >> posix.c:639 >> #7 0x0071dc16 in iothread_run (opaque=0x107d328) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen- >> dir/iothread.c:75 >> #8 0x00a44c80 in qemu_thread_start (args=0x1079538) at >> /home/osstest/build.144736.build-armhf/xen/tools/qemu-xen-dir/util/qemu- >> thread-posix.c:502 >> #9 0xb67ae5d8 in ?? () >> Backtrace stopped: previous frame identical to this frame (corrupt stack?) >> >> This feels like a race condition between the init/free code with >> handler. Anthony, does it ring any bell? >> > > From that stack bt it looks like an iothread managed to run after the sring was NULLed. This should not be able happen as the dataplane should have been moved back onto QEMU's main thread context before the ring is unmapped. My knowledge of this code is fairly limited, so correct me if I am wrong. blk_set_aio_context() would set the context for the block aio. AFAICT, the only aio for the block is xen_block_complete_aio(). In the stack above, we are not dealing with a block aio but an aio tie to the event channel (see the call from xen_device_poll). So I don't think the blk_set_aio_context() would affect the aio. So it would be possible to get the iothread running because we received a notification on the event channel while we are stopping the block (i.e xen_block_dataplane_stop()). If xen_block_dataplane_stop() grab the context lock first, then the iothread dealing with the event may wait on the lock until its released. By the time the lock is grabbed, we may have free all the resources (including srings). So the event iothread will end up to dereference a NULL pointer. It feels to me we need a way to quiesce all the iothreads (blk, event,...) before continuing. But I am a bit unsure how to do this in QEMU. Cheers, -- Julien Grall From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.2 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 21BAFC43603 for ; Sat, 14 Dec 2019 00:35:13 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0F4A1214AF for ; Sat, 14 Dec 2019 00:35:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F4A1214AF Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xen.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ifvOS-0004M6-Fw; Sat, 14 Dec 2019 00:34:48 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ifvOR-0004M1-MX for xen-devel@lists.xenproject.org; Sat, 14 Dec 2019 00:34:47 +0000 X-Inumbo-ID: 82a0c50e-1e09-11ea-88e7-bc764e2007e4 Received: from mail-wm1-f68.google.com (unknown [209.85.128.68]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 82a0c50e-1e09-11ea-88e7-bc764e2007e4; Sat, 14 Dec 2019 00:34:38 +0000 (UTC) Received: by mail-wm1-f68.google.com with SMTP id f4so531721wmj.1 for ; Fri, 13 Dec 2019 16:34:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=ZH/tB32VwDWBcIO+jyMZHCZ/gY+cgypzth0Gb94tIZ8=; b=MrUrX/CRpdGEU5rK6kKF3rTYJJfQEz7l9wr0Il2hy1L/iU8oEe0/pO2oP1KJFNo/Jb X76Q+p9TgQ+Z4dVX1XdpuPLnJpsMPmwGI06/+/NFxnetZ7I74+QvetnhpOaSlWAdOneJ OwsQB0qnV9lyLzN0EZY+X2yLgTRLfMUw2Qi0dGqXqG0II1pojgaZjZSa7uViJeZOyU2l 1o8jXRhXVL/XoOJILHlBNtRL9SCKyO6MPbMNU2wUMlswjfe/IJ8m9tD+LbSzCtiuhCKX JeWFAMMAq3/W9NeKv0COTKmkFvJbHKltF4Iekj0/HOkC292Kbs5hJ60/3HizaH+LgYMm mu9w== X-Gm-Message-State: APjAAAVHgrMbJ4GlRi57X5nBbWFG7tDXDp5+11EKcSJjAPwTyDF7bp4F uhp5qVS6oTIomCNsK2PODNQ= X-Google-Smtp-Source: APXvYqxoFm3nJNtpmA00EH9GTcs0FElV3akt62kTgHGRqlzZXagNWnXTXEYZ2wkS2cKce3EboBdh5g== X-Received: by 2002:a05:600c:24d1:: with SMTP id 17mr16533141wmu.151.1576283677969; Fri, 13 Dec 2019 16:34:37 -0800 (PST) Received: from a483e7b01a66.ant.amazon.com (54-240-197-230.amazon.com. [54.240.197.230]) by smtp.gmail.com with ESMTPSA id a184sm12388953wmf.29.2019.12.13.16.34.36 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 13 Dec 2019 16:34:37 -0800 (PST) To: "Durrant, Paul" , Ian Jackson References: <6ea2af3c-b277-1118-7c83-ebcb0809d449@xen.org> <24051.30893.31444.319978@mariner.uk.xensource.com> <7a0ef296-eb50-fbda-63e2-8d890fad5111@xen.org> From: Julien Grall Message-ID: <65df8a75-a658-1a14-6780-66c8706bcc80@xen.org> Date: Sat, 14 Dec 2019 00:34:36 +0000 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:68.0) Gecko/20100101 Thunderbird/68.3.0 MIME-Version: 1.0 In-Reply-To: Content-Language: en-GB Subject: [Xen-devel] xen-block: race condition when stopping the device (WAS: Re: [xen-4.13-testing test] 144736: regressions - FAIL) X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: =?UTF-8?B?SsO8cmdlbiBHcm/Dnw==?= , Stefano Stabellini , qemu-devel@nongnu.org, osstest service owner , Anthony Perard , "xen-devel@lists.xenproject.org" Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" SGkgUGF1bCwKCk9uIDEzLzEyLzIwMTkgMTU6NTUsIER1cnJhbnQsIFBhdWwgd3JvdGU6Cj4+IC0t LS0tT3JpZ2luYWwgTWVzc2FnZS0tLS0tCj4+IEZyb206IFhlbi1kZXZlbCA8eGVuLWRldmVsLWJv dW5jZXNAbGlzdHMueGVucHJvamVjdC5vcmc+IE9uIEJlaGFsZiBPZgo+PiBKdWxpZW4gR3JhbGwK Pj4gU2VudDogMTMgRGVjZW1iZXIgMjAxOSAxNTozNwo+PiBUbzogSWFuIEphY2tzb24gPGlhbi5q YWNrc29uQGNpdHJpeC5jb20+Cj4+IENjOiBKw7xyZ2VuIEdyb8OfIDxqZ3Jvc3NAc3VzZS5jb20+ OyB4ZW4tZGV2ZWxAbGlzdHMueGVucHJvamVjdC5vcmc7IFN0ZWZhbm8KPj4gU3RhYmVsbGluaSA8 c3N0YWJlbGxpbmlAa2VybmVsLm9yZz47IG9zc3Rlc3Qgc2VydmljZSBvd25lciA8b3NzdGVzdC0K Pj4gYWRtaW5AeGVucHJvamVjdC5vcmc+OyBBbnRob255IFBlcmFyZCA8YW50aG9ueS5wZXJhcmRA Y2l0cml4LmNvbT4KPj4gU3ViamVjdDogUmU6IFtYZW4tZGV2ZWxdIFt4ZW4tNC4xMy10ZXN0aW5n IHRlc3RdIDE0NDczNjogcmVncmVzc2lvbnMgLQo+PiBGQUlMCj4+Cj4+ICtBbnRob255Cj4+Cj4+ IE9uIDEzLzEyLzIwMTkgMTE6NDAsIElhbiBKYWNrc29uIHdyb3RlOgo+Pj4gSnVsaWVuIEdyYWxs IHdyaXRlcyAoIlJlOiBbWGVuLWRldmVsXSBbeGVuLTQuMTMtdGVzdGluZyB0ZXN0XSAxNDQ3MzY6 Cj4+IHJlZ3Jlc3Npb25zIC0gRkFJTCIpOgo+Pj4+IEFNRCBTZWF0dGxlIGJvYXJkcyAobGF4dG9u KikgYXJlIGtub3duIHRvIGZhaWwgYm9vdGluZyB0aW1lIHRvIHRpbWUKPj4+PiBiZWNhdXNlIG9m IFBDSSB0cmFpbmluZyBpc3N1ZS4gV2UgaGF2ZSB3b3JrYXJvdW5kIGZvciBpdCAoaW52b2x2aW5n Cj4+Pj4gbG9uZ2VyIHBvd2VyIGN5Y2xlKSBidXQgdGhpcyBpcyBub3QgMTAwJSByZWxpYWJsZS4K Pj4+Cj4+PiBUaGlzIHdhc24ndCBhIHBvd2VyIGN5Y2xlLiAgSXQgd2FzIGEgc29mdHdhcmUtaW5p dGlhdGVkIHJlYm9vdC4gIEl0Cj4+PiBkb2VzIGFwcGVhciB0byBoYW5nIGluIHRoZSBmaXJtd2Fy ZSBzb21ld2hlcmUuICBEbyB3ZSBleHBlY3QgdGhlIHBjaQo+Pj4gdHJhaW5pbmcgaXNzdWUgdG8g b2NjdXIgaW4gdGhpcyBjYXNlID8KPj4KPj4gVGhlIFBDSSB0cmFpbmluZyBoYXBwZW5zIGF0IGV2 ZXJ5IHJlc2V0IChpbmNsdWRpbmcgc29mdHdhcmUpLiBTbyBJIG1heQo+PiBoYXZlIGNvbmZ1c2Vk IHRoZSB3b3JrYXJvdW5kIGZvciBmaXJtd2FyZSBjb3JydXB0aW9uIHdpdGggdGhlIFBDSQo+PiB0 cmFpbmluZy4gV2UgZGVmaW5pdGVseSBoYXZlIGEgd29ya2Zyb3VuZCBmb3IgdGhlIGZvcm1lci4K Pj4KPj4gRm9yIHRoZSBsYXR0ZXIsIEkgY2FuJ3QgcmVtZW1iZXIgaWYgd2UgZGlkIHVzZSBhIG5l dyBmaXJtd2FyZSBvciBqdXN0Cj4+IGhvcGUgaXQgZG9lcyBub3QgaGFwcGVuIG9mdGVuLgo+Pgo+ PiBJIHRoaW5rIHdlIGhhZCBhIHRocmVhZCBvbiBpbmZyYUAgYWJvdXQgdGhlIHdvcmthcm91bmQg c29tZSB0aW1lcyBsYXN0Cj4+IHllYXIuIFNhZGx5IHRoaXMgd2FzIHNlbnQgb24gbXkgQXJtIGUt bWFpbCBhZGRyZXNzIGFuZCBJIGRpZG4ndCBhcmNoaXZlCj4+IGl0IGJlZm9yZSBsZWF2aW5nIDoo LiBDYW4geW91IGhhdmUgYSBsb29rIGlmIHlvdSBjYW4gZmluZCB0aGUgdGhyZWFkPwo+Pgo+Pj4K Pj4+Pj4+ICAgwqAgdGVzdC1hcm1oZi1hcm1oZi14bC12aGTCoMKgwqDCoMKgIDE4IGxlYWstY2hl Y2svY2hlY2vCoMKgwqDCoMKgwqDCoMKgIGZhaWwKPj4gUkVHUi4KPj4+Pj4+IHZzLiAxNDQ2NzMK Pj4+Pj4KPj4+Pj4gVGhhdCBvbmUgaXMgc3RyYW5nZS4gQSBxZW11IHByb2Nlc3Mgc2VlbXMgdG8g aGF2ZSBoYXZlIGRpZWQgcHJvZHVjaW5nCj4+Pj4+IGEgY29yZSBmaWxlLCBidXQgSSBjb3VsZG4n dCBmaW5kIGFueSBsb2cgY29udGFpbmluZyBhbnkgb3RoZXIKPj4gaW5kaWNhdGlvbgo+Pj4+PiBv ZiBhIGNyYXNoZWQgcHJvZ3JhbS4KPj4+Pgo+Pj4+IEkgaGF2ZW4ndCBmb3VuZCBhbnl0aGluZyBp bnRlcmVzdGluZyBpbiB0aGUgbG9nLiBASWFuIGNvdWxkIHlvdSBzZXQgdXAKPj4+PiBhIHJlcHJv IGZvciB0aGlzPwo+Pj4KPj4+IFRoZXJlIGlzIHNvbWUgaGVpc2VuYnVnIHdoZXJlIHFlbXUgY3Jh c2hlcyB3aXRoIHZlcnkgbG93IHByb2JhYmlsaXR5Lgo+Pj4gKEkgZm9yZ2V0IHdoZXRoZXIgb25s eSBvbiBhcm0gb3Igb24geDg2IHRvbykuICBUaGlzIGhhcyBiZWVuIGFyb3VuZAo+Pj4gZm9yIGEg bGl0dGxlIHdoaWxlLiAgSSBkb3VidCB0aGlzIHBhcnRpY3VsYXIgZmFpbHVyZSB3aWxsIGJlCj4+ PiByZXByb2R1Y2libGUuCj4+Cj4+IEkgY2FuJ3QgcmVtZW1iZXIgc3VjaCBidWcgYmVlbiByZXBv cnRlZCBvbiBBcm0gYmVmb3JlLiBBbnl3YXksIEkgbWFuYWdlZAo+PiB0byBnZXQgdGhlIHN0YWNr IHRyYWNlIGZyb20gZ2RiOgo+Pgo+PiBDb3JlIHdhcyBnZW5lcmF0ZWQgYnkgYC91c3IvbG9jYWwv bGliL3hlbi9iaW4vcWVtdS1zeXN0ZW0taTM4Ngo+PiAteGVuLWRvbWlkIDEgLWNoYXJkZXYgc29j a2V0LGlkPWxpYnhsLWMnLgo+PiBQcm9ncmFtIHRlcm1pbmF0ZWQgd2l0aCBzaWduYWwgU0lHU0VH ViwgU2VnbWVudGF0aW9uIGZhdWx0Lgo+PiAjMCAgMHgwMDYzNDJiZSBpbiB4ZW5fYmxvY2tfaGFu ZGxlX3JlcXVlc3RzIChkYXRhcGxhbmU9MHgxMDhlNjAwKSBhdAo+PiAvaG9tZS9vc3N0ZXN0L2J1 aWxkLjE0NDczNi5idWlsZC1hcm1oZi94ZW4vdG9vbHMvcWVtdS14ZW4tCj4+IGRpci9ody9ibG9j ay9kYXRhcGxhbmUveGVuLWJsb2NrLmM6NTMxCj4+IDUzMQo+PiAvaG9tZS9vc3N0ZXN0L2J1aWxk LjE0NDczNi5idWlsZC1hcm1oZi94ZW4vdG9vbHMvcWVtdS14ZW4tCj4+IGRpci9ody9ibG9jay9k YXRhcGxhbmUveGVuLWJsb2NrLmM6Cj4+IE5vIHN1Y2ggZmlsZSBvciBkaXJlY3RvcnkuCj4+IFtD dXJyZW50IHRocmVhZCBpcyAxIChMV1AgMTk4NyldCj4+IChnZGIpIGJ0Cj4+ICMwICAweDAwNjM0 MmJlIGluIHhlbl9ibG9ja19oYW5kbGVfcmVxdWVzdHMgKGRhdGFwbGFuZT0weDEwOGU2MDApIGF0 Cj4+IC9ob21lL29zc3Rlc3QvYnVpbGQuMTQ0NzM2LmJ1aWxkLWFybWhmL3hlbi90b29scy9xZW11 LXhlbi0KPj4gZGlyL2h3L2Jsb2NrL2RhdGFwbGFuZS94ZW4tYmxvY2suYzo1MzEKPj4gIzEgIDB4 MDA2MzQ0N2MgaW4geGVuX2Jsb2NrX2RhdGFwbGFuZV9ldmVudCAob3BhcXVlPTB4MTA4ZTYwMCkg YXQKPj4gL2hvbWUvb3NzdGVzdC9idWlsZC4xNDQ3MzYuYnVpbGQtYXJtaGYveGVuL3Rvb2xzL3Fl bXUteGVuLQo+PiBkaXIvaHcvYmxvY2svZGF0YXBsYW5lL3hlbi1ibG9jay5jOjYyNgo+PiAjMiAg MHgwMDhkMDA1YyBpbiB4ZW5fZGV2aWNlX3BvbGwgKG9wYXF1ZT0weDEwN2EzYjApIGF0Cj4+IC9o b21lL29zc3Rlc3QvYnVpbGQuMTQ0NzM2LmJ1aWxkLWFybWhmL3hlbi90b29scy9xZW11LXhlbi1k aXIvaHcveGVuL3hlbi0KPj4gYnVzLmM6MTA3Nwo+PiAjMyAgMHgwMGE0MTc1YyBpbiBydW5fcG9s bF9oYW5kbGVyc19vbmNlIChjdHg9MHgxMDc5NzA4LAo+PiB0aW1lb3V0PTB4YjFiYTE3ZjgpIGF0 Cj4+IC9ob21lL29zc3Rlc3QvYnVpbGQuMTQ0NzM2LmJ1aWxkLWFybWhmL3hlbi90b29scy9xZW11 LXhlbi1kaXIvdXRpbC9haW8tCj4+IHBvc2l4LmM6NTIwCj4+ICM0ICAweDAwYTQxODI2IGluIHJ1 bl9wb2xsX2hhbmRsZXJzIChjdHg9MHgxMDc5NzA4LCBtYXhfbnM9ODAwMCwKPj4gdGltZW91dD0w eGIxYmExN2Y4KSBhdAo+PiAvaG9tZS9vc3N0ZXN0L2J1aWxkLjE0NDczNi5idWlsZC1hcm1oZi94 ZW4vdG9vbHMvcWVtdS14ZW4tZGlyL3V0aWwvYWlvLQo+PiBwb3NpeC5jOjU2Mgo+PiAjNSAgMHgw MGE0MTk1NiBpbiB0cnlfcG9sbF9tb2RlIChjdHg9MHgxMDc5NzA4LCB0aW1lb3V0PTB4YjFiYTE3 ZjgpIGF0Cj4+IC9ob21lL29zc3Rlc3QvYnVpbGQuMTQ0NzM2LmJ1aWxkLWFybWhmL3hlbi90b29s cy9xZW11LXhlbi1kaXIvdXRpbC9haW8tCj4+IHBvc2l4LmM6NTk3Cj4+ICM2ICAweDAwYTQxYTJj IGluIGFpb19wb2xsIChjdHg9MHgxMDc5NzA4LCBibG9ja2luZz10cnVlKSBhdAo+PiAvaG9tZS9v c3N0ZXN0L2J1aWxkLjE0NDczNi5idWlsZC1hcm1oZi94ZW4vdG9vbHMvcWVtdS14ZW4tZGlyL3V0 aWwvYWlvLQo+PiBwb3NpeC5jOjYzOQo+PiAjNyAgMHgwMDcxZGMxNiBpbiBpb3RocmVhZF9ydW4g KG9wYXF1ZT0weDEwN2QzMjgpIGF0Cj4+IC9ob21lL29zc3Rlc3QvYnVpbGQuMTQ0NzM2LmJ1aWxk LWFybWhmL3hlbi90b29scy9xZW11LXhlbi0KPj4gZGlyL2lvdGhyZWFkLmM6NzUKPj4gIzggIDB4 MDBhNDRjODAgaW4gcWVtdV90aHJlYWRfc3RhcnQgKGFyZ3M9MHgxMDc5NTM4KSBhdAo+PiAvaG9t ZS9vc3N0ZXN0L2J1aWxkLjE0NDczNi5idWlsZC1hcm1oZi94ZW4vdG9vbHMvcWVtdS14ZW4tZGly L3V0aWwvcWVtdS0KPj4gdGhyZWFkLXBvc2l4LmM6NTAyCj4+ICM5ICAweGI2N2FlNWQ4IGluID8/ ICgpCj4+IEJhY2t0cmFjZSBzdG9wcGVkOiBwcmV2aW91cyBmcmFtZSBpZGVudGljYWwgdG8gdGhp cyBmcmFtZSAoY29ycnVwdCBzdGFjaz8pCj4+Cj4+IFRoaXMgZmVlbHMgbGlrZSBhIHJhY2UgY29u ZGl0aW9uIGJldHdlZW4gdGhlIGluaXQvZnJlZSBjb2RlIHdpdGgKPj4gaGFuZGxlci4gQW50aG9u eSwgZG9lcyBpdCByaW5nIGFueSBiZWxsPwo+Pgo+IAo+ICBGcm9tIHRoYXQgc3RhY2sgYnQgaXQg bG9va3MgbGlrZSBhbiBpb3RocmVhZCBtYW5hZ2VkIHRvIHJ1biBhZnRlciB0aGUgc3Jpbmcgd2Fz IE5VTExlZC4gVGhpcyBzaG91bGQgbm90IGJlIGFibGUgaGFwcGVuIGFzIHRoZSBkYXRhcGxhbmUg c2hvdWxkIGhhdmUgYmVlbiBtb3ZlZCBiYWNrIG9udG8gUUVNVSdzIG1haW4gdGhyZWFkIGNvbnRl eHQgYmVmb3JlIHRoZSByaW5nIGlzIHVubWFwcGVkLgoKTXkga25vd2xlZGdlIG9mIHRoaXMgY29k ZSBpcyBmYWlybHkgbGltaXRlZCwgc28gY29ycmVjdCBtZSBpZiBJIGFtIHdyb25nLgoKYmxrX3Nl dF9haW9fY29udGV4dCgpIHdvdWxkIHNldCB0aGUgY29udGV4dCBmb3IgdGhlIGJsb2NrIGFpby4g QUZBSUNULCAKdGhlIG9ubHkgYWlvIGZvciB0aGUgYmxvY2sgaXMgeGVuX2Jsb2NrX2NvbXBsZXRl X2FpbygpLgoKSW4gdGhlIHN0YWNrIGFib3ZlLCB3ZSBhcmUgbm90IGRlYWxpbmcgd2l0aCBhIGJs b2NrIGFpbyBidXQgYW4gYWlvIHRpZSAKdG8gdGhlIGV2ZW50IGNoYW5uZWwgKHNlZSB0aGUgY2Fs bCBmcm9tIHhlbl9kZXZpY2VfcG9sbCkuIFNvIEkgZG9uJ3QgCnRoaW5rIHRoZSBibGtfc2V0X2Fp b19jb250ZXh0KCkgd291bGQgYWZmZWN0IHRoZSBhaW8uCgpTbyBpdCB3b3VsZCBiZSBwb3NzaWJs ZSB0byBnZXQgdGhlIGlvdGhyZWFkIHJ1bm5pbmcgYmVjYXVzZSB3ZSByZWNlaXZlZCAKYSBub3Rp ZmljYXRpb24gb24gdGhlIGV2ZW50IGNoYW5uZWwgd2hpbGUgd2UgYXJlIHN0b3BwaW5nIHRoZSBi bG9jayAoaS5lIAp4ZW5fYmxvY2tfZGF0YXBsYW5lX3N0b3AoKSkuCgpJZiB4ZW5fYmxvY2tfZGF0 YXBsYW5lX3N0b3AoKSBncmFiIHRoZSBjb250ZXh0IGxvY2sgZmlyc3QsIHRoZW4gdGhlIAppb3Ro cmVhZCBkZWFsaW5nIHdpdGggdGhlIGV2ZW50IG1heSB3YWl0IG9uIHRoZSBsb2NrIHVudGlsIGl0 cyByZWxlYXNlZC4KCkJ5IHRoZSB0aW1lIHRoZSBsb2NrIGlzIGdyYWJiZWQsIHdlIG1heSBoYXZl IGZyZWUgYWxsIHRoZSByZXNvdXJjZXMgCihpbmNsdWRpbmcgc3JpbmdzKS4gU28gdGhlIGV2ZW50 IGlvdGhyZWFkIHdpbGwgZW5kIHVwIHRvIGRlcmVmZXJlbmNlIGEgCk5VTEwgcG9pbnRlci4KCkl0 IGZlZWxzIHRvIG1lIHdlIG5lZWQgYSB3YXkgdG8gcXVpZXNjZSBhbGwgdGhlIGlvdGhyZWFkcyAo YmxrLCAKZXZlbnQsLi4uKSBiZWZvcmUgY29udGludWluZy4gQnV0IEkgYW0gYSBiaXQgdW5zdXJl IGhvdyB0byBkbyB0aGlzIGluIFFFTVUuCgpDaGVlcnMsCgotLSAKSnVsaWVuIEdyYWxsCgpfX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpYZW4tZGV2ZWwgbWFp bGluZyBsaXN0Clhlbi1kZXZlbEBsaXN0cy54ZW5wcm9qZWN0Lm9yZwpodHRwczovL2xpc3RzLnhl bnByb2plY3Qub3JnL21haWxtYW4vbGlzdGluZm8veGVuLWRldmVs