From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83842C433E6 for ; Tue, 5 Jan 2021 22:42:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3759B230F9 for ; Tue, 5 Jan 2021 22:42:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730882AbhAEWmG (ORCPT ); Tue, 5 Jan 2021 17:42:06 -0500 Received: from mail.kernel.org ([198.145.29.99]:50868 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725838AbhAEWmG (ORCPT ); Tue, 5 Jan 2021 17:42:06 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id B64C422D6E; Tue, 5 Jan 2021 22:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1609886485; bh=Hd0WCK5SrMVNFPBh9xnoTwtX6vl1dAnkrgvh+w7m70g=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=nEGBJJsu9wtcnOowraQpQglsTKmrBSp9f3SOj7ZqNb60xHmYkXWumhwpw4iZyFXrY Q2qgJsYv0Uc4U1FS1g+rl2+EyHUsnieBTu9iDunwMfTa7r6xIEQg8z+JA8rtbmc53I APYe/v66W4ITmKuq5bLpQJLqW8NGMHHGMCT3bgOKj5burfUi4IFUSjy8LI/JhC+iHX KdQUa/KGMwpB7qG4JDZPNcihKwEBNzm6aG9KvPH6AFwCE3hg/jXcodrUOZZ6QaZigD mGef+R7mmqmwscyztviJVtPN2ZWeAF0YZx2hBcUPTxUnPqaljWb9WXTltRIFi9nR/3 OSwNnvpOP3N6A== Date: Tue, 5 Jan 2021 22:41:19 +0000 From: Will Deacon To: Andy Lutomirski Cc: Andy Lutomirski , Nicholas Piggin , Mathieu Desnoyers , X86 ML , Arnd Bergmann , Benjamin Herrenschmidt , Catalin Marinas , linux-arm-kernel , LKML , linuxppc-dev , Michael Ellerman , Paul Mackerras , stable Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() Message-ID: <20210105224119.GA13005@willie-the-truck> References: <20210105132623.GB11108@willie-the-truck> <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> User-Agent: Mutt/1.10.1 (2018-07-13) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jan 05, 2021 at 08:20:51AM -0800, Andy Lutomirski wrote: > > On Jan 5, 2021, at 5:26 AM, Will Deacon wrote: > > Sorry for the slow reply, I was socially distanced from my keyboard. > > > >> On Mon, Dec 28, 2020 at 04:36:11PM -0800, Andy Lutomirski wrote: > >> On Mon, Dec 28, 2020 at 4:11 PM Nicholas Piggin wrote: > >>>> +static inline void membarrier_sync_core_before_usermode(void) > >>>> +{ > >>>> + /* > >>>> + * XXX: I know basically nothing about powerpc cache management. > >>>> + * Is this correct? > >>>> + */ > >>>> + isync(); > >>> > >>> This is not about memory ordering or cache management, it's about > >>> pipeline management. Powerpc's return to user mode serializes the > >>> CPU (aka the hardware thread, _not_ the core; another wrongness of > >>> the name, but AFAIKS the HW thread is what is required for > >>> membarrier). So this is wrong, powerpc needs nothing here. > >> > >> Fair enough. I'm happy to defer to you on the powerpc details. In > >> any case, this just illustrates that we need feedback from a person > >> who knows more about ARM64 than I do. > > > > I think we're in a very similar boat to PowerPC, fwiw. Roughly speaking: > > > > 1. SYNC_CORE does _not_ perform any cache management; that is the > > responsibility of userspace, either by executing the relevant > > maintenance instructions (arm64) or a system call (arm32). Crucially, > > the hardware will ensure that this cache maintenance is broadcast > > to all other CPUs. > > Is this guaranteed regardless of any aliases? That is, if I flush from > one CPU at one VA and then execute the same physical address from another > CPU at a different VA, does this still work? The data side will be fine, but the instruction side can have virtual aliases. We handle this in flush_ptrace_access() by blowing away the whole I-cache if we're not physically-indexed, but userspace would be in trouble if it wanted to handle this situation alone. > > 2. Even with all the cache maintenance in the world, a CPU could have > > speculatively fetched stale instructions into its "pipeline" ahead of > > time, and these are _not_ flushed by the broadcast maintenance instructions > > in (1). SYNC_CORE provides a means for userspace to discard these stale > > instructions. > > > > 3. The context synchronization event on exception entry/exit is > > sufficient here. The Arm ARM isn't very good at describing what it > > does, because it's in denial about the existence of a pipeline, but > > it does have snippets such as: > > > > (s/PE/CPU/) > > | For all types of memory: > > | The PE might have fetched the instructions from memory at any time > > | since the last Context synchronization event on that PE. > > > > Interestingly, the architecture recently added a control bit to remove > > this synchronisation from exception return, so if we set that then we'd > > have a problem with SYNC_CORE and adding an ISB would be necessary (and > > we could probable then make kernel->kernel returns cheaper, but I > > suspect we're relying on this implicit synchronisation in other places > > too). > > > > Is ISB just a context synchronization event or does it do more? That's a good question. Barrier instructions on ARM do tend to get overloaded with extra behaviours over time, so it could certainly end up doing the context synchronization event + extra stuff in future. Right now, the only thing that springs to mind is the spectre-v1 heavy mitigation barrier of 'DSB; ISB' which, for example, probably doesn't work for 'DSB; ERET' because the ERET can be treated like a conditional (!) branch. > On x86, it’s very hard to tell that MFENCE does any more than LOCK, but > it’s much slower. And we have LFENCE, which, as documented, doesn’t > appear to have any semantics at all. (Or at least it didn’t before > Spectre.) I tend to think of ISB as a front-end barrier relating to instruction fetch whereas DMB, acquire/release and DSB are all back-end barriers relating to memory accesses. You _can_ use ISB in conjunction with control dependencies to order a pair of loads (like you can with ISYNC on Power), but it's a really expensive way to do it. > > Are you seeing a problem in practice, or did this come up while trying to > > decipher the semantics of SYNC_CORE? > > It came up while trying to understand the code and work through various > bugs in it. The code was written using something approximating x86 > terminology, but it was definitely wrong on x86 (at least if you believe > the SDM, and I haven’t convinced any architects to say otherwise). Ok, thanks. Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.3 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C12D1C433E0 for ; Tue, 5 Jan 2021 22:43:25 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1053A22E00 for ; Tue, 5 Jan 2021 22:43:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1053A22E00 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4D9SFk5PjWzDqVm for ; Wed, 6 Jan 2021 09:43:22 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=198.145.29.99; helo=mail.kernel.org; envelope-from=will@kernel.org; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=nEGBJJsu; dkim-atps=neutral Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4D9SCX0jj8zDqLv for ; Wed, 6 Jan 2021 09:41:27 +1100 (AEDT) Received: by mail.kernel.org (Postfix) with ESMTPSA id B64C422D6E; Tue, 5 Jan 2021 22:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1609886485; bh=Hd0WCK5SrMVNFPBh9xnoTwtX6vl1dAnkrgvh+w7m70g=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=nEGBJJsu9wtcnOowraQpQglsTKmrBSp9f3SOj7ZqNb60xHmYkXWumhwpw4iZyFXrY Q2qgJsYv0Uc4U1FS1g+rl2+EyHUsnieBTu9iDunwMfTa7r6xIEQg8z+JA8rtbmc53I APYe/v66W4ITmKuq5bLpQJLqW8NGMHHGMCT3bgOKj5burfUi4IFUSjy8LI/JhC+iHX KdQUa/KGMwpB7qG4JDZPNcihKwEBNzm6aG9KvPH6AFwCE3hg/jXcodrUOZZ6QaZigD mGef+R7mmqmwscyztviJVtPN2ZWeAF0YZx2hBcUPTxUnPqaljWb9WXTltRIFi9nR/3 OSwNnvpOP3N6A== Date: Tue, 5 Jan 2021 22:41:19 +0000 From: Will Deacon To: Andy Lutomirski Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() Message-ID: <20210105224119.GA13005@willie-the-truck> References: <20210105132623.GB11108@willie-the-truck> <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> User-Agent: Mutt/1.10.1 (2018-07-13) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arnd Bergmann , X86 ML , LKML , Nicholas Piggin , Mathieu Desnoyers , Andy Lutomirski , Catalin Marinas , Paul Mackerras , stable , linuxppc-dev , linux-arm-kernel Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Tue, Jan 05, 2021 at 08:20:51AM -0800, Andy Lutomirski wrote: > > On Jan 5, 2021, at 5:26 AM, Will Deacon wrote: > > Sorry for the slow reply, I was socially distanced from my keyboard. > > > >> On Mon, Dec 28, 2020 at 04:36:11PM -0800, Andy Lutomirski wrote: > >> On Mon, Dec 28, 2020 at 4:11 PM Nicholas Piggin wrote: > >>>> +static inline void membarrier_sync_core_before_usermode(void) > >>>> +{ > >>>> + /* > >>>> + * XXX: I know basically nothing about powerpc cache management. > >>>> + * Is this correct? > >>>> + */ > >>>> + isync(); > >>> > >>> This is not about memory ordering or cache management, it's about > >>> pipeline management. Powerpc's return to user mode serializes the > >>> CPU (aka the hardware thread, _not_ the core; another wrongness of > >>> the name, but AFAIKS the HW thread is what is required for > >>> membarrier). So this is wrong, powerpc needs nothing here. > >> > >> Fair enough. I'm happy to defer to you on the powerpc details. In > >> any case, this just illustrates that we need feedback from a person > >> who knows more about ARM64 than I do. > > > > I think we're in a very similar boat to PowerPC, fwiw. Roughly speaking: > > > > 1. SYNC_CORE does _not_ perform any cache management; that is the > > responsibility of userspace, either by executing the relevant > > maintenance instructions (arm64) or a system call (arm32). Crucially, > > the hardware will ensure that this cache maintenance is broadcast > > to all other CPUs. > > Is this guaranteed regardless of any aliases? That is, if I flush from > one CPU at one VA and then execute the same physical address from another > CPU at a different VA, does this still work? The data side will be fine, but the instruction side can have virtual aliases. We handle this in flush_ptrace_access() by blowing away the whole I-cache if we're not physically-indexed, but userspace would be in trouble if it wanted to handle this situation alone. > > 2. Even with all the cache maintenance in the world, a CPU could have > > speculatively fetched stale instructions into its "pipeline" ahead of > > time, and these are _not_ flushed by the broadcast maintenance instructions > > in (1). SYNC_CORE provides a means for userspace to discard these stale > > instructions. > > > > 3. The context synchronization event on exception entry/exit is > > sufficient here. The Arm ARM isn't very good at describing what it > > does, because it's in denial about the existence of a pipeline, but > > it does have snippets such as: > > > > (s/PE/CPU/) > > | For all types of memory: > > | The PE might have fetched the instructions from memory at any time > > | since the last Context synchronization event on that PE. > > > > Interestingly, the architecture recently added a control bit to remove > > this synchronisation from exception return, so if we set that then we'd > > have a problem with SYNC_CORE and adding an ISB would be necessary (and > > we could probable then make kernel->kernel returns cheaper, but I > > suspect we're relying on this implicit synchronisation in other places > > too). > > > > Is ISB just a context synchronization event or does it do more? That's a good question. Barrier instructions on ARM do tend to get overloaded with extra behaviours over time, so it could certainly end up doing the context synchronization event + extra stuff in future. Right now, the only thing that springs to mind is the spectre-v1 heavy mitigation barrier of 'DSB; ISB' which, for example, probably doesn't work for 'DSB; ERET' because the ERET can be treated like a conditional (!) branch. > On x86, it’s very hard to tell that MFENCE does any more than LOCK, but > it’s much slower. And we have LFENCE, which, as documented, doesn’t > appear to have any semantics at all. (Or at least it didn’t before > Spectre.) I tend to think of ISB as a front-end barrier relating to instruction fetch whereas DMB, acquire/release and DSB are all back-end barriers relating to memory accesses. You _can_ use ISB in conjunction with control dependencies to order a pair of loads (like you can with ISYNC on Power), but it's a really expensive way to do it. > > Are you seeing a problem in practice, or did this come up while trying to > > decipher the semantics of SYNC_CORE? > > It came up while trying to understand the code and work through various > bugs in it. The code was written using something approximating x86 > terminology, but it was definitely wrong on x86 (at least if you believe > the SDM, and I haven’t convinced any architects to say otherwise). Ok, thanks. Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C823C433DB for ; Tue, 5 Jan 2021 22:43:25 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C7DC822D6E for ; Tue, 5 Jan 2021 22:43:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C7DC822D6E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=+9KCOp+VYG/kjtBAl1YKOFl/xJq2YaK/T4MAG0j3G8o=; b=AOB8+dQqEQkGfZs9hPbOHbV43 IZwFis4ixMGa4Yxh4RJ1MaWCNV0BF9UUP9su1Twy0t8bCh+hIEySeV8SBR/PoBOZEBw+1elL0ikbH qedG3q5Vcf/cxu7i/nwLfNXbGX82VFZIZ6HtCtmU1GS1q7DKhxGyHp+G0s5qDe9724geE5P7SEldt 3htynuD6lz4VxArr2j7ZMnkYhQtfZx4Apnx0Oqk2yV6sH4xCzEqbNmc7Pw4WU3gju/OywIV5Yo8OK FubnPYiE1dhr6XBsnm2qlnqORuxVaHtPfqRjuj/Eu1EuCK/AKbKM+xllE74BJNAhApNeSc1UAFVqx IY6BVzzUw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kwv18-0000vf-Os; Tue, 05 Jan 2021 22:41:30 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kwv15-0000uV-J3 for linux-arm-kernel@lists.infradead.org; Tue, 05 Jan 2021 22:41:28 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id B64C422D6E; Tue, 5 Jan 2021 22:41:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1609886485; bh=Hd0WCK5SrMVNFPBh9xnoTwtX6vl1dAnkrgvh+w7m70g=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=nEGBJJsu9wtcnOowraQpQglsTKmrBSp9f3SOj7ZqNb60xHmYkXWumhwpw4iZyFXrY Q2qgJsYv0Uc4U1FS1g+rl2+EyHUsnieBTu9iDunwMfTa7r6xIEQg8z+JA8rtbmc53I APYe/v66W4ITmKuq5bLpQJLqW8NGMHHGMCT3bgOKj5burfUi4IFUSjy8LI/JhC+iHX KdQUa/KGMwpB7qG4JDZPNcihKwEBNzm6aG9KvPH6AFwCE3hg/jXcodrUOZZ6QaZigD mGef+R7mmqmwscyztviJVtPN2ZWeAF0YZx2hBcUPTxUnPqaljWb9WXTltRIFi9nR/3 OSwNnvpOP3N6A== Date: Tue, 5 Jan 2021 22:41:19 +0000 From: Will Deacon To: Andy Lutomirski Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() Message-ID: <20210105224119.GA13005@willie-the-truck> References: <20210105132623.GB11108@willie-the-truck> <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210105_174127_824020_16B2D87A X-CRM114-Status: GOOD ( 38.11 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arnd Bergmann , Benjamin Herrenschmidt , X86 ML , LKML , Nicholas Piggin , Mathieu Desnoyers , Michael Ellerman , Andy Lutomirski , Catalin Marinas , Paul Mackerras , stable , linuxppc-dev , linux-arm-kernel Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gVHVlLCBKYW4gMDUsIDIwMjEgYXQgMDg6MjA6NTFBTSAtMDgwMCwgQW5keSBMdXRvbWlyc2tp IHdyb3RlOgo+ID4gT24gSmFuIDUsIDIwMjEsIGF0IDU6MjYgQU0sIFdpbGwgRGVhY29uIDx3aWxs QGtlcm5lbC5vcmc+IHdyb3RlOgo+ID4gU29ycnkgZm9yIHRoZSBzbG93IHJlcGx5LCBJIHdhcyBz b2NpYWxseSBkaXN0YW5jZWQgZnJvbSBteSBrZXlib2FyZC4KPiA+IAo+ID4+IE9uIE1vbiwgRGVj IDI4LCAyMDIwIGF0IDA0OjM2OjExUE0gLTA4MDAsIEFuZHkgTHV0b21pcnNraSB3cm90ZToKPiA+ PiBPbiBNb24sIERlYyAyOCwgMjAyMCBhdCA0OjExIFBNIE5pY2hvbGFzIFBpZ2dpbiA8bnBpZ2dp bkBnbWFpbC5jb20+IHdyb3RlOgo+ID4+Pj4gK3N0YXRpYyBpbmxpbmUgdm9pZCBtZW1iYXJyaWVy X3N5bmNfY29yZV9iZWZvcmVfdXNlcm1vZGUodm9pZCkKPiA+Pj4+ICt7Cj4gPj4+PiArICAgICAv Kgo+ID4+Pj4gKyAgICAgICogWFhYOiBJIGtub3cgYmFzaWNhbGx5IG5vdGhpbmcgYWJvdXQgcG93 ZXJwYyBjYWNoZSBtYW5hZ2VtZW50Lgo+ID4+Pj4gKyAgICAgICogSXMgdGhpcyBjb3JyZWN0Pwo+ ID4+Pj4gKyAgICAgICovCj4gPj4+PiArICAgICBpc3luYygpOwo+ID4+PiAKPiA+Pj4gVGhpcyBp cyBub3QgYWJvdXQgbWVtb3J5IG9yZGVyaW5nIG9yIGNhY2hlIG1hbmFnZW1lbnQsIGl0J3MgYWJv dXQKPiA+Pj4gcGlwZWxpbmUgbWFuYWdlbWVudC4gUG93ZXJwYydzIHJldHVybiB0byB1c2VyIG1v ZGUgc2VyaWFsaXplcyB0aGUKPiA+Pj4gQ1BVIChha2EgdGhlIGhhcmR3YXJlIHRocmVhZCwgX25v dF8gdGhlIGNvcmU7IGFub3RoZXIgd3JvbmduZXNzIG9mCj4gPj4+IHRoZSBuYW1lLCBidXQgQUZB SUtTIHRoZSBIVyB0aHJlYWQgaXMgd2hhdCBpcyByZXF1aXJlZCBmb3IKPiA+Pj4gbWVtYmFycmll cikuIFNvIHRoaXMgaXMgd3JvbmcsIHBvd2VycGMgbmVlZHMgbm90aGluZyBoZXJlLgo+ID4+IAo+ ID4+IEZhaXIgZW5vdWdoLiAgSSdtIGhhcHB5IHRvIGRlZmVyIHRvIHlvdSBvbiB0aGUgcG93ZXJw YyBkZXRhaWxzLiAgSW4KPiA+PiBhbnkgY2FzZSwgdGhpcyBqdXN0IGlsbHVzdHJhdGVzIHRoYXQg d2UgbmVlZCBmZWVkYmFjayBmcm9tIGEgcGVyc29uCj4gPj4gd2hvIGtub3dzIG1vcmUgYWJvdXQg QVJNNjQgdGhhbiBJIGRvLgo+ID4gCj4gPiBJIHRoaW5rIHdlJ3JlIGluIGEgdmVyeSBzaW1pbGFy IGJvYXQgdG8gUG93ZXJQQywgZndpdy4gUm91Z2hseSBzcGVha2luZzoKPiA+IAo+ID4gIDEuIFNZ TkNfQ09SRSBkb2VzIF9ub3RfIHBlcmZvcm0gYW55IGNhY2hlIG1hbmFnZW1lbnQ7IHRoYXQgaXMg dGhlCj4gPiAgICAgcmVzcG9uc2liaWxpdHkgb2YgdXNlcnNwYWNlLCBlaXRoZXIgYnkgZXhlY3V0 aW5nIHRoZSByZWxldmFudAo+ID4gICAgIG1haW50ZW5hbmNlIGluc3RydWN0aW9ucyAoYXJtNjQp IG9yIGEgc3lzdGVtIGNhbGwgKGFybTMyKS4gQ3J1Y2lhbGx5LAo+ID4gICAgIHRoZSBoYXJkd2Fy ZSB3aWxsIGVuc3VyZSB0aGF0IHRoaXMgY2FjaGUgbWFpbnRlbmFuY2UgaXMgYnJvYWRjYXN0Cj4g PiAgICAgdG8gYWxsIG90aGVyIENQVXMuCj4gCj4gSXMgdGhpcyBndWFyYW50ZWVkIHJlZ2FyZGxl c3Mgb2YgYW55IGFsaWFzZXM/ICBUaGF0IGlzLCBpZiBJIGZsdXNoIGZyb20KPiBvbmUgQ1BVIGF0 IG9uZSBWQSBhbmQgdGhlbiBleGVjdXRlIHRoZSBzYW1lIHBoeXNpY2FsIGFkZHJlc3MgZnJvbSBh bm90aGVyCj4gQ1BVIGF0IGEgZGlmZmVyZW50IFZBLCBkb2VzIHRoaXMgc3RpbGwgd29yaz8KClRo ZSBkYXRhIHNpZGUgd2lsbCBiZSBmaW5lLCBidXQgdGhlIGluc3RydWN0aW9uIHNpZGUgY2FuIGhh dmUgdmlydHVhbAphbGlhc2VzLiBXZSBoYW5kbGUgdGhpcyBpbiBmbHVzaF9wdHJhY2VfYWNjZXNz KCkgYnkgYmxvd2luZyBhd2F5IHRoZSB3aG9sZQpJLWNhY2hlIGlmIHdlJ3JlIG5vdCBwaHlzaWNh bGx5LWluZGV4ZWQsIGJ1dCB1c2Vyc3BhY2Ugd291bGQgYmUgaW4gdHJvdWJsZQppZiBpdCB3YW50 ZWQgdG8gaGFuZGxlIHRoaXMgc2l0dWF0aW9uIGFsb25lLgoKPiA+ICAyLiBFdmVuIHdpdGggYWxs IHRoZSBjYWNoZSBtYWludGVuYW5jZSBpbiB0aGUgd29ybGQsIGEgQ1BVIGNvdWxkIGhhdmUKPiA+ ICAgICBzcGVjdWxhdGl2ZWx5IGZldGNoZWQgc3RhbGUgaW5zdHJ1Y3Rpb25zIGludG8gaXRzICJw aXBlbGluZSIgYWhlYWQgb2YKPiA+ICAgICB0aW1lLCBhbmQgdGhlc2UgYXJlIF9ub3RfIGZsdXNo ZWQgYnkgdGhlIGJyb2FkY2FzdCBtYWludGVuYW5jZSBpbnN0cnVjdGlvbnMKPiA+ICAgICBpbiAo MSkuIFNZTkNfQ09SRSBwcm92aWRlcyBhIG1lYW5zIGZvciB1c2Vyc3BhY2UgdG8gZGlzY2FyZCB0 aGVzZSBzdGFsZQo+ID4gICAgIGluc3RydWN0aW9ucy4KPiA+IAo+ID4gIDMuIFRoZSBjb250ZXh0 IHN5bmNocm9uaXphdGlvbiBldmVudCBvbiBleGNlcHRpb24gZW50cnkvZXhpdCBpcwo+ID4gICAg IHN1ZmZpY2llbnQgaGVyZS4gVGhlIEFybSBBUk0gaXNuJ3QgdmVyeSBnb29kIGF0IGRlc2NyaWJp bmcgd2hhdCBpdAo+ID4gICAgIGRvZXMsIGJlY2F1c2UgaXQncyBpbiBkZW5pYWwgYWJvdXQgdGhl IGV4aXN0ZW5jZSBvZiBhIHBpcGVsaW5lLCBidXQKPiA+ICAgICBpdCBkb2VzIGhhdmUgc25pcHBl dHMgc3VjaCBhczoKPiA+IAo+ID4gICAgKHMvUEUvQ1BVLykKPiA+ICAgICAgIHwgRm9yIGFsbCB0 eXBlcyBvZiBtZW1vcnk6Cj4gPiAgICAgICB8IFRoZSBQRSBtaWdodCBoYXZlIGZldGNoZWQgdGhl IGluc3RydWN0aW9ucyBmcm9tIG1lbW9yeSBhdCBhbnkgdGltZQo+ID4gICAgICAgfCBzaW5jZSB0 aGUgbGFzdCBDb250ZXh0IHN5bmNocm9uaXphdGlvbiBldmVudCBvbiB0aGF0IFBFLgo+ID4gCj4g PiAgICAgSW50ZXJlc3RpbmdseSwgdGhlIGFyY2hpdGVjdHVyZSByZWNlbnRseSBhZGRlZCBhIGNv bnRyb2wgYml0IHRvIHJlbW92ZQo+ID4gICAgIHRoaXMgc3luY2hyb25pc2F0aW9uIGZyb20gZXhj ZXB0aW9uIHJldHVybiwgc28gaWYgd2Ugc2V0IHRoYXQgdGhlbiB3ZSdkCj4gPiAgICAgaGF2ZSBh IHByb2JsZW0gd2l0aCBTWU5DX0NPUkUgYW5kIGFkZGluZyBhbiBJU0Igd291bGQgYmUgbmVjZXNz YXJ5IChhbmQKPiA+ICAgICB3ZSBjb3VsZCBwcm9iYWJsZSB0aGVuIG1ha2Uga2VybmVsLT5rZXJu ZWwgcmV0dXJucyBjaGVhcGVyLCBidXQgSQo+ID4gICAgIHN1c3BlY3Qgd2UncmUgcmVseWluZyBv biB0aGlzIGltcGxpY2l0IHN5bmNocm9uaXNhdGlvbiBpbiBvdGhlciBwbGFjZXMKPiA+ICAgICB0 b28pLgo+ID4gCj4gCj4gSXMgSVNCIGp1c3QgYSBjb250ZXh0IHN5bmNocm9uaXphdGlvbiBldmVu dCBvciBkb2VzIGl0IGRvIG1vcmU/CgpUaGF0J3MgYSBnb29kIHF1ZXN0aW9uLiBCYXJyaWVyIGlu c3RydWN0aW9ucyBvbiBBUk0gZG8gdGVuZCB0byBnZXQKb3ZlcmxvYWRlZCB3aXRoIGV4dHJhIGJl aGF2aW91cnMgb3ZlciB0aW1lLCBzbyBpdCBjb3VsZCBjZXJ0YWlubHkgZW5kIHVwCmRvaW5nIHRo ZSBjb250ZXh0IHN5bmNocm9uaXphdGlvbiBldmVudCArIGV4dHJhIHN0dWZmIGluIGZ1dHVyZS4g UmlnaHQgbm93LAp0aGUgb25seSB0aGluZyB0aGF0IHNwcmluZ3MgdG8gbWluZCBpcyB0aGUgc3Bl Y3RyZS12MSBoZWF2eSBtaXRpZ2F0aW9uCmJhcnJpZXIgb2YgJ0RTQjsgSVNCJyB3aGljaCwgZm9y IGV4YW1wbGUsIHByb2JhYmx5IGRvZXNuJ3Qgd29yayBmb3IgJ0RTQjsKRVJFVCcgYmVjYXVzZSB0 aGUgRVJFVCBjYW4gYmUgdHJlYXRlZCBsaWtlIGEgY29uZGl0aW9uYWwgKCEpIGJyYW5jaC4KCj4g T24geDg2LCBpdOKAmXMgdmVyeSBoYXJkIHRvIHRlbGwgdGhhdCBNRkVOQ0UgZG9lcyBhbnkgbW9y ZSB0aGFuIExPQ0ssIGJ1dAo+IGl04oCZcyBtdWNoIHNsb3dlci4gIEFuZCB3ZSBoYXZlIExGRU5D RSwgd2hpY2gsIGFzIGRvY3VtZW50ZWQsIGRvZXNu4oCZdAo+IGFwcGVhciB0byBoYXZlIGFueSBz ZW1hbnRpY3MgYXQgYWxsLiAgKE9yIGF0IGxlYXN0IGl0IGRpZG7igJl0IGJlZm9yZQo+IFNwZWN0 cmUuKQoKSSB0ZW5kIHRvIHRoaW5rIG9mIElTQiBhcyBhIGZyb250LWVuZCBiYXJyaWVyIHJlbGF0 aW5nIHRvIGluc3RydWN0aW9uIGZldGNoCndoZXJlYXMgRE1CLCBhY3F1aXJlL3JlbGVhc2UgYW5k IERTQiBhcmUgYWxsIGJhY2stZW5kIGJhcnJpZXJzIHJlbGF0aW5nIHRvCm1lbW9yeSBhY2Nlc3Nl cy4gWW91IF9jYW5fIHVzZSBJU0IgaW4gY29uanVuY3Rpb24gd2l0aCBjb250cm9sIGRlcGVuZGVu Y2llcwp0byBvcmRlciBhIHBhaXIgb2YgbG9hZHMgKGxpa2UgeW91IGNhbiB3aXRoIElTWU5DIG9u IFBvd2VyKSwgYnV0IGl0J3MgYQpyZWFsbHkgZXhwZW5zaXZlIHdheSB0byBkbyBpdC4KCj4gPiBB cmUgeW91IHNlZWluZyBhIHByb2JsZW0gaW4gcHJhY3RpY2UsIG9yIGRpZCB0aGlzIGNvbWUgdXAg d2hpbGUgdHJ5aW5nIHRvCj4gPiBkZWNpcGhlciB0aGUgc2VtYW50aWNzIG9mIFNZTkNfQ09SRT8K PiAKPiBJdCBjYW1lIHVwIHdoaWxlIHRyeWluZyB0byB1bmRlcnN0YW5kIHRoZSBjb2RlIGFuZCB3 b3JrIHRocm91Z2ggdmFyaW91cwo+IGJ1Z3MgaW4gaXQuICBUaGUgY29kZSB3YXMgd3JpdHRlbiB1 c2luZyBzb21ldGhpbmcgYXBwcm94aW1hdGluZyB4ODYKPiB0ZXJtaW5vbG9neSwgYnV0IGl0IHdh cyBkZWZpbml0ZWx5IHdyb25nIG9uIHg4NiAoYXQgbGVhc3QgaWYgeW91IGJlbGlldmUKPiB0aGUg U0RNLCBhbmQgSSBoYXZlbuKAmXQgY29udmluY2VkIGFueSBhcmNoaXRlY3RzIHRvIHNheSBvdGhl cndpc2UpLgoKT2ssIHRoYW5rcy4KCldpbGwKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fCmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFy bS1rZXJuZWxAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9t YWlsbWFuL2xpc3RpbmZvL2xpbnV4LWFybS1rZXJuZWwK