From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A76A8C4360F for ; Thu, 4 Apr 2019 05:05:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 78F7F20700 for ; Thu, 4 Apr 2019 05:05:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726725AbfDDFFa (ORCPT ); Thu, 4 Apr 2019 01:05:30 -0400 Received: from mx2.suse.de ([195.135.220.15]:50816 "EHLO mx1.suse.de" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725904AbfDDFFa (ORCPT ); Thu, 4 Apr 2019 01:05:30 -0400 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id EBA84AD8A; Thu, 4 Apr 2019 05:05:27 +0000 (UTC) Subject: Re: [PATCH v2 3/5] locking/qspinlock: Introduce CNA into the slow path of qspinlock To: Peter Zijlstra , Alex Kogan Cc: Waiman Long , linux@armlinux.org.uk, mingo@redhat.com, will.deacon@arm.com, arnd@arndb.de, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, bp@alien8.de, hpa@zytor.com, x86@kernel.org, steven.sistare@oracle.com, daniel.m.jordan@oracle.com, dave.dice@oracle.com, rahul.x.yadav@oracle.com References: <20190329152006.110370-1-alex.kogan@oracle.com> <20190329152006.110370-4-alex.kogan@oracle.com> <60a3a2d8-d222-73aa-2df1-64c9d3fa3241@redhat.com> <20190402094320.GM11158@hirez.programming.kicks-ass.net> <6AEDE4F2-306A-4DF9-9307-9E3517C68A2B@oracle.com> <20190403160112.GK4038@hirez.programming.kicks-ass.net> From: Juergen Gross Openpgp: preference=signencrypt Autocrypt: addr=jgross@suse.com; prefer-encrypt=mutual; keydata= mQENBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB AAG0H0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT6JATkEEwECACMFAlOMcK8CGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO RoVBYuiocc51872tRGywc03xaQydB+9R7BHPuQENBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAGJAR8EGAECAAkFAlOMcBYCGwwACgkQsN6d 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHf4kBrQQY AQgAIBYhBIUSZ3Lo9gSUpdCX97DendYovxMvBQJa3fDQAhsCAIEJELDendYovxMvdiAEGRYI AB0WIQRTLbB6QfY48x44uB6AXGG7T9hjvgUCWt3w0AAKCRCAXGG7T9hjvk2LAP99B/9FenK/ 1lfifxQmsoOrjbZtzCS6OKxPqOLHaY47BgEAqKKn36YAPpbk09d2GTVetoQJwiylx/Z9/mQI CUbQMg1pNQf9EjA1bNcMbnzJCgt0P9Q9wWCLwZa01SnQWFz8Z4HEaKldie+5bHBL5CzVBrLv 81tqX+/j95llpazzCXZW2sdNL3r8gXqrajSox7LR2rYDGdltAhQuISd2BHrbkQVEWD4hs7iV 1KQHe2uwXbKlguKPhk5ubZxqwsg/uIHw0qZDk+d0vxjTtO2JD5Jv/CeDgaBX4Emgp0NYs8IC UIyKXBtnzwiNv4cX9qKlz2Gyq9b+GdcLYZqMlIBjdCz0yJvgeb3WPNsCOanvbjelDhskx9gd 6YUUFFqgsLtrKpCNyy203a58g2WosU9k9H+LcheS37Ph2vMVTISMszW9W8gyORSgmw== Message-ID: Date: Thu, 4 Apr 2019 07:05:24 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <20190403160112.GK4038@hirez.programming.kicks-ass.net> Content-Type: text/plain; charset=utf-8 Content-Language: de-DE Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 03/04/2019 18:01, Peter Zijlstra wrote: > On Wed, Apr 03, 2019 at 11:39:09AM -0400, Alex Kogan wrote: > >>>> The patch that I am looking for is to have a separate >>>> numa_queued_spinlock_slowpath() that coexists with >>>> native_queued_spinlock_slowpath() and >>>> paravirt_queued_spinlock_slowpath(). At boot time, we select the most >>>> appropriate one for the system at hand. >> Is this how this selection works today for paravirt? >> I see a PARAVIRT_SPINLOCKS config option, but IIUC you are talking about a different mechanism here. >> Can you, please, elaborate or give me a link to a page that explains that? > > Oh man, you ask us to explain how paravirt patching works... that's > magic :-) > > Basically, the compiler will emit a bunch of indirect calls to the > various pv_ops.*.* functions. > > Then, at alternative_instructions() <- apply_paravirt() it will rewrite > all these indirect calls to direct calls to the function pointers that > are in the pv_ops structure at that time (+- more magic). > > So we initialize the pv_ops.lock.* methods to the normal > native_queued_spin*() stuff, if KVM/Xen/whatever setup detectors pv > spnlock support changes the methods to the paravirt_queued_*() stuff. > > If you wnt more details, you'll just have to read > arch/x86/include/asm/paravirt*.h and arch/x86/kernel/paravirt*.c, I > don't think there's a coherent writeup of all that. > >>> Agreed; and until we have static_call, I think we can abuse the paravirt >>> stuff for this. >>> >>> By the time we patch the paravirt stuff: >>> >>> check_bugs() >>> alternative_instructions() >>> apply_paravirt() >>> >>> we should already have enumerated the NODE topology and so nr_node_ids() >>> should be set. >>> >>> So if we frob pv_ops.lock.queued_spin_lock_slowpath to >>> numa_queued_spin_lock_slowpath before that, it should all get patched >>> just right. >>> >>> That of course means the whole NUMA_AWARE_SPINLOCKS thing depends on >>> PARAVIRT_SPINLOCK, which is a bit awkward… > >> Just to mention here, the patch so far does not address paravirt, but >> our goal is to add this support once we address all the concerns for >> the native version. So we will end up with four variants for the >> queued_spinlock_slowpath() — one for each combination of >> native/paravirt and NUMA/non-NUMA. Or perhaps we do not need a >> NUMA/paravirt variant? > > I wouldn't bother with a pv version of the numa aware code at all. If > you have overcommitted guests, topology is likely irrelevant anyway. If > you have 1:1 pinned guests, they'll not use pv spinlocks anyway. > > So keep it to tertiary choice: > > - native > - native/numa > - paravirt Just for the records: the paravirt variant could easily choose whether it wants to include a numa version just by using the existing hooks. With PARAVIRT_SPINLOCK configured I guess even the native case would need to use the paravirt hooks for selection of native or native/numa. Without PARAVIRT_SPINLOCK this would be just an alternative() then? Maybe the resulting code would be much more readable if we'd just make PARAVIRT_SPINLOCK usable without the other PARAVIRT hooks? So splitting up PARAVIRT into PARAVIRT_GUEST (timer hooks et al) and the patching infrastructure, with PARAVIRT_GUEST and PARAVIRT_SPINLOCK selecting PARAVIRT, and PARAVIRT_XXL selecting PARAVIRT_GUEST. Juergen From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=3.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C12BC4360F for ; Thu, 4 Apr 2019 05:05:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 59B1420700 for ; Thu, 4 Apr 2019 05:05:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="A5Wy0qrQ" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 59B1420700 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=suse.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date: Message-ID:From:References:To:Subject:Reply-To:Content-ID:Content-Description :Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=iByXRwm4U6Xw50EpXT+a11mtd3u2hi1MhfDwuMOBFGI=; b=A5Wy0qrQb53CRs zeNJUyMVhT3j5oGjUbBRhNXO9XmwIlu0TJtN3w08vtGFnW4tz/Xw6wApoO+oMpAdkhkKz2/Dp8BRA iY94ZHTKMYcdH/B1Q86Jt+EWE5LZI5xOl9n14BMOAnUVj23cZdYNEd58piZ4fZZ8xaArmUHc2LKgs zTZIltju8rJl8tTxLD9Woqe3CQvNqP5yf5cjCFD4zqshkm9i1xq1aFYGQzFJwQtHh8QoonSkFc+5M EcKZ0Ct8fGrjQ6kbBB41emERXF5iWrd+4f5nCxxjrXkpQOkB0OKwFZ+ortYbvYleP129qRQhn3Cau ltN9fUlbtP/iE1jJi0kQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1hBuZD-0006NU-VK; Thu, 04 Apr 2019 05:05:35 +0000 Received: from mx2.suse.de ([195.135.220.15] helo=mx1.suse.de) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1hBuZA-0006Mg-9F for linux-arm-kernel@lists.infradead.org; Thu, 04 Apr 2019 05:05:34 +0000 X-Virus-Scanned: by amavisd-new at test-mx.suse.de Received: from relay2.suse.de (unknown [195.135.220.254]) by mx1.suse.de (Postfix) with ESMTP id EBA84AD8A; Thu, 4 Apr 2019 05:05:27 +0000 (UTC) Subject: Re: [PATCH v2 3/5] locking/qspinlock: Introduce CNA into the slow path of qspinlock To: Peter Zijlstra , Alex Kogan References: <20190329152006.110370-1-alex.kogan@oracle.com> <20190329152006.110370-4-alex.kogan@oracle.com> <60a3a2d8-d222-73aa-2df1-64c9d3fa3241@redhat.com> <20190402094320.GM11158@hirez.programming.kicks-ass.net> <6AEDE4F2-306A-4DF9-9307-9E3517C68A2B@oracle.com> <20190403160112.GK4038@hirez.programming.kicks-ass.net> From: Juergen Gross Openpgp: preference=signencrypt Autocrypt: addr=jgross@suse.com; prefer-encrypt=mutual; keydata= mQENBFOMcBYBCACgGjqjoGvbEouQZw/ToiBg9W98AlM2QHV+iNHsEs7kxWhKMjrioyspZKOB ycWxw3ie3j9uvg9EOB3aN4xiTv4qbnGiTr3oJhkB1gsb6ToJQZ8uxGq2kaV2KL9650I1SJve dYm8Of8Zd621lSmoKOwlNClALZNew72NjJLEzTalU1OdT7/i1TXkH09XSSI8mEQ/ouNcMvIJ NwQpd369y9bfIhWUiVXEK7MlRgUG6MvIj6Y3Am/BBLUVbDa4+gmzDC9ezlZkTZG2t14zWPvx XP3FAp2pkW0xqG7/377qptDmrk42GlSKN4z76ELnLxussxc7I2hx18NUcbP8+uty4bMxABEB AAG0H0p1ZXJnZW4gR3Jvc3MgPGpncm9zc0BzdXNlLmNvbT6JATkEEwECACMFAlOMcK8CGwMH CwkIBwMCAQYVCAIJCgsEFgIDAQIeAQIXgAAKCRCw3p3WKL8TL8eZB/9G0juS/kDY9LhEXseh mE9U+iA1VsLhgDqVbsOtZ/S14LRFHczNd/Lqkn7souCSoyWsBs3/wO+OjPvxf7m+Ef+sMtr0 G5lCWEWa9wa0IXx5HRPW/ScL+e4AVUbL7rurYMfwCzco+7TfjhMEOkC+va5gzi1KrErgNRHH kg3PhlnRY0Udyqx++UYkAsN4TQuEhNN32MvN0Np3WlBJOgKcuXpIElmMM5f1BBzJSKBkW0Jc Wy3h2Wy912vHKpPV/Xv7ZwVJ27v7KcuZcErtptDevAljxJtE7aJG6WiBzm+v9EswyWxwMCIO RoVBYuiocc51872tRGywc03xaQydB+9R7BHPuQENBFOMcBYBCADLMfoA44MwGOB9YT1V4KCy vAfd7E0BTfaAurbG+Olacciz3yd09QOmejFZC6AnoykydyvTFLAWYcSCdISMr88COmmCbJzn sHAogjexXiif6ANUUlHpjxlHCCcELmZUzomNDnEOTxZFeWMTFF9Rf2k2F0Tl4E5kmsNGgtSa aMO0rNZoOEiD/7UfPP3dfh8JCQ1VtUUsQtT1sxos8Eb/HmriJhnaTZ7Hp3jtgTVkV0ybpgFg w6WMaRkrBh17mV0z2ajjmabB7SJxcouSkR0hcpNl4oM74d2/VqoW4BxxxOD1FcNCObCELfIS auZx+XT6s+CE7Qi/c44ibBMR7hyjdzWbABEBAAGJAR8EGAECAAkFAlOMcBYCGwwACgkQsN6d 1ii/Ey9D+Af/WFr3q+bg/8v5tCknCtn92d5lyYTBNt7xgWzDZX8G6/pngzKyWfedArllp0Pn fgIXtMNV+3t8Li1Tg843EXkP7+2+CQ98MB8XvvPLYAfW8nNDV85TyVgWlldNcgdv7nn1Sq8g HwB2BHdIAkYce3hEoDQXt/mKlgEGsLpzJcnLKimtPXQQy9TxUaLBe9PInPd+Ohix0XOlY+Uk QFEx50Ki3rSDl2Zt2tnkNYKUCvTJq7jvOlaPd6d/W0tZqpyy7KVay+K4aMobDsodB3dvEAs6 ScCnh03dDAFgIq5nsB11j3KPKdVoPlfucX2c7kGNH+LUMbzqV6beIENfNexkOfxHf4kBrQQY AQgAIBYhBIUSZ3Lo9gSUpdCX97DendYovxMvBQJa3fDQAhsCAIEJELDendYovxMvdiAEGRYI AB0WIQRTLbB6QfY48x44uB6AXGG7T9hjvgUCWt3w0AAKCRCAXGG7T9hjvk2LAP99B/9FenK/ 1lfifxQmsoOrjbZtzCS6OKxPqOLHaY47BgEAqKKn36YAPpbk09d2GTVetoQJwiylx/Z9/mQI CUbQMg1pNQf9EjA1bNcMbnzJCgt0P9Q9wWCLwZa01SnQWFz8Z4HEaKldie+5bHBL5CzVBrLv 81tqX+/j95llpazzCXZW2sdNL3r8gXqrajSox7LR2rYDGdltAhQuISd2BHrbkQVEWD4hs7iV 1KQHe2uwXbKlguKPhk5ubZxqwsg/uIHw0qZDk+d0vxjTtO2JD5Jv/CeDgaBX4Emgp0NYs8IC UIyKXBtnzwiNv4cX9qKlz2Gyq9b+GdcLYZqMlIBjdCz0yJvgeb3WPNsCOanvbjelDhskx9gd 6YUUFFqgsLtrKpCNyy203a58g2WosU9k9H+LcheS37Ph2vMVTISMszW9W8gyORSgmw== Message-ID: Date: Thu, 4 Apr 2019 07:05:24 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.5.1 MIME-Version: 1.0 In-Reply-To: <20190403160112.GK4038@hirez.programming.kicks-ass.net> Content-Language: de-DE X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190403_220532_616051_899AB676 X-CRM114-Status: GOOD ( 22.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch@vger.kernel.org, arnd@arndb.de, dave.dice@oracle.com, x86@kernel.org, will.deacon@arm.com, linux@armlinux.org.uk, steven.sistare@oracle.com, linux-kernel@vger.kernel.org, rahul.x.yadav@oracle.com, mingo@redhat.com, bp@alien8.de, hpa@zytor.com, Waiman Long , tglx@linutronix.de, daniel.m.jordan@oracle.com, linux-arm-kernel@lists.infradead.org Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+infradead-linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gMDMvMDQvMjAxOSAxODowMSwgUGV0ZXIgWmlqbHN0cmEgd3JvdGU6Cj4gT24gV2VkLCBBcHIg MDMsIDIwMTkgYXQgMTE6Mzk6MDlBTSAtMDQwMCwgQWxleCBLb2dhbiB3cm90ZToKPiAKPj4+PiBU aGUgcGF0Y2ggdGhhdCBJIGFtIGxvb2tpbmcgZm9yIGlzIHRvIGhhdmUgYSBzZXBhcmF0ZQo+Pj4+ IG51bWFfcXVldWVkX3NwaW5sb2NrX3Nsb3dwYXRoKCkgdGhhdCBjb2V4aXN0cyB3aXRoCj4+Pj4g bmF0aXZlX3F1ZXVlZF9zcGlubG9ja19zbG93cGF0aCgpIGFuZAo+Pj4+IHBhcmF2aXJ0X3F1ZXVl ZF9zcGlubG9ja19zbG93cGF0aCgpLiBBdCBib290IHRpbWUsIHdlIHNlbGVjdCB0aGUgbW9zdAo+ Pj4+IGFwcHJvcHJpYXRlIG9uZSBmb3IgdGhlIHN5c3RlbSBhdCBoYW5kLgo+PiBJcyB0aGlzIGhv dyB0aGlzIHNlbGVjdGlvbiB3b3JrcyB0b2RheSBmb3IgcGFyYXZpcnQ/Cj4+IEkgc2VlIGEgUEFS QVZJUlRfU1BJTkxPQ0tTIGNvbmZpZyBvcHRpb24sIGJ1dCBJSVVDIHlvdSBhcmUgdGFsa2luZyBh Ym91dCBhIGRpZmZlcmVudCBtZWNoYW5pc20gaGVyZS4KPj4gQ2FuIHlvdSwgcGxlYXNlLCBlbGFi b3JhdGUgb3IgZ2l2ZSBtZSBhIGxpbmsgdG8gYSBwYWdlIHRoYXQgZXhwbGFpbnMgdGhhdD8KPiAK PiBPaCBtYW4sIHlvdSBhc2sgdXMgdG8gZXhwbGFpbiBob3cgcGFyYXZpcnQgcGF0Y2hpbmcgd29y a3MuLi4gdGhhdCdzCj4gbWFnaWMgOi0pCj4gCj4gQmFzaWNhbGx5LCB0aGUgY29tcGlsZXIgd2ls bCBlbWl0IGEgYnVuY2ggb2YgaW5kaXJlY3QgY2FsbHMgdG8gdGhlCj4gdmFyaW91cyBwdl9vcHMu Ki4qIGZ1bmN0aW9ucy4KPiAKPiBUaGVuLCBhdCBhbHRlcm5hdGl2ZV9pbnN0cnVjdGlvbnMoKSA8 LSBhcHBseV9wYXJhdmlydCgpIGl0IHdpbGwgcmV3cml0ZQo+IGFsbCB0aGVzZSBpbmRpcmVjdCBj YWxscyB0byBkaXJlY3QgY2FsbHMgdG8gdGhlIGZ1bmN0aW9uIHBvaW50ZXJzIHRoYXQKPiBhcmUg aW4gdGhlIHB2X29wcyBzdHJ1Y3R1cmUgYXQgdGhhdCB0aW1lICgrLSBtb3JlIG1hZ2ljKS4KPiAK PiBTbyB3ZSBpbml0aWFsaXplIHRoZSBwdl9vcHMubG9jay4qIG1ldGhvZHMgdG8gdGhlIG5vcm1h bAo+IG5hdGl2ZV9xdWV1ZWRfc3BpbiooKSBzdHVmZiwgaWYgS1ZNL1hlbi93aGF0ZXZlciBzZXR1 cCBkZXRlY3RvcnMgcHYKPiBzcG5sb2NrIHN1cHBvcnQgY2hhbmdlcyB0aGUgbWV0aG9kcyB0byB0 aGUgcGFyYXZpcnRfcXVldWVkXyooKSBzdHVmZi4KPiAKPiBJZiB5b3Ugd250IG1vcmUgZGV0YWls cywgeW91J2xsIGp1c3QgaGF2ZSB0byByZWFkCj4gYXJjaC94ODYvaW5jbHVkZS9hc20vcGFyYXZp cnQqLmggYW5kIGFyY2gveDg2L2tlcm5lbC9wYXJhdmlydCouYywgSQo+IGRvbid0IHRoaW5rIHRo ZXJlJ3MgYSBjb2hlcmVudCB3cml0ZXVwIG9mIGFsbCB0aGF0Lgo+IAo+Pj4gQWdyZWVkOyBhbmQg dW50aWwgd2UgaGF2ZSBzdGF0aWNfY2FsbCwgSSB0aGluayB3ZSBjYW4gYWJ1c2UgdGhlIHBhcmF2 aXJ0Cj4+PiBzdHVmZiBmb3IgdGhpcy4KPj4+Cj4+PiBCeSB0aGUgdGltZSB3ZSBwYXRjaCB0aGUg cGFyYXZpcnQgc3R1ZmY6Cj4+Pgo+Pj4gIGNoZWNrX2J1Z3MoKQo+Pj4gICAgYWx0ZXJuYXRpdmVf aW5zdHJ1Y3Rpb25zKCkKPj4+ICAgICAgYXBwbHlfcGFyYXZpcnQoKQo+Pj4KPj4+IHdlIHNob3Vs ZCBhbHJlYWR5IGhhdmUgZW51bWVyYXRlZCB0aGUgTk9ERSB0b3BvbG9neSBhbmQgc28gbnJfbm9k ZV9pZHMoKQo+Pj4gc2hvdWxkIGJlIHNldC4KPj4+Cj4+PiBTbyBpZiB3ZSBmcm9iIHB2X29wcy5s b2NrLnF1ZXVlZF9zcGluX2xvY2tfc2xvd3BhdGggdG8KPj4+IG51bWFfcXVldWVkX3NwaW5fbG9j a19zbG93cGF0aCBiZWZvcmUgdGhhdCwgaXQgc2hvdWxkIGFsbCBnZXQgcGF0Y2hlZAo+Pj4ganVz dCByaWdodC4KPj4+Cj4+PiBUaGF0IG9mIGNvdXJzZSBtZWFucyB0aGUgd2hvbGUgTlVNQV9BV0FS RV9TUElOTE9DS1MgdGhpbmcgZGVwZW5kcyBvbgo+Pj4gUEFSQVZJUlRfU1BJTkxPQ0ssIHdoaWNo IGlzIGEgYml0IGF3a3dhcmTigKYKPiAKPj4gSnVzdCB0byBtZW50aW9uIGhlcmUsIHRoZSBwYXRj aCBzbyBmYXIgZG9lcyBub3QgYWRkcmVzcyBwYXJhdmlydCwgYnV0Cj4+IG91ciBnb2FsIGlzIHRv IGFkZCB0aGlzIHN1cHBvcnQgb25jZSB3ZSBhZGRyZXNzIGFsbCB0aGUgY29uY2VybnMgZm9yCj4+ IHRoZSBuYXRpdmUgdmVyc2lvbi4gIFNvIHdlIHdpbGwgZW5kIHVwIHdpdGggZm91ciB2YXJpYW50 cyBmb3IgdGhlCj4+IHF1ZXVlZF9zcGlubG9ja19zbG93cGF0aCgpIOKAlCBvbmUgZm9yIGVhY2gg Y29tYmluYXRpb24gb2YKPj4gbmF0aXZlL3BhcmF2aXJ0IGFuZCBOVU1BL25vbi1OVU1BLiAgT3Ig cGVyaGFwcyB3ZSBkbyBub3QgbmVlZCBhCj4+IE5VTUEvcGFyYXZpcnQgdmFyaWFudD8KPiAKPiBJ IHdvdWxkbid0IGJvdGhlciB3aXRoIGEgcHYgdmVyc2lvbiBvZiB0aGUgbnVtYSBhd2FyZSBjb2Rl IGF0IGFsbC4gSWYKPiB5b3UgaGF2ZSBvdmVyY29tbWl0dGVkIGd1ZXN0cywgdG9wb2xvZ3kgaXMg bGlrZWx5IGlycmVsZXZhbnQgYW55d2F5LiBJZgo+IHlvdSBoYXZlIDE6MSBwaW5uZWQgZ3Vlc3Rz LCB0aGV5J2xsIG5vdCB1c2UgcHYgc3BpbmxvY2tzIGFueXdheS4KPiAKPiBTbyBrZWVwIGl0IHRv IHRlcnRpYXJ5IGNob2ljZToKPiAKPiAgLSBuYXRpdmUKPiAgLSBuYXRpdmUvbnVtYQo+ICAtIHBh cmF2aXJ0CgpKdXN0IGZvciB0aGUgcmVjb3JkczogdGhlIHBhcmF2aXJ0IHZhcmlhbnQgY291bGQg ZWFzaWx5IGNob29zZSB3aGV0aGVyCml0IHdhbnRzIHRvIGluY2x1ZGUgYSBudW1hIHZlcnNpb24g anVzdCBieSB1c2luZyB0aGUgZXhpc3RpbmcgaG9va3MuCldpdGggUEFSQVZJUlRfU1BJTkxPQ0sg Y29uZmlndXJlZCBJIGd1ZXNzIGV2ZW4gdGhlIG5hdGl2ZSBjYXNlIHdvdWxkCm5lZWQgdG8gdXNl IHRoZSBwYXJhdmlydCBob29rcyBmb3Igc2VsZWN0aW9uIG9mIG5hdGl2ZSBvciBuYXRpdmUvbnVt YS4KCldpdGhvdXQgUEFSQVZJUlRfU1BJTkxPQ0sgdGhpcyB3b3VsZCBiZSBqdXN0IGFuIGFsdGVy bmF0aXZlKCkgdGhlbj8KCk1heWJlIHRoZSByZXN1bHRpbmcgY29kZSB3b3VsZCBiZSBtdWNoIG1v cmUgcmVhZGFibGUgaWYgd2UnZCBqdXN0Cm1ha2UgUEFSQVZJUlRfU1BJTkxPQ0sgdXNhYmxlIHdp dGhvdXQgdGhlIG90aGVyIFBBUkFWSVJUIGhvb2tzPyBTbwpzcGxpdHRpbmcgdXAgUEFSQVZJUlQg aW50byBQQVJBVklSVF9HVUVTVCAodGltZXIgaG9va3MgZXQgYWwpIGFuZAp0aGUgcGF0Y2hpbmcg aW5mcmFzdHJ1Y3R1cmUsIHdpdGggUEFSQVZJUlRfR1VFU1QgYW5kIFBBUkFWSVJUX1NQSU5MT0NL CnNlbGVjdGluZyBQQVJBVklSVCwgYW5kIFBBUkFWSVJUX1hYTCBzZWxlY3RpbmcgUEFSQVZJUlRf R1VFU1QuCgoKSnVlcmdlbgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBs aXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlz dGluZm8vbGludXgtYXJtLWtlcm5lbAo=