From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC8E8C54EBE for ; Tue, 10 Jan 2023 06:17:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230472AbjAJGRx (ORCPT ); Tue, 10 Jan 2023 01:17:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43058 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230305AbjAJGRo (ORCPT ); Tue, 10 Jan 2023 01:17:44 -0500 Received: from mail-vk1-xa2d.google.com (mail-vk1-xa2d.google.com [IPv6:2607:f8b0:4864:20::a2d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C586D1C921 for ; Mon, 9 Jan 2023 22:17:41 -0800 (PST) Received: by mail-vk1-xa2d.google.com with SMTP id l3so2032274vkk.11 for ; Mon, 09 Jan 2023 22:17:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8CJjolKPlWAhG4ywZunllbYBE4Jf2JMO0vrAUHEfkjw=; b=UQV89TjBxsMqYrp4HaShVHvXh2Ce0AKWRK3bLEHVKP0BIHOPTp7pbUEbgPr4oDetb4 KI6ji9+60LnPfZDux/u8NGhltUa2rNdtUr5nNvs+NOpOusIkyWogcQl3Qi0b5cVfk/HD Gh2i9t7rior+q6Me/lCPQIn5QuyDXHM2x5ueSvLMq8tc9SKm/TYktR9hWEx+W9AiGxj5 QBuA6xIt4tgB+RWMMN8kJJ+9ox30KpjPsbLdlifphOkEGsHdICraWhSs5OrosNfPZgNW /+CCRvggxl47aOOHG//Hb/KGERPJPyYBOH7ZVvXsoEqhIZ3U0g+gLp7sWxQhP0YGc/g6 GoaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8CJjolKPlWAhG4ywZunllbYBE4Jf2JMO0vrAUHEfkjw=; b=IzrjvTgo1AbxH0jrLS1Ub0MxJ2/dOPTABFOD+BB0SvklfRCfsPKy7xCy0RthiGUkyn QfRMQ6EEKdHIrwhZ3Dtqw/Cwx57s2N8aAF+thPsMmFOsa/L+QOQ9QXjLxrGn+53yInQs AuJV3xO3JvnGOfyFDv+q2g9KmFRDmvwptEiKZXVC+v4M6sTJwIQkSYes7ESGrHqNB7Zo FM4O1x14VI0NRS3Dn11LkwTAIY4GQIzKBmwqMlpftdt1qAYwIbP5QcDulWsKO2P41ASk p5dfjWAkRuTZ78lgeFrLVkKFoqzljnCl5W8FLqwayOVPi7gjKcNlM0OEK4+4m9QRrr/M 3JgA== X-Gm-Message-State: AFqh2kp/4EKwodlBBW/M6VWxAr0v1fklZAkfNdel3Vyeu5kLf4SFoFxC /Ruey5vnJVBb1ZmqUKgLJmbO+2U/aPyx/Uasz/KRuQ== X-Google-Smtp-Source: AMrXdXv+UbPcEC0JIbuEUF6JQZeNC3GXIR+0tfBx5v6uXH+RRZFfR3XkgGAYKN5vIlEIZAfTFcYl9N5U9BOeOz/MzgA= X-Received: by 2002:a1f:948e:0:b0:3b7:6c07:c79c with SMTP id w136-20020a1f948e000000b003b76c07c79cmr8478522vkd.34.1673331460720; Mon, 09 Jan 2023 22:17:40 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Anup Patel Date: Tue, 10 Jan 2023 11:47:29 +0530 Message-ID: Subject: Re: Expected rdpmc behavior during context swtich and a RISC-V conundrum To: Atish Patra Cc: Peter Zijlstra , linux-perf-users@vger.kernel.org, "linux-kernel@vger.kernel.org List" , Mark Rutland , Arnaldo Carvalho de Melo , Alexander Shishkin , Will Deacon , Stephane Eranian , Andi Kleen , Palmer Dabbelt , Beeman Strong , Atish Patra , Kan Liang , linux-riscv , "open list:KERNEL VIRTUAL MACHINE FOR RISC-V (KVM/riscv)" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org +linux-riscv +kvm-riscv On Tue, Jan 10, 2023 at 1:26 AM Atish Patra wrote: > > On Mon, Jan 9, 2023 at 4:41 AM Peter Zijlstra wrot= e: > > > > On Thu, Jan 05, 2023 at 11:59:24AM -0800, Atish Patra wrote: > > > Hi All, > > > There was a recent uabi update[1] for RISC-V that allows the users to > > > read cycle and instruction count without any checks. > > > We tried to restrict that behavior to address security concerns > > > earlier but it resulted in breakage for some user space > > > applications[2]. > > > Thus, previous behavior was restored where a user on RISC-V platforms > > > can directly read cycle or instruction count[3]. > > > > > > Comparison with other ISAs w.r.t user space access of counters: > > > ARM64 > > > -- Enabled/Disabled via (/proc/sys/kernel/perf_user_access) > > > -- Only for task bound events configured via perf. > > > > > > X86 > > > --- rdpmc instruction > > > --- Enable/Disable via =E2=80=9C/sys/devices/cpu/rdpmc=E2=80=9D > > > -- Before v4.0 > > > -- any process (even without active perf event) rdpmc > > > After v4.0 > > > -- Default behavior changed to support only active events in a > > > process=E2=80=99s context. > > > -- Configured through perf similar to ARM64 > > > -- Continue to maintain backward compatibility for unrestricted acces= s > > > by writing 2 to =E2=80=9C/sys/devices/cpu/rdpmc=E2=80=9D > > > > > > IMO, RISC-V should only enable user space access through perf similar > > > to ARM64 and x86 (post v4.0). > > > However, we do have to support the legacy behavior to avoid > > > application breakage. > > > As per my understanding a direct user space access can lead to the > > > following problems: > > > > > > 1) There is no context switch support, so counts from other contexts = are exposed > > > 2) If a perf user is allocated one of these counters, the counter > > > value will be written > > > > > > Looking at the x86 code as it continues to allow the above behavior, > > > rdpmc_always_available_key is enabled in the above case. However, > > > during the context switch (cr4_update_pce_mm) > > > only dirty counters are cleared. It only prevents leakage from perf > > > task to rdpmc task. > > > > > > How does the context switch of counters work for users who enable > > > unrestricted access by writing 2 to =E2=80=9C/sys/devices/cpu/rdpmc= =E2=80=9D ? > > > Otherwise, rdpmc users likely get noise from other applications. Is > > > that expected ? > > > This can be a security concern also where a rogue rdpmc user > > > application can monitor other critical applications to initiate side > > > channel attack. > > > > > > Am I missing something? Please correct my understanding of the x86 > > > implementation if it is wrong. > > > > So on x86 we have RDTSC and RDPMC instructions. RDTSC reads the > > Time-Stamp-Counter which is a globally synchronized monotonic increasin= g > > counter at some 'random' rate (idealized, don't ask). This thing is use= d > > for time-keeping etc.. > > > > And then there's RDPMC which (optionally) allows reading the PMU > > counters which are normally disabled and all 0. > > > > Even if RDPMC is unconditionally allowed from userspace (the 2 option > > you refer to) userspace will only be able to read these 0s unless > > someone also programs the PMU. Linux only supports a single means of > > doing so: perf (some people use /dev/msr to poke directly to the MSRs > > but they get to keep all pieces). > > > > It makes sense now. Thanks!! > > AFAIK, the /dev/msr interface is also allowed for root users only. So tha= t > covers the security concerns I was asking about. > > > RDPMC is only useful if you read counters you own on yourself -- IOW > > selfmonitoring, using the interface outlined in uapi/linux/perf_events.= h > > near struct perf_event_mmap_page. > > > > Any other usage -- you get to keep the pieces. > > > > Can you observe random other counters, yes, unavoidably so. The sysfs > > control you mention was instituted to restrict this somewhat. > > > > If the RISC-V counters are fundamentally the PMU counters that need to > > be reset to trigger events, then you've managed to paint yourself into = a > > tight spot :/ > > > > Either you must dis-allow userspace access to these things (and break > > them) or limit the PMU usage -- both options suck. > > > > > > Now, I'm thinking that esp. something like instruction count is not > > synchronized between cores (seems fundamentally impossible) and can onl= y > > be reasonably be consumed (and compared) when strictly affine to a > > particular CPU, you can argue that applications doing this without also > > strictly managing their affinity mask are broken anyway and therefore > > your breakage is not in fact a breaking them -- you can't break > > something that's already broken. > > > > I think most broken applications were using rdcycle to measure time > which was wrong anyways. > It probably happened because there was no "time" CSR in the early > hardwares. Thus, the rdtime would > trap & emulated by the firmware which was slow. This lead to user > space application to use rdcycle which > was not correct either. So the existing applications are broken for > using rdcycle as well. > > Since both cycle & instret behave similarly (fixed counters), they get > enabled/disabled together. > > > > > Anyway, given RISC-V being a very young platform, I would try really > > *really* *REALLY* hard to stomp on these applications and get them to > > change in order to reclaim the PMU usage. > > Yes. Thanks for your valuable input. I agree with Peter Z. We had a similar discussion in the Performance Analysis SIG of RISC-V international as well. This also forces KVM (and other hypervisors) to save-n-restore CYCLE and INSTRUCTION counters so that one Guest/VM can't see cycle/instruction counts from another Guest/VM. Only a few applications are affected and RISC-V ecosystem is young so it is better to change these applications rather than making CYCLE and INSTRUCTION counters as part of uABI. Regards, Anup From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5FE14C46467 for ; Tue, 10 Jan 2023 06:17:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eqhWn4yPKMXp361pLwrWPBHBVTCsCRZ/kKi6smU/n3g=; b=iZBcKvod0uR9MI SXwp3+OyAiQ1AL5bWCbD/wjvIehlgEXzoXh8Dv/PmY5fXd3njMvVRpMOLVVB+5yvuH9e7M5U4aGqr s/y1Ji2OBAmeRx/pT2twtvanHXPxX/qqI5V8Qml98lGMsAqySlsRuMwZcrIIPLiSuGqAEUFKgJPVz KLDHYfrqL4UrXF3MQtgQ+wEpWt17SqZtD2ThMzdBcn6npBHZgwEvnfo3QKr3H11ZfLBOMltKp/GbS 5ulx2b/QOIzGy6HBMQyPd1ma2X7YOb/KIIWnI472PlKHfTetqd9fJZ6xOdu7pMT7PlcfxMNYTDQsC DgnIWnA5Ljrnl7OVxXXQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pF7xD-005ScC-Hu; Tue, 10 Jan 2023 06:17:47 +0000 Received: from mail-vk1-xa34.google.com ([2607:f8b0:4864:20::a34]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pF7x9-005SZp-J0 for linux-riscv@lists.infradead.org; Tue, 10 Jan 2023 06:17:45 +0000 Received: by mail-vk1-xa34.google.com with SMTP id z190so5129520vka.4 for ; Mon, 09 Jan 2023 22:17:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=8CJjolKPlWAhG4ywZunllbYBE4Jf2JMO0vrAUHEfkjw=; b=UQV89TjBxsMqYrp4HaShVHvXh2Ce0AKWRK3bLEHVKP0BIHOPTp7pbUEbgPr4oDetb4 KI6ji9+60LnPfZDux/u8NGhltUa2rNdtUr5nNvs+NOpOusIkyWogcQl3Qi0b5cVfk/HD Gh2i9t7rior+q6Me/lCPQIn5QuyDXHM2x5ueSvLMq8tc9SKm/TYktR9hWEx+W9AiGxj5 QBuA6xIt4tgB+RWMMN8kJJ+9ox30KpjPsbLdlifphOkEGsHdICraWhSs5OrosNfPZgNW /+CCRvggxl47aOOHG//Hb/KGERPJPyYBOH7ZVvXsoEqhIZ3U0g+gLp7sWxQhP0YGc/g6 GoaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8CJjolKPlWAhG4ywZunllbYBE4Jf2JMO0vrAUHEfkjw=; b=fmlu//3LuxtTDTK7abd1cGmwgtqlNfuvEd/7OX4KG3VC8aOVjo738IwwkFyoZ5v1ZB ACEUrehFdIoAtQUKfJThqfdyMIircj7OPbiG3B5L98rVDjy1i51Wmf+obc1wsegeZ+kc LrFfKcIVoqfnFG/kIijKJqayjQyxGmLYduBu1xS0F6t1ii8pzR6P7PDvUt74S+iVrNrg +lLFXPd4wb7Y2q3X24mYqkHK63Mgr2vP0l+THhRc3RcfmFtl2Rt0UTgNIWeKxvxsWRb9 exG5WgBz7oP5pY0jyr7/SpsZ/gcJCDa1aK63A0SHJObWHD8mhU27huV7yAFW6UVcZF3E pd5Q== X-Gm-Message-State: AFqh2kqz/N7DuzxtJ3tVWn/0Jqq6APqNDY0onmPY3SZDVhT8nOdIMxgw tn2Ud+x7oJyfzWU9SyxRxVBSKTSAw+bUMGCSo2BtXQ== X-Google-Smtp-Source: AMrXdXv+UbPcEC0JIbuEUF6JQZeNC3GXIR+0tfBx5v6uXH+RRZFfR3XkgGAYKN5vIlEIZAfTFcYl9N5U9BOeOz/MzgA= X-Received: by 2002:a1f:948e:0:b0:3b7:6c07:c79c with SMTP id w136-20020a1f948e000000b003b76c07c79cmr8478522vkd.34.1673331460720; Mon, 09 Jan 2023 22:17:40 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Anup Patel Date: Tue, 10 Jan 2023 11:47:29 +0530 Message-ID: Subject: Re: Expected rdpmc behavior during context swtich and a RISC-V conundrum To: Atish Patra Cc: Peter Zijlstra , linux-perf-users@vger.kernel.org, "linux-kernel@vger.kernel.org List" , Mark Rutland , Arnaldo Carvalho de Melo , Alexander Shishkin , Will Deacon , Stephane Eranian , Andi Kleen , Palmer Dabbelt , Beeman Strong , Atish Patra , Kan Liang , linux-riscv , "open list:KERNEL VIRTUAL MACHINE FOR RISC-V (KVM/riscv)" X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230109_221743_675645_1EC5DBC9 X-CRM114-Status: GOOD ( 51.65 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org K2xpbnV4LXJpc2N2ICtrdm0tcmlzY3YKCk9uIFR1ZSwgSmFuIDEwLCAyMDIzIGF0IDE6MjYgQU0g QXRpc2ggUGF0cmEgPGF0aXNocEBhdGlzaHBhdHJhLm9yZz4gd3JvdGU6Cj4KPiBPbiBNb24sIEph biA5LCAyMDIzIGF0IDQ6NDEgQU0gUGV0ZXIgWmlqbHN0cmEgPHBldGVyekBpbmZyYWRlYWQub3Jn PiB3cm90ZToKPiA+Cj4gPiBPbiBUaHUsIEphbiAwNSwgMjAyMyBhdCAxMTo1OToyNEFNIC0wODAw LCBBdGlzaCBQYXRyYSB3cm90ZToKPiA+ID4gSGkgQWxsLAo+ID4gPiBUaGVyZSB3YXMgYSByZWNl bnQgdWFiaSB1cGRhdGVbMV0gZm9yIFJJU0MtViB0aGF0IGFsbG93cyB0aGUgdXNlcnMgdG8KPiA+ ID4gcmVhZCBjeWNsZSBhbmQgaW5zdHJ1Y3Rpb24gY291bnQgd2l0aG91dCBhbnkgY2hlY2tzLgo+ ID4gPiBXZSB0cmllZCB0byByZXN0cmljdCB0aGF0IGJlaGF2aW9yIHRvIGFkZHJlc3Mgc2VjdXJp dHkgY29uY2VybnMKPiA+ID4gZWFybGllciBidXQgaXQgcmVzdWx0ZWQgaW4gYnJlYWthZ2UgZm9y IHNvbWUgdXNlciBzcGFjZQo+ID4gPiBhcHBsaWNhdGlvbnNbMl0uCj4gPiA+IFRodXMsIHByZXZp b3VzIGJlaGF2aW9yIHdhcyByZXN0b3JlZCB3aGVyZSBhIHVzZXIgb24gUklTQy1WIHBsYXRmb3Jt cwo+ID4gPiBjYW4gZGlyZWN0bHkgcmVhZCBjeWNsZSBvciBpbnN0cnVjdGlvbiBjb3VudFszXS4K PiA+ID4KPiA+ID4gQ29tcGFyaXNvbiB3aXRoIG90aGVyIElTQXMgdy5yLnQgdXNlciBzcGFjZSBh Y2Nlc3Mgb2YgY291bnRlcnM6Cj4gPiA+IEFSTTY0Cj4gPiA+ICAgLS0gRW5hYmxlZC9EaXNhYmxl ZCB2aWEgKC9wcm9jL3N5cy9rZXJuZWwvcGVyZl91c2VyX2FjY2VzcykKPiA+ID4gICAtLSBPbmx5 IGZvciB0YXNrIGJvdW5kIGV2ZW50cyBjb25maWd1cmVkIHZpYSBwZXJmLgo+ID4gPgo+ID4gPiBY ODYKPiA+ID4gIC0tLSByZHBtYyBpbnN0cnVjdGlvbgo+ID4gPiAgLS0tIEVuYWJsZS9EaXNhYmxl IHZpYSDigJwvc3lzL2RldmljZXMvY3B1L3JkcG1j4oCdCj4gPiA+IC0tIEJlZm9yZSB2NC4wCj4g PiA+ICAtLSBhbnkgcHJvY2VzcyAoZXZlbiB3aXRob3V0IGFjdGl2ZSBwZXJmIGV2ZW50KSByZHBt Ywo+ID4gPiBBZnRlciB2NC4wCj4gPiA+IC0tIERlZmF1bHQgYmVoYXZpb3IgY2hhbmdlZCB0byBz dXBwb3J0IG9ubHkgYWN0aXZlIGV2ZW50cyBpbiBhCj4gPiA+IHByb2Nlc3PigJlzIGNvbnRleHQu Cj4gPiA+IC0tIENvbmZpZ3VyZWQgdGhyb3VnaCBwZXJmIHNpbWlsYXIgdG8gQVJNNjQKPiA+ID4g LS0gQ29udGludWUgdG8gbWFpbnRhaW4gYmFja3dhcmQgY29tcGF0aWJpbGl0eSBmb3IgdW5yZXN0 cmljdGVkIGFjY2Vzcwo+ID4gPiBieSB3cml0aW5nIDIgdG8g4oCcL3N5cy9kZXZpY2VzL2NwdS9y ZHBtY+KAnQo+ID4gPgo+ID4gPiBJTU8sIFJJU0MtViBzaG91bGQgb25seSBlbmFibGUgdXNlciBz cGFjZSBhY2Nlc3MgdGhyb3VnaCBwZXJmIHNpbWlsYXIKPiA+ID4gdG8gQVJNNjQgYW5kIHg4NiAo cG9zdCB2NC4wKS4KPiA+ID4gSG93ZXZlciwgd2UgZG8gaGF2ZSB0byBzdXBwb3J0IHRoZSBsZWdh Y3kgYmVoYXZpb3IgdG8gYXZvaWQKPiA+ID4gYXBwbGljYXRpb24gYnJlYWthZ2UuCj4gPiA+IEFz IHBlciBteSB1bmRlcnN0YW5kaW5nIGEgZGlyZWN0IHVzZXIgc3BhY2UgYWNjZXNzIGNhbiBsZWFk IHRvIHRoZQo+ID4gPiBmb2xsb3dpbmcgcHJvYmxlbXM6Cj4gPiA+Cj4gPiA+IDEpIFRoZXJlIGlz IG5vIGNvbnRleHQgc3dpdGNoIHN1cHBvcnQsIHNvIGNvdW50cyBmcm9tIG90aGVyIGNvbnRleHRz IGFyZSBleHBvc2VkCj4gPiA+IDIpIElmIGEgcGVyZiB1c2VyIGlzIGFsbG9jYXRlZCBvbmUgb2Yg dGhlc2UgY291bnRlcnMsIHRoZSBjb3VudGVyCj4gPiA+IHZhbHVlIHdpbGwgYmUgd3JpdHRlbgo+ ID4gPgo+ID4gPiBMb29raW5nIGF0IHRoZSB4ODYgY29kZSBhcyBpdCBjb250aW51ZXMgdG8gYWxs b3cgdGhlIGFib3ZlIGJlaGF2aW9yLAo+ID4gPiByZHBtY19hbHdheXNfYXZhaWxhYmxlX2tleSBp cyBlbmFibGVkIGluIHRoZSBhYm92ZSBjYXNlLiBIb3dldmVyLAo+ID4gPiBkdXJpbmcgdGhlIGNv bnRleHQgc3dpdGNoIChjcjRfdXBkYXRlX3BjZV9tbSkKPiA+ID4gb25seSBkaXJ0eSBjb3VudGVy cyBhcmUgY2xlYXJlZC4gSXQgb25seSBwcmV2ZW50cyBsZWFrYWdlIGZyb20gcGVyZgo+ID4gPiB0 YXNrIHRvIHJkcG1jIHRhc2suCj4gPiA+Cj4gPiA+IEhvdyBkb2VzIHRoZSBjb250ZXh0IHN3aXRj aCBvZiBjb3VudGVycyB3b3JrIGZvciB1c2VycyB3aG8gZW5hYmxlCj4gPiA+IHVucmVzdHJpY3Rl ZCBhY2Nlc3MgYnkgd3JpdGluZyAyIHRvIOKAnC9zeXMvZGV2aWNlcy9jcHUvcmRwbWPigJ0gPwo+ ID4gPiBPdGhlcndpc2UsIHJkcG1jIHVzZXJzIGxpa2VseSBnZXQgbm9pc2UgZnJvbSBvdGhlciBh cHBsaWNhdGlvbnMuIElzCj4gPiA+IHRoYXQgZXhwZWN0ZWQgPwo+ID4gPiBUaGlzIGNhbiBiZSBh IHNlY3VyaXR5IGNvbmNlcm4gYWxzbyB3aGVyZSBhIHJvZ3VlIHJkcG1jIHVzZXIKPiA+ID4gYXBw bGljYXRpb24gY2FuIG1vbml0b3Igb3RoZXIgY3JpdGljYWwgYXBwbGljYXRpb25zIHRvIGluaXRp YXRlIHNpZGUKPiA+ID4gY2hhbm5lbCBhdHRhY2suCj4gPiA+Cj4gPiA+IEFtIEkgbWlzc2luZyBz b21ldGhpbmc/IFBsZWFzZSBjb3JyZWN0IG15IHVuZGVyc3RhbmRpbmcgb2YgdGhlIHg4Ngo+ID4g PiBpbXBsZW1lbnRhdGlvbiBpZiBpdCBpcyB3cm9uZy4KPiA+Cj4gPiBTbyBvbiB4ODYgd2UgaGF2 ZSBSRFRTQyBhbmQgUkRQTUMgaW5zdHJ1Y3Rpb25zLiBSRFRTQyByZWFkcyB0aGUKPiA+IFRpbWUt U3RhbXAtQ291bnRlciB3aGljaCBpcyBhIGdsb2JhbGx5IHN5bmNocm9uaXplZCBtb25vdG9uaWMg aW5jcmVhc2luZwo+ID4gY291bnRlciBhdCBzb21lICdyYW5kb20nIHJhdGUgKGlkZWFsaXplZCwg ZG9uJ3QgYXNrKS4gVGhpcyB0aGluZyBpcyB1c2VkCj4gPiBmb3IgdGltZS1rZWVwaW5nIGV0Yy4u Cj4gPgo+ID4gQW5kIHRoZW4gdGhlcmUncyBSRFBNQyB3aGljaCAob3B0aW9uYWxseSkgYWxsb3dz IHJlYWRpbmcgdGhlIFBNVQo+ID4gY291bnRlcnMgd2hpY2ggYXJlIG5vcm1hbGx5IGRpc2FibGVk IGFuZCBhbGwgMC4KPiA+Cj4gPiBFdmVuIGlmIFJEUE1DIGlzIHVuY29uZGl0aW9uYWxseSBhbGxv d2VkIGZyb20gdXNlcnNwYWNlICh0aGUgMiBvcHRpb24KPiA+IHlvdSByZWZlciB0bykgdXNlcnNw YWNlIHdpbGwgb25seSBiZSBhYmxlIHRvIHJlYWQgdGhlc2UgMHMgdW5sZXNzCj4gPiBzb21lb25l IGFsc28gcHJvZ3JhbXMgdGhlIFBNVS4gTGludXggb25seSBzdXBwb3J0cyBhIHNpbmdsZSBtZWFu cyBvZgo+ID4gZG9pbmcgc286IHBlcmYgKHNvbWUgcGVvcGxlIHVzZSAvZGV2L21zciB0byBwb2tl IGRpcmVjdGx5IHRvIHRoZSBNU1JzCj4gPiBidXQgdGhleSBnZXQgdG8ga2VlcCBhbGwgcGllY2Vz KS4KPiA+Cj4KPiBJdCBtYWtlcyBzZW5zZSBub3cuIFRoYW5rcyEhCj4KPiBBRkFJSywgdGhlIC9k ZXYvbXNyIGludGVyZmFjZSBpcyBhbHNvIGFsbG93ZWQgZm9yIHJvb3QgdXNlcnMgb25seS4gU28g dGhhdAo+IGNvdmVycyB0aGUgc2VjdXJpdHkgY29uY2VybnMgSSB3YXMgYXNraW5nIGFib3V0Lgo+ Cj4gPiBSRFBNQyBpcyBvbmx5IHVzZWZ1bCBpZiB5b3UgcmVhZCBjb3VudGVycyB5b3Ugb3duIG9u IHlvdXJzZWxmIC0tIElPVwo+ID4gc2VsZm1vbml0b3JpbmcsIHVzaW5nIHRoZSBpbnRlcmZhY2Ug b3V0bGluZWQgaW4gdWFwaS9saW51eC9wZXJmX2V2ZW50cy5oCj4gPiBuZWFyIHN0cnVjdCBwZXJm X2V2ZW50X21tYXBfcGFnZS4KPiA+Cj4gPiBBbnkgb3RoZXIgdXNhZ2UgLS0geW91IGdldCB0byBr ZWVwIHRoZSBwaWVjZXMuCj4gPgo+ID4gQ2FuIHlvdSBvYnNlcnZlIHJhbmRvbSBvdGhlciBjb3Vu dGVycywgeWVzLCB1bmF2b2lkYWJseSBzby4gVGhlIHN5c2ZzCj4gPiBjb250cm9sIHlvdSBtZW50 aW9uIHdhcyBpbnN0aXR1dGVkIHRvIHJlc3RyaWN0IHRoaXMgc29tZXdoYXQuCj4gPgo+ID4gSWYg dGhlIFJJU0MtViBjb3VudGVycyBhcmUgZnVuZGFtZW50YWxseSB0aGUgUE1VIGNvdW50ZXJzIHRo YXQgbmVlZCB0bwo+ID4gYmUgcmVzZXQgdG8gdHJpZ2dlciBldmVudHMsIHRoZW4geW91J3ZlIG1h bmFnZWQgdG8gcGFpbnQgeW91cnNlbGYgaW50byBhCj4gPiB0aWdodCBzcG90IDovCj4gPgo+ID4g RWl0aGVyIHlvdSBtdXN0IGRpcy1hbGxvdyB1c2Vyc3BhY2UgYWNjZXNzIHRvIHRoZXNlIHRoaW5n cyAoYW5kIGJyZWFrCj4gPiB0aGVtKSBvciBsaW1pdCB0aGUgUE1VIHVzYWdlIC0tIGJvdGggb3B0 aW9ucyBzdWNrLgo+ID4KPiA+Cj4gPiBOb3csIEknbSB0aGlua2luZyB0aGF0IGVzcC4gc29tZXRo aW5nIGxpa2UgaW5zdHJ1Y3Rpb24gY291bnQgaXMgbm90Cj4gPiBzeW5jaHJvbml6ZWQgYmV0d2Vl biBjb3JlcyAoc2VlbXMgZnVuZGFtZW50YWxseSBpbXBvc3NpYmxlKSBhbmQgY2FuIG9ubHkKPiA+ IGJlIHJlYXNvbmFibHkgYmUgY29uc3VtZWQgKGFuZCBjb21wYXJlZCkgd2hlbiBzdHJpY3RseSBh ZmZpbmUgdG8gYQo+ID4gcGFydGljdWxhciBDUFUsIHlvdSBjYW4gYXJndWUgdGhhdCBhcHBsaWNh dGlvbnMgZG9pbmcgdGhpcyB3aXRob3V0IGFsc28KPiA+IHN0cmljdGx5IG1hbmFnaW5nIHRoZWly IGFmZmluaXR5IG1hc2sgYXJlIGJyb2tlbiBhbnl3YXkgYW5kIHRoZXJlZm9yZQo+ID4geW91ciBi cmVha2FnZSBpcyBub3QgaW4gZmFjdCBhIGJyZWFraW5nIHRoZW0gLS0geW91IGNhbid0IGJyZWFr Cj4gPiBzb21ldGhpbmcgdGhhdCdzIGFscmVhZHkgYnJva2VuLgo+ID4KPgo+IEkgdGhpbmsgbW9z dCBicm9rZW4gYXBwbGljYXRpb25zIHdlcmUgdXNpbmcgcmRjeWNsZSB0byBtZWFzdXJlIHRpbWUK PiB3aGljaCB3YXMgd3JvbmcgYW55d2F5cy4KPiBJdCBwcm9iYWJseSBoYXBwZW5lZCBiZWNhdXNl IHRoZXJlIHdhcyBubyAidGltZSIgQ1NSIGluIHRoZSBlYXJseQo+IGhhcmR3YXJlcy4gVGh1cywg dGhlIHJkdGltZSB3b3VsZAo+IHRyYXAgJiBlbXVsYXRlZCBieSB0aGUgZmlybXdhcmUgd2hpY2gg d2FzIHNsb3cuIFRoaXMgbGVhZCB0byB1c2VyCj4gc3BhY2UgYXBwbGljYXRpb24gdG8gdXNlIHJk Y3ljbGUgd2hpY2gKPiB3YXMgbm90IGNvcnJlY3QgZWl0aGVyLiBTbyB0aGUgZXhpc3RpbmcgYXBw bGljYXRpb25zIGFyZSBicm9rZW4gZm9yCj4gdXNpbmcgcmRjeWNsZSBhcyB3ZWxsLgo+Cj4gU2lu Y2UgYm90aCBjeWNsZSAmIGluc3RyZXQgYmVoYXZlIHNpbWlsYXJseSAoZml4ZWQgY291bnRlcnMp LCB0aGV5IGdldAo+IGVuYWJsZWQvZGlzYWJsZWQgdG9nZXRoZXIuCj4KPiA+Cj4gPiBBbnl3YXks IGdpdmVuIFJJU0MtViBiZWluZyBhIHZlcnkgeW91bmcgcGxhdGZvcm0sIEkgd291bGQgdHJ5IHJl YWxseQo+ID4gKnJlYWxseSogKlJFQUxMWSogaGFyZCB0byBzdG9tcCBvbiB0aGVzZSBhcHBsaWNh dGlvbnMgYW5kIGdldCB0aGVtIHRvCj4gPiBjaGFuZ2UgaW4gb3JkZXIgdG8gcmVjbGFpbSB0aGUg UE1VIHVzYWdlLgo+Cj4gWWVzLiBUaGFua3MgZm9yIHlvdXIgdmFsdWFibGUgaW5wdXQuCgpJIGFn cmVlIHdpdGggUGV0ZXIgWi4gV2UgaGFkIGEgc2ltaWxhciBkaXNjdXNzaW9uIGluIHRoZQpQZXJm b3JtYW5jZSBBbmFseXNpcyBTSUcgb2YgUklTQy1WIGludGVybmF0aW9uYWwgYXMgd2VsbC4KClRo aXMgYWxzbyBmb3JjZXMgS1ZNIChhbmQgb3RoZXIgaHlwZXJ2aXNvcnMpIHRvIHNhdmUtbi1yZXN0 b3JlCkNZQ0xFIGFuZCBJTlNUUlVDVElPTiBjb3VudGVycyBzbyB0aGF0IG9uZSBHdWVzdC9WTQpj YW4ndCBzZWUgY3ljbGUvaW5zdHJ1Y3Rpb24gY291bnRzIGZyb20gYW5vdGhlciBHdWVzdC9WTS4K Ck9ubHkgYSBmZXcgYXBwbGljYXRpb25zIGFyZSBhZmZlY3RlZCBhbmQgUklTQy1WIGVjb3N5c3Rl bQppcyB5b3VuZyBzbyBpdCBpcyBiZXR0ZXIgdG8gY2hhbmdlIHRoZXNlIGFwcGxpY2F0aW9ucyBy YXRoZXIKdGhhbiBtYWtpbmcgQ1lDTEUgYW5kIElOU1RSVUNUSU9OIGNvdW50ZXJzIGFzIHBhcnQK b2YgdUFCSS4KClJlZ2FyZHMsCkFudXAKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fCmxpbnV4LXJpc2N2IG1haWxpbmcgbGlzdApsaW51eC1yaXNjdkBsaXN0 cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGlu Zm8vbGludXgtcmlzY3YK