From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8B33C48BE5 for ; Sun, 20 Jun 2021 02:10:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B327D61206 for ; Sun, 20 Jun 2021 02:10:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230137AbhFTCMY (ORCPT ); Sat, 19 Jun 2021 22:12:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229591AbhFTCMW (ORCPT ); Sat, 19 Jun 2021 22:12:22 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6290DC061574; Sat, 19 Jun 2021 19:10:10 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id bb10-20020a17090b008ab029016eef083425so10012705pjb.5; Sat, 19 Jun 2021 19:10:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=zwJcdQn5NGse0S2evsaTo3yHirbFc9Aqtf3cv84Z/JQ=; b=mYHoj3IuaNFDkcH56IPxD9mcPgU+OLwdJFlgaGAr7xcwHJzBEGSKO6E8FOC80VNlfA UwW8r1xMCnXPzk2C4c6RjNU3BFfW12JbCddmt7z42K6NL2EXPKUdmJo/6/LKNaLaB8e3 bE/v8UsVz0ikwgfF+PKy5rw5h7a979ZxqBgGoLDdszh4XVjbeY5n2PtfRj8ygwjW/tLo I9xAPVfx7ov81mPIWYV/aW9KgkjP4gW0f8pKY1WGkT0Ng+Z/tHdndgdbZGUTfKCzitig n28ITlaO5xR2TEcEOxCxDc3YDNtlNMeamoltHBMHAAgp4LxuHaADctPJvA9GWVvycK1j 7EFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=zwJcdQn5NGse0S2evsaTo3yHirbFc9Aqtf3cv84Z/JQ=; b=TUZQcCxgesmEBFte3Vm8efBvdYMdls0gPDlGZYWV0YwH09Me2+5tzj6UQVIQjxSVLW WZ6BVDViLN/9JD5Ln2FXv5BHpSUxRlEDbrx022OKiy3BZYSofOsGalxdQHBLJGKFo4ij 3K80fI29I///JjpdhEHTjbv1Wg6rNzohSA0/ddlp6X1yhszk5XH7+XPsj0Rxg2NL85vw PwW+Z+lFNgNvJ+vUoRzQvJRjPfzuZPWpt7wM5CqWrTG+pr8w+NQMMwMOLYqcooICkda4 a601iYEqXONQD2Tzl9/1rxdP0V238K2ByJ4Lzso1b5pMl1jrQm/uAAY/47f5+FF9bIwu SiRw== X-Gm-Message-State: AOAM531tR6moJ8jp4GChtWbs145VV1lI52nMHAZ8o5QHXs60zU04qVjl 1Y+BAPvbU7uQoeHrHiJh8UA= X-Google-Smtp-Source: ABdhPJyqXUsiG6ibTfaxcmtOeTqG/JCNSRGz0ZCXX34hSVqlrCEbCSyQsN2c/6irjDf0xtAeEbeZiA== X-Received: by 2002:a17:90b:3142:: with SMTP id ip2mr29424960pjb.63.1624155009968; Sat, 19 Jun 2021 19:10:09 -0700 (PDT) Received: from localhost (60-242-147-73.tpgi.com.au. [60.242.147.73]) by smtp.gmail.com with ESMTPSA id ls13sm11268684pjb.23.2021.06.19.19.10.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Jun 2021 19:10:09 -0700 (PDT) Date: Sun, 20 Jun 2021 12:10:04 +1000 From: Nicholas Piggin Subject: Re: [PATCH 8/8] membarrier: Rewrite sync_core_before_usermode() and improve documentation To: Andy Lutomirski , Mathieu Desnoyers Cc: Andrew Morton , Benjamin Herrenschmidt , Catalin Marinas , Dave Hansen , linux-arm-kernel , Linux Kernel Mailing List , linux-mm , linuxppc-dev , Michael Ellerman , Paul Mackerras , "Peter Zijlstra (Intel)" , stable , Will Deacon , the arch/x86 maintainers References: <07a8b963002cb955b7516e61bad19514a3acaa82.1623813516.git.luto@kernel.org> <827549827.10547.1623941277868.JavaMail.zimbra@efficios.com> <26196903-4aee-33c4-bed8-8bf8c7b46793@kernel.org> <593983567.12619.1624033908849.JavaMail.zimbra@efficios.com> <1d617df2-57fa-4953-9c75-6de3909a6f14@www.fastmail.com> <639092151.13266.1624046981084.JavaMail.zimbra@efficios.com> <1624080924.z61zvzi4cq.astroid@bobo.none> <2f9e52eb-0105-4bc6-a903-f4ecbfc9b999@www.fastmail.com> In-Reply-To: <2f9e52eb-0105-4bc6-a903-f4ecbfc9b999@www.fastmail.com> MIME-Version: 1.0 Message-Id: <1624154254.9g8ebnq8vg.astroid@bobo.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Excerpts from Andy Lutomirski's message of June 20, 2021 1:50 am: >=20 >=20 > On Fri, Jun 18, 2021, at 11:02 PM, Nicholas Piggin wrote: >> Excerpts from Mathieu Desnoyers's message of June 19, 2021 6:09 am: >> > ----- On Jun 18, 2021, at 3:58 PM, Andy Lutomirski luto@kernel.org wro= te: >> >=20 >> >> On Fri, Jun 18, 2021, at 9:31 AM, Mathieu Desnoyers wrote: >> >>> ----- On Jun 17, 2021, at 8:12 PM, Andy Lutomirski luto@kernel.org w= rote: >> >>>=20 >> >>> > On 6/17/21 7:47 AM, Mathieu Desnoyers wrote: >> >>> >=20 >> >>> >> Please change back this #ifndef / #else / #endif within function = for >> >>> >>=20 >> >>> >> if (!IS_ENABLED(CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE)) { >> >>> >> ... >> >>> >> } else { >> >>> >> ... >> >>> >> } >> >>> >>=20 >> >>> >> I don't think mixing up preprocessor and code logic makes it more= readable. >> >>> >=20 >> >>> > I agree, but I don't know how to make the result work well. >> >>> > membarrier_sync_core_before_usermode() isn't defined in the !IS_EN= ABLED >> >>> > case, so either I need to fake up a definition or use #ifdef. >> >>> >=20 >> >>> > If I faked up a definition, I would want to assert, at build time,= that >> >>> > it isn't called. I don't think we can do: >> >>> >=20 >> >>> > static void membarrier_sync_core_before_usermode() >> >>> > { >> >>> > BUILD_BUG_IF_REACHABLE(); >> >>> > } >> >>>=20 >> >>> Let's look at the context here: >> >>>=20 >> >>> static void ipi_sync_core(void *info) >> >>> { >> >>> [....] >> >>> membarrier_sync_core_before_usermode() >> >>> } >> >>>=20 >> >>> ^ this can be within #ifdef / #endif >> >>>=20 >> >>> static int membarrier_private_expedited(int flags, int cpu_id) >> >>> [...] >> >>> if (!IS_ENABLED(CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE)= ) >> >>> return -EINVAL; >> >>> if (!(atomic_read(&mm->membarrier_state) & >> >>> MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_R= EADY)) >> >>> return -EPERM; >> >>> ipi_func =3D ipi_sync_core; >> >>>=20 >> >>> All we need to make the line above work is to define an empty ipi_sy= nc_core >> >>> function in the #else case after the ipi_sync_core() function defini= tion. >> >>>=20 >> >>> Or am I missing your point ? >> >>=20 >> >> Maybe? >> >>=20 >> >> My objection is that an empty ipi_sync_core is a lie =E2=80=94 it doe= sn=E2=80=99t sync the core. >> >> I would be fine with that if I could have the compiler statically ver= ify that >> >> it=E2=80=99s not called, but I=E2=80=99m uncomfortable having it ther= e if the implementation is >> >> actively incorrect. >> >=20 >> > I see. Another approach would be to implement a "setter" function to p= opulate >> > "ipi_func". That setter function would return -EINVAL in its #ifndef C= ONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE >> > implementation. >>=20 >> I still don't get the problem with my suggestion. Sure the=20 >> ipi is a "lie", but it doesn't get used. That's how a lot of >> ifdef folding works out. E.g., >>=20 >> diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c >> index b5add64d9698..54cb32d064af 100644 >> --- a/kernel/sched/membarrier.c >> +++ b/kernel/sched/membarrier.c >> @@ -5,6 +5,15 @@ >> * membarrier system call >> */ >> #include "sched.h" >> +#ifdef CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE >> +#include >> +#else >> +static inline void membarrier_sync_core_before_usermode(void) >> +{ >> + compiletime_assert(0, "architecture does not implement=20 >> membarrier_sync_core_before_usermode"); >> +} >> + >=20 > With the assert there, I=E2=80=99m fine with this. Let me see if the resu= lt builds. It had better, because compiletime_assert already relies on a similar=20 level of code elimination to work. I think it's fine to use for now, but it may not be quite the the=20 logically correct primitive if we want to be really clean, because a=20 valid compiletime_assert implementation should be able to fire even for=20 code that is never linked. We would want something like to be clean=20 IMO: #ifdef CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE #include #else extern void membarrier_sync_core_before_usermode(void) __compiletime_error(= "architecture does not implement membarrier_sync_core_before_usermode"); #endif However that does not have the ifdef for optimising compile so AFAIKS it=20 could break with a false positive in some cases. Something like compiletime_assert_not_called("msg") that either compiles to a noop or a __compiletime_error depending on __OPTIMIZE__ would be=20 the way to go IMO. I don't know if anything exists that fits, but it's certainly not a unique thing in the kernel so I may not be looking hard enough. Thanks, Nick From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.5 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACD01C2B9F4 for ; Sun, 20 Jun 2021 02:10:45 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 035AA60FDC for ; Sun, 20 Jun 2021 02:10:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 035AA60FDC Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4G6x2r0xxQz3c3N for ; Sun, 20 Jun 2021 12:10:44 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=mYHoj3Iu; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=gmail.com (client-ip=2607:f8b0:4864:20::636; helo=mail-pl1-x636.google.com; envelope-from=npiggin@gmail.com; receiver=) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20161025 header.b=mYHoj3Iu; dkim-atps=neutral Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4G6x2J0V1hz301t for ; Sun, 20 Jun 2021 12:10:15 +1000 (AEST) Received: by mail-pl1-x636.google.com with SMTP id i4so2932276plt.12 for ; Sat, 19 Jun 2021 19:10:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=zwJcdQn5NGse0S2evsaTo3yHirbFc9Aqtf3cv84Z/JQ=; b=mYHoj3IuaNFDkcH56IPxD9mcPgU+OLwdJFlgaGAr7xcwHJzBEGSKO6E8FOC80VNlfA UwW8r1xMCnXPzk2C4c6RjNU3BFfW12JbCddmt7z42K6NL2EXPKUdmJo/6/LKNaLaB8e3 bE/v8UsVz0ikwgfF+PKy5rw5h7a979ZxqBgGoLDdszh4XVjbeY5n2PtfRj8ygwjW/tLo I9xAPVfx7ov81mPIWYV/aW9KgkjP4gW0f8pKY1WGkT0Ng+Z/tHdndgdbZGUTfKCzitig n28ITlaO5xR2TEcEOxCxDc3YDNtlNMeamoltHBMHAAgp4LxuHaADctPJvA9GWVvycK1j 7EFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=zwJcdQn5NGse0S2evsaTo3yHirbFc9Aqtf3cv84Z/JQ=; b=ZWfkXHTJdGOklGHVnplza2TOlOTYUCWb54C8sOqjgLRv1MrHnjuSBT9ZDXUhAndUOO N1Wa5A0ZJw2E/X6Yh1BCXVe41MpypXwEOQvVlDxDDw/+imq1vLXR/XX31qB78+5EkVeL aRNYftg+52qb3SO68Yld7WEhBhIZAkQxDW4AK7+HKovu0DVEZqwhP76N2qOOy5E923jo NNqLVc3AWl4NjygpliDoKxbnJDv5bxZMn0qCufQH3bt838G64FkZ2VUIwGFuwqa7PcwT oh0x9Pm8SwuAKiP7cwKeR+evnOIi5v+6Oev2iti3MpQUbdpg6p0UrZjh8T+hf91eEP3c ijaw== X-Gm-Message-State: AOAM530/gZXHfxcy0iPzIB+fFX/Ntbg62i6AIAbDa5rCH/PN2SOcCJDQ jP3+bST8RBaaFx70AC0qMoI= X-Google-Smtp-Source: ABdhPJyqXUsiG6ibTfaxcmtOeTqG/JCNSRGz0ZCXX34hSVqlrCEbCSyQsN2c/6irjDf0xtAeEbeZiA== X-Received: by 2002:a17:90b:3142:: with SMTP id ip2mr29424960pjb.63.1624155009968; Sat, 19 Jun 2021 19:10:09 -0700 (PDT) Received: from localhost (60-242-147-73.tpgi.com.au. [60.242.147.73]) by smtp.gmail.com with ESMTPSA id ls13sm11268684pjb.23.2021.06.19.19.10.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Jun 2021 19:10:09 -0700 (PDT) Date: Sun, 20 Jun 2021 12:10:04 +1000 From: Nicholas Piggin Subject: Re: [PATCH 8/8] membarrier: Rewrite sync_core_before_usermode() and improve documentation To: Andy Lutomirski , Mathieu Desnoyers References: <07a8b963002cb955b7516e61bad19514a3acaa82.1623813516.git.luto@kernel.org> <827549827.10547.1623941277868.JavaMail.zimbra@efficios.com> <26196903-4aee-33c4-bed8-8bf8c7b46793@kernel.org> <593983567.12619.1624033908849.JavaMail.zimbra@efficios.com> <1d617df2-57fa-4953-9c75-6de3909a6f14@www.fastmail.com> <639092151.13266.1624046981084.JavaMail.zimbra@efficios.com> <1624080924.z61zvzi4cq.astroid@bobo.none> <2f9e52eb-0105-4bc6-a903-f4ecbfc9b999@www.fastmail.com> In-Reply-To: <2f9e52eb-0105-4bc6-a903-f4ecbfc9b999@www.fastmail.com> MIME-Version: 1.0 Message-Id: <1624154254.9g8ebnq8vg.astroid@bobo.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Will Deacon , linux-mm , "Peter Zijlstra \(Intel\)" , the arch/x86 maintainers , Linux Kernel Mailing List , stable , Dave Hansen , Paul Mackerras , Catalin Marinas , Andrew Morton , linuxppc-dev , linux-arm-kernel Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Excerpts from Andy Lutomirski's message of June 20, 2021 1:50 am: >=20 >=20 > On Fri, Jun 18, 2021, at 11:02 PM, Nicholas Piggin wrote: >> Excerpts from Mathieu Desnoyers's message of June 19, 2021 6:09 am: >> > ----- On Jun 18, 2021, at 3:58 PM, Andy Lutomirski luto@kernel.org wro= te: >> >=20 >> >> On Fri, Jun 18, 2021, at 9:31 AM, Mathieu Desnoyers wrote: >> >>> ----- On Jun 17, 2021, at 8:12 PM, Andy Lutomirski luto@kernel.org w= rote: >> >>>=20 >> >>> > On 6/17/21 7:47 AM, Mathieu Desnoyers wrote: >> >>> >=20 >> >>> >> Please change back this #ifndef / #else / #endif within function = for >> >>> >>=20 >> >>> >> if (!IS_ENABLED(CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE)) { >> >>> >> ... >> >>> >> } else { >> >>> >> ... >> >>> >> } >> >>> >>=20 >> >>> >> I don't think mixing up preprocessor and code logic makes it more= readable. >> >>> >=20 >> >>> > I agree, but I don't know how to make the result work well. >> >>> > membarrier_sync_core_before_usermode() isn't defined in the !IS_EN= ABLED >> >>> > case, so either I need to fake up a definition or use #ifdef. >> >>> >=20 >> >>> > If I faked up a definition, I would want to assert, at build time,= that >> >>> > it isn't called. I don't think we can do: >> >>> >=20 >> >>> > static void membarrier_sync_core_before_usermode() >> >>> > { >> >>> > BUILD_BUG_IF_REACHABLE(); >> >>> > } >> >>>=20 >> >>> Let's look at the context here: >> >>>=20 >> >>> static void ipi_sync_core(void *info) >> >>> { >> >>> [....] >> >>> membarrier_sync_core_before_usermode() >> >>> } >> >>>=20 >> >>> ^ this can be within #ifdef / #endif >> >>>=20 >> >>> static int membarrier_private_expedited(int flags, int cpu_id) >> >>> [...] >> >>> if (!IS_ENABLED(CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE)= ) >> >>> return -EINVAL; >> >>> if (!(atomic_read(&mm->membarrier_state) & >> >>> MEMBARRIER_STATE_PRIVATE_EXPEDITED_SYNC_CORE_R= EADY)) >> >>> return -EPERM; >> >>> ipi_func =3D ipi_sync_core; >> >>>=20 >> >>> All we need to make the line above work is to define an empty ipi_sy= nc_core >> >>> function in the #else case after the ipi_sync_core() function defini= tion. >> >>>=20 >> >>> Or am I missing your point ? >> >>=20 >> >> Maybe? >> >>=20 >> >> My objection is that an empty ipi_sync_core is a lie =E2=80=94 it doe= sn=E2=80=99t sync the core. >> >> I would be fine with that if I could have the compiler statically ver= ify that >> >> it=E2=80=99s not called, but I=E2=80=99m uncomfortable having it ther= e if the implementation is >> >> actively incorrect. >> >=20 >> > I see. Another approach would be to implement a "setter" function to p= opulate >> > "ipi_func". That setter function would return -EINVAL in its #ifndef C= ONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE >> > implementation. >>=20 >> I still don't get the problem with my suggestion. Sure the=20 >> ipi is a "lie", but it doesn't get used. That's how a lot of >> ifdef folding works out. E.g., >>=20 >> diff --git a/kernel/sched/membarrier.c b/kernel/sched/membarrier.c >> index b5add64d9698..54cb32d064af 100644 >> --- a/kernel/sched/membarrier.c >> +++ b/kernel/sched/membarrier.c >> @@ -5,6 +5,15 @@ >> * membarrier system call >> */ >> #include "sched.h" >> +#ifdef CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE >> +#include >> +#else >> +static inline void membarrier_sync_core_before_usermode(void) >> +{ >> + compiletime_assert(0, "architecture does not implement=20 >> membarrier_sync_core_before_usermode"); >> +} >> + >=20 > With the assert there, I=E2=80=99m fine with this. Let me see if the resu= lt builds. It had better, because compiletime_assert already relies on a similar=20 level of code elimination to work. I think it's fine to use for now, but it may not be quite the the=20 logically correct primitive if we want to be really clean, because a=20 valid compiletime_assert implementation should be able to fire even for=20 code that is never linked. We would want something like to be clean=20 IMO: #ifdef CONFIG_ARCH_HAS_MEMBARRIER_SYNC_CORE #include #else extern void membarrier_sync_core_before_usermode(void) __compiletime_error(= "architecture does not implement membarrier_sync_core_before_usermode"); #endif However that does not have the ifdef for optimising compile so AFAIKS it=20 could break with a false positive in some cases. Something like compiletime_assert_not_called("msg") that either compiles to a noop or a __compiletime_error depending on __OPTIMIZE__ would be=20 the way to go IMO. I don't know if anything exists that fits, but it's certainly not a unique thing in the kernel so I may not be looking hard enough. Thanks, Nick From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AD8BC2B9F4 for ; Sun, 20 Jun 2021 02:12:33 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B80FD6101A for ; Sun, 20 Jun 2021 02:12:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B80FD6101A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Message-Id:MIME-Version:In-Reply-To: References:Cc:To:Subject:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dvFLI1Fv8WRWioyY0x6l05Ltz5vBzDVAYXKg2JDES0U=; b=WrlRhqu/WrK/D1 IMRtP+3UAcPbUb7eMrgLuLJ+FGv+GFRjGco+TYD6mWQnEu/2Lff+O3iklYra+m3K5YuI1JDCt3m3u JF6qdZoGJdrrIWenWkynOLNK4UmmimgOa/ifADZSw5cz0hqkaPJvO9GNBTLJRYQokBioO9lg/4XVn l2spoJlqyxEWXPYI2/Mt5q6VACz+gvXL3/uVshV+waTAvbR+Ss5IunX+dvFP8E/qx6D6PDfhsEGn8 hAGUynqGM4+ViFFrgmWNa4xpSyiP9/KYs8WZ6p06sSId/wtpUILGj677tla6wCoUPAL7IEKnHS+CE H3h/sV1J52W9U1oQFa/Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lumuc-000JFB-OD; Sun, 20 Jun 2021 02:10:14 +0000 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lumuY-000JEG-Rx for linux-arm-kernel@lists.infradead.org; Sun, 20 Jun 2021 02:10:12 +0000 Received: by mail-pj1-x102b.google.com with SMTP id g24so7863647pji.4 for ; Sat, 19 Jun 2021 19:10:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:subject:to:cc:references:in-reply-to:mime-version :message-id:content-transfer-encoding; bh=zwJcdQn5NGse0S2evsaTo3yHirbFc9Aqtf3cv84Z/JQ=; b=mYHoj3IuaNFDkcH56IPxD9mcPgU+OLwdJFlgaGAr7xcwHJzBEGSKO6E8FOC80VNlfA UwW8r1xMCnXPzk2C4c6RjNU3BFfW12JbCddmt7z42K6NL2EXPKUdmJo/6/LKNaLaB8e3 bE/v8UsVz0ikwgfF+PKy5rw5h7a979ZxqBgGoLDdszh4XVjbeY5n2PtfRj8ygwjW/tLo I9xAPVfx7ov81mPIWYV/aW9KgkjP4gW0f8pKY1WGkT0Ng+Z/tHdndgdbZGUTfKCzitig n28ITlaO5xR2TEcEOxCxDc3YDNtlNMeamoltHBMHAAgp4LxuHaADctPJvA9GWVvycK1j 7EFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:subject:to:cc:references:in-reply-to :mime-version:message-id:content-transfer-encoding; bh=zwJcdQn5NGse0S2evsaTo3yHirbFc9Aqtf3cv84Z/JQ=; b=Wonzsd4ZL49DbrfOTO8kDkSqtzMbJhwMA5d5GmrFXHkLOMwutDWIHnbMBJoA3EgTbF ekqjlf8m5uUGiYaCfBffhojy5LpfF60kSSMJUJo+FnP26xzxMY0hgCWaGEjNwr7gWlyB 3X7Wz1UecObObheuykiMbucf8nS0bNVAiRhB8pAFoLbIOBa4uLWFEB1qkETD2mQRspZU jMHiMQiTP5QYlI3cIoOlPm1PIXLA3M0b6XdGhrRQifX0sWn7uqn+qBgIty2adGi+FJBj 96kFgJB/8sgXHlO70DndN11+nN4r4VWKzVHvGGwgALKzwMLsFgptC8Yq1BlE3/9KTNKK +4IA== X-Gm-Message-State: AOAM5317kn7hll9xSUgPdHbWV6fZz+xF3/sQiJ0zHcz+tXQP/PBZLhv0 6xGqf1rSZnksgGLeldE8iis= X-Google-Smtp-Source: ABdhPJyqXUsiG6ibTfaxcmtOeTqG/JCNSRGz0ZCXX34hSVqlrCEbCSyQsN2c/6irjDf0xtAeEbeZiA== X-Received: by 2002:a17:90b:3142:: with SMTP id ip2mr29424960pjb.63.1624155009968; Sat, 19 Jun 2021 19:10:09 -0700 (PDT) Received: from localhost (60-242-147-73.tpgi.com.au. [60.242.147.73]) by smtp.gmail.com with ESMTPSA id ls13sm11268684pjb.23.2021.06.19.19.10.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 19 Jun 2021 19:10:09 -0700 (PDT) Date: Sun, 20 Jun 2021 12:10:04 +1000 From: Nicholas Piggin Subject: Re: [PATCH 8/8] membarrier: Rewrite sync_core_before_usermode() and improve documentation To: Andy Lutomirski , Mathieu Desnoyers Cc: Andrew Morton , Benjamin Herrenschmidt , Catalin Marinas , Dave Hansen , linux-arm-kernel , Linux Kernel Mailing List , linux-mm , linuxppc-dev , Michael Ellerman , Paul Mackerras , "Peter Zijlstra (Intel)" , stable , Will Deacon , the arch/x86 maintainers References: <07a8b963002cb955b7516e61bad19514a3acaa82.1623813516.git.luto@kernel.org> <827549827.10547.1623941277868.JavaMail.zimbra@efficios.com> <26196903-4aee-33c4-bed8-8bf8c7b46793@kernel.org> <593983567.12619.1624033908849.JavaMail.zimbra@efficios.com> <1d617df2-57fa-4953-9c75-6de3909a6f14@www.fastmail.com> <639092151.13266.1624046981084.JavaMail.zimbra@efficios.com> <1624080924.z61zvzi4cq.astroid@bobo.none> <2f9e52eb-0105-4bc6-a903-f4ecbfc9b999@www.fastmail.com> In-Reply-To: <2f9e52eb-0105-4bc6-a903-f4ecbfc9b999@www.fastmail.com> MIME-Version: 1.0 Message-Id: <1624154254.9g8ebnq8vg.astroid@bobo.none> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210619_191010_981926_5C566D33 X-CRM114-Status: GOOD ( 28.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org RXhjZXJwdHMgZnJvbSBBbmR5IEx1dG9taXJza2kncyBtZXNzYWdlIG9mIEp1bmUgMjAsIDIwMjEg MTo1MCBhbToKPiAKPiAKPiBPbiBGcmksIEp1biAxOCwgMjAyMSwgYXQgMTE6MDIgUE0sIE5pY2hv bGFzIFBpZ2dpbiB3cm90ZToKPj4gRXhjZXJwdHMgZnJvbSBNYXRoaWV1IERlc25veWVycydzIG1l c3NhZ2Ugb2YgSnVuZSAxOSwgMjAyMSA2OjA5IGFtOgo+PiA+IC0tLS0tIE9uIEp1biAxOCwgMjAy MSwgYXQgMzo1OCBQTSwgQW5keSBMdXRvbWlyc2tpIGx1dG9Aa2VybmVsLm9yZyB3cm90ZToKPj4g PiAKPj4gPj4gT24gRnJpLCBKdW4gMTgsIDIwMjEsIGF0IDk6MzEgQU0sIE1hdGhpZXUgRGVzbm95 ZXJzIHdyb3RlOgo+PiA+Pj4gLS0tLS0gT24gSnVuIDE3LCAyMDIxLCBhdCA4OjEyIFBNLCBBbmR5 IEx1dG9taXJza2kgbHV0b0BrZXJuZWwub3JnIHdyb3RlOgo+PiA+Pj4gCj4+ID4+PiA+IE9uIDYv MTcvMjEgNzo0NyBBTSwgTWF0aGlldSBEZXNub3llcnMgd3JvdGU6Cj4+ID4+PiA+IAo+PiA+Pj4g Pj4gUGxlYXNlIGNoYW5nZSBiYWNrIHRoaXMgI2lmbmRlZiAvICNlbHNlIC8gI2VuZGlmIHdpdGhp biBmdW5jdGlvbiBmb3IKPj4gPj4+ID4+IAo+PiA+Pj4gPj4gaWYgKCFJU19FTkFCTEVEKENPTkZJ R19BUkNIX0hBU19NRU1CQVJSSUVSX1NZTkNfQ09SRSkpIHsKPj4gPj4+ID4+ICAgLi4uCj4+ID4+ PiA+PiB9IGVsc2Ugewo+PiA+Pj4gPj4gICAuLi4KPj4gPj4+ID4+IH0KPj4gPj4+ID4+IAo+PiA+ Pj4gPj4gSSBkb24ndCB0aGluayBtaXhpbmcgdXAgcHJlcHJvY2Vzc29yIGFuZCBjb2RlIGxvZ2lj IG1ha2VzIGl0IG1vcmUgcmVhZGFibGUuCj4+ID4+PiA+IAo+PiA+Pj4gPiBJIGFncmVlLCBidXQg SSBkb24ndCBrbm93IGhvdyB0byBtYWtlIHRoZSByZXN1bHQgd29yayB3ZWxsLgo+PiA+Pj4gPiBt ZW1iYXJyaWVyX3N5bmNfY29yZV9iZWZvcmVfdXNlcm1vZGUoKSBpc24ndCBkZWZpbmVkIGluIHRo ZSAhSVNfRU5BQkxFRAo+PiA+Pj4gPiBjYXNlLCBzbyBlaXRoZXIgSSBuZWVkIHRvIGZha2UgdXAg YSBkZWZpbml0aW9uIG9yIHVzZSAjaWZkZWYuCj4+ID4+PiA+IAo+PiA+Pj4gPiBJZiBJIGZha2Vk IHVwIGEgZGVmaW5pdGlvbiwgSSB3b3VsZCB3YW50IHRvIGFzc2VydCwgYXQgYnVpbGQgdGltZSwg dGhhdAo+PiA+Pj4gPiBpdCBpc24ndCBjYWxsZWQuICBJIGRvbid0IHRoaW5rIHdlIGNhbiBkbzoK Pj4gPj4+ID4gCj4+ID4+PiA+IHN0YXRpYyB2b2lkIG1lbWJhcnJpZXJfc3luY19jb3JlX2JlZm9y ZV91c2VybW9kZSgpCj4+ID4+PiA+IHsKPj4gPj4+ID4gICAgQlVJTERfQlVHX0lGX1JFQUNIQUJM RSgpOwo+PiA+Pj4gPiB9Cj4+ID4+PiAKPj4gPj4+IExldCdzIGxvb2sgYXQgdGhlIGNvbnRleHQg aGVyZToKPj4gPj4+IAo+PiA+Pj4gc3RhdGljIHZvaWQgaXBpX3N5bmNfY29yZSh2b2lkICppbmZv KQo+PiA+Pj4gewo+PiA+Pj4gICAgIFsuLi4uXQo+PiA+Pj4gICAgIG1lbWJhcnJpZXJfc3luY19j b3JlX2JlZm9yZV91c2VybW9kZSgpCj4+ID4+PiB9Cj4+ID4+PiAKPj4gPj4+IF4gdGhpcyBjYW4g YmUgd2l0aGluICNpZmRlZiAvICNlbmRpZgo+PiA+Pj4gCj4+ID4+PiBzdGF0aWMgaW50IG1lbWJh cnJpZXJfcHJpdmF0ZV9leHBlZGl0ZWQoaW50IGZsYWdzLCBpbnQgY3B1X2lkKQo+PiA+Pj4gWy4u Ll0KPj4gPj4+ICAgICAgICAgICAgICAgIGlmICghSVNfRU5BQkxFRChDT05GSUdfQVJDSF9IQVNf TUVNQkFSUklFUl9TWU5DX0NPUkUpKQo+PiA+Pj4gICAgICAgICAgICAgICAgICAgICAgICAgcmV0 dXJuIC1FSU5WQUw7Cj4+ID4+PiAgICAgICAgICAgICAgICAgaWYgKCEoYXRvbWljX3JlYWQoJm1t LT5tZW1iYXJyaWVyX3N0YXRlKSAmCj4+ID4+PiAgICAgICAgICAgICAgICAgICAgICAgTUVNQkFS UklFUl9TVEFURV9QUklWQVRFX0VYUEVESVRFRF9TWU5DX0NPUkVfUkVBRFkpKQo+PiA+Pj4gICAg ICAgICAgICAgICAgICAgICAgICAgcmV0dXJuIC1FUEVSTTsKPj4gPj4+ICAgICAgICAgICAgICAg ICBpcGlfZnVuYyA9IGlwaV9zeW5jX2NvcmU7Cj4+ID4+PiAKPj4gPj4+IEFsbCB3ZSBuZWVkIHRv IG1ha2UgdGhlIGxpbmUgYWJvdmUgd29yayBpcyB0byBkZWZpbmUgYW4gZW1wdHkgaXBpX3N5bmNf Y29yZQo+PiA+Pj4gZnVuY3Rpb24gaW4gdGhlICNlbHNlIGNhc2UgYWZ0ZXIgdGhlIGlwaV9zeW5j X2NvcmUoKSBmdW5jdGlvbiBkZWZpbml0aW9uLgo+PiA+Pj4gCj4+ID4+PiBPciBhbSBJIG1pc3Np bmcgeW91ciBwb2ludCA/Cj4+ID4+IAo+PiA+PiBNYXliZT8KPj4gPj4gCj4+ID4+IE15IG9iamVj dGlvbiBpcyB0aGF0IGFuIGVtcHR5IGlwaV9zeW5jX2NvcmUgaXMgYSBsaWUg4oCUIGl0IGRvZXNu 4oCZdCBzeW5jIHRoZSBjb3JlLgo+PiA+PiBJIHdvdWxkIGJlIGZpbmUgd2l0aCB0aGF0IGlmIEkg Y291bGQgaGF2ZSB0aGUgY29tcGlsZXIgc3RhdGljYWxseSB2ZXJpZnkgdGhhdAo+PiA+PiBpdOKA mXMgbm90IGNhbGxlZCwgYnV0IEnigJltIHVuY29tZm9ydGFibGUgaGF2aW5nIGl0IHRoZXJlIGlm IHRoZSBpbXBsZW1lbnRhdGlvbiBpcwo+PiA+PiBhY3RpdmVseSBpbmNvcnJlY3QuCj4+ID4gCj4+ ID4gSSBzZWUuIEFub3RoZXIgYXBwcm9hY2ggd291bGQgYmUgdG8gaW1wbGVtZW50IGEgInNldHRl ciIgZnVuY3Rpb24gdG8gcG9wdWxhdGUKPj4gPiAiaXBpX2Z1bmMiLiBUaGF0IHNldHRlciBmdW5j dGlvbiB3b3VsZCByZXR1cm4gLUVJTlZBTCBpbiBpdHMgI2lmbmRlZiBDT05GSUdfQVJDSF9IQVNf TUVNQkFSUklFUl9TWU5DX0NPUkUKPj4gPiBpbXBsZW1lbnRhdGlvbi4KPj4gCj4+IEkgc3RpbGwg ZG9uJ3QgZ2V0IHRoZSBwcm9ibGVtIHdpdGggbXkgc3VnZ2VzdGlvbi4gU3VyZSB0aGUgCj4+IGlw aSBpcyBhICJsaWUiLCBidXQgaXQgZG9lc24ndCBnZXQgdXNlZC4gVGhhdCdzIGhvdyBhIGxvdCBv Zgo+PiBpZmRlZiBmb2xkaW5nIHdvcmtzIG91dC4gRS5nLiwKPj4gCj4+IGRpZmYgLS1naXQgYS9r ZXJuZWwvc2NoZWQvbWVtYmFycmllci5jIGIva2VybmVsL3NjaGVkL21lbWJhcnJpZXIuYwo+PiBp bmRleCBiNWFkZDY0ZDk2OTguLjU0Y2IzMmQwNjRhZiAxMDA2NDQKPj4gLS0tIGEva2VybmVsL3Nj aGVkL21lbWJhcnJpZXIuYwo+PiArKysgYi9rZXJuZWwvc2NoZWQvbWVtYmFycmllci5jCj4+IEBA IC01LDYgKzUsMTUgQEAKPj4gICAqIG1lbWJhcnJpZXIgc3lzdGVtIGNhbGwKPj4gICAqLwo+PiAg I2luY2x1ZGUgInNjaGVkLmgiCj4+ICsjaWZkZWYgQ09ORklHX0FSQ0hfSEFTX01FTUJBUlJJRVJf U1lOQ19DT1JFCj4+ICsjaW5jbHVkZSA8YXNtL3N5bmNfY29yZS5oPgo+PiArI2Vsc2UKPj4gK3N0 YXRpYyBpbmxpbmUgdm9pZCBtZW1iYXJyaWVyX3N5bmNfY29yZV9iZWZvcmVfdXNlcm1vZGUodm9p ZCkKPj4gK3sKPj4gKwljb21waWxldGltZV9hc3NlcnQoMCwgImFyY2hpdGVjdHVyZSBkb2VzIG5v dCBpbXBsZW1lbnQgCj4+IG1lbWJhcnJpZXJfc3luY19jb3JlX2JlZm9yZV91c2VybW9kZSIpOwo+ PiArfQo+PiArCj4gCj4gV2l0aCB0aGUgYXNzZXJ0IHRoZXJlLCBJ4oCZbSBmaW5lIHdpdGggdGhp cy4gTGV0IG1lIHNlZSBpZiB0aGUgcmVzdWx0IGJ1aWxkcy4KCkl0IGhhZCBiZXR0ZXIsIGJlY2F1 c2UgY29tcGlsZXRpbWVfYXNzZXJ0IGFscmVhZHkgcmVsaWVzIG9uIGEgc2ltaWxhciAKbGV2ZWwg b2YgY29kZSBlbGltaW5hdGlvbiB0byB3b3JrLgoKSSB0aGluayBpdCdzIGZpbmUgdG8gdXNlIGZv ciBub3csIGJ1dCBpdCBtYXkgbm90IGJlIHF1aXRlIHRoZSB0aGUgCmxvZ2ljYWxseSBjb3JyZWN0 IHByaW1pdGl2ZSBpZiB3ZSB3YW50IHRvIGJlIHJlYWxseSBjbGVhbiwgYmVjYXVzZSBhIAp2YWxp ZCBjb21waWxldGltZV9hc3NlcnQgaW1wbGVtZW50YXRpb24gc2hvdWxkIGJlIGFibGUgdG8gZmly ZSBldmVuIGZvciAKY29kZSB0aGF0IGlzIG5ldmVyIGxpbmtlZC4gV2Ugd291bGQgd2FudCBzb21l dGhpbmcgbGlrZSB0byBiZSBjbGVhbiAKSU1POgoKI2lmZGVmIENPTkZJR19BUkNIX0hBU19NRU1C QVJSSUVSX1NZTkNfQ09SRQojaW5jbHVkZSA8YXNtL3N5bmNfY29yZS5oPgojZWxzZQpleHRlcm4g dm9pZCBtZW1iYXJyaWVyX3N5bmNfY29yZV9iZWZvcmVfdXNlcm1vZGUodm9pZCkgX19jb21waWxl dGltZV9lcnJvcigiYXJjaGl0ZWN0dXJlIGRvZXMgbm90IGltcGxlbWVudCBtZW1iYXJyaWVyX3N5 bmNfY29yZV9iZWZvcmVfdXNlcm1vZGUiKTsKI2VuZGlmCgpIb3dldmVyIHRoYXQgZG9lcyBub3Qg aGF2ZSB0aGUgaWZkZWYgZm9yIG9wdGltaXNpbmcgY29tcGlsZSBzbyBBRkFJS1MgaXQgCmNvdWxk IGJyZWFrIHdpdGggYSBmYWxzZSBwb3NpdGl2ZSBpbiBzb21lIGNhc2VzLgoKU29tZXRoaW5nIGxp a2UgY29tcGlsZXRpbWVfYXNzZXJ0X25vdF9jYWxsZWQoIm1zZyIpIHRoYXQgZWl0aGVyIGNvbXBp bGVzCnRvIGEgbm9vcCBvciBhIF9fY29tcGlsZXRpbWVfZXJyb3IgZGVwZW5kaW5nIG9uIF9fT1BU SU1JWkVfXyB3b3VsZCBiZSAKdGhlIHdheSB0byBnbyBJTU8uIEkgZG9uJ3Qga25vdyBpZiBhbnl0 aGluZyBleGlzdHMgdGhhdCBmaXRzLCBidXQgaXQncwpjZXJ0YWlubHkgbm90IGEgdW5pcXVlIHRo aW5nIGluIHRoZSBrZXJuZWwgc28gSSBtYXkgbm90IGJlIGxvb2tpbmcgaGFyZAplbm91Z2guCgpU aGFua3MsCk5pY2sKCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fXwpsaW51eC1hcm0ta2VybmVsIG1haWxpbmcgbGlzdApsaW51eC1hcm0ta2VybmVsQGxpc3Rz LmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5m by9saW51eC1hcm0ta2VybmVsCg==