From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B0A7C433E0 for ; Tue, 5 Jan 2021 16:21:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0E97622C9D for ; Tue, 5 Jan 2021 16:21:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727457AbhAEQVe (ORCPT ); Tue, 5 Jan 2021 11:21:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45846 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725838AbhAEQVe (ORCPT ); Tue, 5 Jan 2021 11:21:34 -0500 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EF198C061574 for ; Tue, 5 Jan 2021 08:20:53 -0800 (PST) Received: by mail-pf1-x42b.google.com with SMTP id h186so50229pfe.0 for ; Tue, 05 Jan 2021 08:20:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amacapital-net.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=YY+00AIRRtxJBsSmjelg+ZTD3PBDsqcfuNjtpAy6QDo=; b=B4eaIy0X/gcJ+6Vfog9DvYrA5lzlSG1wkFyLGirBvNF++rcgXb/U/8jhqrKJZpN6Ff 0c8/ST4Y+qwjFXVzERQhJpi6yTRl3MeWGI52OW0gJiRDmDyMS9lrtLx5MMYGnO6m9nDj sJO8YD5Q0cHMp9+vJ7cFvE/kzbIMKBbVsXltojydDYvFaFlO4AT7V7PpJ8O7vDfqxTNv pX5bMba8hKh/rsPyD87PZAOoUfCCsdQzHoHE+jnjagNvEPIoUjiM2GyEikkEjDoZMdhr +I8KLgCyCzMrGsrIpyqBQr2zTMwcxmWNZzNXEWomxCEFyBeKPYoBx7R8MHsoobR+fvSI x3MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=YY+00AIRRtxJBsSmjelg+ZTD3PBDsqcfuNjtpAy6QDo=; b=QEsXrb1vOO4XvoOO+CqIV+v1L0EZmew8lqRBMJGEsDAdvwf1IoG/cXhLnI06DuGMqK OeUWEMixEoQs+ijJJ9dQ4nM9ubgoq77ZQgpAZy5VOvVlYgXI4BQp/m3Vyu7NEL3yNCAW PRAp8u8unKpSXje3x8Z1FEwbAGD+yCuRNdkFc+OP+aJKQCbDAh3KITR6JnJo9LdjaugJ GNCk4P1lJ/7lVxhuZobV15nH44t8PR1xL3p/FSocgetwTg8I+lSRf4qaYdltgUAaxRJx QDTzT9/vvaI4UsMWUJQvl9sMYavlV3IcEpWbRe2DCGFmimvtvSzOmReU2sFb54m5ethl 42qw== X-Gm-Message-State: AOAM532DXYa5mY1JK3KNCz9AZA/iRvd/yHexAq6NJsyt1U4TCwkniMiG YAQFs/meaYSDBCLKSARoUhcQpg== X-Google-Smtp-Source: ABdhPJxa5mdRi43w+JAj2QFqGTiJMOhmPhoSZTCD5EpNlZP1Ra7Be1Mha7b3xYN6GHbxzVuUSGXQhg== X-Received: by 2002:aa7:954b:0:b029:19e:cb57:f3c with SMTP id w11-20020aa7954b0000b029019ecb570f3cmr253102pfq.51.1609863653323; Tue, 05 Jan 2021 08:20:53 -0800 (PST) Received: from ?IPv6:2601:646:c200:1ef2:1960:9abe:3fe9:fd99? ([2601:646:c200:1ef2:1960:9abe:3fe9:fd99]) by smtp.gmail.com with ESMTPSA id y27sm131408pfr.78.2021.01.05.08.20.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Jan 2021 08:20:52 -0800 (PST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Andy Lutomirski Mime-Version: 1.0 (1.0) Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() Date: Tue, 5 Jan 2021 08:20:51 -0800 Message-Id: <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> References: <20210105132623.GB11108@willie-the-truck> Cc: Andy Lutomirski , Nicholas Piggin , Mathieu Desnoyers , X86 ML , Arnd Bergmann , Benjamin Herrenschmidt , Catalin Marinas , linux-arm-kernel , LKML , linuxppc-dev , Michael Ellerman , Paul Mackerras , stable In-Reply-To: <20210105132623.GB11108@willie-the-truck> To: Will Deacon X-Mailer: iPhone Mail (18B121) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > On Jan 5, 2021, at 5:26 AM, Will Deacon wrote: >=20 > =EF=BB=BFHi Andy, >=20 > Sorry for the slow reply, I was socially distanced from my keyboard. >=20 >> On Mon, Dec 28, 2020 at 04:36:11PM -0800, Andy Lutomirski wrote: >> On Mon, Dec 28, 2020 at 4:11 PM Nicholas Piggin wrote= : >>>> +static inline void membarrier_sync_core_before_usermode(void) >>>> +{ >>>> + /* >>>> + * XXX: I know basically nothing about powerpc cache management. >>>> + * Is this correct? >>>> + */ >>>> + isync(); >>>=20 >>> This is not about memory ordering or cache management, it's about >>> pipeline management. Powerpc's return to user mode serializes the >>> CPU (aka the hardware thread, _not_ the core; another wrongness of >>> the name, but AFAIKS the HW thread is what is required for >>> membarrier). So this is wrong, powerpc needs nothing here. >>=20 >> Fair enough. I'm happy to defer to you on the powerpc details. In >> any case, this just illustrates that we need feedback from a person >> who knows more about ARM64 than I do. >=20 > I think we're in a very similar boat to PowerPC, fwiw. Roughly speaking: >=20 > 1. SYNC_CORE does _not_ perform any cache management; that is the > responsibility of userspace, either by executing the relevant > maintenance instructions (arm64) or a system call (arm32). Crucially, > the hardware will ensure that this cache maintenance is broadcast > to all other CPUs. Is this guaranteed regardless of any aliases? That is, if I flush from one C= PU at one VA and then execute the same physical address from another CPU at a= different VA, does this still work? >=20 > 2. Even with all the cache maintenance in the world, a CPU could have > speculatively fetched stale instructions into its "pipeline" ahead of > time, and these are _not_ flushed by the broadcast maintenance instruc= tions > in (1). SYNC_CORE provides a means for userspace to discard these stal= e > instructions. >=20 > 3. The context synchronization event on exception entry/exit is > sufficient here. The Arm ARM isn't very good at describing what it > does, because it's in denial about the existence of a pipeline, but > it does have snippets such as: >=20 > (s/PE/CPU/) > | For all types of memory: > | The PE might have fetched the instructions from memory at any time= > | since the last Context synchronization event on that PE. >=20 > Interestingly, the architecture recently added a control bit to remove= > this synchronisation from exception return, so if we set that then we'= d > have a problem with SYNC_CORE and adding an ISB would be necessary (an= d > we could probable then make kernel->kernel returns cheaper, but I > suspect we're relying on this implicit synchronisation in other places= > too). >=20 Is ISB just a context synchronization event or does it do more? On x86, it=E2=80=99s very hard to tell that MFENCE does any more than LOCK, b= ut it=E2=80=99s much slower. And we have LFENCE, which, as documented, does= n=E2=80=99t appear to have any semantics at all. (Or at least it didn=E2=80= =99t before Spectre.) > Are you seeing a problem in practice, or did this come up while trying to > decipher the semantics of SYNC_CORE? It came up while trying to understand the code and work through various bugs= in it. The code was written using something approximating x86 terminology,= but it was definitely wrong on x86 (at least if you believe the SDM, and I h= aven=E2=80=99t convinced any architects to say otherwise). Thanks! >=20 > Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7BC1C433E9 for ; Tue, 5 Jan 2021 16:23:01 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E49722C9D for ; Tue, 5 Jan 2021 16:23:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E49722C9D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amacapital.net Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4D9Hpp5B3CzDqD5 for ; Wed, 6 Jan 2021 03:22:58 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=amacapital.net (client-ip=2607:f8b0:4864:20::532; helo=mail-pg1-x532.google.com; envelope-from=luto@amacapital.net; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=amacapital.net Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=amacapital-net.20150623.gappssmtp.com header.i=@amacapital-net.20150623.gappssmtp.com header.a=rsa-sha256 header.s=20150623 header.b=B4eaIy0X; dkim-atps=neutral Received: from mail-pg1-x532.google.com (mail-pg1-x532.google.com [IPv6:2607:f8b0:4864:20::532]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4D9HmY0yTGzDqW1 for ; Wed, 6 Jan 2021 03:20:56 +1100 (AEDT) Received: by mail-pg1-x532.google.com with SMTP id e2so160997pgi.5 for ; Tue, 05 Jan 2021 08:20:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amacapital-net.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=YY+00AIRRtxJBsSmjelg+ZTD3PBDsqcfuNjtpAy6QDo=; b=B4eaIy0X/gcJ+6Vfog9DvYrA5lzlSG1wkFyLGirBvNF++rcgXb/U/8jhqrKJZpN6Ff 0c8/ST4Y+qwjFXVzERQhJpi6yTRl3MeWGI52OW0gJiRDmDyMS9lrtLx5MMYGnO6m9nDj sJO8YD5Q0cHMp9+vJ7cFvE/kzbIMKBbVsXltojydDYvFaFlO4AT7V7PpJ8O7vDfqxTNv pX5bMba8hKh/rsPyD87PZAOoUfCCsdQzHoHE+jnjagNvEPIoUjiM2GyEikkEjDoZMdhr +I8KLgCyCzMrGsrIpyqBQr2zTMwcxmWNZzNXEWomxCEFyBeKPYoBx7R8MHsoobR+fvSI x3MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=YY+00AIRRtxJBsSmjelg+ZTD3PBDsqcfuNjtpAy6QDo=; b=O9WzzR8OoUQ5+gap6D3KX5SrN/M4sjgPQPxiqpOFZ1bes7157ff8+kFZ87kv/DI8iP MMowo0qizbLsoWEgdUA2C0mEjJ2Z9mhXjwCQ+mDdS1NDqOtwjhhsMgP6sGL2M3V7dETn sHoA9S0Me8P2ueKklF7yZ5/0An/37IDHIfY3ffh3q2x5o/S4XgMN5UrK32ONNPG/Rshx GEkDHm04rC0sS7r0Y4590kjWWL5ji+Jrliptr7hPz/XUb6rPDuBXExKRHCoO1wj5BfqG SbP68in4Ib/N5V9pJ1n/L6vmfgEsKDps9spaB1F83uVFWdLRZCNWMuzFApMHgQ+asWtt 8yNw== X-Gm-Message-State: AOAM5319AsNVhtk71m8ySzbn01sozqRKy/aChpvODeAe75/kwp9Pr8xC CH3q1ih4bkqUzGbcXjS2hXT5OQ== X-Google-Smtp-Source: ABdhPJxa5mdRi43w+JAj2QFqGTiJMOhmPhoSZTCD5EpNlZP1Ra7Be1Mha7b3xYN6GHbxzVuUSGXQhg== X-Received: by 2002:aa7:954b:0:b029:19e:cb57:f3c with SMTP id w11-20020aa7954b0000b029019ecb570f3cmr253102pfq.51.1609863653323; Tue, 05 Jan 2021 08:20:53 -0800 (PST) Received: from ?IPv6:2601:646:c200:1ef2:1960:9abe:3fe9:fd99? ([2601:646:c200:1ef2:1960:9abe:3fe9:fd99]) by smtp.gmail.com with ESMTPSA id y27sm131408pfr.78.2021.01.05.08.20.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Jan 2021 08:20:52 -0800 (PST) Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable From: Andy Lutomirski Mime-Version: 1.0 (1.0) Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() Date: Tue, 5 Jan 2021 08:20:51 -0800 Message-Id: <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> References: <20210105132623.GB11108@willie-the-truck> In-Reply-To: <20210105132623.GB11108@willie-the-truck> To: Will Deacon X-Mailer: iPhone Mail (18B121) X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arnd Bergmann , X86 ML , LKML , Nicholas Piggin , Mathieu Desnoyers , Andy Lutomirski , Catalin Marinas , Paul Mackerras , stable , linuxppc-dev , linux-arm-kernel Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" > On Jan 5, 2021, at 5:26 AM, Will Deacon wrote: >=20 > =EF=BB=BFHi Andy, >=20 > Sorry for the slow reply, I was socially distanced from my keyboard. >=20 >> On Mon, Dec 28, 2020 at 04:36:11PM -0800, Andy Lutomirski wrote: >> On Mon, Dec 28, 2020 at 4:11 PM Nicholas Piggin wrote= : >>>> +static inline void membarrier_sync_core_before_usermode(void) >>>> +{ >>>> + /* >>>> + * XXX: I know basically nothing about powerpc cache management. >>>> + * Is this correct? >>>> + */ >>>> + isync(); >>>=20 >>> This is not about memory ordering or cache management, it's about >>> pipeline management. Powerpc's return to user mode serializes the >>> CPU (aka the hardware thread, _not_ the core; another wrongness of >>> the name, but AFAIKS the HW thread is what is required for >>> membarrier). So this is wrong, powerpc needs nothing here. >>=20 >> Fair enough. I'm happy to defer to you on the powerpc details. In >> any case, this just illustrates that we need feedback from a person >> who knows more about ARM64 than I do. >=20 > I think we're in a very similar boat to PowerPC, fwiw. Roughly speaking: >=20 > 1. SYNC_CORE does _not_ perform any cache management; that is the > responsibility of userspace, either by executing the relevant > maintenance instructions (arm64) or a system call (arm32). Crucially, > the hardware will ensure that this cache maintenance is broadcast > to all other CPUs. Is this guaranteed regardless of any aliases? That is, if I flush from one C= PU at one VA and then execute the same physical address from another CPU at a= different VA, does this still work? >=20 > 2. Even with all the cache maintenance in the world, a CPU could have > speculatively fetched stale instructions into its "pipeline" ahead of > time, and these are _not_ flushed by the broadcast maintenance instruc= tions > in (1). SYNC_CORE provides a means for userspace to discard these stal= e > instructions. >=20 > 3. The context synchronization event on exception entry/exit is > sufficient here. The Arm ARM isn't very good at describing what it > does, because it's in denial about the existence of a pipeline, but > it does have snippets such as: >=20 > (s/PE/CPU/) > | For all types of memory: > | The PE might have fetched the instructions from memory at any time= > | since the last Context synchronization event on that PE. >=20 > Interestingly, the architecture recently added a control bit to remove= > this synchronisation from exception return, so if we set that then we'= d > have a problem with SYNC_CORE and adding an ISB would be necessary (an= d > we could probable then make kernel->kernel returns cheaper, but I > suspect we're relying on this implicit synchronisation in other places= > too). >=20 Is ISB just a context synchronization event or does it do more? On x86, it=E2=80=99s very hard to tell that MFENCE does any more than LOCK, b= ut it=E2=80=99s much slower. And we have LFENCE, which, as documented, does= n=E2=80=99t appear to have any semantics at all. (Or at least it didn=E2=80= =99t before Spectre.) > Are you seeing a problem in practice, or did this come up while trying to > decipher the semantics of SYNC_CORE? It came up while trying to understand the code and work through various bugs= in it. The code was written using something approximating x86 terminology,= but it was definitely wrong on x86 (at least if you believe the SDM, and I h= aven=E2=80=99t convinced any architects to say otherwise). Thanks! >=20 > Will From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56CE2C433E0 for ; Tue, 5 Jan 2021 16:22:42 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1948422C9D for ; Tue, 5 Jan 2021 16:22:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1948422C9D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amacapital.net Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:In-Reply-To:References:Message-Id:Date:Subject: Mime-Version:From:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Nj1jjyuJ0prCJGuGpdM0m16gN18UeF+f8SKZjiVwYL0=; b=VC5+ghBPaJfSoPHl6Cv0gf2du ube444iIgw/KKire8VT1VMt/GbegppMUT7X8LWkyCPR13bKYFf7OUZuDA8cE5Ixw2OUQS4KTcxN0k Tu+g8yr3PTE3kIWuxKL9fxX7bxruDR2ICGZSzqe1Imt+4ZpPEQH+Rjx3XUK7jwwlFAhBSKII5cn1V tFp1VjA93yF49C7BzNw2dEKZMNCzrBw/EsmK5aQqZY7UVhccmRJwcES5kBGrbK4C2E05jiDdKfOp7 b2WDTZuXd057FRXDL8F5h6TQKXSisbbq9/JbKD/c5OGVYWyD87ljYSxLPBYUvjhDm8gOxatNrNjOq rkFnMaJdQ==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kwp4u-0000HP-Hs; Tue, 05 Jan 2021 16:21:00 +0000 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kwp4r-0000GG-HR for linux-arm-kernel@lists.infradead.org; Tue, 05 Jan 2021 16:20:58 +0000 Received: by mail-pf1-x431.google.com with SMTP id x126so28440pfc.7 for ; Tue, 05 Jan 2021 08:20:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amacapital-net.20150623.gappssmtp.com; s=20150623; h=content-transfer-encoding:from:mime-version:subject:date:message-id :references:cc:in-reply-to:to; bh=YY+00AIRRtxJBsSmjelg+ZTD3PBDsqcfuNjtpAy6QDo=; b=B4eaIy0X/gcJ+6Vfog9DvYrA5lzlSG1wkFyLGirBvNF++rcgXb/U/8jhqrKJZpN6Ff 0c8/ST4Y+qwjFXVzERQhJpi6yTRl3MeWGI52OW0gJiRDmDyMS9lrtLx5MMYGnO6m9nDj sJO8YD5Q0cHMp9+vJ7cFvE/kzbIMKBbVsXltojydDYvFaFlO4AT7V7PpJ8O7vDfqxTNv pX5bMba8hKh/rsPyD87PZAOoUfCCsdQzHoHE+jnjagNvEPIoUjiM2GyEikkEjDoZMdhr +I8KLgCyCzMrGsrIpyqBQr2zTMwcxmWNZzNXEWomxCEFyBeKPYoBx7R8MHsoobR+fvSI x3MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:content-transfer-encoding:from:mime-version :subject:date:message-id:references:cc:in-reply-to:to; bh=YY+00AIRRtxJBsSmjelg+ZTD3PBDsqcfuNjtpAy6QDo=; b=sE800SiaV3+sx6RR34S/HZviFzHBe3m1g6EfD22GXsL/iyAEdoTCiFOd2S6CJLzsxq xXYh3wF9FT1XsJqOnJWtEv3/mp1w82YxMg7QOaS9dZJLdu46SBlTJoOgmeTHdwbOG5gE 2DoP5sfj7d9AkENhhgLn5G/d7SvO/THu+JpqlFZ3Qi+CMkschqWD3dcKWmC2Lw5yL5dV M+Vxe5MITCp0Yu0rPVKiqI1hVorkSwiOniYtgWn7wgzhcqGAAKCQBiIrtskviyx+w16r gRyNqTm3Ir7riiHI1pr/h/X5NXGIITo16TQgqHRAnA0nDTJOxdBnFjn6XZnJMq+m8yoR 22oA== X-Gm-Message-State: AOAM533fBGkKNf7n1tvnl7BcegtpVLqt/ntP60JE0FFZAKhspWUSp72K VrVwDaEX+Z+ppMTmsSL2wE3Z5A== X-Google-Smtp-Source: ABdhPJxa5mdRi43w+JAj2QFqGTiJMOhmPhoSZTCD5EpNlZP1Ra7Be1Mha7b3xYN6GHbxzVuUSGXQhg== X-Received: by 2002:aa7:954b:0:b029:19e:cb57:f3c with SMTP id w11-20020aa7954b0000b029019ecb570f3cmr253102pfq.51.1609863653323; Tue, 05 Jan 2021 08:20:53 -0800 (PST) Received: from ?IPv6:2601:646:c200:1ef2:1960:9abe:3fe9:fd99? ([2601:646:c200:1ef2:1960:9abe:3fe9:fd99]) by smtp.gmail.com with ESMTPSA id y27sm131408pfr.78.2021.01.05.08.20.52 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 05 Jan 2021 08:20:52 -0800 (PST) From: Andy Lutomirski Mime-Version: 1.0 (1.0) Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() Date: Tue, 5 Jan 2021 08:20:51 -0800 Message-Id: <7BFAB97C-1949-46A3-A1E2-DFE108DC7D5E@amacapital.net> References: <20210105132623.GB11108@willie-the-truck> In-Reply-To: <20210105132623.GB11108@willie-the-truck> To: Will Deacon X-Mailer: iPhone Mail (18B121) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210105_112057_709832_07B9AC7A X-CRM114-Status: GOOD ( 26.53 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Arnd Bergmann , Benjamin Herrenschmidt , X86 ML , LKML , Nicholas Piggin , Mathieu Desnoyers , Michael Ellerman , Andy Lutomirski , Catalin Marinas , Paul Mackerras , stable , linuxppc-dev , linux-arm-kernel Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Cgo+IE9uIEphbiA1LCAyMDIxLCBhdCA1OjI2IEFNLCBXaWxsIERlYWNvbiA8d2lsbEBrZXJuZWwu b3JnPiB3cm90ZToKPiAKPiDvu79IaSBBbmR5LAo+IAo+IFNvcnJ5IGZvciB0aGUgc2xvdyByZXBs eSwgSSB3YXMgc29jaWFsbHkgZGlzdGFuY2VkIGZyb20gbXkga2V5Ym9hcmQuCj4gCj4+IE9uIE1v biwgRGVjIDI4LCAyMDIwIGF0IDA0OjM2OjExUE0gLTA4MDAsIEFuZHkgTHV0b21pcnNraSB3cm90 ZToKPj4gT24gTW9uLCBEZWMgMjgsIDIwMjAgYXQgNDoxMSBQTSBOaWNob2xhcyBQaWdnaW4gPG5w aWdnaW5AZ21haWwuY29tPiB3cm90ZToKPj4+PiArc3RhdGljIGlubGluZSB2b2lkIG1lbWJhcnJp ZXJfc3luY19jb3JlX2JlZm9yZV91c2VybW9kZSh2b2lkKQo+Pj4+ICt7Cj4+Pj4gKyAgICAgLyoK Pj4+PiArICAgICAgKiBYWFg6IEkga25vdyBiYXNpY2FsbHkgbm90aGluZyBhYm91dCBwb3dlcnBj IGNhY2hlIG1hbmFnZW1lbnQuCj4+Pj4gKyAgICAgICogSXMgdGhpcyBjb3JyZWN0Pwo+Pj4+ICsg ICAgICAqLwo+Pj4+ICsgICAgIGlzeW5jKCk7Cj4+PiAKPj4+IFRoaXMgaXMgbm90IGFib3V0IG1l bW9yeSBvcmRlcmluZyBvciBjYWNoZSBtYW5hZ2VtZW50LCBpdCdzIGFib3V0Cj4+PiBwaXBlbGlu ZSBtYW5hZ2VtZW50LiBQb3dlcnBjJ3MgcmV0dXJuIHRvIHVzZXIgbW9kZSBzZXJpYWxpemVzIHRo ZQo+Pj4gQ1BVIChha2EgdGhlIGhhcmR3YXJlIHRocmVhZCwgX25vdF8gdGhlIGNvcmU7IGFub3Ro ZXIgd3JvbmduZXNzIG9mCj4+PiB0aGUgbmFtZSwgYnV0IEFGQUlLUyB0aGUgSFcgdGhyZWFkIGlz IHdoYXQgaXMgcmVxdWlyZWQgZm9yCj4+PiBtZW1iYXJyaWVyKS4gU28gdGhpcyBpcyB3cm9uZywg cG93ZXJwYyBuZWVkcyBub3RoaW5nIGhlcmUuCj4+IAo+PiBGYWlyIGVub3VnaC4gIEknbSBoYXBw eSB0byBkZWZlciB0byB5b3Ugb24gdGhlIHBvd2VycGMgZGV0YWlscy4gIEluCj4+IGFueSBjYXNl LCB0aGlzIGp1c3QgaWxsdXN0cmF0ZXMgdGhhdCB3ZSBuZWVkIGZlZWRiYWNrIGZyb20gYSBwZXJz b24KPj4gd2hvIGtub3dzIG1vcmUgYWJvdXQgQVJNNjQgdGhhbiBJIGRvLgo+IAo+IEkgdGhpbmsg d2UncmUgaW4gYSB2ZXJ5IHNpbWlsYXIgYm9hdCB0byBQb3dlclBDLCBmd2l3LiBSb3VnaGx5IHNw ZWFraW5nOgo+IAo+ICAxLiBTWU5DX0NPUkUgZG9lcyBfbm90XyBwZXJmb3JtIGFueSBjYWNoZSBt YW5hZ2VtZW50OyB0aGF0IGlzIHRoZQo+ICAgICByZXNwb25zaWJpbGl0eSBvZiB1c2Vyc3BhY2Us IGVpdGhlciBieSBleGVjdXRpbmcgdGhlIHJlbGV2YW50Cj4gICAgIG1haW50ZW5hbmNlIGluc3Ry dWN0aW9ucyAoYXJtNjQpIG9yIGEgc3lzdGVtIGNhbGwgKGFybTMyKS4gQ3J1Y2lhbGx5LAo+ICAg ICB0aGUgaGFyZHdhcmUgd2lsbCBlbnN1cmUgdGhhdCB0aGlzIGNhY2hlIG1haW50ZW5hbmNlIGlz IGJyb2FkY2FzdAo+ICAgICB0byBhbGwgb3RoZXIgQ1BVcy4KCklzIHRoaXMgZ3VhcmFudGVlZCBy ZWdhcmRsZXNzIG9mIGFueSBhbGlhc2VzPyAgVGhhdCBpcywgaWYgSSBmbHVzaCBmcm9tIG9uZSBD UFUgYXQgb25lIFZBIGFuZCB0aGVuIGV4ZWN1dGUgdGhlIHNhbWUgcGh5c2ljYWwgYWRkcmVzcyBm cm9tIGFub3RoZXIgQ1BVIGF0IGEgZGlmZmVyZW50IFZBLCBkb2VzIHRoaXMgc3RpbGwgd29yaz8K Cj4gCj4gIDIuIEV2ZW4gd2l0aCBhbGwgdGhlIGNhY2hlIG1haW50ZW5hbmNlIGluIHRoZSB3b3Js ZCwgYSBDUFUgY291bGQgaGF2ZQo+ICAgICBzcGVjdWxhdGl2ZWx5IGZldGNoZWQgc3RhbGUgaW5z dHJ1Y3Rpb25zIGludG8gaXRzICJwaXBlbGluZSIgYWhlYWQgb2YKPiAgICAgdGltZSwgYW5kIHRo ZXNlIGFyZSBfbm90XyBmbHVzaGVkIGJ5IHRoZSBicm9hZGNhc3QgbWFpbnRlbmFuY2UgaW5zdHJ1 Y3Rpb25zCj4gICAgIGluICgxKS4gU1lOQ19DT1JFIHByb3ZpZGVzIGEgbWVhbnMgZm9yIHVzZXJz cGFjZSB0byBkaXNjYXJkIHRoZXNlIHN0YWxlCj4gICAgIGluc3RydWN0aW9ucy4KPiAKPiAgMy4g VGhlIGNvbnRleHQgc3luY2hyb25pemF0aW9uIGV2ZW50IG9uIGV4Y2VwdGlvbiBlbnRyeS9leGl0 IGlzCj4gICAgIHN1ZmZpY2llbnQgaGVyZS4gVGhlIEFybSBBUk0gaXNuJ3QgdmVyeSBnb29kIGF0 IGRlc2NyaWJpbmcgd2hhdCBpdAo+ICAgICBkb2VzLCBiZWNhdXNlIGl0J3MgaW4gZGVuaWFsIGFi b3V0IHRoZSBleGlzdGVuY2Ugb2YgYSBwaXBlbGluZSwgYnV0Cj4gICAgIGl0IGRvZXMgaGF2ZSBz bmlwcGV0cyBzdWNoIGFzOgo+IAo+ICAgIChzL1BFL0NQVS8pCj4gICAgICAgfCBGb3IgYWxsIHR5 cGVzIG9mIG1lbW9yeToKPiAgICAgICB8IFRoZSBQRSBtaWdodCBoYXZlIGZldGNoZWQgdGhlIGlu c3RydWN0aW9ucyBmcm9tIG1lbW9yeSBhdCBhbnkgdGltZQo+ICAgICAgIHwgc2luY2UgdGhlIGxh c3QgQ29udGV4dCBzeW5jaHJvbml6YXRpb24gZXZlbnQgb24gdGhhdCBQRS4KPiAKPiAgICAgSW50 ZXJlc3RpbmdseSwgdGhlIGFyY2hpdGVjdHVyZSByZWNlbnRseSBhZGRlZCBhIGNvbnRyb2wgYml0 IHRvIHJlbW92ZQo+ICAgICB0aGlzIHN5bmNocm9uaXNhdGlvbiBmcm9tIGV4Y2VwdGlvbiByZXR1 cm4sIHNvIGlmIHdlIHNldCB0aGF0IHRoZW4gd2UnZAo+ICAgICBoYXZlIGEgcHJvYmxlbSB3aXRo IFNZTkNfQ09SRSBhbmQgYWRkaW5nIGFuIElTQiB3b3VsZCBiZSBuZWNlc3NhcnkgKGFuZAo+ICAg ICB3ZSBjb3VsZCBwcm9iYWJsZSB0aGVuIG1ha2Uga2VybmVsLT5rZXJuZWwgcmV0dXJucyBjaGVh cGVyLCBidXQgSQo+ICAgICBzdXNwZWN0IHdlJ3JlIHJlbHlpbmcgb24gdGhpcyBpbXBsaWNpdCBz eW5jaHJvbmlzYXRpb24gaW4gb3RoZXIgcGxhY2VzCj4gICAgIHRvbykuCj4gCgpJcyBJU0IganVz dCBhIGNvbnRleHQgc3luY2hyb25pemF0aW9uIGV2ZW50IG9yIGRvZXMgaXQgZG8gbW9yZT8KCk9u IHg4NiwgaXTigJlzIHZlcnkgaGFyZCB0byB0ZWxsIHRoYXQgTUZFTkNFIGRvZXMgYW55IG1vcmUg dGhhbiBMT0NLLCBidXQgaXTigJlzIG11Y2ggc2xvd2VyLiAgQW5kIHdlIGhhdmUgTEZFTkNFLCB3 aGljaCwgYXMgZG9jdW1lbnRlZCwgZG9lc27igJl0IGFwcGVhciB0byBoYXZlIGFueSBzZW1hbnRp Y3MgYXQgYWxsLiAgKE9yIGF0IGxlYXN0IGl0IGRpZG7igJl0IGJlZm9yZSBTcGVjdHJlLikKCj4g QXJlIHlvdSBzZWVpbmcgYSBwcm9ibGVtIGluIHByYWN0aWNlLCBvciBkaWQgdGhpcyBjb21lIHVw IHdoaWxlIHRyeWluZyB0bwo+IGRlY2lwaGVyIHRoZSBzZW1hbnRpY3Mgb2YgU1lOQ19DT1JFPwoK SXQgY2FtZSB1cCB3aGlsZSB0cnlpbmcgdG8gdW5kZXJzdGFuZCB0aGUgY29kZSBhbmQgd29yayB0 aHJvdWdoIHZhcmlvdXMgYnVncyBpbiBpdC4gIFRoZSBjb2RlIHdhcyB3cml0dGVuIHVzaW5nIHNv bWV0aGluZyBhcHByb3hpbWF0aW5nIHg4NiB0ZXJtaW5vbG9neSwgYnV0IGl0IHdhcyBkZWZpbml0 ZWx5IHdyb25nIG9uIHg4NiAoYXQgbGVhc3QgaWYgeW91IGJlbGlldmUgdGhlIFNETSwgYW5kIEkg aGF2ZW7igJl0IGNvbnZpbmNlZCBhbnkgYXJjaGl0ZWN0cyB0byBzYXkgb3RoZXJ3aXNlKS4KClRo YW5rcyEKCj4gCj4gV2lsbAoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBs aXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlz dGluZm8vbGludXgtYXJtLWtlcm5lbAo=