From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C009C433E0 for ; Mon, 28 Dec 2020 18:11:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C62B722AAD for ; Mon, 28 Dec 2020 18:11:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728657AbgL1SK6 (ORCPT ); Mon, 28 Dec 2020 13:10:58 -0500 Received: from mail.kernel.org ([198.145.29.99]:33670 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728094AbgL1SK5 (ORCPT ); Mon, 28 Dec 2020 13:10:57 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 1078D22B2A for ; Mon, 28 Dec 2020 18:10:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1609179016; bh=UU5ZJ7wo09eeQJfHCA8zAN6QsAu7KkQNIOEasIFeWmg=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=NtSbNU38MzOGGEZybmJrqVrXIkZ7riOP6KLyDWXdcKOzPn2oBkVpWfP68WXcUvTXw QPCvxMa9V7U58EGvYn028q8tNODv/ggjHZYdRKTszI+dI2VtLCOS/w4qSKGYw3UIh0 bYN14vc0bpBVomQt5g8Pobew4A1eaZjEyh7ugB5j3AJXzki4F6GHeV/cJ6tzgGs3Xi jccf0pLuMoDyHVDChw9mxOzf/70bEUpgqc8fLJvWl/A0AJV94HAAkLYtXrSnTSxJ9R Voqz0zee4pqZj7x4dfz/8/WNsBT6+tgi33/b34do1ws/GbSSgdpVtXhVSGrPvVgBLL xHvCEkW1rbJ7A== Received: by mail-wr1-f41.google.com with SMTP id r7so12078797wrc.5 for ; Mon, 28 Dec 2020 10:10:15 -0800 (PST) X-Gm-Message-State: AOAM530+lmY4t+YZe2C0/NJDBdin+JWNV9v38468Yr0uE9qHBiP8POp2 Xz+3rN/LkyvZIfBE/efK06e+HXP7Ejfvl0EkcZILJg== X-Google-Smtp-Source: ABdhPJxHP2b6pFiKrTNQ13geWSURUQyXlKEfQlURiSpdu2+ttGMTvrMYL94o5eJg5g/pfdfXcAWB78Wu9XbAQX8kJ/o= X-Received: by 2002:a5d:62c7:: with SMTP id o7mr336085wrv.257.1609179014608; Mon, 28 Dec 2020 10:10:14 -0800 (PST) MIME-Version: 1.0 References: <1836294649.3345.1609100294833.JavaMail.zimbra@efficios.com> <20201228102537.GG1551@shell.armlinux.org.uk> <20201228172301.GH1551@shell.armlinux.org.uk> In-Reply-To: <20201228172301.GH1551@shell.armlinux.org.uk> From: Andy Lutomirski Date: Mon, 28 Dec 2020 10:10:02 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() To: Russell King - ARM Linux admin Cc: Andy Lutomirski , Mathieu Desnoyers , x86 , linux-kernel , Nicholas Piggin , Arnd Bergmann , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , linuxppc-dev , Catalin Marinas , Will Deacon , linux-arm-kernel , stable Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Dec 28, 2020 at 9:23 AM Russell King - ARM Linux admin wrote: > > On Mon, Dec 28, 2020 at 09:14:23AM -0800, Andy Lutomirski wrote: > > On Mon, Dec 28, 2020 at 2:25 AM Russell King - ARM Linux admin > > wrote: > > > > > > On Sun, Dec 27, 2020 at 01:36:13PM -0800, Andy Lutomirski wrote: > > > > On Sun, Dec 27, 2020 at 12:18 PM Mathieu Desnoyers > > > > wrote: > > > > > > > > > > ----- On Dec 27, 2020, at 1:28 PM, Andy Lutomirski luto@kernel.org wrote: > > > > > > > > > > > > > > > > > > > > > I admit that I'm rather surprised that the code worked at all on arm64, > > > > > > and I'm suspicious that it has never been very well tested. My apologies > > > > > > for not reviewing this more carefully in the first place. > > > > > > > > > > Please refer to Documentation/features/sched/membarrier-sync-core/arch-support.txt > > > > > > > > > > It clearly states that only arm, arm64, powerpc and x86 support the membarrier > > > > > sync core feature as of now: > > > > > > > > Sigh, I missed arm (32). Russell or ARM folks, what's the right > > > > incantation to make the CPU notice instruction changes initiated by > > > > other cores on 32-bit ARM? > > > > > > You need to call flush_icache_range(), since the changes need to be > > > flushed from the data cache to the point of unification (of the Harvard > > > I and D), and the instruction cache needs to be invalidated so it can > > > then see those updated instructions. This will also take care of the > > > necessary barriers that the CPU requires for you. > > > > With what parameters? From looking at the header, this is for the > > case in which the kernel writes some memory and then intends to > > execute it. That's not what membarrier() does at all. membarrier() > > works like this: > > You didn't specify that you weren't looking at kernel memory. > > If you're talking about userspace, then the interface you require > is flush_icache_user_range(), which does the same as > flush_icache_range() but takes userspace addresses. Note that this > requires that the memory is currently mapped at those userspace > addresses. > > If that doesn't fit your needs, there isn't an interface to do what > you require, and it basically means creating something brand new > on every architecture. > > What you are asking for is not "just a matter of a few instructions". > I have stated the required steps to achieve what you require above; > that is the minimum when you have non-snooping harvard caches, which > the majority of 32-bit ARMs have. > > > User thread 1: > > > > write to RWX memory *or* write to an RW alias of an X region. > > membarrier(...); > > somehow tell thread 2 that we're ready (with a store release, perhaps, > > or even just a relaxed store.) > > > > User thread 2: > > > > wait for the indication from thread 1. > > barrier(); > > jump to the code. > > > > membarrier() is, for better or for worse, not given a range of addresses. > > Then, I'm sorry, it can't work on 32-bit ARM. Is there a way to flush the *entire* user icache? If so, and if it has reasonable performance, then it could probably be used here. Otherwise I'll just send a revert for this whole mechanism on 32-bit ARM. --Andy