From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,SPF_HELO_NONE, SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F3A9C433DB for ; Tue, 29 Dec 2020 00:31:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DECC1207CF for ; Tue, 29 Dec 2020 00:31:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730539AbgL2Abe (ORCPT ); Mon, 28 Dec 2020 19:31:34 -0500 Received: from mail.kernel.org ([198.145.29.99]:45950 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730191AbgL2Abd (ORCPT ); Mon, 28 Dec 2020 19:31:33 -0500 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9CA4B2226A for ; Tue, 29 Dec 2020 00:30:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1609201852; bh=yV4j/cRgE8H7aOVGN5jCQGtvZBZobooTVQ66898JI4I=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=f5OovboTlW7EYpT8oivw2g8c4aQCg45IJs780xMsZVv9BXlppqEZqy3983beo0x51 kFMuwRgSh0/pRPeHHA20PsWMQmrb9zMZ6QqyMI/pLw0gOiomHhkRC7FHT0chpCeFj0 QWCj2/OWKlLN3lFN+EQ8jvqmX67Op9swBowXWGtd1BIrwQkdP/5XH+oR4o5a74MF+d 4BHKEJlBKCraDuT70SOJkWP8mxSfeiGcFCmj2MxMhIxG2DAuhYkqV06UPt8Iw3reyP 0RVrPJOloYf8YlLTPvAgv1smAbWsM3TS3YPalrnRrPy9vni75J5C/h+t61dUR/GoRh K4yiWQpZEE7+A== Received: by mail-wm1-f48.google.com with SMTP id y23so812773wmi.1 for ; Mon, 28 Dec 2020 16:30:52 -0800 (PST) X-Gm-Message-State: AOAM532XnNdhleM075cn/n5kq2wrwlBDpe8Pk3AREnb1cKaFbpV+yaam mvy3hGXP+XTmJZLbhhDTOUWWop3fcRouLkW4WL0cyg== X-Google-Smtp-Source: ABdhPJyTSyI1fcHiawazSCE0V3NGTirXlbcKcNazoOxkfqoxtb4dfAOpbR4IJMj2HYqQ9DARh1IpAW6OOBL8GRTvWuI= X-Received: by 2002:a1c:2188:: with SMTP id h130mr1080293wmh.176.1609201851177; Mon, 28 Dec 2020 16:30:51 -0800 (PST) MIME-Version: 1.0 References: <1836294649.3345.1609100294833.JavaMail.zimbra@efficios.com> <1670059472.3671.1609189779376.JavaMail.zimbra@efficios.com> In-Reply-To: <1670059472.3671.1609189779376.JavaMail.zimbra@efficios.com> From: Andy Lutomirski Date: Mon, 28 Dec 2020 16:30:39 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() To: Mathieu Desnoyers Cc: Andy Lutomirski , paulmck , Peter Zijlstra , "Russell King, ARM Linux" , x86 , linux-kernel , Nicholas Piggin , Arnd Bergmann , Michael Ellerman , Benjamin Herrenschmidt , Paul Mackerras , linuxppc-dev , Catalin Marinas , Will Deacon , linux-arm-kernel , stable Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Dec 28, 2020 at 1:09 PM Mathieu Desnoyers wrote: > > ----- On Dec 27, 2020, at 4:36 PM, Andy Lutomirski luto@kernel.org wrote: > > [...] > > >> You seem to have noticed odd cases on arm64 where this guarantee does not > >> match reality. Where exactly can we find this in the code, and which part > >> of the architecture manual can you point us to which supports your concern ? > >> > >> Based on the notes I have, use of `eret` on aarch64 guarantees a context > >> synchronizing > >> instruction when returning to user-space. > > > > Based on my reading of the manual, ERET on ARM doesn't synchronize > > anything at all. I can't find any evidence that it synchronizes data > > or instructions, and I've seen reports that the CPU will happily > > speculate right past it. > > Reading [1] there appears to be 3 kind of context synchronization events: > > - Taking an exception, > - Returning from an exception, > - ISB. My reading of [1] is that all three of these are "context synchronization event[s]", but that only ISB flushes the pipeline, etc. The little description of context synchronization seems to suggest that it only implies that certain register changes become effective. > > This other source [2] adds (search for Context synchronization operation): > > - Exit from Debug state > - Executing a DCPS instruction > - Executing a DRPS instruction > > "ERET" falls into the second kind of events, and AFAIU should be context > synchronizing. That was confirmed to me by Will Deacon when membarrier > sync-core was implemented for aarch64. If the architecture reference manuals > are wrong, is there an errata ? > > As for the algorithm to use on ARMv8 to update instructions, see [2] > B2.3.4 Implication of caches for the application programmer > "Synchronization and coherency issues between data and instruction accesses" This specifically discusses ISB. Let's wait for an actual ARM64 expert to chime in, though. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FD18C433DB for ; Tue, 29 Dec 2020 00:32:36 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7012F207CF for ; Tue, 29 Dec 2020 00:32:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7012F207CF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from bilbo.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 4D4b3P3TLKzDqGc for ; Tue, 29 Dec 2020 11:32:33 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=kernel.org (client-ip=198.145.29.99; helo=mail.kernel.org; envelope-from=luto@kernel.org; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=pass (p=none dis=none) header.from=kernel.org Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=kernel.org header.i=@kernel.org header.a=rsa-sha256 header.s=k20201202 header.b=f5OovboT; dkim-atps=neutral Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4D4b1W48RdzDqB5 for ; Tue, 29 Dec 2020 11:30:55 +1100 (AEDT) Received: by mail.kernel.org (Postfix) with ESMTPSA id 8E1F222227 for ; Tue, 29 Dec 2020 00:30:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1609201852; bh=yV4j/cRgE8H7aOVGN5jCQGtvZBZobooTVQ66898JI4I=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=f5OovboTlW7EYpT8oivw2g8c4aQCg45IJs780xMsZVv9BXlppqEZqy3983beo0x51 kFMuwRgSh0/pRPeHHA20PsWMQmrb9zMZ6QqyMI/pLw0gOiomHhkRC7FHT0chpCeFj0 QWCj2/OWKlLN3lFN+EQ8jvqmX67Op9swBowXWGtd1BIrwQkdP/5XH+oR4o5a74MF+d 4BHKEJlBKCraDuT70SOJkWP8mxSfeiGcFCmj2MxMhIxG2DAuhYkqV06UPt8Iw3reyP 0RVrPJOloYf8YlLTPvAgv1smAbWsM3TS3YPalrnRrPy9vni75J5C/h+t61dUR/GoRh K4yiWQpZEE7+A== Received: by mail-wm1-f46.google.com with SMTP id r4so793789wmh.5 for ; Mon, 28 Dec 2020 16:30:52 -0800 (PST) X-Gm-Message-State: AOAM533t3r1sD6kGOoCxeuC+91+pLiXR79aWuPjmHELrzk/M4FeHaMYA NdIFrrGAHIJrAoOPswaJXfri1wgZ+714mofdiCGQYQ== X-Google-Smtp-Source: ABdhPJyTSyI1fcHiawazSCE0V3NGTirXlbcKcNazoOxkfqoxtb4dfAOpbR4IJMj2HYqQ9DARh1IpAW6OOBL8GRTvWuI= X-Received: by 2002:a1c:2188:: with SMTP id h130mr1080293wmh.176.1609201851177; Mon, 28 Dec 2020 16:30:51 -0800 (PST) MIME-Version: 1.0 References: <1836294649.3345.1609100294833.JavaMail.zimbra@efficios.com> <1670059472.3671.1609189779376.JavaMail.zimbra@efficios.com> In-Reply-To: <1670059472.3671.1609189779376.JavaMail.zimbra@efficios.com> From: Andy Lutomirski Date: Mon, 28 Dec 2020 16:30:39 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() To: Mathieu Desnoyers Content-Type: text/plain; charset="UTF-8" X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Catalin Marinas , Arnd Bergmann , paulmck , Peter Zijlstra , x86 , linux-kernel , Nicholas Piggin , "Russell King, ARM Linux" , Will Deacon , Paul Mackerras , stable , Andy Lutomirski , linuxppc-dev , linux-arm-kernel Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" On Mon, Dec 28, 2020 at 1:09 PM Mathieu Desnoyers wrote: > > ----- On Dec 27, 2020, at 4:36 PM, Andy Lutomirski luto@kernel.org wrote: > > [...] > > >> You seem to have noticed odd cases on arm64 where this guarantee does not > >> match reality. Where exactly can we find this in the code, and which part > >> of the architecture manual can you point us to which supports your concern ? > >> > >> Based on the notes I have, use of `eret` on aarch64 guarantees a context > >> synchronizing > >> instruction when returning to user-space. > > > > Based on my reading of the manual, ERET on ARM doesn't synchronize > > anything at all. I can't find any evidence that it synchronizes data > > or instructions, and I've seen reports that the CPU will happily > > speculate right past it. > > Reading [1] there appears to be 3 kind of context synchronization events: > > - Taking an exception, > - Returning from an exception, > - ISB. My reading of [1] is that all three of these are "context synchronization event[s]", but that only ISB flushes the pipeline, etc. The little description of context synchronization seems to suggest that it only implies that certain register changes become effective. > > This other source [2] adds (search for Context synchronization operation): > > - Exit from Debug state > - Executing a DCPS instruction > - Executing a DRPS instruction > > "ERET" falls into the second kind of events, and AFAIU should be context > synchronizing. That was confirmed to me by Will Deacon when membarrier > sync-core was implemented for aarch64. If the architecture reference manuals > are wrong, is there an errata ? > > As for the algorithm to use on ARMv8 to update instructions, see [2] > B2.3.4 Implication of caches for the application programmer > "Synchronization and coherency issues between data and instruction accesses" This specifically discusses ISB. Let's wait for an actual ARM64 expert to chime in, though. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 34B8FC433DB for ; Tue, 29 Dec 2020 00:32:54 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E56AC207CF for ; Tue, 29 Dec 2020 00:32:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E56AC207CF Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=mNzNz22dTomMj0Vbl6VqUInRttsFJhJG0MPwZbx6PnI=; b=KMZvwGht078WxunYeyDyFNdXm z08Xk5vGQlpbnO7dnnKg/aFCSkA3kUGyLNUuEpjGdkXb4sL/fbTrGi9mu749k3op8tXYLAkAycVYR mdpGQ+aqyHMze4+jl/NvKGuFdwpRk/OCjxkz23fexO6sDMi6WJ7ooS3M0BKQNw5Cqzg3LE99dFTb7 XOM+MsfcM3N/6xDXifA9op1Mvdndy/xbFQT/aIzxzrXwhP/60s3vAZrJEAGJTAxHwqSpdahkdk1RQ p9zm9+lDWQIZ2NwupCm1KOE6lCv3BTYRHJgjqnglTpDt+sJD0EjYwWDnYBk3NnjwZrMRNsR9UnTwN sFiOOw+vw==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1ku2uf-0007eO-V5; Tue, 29 Dec 2020 00:30:58 +0000 Received: from mail.kernel.org ([198.145.29.99]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1ku2ub-0007d0-MZ for linux-arm-kernel@lists.infradead.org; Tue, 29 Dec 2020 00:30:54 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 9945022242 for ; Tue, 29 Dec 2020 00:30:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1609201852; bh=yV4j/cRgE8H7aOVGN5jCQGtvZBZobooTVQ66898JI4I=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=f5OovboTlW7EYpT8oivw2g8c4aQCg45IJs780xMsZVv9BXlppqEZqy3983beo0x51 kFMuwRgSh0/pRPeHHA20PsWMQmrb9zMZ6QqyMI/pLw0gOiomHhkRC7FHT0chpCeFj0 QWCj2/OWKlLN3lFN+EQ8jvqmX67Op9swBowXWGtd1BIrwQkdP/5XH+oR4o5a74MF+d 4BHKEJlBKCraDuT70SOJkWP8mxSfeiGcFCmj2MxMhIxG2DAuhYkqV06UPt8Iw3reyP 0RVrPJOloYf8YlLTPvAgv1smAbWsM3TS3YPalrnRrPy9vni75J5C/h+t61dUR/GoRh K4yiWQpZEE7+A== Received: by mail-wm1-f44.google.com with SMTP id 3so797567wmg.4 for ; Mon, 28 Dec 2020 16:30:52 -0800 (PST) X-Gm-Message-State: AOAM531iGy90lPEwXI+wY+NkEyFwZR3n4i6y/yaaRO3zP4nlFePZVB+y fdmmi+MVm5zAN+8BHA/PRH6FhWve8x/inrkWBD0Gxg== X-Google-Smtp-Source: ABdhPJyTSyI1fcHiawazSCE0V3NGTirXlbcKcNazoOxkfqoxtb4dfAOpbR4IJMj2HYqQ9DARh1IpAW6OOBL8GRTvWuI= X-Received: by 2002:a1c:2188:: with SMTP id h130mr1080293wmh.176.1609201851177; Mon, 28 Dec 2020 16:30:51 -0800 (PST) MIME-Version: 1.0 References: <1836294649.3345.1609100294833.JavaMail.zimbra@efficios.com> <1670059472.3671.1609189779376.JavaMail.zimbra@efficios.com> In-Reply-To: <1670059472.3671.1609189779376.JavaMail.zimbra@efficios.com> From: Andy Lutomirski Date: Mon, 28 Dec 2020 16:30:39 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [RFC please help] membarrier: Rewrite sync_core_before_usermode() To: Mathieu Desnoyers X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201228_193053_938226_64AAFDA7 X-CRM114-Status: GOOD ( 22.85 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Catalin Marinas , Arnd Bergmann , paulmck , Peter Zijlstra , Michael Ellerman , x86 , linux-kernel , Nicholas Piggin , "Russell King, ARM Linux" , Will Deacon , Paul Mackerras , stable , Andy Lutomirski , Benjamin Herrenschmidt , linuxppc-dev , linux-arm-kernel Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Dec 28, 2020 at 1:09 PM Mathieu Desnoyers wrote: > > ----- On Dec 27, 2020, at 4:36 PM, Andy Lutomirski luto@kernel.org wrote: > > [...] > > >> You seem to have noticed odd cases on arm64 where this guarantee does not > >> match reality. Where exactly can we find this in the code, and which part > >> of the architecture manual can you point us to which supports your concern ? > >> > >> Based on the notes I have, use of `eret` on aarch64 guarantees a context > >> synchronizing > >> instruction when returning to user-space. > > > > Based on my reading of the manual, ERET on ARM doesn't synchronize > > anything at all. I can't find any evidence that it synchronizes data > > or instructions, and I've seen reports that the CPU will happily > > speculate right past it. > > Reading [1] there appears to be 3 kind of context synchronization events: > > - Taking an exception, > - Returning from an exception, > - ISB. My reading of [1] is that all three of these are "context synchronization event[s]", but that only ISB flushes the pipeline, etc. The little description of context synchronization seems to suggest that it only implies that certain register changes become effective. > > This other source [2] adds (search for Context synchronization operation): > > - Exit from Debug state > - Executing a DCPS instruction > - Executing a DRPS instruction > > "ERET" falls into the second kind of events, and AFAIU should be context > synchronizing. That was confirmed to me by Will Deacon when membarrier > sync-core was implemented for aarch64. If the architecture reference manuals > are wrong, is there an errata ? > > As for the algorithm to use on ARMv8 to update instructions, see [2] > B2.3.4 Implication of caches for the application programmer > "Synchronization and coherency issues between data and instruction accesses" This specifically discusses ISB. Let's wait for an actual ARM64 expert to chime in, though. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel