From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13577C47082 for ; Tue, 8 Jun 2021 19:56:05 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D60076135D for ; Tue, 8 Jun 2021 19:56:04 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D60076135D Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:Subject:Message-ID:Date:From: In-Reply-To:References:MIME-Version:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w6GFMV49LpapuBjzCPWnwBuII8qSLxM8rqDm5C5JJsA=; b=Rk5h/ICB57Qnuz yXGRCR2hGFLkBf3GHSnwsfbVoKgmsFqW8BZQDkhBKXeP8Q5QENxitHSmGtr62PznQeIVBb+7Kmzmn xWLJLTy5WHQlKoF4tWGu/GAQaI+KmK727p1uOtnhSs3STYtn6tbgFQw4SKuC/dRLdjmhygdoRLEBq aJiK/K4B9owrAn9yvuHYUvkdDU5ENSvn7xEsZRyUZq4Zb9LaWt4iMFTRNh3E3wcxkL8Gpyp2cD0nh nOM0/ydE0/gVGTJsxumkmlAs3PMoR138QBbSaSr7XNZPrtph9wzcnR/SYw2zAY6XDyamyWyi92ot8 KHkP02SgAkC4LTeThr5Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqhnq-00AI7I-AZ; Tue, 08 Jun 2021 19:54:22 +0000 Received: from mail-il1-x12d.google.com ([2607:f8b0:4864:20::12d]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lqhnm-00AI30-DG for linux-arm-kernel@lists.infradead.org; Tue, 08 Jun 2021 19:54:20 +0000 Received: by mail-il1-x12d.google.com with SMTP id t6so13894223iln.8 for ; Tue, 08 Jun 2021 12:54:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=mnAe8qKhM1bMDwbGN0CsGA/rHI3QE8BnDh2gAbQ17Po=; b=fI3hZ7x5kZcZ5Klg7wub+TlSG3JhVw/RUzyEb8bfvHGLRqQ0CLX2SqSvQb1ZFmzquk N/j5D7YKI0Rmt7o1YR/YcvepSY0mtH99DuZOptqKrjvQkmPys5IP28127sGmQHOSKKE+ odMrv2fAFXs/IXXuXC7wGpn1eymhclvMzKzIH+wXdbRClPzgCaRJ24LR6iyXaEwbV7J7 BerTRHE4dsqEP/uIvdR3uhXWWejspj8+fOcU7LzQeilg7CAuSPClo/OMHwOl9zBCjB8+ Ye/qtqXTwYCWFJLzMXXA6mXA/eDHbUanYoNdnUosJ6tNlW2pcIFC0nB59qE0WDfbG+ce 1ZSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=mnAe8qKhM1bMDwbGN0CsGA/rHI3QE8BnDh2gAbQ17Po=; b=DOqfS7qL1LEIjd170T/0YmpxjlaqU1SG6heJAWt+l1vx5GTUfebIXBnlIJa/vk3CKt CUPR2YOYbtbdqO+L7Sy941xXBB8BQ4pYPKb6PXODafd/MgtRsggIp/dbqkw1pa+VIPjz 8NWtB+Uel8L/9vLfzlV3TX9B7e4CJnFxJUo2ti86/MFomzau0aU2SS5xpnRgg1yRLbpK 3cVJ+Z6Z1J86rQh5Qwx3HL4tgfgqoyseGcj/RWQz91fsnJZCb8MJpsKPS0xyaRZ2nPdb 9LI+BQgFYxISm1gBsS7HfIB2etmKyQ+/K4JqM5R3CQ1zawSKXpe6hQ/dQsWHiIpcwaxk 81wQ== X-Gm-Message-State: AOAM532c+rMJZbmJ2kdB57rtuwB76CTYGE3g+CQ8GcEMITJJ8GQzNz+g WlPeLK2KRNdOTvKcqfaDwxgxNiZeeWGR39cojgBmuA== X-Google-Smtp-Source: ABdhPJy10yeGgJahjAdUEm/zNARxm8qQewHZtWF34QByutVASMN9RfnVFI6TaWnkByylM43g3RN8R4kOQ3CHQAW4C/w= X-Received: by 2002:a92:d681:: with SMTP id p1mr11979194iln.37.1623182052299; Tue, 08 Jun 2021 12:54:12 -0700 (PDT) MIME-Version: 1.0 References: <20210602232445.3829248-1-pcc@google.com> <5ee9d9a1-5b13-ea21-67df-e713c76fc163@arm.com> <20210608144353.GF17957@arm.com> In-Reply-To: <20210608144353.GF17957@arm.com> From: Peter Collingbourne Date: Tue, 8 Jun 2021 12:54:01 -0700 Message-ID: Subject: Re: [PATCH] arm64: mte: allow async MTE to be upgraded to sync on a per-CPU basis To: Catalin Marinas Cc: Vincenzo Frascino , Will Deacon , Evgenii Stepanov , linux-arm-kernel@lists.infradead.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210608_125418_505420_0D551527 X-CRM114-Status: GOOD ( 57.80 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Jun 8, 2021 at 7:44 AM Catalin Marinas wrote: > > On Thu, Jun 03, 2021 at 10:49:24AM -0700, Peter Collingbourne wrote: > > On Thu, Jun 3, 2021 at 6:01 AM Vincenzo Frascino > > wrote: > > > On 6/3/21 12:24 AM, Peter Collingbourne wrote: > > > > On some CPUs the performance of MTE in synchronous mode is the same > > > > as that of asynchronous mode. This makes it worthwhile to enable > > > > synchronous mode on those CPUs when asynchronous mode is requested, > > > > in order to gain the error detection benefits of synchronous mode > > > > without the performance downsides. Therefore, make it possible for CPUs > > > > to opt into upgrading to synchronous mode via a new mte-prefer-sync > > > > device tree attribute. > > > > > > > > > > I had a look at your patch and I think that there are few points that are worth > > > mentioning: > > > 1) The approach you are using is per-CPU hence we might end up with a system > > > that has some PE configured as sync and some configured as async. We currently > > > support only a system wide setting. > > > > This is the intent. On e.g. a big/little system this means that we > > would effectively have sampling of sync MTE faults at a higher rate > > than a pure userspace implementation could achieve, at zero cost. > > > > > 2) async and sync have slightly different semantics (e.g. in sync mode the > > > access does not take place and it requires emulation) this means that a mixed > > > configuration affects the ABI. > > > > We considered the ABI question and think that is somewhat academic. > > While it's true that we would prevent the first access from succeeding > > (and, more visibly, use SEGV_MTESERR in the signal rather than > > SEGV_MTEAERR) I'm not aware of a reasonable way that a userspace > > program could depend on the access succeeding. > > It's more about whether some software relies on the async mode only for > logging without any intention of handling the synchronous faults. In > such scenario, the async signal is fairly simple, it logs and continues > safely (well, as "safe" as before MTE). With the sync mode, however, the > signal handler will have to ensure the access took place either before > continuing, either by emulating, restarting the instruction with > PSTATE.TCO set or by falling back to the async mode. > > IOW, I don't expect all programs making use of MTE to simply die on an > MTE fault (though I guess they'll be in a minority but we still need to > allow such scenarios). Okay, allowing such programs seems reasonable to me. The question is then whether the new behavior should be an opt-in or an opt-out. > > While it's slightly > > more plausible that there could be a dependency on the signal type, we > > don't depend on that in Android, at least not in a way that would lead > > to worse outcomes if we get MTESERR instead of MTEAERR (it would lead > > to better outcomes, in the form of a more accurate/detailed crash > > report, which is what motivates this change). I also checked glibc and > > they don't appear to have any dependencies on the signal type, or > > indeed have any detailed crash reporting at all as far as I can tell. > > Furthermore, basically nobody has hardware at the moment so I don't > > think we would be breaking any actual users by doing this. > > While there's no user-space out there yet, given that MTE was merged in > 5.10 and that's an LTS kernel, we'll see software running on this kernel > version at some point in the next few years. So any fix will need > backporting but an ABI change for better performance on specific SoCs > hardly counts as a fix. Okay, seems reasonable enough. > I'm ok with improving the ABI but not breaking the current one, hence > the suggestion for a new PR_MTE_TCF_* field or maybe a new bit that > allows the mode to be "upgraded", for some definition of this (for > example, if TCF_NONE is as fast as TCF_ASYNC, should NONE be upgraded?) I think the possibility of upgrading NONE to ASYNC is a good reason to make this an opt-in, since it alters the behavior in a more significant way than ASYNC to SYNC. > > > 3) In your patch you use DT to enforce sync mode on a CPU, probably it is better > > > to have an MIDR scheme to mark these CPUs. > > > > Okay, so in your scheme we would say that e.g. all Cortex-A510 CPUs > > should be subject to this treatment. Can we guarantee that all > > Cortex-A510 CPUs would have the same performance for sync and async or > > could the system designer tweak some aspect of the system such that > > they could get different performance? The possibility of the latter is > > what led me to specify the information via DT. > > While it's more of a CPU microarchitecture issue, there's indeed a good > chance that the SoC has a non-trivial impact on the performance of the > synchronous mode, so it may tip the balance one way or another. > > Another idea would be to introduce a PR_MTE_TCF_DEFAULT mode together > with some /sys/*/cpu*/ entries to control the default MTE mode per CPU. > We'd leave it entirely to user-space (privileged user), maybe it even > wants to run some checks before tuning the default mode per CPU. That's an interesting idea, but it sounds like it wouldn't permit upgrading from NONE as a separate feature from upgrading from ASYNC. > I'm pretty sure the sync vs async decision is not a clear cut (e.g. sync > may still be slower but within a tolerable margin for certain > benchmarks). Leaving the decision to the hardware vendor, hard-coded in > the DT, is probably not the best option (nor is the MIDR). Some future > benchmark with a weird memory access pattern could expose slowness in > the sync mode and vendors will scramble on how to change the DT already > deployed (maybe they can do this OTA already). Perhaps we can let the default upgrading settings be specified via DT, and allow them to be overridden later via sysfs. That seems like the best of both worlds, since I think that in most cases DT should be enough, while the ability to override should satisfy the remaining cases. It also means that ACPI users have a way to use this feature until it gets added to the spec. By the way, if we want to allow upgrading from NONE in the future, or upgrading to asymmetric mode, perhaps we should spell the option "mte-upgrade-async" and have it take an argument in the range 0-3 (corresponding to the SCTLR_EL1.TCF bits) specifying the desired mode to upgrade to. Then we could have e.g. "mte-upgrade-none" later with the same semantics. Peter _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel