From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4EEEEC433E6 for ; Wed, 20 Jan 2021 18:52:22 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 07BCA22473 for ; Wed, 20 Jan 2021 18:52:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2392203AbhATSvI (ORCPT ); Wed, 20 Jan 2021 13:51:08 -0500 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:55430 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390729AbhATSMF (ORCPT ); Wed, 20 Jan 2021 13:12:05 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1611166326; x=1642702326; h=subject:to:cc:references:from:message-id:date: mime-version:in-reply-to:content-transfer-encoding; bh=0FqY4Y9ekqo9Njl4eE4yftVIgAdbgRXuwSrgrFbU1KY=; b=n2oArYMsNAxf9OPtAtzAl8Tp/o3kGHhlSgRm4dtYt1xEKhBpGNFxbqPQ +ALRxgjUqIY1L2kMRuQWfnlDarH5DzU5khCRacimSM7qsAwatZcKyyICg Yz1MHYQVhpD9p3yCdb5IPClfCb/MqmtiXxBInCcWsp8FvDI9DrNJyEwzg Q=; X-IronPort-AV: E=Sophos;i="5.79,361,1602547200"; d="scan'208";a="112258151" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 20 Jan 2021 18:11:17 +0000 Received: from EX13MTAUWC002.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com (Postfix) with ESMTPS id CC0E3A21D6; Wed, 20 Jan 2021 18:11:15 +0000 (UTC) Received: from EX13D20UWC001.ant.amazon.com (10.43.162.244) by EX13MTAUWC002.ant.amazon.com (10.43.162.240) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 20 Jan 2021 18:11:15 +0000 Received: from edge-m1-r3-201.e-iad16.amazon.com (10.43.161.179) by EX13D20UWC001.ant.amazon.com (10.43.162.244) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 20 Jan 2021 18:10:35 +0000 Subject: Re: [RFC PATCH 3/7] arm64: mm: use nGnRnE instead of nGnRE on Apple processors To: Mohamed Mediouni CC: , Catalin Marinas , Will Deacon , Mark Rutland , Marc Zyngier , Hector Martin , , Stan Skowronek References: <20210120132717.395873-1-mohamed.mediouni@caramail.com> <20210120132717.395873-4-mohamed.mediouni@caramail.com> <17579DC5-13AF-48E1-9B72-91B14AB485FA@caramail.com> From: Alexander Graf Message-ID: <6047872c-8a40-4159-aeb6-ea439bfafd82@amazon.com> Date: Wed, 20 Jan 2021 19:10:33 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <17579DC5-13AF-48E1-9B72-91B14AB485FA@caramail.com> Content-Language: en-US X-Originating-IP: [10.43.161.179] X-ClientProxiedBy: EX13D49UWC002.ant.amazon.com (10.43.162.215) To EX13D20UWC001.ant.amazon.com (10.43.162.244) Content-Type: text/plain; charset="windows-1252"; format="flowed" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 20.01.21 19:06, Mohamed Mediouni wrote: > = >> On 20 Jan 2021, at 17:47, Alexander Graf wrote: >> >> On 20.01.21 14:27, Mohamed Mediouni wrote: >>> Use nGnRnE instead of nGnRE on Apple SoCs to workaround a serious hardw= are quirk. >>> On Apple processors, writes using the nGnRE device memory type get drop= ped in flight, >>> getting to nowhere. >>> Signed-off-by: Stan Skowronek >>> Signed-off-by: Mohamed Mediouni >>> --- >>> arch/arm64/mm/proc.S | 26 ++++++++++++++++++++++++++ >>> 1 file changed, 26 insertions(+) >>> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S >>> index 1f7ee8c8b7b8..06436916f137 100644 >>> --- a/arch/arm64/mm/proc.S >>> +++ b/arch/arm64/mm/proc.S >>> @@ -51,6 +51,25 @@ >>> #define TCR_KASAN_HW_FLAGS 0 >>> #endif >>> +#ifdef CONFIG_ARCH_APPLE >> >> Is there any particular reason for this #ifdef? >> >> >> Alex >> > Not a particular reason, as we explicitly check for the implementer ID. H= owever, > without CONFIG_ARCH_APPLE, other parts of the support for Apple CPUs > will not be available anyway. The ifdef below for code looks ok to me, I'm explicitly wondering why = you're guarding the #define :) Alex >>> + >>> +/* >>> + * Apple cores appear to black-hole writes done with nGnRE. >>> + * We settled on a work-around that uses MAIR vs changing every single= user of >>> + * nGnRE across the arm64 code. >>> + */ >>> + >>> +#define MAIR_EL1_SET_APPLE \ >>> + (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRE) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL_TAGGED)) >>> + >>> +#endif >>> + >>> /* >>> * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal m= emory and >>> * changed during __cpu_setup to Normal Tagged if the system supports= MTE. >>> @@ -432,6 +451,13 @@ SYM_FUNC_START(__cpu_setup) >>> * Memory region attributes >>> */ >>> mov_q x5, MAIR_EL1_SET >>> +#ifdef CONFIG_ARCH_APPLE >>> + mrs x0, MIDR_EL1 >>> + lsr w0, w0, #24 >>> + mov_q x1, MAIR_EL1_SET_APPLE >>> + cmp x0, #0x61 // 0x61 =3D Implementer: A= pple >>> + csel x5, x1, x5, eq >>> +#endif >>> #ifdef CONFIG_ARM64_MTE >>> mte_tcr .req x20 >>> -- >>> 2.29.2 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-14.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_ALL,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 51059C433E0 for ; Wed, 20 Jan 2021 18:13:25 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0F302233FA for ; Wed, 20 Jan 2021 18:13:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F302233FA Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Type: Content-Transfer-Encoding:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Sh/BvI2IbgegMYNEYyA3PHXkd6ooxn+Eo3yOhQJ5rN0=; b=MTpYPujL/nLeeN5SIy4rnwdKp yBFFbn2AlL1JgejbBnxqn6GgjHpExO1YJ3GlFl25FqyNHcqovRm9sCMqVZKYHdAYMdbrA0vvrl2K1 +ajIlW1X89/jA35hG+7ftn6G+Zb6BgHkiaefTfCQ5MOI0wA0KtN+Rbcx3ViAL1YzVDasgGTvSnJqr 81Yko5IFcwp1I33q4xawEKxxmm/GNl2fdQWydEbiydXGkxslbLvtgd3HxCDjy6r7DRZSkdtfKoP6P AFMtgsUXfyKYI5lMQDuXp0p0nzomB2WkjLlqzsw6vXFxU0qbi+/kBUakgRo5WNThmrvCX/0swiawD 9XdXCt05A==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2HxC-00068B-SL; Wed, 20 Jan 2021 18:11:38 +0000 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1l2Hx0-000659-E8 for linux-arm-kernel@lists.infradead.org; Wed, 20 Jan 2021 18:11:27 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1611166287; x=1642702287; h=subject:to:cc:references:from:message-id:date: mime-version:in-reply-to:content-transfer-encoding; bh=0FqY4Y9ekqo9Njl4eE4yftVIgAdbgRXuwSrgrFbU1KY=; b=hnFLQo1SAbxcs3Nl0IA08URYqIFYvdJEmEBDIFZV+5gmGpKTieVi+DHJ bdTxLWADsIA5/32toAuzHeCa334gFgo37ZFXiLBTKBqjF+IXc49DfcduZ FNlVCx5ssd1kE6Lo5kE314IhbfwqQX7uJYsTGbhIXGp4LqDc8nk3xFCvq w=; X-IronPort-AV: E=Sophos;i="5.79,361,1602547200"; d="scan'208";a="112258151" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 20 Jan 2021 18:11:17 +0000 Received: from EX13MTAUWC002.ant.amazon.com (iad12-ws-svc-p26-lb9-vlan2.iad.amazon.com [10.40.163.34]) by email-inbound-relay-1d-474bcd9f.us-east-1.amazon.com (Postfix) with ESMTPS id CC0E3A21D6; Wed, 20 Jan 2021 18:11:15 +0000 (UTC) Received: from EX13D20UWC001.ant.amazon.com (10.43.162.244) by EX13MTAUWC002.ant.amazon.com (10.43.162.240) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 20 Jan 2021 18:11:15 +0000 Received: from edge-m1-r3-201.e-iad16.amazon.com (10.43.161.179) by EX13D20UWC001.ant.amazon.com (10.43.162.244) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 20 Jan 2021 18:10:35 +0000 Subject: Re: [RFC PATCH 3/7] arm64: mm: use nGnRnE instead of nGnRE on Apple processors To: Mohamed Mediouni References: <20210120132717.395873-1-mohamed.mediouni@caramail.com> <20210120132717.395873-4-mohamed.mediouni@caramail.com> <17579DC5-13AF-48E1-9B72-91B14AB485FA@caramail.com> From: Alexander Graf Message-ID: <6047872c-8a40-4159-aeb6-ea439bfafd82@amazon.com> Date: Wed, 20 Jan 2021 19:10:33 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <17579DC5-13AF-48E1-9B72-91B14AB485FA@caramail.com> Content-Language: en-US X-Originating-IP: [10.43.161.179] X-ClientProxiedBy: EX13D49UWC002.ant.amazon.com (10.43.162.215) To EX13D20UWC001.ant.amazon.com (10.43.162.244) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210120_131126_641432_64852B44 X-CRM114-Status: GOOD ( 16.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Mark Rutland , Catalin Marinas , Hector Martin , linux-kernel@vger.kernel.org, Marc Zyngier , Will Deacon , linux-arm-kernel@lists.infradead.org, Stan Skowronek Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 20.01.21 19:06, Mohamed Mediouni wrote: > >> On 20 Jan 2021, at 17:47, Alexander Graf wrote: >> >> On 20.01.21 14:27, Mohamed Mediouni wrote: >>> Use nGnRnE instead of nGnRE on Apple SoCs to workaround a serious hardware quirk. >>> On Apple processors, writes using the nGnRE device memory type get dropped in flight, >>> getting to nowhere. >>> Signed-off-by: Stan Skowronek >>> Signed-off-by: Mohamed Mediouni >>> --- >>> arch/arm64/mm/proc.S | 26 ++++++++++++++++++++++++++ >>> 1 file changed, 26 insertions(+) >>> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S >>> index 1f7ee8c8b7b8..06436916f137 100644 >>> --- a/arch/arm64/mm/proc.S >>> +++ b/arch/arm64/mm/proc.S >>> @@ -51,6 +51,25 @@ >>> #define TCR_KASAN_HW_FLAGS 0 >>> #endif >>> +#ifdef CONFIG_ARCH_APPLE >> >> Is there any particular reason for this #ifdef? >> >> >> Alex >> > Not a particular reason, as we explicitly check for the implementer ID. However, > without CONFIG_ARCH_APPLE, other parts of the support for Apple CPUs > will not be available anyway. The ifdef below for code looks ok to me, I'm explicitly wondering why you're guarding the #define :) Alex >>> + >>> +/* >>> + * Apple cores appear to black-hole writes done with nGnRE. >>> + * We settled on a work-around that uses MAIR vs changing every single user of >>> + * nGnRE across the arm64 code. >>> + */ >>> + >>> +#define MAIR_EL1_SET_APPLE \ >>> + (MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRnE) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_nGnRnE, MT_DEVICE_nGnRE) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_DEVICE_GRE, MT_DEVICE_GRE) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_NC, MT_NORMAL_NC) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL_WT, MT_NORMAL_WT) | \ >>> + MAIR_ATTRIDX(MAIR_ATTR_NORMAL, MT_NORMAL_TAGGED)) >>> + >>> +#endif >>> + >>> /* >>> * Default MAIR_EL1. MT_NORMAL_TAGGED is initially mapped as Normal memory and >>> * changed during __cpu_setup to Normal Tagged if the system supports MTE. >>> @@ -432,6 +451,13 @@ SYM_FUNC_START(__cpu_setup) >>> * Memory region attributes >>> */ >>> mov_q x5, MAIR_EL1_SET >>> +#ifdef CONFIG_ARCH_APPLE >>> + mrs x0, MIDR_EL1 >>> + lsr w0, w0, #24 >>> + mov_q x1, MAIR_EL1_SET_APPLE >>> + cmp x0, #0x61 // 0x61 = Implementer: Apple >>> + csel x5, x1, x5, eq >>> +#endif >>> #ifdef CONFIG_ARM64_MTE >>> mte_tcr .req x20 >>> -- >>> 2.29.2 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel