From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9362FC48BE5 for ; Mon, 21 Jun 2021 18:18:07 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6772D6115B for ; Mon, 21 Jun 2021 18:18:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6772D6115B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=nxp.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=si5WZIcR/9xd3DFCjUUvLhrs4rygzNwg6pfEmUChbpE=; b=sHdKW8QmxMWJTi lrfvkhHu95KVeW564+di98TatiSWBYJVAPQ73Ml5TrwJDWebQvgehmlHjmzHgjhBNczT5TpOB0QV8 x1gmgWFNOocyV8/WapOyvnb9Js99+jGUs357Ib0CuLNX6WGYK6dPdd/hTosAup1Z45Op60IkmmoKg xu+b4ZmqUAmAd3WcHtWVhAu/EVOGrMOpX431OL08sGyWHxbd4Po0eMwEB1cb+ackiFdT7h36OsDbr 1IaweG02qxJzLZDuzMijjfp9sBTBo4A5MdZg8qTNLLM8z9zEl0C1Ux0RcwGKVRXpeaB0vKpfftZvu /y2SNndCgFjaL5f2+7eg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvORy-004ViW-K1; Mon, 21 Jun 2021 18:15:11 +0000 Received: from mail-eopbgr60066.outbound.protection.outlook.com ([40.107.6.66] helo=EUR04-DB3-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvOAB-004MUI-Vc for linux-arm-kernel@lists.infradead.org; Mon, 21 Jun 2021 17:56:51 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=R4L+QtJ4QBG3l/Ir+uSLvttChb8wGpZRuTntRJRrD+8kQIHfwdufuP33hmuFqbbTxmllkthhBgI1Yr+/N3FEz3ne7Z8FwmYloQ5CQbY45CkMgZBtqy7erluV0ghwCbZeZHT29iNG0EGx4P+MWy1c09tj8/37I6d+gWQLh9QUYBOlJksWHV+iIJOwCDBOex7b+3/B3FeomGRGHjAMVvoDPZ92HH2rrJwDxGd/R3lvenQpIHibTJVcE/6irUH6dM7hGhUJltqCkqx0U/GBBmi4a8e+0usMmqpbSrgAFlEumEb+jdrrGobCJAVggwHcUbJ7FX+a2JkLE8s/N27eDgUu3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lie2CkHS6yTuKep/tT1zzwRQ7k741yqwTsWEQaD5RCg=; b=U6YV2ReBsz3YxnsEET4Dt2763iJqHu9agvctcvC3jQJQQlJg4/inFsFR8u5hafmRTgaFkiDyb5ySTq4gl2t3OBgCIyMwHktYHx1+u3r4+yPIAbpvTaiX3tHWD4VzvV82sRDrqd3LDMMZ8A41aHcFiOQj3GlR5TkDP2oCxZ/NgnjnTSsNzaQ1JpEQIXSzhiONWW841W3GJkq5dv4IcwbJfXSH4m7STzXANfU2TqXifu7+gJpFmzwqu6heC9lmaR7GwDY8CapZm/soMIALCHj8WIoCgGwMgfU/GTNP66JAb/Jets9qQxLhvZsJv0PwlluUV3bG6jLHoEg85hVC9U5PRA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nxp.com; dmarc=pass action=none header.from=nxp.com; dkim=pass header.d=nxp.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nxp.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lie2CkHS6yTuKep/tT1zzwRQ7k741yqwTsWEQaD5RCg=; b=Li7K0+W2oNBgfL6e+qu1xFvTiR7NC3vcB1oE7XqYPRmS2QxASg4pmeiodgIARIz8vpq9RVZA5ktBBBFqsj7p7M36/LXFYSTF94cYEEMR/kiS3c5lksk3I5nL5iCq3HgB1eO04hfipnNikiiW600zgwIpWQaaxHrqLFReu5PQSlU= Received: from AS8PR04MB8500.eurprd04.prod.outlook.com (2603:10a6:20b:343::14) by AS8PR04MB8834.eurprd04.prod.outlook.com (2603:10a6:20b:42d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4242.18; Mon, 21 Jun 2021 17:56:43 +0000 Received: from AS8PR04MB8500.eurprd04.prod.outlook.com ([fe80::2ca8:978f:5800:3e9a]) by AS8PR04MB8500.eurprd04.prod.outlook.com ([fe80::2ca8:978f:5800:3e9a%7]) with mapi id 15.20.4242.023; Mon, 21 Jun 2021 17:56:43 +0000 From: Frank Li To: Will Deacon CC: Catalin Marinas , Zhi Li , Shenwei Wang , Han Xu , Nitin Garg , Jason Liu , "linux-arm-kernel@lists.infradead.org" Subject: RE: [EXT] Re: The problem about arm64: io: Relax implicit barriers in default I/O accessors Thread-Topic: [EXT] Re: The problem about arm64: io: Relax implicit barriers in default I/O accessors Thread-Index: Addi4iZxCs/FNCdNS+mEtJE2ixnUpwAAxl6AAB1xwAAAEK9FAAAAj6YAAATT00AAA4IFAAC9jG0AAACrFQAAAScLAAAAEzdQ Date: Mon, 21 Jun 2021 17:56:43 +0000 Message-ID: References: <20210617092744.GB6314@arm.com> <20210617172528.GA24813@willie-the-truck> <20210617174131.GC24813@willie-the-truck> <20210617214012.GA25403@willie-the-truck> <20210621162641.GA29595@willie-the-truck> <20210621165941.GB29595@willie-the-truck> In-Reply-To: <20210621165941.GB29595@willie-the-truck> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=none action=none header.from=nxp.com; x-originating-ip: [72.177.87.193] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: 35ae6b75-0511-468a-b0c8-08d934ddeddd x-ms-traffictypediagnostic: AS8PR04MB8834: x-ms-exchange-transport-forked: True x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:10000; x-ms-exchange-senderadcheck: 1 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: VhNcwYPth6JQOM6SiaLgHe4mzg/z6hgdpCEr4b+9uGhO1HSAX+EGUZFNHVta5N+hDOKdas2FGIdGlj7TT0pVTtXmbScCDuxPiSYFKfmv59pWNH9vgGB2ACT0KoIixIWBVRS1vS4oI8oRbudlpcZDATuE+GcYF+dSMHYhWW68CZXaHrbyvSVWDOvUNkXX9sISNIMsq535PvEH/D4Rso3sLRr5wHnkCFWkKlhiZv1rWQ8vB6FNEQ5a5tDLj4BOFNVsBwvrZQUoleaY+bVipp69Sdly2+6woxW/0eka7utlV2+s2pDYR2OxeGXB6DjnkmDICiWJipUm0S7sF/PMhHMJHvKuNPyS6wxvgfAHJrdtjCiDMxAiy1luyp3ZGZs8shB2c4ceFQectNvJmS02dBzzUKno/aZsNNPCH0wB20kYdUVDSoruGqkYRPIK+hMPaRCTSTkqg3D/c19rUeXNyJk5/jy00mnF5hDTmXQ1t5L3iGa2d69dDXDi02rh9f4G6exaj8bNmA2b6di4Z6XrKyDISrXzCdR2ZpMAgKiCjWNXX8ONt2aLaaebIbGGY4eZHDt2LfdiaHK9iEdUEXxa+M2abWP6L6wc4KCGNw1OZlcvOoqwce/V8BlBqlToNOLRpCjnQf5u0QZVNyD1MGN0G9S1tr2046U/VgpeIcFkGmWJJb+6B09B6vBxpJLK1RsjV17Gqm76HoCo+2zaA1M/jVO9Vn44flrRtpOeae91+14x04A= x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:AS8PR04MB8500.eurprd04.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(376002)(396003)(346002)(366004)(136003)(39850400004)(76116006)(64756008)(66556008)(66476007)(66946007)(66446008)(38100700002)(316002)(8936002)(122000001)(83380400001)(44832011)(45080400002)(5660300002)(53546011)(8676002)(33656002)(6506007)(9686003)(55016002)(86362001)(966005)(6916009)(7696005)(186003)(4326008)(2906002)(478600001)(71200400001)(26005)(54906003)(52536014); DIR:OUT; SFP:1101; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?FJp793ptLo6vhiVu+7Q41EGrfzzUpMNS0cyLWCkvF4o34mBtc37S7GBd+zGq?= =?us-ascii?Q?n+iqK6Gn6AUn9sp0fcMoIz7dZNOpN7mUMF8UjEVh5ddIEbcECvWd4QLo4LRI?= =?us-ascii?Q?woR2BI/wHNARzm/WdEusx3h9FFvDMRSUj6SmIZZRhZW7NmsqmFCelUWAZD6W?= =?us-ascii?Q?4rCGkPb9VlIQKag3ZXuMy03/n9wRA0ZahLWXEITbuRXRhwm1HlXt5pqWGFVN?= =?us-ascii?Q?pMDqaX42SOae9UWnd8N8ujoehBqwAFen+3Y6nXlp7S6GCkqD/6k+EV8Gaq7U?= =?us-ascii?Q?f4kjnoyNmZH8DOwEFI8vY2i5uXNJJ3JdjFMeIP4lMCKtDqcIA2JylXT+o+fd?= =?us-ascii?Q?ZEWiiCF2NtkhMI7og/0O97/kp95PonTOPvncz1h1YT3NtD85HQVT9jUP93xq?= =?us-ascii?Q?V11AD75m5tBt0WpKZmqZVBOfs4F8hciW0Hlf+dPwsVIJeJ64nRjrZGwwbX2k?= =?us-ascii?Q?v0j7rZVV2/aMhubYJD843pmDy76h1KK7ngGvBFBBuBtksv15fLNabMOGnWij?= =?us-ascii?Q?6PXSx3BQmYd8wFeMbut3maq2B5RJ8hLU5LZoko++o6CNnYcw4uOQuu3zrko2?= =?us-ascii?Q?qwt01Q8NN/r5i/oR4fSRrqKAW3G3dxLH1OHXWqqwofYPXcpU35n/c1v8VOdX?= =?us-ascii?Q?6IWwdVvvXEEAXAMsMYuac3BNatuwSiVfL9iQCGnwF8P0TrnftolMlX7PLTEF?= =?us-ascii?Q?0uKN13P/GV8XiQo33bpQxdnzMur0AJEP+eSt8KqTS4FdQDzFIt0Hh2d1tojw?= =?us-ascii?Q?iS62mHZNRnwHme1TrmRsHJ7w2niHdLSt+E7bbZ57UmUMYRsHM5ufVVz8jmB4?= =?us-ascii?Q?ovrNhhXcIlswQhmtXXp7ebMFGZBM1pxPJV23CZKjRasufWbaYGsX6Hopswul?= =?us-ascii?Q?GzKT0sqUmmprbiQ3rkvPsLtEE4UcWPLFk19jpseEBbqxmFdU2/Em5H8RLCU/?= =?us-ascii?Q?y6Pc38jaTmJnR94uUUD9AugRgAmHkRNBeiqE9R3dZhtJTI72biahi6BrhztU?= =?us-ascii?Q?prOlILU83zQmDJ+HWod09PsqDjHEJfFtR18BY9O5cw4beZtQcn+AuxlnY52k?= =?us-ascii?Q?xk2G1VF4JfAFmFtluj+mN4KRjm7hkIN6Nykbd1pVWyla11egpyTmV96dYGS+?= =?us-ascii?Q?9xXzcugwpESsRvoeDXhLtqg6Dfm3OZgOSJ4oeyz1T91w72nYs/nByDtcH/XG?= =?us-ascii?Q?e8o4RaxyPXdgznvshUDLqI/AezDkl6qz7PfYGEZobjV/82OwlSr+cS/f9rYF?= =?us-ascii?Q?7fYFcXFA708scd+tEnQIkkmYhlOxhnmtcScRxDcZgOLOxAGpt/4fPc90UwtM?= =?us-ascii?Q?oy3o7gehbVtnfMujZHY/2EA4?= MIME-Version: 1.0 X-OriginatorOrg: nxp.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: AS8PR04MB8500.eurprd04.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 35ae6b75-0511-468a-b0c8-08d934ddeddd X-MS-Exchange-CrossTenant-originalarrivaltime: 21 Jun 2021 17:56:43.1422 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: w28fQ/+T/yBWHTa0axOlOtTJ/XI2VDmJa6Bs+p5K9L9iMYlKDlwvLV4a8IqagG4m1NLeiM8saaWo+pO6sA0TCA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: AS8PR04MB8834 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210621_105648_161390_94276BD1 X-CRM114-Status: GOOD ( 29.88 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > -----Original Message----- > From: Will Deacon > Sent: Monday, June 21, 2021 12:00 PM > To: Frank Li > Cc: Catalin Marinas ; Zhi Li ; > Shenwei Wang ; Han Xu ; Nitin Garg > ; Jason Liu ; linux-arm- > kernel@lists.infradead.org > Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit barriers > in default I/O accessors > > Caution: EXT Email > > On Mon, Jun 21, 2021 at 05:26:41PM +0100, Will Deacon wrote: > > On Mon, Jun 21, 2021 at 04:11:57PM +0000, Frank Li wrote: > > > > Oh, interesting. Maybe this is a case where OSH vs SY actually makes > a > > > > difference. I'm not quite sure what it means for the coherency of > normal, > > > > non-cacheable accesses (which are outer-shareable) so that probably > needs a > > > > bit more thought. > > > > > > > > Can you confirm that the issue *does* still occur if you use dmb(osh) > > > > instead of dmb(oshst), please? > > > > > > After get ARM support > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fservices. > arm.com%2Fsupport%2Fs%2Fcase%2F5003t00001RuJHw&data=04%7C01%7Cfrank.li% > 40nxp.com%7Ca319ac5213a14aa6bb2508d934d5facc%7C686ea1d3bc2b4c6fa92cd99c5c30 > 1635%7C0%7C0%7C637598915908588560%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM > DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6%2F%2FK > ScsCmnUgNPnzcvyjRrOLjLVPrHtbVgI3J959U%2BQ%3D&reserved=0, > > > This issue have some progress. > > > > > > Our system configure SYSBARDISABLE = 0x0, So ARM core barrier propagate > to CCI-400 > > > > > > Our DMA and USB is located below downstream of CCI-400. So USB or DMA > is located > > > in system shared domain. Only use dmb(st), CCI-400 wait for previous > transaction > > > Complete. When dma(osh), the response is sent when snoop responses are > received for > > > all earlier transactions. CCI-400 don't wait for previous write finish. > > > > Thanks for following up. I'll cook a patch to fix this... > > ... and in doing so, I realised I still have a question about this. > > If a CPU is writing to a zero-initialised non-cacheable buffer in memory > and does something like: > > buffer[0] = 1; > dma_wmb(); // DMB OSHST > buffer[64] = 1; > > would a non-coherent device reading this be able to see buffer[64] == 1 > but buffer[0] = 0? In other words, do we need to upgrade the dmb_* barriers > as well as the I/O accessors, or are they still ordered by the bus fabric > because all of the accesses are going to the DDR? I think re-order is possible. According to my understanding, If cci ack dmb(oshst), the follow order is not guaranteed if no address overlap for normal memory. A6.6.1 of AXI protocol spec. A write W1 must be ordered before a write W2 with the same ID, to the *same Memory location*, where W2 is received after W1 is received. > > Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel