From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.7 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 998D9C2B9F4 for ; Tue, 22 Jun 2021 09:13:18 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 6671A61042 for ; Tue, 22 Jun 2021 09:13:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6671A61042 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=HJYB12ApowRO+CrsQBRZQjoSFJdtPSF5dGWWxM3AQF8=; b=dgWekrNT6t5u/H s98aCY/86c/ywxHiw4FaPdSsknJUoVcgFQNCRCDbMJkCBce9CMDgq92EyowgpFD39lACoOKYM+MSU xRA9mIuhiwOqn7lR5WHAssJY+prvB7qmtMMgSoRQ1l9TSr7CanE62zYGaOSm5FFYrae4KlZgLi+zI k/Av6aY/0i8+2idlH+uz9MDzPyKt1n0V7c11vtYdG6iOENxdgkWbFQDh5zgOj5vEEvFFjHqPvQ7CP jkz7/RlMKXtBTcOHEFMGyjkS+bW7N99ywWzO27pqxwRVZNKdz/S0ox0tZ3JCEoQaAerggzFsGbpmb Ut28T1082IHLwW/IX2fA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvcRi-006NEB-Cx; Tue, 22 Jun 2021 09:11:50 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvcRe-006NDj-6C for linux-arm-kernel@lists.infradead.org; Tue, 22 Jun 2021 09:11:47 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id 79EF26113D; Tue, 22 Jun 2021 09:11:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1624353105; bh=cONpC9c9hNBwAC56ta0TA+5m+cae0wExAVA9de5ufzE=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=rX3cGsnmteqdHKGQUWhBP81TPRHn1BtmcnwPoXuVDZWFrP6ScJqL6LmmZG4b9L+by YCAazLtGuJDyiEs//R3oUI+zADE6OcaNhYdQTV/WUp7LtxfceWDilFI5kor4oqC/jJ vOTXMGYcOmmmN2/VnTLM3lNiZ8poglMSSyrgqcu9yjBx/s+jUbxxXUAT9enlcMLJnW wcI2iWJNo6sw/Fn+sudZjpdFsnaiLPN2NJBhoM+jgDeBFQmYqt7UUjKqGWEE/bueub uPa2kaPqfnaRSeIBa951dc6Mhdt5vbsv930G1JBUtk+JBEs8z/68FAActKS+elDb9S 46PIfO4yUKRKA== Date: Tue, 22 Jun 2021 10:11:41 +0100 From: Will Deacon To: Frank Li Cc: Catalin Marinas , Zhi Li , Shenwei Wang , Han Xu , Nitin Garg , Jason Liu , "linux-arm-kernel@lists.infradead.org" Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit barriers in default I/O accessors Message-ID: <20210622091140.GA30677@willie-the-truck> References: <20210617172528.GA24813@willie-the-truck> <20210617174131.GC24813@willie-the-truck> <20210617214012.GA25403@willie-the-truck> <20210621162641.GA29595@willie-the-truck> <20210621165941.GB29595@willie-the-truck> <20210621181326.GD29713@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210622_021146_310625_69B991A2 X-CRM114-Status: GOOD ( 38.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Jun 21, 2021 at 09:32:22PM +0000, Frank Li wrote: > > > > -----Original Message----- > > From: Will Deacon > > Sent: Monday, June 21, 2021 1:13 PM > > To: Frank Li > > Cc: Catalin Marinas ; Zhi Li ; > > Shenwei Wang ; Han Xu ; Nitin Garg > > ; Jason Liu ; linux-arm- > > kernel@lists.infradead.org > > Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit barriers > > in default I/O accessors > > > > Caution: EXT Email > > > > On Mon, Jun 21, 2021 at 05:56:43PM +0000, Frank Li wrote: > > > > > > > > > > -----Original Message----- > > > > From: Will Deacon > > > > Sent: Monday, June 21, 2021 12:00 PM > > > > To: Frank Li > > > > Cc: Catalin Marinas ; Zhi Li > > ; > > > > Shenwei Wang ; Han Xu ; Nitin > > Garg > > > > ; Jason Liu ; linux-arm- > > > > kernel@lists.infradead.org > > > > Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit > > barriers > > > > in default I/O accessors > > > > > > > > Caution: EXT Email > > > > > > > > On Mon, Jun 21, 2021 at 05:26:41PM +0100, Will Deacon wrote: > > > > > On Mon, Jun 21, 2021 at 04:11:57PM +0000, Frank Li wrote: > > > > > > > Oh, interesting. Maybe this is a case where OSH vs SY actually > > makes > > > > a > > > > > > > difference. I'm not quite sure what it means for the coherency of > > > > normal, > > > > > > > non-cacheable accesses (which are outer-shareable) so that > > probably > > > > needs a > > > > > > > bit more thought. > > > > > > > > > > > > > > Can you confirm that the issue *does* still occur if you use > > dmb(osh) > > > > > > > instead of dmb(oshst), please? > > > > > > > > > > > > After get ARM support > > > > > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fservices. > > > > > > arm.com%2Fsupport%2Fs%2Fcase%2F5003t00001RuJHw&data=04%7C01%7Cfrank.li% > > > > > > 40nxp.com%7Ca319ac5213a14aa6bb2508d934d5facc%7C686ea1d3bc2b4c6fa92cd99c5c30 > > > > > > 1635%7C0%7C0%7C637598915908588560%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM > > > > > > DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6%2F%2FK > > > > ScsCmnUgNPnzcvyjRrOLjLVPrHtbVgI3J959U%2BQ%3D&reserved=0, > > > > > > This issue have some progress. > > > > > > > > > > > > Our system configure SYSBARDISABLE = 0x0, So ARM core barrier > > propagate > > > > to CCI-400 > > > > > > > > > > > > Our DMA and USB is located below downstream of CCI-400. So USB or > > DMA > > > > is located > > > > > > in system shared domain. Only use dmb(st), CCI-400 wait for > > previous > > > > transaction > > > > > > Complete. When dma(osh), the response is sent when snoop responses > > are > > > > received for > > > > > > all earlier transactions. CCI-400 don't wait for previous write > > finish. > > > > > > > > > > Thanks for following up. I'll cook a patch to fix this... > > > > > > > > ... and in doing so, I realised I still have a question about this. > > > > > > > > If a CPU is writing to a zero-initialised non-cacheable buffer in > > memory > > > > and does something like: > > > > > > > > buffer[0] = 1; > > > > dma_wmb(); // DMB OSHST > > > > buffer[64] = 1; > > > > > > > > would a non-coherent device reading this be able to see buffer[64] == 1 > > > > but buffer[0] = 0? In other words, do we need to upgrade the dmb_* > > barriers > > > > as well as the I/O accessors, or are they still ordered by the bus > > fabric > > > > because all of the accesses are going to the DDR? > > > > > > I think re-order is possible. According to my understanding, > > > If cci ack dmb(oshst), the follow order is not guaranteed if no address > > overlap > > > for normal memory. > > > > Hmm, so that's a bit rubbish because it means that > > load-acquire/store-release to non-cacheable memory will *not* create order > > for non-coherent devices, as the memory type is outer-shareable :/ > > > > So rewriting the above as: > > > > buffer[0] = 1; > > smp_store_release(&buffer[64], 1); > > > > wouldn't be ordered either. > > > > Can you confirm that it is the case, please? > > I have not test case, which can test it directly. > I supposed smp_mb is not work for no-coherent dma master. > If want dma master see order, need dma_wmb(). I think you had a support case open with Arm [1] which I'm not able to access -- please can you ask them about the two examples above? Will [1] https://services.arm.com/support/s/case/5003t00001RuJHw _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel