From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jason Gunthorpe Subject: Re: RFC on writel and writel_relaxed Date: Tue, 27 Mar 2018 08:24:01 -0600 Message-ID: <20180327142401.GC12318@ziepe.ca> References: <20180326202545.GB15554@ziepe.ca> <20180326210951.GD15554@ziepe.ca> <1522101717.7364.14.camel@kernel.crashing.org> <20180326222756.GJ15554@ziepe.ca> <20180327094159.GA29373@arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20180327094159.GA29373@arm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: linuxppc-dev-bounces+glppe-linuxppc-embedded-2=m.gmane.org@lists.ozlabs.org Sender: "Linuxppc-dev" To: Will Deacon Cc: "Paul E. McKenney" , Arnd Bergmann , "linux-rdma@vger.kernel.org" , Sinan Kaya , David Laight , Oliver , Alexander Duyck , "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" List-Id: linux-rdma@vger.kernel.org On Tue, Mar 27, 2018 at 10:42:00AM +0100, Will Deacon wrote: > On Tue, Mar 27, 2018 at 09:56:47AM +0200, Arnd Bergmann wrote: > > On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > > > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: > > >> On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > > > > I even see patches adding wmb() based on actual observed memory > > > corruption during testing on Intel: > > > > > > https://patchwork.kernel.org/patch/10177207/ > > > > > > So you think all of this is unnecessary and writel is totally strongly > > > ordered, even on multi-socket Intel? > > > > This example adds a wmb() between two writes to a coherent DMA > > area, it is definitely required there. I'm pretty sure I've never seen > > any bug reports pointing to a missing wmb() between memory > > and MMIO write accesses, but if you remember seeing them in the > > list, maybe you can look again for some evidence of something going > > wrong on x86 without it? > > If this is just about ordering accesses to coherent DMA, then using > dma_wmb() instead will be much better performance on arm/arm64. dma_wmb() is a NOP on x86, it was tested anyhow and didn't help this case.. Confusing, but probably not relevant to this discussion. Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pl0-x242.google.com (mail-pl0-x242.google.com [IPv6:2607:f8b0:400e:c01::242]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 409YD63w3CzF28X for ; Wed, 28 Mar 2018 01:24:05 +1100 (AEDT) Received: by mail-pl0-x242.google.com with SMTP id f5-v6so14160776plj.13 for ; Tue, 27 Mar 2018 07:24:05 -0700 (PDT) Date: Tue, 27 Mar 2018 08:24:01 -0600 From: Jason Gunthorpe To: Will Deacon Cc: Arnd Bergmann , Benjamin Herrenschmidt , Sinan Kaya , David Laight , Oliver , "open list:LINUX FOR POWERPC (32-BIT AND 64-BIT)" , "linux-rdma@vger.kernel.org" , Alexander Duyck , "Paul E. McKenney" Subject: Re: RFC on writel and writel_relaxed Message-ID: <20180327142401.GC12318@ziepe.ca> References: <20180326202545.GB15554@ziepe.ca> <20180326210951.GD15554@ziepe.ca> <1522101717.7364.14.camel@kernel.crashing.org> <20180326222756.GJ15554@ziepe.ca> <20180327094159.GA29373@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii In-Reply-To: <20180327094159.GA29373@arm.com> List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Tue, Mar 27, 2018 at 10:42:00AM +0100, Will Deacon wrote: > On Tue, Mar 27, 2018 at 09:56:47AM +0200, Arnd Bergmann wrote: > > On Tue, Mar 27, 2018 at 12:27 AM, Jason Gunthorpe wrote: > > > On Tue, Mar 27, 2018 at 09:01:57AM +1100, Benjamin Herrenschmidt wrote: > > >> On Mon, 2018-03-26 at 17:46 -0400, Sinan Kaya wrote: > > > > > > I even see patches adding wmb() based on actual observed memory > > > corruption during testing on Intel: > > > > > > https://patchwork.kernel.org/patch/10177207/ > > > > > > So you think all of this is unnecessary and writel is totally strongly > > > ordered, even on multi-socket Intel? > > > > This example adds a wmb() between two writes to a coherent DMA > > area, it is definitely required there. I'm pretty sure I've never seen > > any bug reports pointing to a missing wmb() between memory > > and MMIO write accesses, but if you remember seeing them in the > > list, maybe you can look again for some evidence of something going > > wrong on x86 without it? > > If this is just about ordering accesses to coherent DMA, then using > dma_wmb() instead will be much better performance on arm/arm64. dma_wmb() is a NOP on x86, it was tested anyhow and didn't help this case.. Confusing, but probably not relevant to this discussion. Jason