From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from ale.deltatee.com (ale.deltatee.com [207.54.116.67]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E6635222DDBF7 for ; Mon, 5 Mar 2018 10:03:58 -0800 (PST) References: <20180228234006.21093-1-logang@deltatee.com> <20180228234006.21093-8-logang@deltatee.com> <20180305160004.GA30975@localhost.localdomain> <3f56c76d-6a5c-7c2f-5442-c9209749b598@deltatee.com> From: Logan Gunthorpe Message-ID: <645c0d69-8942-5eb5-99ae-04ae385a32fa@deltatee.com> Date: Mon, 5 Mar 2018 11:09:52 -0700 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US Subject: Re: [PATCH v2 07/10] nvme-pci: Use PCI p2pmem subsystem to manage the CMB List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Sinan Kaya , Keith Busch , Oliver Cc: Jens Axboe , "linux-nvdimm@lists.01.org" , linux-rdma@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Alex Williamson , Jason Gunthorpe , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Benjamin Herrenschmidt , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig List-ID: On 05/03/18 11:02 AM, Sinan Kaya wrote: > writel has a barrier inside on ARM64. > > https://elixir.bootlin.com/linux/latest/source/arch/arm64/include/asm/io.h#L143 Yes, and no barrier inside memcpy_toio as it uses __raw_writes. This should be sufficient as we are only accessing addresses that look like memory and have no side effects (those enabling doorbell accesses may need to worry about this though). Typically, what could happen, in this case, is the CPU would issue writes to the BAR normally and the next time it programmed the DMA engine it would flush everything via the flush in writel. > Why do you need another barrier? We don't. Thanks, Logan _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: To: Sinan Kaya , Keith Busch , Oliver Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-nvme@lists.infradead.org, linux-rdma@vger.kernel.org, "linux-nvdimm@lists.01.org" , linux-block@vger.kernel.org, Jens Axboe , Benjamin Herrenschmidt , Alex Williamson , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , Jason Gunthorpe , Bjorn Helgaas , Max Gurtovoy , Christoph Hellwig References: <20180228234006.21093-1-logang@deltatee.com> <20180228234006.21093-8-logang@deltatee.com> <20180305160004.GA30975@localhost.localdomain> <3f56c76d-6a5c-7c2f-5442-c9209749b598@deltatee.com> From: Logan Gunthorpe Message-ID: <645c0d69-8942-5eb5-99ae-04ae385a32fa@deltatee.com> Date: Mon, 5 Mar 2018 11:09:52 -0700 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Subject: Re: [PATCH v2 07/10] nvme-pci: Use PCI p2pmem subsystem to manage the CMB List-ID: On 05/03/18 11:02 AM, Sinan Kaya wrote: > writel has a barrier inside on ARM64. > > https://elixir.bootlin.com/linux/latest/source/arch/arm64/include/asm/io.h#L143 Yes, and no barrier inside memcpy_toio as it uses __raw_writes. This should be sufficient as we are only accessing addresses that look like memory and have no side effects (those enabling doorbell accesses may need to worry about this though). Typically, what could happen, in this case, is the CPU would issue writes to the BAR normally and the next time it programmed the DMA engine it would flush everything via the flush in writel. > Why do you need another barrier? We don't. Thanks, Logan From mboxrd@z Thu Jan 1 00:00:00 1970 From: logang@deltatee.com (Logan Gunthorpe) Date: Mon, 5 Mar 2018 11:09:52 -0700 Subject: [PATCH v2 07/10] nvme-pci: Use PCI p2pmem subsystem to manage the CMB In-Reply-To: References: <20180228234006.21093-1-logang@deltatee.com> <20180228234006.21093-8-logang@deltatee.com> <20180305160004.GA30975@localhost.localdomain> <3f56c76d-6a5c-7c2f-5442-c9209749b598@deltatee.com> Message-ID: <645c0d69-8942-5eb5-99ae-04ae385a32fa@deltatee.com> On 05/03/18 11:02 AM, Sinan Kaya wrote: > writel has a barrier inside on ARM64. > > https://elixir.bootlin.com/linux/latest/source/arch/arm64/include/asm/io.h#L143 Yes, and no barrier inside memcpy_toio as it uses __raw_writes. This should be sufficient as we are only accessing addresses that look like memory and have no side effects (those enabling doorbell accesses may need to worry about this though). Typically, what could happen, in this case, is the CPU would issue writes to the BAR normally and the next time it programmed the DMA engine it would flush everything via the flush in writel. > Why do you need another barrier? We don't. Thanks, Logan