From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753979AbdCPVtM (ORCPT ); Thu, 16 Mar 2017 17:49:12 -0400 Received: from mail.kernel.org ([198.145.29.136]:59710 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753068AbdCPVtK (ORCPT ); Thu, 16 Mar 2017 17:49:10 -0400 Date: Thu, 16 Mar 2017 16:48:44 -0500 From: Bjorn Helgaas To: Lorenzo Pieralisi Cc: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Wenrui Li , Gabriele Paoloni , Catalin Marinas , Shawn Lin , Will Deacon , Michal Simek , Thierry Reding , Tanmay Inamdar , linux-arch@vger.kernel.org, Pratyush Anand , Russell King , Jon Mason , Murali Karicheri , Arnd Bergmann , Bharat Kumar Gogada , Ray Jui , John Garry , Joao Pinto , Bjorn Helgaas , Mingkai Hu , Thomas Petazzoni , Jingoo Han , linux-kernel@vger.kernel.org, Stanimir Varbanov , Minghuan Lian , Zhou Wang , Roy Zang , "Luis R. Rodriguez" Subject: Re: [PATCH 02/20] PCI: fix pci_remap_iospace() remap attribute Message-ID: <20170316214844.GA17769@bhelgaas-glaptop.roam.corp.google.com> References: <20170227151436.18698-1-lorenzo.pieralisi@arm.com> <20170227151436.18698-3-lorenzo.pieralisi@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170227151436.18698-3-lorenzo.pieralisi@arm.com> User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [+cc Luis] On Mon, Feb 27, 2017 at 03:14:13PM +0000, Lorenzo Pieralisi wrote: > According to the PCI local bus specifications (Revision 3.0, 3.2.5), > I/O Address space transactions are non-posted. On architectures where > I/O space is implemented through a chunk of memory mapped space mapped > to PCI address space (ie IA64/ARM/ARM64) the memory mapping for the > region backing I/O Address Space transactions determines the I/O > transactions attributes (before the transactions actually reaches the > PCI bus where it is handled according to the PCI specifications). > > Current pci_remap_iospace() interface, that is used to map the PCI I/O > Address Space into virtual address space, use pgprot_device() as memory > attribute for the virtual address mapping, that in some architectures > (ie ARM64) provides non-cacheable but write bufferable mappings (ie > posted writes), which clash with the non-posted write behaviour for I/O > Address Space mandated by the PCI specifications. > > Update the prot ioremap_page_range() parameter in pci_remap_iospace() > to pgprot_noncached to ensure that the virtual mapping backing > I/O Address Space guarantee non-posted write transactions issued > when addressing I/O Address Space through the MMIO mapping. > > Signed-off-by: Lorenzo Pieralisi > Cc: Arnd Bergmann > Cc: Will Deacon > Cc: Bjorn Helgaas > Cc: Russell King > Cc: Catalin Marinas > --- > drivers/pci/pci.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index bd98674..bfb3c6e 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -3375,7 +3375,7 @@ int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr) > return -EINVAL; > > return ioremap_page_range(vaddr, vaddr + resource_size(res), phys_addr, > - pgprot_device(PAGE_KERNEL)); > + pgprot_noncached(PAGE_KERNEL)); pgprot_device() is equivalent to pgprot_noncached() on all arches except ARM64, and I trust you're doing the right thing on ARM64, so I'm fine with this from a PCI perspective. I do find this puzzling because I naively expected pgprot_noncached() to match up with ioremap_nocache(), and apparently it doesn't. For example, ARM64 ioremap_nocache() uses PROT_DEVICE_nGnRE, which doesn't match the MT_DEVICE_nGnRnE in pgprot_noncached(). The point of these patches is to use non-posted mappings. Apparently you can do that with pgprot_noncached() here, but ioremap_nocache() isn't enough for the config space mappings? I suppose that's a consequence of the pgprot_noncached() vs ioremap_nocache() mismatch, but this is all extremely confusing. > #else > /* this architecture does not have memory mapped I/O space, > so this function should never be called */ > -- > 2.10.0 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Return-Path: Date: Thu, 16 Mar 2017 16:48:44 -0500 From: Bjorn Helgaas To: Lorenzo Pieralisi Subject: Re: [PATCH 02/20] PCI: fix pci_remap_iospace() remap attribute Message-ID: <20170316214844.GA17769@bhelgaas-glaptop.roam.corp.google.com> References: <20170227151436.18698-1-lorenzo.pieralisi@arm.com> <20170227151436.18698-3-lorenzo.pieralisi@arm.com> MIME-Version: 1.0 In-Reply-To: <20170227151436.18698-3-lorenzo.pieralisi@arm.com> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Wenrui Li , Gabriele Paoloni , linux-pci@vger.kernel.org, Shawn Lin , Will Deacon , Russell King , Thierry Reding , Tanmay Inamdar , linux-arch@vger.kernel.org, Joao Pinto , Pratyush Anand , Michal Simek , Bharat Kumar Gogada , Murali Karicheri , Catalin Marinas , Arnd Bergmann , Jon Mason , Ray Jui , John Garry , Bjorn Helgaas , Mingkai Hu , linux-arm-kernel@lists.infradead.org, "Luis R. Rodriguez" , Thomas Petazzoni , Jingoo Han , linux-kernel@vger.kernel.org, Stanimir Varbanov , Minghuan Lian , Zhou Wang , Roy Zang Content-Type: text/plain; charset="us-ascii" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+bjorn=helgaas.com@lists.infradead.org List-ID: [+cc Luis] On Mon, Feb 27, 2017 at 03:14:13PM +0000, Lorenzo Pieralisi wrote: > According to the PCI local bus specifications (Revision 3.0, 3.2.5), > I/O Address space transactions are non-posted. On architectures where > I/O space is implemented through a chunk of memory mapped space mapped > to PCI address space (ie IA64/ARM/ARM64) the memory mapping for the > region backing I/O Address Space transactions determines the I/O > transactions attributes (before the transactions actually reaches the > PCI bus where it is handled according to the PCI specifications). > > Current pci_remap_iospace() interface, that is used to map the PCI I/O > Address Space into virtual address space, use pgprot_device() as memory > attribute for the virtual address mapping, that in some architectures > (ie ARM64) provides non-cacheable but write bufferable mappings (ie > posted writes), which clash with the non-posted write behaviour for I/O > Address Space mandated by the PCI specifications. > > Update the prot ioremap_page_range() parameter in pci_remap_iospace() > to pgprot_noncached to ensure that the virtual mapping backing > I/O Address Space guarantee non-posted write transactions issued > when addressing I/O Address Space through the MMIO mapping. > > Signed-off-by: Lorenzo Pieralisi > Cc: Arnd Bergmann > Cc: Will Deacon > Cc: Bjorn Helgaas > Cc: Russell King > Cc: Catalin Marinas > --- > drivers/pci/pci.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index bd98674..bfb3c6e 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -3375,7 +3375,7 @@ int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr) > return -EINVAL; > > return ioremap_page_range(vaddr, vaddr + resource_size(res), phys_addr, > - pgprot_device(PAGE_KERNEL)); > + pgprot_noncached(PAGE_KERNEL)); pgprot_device() is equivalent to pgprot_noncached() on all arches except ARM64, and I trust you're doing the right thing on ARM64, so I'm fine with this from a PCI perspective. I do find this puzzling because I naively expected pgprot_noncached() to match up with ioremap_nocache(), and apparently it doesn't. For example, ARM64 ioremap_nocache() uses PROT_DEVICE_nGnRE, which doesn't match the MT_DEVICE_nGnRnE in pgprot_noncached(). The point of these patches is to use non-posted mappings. Apparently you can do that with pgprot_noncached() here, but ioremap_nocache() isn't enough for the config space mappings? I suppose that's a consequence of the pgprot_noncached() vs ioremap_nocache() mismatch, but this is all extremely confusing. > #else > /* this architecture does not have memory mapped I/O space, > so this function should never be called */ > -- > 2.10.0 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: Bjorn Helgaas Subject: Re: [PATCH 02/20] PCI: fix pci_remap_iospace() remap attribute Date: Thu, 16 Mar 2017 16:48:44 -0500 Message-ID: <20170316214844.GA17769@bhelgaas-glaptop.roam.corp.google.com> References: <20170227151436.18698-1-lorenzo.pieralisi@arm.com> <20170227151436.18698-3-lorenzo.pieralisi@arm.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20170227151436.18698-3-lorenzo.pieralisi@arm.com> Sender: linux-pci-owner@vger.kernel.org To: Lorenzo Pieralisi Cc: linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Wenrui Li , Gabriele Paoloni , Catalin Marinas , Shawn Lin , Will Deacon , Michal Simek , Thierry Reding , Tanmay Inamdar , linux-arch@vger.kernel.org, Pratyush Anand , Russell King , Jon Mason , Murali Karicheri , Arnd Bergmann , Bharat Kumar Gogada , Ray Jui , John Garry , Joao Pinto List-Id: linux-arch.vger.kernel.org [+cc Luis] On Mon, Feb 27, 2017 at 03:14:13PM +0000, Lorenzo Pieralisi wrote: > According to the PCI local bus specifications (Revision 3.0, 3.2.5), > I/O Address space transactions are non-posted. On architectures where > I/O space is implemented through a chunk of memory mapped space mapped > to PCI address space (ie IA64/ARM/ARM64) the memory mapping for the > region backing I/O Address Space transactions determines the I/O > transactions attributes (before the transactions actually reaches the > PCI bus where it is handled according to the PCI specifications). > > Current pci_remap_iospace() interface, that is used to map the PCI I/O > Address Space into virtual address space, use pgprot_device() as memory > attribute for the virtual address mapping, that in some architectures > (ie ARM64) provides non-cacheable but write bufferable mappings (ie > posted writes), which clash with the non-posted write behaviour for I/O > Address Space mandated by the PCI specifications. > > Update the prot ioremap_page_range() parameter in pci_remap_iospace() > to pgprot_noncached to ensure that the virtual mapping backing > I/O Address Space guarantee non-posted write transactions issued > when addressing I/O Address Space through the MMIO mapping. > > Signed-off-by: Lorenzo Pieralisi > Cc: Arnd Bergmann > Cc: Will Deacon > Cc: Bjorn Helgaas > Cc: Russell King > Cc: Catalin Marinas > --- > drivers/pci/pci.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index bd98674..bfb3c6e 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -3375,7 +3375,7 @@ int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr) > return -EINVAL; > > return ioremap_page_range(vaddr, vaddr + resource_size(res), phys_addr, > - pgprot_device(PAGE_KERNEL)); > + pgprot_noncached(PAGE_KERNEL)); pgprot_device() is equivalent to pgprot_noncached() on all arches except ARM64, and I trust you're doing the right thing on ARM64, so I'm fine with this from a PCI perspective. I do find this puzzling because I naively expected pgprot_noncached() to match up with ioremap_nocache(), and apparently it doesn't. For example, ARM64 ioremap_nocache() uses PROT_DEVICE_nGnRE, which doesn't match the MT_DEVICE_nGnRnE in pgprot_noncached(). The point of these patches is to use non-posted mappings. Apparently you can do that with pgprot_noncached() here, but ioremap_nocache() isn't enough for the config space mappings? I suppose that's a consequence of the pgprot_noncached() vs ioremap_nocache() mismatch, but this is all extremely confusing. > #else > /* this architecture does not have memory mapped I/O space, > so this function should never be called */ > -- > 2.10.0 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 From: helgaas@kernel.org (Bjorn Helgaas) Date: Thu, 16 Mar 2017 16:48:44 -0500 Subject: [PATCH 02/20] PCI: fix pci_remap_iospace() remap attribute In-Reply-To: <20170227151436.18698-3-lorenzo.pieralisi@arm.com> References: <20170227151436.18698-1-lorenzo.pieralisi@arm.com> <20170227151436.18698-3-lorenzo.pieralisi@arm.com> Message-ID: <20170316214844.GA17769@bhelgaas-glaptop.roam.corp.google.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org [+cc Luis] On Mon, Feb 27, 2017 at 03:14:13PM +0000, Lorenzo Pieralisi wrote: > According to the PCI local bus specifications (Revision 3.0, 3.2.5), > I/O Address space transactions are non-posted. On architectures where > I/O space is implemented through a chunk of memory mapped space mapped > to PCI address space (ie IA64/ARM/ARM64) the memory mapping for the > region backing I/O Address Space transactions determines the I/O > transactions attributes (before the transactions actually reaches the > PCI bus where it is handled according to the PCI specifications). > > Current pci_remap_iospace() interface, that is used to map the PCI I/O > Address Space into virtual address space, use pgprot_device() as memory > attribute for the virtual address mapping, that in some architectures > (ie ARM64) provides non-cacheable but write bufferable mappings (ie > posted writes), which clash with the non-posted write behaviour for I/O > Address Space mandated by the PCI specifications. > > Update the prot ioremap_page_range() parameter in pci_remap_iospace() > to pgprot_noncached to ensure that the virtual mapping backing > I/O Address Space guarantee non-posted write transactions issued > when addressing I/O Address Space through the MMIO mapping. > > Signed-off-by: Lorenzo Pieralisi > Cc: Arnd Bergmann > Cc: Will Deacon > Cc: Bjorn Helgaas > Cc: Russell King > Cc: Catalin Marinas > --- > drivers/pci/pci.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c > index bd98674..bfb3c6e 100644 > --- a/drivers/pci/pci.c > +++ b/drivers/pci/pci.c > @@ -3375,7 +3375,7 @@ int pci_remap_iospace(const struct resource *res, phys_addr_t phys_addr) > return -EINVAL; > > return ioremap_page_range(vaddr, vaddr + resource_size(res), phys_addr, > - pgprot_device(PAGE_KERNEL)); > + pgprot_noncached(PAGE_KERNEL)); pgprot_device() is equivalent to pgprot_noncached() on all arches except ARM64, and I trust you're doing the right thing on ARM64, so I'm fine with this from a PCI perspective. I do find this puzzling because I naively expected pgprot_noncached() to match up with ioremap_nocache(), and apparently it doesn't. For example, ARM64 ioremap_nocache() uses PROT_DEVICE_nGnRE, which doesn't match the MT_DEVICE_nGnRnE in pgprot_noncached(). The point of these patches is to use non-posted mappings. Apparently you can do that with pgprot_noncached() here, but ioremap_nocache() isn't enough for the config space mappings? I suppose that's a consequence of the pgprot_noncached() vs ioremap_nocache() mismatch, but this is all extremely confusing. > #else > /* this architecture does not have memory mapped I/O space, > so this function should never be called */ > -- > 2.10.0 > > > _______________________________________________ > linux-arm-kernel mailing list > linux-arm-kernel at lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel