From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932364AbbFYBB5 (ORCPT ); Wed, 24 Jun 2015 21:01:57 -0400 Received: from gate.crashing.org ([63.228.1.57]:52368 "EHLO gate.crashing.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753861AbbFYBBq (ORCPT ); Wed, 24 Jun 2015 21:01:46 -0400 Message-ID: <1435183509.3790.14.camel@kernel.crashing.org> Subject: Re: [PATCH v7 5/9] PCI: Add pci_iomap_wc() variants From: Benjamin Herrenschmidt To: "Luis R. Rodriguez" Cc: "Michael S. Tsirkin" , Bjorn Helgaas , Toshi Kani , Andy Lutomirski , Juergen Gross , Tomi Valkeinen , "Luis R. Rodriguez" , arnd@arndb.de, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xensource.com, linux-fbdev@vger.kernel.org, Suresh Siddha , Ingo Molnar , Thomas Gleixner , Daniel Vetter , Dave Airlie , Antonino Daplas , Jean-Christophe Plagniol-Villard , Dave Hansen , venkatesh.pallipadi@intel.com, Stefan Bader , Ville =?ISO-8859-1?Q?Syrj=E4l=E4?= , Mel Gorman , Vlastimil Babka , Borislav Petkov , Davidlohr Bueso , konrad.wilk@oracle.com, ville.syrjala@linux.intel.com, david.vrabel@citrix.com, jbeulich@suse.com, Roger Pau =?ISO-8859-1?Q?Monn=E9?= Date: Thu, 25 Jun 2015 08:05:09 +1000 In-Reply-To: <20150624163821.GF11147@wotan.suse.de> References: <1434751712-24333-1-git-send-email-mcgrof@do-not-panic.com> <1434751712-24333-6-git-send-email-mcgrof@do-not-panic.com> <1435099343.3996.13.camel@kernel.crashing.org> <20150624163821.GF11147@wotan.suse.de> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.12.11-0ubuntu3 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2015-06-24 at 18:38 +0200, Luis R. Rodriguez wrote: > On Wed, Jun 24, 2015 at 08:42:23AM +1000, Benjamin Herrenschmidt wrote: > > On Fri, 2015-06-19 at 15:08 -0700, Luis R. Rodriguez wrote: > > > From: "Luis R. Rodriguez" > > > > > > PCI BARs tell us whether prefetching is safe, but they don't say anything > > > about write combining (WC). WC changes ordering rules and allows writes to > > > be collapsed, so it's not safe in general to use it on a prefetchable > > > region. > > > > Well, the PCIe spec at least specifies that a prefetchable BAR also > > tolerates write merging... > > How can that be determined and can that be used as a full bullet proof hint > to enable wc ? And are you sure? :) Well, I"m sure the spec says that ;-) But it could be new to PCIe, I haven't checked legacy PCI. > Reason all this was stated was to be > apologetic over why we can't automate this behind the scenes. Otherwise > we could amend what you stated into the commit log to elaborate on our > technical apology. Let me know! At least on powerpc, for mmap of resource to userspace, we take off the garded bit in the PTE for prefetchable BARs. This has the effect architecturally of enabling both prefetch and write combine (ie. side effect) though afaik, the implementations probably don't actually prefetch. We've done that for years. In fact we don't have a way to split the notions, it's either G or no G, which carries both meanings. Do you have example/case of a device having problems ? Cheers, Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Date: Wed, 24 Jun 2015 22:05:09 +0000 Subject: Re: [PATCH v7 5/9] PCI: Add pci_iomap_wc() variants Message-Id: <1435183509.3790.14.camel@kernel.crashing.org> List-Id: References: <1434751712-24333-1-git-send-email-mcgrof@do-not-panic.com> <1434751712-24333-6-git-send-email-mcgrof@do-not-panic.com> <1435099343.3996.13.camel@kernel.crashing.org> <20150624163821.GF11147@wotan.suse.de> In-Reply-To: <20150624163821.GF11147@wotan.suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: "Luis R. Rodriguez" Cc: "Michael S. Tsirkin" , Bjorn Helgaas , Toshi Kani , Andy Lutomirski , Juergen Gross , Tomi Valkeinen , "Luis R. Rodriguez" , arnd@arndb.de, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, xen-devel@lists.xensource.com, linux-fbdev@vger.kernel.org, Suresh Siddha , Ingo Molnar , Thomas Gleixner , Daniel Vetter , Dave Airlie , Antonino Daplas , Jean-Christophe Plagniol-Villard , Dave Hansen , venkatesh.pallipadi@intel.com, Stefan Bader , Ville =?ISO-8859-1?Q?Syrj=E4l=E4?= , Mel Gorman , Vlastimil Babka , Borislav Petkov , Davidlohr Bueso , konrad.wilk@oracle.com, ville.syrjala@linux.intel.com, david.vrabel@citrix.com, jbeulich@suse.com, Roger Pau =?ISO-8859-1?Q?Monn=E9?= On Wed, 2015-06-24 at 18:38 +0200, Luis R. Rodriguez wrote: > On Wed, Jun 24, 2015 at 08:42:23AM +1000, Benjamin Herrenschmidt wrote: > > On Fri, 2015-06-19 at 15:08 -0700, Luis R. Rodriguez wrote: > > > From: "Luis R. Rodriguez" > > > > > > PCI BARs tell us whether prefetching is safe, but they don't say anything > > > about write combining (WC). WC changes ordering rules and allows writes to > > > be collapsed, so it's not safe in general to use it on a prefetchable > > > region. > > > > Well, the PCIe spec at least specifies that a prefetchable BAR also > > tolerates write merging... > > How can that be determined and can that be used as a full bullet proof hint > to enable wc ? And are you sure? :) Well, I"m sure the spec says that ;-) But it could be new to PCIe, I haven't checked legacy PCI. > Reason all this was stated was to be > apologetic over why we can't automate this behind the scenes. Otherwise > we could amend what you stated into the commit log to elaborate on our > technical apology. Let me know! At least on powerpc, for mmap of resource to userspace, we take off the garded bit in the PTE for prefetchable BARs. This has the effect architecturally of enabling both prefetch and write combine (ie. side effect) though afaik, the implementations probably don't actually prefetch. We've done that for years. In fact we don't have a way to split the notions, it's either G or no G, which carries both meanings. Do you have example/case of a device having problems ? Cheers, Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Subject: Re: [PATCH v7 5/9] PCI: Add pci_iomap_wc() variants Date: Thu, 25 Jun 2015 08:05:09 +1000 Message-ID: <1435183509.3790.14.camel@kernel.crashing.org> References: <1434751712-24333-1-git-send-email-mcgrof@do-not-panic.com> <1434751712-24333-6-git-send-email-mcgrof@do-not-panic.com> <1435099343.3996.13.camel@kernel.crashing.org> <20150624163821.GF11147@wotan.suse.de> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20150624163821.GF11147@wotan.suse.de> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: "Luis R. Rodriguez" Cc: linux-fbdev@vger.kernel.org, "Michael S. Tsirkin" , linux-pci@vger.kernel.org, Ville =?ISO-8859-1?Q?Syrj=E4l=E4?= , Dave Hansen , jbeulich@suse.com, ville.syrjala@linux.intel.com, xen-devel@lists.xensource.com, arnd@arndb.de, Davidlohr Bueso , Tomi Valkeinen , Mel Gorman , Daniel Vetter , Dave Airlie , Ingo Molnar , Borislav Petkov , Jean-Christophe Plagniol-Villard , Antonino Daplas , Suresh Siddha , Stefan Bader , Bjorn Helgaas , Thomas Gleixner , Vlastimil Babka , Juergen Gross , Toshi Kani , Luis R. List-Id: xen-devel@lists.xenproject.org On Wed, 2015-06-24 at 18:38 +0200, Luis R. Rodriguez wrote: > On Wed, Jun 24, 2015 at 08:42:23AM +1000, Benjamin Herrenschmidt wrote: > > On Fri, 2015-06-19 at 15:08 -0700, Luis R. Rodriguez wrote: > > > From: "Luis R. Rodriguez" > > > > > > PCI BARs tell us whether prefetching is safe, but they don't say anything > > > about write combining (WC). WC changes ordering rules and allows writes to > > > be collapsed, so it's not safe in general to use it on a prefetchable > > > region. > > > > Well, the PCIe spec at least specifies that a prefetchable BAR also > > tolerates write merging... > > How can that be determined and can that be used as a full bullet proof hint > to enable wc ? And are you sure? :) Well, I"m sure the spec says that ;-) But it could be new to PCIe, I haven't checked legacy PCI. > Reason all this was stated was to be > apologetic over why we can't automate this behind the scenes. Otherwise > we could amend what you stated into the commit log to elaborate on our > technical apology. Let me know! At least on powerpc, for mmap of resource to userspace, we take off the garded bit in the PTE for prefetchable BARs. This has the effect architecturally of enabling both prefetch and write combine (ie. side effect) though afaik, the implementations probably don't actually prefetch. We've done that for years. In fact we don't have a way to split the notions, it's either G or no G, which carries both meanings. Do you have example/case of a device having problems ? Cheers, Ben.