From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752627AbbFYAxn (ORCPT ); Wed, 24 Jun 2015 20:53:43 -0400 Received: from gate.crashing.org ([63.228.1.57]:36090 "EHLO gate.crashing.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751239AbbFYAxh (ORCPT ); Wed, 24 Jun 2015 20:53:37 -0400 Message-ID: <1435193521.3790.26.camel@kernel.crashing.org> Subject: Re: [PATCH v7 5/9] PCI: Add pci_iomap_wc() variants From: Benjamin Herrenschmidt To: "Luis R. Rodriguez" Cc: "Michael S. Tsirkin" , Bjorn Helgaas , Toshi Kani , Andy Lutomirski , Juergen Gross , Tomi Valkeinen , Arnd Bergmann , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "xen-devel@lists.xensource.com" , linux-fbdev , Suresh Siddha , Ingo Molnar , Thomas Gleixner , Daniel Vetter , Dave Airlie , Antonino Daplas , Jean-Christophe Plagniol-Villard , Dave Hansen , venkatesh.pallipadi@intel.com, Stefan Bader , Ville =?ISO-8859-1?Q?Syrj=E4l=E4?= , Mel Gorman , Vlastimil Babka , Borislav Petkov , Davidlohr Bueso , Konrad Rzeszutek Wilk , Ville =?ISO-8859-1?Q?Syrj=E4l=E4?= , David Vrabel , Jan Beulich , Roger Pau =?ISO-8859-1?Q?Monn=E9?= Date: Thu, 25 Jun 2015 10:52:01 +1000 In-Reply-To: <20150625000803.GH11147@wotan.suse.de> References: <1434751712-24333-1-git-send-email-mcgrof@do-not-panic.com> <1434751712-24333-6-git-send-email-mcgrof@do-not-panic.com> <1435099343.3996.13.camel@kernel.crashing.org> <20150624163821.GF11147@wotan.suse.de> <1435183509.3790.14.camel@kernel.crashing.org> <1435189081.3790.24.camel@kernel.crashing.org> <20150625000803.GH11147@wotan.suse.de> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.12.11-0ubuntu3 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 2015-06-25 at 02:08 +0200, Luis R. Rodriguez wrote: > > OK thanks I'll proceed with these patches then. > > > As for user mappings, > > Which APIs were you considering in this regard BTW? mmap of the generic /sys/bus/pci/.../resource* > > maybe the right thing to do is to let us do what we do by > > default with a quirk that can set a flag in pci_dev to disable that > > behaviour (maybe on a per BAR basis ?). > > That might mean it could restrict userspace WC to require devices > to have WC parts on a full PCI BAR. Although this is restrictive > having reviewed most WC uses in the kernel I'd think this would be > a fair compromise to make, but again, if things are still murky > perhaps best we kiss this idea good bye for now and hope for it > to come in on future buses or ammendments (if that's even possible?). > > > I think the common case is that WC works. > > If WC does not I will note one hack which migh be worth mentioning -- > just for > the record, this was devised as a shortcoming of a device where they > failed to > split things properly and that *without* WC performance suffered quite > a bit so > they made one full PCI BAR WC and as a work around this: > > http://lkml.kernel.org/r/20150416041837.GA5712@hykim-PC > > That is for registers that needed it: > > write; wmb; > > Then if they wanted to wait till the NIC has seen the write, they did: > > write; wmb; read; > Right, and as I mentioned, on some archs like powerpc (and possibly more), writel() and co contains an implicit mb() > Luis-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Date: Thu, 25 Jun 2015 00:52:01 +0000 Subject: Re: [PATCH v7 5/9] PCI: Add pci_iomap_wc() variants Message-Id: <1435193521.3790.26.camel@kernel.crashing.org> List-Id: References: <1434751712-24333-1-git-send-email-mcgrof@do-not-panic.com> <1434751712-24333-6-git-send-email-mcgrof@do-not-panic.com> <1435099343.3996.13.camel@kernel.crashing.org> <20150624163821.GF11147@wotan.suse.de> <1435183509.3790.14.camel@kernel.crashing.org> <1435189081.3790.24.camel@kernel.crashing.org> <20150625000803.GH11147@wotan.suse.de> In-Reply-To: <20150625000803.GH11147@wotan.suse.de> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: "Luis R. Rodriguez" Cc: "Michael S. Tsirkin" , Bjorn Helgaas , Toshi Kani , Andy Lutomirski , Juergen Gross , Tomi Valkeinen , Arnd Bergmann , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "xen-devel@lists.xensource.com" , linux-fbdev , Suresh Siddha , Ingo Molnar , Thomas Gleixner , Daniel Vetter , Dave Airlie , Antonino Daplas , Jean-Christophe Plagniol-Villard , Dave Hansen , venkatesh.pallipadi@intel.com, Stefan Bader , Ville =?ISO-8859-1?Q?Syrj=E4l=E4?= , Mel Gorman , Vlastimil Babka , Borislav Petkov , Davidlohr Bueso , Konrad Rzeszutek Wilk , Ville =?ISO-8859-1?Q?Syrj=E4l=E4?= , David Vrabel , Jan Beulich , Roger Pau =?ISO-8859-1?Q?Monn=E9?= On Thu, 2015-06-25 at 02:08 +0200, Luis R. Rodriguez wrote: > > OK thanks I'll proceed with these patches then. > > > As for user mappings, > > Which APIs were you considering in this regard BTW? mmap of the generic /sys/bus/pci/.../resource* > > maybe the right thing to do is to let us do what we do by > > default with a quirk that can set a flag in pci_dev to disable that > > behaviour (maybe on a per BAR basis ?). > > That might mean it could restrict userspace WC to require devices > to have WC parts on a full PCI BAR. Although this is restrictive > having reviewed most WC uses in the kernel I'd think this would be > a fair compromise to make, but again, if things are still murky > perhaps best we kiss this idea good bye for now and hope for it > to come in on future buses or ammendments (if that's even possible?). > > > I think the common case is that WC works. > > If WC does not I will note one hack which migh be worth mentioning -- > just for > the record, this was devised as a shortcoming of a device where they > failed to > split things properly and that *without* WC performance suffered quite > a bit so > they made one full PCI BAR WC and as a work around this: > > http://lkml.kernel.org/r/20150416041837.GA5712@hykim-PC > > That is for registers that needed it: > > write; wmb; > > Then if they wanted to wait till the NIC has seen the write, they did: > > write; wmb; read; > Right, and as I mentioned, on some archs like powerpc (and possibly more), writel() and co contains an implicit mb() > Luis-- From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Subject: Re: [PATCH v7 5/9] PCI: Add pci_iomap_wc() variants Date: Thu, 25 Jun 2015 10:52:01 +1000 Message-ID: <1435193521.3790.26.camel@kernel.crashing.org> References: <1434751712-24333-1-git-send-email-mcgrof@do-not-panic.com> <1434751712-24333-6-git-send-email-mcgrof@do-not-panic.com> <1435099343.3996.13.camel@kernel.crashing.org> <20150624163821.GF11147@wotan.suse.de> <1435183509.3790.14.camel@kernel.crashing.org> <1435189081.3790.24.camel@kernel.crashing.org> <20150625000803.GH11147@wotan.suse.de> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20150625000803.GH11147@wotan.suse.de> Sender: linux-pci-owner@vger.kernel.org To: "Luis R. Rodriguez" Cc: "Michael S. Tsirkin" , Bjorn Helgaas , Toshi Kani , Andy Lutomirski , Juergen Gross , Tomi Valkeinen , Arnd Bergmann , "linux-pci@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "xen-devel@lists.xensource.com" , linux-fbdev , Suresh Siddha , Ingo Molnar , Thomas Gleixner , Daniel Vetter , Dave Airlie , Antonino Daplas , Jean-Christophe Plagniol-Villard , Dave Hansen , venkatesh.pallipadi@intel.com, Stefan Bader , Ville List-Id: xen-devel@lists.xenproject.org On Thu, 2015-06-25 at 02:08 +0200, Luis R. Rodriguez wrote: > > OK thanks I'll proceed with these patches then. > > > As for user mappings, > > Which APIs were you considering in this regard BTW? mmap of the generic /sys/bus/pci/.../resource* > > maybe the right thing to do is to let us do what we do by > > default with a quirk that can set a flag in pci_dev to disable that > > behaviour (maybe on a per BAR basis ?). > > That might mean it could restrict userspace WC to require devices > to have WC parts on a full PCI BAR. Although this is restrictive > having reviewed most WC uses in the kernel I'd think this would be > a fair compromise to make, but again, if things are still murky > perhaps best we kiss this idea good bye for now and hope for it > to come in on future buses or ammendments (if that's even possible?). > > > I think the common case is that WC works. > > If WC does not I will note one hack which migh be worth mentioning -- > just for > the record, this was devised as a shortcoming of a device where they > failed to > split things properly and that *without* WC performance suffered quite > a bit so > they made one full PCI BAR WC and as a work around this: > > http://lkml.kernel.org/r/20150416041837.GA5712@hykim-PC > > That is for registers that needed it: > > write; wmb; > > Then if they wanted to wait till the NIC has seen the write, they did: > > write; wmb; read; > Right, and as I mentioned, on some archs like powerpc (and possibly more), writel() and co contains an implicit mb() > Luis--