From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Jason Gunthorpe <jgg@nvidia.com>, Clint Sbisa <csbisa@amazon.com>,
linux-pci@vger.kernel.org, Bjorn Helgaas <helgaas@kernel.org>,
linux-arm-kernel@lists.infradead.org, will@kernel.org,
catalin.marinas@arm.com, Leon Romanovsky <leon@kernel.org>
Subject: Re: [PATCH] arm64: Enable PCI write-combine resources under sysfs
Date: Thu, 17 Sep 2020 11:28:19 +0100 [thread overview]
Message-ID: <20200917102819.GA2284@e121166-lin.cambridge.arm.com> (raw)
In-Reply-To: <28082ccc715a9fba349ae6052d5c917ae02d40fa.camel@kernel.crashing.org>
On Thu, Sep 17, 2020 at 09:59:28AM +1000, Benjamin Herrenschmidt wrote:
> On Wed, 2020-09-16 at 09:12 -0300, Jason Gunthorpe wrote:
> > > Also we could make this a variable rather than a constant and
> > > choose
> > > a more appropriate set of flags at boot time....
> >
> > It is a function, so it could check the CPU ID for the known broken
> > devices and block them.
>
> Sure, I meant in the abstract way. It's not a hot path so it doesnt
> have to be a static key.
>
> > > > > Why would that be a regression ?
> > > >
> > > > Using the WC submission flow when it doesn't work costs something
> > > > like
> > > > 10% performance vs using the non-WC flow.
> > >
> > > You mean the driver uses a different path to the HW which ahs that
> > > overhead, not that MMIOs have that overhead right ?
> >
> > The different path has overhead of doing extra useless MMIOs because
> > they don't combine
>
> I see. This might have to end up being a TX2 specific hack until the
> end of times...
True - hopefully on platforms that implement normal NC the architectural
way will not trigger user space performance regressions.
Unfortunately if we merge this patch we _do_ know from this thread
that userspace will suffer from a perf regression on TX2.
Either we ignore it or we write some code to prevent it
(ie first step make arch_can_pci_mmap_wc() return 0 on TX2 -
possibly using the arm64 errata detection mechanism).
Adding a new IO mapping API and use it in IB drivers won't solve the TX2
problem - since we still prefer normal NC over device GRE for "WC"
mappings and we would have to "downgrade" TX2 somehow.
Lorenzo
WARNING: multiple messages have this Message-ID (diff)
From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
To: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Leon Romanovsky <leon@kernel.org>,
linux-pci@vger.kernel.org, Bjorn Helgaas <helgaas@kernel.org>,
Jason Gunthorpe <jgg@nvidia.com>,
catalin.marinas@arm.com, will@kernel.org,
linux-arm-kernel@lists.infradead.org,
Clint Sbisa <csbisa@amazon.com>
Subject: Re: [PATCH] arm64: Enable PCI write-combine resources under sysfs
Date: Thu, 17 Sep 2020 11:28:19 +0100 [thread overview]
Message-ID: <20200917102819.GA2284@e121166-lin.cambridge.arm.com> (raw)
In-Reply-To: <28082ccc715a9fba349ae6052d5c917ae02d40fa.camel@kernel.crashing.org>
On Thu, Sep 17, 2020 at 09:59:28AM +1000, Benjamin Herrenschmidt wrote:
> On Wed, 2020-09-16 at 09:12 -0300, Jason Gunthorpe wrote:
> > > Also we could make this a variable rather than a constant and
> > > choose
> > > a more appropriate set of flags at boot time....
> >
> > It is a function, so it could check the CPU ID for the known broken
> > devices and block them.
>
> Sure, I meant in the abstract way. It's not a hot path so it doesnt
> have to be a static key.
>
> > > > > Why would that be a regression ?
> > > >
> > > > Using the WC submission flow when it doesn't work costs something
> > > > like
> > > > 10% performance vs using the non-WC flow.
> > >
> > > You mean the driver uses a different path to the HW which ahs that
> > > overhead, not that MMIOs have that overhead right ?
> >
> > The different path has overhead of doing extra useless MMIOs because
> > they don't combine
>
> I see. This might have to end up being a TX2 specific hack until the
> end of times...
True - hopefully on platforms that implement normal NC the architectural
way will not trigger user space performance regressions.
Unfortunately if we merge this patch we _do_ know from this thread
that userspace will suffer from a perf regression on TX2.
Either we ignore it or we write some code to prevent it
(ie first step make arch_can_pci_mmap_wc() return 0 on TX2 -
possibly using the arm64 errata detection mechanism).
Adding a new IO mapping API and use it in IB drivers won't solve the TX2
problem - since we still prefer normal NC over device GRE for "WC"
mappings and we would have to "downgrade" TX2 somehow.
Lorenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2020-09-17 10:28 UTC|newest]
Thread overview: 117+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-31 15:18 [PATCH] arm64: Enable PCI write-combine resources under sysfs Clint Sbisa
2020-08-31 23:24 ` Benjamin Herrenschmidt
2020-09-01 18:37 ` Bjorn Helgaas
2020-09-01 23:22 ` Benjamin Herrenschmidt
2020-09-02 8:57 ` Will Deacon
2020-09-02 11:32 ` Lorenzo Pieralisi
2020-09-02 14:29 ` Clint Sbisa
2020-09-02 16:47 ` Lorenzo Pieralisi
2020-09-02 16:47 ` Lorenzo Pieralisi
2020-09-02 17:21 ` Catalin Marinas
2020-09-02 17:21 ` Catalin Marinas
2020-09-02 17:54 ` Lorenzo Pieralisi
2020-09-02 17:54 ` Lorenzo Pieralisi
2020-09-02 23:03 ` Benjamin Herrenschmidt
2020-09-02 23:03 ` Benjamin Herrenschmidt
2020-09-02 23:08 ` Benjamin Herrenschmidt
2020-09-02 23:08 ` Benjamin Herrenschmidt
2020-09-02 23:08 ` Benjamin Herrenschmidt
2020-09-02 23:08 ` Benjamin Herrenschmidt
2020-09-02 23:07 ` Benjamin Herrenschmidt
2020-09-02 23:07 ` Benjamin Herrenschmidt
2020-09-03 11:08 ` Lorenzo Pieralisi
2020-09-03 11:08 ` Lorenzo Pieralisi
2020-09-03 14:36 ` Clint Sbisa
2020-09-03 14:36 ` Clint Sbisa
2020-09-03 22:26 ` Benjamin Herrenschmidt
2020-09-03 22:26 ` Benjamin Herrenschmidt
2020-09-07 23:33 ` Benjamin Herrenschmidt
2020-09-07 23:33 ` Benjamin Herrenschmidt
2020-09-10 9:46 ` Lorenzo Pieralisi
2020-09-10 9:46 ` Lorenzo Pieralisi
2020-09-10 10:54 ` Leon Romanovsky
2020-09-10 10:54 ` Leon Romanovsky
2020-09-10 12:37 ` Jason Gunthorpe
2020-09-10 12:37 ` Jason Gunthorpe
2020-09-10 15:17 ` Lorenzo Pieralisi
2020-09-10 15:17 ` Lorenzo Pieralisi
2020-09-10 17:10 ` Jason Gunthorpe
2020-09-10 17:10 ` Jason Gunthorpe
2020-09-10 21:46 ` Benjamin Herrenschmidt
2020-09-10 21:46 ` Benjamin Herrenschmidt
2020-09-10 23:29 ` Jason Gunthorpe
2020-09-10 23:29 ` Jason Gunthorpe
2020-09-11 0:39 ` Benjamin Herrenschmidt
2020-09-11 0:39 ` Benjamin Herrenschmidt
2020-09-11 14:21 ` Jason Gunthorpe
2020-09-11 14:21 ` Jason Gunthorpe
2020-09-11 21:42 ` Clint Sbisa
2020-09-11 21:42 ` Clint Sbisa
2020-09-14 14:17 ` Jason Gunthorpe
2020-09-14 14:17 ` Jason Gunthorpe
2020-09-14 14:24 ` Clint Sbisa
2020-09-14 14:24 ` Clint Sbisa
2020-09-14 14:38 ` Jason Gunthorpe
2020-09-14 14:38 ` Jason Gunthorpe
2020-09-14 21:42 ` Benjamin Herrenschmidt
2020-09-14 21:42 ` Benjamin Herrenschmidt
2020-09-14 22:00 ` Benjamin Herrenschmidt
2020-09-14 22:00 ` Benjamin Herrenschmidt
2020-09-14 22:32 ` Clint Sbisa
2020-09-14 22:32 ` Clint Sbisa
2020-09-14 22:57 ` Jason Gunthorpe
2020-09-14 22:57 ` Jason Gunthorpe
2020-09-14 23:25 ` Benjamin Herrenschmidt
2020-09-14 23:25 ` Benjamin Herrenschmidt
2020-09-15 10:18 ` Lorenzo Pieralisi
2020-09-15 10:18 ` Lorenzo Pieralisi
2020-09-15 11:05 ` Jason Gunthorpe
2020-09-15 11:05 ` Jason Gunthorpe
2020-09-15 23:17 ` Benjamin Herrenschmidt
2020-09-15 23:17 ` Benjamin Herrenschmidt
2020-09-15 23:40 ` Jason Gunthorpe
2020-09-15 23:40 ` Jason Gunthorpe
2020-09-16 7:59 ` Benjamin Herrenschmidt
2020-09-16 7:59 ` Benjamin Herrenschmidt
2020-09-16 12:12 ` Jason Gunthorpe
2020-09-16 12:12 ` Jason Gunthorpe
2020-09-16 14:09 ` Lorenzo Pieralisi
2020-09-16 14:09 ` Lorenzo Pieralisi
2020-09-16 14:14 ` Jason Gunthorpe
2020-09-16 14:14 ` Jason Gunthorpe
2020-09-16 23:59 ` Benjamin Herrenschmidt
2020-09-16 23:59 ` Benjamin Herrenschmidt
2020-09-17 10:28 ` Lorenzo Pieralisi [this message]
2020-09-17 10:28 ` Lorenzo Pieralisi
2020-09-17 11:32 ` Jason Gunthorpe
2020-09-17 11:32 ` Jason Gunthorpe
2020-09-17 14:01 ` Lorenzo Pieralisi
2020-09-17 14:01 ` Lorenzo Pieralisi
2020-09-17 16:08 ` Will Deacon
2020-09-17 16:08 ` Will Deacon
2020-09-16 12:48 ` Leon Romanovsky
2020-09-16 12:48 ` Leon Romanovsky
2020-09-16 8:33 ` Will Deacon
2020-09-16 8:33 ` Will Deacon
2020-09-16 8:48 ` Catalin Marinas
2020-09-16 8:48 ` Catalin Marinas
2020-09-16 14:15 ` Lorenzo Pieralisi
2020-09-16 14:15 ` Lorenzo Pieralisi
2020-09-16 17:00 ` Catalin Marinas
2020-09-16 17:00 ` Catalin Marinas
2020-09-16 21:29 ` Benjamin Herrenschmidt
2020-09-16 21:29 ` Benjamin Herrenschmidt
2020-09-16 12:08 ` Jason Gunthorpe
2020-09-16 12:08 ` Jason Gunthorpe
2020-09-15 23:00 ` Benjamin Herrenschmidt
2020-09-15 23:00 ` Benjamin Herrenschmidt
2020-09-15 23:12 ` Clint Sbisa
2020-09-15 23:12 ` Clint Sbisa
2020-09-14 21:41 ` Benjamin Herrenschmidt
2020-09-14 21:41 ` Benjamin Herrenschmidt
-- strict thread matches above, loose matches on Subject: below --
2020-08-21 15:51 Clint Sbisa
2020-08-21 15:51 ` Clint Sbisa
2020-08-27 14:41 ` Clint Sbisa
2020-08-27 14:41 ` Clint Sbisa
2020-08-31 15:22 ` Clint Sbisa
2020-08-31 15:22 ` Clint Sbisa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200917102819.GA2284@e121166-lin.cambridge.arm.com \
--to=lorenzo.pieralisi@arm.com \
--cc=benh@kernel.crashing.org \
--cc=catalin.marinas@arm.com \
--cc=csbisa@amazon.com \
--cc=helgaas@kernel.org \
--cc=jgg@nvidia.com \
--cc=leon@kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-pci@vger.kernel.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.