From: "Luis R. Rodriguez" <mcgrof@suse.com>
To: Jason Gunthorpe <jgunthorpe@obsidianresearch.com>
Cc: Andy Lutomirski <luto@amacapital.net>,
mike.marciniszyn@intel.com, infinipath@intel.com,
linux-rdma@vger.kernel.org, awalls@md.metrocast.net,
Toshi Kani <toshi.kani@hp.com>, "H. Peter Anvin" <hpa@zytor.com>,
Ingo Molnar <mingo@kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Hal Rosenstock <hal.rosenstock@gmail.com>,
Sean Hefty <sean.hefty@intel.com>,
Suresh Siddha <sbsiddha@gmail.com>,
Rickard Strandqvist <rickard_strandqvist@spectrumdigital.se>,
Roland Dreier <roland@purestorage.com>,
Juergen Gross <jgross@suse.com>,
Mauro Carvalho Chehab <mchehab@osg.samsung.com>,
Borislav Petkov <bp@suse.de>, Mel Gorman <mgorman@suse.de>,
Vlastimil Babka <vbabka@suse.cz>,
Davidlohr Bueso <dbueso@suse.de>,
Dave Hansen <dave.hansen@linux.intel.com>,
Jean-Christophe Plagniol-Villard <plagnioj@jcrosoft.com>,
Thomas Gleixner <tglx@linutronix.de>,
Ville Syrj?l? <syrjala@sci.fi>,
Linux Fbdev development list <linux-fbdev@vger.kernel.org>,
linux-media@vger.kernel.org, X86 ML <x86@kernel.org>,
mcgrof@do-not-panic.com
Subject: Re: ioremap_uc() followed by set_memory_wc() - burrying MTRR
Date: Wed, 22 Apr 2015 17:23:28 +0200 [thread overview]
Message-ID: <20150422152328.GB5622@wotan.suse.de> (raw)
In-Reply-To: <20150422053939.GA29609@obsidianresearch.com>
On Tue, Apr 21, 2015 at 11:39:39PM -0600, Jason Gunthorpe wrote:
> On Wed, Apr 22, 2015 at 01:39:07AM +0200, Luis R. Rodriguez wrote:
> > > Mike, do you think the time is right to just remove the iPath driver?
> >
> > With PAT now being default the driver effectively won't work
> > with write-combining on modern kernels. Even if systems are old
> > they likely had PAT support, when upgrading kernels PAT will work
> > but write-combing won't on ipath.
>
> Sorry, do you mean the driver already doesn't get WC? Or do you mean
> after some more pending patches are applied?
No, you have to consider the system used and the effects of calls used
on the driver in light of this table:
----------------------------------------------------------------------
MTRR Non-PAT PAT Linux ioremap value Effective memory type
----------------------------------------------------------------------
Non-PAT | PAT
PAT
|PCD
||PWT
|||
WC 000 WB _PAGE_CACHE_MODE_WB WC | WC
WC 001 WC _PAGE_CACHE_MODE_WC WC* | WC
WC 010 UC- _PAGE_CACHE_MODE_UC_MINUS WC* | UC
WC 011 UC _PAGE_CACHE_MODE_UC UC | UC
----------------------------------------------------------------------
(*) denotes implementation defined and is discouraged
ioremap_nocache() will use _PAGE_CACHE_MODE_UC_MINUS by default today,
in the future we want to flip the switch and make _PAGE_CACHE_MODE_UC
the default. When that flip occurs it will mean ipath cannot get
write-combining on both non-PAT and PAT systems. Now that is for
the future, lets review the current situation for ipath.
For PAT capable systems if mtrr_add() is used today on a Linux system on a
region mapped with ioremap_nocache() that will mean you effectively nullify the
mtrr_add() effect as the combinatorial effect above yields an effective memory
type of UC. For PAT systems you want to use ioremap_wc() on the region in
which you need write-combining followed by arch_phys_wc_add() which will *only*
call mtrr_add() *iff* PAT was not enabled. This also means we need to split
the ioremap'd areas so that the area that is using ioremap_nocache() can never
get write-combining (_PAGE_CACHE_MODE_UC). The ipath driver needs the regions
split just as was done for the qib driver.
Now we could just say that leaving things as-is is a non-issue if you are OK
with non-write-combining effects being the default behaviour left on the ipath
driver for PAT systems. In that case we can just use arch_phys_wc_add() on the
driver and while it won't trigger the mtrr_add() on PAT systems it sill won't
have any effect. We just typically don't want to see use of ioremap_nocache()
paired with arch_phys_wc_add(), grammatically the correct thing to do is pair
ioremap_wc() areas with a arch_phys_wc_add() to make the write-combining effects
on non-PAT systems. If the ipath driver is not going to get he work required
to split the regions though perhaps we can live with a corner case driver that
annotates PAT must be disabled on the systems that use it and convert it to
arch_phys_wc_add() to just help with phasing out of direct use of mtrr_add().
With this strategy if and when ipath driver gets a split done it would gain WC
on both PAT and non-PAT.
Luis
next prev parent reply other threads:[~2015-04-22 15:23 UTC|newest]
Thread overview: 40+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-04-15 20:42 ioremap_uc() followed by set_memory_wc() - burrying MTRR Andy Lutomirski
2015-04-15 20:56 ` H. Peter Anvin
2015-04-15 22:15 ` Luis R. Rodriguez
2015-04-15 22:50 ` Andy Walls
2015-04-15 23:52 ` Andy Lutomirski
2015-04-16 0:33 ` Andy Walls
2015-04-21 22:46 ` Luis R. Rodriguez
2015-04-21 22:57 ` Jason Gunthorpe
2015-04-21 23:39 ` Luis R. Rodriguez
2015-04-22 5:39 ` Jason Gunthorpe
2015-04-22 15:23 ` Luis R. Rodriguez [this message]
2015-04-22 15:54 ` Luis R. Rodriguez
2015-04-22 15:59 ` Luis R. Rodriguez
2015-04-22 16:17 ` Jason Gunthorpe
2015-04-22 16:51 ` Luis R. Rodriguez
2015-04-22 18:53 ` Doug Ledford
2015-04-22 19:05 ` Luis R. Rodriguez
2015-04-22 19:10 ` Doug Ledford
2015-04-22 19:14 ` Luis R. Rodriguez
2015-04-22 20:46 ` Jason Gunthorpe
2015-04-22 20:58 ` Doug Ledford
2015-04-22 16:53 ` Andy Lutomirski
2015-04-22 17:07 ` Luis R. Rodriguez
[not found] <20150410171750.GA5622@wotan.suse.de>
[not found] ` <CALCETrUG=RiG8S9Gpiqm_0CxvxurxLTNKyuyPoFNX46EAauA+g@mail.gmail.com>
[not found] ` <CAB=NE6XgNgu7i2OiDxFVJLWiEjbjBY17-dV7L3yi2+yzgMhEbw@mail.gmail.com>
[not found] ` <1428695379.6646.69.camel@misato.fc.hp.com>
[not found] ` <20150410210538.GB5622@wotan.suse.de>
[not found] ` <1428699490.21794.5.camel@misato.fc.hp.com>
[not found] ` <CALCETrUP688aNjckygqO=AXXrNYvLQX6F0=b5fjmsCqqZU78+Q@mail.gmail.com>
[not found] ` <20150411012938.GC5622@wotan.suse.de>
[not found] ` <CALCETrXd19C6pARde3pv-4pt-i52APtw5xs20itwROPq9VmCfg@mail.gmail.com>
2015-04-13 17:49 ` Luis R. Rodriguez
2015-04-15 22:38 ` Andy Walls
2015-04-15 23:42 ` Andy Lutomirski
2015-04-15 23:59 ` Andy Walls
2015-04-16 0:58 ` Andy Lutomirski
2015-04-16 1:01 ` Andy Walls
2015-04-16 19:19 ` Andy Lutomirski
2015-04-21 17:35 ` Luis R. Rodriguez
2015-04-15 23:58 ` Luis R. Rodriguez
2015-04-16 1:07 ` Andy Walls
2015-04-21 22:02 ` Luis R. Rodriguez
2015-04-21 22:08 ` Luis R. Rodriguez
2015-04-21 22:09 ` Andy Lutomirski
[not found] ` <5536d47a.95968c0a.1d12.ffffbf85@mx.google.com>
2015-04-21 23:14 ` Luis R. Rodriguez
2015-04-16 4:18 ` Hyong-Youb Kim
2015-04-16 18:54 ` Luis R. Rodriguez
2015-04-17 8:00 ` Hyong-Youb Kim
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150422152328.GB5622@wotan.suse.de \
--to=mcgrof@suse.com \
--cc=awalls@md.metrocast.net \
--cc=bp@suse.de \
--cc=dave.hansen@linux.intel.com \
--cc=dbueso@suse.de \
--cc=hal.rosenstock@gmail.com \
--cc=hpa@zytor.com \
--cc=infinipath@intel.com \
--cc=jgross@suse.com \
--cc=jgunthorpe@obsidianresearch.com \
--cc=linux-fbdev@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=luto@amacapital.net \
--cc=mcgrof@do-not-panic.com \
--cc=mchehab@osg.samsung.com \
--cc=mgorman@suse.de \
--cc=mike.marciniszyn@intel.com \
--cc=mingo@kernel.org \
--cc=plagnioj@jcrosoft.com \
--cc=rickard_strandqvist@spectrumdigital.se \
--cc=roland@purestorage.com \
--cc=sbsiddha@gmail.com \
--cc=sean.hefty@intel.com \
--cc=syrjala@sci.fi \
--cc=tglx@linutronix.de \
--cc=toshi.kani@hp.com \
--cc=vbabka@suse.cz \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).