WT memory type on x86_64?

WT memory type on x86_64?

[Andy Lutomirski]

For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
write-through memory.  I'd like to gauge how unpleasant this will be.

AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
entry.  Sadly, MTRRs are in short supply, and the four fully-usable
PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
crossed and hope that there are enough free MTRRs, or I could try to
free up a PAT entry.

How nasty will the latter be?  I just looked at two rather different
modern Sandy Bridge machines, and BIOS doesn't appear to set up any
MTRRs in the WC or WP states.  As long as those MTRR types aren't
used, I think the UC- PAT entry is useless -- it behaves identically
to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
video memory.  Is there any need for this on modern machines?  That
is, are there any drivers that actually need the mtrr_add call to
succeed on a machine that has a working PAT?

If not, then here's a proposal:  At startup, if there are no WC or WP
MTRRs and the CPU is new enough, then set a flag for an alternative
PAT.  In alternative PAT mode, UC- is replaced with WT in the PAT and
mtrr_add fails when attempting to add a WC or WP entry.  Add
set_memory_wt, etc.  They fail in non-alternative-PAT mode.  This gets
a bit unpleasant, since _PAGE_CACHE_UC_MINUS will have a different
meaning depending on the mode.

A less conservative proposal would be to stop using UC- in the PAT
entirely.  The memtype code could learn to emulate UC- when there's an
overlapping WC or WP MTRR.

Any thoughts?  Are there known errata here?  Is there a good reason
why the kernel needs UC-?  Will this be such a big can of worms that I
should just hope there's an available MTRR?

--Andy

[1]  I hope to be able to upstream all kernel code related to this
project.  It will be neat -- I promise.  It will depend on convincing
the people on the other end of the NDA that they should let me.

Re: WT memory type on x86_64?

[Andy Lutomirski]

On Wed, Apr 24, 2013 at 12:33 PM, Andy Lutomirski <luto@amacapital.net> wrote:
> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
> write-through memory.  I'd like to gauge how unpleasant this will be.
>
> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
> crossed and hope that there are enough free MTRRs, or I could try to
> free up a PAT entry.
>
> How nasty will the latter be?  I just looked at two rather different
> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
> MTRRs in the WC or WP states.  As long as those MTRR types aren't
> used, I think the UC- PAT entry is useless -- it behaves identically
> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
> video memory.  Is there any need for this on modern machines?  That
> is, are there any drivers that actually need the mtrr_add call to
> succeed on a machine that has a working PAT?
>

FWIW, I've done a bit of a survey.  Things that use UC or UC- include:

 - ioremap_nocache: ISTM that any correct caller wants genuine UC memory.

 - plain ioremap: Are there architectures where it's not
ioremap_nocache?  (Tn any case, this is irrelevant.)

 - pci_iomap: This is used all over the framebuffer code.  It seems to
be equivalent to ioremap or ioremap_nocache, which are the same thing
on x86.

 - AGP: The AGP code seems inconsistent.  alloc_page gets a cacheable
page of RAM.  alloc_pages gets uncached pages of RAM.  In there's a WC
MTRR on RAM, then everything is screwed up anyway.

 - ttm: This code is newish.  I imagine that everything using ttm that
wants WC memory asks TTM for WC, which will work just fine.  In any
case, the allocations are AFAICS backed by RAM, so there should be no
conflicts.

 - radeon's gart: Ditto

 - efi: presumably !WB means UC is fine.  (Why would EFI need WC?)

 - uvesafb: The MTRR code is terrifying.  It looks nearly useless (it
has alignment issues) and it's unnecessary on a system with PAT.  In
any case, this code certainly isn't expecting a WC MTRR with any kind
of mapping other than ioremap_wc.


mtrr_add users include:

 - tdfxfb, vt8623fb, sgivwfb, s3fb, etc. should be converted to use ioremap_wc
 - myri10ge tries to use an MTRR.  This is, IMO, strange.
 - Infiniband.  I think it's okay if the MTRR doesn't work.


The only problematic (and not trivially fixable) thing I found is
pci_mmap_page_range, which uses UC- and is part of the ABI -- old X
drivers may care.

I wonder if X (using UMS) will slow down if WC MTRRs become illegal or
stop being added by old framebuffer drivers.  (If so, they can be
randomly slow anyway -- lots of machines have no free MTRRs).

--Andy

Re: WT memory type on x86_64?

[Dave Airlie]

On Sat, Apr 27, 2013 at 10:37 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Wed, Apr 24, 2013 at 12:33 PM, Andy Lutomirski <luto@amacapital.net> wrote:
>> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
>> write-through memory.  I'd like to gauge how unpleasant this will be.
>>
>> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
>> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
>> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
>> crossed and hope that there are enough free MTRRs, or I could try to
>> free up a PAT entry.
>>
>> How nasty will the latter be?  I just looked at two rather different
>> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
>> MTRRs in the WC or WP states.  As long as those MTRR types aren't
>> used, I think the UC- PAT entry is useless -- it behaves identically
>> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
>> video memory.  Is there any need for this on modern machines?  That
>> is, are there any drivers that actually need the mtrr_add call to
>> succeed on a machine that has a working PAT?
>>
>
> FWIW, I've done a bit of a survey.  Things that use UC or UC- include:
>
>  - ioremap_nocache: ISTM that any correct caller wants genuine UC memory.
>
>  - plain ioremap: Are there architectures where it's not
> ioremap_nocache?  (Tn any case, this is irrelevant.)
>
>  - pci_iomap: This is used all over the framebuffer code.  It seems to
> be equivalent to ioremap or ioremap_nocache, which are the same thing
> on x86.
>
>  - AGP: The AGP code seems inconsistent.  alloc_page gets a cacheable
> page of RAM.  alloc_pages gets uncached pages of RAM.  In there's a WC
> MTRR on RAM, then everything is screwed up anyway.
>
>  - ttm: This code is newish.  I imagine that everything using ttm that
> wants WC memory asks TTM for WC, which will work just fine.  In any
> case, the allocations are AFAICS backed by RAM, so there should be no
> conflicts.
>
>  - radeon's gart: Ditto
>
>  - efi: presumably !WB means UC is fine.  (Why would EFI need WC?)
>
>  - uvesafb: The MTRR code is terrifying.  It looks nearly useless (it
> has alignment issues) and it's unnecessary on a system with PAT.  In
> any case, this code certainly isn't expecting a WC MTRR with any kind
> of mapping other than ioremap_wc.
>
>
> mtrr_add users include:
>
>  - tdfxfb, vt8623fb, sgivwfb, s3fb, etc. should be converted to use ioremap_wc
>  - myri10ge tries to use an MTRR.  This is, IMO, strange.
>  - Infiniband.  I think it's okay if the MTRR doesn't work.
>
>
> The only problematic (and not trivially fixable) thing I found is
> pci_mmap_page_range, which uses UC- and is part of the ABI -- old X
> drivers may care.
>
> I wonder if X (using UMS) will slow down if WC MTRRs become illegal or
> stop being added by old framebuffer drivers.  (If so, they can be
> randomly slow anyway -- lots of machines have no free MTRRs).

Don't forget you can add mtrrs from userspace via /proc/mtrr. I'm not sure
what sort ABI guarantees are on this.

TTM allocations are not necessarily backed by RAM, they can also from
device memory.

Also i915 has mtrr code, but we avoid touching mtrrs if we are on a PAT cpu.

Dave.

Re: WT memory type on x86_64?

[Dave Airlie]

On Sat, Apr 27, 2013 at 11:00 AM, Dave Airlie <airlied@gmail.com> wrote:
> On Sat, Apr 27, 2013 at 10:37 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Wed, Apr 24, 2013 at 12:33 PM, Andy Lutomirski <luto@amacapital.net> wrote:
>>> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
>>> write-through memory.  I'd like to gauge how unpleasant this will be.
>>>
>>> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
>>> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
>>> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
>>> crossed and hope that there are enough free MTRRs, or I could try to
>>> free up a PAT entry.
>>>
>>> How nasty will the latter be?  I just looked at two rather different
>>> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
>>> MTRRs in the WC or WP states.  As long as those MTRR types aren't
>>> used, I think the UC- PAT entry is useless -- it behaves identically
>>> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
>>> video memory.  Is there any need for this on modern machines?  That
>>> is, are there any drivers that actually need the mtrr_add call to
>>> succeed on a machine that has a working PAT?
>>>
>>
>> FWIW, I've done a bit of a survey.  Things that use UC or UC- include:
>>
>>  - ioremap_nocache: ISTM that any correct caller wants genuine UC memory.
>>
>>  - plain ioremap: Are there architectures where it's not
>> ioremap_nocache?  (Tn any case, this is irrelevant.)
>>
>>  - pci_iomap: This is used all over the framebuffer code.  It seems to
>> be equivalent to ioremap or ioremap_nocache, which are the same thing
>> on x86.
>>
>>  - AGP: The AGP code seems inconsistent.  alloc_page gets a cacheable
>> page of RAM.  alloc_pages gets uncached pages of RAM.  In there's a WC
>> MTRR on RAM, then everything is screwed up anyway.
>>
>>  - ttm: This code is newish.  I imagine that everything using ttm that
>> wants WC memory asks TTM for WC, which will work just fine.  In any
>> case, the allocations are AFAICS backed by RAM, so there should be no
>> conflicts.
>>
>>  - radeon's gart: Ditto
>>
>>  - efi: presumably !WB means UC is fine.  (Why would EFI need WC?)
>>
>>  - uvesafb: The MTRR code is terrifying.  It looks nearly useless (it
>> has alignment issues) and it's unnecessary on a system with PAT.  In
>> any case, this code certainly isn't expecting a WC MTRR with any kind
>> of mapping other than ioremap_wc.
>>
>>
>> mtrr_add users include:
>>
>>  - tdfxfb, vt8623fb, sgivwfb, s3fb, etc. should be converted to use ioremap_wc
>>  - myri10ge tries to use an MTRR.  This is, IMO, strange.
>>  - Infiniband.  I think it's okay if the MTRR doesn't work.
>>
>>
>> The only problematic (and not trivially fixable) thing I found is
>> pci_mmap_page_range, which uses UC- and is part of the ABI -- old X
>> drivers may care.
>>
>> I wonder if X (using UMS) will slow down if WC MTRRs become illegal or
>> stop being added by old framebuffer drivers.  (If so, they can be
>> randomly slow anyway -- lots of machines have no free MTRRs).
>
> Don't forget you can add mtrrs from userspace via /proc/mtrr. I'm not sure
> what sort ABI guarantees are on this.
>
> TTM allocations are not necessarily backed by RAM, they can also from
> device memory.
>
> Also i915 has mtrr code, but we avoid touching mtrrs if we are on a PAT cpu.

i915 also has this comment:
        /* Set up a WC MTRR for non-PAT systems.  This is more common than
         * one would think, because the kernel disables PAT on first
         * generation Core chips because WC PAT gets overridden by a UC
         * MTRR if present.  Even if a UC MTRR isn't present.
         */

Dave.

Re: WT memory type on x86_64?

[Andy Lutomirski]

On Fri, Apr 26, 2013 at 6:01 PM, Dave Airlie <airlied@gmail.com> wrote:
> On Sat, Apr 27, 2013 at 11:00 AM, Dave Airlie <airlied@gmail.com> wrote:
>> On Sat, Apr 27, 2013 at 10:37 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>> On Wed, Apr 24, 2013 at 12:33 PM, Andy Lutomirski <luto@amacapital.net> wrote:
>>>> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
>>>> write-through memory.  I'd like to gauge how unpleasant this will be.
>>>>
>>>> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
>>>> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
>>>> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
>>>> crossed and hope that there are enough free MTRRs, or I could try to
>>>> free up a PAT entry.
>>>>
>>>> How nasty will the latter be?  I just looked at two rather different
>>>> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
>>>> MTRRs in the WC or WP states.  As long as those MTRR types aren't
>>>> used, I think the UC- PAT entry is useless -- it behaves identically
>>>> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
>>>> video memory.  Is there any need for this on modern machines?  That
>>>> is, are there any drivers that actually need the mtrr_add call to
>>>> succeed on a machine that has a working PAT?
>>>>
>>>
>>> FWIW, I've done a bit of a survey.  Things that use UC or UC- include:
>>>
>>>  - ioremap_nocache: ISTM that any correct caller wants genuine UC memory.
>>>
>>>  - plain ioremap: Are there architectures where it's not
>>> ioremap_nocache?  (Tn any case, this is irrelevant.)
>>>
>>>  - pci_iomap: This is used all over the framebuffer code.  It seems to
>>> be equivalent to ioremap or ioremap_nocache, which are the same thing
>>> on x86.
>>>
>>>  - AGP: The AGP code seems inconsistent.  alloc_page gets a cacheable
>>> page of RAM.  alloc_pages gets uncached pages of RAM.  In there's a WC
>>> MTRR on RAM, then everything is screwed up anyway.
>>>
>>>  - ttm: This code is newish.  I imagine that everything using ttm that
>>> wants WC memory asks TTM for WC, which will work just fine.  In any
>>> case, the allocations are AFAICS backed by RAM, so there should be no
>>> conflicts.
>>>
>>>  - radeon's gart: Ditto
>>>
>>>  - efi: presumably !WB means UC is fine.  (Why would EFI need WC?)
>>>
>>>  - uvesafb: The MTRR code is terrifying.  It looks nearly useless (it
>>> has alignment issues) and it's unnecessary on a system with PAT.  In
>>> any case, this code certainly isn't expecting a WC MTRR with any kind
>>> of mapping other than ioremap_wc.
>>>
>>>
>>> mtrr_add users include:
>>>
>>>  - tdfxfb, vt8623fb, sgivwfb, s3fb, etc. should be converted to use ioremap_wc
>>>  - myri10ge tries to use an MTRR.  This is, IMO, strange.
>>>  - Infiniband.  I think it's okay if the MTRR doesn't work.
>>>
>>>
>>> The only problematic (and not trivially fixable) thing I found is
>>> pci_mmap_page_range, which uses UC- and is part of the ABI -- old X
>>> drivers may care.
>>>
>>> I wonder if X (using UMS) will slow down if WC MTRRs become illegal or
>>> stop being added by old framebuffer drivers.  (If so, they can be
>>> randomly slow anyway -- lots of machines have no free MTRRs).
>>
>> Don't forget you can add mtrrs from userspace via /proc/mtrr. I'm not sure
>> what sort ABI guarantees are on this.
>>
>> TTM allocations are not necessarily backed by RAM, they can also from
>> device memory.
>>
>> Also i915 has mtrr code, but we avoid touching mtrrs if we are on a PAT cpu.
>
> i915 also has this comment:
>         /* Set up a WC MTRR for non-PAT systems.  This is more common than
>          * one would think, because the kernel disables PAT on first
>          * generation Core chips because WC PAT gets overridden by a UC
>          * MTRR if present.  Even if a UC MTRR isn't present.
>          */

I'm playing with cleaning this stuff up, and I found a possible bug.
drm_io_prot in drm_vm.c seems to hardcode the non-PAT incantation for
UC- (if I'm remembering my flags right), which is (fortunately)
equivalent to pgprot_noncached.  Shouldn't it be checking the
_DRM_WRITE_COMBINING and using pgprot_writecombine if the driver
requested write combining?

Given this, I'm not entirely clear on how non-GEM, non-TTM drivers
(i.e. drivers that use drm_addmap) end up with the correct memtypes
set.

Am I missing some reason why this code is correct?  Unfortunately, I
don't think I have any of the right hardware to test on.

--Andy

Re: WT memory type on x86_64?

[Konrad Rzeszutek Wilk]

On Wed, Apr 24, 2013 at 12:33:33PM -0700, Andy Lutomirski wrote:
> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
> write-through memory.  I'd like to gauge how unpleasant this will be.
> 
> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
> crossed and hope that there are enough free MTRRs, or I could try to
> free up a PAT entry.
> 
> How nasty will the latter be?  I just looked at two rather different
> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
> MTRRs in the WC or WP states.  As long as those MTRR types aren't
> used, I think the UC- PAT entry is useless -- it behaves identically
> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
> video memory.  Is there any need for this on modern machines?  That
> is, are there any drivers that actually need the mtrr_add call to
> succeed on a machine that has a working PAT?
> 
> If not, then here's a proposal:  At startup, if there are no WC or WP
> MTRRs and the CPU is new enough, then set a flag for an alternative
> PAT.  In alternative PAT mode, UC- is replaced with WT in the PAT and
> mtrr_add fails when attempting to add a WC or WP entry.  Add
> set_memory_wt, etc.  They fail in non-alternative-PAT mode.  This gets
> a bit unpleasant, since _PAGE_CACHE_UC_MINUS will have a different
> meaning depending on the mode.
> 
> A less conservative proposal would be to stop using UC- in the PAT
> entirely.  The memtype code could learn to emulate UC- when there's an
> overlapping WC or WP MTRR.
> 
> Any thoughts?  Are there known errata here?  Is there a good reason
> why the kernel needs UC-?  Will this be such a big can of worms that I
> should just hope there's an available MTRR?

Stefan Bader (CC-ed here) was looking in making a "black-box" type
code wherein for any types but WC it would lookup in the PAT index
the right bit and provide that for the page_attr_set functions.

If I recall he had run in a bit of issue with of the hard-coded values
set by the macros.

> 
> --Andy
> 
> [1]  I hope to be able to upstream all kernel code related to this
> project.  It will be neat -- I promise.  It will depend on convincing
> the people on the other end of the NDA that they should let me.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: WT memory type on x86_64?

[Stefan Bader]

[-- Attachment #1: Type: text/plain, Size: 3514 bytes --]

On 08.05.2013 16:35, Konrad Rzeszutek Wilk wrote:
> On Wed, Apr 24, 2013 at 12:33:33PM -0700, Andy Lutomirski wrote:
>> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
>> write-through memory.  I'd like to gauge how unpleasant this will be.
>>
>> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
>> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
>> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
>> crossed and hope that there are enough free MTRRs, or I could try to
>> free up a PAT entry.
>>
>> How nasty will the latter be?  I just looked at two rather different
>> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
>> MTRRs in the WC or WP states.  As long as those MTRR types aren't
>> used, I think the UC- PAT entry is useless -- it behaves identically
>> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
>> video memory.  Is there any need for this on modern machines?  That
>> is, are there any drivers that actually need the mtrr_add call to
>> succeed on a machine that has a working PAT?
>>
>> If not, then here's a proposal:  At startup, if there are no WC or WP
>> MTRRs and the CPU is new enough, then set a flag for an alternative
>> PAT.  In alternative PAT mode, UC- is replaced with WT in the PAT and
>> mtrr_add fails when attempting to add a WC or WP entry.  Add
>> set_memory_wt, etc.  They fail in non-alternative-PAT mode.  This gets
>> a bit unpleasant, since _PAGE_CACHE_UC_MINUS will have a different
>> meaning depending on the mode.
>>
>> A less conservative proposal would be to stop using UC- in the PAT
>> entirely.  The memtype code could learn to emulate UC- when there's an
>> overlapping WC or WP MTRR.
>>
>> Any thoughts?  Are there known errata here?  Is there a good reason
>> why the kernel needs UC-?  Will this be such a big can of worms that I
>> should just hope there's an available MTRR?
> 
> Stefan Bader (CC-ed here) was looking in making a "black-box" type
> code wherein for any types but WC it would lookup in the PAT index
> the right bit and provide that for the page_attr_set functions.
> 
> If I recall he had run in a bit of issue with of the hard-coded values
> set by the macros.

It could be opening a can of worms. Right now there are many places / defines
that are based on the assumption that a certain combination of PCD and PWT in
pgprot maps to a certain cache type. PAT is deliberately left out because its
location changes for large pages (PAT and PSE share the same bit). That is the
reason for the PAT MSR is set in a way that results in the same cache type
regardless of the PAT bit being 0 or 1.

Not sure whether this is the reason but UC and UC- (as well as WB) are kept like
the backward compatible setup. So for systems with and without PAT cache types
map to the same. Only WT gets replaced by WC. For that whenever PCD is set it
means some uncached type.

> 
>>
>> --Andy
>>
>> [1]  I hope to be able to upstream all kernel code related to this
>> project.  It will be neat -- I promise.  It will depend on convincing
>> the people on the other end of the NDA that they should let me.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at  http://www.tux.org/lkml/
>>



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 899 bytes --]

Re: WT memory type on x86_64?

[Andy Lutomirski]

On Wed, May 8, 2013 at 8:14 AM, Stefan Bader <stefan.bader@canonical.com> wrote:
> On 08.05.2013 16:35, Konrad Rzeszutek Wilk wrote:
>> On Wed, Apr 24, 2013 at 12:33:33PM -0700, Andy Lutomirski wrote:
>>> For an upcoming (and, sadly, NDA'd [1]) project, I may need to use
>>> write-through memory.  I'd like to gauge how unpleasant this will be.
>>>
>>> AFAICT, modern CPUs allow the WT type to be set using MTRR or a PAT
>>> entry.  Sadly, MTRRs are in short supply, and the four fully-usable
>>> PAT slots are used for UC, UC-, WB, and WC.  I can keep my fingers
>>> crossed and hope that there are enough free MTRRs, or I could try to
>>> free up a PAT entry.
>>>
>>> How nasty will the latter be?  I just looked at two rather different
>>> modern Sandy Bridge machines, and BIOS doesn't appear to set up any
>>> MTRRs in the WC or WP states.  As long as those MTRR types aren't
>>> used, I think the UC- PAT entry is useless -- it behaves identically
>>> to UC.  Lots of DRM drivers, though, seen to add a WC MTRR to cover
>>> video memory.  Is there any need for this on modern machines?  That
>>> is, are there any drivers that actually need the mtrr_add call to
>>> succeed on a machine that has a working PAT?
>>>
>>> If not, then here's a proposal:  At startup, if there are no WC or WP
>>> MTRRs and the CPU is new enough, then set a flag for an alternative
>>> PAT.  In alternative PAT mode, UC- is replaced with WT in the PAT and
>>> mtrr_add fails when attempting to add a WC or WP entry.  Add
>>> set_memory_wt, etc.  They fail in non-alternative-PAT mode.  This gets
>>> a bit unpleasant, since _PAGE_CACHE_UC_MINUS will have a different
>>> meaning depending on the mode.
>>>
>>> A less conservative proposal would be to stop using UC- in the PAT
>>> entirely.  The memtype code could learn to emulate UC- when there's an
>>> overlapping WC or WP MTRR.
>>>
>>> Any thoughts?  Are there known errata here?  Is there a good reason
>>> why the kernel needs UC-?  Will this be such a big can of worms that I
>>> should just hope there's an available MTRR?
>>
>> Stefan Bader (CC-ed here) was looking in making a "black-box" type
>> code wherein for any types but WC it would lookup in the PAT index
>> the right bit and provide that for the page_attr_set functions.
>>
>> If I recall he had run in a bit of issue with of the hard-coded values
>> set by the macros.
>
> It could be opening a can of worms. Right now there are many places / defines
> that are based on the assumption that a certain combination of PCD and PWT in
> pgprot maps to a certain cache type. PAT is deliberately left out because its
> location changes for large pages (PAT and PSE share the same bit). That is the
> reason for the PAT MSR is set in a way that results in the same cache type
> regardless of the PAT bit being 0 or 1.

Fortunately, these appear to all be either in arch/x86 or in graphics
drivers, and the latter are easy to fix.

>
> Not sure whether this is the reason but UC and UC- (as well as WB) are kept like
> the backward compatible setup. So for systems with and without PAT cache types
> map to the same. Only WT gets replaced by WC. For that whenever PCD is set it
> means some uncached type.

This also makes the pgprot_xyz macros faster -- they can do the same
thing regardless of PAT settings.

--Andy

Re: WT memory type on x86_64?

[H. Peter Anvin]

On 05/08/2013 09:37 AM, Andy Lutomirski wrote:
>>
>> Not sure whether this is the reason but UC and UC- (as well as WB) are kept like
>> the backward compatible setup. So for systems with and without PAT cache types
>> map to the same. Only WT gets replaced by WC. For that whenever PCD is set it
>> means some uncached type.
> 
> This also makes the pgprot_xyz macros faster -- they can do the same
> thing regardless of PAT settings.
> 

It also deals with errata in some very old CPUs... old enough that we
don't use PAT on them anyway AFAIK.

	-hpa