PC8 Residency on Broadwell hardware

PC8 Residency on Broadwell hardware

[Andrew Cooper]

Hello,

c/s 350bc1a9d4 "x86: support newer Intel CPU models" changed the set of
MSRs read by Xeon Broadwell hardware (specifically, model 79 / 0x47).

Rereading the manual, it does indeed indicate that this MSR is available.

However, experimentally it is not.  All Broadwell hardware XenServer has
(both SDPs and production systems) reliably take a #GP fault when trying
to read this MSR.  Haswell hardware appears fine (and indeed, was
reading that MSR before).

Intel: Please can you confirm whether the documentation is correct?  If
it is, do you have any idea why the MSR might not be available?  I can't
spot anything relevant in the platform errata documents.

Thanks,

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: PC8 Residency on Broadwell hardware

[Jan Beulich]

>>> On 12.09.16 at 10:47, <andrew.cooper3@citrix.com> wrote:
> c/s 350bc1a9d4 "x86: support newer Intel CPU models" changed the set of
> MSRs read by Xeon Broadwell hardware (specifically, model 79 / 0x47).
> 
> Rereading the manual, it does indeed indicate that this MSR is available.
> 
> However, experimentally it is not.  All Broadwell hardware XenServer has
> (both SDPs and production systems) reliably take a #GP fault when trying
> to read this MSR.  Haswell hardware appears fine (and indeed, was
> reading that MSR before).
> 
> Intel: Please can you confirm whether the documentation is correct?  If
> it is, do you have any idea why the MSR might not be available?  I can't
> spot anything relevant in the platform errata documents.

And indeed I had requested this same information already well over
a week ago, without having heard back.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: PC8 Residency on Broadwell hardware

[Nakajima, Jun]

[-- Attachment #1.1: Type: text/plain, Size: 1461 bytes --]


On Sep 12, 2016, at 2:00 AM, Jan Beulich <JBeulich@suse.com<mailto:JBeulich@suse.com>> wrote:

On 12.09.16 at 10:47, <andrew.cooper3@citrix.com<mailto:andrew.cooper3@citrix.com>> wrote:
c/s 350bc1a9d4 "x86: support newer Intel CPU models" changed the set of
MSRs read by Xeon Broadwell hardware (specifically, model 79 / 0x47).

Rereading the manual, it does indeed indicate that this MSR is available.

However, experimentally it is not.  All Broadwell hardware XenServer has
(both SDPs and production systems) reliably take a #GP fault when trying
to read this MSR.  Haswell hardware appears fine (and indeed, was
reading that MSR before).

Where did you find the info? I think you are talking about MSR_PKG_C8_RESIDENCY (630H).

The SDM says (35.13):

"Processors with signatures 06_3DH and 06_47H support the MSR interfaces listed in Table 35-18, Table 35-19, Table 35-20, Table 35-23, Table 35-27, Table 35-28, Table 35-32, and Table 35-33.”



Intel: Please can you confirm whether the documentation is correct?  If
it is, do you have any idea why the MSR might not be available?  I can't
spot anything relevant in the platform errata documents.

And indeed I had requested this same information already well over
a week ago, without having heard back.

Sorry, it seems that the email was buried in the pile of the emails that I received while I was in Toronto.


Jan


---
Jun
Intel Open Source Technology Center

[-- Attachment #1.2: Type: text/html, Size: 3048 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: PC8 Residency on Broadwell hardware

[Jan Beulich]

>>> On 12.09.16 at 19:28, <jun.nakajima@intel.com> wrote:

Please try to get quoting right - your response was rather hard to
follow.

> On Sep 12, 2016, at 2:00 AM, Jan Beulich 
> <JBeulich@suse.com<mailto:JBeulich@suse.com>> wrote:
> 
> On 12.09.16 at 10:47, 
> <andrew.cooper3@citrix.com<mailto:andrew.cooper3@citrix.com>> wrote:
> c/s 350bc1a9d4 "x86: support newer Intel CPU models" changed the set of
> MSRs read by Xeon Broadwell hardware (specifically, model 79 / 0x47).

I think this was misleading: 79 == 0x4f. Andrew, can you confirm in
your case please?

> Rereading the manual, it does indeed indicate that this MSR is available.
> 
> However, experimentally it is not.  All Broadwell hardware XenServer has
> (both SDPs and production systems) reliably take a #GP fault when trying
> to read this MSR.  Haswell hardware appears fine (and indeed, was
> reading that MSR before).
> 
> Where did you find the info? I think you are talking about 
> MSR_PKG_C8_RESIDENCY (630H).

Yes.

> The SDM says (35.13):
> 
> "Processors with signatures 06_3DH and 06_47H support the MSR interfaces 
> listed in Table 35-18, Table 35-19, Table 35-20, Table 35-23, Table 35-27, 
> Table 35-28, Table 35-32, and Table 35-33.”

Model 0x4f is what we're talking about, and table 35-36 has the information
on MSR_PKG_C8_RESIDENCY (630H).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: PC8 Residency on Broadwell hardware

[Andrew Cooper]

On 13/09/16 08:51, Jan Beulich wrote:
>>>> On 12.09.16 at 19:28, <jun.nakajima@intel.com> wrote:
> Please try to get quoting right - your response was rather hard to
> follow.
>
>> On Sep 12, 2016, at 2:00 AM, Jan Beulich 
>> <JBeulich@suse.com<mailto:JBeulich@suse.com>> wrote:
>>
>> On 12.09.16 at 10:47, 
>> <andrew.cooper3@citrix.com<mailto:andrew.cooper3@citrix.com>> wrote:
>> c/s 350bc1a9d4 "x86: support newer Intel CPU models" changed the set of
>> MSRs read by Xeon Broadwell hardware (specifically, model 79 / 0x47).
> I think this was misleading: 79 == 0x4f. Andrew, can you confirm in
> your case please?

Very sorry for the confusion.  Yes, I was talking about model 0x4f.

>> Rereading the manual, it does indeed indicate that this MSR is available.
>>
>> However, experimentally it is not.  All Broadwell hardware XenServer has
>> (both SDPs and production systems) reliably take a #GP fault when trying
>> to read this MSR.  Haswell hardware appears fine (and indeed, was
>> reading that MSR before).
>>
>> Where did you find the info? I think you are talking about 
>> MSR_PKG_C8_RESIDENCY (630H).
> Yes.
>
>> The SDM says (35.13):
>>
>> "Processors with signatures 06_3DH and 06_47H support the MSR interfaces 
>> listed in Table 35-18, Table 35-19, Table 35-20, Table 35-23, Table 35-27, 
>> Table 35-28, Table 35-32, and Table 35-33.”
> Model 0x4f is what we're talking about, and table 35-36 has the information
> on MSR_PKG_C8_RESIDENCY (630H).

Correct.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: PC8 Residency on Broadwell hardware

[Nakajima, Jun]

> On Sep 13, 2016, at 2:05 AM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
> 
> On 13/09/16 08:51, Jan Beulich wrote:
>>>>> On 12.09.16 at 19:28, <jun.nakajima@intel.com> wrote:
>> Please try to get quoting right - your response was rather hard to
>> follow.
>> 
>>> On Sep 12, 2016, at 2:00 AM, Jan Beulich 
>>> <JBeulich@suse.com<mailto:JBeulich@suse.com>> wrote:
>>> 
>>> On 12.09.16 at 10:47, 
>>> <andrew.cooper3@citrix.com<mailto:andrew.cooper3@citrix.com>> wrote:
>>> c/s 350bc1a9d4 "x86: support newer Intel CPU models" changed the set of
>>> MSRs read by Xeon Broadwell hardware (specifically, model 79 / 0x47).
>> I think this was misleading: 79 == 0x4f. Andrew, can you confirm in
>> your case please?
> 
> Very sorry for the confusion.  Yes, I was talking about model 0x4f.
> 
>>> Rereading the manual, it does indeed indicate that this MSR is available.
>>> 
>>> However, experimentally it is not.  All Broadwell hardware XenServer has
>>> (both SDPs and production systems) reliably take a #GP fault when trying
>>> to read this MSR.  Haswell hardware appears fine (and indeed, was
>>> reading that MSR before).
>>> 
>>> Where did you find the info? I think you are talking about 
>>> MSR_PKG_C8_RESIDENCY (630H).
>> Yes.
>> 
>>> The SDM says (35.13):
>>> 
>>> "Processors with signatures 06_3DH and 06_47H support the MSR interfaces 
>>> listed in Table 35-18, Table 35-19, Table 35-20, Table 35-23, Table 35-27, 
>>> Table 35-28, Table 35-32, and Table 35-33.”
>> Model 0x4f is what we're talking about, and table 35-36 has the information
>> on MSR_PKG_C8_RESIDENCY (630H).
> 
> Correct.
> 
> ~Andrew

OK. We are taking a look at that now. Thanks for the report.

---
Jun
Intel Open Source Technology Center
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: PC8 Residency on Broadwell hardware

[Nakajima, Jun]

> On Sep 13, 2016, at 9:10 AM, Nakajima, Jun <jun.nakajima@intel.com> wrote:
> 
> 
>> On Sep 13, 2016, at 2:05 AM, Andrew Cooper <andrew.cooper3@citrix.com> wrote:
>> 
>> On 13/09/16 08:51, Jan Beulich wrote:
>>>>>> On 12.09.16 at 19:28, <jun.nakajima@intel.com> wrote:
>>> Please try to get quoting right - your response was rather hard to
>>> follow.
>>> 
>>>> On Sep 12, 2016, at 2:00 AM, Jan Beulich 
>>>> <JBeulich@suse.com<mailto:JBeulich@suse.com>> wrote:
>>>> 
>>>> On 12.09.16 at 10:47, 
>>>> <andrew.cooper3@citrix.com<mailto:andrew.cooper3@citrix.com>> wrote:
>>>> c/s 350bc1a9d4 "x86: support newer Intel CPU models" changed the set of
>>>> MSRs read by Xeon Broadwell hardware (specifically, model 79 / 0x47).
>>> I think this was misleading: 79 == 0x4f. Andrew, can you confirm in
>>> your case please?
>> 
>> Very sorry for the confusion.  Yes, I was talking about model 0x4f.
>> 
>>>> Rereading the manual, it does indeed indicate that this MSR is available.
>>>> 
>>>> However, experimentally it is not.  All Broadwell hardware XenServer has
>>>> (both SDPs and production systems) reliably take a #GP fault when trying
>>>> to read this MSR.  Haswell hardware appears fine (and indeed, was
>>>> reading that MSR before).
>>>> 
>>>> Where did you find the info? I think you are talking about 
>>>> MSR_PKG_C8_RESIDENCY (630H).
>>> Yes.
>>> 
>>>> The SDM says (35.13):
>>>> 
>>>> "Processors with signatures 06_3DH and 06_47H support the MSR interfaces 
>>>> listed in Table 35-18, Table 35-19, Table 35-20, Table 35-23, Table 35-27, 
>>>> Table 35-28, Table 35-32, and Table 35-33.”
>>> Model 0x4f is what we're talking about, and table 35-36 has the information
>>> on MSR_PKG_C8_RESIDENCY (630H).
>> 
>> Correct.
>> 
>> ~Andrew
> 
> OK. We are taking a look at that now. Thanks for the report.
> 

We’ll remove the following MSRs from the table 35-36 in the next release: 
MSR_PKG_C8_RESIDENCY, MSR_PKG_C9_RESIDENCY and MSR_PKG_C10_RESIDENCY

Thank you for reporting this.
---
Jun
Intel Open Source Technology Center
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel