* Question about the CAT and CMT in Xen @ 2015-08-31 17:42 Meng Xu 2015-08-31 18:25 ` Andrew Cooper 0 siblings, 1 reply; 16+ messages in thread From: Meng Xu @ 2015-08-31 17:42 UTC (permalink / raw) To: xen-devel, Chao Peng [-- Attachment #1.1: Type: text/plain, Size: 2037 bytes --] Hi Chao, I'm Meng Xu from the University of Pennsylvania. We purchased a computer that has Intel(R) Xeon(R) CPU E5-2618L v3 @ 2.30GHz inside. According to http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, Intel E5-2618L v3 should support both CAT and CMT. I'm playing the latest Xen on the machine to evaluate how CAT and CMT work . The commit point I used is in the staging branch, commit 7b99717f62caeac08eea224a177cd28f047ac4b5, which was push on Aug. 17th, 2015. I checked that the branch has merged your code. After I boot up the system, I tried the following commands and it shows some errors. *** My question is *** : 1) Is there any specific BIOS configuration I need to configure? 2) The CMT is not enabled by default. Do I need to issue any command to enable it? I didn't find any document about how to enable it, according to http://xenbits.xen.org/docs/unstable-staging/misc/xl-psr.html. :-( 3) I'm not 100% sure if this machine we purchased supports the CAT. Right now, it shows "No such device" on the machine. But it may probably because the hypervisor forgot to initialize it? Below is the details of what I tried: # xl psr-hwinfo Cache Monitoring Technology (CMT): Enabled : 0 Cache Allocation Technology (CAT): libxl: error: libxl_psr.c:96:libxl__psr_cat_log_err_msg: CAT is not supported in this system: No such device Failed to get cat info #xl psr-cmt-attach 0 libxl: error: libxl_psr.c:70:libxl__psr_cmt_log_err_msg: CMT is not supported in this system: No such device Please let me know if you need any further information to confirm if the machine support the CAT or if it's because there is some issue with the software layer. :-) Thank you very much for your time and help in this question! Best regards, Meng ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ [-- Attachment #1.2: Type: text/html, Size: 2691 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-08-31 17:42 Question about the CAT and CMT in Xen Meng Xu @ 2015-08-31 18:25 ` Andrew Cooper 2015-08-31 20:09 ` Meng Xu 0 siblings, 1 reply; 16+ messages in thread From: Andrew Cooper @ 2015-08-31 18:25 UTC (permalink / raw) To: Meng Xu, xen-devel, Chao Peng [-- Attachment #1.1: Type: text/plain, Size: 2226 bytes --] On 31/08/15 18:42, Meng Xu wrote: > Hi Chao, > > I'm Meng Xu from the University of Pennsylvania. > > We purchased a computer that has Intel(R) Xeon(R) CPU E5-2618L v3 @ > 2.30GHz inside. > According to > http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, > Intel E5-2618L v3 should support both CAT and CMT. > > I'm playing the latest Xen on the machine to evaluate how CAT and CMT > work > . > > > The commit point I used is in the staging branch, commit > 7b99717f62caeac08eea224a177cd28f047ac4b5, which was push on Aug. 17th, > 2015. > I checked that the branch has merged your code. > > After I boot up the system, I tried the following commands and it > shows some errors. > > *** > My question is > *** > : > 1) Is there any specific BIOS configuration I need to configure? > 2) The CMT is not enabled by default. Do I need to issue any command > to enable it? I didn't find any document about how to enable it, > according to > http://xenbits.xen.org/docs/unstable-staging/misc/xl-psr.html. :-( > 3) I'm not 100% sure if this machine we purchased supports the CAT. > Right now, it shows "No such device" on the machine. > But it may probably because the hypervisor forgot to initialize it? > > > Below is the details of what I tried: > > # xl psr-hwinfo > Cache Monitoring Technology (CMT): > Enabled : 0 > Cache Allocation Technology (CAT): > libxl: error: libxl_psr.c:96:libxl__psr_cat_log_err_msg: CAT is not > supported in this system: No such device > Failed to get cat info > > #xl psr-cmt-attach 0 > libxl: error: libxl_psr.c:70:libxl__psr_cmt_log_err_msg: CMT is not > supported in this system: No such device > > Please let me know if you need any further information to confirm if > the machine support the CAT or if it's because there is some issue > with the software layer. :-) > > Thank you very much for your time and help in this question! > > Best regards, > > Meng To get started, use "psr=cat,cmt" on the Xen command line. Neither are enabled by default as they can potentially allocate quite a lot of resources behind the scenes. ~Andrew [-- Attachment #1.2: Type: text/html, Size: 3838 bytes --] [-- Attachment #2: Type: text/plain, Size: 126 bytes --] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-08-31 18:25 ` Andrew Cooper @ 2015-08-31 20:09 ` Meng Xu 2015-08-31 20:22 ` Andrew Cooper 2015-09-01 5:47 ` Chao Peng 0 siblings, 2 replies; 16+ messages in thread From: Meng Xu @ 2015-08-31 20:09 UTC (permalink / raw) To: Andrew Cooper; +Cc: Chao Peng, xen-devel Hi Andrew and Chao, 2015-08-31 14:25 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: > > On 31/08/15 18:42, Meng Xu wrote: > > Hi Chao, > > I'm Meng Xu from the University of Pennsylvania. > > We purchased a computer that has Intel(R) Xeon(R) CPU E5-2618L v3 @ 2.30GHz inside. > According to http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, Intel E5-2618L v3 should support both CAT and CMT. > > I'm playing the latest Xen on the machine to evaluate how CAT and CMT work > . > > > The commit point I used is in the staging branch, commit 7b99717f62caeac08eea224a177cd28f047ac4b5, which was push on Aug. 17th, 2015. > I checked that the branch has merged your code. > > After I boot up the system, I tried the following commands and it shows some errors. > > *** > My question is > *** > : > 1) Is there any specific BIOS configuration I need to configure? > 2) The CMT is not enabled by default. Do I need to issue any command to enable it? I didn't find any document about how to enable it, according to http://xenbits.xen.org/docs/unstable-staging/misc/xl-psr.html. :-( > 3) I'm not 100% sure if this machine we purchased supports the CAT. Right now, it shows "No such device" on the machine. > But it may probably because the hypervisor forgot to initialize it? > > > Below is the details of what I tried: > > # xl psr-hwinfo > Cache Monitoring Technology (CMT): > Enabled : 0 > Cache Allocation Technology (CAT): > libxl: error: libxl_psr.c:96:libxl__psr_cat_log_err_msg: CAT is not supported in this system: No such device > Failed to get cat info > > #xl psr-cmt-attach 0 > libxl: error: libxl_psr.c:70:libxl__psr_cmt_log_err_msg: CMT is not supported in this system: No such device > > Please let me know if you need any further information to confirm if the machine support the CAT or if it's because there is some issue with the software layer. :-) > > Thank you very much for your time and help in this question! > > Best regards, > > Meng > > > To get started, use "psr=cat,cmt" on the Xen command line. Neither are enabled by default as they can potentially allocate quite a lot of resources behind the scenes. Thank you very much for your suggestion! Yes, after adding the psr=cat:1,cmt:1, the CMT works. Command " xl psr-hwinfo" shows the total RMID as 31. It means CMT is enabled/working. However, the CAT is still not enabled on the socket. It have the same error message: Cache Allocation Technology (CAT): libxl: error: libxl_psr.c:96:libxl__psr_cat_log_err_msg: CAT is not enabled on the socket: No such file or directory Failed to get cat info I looked into the xen/arch/x86/psr.c and found that the function cat_cpu_init() just returned without initializing the variable "cat_socket_enable". Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function cat_cpu_init(). OK. I understand that the cpuid info shows that the CPU does not support CAT. However, according to the table at http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. I'm not sure which part is incorrect: the hardware or the software? (Hope Chao could give some insight about this.) Best regards, Meng -- ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-08-31 20:09 ` Meng Xu @ 2015-08-31 20:22 ` Andrew Cooper 2015-09-01 5:47 ` Chao Peng 1 sibling, 0 replies; 16+ messages in thread From: Andrew Cooper @ 2015-08-31 20:22 UTC (permalink / raw) To: Meng Xu; +Cc: Chao Peng, xen-devel On 31/08/15 21:09, Meng Xu wrote: > Hi Andrew and Chao, > > 2015-08-31 14:25 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >> On 31/08/15 18:42, Meng Xu wrote: >> >> Hi Chao, >> >> I'm Meng Xu from the University of Pennsylvania. >> >> We purchased a computer that has Intel(R) Xeon(R) CPU E5-2618L v3 @ 2.30GHz inside. >> According to http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, Intel E5-2618L v3 should support both CAT and CMT. >> >> I'm playing the latest Xen on the machine to evaluate how CAT and CMT work >> . >> >> >> The commit point I used is in the staging branch, commit 7b99717f62caeac08eea224a177cd28f047ac4b5, which was push on Aug. 17th, 2015. >> I checked that the branch has merged your code. >> >> After I boot up the system, I tried the following commands and it shows some errors. >> >> *** >> My question is >> *** >> : >> 1) Is there any specific BIOS configuration I need to configure? >> 2) The CMT is not enabled by default. Do I need to issue any command to enable it? I didn't find any document about how to enable it, according to http://xenbits.xen.org/docs/unstable-staging/misc/xl-psr.html. :-( >> 3) I'm not 100% sure if this machine we purchased supports the CAT. Right now, it shows "No such device" on the machine. >> But it may probably because the hypervisor forgot to initialize it? >> >> >> Below is the details of what I tried: >> >> # xl psr-hwinfo >> Cache Monitoring Technology (CMT): >> Enabled : 0 >> Cache Allocation Technology (CAT): >> libxl: error: libxl_psr.c:96:libxl__psr_cat_log_err_msg: CAT is not supported in this system: No such device >> Failed to get cat info >> >> #xl psr-cmt-attach 0 >> libxl: error: libxl_psr.c:70:libxl__psr_cmt_log_err_msg: CMT is not supported in this system: No such device >> >> Please let me know if you need any further information to confirm if the machine support the CAT or if it's because there is some issue with the software layer. :-) >> >> Thank you very much for your time and help in this question! >> >> Best regards, >> >> Meng >> >> >> To get started, use "psr=cat,cmt" on the Xen command line. Neither are enabled by default as they can potentially allocate quite a lot of resources behind the scenes. > > Thank you very much for your suggestion! Yes, after adding the > psr=cat:1,cmt:1, the CMT works. Command " > xl psr-hwinfo" shows the total RMID as 31. It means CMT is enabled/working. > > However, the CAT is still not enabled on the socket. It have the same > error message: > Cache Allocation Technology (CAT): > libxl: error: libxl_psr.c:96:libxl__psr_cat_log_err_msg: CAT is not > enabled on the socket: No such file or directory > Failed to get cat info > > > I looked into the xen/arch/x86/psr.c and found that the function > cat_cpu_init() just returned without initializing the variable > "cat_socket_enable". > > Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < > PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function > cat_cpu_init(). > > OK. I understand that the cpuid info shows that the CPU does not > support CAT. However, according to the table at > http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, > Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. > > I'm not sure which part is incorrect: the hardware or the software? > (Hope Chao could give some insight about this.) > > Best regards, IIRC, CAT support required a firmware update on the Broadwell hardware XenServer now has. See if there is a BIOS update available from your vendor. ~Andrew ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-08-31 20:09 ` Meng Xu 2015-08-31 20:22 ` Andrew Cooper @ 2015-09-01 5:47 ` Chao Peng 2015-09-01 12:55 ` Meng Xu 1 sibling, 1 reply; 16+ messages in thread From: Chao Peng @ 2015-09-01 5:47 UTC (permalink / raw) To: Meng Xu; +Cc: Andrew Cooper, xen-devel On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: > I looked into the xen/arch/x86/psr.c and found that the function > cat_cpu_init() just returned without initializing the variable > "cat_socket_enable". > > Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < > PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function > cat_cpu_init(). > > OK. I understand that the cpuid info shows that the CPU does not > support CAT. However, according to the table at > http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, > Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. > > I'm not sure which part is incorrect: the hardware or the software? > (Hope Chao could give some insight about this.) > Hmmm, from cpuid info it looks like this model does not support CAT. I'm not sure which microarchitecture it is. But if it is broadwell, hardware should support that, what you need is some fireware update, just as Andrew has said. But otherwise it is haswell, then the hardware probably doesn't support that. In that case the above link probably is also wrong. I can check that internally. Chao ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-01 5:47 ` Chao Peng @ 2015-09-01 12:55 ` Meng Xu 2015-09-01 13:04 ` Andrew Cooper 0 siblings, 1 reply; 16+ messages in thread From: Meng Xu @ 2015-09-01 12:55 UTC (permalink / raw) To: Chao Peng; +Cc: Andrew Cooper, xen-devel 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: > On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >> I looked into the xen/arch/x86/psr.c and found that the function >> cat_cpu_init() just returned without initializing the variable >> "cat_socket_enable". >> >> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >> cat_cpu_init(). >> >> OK. I understand that the cpuid info shows that the CPU does not >> support CAT. However, according to the table at >> http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, >> Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. >> >> I'm not sure which part is incorrect: the hardware or the software? >> (Hope Chao could give some insight about this.) >> > > Hmmm, from cpuid info it looks like this model does not support CAT. I'm > not sure which microarchitecture it is. According to http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2618L%20v3.html, 2618L v3 is Haswell. :-( > But if it is broadwell, hardware > should support that, what you need is some fireware update, just as Andrew > has said. But otherwise it is haswell, then the hardware probably doesn't > support that. In that case the above link probably is also wrong. I can check > that internally. Ah, could you please let me know if the link[1] is wrong? If it does not support CAT, I'm screwed. :'-( [1] http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html > > Chao -- ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-01 12:55 ` Meng Xu @ 2015-09-01 13:04 ` Andrew Cooper 2015-09-01 14:20 ` Meng Xu 0 siblings, 1 reply; 16+ messages in thread From: Andrew Cooper @ 2015-09-01 13:04 UTC (permalink / raw) To: Meng Xu, Chao Peng; +Cc: xen-devel On 01/09/15 13:55, Meng Xu wrote: > 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >>> I looked into the xen/arch/x86/psr.c and found that the function >>> cat_cpu_init() just returned without initializing the variable >>> "cat_socket_enable". >>> >>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >>> cat_cpu_init(). >>> >>> OK. I understand that the cpuid info shows that the CPU does not >>> support CAT. However, according to the table at >>> http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, >>> Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. >>> >>> I'm not sure which part is incorrect: the hardware or the software? >>> (Hope Chao could give some insight about this.) >>> >> Hmmm, from cpuid info it looks like this model does not support CAT. I'm >> not sure which microarchitecture it is. > According to http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2618L%20v3.html, > 2618L v3 is Haswell. :-( Wikipedia agrees. Haswell only has plain L3 cache usage information. It is Broadwell which adds memory bandwidth monitoring, and Cache Allocation. ~Andrew ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-01 13:04 ` Andrew Cooper @ 2015-09-01 14:20 ` Meng Xu 2015-09-01 14:30 ` Andrew Cooper 0 siblings, 1 reply; 16+ messages in thread From: Meng Xu @ 2015-09-01 14:20 UTC (permalink / raw) To: Andrew Cooper; +Cc: Chao Peng, xen-devel 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: > On 01/09/15 13:55, Meng Xu wrote: >> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >>>> I looked into the xen/arch/x86/psr.c and found that the function >>>> cat_cpu_init() just returned without initializing the variable >>>> "cat_socket_enable". >>>> >>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >>>> cat_cpu_init(). >>>> >>>> OK. I understand that the cpuid info shows that the CPU does not >>>> support CAT. However, according to the table at >>>> http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, >>>> Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. >>>> >>>> I'm not sure which part is incorrect: the hardware or the software? >>>> (Hope Chao could give some insight about this.) >>>> >>> Hmmm, from cpuid info it looks like this model does not support CAT. I'm >>> not sure which microarchitecture it is. >> According to http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2618L%20v3.html, >> 2618L v3 is Haswell. :-( > > Wikipedia agrees. > > Haswell only has plain L3 cache usage information. > > It is Broadwell which adds memory bandwidth monitoring, and Cache > Allocation. Ah, then the Intel's website [1] that lists the CAT-enabled processor is quite misleading. I checked several processors (2618Lv3, 2608Lv3, 2658v3) listed in [1] that is claimed to have CAT, and all of them belong to Haswell... I think "v3" actually represents Haswell. Andrew, is it possible to know which processor XenServer is currently using that is confirmed to have the CAT mechanism? Thank you very much! Best, [1] http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html Meng -- ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-01 14:20 ` Meng Xu @ 2015-09-01 14:30 ` Andrew Cooper 2015-09-01 14:42 ` Meng Xu 0 siblings, 1 reply; 16+ messages in thread From: Andrew Cooper @ 2015-09-01 14:30 UTC (permalink / raw) To: Meng Xu; +Cc: Chao Peng, xen-devel On 01/09/15 15:20, Meng Xu wrote: > 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >> On 01/09/15 13:55, Meng Xu wrote: >>> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >>>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >>>>> I looked into the xen/arch/x86/psr.c and found that the function >>>>> cat_cpu_init() just returned without initializing the variable >>>>> "cat_socket_enable". >>>>> >>>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >>>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >>>>> cat_cpu_init(). >>>>> >>>>> OK. I understand that the cpuid info shows that the CPU does not >>>>> support CAT. However, according to the table at >>>>> http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, >>>>> Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. >>>>> >>>>> I'm not sure which part is incorrect: the hardware or the software? >>>>> (Hope Chao could give some insight about this.) >>>>> >>>> Hmmm, from cpuid info it looks like this model does not support CAT. I'm >>>> not sure which microarchitecture it is. >>> According to http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2618L%20v3.html, >>> 2618L v3 is Haswell. :-( >> Wikipedia agrees. >> >> Haswell only has plain L3 cache usage information. >> >> It is Broadwell which adds memory bandwidth monitoring, and Cache >> Allocation. > Ah, then the Intel's website [1] that lists the CAT-enabled processor > is quite misleading. I checked several processors (2618Lv3, 2608Lv3, > 2658v3) listed in [1] that is claimed to have CAT, and all of them > belong to Haswell... I think "v3" actually represents Haswell. > > Andrew, is it possible to know which processor XenServer is currently > using that is confirmed to have the CAT mechanism? I cant be much help, unfortunately, >From /proc/cpuinfo processor : 14 vendor_id : GenuineIntel cpu family : 6 model : 79 model name : Genuine Intel(R) CPU 0000 @ 2.20GHz This is a 2-socket Broadwell-EP SDP and is probably a firmware revision out of date, but I can confirm that I have had a bit of a play with CMT and CAT on it. ~Andrew ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-01 14:30 ` Andrew Cooper @ 2015-09-01 14:42 ` Meng Xu 2015-09-02 1:47 ` Chao Peng 2015-09-02 1:51 ` Meng Xu 0 siblings, 2 replies; 16+ messages in thread From: Meng Xu @ 2015-09-01 14:42 UTC (permalink / raw) To: Andrew Cooper; +Cc: Chao Peng, xen-devel 2015-09-01 10:30 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: > On 01/09/15 15:20, Meng Xu wrote: >> 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >>> On 01/09/15 13:55, Meng Xu wrote: >>>> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >>>>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >>>>>> I looked into the xen/arch/x86/psr.c and found that the function >>>>>> cat_cpu_init() just returned without initializing the variable >>>>>> "cat_socket_enable". >>>>>> >>>>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >>>>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >>>>>> cat_cpu_init(). >>>>>> >>>>>> OK. I understand that the cpuid info shows that the CPU does not >>>>>> support CAT. However, according to the table at >>>>>> http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, >>>>>> Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. >>>>>> >>>>>> I'm not sure which part is incorrect: the hardware or the software? >>>>>> (Hope Chao could give some insight about this.) >>>>>> >>>>> Hmmm, from cpuid info it looks like this model does not support CAT. I'm >>>>> not sure which microarchitecture it is. >>>> According to http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2618L%20v3.html, >>>> 2618L v3 is Haswell. :-( >>> Wikipedia agrees. >>> >>> Haswell only has plain L3 cache usage information. >>> >>> It is Broadwell which adds memory bandwidth monitoring, and Cache >>> Allocation. >> Ah, then the Intel's website [1] that lists the CAT-enabled processor >> is quite misleading. I checked several processors (2618Lv3, 2608Lv3, >> 2658v3) listed in [1] that is claimed to have CAT, and all of them >> belong to Haswell... I think "v3" actually represents Haswell. >> >> Andrew, is it possible to know which processor XenServer is currently >> using that is confirmed to have the CAT mechanism? > > I cant be much help, unfortunately, Thank you very much for your trying to help! :-) Hope Chao could give more information about which Intel processors support the CAT. Best, Meng ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-01 14:42 ` Meng Xu @ 2015-09-02 1:47 ` Chao Peng 2015-09-02 1:53 ` Meng Xu 2015-09-02 1:51 ` Meng Xu 1 sibling, 1 reply; 16+ messages in thread From: Chao Peng @ 2015-09-02 1:47 UTC (permalink / raw) To: Meng Xu; +Cc: Andrew Cooper, xen-devel On Tue, Sep 01, 2015 at 10:42:57AM -0400, Meng Xu wrote: > 2015-09-01 10:30 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: > > On 01/09/15 15:20, Meng Xu wrote: > >> 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: > >>> On 01/09/15 13:55, Meng Xu wrote: > >>>> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: > >>>>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: > >>>>>> I looked into the xen/arch/x86/psr.c and found that the function > >>>>>> cat_cpu_init() just returned without initializing the variable > >>>>>> "cat_socket_enable". > >>>>>> > >>>>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < > >>>>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function > >>>>>> cat_cpu_init(). > >>>>>> > >>>>>> OK. I understand that the cpuid info shows that the CPU does not > >>>>>> support CAT. However, according to the table at > >>>>>> http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, > >>>>>> Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. > >>>>>> > >>>>>> I'm not sure which part is incorrect: the hardware or the software? > >>>>>> (Hope Chao could give some insight about this.) > >>>>>> > >>>>> Hmmm, from cpuid info it looks like this model does not support CAT. I'm > >>>>> not sure which microarchitecture it is. > >>>> According to http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2618L%20v3.html, > >>>> 2618L v3 is Haswell. :-( > >>> Wikipedia agrees. > >>> > >>> Haswell only has plain L3 cache usage information. > >>> > >>> It is Broadwell which adds memory bandwidth monitoring, and Cache > >>> Allocation. > >> Ah, then the Intel's website [1] that lists the CAT-enabled processor > >> is quite misleading. I checked several processors (2618Lv3, 2608Lv3, > >> 2658v3) listed in [1] that is claimed to have CAT, and all of them > >> belong to Haswell... I think "v3" actually represents Haswell. > >> > >> Andrew, is it possible to know which processor XenServer is currently > >> using that is confirmed to have the CAT mechanism? > > > > I cant be much help, unfortunately, > > Thank you very much for your trying to help! :-) > > Hope Chao could give more information about which Intel processors > support the CAT. Until now what I can tell you is: All Broadwell Server support CAT and some models of Haswell Server may also support it. And the good news is the information @ Intel's website [1] is accurate so your purchased "E5-2618L v3" should have it. But you definitely need to ask your vendor for the firmware update. Chao ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-02 1:47 ` Chao Peng @ 2015-09-02 1:53 ` Meng Xu 0 siblings, 0 replies; 16+ messages in thread From: Meng Xu @ 2015-09-02 1:53 UTC (permalink / raw) To: Chao Peng; +Cc: Andrew Cooper, xen-devel 2015-09-01 21:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: > On Tue, Sep 01, 2015 at 10:42:57AM -0400, Meng Xu wrote: >> 2015-09-01 10:30 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >> > On 01/09/15 15:20, Meng Xu wrote: >> >> 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >> >>> On 01/09/15 13:55, Meng Xu wrote: >> >>>> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >> >>>>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >> >>>>>> I looked into the xen/arch/x86/psr.c and found that the function >> >>>>>> cat_cpu_init() just returned without initializing the variable >> >>>>>> "cat_socket_enable". >> >>>>>> >> >>>>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >> >>>>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >> >>>>>> cat_cpu_init(). >> >>>>>> >> >>>>>> OK. I understand that the cpuid info shows that the CPU does not >> >>>>>> support CAT. However, according to the table at >> >>>>>> http://www.intel.com/content/www/us/en/communications/cache-monitoring-cache-allocation-technologies.html, >> >>>>>> Intel(R) Xeon(R) CPU E5-2618L v3 should support CAT. >> >>>>>> >> >>>>>> I'm not sure which part is incorrect: the hardware or the software? >> >>>>>> (Hope Chao could give some insight about this.) >> >>>>>> >> >>>>> Hmmm, from cpuid info it looks like this model does not support CAT. I'm >> >>>>> not sure which microarchitecture it is. >> >>>> According to http://www.cpu-world.com/CPUs/Xeon/Intel-Xeon%20E5-2618L%20v3.html, >> >>>> 2618L v3 is Haswell. :-( >> >>> Wikipedia agrees. >> >>> >> >>> Haswell only has plain L3 cache usage information. >> >>> >> >>> It is Broadwell which adds memory bandwidth monitoring, and Cache >> >>> Allocation. >> >> Ah, then the Intel's website [1] that lists the CAT-enabled processor >> >> is quite misleading. I checked several processors (2618Lv3, 2608Lv3, >> >> 2658v3) listed in [1] that is claimed to have CAT, and all of them >> >> belong to Haswell... I think "v3" actually represents Haswell. >> >> >> >> Andrew, is it possible to know which processor XenServer is currently >> >> using that is confirmed to have the CAT mechanism? >> > >> > I cant be much help, unfortunately, >> >> Thank you very much for your trying to help! :-) >> >> Hope Chao could give more information about which Intel processors >> support the CAT. > > Until now what I can tell you is: All Broadwell Server support CAT and some > models of Haswell Server may also support it. > > And the good news is the information @ Intel's website [1] is accurate > so your purchased "E5-2618L v3" should have it. But you definitely need > to ask your vendor for the firmware update. Ah, I didn't do the firmware update and it seems working on Linux with MSR tools and the CAT tool provided on 01.org. Now I suspect it is because of some bug in the CAT code of Xen. ;-) I just sent another email to explain why I think the tool on Linux works. :-P Thanks, Meng -- ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-01 14:42 ` Meng Xu 2015-09-02 1:47 ` Chao Peng @ 2015-09-02 1:51 ` Meng Xu 2015-09-02 2:31 ` Chao Peng 1 sibling, 1 reply; 16+ messages in thread From: Meng Xu @ 2015-09-02 1:51 UTC (permalink / raw) To: Andrew Cooper; +Cc: Chao Peng, Autee, Priya V, xen-devel Hi Andrew and Chao, [Important things go first] It turns out my machine (Intel E5-2618L v3) does have CAT capability! Xen gives the false alarm that my machine does not have it. This should be a bug, IMO. :-) 2015-09-01 10:42 GMT-04:00 Meng Xu <xumengpanda@gmail.com>: > 2015-09-01 10:30 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >> On 01/09/15 15:20, Meng Xu wrote: >>> 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >>>> On 01/09/15 13:55, Meng Xu wrote: >>>>> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >>>>>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >>>>>>> I looked into the xen/arch/x86/psr.c and found that the function >>>>>>> cat_cpu_init() just returned without initializing the variable >>>>>>> "cat_socket_enable". >>>>>>> >>>>>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >>>>>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >>>>>>> cat_cpu_init(). I'm thinking this check could be wrong for Intel E5-2618L v3. It should work on Chao's machine but not on mine. There should be a better way to check this probably. :-) --- I used another way to check the CAT capability, as suggested by Priya (cc.ed) from Intel. I did the following steps as Priya suggested: 1. Download msr-tools utility on your linux distribution to perform msr read write operations./ if you already have it installed modprobe msr 2. rdmsr 0xc91 which returns 0xfffff 3. wrmsr -p 1 0xc91 0xf which does not return anything 4. wrmsr -p 1 0xc8f 0x100000000 which does not return anything 5. rdmsr 0xc91 which returns 0xf This shows that the CPU does have the MSRs that are used for CAT. I also run the CAT tools on Linux provided by Intel, which can be downloaded at https://01.org/packet-processing/cache-monitoring-technology-memory-bandwidth-monitoring-cache-allocation-technology. It shows me that I can set up different cache partitions for different COS. Basically, the pre-configured cache setting provided by the CAT tools work perfectly on my machine. :-) ------ OK. That just check the registers are there and tools do not return error. It may still not work, right? :-) Well, I also did some performance evaluation by running a small simple benchmark I wrote. The benchmark task sequentially access a 6MB array; I run the benchmark on core 0 in the following scenarios: Scenario 1): Core 0 is allocated for 8MB cache with CAT, the latency of accessing the 6MB array is around 5.5M cycles; Scenario 2): Core 0 is allocated for 4MB cache with CAT, the latency of accessing the 6MB array is around 16.9M cycles. The slowdown in scenario 2) is 16.9M / 5.5M ~=3x. ------ISSUES------- I tried to run some noisy neighbors on another core to see how good the LLC isolation CAT can provide, but found some *weird result*. I run the benchmark task on core 0 and the noisy neighbor that access 20MB array on core 1; These two cores are configured to have *different* cache areas: core 0 has 8MB cache, core 1 has 4MB cache; These two cores are in two isolated cpuset. No other tasks runs on these two cores. If I run the benchmark alone, the latency is around 5.5M cycles; but if run the benchmark along with the noisy neighbor, the latency *decreases* to 4.9M cycles. I double checked that the Turbo Boost is disabled by checking the MSR value with the following command: rdmsr -pi 0x1a0 -f 38:38 1=disabled 0=enabled it returns 1. I also disabled the cache prefetch in BIOS. Now I'm very confused. How come the latency decreases when a noisy neighbor is running. It seems that the noisy neighbor may help some hardware/software prefetcher to prefetch the data for the benchmark. But right now, I couldn't think out any other prefetchers that may cause this... The benchmark and the noisy neighbor are independent and don't share the array data. If you have any insight or comment, could you please let me know? Thanks and best regards, Meng ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-02 1:51 ` Meng Xu @ 2015-09-02 2:31 ` Chao Peng 2015-09-02 2:55 ` Meng Xu 0 siblings, 1 reply; 16+ messages in thread From: Chao Peng @ 2015-09-02 2:31 UTC (permalink / raw) To: Meng Xu; +Cc: Andrew Cooper, xen-devel, Autee, Priya V On Tue, Sep 01, 2015 at 09:51:57PM -0400, Meng Xu wrote: > Hi Andrew and Chao, > > [Important things go first] It turns out my machine (Intel E5-2618L > v3) does have CAT capability! > Xen gives the false alarm that my machine does not have it. This > should be a bug, IMO. :-) Even some Haswell Servers do support CAT, but it's usually model-specific and the feature is not enumerated in a standard way as listed in Intel SDM. Xen now however only takes care of the standard enumeration so your case is not detected. This could be done by adding cpu model check in Xen code or even use updated firmware, which is I prefered. > > 2015-09-01 10:42 GMT-04:00 Meng Xu <xumengpanda@gmail.com>: > > 2015-09-01 10:30 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: > >> On 01/09/15 15:20, Meng Xu wrote: > >>> 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: > >>>> On 01/09/15 13:55, Meng Xu wrote: > >>>>> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: > >>>>>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: > >>>>>>> I looked into the xen/arch/x86/psr.c and found that the function > >>>>>>> cat_cpu_init() just returned without initializing the variable > >>>>>>> "cat_socket_enable". > >>>>>>> > >>>>>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < > >>>>>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function > >>>>>>> cat_cpu_init(). > > I'm thinking this check could be wrong for Intel E5-2618L v3. It > should work on Chao's machine but not on mine. There should be a > better way to check this probably. :-) > > --- > > I used another way to check the CAT capability, as suggested by Priya > (cc.ed) from Intel. > I did the following steps as Priya suggested: > 1. Download msr-tools utility on your linux distribution to perform > msr read write operations./ if you already have it installed modprobe > msr > 2. rdmsr 0xc91 > which returns 0xfffff > 3. wrmsr -p 1 0xc91 0xf > which does not return anything > 4. wrmsr -p 1 0xc8f 0x100000000 > which does not return anything > 5. rdmsr 0xc91 > which returns 0xf > > This shows that the CPU does have the MSRs that are used for CAT. > > I also run the CAT tools on Linux provided by Intel, which can be > downloaded at https://01.org/packet-processing/cache-monitoring-technology-memory-bandwidth-monitoring-cache-allocation-technology. > It shows me that I can set up different cache partitions for different COS. > Basically, the pre-configured cache setting provided by the CAT tools > work perfectly on my machine. :-) > > ------ > OK. That just check the registers are there and tools do not return > error. It may still not work, right? :-) > Well, I also did some performance evaluation by running a small simple > benchmark I wrote. > > The benchmark task sequentially access a 6MB array; > > I run the benchmark on core 0 in the following scenarios: > > Scenario 1): Core 0 is allocated for 8MB cache with CAT, the latency > of accessing the 6MB array is around 5.5M cycles; > Scenario 2): Core 0 is allocated for 4MB cache with CAT, the latency > of accessing the 6MB array is around 16.9M cycles. > The slowdown in scenario 2) is 16.9M / 5.5M ~=3x. > > ------ISSUES------- > I tried to run some noisy neighbors on another core to see how good > the LLC isolation CAT can provide, but found some *weird result*. > I run the benchmark task on core 0 and the noisy neighbor that access > 20MB array on core 1; > These two cores are configured to have *different* cache areas: core 0 > has 8MB cache, core 1 has 4MB cache; > These two cores are in two isolated cpuset. No other tasks runs on > these two cores. > If I run the benchmark alone, the latency is around 5.5M cycles; > but if run the benchmark along with the noisy neighbor, the latency > *decreases* to 4.9M cycles. > > I double checked that the Turbo Boost is disabled by checking the MSR > value with the following command: > rdmsr -pi 0x1a0 -f 38:38 > 1=disabled > 0=enabled > it returns 1. > I also disabled the cache prefetch in BIOS. > > Now I'm very confused. How come the latency decreases when a noisy > neighbor is running. It seems that the noisy neighbor may help some > hardware/software prefetcher to prefetch the data for the benchmark. > But right now, I couldn't think out any other prefetchers that may > cause this... > The benchmark and the noisy neighbor are independent and don't share > the array data. Did you reboot the machine between your two tests and are the two cores you used in the same socket? Thanks, Chao ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-02 2:31 ` Chao Peng @ 2015-09-02 2:55 ` Meng Xu 2015-09-02 8:34 ` Jan Beulich 0 siblings, 1 reply; 16+ messages in thread From: Meng Xu @ 2015-09-02 2:55 UTC (permalink / raw) To: Chao Peng; +Cc: Andrew Cooper, xen-devel, Autee, Priya V 2015-09-01 22:31 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: > On Tue, Sep 01, 2015 at 09:51:57PM -0400, Meng Xu wrote: >> Hi Andrew and Chao, >> >> [Important things go first] It turns out my machine (Intel E5-2618L >> v3) does have CAT capability! >> Xen gives the false alarm that my machine does not have it. This >> should be a bug, IMO. :-) > > Even some Haswell Servers do support CAT, but it's usually > model-specific and the feature is not enumerated in a standard way as > listed in Intel SDM. Xen now however only takes care of the standard > enumeration so your case is not detected. Ah. I prefer not to upgrade the firmware... Last time when I upgrade the firmware, I screw up the whole system. :-( Is it possible to check if CAT is supported on a machine with another way? Right now, I think you are checking some specific bits returned by CPUID. > > This could be done by adding cpu model check in Xen code or even use > updated firmware, which is I prefered. > >> >> 2015-09-01 10:42 GMT-04:00 Meng Xu <xumengpanda@gmail.com>: >> > 2015-09-01 10:30 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >> >> On 01/09/15 15:20, Meng Xu wrote: >> >>> 2015-09-01 9:04 GMT-04:00 Andrew Cooper <andrew.cooper3@citrix.com>: >> >>>> On 01/09/15 13:55, Meng Xu wrote: >> >>>>> 2015-09-01 1:47 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >> >>>>>> On Mon, Aug 31, 2015 at 04:09:31PM -0400, Meng Xu wrote: >> >>>>>>> I looked into the xen/arch/x86/psr.c and found that the function >> >>>>>>> cat_cpu_init() just returned without initializing the variable >> >>>>>>> "cat_socket_enable". >> >>>>>>> >> >>>>>>> Both !cpu_has(c, X86_FEATURE_CAT) and c->cpuid_level < >> >>>>>>> PSR_CPUID_LEVEL_CAT are evaluated as 1 inside the function >> >>>>>>> cat_cpu_init(). >> >> I'm thinking this check could be wrong for Intel E5-2618L v3. It >> should work on Chao's machine but not on mine. There should be a >> better way to check this probably. :-) >> >> --- >> >> I used another way to check the CAT capability, as suggested by Priya >> (cc.ed) from Intel. >> I did the following steps as Priya suggested: >> 1. Download msr-tools utility on your linux distribution to perform >> msr read write operations./ if you already have it installed modprobe >> msr >> 2. rdmsr 0xc91 >> which returns 0xfffff >> 3. wrmsr -p 1 0xc91 0xf >> which does not return anything >> 4. wrmsr -p 1 0xc8f 0x100000000 >> which does not return anything >> 5. rdmsr 0xc91 >> which returns 0xf >> >> This shows that the CPU does have the MSRs that are used for CAT. >> >> I also run the CAT tools on Linux provided by Intel, which can be >> downloaded at https://01.org/packet-processing/cache-monitoring-technology-memory-bandwidth-monitoring-cache-allocation-technology. >> It shows me that I can set up different cache partitions for different COS. >> Basically, the pre-configured cache setting provided by the CAT tools >> work perfectly on my machine. :-) >> >> ------ >> OK. That just check the registers are there and tools do not return >> error. It may still not work, right? :-) >> Well, I also did some performance evaluation by running a small simple >> benchmark I wrote. >> >> The benchmark task sequentially access a 6MB array; >> >> I run the benchmark on core 0 in the following scenarios: >> >> Scenario 1): Core 0 is allocated for 8MB cache with CAT, the latency >> of accessing the 6MB array is around 5.5M cycles; >> Scenario 2): Core 0 is allocated for 4MB cache with CAT, the latency >> of accessing the 6MB array is around 16.9M cycles. >> The slowdown in scenario 2) is 16.9M / 5.5M ~=3x. >> >> ------ISSUES------- >> I tried to run some noisy neighbors on another core to see how good >> the LLC isolation CAT can provide, but found some *weird result*. >> I run the benchmark task on core 0 and the noisy neighbor that access >> 20MB array on core 1; >> These two cores are configured to have *different* cache areas: core 0 >> has 8MB cache, core 1 has 4MB cache; >> These two cores are in two isolated cpuset. No other tasks runs on >> these two cores. >> If I run the benchmark alone, the latency is around 5.5M cycles; >> but if run the benchmark along with the noisy neighbor, the latency >> *decreases* to 4.9M cycles. >> >> I double checked that the Turbo Boost is disabled by checking the MSR >> value with the following command: >> rdmsr -pi 0x1a0 -f 38:38 >> 1=disabled >> 0=enabled >> it returns 1. >> I also disabled the cache prefetch in BIOS. >> >> Now I'm very confused. How come the latency decreases when a noisy >> neighbor is running. It seems that the noisy neighbor may help some >> hardware/software prefetcher to prefetch the data for the benchmark. >> But right now, I couldn't think out any other prefetchers that may >> cause this... >> The benchmark and the noisy neighbor are independent and don't share >> the array data. > > Did you reboot the machine between your two tests and are the two cores > you used in the same socket? I didn't reboot between these two test. I'm using the same socket 0. Thank you very much! Meng ----------- Meng Xu PhD Student in Computer and Information Science University of Pennsylvania http://www.cis.upenn.edu/~mengxu/ ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: Question about the CAT and CMT in Xen 2015-09-02 2:55 ` Meng Xu @ 2015-09-02 8:34 ` Jan Beulich 0 siblings, 0 replies; 16+ messages in thread From: Jan Beulich @ 2015-09-02 8:34 UTC (permalink / raw) To: xumengpanda, chao.p.peng; +Cc: andrew.cooper3, xen-devel, priya.v.autee >>> Meng Xu <xumengpanda@gmail.com> 09/02/15 4:57 AM >>> >2015-09-01 22:31 GMT-04:00 Chao Peng <chao.p.peng@linux.intel.com>: >> On Tue, Sep 01, 2015 at 09:51:57PM -0400, Meng Xu wrote: >>> Hi Andrew and Chao, >>> >>> [Important things go first] It turns out my machine (Intel E5-2618L >>> v3) does have CAT capability! >>> Xen gives the false alarm that my machine does not have it. This >>> should be a bug, IMO. :-) >> >> Even some Haswell Servers do support CAT, but it's usually >> model-specific and the feature is not enumerated in a standard way as >> listed in Intel SDM. Xen now however only takes care of the standard >> enumeration so your case is not detected. > >Ah. I prefer not to upgrade the firmware... Last time when I upgrade >the firmware, I screw up the whole system. :-( I would very much expect this to be an exception, not the norm. >Is it possible to check if CAT is supported on a machine with another >way? Right now, I think you are checking some specific bits returned >by CPUID. That's the one and only way it should be done in production software. For your own debugging/development purposes you could of course extend that check. Jan ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2015-09-02 8:34 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2015-08-31 17:42 Question about the CAT and CMT in Xen Meng Xu 2015-08-31 18:25 ` Andrew Cooper 2015-08-31 20:09 ` Meng Xu 2015-08-31 20:22 ` Andrew Cooper 2015-09-01 5:47 ` Chao Peng 2015-09-01 12:55 ` Meng Xu 2015-09-01 13:04 ` Andrew Cooper 2015-09-01 14:20 ` Meng Xu 2015-09-01 14:30 ` Andrew Cooper 2015-09-01 14:42 ` Meng Xu 2015-09-02 1:47 ` Chao Peng 2015-09-02 1:53 ` Meng Xu 2015-09-02 1:51 ` Meng Xu 2015-09-02 2:31 ` Chao Peng 2015-09-02 2:55 ` Meng Xu 2015-09-02 8:34 ` Jan Beulich
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.