* Odd performance results @ 2016-07-10 4:26 Paul E. McKenney 2016-07-10 5:17 ` Peter Zijlstra 0 siblings, 1 reply; 15+ messages in thread From: Paul E. McKenney @ 2016-07-10 4:26 UTC (permalink / raw) To: peterz, hpa, tglx, mingo, ak; +Cc: linux-kernel Hello! So I ran a quick benchmark which showed stair-step results. I immediately thought "Ah, this is due to CPU 0 and 1, 2 and 3, 4 and 5, and 6 and 7 being threads in a core." Then I thought "Wait, this is an x86!" Then I dumped out cpu*/topology/thread_siblings_list, getting the following: cpu0/topology/thread_siblings_list: 0-1 cpu1/topology/thread_siblings_list: 0-1 cpu2/topology/thread_siblings_list: 2-3 cpu3/topology/thread_siblings_list: 2-3 cpu4/topology/thread_siblings_list: 4-5 cpu5/topology/thread_siblings_list: 4-5 cpu6/topology/thread_siblings_list: 6-7 cpu7/topology/thread_siblings_list: 6-7 Is this now expected behavior or a fluke of my particular laptop? Here is hoping for expected behavior, as it makes NUMA locality the default for a great many workloads. Enlightenment? Thanx, Paul ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-10 4:26 Odd performance results Paul E. McKenney @ 2016-07-10 5:17 ` Peter Zijlstra 2016-07-10 14:43 ` Paul E. McKenney 0 siblings, 1 reply; 15+ messages in thread From: Peter Zijlstra @ 2016-07-10 5:17 UTC (permalink / raw) To: paulmck, hpa, tglx, mingo, ak; +Cc: linux-kernel On 10 July 2016 06:26:39 CEST, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote: >Hello! > >So I ran a quick benchmark which showed stair-step results. I >immediately >thought "Ah, this is due to CPU 0 and 1, 2 and 3, 4 and 5, and 6 and 7 >being threads in a core." Then I thought "Wait, this is an x86!" >Then I dumped out cpu*/topology/thread_siblings_list, getting the >following: > > cpu0/topology/thread_siblings_list: 0-1 > cpu1/topology/thread_siblings_list: 0-1 > cpu2/topology/thread_siblings_list: 2-3 > cpu3/topology/thread_siblings_list: 2-3 > cpu4/topology/thread_siblings_list: 4-5 > cpu5/topology/thread_siblings_list: 4-5 > cpu6/topology/thread_siblings_list: 6-7 > cpu7/topology/thread_siblings_list: 6-7 I'm guessing this is an AMD bulldozer like machine? -- Sent from my Android device with K-9 Mail. Please excuse my brevity. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-10 5:17 ` Peter Zijlstra @ 2016-07-10 14:43 ` Paul E. McKenney 2016-07-12 14:55 ` Peter Zijlstra 0 siblings, 1 reply; 15+ messages in thread From: Paul E. McKenney @ 2016-07-10 14:43 UTC (permalink / raw) To: Peter Zijlstra; +Cc: hpa, tglx, mingo, ak, linux-kernel On Sun, Jul 10, 2016 at 07:17:19AM +0200, Peter Zijlstra wrote: > > > On 10 July 2016 06:26:39 CEST, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote: > >Hello! > > > >So I ran a quick benchmark which showed stair-step results. I > >immediately > >thought "Ah, this is due to CPU 0 and 1, 2 and 3, 4 and 5, and 6 and 7 > >being threads in a core." Then I thought "Wait, this is an x86!" > >Then I dumped out cpu*/topology/thread_siblings_list, getting the > >following: > > > > cpu0/topology/thread_siblings_list: 0-1 > > cpu1/topology/thread_siblings_list: 0-1 > > cpu2/topology/thread_siblings_list: 2-3 > > cpu3/topology/thread_siblings_list: 2-3 > > cpu4/topology/thread_siblings_list: 4-5 > > cpu5/topology/thread_siblings_list: 4-5 > > cpu6/topology/thread_siblings_list: 6-7 > > cpu7/topology/thread_siblings_list: 6-7 > > > I'm guessing this is an AMD bulldozer like machine? /proc/cpuinfo thinks otherwise: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 1 cpu cores : 4 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 1 cpu cores : 4 apicid : 3 initial apicid : 3 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 4 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 2 cpu cores : 4 apicid : 4 initial apicid : 4 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 5 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 2 cpu cores : 4 apicid : 5 initial apicid : 5 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 6 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 6 initial apicid : 6 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: processor : 7 vendor_id : GenuineIntel cpu family : 6 model : 60 model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz stepping : 3 microcode : 0x1c cpu MHz : 800.000 cache size : 6144 KB physical id : 0 siblings : 8 core id : 3 cpu cores : 4 apicid : 7 initial apicid : 7 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1 sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid bogomips : 4988.59 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management: Thanx, Paul ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-10 14:43 ` Paul E. McKenney @ 2016-07-12 14:55 ` Peter Zijlstra 2016-07-12 15:05 ` Paul E. McKenney 0 siblings, 1 reply; 15+ messages in thread From: Peter Zijlstra @ 2016-07-12 14:55 UTC (permalink / raw) To: Paul E. McKenney; +Cc: hpa, tglx, mingo, ak, linux-kernel On Sun, Jul 10, 2016 at 07:43:27AM -0700, Paul E. McKenney wrote: > On Sun, Jul 10, 2016 at 07:17:19AM +0200, Peter Zijlstra wrote: > > > > > > On 10 July 2016 06:26:39 CEST, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote: > > >Hello! > > > > > >So I ran a quick benchmark which showed stair-step results. I > > >immediately > > >thought "Ah, this is due to CPU 0 and 1, 2 and 3, 4 and 5, and 6 and 7 > > >being threads in a core." Then I thought "Wait, this is an x86!" > > >Then I dumped out cpu*/topology/thread_siblings_list, getting the > > >following: > > > > > > cpu0/topology/thread_siblings_list: 0-1 > > > cpu1/topology/thread_siblings_list: 0-1 > > > cpu2/topology/thread_siblings_list: 2-3 > > > cpu3/topology/thread_siblings_list: 2-3 > > > cpu4/topology/thread_siblings_list: 4-5 > > > cpu5/topology/thread_siblings_list: 4-5 > > > cpu6/topology/thread_siblings_list: 6-7 > > > cpu7/topology/thread_siblings_list: 6-7 > > > > > > I'm guessing this is an AMD bulldozer like machine? > > /proc/cpuinfo thinks otherwise: > > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 60 > model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz Weird, I've never seen an Intel box do that before... hpa, any idea? or is this just one weird BIOS. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-12 14:55 ` Peter Zijlstra @ 2016-07-12 15:05 ` Paul E. McKenney 2016-07-12 17:49 ` H. Peter Anvin 0 siblings, 1 reply; 15+ messages in thread From: Paul E. McKenney @ 2016-07-12 15:05 UTC (permalink / raw) To: Peter Zijlstra; +Cc: hpa, tglx, mingo, ak, linux-kernel On Tue, Jul 12, 2016 at 04:55:51PM +0200, Peter Zijlstra wrote: > On Sun, Jul 10, 2016 at 07:43:27AM -0700, Paul E. McKenney wrote: > > On Sun, Jul 10, 2016 at 07:17:19AM +0200, Peter Zijlstra wrote: > > > > > > > > > On 10 July 2016 06:26:39 CEST, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote: > > > >Hello! > > > > > > > >So I ran a quick benchmark which showed stair-step results. I > > > >immediately > > > >thought "Ah, this is due to CPU 0 and 1, 2 and 3, 4 and 5, and 6 and 7 > > > >being threads in a core." Then I thought "Wait, this is an x86!" > > > >Then I dumped out cpu*/topology/thread_siblings_list, getting the > > > >following: > > > > > > > > cpu0/topology/thread_siblings_list: 0-1 > > > > cpu1/topology/thread_siblings_list: 0-1 > > > > cpu2/topology/thread_siblings_list: 2-3 > > > > cpu3/topology/thread_siblings_list: 2-3 > > > > cpu4/topology/thread_siblings_list: 4-5 > > > > cpu5/topology/thread_siblings_list: 4-5 > > > > cpu6/topology/thread_siblings_list: 6-7 > > > > cpu7/topology/thread_siblings_list: 6-7 > > > > > > > > > I'm guessing this is an AMD bulldozer like machine? > > > > /proc/cpuinfo thinks otherwise: > > > > processor : 0 > > vendor_id : GenuineIntel > > cpu family : 6 > > model : 60 > > model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz > > Weird, I've never seen an Intel box do that before... hpa, any idea? or > is this just one weird BIOS. ;-) It is a Lenovo W541 laptop, for whatever that might be worth. Roughly on year old. Thanx, Paul ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-12 15:05 ` Paul E. McKenney @ 2016-07-12 17:49 ` H. Peter Anvin 2016-07-12 18:26 ` Paul E. McKenney 2016-07-12 18:51 ` Peter Zijlstra 0 siblings, 2 replies; 15+ messages in thread From: H. Peter Anvin @ 2016-07-12 17:49 UTC (permalink / raw) To: paulmck, Peter Zijlstra; +Cc: tglx, mingo, ak, linux-kernel On 07/12/16 08:05, Paul E. McKenney wrote: > On Tue, Jul 12, 2016 at 04:55:51PM +0200, Peter Zijlstra wrote: >> On Sun, Jul 10, 2016 at 07:43:27AM -0700, Paul E. McKenney wrote: >>> On Sun, Jul 10, 2016 at 07:17:19AM +0200, Peter Zijlstra wrote: >>>> >>>> >>>> On 10 July 2016 06:26:39 CEST, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote: >>>>> Hello! >>>>> >>>>> So I ran a quick benchmark which showed stair-step results. I >>>>> immediately >>>>> thought "Ah, this is due to CPU 0 and 1, 2 and 3, 4 and 5, and 6 and 7 >>>>> being threads in a core." Then I thought "Wait, this is an x86!" >>>>> Then I dumped out cpu*/topology/thread_siblings_list, getting the >>>>> following: >>>>> >>>>> cpu0/topology/thread_siblings_list: 0-1 >>>>> cpu1/topology/thread_siblings_list: 0-1 >>>>> cpu2/topology/thread_siblings_list: 2-3 >>>>> cpu3/topology/thread_siblings_list: 2-3 >>>>> cpu4/topology/thread_siblings_list: 4-5 >>>>> cpu5/topology/thread_siblings_list: 4-5 >>>>> cpu6/topology/thread_siblings_list: 6-7 >>>>> cpu7/topology/thread_siblings_list: 6-7 >>>> >>>> >>>> I'm guessing this is an AMD bulldozer like machine? >>> >>> /proc/cpuinfo thinks otherwise: >>> >>> processor : 0 >>> vendor_id : GenuineIntel >>> cpu family : 6 >>> model : 60 >>> model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz >> >> Weird, I've never seen an Intel box do that before... hpa, any idea? or >> is this just one weird BIOS. > > ;-) > > It is a Lenovo W541 laptop, for whatever that might be worth. Roughly > on year old. > Well, the obvious thing here is that CPUs 0-1, 2-3, 4-5, and 6-7 *are* indeed threads in a core... Intel x86 products have supported multithreading since the Pentium 4. So the "wait, this is an x86!" bit is strange to me. The CPU in question (and /proc/cpuinfo should show this) has four cores with a total of eight threads. The "siblings" and "cpu cores" fields in /proc/cpuinfo should show the same thing. So I am utterly confused about what is unexpected here? Also, you mentioned absolutely nothing about what kind of benchmark it was, or what the "stairstepping" results imply, so it doesn't really make it any easier... -hpa ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-12 17:49 ` H. Peter Anvin @ 2016-07-12 18:26 ` Paul E. McKenney 2016-07-12 18:51 ` Peter Zijlstra 1 sibling, 0 replies; 15+ messages in thread From: Paul E. McKenney @ 2016-07-12 18:26 UTC (permalink / raw) To: H. Peter Anvin; +Cc: Peter Zijlstra, tglx, mingo, ak, linux-kernel On Tue, Jul 12, 2016 at 10:49:58AM -0700, H. Peter Anvin wrote: > On 07/12/16 08:05, Paul E. McKenney wrote: > > On Tue, Jul 12, 2016 at 04:55:51PM +0200, Peter Zijlstra wrote: > >> On Sun, Jul 10, 2016 at 07:43:27AM -0700, Paul E. McKenney wrote: > >>> On Sun, Jul 10, 2016 at 07:17:19AM +0200, Peter Zijlstra wrote: > >>>> > >>>> > >>>> On 10 July 2016 06:26:39 CEST, "Paul E. McKenney" <paulmck@linux.vnet.ibm.com> wrote: > >>>>> Hello! > >>>>> > >>>>> So I ran a quick benchmark which showed stair-step results. I > >>>>> immediately > >>>>> thought "Ah, this is due to CPU 0 and 1, 2 and 3, 4 and 5, and 6 and 7 > >>>>> being threads in a core." Then I thought "Wait, this is an x86!" > >>>>> Then I dumped out cpu*/topology/thread_siblings_list, getting the > >>>>> following: > >>>>> > >>>>> cpu0/topology/thread_siblings_list: 0-1 > >>>>> cpu1/topology/thread_siblings_list: 0-1 > >>>>> cpu2/topology/thread_siblings_list: 2-3 > >>>>> cpu3/topology/thread_siblings_list: 2-3 > >>>>> cpu4/topology/thread_siblings_list: 4-5 > >>>>> cpu5/topology/thread_siblings_list: 4-5 > >>>>> cpu6/topology/thread_siblings_list: 6-7 > >>>>> cpu7/topology/thread_siblings_list: 6-7 > >>>> > >>>> > >>>> I'm guessing this is an AMD bulldozer like machine? > >>> > >>> /proc/cpuinfo thinks otherwise: > >>> > >>> processor : 0 > >>> vendor_id : GenuineIntel > >>> cpu family : 6 > >>> model : 60 > >>> model name : Intel(R) Core(TM) i7-4710MQ CPU @ 2.50GHz > >> > >> Weird, I've never seen an Intel box do that before... hpa, any idea? or > >> is this just one weird BIOS. > > > > ;-) > > > > It is a Lenovo W541 laptop, for whatever that might be worth. Roughly > > on year old. > > Well, the obvious thing here is that CPUs 0-1, 2-3, 4-5, and 6-7 *are* > indeed threads in a core... Intel x86 products have supported > multithreading since the Pentium 4. So the "wait, this is an x86!" bit > is strange to me. > > The CPU in question (and /proc/cpuinfo should show this) has four cores > with a total of eight threads. The "siblings" and "cpu cores" fields in > /proc/cpuinfo should show the same thing. So I am utterly confused > about what is unexpected here? My prior experience with Intel x86 systems led me to expect that the hardware-thread pairs would instead be 0 and 4, 1 and 5, 2 and 6, and 3 and 7. This would result in a graph with a two-segment line, having higher slope for the lower-numbered CPUs and a lower slope for the higher-numbered CPUs, and I have in fact seen this behavior on older Intel x86 systems. See for example slides 64-67 of: http://www.rdrop.com/users/paulmck/scalability/paper/Updates.2016.06.05a.TUDresden.pdf But don't get me wrong, I do very much prefer the CPU-numbering approach that my laptop uses, where the hardware threads in a given core have consecutive numbers. > Also, you mentioned absolutely nothing about what kind of benchmark it > was, or what the "stairstepping" results imply, so it doesn't really > make it any easier... The benchmark was a POSIX-threads multithreaded benchmark with each thread repeatedly searching a small linked list, which should fit into the nearest-to-CPU cache. The "stairstepping" results suggest to me that a no-cache-miss pointer-following workload allows a single hardware thread to consume most of a given core's relevant hardware resources, at least on this particular chip. Which is fine -- this sort of thing always has been workload-specific. If you want to see an example plot, take a look at: CodeSamples/defer/perf-rcu-qsbr.eps within: git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/perfbook.git Thanx, Paul ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-12 17:49 ` H. Peter Anvin 2016-07-12 18:26 ` Paul E. McKenney @ 2016-07-12 18:51 ` Peter Zijlstra 2016-07-12 19:10 ` [CRM114spam]: " Dr. David Alan Gilbert 2016-07-13 7:18 ` Ingo Molnar 1 sibling, 2 replies; 15+ messages in thread From: Peter Zijlstra @ 2016-07-12 18:51 UTC (permalink / raw) To: H. Peter Anvin; +Cc: paulmck, tglx, mingo, ak, linux-kernel On Tue, Jul 12, 2016 at 10:49:58AM -0700, H. Peter Anvin wrote: > On 07/12/16 08:05, Paul E. McKenney wrote: > The CPU in question (and /proc/cpuinfo should show this) has four cores > with a total of eight threads. The "siblings" and "cpu cores" fields in > /proc/cpuinfo should show the same thing. So I am utterly confused > about what is unexpected here? Typically threads are enumerated differently on Intel parts. Namely: cpu_id = code_id + nr_cores * smt_id which gives, for a 4 core, 2 thread part: 0-3: core 0-3, smt0 4-7: core 0-3, smt1 My Core i7-2600k for example has: $ cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list 0,4 1,5 2,6 3,7 0,4 1,5 2,6 3,7 The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff like what Paul has. ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [CRM114spam]: Re: Odd performance results 2016-07-12 18:51 ` Peter Zijlstra @ 2016-07-12 19:10 ` Dr. David Alan Gilbert 2016-07-12 19:59 ` Paul E. McKenney 2016-07-13 7:18 ` Ingo Molnar 1 sibling, 1 reply; 15+ messages in thread From: Dr. David Alan Gilbert @ 2016-07-12 19:10 UTC (permalink / raw) To: Peter Zijlstra; +Cc: H. Peter Anvin, paulmck, tglx, mingo, ak, linux-kernel * Peter Zijlstra (peterz@infradead.org) wrote: > On Tue, Jul 12, 2016 at 10:49:58AM -0700, H. Peter Anvin wrote: > > On 07/12/16 08:05, Paul E. McKenney wrote: > > The CPU in question (and /proc/cpuinfo should show this) has four cores > > with a total of eight threads. The "siblings" and "cpu cores" fields in > > /proc/cpuinfo should show the same thing. So I am utterly confused > > about what is unexpected here? > > Typically threads are enumerated differently on Intel parts. Namely: > > cpu_id = code_id + nr_cores * smt_id > > which gives, for a 4 core, 2 thread part: > > 0-3: core 0-3, smt0 > 4-7: core 0-3, smt1 > > My Core i7-2600k for example has: > > $ cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list > 0,4 > 1,5 > 2,6 > 3,7 > 0,4 > 1,5 > 2,6 > 3,7 > > The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something > I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff > like what Paul has. Paul's isn't unique: cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list 0-1 0-1 2-3 2-3 i7-3520M CPU @ 2.90GHz (Dual core with hyperthread, Thinkpad t530, fedora 24) Dave -- -----Open up your eyes, open up your mind, open up your code ------- / Dr. David Alan Gilbert | Running GNU/Linux | Happy \ \ dave @ treblig.org | | In Hex / \ _________________________|_____ http://www.treblig.org |_______/ ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [CRM114spam]: Re: Odd performance results 2016-07-12 19:10 ` [CRM114spam]: " Dr. David Alan Gilbert @ 2016-07-12 19:59 ` Paul E. McKenney 2016-07-13 7:20 ` Ingo Molnar 0 siblings, 1 reply; 15+ messages in thread From: Paul E. McKenney @ 2016-07-12 19:59 UTC (permalink / raw) To: Dr. David Alan Gilbert Cc: Peter Zijlstra, H. Peter Anvin, tglx, mingo, ak, linux-kernel On Tue, Jul 12, 2016 at 08:10:49PM +0100, Dr. David Alan Gilbert wrote: > * Peter Zijlstra (peterz@infradead.org) wrote: > > On Tue, Jul 12, 2016 at 10:49:58AM -0700, H. Peter Anvin wrote: > > > On 07/12/16 08:05, Paul E. McKenney wrote: > > > The CPU in question (and /proc/cpuinfo should show this) has four cores > > > with a total of eight threads. The "siblings" and "cpu cores" fields in > > > /proc/cpuinfo should show the same thing. So I am utterly confused > > > about what is unexpected here? > > > > Typically threads are enumerated differently on Intel parts. Namely: > > > > cpu_id = code_id + nr_cores * smt_id > > > > which gives, for a 4 core, 2 thread part: > > > > 0-3: core 0-3, smt0 > > 4-7: core 0-3, smt1 > > > > My Core i7-2600k for example has: > > > > $ cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list > > 0,4 > > 1,5 > > 2,6 > > 3,7 > > 0,4 > > 1,5 > > 2,6 > > 3,7 > > > > The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something > > I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff > > like what Paul has. > > Paul's isn't unique: > cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list > 0-1 > 0-1 > 2-3 > 2-3 > > i7-3520M CPU @ 2.90GHz (Dual core with hyperthread, Thinkpad t530, fedora 24) Glad that it is not just me! ;-) Thanx, Paul ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [CRM114spam]: Re: Odd performance results 2016-07-12 19:59 ` Paul E. McKenney @ 2016-07-13 7:20 ` Ingo Molnar 0 siblings, 0 replies; 15+ messages in thread From: Ingo Molnar @ 2016-07-13 7:20 UTC (permalink / raw) To: Paul E. McKenney Cc: Dr. David Alan Gilbert, Peter Zijlstra, H. Peter Anvin, tglx, mingo, ak, linux-kernel * Paul E. McKenney <paulmck@linux.vnet.ibm.com> wrote: > > i7-3520M CPU @ 2.90GHz (Dual core with hyperthread, Thinkpad t530, fedora 24) > > Glad that it is not just me! ;-) Got one too: aldebaran:~> grep -i . /sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -t u -k +3 -n /sys/devices/system/cpu/cpu0/topology/thread_siblings_list:0-1 /sys/devices/system/cpu/cpu1/topology/thread_siblings_list:0-1 /sys/devices/system/cpu/cpu2/topology/thread_siblings_list:2-3 /sys/devices/system/cpu/cpu3/topology/thread_siblings_list:2-3 /sys/devices/system/cpu/cpu4/topology/thread_siblings_list:4-5 /sys/devices/system/cpu/cpu5/topology/thread_siblings_list:4-5 /sys/devices/system/cpu/cpu6/topology/thread_siblings_list:6-7 /sys/devices/system/cpu/cpu7/topology/thread_siblings_list:6-7 /sys/devices/system/cpu/cpu8/topology/thread_siblings_list:8-9 /sys/devices/system/cpu/cpu9/topology/thread_siblings_list:8-9 /sys/devices/system/cpu/cpu10/topology/thread_siblings_list:10-11 /sys/devices/system/cpu/cpu11/topology/thread_siblings_list:10-11 /sys/devices/system/cpu/cpu12/topology/thread_siblings_list:12-13 /sys/devices/system/cpu/cpu13/topology/thread_siblings_list:12-13 /sys/devices/system/cpu/cpu14/topology/thread_siblings_list:14-15 /sys/devices/system/cpu/cpu15/topology/thread_siblings_list:14-15 Thanks, Ingo ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-12 18:51 ` Peter Zijlstra 2016-07-12 19:10 ` [CRM114spam]: " Dr. David Alan Gilbert @ 2016-07-13 7:18 ` Ingo Molnar 2016-07-13 12:27 ` Henrique de Moraes Holschuh 2016-07-13 14:06 ` Paul E. McKenney 1 sibling, 2 replies; 15+ messages in thread From: Ingo Molnar @ 2016-07-13 7:18 UTC (permalink / raw) To: Peter Zijlstra; +Cc: H. Peter Anvin, paulmck, tglx, mingo, ak, linux-kernel * Peter Zijlstra <peterz@infradead.org> wrote: > On Tue, Jul 12, 2016 at 10:49:58AM -0700, H. Peter Anvin wrote: > > On 07/12/16 08:05, Paul E. McKenney wrote: > > The CPU in question (and /proc/cpuinfo should show this) has four cores > > with a total of eight threads. The "siblings" and "cpu cores" fields in > > /proc/cpuinfo should show the same thing. So I am utterly confused > > about what is unexpected here? > > Typically threads are enumerated differently on Intel parts. Namely: > > cpu_id = core_id + nr_cores * smt_id Yeah, they are 'interleaved' at the thread/core level - I suppose to 'mix' them on OS schedulers that don't know about SMT. (Fortunately this interleaving is not done across NUMA domains.) > $ cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list Btw., this command will print out the mappings in order even on larger systems and shows the CPU # as well: $ grep -i . /sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -t u -k +3 -n /sys/devices/system/cpu/cpu0/topology/thread_siblings_list:0,60 /sys/devices/system/cpu/cpu1/topology/thread_siblings_list:1,61 /sys/devices/system/cpu/cpu2/topology/thread_siblings_list:2,62 /sys/devices/system/cpu/cpu3/topology/thread_siblings_list:3,63 /sys/devices/system/cpu/cpu4/topology/thread_siblings_list:4,64 /sys/devices/system/cpu/cpu5/topology/thread_siblings_list:5,65 /sys/devices/system/cpu/cpu6/topology/thread_siblings_list:6,66 /sys/devices/system/cpu/cpu7/topology/thread_siblings_list:7,67 /sys/devices/system/cpu/cpu8/topology/thread_siblings_list:8,68 /sys/devices/system/cpu/cpu9/topology/thread_siblings_list:9,69 /sys/devices/system/cpu/cpu10/topology/thread_siblings_list:10,70 /sys/devices/system/cpu/cpu11/topology/thread_siblings_list:11,71 ... /sys/devices/system/cpu/cpu116/topology/thread_siblings_list:56,116 /sys/devices/system/cpu/cpu117/topology/thread_siblings_list:57,117 /sys/devices/system/cpu/cpu118/topology/thread_siblings_list:58,118 /sys/devices/system/cpu/cpu119/topology/thread_siblings_list:59,119 > The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something > I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff > like what Paul has. That's more the natural 'direct' mapping from CPU internal topology to CPU id: what's close to each other physically is close to each other in the CPU id space as well. Thanks, Ingo ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-13 7:18 ` Ingo Molnar @ 2016-07-13 12:27 ` Henrique de Moraes Holschuh 2016-07-13 12:33 ` Peter Zijlstra 2016-07-13 14:06 ` Paul E. McKenney 1 sibling, 1 reply; 15+ messages in thread From: Henrique de Moraes Holschuh @ 2016-07-13 12:27 UTC (permalink / raw) To: Ingo Molnar Cc: Peter Zijlstra, H. Peter Anvin, paulmck, tglx, mingo, ak, linux-kernel On Wed, 13 Jul 2016, Ingo Molnar wrote: > > The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something > > I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff > > like what Paul has. > > That's more the natural 'direct' mapping from CPU internal topology to CPU id: > what's close to each other physically is close to each other in the CPU id space > as well. But does it correctly reflect the hardware? That seems to be the real question... -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-13 12:27 ` Henrique de Moraes Holschuh @ 2016-07-13 12:33 ` Peter Zijlstra 0 siblings, 0 replies; 15+ messages in thread From: Peter Zijlstra @ 2016-07-13 12:33 UTC (permalink / raw) To: Henrique de Moraes Holschuh Cc: Ingo Molnar, H. Peter Anvin, paulmck, tglx, mingo, ak, linux-kernel On Wed, Jul 13, 2016 at 09:27:48AM -0300, Henrique de Moraes Holschuh wrote: > On Wed, 13 Jul 2016, Ingo Molnar wrote: > > > The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something > > > I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff > > > like what Paul has. > > > > That's more the natural 'direct' mapping from CPU internal topology to CPU id: > > what's close to each other physically is close to each other in the CPU id space > > as well. > > But does it correctly reflect the hardware? That seems to be the real > question... Its just enumeration afaict. But its weird that this changed. Some BIOS team somewhere changed things and I want to know why (and how widespread this is). ^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: Odd performance results 2016-07-13 7:18 ` Ingo Molnar 2016-07-13 12:27 ` Henrique de Moraes Holschuh @ 2016-07-13 14:06 ` Paul E. McKenney 1 sibling, 0 replies; 15+ messages in thread From: Paul E. McKenney @ 2016-07-13 14:06 UTC (permalink / raw) To: Ingo Molnar; +Cc: Peter Zijlstra, H. Peter Anvin, tglx, mingo, ak, linux-kernel On Wed, Jul 13, 2016 at 09:18:17AM +0200, Ingo Molnar wrote: > > * Peter Zijlstra <peterz@infradead.org> wrote: > > > On Tue, Jul 12, 2016 at 10:49:58AM -0700, H. Peter Anvin wrote: > > > On 07/12/16 08:05, Paul E. McKenney wrote: > > > The CPU in question (and /proc/cpuinfo should show this) has four cores > > > with a total of eight threads. The "siblings" and "cpu cores" fields in > > > /proc/cpuinfo should show the same thing. So I am utterly confused > > > about what is unexpected here? > > > > Typically threads are enumerated differently on Intel parts. Namely: > > > > cpu_id = core_id + nr_cores * smt_id > > Yeah, they are 'interleaved' at the thread/core level - I suppose to 'mix' them on > OS schedulers that don't know about SMT. > > (Fortunately this interleaving is not done across NUMA domains.) > > > $ cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list > > Btw., this command will print out the mappings in order even on larger systems and > shows the CPU # as well: > > $ grep -i . /sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -t u -k +3 -n > > /sys/devices/system/cpu/cpu0/topology/thread_siblings_list:0,60 > /sys/devices/system/cpu/cpu1/topology/thread_siblings_list:1,61 > /sys/devices/system/cpu/cpu2/topology/thread_siblings_list:2,62 > /sys/devices/system/cpu/cpu3/topology/thread_siblings_list:3,63 > /sys/devices/system/cpu/cpu4/topology/thread_siblings_list:4,64 > /sys/devices/system/cpu/cpu5/topology/thread_siblings_list:5,65 > /sys/devices/system/cpu/cpu6/topology/thread_siblings_list:6,66 > /sys/devices/system/cpu/cpu7/topology/thread_siblings_list:7,67 > /sys/devices/system/cpu/cpu8/topology/thread_siblings_list:8,68 > /sys/devices/system/cpu/cpu9/topology/thread_siblings_list:9,69 > /sys/devices/system/cpu/cpu10/topology/thread_siblings_list:10,70 > /sys/devices/system/cpu/cpu11/topology/thread_siblings_list:11,71 > ... > /sys/devices/system/cpu/cpu116/topology/thread_siblings_list:56,116 > /sys/devices/system/cpu/cpu117/topology/thread_siblings_list:57,117 > /sys/devices/system/cpu/cpu118/topology/thread_siblings_list:58,118 > /sys/devices/system/cpu/cpu119/topology/thread_siblings_list:59,119 Here is what that gets me on the x86 test system I usually use: /sys/devices/system/cpu/cpu0/topology/thread_siblings_list:0,32 /sys/devices/system/cpu/cpu1/topology/thread_siblings_list:1,33 /sys/devices/system/cpu/cpu2/topology/thread_siblings_list:2,34 /sys/devices/system/cpu/cpu3/topology/thread_siblings_list:3,35 /sys/devices/system/cpu/cpu4/topology/thread_siblings_list:4,36 /sys/devices/system/cpu/cpu5/topology/thread_siblings_list:5,37 /sys/devices/system/cpu/cpu6/topology/thread_siblings_list:6,38 /sys/devices/system/cpu/cpu7/topology/thread_siblings_list:7,39 /sys/devices/system/cpu/cpu8/topology/thread_siblings_list:8,40 /sys/devices/system/cpu/cpu9/topology/thread_siblings_list:9,41 /sys/devices/system/cpu/cpu10/topology/thread_siblings_list:10,42 /sys/devices/system/cpu/cpu11/topology/thread_siblings_list:11,43 [ . . . ] /sys/devices/system/cpu/cpu56/topology/thread_siblings_list:24,56 /sys/devices/system/cpu/cpu57/topology/thread_siblings_list:25,57 /sys/devices/system/cpu/cpu58/topology/thread_siblings_list:26,58 /sys/devices/system/cpu/cpu59/topology/thread_siblings_list:27,59 /sys/devices/system/cpu/cpu60/topology/thread_siblings_list:28,60 /sys/devices/system/cpu/cpu61/topology/thread_siblings_list:29,61 /sys/devices/system/cpu/cpu62/topology/thread_siblings_list:30,62 /sys/devices/system/cpu/cpu63/topology/thread_siblings_list:31,63 On my laptop: /sys/devices/system/cpu/cpu0/topology/thread_siblings_list:0-1 /sys/devices/system/cpu/cpu1/topology/thread_siblings_list:0-1 /sys/devices/system/cpu/cpu2/topology/thread_siblings_list:2-3 /sys/devices/system/cpu/cpu3/topology/thread_siblings_list:2-3 /sys/devices/system/cpu/cpu4/topology/thread_siblings_list:4-5 /sys/devices/system/cpu/cpu5/topology/thread_siblings_list:4-5 /sys/devices/system/cpu/cpu6/topology/thread_siblings_list:6-7 /sys/devices/system/cpu/cpu7/topology/thread_siblings_list:6-7 > > The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something > > I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff > > like what Paul has. > > That's more the natural 'direct' mapping from CPU internal topology to CPU id: > what's close to each other physically is close to each other in the CPU id space > as well. Agreed! Thanx, Paul ^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2016-07-13 14:06 UTC | newest] Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-07-10 4:26 Odd performance results Paul E. McKenney 2016-07-10 5:17 ` Peter Zijlstra 2016-07-10 14:43 ` Paul E. McKenney 2016-07-12 14:55 ` Peter Zijlstra 2016-07-12 15:05 ` Paul E. McKenney 2016-07-12 17:49 ` H. Peter Anvin 2016-07-12 18:26 ` Paul E. McKenney 2016-07-12 18:51 ` Peter Zijlstra 2016-07-12 19:10 ` [CRM114spam]: " Dr. David Alan Gilbert 2016-07-12 19:59 ` Paul E. McKenney 2016-07-13 7:20 ` Ingo Molnar 2016-07-13 7:18 ` Ingo Molnar 2016-07-13 12:27 ` Henrique de Moraes Holschuh 2016-07-13 12:33 ` Peter Zijlstra 2016-07-13 14:06 ` Paul E. McKenney
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).