From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752182AbcGMHSc (ORCPT ); Wed, 13 Jul 2016 03:18:32 -0400 Received: from mail-wm0-f50.google.com ([74.125.82.50]:37773 "EHLO mail-wm0-f50.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751532AbcGMHS0 (ORCPT ); Wed, 13 Jul 2016 03:18:26 -0400 Date: Wed, 13 Jul 2016 09:18:17 +0200 From: Ingo Molnar To: Peter Zijlstra Cc: "H. Peter Anvin" , paulmck@linux.vnet.ibm.com, tglx@linutronix.de, mingo@elte.hu, ak@linux.intel.com, linux-kernel@vger.kernel.org Subject: Re: Odd performance results Message-ID: <20160713071817.GC13006@gmail.com> References: <20160710042639.GA4068@linux.vnet.ibm.com> <7DF218CD-22F6-4E46-A628-2138AEA3A161@infradead.org> <20160710144327.GX4650@linux.vnet.ibm.com> <20160712145551.GU30909@twins.programming.kicks-ass.net> <20160712150529.GN7094@linux.vnet.ibm.com> <27d2c710-479d-77a9-f2c6-875e9c2bc40f@zytor.com> <20160712185120.GX30909@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160712185120.GX30909@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org * Peter Zijlstra wrote: > On Tue, Jul 12, 2016 at 10:49:58AM -0700, H. Peter Anvin wrote: > > On 07/12/16 08:05, Paul E. McKenney wrote: > > The CPU in question (and /proc/cpuinfo should show this) has four cores > > with a total of eight threads. The "siblings" and "cpu cores" fields in > > /proc/cpuinfo should show the same thing. So I am utterly confused > > about what is unexpected here? > > Typically threads are enumerated differently on Intel parts. Namely: > > cpu_id = core_id + nr_cores * smt_id Yeah, they are 'interleaved' at the thread/core level - I suppose to 'mix' them on OS schedulers that don't know about SMT. (Fortunately this interleaving is not done across NUMA domains.) > $ cat /sys/devices/system/cpu/cpu*/topology/thread_siblings_list Btw., this command will print out the mappings in order even on larger systems and shows the CPU # as well: $ grep -i . /sys/devices/system/cpu/cpu*/topology/thread_siblings_list | sort -t u -k +3 -n /sys/devices/system/cpu/cpu0/topology/thread_siblings_list:0,60 /sys/devices/system/cpu/cpu1/topology/thread_siblings_list:1,61 /sys/devices/system/cpu/cpu2/topology/thread_siblings_list:2,62 /sys/devices/system/cpu/cpu3/topology/thread_siblings_list:3,63 /sys/devices/system/cpu/cpu4/topology/thread_siblings_list:4,64 /sys/devices/system/cpu/cpu5/topology/thread_siblings_list:5,65 /sys/devices/system/cpu/cpu6/topology/thread_siblings_list:6,66 /sys/devices/system/cpu/cpu7/topology/thread_siblings_list:7,67 /sys/devices/system/cpu/cpu8/topology/thread_siblings_list:8,68 /sys/devices/system/cpu/cpu9/topology/thread_siblings_list:9,69 /sys/devices/system/cpu/cpu10/topology/thread_siblings_list:10,70 /sys/devices/system/cpu/cpu11/topology/thread_siblings_list:11,71 ... /sys/devices/system/cpu/cpu116/topology/thread_siblings_list:56,116 /sys/devices/system/cpu/cpu117/topology/thread_siblings_list:57,117 /sys/devices/system/cpu/cpu118/topology/thread_siblings_list:58,118 /sys/devices/system/cpu/cpu119/topology/thread_siblings_list:59,119 > The ordering Paul has, namely 0,1 for core0,smt{0,1} is not something > I've ever seen on an Intel part. AMD otoh does enumerate their CMT stuff > like what Paul has. That's more the natural 'direct' mapping from CPU internal topology to CPU id: what's close to each other physically is close to each other in the CPU id space as well. Thanks, Ingo