LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Borislav Petkov <bp@alien8.de>
To: X86 ML <x86@kernel.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Thomas Gleixner <tglx@linutronix.de>
Subject: [PATCH 4/4 -v1.1] x86/Documentation: Start documenting x86 topology
Date: Mon, 28 Mar 2016 11:56:09 +0200
Message-ID: <20160328095609.GD26651@pd.tnic> (raw)
In-Reply-To: <1458917557-8757-4-git-send-email-bp@alien8.de>

From: Borislav Petkov <bp@suse.de>

This should contain important aspects of how we represent the system
topology on x86. If people have questions about it and this file doesn't
answer it, then it must be updated.

Signed-off-by: Borislav Petkov <bp@suse.de>
Cc: Thomas Gleixner <tglx@linutronix.de>
---

Let me add a reference to the generic topology document
Documentation/cputopology.txt too.

 Documentation/x86/topology.txt | 212 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 212 insertions(+)
 create mode 100644 Documentation/x86/topology.txt

diff --git a/Documentation/x86/topology.txt b/Documentation/x86/topology.txt
new file mode 100644
index 000000000000..75021f128848
--- /dev/null
+++ b/Documentation/x86/topology.txt
@@ -0,0 +1,212 @@
+x86 Topology
+============
+
+This documents and clarifies the main aspects of x86 topology modelling
+and representation in the kernel. Update/change when doing changes to
+the respective code.
+
+The architecture-agnostic topology definitions are in
+Documentation/cputopology.txt. This file holds x86-specific
+differences/specialities which must not necessarily apply to the generic
+definitions. Thus, the way to read up on Linux topology on x86 is to
+start with the generic one and look at this one in parallel for the x86
+specifics.
+
+Needless to say, code should use the generic functions - this file is
+*only* here to *document* the inner workings of x86 topology.
+
+Started by Thomas Gleixner <tglx@linutronix.de> and Borislav Petkov <bp@alien8.de>.
+
+The main aim of the topology facilities is to present adequate
+interfaces to code which needs to know/query/use the structure of the
+running system wrt threads, cores, packages, etc.
+
+The kernel does not care about the concept of physical sockets because
+a socket has no relevance to software. It's an electromechanical
+component. In the past a socket always contained a single package
+(see below), but with the advent of Multi Chip Modules (MCM) a socket
+can hold more than one package. So there might be still references to
+sockets in the code, but they are of historical nature and should be
+cleaned up.
+
+The topology of a system is described in the units of:
+
+    - packages
+    - cores
+    - threads
+
+* Package:
+
+  Packages contain a number of cores plus shared resources, e.g. DRAM
+  controller, shared caches etc.
+
+  AMD nomenclature for package is 'Node'.
+
+  Package-related topology information in the kernel:
+
+  - cpuinfo_x86.x86_max_cores:
+
+    The number of cores in a package. This information is retrieved via CPUID.
+
+  - cpuinfo_x86.phys_proc_id:
+
+    The physical ID of the package. This information is retrieved via CPUID
+    and deduced from the APIC IDs of the cores in the package.
+
+  - cpuinfo_x86.logical_id:
+
+    The logical ID of the package. As we do not trust BIOSes to enumerate the
+    packages in a consistent way, we introduced the concept of logical package
+    ID so we can sanely calculate the number of maximum possible packages in
+    the system and have the packages enumerated linearly.
+
+  - topology_max_packages():
+
+    The maximum possible number of packages in the system. Helpful for per
+    package facilities to preallocate per package information.
+
+
+* Cores:
+
+  A core consists of 1 or more threads. It does not matter whether the threads
+  are SMT- or CMT-type threads.
+
+  AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses
+  "core".
+
+  Core-related topology information in the kernel:
+
+  - smp_num_siblings:
+
+    The number of threads in a core. The number of threads in a package can be
+    calculated by:
+
+    	threads_per_package = cpuinfo_x86.x86_max_cores * smp_num_siblings
+
+
+* Threads:
+
+  A thread is a single scheduling unit. It's the equivalent to a logical Linux
+  CPU.
+
+  AMDs nomenclature for CMT threads is "Compute Unit Core". The kernel always
+  uses "thread".
+
+  Thread-related topology information in the kernel:
+
+  - topology_core_cpumask():
+
+    The cpumask contains all online threads in the package to which a thread
+    belongs.
+
+    The number of online threads is also printed in /proc/cpuinfo "siblings."
+
+  - topology_sibling_mask():
+
+    The cpumask contains all online threads in the core to which a thread
+    belongs.
+
+   - topology_logical_package_id():
+
+    The logical package ID to which a thread belongs.
+
+   - topology_physical_package_id():
+
+    The physical package ID to which a thread belongs.
+
+   - topology_core_id();
+
+    The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo
+    "core_id."
+
+
+
+System topology examples
+
+Note:
+
+The alternative Linux CPU enumeration depends on how the BIOS enumerates the
+threads. Many BIOSes enumerate all threads 0 first and then all threads 1.
+That has the "advantage" that the logical Linux CPU numbers of threads 0 stay
+the same whether threads are enabled or not. That's merely an implementation
+detail and has no practical impact.
+
+1) Single Package, Single Core
+
+   [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
+
+2) Single Package, Dual Core
+
+   a) One thread per core
+
+      [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
+	          -> [core 1] -> [thread 0] -> Linux CPU 1
+
+   b) Two threads per core
+
+      [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
+   	         	      -> [thread 1] -> Linux CPU 1
+      	          -> [core 1] -> [thread 0] -> Linux CPU 2
+   	         	      -> [thread 1] -> Linux CPU 3
+
+      Alternative enumeration:
+
+      [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
+   	         	      -> [thread 1] -> Linux CPU 2
+      	          -> [core 1] -> [thread 0] -> Linux CPU 1
+   	         	      -> [thread 1] -> Linux CPU 3
+
+      AMD nomenclature for CMT systems:
+
+      [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0
+   	       		           -> [Compute Unit Core 1] -> Linux CPU 1
+               -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2
+   	       		           -> [Compute Unit Core 1] -> Linux CPU 3
+
+4) Dual Package, Dual Core
+
+   a) One thread per core
+
+      [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
+    	          -> [core 1] -> [thread 0] -> Linux CPU 1
+
+      [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2
+   	          -> [core 1] -> [thread 0] -> Linux CPU 3
+
+   b) Two threads per core
+
+      [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
+   	           	      -> [thread 1] -> Linux CPU 1
+      	          -> [core 1] -> [thread 0] -> Linux CPU 2
+   	         	      -> [thread 1] -> Linux CPU 3
+
+      [package 1] -> [core 0] -> [thread 0] -> Linux CPU 4
+   	         	      -> [thread 1] -> Linux CPU 5
+      	          -> [core 1] -> [thread 0] -> Linux CPU 6
+   	         	      -> [thread 1] -> Linux CPU 7
+
+      Alternative enumeration:
+
+      [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0
+   	         	      -> [thread 1] -> Linux CPU 4
+       	          -> [core 1] -> [thread 0] -> Linux CPU 1
+   	         	      -> [thread 1] -> Linux CPU 5
+
+      [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2
+   	         	      -> [thread 1] -> Linux CPU 6
+      	          -> [core 1] -> [thread 0] -> Linux CPU 3
+   	         	      -> [thread 1] -> Linux CPU 7
+
+      AMD nomenclature for CMT systems:
+
+      [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0
+   	       		           -> [Compute Unit Core 1] -> Linux CPU 1
+               -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2
+   	       		           -> [Compute Unit Core 1] -> Linux CPU 3
+
+      [node 1] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 4
+   	       		           -> [Compute Unit Core 1] -> Linux CPU 5
+               -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 6
+   	       		           -> [Compute Unit Core 1] -> Linux CPU 7
+
+
-- 
2.7.3


-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.

  reply index

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-03-25 14:52 [PATCH 1/4] x86/topology: Fix AMD core count Borislav Petkov
2016-03-25 14:52 ` [PATCH 2/4] perf/amd: Cleanup Fam10h NB event constraints Borislav Petkov
2016-03-25 14:52 ` [PATCH 3/4] x86/cpu: Get rid of compute_unit_id Borislav Petkov
2016-03-29  8:49   ` [tip:x86/urgent] " tip-bot for Borislav Petkov
2016-03-25 14:52 ` [PATCH 4/4] x86/Documentation: Start documenting x86 topology Borislav Petkov
2016-03-28  9:56   ` Borislav Petkov [this message]
2016-03-29  8:50     ` [tip:x86/urgent] " tip-bot for Borislav Petkov

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160328095609.GD26651@pd.tnic \
    --to=bp@alien8.de \
    --cc=linux-kernel@vger.kernel.org \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git