[PATCH] x86/amd: Fixup cpu_core_id for family17h downcore configuration

[PATCH] x86/amd: Fixup cpu_core_id for family17h downcore configuration

[Suravee Suthikulpanit]

For family17h, current cpu_core_id is directly taken from the value
CPUID_Fn8000001E_EBX[7:0] (CoreId), which is the physical ID of the
core within a die. However, on system with downcore configuration
(where not all physical cores within a die are available),
this could result in the case where cpu_core_id > (cores_per_node - 1).

Fix up the cpu_core_id by breaking down the bitfields of CoreId,
and calculate relative ID using available topology information.

Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
---
 arch/x86/kernel/cpu/amd.c | 61 ++++++++++++++++++++++++++++++++++-------------
 1 file changed, 45 insertions(+), 16 deletions(-)

diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index bb5abe8..e7de105 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -310,38 +310,67 @@ static void amd_get_topology(struct cpuinfo_x86 *c)
 
 	/* get information required for multi-node processors */
 	if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {
+		u16 l3_nshared = 0;
 		u32 eax, ebx, ecx, edx;
 
+		if (cpuid_edx(0x80000006)) {
+			cpuid_count(0x8000001d, 3, &eax, &ebx, &ecx, &edx);
+			l3_nshared = ((eax >> 14) & 0xfff) + 1;
+		}
+
 		cpuid(0x8000001e, &eax, &ebx, &ecx, &edx);
 
 		node_id  = ecx & 0xff;
 		smp_num_siblings = ((ebx >> 8) & 0xff) + 1;
 
+		/* LLC is default to L3, which generally per-node */
+		if (l3_nshared > 0)
+			per_cpu(cpu_llc_id, cpu) = node_id;
+
 		if (c->x86 == 0x15)
 			c->cu_id = ebx & 0xff;
 
-		if (c->x86 >= 0x17) {
-			c->cpu_core_id = ebx & 0xff;
-
-			if (smp_num_siblings > 1)
-				c->x86_max_cores /= smp_num_siblings;
-		}
-
 		/*
-		 * We may have multiple LLCs if L3 caches exist, so check if we
-		 * have an L3 cache by looking at the L3 cache CPUID leaf.
+		 * In family 17h, the CPUID_Fn8000001E_EBX[7:0] (CoreId)
+		 * is non-contiguous in downcore and non-SMT cases.
+		 * Fixup the cpu_core_id to be contiguous for cores within
+		 * the die.
 		 */
-		if (cpuid_edx(0x80000006)) {
-			if (c->x86 == 0x17) {
+		if (c->x86 >= 0x17) {
+			u32 tmp = c->cpu_core_id = ebx & 0xff;
+			u32 ccx_offset, cpu_offset;
+
+			if (smp_num_siblings == 1) {
 				/*
-				 * LLC is at the core complex level.
-				 * Core complex id is ApicId[3].
+				 * CoreId bit-encoding for SMT-disabled
+				 * [7:4] : die
+				 * [3]   : ccx
+				 * [2:0] : core
 				 */
-				per_cpu(cpu_llc_id, cpu) = c->apicid >> 3;
+				ccx_offset = ((tmp >> 3) & 1) * l3_nshared;
+				cpu_offset = tmp & 7;
 			} else {
-				/* LLC is at the node level. */
-				per_cpu(cpu_llc_id, cpu) = node_id;
+				/*
+				 * CoreId bit-encoding for SMT-enabled
+				 * [7:3] : die
+				 * [2]   : ccx
+				 * [1:0] : core
+				 */
+				c->x86_max_cores /= smp_num_siblings;
+				l3_nshared /= smp_num_siblings;
+				ccx_offset = ((tmp >> 2) & 1) * l3_nshared;
+				cpu_offset = tmp & 3;
 			}
+			c->cpu_core_id = ccx_offset + cpu_offset;
+			pr_debug("Fixup apicid=%#x, CoreId:%#x to cpu_core_id:%#x\n",
+				 c->apicid, tmp, c->cpu_core_id);
+
+			/*
+			 * Family17h L3 cache (LLC) is at Core Complex (CCX).
+			 * There could be multiple CCXs in a node.
+			 * CCX ID is ApicId[3].
+			 */
+			per_cpu(cpu_llc_id, cpu) = c->apicid >> 3;
 		}
 	} else if (cpu_has(c, X86_FEATURE_NODEID_MSR)) {
 		u64 value;
-- 
2.7.4

Re: [PATCH] x86/amd: Fixup cpu_core_id for family17h downcore configuration

[Borislav Petkov]

On Thu, Jul 20, 2017 at 03:29:28AM -0500, Suravee Suthikulpanit wrote:
> For family17h, current cpu_core_id is directly taken from the value
> CPUID_Fn8000001E_EBX[7:0] (CoreId), which is the physical ID of the
> core within a die. However, on system with downcore configuration
> (where not all physical cores within a die are available),
> this could result in the case where cpu_core_id > (cores_per_node - 1).
> 
> Fix up the cpu_core_id by breaking down the bitfields of CoreId,
> and calculate relative ID using available topology information.
> 
> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> ---
>  arch/x86/kernel/cpu/amd.c | 61 ++++++++++++++++++++++++++++++++++-------------
>  1 file changed, 45 insertions(+), 16 deletions(-)
> 
> diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
> index bb5abe8..e7de105 100644
> --- a/arch/x86/kernel/cpu/amd.c
> +++ b/arch/x86/kernel/cpu/amd.c
> @@ -310,38 +310,67 @@ static void amd_get_topology(struct cpuinfo_x86 *c)
>  
>  	/* get information required for multi-node processors */
>  	if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {

Please carve this whole TOPOEXT-specific logic out, into a separate
function, say __get_topoext() or so, for better readability/easier
review.

Thanks.

-- 
Regards/Gruss,
    Boris.

SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg)
-- 

Re: [PATCH] x86/amd: Fixup cpu_core_id for family17h downcore configuration

[Ingo Molnar]

* Borislav Petkov <bp@suse.de> wrote:

> On Thu, Jul 20, 2017 at 03:29:28AM -0500, Suravee Suthikulpanit wrote:
> > For family17h, current cpu_core_id is directly taken from the value
> > CPUID_Fn8000001E_EBX[7:0] (CoreId), which is the physical ID of the
> > core within a die. However, on system with downcore configuration
> > (where not all physical cores within a die are available),
> > this could result in the case where cpu_core_id > (cores_per_node - 1).
> > 
> > Fix up the cpu_core_id by breaking down the bitfields of CoreId,
> > and calculate relative ID using available topology information.
> > 
> > Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> > ---
> >  arch/x86/kernel/cpu/amd.c | 61 ++++++++++++++++++++++++++++++++++-------------
> >  1 file changed, 45 insertions(+), 16 deletions(-)
> > 
> > diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
> > index bb5abe8..e7de105 100644
> > --- a/arch/x86/kernel/cpu/amd.c
> > +++ b/arch/x86/kernel/cpu/amd.c
> > @@ -310,38 +310,67 @@ static void amd_get_topology(struct cpuinfo_x86 *c)
> >  
> >  	/* get information required for multi-node processors */
> >  	if (boot_cpu_has(X86_FEATURE_TOPOEXT)) {
> 
> Please carve this whole TOPOEXT-specific logic out, into a separate
> function, say __get_topoext() or so, for better readability/easier
> review.

Also please do the factoring out in a separate preparatory cleanup patch, so that 
we can see the two changes isolated.

Thanks,

	Ingo