From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751681AbdJYMJ4 (ORCPT ); Wed, 25 Oct 2017 08:09:56 -0400 Received: from mx1.redhat.com ([209.132.183.28]:39456 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751513AbdJYMJt (ORCPT ); Wed, 25 Oct 2017 08:09:49 -0400 DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 94D604E4D1 Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx09.extmail.prod.ext.phx2.redhat.com; spf=fail smtp.mailfrom=prarit@redhat.com From: Prarit Bhargava To: linux-kernel@vger.kernel.org Cc: Prarit Bhargava , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, Peter Zijlstra , Andi Kleen , Dave Hansen , Piotr Luc , Kan Liang , Borislav Petkov , Stephane Eranian , Arvind Yadav , Andy Lutomirski , Christian Borntraeger , "Kirill A. Shutemov" , Tom Lendacky , Mathias Krause , Tim Chen , Vitaly Kuznetsov Subject: [PATCH 3/3 v4] x86/smpboot: Fix __max_logical_packages estimate Date: Wed, 25 Oct 2017 08:09:40 -0400 Message-Id: <20171025120940.15721-4-prarit@redhat.com> In-Reply-To: <20171025120940.15721-1-prarit@redhat.com> References: <20171025120940.15721-1-prarit@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Wed, 25 Oct 2017 12:09:49 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org A system booted with a small number of cores enabled per package panics because the estimate of __max_logical_packages is too low. This occurs when the total number of active cores across all packages is less than the maximum core count for a single package. ie) On a 4 package system with 20 cores/package where only 4 cores are enabled on each package, the value of __max_logical_packages is calculated as DIV_ROUND_UP(16 / 20) = 1 and not 4. Here's an example of the panic: smpboot: Booting Node 1, Processors #1 OK smpboot: Package 1 of CPU 1 exceeds BIOS package data 1. ------------[ cut here ]------------ kernel BUG at arch/x86/kernel/cpu/common.c:1087! invalid opcode: 0000 [#1] SMP Modules linked in: CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.0-rc2+ #4 Calculate __max_logical_packages after the cpu enumeration has completed. Use the boot cpu's data to extrapolate the number of packages. Signed-off-by: Prarit Bhargava Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: x86@kernel.org Cc: Peter Zijlstra Cc: Andi Kleen Cc: Dave Hansen Cc: Piotr Luc Cc: Kan Liang Cc: Borislav Petkov Cc: Stephane Eranian Cc: Arvind Yadav Cc: Andy Lutomirski Cc: Christian Borntraeger Cc: "Kirill A. Shutemov" Cc: Tom Lendacky Cc: Mathias Krause Cc: Tim Chen Cc: Vitaly Kuznetsov --- arch/x86/kernel/smpboot.c | 57 +++++++++-------------------------------------- 1 file changed, 10 insertions(+), 47 deletions(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index ca615bc94e82..6307e92f54ae 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -307,12 +307,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu) if (new >= 0) goto found; - if (logical_packages >= __max_logical_packages) { - pr_warn("Package %u of CPU %u exceeds BIOS package data %u.\n", - logical_packages, cpu, __max_logical_packages); - return -ENOSPC; - } - new = logical_packages++; /* Allocate and copy a new array */ @@ -335,46 +329,6 @@ int topology_update_package_map(unsigned int pkg, unsigned int cpu) return 0; } -static void __init smp_init_package_map(struct cpuinfo_x86 *c, unsigned int cpu) -{ - unsigned int ncpus; - - /* - * Today neither Intel nor AMD support heterogenous systems. That - * might change in the future.... - * - * While ideally we'd want '* smp_num_siblings' in the below @ncpus - * computation, this won't actually work since some Intel BIOSes - * report inconsistent HT data when they disable HT. - * - * In particular, they reduce the APIC-IDs to only include the cores, - * but leave the CPUID topology to say there are (2) siblings. - * This means we don't know how many threads there will be until - * after the APIC enumeration. - * - * By not including this we'll sometimes over-estimate the number of - * logical packages by the amount of !present siblings, but this is - * still better than MAX_LOCAL_APIC. - * - * We use total_cpus not nr_cpu_ids because nr_cpu_ids can be limited - * on the command line leading to a similar issue as the HT disable - * problem because the hyperthreads are usually enumerated after the - * primary cores. - */ - ncpus = boot_cpu_data.x86_max_cores; - if (!ncpus) { - pr_warn("x86_max_cores == zero !?!?"); - ncpus = 1; - } - - __max_logical_packages = DIV_ROUND_UP(total_cpus, ncpus); - pr_info("Max logical packages: %u\n", __max_logical_packages); - - logical_packages = 0; - - topology_update_package_map(c->phys_proc_id, cpu); -} - void __init smp_store_boot_cpu_info(void) { int id = 0; /* CPU 0 */ @@ -382,7 +336,7 @@ void __init smp_store_boot_cpu_info(void) *c = boot_cpu_data; c->cpu_index = id; - smp_init_package_map(c, id); + topology_update_package_map(c->phys_proc_id, id); } /* @@ -1382,7 +1336,16 @@ void __init native_smp_prepare_boot_cpu(void) void __init native_smp_cpus_done(unsigned int max_cpus) { + int ncpus; + pr_debug("Boot done\n"); + /* + * Today neither Intel nor AMD support heterogenous systems so + * extrapolate the boot cpu's data to all packages. + */ + ncpus = cpu_data(0).booted_cores * smp_num_siblings; + __max_logical_packages = DIV_ROUND_UP(nr_cpu_ids, ncpus); + pr_info("Max logical packages: %u\n", __max_logical_packages); if (x86_has_numa_in_package) set_sched_topology(x86_numa_in_package_topology); -- 2.15.0.rc0.39.g2f0e14e64