From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S941053AbcIGLoo (ORCPT ); Wed, 7 Sep 2016 07:44:44 -0400 Received: from mx1.redhat.com ([209.132.183.28]:37092 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S941006AbcIGLo1 (ORCPT ); Wed, 7 Sep 2016 07:44:27 -0400 From: Prarit Bhargava To: linux-kernel@vger.kernel.org Cc: Prarit Bhargava , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, Greg Kroah-Hartman , Peter Zijlstra , Len Brown , Borislav Petkov , Andi Kleen , Jiri Olsa , Juergen Gross Subject: [PATCH 2/2] cpu hotplug, add CONFIG_PERMANENT_CPU_TOPOLOGY Date: Wed, 7 Sep 2016 07:44:17 -0400 Message-Id: <1473248657-17359-3-git-send-email-prarit@redhat.com> In-Reply-To: <1473248657-17359-1-git-send-email-prarit@redhat.com> References: <1473248657-17359-1-git-send-email-prarit@redhat.com> X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Wed, 07 Sep 2016 11:44:26 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The information in /sys/devices/system/cpu/cpuX/topology directory is useful for userspace monitoring applications and in-tree utilities like cpupower & turbostat. When down'ing a CPU the /sys/devices/system/cpu/cpuX/topology directory is removed during the CPU_DEAD hotplug callback in the kernel. The problem with this model is that the CPU has not been physically removed and the data in the topology directory is still valid and the cpu's location is now lost to userspace. This patch adds CONFIG_PERMANENT_CPU_TOPOLOGY, and is Y by default for x86, an N for all other arches. When enabled the kernel is modified so that the topology directory is added to the core cpu sysfs files so that the topology directory exists while the CPU is physically present. When disabled, the behavior of the current kernel is maintained (that is, the topology directory is removed on a soft down and added on an soft up). Adding CONFIG_PERMANENT_CPU_TOPOLOGY may require additional architecture so that the cpumask data the CPU's topology is not cleared during a CPU down. Before patch: [root@hp-z620-01 ~]# grep ^ /sys/devices/system/cpu/cpu10/topology/* /sys/devices/system/cpu/cpu10/topology/core_id:3 /sys/devices/system/cpu/cpu10/topology/core_siblings:ffff /sys/devices/system/cpu/cpu10/topology/core_siblings_list:0-15 /sys/devices/system/cpu/cpu10/topology/physical_package_id:0 /sys/devices/system/cpu/cpu10/topology/thread_siblings:0404 /sys/devices/system/cpu/cpu10/topology/thread_siblings_list:2,10 Down a cpu [root@hp-z620-01 ~]# echo 0 > /sys/devices/system/cpu/cpu10/online [root@hp-z620-01 ~]# ls /sys/devices/system/cpu/cpu10/topology ls: cannot access topology: No such file or directory After patch: [root@hp-z620-01 ~]# grep ^ /sys/devices/system/cpu/cpu10/topology/* /sys/devices/system/cpu/cpu10/topology/core_id:3 /sys/devices/system/cpu/cpu10/topology/core_siblings:ffff /sys/devices/system/cpu/cpu10/topology/core_siblings_list:0-15 /sys/devices/system/cpu/cpu10/topology/physical_package_id:0 /sys/devices/system/cpu/cpu10/topology/thread_siblings:0404 /sys/devices/system/cpu/cpu10/topology/thread_siblings_list:2,10 Down a cpu [root@hp-z620-01 ~]# echo 0 > /sys/devices/system/cpu/cpu10/online [root@hp-z620-01 ~]# grep ^ /sys/devices/system/cpu/cpu10/topology/* /sys/devices/system/cpu/cpu10/topology/core_id:3 /sys/devices/system/cpu/cpu10/topology/core_siblings:0000 /sys/devices/system/cpu/cpu10/topology/core_siblings_list: /sys/devices/system/cpu/cpu10/topology/physical_package_id:0 /sys/devices/system/cpu/cpu10/topology/thread_siblings:0000 /sys/devices/system/cpu/cpu10/topology/thread_siblings_list: I did some testing with and without BOOTPARAM_HOTPLUG_CPU0 enabled, and up'd and down'd CPUs in sequence, randomly, by thread group, by socket group and didn't see any issues. Note: core_siblings, and thread_siblings are "numa siblings that are online" and "thread siblings that are online" and are used as such within the kernel. They must be zero'd out when the CPU is offline. Signed-off-by: Prarit Bhargava Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: x86@kernel.org Cc: Greg Kroah-Hartman Cc: Peter Zijlstra Cc: Len Brown Cc: Borislav Petkov Cc: Andi Kleen Cc: Jiri Olsa Cc: Juergen Gross --- arch/x86/kernel/smpboot.c | 3 --- drivers/base/Kconfig | 12 ++++++++++++ drivers/base/cpu.c | 8 ++++++++ 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c index 4296beb8fdd3..ae82f8f45b61 100644 --- a/arch/x86/kernel/smpboot.c +++ b/arch/x86/kernel/smpboot.c @@ -1472,7 +1472,6 @@ static void recompute_smt_state(void) static void remove_siblinginfo(int cpu) { int sibling; - struct cpuinfo_x86 *c = &cpu_data(cpu); for_each_cpu(sibling, topology_core_cpumask(cpu)) { cpumask_clear_cpu(cpu, topology_core_cpumask(sibling)); @@ -1490,8 +1489,6 @@ static void remove_siblinginfo(int cpu) cpumask_clear(cpu_llc_shared_mask(cpu)); cpumask_clear(topology_sibling_cpumask(cpu)); cpumask_clear(topology_core_cpumask(cpu)); - c->phys_proc_id = 0; - c->cpu_core_id = 0; cpumask_clear_cpu(cpu, cpu_sibling_setup_mask); recompute_smt_state(); } diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index 98504ec99c7d..b3935a272c3c 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -324,4 +324,16 @@ config CMA_ALIGNMENT endif +config PERMANENT_CPU_TOPOLOGY + bool "Permanent CPU Topology" + depends on HOTPLUG_CPU + def_bool y if X86_64 + help + This option configures CPU topology to be permanent for the lifetime + of the CPU (until it is physically removed). Selecting Y here + results in the kernel reporting the physical location for offlined + CPUs. + + If unsure, leave the default value as is. + endmenu diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c index e8c8c1155b13..c84cbd1defd5 100644 --- a/drivers/base/cpu.c +++ b/drivers/base/cpu.c @@ -261,6 +261,7 @@ static struct attribute_group topology_attr_group = { .name = "topology" }; +#ifndef CONFIG_PERMANENT_CPU_TOPOLOGY /* Add/Remove cpu_topology interface for CPU device */ static int topology_add_dev(unsigned int cpu) { @@ -317,11 +318,15 @@ out: } device_initcall(topology_sysfs_init); +#endif static const struct attribute_group *common_cpu_attr_groups[] = { #ifdef CONFIG_KEXEC &crash_note_cpu_attr_group, #endif +#ifdef CONFIG_PERMANENT_CPU_TOPOLOGY + &topology_attr_group, +#endif NULL }; @@ -329,6 +334,9 @@ static const struct attribute_group *hotplugable_cpu_attr_groups[] = { #ifdef CONFIG_KEXEC &crash_note_cpu_attr_group, #endif +#ifdef CONFIG_PERMANENT_CPU_TOPOLOGY + &topology_attr_group, +#endif NULL }; -- 1.7.9.3