From: Tang Chen <tangchen@cn.fujitsu.com> To: cl@linux.com, tj@kernel.org, jiang.liu@linux.intel.com, mika.j.penttila@gmail.com, mingo@redhat.com, akpm@linux-foundation.org, rjw@rjwysocki.net, hpa@zytor.com, yasu.isimatu@gmail.com, isimatu.yasuaki@jp.fujitsu.com, kamezawa.hiroyu@jp.fujitsu.com, izumi.taku@jp.fujitsu.com, gongzhaogang@inspur.com, len.brown@intel.com Cc: tangchen@cn.fujitsu.com, x86@kernel.org, linux-acpi@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Gu Zheng <guz.fnst@cn.fujitsu.com> Subject: [PATCH v5 RESEND 3/5] x86, acpi, cpu-hotplug: Introduce cpuid_to_apicid[] array to store persistent cpuid <-> apicid mapping. Date: Mon, 25 Jan 2016 14:08:18 +0800 [thread overview] Message-ID: <1453702100-2597-4-git-send-email-tangchen@cn.fujitsu.com> (raw) In-Reply-To: <1453702100-2597-1-git-send-email-tangchen@cn.fujitsu.com> From: Gu Zheng <guz.fnst@cn.fujitsu.com> The whole patch-set aims at making cpuid <-> nodeid mapping persistent. So that, when node online/offline happens, cache based on cpuid <-> nodeid mapping such as wq_numa_possible_cpumask will not cause any problem. It contains 4 steps: 1. Enable apic registeration flow to handle both enabled and disabled cpus. 2. Introduce a new array storing all possible cpuid <-> apicid mapping. 3. Enable _MAT and MADT relative apis to return non-presnet or disabled cpus' apicid. 4. Establish all possible cpuid <-> nodeid mapping. This patch finishes step 2. In this patch, we introduce a new static array named cpuid_to_apicid[], which is large enough to store info for all possible cpus. And then, we modify the cpuid calculation. In generic_processor_info(), it simply finds the next unused cpuid. And it is also why the cpuid <-> nodeid mapping changes with node hotplug. After this patch, we find the next unused cpuid, map it to an apicid, and store the mapping in cpuid_to_apicid[], so that cpuid <-> apicid mapping will be persistent. And finally we will use this array to make cpuid <-> nodeid persistent. cpuid <-> apicid mapping is established at local apic registeration time. But non-present or disabled cpus are ignored. In this patch, we establish all possible cpuid <-> apicid mapping when registering local apic. Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com> Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com> --- arch/x86/include/asm/mpspec.h | 1 + arch/x86/kernel/acpi/boot.c | 6 ++--- arch/x86/kernel/apic/apic.c | 61 ++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 61 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/mpspec.h b/arch/x86/include/asm/mpspec.h index b07233b..db902d8 100644 --- a/arch/x86/include/asm/mpspec.h +++ b/arch/x86/include/asm/mpspec.h @@ -86,6 +86,7 @@ static inline void early_reserve_e820_mpc_new(void) { } #endif int generic_processor_info(int apicid, int version); +int __generic_processor_info(int apicid, int version, bool enabled); #define PHYSID_ARRAY_SIZE BITS_TO_LONGS(MAX_LOCAL_APIC) diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c index e759076..0ce06ee 100644 --- a/arch/x86/kernel/acpi/boot.c +++ b/arch/x86/kernel/acpi/boot.c @@ -174,15 +174,13 @@ static int acpi_register_lapic(int id, u8 enabled) return -EINVAL; } - if (!enabled) { + if (!enabled) ++disabled_cpus; - return -EINVAL; - } if (boot_cpu_physical_apicid != -1U) ver = apic_version[boot_cpu_physical_apicid]; - return generic_processor_info(id, ver); + return __generic_processor_info(id, ver, enabled); } static int __init diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c index 1625778..4822cda 100644 --- a/arch/x86/kernel/apic/apic.c +++ b/arch/x86/kernel/apic/apic.c @@ -1998,7 +1998,53 @@ void disconnect_bsp_APIC(int virt_wire_setup) apic_write(APIC_LVT1, value); } -static int __generic_processor_info(int apicid, int version, bool enabled) +/* + * The number of allocated logical CPU IDs. Since logical CPU IDs are allocated + * contiguously, it equals to current allocated max logical CPU ID plus 1. + * All allocated CPU ID should be in [0, nr_logical_cpuidi), so the maximum of + * nr_logical_cpuids is nr_cpu_ids. + * + * NOTE: Reserve 0 for BSP. + */ +static int nr_logical_cpuids = 1; + +/* + * Used to store mapping between logical CPU IDs and APIC IDs. + */ +static int cpuid_to_apicid[] = { + [0 ... NR_CPUS - 1] = -1, +}; + +/* + * Should use this API to allocate logical CPU IDs to keep nr_logical_cpuids + * and cpuid_to_apicid[] synchronized. + */ +static int allocate_logical_cpuid(int apicid) +{ + int i; + + /* + * cpuid <-> apicid mapping is persistent, so when a cpu is up, + * check if the kernel has allocated a cpuid for it. + */ + for (i = 0; i < nr_logical_cpuids; i++) { + if (cpuid_to_apicid[i] == apicid) + return i; + } + + /* Allocate a new cpuid. */ + if (nr_logical_cpuids >= nr_cpu_ids) { + WARN_ONCE(1, "Only %d processors supported." + "Processor %d/0x%x and the rest are ignored.\n", + nr_cpu_ids - 1, nr_logical_cpuids, apicid); + return -1; + } + + cpuid_to_apicid[nr_logical_cpuids] = apicid; + return nr_logical_cpuids++; +} + +int __generic_processor_info(int apicid, int version, bool enabled) { int cpu, max = nr_cpu_ids; bool boot_cpu_detected = physid_isset(boot_cpu_physical_apicid, @@ -2079,8 +2125,17 @@ static int __generic_processor_info(int apicid, int version, bool enabled) * for BSP. */ cpu = 0; - } else - cpu = cpumask_next_zero(-1, cpu_present_mask); + + /* Logical cpuid 0 is reserved for BSP. */ + cpuid_to_apicid[0] = apicid; + } else { + cpu = allocate_logical_cpuid(apicid); + if (cpu < 0) { + if (enabled) + disabled_cpus++; + return -EINVAL; + } + } /* * Validate version -- 1.9.3 -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
WARNING: multiple messages have this Message-ID (diff)
From: Tang Chen <tangchen@cn.fujitsu.com> To: <cl@linux.com>, <tj@kernel.org>, <jiang.liu@linux.intel.com>, <mika.j.penttila@gmail.com>, <mingo@redhat.com>, <akpm@linux-foundation.org>, <rjw@rjwysocki.net>, <hpa@zytor.com>, <yasu.isimatu@gmail.com>, <isimatu.yasuaki@jp.fujitsu.com>, <kamezawa.hiroyu@jp.fujitsu.com>, <izumi.taku@jp.fujitsu.com>, <gongzhaogang@inspur.com>, <len.brown@intel.com> Cc: <tangchen@cn.fujitsu.com>, <x86@kernel.org>, <linux-acpi@vger.kernel.org>, <linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>, Gu Zheng <guz.fnst@cn.fujitsu.com> Subject: [PATCH v5 RESEND 3/5] x86, acpi, cpu-hotplug: Introduce cpuid_to_apicid[] array to store persistent cpuid <-> apicid mapping. Date: Mon, 25 Jan 2016 14:08:18 +0800 [thread overview] Message-ID: <1453702100-2597-4-git-send-email-tangchen@cn.fujitsu.com> (raw) In-Reply-To: <1453702100-2597-1-git-send-email-tangchen@cn.fujitsu.com> From: Gu Zheng <guz.fnst@cn.fujitsu.com> The whole patch-set aims at making cpuid <-> nodeid mapping persistent. So that, when node online/offline happens, cache based on cpuid <-> nodeid mapping such as wq_numa_possible_cpumask will not cause any problem. It contains 4 steps: 1. Enable apic registeration flow to handle both enabled and disabled cpus. 2. Introduce a new array storing all possible cpuid <-> apicid mapping. 3. Enable _MAT and MADT relative apis to return non-presnet or disabled cpus' apicid. 4. Establish all possible cpuid <-> nodeid mapping. This patch finishes step 2. In this patch, we introduce a new static array named cpuid_to_apicid[], which is large enough to store info for all possible cpus. And then, we modify the cpuid calculation. In generic_processor_info(), it simply finds the next unused cpuid. And it is also why the cpuid <-> nodeid mapping changes with node hotplug. After this patch, we find the next unused cpuid, map it to an apicid, and store the mapping in cpuid_to_apicid[], so that cpuid <-> apicid mapping will be persistent. And finally we will use this array to make cpuid <-> nodeid persistent. cpuid <-> apicid mapping is established at local apic registeration time. But non-present or disabled cpus are ignored. In this patch, we establish all possible cpuid <-> apicid mapping when registering local apic. Signed-off-by: Gu Zheng <guz.fnst@cn.fujitsu.com> Signed-off-by: Tang Chen <tangchen@cn.fujitsu.com> --- arch/x86/include/asm/mpspec.h | 1 + arch/x86/kernel/acpi/boot.c | 6 ++--- arch/x86/kernel/apic/apic.c | 61 ++++++++++++++++++++++++++++++++++++++++--- 3 files changed, 61 insertions(+), 7 deletions(-) diff --git a/arch/x86/include/asm/mpspec.h b/arch/x86/include/asm/mpspec.h index b07233b..db902d8 100644 --- a/arch/x86/include/asm/mpspec.h +++ b/arch/x86/include/asm/mpspec.h @@ -86,6 +86,7 @@ static inline void early_reserve_e820_mpc_new(void) { } #endif int generic_processor_info(int apicid, int version); +int __generic_processor_info(int apicid, int version, bool enabled); #define PHYSID_ARRAY_SIZE BITS_TO_LONGS(MAX_LOCAL_APIC) diff --git a/arch/x86/kernel/acpi/boot.c b/arch/x86/kernel/acpi/boot.c index e759076..0ce06ee 100644 --- a/arch/x86/kernel/acpi/boot.c +++ b/arch/x86/kernel/acpi/boot.c @@ -174,15 +174,13 @@ static int acpi_register_lapic(int id, u8 enabled) return -EINVAL; } - if (!enabled) { + if (!enabled) ++disabled_cpus; - return -EINVAL; - } if (boot_cpu_physical_apicid != -1U) ver = apic_version[boot_cpu_physical_apicid]; - return generic_processor_info(id, ver); + return __generic_processor_info(id, ver, enabled); } static int __init diff --git a/arch/x86/kernel/apic/apic.c b/arch/x86/kernel/apic/apic.c index 1625778..4822cda 100644 --- a/arch/x86/kernel/apic/apic.c +++ b/arch/x86/kernel/apic/apic.c @@ -1998,7 +1998,53 @@ void disconnect_bsp_APIC(int virt_wire_setup) apic_write(APIC_LVT1, value); } -static int __generic_processor_info(int apicid, int version, bool enabled) +/* + * The number of allocated logical CPU IDs. Since logical CPU IDs are allocated + * contiguously, it equals to current allocated max logical CPU ID plus 1. + * All allocated CPU ID should be in [0, nr_logical_cpuidi), so the maximum of + * nr_logical_cpuids is nr_cpu_ids. + * + * NOTE: Reserve 0 for BSP. + */ +static int nr_logical_cpuids = 1; + +/* + * Used to store mapping between logical CPU IDs and APIC IDs. + */ +static int cpuid_to_apicid[] = { + [0 ... NR_CPUS - 1] = -1, +}; + +/* + * Should use this API to allocate logical CPU IDs to keep nr_logical_cpuids + * and cpuid_to_apicid[] synchronized. + */ +static int allocate_logical_cpuid(int apicid) +{ + int i; + + /* + * cpuid <-> apicid mapping is persistent, so when a cpu is up, + * check if the kernel has allocated a cpuid for it. + */ + for (i = 0; i < nr_logical_cpuids; i++) { + if (cpuid_to_apicid[i] == apicid) + return i; + } + + /* Allocate a new cpuid. */ + if (nr_logical_cpuids >= nr_cpu_ids) { + WARN_ONCE(1, "Only %d processors supported." + "Processor %d/0x%x and the rest are ignored.\n", + nr_cpu_ids - 1, nr_logical_cpuids, apicid); + return -1; + } + + cpuid_to_apicid[nr_logical_cpuids] = apicid; + return nr_logical_cpuids++; +} + +int __generic_processor_info(int apicid, int version, bool enabled) { int cpu, max = nr_cpu_ids; bool boot_cpu_detected = physid_isset(boot_cpu_physical_apicid, @@ -2079,8 +2125,17 @@ static int __generic_processor_info(int apicid, int version, bool enabled) * for BSP. */ cpu = 0; - } else - cpu = cpumask_next_zero(-1, cpu_present_mask); + + /* Logical cpuid 0 is reserved for BSP. */ + cpuid_to_apicid[0] = apicid; + } else { + cpu = allocate_logical_cpuid(apicid); + if (cpu < 0) { + if (enabled) + disabled_cpus++; + return -EINVAL; + } + } /* * Validate version -- 1.9.3
next prev parent reply other threads:[~2016-01-25 6:08 UTC|newest] Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top 2016-01-25 6:08 [PATCH v5 RESEND 0/5] Make cpuid <-> nodeid mapping persistent Tang Chen 2016-01-25 6:08 ` Tang Chen 2016-01-25 6:08 ` [PATCH v5 RESEND 1/5] x86, memhp, numa: Online memory-less nodes at boot time Tang Chen 2016-01-25 6:08 ` Tang Chen 2016-01-25 6:08 ` [PATCH v5 RESEND 2/5] x86, acpi, cpu-hotplug: Enable acpi to register all possible cpus " Tang Chen 2016-01-25 6:08 ` Tang Chen 2016-01-25 6:08 ` Tang Chen 2016-01-25 6:08 ` Tang Chen [this message] 2016-01-25 6:08 ` [PATCH v5 RESEND 3/5] x86, acpi, cpu-hotplug: Introduce cpuid_to_apicid[] array to store persistent cpuid <-> apicid mapping Tang Chen 2016-01-25 6:08 ` [PATCH v5 RESEND 4/5] x86, acpi, cpu-hotplug: Enable MADT APIs to return disabled apicid Tang Chen 2016-01-25 6:08 ` Tang Chen 2016-01-25 6:08 ` [PATCH v5 RESEND 5/5] x86, acpi, cpu-hotplug: Set persistent cpuid <-> nodeid mapping when booting Tang Chen 2016-01-25 6:08 ` Tang Chen 2016-01-25 6:12 ` [PATCH v5 RESEND 0/5] Make cpuid <-> nodeid mapping persistent Tang Chen 2016-01-25 6:12 ` Tang Chen 2016-02-03 9:14 ` Zhu Guihua 2016-02-03 9:14 ` Zhu Guihua 2016-02-03 12:02 ` Rafael J. Wysocki 2016-02-03 12:02 ` Rafael J. Wysocki 2016-02-03 16:42 ` Tejun Heo 2016-02-03 16:42 ` Tejun Heo 2016-02-18 2:01 ` Zhu Guihua 2016-02-18 2:01 ` Zhu Guihua 2016-02-18 2:01 ` Zhu Guihua 2016-02-18 2:11 ` Rafael J. Wysocki 2016-02-18 2:11 ` Rafael J. Wysocki
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1453702100-2597-4-git-send-email-tangchen@cn.fujitsu.com \ --to=tangchen@cn.fujitsu.com \ --cc=akpm@linux-foundation.org \ --cc=cl@linux.com \ --cc=gongzhaogang@inspur.com \ --cc=guz.fnst@cn.fujitsu.com \ --cc=hpa@zytor.com \ --cc=isimatu.yasuaki@jp.fujitsu.com \ --cc=izumi.taku@jp.fujitsu.com \ --cc=jiang.liu@linux.intel.com \ --cc=kamezawa.hiroyu@jp.fujitsu.com \ --cc=len.brown@intel.com \ --cc=linux-acpi@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mika.j.penttila@gmail.com \ --cc=mingo@redhat.com \ --cc=rjw@rjwysocki.net \ --cc=tj@kernel.org \ --cc=x86@kernel.org \ --cc=yasu.isimatu@gmail.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.