From: Michal Hocko <mhocko@suse.com>
To: Alexey Makhalov <amakhalov@vmware.com>
Cc: Dennis Zhou <dennis@kernel.org>,
Eric Dumazet <eric.dumazet@gmail.com>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
Andrew Morton <akpm@linux-foundation.org>,
David Hildenbrand <david@redhat.com>,
Oscar Salvador <osalvador@suse.de>, Tejun Heo <tj@kernel.org>,
Christoph Lameter <cl@linux.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"stable@vger.kernel.org" <stable@vger.kernel.org>
Subject: Re: [PATCH v3] mm: fix panic in __alloc_pages
Date: Thu, 18 Nov 2021 09:35:30 +0100 [thread overview]
Message-ID: <YZYQUn10DrKhSE7L@dhcp22.suse.cz> (raw)
In-Reply-To: <5239D699-523C-4F0C-923A-B068E476043E@vmware.com>
On Tue 16-11-21 20:22:49, Alexey Makhalov wrote:
>
>
> > On Nov 16, 2021, at 1:17 AM, Michal Hocko <mhocko@suse.com> wrote:
> >
> > On Tue 16-11-21 01:31:44, Alexey Makhalov wrote:
> > [...]
> >> diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
> >> index 6737b1cbf..bbc1a70d5 100644
> >> --- a/drivers/acpi/acpi_processor.c
> >> +++ b/drivers/acpi/acpi_processor.c
> >> @@ -200,6 +200,10 @@ static int acpi_processor_hotadd_init(struct acpi_processor *pr)
> >> * gets online for the first time.
> >> */
> >> pr_info("CPU%d has been hot-added\n", pr->id);
> >> + {
> >> + int nid = cpu_to_node(pr->id);
> >> + printk("%s:%d cpu %d, node %d, online %d, ndata %p\n", __FUNCTION__, __LINE__, pr->id, nid, node_online(nid), NODE_DATA(nid));
> >> + }
> >> pr->flags.need_hotplug_init = 1;
> >
> > OK, IIUC you are adding a processor which is outside of
> > possible_cpu_mask and that means that the node is not allocated for such
> > a future to be hotplugged cpu and its memory node. init_cpu_to_node
> > would have done that initialization otherwise.
> It is not correct.
>
> possible_cpus is 128 for this VM. Look at SRAT and percpu output for proof.
> [ 0.085524] SRAT: PXM 127 -> APIC 0xfe -> Node 127
> [ 0.118928] setup_percpu: NR_CPUS:128 nr_cpumask_bits:128 nr_cpu_ids:128 nr_node_ids:128
OK, I see. I have missed that when looking at the boot log you have
sent.
> It is impossible to add processor outside of possible_cpu_mask. possible_cpus is absolute maximum
> that system can support. See Documentation/core-api/cpu_hotplug.rst
That was my understanding hence the suspicion you might be doing
something that is not really supported.
> Number of present and onlined CPUs (and nodes) is 4. Other 124 CPUs (and nodes) are not present, but can
> be potentially hot added.
Yes this is a configuration I have already seen. The cpu->node binding
was configured during the boot time though IIRC.
> Number of initialized nodes is 4, as init_cpu_to_node() will skip not yet present nodes,
> see arch/x86/mm/numa.c:798 (numa_cpu_node(CPU #4) == NUMA_NO_NODE)
Isn't this the problem? Why is the cpu->node association missing here?
> 788 void __init init_cpu_to_node(void)
> 789 {
> 790 int cpu;
> 791 u16 *cpu_to_apicid = early_per_cpu_ptr(x86_cpu_to_apicid);
> 792
> 793 BUG_ON(cpu_to_apicid == NULL);
> 794
> 795 for_each_possible_cpu(cpu) {
> 796 int node = numa_cpu_node(cpu);
> 797
> 798 if (node == NUMA_NO_NODE)
> 799 continue;
> 800
>
> After CPU (and node) hot plug:
> - CPU 4 is marker as present, but not yet online
> - New node got ID 4. numa_cpu_node(CPU #4) returns 4
> - node_online(4) == 0 and NODE_DATA(4) == NULL, but it will be accessed inside
> for_each_possible_cpu loop in percpu allocation.
>
> Digging further.
> Even if x86/CPU hot add maintainers decide to clean up memoryless node hot add code to initialize the node on time of
> attaching it (to be aligned with mm node while memory hot add), this percpu fix is still needed as it is used during
> the node onlining, See chicken and egg problem that I described above.
I have to say I do not see the chicken and egg problem. As long as
init_cpu_to_node initializes the memoryless node for the cpu properly
then the pcp allocator doesn't really have to care as the page allocator
falls back to to first populated node in a distance order. So I believe
the whole issue boils down to addressing why init_cpu_to_node doesn't
see a proper cpu->node association.
> Or as 2nd option, numa_cpu_node(4) should return NUMA_NO_NODE until node 4 get fully initialized.
--
Michal Hocko
SUSE Labs
next prev parent reply other threads:[~2021-11-18 8:36 UTC|newest]
Thread overview: 81+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-11-01 20:13 [PATCH] mm: fix panic in __alloc_pages Alexey Makhalov
2021-11-01 20:38 ` Matthew Wilcox
2021-11-02 7:47 ` Michal Hocko
2021-11-02 8:12 ` David Hildenbrand
2021-11-02 8:48 ` Alexey Makhalov
2021-11-02 9:04 ` Michal Hocko
2021-11-02 9:24 ` David Hildenbrand
2021-11-02 10:34 ` Alexey Makhalov
2021-11-02 11:00 ` David Hildenbrand
2021-11-02 11:44 ` Michal Hocko
2021-11-02 12:06 ` David Hildenbrand
2021-11-02 12:27 ` Michal Hocko
2021-11-02 12:39 ` David Hildenbrand
2021-11-02 13:25 ` Michal Hocko
2021-11-02 13:41 ` David Hildenbrand
2021-11-02 14:12 ` Michal Hocko
2021-11-02 14:44 ` David Hildenbrand
2021-11-02 13:52 ` Oscar Salvador
2021-11-02 14:35 ` Michal Hocko
2021-11-08 6:12 ` Alexey Makhalov
2021-11-08 6:36 ` [PATCH v2] " Alexey Makhalov
2021-11-08 8:32 ` David Hildenbrand
2021-11-08 20:23 ` [PATCH v3] " Alexey Makhalov
2021-11-09 2:08 ` Eric Dumazet
2021-11-09 7:03 ` David Hildenbrand
2021-11-09 16:55 ` Eric Dumazet
2021-11-09 17:15 ` Michal Hocko
2021-11-09 19:06 ` Dennis Zhou
2021-11-09 19:54 ` Michal Hocko
2021-11-16 1:31 ` Alexey Makhalov
2021-11-16 9:17 ` Michal Hocko
2021-11-16 20:22 ` Alexey Makhalov
2021-11-18 8:35 ` Michal Hocko [this message]
2021-12-07 10:54 ` Michal Hocko
2021-12-07 11:08 ` David Hildenbrand
2021-12-07 12:13 ` Michal Hocko
2021-12-07 12:28 ` David Hildenbrand
2021-12-07 13:23 ` Michal Hocko
2021-12-07 15:09 ` David Hildenbrand
2021-12-07 15:29 ` Michal Hocko
2021-12-07 15:34 ` David Hildenbrand
2021-12-07 15:56 ` Michal Hocko
2021-12-07 16:09 ` David Hildenbrand
2021-12-07 16:27 ` Michal Hocko
2021-12-07 16:36 ` Michal Hocko
2021-12-07 16:40 ` David Hildenbrand
2021-12-08 8:28 ` Michal Hocko
2021-12-07 17:02 ` Alexey Makhalov
2021-12-07 17:13 ` David Hildenbrand
2021-12-07 17:17 ` Alexey Makhalov
2021-12-07 18:03 ` David Hildenbrand
2021-12-08 8:12 ` Michal Hocko
2021-12-08 8:24 ` David Hildenbrand
2021-12-08 8:34 ` Michal Hocko
2021-12-08 8:38 ` David Hildenbrand
2021-12-08 8:04 ` Michal Hocko
2021-12-08 8:19 ` Alexey Makhalov
2021-12-08 8:30 ` Michal Hocko
2021-12-08 8:54 ` Michal Hocko
2021-12-08 8:57 ` Alexey Makhalov
2021-12-08 9:55 ` Michal Hocko
2021-12-09 2:16 ` Alexey Makhalov
2021-12-09 8:46 ` Michal Hocko
2021-12-09 9:28 ` Alexey Makhalov
2021-12-09 9:56 ` Michal Hocko
2021-12-09 10:23 ` Alexey Makhalov
2021-12-09 13:29 ` Michal Hocko
2021-12-09 19:01 ` Alexey Makhalov
2021-12-10 9:11 ` Michal Hocko
2021-12-17 12:53 ` Michal Hocko
2021-12-21 5:46 ` Alexey Makhalov
2021-12-21 9:46 ` Michal Hocko
2021-12-21 20:23 ` Alexey Makhalov
2021-12-22 11:41 ` Michal Hocko
2021-12-09 10:48 ` Michal Hocko
2021-12-13 15:06 ` Michal Hocko
2021-12-13 15:07 ` David Hildenbrand
2021-12-14 8:38 ` Michal Hocko
2021-11-08 10:37 ` [PATCH v2] " Michal Hocko
2021-11-02 9:40 ` [PATCH] " Alexey Makhalov
2021-11-02 9:40 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YZYQUn10DrKhSE7L@dhcp22.suse.cz \
--to=mhocko@suse.com \
--cc=akpm@linux-foundation.org \
--cc=amakhalov@vmware.com \
--cc=cl@linux.com \
--cc=david@redhat.com \
--cc=dennis@kernel.org \
--cc=eric.dumazet@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=osalvador@suse.de \
--cc=stable@vger.kernel.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).