From: David Hildenbrand <david@redhat.com>
To: Qian Cai <cai@lca.pw>, linux-kernel@vger.kernel.org
Cc: linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
Oscar Salvador <osalvador@suse.de>,
Michal Hocko <mhocko@suse.com>,
Pavel Tatashin <pasha.tatashin@soleen.com>,
Dan Williams <dan.j.williams@intel.com>,
Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH v1] mm/memory_hotplug: Don't take the cpu_hotplug_lock
Date: Fri, 4 Oct 2019 09:42:52 +0200 [thread overview]
Message-ID: <be2cb67b-5432-d147-cfa1-efe3c7e77db6@redhat.com> (raw)
In-Reply-To: <1570052242.5576.266.camel@lca.pw>
On 02.10.19 23:37, Qian Cai wrote:
> On Tue, 2019-09-24 at 16:36 +0200, David Hildenbrand wrote:
>> Since commit 3f906ba23689 ("mm/memory-hotplug: switch locking to a percpu
>> rwsem") we do a cpus_read_lock() in mem_hotplug_begin(). This was
>> introduced to fix a potential deadlock between get_online_mems() and
>> get_online_cpus() - the memory and cpu hotplug lock. The root issue was
>> that build_all_zonelists() -> stop_machine() required the cpu hotplug lock:
>> The reason is that memory hotplug takes the memory hotplug lock and
>> then calls stop_machine() which calls get_online_cpus(). That's the
>> reverse lock order to get_online_cpus(); get_online_mems(); in
>> mm/slub_common.c
>>
>> So memory hotplug never really required any cpu lock itself, only
>> stop_machine() and lru_add_drain_all() required it. Back then,
>> stop_machine_cpuslocked() and lru_add_drain_all_cpuslocked() were used
>> as the cpu hotplug lock was now obtained in the caller.
>>
>> Since commit 11cd8638c37f ("mm, page_alloc: remove stop_machine from build
>> all_zonelists"), the stop_machine_cpuslocked() call is gone.
>> build_all_zonelists() does no longer require the cpu lock and does no
>> longer make use of stop_machine().
>>
>> Since commit 9852a7212324 ("mm: drop hotplug lock from
>> lru_add_drain_all()"), lru_add_drain_all() "Doesn't need any cpu hotplug
>> locking because we do rely on per-cpu kworkers being shut down before our
>> page_alloc_cpu_dead callback is executed on the offlined cpu.". The
>> lru_add_drain_all_cpuslocked() variant was removed.
>>
>> So there is nothing left that requires the cpu hotplug lock. The memory
>> hotplug lock and the device hotplug lock are sufficient.
>
> Actually, powerpc does,
>
> arch_add_memory()
> resize_hpt_for_hotplug()
> pseries_lpar_resize_hpt()
> stop_machine_cpuslocked()
>
Thanks for that observation. This will need some further thought.
Another proof that locking is messed up :)
Maybe we should start decoupling locking of the memory
onlining/offlining path (e.g., get_online_mems()) from the memory
adding/removing path (e.g., later something like get_present_mems()).
Then we can push down the cpu hotplug lock to the PPC path and use
stop_machine() directly.
Time to document properly which lock protects what :)
--
Thanks,
David / dhildenb
prev parent reply other threads:[~2019-10-04 7:42 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-24 14:36 [PATCH v1] mm/memory_hotplug: Don't take the cpu_hotplug_lock David Hildenbrand
2019-09-24 14:48 ` Michal Hocko
2019-09-24 15:01 ` David Hildenbrand
2019-09-24 15:03 ` Qian Cai
2019-09-24 15:11 ` Michal Hocko
2019-09-24 18:54 ` Qian Cai
2019-09-25 7:02 ` David Hildenbrand
2019-09-25 16:01 ` Qian Cai
2019-09-25 17:48 ` Michal Hocko
2019-09-25 18:20 ` Qian Cai
2019-09-25 19:48 ` David Hildenbrand
2019-09-25 20:32 ` Qian Cai
2019-09-26 7:26 ` David Hildenbrand
2019-09-26 7:38 ` Michal Hocko
2019-09-26 7:26 ` Michal Hocko
2019-09-26 11:19 ` Qian Cai
2019-09-26 11:52 ` Michal Hocko
2019-09-26 13:02 ` Qian Cai
2019-09-26 13:14 ` David Hildenbrand
2019-09-25 10:03 ` Michal Hocko
2019-09-24 15:23 ` David Hildenbrand
2019-10-02 21:37 ` Qian Cai
2019-10-04 7:42 ` David Hildenbrand [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=be2cb67b-5432-d147-cfa1-efe3c7e77db6@redhat.com \
--to=david@redhat.com \
--cc=akpm@linux-foundation.org \
--cc=cai@lca.pw \
--cc=dan.j.williams@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=osalvador@suse.de \
--cc=pasha.tatashin@soleen.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).