From: Peter Zijlstra <peterz@infradead.org>
To: Alexey Klimov <aklimov@redhat.com>
Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org,
yury.norov@gmail.com, tglx@linutronix.de, jobaker@redhat.com,
audralmitchel@gmail.com, arnd@arndb.de,
gregkh@linuxfoundation.org, rafael@kernel.org, tj@kernel.org,
lizefan@huawei.com, qais.yousef@arm.com, hannes@cmpxchg.org,
klimov.linux@gmail.com
Subject: Re: [RFC][PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu online
Date: Mon, 7 Dec 2020 09:38:27 +0100 [thread overview]
Message-ID: <20201207083827.GD3040@hirez.programming.kicks-ass.net> (raw)
In-Reply-To: <20201203171431.256675-1-aklimov@redhat.com>
On Thu, Dec 03, 2020 at 05:14:31PM +0000, Alexey Klimov wrote:
> When a CPU offlined and onlined via device_offline() and device_online()
> the userspace gets uevent notification. If, after receiving uevent,
> userspace executes sched_setaffinity() on some task trying to move it
> to a recently onlined CPU, then it will fail with -EINVAL. Userspace needs
> to wait around 5..30 ms before sched_setaffinity() will succeed for
> recently onlined CPU after receiving uevent.
Right.
> Unfortunately, the execution time of
> echo 1 > /sys/devices/system/cpu/cpuX/online roughly doubled with this
> change (on my test machine).
Nobody cares, it's hotplug, it's supposed to be slow :-) That is,
we fundamentally shift the work _to_ the hotplug path, so as to keep
everybody else fast.
> The nature of this bug is also described here (with different consequences):
> https://lore.kernel.org/lkml/20200211141554.24181-1-qais.yousef@arm.com/
Yeah, pesky deadlocks.. someone was going to try again.
> kernel/cpu.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 6ff2578ecf17..f39a27a7f24b 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -15,6 +15,7 @@
> #include <linux/sched/smt.h>
> #include <linux/unistd.h>
> #include <linux/cpu.h>
> +#include <linux/cpuset.h>
> #include <linux/oom.h>
> #include <linux/rcupdate.h>
> #include <linux/export.h>
> @@ -1275,6 +1276,8 @@ static int cpu_up(unsigned int cpu, enum cpuhp_state target)
> }
>
> err = _cpu_up(cpu, 0, target);
> + if (!err)
> + cpuset_wait_for_hotplug();
> out:
> cpu_maps_update_done();
> return err;
My only consideration is if doing that flush under cpu_add_remove_lock()
is wise.
next prev parent reply other threads:[~2020-12-07 8:39 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-12-03 17:14 [RFC][PATCH] cpu/hotplug: wait for cpuset_hotplug_work to finish on cpu online Alexey Klimov
2020-12-07 8:38 ` Peter Zijlstra [this message]
2020-12-09 2:40 ` Daniel Jordan
2021-01-15 3:21 ` Daniel Jordan
2021-01-19 16:52 ` Alexey Klimov
2021-01-21 2:09 ` Daniel Jordan
2020-12-09 2:09 ` Daniel Jordan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201207083827.GD3040@hirez.programming.kicks-ass.net \
--to=peterz@infradead.org \
--cc=aklimov@redhat.com \
--cc=arnd@arndb.de \
--cc=audralmitchel@gmail.com \
--cc=cgroups@vger.kernel.org \
--cc=gregkh@linuxfoundation.org \
--cc=hannes@cmpxchg.org \
--cc=jobaker@redhat.com \
--cc=klimov.linux@gmail.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=qais.yousef@arm.com \
--cc=rafael@kernel.org \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
--cc=yury.norov@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).