From: Vitaly Kuznetsov <vkuznets@redhat.com>
To: Hsin-Yi Wang <hsinyi@chromium.org>
Cc: Josh Poimboeuf <jpoimboe@redhat.com>,
Ingo Molnar <mingo@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
Jiri Kosina <jkosina@suse.cz>,
Pavankumar Kondeti <pkondeti@codeaurora.org>,
Zhenzhong Duan <zhenzhong.duan@oracle.com>,
Aaro Koskinen <aaro.koskinen@nokia.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Guenter Roeck <groeck@chromium.org>,
Stephen Boyd <swboyd@chromium.org>,
linux-kernel@vger.kernel.org,
Thomas Gleixner <tglx@linutronix.de>
Subject: Re: [PATCH RFC v3 1/3] reboot: support hotplug CPUs before reboot
Date: Mon, 13 Jan 2020 13:46:20 +0100 [thread overview]
Message-ID: <87r203plr7.fsf@vitty.brq.redhat.com> (raw)
In-Reply-To: <20200113120157.85798-2-hsinyi@chromium.org>
Hsin-Yi Wang <hsinyi@chromium.org> writes:
> Currently system reboots uses architecture specific codes (smp_send_stop)
> to offline non reboot CPUs. Most architecture's implementation is looping
> through all non reboot online CPUs and call ipi function to each of them. Some
> architecture like arm64, arm, and x86... would set offline masks to cpu without
> really offline them. This causes some race condition and kernel warning comes
> out sometimes when system reboots.
>
> This patch adds a config REBOOT_HOTPLUG_CPU, which would hotplug cpus in
> migrate_to_reboot_cpu(). If non reboot cpus are all offlined here, the loop for
> checking online cpus would be an empty loop. If architecture don't enable this
> config, or some cpus somehow fails to offline, it would fallback to ipi
> function.
>
> Signed-off-by: Hsin-Yi Wang <hsinyi@chromium.org>
> ---
> arch/Kconfig | 6 ++++++
> include/linux/cpu.h | 3 +++
> kernel/cpu.c | 19 +++++++++++++++++++
> kernel/reboot.c | 8 ++++++++
> 4 files changed, 36 insertions(+)
>
> diff --git a/arch/Kconfig b/arch/Kconfig
> index 48b5e103bdb0..a043b9be1499 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -52,6 +52,12 @@ config OPROFILE_EVENT_MULTIPLEX
>
> If unsure, say N.
>
> +config REBOOT_HOTPLUG_CPU
> + bool "Support for hotplug CPUs before reboot"
> + depends on HOTPLUG_CPU
> + help
> + Say Y to do a full hotplug on secondary CPUs before reboot.
I'm not sure this should be a configurable option, e.g. in case this is
a good approach in general, why not just use CONFIG_HOTPLUG_CPU in the
code?
> +
> config HAVE_OPROFILE
> bool
>
> diff --git a/include/linux/cpu.h b/include/linux/cpu.h
> index 1ca2baf817ed..3bf5ab289954 100644
> --- a/include/linux/cpu.h
> +++ b/include/linux/cpu.h
> @@ -118,6 +118,9 @@ extern void cpu_hotplug_disable(void);
> extern void cpu_hotplug_enable(void);
> void clear_tasks_mm_cpumask(int cpu);
> int cpu_down(unsigned int cpu);
> +#if IS_ENABLED(CONFIG_REBOOT_HOTPLUG_CPU)
> +extern void offline_secondary_cpus(int primary);
> +#endif
>
> #else /* CONFIG_HOTPLUG_CPU */
>
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 9c706af713fb..52afc47dd56a 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -1057,6 +1057,25 @@ int cpu_down(unsigned int cpu)
> }
> EXPORT_SYMBOL(cpu_down);
>
> +#if IS_ENABLED(CONFIG_REBOOT_HOTPLUG_CPU)
> +void offline_secondary_cpus(int primary)
> +{
> + int i, err;
> +
> + cpu_maps_update_begin();
> +
> + for_each_online_cpu(i) {
> + if (i == primary)
> + continue;
> + err = _cpu_down(i, 0, CPUHP_OFFLINE);
> + if (err)
> + pr_warn("Failed to offline cpu %d\n", i);
> + }
> + cpu_hotplug_disabled++;
> +
> + cpu_maps_update_done();
> +}
> +#endif
This looks like a simplified version of freeze_secondary_cpus(), can
they be merged?
> #else
> #define takedown_cpu NULL
> #endif /*CONFIG_HOTPLUG_CPU*/
> diff --git a/kernel/reboot.c b/kernel/reboot.c
> index c4d472b7f1b4..fda84794ce46 100644
> --- a/kernel/reboot.c
> +++ b/kernel/reboot.c
> @@ -7,6 +7,7 @@
>
> #define pr_fmt(fmt) "reboot: " fmt
>
> +#include <linux/cpu.h>
> #include <linux/ctype.h>
> #include <linux/export.h>
> #include <linux/kexec.h>
> @@ -220,7 +221,9 @@ void migrate_to_reboot_cpu(void)
> /* The boot cpu is always logical cpu 0 */
> int cpu = reboot_cpu;
>
> +#if !IS_ENABLED(CONFIG_REBOOT_HOTPLUG_CPU)
> cpu_hotplug_disable();
> +#endif
>
> /* Make certain the cpu I'm about to reboot on is online */
> if (!cpu_online(cpu))
> @@ -231,6 +234,11 @@ void migrate_to_reboot_cpu(void)
>
> /* Make certain I only run on the appropriate processor */
> set_cpus_allowed_ptr(current, cpumask_of(cpu));
> +
> + /* Hotplug other cpus if possible */
> +#if IS_ENABLED(CONFIG_REBOOT_HOTPLUG_CPU)
> + offline_secondary_cpus(cpu);
> +#endif
> }
>
> /**
--
Vitaly
next prev parent reply other threads:[~2020-01-13 12:46 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-01-13 12:01 [PATCH RFC v3 0/3] support hotplug CPUs before reboot Hsin-Yi Wang
2020-01-13 12:01 ` [PATCH RFC v3 1/3] reboot: " Hsin-Yi Wang
2020-01-13 12:46 ` Vitaly Kuznetsov [this message]
2020-01-13 15:12 ` Hsin-Yi Wang
2020-01-13 15:57 ` Vitaly Kuznetsov
2020-01-13 17:00 ` Hsin-Yi Wang
2020-01-13 12:01 ` [PATCH RFC v3 2/3] arm64: defconfig: enable REBOOT_HOTPLUG_CPU Hsin-Yi Wang
2020-01-13 12:01 ` [PATCH RFC v3 3/3] x86: " Hsin-Yi Wang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87r203plr7.fsf@vitty.brq.redhat.com \
--to=vkuznets@redhat.com \
--cc=aaro.koskinen@nokia.com \
--cc=gregkh@linuxfoundation.org \
--cc=groeck@chromium.org \
--cc=hsinyi@chromium.org \
--cc=jkosina@suse.cz \
--cc=jpoimboe@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=pkondeti@codeaurora.org \
--cc=swboyd@chromium.org \
--cc=tglx@linutronix.de \
--cc=zhenzhong.duan@oracle.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).