From: Mark Rutland <mark.rutland@arm.com>
To: Venkata Narendra Kumar Gutta <vnkgutta@codeaurora.org>
Cc: linux-arm-kernel@lists.infradead.org, tsoni@codeaurora.org,
ckadabi@codeaurora.org, rishabhb@codeaurora.org,
linux-kernel@vger.kernel.org, robh@kernel.org,
hoeun.ryu@gmail.com, adobriyan@gmail.com,
zhizhouzhang@asrmicro.com, suzuki.poulose@arm.com,
james.morse@arm.com, will.deacon@arm.com,
catalin.marinas@arm.com,
Abhimanyu Kapur <abhimany@codeaurora.org>
Subject: Re: [PATCH] ARM64: smp: Fix cpu_up() racing with sys_reboot
Date: Fri, 20 Jul 2018 07:23:15 +0100 [thread overview]
Message-ID: <20180720062314.7qh23yfq2o5jciuh@salmiak> (raw)
In-Reply-To: <1532038726-3376-1-git-send-email-vnkgutta@codeaurora.org>
On Thu, Jul 19, 2018 at 03:18:46PM -0700, Venkata Narendra Kumar Gutta wrote:
> Nothing stops a process from hotplugging in a CPU concurrently
> with a sys_reboot() call. In such a situation we could have
> ipi_cpu_stop() mark a cpu as 'offline' and _cpu_up() ignore the
> fact that the CPU is not really offline and call the
> CPU_UP_PREPARE notifier. When this happens stop_machine code will
> complain that the cpu thread already exists and BUG_ON().
>
> CPU0 CPU1
>
> sys_reboot()
> kernel_restart()
> machine_restart()
> machine_shutdown()
> smp_send_stop()
From a quick look arm64's machine_restart() disables IRQs, calls
smp_send_stop(), then calls one of:
* efi_reboot()
* arm_pm_restart()
* do_kernel_restart()
... and I can't see any of these calling machine_shutdown() directly.
How does machine_shutdown() get called? Is that somewhere in the restart_list
associated with do_kernel_restart?
> ... ipi_cpu_stop()
> set_cpu_online(1, false)
> local_irq_disable()
> while(1)
> <PREEMPT>
Given IRQs are disabled in machine_restart(), I don't understand how this
preemption can occur. Where do they get re-enabled?
> cpu_up()
> _cpu_up()
> if (!cpu_online(1))
> __cpu_notify(CPU_UP_PREPARE...)
>
> cpu_stop_cpu_callback()
> BUG_ON(stopper->thread)
>
> This is easily reproducible by hotplugging in and out in a tight
> loop while also rebooting.
Is this reproducible with mainline? e.g. v4.18-rc5?
I've packed away my HW in preparation for an office move, but I might be able
to give this a go in a VM.
> Since the CPU is not really offline and hasn't gone through the
> proper steps to be marked as such, let's mark the CPU as inactive.
> This is just as easily testable as online and avoids any possibility
> of _cpu_up() trying to bring the CPU back online when it never was
> offline to begin with. Based on the similar patchset by for arm
> targets 040c163( "ARM: smp: Fix cpu_up() racing with sys_reboot)"
Is this in mainline?
Looking at v4.18-rc5 I don't see arm's ipi_cpu_stop touching the active mask.
Thanks,
Mark.
>
> Signed-off-by: Abhimanyu Kapur <abhimany@codeaurora.org>
> Signed-off-by: Venkata Narendra Kumar Gutta <vnkgutta@codeaurora.org>
> ---
> arch/arm64/kernel/smp.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index 2faa986..adee4d3 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -790,7 +790,7 @@ void arch_irq_work_raise(void)
> */
> static void ipi_cpu_stop(unsigned int cpu)
> {
> - set_cpu_online(cpu, false);
> + set_cpu_active(cpu, false);
>
> local_daif_mask();
> sdei_mask_local_cpu();
> @@ -925,10 +925,10 @@ void smp_send_stop(void)
>
> /* Wait up to one second for other CPUs to stop */
> timeout = USEC_PER_SEC;
> - while (num_online_cpus() > 1 && timeout--)
> + while (num_active_cpus() > 1 && timeout--)
> udelay(1);
>
> - if (num_online_cpus() > 1)
> + if (num_active_cpus() > 1)
> pr_warning("SMP: failed to stop secondary CPUs %*pbl\n",
> cpumask_pr_args(cpu_online_mask));
>
> --
> The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
> a Linux Foundation Collaborative Project
>
prev parent reply other threads:[~2018-07-20 6:23 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-07-19 22:18 [PATCH] ARM64: smp: Fix cpu_up() racing with sys_reboot Venkata Narendra Kumar Gutta
2018-07-19 22:55 ` Russell King - ARM Linux
2018-07-20 6:23 ` Mark Rutland [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20180720062314.7qh23yfq2o5jciuh@salmiak \
--to=mark.rutland@arm.com \
--cc=abhimany@codeaurora.org \
--cc=adobriyan@gmail.com \
--cc=catalin.marinas@arm.com \
--cc=ckadabi@codeaurora.org \
--cc=hoeun.ryu@gmail.com \
--cc=james.morse@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=rishabhb@codeaurora.org \
--cc=robh@kernel.org \
--cc=suzuki.poulose@arm.com \
--cc=tsoni@codeaurora.org \
--cc=vnkgutta@codeaurora.org \
--cc=will.deacon@arm.com \
--cc=zhizhouzhang@asrmicro.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).