From: Jon Hunter <jonathanh@nvidia.com>
To: Richard Cochran <rcochran@linutronix.de>, <linux-kernel@vger.kernel.org>
Cc: <rt@linutronix.de>, John Stultz <john.stultz@linaro.org>,
Linus Torvalds <torvalds@linux-foundation.org>,
Oleg Nesterov <oleg@redhat.com>,
"Peter Zijlstra" <peterz@infradead.org>,
Rasmus Villemoes <linux@rasmusvillemoes.dk>,
Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
Thomas Gleixner <tglx@linutronix.de>,
"linux-tegra@vger.kernel.org" <linux-tegra@vger.kernel.org>
Subject: Re: [PATCH] timers/core: Correct callback order during CPU hot plug.
Date: Wed, 27 Jul 2016 10:50:42 +0100 [thread overview]
Message-ID: <25aca1c4-5bc0-838c-ac69-45ec97031da2@nvidia.com> (raw)
In-Reply-To: <1469610498-25914-1-git-send-email-rcochran@linutronix.de>
Hi Richard,
On 27/07/16 10:08, Richard Cochran wrote:
> On the tear-down path, the dead CPU callback for the timers was
> misplaced within the 'cpuhp_state' enumeration. There is a hidden
> dependency between the timers and block multiqueue. The timers
> callback must happen before the block multiqueue callback otherwise a
> RCU stall occurs.
>
> This patch fixes the order within the hot plug state machine.
>
> Signed-off-by: Richard Cochran <rcochran@linutronix.de>
> Fixes: 24f73b99716a ("timers/core: Convert to hotplug state machine")
>
> Cc: John Stultz <john.stultz@linaro.org>
> Cc: Jon Hunter <jonathanh@nvidia.com>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Peter Zijlstra <peterz@infradead.org>
> Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
> Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
> Cc: Thomas Gleixner <tglx@linutronix.de>
> Cc: rt@linutronix.de
> ---
> include/linux/cpuhotplug.h | 2 +-
> kernel/cpu.c | 15 ++++++++++-----
> 2 files changed, 11 insertions(+), 6 deletions(-)
>
> diff --git a/include/linux/cpuhotplug.h b/include/linux/cpuhotplug.h
> index 6d405db..242bf53 100644
> --- a/include/linux/cpuhotplug.h
> +++ b/include/linux/cpuhotplug.h
> @@ -20,9 +20,9 @@ enum cpuhp_state {
> CPUHP_PROFILE_PREPARE,
> CPUHP_X2APIC_PREPARE,
> CPUHP_SMPCFD_PREPARE,
> - CPUHP_TIMERS_DEAD,
> CPUHP_RCUTREE_PREP,
> CPUHP_NOTIFY_PREPARE,
> + CPUHP_TIMERS_DEAD,
> CPUHP_BRINGUP_CPU,
> CPUHP_AP_IDLE_DEAD,
> CPUHP_AP_OFFLINE,
> diff --git a/kernel/cpu.c b/kernel/cpu.c
> index 67f4943..ec12b72 100644
> --- a/kernel/cpu.c
> +++ b/kernel/cpu.c
> @@ -1208,11 +1208,6 @@ static struct cpuhp_step cpuhp_bp_states[] = {
> .startup = smpcfd_prepare_cpu,
> .teardown = smpcfd_dead_cpu,
> },
> - [CPUHP_TIMERS_DEAD] = {
> - .name = "timers dead",
> - .startup = NULL,
> - .teardown = timers_dead_cpu,
> - },
> [CPUHP_RCUTREE_PREP] = {
> .name = "RCU-tree prepare",
> .startup = rcutree_prepare_cpu,
> @@ -1229,6 +1224,16 @@ static struct cpuhp_step cpuhp_bp_states[] = {
> .skip_onerr = true,
> .cant_stop = true,
> },
> + /*
> + * On the tear-down path, timers_dead_cpu() must be invoked
> + * before blk_mq_queue_reinit_notify() from notify_dead(),
> + * otherwise a RCU stall occurs.
> + */
> + [CPUHP_TIMERS_DEAD] = {
> + .name = "timers dead",
> + .startup = NULL,
> + .teardown = timers_dead_cpu,
> + },
> /* Kicks the plugged cpu into life */
> [CPUHP_BRINGUP_CPU] = {
> .name = "cpu:bringup",
>
This fixes the RCU stalls I was seeing during suspend on Tegra and so ...
Tested-by: Jon Hunter <jonathanh@nvidia.com>
Cheers!
Jon
--
nvpublic
next prev parent reply other threads:[~2016-07-27 9:51 UTC|newest]
Thread overview: 170+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-13 17:16 [patch V2 00/67] cpuhotplug: Convert all priority notifiers to the state machine Anna-Maria Gleixner
2016-07-13 17:16 ` [patch V2 02/67] x86/vdso: Convert to hotplug " Anna-Maria Gleixner
2016-07-19 6:58 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 01/67] cpuhotplug: Handle early registration gracefully Anna-Maria Gleixner
2016-07-19 6:57 ` [tip:smp/hotplug] cpu/hotplug: " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 03/67] irqchip/gic: Convert to hotplug state machine Anna-Maria Gleixner
2016-07-19 6:58 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 04/67] irqchip/gicv3: " Anna-Maria Gleixner
2016-07-19 6:59 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 05/67] irqchip/hip04: " Anna-Maria Gleixner
2016-07-19 6:59 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 07/67] irqchip/bcm2836: " Anna-Maria Gleixner
2016-07-19 7:00 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 06/67] irqchip/armada-370-xp: " Anna-Maria Gleixner
2016-07-19 7:00 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 08/67] ARM: mvebu: " Anna-Maria Gleixner
2016-07-19 7:01 ` [tip:smp/hotplug] ARM/mvebu: " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 09/67] perf/core: " Anna-Maria Gleixner
2016-07-19 7:01 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 10/67] perf/x86: Convert the core to the " Anna-Maria Gleixner
2016-07-19 7:02 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 11/67] perf/x86/intel/uncore: Convert to " Anna-Maria Gleixner
2016-07-19 7:02 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 12/67] perf/x86/amd/uncore: " Anna-Maria Gleixner
2016-07-19 7:03 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 13/67] perf/x86/amd/ibs: " Anna-Maria Gleixner
2016-07-19 7:03 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 14/67] perf/x86/intel/rapl: " Anna-Maria Gleixner
2016-07-19 7:04 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 15/67] perf/x86/intel/cqm: Convert Intel CQM " Anna-Maria Gleixner
2016-07-19 7:04 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 16/67] perf/x86/intel/cstate: Convert Intel CSTATE " Anna-Maria Gleixner
2016-07-19 7:05 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 17/67] blackfin/perf: Convert hotplug notifier to " Anna-Maria Gleixner
2016-07-19 7:05 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 18/67] powerpc/perf: Convert book3s notifier to state machine callbacks Anna-Maria Gleixner
2016-07-19 7:06 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 19/67] s390/perf: Convert the hotplug notifier to state machine callbacks (Counter) Anna-Maria Gleixner
2016-07-19 7:07 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 20/67] s390/perf: Convert the hotplug notifier to state machine callbacks (Sampling) Anna-Maria Gleixner
2016-07-19 7:07 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 21/67] sh/perf: Convert the hotplug notifiers to state machine callbacks Anna-Maria Gleixner
2016-07-19 7:07 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 22/67] bus/arm-cci: Convert to hotplug statemachine Anna-Maria Gleixner
2016-07-15 12:38 ` Punit Agrawal
2016-07-19 7:08 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 23/67] bus/arm-ccn: " Anna-Maria Gleixner
2016-07-18 18:37 ` Pawel Moll
2016-07-19 7:09 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 24/67] xtensa/perf: Convert the hotplug notifier to state machine callbacks Anna-Maria Gleixner
2016-07-19 7:09 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 25/67] perf/x86/amd/power: Change hotplug notifier to a symmetric structure Anna-Maria Gleixner
2016-07-19 7:10 ` [tip:smp/hotplug] " tip-bot for Anna-Maria Gleixner
2016-07-13 17:16 ` [patch V2 26/67] perf/x86/amd/power: Convert the hotplug notifier to state machine Anna-Maria Gleixner
2016-07-19 7:11 ` [tip:smp/hotplug] " tip-bot for Anna-Maria Gleixner
2016-07-13 17:16 ` [patch V2 27/67] perf/core: Remove perf CPU notifier code Anna-Maria Gleixner
2016-07-19 7:11 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 28/67] workqueue: Convert to state machine callbacks Anna-Maria Gleixner
2016-07-19 7:12 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 29/67] x86/hpet: Convert to hotplug state machine Anna-Maria Gleixner
2016-07-19 7:13 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 30/67] powerpc/numa: " Anna-Maria Gleixner
2016-07-14 21:42 ` Anton Blanchard
2016-07-14 23:37 ` Anna-Maria Gleixner
2016-07-15 0:28 ` Anton Blanchard
2016-07-15 8:43 ` Ingo Molnar
2016-07-15 12:14 ` Anton Blanchard
2016-07-15 16:20 ` Sebastian Andrzej Siewior
2016-07-18 14:07 ` [PATCH v2] " Sebastian Andrzej Siewior
2016-07-22 19:57 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 31/67] KVM/x86: Remove superfluous SMP function call Anna-Maria Gleixner
2016-07-14 8:21 ` Paolo Bonzini
2016-07-19 7:13 ` [tip:smp/hotplug] " tip-bot for Anna-Maria Gleixner
2016-07-13 17:16 ` [patch V2 32/67] x86/kvm/kvmclock: Convert to hotplug state machine Anna-Maria Gleixner
2016-07-19 7:14 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 33/67] x86/apb_timer: " Anna-Maria Gleixner
2016-07-19 7:14 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 34/67] arm: Convert VFP hotplug notifiers to " Anna-Maria Gleixner
2016-07-19 7:15 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 35/67] arm/perf: Convert to hotplug " Anna-Maria Gleixner
2016-07-15 13:08 ` Mark Rutland
2016-07-15 15:26 ` Sebastian Andrzej Siewior
2016-07-15 16:16 ` Mark Rutland
2016-07-19 7:16 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-19 7:35 ` Sebastian Andrzej Siewior
2016-07-19 7:48 ` Ingo Molnar
2016-07-19 9:02 ` [PATCH v2] " Sebastian Andrzej Siewior
2016-07-19 9:57 ` Mark Rutland
2016-07-19 11:17 ` [PATCH v3] " Sebastian Andrzej Siewior
2016-07-20 8:01 ` Ingo Molnar
2016-07-20 10:41 ` [tip:smp/hotplug] arm/perf: Fix hotplug state machine conversion tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 36/67] virt: Convert kvm hotplug to state machine Anna-Maria Gleixner
2016-07-19 7:16 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 37/67] ACPI/processor: Avoid STARTING/DYING actions in a more logical way Anna-Maria Gleixner
2016-07-19 7:17 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 38/67] clocksource/arm_arch_timer: Convert to hotplug state machine Anna-Maria Gleixner
2016-07-19 7:17 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 39/67] clocksource/dummy_timer: " Anna-Maria Gleixner
2016-07-19 7:18 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 40/67] clocksource/metag: " Anna-Maria Gleixner
2016-07-19 7:19 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 41/67] clocksource/qcom-timer: " Anna-Maria Gleixner
2016-07-19 7:19 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 42/67] clocksource/mips-gic: " Anna-Maria Gleixner
2016-07-19 7:20 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 43/67] leds/trigger/cpu: " Anna-Maria Gleixner
2016-07-14 7:19 ` Jacek Anaszewski
2016-07-14 7:47 ` Ingo Molnar
2016-07-14 8:10 ` Jacek Anaszewski
2016-07-14 9:41 ` Peter Zijlstra
2016-07-14 11:23 ` Jacek Anaszewski
2016-07-14 11:33 ` Thomas Gleixner
2016-07-14 11:55 ` Jacek Anaszewski
2016-07-15 14:10 ` Sebastian Andrzej Siewior
2016-07-18 8:26 ` Jacek Anaszewski
2016-07-19 7:20 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 44/67] arm/kvm/vgic: " Anna-Maria Gleixner
2016-07-19 7:21 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 45/67] arm/kvm/arch_timer: " Anna-Maria Gleixner
2016-07-19 7:21 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 46/67] metag/perf: " Anna-Maria Gleixner
2016-07-19 7:22 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 47/67] arm/l2c: " Anna-Maria Gleixner
2016-07-19 7:22 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 48/67] arm/twd: " Anna-Maria Gleixner
2016-07-19 7:23 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 49/67] arm/xen: " Anna-Maria Gleixner
2016-07-13 18:21 ` Stefano Stabellini
2016-07-19 7:23 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 50/67] MIPS/Loongson-3: Convert oprofile " Anna-Maria Gleixner
2016-07-19 7:24 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 51/67] hwtracing/coresight-etm3x: Convert " Anna-Maria Gleixner
2016-07-19 7:24 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 52/67] hwtracing/coresight-etm4x: " Anna-Maria Gleixner
2016-07-19 7:25 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 53/67] arm64/armv8 deprecated: " Anna-Maria Gleixner
2016-07-19 7:25 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 54/67] x86/tboot: " Anna-Maria Gleixner
2016-07-19 7:26 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:16 ` [patch V2 55/67] hrtimer: " Anna-Maria Gleixner
2016-07-19 7:26 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:16 ` [patch V2 57/67] profile: " Anna-Maria Gleixner
2016-07-19 7:27 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:16 ` [patch V2 56/67] timers/core: " Anna-Maria Gleixner
2016-07-19 7:27 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-27 9:08 ` [PATCH] timers/core: Correct callback order during CPU hot plug Richard Cochran
2016-07-27 9:50 ` Jon Hunter [this message]
2016-07-28 15:34 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-28 16:57 ` tip-bot for Richard Cochran
2016-07-13 17:17 ` [patch V2 58/67] x86/x2apic: Convert to CPU hotplug state machine Anna-Maria Gleixner
2016-07-19 7:28 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2016-07-13 17:17 ` [patch V2 59/67] smp: Convert core to " Anna-Maria Gleixner
2016-07-19 7:28 ` [tip:smp/hotplug] smp/cfd: " tip-bot for Richard Weinberger
2016-07-13 17:17 ` [patch V2 60/67] KVM/arm/arm64/vgic-new: Convert " Anna-Maria Gleixner
2016-07-14 13:08 ` Marc Zyngier
2016-07-19 7:29 ` [tip:smp/hotplug] " tip-bot for Anna-Maria Gleixner
2016-07-13 17:17 ` [patch V2 61/67] rcu: Convert rcutree " Anna-Maria Gleixner
2016-07-19 7:29 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2016-07-13 17:17 ` [patch V2 62/67] [PATCH] clocksource: arm_global_timer: Convert " Anna-Maria Gleixner
2016-07-19 7:30 ` [tip:smp/hotplug] clocksource/arm_global_timer: " tip-bot for Richard Cochran
2016-07-13 17:17 ` [patch V2 63/67] clocksource/exynos_mct: " Anna-Maria Gleixner
2016-07-19 7:31 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:17 ` [patch V2 64/67] clocksource/armada-370-xp: " Anna-Maria Gleixner
2016-07-19 7:31 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:17 ` [patch V2 65/67] clocksource/atlas7: " Anna-Maria Gleixner
2016-07-19 7:32 ` [tip:smp/hotplug] " tip-bot for Richard Cochran
2016-07-13 17:17 ` [patch V2 66/67] ARC/time: " Anna-Maria Gleixner
2016-07-19 7:32 ` [tip:smp/hotplug] " tip-bot for Anna-Maria Gleixner
2016-07-13 17:17 ` [patch V2 67/67] cpuhotplug: Remove CPU_STARTING and CPU_DYING notifier Anna-Maria Gleixner
2016-07-13 19:32 ` [patch V2 00/67] cpuhotplug: Convert all priority notifiers to the state machine Jason Cooper
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=25aca1c4-5bc0-838c-ac69-45ec97031da2@nvidia.com \
--to=jonathanh@nvidia.com \
--cc=bigeasy@linutronix.de \
--cc=john.stultz@linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tegra@vger.kernel.org \
--cc=linux@rasmusvillemoes.dk \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=rcochran@linutronix.de \
--cc=rt@linutronix.de \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).