From: Mark Rutland <mark.rutland@arm.com> To: Sebastian Siewior <bigeasy@linutronix.de>, catalin.marinas@arm.com, will.deacon@arm.com Cc: Peter Zijlstra <peterz@infradead.org>, Thomas Gleixner <tglx@linutronix.de>, LKML <linux-kernel@vger.kernel.org>, Ingo Molnar <mingo@kernel.org>, Steven Rostedt <rostedt@goodmis.org>, suzuki.poulose@arm.com, linux-arm-kernel@lists.infradead.org Subject: Re: [patch V2 00/24] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Date: Wed, 26 Apr 2017 09:59:59 +0100 [thread overview] Message-ID: <20170426085958.GC27156@leverpostej> (raw) In-Reply-To: <20170425172838.mr3kyccsdteyjso5@linutronix.de> On Tue, Apr 25, 2017 at 07:28:38PM +0200, Sebastian Siewior wrote: > On 2017-04-25 17:10:37 [+0100], Mark Rutland wrote: > > When we bring the secondary CPU online, we detect an erratum that wasn't > > present on the boot CPU, and try to enable a static branch we use to > > track the erratum. The call to static_branch_enable() blows up as above. > > this (cpus_set_cap()) seems only to be used used in CPU up part. > > > I see that we now have static_branch_disable_cpuslocked(), but we don't > > have an equivalent for enable. I'm not sure what we should be doing > > here. > > We should introduce static_branch_enable_cpuslocked(). Does this work > for you (after s/static_branch_enable/static_branch_enable_cpuslocked/ > in cpus_set_cap()) ?: The patch you linked worked for me, given the below patch for arm64 to make use of static_branch_enable_cpuslocked(). Catalin/Will, are you happy for this to go via the tip tree with the other hotplug locking changes? Thanks, Mark. ---->8---- >From 03c98baf0a9fcdfb87145bfb3f108d49a721dad6 Mon Sep 17 00:00:00 2001 From: Mark Rutland <mark.rutland@arm.com> Date: Wed, 26 Apr 2017 09:46:47 +0100 Subject: [PATCH] arm64: cpufeature: use static_branch_enable_cpuslocked() Recently, the hotplug locking was conveted to use a percpu rwsem. Unlike the existing {get,put}_online_cpus() logic, this can't nest. Unfortunately, in arm64's secondary boot path we can end up nesting via static_branch_enable() in cpus_set_cap() when we detect an erratum. This leads to a stream of messages as below, where the secondary attempts to schedule befroe it has been fully onlined. As the CPU orchsetrating the onlining holds the rswem, this hangs the system. [ 0.250334] BUG: scheduling while atomic: swapper/1/0/0x00000002 [ 0.250337] Modules linked in: [ 0.250346] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.11.0-rc7-next-20170424 #2 [ 0.250349] Hardware name: ARM Juno development board (r1) (DT) [ 0.250353] Call trace: [ 0.250365] [<ffff000008088510>] dump_backtrace+0x0/0x238 [ 0.250371] [<ffff00000808880c>] show_stack+0x14/0x20 [ 0.250377] [<ffff00000839d854>] dump_stack+0x9c/0xc0 [ 0.250384] [<ffff0000080e3540>] __schedule_bug+0x50/0x70 [ 0.250391] [<ffff000008932ecc>] __schedule+0x52c/0x5a8 [ 0.250395] [<ffff000008932f80>] schedule+0x38/0xa0 [ 0.250400] [<ffff000008935e8c>] rwsem_down_read_failed+0xc4/0x108 [ 0.250407] [<ffff0000080fe8e0>] __percpu_down_read+0x100/0x118 [ 0.250414] [<ffff0000080c0b60>] get_online_cpus+0x70/0x78 [ 0.250420] [<ffff0000081749e8>] static_key_enable+0x28/0x48 [ 0.250425] [<ffff00000808de90>] update_cpu_capabilities+0x78/0xf8 [ 0.250430] [<ffff00000808d14c>] update_cpu_errata_workarounds+0x1c/0x28 [ 0.250435] [<ffff00000808e004>] check_local_cpu_capabilities+0xf4/0x128 [ 0.250440] [<ffff00000808e894>] secondary_start_kernel+0x8c/0x118 [ 0.250444] [<000000008093d1b4>] 0x8093d1b4 We only call cpus_set_cap() in the secondary boot path, where we know that the rwsem is held by the thread orchestrating the onlining. Thus, we can use the new static_branch_enable_cpuslocked() in cpus_set_cap(), avoiding the above. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reported-by: Catalin Marinas <catalin.marinas@arm.com> Suggested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Will Deacon <will.deacon@arm.com> Cc: Suzuki Poulose <suzuki,poulose@arm.com> --- arch/arm64/include/asm/cpufeature.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index f31c48d..349b5cd 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -145,7 +145,7 @@ static inline void cpus_set_cap(unsigned int num) num, ARM64_NCAPS); } else { __set_bit(num, cpu_hwcaps); - static_branch_enable(&cpu_hwcap_keys[num]); + static_branch_enable_cpuslocked(&cpu_hwcap_keys[num]); } } -- 1.9.1
WARNING: multiple messages have this Message-ID (diff)
From: mark.rutland@arm.com (Mark Rutland) To: linux-arm-kernel@lists.infradead.org Subject: [patch V2 00/24] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Date: Wed, 26 Apr 2017 09:59:59 +0100 [thread overview] Message-ID: <20170426085958.GC27156@leverpostej> (raw) In-Reply-To: <20170425172838.mr3kyccsdteyjso5@linutronix.de> On Tue, Apr 25, 2017 at 07:28:38PM +0200, Sebastian Siewior wrote: > On 2017-04-25 17:10:37 [+0100], Mark Rutland wrote: > > When we bring the secondary CPU online, we detect an erratum that wasn't > > present on the boot CPU, and try to enable a static branch we use to > > track the erratum. The call to static_branch_enable() blows up as above. > > this (cpus_set_cap()) seems only to be used used in CPU up part. > > > I see that we now have static_branch_disable_cpuslocked(), but we don't > > have an equivalent for enable. I'm not sure what we should be doing > > here. > > We should introduce static_branch_enable_cpuslocked(). Does this work > for you (after s/static_branch_enable/static_branch_enable_cpuslocked/ > in cpus_set_cap()) ?: The patch you linked worked for me, given the below patch for arm64 to make use of static_branch_enable_cpuslocked(). Catalin/Will, are you happy for this to go via the tip tree with the other hotplug locking changes? Thanks, Mark. ---->8---- >From 03c98baf0a9fcdfb87145bfb3f108d49a721dad6 Mon Sep 17 00:00:00 2001 From: Mark Rutland <mark.rutland@arm.com> Date: Wed, 26 Apr 2017 09:46:47 +0100 Subject: [PATCH] arm64: cpufeature: use static_branch_enable_cpuslocked() Recently, the hotplug locking was conveted to use a percpu rwsem. Unlike the existing {get,put}_online_cpus() logic, this can't nest. Unfortunately, in arm64's secondary boot path we can end up nesting via static_branch_enable() in cpus_set_cap() when we detect an erratum. This leads to a stream of messages as below, where the secondary attempts to schedule befroe it has been fully onlined. As the CPU orchsetrating the onlining holds the rswem, this hangs the system. [ 0.250334] BUG: scheduling while atomic: swapper/1/0/0x00000002 [ 0.250337] Modules linked in: [ 0.250346] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.11.0-rc7-next-20170424 #2 [ 0.250349] Hardware name: ARM Juno development board (r1) (DT) [ 0.250353] Call trace: [ 0.250365] [<ffff000008088510>] dump_backtrace+0x0/0x238 [ 0.250371] [<ffff00000808880c>] show_stack+0x14/0x20 [ 0.250377] [<ffff00000839d854>] dump_stack+0x9c/0xc0 [ 0.250384] [<ffff0000080e3540>] __schedule_bug+0x50/0x70 [ 0.250391] [<ffff000008932ecc>] __schedule+0x52c/0x5a8 [ 0.250395] [<ffff000008932f80>] schedule+0x38/0xa0 [ 0.250400] [<ffff000008935e8c>] rwsem_down_read_failed+0xc4/0x108 [ 0.250407] [<ffff0000080fe8e0>] __percpu_down_read+0x100/0x118 [ 0.250414] [<ffff0000080c0b60>] get_online_cpus+0x70/0x78 [ 0.250420] [<ffff0000081749e8>] static_key_enable+0x28/0x48 [ 0.250425] [<ffff00000808de90>] update_cpu_capabilities+0x78/0xf8 [ 0.250430] [<ffff00000808d14c>] update_cpu_errata_workarounds+0x1c/0x28 [ 0.250435] [<ffff00000808e004>] check_local_cpu_capabilities+0xf4/0x128 [ 0.250440] [<ffff00000808e894>] secondary_start_kernel+0x8c/0x118 [ 0.250444] [<000000008093d1b4>] 0x8093d1b4 We only call cpus_set_cap() in the secondary boot path, where we know that the rwsem is held by the thread orchestrating the onlining. Thus, we can use the new static_branch_enable_cpuslocked() in cpus_set_cap(), avoiding the above. Signed-off-by: Mark Rutland <mark.rutland@arm.com> Reported-by: Catalin Marinas <catalin.marinas@arm.com> Suggested-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de> Cc: Will Deacon <will.deacon@arm.com> Cc: Suzuki Poulose <suzuki,poulose@arm.com> --- arch/arm64/include/asm/cpufeature.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h index f31c48d..349b5cd 100644 --- a/arch/arm64/include/asm/cpufeature.h +++ b/arch/arm64/include/asm/cpufeature.h @@ -145,7 +145,7 @@ static inline void cpus_set_cap(unsigned int num) num, ARM64_NCAPS); } else { __set_bit(num, cpu_hwcaps); - static_branch_enable(&cpu_hwcap_keys[num]); + static_branch_enable_cpuslocked(&cpu_hwcap_keys[num]); } } -- 1.9.1
next prev parent reply other threads:[~2017-04-26 9:02 UTC|newest] Thread overview: 98+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-04-18 17:04 [patch V2 00/24] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Thomas Gleixner 2017-04-18 17:04 ` [patch V2 01/24] cpu/hotplug: Provide cpuhp_setup/remove_state[_nocalls]_cpuslocked() Thomas Gleixner 2017-04-20 11:18 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 02/24] stop_machine: Provide stop_machine_cpuslocked() Thomas Gleixner 2017-04-20 11:19 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 03/24] padata: Make padata_alloc() static Thomas Gleixner 2017-04-20 11:19 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner 2017-04-18 17:04 ` [patch V2 04/24] padata: Avoid nested calls to get_online_cpus() in pcrypt_init_padata() Thomas Gleixner 2017-04-20 11:20 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 05/24] x86/mtrr: Remove get_online_cpus() from mtrr_save_state() Thomas Gleixner 2017-04-20 11:20 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 06/24] cpufreq: Use cpuhp_setup_state_nocalls_cpuslocked() Thomas Gleixner 2017-04-20 11:21 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 07/24] KVM/PPC/Book3S HV: " Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-20 11:21 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 08/24] hwtracing/coresight-etm3x: " Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-20 11:22 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-20 15:14 ` [patch V2 08/24] " Mathieu Poirier 2017-04-20 15:14 ` Mathieu Poirier 2017-04-20 15:32 ` Mathieu Poirier 2017-04-20 15:32 ` Mathieu Poirier 2017-04-18 17:04 ` [patch V2 09/24] hwtracing/coresight-etm4x: " Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-20 11:22 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 10/24] perf/x86/intel/cqm: Use cpuhp_setup_state_cpuslocked() Thomas Gleixner 2017-04-20 11:23 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 11/24] ARM/hw_breakpoint: " Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-19 17:54 ` Mark Rutland 2017-04-19 17:54 ` Mark Rutland 2017-04-19 18:20 ` Thomas Gleixner 2017-04-19 18:20 ` Thomas Gleixner 2017-04-20 11:23 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 12/24] s390/kernel: Use stop_machine_cpuslocked() Thomas Gleixner 2017-04-20 11:24 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 13/24] powerpc/powernv: " Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-20 11:24 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 14/24] cpu/hotplug: Use stop_machine_cpuslocked() in takedown_cpu() Thomas Gleixner 2017-04-20 11:25 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 15/24] x86/perf: Drop EXPORT of perf_check_microcode Thomas Gleixner 2017-04-20 11:25 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner 2017-04-18 17:04 ` [patch V2 16/24] perf/x86/intel: Drop get_online_cpus() in intel_snb_check_microcode() Thomas Gleixner 2017-04-20 11:26 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior 2017-04-18 17:04 ` [patch V2 17/24] PCI: Use cpu_hotplug_disable() instead of get_online_cpus() Thomas Gleixner 2017-04-18 17:04 ` Thomas Gleixner 2017-04-20 11:27 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner 2017-04-18 17:05 ` [patch V2 18/24] PCI: Replace the racy recursion prevention Thomas Gleixner 2017-04-18 17:05 ` Thomas Gleixner 2017-04-20 11:27 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner 2017-04-18 17:05 ` [patch V2 19/24] ACPI/processor: Use cpu_hotplug_disable() instead of get_online_cpus() Thomas Gleixner 2017-04-20 11:28 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner 2017-04-18 17:05 ` [patch V2 20/24] perf/core: Remove redundant get_online_cpus() Thomas Gleixner 2017-04-20 11:28 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner 2017-04-18 17:05 ` [patch V2 21/24] jump_label: Pull get_online_cpus() into generic code Thomas Gleixner 2017-04-18 17:05 ` [patch V2 22/24] jump_label: Provide static_key_slow_inc_cpuslocked() Thomas Gleixner 2017-04-18 17:05 ` [patch V2 23/24] perf: Avoid cpu_hotplug_lock r-r recursion Thomas Gleixner 2017-04-18 17:05 ` [patch V2 24/24] cpu/hotplug: Convert hotplug locking to percpu rwsem Thomas Gleixner 2017-04-20 11:30 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner 2017-05-10 4:59 ` [patch V2 24/24] " Michael Ellerman 2017-05-10 8:49 ` Thomas Gleixner 2017-05-10 16:30 ` Steven Rostedt 2017-05-10 17:15 ` Steven Rostedt 2017-05-11 5:49 ` Michael Ellerman 2017-04-25 16:10 ` [patch V2 00/24] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Mark Rutland 2017-04-25 16:10 ` Mark Rutland 2017-04-25 17:28 ` Sebastian Siewior 2017-04-25 17:28 ` Sebastian Siewior 2017-04-26 8:59 ` Mark Rutland [this message] 2017-04-26 8:59 ` Mark Rutland 2017-04-26 9:40 ` Suzuki K Poulose 2017-04-26 9:40 ` Suzuki K Poulose 2017-04-26 10:32 ` Mark Rutland 2017-04-26 10:32 ` Mark Rutland 2017-04-27 8:27 ` Sebastian Siewior 2017-04-27 8:27 ` Sebastian Siewior 2017-04-27 9:57 ` Mark Rutland 2017-04-27 9:57 ` Mark Rutland 2017-04-27 10:01 ` Thomas Gleixner 2017-04-27 10:01 ` Thomas Gleixner 2017-04-27 12:30 ` Mark Rutland 2017-04-27 12:30 ` Mark Rutland 2017-04-27 15:48 ` [PATCH] arm64: cpufeature: use static_branch_enable_cpuslocked() (was: Re: [patch V2 00/24] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem) Mark Rutland 2017-04-27 15:48 ` Mark Rutland 2017-04-27 16:35 ` Suzuki K Poulose 2017-04-27 16:35 ` Suzuki K Poulose 2017-04-27 17:03 ` [PATCH] arm64: cpufeature: use static_branch_enable_cpuslocked() Suzuki K Poulose 2017-04-27 17:03 ` Suzuki K Poulose 2017-04-27 17:17 ` Mark Rutland 2017-04-27 17:17 ` Mark Rutland 2017-04-28 14:24 ` [RFC PATCH] trace/perf: cure locking issue in perf_event_open() error path Sebastian Siewior 2017-04-28 14:27 ` Sebastian Siewior 2017-05-01 12:57 ` [tip:smp/hotplug] perf: Reorder cpu hotplug rwsem against cred_guard_mutex tip-bot for Thomas Gleixner 2017-05-01 12:58 ` [tip:smp/hotplug] perf: Push hotplug protection down to callers tip-bot for Thomas Gleixner
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20170426085958.GC27156@leverpostej \ --to=mark.rutland@arm.com \ --cc=bigeasy@linutronix.de \ --cc=catalin.marinas@arm.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=mingo@kernel.org \ --cc=peterz@infradead.org \ --cc=rostedt@goodmis.org \ --cc=suzuki.poulose@arm.com \ --cc=tglx@linutronix.de \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.