From: Thomas Gleixner <tglx@linutronix.de>
To: LKML <linux-kernel@vger.kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@kernel.org>,
Steven Rostedt <rostedt@goodmis.org>,
Sebastian Siewior <bigeasy@linutronix.de>,
Paul McKenney <paulmck@linux.vnet.ibm.com>,
Bjorn Helgaas <bhelgaas@google.com>,
linux-pci@vger.kernel.org
Subject: [patch V3 20/32] PCI: Use cpu_hotplug_disable() instead of get_online_cpus()
Date: Wed, 24 May 2017 10:15:31 +0200 [thread overview]
Message-ID: <20170524081548.691198590@linutronix.de> (raw)
In-Reply-To: 20170524081511.203800767@linutronix.de
[-- Attachment #1: PCI_Use_cpu_hotplug_disable_instead_of_get_online_cpus.patch --]
[-- Type: text/plain, Size: 2448 bytes --]
Converting the hotplug locking, i.e. get_online_cpus(), to a percpu rwsem
unearthed a circular lock dependency which was hidden from lockdep due to
the lockdep annotation of get_online_cpus() which prevents lockdep from
creating full dependency chains. There are several variants of this. And
example is:
Chain exists of:
cpu_hotplug_lock.rw_sem --> drm_global_mutex --> &item->mutex
CPU0 CPU1
---- ----
lock(&item->mutex);
lock(drm_global_mutex);
lock(&item->mutex);
lock(cpu_hotplug_lock.rw_sem);
because there are dependencies through workqueues. The call chain is:
get_online_cpus
apply_workqueue_attrs
__alloc_workqueue_key
ttm_mem_global_init
ast_ttm_mem_global_init
drm_global_item_ref
ast_mm_init
ast_driver_load
drm_dev_register
drm_get_pci_dev
ast_pci_probe
local_pci_probe
work_for_cpu_fn
process_one_work
worker_thread
This is not a problem of get_online_cpus() recursion, it's a possible
deadlock undetected by lockdep so far.
The cure is to use cpu_hotplug_disable() instead of get_online_cpus() to
protect the PCI probing.
There is a side effect to this: cpu_hotplug_disable() makes a concurrent
cpu hotplug attempt via the sysfs interfaces fail with -EBUSY, but PCI
probing usually happens during the boot process where no interaction is
possible. Any later invocations are infrequent enough and concurrent
hotplug attempts are so unlikely that the danger of user space visible
regressions is very close to zero. Anyway, thats preferrable over a real
deadlock.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Acked-by: Bjorn Helgaas <bhelgaas@google.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Siewior <bigeasy@linutronix.de>
Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: linux-pci@vger.kernel.org
---
drivers/pci/pci-driver.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -349,13 +349,13 @@ static int pci_call_probe(struct pci_dri
if (node >= 0 && node != numa_node_id()) {
int cpu;
- get_online_cpus();
+ cpu_hotplug_disable();
cpu = cpumask_any_and(cpumask_of_node(node), cpu_online_mask);
if (cpu < nr_cpu_ids)
error = work_on_cpu(cpu, local_pci_probe, &ddi);
else
error = local_pci_probe(&ddi);
- put_online_cpus();
+ cpu_hotplug_enable();
} else
error = local_pci_probe(&ddi);
next prev parent reply other threads:[~2017-05-24 8:29 UTC|newest]
Thread overview: 82+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-24 8:15 [patch V3 00/32] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Thomas Gleixner
2017-05-24 8:15 ` [patch V3 01/32] cpu/hotplug: Provide cpus_read|write_[un]lock() Thomas Gleixner
2017-05-24 16:25 ` Paul E. McKenney
2017-05-26 8:31 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 02/32] cpu/hotplug: Provide lockdep_assert_cpus_held() Thomas Gleixner
2017-05-24 16:26 ` Paul E. McKenney
2017-05-26 8:32 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 03/32] cpu/hotplug: Provide cpuhp_setup/remove_state[_nocalls]_cpuslocked() Thomas Gleixner
2017-05-26 8:32 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 04/32] cpu/hotplug: Add __cpuhp_state_add_instance_cpuslocked() Thomas Gleixner
2017-05-26 8:33 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 05/32] stop_machine: Provide stop_machine_cpuslocked() Thomas Gleixner
2017-05-24 17:42 ` Paul E. McKenney
2017-05-26 8:33 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 06/32] padata: Make padata_alloc() static Thomas Gleixner
2017-05-26 8:34 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 07/32] padata: Avoid nested calls to cpus_read_lock() in pcrypt_init_padata() Thomas Gleixner
2017-05-26 8:35 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 08/32] x86/mtrr: Remove get_online_cpus() from mtrr_save_state() Thomas Gleixner
2017-05-26 8:35 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 09/32] cpufreq: Use cpuhp_setup_state_nocalls_cpuslocked() Thomas Gleixner
2017-05-26 8:36 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 10/32] KVM/PPC/Book3S HV: " Thomas Gleixner
2017-05-26 8:36 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 11/32] hwtracing/coresight-etm3x: " Thomas Gleixner
2017-05-25 16:46 ` Mathieu Poirier
2017-05-26 8:37 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 12/32] hwtracing/coresight-etm4x: " Thomas Gleixner
2017-05-25 16:47 ` Mathieu Poirier
2017-05-26 8:37 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 13/32] perf/x86/intel/cqm: Use cpuhp_setup_state_cpuslocked() Thomas Gleixner
2017-05-26 8:38 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 14/32] ARM/hw_breakpoint: " Thomas Gleixner
2017-05-26 8:38 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 15/32] s390/kernel: Use stop_machine_cpuslocked() Thomas Gleixner
2017-05-24 10:57 ` Heiko Carstens
2017-05-26 8:39 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 16/32] powerpc/powernv: " Thomas Gleixner
2017-05-26 8:40 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 17/32] cpu/hotplug: Use stop_machine_cpuslocked() in takedown_cpu() Thomas Gleixner
2017-05-26 8:40 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` [patch V3 18/32] x86/perf: Drop EXPORT of perf_check_microcode Thomas Gleixner
2017-05-26 8:41 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 19/32] perf/x86/intel: Drop get_online_cpus() in intel_snb_check_microcode() Thomas Gleixner
2017-05-26 8:41 ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24 8:15 ` Thomas Gleixner [this message]
2017-05-26 8:42 ` [tip:smp/hotplug] PCI: Use cpu_hotplug_disable() instead of get_online_cpus() tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 21/32] PCI: Replace the racy recursion prevention Thomas Gleixner
2017-05-26 8:42 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 22/32] ACPI/processor: Use cpu_hotplug_disable() instead of get_online_cpus() Thomas Gleixner
2017-05-26 8:43 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 23/32] perf/tracing/cpuhotplug: Fix locking order Thomas Gleixner
2017-05-24 18:30 ` Paul E. McKenney
2017-05-24 18:47 ` Thomas Gleixner
2017-05-24 21:10 ` Paul E. McKenney
2017-05-30 11:22 ` Peter Zijlstra
2017-05-30 16:25 ` Paul E. McKenney
2017-05-26 8:43 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 24/32] jump_label: Reorder hotplug lock and jump_label_lock Thomas Gleixner
2017-05-24 12:50 ` David Miller
2017-05-26 8:44 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 25/32] kprobes: Cure hotplug lock ordering issues Thomas Gleixner
2017-05-24 15:54 ` Masami Hiramatsu
2017-05-26 7:47 ` Thomas Gleixner
2017-05-26 8:45 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 26/32] arm64: Prevent cpu hotplug rwsem recursion Thomas Gleixner
2017-05-26 8:45 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 27/32] arm: Prevent " Thomas Gleixner
2017-05-26 8:46 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 28/32] s390: " Thomas Gleixner
2017-05-24 10:57 ` Heiko Carstens
2017-05-26 8:46 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 29/32] cpu/hotplug: Convert hotplug locking to percpu rwsem Thomas Gleixner
2017-05-26 8:47 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 30/32] sched: Provide is_percpu_thread() helper Thomas Gleixner
2017-05-26 8:47 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 31/32] acpi/processor: Prevent cpu hotplug deadlock Thomas Gleixner
2017-05-26 8:48 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 8:15 ` [patch V3 32/32] cpuhotplug: Link lock stacks for hotplug callbacks Thomas Gleixner
2017-05-26 8:48 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 16:22 ` [patch V3 00/32] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Paul E. McKenney
2017-05-26 7:03 ` Ingo Molnar
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170524081548.691198590@linutronix.de \
--to=tglx@linutronix.de \
--cc=bhelgaas@google.com \
--cc=bigeasy@linutronix.de \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).