linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [patch V3 00/32] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem
@ 2017-05-24  8:15 Thomas Gleixner
  2017-05-24  8:15 ` [patch V3 01/32] cpu/hotplug: Provide cpus_read|write_[un]lock() Thomas Gleixner
                   ` (33 more replies)
  0 siblings, 34 replies; 83+ messages in thread
From: Thomas Gleixner @ 2017-05-24  8:15 UTC (permalink / raw)
  To: LKML
  Cc: Peter Zijlstra, Ingo Molnar, Steven Rostedt, Sebastian Siewior,
	Paul McKenney

get_online_cpus() is used in hot pathes in mainline and even more so in
RT. That can show up badly under certain conditions because every locker
contends on a global mutex. RT has it's own homebrewn mitigation which is
a (badly done) open coded implementation of percpu_rwsems with recursion
support.

The proper replacement for that are percpu_rwsems, but that requires to
remove recursion support.

The conversion unearthed real locking issues which were previously not
visible because the get_online_cpus() lockdep annotation was implemented
with recursion support which prevents lockdep from tracking full dependency
chains. These potential deadlocks are not related to recursive calls, they
trigger on the first invocation because lockdep now has the full dependency
chains available.

The following patch series addresses this by

 - Cleaning up places which call get_online_cpus() nested

 - Replacing a few instances with cpu_hotplug_disable() to prevent circular
   locking dependencies.

The series is on top of 4.12-rc2. It's available in git from

   git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git WIP.hotplug

Changes since V2:

  - Reworked the approach vs. perf/ftrace/kprobes, which simplified the lot
  
  - Renamed get_online_cpus() to cpus_read_lock() to reflect the nature of
    the interface

  - Link the lockchains between hotplug control task and per cpu hotplug
    threads and fixed the fallout of that.

Thanks,

        tglx

---
 arch/arm/kernel/hw_breakpoint.c               |   11 -
 arch/arm/kernel/patch.c                       |    2 
 arch/arm/probes/kprobes/core.c                |    3 
 arch/arm64/include/asm/insn.h                 |    1 
 arch/arm64/kernel/insn.c                      |    5 
 arch/mips/kernel/jump_label.c                 |    2 
 arch/powerpc/kvm/book3s_hv.c                  |   14 -
 arch/powerpc/platforms/powernv/subcore.c      |    7 
 arch/s390/kernel/jump_label.c                 |    2 
 arch/s390/kernel/kprobes.c                    |    4 
 arch/s390/kernel/time.c                       |    6 
 arch/x86/events/core.c                        |    1 
 arch/x86/events/intel/cqm.c                   |   16 -
 arch/x86/kernel/cpu/mtrr/main.c               |    2 
 b/arch/sparc/kernel/jump_label.c              |    2 
 b/arch/tile/kernel/jump_label.c               |    2 
 b/arch/x86/events/intel/core.c                |   11 -
 b/arch/x86/kernel/jump_label.c                |    2 
 b/kernel/jump_label.c                         |   20 +-
 drivers/acpi/processor_driver.c               |    4 
 drivers/acpi/processor_throttling.c           |   16 -
 drivers/cpufreq/cpufreq.c                     |   21 +-
 drivers/hwtracing/coresight/coresight-etm3x.c |   20 +-
 drivers/hwtracing/coresight/coresight-etm4x.c |   20 +-
 drivers/pci/pci-driver.c                      |   47 +++--
 include/linux/cpu.h                           |   34 ++--
 include/linux/cpuhotplug.h                    |   38 ++++
 include/linux/padata.h                        |    3 
 include/linux/pci.h                           |    1 
 include/linux/perf_event.h                    |    2 
 include/linux/sched.h                         |   10 +
 include/linux/stop_machine.h                  |   26 ++-
 kernel/cpu.c                                  |  213 +++++++++++---------------
 kernel/events/core.c                          |  106 +++++++++---
 kernel/kprobes.c                              |   59 +++----
 kernel/padata.c                               |   43 ++---
 kernel/stop_machine.c                         |   11 -
 37 files changed, 444 insertions(+), 343 deletions(-)

^ permalink raw reply	[flat|nested] 83+ messages in thread
* [patch V2 17/24] PCI: Use cpu_hotplug_disable() instead of get_online_cpus()
@ 2017-04-18 17:04 Thomas Gleixner
  2017-04-20 11:27 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
  0 siblings, 1 reply; 83+ messages in thread
From: Thomas Gleixner @ 2017-04-18 17:04 UTC (permalink / raw)
  To: LKML
  Cc: Peter Zijlstra, Ingo Molnar, Steven Rostedt, Sebastian Siewior,
	Bjorn Helgaas, linux-pci

[-- Attachment #1: PCI--Use-cpu_hotplug_disable-instead-of-get_online_cpus.patch --]
[-- Type: text/plain, Size: 2312 bytes --]

Converting the hotplug locking, i.e. get_online_cpus(), to a percpu rwsem
unearthed a circular lock dependency which was hidden from lockdep due to
the lockdep annotation of get_online_cpus() which prevents lockdep from
creating full dependency chains. There are several variants of this. And
example is:

Chain exists of:

cpu_hotplug_lock.rw_sem --> drm_global_mutex --> &item->mutex

CPU0                    CPU1
----                    ----
lock(&item->mutex);
                        lock(drm_global_mutex);
                        lock(&item->mutex);
lock(cpu_hotplug_lock.rw_sem);

because there are dependencies through workqueues. The call chain is:

	get_online_cpus
	apply_workqueue_attrs
	__alloc_workqueue_key
	ttm_mem_global_init
	ast_ttm_mem_global_init
	drm_global_item_ref
	ast_mm_init
	ast_driver_load
	drm_dev_register
	drm_get_pci_dev
	ast_pci_probe
	local_pci_probe
	work_for_cpu_fn
	process_one_work
	worker_thread

This is not a problem of get_online_cpus() recursion, it's a possible
deadlock undetected by lockdep so far.

The cure is to use cpu_hotplug_disable() instead of get_online_cpus() to
protect the PCI probing.

There is a side effect to this: cpu_hotplug_disable() makes a concurrent
cpu hotplug attempt via the sysfs interfaces fail with -EBUSY, but PCI
probing usually happens during the boot process where no interaction is
possible. Any later invocations are infrequent enough and concurrent
hotplug attempts are so unlikely that the danger of user space visible
regressions is very close to zero. Anyway, thats preferrable over a real
deadlock.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Bjorn Helgaas <bhelgaas@google.com>
Cc: linux-pci@vger.kernel.org
---
 drivers/pci/pci-driver.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

--- a/drivers/pci/pci-driver.c
+++ b/drivers/pci/pci-driver.c
@@ -349,13 +349,13 @@ static int pci_call_probe(struct pci_dri
 	if (node >= 0 && node != numa_node_id()) {
 		int cpu;
 
-		get_online_cpus();
+		cpu_hotplug_disable();
 		cpu = cpumask_any_and(cpumask_of_node(node), cpu_online_mask);
 		if (cpu < nr_cpu_ids)
 			error = work_on_cpu(cpu, local_pci_probe, &ddi);
 		else
 			error = local_pci_probe(&ddi);
-		put_online_cpus();
+		cpu_hotplug_enable();
 	} else
 		error = local_pci_probe(&ddi);
 

^ permalink raw reply	[flat|nested] 83+ messages in thread

end of thread, other threads:[~2017-05-30 16:25 UTC | newest]

Thread overview: 83+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-24  8:15 [patch V3 00/32] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Thomas Gleixner
2017-05-24  8:15 ` [patch V3 01/32] cpu/hotplug: Provide cpus_read|write_[un]lock() Thomas Gleixner
2017-05-24 16:25   ` Paul E. McKenney
2017-05-26  8:31   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 02/32] cpu/hotplug: Provide lockdep_assert_cpus_held() Thomas Gleixner
2017-05-24 16:26   ` Paul E. McKenney
2017-05-26  8:32   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 03/32] cpu/hotplug: Provide cpuhp_setup/remove_state[_nocalls]_cpuslocked() Thomas Gleixner
2017-05-26  8:32   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 04/32] cpu/hotplug: Add __cpuhp_state_add_instance_cpuslocked() Thomas Gleixner
2017-05-26  8:33   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 05/32] stop_machine: Provide stop_machine_cpuslocked() Thomas Gleixner
2017-05-24 17:42   ` Paul E. McKenney
2017-05-26  8:33   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 06/32] padata: Make padata_alloc() static Thomas Gleixner
2017-05-26  8:34   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 07/32] padata: Avoid nested calls to cpus_read_lock() in pcrypt_init_padata() Thomas Gleixner
2017-05-26  8:35   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 08/32] x86/mtrr: Remove get_online_cpus() from mtrr_save_state() Thomas Gleixner
2017-05-26  8:35   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 09/32] cpufreq: Use cpuhp_setup_state_nocalls_cpuslocked() Thomas Gleixner
2017-05-26  8:36   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 10/32] KVM/PPC/Book3S HV: " Thomas Gleixner
2017-05-26  8:36   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 11/32] hwtracing/coresight-etm3x: " Thomas Gleixner
2017-05-25 16:46   ` Mathieu Poirier
2017-05-26  8:37   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 12/32] hwtracing/coresight-etm4x: " Thomas Gleixner
2017-05-25 16:47   ` Mathieu Poirier
2017-05-26  8:37   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 13/32] perf/x86/intel/cqm: Use cpuhp_setup_state_cpuslocked() Thomas Gleixner
2017-05-26  8:38   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 14/32] ARM/hw_breakpoint: " Thomas Gleixner
2017-05-26  8:38   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 15/32] s390/kernel: Use stop_machine_cpuslocked() Thomas Gleixner
2017-05-24 10:57   ` Heiko Carstens
2017-05-26  8:39   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 16/32] powerpc/powernv: " Thomas Gleixner
2017-05-26  8:40   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 17/32] cpu/hotplug: Use stop_machine_cpuslocked() in takedown_cpu() Thomas Gleixner
2017-05-26  8:40   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 18/32] x86/perf: Drop EXPORT of perf_check_microcode Thomas Gleixner
2017-05-26  8:41   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 19/32] perf/x86/intel: Drop get_online_cpus() in intel_snb_check_microcode() Thomas Gleixner
2017-05-26  8:41   ` [tip:smp/hotplug] " tip-bot for Sebastian Andrzej Siewior
2017-05-24  8:15 ` [patch V3 20/32] PCI: Use cpu_hotplug_disable() instead of get_online_cpus() Thomas Gleixner
2017-05-26  8:42   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 21/32] PCI: Replace the racy recursion prevention Thomas Gleixner
2017-05-26  8:42   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 22/32] ACPI/processor: Use cpu_hotplug_disable() instead of get_online_cpus() Thomas Gleixner
2017-05-26  8:43   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 23/32] perf/tracing/cpuhotplug: Fix locking order Thomas Gleixner
2017-05-24 18:30   ` Paul E. McKenney
2017-05-24 18:47     ` Thomas Gleixner
2017-05-24 21:10       ` Paul E. McKenney
2017-05-30 11:22     ` Peter Zijlstra
2017-05-30 16:25       ` Paul E. McKenney
2017-05-26  8:43   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 24/32] jump_label: Reorder hotplug lock and jump_label_lock Thomas Gleixner
2017-05-24 12:50   ` David Miller
2017-05-26  8:44   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 25/32] kprobes: Cure hotplug lock ordering issues Thomas Gleixner
2017-05-24 15:54   ` Masami Hiramatsu
2017-05-26  7:47     ` Thomas Gleixner
2017-05-26  8:45   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 26/32] arm64: Prevent cpu hotplug rwsem recursion Thomas Gleixner
2017-05-26  8:45   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 27/32] arm: Prevent " Thomas Gleixner
2017-05-26  8:46   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 28/32] s390: " Thomas Gleixner
2017-05-24 10:57   ` Heiko Carstens
2017-05-26  8:46   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 29/32] cpu/hotplug: Convert hotplug locking to percpu rwsem Thomas Gleixner
2017-05-26  8:47   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 30/32] sched: Provide is_percpu_thread() helper Thomas Gleixner
2017-05-26  8:47   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 31/32] acpi/processor: Prevent cpu hotplug deadlock Thomas Gleixner
2017-05-26  8:48   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24  8:15 ` [patch V3 32/32] cpuhotplug: Link lock stacks for hotplug callbacks Thomas Gleixner
2017-05-26  8:48   ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner
2017-05-24 16:22 ` [patch V3 00/32] cpu/hotplug: Convert get_online_cpus() to a percpu_rwsem Paul E. McKenney
2017-05-26  7:03 ` Ingo Molnar
  -- strict thread matches above, loose matches on Subject: below --
2017-04-18 17:04 [patch V2 17/24] PCI: Use cpu_hotplug_disable() instead of get_online_cpus() Thomas Gleixner
2017-04-20 11:27 ` [tip:smp/hotplug] " tip-bot for Thomas Gleixner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).