All of lore.kernel.org
 help / color / mirror / Atom feed
* WARNING: possible circular locking dependency detected
@ 2017-08-25 10:03 Borislav Petkov
  2017-08-25 11:45 ` Borislav Petkov
                   ` (3 more replies)
  0 siblings, 4 replies; 24+ messages in thread
From: Borislav Petkov @ 2017-08-25 10:03 UTC (permalink / raw)
  To: Sebastian Andrzej Siewior; +Cc: Thomas Gleixner, Peter Zijlstra, lkml

Hey,

tglx says I have something for ya :-)

======================================================
WARNING: possible circular locking dependency detected
4.13.0-rc6+ #1 Not tainted
------------------------------------------------------
watchdog/3/27 is trying to acquire lock:
 (cpu_hotplug_lock.rw_sem){++++}, at: [<ffffffff8100c489>] release_ds_buffers+0x29/0xd0

but now in release context of a crosslock acquired at the following:
 ((complete)&self->parked){+.+.}, at: [<ffffffff810895f6>] kthread_park+0x46/0x60

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #2 ((complete)&self->parked){+.+.}:
       __lock_acquire+0x10af/0x1110
       lock_acquire+0xea/0x1f0
       wait_for_completion+0x3b/0x130
       kthread_park+0x46/0x60
       __smpboot_create_thread.part.5+0x7d/0xf0
       smpboot_register_percpu_thread_cpumask+0xa2/0x100
       spawn_ksoftirqd+0x3b/0x45
       do_one_initcall+0x52/0x198
       kernel_init_freeable+0x6f/0x1a1
       kernel_init+0xe/0x100
       ret_from_fork+0x2a/0x40

-> #1 (smpboot_threads_lock){+.+.}:
       __lock_acquire+0x10af/0x1110
       lock_acquire+0xea/0x1f0
       __mutex_lock+0x6c/0x940
       mutex_lock_nested+0x1b/0x20
       smpboot_register_percpu_thread_cpumask+0x42/0x100
       spawn_ksoftirqd+0x3b/0x45
       do_one_initcall+0x52/0x198
       kernel_init_freeable+0x6f/0x1a1
       kernel_init+0xe/0x100
       ret_from_fork+0x2a/0x40

-> #0 (cpu_hotplug_lock.rw_sem){++++}:
       cpus_read_lock+0x2a/0x90
       release_ds_buffers+0x29/0xd0
       x86_release_hardware+0x8f/0xa0
       hw_perf_event_destroy+0xe/0x20
       _free_event+0xa7/0x250

other info that might help us debug this:

Chain exists of:
  cpu_hotplug_lock.rw_sem --> smpboot_threads_lock --> (complete)&self->parked

 Possible unsafe locking scenario by crosslock:

       CPU0                    CPU1
       ----                    ----
  lock(smpboot_threads_lock);
  lock((complete)&self->parked);
                               lock(cpu_hotplug_lock.rw_sem);
                               unlock((complete)&self->parked);

 *** DEADLOCK ***

1 lock held by watchdog/3/27:
 #0:  (&x->wait){....}, at: [<ffffffff810b115d>] complete+0x1d/0x60

stack backtrace:
CPU: 3 PID: 27 Comm: watchdog/3 Not tainted 4.13.0-rc6+ #1
Hardware name: LENOVO 2320CTO/2320CTO, BIOS G2ET86WW (2.06 ) 11/13/2012
Call Trace:
 dump_stack+0x86/0xcf
 print_circular_bug+0x1fa/0x2f0
 check_prev_add+0x3be/0x700
 ? __lock_acquire+0x4c6/0x1110
 ? trace_event_raw_event_lock+0xf0/0xf0
 lock_commit_crosslock+0x40d/0x590
 ? lock_commit_crosslock+0x40d/0x590
 complete+0x29/0x60
 __kthread_parkme+0x54/0x80
 kthread_parkme+0x24/0x40
 smpboot_thread_fn+0x95/0x230
 kthread+0x147/0x180
 ? sort_range+0x30/0x30
 ? kthread_create_on_node+0x40/0x40
 ret_from_fork+0x2a/0x40


-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.

^ permalink raw reply	[flat|nested] 24+ messages in thread
* WARNING: possible circular locking dependency detected
@ 2018-11-14  2:41 Qian Cai
  0 siblings, 0 replies; 24+ messages in thread
From: Qian Cai @ 2018-11-14  2:41 UTC (permalink / raw)
  To: linux kernel; +Cc: Greg Kroah-Hartman, Jiri Slaby, Russell King

Compiling kernel on an aarch64 server with the latest mainline (rc2) generated this,

[  910.263839] WARNING: possible circular locking dependency detected
[  910.263841] 4.20.0-rc2+ #4 Tainted: G        W    L   
[  910.263843] ------------------------------------------------------
[  910.263844] swapper/0/0 is trying to acquire lock:
+0x3f0/0x8806] 0000000071c616d1 (console_owner){-.-.}, at: console_unlock+0x26cc481d4 (&pool->ldy depends on therse order) is:
[ 0.263880]        __queue_work+0x1dc/0xb78
[  910.263882]        queue_work_on+0x140/0x158
[  910.263883]        tty_flip_buffer_push+0xd8/0x108
[  910.263885]        pty_write+0xf8/0x100
[  910.263886]        n_tty_write+0x2d8/0x698
[  910.263888]        tty_write+0x254/0x410
[  910.263889]        __vfs_write+0xd4/0x330
[  910.263891]        vfs_write+0xd8/0x240
[  910.263892]        redirected_tty_write+0x74/0xd0
[  910.263894]        do_iter_write+0x224/0x270
[  910.263895]        vfs_writev+0x114/0x1d8
[  910.263897]        do_writev+0xd8/0x1e0
[  910.263898]        __arm64_sys_writev+0x50/0x60
[  910.263900]        el0_svc_handler+0xd4/0x198
[  910.263901]        el0_svc+0x8/0xc
[  910.263902] 
[  910.263903] -> #2 (&(&port->lock)->rlock){-.-.}:
[  910.263908]        __lock_acquire+0x3f0/0x880
[  910.263910]        lock_acquire+0x1a8/0x430
[  910.263911]        _raw_spin_lock_irqsave+0x74/0x98
[  910.263913]        tty_port_tty_get+0x28/0x68
[  910.263915]        tty_port_default_wakeup+0x20/0x40
[  910.263916]        tty_port_tty_wakeup+0x38/0x48
[  910.263918]        uart_write_wakeup+0x2c/0x40
[  910.263919]        pl011_tx_chars+0x2a4/0x358
[  910.263921]        pl011_start_tx_pio+0x24/0xa8
[  910.263922]        pl011_start_tx+0x150/0x2c0
[  910.263924]        __uart_start.isra.1+0x84/0x90
[  910.263925]        uart_write+0x1ec/0x2d8
[  910.263927]        do_output_char+0x1c4/0x2a8
[  910.263928]        n_tty_write+0x338/0x698
[  910.263930]        tty_write+0x254/0x410
[  910.263931]        redirected_tty_write+0xb4/0xd0
[  910.263933]        do_iter_write+0x224/0x270
[  910.263934]        vfs_writev+0x114/0x1d8
[  910.263935]        do_writev+0xd8/0x1e0
[  910.263937]        __arm64_sys_writev+0x50/0x60
[  910.263939]        el0_svc_handler+0xd4/0x198
[  910.263940]        el0_svc+0x8/0xc
[  910.263941] 
[  910.263942] -> #1 (&port_lock_key){-.-.}:
[  910.263947]        __lock_acquire+0x3f0/0x880
[  910.263949]        lock_acquire+0x1a8/0x430
[  910.263950]        _raw_spin_lock+0x68/0x88
[  910.263952]        pl011_console_write+0x1d8/0x258
[  910.263953]        console_unlock+0x3ac/0x938
[  910.263955]        register_console+0x360/0x4c8
[  910.263956]        uart_add_one_port+0x798/0x870
[  910.263958]        pl011_register_port+0x98/0x128
[  910.263959]        sbsa_uart_probe+0x320/0x3b0
[  910.263961]        platform_drv_probe+0x74/0xd0
[  910.263962]        really_probe+0x424/0x678
[  910.263964]        driver_probe_device+0x130/0x1a0
[  910.263966]        __device_attach_driver+0x10c/0x188
[  910.263967]        bus_for_each_drv+0xf8/0x158
[  910.263969]        __device_attach+0x188/0x230
[  910.263970]        device_initial_probe+0x24/0x30
[  910.263972]        bus_probe_device+0xf0/0x100
[  910.263973]        device_add+0x57c/0x9d8
[  910.263975]        platform_device_add+0x198/0x388
[  910.263977]        platform_device_register_full+0x16c/0x1e8
[  910.263978]        acpi_create_platform_device+0x368/0x450
[  910.263980]        acpi_default_enumeration+0x34/0x78
[  910.263982]        acpi_bus_attach+0x2dc/0x3e0
[  910.263983]        acpi_bus_attach+0x108/0x3e0
[  910.263985]        acpi_bus_attach+0x108/0x3e0
[  910.263986]        acpi_bus_attach+0x108/0x3e0
[  910.263987]        acpi_bus_scan+0xb0/0x110
[  910.263989]        acpi_scan_init+0x1a8/0x410
[  910.263990]        acpi_init+0x3e8/0x47c
[  910.263992]        do_one_initcall+0x160/0x6bc
[  910.263993]        kernel_init_freeable+0x74c/0x814
[  910.263995]        kernel_init+0x18/0x11c
[  910.263996]        ret_from_fork+0x10/0x1c
[  910.263997] 
[  910.263998] -> #0 (console_owner){-.-.}:
[  910.264004]        validate_chain.isra.17+0x998/0x1550
[  910.264005]        __lock_acquire+0x3f0/0x880
[  910.264007]        lock_acquire+0x1a8/0x430
[  910.264008]        console_unlock+0x2c8/0x938
[  910.264010]        vprintk_emit+0x358/0x5c0
[  910.264011]        vprintk_default+0x60/0xa8
[  910.264013]        vprintk_func+0x244/0x370
[  910.264014]        printk+0xc8/0xf8
[  910.264016]        show_workqueue_state+0x534/0xce0
[  910.264018]        wq_watchdog_timer_fn+0x444/0x518
[  910.264019]        call_timer_fn+0x1c4/0x708
[  910.264021]        expire_timers+0x32c/0x448
[  910.264022]        run_timer_softirq+0x290/0x328
[  910.264024]        __do_softirq+0x248/0x9c8
[  910.264025]        irq_exit+0x25c/0x2f0
[  910.264027]        __handle_domain_irq+0x8c/0xf0
[  910.264028]        gic_handle_irq+0xc0/0x168
[  910.264029]        el1_irq+0xb4/0x140
[  910.264031]        cpuidle_enter_state+0x150/0x6a0
[  910.264032]        cpuidle_enter+0x34/0x48
[  910.264034]        call_cpuidle+0x4c/0x78
[  910.264035]        do_idle+0x328/0x360
[  910.264037]        cpu_startup_entry+0x2c/0x30
[  910.264038]        rest_init+0x248/0x31c
[  910.264040]        arch_call_rest_init+0x14/0x1c
[  910.264041]        start_kernel+0x568/0x5a8
[  910.264042] 
[  910.264044] other info that might help us debug this:
[  910.264045] 
[  910.264046] Chain exists of:
[  910.264047]   console_owner --> &(&port->lock)->rlock --> &pool->lock/1
[  910.264055] 
[  910.264057]  Possible unsafe locking scenario:
[  910.264058] 
[  910.264059]        CPU0                    CPU1
[  910.264061]        ----                    ----
[  910.264062]   lock(&pool->lock/1);
[  910.264066]                                lock(&(&port->lock)->rlock);
[  910.264070]                                lock(&pool->lock/1);
[  910.264073]   lock(console_owner);
[  910.264076] 
[  910.264078]  *** DEADLOCK ***
[  910.264079] 
[  910.264080] 4 locks held by swapper/0/0:
[  910.264081]  #0: 00000000c45a46a0 ((&wq_watchdog_timer)){+.-.}, at: call_timer_fn+0x0/0x708
[  910.264087]  #1: 00000000f6706545 (rcu_read_lock_sched){....}, at: show_workqueue_state+0x0/0xce0
[  910.264094]  #2: 00000000f2c481d4 (&pool->lock/1){-.-.}, at: show_workqueue_state+0x468/0xce0
[  910.264101]  #3: 0000000014da44b5 (console_lock){+.+.}, at: vprintk_emit+0x350/0x5c0
[  910.264107] 
[  910.264108] stack backtrace:
[  910.264110] CPU: 0 PID: 0 Comm: swapper/0 Kdump: loaded Tainted: G        W    L    4.20.0-rc2+ #4
[  910.264113] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.0.6 07/10/2018
[  910.264114] Call trace:
[  910.264116]  dump_backtrace+0x0/0x2c8
[  910.264117]  show_stack+0x24/0x30
[  910.264118]  dump_stack+0x118/0x19c
[  910.264120]  print_circular_bug.isra.13+0x360/0x408
[  910.264121]  check_prev_add.constprop.22+0x690/0xa10
[  910.264123]  validate_chain.isra.17+0x998/0x1550
[  910.264124]  __lock_acquire+0x3f0/0x880
[  910.264126]  lock_acquire+0x1a8/0x430
[  910.264127]  console_unlock+0x2c8/0x938
[  910.264129]  vprintk_emit+0x358/0x5c0
[  910.264130]  vprintk_default+0x60/0xa8
[  910.264131]  vprintk_func+0x244/0x370
[  910.264133]  printk+0xc8/0xf8
[  910.264134]  show_workqueue_state+0x534/0xce0
[  910.264136]  wq_watchdog_timer_fn+0x444/0x518
[  910.264137]  call_timer_fn+0x1c4/0x708
[  910.264138]  expire_timers+0x32c/0x448
[  910.264140]  run_timer_softirq+0x290/0x328
[  910.264141]  __do_softirq+0x248/0x9c8
[  910.264143]  irq_exit+0x25c/0x2f0
[  910.264144]  __handle_domain_irq+0x8c/0xf0
[  910.264146]  gic_handle_irq+0xc0/0x168
[  910.264147]  el1_irq+0xb4/0x140
[  910.264148]  cpuidle_enter_state+0x150/0x6a0
[  910.264150]  cpuidle_enter+0x34/0x48
[  910.264151]  call_cpuidle+0x4c/0x78
[  910.264152]  do_idle+0x328/0x360
[  910.264154]  cpu_startup_entry+0x2c/0x30
[  910.264155]  rest_init+0x248/0x31c
[  910.264157]  arch_call_rest_init+0x14/0x1c
[  910.264158]  start_kernel+0x568/0x5a8
[  910.327979] ------------[ cut here ]------------

^ permalink raw reply	[flat|nested] 24+ messages in thread

end of thread, other threads:[~2018-11-14  2:44 UTC | newest]

Thread overview: 24+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-25 10:03 WARNING: possible circular locking dependency detected Borislav Petkov
2017-08-25 11:45 ` Borislav Petkov
2017-08-25 14:47 ` Sebastian Andrzej Siewior
2017-08-25 16:12   ` Byungchul Park
2017-08-25 16:21     ` Thomas Gleixner
2017-08-28  7:41   ` Peter Zijlstra
2017-08-28 14:11   ` Peter Zijlstra
2017-08-29 19:34   ` Peter Zijlstra
2017-08-25 16:42 ` Sebastian Andrzej Siewior
2017-08-28 14:58 ` Peter Zijlstra
2017-08-28 15:06   ` Peter Zijlstra
2017-08-28 16:32     ` Peter Zijlstra
2017-08-29 17:40   ` Thomas Gleixner
2017-08-29 19:49     ` Peter Zijlstra
2017-08-29 20:10       ` Thomas Gleixner
2017-08-30  5:47         ` Peter Zijlstra
2017-08-31  7:08           ` Thomas Gleixner
2017-08-31  7:37             ` Peter Zijlstra
2017-08-31  7:55               ` Thomas Gleixner
2017-08-31  8:09                 ` Peter Zijlstra
2017-08-31  8:15                   ` Thomas Gleixner
2017-08-31 21:24                     ` Thomas Gleixner
2017-09-01 20:32                       ` Peter Zijlstra
2018-11-14  2:41 Qian Cai

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.