[ 74.182295] WARNING: possible irq lock inversion dependency detected [ 74.182296] 4.16.0-11766-ge241e3f2bf97 #166 Not tainted [ 74.182297] -------------------------------------------------------- [ 74.182298] kworker/u64:7/175 just changed the state of lock: [ 74.182301] 000000001b8db3f6 (&rq->gstate_seq){+.+.}, at: scsi_queue_rq+0x460/0x578 [ 74.182308] but this lock was taken by another, SOFTIRQ-safe lock in the past: [ 74.182310] (&(&dq->lock)->rlock){..-.} [ 74.182311] and interrupts could create inverse lock ordering between them. [ 74.182315] other info that might help us debug this: [ 74.182316] Possible interrupt unsafe locking scenario: [ 74.182318] CPU0 CPU1 [ 74.182319] ---- ---- [ 74.182320] lock(&rq->gstate_seq); [ 74.182340] local_irq_disable(); [ 74.182341] lock(&(&dq->lock)->rlock); [ 74.182343] lock(&rq->gstate_seq); [ 74.182345] [ 74.182346] lock(&(&dq->lock)->rlock); [ 74.182348] *** DEADLOCK *** [ 74.182350] 4 locks held by kworker/u64:7/175: [ 74.182351] #0: 000000001751423f ((wq_completion)"%s"shost->work_q_name){+.+.}, at: process_one_work+0x1a0/0x6f0 [ 74.182359] #1: 000000001c880b65 ((work_completion)(&rport->scan_work)){+.+.}, at: process_one_work+0x1a0/0x6f0 [ 74.182363] #2: 000000000c5e0f4d (&shost->scan_mutex){+.+.}, at: scsi_scan_target+0xac/0x128 [ 74.182368] #3: 00000000f2c11806 (rcu_read_lock){....}, at: hctx_lock+0x30/0x110 [ 74.182374] the shortest dependencies between 2nd lock and 1st lock: [ 74.182376] -> (&(&dq->lock)->rlock){..-.} ops: 51556787421184 { [ 74.182380] IN-SOFTIRQ-W at: [ 74.182384] __lock_acquire+0x4e6/0x13f0 [ 74.182385] lock_acquire+0xf6/0x290 [ 74.182390] _raw_spin_lock_irq+0x6a/0xb0 [ 74.182392] dasd_block_tasklet+0xf8/0x400 [ 74.182394] tasklet_action_common.isra.5+0x82/0x110 [ 74.182396] __do_softirq+0xd2/0x5d8 [ 74.182398] irq_exit+0x132/0x138 [ 74.182400] do_IRQ+0xaa/0xf0 [ 74.182402] io_int_handler+0x138/0x31c [ 74.182405] enabled_wait+0x6a/0x138 [ 74.182406] enabled_wait+0x52/0x138 [ 74.182408] arch_cpu_idle+0x32/0x50 [ 74.182410] default_idle_call+0x3e/0x68 [ 74.182413] do_idle+0xea/0x1c0 [ 74.182415] cpu_startup_entry+0x3e/0x48 [ 74.182417] smp_start_secondary+0x13e/0x158 [ 74.182419] restart_int_handler+0x5a/0x70 [ 74.182420] (null) [ 74.182422] INITIAL USE at: [ 74.182424] __lock_acquire+0x26c/0x13f0 [ 74.182426] lock_acquire+0xf6/0x290 [ 74.182428] _raw_spin_lock_irq+0x6a/0xb0 [ 74.182432] do_dasd_request+0x50/0x470 [ 74.182434] __blk_mq_try_issue_directly+0x1ca/0x208 [ 74.182436] blk_mq_try_issue_directly+0x5c/0x108 [ 74.182437] blk_mq_make_request+0x4b8/0xa20 [ 74.182440] generic_make_request+0xea/0x2c8 [ 74.182441] submit_bio+0xa4/0x1a0 [ 74.182444] submit_bh_wbc+0x1c4/0x218 [ 74.182446] block_read_full_page+0x366/0x408 [ 74.182450] do_read_cache_page+0x1b2/0x3d0 [ 74.182452] read_cache_page+0x30/0x40 [ 74.182455] read_dev_sector+0x58/0xe8 [ 74.182457] read_lba.isra.0+0x12a/0x1b8 [ 74.182459] efi_partition+0x198/0x660 [ 74.182461] check_partition+0x15c/0x2b8 [ 74.182463] rescan_partitions+0xea/0x3f8 [ 74.182464] blkdev_reread_part+0x44/0x68 [ 74.182467] dasd_scan_partitions+0x90/0x148 [ 74.182469] dasd_change_state+0x8cc/0xb28 [ 74.182471] do_kick_device+0x4c/0x98 [ 74.182473] process_one_work+0x25c/0x6f0 [ 74.182475] worker_thread+0x52/0x400 [ 74.182477] kthread+0x15a/0x178 [ 74.182479] kernel_thread_starter+0x6/0xc [ 74.182481] kernel_thread_starter+0x0/0xc [ 74.182482] } [ 74.182486] ... key at: [<0000000001bd0e10>] __key.43387+0x0/0x8 [ 74.182487] ... acquired at: [ 74.182489] __lock_acquire+0x130a/0x13f0 [ 74.182490] lock_acquire+0xf6/0x290 [ 74.182492] blk_mq_start_request+0xde/0x300 [ 74.182494] do_dasd_request+0x17a/0x470 [ 74.182495] __blk_mq_try_issue_directly+0x1ca/0x208 [ 74.182497] blk_mq_try_issue_directly+0x5c/0x108 [ 74.182499] blk_mq_make_request+0x4b8/0xa20 [ 74.182500] generic_make_request+0xea/0x2c8 [ 74.182502] submit_bio+0xa4/0x1a0 [ 74.182504] submit_bh_wbc+0x1c4/0x218 [ 74.182506] block_read_full_page+0x366/0x408 [ 74.182507] do_read_cache_page+0x1b2/0x3d0 [ 74.182509] read_cache_page+0x30/0x40 [ 74.182512] read_dev_sector+0x58/0xe8 [ 74.182513] read_lba.isra.0+0x12a/0x1b8 [ 74.182515] efi_partition+0x198/0x660 [ 74.182516] check_partition+0x15c/0x2b8 [ 74.182517] rescan_partitions+0xea/0x3f8 [ 74.182519] blkdev_reread_part+0x44/0x68 [ 74.182520] dasd_scan_partitions+0x90/0x148 [ 74.182522] dasd_change_state+0x8cc/0xb28 [ 74.182524] do_kick_device+0x4c/0x98 [ 74.182526] process_one_work+0x25c/0x6f0 [ 74.182527] worker_thread+0x52/0x400 [ 74.182529] kthread+0x15a/0x178 [ 74.182530] kernel_thread_starter+0x6/0xc [ 74.182532] kernel_thread_starter+0x0/0xc [ 74.182534] -> (&rq->gstate_seq){+.+.} ops: 24322399797248 { [ 74.182538] HARDIRQ-ON-W at: [ 74.182540] __lock_acquire+0x3da/0x13f0 [ 74.182542] lock_acquire+0xf6/0x290 [ 74.182543] blk_mq_start_request+0xde/0x300 [ 74.182546] scsi_queue_rq+0x460/0x578 [ 74.182547] blk_mq_dispatch_rq_list+0x186/0x678 [ 74.182549] blk_mq_do_dispatch_sched+0x90/0x128 [ 74.182551] blk_mq_sched_dispatch_requests+0x168/0x200 [ 74.182553] __blk_mq_run_hw_queue+0xd0/0x128 [ 74.182555] __blk_mq_delay_run_hw_queue+0xd4/0x120 [ 74.182556] blk_mq_run_hw_queue+0xa0/0x118 [ 74.182558] blk_mq_sched_insert_request+0xc6/0x1e0 [ 74.182561] blk_execute_rq_nowait+0x8e/0x118 [ 74.182563] blk_execute_rq+0x76/0x90 [ 74.182564] scsi_execute+0x114/0x218 [ 74.182566] scsi_probe_and_add_lun+0x23e/0xe60 [ 74.182568] __scsi_scan_target+0x112/0x630 [ 74.182570] scsi_scan_target+0x110/0x128 [ 74.182573] fc_scsi_scan_rport+0xd8/0xe0 [ 74.182575] process_one_work+0x25c/0x6f0 [ 74.182577] worker_thread+0x52/0x400 [ 74.182579] kthread+0x15a/0x178 [ 74.182580] kernel_thread_starter+0x6/0xc [ 74.182582] kernel_thread_starter+0x0/0xc [ 74.182584] SOFTIRQ-ON-W at: [ 74.182586] __lock_acquire+0x3fc/0x13f0 [ 74.182588] lock_acquire+0xf6/0x290 [ 74.182589] blk_mq_start_request+0xde/0x300 [ 74.182591] scsi_queue_rq+0x460/0x578 [ 74.182593] blk_mq_dispatch_rq_list+0x186/0x678 [ 74.182595] blk_mq_do_dispatch_sched+0x90/0x128 [ 74.182597] blk_mq_sched_dispatch_requests+0x168/0x200 [ 74.182599] __blk_mq_run_hw_queue+0xd0/0x128 [ 74.182600] __blk_mq_delay_run_hw_queue+0xd4/0x120 [ 74.182602] blk_mq_run_hw_queue+0xa0/0x118 [ 74.182603] blk_mq_sched_insert_request+0xc6/0x1e0 [ 74.182605] blk_execute_rq_nowait+0x8e/0x118 [ 74.182607] blk_execute_rq+0x76/0x90 [ 74.182609] scsi_execute+0x114/0x218 [ 74.182611] scsi_probe_and_add_lun+0x23e/0xe60 [ 74.182613] __scsi_scan_target+0x112/0x630 [ 74.182615] scsi_scan_target+0x110/0x128 [ 74.182617] fc_scsi_scan_rport+0xd8/0xe0 [ 74.182618] process_one_work+0x25c/0x6f0 [ 74.182620] worker_thread+0x52/0x400 [ 74.182622] kthread+0x15a/0x178 [ 74.182623] kernel_thread_starter+0x6/0xc [ 74.182625] kernel_thread_starter+0x0/0xc [ 74.182626] INITIAL USE at: [ 74.182629] __lock_acquire+0x26c/0x13f0 [ 74.182631] lock_acquire+0xf6/0x290 [ 74.182632] blk_mq_start_request+0xde/0x300 [ 74.182634] do_dasd_request+0x17a/0x470 [ 74.182636] __blk_mq_try_issue_directly+0x1ca/0x208 [ 74.182638] blk_mq_try_issue_directly+0x5c/0x108 [ 74.182640] blk_mq_make_request+0x4b8/0xa20 [ 74.182641] generic_make_request+0xea/0x2c8 [ 74.182643] submit_bio+0xa4/0x1a0 [ 74.182645] submit_bh_wbc+0x1c4/0x218 [ 74.182646] block_read_full_page+0x366/0x408 [ 74.182648] do_read_cache_page+0x1b2/0x3d0 [ 74.182650] read_cache_page+0x30/0x40 [ 74.182652] read_dev_sector+0x58/0xe8 [ 74.182654] read_lba.isra.0+0x12a/0x1b8 [ 74.182655] efi_partition+0x198/0x660 [ 74.182657] check_partition+0x15c/0x2b8 [ 74.182659] rescan_partitions+0xea/0x3f8 [ 74.182660] blkdev_reread_part+0x44/0x68 [ 74.182662] dasd_scan_partitions+0x90/0x148 [ 74.182664] dasd_change_state+0x8cc/0xb28 [ 74.182665] do_kick_device+0x4c/0x98 [ 74.182667] process_one_work+0x25c/0x6f0 [ 74.182669] worker_thread+0x52/0x400 [ 74.182670] kthread+0x15a/0x178 [ 74.182672] kernel_thread_starter+0x6/0xc [ 74.182674] kernel_thread_starter+0x0/0xc [ 74.182675] } [ 74.182677] ... key at: [<0000000001bc22f0>] __key.43630+0x0/0x8 [ 74.182678] ... acquired at: [ 74.182680] check_usage_backwards+0xa2/0x140 [ 74.182681] mark_lock+0x3ee/0x6f8 [ 74.182682] __lock_acquire+0x3fc/0x13f0 [ 74.182684] lock_acquire+0xf6/0x290 [ 74.182685] blk_mq_start_request+0xde/0x300 [ 74.182687] scsi_queue_rq+0x460/0x578 [ 74.182688] blk_mq_dispatch_rq_list+0x186/0x678 [ 74.182690] blk_mq_do_dispatch_sched+0x90/0x128 [ 74.182692] blk_mq_sched_dispatch_requests+0x168/0x200 [ 74.182693] __blk_mq_run_hw_queue+0xd0/0x128 [ 74.182695] __blk_mq_delay_run_hw_queue+0xd4/0x120 [ 74.182696] blk_mq_run_hw_queue+0xa0/0x118 [ 74.182698] blk_mq_sched_insert_request+0xc6/0x1e0 [ 74.182699] blk_execute_rq_nowait+0x8e/0x118 [ 74.182701] blk_execute_rq+0x76/0x90 [ 74.182703] scsi_execute+0x114/0x218 [ 74.182704] scsi_probe_and_add_lun+0x23e/0xe60 [ 74.182706] __scsi_scan_target+0x112/0x630 [ 74.182708] scsi_scan_target+0x110/0x128 [ 74.182709] fc_scsi_scan_rport+0xd8/0xe0 [ 74.182711] process_one_work+0x25c/0x6f0 [ 74.182713] worker_thread+0x52/0x400 [ 74.182714] kthread+0x15a/0x178 [ 74.182716] kernel_thread_starter+0x6/0xc [ 74.182717] kernel_thread_starter+0x0/0xc [ 74.182719] stack backtrace: [ 74.182722] CPU: 0 PID: 175 Comm: kworker/u64:7 Not tainted 4.16.0-11766-ge241e3f2bf97 #166 [ 74.182724] Hardware name: IBM 3906 M04 704 (LPAR) [ 74.182726] Workqueue: scsi_wq_0 fc_scsi_scan_rport [ 74.182727] Call Trace: [ 74.182730] ([<00000000001151d4>] show_stack+0x9c/0x108) [ 74.182733] [<00000000008c3c9a>] dump_stack+0x9a/0xd8 [ 74.182735] [<00000000001b8f02>] print_irq_inversion_bug+0x1ba/0x230 [ 74.182737] [<00000000001b915a>] check_usage_backwards+0xa2/0x140 [ 74.182739] [<00000000001ba7e6>] mark_lock+0x3ee/0x6f8 [ 74.182740] [<00000000001bb524>] __lock_acquire+0x3fc/0x13f0 [ 74.182742] [<00000000001bcd8e>] lock_acquire+0xf6/0x290 [ 74.182744] [<000000000058239e>] blk_mq_start_request+0xde/0x300 [ 74.182745] [<0000000000671898>] scsi_queue_rq+0x460/0x578 [ 74.182747] [<00000000005856ae>] blk_mq_dispatch_rq_list+0x186/0x678 [ 74.182749] [<000000000058b0b8>] blk_mq_do_dispatch_sched+0x90/0x128 [ 74.182751] [<000000000058ba90>] blk_mq_sched_dispatch_requests+0x168/0x200 [ 74.182752] [<00000000005839b8>] __blk_mq_run_hw_queue+0xd0/0x128 [ 74.182754] [<0000000000583ae4>] __blk_mq_delay_run_hw_queue+0xd4/0x120 [ 74.182756] [<0000000000583c28>] blk_mq_run_hw_queue+0xa0/0x118 [ 74.182758] [<000000000058c0ae>] blk_mq_sched_insert_request+0xc6/0x1e0 [ 74.182759] [<000000000057d186>] blk_execute_rq_nowait+0x8e/0x118 [ 74.182761] [<000000000057d2a6>] blk_execute_rq+0x76/0x90 [ 74.182763] [<000000000066c7fc>] scsi_execute+0x114/0x218 [ 74.182765] [<00000000006735f6>] scsi_probe_and_add_lun+0x23e/0xe60 [ 74.182766] [<0000000000674722>] __scsi_scan_target+0x112/0x630 [ 74.182768] [<0000000000674d50>] scsi_scan_target+0x110/0x128 [ 74.182770] [<000000000067f730>] fc_scsi_scan_rport+0xd8/0xe0 [ 74.182772] [<000000000016ace4>] process_one_work+0x25c/0x6f0 [ 74.182774] [<000000000016b1ca>] worker_thread+0x52/0x400 [ 74.182775] [<00000000001730ba>] kthread+0x15a/0x178 [ 74.182777] [<00000000008e66c2>] kernel_thread_starter+0x6/0xc [ 74.182779] [<00000000008e66bc>] kernel_thread_starter+0x0/0xc [ 74.182780] INFO: lockdep is turned off.