From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Guoqing Jiang <guoqing.jiang@cloud.ionos.com>,
Song Liu <songliubraving@fb.com>, Sasha Levin <sashal@kernel.org>,
linux-raid@vger.kernel.org
Subject: [PATCH AUTOSEL 4.4 25/37] md: don't flush workqueue unconditionally in md_open
Date: Mon, 8 Jun 2020 19:27:37 -0400 [thread overview]
Message-ID: <20200608232750.3370747-25-sashal@kernel.org> (raw)
In-Reply-To: <20200608232750.3370747-1-sashal@kernel.org>
From: Guoqing Jiang <guoqing.jiang@cloud.ionos.com>
[ Upstream commit f6766ff6afff70e2aaf39e1511e16d471de7c3ae ]
We need to check mddev->del_work before flush workqueu since the purpose
of flush is to ensure the previous md is disappeared. Otherwise the similar
deadlock appeared if LOCKDEP is enabled, it is due to md_open holds the
bdev->bd_mutex before flush workqueue.
kernel: [ 154.522645] ======================================================
kernel: [ 154.522647] WARNING: possible circular locking dependency detected
kernel: [ 154.522650] 5.6.0-rc7-lp151.27-default #25 Tainted: G O
kernel: [ 154.522651] ------------------------------------------------------
kernel: [ 154.522653] mdadm/2482 is trying to acquire lock:
kernel: [ 154.522655] ffff888078529128 ((wq_completion)md_misc){+.+.}, at: flush_workqueue+0x84/0x4b0
kernel: [ 154.522673]
kernel: [ 154.522673] but task is already holding lock:
kernel: [ 154.522675] ffff88804efa9338 (&bdev->bd_mutex){+.+.}, at: __blkdev_get+0x79/0x590
kernel: [ 154.522691]
kernel: [ 154.522691] which lock already depends on the new lock.
kernel: [ 154.522691]
kernel: [ 154.522694]
kernel: [ 154.522694] the existing dependency chain (in reverse order) is:
kernel: [ 154.522696]
kernel: [ 154.522696] -> #4 (&bdev->bd_mutex){+.+.}:
kernel: [ 154.522704] __mutex_lock+0x87/0x950
kernel: [ 154.522706] __blkdev_get+0x79/0x590
kernel: [ 154.522708] blkdev_get+0x65/0x140
kernel: [ 154.522709] blkdev_get_by_dev+0x2f/0x40
kernel: [ 154.522716] lock_rdev+0x3d/0x90 [md_mod]
kernel: [ 154.522719] md_import_device+0xd6/0x1b0 [md_mod]
kernel: [ 154.522723] new_dev_store+0x15e/0x210 [md_mod]
kernel: [ 154.522728] md_attr_store+0x7a/0xc0 [md_mod]
kernel: [ 154.522732] kernfs_fop_write+0x117/0x1b0
kernel: [ 154.522735] vfs_write+0xad/0x1a0
kernel: [ 154.522737] ksys_write+0xa4/0xe0
kernel: [ 154.522745] do_syscall_64+0x64/0x2b0
kernel: [ 154.522748] entry_SYSCALL_64_after_hwframe+0x49/0xbe
kernel: [ 154.522749]
kernel: [ 154.522749] -> #3 (&mddev->reconfig_mutex){+.+.}:
kernel: [ 154.522752] __mutex_lock+0x87/0x950
kernel: [ 154.522756] new_dev_store+0xc9/0x210 [md_mod]
kernel: [ 154.522759] md_attr_store+0x7a/0xc0 [md_mod]
kernel: [ 154.522761] kernfs_fop_write+0x117/0x1b0
kernel: [ 154.522763] vfs_write+0xad/0x1a0
kernel: [ 154.522765] ksys_write+0xa4/0xe0
kernel: [ 154.522767] do_syscall_64+0x64/0x2b0
kernel: [ 154.522769] entry_SYSCALL_64_after_hwframe+0x49/0xbe
kernel: [ 154.522770]
kernel: [ 154.522770] -> #2 (kn->count#253){++++}:
kernel: [ 154.522775] __kernfs_remove+0x253/0x2c0
kernel: [ 154.522778] kernfs_remove+0x1f/0x30
kernel: [ 154.522780] kobject_del+0x28/0x60
kernel: [ 154.522783] mddev_delayed_delete+0x24/0x30 [md_mod]
kernel: [ 154.522786] process_one_work+0x2a7/0x5f0
kernel: [ 154.522788] worker_thread+0x2d/0x3d0
kernel: [ 154.522793] kthread+0x117/0x130
kernel: [ 154.522795] ret_from_fork+0x3a/0x50
kernel: [ 154.522796]
kernel: [ 154.522796] -> #1 ((work_completion)(&mddev->del_work)){+.+.}:
kernel: [ 154.522800] process_one_work+0x27e/0x5f0
kernel: [ 154.522802] worker_thread+0x2d/0x3d0
kernel: [ 154.522804] kthread+0x117/0x130
kernel: [ 154.522806] ret_from_fork+0x3a/0x50
kernel: [ 154.522807]
kernel: [ 154.522807] -> #0 ((wq_completion)md_misc){+.+.}:
kernel: [ 154.522813] __lock_acquire+0x1392/0x1690
kernel: [ 154.522816] lock_acquire+0xb4/0x1a0
kernel: [ 154.522818] flush_workqueue+0xab/0x4b0
kernel: [ 154.522821] md_open+0xb6/0xc0 [md_mod]
kernel: [ 154.522823] __blkdev_get+0xea/0x590
kernel: [ 154.522825] blkdev_get+0x65/0x140
kernel: [ 154.522828] do_dentry_open+0x1d1/0x380
kernel: [ 154.522831] path_openat+0x567/0xcc0
kernel: [ 154.522834] do_filp_open+0x9b/0x110
kernel: [ 154.522836] do_sys_openat2+0x201/0x2a0
kernel: [ 154.522838] do_sys_open+0x57/0x80
kernel: [ 154.522840] do_syscall_64+0x64/0x2b0
kernel: [ 154.522842] entry_SYSCALL_64_after_hwframe+0x49/0xbe
kernel: [ 154.522844]
kernel: [ 154.522844] other info that might help us debug this:
kernel: [ 154.522844]
kernel: [ 154.522846] Chain exists of:
kernel: [ 154.522846] (wq_completion)md_misc --> &mddev->reconfig_mutex --> &bdev->bd_mutex
kernel: [ 154.522846]
kernel: [ 154.522850] Possible unsafe locking scenario:
kernel: [ 154.522850]
kernel: [ 154.522852] CPU0 CPU1
kernel: [ 154.522853] ---- ----
kernel: [ 154.522854] lock(&bdev->bd_mutex);
kernel: [ 154.522856] lock(&mddev->reconfig_mutex);
kernel: [ 154.522858] lock(&bdev->bd_mutex);
kernel: [ 154.522860] lock((wq_completion)md_misc);
kernel: [ 154.522861]
kernel: [ 154.522861] *** DEADLOCK ***
kernel: [ 154.522861]
kernel: [ 154.522864] 1 lock held by mdadm/2482:
kernel: [ 154.522865] #0: ffff88804efa9338 (&bdev->bd_mutex){+.+.}, at: __blkdev_get+0x79/0x590
kernel: [ 154.522868]
kernel: [ 154.522868] stack backtrace:
kernel: [ 154.522873] CPU: 1 PID: 2482 Comm: mdadm Tainted: G O 5.6.0-rc7-lp151.27-default #25
kernel: [ 154.522875] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1ubuntu1 04/01/2014
kernel: [ 154.522878] Call Trace:
kernel: [ 154.522881] dump_stack+0x8f/0xcb
kernel: [ 154.522884] check_noncircular+0x194/0x1b0
kernel: [ 154.522888] ? __lock_acquire+0x1392/0x1690
kernel: [ 154.522890] __lock_acquire+0x1392/0x1690
kernel: [ 154.522893] lock_acquire+0xb4/0x1a0
kernel: [ 154.522895] ? flush_workqueue+0x84/0x4b0
kernel: [ 154.522898] flush_workqueue+0xab/0x4b0
kernel: [ 154.522900] ? flush_workqueue+0x84/0x4b0
kernel: [ 154.522905] ? md_open+0xb6/0xc0 [md_mod]
kernel: [ 154.522908] md_open+0xb6/0xc0 [md_mod]
kernel: [ 154.522910] __blkdev_get+0xea/0x590
kernel: [ 154.522912] ? bd_acquire+0xc0/0xc0
kernel: [ 154.522914] blkdev_get+0x65/0x140
kernel: [ 154.522916] ? bd_acquire+0xc0/0xc0
kernel: [ 154.522918] do_dentry_open+0x1d1/0x380
kernel: [ 154.522921] path_openat+0x567/0xcc0
kernel: [ 154.522923] ? __lock_acquire+0x380/0x1690
kernel: [ 154.522926] do_filp_open+0x9b/0x110
kernel: [ 154.522929] ? __alloc_fd+0xe5/0x1f0
kernel: [ 154.522935] ? kmem_cache_alloc+0x28c/0x630
kernel: [ 154.522939] ? do_sys_openat2+0x201/0x2a0
kernel: [ 154.522941] do_sys_openat2+0x201/0x2a0
kernel: [ 154.522944] do_sys_open+0x57/0x80
kernel: [ 154.522946] do_syscall_64+0x64/0x2b0
kernel: [ 154.522948] entry_SYSCALL_64_after_hwframe+0x49/0xbe
kernel: [ 154.522951] RIP: 0033:0x7f98d279d9ae
And md_alloc also flushed the same workqueue, but the thing is different
here. Because all the paths call md_alloc don't hold bdev->bd_mutex, and
the flush is necessary to avoid race condition, so leave it as it is.
Signed-off-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com>
Signed-off-by: Song Liu <songliubraving@fb.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/md/md.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/drivers/md/md.c b/drivers/md/md.c
index d59d79b77fd6..60161690e226 100644
--- a/drivers/md/md.c
+++ b/drivers/md/md.c
@@ -7038,7 +7038,8 @@ static int md_open(struct block_device *bdev, fmode_t mode)
*/
mddev_put(mddev);
/* Wait until bdev->bd_disk is definitely gone */
- flush_workqueue(md_misc_wq);
+ if (work_pending(&mddev->del_work))
+ flush_workqueue(md_misc_wq);
/* Then retry the open from the top */
return -ERESTARTSYS;
}
--
2.25.1
next prev parent reply other threads:[~2020-06-08 23:27 UTC|newest]
Thread overview: 45+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-08 23:27 [PATCH AUTOSEL 4.4 01/37] ath9x: Fix stack-out-of-bounds Write in ath9k_hif_usb_rx_cb Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 02/37] ath9k: Fix use-after-free Write in ath9k_htc_rx_msg Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 03/37] spi: dw: Zero DMA Tx and Rx configurations on stack Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 04/37] Bluetooth: Add SCO fallback for invalid LMP parameters error Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 05/37] kgdb: Prevent infinite recursive entries to the debugger Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 06/37] spi: dw: Enable interrupts in accordance with DMA xfer mode Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 07/37] clocksource: dw_apb_timer_of: Fix missing clockevent timers Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 08/37] btrfs: do not ignore error from btrfs_next_leaf() when inserting checksums Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 09/37] ARM: 8978/1: mm: make act_mm() respect THREAD_SIZE Sasha Levin
2020-06-08 23:27 ` Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 10/37] net: vmxnet3: fix possible buffer overflow caused by bad DMA value in vmxnet3_get_rss() Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 11/37] staging: android: ion: use vmap instead of vm_map_ram Sasha Levin
2020-06-08 23:27 ` Sasha Levin
2020-06-08 23:27 ` Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 12/37] ath9k: Fix use-after-free Read in ath9k_wmi_ctrl_rx Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 13/37] ath9k: Fix general protection fault in ath9k_hif_usb_rx_cb Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 14/37] e1000: Distribute switch variables for initialization Sasha Levin
2020-06-08 23:27 ` [Intel-wired-lan] " Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 15/37] media: dvb: return -EREMOTEIO on i2c transfer failure Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 16/37] MIPS: Make sparse_init() using top-down allocation Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 17/37] netfilter: nft_nat: return EOPNOTSUPP if type or flags are not supported Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 18/37] lib/mpi: Fix 64-bit MIPS build with Clang Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 19/37] net: lpc-enet: fix error return code in lpc_mii_init() Sasha Levin
2020-06-08 23:27 ` Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 20/37] net: allwinner: Fix use correct return type for ndo_start_xmit() Sasha Levin
2020-06-08 23:27 ` Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 21/37] powerpc/spufs: fix copy_to_user while atomic Sasha Levin
2020-06-08 23:27 ` Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 22/37] ath9k_htc: Silence undersized packet warnings Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 23/37] mips: cm: Fix an invalid error code of INTVN_*_ERR Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 24/37] kgdb: Fix spurious true from in_dbg_master() Sasha Levin
2020-06-08 23:27 ` Sasha Levin [this message]
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 26/37] mwifiex: Fix memory corruption in dump_station Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 27/37] mips: Add udelay lpj numbers adjustment Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 28/37] x86/mm: Stop printing BRK addresses Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 29/37] m68k: mac: Don't call via_flush_cache() on Mac IIfx Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 30/37] macvlan: Skip loopback packets in RX handler Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 31/37] PCI: Don't disable decoding when mmio_always_on is set Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 32/37] MIPS: Fix IRQ tracing when call handle_fpe() and handle_msa_fpe() Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 33/37] xfs: gut error handling in xfs_trans_unreserve_and_mod_sb() Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 34/37] ixgbe: fix signed-integer-overflow warning Sasha Levin
2020-06-08 23:27 ` [Intel-wired-lan] " Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 35/37] spi: dw: Return any value retrieved from the dma_transfer callback Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 36/37] cpuidle: Fix three reference count leaks Sasha Levin
2020-06-08 23:27 ` [PATCH AUTOSEL 4.4 37/37] vxlan: Avoid infinite loop when suppressing NS messages with invalid options Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200608232750.3370747-25-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=guoqing.jiang@cloud.ionos.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-raid@vger.kernel.org \
--cc=songliubraving@fb.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.