From: Li Jinlin <lijinlin3@huawei.com>
To: <jejb@linux.ibm.com>, <martin.petersen@oracle.com>,
<linux-scsi@vger.kernel.org>, <linux-kernel@vger.kernel.org>
Cc: <john.garry@huawei.com>, <bvanassche@acm.org>,
<qiulaibin@huawei.com>, <linfeilong@huawei.com>,
<wubo40@huawei.com>
Subject: [PATCH v3] scsi: core: Fix hang of freezing queue between blocking and running device
Date: Tue, 24 Aug 2021 10:59:21 +0800 [thread overview]
Message-ID: <20210824025921.3277629-1-lijinlin3@huawei.com> (raw)
From: Li Jinlin <lijinlin3@huawei.com>
We found a hang issue, the test steps are as follows:
1. blocking device via scsi_device_set_state()
2. dd if=/dev/sda of=/mnt/t.log bs=1M count=10
3. echo none > /sys/block/sda/queue/scheduler
4. echo "running" >/sys/block/sda/device/state
Step 3 and 4 should finish this work after step 4, but they hangs.
CPU#0 CPU#1 CPU#2
--------------- ---------------- ----------------
Step 1: blocking device
Step 2: dd xxxx
^^^^^^ get request
q_usage_counter++
Step 3: switching scheculer
elv_iosched_store
elevator_switch
blk_mq_freeze_queue
blk_freeze_queue
> blk_freeze_queue_start
^^^^^^ mq_freeze_depth++
> blk_mq_run_hw_queues
^^^^^^ can't run queue when dev blocked
> blk_mq_freeze_queue_wait
^^^^^^ Hang here!!!
wait q_usage_counter==0
Step 4: running device
store_state_field
scsi_rescan_device
scsi_attach_vpd
scsi_vpd_inquiry
__scsi_execute
blk_get_request
blk_mq_alloc_request
blk_queue_enter
^^^^^^ Hang here!!!
wait mq_freeze_depth==0
blk_mq_run_hw_queues
^^^^^^ dispatch IO, q_usage_counter will reduce to zero
blk_mq_unfreeze_queue
^^^^^ mq_freeze_depth--
Step 3 and 4 wait for each other.
To fix this, we need to run queue before rescanning device when the device
state changes to SDEV_RUNNING.
Fixes: f0f82e2476f6 ("scsi: core: Fix capacity set to zero after offlinining device")
Signed-off-by: Li Jinlin <lijinlin3@huawei.com>
Signed-off-by: Qiu Laibin <qiulaibin@huawei.com>
---
changes since v2 send with Message-ID:
20210809141308.3700854-1-lijinlin3@huawei.com
- Expanded code comment in store_state_field() as suggested by Bart
drivers/scsi/scsi_sysfs.c | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)
diff --git a/drivers/scsi/scsi_sysfs.c b/drivers/scsi/scsi_sysfs.c
index ae9bfc658203..c0d31119d6d7 100644
--- a/drivers/scsi/scsi_sysfs.c
+++ b/drivers/scsi/scsi_sysfs.c
@@ -808,12 +808,15 @@ store_state_field(struct device *dev, struct device_attribute *attr,
ret = scsi_device_set_state(sdev, state);
/*
* If the device state changes to SDEV_RUNNING, we need to
- * rescan the device to revalidate it, and run the queue to
- * avoid I/O hang.
+ * run the queue to avoid I/O hang, and rescan the device
+ * to revalidate it. Running the queue first is necessary
+ * because another thread may be waiting inside
+ * blk_mq_freeze_queue_wait() and because that call may be
+ * waiting for pending I/O to finish.
*/
if (ret == 0 && state == SDEV_RUNNING) {
- scsi_rescan_device(dev);
blk_mq_run_hw_queues(sdev->request_queue, true);
+ scsi_rescan_device(dev);
}
mutex_unlock(&sdev->state_mutex);
--
2.27.0
next reply other threads:[~2021-08-24 2:30 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-08-24 2:59 Li Jinlin [this message]
2021-08-24 3:41 ` [PATCH v3] scsi: core: Fix hang of freezing queue between blocking and running device Bart Van Assche
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210824025921.3277629-1-lijinlin3@huawei.com \
--to=lijinlin3@huawei.com \
--cc=bvanassche@acm.org \
--cc=jejb@linux.ibm.com \
--cc=john.garry@huawei.com \
--cc=linfeilong@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=qiulaibin@huawei.com \
--cc=wubo40@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.