All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bart Van Assche <bvanassche@acm.org>
To: "Martin K . Petersen" <martin.petersen@oracle.com>
Cc: Jaegeuk Kim <jaegeuk@kernel.org>,
	Adrian Hunter <adrian.hunter@intel.com>,
	linux-scsi@vger.kernel.org, Bart Van Assche <bvanassche@acm.org>,
	"James E.J. Bottomley" <jejb@linux.ibm.com>,
	Bean Huo <beanhuo@micron.com>, Can Guo <cang@codeaurora.org>,
	Avri Altman <avri.altman@wdc.com>,
	Stanley Chu <stanley.chu@mediatek.com>,
	Asutosh Das <asutoshd@codeaurora.org>
Subject: [PATCH v2 13/20] scsi: ufs: Fix a deadlock in the error handler
Date: Fri, 19 Nov 2021 11:57:36 -0800	[thread overview]
Message-ID: <20211119195743.2817-14-bvanassche@acm.org> (raw)
In-Reply-To: <20211119195743.2817-1-bvanassche@acm.org>

The following deadlock has been observed on a test setup:
* All tags allocated.
* The SCSI error handler calls ufshcd_eh_host_reset_handler()
* ufshcd_eh_host_reset_handler() queues work that calls ufshcd_err_handler()
* ufshcd_err_handler() locks up as follows:

Workqueue: ufs_eh_wq_0 ufshcd_err_handler.cfi_jt
Call trace:
 __switch_to+0x298/0x5d8
 __schedule+0x6cc/0xa94
 schedule+0x12c/0x298
 blk_mq_get_tag+0x210/0x480
 __blk_mq_alloc_request+0x1c8/0x284
 blk_get_request+0x74/0x134
 ufshcd_exec_dev_cmd+0x68/0x640
 ufshcd_verify_dev_init+0x68/0x35c
 ufshcd_probe_hba+0x12c/0x1cb8
 ufshcd_host_reset_and_restore+0x88/0x254
 ufshcd_reset_and_restore+0xd0/0x354
 ufshcd_err_handler+0x408/0xc58
 process_one_work+0x24c/0x66c
 worker_thread+0x3e8/0xa4c
 kthread+0x150/0x1b4
 ret_from_fork+0x10/0x30

Fix this lockup by making ufshcd_exec_dev_cmd() allocate a reserved
request.

Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---
 drivers/scsi/ufs/ufshcd.c | 17 +++++++----------
 1 file changed, 7 insertions(+), 10 deletions(-)

diff --git a/drivers/scsi/ufs/ufshcd.c b/drivers/scsi/ufs/ufshcd.c
index a241ef6bbc6f..03f4772fc2e2 100644
--- a/drivers/scsi/ufs/ufshcd.c
+++ b/drivers/scsi/ufs/ufshcd.c
@@ -128,8 +128,9 @@ EXPORT_SYMBOL_GPL(ufshcd_dump_regs);
 enum {
 	UFSHCD_MAX_CHANNEL	= 0,
 	UFSHCD_MAX_ID		= 1,
-	UFSHCD_CMD_PER_LUN	= 32,
-	UFSHCD_CAN_QUEUE	= 32,
+	UFSHCD_NUM_RESERVED	= 1,
+	UFSHCD_CMD_PER_LUN	= 32 - UFSHCD_NUM_RESERVED,
+	UFSHCD_CAN_QUEUE	= 32 - UFSHCD_NUM_RESERVED,
 };
 
 static const char *const ufshcd_state_name[] = {
@@ -2941,12 +2942,7 @@ static int ufshcd_exec_dev_cmd(struct ufs_hba *hba,
 
 	down_read(&hba->clk_scaling_lock);
 
-	/*
-	 * Get free slot, sleep if slots are unavailable.
-	 * Even though we use wait_event() which sleeps indefinitely,
-	 * the maximum wait time is bounded by SCSI request timeout.
-	 */
-	scmd = scsi_get_internal_cmd(q, DMA_TO_DEVICE, 0);
+	scmd = scsi_get_internal_cmd(q, DMA_TO_DEVICE, BLK_MQ_REQ_RESERVED);
 	if (IS_ERR(scmd)) {
 		err = PTR_ERR(scmd);
 		goto out_unlock;
@@ -8171,6 +8167,7 @@ static struct scsi_host_template ufshcd_driver_template = {
 	.sg_tablesize		= SG_ALL,
 	.cmd_per_lun		= UFSHCD_CMD_PER_LUN,
 	.can_queue		= UFSHCD_CAN_QUEUE,
+	.reserved_tags		= UFSHCD_NUM_RESERVED,
 	.max_segment_size	= PRDT_DATA_BYTE_COUNT_MAX,
 	.max_host_blocked	= 1,
 	.track_queue_depth	= 1,
@@ -9531,8 +9528,8 @@ int ufshcd_init(struct ufs_hba *hba, void __iomem *mmio_base, unsigned int irq)
 	/* Configure LRB */
 	ufshcd_host_memory_configure(hba);
 
-	host->can_queue = hba->nutrs;
-	host->cmd_per_lun = hba->nutrs;
+	host->can_queue = hba->nutrs - UFSHCD_NUM_RESERVED;
+	host->cmd_per_lun = hba->nutrs - UFSHCD_NUM_RESERVED;
 	host->max_id = UFSHCD_MAX_ID;
 	host->max_lun = UFS_MAX_LUNS;
 	host->max_channel = UFSHCD_MAX_CHANNEL;

  parent reply	other threads:[~2021-11-19 19:58 UTC|newest]

Thread overview: 72+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-11-19 19:57 [PATCH v2 00/20] UFS patches for kernel v5.17 Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 01/20] block: Add a flag for internal commands Bart Van Assche
2021-11-22  8:46   ` John Garry
2021-11-22 17:38     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 02/20] scsi: core: Unexport scsi_track_queue_full() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 03/20] scsi: core: Fix scsi_device_max_queue_depth() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 04/20] scsi: core: Fix a race between scsi_done() and scsi_times_out() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 05/20] scsi: core: Add support for internal commands Bart Van Assche
2021-11-22  8:58   ` John Garry
2021-11-22 17:46     ` Bart Van Assche
2021-11-22 18:08       ` John Garry
2021-11-22 19:04       ` Bart Van Assche
2021-11-23  8:13       ` Hannes Reinecke
2021-11-23 17:46         ` Bart Van Assche
2021-11-23 19:18           ` Bart Van Assche
2021-11-24  6:33             ` Hannes Reinecke
2021-11-19 19:57 ` [PATCH v2 06/20] scsi: core: Add support for reserved tags Bart Van Assche
2021-11-22  8:15   ` John Garry
2021-11-22 17:25     ` Bart Van Assche
2021-11-22 18:13       ` John Garry
2021-11-19 19:57 ` [PATCH v2 07/20] scsi: ufs: Rename a function argument Bart Van Assche
2021-11-22 20:25   ` Bean Huo
2021-11-19 19:57 ` [PATCH v2 08/20] scsi: ufs: Remove is_rpmb_wlun() Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 09/20] scsi: ufs: Remove the sdev_rpmb member Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 10/20] scsi: ufs: Remove dead code Bart Van Assche
2021-11-24 11:11   ` Adrian Hunter
2021-11-29 19:12     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 11/20] scsi: ufs: Switch to scsi_(get|put)_internal_cmd() Bart Van Assche
2021-11-23 12:20   ` Bean Huo
2021-11-23 17:54     ` Bart Van Assche
2021-11-23 19:41     ` Bart Van Assche
2021-11-24 18:18       ` Bean Huo
2021-11-24 11:02   ` Adrian Hunter
2021-11-24 11:15     ` Adrian Hunter
2021-11-29 19:32     ` Bart Van Assche
2021-11-30  6:41       ` Adrian Hunter
2021-11-30 17:51         ` Bart Van Assche
2021-11-30 19:15           ` Adrian Hunter
2021-11-30 19:21             ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 12/20] scsi: ufs: Rework ufshcd_change_queue_depth() Bart Van Assche
2021-11-19 19:57 ` Bart Van Assche [this message]
2021-11-30  8:54   ` [PATCH v2 13/20] scsi: ufs: Fix a deadlock in the error handler Bean Huo
2021-11-30 17:52     ` Bart Van Assche
2021-11-30 19:32     ` Bart Van Assche
2021-12-01 13:44       ` Bean Huo
2021-12-01 18:31         ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 14/20] scsi: ufs: Introduce ufshcd_release_scsi_cmd() Bart Van Assche
2021-11-24 12:03   ` Adrian Hunter
2021-11-30 18:00     ` Bart Van Assche
2021-11-30 19:02       ` Adrian Hunter
2021-11-30 19:16         ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 15/20] scsi: ufs: Improve SCSI abort handling Bart Van Assche
2021-11-24 12:28   ` Adrian Hunter
2021-11-30  4:13     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 16/20] scsi: ufs: Fix a kernel crash during shutdown Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 17/20] scsi: ufs: Stop using the clock scaling lock in the error handler Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 18/20] scsi: ufs: Optimize the command queueing code Bart Van Assche
2021-11-22 17:46   ` Asutosh Das (asd)
2021-11-22 18:13     ` Bart Van Assche
2021-11-22 23:02       ` Asutosh Das (asd)
2021-11-22 23:48         ` Bart Van Assche
2021-11-23 18:24           ` Asutosh Das (asd)
2021-12-01 18:33             ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 19/20] scsi: ufs: Implement polling support Bart Van Assche
2021-11-30  8:43   ` Bean Huo
2021-11-30  8:57     ` Avri Altman
2021-11-30  9:15       ` Bean Huo
2021-11-30 14:26     ` Bart Van Assche
2021-11-30 15:40       ` Bean Huo
2021-11-30 17:34         ` Bart Van Assche
2021-11-30 17:37     ` Bart Van Assche
2021-11-19 19:57 ` [PATCH v2 20/20] scsi: ufs: Fix race conditions related to driver data Bart Van Assche

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211119195743.2817-14-bvanassche@acm.org \
    --to=bvanassche@acm.org \
    --cc=adrian.hunter@intel.com \
    --cc=asutoshd@codeaurora.org \
    --cc=avri.altman@wdc.com \
    --cc=beanhuo@micron.com \
    --cc=cang@codeaurora.org \
    --cc=jaegeuk@kernel.org \
    --cc=jejb@linux.ibm.com \
    --cc=linux-scsi@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=stanley.chu@mediatek.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.