All of lore.kernel.org
 help / color / mirror / Atom feed
From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Ming Lei <ming.lei@redhat.com>,
	linux-block@vger.kernel.org, Christoph Hellwig <hch@lst.de>,
	Bart Van Assche <bvanassche@acm.org>,
	"Martin K . Petersen" <martin.petersen@oracle.com>,
	Sasha Levin <sashal@kernel.org>,
	linux-scsi@vger.kernel.org
Subject: [PATCH AUTOSEL 5.7 10/25] scsi: core: Run queue in case of I/O resource contention failure
Date: Mon, 27 Jul 2020 19:23:30 -0400	[thread overview]
Message-ID: <20200727232345.717432-10-sashal@kernel.org> (raw)
In-Reply-To: <20200727232345.717432-1-sashal@kernel.org>

From: Ming Lei <ming.lei@redhat.com>

[ Upstream commit 3f0dcfbcd2e162fc0a11c1f59b7acd42ee45f126 ]

I/O requests may be held in scheduler queue because of resource contention.
The starvation scenario was handled properly in the regular completion
path but we failed to account for it during I/O submission. This lead to
the hang captured below. Make sure we run the queue when resource
contention is encountered in the submission path.

[   39.054963] scsi 13:0:0:0: rejecting I/O to dead device
[   39.058700] scsi 13:0:0:0: rejecting I/O to dead device
[   39.087855] sd 13:0:0:1: [sdd] Synchronizing SCSI cache
[   39.088909] scsi 13:0:0:1: rejecting I/O to dead device
[   39.095351] scsi 13:0:0:1: rejecting I/O to dead device
[   39.096962] scsi 13:0:0:1: rejecting I/O to dead device
[  247.021859] INFO: task scsi-stress-rem:813 blocked for more than 122 seconds.
[  247.023258]       Not tainted 5.8.0-rc2 #8
[  247.024069] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  247.025331] scsi-stress-rem D    0   813    802 0x00004000
[  247.025334] Call Trace:
[  247.025354]  __schedule+0x504/0x55f
[  247.027987]  schedule+0x72/0xa8
[  247.027991]  blk_mq_freeze_queue_wait+0x63/0x8c
[  247.027994]  ? do_wait_intr_irq+0x7a/0x7a
[  247.027996]  blk_cleanup_queue+0x4b/0xc9
[  247.028000]  __scsi_remove_device+0xf6/0x14e
[  247.028002]  scsi_remove_device+0x21/0x2b
[  247.029037]  sdev_store_delete+0x58/0x7c
[  247.029041]  kernfs_fop_write+0x10d/0x14f
[  247.031281]  vfs_write+0xa2/0xdf
[  247.032670]  ksys_write+0x6b/0xb3
[  247.032673]  do_syscall_64+0x56/0x82
[  247.034053]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  247.034059] RIP: 0033:0x7f69f39e9008
[  247.036330] Code: Bad RIP value.
[  247.036331] RSP: 002b:00007ffdd8116498 EFLAGS: 00000246 ORIG_RAX: 0000000000000001
[  247.037613] RAX: ffffffffffffffda RBX: 0000000000000002 RCX: 00007f69f39e9008
[  247.039714] RDX: 0000000000000002 RSI: 000055cde92a0ab0 RDI: 0000000000000001
[  247.039715] RBP: 000055cde92a0ab0 R08: 000000000000000a R09: 00007f69f3a79e80
[  247.039716] R10: 000000000000000a R11: 0000000000000246 R12: 00007f69f3abb780
[  247.039717] R13: 0000000000000002 R14: 00007f69f3ab6740 R15: 0000000000000002

Link: https://lore.kernel.org/r/20200720025435.812030-1-ming.lei@redhat.com
Cc: linux-block@vger.kernel.org
Cc: Christoph Hellwig <hch@lst.de>
Reviewed-by: Bart Van Assche <bvanassche@acm.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
 drivers/scsi/scsi_lib.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index b8b4366f12001..887b6a47f5dac 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -564,6 +564,15 @@ static void scsi_mq_uninit_cmd(struct scsi_cmnd *cmd)
 	scsi_uninit_cmd(cmd);
 }
 
+static void scsi_run_queue_async(struct scsi_device *sdev)
+{
+	if (scsi_target(sdev)->single_lun ||
+	    !list_empty(&sdev->host->starved_list))
+		kblockd_schedule_work(&sdev->requeue_work);
+	else
+		blk_mq_run_hw_queues(sdev->request_queue, true);
+}
+
 /* Returns false when no more bytes to process, true if there are more */
 static bool scsi_end_request(struct request *req, blk_status_t error,
 		unsigned int bytes)
@@ -608,11 +617,7 @@ static bool scsi_end_request(struct request *req, blk_status_t error,
 
 	__blk_mq_end_request(req, error);
 
-	if (scsi_target(sdev)->single_lun ||
-	    !list_empty(&sdev->host->starved_list))
-		kblockd_schedule_work(&sdev->requeue_work);
-	else
-		blk_mq_run_hw_queues(q, true);
+	scsi_run_queue_async(sdev);
 
 	percpu_ref_put(&q->q_usage_counter);
 	return false;
@@ -1706,6 +1711,7 @@ static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx,
 		 */
 		if (req->rq_flags & RQF_DONTPREP)
 			scsi_mq_uninit_cmd(cmd);
+		scsi_run_queue_async(sdev);
 		break;
 	}
 	return ret;
-- 
2.25.1


  parent reply	other threads:[~2020-07-27 23:29 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-27 23:23 [PATCH AUTOSEL 5.7 01/25] mt76: mt7615: fix lmac queue debugsfs entry Sasha Levin
2020-07-27 23:23 ` Sasha Levin
2020-07-27 23:23 ` Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 02/25] iwlwifi: fix crash in iwl_dbg_tlv_alloc_trigger Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 03/25] usb: hso: Fix debug compile warning on sparc32 Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 04/25] selftests: fib_nexthop_multiprefix: fix cleanup() netns deletion Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 05/25] qed: Disable "MFW indication via attention" SPAM every 5 minutes Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 06/25] selftests: net: ip_defrag: modprobe missing nf_defrag_ipv6 support Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 07/25] io_uring: always allow drain/link/hardlink/async sqe flags Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 08/25] habanalabs: prevent possible out-of-bounds array access Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 09/25] nfc: s3fwrn5: add missing release on skb in s3fwrn5_recv_frame Sasha Levin
2020-07-27 23:23 ` Sasha Levin [this message]
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 11/25] riscv: kasan: use local_tlb_flush_all() to avoid uninitialized __sbi_rfence Sasha Levin
2020-07-27 23:23   ` Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 12/25] parisc: add support for cmpxchg on u8 pointers Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 13/25] net: ethernet: ravb: exit if re-initialization fails in tx timeout Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 14/25] selftest: txtimestamp: fix net ns entry logic Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 15/25] drivers/net/wan/x25_asy: Fix to make it work Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 16/25] Revert "i2c: cadence: Fix the hold bit setting" Sasha Levin
2020-07-27 23:23   ` Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 17/25] i2c: cadence: Clear HOLD bit at correct time in Rx path Sasha Levin
2020-07-27 23:23   ` Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 18/25] x86/unwind/orc: Fix ORC for newly forked tasks Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 19/25] x86/stacktrace: Fix reliable check for empty user task stacks Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 20/25] cxgb4: add missing release on skb in uld_send() Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 21/25] io_uring: missed req_init_async() for IOSQE_ASYNC Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 22/25] xen-netfront: fix potential deadlock in xennet_remove() Sasha Levin
2020-07-27 23:23   ` Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 23/25] RISC-V: Set maximum number of mapped pages correctly Sasha Levin
2020-07-27 23:23   ` Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 24/25] drivers/net/wan: lapb: Corrected the usage of skb_cow Sasha Levin
2020-07-27 23:23 ` [PATCH AUTOSEL 5.7 25/25] riscv: Parse all memory blocks to remove unusable memory Sasha Levin
2020-07-27 23:23   ` Sasha Levin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200727232345.717432-10-sashal@kernel.org \
    --to=sashal@kernel.org \
    --cc=bvanassche@acm.org \
    --cc=hch@lst.de \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-scsi@vger.kernel.org \
    --cc=martin.petersen@oracle.com \
    --cc=ming.lei@redhat.com \
    --cc=stable@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.