From: Bart Van Assche <bvanassche@acm.org>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Doug Ledford <dledford@redhat.com>,
linux-rdma@vger.kernel.org, Bart Van Assche <bvanassche@acm.org>,
Sergey Gorenko <sergeygo@mellanox.com>,
Max Gurtovoy <maxg@mellanox.com>,
Laurence Oberman <loberman@redhat.com>,
stable@vger.kernel.org
Subject: [PATCH 1/2] RDMA/srp: Avoid calling mutex_lock() from inside scsi_queue_rq()
Date: Wed, 16 Jan 2019 16:27:16 -0800 [thread overview]
Message-ID: <20190117002717.84686-2-bvanassche@acm.org> (raw)
In-Reply-To: <20190117002717.84686-1-bvanassche@acm.org>
srp_queuecommand() can be called from the following contexts:
* From inside scsi_queue_rq(). This function can be called by several
kernel threads, including the SCSI error handler thread.
* From inside scsi_send_eh_cmnd(). This function is only called from the
SCSI error handler thread.
In scsi-mq mode it is not allowed to sleep inside srp_queuecommand()
since the flag BLK_MQ_F_BLOCKING has not been set. Since setting the
request queue flag BLK_MQ_F_BLOCKING would slow down the hot path, make
srp_queuecommand() skip the mutex_lock() and mutex_unlock() calls when
called from inside scsi_queue_rq() from the SCSI EH thread.
This patch avoids that the following appears in the kernel log:
BUG: sleeping function called from invalid context at kernel/locking/mutex.c:908
in_atomic(): 1, irqs_disabled(): 0, pid: 30974, name: scsi_eh_4
INFO: lockdep is turned off.
Preemption disabled at:
[<ffffffff816b1d65>] __blk_mq_delay_run_hw_queue+0x185/0x290
CPU: 3 PID: 30974 Comm: scsi_eh_4 Not tainted 4.20.0-rc6-dbg+ #6
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
Call Trace:
dump_stack+0x86/0xca
___might_sleep.cold.80+0x128/0x139
__might_sleep+0x71/0xe0
__mutex_lock+0xc8/0xbe0
mutex_lock_nested+0x1b/0x20
srp_queuecommand+0x7f2/0x19a0 [ib_srp]
scsi_dispatch_cmd+0x15d/0x510
scsi_queue_rq+0x123/0xa90
blk_mq_dispatch_rq_list+0x678/0xc20
blk_mq_sched_dispatch_requests+0x25c/0x310
__blk_mq_run_hw_queue+0xd6/0x180
__blk_mq_delay_run_hw_queue+0x262/0x290
blk_mq_run_hw_queue+0x11f/0x1b0
blk_mq_run_hw_queues+0x87/0xb0
scsi_run_queue+0x402/0x6f0
scsi_run_host_queues+0x30/0x50
scsi_error_handler+0x2d3/0xa80
kthread+0x1cf/0x1f0
ret_from_fork+0x24/0x30
Cc: Sergey Gorenko <sergeygo@mellanox.com>
Cc: Max Gurtovoy <maxg@mellanox.com>
Cc: Laurence Oberman <loberman@redhat.com>
Cc: <stable@vger.kernel.org>
Fixes: a95cadb9dafe ("IB/srp: Add periodic reconnect functionality") # v3.13
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---
drivers/infiniband/ulp/srp/ib_srp.c | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 31d91538bbf4..23e5c9afb8fb 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -2352,13 +2352,19 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
u32 tag;
u16 idx;
int len, ret;
- const bool in_scsi_eh = !in_interrupt() && current == shost->ehandler;
+ const bool in_scsi_eh = !in_interrupt() && current == shost->ehandler
+ && rcu_preempt_depth() == 0;
/*
- * The SCSI EH thread is the only context from which srp_queuecommand()
- * can get invoked for blocked devices (SDEV_BLOCK /
+ * The scsi_send_eh_cmnd() function is the only function that can call
+ * .queuecommand() for blocked devices (SDEV_BLOCK /
* SDEV_CREATED_BLOCK). Avoid racing with srp_reconnect_rport() by
- * locking the rport mutex if invoked from inside the SCSI EH.
+ * locking the rport mutex if invoked from inside that function.
+ * Recognize this context by checking whether called from the SCSI EH
+ * thread and whether no RCU read lock is held. If
+ * blk_mq_run_hw_queues() is called from the context of the SCSI EH
+ * thread then an RCU read lock is held around scsi_queue_rq() calls
+ * becase the SRP driver does not set BLK_MQ_F_BLOCKING.
*/
if (in_scsi_eh)
mutex_lock(&rport->mutex);
--
2.20.1.97.g81188d93c3-goog
next parent reply other threads:[~2019-01-17 0:27 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20190117002717.84686-1-bvanassche@acm.org>
2019-01-17 0:27 ` Bart Van Assche [this message]
2019-01-19 10:03 ` [PATCH 1/2] RDMA/srp: Avoid calling mutex_lock() from inside scsi_queue_rq() Christoph Hellwig
2019-01-21 21:21 ` Bart Van Assche
2019-01-17 0:27 ` [PATCH 2/2] RDMA/srp: Rework SCSI device reset handling Bart Van Assche
2019-01-19 10:04 ` Christoph Hellwig
2019-01-21 21:08 ` Bart Van Assche
[not found] ` <20190122155559.D1DD9217D6@mail.kernel.org>
2019-01-22 16:04 ` Bart Van Assche
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190117002717.84686-2-bvanassche@acm.org \
--to=bvanassche@acm.org \
--cc=dledford@redhat.com \
--cc=jgg@ziepe.ca \
--cc=linux-rdma@vger.kernel.org \
--cc=loberman@redhat.com \
--cc=maxg@mellanox.com \
--cc=sergeygo@mellanox.com \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).