From: Jens Axboe <axboe@kernel.dk>
To: James Bottomley <James.Bottomley@HansenPartnership.com>,
Ming Lei <ming.lei@redhat.com>
Cc: Yi Zhang <yi.zhang@redhat.com>,
linux-block@vger.kernel.org,
"Martin K . Petersen" <martin.petersen@oracle.com>,
linux-scsi@vger.kernel.org, Mike Snitzer <snitzer@redhat.com>,
dm-devel@redhat.com
Subject: Re: [PATCH 2/3] scsi: make sure that request queue queiesce and unquiesce balanced
Date: Tue, 2 Nov 2021 06:59:18 -0600 [thread overview]
Message-ID: <8cbc1be6-15a5-ed34-53f1-081a05025d34@kernel.dk> (raw)
In-Reply-To: <10c279f54ed0b24cb1ac0861f9a407e6b64f64da.camel@HansenPartnership.com>
On 11/1/21 7:43 PM, James Bottomley wrote:
> On Thu, 2021-10-21 at 22:59 +0800, Ming Lei wrote:
>> For fixing queue quiesce race between driver and block layer(elevator
>> switch, update nr_requests, ...), we need to support concurrent
>> quiesce
>> and unquiesce, which requires the two call balanced.
>>
>> It isn't easy to audit that in all scsi drivers, especially the two
>> may
>> be called from different contexts, so do it in scsi core with one
>> per-device
>> bit flag & global spinlock, basically zero cost since request queue
>> quiesce
>> is seldom triggered.
>>
>> Reported-by: Yi Zhang <yi.zhang@redhat.com>
>> Fixes: e70feb8b3e68 ("blk-mq: support concurrent queue
>> quiesce/unquiesce")
>> Signed-off-by: Ming Lei <ming.lei@redhat.com>
>> ---
>> drivers/scsi/scsi_lib.c | 45 ++++++++++++++++++++++++++++++----
>> ----
>> include/scsi/scsi_device.h | 1 +
>> 2 files changed, 37 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index 51fcd46be265..414f4daf8005 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
>> @@ -2638,6 +2638,40 @@ static int
>> __scsi_internal_device_block_nowait(struct scsi_device *sdev)
>> return 0;
>> }
>>
>> +static DEFINE_SPINLOCK(sdev_queue_stop_lock);
>> +
>> +void scsi_start_queue(struct scsi_device *sdev)
>> +{
>> + bool need_start;
>> + unsigned long flags;
>> +
>> + spin_lock_irqsave(&sdev_queue_stop_lock, flags);
>> + need_start = sdev->queue_stopped;
>> + sdev->queue_stopped = 0;
>> + spin_unlock_irqrestore(&sdev_queue_stop_lock, flags);
>> +
>> + if (need_start)
>> + blk_mq_unquiesce_queue(sdev->request_queue);
>
> Well, this is a classic atomic pattern:
>
> if (cmpxchg(&sdev->queue_stopped, 1, 0))
> blk_mq_unquiesce_queue(sdev->request_queue);
>
> The reason to do it with atomics rather than spinlocks is
>
> 1. no need to disable interrupts: atomics are locked
> 2. faster because a spinlock takes an exclusive line every time but the
> read to check the value can be in shared mode in cmpxchg
> 3. it's just shorter and better code.
>
> The only minor downside is queue_stopped now needs to be a u32.
Are you fine with the change as-is, or do you want it redone? I
can drop the SCSI parts and just queue up the dm fix. Personally
I think it'd be better to get it fixed upfront.
--
Jens Axboe
next prev parent reply other threads:[~2021-11-02 12:59 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-21 14:59 [PATCH 0/3] block: keep quiesce & unquiesce balanced for scsi/dm Ming Lei
2021-10-21 14:59 ` [PATCH 1/3] scsi: avoid to quiesce sdev->request_queue two times Ming Lei
2021-10-21 14:59 ` [PATCH 2/3] scsi: make sure that request queue queiesce and unquiesce balanced Ming Lei
2021-11-02 1:43 ` James Bottomley
2021-11-02 12:58 ` Ming Lei
2021-11-02 12:59 ` Jens Axboe [this message]
2021-11-02 14:33 ` James Bottomley
2021-11-02 14:36 ` Jens Axboe
2021-11-02 14:41 ` Jens Axboe
2021-11-02 14:47 ` James Bottomley
2021-11-02 14:49 ` Jens Axboe
2021-11-02 14:52 ` Jens Axboe
2021-10-21 14:59 ` [PATCH 3/3] dm: don't stop request queue after the dm device is suspended Ming Lei
2021-11-01 16:56 ` Mike Snitzer
2021-10-25 1:43 ` [PATCH 0/3] block: keep quiesce & unquiesce balanced for scsi/dm Yi Zhang
2021-11-01 19:54 ` Jens Axboe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8cbc1be6-15a5-ed34-53f1-081a05025d34@kernel.dk \
--to=axboe@kernel.dk \
--cc=James.Bottomley@HansenPartnership.com \
--cc=dm-devel@redhat.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=ming.lei@redhat.com \
--cc=snitzer@redhat.com \
--cc=yi.zhang@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).