From: Sagi Grimberg <sagi@grimberg.me>
To: Keith Busch <kbusch@kernel.org>, Christoph Hellwig <hch@lst.de>
Cc: Jens Axboe <axboe@kernel.dk>, Chao Leng <lengchao@huawei.com>,
Ming Lei <ming.lei@redhat.com>,
linux-nvme@lists.infradead.org, linux-block@vger.kernel.org
Subject: Re: [PATCH 04/17] nvme: don't call nvme_kill_queues from nvme_remove_namespaces
Date: Tue, 25 Oct 2022 23:17:04 +0300 [thread overview]
Message-ID: <63c062dd-babb-e815-131a-bc0e513bb33e@grimberg.me> (raw)
In-Reply-To: <Y1ggN68V/mbAw1q2@kbusch-mbp.dhcp.thefacebook.com>
On 10/25/22 20:43, Keith Busch wrote:
> On Tue, Oct 25, 2022 at 07:40:07AM -0700, Christoph Hellwig wrote:
>> @@ -4560,15 +4560,6 @@ void nvme_remove_namespaces(struct nvme_ctrl *ctrl)
>> /* prevent racing with ns scanning */
>> flush_work(&ctrl->scan_work);
>>
>> - /*
>> - * The dead states indicates the controller was not gracefully
>> - * disconnected. In that case, we won't be able to flush any data while
>> - * removing the namespaces' disks; fail all the queues now to avoid
>> - * potentially having to clean up the failed sync later.
>> - */
>> - if (ctrl->state == NVME_CTRL_DEAD)
>> - nvme_kill_queues(ctrl);
>> -
>> /* this is a no-op when called from the controller reset handler */
>> nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING_NOIO);
>>
>> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
>> index ec034d4dd9eff..f971e96ffd3f6 100644
>> --- a/drivers/nvme/host/pci.c
>> +++ b/drivers/nvme/host/pci.c
>> @@ -3249,6 +3249,16 @@ static void nvme_remove(struct pci_dev *pdev)
>>
>> flush_work(&dev->ctrl.reset_work);
>> nvme_stop_ctrl(&dev->ctrl);
>> +
>> + /*
>> + * The dead states indicates the controller was not gracefully
>> + * disconnected. In that case, we won't be able to flush any data while
>> + * removing the namespaces' disks; fail all the queues now to avoid
>> + * potentially having to clean up the failed sync later.
>> + */
>> + if (dev->ctrl.state == NVME_CTRL_DEAD)
>> + nvme_kill_queues(&dev->ctrl);
>> +
>> nvme_remove_namespaces(&dev->ctrl);
>> nvme_dev_disable(dev, true);
>> nvme_remove_attrs(dev);
>> --
>> 2.30.2
>>
>
> We still need the flush_work(scan_work) prior to killing the queues. It
> looks like it could safely be moved to nvme_stop_ctrl(), which might
> make it easier on everyone if it were there.
If we do end up moving it to nvme_stop_ctrl, can we make a sub-version
of nvme_stop_ctrl that cannot block on I/O (i.e. without ana/scan/auth)?
for multipathing where we want to teardown the controller quickly so we
can failover I/O asap.
IIRC this is why scan_work is not in nvme_stop_ctrl to begin with, but
it is also possible that there was some other deadlock caused by that.
next prev parent reply other threads:[~2022-10-25 20:17 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-25 14:40 per-tagset SRCU struct and quiesce v2 Christoph Hellwig
2022-10-25 14:40 ` [PATCH 01/17] block: set the disk capacity to 0 in blk_mark_disk_dead Christoph Hellwig
2022-10-26 12:39 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 02/17] nvme-pci: refactor the tagset handling in nvme_reset_work Christoph Hellwig
2022-10-26 12:46 ` Sagi Grimberg
2022-10-30 9:17 ` Christoph Hellwig
2022-10-25 14:40 ` [PATCH 03/17] nvme-pci: don't warn about the lack of I/O queues for admin controllers Christoph Hellwig
2022-10-26 12:49 ` Sagi Grimberg
2022-10-30 9:18 ` Christoph Hellwig
2022-10-25 14:40 ` [PATCH 04/17] nvme: don't call nvme_kill_queues from nvme_remove_namespaces Christoph Hellwig
2022-10-25 17:43 ` Keith Busch
2022-10-25 20:17 ` Sagi Grimberg [this message]
2022-10-30 9:22 ` Christoph Hellwig
2022-10-25 14:40 ` [PATCH 05/17] nvme: don't remove namespaces in nvme_passthru_end Christoph Hellwig
2022-10-26 12:50 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 06/17] nvme: remove the NVME_NS_DEAD check in nvme_remove_invalid_namespaces Christoph Hellwig
2022-10-26 12:50 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 07/17] nvme: remove the NVME_NS_DEAD check in nvme_validate_ns Christoph Hellwig
2022-10-26 12:52 ` Sagi Grimberg
2022-10-30 9:28 ` Christoph Hellwig
2022-10-25 14:40 ` [PATCH 08/17] nvme: don't unquiesce the admin queue in nvme_kill_queues Christoph Hellwig
2022-10-26 12:53 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 09/17] nvme: don't unquiesce the I/O queues " Christoph Hellwig
2022-10-26 12:54 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 10/17] nvme-pci: mark the namespaces dead earlier in nvme_remove Christoph Hellwig
2022-10-25 18:53 ` Keith Busch
2022-10-26 12:55 ` Sagi Grimberg
2022-10-26 12:57 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 11/17] nvme-pci: don't unquiesce the I/O queues in nvme_remove_dead_ctrl Christoph Hellwig
2022-10-26 8:34 ` Chao Leng
2022-10-26 12:58 ` Sagi Grimberg
2022-10-27 2:46 ` Chao Leng
2022-10-25 14:40 ` [PATCH 12/17] nvme-pci: don't unquiesce the I/O queues in apple_nvme_reset_work Christoph Hellwig
2022-10-26 8:37 ` Chao Leng
2022-10-26 12:58 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 13/17] blk-mq: skip non-mq queues in blk_mq_quiesce_queue Christoph Hellwig
2022-10-25 14:40 ` [PATCH 14/17] blk-mq: move the srcu_struct used for quiescing to the tagset Christoph Hellwig
2022-10-26 8:48 ` Chao Leng
2022-10-26 13:01 ` Sagi Grimberg
2022-10-27 2:49 ` Chao Leng
2022-10-27 10:02 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 15/17] blk-mq: pass a tagset to blk_mq_wait_quiesce_done Christoph Hellwig
2022-10-25 14:40 ` [PATCH 16/17] blk-mq: add tagset quiesce interface Christoph Hellwig
2022-10-26 8:51 ` Chao Leng
2022-10-26 13:02 ` Sagi Grimberg
2022-10-25 14:40 ` [PATCH 17/17] nvme: use blk_mq_[un]quiesce_tagset Christoph Hellwig
2022-10-26 13:03 ` Sagi Grimberg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=63c062dd-babb-e815-131a-bc0e513bb33e@grimberg.me \
--to=sagi@grimberg.me \
--cc=axboe@kernel.dk \
--cc=hch@lst.de \
--cc=kbusch@kernel.org \
--cc=lengchao@huawei.com \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=ming.lei@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).