All of lore.kernel.org
 help / color / mirror / Atom feed
From: "Markus Blöchl" <markus.bloechl@ipetronik.com>
To: Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@kernel.dk>
Cc: Christoph Hellwig <hch@lst.de>, Sagi Grimberg <sagi@grimberg.me>,
	linux-nvme@lists.infradead.org, linux-block@vger.kernel.org,
	linux-kernel@vger.kernel.org, Stefan Roese <sr@denx.de>
Subject: [RFC PATCH] nvme: prevent hang on surprise removal of NVMe disk
Date: Mon, 14 Feb 2022 10:51:07 +0100	[thread overview]
Message-ID: <20220214095107.3t5en5a3tosaeoo6@ipetronik.com> (raw)

After the surprise removal of a mounted NVMe disk the pciehp task
reliably hangs forever with a trace similar to this one:

 INFO: task irq/43-pciehp:64 blocked for more than 120 seconds.
 Call Trace:
  <TASK>
  __bio_queue_enter
  blk_mq_submit_bio
  submit_bio_noacct
  submit_bio_wait
  blkdev_issue_flush
  ext4_sync_fs
  sync_filesystem
  fsync_bdev
  delete_partition
  blk_drop_partitions
  del_gendisk
  nvme_ns_remove
  nvme_remove_namespaces
  nvme_remove
  pci_device_remove
  __device_release_driver
  device_release_driver
  pci_stop_bus_device
  pci_stop_and_remove_bus_device
  pciehp_unconfigure_device
  pciehp_disable_slot
  pciehp_handle_presence_or_link_change
  pciehp_ist
  </TASK>

I observed this with 5.15.5 from debian bullseye-backports and confirmed
with 5.17.0-rc3 but previous kernels may be affected as well.

I read that del_gendisk() prevents any new I/O only after
flushing and dropping all partitions.
But in case of a surprise removal any new blocking I/O must be prevented
first. I assume that nvme_set_queue_dying() is supposed to do that.
Is there any other mechanism in place which should achieve this?

Unfortunately I am not very familiar with the blk_mq infrastructure so
any comments and suggestions are very welcome.

Best regards,

Markus


Signed-off-by: Markus Blöchl <markus.bloechl@ipetronik.com>
---
 drivers/nvme/host/core.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 961a5f8a44d2..0654cbe9b80e 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -4573,6 +4573,8 @@ static void nvme_set_queue_dying(struct nvme_ns *ns)
 	if (test_and_set_bit(NVME_NS_DEAD, &ns->flags))
 		return;
 
+	set_bit(GD_DEAD, &ns->disk->state);
+
 	blk_set_queue_dying(ns->queue);
 	nvme_start_ns_queue(ns);
 

base-commit: f1baf68e1383f6ed93eb9cff2866d46562607a43
-- 
2.35.1

             reply	other threads:[~2022-02-14 10:13 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-02-14  9:51 Markus Blöchl [this message]
2022-02-15 15:22 ` [RFC PATCH] nvme: prevent hang on surprise removal of NVMe disk Keith Busch
2022-02-15 18:47   ` Christoph Hellwig
2022-02-15 19:14     ` Keith Busch
2022-02-15 19:17 ` Christoph Hellwig
2022-02-15 19:37   ` Keith Busch
2022-02-16  6:39   ` Hannes Reinecke
2022-02-16 11:18   ` Markus Blöchl
2022-02-16 11:32     ` Hannes Reinecke
2022-02-15 20:17 ` Christoph Hellwig
2022-02-16 12:59   ` Markus Blöchl
2022-02-16 13:33     ` Christoph Hellwig

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20220214095107.3t5en5a3tosaeoo6@ipetronik.com \
    --to=markus.bloechl@ipetronik.com \
    --cc=axboe@kernel.dk \
    --cc=hch@lst.de \
    --cc=kbusch@kernel.org \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-nvme@lists.infradead.org \
    --cc=sagi@grimberg.me \
    --cc=sr@denx.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.