From: Sasha Levin <sashal@kernel.org>
To: linux-kernel@vger.kernel.org, stable@vger.kernel.org
Cc: Sagi Grimberg <sagi@grimberg.me>, Sasha Levin <sashal@kernel.org>,
linux-nvme@lists.infradead.org
Subject: [PATCH AUTOSEL 5.4 25/43] nvme-tcp: fix reset hang if controller died in the middle of a reset
Date: Mon, 7 Sep 2020 12:33:11 -0400 [thread overview]
Message-ID: <20200907163329.1280888-25-sashal@kernel.org> (raw)
In-Reply-To: <20200907163329.1280888-1-sashal@kernel.org>
From: Sagi Grimberg <sagi@grimberg.me>
[ Upstream commit e5c01f4f7f623e768e868bcc08d8e7ceb03b75d0 ]
If the controller becomes unresponsive in the middle of a reset, we will
hang because we are waiting for the freeze to complete, but that cannot
happen since we have commands that are inflight holding the
q_usage_counter, and we can't blindly fail requests that times out.
So give a timeout and if we cannot wait for queue freeze before
unfreezing, fail and have the error handling take care how to proceed
(either schedule a reconnect of remove the controller).
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
drivers/nvme/host/tcp.c | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 98a045429293e..9b81763b44d99 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1693,7 +1693,15 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
if (!new) {
nvme_start_queues(ctrl);
- nvme_wait_freeze(ctrl);
+ if (!nvme_wait_freeze_timeout(ctrl, NVME_IO_TIMEOUT)) {
+ /*
+ * If we timed out waiting for freeze we are likely to
+ * be stuck. Fail the controller initialization just
+ * to be safe.
+ */
+ ret = -ENODEV;
+ goto out_wait_freeze_timed_out;
+ }
blk_mq_update_nr_hw_queues(ctrl->tagset,
ctrl->queue_count - 1);
nvme_unfreeze(ctrl);
@@ -1701,6 +1709,9 @@ static int nvme_tcp_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
return 0;
+out_wait_freeze_timed_out:
+ nvme_stop_queues(ctrl);
+ nvme_tcp_stop_io_queues(ctrl);
out_cleanup_connect_q:
if (new)
blk_cleanup_queue(ctrl->connect_q);
--
2.25.1
next prev parent reply other threads:[~2020-09-07 16:57 UTC|newest]
Thread overview: 43+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-07 16:32 [PATCH AUTOSEL 5.4 01/43] ARC: HSDK: wireup perf irq Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 02/43] dmaengine: acpi: Put the CSRT table after using it Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 03/43] netfilter: conntrack: allow sctp hearbeat after connection re-use Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 04/43] rxrpc: Keep the ACK serial in a var in rxrpc_input_ack() Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 05/43] drivers/net/wan/lapbether: Added needed_tailroom Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 06/43] NFC: st95hf: Fix memleak in st95hf_in_send_cmd Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 07/43] firestream: Fix memleak in fs_open Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 08/43] ALSA: hda: Fix 2 channel swapping for Tegra Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 09/43] ALSA: hda/tegra: Program WAKEEN register " Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 10/43] drivers/dma/dma-jz4780: Fix race condition between probe and irq handler Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 11/43] ibmvnic fix NULL tx_pools and rx_tools issue at do_reset Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 12/43] net: hns3: Fix for geneve tx checksum bug Sasha Levin
2020-09-07 16:32 ` [PATCH AUTOSEL 5.4 13/43] xfs: fix off-by-one in inode alloc block reservation calculation Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 14/43] drivers/net/wan/lapbether: Set network_header before transmitting Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 15/43] cfg80211: regulatory: reject invalid hints Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 16/43] cfg80211: Adjust 6 GHz frequency to channel conversion Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 17/43] net: usb: Fix uninit-was-stored issue in asix_read_phy_addr() Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 18/43] xfs: initialize the shortform attr header padding entry Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 19/43] irqchip/eznps: Fix build error for !ARC700 builds Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 20/43] nvmet-tcp: Fix NULL dereference when a connect data comes in h2cdata pdu Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 21/43] nvme-fabrics: don't check state NVME_CTRL_NEW for request acceptance Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 22/43] nvme: have nvme_wait_freeze_timeout return if it timed out Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 23/43] nvme-tcp: serialize controller teardown sequences Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 24/43] nvme-tcp: fix timeout handler Sasha Levin
2020-09-07 16:33 ` Sasha Levin [this message]
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 26/43] nvme-rdma: serialize controller teardown sequences Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 27/43] nvme-rdma: fix timeout handler Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 28/43] nvme-rdma: fix reset hang if controller died in the middle of a reset Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 29/43] nvme-pci: cancel nvme device request before disabling Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 30/43] HID: quirks: Set INCREMENT_USAGE_ON_DUPLICATE for all Saitek X52 devices Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 31/43] HID: microsoft: Add rumble support for the 8bitdo SN30 Pro+ controller Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 32/43] drivers/net/wan/hdlc_cisco: Add hard_header_len Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 33/43] HID: elan: Fix memleak in elan_input_configured Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 34/43] ARC: [plat-hsdk]: Switch ethernet phy-mode to rgmii-id Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 35/43] cpufreq: intel_pstate: Refuse to turn off with HWP enabled Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 36/43] cpufreq: intel_pstate: Fix intel_pstate_get_hwp_max() for turbo disabled Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 37/43] net: usb: dm9601: Add USB ID of Keenetic Plus DSL Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 38/43] arm64/module: set trampoline section flags regardless of CONFIG_DYNAMIC_FTRACE Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 39/43] ALSA: hda: hdmi - add Rocketlake support Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 40/43] ALSA: hda: fix a runtime pm issue in SOF when integrated GPU is disabled Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 41/43] drm/amdgpu: Fix bug in reporting voltage for CIK Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 42/43] iommu/amd: Do not use IOMMUv2 functionality when SME is active Sasha Levin
2020-09-07 16:33 ` [PATCH AUTOSEL 5.4 43/43] gcov: Disable gcov build with GCC 10 Sasha Levin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200907163329.1280888-25-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
--cc=sagi@grimberg.me \
--cc=stable@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).