* [PATCH] nvme-host tcp: check if the queue is allocated before stopping it
@ 2022-08-01 8:09 Maurizio Lombardi
2022-08-10 13:03 ` Sagi Grimberg
2022-08-10 14:22 ` Christoph Hellwig
0 siblings, 2 replies; 4+ messages in thread
From: Maurizio Lombardi @ 2022-08-01 8:09 UTC (permalink / raw)
To: sagi; +Cc: linux-nvme, hch, kbusch
When an error is detected and the host reconnects, the
nvme_tcp_error_recovery_work() function is called and starts
tearing down the io queues and de-allocating them;
If at the same time the "nvme" process deletes the controller via sysfs,
the nvme_tcp_delete_ctrl() gets called and waits until the
nvme_tcp_error_recovery_work() finishes its job; then starts
tearing down the io queues, but at this point they have already
been freed and the mutexes are destroyed.
Calling mutex_lock() against a destroyed mutex triggers a warning:
[ 1299.025575] nvme nvme1: Reconnecting in 10 seconds...
[ 1299.636449] nvme nvme1: Removing ctrl: NQN "blktests-subsystem-1"
[ 1299.645262] ------------[ cut here ]------------
[ 1299.649949] DEBUG_LOCKS_WARN_ON(lock->magic != lock)
[ 1299.649971] WARNING: CPU: 4 PID: 104150 at kernel/locking/mutex.c:579 __mutex_lock+0x2d0/0x7dc
[ 1299.717934] CPU: 4 PID: 104150 Comm: nvme
[ 1299.828075] Call trace:
[ 1299.830526] __mutex_lock+0x2d0/0x7dc
[ 1299.834203] mutex_lock_nested+0x64/0xd4
[ 1299.838139] nvme_tcp_stop_queue+0x54/0xe0 [nvme_tcp]
[ 1299.843211] nvme_tcp_teardown_io_queues.part.0+0x90/0x280 [nvme_tcp]
[ 1299.849672] nvme_tcp_delete_ctrl+0x6c/0xf0 [nvme_tcp]
[ 1299.854831] nvme_do_delete_ctrl+0x108/0x120 [nvme_core]
[ 1299.860181] nvme_sysfs_delete+0xec/0xf0 [nvme_core]
[ 1299.865179] dev_attr_store+0x40/0x70
Fix the warning by checking if the queues are allocated
in the nvme_tcp_stop_queue(). If they are not, it makes no
sense to try to stop them.
Signed-off-by: Maurizio Lombardi <mlombard@redhat.com>
---
drivers/nvme/host/tcp.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index ac74dce3e95b..ab9bb5c98c78 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1619,6 +1619,9 @@ static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
struct nvme_tcp_queue *queue = &ctrl->queues[qid];
+ if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
+ return;
+
mutex_lock(&queue->queue_lock);
if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
__nvme_tcp_stop_queue(queue);
--
2.31.1
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] nvme-host tcp: check if the queue is allocated before stopping it
2022-08-01 8:09 [PATCH] nvme-host tcp: check if the queue is allocated before stopping it Maurizio Lombardi
@ 2022-08-10 13:03 ` Sagi Grimberg
2022-08-10 13:05 ` Sagi Grimberg
2022-08-10 14:22 ` Christoph Hellwig
1 sibling, 1 reply; 4+ messages in thread
From: Sagi Grimberg @ 2022-08-10 13:03 UTC (permalink / raw)
To: Maurizio Lombardi; +Cc: linux-nvme, hch, kbusch
> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
> index ac74dce3e95b..ab9bb5c98c78 100644
> --- a/drivers/nvme/host/tcp.c
> +++ b/drivers/nvme/host/tcp.c
> @@ -1619,6 +1619,9 @@ static void nvme_tcp_stop_queue(struct nvme_ctrl *nctrl, int qid)
> struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
> struct nvme_tcp_queue *queue = &ctrl->queues[qid];
>
> + if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
> + return;
> +
> mutex_lock(&queue->queue_lock);
> if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
> __nvme_tcp_stop_queue(queue);
This looks correct to me,
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] nvme-host tcp: check if the queue is allocated before stopping it
2022-08-10 13:03 ` Sagi Grimberg
@ 2022-08-10 13:05 ` Sagi Grimberg
0 siblings, 0 replies; 4+ messages in thread
From: Sagi Grimberg @ 2022-08-10 13:05 UTC (permalink / raw)
To: Maurizio Lombardi; +Cc: linux-nvme, hch, kbusch
>
>> diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
>> index ac74dce3e95b..ab9bb5c98c78 100644
>> --- a/drivers/nvme/host/tcp.c
>> +++ b/drivers/nvme/host/tcp.c
>> @@ -1619,6 +1619,9 @@ static void nvme_tcp_stop_queue(struct nvme_ctrl
>> *nctrl, int qid)
>> struct nvme_tcp_ctrl *ctrl = to_tcp_ctrl(nctrl);
>> struct nvme_tcp_queue *queue = &ctrl->queues[qid];
>> + if (!test_bit(NVME_TCP_Q_ALLOCATED, &queue->flags))
>> + return;
>> +
>> mutex_lock(&queue->queue_lock);
>> if (test_and_clear_bit(NVME_TCP_Q_LIVE, &queue->flags))
>> __nvme_tcp_stop_queue(queue);
>
> This looks correct to me,
>
> Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Just one nit,
The patch title should start with nvme-tcp: and not nvme-host tcp:
This can be easily addressed when applying the patch
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] nvme-host tcp: check if the queue is allocated before stopping it
2022-08-01 8:09 [PATCH] nvme-host tcp: check if the queue is allocated before stopping it Maurizio Lombardi
2022-08-10 13:03 ` Sagi Grimberg
@ 2022-08-10 14:22 ` Christoph Hellwig
1 sibling, 0 replies; 4+ messages in thread
From: Christoph Hellwig @ 2022-08-10 14:22 UTC (permalink / raw)
To: Maurizio Lombardi; +Cc: sagi, linux-nvme, hch, kbusch
Thanks,
applied to nvme-6.0 with the fixed up subject.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-08-10 14:23 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-08-01 8:09 [PATCH] nvme-host tcp: check if the queue is allocated before stopping it Maurizio Lombardi
2022-08-10 13:03 ` Sagi Grimberg
2022-08-10 13:05 ` Sagi Grimberg
2022-08-10 14:22 ` Christoph Hellwig
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.