* [PATCH v2 0/3] nvme: fix hang in path of removing disk
@ 2017-05-20 3:56 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
To: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, Zhang Yi, linux-block, Johannes Thumshirn, Ming Lei
The first two patches fixes hang during removing disk.
The 3rd patch removes blk_mq_abort_requeue_list() since
no one uses it any more.
Ming Lei (3):
nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
nvme: avoid to use blk_mq_abort_requeue_list()
blk-mq: remove blk_mq_abort_requeue_list()
block/blk-mq.c | 19 -------------------
drivers/nvme/host/core.c | 13 ++++++++++---
include/linux/blk-mq.h | 1 -
3 files changed, 10 insertions(+), 23 deletions(-)
--
2.9.4
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 0/3] nvme: fix hang in path of removing disk
@ 2017-05-20 3:56 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
The first two patches fixes hang during removing disk.
The 3rd patch removes blk_mq_abort_requeue_list() since
no one uses it any more.
Ming Lei (3):
nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
nvme: avoid to use blk_mq_abort_requeue_list()
blk-mq: remove blk_mq_abort_requeue_list()
block/blk-mq.c | 19 -------------------
drivers/nvme/host/core.c | 13 ++++++++++---
include/linux/blk-mq.h | 1 -
3 files changed, 10 insertions(+), 23 deletions(-)
--
2.9.4
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-20 3:56 ` Ming Lei
-1 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
To: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, Zhang Yi, linux-block, Johannes Thumshirn, Ming Lei, stable
Inside nvme_kill_queues(), we have to start hw queues for
draining requests in sw queues, .dispatch list and requeue list,
so use blk_mq_start_hw_queues() instead of blk_mq_start_stopped_hw_queues()
which only run queues if queues are stopped, but the queues may have
been started already, for example nvme_start_queues() is called in reset work
function.
blk_mq_start_hw_queues() run hw queues in current context, instead
of running asynchronously like before. Given nvme_kill_queues() is
run from either remove context or reset worker context, both are fine
to run hw queue directly. And the mutex of namespaces_mutex isn't a
problem too becasue nvme_start_freeze() runs hw queue in this way
already.
Cc: stable@vger.kernel.org
Reported-by: Zhang Yi <yizhan@redhat.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/nvme/host/core.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d5e0906262ea..ce0d96913ee6 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2437,7 +2437,13 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
revalidate_disk(ns->disk);
blk_set_queue_dying(ns->queue);
blk_mq_abort_requeue_list(ns->queue);
- blk_mq_start_stopped_hw_queues(ns->queue, true);
+
+ /*
+ * We have to force to start queues for avoiding hang
+ * forever, and we have to make sure that queues won't
+ * be stopped forever from now on.
+ */
+ blk_mq_start_hw_queues(ns->queue);
}
mutex_unlock(&ctrl->namespaces_mutex);
}
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
@ 2017-05-20 3:56 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
Inside nvme_kill_queues(), we have to start hw queues for
draining requests in sw queues, .dispatch list and requeue list,
so use blk_mq_start_hw_queues() instead of blk_mq_start_stopped_hw_queues()
which only run queues if queues are stopped, but the queues may have
been started already, for example nvme_start_queues() is called in reset work
function.
blk_mq_start_hw_queues() run hw queues in current context, instead
of running asynchronously like before. Given nvme_kill_queues() is
run from either remove context or reset worker context, both are fine
to run hw queue directly. And the mutex of namespaces_mutex isn't a
problem too becasue nvme_start_freeze() runs hw queue in this way
already.
Cc: stable at vger.kernel.org
Reported-by: Zhang Yi <yizhan at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
drivers/nvme/host/core.c | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index d5e0906262ea..ce0d96913ee6 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2437,7 +2437,13 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
revalidate_disk(ns->disk);
blk_set_queue_dying(ns->queue);
blk_mq_abort_requeue_list(ns->queue);
- blk_mq_start_stopped_hw_queues(ns->queue, true);
+
+ /*
+ * We have to force to start queues for avoiding hang
+ * forever, and we have to make sure that queues won't
+ * be stopped forever from now on.
+ */
+ blk_mq_start_hw_queues(ns->queue);
}
mutex_unlock(&ctrl->namespaces_mutex);
}
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-20 3:56 ` Ming Lei
-1 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
To: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, Zhang Yi, linux-block, Johannes Thumshirn, Ming Lei, stable
NVMe may add request into requeue list simply and not kick off the
requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
is called in both nvme_kill_queues() and nvme_ns_remove() for
dealing with this issue.
Unfortunately blk_mq_abort_requeue_list() is absolutely a
race maker, for example, one request may be requeued during
the aborting. So this patch just calls blk_mq_kick_requeue_list() in
nvme_kill_queues() to handle this issue like what nvme_start_queues()
does. Now all requests in requeue list when queues are stopped will be
handled by blk_mq_kick_requeue_list() when queues are restarted, either
in nvme_start_queues() or in nvme_kill_queues().
Cc: stable@vger.kernel.org
Reported-by: Zhang Yi <yizhan@redhat.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/nvme/host/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ce0d96913ee6..9ce0a0a16984 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
if (ns->ndev)
nvme_nvm_unregister_sysfs(ns);
del_gendisk(ns->disk);
- blk_mq_abort_requeue_list(ns->queue);
blk_cleanup_queue(ns->queue);
}
@@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
continue;
revalidate_disk(ns->disk);
blk_set_queue_dying(ns->queue);
- blk_mq_abort_requeue_list(ns->queue);
/*
* We have to force to start queues for avoiding hang
@@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
* be stopped forever from now on.
*/
blk_mq_start_hw_queues(ns->queue);
+
+ /* draining requests in requeue list */
+ blk_mq_kick_requeue_list(q);
}
mutex_unlock(&ctrl->namespaces_mutex);
}
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
@ 2017-05-20 3:56 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
NVMe may add request into requeue list simply and not kick off the
requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
is called in both nvme_kill_queues() and nvme_ns_remove() for
dealing with this issue.
Unfortunately blk_mq_abort_requeue_list() is absolutely a
race maker, for example, one request may be requeued during
the aborting. So this patch just calls blk_mq_kick_requeue_list() in
nvme_kill_queues() to handle this issue like what nvme_start_queues()
does. Now all requests in requeue list when queues are stopped will be
handled by blk_mq_kick_requeue_list() when queues are restarted, either
in nvme_start_queues() or in nvme_kill_queues().
Cc: stable at vger.kernel.org
Reported-by: Zhang Yi <yizhan at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
drivers/nvme/host/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ce0d96913ee6..9ce0a0a16984 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
if (ns->ndev)
nvme_nvm_unregister_sysfs(ns);
del_gendisk(ns->disk);
- blk_mq_abort_requeue_list(ns->queue);
blk_cleanup_queue(ns->queue);
}
@@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
continue;
revalidate_disk(ns->disk);
blk_set_queue_dying(ns->queue);
- blk_mq_abort_requeue_list(ns->queue);
/*
* We have to force to start queues for avoiding hang
@@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
* be stopped forever from now on.
*/
blk_mq_start_hw_queues(ns->queue);
+
+ /* draining requests in requeue list */
+ blk_mq_kick_requeue_list(q);
}
mutex_unlock(&ctrl->namespaces_mutex);
}
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-20 3:56 ` Ming Lei
-1 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
To: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, Zhang Yi, linux-block, Johannes Thumshirn, Ming Lei
No one uses it any more, so remove it.
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
block/blk-mq.c | 19 -------------------
include/linux/blk-mq.h | 1 -
2 files changed, 20 deletions(-)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index a69ad122ed66..f2224ffd225d 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -628,25 +628,6 @@ void blk_mq_delay_kick_requeue_list(struct request_queue *q,
}
EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
-void blk_mq_abort_requeue_list(struct request_queue *q)
-{
- unsigned long flags;
- LIST_HEAD(rq_list);
-
- spin_lock_irqsave(&q->requeue_lock, flags);
- list_splice_init(&q->requeue_list, &rq_list);
- spin_unlock_irqrestore(&q->requeue_lock, flags);
-
- while (!list_empty(&rq_list)) {
- struct request *rq;
-
- rq = list_first_entry(&rq_list, struct request, queuelist);
- list_del_init(&rq->queuelist);
- blk_mq_end_request(rq, -EIO);
- }
-}
-EXPORT_SYMBOL(blk_mq_abort_requeue_list);
-
struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag)
{
if (tag < tags->nr_tags) {
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index c47aa248c640..fcd641032f8d 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -238,7 +238,6 @@ void blk_mq_add_to_requeue_list(struct request *rq, bool at_head,
bool kick_requeue_list);
void blk_mq_kick_requeue_list(struct request_queue *q);
void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs);
-void blk_mq_abort_requeue_list(struct request_queue *q);
void blk_mq_complete_request(struct request *rq);
bool blk_mq_queue_stopped(struct request_queue *q);
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
@ 2017-05-20 3:56 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 3:56 UTC (permalink / raw)
No one uses it any more, so remove it.
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
block/blk-mq.c | 19 -------------------
include/linux/blk-mq.h | 1 -
2 files changed, 20 deletions(-)
diff --git a/block/blk-mq.c b/block/blk-mq.c
index a69ad122ed66..f2224ffd225d 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -628,25 +628,6 @@ void blk_mq_delay_kick_requeue_list(struct request_queue *q,
}
EXPORT_SYMBOL(blk_mq_delay_kick_requeue_list);
-void blk_mq_abort_requeue_list(struct request_queue *q)
-{
- unsigned long flags;
- LIST_HEAD(rq_list);
-
- spin_lock_irqsave(&q->requeue_lock, flags);
- list_splice_init(&q->requeue_list, &rq_list);
- spin_unlock_irqrestore(&q->requeue_lock, flags);
-
- while (!list_empty(&rq_list)) {
- struct request *rq;
-
- rq = list_first_entry(&rq_list, struct request, queuelist);
- list_del_init(&rq->queuelist);
- blk_mq_end_request(rq, -EIO);
- }
-}
-EXPORT_SYMBOL(blk_mq_abort_requeue_list);
-
struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag)
{
if (tag < tags->nr_tags) {
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index c47aa248c640..fcd641032f8d 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -238,7 +238,6 @@ void blk_mq_add_to_requeue_list(struct request *rq, bool at_head,
bool kick_requeue_list);
void blk_mq_kick_requeue_list(struct request_queue *q);
void blk_mq_delay_kick_requeue_list(struct request_queue *q, unsigned long msecs);
-void blk_mq_abort_requeue_list(struct request_queue *q);
void blk_mq_complete_request(struct request *rq);
bool blk_mq_queue_stopped(struct request_queue *q);
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-20 5:34 ` Ming Lei
-1 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 5:34 UTC (permalink / raw)
To: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: Zhang Yi, stable, linux-block, linux-nvme, Johannes Thumshirn
On Sat, May 20, 2017 at 11:56:04AM +0800, Ming Lei wrote:
> NVMe may add request into requeue list simply and not kick off the
> requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
> is called in both nvme_kill_queues() and nvme_ns_remove() for
> dealing with this issue.
>
> Unfortunately blk_mq_abort_requeue_list() is absolutely a
> race maker, for example, one request may be requeued during
> the aborting. So this patch just calls blk_mq_kick_requeue_list() in
> nvme_kill_queues() to handle this issue like what nvme_start_queues()
> does. Now all requests in requeue list when queues are stopped will be
> handled by blk_mq_kick_requeue_list() when queues are restarted, either
> in nvme_start_queues() or in nvme_kill_queues().
>
> Cc: stable@vger.kernel.org
> Reported-by: Zhang Yi <yizhan@redhat.com>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
> drivers/nvme/host/core.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index ce0d96913ee6..9ce0a0a16984 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
> if (ns->ndev)
> nvme_nvm_unregister_sysfs(ns);
> del_gendisk(ns->disk);
> - blk_mq_abort_requeue_list(ns->queue);
> blk_cleanup_queue(ns->queue);
> }
>
> @@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> continue;
> revalidate_disk(ns->disk);
> blk_set_queue_dying(ns->queue);
> - blk_mq_abort_requeue_list(ns->queue);
>
> /*
> * We have to force to start queues for avoiding hang
> @@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> * be stopped forever from now on.
> */
> blk_mq_start_hw_queues(ns->queue);
> +
> + /* draining requests in requeue list */
> + blk_mq_kick_requeue_list(q);
oops, the above line causes build failure, please
take the following one.
---
>From e70cfe2edacf278e3e6605f8c08e01ebf65bff01 Mon Sep 17 00:00:00 2001
From: Ming Lei <ming.lei@redhat.com>
Date: Wed, 17 May 2017 09:02:07 +0800
Subject: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
NVMe may add request into requeue list simply and not kick off the
requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
is called in both nvme_kill_queues() and nvme_ns_remove() for
dealing with this issue.
Unfortunately blk_mq_abort_requeue_list() is absolutely a
race maker, for example, one request may be requeued during
the aborting. So this patch just calls blk_mq_kick_requeue_list() in
nvme_kill_queues() to handle this issue like what nvme_start_queues()
does. Now all requests in requeue list when queues are stopped will be
handled by blk_mq_kick_requeue_list() when queues are restarted, either
in nvme_start_queues() or in nvme_kill_queues().
Cc: stable@vger.kernel.org
Reported-by: Zhang Yi <yizhan@redhat.com>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
---
drivers/nvme/host/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ce0d96913ee6..e344f5d7c1bc 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
if (ns->ndev)
nvme_nvm_unregister_sysfs(ns);
del_gendisk(ns->disk);
- blk_mq_abort_requeue_list(ns->queue);
blk_cleanup_queue(ns->queue);
}
@@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
continue;
revalidate_disk(ns->disk);
blk_set_queue_dying(ns->queue);
- blk_mq_abort_requeue_list(ns->queue);
/*
* We have to force to start queues for avoiding hang
@@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
* be stopped forever from now on.
*/
blk_mq_start_hw_queues(ns->queue);
+
+ /* draining requests in requeue list */
+ blk_mq_kick_requeue_list(ns->queue);
}
mutex_unlock(&ctrl->namespaces_mutex);
}
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
@ 2017-05-20 5:34 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-20 5:34 UTC (permalink / raw)
On Sat, May 20, 2017@11:56:04AM +0800, Ming Lei wrote:
> NVMe may add request into requeue list simply and not kick off the
> requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
> is called in both nvme_kill_queues() and nvme_ns_remove() for
> dealing with this issue.
>
> Unfortunately blk_mq_abort_requeue_list() is absolutely a
> race maker, for example, one request may be requeued during
> the aborting. So this patch just calls blk_mq_kick_requeue_list() in
> nvme_kill_queues() to handle this issue like what nvme_start_queues()
> does. Now all requests in requeue list when queues are stopped will be
> handled by blk_mq_kick_requeue_list() when queues are restarted, either
> in nvme_start_queues() or in nvme_kill_queues().
>
> Cc: stable at vger.kernel.org
> Reported-by: Zhang Yi <yizhan at redhat.com>
> Signed-off-by: Ming Lei <ming.lei at redhat.com>
> ---
> drivers/nvme/host/core.c | 5 +++--
> 1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index ce0d96913ee6..9ce0a0a16984 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
> if (ns->ndev)
> nvme_nvm_unregister_sysfs(ns);
> del_gendisk(ns->disk);
> - blk_mq_abort_requeue_list(ns->queue);
> blk_cleanup_queue(ns->queue);
> }
>
> @@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> continue;
> revalidate_disk(ns->disk);
> blk_set_queue_dying(ns->queue);
> - blk_mq_abort_requeue_list(ns->queue);
>
> /*
> * We have to force to start queues for avoiding hang
> @@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> * be stopped forever from now on.
> */
> blk_mq_start_hw_queues(ns->queue);
> +
> + /* draining requests in requeue list */
> + blk_mq_kick_requeue_list(q);
oops, the above line causes build failure, please
take the following one.
---
>From e70cfe2edacf278e3e6605f8c08e01ebf65bff01 Mon Sep 17 00:00:00 2001
From: Ming Lei <ming.lei@redhat.com>
Date: Wed, 17 May 2017 09:02:07 +0800
Subject: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
NVMe may add request into requeue list simply and not kick off the
requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
is called in both nvme_kill_queues() and nvme_ns_remove() for
dealing with this issue.
Unfortunately blk_mq_abort_requeue_list() is absolutely a
race maker, for example, one request may be requeued during
the aborting. So this patch just calls blk_mq_kick_requeue_list() in
nvme_kill_queues() to handle this issue like what nvme_start_queues()
does. Now all requests in requeue list when queues are stopped will be
handled by blk_mq_kick_requeue_list() when queues are restarted, either
in nvme_start_queues() or in nvme_kill_queues().
Cc: stable at vger.kernel.org
Reported-by: Zhang Yi <yizhan at redhat.com>
Signed-off-by: Ming Lei <ming.lei at redhat.com>
---
drivers/nvme/host/core.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ce0d96913ee6..e344f5d7c1bc 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
if (ns->ndev)
nvme_nvm_unregister_sysfs(ns);
del_gendisk(ns->disk);
- blk_mq_abort_requeue_list(ns->queue);
blk_cleanup_queue(ns->queue);
}
@@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
continue;
revalidate_disk(ns->disk);
blk_set_queue_dying(ns->queue);
- blk_mq_abort_requeue_list(ns->queue);
/*
* We have to force to start queues for avoiding hang
@@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
* be stopped forever from now on.
*/
blk_mq_start_hw_queues(ns->queue);
+
+ /* draining requests in requeue list */
+ blk_mq_kick_requeue_list(ns->queue);
}
mutex_unlock(&ctrl->namespaces_mutex);
}
--
2.9.4
^ permalink raw reply related [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-21 6:20 ` Christoph Hellwig
-1 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-21 6:20 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg,
linux-nvme, Zhang Yi, linux-block, Johannes Thumshirn, stable
> index d5e0906262ea..ce0d96913ee6 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2437,7 +2437,13 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> revalidate_disk(ns->disk);
> blk_set_queue_dying(ns->queue);
> blk_mq_abort_requeue_list(ns->queue);
> - blk_mq_start_stopped_hw_queues(ns->queue, true);
> +
> + /*
> + * We have to force to start queues for avoiding hang
> + * forever, and we have to make sure that queues won't
> + * be stopped forever from now on.
> + */
/*
* Forcibly start all queues to avoid having stuck requests.
* Note: We must make sure to not stop the queues from
* now until the final removal.
*/
Otherwise this looks good to me:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
@ 2017-05-21 6:20 ` Christoph Hellwig
0 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-21 6:20 UTC (permalink / raw)
> index d5e0906262ea..ce0d96913ee6 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -2437,7 +2437,13 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> revalidate_disk(ns->disk);
> blk_set_queue_dying(ns->queue);
> blk_mq_abort_requeue_list(ns->queue);
> - blk_mq_start_stopped_hw_queues(ns->queue, true);
> +
> + /*
> + * We have to force to start queues for avoiding hang
> + * forever, and we have to make sure that queues won't
> + * be stopped forever from now on.
> + */
/*
* Forcibly start all queues to avoid having stuck requests.
* Note: We must make sure to not stop the queues from
* now until the final removal.
*/
Otherwise this looks good to me:
Reviewed-by: Christoph Hellwig <hch at lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
2017-05-20 5:34 ` Ming Lei
@ 2017-05-21 6:22 ` Christoph Hellwig
-1 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-21 6:22 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg,
Zhang Yi, stable, linux-block, linux-nvme, Johannes Thumshirn
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
@ 2017-05-21 6:22 ` Christoph Hellwig
0 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-21 6:22 UTC (permalink / raw)
Looks good,
Reviewed-by: Christoph Hellwig <hch at lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-21 6:22 ` Christoph Hellwig
-1 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-21 6:22 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg,
linux-nvme, Zhang Yi, linux-block, Johannes Thumshirn
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
@ 2017-05-21 6:22 ` Christoph Hellwig
0 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-21 6:22 UTC (permalink / raw)
Looks good,
Reviewed-by: Christoph Hellwig <hch at lst.de>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
2017-05-21 6:20 ` Christoph Hellwig
@ 2017-05-22 1:35 ` Ming Lei
-1 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-22 1:35 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, linux-block, Sagi Grimberg, Johannes Thumshirn,
linux-nvme, Keith Busch, stable, Zhang Yi
On Sun, May 21, 2017 at 08:20:02AM +0200, Christoph Hellwig wrote:
> > index d5e0906262ea..ce0d96913ee6 100644
> > --- a/drivers/nvme/host/core.c
> > +++ b/drivers/nvme/host/core.c
> > @@ -2437,7 +2437,13 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> > revalidate_disk(ns->disk);
> > blk_set_queue_dying(ns->queue);
> > blk_mq_abort_requeue_list(ns->queue);
> > - blk_mq_start_stopped_hw_queues(ns->queue, true);
> > +
> > + /*
> > + * We have to force to start queues for avoiding hang
> > + * forever, and we have to make sure that queues won't
> > + * be stopped forever from now on.
> > + */
>
> /*
> * Forcibly start all queues to avoid having stuck requests.
The above is better.
> * Note: We must make sure to not stop the queues from
> * now until the final removal.
In theory, it should be OK to stop and start queues again before the final
removal, so how about the following:
* Note: We must make sure to not put the queues into being stopped
forever from now until the final removal.
Thanks,
Ming
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
@ 2017-05-22 1:35 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-22 1:35 UTC (permalink / raw)
On Sun, May 21, 2017@08:20:02AM +0200, Christoph Hellwig wrote:
> > index d5e0906262ea..ce0d96913ee6 100644
> > --- a/drivers/nvme/host/core.c
> > +++ b/drivers/nvme/host/core.c
> > @@ -2437,7 +2437,13 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
> > revalidate_disk(ns->disk);
> > blk_set_queue_dying(ns->queue);
> > blk_mq_abort_requeue_list(ns->queue);
> > - blk_mq_start_stopped_hw_queues(ns->queue, true);
> > +
> > + /*
> > + * We have to force to start queues for avoiding hang
> > + * forever, and we have to make sure that queues won't
> > + * be stopped forever from now on.
> > + */
>
> /*
> * Forcibly start all queues to avoid having stuck requests.
The above is better.
> * Note: We must make sure to not stop the queues from
> * now until the final removal.
In theory, it should be OK to stop and start queues again before the final
removal, so how about the following:
* Note: We must make sure to not put the queues into being stopped
forever from now until the final removal.
Thanks,
Ming
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-22 5:35 ` Keith Busch
-1 siblings, 0 replies; 34+ messages in thread
From: Keith Busch @ 2017-05-22 5:35 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Christoph Hellwig, Sagi Grimberg, linux-nvme,
Zhang Yi, linux-block, Johannes Thumshirn, stable
Looks good.
Reviewed-by: Keith Busch <keith.busch@intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
@ 2017-05-22 5:35 ` Keith Busch
0 siblings, 0 replies; 34+ messages in thread
From: Keith Busch @ 2017-05-22 5:35 UTC (permalink / raw)
Looks good.
Reviewed-by: Keith Busch <keith.busch at intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-22 5:35 ` Keith Busch
-1 siblings, 0 replies; 34+ messages in thread
From: Keith Busch @ 2017-05-22 5:35 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Christoph Hellwig, Sagi Grimberg, linux-nvme,
Zhang Yi, linux-block, Johannes Thumshirn, stable
Looks good.
Reviewed-by: Keith Busch <keith.busch@intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
@ 2017-05-22 5:35 ` Keith Busch
0 siblings, 0 replies; 34+ messages in thread
From: Keith Busch @ 2017-05-22 5:35 UTC (permalink / raw)
Looks good.
Reviewed-by: Keith Busch <keith.busch at intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-22 5:36 ` Keith Busch
-1 siblings, 0 replies; 34+ messages in thread
From: Keith Busch @ 2017-05-22 5:36 UTC (permalink / raw)
To: Ming Lei
Cc: Jens Axboe, Christoph Hellwig, Sagi Grimberg, linux-nvme,
Zhang Yi, linux-block, Johannes Thumshirn
Looks good.
Reviewed-by: Keith Busch <keith.busch@intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
@ 2017-05-22 5:36 ` Keith Busch
0 siblings, 0 replies; 34+ messages in thread
From: Keith Busch @ 2017-05-22 5:36 UTC (permalink / raw)
Looks good.
Reviewed-by: Keith Busch <keith.busch at intel.com>
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-22 7:41 ` Johannes Thumshirn
-1 siblings, 0 replies; 34+ messages in thread
From: Johannes Thumshirn @ 2017-05-22 7:41 UTC (permalink / raw)
To: Ming Lei, Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, Zhang Yi, linux-block, stable
On 05/20/2017 05:56 AM, Ming Lei wrote:
> Inside nvme_kill_queues(), we have to start hw queues for
> draining requests in sw queues, .dispatch list and requeue list,
> so use blk_mq_start_hw_queues() instead of blk_mq_start_stopped_hw_queues()
> which only run queues if queues are stopped, but the queues may have
> been started already, for example nvme_start_queues() is called in reset work
> function.
>
> blk_mq_start_hw_queues() run hw queues in current context, instead
> of running asynchronously like before. Given nvme_kill_queues() is
> run from either remove context or reset worker context, both are fine
> to run hw queue directly. And the mutex of namespaces_mutex isn't a
> problem too becasue nvme_start_freeze() runs hw queue in this way
> already.
>
> Cc: stable@vger.kernel.org
> Reported-by: Zhang Yi <yizhan@redhat.com>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
Looks good,
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
@ 2017-05-22 7:41 ` Johannes Thumshirn
0 siblings, 0 replies; 34+ messages in thread
From: Johannes Thumshirn @ 2017-05-22 7:41 UTC (permalink / raw)
On 05/20/2017 05:56 AM, Ming Lei wrote:
> Inside nvme_kill_queues(), we have to start hw queues for
> draining requests in sw queues, .dispatch list and requeue list,
> so use blk_mq_start_hw_queues() instead of blk_mq_start_stopped_hw_queues()
> which only run queues if queues are stopped, but the queues may have
> been started already, for example nvme_start_queues() is called in reset work
> function.
>
> blk_mq_start_hw_queues() run hw queues in current context, instead
> of running asynchronously like before. Given nvme_kill_queues() is
> run from either remove context or reset worker context, both are fine
> to run hw queue directly. And the mutex of namespaces_mutex isn't a
> problem too becasue nvme_start_freeze() runs hw queue in this way
> already.
>
> Cc: stable at vger.kernel.org
> Reported-by: Zhang Yi <yizhan at redhat.com>
> Signed-off-by: Ming Lei <ming.lei at redhat.com>
> ---
Looks good,
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
--
Johannes Thumshirn Storage
jthumshirn at suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
2017-05-20 5:34 ` Ming Lei
@ 2017-05-22 7:42 ` Johannes Thumshirn
-1 siblings, 0 replies; 34+ messages in thread
From: Johannes Thumshirn @ 2017-05-22 7:42 UTC (permalink / raw)
To: Ming Lei, Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: Zhang Yi, stable, linux-block, linux-nvme
On 05/20/2017 07:34 AM, Ming Lei wrote:
> On Sat, May 20, 2017 at 11:56:04AM +0800, Ming Lei wrote:
>> NVMe may add request into requeue list simply and not kick off the
>> requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
>> is called in both nvme_kill_queues() and nvme_ns_remove() for
>> dealing with this issue.
>>
>> Unfortunately blk_mq_abort_requeue_list() is absolutely a
>> race maker, for example, one request may be requeued during
>> the aborting. So this patch just calls blk_mq_kick_requeue_list() in
>> nvme_kill_queues() to handle this issue like what nvme_start_queues()
>> does. Now all requests in requeue list when queues are stopped will be
>> handled by blk_mq_kick_requeue_list() when queues are restarted, either
>> in nvme_start_queues() or in nvme_kill_queues().
>>
>> Cc: stable@vger.kernel.org
>> Reported-by: Zhang Yi <yizhan@redhat.com>
>> Signed-off-by: Ming Lei <ming.lei@redhat.com>
>> ---
>> drivers/nvme/host/core.c | 5 +++--
>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index ce0d96913ee6..9ce0a0a16984 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
>> if (ns->ndev)
>> nvme_nvm_unregister_sysfs(ns);
>> del_gendisk(ns->disk);
>> - blk_mq_abort_requeue_list(ns->queue);
>> blk_cleanup_queue(ns->queue);
>> }
>>
>> @@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
>> continue;
>> revalidate_disk(ns->disk);
>> blk_set_queue_dying(ns->queue);
>> - blk_mq_abort_requeue_list(ns->queue);
>>
>> /*
>> * We have to force to start queues for avoiding hang
>> @@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
>> * be stopped forever from now on.
>> */
>> blk_mq_start_hw_queues(ns->queue);
>> +
>> + /* draining requests in requeue list */
>> + blk_mq_kick_requeue_list(q);
>
> oops, the above line causes build failure, please
> take the following one.
>
> ---
>
> From e70cfe2edacf278e3e6605f8c08e01ebf65bff01 Mon Sep 17 00:00:00 2001
> From: Ming Lei <ming.lei@redhat.com>
> Date: Wed, 17 May 2017 09:02:07 +0800
> Subject: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
>
> NVMe may add request into requeue list simply and not kick off the
> requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
> is called in both nvme_kill_queues() and nvme_ns_remove() for
> dealing with this issue.
>
> Unfortunately blk_mq_abort_requeue_list() is absolutely a
> race maker, for example, one request may be requeued during
> the aborting. So this patch just calls blk_mq_kick_requeue_list() in
> nvme_kill_queues() to handle this issue like what nvme_start_queues()
> does. Now all requests in requeue list when queues are stopped will be
> handled by blk_mq_kick_requeue_list() when queues are restarted, either
> in nvme_start_queues() or in nvme_kill_queues().
>
> Cc: stable@vger.kernel.org
> Reported-by: Zhang Yi <yizhan@redhat.com>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
@ 2017-05-22 7:42 ` Johannes Thumshirn
0 siblings, 0 replies; 34+ messages in thread
From: Johannes Thumshirn @ 2017-05-22 7:42 UTC (permalink / raw)
On 05/20/2017 07:34 AM, Ming Lei wrote:
> On Sat, May 20, 2017@11:56:04AM +0800, Ming Lei wrote:
>> NVMe may add request into requeue list simply and not kick off the
>> requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
>> is called in both nvme_kill_queues() and nvme_ns_remove() for
>> dealing with this issue.
>>
>> Unfortunately blk_mq_abort_requeue_list() is absolutely a
>> race maker, for example, one request may be requeued during
>> the aborting. So this patch just calls blk_mq_kick_requeue_list() in
>> nvme_kill_queues() to handle this issue like what nvme_start_queues()
>> does. Now all requests in requeue list when queues are stopped will be
>> handled by blk_mq_kick_requeue_list() when queues are restarted, either
>> in nvme_start_queues() or in nvme_kill_queues().
>>
>> Cc: stable at vger.kernel.org
>> Reported-by: Zhang Yi <yizhan at redhat.com>
>> Signed-off-by: Ming Lei <ming.lei at redhat.com>
>> ---
>> drivers/nvme/host/core.c | 5 +++--
>> 1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
>> index ce0d96913ee6..9ce0a0a16984 100644
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns)
>> if (ns->ndev)
>> nvme_nvm_unregister_sysfs(ns);
>> del_gendisk(ns->disk);
>> - blk_mq_abort_requeue_list(ns->queue);
>> blk_cleanup_queue(ns->queue);
>> }
>>
>> @@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
>> continue;
>> revalidate_disk(ns->disk);
>> blk_set_queue_dying(ns->queue);
>> - blk_mq_abort_requeue_list(ns->queue);
>>
>> /*
>> * We have to force to start queues for avoiding hang
>> @@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl)
>> * be stopped forever from now on.
>> */
>> blk_mq_start_hw_queues(ns->queue);
>> +
>> + /* draining requests in requeue list */
>> + blk_mq_kick_requeue_list(q);
>
> oops, the above line causes build failure, please
> take the following one.
>
> ---
>
> From e70cfe2edacf278e3e6605f8c08e01ebf65bff01 Mon Sep 17 00:00:00 2001
> From: Ming Lei <ming.lei at redhat.com>
> Date: Wed, 17 May 2017 09:02:07 +0800
> Subject: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list()
>
> NVMe may add request into requeue list simply and not kick off the
> requeue if hw queues are stopped. Then blk_mq_abort_requeue_list()
> is called in both nvme_kill_queues() and nvme_ns_remove() for
> dealing with this issue.
>
> Unfortunately blk_mq_abort_requeue_list() is absolutely a
> race maker, for example, one request may be requeued during
> the aborting. So this patch just calls blk_mq_kick_requeue_list() in
> nvme_kill_queues() to handle this issue like what nvme_start_queues()
> does. Now all requests in requeue list when queues are stopped will be
> handled by blk_mq_kick_requeue_list() when queues are restarted, either
> in nvme_start_queues() or in nvme_kill_queues().
>
> Cc: stable at vger.kernel.org
> Reported-by: Zhang Yi <yizhan at redhat.com>
> Signed-off-by: Ming Lei <ming.lei at redhat.com>
> ---
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
--
Johannes Thumshirn Storage
jthumshirn at suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
2017-05-20 3:56 ` Ming Lei
@ 2017-05-22 7:43 ` Johannes Thumshirn
-1 siblings, 0 replies; 34+ messages in thread
From: Johannes Thumshirn @ 2017-05-22 7:43 UTC (permalink / raw)
To: Ming Lei, Jens Axboe, Keith Busch, Christoph Hellwig, Sagi Grimberg
Cc: linux-nvme, Zhang Yi, linux-block
On 05/20/2017 05:56 AM, Ming Lei wrote:
> No one uses it any more, so remove it.
>
> Signed-off-by: Ming Lei <ming.lei@redhat.com>
> ---
Looks good,
Reviewed-by: Johannes Thumshirn <jthumshirn@suse.de>
--
Johannes Thumshirn Storage
jthumshirn@suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list()
@ 2017-05-22 7:43 ` Johannes Thumshirn
0 siblings, 0 replies; 34+ messages in thread
From: Johannes Thumshirn @ 2017-05-22 7:43 UTC (permalink / raw)
On 05/20/2017 05:56 AM, Ming Lei wrote:
> No one uses it any more, so remove it.
>
> Signed-off-by: Ming Lei <ming.lei at redhat.com>
> ---
Looks good,
Reviewed-by: Johannes Thumshirn <jthumshirn at suse.de>
--
Johannes Thumshirn Storage
jthumshirn at suse.de +49 911 74053 689
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 N?rnberg
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)
Key fingerprint = EC38 9CAB C2C4 F25D 8600 D0D0 0393 969D 2D76 0850
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
2017-05-22 1:35 ` Ming Lei
@ 2017-05-22 12:51 ` Christoph Hellwig
-1 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-22 12:51 UTC (permalink / raw)
To: Ming Lei
Cc: Christoph Hellwig, Jens Axboe, linux-block, Sagi Grimberg,
Johannes Thumshirn, linux-nvme, Keith Busch, stable, Zhang Yi
On Mon, May 22, 2017 at 09:35:58AM +0800, Ming Lei wrote:
> In theory, it should be OK to stop and start queues again before the final
> removal, so how about the following:
>
> * Note: We must make sure to not put the queues into being stopped
> forever from now until the final removal.
Maybe:
* Note that we must ensure the queues are not stopped
* when the final removal happens.
?
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
@ 2017-05-22 12:51 ` Christoph Hellwig
0 siblings, 0 replies; 34+ messages in thread
From: Christoph Hellwig @ 2017-05-22 12:51 UTC (permalink / raw)
On Mon, May 22, 2017@09:35:58AM +0800, Ming Lei wrote:
> In theory, it should be OK to stop and start queues again before the final
> removal, so how about the following:
>
> * Note: We must make sure to not put the queues into being stopped
> forever from now until the final removal.
Maybe:
* Note that we must ensure the queues are not stopped
* when the final removal happens.
?
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
2017-05-22 12:51 ` Christoph Hellwig
@ 2017-05-22 14:49 ` Ming Lei
-1 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-22 14:49 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, linux-block, Sagi Grimberg, Johannes Thumshirn,
linux-nvme, Keith Busch, stable, Zhang Yi
On Mon, May 22, 2017 at 02:51:40PM +0200, Christoph Hellwig wrote:
> On Mon, May 22, 2017 at 09:35:58AM +0800, Ming Lei wrote:
> > In theory, it should be OK to stop and start queues again before the final
> > removal, so how about the following:
> >
> > * Note: We must make sure to not put the queues into being stopped
> > forever from now until the final removal.
>
> Maybe:
> * Note that we must ensure the queues are not stopped
> * when the final removal happens.
>
> ?
Yeah, that is it, :-)
Thanks,
Ming
^ permalink raw reply [flat|nested] 34+ messages in thread
* [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues()
@ 2017-05-22 14:49 ` Ming Lei
0 siblings, 0 replies; 34+ messages in thread
From: Ming Lei @ 2017-05-22 14:49 UTC (permalink / raw)
On Mon, May 22, 2017@02:51:40PM +0200, Christoph Hellwig wrote:
> On Mon, May 22, 2017@09:35:58AM +0800, Ming Lei wrote:
> > In theory, it should be OK to stop and start queues again before the final
> > removal, so how about the following:
> >
> > * Note: We must make sure to not put the queues into being stopped
> > forever from now until the final removal.
>
> Maybe:
> * Note that we must ensure the queues are not stopped
> * when the final removal happens.
>
> ?
Yeah, that is it, :-)
Thanks,
Ming
^ permalink raw reply [flat|nested] 34+ messages in thread
end of thread, other threads:[~2017-05-22 14:49 UTC | newest]
Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-05-20 3:56 [PATCH v2 0/3] nvme: fix hang in path of removing disk Ming Lei
2017-05-20 3:56 ` Ming Lei
2017-05-20 3:56 ` [PATCH v2 1/3] nvme: use blk_mq_start_hw_queues() in nvme_kill_queues() Ming Lei
2017-05-20 3:56 ` Ming Lei
2017-05-21 6:20 ` Christoph Hellwig
2017-05-21 6:20 ` Christoph Hellwig
2017-05-22 1:35 ` Ming Lei
2017-05-22 1:35 ` Ming Lei
2017-05-22 12:51 ` Christoph Hellwig
2017-05-22 12:51 ` Christoph Hellwig
2017-05-22 14:49 ` Ming Lei
2017-05-22 14:49 ` Ming Lei
2017-05-22 5:35 ` Keith Busch
2017-05-22 5:35 ` Keith Busch
2017-05-22 7:41 ` Johannes Thumshirn
2017-05-22 7:41 ` Johannes Thumshirn
2017-05-20 3:56 ` [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list() Ming Lei
2017-05-20 3:56 ` Ming Lei
2017-05-20 5:34 ` Ming Lei
2017-05-20 5:34 ` Ming Lei
2017-05-21 6:22 ` Christoph Hellwig
2017-05-21 6:22 ` Christoph Hellwig
2017-05-22 7:42 ` Johannes Thumshirn
2017-05-22 7:42 ` Johannes Thumshirn
2017-05-22 5:35 ` Keith Busch
2017-05-22 5:35 ` Keith Busch
2017-05-20 3:56 ` [PATCH v2 3/3] blk-mq: remove blk_mq_abort_requeue_list() Ming Lei
2017-05-20 3:56 ` Ming Lei
2017-05-21 6:22 ` Christoph Hellwig
2017-05-21 6:22 ` Christoph Hellwig
2017-05-22 5:36 ` Keith Busch
2017-05-22 5:36 ` Keith Busch
2017-05-22 7:43 ` Johannes Thumshirn
2017-05-22 7:43 ` Johannes Thumshirn
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.