From: Mike Snitzer <snitzer@redhat.com>
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>
Cc: dm-devel@redhat.com, linux-block@vger.kernel.org,
linux-nvme@lists.infradead.org
Subject: [PATCH v3 4/4] nvme: decouple basic ANA log page re-read support from native multipathing
Date: Fri, 16 Apr 2021 18:06:37 -0400 [thread overview]
Message-ID: <20210416220637.41111-5-snitzer@redhat.com> (raw)
In-Reply-To: <20210416220637.41111-1-snitzer@redhat.com>
Whether or not ANA is present is a choice of the target implementation;
the host (and whether it supports multipathing) has _zero_ influence on
this. If the target declares a path as 'inaccessible' the path _is_
inaccessible to the host. As such, ANA support should be functional
even if native multipathing is not.
Introduce ability to always re-read ANA log page as required due to ANA
error and make current ANA state available via sysfs -- even if native
multipathing is disabled on the host (e.g. nvme_core.multipath=N).
This is achieved by factoring out nvme_update_ana() and calling it in
nvme_complete_rq() for all FAILOVER requests.
This affords userspace access to the current ANA state independent of
which layer might be doing multipathing. This makes 'nvme list-subsys'
show ANA state for all NVMe subsystems with multiple controllers. It
also allows userspace multipath-tools to rely on the NVMe driver for
ANA support while dm-multipath takes care of multipathing.
And as always, if embedded NVMe users do not want any performance
overhead associated with ANA or native NVMe multipathing they can
disable CONFIG_NVME_MULTIPATH.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
---
drivers/nvme/host/core.c | 2 ++
drivers/nvme/host/multipath.c | 16 +++++++++++-----
drivers/nvme/host/nvme.h | 4 ++++
3 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 605ffba6835f..9a878a599897 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -349,6 +349,8 @@ static void nvme_failup_req(struct request *req)
{
blk_status_t status = nvme_error_status(nvme_req(req)->status);
+ nvme_update_ana(req);
+
/* Ensure a retryable path error is returned */
if (WARN_ON_ONCE(!blk_path_error(status))) {
/*
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index a1d476e1ac02..7d94250264aa 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -65,23 +65,29 @@ void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
}
}
-void nvme_failover_req(struct request *req)
+void nvme_update_ana(struct request *req)
{
struct nvme_ns *ns = req->q->queuedata;
u16 status = nvme_req(req)->status & 0x7ff;
- unsigned long flags;
-
- nvme_mpath_clear_current_path(ns);
/*
* If we got back an ANA error, we know the controller is alive but not
- * ready to serve this namespace. Kick of a re-read of the ANA
+ * ready to serve this namespace. Kick off a re-read of the ANA
* information page, and just try any other available path for now.
*/
if (nvme_is_ana_error(status) && ns->ctrl->ana_log_buf) {
set_bit(NVME_NS_ANA_PENDING, &ns->flags);
queue_work(nvme_wq, &ns->ctrl->ana_work);
}
+}
+
+void nvme_failover_req(struct request *req)
+{
+ struct nvme_ns *ns = req->q->queuedata;
+ unsigned long flags;
+
+ nvme_mpath_clear_current_path(ns);
+ nvme_update_ana(req);
spin_lock_irqsave(&ns->head->requeue_lock, flags);
blk_steal_bios(&ns->head->requeue_list, req);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..4eed8536625c 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -664,6 +664,7 @@ void nvme_mpath_start_freeze(struct nvme_subsystem *subsys);
void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
struct nvme_ctrl *ctrl, int *flags);
void nvme_failover_req(struct request *req);
+void nvme_update_ana(struct request *req);
void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl);
int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl,struct nvme_ns_head *head);
void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id);
@@ -714,6 +715,9 @@ static inline void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
static inline void nvme_failover_req(struct request *req)
{
}
+static inline void nvme_update_ana(struct request *req)
+{
+}
static inline void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl)
{
}
--
2.15.0
WARNING: multiple messages have this Message-ID (diff)
From: Mike Snitzer <snitzer@redhat.com>
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>
Cc: dm-devel@redhat.com, linux-block@vger.kernel.org,
linux-nvme@lists.infradead.org
Subject: [PATCH v3 4/4] nvme: decouple basic ANA log page re-read support from native multipathing
Date: Fri, 16 Apr 2021 18:06:37 -0400 [thread overview]
Message-ID: <20210416220637.41111-5-snitzer@redhat.com> (raw)
In-Reply-To: <20210416220637.41111-1-snitzer@redhat.com>
Whether or not ANA is present is a choice of the target implementation;
the host (and whether it supports multipathing) has _zero_ influence on
this. If the target declares a path as 'inaccessible' the path _is_
inaccessible to the host. As such, ANA support should be functional
even if native multipathing is not.
Introduce ability to always re-read ANA log page as required due to ANA
error and make current ANA state available via sysfs -- even if native
multipathing is disabled on the host (e.g. nvme_core.multipath=N).
This is achieved by factoring out nvme_update_ana() and calling it in
nvme_complete_rq() for all FAILOVER requests.
This affords userspace access to the current ANA state independent of
which layer might be doing multipathing. This makes 'nvme list-subsys'
show ANA state for all NVMe subsystems with multiple controllers. It
also allows userspace multipath-tools to rely on the NVMe driver for
ANA support while dm-multipath takes care of multipathing.
And as always, if embedded NVMe users do not want any performance
overhead associated with ANA or native NVMe multipathing they can
disable CONFIG_NVME_MULTIPATH.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
---
drivers/nvme/host/core.c | 2 ++
drivers/nvme/host/multipath.c | 16 +++++++++++-----
drivers/nvme/host/nvme.h | 4 ++++
3 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 605ffba6835f..9a878a599897 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -349,6 +349,8 @@ static void nvme_failup_req(struct request *req)
{
blk_status_t status = nvme_error_status(nvme_req(req)->status);
+ nvme_update_ana(req);
+
/* Ensure a retryable path error is returned */
if (WARN_ON_ONCE(!blk_path_error(status))) {
/*
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index a1d476e1ac02..7d94250264aa 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -65,23 +65,29 @@ void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
}
}
-void nvme_failover_req(struct request *req)
+void nvme_update_ana(struct request *req)
{
struct nvme_ns *ns = req->q->queuedata;
u16 status = nvme_req(req)->status & 0x7ff;
- unsigned long flags;
-
- nvme_mpath_clear_current_path(ns);
/*
* If we got back an ANA error, we know the controller is alive but not
- * ready to serve this namespace. Kick of a re-read of the ANA
+ * ready to serve this namespace. Kick off a re-read of the ANA
* information page, and just try any other available path for now.
*/
if (nvme_is_ana_error(status) && ns->ctrl->ana_log_buf) {
set_bit(NVME_NS_ANA_PENDING, &ns->flags);
queue_work(nvme_wq, &ns->ctrl->ana_work);
}
+}
+
+void nvme_failover_req(struct request *req)
+{
+ struct nvme_ns *ns = req->q->queuedata;
+ unsigned long flags;
+
+ nvme_mpath_clear_current_path(ns);
+ nvme_update_ana(req);
spin_lock_irqsave(&ns->head->requeue_lock, flags);
blk_steal_bios(&ns->head->requeue_list, req);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..4eed8536625c 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -664,6 +664,7 @@ void nvme_mpath_start_freeze(struct nvme_subsystem *subsys);
void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
struct nvme_ctrl *ctrl, int *flags);
void nvme_failover_req(struct request *req);
+void nvme_update_ana(struct request *req);
void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl);
int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl,struct nvme_ns_head *head);
void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id);
@@ -714,6 +715,9 @@ static inline void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
static inline void nvme_failover_req(struct request *req)
{
}
+static inline void nvme_update_ana(struct request *req)
+{
+}
static inline void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl)
{
}
--
2.15.0
_______________________________________________
Linux-nvme mailing list
Linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme
WARNING: multiple messages have this Message-ID (diff)
From: Mike Snitzer <snitzer@redhat.com>
To: Christoph Hellwig <hch@lst.de>, Jens Axboe <axboe@kernel.dk>
Cc: linux-block@vger.kernel.org, dm-devel@redhat.com,
linux-nvme@lists.infradead.org
Subject: [dm-devel] [PATCH v3 4/4] nvme: decouple basic ANA log page re-read support from native multipathing
Date: Fri, 16 Apr 2021 18:06:37 -0400 [thread overview]
Message-ID: <20210416220637.41111-5-snitzer@redhat.com> (raw)
In-Reply-To: <20210416220637.41111-1-snitzer@redhat.com>
Whether or not ANA is present is a choice of the target implementation;
the host (and whether it supports multipathing) has _zero_ influence on
this. If the target declares a path as 'inaccessible' the path _is_
inaccessible to the host. As such, ANA support should be functional
even if native multipathing is not.
Introduce ability to always re-read ANA log page as required due to ANA
error and make current ANA state available via sysfs -- even if native
multipathing is disabled on the host (e.g. nvme_core.multipath=N).
This is achieved by factoring out nvme_update_ana() and calling it in
nvme_complete_rq() for all FAILOVER requests.
This affords userspace access to the current ANA state independent of
which layer might be doing multipathing. This makes 'nvme list-subsys'
show ANA state for all NVMe subsystems with multiple controllers. It
also allows userspace multipath-tools to rely on the NVMe driver for
ANA support while dm-multipath takes care of multipathing.
And as always, if embedded NVMe users do not want any performance
overhead associated with ANA or native NVMe multipathing they can
disable CONFIG_NVME_MULTIPATH.
Signed-off-by: Mike Snitzer <snitzer@redhat.com>
---
drivers/nvme/host/core.c | 2 ++
drivers/nvme/host/multipath.c | 16 +++++++++++-----
drivers/nvme/host/nvme.h | 4 ++++
3 files changed, 17 insertions(+), 5 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 605ffba6835f..9a878a599897 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -349,6 +349,8 @@ static void nvme_failup_req(struct request *req)
{
blk_status_t status = nvme_error_status(nvme_req(req)->status);
+ nvme_update_ana(req);
+
/* Ensure a retryable path error is returned */
if (WARN_ON_ONCE(!blk_path_error(status))) {
/*
diff --git a/drivers/nvme/host/multipath.c b/drivers/nvme/host/multipath.c
index a1d476e1ac02..7d94250264aa 100644
--- a/drivers/nvme/host/multipath.c
+++ b/drivers/nvme/host/multipath.c
@@ -65,23 +65,29 @@ void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
}
}
-void nvme_failover_req(struct request *req)
+void nvme_update_ana(struct request *req)
{
struct nvme_ns *ns = req->q->queuedata;
u16 status = nvme_req(req)->status & 0x7ff;
- unsigned long flags;
-
- nvme_mpath_clear_current_path(ns);
/*
* If we got back an ANA error, we know the controller is alive but not
- * ready to serve this namespace. Kick of a re-read of the ANA
+ * ready to serve this namespace. Kick off a re-read of the ANA
* information page, and just try any other available path for now.
*/
if (nvme_is_ana_error(status) && ns->ctrl->ana_log_buf) {
set_bit(NVME_NS_ANA_PENDING, &ns->flags);
queue_work(nvme_wq, &ns->ctrl->ana_work);
}
+}
+
+void nvme_failover_req(struct request *req)
+{
+ struct nvme_ns *ns = req->q->queuedata;
+ unsigned long flags;
+
+ nvme_mpath_clear_current_path(ns);
+ nvme_update_ana(req);
spin_lock_irqsave(&ns->head->requeue_lock, flags);
blk_steal_bios(&ns->head->requeue_list, req);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 07b34175c6ce..4eed8536625c 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -664,6 +664,7 @@ void nvme_mpath_start_freeze(struct nvme_subsystem *subsys);
void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
struct nvme_ctrl *ctrl, int *flags);
void nvme_failover_req(struct request *req);
+void nvme_update_ana(struct request *req);
void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl);
int nvme_mpath_alloc_disk(struct nvme_ctrl *ctrl,struct nvme_ns_head *head);
void nvme_mpath_add_disk(struct nvme_ns *ns, struct nvme_id_ns *id);
@@ -714,6 +715,9 @@ static inline void nvme_set_disk_name(char *disk_name, struct nvme_ns *ns,
static inline void nvme_failover_req(struct request *req)
{
}
+static inline void nvme_update_ana(struct request *req)
+{
+}
static inline void nvme_kick_requeue_lists(struct nvme_ctrl *ctrl)
{
}
--
2.15.0
--
dm-devel mailing list
dm-devel@redhat.com
https://listman.redhat.com/mailman/listinfo/dm-devel
next prev parent reply other threads:[~2021-04-16 22:06 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-16 22:06 [PATCH v3 0/4] nvme: improve error handling and ana_state to work well with dm-multipath Mike Snitzer
2021-04-16 22:06 ` [dm-devel] " Mike Snitzer
2021-04-16 22:06 ` Mike Snitzer
2021-04-16 22:06 ` [PATCH v3 1/4] nvme: return BLK_STS_DO_NOT_RETRY if the DNR bit is set Mike Snitzer
2021-04-16 22:06 ` [dm-devel] " Mike Snitzer
2021-04-16 22:06 ` Mike Snitzer
2021-04-16 22:06 ` [PATCH v3 2/4] nvme: allow local retry for requests with REQ_FAILFAST_TRANSPORT set Mike Snitzer
2021-04-16 22:06 ` [dm-devel] " Mike Snitzer
2021-04-16 22:06 ` Mike Snitzer
2021-04-16 22:06 ` [PATCH v3 3/4] nvme: introduce FAILUP handling for REQ_FAILFAST_TRANSPORT Mike Snitzer
2021-04-16 22:06 ` [dm-devel] " Mike Snitzer
2021-04-16 22:06 ` Mike Snitzer
2021-04-16 22:06 ` Mike Snitzer [this message]
2021-04-16 22:06 ` [dm-devel] [PATCH v3 4/4] nvme: decouple basic ANA log page re-read support from native multipathing Mike Snitzer
2021-04-16 22:06 ` Mike Snitzer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210416220637.41111-5-snitzer@redhat.com \
--to=snitzer@redhat.com \
--cc=axboe@kernel.dk \
--cc=dm-devel@redhat.com \
--cc=hch@lst.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-nvme@lists.infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.