* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
This is the third part of the attempt to centralize controller reset,
delete and fabrics error recovery in nvme core.
As a reminder, the motivation is to get as much of the duplicate logic
existing in the various nvme transports to coommon code as possible.
We strive to have nvme core and fabrics modules take care of nvme and
fabrics constructs and initialization procedures and have our transports
simply worry about the underlying medium and spec extensions/enhancements.
This set is a step in that direction in the sense that it places a generic
controller initialization flows in nvme core, and converts two transports
to use common code.
The set is gradually reshaping nvme-rdma code to be more generic and then
simply move it to nvme-core (controller reset and delete) and nvme-fabrics
(error recovery and periodic reconnects). Then, nvme-loop is converted to
use nvme-core in one shot (Simply rip out all the duplicated logic).
Note that this patch set shouldn't break any transport that still does not
use it (pci, fc). The next part would be to take a stab at these transports
and add whatever is still needed to common code.
I tested this set with controller resets, deletes and error recovery in
the presence of I/O load. I tested rdma and loop.
Note that this direction got some exposure [1] and I tried to incorporate
feedback so if no hard objections are raised I plan to land this in 4.14.
This set applies on nvme-4.14 plus 4.13 recent fixes.
[1]:
http://www.mail-archive.com/linux-block@vger.kernel.org/msg09923.html
https://lwn.net/Articles/725720/
Sagi Grimberg (12):
nvme: move err and reconnect work to nvme ctrl
nvme-rdma: move admin specific resources to alloc_queue
nvme-rdma: split nvme_rdma_alloc_io_queues
nvme-rdma: restructure create_ctrl a bit
nvme-rdma: introduce nvme_rdma_alloc/stop/free_admin_queue
nvme-rdma: plumb nvme ctrl to various routines
nvme-rdma: split generic probe out of create_ctrl
nvme: add some ctrl ops for centralizing control plane logic.
nvme: move control plane handling to nvme core
nvme-fabrics: handle reconnects in fabrics library
nvme: add sed-opal ctrl manipulation in admin configuration
nvme-loop: convert to nvme-core control plane management
drivers/nvme/host/core.c | 312 +++++++++++++++++++++++++++
drivers/nvme/host/fabrics.c | 103 +++++++++
drivers/nvme/host/fabrics.h | 1 +
drivers/nvme/host/nvme.h | 27 +++
drivers/nvme/host/rdma.c | 515 +++++++++-----------------------------------
drivers/nvme/target/loop.c | 443 ++++++++++++++-----------------------
6 files changed, 706 insertions(+), 695 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
This is the third part of the attempt to centralize controller reset,
delete and fabrics error recovery in nvme core.
As a reminder, the motivation is to get as much of the duplicate logic
existing in the various nvme transports to coommon code as possible.
We strive to have nvme core and fabrics modules take care of nvme and
fabrics constructs and initialization procedures and have our transports
simply worry about the underlying medium and spec extensions/enhancements.
This set is a step in that direction in the sense that it places a generic
controller initialization flows in nvme core, and converts two transports
to use common code.
The set is gradually reshaping nvme-rdma code to be more generic and then
simply move it to nvme-core (controller reset and delete) and nvme-fabrics
(error recovery and periodic reconnects). Then, nvme-loop is converted to
use nvme-core in one shot (Simply rip out all the duplicated logic).
Note that this patch set shouldn't break any transport that still does not
use it (pci, fc). The next part would be to take a stab at these transports
and add whatever is still needed to common code.
I tested this set with controller resets, deletes and error recovery in
the presence of I/O load. I tested rdma and loop.
Note that this direction got some exposure [1] and I tried to incorporate
feedback so if no hard objections are raised I plan to land this in 4.14.
This set applies on nvme-4.14 plus 4.13 recent fixes.
[1]:
http://www.mail-archive.com/linux-block at vger.kernel.org/msg09923.html
https://lwn.net/Articles/725720/
Sagi Grimberg (12):
nvme: move err and reconnect work to nvme ctrl
nvme-rdma: move admin specific resources to alloc_queue
nvme-rdma: split nvme_rdma_alloc_io_queues
nvme-rdma: restructure create_ctrl a bit
nvme-rdma: introduce nvme_rdma_alloc/stop/free_admin_queue
nvme-rdma: plumb nvme ctrl to various routines
nvme-rdma: split generic probe out of create_ctrl
nvme: add some ctrl ops for centralizing control plane logic.
nvme: move control plane handling to nvme core
nvme-fabrics: handle reconnects in fabrics library
nvme: add sed-opal ctrl manipulation in admin configuration
nvme-loop: convert to nvme-core control plane management
drivers/nvme/host/core.c | 312 +++++++++++++++++++++++++++
drivers/nvme/host/fabrics.c | 103 +++++++++
drivers/nvme/host/fabrics.h | 1 +
drivers/nvme/host/nvme.h | 27 +++
drivers/nvme/host/rdma.c | 515 +++++++++-----------------------------------
drivers/nvme/target/loop.c | 443 ++++++++++++++-----------------------
6 files changed, 706 insertions(+), 695 deletions(-)
--
2.7.4
^ permalink raw reply [flat|nested] 50+ messages in thread
* [PATCH 01/12] nvme: move err and reconnect work to nvme ctrl
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
In theory, all fabric transports can/should use these.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/core.c | 4 ++++
drivers/nvme/host/nvme.h | 3 +++
drivers/nvme/host/rdma.c | 29 +++++++++++------------------
3 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index a2ac892f470e..bbaf5b98f2fe 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2703,6 +2703,10 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
void nvme_stop_ctrl(struct nvme_ctrl *ctrl)
{
nvme_stop_keep_alive(ctrl);
+ if (ctrl->ops->flags & NVME_F_FABRICS) {
+ cancel_work_sync(&ctrl->err_work);
+ cancel_delayed_work_sync(&ctrl->reconnect_work);
+ }
flush_work(&ctrl->async_event_work);
flush_work(&ctrl->scan_work);
cancel_work_sync(&ctrl->fw_act_work);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 2c8a02be46fd..c52ba1405788 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -186,6 +186,9 @@ struct nvme_ctrl {
u16 maxcmd;
int nr_reconnects;
struct nvmf_ctrl_options *opts;
+ struct delayed_work reconnect_work;
+ struct work_struct delete_work;
+ struct work_struct err_work;
};
struct nvme_ns {
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 74fd62062377..10e54f81e3d9 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -105,13 +105,9 @@ struct nvme_rdma_ctrl {
/* other member variables */
struct blk_mq_tag_set tag_set;
- struct work_struct delete_work;
- struct work_struct err_work;
struct nvme_rdma_qe async_event_sqe;
- struct delayed_work reconnect_work;
-
struct list_head list;
struct blk_mq_tag_set admin_tag_set;
@@ -908,18 +904,18 @@ static void nvme_rdma_reconnect_or_remove(struct nvme_rdma_ctrl *ctrl)
if (nvmf_should_reconnect(&ctrl->ctrl)) {
dev_info(ctrl->ctrl.device, "Reconnecting in %d seconds...\n",
ctrl->ctrl.opts->reconnect_delay);
- queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
+ queue_delayed_work(nvme_wq, &ctrl->ctrl.reconnect_work,
ctrl->ctrl.opts->reconnect_delay * HZ);
} else {
dev_info(ctrl->ctrl.device, "Removing controller...\n");
- queue_work(nvme_wq, &ctrl->delete_work);
+ queue_work(nvme_wq, &ctrl->ctrl.delete_work);
}
}
static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
{
struct nvme_rdma_ctrl *ctrl = container_of(to_delayed_work(work),
- struct nvme_rdma_ctrl, reconnect_work);
+ struct nvme_rdma_ctrl, ctrl.reconnect_work);
bool changed;
int ret;
@@ -958,7 +954,7 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
static void nvme_rdma_error_recovery_work(struct work_struct *work)
{
struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, err_work);
+ struct nvme_rdma_ctrl, ctrl.err_work);
nvme_stop_ctrl(&ctrl->ctrl);
@@ -991,7 +987,7 @@ static void nvme_rdma_error_recovery(struct nvme_rdma_ctrl *ctrl)
if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_RECONNECTING))
return;
- queue_work(nvme_wq, &ctrl->err_work);
+ queue_work(nvme_wq, &ctrl->ctrl.err_work);
}
static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
@@ -1725,9 +1721,6 @@ static const struct blk_mq_ops nvme_rdma_admin_mq_ops = {
static void nvme_rdma_shutdown_ctrl(struct nvme_rdma_ctrl *ctrl, bool shutdown)
{
- cancel_work_sync(&ctrl->err_work);
- cancel_delayed_work_sync(&ctrl->reconnect_work);
-
if (ctrl->ctrl.queue_count > 1) {
nvme_stop_queues(&ctrl->ctrl);
blk_mq_tagset_busy_iter(&ctrl->tag_set,
@@ -1758,7 +1751,7 @@ static void nvme_rdma_remove_ctrl(struct nvme_rdma_ctrl *ctrl)
static void nvme_rdma_del_ctrl_work(struct work_struct *work)
{
struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, delete_work);
+ struct nvme_rdma_ctrl, ctrl.delete_work);
nvme_stop_ctrl(&ctrl->ctrl);
nvme_rdma_remove_ctrl(ctrl);
@@ -1769,7 +1762,7 @@ static int __nvme_rdma_del_ctrl(struct nvme_rdma_ctrl *ctrl)
if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING))
return -EBUSY;
- if (!queue_work(nvme_wq, &ctrl->delete_work))
+ if (!queue_work(nvme_wq, &ctrl->ctrl.delete_work))
return -EBUSY;
return 0;
@@ -1788,7 +1781,7 @@ static int nvme_rdma_del_ctrl(struct nvme_ctrl *nctrl)
return -EBUSY;
ret = __nvme_rdma_del_ctrl(ctrl);
if (!ret)
- flush_work(&ctrl->delete_work);
+ flush_work(&ctrl->ctrl.delete_work);
nvme_put_ctrl(&ctrl->ctrl);
return ret;
}
@@ -1879,10 +1872,10 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
if (ret)
goto out_free_ctrl;
- INIT_DELAYED_WORK(&ctrl->reconnect_work,
+ INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
nvme_rdma_reconnect_ctrl_work);
- INIT_WORK(&ctrl->err_work, nvme_rdma_error_recovery_work);
- INIT_WORK(&ctrl->delete_work, nvme_rdma_del_ctrl_work);
+ INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
+ INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 01/12] nvme: move err and reconnect work to nvme ctrl
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
In theory, all fabric transports can/should use these.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/core.c | 4 ++++
drivers/nvme/host/nvme.h | 3 +++
drivers/nvme/host/rdma.c | 29 +++++++++++------------------
3 files changed, 18 insertions(+), 18 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index a2ac892f470e..bbaf5b98f2fe 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2703,6 +2703,10 @@ static void nvme_release_instance(struct nvme_ctrl *ctrl)
void nvme_stop_ctrl(struct nvme_ctrl *ctrl)
{
nvme_stop_keep_alive(ctrl);
+ if (ctrl->ops->flags & NVME_F_FABRICS) {
+ cancel_work_sync(&ctrl->err_work);
+ cancel_delayed_work_sync(&ctrl->reconnect_work);
+ }
flush_work(&ctrl->async_event_work);
flush_work(&ctrl->scan_work);
cancel_work_sync(&ctrl->fw_act_work);
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 2c8a02be46fd..c52ba1405788 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -186,6 +186,9 @@ struct nvme_ctrl {
u16 maxcmd;
int nr_reconnects;
struct nvmf_ctrl_options *opts;
+ struct delayed_work reconnect_work;
+ struct work_struct delete_work;
+ struct work_struct err_work;
};
struct nvme_ns {
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 74fd62062377..10e54f81e3d9 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -105,13 +105,9 @@ struct nvme_rdma_ctrl {
/* other member variables */
struct blk_mq_tag_set tag_set;
- struct work_struct delete_work;
- struct work_struct err_work;
struct nvme_rdma_qe async_event_sqe;
- struct delayed_work reconnect_work;
-
struct list_head list;
struct blk_mq_tag_set admin_tag_set;
@@ -908,18 +904,18 @@ static void nvme_rdma_reconnect_or_remove(struct nvme_rdma_ctrl *ctrl)
if (nvmf_should_reconnect(&ctrl->ctrl)) {
dev_info(ctrl->ctrl.device, "Reconnecting in %d seconds...\n",
ctrl->ctrl.opts->reconnect_delay);
- queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
+ queue_delayed_work(nvme_wq, &ctrl->ctrl.reconnect_work,
ctrl->ctrl.opts->reconnect_delay * HZ);
} else {
dev_info(ctrl->ctrl.device, "Removing controller...\n");
- queue_work(nvme_wq, &ctrl->delete_work);
+ queue_work(nvme_wq, &ctrl->ctrl.delete_work);
}
}
static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
{
struct nvme_rdma_ctrl *ctrl = container_of(to_delayed_work(work),
- struct nvme_rdma_ctrl, reconnect_work);
+ struct nvme_rdma_ctrl, ctrl.reconnect_work);
bool changed;
int ret;
@@ -958,7 +954,7 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
static void nvme_rdma_error_recovery_work(struct work_struct *work)
{
struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, err_work);
+ struct nvme_rdma_ctrl, ctrl.err_work);
nvme_stop_ctrl(&ctrl->ctrl);
@@ -991,7 +987,7 @@ static void nvme_rdma_error_recovery(struct nvme_rdma_ctrl *ctrl)
if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_RECONNECTING))
return;
- queue_work(nvme_wq, &ctrl->err_work);
+ queue_work(nvme_wq, &ctrl->ctrl.err_work);
}
static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
@@ -1725,9 +1721,6 @@ static const struct blk_mq_ops nvme_rdma_admin_mq_ops = {
static void nvme_rdma_shutdown_ctrl(struct nvme_rdma_ctrl *ctrl, bool shutdown)
{
- cancel_work_sync(&ctrl->err_work);
- cancel_delayed_work_sync(&ctrl->reconnect_work);
-
if (ctrl->ctrl.queue_count > 1) {
nvme_stop_queues(&ctrl->ctrl);
blk_mq_tagset_busy_iter(&ctrl->tag_set,
@@ -1758,7 +1751,7 @@ static void nvme_rdma_remove_ctrl(struct nvme_rdma_ctrl *ctrl)
static void nvme_rdma_del_ctrl_work(struct work_struct *work)
{
struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, delete_work);
+ struct nvme_rdma_ctrl, ctrl.delete_work);
nvme_stop_ctrl(&ctrl->ctrl);
nvme_rdma_remove_ctrl(ctrl);
@@ -1769,7 +1762,7 @@ static int __nvme_rdma_del_ctrl(struct nvme_rdma_ctrl *ctrl)
if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING))
return -EBUSY;
- if (!queue_work(nvme_wq, &ctrl->delete_work))
+ if (!queue_work(nvme_wq, &ctrl->ctrl.delete_work))
return -EBUSY;
return 0;
@@ -1788,7 +1781,7 @@ static int nvme_rdma_del_ctrl(struct nvme_ctrl *nctrl)
return -EBUSY;
ret = __nvme_rdma_del_ctrl(ctrl);
if (!ret)
- flush_work(&ctrl->delete_work);
+ flush_work(&ctrl->ctrl.delete_work);
nvme_put_ctrl(&ctrl->ctrl);
return ret;
}
@@ -1879,10 +1872,10 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
if (ret)
goto out_free_ctrl;
- INIT_DELAYED_WORK(&ctrl->reconnect_work,
+ INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
nvme_rdma_reconnect_ctrl_work);
- INIT_WORK(&ctrl->err_work, nvme_rdma_error_recovery_work);
- INIT_WORK(&ctrl->delete_work, nvme_rdma_del_ctrl_work);
+ INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
+ INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 02/12] nvme-rdma: move admin specific resources to alloc_queue
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
We're trying to make admin queue configuration generic, so
move the rdma specifics to the queue allocation (based on
the queue index passed).
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/rdma.c | 37 +++++++++++++++++++++----------------
1 file changed, 21 insertions(+), 16 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 10e54f81e3d9..d94b4df364a5 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -544,6 +544,23 @@ static int nvme_rdma_alloc_queue(struct nvme_rdma_ctrl *ctrl,
goto out_destroy_cm_id;
}
+ if (!idx) {
+ ctrl->device = ctrl->queues[0].device;
+ ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
+ ctrl->device->dev->attrs.max_fast_reg_page_list_len);
+ ctrl->ctrl.max_hw_sectors =
+ (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
+
+ ret = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
+ &ctrl->async_event_sqe, sizeof(struct nvme_command),
+ DMA_TO_DEVICE);
+ if (ret) {
+ nvme_rdma_destroy_queue_ib(&ctrl->queues[0]);
+ goto out_destroy_cm_id;
+ }
+
+ }
+
clear_bit(NVME_RDMA_Q_DELETING, &queue->flags);
return 0;
@@ -567,6 +584,10 @@ static void nvme_rdma_free_queue(struct nvme_rdma_queue *queue)
if (test_and_set_bit(NVME_RDMA_Q_DELETING, &queue->flags))
return;
+ if (!nvme_rdma_queue_idx(queue))
+ nvme_rdma_free_qe(queue->device->dev,
+ &queue->ctrl->async_event_sqe,
+ sizeof(struct nvme_command), DMA_TO_DEVICE);
nvme_rdma_destroy_queue_ib(queue);
rdma_destroy_id(queue->cm_id);
}
@@ -735,8 +756,6 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
bool remove)
{
- nvme_rdma_free_qe(ctrl->queues[0].device->dev, &ctrl->async_event_sqe,
- sizeof(struct nvme_command), DMA_TO_DEVICE);
nvme_rdma_stop_queue(&ctrl->queues[0]);
if (remove) {
blk_cleanup_queue(ctrl->ctrl.admin_q);
@@ -754,11 +773,6 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
if (error)
return error;
- ctrl->device = ctrl->queues[0].device;
-
- ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
- ctrl->device->dev->attrs.max_fast_reg_page_list_len);
-
if (new) {
ctrl->ctrl.admin_tagset = nvme_rdma_alloc_tagset(&ctrl->ctrl, true);
if (IS_ERR(ctrl->ctrl.admin_tagset))
@@ -794,19 +808,10 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
if (error)
goto out_cleanup_queue;
- ctrl->ctrl.max_hw_sectors =
- (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
-
error = nvme_init_identify(&ctrl->ctrl);
if (error)
goto out_cleanup_queue;
- error = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
- &ctrl->async_event_sqe, sizeof(struct nvme_command),
- DMA_TO_DEVICE);
- if (error)
- goto out_cleanup_queue;
-
return 0;
out_cleanup_queue:
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 02/12] nvme-rdma: move admin specific resources to alloc_queue
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
We're trying to make admin queue configuration generic, so
move the rdma specifics to the queue allocation (based on
the queue index passed).
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/rdma.c | 37 +++++++++++++++++++++----------------
1 file changed, 21 insertions(+), 16 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 10e54f81e3d9..d94b4df364a5 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -544,6 +544,23 @@ static int nvme_rdma_alloc_queue(struct nvme_rdma_ctrl *ctrl,
goto out_destroy_cm_id;
}
+ if (!idx) {
+ ctrl->device = ctrl->queues[0].device;
+ ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
+ ctrl->device->dev->attrs.max_fast_reg_page_list_len);
+ ctrl->ctrl.max_hw_sectors =
+ (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
+
+ ret = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
+ &ctrl->async_event_sqe, sizeof(struct nvme_command),
+ DMA_TO_DEVICE);
+ if (ret) {
+ nvme_rdma_destroy_queue_ib(&ctrl->queues[0]);
+ goto out_destroy_cm_id;
+ }
+
+ }
+
clear_bit(NVME_RDMA_Q_DELETING, &queue->flags);
return 0;
@@ -567,6 +584,10 @@ static void nvme_rdma_free_queue(struct nvme_rdma_queue *queue)
if (test_and_set_bit(NVME_RDMA_Q_DELETING, &queue->flags))
return;
+ if (!nvme_rdma_queue_idx(queue))
+ nvme_rdma_free_qe(queue->device->dev,
+ &queue->ctrl->async_event_sqe,
+ sizeof(struct nvme_command), DMA_TO_DEVICE);
nvme_rdma_destroy_queue_ib(queue);
rdma_destroy_id(queue->cm_id);
}
@@ -735,8 +756,6 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
bool remove)
{
- nvme_rdma_free_qe(ctrl->queues[0].device->dev, &ctrl->async_event_sqe,
- sizeof(struct nvme_command), DMA_TO_DEVICE);
nvme_rdma_stop_queue(&ctrl->queues[0]);
if (remove) {
blk_cleanup_queue(ctrl->ctrl.admin_q);
@@ -754,11 +773,6 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
if (error)
return error;
- ctrl->device = ctrl->queues[0].device;
-
- ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
- ctrl->device->dev->attrs.max_fast_reg_page_list_len);
-
if (new) {
ctrl->ctrl.admin_tagset = nvme_rdma_alloc_tagset(&ctrl->ctrl, true);
if (IS_ERR(ctrl->ctrl.admin_tagset))
@@ -794,19 +808,10 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
if (error)
goto out_cleanup_queue;
- ctrl->ctrl.max_hw_sectors =
- (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
-
error = nvme_init_identify(&ctrl->ctrl);
if (error)
goto out_cleanup_queue;
- error = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
- &ctrl->async_event_sqe, sizeof(struct nvme_command),
- DMA_TO_DEVICE);
- if (error)
- goto out_cleanup_queue;
-
return 0;
out_cleanup_queue:
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 03/12] nvme-rdma: split nvme_rdma_alloc_io_queues
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
We're going to call it from the core, so split
nr_io_queues setting to the call-sites.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/rdma.c | 30 +++++++++++++++++++-----------
1 file changed, 19 insertions(+), 11 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index d94b4df364a5..a0981fd5291c 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -643,12 +643,11 @@ static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
return ret;
}
-static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
+static unsigned int nvme_rdma_nr_io_queues(struct nvme_rdma_ctrl *ctrl)
{
struct nvmf_ctrl_options *opts = ctrl->ctrl.opts;
struct ib_device *ibdev = ctrl->device->dev;
unsigned int nr_io_queues;
- int i, ret;
nr_io_queues = min(opts->nr_io_queues, num_online_cpus());
@@ -660,16 +659,12 @@ static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
nr_io_queues = min_t(unsigned int, nr_io_queues,
ibdev->num_comp_vectors);
- ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
- if (ret)
- return ret;
-
- ctrl->ctrl.queue_count = nr_io_queues + 1;
- if (ctrl->ctrl.queue_count < 2)
- return 0;
+ return nr_io_queues;
+}
- dev_info(ctrl->ctrl.device,
- "creating %d I/O queues.\n", nr_io_queues);
+static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
+{
+ int i, ret;
for (i = 1; i < ctrl->ctrl.queue_count; i++) {
ret = nvme_rdma_alloc_queue(ctrl, i,
@@ -838,8 +833,21 @@ static void nvme_rdma_destroy_io_queues(struct nvme_rdma_ctrl *ctrl,
static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
{
+ unsigned int nr_io_queues;
int ret;
+ nr_io_queues = nvme_rdma_nr_io_queues(ctrl);
+ ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
+ if (ret)
+ return ret;
+
+ ctrl->ctrl.queue_count = nr_io_queues + 1;
+ if (ctrl->ctrl.queue_count < 2)
+ return 0;
+
+ dev_info(ctrl->ctrl.device,
+ "creating %d I/O queues.\n", nr_io_queues);
+
ret = nvme_rdma_alloc_io_queues(ctrl);
if (ret)
return ret;
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 03/12] nvme-rdma: split nvme_rdma_alloc_io_queues
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
We're going to call it from the core, so split
nr_io_queues setting to the call-sites.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/rdma.c | 30 +++++++++++++++++++-----------
1 file changed, 19 insertions(+), 11 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index d94b4df364a5..a0981fd5291c 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -643,12 +643,11 @@ static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
return ret;
}
-static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
+static unsigned int nvme_rdma_nr_io_queues(struct nvme_rdma_ctrl *ctrl)
{
struct nvmf_ctrl_options *opts = ctrl->ctrl.opts;
struct ib_device *ibdev = ctrl->device->dev;
unsigned int nr_io_queues;
- int i, ret;
nr_io_queues = min(opts->nr_io_queues, num_online_cpus());
@@ -660,16 +659,12 @@ static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
nr_io_queues = min_t(unsigned int, nr_io_queues,
ibdev->num_comp_vectors);
- ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
- if (ret)
- return ret;
-
- ctrl->ctrl.queue_count = nr_io_queues + 1;
- if (ctrl->ctrl.queue_count < 2)
- return 0;
+ return nr_io_queues;
+}
- dev_info(ctrl->ctrl.device,
- "creating %d I/O queues.\n", nr_io_queues);
+static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
+{
+ int i, ret;
for (i = 1; i < ctrl->ctrl.queue_count; i++) {
ret = nvme_rdma_alloc_queue(ctrl, i,
@@ -838,8 +833,21 @@ static void nvme_rdma_destroy_io_queues(struct nvme_rdma_ctrl *ctrl,
static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
{
+ unsigned int nr_io_queues;
int ret;
+ nr_io_queues = nvme_rdma_nr_io_queues(ctrl);
+ ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
+ if (ret)
+ return ret;
+
+ ctrl->ctrl.queue_count = nr_io_queues + 1;
+ if (ctrl->ctrl.queue_count < 2)
+ return 0;
+
+ dev_info(ctrl->ctrl.device,
+ "creating %d I/O queues.\n", nr_io_queues);
+
ret = nvme_rdma_alloc_io_queues(ctrl);
if (ret)
return ret;
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 04/12] nvme-rdma: restructure create_ctrl a bit
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
We'd like to split the generic part out, so rearrange
to ease the split. post_configure will be called after the
controller basic configuration occured and identification.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/rdma.c | 89 ++++++++++++++++++++++++++----------------------
1 file changed, 48 insertions(+), 41 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index a0981fd5291c..dc903625a759 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1844,6 +1844,39 @@ static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
.get_address = nvmf_get_address,
};
+static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
+{
+ /* sanity check icdoff */
+ if (ctrl->icdoff) {
+ dev_err(ctrl->device, "icdoff is not supported!\n");
+ return -EINVAL;
+ }
+
+ /* sanity check keyed sgls */
+ if (!(ctrl->sgls & (1 << 20))) {
+ dev_err(ctrl->device, "Mandatory keyed sgls are not support\n");
+ return -EINVAL;
+ }
+
+ if (ctrl->opts->queue_size > ctrl->maxcmd) {
+ /* warn if maxcmd is lower than queue_size */
+ dev_warn(ctrl->device,
+ "queue_size %zu > ctrl maxcmd %u, clamping down\n",
+ ctrl->opts->queue_size, ctrl->maxcmd);
+ ctrl->opts->queue_size = ctrl->maxcmd;
+ }
+
+ if (ctrl->opts->queue_size > ctrl->sqsize + 1) {
+ /* warn if sqsize is lower than queue_size */
+ dev_warn(ctrl->device,
+ "queue_size %zu > ctrl sqsize %u, clamping down\n",
+ ctrl->opts->queue_size, ctrl->sqsize + 1);
+ ctrl->opts->queue_size = ctrl->sqsize + 1;
+ }
+
+ return 0;
+}
+
static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
@@ -1855,7 +1888,11 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
if (!ctrl)
return ERR_PTR(-ENOMEM);
+
ctrl->ctrl.opts = opts;
+ ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
+ ctrl->ctrl.sqsize = opts->queue_size - 1;
+ ctrl->ctrl.kato = opts->kato;
INIT_LIST_HEAD(&ctrl->list);
if (opts->mask & NVMF_OPT_TRSVCID)
@@ -1880,10 +1917,16 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
}
}
+ ret = -ENOMEM;
+ ctrl->queues = kcalloc(ctrl->ctrl.queue_count, sizeof(*ctrl->queues),
+ GFP_KERNEL);
+ if (!ctrl->queues)
+ goto out_free_ctrl;
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops,
0 /* no quirks, we're perfect! */);
if (ret)
- goto out_free_ctrl;
+ goto out_kfree_queues;
INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
nvme_rdma_reconnect_ctrl_work);
@@ -1891,49 +1934,13 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
- ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
- ctrl->ctrl.sqsize = opts->queue_size - 1;
- ctrl->ctrl.kato = opts->kato;
-
- ret = -ENOMEM;
- ctrl->queues = kcalloc(ctrl->ctrl.queue_count, sizeof(*ctrl->queues),
- GFP_KERNEL);
- if (!ctrl->queues)
- goto out_uninit_ctrl;
-
ret = nvme_rdma_configure_admin_queue(ctrl, true);
if (ret)
- goto out_kfree_queues;
-
- /* sanity check icdoff */
- if (ctrl->ctrl.icdoff) {
- dev_err(ctrl->ctrl.device, "icdoff is not supported!\n");
- ret = -EINVAL;
- goto out_remove_admin_queue;
- }
-
- /* sanity check keyed sgls */
- if (!(ctrl->ctrl.sgls & (1 << 20))) {
- dev_err(ctrl->ctrl.device, "Mandatory keyed sgls are not support\n");
- ret = -EINVAL;
- goto out_remove_admin_queue;
- }
-
- if (opts->queue_size > ctrl->ctrl.maxcmd) {
- /* warn if maxcmd is lower than queue_size */
- dev_warn(ctrl->ctrl.device,
- "queue_size %zu > ctrl maxcmd %u, clamping down\n",
- opts->queue_size, ctrl->ctrl.maxcmd);
- opts->queue_size = ctrl->ctrl.maxcmd;
- }
+ goto out_uninit_ctrl;
- if (opts->queue_size > ctrl->ctrl.sqsize + 1) {
- /* warn if sqsize is lower than queue_size */
- dev_warn(ctrl->ctrl.device,
- "queue_size %zu > ctrl sqsize %u, clamping down\n",
- opts->queue_size, ctrl->ctrl.sqsize + 1);
- opts->queue_size = ctrl->ctrl.sqsize + 1;
- }
+ ret = nvme_rdma_post_configure(&ctrl->ctrl);
+ if (ret)
+ goto out_uninit_ctrl;
if (opts->nr_io_queues) {
ret = nvme_rdma_configure_io_queues(ctrl, true);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 04/12] nvme-rdma: restructure create_ctrl a bit
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
We'd like to split the generic part out, so rearrange
to ease the split. post_configure will be called after the
controller basic configuration occured and identification.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/rdma.c | 89 ++++++++++++++++++++++++++----------------------
1 file changed, 48 insertions(+), 41 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index a0981fd5291c..dc903625a759 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1844,6 +1844,39 @@ static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
.get_address = nvmf_get_address,
};
+static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
+{
+ /* sanity check icdoff */
+ if (ctrl->icdoff) {
+ dev_err(ctrl->device, "icdoff is not supported!\n");
+ return -EINVAL;
+ }
+
+ /* sanity check keyed sgls */
+ if (!(ctrl->sgls & (1 << 20))) {
+ dev_err(ctrl->device, "Mandatory keyed sgls are not support\n");
+ return -EINVAL;
+ }
+
+ if (ctrl->opts->queue_size > ctrl->maxcmd) {
+ /* warn if maxcmd is lower than queue_size */
+ dev_warn(ctrl->device,
+ "queue_size %zu > ctrl maxcmd %u, clamping down\n",
+ ctrl->opts->queue_size, ctrl->maxcmd);
+ ctrl->opts->queue_size = ctrl->maxcmd;
+ }
+
+ if (ctrl->opts->queue_size > ctrl->sqsize + 1) {
+ /* warn if sqsize is lower than queue_size */
+ dev_warn(ctrl->device,
+ "queue_size %zu > ctrl sqsize %u, clamping down\n",
+ ctrl->opts->queue_size, ctrl->sqsize + 1);
+ ctrl->opts->queue_size = ctrl->sqsize + 1;
+ }
+
+ return 0;
+}
+
static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
@@ -1855,7 +1888,11 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
if (!ctrl)
return ERR_PTR(-ENOMEM);
+
ctrl->ctrl.opts = opts;
+ ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
+ ctrl->ctrl.sqsize = opts->queue_size - 1;
+ ctrl->ctrl.kato = opts->kato;
INIT_LIST_HEAD(&ctrl->list);
if (opts->mask & NVMF_OPT_TRSVCID)
@@ -1880,10 +1917,16 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
}
}
+ ret = -ENOMEM;
+ ctrl->queues = kcalloc(ctrl->ctrl.queue_count, sizeof(*ctrl->queues),
+ GFP_KERNEL);
+ if (!ctrl->queues)
+ goto out_free_ctrl;
+
ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops,
0 /* no quirks, we're perfect! */);
if (ret)
- goto out_free_ctrl;
+ goto out_kfree_queues;
INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
nvme_rdma_reconnect_ctrl_work);
@@ -1891,49 +1934,13 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
- ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
- ctrl->ctrl.sqsize = opts->queue_size - 1;
- ctrl->ctrl.kato = opts->kato;
-
- ret = -ENOMEM;
- ctrl->queues = kcalloc(ctrl->ctrl.queue_count, sizeof(*ctrl->queues),
- GFP_KERNEL);
- if (!ctrl->queues)
- goto out_uninit_ctrl;
-
ret = nvme_rdma_configure_admin_queue(ctrl, true);
if (ret)
- goto out_kfree_queues;
-
- /* sanity check icdoff */
- if (ctrl->ctrl.icdoff) {
- dev_err(ctrl->ctrl.device, "icdoff is not supported!\n");
- ret = -EINVAL;
- goto out_remove_admin_queue;
- }
-
- /* sanity check keyed sgls */
- if (!(ctrl->ctrl.sgls & (1 << 20))) {
- dev_err(ctrl->ctrl.device, "Mandatory keyed sgls are not support\n");
- ret = -EINVAL;
- goto out_remove_admin_queue;
- }
-
- if (opts->queue_size > ctrl->ctrl.maxcmd) {
- /* warn if maxcmd is lower than queue_size */
- dev_warn(ctrl->ctrl.device,
- "queue_size %zu > ctrl maxcmd %u, clamping down\n",
- opts->queue_size, ctrl->ctrl.maxcmd);
- opts->queue_size = ctrl->ctrl.maxcmd;
- }
+ goto out_uninit_ctrl;
- if (opts->queue_size > ctrl->ctrl.sqsize + 1) {
- /* warn if sqsize is lower than queue_size */
- dev_warn(ctrl->ctrl.device,
- "queue_size %zu > ctrl sqsize %u, clamping down\n",
- opts->queue_size, ctrl->ctrl.sqsize + 1);
- opts->queue_size = ctrl->ctrl.sqsize + 1;
- }
+ ret = nvme_rdma_post_configure(&ctrl->ctrl);
+ if (ret)
+ goto out_uninit_ctrl;
if (opts->nr_io_queues) {
ret = nvme_rdma_configure_io_queues(ctrl, true);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 05/12] nvme-rdma: introduce nvme_rdma_alloc/stop/free_admin_queue
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
The core will eventually call this type of callouts
to allocate/stop/free the HW admin queue.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/rdma.c | 81 ++++++++++++++++++++++++++++++++----------------
1 file changed, 55 insertions(+), 26 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index dc903625a759..8a8163c82733 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -544,23 +544,6 @@ static int nvme_rdma_alloc_queue(struct nvme_rdma_ctrl *ctrl,
goto out_destroy_cm_id;
}
- if (!idx) {
- ctrl->device = ctrl->queues[0].device;
- ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
- ctrl->device->dev->attrs.max_fast_reg_page_list_len);
- ctrl->ctrl.max_hw_sectors =
- (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
-
- ret = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
- &ctrl->async_event_sqe, sizeof(struct nvme_command),
- DMA_TO_DEVICE);
- if (ret) {
- nvme_rdma_destroy_queue_ib(&ctrl->queues[0]);
- goto out_destroy_cm_id;
- }
-
- }
-
clear_bit(NVME_RDMA_Q_DELETING, &queue->flags);
return 0;
@@ -579,19 +562,32 @@ static void nvme_rdma_stop_queue(struct nvme_rdma_queue *queue)
ib_drain_qp(queue->qp);
}
+static void nvme_rdma_stop_admin_queue(struct nvme_ctrl *ctrl)
+{
+ nvme_rdma_stop_queue(&to_rdma_ctrl(ctrl)->queues[0]);
+}
+
static void nvme_rdma_free_queue(struct nvme_rdma_queue *queue)
{
if (test_and_set_bit(NVME_RDMA_Q_DELETING, &queue->flags))
return;
- if (!nvme_rdma_queue_idx(queue))
- nvme_rdma_free_qe(queue->device->dev,
- &queue->ctrl->async_event_sqe,
- sizeof(struct nvme_command), DMA_TO_DEVICE);
nvme_rdma_destroy_queue_ib(queue);
rdma_destroy_id(queue->cm_id);
}
+static void nvme_rdma_free_admin_queue(struct nvme_ctrl *nctrl)
+{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
+ struct nvme_rdma_queue *queue = &ctrl->queues[0];
+
+ nvme_rdma_free_qe(queue->device->dev,
+ &queue->ctrl->async_event_sqe,
+ sizeof(struct nvme_command), DMA_TO_DEVICE);
+
+ nvme_rdma_free_queue(queue);
+}
+
static void nvme_rdma_free_io_queues(struct nvme_rdma_ctrl *ctrl)
{
int i;
@@ -625,6 +621,11 @@ static int nvme_rdma_start_queue(struct nvme_rdma_ctrl *ctrl, int idx)
return ret;
}
+static int nvme_rdma_start_admin_queue(struct nvme_ctrl *ctrl)
+{
+ return nvme_rdma_start_queue(to_rdma_ctrl(ctrl), 0);
+}
+
static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
{
int i, ret = 0;
@@ -751,12 +752,40 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_queue(&ctrl->queues[0]);
+ nvme_rdma_stop_admin_queue(&ctrl->ctrl);
if (remove) {
blk_cleanup_queue(ctrl->ctrl.admin_q);
nvme_rdma_free_tagset(&ctrl->ctrl, true);
}
+ nvme_rdma_free_admin_queue(&ctrl->ctrl);
+}
+
+static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
+{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
+ int ret;
+
+ ret = nvme_rdma_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
+ if (ret)
+ return ret;
+
+ ctrl->device = ctrl->queues[0].device;
+ ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
+ ctrl->device->dev->attrs.max_fast_reg_page_list_len);
+ ctrl->ctrl.max_hw_sectors =
+ (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
+
+ ret = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
+ &ctrl->async_event_sqe, sizeof(struct nvme_command),
+ DMA_TO_DEVICE);
+ if (ret)
+ goto out_free_queue;
+
+ return 0;
+
+out_free_queue:
nvme_rdma_free_queue(&ctrl->queues[0]);
+ return ret;
}
static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
@@ -764,7 +793,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
{
int error;
- error = nvme_rdma_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
+ error = nvme_rdma_alloc_admin_queue(&ctrl->ctrl);
if (error)
return error;
@@ -784,7 +813,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
goto out_free_queue;
}
- error = nvme_rdma_start_queue(ctrl, 0);
+ error = nvme_rdma_start_admin_queue(&ctrl->ctrl);
if (error)
goto out_cleanup_queue;
@@ -816,7 +845,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
if (new)
nvme_rdma_free_tagset(&ctrl->ctrl, true);
out_free_queue:
- nvme_rdma_free_queue(&ctrl->queues[0]);
+ nvme_rdma_free_admin_queue(&ctrl->ctrl);
return error;
}
@@ -976,7 +1005,7 @@ static void nvme_rdma_error_recovery_work(struct work_struct *work)
nvme_rdma_stop_io_queues(ctrl);
}
blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- nvme_rdma_stop_queue(&ctrl->queues[0]);
+ nvme_rdma_stop_admin_queue(&ctrl->ctrl);
/* We must take care of fastfail/requeue all our inflight requests */
if (ctrl->ctrl.queue_count > 1)
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 05/12] nvme-rdma: introduce nvme_rdma_alloc/stop/free_admin_queue
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
The core will eventually call this type of callouts
to allocate/stop/free the HW admin queue.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/rdma.c | 81 ++++++++++++++++++++++++++++++++----------------
1 file changed, 55 insertions(+), 26 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index dc903625a759..8a8163c82733 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -544,23 +544,6 @@ static int nvme_rdma_alloc_queue(struct nvme_rdma_ctrl *ctrl,
goto out_destroy_cm_id;
}
- if (!idx) {
- ctrl->device = ctrl->queues[0].device;
- ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
- ctrl->device->dev->attrs.max_fast_reg_page_list_len);
- ctrl->ctrl.max_hw_sectors =
- (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
-
- ret = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
- &ctrl->async_event_sqe, sizeof(struct nvme_command),
- DMA_TO_DEVICE);
- if (ret) {
- nvme_rdma_destroy_queue_ib(&ctrl->queues[0]);
- goto out_destroy_cm_id;
- }
-
- }
-
clear_bit(NVME_RDMA_Q_DELETING, &queue->flags);
return 0;
@@ -579,19 +562,32 @@ static void nvme_rdma_stop_queue(struct nvme_rdma_queue *queue)
ib_drain_qp(queue->qp);
}
+static void nvme_rdma_stop_admin_queue(struct nvme_ctrl *ctrl)
+{
+ nvme_rdma_stop_queue(&to_rdma_ctrl(ctrl)->queues[0]);
+}
+
static void nvme_rdma_free_queue(struct nvme_rdma_queue *queue)
{
if (test_and_set_bit(NVME_RDMA_Q_DELETING, &queue->flags))
return;
- if (!nvme_rdma_queue_idx(queue))
- nvme_rdma_free_qe(queue->device->dev,
- &queue->ctrl->async_event_sqe,
- sizeof(struct nvme_command), DMA_TO_DEVICE);
nvme_rdma_destroy_queue_ib(queue);
rdma_destroy_id(queue->cm_id);
}
+static void nvme_rdma_free_admin_queue(struct nvme_ctrl *nctrl)
+{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
+ struct nvme_rdma_queue *queue = &ctrl->queues[0];
+
+ nvme_rdma_free_qe(queue->device->dev,
+ &queue->ctrl->async_event_sqe,
+ sizeof(struct nvme_command), DMA_TO_DEVICE);
+
+ nvme_rdma_free_queue(queue);
+}
+
static void nvme_rdma_free_io_queues(struct nvme_rdma_ctrl *ctrl)
{
int i;
@@ -625,6 +621,11 @@ static int nvme_rdma_start_queue(struct nvme_rdma_ctrl *ctrl, int idx)
return ret;
}
+static int nvme_rdma_start_admin_queue(struct nvme_ctrl *ctrl)
+{
+ return nvme_rdma_start_queue(to_rdma_ctrl(ctrl), 0);
+}
+
static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
{
int i, ret = 0;
@@ -751,12 +752,40 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_queue(&ctrl->queues[0]);
+ nvme_rdma_stop_admin_queue(&ctrl->ctrl);
if (remove) {
blk_cleanup_queue(ctrl->ctrl.admin_q);
nvme_rdma_free_tagset(&ctrl->ctrl, true);
}
+ nvme_rdma_free_admin_queue(&ctrl->ctrl);
+}
+
+static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
+{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
+ int ret;
+
+ ret = nvme_rdma_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
+ if (ret)
+ return ret;
+
+ ctrl->device = ctrl->queues[0].device;
+ ctrl->max_fr_pages = min_t(u32, NVME_RDMA_MAX_SEGMENTS,
+ ctrl->device->dev->attrs.max_fast_reg_page_list_len);
+ ctrl->ctrl.max_hw_sectors =
+ (ctrl->max_fr_pages - 1) << (PAGE_SHIFT - 9);
+
+ ret = nvme_rdma_alloc_qe(ctrl->queues[0].device->dev,
+ &ctrl->async_event_sqe, sizeof(struct nvme_command),
+ DMA_TO_DEVICE);
+ if (ret)
+ goto out_free_queue;
+
+ return 0;
+
+out_free_queue:
nvme_rdma_free_queue(&ctrl->queues[0]);
+ return ret;
}
static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
@@ -764,7 +793,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
{
int error;
- error = nvme_rdma_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
+ error = nvme_rdma_alloc_admin_queue(&ctrl->ctrl);
if (error)
return error;
@@ -784,7 +813,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
goto out_free_queue;
}
- error = nvme_rdma_start_queue(ctrl, 0);
+ error = nvme_rdma_start_admin_queue(&ctrl->ctrl);
if (error)
goto out_cleanup_queue;
@@ -816,7 +845,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
if (new)
nvme_rdma_free_tagset(&ctrl->ctrl, true);
out_free_queue:
- nvme_rdma_free_queue(&ctrl->queues[0]);
+ nvme_rdma_free_admin_queue(&ctrl->ctrl);
return error;
}
@@ -976,7 +1005,7 @@ static void nvme_rdma_error_recovery_work(struct work_struct *work)
nvme_rdma_stop_io_queues(ctrl);
}
blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- nvme_rdma_stop_queue(&ctrl->queues[0]);
+ nvme_rdma_stop_admin_queue(&ctrl->ctrl);
/* We must take care of fastfail/requeue all our inflight requests */
if (ctrl->ctrl.queue_count > 1)
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 06/12] nvme-rdma: plumb nvme ctrl to various routines
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
These routines will move to the nvme core, so make
sure they are passed with a core nvme controller struct.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/rdma.c | 264 ++++++++++++++++++++++++-----------------------
1 file changed, 133 insertions(+), 131 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 8a8163c82733..2591b0ce155e 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -588,16 +588,18 @@ static void nvme_rdma_free_admin_queue(struct nvme_ctrl *nctrl)
nvme_rdma_free_queue(queue);
}
-static void nvme_rdma_free_io_queues(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_free_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i;
for (i = 1; i < ctrl->ctrl.queue_count; i++)
nvme_rdma_free_queue(&ctrl->queues[i]);
}
-static void nvme_rdma_stop_io_queues(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_stop_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i;
for (i = 1; i < ctrl->ctrl.queue_count; i++)
@@ -626,8 +628,9 @@ static int nvme_rdma_start_admin_queue(struct nvme_ctrl *ctrl)
return nvme_rdma_start_queue(to_rdma_ctrl(ctrl), 0);
}
-static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
+static int nvme_rdma_start_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i, ret = 0;
for (i = 1; i < ctrl->ctrl.queue_count; i++) {
@@ -644,9 +647,10 @@ static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
return ret;
}
-static unsigned int nvme_rdma_nr_io_queues(struct nvme_rdma_ctrl *ctrl)
+static unsigned int nvme_rdma_nr_io_queues(struct nvme_ctrl *nctrl)
{
- struct nvmf_ctrl_options *opts = ctrl->ctrl.opts;
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
+ struct nvmf_ctrl_options *opts = nctrl->opts;
struct ib_device *ibdev = ctrl->device->dev;
unsigned int nr_io_queues;
@@ -663,8 +667,9 @@ static unsigned int nvme_rdma_nr_io_queues(struct nvme_rdma_ctrl *ctrl)
return nr_io_queues;
}
-static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
+static int nvme_rdma_alloc_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i, ret;
for (i = 1; i < ctrl->ctrl.queue_count; i++) {
@@ -749,15 +754,15 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
return ERR_PTR(ret);
}
-static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
+static void nvme_rdma_destroy_admin_queue(struct nvme_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_admin_queue(&ctrl->ctrl);
+ nvme_rdma_stop_admin_queue(ctrl);
if (remove) {
- blk_cleanup_queue(ctrl->ctrl.admin_q);
- nvme_rdma_free_tagset(&ctrl->ctrl, true);
+ blk_cleanup_queue(ctrl->admin_q);
+ nvme_rdma_free_tagset(ctrl, true);
}
- nvme_rdma_free_admin_queue(&ctrl->ctrl);
+ nvme_rdma_free_admin_queue(ctrl);
}
static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
@@ -788,51 +793,49 @@ static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
return ret;
}
-static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
+static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
bool new)
{
int error;
- error = nvme_rdma_alloc_admin_queue(&ctrl->ctrl);
+ error = nvme_rdma_alloc_admin_queue(ctrl);
if (error)
return error;
if (new) {
- ctrl->ctrl.admin_tagset = nvme_rdma_alloc_tagset(&ctrl->ctrl, true);
- if (IS_ERR(ctrl->ctrl.admin_tagset))
+ ctrl->admin_tagset = nvme_rdma_alloc_tagset(ctrl, true);
+ if (IS_ERR(ctrl->admin_tagset))
goto out_free_queue;
- ctrl->ctrl.admin_q = blk_mq_init_queue(&ctrl->admin_tag_set);
- if (IS_ERR(ctrl->ctrl.admin_q)) {
- error = PTR_ERR(ctrl->ctrl.admin_q);
+ ctrl->admin_q = blk_mq_init_queue(ctrl->admin_tagset);
+ if (IS_ERR(ctrl->admin_q)) {
+ error = PTR_ERR(ctrl->admin_q);
goto out_free_tagset;
}
} else {
- error = blk_mq_reinit_tagset(&ctrl->admin_tag_set);
+ error = blk_mq_reinit_tagset(ctrl->admin_tagset);
if (error)
goto out_free_queue;
}
- error = nvme_rdma_start_admin_queue(&ctrl->ctrl);
+ error = nvme_rdma_start_admin_queue(ctrl);
if (error)
goto out_cleanup_queue;
- error = ctrl->ctrl.ops->reg_read64(&ctrl->ctrl, NVME_REG_CAP,
- &ctrl->ctrl.cap);
+ error = ctrl->ops->reg_read64(ctrl, NVME_REG_CAP,
+ &ctrl->cap);
if (error) {
- dev_err(ctrl->ctrl.device,
- "prop_get NVME_REG_CAP failed\n");
+ dev_err(ctrl->device, "prop_get NVME_REG_CAP failed\n");
goto out_cleanup_queue;
}
- ctrl->ctrl.sqsize =
- min_t(int, NVME_CAP_MQES(ctrl->ctrl.cap), ctrl->ctrl.sqsize);
+ ctrl->sqsize = min_t(int, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
- error = nvme_enable_ctrl(&ctrl->ctrl, ctrl->ctrl.cap);
+ error = nvme_enable_ctrl(ctrl, ctrl->cap);
if (error)
goto out_cleanup_queue;
- error = nvme_init_identify(&ctrl->ctrl);
+ error = nvme_init_identify(ctrl);
if (error)
goto out_cleanup_queue;
@@ -840,41 +843,41 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
out_cleanup_queue:
if (new)
- blk_cleanup_queue(ctrl->ctrl.admin_q);
+ blk_cleanup_queue(ctrl->admin_q);
out_free_tagset:
if (new)
- nvme_rdma_free_tagset(&ctrl->ctrl, true);
+ nvme_rdma_free_tagset(ctrl, true);
out_free_queue:
- nvme_rdma_free_admin_queue(&ctrl->ctrl);
+ nvme_rdma_free_admin_queue(ctrl);
return error;
}
-static void nvme_rdma_destroy_io_queues(struct nvme_rdma_ctrl *ctrl,
+static void nvme_rdma_destroy_io_queues(struct nvme_ctrl *ctrl,
bool remove)
{
nvme_rdma_stop_io_queues(ctrl);
if (remove) {
- blk_cleanup_queue(ctrl->ctrl.connect_q);
- nvme_rdma_free_tagset(&ctrl->ctrl, false);
+ blk_cleanup_queue(ctrl->connect_q);
+ nvme_rdma_free_tagset(ctrl, false);
}
nvme_rdma_free_io_queues(ctrl);
}
-static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
+static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
{
unsigned int nr_io_queues;
int ret;
nr_io_queues = nvme_rdma_nr_io_queues(ctrl);
- ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
+ ret = nvme_set_queue_count(ctrl, &nr_io_queues);
if (ret)
return ret;
- ctrl->ctrl.queue_count = nr_io_queues + 1;
- if (ctrl->ctrl.queue_count < 2)
+ ctrl->queue_count = nr_io_queues + 1;
+ if (ctrl->queue_count < 2)
return 0;
- dev_info(ctrl->ctrl.device,
+ dev_info(ctrl->device,
"creating %d I/O queues.\n", nr_io_queues);
ret = nvme_rdma_alloc_io_queues(ctrl);
@@ -882,22 +885,22 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
return ret;
if (new) {
- ctrl->ctrl.tagset = nvme_rdma_alloc_tagset(&ctrl->ctrl, false);
- if (IS_ERR(ctrl->ctrl.tagset))
+ ctrl->tagset = nvme_rdma_alloc_tagset(ctrl, false);
+ if (IS_ERR(ctrl->tagset))
goto out_free_io_queues;
- ctrl->ctrl.connect_q = blk_mq_init_queue(&ctrl->tag_set);
- if (IS_ERR(ctrl->ctrl.connect_q)) {
- ret = PTR_ERR(ctrl->ctrl.connect_q);
+ ctrl->connect_q = blk_mq_init_queue(ctrl->tagset);
+ if (IS_ERR(ctrl->connect_q)) {
+ ret = PTR_ERR(ctrl->connect_q);
goto out_free_tag_set;
}
} else {
- ret = blk_mq_reinit_tagset(&ctrl->tag_set);
+ ret = blk_mq_reinit_tagset(ctrl->tagset);
if (ret)
goto out_free_io_queues;
- blk_mq_update_nr_hw_queues(&ctrl->tag_set,
- ctrl->ctrl.queue_count - 1);
+ blk_mq_update_nr_hw_queues(ctrl->tagset,
+ ctrl->queue_count - 1);
}
ret = nvme_rdma_start_io_queues(ctrl);
@@ -908,10 +911,10 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
out_cleanup_connect_q:
if (new)
- blk_cleanup_queue(ctrl->ctrl.connect_q);
+ blk_cleanup_queue(ctrl->connect_q);
out_free_tag_set:
if (new)
- nvme_rdma_free_tagset(&ctrl->ctrl, false);
+ nvme_rdma_free_tagset(ctrl, false);
out_free_io_queues:
nvme_rdma_free_io_queues(ctrl);
return ret;
@@ -934,36 +937,36 @@ static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
kfree(ctrl);
}
-static void nvme_rdma_reconnect_or_remove(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_reconnect_or_remove(struct nvme_ctrl *ctrl)
{
/* If we are resetting/deleting then do nothing */
- if (ctrl->ctrl.state != NVME_CTRL_RECONNECTING) {
- WARN_ON_ONCE(ctrl->ctrl.state == NVME_CTRL_NEW ||
- ctrl->ctrl.state == NVME_CTRL_LIVE);
+ if (ctrl->state != NVME_CTRL_RECONNECTING) {
+ WARN_ON_ONCE(ctrl->state == NVME_CTRL_NEW ||
+ ctrl->state == NVME_CTRL_LIVE);
return;
}
- if (nvmf_should_reconnect(&ctrl->ctrl)) {
- dev_info(ctrl->ctrl.device, "Reconnecting in %d seconds...\n",
- ctrl->ctrl.opts->reconnect_delay);
- queue_delayed_work(nvme_wq, &ctrl->ctrl.reconnect_work,
- ctrl->ctrl.opts->reconnect_delay * HZ);
+ if (nvmf_should_reconnect(ctrl)) {
+ dev_info(ctrl->device, "Reconnecting in %d seconds...\n",
+ ctrl->opts->reconnect_delay);
+ queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
+ ctrl->opts->reconnect_delay * HZ);
} else {
- dev_info(ctrl->ctrl.device, "Removing controller...\n");
- queue_work(nvme_wq, &ctrl->ctrl.delete_work);
+ dev_info(ctrl->device, "Removing controller...\n");
+ queue_work(nvme_wq, &ctrl->delete_work);
}
}
static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl = container_of(to_delayed_work(work),
- struct nvme_rdma_ctrl, ctrl.reconnect_work);
+ struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
+ struct nvme_ctrl, reconnect_work);
bool changed;
int ret;
- ++ctrl->ctrl.nr_reconnects;
+ ++ctrl->nr_reconnects;
- if (ctrl->ctrl.queue_count > 1)
+ if (ctrl->queue_count > 1)
nvme_rdma_destroy_io_queues(ctrl, false);
nvme_rdma_destroy_admin_queue(ctrl, false);
@@ -971,65 +974,65 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
if (ret)
goto requeue;
- if (ctrl->ctrl.queue_count > 1) {
+ if (ctrl->queue_count > 1) {
ret = nvme_rdma_configure_io_queues(ctrl, false);
if (ret)
goto requeue;
}
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
WARN_ON_ONCE(!changed);
- ctrl->ctrl.nr_reconnects = 0;
+ ctrl->nr_reconnects = 0;
- nvme_start_ctrl(&ctrl->ctrl);
+ nvme_start_ctrl(ctrl);
- dev_info(ctrl->ctrl.device, "Successfully reconnected\n");
+ dev_info(ctrl->device, "Successfully reconnected\n");
return;
requeue:
- dev_info(ctrl->ctrl.device, "Failed reconnect attempt %d\n",
- ctrl->ctrl.nr_reconnects);
+ dev_info(ctrl->device, "Failed reconnect attempt %d\n",
+ ctrl->nr_reconnects);
nvme_rdma_reconnect_or_remove(ctrl);
}
static void nvme_rdma_error_recovery_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, ctrl.err_work);
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, err_work);
- nvme_stop_ctrl(&ctrl->ctrl);
+ nvme_stop_ctrl(ctrl);
- if (ctrl->ctrl.queue_count > 1) {
- nvme_stop_queues(&ctrl->ctrl);
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
nvme_rdma_stop_io_queues(ctrl);
}
- blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- nvme_rdma_stop_admin_queue(&ctrl->ctrl);
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ nvme_rdma_stop_admin_queue(ctrl);
/* We must take care of fastfail/requeue all our inflight requests */
- if (ctrl->ctrl.queue_count > 1)
- blk_mq_tagset_busy_iter(&ctrl->tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
- nvme_cancel_request, &ctrl->ctrl);
+ if (ctrl->queue_count > 1)
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
/*
* queues are not a live anymore, so restart the queues to fail fast
* new IO
*/
- blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
- nvme_start_queues(&ctrl->ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
+ nvme_start_queues(ctrl);
nvme_rdma_reconnect_or_remove(ctrl);
}
-static void nvme_rdma_error_recovery(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_error_recovery(struct nvme_ctrl *ctrl)
{
- if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_RECONNECTING))
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RECONNECTING))
return;
- queue_work(nvme_wq, &ctrl->ctrl.err_work);
+ queue_work(nvme_wq, &ctrl->err_work);
}
static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
@@ -1043,7 +1046,7 @@ static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
"%s for CQE 0x%p failed with status %s (%d)\n",
op, wc->wr_cqe,
ib_wc_status_msg(wc->status), wc->status);
- nvme_rdma_error_recovery(ctrl);
+ nvme_rdma_error_recovery(&ctrl->ctrl);
}
static void nvme_rdma_memreg_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -1094,7 +1097,7 @@ static void nvme_rdma_unmap_data(struct nvme_rdma_queue *queue,
dev_err(ctrl->ctrl.device,
"Queueing INV WR for rkey %#x failed (%d)\n",
req->mr->rkey, res);
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
}
}
@@ -1373,7 +1376,7 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
dev_err(queue->ctrl->ctrl.device,
"tag 0x%x on QP %#x not found\n",
cqe->command_id, queue->qp->qp_num);
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
return ret;
}
req = blk_mq_rq_to_pdu(rq);
@@ -1584,7 +1587,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
case RDMA_CM_EVENT_TIMEWAIT_EXIT:
dev_dbg(queue->ctrl->ctrl.device,
"disconnect received - connection closed\n");
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
break;
case RDMA_CM_EVENT_DEVICE_REMOVAL:
/* device removal is handled via the ib_client API */
@@ -1592,7 +1595,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
default:
dev_err(queue->ctrl->ctrl.device,
"Unexpected RDMA CM event (%d)\n", ev->event);
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
break;
}
@@ -1610,7 +1613,7 @@ nvme_rdma_timeout(struct request *rq, bool reserved)
struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq);
/* queue error recovery */
- nvme_rdma_error_recovery(req->queue->ctrl);
+ nvme_rdma_error_recovery(&req->queue->ctrl->ctrl);
/* fail with DNR on cmd timeout */
nvme_req(rq)->status = NVME_SC_ABORT_REQ | NVME_SC_DNR;
@@ -1761,102 +1764,101 @@ static const struct blk_mq_ops nvme_rdma_admin_mq_ops = {
.timeout = nvme_rdma_timeout,
};
-static void nvme_rdma_shutdown_ctrl(struct nvme_rdma_ctrl *ctrl, bool shutdown)
+static void nvme_rdma_shutdown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
{
- if (ctrl->ctrl.queue_count > 1) {
- nvme_stop_queues(&ctrl->ctrl);
- blk_mq_tagset_busy_iter(&ctrl->tag_set,
- nvme_cancel_request, &ctrl->ctrl);
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
nvme_rdma_destroy_io_queues(ctrl, shutdown);
}
if (shutdown)
- nvme_shutdown_ctrl(&ctrl->ctrl);
+ nvme_shutdown_ctrl(ctrl);
else
- nvme_disable_ctrl(&ctrl->ctrl, ctrl->ctrl.cap);
+ nvme_disable_ctrl(ctrl, ctrl->cap);
- blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
nvme_rdma_destroy_admin_queue(ctrl, shutdown);
}
-static void nvme_rdma_remove_ctrl(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_remove_ctrl(struct nvme_ctrl *ctrl)
{
- nvme_remove_namespaces(&ctrl->ctrl);
+ nvme_remove_namespaces(ctrl);
nvme_rdma_shutdown_ctrl(ctrl, true);
- nvme_uninit_ctrl(&ctrl->ctrl);
- nvme_put_ctrl(&ctrl->ctrl);
+ nvme_uninit_ctrl(ctrl);
+ nvme_put_ctrl(ctrl);
}
static void nvme_rdma_del_ctrl_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, ctrl.delete_work);
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, delete_work);
- nvme_stop_ctrl(&ctrl->ctrl);
+ nvme_stop_ctrl(ctrl);
nvme_rdma_remove_ctrl(ctrl);
}
-static int __nvme_rdma_del_ctrl(struct nvme_rdma_ctrl *ctrl)
+static int __nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
{
- if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING))
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
return -EBUSY;
- if (!queue_work(nvme_wq, &ctrl->ctrl.delete_work))
+ if (!queue_work(nvme_wq, &ctrl->delete_work))
return -EBUSY;
return 0;
}
-static int nvme_rdma_del_ctrl(struct nvme_ctrl *nctrl)
+static int nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
{
- struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int ret = 0;
/*
* Keep a reference until all work is flushed since
* __nvme_rdma_del_ctrl can free the ctrl mem
*/
- if (!kref_get_unless_zero(&ctrl->ctrl.kref))
+ if (!kref_get_unless_zero(&ctrl->kref))
return -EBUSY;
ret = __nvme_rdma_del_ctrl(ctrl);
if (!ret)
- flush_work(&ctrl->ctrl.delete_work);
- nvme_put_ctrl(&ctrl->ctrl);
+ flush_work(&ctrl->delete_work);
+ nvme_put_ctrl(ctrl);
return ret;
}
static void nvme_rdma_reset_ctrl_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl =
- container_of(work, struct nvme_rdma_ctrl, ctrl.reset_work);
+ struct nvme_ctrl *ctrl =
+ container_of(work, struct nvme_ctrl, reset_work);
int ret;
bool changed;
- nvme_stop_ctrl(&ctrl->ctrl);
+ nvme_stop_ctrl(ctrl);
nvme_rdma_shutdown_ctrl(ctrl, false);
ret = nvme_rdma_configure_admin_queue(ctrl, false);
if (ret)
goto out_fail;
- if (ctrl->ctrl.queue_count > 1) {
+ if (ctrl->queue_count > 1) {
ret = nvme_rdma_configure_io_queues(ctrl, false);
if (ret)
goto out_fail;
}
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
WARN_ON_ONCE(!changed);
- nvme_start_ctrl(&ctrl->ctrl);
+ nvme_start_ctrl(ctrl);
return;
out_fail:
- dev_warn(ctrl->ctrl.device, "Removing after reset failure\n");
+ dev_warn(ctrl->device, "Removing after reset failure\n");
nvme_rdma_remove_ctrl(ctrl);
}
@@ -1963,7 +1965,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
- ret = nvme_rdma_configure_admin_queue(ctrl, true);
+ ret = nvme_rdma_configure_admin_queue(&ctrl->ctrl, true);
if (ret)
goto out_uninit_ctrl;
@@ -1972,7 +1974,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
goto out_uninit_ctrl;
if (opts->nr_io_queues) {
- ret = nvme_rdma_configure_io_queues(ctrl, true);
+ ret = nvme_rdma_configure_io_queues(&ctrl->ctrl, true);
if (ret)
goto out_remove_admin_queue;
}
@@ -1994,7 +1996,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
return &ctrl->ctrl;
out_remove_admin_queue:
- nvme_rdma_destroy_admin_queue(ctrl, true);
+ nvme_rdma_destroy_admin_queue(&ctrl->ctrl, true);
out_kfree_queues:
kfree(ctrl->queues);
out_uninit_ctrl:
@@ -2032,7 +2034,7 @@ static void nvme_rdma_remove_one(struct ib_device *ib_device, void *client_data)
dev_info(ctrl->ctrl.device,
"Removing ctrl: NQN \"%s\", addr %pISp\n",
ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
- __nvme_rdma_del_ctrl(ctrl);
+ __nvme_rdma_del_ctrl(&ctrl->ctrl);
}
mutex_unlock(&nvme_rdma_ctrl_mutex);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 06/12] nvme-rdma: plumb nvme ctrl to various routines
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
These routines will move to the nvme core, so make
sure they are passed with a core nvme controller struct.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/rdma.c | 264 ++++++++++++++++++++++++-----------------------
1 file changed, 133 insertions(+), 131 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 8a8163c82733..2591b0ce155e 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -588,16 +588,18 @@ static void nvme_rdma_free_admin_queue(struct nvme_ctrl *nctrl)
nvme_rdma_free_queue(queue);
}
-static void nvme_rdma_free_io_queues(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_free_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i;
for (i = 1; i < ctrl->ctrl.queue_count; i++)
nvme_rdma_free_queue(&ctrl->queues[i]);
}
-static void nvme_rdma_stop_io_queues(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_stop_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i;
for (i = 1; i < ctrl->ctrl.queue_count; i++)
@@ -626,8 +628,9 @@ static int nvme_rdma_start_admin_queue(struct nvme_ctrl *ctrl)
return nvme_rdma_start_queue(to_rdma_ctrl(ctrl), 0);
}
-static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
+static int nvme_rdma_start_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i, ret = 0;
for (i = 1; i < ctrl->ctrl.queue_count; i++) {
@@ -644,9 +647,10 @@ static int nvme_rdma_start_io_queues(struct nvme_rdma_ctrl *ctrl)
return ret;
}
-static unsigned int nvme_rdma_nr_io_queues(struct nvme_rdma_ctrl *ctrl)
+static unsigned int nvme_rdma_nr_io_queues(struct nvme_ctrl *nctrl)
{
- struct nvmf_ctrl_options *opts = ctrl->ctrl.opts;
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
+ struct nvmf_ctrl_options *opts = nctrl->opts;
struct ib_device *ibdev = ctrl->device->dev;
unsigned int nr_io_queues;
@@ -663,8 +667,9 @@ static unsigned int nvme_rdma_nr_io_queues(struct nvme_rdma_ctrl *ctrl)
return nr_io_queues;
}
-static int nvme_rdma_alloc_io_queues(struct nvme_rdma_ctrl *ctrl)
+static int nvme_rdma_alloc_io_queues(struct nvme_ctrl *nctrl)
{
+ struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int i, ret;
for (i = 1; i < ctrl->ctrl.queue_count; i++) {
@@ -749,15 +754,15 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
return ERR_PTR(ret);
}
-static void nvme_rdma_destroy_admin_queue(struct nvme_rdma_ctrl *ctrl,
+static void nvme_rdma_destroy_admin_queue(struct nvme_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_admin_queue(&ctrl->ctrl);
+ nvme_rdma_stop_admin_queue(ctrl);
if (remove) {
- blk_cleanup_queue(ctrl->ctrl.admin_q);
- nvme_rdma_free_tagset(&ctrl->ctrl, true);
+ blk_cleanup_queue(ctrl->admin_q);
+ nvme_rdma_free_tagset(ctrl, true);
}
- nvme_rdma_free_admin_queue(&ctrl->ctrl);
+ nvme_rdma_free_admin_queue(ctrl);
}
static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
@@ -788,51 +793,49 @@ static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
return ret;
}
-static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
+static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
bool new)
{
int error;
- error = nvme_rdma_alloc_admin_queue(&ctrl->ctrl);
+ error = nvme_rdma_alloc_admin_queue(ctrl);
if (error)
return error;
if (new) {
- ctrl->ctrl.admin_tagset = nvme_rdma_alloc_tagset(&ctrl->ctrl, true);
- if (IS_ERR(ctrl->ctrl.admin_tagset))
+ ctrl->admin_tagset = nvme_rdma_alloc_tagset(ctrl, true);
+ if (IS_ERR(ctrl->admin_tagset))
goto out_free_queue;
- ctrl->ctrl.admin_q = blk_mq_init_queue(&ctrl->admin_tag_set);
- if (IS_ERR(ctrl->ctrl.admin_q)) {
- error = PTR_ERR(ctrl->ctrl.admin_q);
+ ctrl->admin_q = blk_mq_init_queue(ctrl->admin_tagset);
+ if (IS_ERR(ctrl->admin_q)) {
+ error = PTR_ERR(ctrl->admin_q);
goto out_free_tagset;
}
} else {
- error = blk_mq_reinit_tagset(&ctrl->admin_tag_set);
+ error = blk_mq_reinit_tagset(ctrl->admin_tagset);
if (error)
goto out_free_queue;
}
- error = nvme_rdma_start_admin_queue(&ctrl->ctrl);
+ error = nvme_rdma_start_admin_queue(ctrl);
if (error)
goto out_cleanup_queue;
- error = ctrl->ctrl.ops->reg_read64(&ctrl->ctrl, NVME_REG_CAP,
- &ctrl->ctrl.cap);
+ error = ctrl->ops->reg_read64(ctrl, NVME_REG_CAP,
+ &ctrl->cap);
if (error) {
- dev_err(ctrl->ctrl.device,
- "prop_get NVME_REG_CAP failed\n");
+ dev_err(ctrl->device, "prop_get NVME_REG_CAP failed\n");
goto out_cleanup_queue;
}
- ctrl->ctrl.sqsize =
- min_t(int, NVME_CAP_MQES(ctrl->ctrl.cap), ctrl->ctrl.sqsize);
+ ctrl->sqsize = min_t(int, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
- error = nvme_enable_ctrl(&ctrl->ctrl, ctrl->ctrl.cap);
+ error = nvme_enable_ctrl(ctrl, ctrl->cap);
if (error)
goto out_cleanup_queue;
- error = nvme_init_identify(&ctrl->ctrl);
+ error = nvme_init_identify(ctrl);
if (error)
goto out_cleanup_queue;
@@ -840,41 +843,41 @@ static int nvme_rdma_configure_admin_queue(struct nvme_rdma_ctrl *ctrl,
out_cleanup_queue:
if (new)
- blk_cleanup_queue(ctrl->ctrl.admin_q);
+ blk_cleanup_queue(ctrl->admin_q);
out_free_tagset:
if (new)
- nvme_rdma_free_tagset(&ctrl->ctrl, true);
+ nvme_rdma_free_tagset(ctrl, true);
out_free_queue:
- nvme_rdma_free_admin_queue(&ctrl->ctrl);
+ nvme_rdma_free_admin_queue(ctrl);
return error;
}
-static void nvme_rdma_destroy_io_queues(struct nvme_rdma_ctrl *ctrl,
+static void nvme_rdma_destroy_io_queues(struct nvme_ctrl *ctrl,
bool remove)
{
nvme_rdma_stop_io_queues(ctrl);
if (remove) {
- blk_cleanup_queue(ctrl->ctrl.connect_q);
- nvme_rdma_free_tagset(&ctrl->ctrl, false);
+ blk_cleanup_queue(ctrl->connect_q);
+ nvme_rdma_free_tagset(ctrl, false);
}
nvme_rdma_free_io_queues(ctrl);
}
-static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
+static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
{
unsigned int nr_io_queues;
int ret;
nr_io_queues = nvme_rdma_nr_io_queues(ctrl);
- ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
+ ret = nvme_set_queue_count(ctrl, &nr_io_queues);
if (ret)
return ret;
- ctrl->ctrl.queue_count = nr_io_queues + 1;
- if (ctrl->ctrl.queue_count < 2)
+ ctrl->queue_count = nr_io_queues + 1;
+ if (ctrl->queue_count < 2)
return 0;
- dev_info(ctrl->ctrl.device,
+ dev_info(ctrl->device,
"creating %d I/O queues.\n", nr_io_queues);
ret = nvme_rdma_alloc_io_queues(ctrl);
@@ -882,22 +885,22 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
return ret;
if (new) {
- ctrl->ctrl.tagset = nvme_rdma_alloc_tagset(&ctrl->ctrl, false);
- if (IS_ERR(ctrl->ctrl.tagset))
+ ctrl->tagset = nvme_rdma_alloc_tagset(ctrl, false);
+ if (IS_ERR(ctrl->tagset))
goto out_free_io_queues;
- ctrl->ctrl.connect_q = blk_mq_init_queue(&ctrl->tag_set);
- if (IS_ERR(ctrl->ctrl.connect_q)) {
- ret = PTR_ERR(ctrl->ctrl.connect_q);
+ ctrl->connect_q = blk_mq_init_queue(ctrl->tagset);
+ if (IS_ERR(ctrl->connect_q)) {
+ ret = PTR_ERR(ctrl->connect_q);
goto out_free_tag_set;
}
} else {
- ret = blk_mq_reinit_tagset(&ctrl->tag_set);
+ ret = blk_mq_reinit_tagset(ctrl->tagset);
if (ret)
goto out_free_io_queues;
- blk_mq_update_nr_hw_queues(&ctrl->tag_set,
- ctrl->ctrl.queue_count - 1);
+ blk_mq_update_nr_hw_queues(ctrl->tagset,
+ ctrl->queue_count - 1);
}
ret = nvme_rdma_start_io_queues(ctrl);
@@ -908,10 +911,10 @@ static int nvme_rdma_configure_io_queues(struct nvme_rdma_ctrl *ctrl, bool new)
out_cleanup_connect_q:
if (new)
- blk_cleanup_queue(ctrl->ctrl.connect_q);
+ blk_cleanup_queue(ctrl->connect_q);
out_free_tag_set:
if (new)
- nvme_rdma_free_tagset(&ctrl->ctrl, false);
+ nvme_rdma_free_tagset(ctrl, false);
out_free_io_queues:
nvme_rdma_free_io_queues(ctrl);
return ret;
@@ -934,36 +937,36 @@ static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
kfree(ctrl);
}
-static void nvme_rdma_reconnect_or_remove(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_reconnect_or_remove(struct nvme_ctrl *ctrl)
{
/* If we are resetting/deleting then do nothing */
- if (ctrl->ctrl.state != NVME_CTRL_RECONNECTING) {
- WARN_ON_ONCE(ctrl->ctrl.state == NVME_CTRL_NEW ||
- ctrl->ctrl.state == NVME_CTRL_LIVE);
+ if (ctrl->state != NVME_CTRL_RECONNECTING) {
+ WARN_ON_ONCE(ctrl->state == NVME_CTRL_NEW ||
+ ctrl->state == NVME_CTRL_LIVE);
return;
}
- if (nvmf_should_reconnect(&ctrl->ctrl)) {
- dev_info(ctrl->ctrl.device, "Reconnecting in %d seconds...\n",
- ctrl->ctrl.opts->reconnect_delay);
- queue_delayed_work(nvme_wq, &ctrl->ctrl.reconnect_work,
- ctrl->ctrl.opts->reconnect_delay * HZ);
+ if (nvmf_should_reconnect(ctrl)) {
+ dev_info(ctrl->device, "Reconnecting in %d seconds...\n",
+ ctrl->opts->reconnect_delay);
+ queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
+ ctrl->opts->reconnect_delay * HZ);
} else {
- dev_info(ctrl->ctrl.device, "Removing controller...\n");
- queue_work(nvme_wq, &ctrl->ctrl.delete_work);
+ dev_info(ctrl->device, "Removing controller...\n");
+ queue_work(nvme_wq, &ctrl->delete_work);
}
}
static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl = container_of(to_delayed_work(work),
- struct nvme_rdma_ctrl, ctrl.reconnect_work);
+ struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
+ struct nvme_ctrl, reconnect_work);
bool changed;
int ret;
- ++ctrl->ctrl.nr_reconnects;
+ ++ctrl->nr_reconnects;
- if (ctrl->ctrl.queue_count > 1)
+ if (ctrl->queue_count > 1)
nvme_rdma_destroy_io_queues(ctrl, false);
nvme_rdma_destroy_admin_queue(ctrl, false);
@@ -971,65 +974,65 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
if (ret)
goto requeue;
- if (ctrl->ctrl.queue_count > 1) {
+ if (ctrl->queue_count > 1) {
ret = nvme_rdma_configure_io_queues(ctrl, false);
if (ret)
goto requeue;
}
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
WARN_ON_ONCE(!changed);
- ctrl->ctrl.nr_reconnects = 0;
+ ctrl->nr_reconnects = 0;
- nvme_start_ctrl(&ctrl->ctrl);
+ nvme_start_ctrl(ctrl);
- dev_info(ctrl->ctrl.device, "Successfully reconnected\n");
+ dev_info(ctrl->device, "Successfully reconnected\n");
return;
requeue:
- dev_info(ctrl->ctrl.device, "Failed reconnect attempt %d\n",
- ctrl->ctrl.nr_reconnects);
+ dev_info(ctrl->device, "Failed reconnect attempt %d\n",
+ ctrl->nr_reconnects);
nvme_rdma_reconnect_or_remove(ctrl);
}
static void nvme_rdma_error_recovery_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, ctrl.err_work);
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, err_work);
- nvme_stop_ctrl(&ctrl->ctrl);
+ nvme_stop_ctrl(ctrl);
- if (ctrl->ctrl.queue_count > 1) {
- nvme_stop_queues(&ctrl->ctrl);
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
nvme_rdma_stop_io_queues(ctrl);
}
- blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- nvme_rdma_stop_admin_queue(&ctrl->ctrl);
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ nvme_rdma_stop_admin_queue(ctrl);
/* We must take care of fastfail/requeue all our inflight requests */
- if (ctrl->ctrl.queue_count > 1)
- blk_mq_tagset_busy_iter(&ctrl->tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
- nvme_cancel_request, &ctrl->ctrl);
+ if (ctrl->queue_count > 1)
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
/*
* queues are not a live anymore, so restart the queues to fail fast
* new IO
*/
- blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
- nvme_start_queues(&ctrl->ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
+ nvme_start_queues(ctrl);
nvme_rdma_reconnect_or_remove(ctrl);
}
-static void nvme_rdma_error_recovery(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_error_recovery(struct nvme_ctrl *ctrl)
{
- if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_RECONNECTING))
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RECONNECTING))
return;
- queue_work(nvme_wq, &ctrl->ctrl.err_work);
+ queue_work(nvme_wq, &ctrl->err_work);
}
static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
@@ -1043,7 +1046,7 @@ static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
"%s for CQE 0x%p failed with status %s (%d)\n",
op, wc->wr_cqe,
ib_wc_status_msg(wc->status), wc->status);
- nvme_rdma_error_recovery(ctrl);
+ nvme_rdma_error_recovery(&ctrl->ctrl);
}
static void nvme_rdma_memreg_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -1094,7 +1097,7 @@ static void nvme_rdma_unmap_data(struct nvme_rdma_queue *queue,
dev_err(ctrl->ctrl.device,
"Queueing INV WR for rkey %#x failed (%d)\n",
req->mr->rkey, res);
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
}
}
@@ -1373,7 +1376,7 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
dev_err(queue->ctrl->ctrl.device,
"tag 0x%x on QP %#x not found\n",
cqe->command_id, queue->qp->qp_num);
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
return ret;
}
req = blk_mq_rq_to_pdu(rq);
@@ -1584,7 +1587,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
case RDMA_CM_EVENT_TIMEWAIT_EXIT:
dev_dbg(queue->ctrl->ctrl.device,
"disconnect received - connection closed\n");
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
break;
case RDMA_CM_EVENT_DEVICE_REMOVAL:
/* device removal is handled via the ib_client API */
@@ -1592,7 +1595,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
default:
dev_err(queue->ctrl->ctrl.device,
"Unexpected RDMA CM event (%d)\n", ev->event);
- nvme_rdma_error_recovery(queue->ctrl);
+ nvme_rdma_error_recovery(&queue->ctrl->ctrl);
break;
}
@@ -1610,7 +1613,7 @@ nvme_rdma_timeout(struct request *rq, bool reserved)
struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq);
/* queue error recovery */
- nvme_rdma_error_recovery(req->queue->ctrl);
+ nvme_rdma_error_recovery(&req->queue->ctrl->ctrl);
/* fail with DNR on cmd timeout */
nvme_req(rq)->status = NVME_SC_ABORT_REQ | NVME_SC_DNR;
@@ -1761,102 +1764,101 @@ static const struct blk_mq_ops nvme_rdma_admin_mq_ops = {
.timeout = nvme_rdma_timeout,
};
-static void nvme_rdma_shutdown_ctrl(struct nvme_rdma_ctrl *ctrl, bool shutdown)
+static void nvme_rdma_shutdown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
{
- if (ctrl->ctrl.queue_count > 1) {
- nvme_stop_queues(&ctrl->ctrl);
- blk_mq_tagset_busy_iter(&ctrl->tag_set,
- nvme_cancel_request, &ctrl->ctrl);
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
nvme_rdma_destroy_io_queues(ctrl, shutdown);
}
if (shutdown)
- nvme_shutdown_ctrl(&ctrl->ctrl);
+ nvme_shutdown_ctrl(ctrl);
else
- nvme_disable_ctrl(&ctrl->ctrl, ctrl->ctrl.cap);
+ nvme_disable_ctrl(ctrl, ctrl->cap);
- blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
nvme_rdma_destroy_admin_queue(ctrl, shutdown);
}
-static void nvme_rdma_remove_ctrl(struct nvme_rdma_ctrl *ctrl)
+static void nvme_rdma_remove_ctrl(struct nvme_ctrl *ctrl)
{
- nvme_remove_namespaces(&ctrl->ctrl);
+ nvme_remove_namespaces(ctrl);
nvme_rdma_shutdown_ctrl(ctrl, true);
- nvme_uninit_ctrl(&ctrl->ctrl);
- nvme_put_ctrl(&ctrl->ctrl);
+ nvme_uninit_ctrl(ctrl);
+ nvme_put_ctrl(ctrl);
}
static void nvme_rdma_del_ctrl_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl = container_of(work,
- struct nvme_rdma_ctrl, ctrl.delete_work);
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, delete_work);
- nvme_stop_ctrl(&ctrl->ctrl);
+ nvme_stop_ctrl(ctrl);
nvme_rdma_remove_ctrl(ctrl);
}
-static int __nvme_rdma_del_ctrl(struct nvme_rdma_ctrl *ctrl)
+static int __nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
{
- if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING))
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
return -EBUSY;
- if (!queue_work(nvme_wq, &ctrl->ctrl.delete_work))
+ if (!queue_work(nvme_wq, &ctrl->delete_work))
return -EBUSY;
return 0;
}
-static int nvme_rdma_del_ctrl(struct nvme_ctrl *nctrl)
+static int nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
{
- struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
int ret = 0;
/*
* Keep a reference until all work is flushed since
* __nvme_rdma_del_ctrl can free the ctrl mem
*/
- if (!kref_get_unless_zero(&ctrl->ctrl.kref))
+ if (!kref_get_unless_zero(&ctrl->kref))
return -EBUSY;
ret = __nvme_rdma_del_ctrl(ctrl);
if (!ret)
- flush_work(&ctrl->ctrl.delete_work);
- nvme_put_ctrl(&ctrl->ctrl);
+ flush_work(&ctrl->delete_work);
+ nvme_put_ctrl(ctrl);
return ret;
}
static void nvme_rdma_reset_ctrl_work(struct work_struct *work)
{
- struct nvme_rdma_ctrl *ctrl =
- container_of(work, struct nvme_rdma_ctrl, ctrl.reset_work);
+ struct nvme_ctrl *ctrl =
+ container_of(work, struct nvme_ctrl, reset_work);
int ret;
bool changed;
- nvme_stop_ctrl(&ctrl->ctrl);
+ nvme_stop_ctrl(ctrl);
nvme_rdma_shutdown_ctrl(ctrl, false);
ret = nvme_rdma_configure_admin_queue(ctrl, false);
if (ret)
goto out_fail;
- if (ctrl->ctrl.queue_count > 1) {
+ if (ctrl->queue_count > 1) {
ret = nvme_rdma_configure_io_queues(ctrl, false);
if (ret)
goto out_fail;
}
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
WARN_ON_ONCE(!changed);
- nvme_start_ctrl(&ctrl->ctrl);
+ nvme_start_ctrl(ctrl);
return;
out_fail:
- dev_warn(ctrl->ctrl.device, "Removing after reset failure\n");
+ dev_warn(ctrl->device, "Removing after reset failure\n");
nvme_rdma_remove_ctrl(ctrl);
}
@@ -1963,7 +1965,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
- ret = nvme_rdma_configure_admin_queue(ctrl, true);
+ ret = nvme_rdma_configure_admin_queue(&ctrl->ctrl, true);
if (ret)
goto out_uninit_ctrl;
@@ -1972,7 +1974,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
goto out_uninit_ctrl;
if (opts->nr_io_queues) {
- ret = nvme_rdma_configure_io_queues(ctrl, true);
+ ret = nvme_rdma_configure_io_queues(&ctrl->ctrl, true);
if (ret)
goto out_remove_admin_queue;
}
@@ -1994,7 +1996,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
return &ctrl->ctrl;
out_remove_admin_queue:
- nvme_rdma_destroy_admin_queue(ctrl, true);
+ nvme_rdma_destroy_admin_queue(&ctrl->ctrl, true);
out_kfree_queues:
kfree(ctrl->queues);
out_uninit_ctrl:
@@ -2032,7 +2034,7 @@ static void nvme_rdma_remove_one(struct ib_device *ib_device, void *client_data)
dev_info(ctrl->ctrl.device,
"Removing ctrl: NQN \"%s\", addr %pISp\n",
ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
- __nvme_rdma_del_ctrl(ctrl);
+ __nvme_rdma_del_ctrl(&ctrl->ctrl);
}
mutex_unlock(&nvme_rdma_ctrl_mutex);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 07/12] nvme-rdma: split generic probe out of create_ctrl
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
This code is replicated for several transports, prepare
to move it to the core.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/rdma.c | 81 ++++++++++++++++++++++++++++--------------------
1 file changed, 47 insertions(+), 34 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 2591b0ce155e..b03c4a2a1172 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1908,12 +1908,51 @@ static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
return 0;
}
+static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
+ const struct nvme_ctrl_ops *ops, unsigned long quirks)
+{
+ bool changed;
+ int ret;
+
+ ret = nvme_init_ctrl(ctrl, dev, ops, quirks);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&ctrl->delete_work, nvme_rdma_del_ctrl_work);
+ INIT_WORK(&ctrl->reset_work, nvme_rdma_reset_ctrl_work);
+
+ ret = nvme_rdma_configure_admin_queue(ctrl, true);
+ if (ret)
+ goto out_uninit_ctrl;
+
+ ret = nvme_rdma_post_configure(ctrl);
+ if (ret)
+ goto out_remove_admin_queue;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_rdma_configure_io_queues(ctrl, true);
+ if (ret)
+ goto out_remove_admin_queue;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+
+ nvme_start_ctrl(ctrl);
+
+ return 0;
+out_remove_admin_queue:
+ nvme_rdma_destroy_admin_queue(ctrl, true);
+out_uninit_ctrl:
+ nvme_uninit_ctrl(ctrl);
+ return ret;
+}
+
static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_rdma_ctrl *ctrl;
int ret;
- bool changed;
char *port;
ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
@@ -1948,39 +1987,20 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
}
}
- ret = -ENOMEM;
ctrl->queues = kcalloc(ctrl->ctrl.queue_count, sizeof(*ctrl->queues),
GFP_KERNEL);
- if (!ctrl->queues)
+ if (!ctrl->queues) {
+ ret = -ENOMEM;
goto out_free_ctrl;
-
- ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops,
- 0 /* no quirks, we're perfect! */);
- if (ret)
- goto out_kfree_queues;
+ }
INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
nvme_rdma_reconnect_ctrl_work);
INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
- INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
- INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
-
- ret = nvme_rdma_configure_admin_queue(&ctrl->ctrl, true);
- if (ret)
- goto out_uninit_ctrl;
-
- ret = nvme_rdma_post_configure(&ctrl->ctrl);
- if (ret)
- goto out_uninit_ctrl;
- if (opts->nr_io_queues) {
- ret = nvme_rdma_configure_io_queues(&ctrl->ctrl, true);
- if (ret)
- goto out_remove_admin_queue;
- }
-
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
+ ret = nvme_rdma_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
+ if (!ctrl->queues)
+ goto out_kfree_queues;
dev_info(ctrl->ctrl.device, "new ctrl: NQN \"%s\", addr %pISpcs\n",
ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
@@ -1991,21 +2011,14 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
list_add_tail(&ctrl->list, &nvme_rdma_ctrl_list);
mutex_unlock(&nvme_rdma_ctrl_mutex);
- nvme_start_ctrl(&ctrl->ctrl);
-
return &ctrl->ctrl;
-out_remove_admin_queue:
- nvme_rdma_destroy_admin_queue(&ctrl->ctrl, true);
out_kfree_queues:
kfree(ctrl->queues);
-out_uninit_ctrl:
- nvme_uninit_ctrl(&ctrl->ctrl);
+out_free_ctrl:
nvme_put_ctrl(&ctrl->ctrl);
if (ret > 0)
ret = -EIO;
- return ERR_PTR(ret);
-out_free_ctrl:
kfree(ctrl);
return ERR_PTR(ret);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 07/12] nvme-rdma: split generic probe out of create_ctrl
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
This code is replicated for several transports, prepare
to move it to the core.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/rdma.c | 81 ++++++++++++++++++++++++++++--------------------
1 file changed, 47 insertions(+), 34 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 2591b0ce155e..b03c4a2a1172 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1908,12 +1908,51 @@ static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
return 0;
}
+static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
+ const struct nvme_ctrl_ops *ops, unsigned long quirks)
+{
+ bool changed;
+ int ret;
+
+ ret = nvme_init_ctrl(ctrl, dev, ops, quirks);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&ctrl->delete_work, nvme_rdma_del_ctrl_work);
+ INIT_WORK(&ctrl->reset_work, nvme_rdma_reset_ctrl_work);
+
+ ret = nvme_rdma_configure_admin_queue(ctrl, true);
+ if (ret)
+ goto out_uninit_ctrl;
+
+ ret = nvme_rdma_post_configure(ctrl);
+ if (ret)
+ goto out_remove_admin_queue;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_rdma_configure_io_queues(ctrl, true);
+ if (ret)
+ goto out_remove_admin_queue;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+
+ nvme_start_ctrl(ctrl);
+
+ return 0;
+out_remove_admin_queue:
+ nvme_rdma_destroy_admin_queue(ctrl, true);
+out_uninit_ctrl:
+ nvme_uninit_ctrl(ctrl);
+ return ret;
+}
+
static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_rdma_ctrl *ctrl;
int ret;
- bool changed;
char *port;
ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
@@ -1948,39 +1987,20 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
}
}
- ret = -ENOMEM;
ctrl->queues = kcalloc(ctrl->ctrl.queue_count, sizeof(*ctrl->queues),
GFP_KERNEL);
- if (!ctrl->queues)
+ if (!ctrl->queues) {
+ ret = -ENOMEM;
goto out_free_ctrl;
-
- ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops,
- 0 /* no quirks, we're perfect! */);
- if (ret)
- goto out_kfree_queues;
+ }
INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
nvme_rdma_reconnect_ctrl_work);
INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
- INIT_WORK(&ctrl->ctrl.delete_work, nvme_rdma_del_ctrl_work);
- INIT_WORK(&ctrl->ctrl.reset_work, nvme_rdma_reset_ctrl_work);
-
- ret = nvme_rdma_configure_admin_queue(&ctrl->ctrl, true);
- if (ret)
- goto out_uninit_ctrl;
-
- ret = nvme_rdma_post_configure(&ctrl->ctrl);
- if (ret)
- goto out_uninit_ctrl;
- if (opts->nr_io_queues) {
- ret = nvme_rdma_configure_io_queues(&ctrl->ctrl, true);
- if (ret)
- goto out_remove_admin_queue;
- }
-
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
+ ret = nvme_rdma_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
+ if (!ctrl->queues)
+ goto out_kfree_queues;
dev_info(ctrl->ctrl.device, "new ctrl: NQN \"%s\", addr %pISpcs\n",
ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
@@ -1991,21 +2011,14 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
list_add_tail(&ctrl->list, &nvme_rdma_ctrl_list);
mutex_unlock(&nvme_rdma_ctrl_mutex);
- nvme_start_ctrl(&ctrl->ctrl);
-
return &ctrl->ctrl;
-out_remove_admin_queue:
- nvme_rdma_destroy_admin_queue(&ctrl->ctrl, true);
out_kfree_queues:
kfree(ctrl->queues);
-out_uninit_ctrl:
- nvme_uninit_ctrl(&ctrl->ctrl);
+out_free_ctrl:
nvme_put_ctrl(&ctrl->ctrl);
if (ret > 0)
ret = -EIO;
- return ERR_PTR(ret);
-out_free_ctrl:
kfree(ctrl);
return ERR_PTR(ret);
}
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 08/12] nvme: add some ctrl ops for centralizing control plane logic.
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
The goal is to move a larger portion of the logic to nvme
core, thus add admin/io queue and tagset callouts.
1. for queues we start with admin/io queues:
->alloc
->free
->start
->stop
we separate admin and IO mostly to allow pci driver
to free the queue in batches rather than one by one (to
speed up the process).
2. for tagset we have:
->alloc_tagset
->free_tagset
which accepts admin boolean.
3. we also has some misc ops:
->post_configure
->nr_hw_queues
And I think we'll have more as we go.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/nvme.h | 13 +++++++++
drivers/nvme/host/rdma.c | 76 ++++++++++++++++++++++++++++--------------------
2 files changed, 57 insertions(+), 32 deletions(-)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index c52ba1405788..7b8e57b3e634 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -235,6 +235,19 @@ struct nvme_ctrl_ops {
void (*submit_async_event)(struct nvme_ctrl *ctrl, int aer_idx);
int (*delete_ctrl)(struct nvme_ctrl *ctrl);
int (*get_address)(struct nvme_ctrl *ctrl, char *buf, int size);
+ int (*alloc_admin_queue)(struct nvme_ctrl *ctrl);
+ void (*free_admin_queue)(struct nvme_ctrl *ctrl);
+ int (*start_admin_queue)(struct nvme_ctrl *ctrl);
+ void (*stop_admin_queue)(struct nvme_ctrl *ctrl);
+ int (*alloc_io_queues)(struct nvme_ctrl *ctrl);
+ void (*free_io_queues)(struct nvme_ctrl *ctrl);
+ int (*start_io_queues)(struct nvme_ctrl *ctrl);
+ void (*stop_io_queues)(struct nvme_ctrl *ctrl);
+ struct blk_mq_tag_set *(*alloc_tagset)(struct nvme_ctrl *ctrl,
+ bool admin);
+ void (*free_tagset)(struct nvme_ctrl *ctrl, bool admin);
+ int (*post_configure)(struct nvme_ctrl *ctrl);
+ unsigned int (*nr_hw_queues)(struct nvme_ctrl *ctrl);
};
static inline bool nvme_ctrl_ready(struct nvme_ctrl *ctrl)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index b03c4a2a1172..cf4e4371c2db 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -757,12 +757,12 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
static void nvme_rdma_destroy_admin_queue(struct nvme_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_admin_queue(ctrl);
+ ctrl->ops->stop_admin_queue(ctrl);
if (remove) {
blk_cleanup_queue(ctrl->admin_q);
- nvme_rdma_free_tagset(ctrl, true);
+ ctrl->ops->free_tagset(ctrl, true);
}
- nvme_rdma_free_admin_queue(ctrl);
+ ctrl->ops->free_admin_queue(ctrl);
}
static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
@@ -798,12 +798,12 @@ static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
{
int error;
- error = nvme_rdma_alloc_admin_queue(ctrl);
+ error = ctrl->ops->alloc_admin_queue(ctrl);
if (error)
return error;
if (new) {
- ctrl->admin_tagset = nvme_rdma_alloc_tagset(ctrl, true);
+ ctrl->admin_tagset = ctrl->ops->alloc_tagset(ctrl, true);
if (IS_ERR(ctrl->admin_tagset))
goto out_free_queue;
@@ -818,7 +818,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
goto out_free_queue;
}
- error = nvme_rdma_start_admin_queue(ctrl);
+ error = ctrl->ops->start_admin_queue(ctrl);
if (error)
goto out_cleanup_queue;
@@ -846,21 +846,21 @@ static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
blk_cleanup_queue(ctrl->admin_q);
out_free_tagset:
if (new)
- nvme_rdma_free_tagset(ctrl, true);
+ ctrl->ops->free_tagset(ctrl, true);
out_free_queue:
- nvme_rdma_free_admin_queue(ctrl);
+ ctrl->ops->free_admin_queue(ctrl);
return error;
}
static void nvme_rdma_destroy_io_queues(struct nvme_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_io_queues(ctrl);
+ ctrl->ops->stop_io_queues(ctrl);
if (remove) {
blk_cleanup_queue(ctrl->connect_q);
- nvme_rdma_free_tagset(ctrl, false);
+ ctrl->ops->free_tagset(ctrl, false);
}
- nvme_rdma_free_io_queues(ctrl);
+ ctrl->ops->free_io_queues(ctrl);
}
static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
@@ -868,7 +868,7 @@ static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
unsigned int nr_io_queues;
int ret;
- nr_io_queues = nvme_rdma_nr_io_queues(ctrl);
+ nr_io_queues = ctrl->ops->nr_hw_queues(ctrl);
ret = nvme_set_queue_count(ctrl, &nr_io_queues);
if (ret)
return ret;
@@ -880,12 +880,12 @@ static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
dev_info(ctrl->device,
"creating %d I/O queues.\n", nr_io_queues);
- ret = nvme_rdma_alloc_io_queues(ctrl);
+ ret = ctrl->ops->alloc_io_queues(ctrl);
if (ret)
return ret;
if (new) {
- ctrl->tagset = nvme_rdma_alloc_tagset(ctrl, false);
+ ctrl->tagset = ctrl->ops->alloc_tagset(ctrl, false);
if (IS_ERR(ctrl->tagset))
goto out_free_io_queues;
@@ -914,9 +914,9 @@ static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
blk_cleanup_queue(ctrl->connect_q);
out_free_tag_set:
if (new)
- nvme_rdma_free_tagset(ctrl, false);
+ ctrl->ops->free_tagset(ctrl, false);
out_free_io_queues:
- nvme_rdma_free_io_queues(ctrl);
+ ctrl->ops->free_io_queues(ctrl);
return ret;
}
@@ -1005,10 +1005,10 @@ static void nvme_rdma_error_recovery_work(struct work_struct *work)
if (ctrl->queue_count > 1) {
nvme_stop_queues(ctrl);
- nvme_rdma_stop_io_queues(ctrl);
+ ctrl->ops->stop_io_queues(ctrl);
}
blk_mq_quiesce_queue(ctrl->admin_q);
- nvme_rdma_stop_admin_queue(ctrl);
+ ctrl->ops->stop_admin_queue(ctrl);
/* We must take care of fastfail/requeue all our inflight requests */
if (ctrl->queue_count > 1)
@@ -1862,19 +1862,6 @@ static void nvme_rdma_reset_ctrl_work(struct work_struct *work)
nvme_rdma_remove_ctrl(ctrl);
}
-static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
- .name = "rdma",
- .module = THIS_MODULE,
- .flags = NVME_F_FABRICS,
- .reg_read32 = nvmf_reg_read32,
- .reg_read64 = nvmf_reg_read64,
- .reg_write32 = nvmf_reg_write32,
- .free_ctrl = nvme_rdma_free_ctrl,
- .submit_async_event = nvme_rdma_submit_async_event,
- .delete_ctrl = nvme_rdma_del_ctrl,
- .get_address = nvmf_get_address,
-};
-
static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
{
/* sanity check icdoff */
@@ -1908,6 +1895,31 @@ static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
return 0;
}
+static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
+ .name = "rdma",
+ .module = THIS_MODULE,
+ .flags = NVME_F_FABRICS,
+ .reg_read32 = nvmf_reg_read32,
+ .reg_read64 = nvmf_reg_read64,
+ .reg_write32 = nvmf_reg_write32,
+ .free_ctrl = nvme_rdma_free_ctrl,
+ .submit_async_event = nvme_rdma_submit_async_event,
+ .delete_ctrl = nvme_rdma_del_ctrl,
+ .get_address = nvmf_get_address,
+ .alloc_admin_queue = nvme_rdma_alloc_admin_queue,
+ .free_admin_queue = nvme_rdma_free_admin_queue,
+ .start_admin_queue = nvme_rdma_start_admin_queue,
+ .stop_admin_queue = nvme_rdma_stop_admin_queue,
+ .alloc_io_queues = nvme_rdma_alloc_io_queues,
+ .free_io_queues = nvme_rdma_free_io_queues,
+ .start_io_queues = nvme_rdma_start_io_queues,
+ .stop_io_queues = nvme_rdma_stop_io_queues,
+ .alloc_tagset = nvme_rdma_alloc_tagset,
+ .free_tagset = nvme_rdma_free_tagset,
+ .post_configure = nvme_rdma_post_configure,
+ .nr_hw_queues = nvme_rdma_nr_io_queues,
+};
+
static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
const struct nvme_ctrl_ops *ops, unsigned long quirks)
{
@@ -1925,7 +1937,7 @@ static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
if (ret)
goto out_uninit_ctrl;
- ret = nvme_rdma_post_configure(ctrl);
+ ret = ctrl->ops->post_configure(ctrl);
if (ret)
goto out_remove_admin_queue;
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 08/12] nvme: add some ctrl ops for centralizing control plane logic.
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
The goal is to move a larger portion of the logic to nvme
core, thus add admin/io queue and tagset callouts.
1. for queues we start with admin/io queues:
->alloc
->free
->start
->stop
we separate admin and IO mostly to allow pci driver
to free the queue in batches rather than one by one (to
speed up the process).
2. for tagset we have:
->alloc_tagset
->free_tagset
which accepts admin boolean.
3. we also has some misc ops:
->post_configure
->nr_hw_queues
And I think we'll have more as we go.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/nvme.h | 13 +++++++++
drivers/nvme/host/rdma.c | 76 ++++++++++++++++++++++++++++--------------------
2 files changed, 57 insertions(+), 32 deletions(-)
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index c52ba1405788..7b8e57b3e634 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -235,6 +235,19 @@ struct nvme_ctrl_ops {
void (*submit_async_event)(struct nvme_ctrl *ctrl, int aer_idx);
int (*delete_ctrl)(struct nvme_ctrl *ctrl);
int (*get_address)(struct nvme_ctrl *ctrl, char *buf, int size);
+ int (*alloc_admin_queue)(struct nvme_ctrl *ctrl);
+ void (*free_admin_queue)(struct nvme_ctrl *ctrl);
+ int (*start_admin_queue)(struct nvme_ctrl *ctrl);
+ void (*stop_admin_queue)(struct nvme_ctrl *ctrl);
+ int (*alloc_io_queues)(struct nvme_ctrl *ctrl);
+ void (*free_io_queues)(struct nvme_ctrl *ctrl);
+ int (*start_io_queues)(struct nvme_ctrl *ctrl);
+ void (*stop_io_queues)(struct nvme_ctrl *ctrl);
+ struct blk_mq_tag_set *(*alloc_tagset)(struct nvme_ctrl *ctrl,
+ bool admin);
+ void (*free_tagset)(struct nvme_ctrl *ctrl, bool admin);
+ int (*post_configure)(struct nvme_ctrl *ctrl);
+ unsigned int (*nr_hw_queues)(struct nvme_ctrl *ctrl);
};
static inline bool nvme_ctrl_ready(struct nvme_ctrl *ctrl)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index b03c4a2a1172..cf4e4371c2db 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -757,12 +757,12 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
static void nvme_rdma_destroy_admin_queue(struct nvme_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_admin_queue(ctrl);
+ ctrl->ops->stop_admin_queue(ctrl);
if (remove) {
blk_cleanup_queue(ctrl->admin_q);
- nvme_rdma_free_tagset(ctrl, true);
+ ctrl->ops->free_tagset(ctrl, true);
}
- nvme_rdma_free_admin_queue(ctrl);
+ ctrl->ops->free_admin_queue(ctrl);
}
static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
@@ -798,12 +798,12 @@ static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
{
int error;
- error = nvme_rdma_alloc_admin_queue(ctrl);
+ error = ctrl->ops->alloc_admin_queue(ctrl);
if (error)
return error;
if (new) {
- ctrl->admin_tagset = nvme_rdma_alloc_tagset(ctrl, true);
+ ctrl->admin_tagset = ctrl->ops->alloc_tagset(ctrl, true);
if (IS_ERR(ctrl->admin_tagset))
goto out_free_queue;
@@ -818,7 +818,7 @@ static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
goto out_free_queue;
}
- error = nvme_rdma_start_admin_queue(ctrl);
+ error = ctrl->ops->start_admin_queue(ctrl);
if (error)
goto out_cleanup_queue;
@@ -846,21 +846,21 @@ static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
blk_cleanup_queue(ctrl->admin_q);
out_free_tagset:
if (new)
- nvme_rdma_free_tagset(ctrl, true);
+ ctrl->ops->free_tagset(ctrl, true);
out_free_queue:
- nvme_rdma_free_admin_queue(ctrl);
+ ctrl->ops->free_admin_queue(ctrl);
return error;
}
static void nvme_rdma_destroy_io_queues(struct nvme_ctrl *ctrl,
bool remove)
{
- nvme_rdma_stop_io_queues(ctrl);
+ ctrl->ops->stop_io_queues(ctrl);
if (remove) {
blk_cleanup_queue(ctrl->connect_q);
- nvme_rdma_free_tagset(ctrl, false);
+ ctrl->ops->free_tagset(ctrl, false);
}
- nvme_rdma_free_io_queues(ctrl);
+ ctrl->ops->free_io_queues(ctrl);
}
static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
@@ -868,7 +868,7 @@ static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
unsigned int nr_io_queues;
int ret;
- nr_io_queues = nvme_rdma_nr_io_queues(ctrl);
+ nr_io_queues = ctrl->ops->nr_hw_queues(ctrl);
ret = nvme_set_queue_count(ctrl, &nr_io_queues);
if (ret)
return ret;
@@ -880,12 +880,12 @@ static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
dev_info(ctrl->device,
"creating %d I/O queues.\n", nr_io_queues);
- ret = nvme_rdma_alloc_io_queues(ctrl);
+ ret = ctrl->ops->alloc_io_queues(ctrl);
if (ret)
return ret;
if (new) {
- ctrl->tagset = nvme_rdma_alloc_tagset(ctrl, false);
+ ctrl->tagset = ctrl->ops->alloc_tagset(ctrl, false);
if (IS_ERR(ctrl->tagset))
goto out_free_io_queues;
@@ -914,9 +914,9 @@ static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
blk_cleanup_queue(ctrl->connect_q);
out_free_tag_set:
if (new)
- nvme_rdma_free_tagset(ctrl, false);
+ ctrl->ops->free_tagset(ctrl, false);
out_free_io_queues:
- nvme_rdma_free_io_queues(ctrl);
+ ctrl->ops->free_io_queues(ctrl);
return ret;
}
@@ -1005,10 +1005,10 @@ static void nvme_rdma_error_recovery_work(struct work_struct *work)
if (ctrl->queue_count > 1) {
nvme_stop_queues(ctrl);
- nvme_rdma_stop_io_queues(ctrl);
+ ctrl->ops->stop_io_queues(ctrl);
}
blk_mq_quiesce_queue(ctrl->admin_q);
- nvme_rdma_stop_admin_queue(ctrl);
+ ctrl->ops->stop_admin_queue(ctrl);
/* We must take care of fastfail/requeue all our inflight requests */
if (ctrl->queue_count > 1)
@@ -1862,19 +1862,6 @@ static void nvme_rdma_reset_ctrl_work(struct work_struct *work)
nvme_rdma_remove_ctrl(ctrl);
}
-static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
- .name = "rdma",
- .module = THIS_MODULE,
- .flags = NVME_F_FABRICS,
- .reg_read32 = nvmf_reg_read32,
- .reg_read64 = nvmf_reg_read64,
- .reg_write32 = nvmf_reg_write32,
- .free_ctrl = nvme_rdma_free_ctrl,
- .submit_async_event = nvme_rdma_submit_async_event,
- .delete_ctrl = nvme_rdma_del_ctrl,
- .get_address = nvmf_get_address,
-};
-
static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
{
/* sanity check icdoff */
@@ -1908,6 +1895,31 @@ static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
return 0;
}
+static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
+ .name = "rdma",
+ .module = THIS_MODULE,
+ .flags = NVME_F_FABRICS,
+ .reg_read32 = nvmf_reg_read32,
+ .reg_read64 = nvmf_reg_read64,
+ .reg_write32 = nvmf_reg_write32,
+ .free_ctrl = nvme_rdma_free_ctrl,
+ .submit_async_event = nvme_rdma_submit_async_event,
+ .delete_ctrl = nvme_rdma_del_ctrl,
+ .get_address = nvmf_get_address,
+ .alloc_admin_queue = nvme_rdma_alloc_admin_queue,
+ .free_admin_queue = nvme_rdma_free_admin_queue,
+ .start_admin_queue = nvme_rdma_start_admin_queue,
+ .stop_admin_queue = nvme_rdma_stop_admin_queue,
+ .alloc_io_queues = nvme_rdma_alloc_io_queues,
+ .free_io_queues = nvme_rdma_free_io_queues,
+ .start_io_queues = nvme_rdma_start_io_queues,
+ .stop_io_queues = nvme_rdma_stop_io_queues,
+ .alloc_tagset = nvme_rdma_alloc_tagset,
+ .free_tagset = nvme_rdma_free_tagset,
+ .post_configure = nvme_rdma_post_configure,
+ .nr_hw_queues = nvme_rdma_nr_io_queues,
+};
+
static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
const struct nvme_ctrl_ops *ops, unsigned long quirks)
{
@@ -1925,7 +1937,7 @@ static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
if (ret)
goto out_uninit_ctrl;
- ret = nvme_rdma_post_configure(ctrl);
+ ret = ctrl->ops->post_configure(ctrl);
if (ret)
goto out_remove_admin_queue;
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 09/12] nvme: move control plane handling to nvme core
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
handle controller setup (probe), reset and delete in nvme-core and
rip it our from nvme-rdma.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/core.c | 296 +++++++++++++++++++++++++++++++++++++++++++++++
drivers/nvme/host/nvme.h | 11 ++
drivers/nvme/host/rdma.c | 290 ++--------------------------------------------
3 files changed, 314 insertions(+), 283 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index bbaf5b98f2fe..4344adff7134 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2913,6 +2913,302 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
}
EXPORT_SYMBOL_GPL(nvme_start_queues);
+static int nvme_alloc_io_queues(struct nvme_ctrl *ctrl)
+{
+ unsigned int nr_io_queues;
+ int ret;
+
+ nr_io_queues = ctrl->ops->nr_hw_queues(ctrl);
+ ret = nvme_set_queue_count(ctrl, &nr_io_queues);
+ if (ret)
+ return ret;
+
+ ctrl->queue_count = nr_io_queues + 1;
+ if (ctrl->queue_count < 2)
+ return 0;
+
+ dev_info(ctrl->device,
+ "creating %d I/O queues.\n", nr_io_queues);
+
+ return ctrl->ops->alloc_io_queues(ctrl);
+}
+
+void nvme_destroy_io_queues(struct nvme_ctrl *ctrl, bool remove)
+{
+ ctrl->ops->stop_io_queues(ctrl);
+ if (remove) {
+ if (ctrl->ops->flags & NVME_F_FABRICS)
+ blk_cleanup_queue(ctrl->connect_q);
+ ctrl->ops->free_tagset(ctrl, false);
+ }
+ ctrl->ops->free_io_queues(ctrl);
+}
+EXPORT_SYMBOL_GPL(nvme_destroy_io_queues);
+
+int nvme_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
+{
+ int ret;
+
+ ret = nvme_alloc_io_queues(ctrl);
+ if (ret)
+ return ret;
+
+ if (new) {
+ ctrl->tagset = ctrl->ops->alloc_tagset(ctrl, false);
+ if (IS_ERR(ctrl->tagset)) {
+ ret = PTR_ERR(ctrl->tagset);
+ goto out_free_io_queues;
+ }
+
+ if (ctrl->ops->flags & NVME_F_FABRICS) {
+ ctrl->connect_q = blk_mq_init_queue(ctrl->tagset);
+ if (IS_ERR(ctrl->connect_q)) {
+ ret = PTR_ERR(ctrl->connect_q);
+ goto out_free_tag_set;
+ }
+ }
+ } else {
+ ret = blk_mq_reinit_tagset(ctrl->tagset);
+ if (ret)
+ goto out_free_io_queues;
+
+ blk_mq_update_nr_hw_queues(ctrl->tagset,
+ ctrl->queue_count - 1);
+ }
+
+ ret = ctrl->ops->start_io_queues(ctrl);
+ if (ret)
+ goto out_cleanup_connect_q;
+
+ return 0;
+
+out_cleanup_connect_q:
+ if (new && (ctrl->ops->flags & NVME_F_FABRICS))
+ blk_cleanup_queue(ctrl->connect_q);
+out_free_tag_set:
+ if (new)
+ ctrl->ops->free_tagset(ctrl, false);
+out_free_io_queues:
+ ctrl->ops->free_io_queues(ctrl);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(nvme_configure_io_queues);
+
+void nvme_destroy_admin_queue(struct nvme_ctrl *ctrl, bool remove)
+{
+ ctrl->ops->stop_admin_queue(ctrl);
+ if (remove) {
+ blk_cleanup_queue(ctrl->admin_q);
+ ctrl->ops->free_tagset(ctrl, true);
+ }
+ ctrl->ops->free_admin_queue(ctrl);
+}
+EXPORT_SYMBOL_GPL(nvme_destroy_admin_queue);
+
+int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
+{
+ int error;
+
+ error = ctrl->ops->alloc_admin_queue(ctrl);
+ if (error)
+ return error;
+
+ if (new) {
+ ctrl->admin_tagset = ctrl->ops->alloc_tagset(ctrl, true);
+ if (IS_ERR(ctrl->admin_tagset)) {
+ error = PTR_ERR(ctrl->admin_tagset);
+ goto out_free_queue;
+ }
+
+ ctrl->admin_q = blk_mq_init_queue(ctrl->admin_tagset);
+ if (IS_ERR(ctrl->admin_q)) {
+ error = PTR_ERR(ctrl->admin_q);
+ goto out_free_tagset;
+ }
+ } else {
+ error = blk_mq_reinit_tagset(ctrl->admin_tagset);
+ if (error)
+ goto out_free_queue;
+ }
+
+ error = ctrl->ops->start_admin_queue(ctrl);
+ if (error)
+ goto out_cleanup_queue;
+
+ error = ctrl->ops->reg_read64(ctrl, NVME_REG_CAP, &ctrl->cap);
+ if (error) {
+ dev_err(ctrl->device,
+ "prop_get NVME_REG_CAP failed\n");
+ goto out_cleanup_queue;
+ }
+
+ ctrl->sqsize = min_t(int, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
+
+ error = nvme_enable_ctrl(ctrl, ctrl->cap);
+ if (error)
+ goto out_cleanup_queue;
+
+ error = nvme_init_identify(ctrl);
+ if (error)
+ goto out_cleanup_queue;
+
+ return 0;
+
+out_cleanup_queue:
+ if (new)
+ blk_cleanup_queue(ctrl->admin_q);
+out_free_tagset:
+ if (new)
+ ctrl->ops->free_tagset(ctrl, true);
+out_free_queue:
+ ctrl->ops->free_admin_queue(ctrl);
+ return error;
+}
+EXPORT_SYMBOL_GPL(nvme_configure_admin_queue);
+
+static void nvme_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
+{
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
+ nvme_destroy_io_queues(ctrl, shutdown);
+ }
+
+ if (shutdown)
+ nvme_shutdown_ctrl(ctrl);
+ else
+ nvme_disable_ctrl(ctrl, ctrl->cap);
+
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
+ nvme_destroy_admin_queue(ctrl, shutdown);
+}
+
+static void nvme_remove_ctrl(struct nvme_ctrl *ctrl)
+{
+ nvme_remove_namespaces(ctrl);
+ nvme_teardown_ctrl(ctrl, true);
+ nvme_uninit_ctrl(ctrl);
+ nvme_put_ctrl(ctrl);
+}
+
+static void nvme_del_ctrl_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, delete_work);
+
+ nvme_stop_ctrl(ctrl);
+ nvme_remove_ctrl(ctrl);
+}
+
+int __nvme_del_ctrl(struct nvme_ctrl *ctrl)
+{
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
+ return -EBUSY;
+
+ if (!queue_work(nvme_wq, &ctrl->delete_work))
+ return -EBUSY;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(__nvme_del_ctrl);
+
+int nvme_del_ctrl(struct nvme_ctrl *ctrl)
+{
+ int ret = 0;
+
+ /*
+ * Keep a reference until all work is flushed since
+ * __nvme_del_ctrl can free the ctrl mem
+ */
+ if (!kref_get_unless_zero(&ctrl->kref))
+ return -EBUSY;
+
+ ret = __nvme_del_ctrl(ctrl);
+ if (!ret)
+ flush_work(&ctrl->delete_work);
+
+ nvme_put_ctrl(ctrl);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(nvme_del_ctrl);
+
+static void nvme_reset_ctrl_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl =
+ container_of(work, struct nvme_ctrl, reset_work);
+ int ret;
+ bool changed;
+
+ nvme_stop_ctrl(ctrl);
+ nvme_teardown_ctrl(ctrl, false);
+
+ ret = nvme_configure_admin_queue(ctrl, false);
+ if (ret)
+ goto out_fail;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_configure_io_queues(ctrl, false);
+ if (ret)
+ goto out_fail;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+
+ nvme_start_ctrl(ctrl);
+
+ return;
+
+out_fail:
+ dev_warn(ctrl->device, "Removing after reset failure\n");
+ nvme_remove_ctrl(ctrl);
+}
+
+int nvme_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
+ const struct nvme_ctrl_ops *ops, unsigned long quirks)
+{
+ bool changed;
+ int ret;
+
+ ret = nvme_init_ctrl(ctrl, dev, ops, quirks);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&ctrl->delete_work, nvme_del_ctrl_work);
+ INIT_WORK(&ctrl->reset_work, nvme_reset_ctrl_work);
+
+ ret = nvme_configure_admin_queue(ctrl, true);
+ if (ret)
+ goto out_uninit_ctrl;
+
+ ret = ctrl->ops->post_configure(ctrl);
+ if (ret)
+ goto out_remove_admin_queue;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_configure_io_queues(ctrl, true);
+ if (ret)
+ goto out_remove_admin_queue;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+
+ nvme_start_ctrl(ctrl);
+
+ return 0;
+out_remove_admin_queue:
+ nvme_destroy_admin_queue(ctrl, true);
+out_uninit_ctrl:
+ nvme_uninit_ctrl(ctrl);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(nvme_probe_ctrl);
+
int __init nvme_core_init(void)
{
int result;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 7b8e57b3e634..b5cefa28d3d6 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -381,6 +381,17 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
return dev_to_disk(dev)->private_data;
}
+void nvme_stop_io_queues(struct nvme_ctrl *ctrl);
+void nvme_destroy_io_queues(struct nvme_ctrl *ctrl, bool remove);
+int nvme_configure_io_queues(struct nvme_ctrl *ctrl, bool new);
+void nvme_destroy_admin_queue(struct nvme_ctrl *ctrl, bool remove);
+int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new);
+int __nvme_del_ctrl(struct nvme_ctrl *ctrl);
+int nvme_del_ctrl(struct nvme_ctrl *ctrl);
+int nvme_reset_ctrl(struct nvme_ctrl *ctrl);
+int nvme_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
+ const struct nvme_ctrl_ops *ops, unsigned long quirks);
+
int __init nvme_core_init(void);
void nvme_core_exit(void);
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index cf4e4371c2db..35459f2eea74 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -754,17 +754,6 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
return ERR_PTR(ret);
}
-static void nvme_rdma_destroy_admin_queue(struct nvme_ctrl *ctrl,
- bool remove)
-{
- ctrl->ops->stop_admin_queue(ctrl);
- if (remove) {
- blk_cleanup_queue(ctrl->admin_q);
- ctrl->ops->free_tagset(ctrl, true);
- }
- ctrl->ops->free_admin_queue(ctrl);
-}
-
static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
{
struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
@@ -793,133 +782,6 @@ static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
return ret;
}
-static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
- bool new)
-{
- int error;
-
- error = ctrl->ops->alloc_admin_queue(ctrl);
- if (error)
- return error;
-
- if (new) {
- ctrl->admin_tagset = ctrl->ops->alloc_tagset(ctrl, true);
- if (IS_ERR(ctrl->admin_tagset))
- goto out_free_queue;
-
- ctrl->admin_q = blk_mq_init_queue(ctrl->admin_tagset);
- if (IS_ERR(ctrl->admin_q)) {
- error = PTR_ERR(ctrl->admin_q);
- goto out_free_tagset;
- }
- } else {
- error = blk_mq_reinit_tagset(ctrl->admin_tagset);
- if (error)
- goto out_free_queue;
- }
-
- error = ctrl->ops->start_admin_queue(ctrl);
- if (error)
- goto out_cleanup_queue;
-
- error = ctrl->ops->reg_read64(ctrl, NVME_REG_CAP,
- &ctrl->cap);
- if (error) {
- dev_err(ctrl->device, "prop_get NVME_REG_CAP failed\n");
- goto out_cleanup_queue;
- }
-
- ctrl->sqsize = min_t(int, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
-
- error = nvme_enable_ctrl(ctrl, ctrl->cap);
- if (error)
- goto out_cleanup_queue;
-
- error = nvme_init_identify(ctrl);
- if (error)
- goto out_cleanup_queue;
-
- return 0;
-
-out_cleanup_queue:
- if (new)
- blk_cleanup_queue(ctrl->admin_q);
-out_free_tagset:
- if (new)
- ctrl->ops->free_tagset(ctrl, true);
-out_free_queue:
- ctrl->ops->free_admin_queue(ctrl);
- return error;
-}
-
-static void nvme_rdma_destroy_io_queues(struct nvme_ctrl *ctrl,
- bool remove)
-{
- ctrl->ops->stop_io_queues(ctrl);
- if (remove) {
- blk_cleanup_queue(ctrl->connect_q);
- ctrl->ops->free_tagset(ctrl, false);
- }
- ctrl->ops->free_io_queues(ctrl);
-}
-
-static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
-{
- unsigned int nr_io_queues;
- int ret;
-
- nr_io_queues = ctrl->ops->nr_hw_queues(ctrl);
- ret = nvme_set_queue_count(ctrl, &nr_io_queues);
- if (ret)
- return ret;
-
- ctrl->queue_count = nr_io_queues + 1;
- if (ctrl->queue_count < 2)
- return 0;
-
- dev_info(ctrl->device,
- "creating %d I/O queues.\n", nr_io_queues);
-
- ret = ctrl->ops->alloc_io_queues(ctrl);
- if (ret)
- return ret;
-
- if (new) {
- ctrl->tagset = ctrl->ops->alloc_tagset(ctrl, false);
- if (IS_ERR(ctrl->tagset))
- goto out_free_io_queues;
-
- ctrl->connect_q = blk_mq_init_queue(ctrl->tagset);
- if (IS_ERR(ctrl->connect_q)) {
- ret = PTR_ERR(ctrl->connect_q);
- goto out_free_tag_set;
- }
- } else {
- ret = blk_mq_reinit_tagset(ctrl->tagset);
- if (ret)
- goto out_free_io_queues;
-
- blk_mq_update_nr_hw_queues(ctrl->tagset,
- ctrl->queue_count - 1);
- }
-
- ret = nvme_rdma_start_io_queues(ctrl);
- if (ret)
- goto out_cleanup_connect_q;
-
- return 0;
-
-out_cleanup_connect_q:
- if (new)
- blk_cleanup_queue(ctrl->connect_q);
-out_free_tag_set:
- if (new)
- ctrl->ops->free_tagset(ctrl, false);
-out_free_io_queues:
- ctrl->ops->free_io_queues(ctrl);
- return ret;
-}
-
static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
{
struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
@@ -967,15 +829,15 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
++ctrl->nr_reconnects;
if (ctrl->queue_count > 1)
- nvme_rdma_destroy_io_queues(ctrl, false);
+ nvme_destroy_io_queues(ctrl, false);
- nvme_rdma_destroy_admin_queue(ctrl, false);
- ret = nvme_rdma_configure_admin_queue(ctrl, false);
+ nvme_destroy_admin_queue(ctrl, false);
+ ret = nvme_configure_admin_queue(ctrl, false);
if (ret)
goto requeue;
if (ctrl->queue_count > 1) {
- ret = nvme_rdma_configure_io_queues(ctrl, false);
+ ret = nvme_configure_io_queues(ctrl, false);
if (ret)
goto requeue;
}
@@ -1764,104 +1626,6 @@ static const struct blk_mq_ops nvme_rdma_admin_mq_ops = {
.timeout = nvme_rdma_timeout,
};
-static void nvme_rdma_shutdown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
-{
- if (ctrl->queue_count > 1) {
- nvme_stop_queues(ctrl);
- blk_mq_tagset_busy_iter(ctrl->tagset,
- nvme_cancel_request, ctrl);
- nvme_rdma_destroy_io_queues(ctrl, shutdown);
- }
-
- if (shutdown)
- nvme_shutdown_ctrl(ctrl);
- else
- nvme_disable_ctrl(ctrl, ctrl->cap);
-
- blk_mq_quiesce_queue(ctrl->admin_q);
- blk_mq_tagset_busy_iter(ctrl->admin_tagset,
- nvme_cancel_request, ctrl);
- blk_mq_unquiesce_queue(ctrl->admin_q);
- nvme_rdma_destroy_admin_queue(ctrl, shutdown);
-}
-
-static void nvme_rdma_remove_ctrl(struct nvme_ctrl *ctrl)
-{
- nvme_remove_namespaces(ctrl);
- nvme_rdma_shutdown_ctrl(ctrl, true);
- nvme_uninit_ctrl(ctrl);
- nvme_put_ctrl(ctrl);
-}
-
-static void nvme_rdma_del_ctrl_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl = container_of(work,
- struct nvme_ctrl, delete_work);
-
- nvme_stop_ctrl(ctrl);
- nvme_rdma_remove_ctrl(ctrl);
-}
-
-static int __nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
-{
- if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
- return -EBUSY;
-
- if (!queue_work(nvme_wq, &ctrl->delete_work))
- return -EBUSY;
-
- return 0;
-}
-
-static int nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
-{
- int ret = 0;
-
- /*
- * Keep a reference until all work is flushed since
- * __nvme_rdma_del_ctrl can free the ctrl mem
- */
- if (!kref_get_unless_zero(&ctrl->kref))
- return -EBUSY;
- ret = __nvme_rdma_del_ctrl(ctrl);
- if (!ret)
- flush_work(&ctrl->delete_work);
- nvme_put_ctrl(ctrl);
- return ret;
-}
-
-static void nvme_rdma_reset_ctrl_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl =
- container_of(work, struct nvme_ctrl, reset_work);
- int ret;
- bool changed;
-
- nvme_stop_ctrl(ctrl);
- nvme_rdma_shutdown_ctrl(ctrl, false);
-
- ret = nvme_rdma_configure_admin_queue(ctrl, false);
- if (ret)
- goto out_fail;
-
- if (ctrl->queue_count > 1) {
- ret = nvme_rdma_configure_io_queues(ctrl, false);
- if (ret)
- goto out_fail;
- }
-
- changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
-
- nvme_start_ctrl(ctrl);
-
- return;
-
-out_fail:
- dev_warn(ctrl->device, "Removing after reset failure\n");
- nvme_rdma_remove_ctrl(ctrl);
-}
-
static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
{
/* sanity check icdoff */
@@ -1904,7 +1668,7 @@ static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
.reg_write32 = nvmf_reg_write32,
.free_ctrl = nvme_rdma_free_ctrl,
.submit_async_event = nvme_rdma_submit_async_event,
- .delete_ctrl = nvme_rdma_del_ctrl,
+ .delete_ctrl = nvme_del_ctrl,
.get_address = nvmf_get_address,
.alloc_admin_queue = nvme_rdma_alloc_admin_queue,
.free_admin_queue = nvme_rdma_free_admin_queue,
@@ -1920,46 +1684,6 @@ static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
.nr_hw_queues = nvme_rdma_nr_io_queues,
};
-static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
- const struct nvme_ctrl_ops *ops, unsigned long quirks)
-{
- bool changed;
- int ret;
-
- ret = nvme_init_ctrl(ctrl, dev, ops, quirks);
- if (ret)
- return ret;
-
- INIT_WORK(&ctrl->delete_work, nvme_rdma_del_ctrl_work);
- INIT_WORK(&ctrl->reset_work, nvme_rdma_reset_ctrl_work);
-
- ret = nvme_rdma_configure_admin_queue(ctrl, true);
- if (ret)
- goto out_uninit_ctrl;
-
- ret = ctrl->ops->post_configure(ctrl);
- if (ret)
- goto out_remove_admin_queue;
-
- if (ctrl->queue_count > 1) {
- ret = nvme_rdma_configure_io_queues(ctrl, true);
- if (ret)
- goto out_remove_admin_queue;
- }
-
- changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
-
- nvme_start_ctrl(ctrl);
-
- return 0;
-out_remove_admin_queue:
- nvme_rdma_destroy_admin_queue(ctrl, true);
-out_uninit_ctrl:
- nvme_uninit_ctrl(ctrl);
- return ret;
-}
-
static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
@@ -2010,7 +1734,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
nvme_rdma_reconnect_ctrl_work);
INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
- ret = nvme_rdma_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
+ ret = nvme_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
if (!ctrl->queues)
goto out_kfree_queues;
@@ -2059,7 +1783,7 @@ static void nvme_rdma_remove_one(struct ib_device *ib_device, void *client_data)
dev_info(ctrl->ctrl.device,
"Removing ctrl: NQN \"%s\", addr %pISp\n",
ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
- __nvme_rdma_del_ctrl(&ctrl->ctrl);
+ __nvme_del_ctrl(&ctrl->ctrl);
}
mutex_unlock(&nvme_rdma_ctrl_mutex);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 09/12] nvme: move control plane handling to nvme core
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
handle controller setup (probe), reset and delete in nvme-core and
rip it our from nvme-rdma.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/core.c | 296 +++++++++++++++++++++++++++++++++++++++++++++++
drivers/nvme/host/nvme.h | 11 ++
drivers/nvme/host/rdma.c | 290 ++--------------------------------------------
3 files changed, 314 insertions(+), 283 deletions(-)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index bbaf5b98f2fe..4344adff7134 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2913,6 +2913,302 @@ void nvme_start_queues(struct nvme_ctrl *ctrl)
}
EXPORT_SYMBOL_GPL(nvme_start_queues);
+static int nvme_alloc_io_queues(struct nvme_ctrl *ctrl)
+{
+ unsigned int nr_io_queues;
+ int ret;
+
+ nr_io_queues = ctrl->ops->nr_hw_queues(ctrl);
+ ret = nvme_set_queue_count(ctrl, &nr_io_queues);
+ if (ret)
+ return ret;
+
+ ctrl->queue_count = nr_io_queues + 1;
+ if (ctrl->queue_count < 2)
+ return 0;
+
+ dev_info(ctrl->device,
+ "creating %d I/O queues.\n", nr_io_queues);
+
+ return ctrl->ops->alloc_io_queues(ctrl);
+}
+
+void nvme_destroy_io_queues(struct nvme_ctrl *ctrl, bool remove)
+{
+ ctrl->ops->stop_io_queues(ctrl);
+ if (remove) {
+ if (ctrl->ops->flags & NVME_F_FABRICS)
+ blk_cleanup_queue(ctrl->connect_q);
+ ctrl->ops->free_tagset(ctrl, false);
+ }
+ ctrl->ops->free_io_queues(ctrl);
+}
+EXPORT_SYMBOL_GPL(nvme_destroy_io_queues);
+
+int nvme_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
+{
+ int ret;
+
+ ret = nvme_alloc_io_queues(ctrl);
+ if (ret)
+ return ret;
+
+ if (new) {
+ ctrl->tagset = ctrl->ops->alloc_tagset(ctrl, false);
+ if (IS_ERR(ctrl->tagset)) {
+ ret = PTR_ERR(ctrl->tagset);
+ goto out_free_io_queues;
+ }
+
+ if (ctrl->ops->flags & NVME_F_FABRICS) {
+ ctrl->connect_q = blk_mq_init_queue(ctrl->tagset);
+ if (IS_ERR(ctrl->connect_q)) {
+ ret = PTR_ERR(ctrl->connect_q);
+ goto out_free_tag_set;
+ }
+ }
+ } else {
+ ret = blk_mq_reinit_tagset(ctrl->tagset);
+ if (ret)
+ goto out_free_io_queues;
+
+ blk_mq_update_nr_hw_queues(ctrl->tagset,
+ ctrl->queue_count - 1);
+ }
+
+ ret = ctrl->ops->start_io_queues(ctrl);
+ if (ret)
+ goto out_cleanup_connect_q;
+
+ return 0;
+
+out_cleanup_connect_q:
+ if (new && (ctrl->ops->flags & NVME_F_FABRICS))
+ blk_cleanup_queue(ctrl->connect_q);
+out_free_tag_set:
+ if (new)
+ ctrl->ops->free_tagset(ctrl, false);
+out_free_io_queues:
+ ctrl->ops->free_io_queues(ctrl);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(nvme_configure_io_queues);
+
+void nvme_destroy_admin_queue(struct nvme_ctrl *ctrl, bool remove)
+{
+ ctrl->ops->stop_admin_queue(ctrl);
+ if (remove) {
+ blk_cleanup_queue(ctrl->admin_q);
+ ctrl->ops->free_tagset(ctrl, true);
+ }
+ ctrl->ops->free_admin_queue(ctrl);
+}
+EXPORT_SYMBOL_GPL(nvme_destroy_admin_queue);
+
+int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
+{
+ int error;
+
+ error = ctrl->ops->alloc_admin_queue(ctrl);
+ if (error)
+ return error;
+
+ if (new) {
+ ctrl->admin_tagset = ctrl->ops->alloc_tagset(ctrl, true);
+ if (IS_ERR(ctrl->admin_tagset)) {
+ error = PTR_ERR(ctrl->admin_tagset);
+ goto out_free_queue;
+ }
+
+ ctrl->admin_q = blk_mq_init_queue(ctrl->admin_tagset);
+ if (IS_ERR(ctrl->admin_q)) {
+ error = PTR_ERR(ctrl->admin_q);
+ goto out_free_tagset;
+ }
+ } else {
+ error = blk_mq_reinit_tagset(ctrl->admin_tagset);
+ if (error)
+ goto out_free_queue;
+ }
+
+ error = ctrl->ops->start_admin_queue(ctrl);
+ if (error)
+ goto out_cleanup_queue;
+
+ error = ctrl->ops->reg_read64(ctrl, NVME_REG_CAP, &ctrl->cap);
+ if (error) {
+ dev_err(ctrl->device,
+ "prop_get NVME_REG_CAP failed\n");
+ goto out_cleanup_queue;
+ }
+
+ ctrl->sqsize = min_t(int, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
+
+ error = nvme_enable_ctrl(ctrl, ctrl->cap);
+ if (error)
+ goto out_cleanup_queue;
+
+ error = nvme_init_identify(ctrl);
+ if (error)
+ goto out_cleanup_queue;
+
+ return 0;
+
+out_cleanup_queue:
+ if (new)
+ blk_cleanup_queue(ctrl->admin_q);
+out_free_tagset:
+ if (new)
+ ctrl->ops->free_tagset(ctrl, true);
+out_free_queue:
+ ctrl->ops->free_admin_queue(ctrl);
+ return error;
+}
+EXPORT_SYMBOL_GPL(nvme_configure_admin_queue);
+
+static void nvme_teardown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
+{
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
+ nvme_destroy_io_queues(ctrl, shutdown);
+ }
+
+ if (shutdown)
+ nvme_shutdown_ctrl(ctrl);
+ else
+ nvme_disable_ctrl(ctrl, ctrl->cap);
+
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
+ nvme_destroy_admin_queue(ctrl, shutdown);
+}
+
+static void nvme_remove_ctrl(struct nvme_ctrl *ctrl)
+{
+ nvme_remove_namespaces(ctrl);
+ nvme_teardown_ctrl(ctrl, true);
+ nvme_uninit_ctrl(ctrl);
+ nvme_put_ctrl(ctrl);
+}
+
+static void nvme_del_ctrl_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, delete_work);
+
+ nvme_stop_ctrl(ctrl);
+ nvme_remove_ctrl(ctrl);
+}
+
+int __nvme_del_ctrl(struct nvme_ctrl *ctrl)
+{
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
+ return -EBUSY;
+
+ if (!queue_work(nvme_wq, &ctrl->delete_work))
+ return -EBUSY;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(__nvme_del_ctrl);
+
+int nvme_del_ctrl(struct nvme_ctrl *ctrl)
+{
+ int ret = 0;
+
+ /*
+ * Keep a reference until all work is flushed since
+ * __nvme_del_ctrl can free the ctrl mem
+ */
+ if (!kref_get_unless_zero(&ctrl->kref))
+ return -EBUSY;
+
+ ret = __nvme_del_ctrl(ctrl);
+ if (!ret)
+ flush_work(&ctrl->delete_work);
+
+ nvme_put_ctrl(ctrl);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(nvme_del_ctrl);
+
+static void nvme_reset_ctrl_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl =
+ container_of(work, struct nvme_ctrl, reset_work);
+ int ret;
+ bool changed;
+
+ nvme_stop_ctrl(ctrl);
+ nvme_teardown_ctrl(ctrl, false);
+
+ ret = nvme_configure_admin_queue(ctrl, false);
+ if (ret)
+ goto out_fail;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_configure_io_queues(ctrl, false);
+ if (ret)
+ goto out_fail;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+
+ nvme_start_ctrl(ctrl);
+
+ return;
+
+out_fail:
+ dev_warn(ctrl->device, "Removing after reset failure\n");
+ nvme_remove_ctrl(ctrl);
+}
+
+int nvme_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
+ const struct nvme_ctrl_ops *ops, unsigned long quirks)
+{
+ bool changed;
+ int ret;
+
+ ret = nvme_init_ctrl(ctrl, dev, ops, quirks);
+ if (ret)
+ return ret;
+
+ INIT_WORK(&ctrl->delete_work, nvme_del_ctrl_work);
+ INIT_WORK(&ctrl->reset_work, nvme_reset_ctrl_work);
+
+ ret = nvme_configure_admin_queue(ctrl, true);
+ if (ret)
+ goto out_uninit_ctrl;
+
+ ret = ctrl->ops->post_configure(ctrl);
+ if (ret)
+ goto out_remove_admin_queue;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_configure_io_queues(ctrl, true);
+ if (ret)
+ goto out_remove_admin_queue;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+
+ nvme_start_ctrl(ctrl);
+
+ return 0;
+out_remove_admin_queue:
+ nvme_destroy_admin_queue(ctrl, true);
+out_uninit_ctrl:
+ nvme_uninit_ctrl(ctrl);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(nvme_probe_ctrl);
+
int __init nvme_core_init(void)
{
int result;
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index 7b8e57b3e634..b5cefa28d3d6 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -381,6 +381,17 @@ static inline struct nvme_ns *nvme_get_ns_from_dev(struct device *dev)
return dev_to_disk(dev)->private_data;
}
+void nvme_stop_io_queues(struct nvme_ctrl *ctrl);
+void nvme_destroy_io_queues(struct nvme_ctrl *ctrl, bool remove);
+int nvme_configure_io_queues(struct nvme_ctrl *ctrl, bool new);
+void nvme_destroy_admin_queue(struct nvme_ctrl *ctrl, bool remove);
+int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new);
+int __nvme_del_ctrl(struct nvme_ctrl *ctrl);
+int nvme_del_ctrl(struct nvme_ctrl *ctrl);
+int nvme_reset_ctrl(struct nvme_ctrl *ctrl);
+int nvme_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
+ const struct nvme_ctrl_ops *ops, unsigned long quirks);
+
int __init nvme_core_init(void);
void nvme_core_exit(void);
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index cf4e4371c2db..35459f2eea74 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -754,17 +754,6 @@ static struct blk_mq_tag_set *nvme_rdma_alloc_tagset(struct nvme_ctrl *nctrl,
return ERR_PTR(ret);
}
-static void nvme_rdma_destroy_admin_queue(struct nvme_ctrl *ctrl,
- bool remove)
-{
- ctrl->ops->stop_admin_queue(ctrl);
- if (remove) {
- blk_cleanup_queue(ctrl->admin_q);
- ctrl->ops->free_tagset(ctrl, true);
- }
- ctrl->ops->free_admin_queue(ctrl);
-}
-
static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
{
struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
@@ -793,133 +782,6 @@ static int nvme_rdma_alloc_admin_queue(struct nvme_ctrl *nctrl)
return ret;
}
-static int nvme_rdma_configure_admin_queue(struct nvme_ctrl *ctrl,
- bool new)
-{
- int error;
-
- error = ctrl->ops->alloc_admin_queue(ctrl);
- if (error)
- return error;
-
- if (new) {
- ctrl->admin_tagset = ctrl->ops->alloc_tagset(ctrl, true);
- if (IS_ERR(ctrl->admin_tagset))
- goto out_free_queue;
-
- ctrl->admin_q = blk_mq_init_queue(ctrl->admin_tagset);
- if (IS_ERR(ctrl->admin_q)) {
- error = PTR_ERR(ctrl->admin_q);
- goto out_free_tagset;
- }
- } else {
- error = blk_mq_reinit_tagset(ctrl->admin_tagset);
- if (error)
- goto out_free_queue;
- }
-
- error = ctrl->ops->start_admin_queue(ctrl);
- if (error)
- goto out_cleanup_queue;
-
- error = ctrl->ops->reg_read64(ctrl, NVME_REG_CAP,
- &ctrl->cap);
- if (error) {
- dev_err(ctrl->device, "prop_get NVME_REG_CAP failed\n");
- goto out_cleanup_queue;
- }
-
- ctrl->sqsize = min_t(int, NVME_CAP_MQES(ctrl->cap), ctrl->sqsize);
-
- error = nvme_enable_ctrl(ctrl, ctrl->cap);
- if (error)
- goto out_cleanup_queue;
-
- error = nvme_init_identify(ctrl);
- if (error)
- goto out_cleanup_queue;
-
- return 0;
-
-out_cleanup_queue:
- if (new)
- blk_cleanup_queue(ctrl->admin_q);
-out_free_tagset:
- if (new)
- ctrl->ops->free_tagset(ctrl, true);
-out_free_queue:
- ctrl->ops->free_admin_queue(ctrl);
- return error;
-}
-
-static void nvme_rdma_destroy_io_queues(struct nvme_ctrl *ctrl,
- bool remove)
-{
- ctrl->ops->stop_io_queues(ctrl);
- if (remove) {
- blk_cleanup_queue(ctrl->connect_q);
- ctrl->ops->free_tagset(ctrl, false);
- }
- ctrl->ops->free_io_queues(ctrl);
-}
-
-static int nvme_rdma_configure_io_queues(struct nvme_ctrl *ctrl, bool new)
-{
- unsigned int nr_io_queues;
- int ret;
-
- nr_io_queues = ctrl->ops->nr_hw_queues(ctrl);
- ret = nvme_set_queue_count(ctrl, &nr_io_queues);
- if (ret)
- return ret;
-
- ctrl->queue_count = nr_io_queues + 1;
- if (ctrl->queue_count < 2)
- return 0;
-
- dev_info(ctrl->device,
- "creating %d I/O queues.\n", nr_io_queues);
-
- ret = ctrl->ops->alloc_io_queues(ctrl);
- if (ret)
- return ret;
-
- if (new) {
- ctrl->tagset = ctrl->ops->alloc_tagset(ctrl, false);
- if (IS_ERR(ctrl->tagset))
- goto out_free_io_queues;
-
- ctrl->connect_q = blk_mq_init_queue(ctrl->tagset);
- if (IS_ERR(ctrl->connect_q)) {
- ret = PTR_ERR(ctrl->connect_q);
- goto out_free_tag_set;
- }
- } else {
- ret = blk_mq_reinit_tagset(ctrl->tagset);
- if (ret)
- goto out_free_io_queues;
-
- blk_mq_update_nr_hw_queues(ctrl->tagset,
- ctrl->queue_count - 1);
- }
-
- ret = nvme_rdma_start_io_queues(ctrl);
- if (ret)
- goto out_cleanup_connect_q;
-
- return 0;
-
-out_cleanup_connect_q:
- if (new)
- blk_cleanup_queue(ctrl->connect_q);
-out_free_tag_set:
- if (new)
- ctrl->ops->free_tagset(ctrl, false);
-out_free_io_queues:
- ctrl->ops->free_io_queues(ctrl);
- return ret;
-}
-
static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
{
struct nvme_rdma_ctrl *ctrl = to_rdma_ctrl(nctrl);
@@ -967,15 +829,15 @@ static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
++ctrl->nr_reconnects;
if (ctrl->queue_count > 1)
- nvme_rdma_destroy_io_queues(ctrl, false);
+ nvme_destroy_io_queues(ctrl, false);
- nvme_rdma_destroy_admin_queue(ctrl, false);
- ret = nvme_rdma_configure_admin_queue(ctrl, false);
+ nvme_destroy_admin_queue(ctrl, false);
+ ret = nvme_configure_admin_queue(ctrl, false);
if (ret)
goto requeue;
if (ctrl->queue_count > 1) {
- ret = nvme_rdma_configure_io_queues(ctrl, false);
+ ret = nvme_configure_io_queues(ctrl, false);
if (ret)
goto requeue;
}
@@ -1764,104 +1626,6 @@ static const struct blk_mq_ops nvme_rdma_admin_mq_ops = {
.timeout = nvme_rdma_timeout,
};
-static void nvme_rdma_shutdown_ctrl(struct nvme_ctrl *ctrl, bool shutdown)
-{
- if (ctrl->queue_count > 1) {
- nvme_stop_queues(ctrl);
- blk_mq_tagset_busy_iter(ctrl->tagset,
- nvme_cancel_request, ctrl);
- nvme_rdma_destroy_io_queues(ctrl, shutdown);
- }
-
- if (shutdown)
- nvme_shutdown_ctrl(ctrl);
- else
- nvme_disable_ctrl(ctrl, ctrl->cap);
-
- blk_mq_quiesce_queue(ctrl->admin_q);
- blk_mq_tagset_busy_iter(ctrl->admin_tagset,
- nvme_cancel_request, ctrl);
- blk_mq_unquiesce_queue(ctrl->admin_q);
- nvme_rdma_destroy_admin_queue(ctrl, shutdown);
-}
-
-static void nvme_rdma_remove_ctrl(struct nvme_ctrl *ctrl)
-{
- nvme_remove_namespaces(ctrl);
- nvme_rdma_shutdown_ctrl(ctrl, true);
- nvme_uninit_ctrl(ctrl);
- nvme_put_ctrl(ctrl);
-}
-
-static void nvme_rdma_del_ctrl_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl = container_of(work,
- struct nvme_ctrl, delete_work);
-
- nvme_stop_ctrl(ctrl);
- nvme_rdma_remove_ctrl(ctrl);
-}
-
-static int __nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
-{
- if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
- return -EBUSY;
-
- if (!queue_work(nvme_wq, &ctrl->delete_work))
- return -EBUSY;
-
- return 0;
-}
-
-static int nvme_rdma_del_ctrl(struct nvme_ctrl *ctrl)
-{
- int ret = 0;
-
- /*
- * Keep a reference until all work is flushed since
- * __nvme_rdma_del_ctrl can free the ctrl mem
- */
- if (!kref_get_unless_zero(&ctrl->kref))
- return -EBUSY;
- ret = __nvme_rdma_del_ctrl(ctrl);
- if (!ret)
- flush_work(&ctrl->delete_work);
- nvme_put_ctrl(ctrl);
- return ret;
-}
-
-static void nvme_rdma_reset_ctrl_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl =
- container_of(work, struct nvme_ctrl, reset_work);
- int ret;
- bool changed;
-
- nvme_stop_ctrl(ctrl);
- nvme_rdma_shutdown_ctrl(ctrl, false);
-
- ret = nvme_rdma_configure_admin_queue(ctrl, false);
- if (ret)
- goto out_fail;
-
- if (ctrl->queue_count > 1) {
- ret = nvme_rdma_configure_io_queues(ctrl, false);
- if (ret)
- goto out_fail;
- }
-
- changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
-
- nvme_start_ctrl(ctrl);
-
- return;
-
-out_fail:
- dev_warn(ctrl->device, "Removing after reset failure\n");
- nvme_rdma_remove_ctrl(ctrl);
-}
-
static int nvme_rdma_post_configure(struct nvme_ctrl *ctrl)
{
/* sanity check icdoff */
@@ -1904,7 +1668,7 @@ static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
.reg_write32 = nvmf_reg_write32,
.free_ctrl = nvme_rdma_free_ctrl,
.submit_async_event = nvme_rdma_submit_async_event,
- .delete_ctrl = nvme_rdma_del_ctrl,
+ .delete_ctrl = nvme_del_ctrl,
.get_address = nvmf_get_address,
.alloc_admin_queue = nvme_rdma_alloc_admin_queue,
.free_admin_queue = nvme_rdma_free_admin_queue,
@@ -1920,46 +1684,6 @@ static const struct nvme_ctrl_ops nvme_rdma_ctrl_ops = {
.nr_hw_queues = nvme_rdma_nr_io_queues,
};
-static int nvme_rdma_probe_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
- const struct nvme_ctrl_ops *ops, unsigned long quirks)
-{
- bool changed;
- int ret;
-
- ret = nvme_init_ctrl(ctrl, dev, ops, quirks);
- if (ret)
- return ret;
-
- INIT_WORK(&ctrl->delete_work, nvme_rdma_del_ctrl_work);
- INIT_WORK(&ctrl->reset_work, nvme_rdma_reset_ctrl_work);
-
- ret = nvme_rdma_configure_admin_queue(ctrl, true);
- if (ret)
- goto out_uninit_ctrl;
-
- ret = ctrl->ops->post_configure(ctrl);
- if (ret)
- goto out_remove_admin_queue;
-
- if (ctrl->queue_count > 1) {
- ret = nvme_rdma_configure_io_queues(ctrl, true);
- if (ret)
- goto out_remove_admin_queue;
- }
-
- changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
-
- nvme_start_ctrl(ctrl);
-
- return 0;
-out_remove_admin_queue:
- nvme_rdma_destroy_admin_queue(ctrl, true);
-out_uninit_ctrl:
- nvme_uninit_ctrl(ctrl);
- return ret;
-}
-
static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
@@ -2010,7 +1734,7 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
nvme_rdma_reconnect_ctrl_work);
INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
- ret = nvme_rdma_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
+ ret = nvme_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
if (!ctrl->queues)
goto out_kfree_queues;
@@ -2059,7 +1783,7 @@ static void nvme_rdma_remove_one(struct ib_device *ib_device, void *client_data)
dev_info(ctrl->ctrl.device,
"Removing ctrl: NQN \"%s\", addr %pISp\n",
ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
- __nvme_rdma_del_ctrl(&ctrl->ctrl);
+ __nvme_del_ctrl(&ctrl->ctrl);
}
mutex_unlock(&nvme_rdma_ctrl_mutex);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 10/12] nvme-fabrics: handle reconnects in fabrics library
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
Rip the nvme-rdma equivalent.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/fabrics.c | 103 +++++++++++++++++++++++++++++++++++++++
drivers/nvme/host/fabrics.h | 1 +
drivers/nvme/host/rdma.c | 114 +++-----------------------------------------
3 files changed, 110 insertions(+), 108 deletions(-)
diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index cf8c6163db9e..8e03360f45b3 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -819,6 +819,105 @@ void nvmf_free_options(struct nvmf_ctrl_options *opts)
}
EXPORT_SYMBOL_GPL(nvmf_free_options);
+static void nvmf_reconnect_or_remove(struct nvme_ctrl *ctrl)
+{
+ /* If we are resetting/deleting then do nothing */
+ if (ctrl->state != NVME_CTRL_RECONNECTING) {
+ WARN_ON_ONCE(ctrl->state == NVME_CTRL_NEW ||
+ ctrl->state == NVME_CTRL_LIVE);
+ return;
+ }
+
+ if (nvmf_should_reconnect(ctrl)) {
+ dev_info(ctrl->device, "Reconnecting in %d seconds...\n",
+ ctrl->opts->reconnect_delay);
+ queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
+ ctrl->opts->reconnect_delay * HZ);
+ } else {
+ dev_info(ctrl->device, "Removing controller...\n");
+ __nvme_del_ctrl(ctrl);
+ }
+}
+
+static void nvmf_reconnect_ctrl_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
+ struct nvme_ctrl, reconnect_work);
+ bool changed;
+ int ret;
+
+ ++ctrl->nr_reconnects;
+
+ if (ctrl->queue_count > 1)
+ nvme_destroy_io_queues(ctrl, false);
+
+ nvme_destroy_admin_queue(ctrl, false);
+ ret = nvme_configure_admin_queue(ctrl, false);
+ if (ret)
+ goto requeue;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_configure_io_queues(ctrl, false);
+ if (ret)
+ goto requeue;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+ ctrl->nr_reconnects = 0;
+
+ nvme_start_ctrl(ctrl);
+
+ dev_info(ctrl->device, "Successfully reconnected\n");
+
+ return;
+
+requeue:
+ dev_info(ctrl->device, "Failed reconnect attempt %d\n",
+ ctrl->nr_reconnects);
+ nvmf_reconnect_or_remove(ctrl);
+}
+
+static void nvmf_error_recovery_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, err_work);
+
+ nvme_stop_keep_alive(ctrl);
+
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
+ ctrl->ops->stop_io_queues(ctrl);
+ }
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ ctrl->ops->stop_admin_queue(ctrl);
+
+ /* We must take care of fastfail/requeue all our inflight requests */
+ if (ctrl->queue_count > 1)
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
+
+ /*
+ * queues are not a live anymore, so restart the queues to fail fast
+ * new IO
+ */
+ nvme_start_queues(ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
+
+ nvmf_reconnect_or_remove(ctrl);
+}
+
+void nvmf_error_recovery(struct nvme_ctrl *ctrl)
+{
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RECONNECTING))
+ return;
+
+ queue_work(nvme_wq, &ctrl->err_work);
+}
+EXPORT_SYMBOL_GPL(nvmf_error_recovery);
+
#define NVMF_REQUIRED_OPTS (NVMF_OPT_TRANSPORT | NVMF_OPT_NQN)
#define NVMF_ALLOWED_OPTS (NVMF_OPT_QUEUE_SIZE | NVMF_OPT_NR_IO_QUEUES | \
NVMF_OPT_KATO | NVMF_OPT_HOSTNQN | \
@@ -882,6 +981,10 @@ nvmf_create_ctrl(struct device *dev, const char *buf, size_t count)
return ERR_PTR(-EINVAL);
}
+ INIT_DELAYED_WORK(&ctrl->reconnect_work,
+ nvmf_reconnect_ctrl_work);
+ INIT_WORK(&ctrl->err_work, nvmf_error_recovery_work);
+
mutex_unlock(&nvmf_transports_mutex);
return ctrl;
diff --git a/drivers/nvme/host/fabrics.h b/drivers/nvme/host/fabrics.h
index bf33663218cd..20e9f45b8658 100644
--- a/drivers/nvme/host/fabrics.h
+++ b/drivers/nvme/host/fabrics.h
@@ -140,6 +140,7 @@ int nvmf_register_transport(struct nvmf_transport_ops *ops);
void nvmf_unregister_transport(struct nvmf_transport_ops *ops);
void nvmf_free_options(struct nvmf_ctrl_options *opts);
int nvmf_get_address(struct nvme_ctrl *ctrl, char *buf, int size);
+void nvmf_error_recovery(struct nvme_ctrl *ctrl);
bool nvmf_should_reconnect(struct nvme_ctrl *ctrl);
#endif /* _NVME_FABRICS_H */
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 35459f2eea74..3bdf1ab7c2f3 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -799,104 +799,6 @@ static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
kfree(ctrl);
}
-static void nvme_rdma_reconnect_or_remove(struct nvme_ctrl *ctrl)
-{
- /* If we are resetting/deleting then do nothing */
- if (ctrl->state != NVME_CTRL_RECONNECTING) {
- WARN_ON_ONCE(ctrl->state == NVME_CTRL_NEW ||
- ctrl->state == NVME_CTRL_LIVE);
- return;
- }
-
- if (nvmf_should_reconnect(ctrl)) {
- dev_info(ctrl->device, "Reconnecting in %d seconds...\n",
- ctrl->opts->reconnect_delay);
- queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
- ctrl->opts->reconnect_delay * HZ);
- } else {
- dev_info(ctrl->device, "Removing controller...\n");
- queue_work(nvme_wq, &ctrl->delete_work);
- }
-}
-
-static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
- struct nvme_ctrl, reconnect_work);
- bool changed;
- int ret;
-
- ++ctrl->nr_reconnects;
-
- if (ctrl->queue_count > 1)
- nvme_destroy_io_queues(ctrl, false);
-
- nvme_destroy_admin_queue(ctrl, false);
- ret = nvme_configure_admin_queue(ctrl, false);
- if (ret)
- goto requeue;
-
- if (ctrl->queue_count > 1) {
- ret = nvme_configure_io_queues(ctrl, false);
- if (ret)
- goto requeue;
- }
-
- changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
- ctrl->nr_reconnects = 0;
-
- nvme_start_ctrl(ctrl);
-
- dev_info(ctrl->device, "Successfully reconnected\n");
-
- return;
-
-requeue:
- dev_info(ctrl->device, "Failed reconnect attempt %d\n",
- ctrl->nr_reconnects);
- nvme_rdma_reconnect_or_remove(ctrl);
-}
-
-static void nvme_rdma_error_recovery_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl = container_of(work,
- struct nvme_ctrl, err_work);
-
- nvme_stop_ctrl(ctrl);
-
- if (ctrl->queue_count > 1) {
- nvme_stop_queues(ctrl);
- ctrl->ops->stop_io_queues(ctrl);
- }
- blk_mq_quiesce_queue(ctrl->admin_q);
- ctrl->ops->stop_admin_queue(ctrl);
-
- /* We must take care of fastfail/requeue all our inflight requests */
- if (ctrl->queue_count > 1)
- blk_mq_tagset_busy_iter(ctrl->tagset,
- nvme_cancel_request, ctrl);
- blk_mq_tagset_busy_iter(ctrl->admin_tagset,
- nvme_cancel_request, ctrl);
-
- /*
- * queues are not a live anymore, so restart the queues to fail fast
- * new IO
- */
- blk_mq_unquiesce_queue(ctrl->admin_q);
- nvme_start_queues(ctrl);
-
- nvme_rdma_reconnect_or_remove(ctrl);
-}
-
-static void nvme_rdma_error_recovery(struct nvme_ctrl *ctrl)
-{
- if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RECONNECTING))
- return;
-
- queue_work(nvme_wq, &ctrl->err_work);
-}
-
static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
const char *op)
{
@@ -908,7 +810,7 @@ static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
"%s for CQE 0x%p failed with status %s (%d)\n",
op, wc->wr_cqe,
ib_wc_status_msg(wc->status), wc->status);
- nvme_rdma_error_recovery(&ctrl->ctrl);
+ nvmf_error_recovery(&ctrl->ctrl);
}
static void nvme_rdma_memreg_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -959,7 +861,7 @@ static void nvme_rdma_unmap_data(struct nvme_rdma_queue *queue,
dev_err(ctrl->ctrl.device,
"Queueing INV WR for rkey %#x failed (%d)\n",
req->mr->rkey, res);
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
}
}
@@ -1238,7 +1140,7 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
dev_err(queue->ctrl->ctrl.device,
"tag 0x%x on QP %#x not found\n",
cqe->command_id, queue->qp->qp_num);
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
return ret;
}
req = blk_mq_rq_to_pdu(rq);
@@ -1449,7 +1351,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
case RDMA_CM_EVENT_TIMEWAIT_EXIT:
dev_dbg(queue->ctrl->ctrl.device,
"disconnect received - connection closed\n");
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
break;
case RDMA_CM_EVENT_DEVICE_REMOVAL:
/* device removal is handled via the ib_client API */
@@ -1457,7 +1359,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
default:
dev_err(queue->ctrl->ctrl.device,
"Unexpected RDMA CM event (%d)\n", ev->event);
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
break;
}
@@ -1475,7 +1377,7 @@ nvme_rdma_timeout(struct request *rq, bool reserved)
struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq);
/* queue error recovery */
- nvme_rdma_error_recovery(&req->queue->ctrl->ctrl);
+ nvmf_error_recovery(&req->queue->ctrl->ctrl);
/* fail with DNR on cmd timeout */
nvme_req(rq)->status = NVME_SC_ABORT_REQ | NVME_SC_DNR;
@@ -1730,10 +1632,6 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
goto out_free_ctrl;
}
- INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
- nvme_rdma_reconnect_ctrl_work);
- INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
-
ret = nvme_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
if (!ctrl->queues)
goto out_kfree_queues;
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 10/12] nvme-fabrics: handle reconnects in fabrics library
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
Rip the nvme-rdma equivalent.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/fabrics.c | 103 +++++++++++++++++++++++++++++++++++++++
drivers/nvme/host/fabrics.h | 1 +
drivers/nvme/host/rdma.c | 114 +++-----------------------------------------
3 files changed, 110 insertions(+), 108 deletions(-)
diff --git a/drivers/nvme/host/fabrics.c b/drivers/nvme/host/fabrics.c
index cf8c6163db9e..8e03360f45b3 100644
--- a/drivers/nvme/host/fabrics.c
+++ b/drivers/nvme/host/fabrics.c
@@ -819,6 +819,105 @@ void nvmf_free_options(struct nvmf_ctrl_options *opts)
}
EXPORT_SYMBOL_GPL(nvmf_free_options);
+static void nvmf_reconnect_or_remove(struct nvme_ctrl *ctrl)
+{
+ /* If we are resetting/deleting then do nothing */
+ if (ctrl->state != NVME_CTRL_RECONNECTING) {
+ WARN_ON_ONCE(ctrl->state == NVME_CTRL_NEW ||
+ ctrl->state == NVME_CTRL_LIVE);
+ return;
+ }
+
+ if (nvmf_should_reconnect(ctrl)) {
+ dev_info(ctrl->device, "Reconnecting in %d seconds...\n",
+ ctrl->opts->reconnect_delay);
+ queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
+ ctrl->opts->reconnect_delay * HZ);
+ } else {
+ dev_info(ctrl->device, "Removing controller...\n");
+ __nvme_del_ctrl(ctrl);
+ }
+}
+
+static void nvmf_reconnect_ctrl_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
+ struct nvme_ctrl, reconnect_work);
+ bool changed;
+ int ret;
+
+ ++ctrl->nr_reconnects;
+
+ if (ctrl->queue_count > 1)
+ nvme_destroy_io_queues(ctrl, false);
+
+ nvme_destroy_admin_queue(ctrl, false);
+ ret = nvme_configure_admin_queue(ctrl, false);
+ if (ret)
+ goto requeue;
+
+ if (ctrl->queue_count > 1) {
+ ret = nvme_configure_io_queues(ctrl, false);
+ if (ret)
+ goto requeue;
+ }
+
+ changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
+ WARN_ON_ONCE(!changed);
+ ctrl->nr_reconnects = 0;
+
+ nvme_start_ctrl(ctrl);
+
+ dev_info(ctrl->device, "Successfully reconnected\n");
+
+ return;
+
+requeue:
+ dev_info(ctrl->device, "Failed reconnect attempt %d\n",
+ ctrl->nr_reconnects);
+ nvmf_reconnect_or_remove(ctrl);
+}
+
+static void nvmf_error_recovery_work(struct work_struct *work)
+{
+ struct nvme_ctrl *ctrl = container_of(work,
+ struct nvme_ctrl, err_work);
+
+ nvme_stop_keep_alive(ctrl);
+
+ if (ctrl->queue_count > 1) {
+ nvme_stop_queues(ctrl);
+ ctrl->ops->stop_io_queues(ctrl);
+ }
+ blk_mq_quiesce_queue(ctrl->admin_q);
+ ctrl->ops->stop_admin_queue(ctrl);
+
+ /* We must take care of fastfail/requeue all our inflight requests */
+ if (ctrl->queue_count > 1)
+ blk_mq_tagset_busy_iter(ctrl->tagset,
+ nvme_cancel_request, ctrl);
+ blk_mq_tagset_busy_iter(ctrl->admin_tagset,
+ nvme_cancel_request, ctrl);
+
+ /*
+ * queues are not a live anymore, so restart the queues to fail fast
+ * new IO
+ */
+ nvme_start_queues(ctrl);
+ blk_mq_unquiesce_queue(ctrl->admin_q);
+
+ nvmf_reconnect_or_remove(ctrl);
+}
+
+void nvmf_error_recovery(struct nvme_ctrl *ctrl)
+{
+ if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RECONNECTING))
+ return;
+
+ queue_work(nvme_wq, &ctrl->err_work);
+}
+EXPORT_SYMBOL_GPL(nvmf_error_recovery);
+
#define NVMF_REQUIRED_OPTS (NVMF_OPT_TRANSPORT | NVMF_OPT_NQN)
#define NVMF_ALLOWED_OPTS (NVMF_OPT_QUEUE_SIZE | NVMF_OPT_NR_IO_QUEUES | \
NVMF_OPT_KATO | NVMF_OPT_HOSTNQN | \
@@ -882,6 +981,10 @@ nvmf_create_ctrl(struct device *dev, const char *buf, size_t count)
return ERR_PTR(-EINVAL);
}
+ INIT_DELAYED_WORK(&ctrl->reconnect_work,
+ nvmf_reconnect_ctrl_work);
+ INIT_WORK(&ctrl->err_work, nvmf_error_recovery_work);
+
mutex_unlock(&nvmf_transports_mutex);
return ctrl;
diff --git a/drivers/nvme/host/fabrics.h b/drivers/nvme/host/fabrics.h
index bf33663218cd..20e9f45b8658 100644
--- a/drivers/nvme/host/fabrics.h
+++ b/drivers/nvme/host/fabrics.h
@@ -140,6 +140,7 @@ int nvmf_register_transport(struct nvmf_transport_ops *ops);
void nvmf_unregister_transport(struct nvmf_transport_ops *ops);
void nvmf_free_options(struct nvmf_ctrl_options *opts);
int nvmf_get_address(struct nvme_ctrl *ctrl, char *buf, int size);
+void nvmf_error_recovery(struct nvme_ctrl *ctrl);
bool nvmf_should_reconnect(struct nvme_ctrl *ctrl);
#endif /* _NVME_FABRICS_H */
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 35459f2eea74..3bdf1ab7c2f3 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -799,104 +799,6 @@ static void nvme_rdma_free_ctrl(struct nvme_ctrl *nctrl)
kfree(ctrl);
}
-static void nvme_rdma_reconnect_or_remove(struct nvme_ctrl *ctrl)
-{
- /* If we are resetting/deleting then do nothing */
- if (ctrl->state != NVME_CTRL_RECONNECTING) {
- WARN_ON_ONCE(ctrl->state == NVME_CTRL_NEW ||
- ctrl->state == NVME_CTRL_LIVE);
- return;
- }
-
- if (nvmf_should_reconnect(ctrl)) {
- dev_info(ctrl->device, "Reconnecting in %d seconds...\n",
- ctrl->opts->reconnect_delay);
- queue_delayed_work(nvme_wq, &ctrl->reconnect_work,
- ctrl->opts->reconnect_delay * HZ);
- } else {
- dev_info(ctrl->device, "Removing controller...\n");
- queue_work(nvme_wq, &ctrl->delete_work);
- }
-}
-
-static void nvme_rdma_reconnect_ctrl_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl = container_of(to_delayed_work(work),
- struct nvme_ctrl, reconnect_work);
- bool changed;
- int ret;
-
- ++ctrl->nr_reconnects;
-
- if (ctrl->queue_count > 1)
- nvme_destroy_io_queues(ctrl, false);
-
- nvme_destroy_admin_queue(ctrl, false);
- ret = nvme_configure_admin_queue(ctrl, false);
- if (ret)
- goto requeue;
-
- if (ctrl->queue_count > 1) {
- ret = nvme_configure_io_queues(ctrl, false);
- if (ret)
- goto requeue;
- }
-
- changed = nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
- ctrl->nr_reconnects = 0;
-
- nvme_start_ctrl(ctrl);
-
- dev_info(ctrl->device, "Successfully reconnected\n");
-
- return;
-
-requeue:
- dev_info(ctrl->device, "Failed reconnect attempt %d\n",
- ctrl->nr_reconnects);
- nvme_rdma_reconnect_or_remove(ctrl);
-}
-
-static void nvme_rdma_error_recovery_work(struct work_struct *work)
-{
- struct nvme_ctrl *ctrl = container_of(work,
- struct nvme_ctrl, err_work);
-
- nvme_stop_ctrl(ctrl);
-
- if (ctrl->queue_count > 1) {
- nvme_stop_queues(ctrl);
- ctrl->ops->stop_io_queues(ctrl);
- }
- blk_mq_quiesce_queue(ctrl->admin_q);
- ctrl->ops->stop_admin_queue(ctrl);
-
- /* We must take care of fastfail/requeue all our inflight requests */
- if (ctrl->queue_count > 1)
- blk_mq_tagset_busy_iter(ctrl->tagset,
- nvme_cancel_request, ctrl);
- blk_mq_tagset_busy_iter(ctrl->admin_tagset,
- nvme_cancel_request, ctrl);
-
- /*
- * queues are not a live anymore, so restart the queues to fail fast
- * new IO
- */
- blk_mq_unquiesce_queue(ctrl->admin_q);
- nvme_start_queues(ctrl);
-
- nvme_rdma_reconnect_or_remove(ctrl);
-}
-
-static void nvme_rdma_error_recovery(struct nvme_ctrl *ctrl)
-{
- if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_RECONNECTING))
- return;
-
- queue_work(nvme_wq, &ctrl->err_work);
-}
-
static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
const char *op)
{
@@ -908,7 +810,7 @@ static void nvme_rdma_wr_error(struct ib_cq *cq, struct ib_wc *wc,
"%s for CQE 0x%p failed with status %s (%d)\n",
op, wc->wr_cqe,
ib_wc_status_msg(wc->status), wc->status);
- nvme_rdma_error_recovery(&ctrl->ctrl);
+ nvmf_error_recovery(&ctrl->ctrl);
}
static void nvme_rdma_memreg_done(struct ib_cq *cq, struct ib_wc *wc)
@@ -959,7 +861,7 @@ static void nvme_rdma_unmap_data(struct nvme_rdma_queue *queue,
dev_err(ctrl->ctrl.device,
"Queueing INV WR for rkey %#x failed (%d)\n",
req->mr->rkey, res);
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
}
}
@@ -1238,7 +1140,7 @@ static int nvme_rdma_process_nvme_rsp(struct nvme_rdma_queue *queue,
dev_err(queue->ctrl->ctrl.device,
"tag 0x%x on QP %#x not found\n",
cqe->command_id, queue->qp->qp_num);
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
return ret;
}
req = blk_mq_rq_to_pdu(rq);
@@ -1449,7 +1351,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
case RDMA_CM_EVENT_TIMEWAIT_EXIT:
dev_dbg(queue->ctrl->ctrl.device,
"disconnect received - connection closed\n");
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
break;
case RDMA_CM_EVENT_DEVICE_REMOVAL:
/* device removal is handled via the ib_client API */
@@ -1457,7 +1359,7 @@ static int nvme_rdma_cm_handler(struct rdma_cm_id *cm_id,
default:
dev_err(queue->ctrl->ctrl.device,
"Unexpected RDMA CM event (%d)\n", ev->event);
- nvme_rdma_error_recovery(&queue->ctrl->ctrl);
+ nvmf_error_recovery(&queue->ctrl->ctrl);
break;
}
@@ -1475,7 +1377,7 @@ nvme_rdma_timeout(struct request *rq, bool reserved)
struct nvme_rdma_request *req = blk_mq_rq_to_pdu(rq);
/* queue error recovery */
- nvme_rdma_error_recovery(&req->queue->ctrl->ctrl);
+ nvmf_error_recovery(&req->queue->ctrl->ctrl);
/* fail with DNR on cmd timeout */
nvme_req(rq)->status = NVME_SC_ABORT_REQ | NVME_SC_DNR;
@@ -1730,10 +1632,6 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
goto out_free_ctrl;
}
- INIT_DELAYED_WORK(&ctrl->ctrl.reconnect_work,
- nvme_rdma_reconnect_ctrl_work);
- INIT_WORK(&ctrl->ctrl.err_work, nvme_rdma_error_recovery_work);
-
ret = nvme_probe_ctrl(&ctrl->ctrl, dev, &nvme_rdma_ctrl_ops, 0);
if (!ctrl->queues)
goto out_kfree_queues;
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 11/12] nvme: add sed-opal ctrl manipulation in admin configuration
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
drivers/nvme/host/core.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4344adff7134..19aa68f1fb4a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2743,6 +2743,7 @@ static void nvme_free_ctrl(struct kref *kref)
put_device(ctrl->device);
nvme_release_instance(ctrl);
ida_destroy(&ctrl->ns_ida);
+ free_opal_dev(ctrl->opal_dev);
ctrl->ops->free_ctrl(ctrl);
}
@@ -3007,6 +3008,7 @@ EXPORT_SYMBOL_GPL(nvme_destroy_admin_queue);
int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
{
+ bool was_suspend = !!(ctrl->ctrl_config & NVME_CC_SHN_NORMAL);
int error;
error = ctrl->ops->alloc_admin_queue(ctrl);
@@ -3052,6 +3054,16 @@ int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
if (error)
goto out_cleanup_queue;
+ if (ctrl->oacs & NVME_CTRL_OACS_SEC_SUPP) {
+ if (!ctrl->opal_dev)
+ ctrl->opal_dev = init_opal_dev(ctrl, &nvme_sec_submit);
+ else if (was_suspend)
+ opal_unlock_from_suspend(ctrl->opal_dev);
+ } else {
+ free_opal_dev(ctrl->opal_dev);
+ ctrl->opal_dev = NULL;
+ }
+
return 0;
out_cleanup_queue:
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 11/12] nvme: add sed-opal ctrl manipulation in admin configuration
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
drivers/nvme/host/core.c | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 4344adff7134..19aa68f1fb4a 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2743,6 +2743,7 @@ static void nvme_free_ctrl(struct kref *kref)
put_device(ctrl->device);
nvme_release_instance(ctrl);
ida_destroy(&ctrl->ns_ida);
+ free_opal_dev(ctrl->opal_dev);
ctrl->ops->free_ctrl(ctrl);
}
@@ -3007,6 +3008,7 @@ EXPORT_SYMBOL_GPL(nvme_destroy_admin_queue);
int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
{
+ bool was_suspend = !!(ctrl->ctrl_config & NVME_CC_SHN_NORMAL);
int error;
error = ctrl->ops->alloc_admin_queue(ctrl);
@@ -3052,6 +3054,16 @@ int nvme_configure_admin_queue(struct nvme_ctrl *ctrl, bool new)
if (error)
goto out_cleanup_queue;
+ if (ctrl->oacs & NVME_CTRL_OACS_SEC_SUPP) {
+ if (!ctrl->opal_dev)
+ ctrl->opal_dev = init_opal_dev(ctrl, &nvme_sec_submit);
+ else if (was_suspend)
+ opal_unlock_from_suspend(ctrl->opal_dev);
+ } else {
+ free_opal_dev(ctrl->opal_dev);
+ ctrl->opal_dev = NULL;
+ }
+
return 0;
out_cleanup_queue:
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 12/12] nvme-loop: convert to nvme-core control plane management
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-15 9:52 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
To: linux-nvme, Christoph Hellwig, Keith Busch; +Cc: linux-block
rip out all the controller and queues control plane code,
only maintain queue alloc/free/start/stop and tagset alloc/free.
Signed-off-by: Sagi Grimberg <sagi@grimberg.me>
---
This patch failed to generate a nice diff :(
drivers/nvme/target/loop.c | 443 ++++++++++++++++-----------------------------
1 file changed, 160 insertions(+), 283 deletions(-)
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index 92628c432926..feba730dd9f1 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -273,260 +273,191 @@ static const struct blk_mq_ops nvme_loop_admin_mq_ops = {
.timeout = nvme_loop_timeout,
};
-static void nvme_loop_destroy_admin_queue(struct nvme_loop_ctrl *ctrl)
+static unsigned int nvme_loop_nr_io_queues(struct nvme_ctrl *ctrl)
{
- nvmet_sq_destroy(&ctrl->queues[0].nvme_sq);
- blk_cleanup_queue(ctrl->ctrl.admin_q);
- blk_mq_free_tag_set(&ctrl->admin_tag_set);
+ return num_online_cpus();
}
-static void nvme_loop_free_ctrl(struct nvme_ctrl *nctrl)
+static int nvme_loop_post_configure(struct nvme_ctrl *ctrl)
{
- struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
-
- if (list_empty(&ctrl->list))
- goto free_ctrl;
+ struct nvmf_ctrl_options *opts = ctrl->opts;
- mutex_lock(&nvme_loop_ctrl_mutex);
- list_del(&ctrl->list);
- mutex_unlock(&nvme_loop_ctrl_mutex);
-
- if (nctrl->tagset) {
- blk_cleanup_queue(ctrl->ctrl.connect_q);
- blk_mq_free_tag_set(&ctrl->tag_set);
+ if (opts->queue_size > ctrl->maxcmd) {
+ /* warn if maxcmd is lower than queue_size */
+ dev_warn(ctrl->device,
+ "queue_size %zu > ctrl maxcmd %u, clamping down\n",
+ opts->queue_size, ctrl->maxcmd);
+ opts->queue_size = ctrl->maxcmd;
}
- kfree(ctrl->queues);
- nvmf_free_options(nctrl->opts);
-free_ctrl:
- kfree(ctrl);
+
+ return 0;
}
-static void nvme_loop_destroy_io_queues(struct nvme_loop_ctrl *ctrl)
+static void nvme_loop_free_tagset(struct nvme_ctrl *nctrl, bool admin)
{
- int i;
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
+ struct blk_mq_tag_set *set = admin ?
+ &ctrl->admin_tag_set : &ctrl->tag_set;
- for (i = 1; i < ctrl->ctrl.queue_count; i++)
- nvmet_sq_destroy(&ctrl->queues[i].nvme_sq);
+ blk_mq_free_tag_set(set);
}
-static int nvme_loop_init_io_queues(struct nvme_loop_ctrl *ctrl)
+static struct blk_mq_tag_set *nvme_loop_alloc_tagset(struct nvme_ctrl *nctrl,
+ bool admin)
{
- struct nvmf_ctrl_options *opts = ctrl->ctrl.opts;
- unsigned int nr_io_queues;
- int ret, i;
-
- nr_io_queues = min(opts->nr_io_queues, num_online_cpus());
- ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
- if (ret || !nr_io_queues)
- return ret;
-
- dev_info(ctrl->ctrl.device, "creating %d I/O queues.\n", nr_io_queues);
-
- for (i = 1; i <= nr_io_queues; i++) {
- ctrl->queues[i].ctrl = ctrl;
- ret = nvmet_sq_init(&ctrl->queues[i].nvme_sq);
- if (ret)
- goto out_destroy_queues;
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
+ struct blk_mq_tag_set *set;
+ int ret;
- ctrl->ctrl.queue_count++;
+ if (admin) {
+ set = &ctrl->admin_tag_set;
+ memset(set, 0, sizeof(*set));
+ set->ops = &nvme_loop_admin_mq_ops;
+ set->queue_depth = NVME_LOOP_AQ_BLKMQ_DEPTH;
+ set->reserved_tags = 2; /* connect + keep-alive */
+ set->numa_node = NUMA_NO_NODE;
+ set->cmd_size = sizeof(struct nvme_loop_iod) +
+ SG_CHUNK_SIZE * sizeof(struct scatterlist);
+ set->driver_data = ctrl;
+ set->nr_hw_queues = 1;
+ set->timeout = ADMIN_TIMEOUT;
+ } else {
+ set = &ctrl->tag_set;
+ memset(set, 0, sizeof(*set));
+ set->ops = &nvme_loop_mq_ops;
+ set->queue_depth = nctrl->opts->queue_size;
+ set->reserved_tags = 1; /* fabric connect */
+ set->numa_node = NUMA_NO_NODE;
+ set->flags = BLK_MQ_F_SHOULD_MERGE;
+ set->cmd_size = sizeof(struct nvme_loop_iod) +
+ SG_CHUNK_SIZE * sizeof(struct scatterlist);
+ set->driver_data = ctrl;
+ set->nr_hw_queues = nctrl->queue_count - 1;
+ set->timeout = NVME_IO_TIMEOUT;
}
- return 0;
+ ret = blk_mq_alloc_tag_set(set);
+ if (ret)
+ return ERR_PTR(ret);
-out_destroy_queues:
- nvme_loop_destroy_io_queues(ctrl);
- return ret;
+ return set;
}
-static int nvme_loop_connect_io_queues(struct nvme_loop_ctrl *ctrl)
+static void nvme_loop_free_admin_queue(struct nvme_ctrl *ctrl)
{
- int i, ret;
-
- for (i = 1; i < ctrl->ctrl.queue_count; i++) {
- ret = nvmf_connect_io_queue(&ctrl->ctrl, i);
- if (ret)
- return ret;
- }
-
- return 0;
}
-static int nvme_loop_configure_admin_queue(struct nvme_loop_ctrl *ctrl)
-{
- int error;
-
- memset(&ctrl->admin_tag_set, 0, sizeof(ctrl->admin_tag_set));
- ctrl->admin_tag_set.ops = &nvme_loop_admin_mq_ops;
- ctrl->admin_tag_set.queue_depth = NVME_LOOP_AQ_BLKMQ_DEPTH;
- ctrl->admin_tag_set.reserved_tags = 2; /* connect + keep-alive */
- ctrl->admin_tag_set.numa_node = NUMA_NO_NODE;
- ctrl->admin_tag_set.cmd_size = sizeof(struct nvme_loop_iod) +
- SG_CHUNK_SIZE * sizeof(struct scatterlist);
- ctrl->admin_tag_set.driver_data = ctrl;
- ctrl->admin_tag_set.nr_hw_queues = 1;
- ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
-
- ctrl->queues[0].ctrl = ctrl;
- error = nvmet_sq_init(&ctrl->queues[0].nvme_sq);
- if (error)
- return error;
- ctrl->ctrl.queue_count = 1;
-
- error = blk_mq_alloc_tag_set(&ctrl->admin_tag_set);
- if (error)
- goto out_free_sq;
- ctrl->ctrl.admin_tagset = &ctrl->admin_tag_set;
-
- ctrl->ctrl.admin_q = blk_mq_init_queue(&ctrl->admin_tag_set);
- if (IS_ERR(ctrl->ctrl.admin_q)) {
- error = PTR_ERR(ctrl->ctrl.admin_q);
- goto out_free_tagset;
- }
-
- error = nvmf_connect_admin_queue(&ctrl->ctrl);
- if (error)
- goto out_cleanup_queue;
-
- error = nvmf_reg_read64(&ctrl->ctrl, NVME_REG_CAP, &ctrl->ctrl.cap);
- if (error) {
- dev_err(ctrl->ctrl.device,
- "prop_get NVME_REG_CAP failed\n");
- goto out_cleanup_queue;
- }
-
- ctrl->ctrl.sqsize =
- min_t(int, NVME_CAP_MQES(ctrl->ctrl.cap), ctrl->ctrl.sqsize);
-
- error = nvme_enable_ctrl(&ctrl->ctrl, ctrl->ctrl.cap);
- if (error)
- goto out_cleanup_queue;
+static void nvme_loop_free_io_queues(struct nvme_ctrl *ctrl)
+{
+}
- ctrl->ctrl.max_hw_sectors =
- (NVME_LOOP_MAX_SEGMENTS - 1) << (PAGE_SHIFT - 9);
+static void nvme_loop_stop_queue(struct nvme_loop_ctrl *ctrl, int qid)
+{
+ nvmet_sq_destroy(&ctrl->queues[qid].nvme_sq);
+}
- error = nvme_init_identify(&ctrl->ctrl);
- if (error)
- goto out_cleanup_queue;
+static void nvme_loop_stop_admin_queue(struct nvme_ctrl *ctrl)
+{
+ nvme_loop_stop_queue(to_loop_ctrl(ctrl), 0);
+}
- return 0;
+static void nvme_loop_stop_io_queues(struct nvme_ctrl *ctrl)
+{
+ int i;
-out_cleanup_queue:
- blk_cleanup_queue(ctrl->ctrl.admin_q);
-out_free_tagset:
- blk_mq_free_tag_set(&ctrl->admin_tag_set);
-out_free_sq:
- nvmet_sq_destroy(&ctrl->queues[0].nvme_sq);
- return error;
+ for (i = 1; i < ctrl->queue_count; i++)
+ nvme_loop_stop_queue(to_loop_ctrl(ctrl), i);
}
-static void nvme_loop_shutdown_ctrl(struct nvme_loop_ctrl *ctrl)
+static int nvme_loop_start_queue(struct nvme_ctrl *ctrl, int qid)
{
- if (ctrl->ctrl.queue_count > 1) {
- nvme_stop_queues(&ctrl->ctrl);
- blk_mq_tagset_busy_iter(&ctrl->tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- nvme_loop_destroy_io_queues(ctrl);
- }
+ int ret;
- if (ctrl->ctrl.state == NVME_CTRL_LIVE)
- nvme_shutdown_ctrl(&ctrl->ctrl);
+ if (qid)
+ ret = nvmf_connect_io_queue(ctrl, qid);
+ else
+ ret = nvmf_connect_admin_queue(ctrl);
- blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
- nvme_loop_destroy_admin_queue(ctrl);
+ if (ret)
+ dev_info(ctrl->device,
+ "failed to connect queue: %d ret=%d\n", qid, ret);
+ return ret;
}
-static void nvme_loop_del_ctrl_work(struct work_struct *work)
+static int nvme_loop_start_admin_queue(struct nvme_ctrl *ctrl)
{
- struct nvme_loop_ctrl *ctrl = container_of(work,
- struct nvme_loop_ctrl, delete_work);
-
- nvme_stop_ctrl(&ctrl->ctrl);
- nvme_remove_namespaces(&ctrl->ctrl);
- nvme_loop_shutdown_ctrl(ctrl);
- nvme_uninit_ctrl(&ctrl->ctrl);
- nvme_put_ctrl(&ctrl->ctrl);
+ return nvme_loop_start_queue(ctrl, 0);
}
-static int __nvme_loop_del_ctrl(struct nvme_loop_ctrl *ctrl)
+static int nvme_loop_start_io_queues(struct nvme_ctrl *ctrl)
{
- if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING))
- return -EBUSY;
+ int i, ret = 0;
- if (!queue_work(nvme_wq, &ctrl->delete_work))
- return -EBUSY;
+ for (i = 1; i < ctrl->queue_count; i++) {
+ ret = nvme_loop_start_queue(ctrl, i);
+ if (ret)
+ goto out_stop_queues;
+ }
return 0;
+
+out_stop_queues:
+ for (i--; i >= 1; i--)
+ nvme_loop_stop_queue(to_loop_ctrl(ctrl), i);
+ return ret;
}
-static int nvme_loop_del_ctrl(struct nvme_ctrl *nctrl)
+static int nvme_loop_alloc_queue(struct nvme_loop_ctrl *ctrl,
+ int qid, size_t queue_size)
{
- struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
int ret;
- ret = __nvme_loop_del_ctrl(ctrl);
+ ctrl->queues[qid].ctrl = ctrl;
+ ret = nvmet_sq_init(&ctrl->queues[qid].nvme_sq);
if (ret)
return ret;
- flush_work(&ctrl->delete_work);
-
return 0;
}
-static void nvme_loop_delete_ctrl(struct nvmet_ctrl *nctrl)
+static int nvme_loop_alloc_admin_queue(struct nvme_ctrl *nctrl)
{
- struct nvme_loop_ctrl *ctrl;
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
- mutex_lock(&nvme_loop_ctrl_mutex);
- list_for_each_entry(ctrl, &nvme_loop_ctrl_list, list) {
- if (ctrl->ctrl.cntlid == nctrl->cntlid)
- __nvme_loop_del_ctrl(ctrl);
- }
- mutex_unlock(&nvme_loop_ctrl_mutex);
+ nvme_loop_init_iod(ctrl, &ctrl->async_event_iod, 0);
+
+ return nvme_loop_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
}
-static void nvme_loop_reset_ctrl_work(struct work_struct *work)
+static int nvme_loop_alloc_io_queues(struct nvme_ctrl *ctrl)
{
- struct nvme_loop_ctrl *ctrl =
- container_of(work, struct nvme_loop_ctrl, ctrl.reset_work);
- bool changed;
- int ret;
-
- nvme_stop_ctrl(&ctrl->ctrl);
- nvme_loop_shutdown_ctrl(ctrl);
-
- ret = nvme_loop_configure_admin_queue(ctrl);
- if (ret)
- goto out_disable;
-
- ret = nvme_loop_init_io_queues(ctrl);
- if (ret)
- goto out_destroy_admin;
+ int i, ret;
- ret = nvme_loop_connect_io_queues(ctrl);
- if (ret)
- goto out_destroy_io;
+ for (i = 1; i < ctrl->queue_count; i++) {
+ ret = nvme_loop_alloc_queue(to_loop_ctrl(ctrl),
+ i, ctrl->sqsize + 1);
+ if (ret)
+ return ret;
+ }
- blk_mq_update_nr_hw_queues(&ctrl->tag_set,
- ctrl->ctrl.queue_count - 1);
+ return 0;
+}
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
+static void nvme_loop_free_ctrl(struct nvme_ctrl *nctrl)
+{
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
- nvme_start_ctrl(&ctrl->ctrl);
+ if (list_empty(&ctrl->list))
+ goto free_ctrl;
- return;
+ mutex_lock(&nvme_loop_ctrl_mutex);
+ list_del(&ctrl->list);
+ mutex_unlock(&nvme_loop_ctrl_mutex);
-out_destroy_io:
- nvme_loop_destroy_io_queues(ctrl);
-out_destroy_admin:
- nvme_loop_destroy_admin_queue(ctrl);
-out_disable:
- dev_warn(ctrl->ctrl.device, "Removing after reset failure\n");
- nvme_uninit_ctrl(&ctrl->ctrl);
- nvme_put_ctrl(&ctrl->ctrl);
+ kfree(ctrl->queues);
+ nvmf_free_options(nctrl->opts);
+free_ctrl:
+ kfree(ctrl);
}
static const struct nvme_ctrl_ops nvme_loop_ctrl_ops = {
@@ -538,135 +469,81 @@ static const struct nvme_ctrl_ops nvme_loop_ctrl_ops = {
.reg_write32 = nvmf_reg_write32,
.free_ctrl = nvme_loop_free_ctrl,
.submit_async_event = nvme_loop_submit_async_event,
- .delete_ctrl = nvme_loop_del_ctrl,
+ .delete_ctrl = nvme_del_ctrl,
+ .alloc_admin_queue = nvme_loop_alloc_admin_queue,
+ .free_admin_queue = nvme_loop_free_admin_queue,
+ .start_admin_queue = nvme_loop_start_admin_queue,
+ .stop_admin_queue = nvme_loop_stop_admin_queue,
+ .alloc_io_queues = nvme_loop_alloc_io_queues,
+ .free_io_queues = nvme_loop_free_io_queues,
+ .start_io_queues = nvme_loop_start_io_queues,
+ .stop_io_queues = nvme_loop_stop_io_queues,
+ .alloc_tagset = nvme_loop_alloc_tagset,
+ .free_tagset = nvme_loop_free_tagset,
+ .post_configure = nvme_loop_post_configure,
+ .nr_hw_queues = nvme_loop_nr_io_queues,
};
-static int nvme_loop_create_io_queues(struct nvme_loop_ctrl *ctrl)
-{
- int ret;
-
- ret = nvme_loop_init_io_queues(ctrl);
- if (ret)
- return ret;
-
- memset(&ctrl->tag_set, 0, sizeof(ctrl->tag_set));
- ctrl->tag_set.ops = &nvme_loop_mq_ops;
- ctrl->tag_set.queue_depth = ctrl->ctrl.opts->queue_size;
- ctrl->tag_set.reserved_tags = 1; /* fabric connect */
- ctrl->tag_set.numa_node = NUMA_NO_NODE;
- ctrl->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
- ctrl->tag_set.cmd_size = sizeof(struct nvme_loop_iod) +
- SG_CHUNK_SIZE * sizeof(struct scatterlist);
- ctrl->tag_set.driver_data = ctrl;
- ctrl->tag_set.nr_hw_queues = ctrl->ctrl.queue_count - 1;
- ctrl->tag_set.timeout = NVME_IO_TIMEOUT;
- ctrl->ctrl.tagset = &ctrl->tag_set;
-
- ret = blk_mq_alloc_tag_set(&ctrl->tag_set);
- if (ret)
- goto out_destroy_queues;
-
- ctrl->ctrl.connect_q = blk_mq_init_queue(&ctrl->tag_set);
- if (IS_ERR(ctrl->ctrl.connect_q)) {
- ret = PTR_ERR(ctrl->ctrl.connect_q);
- goto out_free_tagset;
- }
-
- ret = nvme_loop_connect_io_queues(ctrl);
- if (ret)
- goto out_cleanup_connect_q;
-
- return 0;
-
-out_cleanup_connect_q:
- blk_cleanup_queue(ctrl->ctrl.connect_q);
-out_free_tagset:
- blk_mq_free_tag_set(&ctrl->tag_set);
-out_destroy_queues:
- nvme_loop_destroy_io_queues(ctrl);
- return ret;
-}
-
static struct nvme_ctrl *nvme_loop_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_loop_ctrl *ctrl;
- bool changed;
int ret;
ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
if (!ctrl)
return ERR_PTR(-ENOMEM);
- ctrl->ctrl.opts = opts;
- INIT_LIST_HEAD(&ctrl->list);
-
- INIT_WORK(&ctrl->delete_work, nvme_loop_del_ctrl_work);
- INIT_WORK(&ctrl->ctrl.reset_work, nvme_loop_reset_ctrl_work);
-
- ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_loop_ctrl_ops,
- 0 /* no quirks, we're perfect! */);
- if (ret)
- goto out_put_ctrl;
-
- ret = -ENOMEM;
+ ctrl->ctrl.opts = opts;
+ ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
ctrl->ctrl.sqsize = opts->queue_size - 1;
ctrl->ctrl.kato = opts->kato;
+ INIT_LIST_HEAD(&ctrl->list);
ctrl->queues = kcalloc(opts->nr_io_queues + 1, sizeof(*ctrl->queues),
GFP_KERNEL);
- if (!ctrl->queues)
- goto out_uninit_ctrl;
+ if (!ctrl->queues) {
+ ret = -ENOMEM;
+ goto out_free_ctrl;
+ }
- ret = nvme_loop_configure_admin_queue(ctrl);
+ ret = nvme_probe_ctrl(&ctrl->ctrl, dev, &nvme_loop_ctrl_ops, 0);
if (ret)
goto out_free_queues;
- if (opts->queue_size > ctrl->ctrl.maxcmd) {
- /* warn if maxcmd is lower than queue_size */
- dev_warn(ctrl->ctrl.device,
- "queue_size %zu > ctrl maxcmd %u, clamping down\n",
- opts->queue_size, ctrl->ctrl.maxcmd);
- opts->queue_size = ctrl->ctrl.maxcmd;
- }
-
- if (opts->nr_io_queues) {
- ret = nvme_loop_create_io_queues(ctrl);
- if (ret)
- goto out_remove_admin_queue;
- }
-
- nvme_loop_init_iod(ctrl, &ctrl->async_event_iod, 0);
-
dev_info(ctrl->ctrl.device,
"new ctrl: \"%s\"\n", ctrl->ctrl.opts->subsysnqn);
- kref_get(&ctrl->ctrl.kref);
-
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
-
mutex_lock(&nvme_loop_ctrl_mutex);
list_add_tail(&ctrl->list, &nvme_loop_ctrl_list);
mutex_unlock(&nvme_loop_ctrl_mutex);
- nvme_start_ctrl(&ctrl->ctrl);
+ kref_get(&ctrl->ctrl.kref);
return &ctrl->ctrl;
-out_remove_admin_queue:
- nvme_loop_destroy_admin_queue(ctrl);
out_free_queues:
kfree(ctrl->queues);
-out_uninit_ctrl:
- nvme_uninit_ctrl(&ctrl->ctrl);
-out_put_ctrl:
+out_free_ctrl:
nvme_put_ctrl(&ctrl->ctrl);
if (ret > 0)
ret = -EIO;
+ kfree(ctrl);
return ERR_PTR(ret);
}
+static void nvme_loop_delete_ctrl(struct nvmet_ctrl *nctrl)
+{
+ struct nvme_loop_ctrl *ctrl;
+
+ mutex_lock(&nvme_loop_ctrl_mutex);
+ list_for_each_entry(ctrl, &nvme_loop_ctrl_list, list) {
+ if (ctrl->ctrl.cntlid == nctrl->cntlid)
+ __nvme_del_ctrl(&ctrl->ctrl);
+ }
+ mutex_unlock(&nvme_loop_ctrl_mutex);
+}
+
static int nvme_loop_add_port(struct nvmet_port *port)
{
/*
@@ -730,7 +607,7 @@ static void __exit nvme_loop_cleanup_module(void)
mutex_lock(&nvme_loop_ctrl_mutex);
list_for_each_entry_safe(ctrl, next, &nvme_loop_ctrl_list, list)
- __nvme_loop_del_ctrl(ctrl);
+ __nvme_del_ctrl(&ctrl->ctrl);
mutex_unlock(&nvme_loop_ctrl_mutex);
flush_workqueue(nvme_wq);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* [PATCH 12/12] nvme-loop: convert to nvme-core control plane management
@ 2017-08-15 9:52 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-15 9:52 UTC (permalink / raw)
rip out all the controller and queues control plane code,
only maintain queue alloc/free/start/stop and tagset alloc/free.
Signed-off-by: Sagi Grimberg <sagi at grimberg.me>
---
This patch failed to generate a nice diff :(
drivers/nvme/target/loop.c | 443 ++++++++++++++++-----------------------------
1 file changed, 160 insertions(+), 283 deletions(-)
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index 92628c432926..feba730dd9f1 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -273,260 +273,191 @@ static const struct blk_mq_ops nvme_loop_admin_mq_ops = {
.timeout = nvme_loop_timeout,
};
-static void nvme_loop_destroy_admin_queue(struct nvme_loop_ctrl *ctrl)
+static unsigned int nvme_loop_nr_io_queues(struct nvme_ctrl *ctrl)
{
- nvmet_sq_destroy(&ctrl->queues[0].nvme_sq);
- blk_cleanup_queue(ctrl->ctrl.admin_q);
- blk_mq_free_tag_set(&ctrl->admin_tag_set);
+ return num_online_cpus();
}
-static void nvme_loop_free_ctrl(struct nvme_ctrl *nctrl)
+static int nvme_loop_post_configure(struct nvme_ctrl *ctrl)
{
- struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
-
- if (list_empty(&ctrl->list))
- goto free_ctrl;
+ struct nvmf_ctrl_options *opts = ctrl->opts;
- mutex_lock(&nvme_loop_ctrl_mutex);
- list_del(&ctrl->list);
- mutex_unlock(&nvme_loop_ctrl_mutex);
-
- if (nctrl->tagset) {
- blk_cleanup_queue(ctrl->ctrl.connect_q);
- blk_mq_free_tag_set(&ctrl->tag_set);
+ if (opts->queue_size > ctrl->maxcmd) {
+ /* warn if maxcmd is lower than queue_size */
+ dev_warn(ctrl->device,
+ "queue_size %zu > ctrl maxcmd %u, clamping down\n",
+ opts->queue_size, ctrl->maxcmd);
+ opts->queue_size = ctrl->maxcmd;
}
- kfree(ctrl->queues);
- nvmf_free_options(nctrl->opts);
-free_ctrl:
- kfree(ctrl);
+
+ return 0;
}
-static void nvme_loop_destroy_io_queues(struct nvme_loop_ctrl *ctrl)
+static void nvme_loop_free_tagset(struct nvme_ctrl *nctrl, bool admin)
{
- int i;
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
+ struct blk_mq_tag_set *set = admin ?
+ &ctrl->admin_tag_set : &ctrl->tag_set;
- for (i = 1; i < ctrl->ctrl.queue_count; i++)
- nvmet_sq_destroy(&ctrl->queues[i].nvme_sq);
+ blk_mq_free_tag_set(set);
}
-static int nvme_loop_init_io_queues(struct nvme_loop_ctrl *ctrl)
+static struct blk_mq_tag_set *nvme_loop_alloc_tagset(struct nvme_ctrl *nctrl,
+ bool admin)
{
- struct nvmf_ctrl_options *opts = ctrl->ctrl.opts;
- unsigned int nr_io_queues;
- int ret, i;
-
- nr_io_queues = min(opts->nr_io_queues, num_online_cpus());
- ret = nvme_set_queue_count(&ctrl->ctrl, &nr_io_queues);
- if (ret || !nr_io_queues)
- return ret;
-
- dev_info(ctrl->ctrl.device, "creating %d I/O queues.\n", nr_io_queues);
-
- for (i = 1; i <= nr_io_queues; i++) {
- ctrl->queues[i].ctrl = ctrl;
- ret = nvmet_sq_init(&ctrl->queues[i].nvme_sq);
- if (ret)
- goto out_destroy_queues;
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
+ struct blk_mq_tag_set *set;
+ int ret;
- ctrl->ctrl.queue_count++;
+ if (admin) {
+ set = &ctrl->admin_tag_set;
+ memset(set, 0, sizeof(*set));
+ set->ops = &nvme_loop_admin_mq_ops;
+ set->queue_depth = NVME_LOOP_AQ_BLKMQ_DEPTH;
+ set->reserved_tags = 2; /* connect + keep-alive */
+ set->numa_node = NUMA_NO_NODE;
+ set->cmd_size = sizeof(struct nvme_loop_iod) +
+ SG_CHUNK_SIZE * sizeof(struct scatterlist);
+ set->driver_data = ctrl;
+ set->nr_hw_queues = 1;
+ set->timeout = ADMIN_TIMEOUT;
+ } else {
+ set = &ctrl->tag_set;
+ memset(set, 0, sizeof(*set));
+ set->ops = &nvme_loop_mq_ops;
+ set->queue_depth = nctrl->opts->queue_size;
+ set->reserved_tags = 1; /* fabric connect */
+ set->numa_node = NUMA_NO_NODE;
+ set->flags = BLK_MQ_F_SHOULD_MERGE;
+ set->cmd_size = sizeof(struct nvme_loop_iod) +
+ SG_CHUNK_SIZE * sizeof(struct scatterlist);
+ set->driver_data = ctrl;
+ set->nr_hw_queues = nctrl->queue_count - 1;
+ set->timeout = NVME_IO_TIMEOUT;
}
- return 0;
+ ret = blk_mq_alloc_tag_set(set);
+ if (ret)
+ return ERR_PTR(ret);
-out_destroy_queues:
- nvme_loop_destroy_io_queues(ctrl);
- return ret;
+ return set;
}
-static int nvme_loop_connect_io_queues(struct nvme_loop_ctrl *ctrl)
+static void nvme_loop_free_admin_queue(struct nvme_ctrl *ctrl)
{
- int i, ret;
-
- for (i = 1; i < ctrl->ctrl.queue_count; i++) {
- ret = nvmf_connect_io_queue(&ctrl->ctrl, i);
- if (ret)
- return ret;
- }
-
- return 0;
}
-static int nvme_loop_configure_admin_queue(struct nvme_loop_ctrl *ctrl)
-{
- int error;
-
- memset(&ctrl->admin_tag_set, 0, sizeof(ctrl->admin_tag_set));
- ctrl->admin_tag_set.ops = &nvme_loop_admin_mq_ops;
- ctrl->admin_tag_set.queue_depth = NVME_LOOP_AQ_BLKMQ_DEPTH;
- ctrl->admin_tag_set.reserved_tags = 2; /* connect + keep-alive */
- ctrl->admin_tag_set.numa_node = NUMA_NO_NODE;
- ctrl->admin_tag_set.cmd_size = sizeof(struct nvme_loop_iod) +
- SG_CHUNK_SIZE * sizeof(struct scatterlist);
- ctrl->admin_tag_set.driver_data = ctrl;
- ctrl->admin_tag_set.nr_hw_queues = 1;
- ctrl->admin_tag_set.timeout = ADMIN_TIMEOUT;
-
- ctrl->queues[0].ctrl = ctrl;
- error = nvmet_sq_init(&ctrl->queues[0].nvme_sq);
- if (error)
- return error;
- ctrl->ctrl.queue_count = 1;
-
- error = blk_mq_alloc_tag_set(&ctrl->admin_tag_set);
- if (error)
- goto out_free_sq;
- ctrl->ctrl.admin_tagset = &ctrl->admin_tag_set;
-
- ctrl->ctrl.admin_q = blk_mq_init_queue(&ctrl->admin_tag_set);
- if (IS_ERR(ctrl->ctrl.admin_q)) {
- error = PTR_ERR(ctrl->ctrl.admin_q);
- goto out_free_tagset;
- }
-
- error = nvmf_connect_admin_queue(&ctrl->ctrl);
- if (error)
- goto out_cleanup_queue;
-
- error = nvmf_reg_read64(&ctrl->ctrl, NVME_REG_CAP, &ctrl->ctrl.cap);
- if (error) {
- dev_err(ctrl->ctrl.device,
- "prop_get NVME_REG_CAP failed\n");
- goto out_cleanup_queue;
- }
-
- ctrl->ctrl.sqsize =
- min_t(int, NVME_CAP_MQES(ctrl->ctrl.cap), ctrl->ctrl.sqsize);
-
- error = nvme_enable_ctrl(&ctrl->ctrl, ctrl->ctrl.cap);
- if (error)
- goto out_cleanup_queue;
+static void nvme_loop_free_io_queues(struct nvme_ctrl *ctrl)
+{
+}
- ctrl->ctrl.max_hw_sectors =
- (NVME_LOOP_MAX_SEGMENTS - 1) << (PAGE_SHIFT - 9);
+static void nvme_loop_stop_queue(struct nvme_loop_ctrl *ctrl, int qid)
+{
+ nvmet_sq_destroy(&ctrl->queues[qid].nvme_sq);
+}
- error = nvme_init_identify(&ctrl->ctrl);
- if (error)
- goto out_cleanup_queue;
+static void nvme_loop_stop_admin_queue(struct nvme_ctrl *ctrl)
+{
+ nvme_loop_stop_queue(to_loop_ctrl(ctrl), 0);
+}
- return 0;
+static void nvme_loop_stop_io_queues(struct nvme_ctrl *ctrl)
+{
+ int i;
-out_cleanup_queue:
- blk_cleanup_queue(ctrl->ctrl.admin_q);
-out_free_tagset:
- blk_mq_free_tag_set(&ctrl->admin_tag_set);
-out_free_sq:
- nvmet_sq_destroy(&ctrl->queues[0].nvme_sq);
- return error;
+ for (i = 1; i < ctrl->queue_count; i++)
+ nvme_loop_stop_queue(to_loop_ctrl(ctrl), i);
}
-static void nvme_loop_shutdown_ctrl(struct nvme_loop_ctrl *ctrl)
+static int nvme_loop_start_queue(struct nvme_ctrl *ctrl, int qid)
{
- if (ctrl->ctrl.queue_count > 1) {
- nvme_stop_queues(&ctrl->ctrl);
- blk_mq_tagset_busy_iter(&ctrl->tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- nvme_loop_destroy_io_queues(ctrl);
- }
+ int ret;
- if (ctrl->ctrl.state == NVME_CTRL_LIVE)
- nvme_shutdown_ctrl(&ctrl->ctrl);
+ if (qid)
+ ret = nvmf_connect_io_queue(ctrl, qid);
+ else
+ ret = nvmf_connect_admin_queue(ctrl);
- blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
- blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
- nvme_cancel_request, &ctrl->ctrl);
- blk_mq_unquiesce_queue(ctrl->ctrl.admin_q);
- nvme_loop_destroy_admin_queue(ctrl);
+ if (ret)
+ dev_info(ctrl->device,
+ "failed to connect queue: %d ret=%d\n", qid, ret);
+ return ret;
}
-static void nvme_loop_del_ctrl_work(struct work_struct *work)
+static int nvme_loop_start_admin_queue(struct nvme_ctrl *ctrl)
{
- struct nvme_loop_ctrl *ctrl = container_of(work,
- struct nvme_loop_ctrl, delete_work);
-
- nvme_stop_ctrl(&ctrl->ctrl);
- nvme_remove_namespaces(&ctrl->ctrl);
- nvme_loop_shutdown_ctrl(ctrl);
- nvme_uninit_ctrl(&ctrl->ctrl);
- nvme_put_ctrl(&ctrl->ctrl);
+ return nvme_loop_start_queue(ctrl, 0);
}
-static int __nvme_loop_del_ctrl(struct nvme_loop_ctrl *ctrl)
+static int nvme_loop_start_io_queues(struct nvme_ctrl *ctrl)
{
- if (!nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_DELETING))
- return -EBUSY;
+ int i, ret = 0;
- if (!queue_work(nvme_wq, &ctrl->delete_work))
- return -EBUSY;
+ for (i = 1; i < ctrl->queue_count; i++) {
+ ret = nvme_loop_start_queue(ctrl, i);
+ if (ret)
+ goto out_stop_queues;
+ }
return 0;
+
+out_stop_queues:
+ for (i--; i >= 1; i--)
+ nvme_loop_stop_queue(to_loop_ctrl(ctrl), i);
+ return ret;
}
-static int nvme_loop_del_ctrl(struct nvme_ctrl *nctrl)
+static int nvme_loop_alloc_queue(struct nvme_loop_ctrl *ctrl,
+ int qid, size_t queue_size)
{
- struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
int ret;
- ret = __nvme_loop_del_ctrl(ctrl);
+ ctrl->queues[qid].ctrl = ctrl;
+ ret = nvmet_sq_init(&ctrl->queues[qid].nvme_sq);
if (ret)
return ret;
- flush_work(&ctrl->delete_work);
-
return 0;
}
-static void nvme_loop_delete_ctrl(struct nvmet_ctrl *nctrl)
+static int nvme_loop_alloc_admin_queue(struct nvme_ctrl *nctrl)
{
- struct nvme_loop_ctrl *ctrl;
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
- mutex_lock(&nvme_loop_ctrl_mutex);
- list_for_each_entry(ctrl, &nvme_loop_ctrl_list, list) {
- if (ctrl->ctrl.cntlid == nctrl->cntlid)
- __nvme_loop_del_ctrl(ctrl);
- }
- mutex_unlock(&nvme_loop_ctrl_mutex);
+ nvme_loop_init_iod(ctrl, &ctrl->async_event_iod, 0);
+
+ return nvme_loop_alloc_queue(ctrl, 0, NVME_AQ_DEPTH);
}
-static void nvme_loop_reset_ctrl_work(struct work_struct *work)
+static int nvme_loop_alloc_io_queues(struct nvme_ctrl *ctrl)
{
- struct nvme_loop_ctrl *ctrl =
- container_of(work, struct nvme_loop_ctrl, ctrl.reset_work);
- bool changed;
- int ret;
-
- nvme_stop_ctrl(&ctrl->ctrl);
- nvme_loop_shutdown_ctrl(ctrl);
-
- ret = nvme_loop_configure_admin_queue(ctrl);
- if (ret)
- goto out_disable;
-
- ret = nvme_loop_init_io_queues(ctrl);
- if (ret)
- goto out_destroy_admin;
+ int i, ret;
- ret = nvme_loop_connect_io_queues(ctrl);
- if (ret)
- goto out_destroy_io;
+ for (i = 1; i < ctrl->queue_count; i++) {
+ ret = nvme_loop_alloc_queue(to_loop_ctrl(ctrl),
+ i, ctrl->sqsize + 1);
+ if (ret)
+ return ret;
+ }
- blk_mq_update_nr_hw_queues(&ctrl->tag_set,
- ctrl->ctrl.queue_count - 1);
+ return 0;
+}
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
+static void nvme_loop_free_ctrl(struct nvme_ctrl *nctrl)
+{
+ struct nvme_loop_ctrl *ctrl = to_loop_ctrl(nctrl);
- nvme_start_ctrl(&ctrl->ctrl);
+ if (list_empty(&ctrl->list))
+ goto free_ctrl;
- return;
+ mutex_lock(&nvme_loop_ctrl_mutex);
+ list_del(&ctrl->list);
+ mutex_unlock(&nvme_loop_ctrl_mutex);
-out_destroy_io:
- nvme_loop_destroy_io_queues(ctrl);
-out_destroy_admin:
- nvme_loop_destroy_admin_queue(ctrl);
-out_disable:
- dev_warn(ctrl->ctrl.device, "Removing after reset failure\n");
- nvme_uninit_ctrl(&ctrl->ctrl);
- nvme_put_ctrl(&ctrl->ctrl);
+ kfree(ctrl->queues);
+ nvmf_free_options(nctrl->opts);
+free_ctrl:
+ kfree(ctrl);
}
static const struct nvme_ctrl_ops nvme_loop_ctrl_ops = {
@@ -538,135 +469,81 @@ static const struct nvme_ctrl_ops nvme_loop_ctrl_ops = {
.reg_write32 = nvmf_reg_write32,
.free_ctrl = nvme_loop_free_ctrl,
.submit_async_event = nvme_loop_submit_async_event,
- .delete_ctrl = nvme_loop_del_ctrl,
+ .delete_ctrl = nvme_del_ctrl,
+ .alloc_admin_queue = nvme_loop_alloc_admin_queue,
+ .free_admin_queue = nvme_loop_free_admin_queue,
+ .start_admin_queue = nvme_loop_start_admin_queue,
+ .stop_admin_queue = nvme_loop_stop_admin_queue,
+ .alloc_io_queues = nvme_loop_alloc_io_queues,
+ .free_io_queues = nvme_loop_free_io_queues,
+ .start_io_queues = nvme_loop_start_io_queues,
+ .stop_io_queues = nvme_loop_stop_io_queues,
+ .alloc_tagset = nvme_loop_alloc_tagset,
+ .free_tagset = nvme_loop_free_tagset,
+ .post_configure = nvme_loop_post_configure,
+ .nr_hw_queues = nvme_loop_nr_io_queues,
};
-static int nvme_loop_create_io_queues(struct nvme_loop_ctrl *ctrl)
-{
- int ret;
-
- ret = nvme_loop_init_io_queues(ctrl);
- if (ret)
- return ret;
-
- memset(&ctrl->tag_set, 0, sizeof(ctrl->tag_set));
- ctrl->tag_set.ops = &nvme_loop_mq_ops;
- ctrl->tag_set.queue_depth = ctrl->ctrl.opts->queue_size;
- ctrl->tag_set.reserved_tags = 1; /* fabric connect */
- ctrl->tag_set.numa_node = NUMA_NO_NODE;
- ctrl->tag_set.flags = BLK_MQ_F_SHOULD_MERGE;
- ctrl->tag_set.cmd_size = sizeof(struct nvme_loop_iod) +
- SG_CHUNK_SIZE * sizeof(struct scatterlist);
- ctrl->tag_set.driver_data = ctrl;
- ctrl->tag_set.nr_hw_queues = ctrl->ctrl.queue_count - 1;
- ctrl->tag_set.timeout = NVME_IO_TIMEOUT;
- ctrl->ctrl.tagset = &ctrl->tag_set;
-
- ret = blk_mq_alloc_tag_set(&ctrl->tag_set);
- if (ret)
- goto out_destroy_queues;
-
- ctrl->ctrl.connect_q = blk_mq_init_queue(&ctrl->tag_set);
- if (IS_ERR(ctrl->ctrl.connect_q)) {
- ret = PTR_ERR(ctrl->ctrl.connect_q);
- goto out_free_tagset;
- }
-
- ret = nvme_loop_connect_io_queues(ctrl);
- if (ret)
- goto out_cleanup_connect_q;
-
- return 0;
-
-out_cleanup_connect_q:
- blk_cleanup_queue(ctrl->ctrl.connect_q);
-out_free_tagset:
- blk_mq_free_tag_set(&ctrl->tag_set);
-out_destroy_queues:
- nvme_loop_destroy_io_queues(ctrl);
- return ret;
-}
-
static struct nvme_ctrl *nvme_loop_create_ctrl(struct device *dev,
struct nvmf_ctrl_options *opts)
{
struct nvme_loop_ctrl *ctrl;
- bool changed;
int ret;
ctrl = kzalloc(sizeof(*ctrl), GFP_KERNEL);
if (!ctrl)
return ERR_PTR(-ENOMEM);
- ctrl->ctrl.opts = opts;
- INIT_LIST_HEAD(&ctrl->list);
-
- INIT_WORK(&ctrl->delete_work, nvme_loop_del_ctrl_work);
- INIT_WORK(&ctrl->ctrl.reset_work, nvme_loop_reset_ctrl_work);
-
- ret = nvme_init_ctrl(&ctrl->ctrl, dev, &nvme_loop_ctrl_ops,
- 0 /* no quirks, we're perfect! */);
- if (ret)
- goto out_put_ctrl;
-
- ret = -ENOMEM;
+ ctrl->ctrl.opts = opts;
+ ctrl->ctrl.queue_count = opts->nr_io_queues + 1; /* +1 for admin queue */
ctrl->ctrl.sqsize = opts->queue_size - 1;
ctrl->ctrl.kato = opts->kato;
+ INIT_LIST_HEAD(&ctrl->list);
ctrl->queues = kcalloc(opts->nr_io_queues + 1, sizeof(*ctrl->queues),
GFP_KERNEL);
- if (!ctrl->queues)
- goto out_uninit_ctrl;
+ if (!ctrl->queues) {
+ ret = -ENOMEM;
+ goto out_free_ctrl;
+ }
- ret = nvme_loop_configure_admin_queue(ctrl);
+ ret = nvme_probe_ctrl(&ctrl->ctrl, dev, &nvme_loop_ctrl_ops, 0);
if (ret)
goto out_free_queues;
- if (opts->queue_size > ctrl->ctrl.maxcmd) {
- /* warn if maxcmd is lower than queue_size */
- dev_warn(ctrl->ctrl.device,
- "queue_size %zu > ctrl maxcmd %u, clamping down\n",
- opts->queue_size, ctrl->ctrl.maxcmd);
- opts->queue_size = ctrl->ctrl.maxcmd;
- }
-
- if (opts->nr_io_queues) {
- ret = nvme_loop_create_io_queues(ctrl);
- if (ret)
- goto out_remove_admin_queue;
- }
-
- nvme_loop_init_iod(ctrl, &ctrl->async_event_iod, 0);
-
dev_info(ctrl->ctrl.device,
"new ctrl: \"%s\"\n", ctrl->ctrl.opts->subsysnqn);
- kref_get(&ctrl->ctrl.kref);
-
- changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
- WARN_ON_ONCE(!changed);
-
mutex_lock(&nvme_loop_ctrl_mutex);
list_add_tail(&ctrl->list, &nvme_loop_ctrl_list);
mutex_unlock(&nvme_loop_ctrl_mutex);
- nvme_start_ctrl(&ctrl->ctrl);
+ kref_get(&ctrl->ctrl.kref);
return &ctrl->ctrl;
-out_remove_admin_queue:
- nvme_loop_destroy_admin_queue(ctrl);
out_free_queues:
kfree(ctrl->queues);
-out_uninit_ctrl:
- nvme_uninit_ctrl(&ctrl->ctrl);
-out_put_ctrl:
+out_free_ctrl:
nvme_put_ctrl(&ctrl->ctrl);
if (ret > 0)
ret = -EIO;
+ kfree(ctrl);
return ERR_PTR(ret);
}
+static void nvme_loop_delete_ctrl(struct nvmet_ctrl *nctrl)
+{
+ struct nvme_loop_ctrl *ctrl;
+
+ mutex_lock(&nvme_loop_ctrl_mutex);
+ list_for_each_entry(ctrl, &nvme_loop_ctrl_list, list) {
+ if (ctrl->ctrl.cntlid == nctrl->cntlid)
+ __nvme_del_ctrl(&ctrl->ctrl);
+ }
+ mutex_unlock(&nvme_loop_ctrl_mutex);
+}
+
static int nvme_loop_add_port(struct nvmet_port *port)
{
/*
@@ -730,7 +607,7 @@ static void __exit nvme_loop_cleanup_module(void)
mutex_lock(&nvme_loop_ctrl_mutex);
list_for_each_entry_safe(ctrl, next, &nvme_loop_ctrl_list, list)
- __nvme_loop_del_ctrl(ctrl);
+ __nvme_del_ctrl(&ctrl->ctrl);
mutex_unlock(&nvme_loop_ctrl_mutex);
flush_workqueue(nvme_wq);
--
2.7.4
^ permalink raw reply related [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-16 8:16 ` Christoph Hellwig
-1 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 8:16 UTC (permalink / raw)
To: Sagi Grimberg; +Cc: linux-nvme, Christoph Hellwig, Keith Busch, linux-block
> This set applies on nvme-4.14 plus 4.13 recent fixes.
I can't get this applied at all. A pointer to a git repo would
always be nice for patchsets as large as this one, especially if
the base is a bit murky..
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 8:16 ` Christoph Hellwig
0 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 8:16 UTC (permalink / raw)
> This set applies on nvme-4.14 plus 4.13 recent fixes.
I can't get this applied at all. A pointer to a git repo would
always be nice for patchsets as large as this one, especially if
the base is a bit murky..
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 8:16 ` Christoph Hellwig
@ 2017-08-16 9:33 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 9:33 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Keith Busch, linux-block, Christoph Hellwig, linux-nvme
>> This set applies on nvme-4.14 plus 4.13 recent fixes.
>
> I can't get this applied at all.
That's probably because this is on top of the rdma affinity
patches which are already in Doug's tree...
> A pointer to a git repo would
> always be nice for patchsets as large as this one, especially if
> the base is a bit murky..
Sure, should I have it on top of the affinity patches or not?
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 9:33 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 9:33 UTC (permalink / raw)
>> This set applies on nvme-4.14 plus 4.13 recent fixes.
>
> I can't get this applied at all.
That's probably because this is on top of the rdma affinity
patches which are already in Doug's tree...
> A pointer to a git repo would
> always be nice for patchsets as large as this one, especially if
> the base is a bit murky..
Sure, should I have it on top of the affinity patches or not?
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 9:33 ` Sagi Grimberg
@ 2017-08-16 9:35 ` Christoph Hellwig
-1 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 9:35 UTC (permalink / raw)
To: Sagi Grimberg
Cc: Christoph Hellwig, Keith Busch, linux-block, Christoph Hellwig,
linux-nvme
On Wed, Aug 16, 2017 at 12:33:47PM +0300, Sagi Grimberg wrote:
>>> This set applies on nvme-4.14 plus 4.13 recent fixes.
>>
>> I can't get this applied at all.
>
> That's probably because this is on top of the rdma affinity
> patches which are already in Doug's tree...
>
>> A pointer to a git repo would
>> always be nice for patchsets as large as this one, especially if
>> the base is a bit murky..
>
> Sure, should I have it on top of the affinity patches or not?
For now get any tree you have out to take a look. In the longer
run we'll need to decide how we want to deal with the merge
of the two trees. In general having to depend on the RDMA
tree is a major risk because it's maintainance is so unreliable.
If it's in a branch of its own we could pull it in through and
send that bit through both trees.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 9:35 ` Christoph Hellwig
0 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 9:35 UTC (permalink / raw)
On Wed, Aug 16, 2017@12:33:47PM +0300, Sagi Grimberg wrote:
>>> This set applies on nvme-4.14 plus 4.13 recent fixes.
>>
>> I can't get this applied at all.
>
> That's probably because this is on top of the rdma affinity
> patches which are already in Doug's tree...
>
>> A pointer to a git repo would
>> always be nice for patchsets as large as this one, especially if
>> the base is a bit murky..
>
> Sure, should I have it on top of the affinity patches or not?
For now get any tree you have out to take a look. In the longer
run we'll need to decide how we want to deal with the merge
of the two trees. In general having to depend on the RDMA
tree is a major risk because it's maintainance is so unreliable.
If it's in a branch of its own we could pull it in through and
send that bit through both trees.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 9:35 ` Christoph Hellwig
@ 2017-08-16 9:46 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 9:46 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Christoph Hellwig, Keith Busch, linux-block, linux-nvme
>> Sure, should I have it on top of the affinity patches or not?
>
> For now get any tree you have out to take a look. In the longer
> run we'll need to decide how we want to deal with the merge
> of the two trees. In general having to depend on the RDMA
> tree is a major risk because it's maintainance is so unreliable.
Its really a small conflict, easy enough to get it without. The problem
that no matter what I do, one tree would conflict...
> If it's in a branch of its own we could pull it in through and
> send that bit through both trees.
We can do that, should I use the nvme tree for that? or maybe once we
converge on the patch set?
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 9:46 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 9:46 UTC (permalink / raw)
>> Sure, should I have it on top of the affinity patches or not?
>
> For now get any tree you have out to take a look. In the longer
> run we'll need to decide how we want to deal with the merge
> of the two trees. In general having to depend on the RDMA
> tree is a major risk because it's maintainance is so unreliable.
Its really a small conflict, easy enough to get it without. The problem
that no matter what I do, one tree would conflict...
> If it's in a branch of its own we could pull it in through and
> send that bit through both trees.
We can do that, should I use the nvme tree for that? or maybe once we
converge on the patch set?
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 9:46 ` Sagi Grimberg
@ 2017-08-16 9:57 ` Christoph Hellwig
-1 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 9:57 UTC (permalink / raw)
To: Sagi Grimberg
Cc: Christoph Hellwig, Christoph Hellwig, Keith Busch, linux-block,
linux-nvme
On Wed, Aug 16, 2017 at 12:46:25PM +0300, Sagi Grimberg wrote:
>>> Sure, should I have it on top of the affinity patches or not?
>>
>> For now get any tree you have out to take a look. In the longer
>> run we'll need to decide how we want to deal with the merge
>> of the two trees. In general having to depend on the RDMA
>> tree is a major risk because it's maintainance is so unreliable.
>
> Its really a small conflict, easy enough to get it without. The problem
> that no matter what I do, one tree would conflict...
Let's keep the refactoring self contained in nvme now and sort out
the small conflict when the rdma tree is merge (which is alsmost
certainly going to be after the block tree)
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 9:57 ` Christoph Hellwig
0 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 9:57 UTC (permalink / raw)
On Wed, Aug 16, 2017@12:46:25PM +0300, Sagi Grimberg wrote:
>>> Sure, should I have it on top of the affinity patches or not?
>>
>> For now get any tree you have out to take a look. In the longer
>> run we'll need to decide how we want to deal with the merge
>> of the two trees. In general having to depend on the RDMA
>> tree is a major risk because it's maintainance is so unreliable.
>
> Its really a small conflict, easy enough to get it without. The problem
> that no matter what I do, one tree would conflict...
Let's keep the refactoring self contained in nvme now and sort out
the small conflict when the rdma tree is merge (which is alsmost
certainly going to be after the block tree)
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 9:57 ` Christoph Hellwig
@ 2017-08-16 10:09 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 10:09 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: Christoph Hellwig, Keith Busch, linux-block, linux-nvme
>>> For now get any tree you have out to take a look. In the longer
>>> run we'll need to decide how we want to deal with the merge
>>> of the two trees. In general having to depend on the RDMA
>>> tree is a major risk because it's maintainance is so unreliable.
>>
>> Its really a small conflict, easy enough to get it without. The problem
>> that no matter what I do, one tree would conflict...
>
> Let's keep the refactoring self contained in nvme now and sort out
> the small conflict when the rdma tree is merge (which is alsmost
> certainly going to be after the block tree)
Sure, the patch set on top of nvme-4.14 (without the affinity patches)
is at:
git://git.infradead.org/users/sagi/linux.git nvme-central-er-4.14
Cheers,
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 10:09 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 10:09 UTC (permalink / raw)
>>> For now get any tree you have out to take a look. In the longer
>>> run we'll need to decide how we want to deal with the merge
>>> of the two trees. In general having to depend on the RDMA
>>> tree is a major risk because it's maintainance is so unreliable.
>>
>> Its really a small conflict, easy enough to get it without. The problem
>> that no matter what I do, one tree would conflict...
>
> Let's keep the refactoring self contained in nvme now and sort out
> the small conflict when the rdma tree is merge (which is alsmost
> certainly going to be after the block tree)
Sure, the patch set on top of nvme-4.14 (without the affinity patches)
is at:
git://git.infradead.org/users/sagi/linux.git nvme-central-er-4.14
Cheers,
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-15 9:52 ` Sagi Grimberg
@ 2017-08-16 10:49 ` Christoph Hellwig
-1 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 10:49 UTC (permalink / raw)
To: Sagi Grimberg; +Cc: linux-nvme, Christoph Hellwig, Keith Busch, linux-block
Did you get a chance to take a look at converting PCIe over?
A lot of the design decisions seem to be driven by allowing PCIe
as well, and I'd like to see if that's possible / useful to start
with.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 10:49 ` Christoph Hellwig
0 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 10:49 UTC (permalink / raw)
Did you get a chance to take a look at converting PCIe over?
A lot of the design decisions seem to be driven by allowing PCIe
as well, and I'd like to see if that's possible / useful to start
with.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 10:49 ` Christoph Hellwig
@ 2017-08-16 13:51 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 13:51 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-nvme, Keith Busch, linux-block
> Did you get a chance to take a look at converting PCIe over?
I started to play around with it, not anywhere near getting it in
shape though...
> A lot of the design decisions seem to be driven by allowing PCIe
> as well,
That's true.
> and I'd like to see if that's possible / useful to start
> with.
From what my radar got, it seems feasible. If you look at the nvme-core
routines, they really consists of the abstract part of the controller
initiation. From what I've seen so far, PCIe will require some extra
hooks on top of what we have now (some portion of nvme_pci_enable will
probably need to be added as well, and we'll need some generic
representation for pci device error states). I do expect nvme-core
changes for PCIe, but I don't expect the teardown/initiation structure
to change.
The only real design choice taken to accommodate PCIe was to manage
admin and io queues in the transports instead of nvme-core as queues
release procedure is different in PCIe.
I think its a very desirable goal to have PCIe consolidate with fabrics
here, so I'd like to shoot for that and backoff later (and change again
where makes sense).
Having said that, I'm very much open to suggestions if you have anything
better in mind...
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 13:51 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-16 13:51 UTC (permalink / raw)
> Did you get a chance to take a look at converting PCIe over?
I started to play around with it, not anywhere near getting it in
shape though...
> A lot of the design decisions seem to be driven by allowing PCIe
> as well,
That's true.
> and I'd like to see if that's possible / useful to start
> with.
From what my radar got, it seems feasible. If you look at the nvme-core
routines, they really consists of the abstract part of the controller
initiation. From what I've seen so far, PCIe will require some extra
hooks on top of what we have now (some portion of nvme_pci_enable will
probably need to be added as well, and we'll need some generic
representation for pci device error states). I do expect nvme-core
changes for PCIe, but I don't expect the teardown/initiation structure
to change.
The only real design choice taken to accommodate PCIe was to manage
admin and io queues in the transports instead of nvme-core as queues
release procedure is different in PCIe.
I think its a very desirable goal to have PCIe consolidate with fabrics
here, so I'd like to shoot for that and backoff later (and change again
where makes sense).
Having said that, I'm very much open to suggestions if you have anything
better in mind...
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 13:51 ` Sagi Grimberg
@ 2017-08-16 15:17 ` Christoph Hellwig
-1 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 15:17 UTC (permalink / raw)
To: Sagi Grimberg; +Cc: Christoph Hellwig, linux-nvme, Keith Busch, linux-block
On Wed, Aug 16, 2017 at 04:51:31PM +0300, Sagi Grimberg wrote:
> I think its a very desirable goal to have PCIe consolidate with fabrics
> here, so I'd like to shoot for that and backoff later (and change again
> where makes sense).
>
> Having said that, I'm very much open to suggestions if you have anything
> better in mind...
Not too many suggestions - you want to see it done before we go on
merging the code..
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-16 15:17 ` Christoph Hellwig
0 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-16 15:17 UTC (permalink / raw)
On Wed, Aug 16, 2017@04:51:31PM +0300, Sagi Grimberg wrote:
> I think its a very desirable goal to have PCIe consolidate with fabrics
> here, so I'd like to shoot for that and backoff later (and change again
> where makes sense).
>
> Having said that, I'm very much open to suggestions if you have anything
> better in mind...
Not too many suggestions - you want to see it done before we go on
merging the code..
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-16 15:17 ` Christoph Hellwig
@ 2017-08-17 7:21 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-17 7:21 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-nvme, Keith Busch, linux-block
>> Having said that, I'm very much open to suggestions if you have anything
>> better in mind...
>
> Not too many suggestions - you want to see it done before we go on
> merging the code..
Something is not adding up in your reply :)
You want to see it done before we merge? Or are you asking if I
want to see it done? I designed it to be incremental and not
break existing transports.
Personally I think that pci conversion will have to be incremental
and gradual as we need some preparations in place first. So if you are
reluctant to get it in without full conversion its ok, just note it will
take me some time to get both pci and fc converted.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-17 7:21 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-17 7:21 UTC (permalink / raw)
>> Having said that, I'm very much open to suggestions if you have anything
>> better in mind...
>
> Not too many suggestions - you want to see it done before we go on
> merging the code..
Something is not adding up in your reply :)
You want to see it done before we merge? Or are you asking if I
want to see it done? I designed it to be incremental and not
break existing transports.
Personally I think that pci conversion will have to be incremental
and gradual as we need some preparations in place first. So if you are
reluctant to get it in without full conversion its ok, just note it will
take me some time to get both pci and fc converted.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-17 7:21 ` Sagi Grimberg
@ 2017-08-17 7:36 ` Christoph Hellwig
-1 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-17 7:36 UTC (permalink / raw)
To: Sagi Grimberg; +Cc: Christoph Hellwig, linux-nvme, Keith Busch, linux-block
On Thu, Aug 17, 2017 at 10:21:08AM +0300, Sagi Grimberg wrote:
>
>>> Having said that, I'm very much open to suggestions if you have anything
>>> better in mind...
>>
>> Not too many suggestions - you want to see it done before we go on
>> merging the code..
>
> Something is not adding up in your reply :)
>
> You want to see it done before we merge? Or are you asking if I
> want to see it done? I designed it to be incremental and not
> break existing transports.
>
> Personally I think that pci conversion will have to be incremental
> and gradual as we need some preparations in place first. So if you are
> reluctant to get it in without full conversion its ok, just note it will
> take me some time to get both pci and fc converted.
I'd really like to see at least a demo PCI conversion. Guess I need
to take care of it then. I'd also like to see FC, but I'm not that
concerned about it as it better fits into the Fabrics scheme.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-17 7:36 ` Christoph Hellwig
0 siblings, 0 replies; 50+ messages in thread
From: Christoph Hellwig @ 2017-08-17 7:36 UTC (permalink / raw)
On Thu, Aug 17, 2017@10:21:08AM +0300, Sagi Grimberg wrote:
>
>>> Having said that, I'm very much open to suggestions if you have anything
>>> better in mind...
>>
>> Not too many suggestions - you want to see it done before we go on
>> merging the code..
>
> Something is not adding up in your reply :)
>
> You want to see it done before we merge? Or are you asking if I
> want to see it done? I designed it to be incremental and not
> break existing transports.
>
> Personally I think that pci conversion will have to be incremental
> and gradual as we need some preparations in place first. So if you are
> reluctant to get it in without full conversion its ok, just note it will
> take me some time to get both pci and fc converted.
I'd really like to see at least a demo PCI conversion. Guess I need
to take care of it then. I'd also like to see FC, but I'm not that
concerned about it as it better fits into the Fabrics scheme.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Re: Centralize nvme controller reset, delete and fabrics periodic reconnects
2017-08-17 7:36 ` Christoph Hellwig
@ 2017-08-20 6:37 ` Sagi Grimberg
-1 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-20 6:37 UTC (permalink / raw)
To: Christoph Hellwig; +Cc: linux-nvme, Keith Busch, linux-block
>> You want to see it done before we merge? Or are you asking if I
>> want to see it done? I designed it to be incremental and not
>> break existing transports.
>>
>> Personally I think that pci conversion will have to be incremental
>> and gradual as we need some preparations in place first. So if you are
>> reluctant to get it in without full conversion its ok, just note it will
>> take me some time to get both pci and fc converted.
>
> I'd really like to see at least a demo PCI conversion.
Let me see what if I can allocate some time for it.
> Guess I need to take care of it then.
If you feel this is a gating item for inclusion then your help will be
highly appreciated.
^ permalink raw reply [flat|nested] 50+ messages in thread
* Centralize nvme controller reset, delete and fabrics periodic reconnects
@ 2017-08-20 6:37 ` Sagi Grimberg
0 siblings, 0 replies; 50+ messages in thread
From: Sagi Grimberg @ 2017-08-20 6:37 UTC (permalink / raw)
>> You want to see it done before we merge? Or are you asking if I
>> want to see it done? I designed it to be incremental and not
>> break existing transports.
>>
>> Personally I think that pci conversion will have to be incremental
>> and gradual as we need some preparations in place first. So if you are
>> reluctant to get it in without full conversion its ok, just note it will
>> take me some time to get both pci and fc converted.
>
> I'd really like to see at least a demo PCI conversion.
Let me see what if I can allocate some time for it.
> Guess I need to take care of it then.
If you feel this is a gating item for inclusion then your help will be
highly appreciated.
^ permalink raw reply [flat|nested] 50+ messages in thread
end of thread, other threads:[~2017-08-20 6:38 UTC | newest]
Thread overview: 50+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-15 9:52 Centralize nvme controller reset, delete and fabrics periodic reconnects Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 01/12] nvme: move err and reconnect work to nvme ctrl Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 02/12] nvme-rdma: move admin specific resources to alloc_queue Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 03/12] nvme-rdma: split nvme_rdma_alloc_io_queues Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 04/12] nvme-rdma: restructure create_ctrl a bit Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 05/12] nvme-rdma: introduce nvme_rdma_alloc/stop/free_admin_queue Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 06/12] nvme-rdma: plumb nvme ctrl to various routines Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 07/12] nvme-rdma: split generic probe out of create_ctrl Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 08/12] nvme: add some ctrl ops for centralizing control plane logic Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 09/12] nvme: move control plane handling to nvme core Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 10/12] nvme-fabrics: handle reconnects in fabrics library Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 11/12] nvme: add sed-opal ctrl manipulation in admin configuration Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-15 9:52 ` [PATCH 12/12] nvme-loop: convert to nvme-core control plane management Sagi Grimberg
2017-08-15 9:52 ` Sagi Grimberg
2017-08-16 8:16 ` Centralize nvme controller reset, delete and fabrics periodic reconnects Christoph Hellwig
2017-08-16 8:16 ` Christoph Hellwig
2017-08-16 9:33 ` Sagi Grimberg
2017-08-16 9:33 ` Sagi Grimberg
2017-08-16 9:35 ` Christoph Hellwig
2017-08-16 9:35 ` Christoph Hellwig
2017-08-16 9:46 ` Sagi Grimberg
2017-08-16 9:46 ` Sagi Grimberg
2017-08-16 9:57 ` Christoph Hellwig
2017-08-16 9:57 ` Christoph Hellwig
2017-08-16 10:09 ` Sagi Grimberg
2017-08-16 10:09 ` Sagi Grimberg
2017-08-16 10:49 ` Christoph Hellwig
2017-08-16 10:49 ` Christoph Hellwig
2017-08-16 13:51 ` Sagi Grimberg
2017-08-16 13:51 ` Sagi Grimberg
2017-08-16 15:17 ` Christoph Hellwig
2017-08-16 15:17 ` Christoph Hellwig
2017-08-17 7:21 ` Sagi Grimberg
2017-08-17 7:21 ` Sagi Grimberg
2017-08-17 7:36 ` Christoph Hellwig
2017-08-17 7:36 ` Christoph Hellwig
2017-08-20 6:37 ` Sagi Grimberg
2017-08-20 6:37 ` Sagi Grimberg
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.