linux-nvme.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created
@ 2020-03-22 17:59 Israel Rukshin
  2020-03-22 17:59 ` [PATCH 1/6] nvme: Remove unused return code from nvme_delete_ctrl_sync Israel Rukshin
                   ` (5 more replies)
  0 siblings, 6 replies; 15+ messages in thread
From: Israel Rukshin @ 2020-03-22 17:59 UTC (permalink / raw)
  To: Linux-nvme, Sagi Grimberg, Christoph Hellwig
  Cc: Shlomi Nimrodi, Israel Rukshin, Max Gurtovoy

The patchset starts with small cleanups, continue with the fixs and ends
with adding new warnings.
Calling nvme_sysfs_delete() when the controller is in the middle of
creation may cause several bugs. If the controller is in NEW state we
remove delete_controller file and don't delete the controller. The user
will not be able to use nvme disconnect command on that controller again,
although the controller may be active. Other bugs may happen if the
controller is in the middle of create_ctrl callback and
nvme_do_delete_ctrl() starts. For example, freeing I/O tagset at
nvme_do_delete_ctrl() before it was created or controller use after free
at create_ctrl callback.

Changes from v1:
 - Added Reviewed-by signatures
 - Patches reordering
 - New patches (2/6 and 3/6)
 - Split to 2 WARN_ON_ONCE (Patches 5/6 and 6/6)

Israel Rukshin (6):
  nvme: Remove unused return code from nvme_delete_ctrl_sync
  nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe
  nvme: Fix ctrl use-after-free during sysfs deletion
  nvme: Fix controller creation races with teardown flow
  nvme-rdma: Add warning on state change failure at nvme_rdma_setup_ctrl
  nvme-tcp: Add warning on state change failure at nvme_tcp_setup_ctrl

 drivers/nvme/host/core.c   | 18 ++++++++++--------
 drivers/nvme/host/fc.c     |  3 ---
 drivers/nvme/host/nvme.h   |  1 +
 drivers/nvme/host/pci.c    | 10 ++++------
 drivers/nvme/host/rdma.c   |  9 ++++++---
 drivers/nvme/host/tcp.c    |  9 ++++++---
 drivers/nvme/target/loop.c |  3 ---
 7 files changed, 27 insertions(+), 26 deletions(-)

-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH 1/6] nvme: Remove unused return code from nvme_delete_ctrl_sync
  2020-03-22 17:59 [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created Israel Rukshin
@ 2020-03-22 17:59 ` Israel Rukshin
  2020-03-22 17:59 ` [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe Israel Rukshin
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 15+ messages in thread
From: Israel Rukshin @ 2020-03-22 17:59 UTC (permalink / raw)
  To: Linux-nvme, Sagi Grimberg, Christoph Hellwig
  Cc: Shlomi Nimrodi, Israel Rukshin, Max Gurtovoy

The return code of nvme_delete_ctrl_sync is never used, so change it to
void.

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 drivers/nvme/host/core.c | 9 ++-------
 1 file changed, 2 insertions(+), 7 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 68e7c75..a461220 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -192,21 +192,16 @@ int nvme_delete_ctrl(struct nvme_ctrl *ctrl)
 }
 EXPORT_SYMBOL_GPL(nvme_delete_ctrl);
 
-static int nvme_delete_ctrl_sync(struct nvme_ctrl *ctrl)
+static void nvme_delete_ctrl_sync(struct nvme_ctrl *ctrl)
 {
-	int ret = 0;
-
 	/*
 	 * Keep a reference until nvme_do_delete_ctrl() complete,
 	 * since ->delete_ctrl can free the controller.
 	 */
 	nvme_get_ctrl(ctrl);
-	if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
-		ret = -EBUSY;
-	if (!ret)
+	if (nvme_change_ctrl_state(ctrl, NVME_CTRL_DELETING))
 		nvme_do_delete_ctrl(ctrl);
 	nvme_put_ctrl(ctrl);
-	return ret;
 }
 
 static inline bool nvme_ns_has_pi(struct nvme_ns *ns)
-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe
  2020-03-22 17:59 [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created Israel Rukshin
  2020-03-22 17:59 ` [PATCH 1/6] nvme: Remove unused return code from nvme_delete_ctrl_sync Israel Rukshin
@ 2020-03-22 17:59 ` Israel Rukshin
  2020-03-23  7:27   ` Christoph Hellwig
  2020-03-23 16:21   ` Keith Busch
  2020-03-22 17:59 ` [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion Israel Rukshin
                   ` (3 subsequent siblings)
  5 siblings, 2 replies; 15+ messages in thread
From: Israel Rukshin @ 2020-03-22 17:59 UTC (permalink / raw)
  To: Linux-nvme, Sagi Grimberg, Christoph Hellwig
  Cc: Shlomi Nimrodi, Israel Rukshin, Max Gurtovoy

Destroy the resources in the same order like in nvme_probe() error flow.

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
---
 drivers/nvme/host/pci.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index e6fa0c7..eb65b3b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2470,13 +2470,15 @@ static void nvme_pci_free_ctrl(struct nvme_ctrl *ctrl)
 	struct nvme_dev *dev = to_nvme_dev(ctrl);
 
 	nvme_dbbuf_dma_free(dev);
-	put_device(dev->dev);
 	nvme_free_tagset(dev);
 	if (dev->ctrl.admin_q)
 		blk_put_queue(dev->ctrl.admin_q);
-	kfree(dev->queues);
 	free_opal_dev(dev->ctrl.opal_dev);
 	mempool_destroy(dev->iod_mempool);
+	nvme_release_prp_pools(dev);
+	nvme_dev_unmap(dev);
+	put_device(dev->dev);
+	kfree(dev->queues);
 	kfree(dev);
 }
 
@@ -2875,8 +2877,6 @@ static void nvme_remove(struct pci_dev *pdev)
 	nvme_dev_remove_admin(dev);
 	nvme_free_queues(dev, 0);
 	nvme_uninit_ctrl(&dev->ctrl);
-	nvme_release_prp_pools(dev);
-	nvme_dev_unmap(dev);
 	nvme_put_ctrl(&dev->ctrl);
 }
 
-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion
  2020-03-22 17:59 [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created Israel Rukshin
  2020-03-22 17:59 ` [PATCH 1/6] nvme: Remove unused return code from nvme_delete_ctrl_sync Israel Rukshin
  2020-03-22 17:59 ` [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe Israel Rukshin
@ 2020-03-22 17:59 ` Israel Rukshin
  2020-03-23  7:32   ` Christoph Hellwig
  2020-03-22 17:59 ` [PATCH 4/6] nvme: Fix controller creation races with teardown flow Israel Rukshin
                   ` (2 subsequent siblings)
  5 siblings, 1 reply; 15+ messages in thread
From: Israel Rukshin @ 2020-03-22 17:59 UTC (permalink / raw)
  To: Linux-nvme, Sagi Grimberg, Christoph Hellwig
  Cc: Shlomi Nimrodi, Israel Rukshin, Max Gurtovoy

In case nvme_sysfs_delete() is called by the user before taking the ctrl
reference count, the ctrl may be freed during the creation and cause the
bug. Take the reference as soon as the controller is externally visible,
which is done by cdev_device_add() in nvme_init_ctrl(). Also take the
reference count at the core layer instead of taking it on each transport
separately.

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
---
 drivers/nvme/host/core.c   | 4 +++-
 drivers/nvme/host/fc.c     | 3 ---
 drivers/nvme/host/pci.c    | 2 --
 drivers/nvme/host/rdma.c   | 2 --
 drivers/nvme/host/tcp.c    | 2 --
 drivers/nvme/target/loop.c | 3 ---
 6 files changed, 3 insertions(+), 13 deletions(-)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index a461220..ba064fd 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -171,7 +171,6 @@ static void nvme_do_delete_ctrl(struct nvme_ctrl *ctrl)
 	nvme_remove_namespaces(ctrl);
 	ctrl->ops->delete_ctrl(ctrl);
 	nvme_uninit_ctrl(ctrl);
-	nvme_put_ctrl(ctrl);
 }
 
 static void nvme_delete_ctrl_work(struct work_struct *work)
@@ -4048,6 +4047,7 @@ void nvme_uninit_ctrl(struct nvme_ctrl *ctrl)
 	nvme_fault_inject_fini(&ctrl->fault_inject);
 	dev_pm_qos_hide_latency_tolerance(ctrl->device);
 	cdev_device_del(&ctrl->cdev, ctrl->device);
+	nvme_put_ctrl(ctrl);
 }
 EXPORT_SYMBOL_GPL(nvme_uninit_ctrl);
 
@@ -4130,6 +4130,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 	if (ret)
 		goto out_release_instance;
 
+	nvme_get_ctrl(ctrl);
 	cdev_init(&ctrl->cdev, &nvme_dev_fops);
 	ctrl->cdev.owner = ops->module;
 	ret = cdev_device_add(&ctrl->cdev, ctrl->device);
@@ -4148,6 +4149,7 @@ int nvme_init_ctrl(struct nvme_ctrl *ctrl, struct device *dev,
 
 	return 0;
 out_free_name:
+	nvme_put_ctrl(ctrl);
 	kfree_const(ctrl->device->kobj.name);
 out_release_instance:
 	ida_simple_remove(&nvme_instance_ida, ctrl->instance);
diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
index 5a70ac3..a8bf2fb 100644
--- a/drivers/nvme/host/fc.c
+++ b/drivers/nvme/host/fc.c
@@ -3181,10 +3181,7 @@ enum {
 		goto fail_ctrl;
 	}
 
-	nvme_get_ctrl(&ctrl->ctrl);
-
 	if (!queue_delayed_work(nvme_wq, &ctrl->connect_work, 0)) {
-		nvme_put_ctrl(&ctrl->ctrl);
 		dev_err(ctrl->ctrl.device,
 			"NVME-FC{%d}: failed to schedule initial connect\n",
 			ctrl->cnum);
diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index eb65b3b..334491b 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -2804,7 +2804,6 @@ static int nvme_probe(struct pci_dev *pdev, const struct pci_device_id *id)
 	dev_info(dev->ctrl.device, "pci function %s\n", dev_name(&pdev->dev));
 
 	nvme_reset_ctrl(&dev->ctrl);
-	nvme_get_ctrl(&dev->ctrl);
 	async_schedule(nvme_async_probe, dev);
 
 	return 0;
@@ -2877,7 +2876,6 @@ static void nvme_remove(struct pci_dev *pdev)
 	nvme_dev_remove_admin(dev);
 	nvme_free_queues(dev, 0);
 	nvme_uninit_ctrl(&dev->ctrl);
-	nvme_put_ctrl(&dev->ctrl);
 }
 
 #ifdef CONFIG_PM_SLEEP
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 3e85c5c..c99a882 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -2043,8 +2043,6 @@ static struct nvme_ctrl *nvme_rdma_create_ctrl(struct device *dev,
 	dev_info(ctrl->ctrl.device, "new ctrl: NQN \"%s\", addr %pISpcs\n",
 		ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
 
-	nvme_get_ctrl(&ctrl->ctrl);
-
 	mutex_lock(&nvme_rdma_ctrl_mutex);
 	list_add_tail(&ctrl->list, &nvme_rdma_ctrl_list);
 	mutex_unlock(&nvme_rdma_ctrl_mutex);
diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 814ea23..2fc2687 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -2429,8 +2429,6 @@ static struct nvme_ctrl *nvme_tcp_create_ctrl(struct device *dev,
 	dev_info(ctrl->ctrl.device, "new ctrl: NQN \"%s\", addr %pISp\n",
 		ctrl->ctrl.opts->subsysnqn, &ctrl->addr);
 
-	nvme_get_ctrl(&ctrl->ctrl);
-
 	mutex_lock(&nvme_tcp_ctrl_mutex);
 	list_add_tail(&ctrl->list, &nvme_tcp_ctrl_list);
 	mutex_unlock(&nvme_tcp_ctrl_mutex);
diff --git a/drivers/nvme/target/loop.c b/drivers/nvme/target/loop.c
index 4df4ebd..0d54e73 100644
--- a/drivers/nvme/target/loop.c
+++ b/drivers/nvme/target/loop.c
@@ -485,7 +485,6 @@ static void nvme_loop_reset_ctrl_work(struct work_struct *work)
 out_disable:
 	dev_warn(ctrl->ctrl.device, "Removing after reset failure\n");
 	nvme_uninit_ctrl(&ctrl->ctrl);
-	nvme_put_ctrl(&ctrl->ctrl);
 }
 
 static const struct nvme_ctrl_ops nvme_loop_ctrl_ops = {
@@ -618,8 +617,6 @@ static struct nvme_ctrl *nvme_loop_create_ctrl(struct device *dev,
 	dev_info(ctrl->ctrl.device,
 		 "new ctrl: \"%s\"\n", ctrl->ctrl.opts->subsysnqn);
 
-	nvme_get_ctrl(&ctrl->ctrl);
-
 	changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
 	WARN_ON_ONCE(!changed);
 
-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH 4/6] nvme: Fix controller creation races with teardown flow
  2020-03-22 17:59 [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created Israel Rukshin
                   ` (2 preceding siblings ...)
  2020-03-22 17:59 ` [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion Israel Rukshin
@ 2020-03-22 17:59 ` Israel Rukshin
  2020-03-23  7:34   ` Christoph Hellwig
  2020-03-22 17:59 ` [PATCH 5/6] nvme-rdma: Add warning on state change failure at nvme_rdma_setup_ctrl Israel Rukshin
  2020-03-22 17:59 ` [PATCH 6/6] nvme-tcp: Add warning on state change failure at nvme_tcp_setup_ctrl Israel Rukshin
  5 siblings, 1 reply; 15+ messages in thread
From: Israel Rukshin @ 2020-03-22 17:59 UTC (permalink / raw)
  To: Linux-nvme, Sagi Grimberg, Christoph Hellwig
  Cc: Shlomi Nimrodi, Israel Rukshin, Max Gurtovoy

Calling nvme_sysfs_delete() when the controller is in the middle of
creation may cause several bugs. If the controller is in NEW state we
remove delete_controller file and don't delete the controller. The user
will not be able to use nvme disconnect command on that controller again,
although the controller may be active. Other bugs may happen if the
controller is in the middle of create_ctrl callback and
nvme_do_delete_ctrl() starts. For example, freeing I/O tagset at
nvme_do_delete_ctrl() before it was allocated at create_ctrl callback.

To fix all those races don't allow the user to delete the controller
before it was fully created.

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
---
 drivers/nvme/host/core.c | 5 +++++
 drivers/nvme/host/nvme.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index ba064fd..9961d0e 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3228,6 +3228,10 @@ static ssize_t nvme_sysfs_delete(struct device *dev,
 {
 	struct nvme_ctrl *ctrl = dev_get_drvdata(dev);
 
+	/* Can't delete non-created controllers */
+	if (!ctrl->created)
+		return -EBUSY;
+
 	if (device_remove_file_self(dev, attr))
 		nvme_delete_ctrl_sync(ctrl);
 	return count;
@@ -4039,6 +4043,7 @@ void nvme_start_ctrl(struct nvme_ctrl *ctrl)
 		nvme_queue_scan(ctrl);
 		nvme_start_queues(ctrl);
 	}
+	ctrl->created = true;
 }
 EXPORT_SYMBOL_GPL(nvme_start_ctrl);
 
diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h
index d800b9a..2e04a36 100644
--- a/drivers/nvme/host/nvme.h
+++ b/drivers/nvme/host/nvme.h
@@ -259,6 +259,7 @@ struct nvme_ctrl {
 	struct nvme_command ka_cmd;
 	struct work_struct fw_act_work;
 	unsigned long events;
+	bool created;
 
 #ifdef CONFIG_NVME_MULTIPATH
 	/* asymmetric namespace access: */
-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH 5/6] nvme-rdma: Add warning on state change failure at nvme_rdma_setup_ctrl
  2020-03-22 17:59 [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created Israel Rukshin
                   ` (3 preceding siblings ...)
  2020-03-22 17:59 ` [PATCH 4/6] nvme: Fix controller creation races with teardown flow Israel Rukshin
@ 2020-03-22 17:59 ` Israel Rukshin
  2020-03-23  7:34   ` Christoph Hellwig
  2020-03-22 17:59 ` [PATCH 6/6] nvme-tcp: Add warning on state change failure at nvme_tcp_setup_ctrl Israel Rukshin
  5 siblings, 1 reply; 15+ messages in thread
From: Israel Rukshin @ 2020-03-22 17:59 UTC (permalink / raw)
  To: Linux-nvme, Sagi Grimberg, Christoph Hellwig
  Cc: Shlomi Nimrodi, Israel Rukshin, Max Gurtovoy

The transition to LIVE state should not fail in case of a new controller.
Moving to DELETING state before nvme_tcp_create_ctrl() allocates all the
resources may leads to NULL dereference at teardown flow (e.g., IO tagset,
admin_q, connect_q).

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
---
 drivers/nvme/host/rdma.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index c99a882..3ae3011 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -1022,8 +1022,13 @@ static int nvme_rdma_setup_ctrl(struct nvme_rdma_ctrl *ctrl, bool new)
 
 	changed = nvme_change_ctrl_state(&ctrl->ctrl, NVME_CTRL_LIVE);
 	if (!changed) {
-		/* state change failure is ok if we're in DELETING state */
+		/*
+		 * state change failure is ok if we're in DELETING state,
+		 * unless we're during creation of a new controller to
+		 * avoid races with teardown flow.
+		 */
 		WARN_ON_ONCE(ctrl->ctrl.state != NVME_CTRL_DELETING);
+		WARN_ON_ONCE(new);
 		ret = -EINVAL;
 		goto destroy_io;
 	}
-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH 6/6] nvme-tcp: Add warning on state change failure at nvme_tcp_setup_ctrl
  2020-03-22 17:59 [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created Israel Rukshin
                   ` (4 preceding siblings ...)
  2020-03-22 17:59 ` [PATCH 5/6] nvme-rdma: Add warning on state change failure at nvme_rdma_setup_ctrl Israel Rukshin
@ 2020-03-22 17:59 ` Israel Rukshin
  2020-03-23  7:34   ` Christoph Hellwig
  5 siblings, 1 reply; 15+ messages in thread
From: Israel Rukshin @ 2020-03-22 17:59 UTC (permalink / raw)
  To: Linux-nvme, Sagi Grimberg, Christoph Hellwig
  Cc: Shlomi Nimrodi, Israel Rukshin, Max Gurtovoy

The transition to LIVE state should not fail in case of a new controller.
Moving to DELETING state before nvme_tcp_create_ctrl() allocates all the
resources may leads to NULL dereference at teardown flow (e.g., IO tagset,
admin_q, connect_q).

Signed-off-by: Israel Rukshin <israelr@mellanox.com>
Reviewed-by: Max Gurtovoy <maxg@mellanox.com>
---
 drivers/nvme/host/tcp.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/drivers/nvme/host/tcp.c b/drivers/nvme/host/tcp.c
index 2fc2687..7aa2adc 100644
--- a/drivers/nvme/host/tcp.c
+++ b/drivers/nvme/host/tcp.c
@@ -1931,8 +1931,13 @@ static int nvme_tcp_setup_ctrl(struct nvme_ctrl *ctrl, bool new)
 	}
 
 	if (!nvme_change_ctrl_state(ctrl, NVME_CTRL_LIVE)) {
-		/* state change failure is ok if we're in DELETING state */
+		/*
+		 * state change failure is ok if we're in DELETING state,
+		 * unless we're during creation of a new controller to
+		 * avoid races with teardown flow.
+		 */
 		WARN_ON_ONCE(ctrl->state != NVME_CTRL_DELETING);
+		WARN_ON_ONCE(new);
 		ret = -EINVAL;
 		goto destroy_io;
 	}
-- 
1.8.3.1


_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply related	[flat|nested] 15+ messages in thread

* Re: [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe
  2020-03-22 17:59 ` [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe Israel Rukshin
@ 2020-03-23  7:27   ` Christoph Hellwig
  2020-03-23 16:21   ` Keith Busch
  1 sibling, 0 replies; 15+ messages in thread
From: Christoph Hellwig @ 2020-03-23  7:27 UTC (permalink / raw)
  To: Israel Rukshin
  Cc: Shlomi Nimrodi, Max Gurtovoy, Sagi Grimberg, Linux-nvme,
	Christoph Hellwig

On Sun, Mar 22, 2020 at 07:59:45PM +0200, Israel Rukshin wrote:
> Destroy the resources in the same order like in nvme_probe() error flow.

It also moves some calls from the ->remove PCI driver method to the
nvme_ctrl_ops ->free_ctrl method.  I think that looks safe, but it needs
to be properly documented in the changelog, including why you think it
is safe and useful.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion
  2020-03-22 17:59 ` [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion Israel Rukshin
@ 2020-03-23  7:32   ` Christoph Hellwig
  2020-03-23 16:10     ` Israel Rukshin
  0 siblings, 1 reply; 15+ messages in thread
From: Christoph Hellwig @ 2020-03-23  7:32 UTC (permalink / raw)
  To: Israel Rukshin
  Cc: Shlomi Nimrodi, Max Gurtovoy, Sagi Grimberg, Linux-nvme,
	Christoph Hellwig

> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -171,7 +171,6 @@ static void nvme_do_delete_ctrl(struct nvme_ctrl *ctrl)
>  	nvme_remove_namespaces(ctrl);
>  	ctrl->ops->delete_ctrl(ctrl);
>  	nvme_uninit_ctrl(ctrl);
> -	nvme_put_ctrl(ctrl);

Can you please split the change to call nvme_put_ctrl from
nvme_uninit_ctrl into a separate cleanup patch after this one?

The rest looks good to me.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH 4/6] nvme: Fix controller creation races with teardown flow
  2020-03-22 17:59 ` [PATCH 4/6] nvme: Fix controller creation races with teardown flow Israel Rukshin
@ 2020-03-23  7:34   ` Christoph Hellwig
  0 siblings, 0 replies; 15+ messages in thread
From: Christoph Hellwig @ 2020-03-23  7:34 UTC (permalink / raw)
  To: Israel Rukshin
  Cc: Shlomi Nimrodi, Max Gurtovoy, Sagi Grimberg, Linux-nvme,
	Christoph Hellwig

On Sun, Mar 22, 2020 at 07:59:47PM +0200, Israel Rukshin wrote:
> Calling nvme_sysfs_delete() when the controller is in the middle of
> creation may cause several bugs. If the controller is in NEW state we
> remove delete_controller file and don't delete the controller. The user
> will not be able to use nvme disconnect command on that controller again,
> although the controller may be active. Other bugs may happen if the
> controller is in the middle of create_ctrl callback and
> nvme_do_delete_ctrl() starts. For example, freeing I/O tagset at
> nvme_do_delete_ctrl() before it was allocated at create_ctrl callback.
> 
> To fix all those races don't allow the user to delete the controller
> before it was fully created.

Looks sensible.  My gut feeling is that this should be done through
the controller state machine, but I can't really see a way how that
could work:

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH 5/6] nvme-rdma: Add warning on state change failure at nvme_rdma_setup_ctrl
  2020-03-22 17:59 ` [PATCH 5/6] nvme-rdma: Add warning on state change failure at nvme_rdma_setup_ctrl Israel Rukshin
@ 2020-03-23  7:34   ` Christoph Hellwig
  0 siblings, 0 replies; 15+ messages in thread
From: Christoph Hellwig @ 2020-03-23  7:34 UTC (permalink / raw)
  To: Israel Rukshin
  Cc: Shlomi Nimrodi, Max Gurtovoy, Sagi Grimberg, Linux-nvme,
	Christoph Hellwig

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH 6/6] nvme-tcp: Add warning on state change failure at nvme_tcp_setup_ctrl
  2020-03-22 17:59 ` [PATCH 6/6] nvme-tcp: Add warning on state change failure at nvme_tcp_setup_ctrl Israel Rukshin
@ 2020-03-23  7:34   ` Christoph Hellwig
  0 siblings, 0 replies; 15+ messages in thread
From: Christoph Hellwig @ 2020-03-23  7:34 UTC (permalink / raw)
  To: Israel Rukshin
  Cc: Shlomi Nimrodi, Max Gurtovoy, Sagi Grimberg, Linux-nvme,
	Christoph Hellwig

On Sun, Mar 22, 2020 at 07:59:49PM +0200, Israel Rukshin wrote:
> The transition to LIVE state should not fail in case of a new controller.
> Moving to DELETING state before nvme_tcp_create_ctrl() allocates all the
> resources may leads to NULL dereference at teardown flow (e.g., IO tagset,
> admin_q, connect_q).
> 
> Signed-off-by: Israel Rukshin <israelr@mellanox.com>
> Reviewed-by: Max Gurtovoy <maxg@mellanox.com>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion
  2020-03-23  7:32   ` Christoph Hellwig
@ 2020-03-23 16:10     ` Israel Rukshin
  2020-03-23 17:23       ` Christoph Hellwig
  0 siblings, 1 reply; 15+ messages in thread
From: Israel Rukshin @ 2020-03-23 16:10 UTC (permalink / raw)
  To: Christoph Hellwig; +Cc: Max Gurtovoy, Sagi Grimberg, Linux-nvme, Shlomi Nimrodi

On 3/23/2020 9:32 AM, Christoph Hellwig wrote:
>> --- a/drivers/nvme/host/core.c
>> +++ b/drivers/nvme/host/core.c
>> @@ -171,7 +171,6 @@ static void nvme_do_delete_ctrl(struct nvme_ctrl *ctrl)
>>   	nvme_remove_namespaces(ctrl);
>>   	ctrl->ops->delete_ctrl(ctrl);
>>   	nvme_uninit_ctrl(ctrl);
>> -	nvme_put_ctrl(ctrl);
> Can you please split the change to call nvme_put_ctrl from
> nvme_uninit_ctrl into a separate cleanup patch after this one?

Sure, I can split it.

To do that I need to add  nvme_put_ctrl() at most of the error flows 
like this:

@@ -2054,6 +2052,7 @@ static struct nvme_ctrl 
*nvme_rdma_create_ctrl(struct device *dev,
  out_uninit_ctrl:
         nvme_uninit_ctrl(&ctrl->ctrl);
         nvme_put_ctrl(&ctrl->ctrl);
+       nvme_put_ctrl(&ctrl->ctrl);
         if (ret > 0)
                 ret = -EIO;
         return ERR_PTR(ret);


There are double calls to nvme_put_ctrl() at the error flows at rdma, 
tcp, fc and loop.

I will remove those at the next patch.

Is that OK?

>
> The rest looks good to me.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe
  2020-03-22 17:59 ` [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe Israel Rukshin
  2020-03-23  7:27   ` Christoph Hellwig
@ 2020-03-23 16:21   ` Keith Busch
  1 sibling, 0 replies; 15+ messages in thread
From: Keith Busch @ 2020-03-23 16:21 UTC (permalink / raw)
  To: Israel Rukshin
  Cc: Max Gurtovoy, Shlomi Nimrodi, Sagi Grimberg, Linux-nvme,
	Christoph Hellwig

On Sun, Mar 22, 2020 at 07:59:45PM +0200, Israel Rukshin wrote:
> +	nvme_release_prp_pools(dev);
> +	nvme_dev_unmap(dev);
> +	put_device(dev->dev);
> +	kfree(dev->queues);
>  	kfree(dev);
>  }
>  
> @@ -2875,8 +2877,6 @@ static void nvme_remove(struct pci_dev *pdev)
>  	nvme_dev_remove_admin(dev);
>  	nvme_free_queues(dev, 0);
>  	nvme_uninit_ctrl(&dev->ctrl);
> -	nvme_release_prp_pools(dev);
> -	nvme_dev_unmap(dev);
>  	nvme_put_ctrl(&dev->ctrl);
>  }

I think we actually need the nvme_dev_unmap() call to remain inline with
nvme_remove(). If you hot remove the drive while an application has an
active reference on it, we still need to release the PCI resources so
that they're available for the next device inserted.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion
  2020-03-23 16:10     ` Israel Rukshin
@ 2020-03-23 17:23       ` Christoph Hellwig
  0 siblings, 0 replies; 15+ messages in thread
From: Christoph Hellwig @ 2020-03-23 17:23 UTC (permalink / raw)
  To: Israel Rukshin
  Cc: Shlomi Nimrodi, Max Gurtovoy, Christoph Hellwig, Linux-nvme,
	Sagi Grimberg

On Mon, Mar 23, 2020 at 06:10:23PM +0200, Israel Rukshin wrote:
> There are double calls to nvme_put_ctrl() at the error flows at rdma, tcp, 
> fc and loop.
>
> I will remove those at the next patch.
>
> Is that OK?

Yes.

_______________________________________________
linux-nvme mailing list
linux-nvme@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-nvme

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2020-03-23 17:23 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-03-22 17:59 [PATCH 0/6 V2] nvme: Fixes for deleting a ctrl before it was created Israel Rukshin
2020-03-22 17:59 ` [PATCH 1/6] nvme: Remove unused return code from nvme_delete_ctrl_sync Israel Rukshin
2020-03-22 17:59 ` [PATCH 2/6] nvme-pci: Make nvme_pci_free_ctrl symmetric to nvme_probe Israel Rukshin
2020-03-23  7:27   ` Christoph Hellwig
2020-03-23 16:21   ` Keith Busch
2020-03-22 17:59 ` [PATCH 3/6] nvme: Fix ctrl use-after-free during sysfs deletion Israel Rukshin
2020-03-23  7:32   ` Christoph Hellwig
2020-03-23 16:10     ` Israel Rukshin
2020-03-23 17:23       ` Christoph Hellwig
2020-03-22 17:59 ` [PATCH 4/6] nvme: Fix controller creation races with teardown flow Israel Rukshin
2020-03-23  7:34   ` Christoph Hellwig
2020-03-22 17:59 ` [PATCH 5/6] nvme-rdma: Add warning on state change failure at nvme_rdma_setup_ctrl Israel Rukshin
2020-03-23  7:34   ` Christoph Hellwig
2020-03-22 17:59 ` [PATCH 6/6] nvme-tcp: Add warning on state change failure at nvme_tcp_setup_ctrl Israel Rukshin
2020-03-23  7:34   ` Christoph Hellwig

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).