[BACKPORT PATCH] nvme-pci: Remap CMB SQ entries on every controller reset

[BACKPORT PATCH] nvme-pci: Remap CMB SQ entries on every controller reset

[Scott Bauer]

Commit 815c6704bf9f1c59f3a6be380a4032b9c57b12f1 upstream.

The controller memory buffer is remapped into a kernel address on each
reset, but the driver was setting the submission queue base address
only on the very first queue creation. The remapped address is likely to
change after a reset, so accessing the old address will hit a kernel bug.

This patch fixes that by setting the queue's CMB base address each time
the queue is created.

Fixes: f63572dff1421 ("nvme: unmap CMB and remove sysfs file in reset path")
Reported-by: Christian Black <christian.d.black@intel.com>
Cc: Jon Derrick <jonathan.derrick@intel.com>
Signed-off-by: Keith Busch <keith.busch@intel.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: Scott Bauer <scott.bauer@intel.com>
---
 drivers/nvme/host/pci.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)

diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
index 3d4724e38aa9..4cac4755abef 100644
--- a/drivers/nvme/host/pci.c
+++ b/drivers/nvme/host/pci.c
@@ -1233,17 +1233,15 @@ static int nvme_cmb_qdepth(struct nvme_dev *dev, int nr_io_queues,
 static int nvme_alloc_sq_cmds(struct nvme_dev *dev, struct nvme_queue *nvmeq,
 				int qid, int depth)
 {
-	if (qid && dev->cmb && use_cmb_sqes && NVME_CMB_SQS(dev->cmbsz)) {
-		unsigned offset = (qid - 1) * roundup(SQ_SIZE(depth),
-						      dev->ctrl.page_size);
-		nvmeq->sq_dma_addr = dev->cmb_bus_addr + offset;
-		nvmeq->sq_cmds_io = dev->cmb + offset;
-	} else {
-		nvmeq->sq_cmds = dma_alloc_coherent(dev->dev, SQ_SIZE(depth),
-					&nvmeq->sq_dma_addr, GFP_KERNEL);
-		if (!nvmeq->sq_cmds)
-			return -ENOMEM;
-	}
+
+	/* CMB SQEs will be mapped before creation */
+	if (qid && dev->cmb && use_cmb_sqes && NVME_CMB_SQS(dev->cmbsz))
+		return 0;
+
+	nvmeq->sq_cmds = dma_alloc_coherent(dev->dev, SQ_SIZE(depth),
+					    &nvmeq->sq_dma_addr, GFP_KERNEL);
+	if (!nvmeq->sq_cmds)
+		return -ENOMEM;
 
 	return 0;
 }
@@ -1320,6 +1318,13 @@ static int nvme_create_queue(struct nvme_queue *nvmeq, int qid)
 	struct nvme_dev *dev = nvmeq->dev;
 	int result;
 
+	if (qid && dev->cmb && use_cmb_sqes && NVME_CMB_SQS(dev->cmbsz)) {
+		unsigned offset = (qid - 1) * roundup(SQ_SIZE(nvmeq->q_depth),
+						      dev->ctrl.page_size);
+		nvmeq->sq_dma_addr = dev->cmb_bus_addr + offset;
+		nvmeq->sq_cmds_io = dev->cmb + offset;
+	}
+
 	nvmeq->cq_vector = qid - 1;
 	result = adapter_alloc_cq(dev, qid, nvmeq);
 	if (result < 0)
-- 
2.17.1

Re: [BACKPORT PATCH] nvme-pci: Remap CMB SQ entries on every controller reset

[Derrick, Jonathan]

[-- Attachment #1: Type: text/plain, Size: 2853 bytes --]

Thank you, Scott

Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>

On Thu, 2018-07-12 at 15:27 -0600, Scott Bauer wrote:
> Commit 815c6704bf9f1c59f3a6be380a4032b9c57b12f1 upstream.
> 
> The controller memory buffer is remapped into a kernel address on
> each
> reset, but the driver was setting the submission queue base address
> only on the very first queue creation. The remapped address is likely
> to
> change after a reset, so accessing the old address will hit a kernel
> bug.
> 
> This patch fixes that by setting the queue's CMB base address each
> time
> the queue is created.
> 
> Fixes: f63572dff1421 ("nvme: unmap CMB and remove sysfs file in reset
> path")
> Reported-by: Christian Black <christian.d.black@intel.com>
> Cc: Jon Derrick <jonathan.derrick@intel.com>
> Signed-off-by: Keith Busch <keith.busch@intel.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Scott Bauer <scott.bauer@intel.com>
> ---
>  drivers/nvme/host/pci.c | 27 ++++++++++++++++-----------
>  1 file changed, 16 insertions(+), 11 deletions(-)
> 
> diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c
> index 3d4724e38aa9..4cac4755abef 100644
> --- a/drivers/nvme/host/pci.c
> +++ b/drivers/nvme/host/pci.c
> @@ -1233,17 +1233,15 @@ static int nvme_cmb_qdepth(struct nvme_dev
> *dev, int nr_io_queues,
>  static int nvme_alloc_sq_cmds(struct nvme_dev *dev, struct
> nvme_queue *nvmeq,
>  				int qid, int depth)
>  {
> -	if (qid && dev->cmb && use_cmb_sqes && NVME_CMB_SQS(dev-
> >cmbsz)) {
> -		unsigned offset = (qid - 1) *
> roundup(SQ_SIZE(depth),
> -						      dev-
> >ctrl.page_size);
> -		nvmeq->sq_dma_addr = dev->cmb_bus_addr + offset;
> -		nvmeq->sq_cmds_io = dev->cmb + offset;
> -	} else {
> -		nvmeq->sq_cmds = dma_alloc_coherent(dev->dev,
> SQ_SIZE(depth),
> -					&nvmeq->sq_dma_addr,
> GFP_KERNEL);
> -		if (!nvmeq->sq_cmds)
> -			return -ENOMEM;
> -	}
> +
> +	/* CMB SQEs will be mapped before creation */
> +	if (qid && dev->cmb && use_cmb_sqes && NVME_CMB_SQS(dev-
> >cmbsz))
> +		return 0;
> +
> +	nvmeq->sq_cmds = dma_alloc_coherent(dev->dev,
> SQ_SIZE(depth),
> +					    &nvmeq->sq_dma_addr,
> GFP_KERNEL);
> +	if (!nvmeq->sq_cmds)
> +		return -ENOMEM;
>  
>  	return 0;
>  }
> @@ -1320,6 +1318,13 @@ static int nvme_create_queue(struct nvme_queue
> *nvmeq, int qid)
>  	struct nvme_dev *dev = nvmeq->dev;
>  	int result;
>  
> +	if (qid && dev->cmb && use_cmb_sqes && NVME_CMB_SQS(dev-
> >cmbsz)) {
> +		unsigned offset = (qid - 1) * roundup(SQ_SIZE(nvmeq-
> >q_depth),
> +						      dev-
> >ctrl.page_size);
> +		nvmeq->sq_dma_addr = dev->cmb_bus_addr + offset;
> +		nvmeq->sq_cmds_io = dev->cmb + offset;
> +	}
> +
>  	nvmeq->cq_vector = qid - 1;
>  	result = adapter_alloc_cq(dev, qid, nvmeq);
>  	if (result < 0)

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3278 bytes --]

Re: [BACKPORT PATCH] nvme-pci: Remap CMB SQ entries on every controller reset

[Greg KH]

On Thu, Jul 12, 2018 at 03:27:17PM -0600, Scott Bauer wrote:
> Commit 815c6704bf9f1c59f3a6be380a4032b9c57b12f1 upstream.
> 
> The controller memory buffer is remapped into a kernel address on each
> reset, but the driver was setting the submission queue base address
> only on the very first queue creation. The remapped address is likely to
> change after a reset, so accessing the old address will hit a kernel bug.
> 
> This patch fixes that by setting the queue's CMB base address each time
> the queue is created.
> 
> Fixes: f63572dff1421 ("nvme: unmap CMB and remove sysfs file in reset path")
> Reported-by: Christian Black <christian.d.black@intel.com>
> Cc: Jon Derrick <jonathan.derrick@intel.com>
> Signed-off-by: Keith Busch <keith.busch@intel.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Scott Bauer <scott.bauer@intel.com>
> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>
> ---
>  drivers/nvme/host/pci.c | 27 ++++++++++++++++-----------
>  1 file changed, 16 insertions(+), 11 deletions(-)

"backport" to where?  What kernel tree(s) do you want this applied to?

I need a hint please...

thanks,

greg k-h

Re: [BACKPORT PATCH] nvme-pci: Remap CMB SQ entries on every controller reset

[Keith Busch]

On Fri, Jul 13, 2018 at 02:01:04PM +0200, Greg KH wrote:
> On Thu, Jul 12, 2018 at 03:27:17PM -0600, Scott Bauer wrote:
> > Commit 815c6704bf9f1c59f3a6be380a4032b9c57b12f1 upstream.
> > 
> > The controller memory buffer is remapped into a kernel address on each
> > reset, but the driver was setting the submission queue base address
> > only on the very first queue creation. The remapped address is likely to
> > change after a reset, so accessing the old address will hit a kernel bug.
> > 
> > This patch fixes that by setting the queue's CMB base address each time
> > the queue is created.
> > 
> > Fixes: f63572dff1421 ("nvme: unmap CMB and remove sysfs file in reset path")
> > Reported-by: Christian Black <christian.d.black@intel.com>
> > Cc: Jon Derrick <jonathan.derrick@intel.com>
> > Signed-off-by: Keith Busch <keith.busch@intel.com>
> > Reviewed-by: Christoph Hellwig <hch@lst.de>
> > Signed-off-by: Scott Bauer <scott.bauer@intel.com>
> > Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>
> > ---
> >  drivers/nvme/host/pci.c | 27 ++++++++++++++++-----------
> >  1 file changed, 16 insertions(+), 11 deletions(-)
> 
> "backport" to where?  What kernel tree(s) do you want this applied to?
> 
> I need a hint please...

Sorry about that. 4.9 and 4.14 are the applicable targets.

Re: [BACKPORT PATCH] nvme-pci: Remap CMB SQ entries on every controller reset

[Greg KH]

On Thu, Jul 12, 2018 at 03:27:17PM -0600, Scott Bauer wrote:
> Commit 815c6704bf9f1c59f3a6be380a4032b9c57b12f1 upstream.
> 
> The controller memory buffer is remapped into a kernel address on each
> reset, but the driver was setting the submission queue base address
> only on the very first queue creation. The remapped address is likely to
> change after a reset, so accessing the old address will hit a kernel bug.
> 
> This patch fixes that by setting the queue's CMB base address each time
> the queue is created.
> 
> Fixes: f63572dff1421 ("nvme: unmap CMB and remove sysfs file in reset path")
> Reported-by: Christian Black <christian.d.black@intel.com>
> Cc: Jon Derrick <jonathan.derrick@intel.com>
> Signed-off-by: Keith Busch <keith.busch@intel.com>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Signed-off-by: Scott Bauer <scott.bauer@intel.com>
> Reviewed-by: Jon Derrick <jonathan.derrick@intel.com>
> ---
>  drivers/nvme/host/pci.c | 27 ++++++++++++++++-----------
>  1 file changed, 16 insertions(+), 11 deletions(-)

Now queued up, thanks.

greg k-h