[PATCH 0/2] hw/block/nvme: fixes for interrupt behavior

[PATCH 0/2] hw/block/nvme: fixes for interrupt behavior

[Klaus Jensen]

From: Klaus Jensen <k.jensen@samsung.com>

Klaus Jensen (2):
  hw/block/nvme: fix pin-based interrupt behavior
  hw/block/nvme: allow use of any valid msix vector

 hw/block/nvme.c | 14 +++++++++-----
 hw/block/nvme.h |  2 +-
 2 files changed, 10 insertions(+), 6 deletions(-)

-- 
2.26.2

[PATCH 1/2] hw/block/nvme: fix pin-based interrupt behavior

[Klaus Jensen]

From: Klaus Jensen <k.jensen@samsung.com>

First, since the device only supports MSI-X or pin-based interrupt, if
MSI-X is not enabled, it should not accept interrupt vectors different
from 0 when creating completion queues.

Secondly, the irq_status NvmeCtrl member is meant to be compared to the
INTMS register, so it should only be 32 bits wide. And it is really only
useful when used with multi-message MSI.

Third, since we do not force a 1-to-1 correspondence between cqid and
interrupt vector, the irq_status register should not have bits set
according to cqid, but according to the associated interrupt vector.

Fix these issues, but keep irq_status available so we can easily support
multi-message MSI down the line.

Fixes: 5e9aa92eb1a5 ("hw/block: Fix pin-based interrupt behaviour of NVMe")
Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/block/nvme.c | 12 ++++++++----
 hw/block/nvme.h |  2 +-
 2 files changed, 9 insertions(+), 5 deletions(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 9b453423cf2c..2b072bbb21e7 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -127,8 +127,8 @@ static void nvme_irq_assert(NvmeCtrl *n, NvmeCQueue *cq)
             msix_notify(&(n->parent_obj), cq->vector);
         } else {
             trace_nvme_irq_pin();
-            assert(cq->cqid < 64);
-            n->irq_status |= 1 << cq->cqid;
+            assert(cq->vector < 32);
+            n->irq_status |= 1 << cq->vector;
             nvme_irq_check(n);
         }
     } else {
@@ -142,8 +142,8 @@ static void nvme_irq_deassert(NvmeCtrl *n, NvmeCQueue *cq)
         if (msix_enabled(&(n->parent_obj))) {
             return;
         } else {
-            assert(cq->cqid < 64);
-            n->irq_status &= ~(1 << cq->cqid);
+            assert(cq->vector < 32);
+            n->irq_status &= ~(1 << cq->vector);
             nvme_irq_check(n);
         }
     }
@@ -642,6 +642,10 @@ static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeCmd *cmd)
         trace_nvme_err_invalid_create_cq_addr(prp1);
         return NVME_INVALID_FIELD | NVME_DNR;
     }
+    if (unlikely(!msix_enabled(&n->parent_obj) && vector)) {
+        trace_nvme_err_invalid_create_cq_vector(vector);
+        return NVME_INVALID_IRQ_VECTOR | NVME_DNR;
+    }
     if (unlikely(vector > n->num_queues)) {
         trace_nvme_err_invalid_create_cq_vector(vector);
         return NVME_INVALID_IRQ_VECTOR | NVME_DNR;
diff --git a/hw/block/nvme.h b/hw/block/nvme.h
index 6520a9f0bead..db62589247d7 100644
--- a/hw/block/nvme.h
+++ b/hw/block/nvme.h
@@ -78,7 +78,7 @@ typedef struct NvmeCtrl {
     uint32_t    cmbsz;
     uint32_t    cmbloc;
     uint8_t     *cmbuf;
-    uint64_t    irq_status;
+    uint32_t    irq_status;
     uint64_t    host_timestamp;                 /* Timestamp sent by the host */
     uint64_t    timestamp_set_qemu_clock_ms;    /* QEMU clock time */
 
-- 
2.26.2

[PATCH 2/2] hw/block/nvme: allow use of any valid msix vector

[Klaus Jensen]

From: Klaus Jensen <k.jensen@samsung.com>

If the device uses MSI-X, any of the 2048 MSI-X interrupt vectors are
valid. If the device is not using MSI-X, vector will and can only be
zero at this point.

Signed-off-by: Klaus Jensen <k.jensen@samsung.com>
---
 hw/block/nvme.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 2b072bbb21e7..755ced8b03fb 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -646,7 +646,7 @@ static uint16_t nvme_create_cq(NvmeCtrl *n, NvmeCmd *cmd)
         trace_nvme_err_invalid_create_cq_vector(vector);
         return NVME_INVALID_IRQ_VECTOR | NVME_DNR;
     }
-    if (unlikely(vector > n->num_queues)) {
+    if (unlikely(vector > PCI_MSIX_FLAGS_QSIZE)) {
         trace_nvme_err_invalid_create_cq_vector(vector);
         return NVME_INVALID_IRQ_VECTOR | NVME_DNR;
     }
-- 
2.26.2

Re: [PATCH 0/2] hw/block/nvme: fixes for interrupt behavior

[Philippe Mathieu-Daudé]

+ Michael & Marcel for MSIX,
and ping to Keith :)

On 5/5/20 10:36 PM, Klaus Jensen wrote:
> From: Klaus Jensen <k.jensen@samsung.com>
> 
> Klaus Jensen (2):
>    hw/block/nvme: fix pin-based interrupt behavior
>    hw/block/nvme: allow use of any valid msix vector
> 
>   hw/block/nvme.c | 14 +++++++++-----
>   hw/block/nvme.h |  2 +-
>   2 files changed, 10 insertions(+), 6 deletions(-)

Re: [PATCH 0/2] hw/block/nvme: fixes for interrupt behavior

[Klaus Jensen]

On May 11 09:09, Philippe Mathieu-Daudé wrote:
> + Michael & Marcel for MSIX,
> and ping to Keith :)
> 

I'll await some feedback here before moving these fixes into my "nvme:
refactoring and cleanups" series. Not sure if this should go directly to
master or block-next.