[PATCH 0/3] Fix zone write validation

[PATCH 0/3] Fix zone write validation

[Dmitry Fomichev]

These patches solve a few problems that exist in zoned Write
ans Zone Append validation code.

Dmitry Fomichev (3):
  hw/block/nvme: Check for zone boundary during append
  hw/block/nvme: Check zone state before checking boundaries
  hw/block/nvme: Add trace events for zone boundary violations

 hw/block/nvme.c       | 35 ++++++++++++++++++++---------------
 hw/block/trace-events |  3 +++
 2 files changed, 23 insertions(+), 15 deletions(-)

-- 
2.28.0

[PATCH 1/3] hw/block/nvme: Check for zone boundary during append

[Dmitry Fomichev]

It is observed that with the existing code it is possible to keep
appending to a zone indefinitely. To fix, add the missing check to
verify that the zone append is not going to write beyond zone capacity.

Reported-by: Niklas Cassel <niklas.cassel@wdc.com>
Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
---
 hw/block/nvme.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index f64676a930..67538010ef 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1135,9 +1135,10 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
                                       NvmeZone *zone, uint64_t slba,
                                       uint32_t nlb, bool append)
 {
+    uint64_t bndry = nvme_zone_wr_boundary(zone);
     uint16_t status;
 
-    if (unlikely((slba + nlb) > nvme_zone_wr_boundary(zone))) {
+    if (unlikely(slba + nlb > bndry)) {
         status = NVME_ZONE_BOUNDARY_ERROR;
     } else {
         status = nvme_check_zone_state_for_write(zone);
@@ -1151,8 +1152,9 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
             if (unlikely(slba != zone->d.zslba)) {
                 trace_pci_nvme_err_append_not_at_start(slba, zone->d.zslba);
                 status = NVME_INVALID_FIELD;
-            }
-            if (nvme_l2b(ns, nlb) > (n->page_size << n->zasl)) {
+            } else if (unlikely(zone->w_ptr + nlb > bndry)) {
+                status = NVME_ZONE_BOUNDARY_ERROR;
+            } else if (nvme_l2b(ns, nlb) > (n->page_size << n->zasl)) {
                 trace_pci_nvme_err_append_too_large(slba, nlb, n->zasl);
                 status = NVME_INVALID_FIELD;
             }
-- 
2.28.0

[PATCH 2/3] hw/block/nvme: Check zone state before checking boundaries

[Dmitry Fomichev]

The code in nvme_check_zone_write() first checks for zone boundary
condition violation and only after that it proceeds to verify that the
zone state is suitable the write to happen. This is not consistent with
nvme_check_zone_read() flow - in that function, zone state is checked
before making any boundary checks. Besides, checking that ZSLBA + NLB
does not cross the write boundary is now redundant for Zone Append and
only needs to be done for writes.

Move the check in the code to be performed for Write and Write Zeroes
commands only and to occur after zone state checks.

Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
---
 hw/block/nvme.c | 21 ++++++++++-----------
 1 file changed, 10 insertions(+), 11 deletions(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index 67538010ef..b712634c27 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1138,13 +1138,8 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
     uint64_t bndry = nvme_zone_wr_boundary(zone);
     uint16_t status;
 
-    if (unlikely(slba + nlb > bndry)) {
-        status = NVME_ZONE_BOUNDARY_ERROR;
-    } else {
-        status = nvme_check_zone_state_for_write(zone);
-    }
-
-    if (status != NVME_SUCCESS) {
+    status = nvme_check_zone_state_for_write(zone);
+    if (status) {
         trace_pci_nvme_err_zone_write_not_ok(slba, nlb, status);
     } else {
         assert(nvme_wp_is_valid(zone));
@@ -1158,10 +1153,14 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
                 trace_pci_nvme_err_append_too_large(slba, nlb, n->zasl);
                 status = NVME_INVALID_FIELD;
             }
-        } else if (unlikely(slba != zone->w_ptr)) {
-            trace_pci_nvme_err_write_not_at_wp(slba, zone->d.zslba,
-                                               zone->w_ptr);
-            status = NVME_ZONE_INVALID_WRITE;
+        } else {
+            if (unlikely(slba != zone->w_ptr)) {
+                trace_pci_nvme_err_write_not_at_wp(slba, zone->d.zslba,
+                                                   zone->w_ptr);
+                status = NVME_ZONE_INVALID_WRITE;
+            } else if (unlikely(slba + nlb > bndry)) {
+                status = NVME_ZONE_BOUNDARY_ERROR;
+            }
         }
     }
 
-- 
2.28.0

[PATCH 3/3] hw/block/nvme: Add trace events for zone boundary violations

[Dmitry Fomichev]

Add three new trace events to be raised in case of zone boundary
violations during Read, Write(Zeroes) and Zone Append.

Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
---
 hw/block/nvme.c       | 8 ++++++--
 hw/block/trace-events | 3 +++
 2 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/hw/block/nvme.c b/hw/block/nvme.c
index b712634c27..4059bd72e6 100644
--- a/hw/block/nvme.c
+++ b/hw/block/nvme.c
@@ -1148,6 +1148,8 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
                 trace_pci_nvme_err_append_not_at_start(slba, zone->d.zslba);
                 status = NVME_INVALID_FIELD;
             } else if (unlikely(zone->w_ptr + nlb > bndry)) {
+                trace_pci_nvme_err_append_across_boundary(slba, nlb,
+                                                          zone->w_ptr, bndry);
                 status = NVME_ZONE_BOUNDARY_ERROR;
             } else if (nvme_l2b(ns, nlb) > (n->page_size << n->zasl)) {
                 trace_pci_nvme_err_append_too_large(slba, nlb, n->zasl);
@@ -1159,6 +1161,7 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
                                                    zone->w_ptr);
                 status = NVME_ZONE_INVALID_WRITE;
             } else if (unlikely(slba + nlb > bndry)) {
+                trace_pci_nvme_err_write_across_boundary(slba, nlb, bndry);
                 status = NVME_ZONE_BOUNDARY_ERROR;
             }
         }
@@ -1200,9 +1203,10 @@ static uint16_t nvme_check_zone_read(NvmeNamespace *ns, uint64_t slba,
 
     status = nvme_check_zone_state_for_read(zone);
     if (status != NVME_SUCCESS) {
-        ;
+        trace_pci_nvme_err_zone_read_not_ok(slba, nlb, status);
     } else if (unlikely(end > bndry)) {
         if (!ns->params.cross_zone_read) {
+            trace_pci_nvme_err_read_across_boundary(slba, nlb, bndry);
             status = NVME_ZONE_BOUNDARY_ERROR;
         } else {
             /*
@@ -1213,6 +1217,7 @@ static uint16_t nvme_check_zone_read(NvmeNamespace *ns, uint64_t slba,
                 zone++;
                 status = nvme_check_zone_state_for_read(zone);
                 if (status != NVME_SUCCESS) {
+                    trace_pci_nvme_err_zone_read_not_ok(slba, nlb, status);
                     break;
                 }
             } while (end > nvme_zone_rd_boundary(ns, zone));
@@ -1624,7 +1629,6 @@ static uint16_t nvme_read(NvmeCtrl *n, NvmeRequest *req)
     if (ns->params.zoned) {
         status = nvme_check_zone_read(ns, slba, nlb);
         if (status != NVME_SUCCESS) {
-            trace_pci_nvme_err_zone_read_not_ok(slba, nlb, status);
             goto invalid;
         }
     }
diff --git a/hw/block/trace-events b/hw/block/trace-events
index 955ee4860f..18bbe560fc 100644
--- a/hw/block/trace-events
+++ b/hw/block/trace-events
@@ -127,6 +127,9 @@ pci_nvme_err_unaligned_zone_cmd(uint8_t action, uint64_t slba, uint64_t zslba) "
 pci_nvme_err_invalid_zone_state_transition(uint8_t action, uint64_t slba, uint8_t attrs) "action=0x%"PRIx8", slba=%"PRIu64", attrs=0x%"PRIx32""
 pci_nvme_err_write_not_at_wp(uint64_t slba, uint64_t zone, uint64_t wp) "writing at slba=%"PRIu64", zone=%"PRIu64", but wp=%"PRIu64""
 pci_nvme_err_append_not_at_start(uint64_t slba, uint64_t zone) "appending at slba=%"PRIu64", but zone=%"PRIu64""
+pci_nvme_err_append_across_boundary(uint64_t slba, uint32_t nlb, uint64_t wp, uint64_t bndry) "slba=%"PRIu64", nlb=%"PRIu32", w_ptr=%"PRIu64", bndry=%"PRIu64""
+pci_nvme_err_write_across_boundary(uint64_t slba, uint32_t nlb, uint64_t bndry) "slba=%"PRIu64", nlb=%"PRIu32", bndry=%"PRIu64""
+pci_nvme_err_read_across_boundary(uint64_t slba, uint32_t nlb, uint64_t bndry) "slba=%"PRIu64", nlb=%"PRIu32", bndry=%"PRIu64""
 pci_nvme_err_zone_write_not_ok(uint64_t slba, uint32_t nlb, uint32_t status) "slba=%"PRIu64", nlb=%"PRIu32", status=0x%"PRIx16""
 pci_nvme_err_zone_read_not_ok(uint64_t slba, uint32_t nlb, uint32_t status) "slba=%"PRIu64", nlb=%"PRIu32", status=0x%"PRIx16""
 pci_nvme_err_append_too_large(uint64_t slba, uint32_t nlb, uint8_t zasl) "slba=%"PRIu64", nlb=%"PRIu32", zasl=%"PRIu8""
-- 
2.28.0

Re: [PATCH 1/3] hw/block/nvme: Check for zone boundary during append

[Klaus Jensen]

[-- Attachment #1: Type: text/plain, Size: 2032 bytes --]

On Jan 26 14:02, Dmitry Fomichev wrote:
> It is observed that with the existing code it is possible to keep
> appending to a zone indefinitely. To fix, add the missing check to
> verify that the zone append is not going to write beyond zone capacity.
> 
> Reported-by: Niklas Cassel <niklas.cassel@wdc.com>
> Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
> ---
>  hw/block/nvme.c | 8 +++++---
>  1 file changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/hw/block/nvme.c b/hw/block/nvme.c
> index f64676a930..67538010ef 100644
> --- a/hw/block/nvme.c
> +++ b/hw/block/nvme.c
> @@ -1135,9 +1135,10 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
>                                        NvmeZone *zone, uint64_t slba,
>                                        uint32_t nlb, bool append)
>  {
> +    uint64_t bndry = nvme_zone_wr_boundary(zone);
>      uint16_t status;
>  
> -    if (unlikely((slba + nlb) > nvme_zone_wr_boundary(zone))) {
> +    if (unlikely(slba + nlb > bndry)) {
>          status = NVME_ZONE_BOUNDARY_ERROR;
>      } else {
>          status = nvme_check_zone_state_for_write(zone);
> @@ -1151,8 +1152,9 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
>              if (unlikely(slba != zone->d.zslba)) {
>                  trace_pci_nvme_err_append_not_at_start(slba, zone->d.zslba);
>                  status = NVME_INVALID_FIELD;
> -            }
> -            if (nvme_l2b(ns, nlb) > (n->page_size << n->zasl)) {
> +            } else if (unlikely(zone->w_ptr + nlb > bndry)) {
> +                status = NVME_ZONE_BOUNDARY_ERROR;

Now, for appends, you are just checking the boundary condition twice.
And the first one will be moot for appends anyway.

> +            } else if (nvme_l2b(ns, nlb) > (n->page_size << n->zasl)) {
>                  trace_pci_nvme_err_append_too_large(slba, nlb, n->zasl);
>                  status = NVME_INVALID_FIELD;
>              }
> -- 
> 2.28.0
> 
> 

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 2/3] hw/block/nvme: Check zone state before checking boundaries

[Klaus Jensen]

[-- Attachment #1: Type: text/plain, Size: 2690 bytes --]

On Jan 26 14:02, Dmitry Fomichev wrote:
> The code in nvme_check_zone_write() first checks for zone boundary
> condition violation and only after that it proceeds to verify that the
> zone state is suitable the write to happen. This is not consistent with
> nvme_check_zone_read() flow - in that function, zone state is checked
> before making any boundary checks. Besides, checking that ZSLBA + NLB
> does not cross the write boundary is now redundant for Zone Append and
> only needs to be done for writes.
> 
> Move the check in the code to be performed for Write and Write Zeroes
> commands only and to occur after zone state checks.
> 
> Signed-off-by: Dmitry Fomichev <dmitry.fomichev@wdc.com>
> ---
>  hw/block/nvme.c | 21 ++++++++++-----------
>  1 file changed, 10 insertions(+), 11 deletions(-)
> 
> diff --git a/hw/block/nvme.c b/hw/block/nvme.c
> index 67538010ef..b712634c27 100644
> --- a/hw/block/nvme.c
> +++ b/hw/block/nvme.c
> @@ -1138,13 +1138,8 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
>      uint64_t bndry = nvme_zone_wr_boundary(zone);
>      uint16_t status;
>  
> -    if (unlikely(slba + nlb > bndry)) {
> -        status = NVME_ZONE_BOUNDARY_ERROR;
> -    } else {
> -        status = nvme_check_zone_state_for_write(zone);
> -    }

Alright. The double check on boundary that I pointed out in the previous
patch is fixed here.

> -
> -    if (status != NVME_SUCCESS) {
> +    status = nvme_check_zone_state_for_write(zone);
> +    if (status) {
>          trace_pci_nvme_err_zone_write_not_ok(slba, nlb, status);
>      } else {
>          assert(nvme_wp_is_valid(zone));
> @@ -1158,10 +1153,14 @@ static uint16_t nvme_check_zone_write(NvmeCtrl *n, NvmeNamespace *ns,
>                  trace_pci_nvme_err_append_too_large(slba, nlb, n->zasl);
>                  status = NVME_INVALID_FIELD;
>              }
> -        } else if (unlikely(slba != zone->w_ptr)) {
> -            trace_pci_nvme_err_write_not_at_wp(slba, zone->d.zslba,
> -                                               zone->w_ptr);
> -            status = NVME_ZONE_INVALID_WRITE;
> +        } else {
> +            if (unlikely(slba != zone->w_ptr)) {
> +                trace_pci_nvme_err_write_not_at_wp(slba, zone->d.zslba,
> +                                                   zone->w_ptr);
> +                status = NVME_ZONE_INVALID_WRITE;
> +            } else if (unlikely(slba + nlb > bndry)) {
> +                status = NVME_ZONE_BOUNDARY_ERROR;
> +            }
>          }
>      }
>  
> -- 
> 2.28.0
> 
> 

-- 
One of us - No more doubt, silence or taboo about mental illness.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 0/3] Fix zone write validation

[Klaus Jensen]

[-- Attachment #1: Type: text/plain, Size: 1048 bytes --]

On Jan 26 14:02, Dmitry Fomichev wrote:
> These patches solve a few problems that exist in zoned Write
> ans Zone Append validation code.
> 
> Dmitry Fomichev (3):
>   hw/block/nvme: Check for zone boundary during append
>   hw/block/nvme: Check zone state before checking boundaries
>   hw/block/nvme: Add trace events for zone boundary violations
> 
>  hw/block/nvme.c       | 35 ++++++++++++++++++++---------------
>  hw/block/trace-events |  3 +++
>  2 files changed, 23 insertions(+), 15 deletions(-)
> 

I don't think there are any obvious benefits to this series over my fix
and since you didn't identify any functional issues with it, I'm
thinking we go with that.

My fix additionally removes setting ALBA in the CQE for regular writes
and bundles the endianness fix by changing the related logic in
do_write.

I have a couple of series in queue that also includes zoned writes and
there is no reason they have to indirectly deal with append. It's just
cleaner to move this special case closer to where it's used.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 0/3] Fix zone write validation

[Klaus Jensen]

[-- Attachment #1: Type: text/plain, Size: 1209 bytes --]

On Jan 26 09:21, Klaus Jensen wrote:
> On Jan 26 14:02, Dmitry Fomichev wrote:
> > These patches solve a few problems that exist in zoned Write
> > ans Zone Append validation code.
> > 
> > Dmitry Fomichev (3):
> >   hw/block/nvme: Check for zone boundary during append
> >   hw/block/nvme: Check zone state before checking boundaries
> >   hw/block/nvme: Add trace events for zone boundary violations
> > 
> >  hw/block/nvme.c       | 35 ++++++++++++++++++++---------------
> >  hw/block/trace-events |  3 +++
> >  2 files changed, 23 insertions(+), 15 deletions(-)
> > 
> 
> I don't think there are any obvious benefits to this series over my fix
> and since you didn't identify any functional issues with it, I'm
> thinking we go with that.
> 
> My fix additionally removes setting ALBA in the CQE for regular writes
> and bundles the endianness fix by changing the related logic in
> do_write.
> 
> I have a couple of series in queue that also includes zoned writes and
> there is no reason they have to indirectly deal with append. It's just
> cleaner to move this special case closer to where it's used.

Keith,

I think this calls for your +1 tiebreaking special ability.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: [PATCH 0/3] Fix zone write validation

[Keith Busch]

On Tue, Jan 26, 2021 at 09:40:44AM +0100, Klaus Jensen wrote:
> On Jan 26 09:21, Klaus Jensen wrote:
> > On Jan 26 14:02, Dmitry Fomichev wrote:
> > > These patches solve a few problems that exist in zoned Write
> > > ans Zone Append validation code.
> > > 
> > > Dmitry Fomichev (3):
> > >   hw/block/nvme: Check for zone boundary during append
> > >   hw/block/nvme: Check zone state before checking boundaries
> > >   hw/block/nvme: Add trace events for zone boundary violations
> > > 
> > >  hw/block/nvme.c       | 35 ++++++++++++++++++++---------------
> > >  hw/block/trace-events |  3 +++
> > >  2 files changed, 23 insertions(+), 15 deletions(-)
> > > 
> > 
> > I don't think there are any obvious benefits to this series over my fix
> > and since you didn't identify any functional issues with it, I'm
> > thinking we go with that.
> > 
> > My fix additionally removes setting ALBA in the CQE for regular writes
> > and bundles the endianness fix by changing the related logic in
> > do_write.
> > 
> > I have a couple of series in queue that also includes zoned writes and
> > there is no reason they have to indirectly deal with append. It's just
> > cleaner to move this special case closer to where it's used.
> 
> Keith,
> 
> I think this calls for your +1 tiebreaking special ability.

Klaus,

Your series arrived very timely, and since it looks fine (unless there
are outstanding objections?), I think it's fair to apply your series.
Subsequent changes can rebase to that.