[PATCHv3 0/4] zoned block device specific errors

[PATCHv3 0/4] zoned block device specific errors

[Keith Busch]

Zone block devices may have some limits that require special handling.
This series provides a way for drivers to notify of these conditions.

v2->v3:

  Split status for max open vs max active errors (Christoph)

  Include scsi use for the new status (Damien)

  Update documentation for the new status inline with the request_queue
  properties they relate to

Damien Le Moal (2):
  scsi: update additional sense codes list
  scsi: handle zone resources errors

Keith Busch (2):
  block: add zone specific block statuses
  nvme: translate zone resource errors

 Documentation/block/queue-sysfs.rst |  8 +++++
 block/blk-core.c                    |  4 +++
 drivers/nvme/host/core.c            |  4 +++
 drivers/scsi/scsi_lib.c             |  9 +++++
 drivers/scsi/sense_codes.h          | 54 ++++++++++++++++++++++++++++-
 include/linux/blk_types.h           | 18 ++++++++++
 6 files changed, 96 insertions(+), 1 deletion(-)

-- 
2.24.1

[PATCHv3 1/4] block: add zone specific block statuses

[Keith Busch]

A zoned device with limited resources to open or activate zones may
return an error when the host exceeds those limits. The same command may
be successful if retried later, but the host needs to wait for specific
zone states before it should retry. Have the block layer provide an
appropriate status for these conditions so applications can distinuguish
this error for special handling.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Damien Le Moal <Damien.LeMoal@wdc.com>
Cc: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
 Documentation/block/queue-sysfs.rst |  8 ++++++++
 block/blk-core.c                    |  4 ++++
 include/linux/blk_types.h           | 18 ++++++++++++++++++
 3 files changed, 30 insertions(+)

diff --git a/Documentation/block/queue-sysfs.rst b/Documentation/block/queue-sysfs.rst
index f261a5c84170..2638d3446b79 100644
--- a/Documentation/block/queue-sysfs.rst
+++ b/Documentation/block/queue-sysfs.rst
@@ -124,6 +124,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
 EXPLICIT OPEN, IMPLICIT OPEN or CLOSED, is limited by this value.
 If this value is 0, there is no limit.
 
+If the host attempts to exceed this limit, the driver should report this error
+with BLK_STS_ZONE_ACTIVE_RESOURCE, which user space may see as the EOVERFLOW
+errno.
+
 max_open_zones (RO)
 -------------------
 For zoned block devices (zoned attribute indicating "host-managed" or
@@ -131,6 +135,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
 EXPLICIT OPEN or IMPLICIT OPEN, is limited by this value.
 If this value is 0, there is no limit.
 
+If the host attempts to exceed this limit, the driver should report this error
+with BLK_STS_ZONE_OPEN_RESOURCE, which user space may see as the ETOOMANYREFS
+errno.
+
 max_sectors_kb (RW)
 -------------------
 This is the maximum number of kilobytes that the block layer will allow
diff --git a/block/blk-core.c b/block/blk-core.c
index 10c08ac50697..8bffc7732e37 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -186,6 +186,10 @@ static const struct {
 	/* device mapper special case, should not leak out: */
 	[BLK_STS_DM_REQUEUE]	= { -EREMCHG, "dm internal retry" },
 
+	/* zone device specific errors */
+	[BLK_STS_ZONE_OPEN_RESOURCE]	= { -ETOOMANYREFS, "open zones exceeded" },
+	[BLK_STS_ZONE_ACTIVE_RESOURCE]	= { -EOVERFLOW, "active zones exceeded" },
+
 	/* everything else not covered above: */
 	[BLK_STS_IOERR]		= { -EIO,	"I/O" },
 };
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 4ecf4fed171f..8603fc5f86a3 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -103,6 +103,24 @@ typedef u8 __bitwise blk_status_t;
  */
 #define BLK_STS_ZONE_RESOURCE	((__force blk_status_t)14)
 
+/*
+ * BLK_STS_ZONE_OPEN_RESOURCE is returned from the driver in the completion
+ * path if the device returns a status indicating that too many zone resources
+ * are currently open. The same command should be successful if resubmitted
+ * after the number of open zones decreases below the device's limits, which is
+ * reported in the request_queue's max_open_zones.
+ */
+#define BLK_STS_ZONE_OPEN_RESOURCE	((__force blk_status_t)15)
+
+/*
+ * BLK_STS_ZONE_ACTIVE_RESOURCE is returned from the driver in the completion
+ * path if the device returns a status indicating that too many zone resources
+ * are currently active. The same command should be successful if resubmitted
+ * after the number of active zones decreases below the device's limits, which
+ * is reported in the request_queue's max_active_zones.
+ */
+#define BLK_STS_ZONE_ACTIVE_RESOURCE	((__force blk_status_t)16)
+
 /**
  * blk_path_error - returns true if error may be path related
  * @error: status the request was completed with
-- 
2.24.1

Re: [PATCHv3 1/4] block: add zone specific block statuses

[Damien Le Moal]

On 2020/09/18 8:18, Keith Busch wrote:
> A zoned device with limited resources to open or activate zones may
> return an error when the host exceeds those limits. The same command may
> be successful if retried later, but the host needs to wait for specific
> zone states before it should retry. Have the block layer provide an
> appropriate status for these conditions so applications can distinuguish
> this error for special handling.
> 
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Damien Le Moal <Damien.LeMoal@wdc.com>
> Cc: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
> Signed-off-by: Keith Busch <kbusch@kernel.org>
> ---
>  Documentation/block/queue-sysfs.rst |  8 ++++++++
>  block/blk-core.c                    |  4 ++++
>  include/linux/blk_types.h           | 18 ++++++++++++++++++
>  3 files changed, 30 insertions(+)
> 
> diff --git a/Documentation/block/queue-sysfs.rst b/Documentation/block/queue-sysfs.rst
> index f261a5c84170..2638d3446b79 100644
> --- a/Documentation/block/queue-sysfs.rst
> +++ b/Documentation/block/queue-sysfs.rst
> @@ -124,6 +124,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
>  EXPLICIT OPEN, IMPLICIT OPEN or CLOSED, is limited by this value.
>  If this value is 0, there is no limit.
>  
> +If the host attempts to exceed this limit, the driver should report this error
> +with BLK_STS_ZONE_ACTIVE_RESOURCE, which user space may see as the EOVERFLOW
> +errno.
> +
>  max_open_zones (RO)
>  -------------------
>  For zoned block devices (zoned attribute indicating "host-managed" or
> @@ -131,6 +135,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
>  EXPLICIT OPEN or IMPLICIT OPEN, is limited by this value.
>  If this value is 0, there is no limit.
>  
> +If the host attempts to exceed this limit, the driver should report this error
> +with BLK_STS_ZONE_OPEN_RESOURCE, which user space may see as the ETOOMANYREFS
> +errno.
> +
>  max_sectors_kb (RW)
>  -------------------
>  This is the maximum number of kilobytes that the block layer will allow
> diff --git a/block/blk-core.c b/block/blk-core.c
> index 10c08ac50697..8bffc7732e37 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -186,6 +186,10 @@ static const struct {
>  	/* device mapper special case, should not leak out: */
>  	[BLK_STS_DM_REQUEUE]	= { -EREMCHG, "dm internal retry" },
>  
> +	/* zone device specific errors */
> +	[BLK_STS_ZONE_OPEN_RESOURCE]	= { -ETOOMANYREFS, "open zones exceeded" },
> +	[BLK_STS_ZONE_ACTIVE_RESOURCE]	= { -EOVERFLOW, "active zones exceeded" },
> +
>  	/* everything else not covered above: */
>  	[BLK_STS_IOERR]		= { -EIO,	"I/O" },
>  };
> diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
> index 4ecf4fed171f..8603fc5f86a3 100644
> --- a/include/linux/blk_types.h
> +++ b/include/linux/blk_types.h
> @@ -103,6 +103,24 @@ typedef u8 __bitwise blk_status_t;
>   */
>  #define BLK_STS_ZONE_RESOURCE	((__force blk_status_t)14)
>  
> +/*
> + * BLK_STS_ZONE_OPEN_RESOURCE is returned from the driver in the completion
> + * path if the device returns a status indicating that too many zone resources
> + * are currently open. The same command should be successful if resubmitted
> + * after the number of open zones decreases below the device's limits, which is
> + * reported in the request_queue's max_open_zones.
> + */
> +#define BLK_STS_ZONE_OPEN_RESOURCE	((__force blk_status_t)15)
> +
> +/*
> + * BLK_STS_ZONE_ACTIVE_RESOURCE is returned from the driver in the completion
> + * path if the device returns a status indicating that too many zone resources
> + * are currently active. The same command should be successful if resubmitted
> + * after the number of active zones decreases below the device's limits, which
> + * is reported in the request_queue's max_active_zones.
> + */
> +#define BLK_STS_ZONE_ACTIVE_RESOURCE	((__force blk_status_t)16)
> +
>  /**
>   * blk_path_error - returns true if error may be path related
>   * @error: status the request was completed with
> 

Looks good.
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>


-- 
Damien Le Moal
Western Digital Research

Re: [PATCHv3 1/4] block: add zone specific block statuses

[Johannes Thumshirn]

On 18/09/2020 01:18, Keith Busch wrote:
> diff --git a/Documentation/block/queue-sysfs.rst b/Documentation/block/queue-sysfs.rst
> index f261a5c84170..2638d3446b79 100644
> --- a/Documentation/block/queue-sysfs.rst
> +++ b/Documentation/block/queue-sysfs.rst
> @@ -124,6 +124,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
>  EXPLICIT OPEN, IMPLICIT OPEN or CLOSED, is limited by this value.
>  If this value is 0, there is no limit.
>  
> +If the host attempts to exceed this limit, the driver should report this error
> +with BLK_STS_ZONE_ACTIVE_RESOURCE, which user space may see as the EOVERFLOW
> +errno.
> +
>  max_open_zones (RO)
>  -------------------
>  For zoned block devices (zoned attribute indicating "host-managed" or
> @@ -131,6 +135,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
>  EXPLICIT OPEN or IMPLICIT OPEN, is limited by this value.
>  If this value is 0, there is no limit.
>  
> +If the host attempts to exceed this limit, the driver should report this error
> +with BLK_STS_ZONE_OPEN_RESOURCE, which user space may see as the ETOOMANYREFS
> +errno.

Don't we also need to update some man pages in section 2?

Code wise,
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>

Re: [PATCHv3 1/4] block: add zone specific block statuses

[Keith Busch]

On Fri, Sep 18, 2020 at 01:31:44PM +0000, Johannes Thumshirn wrote:
> On 18/09/2020 01:18, Keith Busch wrote:
> > diff --git a/Documentation/block/queue-sysfs.rst b/Documentation/block/queue-sysfs.rst
> > index f261a5c84170..2638d3446b79 100644
> > --- a/Documentation/block/queue-sysfs.rst
> > +++ b/Documentation/block/queue-sysfs.rst
> > @@ -124,6 +124,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
> >  EXPLICIT OPEN, IMPLICIT OPEN or CLOSED, is limited by this value.
> >  If this value is 0, there is no limit.
> >  
> > +If the host attempts to exceed this limit, the driver should report this error
> > +with BLK_STS_ZONE_ACTIVE_RESOURCE, which user space may see as the EOVERFLOW
> > +errno.
> > +
> >  max_open_zones (RO)
> >  -------------------
> >  For zoned block devices (zoned attribute indicating "host-managed" or
> > @@ -131,6 +135,10 @@ For zoned block devices (zoned attribute indicating "host-managed" or
> >  EXPLICIT OPEN or IMPLICIT OPEN, is limited by this value.
> >  If this value is 0, there is no limit.
> >  
> > +If the host attempts to exceed this limit, the driver should report this error
> > +with BLK_STS_ZONE_OPEN_RESOURCE, which user space may see as the ETOOMANYREFS
> > +errno.
> 
> Don't we also need to update some man pages in section 2?

Yes, good point. Those updates need to come from this repo

  https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git

right? If so, I can send updates there once it looks like this is the
form that will be committed.

Re: [PATCHv3 1/4] block: add zone specific block statuses

[Johannes Thumshirn]

On 18/09/2020 21:41, Keith Busch wrote:
[...]
> 
> Yes, good point. Those updates need to come from this repo
> 
>   https://git.kernel.org/pub/scm/docs/man-pages/man-pages.git
> 
> right? If so, I can send updates there once it looks like this is the
> form that will be committed.
> 

I think so yes. Should we also Cc linux-api on this matter?

[PATCHv3 2/4] nvme: translate zone resource errors

[Keith Busch]

Translate zoned resource errors to the appropriate blk_status_t.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Damien Le Moal <Damien.LeMoal@wdc.com>
Cc: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
Signed-off-by: Keith Busch <kbusch@kernel.org>
---
 drivers/nvme/host/core.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index 614cd455836b..a0d26fcbf923 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -236,6 +236,10 @@ static blk_status_t nvme_error_status(u16 status)
 		return BLK_STS_NEXUS;
 	case NVME_SC_HOST_PATH_ERROR:
 		return BLK_STS_TRANSPORT;
+	case NVME_SC_ZONE_TOO_MANY_ACTIVE:
+		return BLK_STS_ZONE_ACTIVE_RESOURCE;
+	case NVME_SC_ZONE_TOO_MANY_OPEN:
+		return BLK_STS_ZONE_OPEN_RESOURCE;
 	default:
 		return BLK_STS_IOERR;
 	}
-- 
2.24.1

Re: [PATCHv3 2/4] nvme: translate zone resource errors

[Damien Le Moal]

On 2020/09/18 8:18, Keith Busch wrote:
> Translate zoned resource errors to the appropriate blk_status_t.
> 
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Damien Le Moal <Damien.LeMoal@wdc.com>
> Cc: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
> Signed-off-by: Keith Busch <kbusch@kernel.org>
> ---
>  drivers/nvme/host/core.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> index 614cd455836b..a0d26fcbf923 100644
> --- a/drivers/nvme/host/core.c
> +++ b/drivers/nvme/host/core.c
> @@ -236,6 +236,10 @@ static blk_status_t nvme_error_status(u16 status)
>  		return BLK_STS_NEXUS;
>  	case NVME_SC_HOST_PATH_ERROR:
>  		return BLK_STS_TRANSPORT;
> +	case NVME_SC_ZONE_TOO_MANY_ACTIVE:
> +		return BLK_STS_ZONE_ACTIVE_RESOURCE;
> +	case NVME_SC_ZONE_TOO_MANY_OPEN:
> +		return BLK_STS_ZONE_OPEN_RESOURCE;
>  	default:
>  		return BLK_STS_IOERR;
>  	}
> 

Looks good.
Reviewed-by: Damien Le Moal <damien.lemoal@wdc.com>

-- 
Damien Le Moal
Western Digital Research

Re: [PATCHv3 2/4] nvme: translate zone resource errors

[Johannes Thumshirn]

Easy enough,
Reviewed-by: Johannes Thumshirn <johannes.thumshirn@wdc.com>

[PATCHv3 3/4] scsi: update additional sense codes list

[Keith Busch]

From: Damien Le Moal <damien.lemoal@wdc.com>

Add missing Additional Sense Codes listed in
http://www.t10.org/lists/asc-num.txt.

Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
---
 drivers/scsi/sense_codes.h | 54 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 53 insertions(+), 1 deletion(-)

diff --git a/drivers/scsi/sense_codes.h b/drivers/scsi/sense_codes.h
index 201a536688de..805d4c13d070 100644
--- a/drivers/scsi/sense_codes.h
+++ b/drivers/scsi/sense_codes.h
@@ -1,7 +1,7 @@
 /* SPDX-License-Identifier: GPL-2.0 */
 /*
  * The canonical list of T10 Additional Sense Codes is available at:
- * http://www.t10.org/lists/asc-num.txt [most recent: 20141221]
+ * http://www.t10.org/lists/asc-num.txt [most recent: 20200817]
  */
 
 SENSE_CODE(0x0000, "No additional sense information")
@@ -29,6 +29,7 @@ SENSE_CODE(0x001E, "Conflicting SA creation request")
 SENSE_CODE(0x001F, "Logical unit transitioning to another power condition")
 SENSE_CODE(0x0020, "Extended copy information available")
 SENSE_CODE(0x0021, "Atomic command aborted due to ACA")
+SENSE_CODE(0x0022, "Deferred microcode is pending")
 
 SENSE_CODE(0x0100, "No index/sector signal")
 
@@ -72,6 +73,9 @@ SENSE_CODE(0x041F, "Logical unit not ready, microcode download required")
 SENSE_CODE(0x0420, "Logical unit not ready, logical unit reset required")
 SENSE_CODE(0x0421, "Logical unit not ready, hard reset required")
 SENSE_CODE(0x0422, "Logical unit not ready, power cycle required")
+SENSE_CODE(0x0423, "Logical unit not ready, affiliation required")
+SENSE_CODE(0x0424, "Depopulation in progress")
+SENSE_CODE(0x0425, "Depopulation restoration in progress")
 
 SENSE_CODE(0x0500, "Logical unit does not respond to selection")
 
@@ -104,6 +108,17 @@ SENSE_CODE(0x0B06, "Warning - non-volatile cache now volatile")
 SENSE_CODE(0x0B07, "Warning - degraded power to non-volatile cache")
 SENSE_CODE(0x0B08, "Warning - power loss expected")
 SENSE_CODE(0x0B09, "Warning - device statistics notification active")
+SENSE_CODE(0x0B0A, "Warning - high critical temperature limit exceeded")
+SENSE_CODE(0x0B0B, "Warning - low critical temperature limit exceeded")
+SENSE_CODE(0x0B0C, "Warning - high operating temperature limit exceeded")
+SENSE_CODE(0x0B0D, "Warning - low operating temperature limit exceeded")
+SENSE_CODE(0x0B0E, "Warning - high critical humidity limit exceeded")
+SENSE_CODE(0x0B0F, "Warning - low critical humidity limit exceeded")
+SENSE_CODE(0x0B10, "Warning - high operating humidity limit exceeded")
+SENSE_CODE(0x0B11, "Warning - low operating humidity limit exceeded")
+SENSE_CODE(0x0B12, "Warning - microcode security at risk")
+SENSE_CODE(0x0B13, "Warning - microcode digital signature validation failure")
+SENSE_CODE(0x0B14, "Warning - physical element status change")
 
 SENSE_CODE(0x0C00, "Write error")
 SENSE_CODE(0x0C01, "Write error - recovered with auto reallocation")
@@ -122,6 +137,8 @@ SENSE_CODE(0x0C0D, "Write error - not enough unsolicited data")
 SENSE_CODE(0x0C0E, "Multiple write errors")
 SENSE_CODE(0x0C0F, "Defects in error window")
 SENSE_CODE(0x0C10, "Incomplete multiple atomic write operations")
+SENSE_CODE(0x0C11, "Write error - recovery scan needed")
+SENSE_CODE(0x0C12, "Write error - insufficient zone resources")
 
 SENSE_CODE(0x0D00, "Error detected by third party temporary initiator")
 SENSE_CODE(0x0D01, "Third party device failure")
@@ -242,6 +259,9 @@ SENSE_CODE(0x2009, "Access denied - invalid LU identifier")
 SENSE_CODE(0x200A, "Access denied - invalid proxy token")
 SENSE_CODE(0x200B, "Access denied - ACL LUN conflict")
 SENSE_CODE(0x200C, "Illegal command when not in append-only mode")
+SENSE_CODE(0x200D, "Not an administrative logical unit")
+SENSE_CODE(0x200E, "Not a subsidiary logical unit")
+SENSE_CODE(0x200F, "Not a conglomerate logical unit")
 
 SENSE_CODE(0x2100, "Logical block address out of range")
 SENSE_CODE(0x2101, "Invalid element address")
@@ -251,6 +271,8 @@ SENSE_CODE(0x2104, "Unaligned write command")
 SENSE_CODE(0x2105, "Write boundary violation")
 SENSE_CODE(0x2106, "Attempt to read invalid data")
 SENSE_CODE(0x2107, "Read boundary violation")
+SENSE_CODE(0x2108, "Misaligned write command")
+SENSE_CODE(0x2109, "Attempt to access gap zone")
 
 SENSE_CODE(0x2200, "Illegal function (use 20 00, 24 00, or 26 00)")
 
@@ -275,6 +297,7 @@ SENSE_CODE(0x2405, "Security working key frozen")
 SENSE_CODE(0x2406, "Nonce not unique")
 SENSE_CODE(0x2407, "Nonce timestamp out of range")
 SENSE_CODE(0x2408, "Invalid XCDB")
+SENSE_CODE(0x2409, "Invalid fast format")
 
 SENSE_CODE(0x2500, "Logical unit not supported")
 
@@ -297,6 +320,10 @@ SENSE_CODE(0x260F, "Invalid data-out buffer integrity check value")
 SENSE_CODE(0x2610, "Data decryption key fail limit reached")
 SENSE_CODE(0x2611, "Incomplete key-associated data set")
 SENSE_CODE(0x2612, "Vendor specific key reference not found")
+SENSE_CODE(0x2613, "Application tag mode page is invalid")
+SENSE_CODE(0x2614, "Tape stream mirroring prevented")
+SENSE_CODE(0x2615, "Copy source or copy destination not authorized")
+SENSE_CODE(0x2616, "Fast copy not possible")
 
 SENSE_CODE(0x2700, "Write protected")
 SENSE_CODE(0x2701, "Hardware write protected")
@@ -342,6 +369,7 @@ SENSE_CODE(0x2A12, "Data encryption parameters changed by vendor specific event"
 SENSE_CODE(0x2A13, "Data encryption key instance counter has changed")
 SENSE_CODE(0x2A14, "SA creation capabilities data has changed")
 SENSE_CODE(0x2A15, "Medium removal prevention preempted")
+SENSE_CODE(0x2A16, "Zone reset write pointer recommended")
 
 SENSE_CODE(0x2B00, "Copy cannot execute since host cannot disconnect")
 
@@ -360,6 +388,11 @@ SENSE_CODE(0x2C0B, "Not reserved")
 SENSE_CODE(0x2C0C, "Orwrite generation does not match")
 SENSE_CODE(0x2C0D, "Reset write pointer not allowed")
 SENSE_CODE(0x2C0E, "Zone is offline")
+SENSE_CODE(0x2C0F, "Stream not open")
+SENSE_CODE(0x2C10, "Unwritten data in zone")
+SENSE_CODE(0x2C11, "Descriptor format sense data required")
+SENSE_CODE(0x2C12, "Zone is inactive")
+SENSE_CODE(0x2C13, "Well known logical unit access required")
 
 SENSE_CODE(0x2D00, "Overwrite error on update in place")
 
@@ -395,6 +428,8 @@ SENSE_CODE(0x3100, "Medium format corrupted")
 SENSE_CODE(0x3101, "Format command failed")
 SENSE_CODE(0x3102, "Zoned formatting failed due to spare linking")
 SENSE_CODE(0x3103, "Sanitize command failed")
+SENSE_CODE(0x3104, "Depopulation failed")
+SENSE_CODE(0x3105, "Depopulation restoration failed")
 
 SENSE_CODE(0x3200, "No defect spare location available")
 SENSE_CODE(0x3201, "Defect list update failure")
@@ -419,6 +454,7 @@ SENSE_CODE(0x3802, "Esn - power management class event")
 SENSE_CODE(0x3804, "Esn - media class event")
 SENSE_CODE(0x3806, "Esn - device busy class event")
 SENSE_CODE(0x3807, "Thin Provisioning soft threshold reached")
+SENSE_CODE(0x3808, "Depopulation interrupted")
 
 SENSE_CODE(0x3900, "Saving parameters not supported")
 
@@ -456,6 +492,7 @@ SENSE_CODE(0x3B19, "Element enabled")
 SENSE_CODE(0x3B1A, "Data transfer device removed")
 SENSE_CODE(0x3B1B, "Data transfer device inserted")
 SENSE_CODE(0x3B1C, "Too many logical objects on partition to support operation")
+SENSE_CODE(0x3B20, "Element static information changed")
 
 SENSE_CODE(0x3D00, "Invalid bits in identify message")
 
@@ -488,6 +525,11 @@ SENSE_CODE(0x3F13, "iSCSI IP address removed")
 SENSE_CODE(0x3F14, "iSCSI IP address changed")
 SENSE_CODE(0x3F15, "Inspect referrals sense descriptors")
 SENSE_CODE(0x3F16, "Microcode has been changed without reset")
+SENSE_CODE(0x3F17, "Zone transition to full")
+SENSE_CODE(0x3F18, "Bind completed")
+SENSE_CODE(0x3F19, "Bind redirected")
+SENSE_CODE(0x3F1A, "Subsidiary binding changed")
+
 /*
  *	SENSE_CODE(0x40NN, "Ram failure")
  *	SENSE_CODE(0x40NN, "Diagnostic failure on component nn")
@@ -589,6 +631,9 @@ SENSE_CODE(0x550B, "Insufficient power for operation")
 SENSE_CODE(0x550C, "Insufficient resources to create rod")
 SENSE_CODE(0x550D, "Insufficient resources to create rod token")
 SENSE_CODE(0x550E, "Insufficient zone resources")
+SENSE_CODE(0x550F, "Insufficient zone resources to complete write")
+SENSE_CODE(0x5510, "Maximum number of streams open")
+SENSE_CODE(0x5511, "Insufficient resources to bind")
 
 SENSE_CODE(0x5700, "Unable to recover table-of-contents")
 
@@ -692,6 +737,7 @@ SENSE_CODE(0x5D69, "Firmware impending failure throughput performance")
 SENSE_CODE(0x5D6A, "Firmware impending failure seek time performance")
 SENSE_CODE(0x5D6B, "Firmware impending failure spin-up retry count")
 SENSE_CODE(0x5D6C, "Firmware impending failure drive calibration retry count")
+SENSE_CODE(0x5D73, "Media impending failure endurance limit met")
 SENSE_CODE(0x5DFF, "Failure prediction threshold exceeded (false)")
 
 SENSE_CODE(0x5E00, "Low power condition on")
@@ -744,6 +790,8 @@ SENSE_CODE(0x6708, "Assign failure occurred")
 SENSE_CODE(0x6709, "Multiply assigned logical unit")
 SENSE_CODE(0x670A, "Set target port groups command failed")
 SENSE_CODE(0x670B, "ATA device feature not enabled")
+SENSE_CODE(0x670C, "Command rejected")
+SENSE_CODE(0x670D, "Explicit bind not allowed")
 
 SENSE_CODE(0x6800, "Logical unit not configured")
 SENSE_CODE(0x6801, "Subsidiary logical unit not configured")
@@ -772,6 +820,10 @@ SENSE_CODE(0x6F04, "Media region code is mismatched to logical unit region")
 SENSE_CODE(0x6F05, "Drive region must be permanent/region reset count error")
 SENSE_CODE(0x6F06, "Insufficient block count for binding nonce recording")
 SENSE_CODE(0x6F07, "Conflict in binding nonce recording")
+SENSE_CODE(0x6F08, "Insufficient permission")
+SENSE_CODE(0x6F09, "Invalid drive-host pairing server")
+SENSE_CODE(0x6F0A, "Drive-host pairing suspended")
+
 /*
  *	SENSE_CODE(0x70NN, "Decompression exception short algorithm id of nn")
  */
-- 
2.24.1

Re: [PATCHv3 3/4] scsi: update additional sense codes list

[Damien Le Moal]

On 2020/09/18 8:18, Keith Busch wrote:
> From: Damien Le Moal <damien.lemoal@wdc.com>
> 
> Add missing Additional Sense Codes listed in
> http://www.t10.org/lists/asc-num.txt.
> 
> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>

Martin already pulled this one in scsi-staging. So you can drop it from this series.

> ---
>  drivers/scsi/sense_codes.h | 54 +++++++++++++++++++++++++++++++++++++-
>  1 file changed, 53 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/scsi/sense_codes.h b/drivers/scsi/sense_codes.h
> index 201a536688de..805d4c13d070 100644
> --- a/drivers/scsi/sense_codes.h
> +++ b/drivers/scsi/sense_codes.h
> @@ -1,7 +1,7 @@
>  /* SPDX-License-Identifier: GPL-2.0 */
>  /*
>   * The canonical list of T10 Additional Sense Codes is available at:
> - * http://www.t10.org/lists/asc-num.txt [most recent: 20141221]
> + * http://www.t10.org/lists/asc-num.txt [most recent: 20200817]
>   */
>  
>  SENSE_CODE(0x0000, "No additional sense information")
> @@ -29,6 +29,7 @@ SENSE_CODE(0x001E, "Conflicting SA creation request")
>  SENSE_CODE(0x001F, "Logical unit transitioning to another power condition")
>  SENSE_CODE(0x0020, "Extended copy information available")
>  SENSE_CODE(0x0021, "Atomic command aborted due to ACA")
> +SENSE_CODE(0x0022, "Deferred microcode is pending")
>  
>  SENSE_CODE(0x0100, "No index/sector signal")
>  
> @@ -72,6 +73,9 @@ SENSE_CODE(0x041F, "Logical unit not ready, microcode download required")
>  SENSE_CODE(0x0420, "Logical unit not ready, logical unit reset required")
>  SENSE_CODE(0x0421, "Logical unit not ready, hard reset required")
>  SENSE_CODE(0x0422, "Logical unit not ready, power cycle required")
> +SENSE_CODE(0x0423, "Logical unit not ready, affiliation required")
> +SENSE_CODE(0x0424, "Depopulation in progress")
> +SENSE_CODE(0x0425, "Depopulation restoration in progress")
>  
>  SENSE_CODE(0x0500, "Logical unit does not respond to selection")
>  
> @@ -104,6 +108,17 @@ SENSE_CODE(0x0B06, "Warning - non-volatile cache now volatile")
>  SENSE_CODE(0x0B07, "Warning - degraded power to non-volatile cache")
>  SENSE_CODE(0x0B08, "Warning - power loss expected")
>  SENSE_CODE(0x0B09, "Warning - device statistics notification active")
> +SENSE_CODE(0x0B0A, "Warning - high critical temperature limit exceeded")
> +SENSE_CODE(0x0B0B, "Warning - low critical temperature limit exceeded")
> +SENSE_CODE(0x0B0C, "Warning - high operating temperature limit exceeded")
> +SENSE_CODE(0x0B0D, "Warning - low operating temperature limit exceeded")
> +SENSE_CODE(0x0B0E, "Warning - high critical humidity limit exceeded")
> +SENSE_CODE(0x0B0F, "Warning - low critical humidity limit exceeded")
> +SENSE_CODE(0x0B10, "Warning - high operating humidity limit exceeded")
> +SENSE_CODE(0x0B11, "Warning - low operating humidity limit exceeded")
> +SENSE_CODE(0x0B12, "Warning - microcode security at risk")
> +SENSE_CODE(0x0B13, "Warning - microcode digital signature validation failure")
> +SENSE_CODE(0x0B14, "Warning - physical element status change")
>  
>  SENSE_CODE(0x0C00, "Write error")
>  SENSE_CODE(0x0C01, "Write error - recovered with auto reallocation")
> @@ -122,6 +137,8 @@ SENSE_CODE(0x0C0D, "Write error - not enough unsolicited data")
>  SENSE_CODE(0x0C0E, "Multiple write errors")
>  SENSE_CODE(0x0C0F, "Defects in error window")
>  SENSE_CODE(0x0C10, "Incomplete multiple atomic write operations")
> +SENSE_CODE(0x0C11, "Write error - recovery scan needed")
> +SENSE_CODE(0x0C12, "Write error - insufficient zone resources")
>  
>  SENSE_CODE(0x0D00, "Error detected by third party temporary initiator")
>  SENSE_CODE(0x0D01, "Third party device failure")
> @@ -242,6 +259,9 @@ SENSE_CODE(0x2009, "Access denied - invalid LU identifier")
>  SENSE_CODE(0x200A, "Access denied - invalid proxy token")
>  SENSE_CODE(0x200B, "Access denied - ACL LUN conflict")
>  SENSE_CODE(0x200C, "Illegal command when not in append-only mode")
> +SENSE_CODE(0x200D, "Not an administrative logical unit")
> +SENSE_CODE(0x200E, "Not a subsidiary logical unit")
> +SENSE_CODE(0x200F, "Not a conglomerate logical unit")
>  
>  SENSE_CODE(0x2100, "Logical block address out of range")
>  SENSE_CODE(0x2101, "Invalid element address")
> @@ -251,6 +271,8 @@ SENSE_CODE(0x2104, "Unaligned write command")
>  SENSE_CODE(0x2105, "Write boundary violation")
>  SENSE_CODE(0x2106, "Attempt to read invalid data")
>  SENSE_CODE(0x2107, "Read boundary violation")
> +SENSE_CODE(0x2108, "Misaligned write command")
> +SENSE_CODE(0x2109, "Attempt to access gap zone")
>  
>  SENSE_CODE(0x2200, "Illegal function (use 20 00, 24 00, or 26 00)")
>  
> @@ -275,6 +297,7 @@ SENSE_CODE(0x2405, "Security working key frozen")
>  SENSE_CODE(0x2406, "Nonce not unique")
>  SENSE_CODE(0x2407, "Nonce timestamp out of range")
>  SENSE_CODE(0x2408, "Invalid XCDB")
> +SENSE_CODE(0x2409, "Invalid fast format")
>  
>  SENSE_CODE(0x2500, "Logical unit not supported")
>  
> @@ -297,6 +320,10 @@ SENSE_CODE(0x260F, "Invalid data-out buffer integrity check value")
>  SENSE_CODE(0x2610, "Data decryption key fail limit reached")
>  SENSE_CODE(0x2611, "Incomplete key-associated data set")
>  SENSE_CODE(0x2612, "Vendor specific key reference not found")
> +SENSE_CODE(0x2613, "Application tag mode page is invalid")
> +SENSE_CODE(0x2614, "Tape stream mirroring prevented")
> +SENSE_CODE(0x2615, "Copy source or copy destination not authorized")
> +SENSE_CODE(0x2616, "Fast copy not possible")
>  
>  SENSE_CODE(0x2700, "Write protected")
>  SENSE_CODE(0x2701, "Hardware write protected")
> @@ -342,6 +369,7 @@ SENSE_CODE(0x2A12, "Data encryption parameters changed by vendor specific event"
>  SENSE_CODE(0x2A13, "Data encryption key instance counter has changed")
>  SENSE_CODE(0x2A14, "SA creation capabilities data has changed")
>  SENSE_CODE(0x2A15, "Medium removal prevention preempted")
> +SENSE_CODE(0x2A16, "Zone reset write pointer recommended")
>  
>  SENSE_CODE(0x2B00, "Copy cannot execute since host cannot disconnect")
>  
> @@ -360,6 +388,11 @@ SENSE_CODE(0x2C0B, "Not reserved")
>  SENSE_CODE(0x2C0C, "Orwrite generation does not match")
>  SENSE_CODE(0x2C0D, "Reset write pointer not allowed")
>  SENSE_CODE(0x2C0E, "Zone is offline")
> +SENSE_CODE(0x2C0F, "Stream not open")
> +SENSE_CODE(0x2C10, "Unwritten data in zone")
> +SENSE_CODE(0x2C11, "Descriptor format sense data required")
> +SENSE_CODE(0x2C12, "Zone is inactive")
> +SENSE_CODE(0x2C13, "Well known logical unit access required")
>  
>  SENSE_CODE(0x2D00, "Overwrite error on update in place")
>  
> @@ -395,6 +428,8 @@ SENSE_CODE(0x3100, "Medium format corrupted")
>  SENSE_CODE(0x3101, "Format command failed")
>  SENSE_CODE(0x3102, "Zoned formatting failed due to spare linking")
>  SENSE_CODE(0x3103, "Sanitize command failed")
> +SENSE_CODE(0x3104, "Depopulation failed")
> +SENSE_CODE(0x3105, "Depopulation restoration failed")
>  
>  SENSE_CODE(0x3200, "No defect spare location available")
>  SENSE_CODE(0x3201, "Defect list update failure")
> @@ -419,6 +454,7 @@ SENSE_CODE(0x3802, "Esn - power management class event")
>  SENSE_CODE(0x3804, "Esn - media class event")
>  SENSE_CODE(0x3806, "Esn - device busy class event")
>  SENSE_CODE(0x3807, "Thin Provisioning soft threshold reached")
> +SENSE_CODE(0x3808, "Depopulation interrupted")
>  
>  SENSE_CODE(0x3900, "Saving parameters not supported")
>  
> @@ -456,6 +492,7 @@ SENSE_CODE(0x3B19, "Element enabled")
>  SENSE_CODE(0x3B1A, "Data transfer device removed")
>  SENSE_CODE(0x3B1B, "Data transfer device inserted")
>  SENSE_CODE(0x3B1C, "Too many logical objects on partition to support operation")
> +SENSE_CODE(0x3B20, "Element static information changed")
>  
>  SENSE_CODE(0x3D00, "Invalid bits in identify message")
>  
> @@ -488,6 +525,11 @@ SENSE_CODE(0x3F13, "iSCSI IP address removed")
>  SENSE_CODE(0x3F14, "iSCSI IP address changed")
>  SENSE_CODE(0x3F15, "Inspect referrals sense descriptors")
>  SENSE_CODE(0x3F16, "Microcode has been changed without reset")
> +SENSE_CODE(0x3F17, "Zone transition to full")
> +SENSE_CODE(0x3F18, "Bind completed")
> +SENSE_CODE(0x3F19, "Bind redirected")
> +SENSE_CODE(0x3F1A, "Subsidiary binding changed")
> +
>  /*
>   *	SENSE_CODE(0x40NN, "Ram failure")
>   *	SENSE_CODE(0x40NN, "Diagnostic failure on component nn")
> @@ -589,6 +631,9 @@ SENSE_CODE(0x550B, "Insufficient power for operation")
>  SENSE_CODE(0x550C, "Insufficient resources to create rod")
>  SENSE_CODE(0x550D, "Insufficient resources to create rod token")
>  SENSE_CODE(0x550E, "Insufficient zone resources")
> +SENSE_CODE(0x550F, "Insufficient zone resources to complete write")
> +SENSE_CODE(0x5510, "Maximum number of streams open")
> +SENSE_CODE(0x5511, "Insufficient resources to bind")
>  
>  SENSE_CODE(0x5700, "Unable to recover table-of-contents")
>  
> @@ -692,6 +737,7 @@ SENSE_CODE(0x5D69, "Firmware impending failure throughput performance")
>  SENSE_CODE(0x5D6A, "Firmware impending failure seek time performance")
>  SENSE_CODE(0x5D6B, "Firmware impending failure spin-up retry count")
>  SENSE_CODE(0x5D6C, "Firmware impending failure drive calibration retry count")
> +SENSE_CODE(0x5D73, "Media impending failure endurance limit met")
>  SENSE_CODE(0x5DFF, "Failure prediction threshold exceeded (false)")
>  
>  SENSE_CODE(0x5E00, "Low power condition on")
> @@ -744,6 +790,8 @@ SENSE_CODE(0x6708, "Assign failure occurred")
>  SENSE_CODE(0x6709, "Multiply assigned logical unit")
>  SENSE_CODE(0x670A, "Set target port groups command failed")
>  SENSE_CODE(0x670B, "ATA device feature not enabled")
> +SENSE_CODE(0x670C, "Command rejected")
> +SENSE_CODE(0x670D, "Explicit bind not allowed")
>  
>  SENSE_CODE(0x6800, "Logical unit not configured")
>  SENSE_CODE(0x6801, "Subsidiary logical unit not configured")
> @@ -772,6 +820,10 @@ SENSE_CODE(0x6F04, "Media region code is mismatched to logical unit region")
>  SENSE_CODE(0x6F05, "Drive region must be permanent/region reset count error")
>  SENSE_CODE(0x6F06, "Insufficient block count for binding nonce recording")
>  SENSE_CODE(0x6F07, "Conflict in binding nonce recording")
> +SENSE_CODE(0x6F08, "Insufficient permission")
> +SENSE_CODE(0x6F09, "Invalid drive-host pairing server")
> +SENSE_CODE(0x6F0A, "Drive-host pairing suspended")
> +
>  /*
>   *	SENSE_CODE(0x70NN, "Decompression exception short algorithm id of nn")
>   */
> 


-- 
Damien Le Moal
Western Digital Research

[PATCHv3 4/4] scsi: handle zone resources errors

[Keith Busch]

From: Damien Le Moal <damien.lemoal@wdc.com>

ZBC or ZAC disks that have a limit on the number of open zones may fail
a zone open command or a write to a zone that is not already implicitly
or explicitly open if the total number of open zones is already at the
maximum allowed.

For these operations, instead of returning the generic BLK_STS_IOERR,
return BLK_STS_ZONE_OPEN_RESOURCE which is returned as -ETOOMANYREFS to
the I/O issuer, allowing the device user to act appropriately on these
relatively benign zone resource errors.

Cc: Christoph Hellwig <hch@lst.de>
Cc: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>
---
 drivers/scsi/scsi_lib.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 7affaaf8b98e..c129ac6666da 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -758,6 +758,15 @@ static void scsi_io_completion_action(struct scsi_cmnd *cmd, int result)
 			/* See SSC3rXX or current. */
 			action = ACTION_FAIL;
 			break;
+		case DATA_PROTECT:
+			action = ACTION_FAIL;
+			if ((sshdr.asc == 0x0C && sshdr.ascq == 0x12) ||
+			    (sshdr.asc == 0x55 &&
+			     (sshdr.ascq == 0x0E || sshdr.ascq == 0x0F))) {
+				/* Insufficient zone resources */
+				blk_stat = BLK_STS_ZONE_OPEN_RESOURCE;
+			}
+			break;
 		default:
 			action = ACTION_FAIL;
 			break;
-- 
2.24.1

Re: [PATCHv3 4/4] scsi: handle zone resources errors

[Damien Le Moal]

On 2020/09/18 8:18, Keith Busch wrote:
> From: Damien Le Moal <damien.lemoal@wdc.com>
> 
> ZBC or ZAC disks that have a limit on the number of open zones may fail
> a zone open command or a write to a zone that is not already implicitly
> or explicitly open if the total number of open zones is already at the
> maximum allowed.
> 
> For these operations, instead of returning the generic BLK_STS_IOERR,
> return BLK_STS_ZONE_OPEN_RESOURCE which is returned as -ETOOMANYREFS to
> the I/O issuer, allowing the device user to act appropriately on these
> relatively benign zone resource errors.
> 
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>

Shouldn't you add your signed-off-by here ?

> ---
>  drivers/scsi/scsi_lib.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index 7affaaf8b98e..c129ac6666da 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -758,6 +758,15 @@ static void scsi_io_completion_action(struct scsi_cmnd *cmd, int result)
>  			/* See SSC3rXX or current. */
>  			action = ACTION_FAIL;
>  			break;
> +		case DATA_PROTECT:
> +			action = ACTION_FAIL;
> +			if ((sshdr.asc == 0x0C && sshdr.ascq == 0x12) ||
> +			    (sshdr.asc == 0x55 &&
> +			     (sshdr.ascq == 0x0E || sshdr.ascq == 0x0F))) {
> +				/* Insufficient zone resources */
> +				blk_stat = BLK_STS_ZONE_OPEN_RESOURCE;
> +			}
> +			break;
>  		default:
>  			action = ACTION_FAIL;
>  			break;
> 


-- 
Damien Le Moal
Western Digital Research

Re: [PATCHv3 4/4] scsi: handle zone resources errors

[Martin K. Petersen]

Keith,

> ZBC or ZAC disks that have a limit on the number of open zones may
> fail a zone open command or a write to a zone that is not already
> implicitly or explicitly open if the total number of open zones is
> already at the maximum allowed.
>
> For these operations, instead of returning the generic BLK_STS_IOERR,
> return BLK_STS_ZONE_OPEN_RESOURCE which is returned as -ETOOMANYREFS
> to the I/O issuer, allowing the device user to act appropriately on
> these relatively benign zone resource errors.
>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Johannes Thumshirn <Johannes.Thumshirn@wdc.com>
> Signed-off-by: Damien Le Moal <damien.lemoal@wdc.com>

Looks fine but needs your SoB as Damien pointed out.

Acked-by: Martin K. Petersen <martin.petersen@oracle.com>

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCHv3 0/4] zoned block device specific errors

[Damien Le Moal]

On 2020/09/18 8:18, Keith Busch wrote:
> Zone block devices may have some limits that require special handling.
> This series provides a way for drivers to notify of these conditions.
> 
> v2->v3:
> 
>   Split status for max open vs max active errors (Christoph)
> 
>   Include scsi use for the new status (Damien)
> 
>   Update documentation for the new status inline with the request_queue
>   properties they relate to
> 
> Damien Le Moal (2):
>   scsi: update additional sense codes list
>   scsi: handle zone resources errors
> 
> Keith Busch (2):
>   block: add zone specific block statuses
>   nvme: translate zone resource errors
> 
>  Documentation/block/queue-sysfs.rst |  8 +++++
>  block/blk-core.c                    |  4 +++
>  drivers/nvme/host/core.c            |  4 +++
>  drivers/scsi/scsi_lib.c             |  9 +++++
>  drivers/scsi/sense_codes.h          | 54 ++++++++++++++++++++++++++++-
>  include/linux/blk_types.h           | 18 ++++++++++
>  6 files changed, 96 insertions(+), 1 deletion(-)
> 

+Niklas

We will also need to send an update for nullblk on top of Miklas patch that adds
MAR/MOR emulation, to have everything in sync. But Jens has not pulled that one
in yet I think.

Jens,

We could add Niklas patch, modified, to this series too. Or do you prefer an
incremental update for it ?

-- 
Damien Le Moal
Western Digital Research

Re: [PATCHv3 0/4] zoned block device specific errors

[Christoph Hellwig]

Looks good:

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCHv3 0/4] zoned block device specific errors

[Martin K. Petersen]

Keith,

> Zone block devices may have some limits that require special handling.
> This series provides a way for drivers to notify of these conditions.

This looks good to me. I already queued patch 3 so please send a v4 with
that patch removed. The rest should go through Jens' tree.

Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com>

-- 
Martin K. Petersen	Oracle Linux Engineering