[PATCH v6 0/3] Enabling ATA Command Priorities

[PATCH v6 0/3] Enabling ATA Command Priorities

[Adam Manzanares]

This patch builds ATA commands with high priority if the iocontext of a process
is set to real time. The goal of the patch is to improve tail latencies of 
workloads that use higher queue depths. This requires setting the iocontext 
ioprio on the request when it is initialized

This patch has been tested with an Ultrastar HE8 HDD and cuts the 
the p99.99 tail latency of foreground IO from 2s down to 72ms when
using the deadline scheduler. This patch works independently of the
scheduler so it can be used with all of the currently available 
request based schedulers. 

Foreground IO, for the previously described results, is an async fio job 
submitting 4K read requests at a QD of 1 to the HDD. The foreground IO is set 
with the iopriority class of real time. The background workload is another fio
job submitting read requests at a QD of 32 to the same HDD with default 
iopriority.

This feature is enabled for ATA devices by setting the ata ncq_prio_on device 
attribute to 1. An ATA device is also checked to see if the device supports per
command priority.

v6:
 - Removed fusion mpt sas prio related code deletion
 - Renamed ata device attribute that enables priority cmds to the device

v5:
 - Updated block patch commit message to indicate the why and not the how
v4:
 - Added blk_rq_set_prio function to associate request prio with ioc prio
 - In init_request_from_bio use bio_prio if it is valid
 - Added ata enable_prio dev attribute to sysfs to enable prioritized commands

v3:
 - Removed null dereference issue in blk-core
 - Renamed queue sysfs entries for clarity
 - Added documentation for sysfs queue entry

v2:
 - Add queue flag to set iopriority going to the request
 - If queue flag set, send iopriority class to ata_build_rw_tf
 - Remove redundant code in ata_ncq_prio_enabled function.


Adam Manzanares (3):
  block: Add iocontext priority to request
  ata: Enabling ATA Command Priorities
  ata: ATA Command Priority Disabled By Default

 block/blk-core.c          |  4 ++-
 drivers/ata/libahci.c     |  1 +
 drivers/ata/libata-core.c | 35 +++++++++++++++++++++-
 drivers/ata/libata-scsi.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++-
 drivers/ata/libata.h      |  2 +-
 include/linux/ata.h       |  6 ++++
 include/linux/blkdev.h    | 14 +++++++++
 include/linux/libata.h    | 25 ++++++++++++++++
 8 files changed, 157 insertions(+), 4 deletions(-)

-- 
2.1.4

Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:

This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.

[PATCH v6 1/3] block: Add iocontext priority to request

[Adam Manzanares]

Patch adds an association between iocontext ioprio and the ioprio of a
request. This is done to enable request based drivers the ability to
act on priority information stored in the request. An example being
ATA devices that support command priorities. If the ATA driver discovers
that the device supports command priorities and the request has valid
priority information indicating the request is high priority, then a high
priority command can be sent to the device. This should improve tail
latencies for high priority IO on any device that queues requests
internally and can make use of the priority information stored in the
request.

The ioprio of the request is set in blk_rq_set_prio which takes the
request and the ioc as arguments. If the ioc is valid in blk_rq_set_prio
then the iopriority of the request is set as the iopriority of the ioc.
In init_request_from_bio a check is made to see if the ioprio of the bio
is valid and if so then the request prio comes from the bio.

Signed-off-by: Adam Manzananares <adam.manzanares@wdc.com>
---
 block/blk-core.c       |  4 +++-
 include/linux/blkdev.h | 14 ++++++++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/block/blk-core.c b/block/blk-core.c
index 14d7c07..361b1b9 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1153,6 +1153,7 @@ static struct request *__get_request(struct request_list *rl, int op,
 
 	blk_rq_init(q, rq);
 	blk_rq_set_rl(rq, rl);
+	blk_rq_set_prio(rq, ioc);
 	req_set_op_attrs(rq, op, op_flags | REQ_ALLOCED);
 
 	/* init elvpriv */
@@ -1656,7 +1657,8 @@ void init_request_from_bio(struct request *req, struct bio *bio)
 
 	req->errors = 0;
 	req->__sector = bio->bi_iter.bi_sector;
-	req->ioprio = bio_prio(bio);
+	if (ioprio_valid(bio_prio(bio)))
+		req->ioprio = bio_prio(bio);
 	blk_rq_bio_prep(req->q, req, bio);
 }
 
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index c47c358..9a0ceaa 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -934,6 +934,20 @@ static inline unsigned int blk_rq_count_bios(struct request *rq)
 }
 
 /*
+ * blk_rq_set_prio - associate a request with prio from ioc
+ * @rq: request of interest
+ * @ioc: target iocontext
+ *
+ * Assocate request prio with ioc prio so request based drivers
+ * can leverage priority information.
+ */
+static inline void blk_rq_set_prio(struct request *rq, struct io_context *ioc)
+{
+	if (ioc)
+		rq->ioprio = ioc->ioprio;
+}
+
+/*
  * Request issue related functions.
  */
 extern struct request *blk_peek_request(struct request_queue *q);
-- 
2.1.4

Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:

This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.

[PATCH v6 2/3] ata: Enabling ATA Command Priorities

[Adam Manzanares]

This patch checks to see if an ATA device supports NCQ command priorities.
If so and the user has specified an iocontext that indicates
IO_PRIO_CLASS_RT then we build a tf with a high priority command.

This is done to improve the tail latency of commands that are high
priority by passing priority to the device.

Signed-off-by: Adam Manzanares <adam.manzanares@hgst.com>
---
 drivers/ata/libata-core.c | 35 ++++++++++++++++++++++++++++++++++-
 drivers/ata/libata-scsi.c |  6 +++++-
 drivers/ata/libata.h      |  2 +-
 include/linux/ata.h       |  6 ++++++
 include/linux/libata.h    | 18 ++++++++++++++++++
 5 files changed, 64 insertions(+), 3 deletions(-)

diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 223a770..181b530 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -739,6 +739,7 @@ u64 ata_tf_read_block(const struct ata_taskfile *tf, struct ata_device *dev)
  *	@n_block: Number of blocks
  *	@tf_flags: RW/FUA etc...
  *	@tag: tag
+ *	@class: IO priority class
  *
  *	LOCKING:
  *	None.
@@ -753,7 +754,7 @@ u64 ata_tf_read_block(const struct ata_taskfile *tf, struct ata_device *dev)
  */
 int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 		    u64 block, u32 n_block, unsigned int tf_flags,
-		    unsigned int tag)
+		    unsigned int tag, int class)
 {
 	tf->flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
 	tf->flags |= tf_flags;
@@ -785,6 +786,12 @@ int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 		tf->device = ATA_LBA;
 		if (tf->flags & ATA_TFLAG_FUA)
 			tf->device |= 1 << 7;
+
+		if (ata_ncq_prio_enabled(dev)) {
+			if (class == IOPRIO_CLASS_RT)
+				tf->hob_nsect |= ATA_PRIO_HIGH <<
+						 ATA_SHIFT_PRIO;
+		}
 	} else if (dev->flags & ATA_DFLAG_LBA) {
 		tf->flags |= ATA_TFLAG_LBA;
 
@@ -2156,6 +2163,30 @@ static void ata_dev_config_ncq_non_data(struct ata_device *dev)
 	}
 }
 
+static void ata_dev_config_ncq_prio(struct ata_device *dev)
+{
+	struct ata_port *ap = dev->link->ap;
+	unsigned int err_mask;
+
+	err_mask = ata_read_log_page(dev,
+				     ATA_LOG_SATA_ID_DEV_DATA,
+				     ATA_LOG_SATA_SETTINGS,
+				     ap->sector_buf,
+				     1);
+	if (err_mask) {
+		ata_dev_dbg(dev,
+			    "failed to get Identify Device data, Emask 0x%x\n",
+			    err_mask);
+		return;
+	}
+
+	if (ap->sector_buf[ATA_LOG_NCQ_PRIO_OFFSET] & BIT(3))
+		dev->flags |= ATA_DFLAG_NCQ_PRIO;
+	else
+		ata_dev_dbg(dev, "SATA page does not support priority\n");
+
+}
+
 static int ata_dev_config_ncq(struct ata_device *dev,
 			       char *desc, size_t desc_sz)
 {
@@ -2205,6 +2236,8 @@ static int ata_dev_config_ncq(struct ata_device *dev,
 			ata_dev_config_ncq_send_recv(dev);
 		if (ata_id_has_ncq_non_data(dev->id))
 			ata_dev_config_ncq_non_data(dev);
+		if (ata_id_has_ncq_prio(dev->id))
+			ata_dev_config_ncq_prio(dev);
 	}
 
 	return 0;
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9cceb4a..2bccc3c 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -50,6 +50,7 @@
 #include <linux/uaccess.h>
 #include <linux/suspend.h>
 #include <asm/unaligned.h>
+#include <linux/ioprio.h>
 
 #include "libata.h"
 #include "libata-transport.h"
@@ -1755,6 +1756,8 @@ static unsigned int ata_scsi_rw_xlat(struct ata_queued_cmd *qc)
 {
 	struct scsi_cmnd *scmd = qc->scsicmd;
 	const u8 *cdb = scmd->cmnd;
+	struct request *rq = scmd->request;
+	int class = IOPRIO_PRIO_CLASS(req_get_ioprio(rq));
 	unsigned int tf_flags = 0;
 	u64 block;
 	u32 n_block;
@@ -1821,7 +1824,8 @@ static unsigned int ata_scsi_rw_xlat(struct ata_queued_cmd *qc)
 	qc->nbytes = n_block * scmd->device->sector_size;
 
 	rc = ata_build_rw_tf(&qc->tf, qc->dev, block, n_block, tf_flags,
-			     qc->tag);
+			     qc->tag, class);
+
 	if (likely(rc == 0))
 		return 0;
 
diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h
index 3b301a4..8f3a559 100644
--- a/drivers/ata/libata.h
+++ b/drivers/ata/libata.h
@@ -66,7 +66,7 @@ extern u64 ata_tf_to_lba48(const struct ata_taskfile *tf);
 extern struct ata_queued_cmd *ata_qc_new_init(struct ata_device *dev, int tag);
 extern int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 			   u64 block, u32 n_block, unsigned int tf_flags,
-			   unsigned int tag);
+			   unsigned int tag, int class);
 extern u64 ata_tf_read_block(const struct ata_taskfile *tf,
 			     struct ata_device *dev);
 extern unsigned ata_exec_internal(struct ata_device *dev,
diff --git a/include/linux/ata.h b/include/linux/ata.h
index fdb1803..af6859b 100644
--- a/include/linux/ata.h
+++ b/include/linux/ata.h
@@ -348,6 +348,7 @@ enum {
 	ATA_LOG_DEVSLP_DETO	  = 0x01,
 	ATA_LOG_DEVSLP_VALID	  = 0x07,
 	ATA_LOG_DEVSLP_VALID_MASK = 0x80,
+	ATA_LOG_NCQ_PRIO_OFFSET   = 0x09,
 
 	/* NCQ send and receive log */
 	ATA_LOG_NCQ_SEND_RECV_SUBCMDS_OFFSET	= 0x00,
@@ -940,6 +941,11 @@ static inline bool ata_id_has_ncq_non_data(const u16 *id)
 	return id[ATA_ID_SATA_CAPABILITY_2] & BIT(5);
 }
 
+static inline bool ata_id_has_ncq_prio(const u16 *id)
+{
+	return id[ATA_ID_SATA_CAPABILITY] & BIT(12);
+}
+
 static inline bool ata_id_has_trim(const u16 *id)
 {
 	if (ata_id_major_version(id) >= 7 &&
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 616eef4..1a524cf 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -166,6 +166,7 @@ enum {
 	ATA_DFLAG_NO_UNLOAD	= (1 << 17), /* device doesn't support unload */
 	ATA_DFLAG_UNLOCK_HPA	= (1 << 18), /* unlock HPA */
 	ATA_DFLAG_NCQ_SEND_RECV = (1 << 19), /* device supports NCQ SEND and RECV */
+	ATA_DFLAG_NCQ_PRIO	= (1 << 20), /* device supports NCQ priority */
 	ATA_DFLAG_INIT_MASK	= (1 << 24) - 1,
 
 	ATA_DFLAG_DETACH	= (1 << 24),
@@ -342,7 +343,9 @@ enum {
 	ATA_SHIFT_PIO		= 0,
 	ATA_SHIFT_MWDMA		= ATA_SHIFT_PIO + ATA_NR_PIO_MODES,
 	ATA_SHIFT_UDMA		= ATA_SHIFT_MWDMA + ATA_NR_MWDMA_MODES,
+	ATA_SHIFT_PRIO		= 6,
 
+	ATA_PRIO_HIGH		= 2,
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
 
@@ -1610,6 +1613,21 @@ static inline int ata_ncq_enabled(struct ata_device *dev)
 			      ATA_DFLAG_NCQ)) == ATA_DFLAG_NCQ;
 }
 
+/**
+ *	ata_ncq_prio_enabled - Test whether NCQ prio is enabled
+ *	@dev: ATA device to test for
+ *
+ *	LOCKING:
+ *	spin_lock_irqsave(host lock)
+ *
+ *	RETURNS:
+ *	1 if NCQ prio is enabled for @dev, 0 otherwise.
+ */
+static inline int ata_ncq_prio_enabled(struct ata_device *dev)
+{
+	return (dev->flags & ATA_DFLAG_NCQ_PRIO) == ATA_DFLAG_NCQ_PRIO;
+}
+
 static inline bool ata_fpdma_dsm_supported(struct ata_device *dev)
 {
 	return (dev->flags & ATA_DFLAG_NCQ_SEND_RECV) &&
-- 
2.1.4

Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:

This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.

[PATCH v6 3/3] ata: ATA Command Priority Disabled By Default

[Adam Manzanares]

Add a sysfs entry to turn on priority information being passed
to a ATA device. By default this feature is turned off.

This patch depends on ata: Enabling ATA Command Priorities

Signed-off-by: Adam Manzanares <adam.manzanares@hgst.com>
---
 drivers/ata/libahci.c     |  1 +
 drivers/ata/libata-core.c |  2 +-
 drivers/ata/libata-scsi.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/libata.h    |  7 +++++
 4 files changed, 77 insertions(+), 1 deletion(-)

diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
index 0d028ea..0e17285 100644
--- a/drivers/ata/libahci.c
+++ b/drivers/ata/libahci.c
@@ -140,6 +140,7 @@ EXPORT_SYMBOL_GPL(ahci_shost_attrs);
 struct device_attribute *ahci_sdev_attrs[] = {
 	&dev_attr_sw_activity,
 	&dev_attr_unload_heads,
+	&dev_attr_ncq_prio_on,
 	NULL
 };
 EXPORT_SYMBOL_GPL(ahci_sdev_attrs);
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 181b530..d8e1278 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -787,7 +787,7 @@ int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 		if (tf->flags & ATA_TFLAG_FUA)
 			tf->device |= 1 << 7;
 
-		if (ata_ncq_prio_enabled(dev)) {
+		if (ata_ncq_prio_enabled(dev) && ata_ncq_prio_on(dev)) {
 			if (class == IOPRIO_CLASS_RT)
 				tf->hob_nsect |= ATA_PRIO_HIGH <<
 						 ATA_SHIFT_PRIO;
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 2bccc3c..f2709ac 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -271,6 +271,73 @@ DEVICE_ATTR(unload_heads, S_IRUGO | S_IWUSR,
 	    ata_scsi_park_show, ata_scsi_park_store);
 EXPORT_SYMBOL_GPL(dev_attr_unload_heads);
 
+static ssize_t ata_ncq_prio_on_show(struct device *device,
+				    struct device_attribute *attr, char *buf)
+{
+	struct scsi_device *sdev = to_scsi_device(device);
+	struct ata_port *ap;
+	struct ata_device *dev;
+	int rc = 0;
+	int ncq_prio_on;
+
+	ap = ata_shost_to_port(sdev->host);
+
+	spin_lock_irq(ap->lock);
+	dev = ata_scsi_find_dev(ap, sdev);
+	if (!dev) {
+		rc = -ENODEV;
+		goto unlock;
+	}
+
+	ncq_prio_on = ata_ncq_prio_on(dev);
+
+unlock:
+	spin_unlock_irq(ap->lock);
+
+	return rc ? rc : snprintf(buf, 20, "%u\n", ncq_prio_on);
+}
+
+static ssize_t ata_ncq_prio_on_store(struct device *device,
+				     struct device_attribute *attr,
+				     const char *buf, size_t len)
+{
+	struct scsi_device *sdev = to_scsi_device(device);
+	struct ata_port *ap;
+	struct ata_device *dev;
+	long int input;
+	unsigned long flags;
+	int rc;
+
+	rc = kstrtol(buf, 10, &input);
+	if (rc)
+		return rc;
+	if ((input < 0) || (input > 1))
+		return -EINVAL;
+
+	ap = ata_shost_to_port(sdev->host);
+
+	spin_lock_irqsave(ap->lock, flags);
+	dev = ata_scsi_find_dev(ap, sdev);
+	if (unlikely(!dev)) {
+		rc = -ENODEV;
+		goto unlock;
+	}
+
+	if (input)
+		dev->flags |= ATA_DFLAG_NCQ_PRIO_ON;
+	else
+		dev->flags &= ~ATA_DFLAG_NCQ_PRIO_ON;
+
+unlock:
+	spin_unlock_irqrestore(ap->lock, flags);
+
+	return rc ? rc : len;
+}
+
+DEVICE_ATTR(ncq_prio_on, S_IRUGO | S_IWUSR,
+	    ata_ncq_prio_on_show, ata_ncq_prio_on_store);
+EXPORT_SYMBOL_GPL(dev_attr_ncq_prio_on);
+
 void ata_scsi_set_sense(struct ata_device *dev, struct scsi_cmnd *cmd,
 			u8 sk, u8 asc, u8 ascq)
 {
@@ -402,6 +469,7 @@ EXPORT_SYMBOL_GPL(dev_attr_sw_activity);
 
 struct device_attribute *ata_common_sdev_attrs[] = {
 	&dev_attr_unload_heads,
+	&dev_attr_ncq_prio_on,
 	NULL
 };
 EXPORT_SYMBOL_GPL(ata_common_sdev_attrs);
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 1a524cf..5f24ca1 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -167,6 +167,7 @@ enum {
 	ATA_DFLAG_UNLOCK_HPA	= (1 << 18), /* unlock HPA */
 	ATA_DFLAG_NCQ_SEND_RECV = (1 << 19), /* device supports NCQ SEND and RECV */
 	ATA_DFLAG_NCQ_PRIO	= (1 << 20), /* device supports NCQ priority */
+	ATA_DFLAG_NCQ_PRIO_ON	= (1 << 21), /* Priority cmds sent to dev */
 	ATA_DFLAG_INIT_MASK	= (1 << 24) - 1,
 
 	ATA_DFLAG_DETACH	= (1 << 24),
@@ -545,6 +546,7 @@ typedef void (*ata_postreset_fn_t)(struct ata_link *link, unsigned int *classes)
 
 extern struct device_attribute dev_attr_link_power_management_policy;
 extern struct device_attribute dev_attr_unload_heads;
+extern struct device_attribute dev_attr_ncq_prio_on;
 extern struct device_attribute dev_attr_em_message_type;
 extern struct device_attribute dev_attr_em_message;
 extern struct device_attribute dev_attr_sw_activity;
@@ -1628,6 +1630,11 @@ static inline int ata_ncq_prio_enabled(struct ata_device *dev)
 	return (dev->flags & ATA_DFLAG_NCQ_PRIO) == ATA_DFLAG_NCQ_PRIO;
 }
 
+static inline int ata_ncq_prio_on(struct ata_device *dev)
+{
+	return ((dev->flags & ATA_DFLAG_NCQ_PRIO_ON) == ATA_DFLAG_NCQ_PRIO_ON);
+}
+
 static inline bool ata_fpdma_dsm_supported(struct ata_device *dev)
 {
 	return (dev->flags & ATA_DFLAG_NCQ_SEND_RECV) &&
-- 
2.1.4

Western Digital Corporation (and its subsidiaries) E-mail Confidentiality Notice & Disclaimer:

This e-mail and any files transmitted with it may contain confidential or legally privileged information of WDC and/or its affiliates, and are intended solely for the use of the individual or entity to which they are addressed. If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited. If you have received this e-mail in error, please notify the sender immediately and delete the e-mail in its entirety from your system.

Re: [PATCH v6 1/3] block: Add iocontext priority to request

[Tejun Heo]

Hello,

On Mon, Oct 17, 2016 at 11:27:28AM -0700, Adam Manzanares wrote:
> Patch adds an association between iocontext ioprio and the ioprio of a
> request. This is done to enable request based drivers the ability to
> act on priority information stored in the request. An example being
> ATA devices that support command priorities. If the ATA driver discovers
> that the device supports command priorities and the request has valid
> priority information indicating the request is high priority, then a high
> priority command can be sent to the device. This should improve tail
> latencies for high priority IO on any device that queues requests
> internally and can make use of the priority information stored in the
> request.
> 
> The ioprio of the request is set in blk_rq_set_prio which takes the
> request and the ioc as arguments. If the ioc is valid in blk_rq_set_prio
> then the iopriority of the request is set as the iopriority of the ioc.
> In init_request_from_bio a check is made to see if the ioprio of the bio
> is valid and if so then the request prio comes from the bio.
> 
> Signed-off-by: Adam Manzananares <adam.manzanares@wdc.com>

Jens, if you're okay with it, I can route this through
libata/for-4.10, or this can be applied to block and libata tree can
pull from it.

Thanks.

-- 
tejun

Re: [PATCH v6 3/3] ata: ATA Command Priority Disabled By Default

[Tejun Heo]

On Mon, Oct 17, 2016 at 11:27:30AM -0700, Adam Manzanares wrote:
> Add a sysfs entry to turn on priority information being passed
> to a ATA device. By default this feature is turned off.
> 
> This patch depends on ata: Enabling ATA Command Priorities
> 
> Signed-off-by: Adam Manzanares <adam.manzanares@hgst.com>
> ---
>  drivers/ata/libahci.c     |  1 +
>  drivers/ata/libata-core.c |  2 +-
>  drivers/ata/libata-scsi.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++
>  include/linux/libata.h    |  7 +++++
>  4 files changed, 77 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
> index 0d028ea..0e17285 100644
> --- a/drivers/ata/libahci.c
> +++ b/drivers/ata/libahci.c
> @@ -140,6 +140,7 @@ EXPORT_SYMBOL_GPL(ahci_shost_attrs);
>  struct device_attribute *ahci_sdev_attrs[] = {
>  	&dev_attr_sw_activity,
>  	&dev_attr_unload_heads,
> +	&dev_attr_ncq_prio_on,

I'll rename it to ncq_prio_enable while applying but otherwise looks
good to me.

Thanks.

-- 
tejun

Re: [PATCH v6 1/3] block: Add iocontext priority to request

[Jens Axboe]

On 10/19/2016 12:05 PM, Tejun Heo wrote:
> Hello,
>
> On Mon, Oct 17, 2016 at 11:27:28AM -0700, Adam Manzanares wrote:
>> Patch adds an association between iocontext ioprio and the ioprio of a
>> request. This is done to enable request based drivers the ability to
>> act on priority information stored in the request. An example being
>> ATA devices that support command priorities. If the ATA driver discovers
>> that the device supports command priorities and the request has valid
>> priority information indicating the request is high priority, then a high
>> priority command can be sent to the device. This should improve tail
>> latencies for high priority IO on any device that queues requests
>> internally and can make use of the priority information stored in the
>> request.
>>
>> The ioprio of the request is set in blk_rq_set_prio which takes the
>> request and the ioc as arguments. If the ioc is valid in blk_rq_set_prio
>> then the iopriority of the request is set as the iopriority of the ioc.
>> In init_request_from_bio a check is made to see if the ioprio of the bio
>> is valid and if so then the request prio comes from the bio.
>>
>> Signed-off-by: Adam Manzananares <adam.manzanares@wdc.com>
>
> Jens, if you're okay with it, I can route this through
> libata/for-4.10, or this can be applied to block and libata tree can
> pull from it.

I'm fine with that, add my Reviewed-by and funnel it through the libata
tree.

-- 
Jens Axboe

Re: [PATCH v6 3/3] ata: ATA Command Priority Disabled By Default

[Adam Manzanares]

VGhlIDEwLzE5LzIwMTYgMTQ6MDUsIFRlanVuIEhlbyB3cm90ZToKPiBPbiBNb24sIE9jdCAxNywg
MjAxNiBhdCAxMToyNzozMEFNIC0wNzAwLCBBZGFtIE1hbnphbmFyZXMgd3JvdGU6Cj4gPiBBZGQg
YSBzeXNmcyBlbnRyeSB0byB0dXJuIG9uIHByaW9yaXR5IGluZm9ybWF0aW9uIGJlaW5nIHBhc3Nl
ZAo+ID4gdG8gYSBBVEEgZGV2aWNlLiBCeSBkZWZhdWx0IHRoaXMgZmVhdHVyZSBpcyB0dXJuZWQg
b2ZmLgo+ID4gCj4gPiBUaGlzIHBhdGNoIGRlcGVuZHMgb24gYXRhOiBFbmFibGluZyBBVEEgQ29t
bWFuZCBQcmlvcml0aWVzCj4gPiAKPiA+IFNpZ25lZC1vZmYtYnk6IEFkYW0gTWFuemFuYXJlcyA8
YWRhbS5tYW56YW5hcmVzQGhnc3QuY29tPgo+ID4gLS0tCj4gPiAgZHJpdmVycy9hdGEvbGliYWhj
aS5jICAgICB8ICAxICsKPiA+ICBkcml2ZXJzL2F0YS9saWJhdGEtY29yZS5jIHwgIDIgKy0KPiA+
ICBkcml2ZXJzL2F0YS9saWJhdGEtc2NzaS5jIHwgNjggKysrKysrKysrKysrKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKysKPiA+ICBpbmNsdWRlL2xpbnV4L2xpYmF0YS5oICAgIHwg
IDcgKysrKysKPiA+ICA0IGZpbGVzIGNoYW5nZWQsIDc3IGluc2VydGlvbnMoKyksIDEgZGVsZXRp
b24oLSkKPiA+IAo+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvYXRhL2xpYmFoY2kuYyBiL2RyaXZl
cnMvYXRhL2xpYmFoY2kuYwo+ID4gaW5kZXggMGQwMjhlYS4uMGUxNzI4NSAxMDA2NDQKPiA+IC0t
LSBhL2RyaXZlcnMvYXRhL2xpYmFoY2kuYwo+ID4gKysrIGIvZHJpdmVycy9hdGEvbGliYWhjaS5j
Cj4gPiBAQCAtMTQwLDYgKzE0MCw3IEBAIEVYUE9SVF9TWU1CT0xfR1BMKGFoY2lfc2hvc3RfYXR0
cnMpOwo+ID4gIHN0cnVjdCBkZXZpY2VfYXR0cmlidXRlICphaGNpX3NkZXZfYXR0cnNbXSA9IHsK
PiA+ICAJJmRldl9hdHRyX3N3X2FjdGl2aXR5LAo+ID4gIAkmZGV2X2F0dHJfdW5sb2FkX2hlYWRz
LAo+ID4gKwkmZGV2X2F0dHJfbmNxX3ByaW9fb24sCj4gCj4gSSdsbCByZW5hbWUgaXQgdG8gbmNx
X3ByaW9fZW5hYmxlIHdoaWxlIGFwcGx5aW5nIGJ1dCBvdGhlcndpc2UgbG9va3MKPiBnb29kIHRv
IG1lLgoKU291bmRzIGdvb2QuCgpUYWtlIGNhcmUsCkFkYW0KCj4gCj4gVGhhbmtzLgo+IAo+IC0t
IAo+IHRlanVuCldlc3Rlcm4gRGlnaXRhbCBDb3Jwb3JhdGlvbiAoYW5kIGl0cyBzdWJzaWRpYXJp
ZXMpIEUtbWFpbCBDb25maWRlbnRpYWxpdHkgTm90aWNlICYgRGlzY2xhaW1lcjoKClRoaXMgZS1t
YWlsIGFuZCBhbnkgZmlsZXMgdHJhbnNtaXR0ZWQgd2l0aCBpdCBtYXkgY29udGFpbiBjb25maWRl
bnRpYWwgb3IgbGVnYWxseSBwcml2aWxlZ2VkIGluZm9ybWF0aW9uIG9mIFdEQyBhbmQvb3IgaXRz
IGFmZmlsaWF0ZXMsIGFuZCBhcmUgaW50ZW5kZWQgc29sZWx5IGZvciB0aGUgdXNlIG9mIHRoZSBp
bmRpdmlkdWFsIG9yIGVudGl0eSB0byB3aGljaCB0aGV5IGFyZSBhZGRyZXNzZWQuIElmIHlvdSBh
cmUgbm90IHRoZSBpbnRlbmRlZCByZWNpcGllbnQsIGFueSBkaXNjbG9zdXJlLCBjb3B5aW5nLCBk
aXN0cmlidXRpb24gb3IgYW55IGFjdGlvbiB0YWtlbiBvciBvbWl0dGVkIHRvIGJlIHRha2VuIGlu
IHJlbGlhbmNlIG9uIGl0LCBpcyBwcm9oaWJpdGVkLiBJZiB5b3UgaGF2ZSByZWNlaXZlZCB0aGlz
IGUtbWFpbCBpbiBlcnJvciwgcGxlYXNlIG5vdGlmeSB0aGUgc2VuZGVyIGltbWVkaWF0ZWx5IGFu
ZCBkZWxldGUgdGhlIGUtbWFpbCBpbiBpdHMgZW50aXJldHkgZnJvbSB5b3VyIHN5c3RlbS4K

Re: [PATCH v6 2/3] ata: Enabling ATA Command Priorities

[Tejun Heo]

Hello,

I removed ata_ncq_prio_enabled() as it was colliding with the sysfs
file rename and these trivial test functions tend to obscure more than
help anything.

Thanks.

------ 8< ------
>From 8e061784b51ec4a4efed0deaafb5bd9725bf5b06 Mon Sep 17 00:00:00 2001
From: Adam Manzanares <adam.manzanares@hgst.com>
Date: Mon, 17 Oct 2016 11:27:29 -0700
Subject: [PATCH 1/2] ata: Enabling ATA Command Priorities

This patch checks to see if an ATA device supports NCQ command priorities.
If so and the user has specified an iocontext that indicates
IO_PRIO_CLASS_RT then we build a tf with a high priority command.

This is done to improve the tail latency of commands that are high
priority by passing priority to the device.

tj: Removed trivial ata_ncq_prio_enabled() and open-coded the test.

Signed-off-by: Adam Manzanares <adam.manzanares@hgst.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
---
 drivers/ata/libata-core.c | 35 ++++++++++++++++++++++++++++++++++-
 drivers/ata/libata-scsi.c |  6 +++++-
 drivers/ata/libata.h      |  2 +-
 include/linux/ata.h       |  6 ++++++
 include/linux/libata.h    |  3 +++
 5 files changed, 49 insertions(+), 3 deletions(-)

diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 223a770..8346faf 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -739,6 +739,7 @@ u64 ata_tf_read_block(const struct ata_taskfile *tf, struct ata_device *dev)
  *	@n_block: Number of blocks
  *	@tf_flags: RW/FUA etc...
  *	@tag: tag
+ *	@class: IO priority class
  *
  *	LOCKING:
  *	None.
@@ -753,7 +754,7 @@ u64 ata_tf_read_block(const struct ata_taskfile *tf, struct ata_device *dev)
  */
 int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 		    u64 block, u32 n_block, unsigned int tf_flags,
-		    unsigned int tag)
+		    unsigned int tag, int class)
 {
 	tf->flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
 	tf->flags |= tf_flags;
@@ -785,6 +786,12 @@ int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 		tf->device = ATA_LBA;
 		if (tf->flags & ATA_TFLAG_FUA)
 			tf->device |= 1 << 7;
+
+		if (dev->flags & ATA_DFLAG_NCQ_PRIO) {
+			if (class == IOPRIO_CLASS_RT)
+				tf->hob_nsect |= ATA_PRIO_HIGH <<
+						 ATA_SHIFT_PRIO;
+		}
 	} else if (dev->flags & ATA_DFLAG_LBA) {
 		tf->flags |= ATA_TFLAG_LBA;
 
@@ -2156,6 +2163,30 @@ static void ata_dev_config_ncq_non_data(struct ata_device *dev)
 	}
 }
 
+static void ata_dev_config_ncq_prio(struct ata_device *dev)
+{
+	struct ata_port *ap = dev->link->ap;
+	unsigned int err_mask;
+
+	err_mask = ata_read_log_page(dev,
+				     ATA_LOG_SATA_ID_DEV_DATA,
+				     ATA_LOG_SATA_SETTINGS,
+				     ap->sector_buf,
+				     1);
+	if (err_mask) {
+		ata_dev_dbg(dev,
+			    "failed to get Identify Device data, Emask 0x%x\n",
+			    err_mask);
+		return;
+	}
+
+	if (ap->sector_buf[ATA_LOG_NCQ_PRIO_OFFSET] & BIT(3))
+		dev->flags |= ATA_DFLAG_NCQ_PRIO;
+	else
+		ata_dev_dbg(dev, "SATA page does not support priority\n");
+
+}
+
 static int ata_dev_config_ncq(struct ata_device *dev,
 			       char *desc, size_t desc_sz)
 {
@@ -2205,6 +2236,8 @@ static int ata_dev_config_ncq(struct ata_device *dev,
 			ata_dev_config_ncq_send_recv(dev);
 		if (ata_id_has_ncq_non_data(dev->id))
 			ata_dev_config_ncq_non_data(dev);
+		if (ata_id_has_ncq_prio(dev->id))
+			ata_dev_config_ncq_prio(dev);
 	}
 
 	return 0;
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 9cceb4a..2bccc3c 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -50,6 +50,7 @@
 #include <linux/uaccess.h>
 #include <linux/suspend.h>
 #include <asm/unaligned.h>
+#include <linux/ioprio.h>
 
 #include "libata.h"
 #include "libata-transport.h"
@@ -1755,6 +1756,8 @@ static unsigned int ata_scsi_rw_xlat(struct ata_queued_cmd *qc)
 {
 	struct scsi_cmnd *scmd = qc->scsicmd;
 	const u8 *cdb = scmd->cmnd;
+	struct request *rq = scmd->request;
+	int class = IOPRIO_PRIO_CLASS(req_get_ioprio(rq));
 	unsigned int tf_flags = 0;
 	u64 block;
 	u32 n_block;
@@ -1821,7 +1824,8 @@ static unsigned int ata_scsi_rw_xlat(struct ata_queued_cmd *qc)
 	qc->nbytes = n_block * scmd->device->sector_size;
 
 	rc = ata_build_rw_tf(&qc->tf, qc->dev, block, n_block, tf_flags,
-			     qc->tag);
+			     qc->tag, class);
+
 	if (likely(rc == 0))
 		return 0;
 
diff --git a/drivers/ata/libata.h b/drivers/ata/libata.h
index 3b301a4..8f3a559 100644
--- a/drivers/ata/libata.h
+++ b/drivers/ata/libata.h
@@ -66,7 +66,7 @@ extern u64 ata_tf_to_lba48(const struct ata_taskfile *tf);
 extern struct ata_queued_cmd *ata_qc_new_init(struct ata_device *dev, int tag);
 extern int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 			   u64 block, u32 n_block, unsigned int tf_flags,
-			   unsigned int tag);
+			   unsigned int tag, int class);
 extern u64 ata_tf_read_block(const struct ata_taskfile *tf,
 			     struct ata_device *dev);
 extern unsigned ata_exec_internal(struct ata_device *dev,
diff --git a/include/linux/ata.h b/include/linux/ata.h
index fdb1803..af6859b 100644
--- a/include/linux/ata.h
+++ b/include/linux/ata.h
@@ -348,6 +348,7 @@ enum {
 	ATA_LOG_DEVSLP_DETO	  = 0x01,
 	ATA_LOG_DEVSLP_VALID	  = 0x07,
 	ATA_LOG_DEVSLP_VALID_MASK = 0x80,
+	ATA_LOG_NCQ_PRIO_OFFSET   = 0x09,
 
 	/* NCQ send and receive log */
 	ATA_LOG_NCQ_SEND_RECV_SUBCMDS_OFFSET	= 0x00,
@@ -940,6 +941,11 @@ static inline bool ata_id_has_ncq_non_data(const u16 *id)
 	return id[ATA_ID_SATA_CAPABILITY_2] & BIT(5);
 }
 
+static inline bool ata_id_has_ncq_prio(const u16 *id)
+{
+	return id[ATA_ID_SATA_CAPABILITY] & BIT(12);
+}
+
 static inline bool ata_id_has_trim(const u16 *id)
 {
 	if (ata_id_major_version(id) >= 7 &&
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 616eef4..90b69a6 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -166,6 +166,7 @@ enum {
 	ATA_DFLAG_NO_UNLOAD	= (1 << 17), /* device doesn't support unload */
 	ATA_DFLAG_UNLOCK_HPA	= (1 << 18), /* unlock HPA */
 	ATA_DFLAG_NCQ_SEND_RECV = (1 << 19), /* device supports NCQ SEND and RECV */
+	ATA_DFLAG_NCQ_PRIO	= (1 << 20), /* device supports NCQ priority */
 	ATA_DFLAG_INIT_MASK	= (1 << 24) - 1,
 
 	ATA_DFLAG_DETACH	= (1 << 24),
@@ -342,7 +343,9 @@ enum {
 	ATA_SHIFT_PIO		= 0,
 	ATA_SHIFT_MWDMA		= ATA_SHIFT_PIO + ATA_NR_PIO_MODES,
 	ATA_SHIFT_UDMA		= ATA_SHIFT_MWDMA + ATA_NR_MWDMA_MODES,
+	ATA_SHIFT_PRIO		= 6,
 
+	ATA_PRIO_HIGH		= 2,
 	/* size of buffer to pad xfers ending on unaligned boundaries */
 	ATA_DMA_PAD_SZ		= 4,
 
-- 
2.7.4

Re: [PATCH v6 3/3] ata: ATA Command Priority Disabled By Default

[Tejun Heo]

Hello,

Removed ata_ncq_prio_on() and renamed _on to _enable.  If I messed up
anything, please let me know.

Also, can you please send a follow-up patch to make the store function
reject prio enabling if the device doesn't support it?

Thanks.

------ 8< ------
>From 84f95243b5439a20c33837075b88926bfa00c4ec Mon Sep 17 00:00:00 2001
From: Adam Manzanares <adam.manzanares@hgst.com>
Date: Mon, 17 Oct 2016 11:27:30 -0700
Subject: [PATCH 2/2] ata: ATA Command Priority Disabled By Default

Add a sysfs entry to turn on priority information being passed
to a ATA device. By default this feature is turned off.

This patch depends on ata: Enabling ATA Command Priorities

tj: Renamed ncq_prio_on to ncq_prio_enable and removed trivial
    ata_ncq_prio_on() and open-coded the test.

Signed-off-by: Adam Manzanares <adam.manzanares@hgst.com>
Signed-off-by: Tejun Heo <tj@kernel.org>
---
 drivers/ata/libahci.c     |  1 +
 drivers/ata/libata-core.c |  3 ++-
 drivers/ata/libata-scsi.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++
 include/linux/libata.h    |  2 ++
 4 files changed, 73 insertions(+), 1 deletion(-)

diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
index 0d028ea..ee7db31 100644
--- a/drivers/ata/libahci.c
+++ b/drivers/ata/libahci.c
@@ -140,6 +140,7 @@ EXPORT_SYMBOL_GPL(ahci_shost_attrs);
 struct device_attribute *ahci_sdev_attrs[] = {
 	&dev_attr_sw_activity,
 	&dev_attr_unload_heads,
+	&dev_attr_ncq_prio_enable,
 	NULL
 };
 EXPORT_SYMBOL_GPL(ahci_sdev_attrs);
diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c
index 8346faf..b294339 100644
--- a/drivers/ata/libata-core.c
+++ b/drivers/ata/libata-core.c
@@ -787,7 +787,8 @@ int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
 		if (tf->flags & ATA_TFLAG_FUA)
 			tf->device |= 1 << 7;
 
-		if (dev->flags & ATA_DFLAG_NCQ_PRIO) {
+		if ((dev->flags & ATA_DFLAG_NCQ_PRIO) &&
+		    (dev->flags & ATA_DFLAG_NCQ_PRIO_ENABLE)) {
 			if (class == IOPRIO_CLASS_RT)
 				tf->hob_nsect |= ATA_PRIO_HIGH <<
 						 ATA_SHIFT_PRIO;
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 2bccc3c..87597a3 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -271,6 +271,73 @@ DEVICE_ATTR(unload_heads, S_IRUGO | S_IWUSR,
 	    ata_scsi_park_show, ata_scsi_park_store);
 EXPORT_SYMBOL_GPL(dev_attr_unload_heads);
 
+static ssize_t ata_ncq_prio_enable_show(struct device *device,
+					struct device_attribute *attr, char *buf)
+{
+	struct scsi_device *sdev = to_scsi_device(device);
+	struct ata_port *ap;
+	struct ata_device *dev;
+	bool ncq_prio_enable;
+	int rc = 0;
+
+	ap = ata_shost_to_port(sdev->host);
+
+	spin_lock_irq(ap->lock);
+	dev = ata_scsi_find_dev(ap, sdev);
+	if (!dev) {
+		rc = -ENODEV;
+		goto unlock;
+	}
+
+	ncq_prio_enable = dev->flags & ATA_DFLAG_NCQ_PRIO_ENABLE;
+
+unlock:
+	spin_unlock_irq(ap->lock);
+
+	return rc ? rc : snprintf(buf, 20, "%u\n", ncq_prio_enable);
+}
+
+static ssize_t ata_ncq_prio_enable_store(struct device *device,
+					 struct device_attribute *attr,
+					 const char *buf, size_t len)
+{
+	struct scsi_device *sdev = to_scsi_device(device);
+	struct ata_port *ap;
+	struct ata_device *dev;
+	long int input;
+	unsigned long flags;
+	int rc;
+
+	rc = kstrtol(buf, 10, &input);
+	if (rc)
+		return rc;
+	if ((input < 0) || (input > 1))
+		return -EINVAL;
+
+	ap = ata_shost_to_port(sdev->host);
+
+	spin_lock_irqsave(ap->lock, flags);
+	dev = ata_scsi_find_dev(ap, sdev);
+	if (unlikely(!dev)) {
+		rc = -ENODEV;
+		goto unlock;
+	}
+
+	if (input)
+		dev->flags |= ATA_DFLAG_NCQ_PRIO_ENABLE;
+	else
+		dev->flags &= ~ATA_DFLAG_NCQ_PRIO_ENABLE;
+
+unlock:
+	spin_unlock_irqrestore(ap->lock, flags);
+
+	return rc ? rc : len;
+}
+
+DEVICE_ATTR(ncq_prio_enable, S_IRUGO | S_IWUSR,
+	    ata_ncq_prio_enable_show, ata_ncq_prio_enable_store);
+EXPORT_SYMBOL_GPL(dev_attr_ncq_prio_enable);
+
 void ata_scsi_set_sense(struct ata_device *dev, struct scsi_cmnd *cmd,
 			u8 sk, u8 asc, u8 ascq)
 {
@@ -402,6 +469,7 @@ EXPORT_SYMBOL_GPL(dev_attr_sw_activity);
 
 struct device_attribute *ata_common_sdev_attrs[] = {
 	&dev_attr_unload_heads,
+	&dev_attr_ncq_prio_enable,
 	NULL
 };
 EXPORT_SYMBOL_GPL(ata_common_sdev_attrs);
diff --git a/include/linux/libata.h b/include/linux/libata.h
index 90b69a6..c170be5 100644
--- a/include/linux/libata.h
+++ b/include/linux/libata.h
@@ -167,6 +167,7 @@ enum {
 	ATA_DFLAG_UNLOCK_HPA	= (1 << 18), /* unlock HPA */
 	ATA_DFLAG_NCQ_SEND_RECV = (1 << 19), /* device supports NCQ SEND and RECV */
 	ATA_DFLAG_NCQ_PRIO	= (1 << 20), /* device supports NCQ priority */
+	ATA_DFLAG_NCQ_PRIO_ENABLE = (1 << 21), /* Priority cmds sent to dev */
 	ATA_DFLAG_INIT_MASK	= (1 << 24) - 1,
 
 	ATA_DFLAG_DETACH	= (1 << 24),
@@ -545,6 +546,7 @@ typedef void (*ata_postreset_fn_t)(struct ata_link *link, unsigned int *classes)
 
 extern struct device_attribute dev_attr_link_power_management_policy;
 extern struct device_attribute dev_attr_unload_heads;
+extern struct device_attribute dev_attr_ncq_prio_enable;
 extern struct device_attribute dev_attr_em_message_type;
 extern struct device_attribute dev_attr_em_message;
 extern struct device_attribute dev_attr_sw_activity;
-- 
2.7.4

Re: [PATCH v6 0/3] Enabling ATA Command Priorities

[Tejun Heo]

On Mon, Oct 17, 2016 at 11:27:27AM -0700, Adam Manzanares wrote:
> This patch builds ATA commands with high priority if the iocontext of a process
> is set to real time. The goal of the patch is to improve tail latencies of 
> workloads that use higher queue depths. This requires setting the iocontext 
> ioprio on the request when it is initialized
> 
> This patch has been tested with an Ultrastar HE8 HDD and cuts the 
> the p99.99 tail latency of foreground IO from 2s down to 72ms when
> using the deadline scheduler. This patch works independently of the
> scheduler so it can be used with all of the currently available 
> request based schedulers. 
> 
> Foreground IO, for the previously described results, is an async fio job 
> submitting 4K read requests at a QD of 1 to the HDD. The foreground IO is set 
> with the iopriority class of real time. The background workload is another fio
> job submitting read requests at a QD of 32 to the same HDD with default 
> iopriority.
> 
> This feature is enabled for ATA devices by setting the ata ncq_prio_on device 
> attribute to 1. An ATA device is also checked to see if the device supports per
> command priority.

Applied 1-3 to libata/for-4.10 w/ some modifications.

Thanks.

-- 
tejun

Re: [PATCH v6 3/3] ata: ATA Command Priority Disabled By Default

[Adam Manzanares]

VGhlIDEwLzE5LzIwMTYgMTQ6MzgsIFRlanVuIEhlbyB3cm90ZToKPiBIZWxsbywKPiAKPiBSZW1v
dmVkIGF0YV9uY3FfcHJpb19vbigpIGFuZCByZW5hbWVkIF9vbiB0byBfZW5hYmxlLiAgSWYgSSBt
ZXNzZWQgdXAKPiBhbnl0aGluZywgcGxlYXNlIGxldCBtZSBrbm93Lgo+IAoKV29ya3MgYXMgZXhw
ZWN0ZWQuIFRoYW5rcyBmb3IgY2xlYW5pbmcgdGhpcyB1cC4KCj4gQWxzbywgY2FuIHlvdSBwbGVh
c2Ugc2VuZCBhIGZvbGxvdy11cCBwYXRjaCB0byBtYWtlIHRoZSBzdG9yZSBmdW5jdGlvbgo+IHJl
amVjdCBwcmlvIGVuYWJsaW5nIGlmIHRoZSBkZXZpY2UgZG9lc24ndCBzdXBwb3J0IGl0Pwo+IAoK
SSdsbCBzZW5kIHNvbWV0aGluZyBvdXQgc2hvcnRseS4KClRha2UgY2FyZSwKQWRhbQoKPiBUaGFu
a3MuCj4gCj4gLS0tLS0tIDg8IC0tLS0tLQo+IEZyb20gODRmOTUyNDNiNTQzOWEyMGMzMzgzNzA3
NWI4ODkyNmJmYTAwYzRlYyBNb24gU2VwIDE3IDAwOjAwOjAwIDIwMDEKPiBGcm9tOiBBZGFtIE1h
bnphbmFyZXMgPGFkYW0ubWFuemFuYXJlc0BoZ3N0LmNvbT4KPiBEYXRlOiBNb24sIDE3IE9jdCAy
MDE2IDExOjI3OjMwIC0wNzAwCj4gU3ViamVjdDogW1BBVENIIDIvMl0gYXRhOiBBVEEgQ29tbWFu
ZCBQcmlvcml0eSBEaXNhYmxlZCBCeSBEZWZhdWx0Cj4gCj4gQWRkIGEgc3lzZnMgZW50cnkgdG8g
dHVybiBvbiBwcmlvcml0eSBpbmZvcm1hdGlvbiBiZWluZyBwYXNzZWQKPiB0byBhIEFUQSBkZXZp
Y2UuIEJ5IGRlZmF1bHQgdGhpcyBmZWF0dXJlIGlzIHR1cm5lZCBvZmYuCj4gCj4gVGhpcyBwYXRj
aCBkZXBlbmRzIG9uIGF0YTogRW5hYmxpbmcgQVRBIENvbW1hbmQgUHJpb3JpdGllcwo+IAo+IHRq
OiBSZW5hbWVkIG5jcV9wcmlvX29uIHRvIG5jcV9wcmlvX2VuYWJsZSBhbmQgcmVtb3ZlZCB0cml2
aWFsCj4gICAgIGF0YV9uY3FfcHJpb19vbigpIGFuZCBvcGVuLWNvZGVkIHRoZSB0ZXN0Lgo+IAo+
IFNpZ25lZC1vZmYtYnk6IEFkYW0gTWFuemFuYXJlcyA8YWRhbS5tYW56YW5hcmVzQGhnc3QuY29t
Pgo+IFNpZ25lZC1vZmYtYnk6IFRlanVuIEhlbyA8dGpAa2VybmVsLm9yZz4KPiAtLS0KPiAgZHJp
dmVycy9hdGEvbGliYWhjaS5jICAgICB8ICAxICsKPiAgZHJpdmVycy9hdGEvbGliYXRhLWNvcmUu
YyB8ICAzICsrLQo+ICBkcml2ZXJzL2F0YS9saWJhdGEtc2NzaS5jIHwgNjggKysrKysrKysrKysr
KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysKPiAgaW5jbHVkZS9saW51eC9saWJh
dGEuaCAgICB8ICAyICsrCj4gIDQgZmlsZXMgY2hhbmdlZCwgNzMgaW5zZXJ0aW9ucygrKSwgMSBk
ZWxldGlvbigtKQo+IAo+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2F0YS9saWJhaGNpLmMgYi9kcml2
ZXJzL2F0YS9saWJhaGNpLmMKPiBpbmRleCAwZDAyOGVhLi5lZTdkYjMxIDEwMDY0NAo+IC0tLSBh
L2RyaXZlcnMvYXRhL2xpYmFoY2kuYwo+ICsrKyBiL2RyaXZlcnMvYXRhL2xpYmFoY2kuYwo+IEBA
IC0xNDAsNiArMTQwLDcgQEAgRVhQT1JUX1NZTUJPTF9HUEwoYWhjaV9zaG9zdF9hdHRycyk7Cj4g
IHN0cnVjdCBkZXZpY2VfYXR0cmlidXRlICphaGNpX3NkZXZfYXR0cnNbXSA9IHsKPiAgCSZkZXZf
YXR0cl9zd19hY3Rpdml0eSwKPiAgCSZkZXZfYXR0cl91bmxvYWRfaGVhZHMsCj4gKwkmZGV2X2F0
dHJfbmNxX3ByaW9fZW5hYmxlLAo+ICAJTlVMTAo+ICB9Owo+ICBFWFBPUlRfU1lNQk9MX0dQTChh
aGNpX3NkZXZfYXR0cnMpOwo+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2F0YS9saWJhdGEtY29yZS5j
IGIvZHJpdmVycy9hdGEvbGliYXRhLWNvcmUuYwo+IGluZGV4IDgzNDZmYWYuLmIyOTQzMzkgMTAw
NjQ0Cj4gLS0tIGEvZHJpdmVycy9hdGEvbGliYXRhLWNvcmUuYwo+ICsrKyBiL2RyaXZlcnMvYXRh
L2xpYmF0YS1jb3JlLmMKPiBAQCAtNzg3LDcgKzc4Nyw4IEBAIGludCBhdGFfYnVpbGRfcndfdGYo
c3RydWN0IGF0YV90YXNrZmlsZSAqdGYsIHN0cnVjdCBhdGFfZGV2aWNlICpkZXYsCj4gIAkJaWYg
KHRmLT5mbGFncyAmIEFUQV9URkxBR19GVUEpCj4gIAkJCXRmLT5kZXZpY2UgfD0gMSA8PCA3Owo+
ICAKPiAtCQlpZiAoZGV2LT5mbGFncyAmIEFUQV9ERkxBR19OQ1FfUFJJTykgewo+ICsJCWlmICgo
ZGV2LT5mbGFncyAmIEFUQV9ERkxBR19OQ1FfUFJJTykgJiYKPiArCQkgICAgKGRldi0+ZmxhZ3Mg
JiBBVEFfREZMQUdfTkNRX1BSSU9fRU5BQkxFKSkgewo+ICAJCQlpZiAoY2xhc3MgPT0gSU9QUklP
X0NMQVNTX1JUKQo+ICAJCQkJdGYtPmhvYl9uc2VjdCB8PSBBVEFfUFJJT19ISUdIIDw8Cj4gIAkJ
CQkJCSBBVEFfU0hJRlRfUFJJTzsKPiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9hdGEvbGliYXRhLXNj
c2kuYyBiL2RyaXZlcnMvYXRhL2xpYmF0YS1zY3NpLmMKPiBpbmRleCAyYmNjYzNjLi44NzU5N2Ez
IDEwMDY0NAo+IC0tLSBhL2RyaXZlcnMvYXRhL2xpYmF0YS1zY3NpLmMKPiArKysgYi9kcml2ZXJz
L2F0YS9saWJhdGEtc2NzaS5jCj4gQEAgLTI3MSw2ICsyNzEsNzMgQEAgREVWSUNFX0FUVFIodW5s
b2FkX2hlYWRzLCBTX0lSVUdPIHwgU19JV1VTUiwKPiAgCSAgICBhdGFfc2NzaV9wYXJrX3Nob3cs
IGF0YV9zY3NpX3Bhcmtfc3RvcmUpOwo+ICBFWFBPUlRfU1lNQk9MX0dQTChkZXZfYXR0cl91bmxv
YWRfaGVhZHMpOwo+ICAKPiArc3RhdGljIHNzaXplX3QgYXRhX25jcV9wcmlvX2VuYWJsZV9zaG93
KHN0cnVjdCBkZXZpY2UgKmRldmljZSwKPiArCQkJCQlzdHJ1Y3QgZGV2aWNlX2F0dHJpYnV0ZSAq
YXR0ciwgY2hhciAqYnVmKQo+ICt7Cj4gKwlzdHJ1Y3Qgc2NzaV9kZXZpY2UgKnNkZXYgPSB0b19z
Y3NpX2RldmljZShkZXZpY2UpOwo+ICsJc3RydWN0IGF0YV9wb3J0ICphcDsKPiArCXN0cnVjdCBh
dGFfZGV2aWNlICpkZXY7Cj4gKwlib29sIG5jcV9wcmlvX2VuYWJsZTsKPiArCWludCByYyA9IDA7
Cj4gKwo+ICsJYXAgPSBhdGFfc2hvc3RfdG9fcG9ydChzZGV2LT5ob3N0KTsKPiArCj4gKwlzcGlu
X2xvY2tfaXJxKGFwLT5sb2NrKTsKPiArCWRldiA9IGF0YV9zY3NpX2ZpbmRfZGV2KGFwLCBzZGV2
KTsKPiArCWlmICghZGV2KSB7Cj4gKwkJcmMgPSAtRU5PREVWOwo+ICsJCWdvdG8gdW5sb2NrOwo+
ICsJfQo+ICsKPiArCW5jcV9wcmlvX2VuYWJsZSA9IGRldi0+ZmxhZ3MgJiBBVEFfREZMQUdfTkNR
X1BSSU9fRU5BQkxFOwo+ICsKPiArdW5sb2NrOgo+ICsJc3Bpbl91bmxvY2tfaXJxKGFwLT5sb2Nr
KTsKPiArCj4gKwlyZXR1cm4gcmMgPyByYyA6IHNucHJpbnRmKGJ1ZiwgMjAsICIldVxuIiwgbmNx
X3ByaW9fZW5hYmxlKTsKPiArfQo+ICsKPiArc3RhdGljIHNzaXplX3QgYXRhX25jcV9wcmlvX2Vu
YWJsZV9zdG9yZShzdHJ1Y3QgZGV2aWNlICpkZXZpY2UsCj4gKwkJCQkJIHN0cnVjdCBkZXZpY2Vf
YXR0cmlidXRlICphdHRyLAo+ICsJCQkJCSBjb25zdCBjaGFyICpidWYsIHNpemVfdCBsZW4pCj4g
K3sKPiArCXN0cnVjdCBzY3NpX2RldmljZSAqc2RldiA9IHRvX3Njc2lfZGV2aWNlKGRldmljZSk7
Cj4gKwlzdHJ1Y3QgYXRhX3BvcnQgKmFwOwo+ICsJc3RydWN0IGF0YV9kZXZpY2UgKmRldjsKPiAr
CWxvbmcgaW50IGlucHV0Owo+ICsJdW5zaWduZWQgbG9uZyBmbGFnczsKPiArCWludCByYzsKPiAr
Cj4gKwlyYyA9IGtzdHJ0b2woYnVmLCAxMCwgJmlucHV0KTsKPiArCWlmIChyYykKPiArCQlyZXR1
cm4gcmM7Cj4gKwlpZiAoKGlucHV0IDwgMCkgfHwgKGlucHV0ID4gMSkpCj4gKwkJcmV0dXJuIC1F
SU5WQUw7Cj4gKwo+ICsJYXAgPSBhdGFfc2hvc3RfdG9fcG9ydChzZGV2LT5ob3N0KTsKPiArCj4g
KwlzcGluX2xvY2tfaXJxc2F2ZShhcC0+bG9jaywgZmxhZ3MpOwo+ICsJZGV2ID0gYXRhX3Njc2lf
ZmluZF9kZXYoYXAsIHNkZXYpOwo+ICsJaWYgKHVubGlrZWx5KCFkZXYpKSB7Cj4gKwkJcmMgPSAt
RU5PREVWOwo+ICsJCWdvdG8gdW5sb2NrOwo+ICsJfQo+ICsKPiArCWlmIChpbnB1dCkKPiArCQlk
ZXYtPmZsYWdzIHw9IEFUQV9ERkxBR19OQ1FfUFJJT19FTkFCTEU7Cj4gKwllbHNlCj4gKwkJZGV2
LT5mbGFncyAmPSB+QVRBX0RGTEFHX05DUV9QUklPX0VOQUJMRTsKPiArCj4gK3VubG9jazoKPiAr
CXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoYXAtPmxvY2ssIGZsYWdzKTsKPiArCj4gKwlyZXR1cm4g
cmMgPyByYyA6IGxlbjsKPiArfQo+ICsKPiArREVWSUNFX0FUVFIobmNxX3ByaW9fZW5hYmxlLCBT
X0lSVUdPIHwgU19JV1VTUiwKPiArCSAgICBhdGFfbmNxX3ByaW9fZW5hYmxlX3Nob3csIGF0YV9u
Y3FfcHJpb19lbmFibGVfc3RvcmUpOwo+ICtFWFBPUlRfU1lNQk9MX0dQTChkZXZfYXR0cl9uY3Ff
cHJpb19lbmFibGUpOwo+ICsKPiAgdm9pZCBhdGFfc2NzaV9zZXRfc2Vuc2Uoc3RydWN0IGF0YV9k
ZXZpY2UgKmRldiwgc3RydWN0IHNjc2lfY21uZCAqY21kLAo+ICAJCQl1OCBzaywgdTggYXNjLCB1
OCBhc2NxKQo+ICB7Cj4gQEAgLTQwMiw2ICs0NjksNyBAQCBFWFBPUlRfU1lNQk9MX0dQTChkZXZf
YXR0cl9zd19hY3Rpdml0eSk7Cj4gIAo+ICBzdHJ1Y3QgZGV2aWNlX2F0dHJpYnV0ZSAqYXRhX2Nv
bW1vbl9zZGV2X2F0dHJzW10gPSB7Cj4gIAkmZGV2X2F0dHJfdW5sb2FkX2hlYWRzLAo+ICsJJmRl
dl9hdHRyX25jcV9wcmlvX2VuYWJsZSwKPiAgCU5VTEwKPiAgfTsKPiAgRVhQT1JUX1NZTUJPTF9H
UEwoYXRhX2NvbW1vbl9zZGV2X2F0dHJzKTsKPiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS9saW51eC9s
aWJhdGEuaCBiL2luY2x1ZGUvbGludXgvbGliYXRhLmgKPiBpbmRleCA5MGI2OWE2Li5jMTcwYmU1
IDEwMDY0NAo+IC0tLSBhL2luY2x1ZGUvbGludXgvbGliYXRhLmgKPiArKysgYi9pbmNsdWRlL2xp
bnV4L2xpYmF0YS5oCj4gQEAgLTE2Nyw2ICsxNjcsNyBAQCBlbnVtIHsKPiAgCUFUQV9ERkxBR19V
TkxPQ0tfSFBBCT0gKDEgPDwgMTgpLCAvKiB1bmxvY2sgSFBBICovCj4gIAlBVEFfREZMQUdfTkNR
X1NFTkRfUkVDViA9ICgxIDw8IDE5KSwgLyogZGV2aWNlIHN1cHBvcnRzIE5DUSBTRU5EIGFuZCBS
RUNWICovCj4gIAlBVEFfREZMQUdfTkNRX1BSSU8JPSAoMSA8PCAyMCksIC8qIGRldmljZSBzdXBw
b3J0cyBOQ1EgcHJpb3JpdHkgKi8KPiArCUFUQV9ERkxBR19OQ1FfUFJJT19FTkFCTEUgPSAoMSA8
PCAyMSksIC8qIFByaW9yaXR5IGNtZHMgc2VudCB0byBkZXYgKi8KPiAgCUFUQV9ERkxBR19JTklU
X01BU0sJPSAoMSA8PCAyNCkgLSAxLAo+ICAKPiAgCUFUQV9ERkxBR19ERVRBQ0gJPSAoMSA8PCAy
NCksCj4gQEAgLTU0NSw2ICs1NDYsNyBAQCB0eXBlZGVmIHZvaWQgKCphdGFfcG9zdHJlc2V0X2Zu
X3QpKHN0cnVjdCBhdGFfbGluayAqbGluaywgdW5zaWduZWQgaW50ICpjbGFzc2VzKQo+ICAKPiAg
ZXh0ZXJuIHN0cnVjdCBkZXZpY2VfYXR0cmlidXRlIGRldl9hdHRyX2xpbmtfcG93ZXJfbWFuYWdl
bWVudF9wb2xpY3k7Cj4gIGV4dGVybiBzdHJ1Y3QgZGV2aWNlX2F0dHJpYnV0ZSBkZXZfYXR0cl91
bmxvYWRfaGVhZHM7Cj4gK2V4dGVybiBzdHJ1Y3QgZGV2aWNlX2F0dHJpYnV0ZSBkZXZfYXR0cl9u
Y3FfcHJpb19lbmFibGU7Cj4gIGV4dGVybiBzdHJ1Y3QgZGV2aWNlX2F0dHJpYnV0ZSBkZXZfYXR0
cl9lbV9tZXNzYWdlX3R5cGU7Cj4gIGV4dGVybiBzdHJ1Y3QgZGV2aWNlX2F0dHJpYnV0ZSBkZXZf
YXR0cl9lbV9tZXNzYWdlOwo+ICBleHRlcm4gc3RydWN0IGRldmljZV9hdHRyaWJ1dGUgZGV2X2F0
dHJfc3dfYWN0aXZpdHk7Cj4gLS0gCj4gMi43LjQKPiAKV2VzdGVybiBEaWdpdGFsIENvcnBvcmF0
aW9uIChhbmQgaXRzIHN1YnNpZGlhcmllcykgRS1tYWlsIENvbmZpZGVudGlhbGl0eSBOb3RpY2Ug
JiBEaXNjbGFpbWVyOgoKVGhpcyBlLW1haWwgYW5kIGFueSBmaWxlcyB0cmFuc21pdHRlZCB3aXRo
IGl0IG1heSBjb250YWluIGNvbmZpZGVudGlhbCBvciBsZWdhbGx5IHByaXZpbGVnZWQgaW5mb3Jt
YXRpb24gb2YgV0RDIGFuZC9vciBpdHMgYWZmaWxpYXRlcywgYW5kIGFyZSBpbnRlbmRlZCBzb2xl
bHkgZm9yIHRoZSB1c2Ugb2YgdGhlIGluZGl2aWR1YWwgb3IgZW50aXR5IHRvIHdoaWNoIHRoZXkg
YXJlIGFkZHJlc3NlZC4gSWYgeW91IGFyZSBub3QgdGhlIGludGVuZGVkIHJlY2lwaWVudCwgYW55
IGRpc2Nsb3N1cmUsIGNvcHlpbmcsIGRpc3RyaWJ1dGlvbiBvciBhbnkgYWN0aW9uIHRha2VuIG9y
IG9taXR0ZWQgdG8gYmUgdGFrZW4gaW4gcmVsaWFuY2Ugb24gaXQsIGlzIHByb2hpYml0ZWQuIElm
IHlvdSBoYXZlIHJlY2VpdmVkIHRoaXMgZS1tYWlsIGluIGVycm9yLCBwbGVhc2Ugbm90aWZ5IHRo
ZSBzZW5kZXIgaW1tZWRpYXRlbHkgYW5kIGRlbGV0ZSB0aGUgZS1tYWlsIGluIGl0cyBlbnRpcmV0
eSBmcm9tIHlvdXIgc3lzdGVtLgo=