[PATCH 00/17] cxlflash: Enhancements and miscellaneous fixes

[PATCH 00/17] cxlflash: Enhancements and miscellaneous fixes

[Uma Krishnan]

This patch series contains miscellaneous patches and adds 4 port device
support. This series also includes patches to improve performance of the
driver in the legacy IO path.

This series is intended for 4.12 and is bisectable

Matthew R. Ochs (16):
  cxlflash: Separate RRQ processing from the RRQ interrupt handler
  cxlflash: Serialize RRQ access and support offlevel processing
  cxlflash: Implement IRQ polling for RRQ processing
  cxlflash: Update sysfs helper routines to pass config structure
  cxlflash: Support dynamic number of FC ports
  cxlflash: Remove port configuration assumptions
  cxlflash: Hide FC internals behind common access routine
  cxlflash: SISlite updates to support 4 ports
  cxlflash: Support up to 4 ports
  cxlflash: Fence EEH during probe
  cxlflash: Remove unnecessary DMA mapping
  cxlflash: Fix power-of-two validations
  cxlflash: Fix warnings/errors
  cxlflash: Improve asynchronous interrupt processing
  cxlflash: Add hardware queues attribute
  cxlflash: Introduce hardware queue steering

Uma Krishnan (1):
  cxlflash: Support multiple hardware queues

 Documentation/powerpc/cxlflash.txt |    5 +
 drivers/scsi/cxlflash/common.h     |  137 +++--
 drivers/scsi/cxlflash/lunmgt.c     |    4 +-
 drivers/scsi/cxlflash/main.c       | 1162 +++++++++++++++++++++++++++---------
 drivers/scsi/cxlflash/main.h       |    2 -
 drivers/scsi/cxlflash/sislite.h    |  124 ++--
 drivers/scsi/cxlflash/superpipe.c  |   16 +-
 drivers/scsi/cxlflash/superpipe.h  |   56 +-
 drivers/scsi/cxlflash/vlun.c       |   99 +--
 drivers/scsi/cxlflash/vlun.h       |    2 +-
 10 files changed, 1182 insertions(+), 425 deletions(-)

-- 
2.1.0

[PATCH 01/17] cxlflash: Separate RRQ processing from the RRQ interrupt handler

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

In order to support processing the HRRQ by other means (e.g. polling),
the processing portion of the current RRQ interrupt handler needs to be
broken out into a separate routine. This will allow RRQ processing from
places other than the RRQ hardware interrupt handler.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/main.c | 27 +++++++++++++++++++++------
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 3061d80..30c09593c 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1155,19 +1155,18 @@ static irqreturn_t cxlflash_sync_err_irq(int irq, void *data)
 }
 
 /**
- * cxlflash_rrq_irq() - interrupt handler for read-response queue (normal path)
- * @irq:	Interrupt number.
- * @data:	Private data provided at interrupt registration, the AFU.
+ * process_hrrq() - process the read-response queue
+ * @afu:	AFU associated with the host.
  *
- * Return: Always return IRQ_HANDLED.
+ * Return: The number of entries processed.
  */
-static irqreturn_t cxlflash_rrq_irq(int irq, void *data)
+static int process_hrrq(struct afu *afu)
 {
-	struct afu *afu = (struct afu *)data;
 	struct afu_cmd *cmd;
 	struct sisl_ioasa *ioasa;
 	struct sisl_ioarcb *ioarcb;
 	bool toggle = afu->toggle;
+	int num_hrrq = 0;
 	u64 entry,
 	    *hrrq_start = afu->hrrq_start,
 	    *hrrq_end = afu->hrrq_end,
@@ -1201,11 +1200,27 @@ static irqreturn_t cxlflash_rrq_irq(int irq, void *data)
 		}
 
 		atomic_inc(&afu->hsq_credits);
+		num_hrrq++;
 	}
 
 	afu->hrrq_curr = hrrq_curr;
 	afu->toggle = toggle;
 
+	return num_hrrq;
+}
+
+/**
+ * cxlflash_rrq_irq() - interrupt handler for read-response queue (normal path)
+ * @irq:	Interrupt number.
+ * @data:	Private data provided at interrupt registration, the AFU.
+ *
+ * Return: Always return IRQ_HANDLED.
+ */
+static irqreturn_t cxlflash_rrq_irq(int irq, void *data)
+{
+	struct afu *afu = (struct afu *)data;
+
+	process_hrrq(afu);
 	return IRQ_HANDLED;
 }
 
-- 
2.1.0

[PATCH 02/17] cxlflash: Serialize RRQ access and support offlevel processing

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

As further staging to support processing the HRRQ by other means, access
to the HRRQ needs to be serialized by a disabled lock. This will allow
safe access in other non-hardware interrupt contexts. In an effort to
minimize the period where interrupts are disabled, support is added to
queue up commands harvested from the RRQ such that they can be processed
with hardware interrupts enabled. While this doesn't offer any improvement
with processing on a hardware interrupt it will help when IRQ polling is
supported and the command completions can execute on softirq context.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h |  2 ++
 drivers/scsi/cxlflash/main.c   | 42 +++++++++++++++++++++++++++++++++++-------
 2 files changed, 37 insertions(+), 7 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index d11dcc5..9d56b8c 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -134,6 +134,7 @@ struct afu_cmd {
 	struct afu *parent;
 	struct scsi_cmnd *scp;
 	struct completion cevent;
+	struct list_head queue;
 
 	u8 cmd_tmf:1;
 
@@ -181,6 +182,7 @@ struct afu {
 	struct sisl_ioarcb *hsq_start;
 	struct sisl_ioarcb *hsq_end;
 	struct sisl_ioarcb *hsq_curr;
+	spinlock_t hrrq_slock;
 	u64 *hrrq_start;
 	u64 *hrrq_end;
 	u64 *hrrq_curr;
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 30c09593c..8c207ba 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1157,10 +1157,13 @@ static irqreturn_t cxlflash_sync_err_irq(int irq, void *data)
 /**
  * process_hrrq() - process the read-response queue
  * @afu:	AFU associated with the host.
+ * @doneq:	Queue of commands harvested from the RRQ.
+ *
+ * This routine must be called holding the disabled RRQ spin lock.
  *
  * Return: The number of entries processed.
  */
-static int process_hrrq(struct afu *afu)
+static int process_hrrq(struct afu *afu, struct list_head *doneq)
 {
 	struct afu_cmd *cmd;
 	struct sisl_ioasa *ioasa;
@@ -1189,7 +1192,7 @@ static int process_hrrq(struct afu *afu)
 			cmd = container_of(ioarcb, struct afu_cmd, rcb);
 		}
 
-		cmd_complete(cmd);
+		list_add_tail(&cmd->queue, doneq);
 
 		/* Advance to next entry or wrap and flip the toggle bit */
 		if (hrrq_curr < hrrq_end)
@@ -1210,17 +1213,43 @@ static int process_hrrq(struct afu *afu)
 }
 
 /**
+ * process_cmd_doneq() - process a queue of harvested RRQ commands
+ * @doneq:	Queue of completed commands.
+ *
+ * Note that upon return the queue can no longer be trusted.
+ */
+static void process_cmd_doneq(struct list_head *doneq)
+{
+	struct afu_cmd *cmd, *tmp;
+
+	WARN_ON(list_empty(doneq));
+
+	list_for_each_entry_safe(cmd, tmp, doneq, queue)
+		cmd_complete(cmd);
+}
+
+/**
  * cxlflash_rrq_irq() - interrupt handler for read-response queue (normal path)
  * @irq:	Interrupt number.
  * @data:	Private data provided at interrupt registration, the AFU.
  *
- * Return: Always return IRQ_HANDLED.
+ * Return: IRQ_HANDLED or IRQ_NONE when no ready entries found.
  */
 static irqreturn_t cxlflash_rrq_irq(int irq, void *data)
 {
 	struct afu *afu = (struct afu *)data;
+	unsigned long hrrq_flags;
+	LIST_HEAD(doneq);
+	int num_entries = 0;
 
-	process_hrrq(afu);
+	spin_lock_irqsave(&afu->hrrq_slock, hrrq_flags);
+	num_entries = process_hrrq(afu, &doneq);
+	spin_unlock_irqrestore(&afu->hrrq_slock, hrrq_flags);
+
+	if (num_entries == 0)
+		return IRQ_NONE;
+
+	process_cmd_doneq(&doneq);
 	return IRQ_HANDLED;
 }
 
@@ -1540,14 +1569,13 @@ static int start_afu(struct cxlflash_cfg *cfg)
 
 	init_pcr(cfg);
 
-	/* After an AFU reset, RRQ entries are stale, clear them */
+	/* Initialize RRQ */
 	memset(&afu->rrq_entry, 0, sizeof(afu->rrq_entry));
-
-	/* Initialize RRQ pointers */
 	afu->hrrq_start = &afu->rrq_entry[0];
 	afu->hrrq_end = &afu->rrq_entry[NUM_RRQ_ENTRY - 1];
 	afu->hrrq_curr = afu->hrrq_start;
 	afu->toggle = 1;
+	spin_lock_init(&afu->hrrq_slock);
 
 	/* Initialize SQ */
 	if (afu_is_sq_cmd_mode(afu)) {
-- 
2.1.0

[PATCH 03/17] cxlflash: Implement IRQ polling for RRQ processing

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

Currently, RRQ processing takes place on hardware interrupt context. This
can be a heavy burden in some environments due to the overhead encountered
while completing RRQ entries. In an effort to improve system performance,
use the IRQ polling API to schedule this processing on softirq context.

This function will be disabled by default until starting values can be
established for the hardware supported by this driver.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h |   8 +++
 drivers/scsi/cxlflash/main.c   | 123 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 127 insertions(+), 4 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 9d56b8c..3ff05f1 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -15,6 +15,7 @@
 #ifndef _CXLFLASH_COMMON_H
 #define _CXLFLASH_COMMON_H
 
+#include <linux/irq_poll.h>
 #include <linux/list.h>
 #include <linux/rwsem.h>
 #include <linux/types.h>
@@ -196,10 +197,17 @@ struct afu {
 	char version[16];
 	u64 interface_version;
 
+	u32 irqpoll_weight;
+	struct irq_poll irqpoll;
 	struct cxlflash_cfg *parent; /* Pointer back to parent cxlflash_cfg */
 
 };
 
+static inline bool afu_is_irqpoll_enabled(struct afu *afu)
+{
+	return !!afu->irqpoll_weight;
+}
+
 static inline bool afu_is_cmd_mode(struct afu *afu, u64 cmd_mode)
 {
 	u64 afu_cap = afu->interface_version >> SISL_INTVER_CAP_SHIFT;
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 8c207ba..30d68af 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -554,7 +554,7 @@ static void free_mem(struct cxlflash_cfg *cfg)
  * Safe to call with AFU in a partially allocated/initialized state.
  *
  * Cancels scheduled worker threads, waits for any active internal AFU
- * commands to timeout and then unmaps the MMIO space.
+ * commands to timeout, disables IRQ polling and then unmaps the MMIO space.
  */
 static void stop_afu(struct cxlflash_cfg *cfg)
 {
@@ -565,6 +565,8 @@ static void stop_afu(struct cxlflash_cfg *cfg)
 	if (likely(afu)) {
 		while (atomic_read(&afu->cmds_active))
 			ssleep(1);
+		if (afu_is_irqpoll_enabled(afu))
+			irq_poll_disable(&afu->irqpoll);
 		if (likely(afu->afu_map)) {
 			cxl_psa_unmap((void __iomem *)afu->afu_map);
 			afu->afu_map = NULL;
@@ -1158,12 +1160,13 @@ static irqreturn_t cxlflash_sync_err_irq(int irq, void *data)
  * process_hrrq() - process the read-response queue
  * @afu:	AFU associated with the host.
  * @doneq:	Queue of commands harvested from the RRQ.
+ * @budget:	Threshold of RRQ entries to process.
  *
  * This routine must be called holding the disabled RRQ spin lock.
  *
  * Return: The number of entries processed.
  */
-static int process_hrrq(struct afu *afu, struct list_head *doneq)
+static int process_hrrq(struct afu *afu, struct list_head *doneq, int budget)
 {
 	struct afu_cmd *cmd;
 	struct sisl_ioasa *ioasa;
@@ -1175,7 +1178,7 @@ static int process_hrrq(struct afu *afu, struct list_head *doneq)
 	    *hrrq_end = afu->hrrq_end,
 	    *hrrq_curr = afu->hrrq_curr;
 
-	/* Process however many RRQ entries that are ready */
+	/* Process ready RRQ entries up to the specified budget (if any) */
 	while (true) {
 		entry = *hrrq_curr;
 
@@ -1204,6 +1207,9 @@ static int process_hrrq(struct afu *afu, struct list_head *doneq)
 
 		atomic_inc(&afu->hsq_credits);
 		num_hrrq++;
+
+		if (budget > 0 && num_hrrq >= budget)
+			break;
 	}
 
 	afu->hrrq_curr = hrrq_curr;
@@ -1229,6 +1235,32 @@ static void process_cmd_doneq(struct list_head *doneq)
 }
 
 /**
+ * cxlflash_irqpoll() - process a queue of harvested RRQ commands
+ * @irqpoll:	IRQ poll structure associated with queue to poll.
+ * @budget:	Threshold of RRQ entries to process per poll.
+ *
+ * Return: The number of entries processed.
+ */
+static int cxlflash_irqpoll(struct irq_poll *irqpoll, int budget)
+{
+	struct afu *afu = container_of(irqpoll, struct afu, irqpoll);
+	unsigned long hrrq_flags;
+	LIST_HEAD(doneq);
+	int num_entries = 0;
+
+	spin_lock_irqsave(&afu->hrrq_slock, hrrq_flags);
+
+	num_entries = process_hrrq(afu, &doneq, budget);
+	if (num_entries < budget)
+		irq_poll_complete(irqpoll);
+
+	spin_unlock_irqrestore(&afu->hrrq_slock, hrrq_flags);
+
+	process_cmd_doneq(&doneq);
+	return num_entries;
+}
+
+/**
  * cxlflash_rrq_irq() - interrupt handler for read-response queue (normal path)
  * @irq:	Interrupt number.
  * @data:	Private data provided at interrupt registration, the AFU.
@@ -1243,7 +1275,14 @@ static irqreturn_t cxlflash_rrq_irq(int irq, void *data)
 	int num_entries = 0;
 
 	spin_lock_irqsave(&afu->hrrq_slock, hrrq_flags);
-	num_entries = process_hrrq(afu, &doneq);
+
+	if (afu_is_irqpoll_enabled(afu)) {
+		irq_poll_sched(&afu->irqpoll);
+		spin_unlock_irqrestore(&afu->hrrq_slock, hrrq_flags);
+		return IRQ_HANDLED;
+	}
+
+	num_entries = process_hrrq(afu, &doneq, -1);
 	spin_unlock_irqrestore(&afu->hrrq_slock, hrrq_flags);
 
 	if (num_entries == 0)
@@ -1588,6 +1627,11 @@ static int start_afu(struct cxlflash_cfg *cfg)
 		atomic_set(&afu->hsq_credits, NUM_SQ_ENTRY - 1);
 	}
 
+	/* Initialize IRQ poll */
+	if (afu_is_irqpoll_enabled(afu))
+		irq_poll_init(&afu->irqpoll, afu->irqpoll_weight,
+			      cxlflash_irqpoll);
+
 	rc = init_global(cfg);
 
 	dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc);
@@ -2225,6 +2269,75 @@ static ssize_t port1_lun_table_show(struct device *dev,
 }
 
 /**
+ * irqpoll_weight_show() - presents the current IRQ poll weight for the host
+ * @dev:	Generic device associated with the host.
+ * @attr:	Device attribute representing the IRQ poll weight.
+ * @buf:	Buffer of length PAGE_SIZE to report back the current IRQ poll
+ *		weight in ASCII.
+ *
+ * An IRQ poll weight of 0 indicates polling is disabled.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t irqpoll_weight_show(struct device *dev,
+				   struct device_attribute *attr, char *buf)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+	struct afu *afu = cfg->afu;
+
+	return scnprintf(buf, PAGE_SIZE, "%u\n", afu->irqpoll_weight);
+}
+
+/**
+ * irqpoll_weight_store() - sets the current IRQ poll weight for the host
+ * @dev:	Generic device associated with the host.
+ * @attr:	Device attribute representing the IRQ poll weight.
+ * @buf:	Buffer of length PAGE_SIZE containing the desired IRQ poll
+ *		weight in ASCII.
+ * @count:	Length of data resizing in @buf.
+ *
+ * An IRQ poll weight of 0 indicates polling is disabled.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t irqpoll_weight_store(struct device *dev,
+				    struct device_attribute *attr,
+				    const char *buf, size_t count)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+	struct device *cfgdev = &cfg->dev->dev;
+	struct afu *afu = cfg->afu;
+	u32 weight;
+	int rc;
+
+	rc = kstrtouint(buf, 10, &weight);
+	if (rc)
+		return -EINVAL;
+
+	if (weight > 256) {
+		dev_info(cfgdev,
+			 "Invalid IRQ poll weight. It must be 256 or less.\n");
+		return -EINVAL;
+	}
+
+	if (weight == afu->irqpoll_weight) {
+		dev_info(cfgdev,
+			 "Current IRQ poll weight has the same weight.\n");
+		return -EINVAL;
+	}
+
+	if (afu_is_irqpoll_enabled(afu))
+		irq_poll_disable(&afu->irqpoll);
+
+	afu->irqpoll_weight = weight;
+
+	if (weight > 0)
+		irq_poll_init(&afu->irqpoll, weight, cxlflash_irqpoll);
+
+	return count;
+}
+
+/**
  * mode_show() - presents the current mode of the device
  * @dev:	Generic device associated with the device.
  * @attr:	Device attribute representing the device mode.
@@ -2250,6 +2363,7 @@ static DEVICE_ATTR_RW(lun_mode);
 static DEVICE_ATTR_RO(ioctl_version);
 static DEVICE_ATTR_RO(port0_lun_table);
 static DEVICE_ATTR_RO(port1_lun_table);
+static DEVICE_ATTR_RW(irqpoll_weight);
 
 static struct device_attribute *cxlflash_host_attrs[] = {
 	&dev_attr_port0,
@@ -2258,6 +2372,7 @@ static struct device_attribute *cxlflash_host_attrs[] = {
 	&dev_attr_ioctl_version,
 	&dev_attr_port0_lun_table,
 	&dev_attr_port1_lun_table,
+	&dev_attr_irqpoll_weight,
 	NULL
 };
 
-- 
2.1.0

[PATCH 04/17] cxlflash: Update sysfs helper routines to pass config structure

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

As staging for future function, pass the config pointer instead of the
AFU pointer for port-related sysfs helper routines.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/main.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 30d68af..157d806 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -2058,13 +2058,16 @@ static int cxlflash_change_queue_depth(struct scsi_device *sdev, int qdepth)
 /**
  * cxlflash_show_port_status() - queries and presents the current port status
  * @port:	Desired port for status reporting.
- * @afu:	AFU owning the specified port.
+ * @cfg:	Internal structure associated with the host.
  * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
  *
  * Return: The size of the ASCII string returned in @buf.
  */
-static ssize_t cxlflash_show_port_status(u32 port, struct afu *afu, char *buf)
+static ssize_t cxlflash_show_port_status(u32 port,
+					 struct cxlflash_cfg *cfg,
+					 char *buf)
 {
+	struct afu *afu = cfg->afu;
 	char *disp_status;
 	u64 status;
 	__be64 __iomem *fc_regs;
@@ -2099,9 +2102,8 @@ static ssize_t port0_show(struct device *dev,
 			  char *buf)
 {
 	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
-	struct afu *afu = cfg->afu;
 
-	return cxlflash_show_port_status(0, afu, buf);
+	return cxlflash_show_port_status(0, cfg, buf);
 }
 
 /**
@@ -2117,9 +2119,8 @@ static ssize_t port1_show(struct device *dev,
 			  char *buf)
 {
 	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
-	struct afu *afu = cfg->afu;
 
-	return cxlflash_show_port_status(1, afu, buf);
+	return cxlflash_show_port_status(1, cfg, buf);
 }
 
 /**
@@ -2208,15 +2209,16 @@ static ssize_t ioctl_version_show(struct device *dev,
 /**
  * cxlflash_show_port_lun_table() - queries and presents the port LUN table
  * @port:	Desired port for status reporting.
- * @afu:	AFU owning the specified port.
+ * @cfg:	Internal structure associated with the host.
  * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
  *
  * Return: The size of the ASCII string returned in @buf.
  */
 static ssize_t cxlflash_show_port_lun_table(u32 port,
-					    struct afu *afu,
+					    struct cxlflash_cfg *cfg,
 					    char *buf)
 {
+	struct afu *afu = cfg->afu;
 	int i;
 	ssize_t bytes = 0;
 	__be64 __iomem *fc_port;
@@ -2245,9 +2247,8 @@ static ssize_t port0_lun_table_show(struct device *dev,
 				    char *buf)
 {
 	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
-	struct afu *afu = cfg->afu;
 
-	return cxlflash_show_port_lun_table(0, afu, buf);
+	return cxlflash_show_port_lun_table(0, cfg, buf);
 }
 
 /**
@@ -2263,9 +2264,8 @@ static ssize_t port1_lun_table_show(struct device *dev,
 				    char *buf)
 {
 	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
-	struct afu *afu = cfg->afu;
 
-	return cxlflash_show_port_lun_table(1, afu, buf);
+	return cxlflash_show_port_lun_table(1, cfg, buf);
 }
 
 /**
-- 
2.1.0

[PATCH 05/17] cxlflash: Support dynamic number of FC ports

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

Transition from a static number of FC ports to a value that is derived
during probe. For now, a static value is used but this will later be
based on the type of card being configured.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h    |  7 ++--
 drivers/scsi/cxlflash/main.c      | 71 ++++++++++++++++++++++++---------------
 drivers/scsi/cxlflash/main.h      |  2 --
 drivers/scsi/cxlflash/sislite.h   |  1 +
 drivers/scsi/cxlflash/superpipe.h |  2 +-
 5 files changed, 51 insertions(+), 32 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 3ff05f1..6a04867 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -25,7 +25,9 @@
 
 extern const struct file_operations cxlflash_cxl_fops;
 
-#define MAX_CONTEXT  CXLFLASH_MAX_CONTEXT       /* num contexts per afu */
+#define MAX_CONTEXT	CXLFLASH_MAX_CONTEXT	/* num contexts per afu */
+#define NUM_FC_PORTS	CXLFLASH_NUM_FC_PORTS	/* ports per AFU */
+#define MAX_FC_PORTS	CXLFLASH_MAX_FC_PORTS	/* ports per AFU */
 
 #define CXLFLASH_BLOCK_SIZE	4096	/* 4K blocks */
 #define CXLFLASH_MAX_XFER_SIZE	16777216	/* 16MB transfer */
@@ -98,6 +100,7 @@ struct cxlflash_cfg {
 	struct pci_dev *dev;
 	struct pci_device_id *dev_id;
 	struct Scsi_Host *host;
+	int num_fc_ports;
 
 	ulong cxlflash_regs_pci;
 
@@ -118,7 +121,7 @@ struct cxlflash_cfg {
 	struct file_operations cxl_fops;
 
 	/* Parameters that are LUN table related */
-	int last_lun_index[CXLFLASH_NUM_FC_PORTS];
+	int last_lun_index[MAX_FC_PORTS];
 	int promote_lun_index;
 	struct list_head lluns; /* list of llun_info structs */
 
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 157d806..3f9c869 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -689,7 +689,7 @@ static void notify_shutdown(struct cxlflash_cfg *cfg, bool wait)
 	global = &afu->afu_map->global;
 
 	/* Notify AFU */
-	for (i = 0; i < NUM_FC_PORTS; i++) {
+	for (i = 0; i < cfg->num_fc_ports; i++) {
 		reg = readq_be(&global->fc_regs[i][FC_CONFIG2 / 8]);
 		reg |= SISL_FC_SHUTDOWN_NORMAL;
 		writeq_be(reg, &global->fc_regs[i][FC_CONFIG2 / 8]);
@@ -699,7 +699,7 @@ static void notify_shutdown(struct cxlflash_cfg *cfg, bool wait)
 		return;
 
 	/* Wait up to 1.5 seconds for shutdown processing to complete */
-	for (i = 0; i < NUM_FC_PORTS; i++) {
+	for (i = 0; i < cfg->num_fc_ports; i++) {
 		retry_cnt = 0;
 		while (true) {
 			status = readq_be(&global->fc_regs[i][FC_STATUS / 8]);
@@ -1072,6 +1072,7 @@ static const struct asyc_intr_info *find_ainfo(u64 status)
  */
 static void afu_err_intr_init(struct afu *afu)
 {
+	struct cxlflash_cfg *cfg = afu->parent;
 	int i;
 	u64 reg;
 
@@ -1107,7 +1108,7 @@ static void afu_err_intr_init(struct afu *afu)
 	writeq_be(reg, &afu->afu_map->global.fc_regs[0][FC_CONFIG2 / 8]);
 
 	/* now clear FC errors */
-	for (i = 0; i < NUM_FC_PORTS; i++) {
+	for (i = 0; i < cfg->num_fc_ports; i++) {
 		writeq_be(0xFFFFFFFFU,
 			  &afu->afu_map->global.fc_regs[i][FC_ERROR / 8]);
 		writeq_be(0, &afu->afu_map->global.fc_regs[i][FC_ERRCAP / 8]);
@@ -1394,7 +1395,7 @@ static int start_context(struct cxlflash_cfg *cfg)
 /**
  * read_vpd() - obtains the WWPNs from VPD
  * @cfg:	Internal structure associated with the host.
- * @wwpn:	Array of size NUM_FC_PORTS to pass back WWPNs
+ * @wwpn:	Array of size MAX_FC_PORTS to pass back WWPNs
  *
  * Return: 0 on success, -errno on failure
  */
@@ -1407,7 +1408,7 @@ static int read_vpd(struct cxlflash_cfg *cfg, u64 wwpn[])
 	ssize_t vpd_size;
 	char vpd_data[CXLFLASH_VPD_LEN];
 	char tmp_buf[WWPN_BUF_LEN] = { 0 };
-	char *wwpn_vpd_tags[NUM_FC_PORTS] = { "V5", "V6" };
+	char *wwpn_vpd_tags[MAX_FC_PORTS] = { "V5", "V6" };
 
 	/* Get the VPD data from the device */
 	vpd_size = cxl_read_adapter_vpd(pdev, vpd_data, sizeof(vpd_data));
@@ -1445,7 +1446,7 @@ static int read_vpd(struct cxlflash_cfg *cfg, u64 wwpn[])
 	 * because the conversion service requires that the ASCII
 	 * string be terminated.
 	 */
-	for (k = 0; k < NUM_FC_PORTS; k++) {
+	for (k = 0; k < cfg->num_fc_ports; k++) {
 		j = ro_size;
 		i = ro_start + PCI_VPD_LRDT_TAG_SIZE;
 
@@ -1474,6 +1475,8 @@ static int read_vpd(struct cxlflash_cfg *cfg, u64 wwpn[])
 			rc = -ENODEV;
 			goto out;
 		}
+
+		dev_dbg(dev, "%s: wwpn%d=%016llx\n", __func__, k, wwpn[k]);
 	}
 
 out:
@@ -1520,7 +1523,7 @@ static int init_global(struct cxlflash_cfg *cfg)
 {
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
-	u64 wwpn[NUM_FC_PORTS];	/* wwpn of AFU ports */
+	u64 wwpn[MAX_FC_PORTS];	/* wwpn of AFU ports */
 	int i = 0, num_ports = 0;
 	int rc = 0;
 	u64 reg;
@@ -1531,9 +1534,6 @@ static int init_global(struct cxlflash_cfg *cfg)
 		goto out;
 	}
 
-	dev_dbg(dev, "%s: wwpn0=%016llx wwpn1=%016llx\n",
-		__func__, wwpn[0], wwpn[1]);
-
 	/* Set up RRQ and SQ in AFU for master issued cmds */
 	writeq_be((u64) afu->hrrq_start, &afu->host_map->rrq_start);
 	writeq_be((u64) afu->hrrq_end, &afu->host_map->rrq_end);
@@ -1556,10 +1556,10 @@ static int init_global(struct cxlflash_cfg *cfg)
 	if (afu->internal_lun) {
 		/* Only use port 0 */
 		writeq_be(PORT0, &afu->afu_map->global.regs.afu_port_sel);
-		num_ports = NUM_FC_PORTS - 1;
+		num_ports = 0;
 	} else {
 		writeq_be(BOTH_PORTS, &afu->afu_map->global.regs.afu_port_sel);
-		num_ports = NUM_FC_PORTS;
+		num_ports = cfg->num_fc_ports;
 	}
 
 	for (i = 0; i < num_ports; i++) {
@@ -2061,19 +2061,25 @@ static int cxlflash_change_queue_depth(struct scsi_device *sdev, int qdepth)
  * @cfg:	Internal structure associated with the host.
  * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
  *
- * Return: The size of the ASCII string returned in @buf.
+ * Return: The size of the ASCII string returned in @buf or -EINVAL.
  */
 static ssize_t cxlflash_show_port_status(u32 port,
 					 struct cxlflash_cfg *cfg,
 					 char *buf)
 {
+	struct device *dev = &cfg->dev->dev;
 	struct afu *afu = cfg->afu;
 	char *disp_status;
 	u64 status;
 	__be64 __iomem *fc_regs;
 
-	if (port >= NUM_FC_PORTS)
-		return 0;
+	WARN_ON(port >= MAX_FC_PORTS);
+
+	if (port >= cfg->num_fc_ports) {
+		dev_info(dev, "%s: Port %d not supported on this card.\n",
+			__func__, port);
+		return -EINVAL;
+	}
 
 	fc_regs = &afu->afu_map->global.fc_regs[port][0];
 	status = readq_be(&fc_regs[FC_MTIP_STATUS / 8]);
@@ -2178,12 +2184,13 @@ static ssize_t lun_mode_store(struct device *dev,
 
 		/*
 		 * When configured for internal LUN, there is only one channel,
-		 * channel number 0, else there will be 2 (default).
+		 * channel number 0, else there will be one less than the number
+		 * of fc ports for this card.
 		 */
 		if (afu->internal_lun)
 			shost->max_channel = 0;
 		else
-			shost->max_channel = NUM_FC_PORTS - 1;
+			shost->max_channel = cfg->num_fc_ports - 1;
 
 		afu_reset(cfg);
 		scsi_scan_host(cfg->host);
@@ -2212,19 +2219,25 @@ static ssize_t ioctl_version_show(struct device *dev,
  * @cfg:	Internal structure associated with the host.
  * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
  *
- * Return: The size of the ASCII string returned in @buf.
+ * Return: The size of the ASCII string returned in @buf or -EINVAL.
  */
 static ssize_t cxlflash_show_port_lun_table(u32 port,
 					    struct cxlflash_cfg *cfg,
 					    char *buf)
 {
+	struct device *dev = &cfg->dev->dev;
 	struct afu *afu = cfg->afu;
 	int i;
 	ssize_t bytes = 0;
 	__be64 __iomem *fc_port;
 
-	if (port >= NUM_FC_PORTS)
-		return 0;
+	WARN_ON(port >= MAX_FC_PORTS);
+
+	if (port >= cfg->num_fc_ports) {
+		dev_info(dev, "%s: Port %d not supported on this card.\n",
+			__func__, port);
+		return -EINVAL;
+	}
 
 	fc_port = &afu->afu_map->global.fc_port[port][0];
 
@@ -2499,6 +2512,7 @@ static int cxlflash_probe(struct pci_dev *pdev,
 	struct device *dev = &pdev->dev;
 	struct dev_dependent_vals *ddv;
 	int rc = 0;
+	int k;
 
 	dev_dbg(&pdev->dev, "%s: Found CXLFLASH with IRQ: %d\n",
 		__func__, pdev->irq);
@@ -2531,17 +2545,20 @@ static int cxlflash_probe(struct pci_dev *pdev,
 
 	cfg->init_state = INIT_STATE_NONE;
 	cfg->dev = pdev;
+	cfg->num_fc_ports = NUM_FC_PORTS;
 	cfg->cxl_fops = cxlflash_cxl_fops;
 
 	/*
-	 * The promoted LUNs move to the top of the LUN table. The rest stay
-	 * on the bottom half. The bottom half grows from the end
-	 * (index = 255), whereas the top half grows from the beginning
-	 * (index = 0).
+	 * Promoted LUNs move to the top of the LUN table. The rest stay on
+	 * the bottom half. The bottom half grows from the end (index = 255),
+	 * whereas the top half grows from the beginning (index = 0).
+	 *
+	 * Initialize the last LUN index for all possible ports.
 	 */
-	cfg->promote_lun_index  = 0;
-	cfg->last_lun_index[0] = CXLFLASH_NUM_VLUNS/2 - 1;
-	cfg->last_lun_index[1] = CXLFLASH_NUM_VLUNS/2 - 1;
+	cfg->promote_lun_index = 0;
+
+	for (k = 0; k < MAX_FC_PORTS; k++)
+		cfg->last_lun_index[k] = CXLFLASH_NUM_VLUNS/2 - 1;
 
 	cfg->dev_id = (struct pci_device_id *)dev_id;
 
diff --git a/drivers/scsi/cxlflash/main.h b/drivers/scsi/cxlflash/main.h
index 0be2261..49657f1 100644
--- a/drivers/scsi/cxlflash/main.h
+++ b/drivers/scsi/cxlflash/main.h
@@ -37,8 +37,6 @@
 
 #define CXLFLASH_PCI_ERROR_RECOVERY_TIMEOUT	(120 * HZ)
 
-#define NUM_FC_PORTS	CXLFLASH_NUM_FC_PORTS	/* ports per AFU */
-
 /* FC defines */
 #define FC_MTIP_CMDCONFIG 0x010
 #define FC_MTIP_STATUS 0x018
diff --git a/drivers/scsi/cxlflash/sislite.h b/drivers/scsi/cxlflash/sislite.h
index a6e48a8..0db4bc1 100644
--- a/drivers/scsi/cxlflash/sislite.h
+++ b/drivers/scsi/cxlflash/sislite.h
@@ -367,6 +367,7 @@ struct sisl_global_regs {
 #define SISL_INTVER_CAP_RESERVED_CMD_MODE_B	0x100000000000ULL
 };
 
+#define CXLFLASH_MAX_FC_PORTS   2
 #define CXLFLASH_NUM_FC_PORTS   2
 #define CXLFLASH_MAX_CONTEXT  512	/* how many contexts per afu */
 #define CXLFLASH_NUM_VLUNS    512
diff --git a/drivers/scsi/cxlflash/superpipe.h b/drivers/scsi/cxlflash/superpipe.h
index 9e62ff3..690ce9c 100644
--- a/drivers/scsi/cxlflash/superpipe.h
+++ b/drivers/scsi/cxlflash/superpipe.h
@@ -59,7 +59,7 @@ struct glun_info {
 
 /* Local (per-adapter) lun_info structure */
 struct llun_info {
-	u64 lun_id[CXLFLASH_NUM_FC_PORTS]; /* from REPORT_LUNS */
+	u64 lun_id[MAX_FC_PORTS]; /* from REPORT_LUNS */
 	u32 lun_index;		/* Index in the LUN table */
 	u32 host_no;		/* host_no from Scsi_host */
 	u32 port_sel;		/* What port to use for this LUN */
-- 
2.1.0

[PATCH 06/17] cxlflash: Remove port configuration assumptions

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

At present, the cxlflash driver only supports hardware with two FC
ports. The code was initially designed with this assumption and is
dependent on having two FC ports - adding more ports will break logic
within the driver.

To mitigate this issue, remove the existing port assumptions and
transition the code to support more than two ports. As a side effect,
clarify the interpretation of the DK_CXLFLASH_ALL_PORTS_ACTIVE flag.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 Documentation/powerpc/cxlflash.txt |  5 +++
 drivers/scsi/cxlflash/common.h     |  4 ++
 drivers/scsi/cxlflash/lunmgt.c     |  4 +-
 drivers/scsi/cxlflash/main.c       | 13 +++---
 drivers/scsi/cxlflash/sislite.h    |  2 +-
 drivers/scsi/cxlflash/superpipe.c  |  2 +-
 drivers/scsi/cxlflash/superpipe.h  |  3 --
 drivers/scsi/cxlflash/vlun.c       | 89 +++++++++++++++++++++++++-------------
 8 files changed, 77 insertions(+), 45 deletions(-)

diff --git a/Documentation/powerpc/cxlflash.txt b/Documentation/powerpc/cxlflash.txt
index 6d9a2ed..66b4496 100644
--- a/Documentation/powerpc/cxlflash.txt
+++ b/Documentation/powerpc/cxlflash.txt
@@ -239,6 +239,11 @@ DK_CXLFLASH_USER_VIRTUAL
     resource handle that is provided is already referencing provisioned
     storage. This is reflected by the last LBA being a non-zero value.
 
+    When a LUN is accessible from more than one port, this ioctl will
+    return with the DK_CXLFLASH_ALL_PORTS_ACTIVE return flag set. This
+    provides the user with a hint that I/O can be retried in the event
+    of an I/O error as the LUN can be reached over multiple paths.
+
 DK_CXLFLASH_VLUN_RESIZE
 -----------------------
     This ioctl is responsible for resizing a previously created virtual
diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 6a04867..ee23e81 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -29,6 +29,10 @@ extern const struct file_operations cxlflash_cxl_fops;
 #define NUM_FC_PORTS	CXLFLASH_NUM_FC_PORTS	/* ports per AFU */
 #define MAX_FC_PORTS	CXLFLASH_MAX_FC_PORTS	/* ports per AFU */
 
+#define CHAN2PORTMASK(_x)	(1 << (_x))	/* channel to port mask */
+#define PORTMASK2CHAN(_x)	(ilog2((_x)))	/* port mask to channel */
+#define PORTNUM2CHAN(_x)	((_x) - 1)	/* port number to channel */
+
 #define CXLFLASH_BLOCK_SIZE	4096	/* 4K blocks */
 #define CXLFLASH_MAX_XFER_SIZE	16777216	/* 16MB transfer */
 #define CXLFLASH_MAX_SECTORS	(CXLFLASH_MAX_XFER_SIZE/512)	/* SCSI wants
diff --git a/drivers/scsi/cxlflash/lunmgt.c b/drivers/scsi/cxlflash/lunmgt.c
index 0efed17..4d232e2 100644
--- a/drivers/scsi/cxlflash/lunmgt.c
+++ b/drivers/scsi/cxlflash/lunmgt.c
@@ -252,7 +252,7 @@ int cxlflash_manage_lun(struct scsi_device *sdev,
 		 * in unpacked, AFU-friendly format, and hang LUN reference in
 		 * the sdev.
 		 */
-		lli->port_sel |= CHAN2PORT(chan);
+		lli->port_sel |= CHAN2PORTMASK(chan);
 		lli->lun_id[chan] = lun_to_lunid(sdev->lun);
 		sdev->hostdata = lli;
 	} else if (flags & DK_CXLFLASH_MANAGE_LUN_DISABLE_SUPERPIPE) {
@@ -264,7 +264,7 @@ int cxlflash_manage_lun(struct scsi_device *sdev,
 			 * tracking when no more references exist.
 			 */
 			sdev->hostdata = NULL;
-			lli->port_sel &= ~CHAN2PORT(chan);
+			lli->port_sel &= ~CHAN2PORTMASK(chan);
 			if (lli->port_sel == 0U)
 				lli->in_table = false;
 		}
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 3f9c869..04e1a8e 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -365,7 +365,6 @@ static int wait_resp(struct afu *afu, struct afu_cmd *cmd)
  */
 static int send_tmf(struct afu *afu, struct scsi_cmnd *scp, u64 tmfcmd)
 {
-	u32 port_sel = scp->device->channel + 1;
 	struct cxlflash_cfg *cfg = shost_priv(scp->device->host);
 	struct afu_cmd *cmd = sc_to_afucz(scp);
 	struct device *dev = &cfg->dev->dev;
@@ -388,7 +387,7 @@ static int send_tmf(struct afu *afu, struct scsi_cmnd *scp, u64 tmfcmd)
 
 	cmd->rcb.ctx_id = afu->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
-	cmd->rcb.port_sel = port_sel;
+	cmd->rcb.port_sel = CHAN2PORTMASK(scp->device->channel);
 	cmd->rcb.lun_id = lun_to_lunid(scp->device->lun);
 	cmd->rcb.req_flags = (SISL_REQ_FLAGS_PORT_LUN_ID |
 			      SISL_REQ_FLAGS_SUP_UNDERRUN |
@@ -444,7 +443,6 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	struct device *dev = &cfg->dev->dev;
 	struct afu_cmd *cmd = sc_to_afucz(scp);
 	struct scatterlist *sg = scsi_sglist(scp);
-	u32 port_sel = scp->device->channel + 1;
 	u16 req_flags = SISL_REQ_FLAGS_SUP_UNDERRUN;
 	ulong lock_flags;
 	int nseg = 0;
@@ -503,7 +501,7 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 
 	cmd->rcb.ctx_id = afu->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
-	cmd->rcb.port_sel = port_sel;
+	cmd->rcb.port_sel = CHAN2PORTMASK(scp->device->channel);
 	cmd->rcb.lun_id = lun_to_lunid(scp->device->lun);
 
 	if (scp->sc_data_direction == DMA_TO_DEVICE)
@@ -1558,7 +1556,8 @@ static int init_global(struct cxlflash_cfg *cfg)
 		writeq_be(PORT0, &afu->afu_map->global.regs.afu_port_sel);
 		num_ports = 0;
 	} else {
-		writeq_be(BOTH_PORTS, &afu->afu_map->global.regs.afu_port_sel);
+		writeq_be(PORT_MASK(cfg->num_fc_ports),
+			  &afu->afu_map->global.regs.afu_port_sel);
 		num_ports = cfg->num_fc_ports;
 	}
 
@@ -2190,7 +2189,7 @@ static ssize_t lun_mode_store(struct device *dev,
 		if (afu->internal_lun)
 			shost->max_channel = 0;
 		else
-			shost->max_channel = cfg->num_fc_ports - 1;
+			shost->max_channel = PORTNUM2CHAN(cfg->num_fc_ports);
 
 		afu_reset(cfg);
 		scsi_scan_host(cfg->host);
@@ -2529,7 +2528,7 @@ static int cxlflash_probe(struct pci_dev *pdev,
 
 	host->max_id = CXLFLASH_MAX_NUM_TARGETS_PER_BUS;
 	host->max_lun = CXLFLASH_MAX_NUM_LUNS_PER_TARGET;
-	host->max_channel = NUM_FC_PORTS - 1;
+	host->max_channel = PORTNUM2CHAN(NUM_FC_PORTS);
 	host->unique_id = host->host_no;
 	host->max_cmd_len = CXLFLASH_MAX_CDB_LEN;
 
diff --git a/drivers/scsi/cxlflash/sislite.h b/drivers/scsi/cxlflash/sislite.h
index 0db4bc1..f26f41b 100644
--- a/drivers/scsi/cxlflash/sislite.h
+++ b/drivers/scsi/cxlflash/sislite.h
@@ -479,7 +479,7 @@ struct sisl_rht_entry_f1 {
 
 #define PORT0  0x01U
 #define PORT1  0x02U
-#define BOTH_PORTS    (PORT0 | PORT1)
+#define PORT_MASK(_n)	((1 << (_n)) - 1)
 
 /* AFU Sync Mode byte */
 #define AFU_LW_SYNC 0x0U
diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index 90869ce..488330f 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -1935,7 +1935,7 @@ static int cxlflash_disk_direct_open(struct scsi_device *sdev, void *arg)
 	u64 lun_size = 0;
 	u64 last_lba = 0;
 	u64 rsrc_handle = -1;
-	u32 port = CHAN2PORT(sdev->channel);
+	u32 port = CHAN2PORTMASK(sdev->channel);
 
 	int rc = 0;
 
diff --git a/drivers/scsi/cxlflash/superpipe.h b/drivers/scsi/cxlflash/superpipe.h
index 690ce9c..8269ff8 100644
--- a/drivers/scsi/cxlflash/superpipe.h
+++ b/drivers/scsi/cxlflash/superpipe.h
@@ -33,9 +33,6 @@ extern struct cxlflash_global global;
 
 #define MAX_SECTOR_UNIT  512 /* max_sector is in 512 byte multiples */
 
-#define CHAN2PORT(_x)	((_x) + 1)
-#define PORT2CHAN(_x)	((_x) - 1)
-
 enum lun_mode {
 	MODE_NONE = 0,
 	MODE_VIRTUAL,
diff --git a/drivers/scsi/cxlflash/vlun.c b/drivers/scsi/cxlflash/vlun.c
index 8fcc804..547c8ff 100644
--- a/drivers/scsi/cxlflash/vlun.c
+++ b/drivers/scsi/cxlflash/vlun.c
@@ -819,8 +819,8 @@ int cxlflash_vlun_resize(struct scsi_device *sdev,
 void cxlflash_restore_luntable(struct cxlflash_cfg *cfg)
 {
 	struct llun_info *lli, *temp;
-	u32 chan;
 	u32 lind;
+	int k;
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
 	struct sisl_global_map __iomem *agm = &afu->afu_map->global;
@@ -832,33 +832,41 @@ void cxlflash_restore_luntable(struct cxlflash_cfg *cfg)
 			continue;
 
 		lind = lli->lun_index;
+		dev_dbg(dev, "%s: Virtual LUNs on slot %d:\n", __func__, lind);
 
-		if (lli->port_sel == BOTH_PORTS) {
-			writeq_be(lli->lun_id[0], &agm->fc_port[0][lind]);
-			writeq_be(lli->lun_id[1], &agm->fc_port[1][lind]);
-			dev_dbg(dev, "%s: Virtual LUN on slot %d  id0=%llx "
-				"id1=%llx\n", __func__, lind,
-				lli->lun_id[0], lli->lun_id[1]);
-		} else {
-			chan = PORT2CHAN(lli->port_sel);
-			writeq_be(lli->lun_id[chan], &agm->fc_port[chan][lind]);
-			dev_dbg(dev, "%s: Virtual LUN on slot %d chan=%d "
-				"id=%llx\n", __func__, lind, chan,
-				lli->lun_id[chan]);
-		}
+		for (k = 0; k < cfg->num_fc_ports; k++)
+			if (lli->port_sel & (1 << k)) {
+				writeq_be(lli->lun_id[k],
+					  &agm->fc_port[k][lind]);
+				dev_dbg(dev, "\t%d=%llx\n", k, lli->lun_id[k]);
+			}
 	}
 
 	mutex_unlock(&global.mutex);
 }
 
 /**
+ * get_num_ports() - compute number of ports from port selection mask
+ * @psm:	Port selection mask.
+ *
+ * Return: Population count of port selection mask
+ */
+static inline u8 get_num_ports(u32 psm)
+{
+	static const u8 bits[16] = { 0, 1, 1, 2, 1, 2, 2, 3,
+				     1, 2, 2, 3, 2, 3, 3, 4 };
+
+	return bits[psm & 0xf];
+}
+
+/**
  * init_luntable() - write an entry in the LUN table
  * @cfg:	Internal structure associated with the host.
  * @lli:	Per adapter LUN information structure.
  *
- * On successful return, a LUN table entry is created.
- * At the top for LUNs visible on both ports.
- * At the bottom for LUNs visible only on one port.
+ * On successful return, a LUN table entry is created:
+ *	- at the top for LUNs visible on multiple ports.
+ *	- at the bottom for LUNs visible only on one port.
  *
  * Return: 0 on success, -errno on failure
  */
@@ -866,7 +874,9 @@ static int init_luntable(struct cxlflash_cfg *cfg, struct llun_info *lli)
 {
 	u32 chan;
 	u32 lind;
+	u32 nports;
 	int rc = 0;
+	int k;
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
 	struct sisl_global_map __iomem *agm = &afu->afu_map->global;
@@ -876,29 +886,46 @@ static int init_luntable(struct cxlflash_cfg *cfg, struct llun_info *lli)
 	if (lli->in_table)
 		goto out;
 
-	if (lli->port_sel == BOTH_PORTS) {
+	nports = get_num_ports(lli->port_sel);
+	if (nports == 0 || nports > cfg->num_fc_ports) {
+		WARN(1, "Unsupported port configuration nports=%u", nports);
+		rc = -EIO;
+		goto out;
+	}
+
+	if (nports > 1) {
 		/*
-		 * If this LUN is visible from both ports, we will put
+		 * When LUN is visible from multiple ports, we will put
 		 * it in the top half of the LUN table.
 		 */
-		if ((cfg->promote_lun_index == cfg->last_lun_index[0]) ||
-		    (cfg->promote_lun_index == cfg->last_lun_index[1])) {
-			rc = -ENOSPC;
-			goto out;
+		for (k = 0; k < cfg->num_fc_ports; k++) {
+			if (!(lli->port_sel & (1 << k)))
+				continue;
+
+			if (cfg->promote_lun_index == cfg->last_lun_index[k]) {
+				rc = -ENOSPC;
+				goto out;
+			}
 		}
 
 		lind = lli->lun_index = cfg->promote_lun_index;
-		writeq_be(lli->lun_id[0], &agm->fc_port[0][lind]);
-		writeq_be(lli->lun_id[1], &agm->fc_port[1][lind]);
+		dev_dbg(dev, "%s: Virtual LUNs on slot %d:\n", __func__, lind);
+
+		for (k = 0; k < cfg->num_fc_ports; k++) {
+			if (!(lli->port_sel & (1 << k)))
+				continue;
+
+			writeq_be(lli->lun_id[k], &agm->fc_port[k][lind]);
+			dev_dbg(dev, "\t%d=%llx\n", k, lli->lun_id[k]);
+		}
+
 		cfg->promote_lun_index++;
-		dev_dbg(dev, "%s: Virtual LUN on slot %d  id0=%llx id1=%llx\n",
-			__func__, lind, lli->lun_id[0], lli->lun_id[1]);
 	} else {
 		/*
-		 * If this LUN is visible only from one port, we will put
+		 * When LUN is visible only from one port, we will put
 		 * it in the bottom half of the LUN table.
 		 */
-		chan = PORT2CHAN(lli->port_sel);
+		chan = PORTMASK2CHAN(lli->port_sel);
 		if (cfg->promote_lun_index == cfg->last_lun_index[chan]) {
 			rc = -ENOSPC;
 			goto out;
@@ -907,7 +934,7 @@ static int init_luntable(struct cxlflash_cfg *cfg, struct llun_info *lli)
 		lind = lli->lun_index = cfg->last_lun_index[chan];
 		writeq_be(lli->lun_id[chan], &agm->fc_port[chan][lind]);
 		cfg->last_lun_index[chan]--;
-		dev_dbg(dev, "%s: Virtual LUN on slot %d  chan=%d id=%llx\n",
+		dev_dbg(dev, "%s: Virtual LUNs on slot %d:\n\t%d=%llx\n",
 			__func__, lind, chan, lli->lun_id[chan]);
 	}
 
@@ -1016,7 +1043,7 @@ int cxlflash_disk_virtual_open(struct scsi_device *sdev, void *arg)
 	virt->last_lba = last_lba;
 	virt->rsrc_handle = rsrc_handle;
 
-	if (lli->port_sel == BOTH_PORTS)
+	if (get_num_ports(lli->port_sel) > 1)
 		virt->hdr.return_flags |= DK_CXLFLASH_ALL_PORTS_ACTIVE;
 out:
 	if (likely(ctxi))
-- 
2.1.0

[PATCH 07/17] cxlflash: Hide FC internals behind common access routine

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

As staging to support FC-related updates to the SISlite specification,
introduce helper routines to obtain references to FC resources that exist
within the global map. This will allow changes to the underlying global
map structure without impacting existing code paths.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h | 14 ++++++++
 drivers/scsi/cxlflash/main.c   | 72 +++++++++++++++++++++++-------------------
 drivers/scsi/cxlflash/vlun.c   | 16 +++++-----
 3 files changed, 61 insertions(+), 41 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index ee23e81..e6a7c97 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -240,6 +240,20 @@ static inline u64 lun_to_lunid(u64 lun)
 	return be64_to_cpu(lun_id);
 }
 
+static inline __be64 __iomem *get_fc_port_regs(struct cxlflash_cfg *cfg, int i)
+{
+	struct afu *afu = cfg->afu;
+
+	return &afu->afu_map->global.fc_regs[i][0];
+}
+
+static inline __be64 __iomem *get_fc_port_luns(struct cxlflash_cfg *cfg, int i)
+{
+	struct afu *afu = cfg->afu;
+
+	return &afu->afu_map->global.fc_port[i][0];
+}
+
 int cxlflash_afu_sync(struct afu *, ctx_hndl_t, res_hndl_t, u8);
 void cxlflash_list_init(void);
 void cxlflash_term_global_luns(void);
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 04e1a8e..e198605 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -670,8 +670,8 @@ static void notify_shutdown(struct cxlflash_cfg *cfg, bool wait)
 {
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
-	struct sisl_global_map __iomem *global;
 	struct dev_dependent_vals *ddv;
+	__be64 __iomem *fc_port_regs;
 	u64 reg, status;
 	int i, retry_cnt = 0;
 
@@ -684,13 +684,13 @@ static void notify_shutdown(struct cxlflash_cfg *cfg, bool wait)
 		return;
 	}
 
-	global = &afu->afu_map->global;
-
 	/* Notify AFU */
 	for (i = 0; i < cfg->num_fc_ports; i++) {
-		reg = readq_be(&global->fc_regs[i][FC_CONFIG2 / 8]);
+		fc_port_regs = get_fc_port_regs(cfg, i);
+
+		reg = readq_be(&fc_port_regs[FC_CONFIG2 / 8]);
 		reg |= SISL_FC_SHUTDOWN_NORMAL;
-		writeq_be(reg, &global->fc_regs[i][FC_CONFIG2 / 8]);
+		writeq_be(reg, &fc_port_regs[FC_CONFIG2 / 8]);
 	}
 
 	if (!wait)
@@ -698,9 +698,11 @@ static void notify_shutdown(struct cxlflash_cfg *cfg, bool wait)
 
 	/* Wait up to 1.5 seconds for shutdown processing to complete */
 	for (i = 0; i < cfg->num_fc_ports; i++) {
+		fc_port_regs = get_fc_port_regs(cfg, i);
 		retry_cnt = 0;
+
 		while (true) {
-			status = readq_be(&global->fc_regs[i][FC_STATUS / 8]);
+			status = readq_be(&fc_port_regs[FC_STATUS / 8]);
 			if (status & SISL_STATUS_SHUTDOWN_COMPLETE)
 				break;
 			if (++retry_cnt >= MC_RETRY_CNT) {
@@ -1071,6 +1073,7 @@ static const struct asyc_intr_info *find_ainfo(u64 status)
 static void afu_err_intr_init(struct afu *afu)
 {
 	struct cxlflash_cfg *cfg = afu->parent;
+	__be64 __iomem *fc_port_regs;
 	int i;
 	u64 reg;
 
@@ -1099,17 +1102,19 @@ static void afu_err_intr_init(struct afu *afu)
 	writeq_be(-1ULL, &afu->afu_map->global.regs.aintr_clear);
 
 	/* Clear/Set internal lun bits */
-	reg = readq_be(&afu->afu_map->global.fc_regs[0][FC_CONFIG2 / 8]);
+	fc_port_regs = get_fc_port_regs(cfg, 0);
+	reg = readq_be(&fc_port_regs[FC_CONFIG2 / 8]);
 	reg &= SISL_FC_INTERNAL_MASK;
 	if (afu->internal_lun)
 		reg |= ((u64)(afu->internal_lun - 1) << SISL_FC_INTERNAL_SHIFT);
-	writeq_be(reg, &afu->afu_map->global.fc_regs[0][FC_CONFIG2 / 8]);
+	writeq_be(reg, &fc_port_regs[FC_CONFIG2 / 8]);
 
 	/* now clear FC errors */
 	for (i = 0; i < cfg->num_fc_ports; i++) {
-		writeq_be(0xFFFFFFFFU,
-			  &afu->afu_map->global.fc_regs[i][FC_ERROR / 8]);
-		writeq_be(0, &afu->afu_map->global.fc_regs[i][FC_ERRCAP / 8]);
+		fc_port_regs = get_fc_port_regs(cfg, i);
+
+		writeq_be(0xFFFFFFFFU, &fc_port_regs[FC_ERROR / 8]);
+		writeq_be(0, &fc_port_regs[FC_ERRCAP / 8]);
 	}
 
 	/* sync interrupts for master's IOARRIN write */
@@ -1306,6 +1311,7 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 	u64 reg_unmasked;
 	const struct asyc_intr_info *info;
 	struct sisl_global_map __iomem *global = &afu->afu_map->global;
+	__be64 __iomem *fc_port_regs;
 	u64 reg;
 	u8 port;
 	int i;
@@ -1329,10 +1335,11 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 			continue;
 
 		port = info->port;
+		fc_port_regs = get_fc_port_regs(cfg, port);
 
 		dev_err(dev, "%s: FC Port %d -> %s, fc_status=%016llx\n",
 			__func__, port, info->desc,
-		       readq_be(&global->fc_regs[port][FC_STATUS / 8]));
+		       readq_be(&fc_port_regs[FC_STATUS / 8]));
 
 		/*
 		 * Do link reset first, some OTHER errors will set FC_ERROR
@@ -1347,7 +1354,7 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 		}
 
 		if (info->action & CLR_FC_ERROR) {
-			reg = readq_be(&global->fc_regs[port][FC_ERROR / 8]);
+			reg = readq_be(&fc_port_regs[FC_ERROR / 8]);
 
 			/*
 			 * Since all errors are unmasked, FC_ERROR and FC_ERRCAP
@@ -1357,8 +1364,8 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 			dev_err(dev, "%s: fc %d: clearing fc_error=%016llx\n",
 				__func__, port, reg);
 
-			writeq_be(reg, &global->fc_regs[port][FC_ERROR / 8]);
-			writeq_be(0, &global->fc_regs[port][FC_ERRCAP / 8]);
+			writeq_be(reg, &fc_port_regs[FC_ERROR / 8]);
+			writeq_be(0, &fc_port_regs[FC_ERRCAP / 8]);
 		}
 
 		if (info->action & SCAN_HOST) {
@@ -1521,6 +1528,7 @@ static int init_global(struct cxlflash_cfg *cfg)
 {
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	__be64 __iomem *fc_port_regs;
 	u64 wwpn[MAX_FC_PORTS];	/* wwpn of AFU ports */
 	int i = 0, num_ports = 0;
 	int rc = 0;
@@ -1562,19 +1570,17 @@ static int init_global(struct cxlflash_cfg *cfg)
 	}
 
 	for (i = 0; i < num_ports; i++) {
+		fc_port_regs = get_fc_port_regs(cfg, i);
+
 		/* Unmask all errors (but they are still masked at AFU) */
-		writeq_be(0, &afu->afu_map->global.fc_regs[i][FC_ERRMSK / 8]);
+		writeq_be(0, &fc_port_regs[FC_ERRMSK / 8]);
 		/* Clear CRC error cnt & set a threshold */
-		(void)readq_be(&afu->afu_map->global.
-			       fc_regs[i][FC_CNT_CRCERR / 8]);
-		writeq_be(MC_CRC_THRESH, &afu->afu_map->global.fc_regs[i]
-			  [FC_CRC_THRESH / 8]);
+		(void)readq_be(&fc_port_regs[FC_CNT_CRCERR / 8]);
+		writeq_be(MC_CRC_THRESH, &fc_port_regs[FC_CRC_THRESH / 8]);
 
 		/* Set WWPNs. If already programmed, wwpn[i] is 0 */
 		if (wwpn[i] != 0)
-			afu_set_wwpn(afu, i,
-				     &afu->afu_map->global.fc_regs[i][0],
-				     wwpn[i]);
+			afu_set_wwpn(afu, i, &fc_port_regs[0], wwpn[i]);
 		/* Programming WWPN back to back causes additional
 		 * offline/online transitions and a PLOGI
 		 */
@@ -2067,10 +2073,9 @@ static ssize_t cxlflash_show_port_status(u32 port,
 					 char *buf)
 {
 	struct device *dev = &cfg->dev->dev;
-	struct afu *afu = cfg->afu;
 	char *disp_status;
 	u64 status;
-	__be64 __iomem *fc_regs;
+	__be64 __iomem *fc_port_regs;
 
 	WARN_ON(port >= MAX_FC_PORTS);
 
@@ -2080,8 +2085,8 @@ static ssize_t cxlflash_show_port_status(u32 port,
 		return -EINVAL;
 	}
 
-	fc_regs = &afu->afu_map->global.fc_regs[port][0];
-	status = readq_be(&fc_regs[FC_MTIP_STATUS / 8]);
+	fc_port_regs = get_fc_port_regs(cfg, port);
+	status = readq_be(&fc_port_regs[FC_MTIP_STATUS / 8]);
 	status &= FC_MTIP_STATUS_MASK;
 
 	if (status == FC_MTIP_STATUS_ONLINE)
@@ -2225,10 +2230,9 @@ static ssize_t cxlflash_show_port_lun_table(u32 port,
 					    char *buf)
 {
 	struct device *dev = &cfg->dev->dev;
-	struct afu *afu = cfg->afu;
+	__be64 __iomem *fc_port_luns;
 	int i;
 	ssize_t bytes = 0;
-	__be64 __iomem *fc_port;
 
 	WARN_ON(port >= MAX_FC_PORTS);
 
@@ -2238,11 +2242,12 @@ static ssize_t cxlflash_show_port_lun_table(u32 port,
 		return -EINVAL;
 	}
 
-	fc_port = &afu->afu_map->global.fc_port[port][0];
+	fc_port_luns = get_fc_port_luns(cfg, port);
 
 	for (i = 0; i < CXLFLASH_NUM_VLUNS; i++)
 		bytes += scnprintf(buf + bytes, PAGE_SIZE - bytes,
-				   "%03d: %016llx\n", i, readq_be(&fc_port[i]));
+				   "%03d: %016llx\n",
+				   i, readq_be(&fc_port_luns[i]));
 	return bytes;
 }
 
@@ -2462,6 +2467,7 @@ static void cxlflash_worker_thread(struct work_struct *work)
 						work_q);
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	__be64 __iomem *fc_port_regs;
 	int port;
 	ulong lock_flags;
 
@@ -2482,8 +2488,8 @@ static void cxlflash_worker_thread(struct work_struct *work)
 					       lock_flags);
 
 			/* The reset can block... */
-			afu_link_reset(afu, port,
-				       &afu->afu_map->global.fc_regs[port][0]);
+			fc_port_regs = get_fc_port_regs(cfg, port);
+			afu_link_reset(afu, port, fc_port_regs);
 			spin_lock_irqsave(cfg->host->host_lock, lock_flags);
 		}
 
diff --git a/drivers/scsi/cxlflash/vlun.c b/drivers/scsi/cxlflash/vlun.c
index 547c8ff..9a6a9e9 100644
--- a/drivers/scsi/cxlflash/vlun.c
+++ b/drivers/scsi/cxlflash/vlun.c
@@ -821,9 +821,8 @@ void cxlflash_restore_luntable(struct cxlflash_cfg *cfg)
 	struct llun_info *lli, *temp;
 	u32 lind;
 	int k;
-	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
-	struct sisl_global_map __iomem *agm = &afu->afu_map->global;
+	__be64 __iomem *fc_port_luns;
 
 	mutex_lock(&global.mutex);
 
@@ -836,8 +835,8 @@ void cxlflash_restore_luntable(struct cxlflash_cfg *cfg)
 
 		for (k = 0; k < cfg->num_fc_ports; k++)
 			if (lli->port_sel & (1 << k)) {
-				writeq_be(lli->lun_id[k],
-					  &agm->fc_port[k][lind]);
+				fc_port_luns = get_fc_port_luns(cfg, k);
+				writeq_be(lli->lun_id[k], &fc_port_luns[lind]);
 				dev_dbg(dev, "\t%d=%llx\n", k, lli->lun_id[k]);
 			}
 	}
@@ -877,9 +876,8 @@ static int init_luntable(struct cxlflash_cfg *cfg, struct llun_info *lli)
 	u32 nports;
 	int rc = 0;
 	int k;
-	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
-	struct sisl_global_map __iomem *agm = &afu->afu_map->global;
+	__be64 __iomem *fc_port_luns;
 
 	mutex_lock(&global.mutex);
 
@@ -915,7 +913,8 @@ static int init_luntable(struct cxlflash_cfg *cfg, struct llun_info *lli)
 			if (!(lli->port_sel & (1 << k)))
 				continue;
 
-			writeq_be(lli->lun_id[k], &agm->fc_port[k][lind]);
+			fc_port_luns = get_fc_port_luns(cfg, k);
+			writeq_be(lli->lun_id[k], &fc_port_luns[lind]);
 			dev_dbg(dev, "\t%d=%llx\n", k, lli->lun_id[k]);
 		}
 
@@ -932,7 +931,8 @@ static int init_luntable(struct cxlflash_cfg *cfg, struct llun_info *lli)
 		}
 
 		lind = lli->lun_index = cfg->last_lun_index[chan];
-		writeq_be(lli->lun_id[chan], &agm->fc_port[chan][lind]);
+		fc_port_luns = get_fc_port_luns(cfg, chan);
+		writeq_be(lli->lun_id[chan], &fc_port_luns[lind]);
 		cfg->last_lun_index[chan]--;
 		dev_dbg(dev, "%s: Virtual LUNs on slot %d:\n\t%d=%llx\n",
 			__func__, lind, chan, lli->lun_id[chan]);
-- 
2.1.0

[PATCH 08/17] cxlflash: SISlite updates to support 4 ports

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

Update the SISlite header to support 4 ports as outlined in the
SISlite specification. Address fallout from structure renames and
refreshed organization throughout the driver. Determine the number
of ports supported by a card from the global port selection mask
register reset value.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h  | 25 ++++++++---
 drivers/scsi/cxlflash/main.c    | 77 +++++++++++++++++++++++++--------
 drivers/scsi/cxlflash/sislite.h | 96 ++++++++++++++++++++++++++++-------------
 3 files changed, 141 insertions(+), 57 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index e6a7c97..28bb716 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -26,8 +26,11 @@
 extern const struct file_operations cxlflash_cxl_fops;
 
 #define MAX_CONTEXT	CXLFLASH_MAX_CONTEXT	/* num contexts per afu */
-#define NUM_FC_PORTS	CXLFLASH_NUM_FC_PORTS	/* ports per AFU */
-#define MAX_FC_PORTS	CXLFLASH_MAX_FC_PORTS	/* ports per AFU */
+#define MAX_FC_PORTS	CXLFLASH_MAX_FC_PORTS	/* max ports per AFU */
+#define LEGACY_FC_PORTS	2			/* legacy ports per AFU */
+
+#define CHAN2PORTBANK(_x)	((_x) >> ilog2(CXLFLASH_NUM_FC_PORTS_PER_BANK))
+#define CHAN2BANKPORT(_x)	((_x) & (CXLFLASH_NUM_FC_PORTS_PER_BANK - 1))
 
 #define CHAN2PORTMASK(_x)	(1 << (_x))	/* channel to port mask */
 #define PORTMASK2CHAN(_x)	(ilog2((_x)))	/* port mask to channel */
@@ -67,7 +70,7 @@ extern const struct file_operations cxlflash_cxl_fops;
 
 static inline void check_sizes(void)
 {
-	BUILD_BUG_ON_NOT_POWER_OF_2(CXLFLASH_NUM_CMDS);
+	BUILD_BUG_ON_NOT_POWER_OF_2(CXLFLASH_NUM_FC_PORTS_PER_BANK);
 }
 
 /* AFU defines a fixed size of 4K for command buffers (borrow 4K page define) */
@@ -240,18 +243,26 @@ static inline u64 lun_to_lunid(u64 lun)
 	return be64_to_cpu(lun_id);
 }
 
-static inline __be64 __iomem *get_fc_port_regs(struct cxlflash_cfg *cfg, int i)
+static inline struct fc_port_bank __iomem *get_fc_port_bank(
+					    struct cxlflash_cfg *cfg, int i)
 {
 	struct afu *afu = cfg->afu;
 
-	return &afu->afu_map->global.fc_regs[i][0];
+	return &afu->afu_map->global.bank[CHAN2PORTBANK(i)];
+}
+
+static inline __be64 __iomem *get_fc_port_regs(struct cxlflash_cfg *cfg, int i)
+{
+	struct fc_port_bank __iomem *fcpb = get_fc_port_bank(cfg, i);
+
+	return &fcpb->fc_port_regs[CHAN2BANKPORT(i)][0];
 }
 
 static inline __be64 __iomem *get_fc_port_luns(struct cxlflash_cfg *cfg, int i)
 {
-	struct afu *afu = cfg->afu;
+	struct fc_port_bank __iomem *fcpb = get_fc_port_bank(cfg, i);
 
-	return &afu->afu_map->global.fc_port[i][0];
+	return &fcpb->fc_port_luns[CHAN2BANKPORT(i)][0];
 }
 
 int cxlflash_afu_sync(struct afu *, ctx_hndl_t, res_hndl_t, u8);
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index e198605..64ad76b 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1028,25 +1028,29 @@ static void afu_link_reset(struct afu *afu, int port, __be64 __iomem *fc_regs)
 
 /*
  * Asynchronous interrupt information table
+ *
+ * NOTE: The checkpatch script considers the BUILD_SISL_ASTATUS_FC_PORT macro
+ * as complex and complains because it is not wrapped with parentheses/braces.
  */
+#define ASTATUS_FC(_a, _b, _c, _d)					 \
+	{ SISL_ASTATUS_FC##_a##_##_b, _c, _a, (_d) }
+
+#define BUILD_SISL_ASTATUS_FC_PORT(_a)					 \
+	ASTATUS_FC(_a, OTHER, "other error", CLR_FC_ERROR | LINK_RESET), \
+	ASTATUS_FC(_a, LOGO, "target initiated LOGO", 0),		 \
+	ASTATUS_FC(_a, CRC_T, "CRC threshold exceeded", LINK_RESET),	 \
+	ASTATUS_FC(_a, LOGI_R, "login timed out, retrying", LINK_RESET), \
+	ASTATUS_FC(_a, LOGI_F, "login failed", CLR_FC_ERROR),		 \
+	ASTATUS_FC(_a, LOGI_S, "login succeeded", SCAN_HOST),		 \
+	ASTATUS_FC(_a, LINK_DN, "link down", 0),			 \
+	ASTATUS_FC(_a, LINK_UP, "link up", 0)
+
 static const struct asyc_intr_info ainfo[] = {
-	{SISL_ASTATUS_FC0_OTHER, "other error", 0, CLR_FC_ERROR | LINK_RESET},
-	{SISL_ASTATUS_FC0_LOGO, "target initiated LOGO", 0, 0},
-	{SISL_ASTATUS_FC0_CRC_T, "CRC threshold exceeded", 0, LINK_RESET},
-	{SISL_ASTATUS_FC0_LOGI_R, "login timed out, retrying", 0, LINK_RESET},
-	{SISL_ASTATUS_FC0_LOGI_F, "login failed", 0, CLR_FC_ERROR},
-	{SISL_ASTATUS_FC0_LOGI_S, "login succeeded", 0, SCAN_HOST},
-	{SISL_ASTATUS_FC0_LINK_DN, "link down", 0, 0},
-	{SISL_ASTATUS_FC0_LINK_UP, "link up", 0, 0},
-	{SISL_ASTATUS_FC1_OTHER, "other error", 1, CLR_FC_ERROR | LINK_RESET},
-	{SISL_ASTATUS_FC1_LOGO, "target initiated LOGO", 1, 0},
-	{SISL_ASTATUS_FC1_CRC_T, "CRC threshold exceeded", 1, LINK_RESET},
-	{SISL_ASTATUS_FC1_LOGI_R, "login timed out, retrying", 1, LINK_RESET},
-	{SISL_ASTATUS_FC1_LOGI_F, "login failed", 1, CLR_FC_ERROR},
-	{SISL_ASTATUS_FC1_LOGI_S, "login succeeded", 1, SCAN_HOST},
-	{SISL_ASTATUS_FC1_LINK_DN, "link down", 1, 0},
-	{SISL_ASTATUS_FC1_LINK_UP, "link up", 1, 0},
-	{0x0, "", 0, 0}		/* terminator */
+	BUILD_SISL_ASTATUS_FC_PORT(2),
+	BUILD_SISL_ASTATUS_FC_PORT(3),
+	BUILD_SISL_ASTATUS_FC_PORT(0),
+	BUILD_SISL_ASTATUS_FC_PORT(1),
+	{ 0x0, "", 0, 0 }
 };
 
 /**
@@ -1059,6 +1063,8 @@ static const struct asyc_intr_info *find_ainfo(u64 status)
 {
 	const struct asyc_intr_info *info;
 
+	BUILD_BUG_ON(ainfo[ARRAY_SIZE(ainfo) - 1].status != 0);
+
 	for (info = &ainfo[0]; info->status; info++)
 		if (info->status == status)
 			return info;
@@ -1747,6 +1753,39 @@ static int init_mc(struct cxlflash_cfg *cfg)
 }
 
 /**
+ * get_num_afu_ports() - determines and configures the number of AFU ports
+ * @cfg:	Internal structure associated with the host.
+ *
+ * This routine determines the number of AFU ports by converting the global
+ * port selection mask. The converted value is only valid following an AFU
+ * reset (explicit or power-on). This routine must be invoked shortly after
+ * mapping as other routines are dependent on the number of ports during the
+ * initialization sequence.
+ *
+ * To support legacy AFUs that might not have reflected an initial global
+ * port mask (value read is 0), default to the number of ports originally
+ * supported by the cxlflash driver (2) before hardware with other port
+ * offerings was introduced.
+ */
+static void get_num_afu_ports(struct cxlflash_cfg *cfg)
+{
+	struct afu *afu = cfg->afu;
+	struct device *dev = &cfg->dev->dev;
+	u64 port_mask;
+	int num_fc_ports = LEGACY_FC_PORTS;
+
+	port_mask = readq_be(&afu->afu_map->global.regs.afu_port_sel);
+	if (port_mask != 0ULL)
+		num_fc_ports = min(ilog2(port_mask) + 1, MAX_FC_PORTS);
+
+	dev_dbg(dev, "%s: port_mask=%016llx num_fc_ports=%d\n",
+		__func__, port_mask, num_fc_ports);
+
+	cfg->num_fc_ports = num_fc_ports;
+	cfg->host->max_channel = PORTNUM2CHAN(num_fc_ports);
+}
+
+/**
  * init_afu() - setup as master context and start AFU
  * @cfg:	Internal structure associated with the host.
  *
@@ -1803,6 +1842,8 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	dev_dbg(dev, "%s: afu_ver=%s interface_ver=%016llx\n", __func__,
 		afu->version, afu->interface_version);
 
+	get_num_afu_ports(cfg);
+
 	rc = start_afu(cfg);
 	if (rc) {
 		dev_err(dev, "%s: start_afu failed, rc=%d\n", __func__, rc);
@@ -2534,7 +2575,6 @@ static int cxlflash_probe(struct pci_dev *pdev,
 
 	host->max_id = CXLFLASH_MAX_NUM_TARGETS_PER_BUS;
 	host->max_lun = CXLFLASH_MAX_NUM_LUNS_PER_TARGET;
-	host->max_channel = PORTNUM2CHAN(NUM_FC_PORTS);
 	host->unique_id = host->host_no;
 	host->max_cmd_len = CXLFLASH_MAX_CDB_LEN;
 
@@ -2550,7 +2590,6 @@ static int cxlflash_probe(struct pci_dev *pdev,
 
 	cfg->init_state = INIT_STATE_NONE;
 	cfg->dev = pdev;
-	cfg->num_fc_ports = NUM_FC_PORTS;
 	cfg->cxl_fops = cxlflash_cxl_fops;
 
 	/*
diff --git a/drivers/scsi/cxlflash/sislite.h b/drivers/scsi/cxlflash/sislite.h
index f26f41b..42d9c9e 100644
--- a/drivers/scsi/cxlflash/sislite.h
+++ b/drivers/scsi/cxlflash/sislite.h
@@ -292,28 +292,54 @@ struct sisl_ctrl_map {
 /* single copy global regs */
 struct sisl_global_regs {
 	__be64 aintr_status;
-	/* In cxlflash, each FC port/link gets a byte of status */
-#define SISL_ASTATUS_FC0_OTHER	 0x8000ULL /* b48, other err,
-					      FC_ERRCAP[31:20] */
-#define SISL_ASTATUS_FC0_LOGO    0x4000ULL /* b49, target sent FLOGI/PLOGI/LOGO
-						   while logged in */
-#define SISL_ASTATUS_FC0_CRC_T   0x2000ULL /* b50, CRC threshold exceeded */
-#define SISL_ASTATUS_FC0_LOGI_R  0x1000ULL /* b51, login state machine timed out
-						   and retrying */
-#define SISL_ASTATUS_FC0_LOGI_F  0x0800ULL /* b52, login failed,
-					      FC_ERROR[19:0] */
-#define SISL_ASTATUS_FC0_LOGI_S  0x0400ULL /* b53, login succeeded */
-#define SISL_ASTATUS_FC0_LINK_DN 0x0200ULL /* b54, link online to offline */
-#define SISL_ASTATUS_FC0_LINK_UP 0x0100ULL /* b55, link offline to online */
-
-#define SISL_ASTATUS_FC1_OTHER   0x0080ULL /* b56 */
-#define SISL_ASTATUS_FC1_LOGO    0x0040ULL /* b57 */
-#define SISL_ASTATUS_FC1_CRC_T   0x0020ULL /* b58 */
-#define SISL_ASTATUS_FC1_LOGI_R  0x0010ULL /* b59 */
-#define SISL_ASTATUS_FC1_LOGI_F  0x0008ULL /* b60 */
-#define SISL_ASTATUS_FC1_LOGI_S  0x0004ULL /* b61 */
-#define SISL_ASTATUS_FC1_LINK_DN 0x0002ULL /* b62 */
-#define SISL_ASTATUS_FC1_LINK_UP 0x0001ULL /* b63 */
+	/*
+	 * In cxlflash, FC port/link are arranged in port pairs, each
+	 * gets a byte of status:
+	 *
+	 *	*_OTHER:	other err, FC_ERRCAP[31:20]
+	 *	*_LOGO:		target sent FLOGI/PLOGI/LOGO while logged in
+	 *	*_CRC_T:	CRC threshold exceeded
+	 *	*_LOGI_R:	login state machine timed out and retrying
+	 *	*_LOGI_F:	login failed, FC_ERROR[19:0]
+	 *	*_LOGI_S:	login succeeded
+	 *	*_LINK_DN:	link online to offline
+	 *	*_LINK_UP:	link offline to online
+	 */
+#define SISL_ASTATUS_FC2_OTHER	 0x80000000ULL /* b32 */
+#define SISL_ASTATUS_FC2_LOGO    0x40000000ULL /* b33 */
+#define SISL_ASTATUS_FC2_CRC_T   0x20000000ULL /* b34 */
+#define SISL_ASTATUS_FC2_LOGI_R  0x10000000ULL /* b35 */
+#define SISL_ASTATUS_FC2_LOGI_F  0x08000000ULL /* b36 */
+#define SISL_ASTATUS_FC2_LOGI_S  0x04000000ULL /* b37 */
+#define SISL_ASTATUS_FC2_LINK_DN 0x02000000ULL /* b38 */
+#define SISL_ASTATUS_FC2_LINK_UP 0x01000000ULL /* b39 */
+
+#define SISL_ASTATUS_FC3_OTHER   0x00800000ULL /* b40 */
+#define SISL_ASTATUS_FC3_LOGO    0x00400000ULL /* b41 */
+#define SISL_ASTATUS_FC3_CRC_T   0x00200000ULL /* b42 */
+#define SISL_ASTATUS_FC3_LOGI_R  0x00100000ULL /* b43 */
+#define SISL_ASTATUS_FC3_LOGI_F  0x00080000ULL /* b44 */
+#define SISL_ASTATUS_FC3_LOGI_S  0x00040000ULL /* b45 */
+#define SISL_ASTATUS_FC3_LINK_DN 0x00020000ULL /* b46 */
+#define SISL_ASTATUS_FC3_LINK_UP 0x00010000ULL /* b47 */
+
+#define SISL_ASTATUS_FC0_OTHER	 0x00008000ULL /* b48 */
+#define SISL_ASTATUS_FC0_LOGO    0x00004000ULL /* b49 */
+#define SISL_ASTATUS_FC0_CRC_T   0x00002000ULL /* b50 */
+#define SISL_ASTATUS_FC0_LOGI_R  0x00001000ULL /* b51 */
+#define SISL_ASTATUS_FC0_LOGI_F  0x00000800ULL /* b52 */
+#define SISL_ASTATUS_FC0_LOGI_S  0x00000400ULL /* b53 */
+#define SISL_ASTATUS_FC0_LINK_DN 0x00000200ULL /* b54 */
+#define SISL_ASTATUS_FC0_LINK_UP 0x00000100ULL /* b55 */
+
+#define SISL_ASTATUS_FC1_OTHER   0x00000080ULL /* b56 */
+#define SISL_ASTATUS_FC1_LOGO    0x00000040ULL /* b57 */
+#define SISL_ASTATUS_FC1_CRC_T   0x00000020ULL /* b58 */
+#define SISL_ASTATUS_FC1_LOGI_R  0x00000010ULL /* b59 */
+#define SISL_ASTATUS_FC1_LOGI_F  0x00000008ULL /* b60 */
+#define SISL_ASTATUS_FC1_LOGI_S  0x00000004ULL /* b61 */
+#define SISL_ASTATUS_FC1_LINK_DN 0x00000002ULL /* b62 */
+#define SISL_ASTATUS_FC1_LINK_UP 0x00000001ULL /* b63 */
 
 #define SISL_FC_INTERNAL_UNMASK	0x0000000300000000ULL	/* 1 means unmasked */
 #define SISL_FC_INTERNAL_MASK	~(SISL_FC_INTERNAL_UNMASK)
@@ -325,7 +351,7 @@ struct sisl_global_regs {
 #define SISL_STATUS_SHUTDOWN_ACTIVE	0x0000000000000010ULL
 #define SISL_STATUS_SHUTDOWN_COMPLETE	0x0000000000000020ULL
 
-#define SISL_ASTATUS_UNMASK	0xFFFFULL		/* 1 means unmasked */
+#define SISL_ASTATUS_UNMASK	0xFFFFFFFFULL		/* 1 means unmasked */
 #define SISL_ASTATUS_MASK	~(SISL_ASTATUS_UNMASK)	/* 1 means masked */
 
 	__be64 aintr_clear;
@@ -367,10 +393,18 @@ struct sisl_global_regs {
 #define SISL_INTVER_CAP_RESERVED_CMD_MODE_B	0x100000000000ULL
 };
 
-#define CXLFLASH_MAX_FC_PORTS   2
-#define CXLFLASH_NUM_FC_PORTS   2
-#define CXLFLASH_MAX_CONTEXT  512	/* how many contexts per afu */
-#define CXLFLASH_NUM_VLUNS    512
+#define CXLFLASH_NUM_FC_PORTS_PER_BANK	2	/* fixed # of ports per bank */
+#define CXLFLASH_MAX_FC_BANKS		1	/* max # of banks supported */
+#define CXLFLASH_MAX_FC_PORTS	(CXLFLASH_NUM_FC_PORTS_PER_BANK *	\
+				 CXLFLASH_MAX_FC_BANKS)
+#define CXLFLASH_MAX_CONTEXT	512	/* number of contexts per AFU */
+#define CXLFLASH_NUM_VLUNS	512	/* number of vluns per AFU/port */
+#define CXLFLASH_NUM_REGS	512	/* number of registers per port */
+
+struct fc_port_bank {
+	__be64 fc_port_regs[CXLFLASH_NUM_FC_PORTS_PER_BANK][CXLFLASH_NUM_REGS];
+	__be64 fc_port_luns[CXLFLASH_NUM_FC_PORTS_PER_BANK][CXLFLASH_NUM_VLUNS];
+};
 
 struct sisl_global_map {
 	union {
@@ -380,11 +414,9 @@ struct sisl_global_map {
 
 	char page1[SIZE_4K];	/* page 1 */
 
-	/* pages 2 & 3 */
-	__be64 fc_regs[CXLFLASH_NUM_FC_PORTS][CXLFLASH_NUM_VLUNS];
+	struct fc_port_bank bank[CXLFLASH_MAX_FC_BANKS]; /* pages 2 - 5 */
 
-	/* pages 4 & 5 (lun tbl) */
-	__be64 fc_port[CXLFLASH_NUM_FC_PORTS][CXLFLASH_NUM_VLUNS];
+	/* pages 6 - 15 are reserved */
 
 };
 
@@ -479,6 +511,8 @@ struct sisl_rht_entry_f1 {
 
 #define PORT0  0x01U
 #define PORT1  0x02U
+#define PORT2  0x04U
+#define PORT3  0x08U
 #define PORT_MASK(_n)	((1 << (_n)) - 1)
 
 /* AFU Sync Mode byte */
-- 
2.1.0

[PATCH 09/17] cxlflash: Support up to 4 ports

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

Update the driver to allow for future cards with 4 ports.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/main.c    | 78 ++++++++++++++++++++++++++++++++++++++++-
 drivers/scsi/cxlflash/sislite.h |  6 ++--
 2 files changed, 80 insertions(+), 4 deletions(-)

diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 64ad76b..568cd63 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1419,7 +1419,7 @@ static int read_vpd(struct cxlflash_cfg *cfg, u64 wwpn[])
 	ssize_t vpd_size;
 	char vpd_data[CXLFLASH_VPD_LEN];
 	char tmp_buf[WWPN_BUF_LEN] = { 0 };
-	char *wwpn_vpd_tags[MAX_FC_PORTS] = { "V5", "V6" };
+	char *wwpn_vpd_tags[MAX_FC_PORTS] = { "V5", "V6", "V7", "V8" };
 
 	/* Get the VPD data from the device */
 	vpd_size = cxl_read_adapter_vpd(pdev, vpd_data, sizeof(vpd_data));
@@ -2175,6 +2175,40 @@ static ssize_t port1_show(struct device *dev,
 }
 
 /**
+ * port2_show() - queries and presents the current status of port 2
+ * @dev:	Generic device associated with the host owning the port.
+ * @attr:	Device attribute representing the port.
+ * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t port2_show(struct device *dev,
+			  struct device_attribute *attr,
+			  char *buf)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+
+	return cxlflash_show_port_status(2, cfg, buf);
+}
+
+/**
+ * port3_show() - queries and presents the current status of port 3
+ * @dev:	Generic device associated with the host owning the port.
+ * @attr:	Device attribute representing the port.
+ * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t port3_show(struct device *dev,
+			  struct device_attribute *attr,
+			  char *buf)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+
+	return cxlflash_show_port_status(3, cfg, buf);
+}
+
+/**
  * lun_mode_show() - presents the current LUN mode of the host
  * @dev:	Generic device associated with the host.
  * @attr:	Device attribute representing the LUN mode.
@@ -2327,6 +2361,40 @@ static ssize_t port1_lun_table_show(struct device *dev,
 }
 
 /**
+ * port2_lun_table_show() - presents the current LUN table of port 2
+ * @dev:	Generic device associated with the host owning the port.
+ * @attr:	Device attribute representing the port.
+ * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t port2_lun_table_show(struct device *dev,
+				    struct device_attribute *attr,
+				    char *buf)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+
+	return cxlflash_show_port_lun_table(2, cfg, buf);
+}
+
+/**
+ * port3_lun_table_show() - presents the current LUN table of port 3
+ * @dev:	Generic device associated with the host owning the port.
+ * @attr:	Device attribute representing the port.
+ * @buf:	Buffer of length PAGE_SIZE to report back port status in ASCII.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t port3_lun_table_show(struct device *dev,
+				    struct device_attribute *attr,
+				    char *buf)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+
+	return cxlflash_show_port_lun_table(3, cfg, buf);
+}
+
+/**
  * irqpoll_weight_show() - presents the current IRQ poll weight for the host
  * @dev:	Generic device associated with the host.
  * @attr:	Device attribute representing the IRQ poll weight.
@@ -2417,19 +2485,27 @@ static ssize_t mode_show(struct device *dev,
  */
 static DEVICE_ATTR_RO(port0);
 static DEVICE_ATTR_RO(port1);
+static DEVICE_ATTR_RO(port2);
+static DEVICE_ATTR_RO(port3);
 static DEVICE_ATTR_RW(lun_mode);
 static DEVICE_ATTR_RO(ioctl_version);
 static DEVICE_ATTR_RO(port0_lun_table);
 static DEVICE_ATTR_RO(port1_lun_table);
+static DEVICE_ATTR_RO(port2_lun_table);
+static DEVICE_ATTR_RO(port3_lun_table);
 static DEVICE_ATTR_RW(irqpoll_weight);
 
 static struct device_attribute *cxlflash_host_attrs[] = {
 	&dev_attr_port0,
 	&dev_attr_port1,
+	&dev_attr_port2,
+	&dev_attr_port3,
 	&dev_attr_lun_mode,
 	&dev_attr_ioctl_version,
 	&dev_attr_port0_lun_table,
 	&dev_attr_port1_lun_table,
+	&dev_attr_port2_lun_table,
+	&dev_attr_port3_lun_table,
 	&dev_attr_irqpoll_weight,
 	NULL
 };
diff --git a/drivers/scsi/cxlflash/sislite.h b/drivers/scsi/cxlflash/sislite.h
index 42d9c9e..0e52bbb 100644
--- a/drivers/scsi/cxlflash/sislite.h
+++ b/drivers/scsi/cxlflash/sislite.h
@@ -394,7 +394,7 @@ struct sisl_global_regs {
 };
 
 #define CXLFLASH_NUM_FC_PORTS_PER_BANK	2	/* fixed # of ports per bank */
-#define CXLFLASH_MAX_FC_BANKS		1	/* max # of banks supported */
+#define CXLFLASH_MAX_FC_BANKS		2	/* max # of banks supported */
 #define CXLFLASH_MAX_FC_PORTS	(CXLFLASH_NUM_FC_PORTS_PER_BANK *	\
 				 CXLFLASH_MAX_FC_BANKS)
 #define CXLFLASH_MAX_CONTEXT	512	/* number of contexts per AFU */
@@ -414,9 +414,9 @@ struct sisl_global_map {
 
 	char page1[SIZE_4K];	/* page 1 */
 
-	struct fc_port_bank bank[CXLFLASH_MAX_FC_BANKS]; /* pages 2 - 5 */
+	struct fc_port_bank bank[CXLFLASH_MAX_FC_BANKS]; /* pages 2 - 9 */
 
-	/* pages 6 - 15 are reserved */
+	/* pages 10 - 15 are reserved */
 
 };
 
-- 
2.1.0

[PATCH 10/17] cxlflash: Fence EEH during probe

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

An EEH during probe can lead to a crash as the recovery thread races
with the probe thread. To avoid this issue, introduce new states to
fence out EEH recovery until probe has completed. Also ensure the reset
wait queue is flushed during device removal to avoid orphaned threads.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h    |  2 ++
 drivers/scsi/cxlflash/main.c      | 25 +++++++++++++++++++++----
 drivers/scsi/cxlflash/superpipe.c |  8 +++++---
 3 files changed, 28 insertions(+), 7 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 28bb716..17aa74a 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -90,6 +90,8 @@ enum cxlflash_init_state {
 };
 
 enum cxlflash_state {
+	STATE_PROBING,	/* Initial state during probe */
+	STATE_PROBED,	/* Temporary state, probe completed but EEH occurred */
 	STATE_NORMAL,	/* Normal running state, everything good */
 	STATE_RESET,	/* Reset state, trying to reset/recover */
 	STATE_FAILTERM	/* Failed/terminating state, error out users/threads */
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 568cd63..ebba3c9 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -470,6 +470,8 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	spin_unlock_irqrestore(&cfg->tmf_slock, lock_flags);
 
 	switch (cfg->state) {
+	case STATE_PROBING:
+	case STATE_PROBED:
 	case STATE_RESET:
 		dev_dbg_ratelimited(dev, "%s: device is in reset\n", __func__);
 		rc = SCSI_MLQUEUE_HOST_BUSY;
@@ -719,7 +721,8 @@ static void notify_shutdown(struct cxlflash_cfg *cfg, bool wait)
  * cxlflash_remove() - PCI entry point to tear down host
  * @pdev:	PCI device associated with the host.
  *
- * Safe to use as a cleanup in partially allocated/initialized state.
+ * Safe to use as a cleanup in partially allocated/initialized state. Note that
+ * the reset_waitq is flushed as part of the stop/termination of user contexts.
  */
 static void cxlflash_remove(struct pci_dev *pdev)
 {
@@ -752,7 +755,6 @@ static void cxlflash_remove(struct pci_dev *pdev)
 	case INIT_STATE_SCSI:
 		cxlflash_term_local_luns(cfg);
 		scsi_remove_host(cfg->host);
-		/* fall through */
 	case INIT_STATE_AFU:
 		term_afu(cfg);
 	case INIT_STATE_PCI:
@@ -2624,6 +2626,15 @@ static void cxlflash_worker_thread(struct work_struct *work)
  * @pdev:	PCI device associated with the host.
  * @dev_id:	PCI device id associated with device.
  *
+ * The device will initially start out in a 'probing' state and
+ * transition to the 'normal' state at the end of a successful
+ * probe. Should an EEH event occur during probe, the notification
+ * thread (error_detected()) will wait until the probe handler
+ * is nearly complete. At that time, the device will be moved to
+ * a 'probed' state and the EEH thread woken up to drive the slot
+ * reset and recovery (device moves to 'normal' state). Meanwhile,
+ * the probe will be allowed to exit successfully.
+ *
  * Return: 0 on success, -errno on failure
  */
 static int cxlflash_probe(struct pci_dev *pdev,
@@ -2707,7 +2718,7 @@ static int cxlflash_probe(struct pci_dev *pdev,
 	cfg->init_state = INIT_STATE_PCI;
 
 	rc = init_afu(cfg);
-	if (rc) {
+	if (rc && !wq_has_sleeper(&cfg->reset_waitq)) {
 		dev_err(dev, "%s: init_afu failed rc=%d\n", __func__, rc);
 		goto out_remove;
 	}
@@ -2720,6 +2731,11 @@ static int cxlflash_probe(struct pci_dev *pdev,
 	}
 	cfg->init_state = INIT_STATE_SCSI;
 
+	if (wq_has_sleeper(&cfg->reset_waitq)) {
+		cfg->state = STATE_PROBED;
+		wake_up_all(&cfg->reset_waitq);
+	} else
+		cfg->state = STATE_NORMAL;
 out:
 	dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc);
 	return rc;
@@ -2750,7 +2766,8 @@ static pci_ers_result_t cxlflash_pci_error_detected(struct pci_dev *pdev,
 
 	switch (state) {
 	case pci_channel_io_frozen:
-		wait_event(cfg->reset_waitq, cfg->state != STATE_RESET);
+		wait_event(cfg->reset_waitq, cfg->state != STATE_RESET &&
+					     cfg->state != STATE_PROBING);
 		if (cfg->state == STATE_FAILTERM)
 			return PCI_ERS_RESULT_DISCONNECT;
 
diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index 488330f..158fa00 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -78,17 +78,18 @@ void cxlflash_free_errpage(void)
  * memory freed. This is accomplished by putting the contexts in error
  * state which will notify the user and let them 'drive' the tear down.
  * Meanwhile, this routine camps until all user contexts have been removed.
+ *
+ * Note that the main loop in this routine will always execute at least once
+ * to flush the reset_waitq.
  */
 void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *cfg)
 {
 	struct device *dev = &cfg->dev->dev;
-	int i, found;
+	int i, found = true;
 
 	cxlflash_mark_contexts_error(cfg);
 
 	while (true) {
-		found = false;
-
 		for (i = 0; i < MAX_CONTEXT; i++)
 			if (cfg->ctx_tbl[i]) {
 				found = true;
@@ -102,6 +103,7 @@ void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *cfg)
 			__func__);
 		wake_up_all(&cfg->reset_waitq);
 		ssleep(1);
+		found = false;
 	}
 }
 
-- 
2.1.0

[PATCH 11/17] cxlflash: Remove unnecessary DMA mapping

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

Devices supported by the cxlflash driver are fully coherent and do not
require a bus address mapping. Avoid unnecessary path length by using
the virtual address and length already present in the scatter-gather
entry.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/main.c | 15 ++-------------
 1 file changed, 2 insertions(+), 13 deletions(-)

diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index ebba3c9..3c4a833 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -176,7 +176,6 @@ static void cmd_complete(struct afu_cmd *cmd)
 		dev_dbg_ratelimited(dev, "%s:scp=%p result=%08x ioasc=%08x\n",
 				    __func__, scp, scp->result, cmd->sa.ioasc);
 
-		scsi_dma_unmap(scp);
 		scp->scsi_done(scp);
 
 		if (cmd_is_tmf) {
@@ -445,7 +444,6 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	struct scatterlist *sg = scsi_sglist(scp);
 	u16 req_flags = SISL_REQ_FLAGS_SUP_UNDERRUN;
 	ulong lock_flags;
-	int nseg = 0;
 	int rc = 0;
 
 	dev_dbg_ratelimited(dev, "%s: (scp=%p) %d/%d/%d/%llu "
@@ -487,15 +485,8 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	}
 
 	if (likely(sg)) {
-		nseg = scsi_dma_map(scp);
-		if (unlikely(nseg < 0)) {
-			dev_err(dev, "%s: Fail DMA map\n", __func__);
-			rc = SCSI_MLQUEUE_HOST_BUSY;
-			goto out;
-		}
-
-		cmd->rcb.data_len = sg_dma_len(sg);
-		cmd->rcb.data_ea = sg_dma_address(sg);
+		cmd->rcb.data_len = sg->length;
+		cmd->rcb.data_ea = (uintptr_t)sg_virt(sg);
 	}
 
 	cmd->scp = scp;
@@ -513,8 +504,6 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	memcpy(cmd->rcb.cdb, scp->cmnd, sizeof(cmd->rcb.cdb));
 
 	rc = afu->send_cmd(afu, cmd);
-	if (unlikely(rc))
-		scsi_dma_unmap(scp);
 out:
 	return rc;
 }
-- 
2.1.0

[PATCH 12/17] cxlflash: Fix power-of-two validations

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

Validation statements to enforce assumptions about specific defines
are not being evaluated by the compiler due to the fact that they
reside in a routine that is not used. To activate them, call the
routine as part of module initialization. As an additional, related
cleanup, remove the now-defunct CXLFLASH_NUM_CMDS.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h | 7 +------
 drivers/scsi/cxlflash/main.c   | 1 +
 2 files changed, 2 insertions(+), 6 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 17aa74a..455fd4d 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -52,12 +52,6 @@ extern const struct file_operations cxlflash_cxl_fops;
 				   certain AFU errors */
 
 /* Command management definitions */
-#define CXLFLASH_NUM_CMDS	(2 * CXLFLASH_MAX_CMDS)	/* Must be a pow2 for
-							   alignment and more
-							   efficient array
-							   index derivation
-							 */
-
 #define CXLFLASH_MAX_CMDS               256
 #define CXLFLASH_MAX_CMDS_PER_LUN       CXLFLASH_MAX_CMDS
 
@@ -71,6 +65,7 @@ extern const struct file_operations cxlflash_cxl_fops;
 static inline void check_sizes(void)
 {
 	BUILD_BUG_ON_NOT_POWER_OF_2(CXLFLASH_NUM_FC_PORTS_PER_BANK);
+	BUILD_BUG_ON_NOT_POWER_OF_2(CXLFLASH_MAX_CMDS);
 }
 
 /* AFU defines a fixed size of 4K for command buffers (borrow 4K page define) */
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 3c4a833..f5c952c 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -2847,6 +2847,7 @@ static struct pci_driver cxlflash_driver = {
  */
 static int __init init_cxlflash(void)
 {
+	check_sizes();
 	cxlflash_list_init();
 
 	return pci_register_driver(&cxlflash_driver);
-- 
2.1.0

[PATCH 13/17] cxlflash: Fix warnings/errors

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

As a general cleanup, address all reasonable checkpatch warnings and
errors. These include enforcement of comment styles and including named
identifiers in function prototypes.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h    | 27 ++++++++++-----------
 drivers/scsi/cxlflash/sislite.h   | 27 +++++++++++----------
 drivers/scsi/cxlflash/superpipe.h | 51 ++++++++++++++++++++++-----------------
 drivers/scsi/cxlflash/vlun.h      |  2 +-
 4 files changed, 57 insertions(+), 50 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 455fd4d..c69cdcf 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -36,20 +36,19 @@ extern const struct file_operations cxlflash_cxl_fops;
 #define PORTMASK2CHAN(_x)	(ilog2((_x)))	/* port mask to channel */
 #define PORTNUM2CHAN(_x)	((_x) - 1)	/* port number to channel */
 
-#define CXLFLASH_BLOCK_SIZE	4096	/* 4K blocks */
+#define CXLFLASH_BLOCK_SIZE	4096		/* 4K blocks */
 #define CXLFLASH_MAX_XFER_SIZE	16777216	/* 16MB transfer */
 #define CXLFLASH_MAX_SECTORS	(CXLFLASH_MAX_XFER_SIZE/512)	/* SCSI wants
-								   max_sectors
-								   in units of
-								   512 byte
-								   sectors
-								*/
+								 * max_sectors
+								 * in units of
+								 * 512 byte
+								 * sectors
+								 */
 
 #define MAX_RHT_PER_CONTEXT (PAGE_SIZE / sizeof(struct sisl_rht_entry))
 
 /* AFU command retry limit */
-#define MC_RETRY_CNT         5	/* sufficient for SCSI check and
-				   certain AFU errors */
+#define MC_RETRY_CNT	5	/* Sufficient for SCSI and certain AFU errors */
 
 /* Command management definitions */
 #define CXLFLASH_MAX_CMDS               256
@@ -262,14 +261,14 @@ static inline __be64 __iomem *get_fc_port_luns(struct cxlflash_cfg *cfg, int i)
 	return &fcpb->fc_port_luns[CHAN2BANKPORT(i)][0];
 }
 
-int cxlflash_afu_sync(struct afu *, ctx_hndl_t, res_hndl_t, u8);
+int cxlflash_afu_sync(struct afu *afu, ctx_hndl_t c, res_hndl_t r, u8 mode);
 void cxlflash_list_init(void);
 void cxlflash_term_global_luns(void);
 void cxlflash_free_errpage(void);
-int cxlflash_ioctl(struct scsi_device *, int, void __user *);
-void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *);
-int cxlflash_mark_contexts_error(struct cxlflash_cfg *);
-void cxlflash_term_local_luns(struct cxlflash_cfg *);
-void cxlflash_restore_luntable(struct cxlflash_cfg *);
+int cxlflash_ioctl(struct scsi_device *sdev, int cmd, void __user *arg);
+void cxlflash_stop_term_user_contexts(struct cxlflash_cfg *cfg);
+int cxlflash_mark_contexts_error(struct cxlflash_cfg *cfg);
+void cxlflash_term_local_luns(struct cxlflash_cfg *cfg);
+void cxlflash_restore_luntable(struct cxlflash_cfg *cfg);
 
 #endif /* ifndef _CXLFLASH_COMMON_H */
diff --git a/drivers/scsi/cxlflash/sislite.h b/drivers/scsi/cxlflash/sislite.h
index 0e52bbb..a768360 100644
--- a/drivers/scsi/cxlflash/sislite.h
+++ b/drivers/scsi/cxlflash/sislite.h
@@ -90,15 +90,15 @@ struct sisl_rc {
 #define SISL_AFU_RC_RHT_UNALIGNED         0x02U	/* should never happen */
 #define SISL_AFU_RC_RHT_OUT_OF_BOUNDS     0x03u	/* user error */
 #define SISL_AFU_RC_RHT_DMA_ERR           0x04u	/* see afu_extra
-						   may retry if afu_retry is off
-						   possible on master exit
+						 * may retry if afu_retry is off
+						 * possible on master exit
 						 */
 #define SISL_AFU_RC_RHT_RW_PERM           0x05u	/* no RW perms, user error */
 #define SISL_AFU_RC_LXT_UNALIGNED         0x12U	/* should never happen */
 #define SISL_AFU_RC_LXT_OUT_OF_BOUNDS     0x13u	/* user error */
 #define SISL_AFU_RC_LXT_DMA_ERR           0x14u	/* see afu_extra
-						   may retry if afu_retry is off
-						   possible on master exit
+						 * may retry if afu_retry is off
+						 * possible on master exit
 						 */
 #define SISL_AFU_RC_LXT_RW_PERM           0x15u	/* no RW perms, user error */
 
@@ -111,11 +111,11 @@ struct sisl_rc {
 	 */
 #define SISL_AFU_RC_NO_CHANNELS           0x20U	/* see afu_extra, may retry */
 #define SISL_AFU_RC_CAP_VIOLATION         0x21U	/* either user error or
-						   afu reset/master restart
+						 * afu reset/master restart
 						 */
 #define SISL_AFU_RC_OUT_OF_DATA_BUFS      0x30U	/* always retry */
 #define SISL_AFU_RC_DATA_DMA_ERR          0x31U	/* see afu_extra
-						   may retry if afu_retry is off
+						 * may retry if afu_retry is off
 						 */
 
 	u8 scsi_rc;		/* SCSI status byte, retry as appropriate */
@@ -149,8 +149,9 @@ struct sisl_rc {
 #define SISL_FC_RC_ABORTFAIL	0x59	/* pending abort completed w/fail */
 #define SISL_FC_RC_RESID	0x5A	/* ioasa underrun/overrun flags set */
 #define SISL_FC_RC_RESIDERR	0x5B	/* actual data len does not match SCSI
-					   reported len, possibly due to dropped
-					   frames */
+					 * reported len, possibly due to dropped
+					 * frames
+					 */
 #define SISL_FC_RC_TGTABORT	0x5C	/* command aborted by target */
 };
 
@@ -227,10 +228,10 @@ struct sisl_ioasa {
 
 /* per context host transport MMIO  */
 struct sisl_host_map {
-	__be64 endian_ctrl;     /* Per context Endian Control. The AFU will
-			      * operate on whatever the context is of the
-			      * host application.
-			      */
+	__be64 endian_ctrl;	/* Per context Endian Control. The AFU will
+				 * operate on whatever the context is of the
+				 * host application.
+				 */
 
 	__be64 intr_status;	/* this sends LISN# programmed in ctx_ctrl.
 				 * Only recovery in a PERM_ERR is a context
@@ -435,7 +436,7 @@ struct sisl_global_map {
  *	|         64 KB Global          |
  *	|   Trusted Process accessible  |
  *	+-------------------------------+
-*/
+ */
 struct cxlflash_afu_map {
 	union {
 		struct sisl_host_map host;
diff --git a/drivers/scsi/cxlflash/superpipe.h b/drivers/scsi/cxlflash/superpipe.h
index 8269ff8..0b59768 100644
--- a/drivers/scsi/cxlflash/superpipe.h
+++ b/drivers/scsi/cxlflash/superpipe.h
@@ -24,8 +24,8 @@ extern struct cxlflash_global global;
  */
 
 /* Chunk size parms: note sislite minimum chunk size is
-   0x10000 LBAs corresponding to a NMASK or 16.
-*/
+ * 0x10000 LBAs corresponding to a NMASK or 16.
+ */
 #define MC_CHUNK_SIZE     (1 << MC_RHT_NMASK)	/* in LBAs */
 
 #define CMD_TIMEOUT 30  /* 30 secs */
@@ -89,7 +89,8 @@ enum ctx_ctrl {
 struct ctx_info {
 	struct sisl_ctrl_map __iomem *ctrl_map; /* initialized at startup */
 	struct sisl_rht_entry *rht_start; /* 1 page (req'd for alignment),
-					     alloc/free on attach/detach */
+					   * alloc/free on attach/detach
+					   */
 	u32 rht_out;		/* Number of checked out RHT entries */
 	u32 rht_perms;		/* User-defined permissions for RHT entries */
 	struct llun_info **rht_lun;       /* Mapping of RHT entries to LUNs */
@@ -117,34 +118,40 @@ struct cxlflash_global {
 	struct page *err_page; /* One page of all 0xF for error notification */
 };
 
-int cxlflash_vlun_resize(struct scsi_device *, struct dk_cxlflash_resize *);
-int _cxlflash_vlun_resize(struct scsi_device *, struct ctx_info *,
-			  struct dk_cxlflash_resize *);
+int cxlflash_vlun_resize(struct scsi_device *sdev,
+			 struct dk_cxlflash_resize *resize);
+int _cxlflash_vlun_resize(struct scsi_device *sdev, struct ctx_info *ctxi,
+			  struct dk_cxlflash_resize *resize);
 
-int cxlflash_disk_release(struct scsi_device *, struct dk_cxlflash_release *);
-int _cxlflash_disk_release(struct scsi_device *, struct ctx_info *,
-			   struct dk_cxlflash_release *);
+int cxlflash_disk_release(struct scsi_device *sdev,
+			  struct dk_cxlflash_release *release);
+int _cxlflash_disk_release(struct scsi_device *sdev, struct ctx_info *ctxi,
+			   struct dk_cxlflash_release *release);
 
-int cxlflash_disk_clone(struct scsi_device *, struct dk_cxlflash_clone *);
+int cxlflash_disk_clone(struct scsi_device *sdev,
+			struct dk_cxlflash_clone *clone);
 
-int cxlflash_disk_virtual_open(struct scsi_device *, void *);
+int cxlflash_disk_virtual_open(struct scsi_device *sdev, void *arg);
 
-int cxlflash_lun_attach(struct glun_info *, enum lun_mode, bool);
-void cxlflash_lun_detach(struct glun_info *);
+int cxlflash_lun_attach(struct glun_info *gli, enum lun_mode mode, bool locked);
+void cxlflash_lun_detach(struct glun_info *gli);
 
-struct ctx_info *get_context(struct cxlflash_cfg *, u64, void *, enum ctx_ctrl);
-void put_context(struct ctx_info *);
+struct ctx_info *get_context(struct cxlflash_cfg *cfg, u64 rctxit, void *arg,
+			     enum ctx_ctrl ctrl);
+void put_context(struct ctx_info *ctxi);
 
-struct sisl_rht_entry *get_rhte(struct ctx_info *, res_hndl_t,
-				struct llun_info *);
+struct sisl_rht_entry *get_rhte(struct ctx_info *ctxi, res_hndl_t rhndl,
+				struct llun_info *lli);
 
-struct sisl_rht_entry *rhte_checkout(struct ctx_info *, struct llun_info *);
-void rhte_checkin(struct ctx_info *, struct sisl_rht_entry *);
+struct sisl_rht_entry *rhte_checkout(struct ctx_info *ctxi,
+				     struct llun_info *lli);
+void rhte_checkin(struct ctx_info *ctxi, struct sisl_rht_entry *rhte);
 
-void cxlflash_ba_terminate(struct ba_lun *);
+void cxlflash_ba_terminate(struct ba_lun *ba_lun);
 
-int cxlflash_manage_lun(struct scsi_device *, struct dk_cxlflash_manage_lun *);
+int cxlflash_manage_lun(struct scsi_device *sdev,
+			struct dk_cxlflash_manage_lun *manage);
 
-int check_state(struct cxlflash_cfg *);
+int check_state(struct cxlflash_cfg *cfg);
 
 #endif /* ifndef _CXLFLASH_SUPERPIPE_H */
diff --git a/drivers/scsi/cxlflash/vlun.h b/drivers/scsi/cxlflash/vlun.h
index 8b29a74..27a63a0 100644
--- a/drivers/scsi/cxlflash/vlun.h
+++ b/drivers/scsi/cxlflash/vlun.h
@@ -47,7 +47,7 @@
  * not stored anywhere.
  *
  * The LXT table is re-allocated whenever it needs to cross into another group.
-*/
+ */
 #define LXT_GROUP_SIZE          8
 #define LXT_NUM_GROUPS(lxt_cnt) (((lxt_cnt) + 7)/8)	/* alloc'ed groups */
 #define LXT_LUNIDX_SHIFT  8	/* LXT entry, shift for LUN index */
-- 
2.1.0

[PATCH 14/17] cxlflash: Improve asynchronous interrupt processing

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

The method used to decode asynchronous interrupts involves unnecessary
loops to match up bits that are set with corresponding entries in the
asynchronous interrupt information table. This algorithm is wasteful
and does not scale well as new status bits are supported.

As an improvement, use the for_each_set_bit() service to iterate over
the asynchronous status bits and refactor the information table such
that it can be indexed by bit position.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/main.c | 94 ++++++++++++++++++++------------------------
 1 file changed, 42 insertions(+), 52 deletions(-)

diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index f5c952c..c60936f 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -1017,52 +1017,6 @@ static void afu_link_reset(struct afu *afu, int port, __be64 __iomem *fc_regs)
 	dev_dbg(dev, "%s: returning port_sel=%016llx\n", __func__, port_sel);
 }
 
-/*
- * Asynchronous interrupt information table
- *
- * NOTE: The checkpatch script considers the BUILD_SISL_ASTATUS_FC_PORT macro
- * as complex and complains because it is not wrapped with parentheses/braces.
- */
-#define ASTATUS_FC(_a, _b, _c, _d)					 \
-	{ SISL_ASTATUS_FC##_a##_##_b, _c, _a, (_d) }
-
-#define BUILD_SISL_ASTATUS_FC_PORT(_a)					 \
-	ASTATUS_FC(_a, OTHER, "other error", CLR_FC_ERROR | LINK_RESET), \
-	ASTATUS_FC(_a, LOGO, "target initiated LOGO", 0),		 \
-	ASTATUS_FC(_a, CRC_T, "CRC threshold exceeded", LINK_RESET),	 \
-	ASTATUS_FC(_a, LOGI_R, "login timed out, retrying", LINK_RESET), \
-	ASTATUS_FC(_a, LOGI_F, "login failed", CLR_FC_ERROR),		 \
-	ASTATUS_FC(_a, LOGI_S, "login succeeded", SCAN_HOST),		 \
-	ASTATUS_FC(_a, LINK_DN, "link down", 0),			 \
-	ASTATUS_FC(_a, LINK_UP, "link up", 0)
-
-static const struct asyc_intr_info ainfo[] = {
-	BUILD_SISL_ASTATUS_FC_PORT(2),
-	BUILD_SISL_ASTATUS_FC_PORT(3),
-	BUILD_SISL_ASTATUS_FC_PORT(0),
-	BUILD_SISL_ASTATUS_FC_PORT(1),
-	{ 0x0, "", 0, 0 }
-};
-
-/**
- * find_ainfo() - locates and returns asynchronous interrupt information
- * @status:	Status code set by AFU on error.
- *
- * Return: The located information or NULL when the status code is invalid.
- */
-static const struct asyc_intr_info *find_ainfo(u64 status)
-{
-	const struct asyc_intr_info *info;
-
-	BUILD_BUG_ON(ainfo[ARRAY_SIZE(ainfo) - 1].status != 0);
-
-	for (info = &ainfo[0]; info->status; info++)
-		if (info->status == status)
-			return info;
-
-	return NULL;
-}
-
 /**
  * afu_err_intr_init() - clears and initializes the AFU for error interrupts
  * @afu:	AFU associated with the host.
@@ -1293,6 +1247,35 @@ static irqreturn_t cxlflash_rrq_irq(int irq, void *data)
 	return IRQ_HANDLED;
 }
 
+/*
+ * Asynchronous interrupt information table
+ *
+ * NOTE:
+ *	- Order matters here as this array is indexed by bit position.
+ *
+ *	- The checkpatch script considers the BUILD_SISL_ASTATUS_FC_PORT macro
+ *	  as complex and complains due to a lack of parentheses/braces.
+ */
+#define ASTATUS_FC(_a, _b, _c, _d)					 \
+	{ SISL_ASTATUS_FC##_a##_##_b, _c, _a, (_d) }
+
+#define BUILD_SISL_ASTATUS_FC_PORT(_a)					 \
+	ASTATUS_FC(_a, LINK_UP, "link up", 0),				 \
+	ASTATUS_FC(_a, LINK_DN, "link down", 0),			 \
+	ASTATUS_FC(_a, LOGI_S, "login succeeded", SCAN_HOST),		 \
+	ASTATUS_FC(_a, LOGI_F, "login failed", CLR_FC_ERROR),		 \
+	ASTATUS_FC(_a, LOGI_R, "login timed out, retrying", LINK_RESET), \
+	ASTATUS_FC(_a, CRC_T, "CRC threshold exceeded", LINK_RESET),	 \
+	ASTATUS_FC(_a, LOGO, "target initiated LOGO", 0),		 \
+	ASTATUS_FC(_a, OTHER, "other error", CLR_FC_ERROR | LINK_RESET)
+
+static const struct asyc_intr_info ainfo[] = {
+	BUILD_SISL_ASTATUS_FC_PORT(1),
+	BUILD_SISL_ASTATUS_FC_PORT(0),
+	BUILD_SISL_ASTATUS_FC_PORT(3),
+	BUILD_SISL_ASTATUS_FC_PORT(2)
+};
+
 /**
  * cxlflash_async_err_irq() - interrupt handler for asynchronous errors
  * @irq:	Interrupt number.
@@ -1305,18 +1288,18 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 	struct afu *afu = (struct afu *)data;
 	struct cxlflash_cfg *cfg = afu->parent;
 	struct device *dev = &cfg->dev->dev;
-	u64 reg_unmasked;
 	const struct asyc_intr_info *info;
 	struct sisl_global_map __iomem *global = &afu->afu_map->global;
 	__be64 __iomem *fc_port_regs;
+	u64 reg_unmasked;
 	u64 reg;
+	u64 bit;
 	u8 port;
-	int i;
 
 	reg = readq_be(&global->regs.aintr_status);
 	reg_unmasked = (reg & SISL_ASTATUS_UNMASK);
 
-	if (reg_unmasked == 0) {
+	if (unlikely(reg_unmasked == 0)) {
 		dev_err(dev, "%s: spurious interrupt, aintr_status=%016llx\n",
 			__func__, reg);
 		goto out;
@@ -1326,10 +1309,17 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 	writeq_be(reg_unmasked, &global->regs.aintr_clear);
 
 	/* Check each bit that is on */
-	for (i = 0; reg_unmasked; i++, reg_unmasked = (reg_unmasked >> 1)) {
-		info = find_ainfo(1ULL << i);
-		if (((reg_unmasked & 0x1) == 0) || !info)
+	for_each_set_bit(bit, (ulong *)&reg_unmasked, BITS_PER_LONG) {
+		if (unlikely(bit >= ARRAY_SIZE(ainfo))) {
+			WARN_ON_ONCE(1);
 			continue;
+		}
+
+		info = &ainfo[bit];
+		if (unlikely(info->status != 1ULL << bit)) {
+			WARN_ON_ONCE(1);
+			continue;
+		}
 
 		port = info->port;
 		fc_port_regs = get_fc_port_regs(cfg, port);
-- 
2.1.0

[PATCH 15/17] cxlflash: Support multiple hardware queues

[Uma Krishnan]

Introduce multiple hardware queues to improve legacy I/O path performance.
Each hardware queue is comprised of a master context and associated I/O
resources. The hardware queues are initially implemented as a static array
embedded in the AFU. This will be transitioned to a dynamic allocation in a
later series to improve the memory footprint of the driver.

Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h    |  41 ++--
 drivers/scsi/cxlflash/main.c      | 426 ++++++++++++++++++++++++--------------
 drivers/scsi/cxlflash/superpipe.c |   6 +-
 3 files changed, 309 insertions(+), 164 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index c69cdcf..b5858ae 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -60,6 +60,9 @@ extern const struct file_operations cxlflash_cxl_fops;
 /* SQ for master issued cmds */
 #define NUM_SQ_ENTRY			CXLFLASH_MAX_CMDS
 
+#define CXLFLASH_NUM_HWQS		1
+#define PRIMARY_HWQ			0
+
 
 static inline void check_sizes(void)
 {
@@ -98,7 +101,6 @@ enum cxlflash_state {
 
 struct cxlflash_cfg {
 	struct afu *afu;
-	struct cxl_context *mcctx;
 
 	struct pci_dev *dev;
 	struct pci_device_id *dev_id;
@@ -144,6 +146,7 @@ struct afu_cmd {
 	struct list_head queue;
 
 	u8 cmd_tmf:1;
+	u32 hwq_index;
 
 	/* As per the SISLITE spec the IOARCB EA has to be 16-byte aligned.
 	 * However for performance reasons the IOARCB/IOASA should be
@@ -164,7 +167,7 @@ static inline struct afu_cmd *sc_to_afucz(struct scsi_cmnd *sc)
 	return afuc;
 }
 
-struct afu {
+struct hwq {
 	/* Stuff requiring alignment go first. */
 	struct sisl_ioarcb sq[NUM_SQ_ENTRY];		/* 16K SQ */
 	u64 rrq_entry[NUM_RRQ_ENTRY];			/* 2K RRQ */
@@ -172,17 +175,13 @@ struct afu {
 	/* Beware of alignment till here. Preferably introduce new
 	 * fields after this point
 	 */
-
-	int (*send_cmd)(struct afu *, struct afu_cmd *);
-	void (*context_reset)(struct afu_cmd *);
-
-	/* AFU HW */
+	struct afu *afu;
+	struct cxl_context *ctx;
 	struct cxl_ioctl_start_work work;
-	struct cxlflash_afu_map __iomem *afu_map;	/* entire MMIO map */
 	struct sisl_host_map __iomem *host_map;		/* MC host map */
 	struct sisl_ctrl_map __iomem *ctrl_map;		/* MC control map */
-
 	ctx_hndl_t ctx_hndl;	/* master's context handle */
+	u32 index;		/* Index of this hwq */
 
 	atomic_t hsq_credits;
 	spinlock_t hsq_slock;
@@ -194,9 +193,22 @@ struct afu {
 	u64 *hrrq_end;
 	u64 *hrrq_curr;
 	bool toggle;
-	atomic_t cmds_active;	/* Number of currently active AFU commands */
+
 	s64 room;
 	spinlock_t rrin_slock; /* Lock to rrin queuing and cmd_room updates */
+
+	struct irq_poll irqpoll;
+} __aligned(cache_line_size());
+
+struct afu {
+	struct hwq hwqs[CXLFLASH_NUM_HWQS];
+	int (*send_cmd)(struct afu *, struct afu_cmd *);
+	void (*context_reset)(struct afu_cmd *);
+
+	/* AFU HW */
+	struct cxlflash_afu_map __iomem *afu_map;	/* entire MMIO map */
+
+	atomic_t cmds_active;	/* Number of currently active AFU commands */
 	u64 hb;
 	u32 internal_lun;	/* User-desired LUN mode for this AFU */
 
@@ -204,11 +216,16 @@ struct afu {
 	u64 interface_version;
 
 	u32 irqpoll_weight;
-	struct irq_poll irqpoll;
 	struct cxlflash_cfg *parent; /* Pointer back to parent cxlflash_cfg */
-
 };
 
+static inline struct hwq *get_hwq(struct afu *afu, u32 index)
+{
+	WARN_ON(index >= CXLFLASH_NUM_HWQS);
+
+	return &afu->hwqs[index];
+}
+
 static inline bool afu_is_irqpoll_enabled(struct afu *afu)
 {
 	return !!afu->irqpoll_weight;
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index c60936f..5d06869 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -223,8 +223,9 @@ static void context_reset(struct afu_cmd *cmd, __be64 __iomem *reset_reg)
 static void context_reset_ioarrin(struct afu_cmd *cmd)
 {
 	struct afu *afu = cmd->parent;
+	struct hwq *hwq = get_hwq(afu, cmd->hwq_index);
 
-	context_reset(cmd, &afu->host_map->ioarrin);
+	context_reset(cmd, &hwq->host_map->ioarrin);
 }
 
 /**
@@ -234,8 +235,9 @@ static void context_reset_ioarrin(struct afu_cmd *cmd)
 static void context_reset_sq(struct afu_cmd *cmd)
 {
 	struct afu *afu = cmd->parent;
+	struct hwq *hwq = get_hwq(afu, cmd->hwq_index);
 
-	context_reset(cmd, &afu->host_map->sq_ctx_reset);
+	context_reset(cmd, &hwq->host_map->sq_ctx_reset);
 }
 
 /**
@@ -250,6 +252,7 @@ static int send_cmd_ioarrin(struct afu *afu, struct afu_cmd *cmd)
 {
 	struct cxlflash_cfg *cfg = afu->parent;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq = get_hwq(afu, cmd->hwq_index);
 	int rc = 0;
 	s64 room;
 	ulong lock_flags;
@@ -258,23 +261,23 @@ static int send_cmd_ioarrin(struct afu *afu, struct afu_cmd *cmd)
 	 * To avoid the performance penalty of MMIO, spread the update of
 	 * 'room' over multiple commands.
 	 */
-	spin_lock_irqsave(&afu->rrin_slock, lock_flags);
-	if (--afu->room < 0) {
-		room = readq_be(&afu->host_map->cmd_room);
+	spin_lock_irqsave(&hwq->rrin_slock, lock_flags);
+	if (--hwq->room < 0) {
+		room = readq_be(&hwq->host_map->cmd_room);
 		if (room <= 0) {
 			dev_dbg_ratelimited(dev, "%s: no cmd_room to send "
 					    "0x%02X, room=0x%016llX\n",
 					    __func__, cmd->rcb.cdb[0], room);
-			afu->room = 0;
+			hwq->room = 0;
 			rc = SCSI_MLQUEUE_HOST_BUSY;
 			goto out;
 		}
-		afu->room = room - 1;
+		hwq->room = room - 1;
 	}
 
-	writeq_be((u64)&cmd->rcb, &afu->host_map->ioarrin);
+	writeq_be((u64)&cmd->rcb, &hwq->host_map->ioarrin);
 out:
-	spin_unlock_irqrestore(&afu->rrin_slock, lock_flags);
+	spin_unlock_irqrestore(&hwq->rrin_slock, lock_flags);
 	dev_dbg(dev, "%s: cmd=%p len=%u ea=%016llx rc=%d\n", __func__,
 		cmd, cmd->rcb.data_len, cmd->rcb.data_ea, rc);
 	return rc;
@@ -292,11 +295,12 @@ static int send_cmd_sq(struct afu *afu, struct afu_cmd *cmd)
 {
 	struct cxlflash_cfg *cfg = afu->parent;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq = get_hwq(afu, cmd->hwq_index);
 	int rc = 0;
 	int newval;
 	ulong lock_flags;
 
-	newval = atomic_dec_if_positive(&afu->hsq_credits);
+	newval = atomic_dec_if_positive(&hwq->hsq_credits);
 	if (newval <= 0) {
 		rc = SCSI_MLQUEUE_HOST_BUSY;
 		goto out;
@@ -304,22 +308,22 @@ static int send_cmd_sq(struct afu *afu, struct afu_cmd *cmd)
 
 	cmd->rcb.ioasa = &cmd->sa;
 
-	spin_lock_irqsave(&afu->hsq_slock, lock_flags);
+	spin_lock_irqsave(&hwq->hsq_slock, lock_flags);
 
-	*afu->hsq_curr = cmd->rcb;
-	if (afu->hsq_curr < afu->hsq_end)
-		afu->hsq_curr++;
+	*hwq->hsq_curr = cmd->rcb;
+	if (hwq->hsq_curr < hwq->hsq_end)
+		hwq->hsq_curr++;
 	else
-		afu->hsq_curr = afu->hsq_start;
-	writeq_be((u64)afu->hsq_curr, &afu->host_map->sq_tail);
+		hwq->hsq_curr = hwq->hsq_start;
+	writeq_be((u64)hwq->hsq_curr, &hwq->host_map->sq_tail);
 
-	spin_unlock_irqrestore(&afu->hsq_slock, lock_flags);
+	spin_unlock_irqrestore(&hwq->hsq_slock, lock_flags);
 out:
 	dev_dbg(dev, "%s: cmd=%p len=%u ea=%016llx ioasa=%p rc=%d curr=%p "
 	       "head=%016llx tail=%016llx\n", __func__, cmd, cmd->rcb.data_len,
-	       cmd->rcb.data_ea, cmd->rcb.ioasa, rc, afu->hsq_curr,
-	       readq_be(&afu->host_map->sq_head),
-	       readq_be(&afu->host_map->sq_tail));
+	       cmd->rcb.data_ea, cmd->rcb.ioasa, rc, hwq->hsq_curr,
+	       readq_be(&hwq->host_map->sq_head),
+	       readq_be(&hwq->host_map->sq_tail));
 	return rc;
 }
 
@@ -367,6 +371,7 @@ static int send_tmf(struct afu *afu, struct scsi_cmnd *scp, u64 tmfcmd)
 	struct cxlflash_cfg *cfg = shost_priv(scp->device->host);
 	struct afu_cmd *cmd = sc_to_afucz(scp);
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
 	ulong lock_flags;
 	int rc = 0;
 	ulong to;
@@ -383,8 +388,9 @@ static int send_tmf(struct afu *afu, struct scsi_cmnd *scp, u64 tmfcmd)
 	cmd->scp = scp;
 	cmd->parent = afu;
 	cmd->cmd_tmf = true;
+	cmd->hwq_index = hwq->index;
 
-	cmd->rcb.ctx_id = afu->ctx_hndl;
+	cmd->rcb.ctx_id = hwq->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
 	cmd->rcb.port_sel = CHAN2PORTMASK(scp->device->channel);
 	cmd->rcb.lun_id = lun_to_lunid(scp->device->lun);
@@ -442,6 +448,7 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	struct device *dev = &cfg->dev->dev;
 	struct afu_cmd *cmd = sc_to_afucz(scp);
 	struct scatterlist *sg = scsi_sglist(scp);
+	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
 	u16 req_flags = SISL_REQ_FLAGS_SUP_UNDERRUN;
 	ulong lock_flags;
 	int rc = 0;
@@ -491,8 +498,9 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 
 	cmd->scp = scp;
 	cmd->parent = afu;
+	cmd->hwq_index = hwq->index;
 
-	cmd->rcb.ctx_id = afu->ctx_hndl;
+	cmd->rcb.ctx_id = hwq->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
 	cmd->rcb.port_sel = CHAN2PORTMASK(scp->device->channel);
 	cmd->rcb.lun_id = lun_to_lunid(scp->device->lun);
@@ -548,14 +556,23 @@ static void free_mem(struct cxlflash_cfg *cfg)
 static void stop_afu(struct cxlflash_cfg *cfg)
 {
 	struct afu *afu = cfg->afu;
+	struct hwq *hwq;
+	int i;
 
 	cancel_work_sync(&cfg->work_q);
 
 	if (likely(afu)) {
 		while (atomic_read(&afu->cmds_active))
 			ssleep(1);
-		if (afu_is_irqpoll_enabled(afu))
-			irq_poll_disable(&afu->irqpoll);
+
+		if (afu_is_irqpoll_enabled(afu)) {
+			for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+				hwq = get_hwq(afu, i);
+
+				irq_poll_disable(&hwq->irqpoll);
+			}
+		}
+
 		if (likely(afu->afu_map)) {
 			cxl_psa_unmap((void __iomem *)afu->afu_map);
 			afu->afu_map = NULL;
@@ -567,28 +584,40 @@ static void stop_afu(struct cxlflash_cfg *cfg)
  * term_intr() - disables all AFU interrupts
  * @cfg:	Internal structure associated with the host.
  * @level:	Depth of allocation, where to begin waterfall tear down.
+ * @index:	Index of the hardware queue.
  *
  * Safe to call with AFU/MC in partially allocated/initialized state.
  */
-static void term_intr(struct cxlflash_cfg *cfg, enum undo_level level)
+static void term_intr(struct cxlflash_cfg *cfg, enum undo_level level,
+		      u32 index)
 {
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq;
 
-	if (!afu || !cfg->mcctx) {
-		dev_err(dev, "%s: returning with NULL afu or MC\n", __func__);
+	if (!afu) {
+		dev_err(dev, "%s: returning with NULL afu\n", __func__);
+		return;
+	}
+
+	hwq = get_hwq(afu, index);
+
+	if (!hwq->ctx) {
+		dev_err(dev, "%s: returning with NULL MC\n", __func__);
 		return;
 	}
 
 	switch (level) {
 	case UNMAP_THREE:
-		cxl_unmap_afu_irq(cfg->mcctx, 3, afu);
+		/* SISL_MSI_ASYNC_ERROR is setup only for the primary HWQ */
+		if (index == PRIMARY_HWQ)
+			cxl_unmap_afu_irq(hwq->ctx, 3, hwq);
 	case UNMAP_TWO:
-		cxl_unmap_afu_irq(cfg->mcctx, 2, afu);
+		cxl_unmap_afu_irq(hwq->ctx, 2, hwq);
 	case UNMAP_ONE:
-		cxl_unmap_afu_irq(cfg->mcctx, 1, afu);
+		cxl_unmap_afu_irq(hwq->ctx, 1, hwq);
 	case FREE_IRQ:
-		cxl_free_afu_irqs(cfg->mcctx);
+		cxl_free_afu_irqs(hwq->ctx);
 		/* fall through */
 	case UNDO_NOOP:
 		/* No action required */
@@ -599,24 +628,32 @@ static void term_intr(struct cxlflash_cfg *cfg, enum undo_level level)
 /**
  * term_mc() - terminates the master context
  * @cfg:	Internal structure associated with the host.
- * @level:	Depth of allocation, where to begin waterfall tear down.
+ * @index:	Index of the hardware queue.
  *
  * Safe to call with AFU/MC in partially allocated/initialized state.
  */
-static void term_mc(struct cxlflash_cfg *cfg)
+static void term_mc(struct cxlflash_cfg *cfg, u32 index)
 {
-	int rc = 0;
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq;
 
-	if (!afu || !cfg->mcctx) {
-		dev_err(dev, "%s: returning with NULL afu or MC\n", __func__);
+	if (!afu) {
+		dev_err(dev, "%s: returning with NULL afu\n", __func__);
 		return;
 	}
 
-	rc = cxl_stop_context(cfg->mcctx);
-	WARN_ON(rc);
-	cfg->mcctx = NULL;
+	hwq = get_hwq(afu, index);
+
+	if (!hwq->ctx) {
+		dev_err(dev, "%s: returning with NULL MC\n", __func__);
+		return;
+	}
+
+	WARN_ON(cxl_stop_context(hwq->ctx));
+	if (index != PRIMARY_HWQ)
+		WARN_ON(cxl_release_context(hwq->ctx));
+	hwq->ctx = NULL;
 }
 
 /**
@@ -628,21 +665,25 @@ static void term_mc(struct cxlflash_cfg *cfg)
 static void term_afu(struct cxlflash_cfg *cfg)
 {
 	struct device *dev = &cfg->dev->dev;
+	int k;
 
 	/*
 	 * Tear down is carefully orchestrated to ensure
 	 * no interrupts can come in when the problem state
 	 * area is unmapped.
 	 *
-	 * 1) Disable all AFU interrupts
+	 * 1) Disable all AFU interrupts for each master
 	 * 2) Unmap the problem state area
-	 * 3) Stop the master context
+	 * 3) Stop each master context
 	 */
-	term_intr(cfg, UNMAP_THREE);
+	for (k = CXLFLASH_NUM_HWQS - 1; k >= 0; k--)
+		term_intr(cfg, UNMAP_THREE, k);
+
 	if (cfg->afu)
 		stop_afu(cfg);
 
-	term_mc(cfg);
+	for (k = CXLFLASH_NUM_HWQS - 1; k >= 0; k--)
+		term_mc(cfg, k);
 
 	dev_dbg(dev, "%s: returning\n", __func__);
 }
@@ -1026,6 +1067,7 @@ static void afu_err_intr_init(struct afu *afu)
 	struct cxlflash_cfg *cfg = afu->parent;
 	__be64 __iomem *fc_port_regs;
 	int i;
+	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
 	u64 reg;
 
 	/* global async interrupts: AFU clears afu_ctrl on context exit
@@ -1037,8 +1079,8 @@ static void afu_err_intr_init(struct afu *afu)
 
 	/* mask all */
 	writeq_be(-1ULL, &afu->afu_map->global.regs.aintr_mask);
-	/* set LISN# to send and point to master context */
-	reg = ((u64) (((afu->ctx_hndl << 8) | SISL_MSI_ASYNC_ERROR)) << 40);
+	/* set LISN# to send and point to primary master context */
+	reg = ((u64) (((hwq->ctx_hndl << 8) | SISL_MSI_ASYNC_ERROR)) << 40);
 
 	if (afu->internal_lun)
 		reg |= 1;	/* Bit 63 indicates local lun */
@@ -1074,8 +1116,12 @@ static void afu_err_intr_init(struct afu *afu)
 	/* IOARRIN yet), so there is nothing to clear. */
 
 	/* set LISN#, it is always sent to the context that wrote IOARRIN */
-	writeq_be(SISL_MSI_SYNC_ERROR, &afu->host_map->ctx_ctrl);
-	writeq_be(SISL_ISTATUS_MASK, &afu->host_map->intr_mask);
+	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		hwq = get_hwq(afu, i);
+
+		writeq_be(SISL_MSI_SYNC_ERROR, &hwq->host_map->ctx_ctrl);
+		writeq_be(SISL_ISTATUS_MASK, &hwq->host_map->intr_mask);
+	}
 }
 
 /**
@@ -1087,13 +1133,13 @@ static void afu_err_intr_init(struct afu *afu)
  */
 static irqreturn_t cxlflash_sync_err_irq(int irq, void *data)
 {
-	struct afu *afu = (struct afu *)data;
-	struct cxlflash_cfg *cfg = afu->parent;
+	struct hwq *hwq = (struct hwq *)data;
+	struct cxlflash_cfg *cfg = hwq->afu->parent;
 	struct device *dev = &cfg->dev->dev;
 	u64 reg;
 	u64 reg_unmasked;
 
-	reg = readq_be(&afu->host_map->intr_status);
+	reg = readq_be(&hwq->host_map->intr_status);
 	reg_unmasked = (reg & SISL_ISTATUS_UNMASK);
 
 	if (reg_unmasked == 0UL) {
@@ -1105,7 +1151,7 @@ static irqreturn_t cxlflash_sync_err_irq(int irq, void *data)
 	dev_err(dev, "%s: unexpected interrupt, intr_status=%016llx\n",
 		__func__, reg);
 
-	writeq_be(reg_unmasked, &afu->host_map->intr_clear);
+	writeq_be(reg_unmasked, &hwq->host_map->intr_clear);
 
 cxlflash_sync_err_irq_exit:
 	return IRQ_HANDLED;
@@ -1121,17 +1167,18 @@ static irqreturn_t cxlflash_sync_err_irq(int irq, void *data)
  *
  * Return: The number of entries processed.
  */
-static int process_hrrq(struct afu *afu, struct list_head *doneq, int budget)
+static int process_hrrq(struct hwq *hwq, struct list_head *doneq, int budget)
 {
+	struct afu *afu = hwq->afu;
 	struct afu_cmd *cmd;
 	struct sisl_ioasa *ioasa;
 	struct sisl_ioarcb *ioarcb;
-	bool toggle = afu->toggle;
+	bool toggle = hwq->toggle;
 	int num_hrrq = 0;
 	u64 entry,
-	    *hrrq_start = afu->hrrq_start,
-	    *hrrq_end = afu->hrrq_end,
-	    *hrrq_curr = afu->hrrq_curr;
+	    *hrrq_start = hwq->hrrq_start,
+	    *hrrq_end = hwq->hrrq_end,
+	    *hrrq_curr = hwq->hrrq_curr;
 
 	/* Process ready RRQ entries up to the specified budget (if any) */
 	while (true) {
@@ -1160,15 +1207,15 @@ static int process_hrrq(struct afu *afu, struct list_head *doneq, int budget)
 			toggle ^= SISL_RESP_HANDLE_T_BIT;
 		}
 
-		atomic_inc(&afu->hsq_credits);
+		atomic_inc(&hwq->hsq_credits);
 		num_hrrq++;
 
 		if (budget > 0 && num_hrrq >= budget)
 			break;
 	}
 
-	afu->hrrq_curr = hrrq_curr;
-	afu->toggle = toggle;
+	hwq->hrrq_curr = hrrq_curr;
+	hwq->toggle = toggle;
 
 	return num_hrrq;
 }
@@ -1198,18 +1245,18 @@ static void process_cmd_doneq(struct list_head *doneq)
  */
 static int cxlflash_irqpoll(struct irq_poll *irqpoll, int budget)
 {
-	struct afu *afu = container_of(irqpoll, struct afu, irqpoll);
+	struct hwq *hwq = container_of(irqpoll, struct hwq, irqpoll);
 	unsigned long hrrq_flags;
 	LIST_HEAD(doneq);
 	int num_entries = 0;
 
-	spin_lock_irqsave(&afu->hrrq_slock, hrrq_flags);
+	spin_lock_irqsave(&hwq->hrrq_slock, hrrq_flags);
 
-	num_entries = process_hrrq(afu, &doneq, budget);
+	num_entries = process_hrrq(hwq, &doneq, budget);
 	if (num_entries < budget)
 		irq_poll_complete(irqpoll);
 
-	spin_unlock_irqrestore(&afu->hrrq_slock, hrrq_flags);
+	spin_unlock_irqrestore(&hwq->hrrq_slock, hrrq_flags);
 
 	process_cmd_doneq(&doneq);
 	return num_entries;
@@ -1224,21 +1271,22 @@ static int cxlflash_irqpoll(struct irq_poll *irqpoll, int budget)
  */
 static irqreturn_t cxlflash_rrq_irq(int irq, void *data)
 {
-	struct afu *afu = (struct afu *)data;
+	struct hwq *hwq = (struct hwq *)data;
+	struct afu *afu = hwq->afu;
 	unsigned long hrrq_flags;
 	LIST_HEAD(doneq);
 	int num_entries = 0;
 
-	spin_lock_irqsave(&afu->hrrq_slock, hrrq_flags);
+	spin_lock_irqsave(&hwq->hrrq_slock, hrrq_flags);
 
 	if (afu_is_irqpoll_enabled(afu)) {
-		irq_poll_sched(&afu->irqpoll);
-		spin_unlock_irqrestore(&afu->hrrq_slock, hrrq_flags);
+		irq_poll_sched(&hwq->irqpoll);
+		spin_unlock_irqrestore(&hwq->hrrq_slock, hrrq_flags);
 		return IRQ_HANDLED;
 	}
 
-	num_entries = process_hrrq(afu, &doneq, -1);
-	spin_unlock_irqrestore(&afu->hrrq_slock, hrrq_flags);
+	num_entries = process_hrrq(hwq, &doneq, -1);
+	spin_unlock_irqrestore(&hwq->hrrq_slock, hrrq_flags);
 
 	if (num_entries == 0)
 		return IRQ_NONE;
@@ -1285,7 +1333,8 @@ static const struct asyc_intr_info ainfo[] = {
  */
 static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 {
-	struct afu *afu = (struct afu *)data;
+	struct hwq *hwq = (struct hwq *)data;
+	struct afu *afu = hwq->afu;
 	struct cxlflash_cfg *cfg = afu->parent;
 	struct device *dev = &cfg->dev->dev;
 	const struct asyc_intr_info *info;
@@ -1368,16 +1417,18 @@ static irqreturn_t cxlflash_async_err_irq(int irq, void *data)
 /**
  * start_context() - starts the master context
  * @cfg:	Internal structure associated with the host.
+ * @index:	Index of the hardware queue.
  *
  * Return: A success or failure value from CXL services.
  */
-static int start_context(struct cxlflash_cfg *cfg)
+static int start_context(struct cxlflash_cfg *cfg, u32 index)
 {
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq = get_hwq(cfg->afu, index);
 	int rc = 0;
 
-	rc = cxl_start_context(cfg->mcctx,
-			       cfg->afu->work.work_element_descriptor,
+	rc = cxl_start_context(hwq->ctx,
+			       hwq->work.work_element_descriptor,
 			       NULL);
 
 	dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc);
@@ -1487,6 +1538,7 @@ static void init_pcr(struct cxlflash_cfg *cfg)
 {
 	struct afu *afu = cfg->afu;
 	struct sisl_ctrl_map __iomem *ctrl_map;
+	struct hwq *hwq;
 	int i;
 
 	for (i = 0; i < MAX_CONTEXT; i++) {
@@ -1498,13 +1550,17 @@ static void init_pcr(struct cxlflash_cfg *cfg)
 		writeq_be(0, &ctrl_map->ctx_cap);
 	}
 
-	/* Copy frequently used fields into afu */
-	afu->ctx_hndl = (u16) cxl_process_element(cfg->mcctx);
-	afu->host_map = &afu->afu_map->hosts[afu->ctx_hndl].host;
-	afu->ctrl_map = &afu->afu_map->ctrls[afu->ctx_hndl].ctrl;
+	/* Copy frequently used fields into hwq */
+	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		hwq = get_hwq(afu, i);
 
-	/* Program the Endian Control for the master context */
-	writeq_be(SISL_ENDIAN_CTRL, &afu->host_map->endian_ctrl);
+		hwq->ctx_hndl = (u16) cxl_process_element(hwq->ctx);
+		hwq->host_map = &afu->afu_map->hosts[hwq->ctx_hndl].host;
+		hwq->ctrl_map = &afu->afu_map->ctrls[hwq->ctx_hndl].ctrl;
+
+		/* Program the Endian Control for the master context */
+		writeq_be(SISL_ENDIAN_CTRL, &hwq->host_map->endian_ctrl);
+	}
 }
 
 /**
@@ -1515,6 +1571,8 @@ static int init_global(struct cxlflash_cfg *cfg)
 {
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq;
+	struct sisl_host_map __iomem *hmap;
 	__be64 __iomem *fc_port_regs;
 	u64 wwpn[MAX_FC_PORTS];	/* wwpn of AFU ports */
 	int i = 0, num_ports = 0;
@@ -1527,13 +1585,18 @@ static int init_global(struct cxlflash_cfg *cfg)
 		goto out;
 	}
 
-	/* Set up RRQ and SQ in AFU for master issued cmds */
-	writeq_be((u64) afu->hrrq_start, &afu->host_map->rrq_start);
-	writeq_be((u64) afu->hrrq_end, &afu->host_map->rrq_end);
+	/* Set up RRQ and SQ in HWQ for master issued cmds */
+	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		hwq = get_hwq(afu, i);
+		hmap = hwq->host_map;
 
-	if (afu_is_sq_cmd_mode(afu)) {
-		writeq_be((u64)afu->hsq_start, &afu->host_map->sq_start);
-		writeq_be((u64)afu->hsq_end, &afu->host_map->sq_end);
+		writeq_be((u64) hwq->hrrq_start, &hmap->rrq_start);
+		writeq_be((u64) hwq->hrrq_end, &hmap->rrq_end);
+
+		if (afu_is_sq_cmd_mode(afu)) {
+			writeq_be((u64)hwq->hsq_start, &hmap->sq_start);
+			writeq_be((u64)hwq->hsq_end, &hmap->sq_end);
+		}
 	}
 
 	/* AFU configuration */
@@ -1577,11 +1640,15 @@ static int init_global(struct cxlflash_cfg *cfg)
 	/* Set up master's own CTX_CAP to allow real mode, host translation */
 	/* tables, afu cmds and read/write GSCSI cmds. */
 	/* First, unlock ctx_cap write by reading mbox */
-	(void)readq_be(&afu->ctrl_map->mbox_r);	/* unlock ctx_cap */
-	writeq_be((SISL_CTX_CAP_REAL_MODE | SISL_CTX_CAP_HOST_XLATE |
-		   SISL_CTX_CAP_READ_CMD | SISL_CTX_CAP_WRITE_CMD |
-		   SISL_CTX_CAP_AFU_CMD | SISL_CTX_CAP_GSCSI_CMD),
-		  &afu->ctrl_map->ctx_cap);
+	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		hwq = get_hwq(afu, i);
+
+		(void)readq_be(&hwq->ctrl_map->mbox_r);	/* unlock ctx_cap */
+		writeq_be((SISL_CTX_CAP_REAL_MODE | SISL_CTX_CAP_HOST_XLATE |
+			SISL_CTX_CAP_READ_CMD | SISL_CTX_CAP_WRITE_CMD |
+			SISL_CTX_CAP_AFU_CMD | SISL_CTX_CAP_GSCSI_CMD),
+			&hwq->ctrl_map->ctx_cap);
+	}
 	/* Initialize heartbeat */
 	afu->hb = readq_be(&afu->afu_map->global.regs.afu_hb);
 out:
@@ -1596,33 +1663,43 @@ static int start_afu(struct cxlflash_cfg *cfg)
 {
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq;
 	int rc = 0;
+	int i;
 
 	init_pcr(cfg);
 
-	/* Initialize RRQ */
-	memset(&afu->rrq_entry, 0, sizeof(afu->rrq_entry));
-	afu->hrrq_start = &afu->rrq_entry[0];
-	afu->hrrq_end = &afu->rrq_entry[NUM_RRQ_ENTRY - 1];
-	afu->hrrq_curr = afu->hrrq_start;
-	afu->toggle = 1;
-	spin_lock_init(&afu->hrrq_slock);
+	/* Initialize each HWQ */
+	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		hwq = get_hwq(afu, i);
 
-	/* Initialize SQ */
-	if (afu_is_sq_cmd_mode(afu)) {
-		memset(&afu->sq, 0, sizeof(afu->sq));
-		afu->hsq_start = &afu->sq[0];
-		afu->hsq_end = &afu->sq[NUM_SQ_ENTRY - 1];
-		afu->hsq_curr = afu->hsq_start;
+		/* After an AFU reset, RRQ entries are stale, clear them */
+		memset(&hwq->rrq_entry, 0, sizeof(hwq->rrq_entry));
 
-		spin_lock_init(&afu->hsq_slock);
-		atomic_set(&afu->hsq_credits, NUM_SQ_ENTRY - 1);
-	}
+		/* Initialize RRQ pointers */
+		hwq->hrrq_start = &hwq->rrq_entry[0];
+		hwq->hrrq_end = &hwq->rrq_entry[NUM_RRQ_ENTRY - 1];
+		hwq->hrrq_curr = hwq->hrrq_start;
+		hwq->toggle = 1;
+		spin_lock_init(&hwq->hrrq_slock);
+
+		/* Initialize SQ */
+		if (afu_is_sq_cmd_mode(afu)) {
+			memset(&hwq->sq, 0, sizeof(hwq->sq));
+			hwq->hsq_start = &hwq->sq[0];
+			hwq->hsq_end = &hwq->sq[NUM_SQ_ENTRY - 1];
+			hwq->hsq_curr = hwq->hsq_start;
+
+			spin_lock_init(&hwq->hsq_slock);
+			atomic_set(&hwq->hsq_credits, NUM_SQ_ENTRY - 1);
+		}
 
-	/* Initialize IRQ poll */
-	if (afu_is_irqpoll_enabled(afu))
-		irq_poll_init(&afu->irqpoll, afu->irqpoll_weight,
-			      cxlflash_irqpoll);
+		/* Initialize IRQ poll */
+		if (afu_is_irqpoll_enabled(afu))
+			irq_poll_init(&hwq->irqpoll, afu->irqpoll_weight,
+				      cxlflash_irqpoll);
+
+	}
 
 	rc = init_global(cfg);
 
@@ -1633,18 +1710,21 @@ static int start_afu(struct cxlflash_cfg *cfg)
 /**
  * init_intr() - setup interrupt handlers for the master context
  * @cfg:	Internal structure associated with the host.
+ * @hwq:	Hardware queue to initialize.
  *
  * Return: 0 on success, -errno on failure
  */
 static enum undo_level init_intr(struct cxlflash_cfg *cfg,
-				 struct cxl_context *ctx)
+				 struct hwq *hwq)
 {
-	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	struct cxl_context *ctx = hwq->ctx;
 	int rc = 0;
 	enum undo_level level = UNDO_NOOP;
+	bool is_primary_hwq = (hwq->index == PRIMARY_HWQ);
+	int num_irqs = is_primary_hwq ? 3 : 2;
 
-	rc = cxl_allocate_afu_irqs(ctx, 3);
+	rc = cxl_allocate_afu_irqs(ctx, num_irqs);
 	if (unlikely(rc)) {
 		dev_err(dev, "%s: allocate_afu_irqs failed rc=%d\n",
 			__func__, rc);
@@ -1652,7 +1732,7 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 		goto out;
 	}
 
-	rc = cxl_map_afu_irq(ctx, 1, cxlflash_sync_err_irq, afu,
+	rc = cxl_map_afu_irq(ctx, 1, cxlflash_sync_err_irq, hwq,
 			     "SISL_MSI_SYNC_ERROR");
 	if (unlikely(rc <= 0)) {
 		dev_err(dev, "%s: SISL_MSI_SYNC_ERROR map failed\n", __func__);
@@ -1660,7 +1740,7 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 		goto out;
 	}
 
-	rc = cxl_map_afu_irq(ctx, 2, cxlflash_rrq_irq, afu,
+	rc = cxl_map_afu_irq(ctx, 2, cxlflash_rrq_irq, hwq,
 			     "SISL_MSI_RRQ_UPDATED");
 	if (unlikely(rc <= 0)) {
 		dev_err(dev, "%s: SISL_MSI_RRQ_UPDATED map failed\n", __func__);
@@ -1668,7 +1748,11 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 		goto out;
 	}
 
-	rc = cxl_map_afu_irq(ctx, 3, cxlflash_async_err_irq, afu,
+	/* SISL_MSI_ASYNC_ERROR is setup only for the primary HWQ */
+	if (!is_primary_hwq)
+		goto out;
+
+	rc = cxl_map_afu_irq(ctx, 3, cxlflash_async_err_irq, hwq,
 			     "SISL_MSI_ASYNC_ERROR");
 	if (unlikely(rc <= 0)) {
 		dev_err(dev, "%s: SISL_MSI_ASYNC_ERROR map failed\n", __func__);
@@ -1682,55 +1766,73 @@ static enum undo_level init_intr(struct cxlflash_cfg *cfg,
 /**
  * init_mc() - create and register as the master context
  * @cfg:	Internal structure associated with the host.
+ * index:	HWQ Index of the master context.
  *
  * Return: 0 on success, -errno on failure
  */
-static int init_mc(struct cxlflash_cfg *cfg)
+static int init_mc(struct cxlflash_cfg *cfg, u32 index)
 {
 	struct cxl_context *ctx;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq = get_hwq(cfg->afu, index);
 	int rc = 0;
 	enum undo_level level;
 
-	ctx = cxl_get_context(cfg->dev);
+	hwq->afu = cfg->afu;
+	hwq->index = index;
+
+	if (index == PRIMARY_HWQ)
+		ctx = cxl_get_context(cfg->dev);
+	else
+		ctx = cxl_dev_context_init(cfg->dev);
 	if (unlikely(!ctx)) {
 		rc = -ENOMEM;
-		goto ret;
+		goto err1;
 	}
-	cfg->mcctx = ctx;
+
+	WARN_ON(hwq->ctx);
+	hwq->ctx = ctx;
 
 	/* Set it up as a master with the CXL */
 	cxl_set_master(ctx);
 
-	/* During initialization reset the AFU to start from a clean slate */
-	rc = cxl_afu_reset(cfg->mcctx);
-	if (unlikely(rc)) {
-		dev_err(dev, "%s: AFU reset failed rc=%d\n", __func__, rc);
-		goto ret;
+	/* Reset AFU when initializing primary context */
+	if (index == PRIMARY_HWQ) {
+		rc = cxl_afu_reset(ctx);
+		if (unlikely(rc)) {
+			dev_err(dev, "%s: AFU reset failed rc=%d\n",
+				      __func__, rc);
+			goto err1;
+		}
 	}
 
-	level = init_intr(cfg, ctx);
+	level = init_intr(cfg, hwq);
 	if (unlikely(level)) {
 		dev_err(dev, "%s: interrupt init failed rc=%d\n", __func__, rc);
-		goto out;
+		goto err2;
 	}
 
 	/* This performs the equivalent of the CXL_IOCTL_START_WORK.
 	 * The CXL_IOCTL_GET_PROCESS_ELEMENT is implicit in the process
 	 * element (pe) that is embedded in the context (ctx)
 	 */
-	rc = start_context(cfg);
+	rc = start_context(cfg, index);
 	if (unlikely(rc)) {
 		dev_err(dev, "%s: start context failed rc=%d\n", __func__, rc);
 		level = UNMAP_THREE;
-		goto out;
+		goto err2;
 	}
-ret:
+
+out:
 	dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc);
 	return rc;
-out:
-	term_intr(cfg, level);
-	goto ret;
+err2:
+	term_intr(cfg, level, index);
+	if (index != PRIMARY_HWQ)
+		cxl_release_context(ctx);
+err1:
+	hwq->ctx = NULL;
+	goto out;
 }
 
 /**
@@ -1781,18 +1883,23 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	int rc = 0;
 	struct afu *afu = cfg->afu;
 	struct device *dev = &cfg->dev->dev;
+	struct hwq *hwq;
+	int i;
 
 	cxl_perst_reloads_same_image(cfg->cxl_afu, true);
 
-	rc = init_mc(cfg);
-	if (rc) {
-		dev_err(dev, "%s: init_mc failed rc=%d\n",
-			__func__, rc);
-		goto out;
+	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		rc = init_mc(cfg, i);
+		if (rc) {
+			dev_err(dev, "%s: init_mc failed rc=%d index=%d\n",
+				__func__, rc, i);
+			goto err1;
+		}
 	}
 
-	/* Map the entire MMIO space of the AFU */
-	afu->afu_map = cxl_psa_map(cfg->mcctx);
+	/* Map the entire MMIO space of the AFU using the first context */
+	hwq = get_hwq(afu, PRIMARY_HWQ);
+	afu->afu_map = cxl_psa_map(hwq->ctx);
 	if (!afu->afu_map) {
 		dev_err(dev, "%s: cxl_psa_map failed\n", __func__);
 		rc = -ENOMEM;
@@ -1832,8 +1939,12 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	}
 
 	afu_err_intr_init(cfg->afu);
-	spin_lock_init(&afu->rrin_slock);
-	afu->room = readq_be(&afu->host_map->cmd_room);
+	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		hwq = get_hwq(afu, i);
+
+		spin_lock_init(&hwq->rrin_slock);
+		hwq->room = readq_be(&hwq->host_map->cmd_room);
+	}
 
 	/* Restore the LUN mappings */
 	cxlflash_restore_luntable(cfg);
@@ -1842,8 +1953,10 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	return rc;
 
 err1:
-	term_intr(cfg, UNMAP_THREE);
-	term_mc(cfg);
+	for (i = CXLFLASH_NUM_HWQS - 1; i >= 0; i--) {
+		term_intr(cfg, UNMAP_THREE, i);
+		term_mc(cfg, i);
+	}
 	goto out;
 }
 
@@ -1875,6 +1988,7 @@ int cxlflash_afu_sync(struct afu *afu, ctx_hndl_t ctx_hndl_u,
 	struct cxlflash_cfg *cfg = afu->parent;
 	struct device *dev = &cfg->dev->dev;
 	struct afu_cmd *cmd = NULL;
+	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
 	char *buf = NULL;
 	int rc = 0;
 	static DEFINE_MUTEX(sync_active);
@@ -1897,11 +2011,12 @@ int cxlflash_afu_sync(struct afu *afu, ctx_hndl_t ctx_hndl_u,
 	cmd = (struct afu_cmd *)PTR_ALIGN(buf, __alignof__(*cmd));
 	init_completion(&cmd->cevent);
 	cmd->parent = afu;
+	cmd->hwq_index = hwq->index;
 
 	dev_dbg(dev, "%s: afu=%p cmd=%p %d\n", __func__, afu, cmd, ctx_hndl_u);
 
 	cmd->rcb.req_flags = SISL_REQ_FLAGS_AFU_CMD;
-	cmd->rcb.ctx_id = afu->ctx_hndl;
+	cmd->rcb.ctx_id = hwq->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
 	cmd->rcb.timeout = MC_AFU_SYNC_TIMEOUT;
 
@@ -2414,8 +2529,9 @@ static ssize_t irqpoll_weight_store(struct device *dev,
 	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
 	struct device *cfgdev = &cfg->dev->dev;
 	struct afu *afu = cfg->afu;
+	struct hwq *hwq;
 	u32 weight;
-	int rc;
+	int rc, i;
 
 	rc = kstrtouint(buf, 10, &weight);
 	if (rc)
@@ -2433,13 +2549,23 @@ static ssize_t irqpoll_weight_store(struct device *dev,
 		return -EINVAL;
 	}
 
-	if (afu_is_irqpoll_enabled(afu))
-		irq_poll_disable(&afu->irqpoll);
+	if (afu_is_irqpoll_enabled(afu)) {
+		for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+			hwq = get_hwq(afu, i);
+
+			irq_poll_disable(&hwq->irqpoll);
+		}
+	}
 
 	afu->irqpoll_weight = weight;
 
-	if (weight > 0)
-		irq_poll_init(&afu->irqpoll, weight, cxlflash_irqpoll);
+	if (weight > 0) {
+		for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+			hwq = get_hwq(afu, i);
+
+			irq_poll_init(&hwq->irqpoll, weight, cxlflash_irqpoll);
+		}
+	}
 
 	return count;
 }
diff --git a/drivers/scsi/cxlflash/superpipe.c b/drivers/scsi/cxlflash/superpipe.c
index 158fa00..52ec525 100644
--- a/drivers/scsi/cxlflash/superpipe.c
+++ b/drivers/scsi/cxlflash/superpipe.c
@@ -254,6 +254,7 @@ static int afu_attach(struct cxlflash_cfg *cfg, struct ctx_info *ctxi)
 	struct afu *afu = cfg->afu;
 	struct sisl_ctrl_map __iomem *ctrl_map = ctxi->ctrl_map;
 	int rc = 0;
+	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
 	u64 val;
 
 	/* Unlock cap and restrict user to read/write cmds in translated mode */
@@ -270,7 +271,7 @@ static int afu_attach(struct cxlflash_cfg *cfg, struct ctx_info *ctxi)
 
 	/* Set up MMIO registers pointing to the RHT */
 	writeq_be((u64)ctxi->rht_start, &ctrl_map->rht_start);
-	val = SISL_RHT_CNT_ID((u64)MAX_RHT_PER_CONTEXT, (u64)(afu->ctx_hndl));
+	val = SISL_RHT_CNT_ID((u64)MAX_RHT_PER_CONTEXT, (u64)(hwq->ctx_hndl));
 	writeq_be(val, &ctrl_map->rht_cnt_id);
 out:
 	dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc);
@@ -1628,6 +1629,7 @@ static int cxlflash_afu_recover(struct scsi_device *sdev,
 	struct afu *afu = cfg->afu;
 	struct ctx_info *ctxi = NULL;
 	struct mutex *mutex = &cfg->ctx_recovery_mutex;
+	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
 	u64 flags;
 	u64 ctxid = DECODE_CTXID(recover->context_id),
 	    rctxid = recover->context_id;
@@ -1698,7 +1700,7 @@ static int cxlflash_afu_recover(struct scsi_device *sdev,
 	}
 
 	/* Test if in error state */
-	reg = readq_be(&afu->ctrl_map->mbox_r);
+	reg = readq_be(&hwq->ctrl_map->mbox_r);
 	if (reg == -1) {
 		dev_dbg(dev, "%s: MMIO fail, wait for recovery.\n", __func__);
 
-- 
2.1.0

[PATCH 16/17] cxlflash: Add hardware queues attribute

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

As staging for supporting multiple hardware queues, add an attribute to
show and set the current number of hardware queues for the host. Support
specifying a hard limit or a CPU affinitized value. This will allow the
number of hardware queues to be tuned by a system administrator.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h |  10 ++--
 drivers/scsi/cxlflash/main.c   | 112 ++++++++++++++++++++++++++++++++++++-----
 2 files changed, 106 insertions(+), 16 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index b5858ae..8fd7a1f 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -60,7 +60,9 @@ extern const struct file_operations cxlflash_cxl_fops;
 /* SQ for master issued cmds */
 #define NUM_SQ_ENTRY			CXLFLASH_MAX_CMDS
 
-#define CXLFLASH_NUM_HWQS		1
+/* Hardware queue definitions */
+#define CXLFLASH_DEF_HWQS		1
+#define CXLFLASH_MAX_HWQS		8
 #define PRIMARY_HWQ			0
 
 
@@ -201,7 +203,7 @@ struct hwq {
 } __aligned(cache_line_size());
 
 struct afu {
-	struct hwq hwqs[CXLFLASH_NUM_HWQS];
+	struct hwq hwqs[CXLFLASH_MAX_HWQS];
 	int (*send_cmd)(struct afu *, struct afu_cmd *);
 	void (*context_reset)(struct afu_cmd *);
 
@@ -211,6 +213,8 @@ struct afu {
 	atomic_t cmds_active;	/* Number of currently active AFU commands */
 	u64 hb;
 	u32 internal_lun;	/* User-desired LUN mode for this AFU */
+	u32 num_hwqs;		/* Number of hardware queues */
+	u32 desired_hwqs;	/* Desired h/w queues, effective on AFU reset */
 
 	char version[16];
 	u64 interface_version;
@@ -221,7 +225,7 @@ struct afu {
 
 static inline struct hwq *get_hwq(struct afu *afu, u32 index)
 {
-	WARN_ON(index >= CXLFLASH_NUM_HWQS);
+	WARN_ON(index >= CXLFLASH_MAX_HWQS);
 
 	return &afu->hwqs[index];
 }
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 5d06869..113797a 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -566,7 +566,7 @@ static void stop_afu(struct cxlflash_cfg *cfg)
 			ssleep(1);
 
 		if (afu_is_irqpoll_enabled(afu)) {
-			for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+			for (i = 0; i < afu->num_hwqs; i++) {
 				hwq = get_hwq(afu, i);
 
 				irq_poll_disable(&hwq->irqpoll);
@@ -676,13 +676,13 @@ static void term_afu(struct cxlflash_cfg *cfg)
 	 * 2) Unmap the problem state area
 	 * 3) Stop each master context
 	 */
-	for (k = CXLFLASH_NUM_HWQS - 1; k >= 0; k--)
+	for (k = cfg->afu->num_hwqs - 1; k >= 0; k--)
 		term_intr(cfg, UNMAP_THREE, k);
 
 	if (cfg->afu)
 		stop_afu(cfg);
 
-	for (k = CXLFLASH_NUM_HWQS - 1; k >= 0; k--)
+	for (k = cfg->afu->num_hwqs - 1; k >= 0; k--)
 		term_mc(cfg, k);
 
 	dev_dbg(dev, "%s: returning\n", __func__);
@@ -823,6 +823,7 @@ static int alloc_mem(struct cxlflash_cfg *cfg)
 		goto out;
 	}
 	cfg->afu->parent = cfg;
+	cfg->afu->desired_hwqs = CXLFLASH_DEF_HWQS;
 	cfg->afu->afu_map = NULL;
 out:
 	return rc;
@@ -1116,7 +1117,7 @@ static void afu_err_intr_init(struct afu *afu)
 	/* IOARRIN yet), so there is nothing to clear. */
 
 	/* set LISN#, it is always sent to the context that wrote IOARRIN */
-	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
 
 		writeq_be(SISL_MSI_SYNC_ERROR, &hwq->host_map->ctx_ctrl);
@@ -1551,7 +1552,7 @@ static void init_pcr(struct cxlflash_cfg *cfg)
 	}
 
 	/* Copy frequently used fields into hwq */
-	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
 
 		hwq->ctx_hndl = (u16) cxl_process_element(hwq->ctx);
@@ -1586,7 +1587,7 @@ static int init_global(struct cxlflash_cfg *cfg)
 	}
 
 	/* Set up RRQ and SQ in HWQ for master issued cmds */
-	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
 		hmap = hwq->host_map;
 
@@ -1640,7 +1641,7 @@ static int init_global(struct cxlflash_cfg *cfg)
 	/* Set up master's own CTX_CAP to allow real mode, host translation */
 	/* tables, afu cmds and read/write GSCSI cmds. */
 	/* First, unlock ctx_cap write by reading mbox */
-	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
 
 		(void)readq_be(&hwq->ctrl_map->mbox_r);	/* unlock ctx_cap */
@@ -1670,7 +1671,7 @@ static int start_afu(struct cxlflash_cfg *cfg)
 	init_pcr(cfg);
 
 	/* Initialize each HWQ */
-	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
 
 		/* After an AFU reset, RRQ entries are stale, clear them */
@@ -1888,7 +1889,8 @@ static int init_afu(struct cxlflash_cfg *cfg)
 
 	cxl_perst_reloads_same_image(cfg->cxl_afu, true);
 
-	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+	afu->num_hwqs = afu->desired_hwqs;
+	for (i = 0; i < afu->num_hwqs; i++) {
 		rc = init_mc(cfg, i);
 		if (rc) {
 			dev_err(dev, "%s: init_mc failed rc=%d index=%d\n",
@@ -1939,7 +1941,7 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	}
 
 	afu_err_intr_init(cfg->afu);
-	for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+	for (i = 0; i < afu->num_hwqs; i++) {
 		hwq = get_hwq(afu, i);
 
 		spin_lock_init(&hwq->rrin_slock);
@@ -1953,7 +1955,7 @@ static int init_afu(struct cxlflash_cfg *cfg)
 	return rc;
 
 err1:
-	for (i = CXLFLASH_NUM_HWQS - 1; i >= 0; i--) {
+	for (i = afu->num_hwqs - 1; i >= 0; i--) {
 		term_intr(cfg, UNMAP_THREE, i);
 		term_mc(cfg, i);
 	}
@@ -2550,7 +2552,7 @@ static ssize_t irqpoll_weight_store(struct device *dev,
 	}
 
 	if (afu_is_irqpoll_enabled(afu)) {
-		for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		for (i = 0; i < afu->num_hwqs; i++) {
 			hwq = get_hwq(afu, i);
 
 			irq_poll_disable(&hwq->irqpoll);
@@ -2560,7 +2562,7 @@ static ssize_t irqpoll_weight_store(struct device *dev,
 	afu->irqpoll_weight = weight;
 
 	if (weight > 0) {
-		for (i = 0; i < CXLFLASH_NUM_HWQS; i++) {
+		for (i = 0; i < afu->num_hwqs; i++) {
 			hwq = get_hwq(afu, i);
 
 			irq_poll_init(&hwq->irqpoll, weight, cxlflash_irqpoll);
@@ -2571,6 +2573,88 @@ static ssize_t irqpoll_weight_store(struct device *dev,
 }
 
 /**
+ * num_hwqs_show() - presents the number of hardware queues for the host
+ * @dev:	Generic device associated with the host.
+ * @attr:	Device attribute representing the number of hardware queues.
+ * @buf:	Buffer of length PAGE_SIZE to report back the number of hardware
+ *		queues in ASCII.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t num_hwqs_show(struct device *dev,
+			     struct device_attribute *attr, char *buf)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+	struct afu *afu = cfg->afu;
+
+	return scnprintf(buf, PAGE_SIZE, "%u\n", afu->num_hwqs);
+}
+
+/**
+ * num_hwqs_store() - sets the number of hardware queues for the host
+ * @dev:	Generic device associated with the host.
+ * @attr:	Device attribute representing the number of hardware queues.
+ * @buf:	Buffer of length PAGE_SIZE containing the number of hardware
+ *		queues in ASCII.
+ * @count:	Length of data resizing in @buf.
+ *
+ * n > 0: num_hwqs = n
+ * n = 0: num_hwqs = num_online_cpus()
+ * n < 0: num_online_cpus() / abs(n)
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t num_hwqs_store(struct device *dev,
+			      struct device_attribute *attr,
+			      const char *buf, size_t count)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+	struct afu *afu = cfg->afu;
+	int rc;
+	int nhwqs, num_hwqs;
+
+	rc = kstrtoint(buf, 10, &nhwqs);
+	if (rc)
+		return -EINVAL;
+
+	if (nhwqs >= 1)
+		num_hwqs = nhwqs;
+	else if (nhwqs == 0)
+		num_hwqs = num_online_cpus();
+	else
+		num_hwqs = num_online_cpus() / abs(nhwqs);
+
+	afu->desired_hwqs = min(num_hwqs, CXLFLASH_MAX_HWQS);
+	WARN_ON_ONCE(afu->desired_hwqs == 0);
+
+retry:
+	switch (cfg->state) {
+	case STATE_NORMAL:
+		cfg->state = STATE_RESET;
+		drain_ioctls(cfg);
+		cxlflash_mark_contexts_error(cfg);
+		rc = afu_reset(cfg);
+		if (rc)
+			cfg->state = STATE_FAILTERM;
+		else
+			cfg->state = STATE_NORMAL;
+		wake_up_all(&cfg->reset_waitq);
+		break;
+	case STATE_RESET:
+		wait_event(cfg->reset_waitq, cfg->state != STATE_RESET);
+		if (cfg->state == STATE_NORMAL)
+			goto retry;
+	default:
+		/* Ideally should not happen */
+		dev_err(dev, "%s: Device is not ready, state=%d\n",
+			__func__, cfg->state);
+		break;
+	}
+
+	return count;
+}
+
+/**
  * mode_show() - presents the current mode of the device
  * @dev:	Generic device associated with the device.
  * @attr:	Device attribute representing the device mode.
@@ -2601,6 +2685,7 @@ static DEVICE_ATTR_RO(port1_lun_table);
 static DEVICE_ATTR_RO(port2_lun_table);
 static DEVICE_ATTR_RO(port3_lun_table);
 static DEVICE_ATTR_RW(irqpoll_weight);
+static DEVICE_ATTR_RW(num_hwqs);
 
 static struct device_attribute *cxlflash_host_attrs[] = {
 	&dev_attr_port0,
@@ -2614,6 +2699,7 @@ static struct device_attribute *cxlflash_host_attrs[] = {
 	&dev_attr_port2_lun_table,
 	&dev_attr_port3_lun_table,
 	&dev_attr_irqpoll_weight,
+	&dev_attr_num_hwqs,
 	NULL
 };
 
-- 
2.1.0

[PATCH 17/17] cxlflash: Introduce hardware queue steering

[Uma Krishnan]

From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>

As an enhancement to distribute requests to multiple hardware queues, add
the infrastructure to hash a SCSI command into a particular hardware queue.
Support the following scenarios when deriving which queue to use: single
queue, tagging when SCSI-MQ enabled, and simple hash via CPU ID when
SCSI-MQ is disabled. Rather than altering the existing send API, the
derived hardware queue is stored in the AFU command where it can be used
for sending a command to the chosen hardware queue.

Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
---
 drivers/scsi/cxlflash/common.h |  12 ++++-
 drivers/scsi/cxlflash/main.c   | 120 +++++++++++++++++++++++++++++++++++++++--
 2 files changed, 126 insertions(+), 6 deletions(-)

diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
index 8fd7a1f..256af81 100644
--- a/drivers/scsi/cxlflash/common.h
+++ b/drivers/scsi/cxlflash/common.h
@@ -96,6 +96,13 @@ enum cxlflash_state {
 	STATE_FAILTERM	/* Failed/terminating state, error out users/threads */
 };
 
+enum cxlflash_hwq_mode {
+	HWQ_MODE_RR,	/* Roundrobin (default) */
+	HWQ_MODE_TAG,	/* Distribute based on block MQ tag */
+	HWQ_MODE_CPU,	/* CPU affinity */
+	MAX_HWQ_MODE
+};
+
 /*
  * Each context has its own set of resource handles that is visible
  * only from that context.
@@ -146,9 +153,9 @@ struct afu_cmd {
 	struct scsi_cmnd *scp;
 	struct completion cevent;
 	struct list_head queue;
+	u32 hwq_index;
 
 	u8 cmd_tmf:1;
-	u32 hwq_index;
 
 	/* As per the SISLITE spec the IOARCB EA has to be 16-byte aligned.
 	 * However for performance reasons the IOARCB/IOASA should be
@@ -213,8 +220,11 @@ struct afu {
 	atomic_t cmds_active;	/* Number of currently active AFU commands */
 	u64 hb;
 	u32 internal_lun;	/* User-desired LUN mode for this AFU */
+
 	u32 num_hwqs;		/* Number of hardware queues */
 	u32 desired_hwqs;	/* Desired h/w queues, effective on AFU reset */
+	enum cxlflash_hwq_mode hwq_mode; /* Steering mode for h/w queues */
+	u32 hwq_rr_count;	/* Count to distribute traffic for roundrobin */
 
 	char version[16];
 	u64 interface_version;
diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c
index 113797a..a7d57c3 100644
--- a/drivers/scsi/cxlflash/main.c
+++ b/drivers/scsi/cxlflash/main.c
@@ -358,6 +358,43 @@ static int wait_resp(struct afu *afu, struct afu_cmd *cmd)
 }
 
 /**
+ * cmd_to_target_hwq() - selects a target hardware queue for a SCSI command
+ * @host:	SCSI host associated with device.
+ * @scp:	SCSI command to send.
+ * @afu:	SCSI command to send.
+ *
+ * Hashes a command based upon the hardware queue mode.
+ *
+ * Return: Trusted index of target hardware queue
+ */
+static u32 cmd_to_target_hwq(struct Scsi_Host *host, struct scsi_cmnd *scp,
+			     struct afu *afu)
+{
+	u32 tag;
+	u32 hwq = 0;
+
+	if (afu->num_hwqs == 1)
+		return 0;
+
+	switch (afu->hwq_mode) {
+	case HWQ_MODE_RR:
+		hwq = afu->hwq_rr_count++ % afu->num_hwqs;
+		break;
+	case HWQ_MODE_TAG:
+		tag = blk_mq_unique_tag(scp->request);
+		hwq = blk_mq_unique_tag_to_hwq(tag);
+		break;
+	case HWQ_MODE_CPU:
+		hwq = smp_processor_id() % afu->num_hwqs;
+		break;
+	default:
+		WARN_ON_ONCE(1);
+	}
+
+	return hwq;
+}
+
+/**
  * send_tmf() - sends a Task Management Function (TMF)
  * @afu:	AFU to checkout from.
  * @scp:	SCSI command from stack.
@@ -368,10 +405,12 @@ static int wait_resp(struct afu *afu, struct afu_cmd *cmd)
  */
 static int send_tmf(struct afu *afu, struct scsi_cmnd *scp, u64 tmfcmd)
 {
-	struct cxlflash_cfg *cfg = shost_priv(scp->device->host);
+	struct Scsi_Host *host = scp->device->host;
+	struct cxlflash_cfg *cfg = shost_priv(host);
 	struct afu_cmd *cmd = sc_to_afucz(scp);
 	struct device *dev = &cfg->dev->dev;
-	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
+	int hwq_index = cmd_to_target_hwq(host, scp, afu);
+	struct hwq *hwq = get_hwq(afu, hwq_index);
 	ulong lock_flags;
 	int rc = 0;
 	ulong to;
@@ -388,7 +427,7 @@ static int send_tmf(struct afu *afu, struct scsi_cmnd *scp, u64 tmfcmd)
 	cmd->scp = scp;
 	cmd->parent = afu;
 	cmd->cmd_tmf = true;
-	cmd->hwq_index = hwq->index;
+	cmd->hwq_index = hwq_index;
 
 	cmd->rcb.ctx_id = hwq->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
@@ -448,7 +487,8 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 	struct device *dev = &cfg->dev->dev;
 	struct afu_cmd *cmd = sc_to_afucz(scp);
 	struct scatterlist *sg = scsi_sglist(scp);
-	struct hwq *hwq = get_hwq(afu, PRIMARY_HWQ);
+	int hwq_index = cmd_to_target_hwq(host, scp, afu);
+	struct hwq *hwq = get_hwq(afu, hwq_index);
 	u16 req_flags = SISL_REQ_FLAGS_SUP_UNDERRUN;
 	ulong lock_flags;
 	int rc = 0;
@@ -498,7 +538,7 @@ static int cxlflash_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *scp)
 
 	cmd->scp = scp;
 	cmd->parent = afu;
-	cmd->hwq_index = hwq->index;
+	cmd->hwq_index = hwq_index;
 
 	cmd->rcb.ctx_id = hwq->ctx_hndl;
 	cmd->rcb.msi = SISL_MSI_RRQ_UPDATED;
@@ -2654,6 +2694,74 @@ static ssize_t num_hwqs_store(struct device *dev,
 	return count;
 }
 
+static const char *hwq_mode_name[MAX_HWQ_MODE] = { "rr", "tag", "cpu" };
+
+/**
+ * hwq_mode_show() - presents the HWQ steering mode for the host
+ * @dev:	Generic device associated with the host.
+ * @attr:	Device attribute representing the HWQ steering mode.
+ * @buf:	Buffer of length PAGE_SIZE to report back the HWQ steering mode
+ *		as a character string.
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t hwq_mode_show(struct device *dev,
+			     struct device_attribute *attr, char *buf)
+{
+	struct cxlflash_cfg *cfg = shost_priv(class_to_shost(dev));
+	struct afu *afu = cfg->afu;
+
+	return scnprintf(buf, PAGE_SIZE, "%s\n", hwq_mode_name[afu->hwq_mode]);
+}
+
+/**
+ * hwq_mode_store() - sets the HWQ steering mode for the host
+ * @dev:	Generic device associated with the host.
+ * @attr:	Device attribute representing the HWQ steering mode.
+ * @buf:	Buffer of length PAGE_SIZE containing the HWQ steering mode
+ *		as a character string.
+ * @count:	Length of data resizing in @buf.
+ *
+ * rr = Round-Robin
+ * tag = Block MQ Tagging
+ * cpu = CPU Affinity
+ *
+ * Return: The size of the ASCII string returned in @buf.
+ */
+static ssize_t hwq_mode_store(struct device *dev,
+			      struct device_attribute *attr,
+			      const char *buf, size_t count)
+{
+	struct Scsi_Host *shost = class_to_shost(dev);
+	struct cxlflash_cfg *cfg = shost_priv(shost);
+	struct device *cfgdev = &cfg->dev->dev;
+	struct afu *afu = cfg->afu;
+	int i;
+	u32 mode = MAX_HWQ_MODE;
+
+	for (i = 0; i < MAX_HWQ_MODE; i++) {
+		if (!strncmp(hwq_mode_name[i], buf, strlen(hwq_mode_name[i]))) {
+			mode = i;
+			break;
+		}
+	}
+
+	if (mode >= MAX_HWQ_MODE) {
+		dev_info(cfgdev, "Invalid HWQ steering mode.\n");
+		return -EINVAL;
+	}
+
+	if ((mode == HWQ_MODE_TAG) && !shost_use_blk_mq(shost)) {
+		dev_info(cfgdev, "SCSI-MQ is not enabled, use a different "
+			 "HWQ steering mode.\n");
+		return -EINVAL;
+	}
+
+	afu->hwq_mode = mode;
+
+	return count;
+}
+
 /**
  * mode_show() - presents the current mode of the device
  * @dev:	Generic device associated with the device.
@@ -2686,6 +2794,7 @@ static DEVICE_ATTR_RO(port2_lun_table);
 static DEVICE_ATTR_RO(port3_lun_table);
 static DEVICE_ATTR_RW(irqpoll_weight);
 static DEVICE_ATTR_RW(num_hwqs);
+static DEVICE_ATTR_RW(hwq_mode);
 
 static struct device_attribute *cxlflash_host_attrs[] = {
 	&dev_attr_port0,
@@ -2700,6 +2809,7 @@ static struct device_attribute *cxlflash_host_attrs[] = {
 	&dev_attr_port3_lun_table,
 	&dev_attr_irqpoll_weight,
 	&dev_attr_num_hwqs,
+	&dev_attr_hwq_mode,
 	NULL
 };
 
-- 
2.1.0

Re: [PATCH 10/17] cxlflash: Fence EEH during probe

[Andrew Donnellan]

On 13/04/17 05:14, Uma Krishnan wrote:
> From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>
>
> An EEH during probe can lead to a crash as the recovery thread races
> with the probe thread. To avoid this issue, introduce new states to
> fence out EEH recovery until probe has completed. Also ensure the reset
> wait queue is flushed during device removal to avoid orphaned threads.
>
> Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>

The use of the term "fence" in the commit message might be a bit 
confusing given how the term is already used in the context of EEH :)


Andrew

-- 
Andrew Donnellan              OzLabs, ADL Canberra
andrew.donnellan@au1.ibm.com  IBM Australia Limited

Re: [PATCH 10/17] cxlflash: Fence EEH during probe

[Matthew R. Ochs]

> On Apr 13, 2017, at 1:27 AM, Andrew Donnellan <andrew.donnellan@au1.ibm.com> wrote:
> 
> On 13/04/17 05:14, Uma Krishnan wrote:
>> From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>
>> 
>> An EEH during probe can lead to a crash as the recovery thread races
>> with the probe thread. To avoid this issue, introduce new states to
>> fence out EEH recovery until probe has completed. Also ensure the reset
>> wait queue is flushed during device removal to avoid orphaned threads.
>> 
>> Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
>> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
> 
> The use of the term "fence" in the commit message might be a bit confusing given how the term is already used in the context of EEH :)

Hi Andrew,

I agree that it could be a bit misleading but think the commit message is
descriptive enough to avoid confusion. If you feel strongly about it I'm happy
to reword the commit.


-matt

Re: [PATCH 10/17] cxlflash: Fence EEH during probe

[Matthew R. Ochs]

> On Apr 13, 2017, at 1:27 AM, Andrew Donnellan =
<andrew.donnellan@au1.ibm.com> wrote:
>=20
> On 13/04/17 05:14, Uma Krishnan wrote:
>> From: "Matthew R. Ochs" <mrochs@linux.vnet.ibm.com>
>>=20
>> An EEH during probe can lead to a crash as the recovery thread races
>> with the probe thread. To avoid this issue, introduce new states to
>> fence out EEH recovery until probe has completed. Also ensure the =
reset
>> wait queue is flushed during device removal to avoid orphaned =
threads.
>>=20
>> Signed-off-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>
>> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>
>=20
> The use of the term "fence" in the commit message might be a bit =
confusing given how the term is already used in the context of EEH :)

Hi Andrew,

I agree that it could be a bit misleading but think the commit message =
is
descriptive enough to avoid confusion. If you feel strongly about it I'm =
happy
to reword the commit.


-matt=

Re: [PATCH 15/17] cxlflash: Support multiple hardware queues

[Matthew R. Ochs]

> On Apr 12, 2017, at 2:15 PM, Uma Krishnan <ukrishn@linux.vnet.ibm.com> wrote:
> 
> Introduce multiple hardware queues to improve legacy I/O path performance.
> Each hardware queue is comprised of a master context and associated I/O
> resources. The hardware queues are initially implemented as a static array
> embedded in the AFU. This will be transitioned to a dynamic allocation in a
> later series to improve the memory footprint of the driver.
> 
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>

Looks good!

Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>

Re: [PATCH 15/17] cxlflash: Support multiple hardware queues

[Matthew R. Ochs]

> On Apr 12, 2017, at 2:15 PM, Uma Krishnan <ukrishn@linux.vnet.ibm.com> =
wrote:
>=20
> Introduce multiple hardware queues to improve legacy I/O path =
performance.
> Each hardware queue is comprised of a master context and associated =
I/O
> resources. The hardware queues are initially implemented as a static =
array
> embedded in the AFU. This will be transitioned to a dynamic allocation =
in a
> later series to improve the memory footprint of the driver.
>=20
> Signed-off-by: Uma Krishnan <ukrishn@linux.vnet.ibm.com>

Looks good!

Acked-by: Matthew R. Ochs <mrochs@linux.vnet.ibm.com>=

Re: [PATCH 00/17] cxlflash: Enhancements and miscellaneous fixes

[Martin K. Petersen]

Uma,

> This patch series contains miscellaneous patches and adds 4 port
> device support. This series also includes patches to improve
> performance of the driver in the legacy IO path.

Applied to 4.12/scsi-queue. Thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering