* [PATCH RFC 0/8] IB/srp: Add multichannel support
@ 2014-09-19 12:55 Bart Van Assche
2014-09-19 12:56 ` Bart Van Assche
` (7 more replies)
0 siblings, 8 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:55 UTC (permalink / raw)
To: linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
Hello,
Although the SRP protocol supports multichannel operation, although
since considerable time RDMA HCA's are available that support multiple
completion vectors and although multichannel operation yields better
performance than using a single channel, the Linux SRP initiator does
not yet support multichannel operation. While adding multichannel
support in the SRP initiator I encountered a few challenges of which I
think these need wider discussion. The topics I would like invite wider
discussion about are as follows:
- How to avoid unneeded inter-socket cache traffic. Should the blk-mq
layer e.g. assign CPU cores to hardware queues such that all CPU cores
associated with a single hardware queue reside on the same CPU socket?
(see also patch 1/8)
- How to pass the hardware context selected by the block layer to the
SCSI LLD queuecommand() callback function ? (see also patches 2/8 and
3/8).
- Which approach should a SCSI LLD follow for selection of an MSI-X
completion vector to ensure that the interrupt handler is invoked on
the same CPU socket as the blk-mq hardware context data structures ?
As one can see patch 8/8 relies on the assumption that completion
vectors have been spread evenly over CPU sockets. If a HCA e.g.
supports eight completion vectors then that means that in a system
with two CPU sockets vectors 0-3 are associated with a CPU core on
the first CPU socket and vectors 4-7 with a CPU core on the second CPU
socket.
The patches in this series are:
0001-blk-mq-Use-all-available-hardware-queues.patch
0002-scsi-mq-Add-support-for-multiple-hardware-queues.patch
0003-scsi-mq-Pass-hctx-to-low-level-SCSI-drivers.patch
0004-IB-srp-Move-ib_destroy_cm_id-call-into-srp_free_ch_i.patch
0005-IB-srp-Remove-stale-connection-retry-mechanism.patch
0006-IB-srp-Avoid-that-I-O-hangs-due-to-a-cable-pull-duri.patch
0007-IB-srp-Separate-target-and-channel-variables.patch
0008-IB-srp-Add-multichannel-support.patch
Note: a predecessor of this patch series has been used to measure the
performance of Christoph's scsi-mq patches that have been merged in
kernel version v3.17-rc1.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH RFC 0/8] IB/srp: Add multichannel support
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
@ 2014-09-19 12:56 ` Bart Van Assche
2014-09-19 18:02 ` Sagi Grimberg
2014-09-19 12:57 ` [PATCH 2/8] scsi-mq: Add support for multiple hardware queues Bart Van Assche
` (6 subsequent siblings)
7 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:56 UTC (permalink / raw)
To: linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
[PATCH 1/8] blk-mq: Use all available hardware queues
Suppose that a system has two CPU sockets, three cores per socket,
that it does not support hyperthreading and that four hardware
queues are provided by a block driver. With the current algorithm
this will lead to the following assignment of CPU cores to hardware
queues:
HWQ 0: 0 1
HWQ 1: 2 3
HWQ 2: 4 5
HWQ 3: (none)
This patch changes the queue assignment into:
HWQ 0: 0 1
HWQ 1: 2
HWQ 2: 3 4
HWQ 3: 5
In other words, this patch has the following three effects:
- All four hardware queues are used instead of only three.
- CPU cores are spread more evenly over hardware queues. For the
above example the range of the number of CPU cores associated
with a single HWQ is reduced from [0..2] to [1..2].
- If the number of HWQ's is a multiple of the number of CPU sockets
it is now guaranteed that all CPU cores associated with a single
HWQ reside on the same CPU socket.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: Jens Axboe <axboe@fb.com>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Ming Lei <ming.lei@canonical.com>
---
block/blk-mq-cpumap.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
index 1065d7c..8e56455 100644
--- a/block/blk-mq-cpumap.c
+++ b/block/blk-mq-cpumap.c
@@ -17,7 +17,7 @@
static int cpu_to_queue_index(unsigned int nr_cpus, unsigned int nr_queues,
const int cpu)
{
- return cpu / ((nr_cpus + nr_queues - 1) / nr_queues);
+ return cpu * nr_queues / nr_cpus;
}
static int get_first_sibling(unsigned int cpu)
--
1.8.4.5
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH 2/8] scsi-mq: Add support for multiple hardware queues
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
2014-09-19 12:56 ` Bart Van Assche
@ 2014-09-19 12:57 ` Bart Van Assche
[not found] ` <541C281E.9090206-HInyCGIudOg@public.gmane.org>
2014-09-19 12:57 ` [PATCH 3/8] scsi-mq: Pass hctx to low-level SCSI drivers Bart Van Assche
` (5 subsequent siblings)
7 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:57 UTC (permalink / raw)
To: linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
Allow a SCSI LLD to declare how many hardware queues it supports
by setting Scsi_Host.nr_hw_queues before calling scsi_add_host().
Note: it is assumed that each hardware queue has a queue depth of
shost->can_queue. In other words, the total queue depth per host
is (number of hardware queues) * (shost->can_queue).
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: Christoph Hellwig <hch@lst.de>
---
drivers/scsi/scsi_lib.c | 2 +-
include/scsi/scsi_host.h | 4 ++++
2 files changed, 5 insertions(+), 1 deletion(-)
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index d837dc1..b0b6117 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -2071,7 +2071,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
memset(&shost->tag_set, 0, sizeof(shost->tag_set));
shost->tag_set.ops = &scsi_mq_ops;
- shost->tag_set.nr_hw_queues = 1;
+ shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
shost->tag_set.queue_depth = shost->can_queue;
shost->tag_set.cmd_size = cmd_size;
shost->tag_set.numa_node = NUMA_NO_NODE;
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index ba20347..0a867d9 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -638,6 +638,10 @@ struct Scsi_Host {
short unsigned int sg_prot_tablesize;
unsigned int max_sectors;
unsigned long dma_boundary;
+ /*
+ * In scsi-mq mode, the number of hardware queues supported by the LLD.
+ */
+ unsigned nr_hw_queues;
/*
* Used to assign serial numbers to the cmds.
* Protected by the host lock.
--
1.8.4.5
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH 3/8] scsi-mq: Pass hctx to low-level SCSI drivers
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
2014-09-19 12:56 ` Bart Van Assche
2014-09-19 12:57 ` [PATCH 2/8] scsi-mq: Add support for multiple hardware queues Bart Van Assche
@ 2014-09-19 12:57 ` Bart Van Assche
[not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
` (4 subsequent siblings)
7 siblings, 0 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:57 UTC (permalink / raw)
To: linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
Low-level drivers (LLDs) need to know which hardware context has been
selected by the block layer. Hence pass this information to SCSI LLDs.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
Cc: Christoph Hellwig <hch@lst.de>
---
drivers/scsi/scsi.c | 7 +++++--
drivers/scsi/scsi_lib.c | 4 ++--
drivers/scsi/scsi_priv.h | 2 +-
include/scsi/scsi_host.h | 14 ++++++++++++++
4 files changed, 22 insertions(+), 5 deletions(-)
diff --git a/drivers/scsi/scsi.c b/drivers/scsi/scsi.c
index d81f3cc..9bd4bd0 100644
--- a/drivers/scsi/scsi.c
+++ b/drivers/scsi/scsi.c
@@ -640,9 +640,10 @@ EXPORT_SYMBOL(scsi_cmd_get_serial);
* Return: nonzero return request was rejected and device's queue needs to be
* plugged.
*/
-int scsi_dispatch_cmd(struct scsi_cmnd *cmd)
+int scsi_dispatch_cmd(struct blk_mq_hw_ctx *hctx, struct scsi_cmnd *cmd)
{
struct Scsi_Host *host = cmd->device->host;
+ struct scsi_host_template *hostt = host->hostt;
int rtn = 0;
atomic_inc(&cmd->device->iorequest_cnt);
@@ -701,7 +702,9 @@ int scsi_dispatch_cmd(struct scsi_cmnd *cmd)
}
trace_scsi_dispatch_cmd_start(cmd);
- rtn = host->hostt->queuecommand(host, cmd);
+ rtn = hctx && hostt->mq_queuecommand ?
+ hostt->mq_queuecommand(hctx, cmd) :
+ hostt->queuecommand(host, cmd);
if (rtn) {
trace_scsi_dispatch_cmd_error(cmd, rtn);
if (rtn != SCSI_MLQUEUE_DEVICE_BUSY &&
diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index b0b6117..6303648 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -1747,7 +1747,7 @@ static void scsi_request_fn(struct request_queue *q)
* Dispatch the command to the low-level driver.
*/
cmd->scsi_done = scsi_done;
- rtn = scsi_dispatch_cmd(cmd);
+ rtn = scsi_dispatch_cmd(NULL, cmd);
if (rtn) {
scsi_queue_insert(cmd, rtn);
spin_lock_irq(q->queue_lock);
@@ -1889,7 +1889,7 @@ static int scsi_queue_rq(struct blk_mq_hw_ctx *hctx, struct request *req)
scsi_init_cmd_errh(cmd);
cmd->scsi_done = scsi_mq_done;
- reason = scsi_dispatch_cmd(cmd);
+ reason = scsi_dispatch_cmd(hctx, cmd);
if (reason) {
scsi_set_blocked(cmd, reason);
ret = BLK_MQ_RQ_QUEUE_BUSY;
diff --git a/drivers/scsi/scsi_priv.h b/drivers/scsi/scsi_priv.h
index 12b8e1b..f4115f6 100644
--- a/drivers/scsi/scsi_priv.h
+++ b/drivers/scsi/scsi_priv.h
@@ -29,7 +29,7 @@ extern int scsi_init_hosts(void);
extern void scsi_exit_hosts(void);
/* scsi.c */
-extern int scsi_dispatch_cmd(struct scsi_cmnd *cmd);
+extern int scsi_dispatch_cmd(struct blk_mq_hw_ctx *hctx, struct scsi_cmnd *cmd);
extern int scsi_setup_command_freelist(struct Scsi_Host *shost);
extern void scsi_destroy_command_freelist(struct Scsi_Host *shost);
#ifdef CONFIG_SCSI_LOGGING
diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
index 0a867d9..c695f1c 100644
--- a/include/scsi/scsi_host.h
+++ b/include/scsi/scsi_host.h
@@ -133,6 +133,20 @@ struct scsi_host_template {
int (* queuecommand)(struct Scsi_Host *, struct scsi_cmnd *);
/*
+ * scsi-mq version of queuecommand(). Must be provided by LLDDs that
+ * provide multiple hardware queues and that need to know on which
+ * hardware context a command has been queued by the block layer.
+ *
+ * Note: even if an LLDD provides an mq_queuecommand callback function
+ * it still has to provide a queuecommand callback function. The SCSI
+ * error handler namely can invoke the queuecommand callback function
+ * even if scsi-mq is enabled.
+ *
+ * STATUS: OPTIONAL
+ */
+ int (* mq_queuecommand)(struct blk_mq_hw_ctx *hctx, struct scsi_cmnd *);
+
+ /*
* This is an error handling strategy routine. You don't need to
* define one of these if you don't want to - there is a default
* routine that is present that should work in most cases. For those
--
1.8.4.5
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH 4/8] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib()
[not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
@ 2014-09-19 12:58 ` Bart Van Assche
[not found] ` <541C285B.5010309-HInyCGIudOg@public.gmane.org>
2014-09-19 12:58 ` [PATCH 5/8] IB/srp: Remove stale connection retry mechanism Bart Van Assche
` (2 subsequent siblings)
3 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:58 UTC (permalink / raw)
To: linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
The patch that adds multichannel support into the SRP initiator
driver introduces an additional call to srp_free_ch_ib(). This
patch helps to keep that later patch simple.
Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
---
drivers/infiniband/ulp/srp/ib_srp.c | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 62d2a18..d3c712f 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -555,6 +555,11 @@ static void srp_free_target_ib(struct srp_target_port *target)
struct srp_device *dev = target->srp_host->srp_dev;
int i;
+ if (target->cm_id) {
+ ib_destroy_cm_id(target->cm_id);
+ target->cm_id = NULL;
+ }
+
if (dev->use_fast_reg) {
if (target->fr_pool)
srp_destroy_fr_pool(target->fr_pool);
@@ -868,7 +873,6 @@ static void srp_remove_target(struct srp_target_port *target)
scsi_remove_host(target->scsi_host);
srp_stop_rport_timers(target->rport);
srp_disconnect_target(target);
- ib_destroy_cm_id(target->cm_id);
srp_free_target_ib(target);
cancel_work_sync(&target->tl_err_work);
srp_rport_put(target->rport);
@@ -3043,7 +3047,7 @@ static ssize_t srp_create_target(struct device *dev,
if (ret) {
shost_printk(KERN_ERR, target->scsi_host,
PFX "Connection failed\n");
- goto err_cm_id;
+ goto err_free_ib;
}
ret = srp_add_target(host, target);
@@ -3067,9 +3071,6 @@ out:
err_disconnect:
srp_disconnect_target(target);
-err_cm_id:
- ib_destroy_cm_id(target->cm_id);
-
err_free_ib:
srp_free_target_ib(target);
--
1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH 5/8] IB/srp: Remove stale connection retry mechanism
[not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
2014-09-19 12:58 ` [PATCH 4/8] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib() Bart Van Assche
@ 2014-09-19 12:58 ` Bart Van Assche
2014-09-19 18:25 ` Sagi Grimberg
[not found] ` <541C287D.1050900-HInyCGIudOg@public.gmane.org>
2014-09-19 13:00 ` [PATCH 8/8] IB/srp: Add multichannel support Bart Van Assche
2014-10-01 16:14 ` [PATCH RFC] scsi_tcq.h: Add support for multiple hardware queues Bart Van Assche
3 siblings, 2 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:58 UTC (permalink / raw)
To: linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
Attempting to connect three times may be insufficient after an
initiator system that was using multiple RDMA channels tries to
relogin. Additionally, this login retry mechanism is a workaround
for particular behavior of the IB/CM. Since the srp_daemon retries
a failed login attempt anyway, remove the stale connection retry
mechanism.
Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
---
drivers/infiniband/ulp/srp/ib_srp.c | 16 +++-------------
1 file changed, 3 insertions(+), 13 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index d3c712f..9608e7a 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -904,7 +904,6 @@ static void srp_rport_delete(struct srp_rport *rport)
static int srp_connect_target(struct srp_target_port *target)
{
- int retries = 3;
int ret;
WARN_ON_ONCE(target->connected);
@@ -945,19 +944,10 @@ static int srp_connect_target(struct srp_target_port *target)
break;
case SRP_STALE_CONN:
- /* Our current CM id was stale, and is now in timewait.
- * Try to reconnect with a new one.
- */
- if (!retries-- || srp_new_cm_id(target)) {
- shost_printk(KERN_ERR, target->scsi_host, PFX
- "giving up on stale connection\n");
- target->status = -ECONNRESET;
- return target->status;
- }
-
shost_printk(KERN_ERR, target->scsi_host, PFX
- "retrying stale connection\n");
- break;
+ "giving up on stale connection\n");
+ target->status = -ECONNRESET;
+ return target->status;
default:
return target->status;
--
1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH 6/8] IB/srp: Avoid that I/O hangs due to a cable pull during LUN scanning
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
` (3 preceding siblings ...)
[not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
@ 2014-09-19 12:59 ` Bart Van Assche
2014-09-19 12:59 ` [PATCH 7/8] IB/srp: Separate target and channel variables Bart Van Assche
` (2 subsequent siblings)
7 siblings, 0 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:59 UTC (permalink / raw)
To: linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
If a cable is pulled during LUN scanning it can happen that the
SRP rport and the SCSI host have been created but no LUNs have been
added to the SCSI host. Since multipathd only sends SCSI commands
to a SCSI target if one or more SCSI devices are present and since
there is no keepalive mechanism for IB queue pairs this means that
after a LUN scan failed and after a reconnect has succeeded no
data will be sent over the QP and hence that a subsequent cable
pull will not be detected. Avoid this by not creating an rport or
SCSI host if a cable is pulled during a SCSI LUN scan.
Note: so far the above behavior has only been observed with the
kernel module parameter ch_count set to a value >= 2.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---
drivers/infiniband/ulp/srp/ib_srp.c | 56 +++++++++++++++++++++++++++++++------
drivers/infiniband/ulp/srp/ib_srp.h | 1 +
2 files changed, 48 insertions(+), 9 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 9608e7a..fd88fb8 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -1111,6 +1111,10 @@ static int srp_rport_reconnect(struct srp_rport *rport)
int i, ret;
srp_disconnect_target(target);
+
+ if (target->state == SRP_TARGET_SCANNING)
+ return -ENODEV;
+
/*
* Now get a new local CM ID so that we avoid confusing the target in
* case things are really fouled up. Doing so also ensures that all CM
@@ -2607,11 +2611,23 @@ static struct scsi_host_template srp_template = {
.shost_attrs = srp_host_attrs
};
+static int srp_sdev_count(struct Scsi_Host *host)
+{
+ struct scsi_device *sdev;
+ int c = 0;
+
+ shost_for_each_device(sdev, host)
+ c++;
+
+ return c;
+}
+
static int srp_add_target(struct srp_host *host, struct srp_target_port *target)
{
struct srp_rport_identifiers ids;
struct srp_rport *rport;
+ target->state = SRP_TARGET_SCANNING;
sprintf(target->target_name, "SRP.T10:%016llX",
(unsigned long long) be64_to_cpu(target->id_ext));
@@ -2634,11 +2650,26 @@ static int srp_add_target(struct srp_host *host, struct srp_target_port *target)
list_add_tail(&target->list, &host->target_list);
spin_unlock(&host->target_lock);
- target->state = SRP_TARGET_LIVE;
-
scsi_scan_target(&target->scsi_host->shost_gendev,
0, target->scsi_id, SCAN_WILD_CARD, 0);
+ if (!target->connected || target->qp_in_error) {
+ shost_printk(KERN_INFO, target->scsi_host,
+ PFX "SCSI scan failed - removing SCSI host\n");
+ srp_queue_remove_work(target);
+ goto out;
+ }
+
+ pr_debug(PFX "%s: SCSI scan succeeded - detected %d LUNs\n",
+ dev_name(&target->scsi_host->shost_gendev),
+ srp_sdev_count(target->scsi_host));
+
+ spin_lock_irq(&target->lock);
+ if (target->state == SRP_TARGET_SCANNING)
+ target->state = SRP_TARGET_LIVE;
+ spin_unlock_irq(&target->lock);
+
+out:
return 0;
}
@@ -3044,13 +3075,20 @@ static ssize_t srp_create_target(struct device *dev,
if (ret)
goto err_disconnect;
- shost_printk(KERN_DEBUG, target->scsi_host, PFX
- "new target: id_ext %016llx ioc_guid %016llx pkey %04x service_id %016llx sgid %pI6 dgid %pI6\n",
- be64_to_cpu(target->id_ext),
- be64_to_cpu(target->ioc_guid),
- be16_to_cpu(target->path.pkey),
- be64_to_cpu(target->service_id),
- target->path.sgid.raw, target->path.dgid.raw);
+ /* Protects against concurrent srp_remove_target() invocation. */
+ scsi_host_get(target->scsi_host);
+
+ if (target->state != SRP_TARGET_REMOVED) {
+ shost_printk(KERN_DEBUG, target->scsi_host, PFX
+ "new target: id_ext %016llx ioc_guid %016llx pkey %04x service_id %016llx sgid %pI6 dgid %pI6\n",
+ be64_to_cpu(target->id_ext),
+ be64_to_cpu(target->ioc_guid),
+ be16_to_cpu(target->path.pkey),
+ be64_to_cpu(target->service_id),
+ target->path.sgid.raw, target->orig_dgid);
+ }
+
+ scsi_host_put(target->scsi_host);
ret = count;
diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
index e46ecb1..00c7c48 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.h
+++ b/drivers/infiniband/ulp/srp/ib_srp.h
@@ -73,6 +73,7 @@ enum {
};
enum srp_target_state {
+ SRP_TARGET_SCANNING,
SRP_TARGET_LIVE,
SRP_TARGET_REMOVED,
};
--
1.8.4.5
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH 7/8] IB/srp: Separate target and channel variables
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
` (4 preceding siblings ...)
2014-09-19 12:59 ` [PATCH 6/8] IB/srp: Avoid that I/O hangs due to a cable pull during LUN scanning Bart Van Assche
@ 2014-09-19 12:59 ` Bart Van Assche
2014-09-19 18:47 ` Sagi Grimberg
[not found] ` <541C28C8.7000007-HInyCGIudOg@public.gmane.org>
2014-09-19 18:31 ` [PATCH RFC 0/8] IB/srp: Add multichannel support Jens Axboe
2014-09-22 14:37 ` Christoph Hellwig
7 siblings, 2 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 12:59 UTC (permalink / raw)
To: linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
Changes in this patch:
- Move channel variables into a new structure (struct srp_rdma_ch).
- cm_id and completion handler context pointer are now of type
srp_rdma_ch * insteoad of srp_target_port *.
No functionality is changed.
Signed-off-by: Bart Van Assche <bvanassche@acm.org>
---
drivers/infiniband/ulp/srp/ib_srp.c | 707 +++++++++++++++++++-----------------
drivers/infiniband/ulp/srp/ib_srp.h | 59 +--
2 files changed, 417 insertions(+), 349 deletions(-)
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index fd88fb8..9feeea1 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -125,8 +125,8 @@ MODULE_PARM_DESC(dev_loss_tmo,
static void srp_add_one(struct ib_device *device);
static void srp_remove_one(struct ib_device *device);
-static void srp_recv_completion(struct ib_cq *cq, void *target_ptr);
-static void srp_send_completion(struct ib_cq *cq, void *target_ptr);
+static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr);
+static void srp_send_completion(struct ib_cq *cq, void *ch_ptr);
static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event);
static struct scsi_transport_template *ib_srp_transport_template;
@@ -262,7 +262,7 @@ static int srp_init_qp(struct srp_target_port *target,
ret = ib_find_pkey(target->srp_host->srp_dev->dev,
target->srp_host->port,
- be16_to_cpu(target->path.pkey),
+ be16_to_cpu(target->pkey),
&attr->pkey_index);
if (ret)
goto out;
@@ -283,18 +283,23 @@ out:
return ret;
}
-static int srp_new_cm_id(struct srp_target_port *target)
+static int srp_new_cm_id(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct ib_cm_id *new_cm_id;
new_cm_id = ib_create_cm_id(target->srp_host->srp_dev->dev,
- srp_cm_handler, target);
+ srp_cm_handler, ch);
if (IS_ERR(new_cm_id))
return PTR_ERR(new_cm_id);
- if (target->cm_id)
- ib_destroy_cm_id(target->cm_id);
- target->cm_id = new_cm_id;
+ if (ch->cm_id)
+ ib_destroy_cm_id(ch->cm_id);
+ ch->cm_id = new_cm_id;
+ ch->path.sgid = target->sgid;
+ ch->path.dgid = target->orig_dgid;
+ ch->path.pkey = target->pkey;
+ ch->path.service_id = target->service_id;
return 0;
}
@@ -443,8 +448,9 @@ static struct srp_fr_pool *srp_alloc_fr_pool(struct srp_target_port *target)
dev->max_pages_per_mr);
}
-static int srp_create_target_ib(struct srp_target_port *target)
+static int srp_create_ch_ib(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_qp_init_attr *init_attr;
struct ib_cq *recv_cq, *send_cq;
@@ -458,15 +464,15 @@ static int srp_create_target_ib(struct srp_target_port *target)
if (!init_attr)
return -ENOMEM;
- recv_cq = ib_create_cq(dev->dev, srp_recv_completion, NULL, target,
- target->queue_size, target->comp_vector);
+ recv_cq = ib_create_cq(dev->dev, srp_recv_completion, NULL, ch,
+ target->queue_size, ch->comp_vector);
if (IS_ERR(recv_cq)) {
ret = PTR_ERR(recv_cq);
goto err;
}
- send_cq = ib_create_cq(dev->dev, srp_send_completion, NULL, target,
- m * target->queue_size, target->comp_vector);
+ send_cq = ib_create_cq(dev->dev, srp_send_completion, NULL, ch,
+ m * target->queue_size, ch->comp_vector);
if (IS_ERR(send_cq)) {
ret = PTR_ERR(send_cq);
goto err_recv_cq;
@@ -502,9 +508,9 @@ static int srp_create_target_ib(struct srp_target_port *target)
"FR pool allocation failed (%d)\n", ret);
goto err_qp;
}
- if (target->fr_pool)
- srp_destroy_fr_pool(target->fr_pool);
- target->fr_pool = fr_pool;
+ if (ch->fr_pool)
+ srp_destroy_fr_pool(ch->fr_pool);
+ ch->fr_pool = fr_pool;
} else if (!dev->use_fast_reg && dev->has_fmr) {
fmr_pool = srp_alloc_fmr_pool(target);
if (IS_ERR(fmr_pool)) {
@@ -513,21 +519,21 @@ static int srp_create_target_ib(struct srp_target_port *target)
"FMR pool allocation failed (%d)\n", ret);
goto err_qp;
}
- if (target->fmr_pool)
- ib_destroy_fmr_pool(target->fmr_pool);
- target->fmr_pool = fmr_pool;
+ if (ch->fmr_pool)
+ ib_destroy_fmr_pool(ch->fmr_pool);
+ ch->fmr_pool = fmr_pool;
}
- if (target->qp)
- ib_destroy_qp(target->qp);
- if (target->recv_cq)
- ib_destroy_cq(target->recv_cq);
- if (target->send_cq)
- ib_destroy_cq(target->send_cq);
+ if (ch->qp)
+ ib_destroy_qp(ch->qp);
+ if (ch->recv_cq)
+ ib_destroy_cq(ch->recv_cq);
+ if (ch->send_cq)
+ ib_destroy_cq(ch->send_cq);
- target->qp = qp;
- target->recv_cq = recv_cq;
- target->send_cq = send_cq;
+ ch->qp = qp;
+ ch->recv_cq = recv_cq;
+ ch->send_cq = send_cq;
kfree(init_attr);
return 0;
@@ -548,98 +554,102 @@ err:
/*
* Note: this function may be called without srp_alloc_iu_bufs() having been
- * invoked. Hence the target->[rt]x_ring checks.
+ * invoked. Hence the ch->[rt]x_ring checks.
*/
-static void srp_free_target_ib(struct srp_target_port *target)
+static void srp_free_ch_ib(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
int i;
- if (target->cm_id) {
- ib_destroy_cm_id(target->cm_id);
- target->cm_id = NULL;
+ if (ch->cm_id) {
+ ib_destroy_cm_id(ch->cm_id);
+ ch->cm_id = NULL;
}
if (dev->use_fast_reg) {
- if (target->fr_pool)
- srp_destroy_fr_pool(target->fr_pool);
+ if (ch->fr_pool)
+ srp_destroy_fr_pool(ch->fr_pool);
} else {
- if (target->fmr_pool)
- ib_destroy_fmr_pool(target->fmr_pool);
+ if (ch->fmr_pool)
+ ib_destroy_fmr_pool(ch->fmr_pool);
}
- ib_destroy_qp(target->qp);
- ib_destroy_cq(target->send_cq);
- ib_destroy_cq(target->recv_cq);
+ ib_destroy_qp(ch->qp);
+ ib_destroy_cq(ch->send_cq);
+ ib_destroy_cq(ch->recv_cq);
- target->qp = NULL;
- target->send_cq = target->recv_cq = NULL;
+ ch->qp = NULL;
+ ch->send_cq = ch->recv_cq = NULL;
- if (target->rx_ring) {
+ if (ch->rx_ring) {
for (i = 0; i < target->queue_size; ++i)
- srp_free_iu(target->srp_host, target->rx_ring[i]);
- kfree(target->rx_ring);
- target->rx_ring = NULL;
+ srp_free_iu(target->srp_host, ch->rx_ring[i]);
+ kfree(ch->rx_ring);
+ ch->rx_ring = NULL;
}
- if (target->tx_ring) {
+ if (ch->tx_ring) {
for (i = 0; i < target->queue_size; ++i)
- srp_free_iu(target->srp_host, target->tx_ring[i]);
- kfree(target->tx_ring);
- target->tx_ring = NULL;
+ srp_free_iu(target->srp_host, ch->tx_ring[i]);
+ kfree(ch->tx_ring);
+ ch->tx_ring = NULL;
}
}
static void srp_path_rec_completion(int status,
struct ib_sa_path_rec *pathrec,
- void *target_ptr)
+ void *ch_ptr)
{
- struct srp_target_port *target = target_ptr;
+ struct srp_rdma_ch *ch = ch_ptr;
+ struct srp_target_port *target = ch->target;
- target->status = status;
+ ch->status = status;
if (status)
shost_printk(KERN_ERR, target->scsi_host,
PFX "Got failed path rec status %d\n", status);
else
- target->path = *pathrec;
- complete(&target->done);
+ ch->path = *pathrec;
+ complete(&ch->done);
}
-static int srp_lookup_path(struct srp_target_port *target)
+static int srp_lookup_path(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
int ret;
- target->path.numb_path = 1;
-
- init_completion(&target->done);
-
- target->path_query_id = ib_sa_path_rec_get(&srp_sa_client,
- target->srp_host->srp_dev->dev,
- target->srp_host->port,
- &target->path,
- IB_SA_PATH_REC_SERVICE_ID |
- IB_SA_PATH_REC_DGID |
- IB_SA_PATH_REC_SGID |
- IB_SA_PATH_REC_NUMB_PATH |
- IB_SA_PATH_REC_PKEY,
- SRP_PATH_REC_TIMEOUT_MS,
- GFP_KERNEL,
- srp_path_rec_completion,
- target, &target->path_query);
- if (target->path_query_id < 0)
- return target->path_query_id;
-
- ret = wait_for_completion_interruptible(&target->done);
+ ch->path.numb_path = 1;
+
+ init_completion(&ch->done);
+
+ ch->path_query_id = ib_sa_path_rec_get(&srp_sa_client,
+ target->srp_host->srp_dev->dev,
+ target->srp_host->port,
+ &ch->path,
+ IB_SA_PATH_REC_SERVICE_ID |
+ IB_SA_PATH_REC_DGID |
+ IB_SA_PATH_REC_SGID |
+ IB_SA_PATH_REC_NUMB_PATH |
+ IB_SA_PATH_REC_PKEY,
+ SRP_PATH_REC_TIMEOUT_MS,
+ GFP_KERNEL,
+ srp_path_rec_completion,
+ ch, &ch->path_query);
+ if (ch->path_query_id < 0)
+ return ch->path_query_id;
+
+ ret = wait_for_completion_interruptible(&ch->done);
if (ret < 0)
return ret;
- if (target->status < 0)
+ if (ch->status < 0)
shost_printk(KERN_WARNING, target->scsi_host,
PFX "Path record query failed\n");
- return target->status;
+ return ch->status;
}
-static int srp_send_req(struct srp_target_port *target)
+static int srp_send_req(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct {
struct ib_cm_req_param param;
struct srp_login_req priv;
@@ -650,11 +660,11 @@ static int srp_send_req(struct srp_target_port *target)
if (!req)
return -ENOMEM;
- req->param.primary_path = &target->path;
+ req->param.primary_path = &ch->path;
req->param.alternate_path = NULL;
req->param.service_id = target->service_id;
- req->param.qp_num = target->qp->qp_num;
- req->param.qp_type = target->qp->qp_type;
+ req->param.qp_num = ch->qp->qp_num;
+ req->param.qp_type = ch->qp->qp_type;
req->param.private_data = &req->priv;
req->param.private_data_len = sizeof req->priv;
req->param.flow_control = 1;
@@ -689,7 +699,7 @@ static int srp_send_req(struct srp_target_port *target)
*/
if (target->io_class == SRP_REV10_IB_IO_CLASS) {
memcpy(req->priv.initiator_port_id,
- &target->path.sgid.global.interface_id, 8);
+ &target->sgid.global.interface_id, 8);
memcpy(req->priv.initiator_port_id + 8,
&target->initiator_ext, 8);
memcpy(req->priv.target_port_id, &target->ioc_guid, 8);
@@ -698,7 +708,7 @@ static int srp_send_req(struct srp_target_port *target)
memcpy(req->priv.initiator_port_id,
&target->initiator_ext, 8);
memcpy(req->priv.initiator_port_id + 8,
- &target->path.sgid.global.interface_id, 8);
+ &target->sgid.global.interface_id, 8);
memcpy(req->priv.target_port_id, &target->id_ext, 8);
memcpy(req->priv.target_port_id + 8, &target->ioc_guid, 8);
}
@@ -718,7 +728,7 @@ static int srp_send_req(struct srp_target_port *target)
&target->srp_host->srp_dev->dev->node_guid, 8);
}
- status = ib_send_cm_req(target->cm_id, &req->param);
+ status = ib_send_cm_req(ch->cm_id, &req->param);
kfree(req);
@@ -759,28 +769,31 @@ static bool srp_change_conn_state(struct srp_target_port *target,
static void srp_disconnect_target(struct srp_target_port *target)
{
+ struct srp_rdma_ch *ch = &target->ch;
+
if (srp_change_conn_state(target, false)) {
/* XXX should send SRP_I_LOGOUT request */
- if (ib_send_cm_dreq(target->cm_id, NULL, 0)) {
+ if (ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
shost_printk(KERN_DEBUG, target->scsi_host,
PFX "Sending CM DREQ failed\n");
}
}
}
-static void srp_free_req_data(struct srp_target_port *target)
+static void srp_free_req_data(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_device *ibdev = dev->dev;
struct srp_request *req;
int i;
- if (!target->req_ring)
+ if (!ch->req_ring)
return;
for (i = 0; i < target->req_ring_size; ++i) {
- req = &target->req_ring[i];
+ req = &ch->req_ring[i];
if (dev->use_fast_reg)
kfree(req->fr_list);
else
@@ -794,12 +807,13 @@ static void srp_free_req_data(struct srp_target_port *target)
kfree(req->indirect_desc);
}
- kfree(target->req_ring);
- target->req_ring = NULL;
+ kfree(ch->req_ring);
+ ch->req_ring = NULL;
}
-static int srp_alloc_req_data(struct srp_target_port *target)
+static int srp_alloc_req_data(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *srp_dev = target->srp_host->srp_dev;
struct ib_device *ibdev = srp_dev->dev;
struct srp_request *req;
@@ -807,15 +821,15 @@ static int srp_alloc_req_data(struct srp_target_port *target)
dma_addr_t dma_addr;
int i, ret = -ENOMEM;
- INIT_LIST_HEAD(&target->free_reqs);
+ INIT_LIST_HEAD(&ch->free_reqs);
- target->req_ring = kzalloc(target->req_ring_size *
- sizeof(*target->req_ring), GFP_KERNEL);
- if (!target->req_ring)
+ ch->req_ring = kcalloc(target->req_ring_size, sizeof(*ch->req_ring),
+ GFP_KERNEL);
+ if (!ch->req_ring)
goto out;
for (i = 0; i < target->req_ring_size; ++i) {
- req = &target->req_ring[i];
+ req = &ch->req_ring[i];
mr_list = kmalloc(target->cmd_sg_cnt * sizeof(void *),
GFP_KERNEL);
if (!mr_list)
@@ -840,7 +854,7 @@ static int srp_alloc_req_data(struct srp_target_port *target)
req->indirect_dma_addr = dma_addr;
req->index = i;
- list_add_tail(&req->list, &target->free_reqs);
+ list_add_tail(&req->list, &ch->free_reqs);
}
ret = 0;
@@ -865,6 +879,8 @@ static void srp_del_scsi_host_attr(struct Scsi_Host *shost)
static void srp_remove_target(struct srp_target_port *target)
{
+ struct srp_rdma_ch *ch = &target->ch;
+
WARN_ON_ONCE(target->state != SRP_TARGET_REMOVED);
srp_del_scsi_host_attr(target->scsi_host);
@@ -873,10 +889,10 @@ static void srp_remove_target(struct srp_target_port *target)
scsi_remove_host(target->scsi_host);
srp_stop_rport_timers(target->rport);
srp_disconnect_target(target);
- srp_free_target_ib(target);
+ srp_free_ch_ib(ch);
cancel_work_sync(&target->tl_err_work);
srp_rport_put(target->rport);
- srp_free_req_data(target);
+ srp_free_req_data(ch);
spin_lock(&target->srp_host->target_lock);
list_del(&target->list);
@@ -902,24 +918,25 @@ static void srp_rport_delete(struct srp_rport *rport)
srp_queue_remove_work(target);
}
-static int srp_connect_target(struct srp_target_port *target)
+static int srp_connect_ch(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
int ret;
WARN_ON_ONCE(target->connected);
target->qp_in_error = false;
- ret = srp_lookup_path(target);
+ ret = srp_lookup_path(ch);
if (ret)
return ret;
while (1) {
- init_completion(&target->done);
- ret = srp_send_req(target);
+ init_completion(&ch->done);
+ ret = srp_send_req(ch);
if (ret)
return ret;
- ret = wait_for_completion_interruptible(&target->done);
+ ret = wait_for_completion_interruptible(&ch->done);
if (ret < 0)
return ret;
@@ -929,13 +946,13 @@ static int srp_connect_target(struct srp_target_port *target)
* back, or SRP_DLID_REDIRECT if we get a lid/qp
* redirect REJ back.
*/
- switch (target->status) {
+ switch (ch->status) {
case 0:
srp_change_conn_state(target, true);
return 0;
case SRP_PORT_REDIRECT:
- ret = srp_lookup_path(target);
+ ret = srp_lookup_path(ch);
if (ret)
return ret;
break;
@@ -946,16 +963,16 @@ static int srp_connect_target(struct srp_target_port *target)
case SRP_STALE_CONN:
shost_printk(KERN_ERR, target->scsi_host, PFX
"giving up on stale connection\n");
- target->status = -ECONNRESET;
- return target->status;
+ ch->status = -ECONNRESET;
+ return ch->status;
default:
- return target->status;
+ return ch->status;
}
}
}
-static int srp_inv_rkey(struct srp_target_port *target, u32 rkey)
+static int srp_inv_rkey(struct srp_rdma_ch *ch, u32 rkey)
{
struct ib_send_wr *bad_wr;
struct ib_send_wr wr = {
@@ -967,13 +984,14 @@ static int srp_inv_rkey(struct srp_target_port *target, u32 rkey)
.ex.invalidate_rkey = rkey,
};
- return ib_post_send(target->qp, &wr, &bad_wr);
+ return ib_post_send(ch->qp, &wr, &bad_wr);
}
static void srp_unmap_data(struct scsi_cmnd *scmnd,
- struct srp_target_port *target,
+ struct srp_rdma_ch *ch,
struct srp_request *req)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_device *ibdev = dev->dev;
int i, res;
@@ -987,7 +1005,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
struct srp_fr_desc **pfr;
for (i = req->nmdesc, pfr = req->fr_list; i > 0; i--, pfr++) {
- res = srp_inv_rkey(target, (*pfr)->mr->rkey);
+ res = srp_inv_rkey(ch, (*pfr)->mr->rkey);
if (res < 0) {
shost_printk(KERN_ERR, target->scsi_host, PFX
"Queueing INV WR for rkey %#x failed (%d)\n",
@@ -997,7 +1015,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
}
}
if (req->nmdesc)
- srp_fr_pool_put(target->fr_pool, req->fr_list,
+ srp_fr_pool_put(ch->fr_pool, req->fr_list,
req->nmdesc);
} else {
struct ib_pool_fmr **pfmr;
@@ -1012,7 +1030,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
/**
* srp_claim_req - Take ownership of the scmnd associated with a request.
- * @target: SRP target port.
+ * @ch: SRP RDMA channel.
* @req: SRP request.
* @sdev: If not NULL, only take ownership for this SCSI device.
* @scmnd: If NULL, take ownership of @req->scmnd. If not NULL, only take
@@ -1021,14 +1039,14 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
* Return value:
* Either NULL or a pointer to the SCSI command the caller became owner of.
*/
-static struct scsi_cmnd *srp_claim_req(struct srp_target_port *target,
+static struct scsi_cmnd *srp_claim_req(struct srp_rdma_ch *ch,
struct srp_request *req,
struct scsi_device *sdev,
struct scsi_cmnd *scmnd)
{
unsigned long flags;
- spin_lock_irqsave(&target->lock, flags);
+ spin_lock_irqsave(&ch->lock, flags);
if (req->scmnd &&
(!sdev || req->scmnd->device == sdev) &&
(!scmnd || req->scmnd == scmnd)) {
@@ -1037,40 +1055,38 @@ static struct scsi_cmnd *srp_claim_req(struct srp_target_port *target,
} else {
scmnd = NULL;
}
- spin_unlock_irqrestore(&target->lock, flags);
+ spin_unlock_irqrestore(&ch->lock, flags);
return scmnd;
}
/**
* srp_free_req() - Unmap data and add request to the free request list.
- * @target: SRP target port.
+ * @ch: SRP RDMA channel.
* @req: Request to be freed.
* @scmnd: SCSI command associated with @req.
* @req_lim_delta: Amount to be added to @target->req_lim.
*/
-static void srp_free_req(struct srp_target_port *target,
- struct srp_request *req, struct scsi_cmnd *scmnd,
- s32 req_lim_delta)
+static void srp_free_req(struct srp_rdma_ch *ch, struct srp_request *req,
+ struct scsi_cmnd *scmnd, s32 req_lim_delta)
{
unsigned long flags;
- srp_unmap_data(scmnd, target, req);
+ srp_unmap_data(scmnd, ch, req);
- spin_lock_irqsave(&target->lock, flags);
- target->req_lim += req_lim_delta;
- list_add_tail(&req->list, &target->free_reqs);
- spin_unlock_irqrestore(&target->lock, flags);
+ spin_lock_irqsave(&ch->lock, flags);
+ ch->req_lim += req_lim_delta;
+ list_add_tail(&req->list, &ch->free_reqs);
+ spin_unlock_irqrestore(&ch->lock, flags);
}
-static void srp_finish_req(struct srp_target_port *target,
- struct srp_request *req, struct scsi_device *sdev,
- int result)
+static void srp_finish_req(struct srp_rdma_ch *ch, struct srp_request *req,
+ struct scsi_device *sdev, int result)
{
- struct scsi_cmnd *scmnd = srp_claim_req(target, req, sdev, NULL);
+ struct scsi_cmnd *scmnd = srp_claim_req(ch, req, sdev, NULL);
if (scmnd) {
- srp_free_req(target, req, scmnd, 0);
+ srp_free_req(ch, req, scmnd, 0);
scmnd->result = result;
scmnd->scsi_done(scmnd);
}
@@ -1079,6 +1095,7 @@ static void srp_finish_req(struct srp_target_port *target,
static void srp_terminate_io(struct srp_rport *rport)
{
struct srp_target_port *target = rport->lld_data;
+ struct srp_rdma_ch *ch = &target->ch;
struct Scsi_Host *shost = target->scsi_host;
struct scsi_device *sdev;
int i;
@@ -1091,8 +1108,9 @@ static void srp_terminate_io(struct srp_rport *rport)
WARN_ON_ONCE(sdev->request_queue->request_fn_active);
for (i = 0; i < target->req_ring_size; ++i) {
- struct srp_request *req = &target->req_ring[i];
- srp_finish_req(target, req, NULL, DID_TRANSPORT_FAILFAST << 16);
+ struct srp_request *req = &ch->req_ring[i];
+
+ srp_finish_req(ch, req, NULL, DID_TRANSPORT_FAILFAST << 16);
}
}
@@ -1108,6 +1126,7 @@ static void srp_terminate_io(struct srp_rport *rport)
static int srp_rport_reconnect(struct srp_rport *rport)
{
struct srp_target_port *target = rport->lld_data;
+ struct srp_rdma_ch *ch = &target->ch;
int i, ret;
srp_disconnect_target(target);
@@ -1120,11 +1139,12 @@ static int srp_rport_reconnect(struct srp_rport *rport)
* case things are really fouled up. Doing so also ensures that all CM
* callbacks will have finished before a new QP is allocated.
*/
- ret = srp_new_cm_id(target);
+ ret = srp_new_cm_id(ch);
for (i = 0; i < target->req_ring_size; ++i) {
- struct srp_request *req = &target->req_ring[i];
- srp_finish_req(target, req, NULL, DID_RESET << 16);
+ struct srp_request *req = &ch->req_ring[i];
+
+ srp_finish_req(ch, req, NULL, DID_RESET << 16);
}
/*
@@ -1132,14 +1152,14 @@ static int srp_rport_reconnect(struct srp_rport *rport)
* QP. This guarantees that all callback functions for the old QP have
* finished before any send requests are posted on the new QP.
*/
- ret += srp_create_target_ib(target);
+ ret += srp_create_ch_ib(ch);
- INIT_LIST_HEAD(&target->free_tx);
+ INIT_LIST_HEAD(&ch->free_tx);
for (i = 0; i < target->queue_size; ++i)
- list_add(&target->tx_ring[i]->list, &target->free_tx);
+ list_add(&ch->tx_ring[i]->list, &ch->free_tx);
if (ret == 0)
- ret = srp_connect_target(target);
+ ret = srp_connect_ch(ch);
if (ret == 0)
shost_printk(KERN_INFO, target->scsi_host,
@@ -1163,12 +1183,12 @@ static void srp_map_desc(struct srp_map_state *state, dma_addr_t dma_addr,
}
static int srp_map_finish_fmr(struct srp_map_state *state,
- struct srp_target_port *target)
+ struct srp_rdma_ch *ch)
{
struct ib_pool_fmr *fmr;
u64 io_addr = 0;
- fmr = ib_fmr_pool_map_phys(target->fmr_pool, state->pages,
+ fmr = ib_fmr_pool_map_phys(ch->fmr_pool, state->pages,
state->npages, io_addr);
if (IS_ERR(fmr))
return PTR_ERR(fmr);
@@ -1182,15 +1202,16 @@ static int srp_map_finish_fmr(struct srp_map_state *state,
}
static int srp_map_finish_fr(struct srp_map_state *state,
- struct srp_target_port *target)
+ struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_send_wr *bad_wr;
struct ib_send_wr wr;
struct srp_fr_desc *desc;
u32 rkey;
- desc = srp_fr_pool_get(target->fr_pool);
+ desc = srp_fr_pool_get(ch->fr_pool);
if (!desc)
return -ENOMEM;
@@ -1219,12 +1240,13 @@ static int srp_map_finish_fr(struct srp_map_state *state,
srp_map_desc(state, state->base_dma_addr, state->dma_len,
desc->mr->rkey);
- return ib_post_send(target->qp, &wr, &bad_wr);
+ return ib_post_send(ch->qp, &wr, &bad_wr);
}
static int srp_finish_mapping(struct srp_map_state *state,
- struct srp_target_port *target)
+ struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
int ret = 0;
if (state->npages == 0)
@@ -1235,8 +1257,8 @@ static int srp_finish_mapping(struct srp_map_state *state,
target->rkey);
else
ret = target->srp_host->srp_dev->use_fast_reg ?
- srp_map_finish_fr(state, target) :
- srp_map_finish_fmr(state, target);
+ srp_map_finish_fr(state, ch) :
+ srp_map_finish_fmr(state, ch);
if (ret == 0) {
state->npages = 0;
@@ -1256,10 +1278,11 @@ static void srp_map_update_start(struct srp_map_state *state,
}
static int srp_map_sg_entry(struct srp_map_state *state,
- struct srp_target_port *target,
+ struct srp_rdma_ch *ch,
struct scatterlist *sg, int sg_index,
bool use_mr)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_device *ibdev = dev->dev;
dma_addr_t dma_addr = ib_sg_dma_address(ibdev, sg);
@@ -1288,7 +1311,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
*/
if ((!dev->use_fast_reg && dma_addr & ~dev->mr_page_mask) ||
dma_len > dev->mr_max_size) {
- ret = srp_finish_mapping(state, target);
+ ret = srp_finish_mapping(state, ch);
if (ret)
return ret;
@@ -1309,7 +1332,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
while (dma_len) {
unsigned offset = dma_addr & ~dev->mr_page_mask;
if (state->npages == dev->max_pages_per_mr || offset != 0) {
- ret = srp_finish_mapping(state, target);
+ ret = srp_finish_mapping(state, ch);
if (ret)
return ret;
@@ -1333,17 +1356,18 @@ static int srp_map_sg_entry(struct srp_map_state *state,
*/
ret = 0;
if (len != dev->mr_page_size) {
- ret = srp_finish_mapping(state, target);
+ ret = srp_finish_mapping(state, ch);
if (!ret)
srp_map_update_start(state, NULL, 0, 0);
}
return ret;
}
-static int srp_map_sg(struct srp_map_state *state,
- struct srp_target_port *target, struct srp_request *req,
- struct scatterlist *scat, int count)
+static int srp_map_sg(struct srp_map_state *state, struct srp_rdma_ch *ch,
+ struct srp_request *req, struct scatterlist *scat,
+ int count)
{
+ struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_device *ibdev = dev->dev;
struct scatterlist *sg;
@@ -1354,14 +1378,14 @@ static int srp_map_sg(struct srp_map_state *state,
state->pages = req->map_page;
if (dev->use_fast_reg) {
state->next_fr = req->fr_list;
- use_mr = !!target->fr_pool;
+ use_mr = !!ch->fr_pool;
} else {
state->next_fmr = req->fmr_list;
- use_mr = !!target->fmr_pool;
+ use_mr = !!ch->fmr_pool;
}
for_each_sg(scat, sg, count, i) {
- if (srp_map_sg_entry(state, target, sg, i, use_mr)) {
+ if (srp_map_sg_entry(state, ch, sg, i, use_mr)) {
/*
* Memory registration failed, so backtrack to the
* first unmapped entry and continue on without using
@@ -1383,7 +1407,7 @@ backtrack:
}
}
- if (use_mr && srp_finish_mapping(state, target))
+ if (use_mr && srp_finish_mapping(state, ch))
goto backtrack;
req->nmdesc = state->nmdesc;
@@ -1391,9 +1415,10 @@ backtrack:
return 0;
}
-static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port *target,
+static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
struct srp_request *req)
{
+ struct srp_target_port *target = ch->target;
struct scatterlist *scat;
struct srp_cmd *cmd = req->cmd->buf;
int len, nents, count;
@@ -1455,7 +1480,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port *target,
target->indirect_size, DMA_TO_DEVICE);
memset(&state, 0, sizeof(state));
- srp_map_sg(&state, target, req, scat, count);
+ srp_map_sg(&state, ch, req, scat, count);
/* We've mapped the request, now pull as much of the indirect
* descriptor table as we can into the command buffer. If this
@@ -1516,20 +1541,20 @@ map_complete:
/*
* Return an IU and possible credit to the free pool
*/
-static void srp_put_tx_iu(struct srp_target_port *target, struct srp_iu *iu,
+static void srp_put_tx_iu(struct srp_rdma_ch *ch, struct srp_iu *iu,
enum srp_iu_type iu_type)
{
unsigned long flags;
- spin_lock_irqsave(&target->lock, flags);
- list_add(&iu->list, &target->free_tx);
+ spin_lock_irqsave(&ch->lock, flags);
+ list_add(&iu->list, &ch->free_tx);
if (iu_type != SRP_IU_RSP)
- ++target->req_lim;
- spin_unlock_irqrestore(&target->lock, flags);
+ ++ch->req_lim;
+ spin_unlock_irqrestore(&ch->lock, flags);
}
/*
- * Must be called with target->lock held to protect req_lim and free_tx.
+ * Must be called with ch->lock held to protect req_lim and free_tx.
* If IU is not sent, it must be returned using srp_put_tx_iu().
*
* Note:
@@ -1541,35 +1566,36 @@ static void srp_put_tx_iu(struct srp_target_port *target, struct srp_iu *iu,
* - SRP_IU_RSP: 1, since a conforming SRP target never sends more than
* one unanswered SRP request to an initiator.
*/
-static struct srp_iu *__srp_get_tx_iu(struct srp_target_port *target,
+static struct srp_iu *__srp_get_tx_iu(struct srp_rdma_ch *ch,
enum srp_iu_type iu_type)
{
+ struct srp_target_port *target = ch->target;
s32 rsv = (iu_type == SRP_IU_TSK_MGMT) ? 0 : SRP_TSK_MGMT_SQ_SIZE;
struct srp_iu *iu;
- srp_send_completion(target->send_cq, target);
+ srp_send_completion(ch->send_cq, target);
- if (list_empty(&target->free_tx))
+ if (list_empty(&ch->free_tx))
return NULL;
/* Initiator responses to target requests do not consume credits */
if (iu_type != SRP_IU_RSP) {
- if (target->req_lim <= rsv) {
+ if (ch->req_lim <= rsv) {
++target->zero_req_lim;
return NULL;
}
- --target->req_lim;
+ --ch->req_lim;
}
- iu = list_first_entry(&target->free_tx, struct srp_iu, list);
+ iu = list_first_entry(&ch->free_tx, struct srp_iu, list);
list_del(&iu->list);
return iu;
}
-static int srp_post_send(struct srp_target_port *target,
- struct srp_iu *iu, int len)
+static int srp_post_send(struct srp_rdma_ch *ch, struct srp_iu *iu, int len)
{
+ struct srp_target_port *target = ch->target;
struct ib_sge list;
struct ib_send_wr wr, *bad_wr;
@@ -1584,11 +1610,12 @@ static int srp_post_send(struct srp_target_port *target,
wr.opcode = IB_WR_SEND;
wr.send_flags = IB_SEND_SIGNALED;
- return ib_post_send(target->qp, &wr, &bad_wr);
+ return ib_post_send(ch->qp, &wr, &bad_wr);
}
-static int srp_post_recv(struct srp_target_port *target, struct srp_iu *iu)
+static int srp_post_recv(struct srp_rdma_ch *ch, struct srp_iu *iu)
{
+ struct srp_target_port *target = ch->target;
struct ib_recv_wr wr, *bad_wr;
struct ib_sge list;
@@ -1601,35 +1628,36 @@ static int srp_post_recv(struct srp_target_port *target, struct srp_iu *iu)
wr.sg_list = &list;
wr.num_sge = 1;
- return ib_post_recv(target->qp, &wr, &bad_wr);
+ return ib_post_recv(ch->qp, &wr, &bad_wr);
}
-static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
+static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
{
+ struct srp_target_port *target = ch->target;
struct srp_request *req;
struct scsi_cmnd *scmnd;
unsigned long flags;
if (unlikely(rsp->tag & SRP_TAG_TSK_MGMT)) {
- spin_lock_irqsave(&target->lock, flags);
- target->req_lim += be32_to_cpu(rsp->req_lim_delta);
- spin_unlock_irqrestore(&target->lock, flags);
+ spin_lock_irqsave(&ch->lock, flags);
+ ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
+ spin_unlock_irqrestore(&ch->lock, flags);
- target->tsk_mgmt_status = -1;
+ ch->tsk_mgmt_status = -1;
if (be32_to_cpu(rsp->resp_data_len) >= 4)
- target->tsk_mgmt_status = rsp->data[3];
- complete(&target->tsk_mgmt_done);
+ ch->tsk_mgmt_status = rsp->data[3];
+ complete(&ch->tsk_mgmt_done);
} else {
- req = &target->req_ring[rsp->tag];
- scmnd = srp_claim_req(target, req, NULL, NULL);
+ req = &ch->req_ring[rsp->tag];
+ scmnd = srp_claim_req(ch, req, NULL, NULL);
if (!scmnd) {
shost_printk(KERN_ERR, target->scsi_host,
"Null scmnd for RSP w/tag %016llx\n",
(unsigned long long) rsp->tag);
- spin_lock_irqsave(&target->lock, flags);
- target->req_lim += be32_to_cpu(rsp->req_lim_delta);
- spin_unlock_irqrestore(&target->lock, flags);
+ spin_lock_irqsave(&ch->lock, flags);
+ ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
+ spin_unlock_irqrestore(&ch->lock, flags);
return;
}
@@ -1651,7 +1679,7 @@ static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
else if (unlikely(rsp->flags & SRP_RSP_FLAG_DOOVER))
scsi_set_resid(scmnd, -be32_to_cpu(rsp->data_out_res_cnt));
- srp_free_req(target, req, scmnd,
+ srp_free_req(ch, req, scmnd,
be32_to_cpu(rsp->req_lim_delta));
scmnd->host_scribble = NULL;
@@ -1659,18 +1687,19 @@ static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
}
}
-static int srp_response_common(struct srp_target_port *target, s32 req_delta,
+static int srp_response_common(struct srp_rdma_ch *ch, s32 req_delta,
void *rsp, int len)
{
+ struct srp_target_port *target = ch->target;
struct ib_device *dev = target->srp_host->srp_dev->dev;
unsigned long flags;
struct srp_iu *iu;
int err;
- spin_lock_irqsave(&target->lock, flags);
- target->req_lim += req_delta;
- iu = __srp_get_tx_iu(target, SRP_IU_RSP);
- spin_unlock_irqrestore(&target->lock, flags);
+ spin_lock_irqsave(&ch->lock, flags);
+ ch->req_lim += req_delta;
+ iu = __srp_get_tx_iu(ch, SRP_IU_RSP);
+ spin_unlock_irqrestore(&ch->lock, flags);
if (!iu) {
shost_printk(KERN_ERR, target->scsi_host, PFX
@@ -1682,17 +1711,17 @@ static int srp_response_common(struct srp_target_port *target, s32 req_delta,
memcpy(iu->buf, rsp, len);
ib_dma_sync_single_for_device(dev, iu->dma, len, DMA_TO_DEVICE);
- err = srp_post_send(target, iu, len);
+ err = srp_post_send(ch, iu, len);
if (err) {
shost_printk(KERN_ERR, target->scsi_host, PFX
"unable to post response: %d\n", err);
- srp_put_tx_iu(target, iu, SRP_IU_RSP);
+ srp_put_tx_iu(ch, iu, SRP_IU_RSP);
}
return err;
}
-static void srp_process_cred_req(struct srp_target_port *target,
+static void srp_process_cred_req(struct srp_rdma_ch *ch,
struct srp_cred_req *req)
{
struct srp_cred_rsp rsp = {
@@ -1701,14 +1730,15 @@ static void srp_process_cred_req(struct srp_target_port *target,
};
s32 delta = be32_to_cpu(req->req_lim_delta);
- if (srp_response_common(target, delta, &rsp, sizeof rsp))
- shost_printk(KERN_ERR, target->scsi_host, PFX
+ if (srp_response_common(ch, delta, &rsp, sizeof(rsp)))
+ shost_printk(KERN_ERR, ch->target->scsi_host, PFX
"problems processing SRP_CRED_REQ\n");
}
-static void srp_process_aer_req(struct srp_target_port *target,
+static void srp_process_aer_req(struct srp_rdma_ch *ch,
struct srp_aer_req *req)
{
+ struct srp_target_port *target = ch->target;
struct srp_aer_rsp rsp = {
.opcode = SRP_AER_RSP,
.tag = req->tag,
@@ -1718,19 +1748,20 @@ static void srp_process_aer_req(struct srp_target_port *target,
shost_printk(KERN_ERR, target->scsi_host, PFX
"ignoring AER for LUN %llu\n", be64_to_cpu(req->lun));
- if (srp_response_common(target, delta, &rsp, sizeof rsp))
+ if (srp_response_common(ch, delta, &rsp, sizeof(rsp)))
shost_printk(KERN_ERR, target->scsi_host, PFX
"problems processing SRP_AER_REQ\n");
}
-static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
+static void srp_handle_recv(struct srp_rdma_ch *ch, struct ib_wc *wc)
{
+ struct srp_target_port *target = ch->target;
struct ib_device *dev = target->srp_host->srp_dev->dev;
struct srp_iu *iu = (struct srp_iu *) (uintptr_t) wc->wr_id;
int res;
u8 opcode;
- ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_ti_iu_len,
+ ib_dma_sync_single_for_cpu(dev, iu->dma, ch->max_ti_iu_len,
DMA_FROM_DEVICE);
opcode = *(u8 *) iu->buf;
@@ -1744,15 +1775,15 @@ static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
switch (opcode) {
case SRP_RSP:
- srp_process_rsp(target, iu->buf);
+ srp_process_rsp(ch, iu->buf);
break;
case SRP_CRED_REQ:
- srp_process_cred_req(target, iu->buf);
+ srp_process_cred_req(ch, iu->buf);
break;
case SRP_AER_REQ:
- srp_process_aer_req(target, iu->buf);
+ srp_process_aer_req(ch, iu->buf);
break;
case SRP_T_LOGOUT:
@@ -1767,10 +1798,10 @@ static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
break;
}
- ib_dma_sync_single_for_device(dev, iu->dma, target->max_ti_iu_len,
+ ib_dma_sync_single_for_device(dev, iu->dma, ch->max_ti_iu_len,
DMA_FROM_DEVICE);
- res = srp_post_recv(target, iu);
+ res = srp_post_recv(ch, iu);
if (res != 0)
shost_printk(KERN_ERR, target->scsi_host,
PFX "Recv failed with error code %d\n", res);
@@ -1815,33 +1846,35 @@ static void srp_handle_qp_err(u64 wr_id, enum ib_wc_status wc_status,
target->qp_in_error = true;
}
-static void srp_recv_completion(struct ib_cq *cq, void *target_ptr)
+static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr)
{
- struct srp_target_port *target = target_ptr;
+ struct srp_rdma_ch *ch = ch_ptr;
struct ib_wc wc;
ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
while (ib_poll_cq(cq, 1, &wc) > 0) {
if (likely(wc.status == IB_WC_SUCCESS)) {
- srp_handle_recv(target, &wc);
+ srp_handle_recv(ch, &wc);
} else {
- srp_handle_qp_err(wc.wr_id, wc.status, false, target);
+ srp_handle_qp_err(wc.wr_id, wc.status, false,
+ ch->target);
}
}
}
-static void srp_send_completion(struct ib_cq *cq, void *target_ptr)
+static void srp_send_completion(struct ib_cq *cq, void *ch_ptr)
{
- struct srp_target_port *target = target_ptr;
+ struct srp_rdma_ch *ch = ch_ptr;
struct ib_wc wc;
struct srp_iu *iu;
while (ib_poll_cq(cq, 1, &wc) > 0) {
if (likely(wc.status == IB_WC_SUCCESS)) {
iu = (struct srp_iu *) (uintptr_t) wc.wr_id;
- list_add(&iu->list, &target->free_tx);
+ list_add(&iu->list, &ch->free_tx);
} else {
- srp_handle_qp_err(wc.wr_id, wc.status, true, target);
+ srp_handle_qp_err(wc.wr_id, wc.status, true,
+ ch->target);
}
}
}
@@ -1850,6 +1883,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
{
struct srp_target_port *target = host_to_target(shost);
struct srp_rport *rport = target->rport;
+ struct srp_rdma_ch *ch;
struct srp_request *req;
struct srp_iu *iu;
struct srp_cmd *cmd;
@@ -1871,14 +1905,16 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
if (unlikely(scmnd->result))
goto err;
- spin_lock_irqsave(&target->lock, flags);
- iu = __srp_get_tx_iu(target, SRP_IU_CMD);
+ ch = &target->ch;
+
+ spin_lock_irqsave(&ch->lock, flags);
+ iu = __srp_get_tx_iu(ch, SRP_IU_CMD);
if (!iu)
goto err_unlock;
- req = list_first_entry(&target->free_reqs, struct srp_request, list);
+ req = list_first_entry(&ch->free_reqs, struct srp_request, list);
list_del(&req->list);
- spin_unlock_irqrestore(&target->lock, flags);
+ spin_unlock_irqrestore(&ch->lock, flags);
dev = target->srp_host->srp_dev->dev;
ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_iu_len,
@@ -1897,7 +1933,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
req->scmnd = scmnd;
req->cmd = iu;
- len = srp_map_data(scmnd, target, req);
+ len = srp_map_data(scmnd, ch, req);
if (len < 0) {
shost_printk(KERN_ERR, target->scsi_host,
PFX "Failed to map data (%d)\n", len);
@@ -1915,7 +1951,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
ib_dma_sync_single_for_device(dev, iu->dma, target->max_iu_len,
DMA_TO_DEVICE);
- if (srp_post_send(target, iu, len)) {
+ if (srp_post_send(ch, iu, len)) {
shost_printk(KERN_ERR, target->scsi_host, PFX "Send failed\n");
goto err_unmap;
}
@@ -1929,10 +1965,10 @@ unlock_rport:
return ret;
err_unmap:
- srp_unmap_data(scmnd, target, req);
+ srp_unmap_data(scmnd, ch, req);
err_iu:
- srp_put_tx_iu(target, iu, SRP_IU_CMD);
+ srp_put_tx_iu(ch, iu, SRP_IU_CMD);
/*
* Avoid that the loops that iterate over the request ring can
@@ -1940,11 +1976,11 @@ err_iu:
*/
req->scmnd = NULL;
- spin_lock_irqsave(&target->lock, flags);
- list_add(&req->list, &target->free_reqs);
+ spin_lock_irqsave(&ch->lock, flags);
+ list_add(&req->list, &ch->free_reqs);
err_unlock:
- spin_unlock_irqrestore(&target->lock, flags);
+ spin_unlock_irqrestore(&ch->lock, flags);
err:
if (scmnd->result) {
@@ -1959,53 +1995,54 @@ err:
/*
* Note: the resources allocated in this function are freed in
- * srp_free_target_ib().
+ * srp_free_ch_ib().
*/
-static int srp_alloc_iu_bufs(struct srp_target_port *target)
+static int srp_alloc_iu_bufs(struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
int i;
- target->rx_ring = kzalloc(target->queue_size * sizeof(*target->rx_ring),
- GFP_KERNEL);
- if (!target->rx_ring)
+ ch->rx_ring = kcalloc(target->queue_size, sizeof(*ch->rx_ring),
+ GFP_KERNEL);
+ if (!ch->rx_ring)
goto err_no_ring;
- target->tx_ring = kzalloc(target->queue_size * sizeof(*target->tx_ring),
- GFP_KERNEL);
- if (!target->tx_ring)
+ ch->tx_ring = kcalloc(target->queue_size, sizeof(*ch->tx_ring),
+ GFP_KERNEL);
+ if (!ch->tx_ring)
goto err_no_ring;
for (i = 0; i < target->queue_size; ++i) {
- target->rx_ring[i] = srp_alloc_iu(target->srp_host,
- target->max_ti_iu_len,
- GFP_KERNEL, DMA_FROM_DEVICE);
- if (!target->rx_ring[i])
+ ch->rx_ring[i] = srp_alloc_iu(target->srp_host,
+ ch->max_ti_iu_len,
+ GFP_KERNEL, DMA_FROM_DEVICE);
+ if (!ch->rx_ring[i])
goto err;
}
for (i = 0; i < target->queue_size; ++i) {
- target->tx_ring[i] = srp_alloc_iu(target->srp_host,
- target->max_iu_len,
- GFP_KERNEL, DMA_TO_DEVICE);
- if (!target->tx_ring[i])
+ ch->tx_ring[i] = srp_alloc_iu(target->srp_host,
+ target->max_iu_len,
+ GFP_KERNEL, DMA_TO_DEVICE);
+ if (!ch->tx_ring[i])
goto err;
- list_add(&target->tx_ring[i]->list, &target->free_tx);
+ list_add(&ch->tx_ring[i]->list, &ch->free_tx);
}
return 0;
err:
for (i = 0; i < target->queue_size; ++i) {
- srp_free_iu(target->srp_host, target->rx_ring[i]);
- srp_free_iu(target->srp_host, target->tx_ring[i]);
+ srp_free_iu(target->srp_host, ch->rx_ring[i]);
+ srp_free_iu(target->srp_host, ch->tx_ring[i]);
}
err_no_ring:
- kfree(target->tx_ring);
- target->tx_ring = NULL;
- kfree(target->rx_ring);
- target->rx_ring = NULL;
+ kfree(ch->tx_ring);
+ ch->tx_ring = NULL;
+ kfree(ch->rx_ring);
+ ch->rx_ring = NULL;
return -ENOMEM;
}
@@ -2039,23 +2076,24 @@ static uint32_t srp_compute_rq_tmo(struct ib_qp_attr *qp_attr, int attr_mask)
static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
struct srp_login_rsp *lrsp,
- struct srp_target_port *target)
+ struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct ib_qp_attr *qp_attr = NULL;
int attr_mask = 0;
int ret;
int i;
if (lrsp->opcode == SRP_LOGIN_RSP) {
- target->max_ti_iu_len = be32_to_cpu(lrsp->max_ti_iu_len);
- target->req_lim = be32_to_cpu(lrsp->req_lim_delta);
+ ch->max_ti_iu_len = be32_to_cpu(lrsp->max_ti_iu_len);
+ ch->req_lim = be32_to_cpu(lrsp->req_lim_delta);
/*
* Reserve credits for task management so we don't
* bounce requests back to the SCSI mid-layer.
*/
target->scsi_host->can_queue
- = min(target->req_lim - SRP_TSK_MGMT_SQ_SIZE,
+ = min(ch->req_lim - SRP_TSK_MGMT_SQ_SIZE,
target->scsi_host->can_queue);
target->scsi_host->cmd_per_lun
= min_t(int, target->scsi_host->can_queue,
@@ -2067,8 +2105,8 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
goto error;
}
- if (!target->rx_ring) {
- ret = srp_alloc_iu_bufs(target);
+ if (!ch->rx_ring) {
+ ret = srp_alloc_iu_bufs(ch);
if (ret)
goto error;
}
@@ -2083,13 +2121,14 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
if (ret)
goto error_free;
- ret = ib_modify_qp(target->qp, qp_attr, attr_mask);
+ ret = ib_modify_qp(ch->qp, qp_attr, attr_mask);
if (ret)
goto error_free;
for (i = 0; i < target->queue_size; i++) {
- struct srp_iu *iu = target->rx_ring[i];
- ret = srp_post_recv(target, iu);
+ struct srp_iu *iu = ch->rx_ring[i];
+
+ ret = srp_post_recv(ch, iu);
if (ret)
goto error_free;
}
@@ -2101,7 +2140,7 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
target->rq_tmo_jiffies = srp_compute_rq_tmo(qp_attr, attr_mask);
- ret = ib_modify_qp(target->qp, qp_attr, attr_mask);
+ ret = ib_modify_qp(ch->qp, qp_attr, attr_mask);
if (ret)
goto error_free;
@@ -2111,13 +2150,14 @@ error_free:
kfree(qp_attr);
error:
- target->status = ret;
+ ch->status = ret;
}
static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
struct ib_cm_event *event,
- struct srp_target_port *target)
+ struct srp_rdma_ch *ch)
{
+ struct srp_target_port *target = ch->target;
struct Scsi_Host *shost = target->scsi_host;
struct ib_class_port_info *cpi;
int opcode;
@@ -2125,12 +2165,12 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
switch (event->param.rej_rcvd.reason) {
case IB_CM_REJ_PORT_CM_REDIRECT:
cpi = event->param.rej_rcvd.ari;
- target->path.dlid = cpi->redirect_lid;
- target->path.pkey = cpi->redirect_pkey;
+ ch->path.dlid = cpi->redirect_lid;
+ ch->path.pkey = cpi->redirect_pkey;
cm_id->remote_cm_qpn = be32_to_cpu(cpi->redirect_qp) & 0x00ffffff;
- memcpy(target->path.dgid.raw, cpi->redirect_gid, 16);
+ memcpy(ch->path.dgid.raw, cpi->redirect_gid, 16);
- target->status = target->path.dlid ?
+ ch->status = ch->path.dlid ?
SRP_DLID_REDIRECT : SRP_PORT_REDIRECT;
break;
@@ -2141,26 +2181,26 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
* reject reason code 25 when they mean 24
* (port redirect).
*/
- memcpy(target->path.dgid.raw,
+ memcpy(ch->path.dgid.raw,
event->param.rej_rcvd.ari, 16);
shost_printk(KERN_DEBUG, shost,
PFX "Topspin/Cisco redirect to target port GID %016llx%016llx\n",
- (unsigned long long) be64_to_cpu(target->path.dgid.global.subnet_prefix),
- (unsigned long long) be64_to_cpu(target->path.dgid.global.interface_id));
+ be64_to_cpu(ch->path.dgid.global.subnet_prefix),
+ be64_to_cpu(ch->path.dgid.global.interface_id));
- target->status = SRP_PORT_REDIRECT;
+ ch->status = SRP_PORT_REDIRECT;
} else {
shost_printk(KERN_WARNING, shost,
" REJ reason: IB_CM_REJ_PORT_REDIRECT\n");
- target->status = -ECONNRESET;
+ ch->status = -ECONNRESET;
}
break;
case IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID:
shost_printk(KERN_WARNING, shost,
" REJ reason: IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID\n");
- target->status = -ECONNRESET;
+ ch->status = -ECONNRESET;
break;
case IB_CM_REJ_CONSUMER_DEFINED:
@@ -2175,30 +2215,31 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
else
shost_printk(KERN_WARNING, shost, PFX
"SRP LOGIN from %pI6 to %pI6 REJECTED, reason 0x%08x\n",
- target->path.sgid.raw,
- target->orig_dgid, reason);
+ target->sgid.raw,
+ target->orig_dgid.raw, reason);
} else
shost_printk(KERN_WARNING, shost,
" REJ reason: IB_CM_REJ_CONSUMER_DEFINED,"
" opcode 0x%02x\n", opcode);
- target->status = -ECONNRESET;
+ ch->status = -ECONNRESET;
break;
case IB_CM_REJ_STALE_CONN:
shost_printk(KERN_WARNING, shost, " REJ reason: stale connection\n");
- target->status = SRP_STALE_CONN;
+ ch->status = SRP_STALE_CONN;
break;
default:
shost_printk(KERN_WARNING, shost, " REJ reason 0x%x\n",
event->param.rej_rcvd.reason);
- target->status = -ECONNRESET;
+ ch->status = -ECONNRESET;
}
}
static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
{
- struct srp_target_port *target = cm_id->context;
+ struct srp_rdma_ch *ch = cm_id->context;
+ struct srp_target_port *target = ch->target;
int comp = 0;
switch (event->event) {
@@ -2206,19 +2247,19 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
shost_printk(KERN_DEBUG, target->scsi_host,
PFX "Sending CM REQ failed\n");
comp = 1;
- target->status = -ECONNRESET;
+ ch->status = -ECONNRESET;
break;
case IB_CM_REP_RECEIVED:
comp = 1;
- srp_cm_rep_handler(cm_id, event->private_data, target);
+ srp_cm_rep_handler(cm_id, event->private_data, ch);
break;
case IB_CM_REJ_RECEIVED:
shost_printk(KERN_DEBUG, target->scsi_host, PFX "REJ received\n");
comp = 1;
- srp_cm_rej_handler(cm_id, event, target);
+ srp_cm_rej_handler(cm_id, event, ch);
break;
case IB_CM_DREQ_RECEIVED:
@@ -2236,7 +2277,7 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
PFX "connection closed\n");
comp = 1;
- target->status = 0;
+ ch->status = 0;
break;
case IB_CM_MRA_RECEIVED:
@@ -2251,7 +2292,7 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
}
if (comp)
- complete(&target->done);
+ complete(&ch->done);
return 0;
}
@@ -2307,9 +2348,10 @@ srp_change_queue_depth(struct scsi_device *sdev, int qdepth, int reason)
return sdev->queue_depth;
}
-static int srp_send_tsk_mgmt(struct srp_target_port *target,
- u64 req_tag, unsigned int lun, u8 func)
+static int srp_send_tsk_mgmt(struct srp_rdma_ch *ch, u64 req_tag,
+ unsigned int lun, u8 func)
{
+ struct srp_target_port *target = ch->target;
struct srp_rport *rport = target->rport;
struct ib_device *dev = target->srp_host->srp_dev->dev;
struct srp_iu *iu;
@@ -2318,16 +2360,16 @@ static int srp_send_tsk_mgmt(struct srp_target_port *target,
if (!target->connected || target->qp_in_error)
return -1;
- init_completion(&target->tsk_mgmt_done);
+ init_completion(&ch->tsk_mgmt_done);
/*
- * Lock the rport mutex to avoid that srp_create_target_ib() is
+ * Lock the rport mutex to avoid that srp_create_ch_ib() is
* invoked while a task management function is being sent.
*/
mutex_lock(&rport->mutex);
- spin_lock_irq(&target->lock);
- iu = __srp_get_tx_iu(target, SRP_IU_TSK_MGMT);
- spin_unlock_irq(&target->lock);
+ spin_lock_irq(&ch->lock);
+ iu = __srp_get_tx_iu(ch, SRP_IU_TSK_MGMT);
+ spin_unlock_irq(&ch->lock);
if (!iu) {
mutex_unlock(&rport->mutex);
@@ -2348,15 +2390,15 @@ static int srp_send_tsk_mgmt(struct srp_target_port *target,
ib_dma_sync_single_for_device(dev, iu->dma, sizeof *tsk_mgmt,
DMA_TO_DEVICE);
- if (srp_post_send(target, iu, sizeof *tsk_mgmt)) {
- srp_put_tx_iu(target, iu, SRP_IU_TSK_MGMT);
+ if (srp_post_send(ch, iu, sizeof(*tsk_mgmt))) {
+ srp_put_tx_iu(ch, iu, SRP_IU_TSK_MGMT);
mutex_unlock(&rport->mutex);
return -1;
}
mutex_unlock(&rport->mutex);
- if (!wait_for_completion_timeout(&target->tsk_mgmt_done,
+ if (!wait_for_completion_timeout(&ch->tsk_mgmt_done,
msecs_to_jiffies(SRP_ABORT_TIMEOUT_MS)))
return -1;
@@ -2367,20 +2409,22 @@ static int srp_abort(struct scsi_cmnd *scmnd)
{
struct srp_target_port *target = host_to_target(scmnd->device->host);
struct srp_request *req = (struct srp_request *) scmnd->host_scribble;
+ struct srp_rdma_ch *ch;
int ret;
shost_printk(KERN_ERR, target->scsi_host, "SRP abort called\n");
- if (!req || !srp_claim_req(target, req, NULL, scmnd))
+ ch = &target->ch;
+ if (!req || !srp_claim_req(ch, req, NULL, scmnd))
return SUCCESS;
- if (srp_send_tsk_mgmt(target, req->index, scmnd->device->lun,
+ if (srp_send_tsk_mgmt(ch, req->index, scmnd->device->lun,
SRP_TSK_ABORT_TASK) == 0)
ret = SUCCESS;
else if (target->rport->state == SRP_RPORT_LOST)
ret = FAST_IO_FAIL;
else
ret = FAILED;
- srp_free_req(target, req, scmnd, 0);
+ srp_free_req(ch, req, scmnd, 0);
scmnd->result = DID_ABORT << 16;
scmnd->scsi_done(scmnd);
@@ -2390,19 +2434,21 @@ static int srp_abort(struct scsi_cmnd *scmnd)
static int srp_reset_device(struct scsi_cmnd *scmnd)
{
struct srp_target_port *target = host_to_target(scmnd->device->host);
+ struct srp_rdma_ch *ch = &target->ch;
int i;
shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n");
- if (srp_send_tsk_mgmt(target, SRP_TAG_NO_REQ, scmnd->device->lun,
+ if (srp_send_tsk_mgmt(ch, SRP_TAG_NO_REQ, scmnd->device->lun,
SRP_TSK_LUN_RESET))
return FAILED;
- if (target->tsk_mgmt_status)
+ if (ch->tsk_mgmt_status)
return FAILED;
for (i = 0; i < target->req_ring_size; ++i) {
- struct srp_request *req = &target->req_ring[i];
- srp_finish_req(target, req, scmnd->device, DID_RESET << 16);
+ struct srp_request *req = &ch->req_ring[i];
+
+ srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
}
return SUCCESS;
@@ -2464,7 +2510,7 @@ static ssize_t show_pkey(struct device *dev, struct device_attribute *attr,
{
struct srp_target_port *target = host_to_target(class_to_shost(dev));
- return sprintf(buf, "0x%04x\n", be16_to_cpu(target->path.pkey));
+ return sprintf(buf, "0x%04x\n", be16_to_cpu(target->pkey));
}
static ssize_t show_sgid(struct device *dev, struct device_attribute *attr,
@@ -2472,15 +2518,16 @@ static ssize_t show_sgid(struct device *dev, struct device_attribute *attr,
{
struct srp_target_port *target = host_to_target(class_to_shost(dev));
- return sprintf(buf, "%pI6\n", target->path.sgid.raw);
+ return sprintf(buf, "%pI6\n", target->sgid.raw);
}
static ssize_t show_dgid(struct device *dev, struct device_attribute *attr,
char *buf)
{
struct srp_target_port *target = host_to_target(class_to_shost(dev));
+ struct srp_rdma_ch *ch = &target->ch;
- return sprintf(buf, "%pI6\n", target->path.dgid.raw);
+ return sprintf(buf, "%pI6\n", ch->path.dgid.raw);
}
static ssize_t show_orig_dgid(struct device *dev,
@@ -2488,7 +2535,7 @@ static ssize_t show_orig_dgid(struct device *dev,
{
struct srp_target_port *target = host_to_target(class_to_shost(dev));
- return sprintf(buf, "%pI6\n", target->orig_dgid);
+ return sprintf(buf, "%pI6\n", target->orig_dgid.raw);
}
static ssize_t show_req_lim(struct device *dev,
@@ -2496,7 +2543,7 @@ static ssize_t show_req_lim(struct device *dev,
{
struct srp_target_port *target = host_to_target(class_to_shost(dev));
- return sprintf(buf, "%d\n", target->req_lim);
+ return sprintf(buf, "%d\n", target->ch.req_lim);
}
static ssize_t show_zero_req_lim(struct device *dev,
@@ -2778,7 +2825,7 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
int opt_mask = 0;
int token;
int ret = -EINVAL;
- int i;
+ int i, b;
options = kstrdup(buf, GFP_KERNEL);
if (!options)
@@ -2826,11 +2873,15 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
}
for (i = 0; i < 16; ++i) {
- strlcpy(dgid, p + i * 2, 3);
- target->path.dgid.raw[i] = simple_strtoul(dgid, NULL, 16);
+ strlcpy(dgid, p + i * 2, sizeof(dgid));
+ if (sscanf(dgid, "%x", &b) < 1) {
+ ret = -EINVAL;
+ kfree(p);
+ goto out;
+ }
+ target->orig_dgid.raw[i] = b;
}
kfree(p);
- memcpy(target->orig_dgid, target->path.dgid.raw, 16);
break;
case SRP_OPT_PKEY:
@@ -2838,7 +2889,7 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
pr_warn("bad P_Key parameter '%s'\n", p);
goto out;
}
- target->path.pkey = cpu_to_be16(token);
+ target->pkey = cpu_to_be16(token);
break;
case SRP_OPT_SERVICE_ID:
@@ -2848,7 +2899,6 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
goto out;
}
target->service_id = cpu_to_be64(simple_strtoull(p, NULL, 16));
- target->path.service_id = target->service_id;
kfree(p);
break;
@@ -2985,6 +3035,7 @@ static ssize_t srp_create_target(struct device *dev,
container_of(dev, struct srp_host, dev);
struct Scsi_Host *target_host;
struct srp_target_port *target;
+ struct srp_rdma_ch *ch;
struct srp_device *srp_dev = host->srp_dev;
struct ib_device *ibdev = srp_dev->dev;
int ret;
@@ -3047,24 +3098,28 @@ static ssize_t srp_create_target(struct device *dev,
INIT_WORK(&target->tl_err_work, srp_tl_err_work);
INIT_WORK(&target->remove_work, srp_remove_work);
spin_lock_init(&target->lock);
- INIT_LIST_HEAD(&target->free_tx);
- ret = srp_alloc_req_data(target);
+ ch = &target->ch;
+ ch->target = target;
+ ch->comp_vector = target->comp_vector;
+ spin_lock_init(&ch->lock);
+ INIT_LIST_HEAD(&ch->free_tx);
+ ret = srp_alloc_req_data(ch);
if (ret)
goto err_free_mem;
- ret = ib_query_gid(ibdev, host->port, 0, &target->path.sgid);
+ ret = ib_query_gid(ibdev, host->port, 0, &target->sgid);
if (ret)
goto err_free_mem;
- ret = srp_create_target_ib(target);
+ ret = srp_create_ch_ib(ch);
if (ret)
goto err_free_mem;
- ret = srp_new_cm_id(target);
+ ret = srp_new_cm_id(ch);
if (ret)
goto err_free_ib;
- ret = srp_connect_target(target);
+ ret = srp_connect_ch(ch);
if (ret) {
shost_printk(KERN_ERR, target->scsi_host,
PFX "Connection failed\n");
@@ -3083,9 +3138,9 @@ static ssize_t srp_create_target(struct device *dev,
"new target: id_ext %016llx ioc_guid %016llx pkey %04x service_id %016llx sgid %pI6 dgid %pI6\n",
be64_to_cpu(target->id_ext),
be64_to_cpu(target->ioc_guid),
- be16_to_cpu(target->path.pkey),
+ be16_to_cpu(target->pkey),
be64_to_cpu(target->service_id),
- target->path.sgid.raw, target->orig_dgid);
+ target->sgid.raw, target->orig_dgid.raw);
}
scsi_host_put(target->scsi_host);
@@ -3100,10 +3155,10 @@ err_disconnect:
srp_disconnect_target(target);
err_free_ib:
- srp_free_target_ib(target);
+ srp_free_ch_ib(ch);
err_free_mem:
- srp_free_req_data(target);
+ srp_free_req_data(ch);
err:
scsi_host_put(target_host);
diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
index 00c7c48..0609124 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.h
+++ b/drivers/infiniband/ulp/srp/ib_srp.h
@@ -130,7 +130,7 @@ struct srp_request {
short index;
};
-struct srp_target_port {
+struct srp_rdma_ch {
/* These are RW in the hot path, and commonly used together */
struct list_head free_tx;
struct list_head free_reqs;
@@ -138,13 +138,43 @@ struct srp_target_port {
s32 req_lim;
/* These are read-only in the hot path */
- struct ib_cq *send_cq ____cacheline_aligned_in_smp;
+ struct srp_target_port *target ____cacheline_aligned_in_smp;
+ struct ib_cq *send_cq;
struct ib_cq *recv_cq;
struct ib_qp *qp;
union {
struct ib_fmr_pool *fmr_pool;
struct srp_fr_pool *fr_pool;
};
+
+ /* Everything above this point is used in the hot path of
+ * command processing. Try to keep them packed into cachelines.
+ */
+
+ struct completion done;
+ int status;
+
+ struct ib_sa_path_rec path;
+ struct ib_sa_query *path_query;
+ int path_query_id;
+
+ struct ib_cm_id *cm_id;
+ struct srp_iu **tx_ring;
+ struct srp_iu **rx_ring;
+ struct srp_request *req_ring;
+ int max_ti_iu_len;
+ int comp_vector;
+
+ struct completion tsk_mgmt_done;
+ u8 tsk_mgmt_status;
+};
+
+struct srp_target_port {
+ /* read and written in the hot path */
+ spinlock_t lock;
+
+ struct srp_rdma_ch ch;
+ /* read only in the hot path */
u32 lkey;
u32 rkey;
enum srp_target_state state;
@@ -153,10 +183,8 @@ struct srp_target_port {
unsigned int indirect_size;
bool allow_ext_sg;
- /* Everything above this point is used in the hot path of
- * command processing. Try to keep them packed into cachelines.
- */
-
+ /* other member variables */
+ union ib_gid sgid;
__be64 id_ext;
__be64 ioc_guid;
__be64 service_id;
@@ -173,34 +201,19 @@ struct srp_target_port {
int comp_vector;
int tl_retry_count;
- struct ib_sa_path_rec path;
- __be16 orig_dgid[8];
- struct ib_sa_query *path_query;
- int path_query_id;
+ union ib_gid orig_dgid;
+ __be16 pkey;
u32 rq_tmo_jiffies;
bool connected;
- struct ib_cm_id *cm_id;
-
- int max_ti_iu_len;
-
int zero_req_lim;
- struct srp_iu **tx_ring;
- struct srp_iu **rx_ring;
- struct srp_request *req_ring;
-
struct work_struct tl_err_work;
struct work_struct remove_work;
struct list_head list;
- struct completion done;
- int status;
bool qp_in_error;
-
- struct completion tsk_mgmt_done;
- u8 tsk_mgmt_status;
};
struct srp_iu {
--
1.8.4.5
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
2014-09-19 12:58 ` [PATCH 4/8] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib() Bart Van Assche
2014-09-19 12:58 ` [PATCH 5/8] IB/srp: Remove stale connection retry mechanism Bart Van Assche
@ 2014-09-19 13:00 ` Bart Van Assche
[not found] ` <541C28E0.7010705-HInyCGIudOg@public.gmane.org>
2014-09-23 16:32 ` Sagi Grimberg
2014-10-01 16:14 ` [PATCH RFC] scsi_tcq.h: Add support for multiple hardware queues Bart Van Assche
3 siblings, 2 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 13:00 UTC (permalink / raw)
To: linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
Improve performance by using multiple RDMA/RC channels per SCSI host
for communicating with an SRP target.
Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
---
Documentation/ABI/stable/sysfs-driver-ib_srp | 25 +-
drivers/infiniband/ulp/srp/ib_srp.c | 337 ++++++++++++++++++++-------
drivers/infiniband/ulp/srp/ib_srp.h | 20 +-
3 files changed, 287 insertions(+), 95 deletions(-)
diff --git a/Documentation/ABI/stable/sysfs-driver-ib_srp b/Documentation/ABI/stable/sysfs-driver-ib_srp
index b9688de..d5a459e 100644
--- a/Documentation/ABI/stable/sysfs-driver-ib_srp
+++ b/Documentation/ABI/stable/sysfs-driver-ib_srp
@@ -55,12 +55,12 @@ Description: Interface for making ib_srp connect to a new target.
only safe with partial memory descriptor list support enabled
(allow_ext_sg=1).
* comp_vector, a number in the range 0..n-1 specifying the
- MSI-X completion vector. Some HCA's allocate multiple (n)
- MSI-X vectors per HCA port. If the IRQ affinity masks of
- these interrupts have been configured such that each MSI-X
- interrupt is handled by a different CPU then the comp_vector
- parameter can be used to spread the SRP completion workload
- over multiple CPU's.
+ MSI-X completion vector of the first RDMA channel. Some
+ HCA's allocate multiple (n) MSI-X vectors per HCA port. If
+ the IRQ affinity masks of these interrupts have been
+ configured such that each MSI-X interrupt is handled by a
+ different CPU then the comp_vector parameter can be used to
+ spread the SRP completion workload over multiple CPU's.
* tl_retry_count, a number in the range 2..7 specifying the
IB RC retry count.
* queue_size, the maximum number of commands that the
@@ -88,6 +88,13 @@ Description: Whether ib_srp is allowed to include a partial memory
descriptor list in an SRP_CMD when communicating with an SRP
target.
+What: /sys/class/scsi_host/host<n>/ch_count
+Date: November 1, 2014
+KernelVersion: 3.18
+Contact: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
+Description: Number of RDMA channels used for communication with the SRP
+ target.
+
What: /sys/class/scsi_host/host<n>/cmd_sg_entries
Date: May 19, 2011
KernelVersion: 2.6.39
@@ -95,6 +102,12 @@ Contact: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Description: Maximum number of data buffer descriptors that may be sent to
the target in a single SRP_CMD request.
+What: /sys/class/scsi_host/host<n>/comp_vector
+Date: September 2, 2013
+KernelVersion: 3.11
+Contact: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
+Description: Completion vector used for the first RDMA channel.
+
What: /sys/class/scsi_host/host<n>/dgid
Date: June 17, 2006
KernelVersion: 2.6.17
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 9feeea1..58ca618 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -123,6 +123,16 @@ MODULE_PARM_DESC(dev_loss_tmo,
" if fast_io_fail_tmo has not been set. \"off\" means that"
" this functionality is disabled.");
+static unsigned ch_count;
+module_param(ch_count, uint, 0444);
+MODULE_PARM_DESC(ch_count,
+ "Number of RDMA channels to use for communication with an SRP"
+ " target. Using more than one channel improves performance"
+ " if the HCA supports multiple completion vectors. The"
+ " default value is the minimum of four times the number of"
+ " online CPU sockets and the number of completion vectors"
+ " supported by the HCA.");
+
static void srp_add_one(struct ib_device *device);
static void srp_remove_one(struct ib_device *device);
static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr);
@@ -556,17 +566,32 @@ err:
* Note: this function may be called without srp_alloc_iu_bufs() having been
* invoked. Hence the ch->[rt]x_ring checks.
*/
-static void srp_free_ch_ib(struct srp_rdma_ch *ch)
+static void srp_free_ch_ib(struct srp_target_port *target,
+ struct srp_rdma_ch *ch)
{
- struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
int i;
+ if (!ch->target)
+ return;
+
+ /*
+ * Avoid that the SCSI error handler tries to use this channel after
+ * it has been freed. The SCSI error handler can namely continue
+ * trying to perform recovery actions after scsi_remove_host()
+ * returned.
+ */
+ ch->target = NULL;
+
if (ch->cm_id) {
ib_destroy_cm_id(ch->cm_id);
ch->cm_id = NULL;
}
+ /* If srp_new_cm_id() succeeded but srp_create_ch_ib() not, return. */
+ if (!ch->qp)
+ return;
+
if (dev->use_fast_reg) {
if (ch->fr_pool)
srp_destroy_fr_pool(ch->fr_pool);
@@ -647,7 +672,7 @@ static int srp_lookup_path(struct srp_rdma_ch *ch)
return ch->status;
}
-static int srp_send_req(struct srp_rdma_ch *ch)
+static int srp_send_req(struct srp_rdma_ch *ch, bool multich)
{
struct srp_target_port *target = ch->target;
struct {
@@ -688,6 +713,8 @@ static int srp_send_req(struct srp_rdma_ch *ch)
req->priv.req_it_iu_len = cpu_to_be32(target->max_iu_len);
req->priv.req_buf_fmt = cpu_to_be16(SRP_BUF_FORMAT_DIRECT |
SRP_BUF_FORMAT_INDIRECT);
+ req->priv.req_flags = (multich ? SRP_MULTICHAN_MULTI :
+ SRP_MULTICHAN_SINGLE);
/*
* In the published SRP specification (draft rev. 16a), the
* port identifier format is 8 bytes of ID extension followed
@@ -769,27 +796,31 @@ static bool srp_change_conn_state(struct srp_target_port *target,
static void srp_disconnect_target(struct srp_target_port *target)
{
- struct srp_rdma_ch *ch = &target->ch;
+ struct srp_rdma_ch *ch;
+ int i;
if (srp_change_conn_state(target, false)) {
/* XXX should send SRP_I_LOGOUT request */
- if (ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
- shost_printk(KERN_DEBUG, target->scsi_host,
- PFX "Sending CM DREQ failed\n");
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ if (ch->cm_id && ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
+ shost_printk(KERN_DEBUG, target->scsi_host,
+ PFX "Sending CM DREQ failed\n");
+ }
}
}
}
-static void srp_free_req_data(struct srp_rdma_ch *ch)
+static void srp_free_req_data(struct srp_target_port *target,
+ struct srp_rdma_ch *ch)
{
- struct srp_target_port *target = ch->target;
struct srp_device *dev = target->srp_host->srp_dev;
struct ib_device *ibdev = dev->dev;
struct srp_request *req;
int i;
- if (!ch->req_ring)
+ if (!ch->target || !ch->req_ring)
return;
for (i = 0; i < target->req_ring_size; ++i) {
@@ -853,7 +884,7 @@ static int srp_alloc_req_data(struct srp_rdma_ch *ch)
goto out;
req->indirect_dma_addr = dma_addr;
- req->index = i;
+ req->tag = build_srp_tag(ch - target->ch, i);
list_add_tail(&req->list, &ch->free_reqs);
}
ret = 0;
@@ -879,7 +910,8 @@ static void srp_del_scsi_host_attr(struct Scsi_Host *shost)
static void srp_remove_target(struct srp_target_port *target)
{
- struct srp_rdma_ch *ch = &target->ch;
+ struct srp_rdma_ch *ch;
+ int i;
WARN_ON_ONCE(target->state != SRP_TARGET_REMOVED);
@@ -889,10 +921,18 @@ static void srp_remove_target(struct srp_target_port *target)
scsi_remove_host(target->scsi_host);
srp_stop_rport_timers(target->rport);
srp_disconnect_target(target);
- srp_free_ch_ib(ch);
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ srp_free_ch_ib(target, ch);
+ }
cancel_work_sync(&target->tl_err_work);
srp_rport_put(target->rport);
- srp_free_req_data(ch);
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ srp_free_req_data(target, ch);
+ }
+ kfree(target->ch);
+ target->ch = NULL;
spin_lock(&target->srp_host->target_lock);
list_del(&target->list);
@@ -918,12 +958,12 @@ static void srp_rport_delete(struct srp_rport *rport)
srp_queue_remove_work(target);
}
-static int srp_connect_ch(struct srp_rdma_ch *ch)
+static int srp_connect_ch(struct srp_rdma_ch *ch, bool multich)
{
struct srp_target_port *target = ch->target;
int ret;
- WARN_ON_ONCE(target->connected);
+ WARN_ON_ONCE(!multich && target->connected);
target->qp_in_error = false;
@@ -933,7 +973,7 @@ static int srp_connect_ch(struct srp_rdma_ch *ch)
while (1) {
init_completion(&ch->done);
- ret = srp_send_req(ch);
+ ret = srp_send_req(ch, multich);
if (ret)
return ret;
ret = wait_for_completion_interruptible(&ch->done);
@@ -1095,10 +1135,10 @@ static void srp_finish_req(struct srp_rdma_ch *ch, struct srp_request *req,
static void srp_terminate_io(struct srp_rport *rport)
{
struct srp_target_port *target = rport->lld_data;
- struct srp_rdma_ch *ch = &target->ch;
+ struct srp_rdma_ch *ch;
struct Scsi_Host *shost = target->scsi_host;
struct scsi_device *sdev;
- int i;
+ int i, j;
/*
* Invoking srp_terminate_io() while srp_queuecommand() is running
@@ -1107,10 +1147,15 @@ static void srp_terminate_io(struct srp_rport *rport)
shost_for_each_device(sdev, shost)
WARN_ON_ONCE(sdev->request_queue->request_fn_active);
- for (i = 0; i < target->req_ring_size; ++i) {
- struct srp_request *req = &ch->req_ring[i];
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+
+ for (j = 0; j < target->req_ring_size; ++j) {
+ struct srp_request *req = &ch->req_ring[j];
- srp_finish_req(ch, req, NULL, DID_TRANSPORT_FAILFAST << 16);
+ srp_finish_req(ch, req, NULL,
+ DID_TRANSPORT_FAILFAST << 16);
+ }
}
}
@@ -1126,8 +1171,9 @@ static void srp_terminate_io(struct srp_rport *rport)
static int srp_rport_reconnect(struct srp_rport *rport)
{
struct srp_target_port *target = rport->lld_data;
- struct srp_rdma_ch *ch = &target->ch;
- int i, ret;
+ struct srp_rdma_ch *ch;
+ int i, j, ret = 0;
+ bool multich = false;
srp_disconnect_target(target);
@@ -1139,27 +1185,43 @@ static int srp_rport_reconnect(struct srp_rport *rport)
* case things are really fouled up. Doing so also ensures that all CM
* callbacks will have finished before a new QP is allocated.
*/
- ret = srp_new_cm_id(ch);
-
- for (i = 0; i < target->req_ring_size; ++i) {
- struct srp_request *req = &ch->req_ring[i];
-
- srp_finish_req(ch, req, NULL, DID_RESET << 16);
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ if (!ch->target)
+ return -ENODEV;
+ ret += srp_new_cm_id(ch);
+ }
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ for (j = 0; j < target->req_ring_size; ++j) {
+ struct srp_request *req = &ch->req_ring[j];
+
+ srp_finish_req(ch, req, NULL, DID_RESET << 16);
+ }
}
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ /*
+ * Whether or not creating a new CM ID succeeded, create a new
+ * QP. This guarantees that all completion callback function
+ * invocations have finished before request resetting starts.
+ */
+ ret += srp_create_ch_ib(ch);
- /*
- * Whether or not creating a new CM ID succeeded, create a new
- * QP. This guarantees that all callback functions for the old QP have
- * finished before any send requests are posted on the new QP.
- */
- ret += srp_create_ch_ib(ch);
-
- INIT_LIST_HEAD(&ch->free_tx);
- for (i = 0; i < target->queue_size; ++i)
- list_add(&ch->tx_ring[i]->list, &ch->free_tx);
-
- if (ret == 0)
- ret = srp_connect_ch(ch);
+ INIT_LIST_HEAD(&ch->free_tx);
+ for (j = 0; j < target->queue_size; ++j)
+ list_add(&ch->tx_ring[j]->list, &ch->free_tx);
+ }
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ if (ret) {
+ if (i > 1)
+ ret = 0;
+ break;
+ }
+ ret = srp_connect_ch(ch, multich);
+ multich = true;
+ }
if (ret == 0)
shost_printk(KERN_INFO, target->scsi_host,
@@ -1573,7 +1635,7 @@ static struct srp_iu *__srp_get_tx_iu(struct srp_rdma_ch *ch,
s32 rsv = (iu_type == SRP_IU_TSK_MGMT) ? 0 : SRP_TSK_MGMT_SQ_SIZE;
struct srp_iu *iu;
- srp_send_completion(ch->send_cq, target);
+ srp_send_completion(ch->send_cq, ch);
if (list_empty(&ch->free_tx))
return NULL;
@@ -1637,6 +1699,7 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
struct srp_request *req;
struct scsi_cmnd *scmnd;
unsigned long flags;
+ unsigned i;
if (unlikely(rsp->tag & SRP_TAG_TSK_MGMT)) {
spin_lock_irqsave(&ch->lock, flags);
@@ -1648,12 +1711,20 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
ch->tsk_mgmt_status = rsp->data[3];
complete(&ch->tsk_mgmt_done);
} else {
- req = &ch->req_ring[rsp->tag];
- scmnd = srp_claim_req(ch, req, NULL, NULL);
+ if (srp_tag_ch(rsp->tag) != ch - target->ch)
+ pr_err("Channel idx mismatch: tag %#llx <> ch %#lx\n",
+ rsp->tag, ch - target->ch);
+ i = srp_tag_idx(rsp->tag);
+ if (i < target->req_ring_size) {
+ req = &ch->req_ring[i];
+ scmnd = srp_claim_req(ch, req, NULL, NULL);
+ } else {
+ scmnd = NULL;
+ }
if (!scmnd) {
shost_printk(KERN_ERR, target->scsi_host,
- "Null scmnd for RSP w/tag %016llx\n",
- (unsigned long long) rsp->tag);
+ "Null scmnd for RSP w/tag %#016llx received on ch %ld / QP %#x\n",
+ rsp->tag, ch - target->ch, ch->qp->qp_num);
spin_lock_irqsave(&ch->lock, flags);
ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
@@ -1879,7 +1950,8 @@ static void srp_send_completion(struct ib_cq *cq, void *ch_ptr)
}
}
-static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
+static int srp_queuecommand(unsigned hwq, struct Scsi_Host *shost,
+ struct scsi_cmnd *scmnd)
{
struct srp_target_port *target = host_to_target(shost);
struct srp_rport *rport = target->rport;
@@ -1905,7 +1977,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
if (unlikely(scmnd->result))
goto err;
- ch = &target->ch;
+ ch = &target->ch[hwq];
spin_lock_irqsave(&ch->lock, flags);
iu = __srp_get_tx_iu(ch, SRP_IU_CMD);
@@ -1927,7 +1999,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
cmd->opcode = SRP_CMD;
cmd->lun = cpu_to_be64((u64) scmnd->device->lun << 48);
- cmd->tag = req->index;
+ cmd->tag = req->tag;
memcpy(cmd->cdb, scmnd->cmnd, scmnd->cmd_len);
req->scmnd = scmnd;
@@ -1993,6 +2065,17 @@ err:
goto unlock_rport;
}
+static int srp_sq_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
+{
+ return srp_queuecommand(0, shost, scmnd);
+}
+
+static int srp_mq_queuecommand(struct blk_mq_hw_ctx *hctx,
+ struct scsi_cmnd *scmnd)
+{
+ return srp_queuecommand(hctx->queue_num, scmnd->device->host, scmnd);
+}
+
/*
* Note: the resources allocated in this function are freed in
* srp_free_ch_ib().
@@ -2409,15 +2492,23 @@ static int srp_abort(struct scsi_cmnd *scmnd)
{
struct srp_target_port *target = host_to_target(scmnd->device->host);
struct srp_request *req = (struct srp_request *) scmnd->host_scribble;
+ u16 ch_idx;
struct srp_rdma_ch *ch;
int ret;
shost_printk(KERN_ERR, target->scsi_host, "SRP abort called\n");
- ch = &target->ch;
- if (!req || !srp_claim_req(ch, req, NULL, scmnd))
+ if (!req)
+ return SUCCESS;
+ ch_idx = srp_tag_ch(req->tag);
+ if (WARN_ON_ONCE(ch_idx >= target->ch_count))
return SUCCESS;
- if (srp_send_tsk_mgmt(ch, req->index, scmnd->device->lun,
+ ch = &target->ch[ch_idx];
+ if (!srp_claim_req(ch, req, NULL, scmnd))
+ return SUCCESS;
+ shost_printk(KERN_ERR, target->scsi_host,
+ "Sending SRP abort for tag %#x\n", req->tag);
+ if (srp_send_tsk_mgmt(ch, req->tag, scmnd->device->lun,
SRP_TSK_ABORT_TASK) == 0)
ret = SUCCESS;
else if (target->rport->state == SRP_RPORT_LOST)
@@ -2434,21 +2525,25 @@ static int srp_abort(struct scsi_cmnd *scmnd)
static int srp_reset_device(struct scsi_cmnd *scmnd)
{
struct srp_target_port *target = host_to_target(scmnd->device->host);
- struct srp_rdma_ch *ch = &target->ch;
+ struct srp_rdma_ch *ch;
int i;
shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n");
+ ch = &target->ch[0];
if (srp_send_tsk_mgmt(ch, SRP_TAG_NO_REQ, scmnd->device->lun,
SRP_TSK_LUN_RESET))
return FAILED;
if (ch->tsk_mgmt_status)
return FAILED;
- for (i = 0; i < target->req_ring_size; ++i) {
- struct srp_request *req = &ch->req_ring[i];
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ for (i = 0; i < target->req_ring_size; ++i) {
+ struct srp_request *req = &ch->req_ring[i];
- srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
+ srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
+ }
}
return SUCCESS;
@@ -2525,7 +2620,7 @@ static ssize_t show_dgid(struct device *dev, struct device_attribute *attr,
char *buf)
{
struct srp_target_port *target = host_to_target(class_to_shost(dev));
- struct srp_rdma_ch *ch = &target->ch;
+ struct srp_rdma_ch *ch = &target->ch[0];
return sprintf(buf, "%pI6\n", ch->path.dgid.raw);
}
@@ -2542,8 +2637,14 @@ static ssize_t show_req_lim(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct srp_target_port *target = host_to_target(class_to_shost(dev));
+ struct srp_rdma_ch *ch;
+ int i, req_lim = INT_MAX;
- return sprintf(buf, "%d\n", target->ch.req_lim);
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ req_lim = min(req_lim, ch->req_lim);
+ }
+ return sprintf(buf, "%d\n", req_lim);
}
static ssize_t show_zero_req_lim(struct device *dev,
@@ -2570,6 +2671,14 @@ static ssize_t show_local_ib_device(struct device *dev,
return sprintf(buf, "%s\n", target->srp_host->srp_dev->dev->name);
}
+static ssize_t show_ch_count(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct srp_target_port *target = host_to_target(class_to_shost(dev));
+
+ return sprintf(buf, "%d\n", target->ch_count);
+}
+
static ssize_t show_comp_vector(struct device *dev,
struct device_attribute *attr, char *buf)
{
@@ -2613,6 +2722,7 @@ static DEVICE_ATTR(req_lim, S_IRUGO, show_req_lim, NULL);
static DEVICE_ATTR(zero_req_lim, S_IRUGO, show_zero_req_lim, NULL);
static DEVICE_ATTR(local_ib_port, S_IRUGO, show_local_ib_port, NULL);
static DEVICE_ATTR(local_ib_device, S_IRUGO, show_local_ib_device, NULL);
+static DEVICE_ATTR(ch_count, S_IRUGO, show_ch_count, NULL);
static DEVICE_ATTR(comp_vector, S_IRUGO, show_comp_vector, NULL);
static DEVICE_ATTR(tl_retry_count, S_IRUGO, show_tl_retry_count, NULL);
static DEVICE_ATTR(cmd_sg_entries, S_IRUGO, show_cmd_sg_entries, NULL);
@@ -2630,6 +2740,7 @@ static struct device_attribute *srp_host_attrs[] = {
&dev_attr_zero_req_lim,
&dev_attr_local_ib_port,
&dev_attr_local_ib_device,
+ &dev_attr_ch_count,
&dev_attr_comp_vector,
&dev_attr_tl_retry_count,
&dev_attr_cmd_sg_entries,
@@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
.proc_name = DRV_NAME,
.slave_configure = srp_slave_configure,
.info = srp_target_info,
- .queuecommand = srp_queuecommand,
+ .queuecommand = srp_sq_queuecommand,
+ .mq_queuecommand = srp_mq_queuecommand,
.change_queue_depth = srp_change_queue_depth,
.change_queue_type = srp_change_queue_type,
.eh_abort_handler = srp_abort,
@@ -3038,7 +3150,8 @@ static ssize_t srp_create_target(struct device *dev,
struct srp_rdma_ch *ch;
struct srp_device *srp_dev = host->srp_dev;
struct ib_device *ibdev = srp_dev->dev;
- int ret;
+ int ret, node_idx, node, cpu, i;
+ bool multich = false;
target_host = scsi_host_alloc(&srp_template,
sizeof (struct srp_target_port));
@@ -3098,34 +3211,82 @@ static ssize_t srp_create_target(struct device *dev,
INIT_WORK(&target->tl_err_work, srp_tl_err_work);
INIT_WORK(&target->remove_work, srp_remove_work);
spin_lock_init(&target->lock);
- ch = &target->ch;
- ch->target = target;
- ch->comp_vector = target->comp_vector;
- spin_lock_init(&ch->lock);
- INIT_LIST_HEAD(&ch->free_tx);
- ret = srp_alloc_req_data(ch);
- if (ret)
- goto err_free_mem;
-
ret = ib_query_gid(ibdev, host->port, 0, &target->sgid);
if (ret)
- goto err_free_mem;
+ goto err;
- ret = srp_create_ch_ib(ch);
- if (ret)
- goto err_free_mem;
+ ret = -ENOMEM;
+ target->ch_count = max_t(unsigned, num_online_nodes(),
+ min(ch_count ? :
+ min(4 * num_online_nodes(),
+ ibdev->num_comp_vectors),
+ num_online_cpus()));
+ target->ch = kcalloc(target->ch_count, sizeof(*target->ch),
+ GFP_KERNEL);
+ if (!target->ch)
+ goto err;
- ret = srp_new_cm_id(ch);
- if (ret)
- goto err_free_ib;
+ node_idx = 0;
+ for_each_online_node(node) {
+ const int ch_start = (node_idx * target->ch_count /
+ num_online_nodes());
+ const int ch_end = ((node_idx + 1) * target->ch_count /
+ num_online_nodes());
+ const int cv_start = (node_idx * ibdev->num_comp_vectors /
+ num_online_nodes() + target->comp_vector)
+ % ibdev->num_comp_vectors;
+ const int cv_end = ((node_idx + 1) * ibdev->num_comp_vectors /
+ num_online_nodes() + target->comp_vector)
+ % ibdev->num_comp_vectors;
+ int cpu_idx = 0;
+
+ for_each_online_cpu(cpu) {
+ if (cpu_to_node(cpu) != node)
+ continue;
+ if (ch_start + cpu_idx >= ch_end)
+ continue;
+ ch = &target->ch[ch_start + cpu_idx];
+ ch->target = target;
+ ch->comp_vector = cv_start == cv_end ? cv_start :
+ cv_start + cpu_idx % (cv_end - cv_start);
+ spin_lock_init(&ch->lock);
+ INIT_LIST_HEAD(&ch->free_tx);
+ ret = srp_new_cm_id(ch);
+ if (ret)
+ goto err_disconnect;
- ret = srp_connect_ch(ch);
- if (ret) {
- shost_printk(KERN_ERR, target->scsi_host,
- PFX "Connection failed\n");
- goto err_free_ib;
+ ret = srp_create_ch_ib(ch);
+ if (ret)
+ goto err_disconnect;
+
+ ret = srp_alloc_req_data(ch);
+ if (ret)
+ goto err_disconnect;
+
+ ret = srp_connect_ch(ch, multich);
+ if (ret) {
+ shost_printk(KERN_ERR, target->scsi_host,
+ PFX "Connection %d/%d failed\n",
+ ch_start + cpu_idx,
+ target->ch_count);
+ if (node_idx == 0 && cpu_idx == 0) {
+ goto err_disconnect;
+ } else {
+ srp_free_ch_ib(target, ch);
+ srp_free_req_data(target, ch);
+ target->ch_count = ch - target->ch;
+ break;
+ }
+ }
+
+ multich = true;
+ cpu_idx++;
+ }
+ node_idx++;
}
+ target->scsi_host->nr_hw_queues = target->ch_count;
+
ret = srp_add_target(host, target);
if (ret)
goto err_disconnect;
@@ -3154,11 +3315,13 @@ out:
err_disconnect:
srp_disconnect_target(target);
-err_free_ib:
- srp_free_ch_ib(ch);
+ for (i = 0; i < target->ch_count; i++) {
+ ch = &target->ch[i];
+ srp_free_ch_ib(target, ch);
+ srp_free_req_data(target, ch);
+ }
-err_free_mem:
- srp_free_req_data(ch);
+ kfree(target->ch);
err:
scsi_host_put(target_host);
diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
index 0609124..d9660e1 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.h
+++ b/drivers/infiniband/ulp/srp/ib_srp.h
@@ -84,6 +84,21 @@ enum srp_iu_type {
SRP_IU_RSP,
};
+static inline u32 build_srp_tag(u16 ch, u16 req_idx)
+{
+ return ch << 16 | req_idx;
+}
+
+static inline u16 srp_tag_ch(u32 tag)
+{
+ return tag >> 16;
+}
+
+static inline u16 srp_tag_idx(u32 tag)
+{
+ return tag & ((1 << 16) - 1);
+}
+
/*
* @mr_page_mask: HCA memory registration page mask.
* @mr_page_size: HCA memory registration page size.
@@ -127,7 +142,7 @@ struct srp_request {
struct srp_direct_buf *indirect_desc;
dma_addr_t indirect_dma_addr;
short nmdesc;
- short index;
+ uint32_t tag;
};
struct srp_rdma_ch {
@@ -173,8 +188,9 @@ struct srp_target_port {
/* read and written in the hot path */
spinlock_t lock;
- struct srp_rdma_ch ch;
/* read only in the hot path */
+ struct srp_rdma_ch *ch;
+ u32 ch_count;
u32 lkey;
u32 rkey;
enum srp_target_state state;
--
1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <541C28E0.7010705-HInyCGIudOg@public.gmane.org>
@ 2014-09-19 14:28 ` Ming Lei
[not found] ` <CACVXFVPzz37J-613NZCfPStUBxf0rLOtz71LJ07PpCxYg5nn+g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Ming Lei @ 2014-09-19 14:28 UTC (permalink / raw)
To: Bart Van Assche
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Jens Axboe, Robert Elliott
On Fri, Sep 19, 2014 at 9:00 PM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org> wrote:
> Improve performance by using multiple RDMA/RC channels per SCSI host
> for communicating with an SRP target.
>
> Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
> ---
> Documentation/ABI/stable/sysfs-driver-ib_srp | 25 +-
> drivers/infiniband/ulp/srp/ib_srp.c | 337 ++++++++++++++++++++-------
> drivers/infiniband/ulp/srp/ib_srp.h | 20 +-
> 3 files changed, 287 insertions(+), 95 deletions(-)
>
> diff --git a/Documentation/ABI/stable/sysfs-driver-ib_srp b/Documentation/ABI/stable/sysfs-driver-ib_srp
> index b9688de..d5a459e 100644
> --- a/Documentation/ABI/stable/sysfs-driver-ib_srp
> +++ b/Documentation/ABI/stable/sysfs-driver-ib_srp
> @@ -55,12 +55,12 @@ Description: Interface for making ib_srp connect to a new target.
> only safe with partial memory descriptor list support enabled
> (allow_ext_sg=1).
> * comp_vector, a number in the range 0..n-1 specifying the
> - MSI-X completion vector. Some HCA's allocate multiple (n)
> - MSI-X vectors per HCA port. If the IRQ affinity masks of
> - these interrupts have been configured such that each MSI-X
> - interrupt is handled by a different CPU then the comp_vector
> - parameter can be used to spread the SRP completion workload
> - over multiple CPU's.
> + MSI-X completion vector of the first RDMA channel. Some
> + HCA's allocate multiple (n) MSI-X vectors per HCA port. If
> + the IRQ affinity masks of these interrupts have been
> + configured such that each MSI-X interrupt is handled by a
> + different CPU then the comp_vector parameter can be used to
> + spread the SRP completion workload over multiple CPU's.
> * tl_retry_count, a number in the range 2..7 specifying the
> IB RC retry count.
> * queue_size, the maximum number of commands that the
> @@ -88,6 +88,13 @@ Description: Whether ib_srp is allowed to include a partial memory
> descriptor list in an SRP_CMD when communicating with an SRP
> target.
>
> +What: /sys/class/scsi_host/host<n>/ch_count
> +Date: November 1, 2014
> +KernelVersion: 3.18
> +Contact: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> +Description: Number of RDMA channels used for communication with the SRP
> + target.
> +
> What: /sys/class/scsi_host/host<n>/cmd_sg_entries
> Date: May 19, 2011
> KernelVersion: 2.6.39
> @@ -95,6 +102,12 @@ Contact: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> Description: Maximum number of data buffer descriptors that may be sent to
> the target in a single SRP_CMD request.
>
> +What: /sys/class/scsi_host/host<n>/comp_vector
> +Date: September 2, 2013
> +KernelVersion: 3.11
> +Contact: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> +Description: Completion vector used for the first RDMA channel.
> +
> What: /sys/class/scsi_host/host<n>/dgid
> Date: June 17, 2006
> KernelVersion: 2.6.17
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index 9feeea1..58ca618 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -123,6 +123,16 @@ MODULE_PARM_DESC(dev_loss_tmo,
> " if fast_io_fail_tmo has not been set. \"off\" means that"
> " this functionality is disabled.");
>
> +static unsigned ch_count;
> +module_param(ch_count, uint, 0444);
> +MODULE_PARM_DESC(ch_count,
> + "Number of RDMA channels to use for communication with an SRP"
> + " target. Using more than one channel improves performance"
> + " if the HCA supports multiple completion vectors. The"
> + " default value is the minimum of four times the number of"
> + " online CPU sockets and the number of completion vectors"
> + " supported by the HCA.");
> +
> static void srp_add_one(struct ib_device *device);
> static void srp_remove_one(struct ib_device *device);
> static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr);
> @@ -556,17 +566,32 @@ err:
> * Note: this function may be called without srp_alloc_iu_bufs() having been
> * invoked. Hence the ch->[rt]x_ring checks.
> */
> -static void srp_free_ch_ib(struct srp_rdma_ch *ch)
> +static void srp_free_ch_ib(struct srp_target_port *target,
> + struct srp_rdma_ch *ch)
> {
> - struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> int i;
>
> + if (!ch->target)
> + return;
> +
> + /*
> + * Avoid that the SCSI error handler tries to use this channel after
> + * it has been freed. The SCSI error handler can namely continue
> + * trying to perform recovery actions after scsi_remove_host()
> + * returned.
> + */
> + ch->target = NULL;
> +
> if (ch->cm_id) {
> ib_destroy_cm_id(ch->cm_id);
> ch->cm_id = NULL;
> }
>
> + /* If srp_new_cm_id() succeeded but srp_create_ch_ib() not, return. */
> + if (!ch->qp)
> + return;
> +
> if (dev->use_fast_reg) {
> if (ch->fr_pool)
> srp_destroy_fr_pool(ch->fr_pool);
> @@ -647,7 +672,7 @@ static int srp_lookup_path(struct srp_rdma_ch *ch)
> return ch->status;
> }
>
> -static int srp_send_req(struct srp_rdma_ch *ch)
> +static int srp_send_req(struct srp_rdma_ch *ch, bool multich)
> {
> struct srp_target_port *target = ch->target;
> struct {
> @@ -688,6 +713,8 @@ static int srp_send_req(struct srp_rdma_ch *ch)
> req->priv.req_it_iu_len = cpu_to_be32(target->max_iu_len);
> req->priv.req_buf_fmt = cpu_to_be16(SRP_BUF_FORMAT_DIRECT |
> SRP_BUF_FORMAT_INDIRECT);
> + req->priv.req_flags = (multich ? SRP_MULTICHAN_MULTI :
> + SRP_MULTICHAN_SINGLE);
> /*
> * In the published SRP specification (draft rev. 16a), the
> * port identifier format is 8 bytes of ID extension followed
> @@ -769,27 +796,31 @@ static bool srp_change_conn_state(struct srp_target_port *target,
>
> static void srp_disconnect_target(struct srp_target_port *target)
> {
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> + int i;
>
> if (srp_change_conn_state(target, false)) {
> /* XXX should send SRP_I_LOGOUT request */
>
> - if (ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
> - shost_printk(KERN_DEBUG, target->scsi_host,
> - PFX "Sending CM DREQ failed\n");
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + if (ch->cm_id && ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
> + shost_printk(KERN_DEBUG, target->scsi_host,
> + PFX "Sending CM DREQ failed\n");
> + }
> }
> }
> }
>
> -static void srp_free_req_data(struct srp_rdma_ch *ch)
> +static void srp_free_req_data(struct srp_target_port *target,
> + struct srp_rdma_ch *ch)
> {
> - struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> struct srp_request *req;
> int i;
>
> - if (!ch->req_ring)
> + if (!ch->target || !ch->req_ring)
> return;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> @@ -853,7 +884,7 @@ static int srp_alloc_req_data(struct srp_rdma_ch *ch)
> goto out;
>
> req->indirect_dma_addr = dma_addr;
> - req->index = i;
> + req->tag = build_srp_tag(ch - target->ch, i);
> list_add_tail(&req->list, &ch->free_reqs);
> }
> ret = 0;
> @@ -879,7 +910,8 @@ static void srp_del_scsi_host_attr(struct Scsi_Host *shost)
>
> static void srp_remove_target(struct srp_target_port *target)
> {
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> + int i;
>
> WARN_ON_ONCE(target->state != SRP_TARGET_REMOVED);
>
> @@ -889,10 +921,18 @@ static void srp_remove_target(struct srp_target_port *target)
> scsi_remove_host(target->scsi_host);
> srp_stop_rport_timers(target->rport);
> srp_disconnect_target(target);
> - srp_free_ch_ib(ch);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + srp_free_ch_ib(target, ch);
> + }
> cancel_work_sync(&target->tl_err_work);
> srp_rport_put(target->rport);
> - srp_free_req_data(ch);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + srp_free_req_data(target, ch);
> + }
> + kfree(target->ch);
> + target->ch = NULL;
>
> spin_lock(&target->srp_host->target_lock);
> list_del(&target->list);
> @@ -918,12 +958,12 @@ static void srp_rport_delete(struct srp_rport *rport)
> srp_queue_remove_work(target);
> }
>
> -static int srp_connect_ch(struct srp_rdma_ch *ch)
> +static int srp_connect_ch(struct srp_rdma_ch *ch, bool multich)
> {
> struct srp_target_port *target = ch->target;
> int ret;
>
> - WARN_ON_ONCE(target->connected);
> + WARN_ON_ONCE(!multich && target->connected);
>
> target->qp_in_error = false;
>
> @@ -933,7 +973,7 @@ static int srp_connect_ch(struct srp_rdma_ch *ch)
>
> while (1) {
> init_completion(&ch->done);
> - ret = srp_send_req(ch);
> + ret = srp_send_req(ch, multich);
> if (ret)
> return ret;
> ret = wait_for_completion_interruptible(&ch->done);
> @@ -1095,10 +1135,10 @@ static void srp_finish_req(struct srp_rdma_ch *ch, struct srp_request *req,
> static void srp_terminate_io(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> struct Scsi_Host *shost = target->scsi_host;
> struct scsi_device *sdev;
> - int i;
> + int i, j;
>
> /*
> * Invoking srp_terminate_io() while srp_queuecommand() is running
> @@ -1107,10 +1147,15 @@ static void srp_terminate_io(struct srp_rport *rport)
> shost_for_each_device(sdev, shost)
> WARN_ON_ONCE(sdev->request_queue->request_fn_active);
>
> - for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &ch->req_ring[i];
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> +
> + for (j = 0; j < target->req_ring_size; ++j) {
> + struct srp_request *req = &ch->req_ring[j];
>
> - srp_finish_req(ch, req, NULL, DID_TRANSPORT_FAILFAST << 16);
> + srp_finish_req(ch, req, NULL,
> + DID_TRANSPORT_FAILFAST << 16);
> + }
> }
> }
>
> @@ -1126,8 +1171,9 @@ static void srp_terminate_io(struct srp_rport *rport)
> static int srp_rport_reconnect(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> - struct srp_rdma_ch *ch = &target->ch;
> - int i, ret;
> + struct srp_rdma_ch *ch;
> + int i, j, ret = 0;
> + bool multich = false;
>
> srp_disconnect_target(target);
>
> @@ -1139,27 +1185,43 @@ static int srp_rport_reconnect(struct srp_rport *rport)
> * case things are really fouled up. Doing so also ensures that all CM
> * callbacks will have finished before a new QP is allocated.
> */
> - ret = srp_new_cm_id(ch);
> -
> - for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &ch->req_ring[i];
> -
> - srp_finish_req(ch, req, NULL, DID_RESET << 16);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + if (!ch->target)
> + return -ENODEV;
> + ret += srp_new_cm_id(ch);
> + }
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + for (j = 0; j < target->req_ring_size; ++j) {
> + struct srp_request *req = &ch->req_ring[j];
> +
> + srp_finish_req(ch, req, NULL, DID_RESET << 16);
> + }
> }
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + /*
> + * Whether or not creating a new CM ID succeeded, create a new
> + * QP. This guarantees that all completion callback function
> + * invocations have finished before request resetting starts.
> + */
> + ret += srp_create_ch_ib(ch);
>
> - /*
> - * Whether or not creating a new CM ID succeeded, create a new
> - * QP. This guarantees that all callback functions for the old QP have
> - * finished before any send requests are posted on the new QP.
> - */
> - ret += srp_create_ch_ib(ch);
> -
> - INIT_LIST_HEAD(&ch->free_tx);
> - for (i = 0; i < target->queue_size; ++i)
> - list_add(&ch->tx_ring[i]->list, &ch->free_tx);
> -
> - if (ret == 0)
> - ret = srp_connect_ch(ch);
> + INIT_LIST_HEAD(&ch->free_tx);
> + for (j = 0; j < target->queue_size; ++j)
> + list_add(&ch->tx_ring[j]->list, &ch->free_tx);
> + }
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + if (ret) {
> + if (i > 1)
> + ret = 0;
> + break;
> + }
> + ret = srp_connect_ch(ch, multich);
> + multich = true;
> + }
>
> if (ret == 0)
> shost_printk(KERN_INFO, target->scsi_host,
> @@ -1573,7 +1635,7 @@ static struct srp_iu *__srp_get_tx_iu(struct srp_rdma_ch *ch,
> s32 rsv = (iu_type == SRP_IU_TSK_MGMT) ? 0 : SRP_TSK_MGMT_SQ_SIZE;
> struct srp_iu *iu;
>
> - srp_send_completion(ch->send_cq, target);
> + srp_send_completion(ch->send_cq, ch);
>
> if (list_empty(&ch->free_tx))
> return NULL;
> @@ -1637,6 +1699,7 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
> struct srp_request *req;
> struct scsi_cmnd *scmnd;
> unsigned long flags;
> + unsigned i;
>
> if (unlikely(rsp->tag & SRP_TAG_TSK_MGMT)) {
> spin_lock_irqsave(&ch->lock, flags);
> @@ -1648,12 +1711,20 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
> ch->tsk_mgmt_status = rsp->data[3];
> complete(&ch->tsk_mgmt_done);
> } else {
> - req = &ch->req_ring[rsp->tag];
> - scmnd = srp_claim_req(ch, req, NULL, NULL);
> + if (srp_tag_ch(rsp->tag) != ch - target->ch)
> + pr_err("Channel idx mismatch: tag %#llx <> ch %#lx\n",
> + rsp->tag, ch - target->ch);
> + i = srp_tag_idx(rsp->tag);
> + if (i < target->req_ring_size) {
> + req = &ch->req_ring[i];
> + scmnd = srp_claim_req(ch, req, NULL, NULL);
> + } else {
> + scmnd = NULL;
> + }
> if (!scmnd) {
> shost_printk(KERN_ERR, target->scsi_host,
> - "Null scmnd for RSP w/tag %016llx\n",
> - (unsigned long long) rsp->tag);
> + "Null scmnd for RSP w/tag %#016llx received on ch %ld / QP %#x\n",
> + rsp->tag, ch - target->ch, ch->qp->qp_num);
>
> spin_lock_irqsave(&ch->lock, flags);
> ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
> @@ -1879,7 +1950,8 @@ static void srp_send_completion(struct ib_cq *cq, void *ch_ptr)
> }
> }
>
> -static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> +static int srp_queuecommand(unsigned hwq, struct Scsi_Host *shost,
> + struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(shost);
> struct srp_rport *rport = target->rport;
> @@ -1905,7 +1977,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> if (unlikely(scmnd->result))
> goto err;
>
> - ch = &target->ch;
> + ch = &target->ch[hwq];
>
> spin_lock_irqsave(&ch->lock, flags);
> iu = __srp_get_tx_iu(ch, SRP_IU_CMD);
> @@ -1927,7 +1999,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
>
> cmd->opcode = SRP_CMD;
> cmd->lun = cpu_to_be64((u64) scmnd->device->lun << 48);
> - cmd->tag = req->index;
> + cmd->tag = req->tag;
> memcpy(cmd->cdb, scmnd->cmnd, scmnd->cmd_len);
>
> req->scmnd = scmnd;
> @@ -1993,6 +2065,17 @@ err:
> goto unlock_rport;
> }
>
> +static int srp_sq_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> +{
> + return srp_queuecommand(0, shost, scmnd);
> +}
> +
> +static int srp_mq_queuecommand(struct blk_mq_hw_ctx *hctx,
> + struct scsi_cmnd *scmnd)
> +{
> + return srp_queuecommand(hctx->queue_num, scmnd->device->host, scmnd);
> +}
> +
> /*
> * Note: the resources allocated in this function are freed in
> * srp_free_ch_ib().
> @@ -2409,15 +2492,23 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> struct srp_request *req = (struct srp_request *) scmnd->host_scribble;
> + u16 ch_idx;
> struct srp_rdma_ch *ch;
> int ret;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP abort called\n");
>
> - ch = &target->ch;
> - if (!req || !srp_claim_req(ch, req, NULL, scmnd))
> + if (!req)
> + return SUCCESS;
> + ch_idx = srp_tag_ch(req->tag);
> + if (WARN_ON_ONCE(ch_idx >= target->ch_count))
> return SUCCESS;
> - if (srp_send_tsk_mgmt(ch, req->index, scmnd->device->lun,
> + ch = &target->ch[ch_idx];
> + if (!srp_claim_req(ch, req, NULL, scmnd))
> + return SUCCESS;
> + shost_printk(KERN_ERR, target->scsi_host,
> + "Sending SRP abort for tag %#x\n", req->tag);
> + if (srp_send_tsk_mgmt(ch, req->tag, scmnd->device->lun,
> SRP_TSK_ABORT_TASK) == 0)
> ret = SUCCESS;
> else if (target->rport->state == SRP_RPORT_LOST)
> @@ -2434,21 +2525,25 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> static int srp_reset_device(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> int i;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n");
>
> + ch = &target->ch[0];
> if (srp_send_tsk_mgmt(ch, SRP_TAG_NO_REQ, scmnd->device->lun,
> SRP_TSK_LUN_RESET))
> return FAILED;
> if (ch->tsk_mgmt_status)
> return FAILED;
>
> - for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &ch->req_ring[i];
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + for (i = 0; i < target->req_ring_size; ++i) {
> + struct srp_request *req = &ch->req_ring[i];
>
> - srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
> + srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
> + }
> }
>
> return SUCCESS;
> @@ -2525,7 +2620,7 @@ static ssize_t show_dgid(struct device *dev, struct device_attribute *attr,
> char *buf)
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch = &target->ch[0];
>
> return sprintf(buf, "%pI6\n", ch->path.dgid.raw);
> }
> @@ -2542,8 +2637,14 @@ static ssize_t show_req_lim(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
> + struct srp_rdma_ch *ch;
> + int i, req_lim = INT_MAX;
>
> - return sprintf(buf, "%d\n", target->ch.req_lim);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + req_lim = min(req_lim, ch->req_lim);
> + }
> + return sprintf(buf, "%d\n", req_lim);
> }
>
> static ssize_t show_zero_req_lim(struct device *dev,
> @@ -2570,6 +2671,14 @@ static ssize_t show_local_ib_device(struct device *dev,
> return sprintf(buf, "%s\n", target->srp_host->srp_dev->dev->name);
> }
>
> +static ssize_t show_ch_count(struct device *dev, struct device_attribute *attr,
> + char *buf)
> +{
> + struct srp_target_port *target = host_to_target(class_to_shost(dev));
> +
> + return sprintf(buf, "%d\n", target->ch_count);
> +}
> +
> static ssize_t show_comp_vector(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> @@ -2613,6 +2722,7 @@ static DEVICE_ATTR(req_lim, S_IRUGO, show_req_lim, NULL);
> static DEVICE_ATTR(zero_req_lim, S_IRUGO, show_zero_req_lim, NULL);
> static DEVICE_ATTR(local_ib_port, S_IRUGO, show_local_ib_port, NULL);
> static DEVICE_ATTR(local_ib_device, S_IRUGO, show_local_ib_device, NULL);
> +static DEVICE_ATTR(ch_count, S_IRUGO, show_ch_count, NULL);
> static DEVICE_ATTR(comp_vector, S_IRUGO, show_comp_vector, NULL);
> static DEVICE_ATTR(tl_retry_count, S_IRUGO, show_tl_retry_count, NULL);
> static DEVICE_ATTR(cmd_sg_entries, S_IRUGO, show_cmd_sg_entries, NULL);
> @@ -2630,6 +2740,7 @@ static struct device_attribute *srp_host_attrs[] = {
> &dev_attr_zero_req_lim,
> &dev_attr_local_ib_port,
> &dev_attr_local_ib_device,
> + &dev_attr_ch_count,
> &dev_attr_comp_vector,
> &dev_attr_tl_retry_count,
> &dev_attr_cmd_sg_entries,
> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
> .proc_name = DRV_NAME,
> .slave_configure = srp_slave_configure,
> .info = srp_target_info,
> - .queuecommand = srp_queuecommand,
> + .queuecommand = srp_sq_queuecommand,
> + .mq_queuecommand = srp_mq_queuecommand,
Another choice is to obtain hctx from request directly, then mq can
reuse the .queuecommand interface too.
> .change_queue_depth = srp_change_queue_depth,
> .change_queue_type = srp_change_queue_type,
> .eh_abort_handler = srp_abort,
> @@ -3038,7 +3150,8 @@ static ssize_t srp_create_target(struct device *dev,
> struct srp_rdma_ch *ch;
> struct srp_device *srp_dev = host->srp_dev;
> struct ib_device *ibdev = srp_dev->dev;
> - int ret;
> + int ret, node_idx, node, cpu, i;
> + bool multich = false;
>
> target_host = scsi_host_alloc(&srp_template,
> sizeof (struct srp_target_port));
> @@ -3098,34 +3211,82 @@ static ssize_t srp_create_target(struct device *dev,
> INIT_WORK(&target->tl_err_work, srp_tl_err_work);
> INIT_WORK(&target->remove_work, srp_remove_work);
> spin_lock_init(&target->lock);
> - ch = &target->ch;
> - ch->target = target;
> - ch->comp_vector = target->comp_vector;
> - spin_lock_init(&ch->lock);
> - INIT_LIST_HEAD(&ch->free_tx);
> - ret = srp_alloc_req_data(ch);
> - if (ret)
> - goto err_free_mem;
> -
> ret = ib_query_gid(ibdev, host->port, 0, &target->sgid);
> if (ret)
> - goto err_free_mem;
> + goto err;
>
> - ret = srp_create_ch_ib(ch);
> - if (ret)
> - goto err_free_mem;
> + ret = -ENOMEM;
> + target->ch_count = max_t(unsigned, num_online_nodes(),
> + min(ch_count ? :
> + min(4 * num_online_nodes(),
> + ibdev->num_comp_vectors),
> + num_online_cpus()));
> + target->ch = kcalloc(target->ch_count, sizeof(*target->ch),
> + GFP_KERNEL);
> + if (!target->ch)
> + goto err;
>
> - ret = srp_new_cm_id(ch);
> - if (ret)
> - goto err_free_ib;
> + node_idx = 0;
> + for_each_online_node(node) {
> + const int ch_start = (node_idx * target->ch_count /
> + num_online_nodes());
> + const int ch_end = ((node_idx + 1) * target->ch_count /
> + num_online_nodes());
> + const int cv_start = (node_idx * ibdev->num_comp_vectors /
> + num_online_nodes() + target->comp_vector)
> + % ibdev->num_comp_vectors;
> + const int cv_end = ((node_idx + 1) * ibdev->num_comp_vectors /
> + num_online_nodes() + target->comp_vector)
> + % ibdev->num_comp_vectors;
> + int cpu_idx = 0;
> +
> + for_each_online_cpu(cpu) {
> + if (cpu_to_node(cpu) != node)
> + continue;
> + if (ch_start + cpu_idx >= ch_end)
> + continue;
> + ch = &target->ch[ch_start + cpu_idx];
> + ch->target = target;
> + ch->comp_vector = cv_start == cv_end ? cv_start :
> + cv_start + cpu_idx % (cv_end - cv_start);
> + spin_lock_init(&ch->lock);
> + INIT_LIST_HEAD(&ch->free_tx);
> + ret = srp_new_cm_id(ch);
> + if (ret)
> + goto err_disconnect;
>
> - ret = srp_connect_ch(ch);
> - if (ret) {
> - shost_printk(KERN_ERR, target->scsi_host,
> - PFX "Connection failed\n");
> - goto err_free_ib;
> + ret = srp_create_ch_ib(ch);
> + if (ret)
> + goto err_disconnect;
> +
> + ret = srp_alloc_req_data(ch);
> + if (ret)
> + goto err_disconnect;
> +
> + ret = srp_connect_ch(ch, multich);
> + if (ret) {
> + shost_printk(KERN_ERR, target->scsi_host,
> + PFX "Connection %d/%d failed\n",
> + ch_start + cpu_idx,
> + target->ch_count);
> + if (node_idx == 0 && cpu_idx == 0) {
> + goto err_disconnect;
> + } else {
> + srp_free_ch_ib(target, ch);
> + srp_free_req_data(target, ch);
> + target->ch_count = ch - target->ch;
> + break;
> + }
> + }
> +
> + multich = true;
> + cpu_idx++;
> + }
> + node_idx++;
> }
>
> + target->scsi_host->nr_hw_queues = target->ch_count;
> +
> ret = srp_add_target(host, target);
> if (ret)
> goto err_disconnect;
> @@ -3154,11 +3315,13 @@ out:
> err_disconnect:
> srp_disconnect_target(target);
>
> -err_free_ib:
> - srp_free_ch_ib(ch);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + srp_free_ch_ib(target, ch);
> + srp_free_req_data(target, ch);
> + }
>
> -err_free_mem:
> - srp_free_req_data(ch);
> + kfree(target->ch);
>
> err:
> scsi_host_put(target_host);
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
> index 0609124..d9660e1 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.h
> +++ b/drivers/infiniband/ulp/srp/ib_srp.h
> @@ -84,6 +84,21 @@ enum srp_iu_type {
> SRP_IU_RSP,
> };
>
> +static inline u32 build_srp_tag(u16 ch, u16 req_idx)
> +{
> + return ch << 16 | req_idx;
> +}
> +
> +static inline u16 srp_tag_ch(u32 tag)
> +{
> + return tag >> 16;
> +}
> +
> +static inline u16 srp_tag_idx(u32 tag)
> +{
> + return tag & ((1 << 16) - 1);
> +}
> +
> /*
> * @mr_page_mask: HCA memory registration page mask.
> * @mr_page_size: HCA memory registration page size.
> @@ -127,7 +142,7 @@ struct srp_request {
> struct srp_direct_buf *indirect_desc;
> dma_addr_t indirect_dma_addr;
> short nmdesc;
> - short index;
> + uint32_t tag;
> };
>
> struct srp_rdma_ch {
> @@ -173,8 +188,9 @@ struct srp_target_port {
> /* read and written in the hot path */
> spinlock_t lock;
>
> - struct srp_rdma_ch ch;
> /* read only in the hot path */
> + struct srp_rdma_ch *ch;
> + u32 ch_count;
> u32 lkey;
> u32 rkey;
> enum srp_target_state state;
> --
> 1.8.4.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <CACVXFVPzz37J-613NZCfPStUBxf0rLOtz71LJ07PpCxYg5nn+g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-09-19 15:21 ` Bart Van Assche
[not found] ` <541C49EC.6030404-HInyCGIudOg@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 15:21 UTC (permalink / raw)
To: Ming Lei
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Jens Axboe, Robert Elliott
On 09/19/14 16:28, Ming Lei wrote:
> On Fri, Sep 19, 2014 at 9:00 PM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org> wrote:
>> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
>> .proc_name = DRV_NAME,
>> .slave_configure = srp_slave_configure,
>> .info = srp_target_info,
>> - .queuecommand = srp_queuecommand,
>> + .queuecommand = srp_sq_queuecommand,
>> + .mq_queuecommand = srp_mq_queuecommand,
>
> Another choice is to obtain hctx from request directly, then mq can
> reuse the .queuecommand interface too.
Hello Ming,
Is the hctx information already available in the request data structure
? I have found a mq_ctx member but no hctx member. Did I perhaps
overlook something ?
Thanks,
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <541C49EC.6030404-HInyCGIudOg@public.gmane.org>
@ 2014-09-19 15:27 ` Ming Lei
2014-09-19 15:35 ` Bart Van Assche
0 siblings, 1 reply; 55+ messages in thread
From: Ming Lei @ 2014-09-19 15:27 UTC (permalink / raw)
To: Bart Van Assche
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Jens Axboe, Robert Elliott
On Fri, Sep 19, 2014 at 11:21 PM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org> wrote:
> On 09/19/14 16:28, Ming Lei wrote:
>>
>> On Fri, Sep 19, 2014 at 9:00 PM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
>> wrote:
>>>
>>> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
>>> .proc_name = DRV_NAME,
>>> .slave_configure = srp_slave_configure,
>>> .info = srp_target_info,
>>> - .queuecommand = srp_queuecommand,
>>> + .queuecommand = srp_sq_queuecommand,
>>> + .mq_queuecommand = srp_mq_queuecommand,
>>
>>
>> Another choice is to obtain hctx from request directly, then mq can
>> reuse the .queuecommand interface too.
>
>
> Hello Ming,
>
> Is the hctx information already available in the request data structure ? I
> have found a mq_ctx member but no hctx member. Did I perhaps overlook
> something ?
You are right, but the mq_ctx can be mapped to hctx like below way:
ctx = rq->mq_ctx;
hctx = q->mq_ops->map_queue(q, ctx->cpu);
Thanks,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-19 15:27 ` Ming Lei
@ 2014-09-19 15:35 ` Bart Van Assche
2014-09-19 15:38 ` Jens Axboe
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-19 15:35 UTC (permalink / raw)
To: Ming Lei
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott
On 09/19/14 17:27, Ming Lei wrote:
> On Fri, Sep 19, 2014 at 11:21 PM, Bart Van Assche <bvanassche@acm.org> wrote:
>> On 09/19/14 16:28, Ming Lei wrote:
>>>
>>> On Fri, Sep 19, 2014 at 9:00 PM, Bart Van Assche <bvanassche@acm.org>
>>> wrote:
>>>>
>>>> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
>>>> .proc_name = DRV_NAME,
>>>> .slave_configure = srp_slave_configure,
>>>> .info = srp_target_info,
>>>> - .queuecommand = srp_queuecommand,
>>>> + .queuecommand = srp_sq_queuecommand,
>>>> + .mq_queuecommand = srp_mq_queuecommand,
>>>
>>>
>>> Another choice is to obtain hctx from request directly, then mq can
>>> reuse the .queuecommand interface too.
>>
>>
>> Hello Ming,
>>
>> Is the hctx information already available in the request data structure ? I
>> have found a mq_ctx member but no hctx member. Did I perhaps overlook
>> something ?
>
> You are right, but the mq_ctx can be mapped to hctx like below way:
>
> ctx = rq->mq_ctx;
> hctx = q->mq_ops->map_queue(q, ctx->cpu);
Hello Ming,
Had you already noticed that the blk_mq_ctx data structure is a private
data structure (declared in block/blk-mq.h) and hence not available to
SCSI LLDs ? However, what might be possible is to cache the hctx pointer
in the request structure, e.g. like in the (completely untested) patch
below.
[PATCH] blk-mq: Cache hardware context pointer in struct request
Cache the hardware context pointer in struct request such that block
drivers can determine which hardware queue has been selected by
reading rq->mq_hctx->queue_num. This information is needed to select
the appropriate hardware queue in e.g. SCSI LLDs.
---
block/blk-flush.c | 6 +-----
block/blk-mq.c | 16 +++++++++-------
include/linux/blkdev.h | 1 +
3 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/block/blk-flush.c b/block/blk-flush.c
index 3cb5e9e..698812d 100644
--- a/block/blk-flush.c
+++ b/block/blk-flush.c
@@ -327,13 +327,9 @@ static void flush_data_end_io(struct request *rq, int error)
static void mq_flush_data_end_io(struct request *rq, int error)
{
struct request_queue *q = rq->q;
- struct blk_mq_hw_ctx *hctx;
- struct blk_mq_ctx *ctx;
+ struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
unsigned long flags;
- ctx = rq->mq_ctx;
- hctx = q->mq_ops->map_queue(q, ctx->cpu);
-
/*
* After populating an empty queue, kick it to avoid stall. Read
* the comment in flush_end_io().
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 383ea0c..8e428fe 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -210,6 +210,7 @@ __blk_mq_alloc_request(struct blk_mq_alloc_data *data, int rw)
}
rq->tag = tag;
+ rq->mq_hctx = data->hctx;
blk_mq_rq_ctx_init(data->q, data->ctx, rq, rw);
return rq;
}
@@ -267,12 +268,10 @@ static void __blk_mq_free_request(struct blk_mq_hw_ctx *hctx,
void blk_mq_free_request(struct request *rq)
{
struct blk_mq_ctx *ctx = rq->mq_ctx;
- struct blk_mq_hw_ctx *hctx;
- struct request_queue *q = rq->q;
+ struct blk_mq_hw_ctx *hctx = rq->mq_hctx;
ctx->rq_completed[rq_is_sync(rq)]++;
- hctx = q->mq_ops->map_queue(q, ctx->cpu);
__blk_mq_free_request(hctx, ctx, rq);
}
@@ -287,10 +286,10 @@ void blk_mq_free_request(struct request *rq)
void blk_mq_clone_flush_request(struct request *flush_rq,
struct request *orig_rq)
{
- struct blk_mq_hw_ctx *hctx =
- orig_rq->q->mq_ops->map_queue(orig_rq->q, orig_rq->mq_ctx->cpu);
+ struct blk_mq_hw_ctx *hctx = orig_rq->mq_hctx;
flush_rq->mq_ctx = orig_rq->mq_ctx;
+ flush_rq->mq_hctx = hctx;
flush_rq->tag = orig_rq->tag;
memcpy(blk_mq_rq_to_pdu(flush_rq), blk_mq_rq_to_pdu(orig_rq),
hctx->cmd_size);
@@ -956,6 +955,7 @@ void blk_mq_insert_request(struct request *rq, bool at_head, bool run_queue,
rq->mq_ctx = ctx = current_ctx;
hctx = q->mq_ops->map_queue(q, ctx->cpu);
+ rq->mq_hctx = hctx;
if (rq->cmd_flags & (REQ_FLUSH | REQ_FUA) &&
!(rq->cmd_flags & (REQ_FLUSH_SEQ))) {
@@ -1001,6 +1001,7 @@ static void blk_mq_insert_requests(struct request_queue *q,
rq = list_first_entry(list, struct request, queuelist);
list_del_init(&rq->queuelist);
rq->mq_ctx = ctx;
+ rq->mq_hctx = hctx;
__blk_mq_insert_request(hctx, rq, false);
}
spin_unlock(&ctx->lock);
@@ -1475,6 +1476,8 @@ static int blk_mq_hctx_cpu_offline(struct blk_mq_hw_ctx *hctx, int cpu)
return NOTIFY_OK;
ctx = blk_mq_get_ctx(q);
+ hctx = q->mq_ops->map_queue(q, ctx->cpu);
+
spin_lock(&ctx->lock);
while (!list_empty(&tmp)) {
@@ -1482,10 +1485,9 @@ static int blk_mq_hctx_cpu_offline(struct blk_mq_hw_ctx *hctx, int cpu)
rq = list_first_entry(&tmp, struct request, queuelist);
rq->mq_ctx = ctx;
+ rq->mq_hctx = hctx;
list_move_tail(&rq->queuelist, &ctx->rq_list);
}
-
- hctx = q->mq_ops->map_queue(q, ctx->cpu);
blk_mq_hctx_mark_pending(hctx, ctx);
spin_unlock(&ctx->lock);
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 518b465..e3a14a2 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -105,6 +105,7 @@ struct request {
struct request_queue *q;
struct blk_mq_ctx *mq_ctx;
+ struct blk_mq_hw_ctx *mq_hctx;
u64 cmd_flags;
enum rq_cmd_type_bits cmd_type;
--
1.8.4.5
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-19 15:35 ` Bart Van Assche
@ 2014-09-19 15:38 ` Jens Axboe
2014-09-19 17:30 ` Sagi Grimberg
[not found] ` <541C4DF1.4090604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
0 siblings, 2 replies; 55+ messages in thread
From: Jens Axboe @ 2014-09-19 15:38 UTC (permalink / raw)
To: Bart Van Assche, Ming Lei
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Robert Elliott
On 09/19/2014 09:35 AM, Bart Van Assche wrote:
> On 09/19/14 17:27, Ming Lei wrote:
>> On Fri, Sep 19, 2014 at 11:21 PM, Bart Van Assche <bvanassche@acm.org> wrote:
>>> On 09/19/14 16:28, Ming Lei wrote:
>>>>
>>>> On Fri, Sep 19, 2014 at 9:00 PM, Bart Van Assche <bvanassche@acm.org>
>>>> wrote:
>>>>>
>>>>> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
>>>>> .proc_name = DRV_NAME,
>>>>> .slave_configure = srp_slave_configure,
>>>>> .info = srp_target_info,
>>>>> - .queuecommand = srp_queuecommand,
>>>>> + .queuecommand = srp_sq_queuecommand,
>>>>> + .mq_queuecommand = srp_mq_queuecommand,
>>>>
>>>>
>>>> Another choice is to obtain hctx from request directly, then mq can
>>>> reuse the .queuecommand interface too.
>>>
>>>
>>> Hello Ming,
>>>
>>> Is the hctx information already available in the request data structure ? I
>>> have found a mq_ctx member but no hctx member. Did I perhaps overlook
>>> something ?
>>
>> You are right, but the mq_ctx can be mapped to hctx like below way:
>>
>> ctx = rq->mq_ctx;
>> hctx = q->mq_ops->map_queue(q, ctx->cpu);
>
> Hello Ming,
>
> Had you already noticed that the blk_mq_ctx data structure is a private
> data structure (declared in block/blk-mq.h) and hence not available to
> SCSI LLDs ? However, what might be possible is to cache the hctx pointer
> in the request structure, e.g. like in the (completely untested) patch
> below.
ctx was meant to be private, unfortunately it's leaked a bit into other
parts of block/. But it's still private within that, at least.
Lets not add more stuff to struct request, it's already way too large.
We could add an exported
struct blk_mq_hw_ctx *blk_mq_request_to_hctx(struct request *rq)
{
struct request_queue *q = rq->q;
return q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
}
for this.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-19 15:38 ` Jens Axboe
@ 2014-09-19 17:30 ` Sagi Grimberg
2014-09-19 17:33 ` Jens Axboe
[not found] ` <541C4DF1.4090604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
1 sibling, 1 reply; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-19 17:30 UTC (permalink / raw)
To: Jens Axboe, Bart Van Assche, Ming Lei
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Robert Elliott
On 9/19/2014 6:38 PM, Jens Axboe wrote:
> On 09/19/2014 09:35 AM, Bart Van Assche wrote:
>> On 09/19/14 17:27, Ming Lei wrote:
>>> On Fri, Sep 19, 2014 at 11:21 PM, Bart Van Assche <bvanassche@acm.org> wrote:
>>>> On 09/19/14 16:28, Ming Lei wrote:
>>>>>
>>>>> On Fri, Sep 19, 2014 at 9:00 PM, Bart Van Assche <bvanassche@acm.org>
>>>>> wrote:
>>>>>>
>>>>>> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
>>>>>> .proc_name = DRV_NAME,
>>>>>> .slave_configure = srp_slave_configure,
>>>>>> .info = srp_target_info,
>>>>>> - .queuecommand = srp_queuecommand,
>>>>>> + .queuecommand = srp_sq_queuecommand,
>>>>>> + .mq_queuecommand = srp_mq_queuecommand,
>>>>>
>>>>>
>>>>> Another choice is to obtain hctx from request directly, then mq can
>>>>> reuse the .queuecommand interface too.
>>>>
>>>>
>>>> Hello Ming,
>>>>
>>>> Is the hctx information already available in the request data structure ? I
>>>> have found a mq_ctx member but no hctx member. Did I perhaps overlook
>>>> something ?
>>>
>>> You are right, but the mq_ctx can be mapped to hctx like below way:
>>>
>>> ctx = rq->mq_ctx;
>>> hctx = q->mq_ops->map_queue(q, ctx->cpu);
>>
>> Hello Ming,
>>
>> Had you already noticed that the blk_mq_ctx data structure is a private
>> data structure (declared in block/blk-mq.h) and hence not available to
>> SCSI LLDs ? However, what might be possible is to cache the hctx pointer
>> in the request structure, e.g. like in the (completely untested) patch
>> below.
>
> ctx was meant to be private, unfortunately it's leaked a bit into other
> parts of block/. But it's still private within that, at least.
>
> Lets not add more stuff to struct request, it's already way too large.
> We could add an exported
>
> struct blk_mq_hw_ctx *blk_mq_request_to_hctx(struct request *rq)
> {
> struct request_queue *q = rq->q;
>
> return q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
> }
>
> for this.
>
Hey,
So I agree that we shouldn't overload struct request with another
pointer, but it also seems a bit unnecessary to map again just to get
the hctx. Since in the future we would like LLDs to use scsi-mq why not
modify existing .queuecommand to pass hctx (or even better
hctx->driver_data) and for now LLDs won't use it. Once they choose to,
it will be available to them.
Sagi.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-19 17:30 ` Sagi Grimberg
@ 2014-09-19 17:33 ` Jens Axboe
2014-09-19 18:11 ` Christoph Hellwig
0 siblings, 1 reply; 55+ messages in thread
From: Jens Axboe @ 2014-09-19 17:33 UTC (permalink / raw)
To: Sagi Grimberg, Bart Van Assche, Ming Lei
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Robert Elliott
On 09/19/2014 11:30 AM, Sagi Grimberg wrote:
> On 9/19/2014 6:38 PM, Jens Axboe wrote:
>> On 09/19/2014 09:35 AM, Bart Van Assche wrote:
>>> On 09/19/14 17:27, Ming Lei wrote:
>>>> On Fri, Sep 19, 2014 at 11:21 PM, Bart Van Assche <bvanassche@acm.org> wrote:
>>>>> On 09/19/14 16:28, Ming Lei wrote:
>>>>>>
>>>>>> On Fri, Sep 19, 2014 at 9:00 PM, Bart Van Assche <bvanassche@acm.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
>>>>>>> .proc_name = DRV_NAME,
>>>>>>> .slave_configure = srp_slave_configure,
>>>>>>> .info = srp_target_info,
>>>>>>> - .queuecommand = srp_queuecommand,
>>>>>>> + .queuecommand = srp_sq_queuecommand,
>>>>>>> + .mq_queuecommand = srp_mq_queuecommand,
>>>>>>
>>>>>>
>>>>>> Another choice is to obtain hctx from request directly, then mq can
>>>>>> reuse the .queuecommand interface too.
>>>>>
>>>>>
>>>>> Hello Ming,
>>>>>
>>>>> Is the hctx information already available in the request data structure ? I
>>>>> have found a mq_ctx member but no hctx member. Did I perhaps overlook
>>>>> something ?
>>>>
>>>> You are right, but the mq_ctx can be mapped to hctx like below way:
>>>>
>>>> ctx = rq->mq_ctx;
>>>> hctx = q->mq_ops->map_queue(q, ctx->cpu);
>>>
>>> Hello Ming,
>>>
>>> Had you already noticed that the blk_mq_ctx data structure is a private
>>> data structure (declared in block/blk-mq.h) and hence not available to
>>> SCSI LLDs ? However, what might be possible is to cache the hctx pointer
>>> in the request structure, e.g. like in the (completely untested) patch
>>> below.
>>
>> ctx was meant to be private, unfortunately it's leaked a bit into other
>> parts of block/. But it's still private within that, at least.
>>
>> Lets not add more stuff to struct request, it's already way too large.
>> We could add an exported
>>
>> struct blk_mq_hw_ctx *blk_mq_request_to_hctx(struct request *rq)
>> {
>> struct request_queue *q = rq->q;
>>
>> return q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
>> }
>>
>> for this.
>>
>
> Hey,
>
> So I agree that we shouldn't overload struct request with another
> pointer, but it also seems a bit unnecessary to map again just to get
> the hctx. Since in the future we would like LLDs to use scsi-mq why not
> modify existing .queuecommand to pass hctx (or even better
> hctx->driver_data) and for now LLDs won't use it. Once they choose to,
> it will be available to them.
That'd be fine as well. The mapping is cheap, though, but it would make
sense to have an appropriate way to just pass it in like it happens for
->queue_rq() for native blk-mq drivers.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH RFC 0/8] IB/srp: Add multichannel support
2014-09-19 12:56 ` Bart Van Assche
@ 2014-09-19 18:02 ` Sagi Grimberg
0 siblings, 0 replies; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-19 18:02 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/19/2014 3:56 PM, Bart Van Assche wrote:
> [PATCH 1/8] blk-mq: Use all available hardware queues
>
> Suppose that a system has two CPU sockets, three cores per socket,
> that it does not support hyperthreading and that four hardware
> queues are provided by a block driver. With the current algorithm
> this will lead to the following assignment of CPU cores to hardware
> queues:
>
> HWQ 0: 0 1
> HWQ 1: 2 3
> HWQ 2: 4 5
> HWQ 3: (none)
>
> This patch changes the queue assignment into:
>
> HWQ 0: 0 1
> HWQ 1: 2
> HWQ 2: 3 4
> HWQ 3: 5
>
> In other words, this patch has the following three effects:
> - All four hardware queues are used instead of only three.
> - CPU cores are spread more evenly over hardware queues. For the
> above example the range of the number of CPU cores associated
> with a single HWQ is reduced from [0..2] to [1..2].
> - If the number of HWQ's is a multiple of the number of CPU sockets
> it is now guaranteed that all CPU cores associated with a single
> HWQ reside on the same CPU socket.
>
> Signed-off-by: Bart Van Assche <bvanassche@acm.org>
> Cc: Jens Axboe <axboe@fb.com>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Ming Lei <ming.lei@canonical.com>
> ---
> block/blk-mq-cpumap.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/block/blk-mq-cpumap.c b/block/blk-mq-cpumap.c
> index 1065d7c..8e56455 100644
> --- a/block/blk-mq-cpumap.c
> +++ b/block/blk-mq-cpumap.c
> @@ -17,7 +17,7 @@
> static int cpu_to_queue_index(unsigned int nr_cpus, unsigned int nr_queues,
> const int cpu)
> {
> - return cpu / ((nr_cpus + nr_queues - 1) / nr_queues);
> + return cpu * nr_queues / nr_cpus;
> }
>
> static int get_first_sibling(unsigned int cpu)
>
Seems reasonable enough.
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 2/8] scsi-mq: Add support for multiple hardware queues
[not found] ` <541C281E.9090206-HInyCGIudOg@public.gmane.org>
@ 2014-09-19 18:05 ` Sagi Grimberg
2014-09-19 18:11 ` Christoph Hellwig
[not found] ` <CAF9gx6JfP2bGyMauB6LzepZP_vKEvrd-sPZc5CRuOrtgQ_UCSw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 2 replies; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-19 18:05 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/19/2014 3:57 PM, Bart Van Assche wrote:
> Allow a SCSI LLD to declare how many hardware queues it supports
> by setting Scsi_Host.nr_hw_queues before calling scsi_add_host().
>
> Note: it is assumed that each hardware queue has a queue depth of
> shost->can_queue. In other words, the total queue depth per host
> is (number of hardware queues) * (shost->can_queue).
>
> Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
> Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
> ---
> drivers/scsi/scsi_lib.c | 2 +-
> include/scsi/scsi_host.h | 4 ++++
> 2 files changed, 5 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
> index d837dc1..b0b6117 100644
> --- a/drivers/scsi/scsi_lib.c
> +++ b/drivers/scsi/scsi_lib.c
> @@ -2071,7 +2071,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
>
> memset(&shost->tag_set, 0, sizeof(shost->tag_set));
> shost->tag_set.ops = &scsi_mq_ops;
> - shost->tag_set.nr_hw_queues = 1;
> + shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
> shost->tag_set.queue_depth = shost->can_queue;
> shost->tag_set.cmd_size = cmd_size;
> shost->tag_set.numa_node = NUMA_NO_NODE;
> diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
> index ba20347..0a867d9 100644
> --- a/include/scsi/scsi_host.h
> +++ b/include/scsi/scsi_host.h
> @@ -638,6 +638,10 @@ struct Scsi_Host {
> short unsigned int sg_prot_tablesize;
> unsigned int max_sectors;
> unsigned long dma_boundary;
> + /*
> + * In scsi-mq mode, the number of hardware queues supported by the LLD.
> + */
> + unsigned nr_hw_queues;
> /*
> * Used to assign serial numbers to the cmds.
> * Protected by the host lock.
>
I think this patch should be squashed with passing LLD hctx patch (in
whatever form it ends up).
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 4/8] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib()
[not found] ` <541C285B.5010309-HInyCGIudOg@public.gmane.org>
@ 2014-09-19 18:10 ` Sagi Grimberg
0 siblings, 0 replies; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-19 18:10 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/19/2014 3:58 PM, Bart Van Assche wrote:
> The patch that adds multichannel support into the SRP initiator
> driver introduces an additional call to srp_free_ch_ib(). This
> patch helps to keep that later patch simple.
>
> Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
> ---
> drivers/infiniband/ulp/srp/ib_srp.c | 11 ++++++-----
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index 62d2a18..d3c712f 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -555,6 +555,11 @@ static void srp_free_target_ib(struct srp_target_port *target)
> struct srp_device *dev = target->srp_host->srp_dev;
> int i;
>
> + if (target->cm_id) {
> + ib_destroy_cm_id(target->cm_id);
> + target->cm_id = NULL;
> + }
> +
> if (dev->use_fast_reg) {
> if (target->fr_pool)
> srp_destroy_fr_pool(target->fr_pool);
> @@ -868,7 +873,6 @@ static void srp_remove_target(struct srp_target_port *target)
> scsi_remove_host(target->scsi_host);
> srp_stop_rport_timers(target->rport);
> srp_disconnect_target(target);
> - ib_destroy_cm_id(target->cm_id);
> srp_free_target_ib(target);
> cancel_work_sync(&target->tl_err_work);
> srp_rport_put(target->rport);
> @@ -3043,7 +3047,7 @@ static ssize_t srp_create_target(struct device *dev,
> if (ret) {
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Connection failed\n");
> - goto err_cm_id;
> + goto err_free_ib;
> }
>
> ret = srp_add_target(host, target);
> @@ -3067,9 +3071,6 @@ out:
> err_disconnect:
> srp_disconnect_target(target);
>
> -err_cm_id:
> - ib_destroy_cm_id(target->cm_id);
> -
> err_free_ib:
> srp_free_target_ib(target);
>
>
Looks good.
Reviewed-by: Sagi Grimberg <sagig-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-19 17:33 ` Jens Axboe
@ 2014-09-19 18:11 ` Christoph Hellwig
0 siblings, 0 replies; 55+ messages in thread
From: Christoph Hellwig @ 2014-09-19 18:11 UTC (permalink / raw)
To: Jens Axboe
Cc: Sagi Grimberg, Bart Van Assche, Ming Lei, linux-scsi, linux-rdma,
Robert Elliott
On Fri, Sep 19, 2014 at 11:33:15AM -0600, Jens Axboe wrote:
> That'd be fine as well. The mapping is cheap, though, but it would make
> sense to have an appropriate way to just pass it in like it happens for
> ->queue_rq() for native blk-mq drivers.
I think just passing the hw_ctx is fine. But I don't want drivers to
have to implement both methods, so we should make sure the new method
works for both the blk-mq and !blk-mq case. Note that once thing the
new method should get is a bool last paramter similar to the one I
added to the queue_rq method in the block tree tree.
It might be worth it to simply do a global search and replace and pass
a hw_ctx method to all instances, too.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 2/8] scsi-mq: Add support for multiple hardware queues
2014-09-19 18:05 ` Sagi Grimberg
@ 2014-09-19 18:11 ` Christoph Hellwig
[not found] ` <CAF9gx6JfP2bGyMauB6LzepZP_vKEvrd-sPZc5CRuOrtgQ_UCSw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
1 sibling, 0 replies; 55+ messages in thread
From: Christoph Hellwig @ 2014-09-19 18:11 UTC (permalink / raw)
To: Sagi Grimberg
Cc: Bart Van Assche, linux-scsi, linux-rdma, Christoph Hellwig,
Jens Axboe, Robert Elliott, Ming Lei
On Fri, Sep 19, 2014 at 09:05:53PM +0300, Sagi Grimberg wrote:
> I think this patch should be squashed with passing LLD hctx patch (in
> whatever form it ends up).
Agreed.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 5/8] IB/srp: Remove stale connection retry mechanism
2014-09-19 12:58 ` [PATCH 5/8] IB/srp: Remove stale connection retry mechanism Bart Van Assche
@ 2014-09-19 18:25 ` Sagi Grimberg
[not found] ` <541C287D.1050900-HInyCGIudOg@public.gmane.org>
1 sibling, 0 replies; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-19 18:25 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/19/2014 3:58 PM, Bart Van Assche wrote:
> Attempting to connect three times may be insufficient after an
> initiator system that was using multiple RDMA channels tries to
> relogin. Additionally, this login retry mechanism is a workaround
> for particular behavior of the IB/CM. Since the srp_daemon retries
> a failed login attempt anyway, remove the stale connection retry
> mechanism.
>
I agree, this work-around doesn't even always work for some targets. In
case the RDMA stack restarted while IB ports were down and srp initiator
didn't manage to teardown the connections, some targets are keeping
cm_ids online returning "stale connection" rejects longer then expected.
let user-space retry from scratch...
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
> Signed-off-by: Bart Van Assche <bvanassche@acm.org>
> ---
> drivers/infiniband/ulp/srp/ib_srp.c | 16 +++-------------
> 1 file changed, 3 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index d3c712f..9608e7a 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -904,7 +904,6 @@ static void srp_rport_delete(struct srp_rport *rport)
>
> static int srp_connect_target(struct srp_target_port *target)
> {
> - int retries = 3;
> int ret;
>
> WARN_ON_ONCE(target->connected);
> @@ -945,19 +944,10 @@ static int srp_connect_target(struct srp_target_port *target)
> break;
>
> case SRP_STALE_CONN:
> - /* Our current CM id was stale, and is now in timewait.
> - * Try to reconnect with a new one.
> - */
> - if (!retries-- || srp_new_cm_id(target)) {
> - shost_printk(KERN_ERR, target->scsi_host, PFX
> - "giving up on stale connection\n");
> - target->status = -ECONNRESET;
> - return target->status;
> - }
> -
> shost_printk(KERN_ERR, target->scsi_host, PFX
> - "retrying stale connection\n");
> - break;
> + "giving up on stale connection\n");
> + target->status = -ECONNRESET;
> + return target->status;
>
> default:
> return target->status;
>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH RFC 0/8] IB/srp: Add multichannel support
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
` (5 preceding siblings ...)
2014-09-19 12:59 ` [PATCH 7/8] IB/srp: Separate target and channel variables Bart Van Assche
@ 2014-09-19 18:31 ` Jens Axboe
2014-09-22 14:37 ` Christoph Hellwig
7 siblings, 0 replies; 55+ messages in thread
From: Jens Axboe @ 2014-09-19 18:31 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Robert Elliott, Ming Lei
On 09/19/2014 06:55 AM, Bart Van Assche wrote:
> Hello,
>
> Although the SRP protocol supports multichannel operation, although
> since considerable time RDMA HCA's are available that support multiple
> completion vectors and although multichannel operation yields better
> performance than using a single channel, the Linux SRP initiator does
> not yet support multichannel operation. While adding multichannel
> support in the SRP initiator I encountered a few challenges of which I
> think these need wider discussion. The topics I would like invite wider
> discussion about are as follows:
> - How to avoid unneeded inter-socket cache traffic. Should the blk-mq
> layer e.g. assign CPU cores to hardware queues such that all CPU cores
> associated with a single hardware queue reside on the same CPU socket?
> (see also patch 1/8)
Right now these are deliberately symmetric, and hence can result in
hardware queues being left unused. Whether it makes sense to make this
change or not, I think will greatly depend on how many queues are
available. There's a crossover point where having more queues doesn't
help, and it can even make things worse (interrupt load, etc). For your
specific case or 4 queues, it probably DOES make sense to use them all,
however.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 7/8] IB/srp: Separate target and channel variables
2014-09-19 12:59 ` [PATCH 7/8] IB/srp: Separate target and channel variables Bart Van Assche
@ 2014-09-19 18:47 ` Sagi Grimberg
[not found] ` <541C28C8.7000007-HInyCGIudOg@public.gmane.org>
1 sibling, 0 replies; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-19 18:47 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/19/2014 3:59 PM, Bart Van Assche wrote:
> Changes in this patch:
> - Move channel variables into a new structure (struct srp_rdma_ch).
> - cm_id and completion handler context pointer are now of type
> srp_rdma_ch * insteoad of srp_target_port *.
>
s/insteoad/instead
> No functionality is changed.
>
errr... long patch....
I'll review it more thoroughly later...
> Signed-off-by: Bart Van Assche <bvanassche@acm.org>
> ---
> drivers/infiniband/ulp/srp/ib_srp.c | 707 +++++++++++++++++++-----------------
> drivers/infiniband/ulp/srp/ib_srp.h | 59 +--
> 2 files changed, 417 insertions(+), 349 deletions(-)
>
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index fd88fb8..9feeea1 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -125,8 +125,8 @@ MODULE_PARM_DESC(dev_loss_tmo,
>
> static void srp_add_one(struct ib_device *device);
> static void srp_remove_one(struct ib_device *device);
> -static void srp_recv_completion(struct ib_cq *cq, void *target_ptr);
> -static void srp_send_completion(struct ib_cq *cq, void *target_ptr);
> +static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr);
> +static void srp_send_completion(struct ib_cq *cq, void *ch_ptr);
> static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event);
>
> static struct scsi_transport_template *ib_srp_transport_template;
> @@ -262,7 +262,7 @@ static int srp_init_qp(struct srp_target_port *target,
>
> ret = ib_find_pkey(target->srp_host->srp_dev->dev,
> target->srp_host->port,
> - be16_to_cpu(target->path.pkey),
> + be16_to_cpu(target->pkey),
> &attr->pkey_index);
> if (ret)
> goto out;
> @@ -283,18 +283,23 @@ out:
> return ret;
> }
>
> -static int srp_new_cm_id(struct srp_target_port *target)
> +static int srp_new_cm_id(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_cm_id *new_cm_id;
>
> new_cm_id = ib_create_cm_id(target->srp_host->srp_dev->dev,
> - srp_cm_handler, target);
> + srp_cm_handler, ch);
> if (IS_ERR(new_cm_id))
> return PTR_ERR(new_cm_id);
>
> - if (target->cm_id)
> - ib_destroy_cm_id(target->cm_id);
> - target->cm_id = new_cm_id;
> + if (ch->cm_id)
> + ib_destroy_cm_id(ch->cm_id);
> + ch->cm_id = new_cm_id;
> + ch->path.sgid = target->sgid;
> + ch->path.dgid = target->orig_dgid;
> + ch->path.pkey = target->pkey;
> + ch->path.service_id = target->service_id;
>
> return 0;
> }
> @@ -443,8 +448,9 @@ static struct srp_fr_pool *srp_alloc_fr_pool(struct srp_target_port *target)
> dev->max_pages_per_mr);
> }
>
> -static int srp_create_target_ib(struct srp_target_port *target)
> +static int srp_create_ch_ib(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_qp_init_attr *init_attr;
> struct ib_cq *recv_cq, *send_cq;
> @@ -458,15 +464,15 @@ static int srp_create_target_ib(struct srp_target_port *target)
> if (!init_attr)
> return -ENOMEM;
>
> - recv_cq = ib_create_cq(dev->dev, srp_recv_completion, NULL, target,
> - target->queue_size, target->comp_vector);
> + recv_cq = ib_create_cq(dev->dev, srp_recv_completion, NULL, ch,
> + target->queue_size, ch->comp_vector);
> if (IS_ERR(recv_cq)) {
> ret = PTR_ERR(recv_cq);
> goto err;
> }
>
> - send_cq = ib_create_cq(dev->dev, srp_send_completion, NULL, target,
> - m * target->queue_size, target->comp_vector);
> + send_cq = ib_create_cq(dev->dev, srp_send_completion, NULL, ch,
> + m * target->queue_size, ch->comp_vector);
> if (IS_ERR(send_cq)) {
> ret = PTR_ERR(send_cq);
> goto err_recv_cq;
> @@ -502,9 +508,9 @@ static int srp_create_target_ib(struct srp_target_port *target)
> "FR pool allocation failed (%d)\n", ret);
> goto err_qp;
> }
> - if (target->fr_pool)
> - srp_destroy_fr_pool(target->fr_pool);
> - target->fr_pool = fr_pool;
> + if (ch->fr_pool)
> + srp_destroy_fr_pool(ch->fr_pool);
> + ch->fr_pool = fr_pool;
> } else if (!dev->use_fast_reg && dev->has_fmr) {
> fmr_pool = srp_alloc_fmr_pool(target);
> if (IS_ERR(fmr_pool)) {
> @@ -513,21 +519,21 @@ static int srp_create_target_ib(struct srp_target_port *target)
> "FMR pool allocation failed (%d)\n", ret);
> goto err_qp;
> }
> - if (target->fmr_pool)
> - ib_destroy_fmr_pool(target->fmr_pool);
> - target->fmr_pool = fmr_pool;
> + if (ch->fmr_pool)
> + ib_destroy_fmr_pool(ch->fmr_pool);
> + ch->fmr_pool = fmr_pool;
> }
>
> - if (target->qp)
> - ib_destroy_qp(target->qp);
> - if (target->recv_cq)
> - ib_destroy_cq(target->recv_cq);
> - if (target->send_cq)
> - ib_destroy_cq(target->send_cq);
> + if (ch->qp)
> + ib_destroy_qp(ch->qp);
> + if (ch->recv_cq)
> + ib_destroy_cq(ch->recv_cq);
> + if (ch->send_cq)
> + ib_destroy_cq(ch->send_cq);
>
> - target->qp = qp;
> - target->recv_cq = recv_cq;
> - target->send_cq = send_cq;
> + ch->qp = qp;
> + ch->recv_cq = recv_cq;
> + ch->send_cq = send_cq;
>
> kfree(init_attr);
> return 0;
> @@ -548,98 +554,102 @@ err:
>
> /*
> * Note: this function may be called without srp_alloc_iu_bufs() having been
> - * invoked. Hence the target->[rt]x_ring checks.
> + * invoked. Hence the ch->[rt]x_ring checks.
> */
> -static void srp_free_target_ib(struct srp_target_port *target)
> +static void srp_free_ch_ib(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> int i;
>
> - if (target->cm_id) {
> - ib_destroy_cm_id(target->cm_id);
> - target->cm_id = NULL;
> + if (ch->cm_id) {
> + ib_destroy_cm_id(ch->cm_id);
> + ch->cm_id = NULL;
> }
>
> if (dev->use_fast_reg) {
> - if (target->fr_pool)
> - srp_destroy_fr_pool(target->fr_pool);
> + if (ch->fr_pool)
> + srp_destroy_fr_pool(ch->fr_pool);
> } else {
> - if (target->fmr_pool)
> - ib_destroy_fmr_pool(target->fmr_pool);
> + if (ch->fmr_pool)
> + ib_destroy_fmr_pool(ch->fmr_pool);
> }
> - ib_destroy_qp(target->qp);
> - ib_destroy_cq(target->send_cq);
> - ib_destroy_cq(target->recv_cq);
> + ib_destroy_qp(ch->qp);
> + ib_destroy_cq(ch->send_cq);
> + ib_destroy_cq(ch->recv_cq);
>
> - target->qp = NULL;
> - target->send_cq = target->recv_cq = NULL;
> + ch->qp = NULL;
> + ch->send_cq = ch->recv_cq = NULL;
>
> - if (target->rx_ring) {
> + if (ch->rx_ring) {
> for (i = 0; i < target->queue_size; ++i)
> - srp_free_iu(target->srp_host, target->rx_ring[i]);
> - kfree(target->rx_ring);
> - target->rx_ring = NULL;
> + srp_free_iu(target->srp_host, ch->rx_ring[i]);
> + kfree(ch->rx_ring);
> + ch->rx_ring = NULL;
> }
> - if (target->tx_ring) {
> + if (ch->tx_ring) {
> for (i = 0; i < target->queue_size; ++i)
> - srp_free_iu(target->srp_host, target->tx_ring[i]);
> - kfree(target->tx_ring);
> - target->tx_ring = NULL;
> + srp_free_iu(target->srp_host, ch->tx_ring[i]);
> + kfree(ch->tx_ring);
> + ch->tx_ring = NULL;
> }
> }
>
> static void srp_path_rec_completion(int status,
> struct ib_sa_path_rec *pathrec,
> - void *target_ptr)
> + void *ch_ptr)
> {
> - struct srp_target_port *target = target_ptr;
> + struct srp_rdma_ch *ch = ch_ptr;
> + struct srp_target_port *target = ch->target;
>
> - target->status = status;
> + ch->status = status;
> if (status)
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Got failed path rec status %d\n", status);
> else
> - target->path = *pathrec;
> - complete(&target->done);
> + ch->path = *pathrec;
> + complete(&ch->done);
> }
>
> -static int srp_lookup_path(struct srp_target_port *target)
> +static int srp_lookup_path(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int ret;
>
> - target->path.numb_path = 1;
> -
> - init_completion(&target->done);
> -
> - target->path_query_id = ib_sa_path_rec_get(&srp_sa_client,
> - target->srp_host->srp_dev->dev,
> - target->srp_host->port,
> - &target->path,
> - IB_SA_PATH_REC_SERVICE_ID |
> - IB_SA_PATH_REC_DGID |
> - IB_SA_PATH_REC_SGID |
> - IB_SA_PATH_REC_NUMB_PATH |
> - IB_SA_PATH_REC_PKEY,
> - SRP_PATH_REC_TIMEOUT_MS,
> - GFP_KERNEL,
> - srp_path_rec_completion,
> - target, &target->path_query);
> - if (target->path_query_id < 0)
> - return target->path_query_id;
> -
> - ret = wait_for_completion_interruptible(&target->done);
> + ch->path.numb_path = 1;
> +
> + init_completion(&ch->done);
> +
> + ch->path_query_id = ib_sa_path_rec_get(&srp_sa_client,
> + target->srp_host->srp_dev->dev,
> + target->srp_host->port,
> + &ch->path,
> + IB_SA_PATH_REC_SERVICE_ID |
> + IB_SA_PATH_REC_DGID |
> + IB_SA_PATH_REC_SGID |
> + IB_SA_PATH_REC_NUMB_PATH |
> + IB_SA_PATH_REC_PKEY,
> + SRP_PATH_REC_TIMEOUT_MS,
> + GFP_KERNEL,
> + srp_path_rec_completion,
> + ch, &ch->path_query);
> + if (ch->path_query_id < 0)
> + return ch->path_query_id;
> +
> + ret = wait_for_completion_interruptible(&ch->done);
> if (ret < 0)
> return ret;
>
> - if (target->status < 0)
> + if (ch->status < 0)
> shost_printk(KERN_WARNING, target->scsi_host,
> PFX "Path record query failed\n");
>
> - return target->status;
> + return ch->status;
> }
>
> -static int srp_send_req(struct srp_target_port *target)
> +static int srp_send_req(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct {
> struct ib_cm_req_param param;
> struct srp_login_req priv;
> @@ -650,11 +660,11 @@ static int srp_send_req(struct srp_target_port *target)
> if (!req)
> return -ENOMEM;
>
> - req->param.primary_path = &target->path;
> + req->param.primary_path = &ch->path;
> req->param.alternate_path = NULL;
> req->param.service_id = target->service_id;
> - req->param.qp_num = target->qp->qp_num;
> - req->param.qp_type = target->qp->qp_type;
> + req->param.qp_num = ch->qp->qp_num;
> + req->param.qp_type = ch->qp->qp_type;
> req->param.private_data = &req->priv;
> req->param.private_data_len = sizeof req->priv;
> req->param.flow_control = 1;
> @@ -689,7 +699,7 @@ static int srp_send_req(struct srp_target_port *target)
> */
> if (target->io_class == SRP_REV10_IB_IO_CLASS) {
> memcpy(req->priv.initiator_port_id,
> - &target->path.sgid.global.interface_id, 8);
> + &target->sgid.global.interface_id, 8);
> memcpy(req->priv.initiator_port_id + 8,
> &target->initiator_ext, 8);
> memcpy(req->priv.target_port_id, &target->ioc_guid, 8);
> @@ -698,7 +708,7 @@ static int srp_send_req(struct srp_target_port *target)
> memcpy(req->priv.initiator_port_id,
> &target->initiator_ext, 8);
> memcpy(req->priv.initiator_port_id + 8,
> - &target->path.sgid.global.interface_id, 8);
> + &target->sgid.global.interface_id, 8);
> memcpy(req->priv.target_port_id, &target->id_ext, 8);
> memcpy(req->priv.target_port_id + 8, &target->ioc_guid, 8);
> }
> @@ -718,7 +728,7 @@ static int srp_send_req(struct srp_target_port *target)
> &target->srp_host->srp_dev->dev->node_guid, 8);
> }
>
> - status = ib_send_cm_req(target->cm_id, &req->param);
> + status = ib_send_cm_req(ch->cm_id, &req->param);
>
> kfree(req);
>
> @@ -759,28 +769,31 @@ static bool srp_change_conn_state(struct srp_target_port *target,
>
> static void srp_disconnect_target(struct srp_target_port *target)
> {
> + struct srp_rdma_ch *ch = &target->ch;
> +
> if (srp_change_conn_state(target, false)) {
> /* XXX should send SRP_I_LOGOUT request */
>
> - if (ib_send_cm_dreq(target->cm_id, NULL, 0)) {
> + if (ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
> shost_printk(KERN_DEBUG, target->scsi_host,
> PFX "Sending CM DREQ failed\n");
> }
> }
> }
>
> -static void srp_free_req_data(struct srp_target_port *target)
> +static void srp_free_req_data(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> struct srp_request *req;
> int i;
>
> - if (!target->req_ring)
> + if (!ch->req_ring)
> return;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - req = &target->req_ring[i];
> + req = &ch->req_ring[i];
> if (dev->use_fast_reg)
> kfree(req->fr_list);
> else
> @@ -794,12 +807,13 @@ static void srp_free_req_data(struct srp_target_port *target)
> kfree(req->indirect_desc);
> }
>
> - kfree(target->req_ring);
> - target->req_ring = NULL;
> + kfree(ch->req_ring);
> + ch->req_ring = NULL;
> }
>
> -static int srp_alloc_req_data(struct srp_target_port *target)
> +static int srp_alloc_req_data(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *srp_dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = srp_dev->dev;
> struct srp_request *req;
> @@ -807,15 +821,15 @@ static int srp_alloc_req_data(struct srp_target_port *target)
> dma_addr_t dma_addr;
> int i, ret = -ENOMEM;
>
> - INIT_LIST_HEAD(&target->free_reqs);
> + INIT_LIST_HEAD(&ch->free_reqs);
>
> - target->req_ring = kzalloc(target->req_ring_size *
> - sizeof(*target->req_ring), GFP_KERNEL);
> - if (!target->req_ring)
> + ch->req_ring = kcalloc(target->req_ring_size, sizeof(*ch->req_ring),
> + GFP_KERNEL);
> + if (!ch->req_ring)
> goto out;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - req = &target->req_ring[i];
> + req = &ch->req_ring[i];
> mr_list = kmalloc(target->cmd_sg_cnt * sizeof(void *),
> GFP_KERNEL);
> if (!mr_list)
> @@ -840,7 +854,7 @@ static int srp_alloc_req_data(struct srp_target_port *target)
>
> req->indirect_dma_addr = dma_addr;
> req->index = i;
> - list_add_tail(&req->list, &target->free_reqs);
> + list_add_tail(&req->list, &ch->free_reqs);
> }
> ret = 0;
>
> @@ -865,6 +879,8 @@ static void srp_del_scsi_host_attr(struct Scsi_Host *shost)
>
> static void srp_remove_target(struct srp_target_port *target)
> {
> + struct srp_rdma_ch *ch = &target->ch;
> +
> WARN_ON_ONCE(target->state != SRP_TARGET_REMOVED);
>
> srp_del_scsi_host_attr(target->scsi_host);
> @@ -873,10 +889,10 @@ static void srp_remove_target(struct srp_target_port *target)
> scsi_remove_host(target->scsi_host);
> srp_stop_rport_timers(target->rport);
> srp_disconnect_target(target);
> - srp_free_target_ib(target);
> + srp_free_ch_ib(ch);
> cancel_work_sync(&target->tl_err_work);
> srp_rport_put(target->rport);
> - srp_free_req_data(target);
> + srp_free_req_data(ch);
>
> spin_lock(&target->srp_host->target_lock);
> list_del(&target->list);
> @@ -902,24 +918,25 @@ static void srp_rport_delete(struct srp_rport *rport)
> srp_queue_remove_work(target);
> }
>
> -static int srp_connect_target(struct srp_target_port *target)
> +static int srp_connect_ch(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int ret;
>
> WARN_ON_ONCE(target->connected);
>
> target->qp_in_error = false;
>
> - ret = srp_lookup_path(target);
> + ret = srp_lookup_path(ch);
> if (ret)
> return ret;
>
> while (1) {
> - init_completion(&target->done);
> - ret = srp_send_req(target);
> + init_completion(&ch->done);
> + ret = srp_send_req(ch);
> if (ret)
> return ret;
> - ret = wait_for_completion_interruptible(&target->done);
> + ret = wait_for_completion_interruptible(&ch->done);
> if (ret < 0)
> return ret;
>
> @@ -929,13 +946,13 @@ static int srp_connect_target(struct srp_target_port *target)
> * back, or SRP_DLID_REDIRECT if we get a lid/qp
> * redirect REJ back.
> */
> - switch (target->status) {
> + switch (ch->status) {
> case 0:
> srp_change_conn_state(target, true);
> return 0;
>
> case SRP_PORT_REDIRECT:
> - ret = srp_lookup_path(target);
> + ret = srp_lookup_path(ch);
> if (ret)
> return ret;
> break;
> @@ -946,16 +963,16 @@ static int srp_connect_target(struct srp_target_port *target)
> case SRP_STALE_CONN:
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "giving up on stale connection\n");
> - target->status = -ECONNRESET;
> - return target->status;
> + ch->status = -ECONNRESET;
> + return ch->status;
>
> default:
> - return target->status;
> + return ch->status;
> }
> }
> }
>
> -static int srp_inv_rkey(struct srp_target_port *target, u32 rkey)
> +static int srp_inv_rkey(struct srp_rdma_ch *ch, u32 rkey)
> {
> struct ib_send_wr *bad_wr;
> struct ib_send_wr wr = {
> @@ -967,13 +984,14 @@ static int srp_inv_rkey(struct srp_target_port *target, u32 rkey)
> .ex.invalidate_rkey = rkey,
> };
>
> - return ib_post_send(target->qp, &wr, &bad_wr);
> + return ib_post_send(ch->qp, &wr, &bad_wr);
> }
>
> static void srp_unmap_data(struct scsi_cmnd *scmnd,
> - struct srp_target_port *target,
> + struct srp_rdma_ch *ch,
> struct srp_request *req)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> int i, res;
> @@ -987,7 +1005,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
> struct srp_fr_desc **pfr;
>
> for (i = req->nmdesc, pfr = req->fr_list; i > 0; i--, pfr++) {
> - res = srp_inv_rkey(target, (*pfr)->mr->rkey);
> + res = srp_inv_rkey(ch, (*pfr)->mr->rkey);
> if (res < 0) {
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "Queueing INV WR for rkey %#x failed (%d)\n",
> @@ -997,7 +1015,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
> }
> }
> if (req->nmdesc)
> - srp_fr_pool_put(target->fr_pool, req->fr_list,
> + srp_fr_pool_put(ch->fr_pool, req->fr_list,
> req->nmdesc);
> } else {
> struct ib_pool_fmr **pfmr;
> @@ -1012,7 +1030,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
>
> /**
> * srp_claim_req - Take ownership of the scmnd associated with a request.
> - * @target: SRP target port.
> + * @ch: SRP RDMA channel.
> * @req: SRP request.
> * @sdev: If not NULL, only take ownership for this SCSI device.
> * @scmnd: If NULL, take ownership of @req->scmnd. If not NULL, only take
> @@ -1021,14 +1039,14 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
> * Return value:
> * Either NULL or a pointer to the SCSI command the caller became owner of.
> */
> -static struct scsi_cmnd *srp_claim_req(struct srp_target_port *target,
> +static struct scsi_cmnd *srp_claim_req(struct srp_rdma_ch *ch,
> struct srp_request *req,
> struct scsi_device *sdev,
> struct scsi_cmnd *scmnd)
> {
> unsigned long flags;
>
> - spin_lock_irqsave(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> if (req->scmnd &&
> (!sdev || req->scmnd->device == sdev) &&
> (!scmnd || req->scmnd == scmnd)) {
> @@ -1037,40 +1055,38 @@ static struct scsi_cmnd *srp_claim_req(struct srp_target_port *target,
> } else {
> scmnd = NULL;
> }
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> return scmnd;
> }
>
> /**
> * srp_free_req() - Unmap data and add request to the free request list.
> - * @target: SRP target port.
> + * @ch: SRP RDMA channel.
> * @req: Request to be freed.
> * @scmnd: SCSI command associated with @req.
> * @req_lim_delta: Amount to be added to @target->req_lim.
> */
> -static void srp_free_req(struct srp_target_port *target,
> - struct srp_request *req, struct scsi_cmnd *scmnd,
> - s32 req_lim_delta)
> +static void srp_free_req(struct srp_rdma_ch *ch, struct srp_request *req,
> + struct scsi_cmnd *scmnd, s32 req_lim_delta)
> {
> unsigned long flags;
>
> - srp_unmap_data(scmnd, target, req);
> + srp_unmap_data(scmnd, ch, req);
>
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += req_lim_delta;
> - list_add_tail(&req->list, &target->free_reqs);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += req_lim_delta;
> + list_add_tail(&req->list, &ch->free_reqs);
> + spin_unlock_irqrestore(&ch->lock, flags);
> }
>
> -static void srp_finish_req(struct srp_target_port *target,
> - struct srp_request *req, struct scsi_device *sdev,
> - int result)
> +static void srp_finish_req(struct srp_rdma_ch *ch, struct srp_request *req,
> + struct scsi_device *sdev, int result)
> {
> - struct scsi_cmnd *scmnd = srp_claim_req(target, req, sdev, NULL);
> + struct scsi_cmnd *scmnd = srp_claim_req(ch, req, sdev, NULL);
>
> if (scmnd) {
> - srp_free_req(target, req, scmnd, 0);
> + srp_free_req(ch, req, scmnd, 0);
> scmnd->result = result;
> scmnd->scsi_done(scmnd);
> }
> @@ -1079,6 +1095,7 @@ static void srp_finish_req(struct srp_target_port *target,
> static void srp_terminate_io(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> + struct srp_rdma_ch *ch = &target->ch;
> struct Scsi_Host *shost = target->scsi_host;
> struct scsi_device *sdev;
> int i;
> @@ -1091,8 +1108,9 @@ static void srp_terminate_io(struct srp_rport *rport)
> WARN_ON_ONCE(sdev->request_queue->request_fn_active);
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &target->req_ring[i];
> - srp_finish_req(target, req, NULL, DID_TRANSPORT_FAILFAST << 16);
> + struct srp_request *req = &ch->req_ring[i];
> +
> + srp_finish_req(ch, req, NULL, DID_TRANSPORT_FAILFAST << 16);
> }
> }
>
> @@ -1108,6 +1126,7 @@ static void srp_terminate_io(struct srp_rport *rport)
> static int srp_rport_reconnect(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> + struct srp_rdma_ch *ch = &target->ch;
> int i, ret;
>
> srp_disconnect_target(target);
> @@ -1120,11 +1139,12 @@ static int srp_rport_reconnect(struct srp_rport *rport)
> * case things are really fouled up. Doing so also ensures that all CM
> * callbacks will have finished before a new QP is allocated.
> */
> - ret = srp_new_cm_id(target);
> + ret = srp_new_cm_id(ch);
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &target->req_ring[i];
> - srp_finish_req(target, req, NULL, DID_RESET << 16);
> + struct srp_request *req = &ch->req_ring[i];
> +
> + srp_finish_req(ch, req, NULL, DID_RESET << 16);
> }
>
> /*
> @@ -1132,14 +1152,14 @@ static int srp_rport_reconnect(struct srp_rport *rport)
> * QP. This guarantees that all callback functions for the old QP have
> * finished before any send requests are posted on the new QP.
> */
> - ret += srp_create_target_ib(target);
> + ret += srp_create_ch_ib(ch);
>
> - INIT_LIST_HEAD(&target->free_tx);
> + INIT_LIST_HEAD(&ch->free_tx);
> for (i = 0; i < target->queue_size; ++i)
> - list_add(&target->tx_ring[i]->list, &target->free_tx);
> + list_add(&ch->tx_ring[i]->list, &ch->free_tx);
>
> if (ret == 0)
> - ret = srp_connect_target(target);
> + ret = srp_connect_ch(ch);
>
> if (ret == 0)
> shost_printk(KERN_INFO, target->scsi_host,
> @@ -1163,12 +1183,12 @@ static void srp_map_desc(struct srp_map_state *state, dma_addr_t dma_addr,
> }
>
> static int srp_map_finish_fmr(struct srp_map_state *state,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> struct ib_pool_fmr *fmr;
> u64 io_addr = 0;
>
> - fmr = ib_fmr_pool_map_phys(target->fmr_pool, state->pages,
> + fmr = ib_fmr_pool_map_phys(ch->fmr_pool, state->pages,
> state->npages, io_addr);
> if (IS_ERR(fmr))
> return PTR_ERR(fmr);
> @@ -1182,15 +1202,16 @@ static int srp_map_finish_fmr(struct srp_map_state *state,
> }
>
> static int srp_map_finish_fr(struct srp_map_state *state,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_send_wr *bad_wr;
> struct ib_send_wr wr;
> struct srp_fr_desc *desc;
> u32 rkey;
>
> - desc = srp_fr_pool_get(target->fr_pool);
> + desc = srp_fr_pool_get(ch->fr_pool);
> if (!desc)
> return -ENOMEM;
>
> @@ -1219,12 +1240,13 @@ static int srp_map_finish_fr(struct srp_map_state *state,
> srp_map_desc(state, state->base_dma_addr, state->dma_len,
> desc->mr->rkey);
>
> - return ib_post_send(target->qp, &wr, &bad_wr);
> + return ib_post_send(ch->qp, &wr, &bad_wr);
> }
>
> static int srp_finish_mapping(struct srp_map_state *state,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int ret = 0;
>
> if (state->npages == 0)
> @@ -1235,8 +1257,8 @@ static int srp_finish_mapping(struct srp_map_state *state,
> target->rkey);
> else
> ret = target->srp_host->srp_dev->use_fast_reg ?
> - srp_map_finish_fr(state, target) :
> - srp_map_finish_fmr(state, target);
> + srp_map_finish_fr(state, ch) :
> + srp_map_finish_fmr(state, ch);
>
> if (ret == 0) {
> state->npages = 0;
> @@ -1256,10 +1278,11 @@ static void srp_map_update_start(struct srp_map_state *state,
> }
>
> static int srp_map_sg_entry(struct srp_map_state *state,
> - struct srp_target_port *target,
> + struct srp_rdma_ch *ch,
> struct scatterlist *sg, int sg_index,
> bool use_mr)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> dma_addr_t dma_addr = ib_sg_dma_address(ibdev, sg);
> @@ -1288,7 +1311,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
> */
> if ((!dev->use_fast_reg && dma_addr & ~dev->mr_page_mask) ||
> dma_len > dev->mr_max_size) {
> - ret = srp_finish_mapping(state, target);
> + ret = srp_finish_mapping(state, ch);
> if (ret)
> return ret;
>
> @@ -1309,7 +1332,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
> while (dma_len) {
> unsigned offset = dma_addr & ~dev->mr_page_mask;
> if (state->npages == dev->max_pages_per_mr || offset != 0) {
> - ret = srp_finish_mapping(state, target);
> + ret = srp_finish_mapping(state, ch);
> if (ret)
> return ret;
>
> @@ -1333,17 +1356,18 @@ static int srp_map_sg_entry(struct srp_map_state *state,
> */
> ret = 0;
> if (len != dev->mr_page_size) {
> - ret = srp_finish_mapping(state, target);
> + ret = srp_finish_mapping(state, ch);
> if (!ret)
> srp_map_update_start(state, NULL, 0, 0);
> }
> return ret;
> }
>
> -static int srp_map_sg(struct srp_map_state *state,
> - struct srp_target_port *target, struct srp_request *req,
> - struct scatterlist *scat, int count)
> +static int srp_map_sg(struct srp_map_state *state, struct srp_rdma_ch *ch,
> + struct srp_request *req, struct scatterlist *scat,
> + int count)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> struct scatterlist *sg;
> @@ -1354,14 +1378,14 @@ static int srp_map_sg(struct srp_map_state *state,
> state->pages = req->map_page;
> if (dev->use_fast_reg) {
> state->next_fr = req->fr_list;
> - use_mr = !!target->fr_pool;
> + use_mr = !!ch->fr_pool;
> } else {
> state->next_fmr = req->fmr_list;
> - use_mr = !!target->fmr_pool;
> + use_mr = !!ch->fmr_pool;
> }
>
> for_each_sg(scat, sg, count, i) {
> - if (srp_map_sg_entry(state, target, sg, i, use_mr)) {
> + if (srp_map_sg_entry(state, ch, sg, i, use_mr)) {
> /*
> * Memory registration failed, so backtrack to the
> * first unmapped entry and continue on without using
> @@ -1383,7 +1407,7 @@ backtrack:
> }
> }
>
> - if (use_mr && srp_finish_mapping(state, target))
> + if (use_mr && srp_finish_mapping(state, ch))
> goto backtrack;
>
> req->nmdesc = state->nmdesc;
> @@ -1391,9 +1415,10 @@ backtrack:
> return 0;
> }
>
> -static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port *target,
> +static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
> struct srp_request *req)
> {
> + struct srp_target_port *target = ch->target;
> struct scatterlist *scat;
> struct srp_cmd *cmd = req->cmd->buf;
> int len, nents, count;
> @@ -1455,7 +1480,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port *target,
> target->indirect_size, DMA_TO_DEVICE);
>
> memset(&state, 0, sizeof(state));
> - srp_map_sg(&state, target, req, scat, count);
> + srp_map_sg(&state, ch, req, scat, count);
>
> /* We've mapped the request, now pull as much of the indirect
> * descriptor table as we can into the command buffer. If this
> @@ -1516,20 +1541,20 @@ map_complete:
> /*
> * Return an IU and possible credit to the free pool
> */
> -static void srp_put_tx_iu(struct srp_target_port *target, struct srp_iu *iu,
> +static void srp_put_tx_iu(struct srp_rdma_ch *ch, struct srp_iu *iu,
> enum srp_iu_type iu_type)
> {
> unsigned long flags;
>
> - spin_lock_irqsave(&target->lock, flags);
> - list_add(&iu->list, &target->free_tx);
> + spin_lock_irqsave(&ch->lock, flags);
> + list_add(&iu->list, &ch->free_tx);
> if (iu_type != SRP_IU_RSP)
> - ++target->req_lim;
> - spin_unlock_irqrestore(&target->lock, flags);
> + ++ch->req_lim;
> + spin_unlock_irqrestore(&ch->lock, flags);
> }
>
> /*
> - * Must be called with target->lock held to protect req_lim and free_tx.
> + * Must be called with ch->lock held to protect req_lim and free_tx.
> * If IU is not sent, it must be returned using srp_put_tx_iu().
> *
> * Note:
> @@ -1541,35 +1566,36 @@ static void srp_put_tx_iu(struct srp_target_port *target, struct srp_iu *iu,
> * - SRP_IU_RSP: 1, since a conforming SRP target never sends more than
> * one unanswered SRP request to an initiator.
> */
> -static struct srp_iu *__srp_get_tx_iu(struct srp_target_port *target,
> +static struct srp_iu *__srp_get_tx_iu(struct srp_rdma_ch *ch,
> enum srp_iu_type iu_type)
> {
> + struct srp_target_port *target = ch->target;
> s32 rsv = (iu_type == SRP_IU_TSK_MGMT) ? 0 : SRP_TSK_MGMT_SQ_SIZE;
> struct srp_iu *iu;
>
> - srp_send_completion(target->send_cq, target);
> + srp_send_completion(ch->send_cq, target);
>
> - if (list_empty(&target->free_tx))
> + if (list_empty(&ch->free_tx))
> return NULL;
>
> /* Initiator responses to target requests do not consume credits */
> if (iu_type != SRP_IU_RSP) {
> - if (target->req_lim <= rsv) {
> + if (ch->req_lim <= rsv) {
> ++target->zero_req_lim;
> return NULL;
> }
>
> - --target->req_lim;
> + --ch->req_lim;
> }
>
> - iu = list_first_entry(&target->free_tx, struct srp_iu, list);
> + iu = list_first_entry(&ch->free_tx, struct srp_iu, list);
> list_del(&iu->list);
> return iu;
> }
>
> -static int srp_post_send(struct srp_target_port *target,
> - struct srp_iu *iu, int len)
> +static int srp_post_send(struct srp_rdma_ch *ch, struct srp_iu *iu, int len)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_sge list;
> struct ib_send_wr wr, *bad_wr;
>
> @@ -1584,11 +1610,12 @@ static int srp_post_send(struct srp_target_port *target,
> wr.opcode = IB_WR_SEND;
> wr.send_flags = IB_SEND_SIGNALED;
>
> - return ib_post_send(target->qp, &wr, &bad_wr);
> + return ib_post_send(ch->qp, &wr, &bad_wr);
> }
>
> -static int srp_post_recv(struct srp_target_port *target, struct srp_iu *iu)
> +static int srp_post_recv(struct srp_rdma_ch *ch, struct srp_iu *iu)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_recv_wr wr, *bad_wr;
> struct ib_sge list;
>
> @@ -1601,35 +1628,36 @@ static int srp_post_recv(struct srp_target_port *target, struct srp_iu *iu)
> wr.sg_list = &list;
> wr.num_sge = 1;
>
> - return ib_post_recv(target->qp, &wr, &bad_wr);
> + return ib_post_recv(ch->qp, &wr, &bad_wr);
> }
>
> -static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
> +static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_request *req;
> struct scsi_cmnd *scmnd;
> unsigned long flags;
>
> if (unlikely(rsp->tag & SRP_TAG_TSK_MGMT)) {
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += be32_to_cpu(rsp->req_lim_delta);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> - target->tsk_mgmt_status = -1;
> + ch->tsk_mgmt_status = -1;
> if (be32_to_cpu(rsp->resp_data_len) >= 4)
> - target->tsk_mgmt_status = rsp->data[3];
> - complete(&target->tsk_mgmt_done);
> + ch->tsk_mgmt_status = rsp->data[3];
> + complete(&ch->tsk_mgmt_done);
> } else {
> - req = &target->req_ring[rsp->tag];
> - scmnd = srp_claim_req(target, req, NULL, NULL);
> + req = &ch->req_ring[rsp->tag];
> + scmnd = srp_claim_req(ch, req, NULL, NULL);
> if (!scmnd) {
> shost_printk(KERN_ERR, target->scsi_host,
> "Null scmnd for RSP w/tag %016llx\n",
> (unsigned long long) rsp->tag);
>
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += be32_to_cpu(rsp->req_lim_delta);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> return;
> }
> @@ -1651,7 +1679,7 @@ static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
> else if (unlikely(rsp->flags & SRP_RSP_FLAG_DOOVER))
> scsi_set_resid(scmnd, -be32_to_cpu(rsp->data_out_res_cnt));
>
> - srp_free_req(target, req, scmnd,
> + srp_free_req(ch, req, scmnd,
> be32_to_cpu(rsp->req_lim_delta));
>
> scmnd->host_scribble = NULL;
> @@ -1659,18 +1687,19 @@ static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
> }
> }
>
> -static int srp_response_common(struct srp_target_port *target, s32 req_delta,
> +static int srp_response_common(struct srp_rdma_ch *ch, s32 req_delta,
> void *rsp, int len)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_device *dev = target->srp_host->srp_dev->dev;
> unsigned long flags;
> struct srp_iu *iu;
> int err;
>
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += req_delta;
> - iu = __srp_get_tx_iu(target, SRP_IU_RSP);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += req_delta;
> + iu = __srp_get_tx_iu(ch, SRP_IU_RSP);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> if (!iu) {
> shost_printk(KERN_ERR, target->scsi_host, PFX
> @@ -1682,17 +1711,17 @@ static int srp_response_common(struct srp_target_port *target, s32 req_delta,
> memcpy(iu->buf, rsp, len);
> ib_dma_sync_single_for_device(dev, iu->dma, len, DMA_TO_DEVICE);
>
> - err = srp_post_send(target, iu, len);
> + err = srp_post_send(ch, iu, len);
> if (err) {
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "unable to post response: %d\n", err);
> - srp_put_tx_iu(target, iu, SRP_IU_RSP);
> + srp_put_tx_iu(ch, iu, SRP_IU_RSP);
> }
>
> return err;
> }
>
> -static void srp_process_cred_req(struct srp_target_port *target,
> +static void srp_process_cred_req(struct srp_rdma_ch *ch,
> struct srp_cred_req *req)
> {
> struct srp_cred_rsp rsp = {
> @@ -1701,14 +1730,15 @@ static void srp_process_cred_req(struct srp_target_port *target,
> };
> s32 delta = be32_to_cpu(req->req_lim_delta);
>
> - if (srp_response_common(target, delta, &rsp, sizeof rsp))
> - shost_printk(KERN_ERR, target->scsi_host, PFX
> + if (srp_response_common(ch, delta, &rsp, sizeof(rsp)))
> + shost_printk(KERN_ERR, ch->target->scsi_host, PFX
> "problems processing SRP_CRED_REQ\n");
> }
>
> -static void srp_process_aer_req(struct srp_target_port *target,
> +static void srp_process_aer_req(struct srp_rdma_ch *ch,
> struct srp_aer_req *req)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_aer_rsp rsp = {
> .opcode = SRP_AER_RSP,
> .tag = req->tag,
> @@ -1718,19 +1748,20 @@ static void srp_process_aer_req(struct srp_target_port *target,
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "ignoring AER for LUN %llu\n", be64_to_cpu(req->lun));
>
> - if (srp_response_common(target, delta, &rsp, sizeof rsp))
> + if (srp_response_common(ch, delta, &rsp, sizeof(rsp)))
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "problems processing SRP_AER_REQ\n");
> }
>
> -static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
> +static void srp_handle_recv(struct srp_rdma_ch *ch, struct ib_wc *wc)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_device *dev = target->srp_host->srp_dev->dev;
> struct srp_iu *iu = (struct srp_iu *) (uintptr_t) wc->wr_id;
> int res;
> u8 opcode;
>
> - ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_ti_iu_len,
> + ib_dma_sync_single_for_cpu(dev, iu->dma, ch->max_ti_iu_len,
> DMA_FROM_DEVICE);
>
> opcode = *(u8 *) iu->buf;
> @@ -1744,15 +1775,15 @@ static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
>
> switch (opcode) {
> case SRP_RSP:
> - srp_process_rsp(target, iu->buf);
> + srp_process_rsp(ch, iu->buf);
> break;
>
> case SRP_CRED_REQ:
> - srp_process_cred_req(target, iu->buf);
> + srp_process_cred_req(ch, iu->buf);
> break;
>
> case SRP_AER_REQ:
> - srp_process_aer_req(target, iu->buf);
> + srp_process_aer_req(ch, iu->buf);
> break;
>
> case SRP_T_LOGOUT:
> @@ -1767,10 +1798,10 @@ static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
> break;
> }
>
> - ib_dma_sync_single_for_device(dev, iu->dma, target->max_ti_iu_len,
> + ib_dma_sync_single_for_device(dev, iu->dma, ch->max_ti_iu_len,
> DMA_FROM_DEVICE);
>
> - res = srp_post_recv(target, iu);
> + res = srp_post_recv(ch, iu);
> if (res != 0)
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Recv failed with error code %d\n", res);
> @@ -1815,33 +1846,35 @@ static void srp_handle_qp_err(u64 wr_id, enum ib_wc_status wc_status,
> target->qp_in_error = true;
> }
>
> -static void srp_recv_completion(struct ib_cq *cq, void *target_ptr)
> +static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr)
> {
> - struct srp_target_port *target = target_ptr;
> + struct srp_rdma_ch *ch = ch_ptr;
> struct ib_wc wc;
>
> ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
> while (ib_poll_cq(cq, 1, &wc) > 0) {
> if (likely(wc.status == IB_WC_SUCCESS)) {
> - srp_handle_recv(target, &wc);
> + srp_handle_recv(ch, &wc);
> } else {
> - srp_handle_qp_err(wc.wr_id, wc.status, false, target);
> + srp_handle_qp_err(wc.wr_id, wc.status, false,
> + ch->target);
> }
> }
> }
>
> -static void srp_send_completion(struct ib_cq *cq, void *target_ptr)
> +static void srp_send_completion(struct ib_cq *cq, void *ch_ptr)
> {
> - struct srp_target_port *target = target_ptr;
> + struct srp_rdma_ch *ch = ch_ptr;
> struct ib_wc wc;
> struct srp_iu *iu;
>
> while (ib_poll_cq(cq, 1, &wc) > 0) {
> if (likely(wc.status == IB_WC_SUCCESS)) {
> iu = (struct srp_iu *) (uintptr_t) wc.wr_id;
> - list_add(&iu->list, &target->free_tx);
> + list_add(&iu->list, &ch->free_tx);
> } else {
> - srp_handle_qp_err(wc.wr_id, wc.status, true, target);
> + srp_handle_qp_err(wc.wr_id, wc.status, true,
> + ch->target);
> }
> }
> }
> @@ -1850,6 +1883,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(shost);
> struct srp_rport *rport = target->rport;
> + struct srp_rdma_ch *ch;
> struct srp_request *req;
> struct srp_iu *iu;
> struct srp_cmd *cmd;
> @@ -1871,14 +1905,16 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> if (unlikely(scmnd->result))
> goto err;
>
> - spin_lock_irqsave(&target->lock, flags);
> - iu = __srp_get_tx_iu(target, SRP_IU_CMD);
> + ch = &target->ch;
> +
> + spin_lock_irqsave(&ch->lock, flags);
> + iu = __srp_get_tx_iu(ch, SRP_IU_CMD);
> if (!iu)
> goto err_unlock;
>
> - req = list_first_entry(&target->free_reqs, struct srp_request, list);
> + req = list_first_entry(&ch->free_reqs, struct srp_request, list);
> list_del(&req->list);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> dev = target->srp_host->srp_dev->dev;
> ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_iu_len,
> @@ -1897,7 +1933,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> req->scmnd = scmnd;
> req->cmd = iu;
>
> - len = srp_map_data(scmnd, target, req);
> + len = srp_map_data(scmnd, ch, req);
> if (len < 0) {
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Failed to map data (%d)\n", len);
> @@ -1915,7 +1951,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> ib_dma_sync_single_for_device(dev, iu->dma, target->max_iu_len,
> DMA_TO_DEVICE);
>
> - if (srp_post_send(target, iu, len)) {
> + if (srp_post_send(ch, iu, len)) {
> shost_printk(KERN_ERR, target->scsi_host, PFX "Send failed\n");
> goto err_unmap;
> }
> @@ -1929,10 +1965,10 @@ unlock_rport:
> return ret;
>
> err_unmap:
> - srp_unmap_data(scmnd, target, req);
> + srp_unmap_data(scmnd, ch, req);
>
> err_iu:
> - srp_put_tx_iu(target, iu, SRP_IU_CMD);
> + srp_put_tx_iu(ch, iu, SRP_IU_CMD);
>
> /*
> * Avoid that the loops that iterate over the request ring can
> @@ -1940,11 +1976,11 @@ err_iu:
> */
> req->scmnd = NULL;
>
> - spin_lock_irqsave(&target->lock, flags);
> - list_add(&req->list, &target->free_reqs);
> + spin_lock_irqsave(&ch->lock, flags);
> + list_add(&req->list, &ch->free_reqs);
>
> err_unlock:
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> err:
> if (scmnd->result) {
> @@ -1959,53 +1995,54 @@ err:
>
> /*
> * Note: the resources allocated in this function are freed in
> - * srp_free_target_ib().
> + * srp_free_ch_ib().
> */
> -static int srp_alloc_iu_bufs(struct srp_target_port *target)
> +static int srp_alloc_iu_bufs(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int i;
>
> - target->rx_ring = kzalloc(target->queue_size * sizeof(*target->rx_ring),
> - GFP_KERNEL);
> - if (!target->rx_ring)
> + ch->rx_ring = kcalloc(target->queue_size, sizeof(*ch->rx_ring),
> + GFP_KERNEL);
> + if (!ch->rx_ring)
> goto err_no_ring;
> - target->tx_ring = kzalloc(target->queue_size * sizeof(*target->tx_ring),
> - GFP_KERNEL);
> - if (!target->tx_ring)
> + ch->tx_ring = kcalloc(target->queue_size, sizeof(*ch->tx_ring),
> + GFP_KERNEL);
> + if (!ch->tx_ring)
> goto err_no_ring;
>
> for (i = 0; i < target->queue_size; ++i) {
> - target->rx_ring[i] = srp_alloc_iu(target->srp_host,
> - target->max_ti_iu_len,
> - GFP_KERNEL, DMA_FROM_DEVICE);
> - if (!target->rx_ring[i])
> + ch->rx_ring[i] = srp_alloc_iu(target->srp_host,
> + ch->max_ti_iu_len,
> + GFP_KERNEL, DMA_FROM_DEVICE);
> + if (!ch->rx_ring[i])
> goto err;
> }
>
> for (i = 0; i < target->queue_size; ++i) {
> - target->tx_ring[i] = srp_alloc_iu(target->srp_host,
> - target->max_iu_len,
> - GFP_KERNEL, DMA_TO_DEVICE);
> - if (!target->tx_ring[i])
> + ch->tx_ring[i] = srp_alloc_iu(target->srp_host,
> + target->max_iu_len,
> + GFP_KERNEL, DMA_TO_DEVICE);
> + if (!ch->tx_ring[i])
> goto err;
>
> - list_add(&target->tx_ring[i]->list, &target->free_tx);
> + list_add(&ch->tx_ring[i]->list, &ch->free_tx);
> }
>
> return 0;
>
> err:
> for (i = 0; i < target->queue_size; ++i) {
> - srp_free_iu(target->srp_host, target->rx_ring[i]);
> - srp_free_iu(target->srp_host, target->tx_ring[i]);
> + srp_free_iu(target->srp_host, ch->rx_ring[i]);
> + srp_free_iu(target->srp_host, ch->tx_ring[i]);
> }
>
>
> err_no_ring:
> - kfree(target->tx_ring);
> - target->tx_ring = NULL;
> - kfree(target->rx_ring);
> - target->rx_ring = NULL;
> + kfree(ch->tx_ring);
> + ch->tx_ring = NULL;
> + kfree(ch->rx_ring);
> + ch->rx_ring = NULL;
>
> return -ENOMEM;
> }
> @@ -2039,23 +2076,24 @@ static uint32_t srp_compute_rq_tmo(struct ib_qp_attr *qp_attr, int attr_mask)
>
> static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
> struct srp_login_rsp *lrsp,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_qp_attr *qp_attr = NULL;
> int attr_mask = 0;
> int ret;
> int i;
>
> if (lrsp->opcode == SRP_LOGIN_RSP) {
> - target->max_ti_iu_len = be32_to_cpu(lrsp->max_ti_iu_len);
> - target->req_lim = be32_to_cpu(lrsp->req_lim_delta);
> + ch->max_ti_iu_len = be32_to_cpu(lrsp->max_ti_iu_len);
> + ch->req_lim = be32_to_cpu(lrsp->req_lim_delta);
>
> /*
> * Reserve credits for task management so we don't
> * bounce requests back to the SCSI mid-layer.
> */
> target->scsi_host->can_queue
> - = min(target->req_lim - SRP_TSK_MGMT_SQ_SIZE,
> + = min(ch->req_lim - SRP_TSK_MGMT_SQ_SIZE,
> target->scsi_host->can_queue);
> target->scsi_host->cmd_per_lun
> = min_t(int, target->scsi_host->can_queue,
> @@ -2067,8 +2105,8 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
> goto error;
> }
>
> - if (!target->rx_ring) {
> - ret = srp_alloc_iu_bufs(target);
> + if (!ch->rx_ring) {
> + ret = srp_alloc_iu_bufs(ch);
> if (ret)
> goto error;
> }
> @@ -2083,13 +2121,14 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
> if (ret)
> goto error_free;
>
> - ret = ib_modify_qp(target->qp, qp_attr, attr_mask);
> + ret = ib_modify_qp(ch->qp, qp_attr, attr_mask);
> if (ret)
> goto error_free;
>
> for (i = 0; i < target->queue_size; i++) {
> - struct srp_iu *iu = target->rx_ring[i];
> - ret = srp_post_recv(target, iu);
> + struct srp_iu *iu = ch->rx_ring[i];
> +
> + ret = srp_post_recv(ch, iu);
> if (ret)
> goto error_free;
> }
> @@ -2101,7 +2140,7 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
>
> target->rq_tmo_jiffies = srp_compute_rq_tmo(qp_attr, attr_mask);
>
> - ret = ib_modify_qp(target->qp, qp_attr, attr_mask);
> + ret = ib_modify_qp(ch->qp, qp_attr, attr_mask);
> if (ret)
> goto error_free;
>
> @@ -2111,13 +2150,14 @@ error_free:
> kfree(qp_attr);
>
> error:
> - target->status = ret;
> + ch->status = ret;
> }
>
> static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> struct ib_cm_event *event,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct Scsi_Host *shost = target->scsi_host;
> struct ib_class_port_info *cpi;
> int opcode;
> @@ -2125,12 +2165,12 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> switch (event->param.rej_rcvd.reason) {
> case IB_CM_REJ_PORT_CM_REDIRECT:
> cpi = event->param.rej_rcvd.ari;
> - target->path.dlid = cpi->redirect_lid;
> - target->path.pkey = cpi->redirect_pkey;
> + ch->path.dlid = cpi->redirect_lid;
> + ch->path.pkey = cpi->redirect_pkey;
> cm_id->remote_cm_qpn = be32_to_cpu(cpi->redirect_qp) & 0x00ffffff;
> - memcpy(target->path.dgid.raw, cpi->redirect_gid, 16);
> + memcpy(ch->path.dgid.raw, cpi->redirect_gid, 16);
>
> - target->status = target->path.dlid ?
> + ch->status = ch->path.dlid ?
> SRP_DLID_REDIRECT : SRP_PORT_REDIRECT;
> break;
>
> @@ -2141,26 +2181,26 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> * reject reason code 25 when they mean 24
> * (port redirect).
> */
> - memcpy(target->path.dgid.raw,
> + memcpy(ch->path.dgid.raw,
> event->param.rej_rcvd.ari, 16);
>
> shost_printk(KERN_DEBUG, shost,
> PFX "Topspin/Cisco redirect to target port GID %016llx%016llx\n",
> - (unsigned long long) be64_to_cpu(target->path.dgid.global.subnet_prefix),
> - (unsigned long long) be64_to_cpu(target->path.dgid.global.interface_id));
> + be64_to_cpu(ch->path.dgid.global.subnet_prefix),
> + be64_to_cpu(ch->path.dgid.global.interface_id));
>
> - target->status = SRP_PORT_REDIRECT;
> + ch->status = SRP_PORT_REDIRECT;
> } else {
> shost_printk(KERN_WARNING, shost,
> " REJ reason: IB_CM_REJ_PORT_REDIRECT\n");
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> }
> break;
>
> case IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID:
> shost_printk(KERN_WARNING, shost,
> " REJ reason: IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID\n");
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> break;
>
> case IB_CM_REJ_CONSUMER_DEFINED:
> @@ -2175,30 +2215,31 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> else
> shost_printk(KERN_WARNING, shost, PFX
> "SRP LOGIN from %pI6 to %pI6 REJECTED, reason 0x%08x\n",
> - target->path.sgid.raw,
> - target->orig_dgid, reason);
> + target->sgid.raw,
> + target->orig_dgid.raw, reason);
> } else
> shost_printk(KERN_WARNING, shost,
> " REJ reason: IB_CM_REJ_CONSUMER_DEFINED,"
> " opcode 0x%02x\n", opcode);
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> break;
>
> case IB_CM_REJ_STALE_CONN:
> shost_printk(KERN_WARNING, shost, " REJ reason: stale connection\n");
> - target->status = SRP_STALE_CONN;
> + ch->status = SRP_STALE_CONN;
> break;
>
> default:
> shost_printk(KERN_WARNING, shost, " REJ reason 0x%x\n",
> event->param.rej_rcvd.reason);
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> }
> }
>
> static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> {
> - struct srp_target_port *target = cm_id->context;
> + struct srp_rdma_ch *ch = cm_id->context;
> + struct srp_target_port *target = ch->target;
> int comp = 0;
>
> switch (event->event) {
> @@ -2206,19 +2247,19 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> shost_printk(KERN_DEBUG, target->scsi_host,
> PFX "Sending CM REQ failed\n");
> comp = 1;
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> break;
>
> case IB_CM_REP_RECEIVED:
> comp = 1;
> - srp_cm_rep_handler(cm_id, event->private_data, target);
> + srp_cm_rep_handler(cm_id, event->private_data, ch);
> break;
>
> case IB_CM_REJ_RECEIVED:
> shost_printk(KERN_DEBUG, target->scsi_host, PFX "REJ received\n");
> comp = 1;
>
> - srp_cm_rej_handler(cm_id, event, target);
> + srp_cm_rej_handler(cm_id, event, ch);
> break;
>
> case IB_CM_DREQ_RECEIVED:
> @@ -2236,7 +2277,7 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> PFX "connection closed\n");
> comp = 1;
>
> - target->status = 0;
> + ch->status = 0;
> break;
>
> case IB_CM_MRA_RECEIVED:
> @@ -2251,7 +2292,7 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> }
>
> if (comp)
> - complete(&target->done);
> + complete(&ch->done);
>
> return 0;
> }
> @@ -2307,9 +2348,10 @@ srp_change_queue_depth(struct scsi_device *sdev, int qdepth, int reason)
> return sdev->queue_depth;
> }
>
> -static int srp_send_tsk_mgmt(struct srp_target_port *target,
> - u64 req_tag, unsigned int lun, u8 func)
> +static int srp_send_tsk_mgmt(struct srp_rdma_ch *ch, u64 req_tag,
> + unsigned int lun, u8 func)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_rport *rport = target->rport;
> struct ib_device *dev = target->srp_host->srp_dev->dev;
> struct srp_iu *iu;
> @@ -2318,16 +2360,16 @@ static int srp_send_tsk_mgmt(struct srp_target_port *target,
> if (!target->connected || target->qp_in_error)
> return -1;
>
> - init_completion(&target->tsk_mgmt_done);
> + init_completion(&ch->tsk_mgmt_done);
>
> /*
> - * Lock the rport mutex to avoid that srp_create_target_ib() is
> + * Lock the rport mutex to avoid that srp_create_ch_ib() is
> * invoked while a task management function is being sent.
> */
> mutex_lock(&rport->mutex);
> - spin_lock_irq(&target->lock);
> - iu = __srp_get_tx_iu(target, SRP_IU_TSK_MGMT);
> - spin_unlock_irq(&target->lock);
> + spin_lock_irq(&ch->lock);
> + iu = __srp_get_tx_iu(ch, SRP_IU_TSK_MGMT);
> + spin_unlock_irq(&ch->lock);
>
> if (!iu) {
> mutex_unlock(&rport->mutex);
> @@ -2348,15 +2390,15 @@ static int srp_send_tsk_mgmt(struct srp_target_port *target,
>
> ib_dma_sync_single_for_device(dev, iu->dma, sizeof *tsk_mgmt,
> DMA_TO_DEVICE);
> - if (srp_post_send(target, iu, sizeof *tsk_mgmt)) {
> - srp_put_tx_iu(target, iu, SRP_IU_TSK_MGMT);
> + if (srp_post_send(ch, iu, sizeof(*tsk_mgmt))) {
> + srp_put_tx_iu(ch, iu, SRP_IU_TSK_MGMT);
> mutex_unlock(&rport->mutex);
>
> return -1;
> }
> mutex_unlock(&rport->mutex);
>
> - if (!wait_for_completion_timeout(&target->tsk_mgmt_done,
> + if (!wait_for_completion_timeout(&ch->tsk_mgmt_done,
> msecs_to_jiffies(SRP_ABORT_TIMEOUT_MS)))
> return -1;
>
> @@ -2367,20 +2409,22 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> struct srp_request *req = (struct srp_request *) scmnd->host_scribble;
> + struct srp_rdma_ch *ch;
> int ret;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP abort called\n");
>
> - if (!req || !srp_claim_req(target, req, NULL, scmnd))
> + ch = &target->ch;
> + if (!req || !srp_claim_req(ch, req, NULL, scmnd))
> return SUCCESS;
> - if (srp_send_tsk_mgmt(target, req->index, scmnd->device->lun,
> + if (srp_send_tsk_mgmt(ch, req->index, scmnd->device->lun,
> SRP_TSK_ABORT_TASK) == 0)
> ret = SUCCESS;
> else if (target->rport->state == SRP_RPORT_LOST)
> ret = FAST_IO_FAIL;
> else
> ret = FAILED;
> - srp_free_req(target, req, scmnd, 0);
> + srp_free_req(ch, req, scmnd, 0);
> scmnd->result = DID_ABORT << 16;
> scmnd->scsi_done(scmnd);
>
> @@ -2390,19 +2434,21 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> static int srp_reset_device(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> + struct srp_rdma_ch *ch = &target->ch;
> int i;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n");
>
> - if (srp_send_tsk_mgmt(target, SRP_TAG_NO_REQ, scmnd->device->lun,
> + if (srp_send_tsk_mgmt(ch, SRP_TAG_NO_REQ, scmnd->device->lun,
> SRP_TSK_LUN_RESET))
> return FAILED;
> - if (target->tsk_mgmt_status)
> + if (ch->tsk_mgmt_status)
> return FAILED;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &target->req_ring[i];
> - srp_finish_req(target, req, scmnd->device, DID_RESET << 16);
> + struct srp_request *req = &ch->req_ring[i];
> +
> + srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
> }
>
> return SUCCESS;
> @@ -2464,7 +2510,7 @@ static ssize_t show_pkey(struct device *dev, struct device_attribute *attr,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "0x%04x\n", be16_to_cpu(target->path.pkey));
> + return sprintf(buf, "0x%04x\n", be16_to_cpu(target->pkey));
> }
>
> static ssize_t show_sgid(struct device *dev, struct device_attribute *attr,
> @@ -2472,15 +2518,16 @@ static ssize_t show_sgid(struct device *dev, struct device_attribute *attr,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "%pI6\n", target->path.sgid.raw);
> + return sprintf(buf, "%pI6\n", target->sgid.raw);
> }
>
> static ssize_t show_dgid(struct device *dev, struct device_attribute *attr,
> char *buf)
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
> + struct srp_rdma_ch *ch = &target->ch;
>
> - return sprintf(buf, "%pI6\n", target->path.dgid.raw);
> + return sprintf(buf, "%pI6\n", ch->path.dgid.raw);
> }
>
> static ssize_t show_orig_dgid(struct device *dev,
> @@ -2488,7 +2535,7 @@ static ssize_t show_orig_dgid(struct device *dev,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "%pI6\n", target->orig_dgid);
> + return sprintf(buf, "%pI6\n", target->orig_dgid.raw);
> }
>
> static ssize_t show_req_lim(struct device *dev,
> @@ -2496,7 +2543,7 @@ static ssize_t show_req_lim(struct device *dev,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "%d\n", target->req_lim);
> + return sprintf(buf, "%d\n", target->ch.req_lim);
> }
>
> static ssize_t show_zero_req_lim(struct device *dev,
> @@ -2778,7 +2825,7 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> int opt_mask = 0;
> int token;
> int ret = -EINVAL;
> - int i;
> + int i, b;
>
> options = kstrdup(buf, GFP_KERNEL);
> if (!options)
> @@ -2826,11 +2873,15 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> }
>
> for (i = 0; i < 16; ++i) {
> - strlcpy(dgid, p + i * 2, 3);
> - target->path.dgid.raw[i] = simple_strtoul(dgid, NULL, 16);
> + strlcpy(dgid, p + i * 2, sizeof(dgid));
> + if (sscanf(dgid, "%x", &b) < 1) {
> + ret = -EINVAL;
> + kfree(p);
> + goto out;
> + }
> + target->orig_dgid.raw[i] = b;
> }
> kfree(p);
> - memcpy(target->orig_dgid, target->path.dgid.raw, 16);
> break;
>
> case SRP_OPT_PKEY:
> @@ -2838,7 +2889,7 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> pr_warn("bad P_Key parameter '%s'\n", p);
> goto out;
> }
> - target->path.pkey = cpu_to_be16(token);
> + target->pkey = cpu_to_be16(token);
> break;
>
> case SRP_OPT_SERVICE_ID:
> @@ -2848,7 +2899,6 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> goto out;
> }
> target->service_id = cpu_to_be64(simple_strtoull(p, NULL, 16));
> - target->path.service_id = target->service_id;
> kfree(p);
> break;
>
> @@ -2985,6 +3035,7 @@ static ssize_t srp_create_target(struct device *dev,
> container_of(dev, struct srp_host, dev);
> struct Scsi_Host *target_host;
> struct srp_target_port *target;
> + struct srp_rdma_ch *ch;
> struct srp_device *srp_dev = host->srp_dev;
> struct ib_device *ibdev = srp_dev->dev;
> int ret;
> @@ -3047,24 +3098,28 @@ static ssize_t srp_create_target(struct device *dev,
> INIT_WORK(&target->tl_err_work, srp_tl_err_work);
> INIT_WORK(&target->remove_work, srp_remove_work);
> spin_lock_init(&target->lock);
> - INIT_LIST_HEAD(&target->free_tx);
> - ret = srp_alloc_req_data(target);
> + ch = &target->ch;
> + ch->target = target;
> + ch->comp_vector = target->comp_vector;
> + spin_lock_init(&ch->lock);
> + INIT_LIST_HEAD(&ch->free_tx);
> + ret = srp_alloc_req_data(ch);
> if (ret)
> goto err_free_mem;
>
> - ret = ib_query_gid(ibdev, host->port, 0, &target->path.sgid);
> + ret = ib_query_gid(ibdev, host->port, 0, &target->sgid);
> if (ret)
> goto err_free_mem;
>
> - ret = srp_create_target_ib(target);
> + ret = srp_create_ch_ib(ch);
> if (ret)
> goto err_free_mem;
>
> - ret = srp_new_cm_id(target);
> + ret = srp_new_cm_id(ch);
> if (ret)
> goto err_free_ib;
>
> - ret = srp_connect_target(target);
> + ret = srp_connect_ch(ch);
> if (ret) {
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Connection failed\n");
> @@ -3083,9 +3138,9 @@ static ssize_t srp_create_target(struct device *dev,
> "new target: id_ext %016llx ioc_guid %016llx pkey %04x service_id %016llx sgid %pI6 dgid %pI6\n",
> be64_to_cpu(target->id_ext),
> be64_to_cpu(target->ioc_guid),
> - be16_to_cpu(target->path.pkey),
> + be16_to_cpu(target->pkey),
> be64_to_cpu(target->service_id),
> - target->path.sgid.raw, target->orig_dgid);
> + target->sgid.raw, target->orig_dgid.raw);
> }
>
> scsi_host_put(target->scsi_host);
> @@ -3100,10 +3155,10 @@ err_disconnect:
> srp_disconnect_target(target);
>
> err_free_ib:
> - srp_free_target_ib(target);
> + srp_free_ch_ib(ch);
>
> err_free_mem:
> - srp_free_req_data(target);
> + srp_free_req_data(ch);
>
> err:
> scsi_host_put(target_host);
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
> index 00c7c48..0609124 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.h
> +++ b/drivers/infiniband/ulp/srp/ib_srp.h
> @@ -130,7 +130,7 @@ struct srp_request {
> short index;
> };
>
> -struct srp_target_port {
> +struct srp_rdma_ch {
> /* These are RW in the hot path, and commonly used together */
> struct list_head free_tx;
> struct list_head free_reqs;
> @@ -138,13 +138,43 @@ struct srp_target_port {
> s32 req_lim;
>
> /* These are read-only in the hot path */
> - struct ib_cq *send_cq ____cacheline_aligned_in_smp;
> + struct srp_target_port *target ____cacheline_aligned_in_smp;
> + struct ib_cq *send_cq;
> struct ib_cq *recv_cq;
> struct ib_qp *qp;
> union {
> struct ib_fmr_pool *fmr_pool;
> struct srp_fr_pool *fr_pool;
> };
> +
> + /* Everything above this point is used in the hot path of
> + * command processing. Try to keep them packed into cachelines.
> + */
> +
> + struct completion done;
> + int status;
> +
> + struct ib_sa_path_rec path;
> + struct ib_sa_query *path_query;
> + int path_query_id;
> +
> + struct ib_cm_id *cm_id;
> + struct srp_iu **tx_ring;
> + struct srp_iu **rx_ring;
> + struct srp_request *req_ring;
> + int max_ti_iu_len;
> + int comp_vector;
> +
> + struct completion tsk_mgmt_done;
> + u8 tsk_mgmt_status;
> +};
> +
> +struct srp_target_port {
> + /* read and written in the hot path */
> + spinlock_t lock;
> +
> + struct srp_rdma_ch ch;
> + /* read only in the hot path */
> u32 lkey;
> u32 rkey;
> enum srp_target_state state;
> @@ -153,10 +183,8 @@ struct srp_target_port {
> unsigned int indirect_size;
> bool allow_ext_sg;
>
> - /* Everything above this point is used in the hot path of
> - * command processing. Try to keep them packed into cachelines.
> - */
> -
> + /* other member variables */
> + union ib_gid sgid;
> __be64 id_ext;
> __be64 ioc_guid;
> __be64 service_id;
> @@ -173,34 +201,19 @@ struct srp_target_port {
> int comp_vector;
> int tl_retry_count;
>
> - struct ib_sa_path_rec path;
> - __be16 orig_dgid[8];
> - struct ib_sa_query *path_query;
> - int path_query_id;
> + union ib_gid orig_dgid;
> + __be16 pkey;
>
> u32 rq_tmo_jiffies;
> bool connected;
>
> - struct ib_cm_id *cm_id;
> -
> - int max_ti_iu_len;
> -
> int zero_req_lim;
>
> - struct srp_iu **tx_ring;
> - struct srp_iu **rx_ring;
> - struct srp_request *req_ring;
> -
> struct work_struct tl_err_work;
> struct work_struct remove_work;
>
> struct list_head list;
> - struct completion done;
> - int status;
> bool qp_in_error;
> -
> - struct completion tsk_mgmt_done;
> - u8 tsk_mgmt_status;
> };
>
> struct srp_iu {
>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 5/8] IB/srp: Remove stale connection retry mechanism
[not found] ` <541C287D.1050900-HInyCGIudOg@public.gmane.org>
@ 2014-09-20 17:45 ` Or Gerlitz
[not found] ` <CAJ3xEMhPKiut4MwZH9F7-T0+u7B6XPuh-FTZpA=Xe4ViAj5UUQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Or Gerlitz @ 2014-09-20 17:45 UTC (permalink / raw)
To: Bart Van Assche, sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Jens Axboe, Robert Elliott, Ming Lei
On Fri, Sep 19, 2014 at 3:58 PM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org> wrote:
> Attempting to connect three times may be insufficient after an
> initiator system that was using multiple RDMA channels tries to
> relogin. Additionally, this login retry mechanism is a workaround
> for particular behavior of the IB/CM.
Can you be more specific re the particular behavior of the IB CM?
added Sean, the CM maintainer.
> Since the srp_daemon retries
> a failed login attempt anyway, remove the stale connection retry
> mechanism.
>
> Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
> ---
> drivers/infiniband/ulp/srp/ib_srp.c | 16 +++-------------
> 1 file changed, 3 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index d3c712f..9608e7a 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -904,7 +904,6 @@ static void srp_rport_delete(struct srp_rport *rport)
>
> static int srp_connect_target(struct srp_target_port *target)
> {
> - int retries = 3;
> int ret;
>
> WARN_ON_ONCE(target->connected);
> @@ -945,19 +944,10 @@ static int srp_connect_target(struct srp_target_port *target)
> break;
>
> case SRP_STALE_CONN:
> - /* Our current CM id was stale, and is now in timewait.
> - * Try to reconnect with a new one.
> - */
> - if (!retries-- || srp_new_cm_id(target)) {
> - shost_printk(KERN_ERR, target->scsi_host, PFX
> - "giving up on stale connection\n");
> - target->status = -ECONNRESET;
> - return target->status;
> - }
> -
> shost_printk(KERN_ERR, target->scsi_host, PFX
> - "retrying stale connection\n");
> - break;
> + "giving up on stale connection\n");
> + target->status = -ECONNRESET;
> + return target->status;
>
> default:
> return target->status;
> --
> 1.8.4.5
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH RFC 0/8] IB/srp: Add multichannel support
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
` (6 preceding siblings ...)
2014-09-19 18:31 ` [PATCH RFC 0/8] IB/srp: Add multichannel support Jens Axboe
@ 2014-09-22 14:37 ` Christoph Hellwig
[not found] ` <20140922143731.GA15377-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
7 siblings, 1 reply; 55+ messages in thread
From: Christoph Hellwig @ 2014-09-22 14:37 UTC (permalink / raw)
To: Bart Van Assche
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Jens Axboe,
Robert Elliott, Ming Lei
Hi Bart,
I like these changes modulo the minor comments we had.
One thing that is missing is generation multiqueue-aware tags at the
blk-mq level, which should be as simple as always adding a queue
prefix in the tag allocation code. Did you consider switching srp
to use the block layer provided tags?
Also do you have any performance numbers for just using multiple
queues inside srp vs using blk-mq exposed queues?
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH RFC 0/8] IB/srp: Add multichannel support
[not found] ` <20140922143731.GA15377-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
@ 2014-09-22 16:25 ` Bart Van Assche
2014-09-22 16:31 ` Jens Axboe
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-22 16:25 UTC (permalink / raw)
To: Christoph Hellwig
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Jens Axboe,
Robert Elliott, Ming Lei
On 22/09/2014 8:37, Christoph Hellwig wrote:
> One thing that is missing is generation multiqueue-aware tags at the
> blk-mq level, which should be as simple as always adding a queue
> prefix in the tag allocation code.
Hello Christoph,
Adding a queue prefix in the tag allocation code is an interesting idea.
Encoding the hardware context index in the upper bits of the 'tag' field
in 'struct request' is something I have considered. The reason I have
not done that is because I think several block drivers assume that the
rq->tag field is a number in the range [0..queue_depth-1]. Here is just
one example from the mtip32xx driver:
fis->sect_count = ((rq->tag << 3) | (rq->tag >> 5));
> Did you consider switching srp to use the block layer provided tags?
This is on my to-do list. The only reason I have not yet done this is
because I have not yet had the time to work on it. Another item that is
on my to-do list is to eliminate per-request memory allocation and
instead to use your patch that added a "cmd_size" field in the SCSI host
template.
> Also do you have any performance numbers for just using multiple
> queues inside srp vs using blk-mq exposed queues?
So far I have only rerun the multithreaded write test. For that test I
see about 15% more IOPS with this patch series (exploiting multiple
hardware queues and a 1:1 mapping between hardware context and RDMA
queue pair) compared to the previous implementation (one hardware queue
and multiple RDMA queue pairs). Please keep in mind that in that test
the CPU's of the target system are saturated so the performance
potential of using multiple hardware queues is probably larger than the
difference I measured.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH RFC 0/8] IB/srp: Add multichannel support
2014-09-22 16:25 ` Bart Van Assche
@ 2014-09-22 16:31 ` Jens Axboe
2014-09-22 16:39 ` Jens Axboe
0 siblings, 1 reply; 55+ messages in thread
From: Jens Axboe @ 2014-09-22 16:31 UTC (permalink / raw)
To: Bart Van Assche, Christoph Hellwig
Cc: linux-scsi, linux-rdma, Robert Elliott, Ming Lei
On 2014-09-22 10:25, Bart Van Assche wrote:
> On 22/09/2014 8:37, Christoph Hellwig wrote:
>> One thing that is missing is generation multiqueue-aware tags at the
>> blk-mq level, which should be as simple as always adding a queue
>> prefix in the tag allocation code.
>
> Hello Christoph,
>
> Adding a queue prefix in the tag allocation code is an interesting idea.
> Encoding the hardware context index in the upper bits of the 'tag' field
> in 'struct request' is something I have considered. The reason I have
> not done that is because I think several block drivers assume that the
> rq->tag field is a number in the range [0..queue_depth-1]. Here is just
> one example from the mtip32xx driver:
>
> fis->sect_count = ((rq->tag << 3) | (rq->tag >> 5));
Most drivers assume that the tag is within a certain range, the queue
prefix would only work on drivers where the tag number is just some
arbitrary "cookie". So for SCSI it should work, and I don't think we
need it anywhere else.
Alternatively, we can wrap tag retrieval in some function to mask off
the queue prefix for the cases where we just want an index.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH RFC 0/8] IB/srp: Add multichannel support
2014-09-22 16:31 ` Jens Axboe
@ 2014-09-22 16:39 ` Jens Axboe
0 siblings, 0 replies; 55+ messages in thread
From: Jens Axboe @ 2014-09-22 16:39 UTC (permalink / raw)
To: Bart Van Assche, Christoph Hellwig
Cc: linux-scsi, linux-rdma, Robert Elliott, Ming Lei
On 2014-09-22 10:31, Jens Axboe wrote:
> On 2014-09-22 10:25, Bart Van Assche wrote:
>> On 22/09/2014 8:37, Christoph Hellwig wrote:
>>> One thing that is missing is generation multiqueue-aware tags at the
>>> blk-mq level, which should be as simple as always adding a queue
>>> prefix in the tag allocation code.
>>
>> Hello Christoph,
>>
>> Adding a queue prefix in the tag allocation code is an interesting idea.
>> Encoding the hardware context index in the upper bits of the 'tag' field
>> in 'struct request' is something I have considered. The reason I have
>> not done that is because I think several block drivers assume that the
>> rq->tag field is a number in the range [0..queue_depth-1]. Here is just
>> one example from the mtip32xx driver:
>>
>> fis->sect_count = ((rq->tag << 3) | (rq->tag >> 5));
>
> Most drivers assume that the tag is within a certain range, the queue
> prefix would only work on drivers where the tag number is just some
> arbitrary "cookie". So for SCSI it should work, and I don't think we
> need it anywhere else.
>
> Alternatively, we can wrap tag retrieval in some function to mask off
> the queue prefix for the cases where we just want an index.
Would probably be better to have a tag generation function instead, that
uses rq->tag and ORs in the queue prefix, actually. That way rq->tag
would remain being a plain index, and the issued tag in the driver could
then use some function to add the queue prefix, if it needs unique tags
across queues in a shared tag set.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 7/8] IB/srp: Separate target and channel variables
[not found] ` <541C28C8.7000007-HInyCGIudOg@public.gmane.org>
@ 2014-09-23 16:07 ` Sagi Grimberg
2014-09-23 20:00 ` Bart Van Assche
0 siblings, 1 reply; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-23 16:07 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/19/2014 3:59 PM, Bart Van Assche wrote:
> Changes in this patch:
> - Move channel variables into a new structure (struct srp_rdma_ch).
> - cm_id and completion handler context pointer are now of type
> srp_rdma_ch * insteoad of srp_target_port *.
>
> No functionality is changed.
>
> Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
> ---
> drivers/infiniband/ulp/srp/ib_srp.c | 707 +++++++++++++++++++-----------------
> drivers/infiniband/ulp/srp/ib_srp.h | 59 +--
> 2 files changed, 417 insertions(+), 349 deletions(-)
>
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index fd88fb8..9feeea1 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -125,8 +125,8 @@ MODULE_PARM_DESC(dev_loss_tmo,
>
> static void srp_add_one(struct ib_device *device);
> static void srp_remove_one(struct ib_device *device);
> -static void srp_recv_completion(struct ib_cq *cq, void *target_ptr);
> -static void srp_send_completion(struct ib_cq *cq, void *target_ptr);
> +static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr);
> +static void srp_send_completion(struct ib_cq *cq, void *ch_ptr);
> static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event);
>
> static struct scsi_transport_template *ib_srp_transport_template;
> @@ -262,7 +262,7 @@ static int srp_init_qp(struct srp_target_port *target,
>
> ret = ib_find_pkey(target->srp_host->srp_dev->dev,
> target->srp_host->port,
> - be16_to_cpu(target->path.pkey),
> + be16_to_cpu(target->pkey),
> &attr->pkey_index);
> if (ret)
> goto out;
> @@ -283,18 +283,23 @@ out:
> return ret;
> }
>
> -static int srp_new_cm_id(struct srp_target_port *target)
> +static int srp_new_cm_id(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_cm_id *new_cm_id;
>
> new_cm_id = ib_create_cm_id(target->srp_host->srp_dev->dev,
> - srp_cm_handler, target);
> + srp_cm_handler, ch);
> if (IS_ERR(new_cm_id))
> return PTR_ERR(new_cm_id);
>
> - if (target->cm_id)
> - ib_destroy_cm_id(target->cm_id);
> - target->cm_id = new_cm_id;
> + if (ch->cm_id)
> + ib_destroy_cm_id(ch->cm_id);
> + ch->cm_id = new_cm_id;
> + ch->path.sgid = target->sgid;
> + ch->path.dgid = target->orig_dgid;
> + ch->path.pkey = target->pkey;
> + ch->path.service_id = target->service_id;
>
> return 0;
> }
> @@ -443,8 +448,9 @@ static struct srp_fr_pool *srp_alloc_fr_pool(struct srp_target_port *target)
> dev->max_pages_per_mr);
> }
>
> -static int srp_create_target_ib(struct srp_target_port *target)
> +static int srp_create_ch_ib(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_qp_init_attr *init_attr;
> struct ib_cq *recv_cq, *send_cq;
> @@ -458,15 +464,15 @@ static int srp_create_target_ib(struct srp_target_port *target)
> if (!init_attr)
> return -ENOMEM;
>
> - recv_cq = ib_create_cq(dev->dev, srp_recv_completion, NULL, target,
> - target->queue_size, target->comp_vector);
> + recv_cq = ib_create_cq(dev->dev, srp_recv_completion, NULL, ch,
> + target->queue_size, ch->comp_vector);
> if (IS_ERR(recv_cq)) {
> ret = PTR_ERR(recv_cq);
> goto err;
> }
>
> - send_cq = ib_create_cq(dev->dev, srp_send_completion, NULL, target,
> - m * target->queue_size, target->comp_vector);
> + send_cq = ib_create_cq(dev->dev, srp_send_completion, NULL, ch,
> + m * target->queue_size, ch->comp_vector);
> if (IS_ERR(send_cq)) {
> ret = PTR_ERR(send_cq);
> goto err_recv_cq;
> @@ -502,9 +508,9 @@ static int srp_create_target_ib(struct srp_target_port *target)
> "FR pool allocation failed (%d)\n", ret);
> goto err_qp;
> }
> - if (target->fr_pool)
> - srp_destroy_fr_pool(target->fr_pool);
> - target->fr_pool = fr_pool;
> + if (ch->fr_pool)
> + srp_destroy_fr_pool(ch->fr_pool);
> + ch->fr_pool = fr_pool;
> } else if (!dev->use_fast_reg && dev->has_fmr) {
> fmr_pool = srp_alloc_fmr_pool(target);
> if (IS_ERR(fmr_pool)) {
> @@ -513,21 +519,21 @@ static int srp_create_target_ib(struct srp_target_port *target)
> "FMR pool allocation failed (%d)\n", ret);
> goto err_qp;
> }
> - if (target->fmr_pool)
> - ib_destroy_fmr_pool(target->fmr_pool);
> - target->fmr_pool = fmr_pool;
> + if (ch->fmr_pool)
> + ib_destroy_fmr_pool(ch->fmr_pool);
> + ch->fmr_pool = fmr_pool;
> }
>
> - if (target->qp)
> - ib_destroy_qp(target->qp);
> - if (target->recv_cq)
> - ib_destroy_cq(target->recv_cq);
> - if (target->send_cq)
> - ib_destroy_cq(target->send_cq);
> + if (ch->qp)
> + ib_destroy_qp(ch->qp);
> + if (ch->recv_cq)
> + ib_destroy_cq(ch->recv_cq);
> + if (ch->send_cq)
> + ib_destroy_cq(ch->send_cq);
>
> - target->qp = qp;
> - target->recv_cq = recv_cq;
> - target->send_cq = send_cq;
> + ch->qp = qp;
> + ch->recv_cq = recv_cq;
> + ch->send_cq = send_cq;
>
> kfree(init_attr);
> return 0;
> @@ -548,98 +554,102 @@ err:
>
> /*
> * Note: this function may be called without srp_alloc_iu_bufs() having been
> - * invoked. Hence the target->[rt]x_ring checks.
> + * invoked. Hence the ch->[rt]x_ring checks.
> */
> -static void srp_free_target_ib(struct srp_target_port *target)
> +static void srp_free_ch_ib(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> int i;
>
> - if (target->cm_id) {
> - ib_destroy_cm_id(target->cm_id);
> - target->cm_id = NULL;
> + if (ch->cm_id) {
> + ib_destroy_cm_id(ch->cm_id);
> + ch->cm_id = NULL;
> }
>
> if (dev->use_fast_reg) {
> - if (target->fr_pool)
> - srp_destroy_fr_pool(target->fr_pool);
> + if (ch->fr_pool)
> + srp_destroy_fr_pool(ch->fr_pool);
> } else {
> - if (target->fmr_pool)
> - ib_destroy_fmr_pool(target->fmr_pool);
> + if (ch->fmr_pool)
> + ib_destroy_fmr_pool(ch->fmr_pool);
> }
> - ib_destroy_qp(target->qp);
> - ib_destroy_cq(target->send_cq);
> - ib_destroy_cq(target->recv_cq);
> + ib_destroy_qp(ch->qp);
> + ib_destroy_cq(ch->send_cq);
> + ib_destroy_cq(ch->recv_cq);
>
> - target->qp = NULL;
> - target->send_cq = target->recv_cq = NULL;
> + ch->qp = NULL;
> + ch->send_cq = ch->recv_cq = NULL;
>
> - if (target->rx_ring) {
> + if (ch->rx_ring) {
> for (i = 0; i < target->queue_size; ++i)
> - srp_free_iu(target->srp_host, target->rx_ring[i]);
> - kfree(target->rx_ring);
> - target->rx_ring = NULL;
> + srp_free_iu(target->srp_host, ch->rx_ring[i]);
> + kfree(ch->rx_ring);
> + ch->rx_ring = NULL;
> }
> - if (target->tx_ring) {
> + if (ch->tx_ring) {
> for (i = 0; i < target->queue_size; ++i)
> - srp_free_iu(target->srp_host, target->tx_ring[i]);
> - kfree(target->tx_ring);
> - target->tx_ring = NULL;
> + srp_free_iu(target->srp_host, ch->tx_ring[i]);
> + kfree(ch->tx_ring);
> + ch->tx_ring = NULL;
> }
> }
>
> static void srp_path_rec_completion(int status,
> struct ib_sa_path_rec *pathrec,
> - void *target_ptr)
> + void *ch_ptr)
> {
> - struct srp_target_port *target = target_ptr;
> + struct srp_rdma_ch *ch = ch_ptr;
> + struct srp_target_port *target = ch->target;
>
> - target->status = status;
> + ch->status = status;
> if (status)
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Got failed path rec status %d\n", status);
> else
> - target->path = *pathrec;
> - complete(&target->done);
> + ch->path = *pathrec;
> + complete(&ch->done);
> }
>
> -static int srp_lookup_path(struct srp_target_port *target)
> +static int srp_lookup_path(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int ret;
>
> - target->path.numb_path = 1;
> -
> - init_completion(&target->done);
> -
> - target->path_query_id = ib_sa_path_rec_get(&srp_sa_client,
> - target->srp_host->srp_dev->dev,
> - target->srp_host->port,
> - &target->path,
> - IB_SA_PATH_REC_SERVICE_ID |
> - IB_SA_PATH_REC_DGID |
> - IB_SA_PATH_REC_SGID |
> - IB_SA_PATH_REC_NUMB_PATH |
> - IB_SA_PATH_REC_PKEY,
> - SRP_PATH_REC_TIMEOUT_MS,
> - GFP_KERNEL,
> - srp_path_rec_completion,
> - target, &target->path_query);
> - if (target->path_query_id < 0)
> - return target->path_query_id;
> -
> - ret = wait_for_completion_interruptible(&target->done);
> + ch->path.numb_path = 1;
> +
> + init_completion(&ch->done);
> +
> + ch->path_query_id = ib_sa_path_rec_get(&srp_sa_client,
> + target->srp_host->srp_dev->dev,
> + target->srp_host->port,
> + &ch->path,
> + IB_SA_PATH_REC_SERVICE_ID |
> + IB_SA_PATH_REC_DGID |
> + IB_SA_PATH_REC_SGID |
> + IB_SA_PATH_REC_NUMB_PATH |
> + IB_SA_PATH_REC_PKEY,
> + SRP_PATH_REC_TIMEOUT_MS,
> + GFP_KERNEL,
> + srp_path_rec_completion,
> + ch, &ch->path_query);
> + if (ch->path_query_id < 0)
> + return ch->path_query_id;
> +
> + ret = wait_for_completion_interruptible(&ch->done);
> if (ret < 0)
> return ret;
>
> - if (target->status < 0)
> + if (ch->status < 0)
> shost_printk(KERN_WARNING, target->scsi_host,
> PFX "Path record query failed\n");
>
> - return target->status;
> + return ch->status;
> }
>
> -static int srp_send_req(struct srp_target_port *target)
> +static int srp_send_req(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct {
> struct ib_cm_req_param param;
> struct srp_login_req priv;
> @@ -650,11 +660,11 @@ static int srp_send_req(struct srp_target_port *target)
> if (!req)
> return -ENOMEM;
>
> - req->param.primary_path = &target->path;
> + req->param.primary_path = &ch->path;
> req->param.alternate_path = NULL;
> req->param.service_id = target->service_id;
> - req->param.qp_num = target->qp->qp_num;
> - req->param.qp_type = target->qp->qp_type;
> + req->param.qp_num = ch->qp->qp_num;
> + req->param.qp_type = ch->qp->qp_type;
> req->param.private_data = &req->priv;
> req->param.private_data_len = sizeof req->priv;
> req->param.flow_control = 1;
> @@ -689,7 +699,7 @@ static int srp_send_req(struct srp_target_port *target)
> */
> if (target->io_class == SRP_REV10_IB_IO_CLASS) {
> memcpy(req->priv.initiator_port_id,
> - &target->path.sgid.global.interface_id, 8);
> + &target->sgid.global.interface_id, 8);
> memcpy(req->priv.initiator_port_id + 8,
> &target->initiator_ext, 8);
> memcpy(req->priv.target_port_id, &target->ioc_guid, 8);
> @@ -698,7 +708,7 @@ static int srp_send_req(struct srp_target_port *target)
> memcpy(req->priv.initiator_port_id,
> &target->initiator_ext, 8);
> memcpy(req->priv.initiator_port_id + 8,
> - &target->path.sgid.global.interface_id, 8);
> + &target->sgid.global.interface_id, 8);
> memcpy(req->priv.target_port_id, &target->id_ext, 8);
> memcpy(req->priv.target_port_id + 8, &target->ioc_guid, 8);
> }
> @@ -718,7 +728,7 @@ static int srp_send_req(struct srp_target_port *target)
> &target->srp_host->srp_dev->dev->node_guid, 8);
> }
>
> - status = ib_send_cm_req(target->cm_id, &req->param);
> + status = ib_send_cm_req(ch->cm_id, &req->param);
>
> kfree(req);
>
> @@ -759,28 +769,31 @@ static bool srp_change_conn_state(struct srp_target_port *target,
>
> static void srp_disconnect_target(struct srp_target_port *target)
> {
> + struct srp_rdma_ch *ch = &target->ch;
> +
> if (srp_change_conn_state(target, false)) {
> /* XXX should send SRP_I_LOGOUT request */
>
> - if (ib_send_cm_dreq(target->cm_id, NULL, 0)) {
> + if (ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
> shost_printk(KERN_DEBUG, target->scsi_host,
> PFX "Sending CM DREQ failed\n");
> }
> }
> }
>
> -static void srp_free_req_data(struct srp_target_port *target)
> +static void srp_free_req_data(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> struct srp_request *req;
> int i;
>
> - if (!target->req_ring)
> + if (!ch->req_ring)
> return;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - req = &target->req_ring[i];
> + req = &ch->req_ring[i];
> if (dev->use_fast_reg)
> kfree(req->fr_list);
> else
> @@ -794,12 +807,13 @@ static void srp_free_req_data(struct srp_target_port *target)
> kfree(req->indirect_desc);
> }
>
> - kfree(target->req_ring);
> - target->req_ring = NULL;
> + kfree(ch->req_ring);
> + ch->req_ring = NULL;
> }
>
> -static int srp_alloc_req_data(struct srp_target_port *target)
> +static int srp_alloc_req_data(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *srp_dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = srp_dev->dev;
> struct srp_request *req;
> @@ -807,15 +821,15 @@ static int srp_alloc_req_data(struct srp_target_port *target)
> dma_addr_t dma_addr;
> int i, ret = -ENOMEM;
>
> - INIT_LIST_HEAD(&target->free_reqs);
> + INIT_LIST_HEAD(&ch->free_reqs);
>
> - target->req_ring = kzalloc(target->req_ring_size *
> - sizeof(*target->req_ring), GFP_KERNEL);
> - if (!target->req_ring)
> + ch->req_ring = kcalloc(target->req_ring_size, sizeof(*ch->req_ring),
> + GFP_KERNEL);
> + if (!ch->req_ring)
> goto out;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - req = &target->req_ring[i];
> + req = &ch->req_ring[i];
> mr_list = kmalloc(target->cmd_sg_cnt * sizeof(void *),
> GFP_KERNEL);
> if (!mr_list)
> @@ -840,7 +854,7 @@ static int srp_alloc_req_data(struct srp_target_port *target)
>
> req->indirect_dma_addr = dma_addr;
> req->index = i;
> - list_add_tail(&req->list, &target->free_reqs);
> + list_add_tail(&req->list, &ch->free_reqs);
> }
> ret = 0;
>
> @@ -865,6 +879,8 @@ static void srp_del_scsi_host_attr(struct Scsi_Host *shost)
>
> static void srp_remove_target(struct srp_target_port *target)
> {
> + struct srp_rdma_ch *ch = &target->ch;
> +
> WARN_ON_ONCE(target->state != SRP_TARGET_REMOVED);
>
> srp_del_scsi_host_attr(target->scsi_host);
> @@ -873,10 +889,10 @@ static void srp_remove_target(struct srp_target_port *target)
> scsi_remove_host(target->scsi_host);
> srp_stop_rport_timers(target->rport);
> srp_disconnect_target(target);
> - srp_free_target_ib(target);
> + srp_free_ch_ib(ch);
> cancel_work_sync(&target->tl_err_work);
> srp_rport_put(target->rport);
> - srp_free_req_data(target);
> + srp_free_req_data(ch);
>
> spin_lock(&target->srp_host->target_lock);
> list_del(&target->list);
> @@ -902,24 +918,25 @@ static void srp_rport_delete(struct srp_rport *rport)
> srp_queue_remove_work(target);
> }
>
> -static int srp_connect_target(struct srp_target_port *target)
> +static int srp_connect_ch(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int ret;
>
> WARN_ON_ONCE(target->connected);
>
> target->qp_in_error = false;
>
> - ret = srp_lookup_path(target);
> + ret = srp_lookup_path(ch);
> if (ret)
> return ret;
>
> while (1) {
> - init_completion(&target->done);
> - ret = srp_send_req(target);
> + init_completion(&ch->done);
> + ret = srp_send_req(ch);
> if (ret)
> return ret;
> - ret = wait_for_completion_interruptible(&target->done);
> + ret = wait_for_completion_interruptible(&ch->done);
> if (ret < 0)
> return ret;
>
> @@ -929,13 +946,13 @@ static int srp_connect_target(struct srp_target_port *target)
> * back, or SRP_DLID_REDIRECT if we get a lid/qp
> * redirect REJ back.
> */
> - switch (target->status) {
> + switch (ch->status) {
> case 0:
> srp_change_conn_state(target, true);
> return 0;
>
> case SRP_PORT_REDIRECT:
> - ret = srp_lookup_path(target);
> + ret = srp_lookup_path(ch);
> if (ret)
> return ret;
> break;
> @@ -946,16 +963,16 @@ static int srp_connect_target(struct srp_target_port *target)
> case SRP_STALE_CONN:
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "giving up on stale connection\n");
> - target->status = -ECONNRESET;
> - return target->status;
> + ch->status = -ECONNRESET;
> + return ch->status;
>
> default:
> - return target->status;
> + return ch->status;
> }
> }
> }
>
> -static int srp_inv_rkey(struct srp_target_port *target, u32 rkey)
> +static int srp_inv_rkey(struct srp_rdma_ch *ch, u32 rkey)
> {
> struct ib_send_wr *bad_wr;
> struct ib_send_wr wr = {
> @@ -967,13 +984,14 @@ static int srp_inv_rkey(struct srp_target_port *target, u32 rkey)
> .ex.invalidate_rkey = rkey,
> };
>
> - return ib_post_send(target->qp, &wr, &bad_wr);
> + return ib_post_send(ch->qp, &wr, &bad_wr);
> }
>
> static void srp_unmap_data(struct scsi_cmnd *scmnd,
> - struct srp_target_port *target,
> + struct srp_rdma_ch *ch,
> struct srp_request *req)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> int i, res;
> @@ -987,7 +1005,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
> struct srp_fr_desc **pfr;
>
> for (i = req->nmdesc, pfr = req->fr_list; i > 0; i--, pfr++) {
> - res = srp_inv_rkey(target, (*pfr)->mr->rkey);
> + res = srp_inv_rkey(ch, (*pfr)->mr->rkey);
> if (res < 0) {
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "Queueing INV WR for rkey %#x failed (%d)\n",
> @@ -997,7 +1015,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
> }
> }
> if (req->nmdesc)
> - srp_fr_pool_put(target->fr_pool, req->fr_list,
> + srp_fr_pool_put(ch->fr_pool, req->fr_list,
> req->nmdesc);
> } else {
> struct ib_pool_fmr **pfmr;
> @@ -1012,7 +1030,7 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
>
> /**
> * srp_claim_req - Take ownership of the scmnd associated with a request.
> - * @target: SRP target port.
> + * @ch: SRP RDMA channel.
> * @req: SRP request.
> * @sdev: If not NULL, only take ownership for this SCSI device.
> * @scmnd: If NULL, take ownership of @req->scmnd. If not NULL, only take
> @@ -1021,14 +1039,14 @@ static void srp_unmap_data(struct scsi_cmnd *scmnd,
> * Return value:
> * Either NULL or a pointer to the SCSI command the caller became owner of.
> */
> -static struct scsi_cmnd *srp_claim_req(struct srp_target_port *target,
> +static struct scsi_cmnd *srp_claim_req(struct srp_rdma_ch *ch,
> struct srp_request *req,
> struct scsi_device *sdev,
> struct scsi_cmnd *scmnd)
> {
> unsigned long flags;
>
> - spin_lock_irqsave(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> if (req->scmnd &&
> (!sdev || req->scmnd->device == sdev) &&
> (!scmnd || req->scmnd == scmnd)) {
> @@ -1037,40 +1055,38 @@ static struct scsi_cmnd *srp_claim_req(struct srp_target_port *target,
> } else {
> scmnd = NULL;
> }
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> return scmnd;
> }
>
> /**
> * srp_free_req() - Unmap data and add request to the free request list.
> - * @target: SRP target port.
> + * @ch: SRP RDMA channel.
> * @req: Request to be freed.
> * @scmnd: SCSI command associated with @req.
> * @req_lim_delta: Amount to be added to @target->req_lim.
> */
> -static void srp_free_req(struct srp_target_port *target,
> - struct srp_request *req, struct scsi_cmnd *scmnd,
> - s32 req_lim_delta)
> +static void srp_free_req(struct srp_rdma_ch *ch, struct srp_request *req,
> + struct scsi_cmnd *scmnd, s32 req_lim_delta)
> {
> unsigned long flags;
>
> - srp_unmap_data(scmnd, target, req);
> + srp_unmap_data(scmnd, ch, req);
>
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += req_lim_delta;
> - list_add_tail(&req->list, &target->free_reqs);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += req_lim_delta;
> + list_add_tail(&req->list, &ch->free_reqs);
> + spin_unlock_irqrestore(&ch->lock, flags);
> }
>
> -static void srp_finish_req(struct srp_target_port *target,
> - struct srp_request *req, struct scsi_device *sdev,
> - int result)
> +static void srp_finish_req(struct srp_rdma_ch *ch, struct srp_request *req,
> + struct scsi_device *sdev, int result)
> {
> - struct scsi_cmnd *scmnd = srp_claim_req(target, req, sdev, NULL);
> + struct scsi_cmnd *scmnd = srp_claim_req(ch, req, sdev, NULL);
>
> if (scmnd) {
> - srp_free_req(target, req, scmnd, 0);
> + srp_free_req(ch, req, scmnd, 0);
> scmnd->result = result;
> scmnd->scsi_done(scmnd);
> }
> @@ -1079,6 +1095,7 @@ static void srp_finish_req(struct srp_target_port *target,
> static void srp_terminate_io(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> + struct srp_rdma_ch *ch = &target->ch;
> struct Scsi_Host *shost = target->scsi_host;
> struct scsi_device *sdev;
> int i;
> @@ -1091,8 +1108,9 @@ static void srp_terminate_io(struct srp_rport *rport)
> WARN_ON_ONCE(sdev->request_queue->request_fn_active);
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &target->req_ring[i];
> - srp_finish_req(target, req, NULL, DID_TRANSPORT_FAILFAST << 16);
> + struct srp_request *req = &ch->req_ring[i];
> +
> + srp_finish_req(ch, req, NULL, DID_TRANSPORT_FAILFAST << 16);
> }
> }
>
> @@ -1108,6 +1126,7 @@ static void srp_terminate_io(struct srp_rport *rport)
> static int srp_rport_reconnect(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> + struct srp_rdma_ch *ch = &target->ch;
> int i, ret;
>
> srp_disconnect_target(target);
> @@ -1120,11 +1139,12 @@ static int srp_rport_reconnect(struct srp_rport *rport)
> * case things are really fouled up. Doing so also ensures that all CM
> * callbacks will have finished before a new QP is allocated.
> */
> - ret = srp_new_cm_id(target);
> + ret = srp_new_cm_id(ch);
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &target->req_ring[i];
> - srp_finish_req(target, req, NULL, DID_RESET << 16);
> + struct srp_request *req = &ch->req_ring[i];
> +
> + srp_finish_req(ch, req, NULL, DID_RESET << 16);
> }
>
> /*
> @@ -1132,14 +1152,14 @@ static int srp_rport_reconnect(struct srp_rport *rport)
> * QP. This guarantees that all callback functions for the old QP have
> * finished before any send requests are posted on the new QP.
> */
> - ret += srp_create_target_ib(target);
> + ret += srp_create_ch_ib(ch);
>
> - INIT_LIST_HEAD(&target->free_tx);
> + INIT_LIST_HEAD(&ch->free_tx);
> for (i = 0; i < target->queue_size; ++i)
> - list_add(&target->tx_ring[i]->list, &target->free_tx);
> + list_add(&ch->tx_ring[i]->list, &ch->free_tx);
>
> if (ret == 0)
> - ret = srp_connect_target(target);
> + ret = srp_connect_ch(ch);
>
> if (ret == 0)
> shost_printk(KERN_INFO, target->scsi_host,
> @@ -1163,12 +1183,12 @@ static void srp_map_desc(struct srp_map_state *state, dma_addr_t dma_addr,
> }
>
> static int srp_map_finish_fmr(struct srp_map_state *state,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> struct ib_pool_fmr *fmr;
> u64 io_addr = 0;
>
> - fmr = ib_fmr_pool_map_phys(target->fmr_pool, state->pages,
> + fmr = ib_fmr_pool_map_phys(ch->fmr_pool, state->pages,
> state->npages, io_addr);
> if (IS_ERR(fmr))
> return PTR_ERR(fmr);
> @@ -1182,15 +1202,16 @@ static int srp_map_finish_fmr(struct srp_map_state *state,
> }
>
> static int srp_map_finish_fr(struct srp_map_state *state,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_send_wr *bad_wr;
> struct ib_send_wr wr;
> struct srp_fr_desc *desc;
> u32 rkey;
>
> - desc = srp_fr_pool_get(target->fr_pool);
> + desc = srp_fr_pool_get(ch->fr_pool);
> if (!desc)
> return -ENOMEM;
>
> @@ -1219,12 +1240,13 @@ static int srp_map_finish_fr(struct srp_map_state *state,
> srp_map_desc(state, state->base_dma_addr, state->dma_len,
> desc->mr->rkey);
>
> - return ib_post_send(target->qp, &wr, &bad_wr);
> + return ib_post_send(ch->qp, &wr, &bad_wr);
> }
>
> static int srp_finish_mapping(struct srp_map_state *state,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int ret = 0;
>
> if (state->npages == 0)
> @@ -1235,8 +1257,8 @@ static int srp_finish_mapping(struct srp_map_state *state,
> target->rkey);
> else
> ret = target->srp_host->srp_dev->use_fast_reg ?
> - srp_map_finish_fr(state, target) :
> - srp_map_finish_fmr(state, target);
> + srp_map_finish_fr(state, ch) :
> + srp_map_finish_fmr(state, ch);
>
> if (ret == 0) {
> state->npages = 0;
> @@ -1256,10 +1278,11 @@ static void srp_map_update_start(struct srp_map_state *state,
> }
>
> static int srp_map_sg_entry(struct srp_map_state *state,
> - struct srp_target_port *target,
> + struct srp_rdma_ch *ch,
> struct scatterlist *sg, int sg_index,
> bool use_mr)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> dma_addr_t dma_addr = ib_sg_dma_address(ibdev, sg);
> @@ -1288,7 +1311,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
> */
> if ((!dev->use_fast_reg && dma_addr & ~dev->mr_page_mask) ||
> dma_len > dev->mr_max_size) {
> - ret = srp_finish_mapping(state, target);
> + ret = srp_finish_mapping(state, ch);
> if (ret)
> return ret;
>
> @@ -1309,7 +1332,7 @@ static int srp_map_sg_entry(struct srp_map_state *state,
> while (dma_len) {
> unsigned offset = dma_addr & ~dev->mr_page_mask;
> if (state->npages == dev->max_pages_per_mr || offset != 0) {
> - ret = srp_finish_mapping(state, target);
> + ret = srp_finish_mapping(state, ch);
> if (ret)
> return ret;
>
> @@ -1333,17 +1356,18 @@ static int srp_map_sg_entry(struct srp_map_state *state,
> */
> ret = 0;
> if (len != dev->mr_page_size) {
> - ret = srp_finish_mapping(state, target);
> + ret = srp_finish_mapping(state, ch);
> if (!ret)
> srp_map_update_start(state, NULL, 0, 0);
> }
> return ret;
> }
>
> -static int srp_map_sg(struct srp_map_state *state,
> - struct srp_target_port *target, struct srp_request *req,
> - struct scatterlist *scat, int count)
> +static int srp_map_sg(struct srp_map_state *state, struct srp_rdma_ch *ch,
> + struct srp_request *req, struct scatterlist *scat,
> + int count)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> struct scatterlist *sg;
> @@ -1354,14 +1378,14 @@ static int srp_map_sg(struct srp_map_state *state,
> state->pages = req->map_page;
> if (dev->use_fast_reg) {
> state->next_fr = req->fr_list;
> - use_mr = !!target->fr_pool;
> + use_mr = !!ch->fr_pool;
> } else {
> state->next_fmr = req->fmr_list;
> - use_mr = !!target->fmr_pool;
> + use_mr = !!ch->fmr_pool;
> }
>
> for_each_sg(scat, sg, count, i) {
> - if (srp_map_sg_entry(state, target, sg, i, use_mr)) {
> + if (srp_map_sg_entry(state, ch, sg, i, use_mr)) {
> /*
> * Memory registration failed, so backtrack to the
> * first unmapped entry and continue on without using
> @@ -1383,7 +1407,7 @@ backtrack:
> }
> }
>
> - if (use_mr && srp_finish_mapping(state, target))
> + if (use_mr && srp_finish_mapping(state, ch))
> goto backtrack;
>
> req->nmdesc = state->nmdesc;
> @@ -1391,9 +1415,10 @@ backtrack:
> return 0;
> }
>
> -static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port *target,
> +static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_rdma_ch *ch,
> struct srp_request *req)
> {
> + struct srp_target_port *target = ch->target;
> struct scatterlist *scat;
> struct srp_cmd *cmd = req->cmd->buf;
> int len, nents, count;
> @@ -1455,7 +1480,7 @@ static int srp_map_data(struct scsi_cmnd *scmnd, struct srp_target_port *target,
> target->indirect_size, DMA_TO_DEVICE);
>
> memset(&state, 0, sizeof(state));
> - srp_map_sg(&state, target, req, scat, count);
> + srp_map_sg(&state, ch, req, scat, count);
>
> /* We've mapped the request, now pull as much of the indirect
> * descriptor table as we can into the command buffer. If this
> @@ -1516,20 +1541,20 @@ map_complete:
> /*
> * Return an IU and possible credit to the free pool
> */
> -static void srp_put_tx_iu(struct srp_target_port *target, struct srp_iu *iu,
> +static void srp_put_tx_iu(struct srp_rdma_ch *ch, struct srp_iu *iu,
> enum srp_iu_type iu_type)
> {
> unsigned long flags;
>
> - spin_lock_irqsave(&target->lock, flags);
> - list_add(&iu->list, &target->free_tx);
> + spin_lock_irqsave(&ch->lock, flags);
> + list_add(&iu->list, &ch->free_tx);
> if (iu_type != SRP_IU_RSP)
> - ++target->req_lim;
> - spin_unlock_irqrestore(&target->lock, flags);
> + ++ch->req_lim;
> + spin_unlock_irqrestore(&ch->lock, flags);
> }
>
> /*
> - * Must be called with target->lock held to protect req_lim and free_tx.
> + * Must be called with ch->lock held to protect req_lim and free_tx.
> * If IU is not sent, it must be returned using srp_put_tx_iu().
> *
> * Note:
> @@ -1541,35 +1566,36 @@ static void srp_put_tx_iu(struct srp_target_port *target, struct srp_iu *iu,
> * - SRP_IU_RSP: 1, since a conforming SRP target never sends more than
> * one unanswered SRP request to an initiator.
> */
> -static struct srp_iu *__srp_get_tx_iu(struct srp_target_port *target,
> +static struct srp_iu *__srp_get_tx_iu(struct srp_rdma_ch *ch,
> enum srp_iu_type iu_type)
> {
> + struct srp_target_port *target = ch->target;
> s32 rsv = (iu_type == SRP_IU_TSK_MGMT) ? 0 : SRP_TSK_MGMT_SQ_SIZE;
> struct srp_iu *iu;
>
> - srp_send_completion(target->send_cq, target);
> + srp_send_completion(ch->send_cq, target);
>
> - if (list_empty(&target->free_tx))
> + if (list_empty(&ch->free_tx))
> return NULL;
>
> /* Initiator responses to target requests do not consume credits */
> if (iu_type != SRP_IU_RSP) {
> - if (target->req_lim <= rsv) {
> + if (ch->req_lim <= rsv) {
> ++target->zero_req_lim;
> return NULL;
> }
>
> - --target->req_lim;
> + --ch->req_lim;
> }
>
> - iu = list_first_entry(&target->free_tx, struct srp_iu, list);
> + iu = list_first_entry(&ch->free_tx, struct srp_iu, list);
> list_del(&iu->list);
> return iu;
> }
>
> -static int srp_post_send(struct srp_target_port *target,
> - struct srp_iu *iu, int len)
> +static int srp_post_send(struct srp_rdma_ch *ch, struct srp_iu *iu, int len)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_sge list;
> struct ib_send_wr wr, *bad_wr;
>
> @@ -1584,11 +1610,12 @@ static int srp_post_send(struct srp_target_port *target,
> wr.opcode = IB_WR_SEND;
> wr.send_flags = IB_SEND_SIGNALED;
>
> - return ib_post_send(target->qp, &wr, &bad_wr);
> + return ib_post_send(ch->qp, &wr, &bad_wr);
> }
>
> -static int srp_post_recv(struct srp_target_port *target, struct srp_iu *iu)
> +static int srp_post_recv(struct srp_rdma_ch *ch, struct srp_iu *iu)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_recv_wr wr, *bad_wr;
> struct ib_sge list;
>
> @@ -1601,35 +1628,36 @@ static int srp_post_recv(struct srp_target_port *target, struct srp_iu *iu)
> wr.sg_list = &list;
> wr.num_sge = 1;
>
> - return ib_post_recv(target->qp, &wr, &bad_wr);
> + return ib_post_recv(ch->qp, &wr, &bad_wr);
> }
>
> -static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
> +static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_request *req;
> struct scsi_cmnd *scmnd;
> unsigned long flags;
>
> if (unlikely(rsp->tag & SRP_TAG_TSK_MGMT)) {
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += be32_to_cpu(rsp->req_lim_delta);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> - target->tsk_mgmt_status = -1;
> + ch->tsk_mgmt_status = -1;
> if (be32_to_cpu(rsp->resp_data_len) >= 4)
> - target->tsk_mgmt_status = rsp->data[3];
> - complete(&target->tsk_mgmt_done);
> + ch->tsk_mgmt_status = rsp->data[3];
> + complete(&ch->tsk_mgmt_done);
> } else {
> - req = &target->req_ring[rsp->tag];
> - scmnd = srp_claim_req(target, req, NULL, NULL);
> + req = &ch->req_ring[rsp->tag];
> + scmnd = srp_claim_req(ch, req, NULL, NULL);
> if (!scmnd) {
> shost_printk(KERN_ERR, target->scsi_host,
> "Null scmnd for RSP w/tag %016llx\n",
> (unsigned long long) rsp->tag);
>
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += be32_to_cpu(rsp->req_lim_delta);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> return;
> }
> @@ -1651,7 +1679,7 @@ static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
> else if (unlikely(rsp->flags & SRP_RSP_FLAG_DOOVER))
> scsi_set_resid(scmnd, -be32_to_cpu(rsp->data_out_res_cnt));
>
> - srp_free_req(target, req, scmnd,
> + srp_free_req(ch, req, scmnd,
> be32_to_cpu(rsp->req_lim_delta));
>
> scmnd->host_scribble = NULL;
> @@ -1659,18 +1687,19 @@ static void srp_process_rsp(struct srp_target_port *target, struct srp_rsp *rsp)
> }
> }
>
> -static int srp_response_common(struct srp_target_port *target, s32 req_delta,
> +static int srp_response_common(struct srp_rdma_ch *ch, s32 req_delta,
> void *rsp, int len)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_device *dev = target->srp_host->srp_dev->dev;
> unsigned long flags;
> struct srp_iu *iu;
> int err;
>
> - spin_lock_irqsave(&target->lock, flags);
> - target->req_lim += req_delta;
> - iu = __srp_get_tx_iu(target, SRP_IU_RSP);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_lock_irqsave(&ch->lock, flags);
> + ch->req_lim += req_delta;
> + iu = __srp_get_tx_iu(ch, SRP_IU_RSP);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> if (!iu) {
> shost_printk(KERN_ERR, target->scsi_host, PFX
> @@ -1682,17 +1711,17 @@ static int srp_response_common(struct srp_target_port *target, s32 req_delta,
> memcpy(iu->buf, rsp, len);
> ib_dma_sync_single_for_device(dev, iu->dma, len, DMA_TO_DEVICE);
>
> - err = srp_post_send(target, iu, len);
> + err = srp_post_send(ch, iu, len);
> if (err) {
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "unable to post response: %d\n", err);
> - srp_put_tx_iu(target, iu, SRP_IU_RSP);
> + srp_put_tx_iu(ch, iu, SRP_IU_RSP);
> }
>
> return err;
> }
>
> -static void srp_process_cred_req(struct srp_target_port *target,
> +static void srp_process_cred_req(struct srp_rdma_ch *ch,
> struct srp_cred_req *req)
> {
> struct srp_cred_rsp rsp = {
> @@ -1701,14 +1730,15 @@ static void srp_process_cred_req(struct srp_target_port *target,
> };
> s32 delta = be32_to_cpu(req->req_lim_delta);
>
> - if (srp_response_common(target, delta, &rsp, sizeof rsp))
> - shost_printk(KERN_ERR, target->scsi_host, PFX
> + if (srp_response_common(ch, delta, &rsp, sizeof(rsp)))
> + shost_printk(KERN_ERR, ch->target->scsi_host, PFX
> "problems processing SRP_CRED_REQ\n");
> }
>
> -static void srp_process_aer_req(struct srp_target_port *target,
> +static void srp_process_aer_req(struct srp_rdma_ch *ch,
> struct srp_aer_req *req)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_aer_rsp rsp = {
> .opcode = SRP_AER_RSP,
> .tag = req->tag,
> @@ -1718,19 +1748,20 @@ static void srp_process_aer_req(struct srp_target_port *target,
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "ignoring AER for LUN %llu\n", be64_to_cpu(req->lun));
>
> - if (srp_response_common(target, delta, &rsp, sizeof rsp))
> + if (srp_response_common(ch, delta, &rsp, sizeof(rsp)))
> shost_printk(KERN_ERR, target->scsi_host, PFX
> "problems processing SRP_AER_REQ\n");
> }
>
> -static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
> +static void srp_handle_recv(struct srp_rdma_ch *ch, struct ib_wc *wc)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_device *dev = target->srp_host->srp_dev->dev;
> struct srp_iu *iu = (struct srp_iu *) (uintptr_t) wc->wr_id;
> int res;
> u8 opcode;
>
> - ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_ti_iu_len,
> + ib_dma_sync_single_for_cpu(dev, iu->dma, ch->max_ti_iu_len,
> DMA_FROM_DEVICE);
>
> opcode = *(u8 *) iu->buf;
> @@ -1744,15 +1775,15 @@ static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
>
> switch (opcode) {
> case SRP_RSP:
> - srp_process_rsp(target, iu->buf);
> + srp_process_rsp(ch, iu->buf);
> break;
>
> case SRP_CRED_REQ:
> - srp_process_cred_req(target, iu->buf);
> + srp_process_cred_req(ch, iu->buf);
> break;
>
> case SRP_AER_REQ:
> - srp_process_aer_req(target, iu->buf);
> + srp_process_aer_req(ch, iu->buf);
> break;
>
> case SRP_T_LOGOUT:
> @@ -1767,10 +1798,10 @@ static void srp_handle_recv(struct srp_target_port *target, struct ib_wc *wc)
> break;
> }
>
> - ib_dma_sync_single_for_device(dev, iu->dma, target->max_ti_iu_len,
> + ib_dma_sync_single_for_device(dev, iu->dma, ch->max_ti_iu_len,
> DMA_FROM_DEVICE);
>
> - res = srp_post_recv(target, iu);
> + res = srp_post_recv(ch, iu);
> if (res != 0)
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Recv failed with error code %d\n", res);
> @@ -1815,33 +1846,35 @@ static void srp_handle_qp_err(u64 wr_id, enum ib_wc_status wc_status,
> target->qp_in_error = true;
> }
>
> -static void srp_recv_completion(struct ib_cq *cq, void *target_ptr)
> +static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr)
> {
> - struct srp_target_port *target = target_ptr;
> + struct srp_rdma_ch *ch = ch_ptr;
> struct ib_wc wc;
>
> ib_req_notify_cq(cq, IB_CQ_NEXT_COMP);
> while (ib_poll_cq(cq, 1, &wc) > 0) {
> if (likely(wc.status == IB_WC_SUCCESS)) {
> - srp_handle_recv(target, &wc);
> + srp_handle_recv(ch, &wc);
> } else {
> - srp_handle_qp_err(wc.wr_id, wc.status, false, target);
> + srp_handle_qp_err(wc.wr_id, wc.status, false,
> + ch->target);
> }
> }
> }
>
> -static void srp_send_completion(struct ib_cq *cq, void *target_ptr)
> +static void srp_send_completion(struct ib_cq *cq, void *ch_ptr)
> {
> - struct srp_target_port *target = target_ptr;
> + struct srp_rdma_ch *ch = ch_ptr;
> struct ib_wc wc;
> struct srp_iu *iu;
>
> while (ib_poll_cq(cq, 1, &wc) > 0) {
> if (likely(wc.status == IB_WC_SUCCESS)) {
> iu = (struct srp_iu *) (uintptr_t) wc.wr_id;
> - list_add(&iu->list, &target->free_tx);
> + list_add(&iu->list, &ch->free_tx);
> } else {
> - srp_handle_qp_err(wc.wr_id, wc.status, true, target);
> + srp_handle_qp_err(wc.wr_id, wc.status, true,
> + ch->target);
> }
> }
> }
> @@ -1850,6 +1883,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(shost);
> struct srp_rport *rport = target->rport;
> + struct srp_rdma_ch *ch;
> struct srp_request *req;
> struct srp_iu *iu;
> struct srp_cmd *cmd;
> @@ -1871,14 +1905,16 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> if (unlikely(scmnd->result))
> goto err;
>
> - spin_lock_irqsave(&target->lock, flags);
> - iu = __srp_get_tx_iu(target, SRP_IU_CMD);
> + ch = &target->ch;
> +
> + spin_lock_irqsave(&ch->lock, flags);
> + iu = __srp_get_tx_iu(ch, SRP_IU_CMD);
> if (!iu)
> goto err_unlock;
>
> - req = list_first_entry(&target->free_reqs, struct srp_request, list);
> + req = list_first_entry(&ch->free_reqs, struct srp_request, list);
> list_del(&req->list);
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> dev = target->srp_host->srp_dev->dev;
> ib_dma_sync_single_for_cpu(dev, iu->dma, target->max_iu_len,
> @@ -1897,7 +1933,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> req->scmnd = scmnd;
> req->cmd = iu;
>
> - len = srp_map_data(scmnd, target, req);
> + len = srp_map_data(scmnd, ch, req);
> if (len < 0) {
> shost_printk(KERN_ERR, target->scsi_host,
> PFX "Failed to map data (%d)\n", len);
> @@ -1915,7 +1951,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> ib_dma_sync_single_for_device(dev, iu->dma, target->max_iu_len,
> DMA_TO_DEVICE);
>
> - if (srp_post_send(target, iu, len)) {
> + if (srp_post_send(ch, iu, len)) {
> shost_printk(KERN_ERR, target->scsi_host, PFX "Send failed\n");
> goto err_unmap;
> }
> @@ -1929,10 +1965,10 @@ unlock_rport:
> return ret;
>
> err_unmap:
> - srp_unmap_data(scmnd, target, req);
> + srp_unmap_data(scmnd, ch, req);
>
> err_iu:
> - srp_put_tx_iu(target, iu, SRP_IU_CMD);
> + srp_put_tx_iu(ch, iu, SRP_IU_CMD);
>
> /*
> * Avoid that the loops that iterate over the request ring can
> @@ -1940,11 +1976,11 @@ err_iu:
> */
> req->scmnd = NULL;
>
> - spin_lock_irqsave(&target->lock, flags);
> - list_add(&req->list, &target->free_reqs);
> + spin_lock_irqsave(&ch->lock, flags);
> + list_add(&req->list, &ch->free_reqs);
>
> err_unlock:
> - spin_unlock_irqrestore(&target->lock, flags);
> + spin_unlock_irqrestore(&ch->lock, flags);
>
> err:
> if (scmnd->result) {
> @@ -1959,53 +1995,54 @@ err:
>
> /*
> * Note: the resources allocated in this function are freed in
> - * srp_free_target_ib().
> + * srp_free_ch_ib().
> */
> -static int srp_alloc_iu_bufs(struct srp_target_port *target)
> +static int srp_alloc_iu_bufs(struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> int i;
>
> - target->rx_ring = kzalloc(target->queue_size * sizeof(*target->rx_ring),
> - GFP_KERNEL);
> - if (!target->rx_ring)
> + ch->rx_ring = kcalloc(target->queue_size, sizeof(*ch->rx_ring),
> + GFP_KERNEL);
> + if (!ch->rx_ring)
> goto err_no_ring;
> - target->tx_ring = kzalloc(target->queue_size * sizeof(*target->tx_ring),
> - GFP_KERNEL);
> - if (!target->tx_ring)
> + ch->tx_ring = kcalloc(target->queue_size, sizeof(*ch->tx_ring),
> + GFP_KERNEL);
> + if (!ch->tx_ring)
> goto err_no_ring;
>
> for (i = 0; i < target->queue_size; ++i) {
> - target->rx_ring[i] = srp_alloc_iu(target->srp_host,
> - target->max_ti_iu_len,
> - GFP_KERNEL, DMA_FROM_DEVICE);
> - if (!target->rx_ring[i])
> + ch->rx_ring[i] = srp_alloc_iu(target->srp_host,
> + ch->max_ti_iu_len,
> + GFP_KERNEL, DMA_FROM_DEVICE);
> + if (!ch->rx_ring[i])
> goto err;
> }
>
> for (i = 0; i < target->queue_size; ++i) {
> - target->tx_ring[i] = srp_alloc_iu(target->srp_host,
> - target->max_iu_len,
> - GFP_KERNEL, DMA_TO_DEVICE);
> - if (!target->tx_ring[i])
> + ch->tx_ring[i] = srp_alloc_iu(target->srp_host,
> + target->max_iu_len,
> + GFP_KERNEL, DMA_TO_DEVICE);
> + if (!ch->tx_ring[i])
> goto err;
>
> - list_add(&target->tx_ring[i]->list, &target->free_tx);
> + list_add(&ch->tx_ring[i]->list, &ch->free_tx);
> }
>
> return 0;
>
> err:
> for (i = 0; i < target->queue_size; ++i) {
> - srp_free_iu(target->srp_host, target->rx_ring[i]);
> - srp_free_iu(target->srp_host, target->tx_ring[i]);
> + srp_free_iu(target->srp_host, ch->rx_ring[i]);
> + srp_free_iu(target->srp_host, ch->tx_ring[i]);
> }
>
>
> err_no_ring:
> - kfree(target->tx_ring);
> - target->tx_ring = NULL;
> - kfree(target->rx_ring);
> - target->rx_ring = NULL;
> + kfree(ch->tx_ring);
> + ch->tx_ring = NULL;
> + kfree(ch->rx_ring);
> + ch->rx_ring = NULL;
>
> return -ENOMEM;
> }
> @@ -2039,23 +2076,24 @@ static uint32_t srp_compute_rq_tmo(struct ib_qp_attr *qp_attr, int attr_mask)
>
> static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
> struct srp_login_rsp *lrsp,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct ib_qp_attr *qp_attr = NULL;
> int attr_mask = 0;
> int ret;
> int i;
>
> if (lrsp->opcode == SRP_LOGIN_RSP) {
> - target->max_ti_iu_len = be32_to_cpu(lrsp->max_ti_iu_len);
> - target->req_lim = be32_to_cpu(lrsp->req_lim_delta);
> + ch->max_ti_iu_len = be32_to_cpu(lrsp->max_ti_iu_len);
> + ch->req_lim = be32_to_cpu(lrsp->req_lim_delta);
>
> /*
> * Reserve credits for task management so we don't
> * bounce requests back to the SCSI mid-layer.
> */
> target->scsi_host->can_queue
> - = min(target->req_lim - SRP_TSK_MGMT_SQ_SIZE,
> + = min(ch->req_lim - SRP_TSK_MGMT_SQ_SIZE,
> target->scsi_host->can_queue);
> target->scsi_host->cmd_per_lun
> = min_t(int, target->scsi_host->can_queue,
> @@ -2067,8 +2105,8 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
> goto error;
> }
>
> - if (!target->rx_ring) {
> - ret = srp_alloc_iu_bufs(target);
> + if (!ch->rx_ring) {
> + ret = srp_alloc_iu_bufs(ch);
> if (ret)
> goto error;
> }
> @@ -2083,13 +2121,14 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
> if (ret)
> goto error_free;
>
> - ret = ib_modify_qp(target->qp, qp_attr, attr_mask);
> + ret = ib_modify_qp(ch->qp, qp_attr, attr_mask);
> if (ret)
> goto error_free;
>
> for (i = 0; i < target->queue_size; i++) {
> - struct srp_iu *iu = target->rx_ring[i];
> - ret = srp_post_recv(target, iu);
> + struct srp_iu *iu = ch->rx_ring[i];
> +
> + ret = srp_post_recv(ch, iu);
> if (ret)
> goto error_free;
> }
> @@ -2101,7 +2140,7 @@ static void srp_cm_rep_handler(struct ib_cm_id *cm_id,
>
> target->rq_tmo_jiffies = srp_compute_rq_tmo(qp_attr, attr_mask);
>
> - ret = ib_modify_qp(target->qp, qp_attr, attr_mask);
> + ret = ib_modify_qp(ch->qp, qp_attr, attr_mask);
> if (ret)
> goto error_free;
>
> @@ -2111,13 +2150,14 @@ error_free:
> kfree(qp_attr);
>
> error:
> - target->status = ret;
> + ch->status = ret;
> }
>
> static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> struct ib_cm_event *event,
> - struct srp_target_port *target)
> + struct srp_rdma_ch *ch)
> {
> + struct srp_target_port *target = ch->target;
> struct Scsi_Host *shost = target->scsi_host;
> struct ib_class_port_info *cpi;
> int opcode;
> @@ -2125,12 +2165,12 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> switch (event->param.rej_rcvd.reason) {
> case IB_CM_REJ_PORT_CM_REDIRECT:
> cpi = event->param.rej_rcvd.ari;
> - target->path.dlid = cpi->redirect_lid;
> - target->path.pkey = cpi->redirect_pkey;
> + ch->path.dlid = cpi->redirect_lid;
> + ch->path.pkey = cpi->redirect_pkey;
> cm_id->remote_cm_qpn = be32_to_cpu(cpi->redirect_qp) & 0x00ffffff;
> - memcpy(target->path.dgid.raw, cpi->redirect_gid, 16);
> + memcpy(ch->path.dgid.raw, cpi->redirect_gid, 16);
>
> - target->status = target->path.dlid ?
> + ch->status = ch->path.dlid ?
> SRP_DLID_REDIRECT : SRP_PORT_REDIRECT;
> break;
>
> @@ -2141,26 +2181,26 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> * reject reason code 25 when they mean 24
> * (port redirect).
> */
> - memcpy(target->path.dgid.raw,
> + memcpy(ch->path.dgid.raw,
> event->param.rej_rcvd.ari, 16);
>
> shost_printk(KERN_DEBUG, shost,
> PFX "Topspin/Cisco redirect to target port GID %016llx%016llx\n",
> - (unsigned long long) be64_to_cpu(target->path.dgid.global.subnet_prefix),
> - (unsigned long long) be64_to_cpu(target->path.dgid.global.interface_id));
> + be64_to_cpu(ch->path.dgid.global.subnet_prefix),
> + be64_to_cpu(ch->path.dgid.global.interface_id));
>
> - target->status = SRP_PORT_REDIRECT;
> + ch->status = SRP_PORT_REDIRECT;
> } else {
> shost_printk(KERN_WARNING, shost,
> " REJ reason: IB_CM_REJ_PORT_REDIRECT\n");
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> }
> break;
>
> case IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID:
> shost_printk(KERN_WARNING, shost,
> " REJ reason: IB_CM_REJ_DUPLICATE_LOCAL_COMM_ID\n");
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> break;
>
> case IB_CM_REJ_CONSUMER_DEFINED:
> @@ -2175,30 +2215,31 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
> else
> shost_printk(KERN_WARNING, shost, PFX
> "SRP LOGIN from %pI6 to %pI6 REJECTED, reason 0x%08x\n",
> - target->path.sgid.raw,
> - target->orig_dgid, reason);
> + target->sgid.raw,
> + target->orig_dgid.raw, reason);
> } else
> shost_printk(KERN_WARNING, shost,
> " REJ reason: IB_CM_REJ_CONSUMER_DEFINED,"
> " opcode 0x%02x\n", opcode);
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> break;
>
> case IB_CM_REJ_STALE_CONN:
> shost_printk(KERN_WARNING, shost, " REJ reason: stale connection\n");
> - target->status = SRP_STALE_CONN;
> + ch->status = SRP_STALE_CONN;
> break;
>
> default:
> shost_printk(KERN_WARNING, shost, " REJ reason 0x%x\n",
> event->param.rej_rcvd.reason);
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> }
> }
>
> static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> {
> - struct srp_target_port *target = cm_id->context;
> + struct srp_rdma_ch *ch = cm_id->context;
> + struct srp_target_port *target = ch->target;
> int comp = 0;
>
> switch (event->event) {
> @@ -2206,19 +2247,19 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> shost_printk(KERN_DEBUG, target->scsi_host,
> PFX "Sending CM REQ failed\n");
> comp = 1;
> - target->status = -ECONNRESET;
> + ch->status = -ECONNRESET;
> break;
>
> case IB_CM_REP_RECEIVED:
> comp = 1;
> - srp_cm_rep_handler(cm_id, event->private_data, target);
> + srp_cm_rep_handler(cm_id, event->private_data, ch);
> break;
>
> case IB_CM_REJ_RECEIVED:
> shost_printk(KERN_DEBUG, target->scsi_host, PFX "REJ received\n");
> comp = 1;
>
> - srp_cm_rej_handler(cm_id, event, target);
> + srp_cm_rej_handler(cm_id, event, ch);
> break;
>
> case IB_CM_DREQ_RECEIVED:
> @@ -2236,7 +2277,7 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> PFX "connection closed\n");
> comp = 1;
>
> - target->status = 0;
> + ch->status = 0;
> break;
>
> case IB_CM_MRA_RECEIVED:
> @@ -2251,7 +2292,7 @@ static int srp_cm_handler(struct ib_cm_id *cm_id, struct ib_cm_event *event)
> }
>
> if (comp)
> - complete(&target->done);
> + complete(&ch->done);
>
> return 0;
> }
> @@ -2307,9 +2348,10 @@ srp_change_queue_depth(struct scsi_device *sdev, int qdepth, int reason)
> return sdev->queue_depth;
> }
>
> -static int srp_send_tsk_mgmt(struct srp_target_port *target,
> - u64 req_tag, unsigned int lun, u8 func)
> +static int srp_send_tsk_mgmt(struct srp_rdma_ch *ch, u64 req_tag,
> + unsigned int lun, u8 func)
> {
> + struct srp_target_port *target = ch->target;
> struct srp_rport *rport = target->rport;
> struct ib_device *dev = target->srp_host->srp_dev->dev;
> struct srp_iu *iu;
> @@ -2318,16 +2360,16 @@ static int srp_send_tsk_mgmt(struct srp_target_port *target,
> if (!target->connected || target->qp_in_error)
> return -1;
>
> - init_completion(&target->tsk_mgmt_done);
> + init_completion(&ch->tsk_mgmt_done);
>
> /*
> - * Lock the rport mutex to avoid that srp_create_target_ib() is
> + * Lock the rport mutex to avoid that srp_create_ch_ib() is
> * invoked while a task management function is being sent.
> */
> mutex_lock(&rport->mutex);
> - spin_lock_irq(&target->lock);
> - iu = __srp_get_tx_iu(target, SRP_IU_TSK_MGMT);
> - spin_unlock_irq(&target->lock);
> + spin_lock_irq(&ch->lock);
> + iu = __srp_get_tx_iu(ch, SRP_IU_TSK_MGMT);
> + spin_unlock_irq(&ch->lock);
>
> if (!iu) {
> mutex_unlock(&rport->mutex);
> @@ -2348,15 +2390,15 @@ static int srp_send_tsk_mgmt(struct srp_target_port *target,
>
> ib_dma_sync_single_for_device(dev, iu->dma, sizeof *tsk_mgmt,
> DMA_TO_DEVICE);
> - if (srp_post_send(target, iu, sizeof *tsk_mgmt)) {
> - srp_put_tx_iu(target, iu, SRP_IU_TSK_MGMT);
> + if (srp_post_send(ch, iu, sizeof(*tsk_mgmt))) {
> + srp_put_tx_iu(ch, iu, SRP_IU_TSK_MGMT);
> mutex_unlock(&rport->mutex);
>
> return -1;
> }
> mutex_unlock(&rport->mutex);
>
> - if (!wait_for_completion_timeout(&target->tsk_mgmt_done,
> + if (!wait_for_completion_timeout(&ch->tsk_mgmt_done,
> msecs_to_jiffies(SRP_ABORT_TIMEOUT_MS)))
> return -1;
>
> @@ -2367,20 +2409,22 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> struct srp_request *req = (struct srp_request *) scmnd->host_scribble;
> + struct srp_rdma_ch *ch;
> int ret;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP abort called\n");
>
> - if (!req || !srp_claim_req(target, req, NULL, scmnd))
> + ch = &target->ch;
> + if (!req || !srp_claim_req(ch, req, NULL, scmnd))
> return SUCCESS;
> - if (srp_send_tsk_mgmt(target, req->index, scmnd->device->lun,
> + if (srp_send_tsk_mgmt(ch, req->index, scmnd->device->lun,
> SRP_TSK_ABORT_TASK) == 0)
> ret = SUCCESS;
> else if (target->rport->state == SRP_RPORT_LOST)
> ret = FAST_IO_FAIL;
> else
> ret = FAILED;
> - srp_free_req(target, req, scmnd, 0);
> + srp_free_req(ch, req, scmnd, 0);
> scmnd->result = DID_ABORT << 16;
> scmnd->scsi_done(scmnd);
>
> @@ -2390,19 +2434,21 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> static int srp_reset_device(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> + struct srp_rdma_ch *ch = &target->ch;
> int i;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n");
>
> - if (srp_send_tsk_mgmt(target, SRP_TAG_NO_REQ, scmnd->device->lun,
> + if (srp_send_tsk_mgmt(ch, SRP_TAG_NO_REQ, scmnd->device->lun,
> SRP_TSK_LUN_RESET))
> return FAILED;
> - if (target->tsk_mgmt_status)
> + if (ch->tsk_mgmt_status)
> return FAILED;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &target->req_ring[i];
> - srp_finish_req(target, req, scmnd->device, DID_RESET << 16);
> + struct srp_request *req = &ch->req_ring[i];
> +
> + srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
> }
>
> return SUCCESS;
> @@ -2464,7 +2510,7 @@ static ssize_t show_pkey(struct device *dev, struct device_attribute *attr,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "0x%04x\n", be16_to_cpu(target->path.pkey));
> + return sprintf(buf, "0x%04x\n", be16_to_cpu(target->pkey));
> }
>
> static ssize_t show_sgid(struct device *dev, struct device_attribute *attr,
> @@ -2472,15 +2518,16 @@ static ssize_t show_sgid(struct device *dev, struct device_attribute *attr,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "%pI6\n", target->path.sgid.raw);
> + return sprintf(buf, "%pI6\n", target->sgid.raw);
> }
>
> static ssize_t show_dgid(struct device *dev, struct device_attribute *attr,
> char *buf)
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
> + struct srp_rdma_ch *ch = &target->ch;
>
> - return sprintf(buf, "%pI6\n", target->path.dgid.raw);
> + return sprintf(buf, "%pI6\n", ch->path.dgid.raw);
> }
>
> static ssize_t show_orig_dgid(struct device *dev,
> @@ -2488,7 +2535,7 @@ static ssize_t show_orig_dgid(struct device *dev,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "%pI6\n", target->orig_dgid);
> + return sprintf(buf, "%pI6\n", target->orig_dgid.raw);
> }
>
> static ssize_t show_req_lim(struct device *dev,
> @@ -2496,7 +2543,7 @@ static ssize_t show_req_lim(struct device *dev,
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
>
> - return sprintf(buf, "%d\n", target->req_lim);
> + return sprintf(buf, "%d\n", target->ch.req_lim);
> }
>
> static ssize_t show_zero_req_lim(struct device *dev,
> @@ -2778,7 +2825,7 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> int opt_mask = 0;
> int token;
> int ret = -EINVAL;
> - int i;
> + int i, b;
>
> options = kstrdup(buf, GFP_KERNEL);
> if (!options)
> @@ -2826,11 +2873,15 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> }
>
> for (i = 0; i < 16; ++i) {
> - strlcpy(dgid, p + i * 2, 3);
> - target->path.dgid.raw[i] = simple_strtoul(dgid, NULL, 16);
> + strlcpy(dgid, p + i * 2, sizeof(dgid));
> + if (sscanf(dgid, "%x", &b) < 1) {
> + ret = -EINVAL;
> + kfree(p);
> + goto out;
> + }
> + target->orig_dgid.raw[i] = b;
> }
> kfree(p);
> - memcpy(target->orig_dgid, target->path.dgid.raw, 16);
> break;
>
> case SRP_OPT_PKEY:
> @@ -2838,7 +2889,7 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> pr_warn("bad P_Key parameter '%s'\n", p);
> goto out;
> }
> - target->path.pkey = cpu_to_be16(token);
> + target->pkey = cpu_to_be16(token);
> break;
>
> case SRP_OPT_SERVICE_ID:
> @@ -2848,7 +2899,6 @@ static int srp_parse_options(const char *buf, struct srp_target_port *target)
> goto out;
> }
> target->service_id = cpu_to_be64(simple_strtoull(p, NULL, 16));
> - target->path.service_id = target->service_id;
> kfree(p);
> break;
>
> @@ -2985,6 +3035,7 @@ static ssize_t srp_create_target(struct device *dev,
> container_of(dev, struct srp_host, dev);
> struct Scsi_Host *target_host;
> struct srp_target_port *target;
> + struct srp_rdma_ch *ch;
> struct srp_device *srp_dev = host->srp_dev;
> struct ib_device *ibdev = srp_dev->dev;
> int ret;
> @@ -3047,24 +3098,28 @@ static ssize_t srp_create_target(struct device *dev,
> INIT_WORK(&target->tl_err_work, srp_tl_err_work);
> INIT_WORK(&target->remove_work, srp_remove_work);
> spin_lock_init(&target->lock);
Hey Bart,
You keep this target-lock around. I don't see in this patch any usage
of it left. Can you document what each of target->lock and ch->lock
protects?
So I would imagine that target-lock manages the target-wide variables
such as state (what else?) while ch->lock protect channel specific free
requests pool which can be moved to blk-mq preallocs and avoid
maintaining it (not sure how req_lim would be maintained though and if
it is still needed?).
Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-19 13:00 ` [PATCH 8/8] IB/srp: Add multichannel support Bart Van Assche
[not found] ` <541C28E0.7010705-HInyCGIudOg@public.gmane.org>
@ 2014-09-23 16:32 ` Sagi Grimberg
[not found] ` <5421A093.1070203-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
1 sibling, 1 reply; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-23 16:32 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/19/2014 4:00 PM, Bart Van Assche wrote:
> Improve performance by using multiple RDMA/RC channels per SCSI host
> for communicating with an SRP target.
>
Hey Bart,
Since you don't seem to negotiate/declare multichannel with the target,
did you test this code with some target implementations other than SCST
that happen to be out there?
Overall, I think this patch would be easier to review if you also
provide a list of logical changes (which obviously are introduced in
this patch). Patch 7/8 can use some more information of target-channel
relations as well.
> Signed-off-by: Bart Van Assche <bvanassche@acm.org>
> ---
> Documentation/ABI/stable/sysfs-driver-ib_srp | 25 +-
> drivers/infiniband/ulp/srp/ib_srp.c | 337 ++++++++++++++++++++-------
> drivers/infiniband/ulp/srp/ib_srp.h | 20 +-
> 3 files changed, 287 insertions(+), 95 deletions(-)
>
> diff --git a/Documentation/ABI/stable/sysfs-driver-ib_srp b/Documentation/ABI/stable/sysfs-driver-ib_srp
> index b9688de..d5a459e 100644
> --- a/Documentation/ABI/stable/sysfs-driver-ib_srp
> +++ b/Documentation/ABI/stable/sysfs-driver-ib_srp
> @@ -55,12 +55,12 @@ Description: Interface for making ib_srp connect to a new target.
> only safe with partial memory descriptor list support enabled
> (allow_ext_sg=1).
> * comp_vector, a number in the range 0..n-1 specifying the
> - MSI-X completion vector. Some HCA's allocate multiple (n)
> - MSI-X vectors per HCA port. If the IRQ affinity masks of
> - these interrupts have been configured such that each MSI-X
> - interrupt is handled by a different CPU then the comp_vector
> - parameter can be used to spread the SRP completion workload
> - over multiple CPU's.
> + MSI-X completion vector of the first RDMA channel. Some
> + HCA's allocate multiple (n) MSI-X vectors per HCA port. If
> + the IRQ affinity masks of these interrupts have been
> + configured such that each MSI-X interrupt is handled by a
> + different CPU then the comp_vector parameter can be used to
> + spread the SRP completion workload over multiple CPU's.
Why do you want the first channel vector placement? Why can't you start
with obvious 0?
> * tl_retry_count, a number in the range 2..7 specifying the
> IB RC retry count.
> * queue_size, the maximum number of commands that the
> @@ -88,6 +88,13 @@ Description: Whether ib_srp is allowed to include a partial memory
> descriptor list in an SRP_CMD when communicating with an SRP
> target.
>
> +What: /sys/class/scsi_host/host<n>/ch_count
> +Date: November 1, 2014
> +KernelVersion: 3.18
> +Contact: linux-rdma@vger.kernel.org
> +Description: Number of RDMA channels used for communication with the SRP
> + target.
> +
> What: /sys/class/scsi_host/host<n>/cmd_sg_entries
> Date: May 19, 2011
> KernelVersion: 2.6.39
> @@ -95,6 +102,12 @@ Contact: linux-rdma@vger.kernel.org
> Description: Maximum number of data buffer descriptors that may be sent to
> the target in a single SRP_CMD request.
>
> +What: /sys/class/scsi_host/host<n>/comp_vector
> +Date: September 2, 2013
> +KernelVersion: 3.11
> +Contact: linux-rdma@vger.kernel.org
> +Description: Completion vector used for the first RDMA channel.
> +
> What: /sys/class/scsi_host/host<n>/dgid
> Date: June 17, 2006
> KernelVersion: 2.6.17
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
> index 9feeea1..58ca618 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.c
> +++ b/drivers/infiniband/ulp/srp/ib_srp.c
> @@ -123,6 +123,16 @@ MODULE_PARM_DESC(dev_loss_tmo,
> " if fast_io_fail_tmo has not been set. \"off\" means that"
> " this functionality is disabled.");
>
> +static unsigned ch_count;
> +module_param(ch_count, uint, 0444);
> +MODULE_PARM_DESC(ch_count,
> + "Number of RDMA channels to use for communication with an SRP"
> + " target. Using more than one channel improves performance"
> + " if the HCA supports multiple completion vectors. The"
> + " default value is the minimum of four times the number of"
> + " online CPU sockets and the number of completion vectors"
> + " supported by the HCA.");
> +
Can you explain the default math? how did you end-up with 4*numa_nodes?
wouldn't per-cpu be a better fit?
Moreover, while using multiple channels you don't suffice for less
requests of less FMRs/FRs. I'm a bit concerned here about scalability
of multi-channel.
Should we take care of cases where the user will want lots of channels
to lots of targets and might run out of resources?
> static void srp_add_one(struct ib_device *device);
> static void srp_remove_one(struct ib_device *device);
> static void srp_recv_completion(struct ib_cq *cq, void *ch_ptr);
> @@ -556,17 +566,32 @@ err:
> * Note: this function may be called without srp_alloc_iu_bufs() having been
> * invoked. Hence the ch->[rt]x_ring checks.
> */
> -static void srp_free_ch_ib(struct srp_rdma_ch *ch)
> +static void srp_free_ch_ib(struct srp_target_port *target,
> + struct srp_rdma_ch *ch)
> {
> - struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> int i;
>
> + if (!ch->target)
> + return;
How did this condition pop up here? As I don't feel this is a trivial
condition, I would like to see a comment of how can this routine be
called twice and why is this peek safe?
> +
> + /*
> + * Avoid that the SCSI error handler tries to use this channel after
> + * it has been freed. The SCSI error handler can namely continue
> + * trying to perform recovery actions after scsi_remove_host()
> + * returned.
> + */
> + ch->target = NULL;
> +
> if (ch->cm_id) {
> ib_destroy_cm_id(ch->cm_id);
> ch->cm_id = NULL;
> }
>
> + /* If srp_new_cm_id() succeeded but srp_create_ch_ib() not, return. */
> + if (!ch->qp)
> + return;
> +
> if (dev->use_fast_reg) {
> if (ch->fr_pool)
> srp_destroy_fr_pool(ch->fr_pool);
> @@ -647,7 +672,7 @@ static int srp_lookup_path(struct srp_rdma_ch *ch)
> return ch->status;
> }
>
> -static int srp_send_req(struct srp_rdma_ch *ch)
> +static int srp_send_req(struct srp_rdma_ch *ch, bool multich)
> {
> struct srp_target_port *target = ch->target;
> struct {
> @@ -688,6 +713,8 @@ static int srp_send_req(struct srp_rdma_ch *ch)
> req->priv.req_it_iu_len = cpu_to_be32(target->max_iu_len);
> req->priv.req_buf_fmt = cpu_to_be16(SRP_BUF_FORMAT_DIRECT |
> SRP_BUF_FORMAT_INDIRECT);
> + req->priv.req_flags = (multich ? SRP_MULTICHAN_MULTI :
> + SRP_MULTICHAN_SINGLE);
> /*
> * In the published SRP specification (draft rev. 16a), the
> * port identifier format is 8 bytes of ID extension followed
> @@ -769,27 +796,31 @@ static bool srp_change_conn_state(struct srp_target_port *target,
>
> static void srp_disconnect_target(struct srp_target_port *target)
> {
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> + int i;
>
> if (srp_change_conn_state(target, false)) {
> /* XXX should send SRP_I_LOGOUT request */
>
> - if (ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
> - shost_printk(KERN_DEBUG, target->scsi_host,
> - PFX "Sending CM DREQ failed\n");
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + if (ch->cm_id && ib_send_cm_dreq(ch->cm_id, NULL, 0)) {
> + shost_printk(KERN_DEBUG, target->scsi_host,
> + PFX "Sending CM DREQ failed\n");
> + }
> }
> }
> }
>
> -static void srp_free_req_data(struct srp_rdma_ch *ch)
> +static void srp_free_req_data(struct srp_target_port *target,
> + struct srp_rdma_ch *ch)
> {
> - struct srp_target_port *target = ch->target;
> struct srp_device *dev = target->srp_host->srp_dev;
> struct ib_device *ibdev = dev->dev;
> struct srp_request *req;
> int i;
>
> - if (!ch->req_ring)
> + if (!ch->target || !ch->req_ring)
> return;
>
> for (i = 0; i < target->req_ring_size; ++i) {
> @@ -853,7 +884,7 @@ static int srp_alloc_req_data(struct srp_rdma_ch *ch)
> goto out;
>
> req->indirect_dma_addr = dma_addr;
> - req->index = i;
> + req->tag = build_srp_tag(ch - target->ch, i);
> list_add_tail(&req->list, &ch->free_reqs);
> }
> ret = 0;
> @@ -879,7 +910,8 @@ static void srp_del_scsi_host_attr(struct Scsi_Host *shost)
>
> static void srp_remove_target(struct srp_target_port *target)
> {
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> + int i;
>
> WARN_ON_ONCE(target->state != SRP_TARGET_REMOVED);
>
> @@ -889,10 +921,18 @@ static void srp_remove_target(struct srp_target_port *target)
> scsi_remove_host(target->scsi_host);
> srp_stop_rport_timers(target->rport);
> srp_disconnect_target(target);
> - srp_free_ch_ib(ch);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + srp_free_ch_ib(target, ch);
> + }
> cancel_work_sync(&target->tl_err_work);
> srp_rport_put(target->rport);
> - srp_free_req_data(ch);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + srp_free_req_data(target, ch);
> + }
> + kfree(target->ch);
> + target->ch = NULL;
>
> spin_lock(&target->srp_host->target_lock);
> list_del(&target->list);
> @@ -918,12 +958,12 @@ static void srp_rport_delete(struct srp_rport *rport)
> srp_queue_remove_work(target);
> }
>
> -static int srp_connect_ch(struct srp_rdma_ch *ch)
> +static int srp_connect_ch(struct srp_rdma_ch *ch, bool multich)
> {
> struct srp_target_port *target = ch->target;
> int ret;
>
> - WARN_ON_ONCE(target->connected);
> + WARN_ON_ONCE(!multich && target->connected);
>
> target->qp_in_error = false;
>
> @@ -933,7 +973,7 @@ static int srp_connect_ch(struct srp_rdma_ch *ch)
>
> while (1) {
> init_completion(&ch->done);
> - ret = srp_send_req(ch);
> + ret = srp_send_req(ch, multich);
> if (ret)
> return ret;
> ret = wait_for_completion_interruptible(&ch->done);
> @@ -1095,10 +1135,10 @@ static void srp_finish_req(struct srp_rdma_ch *ch, struct srp_request *req,
> static void srp_terminate_io(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> struct Scsi_Host *shost = target->scsi_host;
> struct scsi_device *sdev;
> - int i;
> + int i, j;
>
> /*
> * Invoking srp_terminate_io() while srp_queuecommand() is running
> @@ -1107,10 +1147,15 @@ static void srp_terminate_io(struct srp_rport *rport)
> shost_for_each_device(sdev, shost)
> WARN_ON_ONCE(sdev->request_queue->request_fn_active);
>
> - for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &ch->req_ring[i];
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> +
> + for (j = 0; j < target->req_ring_size; ++j) {
> + struct srp_request *req = &ch->req_ring[j];
>
> - srp_finish_req(ch, req, NULL, DID_TRANSPORT_FAILFAST << 16);
> + srp_finish_req(ch, req, NULL,
> + DID_TRANSPORT_FAILFAST << 16);
> + }
> }
> }
>
> @@ -1126,8 +1171,9 @@ static void srp_terminate_io(struct srp_rport *rport)
> static int srp_rport_reconnect(struct srp_rport *rport)
> {
> struct srp_target_port *target = rport->lld_data;
> - struct srp_rdma_ch *ch = &target->ch;
> - int i, ret;
> + struct srp_rdma_ch *ch;
> + int i, j, ret = 0;
> + bool multich = false;
>
> srp_disconnect_target(target);
>
> @@ -1139,27 +1185,43 @@ static int srp_rport_reconnect(struct srp_rport *rport)
> * case things are really fouled up. Doing so also ensures that all CM
> * callbacks will have finished before a new QP is allocated.
> */
> - ret = srp_new_cm_id(ch);
> -
> - for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &ch->req_ring[i];
> -
> - srp_finish_req(ch, req, NULL, DID_RESET << 16);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + if (!ch->target)
> + return -ENODEV;
> + ret += srp_new_cm_id(ch);
> + }
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + for (j = 0; j < target->req_ring_size; ++j) {
> + struct srp_request *req = &ch->req_ring[j];
> +
> + srp_finish_req(ch, req, NULL, DID_RESET << 16);
> + }
> }
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + /*
> + * Whether or not creating a new CM ID succeeded, create a new
> + * QP. This guarantees that all completion callback function
> + * invocations have finished before request resetting starts.
> + */
> + ret += srp_create_ch_ib(ch);
>
> - /*
> - * Whether or not creating a new CM ID succeeded, create a new
> - * QP. This guarantees that all callback functions for the old QP have
> - * finished before any send requests are posted on the new QP.
> - */
> - ret += srp_create_ch_ib(ch);
> -
> - INIT_LIST_HEAD(&ch->free_tx);
> - for (i = 0; i < target->queue_size; ++i)
> - list_add(&ch->tx_ring[i]->list, &ch->free_tx);
> -
> - if (ret == 0)
> - ret = srp_connect_ch(ch);
> + INIT_LIST_HEAD(&ch->free_tx);
> + for (j = 0; j < target->queue_size; ++j)
> + list_add(&ch->tx_ring[j]->list, &ch->free_tx);
> + }
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + if (ret) {
> + if (i > 1)
> + ret = 0;
> + break;
> + }
> + ret = srp_connect_ch(ch, multich);
> + multich = true;
> + }
>
> if (ret == 0)
> shost_printk(KERN_INFO, target->scsi_host,
> @@ -1573,7 +1635,7 @@ static struct srp_iu *__srp_get_tx_iu(struct srp_rdma_ch *ch,
> s32 rsv = (iu_type == SRP_IU_TSK_MGMT) ? 0 : SRP_TSK_MGMT_SQ_SIZE;
> struct srp_iu *iu;
>
> - srp_send_completion(ch->send_cq, target);
> + srp_send_completion(ch->send_cq, ch);
>
> if (list_empty(&ch->free_tx))
> return NULL;
> @@ -1637,6 +1699,7 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
> struct srp_request *req;
> struct scsi_cmnd *scmnd;
> unsigned long flags;
> + unsigned i;
>
> if (unlikely(rsp->tag & SRP_TAG_TSK_MGMT)) {
> spin_lock_irqsave(&ch->lock, flags);
> @@ -1648,12 +1711,20 @@ static void srp_process_rsp(struct srp_rdma_ch *ch, struct srp_rsp *rsp)
> ch->tsk_mgmt_status = rsp->data[3];
> complete(&ch->tsk_mgmt_done);
> } else {
> - req = &ch->req_ring[rsp->tag];
> - scmnd = srp_claim_req(ch, req, NULL, NULL);
> + if (srp_tag_ch(rsp->tag) != ch - target->ch)
> + pr_err("Channel idx mismatch: tag %#llx <> ch %#lx\n",
> + rsp->tag, ch - target->ch);
> + i = srp_tag_idx(rsp->tag);
> + if (i < target->req_ring_size) {
> + req = &ch->req_ring[i];
> + scmnd = srp_claim_req(ch, req, NULL, NULL);
> + } else {
> + scmnd = NULL;
> + }
> if (!scmnd) {
> shost_printk(KERN_ERR, target->scsi_host,
> - "Null scmnd for RSP w/tag %016llx\n",
> - (unsigned long long) rsp->tag);
> + "Null scmnd for RSP w/tag %#016llx received on ch %ld / QP %#x\n",
> + rsp->tag, ch - target->ch, ch->qp->qp_num);
>
> spin_lock_irqsave(&ch->lock, flags);
> ch->req_lim += be32_to_cpu(rsp->req_lim_delta);
> @@ -1879,7 +1950,8 @@ static void srp_send_completion(struct ib_cq *cq, void *ch_ptr)
> }
> }
>
> -static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> +static int srp_queuecommand(unsigned hwq, struct Scsi_Host *shost,
> + struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(shost);
> struct srp_rport *rport = target->rport;
> @@ -1905,7 +1977,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> if (unlikely(scmnd->result))
> goto err;
>
> - ch = &target->ch;
> + ch = &target->ch[hwq];
>
> spin_lock_irqsave(&ch->lock, flags);
> iu = __srp_get_tx_iu(ch, SRP_IU_CMD);
> @@ -1927,7 +1999,7 @@ static int srp_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
>
> cmd->opcode = SRP_CMD;
> cmd->lun = cpu_to_be64((u64) scmnd->device->lun << 48);
> - cmd->tag = req->index;
> + cmd->tag = req->tag;
> memcpy(cmd->cdb, scmnd->cmnd, scmnd->cmd_len);
>
> req->scmnd = scmnd;
> @@ -1993,6 +2065,17 @@ err:
> goto unlock_rport;
> }
>
> +static int srp_sq_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *scmnd)
> +{
> + return srp_queuecommand(0, shost, scmnd);
> +}
> +
> +static int srp_mq_queuecommand(struct blk_mq_hw_ctx *hctx,
> + struct scsi_cmnd *scmnd)
> +{
> + return srp_queuecommand(hctx->queue_num, scmnd->device->host, scmnd);
> +}
> +
> /*
> * Note: the resources allocated in this function are freed in
> * srp_free_ch_ib().
> @@ -2409,15 +2492,23 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> struct srp_request *req = (struct srp_request *) scmnd->host_scribble;
> + u16 ch_idx;
> struct srp_rdma_ch *ch;
> int ret;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP abort called\n");
>
> - ch = &target->ch;
> - if (!req || !srp_claim_req(ch, req, NULL, scmnd))
> + if (!req)
> + return SUCCESS;
> + ch_idx = srp_tag_ch(req->tag);
> + if (WARN_ON_ONCE(ch_idx >= target->ch_count))
Can you explain how can this happen? and why do you continue as if
nothing happened?
> return SUCCESS;
> - if (srp_send_tsk_mgmt(ch, req->index, scmnd->device->lun,
> + ch = &target->ch[ch_idx];
> + if (!srp_claim_req(ch, req, NULL, scmnd))
> + return SUCCESS;
> + shost_printk(KERN_ERR, target->scsi_host,
> + "Sending SRP abort for tag %#x\n", req->tag);
> + if (srp_send_tsk_mgmt(ch, req->tag, scmnd->device->lun,
> SRP_TSK_ABORT_TASK) == 0)
> ret = SUCCESS;
> else if (target->rport->state == SRP_RPORT_LOST)
> @@ -2434,21 +2525,25 @@ static int srp_abort(struct scsi_cmnd *scmnd)
> static int srp_reset_device(struct scsi_cmnd *scmnd)
> {
> struct srp_target_port *target = host_to_target(scmnd->device->host);
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch;
> int i;
>
> shost_printk(KERN_ERR, target->scsi_host, "SRP reset_device called\n");
>
> + ch = &target->ch[0];
> if (srp_send_tsk_mgmt(ch, SRP_TAG_NO_REQ, scmnd->device->lun,
> SRP_TSK_LUN_RESET))
> return FAILED;
> if (ch->tsk_mgmt_status)
> return FAILED;
>
> - for (i = 0; i < target->req_ring_size; ++i) {
> - struct srp_request *req = &ch->req_ring[i];
> + for (i = 0; i < target->ch_count; i++) {
Just a Nit, This channels loop appears several times in the code.
Might be nicer to macro it to for_each_rdma_ch() - not a must though...
> + ch = &target->ch[i];
> + for (i = 0; i < target->req_ring_size; ++i) {
> + struct srp_request *req = &ch->req_ring[i];
>
> - srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
> + srp_finish_req(ch, req, scmnd->device, DID_RESET << 16);
> + }
> }
>
> return SUCCESS;
> @@ -2525,7 +2620,7 @@ static ssize_t show_dgid(struct device *dev, struct device_attribute *attr,
> char *buf)
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
> - struct srp_rdma_ch *ch = &target->ch;
> + struct srp_rdma_ch *ch = &target->ch[0];
>
> return sprintf(buf, "%pI6\n", ch->path.dgid.raw);
> }
> @@ -2542,8 +2637,14 @@ static ssize_t show_req_lim(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> struct srp_target_port *target = host_to_target(class_to_shost(dev));
> + struct srp_rdma_ch *ch;
> + int i, req_lim = INT_MAX;
>
> - return sprintf(buf, "%d\n", target->ch.req_lim);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
No lock here?
> + req_lim = min(req_lim, ch->req_lim);
> + }
> + return sprintf(buf, "%d\n", req_lim);
> }
>
> static ssize_t show_zero_req_lim(struct device *dev,
> @@ -2570,6 +2671,14 @@ static ssize_t show_local_ib_device(struct device *dev,
> return sprintf(buf, "%s\n", target->srp_host->srp_dev->dev->name);
> }
>
> +static ssize_t show_ch_count(struct device *dev, struct device_attribute *attr,
> + char *buf)
> +{
> + struct srp_target_port *target = host_to_target(class_to_shost(dev));
> +
> + return sprintf(buf, "%d\n", target->ch_count);
> +}
> +
> static ssize_t show_comp_vector(struct device *dev,
> struct device_attribute *attr, char *buf)
> {
> @@ -2613,6 +2722,7 @@ static DEVICE_ATTR(req_lim, S_IRUGO, show_req_lim, NULL);
> static DEVICE_ATTR(zero_req_lim, S_IRUGO, show_zero_req_lim, NULL);
> static DEVICE_ATTR(local_ib_port, S_IRUGO, show_local_ib_port, NULL);
> static DEVICE_ATTR(local_ib_device, S_IRUGO, show_local_ib_device, NULL);
> +static DEVICE_ATTR(ch_count, S_IRUGO, show_ch_count, NULL);
> static DEVICE_ATTR(comp_vector, S_IRUGO, show_comp_vector, NULL);
> static DEVICE_ATTR(tl_retry_count, S_IRUGO, show_tl_retry_count, NULL);
> static DEVICE_ATTR(cmd_sg_entries, S_IRUGO, show_cmd_sg_entries, NULL);
> @@ -2630,6 +2740,7 @@ static struct device_attribute *srp_host_attrs[] = {
> &dev_attr_zero_req_lim,
> &dev_attr_local_ib_port,
> &dev_attr_local_ib_device,
> + &dev_attr_ch_count,
> &dev_attr_comp_vector,
> &dev_attr_tl_retry_count,
> &dev_attr_cmd_sg_entries,
> @@ -2643,7 +2754,8 @@ static struct scsi_host_template srp_template = {
> .proc_name = DRV_NAME,
> .slave_configure = srp_slave_configure,
> .info = srp_target_info,
> - .queuecommand = srp_queuecommand,
> + .queuecommand = srp_sq_queuecommand,
> + .mq_queuecommand = srp_mq_queuecommand,
> .change_queue_depth = srp_change_queue_depth,
> .change_queue_type = srp_change_queue_type,
> .eh_abort_handler = srp_abort,
> @@ -3038,7 +3150,8 @@ static ssize_t srp_create_target(struct device *dev,
> struct srp_rdma_ch *ch;
> struct srp_device *srp_dev = host->srp_dev;
> struct ib_device *ibdev = srp_dev->dev;
> - int ret;
> + int ret, node_idx, node, cpu, i;
> + bool multich = false;
>
> target_host = scsi_host_alloc(&srp_template,
> sizeof (struct srp_target_port));
> @@ -3098,34 +3211,82 @@ static ssize_t srp_create_target(struct device *dev,
> INIT_WORK(&target->tl_err_work, srp_tl_err_work);
> INIT_WORK(&target->remove_work, srp_remove_work);
> spin_lock_init(&target->lock);
> - ch = &target->ch;
> - ch->target = target;
> - ch->comp_vector = target->comp_vector;
> - spin_lock_init(&ch->lock);
> - INIT_LIST_HEAD(&ch->free_tx);
> - ret = srp_alloc_req_data(ch);
> - if (ret)
> - goto err_free_mem;
> -
> ret = ib_query_gid(ibdev, host->port, 0, &target->sgid);
> if (ret)
> - goto err_free_mem;
> + goto err;
>
> - ret = srp_create_ch_ib(ch);
> - if (ret)
> - goto err_free_mem;
> + ret = -ENOMEM;
Any chance you take this non-trivial setup below to a routine and
document what you are trying to attempt?
> + target->ch_count = max_t(unsigned, num_online_nodes(),
> + min(ch_count ? :
> + min(4 * num_online_nodes(),
> + ibdev->num_comp_vectors),
> + num_online_cpus()));
> + target->ch = kcalloc(target->ch_count, sizeof(*target->ch),
> + GFP_KERNEL);
> + if (!target->ch)
> + goto err;
>
> - ret = srp_new_cm_id(ch);
> - if (ret)
> - goto err_free_ib;
> + node_idx = 0;
> + for_each_online_node(node) {
> + const int ch_start = (node_idx * target->ch_count /
> + num_online_nodes());
> + const int ch_end = ((node_idx + 1) * target->ch_count /
> + num_online_nodes());
> + const int cv_start = (node_idx * ibdev->num_comp_vectors /
> + num_online_nodes() + target->comp_vector)
> + % ibdev->num_comp_vectors;
> + const int cv_end = ((node_idx + 1) * ibdev->num_comp_vectors /
> + num_online_nodes() + target->comp_vector)
> + % ibdev->num_comp_vectors;
> + int cpu_idx = 0;
> +
> + for_each_online_cpu(cpu) {
> + if (cpu_to_node(cpu) != node)
> + continue;
> + if (ch_start + cpu_idx >= ch_end)
> + continue;
> + ch = &target->ch[ch_start + cpu_idx];
> + ch->target = target;
> + ch->comp_vector = cv_start == cv_end ? cv_start :
> + cv_start + cpu_idx % (cv_end - cv_start);
> + spin_lock_init(&ch->lock);
> + INIT_LIST_HEAD(&ch->free_tx);
> + ret = srp_new_cm_id(ch);
> + if (ret)
> + goto err_disconnect;
>
> - ret = srp_connect_ch(ch);
> - if (ret) {
> - shost_printk(KERN_ERR, target->scsi_host,
> - PFX "Connection failed\n");
> - goto err_free_ib;
> + ret = srp_create_ch_ib(ch);
> + if (ret)
> + goto err_disconnect;
> +
> + ret = srp_alloc_req_data(ch);
> + if (ret)
> + goto err_disconnect;
> +
> + ret = srp_connect_ch(ch, multich);
> + if (ret) {
> + shost_printk(KERN_ERR, target->scsi_host,
> + PFX "Connection %d/%d failed\n",
> + ch_start + cpu_idx,
> + target->ch_count);
> + if (node_idx == 0 && cpu_idx == 0) {
> + goto err_disconnect;
> + } else {
> + srp_free_ch_ib(target, ch);
> + srp_free_req_data(target, ch);
> + target->ch_count = ch - target->ch;
> + break;
> + }
> + }
> +
> + multich = true;
> + cpu_idx++;
> + }
> + node_idx++;
> }
>
> + target->scsi_host->nr_hw_queues = target->ch_count;
> +
> ret = srp_add_target(host, target);
> if (ret)
> goto err_disconnect;
> @@ -3154,11 +3315,13 @@ out:
> err_disconnect:
> srp_disconnect_target(target);
>
> -err_free_ib:
> - srp_free_ch_ib(ch);
> + for (i = 0; i < target->ch_count; i++) {
> + ch = &target->ch[i];
> + srp_free_ch_ib(target, ch);
> + srp_free_req_data(target, ch);
> + }
>
> -err_free_mem:
> - srp_free_req_data(ch);
> + kfree(target->ch);
>
> err:
> scsi_host_put(target_host);
> diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
> index 0609124..d9660e1 100644
> --- a/drivers/infiniband/ulp/srp/ib_srp.h
> +++ b/drivers/infiniband/ulp/srp/ib_srp.h
> @@ -84,6 +84,21 @@ enum srp_iu_type {
> SRP_IU_RSP,
> };
>
> +static inline u32 build_srp_tag(u16 ch, u16 req_idx)
> +{
> + return ch << 16 | req_idx;
> +}
> +
> +static inline u16 srp_tag_ch(u32 tag)
> +{
> + return tag >> 16;
> +}
> +
> +static inline u16 srp_tag_idx(u32 tag)
> +{
> + return tag & ((1 << 16) - 1);
> +}
> +
> /*
> * @mr_page_mask: HCA memory registration page mask.
> * @mr_page_size: HCA memory registration page size.
> @@ -127,7 +142,7 @@ struct srp_request {
> struct srp_direct_buf *indirect_desc;
> dma_addr_t indirect_dma_addr;
> short nmdesc;
> - short index;
> + uint32_t tag;
> };
>
> struct srp_rdma_ch {
> @@ -173,8 +188,9 @@ struct srp_target_port {
> /* read and written in the hot path */
> spinlock_t lock;
>
> - struct srp_rdma_ch ch;
> /* read only in the hot path */
> + struct srp_rdma_ch *ch;
> + u32 ch_count;
> u32 lkey;
> u32 rkey;
> enum srp_target_state state;
>
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <5421A093.1070203-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
@ 2014-09-23 19:02 ` Bart Van Assche
2014-09-24 12:22 ` Sagi Grimberg
2014-10-07 12:51 ` Bart Van Assche
1 sibling, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-23 19:02 UTC (permalink / raw)
To: Sagi Grimberg, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 23/09/2014 10:32, Sagi Grimberg wrote:
> On 9/19/2014 4:00 PM, Bart Van Assche wrote:
>> Improve performance by using multiple RDMA/RC channels per SCSI host
>> for communicating with an SRP target.
>>
>
> Hey Bart,
>
> Since you don't seem to negotiate/declare multichannel with the target,
> did you test this code with some target implementations other than SCST
> that happen to be out there?
>
> Overall, I think this patch would be easier to review if you also
> provide a list of logical changes (which obviously are introduced in
> this patch). Patch 7/8 can use some more information of target-channel
> relations as well.
Hello Sagi,
That's a good question. So far this patch series has only been tested
against the SCST SRP target driver. However, as you probably noticed, if
setting up a second or later RDMA channel fails SRP login is not failed
but communication proceeds with the number of channels that have been
established. This mechanism should retain backwards compatibility with
SRP target systems that do not support multichannel communication.
However, if the new code for SRP login turns out to be triggering bugs
in existing SRP target implementations we can still add a blacklist for
these implementations.
I will provide a more detailed list of logical changes in the second
version of this patch series.
Bart.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 7/8] IB/srp: Separate target and channel variables
2014-09-23 16:07 ` Sagi Grimberg
@ 2014-09-23 20:00 ` Bart Van Assche
0 siblings, 0 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-23 20:00 UTC (permalink / raw)
To: Sagi Grimberg, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 23/09/2014 10:07, Sagi Grimberg wrote:
> On 9/19/2014 3:59 PM, Bart Van Assche wrote:
>> @@ -3047,24 +3098,28 @@ static ssize_t srp_create_target(struct
>> device *dev,
>> INIT_WORK(&target->tl_err_work, srp_tl_err_work);
>> INIT_WORK(&target->remove_work, srp_remove_work);
>> spin_lock_init(&target->lock);
>
> Hey Bart,
>
> You keep this target-lock around. I don't see in this patch any usage
> of it left. Can you document what each of target->lock and ch->lock
> protects?
>
> So I would imagine that target-lock manages the target-wide variables
> such as state (what else?) while ch->lock protect channel specific free
> requests pool which can be moved to blk-mq preallocs and avoid
> maintaining it (not sure how req_lim would be maintained though and if
> it is still needed?).
Hello Sagi,
The use of these locks is very traditional - each lock is used to
protect those members of the data structure it is a member of and that
need to be protected against concurrent access.
Even with this patch applied "target->lock" is still used, e.g. to
protect target port state modifications.
I will convert the free request pool in the next version of this patch
series. The code for managing req_lim can probably be converted into
something based on the blk-mq reserved tag infrastructure.
Bart.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-23 19:02 ` Bart Van Assche
@ 2014-09-24 12:22 ` Sagi Grimberg
2014-09-24 13:13 ` Bart Van Assche
0 siblings, 1 reply; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-24 12:22 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/23/2014 10:02 PM, Bart Van Assche wrote:
> On 23/09/2014 10:32, Sagi Grimberg wrote:
>> On 9/19/2014 4:00 PM, Bart Van Assche wrote:
>>> Improve performance by using multiple RDMA/RC channels per SCSI host
>>> for communicating with an SRP target.
>>>
>>
>> Hey Bart,
>>
>> Since you don't seem to negotiate/declare multichannel with the target,
>> did you test this code with some target implementations other than SCST
>> that happen to be out there?
>>
>> Overall, I think this patch would be easier to review if you also
>> provide a list of logical changes (which obviously are introduced in
>> this patch). Patch 7/8 can use some more information of target-channel
>> relations as well.
>
> Hello Sagi,
>
> That's a good question. So far this patch series has only been tested
> against the SCST SRP target driver. However, as you probably noticed, if
> setting up a second or later RDMA channel fails SRP login is not failed
> but communication proceeds with the number of channels that have been
> established. This mechanism should retain backwards compatibility with
> SRP target systems that do not support multichannel communication.
> However, if the new code for SRP login turns out to be triggering bugs
> in existing SRP target implementations we can still add a blacklist for
> these implementations.
I'm more concerned that a target will accept multichannel and then
starts flipping since that wasn't tested in I don't know when, probably
never...
Since SRP_LOGIN_REQ/RESP has some free bits why not declare it and
activate it when both sides *says* they support it? I'd be much calmer
knowing we're on the safe side on this...
>
> I will provide a more detailed list of logical changes in the second
> version of this patch series.
Thanks,
Plus, I would like to run it on my performance setups. can you point me
to the SCST repo? is multichannel supported in scst trunk?
Sagi.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-09-24 12:22 ` Sagi Grimberg
@ 2014-09-24 13:13 ` Bart Van Assche
[not found] ` <5422C395.7090902-HInyCGIudOg@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-09-24 13:13 UTC (permalink / raw)
To: Sagi Grimberg, linux-scsi
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 24/09/2014 6:22, Sagi Grimberg wrote:
> Since SRP_LOGIN_REQ/RESP has some free bits why not declare it and
> activate it when both sides *says* they support it? I'd be much calmer
> knowing we're on the safe side on this...
Hello Sagi,
Since more than ten years the SRP protocol is an official ANSI standard.
Since multichannel support has been defined in that standard my
preference is to follow what has been documented in that standard with
regard to multichannel operation. Using one of the free bits in the SRP
login request and response would involve a protocol modification. Hence
the proposal to add a blacklist for non-conforming target implementations.
> Plus, I would like to run it on my performance setups. can you point me
> to the SCST repo? is multichannel supported in scst trunk?
I think multichannel support was already present in the SCST SRP target
driver before I started maintaining that driver. However, last April a
few patches were checked in to improve multichannel support in the SCST
SRP target driver. These patches have been included in the SCST 3.0
release. Download instructions for SCST (3.0 and trunk) can be found
e.g. here: http://scst.sourceforge.net/downloads.html.
Bart.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <5422C395.7090902-HInyCGIudOg@public.gmane.org>
@ 2014-09-24 13:38 ` Sagi Grimberg
[not found] ` <5422C970.4050306-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-24 13:38 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/24/2014 4:13 PM, Bart Van Assche wrote:
> On 24/09/2014 6:22, Sagi Grimberg wrote:
>> Since SRP_LOGIN_REQ/RESP has some free bits why not declare it and
>> activate it when both sides *says* they support it? I'd be much calmer
>> knowing we're on the safe side on this...
>
> Hello Sagi,
>
> Since more than ten years the SRP protocol is an official ANSI standard.
> Since multichannel support has been defined in that standard my
> preference is to follow what has been documented in that standard with
> regard to multichannel operation.
Just re-visited the r16a, srp_login request req_flags include MULTI
CHANNEL ACTION (Table 10) and srp login response rsp_flags include
MULTI-CHANNEL RESULT (Table 12).
Did you notice those? Didn't see any reference in the patch...
Using one of the free bits in the SRP
> login request and response would involve a protocol modification. Hence
> the proposal to add a blacklist for non-conforming target implementations.
>
So I'm not so sure we need to update SRP login sequence...
>> Plus, I would like to run it on my performance setups. can you point me
>> to the SCST repo? is multichannel supported in scst trunk?
>
> I think multichannel support was already present in the SCST SRP target
> driver before I started maintaining that driver. However, last April a
> few patches were checked in to improve multichannel support in the SCST
> SRP target driver. These patches have been included in the SCST 3.0
> release. Download instructions for SCST (3.0 and trunk) can be found
> e.g. here: http://scst.sourceforge.net/downloads.html.
>
Thanks,
P.S.
Would it be possible to break 8/8 into more patches in the next round?
it would help make it more review-able?
Thanks,
Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <5422C970.4050306-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
@ 2014-09-24 13:43 ` Sagi Grimberg
0 siblings, 0 replies; 55+ messages in thread
From: Sagi Grimberg @ 2014-09-24 13:43 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 9/24/2014 4:38 PM, Sagi Grimberg wrote:
> On 9/24/2014 4:13 PM, Bart Van Assche wrote:
>> On 24/09/2014 6:22, Sagi Grimberg wrote:
>>> Since SRP_LOGIN_REQ/RESP has some free bits why not declare it and
>>> activate it when both sides *says* they support it? I'd be much calmer
>>> knowing we're on the safe side on this...
>>
>> Hello Sagi,
>>
>> Since more than ten years the SRP protocol is an official ANSI standard.
>> Since multichannel support has been defined in that standard my
>> preference is to follow what has been documented in that standard with
>> regard to multichannel operation.
>
> Just re-visited the r16a, srp_login request req_flags include MULTI
> CHANNEL ACTION (Table 10) and srp login response rsp_flags include
> MULTI-CHANNEL RESULT (Table 12).
>
> Did you notice those? Didn't see any reference in the patch...
>
> Using one of the free bits in the SRP
>> login request and response would involve a protocol modification. Hence
>> the proposal to add a blacklist for non-conforming target
>> implementations.
>>
>
> So I'm not so sure we need to update SRP login sequence...
>
Wait, yes you did reference those...
OK, I'm on board now...
Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 2/8] scsi-mq: Add support for multiple hardware queues
[not found] ` <CAF9gx6JfP2bGyMauB6LzepZP_vKEvrd-sPZc5CRuOrtgQ_UCSw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-09-26 11:08 ` Ming Lei
[not found] ` <CACVXFVMiYsW=dszQ6mE-o_L8fEDdkO59vJ5qeHKch5c33K_QXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Ming Lei @ 2014-09-26 11:08 UTC (permalink / raw)
To: Sagi Grimberg
Cc: Bart Van Assche, linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma,
Christoph Hellwig, Jens Axboe, Robert Elliott
On Sat, Sep 20, 2014 at 2:05 AM, Sagi Grimberg <sagig-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> wrote:
> On 9/19/2014 3:57 PM, Bart Van Assche wrote:
>> Allow a SCSI LLD to declare how many hardware queues it supports
>> by setting Scsi_Host.nr_hw_queues before calling scsi_add_host().
>>
>> Note: it is assumed that each hardware queue has a queue depth of
>> shost->can_queue. In other words, the total queue depth per host
>> is (number of hardware queues) * (shost->can_queue).
>>
>> Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
>> Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
>> ---
>> drivers/scsi/scsi_lib.c | 2 +-
>> include/scsi/scsi_host.h | 4 ++++
>> 2 files changed, 5 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
>> index d837dc1..b0b6117 100644
>> --- a/drivers/scsi/scsi_lib.c
>> +++ b/drivers/scsi/scsi_lib.c
>> @@ -2071,7 +2071,7 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost)
>>
>> memset(&shost->tag_set, 0, sizeof(shost->tag_set));
>> shost->tag_set.ops = &scsi_mq_ops;
>> - shost->tag_set.nr_hw_queues = 1;
>> + shost->tag_set.nr_hw_queues = shost->nr_hw_queues ? : 1;
>> shost->tag_set.queue_depth = shost->can_queue;
>> shost->tag_set.cmd_size = cmd_size;
>> shost->tag_set.numa_node = NUMA_NO_NODE;
>> diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h
>> index ba20347..0a867d9 100644
>> --- a/include/scsi/scsi_host.h
>> +++ b/include/scsi/scsi_host.h
>> @@ -638,6 +638,10 @@ struct Scsi_Host {
>> short unsigned int sg_prot_tablesize;
>> unsigned int max_sectors;
>> unsigned long dma_boundary;
>> + /*
>> + * In scsi-mq mode, the number of hardware queues supported by the LLD.
>> + */
>> + unsigned nr_hw_queues;
>> /*
>> * Used to assign serial numbers to the cmds.
>> * Protected by the host lock.
>>
>
> I think this patch should be squashed with passing LLD hctx patch (in
> whatever form it ends up).
I suggest to apply this patch and related scsi-mq multi hw-queue
enablement patches first, so that any scsi drivers capable of
multi hw-queue can go without waiting for passing hctx
patches which may take a bit long since lots of drivers are involved,
as Jens said, the mapping from req to hctx is quite cheap.
Thanks,
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 2/8] scsi-mq: Add support for multiple hardware queues
[not found] ` <CACVXFVMiYsW=dszQ6mE-o_L8fEDdkO59vJ5qeHKch5c33K_QXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-09-26 14:02 ` Bart Van Assche
0 siblings, 0 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-09-26 14:02 UTC (permalink / raw)
To: Ming Lei, Sagi Grimberg
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Jens Axboe, Robert Elliott
On 26/09/2014 5:08, Ming Lei wrote:
>
> I suggest to apply this patch and related scsi-mq multi hw-queue
> enablement patches first, so that any scsi drivers capable of
> multi hw-queue can go without waiting for passing hctx
> patches which may take a bit long since lots of drivers are involved,
> as Jens said, the mapping from req to hctx is quite cheap.
Hello Ming,
This week I have been traveling. I hope to find more time next week or
the week after next week to rework this patch series and to address the
feedback I have received from you, Jens, Christoph and Sagi.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <541C4DF1.4090604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
@ 2014-10-01 16:08 ` Bart Van Assche
2014-10-01 16:54 ` Jens Axboe
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-10-01 16:08 UTC (permalink / raw)
To: Jens Axboe, Ming Lei
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Robert Elliott
On 09/19/14 17:38, Jens Axboe wrote:
> ctx was meant to be private, unfortunately it's leaked a bit into other
> parts of block/. But it's still private within that, at least.
>
> Lets not add more stuff to struct request, it's already way too large.
> We could add an exported
>
> struct blk_mq_hw_ctx *blk_mq_request_to_hctx(struct request *rq)
> {
> struct request_queue *q = rq->q;
>
> return q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
> }
>
> for this.
How about the patch below ? That patch makes it easy for SCSI LLDs to obtain
the hardware context index.
[PATCH] blk-mq: Add blk_get_mq_tag()
The queuecommand() callback functions in SCSI low-level drivers
must know which hardware context has been selected by the block
layer. Since passing the hctx pointer directly to the queuecommand
callback function would require modification of all SCSI LLDs,
add a function to the block layer that allows to query the hardware
context index.
Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
---
block/blk-mq-tag.c | 24 ++++++++++++++++++++++++
include/linux/blk-mq.h | 16 ++++++++++++++++
2 files changed, 40 insertions(+)
diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index c1b9242..5618759 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -595,6 +595,30 @@ int blk_mq_tag_update_depth(struct blk_mq_tags *tags, unsigned int tdepth)
return 0;
}
+/**
+ * blk_get_mq_tag() - return a tag that is unique queue-wide
+ *
+ * The tag field in struct request is unique per hardware queue but not
+ * queue-wide. Hence this function. It is not only safe to use this function
+ * for multiqueue request queues but also for single-queue request queues.
+ * Note: rq->tag == -1 if tagging is not enabled for single-queue request
+ * queues.
+ */
+struct blk_mq_tag blk_get_mq_tag(struct request *rq)
+{
+ struct request_queue *q = rq->q;
+ struct blk_mq_tag tag = { rq->tag & ((1 << 16) - 1) };
+ struct blk_mq_hw_ctx *hctx;
+
+ if (q->mq_ops) {
+ hctx = q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
+ tag.tag |= hctx->queue_num << 16;
+ }
+
+ return tag;
+}
+EXPORT_SYMBOL(blk_get_mq_tag);
+
ssize_t blk_mq_tag_sysfs_show(struct blk_mq_tags *tags, char *page)
{
char *orig_page = page;
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index eac4f31..eb419bc 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -88,6 +88,13 @@ struct blk_mq_tag_set {
struct list_head tag_list;
};
+/**
+ * struct blk_mq_tag - hardware queue index and per-queue tag
+ */
+struct blk_mq_tag {
+ u32 tag;
+};
+
typedef int (queue_rq_fn)(struct blk_mq_hw_ctx *, struct request *);
typedef struct blk_mq_hw_ctx *(map_queue_fn)(struct request_queue *, const int);
typedef int (init_hctx_fn)(struct blk_mq_hw_ctx *, void *, unsigned int);
@@ -166,6 +173,15 @@ bool blk_mq_can_queue(struct blk_mq_hw_ctx *);
struct request *blk_mq_alloc_request(struct request_queue *q, int rw,
gfp_t gfp, bool reserved);
struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag);
+struct blk_mq_tag blk_get_mq_tag(struct request *rq);
+static inline u16 blk_mq_tag_to_hwq(struct blk_mq_tag t)
+{
+ return t.tag >> 16;
+}
+static inline u16 blk_mq_tag_to_tag(struct blk_mq_tag t)
+{
+ return t.tag & ((1 << 16) - 1);
+}
struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *, const int ctx_index);
struct blk_mq_hw_ctx *blk_mq_alloc_single_hw_queue(struct blk_mq_tag_set *, unsigned int, int);
--
1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 55+ messages in thread
* [PATCH RFC] scsi_tcq.h: Add support for multiple hardware queues
[not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
` (2 preceding siblings ...)
2014-09-19 13:00 ` [PATCH 8/8] IB/srp: Add multichannel support Bart Van Assche
@ 2014-10-01 16:14 ` Bart Van Assche
3 siblings, 0 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-10-01 16:14 UTC (permalink / raw)
To: linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
The eight patches I had posted so far did not modify the tag lookup
functions in include/scsi/scsi_tcq.h. However, these functions have
to be modified in order to become usable in SCSI LLDs that support
multiple hardware queues. How about the patch below, which adds
support for multiqueue tag lookup without requiring any modifications
in existing SCSI LLD drivers ?
---
include/scsi/scsi_tcq.h | 17 ++++++++++-------
1 file changed, 10 insertions(+), 7 deletions(-)
diff --git a/include/scsi/scsi_tcq.h b/include/scsi/scsi_tcq.h
index e645835..8e3bec34 100644
--- a/include/scsi/scsi_tcq.h
+++ b/include/scsi/scsi_tcq.h
@@ -111,18 +111,21 @@ static inline int scsi_populate_tag_msg(struct scsi_cmnd *cmd, char *msg)
}
static inline struct scsi_cmnd *scsi_mq_find_tag(struct Scsi_Host *shost,
- unsigned int hw_ctx, int tag)
+ int tag)
{
+ struct blk_mq_tag mq_tag = { tag };
+ u16 hwq = blk_mq_tag_to_hwq(mq_tag);
struct request *req;
- req = blk_mq_tag_to_rq(shost->tag_set.tags[hw_ctx], tag);
+ req = blk_mq_tag_to_rq(shost->tag_set.tags[hwq],
+ blk_mq_tag_to_tag(mq_tag));
return req ? (struct scsi_cmnd *)req->special : NULL;
}
/**
* scsi_find_tag - find a tagged command by device
* @SDpnt: pointer to the ScSI device
- * @tag: the tag number
+ * @tag: tag generated by blk_get_mq_tag()
*
* Notes:
* Only works with tags allocated by the generic blk layer.
@@ -133,9 +136,9 @@ static inline struct scsi_cmnd *scsi_find_tag(struct scsi_device *sdev, int tag)
if (tag != SCSI_NO_TAG) {
if (shost_use_blk_mq(sdev->host))
- return scsi_mq_find_tag(sdev->host, 0, tag);
+ return scsi_mq_find_tag(sdev->host, tag);
- req = blk_queue_find_tag(sdev->request_queue, tag);
+ req = blk_queue_find_tag(sdev->request_queue, tag);
return req ? (struct scsi_cmnd *)req->special : NULL;
}
@@ -174,7 +177,7 @@ static inline int scsi_init_shared_tag_map(struct Scsi_Host *shost, int depth)
/**
* scsi_host_find_tag - find the tagged command by host
* @shost: pointer to scsi_host
- * @tag: tag of the scsi_cmnd
+ * @tag: tag generated by blk_get_mq_tag()
*
* Notes:
* Only works with tags allocated by the generic blk layer.
@@ -186,7 +189,7 @@ static inline struct scsi_cmnd *scsi_host_find_tag(struct Scsi_Host *shost,
if (tag != SCSI_NO_TAG) {
if (shost_use_blk_mq(shost))
- return scsi_mq_find_tag(shost, 0, tag);
+ return scsi_mq_find_tag(shost, tag);
req = blk_map_queue_find_tag(shost->bqt, tag);
return req ? (struct scsi_cmnd *)req->special : NULL;
}
--
1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-10-01 16:08 ` Bart Van Assche
@ 2014-10-01 16:54 ` Jens Axboe
2014-10-01 21:14 ` Christoph Hellwig
[not found] ` <542C31C4.1020702-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
0 siblings, 2 replies; 55+ messages in thread
From: Jens Axboe @ 2014-10-01 16:54 UTC (permalink / raw)
To: Bart Van Assche, Ming Lei
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Robert Elliott
On 2014-10-01 10:08, Bart Van Assche wrote:
> On 09/19/14 17:38, Jens Axboe wrote:
>> ctx was meant to be private, unfortunately it's leaked a bit into other
>> parts of block/. But it's still private within that, at least.
>>
>> Lets not add more stuff to struct request, it's already way too large.
>> We could add an exported
>>
>> struct blk_mq_hw_ctx *blk_mq_request_to_hctx(struct request *rq)
>> {
>> struct request_queue *q = rq->q;
>>
>> return q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
>> }
>>
>> for this.
>
> How about the patch below ? That patch makes it easy for SCSI LLDs to obtain
> the hardware context index.
>
> [PATCH] blk-mq: Add blk_get_mq_tag()
>
> The queuecommand() callback functions in SCSI low-level drivers
> must know which hardware context has been selected by the block
> layer. Since passing the hctx pointer directly to the queuecommand
> callback function would require modification of all SCSI LLDs,
> add a function to the block layer that allows to query the hardware
> context index.
>
> Signed-off-by: Bart Van Assche <bvanassche@acm.org>
> ---
> block/blk-mq-tag.c | 24 ++++++++++++++++++++++++
> include/linux/blk-mq.h | 16 ++++++++++++++++
> 2 files changed, 40 insertions(+)
>
> diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
> index c1b9242..5618759 100644
> --- a/block/blk-mq-tag.c
> +++ b/block/blk-mq-tag.c
> @@ -595,6 +595,30 @@ int blk_mq_tag_update_depth(struct blk_mq_tags *tags, unsigned int tdepth)
> return 0;
> }
>
> +/**
> + * blk_get_mq_tag() - return a tag that is unique queue-wide
> + *
> + * The tag field in struct request is unique per hardware queue but not
> + * queue-wide. Hence this function. It is not only safe to use this function
> + * for multiqueue request queues but also for single-queue request queues.
> + * Note: rq->tag == -1 if tagging is not enabled for single-queue request
> + * queues.
> + */
> +struct blk_mq_tag blk_get_mq_tag(struct request *rq)
> +{
> + struct request_queue *q = rq->q;
> + struct blk_mq_tag tag = { rq->tag & ((1 << 16) - 1) };
> + struct blk_mq_hw_ctx *hctx;
> +
> + if (q->mq_ops) {
> + hctx = q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
> + tag.tag |= hctx->queue_num << 16;
> + }
> +
> + return tag;
> +}
> +EXPORT_SYMBOL(blk_get_mq_tag);
Lets get rid of the blk_mq_tag struct and just have it be an u32 or
something. We could potentially typedef it, but I'd prefer to just have
it be an unsigned 32-bit int.
Probably also need some init time safety checks that 16-bits is enough
to hold BLK_MQ_MAX_DEPTH. Just in case that is ever bumped, or the queue
prefixing changes.
And I think we need to name this better. Your comment correctly
describes that this generates a unique tag queue wide, but the name of
the function implies that we just return the request tag. Most drivers
wont use this. Perhaps add a queue flag that tells us that we should
generate these tags and have it setup ala:
u32 blk_mq_unique_rq_tag(struct request *rq)
{
struct request_queue *q = rq->q;
u32 tag = rq->tag & ((1 << 16) - 1);
struct blk_mq_hw_ctx *hctx;
hctx = q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
return tag | (hctx->queue_num << 16);
}
u32 blk_mq_rq_tag(struct request *rq)
{
struct request_queue *q = rq->q;
if (q->mq_ops &&
test_bit(QUEUE_FLAG_UNIQUE_TAGS, &q->queue_flags))
return blk_mq_unique_rq_tag(rq);
return rq->tag;
}
Totally untested, just typed in email.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-10-01 16:54 ` Jens Axboe
@ 2014-10-01 21:14 ` Christoph Hellwig
[not found] ` <542C31C4.1020702-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
1 sibling, 0 replies; 55+ messages in thread
From: Christoph Hellwig @ 2014-10-01 21:14 UTC (permalink / raw)
To: Jens Axboe
Cc: Bart Van Assche, Ming Lei, linux-scsi, linux-rdma,
Christoph Hellwig, Robert Elliott
On Wed, Oct 01, 2014 at 10:54:28AM -0600, Jens Axboe wrote:
> Lets get rid of the blk_mq_tag struct and just have it be an u32 or
> something. We could potentially typedef it, but I'd prefer to just have it
> be an unsigned 32-bit int.
Agreed.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 5/8] IB/srp: Remove stale connection retry mechanism
[not found] ` <CAJ3xEMhPKiut4MwZH9F7-T0+u7B6XPuh-FTZpA=Xe4ViAj5UUQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
@ 2014-10-02 10:34 ` Bart Van Assche
[not found] ` <542D2A3C.2080009-HInyCGIudOg@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-10-02 10:34 UTC (permalink / raw)
To: Or Gerlitz, sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Jens Axboe, Robert Elliott, Ming Lei
On 09/20/14 19:45, Or Gerlitz wrote:
> On Fri, Sep 19, 2014 at 3:58 PM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org> wrote:
>> Attempting to connect three times may be insufficient after an
>> initiator system that was using multiple RDMA channels tries to
>> relogin. Additionally, this login retry mechanism is a workaround
>> for particular behavior of the IB/CM.
>
> Can you be more specific re the particular behavior of the IB CM?
> added Sean, the CM maintainer.
Let's focus on the software behavior instead of the people who are
involved. What I have observed several times is that after a power cycle
of the initiator system the first few login attempts are rejected. I was
assuming that this was due to the IB/CM implementation but now that I
have had another look at the logs I see that there is not enough
information in the system logs to draw this conclusion. I will add
additional logging statements in the initiator and target kernel code
such that I can determine the root cause of this behavior.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <542C31C4.1020702-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
@ 2014-10-02 16:45 ` Bart Van Assche
2014-10-02 16:55 ` Jens Axboe
[not found] ` <542D8143.3050305-HInyCGIudOg@public.gmane.org>
0 siblings, 2 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-10-02 16:45 UTC (permalink / raw)
To: Jens Axboe, Ming Lei
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Robert Elliott
On 10/01/14 18:54, Jens Axboe wrote:
> Lets get rid of the blk_mq_tag struct and just have it be an u32 or
> something. We could potentially typedef it, but I'd prefer to just have
> it be an unsigned 32-bit int.
>
> Probably also need some init time safety checks that 16-bits is enough
> to hold BLK_MQ_MAX_DEPTH. Just in case that is ever bumped, or the queue
> prefixing changes.
>
> And I think we need to name this better. Your comment correctly
> describes that this generates a unique tag queue wide, but the name of
> the function implies that we just return the request tag. Most drivers
> wont use this. Perhaps add a queue flag that tells us that we should
> generate these tags and have it setup ala:
>
> u32 blk_mq_unique_rq_tag(struct request *rq)
> {
> struct request_queue *q = rq->q;
> u32 tag = rq->tag & ((1 << 16) - 1);
> struct blk_mq_hw_ctx *hctx;
>
> hctx = q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
> return tag | (hctx->queue_num << 16);
> }
>
> u32 blk_mq_rq_tag(struct request *rq)
> {
> struct request_queue *q = rq->q;
>
> if (q->mq_ops &&
> test_bit(QUEUE_FLAG_UNIQUE_TAGS, &q->queue_flags))
> return blk_mq_unique_rq_tag(rq);
>
> return rq->tag;
> }
Would it be acceptable to let blk_mq_rq_tag() always return the
hardware context number and the per-hctx tag ? Block and SCSI LLD
drivers that do not need the hardware context number can still use
rq->tag. Drivers that need both can use blk_mq_rq_tag(). That way we do
not have to introduce a new queue flag. How about the patch below
(which is still missing a BLK_MQ_MAX_DEPTH check):
diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index c1b9242..8cfbc7b 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -595,6 +595,30 @@ int blk_mq_tag_update_depth(struct blk_mq_tags *tags, unsigned int tdepth)
return 0;
}
+/**
+ * blk_mq_rq_tag() - return a tag that is unique queue-wide
+ *
+ * The tag field in struct request is unique per hardware queue but not
+ * queue-wide. Hence this function.
+ *
+ * Note: When called for a non-multiqueue request queue, the hardware context
+ * index is set to zero.
+ */
+u32 blk_mq_rq_tag(struct request *rq)
+{
+ struct request_queue *q = rq->q;
+ struct blk_mq_hw_ctx *hctx;
+ int hwq = 0;
+
+ if (q->mq_ops) {
+ hctx = q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
+ hwq = hctx->queue_num;
+ }
+
+ return blk_mq_build_rq_tag(hwq, rq->tag);
+}
+EXPORT_SYMBOL(blk_mq_rq_tag);
+
ssize_t blk_mq_tag_sysfs_show(struct blk_mq_tags *tags, char *page)
{
char *orig_page = page;
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index eac4f31..c5be535 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -166,6 +166,19 @@ bool blk_mq_can_queue(struct blk_mq_hw_ctx *);
struct request *blk_mq_alloc_request(struct request_queue *q, int rw,
gfp_t gfp, bool reserved);
struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag);
+u32 blk_mq_rq_tag(struct request *rq);
+static inline u32 blk_mq_build_rq_tag(int hwq, int tag)
+{
+ return (hwq << 16) | (tag & ((1 << 16) - 1));
+}
+static inline u16 blk_mq_rq_tag_to_hwq(u32 rq_tag)
+{
+ return rq_tag >> 16;
+}
+static inline u16 blk_mq_rq_tag_to_tag(u32 rq_tag)
+{
+ return rq_tag & ((1 << 16) - 1);
+}
struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *, const int ctx_index);
struct blk_mq_hw_ctx *blk_mq_alloc_single_hw_queue(struct blk_mq_tag_set *, unsigned int, int);
--
1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-10-02 16:45 ` Bart Van Assche
@ 2014-10-02 16:55 ` Jens Axboe
[not found] ` <542D8368.8080604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
[not found] ` <542D8143.3050305-HInyCGIudOg@public.gmane.org>
1 sibling, 1 reply; 55+ messages in thread
From: Jens Axboe @ 2014-10-02 16:55 UTC (permalink / raw)
To: Bart Van Assche, Ming Lei
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Robert Elliott
On 10/02/2014 10:45 AM, Bart Van Assche wrote:
> On 10/01/14 18:54, Jens Axboe wrote:
>> Lets get rid of the blk_mq_tag struct and just have it be an u32 or
>> something. We could potentially typedef it, but I'd prefer to just have
>> it be an unsigned 32-bit int.
>>
>> Probably also need some init time safety checks that 16-bits is enough
>> to hold BLK_MQ_MAX_DEPTH. Just in case that is ever bumped, or the queue
>> prefixing changes.
>>
>> And I think we need to name this better. Your comment correctly
>> describes that this generates a unique tag queue wide, but the name of
>> the function implies that we just return the request tag. Most drivers
>> wont use this. Perhaps add a queue flag that tells us that we should
>> generate these tags and have it setup ala:
>>
>> u32 blk_mq_unique_rq_tag(struct request *rq)
>> {
>> struct request_queue *q = rq->q;
>> u32 tag = rq->tag & ((1 << 16) - 1);
>> struct blk_mq_hw_ctx *hctx;
>>
>> hctx = q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
>> return tag | (hctx->queue_num << 16);
>> }
>>
>> u32 blk_mq_rq_tag(struct request *rq)
>> {
>> struct request_queue *q = rq->q;
>>
>> if (q->mq_ops &&
>> test_bit(QUEUE_FLAG_UNIQUE_TAGS, &q->queue_flags))
>> return blk_mq_unique_rq_tag(rq);
>>
>> return rq->tag;
>> }
>
> Would it be acceptable to let blk_mq_rq_tag() always return the
> hardware context number and the per-hctx tag ? Block and SCSI LLD
Sure, that's fine as well, but the function needs a more descriptive
name. I try to think of it like I have never looked at the code and need
to write a driver, it's a lot easier if the functions are named
appropriately. Seeing blk_mq_rq_tag() and even with reading the function
comment, I'm really none the wiser and would assume I need to use this
function to get the tag.
So we can do the single function, but lets call it
blk_mq_unique_rq_tag(). That's special enough that people will know this
is something that doesn't just return the request tag. Then add an extra
sentence to the comment you already have on when this is needed.
And lets roll those bitshift values and masks into a define or enum so
it's collected in one place.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <542D8143.3050305-HInyCGIudOg@public.gmane.org>
@ 2014-10-02 17:30 ` Christoph Hellwig
2014-10-06 11:16 ` Bart Van Assche
0 siblings, 1 reply; 55+ messages in thread
From: Christoph Hellwig @ 2014-10-02 17:30 UTC (permalink / raw)
To: Bart Van Assche
Cc: Jens Axboe, Ming Lei, linux-scsi-u79uwXL29TY76Z2rM5mHXA,
linux-rdma, Christoph Hellwig, Robert Elliott
On Thu, Oct 02, 2014 at 06:45:55PM +0200, Bart Van Assche wrote:
> Would it be acceptable to let blk_mq_rq_tag() always return the
> hardware context number and the per-hctx tag ? Block and SCSI LLD
> drivers that do not need the hardware context number can still use
> rq->tag. Drivers that need both can use blk_mq_rq_tag(). That way we do
> not have to introduce a new queue flag. How about the patch below
> (which is still missing a BLK_MQ_MAX_DEPTH check):
I'd add the unique_ part to the name that Jens added, and fix up the
comment to be valid kerneldoc, but otherwise this looks fine to me.
Also if we want to merge scsi LLDDs that can take advantage of
multiqueue support it would probably be best if I take this via the SCSI
tree.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 5/8] IB/srp: Remove stale connection retry mechanism
[not found] ` <542D2A3C.2080009-HInyCGIudOg@public.gmane.org>
@ 2014-10-03 8:51 ` Bart Van Assche
0 siblings, 0 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-10-03 8:51 UTC (permalink / raw)
To: Or Gerlitz, sean.hefty-ral2JQCrhuEAvxtiuMwx3w
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
[-- Attachment #1: Type: text/plain, Size: 2373 bytes --]
On 10/02/14 12:34, Bart Van Assche wrote:
> On 09/20/14 19:45, Or Gerlitz wrote:
>> On Fri, Sep 19, 2014 at 3:58 PM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
>> wrote:
>>> Attempting to connect three times may be insufficient after an
>>> initiator system that was using multiple RDMA channels tries to
>>> relogin. Additionally, this login retry mechanism is a workaround
>>> for particular behavior of the IB/CM.
>>
>> Can you be more specific re the particular behavior of the IB CM?
>> added Sean, the CM maintainer.
>
> Let's focus on the software behavior instead of the people who are
> involved. What I have observed several times is that after a power cycle
> of the initiator system the first few login attempts are rejected. I was
> assuming that this was due to the IB/CM implementation but now that I
> have had another look at the logs I see that there is not enough
> information in the system logs to draw this conclusion. I will add
> additional logging statements in the initiator and target kernel code
> such that I can determine the root cause of this behavior.
(replying to my own e-mail / removed linux-scsi from CC-list)
So far I have been able to reproduce this behavior once after pushing
the reset button of the initiator system while it was in the connected
state. After the initiator system had finished rebooting I started
ibdump on both IB ports of the target system (attached to this e-mail).
What surprised me is that I found all the messages I expected in the
ibdump output (e.g. IB MAD device management query) but no CM messages. Both
sides were running FW 2.32.5100. The following messages were logged at
the initiator side while ibdump was running at the target side:
Oct 02 17:43:42 msi kernel: scsi host14: ib_srp: REJ received
Oct 02 17:43:42 msi kernel: scsi host14: REJ reason: stale connection
Oct 02 17:43:42 msi kernel: scsi host14: ib_srp: giving up on stale connection
Oct 02 17:43:42 msi kernel: scsi host14: ib_srp: Connection 0/12 failed
Oct 02 17:43:42 msi kernel: scsi host15: ib_srp: REJ received
Oct 02 17:43:42 msi kernel: scsi host15: REJ reason: stale connection
Oct 02 17:43:42 msi kernel: scsi host15: ib_srp: giving up on stale connection
Oct 02 17:43:42 msi kernel: scsi host15: ib_srp: Connection 0/12 failed
After a few more login attempts SRP login succeeded.
Bart.
[-- Attachment #2: p1.pcap --]
[-- Type: application/vnd.tcpdump.pcap, Size: 0 bytes --]
[-- Attachment #3: p2.pcap --]
[-- Type: application/vnd.tcpdump.pcap, Size: 4096 bytes --]
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <542D8368.8080604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
@ 2014-10-03 13:01 ` Bart Van Assche
2014-10-03 14:24 ` Jens Axboe
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-10-03 13:01 UTC (permalink / raw)
To: Jens Axboe, Ming Lei
Cc: linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Christoph Hellwig,
Robert Elliott
On 10/02/14 18:55, Jens Axboe wrote:
> Sure, that's fine as well, but the function needs a more descriptive
> name. I try to think of it like I have never looked at the code and need
> to write a driver, it's a lot easier if the functions are named
> appropriately. Seeing blk_mq_rq_tag() and even with reading the function
> comment, I'm really none the wiser and would assume I need to use this
> function to get the tag.
>
> So we can do the single function, but lets call it
> blk_mq_unique_rq_tag(). That's special enough that people will know this
> is something that doesn't just return the request tag. Then add an extra
> sentence to the comment you already have on when this is needed.
>
> And lets roll those bitshift values and masks into a define or enum so
> it's collected in one place.
How about the patch below ? In that patch all comments should have been
addressed that Christoph and you have formulated so far.
Thanks,
Bart.
[PATCH] blk-mq: Add blk_mq_unique_tag()
The queuecommand() callback functions in SCSI low-level drivers
need to know which hardware context has been selected by the
block layer. Since this information is not available in the
request structure, and since passing the hctx pointer directly to
the queuecommand callback function would require modification of
all SCSI LLDs, add a function to the block layer that allows to
query the hardware context index.
Signed-off-by: Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org>
---
block/blk-mq-tag.c | 27 +++++++++++++++++++++++++++
block/blk-mq.c | 2 ++
include/linux/blk-mq.h | 23 +++++++++++++++++++++++
3 files changed, 52 insertions(+)
diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c
index c1b9242..b5088f0 100644
--- a/block/blk-mq-tag.c
+++ b/block/blk-mq-tag.c
@@ -595,6 +595,33 @@ int blk_mq_tag_update_depth(struct blk_mq_tags *tags, unsigned int tdepth)
return 0;
}
+/**
+ * blk_mq_unique_tag() - return a tag that is unique queue-wide
+ * @rq: request for which to compute a unique tag
+ *
+ * The tag field in struct request is unique per hardware queue but not over
+ * all hardware queues. Hence this function that returns a tag with the
+ * hardware context index in the upper bits and the per hardware queue tag in
+ * the lower bits.
+ *
+ * Note: When called for a request that queued on a non-multiqueue request
+ * queue, the hardware context index is set to zero.
+ */
+u32 blk_mq_unique_tag(struct request *rq)
+{
+ struct request_queue *q = rq->q;
+ struct blk_mq_hw_ctx *hctx;
+ int hwq = 0;
+
+ if (q->mq_ops) {
+ hctx = q->mq_ops->map_queue(q, rq->mq_ctx->cpu);
+ hwq = hctx->queue_num;
+ }
+
+ return blk_mq_build_unique_tag(hwq, rq->tag);
+}
+EXPORT_SYMBOL(blk_mq_unique_tag);
+
ssize_t blk_mq_tag_sysfs_show(struct blk_mq_tags *tags, char *page)
{
char *orig_page = page;
diff --git a/block/blk-mq.c b/block/blk-mq.c
index df8e1e0..8098aac 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -2018,6 +2018,8 @@ static int blk_mq_alloc_rq_maps(struct blk_mq_tag_set *set)
*/
int blk_mq_alloc_tag_set(struct blk_mq_tag_set *set)
{
+ BUILD_BUG_ON(BLK_MQ_MAX_DEPTH > 1 << BLK_MQ_UNIQUE_TAG_TAG_BITS);
+
if (!set->nr_hw_queues)
return -EINVAL;
if (!set->queue_depth)
diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h
index eac4f31..b53d0c2 100644
--- a/include/linux/blk-mq.h
+++ b/include/linux/blk-mq.h
@@ -167,6 +167,29 @@ struct request *blk_mq_alloc_request(struct request_queue *q, int rw,
gfp_t gfp, bool reserved);
struct request *blk_mq_tag_to_rq(struct blk_mq_tags *tags, unsigned int tag);
+enum {
+ BLK_MQ_UNIQUE_TAG_TAG_BITS = 16,
+ BLK_MQ_UNIQUE_TAG_TAG_MASK = (1 << BLK_MQ_UNIQUE_TAG_TAG_BITS) - 1,
+};
+
+u32 blk_mq_unique_tag(struct request *rq);
+
+static inline u32 blk_mq_build_unique_tag(int hwq, int tag)
+{
+ return (hwq << BLK_MQ_UNIQUE_TAG_TAG_BITS) |
+ (tag & BLK_MQ_UNIQUE_TAG_TAG_MASK);
+}
+
+static inline u16 blk_mq_unique_tag_to_hwq(u32 unique_tag)
+{
+ return unique_tag >> BLK_MQ_UNIQUE_TAG_TAG_BITS;
+}
+
+static inline u16 blk_mq_unique_tag_to_tag(u32 unique_tag)
+{
+ return unique_tag & BLK_MQ_UNIQUE_TAG_TAG_MASK;
+}
+
struct blk_mq_hw_ctx *blk_mq_map_queue(struct request_queue *, const int ctx_index);
struct blk_mq_hw_ctx *blk_mq_alloc_single_hw_queue(struct blk_mq_tag_set *, unsigned int, int);
--
1.8.4.5
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-10-03 13:01 ` Bart Van Assche
@ 2014-10-03 14:24 ` Jens Axboe
0 siblings, 0 replies; 55+ messages in thread
From: Jens Axboe @ 2014-10-03 14:24 UTC (permalink / raw)
To: Bart Van Assche, Ming Lei
Cc: linux-scsi, linux-rdma, Christoph Hellwig, Robert Elliott
On 2014-10-03 07:01, Bart Van Assche wrote:
> On 10/02/14 18:55, Jens Axboe wrote:
>> Sure, that's fine as well, but the function needs a more descriptive
>> name. I try to think of it like I have never looked at the code and need
>> to write a driver, it's a lot easier if the functions are named
>> appropriately. Seeing blk_mq_rq_tag() and even with reading the function
>> comment, I'm really none the wiser and would assume I need to use this
>> function to get the tag.
>>
>> So we can do the single function, but lets call it
>> blk_mq_unique_rq_tag(). That's special enough that people will know this
>> is something that doesn't just return the request tag. Then add an extra
>> sentence to the comment you already have on when this is needed.
>>
>> And lets roll those bitshift values and masks into a define or enum so
>> it's collected in one place.
>
> How about the patch below ? In that patch all comments should have been
> addressed that Christoph and you have formulated so far.
Looks good to me now. Get rid of the extra TAG in the
BLK_MQ_UNIQUE_TAG_TAG_BITS/MASK naming though, then you can add my
acked-by if Christoph wants to take this through the scsi tree.
--
Jens Axboe
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
2014-10-02 17:30 ` Christoph Hellwig
@ 2014-10-06 11:16 ` Bart Van Assche
[not found] ` <54327A21.6070202-HInyCGIudOg@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-10-06 11:16 UTC (permalink / raw)
To: Christoph Hellwig
Cc: Jens Axboe, Ming Lei, linux-scsi, linux-rdma, Robert Elliott,
Roland Dreier
On 10/02/14 19:30, Christoph Hellwig wrote:
> Also if we want to merge scsi LLDDs that can take advantage of
> multiqueue support it would probably be best if I take this via the SCSI
> tree.
Sending these patches to you is fine with me, at least if Roland agrees.
Bart.
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <5421A093.1070203-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-09-23 19:02 ` Bart Van Assche
@ 2014-10-07 12:51 ` Bart Van Assche
[not found] ` <5433E1B5.1030103-HInyCGIudOg@public.gmane.org>
1 sibling, 1 reply; 55+ messages in thread
From: Bart Van Assche @ 2014-10-07 12:51 UTC (permalink / raw)
To: Sagi Grimberg, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 09/23/14 18:32, Sagi Grimberg wrote:
> Since you don't seem to negotiate/declare multichannel with the target,
> did you test this code with some target implementations other than SCST
> that happen to be out there?
(replying to an e-mail of two weeks ago)
Hello Sagi,
I have just verified that the multichannel code in this patch series
works fine in combination with the upstream SRP target driver.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <54327A21.6070202-HInyCGIudOg@public.gmane.org>
@ 2014-10-10 20:16 ` Roland Dreier
0 siblings, 0 replies; 55+ messages in thread
From: Roland Dreier @ 2014-10-10 20:16 UTC (permalink / raw)
To: Bart Van Assche
Cc: Christoph Hellwig, Jens Axboe, Ming Lei,
linux-scsi-u79uwXL29TY76Z2rM5mHXA, linux-rdma, Robert Elliott
On Mon, Oct 6, 2014 at 4:16 AM, Bart Van Assche <bvanassche-HInyCGIudOg@public.gmane.org> wrote:
> On 10/02/14 19:30, Christoph Hellwig wrote:
>> Also if we want to merge scsi LLDDs that can take advantage of
>> multiqueue support it would probably be best if I take this via the SCSI
>> tree.
> Sending these patches to you is fine with me, at least if Roland agrees.
That's fine with me. Christoph/Bart should I just let you guys handle
all the pending SRP patches, or is there anything I should pick up via
the InfiniBand tree?
Everything that's in flight looks reasonable to me, so I'm fine with
however you guys want to merge it.
- R.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <5433E1B5.1030103-HInyCGIudOg@public.gmane.org>
@ 2014-10-13 8:17 ` Sagi Grimberg
[not found] ` <543B8AB0.1090704-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
0 siblings, 1 reply; 55+ messages in thread
From: Sagi Grimberg @ 2014-10-13 8:17 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 10/7/2014 3:51 PM, Bart Van Assche wrote:
> On 09/23/14 18:32, Sagi Grimberg wrote:
>> Since you don't seem to negotiate/declare multichannel with the target,
>> did you test this code with some target implementations other than SCST
>> that happen to be out there?
>
> (replying to an e-mail of two weeks ago)
>
> Hello Sagi,
>
> I have just verified that the multichannel code in this patch series
> works fine in combination with the upstream SRP target driver.
>
Working as in single channel mode? or multichannel mode?
Sagi.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
* Re: [PATCH 8/8] IB/srp: Add multichannel support
[not found] ` <543B8AB0.1090704-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
@ 2014-10-13 8:52 ` Bart Van Assche
0 siblings, 0 replies; 55+ messages in thread
From: Bart Van Assche @ 2014-10-13 8:52 UTC (permalink / raw)
To: Sagi Grimberg, linux-scsi-u79uwXL29TY76Z2rM5mHXA
Cc: linux-rdma, Christoph Hellwig, Jens Axboe, Robert Elliott, Ming Lei
On 10/13/14 10:17, Sagi Grimberg wrote:
> On 10/7/2014 3:51 PM, Bart Van Assche wrote:
>> On 09/23/14 18:32, Sagi Grimberg wrote:
>>> Since you don't seem to negotiate/declare multichannel with the target,
>>> did you test this code with some target implementations other than SCST
>>> that happen to be out there?
>>
>> (replying to an e-mail of two weeks ago)
>>
>> I have just verified that the multichannel code in this patch series
>> works fine in combination with the upstream SRP target driver.
>
> Working as in single channel mode? or multichannel mode?
Hello Sagi,
In my e-mail I was referring to multichannel mode.
Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 55+ messages in thread
end of thread, other threads:[~2014-10-13 8:52 UTC | newest]
Thread overview: 55+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-09-19 12:55 [PATCH RFC 0/8] IB/srp: Add multichannel support Bart Van Assche
2014-09-19 12:56 ` Bart Van Assche
2014-09-19 18:02 ` Sagi Grimberg
2014-09-19 12:57 ` [PATCH 2/8] scsi-mq: Add support for multiple hardware queues Bart Van Assche
[not found] ` <541C281E.9090206-HInyCGIudOg@public.gmane.org>
2014-09-19 18:05 ` Sagi Grimberg
2014-09-19 18:11 ` Christoph Hellwig
[not found] ` <CAF9gx6JfP2bGyMauB6LzepZP_vKEvrd-sPZc5CRuOrtgQ_UCSw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-26 11:08 ` Ming Lei
[not found] ` <CACVXFVMiYsW=dszQ6mE-o_L8fEDdkO59vJ5qeHKch5c33K_QXw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-26 14:02 ` Bart Van Assche
2014-09-19 12:57 ` [PATCH 3/8] scsi-mq: Pass hctx to low-level SCSI drivers Bart Van Assche
[not found] ` <541C27BF.6070609-HInyCGIudOg@public.gmane.org>
2014-09-19 12:58 ` [PATCH 4/8] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib() Bart Van Assche
[not found] ` <541C285B.5010309-HInyCGIudOg@public.gmane.org>
2014-09-19 18:10 ` Sagi Grimberg
2014-09-19 12:58 ` [PATCH 5/8] IB/srp: Remove stale connection retry mechanism Bart Van Assche
2014-09-19 18:25 ` Sagi Grimberg
[not found] ` <541C287D.1050900-HInyCGIudOg@public.gmane.org>
2014-09-20 17:45 ` Or Gerlitz
[not found] ` <CAJ3xEMhPKiut4MwZH9F7-T0+u7B6XPuh-FTZpA=Xe4ViAj5UUQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-10-02 10:34 ` Bart Van Assche
[not found] ` <542D2A3C.2080009-HInyCGIudOg@public.gmane.org>
2014-10-03 8:51 ` Bart Van Assche
2014-09-19 13:00 ` [PATCH 8/8] IB/srp: Add multichannel support Bart Van Assche
[not found] ` <541C28E0.7010705-HInyCGIudOg@public.gmane.org>
2014-09-19 14:28 ` Ming Lei
[not found] ` <CACVXFVPzz37J-613NZCfPStUBxf0rLOtz71LJ07PpCxYg5nn+g-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-19 15:21 ` Bart Van Assche
[not found] ` <541C49EC.6030404-HInyCGIudOg@public.gmane.org>
2014-09-19 15:27 ` Ming Lei
2014-09-19 15:35 ` Bart Van Assche
2014-09-19 15:38 ` Jens Axboe
2014-09-19 17:30 ` Sagi Grimberg
2014-09-19 17:33 ` Jens Axboe
2014-09-19 18:11 ` Christoph Hellwig
[not found] ` <541C4DF1.4090604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-01 16:08 ` Bart Van Assche
2014-10-01 16:54 ` Jens Axboe
2014-10-01 21:14 ` Christoph Hellwig
[not found] ` <542C31C4.1020702-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-02 16:45 ` Bart Van Assche
2014-10-02 16:55 ` Jens Axboe
[not found] ` <542D8368.8080604-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-10-03 13:01 ` Bart Van Assche
2014-10-03 14:24 ` Jens Axboe
[not found] ` <542D8143.3050305-HInyCGIudOg@public.gmane.org>
2014-10-02 17:30 ` Christoph Hellwig
2014-10-06 11:16 ` Bart Van Assche
[not found] ` <54327A21.6070202-HInyCGIudOg@public.gmane.org>
2014-10-10 20:16 ` Roland Dreier
2014-09-23 16:32 ` Sagi Grimberg
[not found] ` <5421A093.1070203-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-09-23 19:02 ` Bart Van Assche
2014-09-24 12:22 ` Sagi Grimberg
2014-09-24 13:13 ` Bart Van Assche
[not found] ` <5422C395.7090902-HInyCGIudOg@public.gmane.org>
2014-09-24 13:38 ` Sagi Grimberg
[not found] ` <5422C970.4050306-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-09-24 13:43 ` Sagi Grimberg
2014-10-07 12:51 ` Bart Van Assche
[not found] ` <5433E1B5.1030103-HInyCGIudOg@public.gmane.org>
2014-10-13 8:17 ` Sagi Grimberg
[not found] ` <543B8AB0.1090704-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2014-10-13 8:52 ` Bart Van Assche
2014-10-01 16:14 ` [PATCH RFC] scsi_tcq.h: Add support for multiple hardware queues Bart Van Assche
2014-09-19 12:59 ` [PATCH 6/8] IB/srp: Avoid that I/O hangs due to a cable pull during LUN scanning Bart Van Assche
2014-09-19 12:59 ` [PATCH 7/8] IB/srp: Separate target and channel variables Bart Van Assche
2014-09-19 18:47 ` Sagi Grimberg
[not found] ` <541C28C8.7000007-HInyCGIudOg@public.gmane.org>
2014-09-23 16:07 ` Sagi Grimberg
2014-09-23 20:00 ` Bart Van Assche
2014-09-19 18:31 ` [PATCH RFC 0/8] IB/srp: Add multichannel support Jens Axboe
2014-09-22 14:37 ` Christoph Hellwig
[not found] ` <20140922143731.GA15377-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-09-22 16:25 ` Bart Van Assche
2014-09-22 16:31 ` Jens Axboe
2014-09-22 16:39 ` Jens Axboe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.