[PATCH 0/3] iscsi-target: percpu_ida tag starvation regression fixes

[PATCH 0/3] iscsi-target: percpu_ida tag starvation regression fixes

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

Hi folks,

This series addresses a v3.12 starvation regression in iscsi-target
after the conversion to use percpu_ida tag preallocation for all
iscsi_cmd descriptors.

The first patch addresses an unnecessary overhead during iscsi_cmd
acknowledgement -> release, that could end up causing an extra
context switch to occur for WRITEs.  With the advent of proper
se_cmd->cmd_kref handling in v3.10, this is no longer required
in the acknowledgement fast path.

The second patch converts iscsit_ack_from_expstatsn() to populate
a local list of acknowledged iscsi_cmd descriptors, and releases
them as quickly as possible (and the associated percpu_ida tags)
directly from RX thread context.

The third patch adds another (tag_num / 2) extra pre-allocated tags,
in order to prevent percpu_ida_alloc() from needing to steal tags
when the number of acknowledged tags in use is large, seperate from
the tags for the CmdSN window depth.

I'll be including into target-pending/master shortly, and sending
off a PULL request to include for v3.12-rc4 over the next days.

Thanks,

--nab

Nicholas Bellinger (3):
  iscsi-target: Only perform wait_for_tasks when performing shutdown
  iscsi-target: Perform release of acknowledged tags from RX context
  iscsi-target; Allow an extra tag_num / 2 number of percpu_ida tags

 drivers/target/iscsi/iscsi_target.c      |   13 +++++++++----
 drivers/target/iscsi/iscsi_target_nego.c |    2 +-
 drivers/target/iscsi/iscsi_target_util.c |    4 ++--
 3 files changed, 12 insertions(+), 7 deletions(-)

-- 
1.7.10.4

[PATCH 1/3] iscsi-target: Only perform wait_for_tasks when performing shutdown

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch changes transport_generic_free_cmd() to only wait_for_tasks
when shutdown=true is passed to iscsit_free_cmd().

With the advent of >= v3.10 iscsi-target code using se_cmd->cmd_kref,
the extra wait_for_tasks with shutdown=false is unnecessary, and may
end up causing an extra context switch when releasing WRITEs.

Cc: stable@vger.kernel.org # 3.10+
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/iscsi/iscsi_target_util.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c
index f2de28e..b0cac0c 100644
--- a/drivers/target/iscsi/iscsi_target_util.c
+++ b/drivers/target/iscsi/iscsi_target_util.c
@@ -736,7 +736,7 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)
 		 * Fallthrough
 		 */
 	case ISCSI_OP_SCSI_TMFUNC:
-		rc = transport_generic_free_cmd(&cmd->se_cmd, 1);
+		rc = transport_generic_free_cmd(&cmd->se_cmd, shutdown);
 		if (!rc && shutdown && se_cmd && se_cmd->se_sess) {
 			__iscsit_free_cmd(cmd, true, shutdown);
 			target_put_sess_cmd(se_cmd->se_sess, se_cmd);
@@ -752,7 +752,7 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)
 			se_cmd = &cmd->se_cmd;
 			__iscsit_free_cmd(cmd, true, shutdown);
 
-			rc = transport_generic_free_cmd(&cmd->se_cmd, 1);
+			rc = transport_generic_free_cmd(&cmd->se_cmd, shutdown);
 			if (!rc && shutdown && se_cmd->se_sess) {
 				__iscsit_free_cmd(cmd, true, shutdown);
 				target_put_sess_cmd(se_cmd->se_sess, se_cmd);
-- 
1.7.10.4

[PATCH 2/3] iscsi-target: Perform release of acknowledged tags from RX context

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts iscsit_ack_from_expstatsn() to populate a local
ack_list of commands, and call iscsit_free_cmd() directly from RX
thread context, instead of using iscsit_add_cmd_to_immediate_queue()
to queue the acknowledged commands to be released from TX thread
context.

It is helpful to release the acknowledge commands as quickly as
possible, along with the associated percpu_ida tags, in order to
prevent percpu_ida_alloc() from having to steal tags from other
CPUs while waiting for iscsit_free_cmd() to happen from TX thread
context.

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/iscsi/iscsi_target.c |   13 +++++++++----
 1 file changed, 9 insertions(+), 4 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 35b61f7..38e44b9 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -753,7 +753,8 @@ static void iscsit_unmap_iovec(struct iscsi_cmd *cmd)
 
 static void iscsit_ack_from_expstatsn(struct iscsi_conn *conn, u32 exp_statsn)
 {
-	struct iscsi_cmd *cmd;
+	LIST_HEAD(ack_list);
+	struct iscsi_cmd *cmd, *cmd_p;
 
 	conn->exp_statsn = exp_statsn;
 
@@ -761,19 +762,23 @@ static void iscsit_ack_from_expstatsn(struct iscsi_conn *conn, u32 exp_statsn)
 		return;
 
 	spin_lock_bh(&conn->cmd_lock);
-	list_for_each_entry(cmd, &conn->conn_cmd_list, i_conn_node) {
+	list_for_each_entry_safe(cmd, cmd_p, &conn->conn_cmd_list, i_conn_node) {
 		spin_lock(&cmd->istate_lock);
 		if ((cmd->i_state == ISTATE_SENT_STATUS) &&
 		    iscsi_sna_lt(cmd->stat_sn, exp_statsn)) {
 			cmd->i_state = ISTATE_REMOVE;
 			spin_unlock(&cmd->istate_lock);
-			iscsit_add_cmd_to_immediate_queue(cmd, conn,
-						cmd->i_state);
+			list_move_tail(&cmd->i_conn_node, &ack_list);
 			continue;
 		}
 		spin_unlock(&cmd->istate_lock);
 	}
 	spin_unlock_bh(&conn->cmd_lock);
+
+	list_for_each_entry_safe(cmd, cmd_p, &ack_list, i_conn_node) {
+		list_del(&cmd->i_conn_node);
+		iscsit_free_cmd(cmd, false);
+	}
 }
 
 static int iscsit_allocate_iovecs(struct iscsi_cmd *cmd)
-- 
1.7.10.4

[PATCH 3/3] iscsi-target; Allow an extra tag_num / 2 number of percpu_ida tags

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch bumps the default number of tags allocated per session by
iscsi-target via transport_alloc_session_tags() -> percpu_ida_init()
by another (tag_num / 2).

This is done to take into account the tags waiting to be acknowledged
and released in iscsit_ack_from_expstatsn(), but who's number are not
directly limited by the CmdSN Window queue_depth being enforced by
the target.

Using a larger value here is also useful to prevent percpu_ida_alloc()
from having to steal tags from other CPUs when no tags are available
on the local CPU, while waiting for unacknowledged tags to be released.

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/iscsi/iscsi_target_nego.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/target/iscsi/iscsi_target_nego.c b/drivers/target/iscsi/iscsi_target_nego.c
index 14d1aed..ef6d836 100644
--- a/drivers/target/iscsi/iscsi_target_nego.c
+++ b/drivers/target/iscsi/iscsi_target_nego.c
@@ -1192,7 +1192,7 @@ get_target:
 	 */
 alloc_tags:
 	tag_num = max_t(u32, ISCSIT_MIN_TAGS, queue_depth);
-	tag_num += ISCSIT_EXTRA_TAGS;
+	tag_num += (tag_num / 2) + ISCSIT_EXTRA_TAGS;
 	tag_size = sizeof(struct iscsi_cmd) + conn->conn_transport->priv_size;
 
 	ret = transport_alloc_session_tags(sess->se_sess, tag_num, tag_size);
-- 
1.7.10.4