[RFC 00/22] target: se_node_acl LUN list RCU conversion

[RFC 00/22] target: se_node_acl LUN list RCU conversion

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

Hi Hannes, HCH, & Sagi,

Here is an initial pass at the conversion of se_node_acl->device_list[]
to use RCU protected pointers for se_lun fast-path lookup code.

The big advantage with RCU is that transport_lookup_cmd_lun() can now
run completely lock-less using existing per-cpu se_lun->lun_ref logic.

Currently the new se_node_acl->lun_entry_hlist is still using an array
of pointers allocated at se_node_acl creation time.  The next step on
my end is to convert the entries to mempool allocation + rcu_head ->
call_rcu() asynchronous release.  Beyond that, RCU hlist support for
both NodeACL MappedLUN and TPG LUN tables is the final goal.

Some of the more interesting conversions so far are in target_core_pr.c
code for PROUT operations that lookup remote I_T LUNs and take configfs
dependencies.  This includes a new se_dev_entry->pr_ref to handle the
REGISTER w/ I_PORT descriptors + REGISTER_AND_MOVE special cases.

The series is broken up for review, and also includes a struct mutex
conversion of existing se_node_acl and se_portal_group LUN locks to
simplify a number of existing codepaths.

There are still a number of outstanding FIXMEs to address, and at this
point the series is stable enough to function with CONFIG_PROVE_RCU=y.
However, it still needs alot more testing with active I/O shutdown and
complex PR use cases.

Along with other cleanups, the target_* prefix rename discussed at LSF
is also included at the series end.

Please have a look.

--nab

Nicholas Bellinger (22):
  target: Convert transport_lookup_*_lun to RCU reader
  target: Convert enable/disable ->device_list to RCU updater
  target/device: Convert se_node_acl->device_list access to RCU reader
  target/configfs: Convert mappedlun link/unlink to RCU reader
  target/configfs: Convert SCSI MIB attrs to RCU reader
  target/spc: Convert REPORT_LUN + MODE_SENSE to RCU reader
  target/pscsi: Convert MODE_SENSE special case to RCU reader
  target/pr: Convert se_dev_entry to percpu-refcount for RCU
  target/pr: Convert registration check to RCU pointer
  target/pr: Change alloc_registration to avoid pr_reg_tg_pt_lun
  target: Convert UNIT_ATTENTION logic to RCU reader
  target: Convert se_tpg->tpg_lun_lock to ->tpg_lun_mutex
  target: Convert se_tpg->acl_node_lock to ->acl_node_mutex
  target: Convert se_node_acl->lun_entry_lock to ->lun_entry_mutex
  target: Convert core_tpg_deregister to use list splice
  target: Drop se_lun->lun_acl_list
  target: Drop core_tpg_clear_object_luns
  target: Rename TPG initiator_node_acl to target_* prefix
  target: Rename TPG register/deregister to target_* prefix
  target: Rename LUN lookup/add/remove to target_* prefix
  target: Rename se_node_acl->lun_entry_hlist to target_* prefix
  target: Rename se_port/se_device export to target_* prefix

 drivers/infiniband/ulp/srpt/ib_srpt.c        |  15 +-
 drivers/scsi/qla2xxx/tcm_qla2xxx.c           |  22 +-
 drivers/target/iscsi/iscsi_target.c          |  10 +-
 drivers/target/iscsi/iscsi_target_configfs.c |  17 +-
 drivers/target/iscsi/iscsi_target_nego.c     |   4 +-
 drivers/target/iscsi/iscsi_target_tpg.c      |  12 +-
 drivers/target/loopback/tcm_loop.c           |  14 +-
 drivers/target/sbp/sbp_target.c              |  34 ++-
 drivers/target/target_core_configfs.c        |   6 +-
 drivers/target/target_core_device.c          | 295 ++++++++++++++-------------
 drivers/target/target_core_fabric_configfs.c |  29 +--
 drivers/target/target_core_internal.h        |  24 +--
 drivers/target/target_core_pr.c              | 213 +++++++++++--------
 drivers/target/target_core_pscsi.c           |  14 +-
 drivers/target/target_core_spc.c             |  28 ++-
 drivers/target/target_core_stat.c            | 168 +++++++--------
 drivers/target/target_core_tpg.c             | 244 ++++++++++------------
 drivers/target/target_core_transport.c       |  24 +--
 drivers/target/target_core_ua.c              |  35 ++--
 drivers/target/tcm_fc/tfc_conf.c             |  16 +-
 drivers/usb/gadget/legacy/tcm_usb_gadget.c   |  22 +-
 drivers/vhost/scsi.c                         |  18 +-
 drivers/xen/xen-scsiback.c                   |   8 +-
 include/target/target_core_base.h            |  23 +--
 include/target/target_core_fabric.h          |  21 +-
 25 files changed, 678 insertions(+), 638 deletions(-)

-- 
1.9.1

[RFC 01/22] target: Convert transport_lookup_*_lun to RCU reader

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts transport_lookup_*_lun() fast-path code to
use RCU read path primitives when looking up se_dev_entry.  It
adds a new array of pointers in se_node_acl->lun_entry_hlist
for this purpose.

For transport_lookup_cmd_lun() code, it works with existing per-cpu
se_lun->lun_ref when associating se_cmd with se_lun + se_device.

Also, go ahead and update core_create_device_list_for_node() +
core_free_device_list_for_node() to use ->lun_entry_hlist.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c | 45 +++++++++++++++++++++++++------------
 drivers/target/target_core_tpg.c    | 10 +++++----
 include/target/target_core_base.h   |  2 ++
 3 files changed, 39 insertions(+), 18 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 37449bd..be893c8 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -59,17 +59,24 @@ transport_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
 {
 	struct se_lun *se_lun = NULL;
 	struct se_session *se_sess = se_cmd->se_sess;
+	struct se_node_acl *nacl = se_sess->se_node_acl;
 	struct se_device *dev;
-	unsigned long flags;
+	struct se_dev_entry *deve;
 
 	if (unpacked_lun >= TRANSPORT_MAX_LUNS_PER_TPG)
 		return TCM_NON_EXISTENT_LUN;
 
-	spin_lock_irqsave(&se_sess->se_node_acl->device_list_lock, flags);
-	se_cmd->se_deve = se_sess->se_node_acl->device_list[unpacked_lun];
-	if (se_cmd->se_deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
-		struct se_dev_entry *deve = se_cmd->se_deve;
-
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[unpacked_lun]);
+	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
+		/*
+		 * Make sure that target_enable_device_list_for_node()
+		 * has not already cleared the RCU protected pointers.
+		 */
+		if (!deve->se_lun) {
+			rcu_read_unlock();
+			goto check_lun;
+		}
 		deve->total_cmds++;
 
 		if ((se_cmd->data_direction == DMA_TO_DEVICE) &&
@@ -78,7 +85,7 @@ transport_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
 				" Access for 0x%08x\n",
 				se_cmd->se_tfo->get_fabric_name(),
 				unpacked_lun);
-			spin_unlock_irqrestore(&se_sess->se_node_acl->device_list_lock, flags);
+			rcu_read_unlock();
 			return TCM_WRITE_PROTECTED;
 		}
 
@@ -96,8 +103,9 @@ transport_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
 		percpu_ref_get(&se_lun->lun_ref);
 		se_cmd->lun_ref_active = true;
 	}
-	spin_unlock_irqrestore(&se_sess->se_node_acl->device_list_lock, flags);
+	rcu_read_unlock();
 
+check_lun:
 	if (!se_lun) {
 		/*
 		 * Use the se_portal_group->tpg_virt_lun0 to allow for
@@ -146,25 +154,34 @@ int transport_lookup_tmr_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
 	struct se_dev_entry *deve;
 	struct se_lun *se_lun = NULL;
 	struct se_session *se_sess = se_cmd->se_sess;
+	struct se_node_acl *nacl = se_sess->se_node_acl;
 	struct se_tmr_req *se_tmr = se_cmd->se_tmr_req;
 	unsigned long flags;
 
 	if (unpacked_lun >= TRANSPORT_MAX_LUNS_PER_TPG)
 		return -ENODEV;
 
-	spin_lock_irqsave(&se_sess->se_node_acl->device_list_lock, flags);
-	se_cmd->se_deve = se_sess->se_node_acl->device_list[unpacked_lun];
-	deve = se_cmd->se_deve;
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[unpacked_lun]);
 
 	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
+		/*
+		 * Make sure that target_enable_device_list_for_node()
+		 * has not already cleared the RCU protected pointers.
+		 */
+		if (!deve->se_lun) {
+			rcu_read_unlock();
+			goto check_lun;
+		}
 		se_tmr->tmr_lun = deve->se_lun;
 		se_cmd->se_lun = deve->se_lun;
 		se_lun = deve->se_lun;
 		se_cmd->pr_res_key = deve->pr_res_key;
 		se_cmd->orig_fe_lun = unpacked_lun;
 	}
-	spin_unlock_irqrestore(&se_sess->se_node_acl->device_list_lock, flags);
+	rcu_read_unlock();
 
+check_lun:
 	if (!se_lun) {
 		pr_debug("TARGET_CORE[%s]: Detected NON_EXISTENT_LUN"
 			" Access for 0x%08x\n",
@@ -267,8 +284,8 @@ int core_free_device_list_for_node(
 	}
 	spin_unlock_irq(&nacl->device_list_lock);
 
-	array_free(nacl->device_list, TRANSPORT_MAX_LUNS_PER_TPG);
-	nacl->device_list = NULL;
+	array_free(nacl->lun_entry_hlist, TRANSPORT_MAX_LUNS_PER_TPG);
+	nacl->lun_entry_hlist = NULL;
 
 	return 0;
 }
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 0696de9..b2fdba5 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -234,15 +234,15 @@ static int core_create_device_list_for_node(struct se_node_acl *nacl)
 	struct se_dev_entry *deve;
 	int i;
 
-	nacl->device_list = array_zalloc(TRANSPORT_MAX_LUNS_PER_TPG,
+	nacl->lun_entry_hlist = array_zalloc(TRANSPORT_MAX_LUNS_PER_TPG,
 			sizeof(struct se_dev_entry), GFP_KERNEL);
-	if (!nacl->device_list) {
+	if (!nacl->lun_entry_hlist) {
 		pr_err("Unable to allocate memory for"
-			" struct se_node_acl->device_list\n");
+			" struct se_node_acl->lun_entry_hlist\n");
 		return -ENOMEM;
 	}
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
-		deve = nacl->device_list[i];
+		deve = nacl->lun_entry_hlist[i];
 
 		atomic_set(&deve->ua_count, 0);
 		atomic_set(&deve->pr_ref_count, 0);
@@ -281,6 +281,7 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
 	init_completion(&acl->acl_free_comp);
 	spin_lock_init(&acl->device_list_lock);
 	spin_lock_init(&acl->nacl_sess_lock);
+	spin_lock_init(&acl->lun_entry_lock);
 	atomic_set(&acl->acl_pr_ref_count, 0);
 	acl->queue_depth = tpg->se_tpg_tfo->tpg_get_default_depth(tpg);
 	snprintf(acl->initiatorname, TRANSPORT_IQN_LEN, "%s", initiatorname);
@@ -408,6 +409,7 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
 	init_completion(&acl->acl_free_comp);
 	spin_lock_init(&acl->device_list_lock);
 	spin_lock_init(&acl->nacl_sess_lock);
+	spin_lock_init(&acl->lun_entry_lock);
 	atomic_set(&acl->acl_pr_ref_count, 0);
 	acl->queue_depth = queue_depth;
 	snprintf(acl->initiatorname, TRANSPORT_IQN_LEN, "%s", initiatorname);
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index fe25a78..06ecd7b 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -595,10 +595,12 @@ struct se_node_acl {
 	char			acl_tag[MAX_ACL_TAG_SIZE];
 	/* Used for PR SPEC_I_PT=1 and REGISTER_AND_MOVE */
 	atomic_t		acl_pr_ref_count;
+	struct se_dev_entry __rcu **lun_entry_hlist;
 	struct se_dev_entry	**device_list;
 	struct se_session	*nacl_sess;
 	struct se_portal_group *se_tpg;
 	spinlock_t		device_list_lock;
+	spinlock_t		lun_entry_lock;
 	spinlock_t		nacl_sess_lock;
 	struct config_group	acl_group;
 	struct config_group	acl_attrib_group;
-- 
1.9.1

[RFC 02/22] target: Convert enable/disable ->device_list to RCU updater

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts core_[enable,disable]_device_list_for_node() to RCU
updater path code when modifying se_dev_entry pointers.

It includes protected rcu_assign_pointer() and invokes synchronize_rcu()
to wait for RCU read paths to finish.

Required for subsequent conversion to se_deve->pr_ref percpu-refcount.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c | 50 ++++++++++++++++---------------------
 1 file changed, 22 insertions(+), 28 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index be893c8..9385e16 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -324,31 +324,16 @@ int core_enable_device_list_for_node(
 	struct se_port *port = lun->lun_sep;
 	struct se_dev_entry *deve;
 
-	spin_lock_irq(&nacl->device_list_lock);
-
-	deve = nacl->device_list[mapped_lun];
-
 	/*
 	 * Check if the call is handling demo mode -> explicit LUN ACL
 	 * transition.  This transition must be for the same struct se_lun
 	 * + mapped_lun that was setup in demo mode..
 	 */
+	spin_lock_irq(&nacl->lun_entry_lock);
+	deve = nacl->lun_entry_hlist[mapped_lun];
 	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
-		if (deve->se_lun_acl != NULL) {
-			pr_err("struct se_dev_entry->se_lun_acl"
-			       " already set for demo mode -> explicit"
-			       " LUN ACL transition\n");
-			spin_unlock_irq(&nacl->device_list_lock);
-			return -EINVAL;
-		}
-		if (deve->se_lun != lun) {
-			pr_err("struct se_dev_entry->se_lun does"
-			       " match passed struct se_lun for demo mode"
-			       " -> explicit LUN ACL transition\n");
-			spin_unlock_irq(&nacl->device_list_lock);
-			return -EINVAL;
-		}
-		deve->se_lun_acl = lun_acl;
+		BUG_ON(deve->se_lun_acl != NULL);
+		BUG_ON(deve->se_lun != lun);
 
 		if (lun_access & TRANSPORT_LUNFLAGS_READ_WRITE) {
 			deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_ONLY;
@@ -357,13 +342,13 @@ int core_enable_device_list_for_node(
 			deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_WRITE;
 			deve->lun_flags |= TRANSPORT_LUNFLAGS_READ_ONLY;
 		}
+		rcu_assign_pointer(deve->se_lun_acl, lun_acl);
+		spin_unlock_irq(&nacl->lun_entry_lock);
 
-		spin_unlock_irq(&nacl->device_list_lock);
+		synchronize_rcu();
 		return 0;
 	}
 
-	deve->se_lun = lun;
-	deve->se_lun_acl = lun_acl;
 	deve->mapped_lun = mapped_lun;
 	deve->lun_flags |= TRANSPORT_LUNFLAGS_INITIATOR_ACCESS;
 
@@ -377,12 +362,16 @@ int core_enable_device_list_for_node(
 
 	deve->creation_time = get_jiffies_64();
 	deve->attach_count++;
-	spin_unlock_irq(&nacl->device_list_lock);
+
+	rcu_assign_pointer(deve->se_lun, lun);
+	rcu_assign_pointer(deve->se_lun_acl, lun_acl);
+	spin_unlock_irq(&nacl->lun_entry_lock);
 
 	spin_lock_bh(&port->sep_alua_lock);
 	list_add_tail(&deve->alua_port_list, &port->sep_alua_list);
 	spin_unlock_bh(&port->sep_alua_lock);
 
+	synchronize_rcu();
 	return 0;
 }
 
@@ -399,8 +388,10 @@ int core_disable_device_list_for_node(
 	struct se_portal_group *tpg)
 {
 	struct se_port *port = lun->lun_sep;
-	struct se_dev_entry *deve = nacl->device_list[mapped_lun];
+	struct se_dev_entry *deve;
 
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[mapped_lun]);
 	/*
 	 * If the MappedLUN entry is being disabled, the entry in
 	 * port->sep_alua_list must be removed now before clearing the
@@ -424,17 +415,20 @@ int core_disable_device_list_for_node(
 	while (atomic_read(&deve->pr_ref_count) != 0)
 		cpu_relax();
 
-	spin_lock_irq(&nacl->device_list_lock);
 	/*
 	 * Disable struct se_dev_entry LUN ACL mapping
 	 */
+	spin_lock_irq(&nacl->lun_entry_lock);
 	core_scsi3_ua_release_all(deve);
-	deve->se_lun = NULL;
-	deve->se_lun_acl = NULL;
+	rcu_assign_pointer(deve->se_lun, NULL);
+	rcu_assign_pointer(deve->se_lun_acl, NULL);
 	deve->lun_flags = 0;
 	deve->creation_time = 0;
 	deve->attach_count--;
-	spin_unlock_irq(&nacl->device_list_lock);
+	spin_unlock_irq(&nacl->lun_entry_lock);
+	rcu_read_unlock();
+
+	synchronize_rcu();
 
 	core_scsi3_free_pr_reg_from_nacl(lun->lun_se_dev, nacl);
 	return 0;
-- 
1.9.1

[RFC 03/22] target/device: Convert se_node_acl->device_list access to RCU reader

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts various target_core_device.c to RCU reader path
to extract deve->mapped_lun before rcu_read_unlock() ahead of invoking
core_disable_device_list_for_node() to trigger subsequent RCU updater
code.

It includes core_free_device_list_for_node(), core_clear_lun_from_tpg(),
core_clear_initiator_node_from_tpg(), and core_update_device_list_access()
updates.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c | 52 +++++++++++++++++++------------------
 drivers/target/target_core_tpg.c    | 22 ++++++++--------
 2 files changed, 38 insertions(+), 36 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 9385e16..1053a2d 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -257,32 +257,33 @@ int core_free_device_list_for_node(
 {
 	struct se_dev_entry *deve;
 	struct se_lun *lun;
-	u32 i;
+	u32 i, mapped_lun;
 
-	if (!nacl->device_list)
+	if (!nacl->lun_entry_hlist)
 		return 0;
 
-	spin_lock_irq(&nacl->device_list_lock);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
-		deve = nacl->device_list[i];
+		rcu_read_lock();
+		deve = rcu_dereference(nacl->lun_entry_hlist[i]);
 
-		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS))
+		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS)) {
+			rcu_read_unlock();
 			continue;
-
+		}
 		if (!deve->se_lun) {
 			pr_err("%s device entries device pointer is"
 				" NULL, but Initiator has access.\n",
 				tpg->se_tpg_tfo->get_fabric_name());
+			rcu_read_unlock();
 			continue;
 		}
 		lun = deve->se_lun;
+		mapped_lun = deve->mapped_lun;
+		rcu_read_unlock();
 
-		spin_unlock_irq(&nacl->device_list_lock);
-		core_disable_device_list_for_node(lun, NULL, deve->mapped_lun,
-			TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
-		spin_lock_irq(&nacl->device_list_lock);
+		core_disable_device_list_for_node(lun, NULL, mapped_lun,
+					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 	}
-	spin_unlock_irq(&nacl->device_list_lock);
 
 	array_free(nacl->lun_entry_hlist, TRANSPORT_MAX_LUNS_PER_TPG);
 	nacl->lun_entry_hlist = NULL;
@@ -297,8 +298,8 @@ void core_update_device_list_access(
 {
 	struct se_dev_entry *deve;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[mapped_lun];
+	spin_lock_irq(&nacl->lun_entry_lock);
+	deve = rcu_dereference(nacl->lun_entry_hlist[mapped_lun]);
 	if (lun_access & TRANSPORT_LUNFLAGS_READ_WRITE) {
 		deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_ONLY;
 		deve->lun_flags |= TRANSPORT_LUNFLAGS_READ_WRITE;
@@ -306,7 +307,9 @@ void core_update_device_list_access(
 		deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_WRITE;
 		deve->lun_flags |= TRANSPORT_LUNFLAGS_READ_ONLY;
 	}
-	spin_unlock_irq(&nacl->device_list_lock);
+	spin_unlock_irq(&nacl->lun_entry_lock);
+
+	synchronize_rcu();
 }
 
 /*      core_enable_device_list_for_node():
@@ -442,26 +445,25 @@ void core_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg)
 {
 	struct se_node_acl *nacl;
 	struct se_dev_entry *deve;
-	u32 i;
+	u32 i, mapped_lun;
 
 	spin_lock_irq(&tpg->acl_node_lock);
 	list_for_each_entry(nacl, &tpg->acl_node_list, acl_list) {
 		spin_unlock_irq(&tpg->acl_node_lock);
 
-		spin_lock_irq(&nacl->device_list_lock);
 		for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
-			deve = nacl->device_list[i];
-			if (lun != deve->se_lun)
+			rcu_read_lock();
+			deve = rcu_dereference(nacl->lun_entry_hlist[i]);
+			if (!deve || lun != deve->se_lun) {
+				rcu_read_unlock();
 				continue;
-			spin_unlock_irq(&nacl->device_list_lock);
-
-			core_disable_device_list_for_node(lun, NULL,
-				deve->mapped_lun, TRANSPORT_LUNFLAGS_NO_ACCESS,
-				nacl, tpg);
+			}
+			mapped_lun = deve->mapped_lun;
+			rcu_read_unlock();
 
-			spin_lock_irq(&nacl->device_list_lock);
+			core_disable_device_list_for_node(lun, NULL, mapped_lun,
+					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 		}
-		spin_unlock_irq(&nacl->device_list_lock);
 
 		spin_lock_irq(&tpg->acl_node_lock);
 	}
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index b2fdba5..15a683b 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -55,32 +55,32 @@ static void core_clear_initiator_node_from_tpg(
 	struct se_node_acl *nacl,
 	struct se_portal_group *tpg)
 {
-	int i;
 	struct se_dev_entry *deve;
 	struct se_lun *lun;
+	u32 i, mapped_lun;
 
-	spin_lock_irq(&nacl->device_list_lock);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
-		deve = nacl->device_list[i];
+		rcu_read_lock();
+		deve = rcu_dereference(nacl->lun_entry_hlist[i]);
 
-		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS))
+		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS)) {
+			rcu_read_unlock();
 			continue;
-
+		}
 		if (!deve->se_lun) {
 			pr_err("%s device entries device pointer is"
 				" NULL, but Initiator has access.\n",
 				tpg->se_tpg_tfo->get_fabric_name());
+			rcu_read_unlock();
 			continue;
 		}
-
 		lun = deve->se_lun;
-		spin_unlock_irq(&nacl->device_list_lock);
-		core_disable_device_list_for_node(lun, NULL, deve->mapped_lun,
-			TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
+		mapped_lun = deve->mapped_lun;
+		rcu_read_unlock();
 
-		spin_lock_irq(&nacl->device_list_lock);
+		core_disable_device_list_for_node(lun, NULL, mapped_lun,
+					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 	}
-	spin_unlock_irq(&nacl->device_list_lock);
 }
 
 /*	__core_tpg_get_initiator_node_acl():
-- 
1.9.1

[RFC 04/22] target/configfs: Convert mappedlun link/unlink to RCU reader

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts fabric independent configfs link/unlink to use
RCU read path macros for se_node_acl->lun_entry_hlist[] access.

Also, update show_write_protect attribute access as well.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_fabric_configfs.c | 29 ++++++++++++++++------------
 1 file changed, 17 insertions(+), 12 deletions(-)

diff --git a/drivers/target/target_core_fabric_configfs.c b/drivers/target/target_core_fabric_configfs.c
index 0c3f901..86ad50d 100644
--- a/drivers/target/target_core_fabric_configfs.c
+++ b/drivers/target/target_core_fabric_configfs.c
@@ -111,8 +111,8 @@ static int target_fabric_mappedlun_link(
 	 * which be will write protected (READ-ONLY) when
 	 * tpg_1/attrib/demo_mode_write_protect=1
 	 */
-	spin_lock_irq(&lacl->se_lun_nacl->device_list_lock);
-	deve = lacl->se_lun_nacl->device_list[lacl->mapped_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(lacl->se_lun_nacl->lun_entry_hlist[lacl->mapped_lun]);
 	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS)
 		lun_access = deve->lun_flags;
 	else
@@ -120,7 +120,7 @@ static int target_fabric_mappedlun_link(
 			(se_tpg->se_tpg_tfo->tpg_check_prod_mode_write_protect(
 				se_tpg)) ? TRANSPORT_LUNFLAGS_READ_ONLY :
 					   TRANSPORT_LUNFLAGS_READ_WRITE;
-	spin_unlock_irq(&lacl->se_lun_nacl->device_list_lock);
+	rcu_read_unlock();
 	/*
 	 * Determine the actual mapped LUN value user wants..
 	 *
@@ -141,14 +141,18 @@ static int target_fabric_mappedlun_unlink(
 	struct se_lun_acl *lacl = container_of(to_config_group(lun_acl_ci),
 			struct se_lun_acl, se_lun_group);
 	struct se_node_acl *nacl = lacl->se_lun_nacl;
-	struct se_dev_entry *deve = nacl->device_list[lacl->mapped_lun];
+	struct se_dev_entry *deve;
 	struct se_portal_group *se_tpg;
 	/*
 	 * Determine if the underlying MappedLUN has already been released..
 	 */
-	if (!deve->se_lun)
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun) {
+		rcu_read_lock();
 		return 0;
-
+	}
+	rcu_read_unlock();
 	lun = container_of(to_config_group(lun_ci), struct se_lun, lun_group);
 	se_tpg = lun->lun_sep->sep_tpg;
 
@@ -171,12 +175,13 @@ static ssize_t target_fabric_mappedlun_show_write_protect(
 	struct se_dev_entry *deve;
 	ssize_t len;
 
-	spin_lock_irq(&se_nacl->device_list_lock);
-	deve = se_nacl->device_list[lacl->mapped_lun];
-	len = sprintf(page, "%d\n",
-			(deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY) ?
-			1 : 0);
-	spin_unlock_irq(&se_nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(se_nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (deve) {
+		len = sprintf(page, "%d\n",
+			(deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY) ? 1 : 0);
+	}
+	rcu_read_unlock();
 
 	return len;
 }
-- 
1.9.1

[RFC 05/22] target/configfs: Convert SCSI MIB attrs to RCU reader

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts SCSI MIB configfs show attribute code to use
RCU read path macros for se_node_acl->lun_entry_hlist[] access.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_stat.c | 168 +++++++++++++++++++-------------------
 1 file changed, 84 insertions(+), 84 deletions(-)

diff --git a/drivers/target/target_core_stat.c b/drivers/target/target_core_stat.c
index 0353899..d09787b 100644
--- a/drivers/target/target_core_stat.c
+++ b/drivers/target/target_core_stat.c
@@ -1085,17 +1085,17 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_inst(
 	struct se_portal_group *tpg;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	tpg = nacl->se_tpg;
 	/* scsiInstIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n",
 			tpg->se_tpg_tfo->tpg_get_inst_index(tpg));
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(inst);
@@ -1110,16 +1110,16 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_dev(
 	struct se_lun *lun;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	lun = deve->se_lun;
 	/* scsiDeviceIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", lun->lun_se_dev->dev_index);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(dev);
@@ -1134,16 +1134,16 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_port(
 	struct se_portal_group *tpg;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	tpg = nacl->se_tpg;
 	/* scsiAuthIntrTgtPortIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", tpg->se_tpg_tfo->tpg_get_tag(tpg));
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(port);
@@ -1157,15 +1157,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_indx(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
 	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", nacl->acl_index);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(indx);
@@ -1179,15 +1179,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_dev_or_port(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
 	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrDevOrPort */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", 1);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(dev_or_port);
@@ -1201,15 +1201,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_intr_name(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrName */
 	ret = snprintf(page, PAGE_SIZE, "%s\n", nacl->initiatorname);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(intr_name);
@@ -1223,15 +1223,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_map_indx(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* FIXME: scsiAuthIntrLunMapIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", 0);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(map_indx);
@@ -1245,15 +1245,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_att_count(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrAttachedTimes */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", deve->attach_count);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(att_count);
@@ -1267,15 +1267,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_num_cmds(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrOutCommands */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", deve->total_cmds);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(num_cmds);
@@ -1289,15 +1289,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_read_mbytes(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrReadMegaBytes */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", (u32)(deve->read_bytes >> 20));
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(read_mbytes);
@@ -1311,15 +1311,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_write_mbytes(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrWrittenMegaBytes */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", (u32)(deve->write_bytes >> 20));
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(write_mbytes);
@@ -1333,15 +1333,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_hs_num_cmds(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* FIXME: scsiAuthIntrHSOutCommands */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", 0);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(hs_num_cmds);
@@ -1355,16 +1355,16 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_creation_time(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || !deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAuthIntrLastCreation */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", (u32)(((u32)deve->creation_time -
 				INITIAL_JIFFIES) * 100 / HZ));
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(creation_time);
@@ -1378,15 +1378,15 @@ static ssize_t target_stat_scsi_auth_intr_show_attr_row_status(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
-	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
+	if (!deve || deve->se_lun || !deve->se_lun_acl) {
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* FIXME: scsiAuthIntrRowStatus */
 	ret = snprintf(page, PAGE_SIZE, "Ready\n");
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_AUTH_INTR_ATTR_RO(row_status);
@@ -1451,17 +1451,17 @@ static ssize_t target_stat_scsi_att_intr_port_show_attr_inst(
 	struct se_portal_group *tpg;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
 	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	tpg = nacl->se_tpg;
 	/* scsiInstIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n",
 			tpg->se_tpg_tfo->tpg_get_inst_index(tpg));
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_ATTR_INTR_PORT_ATTR_RO(inst);
@@ -1476,16 +1476,16 @@ static ssize_t target_stat_scsi_att_intr_port_show_attr_dev(
 	struct se_lun *lun;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
 	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	lun = deve->se_lun;
 	/* scsiDeviceIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", lun->lun_se_dev->dev_index);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_ATTR_INTR_PORT_ATTR_RO(dev);
@@ -1500,16 +1500,16 @@ static ssize_t target_stat_scsi_att_intr_port_show_attr_port(
 	struct se_portal_group *tpg;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
 	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	tpg = nacl->se_tpg;
 	/* scsiPortIndex */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", tpg->se_tpg_tfo->tpg_get_tag(tpg));
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_ATTR_INTR_PORT_ATTR_RO(port);
@@ -1549,15 +1549,15 @@ static ssize_t target_stat_scsi_att_intr_port_show_attr_port_auth_indx(
 	struct se_dev_entry *deve;
 	ssize_t ret;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[lacl->mapped_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[lacl->mapped_lun]);
 	if (!deve->se_lun || !deve->se_lun_acl) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return -ENODEV;
 	}
 	/* scsiAttIntrPortAuthIntrIdx */
 	ret = snprintf(page, PAGE_SIZE, "%u\n", nacl->acl_index);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 	return ret;
 }
 DEV_STAT_SCSI_ATTR_INTR_PORT_ATTR_RO(port_auth_indx);
-- 
1.9.1

[RFC 06/22] target/spc: Convert REPORT_LUN + MODE_SENSE to RCU reader

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts SPC emulation for REPORT_LUN + MODE_SENSE to use
RCU read path macros for se_node_acl->lun_entry_hlist[] access.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_spc.c  | 28 ++++++++++++++++++----------
 include/target/target_core_base.h |  1 -
 2 files changed, 18 insertions(+), 11 deletions(-)

diff --git a/drivers/target/target_core_spc.c b/drivers/target/target_core_spc.c
index 5b6a0af..12cb045 100644
--- a/drivers/target/target_core_spc.c
+++ b/drivers/target/target_core_spc.c
@@ -986,6 +986,8 @@ static int spc_modesense_long_blockdesc(unsigned char *buf, u64 blocks, u32 bloc
 static sense_reason_t spc_emulate_modesense(struct se_cmd *cmd)
 {
 	struct se_device *dev = cmd->se_dev;
+	struct se_dev_entry *deve;
+	struct se_session *sess = cmd->se_sess;
 	char *cdb = cmd->t_task_cdb;
 	unsigned char buf[SE_MODE_PAGE_BUF], *rbuf;
 	int type = dev->transport->get_device_type(dev);
@@ -998,6 +1000,7 @@ static sense_reason_t spc_emulate_modesense(struct se_cmd *cmd)
 	int length = 0;
 	int ret;
 	int i;
+	bool read_only;
 
 	memset(buf, 0, SE_MODE_PAGE_BUF);
 
@@ -1007,10 +1010,13 @@ static sense_reason_t spc_emulate_modesense(struct se_cmd *cmd)
 	 */
 	length = ten ? 3 : 2;
 
+	rcu_read_lock();
+	deve = rcu_dereference(sess->se_node_acl->lun_entry_hlist[cmd->orig_fe_lun]);
+	read_only = (deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY);
+	rcu_read_unlock();
+
 	/* DEVICE-SPECIFIC PARAMETER */
-	if ((cmd->se_lun->lun_access & TRANSPORT_LUNFLAGS_READ_ONLY) ||
-	    (cmd->se_deve &&
-	     (cmd->se_deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY)))
+	if ((cmd->se_lun->lun_access & TRANSPORT_LUNFLAGS_READ_ONLY) || read_only)
 		spc_modesense_write_protect(&buf[length], type);
 
 	if ((spc_check_dev_wce(dev)) &&
@@ -1225,7 +1231,7 @@ sense_reason_t spc_emulate_report_luns(struct se_cmd *cmd)
 	struct se_dev_entry *deve;
 	struct se_session *sess = cmd->se_sess;
 	unsigned char *buf;
-	u32 lun_count = 0, offset = 8, i;
+	u32 lun_count = 0, offset = 8, i, mapped_lun;
 
 	if (cmd->data_length < 16) {
 		pr_warn("REPORT LUNS allocation length %u too small\n",
@@ -1248,11 +1254,15 @@ sense_reason_t spc_emulate_report_luns(struct se_cmd *cmd)
 		goto done;
 	}
 
-	spin_lock_irq(&sess->se_node_acl->device_list_lock);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
-		deve = sess->se_node_acl->device_list[i];
-		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS))
+		rcu_read_lock();
+		deve = rcu_dereference(sess->se_node_acl->lun_entry_hlist[i]);
+		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS)) {
+			rcu_read_unlock();
 			continue;
+		}
+		mapped_lun = deve->mapped_lun;
+		rcu_read_unlock();
 		/*
 		 * We determine the correct LUN LIST LENGTH even once we
 		 * have reached the initial allocation length.
@@ -1262,11 +1272,9 @@ sense_reason_t spc_emulate_report_luns(struct se_cmd *cmd)
 		if ((offset + 8) > cmd->data_length)
 			continue;
 
-		int_to_scsilun(deve->mapped_lun, (struct scsi_lun *)&buf[offset]);
+		int_to_scsilun(mapped_lun, (struct scsi_lun *)&buf[offset]);
 		offset += 8;
 	}
-	spin_unlock_irq(&sess->se_node_acl->device_list_lock);
-
 	/*
 	 * See SPC3 r07, page 159.
 	 */
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 06ecd7b..a826e6f 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -512,7 +512,6 @@ struct se_cmd {
 	struct list_head	se_delayed_node;
 	struct list_head	se_qf_node;
 	struct se_device      *se_dev;
-	struct se_dev_entry   *se_deve;
 	struct se_lun		*se_lun;
 	/* Only used for internal passthrough and legacy TCM fabric modules */
 	struct se_session	*se_sess;
-- 
1.9.1

[RFC 07/22] target/pscsi: Convert MODE_SENSE special case to RCU reader

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts the MODE_SENSE special case in pscsi_transport_complete()
to use RCU read path macros for se_node_acl->lun_entry_hlist[] access.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_pscsi.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/target/target_core_pscsi.c b/drivers/target/target_core_pscsi.c
index f6c954c..8bffcd8 100644
--- a/drivers/target/target_core_pscsi.c
+++ b/drivers/target/target_core_pscsi.c
@@ -634,12 +634,22 @@ static void pscsi_transport_complete(struct se_cmd *cmd, struct scatterlist *sg,
 	 * Hack to make sure that Write-Protect modepage is set if R/O mode is
 	 * forced.
 	 */
-	if (!cmd->se_deve || !cmd->data_length)
+	if (!cmd->data_length)
 		goto after_mode_sense;
 
 	if (((cdb[0] == MODE_SENSE) || (cdb[0] == MODE_SENSE_10)) &&
 	     (status_byte(result) << 1) == SAM_STAT_GOOD) {
-		if (cmd->se_deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY) {
+		struct se_session *sess = cmd->se_sess;
+		struct se_dev_entry *deve;
+		u32 lun = cmd->orig_fe_lun;
+		bool read_only;
+
+		rcu_read_lock();
+		deve = rcu_dereference(sess->se_node_acl->lun_entry_hlist[lun]);
+		read_only = (deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY);
+		rcu_read_unlock();
+
+		if (read_only) {
 			unsigned char *buf;
 
 			buf = transport_kmap_data_sg(cmd);
-- 
1.9.1

[RFC 08/22] target/pr: Convert se_dev_entry to percpu-refcount for RCU

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts se_dev_entry->pr_ref_count access to use modern
percpu reference counting.

It updates core_enable_device_list_for_node() to percpu_ref_init()
when se_dev_entry is being enabled.

It also updates core_disable_device_list_for_node() to percpu_ref_kill()
and now waits for outstanding PR references to drop to zero.

Also, go ahead and convert core_get_se_deve_from_rtpi code to RCU read
path for RELATIVE TARGET PORT IDENTIFIER lookup.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c | 54 ++++++++++++++++++---------
 drivers/target/target_core_pr.c     | 73 +++++++++++++++++++++++--------------
 drivers/target/target_core_tpg.c    |  1 -
 include/target/target_core_base.h   |  3 +-
 4 files changed, 85 insertions(+), 46 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 1053a2d..473543e 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -217,18 +217,23 @@ struct se_dev_entry *core_get_se_deve_from_rtpi(
 	struct se_portal_group *tpg = nacl->se_tpg;
 	u32 i;
 
-	spin_lock_irq(&nacl->device_list_lock);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
-		deve = nacl->device_list[i];
-
-		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS))
+		rcu_read_lock();
+		deve = rcu_dereference(nacl->lun_entry_hlist[i]);
+		if (!deve) {
+			rcu_read_unlock();
 			continue;
-
+		}
+		if (!(deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS)) {
+			rcu_read_unlock();
+			continue;
+		}
 		lun = deve->se_lun;
 		if (!lun) {
 			pr_err("%s device entries device pointer is"
 				" NULL, but Initiator has access.\n",
 				tpg->se_tpg_tfo->get_fabric_name());
+			rcu_read_unlock();
 			continue;
 		}
 		port = lun->lun_sep;
@@ -236,17 +241,18 @@ struct se_dev_entry *core_get_se_deve_from_rtpi(
 			pr_err("%s device entries device pointer is"
 				" NULL, but Initiator has access.\n",
 				tpg->se_tpg_tfo->get_fabric_name());
+			rcu_read_unlock();
 			continue;
 		}
-		if (port->sep_rtpi != rtpi)
+		if (port->sep_rtpi != rtpi) {
+			rcu_read_unlock();
 			continue;
-
-		atomic_inc_mb(&deve->pr_ref_count);
-		spin_unlock_irq(&nacl->device_list_lock);
+		}
+		percpu_ref_get(&deve->pr_ref);
+		rcu_read_unlock();
 
 		return deve;
 	}
-	spin_unlock_irq(&nacl->device_list_lock);
 
 	return NULL;
 }
@@ -312,6 +318,13 @@ void core_update_device_list_access(
 	synchronize_rcu();
 }
 
+static void target_pr_ref_release(struct percpu_ref *ref)
+{
+	struct se_dev_entry *deve = container_of(ref, struct se_dev_entry,
+						 pr_ref);
+	complete(&deve->pr_comp);
+}
+
 /*      core_enable_device_list_for_node():
  *
  *
@@ -352,6 +365,9 @@ int core_enable_device_list_for_node(
 		return 0;
 	}
 
+	percpu_ref_init(&deve->pr_ref, target_pr_ref_release, 0, GFP_KERNEL);
+	init_completion(&deve->pr_comp);
+
 	deve->mapped_lun = mapped_lun;
 	deve->lun_flags |= TRANSPORT_LUNFLAGS_INITIATOR_ACCESS;
 
@@ -412,13 +428,6 @@ int core_disable_device_list_for_node(
 	list_del(&deve->alua_port_list);
 	spin_unlock_bh(&port->sep_alua_lock);
 	/*
-	 * Wait for any in process SPEC_I_PT=1 or REGISTER_AND_MOVE
-	 * PR operation to complete.
-	 */
-	while (atomic_read(&deve->pr_ref_count) != 0)
-		cpu_relax();
-
-	/*
 	 * Disable struct se_dev_entry LUN ACL mapping
 	 */
 	spin_lock_irq(&nacl->lun_entry_lock);
@@ -431,7 +440,18 @@ int core_disable_device_list_for_node(
 	spin_unlock_irq(&nacl->lun_entry_lock);
 	rcu_read_unlock();
 
+	/*
+	 * Wait for RCU read critical sections to complete after
+	 * se_deve pointer reassignments.
+	 */
 	synchronize_rcu();
+	/*
+	 * Wait for any in process SPEC_I_PT=1 or REGISTER_AND_MOVE
+	 * PR operation to complete.
+	 */
+	percpu_ref_kill(&deve->pr_ref);
+	wait_for_completion(&deve->pr_comp);
+	percpu_ref_exit(&deve->pr_ref);
 
 	core_scsi3_free_pr_reg_from_nacl(lun->lun_se_dev, nacl);
 	return 0;
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index 2de6fb8..123e449 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -49,7 +49,7 @@ struct pr_transport_id_holder {
 	struct t10_pr_registration *dest_pr_reg;
 	struct se_portal_group *dest_tpg;
 	struct se_node_acl *dest_node_acl;
-	struct se_dev_entry *dest_se_deve;
+	struct se_dev_entry __rcu *dest_se_deve;
 	struct list_head dest_list;
 };
 
@@ -217,7 +217,7 @@ target_scsi2_reservation_release(struct se_cmd *cmd)
 	tpg = sess->se_tpg;
 	pr_debug("SCSI-2 Released reservation for %s LUN: %u ->"
 		" MAPPED LUN: %u for %s\n", tpg->se_tpg_tfo->get_fabric_name(),
-		cmd->se_lun->unpacked_lun, cmd->se_deve->mapped_lun,
+		cmd->se_lun->unpacked_lun, cmd->orig_fe_lun,
 		sess->se_node_acl->initiatorname);
 
 out_unlock:
@@ -266,7 +266,7 @@ target_scsi2_reservation_reserve(struct se_cmd *cmd)
 			dev->dev_reserved_node_acl->initiatorname);
 		pr_err("Current attempt - LUN: %u -> MAPPED LUN: %u"
 			" from %s \n", cmd->se_lun->unpacked_lun,
-			cmd->se_deve->mapped_lun,
+			cmd->orig_fe_lun,
 			sess->se_node_acl->initiatorname);
 		ret = TCM_RESERVATION_CONFLICT;
 		goto out_unlock;
@@ -280,7 +280,7 @@ target_scsi2_reservation_reserve(struct se_cmd *cmd)
 	}
 	pr_debug("SCSI-2 Reserved %s LUN: %u -> MAPPED LUN: %u"
 		" for %s\n", tpg->se_tpg_tfo->get_fabric_name(),
-		cmd->se_lun->unpacked_lun, cmd->se_deve->mapped_lun,
+		cmd->se_lun->unpacked_lun, cmd->orig_fe_lun,
 		sess->se_node_acl->initiatorname);
 
 out_unlock:
@@ -305,6 +305,7 @@ static int core_scsi3_pr_seq_non_holder(
 	unsigned char *cdb = cmd->t_task_cdb;
 	struct se_dev_entry *se_deve;
 	struct se_session *se_sess = cmd->se_sess;
+	struct se_node_acl *nacl = se_sess->se_node_acl;
 	int other_cdb = 0, ignore_reg;
 	int registered_nexus = 0, ret = 1; /* Conflict by default */
 	int all_reg = 0, reg_only = 0; /* ALL_REG, REG_ONLY */
@@ -312,7 +313,8 @@ static int core_scsi3_pr_seq_non_holder(
 	int legacy = 0; /* Act like a legacy device and return
 			 * RESERVATION CONFLICT on some CDBs */
 
-	se_deve = se_sess->se_node_acl->device_list[cmd->orig_fe_lun];
+	rcu_read_lock();
+	se_deve = rcu_dereference(nacl->lun_entry_hlist[cmd->orig_fe_lun]);
 	/*
 	 * Determine if the registration should be ignored due to
 	 * non-matching ISIDs in target_scsi3_pr_reservation_check().
@@ -353,8 +355,10 @@ static int core_scsi3_pr_seq_non_holder(
 			registered_nexus = 1;
 		break;
 	default:
+		rcu_read_unlock();
 		return -EINVAL;
 	}
+	rcu_read_unlock();
 	/*
 	 * Referenced from spc4r17 table 45 for *NON* PR holder access
 	 */
@@ -720,7 +724,7 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration(
 			if (strcmp(nacl->initiatorname, nacl_tmp->initiatorname))
 				continue;
 
-			atomic_inc_mb(&deve_tmp->pr_ref_count);
+			percpu_ref_get(&deve_tmp->pr_ref);
 			spin_unlock_bh(&port->sep_alua_lock);
 			/*
 			 * Grab a configfs group dependency that is released
@@ -733,7 +737,7 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration(
 				pr_err("core_scsi3_lunacl_depend"
 						"_item() failed\n");
 				atomic_dec_mb(&port->sep_tg_pt_ref_cnt);
-				atomic_dec_mb(&deve_tmp->pr_ref_count);
+				percpu_ref_put(&deve_tmp->pr_ref);
 				goto out;
 			}
 			/*
@@ -748,7 +752,6 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration(
 						sa_res_key, all_tg_pt, aptpl);
 			if (!pr_reg_atp) {
 				atomic_dec_mb(&port->sep_tg_pt_ref_cnt);
-				atomic_dec_mb(&deve_tmp->pr_ref_count);
 				core_scsi3_lunacl_undepend_item(deve_tmp);
 				goto out;
 			}
@@ -881,7 +884,7 @@ static int __core_scsi3_check_aptpl_registration(
 	struct se_lun *lun,
 	u32 target_lun,
 	struct se_node_acl *nacl,
-	struct se_dev_entry *deve)
+	u32 mapped_lun)
 {
 	struct t10_pr_registration *pr_reg, *pr_reg_tmp;
 	struct t10_reservation *pr_tmpl = &dev->t10_pr;
@@ -909,13 +912,12 @@ static int __core_scsi3_check_aptpl_registration(
 				pr_reg_aptpl_list) {
 
 		if (!strcmp(pr_reg->pr_iport, i_port) &&
-		     (pr_reg->pr_res_mapped_lun == deve->mapped_lun) &&
+		     (pr_reg->pr_res_mapped_lun == mapped_lun) &&
 		    !(strcmp(pr_reg->pr_tport, t_port)) &&
 		     (pr_reg->pr_reg_tpgt == tpgt) &&
 		     (pr_reg->pr_aptpl_target_lun == target_lun)) {
 
 			pr_reg->pr_reg_nacl = nacl;
-			pr_reg->pr_reg_deve = deve;
 			pr_reg->pr_reg_tg_pt_lun = lun;
 
 			list_del(&pr_reg->pr_reg_aptpl_list);
@@ -953,13 +955,14 @@ int core_scsi3_check_aptpl_registration(
 	struct se_node_acl *nacl,
 	u32 mapped_lun)
 {
-	struct se_dev_entry *deve = nacl->device_list[mapped_lun];
+	struct se_dev_entry *deve;
 
 	if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS)
 		return 0;
 
 	return __core_scsi3_check_aptpl_registration(dev, tpg, lun,
-				lun->unpacked_lun, nacl, deve);
+						     lun->unpacked_lun, nacl,
+						     mapped_lun);
 }
 
 static void __core_scsi3_dump_registration(
@@ -1407,14 +1410,21 @@ static int core_scsi3_lunacl_depend_item(struct se_dev_entry *se_deve)
 
 static void core_scsi3_lunacl_undepend_item(struct se_dev_entry *se_deve)
 {
-	struct se_lun_acl *lun_acl = se_deve->se_lun_acl;
+	struct se_lun_acl *lun_acl;
 	struct se_node_acl *nacl;
 	struct se_portal_group *tpg;
+
+	if (!se_deve) {
+		pr_err("core_scsi3_lunacl_undepend_item passed NULL se_deve\n");
+		dump_stack();
+		return;
+	}
 	/*
 	 * For nacl->dynamic_node_acl=1
 	 */
+	lun_acl = se_deve->se_lun_acl;
 	if (!lun_acl) {
-		atomic_dec_mb(&se_deve->pr_ref_count);
+		percpu_ref_put(&se_deve->pr_ref);
 		return;
 	}
 	nacl = lun_acl->se_lun_nacl;
@@ -1423,13 +1433,14 @@ static void core_scsi3_lunacl_undepend_item(struct se_dev_entry *se_deve)
 	configfs_undepend_item(tpg->se_tpg_tfo->tf_subsys,
 			&lun_acl->se_lun_group.cg_item);
 
-	atomic_dec_mb(&se_deve->pr_ref_count);
+	percpu_ref_put(&se_deve->pr_ref);
 }
 
 static sense_reason_t
 core_scsi3_decode_spec_i_port(
 	struct se_cmd *cmd,
 	struct se_portal_group *tpg,
+	struct se_dev_entry *local_se_deve,
 	unsigned char *l_isid,
 	u64 sa_res_key,
 	int all_tg_pt,
@@ -1440,7 +1451,7 @@ core_scsi3_decode_spec_i_port(
 	struct se_portal_group *dest_tpg = NULL, *tmp_tpg;
 	struct se_session *se_sess = cmd->se_sess;
 	struct se_node_acl *dest_node_acl = NULL;
-	struct se_dev_entry *dest_se_deve = NULL, *local_se_deve;
+	struct se_dev_entry __rcu *dest_se_deve = NULL;
 	struct t10_pr_registration *dest_pr_reg, *local_pr_reg, *pr_reg_e;
 	struct t10_pr_registration *pr_reg_tmp, *pr_reg_tmp_safe;
 	LIST_HEAD(tid_dest_list);
@@ -1454,7 +1465,6 @@ core_scsi3_decode_spec_i_port(
 	int dest_local_nexus;
 	u32 dest_rtpi = 0;
 
-	local_se_deve = se_sess->se_node_acl->device_list[cmd->orig_fe_lun];
 	/*
 	 * Allocate a struct pr_transport_id_holder and setup the
 	 * local_node_acl and local_se_deve pointers and add to
@@ -1469,7 +1479,6 @@ core_scsi3_decode_spec_i_port(
 	INIT_LIST_HEAD(&tidh_new->dest_list);
 	tidh_new->dest_tpg = tpg;
 	tidh_new->dest_node_acl = se_sess->se_node_acl;
-	tidh_new->dest_se_deve = local_se_deve;
 
 	local_pr_reg = __core_scsi3_alloc_registration(cmd->se_dev,
 				se_sess->se_node_acl, local_se_deve, l_isid,
@@ -1478,6 +1487,7 @@ core_scsi3_decode_spec_i_port(
 		kfree(tidh_new);
 		return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
 	}
+	rcu_assign_pointer(tidh_new->dest_se_deve, local_se_deve);
 	tidh_new->dest_pr_reg = local_pr_reg;
 	/*
 	 * The local I_T nexus does not hold any configfs dependances,
@@ -1644,7 +1654,7 @@ core_scsi3_decode_spec_i_port(
 		if (core_scsi3_lunacl_depend_item(dest_se_deve)) {
 			pr_err("core_scsi3_lunacl_depend_item()"
 					" failed\n");
-			atomic_dec_mb(&dest_se_deve->pr_ref_count);
+			percpu_ref_put(&dest_se_deve->pr_ref);
 			core_scsi3_nodeacl_undepend_item(dest_node_acl);
 			core_scsi3_tpg_undepend_item(dest_tpg);
 			ret = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
@@ -1999,6 +2009,7 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 		bool aptpl, bool all_tg_pt, bool spec_i_pt, enum register_type register_type)
 {
 	struct se_session *se_sess = cmd->se_sess;
+	struct se_node_acl *nacl = se_sess->se_node_acl;
 	struct se_device *dev = cmd->se_dev;
 	struct se_dev_entry *se_deve;
 	struct se_lun *se_lun = cmd->se_lun;
@@ -2014,7 +2025,9 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 		return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
 	}
 	se_tpg = se_sess->se_tpg;
-	se_deve = se_sess->se_node_acl->device_list[cmd->orig_fe_lun];
+
+	rcu_read_lock();
+	se_deve = rcu_dereference(nacl->lun_entry_hlist[cmd->orig_fe_lun]);
 
 	if (se_tpg->se_tpg_tfo->sess_get_initiator_sid) {
 		memset(&isid_buf[0], 0, PR_REG_ISID_LEN);
@@ -2030,14 +2043,16 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 		if (res_key) {
 			pr_warn("SPC-3 PR: Reservation Key non-zero"
 				" for SA REGISTER, returning CONFLICT\n");
+			rcu_read_unlock();
 			return TCM_RESERVATION_CONFLICT;
 		}
 		/*
 		 * Do nothing but return GOOD status.
 		 */
-		if (!sa_res_key)
+		if (!sa_res_key) {
+			rcu_read_unlock();
 			return 0;
-
+		}
 		if (!spec_i_pt) {
 			/*
 			 * Perform the Service Action REGISTER on the Initiator
@@ -2050,6 +2065,7 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 					register_type, 0)) {
 				pr_err("Unable to allocate"
 					" struct t10_pr_registration\n");
+				rcu_read_unlock();
 				return TCM_INVALID_PARAMETER_LIST;
 			}
 		} else {
@@ -2061,14 +2077,17 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 			 * logic from of core_scsi3_alloc_registration() for
 			 * each TransportID provided SCSI Initiator Port/Device
 			 */
-			ret = core_scsi3_decode_spec_i_port(cmd, se_tpg,
+			ret = core_scsi3_decode_spec_i_port(cmd, se_tpg, se_deve,
 					isid_ptr, sa_res_key, all_tg_pt, aptpl);
-			if (ret != 0)
+			if (ret != 0) {
+				rcu_read_unlock();
 				return ret;
+			}
 		}
-
+		rcu_read_unlock();
 		return core_scsi3_update_and_write_aptpl(dev, aptpl);
 	}
+	rcu_read_unlock();
 
 	/* ok, existing registration */
 
@@ -3345,7 +3364,7 @@ after_iport_check:
 
 	if (core_scsi3_lunacl_depend_item(dest_se_deve)) {
 		pr_err("core_scsi3_lunacl_depend_item() failed\n");
-		atomic_dec_mb(&dest_se_deve->pr_ref_count);
+		percpu_ref_put(&dest_se_deve->pr_ref);
 		dest_se_deve = NULL;
 		ret = TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
 		goto out;
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 15a683b..2813bd4 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -245,7 +245,6 @@ static int core_create_device_list_for_node(struct se_node_acl *nacl)
 		deve = nacl->lun_entry_hlist[i];
 
 		atomic_set(&deve->ua_count, 0);
-		atomic_set(&deve->pr_ref_count, 0);
 		spin_lock_init(&deve->ua_lock);
 		INIT_LIST_HEAD(&deve->alua_port_list);
 		INIT_LIST_HEAD(&deve->ua_list);
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index a826e6f..c4d182f 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -664,7 +664,8 @@ struct se_dev_entry {
 	u64			write_bytes;
 	atomic_t		ua_count;
 	/* Used for PR SPEC_I_PT=1 and REGISTER_AND_MOVE */
-	atomic_t		pr_ref_count;
+	struct percpu_ref	pr_ref;
+	struct completion	pr_comp;
 	struct se_lun_acl	*se_lun_acl;
 	spinlock_t		ua_lock;
 	struct se_lun		*se_lun;
-- 
1.9.1

[RFC 09/22] target/pr: Convert registration check to RCU pointer

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts core_scsi3_pr_seq_non_holder() check for non
reservation holding registrations to use se_deve->pr_reg as an RCU
protected pointer.

It also includes associated rcu_assign_pointer() + synchronize_rcu()
in __core_scsi3_add_registration() and __core_scsi3_free_registration()
for the RCU updater path.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c |  1 +
 drivers/target/target_core_pr.c     | 69 +++++++++++++++++++++++++++----------
 include/target/target_core_base.h   |  6 ++--
 3 files changed, 54 insertions(+), 22 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 473543e..670b1b5 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -432,6 +432,7 @@ int core_disable_device_list_for_node(
 	 */
 	spin_lock_irq(&nacl->lun_entry_lock);
 	core_scsi3_ua_release_all(deve);
+	rcu_assign_pointer(deve->pr_reg, NULL);
 	rcu_assign_pointer(deve->se_lun, NULL);
 	rcu_assign_pointer(deve->se_lun_acl, NULL);
 	deve->lun_flags = 0;
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index 123e449..e23fe68 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -312,9 +312,12 @@ static int core_scsi3_pr_seq_non_holder(
 	int we = 0; /* Write Exclusive */
 	int legacy = 0; /* Act like a legacy device and return
 			 * RESERVATION CONFLICT on some CDBs */
+	bool registered;
 
 	rcu_read_lock();
 	se_deve = rcu_dereference(nacl->lun_entry_hlist[cmd->orig_fe_lun]);
+	registered = (se_deve->pr_reg != NULL);
+	rcu_read_unlock();
 	/*
 	 * Determine if the registration should be ignored due to
 	 * non-matching ISIDs in target_scsi3_pr_reservation_check().
@@ -331,7 +334,7 @@ static int core_scsi3_pr_seq_non_holder(
 		 * Some commands are only allowed for the persistent reservation
 		 * holder.
 		 */
-		if ((se_deve->def_pr_registered) && !(ignore_reg))
+		if ((registered) && !(ignore_reg))
 			registered_nexus = 1;
 		break;
 	case PR_TYPE_WRITE_EXCLUSIVE_REGONLY:
@@ -341,7 +344,7 @@ static int core_scsi3_pr_seq_non_holder(
 		 * Some commands are only allowed for registered I_T Nexuses.
 		 */
 		reg_only = 1;
-		if ((se_deve->def_pr_registered) && !(ignore_reg))
+		if ((registered) && !(ignore_reg))
 			registered_nexus = 1;
 		break;
 	case PR_TYPE_WRITE_EXCLUSIVE_ALLREG:
@@ -351,14 +354,12 @@ static int core_scsi3_pr_seq_non_holder(
 		 * Each registered I_T Nexus is a reservation holder.
 		 */
 		all_reg = 1;
-		if ((se_deve->def_pr_registered) && !(ignore_reg))
+		if ((registered) && !(ignore_reg))
 			registered_nexus = 1;
 		break;
 	default:
-		rcu_read_unlock();
 		return -EINVAL;
 	}
-	rcu_read_unlock();
 	/*
 	 * Referenced from spc4r17 table 45 for *NON* PR holder access
 	 */
@@ -996,10 +997,6 @@ static void __core_scsi3_dump_registration(
 		pr_reg->pr_reg_aptpl);
 }
 
-/*
- * this function can be called with struct se_device->dev_reservation_lock
- * when register_move = 1
- */
 static void __core_scsi3_add_registration(
 	struct se_device *dev,
 	struct se_node_acl *nacl,
@@ -1010,6 +1007,7 @@ static void __core_scsi3_add_registration(
 	struct target_core_fabric_ops *tfo = nacl->se_tpg->se_tpg_tfo;
 	struct t10_pr_registration *pr_reg_tmp, *pr_reg_tmp_safe;
 	struct t10_reservation *pr_tmpl = &dev->t10_pr;
+	struct se_dev_entry *deve;
 
 	/*
 	 * Increment PRgeneration counter for struct se_device upon a successful
@@ -1026,10 +1024,20 @@ static void __core_scsi3_add_registration(
 
 	spin_lock(&pr_tmpl->registration_lock);
 	list_add_tail(&pr_reg->pr_reg_list, &pr_tmpl->registration_list);
-	pr_reg->pr_reg_deve->def_pr_registered = 1;
 
 	__core_scsi3_dump_registration(tfo, dev, nacl, pr_reg, register_type);
 	spin_unlock(&pr_tmpl->registration_lock);
+
+	spin_lock(&nacl->lun_entry_lock);
+	deve = nacl->lun_entry_hlist[pr_reg->pr_res_mapped_lun];
+	rcu_assign_pointer(deve->pr_reg, pr_reg);
+	spin_unlock(&nacl->lun_entry_lock);
+	/*
+	 * Wait for read path critical RCU in core_scsi3_pr_seq_non_holder()
+	 * conditional checks for deve->pr_reg pointer access complete.
+	 */
+	synchronize_rcu();
+
 	/*
 	 * Skip extra processing for ALL_TG_PT=0 or REGISTER_AND_MOVE.
 	 */
@@ -1041,6 +1049,8 @@ static void __core_scsi3_add_registration(
 	 */
 	list_for_each_entry_safe(pr_reg_tmp, pr_reg_tmp_safe,
 			&pr_reg->pr_reg_atp_list, pr_reg_atp_mem_list) {
+		struct se_node_acl *nacl_tmp = pr_reg_tmp->pr_reg_nacl;
+
 		list_del(&pr_reg_tmp->pr_reg_atp_mem_list);
 
 		pr_reg_tmp->pr_res_generation = core_scsi3_pr_generation(dev);
@@ -1048,12 +1058,21 @@ static void __core_scsi3_add_registration(
 		spin_lock(&pr_tmpl->registration_lock);
 		list_add_tail(&pr_reg_tmp->pr_reg_list,
 			      &pr_tmpl->registration_list);
-		pr_reg_tmp->pr_reg_deve->def_pr_registered = 1;
 
-		__core_scsi3_dump_registration(tfo, dev,
-				pr_reg_tmp->pr_reg_nacl, pr_reg_tmp,
-				register_type);
+		__core_scsi3_dump_registration(tfo, dev, nacl_tmp, pr_reg_tmp,
+					       register_type);
 		spin_unlock(&pr_tmpl->registration_lock);
+
+		spin_lock(&nacl->lun_entry_lock);
+		deve = nacl_tmp->lun_entry_hlist[pr_reg_tmp->pr_res_mapped_lun];
+		rcu_assign_pointer(deve->pr_reg, pr_reg_tmp);
+		spin_unlock(&nacl->lun_entry_lock);
+		/*
+		 * Wait for read path critical RCU in core_scsi3_pr_seq_non_holder()
+		 * conditional checks for deve->pr_reg pointer access complete.
+		 */
+		synchronize_rcu();
+
 		/*
 		 * Drop configfs group dependency reference from
 		 * __core_scsi3_alloc_registration()
@@ -1227,13 +1246,13 @@ static void __core_scsi3_free_registration(
 	struct target_core_fabric_ops *tfo =
 			pr_reg->pr_reg_nacl->se_tpg->se_tpg_tfo;
 	struct t10_reservation *pr_tmpl = &dev->t10_pr;
+	struct se_node_acl *nacl = pr_reg->pr_reg_nacl;
+	struct se_dev_entry *deve;
 	char i_buf[PR_REG_ISID_ID_LEN];
 
 	memset(i_buf, 0, PR_REG_ISID_ID_LEN);
 	core_pr_dump_initiator_port(pr_reg, i_buf, PR_REG_ISID_ID_LEN);
 
-	pr_reg->pr_reg_deve->def_pr_registered = 0;
-	pr_reg->pr_reg_deve->pr_res_key = 0;
 	if (!list_empty(&pr_reg->pr_reg_list))
 		list_del(&pr_reg->pr_reg_list);
 	/*
@@ -1242,6 +1261,8 @@ static void __core_scsi3_free_registration(
 	 */
 	if (dec_holders)
 		core_scsi3_put_pr_reg(pr_reg);
+
+	spin_unlock(&pr_tmpl->registration_lock);
 	/*
 	 * Wait until all reference from any other I_T nexuses for this
 	 * *pr_reg have been released.  Because list_del() is called above,
@@ -1249,13 +1270,22 @@ static void __core_scsi3_free_registration(
 	 * count back to zero, and we release *pr_reg.
 	 */
 	while (atomic_read(&pr_reg->pr_res_holders) != 0) {
-		spin_unlock(&pr_tmpl->registration_lock);
 		pr_debug("SPC-3 PR [%s] waiting for pr_res_holders\n",
 				tfo->get_fabric_name());
 		cpu_relax();
-		spin_lock(&pr_tmpl->registration_lock);
 	}
 
+	spin_lock(&nacl->lun_entry_lock);
+	deve = nacl->lun_entry_hlist[pr_reg->pr_res_mapped_lun];
+	rcu_assign_pointer(deve->pr_reg, NULL);
+	spin_unlock(&nacl->lun_entry_lock);
+	/*
+	 * Wait for read path critical RCU in core_scsi3_pr_seq_non_holder()
+	 * conditional checks for deve->pr_reg pointer access complete.
+	 */
+	synchronize_rcu();
+
+	spin_lock(&pr_tmpl->registration_lock);
 	pr_debug("SPC-3 PR [%s] Service Action: UNREGISTER Initiator"
 		" Node: %s%s\n", tfo->get_fabric_name(),
 		pr_reg->pr_reg_nacl->initiatorname,
@@ -3448,13 +3478,14 @@ after_iport_check:
 	dest_pr_reg = __core_scsi3_locate_pr_reg(dev, dest_node_acl,
 					iport_ptr);
 	if (!dest_pr_reg) {
+		spin_unlock(&dev->dev_reservation_lock);
 		if (core_scsi3_alloc_registration(cmd->se_dev,
 				dest_node_acl, dest_se_deve, iport_ptr,
 				sa_res_key, 0, aptpl, 2, 1)) {
-			spin_unlock(&dev->dev_reservation_lock);
 			ret = TCM_INVALID_PARAMETER_LIST;
 			goto out;
 		}
+		spin_lock(&dev->dev_reservation_lock);
 		dest_pr_reg = __core_scsi3_locate_pr_reg(dev, dest_node_acl,
 						iport_ptr);
 		new_reg = 1;
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index c4d182f..2566ce5 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -652,7 +652,6 @@ struct se_lun_acl {
 };
 
 struct se_dev_entry {
-	bool			def_pr_registered;
 	/* See transport_lunflags_table */
 	u32			lun_flags;
 	u32			mapped_lun;
@@ -666,9 +665,10 @@ struct se_dev_entry {
 	/* Used for PR SPEC_I_PT=1 and REGISTER_AND_MOVE */
 	struct percpu_ref	pr_ref;
 	struct completion	pr_comp;
-	struct se_lun_acl	*se_lun_acl;
+	struct se_lun_acl __rcu	*se_lun_acl;
 	spinlock_t		ua_lock;
-	struct se_lun		*se_lun;
+	struct se_lun __rcu	*se_lun;
+	struct t10_pr_registration __rcu *pr_reg;
 	struct list_head	alua_port_list;
 	struct list_head	ua_list;
 };
-- 
1.9.1

[RFC 10/22] target/pr: Change alloc_registration to avoid pr_reg_tg_pt_lun

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch changes __core_scsi3_do_alloc_registration() code to
drop pr_reg->pr_reg_tg_pt_lun pointer usage in favor of a new
pr_reg RPTI + existing target_lun.

It also includes changes to REGISTER, REGISTER_AND_MOVE and APTPL
feature bit codepaths.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_configfs.c |   6 +-
 drivers/target/target_core_pr.c       | 107 ++++++++++++++++------------------
 include/target/target_core_base.h     |   3 +-
 3 files changed, 53 insertions(+), 63 deletions(-)

diff --git a/drivers/target/target_core_configfs.c b/drivers/target/target_core_configfs.c
index 75d89ad..007c02c 100644
--- a/drivers/target/target_core_configfs.c
+++ b/drivers/target/target_core_configfs.c
@@ -942,7 +942,6 @@ static ssize_t target_core_dev_pr_show_attr_res_pr_holder_tg_port(
 		struct se_device *dev, char *page)
 {
 	struct se_node_acl *se_nacl;
-	struct se_lun *lun;
 	struct se_portal_group *se_tpg;
 	struct t10_pr_registration *pr_reg;
 	struct target_core_fabric_ops *tfo;
@@ -957,7 +956,6 @@ static ssize_t target_core_dev_pr_show_attr_res_pr_holder_tg_port(
 
 	se_nacl = pr_reg->pr_reg_nacl;
 	se_tpg = se_nacl->se_tpg;
-	lun = pr_reg->pr_reg_tg_pt_lun;
 	tfo = se_tpg->se_tpg_tfo;
 
 	len += sprintf(page+len, "SPC-3 Reservation: %s"
@@ -965,9 +963,9 @@ static ssize_t target_core_dev_pr_show_attr_res_pr_holder_tg_port(
 		tfo->tpg_get_wwn(se_tpg));
 	len += sprintf(page+len, "SPC-3 Reservation: Relative Port"
 		" Identifier Tag: %hu %s Portal Group Tag: %hu"
-		" %s Logical Unit: %u\n", lun->lun_sep->sep_rtpi,
+		" %s Logical Unit: %u\n", pr_reg->tg_pt_sep_rtpi,
 		tfo->get_fabric_name(), tfo->tpg_get_tag(se_tpg),
-		tfo->get_fabric_name(), lun->unpacked_lun);
+		tfo->get_fabric_name(), pr_reg->pr_aptpl_target_lun);
 
 out_unlock:
 	spin_unlock(&dev->dev_reservation_lock);
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index e23fe68..c1283c6 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -45,11 +45,10 @@
  * Used for Specify Initiator Ports Capable Bit (SPEC_I_PT)
  */
 struct pr_transport_id_holder {
-	int dest_local_nexus;
 	struct t10_pr_registration *dest_pr_reg;
 	struct se_portal_group *dest_tpg;
 	struct se_node_acl *dest_node_acl;
-	struct se_dev_entry __rcu *dest_se_deve;
+	struct se_dev_entry *dest_se_deve;
 	struct list_head dest_list;
 };
 
@@ -609,7 +608,9 @@ static u32 core_scsi3_pr_generation(struct se_device *dev)
 static struct t10_pr_registration *__core_scsi3_do_alloc_registration(
 	struct se_device *dev,
 	struct se_node_acl *nacl,
+	struct se_lun *lun,
 	struct se_dev_entry *deve,
+	u32 mapped_lun,
 	unsigned char *isid,
 	u64 sa_res_key,
 	int all_tg_pt,
@@ -631,12 +632,12 @@ static struct t10_pr_registration *__core_scsi3_do_alloc_registration(
 	atomic_set(&pr_reg->pr_res_holders, 0);
 	pr_reg->pr_reg_nacl = nacl;
 	pr_reg->pr_reg_deve = deve;
-	pr_reg->pr_res_mapped_lun = deve->mapped_lun;
-	pr_reg->pr_aptpl_target_lun = deve->se_lun->unpacked_lun;
+	pr_reg->pr_res_mapped_lun = mapped_lun;
+	pr_reg->pr_aptpl_target_lun = lun->unpacked_lun;
+	pr_reg->tg_pt_sep_rtpi = lun->lun_sep->sep_rtpi;
 	pr_reg->pr_res_key = sa_res_key;
 	pr_reg->pr_reg_all_tg_pt = all_tg_pt;
 	pr_reg->pr_reg_aptpl = aptpl;
-	pr_reg->pr_reg_tg_pt_lun = deve->se_lun;
 	/*
 	 * If an ISID value for this SCSI Initiator Port exists,
 	 * save it to the registration now.
@@ -660,7 +661,9 @@ static void core_scsi3_lunacl_undepend_item(struct se_dev_entry *);
 static struct t10_pr_registration *__core_scsi3_alloc_registration(
 	struct se_device *dev,
 	struct se_node_acl *nacl,
+	struct se_lun *lun,
 	struct se_dev_entry *deve,
+	u32 mapped_lun,
 	unsigned char *isid,
 	u64 sa_res_key,
 	int all_tg_pt,
@@ -676,8 +679,9 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration(
 	 * Create a registration for the I_T Nexus upon which the
 	 * PROUT REGISTER was received.
 	 */
-	pr_reg = __core_scsi3_do_alloc_registration(dev, nacl, deve, isid,
-			sa_res_key, all_tg_pt, aptpl);
+	pr_reg = __core_scsi3_do_alloc_registration(dev, nacl, lun, deve, mapped_lun,
+						    isid, sa_res_key, all_tg_pt,
+						    aptpl);
 	if (!pr_reg)
 		return NULL;
 	/*
@@ -749,8 +753,9 @@ static struct t10_pr_registration *__core_scsi3_alloc_registration(
 			 * __core_scsi3_add_registration()
 			 */
 			pr_reg_atp = __core_scsi3_do_alloc_registration(dev,
-						nacl_tmp, deve_tmp, NULL,
-						sa_res_key, all_tg_pt, aptpl);
+						nacl_tmp, deve_tmp->se_lun,
+						deve_tmp, deve_tmp->mapped_lun,
+						NULL, sa_res_key, all_tg_pt, aptpl);
 			if (!pr_reg_atp) {
 				atomic_dec_mb(&port->sep_tg_pt_ref_cnt);
 				core_scsi3_lunacl_undepend_item(deve_tmp);
@@ -819,7 +824,6 @@ int core_scsi3_alloc_aptpl_registration(
 	pr_reg->pr_res_key = sa_res_key;
 	pr_reg->pr_reg_all_tg_pt = all_tg_pt;
 	pr_reg->pr_reg_aptpl = 1;
-	pr_reg->pr_reg_tg_pt_lun = NULL;
 	pr_reg->pr_res_scope = 0; /* Always LUN_SCOPE */
 	pr_reg->pr_res_type = type;
 	/*
@@ -919,7 +923,7 @@ static int __core_scsi3_check_aptpl_registration(
 		     (pr_reg->pr_aptpl_target_lun == target_lun)) {
 
 			pr_reg->pr_reg_nacl = nacl;
-			pr_reg->pr_reg_tg_pt_lun = lun;
+			pr_reg->tg_pt_sep_rtpi = lun->lun_sep->sep_rtpi;
 
 			list_del(&pr_reg->pr_reg_aptpl_list);
 			spin_unlock(&pr_tmpl->aptpl_reg_lock);
@@ -956,8 +960,6 @@ int core_scsi3_check_aptpl_registration(
 	struct se_node_acl *nacl,
 	u32 mapped_lun)
 {
-	struct se_dev_entry *deve;
-
 	if (dev->dev_reservation_flags & DRF_SPC2_RESERVATIONS)
 		return 0;
 
@@ -1084,7 +1086,9 @@ static void __core_scsi3_add_registration(
 static int core_scsi3_alloc_registration(
 	struct se_device *dev,
 	struct se_node_acl *nacl,
+	struct se_lun *lun,
 	struct se_dev_entry *deve,
+	u32 mapped_lun,
 	unsigned char *isid,
 	u64 sa_res_key,
 	int all_tg_pt,
@@ -1094,8 +1098,9 @@ static int core_scsi3_alloc_registration(
 {
 	struct t10_pr_registration *pr_reg;
 
-	pr_reg = __core_scsi3_alloc_registration(dev, nacl, deve, isid,
-			sa_res_key, all_tg_pt, aptpl);
+	pr_reg = __core_scsi3_alloc_registration(dev, nacl, lun, deve, mapped_lun,
+						 isid, sa_res_key, all_tg_pt,
+						 aptpl);
 	if (!pr_reg)
 		return -EPERM;
 
@@ -1470,7 +1475,6 @@ static sense_reason_t
 core_scsi3_decode_spec_i_port(
 	struct se_cmd *cmd,
 	struct se_portal_group *tpg,
-	struct se_dev_entry *local_se_deve,
 	unsigned char *l_isid,
 	u64 sa_res_key,
 	int all_tg_pt,
@@ -1481,7 +1485,7 @@ core_scsi3_decode_spec_i_port(
 	struct se_portal_group *dest_tpg = NULL, *tmp_tpg;
 	struct se_session *se_sess = cmd->se_sess;
 	struct se_node_acl *dest_node_acl = NULL;
-	struct se_dev_entry __rcu *dest_se_deve = NULL;
+	struct se_dev_entry *dest_se_deve = NULL;
 	struct t10_pr_registration *dest_pr_reg, *local_pr_reg, *pr_reg_e;
 	struct t10_pr_registration *pr_reg_tmp, *pr_reg_tmp_safe;
 	LIST_HEAD(tid_dest_list);
@@ -1492,14 +1496,12 @@ core_scsi3_decode_spec_i_port(
 	char *iport_ptr = NULL, i_buf[PR_REG_ISID_ID_LEN];
 	sense_reason_t ret;
 	u32 tpdl, tid_len = 0;
-	int dest_local_nexus;
 	u32 dest_rtpi = 0;
 
 	/*
 	 * Allocate a struct pr_transport_id_holder and setup the
-	 * local_node_acl and local_se_deve pointers and add to
-	 * struct list_head tid_dest_list for add registration
-	 * processing in the loop of tid_dest_list below.
+	 * local_node_acl pointer and add to struct list_head tid_dest_list
+	 * for add registration processing in the loop of tid_dest_list below.
 	 */
 	tidh_new = kzalloc(sizeof(struct pr_transport_id_holder), GFP_KERNEL);
 	if (!tidh_new) {
@@ -1511,20 +1513,20 @@ core_scsi3_decode_spec_i_port(
 	tidh_new->dest_node_acl = se_sess->se_node_acl;
 
 	local_pr_reg = __core_scsi3_alloc_registration(cmd->se_dev,
-				se_sess->se_node_acl, local_se_deve, l_isid,
+				se_sess->se_node_acl, cmd->se_lun,
+				NULL, cmd->orig_fe_lun, l_isid,
 				sa_res_key, all_tg_pt, aptpl);
 	if (!local_pr_reg) {
 		kfree(tidh_new);
 		return TCM_LOGICAL_UNIT_COMMUNICATION_FAILURE;
 	}
-	rcu_assign_pointer(tidh_new->dest_se_deve, local_se_deve);
 	tidh_new->dest_pr_reg = local_pr_reg;
 	/*
 	 * The local I_T nexus does not hold any configfs dependances,
-	 * so we set tid_h->dest_local_nexus=1 to prevent the
+	 * so we set tidh_new->dest_se_deve to NULL to prevent the
 	 * configfs_undepend_item() calls in the tid_dest_list loops below.
 	 */
-	tidh_new->dest_local_nexus = 1;
+	tidh_new->dest_se_deve = NULL;
 	list_add_tail(&tidh_new->dest_list, &tid_dest_list);
 
 	if (cmd->data_length < 28) {
@@ -1749,8 +1751,9 @@ core_scsi3_decode_spec_i_port(
 		 * 2nd loop which will never fail.
 		 */
 		dest_pr_reg = __core_scsi3_alloc_registration(cmd->se_dev,
-				dest_node_acl, dest_se_deve, iport_ptr,
-				sa_res_key, all_tg_pt, aptpl);
+					dest_node_acl, dest_se_deve->se_lun,
+					dest_se_deve, dest_se_deve->mapped_lun,
+					iport_ptr, sa_res_key, all_tg_pt, aptpl);
 		if (!dest_pr_reg) {
 			core_scsi3_lunacl_undepend_item(dest_se_deve);
 			core_scsi3_nodeacl_undepend_item(dest_node_acl);
@@ -1788,7 +1791,6 @@ core_scsi3_decode_spec_i_port(
 		dest_node_acl = tidh->dest_node_acl;
 		dest_se_deve = tidh->dest_se_deve;
 		dest_pr_reg = tidh->dest_pr_reg;
-		dest_local_nexus = tidh->dest_local_nexus;
 
 		list_del(&tidh->dest_list);
 		kfree(tidh);
@@ -1802,9 +1804,10 @@ core_scsi3_decode_spec_i_port(
 		pr_debug("SPC-3 PR [%s] SPEC_I_PT: Successfully"
 			" registered Transport ID for Node: %s%s Mapped LUN:"
 			" %u\n", dest_tpg->se_tpg_tfo->get_fabric_name(),
-			dest_node_acl->initiatorname, i_buf, dest_se_deve->mapped_lun);
+			dest_node_acl->initiatorname, i_buf, (dest_se_deve) ?
+			dest_se_deve->mapped_lun : 0);
 
-		if (dest_local_nexus)
+		if (!dest_se_deve)
 			continue;
 
 		core_scsi3_lunacl_undepend_item(dest_se_deve);
@@ -1825,7 +1828,6 @@ out:
 		dest_node_acl = tidh->dest_node_acl;
 		dest_se_deve = tidh->dest_se_deve;
 		dest_pr_reg = tidh->dest_pr_reg;
-		dest_local_nexus = tidh->dest_local_nexus;
 
 		list_del(&tidh->dest_list);
 		kfree(tidh);
@@ -1843,7 +1845,7 @@ out:
 
 		kmem_cache_free(t10_pr_reg_cache, dest_pr_reg);
 
-		if (dest_local_nexus)
+		if (!dest_se_deve)
 			continue;
 
 		core_scsi3_lunacl_undepend_item(dest_se_deve);
@@ -1858,7 +1860,6 @@ static int core_scsi3_update_aptpl_buf(
 	unsigned char *buf,
 	u32 pr_aptpl_buf_len)
 {
-	struct se_lun *lun;
 	struct se_portal_group *tpg;
 	struct t10_pr_registration *pr_reg;
 	unsigned char tmp[512], isid_buf[32];
@@ -1877,7 +1878,6 @@ static int core_scsi3_update_aptpl_buf(
 		tmp[0] = '\0';
 		isid_buf[0] = '\0';
 		tpg = pr_reg->pr_reg_nacl->se_tpg;
-		lun = pr_reg->pr_reg_tg_pt_lun;
 		/*
 		 * Write out any ISID value to APTPL metadata that was included
 		 * in the original registration.
@@ -1929,7 +1929,8 @@ static int core_scsi3_update_aptpl_buf(
 			" %d\n", tpg->se_tpg_tfo->get_fabric_name(),
 			tpg->se_tpg_tfo->tpg_get_wwn(tpg),
 			tpg->se_tpg_tfo->tpg_get_tag(tpg),
-			lun->lun_sep->sep_rtpi, lun->unpacked_lun, reg_count);
+			pr_reg->tg_pt_sep_rtpi, pr_reg->pr_aptpl_target_lun,
+			reg_count);
 
 		if ((len + strlen(tmp) >= pr_aptpl_buf_len)) {
 			pr_err("Unable to update renaming APTPL metadata,"
@@ -2039,9 +2040,7 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 		bool aptpl, bool all_tg_pt, bool spec_i_pt, enum register_type register_type)
 {
 	struct se_session *se_sess = cmd->se_sess;
-	struct se_node_acl *nacl = se_sess->se_node_acl;
 	struct se_device *dev = cmd->se_dev;
-	struct se_dev_entry *se_deve;
 	struct se_lun *se_lun = cmd->se_lun;
 	struct se_portal_group *se_tpg;
 	struct t10_pr_registration *pr_reg, *pr_reg_p, *pr_reg_tmp;
@@ -2056,9 +2055,6 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 	}
 	se_tpg = se_sess->se_tpg;
 
-	rcu_read_lock();
-	se_deve = rcu_dereference(nacl->lun_entry_hlist[cmd->orig_fe_lun]);
-
 	if (se_tpg->se_tpg_tfo->sess_get_initiator_sid) {
 		memset(&isid_buf[0], 0, PR_REG_ISID_LEN);
 		se_tpg->se_tpg_tfo->sess_get_initiator_sid(se_sess, &isid_buf[0],
@@ -2073,16 +2069,14 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 		if (res_key) {
 			pr_warn("SPC-3 PR: Reservation Key non-zero"
 				" for SA REGISTER, returning CONFLICT\n");
-			rcu_read_unlock();
 			return TCM_RESERVATION_CONFLICT;
 		}
 		/*
 		 * Do nothing but return GOOD status.
 		 */
-		if (!sa_res_key) {
-			rcu_read_unlock();
+		if (!sa_res_key)
 			return 0;
-		}
+
 		if (!spec_i_pt) {
 			/*
 			 * Perform the Service Action REGISTER on the Initiator
@@ -2090,12 +2084,12 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 			 * Logical Unit of the SCSI device server.
 			 */
 			if (core_scsi3_alloc_registration(cmd->se_dev,
-					se_sess->se_node_acl, se_deve, isid_ptr,
+					se_sess->se_node_acl, cmd->se_lun,
+					NULL, cmd->orig_fe_lun, isid_ptr,
 					sa_res_key, all_tg_pt, aptpl,
 					register_type, 0)) {
 				pr_err("Unable to allocate"
 					" struct t10_pr_registration\n");
-				rcu_read_unlock();
 				return TCM_INVALID_PARAMETER_LIST;
 			}
 		} else {
@@ -2107,17 +2101,13 @@ core_scsi3_emulate_pro_register(struct se_cmd *cmd, u64 res_key, u64 sa_res_key,
 			 * logic from of core_scsi3_alloc_registration() for
 			 * each TransportID provided SCSI Initiator Port/Device
 			 */
-			ret = core_scsi3_decode_spec_i_port(cmd, se_tpg, se_deve,
+			ret = core_scsi3_decode_spec_i_port(cmd, se_tpg,
 					isid_ptr, sa_res_key, all_tg_pt, aptpl);
-			if (ret != 0) {
-				rcu_read_unlock();
+			if (ret != 0)
 				return ret;
-			}
 		}
-		rcu_read_unlock();
 		return core_scsi3_update_and_write_aptpl(dev, aptpl);
 	}
-	rcu_read_unlock();
 
 	/* ok, existing registration */
 
@@ -3479,9 +3469,10 @@ after_iport_check:
 					iport_ptr);
 	if (!dest_pr_reg) {
 		spin_unlock(&dev->dev_reservation_lock);
-		if (core_scsi3_alloc_registration(cmd->se_dev,
-				dest_node_acl, dest_se_deve, iport_ptr,
-				sa_res_key, 0, aptpl, 2, 1)) {
+		if (core_scsi3_alloc_registration(cmd->se_dev, dest_node_acl,
+					dest_se_deve->se_lun, dest_se_deve,
+					dest_se_deve->mapped_lun, iport_ptr,
+					sa_res_key, 0, aptpl, 2, 1)) {
 			ret = TCM_INVALID_PARAMETER_LIST;
 			goto out;
 		}
@@ -4048,10 +4039,10 @@ core_scsi3_pri_read_full_status(struct se_cmd *cmd)
 		 * IDENTIFIER field are not defined by this standard.
 		 */
 		if (!pr_reg->pr_reg_all_tg_pt) {
-			struct se_port *port = pr_reg->pr_reg_tg_pt_lun->lun_sep;
+			u16 sep_rtpi = pr_reg->tg_pt_sep_rtpi;
 
-			buf[off++] = ((port->sep_rtpi >> 8) & 0xff);
-			buf[off++] = (port->sep_rtpi & 0xff);
+			buf[off++] = ((sep_rtpi >> 8) & 0xff);
+			buf[off++] = (sep_rtpi & 0xff);
 		} else
 			off += 2; /* Skip over RELATIVE TARGET PORT IDENTIFIER */
 
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 2566ce5..040138a 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -387,13 +387,14 @@ struct t10_pr_registration {
 	bool isid_present_at_reg;
 	u32 pr_res_mapped_lun;
 	u32 pr_aptpl_target_lun;
+	u16 tg_pt_sep_rtpi;
 	u32 pr_res_generation;
 	u64 pr_reg_bin_isid;
 	u64 pr_res_key;
 	atomic_t pr_res_holders;
 	struct se_node_acl *pr_reg_nacl;
+	/* Used by ALL_TG_PT=1 registration with deve->pr_ref taken */
 	struct se_dev_entry *pr_reg_deve;
-	struct se_lun *pr_reg_tg_pt_lun;
 	struct list_head pr_reg_list;
 	struct list_head pr_reg_abort_list;
 	struct list_head pr_reg_aptpl_list;
-- 
1.9.1

[RFC 11/22] target: Convert UNIT_ATTENTION logic to RCU reader

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts target_core_ua.c code to use RCU read path macros
for se_node_acl->lun_entry_hlist[] access.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_ua.c | 35 +++++++++++++++++++----------------
 1 file changed, 19 insertions(+), 16 deletions(-)

diff --git a/drivers/target/target_core_ua.c b/drivers/target/target_core_ua.c
index 1738b16..975dbba 100644
--- a/drivers/target/target_core_ua.c
+++ b/drivers/target/target_core_ua.c
@@ -51,9 +51,13 @@ target_scsi3_ua_check(struct se_cmd *cmd)
 	if (!nacl)
 		return 0;
 
-	deve = nacl->device_list[cmd->orig_fe_lun];
-	if (!atomic_read(&deve->ua_count))
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[cmd->orig_fe_lun]);
+	if (!atomic_read(&deve->ua_count)) {
+		rcu_read_unlock();
 		return 0;
+	}
+	rcu_read_unlock();
 	/*
 	 * From sam4r14, section 5.14 Unit attention condition:
 	 *
@@ -104,9 +108,8 @@ int core_scsi3_ua_allocate(
 	ua->ua_asc = asc;
 	ua->ua_ascq = ascq;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[unpacked_lun];
-
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[unpacked_lun]);
 	spin_lock(&deve->ua_lock);
 	list_for_each_entry_safe(ua_p, ua_tmp, &deve->ua_list, ua_nacl_list) {
 		/*
@@ -114,7 +117,7 @@ int core_scsi3_ua_allocate(
 		 */
 		if ((ua_p->ua_asc == asc) && (ua_p->ua_ascq == ascq)) {
 			spin_unlock(&deve->ua_lock);
-			spin_unlock_irq(&nacl->device_list_lock);
+			rcu_read_unlock();
 			kmem_cache_free(se_ua_cache, ua);
 			return 0;
 		}
@@ -159,14 +162,13 @@ int core_scsi3_ua_allocate(
 			list_add_tail(&ua->ua_nacl_list,
 				&deve->ua_list);
 		spin_unlock(&deve->ua_lock);
-		spin_unlock_irq(&nacl->device_list_lock);
 
 		atomic_inc_mb(&deve->ua_count);
+		rcu_read_unlock();
 		return 0;
 	}
 	list_add_tail(&ua->ua_nacl_list, &deve->ua_list);
 	spin_unlock(&deve->ua_lock);
-	spin_unlock_irq(&nacl->device_list_lock);
 
 	pr_debug("[%s]: Allocated UNIT ATTENTION, mapped LUN: %u, ASC:"
 		" 0x%02x, ASCQ: 0x%02x\n",
@@ -174,6 +176,7 @@ int core_scsi3_ua_allocate(
 		asc, ascq);
 
 	atomic_inc_mb(&deve->ua_count);
+	rcu_read_unlock();
 	return 0;
 }
 
@@ -211,10 +214,10 @@ void core_scsi3_ua_for_check_condition(
 	if (!nacl)
 		return;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[cmd->orig_fe_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[cmd->orig_fe_lun]);
 	if (!atomic_read(&deve->ua_count)) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return;
 	}
 	/*
@@ -250,7 +253,7 @@ void core_scsi3_ua_for_check_condition(
 		atomic_dec_mb(&deve->ua_count);
 	}
 	spin_unlock(&deve->ua_lock);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 
 	pr_debug("[%s]: %s UNIT ATTENTION condition with"
 		" INTLCK_CTRL: %d, mapped LUN: %u, got CDB: 0x%02x"
@@ -279,10 +282,10 @@ int core_scsi3_ua_clear_for_request_sense(
 	if (!nacl)
 		return -EINVAL;
 
-	spin_lock_irq(&nacl->device_list_lock);
-	deve = nacl->device_list[cmd->orig_fe_lun];
+	rcu_read_lock();
+	deve = rcu_dereference(nacl->lun_entry_hlist[cmd->orig_fe_lun]);
 	if (!atomic_read(&deve->ua_count)) {
-		spin_unlock_irq(&nacl->device_list_lock);
+		rcu_read_unlock();
 		return -EPERM;
 	}
 	/*
@@ -308,7 +311,7 @@ int core_scsi3_ua_clear_for_request_sense(
 		atomic_dec_mb(&deve->ua_count);
 	}
 	spin_unlock(&deve->ua_lock);
-	spin_unlock_irq(&nacl->device_list_lock);
+	rcu_read_unlock();
 
 	pr_debug("[%s]: Released UNIT ATTENTION condition, mapped"
 		" LUN: %u, got REQUEST_SENSE reported ASC: 0x%02x,"
-- 
1.9.1

[RFC 12/22] target: Convert se_tpg->tpg_lun_lock to ->tpg_lun_mutex

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

Now that se_node_acl->lun_entry_hlist[] is using RCU, convert existing
se_tpg->tpg_lun_lock to struct mutex so core_tpg_add_node_to_devs()
can perform RCU updater logic without releasing ->tpg_lun_mutex.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/sbp/sbp_target.c     | 16 ++++++----------
 drivers/target/target_core_device.c | 18 ++++++++----------
 drivers/target/target_core_tpg.c    | 25 +++++++++++--------------
 include/target/target_core_base.h   |  2 +-
 4 files changed, 26 insertions(+), 35 deletions(-)

diff --git a/drivers/target/sbp/sbp_target.c b/drivers/target/sbp/sbp_target.c
index 9512af6..ec46968 100644
--- a/drivers/target/sbp/sbp_target.c
+++ b/drivers/target/sbp/sbp_target.c
@@ -184,13 +184,13 @@ static struct se_lun *sbp_get_lun_from_tpg(struct sbp_tpg *tpg, int lun)
 	if (lun >= TRANSPORT_MAX_LUNS_PER_TPG)
 		return ERR_PTR(-EINVAL);
 
-	spin_lock(&se_tpg->tpg_lun_lock);
+	mutex_lock(&se_tpg->tpg_lun_mutex);
 	se_lun = se_tpg->tpg_lun_list[lun];
 
 	if (se_lun->lun_status != TRANSPORT_LUN_STATUS_ACTIVE)
 		se_lun = ERR_PTR(-ENODEV);
 
-	spin_unlock(&se_tpg->tpg_lun_lock);
+	mutex_unlock(&se_tpg->tpg_lun_mutex);
 
 	return se_lun;
 }
@@ -1943,7 +1943,7 @@ static int sbp_count_se_tpg_luns(struct se_portal_group *tpg)
 {
 	int i, count = 0;
 
-	spin_lock(&tpg->tpg_lun_lock);
+	mutex_lock(&tpg->tpg_lun_mutex);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
 		struct se_lun *se_lun = tpg->tpg_lun_list[i];
 
@@ -1952,7 +1952,7 @@ static int sbp_count_se_tpg_luns(struct se_portal_group *tpg)
 
 		count++;
 	}
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 
 	return count;
 }
@@ -2021,7 +2021,7 @@ static int sbp_update_unit_directory(struct sbp_tport *tport)
 	/* unit unique ID (leaf is just after LUNs) */
 	data[idx++] = 0x8d000000 | (num_luns + 1);
 
-	spin_lock(&tport->tpg->se_tpg.tpg_lun_lock);
+	mutex_lock(&tport->tpg->se_tpg.tpg_lun_mutex);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
 		struct se_lun *se_lun = tport->tpg->se_tpg.tpg_lun_list[i];
 		struct se_device *dev;
@@ -2030,8 +2030,6 @@ static int sbp_update_unit_directory(struct sbp_tport *tport)
 		if (se_lun->lun_status == TRANSPORT_LUN_STATUS_FREE)
 			continue;
 
-		spin_unlock(&tport->tpg->se_tpg.tpg_lun_lock);
-
 		dev = se_lun->lun_se_dev;
 		type = dev->transport->get_device_type(dev);
 
@@ -2039,10 +2037,8 @@ static int sbp_update_unit_directory(struct sbp_tport *tport)
 		data[idx++] = 0x14000000 |
 			((type << 16) & 0x1f0000) |
 			(se_lun->unpacked_lun & 0xffff);
-
-		spin_lock(&tport->tpg->se_tpg.tpg_lun_lock);
 	}
-	spin_unlock(&tport->tpg->se_tpg.tpg_lun_lock);
+	mutex_unlock(&tport->tpg->se_tpg.tpg_lun_mutex);
 
 	/* unit unique ID leaf */
 	data[idx++] = 2 << 16;
diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 670b1b5..014c1b6 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -1267,27 +1267,26 @@ struct se_lun *core_get_lun_from_tpg(struct se_portal_group *tpg, u32 unpacked_l
 {
 	struct se_lun *lun;
 
-	spin_lock(&tpg->tpg_lun_lock);
 	if (unpacked_lun > (TRANSPORT_MAX_LUNS_PER_TPG-1)) {
 		pr_err("%s LUN: %u exceeds TRANSPORT_MAX_LUNS"
 			"_PER_TPG-1: %u for Target Portal Group: %hu\n",
 			tpg->se_tpg_tfo->get_fabric_name(), unpacked_lun,
 			TRANSPORT_MAX_LUNS_PER_TPG-1,
 			tpg->se_tpg_tfo->tpg_get_tag(tpg));
-		spin_unlock(&tpg->tpg_lun_lock);
 		return NULL;
 	}
-	lun = tpg->tpg_lun_list[unpacked_lun];
 
+	mutex_lock(&tpg->tpg_lun_mutex);
+	lun = tpg->tpg_lun_list[unpacked_lun];
 	if (lun->lun_status != TRANSPORT_LUN_STATUS_FREE) {
 		pr_err("%s Logical Unit Number: %u is not free on"
 			" Target Portal Group: %hu, ignoring request.\n",
 			tpg->se_tpg_tfo->get_fabric_name(), unpacked_lun,
 			tpg->se_tpg_tfo->tpg_get_tag(tpg));
-		spin_unlock(&tpg->tpg_lun_lock);
+		mutex_unlock(&tpg->tpg_lun_mutex);
 		return NULL;
 	}
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 
 	return lun;
 }
@@ -1300,27 +1299,26 @@ static struct se_lun *core_dev_get_lun(struct se_portal_group *tpg, u32 unpacked
 {
 	struct se_lun *lun;
 
-	spin_lock(&tpg->tpg_lun_lock);
 	if (unpacked_lun > (TRANSPORT_MAX_LUNS_PER_TPG-1)) {
 		pr_err("%s LUN: %u exceeds TRANSPORT_MAX_LUNS_PER"
 			"_TPG-1: %u for Target Portal Group: %hu\n",
 			tpg->se_tpg_tfo->get_fabric_name(), unpacked_lun,
 			TRANSPORT_MAX_LUNS_PER_TPG-1,
 			tpg->se_tpg_tfo->tpg_get_tag(tpg));
-		spin_unlock(&tpg->tpg_lun_lock);
 		return NULL;
 	}
-	lun = tpg->tpg_lun_list[unpacked_lun];
 
+	mutex_lock(&tpg->tpg_lun_mutex);
+	lun = tpg->tpg_lun_list[unpacked_lun];
 	if (lun->lun_status != TRANSPORT_LUN_STATUS_ACTIVE) {
 		pr_err("%s Logical Unit Number: %u is not active on"
 			" Target Portal Group: %hu, ignoring request.\n",
 			tpg->se_tpg_tfo->get_fabric_name(), unpacked_lun,
 			tpg->se_tpg_tfo->tpg_get_tag(tpg));
-		spin_unlock(&tpg->tpg_lun_lock);
+		mutex_unlock(&tpg->tpg_lun_mutex);
 		return NULL;
 	}
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 
 	return lun;
 }
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 2813bd4..1a12532 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -132,14 +132,12 @@ void core_tpg_add_node_to_devs(
 	struct se_lun *lun;
 	struct se_device *dev;
 
-	spin_lock(&tpg->tpg_lun_lock);
+	mutex_lock(&tpg->tpg_lun_mutex);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
 		lun = tpg->tpg_lun_list[i];
 		if (lun->lun_status != TRANSPORT_LUN_STATUS_ACTIVE)
 			continue;
 
-		spin_unlock(&tpg->tpg_lun_lock);
-
 		dev = lun->lun_se_dev;
 		/*
 		 * By default in LIO-Target $FABRIC_MOD,
@@ -166,7 +164,7 @@ void core_tpg_add_node_to_devs(
 			"READ-WRITE" : "READ-ONLY");
 
 		core_enable_device_list_for_node(lun, NULL, lun->unpacked_lun,
-				lun_access, acl, tpg);
+						 lun_access, acl, tpg);
 		/*
 		 * Check to see if there are any existing persistent reservation
 		 * APTPL pre-registrations that need to be enabled for this dynamic
@@ -174,9 +172,8 @@ void core_tpg_add_node_to_devs(
 		 */
 		core_scsi3_check_aptpl_registration(dev, tpg, lun, acl,
 						    lun->unpacked_lun);
-		spin_lock(&tpg->tpg_lun_lock);
 	}
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 }
 
 /*      core_set_queue_depth_for_node():
@@ -713,7 +710,7 @@ int core_tpg_register(
 	INIT_LIST_HEAD(&se_tpg->tpg_sess_list);
 	spin_lock_init(&se_tpg->acl_node_lock);
 	spin_lock_init(&se_tpg->session_lock);
-	spin_lock_init(&se_tpg->tpg_lun_lock);
+	mutex_init(&se_tpg->tpg_lun_mutex);
 
 	if (se_tpg->se_tpg_type == TRANSPORT_TPG_TYPE_NORMAL) {
 		if (core_tpg_setup_virtual_lun0(se_tpg) < 0) {
@@ -798,17 +795,17 @@ struct se_lun *core_tpg_alloc_lun(
 		return ERR_PTR(-EOVERFLOW);
 	}
 
-	spin_lock(&tpg->tpg_lun_lock);
+	mutex_lock(&tpg->tpg_lun_mutex);
 	lun = tpg->tpg_lun_list[unpacked_lun];
 	if (lun->lun_status == TRANSPORT_LUN_STATUS_ACTIVE) {
 		pr_err("TPG Logical Unit Number: %u is already active"
 			" on %s Target Portal Group: %u, ignoring request.\n",
 			unpacked_lun, tpg->se_tpg_tfo->get_fabric_name(),
 			tpg->se_tpg_tfo->tpg_get_tag(tpg));
-		spin_unlock(&tpg->tpg_lun_lock);
+		mutex_unlock(&tpg->tpg_lun_mutex);
 		return ERR_PTR(-EINVAL);
 	}
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 
 	return lun;
 }
@@ -832,10 +829,10 @@ int core_tpg_add_lun(
 		return ret;
 	}
 
-	spin_lock(&tpg->tpg_lun_lock);
+	mutex_lock(&tpg->tpg_lun_mutex);
 	lun->lun_access = lun_access;
 	lun->lun_status = TRANSPORT_LUN_STATUS_ACTIVE;
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 
 	return 0;
 }
@@ -849,9 +846,9 @@ void core_tpg_remove_lun(
 
 	core_dev_unexport(lun->lun_se_dev, tpg, lun);
 
-	spin_lock(&tpg->tpg_lun_lock);
+	mutex_lock(&tpg->tpg_lun_mutex);
 	lun->lun_status = TRANSPORT_LUN_STATUS_FREE;
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 
 	percpu_ref_exit(&lun->lun_ref);
 }
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 040138a..5a7a8fa 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -883,7 +883,7 @@ struct se_portal_group {
 	spinlock_t		acl_node_lock;
 	/* Spinlock for adding/removing sessions */
 	spinlock_t		session_lock;
-	spinlock_t		tpg_lun_lock;
+	struct mutex		tpg_lun_mutex;
 	/* Pointer to $FABRIC_MOD portal group */
 	void			*se_tpg_fabric_ptr;
 	struct list_head	se_tpg_node;
-- 
1.9.1

[RFC 13/22] target: Convert se_tpg->acl_node_lock to ->acl_node_mutex

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts se_tpg->acl_node_lock to struct mutex, so that
->acl_node_acl walkers in core_clear_lun_from_tpg() can block when
calling core_disable_device_list_for_node().

It also updates core_dev_add_lun() to hold ->acl_node_mutex when
calling core_tpg_add_node_to_devs() to build ->lun_entry_hlist[]
for dynamically generated se_node_acl.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c    | 14 ++++-----
 drivers/target/target_core_pr.c        |  8 ++---
 drivers/target/target_core_tpg.c       | 55 +++++++++++++++++-----------------
 drivers/target/target_core_transport.c | 20 ++++++-------
 drivers/target/tcm_fc/tfc_conf.c       |  4 +--
 include/target/target_core_base.h      |  2 +-
 6 files changed, 50 insertions(+), 53 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 014c1b6..965d81e 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -468,9 +468,8 @@ void core_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg)
 	struct se_dev_entry *deve;
 	u32 i, mapped_lun;
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	list_for_each_entry(nacl, &tpg->acl_node_list, acl_list) {
-		spin_unlock_irq(&tpg->acl_node_lock);
 
 		for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
 			rcu_read_lock();
@@ -485,10 +484,8 @@ void core_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg)
 			core_disable_device_list_for_node(lun, NULL, mapped_lun,
 					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 		}
-
-		spin_lock_irq(&tpg->acl_node_lock);
 	}
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 }
 
 static struct se_port *core_alloc_port(struct se_device *dev)
@@ -1231,17 +1228,16 @@ struct se_lun *core_dev_add_lun(
 	 */
 	if (tpg->se_tpg_tfo->tpg_check_demo_mode(tpg)) {
 		struct se_node_acl *acl;
-		spin_lock_irq(&tpg->acl_node_lock);
+
+		mutex_lock(&tpg->acl_node_mutex);
 		list_for_each_entry(acl, &tpg->acl_node_list, acl_list) {
 			if (acl->dynamic_node_acl &&
 			    (!tpg->se_tpg_tfo->tpg_check_demo_mode_login_only ||
 			     !tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg))) {
-				spin_unlock_irq(&tpg->acl_node_lock);
 				core_tpg_add_node_to_devs(acl, tpg);
-				spin_lock_irq(&tpg->acl_node_lock);
 			}
 		}
-		spin_unlock_irq(&tpg->acl_node_lock);
+		mutex_unlock(&tpg->acl_node_mutex);
 	}
 
 	return lun;
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index c1283c6..3b83026 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -1611,12 +1611,12 @@ core_scsi3_decode_spec_i_port(
 			 * from the decoded fabric module specific TransportID
 			 * at *i_str.
 			 */
-			spin_lock_irq(&tmp_tpg->acl_node_lock);
+			mutex_lock(&tmp_tpg->acl_node_mutex);
 			dest_node_acl = __core_tpg_get_initiator_node_acl(
 						tmp_tpg, i_str);
 			if (dest_node_acl)
 				atomic_inc_mb(&dest_node_acl->acl_pr_ref_count);
-			spin_unlock_irq(&tmp_tpg->acl_node_lock);
+			mutex_unlock(&tmp_tpg->acl_node_mutex);
 
 			if (!dest_node_acl) {
 				core_scsi3_tpg_undepend_item(tmp_tpg);
@@ -3342,12 +3342,12 @@ after_iport_check:
 	/*
 	 * Locate the destination struct se_node_acl from the received Transport ID
 	 */
-	spin_lock_irq(&dest_se_tpg->acl_node_lock);
+	mutex_lock(&dest_se_tpg->acl_node_mutex);
 	dest_node_acl = __core_tpg_get_initiator_node_acl(dest_se_tpg,
 				initiator_str);
 	if (dest_node_acl)
 		atomic_inc_mb(&dest_node_acl->acl_pr_ref_count);
-	spin_unlock_irq(&dest_se_tpg->acl_node_lock);
+	mutex_unlock(&dest_se_tpg->acl_node_mutex);
 
 	if (!dest_node_acl) {
 		pr_err("Unable to locate %s dest_node_acl for"
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 1a12532..027707a 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -85,7 +85,7 @@ static void core_clear_initiator_node_from_tpg(
 
 /*	__core_tpg_get_initiator_node_acl():
  *
- *	spin_lock_bh(&tpg->acl_node_lock); must be held when calling
+ *	mutex_lock(&tpg->acl_node_mutex); must be held when calling
  */
 struct se_node_acl *__core_tpg_get_initiator_node_acl(
 	struct se_portal_group *tpg,
@@ -111,9 +111,9 @@ struct se_node_acl *core_tpg_get_initiator_node_acl(
 {
 	struct se_node_acl *acl;
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	acl = __core_tpg_get_initiator_node_acl(tpg, initiatorname);
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 
 	return acl;
 }
@@ -306,10 +306,10 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
 	    (tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg) != 1))
 		core_tpg_add_node_to_devs(acl, tpg);
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	list_add_tail(&acl->acl_list, &tpg->acl_node_list);
 	tpg->num_node_acls++;
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 
 	pr_debug("%s_TPG[%u] - Added DYNAMIC ACL with TCQ Depth: %d for %s"
 		" Initiator Node: %s\n", tpg->se_tpg_tfo->get_fabric_name(),
@@ -359,7 +359,7 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
 {
 	struct se_node_acl *acl = NULL;
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	acl = __core_tpg_get_initiator_node_acl(tpg, initiatorname);
 	if (acl) {
 		if (acl->dynamic_node_acl) {
@@ -367,7 +367,7 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
 			pr_debug("%s_TPG[%u] - Replacing dynamic ACL"
 				" for %s\n", tpg->se_tpg_tfo->get_fabric_name(),
 				tpg->se_tpg_tfo->tpg_get_tag(tpg), initiatorname);
-			spin_unlock_irq(&tpg->acl_node_lock);
+			mutex_unlock(&tpg->acl_node_mutex);
 			/*
 			 * Release the locally allocated struct se_node_acl
 			 * because * core_tpg_add_initiator_node_acl() returned
@@ -383,10 +383,10 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
 			" Node %s already exists for TPG %u, ignoring"
 			" request.\n",  tpg->se_tpg_tfo->get_fabric_name(),
 			initiatorname, tpg->se_tpg_tfo->tpg_get_tag(tpg));
-		spin_unlock_irq(&tpg->acl_node_lock);
+		mutex_unlock(&tpg->acl_node_mutex);
 		return ERR_PTR(-EEXIST);
 	}
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 
 	if (!se_nacl) {
 		pr_err("struct se_node_acl pointer is NULL\n");
@@ -425,10 +425,10 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
 		return ERR_PTR(-EINVAL);
 	}
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	list_add_tail(&acl->acl_list, &tpg->acl_node_list);
 	tpg->num_node_acls++;
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 
 done:
 	pr_debug("%s_TPG[%hu] - Added ACL with TCQ Depth: %d for %s"
@@ -454,13 +454,13 @@ int core_tpg_del_initiator_node_acl(
 	unsigned long flags;
 	int rc;
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	if (acl->dynamic_node_acl) {
 		acl->dynamic_node_acl = 0;
 	}
 	list_del(&acl->acl_list);
 	tpg->num_node_acls--;
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 
 	spin_lock_irqsave(&acl->nacl_sess_lock, flags);
 	acl->acl_stop = 1;
@@ -519,21 +519,21 @@ int core_tpg_set_initiator_node_queue_depth(
 	unsigned long flags;
 	int dynamic_acl = 0;
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	acl = __core_tpg_get_initiator_node_acl(tpg, initiatorname);
 	if (!acl) {
 		pr_err("Access Control List entry for %s Initiator"
 			" Node %s does not exists for TPG %hu, ignoring"
 			" request.\n", tpg->se_tpg_tfo->get_fabric_name(),
 			initiatorname, tpg->se_tpg_tfo->tpg_get_tag(tpg));
-		spin_unlock_irq(&tpg->acl_node_lock);
+		mutex_unlock(&tpg->acl_node_mutex);
 		return -ENODEV;
 	}
 	if (acl->dynamic_node_acl) {
 		acl->dynamic_node_acl = 0;
 		dynamic_acl = 1;
 	}
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 
 	spin_lock_irqsave(&tpg->session_lock, flags);
 	list_for_each_entry(sess, &tpg->tpg_sess_list, sess_list) {
@@ -549,10 +549,10 @@ int core_tpg_set_initiator_node_queue_depth(
 				tpg->se_tpg_tfo->get_fabric_name(), initiatorname);
 			spin_unlock_irqrestore(&tpg->session_lock, flags);
 
-			spin_lock_irq(&tpg->acl_node_lock);
+			mutex_lock(&tpg->acl_node_mutex);
 			if (dynamic_acl)
 				acl->dynamic_node_acl = 1;
-			spin_unlock_irq(&tpg->acl_node_lock);
+			mutex_unlock(&tpg->acl_node_mutex);
 			return -EEXIST;
 		}
 		/*
@@ -587,10 +587,10 @@ int core_tpg_set_initiator_node_queue_depth(
 		if (init_sess)
 			tpg->se_tpg_tfo->close_session(init_sess);
 
-		spin_lock_irq(&tpg->acl_node_lock);
+		mutex_lock(&tpg->acl_node_mutex);
 		if (dynamic_acl)
 			acl->dynamic_node_acl = 1;
-		spin_unlock_irq(&tpg->acl_node_lock);
+		mutex_unlock(&tpg->acl_node_mutex);
 		return -EINVAL;
 	}
 	spin_unlock_irqrestore(&tpg->session_lock, flags);
@@ -606,10 +606,10 @@ int core_tpg_set_initiator_node_queue_depth(
 		initiatorname, tpg->se_tpg_tfo->get_fabric_name(),
 		tpg->se_tpg_tfo->tpg_get_tag(tpg));
 
-	spin_lock_irq(&tpg->acl_node_lock);
+	mutex_lock(&tpg->acl_node_mutex);
 	if (dynamic_acl)
 		acl->dynamic_node_acl = 1;
-	spin_unlock_irq(&tpg->acl_node_lock);
+	mutex_unlock(&tpg->acl_node_mutex);
 
 	return 0;
 }
@@ -708,9 +708,9 @@ int core_tpg_register(
 	INIT_LIST_HEAD(&se_tpg->acl_node_list);
 	INIT_LIST_HEAD(&se_tpg->se_tpg_node);
 	INIT_LIST_HEAD(&se_tpg->tpg_sess_list);
-	spin_lock_init(&se_tpg->acl_node_lock);
 	spin_lock_init(&se_tpg->session_lock);
 	mutex_init(&se_tpg->tpg_lun_mutex);
+	mutex_init(&se_tpg->acl_node_mutex);
 
 	if (se_tpg->se_tpg_type == TRANSPORT_TPG_TYPE_NORMAL) {
 		if (core_tpg_setup_virtual_lun0(se_tpg) < 0) {
@@ -751,25 +751,26 @@ int core_tpg_deregister(struct se_portal_group *se_tpg)
 
 	while (atomic_read(&se_tpg->tpg_pr_ref_count) != 0)
 		cpu_relax();
+
 	/*
 	 * Release any remaining demo-mode generated se_node_acl that have
 	 * not been released because of TFO->tpg_check_demo_mode_cache() == 1
 	 * in transport_deregister_session().
 	 */
-	spin_lock_irq(&se_tpg->acl_node_lock);
+	mutex_lock(&se_tpg->acl_node_mutex);
 	list_for_each_entry_safe(nacl, nacl_tmp, &se_tpg->acl_node_list,
 			acl_list) {
 		list_del(&nacl->acl_list);
 		se_tpg->num_node_acls--;
-		spin_unlock_irq(&se_tpg->acl_node_lock);
+		mutex_unlock(&se_tpg->acl_node_mutex);
 
 		core_tpg_wait_for_nacl_pr_ref(nacl);
 		core_free_device_list_for_node(nacl, se_tpg);
 		se_tpg->se_tpg_tfo->tpg_release_fabric_acl(se_tpg, nacl);
 
-		spin_lock_irq(&se_tpg->acl_node_lock);
+		mutex_lock(&se_tpg->acl_node_mutex);
 	}
-	spin_unlock_irq(&se_tpg->acl_node_lock);
+	mutex_unlock(&se_tpg->acl_node_mutex);
 
 	if (se_tpg->se_tpg_type == TRANSPORT_TPG_TYPE_NORMAL)
 		core_tpg_remove_lun(se_tpg, &se_tpg->tpg_virt_lun0);
diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index 1fd0f12..dfd78dd 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -473,7 +473,7 @@ void transport_deregister_session(struct se_session *se_sess)
 	struct target_core_fabric_ops *se_tfo;
 	struct se_node_acl *se_nacl;
 	unsigned long flags;
-	bool comp_nacl = true;
+	bool comp_nacl = true, drop_nacl = false;
 
 	if (!se_tpg) {
 		transport_free_session(se_sess);
@@ -493,22 +493,22 @@ void transport_deregister_session(struct se_session *se_sess)
 	 */
 	se_nacl = se_sess->se_node_acl;
 
-	spin_lock_irqsave(&se_tpg->acl_node_lock, flags);
+	mutex_lock(&se_tpg->acl_node_mutex);
 	if (se_nacl && se_nacl->dynamic_node_acl) {
 		if (!se_tfo->tpg_check_demo_mode_cache(se_tpg)) {
 			list_del(&se_nacl->acl_list);
 			se_tpg->num_node_acls--;
-			spin_unlock_irqrestore(&se_tpg->acl_node_lock, flags);
-			core_tpg_wait_for_nacl_pr_ref(se_nacl);
-			core_free_device_list_for_node(se_nacl, se_tpg);
-			se_tfo->tpg_release_fabric_acl(se_tpg, se_nacl);
-
-			comp_nacl = false;
-			spin_lock_irqsave(&se_tpg->acl_node_lock, flags);
+			drop_nacl = true;
 		}
 	}
-	spin_unlock_irqrestore(&se_tpg->acl_node_lock, flags);
+	mutex_unlock(&se_tpg->acl_node_mutex);
 
+	if (drop_nacl) {
+		core_tpg_wait_for_nacl_pr_ref(se_nacl);
+		core_free_device_list_for_node(se_nacl, se_tpg);
+		se_tfo->tpg_release_fabric_acl(se_tpg, se_nacl);
+		comp_nacl = false;
+	}
 	pr_debug("TARGET_CORE[%s]: Deregistered fabric_sess\n",
 		se_tpg->se_tpg_tfo->get_fabric_name());
 	/*
diff --git a/drivers/target/tcm_fc/tfc_conf.c b/drivers/target/tcm_fc/tfc_conf.c
index efdcb96..72187c6 100644
--- a/drivers/target/tcm_fc/tfc_conf.c
+++ b/drivers/target/tcm_fc/tfc_conf.c
@@ -249,7 +249,7 @@ struct ft_node_acl *ft_acl_get(struct ft_tpg *tpg, struct fc_rport_priv *rdata)
 	struct se_portal_group *se_tpg = &tpg->se_tpg;
 	struct se_node_acl *se_acl;
 
-	spin_lock_irq(&se_tpg->acl_node_lock);
+	mutex_lock(&se_tpg->acl_node_mutex);
 	list_for_each_entry(se_acl, &se_tpg->acl_node_list, acl_list) {
 		acl = container_of(se_acl, struct ft_node_acl, se_node_acl);
 		pr_debug("acl %p port_name %llx\n",
@@ -263,7 +263,7 @@ struct ft_node_acl *ft_acl_get(struct ft_tpg *tpg, struct fc_rport_priv *rdata)
 			break;
 		}
 	}
-	spin_unlock_irq(&se_tpg->acl_node_lock);
+	mutex_unlock(&se_tpg->acl_node_mutex);
 	return found;
 }
 
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 5a7a8fa..20b01cf 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -880,7 +880,7 @@ struct se_portal_group {
 	/* Used for PR SPEC_I_PT=1 and REGISTER_AND_MOVE */
 	atomic_t		tpg_pr_ref_count;
 	/* Spinlock for adding/removing ACLed Nodes */
-	spinlock_t		acl_node_lock;
+	struct mutex		acl_node_mutex;
 	/* Spinlock for adding/removing sessions */
 	spinlock_t		session_lock;
 	struct mutex		tpg_lun_mutex;
-- 
1.9.1

[RFC 14/22] target: Convert se_node_acl->lun_entry_lock to ->lun_entry_mutex

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

Now that se_node_acl->lun_entry_hlist[] fast path access uses RCU,
go ahead and convert se_node_acl->lun_entry_lock to struct mutex,
for non fast-path RCU updater code.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c | 18 ++++++++----------
 drivers/target/target_core_pr.c     | 12 ++++++------
 drivers/target/target_core_tpg.c    | 14 ++++++--------
 include/target/target_core_base.h   |  3 +--
 4 files changed, 21 insertions(+), 26 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 965d81e..2a5fa88 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -304,7 +304,7 @@ void core_update_device_list_access(
 {
 	struct se_dev_entry *deve;
 
-	spin_lock_irq(&nacl->lun_entry_lock);
+	mutex_lock(&nacl->lun_entry_mutex);
 	deve = rcu_dereference(nacl->lun_entry_hlist[mapped_lun]);
 	if (lun_access & TRANSPORT_LUNFLAGS_READ_WRITE) {
 		deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_ONLY;
@@ -313,7 +313,7 @@ void core_update_device_list_access(
 		deve->lun_flags &= ~TRANSPORT_LUNFLAGS_READ_WRITE;
 		deve->lun_flags |= TRANSPORT_LUNFLAGS_READ_ONLY;
 	}
-	spin_unlock_irq(&nacl->lun_entry_lock);
+	mutex_unlock(&nacl->lun_entry_mutex);
 
 	synchronize_rcu();
 }
@@ -345,7 +345,7 @@ int core_enable_device_list_for_node(
 	 * transition.  This transition must be for the same struct se_lun
 	 * + mapped_lun that was setup in demo mode..
 	 */
-	spin_lock_irq(&nacl->lun_entry_lock);
+	mutex_lock(&nacl->lun_entry_mutex);
 	deve = nacl->lun_entry_hlist[mapped_lun];
 	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
 		BUG_ON(deve->se_lun_acl != NULL);
@@ -359,7 +359,7 @@ int core_enable_device_list_for_node(
 			deve->lun_flags |= TRANSPORT_LUNFLAGS_READ_ONLY;
 		}
 		rcu_assign_pointer(deve->se_lun_acl, lun_acl);
-		spin_unlock_irq(&nacl->lun_entry_lock);
+		mutex_unlock(&nacl->lun_entry_mutex);
 
 		synchronize_rcu();
 		return 0;
@@ -384,7 +384,7 @@ int core_enable_device_list_for_node(
 
 	rcu_assign_pointer(deve->se_lun, lun);
 	rcu_assign_pointer(deve->se_lun_acl, lun_acl);
-	spin_unlock_irq(&nacl->lun_entry_lock);
+	mutex_unlock(&nacl->lun_entry_mutex);
 
 	spin_lock_bh(&port->sep_alua_lock);
 	list_add_tail(&deve->alua_port_list, &port->sep_alua_list);
@@ -409,8 +409,8 @@ int core_disable_device_list_for_node(
 	struct se_port *port = lun->lun_sep;
 	struct se_dev_entry *deve;
 
-	rcu_read_lock();
-	deve = rcu_dereference(nacl->lun_entry_hlist[mapped_lun]);
+	mutex_lock(&nacl->lun_entry_mutex);
+	deve = (nacl->lun_entry_hlist[mapped_lun]);
 	/*
 	 * If the MappedLUN entry is being disabled, the entry in
 	 * port->sep_alua_list must be removed now before clearing the
@@ -430,7 +430,6 @@ int core_disable_device_list_for_node(
 	/*
 	 * Disable struct se_dev_entry LUN ACL mapping
 	 */
-	spin_lock_irq(&nacl->lun_entry_lock);
 	core_scsi3_ua_release_all(deve);
 	rcu_assign_pointer(deve->pr_reg, NULL);
 	rcu_assign_pointer(deve->se_lun, NULL);
@@ -438,8 +437,7 @@ int core_disable_device_list_for_node(
 	deve->lun_flags = 0;
 	deve->creation_time = 0;
 	deve->attach_count--;
-	spin_unlock_irq(&nacl->lun_entry_lock);
-	rcu_read_unlock();
+	mutex_unlock(&nacl->lun_entry_mutex);
 
 	/*
 	 * Wait for RCU read critical sections to complete after
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index 3b83026..93e109e 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -1030,10 +1030,10 @@ static void __core_scsi3_add_registration(
 	__core_scsi3_dump_registration(tfo, dev, nacl, pr_reg, register_type);
 	spin_unlock(&pr_tmpl->registration_lock);
 
-	spin_lock(&nacl->lun_entry_lock);
+	mutex_lock(&nacl->lun_entry_mutex);
 	deve = nacl->lun_entry_hlist[pr_reg->pr_res_mapped_lun];
 	rcu_assign_pointer(deve->pr_reg, pr_reg);
-	spin_unlock(&nacl->lun_entry_lock);
+	mutex_unlock(&nacl->lun_entry_mutex);
 	/*
 	 * Wait for read path critical RCU in core_scsi3_pr_seq_non_holder()
 	 * conditional checks for deve->pr_reg pointer access complete.
@@ -1065,10 +1065,10 @@ static void __core_scsi3_add_registration(
 					       register_type);
 		spin_unlock(&pr_tmpl->registration_lock);
 
-		spin_lock(&nacl->lun_entry_lock);
+		mutex_lock(&nacl->lun_entry_mutex);
 		deve = nacl_tmp->lun_entry_hlist[pr_reg_tmp->pr_res_mapped_lun];
 		rcu_assign_pointer(deve->pr_reg, pr_reg_tmp);
-		spin_unlock(&nacl->lun_entry_lock);
+		mutex_unlock(&nacl->lun_entry_mutex);
 		/*
 		 * Wait for read path critical RCU in core_scsi3_pr_seq_non_holder()
 		 * conditional checks for deve->pr_reg pointer access complete.
@@ -1280,10 +1280,10 @@ static void __core_scsi3_free_registration(
 		cpu_relax();
 	}
 
-	spin_lock(&nacl->lun_entry_lock);
+	mutex_lock(&nacl->lun_entry_mutex);
 	deve = nacl->lun_entry_hlist[pr_reg->pr_res_mapped_lun];
 	rcu_assign_pointer(deve->pr_reg, NULL);
-	spin_unlock(&nacl->lun_entry_lock);
+	mutex_unlock(&nacl->lun_entry_mutex);
 	/*
 	 * Wait for read path critical RCU in core_scsi3_pr_seq_non_holder()
 	 * conditional checks for deve->pr_reg pointer access complete.
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 027707a..3d81a72 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -275,9 +275,8 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
 	INIT_LIST_HEAD(&acl->acl_sess_list);
 	kref_init(&acl->acl_kref);
 	init_completion(&acl->acl_free_comp);
-	spin_lock_init(&acl->device_list_lock);
 	spin_lock_init(&acl->nacl_sess_lock);
-	spin_lock_init(&acl->lun_entry_lock);
+	mutex_init(&acl->lun_entry_mutex);
 	atomic_set(&acl->acl_pr_ref_count, 0);
 	acl->queue_depth = tpg->se_tpg_tfo->tpg_get_default_depth(tpg);
 	snprintf(acl->initiatorname, TRANSPORT_IQN_LEN, "%s", initiatorname);
@@ -331,7 +330,7 @@ void core_tpg_clear_object_luns(struct se_portal_group *tpg)
 	int i;
 	struct se_lun *lun;
 
-	spin_lock(&tpg->tpg_lun_lock);
+	mutex_lock(&tpg->tpg_lun_mutex);
 	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
 		lun = tpg->tpg_lun_list[i];
 
@@ -339,11 +338,11 @@ void core_tpg_clear_object_luns(struct se_portal_group *tpg)
 		    (lun->lun_se_dev == NULL))
 			continue;
 
-		spin_unlock(&tpg->tpg_lun_lock);
+		mutex_unlock(&tpg->tpg_lun_mutex);
 		core_dev_del_lun(tpg, lun);
-		spin_lock(&tpg->tpg_lun_lock);
+		mutex_lock(&tpg->tpg_lun_mutex);
 	}
-	spin_unlock(&tpg->tpg_lun_lock);
+	mutex_unlock(&tpg->tpg_lun_mutex);
 }
 EXPORT_SYMBOL(core_tpg_clear_object_luns);
 
@@ -403,9 +402,8 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
 	INIT_LIST_HEAD(&acl->acl_sess_list);
 	kref_init(&acl->acl_kref);
 	init_completion(&acl->acl_free_comp);
-	spin_lock_init(&acl->device_list_lock);
 	spin_lock_init(&acl->nacl_sess_lock);
-	spin_lock_init(&acl->lun_entry_lock);
+	mutex_init(&acl->lun_entry_mutex);
 	atomic_set(&acl->acl_pr_ref_count, 0);
 	acl->queue_depth = queue_depth;
 	snprintf(acl->initiatorname, TRANSPORT_IQN_LEN, "%s", initiatorname);
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index 20b01cf..ab25709 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -599,8 +599,7 @@ struct se_node_acl {
 	struct se_dev_entry	**device_list;
 	struct se_session	*nacl_sess;
 	struct se_portal_group *se_tpg;
-	spinlock_t		device_list_lock;
-	spinlock_t		lun_entry_lock;
+	struct mutex		lun_entry_mutex;
 	spinlock_t		nacl_sess_lock;
 	struct config_group	acl_group;
 	struct config_group	acl_attrib_group;
-- 
1.9.1

[RFC 15/22] target: Convert core_tpg_deregister to use list splice

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This patch converts core_tpg_deregister() to perform a list splice
for any remaining dynamically generated se_node_acls attached to
se_tpg, before calling tfo->tpg_release_fabric_acl() to free memory.

Cc: Hannes Reinecke <hare@suse.de>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Sagi Grimberg <sagig@mellanox.com>
Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_tpg.c | 12 +++++-------
 1 file changed, 5 insertions(+), 7 deletions(-)

diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 3d81a72..b5b4053 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -735,6 +735,7 @@ EXPORT_SYMBOL(core_tpg_register);
 int core_tpg_deregister(struct se_portal_group *se_tpg)
 {
 	struct se_node_acl *nacl, *nacl_tmp;
+	LIST_HEAD(node_list);
 
 	pr_debug("TARGET_CORE[%s]: Deallocating %s struct se_portal_group"
 		" for endpoint: %s Portal Tag %u\n",
@@ -750,25 +751,22 @@ int core_tpg_deregister(struct se_portal_group *se_tpg)
 	while (atomic_read(&se_tpg->tpg_pr_ref_count) != 0)
 		cpu_relax();
 
+	mutex_lock(&se_tpg->acl_node_mutex);
+	list_splice_init(&se_tpg->acl_node_list, &node_list);
+	mutex_unlock(&se_tpg->acl_node_mutex);
 	/*
 	 * Release any remaining demo-mode generated se_node_acl that have
 	 * not been released because of TFO->tpg_check_demo_mode_cache() == 1
 	 * in transport_deregister_session().
 	 */
-	mutex_lock(&se_tpg->acl_node_mutex);
-	list_for_each_entry_safe(nacl, nacl_tmp, &se_tpg->acl_node_list,
-			acl_list) {
+	list_for_each_entry_safe(nacl, nacl_tmp, &node_list, acl_list) {
 		list_del(&nacl->acl_list);
 		se_tpg->num_node_acls--;
-		mutex_unlock(&se_tpg->acl_node_mutex);
 
 		core_tpg_wait_for_nacl_pr_ref(nacl);
 		core_free_device_list_for_node(nacl, se_tpg);
 		se_tpg->se_tpg_tfo->tpg_release_fabric_acl(se_tpg, nacl);
-
-		mutex_lock(&se_tpg->acl_node_mutex);
 	}
-	mutex_unlock(&se_tpg->acl_node_mutex);
 
 	if (se_tpg->se_tpg_type == TRANSPORT_TPG_TYPE_NORMAL)
 		core_tpg_remove_lun(se_tpg, &se_tpg->tpg_virt_lun0);
-- 
1.9.1

[RFC 16/22] target: Drop se_lun->lun_acl_list

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c | 13 -------------
 drivers/target/target_core_tpg.c    |  4 ----
 include/target/target_core_base.h   |  3 ---
 3 files changed, 20 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 2a5fa88..80d1b04 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -1338,7 +1338,6 @@ struct se_lun_acl *core_dev_init_initiator_node_lun_acl(
 		return NULL;
 	}
 
-	INIT_LIST_HEAD(&lacl->lacl_list);
 	lacl->mapped_lun = mapped_lun;
 	lacl->se_lun_nacl = nacl;
 	snprintf(lacl->initiatorname, TRANSPORT_IQN_LEN, "%s",
@@ -1379,11 +1378,6 @@ int core_dev_add_initiator_node_lun_acl(
 			lun_access, nacl, tpg) < 0)
 		return -EINVAL;
 
-	spin_lock(&lun->lun_acl_lock);
-	list_add_tail(&lacl->lacl_list, &lun->lun_acl_list);
-	atomic_inc_mb(&lun->lun_acl_count);
-	spin_unlock(&lun->lun_acl_lock);
-
 	pr_debug("%s_TPG[%hu]_LUN[%u->%u] - Added %s ACL for "
 		" InitiatorNode: %s\n", tpg->se_tpg_tfo->get_fabric_name(),
 		tpg->se_tpg_tfo->tpg_get_tag(tpg), unpacked_lun, lacl->mapped_lun,
@@ -1413,11 +1407,6 @@ int core_dev_del_initiator_node_lun_acl(
 	if (!nacl)
 		return -EINVAL;
 
-	spin_lock(&lun->lun_acl_lock);
-	list_del(&lacl->lacl_list);
-	atomic_dec_mb(&lun->lun_acl_count);
-	spin_unlock(&lun->lun_acl_lock);
-
 	core_disable_device_list_for_node(lun, NULL, lacl->mapped_lun,
 		TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 
@@ -1552,8 +1541,6 @@ struct se_device *target_alloc_device(struct se_hba *hba, const char *name)
 	xcopy_lun = &dev->xcopy_lun;
 	xcopy_lun->lun_se_dev = dev;
 	init_completion(&xcopy_lun->lun_shutdown_comp);
-	INIT_LIST_HEAD(&xcopy_lun->lun_acl_list);
-	spin_lock_init(&xcopy_lun->lun_acl_lock);
 	spin_lock_init(&xcopy_lun->lun_sep_lock);
 	init_completion(&xcopy_lun->lun_ref_comp);
 
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index b5b4053..a1849ce 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -655,8 +655,6 @@ static int core_tpg_setup_virtual_lun0(struct se_portal_group *se_tpg)
 	lun->lun_status = TRANSPORT_LUN_STATUS_FREE;
 	atomic_set(&lun->lun_acl_count, 0);
 	init_completion(&lun->lun_shutdown_comp);
-	INIT_LIST_HEAD(&lun->lun_acl_list);
-	spin_lock_init(&lun->lun_acl_lock);
 	spin_lock_init(&lun->lun_sep_lock);
 	init_completion(&lun->lun_ref_comp);
 
@@ -692,8 +690,6 @@ int core_tpg_register(
 		lun->lun_status = TRANSPORT_LUN_STATUS_FREE;
 		atomic_set(&lun->lun_acl_count, 0);
 		init_completion(&lun->lun_shutdown_comp);
-		INIT_LIST_HEAD(&lun->lun_acl_list);
-		spin_lock_init(&lun->lun_acl_lock);
 		spin_lock_init(&lun->lun_sep_lock);
 		init_completion(&lun->lun_ref_comp);
 	}
diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
index ab25709..f24976d 100644
--- a/include/target/target_core_base.h
+++ b/include/target/target_core_base.h
@@ -646,7 +646,6 @@ struct se_lun_acl {
 	u32			mapped_lun;
 	struct se_node_acl	*se_lun_nacl;
 	struct se_lun		*se_lun;
-	struct list_head	lacl_list;
 	struct config_group	se_lun_group;
 	struct se_ml_stat_grps	ml_stat_grps;
 };
@@ -724,10 +723,8 @@ struct se_lun {
 	u32			lun_flags;
 	u32			unpacked_lun;
 	atomic_t		lun_acl_count;
-	spinlock_t		lun_acl_lock;
 	spinlock_t		lun_sep_lock;
 	struct completion	lun_shutdown_comp;
-	struct list_head	lun_acl_list;
 	struct se_device	*lun_se_dev;
 	struct se_port		*lun_sep;
 	struct config_group	lun_group;
-- 
1.9.1

[RFC 17/22] target: Drop core_tpg_clear_object_luns

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/iscsi/iscsi_target_tpg.c |  2 --
 drivers/target/target_core_tpg.c        | 21 ---------------------
 include/target/target_core_fabric.h     |  1 -
 3 files changed, 24 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index bdd127c..2261c8c 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -284,8 +284,6 @@ int iscsit_tpg_del_portal_group(
 		return -EPERM;
 	}
 
-	core_tpg_clear_object_luns(&tpg->tpg_se_tpg);
-
 	if (tpg->param_list) {
 		iscsi_release_param_list(tpg->param_list);
 		tpg->param_list = NULL;
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index a1849ce..f197618 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -325,27 +325,6 @@ void core_tpg_wait_for_nacl_pr_ref(struct se_node_acl *nacl)
 		cpu_relax();
 }
 
-void core_tpg_clear_object_luns(struct se_portal_group *tpg)
-{
-	int i;
-	struct se_lun *lun;
-
-	mutex_lock(&tpg->tpg_lun_mutex);
-	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
-		lun = tpg->tpg_lun_list[i];
-
-		if ((lun->lun_status != TRANSPORT_LUN_STATUS_ACTIVE) ||
-		    (lun->lun_se_dev == NULL))
-			continue;
-
-		mutex_unlock(&tpg->tpg_lun_mutex);
-		core_dev_del_lun(tpg, lun);
-		mutex_lock(&tpg->tpg_lun_mutex);
-	}
-	mutex_unlock(&tpg->tpg_lun_mutex);
-}
-EXPORT_SYMBOL(core_tpg_clear_object_luns);
-
 /*	core_tpg_add_initiator_node_acl():
  *
  *
diff --git a/include/target/target_core_fabric.h b/include/target/target_core_fabric.h
index 9811c85..578fc93 100644
--- a/include/target/target_core_fabric.h
+++ b/include/target/target_core_fabric.h
@@ -152,7 +152,6 @@ struct se_node_acl *core_tpg_get_initiator_node_acl(struct se_portal_group *tpg,
 		unsigned char *);
 struct se_node_acl *core_tpg_check_initiator_node_acl(struct se_portal_group *,
 		unsigned char *);
-void	core_tpg_clear_object_luns(struct se_portal_group *);
 struct se_node_acl *core_tpg_add_initiator_node_acl(struct se_portal_group *,
 		struct se_node_acl *, const char *, u32);
 int	core_tpg_del_initiator_node_acl(struct se_portal_group *,
-- 
1.9.1

[RFC 18/22] target: Rename TPG initiator_node_acl to target_* prefix

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This includes + fabric driver changes:

  - target_check_initiator_node_acl
  - target_add_initiator_node_acl
  - target_del_initiator_node_acl
  - target_set_initiator_node_queue_depth
  - target_del_initiator_node_acl
  - target_get_initiator_node_acl
  - target_clear_initiator_node_from_tpg

Also, update fabric drivers for loopback, tcm_fc, iscsi/iser-target,
ib_srpt, sbp-target, usb-gadget, tcm_qla2xxx, vhost-scsi and xen-scsiback.

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/infiniband/ulp/srpt/ib_srpt.c        |  8 ++---
 drivers/scsi/qla2xxx/tcm_qla2xxx.c           | 12 +++----
 drivers/target/iscsi/iscsi_target.c          |  2 +-
 drivers/target/iscsi/iscsi_target_configfs.c | 13 ++++---
 drivers/target/iscsi/iscsi_target_nego.c     |  4 +--
 drivers/target/iscsi/iscsi_target_tpg.c      |  2 +-
 drivers/target/loopback/tcm_loop.c           |  4 +--
 drivers/target/sbp/sbp_target.c              | 10 +++---
 drivers/target/target_core_internal.h        |  2 +-
 drivers/target/target_core_pr.c              |  4 +--
 drivers/target/target_core_tpg.c             | 54 ++++++++++++++--------------
 drivers/target/tcm_fc/tfc_conf.c             |  6 ++--
 drivers/usb/gadget/legacy/tcm_usb_gadget.c   | 14 ++++----
 drivers/vhost/scsi.c                         | 12 +++----
 include/target/target_core_fabric.h          | 12 +++----
 15 files changed, 79 insertions(+), 80 deletions(-)

diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 6e0a477..c2f833d 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -3626,11 +3626,11 @@ static struct se_node_acl *srpt_make_nodeacl(struct se_portal_group *tpg,
 		goto err;
 	}
 	/*
-	 * nacl_new may be released by core_tpg_add_initiator_node_acl()
+	 * nacl_new may be released by target_add_initiator_node_acl()
 	 * when converting a node ACL from demo mode to explict
 	 */
-	se_nacl = core_tpg_add_initiator_node_acl(tpg, se_nacl_new, name,
-						  nexus_depth);
+	se_nacl = target_add_initiator_node_acl(tpg, se_nacl_new, name,
+						nexus_depth);
 	if (IS_ERR(se_nacl)) {
 		ret = PTR_ERR(se_nacl);
 		goto err;
@@ -3665,7 +3665,7 @@ static void srpt_drop_nodeacl(struct se_node_acl *se_nacl)
 	spin_lock_irq(&sport->port_acl_lock);
 	list_del(&nacl->list);
 	spin_unlock_irq(&sport->port_acl_lock);
-	core_tpg_del_initiator_node_acl(&sport->port_tpg_1, se_nacl, 1);
+	target_del_initiator_node_acl(&sport->port_tpg_1, se_nacl, 1);
 	srpt_release_fabric_acl(NULL, se_nacl);
 }
 
diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
index 65a683c..e29515b 100644
--- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
+++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
@@ -865,11 +865,11 @@ static struct se_node_acl *tcm_qla2xxx_make_nodeacl(
 	qla2xxx_nexus_depth = 1;
 
 	/*
-	 * se_nacl_new may be released by core_tpg_add_initiator_node_acl()
+	 * se_nacl_new may be released by target_add_initiator_node_acl()
 	 * when converting a NodeACL from demo mode -> explict
 	 */
-	se_nacl = core_tpg_add_initiator_node_acl(se_tpg, se_nacl_new,
-				name, qla2xxx_nexus_depth);
+	se_nacl = target_add_initiator_node_acl(se_tpg, se_nacl_new,
+						name, qla2xxx_nexus_depth);
 	if (IS_ERR(se_nacl)) {
 		tcm_qla2xxx_release_fabric_acl(se_tpg, se_nacl_new);
 		return se_nacl;
@@ -890,7 +890,7 @@ static void tcm_qla2xxx_drop_nodeacl(struct se_node_acl *se_acl)
 	struct tcm_qla2xxx_nacl *nacl = container_of(se_acl,
 				struct tcm_qla2xxx_nacl, se_node_acl);
 
-	core_tpg_del_initiator_node_acl(se_tpg, se_acl, 1);
+	target_del_initiator_node_acl(se_tpg, se_acl, 1);
 	kfree(nacl);
 }
 
@@ -1575,8 +1575,8 @@ static int tcm_qla2xxx_check_initiator_node_acl(
 	 * Locate our struct se_node_acl either from an explict NodeACL created
 	 * via ConfigFS, or via running in TPG demo mode.
 	 */
-	se_sess->se_node_acl = core_tpg_check_initiator_node_acl(se_tpg,
-					port_name);
+	se_sess->se_node_acl = target_check_initiator_node_acl(se_tpg,
+							       port_name);
 	if (!se_sess->se_node_acl) {
 		transport_free_session(se_sess);
 		return -EINVAL;
diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 935cc05..75e24ef 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -3443,7 +3443,7 @@ iscsit_build_sendtargets_response(struct iscsi_cmd *cmd,
 
 			if ((tpg->tpg_attrib.generate_node_acls == 0) &&
 			    (tpg->tpg_attrib.demo_mode_discovery == 0) &&
-			    (!core_tpg_get_initiator_node_acl(&tpg->tpg_se_tpg,
+			    (!target_get_initiator_node_acl(&tpg->tpg_se_tpg,
 				cmd->conn->sess->sess_ops->InitiatorName))) {
 				continue;
 			}
diff --git a/drivers/target/iscsi/iscsi_target_configfs.c b/drivers/target/iscsi/iscsi_target_configfs.c
index 48384b6..167782a 100644
--- a/drivers/target/iscsi/iscsi_target_configfs.c
+++ b/drivers/target/iscsi/iscsi_target_configfs.c
@@ -847,8 +847,7 @@ static ssize_t lio_target_nacl_store_tag(
 {
 	int ret;
 
-	ret = core_tpg_set_initiator_node_tag(se_nacl->se_tpg, se_nacl, page);
-
+	ret = target_set_initiator_node_tag(se_nacl->se_tpg, se_nacl, page);
 	if (ret < 0)
 		return ret;
 	return count;
@@ -895,11 +894,11 @@ static struct se_node_acl *lio_target_make_nodeacl(
 
 	cmdsn_depth = tpg->tpg_attrib.default_cmdsn_depth;
 	/*
-	 * se_nacl_new may be released by core_tpg_add_initiator_node_acl()
+	 * se_nacl_new may be released by target_add_initiator_node_acl()
 	 * when converting a NdoeACL from demo mode -> explict
 	 */
-	se_nacl = core_tpg_add_initiator_node_acl(se_tpg, se_nacl_new,
-				name, cmdsn_depth);
+	se_nacl = target_add_initiator_node_acl(se_tpg, se_nacl_new,
+						name, cmdsn_depth);
 	if (IS_ERR(se_nacl))
 		return se_nacl;
 
@@ -911,7 +910,7 @@ static struct se_node_acl *lio_target_make_nodeacl(
 	if (!stats_cg->default_groups) {
 		pr_err("Unable to allocate memory for"
 				" stats_cg->default_groups\n");
-		core_tpg_del_initiator_node_acl(se_tpg, se_nacl, 1);
+		target_del_initiator_node_acl(se_tpg, se_nacl, 1);
 		kfree(acl);
 		return ERR_PTR(-ENOMEM);
 	}
@@ -942,7 +941,7 @@ static void lio_target_drop_nodeacl(
 	}
 	kfree(stats_cg->default_groups);
 
-	core_tpg_del_initiator_node_acl(se_tpg, se_nacl, 1);
+	target_del_initiator_node_acl(se_tpg, se_nacl, 1);
 	kfree(acl);
 }
 
diff --git a/drivers/target/iscsi/iscsi_target_nego.c b/drivers/target/iscsi/iscsi_target_nego.c
index 8c02fa3..ce326cd 100644
--- a/drivers/target/iscsi/iscsi_target_nego.c
+++ b/drivers/target/iscsi/iscsi_target_nego.c
@@ -1175,8 +1175,8 @@ get_target:
 	/*
 	 * Locate incoming Initiator IQN reference from Storage Node.
 	 */
-	sess->se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
-			&conn->tpg->tpg_se_tpg, i_buf);
+	sess->se_sess->se_node_acl = target_check_initiator_node_acl(
+					&conn->tpg->tpg_se_tpg, i_buf);
 	if (!sess->se_sess->se_node_acl) {
 		pr_err("iSCSI Initiator Node: %s is not authorized to"
 			" access iSCSI target portal group: %hu.\n",
diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index 2261c8c..4ebc20b 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -598,7 +598,7 @@ int iscsit_tpg_set_initiator_node_queue_depth(
 	u32 queue_depth,
 	int force)
 {
-	return core_tpg_set_initiator_node_queue_depth(&tpg->tpg_se_tpg,
+	return target_set_initiator_node_queue_depth(&tpg->tpg_se_tpg,
 		initiatorname, queue_depth, force);
 }
 
diff --git a/drivers/target/loopback/tcm_loop.c b/drivers/target/loopback/tcm_loop.c
index 696c2ca..b271624 100644
--- a/drivers/target/loopback/tcm_loop.c
+++ b/drivers/target/loopback/tcm_loop.c
@@ -991,8 +991,8 @@ static int tcm_loop_make_nexus(
 	 * struct se_node_acl for the tcm_loop struct se_portal_group with the SCSI
 	 * Initiator port name of the passed configfs group 'name'.
 	 */
-	tl_nexus->se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
-				se_tpg, (unsigned char *)name);
+	tl_nexus->se_sess->se_node_acl = target_check_initiator_node_acl(
+					se_tpg, (unsigned char *)name);
 	if (!tl_nexus->se_sess->se_node_acl) {
 		transport_free_session(tl_nexus->se_sess);
 		goto out;
diff --git a/drivers/target/sbp/sbp_target.c b/drivers/target/sbp/sbp_target.c
index ec46968..f433d56 100644
--- a/drivers/target/sbp/sbp_target.c
+++ b/drivers/target/sbp/sbp_target.c
@@ -221,7 +221,7 @@ static struct sbp_session *sbp_session_create(
 
 	snprintf(guid_str, sizeof(guid_str), "%016llx", guid);
 
-	se_nacl = core_tpg_check_initiator_node_acl(&tpg->se_tpg, guid_str);
+	se_nacl = target_check_initiator_node_acl(&tpg->se_tpg, guid_str);
 	if (!se_nacl) {
 		pr_warn("Node ACL not found for %s\n", guid_str);
 
@@ -2115,11 +2115,11 @@ static struct se_node_acl *sbp_make_nodeacl(
 		return ERR_PTR(-ENOMEM);
 
 	/*
-	 * se_nacl_new may be released by core_tpg_add_initiator_node_acl()
+	 * se_nacl_new may be released by target_add_initiator_node_acl()
 	 * when converting a NodeACL from demo mode -> explict
 	 */
-	se_nacl = core_tpg_add_initiator_node_acl(se_tpg, se_nacl_new,
-			name, nexus_depth);
+	se_nacl = target_add_initiator_node_acl(se_tpg, se_nacl_new,
+						name, nexus_depth);
 	if (IS_ERR(se_nacl)) {
 		sbp_release_fabric_acl(se_tpg, se_nacl_new);
 		return se_nacl;
@@ -2137,7 +2137,7 @@ static void sbp_drop_nodeacl(struct se_node_acl *se_acl)
 	struct sbp_nacl *nacl =
 		container_of(se_acl, struct sbp_nacl, se_node_acl);
 
-	core_tpg_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
+	target_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
 	kfree(nacl);
 }
 
diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h
index 60381db..574b396 100644
--- a/drivers/target/target_core_internal.h
+++ b/drivers/target/target_core_internal.h
@@ -48,7 +48,7 @@ int	core_tmr_lun_reset(struct se_device *, struct se_tmr_req *,
 /* target_core_tpg.c */
 extern struct se_device *g_lun0_dev;
 
-struct se_node_acl *__core_tpg_get_initiator_node_acl(struct se_portal_group *tpg,
+struct se_node_acl *__target_get_initiator_node_acl(struct se_portal_group *tpg,
 		const char *);
 void	core_tpg_add_node_to_devs(struct se_node_acl *, struct se_portal_group *);
 void	core_tpg_wait_for_nacl_pr_ref(struct se_node_acl *);
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index 93e109e..3a5ee0b 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -1612,7 +1612,7 @@ core_scsi3_decode_spec_i_port(
 			 * at *i_str.
 			 */
 			mutex_lock(&tmp_tpg->acl_node_mutex);
-			dest_node_acl = __core_tpg_get_initiator_node_acl(
+			dest_node_acl = __target_get_initiator_node_acl(
 						tmp_tpg, i_str);
 			if (dest_node_acl)
 				atomic_inc_mb(&dest_node_acl->acl_pr_ref_count);
@@ -3343,7 +3343,7 @@ after_iport_check:
 	 * Locate the destination struct se_node_acl from the received Transport ID
 	 */
 	mutex_lock(&dest_se_tpg->acl_node_mutex);
-	dest_node_acl = __core_tpg_get_initiator_node_acl(dest_se_tpg,
+	dest_node_acl = __target_get_initiator_node_acl(dest_se_tpg,
 				initiator_str);
 	if (dest_node_acl)
 		atomic_inc_mb(&dest_node_acl->acl_pr_ref_count);
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index f197618..86dda22 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -47,11 +47,11 @@ extern struct se_device *g_lun0_dev;
 static DEFINE_SPINLOCK(tpg_lock);
 static LIST_HEAD(tpg_list);
 
-/*	core_clear_initiator_node_from_tpg():
+/*	target_clear_initiator_node_from_tpg():
  *
  *
  */
-static void core_clear_initiator_node_from_tpg(
+static void target_clear_initiator_node_from_tpg(
 	struct se_node_acl *nacl,
 	struct se_portal_group *tpg)
 {
@@ -83,11 +83,11 @@ static void core_clear_initiator_node_from_tpg(
 	}
 }
 
-/*	__core_tpg_get_initiator_node_acl():
+/*	__target_get_initiator_node_acl():
  *
  *	mutex_lock(&tpg->acl_node_mutex); must be held when calling
  */
-struct se_node_acl *__core_tpg_get_initiator_node_acl(
+struct se_node_acl *__target_get_initiator_node_acl(
 	struct se_portal_group *tpg,
 	const char *initiatorname)
 {
@@ -101,23 +101,23 @@ struct se_node_acl *__core_tpg_get_initiator_node_acl(
 	return NULL;
 }
 
-/*	core_tpg_get_initiator_node_acl():
+/*	target_get_initiator_node_acl():
  *
  *
  */
-struct se_node_acl *core_tpg_get_initiator_node_acl(
+struct se_node_acl *target_get_initiator_node_acl(
 	struct se_portal_group *tpg,
 	unsigned char *initiatorname)
 {
 	struct se_node_acl *acl;
 
 	mutex_lock(&tpg->acl_node_mutex);
-	acl = __core_tpg_get_initiator_node_acl(tpg, initiatorname);
+	acl = __target_get_initiator_node_acl(tpg, initiatorname);
 	mutex_unlock(&tpg->acl_node_mutex);
 
 	return acl;
 }
-EXPORT_SYMBOL(core_tpg_get_initiator_node_acl);
+EXPORT_SYMBOL(target_get_initiator_node_acl);
 
 /*	core_tpg_add_node_to_devs():
  *
@@ -250,17 +250,17 @@ static int core_create_device_list_for_node(struct se_node_acl *nacl)
 	return 0;
 }
 
-/*	core_tpg_check_initiator_node_acl()
+/*	target_check_initiator_node_acl()
  *
  *
  */
-struct se_node_acl *core_tpg_check_initiator_node_acl(
+struct se_node_acl *target_check_initiator_node_acl(
 	struct se_portal_group *tpg,
 	unsigned char *initiatorname)
 {
 	struct se_node_acl *acl;
 
-	acl = core_tpg_get_initiator_node_acl(tpg, initiatorname);
+	acl = target_get_initiator_node_acl(tpg, initiatorname);
 	if (acl)
 		return acl;
 
@@ -317,7 +317,7 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
 
 	return acl;
 }
-EXPORT_SYMBOL(core_tpg_check_initiator_node_acl);
+EXPORT_SYMBOL(target_check_initiator_node_acl);
 
 void core_tpg_wait_for_nacl_pr_ref(struct se_node_acl *nacl)
 {
@@ -325,11 +325,11 @@ void core_tpg_wait_for_nacl_pr_ref(struct se_node_acl *nacl)
 		cpu_relax();
 }
 
-/*	core_tpg_add_initiator_node_acl():
+/*	target_tadd_initiator_node_acl():
  *
  *
  */
-struct se_node_acl *core_tpg_add_initiator_node_acl(
+struct se_node_acl *target_add_initiator_node_acl(
 	struct se_portal_group *tpg,
 	struct se_node_acl *se_nacl,
 	const char *initiatorname,
@@ -338,7 +338,7 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
 	struct se_node_acl *acl = NULL;
 
 	mutex_lock(&tpg->acl_node_mutex);
-	acl = __core_tpg_get_initiator_node_acl(tpg, initiatorname);
+	acl = __target_get_initiator_node_acl(tpg, initiatorname);
 	if (acl) {
 		if (acl->dynamic_node_acl) {
 			acl->dynamic_node_acl = 0;
@@ -415,13 +415,13 @@ done:
 
 	return acl;
 }
-EXPORT_SYMBOL(core_tpg_add_initiator_node_acl);
+EXPORT_SYMBOL(target_add_initiator_node_acl);
 
-/*	core_tpg_del_initiator_node_acl():
+/*	target_del_initiator_node_acl():
  *
  *
  */
-int core_tpg_del_initiator_node_acl(
+int target_del_initiator_node_acl(
 	struct se_portal_group *tpg,
 	struct se_node_acl *acl,
 	int force)
@@ -469,7 +469,7 @@ int core_tpg_del_initiator_node_acl(
 	wait_for_completion(&acl->acl_free_comp);
 
 	core_tpg_wait_for_nacl_pr_ref(acl);
-	core_clear_initiator_node_from_tpg(acl, tpg);
+	target_clear_initiator_node_from_tpg(acl, tpg);
 	core_free_device_list_for_node(acl, tpg);
 
 	pr_debug("%s_TPG[%hu] - Deleted ACL with TCQ Depth: %d for %s"
@@ -479,13 +479,13 @@ int core_tpg_del_initiator_node_acl(
 
 	return 0;
 }
-EXPORT_SYMBOL(core_tpg_del_initiator_node_acl);
+EXPORT_SYMBOL(target_del_initiator_node_acl);
 
-/*	core_tpg_set_initiator_node_queue_depth():
+/*	target_set_initiator_node_queue_depth():
  *
  *
  */
-int core_tpg_set_initiator_node_queue_depth(
+int target_set_initiator_node_queue_depth(
 	struct se_portal_group *tpg,
 	unsigned char *initiatorname,
 	u32 queue_depth,
@@ -497,7 +497,7 @@ int core_tpg_set_initiator_node_queue_depth(
 	int dynamic_acl = 0;
 
 	mutex_lock(&tpg->acl_node_mutex);
-	acl = __core_tpg_get_initiator_node_acl(tpg, initiatorname);
+	acl = __target_get_initiator_node_acl(tpg, initiatorname);
 	if (!acl) {
 		pr_err("Access Control List entry for %s Initiator"
 			" Node %s does not exists for TPG %hu, ignoring"
@@ -590,15 +590,15 @@ int core_tpg_set_initiator_node_queue_depth(
 
 	return 0;
 }
-EXPORT_SYMBOL(core_tpg_set_initiator_node_queue_depth);
+EXPORT_SYMBOL(target_set_initiator_node_queue_depth);
 
-/*	core_tpg_set_initiator_node_tag():
+/*	target_set_initiator_node_tag():
  *
  *	Initiator nodeacl tags are not used internally, but may be used by
  *	userspace to emulate aliases or groups.
  *	Returns length of newly-set tag or -EINVAL.
  */
-int core_tpg_set_initiator_node_tag(
+int target_set_initiator_node_tag(
 	struct se_portal_group *tpg,
 	struct se_node_acl *acl,
 	const char *new_tag)
@@ -613,7 +613,7 @@ int core_tpg_set_initiator_node_tag(
 
 	return snprintf(acl->acl_tag, MAX_ACL_TAG_SIZE, "%s", new_tag);
 }
-EXPORT_SYMBOL(core_tpg_set_initiator_node_tag);
+EXPORT_SYMBOL(target_set_initiator_node_tag);
 
 static void core_tpg_lun_ref_release(struct percpu_ref *ref)
 {
diff --git a/drivers/target/tcm_fc/tfc_conf.c b/drivers/target/tcm_fc/tfc_conf.c
index 72187c6..85dde28 100644
--- a/drivers/target/tcm_fc/tfc_conf.c
+++ b/drivers/target/tcm_fc/tfc_conf.c
@@ -220,8 +220,8 @@ static struct se_node_acl *ft_add_acl(
 	acl->node_auth.port_name = wwpn;
 
 	q_depth = 32;		/* XXX bogus default - get from tpg? */
-	return core_tpg_add_initiator_node_acl(&tpg->se_tpg,
-				&acl->se_node_acl, name, q_depth);
+	return target_add_initiator_node_acl(&tpg->se_tpg,
+					     &acl->se_node_acl, name, q_depth);
 }
 
 static void ft_del_acl(struct se_node_acl *se_acl)
@@ -238,7 +238,7 @@ static void ft_del_acl(struct se_node_acl *se_acl)
 	pr_debug("del acl %p se_acl %p tpg %p se_tpg %p\n",
 		    acl, se_acl, tpg, &tpg->se_tpg);
 
-	core_tpg_del_initiator_node_acl(&tpg->se_tpg, se_acl, 1);
+	target_del_initiator_node_acl(&tpg->se_tpg, se_acl, 1);
 	kfree(acl);
 }
 
diff --git a/drivers/usb/gadget/legacy/tcm_usb_gadget.c b/drivers/usb/gadget/legacy/tcm_usb_gadget.c
index 6e0a019..81a8b07 100644
--- a/drivers/usb/gadget/legacy/tcm_usb_gadget.c
+++ b/drivers/usb/gadget/legacy/tcm_usb_gadget.c
@@ -1509,11 +1509,11 @@ static struct se_node_acl *usbg_make_nodeacl(
 
 	nexus_depth = 1;
 	/*
-	 * se_nacl_new may be released by core_tpg_add_initiator_node_acl()
+	 * se_nacl_new may be released by target_add_initiator_node_acl()
 	 * when converting a NodeACL from demo mode -> explict
 	 */
-	se_nacl = core_tpg_add_initiator_node_acl(se_tpg, se_nacl_new,
-				name, nexus_depth);
+	se_nacl = target_add_initiator_node_acl(se_tpg, se_nacl_new,
+						name, nexus_depth);
 	if (IS_ERR(se_nacl)) {
 		usbg_release_fabric_acl(se_tpg, se_nacl_new);
 		return se_nacl;
@@ -1531,7 +1531,7 @@ static void usbg_drop_nodeacl(struct se_node_acl *se_acl)
 {
 	struct usbg_nacl *nacl = container_of(se_acl,
 				struct usbg_nacl, se_node_acl);
-	core_tpg_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
+	target_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
 	kfree(nacl);
 }
 
@@ -1732,10 +1732,10 @@ static int tcm_usbg_make_nexus(struct usbg_tpg *tpg, char *name)
 	 * struct se_node_acl for the tcm_vhost struct se_portal_group with
 	 * the SCSI Initiator port name of the passed configfs group 'name'.
 	 */
-	tv_nexus->tvn_se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
-			se_tpg, name);
+	tv_nexus->tvn_se_sess->se_node_acl = target_check_initiator_node_acl(
+							se_tpg, name);
 	if (!tv_nexus->tvn_se_sess->se_node_acl) {
-		pr_debug("core_tpg_check_initiator_node_acl() failed"
+		pr_debug("target_check_initiator_node_acl() failed"
 				" for %s\n", name);
 		goto err_session;
 	}
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 71df240..7e745fd 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -1834,11 +1834,11 @@ vhost_scsi_make_nodeacl(struct se_portal_group *se_tpg,
 
 	nexus_depth = 1;
 	/*
-	 * se_nacl_new may be released by core_tpg_add_initiator_node_acl()
+	 * se_nacl_new may be released by target_add_initiator_node_acl()
 	 * when converting a NodeACL from demo mode -> explict
 	 */
-	se_nacl = core_tpg_add_initiator_node_acl(se_tpg, se_nacl_new,
-				name, nexus_depth);
+	se_nacl = target_add_initiator_node_acl(se_tpg, se_nacl_new,
+						name, nexus_depth);
 	if (IS_ERR(se_nacl)) {
 		vhost_scsi_release_fabric_acl(se_tpg, se_nacl_new);
 		return se_nacl;
@@ -1856,7 +1856,7 @@ static void vhost_scsi_drop_nodeacl(struct se_node_acl *se_acl)
 {
 	struct vhost_scsi_nacl *nacl = container_of(se_acl,
 				struct vhost_scsi_nacl, se_node_acl);
-	core_tpg_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
+	target_del_initiator_node_acl(se_acl->se_tpg, se_acl, 1);
 	kfree(nacl);
 }
 
@@ -1947,11 +1947,11 @@ static int vhost_scsi_make_nexus(struct vhost_scsi_tpg *tpg,
 	 * struct se_node_acl for the vhost_scsi struct se_portal_group with
 	 * the SCSI Initiator port name of the passed configfs group 'name'.
 	 */
-	tv_nexus->tvn_se_sess->se_node_acl = core_tpg_check_initiator_node_acl(
+	tv_nexus->tvn_se_sess->se_node_acl = target_check_initiator_node_acl(
 				se_tpg, (unsigned char *)name);
 	if (!tv_nexus->tvn_se_sess->se_node_acl) {
 		mutex_unlock(&tpg->tv_tpg_mutex);
-		pr_debug("core_tpg_check_initiator_node_acl() failed"
+		pr_debug("target_check_initiator_node_acl() failed"
 				" for %s\n", name);
 		goto out;
 	}
diff --git a/include/target/target_core_fabric.h b/include/target/target_core_fabric.h
index 578fc93..e33c67b 100644
--- a/include/target/target_core_fabric.h
+++ b/include/target/target_core_fabric.h
@@ -148,17 +148,17 @@ void	transport_generic_request_failure(struct se_cmd *, sense_reason_t);
 void	__target_execute_cmd(struct se_cmd *);
 int	transport_lookup_tmr_lun(struct se_cmd *, u32);
 
-struct se_node_acl *core_tpg_get_initiator_node_acl(struct se_portal_group *tpg,
+struct se_node_acl *target_get_initiator_node_acl(struct se_portal_group *tpg,
 		unsigned char *);
-struct se_node_acl *core_tpg_check_initiator_node_acl(struct se_portal_group *,
+struct se_node_acl *target_check_initiator_node_acl(struct se_portal_group *,
 		unsigned char *);
-struct se_node_acl *core_tpg_add_initiator_node_acl(struct se_portal_group *,
+struct se_node_acl *target_add_initiator_node_acl(struct se_portal_group *,
 		struct se_node_acl *, const char *, u32);
-int	core_tpg_del_initiator_node_acl(struct se_portal_group *,
+int	target_del_initiator_node_acl(struct se_portal_group *,
 		struct se_node_acl *, int);
-int	core_tpg_set_initiator_node_queue_depth(struct se_portal_group *,
+int	target_set_initiator_node_queue_depth(struct se_portal_group *,
 		unsigned char *, u32, int);
-int	core_tpg_set_initiator_node_tag(struct se_portal_group *,
+int	target_set_initiator_node_tag(struct se_portal_group *,
 		struct se_node_acl *, const char *);
 int	core_tpg_register(struct target_core_fabric_ops *, struct se_wwn *,
 		struct se_portal_group *, void *, int);
-- 
1.9.1

[RFC 19/22] target: Rename TPG register/deregister to target_* prefix

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This includes:

  - target_register_tpg
  - target_deregister_tpg

This includes updates for loopback, tcm_fc, iscsi/iser-target,
ib_srpt, sbp-target, usb-gadget, tcm_qla2xxx, vhost-scsi and
xen-scsiback.

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/infiniband/ulp/srpt/ib_srpt.c        |  7 ++++---
 drivers/scsi/qla2xxx/tcm_qla2xxx.c           | 10 +++++-----
 drivers/target/iscsi/iscsi_target_configfs.c |  4 ++--
 drivers/target/iscsi/iscsi_target_tpg.c      |  8 ++++----
 drivers/target/loopback/tcm_loop.c           |  8 ++++----
 drivers/target/sbp/sbp_target.c              |  8 ++++----
 drivers/target/target_core_tpg.c             |  8 ++++----
 drivers/target/tcm_fc/tfc_conf.c             |  6 +++---
 drivers/usb/gadget/legacy/tcm_usb_gadget.c   |  8 ++++----
 drivers/vhost/scsi.c                         |  6 +++---
 drivers/xen/xen-scsiback.c                   |  6 +++---
 include/target/target_core_fabric.h          |  4 ++--
 12 files changed, 42 insertions(+), 41 deletions(-)

diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index c2f833d..f4f0853 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -3851,8 +3851,9 @@ static struct se_portal_group *srpt_make_tpg(struct se_wwn *wwn,
 	int res;
 
 	/* Initialize sport->port_wwn and sport->port_tpg_1 */
-	res = core_tpg_register(&srpt_target->tf_ops, &sport->port_wwn,
-			&sport->port_tpg_1, sport, TRANSPORT_TPG_TYPE_NORMAL);
+	res = target_register_tpg(&srpt_target->tf_ops, &sport->port_wwn,
+				  &sport->port_tpg_1, sport,
+				  TRANSPORT_TPG_TYPE_NORMAL);
 	if (res)
 		return ERR_PTR(res);
 
@@ -3869,7 +3870,7 @@ static void srpt_drop_tpg(struct se_portal_group *tpg)
 				struct srpt_port, port_tpg_1);
 
 	sport->enabled = false;
-	core_tpg_deregister(&sport->port_tpg_1);
+	target_deregister_tpg(&sport->port_tpg_1);
 }
 
 /**
diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
index e29515b..e9f67ef 100644
--- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c
+++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c
@@ -1124,8 +1124,8 @@ static struct se_portal_group *tcm_qla2xxx_make_tpg(
 	tpg->tpg_attrib.cache_dynamic_acls = 1;
 	tpg->tpg_attrib.demo_mode_login_only = 1;
 
-	ret = core_tpg_register(&tcm_qla2xxx_fabric_configfs->tf_ops, wwn,
-				&tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&tcm_qla2xxx_fabric_configfs->tf_ops, wwn,
+				  &tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0) {
 		kfree(tpg);
 		return NULL;
@@ -1149,7 +1149,7 @@ static void tcm_qla2xxx_drop_tpg(struct se_portal_group *se_tpg)
 	if (vha->vha_tgt.qla_tgt && !vha->vha_tgt.qla_tgt->tgt_stop)
 		qlt_stop_phase1(vha->vha_tgt.qla_tgt);
 
-	core_tpg_deregister(se_tpg);
+	target_deregister_tpg(se_tpg);
 	/*
 	 * Clear local TPG=1 pointer for non NPIV mode.
 	 */
@@ -1244,8 +1244,8 @@ static struct se_portal_group *tcm_qla2xxx_npiv_make_tpg(
 	tpg->tpg_attrib.cache_dynamic_acls = 1;
 	tpg->tpg_attrib.demo_mode_login_only = 1;
 
-	ret = core_tpg_register(&tcm_qla2xxx_npiv_fabric_configfs->tf_ops, wwn,
-				&tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&tcm_qla2xxx_npiv_fabric_configfs->tf_ops, wwn,
+				  &tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0) {
 		kfree(tpg);
 		return NULL;
diff --git a/drivers/target/iscsi/iscsi_target_configfs.c b/drivers/target/iscsi/iscsi_target_configfs.c
index 167782a..4cafb76 100644
--- a/drivers/target/iscsi/iscsi_target_configfs.c
+++ b/drivers/target/iscsi/iscsi_target_configfs.c
@@ -1449,7 +1449,7 @@ static struct se_portal_group *lio_target_tiqn_addtpg(
 	if (!tpg)
 		return NULL;
 
-	ret = core_tpg_register(
+	ret = target_register_tpg(
 			&lio_target_fabric_configfs->tf_ops,
 			wwn, &tpg->tpg_se_tpg, tpg,
 			TRANSPORT_TPG_TYPE_NORMAL);
@@ -1465,7 +1465,7 @@ static struct se_portal_group *lio_target_tiqn_addtpg(
 			name);
 	return &tpg->tpg_se_tpg;
 out:
-	core_tpg_deregister(&tpg->tpg_se_tpg);
+	target_deregister_tpg(&tpg->tpg_se_tpg);
 	kfree(tpg);
 	return NULL;
 }
diff --git a/drivers/target/iscsi/iscsi_target_tpg.c b/drivers/target/iscsi/iscsi_target_tpg.c
index 4ebc20b..e1b93582 100644
--- a/drivers/target/iscsi/iscsi_target_tpg.c
+++ b/drivers/target/iscsi/iscsi_target_tpg.c
@@ -68,7 +68,7 @@ int iscsit_load_discovery_tpg(void)
 		return -1;
 	}
 
-	ret = core_tpg_register(
+	ret = target_register_tpg(
 			&lio_target_fabric_configfs->tf_ops,
 			NULL, &tpg->tpg_se_tpg, tpg,
 			TRANSPORT_TPG_TYPE_DISCOVERY);
@@ -107,7 +107,7 @@ int iscsit_load_discovery_tpg(void)
 	return 0;
 out:
 	if (tpg->sid == 1)
-		core_tpg_deregister(&tpg->tpg_se_tpg);
+		target_deregister_tpg(&tpg->tpg_se_tpg);
 	kfree(tpg);
 	return -1;
 }
@@ -119,7 +119,7 @@ void iscsit_release_discovery_tpg(void)
 	if (!tpg)
 		return;
 
-	core_tpg_deregister(&tpg->tpg_se_tpg);
+	target_deregister_tpg(&tpg->tpg_se_tpg);
 
 	kfree(tpg);
 	iscsit_global->discovery_tpg = NULL;
@@ -289,7 +289,7 @@ int iscsit_tpg_del_portal_group(
 		tpg->param_list = NULL;
 	}
 
-	core_tpg_deregister(&tpg->tpg_se_tpg);
+	target_deregister_tpg(&tpg->tpg_se_tpg);
 
 	spin_lock(&tpg->tpg_state_lock);
 	tpg->tpg_state = TPG_STATE_FREE;
diff --git a/drivers/target/loopback/tcm_loop.c b/drivers/target/loopback/tcm_loop.c
index b271624..a05b86c 100644
--- a/drivers/target/loopback/tcm_loop.c
+++ b/drivers/target/loopback/tcm_loop.c
@@ -1234,9 +1234,9 @@ static struct se_portal_group *tcm_loop_make_naa_tpg(
 	/*
 	 * Register the tl_tpg as a emulated SAS TCM Target Endpoint
 	 */
-	ret = core_tpg_register(&tcm_loop_fabric_configfs->tf_ops,
-			wwn, &tl_tpg->tl_se_tpg, tl_tpg,
-			TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&tcm_loop_fabric_configfs->tf_ops,
+				  wwn, &tl_tpg->tl_se_tpg, tl_tpg,
+				  TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0)
 		return ERR_PTR(-ENOMEM);
 
@@ -1265,7 +1265,7 @@ static void tcm_loop_drop_naa_tpg(
 	/*
 	 * Deregister the tl_tpg as a emulated SAS TCM Target Endpoint
 	 */
-	core_tpg_deregister(se_tpg);
+	target_deregister_tpg(se_tpg);
 
 	tl_tpg->tl_hba = NULL;
 	tl_tpg->tl_tpgt = 0;
diff --git a/drivers/target/sbp/sbp_target.c b/drivers/target/sbp/sbp_target.c
index f433d56..094d040 100644
--- a/drivers/target/sbp/sbp_target.c
+++ b/drivers/target/sbp/sbp_target.c
@@ -2211,9 +2211,9 @@ static struct se_portal_group *sbp_make_tpg(
 		goto out_free_tpg;
 	}
 
-	ret = core_tpg_register(&sbp_fabric_configfs->tf_ops, wwn,
-			&tpg->se_tpg, (void *)tpg,
-			TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&sbp_fabric_configfs->tf_ops, wwn,
+				  &tpg->se_tpg, (void *)tpg,
+				  TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0)
 		goto out_unreg_mgt_agt;
 
@@ -2232,7 +2232,7 @@ static void sbp_drop_tpg(struct se_portal_group *se_tpg)
 	struct sbp_tpg *tpg = container_of(se_tpg, struct sbp_tpg, se_tpg);
 	struct sbp_tport *tport = tpg->tport;
 
-	core_tpg_deregister(se_tpg);
+	target_deregister_tpg(se_tpg);
 	sbp_management_agent_unregister(tport->mgt_agt);
 	tport->tpg = NULL;
 	kfree(tpg);
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 86dda22..513e6bd 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -644,7 +644,7 @@ static int core_tpg_setup_virtual_lun0(struct se_portal_group *se_tpg)
 	return 0;
 }
 
-int core_tpg_register(
+int target_register_tpg(
 	struct target_core_fabric_ops *tfo,
 	struct se_wwn *se_wwn,
 	struct se_portal_group *se_tpg,
@@ -705,9 +705,9 @@ int core_tpg_register(
 
 	return 0;
 }
-EXPORT_SYMBOL(core_tpg_register);
+EXPORT_SYMBOL(target_register_tpg);
 
-int core_tpg_deregister(struct se_portal_group *se_tpg)
+int target_deregister_tpg(struct se_portal_group *se_tpg)
 {
 	struct se_node_acl *nacl, *nacl_tmp;
 	LIST_HEAD(node_list);
@@ -750,7 +750,7 @@ int core_tpg_deregister(struct se_portal_group *se_tpg)
 	array_free(se_tpg->tpg_lun_list, TRANSPORT_MAX_LUNS_PER_TPG);
 	return 0;
 }
-EXPORT_SYMBOL(core_tpg_deregister);
+EXPORT_SYMBOL(target_deregister_tpg);
 
 struct se_lun *core_tpg_alloc_lun(
 	struct se_portal_group *tpg,
diff --git a/drivers/target/tcm_fc/tfc_conf.c b/drivers/target/tcm_fc/tfc_conf.c
index 85dde28..9aa67ce 100644
--- a/drivers/target/tcm_fc/tfc_conf.c
+++ b/drivers/target/tcm_fc/tfc_conf.c
@@ -337,8 +337,8 @@ static struct se_portal_group *ft_add_tpg(
 		return NULL;
 	}
 
-	ret = core_tpg_register(&ft_configfs->tf_ops, wwn, &tpg->se_tpg,
-				tpg, TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&ft_configfs->tf_ops, wwn, &tpg->se_tpg,
+				  tpg, TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0) {
 		destroy_workqueue(wq);
 		kfree(tpg);
@@ -374,7 +374,7 @@ static void ft_del_tpg(struct se_portal_group *se_tpg)
 	}
 	mutex_unlock(&ft_lport_lock);
 
-	core_tpg_deregister(se_tpg);
+	target_deregister_tpg(se_tpg);
 	kfree(tpg);
 }
 
diff --git a/drivers/usb/gadget/legacy/tcm_usb_gadget.c b/drivers/usb/gadget/legacy/tcm_usb_gadget.c
index 81a8b07..3db4b44 100644
--- a/drivers/usb/gadget/legacy/tcm_usb_gadget.c
+++ b/drivers/usb/gadget/legacy/tcm_usb_gadget.c
@@ -1572,9 +1572,9 @@ static struct se_portal_group *usbg_make_tpg(
 	tpg->tport = tport;
 	tpg->tport_tpgt = tpgt;
 
-	ret = core_tpg_register(&usbg_fabric_configfs->tf_ops, wwn,
-				&tpg->se_tpg, tpg,
-				TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&usbg_fabric_configfs->tf_ops, wwn,
+				  &tpg->se_tpg, tpg,
+				  TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0) {
 		destroy_workqueue(tpg->workqueue);
 		kfree(tpg);
@@ -1589,7 +1589,7 @@ static void usbg_drop_tpg(struct se_portal_group *se_tpg)
 	struct usbg_tpg *tpg = container_of(se_tpg,
 				struct usbg_tpg, se_tpg);
 
-	core_tpg_deregister(se_tpg);
+	target_deregister_tpg(se_tpg);
 	destroy_workqueue(tpg->workqueue);
 	kfree(tpg);
 	the_only_tpg_I_currently_have = NULL;
diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c
index 7e745fd..4122073 100644
--- a/drivers/vhost/scsi.c
+++ b/drivers/vhost/scsi.c
@@ -2155,8 +2155,8 @@ vhost_scsi_make_tpg(struct se_wwn *wwn,
 	tpg->tport = tport;
 	tpg->tport_tpgt = tpgt;
 
-	ret = core_tpg_register(&vhost_scsi_fabric_configfs->tf_ops, wwn,
-				&tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&vhost_scsi_fabric_configfs->tf_ops, wwn,
+				  &tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0) {
 		kfree(tpg);
 		return NULL;
@@ -2183,7 +2183,7 @@ static void vhost_scsi_drop_tpg(struct se_portal_group *se_tpg)
 	/*
 	 * Deregister the se_tpg from TCM..
 	 */
-	core_tpg_deregister(se_tpg);
+	target_deregister_tpg(se_tpg);
 	kfree(tpg);
 }
 
diff --git a/drivers/xen/xen-scsiback.c b/drivers/xen/xen-scsiback.c
index 8511277..00772de 100644
--- a/drivers/xen/xen-scsiback.c
+++ b/drivers/xen/xen-scsiback.c
@@ -1898,8 +1898,8 @@ scsiback_make_tpg(struct se_wwn *wwn,
 	tpg->tport = tport;
 	tpg->tport_tpgt = tpgt;
 
-	ret = core_tpg_register(&scsiback_fabric_configfs->tf_ops, wwn,
-				&tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
+	ret = target_register_tpg(&scsiback_fabric_configfs->tf_ops, wwn,
+				  &tpg->se_tpg, tpg, TRANSPORT_TPG_TYPE_NORMAL);
 	if (ret < 0) {
 		kfree(tpg);
 		return NULL;
@@ -1926,7 +1926,7 @@ static void scsiback_drop_tpg(struct se_portal_group *se_tpg)
 	/*
 	 * Deregister the se_tpg from TCM..
 	 */
-	core_tpg_deregister(se_tpg);
+	target_deregister_tpg(se_tpg);
 	kfree(tpg);
 }
 
diff --git a/include/target/target_core_fabric.h b/include/target/target_core_fabric.h
index e33c67b..9b4dc7a 100644
--- a/include/target/target_core_fabric.h
+++ b/include/target/target_core_fabric.h
@@ -160,9 +160,9 @@ int	target_set_initiator_node_queue_depth(struct se_portal_group *,
 		unsigned char *, u32, int);
 int	target_set_initiator_node_tag(struct se_portal_group *,
 		struct se_node_acl *, const char *);
-int	core_tpg_register(struct target_core_fabric_ops *, struct se_wwn *,
+int	target_register_tpg(struct target_core_fabric_ops *, struct se_wwn *,
 		struct se_portal_group *, void *, int);
-int	core_tpg_deregister(struct se_portal_group *);
+int	target_deregister_tpg(struct se_portal_group *);
 
 /* SAS helpers */
 u8	sas_get_fabric_proto_ident(struct se_portal_group *);
-- 
1.9.1

[RFC 20/22] target: Rename LUN lookup/add/remove to target_* prefix

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This includes:

  - target_add_lun
  - target_remove_lun
  - target_lookup_cmd_lun
  - target_lookup_tmr_lun
  - target_clear_lun_from_tpg

Also, update fabric drivers for loopback, iscsi/iser-target,
xen-scsiback.

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/iscsi/iscsi_target.c    |  8 ++++----
 drivers/target/loopback/tcm_loop.c     |  2 +-
 drivers/target/target_core_device.c    | 16 ++++++++--------
 drivers/target/target_core_internal.h  |  6 +++---
 drivers/target/target_core_tpg.c       | 10 +++++-----
 drivers/target/target_core_transport.c |  4 ++--
 drivers/xen/xen-scsiback.c             |  2 +-
 include/target/target_core_fabric.h    |  4 ++--
 8 files changed, 26 insertions(+), 26 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 75e24ef..9a5d56a 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -995,8 +995,8 @@ int iscsit_setup_scsi_cmd(struct iscsi_conn *conn, struct iscsi_cmd *cmd,
 
 	target_get_sess_cmd(conn->sess->se_sess, &cmd->se_cmd, true);
 
-	cmd->sense_reason = transport_lookup_cmd_lun(&cmd->se_cmd,
-						     scsilun_to_int(&hdr->lun));
+	cmd->sense_reason = target_lookup_cmd_lun(&cmd->se_cmd,
+						  scsilun_to_int(&hdr->lun));
 	if (cmd->sense_reason)
 		goto attach_cmd;
 
@@ -1857,8 +1857,8 @@ iscsit_handle_task_mgt_cmd(struct iscsi_conn *conn, struct iscsi_cmd *cmd,
 	 * Locate the struct se_lun for all TMRs not related to ERL=2 TASK_REASSIGN
 	 */
 	if (function != ISCSI_TM_FUNC_TASK_REASSIGN) {
-		ret = transport_lookup_tmr_lun(&cmd->se_cmd,
-					       scsilun_to_int(&hdr->lun));
+		ret = target_lookup_tmr_lun(&cmd->se_cmd,
+					    scsilun_to_int(&hdr->lun));
 		if (ret < 0) {
 			se_tmr->response = ISCSI_TMF_RSP_NO_LUN;
 			goto attach;
diff --git a/drivers/target/loopback/tcm_loop.c b/drivers/target/loopback/tcm_loop.c
index a05b86c..1510380 100644
--- a/drivers/target/loopback/tcm_loop.c
+++ b/drivers/target/loopback/tcm_loop.c
@@ -271,7 +271,7 @@ static int tcm_loop_issue_tmr(struct tcm_loop_tpg *tl_tpg,
 	/*
 	 * Locate the underlying TCM struct se_lun
 	 */
-	if (transport_lookup_tmr_lun(se_cmd, lun) < 0) {
+	if (target_lookup_tmr_lun(se_cmd, lun) < 0) {
 		ret = TMR_LUN_DOES_NOT_EXIST;
 		goto release;
 	}
diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 80d1b04..3647e7c 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -55,7 +55,7 @@ static struct se_hba *lun0_hba;
 struct se_device *g_lun0_dev;
 
 sense_reason_t
-transport_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
+target_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
 {
 	struct se_lun *se_lun = NULL;
 	struct se_session *se_sess = se_cmd->se_sess;
@@ -147,9 +147,9 @@ check_lun:
 
 	return 0;
 }
-EXPORT_SYMBOL(transport_lookup_cmd_lun);
+EXPORT_SYMBOL(target_lookup_cmd_lun);
 
-int transport_lookup_tmr_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
+int target_lookup_tmr_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
 {
 	struct se_dev_entry *deve;
 	struct se_lun *se_lun = NULL;
@@ -200,7 +200,7 @@ check_lun:
 
 	return 0;
 }
-EXPORT_SYMBOL(transport_lookup_tmr_lun);
+EXPORT_SYMBOL(target_lookup_tmr_lun);
 
 /*
  * This function is called from core_scsi3_emulate_pro_register_and_move()
@@ -456,11 +456,11 @@ int core_disable_device_list_for_node(
 	return 0;
 }
 
-/*      core_clear_lun_from_tpg():
+/*      target_clear_lun_from_tpg():
  *
  *
  */
-void core_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg)
+void target_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg)
 {
 	struct se_node_acl *nacl;
 	struct se_dev_entry *deve;
@@ -1211,7 +1211,7 @@ struct se_lun *core_dev_add_lun(
 	if (IS_ERR(lun))
 		return lun;
 
-	rc = core_tpg_add_lun(tpg, lun,
+	rc = target_add_lun(tpg, lun,
 				TRANSPORT_LUNFLAGS_READ_WRITE, dev);
 	if (rc < 0)
 		return ERR_PTR(rc);
@@ -1254,7 +1254,7 @@ void core_dev_del_lun(
 		tpg->se_tpg_tfo->tpg_get_tag(tpg), lun->unpacked_lun,
 		tpg->se_tpg_tfo->get_fabric_name());
 
-	core_tpg_remove_lun(tpg, lun);
+	target_remove_lun(tpg, lun);
 }
 
 struct se_lun *core_get_lun_from_tpg(struct se_portal_group *tpg, u32 unpacked_lun)
diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h
index 574b396..70190e2 100644
--- a/drivers/target/target_core_internal.h
+++ b/drivers/target/target_core_internal.h
@@ -13,7 +13,7 @@ int	core_enable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
 		u32, u32, struct se_node_acl *, struct se_portal_group *);
 int	core_disable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
 		u32, u32, struct se_node_acl *, struct se_portal_group *);
-void	core_clear_lun_from_tpg(struct se_lun *, struct se_portal_group *);
+void	target_clear_lun_from_tpg(struct se_lun *, struct se_portal_group *);
 int	core_dev_export(struct se_device *, struct se_portal_group *,
 		struct se_lun *);
 void	core_dev_unexport(struct se_device *, struct se_portal_group *,
@@ -53,9 +53,9 @@ struct se_node_acl *__target_get_initiator_node_acl(struct se_portal_group *tpg,
 void	core_tpg_add_node_to_devs(struct se_node_acl *, struct se_portal_group *);
 void	core_tpg_wait_for_nacl_pr_ref(struct se_node_acl *);
 struct se_lun *core_tpg_alloc_lun(struct se_portal_group *, u32);
-int	core_tpg_add_lun(struct se_portal_group *, struct se_lun *,
+int	target_add_lun(struct se_portal_group *, struct se_lun *,
 		u32, struct se_device *);
-void core_tpg_remove_lun(struct se_portal_group *, struct se_lun *);
+void target_remove_lun(struct se_portal_group *, struct se_lun *);
 
 /* target_core_transport.c */
 extern struct kmem_cache *se_tmr_req_cache;
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 513e6bd..9d31434 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -637,7 +637,7 @@ static int core_tpg_setup_virtual_lun0(struct se_portal_group *se_tpg)
 	spin_lock_init(&lun->lun_sep_lock);
 	init_completion(&lun->lun_ref_comp);
 
-	ret = core_tpg_add_lun(se_tpg, lun, lun_access, dev);
+	ret = target_add_lun(se_tpg, lun, lun_access, dev);
 	if (ret < 0)
 		return ret;
 
@@ -744,7 +744,7 @@ int target_deregister_tpg(struct se_portal_group *se_tpg)
 	}
 
 	if (se_tpg->se_tpg_type == TRANSPORT_TPG_TYPE_NORMAL)
-		core_tpg_remove_lun(se_tpg, &se_tpg->tpg_virt_lun0);
+		target_remove_lun(se_tpg, &se_tpg->tpg_virt_lun0);
 
 	se_tpg->se_tpg_fabric_ptr = NULL;
 	array_free(se_tpg->tpg_lun_list, TRANSPORT_MAX_LUNS_PER_TPG);
@@ -782,7 +782,7 @@ struct se_lun *core_tpg_alloc_lun(
 	return lun;
 }
 
-int core_tpg_add_lun(
+int target_add_lun(
 	struct se_portal_group *tpg,
 	struct se_lun *lun,
 	u32 lun_access,
@@ -809,11 +809,11 @@ int core_tpg_add_lun(
 	return 0;
 }
 
-void core_tpg_remove_lun(
+void target_remove_lun(
 	struct se_portal_group *tpg,
 	struct se_lun *lun)
 {
-	core_clear_lun_from_tpg(lun, tpg);
+	target_clear_lun_from_tpg(lun, tpg);
 	transport_clear_lun_ref(lun);
 
 	core_dev_unexport(lun->lun_se_dev, tpg, lun);
diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index dfd78dd..86d4aeb 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -1398,7 +1398,7 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess
 	/*
 	 * Locate se_lun pointer and attach it to struct se_cmd
 	 */
-	rc = transport_lookup_cmd_lun(se_cmd, unpacked_lun);
+	rc = target_lookup_cmd_lun(se_cmd, unpacked_lun);
 	if (rc) {
 		transport_send_check_condition_and_sense(se_cmd, rc, 0);
 		target_put_sess_cmd(se_sess, se_cmd);
@@ -1558,7 +1558,7 @@ int target_submit_tmr(struct se_cmd *se_cmd, struct se_session *se_sess,
 		return ret;
 	}
 
-	ret = transport_lookup_tmr_lun(se_cmd, unpacked_lun);
+	ret = target_lookup_tmr_lun(se_cmd, unpacked_lun);
 	if (ret) {
 		/*
 		 * For callback during failure handling, push this work off
diff --git a/drivers/xen/xen-scsiback.c b/drivers/xen/xen-scsiback.c
index 00772de..a2062b6 100644
--- a/drivers/xen/xen-scsiback.c
+++ b/drivers/xen/xen-scsiback.c
@@ -616,7 +616,7 @@ static void scsiback_device_action(struct vscsibk_pend *pending_req,
 
 	se_cmd->se_tmr_req->ref_task_tag = tag;
 
-	if (transport_lookup_tmr_lun(se_cmd, pending_req->v2p->lun) < 0)
+	if (target_lookup_tmr_lun(se_cmd, pending_req->v2p->lun) < 0)
 		goto out;
 
 	transport_generic_handle_tmr(se_cmd);
diff --git a/include/target/target_core_fabric.h b/include/target/target_core_fabric.h
index 9b4dc7a..e06613d 100644
--- a/include/target/target_core_fabric.h
+++ b/include/target/target_core_fabric.h
@@ -111,7 +111,7 @@ void	transport_deregister_session(struct se_session *);
 
 void	transport_init_se_cmd(struct se_cmd *, struct target_core_fabric_ops *,
 		struct se_session *, u32, int, int, unsigned char *);
-sense_reason_t transport_lookup_cmd_lun(struct se_cmd *, u32);
+sense_reason_t target_lookup_cmd_lun(struct se_cmd *, u32);
 sense_reason_t target_setup_cmd_from_cdb(struct se_cmd *, unsigned char *);
 int	target_submit_cmd_map_sgls(struct se_cmd *, struct se_session *,
 		unsigned char *, unsigned char *, u32, u32, int, int, int,
@@ -146,7 +146,7 @@ void	core_tmr_release_req(struct se_tmr_req *);
 int	transport_generic_handle_tmr(struct se_cmd *);
 void	transport_generic_request_failure(struct se_cmd *, sense_reason_t);
 void	__target_execute_cmd(struct se_cmd *);
-int	transport_lookup_tmr_lun(struct se_cmd *, u32);
+int	target_lookup_tmr_lun(struct se_cmd *, u32);
 
 struct se_node_acl *target_get_initiator_node_acl(struct se_portal_group *tpg,
 		unsigned char *);
-- 
1.9.1

[RFC 21/22] target: Rename se_node_acl->lun_entry_hlist to target_* prefix

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This includes:

  - target_add_node_to_devs
  - target_get_deve_from_rtpi
  - target_create_device_list_for_node
  - target_free_device_list_for_node
  - target_enable_device_list_for_node
  - target_disable_device_list_for_node

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c    | 22 +++++++++++-----------
 drivers/target/target_core_internal.h  | 12 ++++++------
 drivers/target/target_core_pr.c        |  6 +++---
 drivers/target/target_core_tpg.c       | 28 ++++++++++++++--------------
 drivers/target/target_core_transport.c |  2 +-
 5 files changed, 35 insertions(+), 35 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 3647e7c..5b225d8 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -207,7 +207,7 @@ EXPORT_SYMBOL(target_lookup_tmr_lun);
  * and core_scsi3_decode_spec_i_port(), and will increment &deve->pr_ref_count
  * when a matching rtpi is found.
  */
-struct se_dev_entry *core_get_se_deve_from_rtpi(
+struct se_dev_entry *target_get_deve_from_rtpi(
 	struct se_node_acl *nacl,
 	u16 rtpi)
 {
@@ -257,7 +257,7 @@ struct se_dev_entry *core_get_se_deve_from_rtpi(
 	return NULL;
 }
 
-int core_free_device_list_for_node(
+int target_free_device_list_for_node(
 	struct se_node_acl *nacl,
 	struct se_portal_group *tpg)
 {
@@ -287,7 +287,7 @@ int core_free_device_list_for_node(
 		mapped_lun = deve->mapped_lun;
 		rcu_read_unlock();
 
-		core_disable_device_list_for_node(lun, NULL, mapped_lun,
+		target_disable_device_list_for_node(lun, NULL, mapped_lun,
 					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 	}
 
@@ -325,11 +325,11 @@ static void target_pr_ref_release(struct percpu_ref *ref)
 	complete(&deve->pr_comp);
 }
 
-/*      core_enable_device_list_for_node():
+/*      target_enable_device_list_for_node():
  *
  *
  */
-int core_enable_device_list_for_node(
+int target_enable_device_list_for_node(
 	struct se_lun *lun,
 	struct se_lun_acl *lun_acl,
 	u32 mapped_lun,
@@ -394,11 +394,11 @@ int core_enable_device_list_for_node(
 	return 0;
 }
 
-/*      core_disable_device_list_for_node():
+/*      target_disable_device_list_for_node():
  *
  *
  */
-int core_disable_device_list_for_node(
+int target_disable_device_list_for_node(
 	struct se_lun *lun,
 	struct se_lun_acl *lun_acl,
 	u32 mapped_lun,
@@ -479,7 +479,7 @@ void target_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg)
 			mapped_lun = deve->mapped_lun;
 			rcu_read_unlock();
 
-			core_disable_device_list_for_node(lun, NULL, mapped_lun,
+			target_disable_device_list_for_node(lun, NULL, mapped_lun,
 					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 		}
 	}
@@ -1232,7 +1232,7 @@ struct se_lun *core_dev_add_lun(
 			if (acl->dynamic_node_acl &&
 			    (!tpg->se_tpg_tfo->tpg_check_demo_mode_login_only ||
 			     !tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg))) {
-				core_tpg_add_node_to_devs(acl, tpg);
+				target_add_node_to_devs(acl, tpg);
 			}
 		}
 		mutex_unlock(&tpg->acl_node_mutex);
@@ -1374,7 +1374,7 @@ int core_dev_add_initiator_node_lun_acl(
 
 	lacl->se_lun = lun;
 
-	if (core_enable_device_list_for_node(lun, lacl, lacl->mapped_lun,
+	if (target_enable_device_list_for_node(lun, lacl, lacl->mapped_lun,
 			lun_access, nacl, tpg) < 0)
 		return -EINVAL;
 
@@ -1407,7 +1407,7 @@ int core_dev_del_initiator_node_lun_acl(
 	if (!nacl)
 		return -EINVAL;
 
-	core_disable_device_list_for_node(lun, NULL, lacl->mapped_lun,
+	target_disable_device_list_for_node(lun, NULL, lacl->mapped_lun,
 		TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 
 	lacl->se_lun = NULL;
diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h
index 70190e2..6c8b7fb 100644
--- a/drivers/target/target_core_internal.h
+++ b/drivers/target/target_core_internal.h
@@ -5,13 +5,13 @@
 extern struct t10_alua_lu_gp *default_lu_gp;
 
 /* target_core_device.c */
-struct se_dev_entry *core_get_se_deve_from_rtpi(struct se_node_acl *, u16);
-int	core_free_device_list_for_node(struct se_node_acl *,
+struct se_dev_entry *target_get_deve_from_rtpi(struct se_node_acl *, u16);
+int	target_free_device_list_for_node(struct se_node_acl *,
 		struct se_portal_group *);
 void	core_update_device_list_access(u32, u32, struct se_node_acl *);
-int	core_enable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
-		u32, u32, struct se_node_acl *, struct se_portal_group *);
-int	core_disable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
+int	target_enable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
+			u32, u32, struct se_node_acl *, struct se_portal_group *);
+int	target_disable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
 		u32, u32, struct se_node_acl *, struct se_portal_group *);
 void	target_clear_lun_from_tpg(struct se_lun *, struct se_portal_group *);
 int	core_dev_export(struct se_device *, struct se_portal_group *,
@@ -50,7 +50,7 @@ extern struct se_device *g_lun0_dev;
 
 struct se_node_acl *__target_get_initiator_node_acl(struct se_portal_group *tpg,
 		const char *);
-void	core_tpg_add_node_to_devs(struct se_node_acl *, struct se_portal_group *);
+void	target_add_node_to_devs(struct se_node_acl *, struct se_portal_group *);
 void	core_tpg_wait_for_nacl_pr_ref(struct se_node_acl *);
 struct se_lun *core_tpg_alloc_lun(struct se_portal_group *, u32);
 int	target_add_lun(struct se_portal_group *, struct se_lun *,
diff --git a/drivers/target/target_core_pr.c b/drivers/target/target_core_pr.c
index 3a5ee0b..29c2fc7 100644
--- a/drivers/target/target_core_pr.c
+++ b/drivers/target/target_core_pr.c
@@ -1669,8 +1669,8 @@ core_scsi3_decode_spec_i_port(
 		 * RELATIVE TARGET PORT IDENTIFIER on the receiving I_T Nexus
 		 * Target Port.
 		 */
-		dest_se_deve = core_get_se_deve_from_rtpi(dest_node_acl,
-					dest_rtpi);
+		dest_se_deve = target_get_deve_from_rtpi(dest_node_acl,
+							 dest_rtpi);
 		if (!dest_se_deve) {
 			pr_err("Unable to locate %s dest_se_deve"
 				" from destination RTPI: %hu\n",
@@ -3374,7 +3374,7 @@ after_iport_check:
 	 * Locate the struct se_dev_entry pointer for the matching RELATIVE TARGET
 	 * PORT IDENTIFIER.
 	 */
-	dest_se_deve = core_get_se_deve_from_rtpi(dest_node_acl, rtpi);
+	dest_se_deve = target_get_deve_from_rtpi(dest_node_acl, rtpi);
 	if (!dest_se_deve) {
 		pr_err("Unable to locate %s dest_se_deve from RTPI:"
 			" %hu\n",  dest_tf_ops->get_fabric_name(), rtpi);
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index 9d31434..b10071d 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -78,7 +78,7 @@ static void target_clear_initiator_node_from_tpg(
 		mapped_lun = deve->mapped_lun;
 		rcu_read_unlock();
 
-		core_disable_device_list_for_node(lun, NULL, mapped_lun,
+		target_disable_device_list_for_node(lun, NULL, mapped_lun,
 					TRANSPORT_LUNFLAGS_NO_ACCESS, nacl, tpg);
 	}
 }
@@ -119,11 +119,11 @@ struct se_node_acl *target_get_initiator_node_acl(
 }
 EXPORT_SYMBOL(target_get_initiator_node_acl);
 
-/*	core_tpg_add_node_to_devs():
+/*	target_add_node_to_devs():
  *
  *
  */
-void core_tpg_add_node_to_devs(
+void target_add_node_to_devs(
 	struct se_node_acl *acl,
 	struct se_portal_group *tpg)
 {
@@ -163,8 +163,8 @@ void core_tpg_add_node_to_devs(
 			(lun_access == TRANSPORT_LUNFLAGS_READ_WRITE) ?
 			"READ-WRITE" : "READ-ONLY");
 
-		core_enable_device_list_for_node(lun, NULL, lun->unpacked_lun,
-						 lun_access, acl, tpg);
+		target_enable_device_list_for_node(lun, NULL, lun->unpacked_lun,
+						   lun_access, acl, tpg);
 		/*
 		 * Check to see if there are any existing persistent reservation
 		 * APTPL pre-registrations that need to be enabled for this dynamic
@@ -222,11 +222,11 @@ static void *array_zalloc(int n, size_t size, gfp_t flags)
 	return a;
 }
 
-/*      core_create_device_list_for_node():
+/*      target_create_device_list_for_node():
  *
  *
  */
-static int core_create_device_list_for_node(struct se_node_acl *nacl)
+static int target_create_device_list_for_node(struct se_node_acl *nacl)
 {
 	struct se_dev_entry *deve;
 	int i;
@@ -286,13 +286,13 @@ struct se_node_acl *target_check_initiator_node_acl(
 
 	tpg->se_tpg_tfo->set_default_node_attributes(acl);
 
-	if (core_create_device_list_for_node(acl) < 0) {
+	if (target_create_device_list_for_node(acl) < 0) {
 		tpg->se_tpg_tfo->tpg_release_fabric_acl(tpg, acl);
 		return NULL;
 	}
 
 	if (core_set_queue_depth_for_node(tpg, acl) < 0) {
-		core_free_device_list_for_node(acl, tpg);
+		target_free_device_list_for_node(acl, tpg);
 		tpg->se_tpg_tfo->tpg_release_fabric_acl(tpg, acl);
 		return NULL;
 	}
@@ -303,7 +303,7 @@ struct se_node_acl *target_check_initiator_node_acl(
 	 */
 	if ((tpg->se_tpg_tfo->tpg_check_demo_mode_login_only == NULL) ||
 	    (tpg->se_tpg_tfo->tpg_check_demo_mode_login_only(tpg) != 1))
-		core_tpg_add_node_to_devs(acl, tpg);
+		target_add_node_to_devs(acl, tpg);
 
 	mutex_lock(&tpg->acl_node_mutex);
 	list_add_tail(&acl->acl_list, &tpg->acl_node_list);
@@ -391,13 +391,13 @@ struct se_node_acl *target_add_initiator_node_acl(
 
 	tpg->se_tpg_tfo->set_default_node_attributes(acl);
 
-	if (core_create_device_list_for_node(acl) < 0) {
+	if (target_create_device_list_for_node(acl) < 0) {
 		tpg->se_tpg_tfo->tpg_release_fabric_acl(tpg, acl);
 		return ERR_PTR(-ENOMEM);
 	}
 
 	if (core_set_queue_depth_for_node(tpg, acl) < 0) {
-		core_free_device_list_for_node(acl, tpg);
+		target_free_device_list_for_node(acl, tpg);
 		tpg->se_tpg_tfo->tpg_release_fabric_acl(tpg, acl);
 		return ERR_PTR(-EINVAL);
 	}
@@ -470,7 +470,7 @@ int target_del_initiator_node_acl(
 
 	core_tpg_wait_for_nacl_pr_ref(acl);
 	target_clear_initiator_node_from_tpg(acl, tpg);
-	core_free_device_list_for_node(acl, tpg);
+	target_free_device_list_for_node(acl, tpg);
 
 	pr_debug("%s_TPG[%hu] - Deleted ACL with TCQ Depth: %d for %s"
 		" Initiator Node: %s\n", tpg->se_tpg_tfo->get_fabric_name(),
@@ -739,7 +739,7 @@ int target_deregister_tpg(struct se_portal_group *se_tpg)
 		se_tpg->num_node_acls--;
 
 		core_tpg_wait_for_nacl_pr_ref(nacl);
-		core_free_device_list_for_node(nacl, se_tpg);
+		target_free_device_list_for_node(nacl, se_tpg);
 		se_tpg->se_tpg_tfo->tpg_release_fabric_acl(se_tpg, nacl);
 	}
 
diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index 86d4aeb..64f2f87 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -505,7 +505,7 @@ void transport_deregister_session(struct se_session *se_sess)
 
 	if (drop_nacl) {
 		core_tpg_wait_for_nacl_pr_ref(se_nacl);
-		core_free_device_list_for_node(se_nacl, se_tpg);
+		target_free_device_list_for_node(se_nacl, se_tpg);
 		se_tfo->tpg_release_fabric_acl(se_tpg, se_nacl);
 		comp_nacl = false;
 	}
-- 
1.9.1

[RFC 22/22] target: Rename se_port/se_device export to target_* prefix

[Nicholas A. Bellinger]

From: Nicholas Bellinger <nab@linux-iscsi.org>

This includes:

  - target_alloc_port
  - target_export_port
  - target_release_port
  - target_dev_export
  - target_dev_unexport

Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
---
 drivers/target/target_core_device.c   | 16 ++++++++--------
 drivers/target/target_core_internal.h |  4 ++--
 drivers/target/target_core_tpg.c      |  4 ++--
 3 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
index 5b225d8..efbc840 100644
--- a/drivers/target/target_core_device.c
+++ b/drivers/target/target_core_device.c
@@ -486,7 +486,7 @@ void target_clear_lun_from_tpg(struct se_lun *lun, struct se_portal_group *tpg)
 	mutex_unlock(&tpg->acl_node_mutex);
 }
 
-static struct se_port *core_alloc_port(struct se_device *dev)
+static struct se_port *target_alloc_port(struct se_device *dev)
 {
 	struct se_port *port, *port_tmp;
 
@@ -538,7 +538,7 @@ again:
 	return port;
 }
 
-static void core_export_port(
+static void target_export_port(
 	struct se_device *dev,
 	struct se_portal_group *tpg,
 	struct se_port *port,
@@ -580,7 +580,7 @@ static void core_export_port(
 /*
  *	Called with struct se_device->se_port_lock spinlock held.
  */
-static void core_release_port(struct se_device *dev, struct se_port *port)
+static void target_release_port(struct se_device *dev, struct se_port *port)
 	__releases(&dev->se_port_lock) __acquires(&dev->se_port_lock)
 {
 	/*
@@ -599,7 +599,7 @@ static void core_release_port(struct se_device *dev, struct se_port *port)
 	kfree(port);
 }
 
-int core_dev_export(
+int target_dev_export(
 	struct se_device *dev,
 	struct se_portal_group *tpg,
 	struct se_lun *lun)
@@ -607,7 +607,7 @@ int core_dev_export(
 	struct se_hba *hba = dev->se_hba;
 	struct se_port *port;
 
-	port = core_alloc_port(dev);
+	port = target_alloc_port(dev);
 	if (IS_ERR(port))
 		return PTR_ERR(port);
 
@@ -617,11 +617,11 @@ int core_dev_export(
 	dev->export_count++;
 	spin_unlock(&hba->device_lock);
 
-	core_export_port(dev, tpg, port, lun);
+	target_export_port(dev, tpg, port, lun);
 	return 0;
 }
 
-void core_dev_unexport(
+void target_dev_unexport(
 	struct se_device *dev,
 	struct se_portal_group *tpg,
 	struct se_lun *lun)
@@ -637,7 +637,7 @@ void core_dev_unexport(
 	spin_unlock(&lun->lun_sep_lock);
 
 	spin_lock(&dev->se_port_lock);
-	core_release_port(dev, port);
+	target_release_port(dev, port);
 	spin_unlock(&dev->se_port_lock);
 
 	spin_lock(&hba->device_lock);
diff --git a/drivers/target/target_core_internal.h b/drivers/target/target_core_internal.h
index 6c8b7fb..240e5ba 100644
--- a/drivers/target/target_core_internal.h
+++ b/drivers/target/target_core_internal.h
@@ -14,9 +14,9 @@ int	target_enable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
 int	target_disable_device_list_for_node(struct se_lun *, struct se_lun_acl *,
 		u32, u32, struct se_node_acl *, struct se_portal_group *);
 void	target_clear_lun_from_tpg(struct se_lun *, struct se_portal_group *);
-int	core_dev_export(struct se_device *, struct se_portal_group *,
+int	target_dev_export(struct se_device *, struct se_portal_group *,
 		struct se_lun *);
-void	core_dev_unexport(struct se_device *, struct se_portal_group *,
+void	target_dev_unexport(struct se_device *, struct se_portal_group *,
 		struct se_lun *);
 struct se_lun *core_dev_add_lun(struct se_portal_group *, struct se_device *, u32);
 void	core_dev_del_lun(struct se_portal_group *, struct se_lun *);
diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
index b10071d..ee6af20 100644
--- a/drivers/target/target_core_tpg.c
+++ b/drivers/target/target_core_tpg.c
@@ -795,7 +795,7 @@ int target_add_lun(
 	if (ret < 0)
 		return ret;
 
-	ret = core_dev_export(dev, tpg, lun);
+	ret = target_dev_export(dev, tpg, lun);
 	if (ret < 0) {
 		percpu_ref_exit(&lun->lun_ref);
 		return ret;
@@ -816,7 +816,7 @@ void target_remove_lun(
 	target_clear_lun_from_tpg(lun, tpg);
 	transport_clear_lun_ref(lun);
 
-	core_dev_unexport(lun->lun_se_dev, tpg, lun);
+	target_dev_unexport(lun->lun_se_dev, tpg, lun);
 
 	mutex_lock(&tpg->tpg_lun_mutex);
 	lun->lun_status = TRANSPORT_LUN_STATUS_FREE;
-- 
1.9.1

Re: [RFC 01/22] target: Convert transport_lookup_*_lun to RCU reader

[Sagi Grimberg]

On 3/27/2015 11:04 AM, Nicholas A. Bellinger wrote:
> From: Nicholas Bellinger <nab@linux-iscsi.org>
>
> This patch converts transport_lookup_*_lun() fast-path code to
> use RCU read path primitives when looking up se_dev_entry.  It
> adds a new array of pointers in se_node_acl->lun_entry_hlist
> for this purpose.
>
> For transport_lookup_cmd_lun() code, it works with existing per-cpu
> se_lun->lun_ref when associating se_cmd with se_lun + se_device.
>
> Also, go ahead and update core_create_device_list_for_node() +
> core_free_device_list_for_node() to use ->lun_entry_hlist.
>
> Cc: Hannes Reinecke <hare@suse.de>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Sagi Grimberg <sagig@mellanox.com>
> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
> ---
>   drivers/target/target_core_device.c | 45 +++++++++++++++++++++++++------------
>   drivers/target/target_core_tpg.c    | 10 +++++----
>   include/target/target_core_base.h   |  2 ++
>   3 files changed, 39 insertions(+), 18 deletions(-)
>
> diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
> index 37449bd..be893c8 100644
> --- a/drivers/target/target_core_device.c
> +++ b/drivers/target/target_core_device.c
> @@ -59,17 +59,24 @@ transport_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
>   {
>   	struct se_lun *se_lun = NULL;
>   	struct se_session *se_sess = se_cmd->se_sess;
> +	struct se_node_acl *nacl = se_sess->se_node_acl;
>   	struct se_device *dev;
> -	unsigned long flags;
> +	struct se_dev_entry *deve;
>
>   	if (unpacked_lun >= TRANSPORT_MAX_LUNS_PER_TPG)
>   		return TCM_NON_EXISTENT_LUN;
>
> -	spin_lock_irqsave(&se_sess->se_node_acl->device_list_lock, flags);
> -	se_cmd->se_deve = se_sess->se_node_acl->device_list[unpacked_lun];
> -	if (se_cmd->se_deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
> -		struct se_dev_entry *deve = se_cmd->se_deve;
> -
> +	rcu_read_lock();
> +	deve = rcu_dereference(nacl->lun_entry_hlist[unpacked_lun]);
> +	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
> +		/*
> +		 * Make sure that target_enable_device_list_for_node()
> +		 * has not already cleared the RCU protected pointers.
> +		 */
> +		if (!deve->se_lun) {
> +			rcu_read_unlock();
> +			goto check_lun;
> +		}
>   		deve->total_cmds++;
>
>   		if ((se_cmd->data_direction == DMA_TO_DEVICE) &&
> @@ -78,7 +85,7 @@ transport_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
>   				" Access for 0x%08x\n",
>   				se_cmd->se_tfo->get_fabric_name(),
>   				unpacked_lun);
> -			spin_unlock_irqrestore(&se_sess->se_node_acl->device_list_lock, flags);
> +			rcu_read_unlock();
>   			return TCM_WRITE_PROTECTED;
>   		}
>
> @@ -96,8 +103,9 @@ transport_lookup_cmd_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
>   		percpu_ref_get(&se_lun->lun_ref);
>   		se_cmd->lun_ref_active = true;
>   	}
> -	spin_unlock_irqrestore(&se_sess->se_node_acl->device_list_lock, flags);
> +	rcu_read_unlock();
>
> +check_lun:
>   	if (!se_lun) {
>   		/*
>   		 * Use the se_portal_group->tpg_virt_lun0 to allow for
> @@ -146,25 +154,34 @@ int transport_lookup_tmr_lun(struct se_cmd *se_cmd, u32 unpacked_lun)
>   	struct se_dev_entry *deve;
>   	struct se_lun *se_lun = NULL;
>   	struct se_session *se_sess = se_cmd->se_sess;
> +	struct se_node_acl *nacl = se_sess->se_node_acl;
>   	struct se_tmr_req *se_tmr = se_cmd->se_tmr_req;
>   	unsigned long flags;
>
>   	if (unpacked_lun >= TRANSPORT_MAX_LUNS_PER_TPG)
>   		return -ENODEV;
>
> -	spin_lock_irqsave(&se_sess->se_node_acl->device_list_lock, flags);
> -	se_cmd->se_deve = se_sess->se_node_acl->device_list[unpacked_lun];
> -	deve = se_cmd->se_deve;
> +	rcu_read_lock();
> +	deve = rcu_dereference(nacl->lun_entry_hlist[unpacked_lun]);
>
>   	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
> +		/*
> +		 * Make sure that target_enable_device_list_for_node()
> +		 * has not already cleared the RCU protected pointers.
> +		 */
> +		if (!deve->se_lun) {
> +			rcu_read_unlock();
> +			goto check_lun;
> +		}
>   		se_tmr->tmr_lun = deve->se_lun;
>   		se_cmd->se_lun = deve->se_lun;
>   		se_lun = deve->se_lun;
>   		se_cmd->pr_res_key = deve->pr_res_key;
>   		se_cmd->orig_fe_lun = unpacked_lun;
>   	}
> -	spin_unlock_irqrestore(&se_sess->se_node_acl->device_list_lock, flags);
> +	rcu_read_unlock();
>
> +check_lun:
>   	if (!se_lun) {
>   		pr_debug("TARGET_CORE[%s]: Detected NON_EXISTENT_LUN"
>   			" Access for 0x%08x\n",
> @@ -267,8 +284,8 @@ int core_free_device_list_for_node(
>   	}
>   	spin_unlock_irq(&nacl->device_list_lock);
>
> -	array_free(nacl->device_list, TRANSPORT_MAX_LUNS_PER_TPG);
> -	nacl->device_list = NULL;
> +	array_free(nacl->lun_entry_hlist, TRANSPORT_MAX_LUNS_PER_TPG);
> +	nacl->lun_entry_hlist = NULL;
>
>   	return 0;
>   }
> diff --git a/drivers/target/target_core_tpg.c b/drivers/target/target_core_tpg.c
> index 0696de9..b2fdba5 100644
> --- a/drivers/target/target_core_tpg.c
> +++ b/drivers/target/target_core_tpg.c
> @@ -234,15 +234,15 @@ static int core_create_device_list_for_node(struct se_node_acl *nacl)
>   	struct se_dev_entry *deve;
>   	int i;
>
> -	nacl->device_list = array_zalloc(TRANSPORT_MAX_LUNS_PER_TPG,
> +	nacl->lun_entry_hlist = array_zalloc(TRANSPORT_MAX_LUNS_PER_TPG,
>   			sizeof(struct se_dev_entry), GFP_KERNEL);
> -	if (!nacl->device_list) {
> +	if (!nacl->lun_entry_hlist) {
>   		pr_err("Unable to allocate memory for"
> -			" struct se_node_acl->device_list\n");
> +			" struct se_node_acl->lun_entry_hlist\n");
>   		return -ENOMEM;
>   	}
>   	for (i = 0; i < TRANSPORT_MAX_LUNS_PER_TPG; i++) {
> -		deve = nacl->device_list[i];
> +		deve = nacl->lun_entry_hlist[i];
>
>   		atomic_set(&deve->ua_count, 0);
>   		atomic_set(&deve->pr_ref_count, 0);
> @@ -281,6 +281,7 @@ struct se_node_acl *core_tpg_check_initiator_node_acl(
>   	init_completion(&acl->acl_free_comp);
>   	spin_lock_init(&acl->device_list_lock);
>   	spin_lock_init(&acl->nacl_sess_lock);
> +	spin_lock_init(&acl->lun_entry_lock);
>   	atomic_set(&acl->acl_pr_ref_count, 0);
>   	acl->queue_depth = tpg->se_tpg_tfo->tpg_get_default_depth(tpg);
>   	snprintf(acl->initiatorname, TRANSPORT_IQN_LEN, "%s", initiatorname);
> @@ -408,6 +409,7 @@ struct se_node_acl *core_tpg_add_initiator_node_acl(
>   	init_completion(&acl->acl_free_comp);
>   	spin_lock_init(&acl->device_list_lock);
>   	spin_lock_init(&acl->nacl_sess_lock);
> +	spin_lock_init(&acl->lun_entry_lock);
>   	atomic_set(&acl->acl_pr_ref_count, 0);
>   	acl->queue_depth = queue_depth;
>   	snprintf(acl->initiatorname, TRANSPORT_IQN_LEN, "%s", initiatorname);
> diff --git a/include/target/target_core_base.h b/include/target/target_core_base.h
> index fe25a78..06ecd7b 100644
> --- a/include/target/target_core_base.h
> +++ b/include/target/target_core_base.h
> @@ -595,10 +595,12 @@ struct se_node_acl {
>   	char			acl_tag[MAX_ACL_TAG_SIZE];
>   	/* Used for PR SPEC_I_PT=1 and REGISTER_AND_MOVE */
>   	atomic_t		acl_pr_ref_count;
> +	struct se_dev_entry __rcu **lun_entry_hlist;
>   	struct se_dev_entry	**device_list;

Shouldn't device_list be removed here?

>   	struct se_session	*nacl_sess;
>   	struct se_portal_group *se_tpg;
>   	spinlock_t		device_list_lock;
> +	spinlock_t		lun_entry_lock;

This lock is unused in this patch, it should be
introduced with its usage.

Sagi.

Re: [RFC 02/22] target: Convert enable/disable ->device_list to RCU updater

[Sagi Grimberg]

On 3/27/2015 11:04 AM, Nicholas A. Bellinger wrote:
> From: Nicholas Bellinger <nab@linux-iscsi.org>
>
> This patch converts core_[enable,disable]_device_list_for_node() to RCU
> updater path code when modifying se_dev_entry pointers.
>
> It includes protected rcu_assign_pointer() and invokes synchronize_rcu()
> to wait for RCU read paths to finish.
>
> Required for subsequent conversion to se_deve->pr_ref percpu-refcount.
>
> Cc: Hannes Reinecke <hare@suse.de>
> Cc: Christoph Hellwig <hch@lst.de>
> Cc: Sagi Grimberg <sagig@mellanox.com>
> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
> ---
>   drivers/target/target_core_device.c | 50 ++++++++++++++++---------------------
>   1 file changed, 22 insertions(+), 28 deletions(-)
>
> diff --git a/drivers/target/target_core_device.c b/drivers/target/target_core_device.c
> index be893c8..9385e16 100644
> --- a/drivers/target/target_core_device.c
> +++ b/drivers/target/target_core_device.c
> @@ -324,31 +324,16 @@ int core_enable_device_list_for_node(
>   	struct se_port *port = lun->lun_sep;
>   	struct se_dev_entry *deve;
>
> -	spin_lock_irq(&nacl->device_list_lock);
> -
> -	deve = nacl->device_list[mapped_lun];
> -
>   	/*
>   	 * Check if the call is handling demo mode -> explicit LUN ACL
>   	 * transition.  This transition must be for the same struct se_lun
>   	 * + mapped_lun that was setup in demo mode..
>   	 */
> +	spin_lock_irq(&nacl->lun_entry_lock);
> +	deve = nacl->lun_entry_hlist[mapped_lun];
>   	if (deve->lun_flags & TRANSPORT_LUNFLAGS_INITIATOR_ACCESS) {
> -		if (deve->se_lun_acl != NULL) {
> -			pr_err("struct se_dev_entry->se_lun_acl"
> -			       " already set for demo mode -> explicit"
> -			       " LUN ACL transition\n");
> -			spin_unlock_irq(&nacl->device_list_lock);
> -			return -EINVAL;
> -		}
> -		if (deve->se_lun != lun) {
> -			pr_err("struct se_dev_entry->se_lun does"
> -			       " match passed struct se_lun for demo mode"
> -			       " -> explicit LUN ACL transition\n");
> -			spin_unlock_irq(&nacl->device_list_lock);
> -			return -EINVAL;
> -		}
> -		deve->se_lun_acl = lun_acl;
> +		BUG_ON(deve->se_lun_acl != NULL);
> +		BUG_ON(deve->se_lun != lun);

Are these a MUST? Can't we WARN and return an error instead of shutting
down the lights on everyone?

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Christoph Hellwig]

I went through this in detail, and the odd patch split that splits
one change up into muliple patches, but then also mixes up other
changes makes it hard to read.

So I started rebasing the tree to understand it moving your various
cleanups to the front of the series.  Of coure while reading through
it the number of cleanups grew, so there are a few more now.

Based on that I don't think this series will work very well, RCU works
when replacing strutures in a lookup data structure, but not when
changing the content of it, so the new locking scheme doesn't work
very well, as the device_list array is static, it's just the content
that changes.

Fortunately there is the old patch from Andy to make the se_dev_entry
dynamically allocated, which comes in useful here.  With that we might
only change the rdonly flag on a live dev entry, or assign an ACL when
it previously was NULL, something that RCU can handle nicely.

I've pushed a git tree to

	git://git.infradead.org/users/hch/scsi.git target-rcu-hch

Gitweb:

	http://git.infradead.org/users/hch/scsi.git/shortlog/refs/heads/target-rcu-hch

that implements this scheme.  Note that I dropped the percpu refcount
changes - the reference is only taken in the SCSI3-PR slow path, and the
percpu refcount API is extremly cumbersome.

This survives I/O testing nicely, but I couldn't find a good test suite
for the control path.

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Andy Grover]

On 03/30/2015 05:08 AM, Christoph Hellwig wrote:
> Fortunately there is the old patch from Andy to make the se_dev_entry
> dynamically allocated, which comes in useful here.  With that we might
> only change the rdonly flag on a live dev entry, or assign an ACL when
> it previously was NULL, something that RCU can handle nicely.
>
> I've pushed a git tree to
>
> 	git://git.infradead.org/users/hch/scsi.git target-rcu-hch
>
> Gitweb:
>
> 	http://git.infradead.org/users/hch/scsi.git/shortlog/refs/heads/target-rcu-hch
>
> that implements this scheme.  Note that I dropped the percpu refcount
> changes - the reference is only taken in the SCSI3-PR slow path, and the
> percpu refcount API is extremly cumbersome.

I dug my patches up and rebased on top of target-pending/for-next. 
Pushed here:

git://git.kernel.org/pub/scm/linux/kernel/git/grover/linux.git 
mar30-dynalloc-deve

https://git.kernel.org/cgit/linux/kernel/git/grover/linux.git/log/?h=mar30-dynalloc-deve

if this is helpful for your efforts.

Regards -- Andy

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Christoph Hellwig]

On Mon, Mar 30, 2015 at 12:16:04PM -0700, Andy Grover wrote:
> I dug my patches up and rebased on top of target-pending/for-next. Pushed
> here:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/grover/linux.git
> mar30-dynalloc-deve
> 
> https://git.kernel.org/cgit/linux/kernel/git/grover/linux.git/log/?h=mar30-dynalloc-deve
> 
> if this is helpful for your efforts.

See the branch above, it does everything your old series did except
for dynamic allocation of the LUNs, although it gets there through a
different way.

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Andy Grover]

On 03/30/2015 12:21 PM, Christoph Hellwig wrote:
> On Mon, Mar 30, 2015 at 12:16:04PM -0700, Andy Grover wrote:
>> I dug my patches up and rebased on top of target-pending/for-next. Pushed
>> here:
>>
>> git://git.kernel.org/pub/scm/linux/kernel/git/grover/linux.git
>> mar30-dynalloc-deve
>>
>> https://git.kernel.org/cgit/linux/kernel/git/grover/linux.git/log/?h=mar30-dynalloc-deve
>>
>> if this is helpful for your efforts.
>
> See the branch above, it does everything your old series did except
> for dynamic allocation of the LUNs, although it gets there through a
> different way.
>

And you got rid of the redundant LUNFLAGS_READ_WRITE flag too, nice.

So what's next? Were you going to do the LUNs too? In addition to +perf 
and less mem usage, going down this path also leads to >256 LUNs per tpg.

Woot! -- Andy

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Christoph Hellwig]

On Mon, Mar 30, 2015 at 12:35:55PM -0700, Andy Grover wrote:
> So what's next? Were you going to do the LUNs too? In addition to +perf and
> less mem usage, going down this path also leads to >256 LUNs per tpg.

This just started as a rework of Nic' series, so I'd like to avoid scope
creep.  I think applying the same dynamic allocation + rcu scheme should
work fine for the LUNs, too.

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Nicholas A. Bellinger]

On Mon, 2015-03-30 at 05:08 -0700, Christoph Hellwig wrote:
> I went through this in detail, and the odd patch split that splits
> one change up into muliple patches, but then also mixes up other
> changes makes it hard to read.
> 

Thanks for having a look.  Will fix up patch ordering for -v2.

> So I started rebasing the tree to understand it moving your various
> cleanups to the front of the series.  Of coure while reading through
> it the number of cleanups grew, so there are a few more now.
> 

;)

> Based on that I don't think this series will work very well, RCU works
> when replacing strutures in a lookup data structure, but not when
> changing the content of it, so the new locking scheme doesn't work
> very well, as the device_list array is static, it's just the content
> that changes.
> 

The initial series was primarily to get some feedback on the non trivial
PR cases.

Since last week, enable/disable device_list code has been converted to
use mempool + call_rcu() and performs the RCU pointer swap in
se_node_acl->lun_entry_hlist[] under se_node_acl->lun_entry_mutex.

There are a few more target_core_pr.c pieces that need to be updated to
handle NULL se_node_acl->lun_entry_hlist[] pointers, along with proper
rcu_dereference_protected() and rcu_access_pointer() notations for
se_dev_entry pointer access outside of RCU read lock.

>From there, it should be an straight-forward conversion to rcu hlist for
se_node_acl->lun_entry_list[] and se_portal_group->tpg_lun_list[] to
support dynamic LUN layouts.

> Fortunately there is the old patch from Andy to make the se_dev_entry
> dynamically allocated, which comes in useful here.  With that we might
> only change the rdonly flag on a live dev entry, or assign an ACL when
> it previously was NULL, something that RCU can handle nicely.
> 
> I've pushed a git tree to
> 
> 	git://git.infradead.org/users/hch/scsi.git target-rcu-hch
> 
> Gitweb:
> 
> 	http://git.infradead.org/users/hch/scsi.git/shortlog/refs/heads/target-rcu-hch
> 
> that implements this scheme.  Note that I dropped the percpu refcount
> changes - the reference is only taken in the SCSI3-PR slow path, and the
> percpu refcount API is extremly cumbersome.
> 

Well, there still needs to be some manner of reference counting on
se_dev_entry once rcu_read_lock() is released for the REGISTER w/ I_PORT
and REGISTER_AND_MOVE cases with a non-local I_T_L se_dev_entry
reference.

Otherwise, an RCU update can occur from below the reader unless the PR
special cases are allowed to drop pointer reference + invoke struct
completion before call_rcu() is dispatched to release memory.

> This survives I/O testing nicely, but I couldn't find a good test suite
> for the control path.
> --
> To unsubscribe from this list: send the line "unsubscribe target-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Christoph Hellwig]

On Tue, Mar 31, 2015 at 11:51:56PM -0700, Nicholas A. Bellinger wrote:
> Since last week, enable/disable device_list code has been converted to
> use mempool + call_rcu() and performs the RCU pointer swap in
> se_node_acl->lun_entry_hlist[] under se_node_acl->lun_entry_mutex.

Why use a mempool there?  Please take a look at my branch, it's much
simpler and logical than the old version.

> There are a few more target_core_pr.c pieces that need to be updated to
> handle NULL se_node_acl->lun_entry_hlist[] pointers, along with proper
> rcu_dereference_protected() and rcu_access_pointer() notations for
> se_dev_entry pointer access outside of RCU read lock.

In my branch I've been over all the places to make sure they handle
the NULL case.

> > that implements this scheme.  Note that I dropped the percpu refcount
> > changes - the reference is only taken in the SCSI3-PR slow path, and the
> > percpu refcount API is extremly cumbersome.
> > 
> 
> Well, there still needs to be some manner of reference counting on
> se_dev_entry once rcu_read_lock() is released for the REGISTER w/ I_PORT
> and REGISTER_AND_MOVE cases with a non-local I_T_L se_dev_entry
> reference.

The refcount is still there in the good old atomic_t-based version,
which is perfectl fine for the PRIN and PROUT subcommands which aren't
the I/O fast path.

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Nicholas A. Bellinger]

On Wed, 2015-04-01 at 00:04 -0700, Christoph Hellwig wrote:
> On Tue, Mar 31, 2015 at 11:51:56PM -0700, Nicholas A. Bellinger wrote:
> > Since last week, enable/disable device_list code has been converted to
> > use mempool + call_rcu() and performs the RCU pointer swap in
> > se_node_acl->lun_entry_hlist[] under se_node_acl->lun_entry_mutex.
> 
> Why use a mempool there?

Would be nice to guarantee a certain number of new se_dev_entry
allocations always succeed as existing code expects.

>  Please take a look at my branch, it's much simpler and logical than
> the old version.
> 

Yes, nice simplifications and code cleanups.  I'm still working to get
things functionally correct first, and will be applying parts of
target-rcu-hch from there..

The use of t10_pr_registration->pr_reg_deve is still wrong though, as
it's held for the lifetime of the pr_reg, and not just while
pr_ref_count is non zero for REGISTER w/ I_PORT + REGISTER_AND_MOVE
special cases.

I've already fixed this for v2 by using se_dev_entry->pr_reg for
signaling when a PR registration is active.

> > There are a few more target_core_pr.c pieces that need to be updated to
> > handle NULL se_node_acl->lun_entry_hlist[] pointers, along with proper
> > rcu_dereference_protected() and rcu_access_pointer() notations for
> > se_dev_entry pointer access outside of RCU read lock.
> 
> In my branch I've been over all the places to make sure they handle
> the NULL case.
> 

Great.

> > > that implements this scheme.  Note that I dropped the percpu refcount
> > > changes - the reference is only taken in the SCSI3-PR slow path, and the
> > > percpu refcount API is extremly cumbersome.
> > > 
> > 
> > Well, there still needs to be some manner of reference counting on
> > se_dev_entry once rcu_read_lock() is released for the REGISTER w/ I_PORT
> > and REGISTER_AND_MOVE cases with a non-local I_T_L se_dev_entry
> > reference.
> 
> The refcount is still there in the good old atomic_t-based version,
> which is perfectl fine for the PRIN and PROUT subcommands which aren't
> the I/O fast path.

Agreed the percpu_ref is overkill and percpu_ref_init() is problematic
because it can still -ENOMEM during se_dev_entry creation..

Switching to a normal kref with a blocking completion probably makes the
most sense, in order to avoid potentially spinning on atomic_read() if
APTPL metadata write-out ends up blocking during the two special PR
cases.

--nab

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Christoph Hellwig]

On Wed, Apr 01, 2015 at 10:37:27PM -0700, Nicholas A. Bellinger wrote:
> On Wed, 2015-04-01 at 00:04 -0700, Christoph Hellwig wrote:
> > On Tue, Mar 31, 2015 at 11:51:56PM -0700, Nicholas A. Bellinger wrote:
> > > Since last week, enable/disable device_list code has been converted to
> > > use mempool + call_rcu() and performs the RCU pointer swap in
> > > se_node_acl->lun_entry_hlist[] under se_node_acl->lun_entry_mutex.
> > 
> > Why use a mempool there?
> 
> Would be nice to guarantee a certain number of new se_dev_entry
> allocations always succeed as existing code expects.

mempools are for I/O path that make guaranteed progress.  While the
callers of core_enable_device_list_for_node are in the control path,
and not in a very deep callstack, fairly shallow below the configfs
interface.  This is not a use case for mempools, as there is no need
to guarantee progress in deep I/O callstacks.

> The use of t10_pr_registration->pr_reg_deve is still wrong though, as
> it's held for the lifetime of the pr_reg, and not just while
> pr_ref_count is non zero for REGISTER w/ I_PORT + REGISTER_AND_MOVE
> special cases.

Seems like we should just take the reference properly.

> I've already fixed this for v2 by using se_dev_entry->pr_reg for
> signaling when a PR registration is active.

At least in the version you posted ->pr_reg is just used as a boolean
flag, in which case we might want to keep the real def_pr_registered
flag.

> > The refcount is still there in the good old atomic_t-based version,
> > which is perfectl fine for the PRIN and PROUT subcommands which aren't
> > the I/O fast path.
> 
> Agreed the percpu_ref is overkill and percpu_ref_init() is problematic
> because it can still -ENOMEM during se_dev_entry creation..
> 
> Switching to a normal kref with a blocking completion probably makes the
> most sense, in order to avoid potentially spinning on atomic_read() if
> APTPL metadata write-out ends up blocking during the two special PR
> cases.

The nice part about using the dynamically allocated dev entry is that
the shutdown path doesn't need to be sync.  You can just use a normal
kref and don't care when it actually gets freed.  One thing I didn't
do in my branch but which would e useful is to rename -> pr_ref
to just ->ref as it's a generic refcount - for now it's only used by
the PR code, but there is nothing specific to it in there.

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Nicholas A. Bellinger]

On Thu, 2015-04-02 at 01:56 -0700, Christoph Hellwig wrote:
> On Wed, Apr 01, 2015 at 10:37:27PM -0700, Nicholas A. Bellinger wrote:
> > On Wed, 2015-04-01 at 00:04 -0700, Christoph Hellwig wrote:
> > > On Tue, Mar 31, 2015 at 11:51:56PM -0700, Nicholas A. Bellinger wrote:
> > > > Since last week, enable/disable device_list code has been converted to
> > > > use mempool + call_rcu() and performs the RCU pointer swap in
> > > > se_node_acl->lun_entry_hlist[] under se_node_acl->lun_entry_mutex.
> > > 
> > > Why use a mempool there?
> > 
> > Would be nice to guarantee a certain number of new se_dev_entry
> > allocations always succeed as existing code expects.
> 
> mempools are for I/O path that make guaranteed progress.  While the
> callers of core_enable_device_list_for_node are in the control path,
> and not in a very deep callstack, fairly shallow below the configfs
> interface.  This is not a use case for mempools, as there is no need
> to guarantee progress in deep I/O callstacks.
> 

I was more concerned about the creation of se_dev_entry for dynamically
generated se_node_acl in the login path.

Having to back out everything upon se_dev_entry allocation failure is
annoying.

> > The use of t10_pr_registration->pr_reg_deve is still wrong though, as
> > it's held for the lifetime of the pr_reg, and not just while
> > pr_ref_count is non zero for REGISTER w/ I_PORT + REGISTER_AND_MOVE
> > special cases.
> 
> Seems like we should just take the reference properly.
> 

Not exactly.  The reference needs to be taken when se_dev_entry
is referenced as a remote I_T_L for the two special PR cases.

Otherwise, if it's referenced for normal use it would not be possible to
forcefully release se_dev_entry without first finding the registration,
which requires taking more locks that I want to avoid in that particular
code-path.

> > I've already fixed this for v2 by using se_dev_entry->pr_reg for
> > signaling when a PR registration is active.
> 
> At least in the version you posted ->pr_reg is just used as a boolean
> flag, in which case we might want to keep the real def_pr_registered
> flag.
> 

Using se_dev_entry->pr_reg as an RCU pointer seems reasonable to me,
considering this value can be updated separate from removing the entire
se_dev_entry.

So that way, at least core_scsi3_pr_seq_non_holder() will get the
correct value under rcu_read_lock() after synchronize_rcu() completes in
the update path.

> > > The refcount is still there in the good old atomic_t-based version,
> > > which is perfectl fine for the PRIN and PROUT subcommands which aren't
> > > the I/O fast path.
> > 
> > Agreed the percpu_ref is overkill and percpu_ref_init() is problematic
> > because it can still -ENOMEM during se_dev_entry creation..
> > 
> > Switching to a normal kref with a blocking completion probably makes the
> > most sense, in order to avoid potentially spinning on atomic_read() if
> > APTPL metadata write-out ends up blocking during the two special PR
> > cases.
> 
> The nice part about using the dynamically allocated dev entry is that
> the shutdown path doesn't need to be sync.  You can just use a normal
> kref and don't care when it actually gets freed.  One thing I didn't
> do in my branch but which would e useful is to rename -> pr_ref
> to just ->ref as it's a generic refcount - for now it's only used by
> the PR code, but there is nothing specific to it in there.

Yes, I've already converted to use dynamically allocated dev entries +
call_rcu().  Rename to ->ref sounds fine.

Will be posting -v2 over the next days.  

--nab

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Christoph Hellwig]

On Thu, Apr 02, 2015 at 01:57:10PM -0700, Nicholas A. Bellinger wrote:
> > mempools are for I/O path that make guaranteed progress.  While the
> > callers of core_enable_device_list_for_node are in the control path,
> > and not in a very deep callstack, fairly shallow below the configfs
> > interface.  This is not a use case for mempools, as there is no need
> > to guarantee progress in deep I/O callstacks.
> > 
> 
> I was more concerned about the creation of se_dev_entry for dynamically
> generated se_node_acl in the login path.
> 
> Having to back out everything upon se_dev_entry allocation failure is
> annoying.

But unless you specificly pre-create the number of entries that
core_dev_add_lun may create that's the only way to handle it.  mempool
rely on the fact the objects get freed to the mempool again to make
forward progress, which isn't the case for a lon-lived se_dev_entry
allocation.

Re: [RFC 00/22] target: se_node_acl LUN list RCU conversion

[Nicholas A. Bellinger]

On Mon, 2015-04-06 at 23:17 -0700, Christoph Hellwig wrote:
> On Thu, Apr 02, 2015 at 01:57:10PM -0700, Nicholas A. Bellinger wrote:
> > > mempools are for I/O path that make guaranteed progress.  While the
> > > callers of core_enable_device_list_for_node are in the control path,
> > > and not in a very deep callstack, fairly shallow below the configfs
> > > interface.  This is not a use case for mempools, as there is no need
> > > to guarantee progress in deep I/O callstacks.
> > > 
> > 
> > I was more concerned about the creation of se_dev_entry for dynamically
> > generated se_node_acl in the login path.
> > 
> > Having to back out everything upon se_dev_entry allocation failure is
> > annoying.
> 
> But unless you specificly pre-create the number of entries that
> core_dev_add_lun may create that's the only way to handle it.  mempool
> rely on the fact the objects get freed to the mempool again to make
> forward progress, which isn't the case for a lon-lived se_dev_entry
> allocation.
> --

Point taken.  Dropping the extra mempool_t usage for this case.

--nab