[PATCH 0/6] scsi: target: Remove in_interrupt() usage.

[PATCH 0/6] scsi: target: Remove in_interrupt() usage.

[Sebastian Andrzej Siewior]

Folks,

in the discussion about preempt count consistency across kernel
configurations:

 https://lore.kernel.org/r/20200914204209.256266093@linutronix.de/

it was concluded that the usage of in_interrupt() and related context
checks should be removed from non-core code.

In the long run, usage of 'preemptible, in_*irq etc.' should be banned from
driver code completely.

This series addresses the target subsystem.
Most of in_interrupt() usage is debugging. There is one function which
either invokes wait_for_completion() or schedules a timer based on
in_interrupt().

Sebastian

[PATCH 1/6] scsi: target: iscsi: Avoid in_interrupt() usage in iscsit_close_session().

[Sebastian Andrzej Siewior]

iscsit_close_session() uses in_interrupt() to decide if it needs to
check the return value of iscsit_check_session_usage_count() if it was
not able to sleep.

The usage of in_interrupt() in drivers is phased out and Linus clearly
requested that code which changes behaviour depending on context should
either be separated or the context be conveyed in an argument passed by the
caller, which usually knows the context.

iscsit_close_session() has two callers:
- iscsit_handle_time2retain_timeout()
  A timer_list callback.

- iscsit_close_connection()
  Runs in preemptible context, acquires a mutex.

Add an argument to iscsit_close_session() indicating if sleeping is
possible.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/target/iscsi/iscsi_target.c      | 10 +++++-----
 drivers/target/iscsi/iscsi_target.h      |  2 +-
 drivers/target/iscsi/iscsi_target_erl0.c |  2 +-
 3 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 518fac4864cfa..5abf03125388a 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -4327,7 +4327,7 @@ int iscsit_close_connection(
 	     atomic_read(&sess->session_fall_back_to_erl0)) {
 		spin_unlock_bh(&sess->conn_lock);
 		complete_all(&sess->session_wait_comp);
-		iscsit_close_session(sess);
+		iscsit_close_session(sess, true);
 
 		return 0;
 	} else if (atomic_read(&sess->session_logout)) {
@@ -4337,7 +4337,7 @@ int iscsit_close_connection(
 		if (atomic_read(&sess->session_close)) {
 			spin_unlock_bh(&sess->conn_lock);
 			complete_all(&sess->session_wait_comp);
-			iscsit_close_session(sess);
+			iscsit_close_session(sess, true);
 		} else {
 			spin_unlock_bh(&sess->conn_lock);
 		}
@@ -4353,7 +4353,7 @@ int iscsit_close_connection(
 		if (atomic_read(&sess->session_close)) {
 			spin_unlock_bh(&sess->conn_lock);
 			complete_all(&sess->session_wait_comp);
-			iscsit_close_session(sess);
+			iscsit_close_session(sess, true);
 		} else {
 			spin_unlock_bh(&sess->conn_lock);
 		}
@@ -4366,7 +4366,7 @@ int iscsit_close_connection(
  * If the iSCSI Session for the iSCSI Initiator Node exists,
  * forcefully shutdown the iSCSI NEXUS.
  */
-int iscsit_close_session(struct iscsi_session *sess)
+int iscsit_close_session(struct iscsi_session *sess, bool can_sleep)
 {
 	struct iscsi_portal_group *tpg = sess->tpg;
 	struct se_portal_group *se_tpg = &tpg->tpg_se_tpg;
@@ -4399,7 +4399,7 @@ int iscsit_close_session(struct iscsi_session *sess)
 	 * time2retain handler) and contain and active session usage count we
 	 * restart the timer and exit.
 	 */
-	if (!in_interrupt()) {
+	if (can_sleep) {
 		iscsit_check_session_usage_count(sess);
 	} else {
 		if (iscsit_check_session_usage_count(sess) == 2) {
diff --git a/drivers/target/iscsi/iscsi_target.h b/drivers/target/iscsi/iscsi_target.h
index 7409ce2a66078..b35a96ded9c19 100644
--- a/drivers/target/iscsi/iscsi_target.h
+++ b/drivers/target/iscsi/iscsi_target.h
@@ -41,7 +41,7 @@ extern void iscsit_thread_get_cpumask(struct iscsi_conn *);
 extern int iscsi_target_tx_thread(void *);
 extern int iscsi_target_rx_thread(void *);
 extern int iscsit_close_connection(struct iscsi_conn *);
-extern int iscsit_close_session(struct iscsi_session *);
+extern int iscsit_close_session(struct iscsi_session *, bool can_sleep);
 extern void iscsit_fail_session(struct iscsi_session *);
 extern void iscsit_stop_session(struct iscsi_session *, int, int);
 extern int iscsit_release_sessions_for_tpg(struct iscsi_portal_group *, int);
diff --git a/drivers/target/iscsi/iscsi_target_erl0.c b/drivers/target/iscsi/iscsi_target_erl0.c
index b4abd7b68e6da..102c9cbf59f38 100644
--- a/drivers/target/iscsi/iscsi_target_erl0.c
+++ b/drivers/target/iscsi/iscsi_target_erl0.c
@@ -765,7 +765,7 @@ void iscsit_handle_time2retain_timeout(struct timer_list *t)
 
 	iscsit_fill_cxn_timeout_err_stats(sess);
 	spin_unlock_bh(&se_tpg->session_lock);
-	iscsit_close_session(sess);
+	iscsit_close_session(sess, false);
 }
 
 void iscsit_start_time2retain_handler(struct iscsi_session *sess)
-- 
2.29.2

[PATCH 2/6] scsi: target: iscsi: Avoid in_interrupt() usage in iscsit_check_session_usage_count().

[Sebastian Andrzej Siewior]

iscsit_check_session_usage_count() uses in_interrupt() to find out if it
is safe to invoke wait_for_completion().

The usage of in_interrupt() in drivers is phased out and Linus clearly
requested that code which changes behaviour depending on context should
either be separated or the context be conveyed in an argument passed by the
caller, which usually knows the context.

There is only one caller of iscsit_check_session_usage_count() which
already has an argument indicating if it is safe to sleep.

Extend iscsit_check_session_usage_count() by an argument indicating if
it may sleep.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/target/iscsi/iscsi_target.c      | 4 ++--
 drivers/target/iscsi/iscsi_target_util.c | 4 ++--
 drivers/target/iscsi/iscsi_target_util.h | 2 +-
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 5abf03125388a..13de37ff9dc9f 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -4400,9 +4400,9 @@ int iscsit_close_session(struct iscsi_session *sess, bool can_sleep)
 	 * restart the timer and exit.
 	 */
 	if (can_sleep) {
-		iscsit_check_session_usage_count(sess);
+		iscsit_check_session_usage_count(sess, can_sleep);
 	} else {
-		if (iscsit_check_session_usage_count(sess) == 2) {
+		if (iscsit_check_session_usage_count(sess, can_sleep) == 2) {
 			atomic_set(&sess->session_logout, 0);
 			iscsit_start_time2retain_handler(sess);
 			return 0;
diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c
index 45ba07c6ec270..50028b1d3b849 100644
--- a/drivers/target/iscsi/iscsi_target_util.c
+++ b/drivers/target/iscsi/iscsi_target_util.c
@@ -779,13 +779,13 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)
 }
 EXPORT_SYMBOL(iscsit_free_cmd);
 
-int iscsit_check_session_usage_count(struct iscsi_session *sess)
+int iscsit_check_session_usage_count(struct iscsi_session *sess, bool can_sleep)
 {
 	spin_lock_bh(&sess->session_usage_lock);
 	if (sess->session_usage_count != 0) {
 		sess->session_waiting_on_uc = 1;
 		spin_unlock_bh(&sess->session_usage_lock);
-		if (in_interrupt())
+		if (!can_sleep)
 			return 2;
 
 		wait_for_completion(&sess->session_waiting_on_uc_comp);
diff --git a/drivers/target/iscsi/iscsi_target_util.h b/drivers/target/iscsi/iscsi_target_util.h
index 68e84803b0a1a..a2c2401f29452 100644
--- a/drivers/target/iscsi/iscsi_target_util.h
+++ b/drivers/target/iscsi/iscsi_target_util.h
@@ -40,7 +40,7 @@ extern void iscsit_free_queue_reqs_for_conn(struct iscsi_conn *);
 extern void iscsit_release_cmd(struct iscsi_cmd *);
 extern void __iscsit_free_cmd(struct iscsi_cmd *, bool);
 extern void iscsit_free_cmd(struct iscsi_cmd *, bool);
-extern int iscsit_check_session_usage_count(struct iscsi_session *);
+extern int iscsit_check_session_usage_count(struct iscsi_session *sess, bool can_sleep);
 extern void iscsit_dec_session_usage_count(struct iscsi_session *);
 extern void iscsit_inc_session_usage_count(struct iscsi_session *);
 extern struct iscsi_conn *iscsit_get_conn_from_cid(struct iscsi_session *, u16);
-- 
2.29.2

[PATCH 3/6] scsi: target: iscsi: Redo iscsit_check_session_usage_count() return code.

[Sebastian Andrzej Siewior]

The return value of iscsit_check_session_usage_count() is only checked
if it was not allowed to sleep. If it returns `2' then a timer is
prepared. If it returns something else or if it was allowed to sleep
then it is ignored.

Let iscsit_check_session_usage_count() return true if it needs to arm
the timer - otherwise false. This simplifies the code flow of the only
caller.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/target/iscsi/iscsi_target.c      | 12 ++++--------
 drivers/target/iscsi/iscsi_target_util.c |  9 +++++----
 drivers/target/iscsi/iscsi_target_util.h |  2 +-
 3 files changed, 10 insertions(+), 13 deletions(-)

diff --git a/drivers/target/iscsi/iscsi_target.c b/drivers/target/iscsi/iscsi_target.c
index 13de37ff9dc9f..d0e7ed8f28cc8 100644
--- a/drivers/target/iscsi/iscsi_target.c
+++ b/drivers/target/iscsi/iscsi_target.c
@@ -4399,14 +4399,10 @@ int iscsit_close_session(struct iscsi_session *sess, bool can_sleep)
 	 * time2retain handler) and contain and active session usage count we
 	 * restart the timer and exit.
 	 */
-	if (can_sleep) {
-		iscsit_check_session_usage_count(sess, can_sleep);
-	} else {
-		if (iscsit_check_session_usage_count(sess, can_sleep) == 2) {
-			atomic_set(&sess->session_logout, 0);
-			iscsit_start_time2retain_handler(sess);
-			return 0;
-		}
+	if (iscsit_check_session_usage_count(sess, can_sleep)) {
+		atomic_set(&sess->session_logout, 0);
+		iscsit_start_time2retain_handler(sess);
+		return 0;
 	}
 
 	transport_deregister_session(sess->se_sess);
diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c
index 50028b1d3b849..9468b017b4a73 100644
--- a/drivers/target/iscsi/iscsi_target_util.c
+++ b/drivers/target/iscsi/iscsi_target_util.c
@@ -779,21 +779,22 @@ void iscsit_free_cmd(struct iscsi_cmd *cmd, bool shutdown)
 }
 EXPORT_SYMBOL(iscsit_free_cmd);
 
-int iscsit_check_session_usage_count(struct iscsi_session *sess, bool can_sleep)
+bool iscsit_check_session_usage_count(struct iscsi_session *sess,
+				      bool can_sleep)
 {
 	spin_lock_bh(&sess->session_usage_lock);
 	if (sess->session_usage_count != 0) {
 		sess->session_waiting_on_uc = 1;
 		spin_unlock_bh(&sess->session_usage_lock);
 		if (!can_sleep)
-			return 2;
+			return true;
 
 		wait_for_completion(&sess->session_waiting_on_uc_comp);
-		return 1;
+		return false;
 	}
 	spin_unlock_bh(&sess->session_usage_lock);
 
-	return 0;
+	return false;
 }
 
 void iscsit_dec_session_usage_count(struct iscsi_session *sess)
diff --git a/drivers/target/iscsi/iscsi_target_util.h b/drivers/target/iscsi/iscsi_target_util.h
index a2c2401f29452..8ee1c133a9b7b 100644
--- a/drivers/target/iscsi/iscsi_target_util.h
+++ b/drivers/target/iscsi/iscsi_target_util.h
@@ -40,7 +40,7 @@ extern void iscsit_free_queue_reqs_for_conn(struct iscsi_conn *);
 extern void iscsit_release_cmd(struct iscsi_cmd *);
 extern void __iscsit_free_cmd(struct iscsi_cmd *, bool);
 extern void iscsit_free_cmd(struct iscsi_cmd *, bool);
-extern int iscsit_check_session_usage_count(struct iscsi_session *sess, bool can_sleep);
+extern bool iscsit_check_session_usage_count(struct iscsi_session *sess, bool can_sleep);
 extern void iscsit_dec_session_usage_count(struct iscsi_session *);
 extern void iscsit_inc_session_usage_count(struct iscsi_session *);
 extern struct iscsi_conn *iscsit_get_conn_from_cid(struct iscsi_session *, u16);
-- 
2.29.2

[PATCH 4/6] scsi: target: Remove in_interrupt() usage in core_alua_check_nonop_delay()

[Sebastian Andrzej Siewior]

core_alua_check_nonop_delay() uses in_interrupt() to decide if it is
safe to sleep.

The usage of in_interrupt() in drivers is phased out and Linus clearly
requested that code which changes behaviour depending on context should
either be separated or the context be conveyed in an argument passed by the
caller, which usually knows the context.

core_alua_check_nonop_delay() has two callers:
- target_submit_cmd_map_sgls()
  Kernel doc says it that it must be called from process context. Also
  has a BUG_ON(in_interrupt()).

- iscsit_setup_scsi_cmd()
  Invokes iscsit_add_reject_cmd() which does GFP_KERNEL allocation and
  target_cmd_init_cdb() which may do GFP_KERNEL allocations.

Remove the in_interrupt() check because all callers are from preemptible
context.

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/target/target_core_alua.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/drivers/target/target_core_alua.c b/drivers/target/target_core_alua.c
index 6b72afee2f8b7..5517c7dd51441 100644
--- a/drivers/target/target_core_alua.c
+++ b/drivers/target/target_core_alua.c
@@ -860,8 +860,6 @@ int core_alua_check_nonop_delay(
 {
 	if (!(cmd->se_cmd_flags & SCF_ALUA_NON_OPTIMIZED))
 		return 0;
-	if (in_interrupt())
-		return 0;
 	/*
 	 * The ALUA Active/NonOptimized access state delay can be disabled
 	 * in via configfs with a value of zero
-- 
2.29.2

[PATCH 5/6] scsi: target: Replace in_interrupt() usage in target_submit_cmd_map_sgls()

[Sebastian Andrzej Siewior]

target_submit_cmd_map_sgls() uses in_interrupt() to crash if it returns
true.

The usage of in_interrupt() in drivers is phased out and Linus clearly
requested that code which changes behaviour depending on context should
either be separated or the context be conveyed in an argument passed by the
caller, which usually knows the context.

The usage of in_interrupt() is clearly for debugging. might_sleep() is
better at this because it also detects other contexts in which it is not
allowed to sleep, like preempt-disabled section.

Replace BUG_ON(in_interrupt) with might_sleep().

Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/target/target_core_transport.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index fca4bd079d02c..b4fdc3f41e90a 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -1613,10 +1613,11 @@ int target_submit_cmd_map_sgls(struct se_cmd *se_cmd, struct se_session *se_sess
 	sense_reason_t rc;
 	int ret;
 
+	might_sleep();
+
 	se_tpg = se_sess->se_tpg;
 	BUG_ON(!se_tpg);
 	BUG_ON(se_cmd->se_tfo || se_cmd->se_sess);
-	BUG_ON(in_interrupt());
 
 	if (flags & TARGET_SCF_USE_CPUID)
 		se_cmd->se_cmd_flags |= SCF_USE_CPUID;
-- 
2.29.2

[PATCH 6/6] scsi: target: Remove in_interrupt() check in transport_handle_cdb_direct()

[Sebastian Andrzej Siewior]

From: "Ahmed S. Darwish" <a.darwish@linutronix.de>

transport_handle_cdb_direct() uses in_interrupt() to detect if it is
safe to sleep. It produces a stack trace and returns with an error which
is clearly for debugging.

The usage of in_interrupt() in drivers is phased out and Linus clearly
requested that code which changes behaviour depending on context should
either be separated or the context be conveyed in an argument passed by the
caller, which usually knows the context.

transport_handle_cdb_direct() has a comment saying that it may only be
invoked from process context. It invokes transport_generic_new_cmd()
which performs GFP_KERNEL memory allocations. in_interrupt() does not
detect all the contexts where it is invalid to sleep (for the blocking
GFP_KERNEL allocation) as it fails to detect sections with disabled
preemption.

Replace the in_interrupt() based check with a might_sleep() annotation.

Signed-off-by: Ahmed S. Darwish <a.darwish@linutronix.de>
Signed-off-by: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
---
 drivers/target/target_core_transport.c | 9 +++------
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/target/target_core_transport.c b/drivers/target/target_core_transport.c
index b4fdc3f41e90a..d47bfd8b0f87c 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -1512,17 +1512,14 @@ int transport_handle_cdb_direct(
 {
 	sense_reason_t ret;
 
+	might_sleep();
+
 	if (!cmd->se_lun) {
 		dump_stack();
 		pr_err("cmd->se_lun is NULL\n");
 		return -EINVAL;
 	}
-	if (in_interrupt()) {
-		dump_stack();
-		pr_err("transport_generic_handle_cdb cannot be called"
-				" from interrupt context\n");
-		return -EINVAL;
-	}
+
 	/*
 	 * Set TRANSPORT_NEW_CMD state and CMD_T_ACTIVE to ensure that
 	 * outstanding descriptors are handled correctly during shutdown via
-- 
2.29.2

Re: [PATCH 0/6] scsi: target: Remove in_interrupt() usage.

[Sebastian Andrzej Siewior]

On 2020-12-20 21:36:32 [+0100], To target-devel@vger.kernel.org wrote:
> Folks,
> 
> in the discussion about preempt count consistency across kernel
> configurations:
> 
>  https://lore.kernel.org/r/20200914204209.256266093@linutronix.de/
> 
> it was concluded that the usage of in_interrupt() and related context
> checks should be removed from non-core code.
> 
> In the long run, usage of 'preemptible, in_*irq etc.' should be banned from
> driver code completely.
> 
> This series addresses the target subsystem.
> Most of in_interrupt() usage is debugging. There is one function which
> either invokes wait_for_completion() or schedules a timer based on
> in_interrupt().

A gentle ping case it got buried somewhere in the merge window / xmas.
My understanding is that nab is out-of-town and target gets routed via
scsi.

Sebastian

Re: [PATCH 0/6] scsi: target: Remove in_interrupt() usage.

[Martin K. Petersen]

Sebastian,

> A gentle ping case it got buried somewhere in the merge window / xmas.

It didn't get any reviews, presumably thanks to being submitted right
before the holidays. However, it looks OK to me.

Applied to 5.12/scsi-staging, thanks!

-- 
Martin K. Petersen	Oracle Linux Engineering

Re: [PATCH 0/6] scsi: target: Remove in_interrupt() usage.

[Martin K. Petersen]

On Sun, 20 Dec 2020 21:36:32 +0100, Sebastian Andrzej Siewior wrote:

> Folks,
> 
> in the discussion about preempt count consistency across kernel
> configurations:
> 
>  https://lore.kernel.org/r/20200914204209.256266093@linutronix.de/
> 
> [...]

Applied to 5.12/scsi-queue, thanks!

[1/6] scsi: target: iscsi: Avoid in_interrupt() usage in iscsit_close_session().
      https://git.kernel.org/mkp/scsi/c/433675486af4
[2/6] scsi: target: iscsi: Avoid in_interrupt() usage in iscsit_check_session_usage_count().
      https://git.kernel.org/mkp/scsi/c/efc9d73063c1
[3/6] scsi: target: iscsi: Redo iscsit_check_session_usage_count() return code.
      https://git.kernel.org/mkp/scsi/c/f88a10f80da9
[4/6] scsi: target: Remove in_interrupt() usage in core_alua_check_nonop_delay()
      https://git.kernel.org/mkp/scsi/c/a97451ac1e34
[5/6] scsi: target: Replace in_interrupt() usage in target_submit_cmd_map_sgls()
      https://git.kernel.org/mkp/scsi/c/513e29946ab2
[6/6] scsi: target: Remove in_interrupt() check in transport_handle_cdb_direct()
      https://git.kernel.org/mkp/scsi/c/bbb087679d5f

-- 
Martin K. Petersen	Oracle Linux Engineering