From: Maurizio Lombardi <mlombard@redhat.com> To: martin.petersen@oracle.com Cc: linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, bvanassche@acm.org, michael.christie@oracle.com Subject: [PATCH 2/2] target: iscsi: fix a race condition when aborting a task Date: Wed, 07 Oct 2020 14:53:26 +0000 [thread overview] Message-ID: <20201007145326.56850-3-mlombard@redhat.com> (raw) In-Reply-To: <20201007145326.56850-1-mlombard@redhat.com> The iscsit_release_commands_from_conn() function does the following operations: 1) locks the cmd_lock spinlock 2) Scans the list of commands and sets the CMD_T_FABRIC_STOP flag 3) Releases the cmd_lock spinlock 4) Rescans the list again and clears the i_conn_node link of each command If an abort task timer is fired between 3) and 4), it will find the CMD_T_FABRIC_STOP flag set and won't call list_del_init(); therefore it may end up calling __iscsit_free_cmd() with a non-empty i_conn_node list, thus triggering the warning. Considering that: - we expect list_del_init() to be executed by iscsit_release_commands_from_conn() when the CMD_T_FABRIC_STOP is set. - iscsit_aborted_task() is the only function that calls __iscsit_free_cmd() directly, while all the other functions call iscsit_free_cmd(). - the warning in __iscsit_free_cmd() is a duplicate (the same warning can be found in iscsit_free_cmd(). We can fix the bug by simply removing the warning from __iscsit_free_cmd() kernel: ------------[ cut here ]------------ kernel: WARNING: CPU: 1 PID: 21173 at drivers/target/iscsi/iscsi_target_util.c:720 __iscsit_free_cmd+0x26e/0x290 [iscsi_target_mod] ... kernel: CPU: 1 PID: 21173 Comm: kworker/u8:3 Kdump: loaded Not tainted 3.10.0-1062.4.1.el7.x86_64 #1 kernel: Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015 kernel: Workqueue: tmr-user target_tmr_work [target_core_mod] kernel: Call Trace: kernel: [<ffffffff91d78ba4>] dump_stack+0x19/0x1b kernel: [<ffffffff9169a958>] __warn+0xd8/0x100 kernel: [<ffffffff9169aa9d>] warn_slowpath_null+0x1d/0x20 kernel: [<ffffffffc0a6f69e>] __iscsit_free_cmd+0x26e/0x290 [iscsi_target_mod] kernel: [<ffffffffc0a70bc4>] iscsit_aborted_task+0x64/0x70 [iscsi_target_mod] kernel: [<ffffffffc0a7830a>] lio_aborted_task+0x2a/0x30 [iscsi_target_mod] kernel: [<ffffffffc09fa516>] transport_cmd_finish_abort+0x66/0xb0 [target_core_mod] kernel: [<ffffffffc09f4d92>] core_tmr_abort_task+0x102/0x180 [target_core_mod] kernel: [<ffffffffc09f7bb2>] target_tmr_work+0x152/0x170 [target_core_mod] kernel: [<ffffffff916bd1df>] process_one_work+0x17f/0x440 kernel: [<ffffffff916be2f6>] worker_thread+0x126/0x3c0 kernel: [<ffffffff916be1d0>] ? manage_workers.isra.26+0x2a0/0x2a0 kernel: [<ffffffff916c51b1>] kthread+0xd1/0xe0 kernel: [<ffffffff916c50e0>] ? insert_kthread_work+0x40/0x40 kernel: [<ffffffff91d8bd37>] ret_from_fork_nospec_begin+0x21/0x21 kernel: [<ffffffff916c50e0>] ? insert_kthread_work+0x40/0x40 kernel: ---[ end trace ed2119501826ec7a ]--- Signed-off-by: Maurizio Lombardi <mlombard@redhat.com> --- drivers/target/iscsi/iscsi_target_util.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c index 3082f5bde9fa..61233755ece0 100644 --- a/drivers/target/iscsi/iscsi_target_util.c +++ b/drivers/target/iscsi/iscsi_target_util.c @@ -741,8 +741,6 @@ void __iscsit_free_cmd(struct iscsi_cmd *cmd, bool check_queues) { struct iscsi_conn *conn = cmd->conn; - WARN_ON(!list_empty(&cmd->i_conn_node)); - if (cmd->data_direction = DMA_TO_DEVICE) { iscsit_stop_dataout_timer(cmd); iscsit_free_r2ts_from_list(cmd); -- 2.26.2
WARNING: multiple messages have this Message-ID (diff)
From: Maurizio Lombardi <mlombard@redhat.com> To: martin.petersen@oracle.com Cc: linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, bvanassche@acm.org, michael.christie@oracle.com Subject: [PATCH 2/2] target: iscsi: fix a race condition when aborting a task Date: Wed, 7 Oct 2020 16:53:26 +0200 [thread overview] Message-ID: <20201007145326.56850-3-mlombard@redhat.com> (raw) In-Reply-To: <20201007145326.56850-1-mlombard@redhat.com> The iscsit_release_commands_from_conn() function does the following operations: 1) locks the cmd_lock spinlock 2) Scans the list of commands and sets the CMD_T_FABRIC_STOP flag 3) Releases the cmd_lock spinlock 4) Rescans the list again and clears the i_conn_node link of each command If an abort task timer is fired between 3) and 4), it will find the CMD_T_FABRIC_STOP flag set and won't call list_del_init(); therefore it may end up calling __iscsit_free_cmd() with a non-empty i_conn_node list, thus triggering the warning. Considering that: - we expect list_del_init() to be executed by iscsit_release_commands_from_conn() when the CMD_T_FABRIC_STOP is set. - iscsit_aborted_task() is the only function that calls __iscsit_free_cmd() directly, while all the other functions call iscsit_free_cmd(). - the warning in __iscsit_free_cmd() is a duplicate (the same warning can be found in iscsit_free_cmd(). We can fix the bug by simply removing the warning from __iscsit_free_cmd() kernel: ------------[ cut here ]------------ kernel: WARNING: CPU: 1 PID: 21173 at drivers/target/iscsi/iscsi_target_util.c:720 __iscsit_free_cmd+0x26e/0x290 [iscsi_target_mod] ... kernel: CPU: 1 PID: 21173 Comm: kworker/u8:3 Kdump: loaded Not tainted 3.10.0-1062.4.1.el7.x86_64 #1 kernel: Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 09/17/2015 kernel: Workqueue: tmr-user target_tmr_work [target_core_mod] kernel: Call Trace: kernel: [<ffffffff91d78ba4>] dump_stack+0x19/0x1b kernel: [<ffffffff9169a958>] __warn+0xd8/0x100 kernel: [<ffffffff9169aa9d>] warn_slowpath_null+0x1d/0x20 kernel: [<ffffffffc0a6f69e>] __iscsit_free_cmd+0x26e/0x290 [iscsi_target_mod] kernel: [<ffffffffc0a70bc4>] iscsit_aborted_task+0x64/0x70 [iscsi_target_mod] kernel: [<ffffffffc0a7830a>] lio_aborted_task+0x2a/0x30 [iscsi_target_mod] kernel: [<ffffffffc09fa516>] transport_cmd_finish_abort+0x66/0xb0 [target_core_mod] kernel: [<ffffffffc09f4d92>] core_tmr_abort_task+0x102/0x180 [target_core_mod] kernel: [<ffffffffc09f7bb2>] target_tmr_work+0x152/0x170 [target_core_mod] kernel: [<ffffffff916bd1df>] process_one_work+0x17f/0x440 kernel: [<ffffffff916be2f6>] worker_thread+0x126/0x3c0 kernel: [<ffffffff916be1d0>] ? manage_workers.isra.26+0x2a0/0x2a0 kernel: [<ffffffff916c51b1>] kthread+0xd1/0xe0 kernel: [<ffffffff916c50e0>] ? insert_kthread_work+0x40/0x40 kernel: [<ffffffff91d8bd37>] ret_from_fork_nospec_begin+0x21/0x21 kernel: [<ffffffff916c50e0>] ? insert_kthread_work+0x40/0x40 kernel: ---[ end trace ed2119501826ec7a ]--- Signed-off-by: Maurizio Lombardi <mlombard@redhat.com> --- drivers/target/iscsi/iscsi_target_util.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/drivers/target/iscsi/iscsi_target_util.c b/drivers/target/iscsi/iscsi_target_util.c index 3082f5bde9fa..61233755ece0 100644 --- a/drivers/target/iscsi/iscsi_target_util.c +++ b/drivers/target/iscsi/iscsi_target_util.c @@ -741,8 +741,6 @@ void __iscsit_free_cmd(struct iscsi_cmd *cmd, bool check_queues) { struct iscsi_conn *conn = cmd->conn; - WARN_ON(!list_empty(&cmd->i_conn_node)); - if (cmd->data_direction == DMA_TO_DEVICE) { iscsit_stop_dataout_timer(cmd); iscsit_free_r2ts_from_list(cmd); -- 2.26.2
next prev parent reply other threads:[~2020-10-07 14:53 UTC|newest] Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-10-07 14:53 [PATCH 0/2] fix race conditions with task aborts Maurizio Lombardi 2020-10-07 14:53 ` Maurizio Lombardi 2020-10-07 14:53 ` [PATCH 1/2] target: iscsi: prevent a race condition in iscsit_unmap_cmd() Maurizio Lombardi 2020-10-07 14:53 ` Maurizio Lombardi 2020-10-08 2:15 ` Bart Van Assche 2020-10-08 2:15 ` Bart Van Assche 2020-10-08 9:42 ` Maurizio Lombardi 2020-10-08 9:42 ` Maurizio Lombardi 2020-10-07 14:53 ` Maurizio Lombardi [this message] 2020-10-07 14:53 ` [PATCH 2/2] target: iscsi: fix a race condition when aborting a task Maurizio Lombardi 2020-10-22 2:42 ` Mike Christie 2020-10-22 2:42 ` Mike Christie 2020-10-27 13:49 ` Maurizio Lombardi 2020-10-27 13:49 ` Maurizio Lombardi 2020-10-27 17:54 ` Mike Christie 2020-10-27 17:54 ` Mike Christie 2020-10-27 20:03 ` Michael Christie 2020-10-27 20:03 ` Michael Christie 2020-10-28 17:09 ` Maurizio Lombardi 2020-10-28 17:09 ` Maurizio Lombardi 2020-10-28 20:37 ` Mike Christie 2020-10-28 20:37 ` Mike Christie 2020-11-10 21:29 ` Maurizio Lombardi 2020-11-10 21:29 ` Maurizio Lombardi 2020-11-10 23:08 ` Mike Christie 2020-11-10 23:08 ` Mike Christie 2020-11-11 2:16 ` Mike Christie 2020-11-11 2:16 ` Mike Christie 2020-11-11 14:58 ` Maurizio Lombardi 2020-11-11 14:58 ` Maurizio Lombardi 2020-11-11 15:37 ` Michael Christie 2020-11-11 15:37 ` Michael Christie 2020-11-11 15:48 ` Maurizio Lombardi 2020-11-11 15:48 ` Maurizio Lombardi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20201007145326.56850-3-mlombard@redhat.com \ --to=mlombard@redhat.com \ --cc=bvanassche@acm.org \ --cc=linux-scsi@vger.kernel.org \ --cc=martin.petersen@oracle.com \ --cc=michael.christie@oracle.com \ --cc=target-devel@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.