From: Roman Bolshakov <r.bolshakov@yadro.com> To: linux-scsi@vger.kernel.org, target-devel@vger.kernel.org Cc: linux@yadro.com, Roman Bolshakov <r.bolshakov@yadro.com>, Quinn Tran <qutran@marvell.com>, Bart Van Assche <bvanassche@acm.org>, Thomas Abraham <tabraham@suse.com>, stable@vger.kernel.org, Himanshu Madhani <hmadhani@marvell.com> Subject: [PATCH v2 01/15] scsi: qla2xxx: Ignore NULL pointer in tcm_qla2xxx_free_mcmd Date: Wed, 20 Nov 2019 22:27:09 +0000 [thread overview] Message-ID: <20191120222723.27779-2-r.bolshakov@yadro.com> (raw) In-Reply-To: <20191120222723.27779-1-r.bolshakov@yadro.com> If ABTS cannot be completed in target mode, the driver attempts to free related management command and crashes: NIP [d000000019181ee8] tcm_qla2xxx_free_mcmd+0x40/0x80 [tcm_qla2xxx] LR [d00000001dc1e6f8] qlt_response_pkt+0x190/0xa10 [qla2xxx] Call Trace: [c000003fff27bb50] [c000003fff27bc10] 0xc000003fff27bc10 (unreliable) [c000003fff27bb70] [d00000001dc1e6f8] qlt_response_pkt+0x190/0xa10 [qla2xxx] [c000003fff27bc10] [d00000001dbc2be0] qla24xx_process_response_queue+0x5d8/0xbd0 [qla2xxx] [c000003fff27bd50] [d00000001dbc632c] qla24xx_msix_rsp_q+0x64/0x150 [qla2xxx] [c000003fff27bde0] [c000000000187200] __handle_irq_event_percpu+0x90/0x310 [c000003fff27bea0] [c0000000001874b8] handle_irq_event_percpu+0x38/0x90 [c000003fff27bee0] [c000000000187574] handle_irq_event+0x64/0xb0 [c000003fff27bf10] [c00000000018cd38] handle_fasteoi_irq+0xe8/0x280 [c000003fff27bf40] [c000000000185ccc] generic_handle_irq+0x4c/0x70 [c000003fff27bf60] [c000000000016cec] __do_irq+0x7c/0x1d0 [c000003fff27bf90] [c00000000002a530] call_do_irq+0x14/0x24 [c00000207d2cba90] [c000000000016edc] do_IRQ+0x9c/0x130 [c00000207d2cbae0] [c000000000008bf4] hardware_interrupt_common+0x114/0x120 --- interrupt: 501 at arch_local_irq_restore+0x74/0x90 LR = arch_local_irq_restore+0x74/0x90 [c00000207d2cbdd0] [c0000000001c64fc] tick_broadcast_oneshot_control+0x4c/0x60 (unreliable) [c00000207d2cbdf0] [c0000000007ac840] cpuidle_enter_state+0xf0/0x450 [c00000207d2cbe50] [c00000000016b81c] call_cpuidle+0x4c/0x90 [c00000207d2cbe70] [c00000000016bc30] do_idle+0x2b0/0x330 [c00000207d2cbec0] [c00000000016beec] cpu_startup_entry+0x3c/0x50 [c00000207d2cbef0] [c00000000004a06c] start_secondary+0x63c/0x670 [c00000207d2cbf90] [c00000000000aa6c] start_secondary_prolog+0x10/0x14 The crash can be triggered by ACL deletion when there's active I/O. During ACL deletion, qla2xxx performs implicit LOGO that's invisible for the initiator. Only the driver and firmware are aware of the logout. Therefore the initiator continues to send SCSI commands and the target always responds with SAM STATUS BUSY as it can't find the session. The command times out after a while and initiator invokes ABORT TASK TMF for the command. The TMF is mapped to ABTS-LS in FCP. The target can't find session for S_ID originating ABTS-LS so it never allocates mcmd. And since N_Port handle was deleted after LOGO, it is no longer valid and ABTS Response IOCB is returned from firmware with status 31. Then free_mcmd is invoked on NULL pointer and the kernel crashes. [ 7734.578642] qla2xxx [0000:00:0c.0]-e837:6: ABTS_RECV_24XX: instance 0 [ 7734.578644] qla2xxx [0000:00:0c.0]-f811:6: qla_target(0): task abort (s_id=1:2:0, tag\x1209504, param=0) [ 7734.578645] find_sess_by_s_id: 0x010200 [ 7734.578645] Unable to locate s_id: 0x010200 [ 7734.578646] qla2xxx [0000:00:0c.0]-f812:6: qla_target(0): task abort for non-existent session [ 7734.578648] qla2xxx [0000:00:0c.0]-e806:6: Sending task mgmt ABTS response (ha└000000d5819000, atio└000000d3fd4700, status=4 [ 7734.578730] qla2xxx [0000:00:0c.0]-e838:6: ABTS_RESP_24XX: compl_status 31 [ 7734.578732] qla2xxx [0000:00:0c.0]-e863:6: qla_target(0): ABTS_RESP_24XX failed 31 (subcode 19:a) [ 7734.578740] Unable to handle kernel paging request for data at address 0x00000200 Fixes: 6b0431d6fa20b ("scsi: qla2xxx: Fix out of order Termination and ABTS response") Cc: Quinn Tran <qutran@marvell.com> Cc: Bart Van Assche <bvanassche@acm.org> Cc: Thomas Abraham <tabraham@suse.com> Cc: stable@vger.kernel.org Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com> Acked-by: Himanshu Madhani <hmadhani@marvell.com> --- drivers/scsi/qla2xxx/tcm_qla2xxx.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c b/drivers/scsi/qla2xxx/tcm_qla2xxx.c index 042a24314edc..bab2073c1f72 100644 --- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c +++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c @@ -246,6 +246,8 @@ static void tcm_qla2xxx_complete_mcmd(struct work_struct *work) */ static void tcm_qla2xxx_free_mcmd(struct qla_tgt_mgmt_cmd *mcmd) { + if (!mcmd) + return; INIT_WORK(&mcmd->free_work, tcm_qla2xxx_complete_mcmd); queue_work(tcm_qla2xxx_free_wq, &mcmd->free_work); } -- 2.24.0
WARNING: multiple messages have this Message-ID (diff)
From: Roman Bolshakov <r.bolshakov@yadro.com> To: <linux-scsi@vger.kernel.org>, <target-devel@vger.kernel.org> Cc: <linux@yadro.com>, Roman Bolshakov <r.bolshakov@yadro.com>, Quinn Tran <qutran@marvell.com>, Bart Van Assche <bvanassche@acm.org>, Thomas Abraham <tabraham@suse.com>, <stable@vger.kernel.org>, Himanshu Madhani <hmadhani@marvell.com> Subject: [PATCH v2 01/15] scsi: qla2xxx: Ignore NULL pointer in tcm_qla2xxx_free_mcmd Date: Thu, 21 Nov 2019 01:27:09 +0300 [thread overview] Message-ID: <20191120222723.27779-2-r.bolshakov@yadro.com> (raw) In-Reply-To: <20191120222723.27779-1-r.bolshakov@yadro.com> If ABTS cannot be completed in target mode, the driver attempts to free related management command and crashes: NIP [d000000019181ee8] tcm_qla2xxx_free_mcmd+0x40/0x80 [tcm_qla2xxx] LR [d00000001dc1e6f8] qlt_response_pkt+0x190/0xa10 [qla2xxx] Call Trace: [c000003fff27bb50] [c000003fff27bc10] 0xc000003fff27bc10 (unreliable) [c000003fff27bb70] [d00000001dc1e6f8] qlt_response_pkt+0x190/0xa10 [qla2xxx] [c000003fff27bc10] [d00000001dbc2be0] qla24xx_process_response_queue+0x5d8/0xbd0 [qla2xxx] [c000003fff27bd50] [d00000001dbc632c] qla24xx_msix_rsp_q+0x64/0x150 [qla2xxx] [c000003fff27bde0] [c000000000187200] __handle_irq_event_percpu+0x90/0x310 [c000003fff27bea0] [c0000000001874b8] handle_irq_event_percpu+0x38/0x90 [c000003fff27bee0] [c000000000187574] handle_irq_event+0x64/0xb0 [c000003fff27bf10] [c00000000018cd38] handle_fasteoi_irq+0xe8/0x280 [c000003fff27bf40] [c000000000185ccc] generic_handle_irq+0x4c/0x70 [c000003fff27bf60] [c000000000016cec] __do_irq+0x7c/0x1d0 [c000003fff27bf90] [c00000000002a530] call_do_irq+0x14/0x24 [c00000207d2cba90] [c000000000016edc] do_IRQ+0x9c/0x130 [c00000207d2cbae0] [c000000000008bf4] hardware_interrupt_common+0x114/0x120 --- interrupt: 501 at arch_local_irq_restore+0x74/0x90 LR = arch_local_irq_restore+0x74/0x90 [c00000207d2cbdd0] [c0000000001c64fc] tick_broadcast_oneshot_control+0x4c/0x60 (unreliable) [c00000207d2cbdf0] [c0000000007ac840] cpuidle_enter_state+0xf0/0x450 [c00000207d2cbe50] [c00000000016b81c] call_cpuidle+0x4c/0x90 [c00000207d2cbe70] [c00000000016bc30] do_idle+0x2b0/0x330 [c00000207d2cbec0] [c00000000016beec] cpu_startup_entry+0x3c/0x50 [c00000207d2cbef0] [c00000000004a06c] start_secondary+0x63c/0x670 [c00000207d2cbf90] [c00000000000aa6c] start_secondary_prolog+0x10/0x14 The crash can be triggered by ACL deletion when there's active I/O. During ACL deletion, qla2xxx performs implicit LOGO that's invisible for the initiator. Only the driver and firmware are aware of the logout. Therefore the initiator continues to send SCSI commands and the target always responds with SAM STATUS BUSY as it can't find the session. The command times out after a while and initiator invokes ABORT TASK TMF for the command. The TMF is mapped to ABTS-LS in FCP. The target can't find session for S_ID originating ABTS-LS so it never allocates mcmd. And since N_Port handle was deleted after LOGO, it is no longer valid and ABTS Response IOCB is returned from firmware with status 31. Then free_mcmd is invoked on NULL pointer and the kernel crashes. [ 7734.578642] qla2xxx [0000:00:0c.0]-e837:6: ABTS_RECV_24XX: instance 0 [ 7734.578644] qla2xxx [0000:00:0c.0]-f811:6: qla_target(0): task abort (s_id=1:2:0, tag=1209504, param=0) [ 7734.578645] find_sess_by_s_id: 0x010200 [ 7734.578645] Unable to locate s_id: 0x010200 [ 7734.578646] qla2xxx [0000:00:0c.0]-f812:6: qla_target(0): task abort for non-existent session [ 7734.578648] qla2xxx [0000:00:0c.0]-e806:6: Sending task mgmt ABTS response (ha=c0000000d5819000, atio=c0000000d3fd4700, status=4 [ 7734.578730] qla2xxx [0000:00:0c.0]-e838:6: ABTS_RESP_24XX: compl_status 31 [ 7734.578732] qla2xxx [0000:00:0c.0]-e863:6: qla_target(0): ABTS_RESP_24XX failed 31 (subcode 19:a) [ 7734.578740] Unable to handle kernel paging request for data at address 0x00000200 Fixes: 6b0431d6fa20b ("scsi: qla2xxx: Fix out of order Termination and ABTS response") Cc: Quinn Tran <qutran@marvell.com> Cc: Bart Van Assche <bvanassche@acm.org> Cc: Thomas Abraham <tabraham@suse.com> Cc: stable@vger.kernel.org Signed-off-by: Roman Bolshakov <r.bolshakov@yadro.com> Acked-by: Himanshu Madhani <hmadhani@marvell.com> --- drivers/scsi/qla2xxx/tcm_qla2xxx.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/scsi/qla2xxx/tcm_qla2xxx.c b/drivers/scsi/qla2xxx/tcm_qla2xxx.c index 042a24314edc..bab2073c1f72 100644 --- a/drivers/scsi/qla2xxx/tcm_qla2xxx.c +++ b/drivers/scsi/qla2xxx/tcm_qla2xxx.c @@ -246,6 +246,8 @@ static void tcm_qla2xxx_complete_mcmd(struct work_struct *work) */ static void tcm_qla2xxx_free_mcmd(struct qla_tgt_mgmt_cmd *mcmd) { + if (!mcmd) + return; INIT_WORK(&mcmd->free_work, tcm_qla2xxx_complete_mcmd); queue_work(tcm_qla2xxx_free_wq, &mcmd->free_work); } -- 2.24.0
next prev parent reply other threads:[~2019-11-20 22:27 UTC|newest] Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-11-20 22:27 [PATCH v2 00/15] scsi: qla2xxx: Bug fixes Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov [this message] 2019-11-20 22:27 ` [PATCH v2 01/15] scsi: qla2xxx: Ignore NULL pointer in tcm_qla2xxx_free_mcmd Roman Bolshakov 2019-11-20 22:27 ` [PATCH v2 02/15] scsi: qla2xxx: Initialize free_work before flushing it Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 22:27 ` [PATCH v2 03/15] scsi: qla2xxx: Drop superfluous INIT_WORK of del_work Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 22:27 ` [PATCH v2 04/15] scsi: qla2xxx: Change discovery state before PLOGI Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 22:27 ` [PATCH v2 05/15] scsi: qla2xxx: Allow PLOGI in target mode Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 22:52 ` [EXT] " Quinn Tran 2019-11-21 16:38 ` Himanshu Madhani 2019-11-21 16:38 ` Himanshu Madhani 2019-11-20 22:27 ` [PATCH v2 06/15] scsi: qla2xxx: Don't call qlt_async_event twice Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-21 15:43 ` Himanshu Madhani 2019-11-21 15:43 ` Himanshu Madhani 2019-11-20 22:27 ` [PATCH v2 07/15] scsi: qla2xxx: Fix PLOGI payload and ELS IOCB dump length Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-21 16:39 ` [EXT] " Himanshu Madhani 2019-11-21 16:39 ` Himanshu Madhani 2019-11-20 22:27 ` [PATCH v2 08/15] scsi: qla2xxx: Configure local loop for N2N target Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 23:06 ` [EXT] " Quinn Tran 2019-11-21 16:39 ` Himanshu Madhani 2019-11-21 16:39 ` Himanshu Madhani 2019-11-20 22:27 ` [PATCH v2 09/15] scsi: qla2xxx: Send Notify ACK after N2N PLOGI Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 23:53 ` [EXT] " Quinn Tran 2019-11-21 16:40 ` Himanshu Madhani 2019-11-21 16:40 ` Himanshu Madhani 2019-11-20 22:27 ` [PATCH v2 10/15] scsi: qla2xxx: Don't defer relogin unconditonally Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-21 0:03 ` [EXT] " Quinn Tran 2019-11-20 22:27 ` [PATCH v2 11/15] scsi: qla2xxx: Ignore PORT UPDATE after N2N PLOGI Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-20 22:27 ` [PATCH v2 12/15] scsi: qla2xxx: Use explicit LOGO in target mode Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-23 4:58 ` kbuild test robot 2019-11-23 4:58 ` kbuild test robot 2019-11-23 4:58 ` kbuild test robot 2019-11-20 22:27 ` [PATCH v2 13/15] scsi: qla2xxx: Add async mode for qla24xx_els_dcmd_iocb Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-21 22:50 ` [EXT] " Quinn Tran 2019-11-22 5:04 ` Mark Harvey 2019-11-22 5:04 ` Mark Harvey 2019-11-22 17:00 ` Quinn Tran 2019-11-22 17:00 ` Quinn Tran 2019-11-24 19:15 ` Roman Bolshakov 2019-11-24 19:15 ` Roman Bolshakov 2019-11-20 22:27 ` [PATCH v2 14/15] scsi: qla2xxx: Add debug dump of LOGO payload and ELS IOCB Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-21 16:37 ` Himanshu Madhani 2019-11-21 16:37 ` Himanshu Madhani 2019-11-21 22:52 ` [EXT] " Quinn Tran 2019-11-20 22:27 ` [PATCH v2 15/15] scsi: qla2xxx: Handle ABTS according to FCP spec for logged out ports Roman Bolshakov 2019-11-20 22:27 ` Roman Bolshakov 2019-11-23 5:57 ` kbuild test robot 2019-11-23 5:57 ` kbuild test robot 2019-11-23 5:57 ` kbuild test robot 2019-11-22 9:14 ` [PATCH v2 00/15] scsi: qla2xxx: Bug fixes Hannes Reinecke 2019-11-22 9:14 ` Hannes Reinecke 2019-11-22 21:36 ` Martin Wilck 2019-11-22 21:36 ` Martin Wilck 2019-11-24 18:31 ` Roman Bolshakov 2019-11-24 18:31 ` Roman Bolshakov
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20191120222723.27779-2-r.bolshakov@yadro.com \ --to=r.bolshakov@yadro.com \ --cc=bvanassche@acm.org \ --cc=hmadhani@marvell.com \ --cc=linux-scsi@vger.kernel.org \ --cc=linux@yadro.com \ --cc=qutran@marvell.com \ --cc=stable@vger.kernel.org \ --cc=tabraham@suse.com \ --cc=target-devel@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.