* [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
[not found] <347186099.14151179.1614497065881.JavaMail.zimbra@redhat.com>
@ 2021-02-28 9:49 ` Yi Zhang
2021-02-28 19:07 ` Chaitanya Kulkarni
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Yi Zhang @ 2021-02-28 9:49 UTC (permalink / raw)
To: linux-scsi, linux-rdma, linux-block
Hello
I found this issue with blktests srp/015, could anyone help check it?
[ 250.415156] run blktests srp/015 at 2021-02-28 09:23:02
[ 250.525538] eno2 speed is unknown, defaulting to 1000
[ 250.530599] eno2 speed is unknown, defaulting to 1000
[ 250.535665] eno2 speed is unknown, defaulting to 1000
[ 250.541768] eno3 speed is unknown, defaulting to 1000
[ 250.546821] eno3 speed is unknown, defaulting to 1000
[ 250.551881] eno3 speed is unknown, defaulting to 1000
[ 250.557844] eno4 speed is unknown, defaulting to 1000
[ 250.562912] eno4 speed is unknown, defaulting to 1000
[ 250.567978] eno4 speed is unknown, defaulting to 1000
[ 250.573945] lo speed is unknown, defaulting to 1000
[ 250.578827] lo speed is unknown, defaulting to 1000
[ 250.583712] lo speed is unknown, defaulting to 1000
[ 250.605512] sd 15:0:0:0: Power-on or device reset occurred
[ 250.673453] device-mapper: table: 253:3: multipath: error getting device
[ 250.680160] device-mapper: ioctl: error adding target to table
[ 250.732648] eno2 speed is unknown, defaulting to 1000
[ 250.737724] eno3 speed is unknown, defaulting to 1000
[ 250.742784] eno4 speed is unknown, defaulting to 1000
[ 250.747843] lo speed is unknown, defaulting to 1000
[ 250.752725] ib_srpt MAD registration failed for lo_siw-1.
[ 251.106511] scsi host16: REJ reason 0xffffff98
[ 251.111150] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
[ 251.246254] sd 16:0:0:0: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatical
[ 251.260567] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 251.260765] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 251.280135] sd 16:0:0:1: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatical
[ 251.298968] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 252.505389] ext4 filesystem being mounted at /root/blktests/results/tmpdir.srp.015.63f/mnt0 supports timestamps until 2038 (0x7fffffff)
[ 253.522565] device-mapper: table: 253:6: multipath: error getting device
[ 253.529277] device-mapper: ioctl: error adding target to table
[ 257.526935] device-mapper: multipath: 253:4: Failing path 8:32.
[ 262.542076] scsi host16: REJ reason 0xffffff98
[ 262.546708] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
[ 262.671752] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 262.672079] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 262.692394] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 267.804429] device-mapper: multipath: 253:4: Failing path 8:32.
[ 267.882487] srpt_recv_done: 502 callbacks suppressed
[ 269.819904] scsi host16: REJ reason 0xffffff98
[ 269.824533] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
[ 269.951714] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 269.952049] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 269.972080] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 275.085527] device-mapper: multipath: 253:4: Failing path 8:32.
[ 275.152684] srpt_recv_done: 502 callbacks suppressed
[ 277.100827] scsi host16: REJ reason 0xffffff98
[ 277.105452] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
[ 277.231922] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 277.242758] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 277.252505] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 282.353149] device-mapper: multipath: 253:4: Failing path 8:32.
[ 282.419429] srpt_recv_done: 502 callbacks suppressed
[ 282.842310] scsi host17: REJ reason 0xffffff98
[ 282.846939] scsi host17: ib_srp: Connection 0/4 to 10.16.221.116 failed
[ 282.981432] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 282.991501] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 283.001358] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
[ 284.690990] device-mapper: multipath: 253:4: Failing path 8:96.
[ 285.493848] BUG: unable to handle page fault for address: ffffffffc0a89150
[ 285.500724] #PF: supervisor read access in kernel mode
[ 285.505863] #PF: error_code(0x0000) - not-present page
[ 285.511001] PGD 37bc13067 P4D 37bc13067 PUD 37bc15067 PMD 13839d067 PTE 0
[ 285.517789] Oops: 0000 [#1] SMP NOPTI
[ 285.521454] CPU: 31 PID: 17943 Comm: multipathd Tainted: G S I 5.11.0 #4
[ 285.529279] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
[ 285.536837] RIP: 0010:scsi_mq_exit_request+0x14/0x50
[ 285.541804] Code: 00 00 e9 bf eb 58 00 e9 ea bc e0 ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 8b 7f 60 48 89 f3 48 8b 87 98 00 00 00 <48> 8b 40 30 48 85 c0 74 0c 48 8d b6 10 01 00 00 e8 87 eb 58 00 f6
[ 285.560548] RSP: 0018:ffffa66e82e37c98 EFLAGS: 00010286
[ 285.565774] RAX: ffffffffc0a89120 RBX: ffff8a0e8a3e0000 RCX: 0000000000000000
[ 285.572906] RDX: 0000000000000000 RSI: ffff8a0e8a3e0000 RDI: ffff8a0e692a1000
[ 285.580038] RBP: ffff8a0ea8469800 R08: 000000000000020d R09: 000000000002a780
[ 285.587170] R10: 00000376b4157308 R11: 0000000000000000 R12: 0000000000000000
[ 285.594304] R13: ffff8a0e692a10a8 R14: 0000000000000000 R15: ffff8a0e9244b700
[ 285.601435] FS: 00007f616c291700(0000) GS:ffff8a1d801c0000(0000) knlGS:0000000000000000
[ 285.609521] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 285.615269] CR2: ffffffffc0a89150 CR3: 000000011e43e003 CR4: 00000000007706e0
[ 285.622401] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 285.629532] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 285.636664] PKRU: 55555554
[ 285.639369] Call Trace:
[ 285.641824] blk_mq_free_rqs+0x59/0xd0
[ 285.645573] blk_mq_free_map_and_requests+0x31/0x60
[ 285.650454] blk_mq_free_tag_set+0x22/0x80
[ 285.654553] scsi_host_dev_release+0x86/0xe0
[ 285.658826] device_release+0x33/0x90
[ 285.662490] kobject_release+0x46/0x150
[ 285.666332] device_release+0x33/0x90
[ 285.669995] kobject_release+0x46/0x150
[ 285.673836] execute_in_process_context+0x21/0x60
[ 285.678542] device_release+0x33/0x90
[ 285.682208] kobject_release+0x46/0x150
[ 285.686046] scsi_disk_put+0x2b/0x40 [sd_mod]
[ 285.690406] __blkdev_put+0x186/0x1b0
[ 285.694072] blkdev_put+0x4c/0x110
[ 285.697476] blkdev_close+0x21/0x30
[ 285.700970] __fput+0x92/0x230
[ 285.704029] task_work_run+0x70/0xb0
[ 285.707608] exit_to_user_mode_prepare+0x150/0x160
[ 285.712402] syscall_exit_to_user_mode+0x12/0x40
[ 285.717021] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 285.722071] RIP: 0033:0x7f616b1ce977
[ 285.725652] Code: 12 b8 03 00 00 00 0f 05 48 3d 00 f0 ff ff 77 3b c3 66 90 53 89 fb 48 83 ec 10 e8 e4 fb ff ff 89 df 89 c2 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2b 89 d7 89 44 24 0c e8 26 fc ff ff 8b 44 24
[ 285.744397] RSP: 002b:00007f616c28f7b0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
[ 285.751962] RAX: 0000000000000000 RBX: 000000000000000b RCX: 00007f616b1ce977
[ 285.759094] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 000000000000000b
[ 285.766227] RBP: 00005588a853efb0 R08: 0000000000000001 R09: 0000000000000007
[ 285.773359] R10: 0000000000000000 R11: 0000000000000293 R12: 00005588a8215b19
[ 285.780492] R13: 00007f615401d630 R14: 0000000000000001 R15: 00007f615401d630
[ 285.787624] Modules linked in: target_core_user uio target_core_pscsi target_core_file ib_srpt target_core_iblock target_core_mod scsi_debug siw null_blk ext4 mbcache jbd2 rpcrdma rdma_ucm ib_umad ib_iser libiscsi scsi_transport_iscsi rdma_cm iw_cm ib_cm ib_uverbs ib_core rfkill sunrpc vfat fat dm_service_time dm_multipath intel_rapl_msr intel_rapl_common isst_if_common skx_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm mgag200 i2c_algo_bit drm_kms_helper iTCO_wdt iTCO_vendor_support irqbypass dcdbas syscopyarea crct10dif_pclmul sysfillrect crc32_pclmul sysimgblt fb_sys_fops ghash_clmulni_intel drm acpi_ipmi rapl ipmi_si intel_cstate dell_smbios dax_pmem_compat mei_me i2c_i801 ipmi_devintf device_dax intel_uncore mei wmi_bmof dell_wmi_descriptor pcspkr intel_pch_thermal lpc_ich i2c_smbus ipmi_msghandler dax_pmem_core acpi_power_meter ip_tables xfs libcrc32c nd_pmem nd_btt sd_mod t10_pi sg ahci libahci nfit megaraid_sas libata tg3 crc32c_intel libnvdimm wmi
[ 285.787672] dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_transport_srp]
[ 285.882703] CR2: ffffffffc0a89150
[ 285.886032] ---[ end trace 718c96f79b0576a6 ]---
[ 285.896061] RIP: 0010:scsi_mq_exit_request+0x14/0x50
[ 285.901024] Code: 00 00 e9 bf eb 58 00 e9 ea bc e0 ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 8b 7f 60 48 89 f3 48 8b 87 98 00 00 00 <48> 8b 40 30 48 85 c0 74 0c 48 8d b6 10 01 00 00 e8 87 eb 58 00 f6
[ 285.919768] RSP: 0018:ffffa66e82e37c98 EFLAGS: 00010286
[ 285.924995] RAX: ffffffffc0a89120 RBX: ffff8a0e8a3e0000 RCX: 0000000000000000
[ 285.932128] RDX: 0000000000000000 RSI: ffff8a0e8a3e0000 RDI: ffff8a0e692a1000
[ 285.939260] RBP: ffff8a0ea8469800 R08: 000000000000020d R09: 000000000002a780
[ 285.946394] R10: 00000376b4157308 R11: 0000000000000000 R12: 0000000000000000
[ 285.953524] R13: ffff8a0e692a10a8 R14: 0000000000000000 R15: ffff8a0e9244b700
[ 285.960658] FS: 00007f616c291700(0000) GS:ffff8a1d801c0000(0000) knlGS:0000000000000000
[ 285.968742] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 285.974489] CR2: ffffffffc0a89150 CR3: 000000011e43e003 CR4: 00000000007706e0
[ 285.981623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 285.988753] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 285.995885] PKRU: 55555554
[ 285.998590] Kernel panic - not syncing: Fatal exception
[ 286.504449] Kernel Offset: 0x2c200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[ 286.520574] ---[ end Kernel panic - not syncing: Fatal exception ]---
(gdb) l *(scsi_mq_exit_request+0x14)
0xffffffff81673784 is in scsi_mq_exit_request (drivers/scsi/scsi_lib.c:1785).
1780 unsigned int hctx_idx)
1781 {
1782 struct Scsi_Host *shost = set->driver_data;
1783 struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(rq);
1784
1785 if (shost->hostt->exit_cmd_priv)
1786 shost->hostt->exit_cmd_priv(shost, cmd);
1787 scsi_free_sense_buffer(cmd->flags & SCMD_UNCHECKED_ISA_DMA,
1788 cmd->sense_buffer);
1789 }
(gdb)
Best Regards,
Yi Zhang
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-02-28 9:49 ` [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015 Yi Zhang
@ 2021-02-28 19:07 ` Chaitanya Kulkarni
2021-03-01 2:54 ` Yi Zhang
2021-03-01 15:36 ` Yi Zhang
2021-02-28 22:04 ` Bart Van Assche
2021-03-03 15:16 ` Ming Lei
2 siblings, 2 replies; 9+ messages in thread
From: Chaitanya Kulkarni @ 2021-02-28 19:07 UTC (permalink / raw)
To: Yi Zhang, linux-scsi, linux-rdma, linux-block
On 2/28/21 01:52, Yi Zhang wrote:
> Hello
>
> I found this issue with blktests srp/015, could anyone help check it?
Until you get some reply you can try and bisect it.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-02-28 9:49 ` [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015 Yi Zhang
2021-02-28 19:07 ` Chaitanya Kulkarni
@ 2021-02-28 22:04 ` Bart Van Assche
2021-03-01 2:56 ` Yi Zhang
2021-03-03 15:16 ` Ming Lei
2 siblings, 1 reply; 9+ messages in thread
From: Bart Van Assche @ 2021-02-28 22:04 UTC (permalink / raw)
To: Yi Zhang, linux-scsi, linux-rdma, linux-block
On 2/28/21 1:49 AM, Yi Zhang wrote:
> I found this issue with blktests srp/015, could anyone help check it?
Which kernel tree has been used in your tests? One of Linus' trees or a
for-next tree from a kernel maintainer?
Bart.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-02-28 19:07 ` Chaitanya Kulkarni
@ 2021-03-01 2:54 ` Yi Zhang
2021-03-01 15:36 ` Yi Zhang
1 sibling, 0 replies; 9+ messages in thread
From: Yi Zhang @ 2021-03-01 2:54 UTC (permalink / raw)
To: Chaitanya Kulkarni, linux-scsi, linux-rdma, linux-block
Sure, will do.
On 3/1/21 3:07 AM, Chaitanya Kulkarni wrote:
> On 2/28/21 01:52, Yi Zhang wrote:
>> Hello
>>
>> I found this issue with blktests srp/015, could anyone help check it?
> Until you get some reply you can try and bisect it.
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-02-28 22:04 ` Bart Van Assche
@ 2021-03-01 2:56 ` Yi Zhang
0 siblings, 0 replies; 9+ messages in thread
From: Yi Zhang @ 2021-03-01 2:56 UTC (permalink / raw)
To: Bart Van Assche, linux-scsi, linux-rdma, linux-block
On 3/1/21 6:04 AM, Bart Van Assche wrote:
> On 2/28/21 1:49 AM, Yi Zhang wrote:
>> I found this issue with blktests srp/015, could anyone help check it?
> Which kernel tree has been used in your tests? One of Linus' trees or a
> for-next tree from a kernel maintainer?
I'm using "tag: v5.11" on Linus' tree.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-02-28 19:07 ` Chaitanya Kulkarni
2021-03-01 2:54 ` Yi Zhang
@ 2021-03-01 15:36 ` Yi Zhang
2021-03-02 3:59 ` Bart Van Assche
1 sibling, 1 reply; 9+ messages in thread
From: Yi Zhang @ 2021-03-01 15:36 UTC (permalink / raw)
To: Chaitanya Kulkarni, linux-scsi, linux-rdma, linux-block
This issue cannot be reproduced on latest 5.12.0-rc1.
Please ignore this report, sorry for the noise.
On 3/1/21 3:07 AM, Chaitanya Kulkarni wrote:
> On 2/28/21 01:52, Yi Zhang wrote:
>> Hello
>>
>> I found this issue with blktests srp/015, could anyone help check it?
> Until you get some reply you can try and bisect it.
>
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-03-01 15:36 ` Yi Zhang
@ 2021-03-02 3:59 ` Bart Van Assche
2021-03-03 11:40 ` Yi Zhang
0 siblings, 1 reply; 9+ messages in thread
From: Bart Van Assche @ 2021-03-02 3:59 UTC (permalink / raw)
To: Yi Zhang, Chaitanya Kulkarni, linux-scsi, linux-rdma, linux-block
On 3/1/21 7:36 AM, Yi Zhang wrote:
> This issue cannot be reproduced on latest 5.12.0-rc1.
>
> Please ignore this report, sorry for the noise.
How about rerunning the same test against v5.11.2, the latest v5.11
stable kernel? I think your report means that v5.11 can be improved...
Thanks,
Bart.
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-03-02 3:59 ` Bart Van Assche
@ 2021-03-03 11:40 ` Yi Zhang
0 siblings, 0 replies; 9+ messages in thread
From: Yi Zhang @ 2021-03-03 11:40 UTC (permalink / raw)
To: Bart Van Assche, Chaitanya Kulkarni, linux-scsi, linux-rdma, linux-block
On 3/2/21 11:59 AM, Bart Van Assche wrote:
> On 3/1/21 7:36 AM, Yi Zhang wrote:
>> This issue cannot be reproduced on latest 5.12.0-rc1.
>>
>> Please ignore this report, sorry for the noise.
> How about rerunning the same test against v5.11.2, the latest v5.11
> stable kernel? I think your report means that v5.11 can be improved...
I tried on stable kernel 5.11.2/5.10.19, and all reproduced, here is the log
5.11.2 https://pastebin.com/hG4iBETG
5.10.19 https://pastebin.com/ExFbQMdg
It' not 100% reproduced and sometimes can be reproduced within 20 times'
testing.
I tried bisect, but unfortunately my attempts don't land on anything that
looks like the real culprit. :(
> Thanks,
>
> Bart.
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015
2021-02-28 9:49 ` [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015 Yi Zhang
2021-02-28 19:07 ` Chaitanya Kulkarni
2021-02-28 22:04 ` Bart Van Assche
@ 2021-03-03 15:16 ` Ming Lei
2 siblings, 0 replies; 9+ messages in thread
From: Ming Lei @ 2021-03-03 15:16 UTC (permalink / raw)
To: Yi Zhang; +Cc: linux-scsi, linux-rdma, linux-block
On Sun, Feb 28, 2021 at 04:49:32AM -0500, Yi Zhang wrote:
> Hello
>
> I found this issue with blktests srp/015, could anyone help check it?
>
> [ 250.415156] run blktests srp/015 at 2021-02-28 09:23:02
> [ 250.525538] eno2 speed is unknown, defaulting to 1000
> [ 250.530599] eno2 speed is unknown, defaulting to 1000
> [ 250.535665] eno2 speed is unknown, defaulting to 1000
> [ 250.541768] eno3 speed is unknown, defaulting to 1000
> [ 250.546821] eno3 speed is unknown, defaulting to 1000
> [ 250.551881] eno3 speed is unknown, defaulting to 1000
> [ 250.557844] eno4 speed is unknown, defaulting to 1000
> [ 250.562912] eno4 speed is unknown, defaulting to 1000
> [ 250.567978] eno4 speed is unknown, defaulting to 1000
> [ 250.573945] lo speed is unknown, defaulting to 1000
> [ 250.578827] lo speed is unknown, defaulting to 1000
> [ 250.583712] lo speed is unknown, defaulting to 1000
> [ 250.605512] sd 15:0:0:0: Power-on or device reset occurred
> [ 250.673453] device-mapper: table: 253:3: multipath: error getting device
> [ 250.680160] device-mapper: ioctl: error adding target to table
> [ 250.732648] eno2 speed is unknown, defaulting to 1000
> [ 250.737724] eno3 speed is unknown, defaulting to 1000
> [ 250.742784] eno4 speed is unknown, defaulting to 1000
> [ 250.747843] lo speed is unknown, defaulting to 1000
> [ 250.752725] ib_srpt MAD registration failed for lo_siw-1.
> [ 251.106511] scsi host16: REJ reason 0xffffff98
> [ 251.111150] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
> [ 251.246254] sd 16:0:0:0: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatical
> [ 251.260567] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 251.260765] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 251.280135] sd 16:0:0:1: Warning! Received an indication that the LUN assignments on this target have changed. The Linux SCSI layer does not automatical
> [ 251.298968] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 252.505389] ext4 filesystem being mounted at /root/blktests/results/tmpdir.srp.015.63f/mnt0 supports timestamps until 2038 (0x7fffffff)
> [ 253.522565] device-mapper: table: 253:6: multipath: error getting device
> [ 253.529277] device-mapper: ioctl: error adding target to table
> [ 257.526935] device-mapper: multipath: 253:4: Failing path 8:32.
> [ 262.542076] scsi host16: REJ reason 0xffffff98
> [ 262.546708] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
> [ 262.671752] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 262.672079] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 262.692394] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 267.804429] device-mapper: multipath: 253:4: Failing path 8:32.
> [ 267.882487] srpt_recv_done: 502 callbacks suppressed
> [ 269.819904] scsi host16: REJ reason 0xffffff98
> [ 269.824533] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
> [ 269.951714] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 269.952049] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 269.972080] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 275.085527] device-mapper: multipath: 253:4: Failing path 8:32.
> [ 275.152684] srpt_recv_done: 502 callbacks suppressed
> [ 277.100827] scsi host16: REJ reason 0xffffff98
> [ 277.105452] scsi host16: ib_srp: Connection 0/4 to 10.16.221.116 failed
> [ 277.231922] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 277.242758] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 277.252505] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 282.353149] device-mapper: multipath: 253:4: Failing path 8:32.
> [ 282.419429] srpt_recv_done: 502 callbacks suppressed
> [ 282.842310] scsi host17: REJ reason 0xffffff98
> [ 282.846939] scsi host17: ib_srp: Connection 0/4 to 10.16.221.116 failed
> [ 282.981432] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 282.991501] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 283.001358] srpt/0xf0d4e2e6e1e000000000000000000000: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [ 284.690990] device-mapper: multipath: 253:4: Failing path 8:96.
> [ 285.493848] BUG: unable to handle page fault for address: ffffffffc0a89150
> [ 285.500724] #PF: supervisor read access in kernel mode
> [ 285.505863] #PF: error_code(0x0000) - not-present page
> [ 285.511001] PGD 37bc13067 P4D 37bc13067 PUD 37bc15067 PMD 13839d067 PTE 0
> [ 285.517789] Oops: 0000 [#1] SMP NOPTI
> [ 285.521454] CPU: 31 PID: 17943 Comm: multipathd Tainted: G S I 5.11.0 #4
> [ 285.529279] Hardware name: Dell Inc. PowerEdge R640/06NR82, BIOS 2.10.0 11/12/2020
> [ 285.536837] RIP: 0010:scsi_mq_exit_request+0x14/0x50
> [ 285.541804] Code: 00 00 e9 bf eb 58 00 e9 ea bc e0 ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 8b 7f 60 48 89 f3 48 8b 87 98 00 00 00 <48> 8b 40 30 48 85 c0 74 0c 48 8d b6 10 01 00 00 e8 87 eb 58 00 f6
> [ 285.560548] RSP: 0018:ffffa66e82e37c98 EFLAGS: 00010286
> [ 285.565774] RAX: ffffffffc0a89120 RBX: ffff8a0e8a3e0000 RCX: 0000000000000000
> [ 285.572906] RDX: 0000000000000000 RSI: ffff8a0e8a3e0000 RDI: ffff8a0e692a1000
> [ 285.580038] RBP: ffff8a0ea8469800 R08: 000000000000020d R09: 000000000002a780
> [ 285.587170] R10: 00000376b4157308 R11: 0000000000000000 R12: 0000000000000000
> [ 285.594304] R13: ffff8a0e692a10a8 R14: 0000000000000000 R15: ffff8a0e9244b700
> [ 285.601435] FS: 00007f616c291700(0000) GS:ffff8a1d801c0000(0000) knlGS:0000000000000000
> [ 285.609521] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 285.615269] CR2: ffffffffc0a89150 CR3: 000000011e43e003 CR4: 00000000007706e0
> [ 285.622401] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 285.629532] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 285.636664] PKRU: 55555554
> [ 285.639369] Call Trace:
> [ 285.641824] blk_mq_free_rqs+0x59/0xd0
> [ 285.645573] blk_mq_free_map_and_requests+0x31/0x60
> [ 285.650454] blk_mq_free_tag_set+0x22/0x80
> [ 285.654553] scsi_host_dev_release+0x86/0xe0
> [ 285.658826] device_release+0x33/0x90
> [ 285.662490] kobject_release+0x46/0x150
> [ 285.666332] device_release+0x33/0x90
> [ 285.669995] kobject_release+0x46/0x150
> [ 285.673836] execute_in_process_context+0x21/0x60
> [ 285.678542] device_release+0x33/0x90
> [ 285.682208] kobject_release+0x46/0x150
> [ 285.686046] scsi_disk_put+0x2b/0x40 [sd_mod]
> [ 285.690406] __blkdev_put+0x186/0x1b0
> [ 285.694072] blkdev_put+0x4c/0x110
> [ 285.697476] blkdev_close+0x21/0x30
> [ 285.700970] __fput+0x92/0x230
> [ 285.704029] task_work_run+0x70/0xb0
> [ 285.707608] exit_to_user_mode_prepare+0x150/0x160
> [ 285.712402] syscall_exit_to_user_mode+0x12/0x40
> [ 285.717021] entry_SYSCALL_64_after_hwframe+0x44/0xa9
> [ 285.722071] RIP: 0033:0x7f616b1ce977
> [ 285.725652] Code: 12 b8 03 00 00 00 0f 05 48 3d 00 f0 ff ff 77 3b c3 66 90 53 89 fb 48 83 ec 10 e8 e4 fb ff ff 89 df 89 c2 b8 03 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 2b 89 d7 89 44 24 0c e8 26 fc ff ff 8b 44 24
> [ 285.744397] RSP: 002b:00007f616c28f7b0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
> [ 285.751962] RAX: 0000000000000000 RBX: 000000000000000b RCX: 00007f616b1ce977
> [ 285.759094] RDX: 0000000000000000 RSI: 0000000000000001 RDI: 000000000000000b
> [ 285.766227] RBP: 00005588a853efb0 R08: 0000000000000001 R09: 0000000000000007
> [ 285.773359] R10: 0000000000000000 R11: 0000000000000293 R12: 00005588a8215b19
> [ 285.780492] R13: 00007f615401d630 R14: 0000000000000001 R15: 00007f615401d630
> [ 285.787624] Modules linked in: target_core_user uio target_core_pscsi target_core_file ib_srpt target_core_iblock target_core_mod scsi_debug siw null_blk ext4 mbcache jbd2 rpcrdma rdma_ucm ib_umad ib_iser libiscsi scsi_transport_iscsi rdma_cm iw_cm ib_cm ib_uverbs ib_core rfkill sunrpc vfat fat dm_service_time dm_multipath intel_rapl_msr intel_rapl_common isst_if_common skx_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel ipmi_ssif kvm mgag200 i2c_algo_bit drm_kms_helper iTCO_wdt iTCO_vendor_support irqbypass dcdbas syscopyarea crct10dif_pclmul sysfillrect crc32_pclmul sysimgblt fb_sys_fops ghash_clmulni_intel drm acpi_ipmi rapl ipmi_si intel_cstate dell_smbios dax_pmem_compat mei_me i2c_i801 ipmi_devintf device_dax intel_uncore mei wmi_bmof dell_wmi_descriptor pcspkr intel_pch_thermal lpc_ich i2c_smbus ipmi_msghandler dax_pmem_core acpi_power_meter ip_tables xfs libcrc32c nd_pmem nd_btt sd_mod t10_pi sg ahci libahci nfit megaraid_sas libata tg3 crc32c_intel libnvdimm wmi
> [ 285.787672] dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_transport_srp]
> [ 285.882703] CR2: ffffffffc0a89150
> [ 285.886032] ---[ end trace 718c96f79b0576a6 ]---
> [ 285.896061] RIP: 0010:scsi_mq_exit_request+0x14/0x50
> [ 285.901024] Code: 00 00 e9 bf eb 58 00 e9 ea bc e0 ff 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 53 48 8b 7f 60 48 89 f3 48 8b 87 98 00 00 00 <48> 8b 40 30 48 85 c0 74 0c 48 8d b6 10 01 00 00 e8 87 eb 58 00 f6
> [ 285.919768] RSP: 0018:ffffa66e82e37c98 EFLAGS: 00010286
> [ 285.924995] RAX: ffffffffc0a89120 RBX: ffff8a0e8a3e0000 RCX: 0000000000000000
> [ 285.932128] RDX: 0000000000000000 RSI: ffff8a0e8a3e0000 RDI: ffff8a0e692a1000
> [ 285.939260] RBP: ffff8a0ea8469800 R08: 000000000000020d R09: 000000000002a780
> [ 285.946394] R10: 00000376b4157308 R11: 0000000000000000 R12: 0000000000000000
> [ 285.953524] R13: ffff8a0e692a10a8 R14: 0000000000000000 R15: ffff8a0e9244b700
> [ 285.960658] FS: 00007f616c291700(0000) GS:ffff8a1d801c0000(0000) knlGS:0000000000000000
> [ 285.968742] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 285.974489] CR2: ffffffffc0a89150 CR3: 000000011e43e003 CR4: 00000000007706e0
> [ 285.981623] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 285.988753] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 285.995885] PKRU: 55555554
> [ 285.998590] Kernel panic - not syncing: Fatal exception
> [ 286.504449] Kernel Offset: 0x2c200000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
> [ 286.520574] ---[ end Kernel panic - not syncing: Fatal exception ]---
>
> (gdb) l *(scsi_mq_exit_request+0x14)
> 0xffffffff81673784 is in scsi_mq_exit_request (drivers/scsi/scsi_lib.c:1785).
> 1780 unsigned int hctx_idx)
> 1781 {
> 1782 struct Scsi_Host *shost = set->driver_data;
> 1783 struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(rq);
> 1784
> 1785 if (shost->hostt->exit_cmd_priv)
> 1786 shost->hostt->exit_cmd_priv(shost, cmd);
Looks it is weird since not see any drivers implement .exit_cmd_priv in
v5.11.
--
Ming
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2021-03-03 16:44 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <347186099.14151179.1614497065881.JavaMail.zimbra@redhat.com>
2021-02-28 9:49 ` [bug report]null pointer at scsi_mq_exit_request+0x14 with blktests srp/015 Yi Zhang
2021-02-28 19:07 ` Chaitanya Kulkarni
2021-03-01 2:54 ` Yi Zhang
2021-03-01 15:36 ` Yi Zhang
2021-03-02 3:59 ` Bart Van Assche
2021-03-03 11:40 ` Yi Zhang
2021-02-28 22:04 ` Bart Van Assche
2021-03-01 2:56 ` Yi Zhang
2021-03-03 15:16 ` Ming Lei
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).