Greetings, linux-next kernel 5.3.0-rc1 failed to boot with kernel Oops on Power 9 box I see a recent changes to lpfc code was from commit 10541f03 scsi: lpfc: Update lpfc version to 12.4.0.0 Recent boot logs: [..snip..] Emulex LightPulse Fibre Channel SCSI driver 12.4.0.0 Copyright (C) 2017-2019 Broadcom. All Rights Reserved. The term "Broadcom" refers to Broadcom Inc. and/or its subsidiaries. bnx2x 0021:01:00.0: part number 0-0-0-0 bnx2x 0021:01:00.0: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link) bnx2x 0021:01:00.1: msix capability found bnx2x 0021:01:00.1: part number 0-0-0-0 lpfc 0022:01:00.0: 0:2574 IO channels: hdwQ 16 IRQ 16 MRQ: 0 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 0 (phys 0 core 0): hdwq 0 eq 0 irq 48 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 1 (phys 0 core 1): hdwq 1 eq 1 irq 49 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 2 (phys 0 core 2): hdwq 2 eq 2 irq 50 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 3 (phys 0 core 3): hdwq 3 eq 3 irq 51 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 4 (phys 0 core 4): hdwq 4 eq 4 irq 52 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 5 (phys 0 core 5): hdwq 5 eq 5 irq 53 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 6 (phys 0 core 6): hdwq 6 eq 6 irq 54 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 7 (phys 0 core 7): hdwq 7 eq 7 irq 55 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 8 (phys 0 core 8): hdwq 8 eq 8 irq 56 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 9 (phys 0 core 9): hdwq 9 eq 9 irq 57 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 10 (phys 0 core 10): hdwq 10 eq 10 irq 58 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 11 (phys 0 core 11): hdwq 11 eq 11 irq 59 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 12 (phys 0 core 12): hdwq 12 eq 12 irq 60 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 13 (phys 0 core 13): hdwq 13 eq 13 irq 61 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 14 (phys 0 core 14): hdwq 14 eq 14 irq 62 flg x4 lpfc 0022:01:00.0: 0:3333 Set Affinity: CPU 15 (phys 0 core 15): hdwq 15 eq 15 irq 63 flg x4 scsi host1: Emulex LPe32000 32Gb PCIe Fibre Channel Adapter on PCI bus 01 device 00 irq 20 PCI resettable bnx2x 0021:01:00.1: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link) bnx2x 0021:01:00.2: msix capability found bnx2x 0021:01:00.2: part number 0-0-0-0 bnx2x 0021:01:00.2: 32.000 Gb/s available PCIe bandwidth (5 GT/s x8 link) bnx2x 0021:01:00.3: msix capability found bnx2x 0021:01:00.3: part number 0-0-0-0 BUG: Kernel NULL pointer dereference at 0x00000148 Faulting instruction address: 0xc0080000006f97dc Oops: Kernel access of bad area, sig: 11 [#1] LE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries Modules linked in: lpfc(E+) bnx2x(E+) ibmvscsi(E+) ibmveth(E) scsi_transport_srp(E) nvmet_fc(E) nvmet(E) nvme_fc(E) nvme_fabrics(E) scsi_transport_fc(E) mdio(E) libcrc32c(E) ptp(E) pps_core(E) nvme(E) nvme_core(E) dm_mirror(E) dm_region_hash(E) dm_log(E) dm_mod(E) CPU: 0 PID: 16 Comm: kworker/0:0 Tainted: G E 5.3.0-rc6-next-20190826-autotest-autotest #1 Workqueue: events work_for_cpu_fn NIP: c0080000006f97dc LR: c008000000712de4 CTR: c0080000006f9760 REGS: c00000027b57b660 TRAP: 0380 Tainted: G E (5.3.0-rc6-next-20190826-autotest-autotest) MSR: 800000010280b033 CR: 84002044 XER: 20000008 CFAR: c0080000006f976c IRQMASK: 0 GPR00: c008000000712c90 c00000027b57b8f0 c0080000007c8900 c000000003a44000 GPR04: c00000026bc95c00 0000000000000000 0000000000000001 0000000000000000 GPR08: 0000000000000000 0000000000000000 0000000020000000 c008000000799d18 GPR12: c0080000006f9760 c00000001ecaee00 c00000027d096980 c00000027d0966f0 GPR16: c00000027d0966a0 c00000027d096680 c000000001323a00 0000000000000000 GPR20: c00000027b4ad000 fffffffffffffef7 0000000000000000 0000000000000000 GPR24: 0000000000000000 c000000001323a00 c00000026e3c3848 c000000003a45148 GPR28: 0000000000000000 000000000000000a c00000026e394c00 c000000003a44000 NIP [c0080000006f97dc] lpfc_sli4_write_cq_db+0x7c/0xa0 [lpfc] LR [c008000000712de4] lpfc_sli4_hba_setup+0x1024/0x1dc0 [lpfc] Call Trace: [c00000027b57b8f0] [c008000000712c90] lpfc_sli4_hba_setup+0xed0/0x1dc0 [lpfc] (unreliable) [c00000027b57ba90] [c008000000752c28] lpfc_pci_probe_one_s4.isra.25+0x358/0x13b0 [lpfc] [c00000027b57bb20] [c008000000753270] lpfc_pci_probe_one_s4.isra.25+0x9a0/0x13b0 [lpfc] [c00000027b57bbb0] [c0000000005d7af4] local_pci_probe+0x64/0x100 [c00000027b57bc30] [c0000000001363c0] work_for_cpu_fn+0x30/0x50 [c00000027b57bc60] [c00000000013c2a0] process_one_work+0x1c0/0x480 [c00000027b57bd00] [c00000000013c7e8] worker_thread+0x288/0x590 [c00000027b57bdb0] [c00000000014401c] kthread+0x14c/0x190 [c00000027b57be20] [c00000000000b760] ret_from_kernel_thread+0x5c/0x7c Instruction dump: a14d1180 b14d1182 4e800020 2fa60000 4dde0020 81040098 54a580de e9240038 3d402000 7caa5378 5507b2be 550805be 54e75c28 7d4a3b78 7d484378 ---[ end trace 067b4fb92c298ba5 ]--- Kernel panic - not syncing: Fatal exception ------------[ cut here ]------------ WARNING: CPU: 0 PID: 16 at drivers/tty/vt/vt.c:4256 do_unblank_screen+0x1dc/0x250 Modules linked in: lpfc(E+) bnx2x(E+) ibmvscsi(E+) ibmveth(E) scsi_transport_srp(E) nvmet_fc(E) nvmet(E) nvme_fc(E) nvme_fabrics(E) scsi_transport_fc(E) mdio(E) libcrc32c(E) ptp(E) pps_core(E) nvme(E) nvme_core(E) dm_mirror(E) dm_region_hash(E) dm_log(E) dm_mod(E) CPU: 0 PID: 16 Comm: kworker/0:0 Tainted: G D E 5.3.0-rc6-next-20190826-autotest-autotest #1 Workqueue: events work_for_cpu_fn NIP: c000000000659adc LR: c000000000659ac4 CTR: c000000000499980 REGS: c00000027b57b140 TRAP: 0700 Tainted: G D E (5.3.0-rc6-next-20190826-autotest-autotest) MSR: 8000000000021033 CR: 28002042 XER: 2000000c CFAR: c000000000194da0 IRQMASK: 3 GPR00: c000000000659af4 c00000027b57b3d0 c000000001303000 0000000000000000 GPR04: 0000000000000003 0000000000000800 0000000000000000 0000000000000000 GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000001001 GPR12: c000000000499980 c00000001ecaee00 c00000027d096980 c00000027d0966f0 GPR16: c00000027d0966a0 c00000027d096680 c000000001323a00 0000000000000000 GPR20: c00000027b4ad000 fffffffffffffef7 0000000000000000 0000000000000000 GPR24: 0000000000000000 c000000001323a00 c0000000011d2728 c0000000014d6758 GPR28: c0000000014d6730 0000000000000000 c000000000cada08 c0000000015e5270 NIP [c000000000659adc] do_unblank_screen+0x1dc/0x250 LR [c000000000659ac4] do_unblank_screen+0x1c4/0x250 Call Trace: [c00000027b57b3d0] [c000000000659af4] do_unblank_screen+0x1f4/0x250 (unreliable) [c00000027b57b450] [c000000000112c08] panic+0x1e0/0x3f8 [c00000027b57b4e0] [c00000000002a628] oops_end+0x1a8/0x1b0 [c00000027b57b560] [c00000000006f718] bad_page_fault+0xe8/0x194 [c00000027b57b5d0] [c000000000078318] do_bad_slb_fault+0x88/0xa0 [c00000027b57b5f0] [c000000000008840] data_access_slb_common+0x130/0x140 --- interrupt: 380 at lpfc_sli4_write_cq_db+0x7c/0xa0 [lpfc] LR = lpfc_sli4_hba_setup+0x1024/0x1dc0 [lpfc] [c00000027b57b8f0] [c008000000712c90] lpfc_sli4_hba_setup+0xed0/0x1dc0 [lpfc] (unreliable) [c00000027b57ba90] [c008000000752c28] lpfc_pci_probe_one_s4.isra.25+0x358/0x13b0 [lpfc] [c00000027b57bb20] [c008000000753270] lpfc_pci_probe_one_s4.isra.25+0x9a0/0x13b0 [lpfc] [c00000027b57bbb0] [c0000000005d7af4] local_pci_probe+0x64/0x100 [c00000027b57bc30] [c0000000001363c0] work_for_cpu_fn+0x30/0x50 [c00000027b57bc60] [c00000000013c2a0] process_one_work+0x1c0/0x480 [c00000027b57bd00] [c00000000013c7e8] worker_thread+0x288/0x590 [c00000027b57bdb0] [c00000000014401c] kthread+0x14c/0x190 [c00000027b57be20] [c00000000000b760] ret_from_kernel_thread+0x5c/0x7c Instruction dump: 7c0803a6 4e800020 60000000 60000000 60420000 4bb3b2d9 60000000 2fa30000 409efe80 813f0000 2f890000 409efe74 <0fe00000> 4bfffe6c 60000000 60000000 ---[ end trace 067b4fb92c298ba6 ]--- Rebooting in 10 seconds.. Detailed logs attached -- Regard's Abdul Haleem IBM Linux Technology Centre