* AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
@ 2022-02-05 10:06 Thomas Kupper
2022-02-05 15:51 ` Tom Lendacky
0 siblings, 1 reply; 15+ messages in thread
From: Thomas Kupper @ 2022-02-05 10:06 UTC (permalink / raw)
To: thomas.lendacky; +Cc: netdev
Hi,
I got an OPNsense DEC740 firewall which is based on the AMD V1500B CPU.
OPNsense runs fine on it but on Linux I'm not able to get the 10GbE
interfaces to work.
My test setup is based on Ubuntu 21.10 Impish Indri with a v5.17-rc2
kernel compiled from Mr Torvalds sources, tag v5.17-rc2. The second
10GbE interface (enp6s0f2) is set to receive the IP by DHCPv4.
The relevant dmesg entries after boot are:
[ 4.763712] libphy: amd-xgbe-mii: probed
[ 4.782850] amd-xgbe 0000:06:00.1 eth0: net device enabled
[ 4.800625] libphy: amd-xgbe-mii: probed
[ 4.803192] amd-xgbe 0000:06:00.2 eth1: net device enabled
[ 4.841151] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
[ 5.116617] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
After that I see a link up on the switch for enp6s0f2 and the switch
reports 10G link speed.
ethtool reports:
$ sudo ethtool enp6s0f2
Settings for enp6s0f2:
Supported ports: [ FIBRE ]
Supported link modes: Not reported
Supported pause frame use: No
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes: Not reported
Advertised pause frame use: No
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Auto-negotiation: off
Port: None
PHYAD: 0
Transceiver: internal
Current message level: 0x00000034 (52)
link ifdown ifup
Link detected: no
Manually assigning an IP and pull the interface up and I end up with:
$ sudo ifconfig enp6s0f2 up
SIOCSIFFLAGS: Device or resource busy
... and dmesg reports:
[ 648.038655] genirq: Flags mismatch irq 59. 00000000 (enp6s0f2-pcs)
vs. 00000000 (enp6s0f2-pcs)
[ 648.048303] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
After that the lights are out on the switch for that port and it reports
'no link'
Would that be an known issue or is that configuration simply not yet
supported?
Kind Regards
Thomas Kupper
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-05 10:06 AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board Thomas Kupper
@ 2022-02-05 15:51 ` Tom Lendacky
2022-02-05 18:14 ` Thomas Kupper
0 siblings, 1 reply; 15+ messages in thread
From: Tom Lendacky @ 2022-02-05 15:51 UTC (permalink / raw)
To: Thomas Kupper, Shyam Sundar S K; +Cc: netdev
On 2/5/22 04:06, Thomas Kupper wrote:
> Hi,
>
> I got an OPNsense DEC740 firewall which is based on the AMD V1500B CPU.
>
> OPNsense runs fine on it but on Linux I'm not able to get the 10GbE
> interfaces to work.
>
> My test setup is based on Ubuntu 21.10 Impish Indri with a v5.17-rc2
> kernel compiled from Mr Torvalds sources, tag v5.17-rc2. The second 10GbE
> interface (enp6s0f2) is set to receive the IP by DHCPv4.
>
> The relevant dmesg entries after boot are:
>
> [ 4.763712] libphy: amd-xgbe-mii: probed
> [ 4.782850] amd-xgbe 0000:06:00.1 eth0: net device enabled
> [ 4.800625] libphy: amd-xgbe-mii: probed
> [ 4.803192] amd-xgbe 0000:06:00.2 eth1: net device enabled
> [ 4.841151] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
> [ 5.116617] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>
> After that I see a link up on the switch for enp6s0f2 and the switch
> reports 10G link speed.
>
> ethtool reports:
>
> $ sudo ethtool enp6s0f2
> Settings for enp6s0f2:
> Supported ports: [ FIBRE ]
> Supported link modes: Not reported
> Supported pause frame use: No
> Supports auto-negotiation: No
> Supported FEC modes: Not reported
> Advertised link modes: Not reported
> Advertised pause frame use: No
> Advertised auto-negotiation: No
> Advertised FEC modes: Not reported
> Speed: Unknown!
> Duplex: Unknown! (255)
> Auto-negotiation: off
> Port: None
> PHYAD: 0
> Transceiver: internal
> Current message level: 0x00000034 (52)
> link ifdown ifup
> Link detected: no
>
>
> Manually assigning an IP and pull the interface up and I end up with:
>
> $ sudo ifconfig enp6s0f2 up
>
> SIOCSIFFLAGS: Device or resource busy
>
> ... and dmesg reports:
>
> [ 648.038655] genirq: Flags mismatch irq 59. 00000000 (enp6s0f2-pcs) vs.
> 00000000 (enp6s0f2-pcs)
> [ 648.048303] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>
> After that the lights are out on the switch for that port and it reports
> 'no link'
>
> Would that be an known issue or is that configuration simply not yet
> supported?
>
Reloading the module and specify the dyndbg option to get some additional
debug output.
I'm adding Shyam to the thread, too, as I'm not familiar with the
configuration for this chip.
Thanks,
Tom
>
> Kind Regards
>
> Thomas Kupper
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-05 15:51 ` Tom Lendacky
@ 2022-02-05 18:14 ` Thomas Kupper
2022-02-07 14:32 ` Tom Lendacky
0 siblings, 1 reply; 15+ messages in thread
From: Thomas Kupper @ 2022-02-05 18:14 UTC (permalink / raw)
To: Tom Lendacky, Shyam Sundar S K; +Cc: netdev
Am 05.02.22 um 16:51 schrieb Tom Lendacky:
> On 2/5/22 04:06, Thomas Kupper wrote:
>> Hi,
>>
>> I got an OPNsense DEC740 firewall which is based on the AMD V1500B CPU.
>>
>> OPNsense runs fine on it but on Linux I'm not able to get the 10GbE
>> interfaces to work.
>>
>> My test setup is based on Ubuntu 21.10 Impish Indri with a v5.17-rc2
>> kernel compiled from Mr Torvalds sources, tag v5.17-rc2. The second
>> 10GbE interface (enp6s0f2) is set to receive the IP by DHCPv4.
>>
>> The relevant dmesg entries after boot are:
>>
>> [ 4.763712] libphy: amd-xgbe-mii: probed
>> [ 4.782850] amd-xgbe 0000:06:00.1 eth0: net device enabled
>> [ 4.800625] libphy: amd-xgbe-mii: probed
>> [ 4.803192] amd-xgbe 0000:06:00.2 eth1: net device enabled
>> [ 4.841151] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>> [ 5.116617] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>
>> After that I see a link up on the switch for enp6s0f2 and the switch
>> reports 10G link speed.
>>
>> ethtool reports:
>>
>> $ sudo ethtool enp6s0f2
>> Settings for enp6s0f2:
>> Supported ports: [ FIBRE ]
>> Supported link modes: Not reported
>> Supported pause frame use: No
>> Supports auto-negotiation: No
>> Supported FEC modes: Not reported
>> Advertised link modes: Not reported
>> Advertised pause frame use: No
>> Advertised auto-negotiation: No
>> Advertised FEC modes: Not reported
>> Speed: Unknown!
>> Duplex: Unknown! (255)
>> Auto-negotiation: off
>> Port: None
>> PHYAD: 0
>> Transceiver: internal
>> Current message level: 0x00000034 (52)
>> link ifdown ifup
>> Link detected: no
>>
>>
>> Manually assigning an IP and pull the interface up and I end up with:
>>
>> $ sudo ifconfig enp6s0f2 up
>>
>> SIOCSIFFLAGS: Device or resource busy
>>
>> ... and dmesg reports:
>>
>> [ 648.038655] genirq: Flags mismatch irq 59. 00000000 (enp6s0f2-pcs)
>> vs. 00000000 (enp6s0f2-pcs)
>> [ 648.048303] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>>
>> After that the lights are out on the switch for that port and it
>> reports 'no link'
>>
>> Would that be an known issue or is that configuration simply not yet
>> supported?
>>
>
> Reloading the module and specify the dyndbg option to get some
> additional debug output.
>
> I'm adding Shyam to the thread, too, as I'm not familiar with the
> configuration for this chip.
>
> Thanks,
> Tom
>
>>
>> Kind Regards
>>
>> Thomas Kupper
>>
Thanks Tom for getting back to me so quick. After adding
'amd_xgbe.dyndbg=+p' to the kernel command line here the output of
dmesg. Probably the most interesting is the output after running 'rmmod'.
Right after boot:
[ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
[ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
...
[ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
[ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
...
[ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
[ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
[ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
[ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
[ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
...
[ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox command
did not complete
[ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
performed
[ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
[ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
[ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY configuration
Then after 'ifconfig enp6s0f2 up':
[ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
[ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
[ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
[ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
[ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
[ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
[ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
vs. 00000000 (enp6s0f2-pcs)
[ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
[ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
[ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
And after 'rmmod amd_xgbe':
[ 278.324933] ------------[ cut here ]------------
[ 278.324939] remove_proc_entry: removing non-empty directory 'irq/69',
leaking at least 'enp6s0f2-pcs'
[ 278.324952] WARNING: CPU: 0 PID: 796 at fs/proc/generic.c:715
remove_proc_entry+0x196/0x1b0
[ 278.324964] Modules linked in: nls_iso8859_1 intel_rapl_msr
intel_rapl_common snd_hda_intel snd_intel_dspcfg edac_mce_amd
snd_intel_sdw_acpi snd_hda_codec snd_hda_core snd_hwdep kvm snd_pcm rapl
snd_
timer efi_pstore k10temp snd_rn_pci_acp3x snd soundcore snd_pci_acp3x
ccp mac_hid sch_fq_codel msr drm ip_tables x_tables autofs4 btrfs
blake2b_generic zstd_compress raid10 raid456 async_raid6_recov asyn
c_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0
multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
aesni_intel crypto_simd igb nvme cryptd dca amd_xgbe(-) xhci_pci
i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas i2c_algo_bit nvme_core
video spi_amd
[ 278.325038] CPU: 0 PID: 796 Comm: rmmod Not tainted 5.17.0-rc2-tk #8
[ 278.325043] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 278.325046] RIP: 0010:remove_proc_entry+0x196/0x1b0
[ 278.325052] Code: a8 1d 9e 84 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 28 53 81
00 <0f> 0b e9 44 ff ff ff e8 6e bd 87 00 66 66 2e 0f 1f 84
00 00 00 00
[ 278.325055] RSP: 0018:ffff954d81027b00 EFLAGS: 00010286
[ 278.325059] RAX: 0000000000000000 RBX: ffff89350022dc00 RCX:
0000000000000000
[ 278.325062] RDX: 0000000000000001 RSI: ffffffff849bc031 RDI:
00000000ffffffff
[ 278.325064] RBP: ffff954d81027b30 R08: 0000000000000000 R09:
ffff954d810278f0
[ 278.325066] R10: ffff954d810278e8 R11: ffffffff84d55f48 R12:
ffff89351996a780
[ 278.325068] R13: ffff89351996a800 R14: 0000000000000046 R15:
0000000000000046
[ 278.325070] FS: 00007f8a17115400(0000) GS:ffff89352ae00000(0000)
knlGS:0000000000000000
[ 278.325073] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 278.325075] CR2: 00007f1fac1391a0 CR3: 00000001104b6000 CR4:
00000000003506f0
[ 278.325078] Call Trace:
[ 278.325080] <TASK>
[ 278.325085] unregister_irq_proc+0xe4/0x110
[ 278.325093] free_desc+0x2e/0x70
[ 278.325098] irq_free_descs+0x50/0x80
[ 278.325102] irq_domain_free_irqs+0x16b/0x1c0
[ 278.325107] __msi_domain_free_irqs+0xf1/0x160
[ 278.325114] msi_domain_free_irqs_descs_locked+0x20/0x50
[ 278.325118] pci_msi_teardown_msi_irqs+0x49/0x50
[ 278.325124] pci_disable_msix.part.0+0xff/0x160
[ 278.325128] pci_free_irq_vectors+0x45/0x60
[ 278.325132] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
[ 278.325151] pci_device_remove+0x39/0xa0
[ 278.325157] __device_release_driver+0x181/0x250
[ 278.325163] driver_detach+0xd3/0x120
[ 278.325166] bus_remove_driver+0x59/0xd0
[ 278.325169] driver_unregister+0x31/0x50
[ 278.325172] pci_unregister_driver+0x40/0x90
[ 278.325177] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
[ 278.325192] xgbe_mod_exit+0x9/0x8b0 [amd_xgbe]
[ 278.325207] __do_sys_delete_module.constprop.0+0x183/0x290
[ 278.325214] ? __fput+0x123/0x260
[ 278.325219] __x64_sys_delete_module+0x12/0x20
[ 278.325223] do_syscall_64+0x5c/0xc0
[ 278.325228] ? fpregs_assert_state_consistent+0x26/0x50
[ 278.325234] ? exit_to_user_mode_prepare+0x49/0x1e0
[ 278.325239] ? syscall_exit_to_user_mode+0x27/0x50
[ 278.325244] ? __x64_sys_close+0x11/0x40
[ 278.325248] ? do_syscall_64+0x69/0xc0
[ 278.325251] ? __x64_sys_close+0x11/0x40
[ 278.325254] ? do_syscall_64+0x69/0xc0
[ 278.325257] ? irqentry_exit+0x33/0x40
[ 278.325261] ? exc_page_fault+0x89/0x180
[ 278.325265] ? asm_exc_page_fault+0x8/0x30
[ 278.325269] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 278.325274] RIP: 0033:0x7f8a172448eb
[ 278.325278] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8
64 89 01 48
[ 278.325280] RSP: 002b:00007ffe3a968e98 EFLAGS: 00000206 ORIG_RAX:
00000000000000b0
[ 278.325284] RAX: ffffffffffffffda RBX: 00007f8a190dc760 RCX:
00007f8a172448eb
[ 278.325286] RDX: 000000000000000a RSI: 0000000000000800 RDI:
00007f8a190dc7c8
[ 278.325288] RBP: 0000000000000000 R08: 0000000000000000 R09:
0000000000000000
[ 278.325289] R10: 00007f8a172dcac0 R11: 0000000000000206 R12:
00007ffe3a9690f8
[ 278.325291] R13: 00007ffe3a969847 R14: 00007f8a190dc2a0 R15:
00007f8a190dc760
[ 278.325296] </TASK>
[ 278.325298] ---[ end trace 0000000000000000 ]---
[ 278.922700] irq 31: nobody cared (try booting with the "irqpoll" option)
[ 278.930195] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G W
5.17.0-rc2-tk #8
[ 278.930201] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 278.930204] Call Trace:
[ 278.930206] <IRQ>
[ 278.930210] dump_stack_lvl+0x4c/0x63
[ 278.930219] dump_stack+0x10/0x12
[ 278.930223] __report_bad_irq+0x3a/0xaf
[ 278.930228] note_interrupt.cold+0xb/0x60
[ 278.930232] ? __this_cpu_preempt_check+0x13/0x20
[ 278.930238] handle_irq_event+0x71/0x80
[ 278.930244] handle_fasteoi_irq+0x95/0x1e0
[ 278.930249] __common_interrupt+0x6e/0x110
[ 278.930254] common_interrupt+0xbd/0xe0
[ 278.930258] </IRQ>
[ 278.930259] <TASK>
[ 278.930261] asm_common_interrupt+0x1e/0x40
[ 278.930265] RIP: 0010:cpuidle_enter_state+0xdf/0x380
[ 278.930273] Code: ff e8 e5 88 73 ff 80 7d d7 00 74 17 9c 58 0f 1f 44
00 00 f6 c4 02 0f 85 82 02 00 00 31 ff e8 d8 9e 7a ff fb 66 0f 1f 44 00
00 <45> 85 ff 0f 88 1a 01 00 00 49 63 d7 4c 89 f1 48 2b 4d
c8 48 8d 04
[ 278.930277] RSP: 0018:ffff954d800e3e68 EFLAGS: 00000246
[ 278.930281] RAX: ffff89352af00000 RBX: 0000000000000002 RCX:
000000000000001f
[ 278.930284] RDX: 0000000000000000 RSI: ffffffff849bc031 RDI:
ffffffff849cab7f
[ 278.930287] RBP: ffff954d800e3ea0 R08: 00000040f1169c00 R09:
00000040d2207b5c
[ 278.930289] R10: 0000000000000001 R11: ffff89352af2fd84 R12:
ffff893501907000
[ 278.930291] R13: ffffffff84e6e3c0 R14: 00000040f1169c00 R15:
0000000000000002
[ 278.930297] ? cpuidle_enter_state+0xbb/0x380
[ 278.930302] cpuidle_enter+0x2e/0x40
[ 278.930307] do_idle+0x203/0x290
[ 278.930313] cpu_startup_entry+0x20/0x30
[ 278.930316] start_secondary+0x118/0x150
[ 278.930322] secondary_startup_64_no_verify+0xd5/0xdb
[ 278.930330] </TASK>
[ 278.930331] handlers:
[ 278.932870] [<000000000a369c68>] amd_mp2_irq_isr [i2c_amd_mp2_pci]
[ 278.939782] Disabling IRQ #31
Cheers
/Thomas
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-05 18:14 ` Thomas Kupper
@ 2022-02-07 14:32 ` Tom Lendacky
2022-02-07 15:19 ` Shyam Sundar S K
0 siblings, 1 reply; 15+ messages in thread
From: Tom Lendacky @ 2022-02-07 14:32 UTC (permalink / raw)
To: Thomas Kupper, Shyam Sundar S K; +Cc: netdev
On 2/5/22 12:14, Thomas Kupper wrote:
> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>> On 2/5/22 04:06, Thomas Kupper wrote:
>>
>> Reloading the module and specify the dyndbg option to get some
>> additional debug output.
>>
>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>> configuration for this chip.
>>
>
> Right after boot:
>
> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
> ...
> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
> ...
> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
> ...
> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox command
> did not complete
> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
> performed
> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY configuration
>
> Then after 'ifconfig enp6s0f2 up':
>
> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs) vs.
> 00000000 (enp6s0f2-pcs)
> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>
Please ensure that the ethtool msglvl is on for drv and probe. I was
expecting to see some additional debug messages that I don't see here.
Also, if you can provide the lspci output for the device (using -nn and
-vv) that might be helpful as well.
Shyam will be the best one to understand what is going on here.
Thanks,
Tom
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-07 14:32 ` Tom Lendacky
@ 2022-02-07 15:19 ` Shyam Sundar S K
2022-02-07 17:59 ` Thomas Kupper
2022-02-07 18:04 ` Thomas Kupper
0 siblings, 2 replies; 15+ messages in thread
From: Shyam Sundar S K @ 2022-02-07 15:19 UTC (permalink / raw)
To: Tom Lendacky, Thomas Kupper; +Cc: netdev
On 2/7/2022 8:02 PM, Tom Lendacky wrote:
> On 2/5/22 12:14, Thomas Kupper wrote:
>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>> On 2/5/22 04:06, Thomas Kupper wrote:
>
>>>
>>> Reloading the module and specify the dyndbg option to get some
>>> additional debug output.
>>>
>>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>>> configuration for this chip.
>>>
>>
>> Right after boot:
>>
>> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
>> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
>> ...
>> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>> ...
>> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
>> ...
>> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox
>> command did not complete
>> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
>> performed
>> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY configuration
>>
>> Then after 'ifconfig enp6s0f2 up':
>>
>> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
>> vs. 00000000 (enp6s0f2-pcs)
>> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>>
>
> Please ensure that the ethtool msglvl is on for drv and probe. I was
> expecting to see some additional debug messages that I don't see here.
>
> Also, if you can provide the lspci output for the device (using -nn and
> -vv) that might be helpful as well.
>
> Shyam will be the best one to understand what is going on here.
On some other platforms, we have seen similar kind of problems getting
reported. There is a fix sent for validation.
The root cause is that removal of xgbe driver is causing interrupt storm
on the MP2 device (Sensor Fusion Hub).
Shall submit a fix soon to upstream once the validation is done, you may
give it a try with that and see if that helps.
Thanks,
Shyam
>
> Thanks,
> Tom
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-07 15:19 ` Shyam Sundar S K
@ 2022-02-07 17:59 ` Thomas Kupper
2022-02-08 16:24 ` Tom Lendacky
2022-02-07 18:04 ` Thomas Kupper
1 sibling, 1 reply; 15+ messages in thread
From: Thomas Kupper @ 2022-02-07 17:59 UTC (permalink / raw)
To: Shyam Sundar S K, Tom Lendacky; +Cc: netdev
Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>
> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>> On 2/5/22 12:14, Thomas Kupper wrote:
>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>>> Reloading the module and specify the dyndbg option to get some
>>>> additional debug output.
>>>>
>>>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>>>> configuration for this chip.
>>>>
>>> Right after boot:
>>>
>>> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>> ...
>>> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>>> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>> ...
>>> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
>>> ...
>>> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox
>>> command did not complete
>>> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
>>> performed
>>> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY configuration
>>>
>>> Then after 'ifconfig enp6s0f2 up':
>>>
>>> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
>>> vs. 00000000 (enp6s0f2-pcs)
>>> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>>> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>>>
>> Please ensure that the ethtool msglvl is on for drv and probe. I was
>> expecting to see some additional debug messages that I don't see here.
>>
>> Also, if you can provide the lspci output for the device (using -nn and
>> -vv) that might be helpful as well.
>>
>> Shyam will be the best one to understand what is going on here.
> On some other platforms, we have seen similar kind of problems getting
> reported. There is a fix sent for validation.
>
> The root cause is that removal of xgbe driver is causing interrupt storm
> on the MP2 device (Sensor Fusion Hub).
>
> Shall submit a fix soon to upstream once the validation is done, you may
> give it a try with that and see if that helps.
>
> Thanks,
> Shyam
>
>> Thanks,
>> Tom
Shyam, I will check the git logs for the relevant commit then from time
to time.
Looking at the code diff from OPNsense and the latest Linux kernel I
assumed that there would much more to do then fix a irq strom (but I
have no idea about the inner working of the kernel).
Nevermind: Setting the 'msglvl 0x3' with ethtool the following info can
be found in dmesg:
Running : $ ifconfig enp6s0f2 up
SIOCSIFFLAGS: Invalid argument
... and 'dmesg':
[ 55.177447] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0: cpu=0, node=0
[ 55.177456] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0:
dma_regs=00000000d11bf3f1, dma_irq=74, tx=00000000dd57b5c4,
rx=00000000d73e70f8
[ 55.177464] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1: cpu=1, node=0
[ 55.177467] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1:
dma_regs=000000000d972dd7, dma_irq=75, tx=00000000573bcff8,
rx=000000003d9a6f65
[ 55.177473] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2: cpu=2, node=0
[ 55.177476] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2:
dma_regs=0000000046f71179, dma_irq=76, tx=00000000897116c9,
rx=0000000004ba17e7
[ 55.177480] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Tx ring:
[ 55.177502] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=00000000794657ba,
rdesc_dma=0x000000010fad8000, rdata=0000000008ace7d8, node=0
[ 55.177507] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Rx ring:
[ 55.177523] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=000000009313d9b3,
rdesc_dma=0x0000000114538000, rdata=00000000510e3b77, node=0
[ 55.177527] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Tx ring:
[ 55.177543] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=00000000d26d9194,
rdesc_dma=0x000000010a774000, rdata=00000000b9419829, node=0
[ 55.177547] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Rx ring:
[ 55.177564] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=0000000007bf60dd,
rdesc_dma=0x000000010fb84000, rdata=00000000aa48e8c0, node=0
[ 55.177568] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Tx ring:
[ 55.177584] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=00000000e7e6c52e,
rdesc_dma=0x000000010fa2a000, rdata=0000000017b5d85c, node=0
[ 55.177587] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Rx ring:
[ 55.177603] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=000000000898fbf4,
rdesc_dma=0x0000000101f08000, rdata=00000000aded7d4c, node=0
[ 55.182366] amd-xgbe 0000:06:00.2 enp6s0f2: TXq0 mapped to TC0
[ 55.182381] amd-xgbe 0000:06:00.2 enp6s0f2: TXq1 mapped to TC1
[ 55.182388] amd-xgbe 0000:06:00.2 enp6s0f2: TXq2 mapped to TC2
[ 55.182395] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO0 mapped to RXq0
[ 55.182400] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO1 mapped to RXq0
[ 55.182405] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO2 mapped to RXq0
[ 55.182410] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO3 mapped to RXq1
[ 55.182414] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO4 mapped to RXq1
[ 55.182418] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO5 mapped to RXq1
[ 55.182423] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO6 mapped to RXq2
[ 55.182427] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO7 mapped to RXq2
[ 55.182473] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues,
21760 byte fifo per queue
[ 55.182501] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues,
21760 byte fifo per queue
[ 55.182544] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled for RXq0
[ 55.182550] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled for RXq1
[ 55.182556] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled for RXq2
[ 56.178946] amd-xgbe 0000:06:00.2 enp6s0f2: SFP detected:
[ 56.178954] amd-xgbe 0000:06:00.2 enp6s0f2: vendor: MikroTik
[ 56.178958] amd-xgbe 0000:06:00.2 enp6s0f2: part number: S+AO0005
[ 56.178961] amd-xgbe 0000:06:00.2 enp6s0f2: revision level: 1.0
[ 56.178963] amd-xgbe 0000:06:00.2 enp6s0f2: serial number:
STST050B1900001
Then running '$ rmmod amd_xgbe' produced the following dmesg output:
[ 504.272482] ------------[ cut here ]------------
[ 504.272489] remove_proc_entry: removing non-empty directory 'irq/72',
leaking at least 'enp6s0f2-i2c'
[ 504.272500] WARNING: CPU: 0 PID: 803 at fs/proc/generic.c:715
remove_proc_entry+0x196/0x1b0
[ 504.272525] Modules linked in: nls_iso8859_1 intel_rapl_msr
intel_rapl_common snd_hda_intel edac_mce_amd snd_intel_dspcfg
snd_intel_sdw_acpi snd_hda_codec snd_hda_core snd_hwdep kvm snd_pcm rapl
snd_timer snd_rn_pci_acp3x snd k10temp efi_pstore soundcore
snd_pci_acp3x ccp mac_hid sch_fq_codel msr drm ip_tables x_tables
autofs4 btrfs blake2b_generic zstd_compress raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq
libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul
ghash_clmulni_intel aesni_intel crypto_simd cryptd amd_xgbe(-) xhci_pci
igb i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme dca i2c_algo_bit
nvme_core video spi_amd
[ 504.272603] CPU: 0 PID: 803 Comm: rmmod Not tainted 5.17.0-rc2-tk #8
[ 504.272608] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 504.272612] RIP: 0010:remove_proc_entry+0x196/0x1b0
[ 504.272619] Code: a8 1d de 92 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 28 53 81
00 <0f> 0b e9 44 ff ff ff e8 6e bd 87 00 66 66 2e 0f 1f 84 00 00 00 00
[ 504.272623] RSP: 0018:ffffa22a810b7b88 EFLAGS: 00010282
[ 504.272627] RAX: 0000000000000000 RBX: ffff8d8c8022ccc0 RCX:
0000000000000000
[ 504.272630] RDX: 0000000000000001 RSI: ffffffff92dbc031 RDI:
00000000ffffffff
[ 504.272632] RBP: ffffa22a810b7bb8 R08: 0000000000000000 R09:
ffffa22a810b7978
[ 504.272635] R10: ffffa22a810b7970 R11: ffffffff93155f48 R12:
ffff8d8c90dc0540
[ 504.272637] R13: ffff8d8c90dc05c0 R14: 0000000000000049 R15:
0000000000000049
[ 504.272639] FS: 00007fb60c99b400(0000) GS:ffff8d8caae00000(0000)
knlGS:0000000000000000
[ 504.272643] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 504.272645] CR2: 00007f86e9192f68 CR3: 0000000111b46000 CR4:
00000000003506f0
[ 504.272648] Call Trace:
[ 504.272650] <TASK>
[ 504.272656] unregister_irq_proc+0xe4/0x110
[ 504.272664] free_desc+0x2e/0x70
[ 504.272669] irq_free_descs+0x50/0x80
[ 504.272674] irq_domain_free_irqs+0x16b/0x1c0
[ 504.272678] __msi_domain_free_irqs+0xf1/0x160
[ 504.272683] msi_domain_free_irqs_descs_locked+0x20/0x50
[ 504.272687] pci_msi_teardown_msi_irqs+0x49/0x50
[ 504.272692] pci_disable_msix.part.0+0xff/0x160
[ 504.272695] pci_free_irq_vectors+0x45/0x60
[ 504.272699] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
[ 504.272717] pci_device_remove+0x39/0xa0
[ 504.272724] __device_release_driver+0x181/0x250
[ 504.272731] driver_detach+0xd3/0x120
[ 504.272736] bus_remove_driver+0x59/0xd0
[ 504.272739] driver_unregister+0x31/0x50
[ 504.272743] pci_unregister_driver+0x40/0x90
[ 504.272748] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
[ 504.272766] xgbe_mod_exit+0x9/0x8b0 [amd_xgbe]
[ 504.272784] __do_sys_delete_module.constprop.0+0x183/0x290
[ 504.272791] ? syscall_exit_to_user_mode+0x27/0x50
[ 504.272799] __x64_sys_delete_module+0x12/0x20
[ 504.272804] do_syscall_64+0x5c/0xc0
[ 504.272809] ? irqentry_exit+0x33/0x40
[ 504.272813] ? exc_page_fault+0x89/0x180
[ 504.272818] ? asm_exc_page_fault+0x8/0x30
[ 504.272822] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 504.272828] RIP: 0033:0x7fb60caca8eb
[ 504.272833] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
[ 504.272836] RSP: 002b:00007ffd82036228 EFLAGS: 00000206 ORIG_RAX:
00000000000000b0
[ 504.272840] RAX: ffffffffffffffda RBX: 00007fb60e79b760 RCX:
00007fb60caca8eb
[ 504.272843] RDX: 000000000000000a RSI: 0000000000000800 RDI:
00007fb60e79b7c8
[ 504.272845] RBP: 0000000000000000 R08: 0000000000000000 R09:
0000000000000000
[ 504.272847] R10: 00007fb60cb62ac0 R11: 0000000000000206 R12:
00007ffd82036480
[ 504.272850] R13: 00007ffd820368b6 R14: 00007fb60e79b2a0 R15:
00007fb60e79b760
[ 504.272855] </TASK>
[ 504.272857] ---[ end trace 0000000000000000 ]---
[ 504.272917] ------------[ cut here ]------------
[ 504.272919] remove_proc_entry: removing non-empty directory 'irq/73',
leaking at least 'enp6s0f2-pcs'
[ 504.272930] WARNING: CPU: 0 PID: 803 at fs/proc/generic.c:715
remove_proc_entry+0x196/0x1b0
[ 504.272938] Modules linked in: nls_iso8859_1 intel_rapl_msr
intel_rapl_common snd_hda_intel edac_mce_amd snd_intel_dspcfg
snd_intel_sdw_acpi snd_hda_codec snd_hda_core snd_hwdep kvm snd_pcm rapl
snd_timer snd_rn_pci_acp3x snd k10temp efi_pstore soundcore
snd_pci_acp3x ccp mac_hid sch_fq_codel msr drm ip_tables x_tables
autofs4 btrfs blake2b_generic zstd_compress raid10 raid456
async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq
libcrc32c raid1 raid0 multipath linear crct10dif_pclmul crc32_pclmul
ghash_clmulni_intel aesni_intel crypto_simd cryptd amd_xgbe(-) xhci_pci
igb i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme dca i2c_algo_bit
nvme_core video spi_amd
[ 504.272998] CPU: 0 PID: 803 Comm: rmmod Tainted: G W
5.17.0-rc2-tk #8
[ 504.273002] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 504.273004] RIP: 0010:remove_proc_entry+0x196/0x1b0
[ 504.273009] Code: a8 1d de 92 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 28 53 81
00 <0f> 0b e9 44 ff ff ff e8 6e bd 87 00 66 66 2e 0f 1f 84 00 00 00 00
[ 504.273012] RSP: 0018:ffffa22a810b7b88 EFLAGS: 00010282
[ 504.273015] RAX: 0000000000000000 RBX: ffff8d8c8022ccc0 RCX:
0000000000000000
[ 504.273018] RDX: 0000000000000001 RSI: ffffffff92dbc031 RDI:
00000000ffffffff
[ 504.273020] RBP: ffffa22a810b7bb8 R08: 0000000000000000 R09:
ffffa22a810b7978
[ 504.273022] R10: ffffa22a810b7970 R11: ffffffff93155f48 R12:
ffff8d8c8b8676c0
[ 504.273024] R13: ffff8d8c8b867740 R14: 000000000000004a R15:
000000000000004a
[ 504.273027] FS: 00007fb60c99b400(0000) GS:ffff8d8caae00000(0000)
knlGS:0000000000000000
[ 504.273030] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 504.273032] CR2: 00007f86e9192f68 CR3: 0000000111b46000 CR4:
00000000003506f0
[ 504.273035] Call Trace:
[ 504.273036] <TASK>
[ 504.273039] unregister_irq_proc+0xe4/0x110
[ 504.273044] free_desc+0x2e/0x70
[ 504.273049] irq_free_descs+0x50/0x80
[ 504.273053] irq_domain_free_irqs+0x16b/0x1c0
[ 504.273058] __msi_domain_free_irqs+0xf1/0x160
[ 504.273064] msi_domain_free_irqs_descs_locked+0x20/0x50
[ 504.273070] pci_msi_teardown_msi_irqs+0x49/0x50
[ 504.273074] pci_disable_msix.part.0+0xff/0x160
[ 504.273079] pci_free_irq_vectors+0x45/0x60
[ 504.273082] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
[ 504.273098] pci_device_remove+0x39/0xa0
[ 504.273103] __device_release_driver+0x181/0x250
[ 504.273107] driver_detach+0xd3/0x120
[ 504.273110] bus_remove_driver+0x59/0xd0
[ 504.273113] driver_unregister+0x31/0x50
[ 504.273116] pci_unregister_driver+0x40/0x90
[ 504.273121] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
[ 504.273136] xgbe_mod_exit+0x9/0x8b0 [amd_xgbe]
[ 504.273151] __do_sys_delete_module.constprop.0+0x183/0x290
[ 504.273156] ? syscall_exit_to_user_mode+0x27/0x50
[ 504.273161] __x64_sys_delete_module+0x12/0x20
[ 504.273165] do_syscall_64+0x5c/0xc0
[ 504.273168] ? irqentry_exit+0x33/0x40
[ 504.273172] ? exc_page_fault+0x89/0x180
[ 504.273176] ? asm_exc_page_fault+0x8/0x30
[ 504.273179] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 504.273184] RIP: 0033:0x7fb60caca8eb
[ 504.273187] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
[ 504.273189] RSP: 002b:00007ffd82036228 EFLAGS: 00000206 ORIG_RAX:
00000000000000b0
[ 504.273192] RAX: ffffffffffffffda RBX: 00007fb60e79b760 RCX:
00007fb60caca8eb
[ 504.273194] RDX: 000000000000000a RSI: 0000000000000800 RDI:
00007fb60e79b7c8
[ 504.273196] RBP: 0000000000000000 R08: 0000000000000000 R09:
0000000000000000
[ 504.273198] R10: 00007fb60cb62ac0 R11: 0000000000000206 R12:
00007ffd82036480
[ 504.273200] R13: 00007ffd820368b6 R14: 00007fb60e79b2a0 R15:
00007fb60e79b760
[ 504.273204] </TASK>
[ 504.273205] ---[ end trace 0000000000000000 ]---
[ 504.925023] irq 31: nobody cared (try booting with the "irqpoll" option)
[ 504.932518] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G W
5.17.0-rc2-tk #8
[ 504.932524] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 504.932526] Call Trace:
[ 504.932529] <IRQ>
[ 504.932533] dump_stack_lvl+0x4c/0x63
[ 504.932542] dump_stack+0x10/0x12
[ 504.932545] __report_bad_irq+0x3a/0xaf
[ 504.932550] note_interrupt.cold+0xb/0x60
[ 504.932554] ? __this_cpu_preempt_check+0x13/0x20
[ 504.932560] handle_irq_event+0x71/0x80
[ 504.932567] handle_fasteoi_irq+0x95/0x1e0
[ 504.932572] __common_interrupt+0x6e/0x110
[ 504.932577] common_interrupt+0xbd/0xe0
[ 504.932581] </IRQ>
[ 504.932582] <TASK>
[ 504.932584] asm_common_interrupt+0x1e/0x40
[ 504.932588] RIP: 0010:cpuidle_enter_state+0xdf/0x380
[ 504.932595] Code: ff e8 e5 88 73 ff 80 7d d7 00 74 17 9c 58 0f 1f 44
00 00 f6 c4 02 0f 85 82 02 00 00 31 ff e8 d8 9e 7a ff fb 66 0f 1f 44 00
00 <45> 85 ff 0f 88 1a 01 00 00 49 63 d7 4c 89 f1 48 2b 4d c8 48 8d 04
[ 504.932599] RSP: 0018:ffffa22a800e3e68 EFLAGS: 00000246
[ 504.932604] RAX: ffff8d8caaf00000 RBX: 0000000000000002 RCX:
000000000000001f
[ 504.932607] RDX: 0000000000000000 RSI: ffffffff92dbc031 RDI:
ffffffff92dcab7f
[ 504.932609] RBP: ffffa22a800e3ea0 R08: 000000758fe062ac R09:
000000754b94ae72
[ 504.932611] R10: 0000000000000001 R11: ffff8d8caaf2fd84 R12:
ffff8d8c933f7000
[ 504.932613] R13: ffffffff9326e3c0 R14: 000000758fe062ac R15:
0000000000000002
[ 504.932618] ? cpuidle_enter_state+0xbb/0x380
[ 504.932624] cpuidle_enter+0x2e/0x40
[ 504.932628] do_idle+0x203/0x290
[ 504.932633] cpu_startup_entry+0x20/0x30
[ 504.932637] start_secondary+0x118/0x150
[ 504.932642] secondary_startup_64_no_verify+0xd5/0xdb
[ 504.932650] </TASK>
[ 504.932651] handlers:
[ 504.935191] [<00000000dbc7353a>] amd_mp2_irq_isr [i2c_amd_mp2_pci]
[ 504.942102] Disabling IRQ #31
Cheers
Thomas
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-07 15:19 ` Shyam Sundar S K
2022-02-07 17:59 ` Thomas Kupper
@ 2022-02-07 18:04 ` Thomas Kupper
1 sibling, 0 replies; 15+ messages in thread
From: Thomas Kupper @ 2022-02-07 18:04 UTC (permalink / raw)
To: Shyam Sundar S K, Tom Lendacky; +Cc: netdev
Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>
> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>> On 2/5/22 12:14, Thomas Kupper wrote:
>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>>> Reloading the module and specify the dyndbg option to get some
>>>> additional debug output.
>>>>
>>>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>>>> configuration for this chip.
>>>>
>>> Right after boot:
>>>
>>> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>> ...
>>> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>>> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>> ...
>>> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
>>> ...
>>> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox
>>> command did not complete
>>> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
>>> performed
>>> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY configuration
>>>
>>> Then after 'ifconfig enp6s0f2 up':
>>>
>>> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
>>> vs. 00000000 (enp6s0f2-pcs)
>>> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>>> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>>>
>> Please ensure that the ethtool msglvl is on for drv and probe. I was
>> expecting to see some additional debug messages that I don't see here.
>>
>> Also, if you can provide the lspci output for the device (using -nn and
>> -vv) that might be helpful as well.
>>
>> Shyam will be the best one to understand what is going on here.
> On some other platforms, we have seen similar kind of problems getting
> reported. There is a fix sent for validation.
>
> The root cause is that removal of xgbe driver is causing interrupt storm
> on the MP2 device (Sensor Fusion Hub).
>
> Shall submit a fix soon to upstream once the validation is done, you may
> give it a try with that and see if that helps.
>
> Thanks,
> Shyam
>
>> Thanks,
>> Tom
Sorry, forgot the 'lspci -nn -vv' output. Here it goes:
$ ethtool -i enp6s0f2
driver: amd-xgbe
version: 5.17.0-rc2-tk
firmware-version: 17.118.33
expansion-rom-version:
bus-info: 0000:06:00.2
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
$ lspci -nn -vv -s 0:6:0.2
06:00.2 Ethernet controller [0200]: Advanced Micro Devices, Inc. [AMD]
Device [1022:1458]
Subsystem: Advanced Micro Devices, Inc. [AMD] Device [1022:1458]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin C routed to IRQ 69
Region 0: Memory at d0020000 (32-bit, non-prefetchable) [size=128K]
Region 1: Memory at d0000000 (32-bit, non-prefetchable) [size=128K]
Region 2: Memory at d0080000 (64-bit, non-prefetchable) [size=8K]
Capabilities: [48] Vendor Specific Information: Len=08 <?>
Capabilities: [50] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [64] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s
<4us, L1 unlimited
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset-
SlotPowerLimit 0.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq+
AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L0s L1,
Exit Latency L0s <64ns, L1 <1us
ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 8GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Not Supported, TimeoutDis-
NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported,
ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported,
EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkSta2: Current De-emphasis Level: -3.5dB,
EqualizationComplete- EqualizationPhase1-
EqualizationPhase2- EqualizationPhase3-
LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [c0] MSI-X: Enable+ Count=7 Masked-
Vector table: BAR=2 offset=00000000
PBA: BAR=2 offset=00001000
Capabilities: [100 v1] Vendor Specific Information: ID=0001
Rev=1 Len=010 <?>
Capabilities: [150 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn-
ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [2a0 v1] Access Control Services
ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir-
UpstreamFwd- EgressCtrl- DirectTrans-
ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir-
UpstreamFwd- EgressCtrl- DirectTrans-
Kernel driver in use: amd-xgbe
Kernel modules: amd_xgbe
/Thomas
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-07 17:59 ` Thomas Kupper
@ 2022-02-08 16:24 ` Tom Lendacky
2022-02-11 9:33 ` Thomas Kupper
0 siblings, 1 reply; 15+ messages in thread
From: Tom Lendacky @ 2022-02-08 16:24 UTC (permalink / raw)
To: Thomas Kupper, Shyam Sundar S K; +Cc: netdev
On 2/7/22 11:59, Thomas Kupper wrote:
>
> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>
>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>>>> Reloading the module and specify the dyndbg option to get some
>>>>> additional debug output.
>>>>>
>>>>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>>>>> configuration for this chip.
>>>>>
>>>> Right after boot:
>>>>
>>>> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>>> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>>> ...
>>>> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>>>> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>>> ...
>>>> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
>>>> ...
>>>> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox
>>>> command did not complete
>>>> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
>>>> performed
>>>> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY configuration
>>>>
>>>> Then after 'ifconfig enp6s0f2 up':
>>>>
>>>> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
>>>> vs. 00000000 (enp6s0f2-pcs)
>>>> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>>>> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>>>>
>>> Please ensure that the ethtool msglvl is on for drv and probe. I was
>>> expecting to see some additional debug messages that I don't see here.
>>>
>>> Also, if you can provide the lspci output for the device (using -nn and
>>> -vv) that might be helpful as well.
>>>
>>> Shyam will be the best one to understand what is going on here.
>> On some other platforms, we have seen similar kind of problems getting
>> reported. There is a fix sent for validation.
>>
>> The root cause is that removal of xgbe driver is causing interrupt storm
>> on the MP2 device (Sensor Fusion Hub).
>>
>> Shall submit a fix soon to upstream once the validation is done, you may
>> give it a try with that and see if that helps.
>>
>> Thanks,
>> Shyam
>>
>>> Thanks,
>>> Tom
>
> Shyam, I will check the git logs for the relevant commit then from time to
> time.
> Looking at the code diff from OPNsense and the latest Linux kernel I
> assumed that there would much more to do then fix a irq strom (but I have
> no idea about the inner working of the kernel).
>
> Nevermind: Setting the 'msglvl 0x3' with ethtool the following info can be
> found in dmesg:
>
> Running : $ ifconfig enp6s0f2 up
> SIOCSIFFLAGS: Invalid argument
>
> ... and 'dmesg':
>
> [ 55.177447] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0: cpu=0, node=0
> [ 55.177456] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0:
> dma_regs=00000000d11bf3f1, dma_irq=74, tx=00000000dd57b5c4,
> rx=00000000d73e70f8
> [ 55.177464] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1: cpu=1, node=0
> [ 55.177467] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1:
> dma_regs=000000000d972dd7, dma_irq=75, tx=00000000573bcff8,
> rx=000000003d9a6f65
> [ 55.177473] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2: cpu=2, node=0
> [ 55.177476] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2:
> dma_regs=0000000046f71179, dma_irq=76, tx=00000000897116c9,
> rx=0000000004ba17e7
> [ 55.177480] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Tx ring:
> [ 55.177502] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=00000000794657ba,
> rdesc_dma=0x000000010fad8000, rdata=0000000008ace7d8, node=0
> [ 55.177507] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Rx ring:
> [ 55.177523] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=000000009313d9b3,
> rdesc_dma=0x0000000114538000, rdata=00000000510e3b77, node=0
> [ 55.177527] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Tx ring:
> [ 55.177543] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=00000000d26d9194,
> rdesc_dma=0x000000010a774000, rdata=00000000b9419829, node=0
> [ 55.177547] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Rx ring:
> [ 55.177564] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=0000000007bf60dd,
> rdesc_dma=0x000000010fb84000, rdata=00000000aa48e8c0, node=0
> [ 55.177568] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Tx ring:
> [ 55.177584] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=00000000e7e6c52e,
> rdesc_dma=0x000000010fa2a000, rdata=0000000017b5d85c, node=0
> [ 55.177587] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Rx ring:
> [ 55.177603] amd-xgbe 0000:06:00.2 enp6s0f2: rdesc=000000000898fbf4,
> rdesc_dma=0x0000000101f08000, rdata=00000000aded7d4c, node=0
> [ 55.182366] amd-xgbe 0000:06:00.2 enp6s0f2: TXq0 mapped to TC0
> [ 55.182381] amd-xgbe 0000:06:00.2 enp6s0f2: TXq1 mapped to TC1
> [ 55.182388] amd-xgbe 0000:06:00.2 enp6s0f2: TXq2 mapped to TC2
> [ 55.182395] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO0 mapped to RXq0
> [ 55.182400] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO1 mapped to RXq0
> [ 55.182405] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO2 mapped to RXq0
> [ 55.182410] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO3 mapped to RXq1
> [ 55.182414] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO4 mapped to RXq1
> [ 55.182418] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO5 mapped to RXq1
> [ 55.182423] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO6 mapped to RXq2
> [ 55.182427] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO7 mapped to RXq2
> [ 55.182473] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues, 21760
> byte fifo per queue
> [ 55.182501] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues, 21760
> byte fifo per queue
> [ 55.182544] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled for RXq0
> [ 55.182550] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled for RXq1
> [ 55.182556] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled for RXq2
> [ 56.178946] amd-xgbe 0000:06:00.2 enp6s0f2: SFP detected:
> [ 56.178954] amd-xgbe 0000:06:00.2 enp6s0f2: vendor: MikroTik
> [ 56.178958] amd-xgbe 0000:06:00.2 enp6s0f2: part number: S+AO0005
> [ 56.178961] amd-xgbe 0000:06:00.2 enp6s0f2: revision level: 1.0
> [ 56.178963] amd-xgbe 0000:06:00.2 enp6s0f2: serial number:
> STST050B1900001
>
Ah, it's been a while since I've had to use the debug support. Could you
also set the module debug parameter to 0x37 (debug=0x37) when loading the
module. That will capture some of the debug messages that are issued on
driver load. Sorry about that...
Thanks,
Tom
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-08 16:24 ` Tom Lendacky
@ 2022-02-11 9:33 ` Thomas Kupper
2022-02-11 9:49 ` Shyam Sundar S K
0 siblings, 1 reply; 15+ messages in thread
From: Thomas Kupper @ 2022-02-11 9:33 UTC (permalink / raw)
To: Tom Lendacky, Shyam Sundar S K; +Cc: netdev
Am 08.02.22 um 17:24 schrieb Tom Lendacky:
> On 2/7/22 11:59, Thomas Kupper wrote:
>>
>> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>>
>>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>>>>> Reloading the module and specify the dyndbg option to get some
>>>>>> additional debug output.
>>>>>>
>>>>>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>>>>>> configuration for this chip.
>>>>>>
>>>>> Right after boot:
>>>>>
>>>>> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>>>> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>>>> ...
>>>>> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>>>>> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>>>> ...
>>>>> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>>> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>>> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>>> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
>>>>> ...
>>>>> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox
>>>>> command did not complete
>>>>> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
>>>>> performed
>>>>> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY
>>>>> configuration
>>>>>
>>>>> Then after 'ifconfig enp6s0f2 up':
>>>>>
>>>>> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>>> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>>> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>>> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
>>>>> vs. 00000000 (enp6s0f2-pcs)
>>>>> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>>>>> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>>>>>
>>>> Please ensure that the ethtool msglvl is on for drv and probe. I was
>>>> expecting to see some additional debug messages that I don't see here.
>>>>
>>>> Also, if you can provide the lspci output for the device (using -nn
>>>> and
>>>> -vv) that might be helpful as well.
>>>>
>>>> Shyam will be the best one to understand what is going on here.
>>> On some other platforms, we have seen similar kind of problems getting
>>> reported. There is a fix sent for validation.
>>>
>>> The root cause is that removal of xgbe driver is causing interrupt
>>> storm
>>> on the MP2 device (Sensor Fusion Hub).
>>>
>>> Shall submit a fix soon to upstream once the validation is done, you
>>> may
>>> give it a try with that and see if that helps.
>>>
>>> Thanks,
>>> Shyam
>>>
>>>> Thanks,
>>>> Tom
>>
>> Shyam, I will check the git logs for the relevant commit then from
>> time to time.
>> Looking at the code diff from OPNsense and the latest Linux kernel I
>> assumed that there would much more to do then fix a irq strom (but I
>> have no idea about the inner working of the kernel).
>>
>> Nevermind: Setting the 'msglvl 0x3' with ethtool the following info
>> can be found in dmesg:
>>
>> Running : $ ifconfig enp6s0f2 up
>> SIOCSIFFLAGS: Invalid argument
>>
>> ... and 'dmesg':
>>
>> [ 55.177447] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0: cpu=0, node=0
>> [ 55.177456] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0:
>> dma_regs=00000000d11bf3f1, dma_irq=74, tx=00000000dd57b5c4,
>> rx=00000000d73e70f8
>> [ 55.177464] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1: cpu=1, node=0
>> [ 55.177467] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1:
>> dma_regs=000000000d972dd7, dma_irq=75, tx=00000000573bcff8,
>> rx=000000003d9a6f65
>> [ 55.177473] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2: cpu=2, node=0
>> [ 55.177476] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2:
>> dma_regs=0000000046f71179, dma_irq=76, tx=00000000897116c9,
>> rx=0000000004ba17e7
>> [ 55.177480] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Tx ring:
>> [ 55.177502] amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=00000000794657ba, rdesc_dma=0x000000010fad8000,
>> rdata=0000000008ace7d8, node=0
>> [ 55.177507] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Rx ring:
>> [ 55.177523] amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=000000009313d9b3, rdesc_dma=0x0000000114538000,
>> rdata=00000000510e3b77, node=0
>> [ 55.177527] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Tx ring:
>> [ 55.177543] amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=00000000d26d9194, rdesc_dma=0x000000010a774000,
>> rdata=00000000b9419829, node=0
>> [ 55.177547] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Rx ring:
>> [ 55.177564] amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=0000000007bf60dd, rdesc_dma=0x000000010fb84000,
>> rdata=00000000aa48e8c0, node=0
>> [ 55.177568] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Tx ring:
>> [ 55.177584] amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=00000000e7e6c52e, rdesc_dma=0x000000010fa2a000,
>> rdata=0000000017b5d85c, node=0
>> [ 55.177587] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Rx ring:
>> [ 55.177603] amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=000000000898fbf4, rdesc_dma=0x0000000101f08000,
>> rdata=00000000aded7d4c, node=0
>> [ 55.182366] amd-xgbe 0000:06:00.2 enp6s0f2: TXq0 mapped to TC0
>> [ 55.182381] amd-xgbe 0000:06:00.2 enp6s0f2: TXq1 mapped to TC1
>> [ 55.182388] amd-xgbe 0000:06:00.2 enp6s0f2: TXq2 mapped to TC2
>> [ 55.182395] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO0 mapped to RXq0
>> [ 55.182400] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO1 mapped to RXq0
>> [ 55.182405] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO2 mapped to RXq0
>> [ 55.182410] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO3 mapped to RXq1
>> [ 55.182414] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO4 mapped to RXq1
>> [ 55.182418] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO5 mapped to RXq1
>> [ 55.182423] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO6 mapped to RXq2
>> [ 55.182427] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO7 mapped to RXq2
>> [ 55.182473] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues,
>> 21760 byte fifo per queue
>> [ 55.182501] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues,
>> 21760 byte fifo per queue
>> [ 55.182544] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>> for RXq0
>> [ 55.182550] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>> for RXq1
>> [ 55.182556] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>> for RXq2
>> [ 56.178946] amd-xgbe 0000:06:00.2 enp6s0f2: SFP detected:
>> [ 56.178954] amd-xgbe 0000:06:00.2 enp6s0f2: vendor: MikroTik
>> [ 56.178958] amd-xgbe 0000:06:00.2 enp6s0f2: part number: S+AO0005
>> [ 56.178961] amd-xgbe 0000:06:00.2 enp6s0f2: revision level: 1.0
>> [ 56.178963] amd-xgbe 0000:06:00.2 enp6s0f2: serial number:
>> STST050B1900001
>>
>
> Ah, it's been a while since I've had to use the debug support. Could
> you also set the module debug parameter to 0x37 (debug=0x37) when
> loading the module. That will capture some of the debug messages that
> are issued on driver load. Sorry about that...
>
> Thanks,
> Tom
Thanks Tom, I now got time to update to 5.17-rc3 and add the 'debug'
module parameter. I assume that parameter works with the non-debug
kernel? I don't really see any new messages related to the amd-xgbe driver:
dmesg right after boot:
[ 0.000000] Linux version 5.17.0-rc3-tk (jane@m920q-ubu21) (gcc
(Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37)
#12 SMP PREEMPT Tue Feb 8 19:52:19 CET 2022
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
console=ttyS0,115200n8 amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
...
[ 5.275730] amd-xgbe 0000:06:00.1 eth0: net device enabled
[ 5.277766] amd-xgbe 0000:06:00.2 eth1: net device enabled
[ 5.665315] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
[ 5.696665] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
dmesg right after 'ifconfig enp6s0f2 up'
[ 88.843454] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
enp6s0f2: channel-0: cpu=0, node=0
[ 88.843464] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
enp6s0f2: channel-0: dma_regs=000000001078e433, dma_irq=55,
tx=00000000e8736669, rx=00000000fadd04ec
[ 88.843474] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
enp6s0f2: channel-1: cpu=1, node=0
[ 88.843478] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
enp6s0f2: channel-1: dma_regs=000000003c3cbea8, dma_irq=56,
tx=000000000836d88c, rx=00000000920d02c4
[ 88.843485] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
enp6s0f2: channel-2: cpu=2, node=0
[ 88.843488] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
enp6s0f2: channel-2: dma_regs=000000008d034191, dma_irq=57,
tx=00000000a0664378, rx=00000000d72ce726
[ 88.843493] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
enp6s0f2: channel-0 - Tx ring:
[ 88.843514] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
rdesc=00000000c6703013, rdesc_dma=0x0000000101c44000,
rdata=0000000029951e4c, node=0
[ 88.843519] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
enp6s0f2: channel-0 - Rx ring:
[ 88.843537] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
rdesc=000000003262c446, rdesc_dma=0x0000000103c74000,
rdata=000000001b7a4275, node=0
[ 88.843542] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
enp6s0f2: channel-1 - Tx ring:
[ 88.843560] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
rdesc=000000007ce3cc7e, rdesc_dma=0x00000001023c0000,
rdata=00000000c0fc51d9, node=0
[ 88.843565] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
enp6s0f2: channel-1 - Rx ring:
[ 88.843583] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
rdesc=00000000448612df, rdesc_dma=0x00000001185b6000,
rdata=00000000a23b7f86, node=0
[ 88.843587] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
enp6s0f2: channel-2 - Tx ring:
[ 88.843606] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
rdesc=00000000e509050e, rdesc_dma=0x0000000104db2000,
rdata=000000000d605e1a, node=0
[ 88.843610] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
enp6s0f2: channel-2 - Rx ring:
[ 88.843629] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
rdesc=00000000436c5cc6, rdesc_dma=0x0000000114aaa000,
rdata=00000000246ed062, node=0
[ 88.848416] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: TXq0 mapped to TC0
[ 88.848432] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: TXq1 mapped to TC1
[ 88.848440] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: TXq2 mapped to TC2
[ 88.848449] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO0 mapped to RXq0
[ 88.848455] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO1 mapped to RXq0
[ 88.848461] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO2 mapped to RXq0
[ 88.848467] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO3 mapped to RXq1
[ 88.848472] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO4 mapped to RXq1
[ 88.848478] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO5 mapped to RXq1
[ 88.848483] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO6 mapped to RXq2
[ 88.848489] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
enp6s0f2: PRIO7 mapped to RXq2
[ 88.848536] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues,
21760 byte fifo per queue
[ 88.848565] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues,
21760 byte fifo per queue
[ 88.848609] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
0000:06:00.2 enp6s0f2: flow control enabled for RXq0
[ 88.848619] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
0000:06:00.2 enp6s0f2: flow control enabled for RXq1
[ 88.848627] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
0000:06:00.2 enp6s0f2: flow control enabled for RXq2
[ 89.862558] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
enp6s0f2: SFP detected:
[ 89.862567] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
enp6s0f2: vendor: MikroTik
[ 89.862572] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
enp6s0f2: part number: S+AO0005
[ 89.862576] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
enp6s0f2: revision level: 1.0
[ 89.862580] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
enp6s0f2: serial number: STST050B1900001
again, dmesg diff after 'rmmod':
[ 127.068380] ------------[ cut here ]------------
[ 127.068386] remove_proc_entry: removing non-empty directory 'irq/53',
leaking at least 'enp6s0f2-i2c'
[ 127.068398] WARNING: CPU: 4 PID: 803 at fs/proc/generic.c:715
remove_proc_entry+0x196/0x1b0
[ 127.068411] Modules linked in: nls_iso8859_1 intel_rapl_msr
intel_rapl_common snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi
edac_mce_amd snd_hda_codec snd_hda_core snd_hwdep snd_pcm kvm snd_timer
snd_rn_pci_acp3x snd rapl efi_pstore k10temp soundcore snd_pci_acp3x ccp
mac_hid sch_fq_codel msr drm ip_tables x_tables autofs4 btrfs
blake2b_generic zstd_compress raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1
raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
aesni_intel crypto_simd igb nvme cryptd dca xhci_pci amd_xgbe(-)
i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme_core i2c_algo_bit video
spi_amd
[ 127.068485] CPU: 4 PID: 803 Comm: rmmod Not tainted 5.17.0-rc3-tk #12
[ 127.068490] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 127.068493] RIP: 0010:remove_proc_entry+0x196/0x1b0
[ 127.068499] Code: 60 50 5e 84 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 38 56 81
00 <0f> 0b e9 44 ff ff ff e8 9e c0 87 00 66 66 2e 0f 1f 84 00 00 00 00
[ 127.068502] RSP: 0018:ffffaf2940fffb60 EFLAGS: 00010286
[ 127.068506] RAX: 0000000000000000 RBX: ffff91fa4022ed80 RCX:
0000000000000000
[ 127.068509] RDX: 0000000000000001 RSI: ffffffff845bf281 RDI:
00000000ffffffff
[ 127.068511] RBP: ffffaf2940fffb90 R08: 0000000000000000 R09:
ffffaf2940fff950
[ 127.068513] R10: ffffaf2940fff948 R11: ffffffff84f55f48 R12:
ffff91fa44e8c540
[ 127.068515] R13: ffff91fa44e8c5c0 R14: 0000000000000036 R15:
0000000000000036
[ 127.068517] FS: 00007f3a68f9c400(0000) GS:ffff91fa6af00000(0000)
knlGS:0000000000000000
[ 127.068520] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 127.068522] CR2: 00007fd6c2e7fd04 CR3: 0000000104ee0000 CR4:
00000000003506e0
[ 127.068525] Call Trace:
[ 127.068528] <TASK>
[ 127.068534] unregister_irq_proc+0xe4/0x110
[ 127.068541] free_desc+0x2e/0x70
[ 127.068546] irq_free_descs+0x50/0x80
[ 127.068550] irq_domain_free_irqs+0x16b/0x1c0
[ 127.068554] __msi_domain_free_irqs+0xf1/0x160
[ 127.068560] msi_domain_free_irqs_descs_locked+0x20/0x50
[ 127.068565] pci_msi_teardown_msi_irqs+0x49/0x50
[ 127.068571] pci_disable_msix.part.0+0xff/0x160
[ 127.068575] pci_free_irq_vectors+0x45/0x60
[ 127.068578] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
[ 127.068596] pci_device_remove+0x39/0xa0
[ 127.068602] __device_release_driver+0x181/0x250
[ 127.068608] driver_detach+0xd3/0x120
[ 127.068612] bus_remove_driver+0x59/0xd0
[ 127.068615] driver_unregister+0x31/0x50
[ 127.068619] pci_unregister_driver+0x40/0x90
[ 127.068623] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
[ 127.068639] xgbe_mod_exit+0x9/0x880 [amd_xgbe]
[ 127.068654] __do_sys_delete_module.constprop.0+0x183/0x290
[ 127.068660] ? exit_to_user_mode_prepare+0x49/0x1e0
[ 127.068666] __x64_sys_delete_module+0x12/0x20
[ 127.068670] do_syscall_64+0x5c/0xc0
[ 127.068676] ? irqentry_exit_to_user_mode+0x9/0x20
[ 127.068681] ? irqentry_exit+0x33/0x40
[ 127.068685] ? exc_page_fault+0x89/0x180
[ 127.068689] ? asm_exc_page_fault+0x8/0x30
[ 127.068694] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 127.068697] RIP: 0033:0x7f3a690cb8eb
[ 127.068702] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
[ 127.068704] RSP: 002b:00007ffed553a818 EFLAGS: 00000206 ORIG_RAX:
00000000000000b0
[ 127.068708] RAX: ffffffffffffffda RBX: 00007f3a6a02e7b0 RCX:
00007f3a690cb8eb
[ 127.068710] RDX: 000000000000000a RSI: 0000000000000800 RDI:
00007f3a6a02e818
[ 127.068712] RBP: 0000000000000000 R08: 0000000000000000 R09:
0000000000000000
[ 127.068714] R10: 00007f3a69163ac0 R11: 0000000000000206 R12:
00007ffed553aa70
[ 127.068716] R13: 00007ffed553b84a R14: 00007f3a6a02e2a0 R15:
00007f3a6a02e7b0
[ 127.068722] </TASK>
[ 127.068723] ---[ end trace 0000000000000000 ]---
[ 127.068744] ------------[ cut here ]------------
[ 127.068746] remove_proc_entry: removing non-empty directory 'irq/54',
leaking at least 'enp6s0f2-pcs'
[ 127.068755] WARNING: CPU: 4 PID: 803 at fs/proc/generic.c:715
remove_proc_entry+0x196/0x1b0
[ 127.068761] Modules linked in: nls_iso8859_1 intel_rapl_msr
intel_rapl_common snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi
edac_mce_amd snd_hda_codec snd_hda_core snd_hwdep snd_pcm kvm snd_timer
snd_rn_pci_acp3x snd rapl efi_pstore k10temp soundcore snd_pci_acp3x ccp
mac_hid sch_fq_codel msr drm ip_tables x_tables autofs4 btrfs
blake2b_generic zstd_compress raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1
raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
aesni_intel crypto_simd igb nvme cryptd dca xhci_pci amd_xgbe(-)
i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme_core i2c_algo_bit video
spi_amd
[ 127.068810] CPU: 4 PID: 803 Comm: rmmod Tainted: G W
5.17.0-rc3-tk #12
[ 127.068814] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 127.068815] RIP: 0010:remove_proc_entry+0x196/0x1b0
[ 127.068820] Code: 60 50 5e 84 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 38 56 81
00 <0f> 0b e9 44 ff ff ff e8 9e c0 87 00 66 66 2e 0f 1f 84 00 00 00 00
[ 127.068822] RSP: 0018:ffffaf2940fffb60 EFLAGS: 00010286
[ 127.068825] RAX: 0000000000000000 RBX: ffff91fa4022ed80 RCX:
0000000000000000
[ 127.068827] RDX: 0000000000000001 RSI: ffffffff845bf281 RDI:
00000000ffffffff
[ 127.068829] RBP: ffffaf2940fffb90 R08: 0000000000000000 R09:
ffffaf2940fff950
[ 127.068830] R10: ffffaf2940fff948 R11: ffffffff84f55f48 R12:
ffff91fa4eca7000
[ 127.068832] R13: ffff91fa4eca7080 R14: 0000000000000037 R15:
0000000000000037
[ 127.068834] FS: 00007f3a68f9c400(0000) GS:ffff91fa6af00000(0000)
knlGS:0000000000000000
[ 127.068837] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 127.068839] CR2: 00007fd6c2e7fd04 CR3: 0000000104ee0000 CR4:
00000000003506e0
[ 127.068841] Call Trace:
[ 127.068842] <TASK>
[ 127.068844] unregister_irq_proc+0xe4/0x110
[ 127.068849] free_desc+0x2e/0x70
[ 127.068852] irq_free_descs+0x50/0x80
[ 127.068856] irq_domain_free_irqs+0x16b/0x1c0
[ 127.068860] __msi_domain_free_irqs+0xf1/0x160
[ 127.068865] msi_domain_free_irqs_descs_locked+0x20/0x50
[ 127.068870] pci_msi_teardown_msi_irqs+0x49/0x50
[ 127.068873] pci_disable_msix.part.0+0xff/0x160
[ 127.068877] pci_free_irq_vectors+0x45/0x60
[ 127.068881] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
[ 127.068896] pci_device_remove+0x39/0xa0
[ 127.068900] __device_release_driver+0x181/0x250
[ 127.068904] driver_detach+0xd3/0x120
[ 127.068908] bus_remove_driver+0x59/0xd0
[ 127.068911] driver_unregister+0x31/0x50
[ 127.068914] pci_unregister_driver+0x40/0x90
[ 127.068919] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
[ 127.068933] xgbe_mod_exit+0x9/0x880 [amd_xgbe]
[ 127.068948] __do_sys_delete_module.constprop.0+0x183/0x290
[ 127.068952] ? exit_to_user_mode_prepare+0x49/0x1e0
[ 127.068957] __x64_sys_delete_module+0x12/0x20
[ 127.068961] do_syscall_64+0x5c/0xc0
[ 127.068964] ? irqentry_exit_to_user_mode+0x9/0x20
[ 127.068969] ? irqentry_exit+0x33/0x40
[ 127.068973] ? exc_page_fault+0x89/0x180
[ 127.068977] ? asm_exc_page_fault+0x8/0x30
[ 127.068980] entry_SYSCALL_64_after_hwframe+0x44/0xae
[ 127.068983] RIP: 0033:0x7f3a690cb8eb
[ 127.068985] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
[ 127.068987] RSP: 002b:00007ffed553a818 EFLAGS: 00000206 ORIG_RAX:
00000000000000b0
[ 127.068990] RAX: ffffffffffffffda RBX: 00007f3a6a02e7b0 RCX:
00007f3a690cb8eb
[ 127.068991] RDX: 000000000000000a RSI: 0000000000000800 RDI:
00007f3a6a02e818
[ 127.068993] RBP: 0000000000000000 R08: 0000000000000000 R09:
0000000000000000
[ 127.068994] R10: 00007f3a69163ac0 R11: 0000000000000206 R12:
00007ffed553aa70
[ 127.068996] R13: 00007ffed553b84a R14: 00007f3a6a02e2a0 R15:
00007f3a6a02e7b0
[ 127.068999] </TASK>
[ 127.069000] ---[ end trace 0000000000000000 ]---
[ 127.667264] irq 31: nobody cared (try booting with the "irqpoll" option)
[ 127.674758] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G W
5.17.0-rc3-tk #12
[ 127.674764] Hardware name: Deciso B.V. DEC2700 - OPNsense
Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
[ 127.674766] Call Trace:
[ 127.674769] <IRQ>
[ 127.674773] dump_stack_lvl+0x4c/0x63
[ 127.674781] dump_stack+0x10/0x12
[ 127.674784] __report_bad_irq+0x3a/0xaf
[ 127.674789] note_interrupt.cold+0xb/0x60
[ 127.674793] ? __this_cpu_preempt_check+0x13/0x20
[ 127.674799] handle_irq_event+0x71/0x80
[ 127.674805] handle_fasteoi_irq+0x95/0x1e0
[ 127.674810] __common_interrupt+0x6e/0x110
[ 127.674815] common_interrupt+0xbd/0xe0
[ 127.674819] </IRQ>
[ 127.674820] <TASK>
[ 127.674822] asm_common_interrupt+0x1e/0x40
[ 127.674826] RIP: 0010:cpuidle_enter_state+0xdf/0x380
[ 127.674834] Code: ff e8 25 76 73 ff 80 7d d7 00 74 17 9c 58 0f 1f 44
00 00 f6 c4 02 0f 85 82 02 00 00 31 ff e8 18 8c 7a ff fb 66 0f 1f 44 00
00 <45> 85 ff 0f 88 1a 01 00 00 49 63 d7 4c 89 f1 48 2b 4d c8 48 8d 04
[ 127.674837] RSP: 0018:ffffaf29400e3e68 EFLAGS: 00000246
[ 127.674841] RAX: ffff91fa6af00000 RBX: 0000000000000002 RCX:
000000000000001f
[ 127.674843] RDX: 0000000000000000 RSI: ffffffff845bf281 RDI:
ffffffff845cddcf
[ 127.674845] RBP: ffffaf29400e3ea0 R08: 0000001db98fd21c R09:
0000001d7b8fd3fc
[ 127.674847] R10: 0000000000000001 R11: ffff91fa6af2fd84 R12:
ffff91fa41de6c00
[ 127.674849] R13: ffffffff8506e4c0 R14: 0000001db98fd21c R15:
0000000000000002
[ 127.674854] ? cpuidle_enter_state+0xbb/0x380
[ 127.674860] cpuidle_enter+0x2e/0x40
[ 127.674864] do_idle+0x203/0x290
[ 127.674869] cpu_startup_entry+0x20/0x30
[ 127.674872] start_secondary+0x118/0x150
[ 127.674877] secondary_startup_64_no_verify+0xd5/0xdb
[ 127.674885] </TASK>
[ 127.674886] handlers:
[ 127.677425] [<00000000b61e344c>] amd_mp2_irq_isr [i2c_amd_mp2_pci]
[ 127.684335] Disabling IRQ #31
and command line output after 'modprobe -vvv amd_xgbe':
jane@dec740-ubu21:~$ sudo modprobe -vvv amd_xgbe
modprobe: INFO: ../libkmod/libkmod.c:365 kmod_set_log_fn() custom
logging function 0x7f74d79de780 registered
modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
file=/lib/modules/5.17.0-rc3-tk/modules.dep.bin
modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
file=/lib/modules/5.17.0-rc3-tk/modules.alias.bin
modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
file=/lib/modules/5.17.0-rc3-tk/modules.symbols.bin
modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
file=/lib/modules/5.17.0-rc3-tk/modules.builtin.alias.bin
modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
file=/lib/modules/5.17.0-rc3-tk/modules.builtin.bin
modprobe: DEBUG: ../libkmod/libkmod-module.c:556
kmod_module_new_from_lookup() input alias=amd_xgbe, normalized=amd_xgbe
modprobe: DEBUG: ../libkmod/libkmod-module.c:562
kmod_module_new_from_lookup() lookup modules.dep amd_xgbe
modprobe: DEBUG: ../libkmod/libkmod.c:595 kmod_search_moddep() use
mmaped index 'modules.dep' modname=amd_xgbe
modprobe: DEBUG: ../libkmod/libkmod.c:403 kmod_pool_get_module() get
module name='amd_xgbe' found=(nil)
modprobe: DEBUG: ../libkmod/libkmod.c:411 kmod_pool_add_module() add
0x7f74d83862a0 key='amd_xgbe'
modprobe: DEBUG: ../libkmod/libkmod-module.c:202
kmod_module_parse_depline() 0 dependencies for amd_xgbe
modprobe: DEBUG: ../libkmod/libkmod-module.c:589
kmod_module_new_from_lookup() lookup amd_xgbe=0, list=0x7f74d8385c40
modprobe: DEBUG: ../libkmod/libkmod.c:500 lookup_builtin_file() use
mmaped index 'modules.builtin' modname=amd_xgbe
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760
kmod_module_get_initstate() could not open
'/sys/module/amd_xgbe/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1770
kmod_module_get_initstate() could not open '/sys/module/amd_xgbe': No
such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=snd_pcsp mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=snd_usb_audio mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=cx88_alsa mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=snd_atiixp_modem mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=snd_intel8x0m mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=snd_via82xx_modem mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=md_mod mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=bonding mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=dummy mod->name=amd_xgbe mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
mod->alias=(null)
modprobe: DEBUG: ../libkmod/libkmod-module.c:1760
kmod_module_get_initstate() could not open
'/sys/module/amd_xgbe/initstate': No such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:1770
kmod_module_get_initstate() could not open '/sys/module/amd_xgbe': No
such file or directory
modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
name='amd_xgbe'
path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
name='amd_xgbe'
path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
insmod
/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko
dyndbg="+pfm" debug=0x37 dyndbg=+p debug=0x37
modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
name='amd_xgbe'
path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref()
kmod_module 0x7f74d83862a0 released
modprobe: DEBUG: ../libkmod/libkmod.c:419 kmod_pool_del_module() del
0x7f74d83862a0 key='amd_xgbe'
modprobe: INFO: ../libkmod/libkmod.c:332 kmod_unref() context
0x7f74d83854c0 released
and the corresponding dmesg diff:
[ 151.599892] amd-xgbe 0000:06:00.1 eth0: net device enabled
[ 151.601333] amd-xgbe 0000:06:00.2 eth1: net device enabled
[ 151.606044] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
[ 151.646262] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
Cheers
Thomas
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-11 9:33 ` Thomas Kupper
@ 2022-02-11 9:49 ` Shyam Sundar S K
2022-02-11 12:01 ` Thomas Kupper
2022-02-11 15:48 ` Tom Lendacky
0 siblings, 2 replies; 15+ messages in thread
From: Shyam Sundar S K @ 2022-02-11 9:49 UTC (permalink / raw)
To: Thomas Kupper, Tom Lendacky; +Cc: netdev
On 2/11/2022 3:03 PM, Thomas Kupper wrote:
>
> Am 08.02.22 um 17:24 schrieb Tom Lendacky:
>> On 2/7/22 11:59, Thomas Kupper wrote:
>>>
>>> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>>>
>>>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>>>>>> Reloading the module and specify the dyndbg option to get some
>>>>>>> additional debug output.
>>>>>>>
>>>>>>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>>>>>>> configuration for this chip.
>>>>>>>
>>>>>> Right after boot:
>>>>>>
>>>>>> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>>>>> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>>>>> ...
>>>>>> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>>>>>> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>>>>> ...
>>>>>> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>>> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>>>> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>>>> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>>>> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
>>>>>> ...
>>>>>> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox
>>>>>> command did not complete
>>>>>> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
>>>>>> performed
>>>>>> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY
>>>>>> configuration
>>>>>>
>>>>>> Then after 'ifconfig enp6s0f2 up':
>>>>>>
>>>>>> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>>> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>>>> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>>>> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>>>> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
>>>>>> vs. 00000000 (enp6s0f2-pcs)
>>>>>> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>>>>>> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>>> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>>>>>>
>>>>> Please ensure that the ethtool msglvl is on for drv and probe. I was
>>>>> expecting to see some additional debug messages that I don't see here.
>>>>>
>>>>> Also, if you can provide the lspci output for the device (using -nn
>>>>> and
>>>>> -vv) that might be helpful as well.
>>>>>
>>>>> Shyam will be the best one to understand what is going on here.
>>>> On some other platforms, we have seen similar kind of problems getting
>>>> reported. There is a fix sent for validation.
>>>>
>>>> The root cause is that removal of xgbe driver is causing interrupt
>>>> storm
>>>> on the MP2 device (Sensor Fusion Hub).
>>>>
>>>> Shall submit a fix soon to upstream once the validation is done, you
>>>> may
>>>> give it a try with that and see if that helps.
>>>>
>>>> Thanks,
>>>> Shyam
>>>>
>>>>> Thanks,
>>>>> Tom
>>>
>>> Shyam, I will check the git logs for the relevant commit then from
>>> time to time.
>>> Looking at the code diff from OPNsense and the latest Linux kernel I
>>> assumed that there would much more to do then fix a irq strom (but I
>>> have no idea about the inner working of the kernel).
>>>
>>> Nevermind: Setting the 'msglvl 0x3' with ethtool the following info
>>> can be found in dmesg:
>>>
>>> Running : $ ifconfig enp6s0f2 up
>>> SIOCSIFFLAGS: Invalid argument
>>>
>>> ... and 'dmesg':
>>>
>>> [ 55.177447] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0: cpu=0, node=0
>>> [ 55.177456] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0:
>>> dma_regs=00000000d11bf3f1, dma_irq=74, tx=00000000dd57b5c4,
>>> rx=00000000d73e70f8
>>> [ 55.177464] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1: cpu=1, node=0
>>> [ 55.177467] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1:
>>> dma_regs=000000000d972dd7, dma_irq=75, tx=00000000573bcff8,
>>> rx=000000003d9a6f65
>>> [ 55.177473] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2: cpu=2, node=0
>>> [ 55.177476] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2:
>>> dma_regs=0000000046f71179, dma_irq=76, tx=00000000897116c9,
>>> rx=0000000004ba17e7
>>> [ 55.177480] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Tx ring:
>>> [ 55.177502] amd-xgbe 0000:06:00.2 enp6s0f2:
>>> rdesc=00000000794657ba, rdesc_dma=0x000000010fad8000,
>>> rdata=0000000008ace7d8, node=0
>>> [ 55.177507] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Rx ring:
>>> [ 55.177523] amd-xgbe 0000:06:00.2 enp6s0f2:
>>> rdesc=000000009313d9b3, rdesc_dma=0x0000000114538000,
>>> rdata=00000000510e3b77, node=0
>>> [ 55.177527] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Tx ring:
>>> [ 55.177543] amd-xgbe 0000:06:00.2 enp6s0f2:
>>> rdesc=00000000d26d9194, rdesc_dma=0x000000010a774000,
>>> rdata=00000000b9419829, node=0
>>> [ 55.177547] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Rx ring:
>>> [ 55.177564] amd-xgbe 0000:06:00.2 enp6s0f2:
>>> rdesc=0000000007bf60dd, rdesc_dma=0x000000010fb84000,
>>> rdata=00000000aa48e8c0, node=0
>>> [ 55.177568] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Tx ring:
>>> [ 55.177584] amd-xgbe 0000:06:00.2 enp6s0f2:
>>> rdesc=00000000e7e6c52e, rdesc_dma=0x000000010fa2a000,
>>> rdata=0000000017b5d85c, node=0
>>> [ 55.177587] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Rx ring:
>>> [ 55.177603] amd-xgbe 0000:06:00.2 enp6s0f2:
>>> rdesc=000000000898fbf4, rdesc_dma=0x0000000101f08000,
>>> rdata=00000000aded7d4c, node=0
>>> [ 55.182366] amd-xgbe 0000:06:00.2 enp6s0f2: TXq0 mapped to TC0
>>> [ 55.182381] amd-xgbe 0000:06:00.2 enp6s0f2: TXq1 mapped to TC1
>>> [ 55.182388] amd-xgbe 0000:06:00.2 enp6s0f2: TXq2 mapped to TC2
>>> [ 55.182395] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO0 mapped to RXq0
>>> [ 55.182400] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO1 mapped to RXq0
>>> [ 55.182405] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO2 mapped to RXq0
>>> [ 55.182410] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO3 mapped to RXq1
>>> [ 55.182414] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO4 mapped to RXq1
>>> [ 55.182418] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO5 mapped to RXq1
>>> [ 55.182423] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO6 mapped to RXq2
>>> [ 55.182427] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO7 mapped to RXq2
>>> [ 55.182473] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues,
>>> 21760 byte fifo per queue
>>> [ 55.182501] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues,
>>> 21760 byte fifo per queue
>>> [ 55.182544] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>>> for RXq0
>>> [ 55.182550] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>>> for RXq1
>>> [ 55.182556] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>>> for RXq2
>>> [ 56.178946] amd-xgbe 0000:06:00.2 enp6s0f2: SFP detected:
>>> [ 56.178954] amd-xgbe 0000:06:00.2 enp6s0f2: vendor: MikroTik
>>> [ 56.178958] amd-xgbe 0000:06:00.2 enp6s0f2: part number: S+AO0005
>>> [ 56.178961] amd-xgbe 0000:06:00.2 enp6s0f2: revision level: 1.0
>>> [ 56.178963] amd-xgbe 0000:06:00.2 enp6s0f2: serial number:
>>> STST050B1900001
>>>
>>
>> Ah, it's been a while since I've had to use the debug support. Could
>> you also set the module debug parameter to 0x37 (debug=0x37) when
>> loading the module. That will capture some of the debug messages that
>> are issued on driver load. Sorry about that...
>>
>> Thanks,
>> Tom
>
> Thanks Tom, I now got time to update to 5.17-rc3 and add the 'debug'
> module parameter. I assume that parameter works with the non-debug
> kernel? I don't really see any new messages related to the amd-xgbe driver:
>
> dmesg right after boot:
>
> [ 0.000000] Linux version 5.17.0-rc3-tk (jane@m920q-ubu21) (gcc
> (Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37)
> #12 SMP PREEMPT Tue Feb 8 19:52:19 CET 2022
> [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
> root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
> console=ttyS0,115200n8 amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
> ...
> [ 5.275730] amd-xgbe 0000:06:00.1 eth0: net device enabled
> [ 5.277766] amd-xgbe 0000:06:00.2 eth1: net device enabled
> [ 5.665315] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
> [ 5.696665] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>
> dmesg right after 'ifconfig enp6s0f2 up'
>
> [ 88.843454] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-0: cpu=0, node=0
> [ 88.843464] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-0: dma_regs=000000001078e433, dma_irq=55,
> tx=00000000e8736669, rx=00000000fadd04ec
> [ 88.843474] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-1: cpu=1, node=0
> [ 88.843478] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-1: dma_regs=000000003c3cbea8, dma_irq=56,
> tx=000000000836d88c, rx=00000000920d02c4
> [ 88.843485] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-2: cpu=2, node=0
> [ 88.843488] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-2: dma_regs=000000008d034191, dma_irq=57,
> tx=00000000a0664378, rx=00000000d72ce726
> [ 88.843493] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-0 - Tx ring:
> [ 88.843514] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
> rdesc=00000000c6703013, rdesc_dma=0x0000000101c44000,
> rdata=0000000029951e4c, node=0
> [ 88.843519] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-0 - Rx ring:
> [ 88.843537] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
> rdesc=000000003262c446, rdesc_dma=0x0000000103c74000,
> rdata=000000001b7a4275, node=0
> [ 88.843542] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-1 - Tx ring:
> [ 88.843560] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
> rdesc=000000007ce3cc7e, rdesc_dma=0x00000001023c0000,
> rdata=00000000c0fc51d9, node=0
> [ 88.843565] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-1 - Rx ring:
> [ 88.843583] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
> rdesc=00000000448612df, rdesc_dma=0x00000001185b6000,
> rdata=00000000a23b7f86, node=0
> [ 88.843587] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-2 - Tx ring:
> [ 88.843606] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
> rdesc=00000000e509050e, rdesc_dma=0x0000000104db2000,
> rdata=000000000d605e1a, node=0
> [ 88.843610] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
> enp6s0f2: channel-2 - Rx ring:
> [ 88.843629] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
> rdesc=00000000436c5cc6, rdesc_dma=0x0000000114aaa000,
> rdata=00000000246ed062, node=0
> [ 88.848416] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: TXq0 mapped to TC0
> [ 88.848432] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: TXq1 mapped to TC1
> [ 88.848440] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: TXq2 mapped to TC2
> [ 88.848449] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO0 mapped to RXq0
> [ 88.848455] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO1 mapped to RXq0
> [ 88.848461] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO2 mapped to RXq0
> [ 88.848467] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO3 mapped to RXq1
> [ 88.848472] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO4 mapped to RXq1
> [ 88.848478] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO5 mapped to RXq1
> [ 88.848483] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO6 mapped to RXq2
> [ 88.848489] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
> enp6s0f2: PRIO7 mapped to RXq2
> [ 88.848536] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues,
> 21760 byte fifo per queue
> [ 88.848565] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues,
> 21760 byte fifo per queue
> [ 88.848609] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
> 0000:06:00.2 enp6s0f2: flow control enabled for RXq0
> [ 88.848619] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
> 0000:06:00.2 enp6s0f2: flow control enabled for RXq1
> [ 88.848627] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
> 0000:06:00.2 enp6s0f2: flow control enabled for RXq2
> [ 89.862558] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
> enp6s0f2: SFP detected:
> [ 89.862567] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
> enp6s0f2: vendor: MikroTik
> [ 89.862572] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
> enp6s0f2: part number: S+AO0005
> [ 89.862576] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
> enp6s0f2: revision level: 1.0
> [ 89.862580] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
> enp6s0f2: serial number: STST050B1900001
>
> again, dmesg diff after 'rmmod':
Can you add this change and see if it solves the problem?
https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=68c2d6af1f1e
Thanks,
Shyam
>
> [ 127.068380] ------------[ cut here ]------------
> [ 127.068386] remove_proc_entry: removing non-empty directory 'irq/53',
> leaking at least 'enp6s0f2-i2c'
> [ 127.068398] WARNING: CPU: 4 PID: 803 at fs/proc/generic.c:715
> remove_proc_entry+0x196/0x1b0
> [ 127.068411] Modules linked in: nls_iso8859_1 intel_rapl_msr
> intel_rapl_common snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi
> edac_mce_amd snd_hda_codec snd_hda_core snd_hwdep snd_pcm kvm snd_timer
> snd_rn_pci_acp3x snd rapl efi_pstore k10temp soundcore snd_pci_acp3x ccp
> mac_hid sch_fq_codel msr drm ip_tables x_tables autofs4 btrfs
> blake2b_generic zstd_compress raid10 raid456 async_raid6_recov
> async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1
> raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
> aesni_intel crypto_simd igb nvme cryptd dca xhci_pci amd_xgbe(-)
> i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme_core i2c_algo_bit video
> spi_amd
> [ 127.068485] CPU: 4 PID: 803 Comm: rmmod Not tainted 5.17.0-rc3-tk #12
> [ 127.068490] Hardware name: Deciso B.V. DEC2700 - OPNsense
> Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
> [ 127.068493] RIP: 0010:remove_proc_entry+0x196/0x1b0
> [ 127.068499] Code: 60 50 5e 84 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
> c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 38 56 81
> 00 <0f> 0b e9 44 ff ff ff e8 9e c0 87 00 66 66 2e 0f 1f 84 00 00 00 00
> [ 127.068502] RSP: 0018:ffffaf2940fffb60 EFLAGS: 00010286
> [ 127.068506] RAX: 0000000000000000 RBX: ffff91fa4022ed80 RCX:
> 0000000000000000
> [ 127.068509] RDX: 0000000000000001 RSI: ffffffff845bf281 RDI:
> 00000000ffffffff
> [ 127.068511] RBP: ffffaf2940fffb90 R08: 0000000000000000 R09:
> ffffaf2940fff950
> [ 127.068513] R10: ffffaf2940fff948 R11: ffffffff84f55f48 R12:
> ffff91fa44e8c540
> [ 127.068515] R13: ffff91fa44e8c5c0 R14: 0000000000000036 R15:
> 0000000000000036
> [ 127.068517] FS: 00007f3a68f9c400(0000) GS:ffff91fa6af00000(0000)
> knlGS:0000000000000000
> [ 127.068520] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 127.068522] CR2: 00007fd6c2e7fd04 CR3: 0000000104ee0000 CR4:
> 00000000003506e0
> [ 127.068525] Call Trace:
> [ 127.068528] <TASK>
> [ 127.068534] unregister_irq_proc+0xe4/0x110
> [ 127.068541] free_desc+0x2e/0x70
> [ 127.068546] irq_free_descs+0x50/0x80
> [ 127.068550] irq_domain_free_irqs+0x16b/0x1c0
> [ 127.068554] __msi_domain_free_irqs+0xf1/0x160
> [ 127.068560] msi_domain_free_irqs_descs_locked+0x20/0x50
> [ 127.068565] pci_msi_teardown_msi_irqs+0x49/0x50
> [ 127.068571] pci_disable_msix.part.0+0xff/0x160
> [ 127.068575] pci_free_irq_vectors+0x45/0x60
> [ 127.068578] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
> [ 127.068596] pci_device_remove+0x39/0xa0
> [ 127.068602] __device_release_driver+0x181/0x250
> [ 127.068608] driver_detach+0xd3/0x120
> [ 127.068612] bus_remove_driver+0x59/0xd0
> [ 127.068615] driver_unregister+0x31/0x50
> [ 127.068619] pci_unregister_driver+0x40/0x90
> [ 127.068623] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
> [ 127.068639] xgbe_mod_exit+0x9/0x880 [amd_xgbe]
> [ 127.068654] __do_sys_delete_module.constprop.0+0x183/0x290
> [ 127.068660] ? exit_to_user_mode_prepare+0x49/0x1e0
> [ 127.068666] __x64_sys_delete_module+0x12/0x20
> [ 127.068670] do_syscall_64+0x5c/0xc0
> [ 127.068676] ? irqentry_exit_to_user_mode+0x9/0x20
> [ 127.068681] ? irqentry_exit+0x33/0x40
> [ 127.068685] ? exc_page_fault+0x89/0x180
> [ 127.068689] ? asm_exc_page_fault+0x8/0x30
> [ 127.068694] entry_SYSCALL_64_after_hwframe+0x44/0xae
> [ 127.068697] RIP: 0033:0x7f3a690cb8eb
> [ 127.068702] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
> c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
> 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
> [ 127.068704] RSP: 002b:00007ffed553a818 EFLAGS: 00000206 ORIG_RAX:
> 00000000000000b0
> [ 127.068708] RAX: ffffffffffffffda RBX: 00007f3a6a02e7b0 RCX:
> 00007f3a690cb8eb
> [ 127.068710] RDX: 000000000000000a RSI: 0000000000000800 RDI:
> 00007f3a6a02e818
> [ 127.068712] RBP: 0000000000000000 R08: 0000000000000000 R09:
> 0000000000000000
> [ 127.068714] R10: 00007f3a69163ac0 R11: 0000000000000206 R12:
> 00007ffed553aa70
> [ 127.068716] R13: 00007ffed553b84a R14: 00007f3a6a02e2a0 R15:
> 00007f3a6a02e7b0
> [ 127.068722] </TASK>
> [ 127.068723] ---[ end trace 0000000000000000 ]---
> [ 127.068744] ------------[ cut here ]------------
> [ 127.068746] remove_proc_entry: removing non-empty directory 'irq/54',
> leaking at least 'enp6s0f2-pcs'
> [ 127.068755] WARNING: CPU: 4 PID: 803 at fs/proc/generic.c:715
> remove_proc_entry+0x196/0x1b0
> [ 127.068761] Modules linked in: nls_iso8859_1 intel_rapl_msr
> intel_rapl_common snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi
> edac_mce_amd snd_hda_codec snd_hda_core snd_hwdep snd_pcm kvm snd_timer
> snd_rn_pci_acp3x snd rapl efi_pstore k10temp soundcore snd_pci_acp3x ccp
> mac_hid sch_fq_codel msr drm ip_tables x_tables autofs4 btrfs
> blake2b_generic zstd_compress raid10 raid456 async_raid6_recov
> async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1
> raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
> aesni_intel crypto_simd igb nvme cryptd dca xhci_pci amd_xgbe(-)
> i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme_core i2c_algo_bit video
> spi_amd
> [ 127.068810] CPU: 4 PID: 803 Comm: rmmod Tainted: G W
> 5.17.0-rc3-tk #12
> [ 127.068814] Hardware name: Deciso B.V. DEC2700 - OPNsense
> Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
> [ 127.068815] RIP: 0010:remove_proc_entry+0x196/0x1b0
> [ 127.068820] Code: 60 50 5e 84 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
> c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 38 56 81
> 00 <0f> 0b e9 44 ff ff ff e8 9e c0 87 00 66 66 2e 0f 1f 84 00 00 00 00
> [ 127.068822] RSP: 0018:ffffaf2940fffb60 EFLAGS: 00010286
> [ 127.068825] RAX: 0000000000000000 RBX: ffff91fa4022ed80 RCX:
> 0000000000000000
> [ 127.068827] RDX: 0000000000000001 RSI: ffffffff845bf281 RDI:
> 00000000ffffffff
> [ 127.068829] RBP: ffffaf2940fffb90 R08: 0000000000000000 R09:
> ffffaf2940fff950
> [ 127.068830] R10: ffffaf2940fff948 R11: ffffffff84f55f48 R12:
> ffff91fa4eca7000
> [ 127.068832] R13: ffff91fa4eca7080 R14: 0000000000000037 R15:
> 0000000000000037
> [ 127.068834] FS: 00007f3a68f9c400(0000) GS:ffff91fa6af00000(0000)
> knlGS:0000000000000000
> [ 127.068837] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 127.068839] CR2: 00007fd6c2e7fd04 CR3: 0000000104ee0000 CR4:
> 00000000003506e0
> [ 127.068841] Call Trace:
> [ 127.068842] <TASK>
> [ 127.068844] unregister_irq_proc+0xe4/0x110
> [ 127.068849] free_desc+0x2e/0x70
> [ 127.068852] irq_free_descs+0x50/0x80
> [ 127.068856] irq_domain_free_irqs+0x16b/0x1c0
> [ 127.068860] __msi_domain_free_irqs+0xf1/0x160
> [ 127.068865] msi_domain_free_irqs_descs_locked+0x20/0x50
> [ 127.068870] pci_msi_teardown_msi_irqs+0x49/0x50
> [ 127.068873] pci_disable_msix.part.0+0xff/0x160
> [ 127.068877] pci_free_irq_vectors+0x45/0x60
> [ 127.068881] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
> [ 127.068896] pci_device_remove+0x39/0xa0
> [ 127.068900] __device_release_driver+0x181/0x250
> [ 127.068904] driver_detach+0xd3/0x120
> [ 127.068908] bus_remove_driver+0x59/0xd0
> [ 127.068911] driver_unregister+0x31/0x50
> [ 127.068914] pci_unregister_driver+0x40/0x90
> [ 127.068919] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
> [ 127.068933] xgbe_mod_exit+0x9/0x880 [amd_xgbe]
> [ 127.068948] __do_sys_delete_module.constprop.0+0x183/0x290
> [ 127.068952] ? exit_to_user_mode_prepare+0x49/0x1e0
> [ 127.068957] __x64_sys_delete_module+0x12/0x20
> [ 127.068961] do_syscall_64+0x5c/0xc0
> [ 127.068964] ? irqentry_exit_to_user_mode+0x9/0x20
> [ 127.068969] ? irqentry_exit+0x33/0x40
> [ 127.068973] ? exc_page_fault+0x89/0x180
> [ 127.068977] ? asm_exc_page_fault+0x8/0x30
> [ 127.068980] entry_SYSCALL_64_after_hwframe+0x44/0xae
> [ 127.068983] RIP: 0033:0x7f3a690cb8eb
> [ 127.068985] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
> c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
> 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
> [ 127.068987] RSP: 002b:00007ffed553a818 EFLAGS: 00000206 ORIG_RAX:
> 00000000000000b0
> [ 127.068990] RAX: ffffffffffffffda RBX: 00007f3a6a02e7b0 RCX:
> 00007f3a690cb8eb
> [ 127.068991] RDX: 000000000000000a RSI: 0000000000000800 RDI:
> 00007f3a6a02e818
> [ 127.068993] RBP: 0000000000000000 R08: 0000000000000000 R09:
> 0000000000000000
> [ 127.068994] R10: 00007f3a69163ac0 R11: 0000000000000206 R12:
> 00007ffed553aa70
> [ 127.068996] R13: 00007ffed553b84a R14: 00007f3a6a02e2a0 R15:
> 00007f3a6a02e7b0
> [ 127.068999] </TASK>
> [ 127.069000] ---[ end trace 0000000000000000 ]---
> [ 127.667264] irq 31: nobody cared (try booting with the "irqpoll" option)
> [ 127.674758] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G W
> 5.17.0-rc3-tk #12
> [ 127.674764] Hardware name: Deciso B.V. DEC2700 - OPNsense
> Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
> [ 127.674766] Call Trace:
> [ 127.674769] <IRQ>
> [ 127.674773] dump_stack_lvl+0x4c/0x63
> [ 127.674781] dump_stack+0x10/0x12
> [ 127.674784] __report_bad_irq+0x3a/0xaf
> [ 127.674789] note_interrupt.cold+0xb/0x60
> [ 127.674793] ? __this_cpu_preempt_check+0x13/0x20
> [ 127.674799] handle_irq_event+0x71/0x80
> [ 127.674805] handle_fasteoi_irq+0x95/0x1e0
> [ 127.674810] __common_interrupt+0x6e/0x110
> [ 127.674815] common_interrupt+0xbd/0xe0
> [ 127.674819] </IRQ>
> [ 127.674820] <TASK>
> [ 127.674822] asm_common_interrupt+0x1e/0x40
> [ 127.674826] RIP: 0010:cpuidle_enter_state+0xdf/0x380
> [ 127.674834] Code: ff e8 25 76 73 ff 80 7d d7 00 74 17 9c 58 0f 1f 44
> 00 00 f6 c4 02 0f 85 82 02 00 00 31 ff e8 18 8c 7a ff fb 66 0f 1f 44 00
> 00 <45> 85 ff 0f 88 1a 01 00 00 49 63 d7 4c 89 f1 48 2b 4d c8 48 8d 04
> [ 127.674837] RSP: 0018:ffffaf29400e3e68 EFLAGS: 00000246
> [ 127.674841] RAX: ffff91fa6af00000 RBX: 0000000000000002 RCX:
> 000000000000001f
> [ 127.674843] RDX: 0000000000000000 RSI: ffffffff845bf281 RDI:
> ffffffff845cddcf
> [ 127.674845] RBP: ffffaf29400e3ea0 R08: 0000001db98fd21c R09:
> 0000001d7b8fd3fc
> [ 127.674847] R10: 0000000000000001 R11: ffff91fa6af2fd84 R12:
> ffff91fa41de6c00
> [ 127.674849] R13: ffffffff8506e4c0 R14: 0000001db98fd21c R15:
> 0000000000000002
> [ 127.674854] ? cpuidle_enter_state+0xbb/0x380
> [ 127.674860] cpuidle_enter+0x2e/0x40
> [ 127.674864] do_idle+0x203/0x290
> [ 127.674869] cpu_startup_entry+0x20/0x30
> [ 127.674872] start_secondary+0x118/0x150
> [ 127.674877] secondary_startup_64_no_verify+0xd5/0xdb
> [ 127.674885] </TASK>
> [ 127.674886] handlers:
> [ 127.677425] [<00000000b61e344c>] amd_mp2_irq_isr [i2c_amd_mp2_pci]
> [ 127.684335] Disabling IRQ #31
>
> and command line output after 'modprobe -vvv amd_xgbe':
>
> jane@dec740-ubu21:~$ sudo modprobe -vvv amd_xgbe
> modprobe: INFO: ../libkmod/libkmod.c:365 kmod_set_log_fn() custom
> logging function 0x7f74d79de780 registered
> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
> file=/lib/modules/5.17.0-rc3-tk/modules.dep.bin
> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
> file=/lib/modules/5.17.0-rc3-tk/modules.alias.bin
> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
> file=/lib/modules/5.17.0-rc3-tk/modules.symbols.bin
> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
> file=/lib/modules/5.17.0-rc3-tk/modules.builtin.alias.bin
> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
> file=/lib/modules/5.17.0-rc3-tk/modules.builtin.bin
> modprobe: DEBUG: ../libkmod/libkmod-module.c:556
> kmod_module_new_from_lookup() input alias=amd_xgbe, normalized=amd_xgbe
> modprobe: DEBUG: ../libkmod/libkmod-module.c:562
> kmod_module_new_from_lookup() lookup modules.dep amd_xgbe
> modprobe: DEBUG: ../libkmod/libkmod.c:595 kmod_search_moddep() use
> mmaped index 'modules.dep' modname=amd_xgbe
> modprobe: DEBUG: ../libkmod/libkmod.c:403 kmod_pool_get_module() get
> module name='amd_xgbe' found=(nil)
> modprobe: DEBUG: ../libkmod/libkmod.c:411 kmod_pool_add_module() add
> 0x7f74d83862a0 key='amd_xgbe'
> modprobe: DEBUG: ../libkmod/libkmod-module.c:202
> kmod_module_parse_depline() 0 dependencies for amd_xgbe
> modprobe: DEBUG: ../libkmod/libkmod-module.c:589
> kmod_module_new_from_lookup() lookup amd_xgbe=0, list=0x7f74d8385c40
> modprobe: DEBUG: ../libkmod/libkmod.c:500 lookup_builtin_file() use
> mmaped index 'modules.builtin' modname=amd_xgbe
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1760
> kmod_module_get_initstate() could not open
> '/sys/module/amd_xgbe/initstate': No such file or directory
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1770
> kmod_module_get_initstate() could not open '/sys/module/amd_xgbe': No
> such file or directory
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=snd_pcsp mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=snd_usb_audio mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=cx88_alsa mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=snd_atiixp_modem mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=snd_intel8x0m mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=snd_via82xx_modem mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
> kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=md_mod mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=bonding mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=dummy mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
> kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
> kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
> kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
> mod->alias=(null)
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1760
> kmod_module_get_initstate() could not open
> '/sys/module/amd_xgbe/initstate': No such file or directory
> modprobe: DEBUG: ../libkmod/libkmod-module.c:1770
> kmod_module_get_initstate() could not open '/sys/module/amd_xgbe': No
> such file or directory
> modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
> name='amd_xgbe'
> path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
>
> modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
> name='amd_xgbe'
> path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
>
> insmod
> /lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko
> dyndbg="+pfm" debug=0x37 dyndbg=+p debug=0x37
> modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
> name='amd_xgbe'
> path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
>
> modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref()
> kmod_module 0x7f74d83862a0 released
> modprobe: DEBUG: ../libkmod/libkmod.c:419 kmod_pool_del_module() del
> 0x7f74d83862a0 key='amd_xgbe'
> modprobe: INFO: ../libkmod/libkmod.c:332 kmod_unref() context
> 0x7f74d83854c0 released
>
> and the corresponding dmesg diff:
>
> [ 151.599892] amd-xgbe 0000:06:00.1 eth0: net device enabled
> [ 151.601333] amd-xgbe 0000:06:00.2 eth1: net device enabled
> [ 151.606044] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
> [ 151.646262] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>
>
> Cheers
> Thomas
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-11 9:49 ` Shyam Sundar S K
@ 2022-02-11 12:01 ` Thomas Kupper
2022-02-11 15:48 ` Tom Lendacky
1 sibling, 0 replies; 15+ messages in thread
From: Thomas Kupper @ 2022-02-11 12:01 UTC (permalink / raw)
To: Shyam Sundar S K, Tom Lendacky; +Cc: netdev
Am 11.02.22 um 10:49 schrieb Shyam Sundar S K:
>
> On 2/11/2022 3:03 PM, Thomas Kupper wrote:
>> Am 08.02.22 um 17:24 schrieb Tom Lendacky:
>>> On 2/7/22 11:59, Thomas Kupper wrote:
>>>> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>>>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>>>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>>>>>>> Reloading the module and specify the dyndbg option to get some
>>>>>>>> additional debug output.
>>>>>>>>
>>>>>>>> I'm adding Shyam to the thread, too, as I'm not familiar with the
>>>>>>>> configuration for this chip.
>>>>>>>>
>>>>>>> Right after boot:
>>>>>>>
>>>>>>> [ 5.352977] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>>>>>> [ 5.354198] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>>>>>> ...
>>>>>>> [ 5.382185] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>>>>>>> [ 5.426931] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>>>>>> ...
>>>>>>> [ 9.701637] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>>>> [ 9.701679] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>>>>> [ 9.701715] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>>>>> [ 9.738191] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>>>>> [ 9.738219] amd-xgbe 0000:06:00.2 enp6s0f2: starting I2C
>>>>>>> ...
>>>>>>> [ 10.742622] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox
>>>>>>> command did not complete
>>>>>>> [ 10.742710] amd-xgbe 0000:06:00.2 enp6s0f2: firmware mailbox reset
>>>>>>> performed
>>>>>>> [ 10.750813] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>>> [ 10.768366] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>>> [ 10.768371] amd-xgbe 0000:06:00.2 enp6s0f2: fixed PHY
>>>>>>> configuration
>>>>>>>
>>>>>>> Then after 'ifconfig enp6s0f2 up':
>>>>>>>
>>>>>>> [ 189.184928] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>>>> [ 189.191828] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>>> [ 189.191863] amd-xgbe 0000:06:00.2 enp6s0f2: CL73 AN disabled
>>>>>>> [ 189.191894] amd-xgbe 0000:06:00.2 enp6s0f2: CL37 AN disabled
>>>>>>> [ 189.196338] amd-xgbe 0000:06:00.2 enp6s0f2: starting PHY
>>>>>>> [ 189.198792] amd-xgbe 0000:06:00.2 enp6s0f2: 10GbE SFI mode set
>>>>>>> [ 189.212036] genirq: Flags mismatch irq 69. 00000000 (enp6s0f2-pcs)
>>>>>>> vs. 00000000 (enp6s0f2-pcs)
>>>>>>> [ 189.221700] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>>>>>>> [ 189.231051] amd-xgbe 0000:06:00.2 enp6s0f2: phy powered off
>>>>>>> [ 189.231054] amd-xgbe 0000:06:00.2 enp6s0f2: stopping I2C
>>>>>>>
>>>>>> Please ensure that the ethtool msglvl is on for drv and probe. I was
>>>>>> expecting to see some additional debug messages that I don't see here.
>>>>>>
>>>>>> Also, if you can provide the lspci output for the device (using -nn
>>>>>> and
>>>>>> -vv) that might be helpful as well.
>>>>>>
>>>>>> Shyam will be the best one to understand what is going on here.
>>>>> On some other platforms, we have seen similar kind of problems getting
>>>>> reported. There is a fix sent for validation.
>>>>>
>>>>> The root cause is that removal of xgbe driver is causing interrupt
>>>>> storm
>>>>> on the MP2 device (Sensor Fusion Hub).
>>>>>
>>>>> Shall submit a fix soon to upstream once the validation is done, you
>>>>> may
>>>>> give it a try with that and see if that helps.
>>>>>
>>>>> Thanks,
>>>>> Shyam
>>>>>
>>>>>> Thanks,
>>>>>> Tom
>>>> Shyam, I will check the git logs for the relevant commit then from
>>>> time to time.
>>>> Looking at the code diff from OPNsense and the latest Linux kernel I
>>>> assumed that there would much more to do then fix a irq strom (but I
>>>> have no idea about the inner working of the kernel).
>>>>
>>>> Nevermind: Setting the 'msglvl 0x3' with ethtool the following info
>>>> can be found in dmesg:
>>>>
>>>> Running : $ ifconfig enp6s0f2 up
>>>> SIOCSIFFLAGS: Invalid argument
>>>>
>>>> ... and 'dmesg':
>>>>
>>>> [ 55.177447] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0: cpu=0, node=0
>>>> [ 55.177456] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0:
>>>> dma_regs=00000000d11bf3f1, dma_irq=74, tx=00000000dd57b5c4,
>>>> rx=00000000d73e70f8
>>>> [ 55.177464] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1: cpu=1, node=0
>>>> [ 55.177467] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1:
>>>> dma_regs=000000000d972dd7, dma_irq=75, tx=00000000573bcff8,
>>>> rx=000000003d9a6f65
>>>> [ 55.177473] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2: cpu=2, node=0
>>>> [ 55.177476] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2:
>>>> dma_regs=0000000046f71179, dma_irq=76, tx=00000000897116c9,
>>>> rx=0000000004ba17e7
>>>> [ 55.177480] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Tx ring:
>>>> [ 55.177502] amd-xgbe 0000:06:00.2 enp6s0f2:
>>>> rdesc=00000000794657ba, rdesc_dma=0x000000010fad8000,
>>>> rdata=0000000008ace7d8, node=0
>>>> [ 55.177507] amd-xgbe 0000:06:00.2 enp6s0f2: channel-0 - Rx ring:
>>>> [ 55.177523] amd-xgbe 0000:06:00.2 enp6s0f2:
>>>> rdesc=000000009313d9b3, rdesc_dma=0x0000000114538000,
>>>> rdata=00000000510e3b77, node=0
>>>> [ 55.177527] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Tx ring:
>>>> [ 55.177543] amd-xgbe 0000:06:00.2 enp6s0f2:
>>>> rdesc=00000000d26d9194, rdesc_dma=0x000000010a774000,
>>>> rdata=00000000b9419829, node=0
>>>> [ 55.177547] amd-xgbe 0000:06:00.2 enp6s0f2: channel-1 - Rx ring:
>>>> [ 55.177564] amd-xgbe 0000:06:00.2 enp6s0f2:
>>>> rdesc=0000000007bf60dd, rdesc_dma=0x000000010fb84000,
>>>> rdata=00000000aa48e8c0, node=0
>>>> [ 55.177568] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Tx ring:
>>>> [ 55.177584] amd-xgbe 0000:06:00.2 enp6s0f2:
>>>> rdesc=00000000e7e6c52e, rdesc_dma=0x000000010fa2a000,
>>>> rdata=0000000017b5d85c, node=0
>>>> [ 55.177587] amd-xgbe 0000:06:00.2 enp6s0f2: channel-2 - Rx ring:
>>>> [ 55.177603] amd-xgbe 0000:06:00.2 enp6s0f2:
>>>> rdesc=000000000898fbf4, rdesc_dma=0x0000000101f08000,
>>>> rdata=00000000aded7d4c, node=0
>>>> [ 55.182366] amd-xgbe 0000:06:00.2 enp6s0f2: TXq0 mapped to TC0
>>>> [ 55.182381] amd-xgbe 0000:06:00.2 enp6s0f2: TXq1 mapped to TC1
>>>> [ 55.182388] amd-xgbe 0000:06:00.2 enp6s0f2: TXq2 mapped to TC2
>>>> [ 55.182395] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO0 mapped to RXq0
>>>> [ 55.182400] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO1 mapped to RXq0
>>>> [ 55.182405] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO2 mapped to RXq0
>>>> [ 55.182410] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO3 mapped to RXq1
>>>> [ 55.182414] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO4 mapped to RXq1
>>>> [ 55.182418] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO5 mapped to RXq1
>>>> [ 55.182423] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO6 mapped to RXq2
>>>> [ 55.182427] amd-xgbe 0000:06:00.2 enp6s0f2: PRIO7 mapped to RXq2
>>>> [ 55.182473] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues,
>>>> 21760 byte fifo per queue
>>>> [ 55.182501] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues,
>>>> 21760 byte fifo per queue
>>>> [ 55.182544] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>>>> for RXq0
>>>> [ 55.182550] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>>>> for RXq1
>>>> [ 55.182556] amd-xgbe 0000:06:00.2 enp6s0f2: flow control enabled
>>>> for RXq2
>>>> [ 56.178946] amd-xgbe 0000:06:00.2 enp6s0f2: SFP detected:
>>>> [ 56.178954] amd-xgbe 0000:06:00.2 enp6s0f2: vendor: MikroTik
>>>> [ 56.178958] amd-xgbe 0000:06:00.2 enp6s0f2: part number: S+AO0005
>>>> [ 56.178961] amd-xgbe 0000:06:00.2 enp6s0f2: revision level: 1.0
>>>> [ 56.178963] amd-xgbe 0000:06:00.2 enp6s0f2: serial number:
>>>> STST050B1900001
>>>>
>>> Ah, it's been a while since I've had to use the debug support. Could
>>> you also set the module debug parameter to 0x37 (debug=0x37) when
>>> loading the module. That will capture some of the debug messages that
>>> are issued on driver load. Sorry about that...
>>>
>>> Thanks,
>>> Tom
>> Thanks Tom, I now got time to update to 5.17-rc3 and add the 'debug'
>> module parameter. I assume that parameter works with the non-debug
>> kernel? I don't really see any new messages related to the amd-xgbe driver:
>>
>> dmesg right after boot:
>>
>> [ 0.000000] Linux version 5.17.0-rc3-tk (jane@m920q-ubu21) (gcc
>> (Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37)
>> #12 SMP PREEMPT Tue Feb 8 19:52:19 CET 2022
>> [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
>> root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
>> console=ttyS0,115200n8 amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
>> ...
>> [ 5.275730] amd-xgbe 0000:06:00.1 eth0: net device enabled
>> [ 5.277766] amd-xgbe 0000:06:00.2 eth1: net device enabled
>> [ 5.665315] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>> [ 5.696665] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>>
>> dmesg right after 'ifconfig enp6s0f2 up'
>>
>> [ 88.843454] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-0: cpu=0, node=0
>> [ 88.843464] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-0: dma_regs=000000001078e433, dma_irq=55,
>> tx=00000000e8736669, rx=00000000fadd04ec
>> [ 88.843474] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-1: cpu=1, node=0
>> [ 88.843478] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-1: dma_regs=000000003c3cbea8, dma_irq=56,
>> tx=000000000836d88c, rx=00000000920d02c4
>> [ 88.843485] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-2: cpu=2, node=0
>> [ 88.843488] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-2: dma_regs=000000008d034191, dma_irq=57,
>> tx=00000000a0664378, rx=00000000d72ce726
>> [ 88.843493] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-0 - Tx ring:
>> [ 88.843514] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=00000000c6703013, rdesc_dma=0x0000000101c44000,
>> rdata=0000000029951e4c, node=0
>> [ 88.843519] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-0 - Rx ring:
>> [ 88.843537] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=000000003262c446, rdesc_dma=0x0000000103c74000,
>> rdata=000000001b7a4275, node=0
>> [ 88.843542] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-1 - Tx ring:
>> [ 88.843560] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=000000007ce3cc7e, rdesc_dma=0x00000001023c0000,
>> rdata=00000000c0fc51d9, node=0
>> [ 88.843565] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-1 - Rx ring:
>> [ 88.843583] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=00000000448612df, rdesc_dma=0x00000001185b6000,
>> rdata=00000000a23b7f86, node=0
>> [ 88.843587] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-2 - Tx ring:
>> [ 88.843606] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=00000000e509050e, rdesc_dma=0x0000000104db2000,
>> rdata=000000000d605e1a, node=0
>> [ 88.843610] amd_xgbe:xgbe_alloc_ring_resources: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-2 - Rx ring:
>> [ 88.843629] amd_xgbe:xgbe_init_ring: amd-xgbe 0000:06:00.2 enp6s0f2:
>> rdesc=00000000436c5cc6, rdesc_dma=0x0000000114aaa000,
>> rdata=00000000246ed062, node=0
>> [ 88.848416] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: TXq0 mapped to TC0
>> [ 88.848432] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: TXq1 mapped to TC1
>> [ 88.848440] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: TXq2 mapped to TC2
>> [ 88.848449] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO0 mapped to RXq0
>> [ 88.848455] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO1 mapped to RXq0
>> [ 88.848461] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO2 mapped to RXq0
>> [ 88.848467] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO3 mapped to RXq1
>> [ 88.848472] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO4 mapped to RXq1
>> [ 88.848478] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO5 mapped to RXq1
>> [ 88.848483] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO6 mapped to RXq2
>> [ 88.848489] amd_xgbe:xgbe_config_queue_mapping: amd-xgbe 0000:06:00.2
>> enp6s0f2: PRIO7 mapped to RXq2
>> [ 88.848536] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Tx hardware queues,
>> 21760 byte fifo per queue
>> [ 88.848565] amd-xgbe 0000:06:00.2 enp6s0f2: 3 Rx hardware queues,
>> 21760 byte fifo per queue
>> [ 88.848609] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
>> 0000:06:00.2 enp6s0f2: flow control enabled for RXq0
>> [ 88.848619] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
>> 0000:06:00.2 enp6s0f2: flow control enabled for RXq1
>> [ 88.848627] amd_xgbe:xgbe_enable_tx_flow_control: amd-xgbe
>> 0000:06:00.2 enp6s0f2: flow control enabled for RXq2
>> [ 89.862558] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
>> enp6s0f2: SFP detected:
>> [ 89.862567] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
>> enp6s0f2: vendor: MikroTik
>> [ 89.862572] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
>> enp6s0f2: part number: S+AO0005
>> [ 89.862576] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
>> enp6s0f2: revision level: 1.0
>> [ 89.862580] amd_xgbe:xgbe_phy_sfp_eeprom_info: amd-xgbe 0000:06:00.2
>> enp6s0f2: serial number: STST050B1900001
>>
>> again, dmesg diff after 'rmmod':
> Can you add this change and see if it solves the problem?
>
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=68c2d6af1f1e
>
> Thanks,
> Shyam
Hi Shyam,
Unfortunately nothing changes with that commit. There are no new debug
messages and it still behaves the same.
Any changes you could have a look a the diff from
opnsense/src/sys/dev/axgbe/* and
linux-kernel/drivers/net/ethernet/amd/xgbe/* for relevant changes?
/Thomas
>> [ 127.068380] ------------[ cut here ]------------
>> [ 127.068386] remove_proc_entry: removing non-empty directory 'irq/53',
>> leaking at least 'enp6s0f2-i2c'
>> [ 127.068398] WARNING: CPU: 4 PID: 803 at fs/proc/generic.c:715
>> remove_proc_entry+0x196/0x1b0
>> [ 127.068411] Modules linked in: nls_iso8859_1 intel_rapl_msr
>> intel_rapl_common snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi
>> edac_mce_amd snd_hda_codec snd_hda_core snd_hwdep snd_pcm kvm snd_timer
>> snd_rn_pci_acp3x snd rapl efi_pstore k10temp soundcore snd_pci_acp3x ccp
>> mac_hid sch_fq_codel msr drm ip_tables x_tables autofs4 btrfs
>> blake2b_generic zstd_compress raid10 raid456 async_raid6_recov
>> async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1
>> raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
>> aesni_intel crypto_simd igb nvme cryptd dca xhci_pci amd_xgbe(-)
>> i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme_core i2c_algo_bit video
>> spi_amd
>> [ 127.068485] CPU: 4 PID: 803 Comm: rmmod Not tainted 5.17.0-rc3-tk #12
>> [ 127.068490] Hardware name: Deciso B.V. DEC2700 - OPNsense
>> Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
>> [ 127.068493] RIP: 0010:remove_proc_entry+0x196/0x1b0
>> [ 127.068499] Code: 60 50 5e 84 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
>> c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 38 56 81
>> 00 <0f> 0b e9 44 ff ff ff e8 9e c0 87 00 66 66 2e 0f 1f 84 00 00 00 00
>> [ 127.068502] RSP: 0018:ffffaf2940fffb60 EFLAGS: 00010286
>> [ 127.068506] RAX: 0000000000000000 RBX: ffff91fa4022ed80 RCX:
>> 0000000000000000
>> [ 127.068509] RDX: 0000000000000001 RSI: ffffffff845bf281 RDI:
>> 00000000ffffffff
>> [ 127.068511] RBP: ffffaf2940fffb90 R08: 0000000000000000 R09:
>> ffffaf2940fff950
>> [ 127.068513] R10: ffffaf2940fff948 R11: ffffffff84f55f48 R12:
>> ffff91fa44e8c540
>> [ 127.068515] R13: ffff91fa44e8c5c0 R14: 0000000000000036 R15:
>> 0000000000000036
>> [ 127.068517] FS: 00007f3a68f9c400(0000) GS:ffff91fa6af00000(0000)
>> knlGS:0000000000000000
>> [ 127.068520] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 127.068522] CR2: 00007fd6c2e7fd04 CR3: 0000000104ee0000 CR4:
>> 00000000003506e0
>> [ 127.068525] Call Trace:
>> [ 127.068528] <TASK>
>> [ 127.068534] unregister_irq_proc+0xe4/0x110
>> [ 127.068541] free_desc+0x2e/0x70
>> [ 127.068546] irq_free_descs+0x50/0x80
>> [ 127.068550] irq_domain_free_irqs+0x16b/0x1c0
>> [ 127.068554] __msi_domain_free_irqs+0xf1/0x160
>> [ 127.068560] msi_domain_free_irqs_descs_locked+0x20/0x50
>> [ 127.068565] pci_msi_teardown_msi_irqs+0x49/0x50
>> [ 127.068571] pci_disable_msix.part.0+0xff/0x160
>> [ 127.068575] pci_free_irq_vectors+0x45/0x60
>> [ 127.068578] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
>> [ 127.068596] pci_device_remove+0x39/0xa0
>> [ 127.068602] __device_release_driver+0x181/0x250
>> [ 127.068608] driver_detach+0xd3/0x120
>> [ 127.068612] bus_remove_driver+0x59/0xd0
>> [ 127.068615] driver_unregister+0x31/0x50
>> [ 127.068619] pci_unregister_driver+0x40/0x90
>> [ 127.068623] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
>> [ 127.068639] xgbe_mod_exit+0x9/0x880 [amd_xgbe]
>> [ 127.068654] __do_sys_delete_module.constprop.0+0x183/0x290
>> [ 127.068660] ? exit_to_user_mode_prepare+0x49/0x1e0
>> [ 127.068666] __x64_sys_delete_module+0x12/0x20
>> [ 127.068670] do_syscall_64+0x5c/0xc0
>> [ 127.068676] ? irqentry_exit_to_user_mode+0x9/0x20
>> [ 127.068681] ? irqentry_exit+0x33/0x40
>> [ 127.068685] ? exc_page_fault+0x89/0x180
>> [ 127.068689] ? asm_exc_page_fault+0x8/0x30
>> [ 127.068694] entry_SYSCALL_64_after_hwframe+0x44/0xae
>> [ 127.068697] RIP: 0033:0x7f3a690cb8eb
>> [ 127.068702] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
>> c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
>> 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
>> [ 127.068704] RSP: 002b:00007ffed553a818 EFLAGS: 00000206 ORIG_RAX:
>> 00000000000000b0
>> [ 127.068708] RAX: ffffffffffffffda RBX: 00007f3a6a02e7b0 RCX:
>> 00007f3a690cb8eb
>> [ 127.068710] RDX: 000000000000000a RSI: 0000000000000800 RDI:
>> 00007f3a6a02e818
>> [ 127.068712] RBP: 0000000000000000 R08: 0000000000000000 R09:
>> 0000000000000000
>> [ 127.068714] R10: 00007f3a69163ac0 R11: 0000000000000206 R12:
>> 00007ffed553aa70
>> [ 127.068716] R13: 00007ffed553b84a R14: 00007f3a6a02e2a0 R15:
>> 00007f3a6a02e7b0
>> [ 127.068722] </TASK>
>> [ 127.068723] ---[ end trace 0000000000000000 ]---
>> [ 127.068744] ------------[ cut here ]------------
>> [ 127.068746] remove_proc_entry: removing non-empty directory 'irq/54',
>> leaking at least 'enp6s0f2-pcs'
>> [ 127.068755] WARNING: CPU: 4 PID: 803 at fs/proc/generic.c:715
>> remove_proc_entry+0x196/0x1b0
>> [ 127.068761] Modules linked in: nls_iso8859_1 intel_rapl_msr
>> intel_rapl_common snd_hda_intel snd_intel_dspcfg snd_intel_sdw_acpi
>> edac_mce_amd snd_hda_codec snd_hda_core snd_hwdep snd_pcm kvm snd_timer
>> snd_rn_pci_acp3x snd rapl efi_pstore k10temp soundcore snd_pci_acp3x ccp
>> mac_hid sch_fq_codel msr drm ip_tables x_tables autofs4 btrfs
>> blake2b_generic zstd_compress raid10 raid456 async_raid6_recov
>> async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1
>> raid0 multipath linear crct10dif_pclmul crc32_pclmul ghash_clmulni_intel
>> aesni_intel crypto_simd igb nvme cryptd dca xhci_pci amd_xgbe(-)
>> i2c_piix4 i2c_amd_mp2_pci xhci_pci_renesas nvme_core i2c_algo_bit video
>> spi_amd
>> [ 127.068810] CPU: 4 PID: 803 Comm: rmmod Tainted: G W
>> 5.17.0-rc3-tk #12
>> [ 127.068814] Hardware name: Deciso B.V. DEC2700 - OPNsense
>> Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
>> [ 127.068815] RIP: 0010:remove_proc_entry+0x196/0x1b0
>> [ 127.068820] Code: 60 50 5e 84 48 85 c0 48 8d 90 78 ff ff ff 48 0f 45
>> c2 49 8b 54 24 78 4c 8b 80 a0 00 00 00 48 8b 92 a0 00 00 00 e8 38 56 81
>> 00 <0f> 0b e9 44 ff ff ff e8 9e c0 87 00 66 66 2e 0f 1f 84 00 00 00 00
>> [ 127.068822] RSP: 0018:ffffaf2940fffb60 EFLAGS: 00010286
>> [ 127.068825] RAX: 0000000000000000 RBX: ffff91fa4022ed80 RCX:
>> 0000000000000000
>> [ 127.068827] RDX: 0000000000000001 RSI: ffffffff845bf281 RDI:
>> 00000000ffffffff
>> [ 127.068829] RBP: ffffaf2940fffb90 R08: 0000000000000000 R09:
>> ffffaf2940fff950
>> [ 127.068830] R10: ffffaf2940fff948 R11: ffffffff84f55f48 R12:
>> ffff91fa4eca7000
>> [ 127.068832] R13: ffff91fa4eca7080 R14: 0000000000000037 R15:
>> 0000000000000037
>> [ 127.068834] FS: 00007f3a68f9c400(0000) GS:ffff91fa6af00000(0000)
>> knlGS:0000000000000000
>> [ 127.068837] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>> [ 127.068839] CR2: 00007fd6c2e7fd04 CR3: 0000000104ee0000 CR4:
>> 00000000003506e0
>> [ 127.068841] Call Trace:
>> [ 127.068842] <TASK>
>> [ 127.068844] unregister_irq_proc+0xe4/0x110
>> [ 127.068849] free_desc+0x2e/0x70
>> [ 127.068852] irq_free_descs+0x50/0x80
>> [ 127.068856] irq_domain_free_irqs+0x16b/0x1c0
>> [ 127.068860] __msi_domain_free_irqs+0xf1/0x160
>> [ 127.068865] msi_domain_free_irqs_descs_locked+0x20/0x50
>> [ 127.068870] pci_msi_teardown_msi_irqs+0x49/0x50
>> [ 127.068873] pci_disable_msix.part.0+0xff/0x160
>> [ 127.068877] pci_free_irq_vectors+0x45/0x60
>> [ 127.068881] xgbe_pci_remove+0x24/0x40 [amd_xgbe]
>> [ 127.068896] pci_device_remove+0x39/0xa0
>> [ 127.068900] __device_release_driver+0x181/0x250
>> [ 127.068904] driver_detach+0xd3/0x120
>> [ 127.068908] bus_remove_driver+0x59/0xd0
>> [ 127.068911] driver_unregister+0x31/0x50
>> [ 127.068914] pci_unregister_driver+0x40/0x90
>> [ 127.068919] xgbe_pci_exit+0x15/0x20 [amd_xgbe]
>> [ 127.068933] xgbe_mod_exit+0x9/0x880 [amd_xgbe]
>> [ 127.068948] __do_sys_delete_module.constprop.0+0x183/0x290
>> [ 127.068952] ? exit_to_user_mode_prepare+0x49/0x1e0
>> [ 127.068957] __x64_sys_delete_module+0x12/0x20
>> [ 127.068961] do_syscall_64+0x5c/0xc0
>> [ 127.068964] ? irqentry_exit_to_user_mode+0x9/0x20
>> [ 127.068969] ? irqentry_exit+0x33/0x40
>> [ 127.068973] ? exc_page_fault+0x89/0x180
>> [ 127.068977] ? asm_exc_page_fault+0x8/0x30
>> [ 127.068980] entry_SYSCALL_64_after_hwframe+0x44/0xae
>> [ 127.068983] RIP: 0033:0x7f3a690cb8eb
>> [ 127.068985] Code: 73 01 c3 48 8b 0d 45 e5 0e 00 f7 d8 64 89 01 48 83
>> c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f
>> 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 15 e5 0e 00 f7 d8 64 89 01 48
>> [ 127.068987] RSP: 002b:00007ffed553a818 EFLAGS: 00000206 ORIG_RAX:
>> 00000000000000b0
>> [ 127.068990] RAX: ffffffffffffffda RBX: 00007f3a6a02e7b0 RCX:
>> 00007f3a690cb8eb
>> [ 127.068991] RDX: 000000000000000a RSI: 0000000000000800 RDI:
>> 00007f3a6a02e818
>> [ 127.068993] RBP: 0000000000000000 R08: 0000000000000000 R09:
>> 0000000000000000
>> [ 127.068994] R10: 00007f3a69163ac0 R11: 0000000000000206 R12:
>> 00007ffed553aa70
>> [ 127.068996] R13: 00007ffed553b84a R14: 00007f3a6a02e2a0 R15:
>> 00007f3a6a02e7b0
>> [ 127.068999] </TASK>
>> [ 127.069000] ---[ end trace 0000000000000000 ]---
>> [ 127.667264] irq 31: nobody cared (try booting with the "irqpoll" option)
>> [ 127.674758] CPU: 4 PID: 0 Comm: swapper/4 Tainted: G W
>> 5.17.0-rc3-tk #12
>> [ 127.674764] Hardware name: Deciso B.V. DEC2700 - OPNsense
>> Appliance/Netboard-A10 Gen.3, BIOS 05.32.50.0012-A10.20 11/15/2021
>> [ 127.674766] Call Trace:
>> [ 127.674769] <IRQ>
>> [ 127.674773] dump_stack_lvl+0x4c/0x63
>> [ 127.674781] dump_stack+0x10/0x12
>> [ 127.674784] __report_bad_irq+0x3a/0xaf
>> [ 127.674789] note_interrupt.cold+0xb/0x60
>> [ 127.674793] ? __this_cpu_preempt_check+0x13/0x20
>> [ 127.674799] handle_irq_event+0x71/0x80
>> [ 127.674805] handle_fasteoi_irq+0x95/0x1e0
>> [ 127.674810] __common_interrupt+0x6e/0x110
>> [ 127.674815] common_interrupt+0xbd/0xe0
>> [ 127.674819] </IRQ>
>> [ 127.674820] <TASK>
>> [ 127.674822] asm_common_interrupt+0x1e/0x40
>> [ 127.674826] RIP: 0010:cpuidle_enter_state+0xdf/0x380
>> [ 127.674834] Code: ff e8 25 76 73 ff 80 7d d7 00 74 17 9c 58 0f 1f 44
>> 00 00 f6 c4 02 0f 85 82 02 00 00 31 ff e8 18 8c 7a ff fb 66 0f 1f 44 00
>> 00 <45> 85 ff 0f 88 1a 01 00 00 49 63 d7 4c 89 f1 48 2b 4d c8 48 8d 04
>> [ 127.674837] RSP: 0018:ffffaf29400e3e68 EFLAGS: 00000246
>> [ 127.674841] RAX: ffff91fa6af00000 RBX: 0000000000000002 RCX:
>> 000000000000001f
>> [ 127.674843] RDX: 0000000000000000 RSI: ffffffff845bf281 RDI:
>> ffffffff845cddcf
>> [ 127.674845] RBP: ffffaf29400e3ea0 R08: 0000001db98fd21c R09:
>> 0000001d7b8fd3fc
>> [ 127.674847] R10: 0000000000000001 R11: ffff91fa6af2fd84 R12:
>> ffff91fa41de6c00
>> [ 127.674849] R13: ffffffff8506e4c0 R14: 0000001db98fd21c R15:
>> 0000000000000002
>> [ 127.674854] ? cpuidle_enter_state+0xbb/0x380
>> [ 127.674860] cpuidle_enter+0x2e/0x40
>> [ 127.674864] do_idle+0x203/0x290
>> [ 127.674869] cpu_startup_entry+0x20/0x30
>> [ 127.674872] start_secondary+0x118/0x150
>> [ 127.674877] secondary_startup_64_no_verify+0xd5/0xdb
>> [ 127.674885] </TASK>
>> [ 127.674886] handlers:
>> [ 127.677425] [<00000000b61e344c>] amd_mp2_irq_isr [i2c_amd_mp2_pci]
>> [ 127.684335] Disabling IRQ #31
>>
>> and command line output after 'modprobe -vvv amd_xgbe':
>>
>> jane@dec740-ubu21:~$ sudo modprobe -vvv amd_xgbe
>> modprobe: INFO: ../libkmod/libkmod.c:365 kmod_set_log_fn() custom
>> logging function 0x7f74d79de780 registered
>> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
>> file=/lib/modules/5.17.0-rc3-tk/modules.dep.bin
>> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
>> file=/lib/modules/5.17.0-rc3-tk/modules.alias.bin
>> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
>> file=/lib/modules/5.17.0-rc3-tk/modules.symbols.bin
>> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
>> file=/lib/modules/5.17.0-rc3-tk/modules.builtin.alias.bin
>> modprobe: DEBUG: ../libkmod/libkmod-index.c:757 index_mm_open()
>> file=/lib/modules/5.17.0-rc3-tk/modules.builtin.bin
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:556
>> kmod_module_new_from_lookup() input alias=amd_xgbe, normalized=amd_xgbe
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:562
>> kmod_module_new_from_lookup() lookup modules.dep amd_xgbe
>> modprobe: DEBUG: ../libkmod/libkmod.c:595 kmod_search_moddep() use
>> mmaped index 'modules.dep' modname=amd_xgbe
>> modprobe: DEBUG: ../libkmod/libkmod.c:403 kmod_pool_get_module() get
>> module name='amd_xgbe' found=(nil)
>> modprobe: DEBUG: ../libkmod/libkmod.c:411 kmod_pool_add_module() add
>> 0x7f74d83862a0 key='amd_xgbe'
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:202
>> kmod_module_parse_depline() 0 dependencies for amd_xgbe
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:589
>> kmod_module_new_from_lookup() lookup amd_xgbe=0, list=0x7f74d8385c40
>> modprobe: DEBUG: ../libkmod/libkmod.c:500 lookup_builtin_file() use
>> mmaped index 'modules.builtin' modname=amd_xgbe
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1760
>> kmod_module_get_initstate() could not open
>> '/sys/module/amd_xgbe/initstate': No such file or directory
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1770
>> kmod_module_get_initstate() could not open '/sys/module/amd_xgbe': No
>> such file or directory
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=snd_pcsp mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=snd_usb_audio mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=cx88_alsa mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=snd_atiixp_modem mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=snd_intel8x0m mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=snd_via82xx_modem mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
>> kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=md_mod mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=bonding mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=dummy mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
>> kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1404
>> kmod_module_get_options() modname=amd_xgbe mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1409
>> kmod_module_get_options() passed = modname=amd_xgbe mod->name=amd_xgbe
>> mod->alias=(null)
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1760
>> kmod_module_get_initstate() could not open
>> '/sys/module/amd_xgbe/initstate': No such file or directory
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:1770
>> kmod_module_get_initstate() could not open '/sys/module/amd_xgbe': No
>> such file or directory
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
>> name='amd_xgbe'
>> path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
>>
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
>> name='amd_xgbe'
>> path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
>>
>> insmod
>> /lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko
>> dyndbg="+pfm" debug=0x37 dyndbg=+p debug=0x37
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:750 kmod_module_get_path()
>> name='amd_xgbe'
>> path='/lib/modules/5.17.0-rc3-tk/kernel/drivers/net/ethernet/amd/xgbe/amd-xgbe.ko'
>>
>> modprobe: DEBUG: ../libkmod/libkmod-module.c:468 kmod_module_unref()
>> kmod_module 0x7f74d83862a0 released
>> modprobe: DEBUG: ../libkmod/libkmod.c:419 kmod_pool_del_module() del
>> 0x7f74d83862a0 key='amd_xgbe'
>> modprobe: INFO: ../libkmod/libkmod.c:332 kmod_unref() context
>> 0x7f74d83854c0 released
>>
>> and the corresponding dmesg diff:
>>
>> [ 151.599892] amd-xgbe 0000:06:00.1 eth0: net device enabled
>> [ 151.601333] amd-xgbe 0000:06:00.2 eth1: net device enabled
>> [ 151.606044] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>> [ 151.646262] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>
>>
>> Cheers
>> Thomas
>>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-11 9:49 ` Shyam Sundar S K
2022-02-11 12:01 ` Thomas Kupper
@ 2022-02-11 15:48 ` Tom Lendacky
2022-02-11 17:41 ` Thomas Kupper
2022-02-14 4:23 ` Shyam Sundar S K
1 sibling, 2 replies; 15+ messages in thread
From: Tom Lendacky @ 2022-02-11 15:48 UTC (permalink / raw)
To: Shyam Sundar S K, Thomas Kupper; +Cc: netdev
On 2/11/22 03:49, Shyam Sundar S K wrote:
> On 2/11/2022 3:03 PM, Thomas Kupper wrote:
>> Am 08.02.22 um 17:24 schrieb Tom Lendacky:
>>> On 2/7/22 11:59, Thomas Kupper wrote:
>>>> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>>>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>>>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>
>> Thanks Tom, I now got time to update to 5.17-rc3 and add the 'debug'
>> module parameter. I assume that parameter works with the non-debug
>> kernel? I don't really see any new messages related to the amd-xgbe driver:
>>
>> dmesg right after boot:
>>
>> [ 0.000000] Linux version 5.17.0-rc3-tk (jane@m920q-ubu21) (gcc
>> (Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37)
>> #12 SMP PREEMPT Tue Feb 8 19:52:19 CET 2022
>> [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
>> root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
>> console=ttyS0,115200n8 amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
>> ...
>> [ 5.275730] amd-xgbe 0000:06:00.1 eth0: net device enabled
>> [ 5.277766] amd-xgbe 0000:06:00.2 eth1: net device enabled
>> [ 5.665315] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>> [ 5.696665] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
Hmmm... that's strange. There should have been some messages issued by the
xgbe-phy-v2.c file from the xgbe_phy_init() routine.
Thomas, if you're up for a bit of kernel hacking, can you remove the
"if (netif_msg_probe(pdata)) {" that wrap the dev_dbg() calls in the
xgbe-phy-v2.c file? There are 5 locations.
>>
>> dmesg right after 'ifconfig enp6s0f2 up'
>>
>> [ 88.843454] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>> enp6s0f2: channel-0: cpu=0, node=0
> Can you add this change and see if it solves the problem?
>
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=68c2d6af1f1e
I would imagine that patch has nothing to do with the real issue. Given
the previous messages of:
> [ 648.038655] genirq: Flags mismatch irq 59. 00000000 (enp6s0f2-pcs) vs. 00000000 (enp6s0f2-pcs)
> [ 648.048303] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
There should be no reason for not being able to obtain the IRQ.
I suspect it is something in the BIOS setup that is not correct and thus
the Linux driver is not working properly because of bad input/setup from
the BIOS. This was probably worked around by the driver used in the
OPNsense DEC740 firewall.
Shyam has worked more closely with the embedded area of this device, I'll
let him take it from here.
Thanks,
Tom
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-11 15:48 ` Tom Lendacky
@ 2022-02-11 17:41 ` Thomas Kupper
2022-02-14 4:23 ` Shyam Sundar S K
1 sibling, 0 replies; 15+ messages in thread
From: Thomas Kupper @ 2022-02-11 17:41 UTC (permalink / raw)
To: Tom Lendacky, Shyam Sundar S K; +Cc: netdev
Am 11.02.22 um 16:48 schrieb Tom Lendacky:
> On 2/11/22 03:49, Shyam Sundar S K wrote:
>> On 2/11/2022 3:03 PM, Thomas Kupper wrote:
>>> Am 08.02.22 um 17:24 schrieb Tom Lendacky:
>>>> On 2/7/22 11:59, Thomas Kupper wrote:
>>>>> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>>>>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>>>>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>>>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>
>>>
>>> Thanks Tom, I now got time to update to 5.17-rc3 and add the 'debug'
>>> module parameter. I assume that parameter works with the non-debug
>>> kernel? I don't really see any new messages related to the amd-xgbe
>>> driver:
>>>
>>> dmesg right after boot:
>>>
>>> [ 0.000000] Linux version 5.17.0-rc3-tk (jane@m920q-ubu21) (gcc
>>> (Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37)
>>> #12 SMP PREEMPT Tue Feb 8 19:52:19 CET 2022
>>> [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
>>> root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
>>> console=ttyS0,115200n8 amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
>>> ...
>>> [ 5.275730] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>> [ 5.277766] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>> [ 5.665315] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>> [ 5.696665] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>
> Hmmm... that's strange. There should have been some messages issued by
> the
> xgbe-phy-v2.c file from the xgbe_phy_init() routine.
>
> Thomas, if you're up for a bit of kernel hacking, can you remove the
> "if (netif_msg_probe(pdata)) {" that wrap the dev_dbg() calls in the
> xgbe-phy-v2.c file? There are 5 locations.
>
Thanks Tom,
I did try dyndbg with another the Intel igb module to make sure there's
nothing wrong with my setup ... and that worked fine.
I then replace all 'if (netif_msg_probe(pdata))' with 'if (1)' in the
AMD XGBE code. Now we got some debug messages:
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
console=ttyS0,115200n8 debug amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
[ 0.021516] Kernel command line:
BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
console=ttyS0,115200n8 debug amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
...
[ 5.425856] amd-xgbe 0000:06:00.1: xgmac_regs = 0000000090a80b23
[ 5.432628] amd-xgbe 0000:06:00.1: xprop_regs = 00000000d329c23a
[ 5.448610] amd-xgbe 0000:06:00.1: xi2c_regs = 000000003612f52a
[ 5.463393] amd-xgbe 0000:06:00.1: xpcs_regs = 0000000091508acb
[ 5.475558] amd-xgbe 0000:06:00.1: xpcs window def = 0x00001060
[ 5.488971] amd-xgbe 0000:06:00.1: xpcs window sel = 0x00001064
[ 5.500838] amd-xgbe 0000:06:00.1: xpcs window = 0x0000b000
[ 5.511736] amd-xgbe 0000:06:00.1: xpcs window size = 0x00001000
[ 5.522924] amd-xgbe 0000:06:00.1: xpcs window mask = 0x00000fff
[ 5.522948] amd-xgbe 0000:06:00.1: port property 0 = 0x15800800
[ 5.535474] amd-xgbe 0000:06:00.1: port property 1 = 0x03030303
[ 5.550344] amd-xgbe 0000:06:00.1: port property 2 = 0x00040004
[ 5.550346] amd-xgbe 0000:06:00.1: port property 3 = 0x2dc0e100
[ 5.564845] amd-xgbe 0000:06:00.1: port property 4 = 0x00001c03
[ 5.578070] amd-xgbe 0000:06:00.1: max tx/rx channel count = 3/3
[ 5.578072] amd-xgbe 0000:06:00.1: max tx/rx hw queue count = 3/3
[ 5.593733] amd-xgbe 0000:06:00.1: Hardware features:
[ 5.609665] amd-xgbe 0000:06:00.1: 1GbE support : yes
[ 5.624537] amd-xgbe 0000:06:00.1: VLAN hash filter : yes
[ 5.635628] amd-xgbe 0000:06:00.1: MDIO interface : yes
[ 5.635630] amd-xgbe 0000:06:00.1: Wake-up packet support : no
[ 5.649437] amd-xgbe 0000:06:00.1: Magic packet support : no
[ 5.649439] amd-xgbe 0000:06:00.1: Management counters : yes
[ 5.649440] amd-xgbe 0000:06:00.1: ARP offload : yes
[ 5.661697] amd-xgbe 0000:06:00.1: IEEE 1588-2008 Timestamp : yes
[ 5.661698] amd-xgbe 0000:06:00.1: Energy Efficient Ethernet : yes
[ 5.719737] amd-xgbe 0000:06:00.1: TX checksum offload : yes
[ 5.726837] amd-xgbe 0000:06:00.1: RX checksum offload : yes
[ 5.733934] amd-xgbe 0000:06:00.1: Additional MAC addresses : 31
[ 5.740937] amd-xgbe 0000:06:00.1: Timestamp source :
internal/external
[ 5.749393] amd-xgbe 0000:06:00.1: SA/VLAN insertion : yes
[ 5.756493] amd-xgbe 0000:06:00.1: VXLAN/NVGRE support : yes
[ 5.763592] amd-xgbe 0000:06:00.1: RX fifo size : 65536
[ 5.770882] amd-xgbe 0000:06:00.1: TX fifo size : 65536
[ 5.778177] amd-xgbe 0000:06:00.1: IEEE 1588 high word : yes
[ 5.785274] amd-xgbe 0000:06:00.1: DMA width : 48
[ 5.792276] amd-xgbe 0000:06:00.1: Data Center Bridging : yes
[ 5.799375] amd-xgbe 0000:06:00.1: Split header : yes
[ 5.806475] amd-xgbe 0000:06:00.1: TCP Segmentation Offload : yes
[ 5.813572] amd-xgbe 0000:06:00.1: Debug memory interface : yes
[ 5.820671] amd-xgbe 0000:06:00.1: Receive Side Scaling : yes
[ 5.827771] amd-xgbe 0000:06:00.1: Traffic Class count : 3
[ 5.834676] amd-xgbe 0000:06:00.1: Hash table size : 256
[ 5.841773] amd-xgbe 0000:06:00.1: L3/L4 Filters : 8
[ 5.848679] amd-xgbe 0000:06:00.1: RX queue count : 3
[ 5.855583] amd-xgbe 0000:06:00.1: TX queue count : 3
[ 5.862484] amd-xgbe 0000:06:00.1: RX DMA channel count : 3
[ 5.869387] amd-xgbe 0000:06:00.1: TX DMA channel count : 3
[ 5.876292] amd-xgbe 0000:06:00.1: PPS outputs : 0
[ 5.883198] amd-xgbe 0000:06:00.1: Auxiliary snapshot inputs : 0
[ 5.890103] amd-xgbe 0000:06:00.1: TX/RX DMA channel count = 3/3
[ 5.896815] amd-xgbe 0000:06:00.1: TX/RX hardware queue count = 3/3
[ 5.903814] amd-xgbe 0000:06:00.1: max tx/rx max fifo size = 65536/65536
[ 5.911499] amd-xgbe 0000:06:00.1: multi MSI-X interrupts enabled
[ 5.918323] amd-xgbe 0000:06:00.1: dev irq=59
[ 5.923349] amd-xgbe 0000:06:00.1: ecc irq=60
[ 5.928316] amd-xgbe 0000:06:00.1: i2c irq=61
[ 5.933290] amd-xgbe 0000:06:00.1: an irq=62
[ 5.938268] amd-xgbe 0000:06:00.1: dma0 irq=63
[ 5.943335] amd-xgbe 0000:06:00.1: dma1 irq=64
[ 5.948404] amd-xgbe 0000:06:00.1: dma2 irq=65
[ 5.953529] amd-xgbe 0000:06:00.1: adjusted TX/RX DMA channel count = 3/3
[ 5.961149] amd-xgbe 0000:06:00.1: I2C features: MAX_SPEED_MODE=2,
RX_BUFFER_DEPTH=15, TX_BUFFER_DEPTH=15
[ 5.971905] amd-xgbe 0000:06:00.1: port mode=8
[ 5.976880] amd-xgbe 0000:06:00.1: port id=0
[ 5.987690] amd-xgbe 0000:06:00.1: port speeds=0x16
[ 5.987695] amd-xgbe 0000:06:00.1: conn type=1
[ 6.004349] amd-xgbe 0000:06:00.1: mdio addr=0
[ 6.009320] amd-xgbe 0000:06:00.1: redrv present
[ 6.014487] amd-xgbe 0000:06:00.1: redrv i/f=0
[ 6.019459] amd-xgbe 0000:06:00.1: redrv addr=0x0
[ 6.024720] amd-xgbe 0000:06:00.1: redrv lane=0
[ 6.029791] amd-xgbe 0000:06:00.1: redrv model=0
[ 6.039430] amd-xgbe 0000:06:00.1: SFP: mux_address=0x73
[ 6.050427] amd-xgbe 0000:06:00.1: SFP: mux_channel=0
[ 6.063769] amd-xgbe 0000:06:00.1: SFP: gpio_address=0x21
[ 6.077779] amd-xgbe 0000:06:00.1: SFP: gpio_mask=0x2
[ 6.089176] amd-xgbe 0000:06:00.1: SFP: gpio_rx_los=13
[ 6.103384] amd-xgbe 0000:06:00.1: SFP: gpio_tx_fault=14
[ 6.109326] amd-xgbe 0000:06:00.1: SFP: gpio_mod_absent=12
[ 6.115461] amd-xgbe 0000:06:00.1: SFP: gpio_rate_select=0
[ 6.121600] amd-xgbe 0000:06:00.1: phy
supported=0x0000000,00000000,000064c0
[ 6.130337] amd-xgbe 0000:06:00.1 eth1: net device enabled
[ 6.136645] amd-xgbe 0000:06:00.2: xgmac_regs = 00000000013cde0a
[ 6.147680] amd-xgbe 0000:06:00.2: xprop_regs = 00000000c2a793ff
[ 6.159477] amd-xgbe 0000:06:00.2: xi2c_regs = 00000000cfbf31f0
[ 6.173891] amd-xgbe 0000:06:00.2: xpcs_regs = 00000000a1eda099
[ 6.188606] amd-xgbe 0000:06:00.2: xpcs window def = 0x00001060
[ 6.209526] amd-xgbe 0000:06:00.2: xpcs window sel = 0x00001064
[ 6.216248] amd-xgbe 0000:06:00.2: xpcs window = 0x0000b000
[ 6.222967] amd-xgbe 0000:06:00.2: xpcs window size = 0x00001000
[ 6.229687] amd-xgbe 0000:06:00.2: xpcs window mask = 0x00000fff
[ 6.240916] amd-xgbe 0000:06:00.2: port property 0 = 0x15800801
[ 6.252578] amd-xgbe 0000:06:00.2: port property 1 = 0x03030303
[ 6.266891] amd-xgbe 0000:06:00.2: port property 2 = 0x00040004
[ 6.281498] amd-xgbe 0000:06:00.2: port property 3 = 0x2980a100
[ 6.293861] amd-xgbe 0000:06:00.2: port property 4 = 0x00001c13
[ 6.308933] amd-xgbe 0000:06:00.2: max tx/rx channel count = 3/3
[ 6.315651] amd-xgbe 0000:06:00.2: max tx/rx hw queue count = 3/3
[ 6.322480] amd-xgbe 0000:06:00.2: Hardware features:
[ 6.333849] amd-xgbe 0000:06:00.2: 1GbE support : yes
[ 6.340948] amd-xgbe 0000:06:00.2: VLAN hash filter : yes
[ 6.348047] amd-xgbe 0000:06:00.2: MDIO interface : yes
[ 6.355146] amd-xgbe 0000:06:00.2: Wake-up packet support : no
[ 6.362146] amd-xgbe 0000:06:00.2: Magic packet support : no
[ 6.369148] amd-xgbe 0000:06:00.2: Management counters : yes
[ 6.376244] amd-xgbe 0000:06:00.2: ARP offload : yes
[ 6.383345] amd-xgbe 0000:06:00.2: IEEE 1588-2008 Timestamp : yes
[ 6.390443] amd-xgbe 0000:06:00.2: Energy Efficient Ethernet : yes
[ 6.397539] amd-xgbe 0000:06:00.2: TX checksum offload : yes
[ 6.404636] amd-xgbe 0000:06:00.2: RX checksum offload : yes
[ 6.411734] amd-xgbe 0000:06:00.2: Additional MAC addresses : 31
[ 6.418737] amd-xgbe 0000:06:00.2: Timestamp source :
internal/external
[ 6.427195] amd-xgbe 0000:06:00.2: SA/VLAN insertion : yes
[ 6.434292] amd-xgbe 0000:06:00.2: VXLAN/NVGRE support : yes
[ 6.441391] amd-xgbe 0000:06:00.2: RX fifo size : 65536
[ 6.448684] amd-xgbe 0000:06:00.2: TX fifo size : 65536
[ 6.455976] amd-xgbe 0000:06:00.2: IEEE 1588 high word : yes
[ 6.463075] amd-xgbe 0000:06:00.2: DMA width : 48
[ 6.470078] amd-xgbe 0000:06:00.2: Data Center Bridging : yes
[ 6.477178] amd-xgbe 0000:06:00.2: Split header : yes
[ 6.484276] amd-xgbe 0000:06:00.2: TCP Segmentation Offload : yes
[ 6.491375] amd-xgbe 0000:06:00.2: Debug memory interface : yes
[ 6.498471] amd-xgbe 0000:06:00.2: Receive Side Scaling : yes
[ 6.505570] amd-xgbe 0000:06:00.2: Traffic Class count : 3
[ 6.512472] amd-xgbe 0000:06:00.2: Hash table size : 256
[ 6.519571] amd-xgbe 0000:06:00.2: L3/L4 Filters : 8
[ 6.526476] amd-xgbe 0000:06:00.2: RX queue count : 3
[ 6.533379] amd-xgbe 0000:06:00.2: TX queue count : 3
[ 6.540282] amd-xgbe 0000:06:00.2: RX DMA channel count : 3
[ 6.547183] amd-xgbe 0000:06:00.2: TX DMA channel count : 3
[ 6.554087] amd-xgbe 0000:06:00.2: PPS outputs : 0
[ 6.560992] amd-xgbe 0000:06:00.2: Auxiliary snapshot inputs : 0
[ 6.567894] amd-xgbe 0000:06:00.2: TX/RX DMA channel count = 3/3
[ 6.574606] amd-xgbe 0000:06:00.2: TX/RX hardware queue count = 3/3
[ 6.581609] amd-xgbe 0000:06:00.2: max tx/rx max fifo size = 65536/65536
[ 6.589256] amd-xgbe 0000:06:00.2: multi MSI-X interrupts enabled
[ 6.596079] amd-xgbe 0000:06:00.2: dev irq=84
[ 6.601045] amd-xgbe 0000:06:00.2: ecc irq=85
[ 6.606008] amd-xgbe 0000:06:00.2: i2c irq=86
[ 6.610974] amd-xgbe 0000:06:00.2: an irq=87
[ 6.615938] amd-xgbe 0000:06:00.2: dma0 irq=88
[ 6.621002] amd-xgbe 0000:06:00.2: dma1 irq=89
[ 6.626072] amd-xgbe 0000:06:00.2: dma2 irq=90
[ 6.631176] amd-xgbe 0000:06:00.2: adjusted TX/RX DMA channel count = 3/3
[ 6.638772] amd-xgbe 0000:06:00.2: I2C features: MAX_SPEED_MODE=2,
RX_BUFFER_DEPTH=15, TX_BUFFER_DEPTH=15
[ 6.649472] amd-xgbe 0000:06:00.2: port mode=8
[ 6.654443] amd-xgbe 0000:06:00.2: port id=1
[ 6.659216] amd-xgbe 0000:06:00.2: port speeds=0x16
[ 6.664668] amd-xgbe 0000:06:00.2: conn type=1
[ 6.669634] amd-xgbe 0000:06:00.2: mdio addr=0
[ 6.674598] amd-xgbe 0000:06:00.2: redrv present
[ 6.679758] amd-xgbe 0000:06:00.2: redrv i/f=0
[ 6.684726] amd-xgbe 0000:06:00.2: redrv addr=0x0
[ 6.689974] amd-xgbe 0000:06:00.2: redrv lane=0
[ 6.695038] amd-xgbe 0000:06:00.2: redrv model=0
[ 6.700196] amd-xgbe 0000:06:00.2: SFP: mux_address=0x73
[ 6.706131] amd-xgbe 0000:06:00.2: SFP: mux_channel=1
[ 6.711776] amd-xgbe 0000:06:00.2: SFP: gpio_address=0x21
[ 6.717808] amd-xgbe 0000:06:00.2: SFP: gpio_mask=0x2
[ 6.723452] amd-xgbe 0000:06:00.2: SFP: gpio_rx_los=9
[ 6.729097] amd-xgbe 0000:06:00.2: SFP: gpio_tx_fault=10
[ 6.735031] amd-xgbe 0000:06:00.2: SFP: gpio_mod_absent=8
[ 6.741063] amd-xgbe 0000:06:00.2: SFP: gpio_rate_select=0
[ 6.747201] amd-xgbe 0000:06:00.2: phy
supported=0x0000000,00000000,000064c0
[ 6.765905] amd-xgbe 0000:06:00.2 eth2: net device enabled
[ 6.830271] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth1
[ 6.873079] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth2
I'll try and debug why 'netif_msg_probe(pdata)' doesn't work, next week.
>>>
>>> dmesg right after 'ifconfig enp6s0f2 up'
>>>
>>> [ 88.843454] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>>> enp6s0f2: channel-0: cpu=0, node=0
>
>
>> Can you add this change and see if it solves the problem?
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=68c2d6af1f1e
>>
>
> I would imagine that patch has nothing to do with the real issue. Given
> the previous messages of:
>
>> [ 648.038655] genirq: Flags mismatch irq 59. 00000000 (enp6s0f2-pcs)
>> vs. 00000000 (enp6s0f2-pcs)
>> [ 648.048303] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>
> There should be no reason for not being able to obtain the IRQ.
>
> I suspect it is something in the BIOS setup that is not correct and thus
> the Linux driver is not working properly because of bad input/setup from
> the BIOS. This was probably worked around by the driver used in the
> OPNsense DEC740 firewall.
>
> Shyam has worked more closely with the embedded area of this device, I'll
> let him take it from here.
>
> Thanks,
> Tom
>
Thanks for your help Tom! I'll wait to hear back from Shyam.
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-11 15:48 ` Tom Lendacky
2022-02-11 17:41 ` Thomas Kupper
@ 2022-02-14 4:23 ` Shyam Sundar S K
2022-04-08 7:02 ` Thomas Kupper
1 sibling, 1 reply; 15+ messages in thread
From: Shyam Sundar S K @ 2022-02-14 4:23 UTC (permalink / raw)
To: Tom Lendacky, Thomas Kupper; +Cc: netdev
On 2/11/2022 9:18 PM, Tom Lendacky wrote:
> On 2/11/22 03:49, Shyam Sundar S K wrote:
>> On 2/11/2022 3:03 PM, Thomas Kupper wrote:
>>> Am 08.02.22 um 17:24 schrieb Tom Lendacky:
>>>> On 2/7/22 11:59, Thomas Kupper wrote:
>>>>> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>>>>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>>>>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>>>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>
>>>
>>> Thanks Tom, I now got time to update to 5.17-rc3 and add the 'debug'
>>> module parameter. I assume that parameter works with the non-debug
>>> kernel? I don't really see any new messages related to the amd-xgbe
>>> driver:
>>>
>>> dmesg right after boot:
>>>
>>> [ 0.000000] Linux version 5.17.0-rc3-tk (jane@m920q-ubu21) (gcc
>>> (Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37)
>>> #12 SMP PREEMPT Tue Feb 8 19:52:19 CET 2022
>>> [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
>>> root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
>>> console=ttyS0,115200n8 amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
>>> ...
>>> [ 5.275730] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>> [ 5.277766] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>> [ 5.665315] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>> [ 5.696665] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>
> Hmmm... that's strange. There should have been some messages issued by the
> xgbe-phy-v2.c file from the xgbe_phy_init() routine.
>
> Thomas, if you're up for a bit of kernel hacking, can you remove the
> "if (netif_msg_probe(pdata)) {" that wrap the dev_dbg() calls in the
> xgbe-phy-v2.c file? There are 5 locations.
>
>>>
>>> dmesg right after 'ifconfig enp6s0f2 up'
>>>
>>> [ 88.843454] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>>> enp6s0f2: channel-0: cpu=0, node=0
>
>
>> Can you add this change and see if it solves the problem?
>>
>> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=68c2d6af1f1e
>>
>
> I would imagine that patch has nothing to do with the real issue. Given
> the previous messages of:
Agreed. I guessed the earlier problem manifested after the driver
removal. However, this one still appears like a BIOS misconfiguration.
>
>> [ 648.038655] genirq: Flags mismatch irq 59. 00000000 (enp6s0f2-pcs)
>> vs. 00000000 (enp6s0f2-pcs)
>> [ 648.048303] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>
> There should be no reason for not being able to obtain the IRQ.
>
> I suspect it is something in the BIOS setup that is not correct and thus
> the Linux driver is not working properly because of bad input/setup from
> the BIOS. This was probably worked around by the driver used in the
> OPNsense DEC740 firewall.
>
> Shyam has worked more closely with the embedded area of this device, I'll
> let him take it from here.
I shall connect Thomas to BIOS folks and take it forward from there.
Thanks,
Shyam
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board
2022-02-14 4:23 ` Shyam Sundar S K
@ 2022-04-08 7:02 ` Thomas Kupper
0 siblings, 0 replies; 15+ messages in thread
From: Thomas Kupper @ 2022-04-08 7:02 UTC (permalink / raw)
To: Shyam Sundar S K; +Cc: Tom Lendacky, netdev
Shyam Sundar S K wrote on 14.02.22 05:23:
>
> On 2/11/2022 9:18 PM, Tom Lendacky wrote:
>> On 2/11/22 03:49, Shyam Sundar S K wrote:
>>> On 2/11/2022 3:03 PM, Thomas Kupper wrote:
>>>> Am 08.02.22 um 17:24 schrieb Tom Lendacky:
>>>>> On 2/7/22 11:59, Thomas Kupper wrote:
>>>>>> Am 07.02.22 um 16:19 schrieb Shyam Sundar S K:
>>>>>>> On 2/7/2022 8:02 PM, Tom Lendacky wrote:
>>>>>>>> On 2/5/22 12:14, Thomas Kupper wrote:
>>>>>>>>> Am 05.02.22 um 16:51 schrieb Tom Lendacky:
>>>>>>>>>> On 2/5/22 04:06, Thomas Kupper wrote:
>>>> Thanks Tom, I now got time to update to 5.17-rc3 and add the 'debug'
>>>> module parameter. I assume that parameter works with the non-debug
>>>> kernel? I don't really see any new messages related to the amd-xgbe
>>>> driver:
>>>>
>>>> dmesg right after boot:
>>>>
>>>> [ 0.000000] Linux version 5.17.0-rc3-tk (jane@m920q-ubu21) (gcc
>>>> (Ubuntu 11.2.0-7ubuntu2) 11.2.0, GNU ld (GNU Binutils for Ubuntu) 2.37)
>>>> #12 SMP PREEMPT Tue Feb 8 19:52:19 CET 2022
>>>> [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-5.17.0-rc3-tk
>>>> root=UUID=8e462830-8ba0-4061-8f23-6f29ce751792 ro console=tty0
>>>> console=ttyS0,115200n8 amd_xgbe.dyndbg=+p amd_xgbe.debug=0x37
>>>> ...
>>>> [ 5.275730] amd-xgbe 0000:06:00.1 eth0: net device enabled
>>>> [ 5.277766] amd-xgbe 0000:06:00.2 eth1: net device enabled
>>>> [ 5.665315] amd-xgbe 0000:06:00.2 enp6s0f2: renamed from eth1
>>>> [ 5.696665] amd-xgbe 0000:06:00.1 enp6s0f1: renamed from eth0
>> Hmmm... that's strange. There should have been some messages issued by the
>> xgbe-phy-v2.c file from the xgbe_phy_init() routine.
>>
>> Thomas, if you're up for a bit of kernel hacking, can you remove the
>> "if (netif_msg_probe(pdata)) {" that wrap the dev_dbg() calls in the
>> xgbe-phy-v2.c file? There are 5 locations.
>>
>>>> dmesg right after 'ifconfig enp6s0f2 up'
>>>>
>>>> [ 88.843454] amd_xgbe:xgbe_alloc_channels: amd-xgbe 0000:06:00.2
>>>> enp6s0f2: channel-0: cpu=0, node=0
>>
>>> Can you add this change and see if it solves the problem?
>>>
>>> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net.git/commit/?id=68c2d6af1f1e
>>>
>> I would imagine that patch has nothing to do with the real issue. Given
>> the previous messages of:
> Agreed. I guessed the earlier problem manifested after the driver
> removal. However, this one still appears like a BIOS misconfiguration.
>
>>> [ 648.038655] genirq: Flags mismatch irq 59. 00000000 (enp6s0f2-pcs)
>>> vs. 00000000 (enp6s0f2-pcs)
>>> [ 648.048303] amd-xgbe 0000:06:00.2 enp6s0f2: phy irq request failed
>> There should be no reason for not being able to obtain the IRQ.
>>
>> I suspect it is something in the BIOS setup that is not correct and thus
>> the Linux driver is not working properly because of bad input/setup from
>> the BIOS. This was probably worked around by the driver used in the
>> OPNsense DEC740 firewall.
>>
>> Shyam has worked more closely with the embedded area of this device, I'll
>> let him take it from here.
> I shall connect Thomas to BIOS folks and take it forward from there.
Hey Shyam and Tom,
After almost two months I unfortunately haven't heard of anyone from AMD
about a possible fix. And neither in the linux kernel nor the linux
netdev repo are any new commit related to the amd-xgbe driver.
Is there a way that I could get an answer? If you can't or won't do any
fix that's ok it just helps me if I know where I stand on this issue.
/Thomas
> Thanks,
> Shyam
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2022-04-08 7:12 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-05 10:06 AMD XGBE "phy irq request failed" kernel v5.17-rc2 on V1500B based board Thomas Kupper
2022-02-05 15:51 ` Tom Lendacky
2022-02-05 18:14 ` Thomas Kupper
2022-02-07 14:32 ` Tom Lendacky
2022-02-07 15:19 ` Shyam Sundar S K
2022-02-07 17:59 ` Thomas Kupper
2022-02-08 16:24 ` Tom Lendacky
2022-02-11 9:33 ` Thomas Kupper
2022-02-11 9:49 ` Shyam Sundar S K
2022-02-11 12:01 ` Thomas Kupper
2022-02-11 15:48 ` Tom Lendacky
2022-02-11 17:41 ` Thomas Kupper
2022-02-14 4:23 ` Shyam Sundar S K
2022-04-08 7:02 ` Thomas Kupper
2022-02-07 18:04 ` Thomas Kupper
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.