* Re: PROBLEM: XGBE unable to autoneg when advertising only 1GbE KX mode
@ 2022-05-24 12:34 Pighin, Anthony (Nokia - CA/Ottawa)
0 siblings, 0 replies; 6+ messages in thread
From: Pighin, Anthony (Nokia - CA/Ottawa) @ 2022-05-24 12:34 UTC (permalink / raw)
To: Shyam-sundar.S-k, netdev, thomas.lendacky
Cc: Pighin, Anthony (Nokia - CA/Ottawa)
[ 6.554938] xgbe_pci_probe:261: amd-xgbe 0000:0d:00.7: xgmac_regs = 0000000094e72ff0
[ 6.554941] xgbe_pci_probe:262: amd-xgbe 0000:0d:00.7: xprop_regs = 0000000052e3fd4b
[ 6.554942] xgbe_pci_probe:263: amd-xgbe 0000:0d:00.7: xi2c_regs = 000000001de70eb6
[ 6.554943] xgbe_pci_probe:273: amd-xgbe 0000:0d:00.7: xpcs_regs = 000000008a626609
[ 6.554947] xgbe_pci_probe:295: amd-xgbe 0000:0d:00.7: xpcs window def = 0x00009060
[ 6.554948] xgbe_pci_probe:297: amd-xgbe 0000:0d:00.7: xpcs window sel = 0x00009064
[ 6.554949] xgbe_pci_probe:299: amd-xgbe 0000:0d:00.7: xpcs window = 0x0000b000
[ 6.554951] xgbe_pci_probe:301: amd-xgbe 0000:0d:00.7: xpcs window size = 0x00001000
[ 6.554952] xgbe_pci_probe:303: amd-xgbe 0000:0d:00.7: xpcs window mask = 0x00000fff
[ 6.554958] xgbe_pci_probe:345: amd-xgbe 0000:0d:00.7: port property 0 = 0x45000103
[ 6.554959] xgbe_pci_probe:346: amd-xgbe 0000:0d:00.7: port property 1 = 0x02020202
[ 6.554960] xgbe_pci_probe:347: amd-xgbe 0000:0d:00.7: port property 2 = 0x00040004
[ 6.554962] xgbe_pci_probe:348: amd-xgbe 0000:0d:00.7: port property 3 = 0x0000a100
[ 6.554963] xgbe_pci_probe:349: amd-xgbe 0000:0d:00.7: port property 4 = 0x00000000
[ 6.554964] xgbe_pci_probe:362: amd-xgbe 0000:0d:00.7: max tx/rx channel count = 2/2
[ 6.554965] xgbe_pci_probe:365: amd-xgbe 0000:0d:00.7: max tx/rx hw queue count = 2/2
[ 6.554969] xgbe_get_all_hw_features:831: amd-xgbe 0000:0d:00.7: Hardware features:
[ 6.554971] xgbe_get_all_hw_features:834: amd-xgbe 0000:0d:00.7: 1GbE support : yes
[ 6.554972] xgbe_get_all_hw_features:836: amd-xgbe 0000:0d:00.7: VLAN hash filter : yes
[ 6.554973] xgbe_get_all_hw_features:838: amd-xgbe 0000:0d:00.7: MDIO interface : yes
[ 6.554974] xgbe_get_all_hw_features:840: amd-xgbe 0000:0d:00.7: Wake-up packet support : no
[ 6.554975] xgbe_get_all_hw_features:842: amd-xgbe 0000:0d:00.7: Magic packet support : no
[ 6.554977] xgbe_get_all_hw_features:844: amd-xgbe 0000:0d:00.7: Management counters : yes
[ 6.554978] xgbe_get_all_hw_features:846: amd-xgbe 0000:0d:00.7: ARP offload : yes
[ 6.554979] xgbe_get_all_hw_features:848: amd-xgbe 0000:0d:00.7: IEEE 1588-2008 Timestamp : yes
[ 6.554980] xgbe_get_all_hw_features:850: amd-xgbe 0000:0d:00.7: Energy Efficient Ethernet : yes
[ 6.554981] xgbe_get_all_hw_features:852: amd-xgbe 0000:0d:00.7: TX checksum offload : yes
[ 6.554983] xgbe_get_all_hw_features:854: amd-xgbe 0000:0d:00.7: RX checksum offload : yes
[ 6.554984] xgbe_get_all_hw_features:856: amd-xgbe 0000:0d:00.7: Additional MAC addresses : 31
[ 6.554985] xgbe_get_all_hw_features:858: amd-xgbe 0000:0d:00.7: Timestamp source : internal/external
[ 6.554986] xgbe_get_all_hw_features:862: amd-xgbe 0000:0d:00.7: SA/VLAN insertion : yes
[ 6.554988] xgbe_get_all_hw_features:864: amd-xgbe 0000:0d:00.7: VXLAN/NVGRE support : yes
[ 6.554989] xgbe_get_all_hw_features:868: amd-xgbe 0000:0d:00.7: RX fifo size : 65536
[ 6.554990] xgbe_get_all_hw_features:870: amd-xgbe 0000:0d:00.7: TX fifo size : 65536
[ 6.554991] xgbe_get_all_hw_features:872: amd-xgbe 0000:0d:00.7: IEEE 1588 high word : yes
[ 6.554993] xgbe_get_all_hw_features:874: amd-xgbe 0000:0d:00.7: DMA width : 48
[ 6.554994] xgbe_get_all_hw_features:876: amd-xgbe 0000:0d:00.7: Data Center Bridging : yes
[ 6.554995] xgbe_get_all_hw_features:878: amd-xgbe 0000:0d:00.7: Split header : yes
[ 6.554996] xgbe_get_all_hw_features:880: amd-xgbe 0000:0d:00.7: TCP Segmentation Offload : yes
[ 6.554997] xgbe_get_all_hw_features:882: amd-xgbe 0000:0d:00.7: Debug memory interface : yes
[ 6.554999] xgbe_get_all_hw_features:884: amd-xgbe 0000:0d:00.7: Receive Side Scaling : yes
[ 6.555000] xgbe_get_all_hw_features:886: amd-xgbe 0000:0d:00.7: Traffic Class count : 2
[ 6.555002] xgbe_get_all_hw_features:888: amd-xgbe 0000:0d:00.7: Hash table size : 256
[ 6.555003] xgbe_get_all_hw_features:890: amd-xgbe 0000:0d:00.7: L3/L4 Filters : 8
[ 6.555004] xgbe_get_all_hw_features:894: amd-xgbe 0000:0d:00.7: RX queue count : 2
[ 6.555005] xgbe_get_all_hw_features:896: amd-xgbe 0000:0d:00.7: TX queue count : 2
[ 6.555007] xgbe_get_all_hw_features:898: amd-xgbe 0000:0d:00.7: RX DMA channel count : 2
[ 6.555008] xgbe_get_all_hw_features:900: amd-xgbe 0000:0d:00.7: TX DMA channel count : 2
[ 6.555009] xgbe_get_all_hw_features:902: amd-xgbe 0000:0d:00.7: PPS outputs : 0
[ 6.555010] xgbe_get_all_hw_features:904: amd-xgbe 0000:0d:00.7: Auxiliary snapshot inputs : 0
[ 6.555011] xgbe_set_counts:255: amd-xgbe 0000:0d:00.7: TX/RX DMA channel count = 2/2
[ 6.555013] xgbe_set_counts:257: amd-xgbe 0000:0d:00.7: TX/RX hardware queue count = 2/2
[ 6.555014] xgbe_pci_probe:384: amd-xgbe 0000:0d:00.7: max tx/rx max fifo size = 65536/65536
[ 6.555115] xgbe_config_multi_msi:158: amd-xgbe 0000:0d:00.7: multi MSI-X interrupts enabled
[ 6.555118] xgbe_config_irqs:196: amd-xgbe 0000:0d:00.7: dev irq=108
[ 6.555119] xgbe_config_irqs:197: amd-xgbe 0000:0d:00.7: ecc irq=109
[ 6.555121] xgbe_config_irqs:198: amd-xgbe 0000:0d:00.7: i2c irq=110
[ 6.555122] xgbe_config_irqs:199: amd-xgbe 0000:0d:00.7: an irq=111
[ 6.555124] xgbe_config_irqs:201: amd-xgbe 0000:0d:00.7: dma0 irq=112
[ 6.555125] xgbe_config_irqs:201: amd-xgbe 0000:0d:00.7: dma1 irq=113
[ 6.555167] xgbe_config_netdev:319: amd-xgbe 0000:0d:00.7: adjusted TX/RX DMA channel count = 2/2
[ 6.555175] xgbe_i2c_get_features:359: amd-xgbe 0000:0d:00.7: I2C features: MAX_SPEED_MODE=2, RX_BUFFER_DEPTH=15, TX_BUFFER_DEPTH=15
[ 6.555181] xgbe_phy_init:3183: amd-xgbe 0000:0d:00.7: port mode=1
[ 6.555183] xgbe_phy_init:3184: amd-xgbe 0000:0d:00.7: port id=3
[ 6.555185] xgbe_phy_init:3185: amd-xgbe 0000:0d:00.7: port speeds=0xa
[ 6.555188] xgbe_phy_init:3186: amd-xgbe 0000:0d:00.7: conn type=4
[ 6.555189] xgbe_phy_init:3187: amd-xgbe 0000:0d:00.7: mdio addr=0
[ 6.555191] xgbe_phy_init:3381: amd-xgbe 0000:0d:00.7: phy supported=0x0000000,00000000,001b6040
[ 6.555402] xgbe_dump_phy_registers:1511: amd-xgbe 0000:0d:00.7:
[ 6.555407] xgbe_dump_phy_registers:1513: amd-xgbe 0000:0d:00.7: PCS Control Reg (0x0000) = 0x0440
[ 6.555412] xgbe_dump_phy_registers:1515: amd-xgbe 0000:0d:00.7: PCS Status Reg (0x0001) = 0x00c2
[ 6.555416] xgbe_dump_phy_registers:1517: amd-xgbe 0000:0d:00.7: Phy Id (PHYS ID 1 0x0002)= 0x6000
[ 6.555421] xgbe_dump_phy_registers:1519: amd-xgbe 0000:0d:00.7: Phy Id (PHYS ID 2 0x0003)= 0x02f0
[ 6.555425] xgbe_dump_phy_registers:1521: amd-xgbe 0000:0d:00.7: Devices in Package (0x0005)= 0x008a
[ 6.555429] xgbe_dump_phy_registers:1523: amd-xgbe 0000:0d:00.7: Devices in Package (0x0006)= 0xc000
[ 6.555433] xgbe_dump_phy_registers:1526: amd-xgbe 0000:0d:00.7: Auto-Neg Control Reg (0x0000) = 0x3000
[ 6.555438] xgbe_dump_phy_registers:1528: amd-xgbe 0000:0d:00.7: Auto-Neg Status Reg (0x0001) = 0x0008
[ 6.555442] xgbe_dump_phy_registers:1530: amd-xgbe 0000:0d:00.7: Auto-Neg Ad Reg 1 (0x0010) = 0x0001
[ 6.555446] xgbe_dump_phy_registers:1533: amd-xgbe 0000:0d:00.7: Auto-Neg Ad Reg 2 (0x0011) = 0x00a0
[ 6.555451] xgbe_dump_phy_registers:1536: amd-xgbe 0000:0d:00.7: Auto-Neg Ad Reg 3 (0x0012) = 0x0000
[ 6.555456] xgbe_dump_phy_registers:1539: amd-xgbe 0000:0d:00.7: Auto-Neg Completion Reg (0x0030) = 0x0001
[ 6.555458] xgbe_dump_phy_registers:1543: amd-xgbe 0000:0d:00.7:
[ 6.556126] xgbe_config_netdev:407: amd-xgbe 0000:0d:00.7 eth3: 2 Tx software queues
[ 6.556130] xgbe_config_netdev:409: amd-xgbe 0000:0d:00.7 eth3: 2 Rx software queues
[ 6.556132] amd-xgbe 0000:0d:00.7 eth3: net device enabled
[ 8.483462] amd-xgbe 0000:0d:00.7 bp3: renamed from eth3
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: PROBLEM: XGBE unable to autoneg when advertising only 1GbE KX mode
@ 2022-05-19 19:02 Pighin, Anthony (Nokia - CA/Ottawa)
0 siblings, 0 replies; 6+ messages in thread
From: Pighin, Anthony (Nokia - CA/Ottawa) @ 2022-05-19 19:02 UTC (permalink / raw)
To: thomas.lendacky, Shyam-sundar.S-k
Cc: netdev, Pighin, Anthony (Nokia - CA/Ottawa)
(Consolidating the two threads; adding even more asymmetric results)
Additional information:
I performed an asymmetric test. One side has 'ethtool -s bp3 advertise 0x20000' and the other side of the link has 'ethtool -s bp3 advertise 0xa0000' (ie. one side is advertising 1G only, the other is advertising 1G+10G). If I bring up the 1G-only side first, followed by the 1G+10G side, then the link properly comes up at 1G. However, if I do the reverse, and bring up the 1G+10G side first, followed by the 1G-only side, there is no link up. I need to bounce the 1G+10G side again (ip set link bp3 down/up) to get the link to come up.
I then performed a second asymmetric test. One side had 'ethtool -s bp3 0x120000' and the other side of the link has 'ethtool -s bp3 advertise 0xa0000' (ie. both sides have 1G+10G, but the 10G differ between FEC and non-FEC). This means the common denominator is the 1G speed. In this case, no matter what side of the link I brought up first, they linked up at 1G.
So it appears that either two speeds need to be advertised, or that at least one 10G speed needs to be advertised, even if it doesn't match in the end.
The setup is two identical systems using the XGBE in backplane mode for inter-system communications (redundancy). The processor itself is a Zen (Family 23, Model 1, Stepping 2).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: PROBLEM: XGBE unable to autoneg when advertising only 1GbE KX mode
2022-05-19 14:28 Pighin, Anthony (Nokia - CA/Ottawa)
@ 2022-05-19 17:54 ` Tom Lendacky
0 siblings, 0 replies; 6+ messages in thread
From: Tom Lendacky @ 2022-05-19 17:54 UTC (permalink / raw)
To: Pighin, Anthony (Nokia - CA/Ottawa), S-k, Shyam-sundar; +Cc: netdev
On 5/19/22 09:28, Pighin, Anthony (Nokia - CA/Ottawa) wrote:
> Additional information:
Adding Shyam.
Tom
>
> I performed an asymmetric test. One side has 'ethtool -s bp3 advertise 0x20000' and the other side of the link has 'ethtool -s bp3 advertise 0xa0000' (ie. one side is advertising 1G only, the other is advertising 1G+10G). If I bring up the 1G-only side first, followed by the 1G+10G side, then the link properly comes up at 1G. However, if I do the reverse, and bring up the 1G+10G side first, followed by the 1G-only side, there is no link up. I need to bounce the 1G+10G side again (ip set link bp3 down/up) to get the link to come up.
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: PROBLEM: XGBE unable to autoneg when advertising only 1GbE KX mode
2022-05-18 18:58 Pighin, Anthony (Nokia - CA/Ottawa)
@ 2022-05-19 17:53 ` Tom Lendacky
0 siblings, 0 replies; 6+ messages in thread
From: Tom Lendacky @ 2022-05-19 17:53 UTC (permalink / raw)
To: Pighin, Anthony (Nokia - CA/Ottawa), S-k, Shyam-sundar; +Cc: netdev
On 5/18/22 13:58, Pighin, Anthony (Nokia - CA/Ottawa) wrote:
> Tested on Linux 5.15.22.
Can you supply some additional information about the system this is on?
Adding Shyam, as he will be taking over maintainership for this driver (I
just need to get around sending the patch for that).
Thanks,
Tom
>
> Step 1. Force (both sides) of the Link to advertise 1000baseKX/Full only
> root@localhost:~# ethtool --change bp3 advertise 0x20000
> root@localhost:~# ethtool bp3
> Settings for bp3:
> Supported ports: [ Backplane ]
> Supported link modes: 1000baseKX/Full
> 10000baseKR/Full
> 10000baseR_FEC
> Supported pause frame use: Symmetric Receive-only
> Supports auto-negotiation: Yes
> Supported FEC modes: Not reported
> Advertised link modes: 1000baseKX/Full
> Advertised pause frame use: No
> Advertised auto-negotiation: Yes
> Advertised FEC modes: Not reported
> Speed: Unknown!
> Duplex: Unknown! (255)
> Port: None
> PHYAD: 0
> Transceiver: internal
> Auto-negotiation: on
> Current message level: 0x00000034 (52)
> link ifdown ifup
> Link detected: no
>
> Step 2. Bring Up both sides of the Link:
> root@localhost:~# ip link set bp3 up
>
> Result. Link stays Down:
> root@localhost:~# ethtool bp3
> Settings for bp3:
> Supported ports: [ Backplane ]
> Supported link modes: 1000baseKX/Full
> 10000baseKR/Full
> 10000baseR_FEC
> Supported pause frame use: Symmetric Receive-only
> Supports auto-negotiation: Yes
> Supported FEC modes: Not reported
> Advertised link modes: 1000baseKX/Full
> Advertised pause frame use: No
> Advertised auto-negotiation: Yes
> Advertised FEC modes: Not reported
> Link partner advertised link modes: Not reported
> Link partner advertised pause frame use: No
> Link partner advertised auto-negotiation: Yes
> Link partner advertised FEC modes: Not reported
> Speed: Unknown!
> Duplex: Full
> Port: None
> PHYAD: 0
> Transceiver: internal
> Auto-negotiation: on
> Current message level: 0x00000034 (52)
> link ifdown ifup
> Link detected: no
>
> Step 3. Change one side of the Link to advertise 1000baseKX/Full and 10000baseKR/Full:
> root@localhost:~# ethtool --change bp3 advertise 0xa000
>
> Result: Link comes Up at 1000baseKX/Full:
> root@localhost:~# ethtool bp3
> Settings for bp3:
> Supported ports: [ Backplane ]
> Supported link modes: 1000baseKX/Full
> 10000baseKR/Full
> 10000baseR_FEC
> Supported pause frame use: Symmetric Receive-only
> Supports auto-negotiation: Yes
> Supported FEC modes: Not reported
> Advertised link modes: 1000baseKX/Full
> 10000baseKR/Full
> Advertised pause frame use: No
> Advertised auto-negotiation: Yes
> Advertised FEC modes: Not reported
> Link partner advertised link modes: 1000baseKX/Full
> Link partner advertised pause frame use: No
> Link partner advertised auto-negotiation: Yes
> Link partner advertised FEC modes: Not reported
> Speed: 1000Mb/s
> Duplex: Full
> Port: None
> PHYAD: 0
> Transceiver: internal
> Auto-negotiation: on
> Current message level: 0x00000034 (52)
> link ifdown ifup
> Link detected: yes
>
>
> Logs during initial Link up failures (when only advertising 1000baseKX/Full):
>
> [ 581.429431] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.429437] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
> [ 581.429722] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.429724] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
> [ 581.542950] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.542954] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x00000001
> [ 581.661399] xgbe_check_link_timeout:1292: amd-xgbe 0000:0d:00.7 bp3: AN link timeout
> [ 581.661403] __xgbe_phy_config_aneg:1214: amd-xgbe 0000:0d:00.7 bp3: AN PHY configuration
> [ 581.663591] xgbe_phy_kx_1000_mode:2160: amd-xgbe 0000:0d:00.7 bp3: 1GbE KX mode set
> [ 581.663602] xgbe_an73_disable:422: amd-xgbe 0000:0d:00.7 bp3: CL73 AN disabled
> [ 581.663618] xgbe_an37_disable:381: amd-xgbe 0000:0d:00.7 bp3: CL37 AN disabled
> [ 581.663639] xgbe_an73_init:1051: amd-xgbe 0000:0d:00.7 bp3: CL73 AN initialized
> [ 581.663650] xgbe_an73_restart:412: amd-xgbe 0000:0d:00.7 bp3: CL73 AN enabled/restarted
> [ 581.763656] xgbe_an_isr_task:695: amd-xgbe 0000:0d:00.7 bp3: AN interrupt received
> [ 581.763672] xgbe_an73_state_machine:847: amd-xgbe 0000:0d:00.7 bp3: CL73 AN Incompatible-Link
> [ 581.763676] xgbe_an73_state_machine:907: amd-xgbe 0000:0d:00.7 bp3: CL73 AN result: No-Link
> [ 581.763682] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.763685] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
> [ 581.764848] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.764851] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
> [ 581.765865] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.765867] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
> [ 581.766258] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.766260] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
> [ 581.767364] xgbe_phy_kr_mode:2132: amd-xgbe 0000:0d:00.7 bp3: 10GbE KR mode set
> [ 581.767593] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.767596] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
> [ 581.769699] xgbe_phy_power_off:2053: amd-xgbe 0000:0d:00.7 bp3: phy powered off
> [ 581.772065] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.772068] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
> [ 581.773534] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.773537] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
> [ 581.773593] xgbe_phy_kr_mode:2132: amd-xgbe 0000:0d:00.7 bp3: 10GbE KR mode set
> [ 581.773604] xgbe_an73_disable:422: amd-xgbe 0000:0d:00.7 bp3: CL73 AN disabled
> [ 581.773619] xgbe_an37_disable:381: amd-xgbe 0000:0d:00.7 bp3: CL37 AN disabled
> [ 581.773626] xgbe_an73_state_machine:913: amd-xgbe 0000:0d:00.7 bp3: PHY Reset
> [ 581.773628] xgbe_an73_state_machine:847: amd-xgbe 0000:0d:00.7 bp3: CL73 AN Ready
> [ 581.773723] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
> [ 581.773726] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x00000001
>
> Anthony
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: PROBLEM: XGBE unable to autoneg when advertising only 1GbE KX mode
@ 2022-05-19 14:28 Pighin, Anthony (Nokia - CA/Ottawa)
2022-05-19 17:54 ` Tom Lendacky
0 siblings, 1 reply; 6+ messages in thread
From: Pighin, Anthony (Nokia - CA/Ottawa) @ 2022-05-19 14:28 UTC (permalink / raw)
To: Pighin, Anthony (Nokia - CA/Ottawa); +Cc: netdev, thomas.lendacky
Additional information:
I performed an asymmetric test. One side has 'ethtool -s bp3 advertise 0x20000' and the other side of the link has 'ethtool -s bp3 advertise 0xa0000' (ie. one side is advertising 1G only, the other is advertising 1G+10G). If I bring up the 1G-only side first, followed by the 1G+10G side, then the link properly comes up at 1G. However, if I do the reverse, and bring up the 1G+10G side first, followed by the 1G-only side, there is no link up. I need to bounce the 1G+10G side again (ip set link bp3 down/up) to get the link to come up.
^ permalink raw reply [flat|nested] 6+ messages in thread
* PROBLEM: XGBE unable to autoneg when advertising only 1GbE KX mode
@ 2022-05-18 18:58 Pighin, Anthony (Nokia - CA/Ottawa)
2022-05-19 17:53 ` Tom Lendacky
0 siblings, 1 reply; 6+ messages in thread
From: Pighin, Anthony (Nokia - CA/Ottawa) @ 2022-05-18 18:58 UTC (permalink / raw)
To: thomas.lendacky; +Cc: netdev
Tested on Linux 5.15.22.
Step 1. Force (both sides) of the Link to advertise 1000baseKX/Full only
root@localhost:~# ethtool --change bp3 advertise 0x20000
root@localhost:~# ethtool bp3
Settings for bp3:
Supported ports: [ Backplane ]
Supported link modes: 1000baseKX/Full
10000baseKR/Full
10000baseR_FEC
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseKX/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Unknown! (255)
Port: None
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Current message level: 0x00000034 (52)
link ifdown ifup
Link detected: no
Step 2. Bring Up both sides of the Link:
root@localhost:~# ip link set bp3 up
Result. Link stays Down:
root@localhost:~# ethtool bp3
Settings for bp3:
Supported ports: [ Backplane ]
Supported link modes: 1000baseKX/Full
10000baseKR/Full
10000baseR_FEC
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseKX/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: Not reported
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: Unknown!
Duplex: Full
Port: None
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Current message level: 0x00000034 (52)
link ifdown ifup
Link detected: no
Step 3. Change one side of the Link to advertise 1000baseKX/Full and 10000baseKR/Full:
root@localhost:~# ethtool --change bp3 advertise 0xa000
Result: Link comes Up at 1000baseKX/Full:
root@localhost:~# ethtool bp3
Settings for bp3:
Supported ports: [ Backplane ]
Supported link modes: 1000baseKX/Full
10000baseKR/Full
10000baseR_FEC
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 1000baseKX/Full
10000baseKR/Full
Advertised pause frame use: No
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 1000baseKX/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Port: None
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Current message level: 0x00000034 (52)
link ifdown ifup
Link detected: yes
Logs during initial Link up failures (when only advertising 1000baseKX/Full):
[ 581.429431] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.429437] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
[ 581.429722] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.429724] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
[ 581.542950] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.542954] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x00000001
[ 581.661399] xgbe_check_link_timeout:1292: amd-xgbe 0000:0d:00.7 bp3: AN link timeout
[ 581.661403] __xgbe_phy_config_aneg:1214: amd-xgbe 0000:0d:00.7 bp3: AN PHY configuration
[ 581.663591] xgbe_phy_kx_1000_mode:2160: amd-xgbe 0000:0d:00.7 bp3: 1GbE KX mode set
[ 581.663602] xgbe_an73_disable:422: amd-xgbe 0000:0d:00.7 bp3: CL73 AN disabled
[ 581.663618] xgbe_an37_disable:381: amd-xgbe 0000:0d:00.7 bp3: CL37 AN disabled
[ 581.663639] xgbe_an73_init:1051: amd-xgbe 0000:0d:00.7 bp3: CL73 AN initialized
[ 581.663650] xgbe_an73_restart:412: amd-xgbe 0000:0d:00.7 bp3: CL73 AN enabled/restarted
[ 581.763656] xgbe_an_isr_task:695: amd-xgbe 0000:0d:00.7 bp3: AN interrupt received
[ 581.763672] xgbe_an73_state_machine:847: amd-xgbe 0000:0d:00.7 bp3: CL73 AN Incompatible-Link
[ 581.763676] xgbe_an73_state_machine:907: amd-xgbe 0000:0d:00.7 bp3: CL73 AN result: No-Link
[ 581.763682] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.763685] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
[ 581.764848] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.764851] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
[ 581.765865] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.765867] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
[ 581.766258] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.766260] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
[ 581.767364] xgbe_phy_kr_mode:2132: amd-xgbe 0000:0d:00.7 bp3: 10GbE KR mode set
[ 581.767593] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.767596] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
[ 581.769699] xgbe_phy_power_off:2053: amd-xgbe 0000:0d:00.7 bp3: phy powered off
[ 581.772065] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.772068] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x03000001
[ 581.773534] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.773537] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x02000001
[ 581.773593] xgbe_phy_kr_mode:2132: amd-xgbe 0000:0d:00.7 bp3: 10GbE KR mode set
[ 581.773604] xgbe_an73_disable:422: amd-xgbe 0000:0d:00.7 bp3: CL73 AN disabled
[ 581.773619] xgbe_an37_disable:381: amd-xgbe 0000:0d:00.7 bp3: CL37 AN disabled
[ 581.773626] xgbe_an73_state_machine:913: amd-xgbe 0000:0d:00.7 bp3: PHY Reset
[ 581.773628] xgbe_an73_state_machine:847: amd-xgbe 0000:0d:00.7 bp3: CL73 AN Ready
[ 581.773723] xgbe_isr_task:493: amd-xgbe 0000:0d:00.7 bp3: DMA_ISR=0x00020000
[ 581.773726] xgbe_isr_task:542: amd-xgbe 0000:0d:00.7 bp3: MAC_ISR=0x00000001
Anthony
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-05-24 12:34 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-24 12:34 PROBLEM: XGBE unable to autoneg when advertising only 1GbE KX mode Pighin, Anthony (Nokia - CA/Ottawa)
-- strict thread matches above, loose matches on Subject: below --
2022-05-19 19:02 Pighin, Anthony (Nokia - CA/Ottawa)
2022-05-19 14:28 Pighin, Anthony (Nokia - CA/Ottawa)
2022-05-19 17:54 ` Tom Lendacky
2022-05-18 18:58 Pighin, Anthony (Nokia - CA/Ottawa)
2022-05-19 17:53 ` Tom Lendacky
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.