* status of rate adaptation
@ 2022-11-11 20:44 Tim Harvey
2022-11-11 20:57 ` Sean Anderson
0 siblings, 1 reply; 21+ messages in thread
From: Tim Harvey @ 2022-11-11 20:44 UTC (permalink / raw)
To: netdev, Sean Anderson, Russell King, David S. Miller
Greetings,
I've noticed some recent commits that appear to add rate adaptation support:
3c42563b3041 net: phy: aquantia: Add support for rate matching
7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
0c3e10cb4423 net: phy: Add support for rate matching
I have a board with an AQR113C PHY over XFI that functions properly at
10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
Should I expect this to work now at those lower rates and if so what
kind of debug information or testing can I provide?
Best Regards,
Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 20:44 status of rate adaptation Tim Harvey
@ 2022-11-11 20:57 ` Sean Anderson
2022-11-11 20:58 ` Sean Anderson
0 siblings, 1 reply; 21+ messages in thread
From: Sean Anderson @ 2022-11-11 20:57 UTC (permalink / raw)
To: Tim Harvey, netdev, Russell King, David S. Miller
Hi Tim,
On 11/11/22 15:44, Tim Harvey wrote:
> Greetings,
>
> I've noticed some recent commits that appear to add rate adaptation support:
> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> 0c3e10cb4423 net: phy: Add support for rate matching
>
> I have a board with an AQR113C PHY over XFI that functions properly at
> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
>
> Should I expect this to work now at those lower rates
Yes.
> and if so what kind of debug information or testing can I provide?
Please send
- Your test procedure (how do you select 1G?)
- Device tree node for the interface
- Output of ethtool (on both ends if possible).
- Kernel logs with debug enabled for drivers/phylink.c
That should be enough to get us started.
--Sean
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 20:57 ` Sean Anderson
@ 2022-11-11 20:58 ` Sean Anderson
2022-11-11 21:20 ` Tim Harvey
0 siblings, 1 reply; 21+ messages in thread
From: Sean Anderson @ 2022-11-11 20:58 UTC (permalink / raw)
To: Tim Harvey, netdev, Russell King, David S. Miller
On 11/11/22 15:57, Sean Anderson wrote:
> Hi Tim,
>
> On 11/11/22 15:44, Tim Harvey wrote:
>> Greetings,
>>
>> I've noticed some recent commits that appear to add rate adaptation support:
>> 3c42563b3041 net: phy: aquantia: Add support for rate matching
>> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
>> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
>> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
>> 0c3e10cb4423 net: phy: Add support for rate matching
>>
>> I have a board with an AQR113C PHY over XFI that functions properly at
>> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
>>
>> Should I expect this to work now at those lower rates
>
> Yes.
>
>> and if so what kind of debug information or testing can I provide?
>
> Please send
>
> - Your test procedure (how do you select 1G?)
> - Device tree node for the interface
> - Output of ethtool (on both ends if possible).
> - Kernel logs with debug enabled for drivers/phylink.c
Sorry, this should be drivers/net/phy/phylink.c
>
> That should be enough to get us started.
>
> --Sean
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 20:58 ` Sean Anderson
@ 2022-11-11 21:20 ` Tim Harvey
2022-11-11 21:54 ` Sean Anderson
0 siblings, 1 reply; 21+ messages in thread
From: Tim Harvey @ 2022-11-11 21:20 UTC (permalink / raw)
To: Sean Anderson; +Cc: netdev, Russell King, David S. Miller
On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
>
> On 11/11/22 15:57, Sean Anderson wrote:
> > Hi Tim,
> >
> > On 11/11/22 15:44, Tim Harvey wrote:
> >> Greetings,
> >>
> >> I've noticed some recent commits that appear to add rate adaptation support:
> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> >> 0c3e10cb4423 net: phy: Add support for rate matching
> >>
> >> I have a board with an AQR113C PHY over XFI that functions properly at
> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> >>
> >> Should I expect this to work now at those lower rates
> >
> > Yes.
Sean,
Good to hear - thank you for your work on this feature!
> >
> >> and if so what kind of debug information or testing can I provide?
> >
> > Please send
> >
> > - Your test procedure (how do you select 1G?)
> > - Device tree node for the interface
> > - Output of ethtool (on both ends if possible).
> > - Kernel logs with debug enabled for drivers/phylink.c
>
> Sorry, this should be drivers/net/phy/phylink.c
>
> >
> > That should be enough to get us started.
> >
> > --Sean
>
I'm currently testing by bringing up the network interface while
connected to a 10gbe switch, verifying link and traffic, then forcing
the switch port to 1000mbps.
The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
#include "cn9130.dtsi" /* include SoC device tree */
&cp0_xmdio {
pinctrl-names = "default";
pinctrl-0 = <&cp0_xsmi_pins>;
status = "okay";
phy1: ethernet-phy@8 {
compatible = "ethernet-phy-ieee802.3-c45";
reg = <8>;
};
};
&cp0_ethernet {
status = "okay";
};
/* 10GbE XFI AQR113C */
&cp0_eth0 {
status = "okay";
phy = <&phy1>;
phy-mode = "10gbase-r";
phys = <&cp0_comphy4 0>;
};
Here are some logs with debug enabled in drivers/net/phy/phylink.c and
some additional debug in mvpp2.c and aquantia_main.c:
# ifconfig eth0 192.168.1.22
[ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
speed=-1 26:10gbase-r
[ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
[ 8.898165] aqr107_resume
[ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
duplex=255 speed=-1 26:10gbase-r 0:
[ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
[f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
[ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
supported 00000000,00018000,000e706f advertising
00000000,00018000,000e706f
[ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
[ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
phy/10gbase-r link mode
[ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
[ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
pause=00 link=0 an=0
[ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
[ 8.976267] aqr107_resume
[ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
10gbase-r/10Gbps/Full/none/off
[ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
duplex=1 speed=10000 26:10gbase-r
[ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
[ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
- flow control off
[ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
10gbase-r/10Gbps/Full/none/off
[ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
duplex=1 speed=10000 26:10gbase-r
# ethtool eth0
Settings for eth0:
Supported ports: [ ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
10000baseT/Full
1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
2500baseT/Full
5000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
10000baseT/Full
1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
2500baseT/Full
5000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
10000baseT/Full
2500baseT/Full
5000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 8
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Link detected: yes
# ping 192.168.1.146 -c5
PING 192.168.1.146 (192.168.1.146): 56 data bytes
64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
--- 192.168.1.146 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.267/0.416/0.991 ms
# # force switch port to 1G
[ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
10gbase-r/Unknown/Unknown/none/off
[ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
[ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
[ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
duplex=255 speed=-1 26:10gbase-r
[ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
[ 197.472504] mvpp2 f2000000.ethernet eth0: major config
[ 197.472614] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
mode=phy//1Gbps/Full/pause adv=00000000,00000000,00000000 pause=00
link=1 an=0
[ 197.479561] aqr107_link_change_notify state=4:running an=1 link=1
duplex=1 speed=1000 0:
[ 197.484972] mvpp2 f2000000.ethernet eth0: Link is Up - 1Gbps/Full -
flow control off
# ethtool eth0
Settings for eth0:
Supported ports: [ ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
10000baseT/Full
1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
2500baseT/Full
5000baseT/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: Yes
Supported FEC modes: Not reported
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
10000baseT/Full
1000baseKX/Full
10000baseKX4/Full
10000baseKR/Full
2500baseT/Full
5000baseT/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: Yes
Advertised FEC modes: Not reported
Link partner advertised link modes: 1000baseT/Half 1000baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Link partner advertised FEC modes: Not reported
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 8
Transceiver: external
Auto-negotiation: on
MDI-X: Unknown
Link detected: yes
# ping 192.168.1.146 -c5
PING 192.168.1.146 (192.168.1.146): 56 data bytes
--- 192.168.1.146 ping statistics ---
5 packets transmitted, 0 packets received, 100% packet loss
Best Regards,
Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 21:20 ` Tim Harvey
@ 2022-11-11 21:54 ` Sean Anderson
2022-11-11 22:14 ` Tim Harvey
` (2 more replies)
0 siblings, 3 replies; 21+ messages in thread
From: Sean Anderson @ 2022-11-11 21:54 UTC (permalink / raw)
To: Tim Harvey; +Cc: netdev, Russell King, David S. Miller
On 11/11/22 16:20, Tim Harvey wrote:
> On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
>>
>> On 11/11/22 15:57, Sean Anderson wrote:
>> > Hi Tim,
>> >
>> > On 11/11/22 15:44, Tim Harvey wrote:
>> >> Greetings,
>> >>
>> >> I've noticed some recent commits that appear to add rate adaptation support:
>> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
>> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
>> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
>> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
>> >> 0c3e10cb4423 net: phy: Add support for rate matching
>> >>
>> >> I have a board with an AQR113C PHY over XFI that functions properly at
>> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
>> >>
>> >> Should I expect this to work now at those lower rates
>> >
>> > Yes.
>
> Sean,
>
> Good to hear - thank you for your work on this feature!
>
>> >
>> >> and if so what kind of debug information or testing can I provide?
>> >
>> > Please send
>> >
>> > - Your test procedure (how do you select 1G?)
>> > - Device tree node for the interface
>> > - Output of ethtool (on both ends if possible).
>> > - Kernel logs with debug enabled for drivers/phylink.c
>>
>> Sorry, this should be drivers/net/phy/phylink.c
>>
>> >
>> > That should be enough to get us started.
>> >
>> > --Sean
>>
>
> I'm currently testing by bringing up the network interface while
> connected to a 10gbe switch, verifying link and traffic, then forcing
> the switch port to 1000mbps.
>
> The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
>
> #include "cn9130.dtsi" /* include SoC device tree */
>
> &cp0_xmdio {
> pinctrl-names = "default";
> pinctrl-0 = <&cp0_xsmi_pins>;
> status = "okay";
>
> phy1: ethernet-phy@8 {
> compatible = "ethernet-phy-ieee802.3-c45";
> reg = <8>;
> };
> };
>
> &cp0_ethernet {
> status = "okay";
> };
>
> /* 10GbE XFI AQR113C */
> &cp0_eth0 {
> status = "okay";
> phy = <&phy1>;
> phy-mode = "10gbase-r";
> phys = <&cp0_comphy4 0>;
> };
>
> Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> some additional debug in mvpp2.c and aquantia_main.c:
> # ifconfig eth0 192.168.1.22
> [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> speed=-1 26:10gbase-r
> [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> [ 8.898165] aqr107_resume
> [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> duplex=255 speed=-1 26:10gbase-r 0:
> [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> supported 00000000,00018000,000e706f advertising
> 00000000,00018000,000e706f
> [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> phy/10gbase-r link mode
> [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> pause=00 link=0 an=0
> [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> [ 8.976267] aqr107_resume
> [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> 10gbase-r/10Gbps/Full/none/off
> [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> duplex=1 speed=10000 26:10gbase-r
> [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> - flow control off
> [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> 10gbase-r/10Gbps/Full/none/off
> [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> duplex=1 speed=10000 26:10gbase-r
>
> # ethtool eth0
> Settings for eth0:
> Supported ports: [ ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
10/100 half duplex aren't achievable with rate matching (and we avoid
turning them on), so they must be coming from somewhere else. I wonder
if this is because PHY_INTERFACE_MODE_SGMII is set in
supported_interfaces.
I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
should support it. I'm not sure if the aquantia driver is set up for it.
> 1000baseT/Full
> 10000baseT/Full
> 1000baseKX/Full
> 10000baseKX4/Full
> 10000baseKR/Full
> 2500baseT/Full
> 5000baseT/Full
> Supported pause frame use: Symmetric Receive-only
> Supports auto-negotiation: Yes
> Supported FEC modes: Not reported
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Full
> 10000baseT/Full
> 1000baseKX/Full
> 10000baseKX4/Full
> 10000baseKR/Full
> 2500baseT/Full
> 5000baseT/Full
> Advertised pause frame use: Symmetric Receive-only
> Advertised auto-negotiation: Yes
> Advertised FEC modes: Not reported
> Link partner advertised link modes: 100baseT/Half 100baseT/Full
> 1000baseT/Half 1000baseT/Full
> 10000baseT/Full
> 2500baseT/Full
> 5000baseT/Full
> Link partner advertised pause frame use: No
> Link partner advertised auto-negotiation: Yes
> Link partner advertised FEC modes: Not reported
> Speed: 10000Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 8
> Transceiver: external
> Auto-negotiation: on
> MDI-X: Unknown
> Link detected: yes
> # ping 192.168.1.146 -c5
> PING 192.168.1.146 (192.168.1.146): 56 data bytes
> 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
>
> --- 192.168.1.146 ping statistics ---
> 5 packets transmitted, 5 packets received, 0% packet loss
> round-trip min/avg/max = 0.267/0.416/0.991 ms
> # # force switch port to 1G
> [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> 10gbase-r/Unknown/Unknown/none/off
> [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> duplex=255 speed=-1 26:10gbase-r
> [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
and the vendor provisioning registers (dev 4, reg 0xc440 through
0xc449).
It's possible that your firmware doesn't support rate adaptation... I'm
not sure what we can do about that.
--Sean
> [ 197.472504] mvpp2 f2000000.ethernet eth0: major config
> [ 197.472614] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> mode=phy//1Gbps/Full/pause adv=00000000,00000000,00000000 pause=00
> link=1 an=0
> [ 197.479561] aqr107_link_change_notify state=4:running an=1 link=1
> duplex=1 speed=1000 0:
> [ 197.484972] mvpp2 f2000000.ethernet eth0: Link is Up - 1Gbps/Full -
> flow control off
> # ethtool eth0
> Settings for eth0:
> Supported ports: [ ]
> Supported link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Full
> 10000baseT/Full
> 1000baseKX/Full
> 10000baseKX4/Full
> 10000baseKR/Full
> 2500baseT/Full
> 5000baseT/Full
> Supported pause frame use: Symmetric Receive-only
> Supports auto-negotiation: Yes
> Supported FEC modes: Not reported
> Advertised link modes: 10baseT/Half 10baseT/Full
> 100baseT/Half 100baseT/Full
> 1000baseT/Full
> 10000baseT/Full
> 1000baseKX/Full
> 10000baseKX4/Full
> 10000baseKR/Full
> 2500baseT/Full
> 5000baseT/Full
> Advertised pause frame use: Symmetric Receive-only
> Advertised auto-negotiation: Yes
> Advertised FEC modes: Not reported
> Link partner advertised link modes: 1000baseT/Half 1000baseT/Full
> Link partner advertised pause frame use: No
> Link partner advertised auto-negotiation: Yes
> Link partner advertised FEC modes: Not reported
> Speed: 1000Mb/s
> Duplex: Full
> Port: Twisted Pair
> PHYAD: 8
> Transceiver: external
> Auto-negotiation: on
> MDI-X: Unknown
> Link detected: yes
> # ping 192.168.1.146 -c5
> PING 192.168.1.146 (192.168.1.146): 56 data bytes
>
> --- 192.168.1.146 ping statistics ---
> 5 packets transmitted, 0 packets received, 100% packet loss
>
> Best Regards,
>
> Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 21:54 ` Sean Anderson
@ 2022-11-11 22:14 ` Tim Harvey
2022-11-11 22:38 ` Sean Anderson
2022-11-11 22:33 ` Russell King (Oracle)
2022-11-12 13:15 ` Russell King (Oracle)
2 siblings, 1 reply; 21+ messages in thread
From: Tim Harvey @ 2022-11-11 22:14 UTC (permalink / raw)
To: Sean Anderson; +Cc: netdev, Russell King, David S. Miller
On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
>
> On 11/11/22 16:20, Tim Harvey wrote:
> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >>
> >> On 11/11/22 15:57, Sean Anderson wrote:
> >> > Hi Tim,
> >> >
> >> > On 11/11/22 15:44, Tim Harvey wrote:
> >> >> Greetings,
> >> >>
> >> >> I've noticed some recent commits that appear to add rate adaptation support:
> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
> >> >>
> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> >> >>
> >> >> Should I expect this to work now at those lower rates
> >> >
> >> > Yes.
> >
> > Sean,
> >
> > Good to hear - thank you for your work on this feature!
> >
> >> >
> >> >> and if so what kind of debug information or testing can I provide?
> >> >
> >> > Please send
> >> >
> >> > - Your test procedure (how do you select 1G?)
> >> > - Device tree node for the interface
> >> > - Output of ethtool (on both ends if possible).
> >> > - Kernel logs with debug enabled for drivers/phylink.c
> >>
> >> Sorry, this should be drivers/net/phy/phylink.c
> >>
> >> >
> >> > That should be enough to get us started.
> >> >
> >> > --Sean
> >>
> >
> > I'm currently testing by bringing up the network interface while
> > connected to a 10gbe switch, verifying link and traffic, then forcing
> > the switch port to 1000mbps.
> >
> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
> >
> > #include "cn9130.dtsi" /* include SoC device tree */
> >
> > &cp0_xmdio {
> > pinctrl-names = "default";
> > pinctrl-0 = <&cp0_xsmi_pins>;
> > status = "okay";
> >
> > phy1: ethernet-phy@8 {
> > compatible = "ethernet-phy-ieee802.3-c45";
> > reg = <8>;
> > };
> > };
> >
> > &cp0_ethernet {
> > status = "okay";
> > };
> >
> > /* 10GbE XFI AQR113C */
> > &cp0_eth0 {
> > status = "okay";
> > phy = <&phy1>;
> > phy-mode = "10gbase-r";
> > phys = <&cp0_comphy4 0>;
> > };
> >
> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> > some additional debug in mvpp2.c and aquantia_main.c:
> > # ifconfig eth0 192.168.1.22
> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> > speed=-1 26:10gbase-r
> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> > [ 8.898165] aqr107_resume
> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> > duplex=255 speed=-1 26:10gbase-r 0:
> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> > supported 00000000,00018000,000e706f advertising
> > 00000000,00018000,000e706f
> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> > phy/10gbase-r link mode
> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> > pause=00 link=0 an=0
> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> > [ 8.976267] aqr107_resume
> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> > 10gbase-r/10Gbps/Full/none/off
> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> > duplex=1 speed=10000 26:10gbase-r
> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> > - flow control off
> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> > 10gbase-r/10Gbps/Full/none/off
> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> > duplex=1 speed=10000 26:10gbase-r
> >
> > # ethtool eth0
> > Settings for eth0:
> > Supported ports: [ ]
> > Supported link modes: 10baseT/Half 10baseT/Full
> > 100baseT/Half 100baseT/Full
>
> 10/100 half duplex aren't achievable with rate matching (and we avoid
> turning them on), so they must be coming from somewhere else. I wonder
> if this is because PHY_INTERFACE_MODE_SGMII is set in
> supported_interfaces.
>
> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> should support it. I'm not sure if the aquantia driver is set up for it.
This appears to trigger an issue from mvpp2:
mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
00000000,00018000,000e706f and advertisement
00000000,00018000,000e706f failed: -EINVAL
>
> > 1000baseT/Full
> > 10000baseT/Full
> > 1000baseKX/Full
> > 10000baseKX4/Full
> > 10000baseKR/Full
> > 2500baseT/Full
> > 5000baseT/Full
> > Supported pause frame use: Symmetric Receive-only
> > Supports auto-negotiation: Yes
> > Supported FEC modes: Not reported
> > Advertised link modes: 10baseT/Half 10baseT/Full
> > 100baseT/Half 100baseT/Full
> > 1000baseT/Full
> > 10000baseT/Full
> > 1000baseKX/Full
> > 10000baseKX4/Full
> > 10000baseKR/Full
> > 2500baseT/Full
> > 5000baseT/Full
> > Advertised pause frame use: Symmetric Receive-only
> > Advertised auto-negotiation: Yes
> > Advertised FEC modes: Not reported
> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
> > 1000baseT/Half 1000baseT/Full
> > 10000baseT/Full
> > 2500baseT/Full
> > 5000baseT/Full
> > Link partner advertised pause frame use: No
> > Link partner advertised auto-negotiation: Yes
> > Link partner advertised FEC modes: Not reported
> > Speed: 10000Mb/s
> > Duplex: Full
> > Port: Twisted Pair
> > PHYAD: 8
> > Transceiver: external
> > Auto-negotiation: on
> > MDI-X: Unknown
> > Link detected: yes
> > # ping 192.168.1.146 -c5
> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
> >
> > --- 192.168.1.146 ping statistics ---
> > 5 packets transmitted, 5 packets received, 0% packet loss
> > round-trip min/avg/max = 0.267/0.416/0.991 ms
> > # # force switch port to 1G
> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> > 10gbase-r/Unknown/Unknown/none/off
> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> > duplex=255 speed=-1 26:10gbase-r
> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
>
> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
> and the vendor provisioning registers (dev 4, reg 0xc440 through
> 0xc449).
yes, this is what I've been looking at as well. When forced to 1000m
the register shows a phy type of 11 which according to the aqr113
datasheet is XFI 5G:
aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
link=1 duplex=1 speed=1000 interface=0
>
> It's possible that your firmware doesn't support rate adaptation... I'm
> not sure what we can do about that.
>
I will enquire with my Aquantia FAE to see what they say about rate
adaptation support
Something interesting is that when I configured the xmdio node with an
interrupt I ended up in a mode where 5g,2.5g and 1g all worked for at
least 1 test. There was something wrong with my interrupt
configuration (i'm not clear if the AQR113C's interrupt should be
IRQ_TYPE_LEVEL_LOW, IRQ_TYPE_EDGE_FALLING or something different).
While I can't reliably reproduce this and I believe I was on the 6.0
kernel at the time without the rate adaptation support a debug log
when I was in this mode shows the following:
[ 27.700221] aqr107_config_init state=1 an=1 link=0 duplex=255
speed=-1 26:10gbase-r
[ 27.709694] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
[ 27.716457] aqr107_resume
[ 27.723551] aqr107_get_rate_matching state=1 an=1 link=0 duplex=255
speed=-1 26:10gbase-r 0:
[ 27.733075] mvpp2 f2000000.ethernet eth0: PHY
[f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=40)
[ 27.752939] mvpp2 f2000000.ethernet eth0: configuring for
phy/10gbase-r link mode
[ 27.760508] aqr107_resume
[ 27.769781] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
speed=10000 26:10gbase-r
[ 32.670293] aqr107_read_status state=5 an=1 link=1 duplex=1 speed=10000 0:
[ 32.678642] aqr107_read_rate state=5 an=1 link=1 duplex=1 speed=10000 0:
[ 32.686405] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
speed=10000 0:
[ 32.686628] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
- flow control off
[ 32.702981] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
^^^ 10gbe link; ping ok
# force port to 1Gbe
[ 945.918132] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
speed=10000 26:10gbase-r
[ 945.918193] mvpp2_port_isr 10gbase-r
[ 945.919186] mvpp2_port_disable 10gbase-r
[ 945.935304] mvpp2 f2000000.ethernet eth0: Link is Down
[ 949.509595] aqr107_read_status state=5 an=1 link=1 duplex=1
speed=1000 26:10gbase-r
[ 949.518562] aqr107_read_rate state=5 an=1 link=1 duplex=1
speed=1000 26:10gbase-r
[ 949.527112] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
speed=1000 26:10gbase-r
[ 949.527166] mvpp2_port_isr 10gbase-r
[ 949.527176] mvpp2_port_enable 10gbase-r
[ 949.527306] mvpp2 f2000000.ethernet eth0: Link is Up - 1Gbps/Full -
flow control off
^^^ 1gbe link; ping ok
# force port to 2.5Gbe
[ 1024.518112] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
speed=1000 26:10gbase-r
[ 1024.518187] mvpp2_port_isr 10gbase-r
[ 1024.532897] mvpp2_port_disable 10gbase-r
[ 1024.536880] mvpp2 f2000000.ethernet eth0: Link is Down
[ 1029.295136] aqr107_read_status state=5 an=1 link=1 duplex=1
speed=2500 26:10gbase-r
[ 1029.304070] aqr107_read_rate state=5 an=1 link=1 duplex=1
speed=2500 26:10gbase-r
[ 1029.312611] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
speed=2500 26:10gbase-r
[ 1029.312638] mvpp2_port_isr 10gbase-r
[ 1029.325584] mvpp2_port_enable 10gbase-r
[ 1029.329564] mvpp2 f2000000.ethernet eth0: Link is Up - 2.5Gbps/Full
- flow control off
^^^ 2.5gbe link; ping ok
# force port to 5gbe
[ 1060.401209] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
speed=2500 26:10gbase-r
[ 1060.401272] mvpp2_port_isr 10gbase-r
[ 1060.402274] mvpp2_port_disable 10gbase-r
[ 1060.419006] mvpp2 f2000000.ethernet eth0: Link is Down
[ 1065.167937] aqr107_read_status state=5 an=1 link=1 duplex=1
speed=5000 26:10gbase-r
[ 1065.176865] aqr107_read_rate state=5 an=1 link=1 duplex=1
speed=5000 26:10gbase-r
[ 1065.185415] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
speed=5000 26:10gbase-r
[ 1065.185456] mvpp2_port_isr 10gbase-r
[ 1065.185474] mvpp2_port_enable 10gbase-r
[ 1065.185597] mvpp2 f2000000.ethernet eth0: Link is Up - 5Gbps/Full -
flow control off
^^^ 5gpbe link; ping ok
Thanks,
Tim
> --Sean
>
> > [ 197.472504] mvpp2 f2000000.ethernet eth0: major config
> > [ 197.472614] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> > mode=phy//1Gbps/Full/pause adv=00000000,00000000,00000000 pause=00
> > link=1 an=0
> > [ 197.479561] aqr107_link_change_notify state=4:running an=1 link=1
> > duplex=1 speed=1000 0:
> > [ 197.484972] mvpp2 f2000000.ethernet eth0: Link is Up - 1Gbps/Full -
> > flow control off
> > # ethtool eth0
> > Settings for eth0:
> > Supported ports: [ ]
> > Supported link modes: 10baseT/Half 10baseT/Full
> > 100baseT/Half 100baseT/Full
> > 1000baseT/Full
> > 10000baseT/Full
> > 1000baseKX/Full
> > 10000baseKX4/Full
> > 10000baseKR/Full
> > 2500baseT/Full
> > 5000baseT/Full
> > Supported pause frame use: Symmetric Receive-only
> > Supports auto-negotiation: Yes
> > Supported FEC modes: Not reported
> > Advertised link modes: 10baseT/Half 10baseT/Full
> > 100baseT/Half 100baseT/Full
> > 1000baseT/Full
> > 10000baseT/Full
> > 1000baseKX/Full
> > 10000baseKX4/Full
> > 10000baseKR/Full
> > 2500baseT/Full
> > 5000baseT/Full
> > Advertised pause frame use: Symmetric Receive-only
> > Advertised auto-negotiation: Yes
> > Advertised FEC modes: Not reported
> > Link partner advertised link modes: 1000baseT/Half 1000baseT/Full
> > Link partner advertised pause frame use: No
> > Link partner advertised auto-negotiation: Yes
> > Link partner advertised FEC modes: Not reported
> > Speed: 1000Mb/s
> > Duplex: Full
> > Port: Twisted Pair
> > PHYAD: 8
> > Transceiver: external
> > Auto-negotiation: on
> > MDI-X: Unknown
> > Link detected: yes
> > # ping 192.168.1.146 -c5
> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> >
> > --- 192.168.1.146 ping statistics ---
> > 5 packets transmitted, 0 packets received, 100% packet loss
> >
> > Best Regards,
> >
> > Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 21:54 ` Sean Anderson
2022-11-11 22:14 ` Tim Harvey
@ 2022-11-11 22:33 ` Russell King (Oracle)
2022-11-12 13:15 ` Russell King (Oracle)
2 siblings, 0 replies; 21+ messages in thread
From: Russell King (Oracle) @ 2022-11-11 22:33 UTC (permalink / raw)
To: Sean Anderson; +Cc: Tim Harvey, netdev, David S. Miller
On Fri, Nov 11, 2022 at 04:54:40PM -0500, Sean Anderson wrote:
> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> should support it. I'm not sure if the aquantia driver is set up for it.
mvpp2 doesn't support USXGMII.
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 22:14 ` Tim Harvey
@ 2022-11-11 22:38 ` Sean Anderson
2022-11-12 0:48 ` Vladimir Oltean
2022-11-14 19:33 ` Tim Harvey
0 siblings, 2 replies; 21+ messages in thread
From: Sean Anderson @ 2022-11-11 22:38 UTC (permalink / raw)
To: Tim Harvey; +Cc: netdev, Russell King, David S. Miller
On 11/11/22 17:14, Tim Harvey wrote:
> On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
>>
>> On 11/11/22 16:20, Tim Harvey wrote:
>> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
>> >>
>> >> On 11/11/22 15:57, Sean Anderson wrote:
>> >> > Hi Tim,
>> >> >
>> >> > On 11/11/22 15:44, Tim Harvey wrote:
>> >> >> Greetings,
>> >> >>
>> >> >> I've noticed some recent commits that appear to add rate adaptation support:
>> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
>> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
>> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
>> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
>> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
>> >> >>
>> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
>> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
>> >> >>
>> >> >> Should I expect this to work now at those lower rates
>> >> >
>> >> > Yes.
>> >
>> > Sean,
>> >
>> > Good to hear - thank you for your work on this feature!
>> >
>> >> >
>> >> >> and if so what kind of debug information or testing can I provide?
>> >> >
>> >> > Please send
>> >> >
>> >> > - Your test procedure (how do you select 1G?)
>> >> > - Device tree node for the interface
>> >> > - Output of ethtool (on both ends if possible).
>> >> > - Kernel logs with debug enabled for drivers/phylink.c
>> >>
>> >> Sorry, this should be drivers/net/phy/phylink.c
>> >>
>> >> >
>> >> > That should be enough to get us started.
>> >> >
>> >> > --Sean
>> >>
>> >
>> > I'm currently testing by bringing up the network interface while
>> > connected to a 10gbe switch, verifying link and traffic, then forcing
>> > the switch port to 1000mbps.
>> >
>> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
>> >
>> > #include "cn9130.dtsi" /* include SoC device tree */
>> >
>> > &cp0_xmdio {
>> > pinctrl-names = "default";
>> > pinctrl-0 = <&cp0_xsmi_pins>;
>> > status = "okay";
>> >
>> > phy1: ethernet-phy@8 {
>> > compatible = "ethernet-phy-ieee802.3-c45";
>> > reg = <8>;
>> > };
>> > };
>> >
>> > &cp0_ethernet {
>> > status = "okay";
>> > };
>> >
>> > /* 10GbE XFI AQR113C */
>> > &cp0_eth0 {
>> > status = "okay";
>> > phy = <&phy1>;
>> > phy-mode = "10gbase-r";
>> > phys = <&cp0_comphy4 0>;
>> > };
>> >
>> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
>> > some additional debug in mvpp2.c and aquantia_main.c:
>> > # ifconfig eth0 192.168.1.22
>> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
>> > speed=-1 26:10gbase-r
>> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
>> > [ 8.898165] aqr107_resume
>> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
>> > duplex=255 speed=-1 26:10gbase-r 0:
>> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
>> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
>> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
>> > supported 00000000,00018000,000e706f advertising
>> > 00000000,00018000,000e706f
>> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
>> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
>> > phy/10gbase-r link mode
>> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
>> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
>> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
>> > pause=00 link=0 an=0
>> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
>> > [ 8.976267] aqr107_resume
>> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
>> > 10gbase-r/10Gbps/Full/none/off
>> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
>> > duplex=1 speed=10000 26:10gbase-r
>> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
>> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
>> > - flow control off
>> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
>> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
>> > 10gbase-r/10Gbps/Full/none/off
>> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
>> > duplex=1 speed=10000 26:10gbase-r
>> >
>> > # ethtool eth0
>> > Settings for eth0:
>> > Supported ports: [ ]
>> > Supported link modes: 10baseT/Half 10baseT/Full
>> > 100baseT/Half 100baseT/Full
>>
>> 10/100 half duplex aren't achievable with rate matching (and we avoid
>> turning them on), so they must be coming from somewhere else. I wonder
>> if this is because PHY_INTERFACE_MODE_SGMII is set in
>> supported_interfaces.
>>
>> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
>> should support it. I'm not sure if the aquantia driver is set up for it.
>
> This appears to trigger an issue from mvpp2:
> mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
> 00000000,00018000,000e706f and advertisement
> 00000000,00018000,000e706f failed: -EINVAL
Ah, I forgot this was a separate phy mode. Disregard this.
>>
>> > 1000baseT/Full
>> > 10000baseT/Full
>> > 1000baseKX/Full
>> > 10000baseKX4/Full
>> > 10000baseKR/Full
>> > 2500baseT/Full
>> > 5000baseT/Full
>> > Supported pause frame use: Symmetric Receive-only
>> > Supports auto-negotiation: Yes
>> > Supported FEC modes: Not reported
>> > Advertised link modes: 10baseT/Half 10baseT/Full
>> > 100baseT/Half 100baseT/Full
>> > 1000baseT/Full
>> > 10000baseT/Full
>> > 1000baseKX/Full
>> > 10000baseKX4/Full
>> > 10000baseKR/Full
>> > 2500baseT/Full
>> > 5000baseT/Full
>> > Advertised pause frame use: Symmetric Receive-only
>> > Advertised auto-negotiation: Yes
>> > Advertised FEC modes: Not reported
>> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
>> > 1000baseT/Half 1000baseT/Full
>> > 10000baseT/Full
>> > 2500baseT/Full
>> > 5000baseT/Full
>> > Link partner advertised pause frame use: No
>> > Link partner advertised auto-negotiation: Yes
>> > Link partner advertised FEC modes: Not reported
>> > Speed: 10000Mb/s
>> > Duplex: Full
>> > Port: Twisted Pair
>> > PHYAD: 8
>> > Transceiver: external
>> > Auto-negotiation: on
>> > MDI-X: Unknown
>> > Link detected: yes
>> > # ping 192.168.1.146 -c5
>> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
>> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
>> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
>> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
>> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
>> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
>> >
>> > --- 192.168.1.146 ping statistics ---
>> > 5 packets transmitted, 5 packets received, 0% packet loss
>> > round-trip min/avg/max = 0.267/0.416/0.991 ms
>> > # # force switch port to 1G
>> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
>> > 10gbase-r/Unknown/Unknown/none/off
>> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
>> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
>> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
>> > duplex=255 speed=-1 26:10gbase-r
>> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
>>
>> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
>> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
>> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
>> and the vendor provisioning registers (dev 4, reg 0xc440 through
>> 0xc449).
>
> yes, this is what I've been looking at as well. When forced to 1000m
> the register shows a phy type of 11 which according to the aqr113
> datasheet is XFI 5G:
> aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
> link=1 duplex=1 speed=1000 interface=0
That's pretty strange. Seems like it's rate adapting from 5g instead of
10g. Is SERDES Mode in the Global System Configuration For 1G register
set to XFI?
>>
>> It's possible that your firmware doesn't support rate adaptation... I'm
>> not sure what we can do about that.
>>
>
> I will enquire with my Aquantia FAE to see what they say about rate
> adaptation support
>
> Something interesting is that when I configured the xmdio node with an
> interrupt I ended up in a mode where 5g,2.5g and 1g all worked for at
> least 1 test. There was something wrong with my interrupt
> configuration (i'm not clear if the AQR113C's interrupt should be
> IRQ_TYPE_LEVEL_LOW, IRQ_TYPE_EDGE_FALLING or something different).
NXP use IRQ_TYPE_LEVEL_HIGH on the LS1046ARDB.
--Sean
> While I can't reliably reproduce this and I believe I was on the 6.0
> kernel at the time without the rate adaptation support a debug log
> when I was in this mode shows the following:
> [ 27.700221] aqr107_config_init state=1 an=1 link=0 duplex=255
> speed=-1 26:10gbase-r
> [ 27.709694] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> [ 27.716457] aqr107_resume
> [ 27.723551] aqr107_get_rate_matching state=1 an=1 link=0 duplex=255
> speed=-1 26:10gbase-r 0:
> [ 27.733075] mvpp2 f2000000.ethernet eth0: PHY
> [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=40)
> [ 27.752939] mvpp2 f2000000.ethernet eth0: configuring for
> phy/10gbase-r link mode
> [ 27.760508] aqr107_resume
> [ 27.769781] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
> speed=10000 26:10gbase-r
> [ 32.670293] aqr107_read_status state=5 an=1 link=1 duplex=1 speed=10000 0:
> [ 32.678642] aqr107_read_rate state=5 an=1 link=1 duplex=1 speed=10000 0:
> [ 32.686405] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
> speed=10000 0:
> [ 32.686628] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> - flow control off
> [ 32.702981] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> ^^^ 10gbe link; ping ok
> # force port to 1Gbe
> [ 945.918132] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
> speed=10000 26:10gbase-r
> [ 945.918193] mvpp2_port_isr 10gbase-r
> [ 945.919186] mvpp2_port_disable 10gbase-r
> [ 945.935304] mvpp2 f2000000.ethernet eth0: Link is Down
> [ 949.509595] aqr107_read_status state=5 an=1 link=1 duplex=1
> speed=1000 26:10gbase-r
> [ 949.518562] aqr107_read_rate state=5 an=1 link=1 duplex=1
> speed=1000 26:10gbase-r
> [ 949.527112] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
> speed=1000 26:10gbase-r
> [ 949.527166] mvpp2_port_isr 10gbase-r
> [ 949.527176] mvpp2_port_enable 10gbase-r
> [ 949.527306] mvpp2 f2000000.ethernet eth0: Link is Up - 1Gbps/Full -
> flow control off
> ^^^ 1gbe link; ping ok
> # force port to 2.5Gbe
> [ 1024.518112] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
> speed=1000 26:10gbase-r
> [ 1024.518187] mvpp2_port_isr 10gbase-r
> [ 1024.532897] mvpp2_port_disable 10gbase-r
> [ 1024.536880] mvpp2 f2000000.ethernet eth0: Link is Down
> [ 1029.295136] aqr107_read_status state=5 an=1 link=1 duplex=1
> speed=2500 26:10gbase-r
> [ 1029.304070] aqr107_read_rate state=5 an=1 link=1 duplex=1
> speed=2500 26:10gbase-r
> [ 1029.312611] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
> speed=2500 26:10gbase-r
> [ 1029.312638] mvpp2_port_isr 10gbase-r
> [ 1029.325584] mvpp2_port_enable 10gbase-r
> [ 1029.329564] mvpp2 f2000000.ethernet eth0: Link is Up - 2.5Gbps/Full
> - flow control off
> ^^^ 2.5gbe link; ping ok
> # force port to 5gbe
> [ 1060.401209] aqr107_link_change_notify state=5 an=1 link=0 duplex=1
> speed=2500 26:10gbase-r
> [ 1060.401272] mvpp2_port_isr 10gbase-r
> [ 1060.402274] mvpp2_port_disable 10gbase-r
> [ 1060.419006] mvpp2 f2000000.ethernet eth0: Link is Down
> [ 1065.167937] aqr107_read_status state=5 an=1 link=1 duplex=1
> speed=5000 26:10gbase-r
> [ 1065.176865] aqr107_read_rate state=5 an=1 link=1 duplex=1
> speed=5000 26:10gbase-r
> [ 1065.185415] aqr107_link_change_notify state=4 an=1 link=1 duplex=1
> speed=5000 26:10gbase-r
> [ 1065.185456] mvpp2_port_isr 10gbase-r
> [ 1065.185474] mvpp2_port_enable 10gbase-r
> [ 1065.185597] mvpp2 f2000000.ethernet eth0: Link is Up - 5Gbps/Full -
> flow control off
> ^^^ 5gpbe link; ping ok
>
> Thanks,
>
> Tim
>
>> --Sean
>>
>> > [ 197.472504] mvpp2 f2000000.ethernet eth0: major config
>> > [ 197.472614] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
>> > mode=phy//1Gbps/Full/pause adv=00000000,00000000,00000000 pause=00
>> > link=1 an=0
>> > [ 197.479561] aqr107_link_change_notify state=4:running an=1 link=1
>> > duplex=1 speed=1000 0:
>> > [ 197.484972] mvpp2 f2000000.ethernet eth0: Link is Up - 1Gbps/Full -
>> > flow control off
>> > # ethtool eth0
>> > Settings for eth0:
>> > Supported ports: [ ]
>> > Supported link modes: 10baseT/Half 10baseT/Full
>> > 100baseT/Half 100baseT/Full
>> > 1000baseT/Full
>> > 10000baseT/Full
>> > 1000baseKX/Full
>> > 10000baseKX4/Full
>> > 10000baseKR/Full
>> > 2500baseT/Full
>> > 5000baseT/Full
>> > Supported pause frame use: Symmetric Receive-only
>> > Supports auto-negotiation: Yes
>> > Supported FEC modes: Not reported
>> > Advertised link modes: 10baseT/Half 10baseT/Full
>> > 100baseT/Half 100baseT/Full
>> > 1000baseT/Full
>> > 10000baseT/Full
>> > 1000baseKX/Full
>> > 10000baseKX4/Full
>> > 10000baseKR/Full
>> > 2500baseT/Full
>> > 5000baseT/Full
>> > Advertised pause frame use: Symmetric Receive-only
>> > Advertised auto-negotiation: Yes
>> > Advertised FEC modes: Not reported
>> > Link partner advertised link modes: 1000baseT/Half 1000baseT/Full
>> > Link partner advertised pause frame use: No
>> > Link partner advertised auto-negotiation: Yes
>> > Link partner advertised FEC modes: Not reported
>> > Speed: 1000Mb/s
>> > Duplex: Full
>> > Port: Twisted Pair
>> > PHYAD: 8
>> > Transceiver: external
>> > Auto-negotiation: on
>> > MDI-X: Unknown
>> > Link detected: yes
>> > # ping 192.168.1.146 -c5
>> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
>> >
>> > --- 192.168.1.146 ping statistics ---
>> > 5 packets transmitted, 0 packets received, 100% packet loss
>> >
>> > Best Regards,
>> >
>> > Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 22:38 ` Sean Anderson
@ 2022-11-12 0:48 ` Vladimir Oltean
2022-11-12 16:08 ` Andrew Lunn
2022-11-14 15:08 ` Sean Anderson
2022-11-14 19:33 ` Tim Harvey
1 sibling, 2 replies; 21+ messages in thread
From: Vladimir Oltean @ 2022-11-12 0:48 UTC (permalink / raw)
To: Sean Anderson; +Cc: Tim Harvey, netdev, Russell King, David S. Miller
On Fri, Nov 11, 2022 at 05:38:12PM -0500, Sean Anderson wrote:
> > Something interesting is that when I configured the xmdio node with an
> > interrupt I ended up in a mode where 5g,2.5g and 1g all worked for at
> > least 1 test. There was something wrong with my interrupt
> > configuration (i'm not clear if the AQR113C's interrupt should be
> > IRQ_TYPE_LEVEL_LOW, IRQ_TYPE_EDGE_FALLING or something different).
>
> NXP use IRQ_TYPE_LEVEL_HIGH on the LS1046ARDB.
Partly true, but mostly false. What is described in fsl-ls1046a-rdb.dts as:
interrupts = <0 131 4>;
should really have been described as:
interrupts-extended = <&extirq 0 IRQ_TYPE_LEVEL_LOW>;
There's a polarity inverter which inverts the signal by default,
changing what the GIC sees. The first description bypasses it.
So that's not what the problem is in Tim's case.
As to LEVEL_LOW vs EDGE_FALLING, I suppose the only real difference is
if the interrupt line is shared with other peripherals?
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 21:54 ` Sean Anderson
2022-11-11 22:14 ` Tim Harvey
2022-11-11 22:33 ` Russell King (Oracle)
@ 2022-11-12 13:15 ` Russell King (Oracle)
2022-11-14 15:33 ` Sean Anderson
2 siblings, 1 reply; 21+ messages in thread
From: Russell King (Oracle) @ 2022-11-12 13:15 UTC (permalink / raw)
To: Sean Anderson; +Cc: Tim Harvey, netdev, David S. Miller
On Fri, Nov 11, 2022 at 04:54:40PM -0500, Sean Anderson wrote:
> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> > supported 00000000,00018000,000e706f advertising
> > 00000000,00018000,000e706f
> > # ethtool eth0
> > Settings for eth0:
> > Supported ports: [ ]
> > Supported link modes: 10baseT/Half 10baseT/Full
> > 100baseT/Half 100baseT/Full
>
> 10/100 half duplex aren't achievable with rate matching (and we avoid
> turning them on), so they must be coming from somewhere else. I wonder
> if this is because PHY_INTERFACE_MODE_SGMII is set in
> supported_interfaces.
The reason is due to the way phylink_bringup_phy() works. This is
being called with interface = 10GBASE-R, and the PHY is a C45 PHY,
which means we call phy_get_rate_matching() with
PHY_INTERFACE_MODE_NA as we don't know whether the PHY will be
switching its interface or not.
Looking at the Aquanta PHY driver, this will return that pause mode
rate matching will be used, so config.rate_matching will be
RATE_MATCH_PAUSE.
phylink_validate() will be called for PHY_INTERFACE_MODE_NA, which
causes it to scan all supported interface modes (as again, we don't
know which will be used by the PHY [*]) and the union of those
results will be used.
So when we e.g. try SGMII mode, caps & mac_capabilities will allow
the half duplex modes through.
Now for the bit marked with [*] - at this point, if rate matching is
will be used, we in fact know which interface mode is going to be in
operation, and it isn't going to change. So maybe we need this instead
in phylink_bringup_phy():
- if (phy->is_c45 &&
+ config.rate_matching = phy_get_rate_matching(phy, interface);
+ if (phy->is_c45 && config.rate_matching == RATE_MATCH_NONE &&
interface != PHY_INTERFACE_MODE_RXAUI &&
interface != PHY_INTERFACE_MODE_XAUI &&
interface != PHY_INTERFACE_MODE_USXGMII)
config.interface = PHY_INTERFACE_MODE_NA;
else
config.interface = interface;
- config.rate_matching = phy_get_rate_matching(phy, config.interface);
ret = phylink_validate(pl, supported, &config);
?
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-12 0:48 ` Vladimir Oltean
@ 2022-11-12 16:08 ` Andrew Lunn
2022-11-14 15:08 ` Sean Anderson
1 sibling, 0 replies; 21+ messages in thread
From: Andrew Lunn @ 2022-11-12 16:08 UTC (permalink / raw)
To: Vladimir Oltean
Cc: Sean Anderson, Tim Harvey, netdev, Russell King, David S. Miller
> As to LEVEL_LOW vs EDGE_FALLING, I suppose the only real difference is
> if the interrupt line is shared with other peripherals?
It pretty much always is, on the PHY side. The PHY is an interrupt
controller, with lots of different interrupt sources within the PHY
coming together to trigger one external interrupt. It is unlikely to
produce another edge if the hardware has another interrupt source
trigger an interrupt while the interrupt handler is running. With a
level interrupt, the interrupt handler will exit, the interrupt will
get enabled in the parent interrupt controller, and immediately fire
again.
I have seen some boards using edge, but that is only because the
interrupt controller does not support level. They mostly get away with
it because generally PHYs are slow things, interrupts tend to be few
and infrequency and the race window is small.
Andrew
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-12 0:48 ` Vladimir Oltean
2022-11-12 16:08 ` Andrew Lunn
@ 2022-11-14 15:08 ` Sean Anderson
1 sibling, 0 replies; 21+ messages in thread
From: Sean Anderson @ 2022-11-14 15:08 UTC (permalink / raw)
To: Vladimir Oltean; +Cc: Tim Harvey, netdev, Russell King, David S. Miller
On 11/11/22 19:48, Vladimir Oltean wrote:
> On Fri, Nov 11, 2022 at 05:38:12PM -0500, Sean Anderson wrote:
>> > Something interesting is that when I configured the xmdio node with an
>> > interrupt I ended up in a mode where 5g,2.5g and 1g all worked for at
>> > least 1 test. There was something wrong with my interrupt
>> > configuration (i'm not clear if the AQR113C's interrupt should be
>> > IRQ_TYPE_LEVEL_LOW, IRQ_TYPE_EDGE_FALLING or something different).
>>
>> NXP use IRQ_TYPE_LEVEL_HIGH on the LS1046ARDB.
>
> Partly true, but mostly false. What is described in fsl-ls1046a-rdb.dts as:
>
> interrupts = <0 131 4>;
>
> should really have been described as:
>
> interrupts-extended = <&extirq 0 IRQ_TYPE_LEVEL_LOW>;
>
> There's a polarity inverter which inverts the signal by default,
> changing what the GIC sees. The first description bypasses it.
Ah. I missed that they described it as going straight to the GIC, skipping
the extirq. Thanks for pointing that out.
--Sean
> So that's not what the problem is in Tim's case.
>
> As to LEVEL_LOW vs EDGE_FALLING, I suppose the only real difference is
> if the interrupt line is shared with other peripherals?
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-12 13:15 ` Russell King (Oracle)
@ 2022-11-14 15:33 ` Sean Anderson
2022-11-14 16:13 ` Russell King (Oracle)
0 siblings, 1 reply; 21+ messages in thread
From: Sean Anderson @ 2022-11-14 15:33 UTC (permalink / raw)
To: Russell King (Oracle); +Cc: Tim Harvey, netdev, David S. Miller
On 11/12/22 08:15, Russell King (Oracle) wrote:
> On Fri, Nov 11, 2022 at 04:54:40PM -0500, Sean Anderson wrote:
>> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
>> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
>> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
>> > supported 00000000,00018000,000e706f advertising
>> > 00000000,00018000,000e706f
>
>> > # ethtool eth0
>> > Settings for eth0:
>> > Supported ports: [ ]
>> > Supported link modes: 10baseT/Half 10baseT/Full
>> > 100baseT/Half 100baseT/Full
>>
>> 10/100 half duplex aren't achievable with rate matching (and we avoid
>> turning them on), so they must be coming from somewhere else. I wonder
>> if this is because PHY_INTERFACE_MODE_SGMII is set in
>> supported_interfaces.
>
> The reason is due to the way phylink_bringup_phy() works. This is
> being called with interface = 10GBASE-R, and the PHY is a C45 PHY,
> which means we call phy_get_rate_matching() with
> PHY_INTERFACE_MODE_NA as we don't know whether the PHY will be
> switching its interface or not.
>
> Looking at the Aquanta PHY driver, this will return that pause mode
> rate matching will be used, so config.rate_matching will be
> RATE_MATCH_PAUSE.
>
> phylink_validate() will be called for PHY_INTERFACE_MODE_NA, which
> causes it to scan all supported interface modes (as again, we don't
> know which will be used by the PHY [*]) and the union of those
> results will be used.
>
> So when we e.g. try SGMII mode, caps & mac_capabilities will allow
> the half duplex modes through.
>
> Now for the bit marked with [*] - at this point, if rate matching is
> will be used, we in fact know which interface mode is going to be in
> operation, and it isn't going to change. So maybe we need this instead
> in phylink_bringup_phy():
>
> - if (phy->is_c45 &&
> + config.rate_matching = phy_get_rate_matching(phy, interface);
> + if (phy->is_c45 && config.rate_matching == RATE_MATCH_NONE &&
> interface != PHY_INTERFACE_MODE_RXAUI &&
> interface != PHY_INTERFACE_MODE_XAUI &&
> interface != PHY_INTERFACE_MODE_USXGMII)
> config.interface = PHY_INTERFACE_MODE_NA;
> else
> config.interface = interface;
> - config.rate_matching = phy_get_rate_matching(phy, config.interface);
>
> ret = phylink_validate(pl, supported, &config);
>
> ?
Yeah, that sounds reasonable. Actually, this was the logic I was
thinking of when I asked Tim to try USXGMII earlier. The funny thing is
that the comment above this implies that the link mode is never actually
(R)XAUI or USXGMII.
On another subject, if setting the SERDES mode field above fixes the
issue, then the Aquantia driver should be modified to set that field to
use a supported interface. Will host_interfaces work for this? It seems
to be set only when there's an SFP module.
That said, imagine if Tim was using a MAC without pause support, but
which supported SGMII and 10GBASE-R. Currently, we would just advertise
10G modes. But 1G could be supported by switching the phy interface. I
don't think this is supported right now. But if it were, we would need a
way to tell the phy to use a lower phy interface mode, instead of rate
adapting. We might also need a way to let the board tell us what
interfaces are supported (like [1]).
--Sean
[1] https://lore.kernel.org/netdev/20211117225050.18395-1-kabel@kernel.org/
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-14 15:33 ` Sean Anderson
@ 2022-11-14 16:13 ` Russell King (Oracle)
2022-11-18 18:16 ` Sean Anderson
0 siblings, 1 reply; 21+ messages in thread
From: Russell King (Oracle) @ 2022-11-14 16:13 UTC (permalink / raw)
To: Sean Anderson; +Cc: Tim Harvey, netdev, David S. Miller
On Mon, Nov 14, 2022 at 10:33:52AM -0500, Sean Anderson wrote:
> On 11/12/22 08:15, Russell King (Oracle) wrote:
> > On Fri, Nov 11, 2022 at 04:54:40PM -0500, Sean Anderson wrote:
> >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> >> > supported 00000000,00018000,000e706f advertising
> >> > 00000000,00018000,000e706f
> >
> >> > # ethtool eth0
> >> > Settings for eth0:
> >> > Supported ports: [ ]
> >> > Supported link modes: 10baseT/Half 10baseT/Full
> >> > 100baseT/Half 100baseT/Full
> >>
> >> 10/100 half duplex aren't achievable with rate matching (and we avoid
> >> turning them on), so they must be coming from somewhere else. I wonder
> >> if this is because PHY_INTERFACE_MODE_SGMII is set in
> >> supported_interfaces.
> >
> > The reason is due to the way phylink_bringup_phy() works. This is
> > being called with interface = 10GBASE-R, and the PHY is a C45 PHY,
> > which means we call phy_get_rate_matching() with
> > PHY_INTERFACE_MODE_NA as we don't know whether the PHY will be
> > switching its interface or not.
> >
> > Looking at the Aquanta PHY driver, this will return that pause mode
> > rate matching will be used, so config.rate_matching will be
> > RATE_MATCH_PAUSE.
> >
> > phylink_validate() will be called for PHY_INTERFACE_MODE_NA, which
> > causes it to scan all supported interface modes (as again, we don't
> > know which will be used by the PHY [*]) and the union of those
> > results will be used.
> >
> > So when we e.g. try SGMII mode, caps & mac_capabilities will allow
> > the half duplex modes through.
> >
> > Now for the bit marked with [*] - at this point, if rate matching is
> > will be used, we in fact know which interface mode is going to be in
> > operation, and it isn't going to change. So maybe we need this instead
> > in phylink_bringup_phy():
> >
> > - if (phy->is_c45 &&
> > + config.rate_matching = phy_get_rate_matching(phy, interface);
> > + if (phy->is_c45 && config.rate_matching == RATE_MATCH_NONE &&
> > interface != PHY_INTERFACE_MODE_RXAUI &&
> > interface != PHY_INTERFACE_MODE_XAUI &&
> > interface != PHY_INTERFACE_MODE_USXGMII)
> > config.interface = PHY_INTERFACE_MODE_NA;
> > else
> > config.interface = interface;
> > - config.rate_matching = phy_get_rate_matching(phy, config.interface);
> >
> > ret = phylink_validate(pl, supported, &config);
> >
> > ?
>
> Yeah, that sounds reasonable. Actually, this was the logic I was
> thinking of when I asked Tim to try USXGMII earlier. The funny thing is
> that the comment above this implies that the link mode is never actually
> (R)XAUI or USXGMII.
I think you're misunderstanding the comment...
If a clause 45 PHY is using USXGMII, then it is highly likely that the
PHY will not switch between different interface modes depending on the
media side negotiation.
If a clause 45 PHY is using RXAUI or XAUI, then I believe according to
the information available to me at the time, that there is no
possibility of different interface modes being used.
If any other interface type is specified (e.g. 10GBASE-R etc) then there
is the possibility that the PHY will be switching between different
interface modes, and we have no idea what so ever at this point what
modes the PHY will be making use of - so the best we can do is to
validate _all_ possible modes. This is what is done by setting the
interface mode to _NA.
Obviously, if we are using rate matching with a particular interface
mode (e.g. 10GBASE-R) then we know that we are only going to be using
10GBASE-R, so we can validate just the single interface mode.
> On another subject, if setting the SERDES mode field above fixes the
> issue, then the Aquantia driver should be modified to set that field to
> use a supported interface. Will host_interfaces work for this? It seems
> to be set only when there's an SFP module.
The reason I didn't push host_interfaces upstream myself is that I was
unconvinced that it was the proper approach - and I still have my
reservations with it. This can only tell the PHY driver what the MAC
driver supports, and it means the PHY driver is then free to do its
own choosing of what group of interface modes it wants to use.
However, think about what I've said above about phylink not having any
clue about what interface modes the PHY is going to be using - having
the PHY driver decide on its own which group of interface modes should
be used adds even more complexity in a completely different chunk of
code, one where driver authors are free to make whatever decisions
they deem (and we know that wildly different solutions will happen.)
I had been toying with the idea of doing this differently, and had
dropped most of the host_interfaces approach from my git tree, instead
having PHYs provide a bitmap of the interface modes they support and
having them initialise in their config_init function which interface
modes they're going to be making use of given their resulting
configuration. I never properly finished this though.
> That said, imagine if Tim was using a MAC without pause support, but
> which supported SGMII and 10GBASE-R. Currently, we would just advertise
> 10G modes. But 1G could be supported by switching the phy interface.
Note that we already have boards that make use of interface switching.
Macchiatobin has switched between 10GBASE-R, 5GBASE-R, 2500BASE-X and
SGMII depending on the negotiated media speed. In fact, that switching
is rather enforced by the 3310 PHY firmware.
We could force 10GBASE-R and enable rate matching, but then we get
into the problems that the 3310 on these boards does not have MACSEC
therefore can't send pause frames to the host MAC (and the host MAC
doesn't support pause frames - eek) and we have not come up with an
implementation for extending the IPG, although I believe mvpp2
hardware is capable of it.
However, there's also the BCM84881 PHY which does the same dynamic
switching which we can't prevent (we don't know how to!)
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-11 22:38 ` Sean Anderson
2022-11-12 0:48 ` Vladimir Oltean
@ 2022-11-14 19:33 ` Tim Harvey
2022-11-16 22:37 ` Tim Harvey
1 sibling, 1 reply; 21+ messages in thread
From: Tim Harvey @ 2022-11-14 19:33 UTC (permalink / raw)
To: Sean Anderson; +Cc: netdev, Russell King, David S. Miller
On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@seco.com> wrote:
>
> On 11/11/22 17:14, Tim Harvey wrote:
> > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >>
> >> On 11/11/22 16:20, Tim Harvey wrote:
> >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >> >>
> >> >> On 11/11/22 15:57, Sean Anderson wrote:
> >> >> > Hi Tim,
> >> >> >
> >> >> > On 11/11/22 15:44, Tim Harvey wrote:
> >> >> >> Greetings,
> >> >> >>
> >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
> >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
> >> >> >>
> >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
> >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> >> >> >>
> >> >> >> Should I expect this to work now at those lower rates
> >> >> >
> >> >> > Yes.
> >> >
> >> > Sean,
> >> >
> >> > Good to hear - thank you for your work on this feature!
> >> >
> >> >> >
> >> >> >> and if so what kind of debug information or testing can I provide?
> >> >> >
> >> >> > Please send
> >> >> >
> >> >> > - Your test procedure (how do you select 1G?)
> >> >> > - Device tree node for the interface
> >> >> > - Output of ethtool (on both ends if possible).
> >> >> > - Kernel logs with debug enabled for drivers/phylink.c
> >> >>
> >> >> Sorry, this should be drivers/net/phy/phylink.c
> >> >>
> >> >> >
> >> >> > That should be enough to get us started.
> >> >> >
> >> >> > --Sean
> >> >>
> >> >
> >> > I'm currently testing by bringing up the network interface while
> >> > connected to a 10gbe switch, verifying link and traffic, then forcing
> >> > the switch port to 1000mbps.
> >> >
> >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
> >> >
> >> > #include "cn9130.dtsi" /* include SoC device tree */
> >> >
> >> > &cp0_xmdio {
> >> > pinctrl-names = "default";
> >> > pinctrl-0 = <&cp0_xsmi_pins>;
> >> > status = "okay";
> >> >
> >> > phy1: ethernet-phy@8 {
> >> > compatible = "ethernet-phy-ieee802.3-c45";
> >> > reg = <8>;
> >> > };
> >> > };
> >> >
> >> > &cp0_ethernet {
> >> > status = "okay";
> >> > };
> >> >
> >> > /* 10GbE XFI AQR113C */
> >> > &cp0_eth0 {
> >> > status = "okay";
> >> > phy = <&phy1>;
> >> > phy-mode = "10gbase-r";
> >> > phys = <&cp0_comphy4 0>;
> >> > };
> >> >
> >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> >> > some additional debug in mvpp2.c and aquantia_main.c:
> >> > # ifconfig eth0 192.168.1.22
> >> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> >> > speed=-1 26:10gbase-r
> >> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> >> > [ 8.898165] aqr107_resume
> >> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> >> > duplex=255 speed=-1 26:10gbase-r 0:
> >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> >> > supported 00000000,00018000,000e706f advertising
> >> > 00000000,00018000,000e706f
> >> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> >> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> >> > phy/10gbase-r link mode
> >> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> >> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> >> > pause=00 link=0 an=0
> >> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> >> > [ 8.976267] aqr107_resume
> >> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> >> > 10gbase-r/10Gbps/Full/none/off
> >> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> > duplex=1 speed=10000 26:10gbase-r
> >> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> >> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> >> > - flow control off
> >> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> >> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> >> > 10gbase-r/10Gbps/Full/none/off
> >> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> >> > duplex=1 speed=10000 26:10gbase-r
> >> >
> >> > # ethtool eth0
> >> > Settings for eth0:
> >> > Supported ports: [ ]
> >> > Supported link modes: 10baseT/Half 10baseT/Full
> >> > 100baseT/Half 100baseT/Full
> >>
> >> 10/100 half duplex aren't achievable with rate matching (and we avoid
> >> turning them on), so they must be coming from somewhere else. I wonder
> >> if this is because PHY_INTERFACE_MODE_SGMII is set in
> >> supported_interfaces.
> >>
> >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> >> should support it. I'm not sure if the aquantia driver is set up for it.
> >
> > This appears to trigger an issue from mvpp2:
> > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
> > 00000000,00018000,000e706f and advertisement
> > 00000000,00018000,000e706f failed: -EINVAL
>
> Ah, I forgot this was a separate phy mode. Disregard this.
>
> >>
> >> > 1000baseT/Full
> >> > 10000baseT/Full
> >> > 1000baseKX/Full
> >> > 10000baseKX4/Full
> >> > 10000baseKR/Full
> >> > 2500baseT/Full
> >> > 5000baseT/Full
> >> > Supported pause frame use: Symmetric Receive-only
> >> > Supports auto-negotiation: Yes
> >> > Supported FEC modes: Not reported
> >> > Advertised link modes: 10baseT/Half 10baseT/Full
> >> > 100baseT/Half 100baseT/Full
> >> > 1000baseT/Full
> >> > 10000baseT/Full
> >> > 1000baseKX/Full
> >> > 10000baseKX4/Full
> >> > 10000baseKR/Full
> >> > 2500baseT/Full
> >> > 5000baseT/Full
> >> > Advertised pause frame use: Symmetric Receive-only
> >> > Advertised auto-negotiation: Yes
> >> > Advertised FEC modes: Not reported
> >> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
> >> > 1000baseT/Half 1000baseT/Full
> >> > 10000baseT/Full
> >> > 2500baseT/Full
> >> > 5000baseT/Full
> >> > Link partner advertised pause frame use: No
> >> > Link partner advertised auto-negotiation: Yes
> >> > Link partner advertised FEC modes: Not reported
> >> > Speed: 10000Mb/s
> >> > Duplex: Full
> >> > Port: Twisted Pair
> >> > PHYAD: 8
> >> > Transceiver: external
> >> > Auto-negotiation: on
> >> > MDI-X: Unknown
> >> > Link detected: yes
> >> > # ping 192.168.1.146 -c5
> >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
> >> >
> >> > --- 192.168.1.146 ping statistics ---
> >> > 5 packets transmitted, 5 packets received, 0% packet loss
> >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
> >> > # # force switch port to 1G
> >> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> >> > 10gbase-r/Unknown/Unknown/none/off
> >> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> >> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> >> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> > duplex=255 speed=-1 26:10gbase-r
> >> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
> >>
> >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
> >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
> >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
> >> and the vendor provisioning registers (dev 4, reg 0xc440 through
> >> 0xc449).
> >
> > yes, this is what I've been looking at as well. When forced to 1000m
> > the register shows a phy type of 11 which according to the aqr113
> > datasheet is XFI 5G:
> > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
> > link=1 duplex=1 speed=1000 interface=0
>
> That's pretty strange. Seems like it's rate adapting from 5g instead of
> 10g. Is SERDES Mode in the Global System Configuration For 1G register
> set to XFI?
1E.31C=0x0106:
Rate Adaptation Method: 2=Pause Rate Adaptation
SERDES Mode: 6=XFI/2 (XFI 5G)
Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-14 19:33 ` Tim Harvey
@ 2022-11-16 22:37 ` Tim Harvey
2022-11-17 15:38 ` Sean Anderson
0 siblings, 1 reply; 21+ messages in thread
From: Tim Harvey @ 2022-11-16 22:37 UTC (permalink / raw)
To: Sean Anderson; +Cc: netdev, Russell King, David S. Miller
On Mon, Nov 14, 2022 at 11:33 AM Tim Harvey <tharvey@gateworks.com> wrote:
>
> On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >
> > On 11/11/22 17:14, Tim Harvey wrote:
> > > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
> > >>
> > >> On 11/11/22 16:20, Tim Harvey wrote:
> > >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
> > >> >>
> > >> >> On 11/11/22 15:57, Sean Anderson wrote:
> > >> >> > Hi Tim,
> > >> >> >
> > >> >> > On 11/11/22 15:44, Tim Harvey wrote:
> > >> >> >> Greetings,
> > >> >> >>
> > >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
> > >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> > >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> > >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> > >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> > >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
> > >> >> >>
> > >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
> > >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> > >> >> >>
> > >> >> >> Should I expect this to work now at those lower rates
> > >> >> >
> > >> >> > Yes.
> > >> >
> > >> > Sean,
> > >> >
> > >> > Good to hear - thank you for your work on this feature!
> > >> >
> > >> >> >
> > >> >> >> and if so what kind of debug information or testing can I provide?
> > >> >> >
> > >> >> > Please send
> > >> >> >
> > >> >> > - Your test procedure (how do you select 1G?)
> > >> >> > - Device tree node for the interface
> > >> >> > - Output of ethtool (on both ends if possible).
> > >> >> > - Kernel logs with debug enabled for drivers/phylink.c
> > >> >>
> > >> >> Sorry, this should be drivers/net/phy/phylink.c
> > >> >>
> > >> >> >
> > >> >> > That should be enough to get us started.
> > >> >> >
> > >> >> > --Sean
> > >> >>
> > >> >
> > >> > I'm currently testing by bringing up the network interface while
> > >> > connected to a 10gbe switch, verifying link and traffic, then forcing
> > >> > the switch port to 1000mbps.
> > >> >
> > >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
> > >> >
> > >> > #include "cn9130.dtsi" /* include SoC device tree */
> > >> >
> > >> > &cp0_xmdio {
> > >> > pinctrl-names = "default";
> > >> > pinctrl-0 = <&cp0_xsmi_pins>;
> > >> > status = "okay";
> > >> >
> > >> > phy1: ethernet-phy@8 {
> > >> > compatible = "ethernet-phy-ieee802.3-c45";
> > >> > reg = <8>;
> > >> > };
> > >> > };
> > >> >
> > >> > &cp0_ethernet {
> > >> > status = "okay";
> > >> > };
> > >> >
> > >> > /* 10GbE XFI AQR113C */
> > >> > &cp0_eth0 {
> > >> > status = "okay";
> > >> > phy = <&phy1>;
> > >> > phy-mode = "10gbase-r";
> > >> > phys = <&cp0_comphy4 0>;
> > >> > };
> > >> >
> > >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> > >> > some additional debug in mvpp2.c and aquantia_main.c:
> > >> > # ifconfig eth0 192.168.1.22
> > >> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> > >> > speed=-1 26:10gbase-r
> > >> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> > >> > [ 8.898165] aqr107_resume
> > >> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> > >> > duplex=255 speed=-1 26:10gbase-r 0:
> > >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> > >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> > >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> > >> > supported 00000000,00018000,000e706f advertising
> > >> > 00000000,00018000,000e706f
> > >> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> > >> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> > >> > phy/10gbase-r link mode
> > >> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> > >> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> > >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> > >> > pause=00 link=0 an=0
> > >> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> > >> > [ 8.976267] aqr107_resume
> > >> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> > >> > 10gbase-r/10Gbps/Full/none/off
> > >> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> > >> > duplex=1 speed=10000 26:10gbase-r
> > >> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> > >> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> > >> > - flow control off
> > >> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> > >> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> > >> > 10gbase-r/10Gbps/Full/none/off
> > >> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> > >> > duplex=1 speed=10000 26:10gbase-r
> > >> >
> > >> > # ethtool eth0
> > >> > Settings for eth0:
> > >> > Supported ports: [ ]
> > >> > Supported link modes: 10baseT/Half 10baseT/Full
> > >> > 100baseT/Half 100baseT/Full
> > >>
> > >> 10/100 half duplex aren't achievable with rate matching (and we avoid
> > >> turning them on), so they must be coming from somewhere else. I wonder
> > >> if this is because PHY_INTERFACE_MODE_SGMII is set in
> > >> supported_interfaces.
> > >>
> > >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> > >> should support it. I'm not sure if the aquantia driver is set up for it.
> > >
> > > This appears to trigger an issue from mvpp2:
> > > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
> > > 00000000,00018000,000e706f and advertisement
> > > 00000000,00018000,000e706f failed: -EINVAL
> >
> > Ah, I forgot this was a separate phy mode. Disregard this.
> >
> > >>
> > >> > 1000baseT/Full
> > >> > 10000baseT/Full
> > >> > 1000baseKX/Full
> > >> > 10000baseKX4/Full
> > >> > 10000baseKR/Full
> > >> > 2500baseT/Full
> > >> > 5000baseT/Full
> > >> > Supported pause frame use: Symmetric Receive-only
> > >> > Supports auto-negotiation: Yes
> > >> > Supported FEC modes: Not reported
> > >> > Advertised link modes: 10baseT/Half 10baseT/Full
> > >> > 100baseT/Half 100baseT/Full
> > >> > 1000baseT/Full
> > >> > 10000baseT/Full
> > >> > 1000baseKX/Full
> > >> > 10000baseKX4/Full
> > >> > 10000baseKR/Full
> > >> > 2500baseT/Full
> > >> > 5000baseT/Full
> > >> > Advertised pause frame use: Symmetric Receive-only
> > >> > Advertised auto-negotiation: Yes
> > >> > Advertised FEC modes: Not reported
> > >> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
> > >> > 1000baseT/Half 1000baseT/Full
> > >> > 10000baseT/Full
> > >> > 2500baseT/Full
> > >> > 5000baseT/Full
> > >> > Link partner advertised pause frame use: No
> > >> > Link partner advertised auto-negotiation: Yes
> > >> > Link partner advertised FEC modes: Not reported
> > >> > Speed: 10000Mb/s
> > >> > Duplex: Full
> > >> > Port: Twisted Pair
> > >> > PHYAD: 8
> > >> > Transceiver: external
> > >> > Auto-negotiation: on
> > >> > MDI-X: Unknown
> > >> > Link detected: yes
> > >> > # ping 192.168.1.146 -c5
> > >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> > >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> > >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> > >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> > >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> > >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
> > >> >
> > >> > --- 192.168.1.146 ping statistics ---
> > >> > 5 packets transmitted, 5 packets received, 0% packet loss
> > >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
> > >> > # # force switch port to 1G
> > >> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> > >> > 10gbase-r/Unknown/Unknown/none/off
> > >> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> > >> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> > >> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> > >> > duplex=255 speed=-1 26:10gbase-r
> > >> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
> > >>
> > >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
> > >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
> > >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
> > >> and the vendor provisioning registers (dev 4, reg 0xc440 through
> > >> 0xc449).
> > >
> > > yes, this is what I've been looking at as well. When forced to 1000m
> > > the register shows a phy type of 11 which according to the aqr113
> > > datasheet is XFI 5G:
> > > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
> > > link=1 duplex=1 speed=1000 interface=0
> >
> > That's pretty strange. Seems like it's rate adapting from 5g instead of
> > 10g. Is SERDES Mode in the Global System Configuration For 1G register
> > set to XFI?
>
> 1E.31C=0x0106:
> Rate Adaptation Method: 2=Pause Rate Adaptation
> SERDES Mode: 6=XFI/2 (XFI 5G)
>
The SERDES mode here is not valid and it seems to always be set to
XFI/2 unless I init/use the AQR113 in U-Boot. If I manually set SERDES
Mode to 0 (XFI) in the driver I find that all rates
10g,5g,2.5g,1g,100m work fine both in Linux 6.0 and in Linux 6.1-rc5.
I'm still trying to understand why I would need to set SERDES Mode
manually (vs the XFI mode specific firmware setting this) but I am
unclear what the rate adaptation in 6.1 provides in this case. Is it
that perhaps the AQR113 is handling rate adaptation internally and
that's why the non 10gbe rates are working on 6.0?
Best Regards,
Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-16 22:37 ` Tim Harvey
@ 2022-11-17 15:38 ` Sean Anderson
2022-11-17 23:42 ` Tim Harvey
0 siblings, 1 reply; 21+ messages in thread
From: Sean Anderson @ 2022-11-17 15:38 UTC (permalink / raw)
To: Tim Harvey; +Cc: netdev, Russell King, David S. Miller
On 11/16/22 17:37, Tim Harvey wrote:
> On Mon, Nov 14, 2022 at 11:33 AM Tim Harvey <tharvey@gateworks.com> wrote:
>>
>> On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@seco.com> wrote:
>> >
>> > On 11/11/22 17:14, Tim Harvey wrote:
>> > > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
>> > >>
>> > >> On 11/11/22 16:20, Tim Harvey wrote:
>> > >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
>> > >> >>
>> > >> >> On 11/11/22 15:57, Sean Anderson wrote:
>> > >> >> > Hi Tim,
>> > >> >> >
>> > >> >> > On 11/11/22 15:44, Tim Harvey wrote:
>> > >> >> >> Greetings,
>> > >> >> >>
>> > >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
>> > >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
>> > >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
>> > >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
>> > >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
>> > >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
>> > >> >> >>
>> > >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
>> > >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
>> > >> >> >>
>> > >> >> >> Should I expect this to work now at those lower rates
>> > >> >> >
>> > >> >> > Yes.
>> > >> >
>> > >> > Sean,
>> > >> >
>> > >> > Good to hear - thank you for your work on this feature!
>> > >> >
>> > >> >> >
>> > >> >> >> and if so what kind of debug information or testing can I provide?
>> > >> >> >
>> > >> >> > Please send
>> > >> >> >
>> > >> >> > - Your test procedure (how do you select 1G?)
>> > >> >> > - Device tree node for the interface
>> > >> >> > - Output of ethtool (on both ends if possible).
>> > >> >> > - Kernel logs with debug enabled for drivers/phylink.c
>> > >> >>
>> > >> >> Sorry, this should be drivers/net/phy/phylink.c
>> > >> >>
>> > >> >> >
>> > >> >> > That should be enough to get us started.
>> > >> >> >
>> > >> >> > --Sean
>> > >> >>
>> > >> >
>> > >> > I'm currently testing by bringing up the network interface while
>> > >> > connected to a 10gbe switch, verifying link and traffic, then forcing
>> > >> > the switch port to 1000mbps.
>> > >> >
>> > >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
>> > >> >
>> > >> > #include "cn9130.dtsi" /* include SoC device tree */
>> > >> >
>> > >> > &cp0_xmdio {
>> > >> > pinctrl-names = "default";
>> > >> > pinctrl-0 = <&cp0_xsmi_pins>;
>> > >> > status = "okay";
>> > >> >
>> > >> > phy1: ethernet-phy@8 {
>> > >> > compatible = "ethernet-phy-ieee802.3-c45";
>> > >> > reg = <8>;
>> > >> > };
>> > >> > };
>> > >> >
>> > >> > &cp0_ethernet {
>> > >> > status = "okay";
>> > >> > };
>> > >> >
>> > >> > /* 10GbE XFI AQR113C */
>> > >> > &cp0_eth0 {
>> > >> > status = "okay";
>> > >> > phy = <&phy1>;
>> > >> > phy-mode = "10gbase-r";
>> > >> > phys = <&cp0_comphy4 0>;
>> > >> > };
>> > >> >
>> > >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
>> > >> > some additional debug in mvpp2.c and aquantia_main.c:
>> > >> > # ifconfig eth0 192.168.1.22
>> > >> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
>> > >> > speed=-1 26:10gbase-r
>> > >> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
>> > >> > [ 8.898165] aqr107_resume
>> > >> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
>> > >> > duplex=255 speed=-1 26:10gbase-r 0:
>> > >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
>> > >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
>> > >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
>> > >> > supported 00000000,00018000,000e706f advertising
>> > >> > 00000000,00018000,000e706f
>> > >> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
>> > >> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
>> > >> > phy/10gbase-r link mode
>> > >> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
>> > >> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
>> > >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
>> > >> > pause=00 link=0 an=0
>> > >> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
>> > >> > [ 8.976267] aqr107_resume
>> > >> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
>> > >> > 10gbase-r/10Gbps/Full/none/off
>> > >> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
>> > >> > duplex=1 speed=10000 26:10gbase-r
>> > >> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
>> > >> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
>> > >> > - flow control off
>> > >> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
>> > >> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
>> > >> > 10gbase-r/10Gbps/Full/none/off
>> > >> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
>> > >> > duplex=1 speed=10000 26:10gbase-r
>> > >> >
>> > >> > # ethtool eth0
>> > >> > Settings for eth0:
>> > >> > Supported ports: [ ]
>> > >> > Supported link modes: 10baseT/Half 10baseT/Full
>> > >> > 100baseT/Half 100baseT/Full
>> > >>
>> > >> 10/100 half duplex aren't achievable with rate matching (and we avoid
>> > >> turning them on), so they must be coming from somewhere else. I wonder
>> > >> if this is because PHY_INTERFACE_MODE_SGMII is set in
>> > >> supported_interfaces.
>> > >>
>> > >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
>> > >> should support it. I'm not sure if the aquantia driver is set up for it.
>> > >
>> > > This appears to trigger an issue from mvpp2:
>> > > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
>> > > 00000000,00018000,000e706f and advertisement
>> > > 00000000,00018000,000e706f failed: -EINVAL
>> >
>> > Ah, I forgot this was a separate phy mode. Disregard this.
>> >
>> > >>
>> > >> > 1000baseT/Full
>> > >> > 10000baseT/Full
>> > >> > 1000baseKX/Full
>> > >> > 10000baseKX4/Full
>> > >> > 10000baseKR/Full
>> > >> > 2500baseT/Full
>> > >> > 5000baseT/Full
>> > >> > Supported pause frame use: Symmetric Receive-only
>> > >> > Supports auto-negotiation: Yes
>> > >> > Supported FEC modes: Not reported
>> > >> > Advertised link modes: 10baseT/Half 10baseT/Full
>> > >> > 100baseT/Half 100baseT/Full
>> > >> > 1000baseT/Full
>> > >> > 10000baseT/Full
>> > >> > 1000baseKX/Full
>> > >> > 10000baseKX4/Full
>> > >> > 10000baseKR/Full
>> > >> > 2500baseT/Full
>> > >> > 5000baseT/Full
>> > >> > Advertised pause frame use: Symmetric Receive-only
>> > >> > Advertised auto-negotiation: Yes
>> > >> > Advertised FEC modes: Not reported
>> > >> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
>> > >> > 1000baseT/Half 1000baseT/Full
>> > >> > 10000baseT/Full
>> > >> > 2500baseT/Full
>> > >> > 5000baseT/Full
>> > >> > Link partner advertised pause frame use: No
>> > >> > Link partner advertised auto-negotiation: Yes
>> > >> > Link partner advertised FEC modes: Not reported
>> > >> > Speed: 10000Mb/s
>> > >> > Duplex: Full
>> > >> > Port: Twisted Pair
>> > >> > PHYAD: 8
>> > >> > Transceiver: external
>> > >> > Auto-negotiation: on
>> > >> > MDI-X: Unknown
>> > >> > Link detected: yes
>> > >> > # ping 192.168.1.146 -c5
>> > >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
>> > >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
>> > >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
>> > >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
>> > >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
>> > >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
>> > >> >
>> > >> > --- 192.168.1.146 ping statistics ---
>> > >> > 5 packets transmitted, 5 packets received, 0% packet loss
>> > >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
>> > >> > # # force switch port to 1G
>> > >> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
>> > >> > 10gbase-r/Unknown/Unknown/none/off
>> > >> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
>> > >> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
>> > >> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
>> > >> > duplex=255 speed=-1 26:10gbase-r
>> > >> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
>> > >>
>> > >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
>> > >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
>> > >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
>> > >> and the vendor provisioning registers (dev 4, reg 0xc440 through
>> > >> 0xc449).
>> > >
>> > > yes, this is what I've been looking at as well. When forced to 1000m
>> > > the register shows a phy type of 11 which according to the aqr113
>> > > datasheet is XFI 5G:
>> > > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
>> > > link=1 duplex=1 speed=1000 interface=0
>> >
>> > That's pretty strange. Seems like it's rate adapting from 5g instead of
>> > 10g. Is SERDES Mode in the Global System Configuration For 1G register
>> > set to XFI?
>>
>> 1E.31C=0x0106:
>> Rate Adaptation Method: 2=Pause Rate Adaptation
>> SERDES Mode: 6=XFI/2 (XFI 5G)
>>
>
> The SERDES mode here is not valid and it seems to always be set to
> XFI/2 unless I init/use the AQR113 in U-Boot. If I manually set SERDES
> Mode to 0 (XFI) in the driver I find that all rates
> 10g,5g,2.5g,1g,100m work fine both in Linux 6.0 and in Linux 6.1-rc5.
> I'm still trying to understand why I would need to set SERDES Mode
> manually (vs the XFI mode specific firmware setting this) but I am
> unclear what the rate adaptation in 6.1 provides in this case. Is it
> that perhaps the AQR113 is handling rate adaptation internally and
> that's why the non 10gbe rates are working on 6.0?
The changes in 6.1 are
- We now always enable pause frame reception when doing rate adaptation.
This is necessary for rate adaptation to work, but wasn't done
automatically before.
- We advertise lower speed modes which are enabled with rate adaptation,
even if we would not otherwise be able to support them.
I'm not sure why you'd have XFI/2 selected in 6.1 if it isn't selected
in 6.0.
--Sean
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-17 15:38 ` Sean Anderson
@ 2022-11-17 23:42 ` Tim Harvey
2022-11-28 19:57 ` Sean Anderson
0 siblings, 1 reply; 21+ messages in thread
From: Tim Harvey @ 2022-11-17 23:42 UTC (permalink / raw)
To: Sean Anderson; +Cc: netdev, Russell King, David S. Miller
On Thu, Nov 17, 2022 at 7:38 AM Sean Anderson <sean.anderson@seco.com> wrote:
>
> On 11/16/22 17:37, Tim Harvey wrote:
> > On Mon, Nov 14, 2022 at 11:33 AM Tim Harvey <tharvey@gateworks.com> wrote:
> >>
> >> On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >> >
> >> > On 11/11/22 17:14, Tim Harvey wrote:
> >> > > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >> > >>
> >> > >> On 11/11/22 16:20, Tim Harvey wrote:
> >> > >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >> > >> >>
> >> > >> >> On 11/11/22 15:57, Sean Anderson wrote:
> >> > >> >> > Hi Tim,
> >> > >> >> >
> >> > >> >> > On 11/11/22 15:44, Tim Harvey wrote:
> >> > >> >> >> Greetings,
> >> > >> >> >>
> >> > >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
> >> > >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> >> > >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> >> > >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> >> > >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> >> > >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
> >> > >> >> >>
> >> > >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
> >> > >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> >> > >> >> >>
> >> > >> >> >> Should I expect this to work now at those lower rates
> >> > >> >> >
> >> > >> >> > Yes.
> >> > >> >
> >> > >> > Sean,
> >> > >> >
> >> > >> > Good to hear - thank you for your work on this feature!
> >> > >> >
> >> > >> >> >
> >> > >> >> >> and if so what kind of debug information or testing can I provide?
> >> > >> >> >
> >> > >> >> > Please send
> >> > >> >> >
> >> > >> >> > - Your test procedure (how do you select 1G?)
> >> > >> >> > - Device tree node for the interface
> >> > >> >> > - Output of ethtool (on both ends if possible).
> >> > >> >> > - Kernel logs with debug enabled for drivers/phylink.c
> >> > >> >>
> >> > >> >> Sorry, this should be drivers/net/phy/phylink.c
> >> > >> >>
> >> > >> >> >
> >> > >> >> > That should be enough to get us started.
> >> > >> >> >
> >> > >> >> > --Sean
> >> > >> >>
> >> > >> >
> >> > >> > I'm currently testing by bringing up the network interface while
> >> > >> > connected to a 10gbe switch, verifying link and traffic, then forcing
> >> > >> > the switch port to 1000mbps.
> >> > >> >
> >> > >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
> >> > >> >
> >> > >> > #include "cn9130.dtsi" /* include SoC device tree */
> >> > >> >
> >> > >> > &cp0_xmdio {
> >> > >> > pinctrl-names = "default";
> >> > >> > pinctrl-0 = <&cp0_xsmi_pins>;
> >> > >> > status = "okay";
> >> > >> >
> >> > >> > phy1: ethernet-phy@8 {
> >> > >> > compatible = "ethernet-phy-ieee802.3-c45";
> >> > >> > reg = <8>;
> >> > >> > };
> >> > >> > };
> >> > >> >
> >> > >> > &cp0_ethernet {
> >> > >> > status = "okay";
> >> > >> > };
> >> > >> >
> >> > >> > /* 10GbE XFI AQR113C */
> >> > >> > &cp0_eth0 {
> >> > >> > status = "okay";
> >> > >> > phy = <&phy1>;
> >> > >> > phy-mode = "10gbase-r";
> >> > >> > phys = <&cp0_comphy4 0>;
> >> > >> > };
> >> > >> >
> >> > >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> >> > >> > some additional debug in mvpp2.c and aquantia_main.c:
> >> > >> > # ifconfig eth0 192.168.1.22
> >> > >> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> >> > >> > speed=-1 26:10gbase-r
> >> > >> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> >> > >> > [ 8.898165] aqr107_resume
> >> > >> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> >> > >> > duplex=255 speed=-1 26:10gbase-r 0:
> >> > >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> >> > >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> >> > >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> >> > >> > supported 00000000,00018000,000e706f advertising
> >> > >> > 00000000,00018000,000e706f
> >> > >> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> >> > >> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> >> > >> > phy/10gbase-r link mode
> >> > >> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> >> > >> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> >> > >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> >> > >> > pause=00 link=0 an=0
> >> > >> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> >> > >> > [ 8.976267] aqr107_resume
> >> > >> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> >> > >> > 10gbase-r/10Gbps/Full/none/off
> >> > >> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> > >> > duplex=1 speed=10000 26:10gbase-r
> >> > >> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> >> > >> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> >> > >> > - flow control off
> >> > >> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> >> > >> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> >> > >> > 10gbase-r/10Gbps/Full/none/off
> >> > >> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> >> > >> > duplex=1 speed=10000 26:10gbase-r
> >> > >> >
> >> > >> > # ethtool eth0
> >> > >> > Settings for eth0:
> >> > >> > Supported ports: [ ]
> >> > >> > Supported link modes: 10baseT/Half 10baseT/Full
> >> > >> > 100baseT/Half 100baseT/Full
> >> > >>
> >> > >> 10/100 half duplex aren't achievable with rate matching (and we avoid
> >> > >> turning them on), so they must be coming from somewhere else. I wonder
> >> > >> if this is because PHY_INTERFACE_MODE_SGMII is set in
> >> > >> supported_interfaces.
> >> > >>
> >> > >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> >> > >> should support it. I'm not sure if the aquantia driver is set up for it.
> >> > >
> >> > > This appears to trigger an issue from mvpp2:
> >> > > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
> >> > > 00000000,00018000,000e706f and advertisement
> >> > > 00000000,00018000,000e706f failed: -EINVAL
> >> >
> >> > Ah, I forgot this was a separate phy mode. Disregard this.
> >> >
> >> > >>
> >> > >> > 1000baseT/Full
> >> > >> > 10000baseT/Full
> >> > >> > 1000baseKX/Full
> >> > >> > 10000baseKX4/Full
> >> > >> > 10000baseKR/Full
> >> > >> > 2500baseT/Full
> >> > >> > 5000baseT/Full
> >> > >> > Supported pause frame use: Symmetric Receive-only
> >> > >> > Supports auto-negotiation: Yes
> >> > >> > Supported FEC modes: Not reported
> >> > >> > Advertised link modes: 10baseT/Half 10baseT/Full
> >> > >> > 100baseT/Half 100baseT/Full
> >> > >> > 1000baseT/Full
> >> > >> > 10000baseT/Full
> >> > >> > 1000baseKX/Full
> >> > >> > 10000baseKX4/Full
> >> > >> > 10000baseKR/Full
> >> > >> > 2500baseT/Full
> >> > >> > 5000baseT/Full
> >> > >> > Advertised pause frame use: Symmetric Receive-only
> >> > >> > Advertised auto-negotiation: Yes
> >> > >> > Advertised FEC modes: Not reported
> >> > >> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
> >> > >> > 1000baseT/Half 1000baseT/Full
> >> > >> > 10000baseT/Full
> >> > >> > 2500baseT/Full
> >> > >> > 5000baseT/Full
> >> > >> > Link partner advertised pause frame use: No
> >> > >> > Link partner advertised auto-negotiation: Yes
> >> > >> > Link partner advertised FEC modes: Not reported
> >> > >> > Speed: 10000Mb/s
> >> > >> > Duplex: Full
> >> > >> > Port: Twisted Pair
> >> > >> > PHYAD: 8
> >> > >> > Transceiver: external
> >> > >> > Auto-negotiation: on
> >> > >> > MDI-X: Unknown
> >> > >> > Link detected: yes
> >> > >> > # ping 192.168.1.146 -c5
> >> > >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> >> > >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> >> > >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
> >> > >> >
> >> > >> > --- 192.168.1.146 ping statistics ---
> >> > >> > 5 packets transmitted, 5 packets received, 0% packet loss
> >> > >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
> >> > >> > # # force switch port to 1G
> >> > >> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> >> > >> > 10gbase-r/Unknown/Unknown/none/off
> >> > >> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> >> > >> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> >> > >> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> > >> > duplex=255 speed=-1 26:10gbase-r
> >> > >> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
> >> > >>
> >> > >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
> >> > >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
> >> > >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
> >> > >> and the vendor provisioning registers (dev 4, reg 0xc440 through
> >> > >> 0xc449).
> >> > >
> >> > > yes, this is what I've been looking at as well. When forced to 1000m
> >> > > the register shows a phy type of 11 which according to the aqr113
> >> > > datasheet is XFI 5G:
> >> > > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
> >> > > link=1 duplex=1 speed=1000 interface=0
> >> >
> >> > That's pretty strange. Seems like it's rate adapting from 5g instead of
> >> > 10g. Is SERDES Mode in the Global System Configuration For 1G register
> >> > set to XFI?
> >>
> >> 1E.31C=0x0106:
> >> Rate Adaptation Method: 2=Pause Rate Adaptation
> >> SERDES Mode: 6=XFI/2 (XFI 5G)
> >>
> >
> > The SERDES mode here is not valid and it seems to always be set to
> > XFI/2 unless I init/use the AQR113 in U-Boot. If I manually set SERDES
> > Mode to 0 (XFI) in the driver I find that all rates
> > 10g,5g,2.5g,1g,100m work fine both in Linux 6.0 and in Linux 6.1-rc5.
> > I'm still trying to understand why I would need to set SERDES Mode
> > manually (vs the XFI mode specific firmware setting this) but I am
> > unclear what the rate adaptation in 6.1 provides in this case. Is it
> > that perhaps the AQR113 is handling rate adaptation internally and
> > that's why the non 10gbe rates are working on 6.0?
>
> The changes in 6.1 are
>
> - We now always enable pause frame reception when doing rate adaptation.
> This is necessary for rate adaptation to work, but wasn't done
> automatically before.
> - We advertise lower speed modes which are enabled with rate adaptation,
> even if we would not otherwise be able to support them.
>
> I'm not sure why you'd have XFI/2 selected in 6.1 if it isn't selected
> in 6.0.
Sean,
Thanks for the explanation. The issue I had which resulted in the
wrong SERDES mode was simply that I was using the wrong aquantia
firmware. They customize the firmware to default registers like SERDES
mode specifically for customers and I was unknowingly using the
firmware for XFI/2 instead of XFI.
I suppose it would be worth putting something in the aquantia driver
that verifies SERDES mode matches the phy-mode from dt to throw an
error.
Best Regards,
Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-14 16:13 ` Russell King (Oracle)
@ 2022-11-18 18:16 ` Sean Anderson
0 siblings, 0 replies; 21+ messages in thread
From: Sean Anderson @ 2022-11-18 18:16 UTC (permalink / raw)
To: Russell King (Oracle); +Cc: Tim Harvey, netdev, David S. Miller
On 11/14/22 11:13, Russell King (Oracle) wrote:
> On Mon, Nov 14, 2022 at 10:33:52AM -0500, Sean Anderson wrote:
>> On 11/12/22 08:15, Russell King (Oracle) wrote:
>> > On Fri, Nov 11, 2022 at 04:54:40PM -0500, Sean Anderson wrote:
>> >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
>> >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
>> >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
>> >> > supported 00000000,00018000,000e706f advertising
>> >> > 00000000,00018000,000e706f
>> >
>> >> > # ethtool eth0
>> >> > Settings for eth0:
>> >> > Supported ports: [ ]
>> >> > Supported link modes: 10baseT/Half 10baseT/Full
>> >> > 100baseT/Half 100baseT/Full
>> >>
>> >> 10/100 half duplex aren't achievable with rate matching (and we avoid
>> >> turning them on), so they must be coming from somewhere else. I wonder
>> >> if this is because PHY_INTERFACE_MODE_SGMII is set in
>> >> supported_interfaces.
>> >
>> > The reason is due to the way phylink_bringup_phy() works. This is
>> > being called with interface = 10GBASE-R, and the PHY is a C45 PHY,
>> > which means we call phy_get_rate_matching() with
>> > PHY_INTERFACE_MODE_NA as we don't know whether the PHY will be
>> > switching its interface or not.
>> >
>> > Looking at the Aquanta PHY driver, this will return that pause mode
>> > rate matching will be used, so config.rate_matching will be
>> > RATE_MATCH_PAUSE.
>> >
>> > phylink_validate() will be called for PHY_INTERFACE_MODE_NA, which
>> > causes it to scan all supported interface modes (as again, we don't
>> > know which will be used by the PHY [*]) and the union of those
>> > results will be used.
>> >
>> > So when we e.g. try SGMII mode, caps & mac_capabilities will allow
>> > the half duplex modes through.
>> >
>> > Now for the bit marked with [*] - at this point, if rate matching is
>> > will be used, we in fact know which interface mode is going to be in
>> > operation, and it isn't going to change. So maybe we need this instead
>> > in phylink_bringup_phy():
>> >
>> > - if (phy->is_c45 &&
>> > + config.rate_matching = phy_get_rate_matching(phy, interface);
>> > + if (phy->is_c45 && config.rate_matching == RATE_MATCH_NONE &&
>> > interface != PHY_INTERFACE_MODE_RXAUI &&
>> > interface != PHY_INTERFACE_MODE_XAUI &&
>> > interface != PHY_INTERFACE_MODE_USXGMII)
>> > config.interface = PHY_INTERFACE_MODE_NA;
>> > else
>> > config.interface = interface;
>> > - config.rate_matching = phy_get_rate_matching(phy, config.interface);
>> >
>> > ret = phylink_validate(pl, supported, &config);
>> >
>> > ?
>>
>> Yeah, that sounds reasonable. Actually, this was the logic I was
>> thinking of when I asked Tim to try USXGMII earlier. The funny thing is
>> that the comment above this implies that the link mode is never actually
>> (R)XAUI or USXGMII.
>
> I think you're misunderstanding the comment...
>
> If a clause 45 PHY is using USXGMII, then it is highly likely that the
> PHY will not switch between different interface modes depending on the
> media side negotiation.
>
> If a clause 45 PHY is using RXAUI or XAUI, then I believe according to
> the information available to me at the time, that there is no
> possibility of different interface modes being used.
>
> If any other interface type is specified (e.g. 10GBASE-R etc) then there
> is the possibility that the PHY will be switching between different
> interface modes, and we have no idea what so ever at this point what
> modes the PHY will be making use of - so the best we can do is to
> validate _all_ possible modes. This is what is done by setting the
> interface mode to _NA.
>
> Obviously, if we are using rate matching with a particular interface
> mode (e.g. 10GBASE-R) then we know that we are only going to be using
> 10GBASE-R, so we can validate just the single interface mode.
Ah, you're right, I was reading this backwards.
>> On another subject, if setting the SERDES mode field above fixes the
>> issue, then the Aquantia driver should be modified to set that field to
>> use a supported interface. Will host_interfaces work for this? It seems
>> to be set only when there's an SFP module.
>
> The reason I didn't push host_interfaces upstream myself is that I was
> unconvinced that it was the proper approach - and I still have my
> reservations with it. This can only tell the PHY driver what the MAC
> driver supports, and it means the PHY driver is then free to do its
> own choosing of what group of interface modes it wants to use.
Well, this is what we have already. The Aquantia phys are initialized by
the firmware to use particular interface for a particular link speed.
Rate adaptation may or may not be involved. If it picks an interface the
MAC doesn't support, you're SOL (until you get a new firmware). Ideally,
we'd be able to configure the phy to always select an interface the MAC
supports.
> However, think about what I've said above about phylink not having any
> clue about what interface modes the PHY is going to be using - having
> the PHY driver decide on its own which group of interface modes should
> be used adds even more complexity in a completely different chunk of
> code, one where driver authors are free to make whatever decisions
> they deem (and we know that wildly different solutions will happen.)
>
> I had been toying with the idea of doing this differently, and had
> dropped most of the host_interfaces approach from my git tree, instead
> having PHYs provide a bitmap of the interface modes they support and
> having them initialise in their config_init function which interface
> modes they're going to be making use of given their resulting
> configuration. I never properly finished this though.
That sounds like a good start.
>> That said, imagine if Tim was using a MAC without pause support, but
>> which supported SGMII and 10GBASE-R. Currently, we would just advertise
>> 10G modes. But 1G could be supported by switching the phy interface.
>
> Note that we already have boards that make use of interface switching.
> Macchiatobin has switched between 10GBASE-R, 5GBASE-R, 2500BASE-X and
> SGMII depending on the negotiated media speed. In fact, that switching
> is rather enforced by the 3310 PHY firmware.
>
> We could force 10GBASE-R and enable rate matching, but then we get
> into the problems that the 3310 on these boards does not have MACSEC
> therefore can't send pause frames to the host MAC (and the host MAC
> doesn't support pause frames - eek) and we have not come up with an
> implementation for extending the IPG, although I believe mvpp2
> hardware is capable of it.
The DPAA hardware is as well. As I understand it, the 10GBASE-W standard
specifies linear scaling of the IPG with the packet size, not A simple
implementation could be to have MACs expose something like
mac_rate_match_ipg_numerator,
mac_rate_match_ipg_denominator,
With the intention that they'd do something like
numerator = SPEED_10000,
denominator = 9500,
and then we could multiply the speed to adjust. We could also do
something like
int mac_minimum_speed(int base_speed);
But AIUI we are trying to move away from these sorts of things.
FWIW I don't have any 10GBASE-W hardware.
--Sean
> However, there's also the BCM84881 PHY which does the same dynamic
> switching which we can't prevent (we don't know how to!)
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-17 23:42 ` Tim Harvey
@ 2022-11-28 19:57 ` Sean Anderson
2022-12-01 1:11 ` Tim Harvey
0 siblings, 1 reply; 21+ messages in thread
From: Sean Anderson @ 2022-11-28 19:57 UTC (permalink / raw)
To: Tim Harvey; +Cc: netdev, Russell King, David S. Miller
Hi Tim,
On 11/17/22 18:42, Tim Harvey wrote:
> On Thu, Nov 17, 2022 at 7:38 AM Sean Anderson <sean.anderson@seco.com> wrote:
>>
>> On 11/16/22 17:37, Tim Harvey wrote:
>> > On Mon, Nov 14, 2022 at 11:33 AM Tim Harvey <tharvey@gateworks.com> wrote:
>> >>
>> >> On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@seco.com> wrote:
>> >> >
>> >> > On 11/11/22 17:14, Tim Harvey wrote:
>> >> > > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
>> >> > >>
>> >> > >> On 11/11/22 16:20, Tim Harvey wrote:
>> >> > >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
>> >> > >> >>
>> >> > >> >> On 11/11/22 15:57, Sean Anderson wrote:
>> >> > >> >> > Hi Tim,
>> >> > >> >> >
>> >> > >> >> > On 11/11/22 15:44, Tim Harvey wrote:
>> >> > >> >> >> Greetings,
>> >> > >> >> >>
>> >> > >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
>> >> > >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
>> >> > >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
>> >> > >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
>> >> > >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
>> >> > >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
>> >> > >> >> >>
>> >> > >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
>> >> > >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
>> >> > >> >> >>
>> >> > >> >> >> Should I expect this to work now at those lower rates
>> >> > >> >> >
>> >> > >> >> > Yes.
>> >> > >> >
>> >> > >> > Sean,
>> >> > >> >
>> >> > >> > Good to hear - thank you for your work on this feature!
>> >> > >> >
>> >> > >> >> >
>> >> > >> >> >> and if so what kind of debug information or testing can I provide?
>> >> > >> >> >
>> >> > >> >> > Please send
>> >> > >> >> >
>> >> > >> >> > - Your test procedure (how do you select 1G?)
>> >> > >> >> > - Device tree node for the interface
>> >> > >> >> > - Output of ethtool (on both ends if possible).
>> >> > >> >> > - Kernel logs with debug enabled for drivers/phylink.c
>> >> > >> >>
>> >> > >> >> Sorry, this should be drivers/net/phy/phylink.c
>> >> > >> >>
>> >> > >> >> >
>> >> > >> >> > That should be enough to get us started.
>> >> > >> >> >
>> >> > >> >> > --Sean
>> >> > >> >>
>> >> > >> >
>> >> > >> > I'm currently testing by bringing up the network interface while
>> >> > >> > connected to a 10gbe switch, verifying link and traffic, then forcing
>> >> > >> > the switch port to 1000mbps.
>> >> > >> >
>> >> > >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
>> >> > >> >
>> >> > >> > #include "cn9130.dtsi" /* include SoC device tree */
>> >> > >> >
>> >> > >> > &cp0_xmdio {
>> >> > >> > pinctrl-names = "default";
>> >> > >> > pinctrl-0 = <&cp0_xsmi_pins>;
>> >> > >> > status = "okay";
>> >> > >> >
>> >> > >> > phy1: ethernet-phy@8 {
>> >> > >> > compatible = "ethernet-phy-ieee802.3-c45";
>> >> > >> > reg = <8>;
>> >> > >> > };
>> >> > >> > };
>> >> > >> >
>> >> > >> > &cp0_ethernet {
>> >> > >> > status = "okay";
>> >> > >> > };
>> >> > >> >
>> >> > >> > /* 10GbE XFI AQR113C */
>> >> > >> > &cp0_eth0 {
>> >> > >> > status = "okay";
>> >> > >> > phy = <&phy1>;
>> >> > >> > phy-mode = "10gbase-r";
>> >> > >> > phys = <&cp0_comphy4 0>;
>> >> > >> > };
>> >> > >> >
>> >> > >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
>> >> > >> > some additional debug in mvpp2.c and aquantia_main.c:
>> >> > >> > # ifconfig eth0 192.168.1.22
>> >> > >> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
>> >> > >> > speed=-1 26:10gbase-r
>> >> > >> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
>> >> > >> > [ 8.898165] aqr107_resume
>> >> > >> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
>> >> > >> > duplex=255 speed=-1 26:10gbase-r 0:
>> >> > >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
>> >> > >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
>> >> > >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
>> >> > >> > supported 00000000,00018000,000e706f advertising
>> >> > >> > 00000000,00018000,000e706f
>> >> > >> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
>> >> > >> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
>> >> > >> > phy/10gbase-r link mode
>> >> > >> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
>> >> > >> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
>> >> > >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
>> >> > >> > pause=00 link=0 an=0
>> >> > >> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
>> >> > >> > [ 8.976267] aqr107_resume
>> >> > >> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
>> >> > >> > 10gbase-r/10Gbps/Full/none/off
>> >> > >> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
>> >> > >> > duplex=1 speed=10000 26:10gbase-r
>> >> > >> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
>> >> > >> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
>> >> > >> > - flow control off
>> >> > >> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
>> >> > >> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
>> >> > >> > 10gbase-r/10Gbps/Full/none/off
>> >> > >> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
>> >> > >> > duplex=1 speed=10000 26:10gbase-r
>> >> > >> >
>> >> > >> > # ethtool eth0
>> >> > >> > Settings for eth0:
>> >> > >> > Supported ports: [ ]
>> >> > >> > Supported link modes: 10baseT/Half 10baseT/Full
>> >> > >> > 100baseT/Half 100baseT/Full
>> >> > >>
>> >> > >> 10/100 half duplex aren't achievable with rate matching (and we avoid
>> >> > >> turning them on), so they must be coming from somewhere else. I wonder
>> >> > >> if this is because PHY_INTERFACE_MODE_SGMII is set in
>> >> > >> supported_interfaces.
>> >> > >>
>> >> > >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
>> >> > >> should support it. I'm not sure if the aquantia driver is set up for it.
>> >> > >
>> >> > > This appears to trigger an issue from mvpp2:
>> >> > > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
>> >> > > 00000000,00018000,000e706f and advertisement
>> >> > > 00000000,00018000,000e706f failed: -EINVAL
>> >> >
>> >> > Ah, I forgot this was a separate phy mode. Disregard this.
>> >> >
>> >> > >>
>> >> > >> > 1000baseT/Full
>> >> > >> > 10000baseT/Full
>> >> > >> > 1000baseKX/Full
>> >> > >> > 10000baseKX4/Full
>> >> > >> > 10000baseKR/Full
>> >> > >> > 2500baseT/Full
>> >> > >> > 5000baseT/Full
>> >> > >> > Supported pause frame use: Symmetric Receive-only
>> >> > >> > Supports auto-negotiation: Yes
>> >> > >> > Supported FEC modes: Not reported
>> >> > >> > Advertised link modes: 10baseT/Half 10baseT/Full
>> >> > >> > 100baseT/Half 100baseT/Full
>> >> > >> > 1000baseT/Full
>> >> > >> > 10000baseT/Full
>> >> > >> > 1000baseKX/Full
>> >> > >> > 10000baseKX4/Full
>> >> > >> > 10000baseKR/Full
>> >> > >> > 2500baseT/Full
>> >> > >> > 5000baseT/Full
>> >> > >> > Advertised pause frame use: Symmetric Receive-only
>> >> > >> > Advertised auto-negotiation: Yes
>> >> > >> > Advertised FEC modes: Not reported
>> >> > >> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
>> >> > >> > 1000baseT/Half 1000baseT/Full
>> >> > >> > 10000baseT/Full
>> >> > >> > 2500baseT/Full
>> >> > >> > 5000baseT/Full
>> >> > >> > Link partner advertised pause frame use: No
>> >> > >> > Link partner advertised auto-negotiation: Yes
>> >> > >> > Link partner advertised FEC modes: Not reported
>> >> > >> > Speed: 10000Mb/s
>> >> > >> > Duplex: Full
>> >> > >> > Port: Twisted Pair
>> >> > >> > PHYAD: 8
>> >> > >> > Transceiver: external
>> >> > >> > Auto-negotiation: on
>> >> > >> > MDI-X: Unknown
>> >> > >> > Link detected: yes
>> >> > >> > # ping 192.168.1.146 -c5
>> >> > >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
>> >> > >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
>> >> > >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
>> >> > >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
>> >> > >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
>> >> > >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
>> >> > >> >
>> >> > >> > --- 192.168.1.146 ping statistics ---
>> >> > >> > 5 packets transmitted, 5 packets received, 0% packet loss
>> >> > >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
>> >> > >> > # # force switch port to 1G
>> >> > >> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
>> >> > >> > 10gbase-r/Unknown/Unknown/none/off
>> >> > >> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
>> >> > >> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
>> >> > >> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
>> >> > >> > duplex=255 speed=-1 26:10gbase-r
>> >> > >> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
>> >> > >>
>> >> > >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
>> >> > >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
>> >> > >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
>> >> > >> and the vendor provisioning registers (dev 4, reg 0xc440 through
>> >> > >> 0xc449).
>> >> > >
>> >> > > yes, this is what I've been looking at as well. When forced to 1000m
>> >> > > the register shows a phy type of 11 which according to the aqr113
>> >> > > datasheet is XFI 5G:
>> >> > > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
>> >> > > link=1 duplex=1 speed=1000 interface=0
>> >> >
>> >> > That's pretty strange. Seems like it's rate adapting from 5g instead of
>> >> > 10g. Is SERDES Mode in the Global System Configuration For 1G register
>> >> > set to XFI?
>> >>
>> >> 1E.31C=0x0106:
>> >> Rate Adaptation Method: 2=Pause Rate Adaptation
>> >> SERDES Mode: 6=XFI/2 (XFI 5G)
>> >>
>> >
>> > The SERDES mode here is not valid and it seems to always be set to
>> > XFI/2 unless I init/use the AQR113 in U-Boot. If I manually set SERDES
>> > Mode to 0 (XFI) in the driver I find that all rates
>> > 10g,5g,2.5g,1g,100m work fine both in Linux 6.0 and in Linux 6.1-rc5.
>> > I'm still trying to understand why I would need to set SERDES Mode
>> > manually (vs the XFI mode specific firmware setting this) but I am
>> > unclear what the rate adaptation in 6.1 provides in this case. Is it
>> > that perhaps the AQR113 is handling rate adaptation internally and
>> > that's why the non 10gbe rates are working on 6.0?
>>
>> The changes in 6.1 are
>>
>> - We now always enable pause frame reception when doing rate adaptation.
>> This is necessary for rate adaptation to work, but wasn't done
>> automatically before.
>> - We advertise lower speed modes which are enabled with rate adaptation,
>> even if we would not otherwise be able to support them.
>>
>> I'm not sure why you'd have XFI/2 selected in 6.1 if it isn't selected
>> in 6.0.
>
> Sean,
>
> Thanks for the explanation. The issue I had which resulted in the
> wrong SERDES mode was simply that I was using the wrong aquantia
> firmware. They customize the firmware to default registers like SERDES
> mode specifically for customers and I was unknowingly using the
> firmware for XFI/2 instead of XFI.
>
> I suppose it would be worth putting something in the aquantia driver
> that verifies SERDES mode matches the phy-mode from dt to throw an
> error.
>
> Best Regards,
>
> Tim
Can you test the following series to see if it fixes your problem:
https://lore.kernel.org/netdev/20221128195409.100873-1-sean.anderson@seco.com/
--Sean
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: status of rate adaptation
2022-11-28 19:57 ` Sean Anderson
@ 2022-12-01 1:11 ` Tim Harvey
0 siblings, 0 replies; 21+ messages in thread
From: Tim Harvey @ 2022-12-01 1:11 UTC (permalink / raw)
To: Sean Anderson; +Cc: netdev, Russell King, David S. Miller
On Mon, Nov 28, 2022 at 11:58 AM Sean Anderson <sean.anderson@seco.com> wrote:
>
> Hi Tim,
>
> On 11/17/22 18:42, Tim Harvey wrote:
> > On Thu, Nov 17, 2022 at 7:38 AM Sean Anderson <sean.anderson@seco.com> wrote:
> >>
> >> On 11/16/22 17:37, Tim Harvey wrote:
> >> > On Mon, Nov 14, 2022 at 11:33 AM Tim Harvey <tharvey@gateworks.com> wrote:
> >> >>
> >> >> On Fri, Nov 11, 2022 at 2:38 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >> >> >
> >> >> > On 11/11/22 17:14, Tim Harvey wrote:
> >> >> > > On Fri, Nov 11, 2022 at 1:54 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >> >> > >>
> >> >> > >> On 11/11/22 16:20, Tim Harvey wrote:
> >> >> > >> > On Fri, Nov 11, 2022 at 12:58 PM Sean Anderson <sean.anderson@seco.com> wrote:
> >> >> > >> >>
> >> >> > >> >> On 11/11/22 15:57, Sean Anderson wrote:
> >> >> > >> >> > Hi Tim,
> >> >> > >> >> >
> >> >> > >> >> > On 11/11/22 15:44, Tim Harvey wrote:
> >> >> > >> >> >> Greetings,
> >> >> > >> >> >>
> >> >> > >> >> >> I've noticed some recent commits that appear to add rate adaptation support:
> >> >> > >> >> >> 3c42563b3041 net: phy: aquantia: Add support for rate matching
> >> >> > >> >> >> 7de26bf144f6 net: phy: aquantia: Add some additional phy interfaces
> >> >> > >> >> >> b7e9294885b6 net: phylink: Adjust advertisement based on rate matching
> >> >> > >> >> >> ae0e4bb2a0e0 net: phylink: Adjust link settings based on rate matching
> >> >> > >> >> >> 0c3e10cb4423 net: phy: Add support for rate matching
> >> >> > >> >> >>
> >> >> > >> >> >> I have a board with an AQR113C PHY over XFI that functions properly at
> >> >> > >> >> >> 10Gbe links but still not at 1Gbe,2.5Gbe,5.0Gbe,100M with v6.1-rc4
> >> >> > >> >> >>
> >> >> > >> >> >> Should I expect this to work now at those lower rates
> >> >> > >> >> >
> >> >> > >> >> > Yes.
> >> >> > >> >
> >> >> > >> > Sean,
> >> >> > >> >
> >> >> > >> > Good to hear - thank you for your work on this feature!
> >> >> > >> >
> >> >> > >> >> >
> >> >> > >> >> >> and if so what kind of debug information or testing can I provide?
> >> >> > >> >> >
> >> >> > >> >> > Please send
> >> >> > >> >> >
> >> >> > >> >> > - Your test procedure (how do you select 1G?)
> >> >> > >> >> > - Device tree node for the interface
> >> >> > >> >> > - Output of ethtool (on both ends if possible).
> >> >> > >> >> > - Kernel logs with debug enabled for drivers/phylink.c
> >> >> > >> >>
> >> >> > >> >> Sorry, this should be drivers/net/phy/phylink.c
> >> >> > >> >>
> >> >> > >> >> >
> >> >> > >> >> > That should be enough to get us started.
> >> >> > >> >> >
> >> >> > >> >> > --Sean
> >> >> > >> >>
> >> >> > >> >
> >> >> > >> > I'm currently testing by bringing up the network interface while
> >> >> > >> > connected to a 10gbe switch, verifying link and traffic, then forcing
> >> >> > >> > the switch port to 1000mbps.
> >> >> > >> >
> >> >> > >> > The board has a CN9130 on it (NIC is mvpp2) and the dt node snippets are:
> >> >> > >> >
> >> >> > >> > #include "cn9130.dtsi" /* include SoC device tree */
> >> >> > >> >
> >> >> > >> > &cp0_xmdio {
> >> >> > >> > pinctrl-names = "default";
> >> >> > >> > pinctrl-0 = <&cp0_xsmi_pins>;
> >> >> > >> > status = "okay";
> >> >> > >> >
> >> >> > >> > phy1: ethernet-phy@8 {
> >> >> > >> > compatible = "ethernet-phy-ieee802.3-c45";
> >> >> > >> > reg = <8>;
> >> >> > >> > };
> >> >> > >> > };
> >> >> > >> >
> >> >> > >> > &cp0_ethernet {
> >> >> > >> > status = "okay";
> >> >> > >> > };
> >> >> > >> >
> >> >> > >> > /* 10GbE XFI AQR113C */
> >> >> > >> > &cp0_eth0 {
> >> >> > >> > status = "okay";
> >> >> > >> > phy = <&phy1>;
> >> >> > >> > phy-mode = "10gbase-r";
> >> >> > >> > phys = <&cp0_comphy4 0>;
> >> >> > >> > };
> >> >> > >> >
> >> >> > >> > Here are some logs with debug enabled in drivers/net/phy/phylink.c and
> >> >> > >> > some additional debug in mvpp2.c and aquantia_main.c:
> >> >> > >> > # ifconfig eth0 192.168.1.22
> >> >> > >> > [ 8.882437] aqr107_config_init state=1:ready an=1 link=0 duplex=255
> >> >> > >> > speed=-1 26:10gbase-r
> >> >> > >> > [ 8.891391] aqr107_chip_info FW 5.6, Build 7, Provisioning 6
> >> >> > >> > [ 8.898165] aqr107_resume
> >> >> > >> > [ 8.902853] aqr107_get_rate_matching state=1:ready an=1 link=0
> >> >> > >> > duplex=255 speed=-1 26:10gbase-r 0:
> >> >> > >> > [ 8.911932] mvpp2 f2000000.ethernet eth0: PHY
> >> >> > >> > [f212a600.mdio-mii:08] driver [Aquantia AQR113C] (irq=POLL)
> >> >> > >> > [ 8.921577] mvpp2 f2000000.ethernet eth0: phy: 10gbase-r setting
> >> >> > >> > supported 00000000,00018000,000e706f advertising
> >> >> > >> > 00000000,00018000,000e706f
> >> >> > >> > [ 8.934349] mvpp2 f2000000.ethernet eth0: mac link down
> >> >> > >> > [ 8.948812] mvpp2 f2000000.ethernet eth0: configuring for
> >> >> > >> > phy/10gbase-r link mode
> >> >> > >> > [ 8.956350] mvpp2 f2000000.ethernet eth0: major config 10gbase-r
> >> >> > >> > [ 8.962414] mvpp2 f2000000.ethernet eth0: phylink_mac_config:
> >> >> > >> > mode=phy/10gbase-r/Unknown/Unknown/none adv=00000000,00000000,00000000
> >> >> > >> > pause=00 link=0 an=0
> >> >> > >> > [ 8.976252] mvpp2 f2000000.ethernet eth0: mac link down
> >> >> > >> > [ 8.976267] aqr107_resume
> >> >> > >> > [ 8.988970] mvpp2 f2000000.ethernet eth0: phy link down
> >> >> > >> > 10gbase-r/10Gbps/Full/none/off
> >> >> > >> > [ 8.997086] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> >> > >> > duplex=1 speed=10000 26:10gbase-r
> >> >> > >> > [ 14.112540] mvpp2 f2000000.ethernet eth0: mac link up
> >> >> > >> > [ 14.112594] mvpp2 f2000000.ethernet eth0: Link is Up - 10Gbps/Full
> >> >> > >> > - flow control off
> >> >> > >> > [ 14.112673] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> >> >> > >> > [ 14.118198] mvpp2 f2000000.ethernet eth0: phy link up
> >> >> > >> > 10gbase-r/10Gbps/Full/none/off
> >> >> > >> > [ 14.139808] aqr107_link_change_notify state=4:running an=1 link=1
> >> >> > >> > duplex=1 speed=10000 26:10gbase-r
> >> >> > >> >
> >> >> > >> > # ethtool eth0
> >> >> > >> > Settings for eth0:
> >> >> > >> > Supported ports: [ ]
> >> >> > >> > Supported link modes: 10baseT/Half 10baseT/Full
> >> >> > >> > 100baseT/Half 100baseT/Full
> >> >> > >>
> >> >> > >> 10/100 half duplex aren't achievable with rate matching (and we avoid
> >> >> > >> turning them on), so they must be coming from somewhere else. I wonder
> >> >> > >> if this is because PHY_INTERFACE_MODE_SGMII is set in
> >> >> > >> supported_interfaces.
> >> >> > >>
> >> >> > >> I wonder if you could enable USXGMII? Seems like mvpp2 with comphy
> >> >> > >> should support it. I'm not sure if the aquantia driver is set up for it.
> >> >> > >
> >> >> > > This appears to trigger an issue from mvpp2:
> >> >> > > mvpp2 f2000000.ethernet eth0: validation of usxgmii with support
> >> >> > > 00000000,00018000,000e706f and advertisement
> >> >> > > 00000000,00018000,000e706f failed: -EINVAL
> >> >> >
> >> >> > Ah, I forgot this was a separate phy mode. Disregard this.
> >> >> >
> >> >> > >>
> >> >> > >> > 1000baseT/Full
> >> >> > >> > 10000baseT/Full
> >> >> > >> > 1000baseKX/Full
> >> >> > >> > 10000baseKX4/Full
> >> >> > >> > 10000baseKR/Full
> >> >> > >> > 2500baseT/Full
> >> >> > >> > 5000baseT/Full
> >> >> > >> > Supported pause frame use: Symmetric Receive-only
> >> >> > >> > Supports auto-negotiation: Yes
> >> >> > >> > Supported FEC modes: Not reported
> >> >> > >> > Advertised link modes: 10baseT/Half 10baseT/Full
> >> >> > >> > 100baseT/Half 100baseT/Full
> >> >> > >> > 1000baseT/Full
> >> >> > >> > 10000baseT/Full
> >> >> > >> > 1000baseKX/Full
> >> >> > >> > 10000baseKX4/Full
> >> >> > >> > 10000baseKR/Full
> >> >> > >> > 2500baseT/Full
> >> >> > >> > 5000baseT/Full
> >> >> > >> > Advertised pause frame use: Symmetric Receive-only
> >> >> > >> > Advertised auto-negotiation: Yes
> >> >> > >> > Advertised FEC modes: Not reported
> >> >> > >> > Link partner advertised link modes: 100baseT/Half 100baseT/Full
> >> >> > >> > 1000baseT/Half 1000baseT/Full
> >> >> > >> > 10000baseT/Full
> >> >> > >> > 2500baseT/Full
> >> >> > >> > 5000baseT/Full
> >> >> > >> > Link partner advertised pause frame use: No
> >> >> > >> > Link partner advertised auto-negotiation: Yes
> >> >> > >> > Link partner advertised FEC modes: Not reported
> >> >> > >> > Speed: 10000Mb/s
> >> >> > >> > Duplex: Full
> >> >> > >> > Port: Twisted Pair
> >> >> > >> > PHYAD: 8
> >> >> > >> > Transceiver: external
> >> >> > >> > Auto-negotiation: on
> >> >> > >> > MDI-X: Unknown
> >> >> > >> > Link detected: yes
> >> >> > >> > # ping 192.168.1.146 -c5
> >> >> > >> > PING 192.168.1.146 (192.168.1.146): 56 data bytes
> >> >> > >> > 64 bytes from 192.168.1.146: seq=0 ttl=64 time=0.991 ms
> >> >> > >> > 64 bytes from 192.168.1.146: seq=1 ttl=64 time=0.267 ms
> >> >> > >> > 64 bytes from 192.168.1.146: seq=2 ttl=64 time=0.271 ms
> >> >> > >> > 64 bytes from 192.168.1.146: seq=3 ttl=64 time=0.280 ms
> >> >> > >> > 64 bytes from 192.168.1.146: seq=4 ttl=64 time=0.271 ms
> >> >> > >> >
> >> >> > >> > --- 192.168.1.146 ping statistics ---
> >> >> > >> > 5 packets transmitted, 5 packets received, 0% packet loss
> >> >> > >> > round-trip min/avg/max = 0.267/0.416/0.991 ms
> >> >> > >> > # # force switch port to 1G
> >> >> > >> > [ 193.343494] mvpp2 f2000000.ethernet eth0: phy link down
> >> >> > >> > 10gbase-r/Unknown/Unknown/none/off
> >> >> > >> > [ 193.343539] mvpp2 f2000000.ethernet eth0: mac link down
> >> >> > >> > [ 193.344524] mvpp2 f2000000.ethernet eth0: Link is Down
> >> >> > >> > [ 193.351973] aqr107_link_change_notify state=5:nolink an=1 link=0
> >> >> > >> > duplex=255 speed=-1 26:10gbase-r
> >> >> > >> > [ 197.472489] mvpp2 f2000000.ethernet eth0: phy link up /1Gbps/Full/pause/off
> >> >> > >>
> >> >> > >> Well, it looks like we have selected PHY_INTERFACE_MODE_NA. Can you send
> >> >> > >> the value of MDIO_PHYXS_VEND_IF_STATUS (dev 4, reg 0xe812)? Please also
> >> >> > >> send the global config registers (dev 0x1e, reg 0x0310 through 0x031f)
> >> >> > >> and the vendor provisioning registers (dev 4, reg 0xc440 through
> >> >> > >> 0xc449).
> >> >> > >
> >> >> > > yes, this is what I've been looking at as well. When forced to 1000m
> >> >> > > the register shows a phy type of 11 which according to the aqr113
> >> >> > > datasheet is XFI 5G:
> >> >> > > aqr107_read_status STATUS=0x00001258 (type=11) state=4:running an=1
> >> >> > > link=1 duplex=1 speed=1000 interface=0
> >> >> >
> >> >> > That's pretty strange. Seems like it's rate adapting from 5g instead of
> >> >> > 10g. Is SERDES Mode in the Global System Configuration For 1G register
> >> >> > set to XFI?
> >> >>
> >> >> 1E.31C=0x0106:
> >> >> Rate Adaptation Method: 2=Pause Rate Adaptation
> >> >> SERDES Mode: 6=XFI/2 (XFI 5G)
> >> >>
> >> >
> >> > The SERDES mode here is not valid and it seems to always be set to
> >> > XFI/2 unless I init/use the AQR113 in U-Boot. If I manually set SERDES
> >> > Mode to 0 (XFI) in the driver I find that all rates
> >> > 10g,5g,2.5g,1g,100m work fine both in Linux 6.0 and in Linux 6.1-rc5.
> >> > I'm still trying to understand why I would need to set SERDES Mode
> >> > manually (vs the XFI mode specific firmware setting this) but I am
> >> > unclear what the rate adaptation in 6.1 provides in this case. Is it
> >> > that perhaps the AQR113 is handling rate adaptation internally and
> >> > that's why the non 10gbe rates are working on 6.0?
> >>
> >> The changes in 6.1 are
> >>
> >> - We now always enable pause frame reception when doing rate adaptation.
> >> This is necessary for rate adaptation to work, but wasn't done
> >> automatically before.
> >> - We advertise lower speed modes which are enabled with rate adaptation,
> >> even if we would not otherwise be able to support them.
> >>
> >> I'm not sure why you'd have XFI/2 selected in 6.1 if it isn't selected
> >> in 6.0.
> >
> > Sean,
> >
> > Thanks for the explanation. The issue I had which resulted in the
> > wrong SERDES mode was simply that I was using the wrong aquantia
> > firmware. They customize the firmware to default registers like SERDES
> > mode specifically for customers and I was unknowingly using the
> > firmware for XFI/2 instead of XFI.
> >
> > I suppose it would be worth putting something in the aquantia driver
> > that verifies SERDES mode matches the phy-mode from dt to throw an
> > error.
> >
> > Best Regards,
> >
> > Tim
>
> Can you test the following series to see if it fixes your problem:
>
> https://lore.kernel.org/netdev/20221128195409.100873-1-sean.anderson@seco.com/
>
> --Sean
Sean,
I believe the discussion is still ongoing regarding this being the
correct approach but that series does in fact resolve the issue I had
where my firmware was provisioned for something not compatible with
the SERDES link I needed.
Best Regards,
Tim
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2022-12-01 1:11 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-11 20:44 status of rate adaptation Tim Harvey
2022-11-11 20:57 ` Sean Anderson
2022-11-11 20:58 ` Sean Anderson
2022-11-11 21:20 ` Tim Harvey
2022-11-11 21:54 ` Sean Anderson
2022-11-11 22:14 ` Tim Harvey
2022-11-11 22:38 ` Sean Anderson
2022-11-12 0:48 ` Vladimir Oltean
2022-11-12 16:08 ` Andrew Lunn
2022-11-14 15:08 ` Sean Anderson
2022-11-14 19:33 ` Tim Harvey
2022-11-16 22:37 ` Tim Harvey
2022-11-17 15:38 ` Sean Anderson
2022-11-17 23:42 ` Tim Harvey
2022-11-28 19:57 ` Sean Anderson
2022-12-01 1:11 ` Tim Harvey
2022-11-11 22:33 ` Russell King (Oracle)
2022-11-12 13:15 ` Russell King (Oracle)
2022-11-14 15:33 ` Sean Anderson
2022-11-14 16:13 ` Russell King (Oracle)
2022-11-18 18:16 ` Sean Anderson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).