PHY VSC8531 MDIO data not reflected in ethernet/ sub-module

PHY VSC8531 MDIO data not reflected in ethernet/ sub-module

[Ron Eggler]

Hi, Everybody,

I've posted here previously about the bring up of two network interfaces 
on an embedded platform that is using two the Microsemi VSC8531 PHYs. 
(two previous threads: "issues to bring up two VSC8531 PHYs" & "Unable 
to TX data on VSC8531". Thanks to Heiner Kallweit, Andrew Lunn, Horatiu 
Vultur & everybody else who might be following along)!

A quick summary of the previous threads is: No matter what speed the PHY 
negotiates to 10/100 or 1G, the RGMII TX_CLK always remains at 2.5MHz 
(which would be frequency for 10baseT). I'm starting yet another thread 
as I have realized that the MDIO information does not seem to propagate 
through to the drivers/net/ethernet/renesas/ravb_main.c which utilizes 
the function *ravb_set_rate_gbeth() to set the MPU's registers correctly 
(which is needed to get the correct frequency on the RGMII bus). **
I have not found a potential reason for this, the MDIO comms appear to 
work properly as I can retrieve information with the mdio and the 
mii-tool utility properly.
**I've added printk debug logs with to ravb_main.c and found out that 
the link state nor the speed do not appear to be propagated through 
properly (when compared with mii-tool).*
***The logs generated look like:*
***
# dmesg | grep DEBUG*
***[ 6.728165] DEBUG: in ravb_emac_init_gbeth(), calling 
ravb_set_rate_gbeth(), priv->duplex: 0, priv->speed: 0*
***[ 6.751973] DEBUG: in ravb_set_rate_gbeth() - priv->speed 0*
***[ 6.831153] DEBUG: in ravb_adjust_link(), phydev->speed -1, 
priv->speed 0*
***[ 6.839952] DEBUG: in ravb_adjust_link(), priv->no_avb_link 0, 
phydev->link 0*
***[ 6.847093] DEBUG: in ravb_adjust_link(), phydev->autoneg_complete: 0*
***[ 6.853514] DEBUG: in ravb_adjust_link(),phydev->link 0*
***[ 6.883683] DEBUG: in ravb_emac_init_gbeth(), calling 
ravb_set_rate_gbeth(), priv->duplex: 0, priv->speed: 0*
***[ 6.898551] DEBUG: in ravb_set_rate_gbeth() - priv->speed 0*
***[ 6.963742] DEBUG: in ravb_adjust_link(), phydev->speed -1, 
priv->speed 0*
***[ 6.973404] DEBUG: in ravb_adjust_link(), priv->no_avb_link 0, 
phydev->link 0*
***[ 6.981869] DEBUG: in ravb_adjust_link(), phydev->autoneg_complete: 0*
***[ 6.989645] DEBUG: in ravb_adjust_link(),phydev->link 0
*

**


Which has been generated by application of the following patch file: 
https://github.com/MistySOM/meta-mistysom/blob/phy-enable/recipes-kernel/linux/smarc-rzg2l/0002-add-phy_debug_logs.patch

While I also receive the following using the mii-tool utility:
# mii-tool -vv eth0
Using SIOCGMIIPHY=0x8947
eth0: negotiated 1000baseT-FD flow-control, link ok
   registers for MII PHY 0:
     1040 796d 0007 0572 01e1 cde1 000f 2001
     4006 0300 7800 0000 0000 4002 0000 3000
     0000 f000 0088 0000 0000 0001 3200 1000
     0000 0000 0000 0000 a035 0054 0400 0000
   product info: vendor 00:01:c1, model 23 rev 2
   basic mode:   autonegotiation enabled
   basic status: autonegotiation complete, link ok
   capabilities: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD 
10baseT-FD 10baseT-HD
   advertising:  1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 
10baseT-HD
   link partner: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD 
10baseT-FD 10baseT-HD flow-control

# mii-tool -vv eth1
Using SIOCGMIIPHY=0x8947
eth1: no link
   registers for MII PHY 0:
     1040 7949 0007 0572 01e1 0000 0004 2001
     0000 0300 4000 0000 0000 4002 0000 3000
     0000 0000 0088 0000 0000 0000 3200 1000
     0000 0000 0000 0000 0404 0054 0400 0000
   product info: vendor 00:01:c1, model 23 rev 2
   basic mode:   autonegotiation enabled
   basic status: no link
   capabilities: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD 
10baseT-FD 10baseT-HD
   advertising:  100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD

Also one oditty I just realize now is that eth1 only appears to 
advertise 100Mbits until I connect it to the 1G line, then it looks just 
like eth0 above but that's something for another day
Can sdomeone help me to find out why the MDIO info doesn't appear to 
pass through to the ravb driver properly?

Thanks,
-- 
Ron

Re: PHY VSC8531 MDIO data not reflected in ethernet/ sub-module

[Andrew Lunn]

> ***[ 6.728165] DEBUG: in ravb_emac_init_gbeth(), calling
> ravb_set_rate_gbeth(), priv->duplex: 0, priv->speed: 0*
> ***[ 6.751973] DEBUG: in ravb_set_rate_gbeth() - priv->speed 0*
> ***[ 6.831153] DEBUG: in ravb_adjust_link(), phydev->speed -1, priv->speed
> 0*
> ***[ 6.839952] DEBUG: in ravb_adjust_link(), priv->no_avb_link 0,
> phydev->link 0*

If there is no link, everything else is meaningless. You cannot have
speed without link.

> While I also receive the following using the mii-tool utility:
> # mii-tool -vv eth0
> Using SIOCGMIIPHY=0x8947
> eth0: negotiated 1000baseT-FD flow-control, link ok
>   registers for MII PHY 0:
>     1040 796d 0007 0572 01e1 cde1 000f 2001
>     4006 0300 7800 0000 0000 4002 0000 3000
>     0000 f000 0088 0000 0000 0001 3200 1000
>     0000 0000 0000 0000 a035 0054 0400 0000
>   product info: vendor 00:01:c1, model 23 rev 2
>   basic mode:   autonegotiation enabled
>   basic status: autonegotiation complete, link ok
>   capabilities: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD
> 10baseT-FD 10baseT-HD
>   advertising:  1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
>   link partner: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD
> 10baseT-FD 10baseT-HD flow-control

Assuming you are not using interrupts, phylib will poll the PHY once a
second, calling
phy_check_link_status()->
  phy_read_status()->
    phydev->drv->read_status()
or
    genphy_read_status()

Check what these are doing, why do they think the link is down.

      Andrew

Re: PHY VSC8531 MDIO data not reflected in ethernet/ sub-module

[Ron Eggler]

Hi Andrew,

Thanks for your response:

On 2023-05-11 8:34 a.m., Andrew Lunn wrote:
>> ***[ 6.728165] DEBUG: in ravb_emac_init_gbeth(), calling
>> ravb_set_rate_gbeth(), priv->duplex: 0, priv->speed: 0*
>> ***[ 6.751973] DEBUG: in ravb_set_rate_gbeth() - priv->speed 0*
>> ***[ 6.831153] DEBUG: in ravb_adjust_link(), phydev->speed -1, priv->speed
>> 0*
>> ***[ 6.839952] DEBUG: in ravb_adjust_link(), priv->no_avb_link 0,
>> phydev->link 0*
> If there is no link, everything else is meaningless. You cannot have
> speed without link.
Makes sense!
>> While I also receive the following using the mii-tool utility:
>> # mii-tool -vv eth0
>> Using SIOCGMIIPHY=0x8947
>> eth0: negotiated 1000baseT-FD flow-control, link ok
>>    registers for MII PHY 0:
>>      1040 796d 0007 0572 01e1 cde1 000f 2001
>>      4006 0300 7800 0000 0000 4002 0000 3000
>>      0000 f000 0088 0000 0000 0001 3200 1000
>>      0000 0000 0000 0000 a035 0054 0400 0000
>>    product info: vendor 00:01:c1, model 23 rev 2
>>    basic mode:   autonegotiation enabled
>>    basic status: autonegotiation complete, link ok
>>    capabilities: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD
>> 10baseT-FD 10baseT-HD
>>    advertising:  1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 10baseT-HD
>>    link partner: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD
>> 10baseT-FD 10baseT-HD flow-control
> Assuming you are not using interrupts, phylib will poll the PHY once a
> second, calling
> phy_check_link_status()->

I don't see it being invoked every Seconds but it gets invoked on boot, 
I added debug logs and see the following:

[    6.669425] DEBUG: in phy_check_link_status(), phy_read_status() ret 
err 0, phydev->loopback_enabled 0,
[    6.696237] DEBUG: in phy_check_link_status(), phydev->link 0, 
phydev->state UP

state = UP which means it's ready to start auto negotiation(in 
phy_state_machine())  but instead in phy_check_link_status(), 
phydev->state should be set to  PHY_RUNNING but it only can get set to 
PHY_RUNNING when phydev->link is 1 (in phy_check_link_status()):

>    phy_read_status()->
>      phydev->drv->read_status()
> or
>      genphy_read_status()
>
> Check what these are doing, why do they think the link is down.

Yes, so in phy_read_status, phydev->drv->read_status appears to be set 
but I cannot figure out where it gets set. (I obviously need to find the 
function to find why the link isn't read correctly).
I temporarily set phydev->drv->read_status to 0 to force invocation of 
genphy_read_status() function to see how that would work.

genphy_update_link(0 is called from genphy_read_status() and I get the 
below data:

[    6.795480] DEBUG: in genphy_update_link(), after phy_read() bmcr 4160
[    6.805225] DEBUG: in genphy_update_link(), bmcr 0x1040 & 0x200
[    6.815730] DEBUG: in genphy_read_status(), genphy_update_link() 0 
phydev->autoneg 1, phydev->link 0


Could it be that the link needs a second to come up when when the 
network drivers get started and hence I should make sure that the 
polling once a second works (which currently doesn't appear to be the 
case)? Am I missing a configuration option?

Thanks!

-- 
Ron

Re: PHY VSC8531 MDIO data not reflected in ethernet/ sub-module

[Andrew Lunn]

> I don't see it being invoked every Seconds but it gets invoked on boot, I
> added debug logs and see the following:

What should happen is when the MAC driver call phy_start(), it either
starts polling the PHY or it enabled interrupts. If it is not polling,
then is sounds like you have interrupts setup for the PHY. Scatter
some more debug prints around and about and see which is true.

> state = UP which means it's ready to start auto negotiation(in
> phy_state_machine())  but instead in phy_check_link_status(), phydev->state
> should be set to  PHY_RUNNING but it only can get set to PHY_RUNNING when
> phydev->link is 1 (in phy_check_link_status()):

Yep. Either via polling, or interrupts, the state machine will change
to state RUNNING.

> 
> >    phy_read_status()->
> >      phydev->drv->read_status()
> > or
> >      genphy_read_status()
> > 
> > Check what these are doing, why do they think the link is down.
> 
> Yes, so in phy_read_status, phydev->drv->read_status appears to be set but I
> cannot figure out where it gets set. (I obviously need to find the function
> to find why the link isn't read correctly).

Since this is a microchip PHY, i would expect vsc85xx_read_status().

> I temporarily set phydev->drv->read_status to 0 to force invocation of
> genphy_read_status() function to see how that would work.
> 
> genphy_update_link(0 is called from genphy_read_status() and I get the below
> data:
> 
> [    6.795480] DEBUG: in genphy_update_link(), after phy_read() bmcr 4160
> [    6.805225] DEBUG: in genphy_update_link(), bmcr 0x1040 & 0x200
> [    6.815730] DEBUG: in genphy_read_status(), genphy_update_link() 0
> phydev->autoneg 1, phydev->link 0
> 
> 
> Could it be that the link needs a second to come up when when the network
> drivers get started and hence I should make sure that the polling once a
> second works (which currently doesn't appear to be the case)? Am I missing a
> configuration option?

auto-neg takes a little over 1 second. Polling does not care, if it is
not up this time, it might be the next. If you are using interrupts,
then you need to ensure the interrupt actually fires when auto-neg is
complete.

	Andrew

Re: PHY VSC8531 MDIO data not reflected in ethernet/ sub-module

[Ron Eggler]

On 2023-05-11 16:21, Ron Eggler wrote:
> Hi Andrew,
>
> Thanks for your response:
>
> On 2023-05-11 8:34 a.m., Andrew Lunn wrote:
>>> ***[ 6.728165] DEBUG: in ravb_emac_init_gbeth(), calling
>>> ravb_set_rate_gbeth(), priv->duplex: 0, priv->speed: 0*
>>> ***[ 6.751973] DEBUG: in ravb_set_rate_gbeth() - priv->speed 0*
>>> ***[ 6.831153] DEBUG: in ravb_adjust_link(), phydev->speed -1, 
>>> priv->speed
>>> 0*
>>> ***[ 6.839952] DEBUG: in ravb_adjust_link(), priv->no_avb_link 0,
>>> phydev->link 0*
>> If there is no link, everything else is meaningless. You cannot have
>> speed without link.
> Makes sense!
>>> While I also receive the following using the mii-tool utility:
>>> # mii-tool -vv eth0
>>> Using SIOCGMIIPHY=0x8947
>>> eth0: negotiated 1000baseT-FD flow-control, link ok
>>>    registers for MII PHY 0:
>>>      1040 796d 0007 0572 01e1 cde1 000f 2001
>>>      4006 0300 7800 0000 0000 4002 0000 3000
>>>      0000 f000 0088 0000 0000 0001 3200 1000
>>>      0000 0000 0000 0000 a035 0054 0400 0000
>>>    product info: vendor 00:01:c1, model 23 rev 2
>>>    basic mode:   autonegotiation enabled
>>>    basic status: autonegotiation complete, link ok
>>>    capabilities: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD
>>> 10baseT-FD 10baseT-HD
>>>    advertising:  1000baseT-FD 100baseTx-FD 100baseTx-HD 10baseT-FD 
>>> 10baseT-HD
>>>    link partner: 1000baseT-HD 1000baseT-FD 100baseTx-FD 100baseTx-HD
>>> 10baseT-FD 10baseT-HD flow-control
>> Assuming you are not using interrupts, phylib will poll the PHY once a
>> second, calling
>> phy_check_link_status()->
>
> I don't see it being invoked every Seconds but it gets invoked on 
> boot, I added debug logs and see the following:
>
> [    6.669425] DEBUG: in phy_check_link_status(), phy_read_status() 
> ret err 0, phydev->loopback_enabled 0,
> [    6.696237] DEBUG: in phy_check_link_status(), phydev->link 0, 
> phydev->state UP
>
> state = UP which means it's ready to start auto negotiation(in 
> phy_state_machine())  but instead in phy_check_link_status(), 
> phydev->state should be set to  PHY_RUNNING but it only can get set to 
> PHY_RUNNING when phydev->link is 1 (in phy_check_link_status()):
>
>>    phy_read_status()->
>>      phydev->drv->read_status()
>> or
>>      genphy_read_status()
>>
>> Check what these are doing, why do they think the link is down.
>
> Yes, so in phy_read_status, phydev->drv->read_status appears to be set 
> but I cannot figure out where it gets set. (I obviously need to find 
> the function to find why the link isn't read correctly).
> I temporarily set phydev->drv->read_status to 0 to force invocation of 
> genphy_read_status() function to see how that would work.
>
> genphy_update_link(0 is called from genphy_read_status() and I get the 
> below data:
>
> [    6.795480] DEBUG: in genphy_update_link(), after phy_read() bmcr 4160
> [    6.805225] DEBUG: in genphy_update_link(), bmcr 0x1040 & 0x200
> [    6.815730] DEBUG: in genphy_read_status(), genphy_update_link() 0 
> phydev->autoneg 1, phydev->link 0
>
>
> Could it be that the link needs a second to come up when when the 
> network drivers get started and hence I should make sure that the 
> polling once a second works (which currently doesn't appear to be the 
> case)? Am I missing a configuration option?

For now I've tried to add:
phydev->irq = PHY_POLL;
to phy_attached_info_irq() in drivers/net/phy/phy_device.c which turns 
out totally locks-up the system, i.e. I can't even type root to login 
anymore. This is after I removed the log messages now but while I still 
had them in there, they would just fill up my screen, the polling 
appears to use up all the CPU power.

A little cut-out from the logs, looked like this:

...
[  418.485394] DEBUG: in phy_check_link_status(), phy_read_status() err 
0, phydev->loopback_enabled 0,
[  418.503700] DEBUG: in phy_check_link_status(), phydev->link 1, 
phydev->state RUNNING
[  418.511463] DEBUG: in phy_state_machine(), phydev->state RUNNING, 
needs_aneg 0
[  418.518727] DEBUG: in phy_state_machine(), phy_start_aneg() 0, 
needs_aneg 826560768
[  419.361074] DEBUG: in phy_read_status(), phydev->drv->read_status 0
[  419.367474] DEBUG: in genphy_update_link(), after phy_read() bmcr 4160
[  419.375665] DEBUG: in genphy_update_link(), bmcr 0x1040 & 0x200
[  419.382324] DEBUG: in genphy_read_status(), genphy_update_link() 0 
phydev->autoneg 1, phydev->link 0
[  419.382547] DEBUG: in phy_check_link_status(), phy_read_status() err 
0, phydev->loopback_enabled 0,
[  419.400837] DEBUG: in phy_check_link_status(), phydev->link 0, 
phydev->state NOLINK
[  419.408521] DEBUG: in phy_state_machine(), phydev->state NOLINK, 
needs_aneg 0
[  419.415718] DEBUG: in phy_state_machine(), phy_start_aneg() 0, 
needs_aneg 826560768
[  419.553089] DEBUG: in phy_read_status(), phydev->drv->read_status 0
[  419.559486] DEBUG: in genphy_update_link(), after phy_read() bmcr 4160
[  419.566858] DEBUG: in genphy_update_link(), bmcr 0x1040 & 0x200
[  419.573423] DEBUG: in genphy_read_status(), genphy_update_link() 0 
phydev->autoneg 1, phydev->link 1
[  419.573432] DEBUG: in phy_check_link_status(), phy_read_status() err 
0, phydev->loopback_enabled 0,
[  419.591738] DEBUG: in phy_check_link_status(), phydev->link 1, 
phydev->state RUNNING
[  419.599505] DEBUG: in phy_state_machine(), phydev->state RUNNING, 
needs_aneg 0
[  419.606764] DEBUG: in phy_state_machine(), phy_start_aneg() 0, 
needs_aneg 826560768
...

The state toggled between RUNNING & NOLINK and the phydev->link toggeled 
between 1 and 0

The reason why I had interrupts activated is because the device tree 
that's used as a base had a PHY with the interrupt pin hooked up and 
while I override the original phy node in the device tree with 
https://github.com/MistySOM/meta-mistysom/blob/phy-enable/recipes-kernel/linux/smarc-rzg2l/0001-add-vsc8531-userspace-dts.patch 
, the interrupt attribute from the original device tree will persist. I 
will have to create a patch that removes the attrribute in the dtsi file 
where the original phy node is defined. I think that should give me the 
poll functionality

-- 

Ron

Re: PHY VSC8531 MDIO data not reflected in ethernet/ sub-module

[Ron Eggler]

I should follow up on this Thread and send out a Thank you for everybody 
that has pitched in. I've got the two Ethernet interfaces working fine 
now after I set "phy-mode" to "rgmii-id" and removed the "interrupts" 
and "interrupt-parent" attributes to trigger the polling of the PHY.
Both changes were made in the device tree.

Thanks again!
Ron


On 2023-05-11 19:47, Andrew Lunn wrote:
>> I don't see it being invoked every Seconds but it gets invoked on boot, I
>> added debug logs and see the following:
> What should happen is when the MAC driver call phy_start(), it either
> starts polling the PHY or it enabled interrupts. If it is not polling,
> then is sounds like you have interrupts setup for the PHY. Scatter
> some more debug prints around and about and see which is true.
>
>> state = UP which means it's ready to start auto negotiation(in
>> phy_state_machine())  but instead in phy_check_link_status(), phydev->state
>> should be set to  PHY_RUNNING but it only can get set to PHY_RUNNING when
>> phydev->link is 1 (in phy_check_link_status()):
> Yep. Either via polling, or interrupts, the state machine will change
> to state RUNNING.
>
>>>     phy_read_status()->
>>>       phydev->drv->read_status()
>>> or
>>>       genphy_read_status()
>>>
>>> Check what these are doing, why do they think the link is down.
>> Yes, so in phy_read_status, phydev->drv->read_status appears to be set but I
>> cannot figure out where it gets set. (I obviously need to find the function
>> to find why the link isn't read correctly).
> Since this is a microchip PHY, i would expect vsc85xx_read_status().
>
>> I temporarily set phydev->drv->read_status to 0 to force invocation of
>> genphy_read_status() function to see how that would work.
>>
>> genphy_update_link(0 is called from genphy_read_status() and I get the below
>> data:
>>
>> [    6.795480] DEBUG: in genphy_update_link(), after phy_read() bmcr 4160
>> [    6.805225] DEBUG: in genphy_update_link(), bmcr 0x1040 & 0x200
>> [    6.815730] DEBUG: in genphy_read_status(), genphy_update_link() 0
>> phydev->autoneg 1, phydev->link 0
>>
>>
>> Could it be that the link needs a second to come up when when the network
>> drivers get started and hence I should make sure that the polling once a
>> second works (which currently doesn't appear to be the case)? Am I missing a
>> configuration option?
> auto-neg takes a little over 1 second. Polling does not care, if it is
> not up this time, it might be the next. If you are using interrupts,
> then you need to ensure the interrupt actually fires when auto-neg is
> complete.
>
> 	Andrew
-- 
Ron -- Ron Eggler Senior Firmware Developer 778 230 9442 
www.mistywest.com __________________________________________________ 
About MistyWest We are a Research & Engineering firm composed of 
engineers and physicists with a focus on solving hard problems across a 
number of technology verticals. We specifically target projects that 
have the potential for high-impact, whether it's improving the human 
condition, impacting sustainability in a positive way, or otherwise 
moving us collectively to an inclusively abundant future.