RE: [PATCH v2 3/3] net: macb: add support for high speed interface

[Milind Parab <mparab@cadence.com>]

>> > +	if (bp->phy_interface == PHY_INTERFACE_MODE_USXGMII) {
>>
>> Why bp->phy_interface and not state->interface?

okay, this needs to change to state->interface

>>
>> If you don't support selecting between USXGMII and other modes at
>> runtime, should macb_validate() be allowing ethtool link modes for
>> it when it's different from the configured setting?

We have separate SGMII and USXGMII PCS, which are enabled and programmed 
by MAC driver. Also, there are separate low speed (up to 1G) and high 
speed MAC which can be programmed though MAC driver. 
As long as, PHY (PMA, external to Cadence MAC controller) can handle 
this change, GEM can work with interface changes at a runtime.

>>
>> > +		if (gem_mac_usx_configure(bp, state) < 0) {
>> > +			spin_unlock_irqrestore(&bp->lock, flags);
>> > +			phylink_mac_change(bp->phylink, false);
>>
>> I guess this is the reason you're waiting for the USXGMII block
>> to lock - do you not have any way to raise an interrupt when
>> something changes with the USXGMII (or for that matter SGMII)
>> blocks?  Without that, you're fixed to a single speed.

Yes, we need to wait (poll) until USXGMII block lock is set.
Interrupt for USXGMII block lock set event is not supported.

>
>BTW, if you don't have an macb_mac_pcs_get_state() implementation,
>and from what you described last time around, I don't see how SGMII
>nor this new addition of USXGMII can work for you. Both these
>protocols use in-band control words, which should be read and
>interpreted in macb_mac_pcs_get_state().
>
>What I think you're trying to do is to use your PCS PHY as a normal
>PHY, or maybe you're ignoring the PCS PHY completely and relying on
>an external PHY (and hence always using MLO_AN_PHY or MLO_AN_FIXED
>mode.)

We are limiting our functionality to 10G fixed link using PCS and SFP+
Though the Cadence MAC is a full functional ethernet MAC controller, 
we are not sure what PHY or PCS be used in the end system.
Hence we are using PCS PHY as a normal PHY and not dependent on 
macb_mac_pcs_get_state(). Also it should be noted that we are 
not doing any change in SGMII. Status available in PCS is 
just a "status transferred" from PHY. So in case of SGMII, whether 
we read from PCS or from PHY, it is the same information. 

Below are listed all the possible use cases of Cadence GEM 10G controller

Basic MII MAC/PHY interconnect using MDIO for link status xfer.
 +-------------+                                    +--------+
 |             |                                    |        |
 | GEM MAC/DMA | <------ GMII/RGMII/RMII/MII -----> |  PHY   |
 |             |                                    |        |
 +-------------+                                    +--------+
       ^                                                 ^
       |_____________________ MDIO ______________________|

No PHY. No status xfer required. GEM PCS responsible for auto-negotiation
across link. Driver must interrogate PCS registers within GEM.
 +-------------+                                    +--------+
 |             |       |        |                   |        |
 | GEM MAC/DMA | <---> | SerDes | <- 1000BASE-X ->  |  SFP   |
 |    PCS      |       | (PMA)  |                   |        |
 +-------------+                                    +--------+      

SGMII MAC/PHY interconnect using MDIO for link status xfer.
 +-------------+                                    +--------+
 |             |       |        |                   |        |
 | GEM MAC/DMA | <---> | SerDes | <--- SGMII --->   |  PHY   |
 |  SGMII PCS  |       | (PMA)  |                   |        |
 +-------------+                                    +--------+
       ^                                                 ^
       |_____________________ MDIO ______________________|

SGMII MAC/PHY interconnect using inline status xfer. Multi-rate.
Driver must interrogate PCS registers within GEM.
 +-------------+                                    +--------+
 |             |       |        |                   |        |
 | GEM MAC/DMA | <---> | SerDes | <--- SGMII --->   |  PHY   |
 |  SGMII PCS  |       | (PMA)  |                   |        |
 +-------------+                                    +--------+

Up to 2.5G. MAC/PHY interconnect. Rate determined by 2.5GBASE-T PHY capability.
 +--------------+                                  +-----------+
 |              |       |        |                 |           |
 | GEM MAC/DMA  | <---> | SerDes | <-2500BASE-X->  |2.5GBASE-T |
 |2.5GBASE-X PCS|       | (PMA)  |                 |   PHY     |
 +--------------+                                  +-----------+

No ability for host to interrogate Optical.
 +--------------+                                  +-----------+
 |              |       |        |                 |  SFP+     |
 | GEM MAC/DMA  | <---> | SerDes | <---- SFI-----> | Optical   |
 |   USX PCS|   |       | (PMA)  |                 | Module    |
 +--------------+                                  +-----------+

Additional 3rd party I2C IP required (not part of GEM) for module
interrogation (MDIO to I2C handled by SW
 +--------------+                                  +-----------+
 |              |       |        |                 |  SFP+     |
 | GEM MAC/DMA  | <---> | SerDes | <---- SFI-----> | Optical   |
 |   USX PCS|   |       | (PMA)  |                 | Module    |
 +--------------+                                  +-----------+
                                                         ^
        +--------+                                       |
        | I2C    |                                       |
        | Master | <-------------------------------------|
        +--------+

Rate determined by 10GBASE-T PHY capability through auto-negotiation. 
I2C IP required
 +--------------+                                  +-----------+
 |              |       |        |                 |  SFP+ to  |
 | GEM MAC/DMA  | <---> | SerDes | <---- SFI-----> | 10GBASE-T |
 |   USX PCS|   |       | (PMA)  |                 |           |
 +--------------+                                  +-----------+
                                                         ^
        +--------+                                       |
        | I2C    |                                       |
        | Master | <-------------------------------------|
        +--------+

USXGMII PHY. Uses MDIO or equivalent for status xfer
 +-------------+                                    +--------+
 |             |       |        |                   |        |
 | GEM MAC/DMA | <---> | SerDes | <--- USXGMII ---> |  PHY   |
 |  USX PCS    |       | (PMA)  |                   |        |
 +-------------+                                    +--------+
       ^                                                 ^
       |_____________________ MDIO ______________________|


>
>
>
>--