[PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Marek Behún]

Hello,

the Marvell Alaska PHYs (88X3310, 88E2110) can, depending on their
configuration, change the PHY mode to the MAC, depending on the
copper/fiber speed.

The 88X3310, for example, can be configured (via MACTYPE register)
so that it communicates with the MAC in sgmii for 10/100/1000mbps,
2500base-x for 2500mbps, 5gbase-r for 5gbps and either 10gbase-r,
xaui or rxaui for 10gbps. Or the PHY may communicate with the MAC
in usxgmii, or one of the 10gbase-r, rxaui or xaui modes with rate
matching.

So for the 10gbps mode we have options 10gbase-r, xaui, rxaui and
usxgmii. The MAC can support some of these modes, and if it does more
than one, we need to know which one to use. Not all of these modes
must necessarily be supported by the board (xaui required wiring for
4 SerDes lanes, for example, and 10gbase-r requires wiring capable
of transmitting at 10.3125 GBd).

The MACTYPE is upon HW reset configured by strapping pins - so the
board should have a correct mode configured after HW reset.

One problem with this is that some boards configure the MACTYPE to
a rate matching mode, which, according to the errata, is broken in
some situations (it does not work for 10/100/1000, for example).

Another problem is that if lower modes are supported, we should
maybe use them in order to save power.

But for this we need to know which phy-modes are supported on the
board.

This series adds documentation for a new ethernet PHY property,
called `supported-mac-connection-types`.

When this property is present for a PHY node, only the phy-modes
listed in this property should be considered to be functional on
the board.

The second patch adds binding for this property.

The first patch does some YAML magic in ethernet-controller.yaml
in order to be able to reuse the PHY modes enum, so that the same
list does not have to be defined twice.

Marek

Marek Behún (2):
  dt-bindings: ethernet-controller: create a type for PHY interface
    modes
  dt-bindings: ethernet-phy: define `supported-mac-connection-types`
    property

 .../bindings/net/ethernet-controller.yaml     | 89 ++++++++++---------
 .../devicetree/bindings/net/ethernet-phy.yaml | 18 ++++
 2 files changed, 66 insertions(+), 41 deletions(-)

-- 
2.26.2

[PATCH net-next 1/2] dt-bindings: ethernet-controller: create a type for PHY interface modes

[Marek Behún]

In order to be able to define a property describing an array of PHY
interface modes, we need to change the current scalar
`phy-connection-type`, which lists the possible PHY interface modes, to
an array of length 1 (otherwise we would need to define the same list at
two different places).

Moreover Rob Herring says that we cannot reuse the values of a property;
we need to $ref a type.

Move the definition of possible PHY interface modes from the
`phy-connection-type` property to an array type definition
`phy-connection-type-array`, and simply reference this type in the
original property.

Signed-off-by: Marek Behún <kabel@kernel.org>
---
 .../bindings/net/ethernet-controller.yaml     | 89 ++++++++++---------
 1 file changed, 48 insertions(+), 41 deletions(-)

diff --git a/Documentation/devicetree/bindings/net/ethernet-controller.yaml b/Documentation/devicetree/bindings/net/ethernet-controller.yaml
index 4b7d1e5d003c..0ee25ecbffde 100644
--- a/Documentation/devicetree/bindings/net/ethernet-controller.yaml
+++ b/Documentation/devicetree/bindings/net/ethernet-controller.yaml
@@ -53,50 +53,12 @@ properties:
     const: mac-address
 
   phy-connection-type:
+    $ref: "#/$defs/phy-connection-type-array"
     description:
       Specifies interface type between the Ethernet device and a physical
       layer (PHY) device.
-    enum:
-      # There is not a standard bus between the MAC and the PHY,
-      # something proprietary is being used to embed the PHY in the
-      # MAC.
-      - internal
-      - mii
-      - gmii
-      - sgmii
-      - qsgmii
-      - tbi
-      - rev-mii
-      - rmii
-
-      # RX and TX delays are added by the MAC when required
-      - rgmii
-
-      # RGMII with internal RX and TX delays provided by the PHY,
-      # the MAC should not add the RX or TX delays in this case
-      - rgmii-id
-
-      # RGMII with internal RX delay provided by the PHY, the MAC
-      # should not add an RX delay in this case
-      - rgmii-rxid
-
-      # RGMII with internal TX delay provided by the PHY, the MAC
-      # should not add an TX delay in this case
-      - rgmii-txid
-      - rtbi
-      - smii
-      - xgmii
-      - trgmii
-      - 1000base-x
-      - 2500base-x
-      - 5gbase-r
-      - rxaui
-      - xaui
-
-      # 10GBASE-KR, XFI, SFI
-      - 10gbase-kr
-      - usxgmii
-      - 10gbase-r
+    minItems: 1
+    maxItems: 1
 
   phy-mode:
     $ref: "#/properties/phy-connection-type"
@@ -226,4 +188,49 @@ properties:
 
 additionalProperties: true
 
+'$defs':
+  phy-connection-type-array:
+    items:
+      enum:
+        # There is not a standard bus between the MAC and the PHY,
+        # something proprietary is being used to embed the PHY in the
+        # MAC.
+        - internal
+        - mii
+        - gmii
+        - sgmii
+        - qsgmii
+        - tbi
+        - rev-mii
+        - rmii
+
+        # RX and TX delays are added by the MAC when required
+        - rgmii
+
+        # RGMII with internal RX and TX delays provided by the PHY,
+        # the MAC should not add the RX or TX delays in this case
+        - rgmii-id
+
+        # RGMII with internal RX delay provided by the PHY, the MAC
+        # should not add an RX delay in this case
+        - rgmii-rxid
+
+        # RGMII with internal TX delay provided by the PHY, the MAC
+        # should not add an TX delay in this case
+        - rgmii-txid
+        - rtbi
+        - smii
+        - xgmii
+        - trgmii
+        - 1000base-x
+        - 2500base-x
+        - 5gbase-r
+        - rxaui
+        - xaui
+
+        # 10GBASE-KR, XFI, SFI
+        - 10gbase-kr
+        - usxgmii
+        - 10gbase-r
+
 ...
-- 
2.26.2

[PATCH net-next 2/2] dt-bindings: ethernet-phy: define `supported-mac-connection-types` property

[Marek Behún]

An ethernet PHY may support PHY interface modes that are not wired on a
specific board (or are broken for some other reason).

For example the Marvell 88X3310 PHY supports connecting the MAC with the
PHY with `xaui` and `rxaui`, the MAC can also support these modes, but
it is possible for a board not to have them wired (since these modes use
multiple SerDes lanes).

In order for the kernel to know which modes are supported on the board,
we need to specify them in the device tree.

Define a new ethernet PHY property `supported-mac-connection-types`,
which lists the supported modes. If this property is missing, all modes
supported by the PHY and MAC are presumed to be supported by the board.

Signed-off-by: Marek Behún <kabel@kernel.org>
---
 .../devicetree/bindings/net/ethernet-phy.yaml  | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/Documentation/devicetree/bindings/net/ethernet-phy.yaml b/Documentation/devicetree/bindings/net/ethernet-phy.yaml
index 2766fe45bb98..3706760b5637 100644
--- a/Documentation/devicetree/bindings/net/ethernet-phy.yaml
+++ b/Documentation/devicetree/bindings/net/ethernet-phy.yaml
@@ -136,6 +136,23 @@ properties:
       used. The absence of this property indicates the muxers
       should be configured so that the external PHY is used.
 
+  supported-mac-connection-types:
+    $ref: "ethernet-controller.yaml#/$defs/phy-connection-type-array"
+    description:
+      The PHY device may support different interface types for
+      connecting the Ethernet MAC device to the PHY device (i.e.
+      rgmii, sgmii, xaui, ...), but not all of these interface
+      types must necessarily be supported for a specific board
+      (not all of them are wired, or there may be some known bug
+      for a specific mode, ...).
+      This property specifies a list of interface modes to the
+      MAC supported by the PHY hardware that are also supported
+      by the board.
+      If this property is missing, all modes supported by the
+      PHY are presumend to be supported by the board.
+    minItems: 1
+    maxItems: 64
+
   resets:
     maxItems: 1
 
@@ -196,5 +213,6 @@ examples:
             reset-gpios = <&gpio1 4 1>;
             reset-assert-us = <1000>;
             reset-deassert-us = <2000>;
+            supported-mac-connection-types = "xaui", "rxaui";
         };
     };
-- 
2.26.2

Re: [PATCH net-next 1/2] dt-bindings: ethernet-controller: create a type for PHY interface modes

[Rob Herring]

On Wed, Mar 24, 2021 at 11:35:55AM +0100, Marek Behún wrote:
> In order to be able to define a property describing an array of PHY
> interface modes, we need to change the current scalar
> `phy-connection-type`, which lists the possible PHY interface modes, to
> an array of length 1 (otherwise we would need to define the same list at
> two different places).
> 
> Moreover Rob Herring says that we cannot reuse the values of a property;
> we need to $ref a type.
> 
> Move the definition of possible PHY interface modes from the
> `phy-connection-type` property to an array type definition
> `phy-connection-type-array`, and simply reference this type in the
> original property.

Why not just extend phy-connection-type to support more than 1 entry?

Re: [PATCH net-next 1/2] dt-bindings: ethernet-controller: create a type for PHY interface modes

[Marek Behún]

On Wed, 24 Mar 2021 14:07:06 -0600
Rob Herring <robh@kernel.org> wrote:

> On Wed, Mar 24, 2021 at 11:35:55AM +0100, Marek Behún wrote:
> > In order to be able to define a property describing an array of PHY
> > interface modes, we need to change the current scalar
> > `phy-connection-type`, which lists the possible PHY interface modes, to
> > an array of length 1 (otherwise we would need to define the same list at
> > two different places).
> > 
> > Moreover Rob Herring says that we cannot reuse the values of a property;
> > we need to $ref a type.
> > 
> > Move the definition of possible PHY interface modes from the
> > `phy-connection-type` property to an array type definition
> > `phy-connection-type-array`, and simply reference this type in the
> > original property.  
> 
> Why not just extend phy-connection-type to support more than 1 entry?

Hmm, that would be even better, although it would complicate the
Russell's marvell10g patches a little if we want the code to be
backward compatible with older device trees.

I will look into this.

Marek

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Florian Fainelli]

On 3/24/2021 3:35 AM, Marek Behún wrote:
> Hello,
> 
> the Marvell Alaska PHYs (88X3310, 88E2110) can, depending on their
> configuration, change the PHY mode to the MAC, depending on the
> copper/fiber speed.
> 
> The 88X3310, for example, can be configured (via MACTYPE register)
> so that it communicates with the MAC in sgmii for 10/100/1000mbps,
> 2500base-x for 2500mbps, 5gbase-r for 5gbps and either 10gbase-r,
> xaui or rxaui for 10gbps. Or the PHY may communicate with the MAC
> in usxgmii, or one of the 10gbase-r, rxaui or xaui modes with rate
> matching.
> 
> So for the 10gbps mode we have options 10gbase-r, xaui, rxaui and
> usxgmii. The MAC can support some of these modes, and if it does more
> than one, we need to know which one to use. Not all of these modes
> must necessarily be supported by the board (xaui required wiring for
> 4 SerDes lanes, for example, and 10gbase-r requires wiring capable
> of transmitting at 10.3125 GBd).
> 
> The MACTYPE is upon HW reset configured by strapping pins - so the
> board should have a correct mode configured after HW reset.
> 
> One problem with this is that some boards configure the MACTYPE to
> a rate matching mode, which, according to the errata, is broken in
> some situations (it does not work for 10/100/1000, for example).
> 
> Another problem is that if lower modes are supported, we should
> maybe use them in order to save power.

That is an interesting proposal but if you want it to be truly valuable,
does not that mean that an user ought to be able to switch between any
of the supported PHY <=> MAC interfaces at runtime, and then within
those interfaces to the speeds that yield the best power savings?

> 
> But for this we need to know which phy-modes are supported on the
> board.
> 
> This series adds documentation for a new ethernet PHY property,
> called `supported-mac-connection-types`.

That naming does not quite make sense to me, if we want to describe the
MAC supported connection types, then those would naturally be within the
Ethernet MAC Device Tree node, no? If we are describing what the PHY is
capable, then we should be dropping "mac" from the property name not to
create confusion.

> 
> When this property is present for a PHY node, only the phy-modes
> listed in this property should be considered to be functional on
> the board.

Can you post the code that is going to utilize these properties so we
have a clearer picture of how and what you need to solve?

> 
> The second patch adds binding for this property.
> 
> The first patch does some YAML magic in ethernet-controller.yaml
> in order to be able to reuse the PHY modes enum, so that the same
> list does not have to be defined twice.
> 
> Marek
> 
> Marek Behún (2):
>   dt-bindings: ethernet-controller: create a type for PHY interface
>     modes
>   dt-bindings: ethernet-phy: define `supported-mac-connection-types`
>     property
> 
>  .../bindings/net/ethernet-controller.yaml     | 89 ++++++++++---------
>  .../devicetree/bindings/net/ethernet-phy.yaml | 18 ++++
>  2 files changed, 66 insertions(+), 41 deletions(-)
> 

-- 
Florian

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Marek Behún]

On Wed, 24 Mar 2021 14:19:28 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

> > Another problem is that if lower modes are supported, we should
> > maybe use them in order to save power.  
> 
> That is an interesting proposal but if you want it to be truly valuable,
> does not that mean that an user ought to be able to switch between any
> of the supported PHY <=> MAC interfaces at runtime, and then within
> those interfaces to the speeds that yield the best power savings?

If the code determines that there are multiple working configurations,
it theoretically could allow the user to switch between them.

My idea was that this should be done by kernel, though.

But power saving is not the main problem I am trying to solve.
What I am trying to solve is that if a board does not support all modes
supported by the MAC and PHY, because they are not wired or something,
we need to know about that so that we can select the correct mode for
PHYs that change this mode at runtime.

> > 
> > But for this we need to know which phy-modes are supported on the
> > board.
> > 
> > This series adds documentation for a new ethernet PHY property,
> > called `supported-mac-connection-types`.  
> 
> That naming does not quite make sense to me, if we want to describe the
> MAC supported connection types, then those would naturally be within the
> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
> capable, then we should be dropping "mac" from the property name not to
> create confusion.

I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
I guess I put "mac" there so that if in the future we wanted to specify
supported modes for the fiber side, we could add
`supported-fiber-connection-types`.

But otherwise it does not matter where this property is. Rob Herring
says that maybe we don't need a new property at all. We can reuse
phy-mode property of the MAC, and enumerate all supported modes there.

> 
> > 
> > When this property is present for a PHY node, only the phy-modes
> > listed in this property should be considered to be functional on
> > the board.  
> 
> Can you post the code that is going to utilize these properties so we
> have a clearer picture of how and what you need to solve?

I am still working on this, so the repo may change, but look at
https://git.kernel.org/pub/scm/linux/kernel/git/kabel/linux.git/log/?h=phy-supported-interfaces
at the last three patches.

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Florian Fainelli]

On 3/24/2021 4:00 PM, Marek Behún wrote:
> On Wed, 24 Mar 2021 14:19:28 -0700
> Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
>>> Another problem is that if lower modes are supported, we should
>>> maybe use them in order to save power.  
>>
>> That is an interesting proposal but if you want it to be truly valuable,
>> does not that mean that an user ought to be able to switch between any
>> of the supported PHY <=> MAC interfaces at runtime, and then within
>> those interfaces to the speeds that yield the best power savings?
> 
> If the code determines that there are multiple working configurations,
> it theoretically could allow the user to switch between them.
> 
> My idea was that this should be done by kernel, though.
> 
> But power saving is not the main problem I am trying to solve.
> What I am trying to solve is that if a board does not support all modes
> supported by the MAC and PHY, because they are not wired or something,
> we need to know about that so that we can select the correct mode for
> PHYs that change this mode at runtime.

OK so the runtime part comes from plugging in various SFP modules into a
cage but other than that, for a "fixed" link such as a SFF or a soldered
down PHY, do we agree that there would be no runtime changing of the
'phy-mode'?

What I am trying to understand is why this needs to be added to the
Device Tree as opposed to a bitmask within the PHY driver that indicates
the various interface mode capabilities which, looking at the code you
shared below, is how you make decisions ultimately.

> 
>>>
>>> But for this we need to know which phy-modes are supported on the
>>> board.
>>>
>>> This series adds documentation for a new ethernet PHY property,
>>> called `supported-mac-connection-types`.  
>>
>> That naming does not quite make sense to me, if we want to describe the
>> MAC supported connection types, then those would naturally be within the
>> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
>> capable, then we should be dropping "mac" from the property name not to
>> create confusion.
> 
> I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
> host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
> I guess I put "mac" there so that if in the future we wanted to specify
> supported modes for the fiber side, we could add
> `supported-fiber-connection-types`.

You would traditionally find the words "line side" (copper, optical,
etc.) and "MAC side" being used in datasheets, maybe you can use a
similar naming here?

> 
> But otherwise it does not matter where this property is. Rob Herring
> says that maybe we don't need a new property at all. We can reuse
> phy-mode property of the MAC, and enumerate all supported modes there.
> 
>>
>>>
>>> When this property is present for a PHY node, only the phy-modes
>>> listed in this property should be considered to be functional on
>>> the board.  
>>
>> Can you post the code that is going to utilize these properties so we
>> have a clearer picture of how and what you need to solve?
> 
> I am still working on this, so the repo may change, but look at
> https://git.kernel.org/pub/scm/linux/kernel/git/kabel/linux.git/log/?h=phy-supported-interfaces
> at the last three patches.
> 

-- 
Florian

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Marek Behún]

On Wed, 24 Mar 2021 16:16:41 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

> On 3/24/2021 4:00 PM, Marek Behún wrote:
> > On Wed, 24 Mar 2021 14:19:28 -0700
> > Florian Fainelli <f.fainelli@gmail.com> wrote:
> >   
> >>> Another problem is that if lower modes are supported, we should
> >>> maybe use them in order to save power.    
> >>
> >> That is an interesting proposal but if you want it to be truly valuable,
> >> does not that mean that an user ought to be able to switch between any
> >> of the supported PHY <=> MAC interfaces at runtime, and then within
> >> those interfaces to the speeds that yield the best power savings?  
> > 
> > If the code determines that there are multiple working configurations,
> > it theoretically could allow the user to switch between them.
> > 
> > My idea was that this should be done by kernel, though.
> > 
> > But power saving is not the main problem I am trying to solve.
> > What I am trying to solve is that if a board does not support all modes
> > supported by the MAC and PHY, because they are not wired or something,
> > we need to know about that so that we can select the correct mode for
> > PHYs that change this mode at runtime.  
> 
> OK so the runtime part comes from plugging in various SFP modules into a
> cage but other than that, for a "fixed" link such as a SFF or a soldered
> down PHY, do we agree that there would be no runtime changing of the
> 'phy-mode'?

No, we do not. The PHY can be configured (by strapping pins or by
sw) to change phy-mode depending on the autonegotiated copper speed.

So if you plug in an ethernet cable where on the otherside is only 1g
capable device, the PHY will change mode to sgmii. But if you then plug
a 5g capable device, the PHY will change mode to 5gbase-r.

This happens if the PHY is configured into one of these changing
configurations. It can also be configured to USXGMII, or 10GBASER with
rate matching.

Not many MACs in kernel support USXGMII currently.

And if you use rate matching mode, and the copper side is
linked in lower speed (2.5g for example), and the MAC will start
sending too many packets, the internal buffer in the PHY is only 16 KB,
so it will fill up quickly. So you need pause frames support. But this
is broken for speeds <= 1g, according to erratum.

So you really want to change modes. The rate matching mode is
basically useless.

> 
> What I am trying to understand is why this needs to be added to the
> Device Tree as opposed to a bitmask within the PHY driver that indicates
> the various interface mode capabilities which, looking at the code you
> shared below, is how you make decisions ultimately.

Because someone can create a board with a SOC where MAC is capable of
all of the following modes: 10gbase-r, xaui, rxaui, 5gbase-r,
2.5gbase-x, sgmii.

And use Marvell 88X3310 PHY to translate to copper.

But only wire the PHY to the MAC with one SerDes lane. So for 10g,
10gbase-r mode must be used, xaui and rxaui cannot.
Or wire the PHY to the MAC with 2 SerDes lanes, but both lanes capable
only of 6 GHz freq. So for 10g, rxaui must be used.

And then make the mistake of wiring the strapping pins to the
rate-matching mode, which is useless.

So we need to know which modes are supported if we want to change the
configuration to a working one.

> >   
> >>>
> >>> But for this we need to know which phy-modes are supported on the
> >>> board.
> >>>
> >>> This series adds documentation for a new ethernet PHY property,
> >>> called `supported-mac-connection-types`.    
> >>
> >> That naming does not quite make sense to me, if we want to describe the
> >> MAC supported connection types, then those would naturally be within the
> >> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
> >> capable, then we should be dropping "mac" from the property name not to
> >> create confusion.  
> > 
> > I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
> > host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
> > I guess I put "mac" there so that if in the future we wanted to specify
> > supported modes for the fiber side, we could add
> > `supported-fiber-connection-types`.  
> 
> You would traditionally find the words "line side" (copper, optical,
> etc.) and "MAC side" being used in datasheets, maybe you can use a
> similar naming here?

So
  supported-connection-types-mac-side
  supported-connection-types-line-side
or maybe media-side?

I am still exploring whether this could be simply defined in the
ethernet controllers `phy-mode` property, as Rob Herring says. It would
be simpler...

Marek

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Florian Fainelli]

On 3/24/2021 4:45 PM, Marek Behún wrote:
> On Wed, 24 Mar 2021 16:16:41 -0700
> Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
>> On 3/24/2021 4:00 PM, Marek Behún wrote:
>>> On Wed, 24 Mar 2021 14:19:28 -0700
>>> Florian Fainelli <f.fainelli@gmail.com> wrote:
>>>   
>>>>> Another problem is that if lower modes are supported, we should
>>>>> maybe use them in order to save power.    
>>>>
>>>> That is an interesting proposal but if you want it to be truly valuable,
>>>> does not that mean that an user ought to be able to switch between any
>>>> of the supported PHY <=> MAC interfaces at runtime, and then within
>>>> those interfaces to the speeds that yield the best power savings?  
>>>
>>> If the code determines that there are multiple working configurations,
>>> it theoretically could allow the user to switch between them.
>>>
>>> My idea was that this should be done by kernel, though.
>>>
>>> But power saving is not the main problem I am trying to solve.
>>> What I am trying to solve is that if a board does not support all modes
>>> supported by the MAC and PHY, because they are not wired or something,
>>> we need to know about that so that we can select the correct mode for
>>> PHYs that change this mode at runtime.  
>>
>> OK so the runtime part comes from plugging in various SFP modules into a
>> cage but other than that, for a "fixed" link such as a SFF or a soldered
>> down PHY, do we agree that there would be no runtime changing of the
>> 'phy-mode'?
> 
> No, we do not. The PHY can be configured (by strapping pins or by
> sw) to change phy-mode depending on the autonegotiated copper speed.
> 
> So if you plug in an ethernet cable where on the otherside is only 1g
> capable device, the PHY will change mode to sgmii. But if you then plug
> a 5g capable device, the PHY will change mode to 5gbase-r.
> 
> This happens if the PHY is configured into one of these changing
> configurations. It can also be configured to USXGMII, or 10GBASER with
> rate matching.
> 
> Not many MACs in kernel support USXGMII currently.
> 
> And if you use rate matching mode, and the copper side is
> linked in lower speed (2.5g for example), and the MAC will start
> sending too many packets, the internal buffer in the PHY is only 16 KB,
> so it will fill up quickly. So you need pause frames support. But this
> is broken for speeds <= 1g, according to erratum.
> 
> So you really want to change modes. The rate matching mode is
> basically useless.

OK, so whenever there is a link change you are presumably reading the
mode in which the PHY has been reconfigured to, asking the MAC to
configured itself appropriately based on that, and if there is no
intersection, error out?

> 
>>
>> What I am trying to understand is why this needs to be added to the
>> Device Tree as opposed to a bitmask within the PHY driver that indicates
>> the various interface mode capabilities which, looking at the code you
>> shared below, is how you make decisions ultimately.
> 
> Because someone can create a board with a SOC where MAC is capable of
> all of the following modes: 10gbase-r, xaui, rxaui, 5gbase-r,
> 2.5gbase-x, sgmii.
> 
> And use Marvell 88X3310 PHY to translate to copper.
> 
> But only wire the PHY to the MAC with one SerDes lane. So for 10g,
> 10gbase-r mode must be used, xaui and rxaui cannot.
> Or wire the PHY to the MAC with 2 SerDes lanes, but both lanes capable
> only of 6 GHz freq. So for 10g, rxaui must be used.
> 
> And then make the mistake of wiring the strapping pins to the
> rate-matching mode, which is useless.
> 
> So we need to know which modes are supported if we want to change the
> configuration to a working one.

OK, so you need to know the PCB limitations which would be coming via
Device Tree and what mode the PHY has been configured into at the time
you attach/connect to the PHY which you could read from the device itself.

> 
>>>   
>>>>>
>>>>> But for this we need to know which phy-modes are supported on the
>>>>> board.
>>>>>
>>>>> This series adds documentation for a new ethernet PHY property,
>>>>> called `supported-mac-connection-types`.    
>>>>
>>>> That naming does not quite make sense to me, if we want to describe the
>>>> MAC supported connection types, then those would naturally be within the
>>>> Ethernet MAC Device Tree node, no? If we are describing what the PHY is
>>>> capable, then we should be dropping "mac" from the property name not to
>>>> create confusion.  
>>>
>>> I put "mac" there to indicate that this is the SerDes to the MAC (i.e.
>>> host side in Marvell PHY). 88X3310 has another SerDes side (Fiber Side).
>>> I guess I put "mac" there so that if in the future we wanted to specify
>>> supported modes for the fiber side, we could add
>>> `supported-fiber-connection-types`.  
>>
>> You would traditionally find the words "line side" (copper, optical,
>> etc.) and "MAC side" being used in datasheets, maybe you can use a
>> similar naming here?
> 
> So
>   supported-connection-types-mac-side
>   supported-connection-types-line-side
> or maybe media-side?

Yes, that sounds a bit better and more descriptive.

> 
> I am still exploring whether this could be simply defined in the
> ethernet controllers `phy-mode` property, as Rob Herring says. It would
> be simpler...

OK.
-- 
Florian

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Russell King - ARM Linux admin]

On Thu, Mar 25, 2021 at 12:45:25AM +0100, Marek Behún wrote:
> On Wed, 24 Mar 2021 16:16:41 -0700
> Florian Fainelli <f.fainelli@gmail.com> wrote:
> 
> > On 3/24/2021 4:00 PM, Marek Behún wrote:
> > > On Wed, 24 Mar 2021 14:19:28 -0700
> > > Florian Fainelli <f.fainelli@gmail.com> wrote:
> > >   
> > >>> Another problem is that if lower modes are supported, we should
> > >>> maybe use them in order to save power.    
> > >>
> > >> That is an interesting proposal but if you want it to be truly valuable,
> > >> does not that mean that an user ought to be able to switch between any
> > >> of the supported PHY <=> MAC interfaces at runtime, and then within
> > >> those interfaces to the speeds that yield the best power savings?  
> > > 
> > > If the code determines that there are multiple working configurations,
> > > it theoretically could allow the user to switch between them.
> > > 
> > > My idea was that this should be done by kernel, though.
> > > 
> > > But power saving is not the main problem I am trying to solve.
> > > What I am trying to solve is that if a board does not support all modes
> > > supported by the MAC and PHY, because they are not wired or something,
> > > we need to know about that so that we can select the correct mode for
> > > PHYs that change this mode at runtime.  
> > 
> > OK so the runtime part comes from plugging in various SFP modules into a
> > cage but other than that, for a "fixed" link such as a SFF or a soldered
> > down PHY, do we agree that there would be no runtime changing of the
> > 'phy-mode'?
> 
> No, we do not. The PHY can be configured (by strapping pins or by
> sw) to change phy-mode depending on the autonegotiated copper speed.
> 
> So if you plug in an ethernet cable where on the otherside is only 1g
> capable device, the PHY will change mode to sgmii. But if you then plug
> a 5g capable device, the PHY will change mode to 5gbase-r.
> 
> This happens if the PHY is configured into one of these changing
> configurations. It can also be configured to USXGMII, or 10GBASER with
> rate matching.
> 
> Not many MACs in kernel support USXGMII currently.
> 
> And if you use rate matching mode, and the copper side is
> linked in lower speed (2.5g for example), and the MAC will start
> sending too many packets, the internal buffer in the PHY is only 16 KB,
> so it will fill up quickly. So you need pause frames support. But this
> is broken for speeds <= 1g, according to erratum.

Also, the sending of pause frames is only supported for 88x3310P
devices, not the 88x3310. The plain 88x3310 requires the MAC to
rate-limit in this mode.

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Russell King - ARM Linux admin]

On Wed, Mar 24, 2021 at 05:11:25PM -0700, Florian Fainelli wrote:
> > And if you use rate matching mode, and the copper side is
> > linked in lower speed (2.5g for example), and the MAC will start
> > sending too many packets, the internal buffer in the PHY is only 16 KB,
> > so it will fill up quickly. So you need pause frames support. But this
> > is broken for speeds <= 1g, according to erratum.
> > 
> > So you really want to change modes. The rate matching mode is
> > basically useless.
> 
> OK, so whenever there is a link change you are presumably reading the
> mode in which the PHY has been reconfigured to, asking the MAC to
> configured itself appropriately based on that, and if there is no
> intersection, error out?

The 88x3310 already tells the MAC what the interface mode is.

I think what Marc is trying to do is to program the PHY to use the
appropriate _group_ of modes for the MAC connection and then letting
the PHY get on with it, rather than explicitly trying to resolve the
mode (which likely won't work for the 88x3310, since changing the
MAC side mode requires resetting the entire PHY - which will cause
any media side link to be dropped.)

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!

Re: [PATCH net-next 0/2] dt-bindings: define property describing supported ethernet PHY modes

[Marek Behún]

On Wed, 24 Mar 2021 17:11:25 -0700
Florian Fainelli <f.fainelli@gmail.com> wrote:

> On 3/24/2021 4:45 PM, Marek Behún wrote:
> > On Wed, 24 Mar 2021 16:16:41 -0700
> > Florian Fainelli <f.fainelli@gmail.com> wrote:
> >   
> >> On 3/24/2021 4:00 PM, Marek Behún wrote:  
> >>> On Wed, 24 Mar 2021 14:19:28 -0700
> >>> Florian Fainelli <f.fainelli@gmail.com> wrote:
> >>>     
> >>>>> Another problem is that if lower modes are supported, we should
> >>>>> maybe use them in order to save power.      
> >>>>
> >>>> That is an interesting proposal but if you want it to be truly valuable,
> >>>> does not that mean that an user ought to be able to switch between any
> >>>> of the supported PHY <=> MAC interfaces at runtime, and then within
> >>>> those interfaces to the speeds that yield the best power savings?    
> >>>
> >>> If the code determines that there are multiple working configurations,
> >>> it theoretically could allow the user to switch between them.
> >>>
> >>> My idea was that this should be done by kernel, though.
> >>>
> >>> But power saving is not the main problem I am trying to solve.
> >>> What I am trying to solve is that if a board does not support all modes
> >>> supported by the MAC and PHY, because they are not wired or something,
> >>> we need to know about that so that we can select the correct mode for
> >>> PHYs that change this mode at runtime.    
> >>
> >> OK so the runtime part comes from plugging in various SFP modules into a
> >> cage but other than that, for a "fixed" link such as a SFF or a soldered
> >> down PHY, do we agree that there would be no runtime changing of the
> >> 'phy-mode'?  
> > 
> > No, we do not. The PHY can be configured (by strapping pins or by
> > sw) to change phy-mode depending on the autonegotiated copper speed.
> > 
> > So if you plug in an ethernet cable where on the otherside is only 1g
> > capable device, the PHY will change mode to sgmii. But if you then plug
> > a 5g capable device, the PHY will change mode to 5gbase-r.
> > 
> > This happens if the PHY is configured into one of these changing
> > configurations. It can also be configured to USXGMII, or 10GBASER with
> > rate matching.
> > 
> > Not many MACs in kernel support USXGMII currently.
> > 
> > And if you use rate matching mode, and the copper side is
> > linked in lower speed (2.5g for example), and the MAC will start
> > sending too many packets, the internal buffer in the PHY is only 16 KB,
> > so it will fill up quickly. So you need pause frames support. But this
> > is broken for speeds <= 1g, according to erratum.
> > 
> > So you really want to change modes. The rate matching mode is
> > basically useless.  
> 
> OK, so whenever there is a link change you are presumably reading the
> mode in which the PHY has been reconfigured to, asking the MAC to
> configured itself appropriately based on that, and if there is no
> intersection, error out?

No. At initialization I tell the PHY to change between
  10gbase-r / 5gbase-r / 2500base-x / sgmii
according to the copper side. The PHY will do this alone on change on
copper side. I don't need to do this.

(This already works with current version of marvell10g driver - but
 kernel is not configuring this, it has to be configure via strapping
 pins.)

But I can tell the PHY at initialization to change instead between
  xaui / 5gbase-r / 2500base-x / sgmii
Again the PHY will do this on its own whenever speed on the copper side
changes.

But I need to know which of this settings I should use.

Marek