DSA with MV88E6321 and imx28

DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

Hi all,

I'm working on a custom board with a 88E6321 and an i.MX28. Port 5 is directly connected per RMII to the CPU. 
The switch is running in CPU attached mode. On Port 2 and 6 we have 2 external Micrel KSZ9031 PHYs.
Here is the snip of my device tree:

&mac0 {
	pinctrl-0 = <&mac0_pins_a &mac0_freigabe &mac0_lcd_d04>;
	phy-supply = <&reg_3p3v>;
	status = "okay";

	fixed-link = <1 1 100 0 0>;

	/* this is done to remove enet_out */
	clocks = <&clks 57>, <&clks 57>;
	clock-names = "ipg", "ahb";

	/delete-property/ phy-reset-gpios;
	/delete-property/ phy-reset-duration;
	freigabe-gpios = <&gpio0 26 GPIO_ACTIVE_HIGH>;
	trigger-gpios = <&gpio1 3 GPIO_ACTIVE_HIGH>;

	mdio {
                #address-cells = <1>;
                #size-cells = <0>;

		switch0: switch0@10 {
                        compatible = "marvell,mv88e6085";
                        reg = <0x10>;
			pinctrl-0 = <&lcd_d06_pins>;
			reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;

                        dsa,member = <0 0>;

                        ports {
                                #address-cells = <1>;
                                #size-cells = <0>;

                                port@0 {
                                        reg = <0x0>;
                                        label = "Serdes0";
                                        phy-handle = <&switch0phy0>;
                                };

                                port@1 {
                                        reg = <0x1>;
                                        label = "Serdes1";
                                        phy-handle = <&switch0phy1>;
                                };

                                port@2 {
					reg = <0x2>;
                                        label = "lan1";
                                        phy-handle = <&switch0phy2>;
                                };

                                port@3 {
                                        reg = <0x3>;
                                        label = "lan2";
                                        phy-handle = <&switch0phy3>;
                                };

                                port@4 {
                                        reg = <0x4>;
                                        label = "lan3";
                                        phy-handle = <&switch0phy4>;
                                };

                                port5 {
                                        reg = <0x5>;
                                        label = "cpu";
                                        ethernet = <&mac0>;
                                        phy-mode = "rmii";
                                        fixed-link {
                                                speed = <100>;
                                                full-duplex;
                                        };
                                };

				port@6 {
                                        reg = <0x6>;
                                        label = "lan4";
                                        phy-handle = <&switch0phy6>;
                                };
                        };

			mdio {
				#address-cells = <1>;
				#size-cells = <0>;
				switch0phy0: switch0phy0@0 {
					reg = <0xc>;
				};
				switch0phy1: switch0phy1@1 {
				        reg = <0xd>;
				};
				switch0phy2: switch0phy2@2 {
				        reg = <0x2>;
				};
				switch0phy3: switch0phy3@3 {
				        reg = <0x3>;
				};
				switch0phy4: switch0phy4@4 {
				        reg = <0x4>;
				};
				switch0phy6: switch0phy6@6 {
				        reg = <0x6>;
				};
			};
		};
        };
}; 

I'm sure it must be wrong. Does the mdio part in between the switch part is for the internal
mdio bus? 
From the outside I can read and write the SMI Register 0x10-0x16. 

Here is a snip from the bootup

[    1.377362] at24 0-0051: 256 byte 24c02 EEPROM, writable, 32 bytes/write
[    1.391046] libphy: Fixed MDIO Bus: probed
[    1.396763] libphy: mdio_driver_register: mv88e6085
[    1.407168] fec 800f0000.ethernet (unnamed net_device) (uninitialized): Invalid MAC address: 00:00:00:00:00:00
[    1.417279] fec 800f0000.ethernet (unnamed net_device) (uninitialized): Using random MAC address: 86:50:72:5d:79:ad
[    1.429918] libphy: fec_enet_mii_bus: probed
[    1.434374] mdio_bus 800f0000.ethernet-1:10: mdio_device_register
---
[   18.735835] Generic PHY fixed-0:00: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:00, irq=-1)

On the other board it was at least recognized but now I don't get anything about the switch.

Here I'm running 4.9.109. I've tested newer kernels but then the MDIO bus can not be accessed anymore.
I've tested 4.19.47 and 4.14.179. 

@Andrew Lunn I haven't forgotten to answer your the last mail. But this product has a higher
priority so I will come back later to the other custom board again.

Thanks in advance.

Cheers,
Benjamin Beckmeyer

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

On Tue, Jun 04, 2019 at 03:07:25PM +0200, Benjamin Beckmeyer wrote:
> Hi all,
> 
> I'm working on a custom board with a 88E6321 and an i.MX28. Port 5 is directly connected per RMII to the CPU. 
> The switch is running in CPU attached mode. On Port 2 and 6 we have 2 external Micrel KSZ9031 PHYs.
> Here is the snip of my device tree:
> 
> &mac0 {
> 	pinctrl-0 = <&mac0_pins_a &mac0_freigabe &mac0_lcd_d04>;
> 	phy-supply = <&reg_3p3v>;
> 	status = "okay";
> 
> 	fixed-link = <1 1 100 0 0>;

Hi Benjamin

That is the old format for a fixed-link. Please use the new one.

> 
> 	/* this is done to remove enet_out */
> 	clocks = <&clks 57>, <&clks 57>;
> 	clock-names = "ipg", "ahb";
> 
> 	/delete-property/ phy-reset-gpios;
> 	/delete-property/ phy-reset-duration;
> 	freigabe-gpios = <&gpio0 26 GPIO_ACTIVE_HIGH>;

German in device tree? 

> 	trigger-gpios = <&gpio1 3 GPIO_ACTIVE_HIGH>;
> 
> 	mdio {
>                 #address-cells = <1>;
>                 #size-cells = <0>;
> 
> 		switch0: switch0@10 {
>                         compatible = "marvell,mv88e6085";
>                         reg = <0x10>;

So you have the switch strapped to use a single address?

> 			pinctrl-0 = <&lcd_d06_pins>;

lcd ?

> 			reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
> 
>                         dsa,member = <0 0>;
> 
>                         ports {
>                                 #address-cells = <1>;
>                                 #size-cells = <0>;
> 
>                                 port@0 {
>                                         reg = <0x0>;
>                                         label = "Serdes0";
>                                         phy-handle = <&switch0phy0>;
>                                 };
> 
>                                 port@1 {
>                                         reg = <0x1>;
>                                         label = "Serdes1";
>                                         phy-handle = <&switch0phy1>;
>                                 };
> 
>                                 port@2 {
> 					reg = <0x2>;
>                                         label = "lan1";
>                                         phy-handle = <&switch0phy2>;
>                                 };
> 
>                                 port@3 {
>                                         reg = <0x3>;
>                                         label = "lan2";
>                                         phy-handle = <&switch0phy3>;
>                                 };
> 
>                                 port@4 {
>                                         reg = <0x4>;
>                                         label = "lan3";
>                                         phy-handle = <&switch0phy4>;
>                                 };
> 
>                                 port5 {
>                                         reg = <0x5>;
>                                         label = "cpu";
>                                         ethernet = <&mac0>;
>                                         phy-mode = "rmii";
>                                         fixed-link {
>                                                 speed = <100>;
>                                                 full-duplex;
>                                         };
>                                 };
> 
> 				port@6 {
>                                         reg = <0x6>;
>                                         label = "lan4";
>                                         phy-handle = <&switch0phy6>;
>                                 };
>                         };
> 
> 			mdio {
> 				#address-cells = <1>;
> 				#size-cells = <0>;
> 				switch0phy0: switch0phy0@0 {
> 					reg = <0xc>;
> 				};
> 				switch0phy1: switch0phy1@1 {
> 				        reg = <0xd>;
> 				};
> 				switch0phy2: switch0phy2@2 {
> 				        reg = <0x2>;
> 				};
> 				switch0phy3: switch0phy3@3 {
> 				        reg = <0x3>;
> 				};
> 				switch0phy4: switch0phy4@4 {
> 				        reg = <0x4>;
> 				};
> 				switch0phy6: switch0phy6@6 {
> 				        reg = <0x6>;
> 				};
> 			};
> 		};
>         };
> }; 
> 
> I'm sure it must be wrong. Does the mdio part in between the switch part is for the internal
> mdio bus? 

It is for the switch MDIO bus. For this generation of switch, it is
both internal and external. Later generations have two MDIO busses,
one internal and one external.

> >From the outside I can read and write the SMI Register 0x10-0x16. 
> 
> Here is a snip from the bootup
> 
> [    1.377362] at24 0-0051: 256 byte 24c02 EEPROM, writable, 32 bytes/write
> [    1.391046] libphy: Fixed MDIO Bus: probed
> [    1.396763] libphy: mdio_driver_register: mv88e6085
> [    1.407168] fec 800f0000.ethernet (unnamed net_device) (uninitialized): Invalid MAC address: 00:00:00:00:00:00
> [    1.417279] fec 800f0000.ethernet (unnamed net_device) (uninitialized): Using random MAC address: 86:50:72:5d:79:ad
> [    1.429918] libphy: fec_enet_mii_bus: probed
> [    1.434374] mdio_bus 800f0000.ethernet-1:10: mdio_device_register
> ---
> [   18.735835] Generic PHY fixed-0:00: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:00, irq=-1)

You probably want to put some printk mv88e6xxx_detect(). Is it getting
called? What value does id have? When these switches are held in
reset, they don't respond on the bus at all, so you will get MDIO
reads of 0xffff.

      Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

Hey Andrew,
thanks for you reply. 
I got the devicetree from somebody that is why German is in it. But first I wanted to get it running before I tidy it up.
The switch is strapped to single mode (so I can read SMI addresses 0x10-0x16 and 0x1b-0x1e directly). Do I have to
tell this the devicetree? I thought the driver will recognized that and I only give it the start SMI address of the switch?
I can read the internal PHYs with the indirectly read over the SMI COMMAND and SMI DATA registers.

The switch is not held in reset anymore, I take it out of reset in barebox manually every start, not a good solution but
for the moment to get it running it is okay.

The LCD port is just a GPIO. Like I said before, the devicetree is not completely from me. I've already started to write 
a new one but then the MDIO bus can't be accessed anymore.

I will put some printk's in that function and see what is happening. What I think is confusing me, the switch is working 
with internal addresses from 0-6 for the ports and you can read the port status registers at 0x10-0x16. And I'm not sure
which addresses should be in the switch mdio part.

Cheers,
Benjamin

> On Tue, Jun 04, 2019 at 03:07:25PM +0200, Benjamin Beckmeyer wrote:
>> Hi all,
>>
>> I'm working on a custom board with a 88E6321 and an i.MX28. Port 5 is directly connected per RMII to the CPU. 
>> The switch is running in CPU attached mode. On Port 2 and 6 we have 2 external Micrel KSZ9031 PHYs.
>> Here is the snip of my device tree:
>>
>> &mac0 {
>> 	pinctrl-0 = <&mac0_pins_a &mac0_freigabe &mac0_lcd_d04>;
>> 	phy-supply = <&reg_3p3v>;
>> 	status = "okay";
>>
>> 	fixed-link = <1 1 100 0 0>;
> Hi Benjamin
>
> That is the old format for a fixed-link. Please use the new one.
>
>> 	/* this is done to remove enet_out */
>> 	clocks = <&clks 57>, <&clks 57>;
>> 	clock-names = "ipg", "ahb";
>>
>> 	/delete-property/ phy-reset-gpios;
>> 	/delete-property/ phy-reset-duration;
>> 	freigabe-gpios = <&gpio0 26 GPIO_ACTIVE_HIGH>;
> German in device tree? 
>
>> 	trigger-gpios = <&gpio1 3 GPIO_ACTIVE_HIGH>;
>>
>> 	mdio {
>>                 #address-cells = <1>;
>>                 #size-cells = <0>;
>>
>> 		switch0: switch0@10 {
>>                         compatible = "marvell,mv88e6085";
>>                         reg = <0x10>;
> So you have the switch strapped to use a single address?
>
>> 			pinctrl-0 = <&lcd_d06_pins>;
> lcd ?
>
>> 			reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
>>
>>                         dsa,member = <0 0>;
>>
>>                         ports {
>>                                 #address-cells = <1>;
>>                                 #size-cells = <0>;
>>
>>                                 port@0 {
>>                                         reg = <0x0>;
>>                                         label = "Serdes0";
>>                                         phy-handle = <&switch0phy0>;
>>                                 };
>>
>>                                 port@1 {
>>                                         reg = <0x1>;
>>                                         label = "Serdes1";
>>                                         phy-handle = <&switch0phy1>;
>>                                 };
>>
>>                                 port@2 {
>> 					reg = <0x2>;
>>                                         label = "lan1";
>>                                         phy-handle = <&switch0phy2>;
>>                                 };
>>
>>                                 port@3 {
>>                                         reg = <0x3>;
>>                                         label = "lan2";
>>                                         phy-handle = <&switch0phy3>;
>>                                 };
>>
>>                                 port@4 {
>>                                         reg = <0x4>;
>>                                         label = "lan3";
>>                                         phy-handle = <&switch0phy4>;
>>                                 };
>>
>>                                 port5 {
>>                                         reg = <0x5>;
>>                                         label = "cpu";
>>                                         ethernet = <&mac0>;
>>                                         phy-mode = "rmii";
>>                                         fixed-link {
>>                                                 speed = <100>;
>>                                                 full-duplex;
>>                                         };
>>                                 };
>>
>> 				port@6 {
>>                                         reg = <0x6>;
>>                                         label = "lan4";
>>                                         phy-handle = <&switch0phy6>;
>>                                 };
>>                         };
>>
>> 			mdio {
>> 				#address-cells = <1>;
>> 				#size-cells = <0>;
>> 				switch0phy0: switch0phy0@0 {
>> 					reg = <0xc>;
>> 				};
>> 				switch0phy1: switch0phy1@1 {
>> 				        reg = <0xd>;
>> 				};
>> 				switch0phy2: switch0phy2@2 {
>> 				        reg = <0x2>;
>> 				};
>> 				switch0phy3: switch0phy3@3 {
>> 				        reg = <0x3>;
>> 				};
>> 				switch0phy4: switch0phy4@4 {
>> 				        reg = <0x4>;
>> 				};
>> 				switch0phy6: switch0phy6@6 {
>> 				        reg = <0x6>;
>> 				};
>> 			};
>> 		};
>>         };
>> }; 
>>
>> I'm sure it must be wrong. Does the mdio part in between the switch part is for the internal
>> mdio bus? 
> It is for the switch MDIO bus. For this generation of switch, it is
> both internal and external. Later generations have two MDIO busses,
> one internal and one external.
>
>> >From the outside I can read and write the SMI Register 0x10-0x16. 
>>
>> Here is a snip from the bootup
>>
>> [    1.377362] at24 0-0051: 256 byte 24c02 EEPROM, writable, 32 bytes/write
>> [    1.391046] libphy: Fixed MDIO Bus: probed
>> [    1.396763] libphy: mdio_driver_register: mv88e6085
>> [    1.407168] fec 800f0000.ethernet (unnamed net_device) (uninitialized): Invalid MAC address: 00:00:00:00:00:00
>> [    1.417279] fec 800f0000.ethernet (unnamed net_device) (uninitialized): Using random MAC address: 86:50:72:5d:79:ad
>> [    1.429918] libphy: fec_enet_mii_bus: probed
>> [    1.434374] mdio_bus 800f0000.ethernet-1:10: mdio_device_register
>> ---
>> [   18.735835] Generic PHY fixed-0:00: attached PHY driver [Generic PHY] (mii_bus:phy_addr=fixed-0:00, irq=-1)
> You probably want to put some printk mv88e6xxx_detect(). Is it getting
> called? What value does id have? When these switches are held in
> reset, they don't respond on the bus at all, so you will get MDIO
> reads of 0xffff.
>
>       Andrew

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

> I got the devicetree from somebody that is why German is in it. But
> first I wanted to get it running before I tidy it up.  The switch is
> strapped to single mode (so I can read SMI addresses 0x10-0x16 and
> 0x1b-0x1e directly).

Hi Benjamin

You have miss-understood what reg means.

There are three addressing modes used by the various switches,
although most only support two.

In multi-chip mode, it occupies one address, and there are two
registers used to multiplex access to the underlying registers.  In
this setup, you use reg=<X> to indicate the switch is using address X.

In single mode, it occupies all addresses on the MDIO bus, but many
are reserved. In this mode you use reg=<0>.

A few chips support dual mode, where you can have two switches on one
MDIO bus, one using 0x0-0xf, and the second using 0x10-0x1f. Here you
use reg=<0> or reg=<16>.

Try setting reg=<0> if you have it in single mode.

    Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

>> I got the devicetree from somebody that is why German is in it. But
>> first I wanted to get it running before I tidy it up.  The switch is
>> strapped to single mode (so I can read SMI addresses 0x10-0x16 and
>> 0x1b-0x1e directly).
> Hi Benjamin
>
> You have miss-understood what reg means.
>
> There are three addressing modes used by the various switches,
> although most only support two.
>
> In multi-chip mode, it occupies one address, and there are two
> registers used to multiplex access to the underlying registers.  In
> this setup, you use reg=<X> to indicate the switch is using address X.
>
> In single mode, it occupies all addresses on the MDIO bus, but many
> are reserved. In this mode you use reg=<0>.
>
> A few chips support dual mode, where you can have two switches on one
> MDIO bus, one using 0x0-0xf, and the second using 0x10-0x1f. Here you
> use reg=<0> or reg=<16>.
>
> Try setting reg=<0> if you have it in single mode.
>
>     Andrew

Hi Andrew, 

thanks a lot. That was the hint I needed. Now the DSA ist recognized. 
But the external PHYs are not recognized and the Serdes Ports neither.

Here is my device tree

mdio { 
                #address-cells = <1>;
                #size-cells = <0>;

                switch0: switch0@0 {
                        compatible = "marvell,mv88e6085";
                        reg = <0x0>;
                        pinctrl-0 = <&lcd_d06_pins>;
                        reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;

                        dsa,member = <0 0>;

                        ports {
                                #address-cells = <1>;
                                #size-cells = <0>;

                                port@0 {
/* Changed reg to 0xc too, same error message */
                                        reg = <0x0>;
                                        label = "Serdes0";
                                        phy-handle = <&switch0phy0>;
                                };

                                port@1 {
/* Changed reg to 0xd too, same error message */
                                        reg = <0x1>;
                                        label = "Serdes1";
                                        phy-handle = <&switch0phy1>;
                                };
				
				port@2 {
                                        reg = <0x2>;
                                        label = "lan1";
                                        phy-handle = <&switch0phy2>;
                                };

                                port@3 {
                                        reg = <0x3>;
                                        label = "lan2";
                                        phy-handle = <&switch0phy3>;
                                };

                                port@4 {
                                        reg = <0x4>;
                                        label = "lan3";
                                        phy-handle = <&switch0phy4>;
                                };

                                port5 {
                                        reg = <0x5>;
                                        label = "cpu";
                                        ethernet = <&mac0>;
                                        phy-mode = "rmii";
                                        fixed-link {
                                                speed = <100>;
                                                full-duplex;
                                        };
				};

                                port@6 {
                                        reg = <0x6>;
                                        label = "lan4";
                                        phy-handle = <&switch0phy6>;
                                };
                        };

                        mdio {
                                #address-cells = <1>;
                                #size-cells = <0>;
                                switch0phy0: switch0phy0@0 {
/* Changed reg to 0 too, same error message */
					 reg = <0xc>;
                                };
                                switch0phy1: switch0phy1@1 {
/* Changed reg to 1 too, same error message */
                                        reg = <0xd>;
                                };
                                switch0phy3: switch0phy3@3 {
                                        reg = <0x3>;
                                };
                                switch0phy4: switch0phy4@4 {
                                        reg = <0x4>;
                                };
                        };

                        mdio1 {
				compatible = "marvell,mv88e6xxx-mdio-external";
                                #address-cells = <1>;
                                #size-cells = <0>;

                                switch0phy2: switch0phy2@2 {
                                        reg = <0x2>;
                                };
                                switch0phy6: switch0phy6@6 {
                                        reg = <0x6>;
                                };
                        };
                };
        };
				
and here the bootup.
--snip
mv88e6085 800f0000.ethernet-1:00: switch 0x310 detected: Marvell 88E6321, revision 2
[    1.737480] libphy: /ahb@80080000/ethernet@800f0000/mdio/switch0@10: probed
[    1.754443] DSA: switch 0 0 parsed
[    1.757915] DSA: tree 0 parsed
[    1.825228] random: fast init done
[    1.847046] mv88e6085 800f0000.ethernet-1:00 Serdes0 (uninitialized): no phy at 0
[    1.854597] mv88e6085 800f0000.ethernet-1:00 Serdes0 (uninitialized): failed to connect to port 0: -19
[    1.863968] fec 800f0000.ethernet eth0: error -19 setting up slave phy
[    1.870591] mv88e6085 800f0000.ethernet-1:00: Failed to create slave 0: -19
[    1.877821] mv88e6085 800f0000.ethernet-1:00 Serdes1 (uninitialized): no phy at 1
[    1.885365] mv88e6085 800f0000.ethernet-1:00 Serdes1 (uninitialized): failed to connect to port 1: -19
[    1.894736] fec 800f0000.ethernet eth0: error -19 setting up slave phy
[    1.901359] mv88e6085 800f0000.ethernet-1:00: Failed to create slave 1: -19
[    1.908568] mv88e6085 800f0000.ethernet-1:00 lan1 (uninitialized): no phy at 2
[    1.915849] mv88e6085 800f0000.ethernet-1:00 lan1 (uninitialized): failed to connect to phy2: -19
[    1.925106] fec 800f0000.ethernet eth0: error -19 setting up slave phy
[    1.931721] mv88e6085 800f0000.ethernet-1:00: Failed to create slave 2: -19
[    1.943446] Generic PHY !ahb@80080000!ethernet@800f0000!mdio!switch0@10:03: attached PHY driver [Generic PHY] (mii_bus:phy_addr=!ahb@80080000!ethernet@800f0000!mdio!switch0@10:03, irq=-1)
[    1.973026] Generic PHY !ahb@80080000!ethernet@800f0000!mdio!switch0@10:04: attached PHY driver [Generic PHY] (mii_bus:phy_addr=!ahb@80080000!ethernet@800f0000!mdio!switch0@10:04, irq=-1)
[    1.993580] mv88e6085 800f0000.ethernet-1:00 lan4 (uninitialized): no phy at 6
[    2.000896] mv88e6085 800f0000.ethernet-1:00 lan4 (uninitialized): failed to connect to phy6: -19
[    2.009848] fec 800f0000.ethernet eth0: error -19 setting up slave phy
[    2.016475] mv88e6085 800f0000.ethernet-1:00: Failed to create slave 6: -19
--snip

Same error occurs when I put PHY 2 and 6 into the mdio part above 
(the PHYs are working properly in forwarding mode without DSA).

But I can read the internal PHY Identifier register (0x2 and 0x3)
over indirect mode per mdio. But I can't read the registers of the external PHYs.

Thanks in advance.
Benjamin 

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

> Here is my device tree
> 
> mdio { 
>                 #address-cells = <1>;
>                 #size-cells = <0>;
> 
>                 switch0: switch0@0 {
>                         compatible = "marvell,mv88e6085";
>                         reg = <0x0>;
>                         pinctrl-0 = <&lcd_d06_pins>;
>                         reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
> 
>                         dsa,member = <0 0>;
> 
>                         ports {
>                                 #address-cells = <1>;
>                                 #size-cells = <0>;
> 
>                                 port@0 {
> /* Changed reg to 0xc too, same error message */
>                                         reg = <0x0>;
>                                         label = "Serdes0";
>                                         phy-handle = <&switch0phy0>;
>                                 };
> 
>                                 port@1 {
> /* Changed reg to 0xd too, same error message */
>                                         reg = <0x1>;
>                                         label = "Serdes1";
>                                         phy-handle = <&switch0phy1>;
>                                 };
> 				
> 				port@2 {
>                                         reg = <0x2>;
>                                         label = "lan1";
>                                         phy-handle = <&switch0phy2>;
>                                 };
> 
>                                 port@3 {
>                                         reg = <0x3>;
>                                         label = "lan2";
>                                         phy-handle = <&switch0phy3>;
>                                 };
> 
>                                 port@4 {
>                                         reg = <0x4>;
>                                         label = "lan3";
>                                         phy-handle = <&switch0phy4>;
>                                 };

You don't need to list phy-handle for the internal PHYs. It should
just find them.

>                         mdio {
>                                 #address-cells = <1>;
>                                 #size-cells = <0>;
>                                 switch0phy0: switch0phy0@0 {
> /* Changed reg to 0 too, same error message */
> 					 reg = <0xc>;
>                                 };
>                                 switch0phy1: switch0phy1@1 {
> /* Changed reg to 1 too, same error message */
>                                         reg = <0xd>;
>                                 };
>                                 switch0phy3: switch0phy3@3 {
>                                         reg = <0x3>;
>                                 };
>                                 switch0phy4: switch0phy4@4 {
>                                         reg = <0x4>;
>                                 };
>                         };
> 
>                         mdio1 {
> 				compatible = "marvell,mv88e6xxx-mdio-external";
>                                 #address-cells = <1>;
>                                 #size-cells = <0>;
> 
>                                 switch0phy2: switch0phy2@2 {
>                                         reg = <0x2>;
>                                 };
>                                 switch0phy6: switch0phy6@6 {
>                                         reg = <0x6>;
>                                 };
>                         };

I doubt this second mdio bus is correct. The 6390 uses that, but i
don't think any other family does. The older generations have one MDIO
bus for both internal and external PHYs.

One other idea. Take a look at the data sheet. Can the MDIO pins also
be used for GPIO? Do they default to GPIO or MDIO? For the 6390 they
default to GPIO and there is code to reconfigure them for MDIO.

	Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

>> Here is my device tree
>>
>> mdio { 
>>                 #address-cells = <1>;
>>                 #size-cells = <0>;
>>
>>                 switch0: switch0@0 {
>>                         compatible = "marvell,mv88e6085";
>>                         reg = <0x0>;
>>                         pinctrl-0 = <&lcd_d06_pins>;
>>                         reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
>>
>>                         dsa,member = <0 0>;
>>
>>                         ports {
>>                                 #address-cells = <1>;
>>                                 #size-cells = <0>;
>>
>>                                 port@0 {
>> /* Changed reg to 0xc too, same error message */
>>                                         reg = <0x0>;
>>                                         label = "Serdes0";
>>                                         phy-handle = <&switch0phy0>;
>>                                 };
>>
>>                                 port@1 {
>> /* Changed reg to 0xd too, same error message */
>>                                         reg = <0x1>;
>>                                         label = "Serdes1";
>>                                         phy-handle = <&switch0phy1>;
>>                                 };
>> 				
>> 				port@2 {
>>                                         reg = <0x2>;
>>                                         label = "lan1";
>>                                         phy-handle = <&switch0phy2>;
>>                                 };
>>
>>                                 port@3 {
>>                                         reg = <0x3>;
>>                                         label = "lan2";
>>                                         phy-handle = <&switch0phy3>;
>>                                 };
>>
>>                                 port@4 {
>>                                         reg = <0x4>;
>>                                         label = "lan3";
>>                                         phy-handle = <&switch0phy4>;
>>                                 };
> You don't need to list phy-handle for the internal PHYs. It should
> just find them.
>
>>                         mdio {
>>                                 #address-cells = <1>;
>>                                 #size-cells = <0>;
>>                                 switch0phy0: switch0phy0@0 {
>> /* Changed reg to 0 too, same error message */
>> 					 reg = <0xc>;
>>                                 };
>>                                 switch0phy1: switch0phy1@1 {
>> /* Changed reg to 1 too, same error message */
>>                                         reg = <0xd>;
>>                                 };
>>                                 switch0phy3: switch0phy3@3 {
>>                                         reg = <0x3>;
>>                                 };
>>                                 switch0phy4: switch0phy4@4 {
>>                                         reg = <0x4>;
>>                                 };
>>                         };
>>
>>                         mdio1 {
>> 				compatible = "marvell,mv88e6xxx-mdio-external";
>>                                 #address-cells = <1>;
>>                                 #size-cells = <0>;
>>
>>                                 switch0phy2: switch0phy2@2 {
>>                                         reg = <0x2>;
>>                                 };
>>                                 switch0phy6: switch0phy6@6 {
>>                                         reg = <0x6>;
>>                                 };
>>                         };
> I doubt this second mdio bus is correct. The 6390 uses that, but i
> don't think any other family does. The older generations have one MDIO
> bus for both internal and external PHYs.
>
> One other idea. Take a look at the data sheet. Can the MDIO pins also
> be used for GPIO? Do they default to GPIO or MDIO? For the 6390 they
> default to GPIO and there is code to reconfigure them for MDIO.
>
> 	Andrew

Hi Andrew,

okay, thanks so far. I'll look into the code. The GPIO configuration is 
a good point. I remember that we did something there (Normal SMI bit).

I removed all phy-handle for the internal ports and in the mdio part 
is only port 2 and 6 by now. But the Serdes ports are still not be
recognized. So maybe there is still something wrong?

Benjamin

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

> I removed all phy-handle for the internal ports and in the mdio part 
> is only port 2 and 6 by now. But the Serdes ports are still not be
> recognized. So maybe there is still something wrong?

What do you mean by SERDES? Do you mean they are connected to an SFP
cage? If so, you need to add an SFP node. Take a look at
vf610-zii-dev-rev-c.dts for an example.

	Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

>> I removed all phy-handle for the internal ports and in the mdio part 
>> is only port 2 and 6 by now. But the Serdes ports are still not be
>> recognized. So maybe there is still something wrong?
> What do you mean by SERDES? Do you mean they are connected to an SFP
> cage? If so, you need to add an SFP node. Take a look at
> vf610-zii-dev-rev-c.dts for an example.
>
> 	Andrew

Hey Andrew,
I've looked into the device tree. Why do they reference to i2c? We have
1x8 SFF tranceivers. Should I just add an i2c entry for it, because the 
value is required?

Now I don't have anything in the mdio part but the serdes ports are not 
recognized. Do I have to activate them in any kind? When I read the PHY
0 register of the serdes port 0 (SMI address 0xc) it is in power down mode.
Is there a problem with the mapping? Internally these serdes port PHYs are
at 0x0c and 0x0d but mapped to 0x0 and 0x1, for me this can be a reason 
why it won't be found. Is there a switch comparable to this mapping?
Maybe I'm totally wrong and misunderstanding something here?

Do you know which switch chip they use in the  vf610-zii-dev-rev-c?

Benjamin

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

On 06.06.19 14:24, Andrew Lunn wrote:
> On Thu, Jun 06, 2019 at 10:49:08AM +0200, Benjamin Beckmeyer wrote:
>>>> I removed all phy-handle for the internal ports and in the mdio part 
>>>> is only port 2 and 6 by now. But the Serdes ports are still not be
>>>> recognized. So maybe there is still something wrong?
>>> What do you mean by SERDES? Do you mean they are connected to an SFP
>>> cage? If so, you need to add an SFP node. Take a look at
>>> vf610-zii-dev-rev-c.dts for an example.
>>>
>>> 	Andrew
>> Hey Andrew,
>> I've looked into the device tree. Why do they reference to i2c? We have
>> 1x8 SFF tranceivers. Should I just add an i2c entry for it, because the 
>> value is required?
> Hi Benjamin
>
> Do you have a diagram of the board you can share with me. I'm confused
> about which ports have external copper PHYs and which have SFF
> connected to them., and if you are using copper PHYs with SERDES
> interfaces not RGMII.
>
> Each port needs to have some sort of 'PHY' connected to it. Either a
> copper PHY, an SFP/SFF, or a fake PHY using fixed-link.
>
>> Do you know which switch chip they use in the  vf610-zii-dev-rev-c?
> That board uses two 6390X. There is also ZII CF1 is if i remember
> correctly a 6352, and has one SFF.
>
> If you have an SFF connected to a port, you need its i2c bus, so that
> PHYLINK can read the SFF EEPROM to determine its capabilities and
> correctly configure the MAC to fit the SFF. Plus you get all the
> diagnostics, etc.
>
>   Andrew

Hi Andrew,
From our hardware developer I know now that we are using a "mini" SFF 
which has no i2c eeprom. 

Switch				|	external
Port 0 - internal serdes 0x0c --|-------Mini SFF 1x8 Transceiver
				|
Port 0 - internal serdes 0x0d --|-------Mini SFF 1x8 Transceiver
				|
Port 2 ----------RGMII----------|-------KSZ9031 PHY 0x02(strap)--Transceiver
				|
Port 3 - internal PHY 0x03 -----|-------Transceiver
				|
Port 3 - internal PHY 0x04 -----|-------Transceiver
				|			
Port 5 - CPU-Port RMII ---------|-------CPU
				|
Port 6 ----------RGMII----------|-------KSZ9031 PHY 0x06(strap)--Transceiver


Hopefully this makes it more clear to you. 

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

> >From our hardware developer I know now that we are using a "mini" SFF 
> which has no i2c eeprom. 

O.K. Does this mini SFF have LOS, TX-Disable, etc? Are these connected
to GPIOs? I assume the SFF is fibre? And it needs the SERDES to speak
1000BaseX, not SGMII?

> Switch				|	external
> Port 0 - internal serdes 0x0c --|-------Mini SFF 1x8 Transceiver
> 				|
> Port 0 - internal serdes 0x0d --|-------Mini SFF 1x8 Transceiver
> 				|
> Port 2 ----------RGMII----------|-------KSZ9031 PHY 0x02(strap)--Transceiver
> 				|
> Port 3 - internal PHY 0x03 -----|-------Transceiver
> 				|
> Port 3 - internal PHY 0x04 -----|-------Transceiver
> 				|			
> Port 5 - CPU-Port RMII ---------|-------CPU
> 				|
> Port 6 ----------RGMII----------|-------KSZ9031 PHY 0x06(strap)--Transceiver

So the current state is that just the SFF ports are not working? All
the copper PHYs are O.K.

    Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

On 06.06.19 15:35, Andrew Lunn wrote:
>> >From our hardware developer I know now that we are using a "mini" SFF 
>> which has no i2c eeprom. 
> O.K. Does this mini SFF have LOS, TX-Disable, etc? Are these connected
> to GPIOs? I assume the SFF is fibre? And it needs the SERDES to speak
> 1000BaseX, not SGMII?

Nope, no LOS no tx-disable etc. Yeah, the SFF is fibre. Exactly, it needs 
SERDES to speak 1000BaseX.

>
>> Switch				|	external
>> Port 0 - internal serdes 0x0c --|-------Mini SFF 1x8 Transceiver
>> 				|
>> Port 0 - internal serdes 0x0d --|-------Mini SFF 1x8 Transceiver
>> 				|
>> Port 2 ----------RGMII----------|-------KSZ9031 PHY 0x02(strap)--Transceiver
>> 				|
>> Port 3 - internal PHY 0x03 -----|-------Transceiver
>> 				|
>> Port 3 - internal PHY 0x04 -----|-------Transceiver
>> 				|			
>> Port 5 - CPU-Port RMII ---------|-------CPU
>> 				|
>> Port 6 ----------RGMII----------|-------KSZ9031 PHY 0x06(strap)--Transceiver
> So the current state is that just the SFF ports are not working? All
> the copper PHYs are O.K.
>
>     Andrew
>
The external copper PHYs are still not working properly, but if I set them to
fixed-link, I see data coming in with I start tcpdump on my device. Just with
some odd header but I'm not that far with DSA-tags and these stuff.
 
Just at that moment we found out that we have a problem with our MDC timing.
After we fixed that the external PHY registers could be read correctly over SMI
Command registers. But I haven't tested it with the DSA driver yet.

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

On Thu, Jun 06, 2019 at 03:47:06PM +0200, Benjamin Beckmeyer wrote:
> 
> On 06.06.19 15:35, Andrew Lunn wrote:
> >> >From our hardware developer I know now that we are using a "mini" SFF 
> >> which has no i2c eeprom. 
> > O.K. Does this mini SFF have LOS, TX-Disable, etc? Are these connected
> > to GPIOs? I assume the SFF is fibre? And it needs the SERDES to speak
> > 1000BaseX, not SGMII?
> 
> Nope, no LOS no tx-disable etc. Yeah, the SFF is fibre. Exactly, it needs 
> SERDES to speak 1000BaseX.

O.K. Then try something like what ZII devel B does:

                                       port@3 {
                                                reg = <3>;
                                                label = "optical3";

                                                fixed-link {
                                                        speed = <1000>;
                                                        full-duplex;
                                                };


What this does not give you is any link monitoring. I don't have the
datasheet of this device, but i assume it has two banks of registers
for the SERDES? And you can get the sync status? Similar to how the
6352 works. But with a fixed link this will be ignored.

> >> Switch				|	external
> >> Port 0 - internal serdes 0x0c --|-------Mini SFF 1x8 Transceiver
> >> 				|
> >> Port 0 - internal serdes 0x0d --|-------Mini SFF 1x8 Transceiver
> >> 				|
> >> Port 2 ----------RGMII----------|-------KSZ9031 PHY 0x02(strap)--Transceiver
> >> 				|
> >> Port 3 - internal PHY 0x03 -----|-------Transceiver
> >> 				|
> >> Port 3 - internal PHY 0x04 -----|-------Transceiver
> >> 				|			
> >> Port 5 - CPU-Port RMII ---------|-------CPU
> >> 				|
> >> Port 6 ----------RGMII----------|-------KSZ9031 PHY 0x06(strap)--Transceiver
> > So the current state is that just the SFF ports are not working? All
> > the copper PHYs are O.K.
> >
> >     Andrew
> >
> The external copper PHYs are still not working properly, but if I set them to
> fixed-link, I see data coming in with I start tcpdump on my device. Just with
> some odd header but I'm not that far with DSA-tags and these stuff.

If you build libpcap & tcpdump from the latest sources, it will
understand these headers.

	Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

On 06.06.19 15:59, Andrew Lunn wrote:
> On Thu, Jun 06, 2019 at 03:47:06PM +0200, Benjamin Beckmeyer wrote:
>> On 06.06.19 15:35, Andrew Lunn wrote:
>>>> >From our hardware developer I know now that we are using a "mini" SFF 
>>>> which has no i2c eeprom. 
>>> O.K. Does this mini SFF have LOS, TX-Disable, etc? Are these connected
>>> to GPIOs? I assume the SFF is fibre? And it needs the SERDES to speak
>>> 1000BaseX, not SGMII?
>> Nope, no LOS no tx-disable etc. Yeah, the SFF is fibre. Exactly, it needs 
>> SERDES to speak 1000BaseX.
> O.K. Then try something like what ZII devel B does:
>
>                                        port@3 {
>                                                 reg = <3>;
>                                                 label = "optical3";
>
>                                                 fixed-link {
>                                                         speed = <1000>;
>                                                         full-duplex;
>                                                 };
>
>
> What this does not give you is any link monitoring. I don't have the
> datasheet of this device, but i assume it has two banks of registers
> for the SERDES? And you can get the sync status? Similar to how the
> 6352 works. But with a fixed link this will be ignored.
>
>>>> Switch				|	external
>>>> Port 0 - internal serdes 0x0c --|-------Mini SFF 1x8 Transceiver
>>>> 				|
>>>> Port 0 - internal serdes 0x0d --|-------Mini SFF 1x8 Transceiver
>>>> 				|
>>>> Port 2 ----------RGMII----------|-------KSZ9031 PHY 0x02(strap)--Transceiver
>>>> 				|
>>>> Port 3 - internal PHY 0x03 -----|-------Transceiver
>>>> 				|
>>>> Port 3 - internal PHY 0x04 -----|-------Transceiver
>>>> 				|			
>>>> Port 5 - CPU-Port RMII ---------|-------CPU
>>>> 				|
>>>> Port 6 ----------RGMII----------|-------KSZ9031 PHY 0x06(strap)--Transceiver
>>> So the current state is that just the SFF ports are not working? All
>>> the copper PHYs are O.K.
>>>
>>>     Andrew
>>>
>> The external copper PHYs are still not working properly, but if I set them to
>> fixed-link, I see data coming in with I start tcpdump on my device. Just with
>> some odd header but I'm not that far with DSA-tags and these stuff.
> If you build libpcap & tcpdump from the latest sources, it will
> understand these headers.
>
> 	Andrew

Hi Andrew,

thanks for all the help. We had a problem with the external PHY and our mdio bus.
Now all port are recognized. I set the Serdes ports to a fixed-link, but at the 
moment  I can't test them. Booting kernel 4.9.109 I'm having the interfaces now.
So first of all. success! Great.

So all ports are now in forwarding mode (Switch port register 0x4 of all ports 
are 0x7f), but I don't reach it over ping. Even the ARP request are not forwarded.
The behavior is vice versa. The port status register shows for all copper ports
0x1007 when no link is up so the PHYs are recognized correctly and I tested it 
while the port is up and down. Even the indirect read via PHY COMMAND is working 
with all PHYs now. 
Do I have to set up the DSA in any kind? Or have to set up forwarding mode in 
software?
	Benjamin

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

> So all ports are now in forwarding mode (Switch port register 0x4 of all ports 
> are 0x7f), but I don't reach it over ping.

Hi

The most common error for people new to DSA is forgetting to bring
the master interface up.

The second thing to understand is that by default, all interfaces are
separated. So the switch won't bridge frames between ports, until you
add the ports to a Linux bridge. But you can give each interface its
own IP address.

    Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

>> So all ports are now in forwarding mode (Switch port register 0x4 of all ports 
>> are 0x7f), but I don't reach it over ping.
> Hi
>
> The most common error for people new to DSA is forgetting to bring
> the master interface up.
>
> The second thing to understand is that by default, all interfaces are
> separated. So the switch won't bridge frames between ports, until you
> add the ports to a Linux bridge. But you can give each interface its
> own IP address.
>
>     Andrew

Hi Andrew,
thanks for your help again. Sorry for the late reply we had a stats day yesterday. 
What interface do you mean with master interface? I assume, you mean eth0 (cpu port)?
I deleted the IP address of this interface and tried to add it to the bridge:

brctl addif bridge0 eth0
brctl: bridge bridge0: Invalid argument

I tried this with all lan1-4 interfaces and they just work and directly after
I added them I got some information about the port:

brctl addif br0 lan4
[  156.085842] br0: port 4(lan4) entered blocking state
[  156.091022] br0: port 4(lan4) entered disabled state

After I brought up the bridge with:

ip link set br0 up
[  445.313697] br0: port 4(lan4) entered blocking state
[  445.318896] br0: port 4(lan4) entered forwarding state

So I gave my eth0 an IP address and started tcpdump on eth0:

tcpdump -i eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
01:11:36.040006 ARP, Request who-has 192.168.10.1 tell 192.168.10.2, length 46
01:11:37.062283 ARP, Request who-has 192.168.10.1 tell 192.168.10.2, length 46
01:11:38.086465 ARP, Request who-has 192.168.10.1 tell 192.168.10.2, length 46

How you can see, I get the ARP request but no reply from my device. No ping is
working. All interfaces are up:

ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.201/24 brd 192.168.10.255 scope global eth0
       valid_lft forever preferred_lft forever
3: Serdes0@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop switchid 00000000 state DOWN group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff
4: Serdes1@eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop switchid 00000000 state DOWN group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff
5: lan1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br0 switchid 00000000 state UP group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff
6: lan2@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br0 switchid 00000000 state LOWERLAYERDOWN group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff
7: lan3@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br0 switchid 00000000 state LOWERLAYERDOWN group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff
8: lan4@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br0 switchid 00000000 state LOWERLAYERDOWN group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff
9: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 6a:d0:d5:9c:fe:f3 brd ff:ff:ff:ff:ff:ff

Am I doing something wrong or maybe I forget something?

Cheers,
Benjamin

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

On Tue, Jun 11, 2019 at 09:36:16AM +0200, Benjamin Beckmeyer wrote:
> >> So all ports are now in forwarding mode (Switch port register 0x4 of all ports 
> >> are 0x7f), but I don't reach it over ping.
> > Hi
> >
> > The most common error for people new to DSA is forgetting to bring
> > the master interface up.
> >
> > The second thing to understand is that by default, all interfaces are
> > separated. So the switch won't bridge frames between ports, until you
> > add the ports to a Linux bridge. But you can give each interface its
> > own IP address.
> >
> >     Andrew
> 
> Hi Andrew,
> thanks for your help again. Sorry for the late reply we had a stats day yesterday. 
> What interface do you mean with master interface? I assume, you mean eth0 (cpu port)?

Yes. The master interface is the pipe between the host and the
switch. It is only used as a pipe. It needs to be up, but there is no
point having an IP address on it, since it cannot send packets itself.

lan1-4 are slave interfaces. They can have IP addresses.

> I deleted the IP address of this interface and tried to add it to the bridge:
> 
> brctl addif bridge0 eth0
> brctl: bridge bridge0: Invalid argument

Yes, you should not do this. Just have the master interface up, but
otherwise leave it alone. It also needs to be up before you bring the
slave interfaces up.

> I tried this with all lan1-4 interfaces and they just work and directly after
> I added them I got some information about the port:
> 
> brctl addif br0 lan4
> [  156.085842] br0: port 4(lan4) entered blocking state
> [  156.091022] br0: port 4(lan4) entered disabled state
> 
> After I brought up the bridge with:
> 
> ip link set br0 up
> [  445.313697] br0: port 4(lan4) entered blocking state
> [  445.318896] br0: port 4(lan4) entered forwarding state
> 
> So I gave my eth0 an IP address and started tcpdump on eth0:

No. If you have created a bridge, put the IP address on the bridge.
If you have a slave which is not part of the bridge, you can give it
an IP address. Just treat the interfaces as Linux interfaces. Run
dhclient on them, use ethtool, iproute2, an snmp agent, add them to a
bridge. They are just normal Linux interfaces, which can make use of
the switch hardware to accelerate some operations, like bridging
frames.

   Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

> On Tue, Jun 11, 2019 at 09:36:16AM +0200, Benjamin Beckmeyer wrote:
>>>> So all ports are now in forwarding mode (Switch port register 0x4 of all ports 
>>>> are 0x7f), but I don't reach it over ping.
>>> Hi
>>>
>>> The most common error for people new to DSA is forgetting to bring
>>> the master interface up.
>>>
>>> The second thing to understand is that by default, all interfaces are
>>> separated. So the switch won't bridge frames between ports, until you
>>> add the ports to a Linux bridge. But you can give each interface its
>>> own IP address.
>>>
>>>     Andrew
>> Hi Andrew,
>> thanks for your help again. Sorry for the late reply we had a stats day yesterday. 
>> What interface do you mean with master interface? I assume, you mean eth0 (cpu port)?
> Yes. The master interface is the pipe between the host and the
> switch. It is only used as a pipe. It needs to be up, but there is no
> point having an IP address on it, since it cannot send packets itself.
>
> lan1-4 are slave interfaces. They can have IP addresses.
>
>> I deleted the IP address of this interface and tried to add it to the bridge:
>>
>> brctl addif bridge0 eth0
>> brctl: bridge bridge0: Invalid argument
> Yes, you should not do this. Just have the master interface up, but
> otherwise leave it alone. It also needs to be up before you bring the
> slave interfaces up.
>
>> I tried this with all lan1-4 interfaces and they just work and directly after
>> I added them I got some information about the port:
>>
>> brctl addif br0 lan4
>> [  156.085842] br0: port 4(lan4) entered blocking state
>> [  156.091022] br0: port 4(lan4) entered disabled state
>>
>> After I brought up the bridge with:
>>
>> ip link set br0 up
>> [  445.313697] br0: port 4(lan4) entered blocking state
>> [  445.318896] br0: port 4(lan4) entered forwarding state
>>
>> So I gave my eth0 an IP address and started tcpdump on eth0:
> No. If you have created a bridge, put the IP address on the bridge.
> If you have a slave which is not part of the bridge, you can give it
> an IP address. Just treat the interfaces as Linux interfaces. Run
> dhclient on them, use ethtool, iproute2, an snmp agent, add them to a
> bridge. They are just normal Linux interfaces, which can make use of
> the switch hardware to accelerate some operations, like bridging
> frames.
>
>    Andrew

Hi Andrew,
it set up a bridge now, added all four ethernet ports to it, brought all four
interfaces up and the bridge up and gave the bridge an IP address.
If I try to capture on the bridge interface nothing is coming in. On the eth0
interface at least the ARP request comes in.

I captured a ping from my device to my computer to look if outgoing is working
(captured on both devices). Here is the output from my device where i started the:

00:24:24.752057 ARP, Request who-has 192.168.10.2 tell 192.168.10.1, length 28
	0x0000:  0001 0800 0604 0001 6a2a ad79 def5 c0a8  ........j*.y....
	0x0010:  0a01 0000 0000 0000 c0a8 0a02            ............

and here the output of the receiver:

14:49:06.725940 MEDSA 0.2:0: ARP, Request who-has benjamin-HP tell 192.168.10.1, length 42
	0x0000:  0000 4010 0000 0806 0001 0800 0604 0001  ..@.............
	0x0010:  6a2a ad79 def5 c0a8 0a01 0000 0000 0000  j*.y............
	0x0020:  c0a8 0a02 0000 0000 0000 0000 0000 0000  ................
	0x0030:  0000

I'm really stuck at the moment because I don't know what to do further. I think, 
I did everything what is needed.
And I know when I configure the switch manually via MDIO the connection is working.
When I'm looking for traffic in ifconfig on all ports there is everywhere 0 bytes 
except for eth0.
Do you have any ideas?

	Benjamin

Re: DSA with MV88E6321 and imx28

[Andrew Lunn]

> I captured a ping from my device to my computer to look if outgoing is working
> (captured on both devices). Here is the output from my device where i started the:
> 
> 00:24:24.752057 ARP, Request who-has 192.168.10.2 tell 192.168.10.1, length 28
> 	0x0000:  0001 0800 0604 0001 6a2a ad79 def5 c0a8  ........j*.y....
> 	0x0010:  0a01 0000 0000 0000 c0a8 0a02            ............
> 
> and here the output of the receiver:
> 
> 14:49:06.725940 MEDSA 0.2:0: ARP, Request who-has benjamin-HP tell 192.168.10.1, length 42
> 	0x0000:  0000 4010 0000 0806 0001 0800 0604 0001  ..@.............
> 	0x0010:  6a2a ad79 def5 c0a8 0a01 0000 0000 0000  j*.y............
> 	0x0020:  c0a8 0a02 0000 0000 0000 0000 0000 0000  ................
> 	0x0030:  0000
> 
> I'm really stuck at the moment because I don't know what to do further. I think, 
> I did everything what is needed.
> And I know when I configure the switch manually via MDIO the connection is working.
> When I'm looking for traffic in ifconfig on all ports there is everywhere 0 bytes 
> except for eth0.
> Do you have any ideas?

I would start simple and build up. Don't use a bridge. Just put the IP
address 192.168.10.1 on the slave interface for port 2.

So something like:

ip link set eth0 up
ip addr add 192.168.10.1/24 dev lan2
ip link set lan2 up

then you can try ping 192.168.10.2.

Then trace the packet along the path. Does the ARP request make it to
192.168.10.2? Is a reply sent? ethtool -S lan2 will show you the
packet counts. Do the counters show the ARP going out and the reply
coming back?

       Andrew

Re: DSA with MV88E6321 and imx28

[Benjamin Beckmeyer]

>> I captured a ping from my device to my computer to look if outgoing is working
>> (captured on both devices). Here is the output from my device where i started the:
>>
>> 00:24:24.752057 ARP, Request who-has 192.168.10.2 tell 192.168.10.1, length 28
>> 	0x0000:  0001 0800 0604 0001 6a2a ad79 def5 c0a8  ........j*.y....
>> 	0x0010:  0a01 0000 0000 0000 c0a8 0a02            ............
>>
>> and here the output of the receiver:
>>
>> 14:49:06.725940 MEDSA 0.2:0: ARP, Request who-has benjamin-HP tell 192.168.10.1, length 42
>> 	0x0000:  0000 4010 0000 0806 0001 0800 0604 0001  ..@.............
>> 	0x0010:  6a2a ad79 def5 c0a8 0a01 0000 0000 0000  j*.y............
>> 	0x0020:  c0a8 0a02 0000 0000 0000 0000 0000 0000  ................
>> 	0x0030:  0000
>>
>> I'm really stuck at the moment because I don't know what to do further. I think, 
>> I did everything what is needed.
>> And I know when I configure the switch manually via MDIO the connection is working.
>> When I'm looking for traffic in ifconfig on all ports there is everywhere 0 bytes 
>> except for eth0.
>> Do you have any ideas?
> I would start simple and build up. Don't use a bridge. Just put the IP
> address 192.168.10.1 on the slave interface for port 2.
>
> So something like:
>
> ip link set eth0 up
> ip addr add 192.168.10.1/24 dev lan2
> ip link set lan2 up
>
> then you can try ping 192.168.10.2.
>
> Then trace the packet along the path. Does the ARP request make it to
> 192.168.10.2? Is a reply sent? ethtool -S lan2 will show you the
> packet counts. Do the counters show the ARP going out and the reply
> coming back?
>
>        Andrew

Hi Andrew,

it is finally running in DSA mode. Thanks so much, Andrew. I can set an
IP address to all ports individually and running it now in a bridge with 
all 4 copper ports. Set an IP address to the bridge and get a ping reply 
from  all ports. So far, that is all what I have to test, for now. 

I warned you before, I will go back to another custom board. But now with 
more experience thanks to you.

Cheers,
Benjamin