Re: [NetDev-info] Distributed Switch Architecture for 88E6390

Re: [NetDev-info] Distributed Switch Architecture for 88E6390

[S.Y. Park]

Dear Mr. Thompson,

I'm forwarding to you to the technical discussion mailing list called
"netdev@vger.kernel.org".

info@netdevconf.org is for discussions regarding The NetDev Society's
NetDev Conference  attendance & participation questions & concerns,
not technical discussion.

Good luck w/ your 88E6390 driver functionality testing contribution,
and I'm sure someone from the netdev mailing list can address your
question.

Sincerely,

Soyoung Park
Executive Director
The NetDev Society

On Mon, Feb 5, 2018 at 1:09 PM, David Thompson <dthompson@miovision.com> wrote:
> Hello,
>
> I am looking for contacts to anyone that is currently working on the Marvell
> 88E6390 port within the Distributed Switch Architecture.  I am working
> through a back port to 4.4.38 in a non-trivial integration and have found a
> number of issues I believe are legitimate issues that I would like to
> feedback.  I am also seeing a potential issue with outgoing TCP traffic and
> was looking to see if this is a known issue or has been investigated prior.
> I would be very happy to assist in testing the 88E6390 driver functionality.
>
> Details follow:
>
> For my configuration the CPU port is port 6, there is only one 88E6390 and
> finally port 0 and the SERDES are not used.  I am also using the GPIO
> bitbang driver for MDIO to the 88E6390 (I've confirmed that it works
> independently).
>
> First, I've noticed a few interesting discrepancies in setup and the
> datasheet for the 88E6390.  They are as follows:
>
> CPU Port port control configuration (port 0x6 reg 0x4) is configured 0x3107.
> This indicates
>
> 15:14 SA Filter Disabled
>     9:8 Frame Mode DSA
> 13:12 Egress Mode - is then in an undefined state.  According the datasheet
> it must be set to 00.
>
> Global1 Global Control 2 (addr 0x1b reg 0x1c) is configured 0xf000.  Bits
> 15:14 only supports 0x0, 0x1, and 0x2 while bits 13:12 are reserved.
> Global1 registers 0x10 -> 0x18 are written 0x0000, 0x0000, 0x5555, 0x5555,
> 0xaaaa, 0xaaaa, 0xffff, and 0xffff.  According the 88E6390 datasheet
> 0x10->0x18 are reserved.
>
>
> When I got the back-port compiling and flashed on my target device I was
> able to bring up individual ports (the phy would come up and auto-negotiate
> successfully) but could not send or receive ICMP echo/reply.  Making the
> changes associated with 1) above to be in accordance with the Marvell
> datasheet got me to a point where I could send and receive pings.
>
> I now face an interesting problem, I am currently unable to setup or
> maintain a TCP session.  Doing a wireshark capture on my host pc to the
> target device (including the 88E6390) with the appropriate port enabled I am
> seeing packet re-transmission issues from the target device.  For example,
> if I try to initiate from my host pc I will send a SYN packet and receive
> multiple identical  SYN,ACK packets returned or if I initiate from the
> target device I see multiple identical SYN packets sent.  I was wondering if
> you had ever seen this anything like this issue and if so had any
> suggestions with regards to what might be the cause?
>
> Much appreciated,
>
> Dave Thompson
>
> David Thompson BSc, MSc
> Chief Systems Architect
> Miovision
> dthompson@miovision.com
> 519-513-2407 x 225
> 877-646-8476 (toll-free)
> 519-590-5458 (mobile)
> Web | Blog  |  LinkedIn  |  Twitter  |  Facebook
>
> We’re on to bigger and better things (and spaces)! Please note that
> Miovision’s head office has moved to 137 Glasgow Street, Suite 110,
> Kitchener ON, N2G 4X8.
>
> Miovision | 137 Glasgow Street, Suite 110, Kitchener ON | N2G 4X8
>
> This e-mail may contain information that is privileged or confidential. If
> you are not the intended recipient, please delete the e-mail and any
> attachments and notify us immediately. Please advise if you require
> reasonable accommodation or assistance.
>
> _______________________________________________
> info mailing list
> info@lists.netdevconf.org
> http://lists.netdevconf.org/cgi-bin/mailman/listinfo/info
>

Re: [NetDev-info] Distributed Switch Architecture for 88E6390

[Florian Fainelli]

On 02/05/2018 11:59 AM, S.Y. Park wrote:
> Dear Mr. Thompson,
> 
> I'm forwarding to you to the technical discussion mailing list called
> "netdev@vger.kernel.org".
> 
> info@netdevconf.org is for discussions regarding The NetDev Society's
> NetDev Conference  attendance & participation questions & concerns,
> not technical discussion.
> 
> Good luck w/ your 88E6390 driver functionality testing contribution,
> and I'm sure someone from the netdev mailing list can address your
> question.
> 
> Sincerely,
> 
> Soyoung Park
> Executive Director
> The NetDev Society
> 
> On Mon, Feb 5, 2018 at 1:09 PM, David Thompson <dthompson@miovision.com> wrote:
>> Hello,
>>
>> I am looking for contacts to anyone that is currently working on the Marvell
>> 88E6390 port within the Distributed Switch Architecture.  I am working
>> through a back port to 4.4.38 in a non-trivial integration and have found a
>> number of issues I believe are legitimate issues that I would like to
>> feedback.  I am also seeing a potential issue with outgoing TCP traffic and
>> was looking to see if this is a known issue or has been investigated prior.
>> I would be very happy to assist in testing the 88E6390 driver functionality.
>>
>> Details follow:
>>
>> For my configuration the CPU port is port 6, there is only one 88E6390 and
>> finally port 0 and the SERDES are not used.  I am also using the GPIO
>> bitbang driver for MDIO to the 88E6390 (I've confirmed that it works
>> independently).
>>
>> First, I've noticed a few interesting discrepancies in setup and the
>> datasheet for the 88E6390.  They are as follows:
>>
>> CPU Port port control configuration (port 0x6 reg 0x4) is configured 0x3107.
>> This indicates
>>
>> 15:14 SA Filter Disabled
>>     9:8 Frame Mode DSA
>> 13:12 Egress Mode - is then in an undefined state.  According the datasheet
>> it must be set to 00.
>>
>> Global1 Global Control 2 (addr 0x1b reg 0x1c) is configured 0xf000.  Bits
>> 15:14 only supports 0x0, 0x1, and 0x2 while bits 13:12 are reserved.
>> Global1 registers 0x10 -> 0x18 are written 0x0000, 0x0000, 0x5555, 0x5555,
>> 0xaaaa, 0xaaaa, 0xffff, and 0xffff.  According the 88E6390 datasheet
>> 0x10->0x18 are reserved.
>>
>>
>> When I got the back-port compiling and flashed on my target device I was
>> able to bring up individual ports (the phy would come up and auto-negotiate
>> successfully) but could not send or receive ICMP echo/reply.  Making the
>> changes associated with 1) above to be in accordance with the Marvell
>> datasheet got me to a point where I could send and receive pings.
>>
>> I now face an interesting problem, I am currently unable to setup or
>> maintain a TCP session.  Doing a wireshark capture on my host pc to the
>> target device (including the 88E6390) with the appropriate port enabled I am
>> seeing packet re-transmission issues from the target device.  For example,
>> if I try to initiate from my host pc I will send a SYN packet and receive
>> multiple identical  SYN,ACK packets returned or if I initiate from the
>> target device I see multiple identical SYN packets sent.  I was wondering if
>> you had ever seen this anything like this issue and if so had any
>> suggestions with regards to what might be the cause?
>>
>> Much appreciated,
>>
>> Dave Thompson
>>
>> David Thompson BSc, MSc
>> Chief Systems Architect
>> Miovision
>> dthompson@miovision.com
>> 519-513-2407 x 225
>> 877-646-8476 (toll-free)
>> 519-590-5458 (mobile)
>> Web | Blog  |  LinkedIn  |  Twitter  |  Facebook
>>
>> We’re on to bigger and better things (and spaces)! Please note that
>> Miovision’s head office has moved to 137 Glasgow Street, Suite 110,
>> Kitchener ON, N2G 4X8.
>>
>> Miovision | 137 Glasgow Street, Suite 110, Kitchener ON | N2G 4X8
>>
>> This e-mail may contain information that is privileged or confidential. If
>> you are not the intended recipient, please delete the e-mail and any
>> attachments and notify us immediately. Please advise if you require
>> reasonable accommodation or assistance.
>>
>> _______________________________________________
>> info mailing list
>> info@lists.netdevconf.org
>> http://lists.netdevconf.org/cgi-bin/mailman/listinfo/info
>>

-- 
Florian

Re: [NetDev-info] Distributed Switch Architecture for 88E6390

[Andrew Lunn]

On Mon, Feb 05, 2018 at 02:59:48PM -0500, S.Y. Park wrote:
> Dear Mr. Thompson,
> 
> I'm forwarding to you to the technical discussion mailing list called
> "netdev@vger.kernel.org".
> 
> info@netdevconf.org is for discussions regarding The NetDev Society's
> NetDev Conference  attendance & participation questions & concerns,
> not technical discussion.
> 
> Good luck w/ your 88E6390 driver functionality testing contribution,
> and I'm sure someone from the netdev mailing list can address your
> question.
> 
> Sincerely,
> 
> Soyoung Park

Hi Soyoung

Thanks for forwarding this.

> > I am looking for contacts to anyone that is currently working on the Marvell
> > 88E6390 port within the Distributed Switch Architecture.  I am working
> > through a back port to 4.4.38 in a non-trivial integration and have found a
> > number of issues I believe are legitimate issues that I would like to
> > feedback.  I am also seeing a potential issue with outgoing TCP traffic and
> > was looking to see if this is a known issue or has been investigated prior.
> > I would be very happy to assist in testing the 88E6390 driver functionality.

Hi David

Are the issues you report here with your backported 4.4.38, or a
net-next?

> > For my configuration the CPU port is port 6, there is only one 88E6390 and
> > finally port 0 and the SERDES are not used.  I am also using the GPIO
> > bitbang driver for MDIO to the 88E6390 (I've confirmed that it works
> > independently).

Bit banging should not be a problem. I've used it with other Marvell
switches.

> > First, I've noticed a few interesting discrepancies in setup and the
> > datasheet for the 88E6390.  They are as follows:
> >
> > CPU Port port control configuration (port 0x6 reg 0x4) is configured 0x3107.
> > This indicates
> >
> > 15:14 SA Filter Disabled
> >     9:8 Frame Mode DSA
> > 13:12 Egress Mode - is then in an undefined state.  According the datasheet
> > it must be set to 00.

So mv88e6xxx_setup_port_mode() calls
mv88e6xxx_set_port_mode_dsa(), which calls
mv88e6xxx_set_port_mode(chip, port, MV88E6XXX_FRAME_MODE_DSA,
                        MV88E6XXX_EGRESS_MODE_UNMODIFIED,
                        MV88E6XXX_PORT_ETH_TYPE_DEFAULT);
which calls
mv88e6xxx_port_set_egress_mode(chip, port, egress);
where egress is MV88E6XXX_EGRESS_MODE_UNMODIFIED.

int mv88e6xxx_port_set_egress_mode(struct mv88e6xxx_chip *chip, int port,
                                   enum mv88e6xxx_egress_mode mode)
{
        int err;
        u16 reg;

        err = mv88e6xxx_port_read(chip, port, MV88E6XXX_PORT_CTL0, &reg);
        if (err)
                return err;

        reg &= ~MV88E6XXX_PORT_CTL0_EGRESS_MODE_MASK;

        switch (mode) {
        case MV88E6XXX_EGRESS_MODE_UNMODIFIED:
                reg |= MV88E6XXX_PORT_CTL0_EGRESS_MODE_UNMODIFIED;
                break;

#define MV88E6XXX_PORT_CTL0_EGRESS_MODE_UNMODIFIED              0x0000

So that should me 13:12 are set to 00.

On my board, i have a value of 0x53f.

> > Global1 Global Control 2 (addr 0x1b reg 0x1c) is configured 0xf000.  Bits
> > 15:14 only supports 0x0, 0x1, and 0x2 while bits 13:12 are reserved.

>From my board

Global1@0
 0: c801
 1:    0
 2:    0
 3:    0
 4: 40a8
 5: 1000
 6:    0
 7:    0
 8:    0
 9:    0
10:  509
11: 3000
12: 7ff7
13: ffff
14: ffff
15: ffff
16:    0
17:    0
18:    0
19:    0
20:    0
21:    0
22:    0
23:    0
24:    0
25:    0
26:  3ff
27:  3f9
28: f0c0

1c = 28. Yes, we have the upper bits set.

This is from mv88e6xxx_g1_setup(), setting
MV88E6XXX_G1_CTL2_MULTIPLE_CASCADE which is not valid for this family.

29: 1000
30:    0
31:    0


> > Global1 registers 0x10 -> 0x18 are written 0x0000, 0x0000, 0x5555, 0x5555,
> > 0xaaaa, 0xaaaa, 0xffff, and 0xffff.  According the 88E6390 datasheet
> > 0x10->0x18 are reserved.

Agreed. mv88e6xxx_g1_setup() needs a few fixes.

> > I now face an interesting problem, I am currently unable to setup or
> > maintain a TCP session.  Doing a wireshark capture on my host pc to the
> > target device (including the 88E6390) with the appropriate port enabled I am
> > seeing packet re-transmission issues from the target device.  For example,
> > if I try to initiate from my host pc I will send a SYN packet and receive
> > multiple identical  SYN,ACK packets returned or if I initiate from the
> > target device I see multiple identical SYN packets sent.  I was wondering if
> > you had ever seen this anything like this issue and if so had any
> > suggestions with regards to what might be the cause?

I would suggest running tcpdump on the target as well. See if packets
it is trying to send are not making it out. This is made a bit harder
in the tcpdump does not understand the DSA header. But you can at
least see the raw frames.

What Ethernet device are you using to the SoC side of the CPU port?
Adding the DSA header makes the Ethernet frame bigger. It is then
bigger than the standard Ethernet MTU. I've seen some Ethernet
drivers/HW discard such frames. Check you can actually Rx and Tx
frames which are bigger than usual.

       Andrew

Re: [NetDev-info] Distributed Switch Architecture for 88E6390

[Andrew Lunn]

On Mon, Feb 05, 2018 at 03:52:20PM -0500, David Thompson wrote:
> Hello all!  I really appreciate any feedback with regards to my inquiry.

Hi David

No top posting please.

Please make your signature confirm the netiquette. That general means
4 lines max, no legal gibberish, etc.

     Andrew

Re: [NetDev-info] Distributed Switch Architecture for 88E6390

[Andrew Lunn]

> Hi Andrew,
> 
> > For port 0x6 reg 0x4, confirmed.  It looks like the latest code is
> > correctly setting bits 13:12.  I'm porting from an older hash.

Hi Dave

6390 support is quite new. So you really should be use the latest
code. Otherwise you are going to have issues like this. Plus you are
going to want the fixes for the issues you just pointed out yourself.

> > Thanks for the suggestion regarding the MTU size.  I will look
> > into it in detail and follow up.  The driver for the ethernet
> > device I'm using is compatible with the nvidia,eqos device (which
> > is basically saying it is the built-in MAC & PHY on the Tegra 186
> > ).

I'm not aware of anybody using a tegra with DSA. So there could be
frame size issues.

      Andrew

Re: [NetDev-info] Distributed Switch Architecture for 88E6390

[Andrew Lunn]

> Let me know if you're interested in any testing.  Unfortunately I'm going
> to be limited to kernel 4.4.38 though as the tegra BSP would need to be
> ported in its entirety (a code-base that I don't control).

Hi Dave

You might be interested in:

https://lkml.org/lkml/2018/2/6/24

	Andrew