All of lore.kernel.org
 help / color / mirror / Atom feed
* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-11 20:39 ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-09-11 20:39 UTC (permalink / raw)
  To: netdev, linux-amlogic; +Cc: Giuseppe Cavallaro, Alexandre Torgue, Johnson Leung

[-- Attachment #1: Type: text/plain, Size: 2831 bytes --]

Hello,

I have a device with a Meson GXBB SoC with an stmmac IP block.
Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
Similar issues were reported in #linux-amlogic by a user with an
Odroid C2 board (= similar hardware).

The symptoms are:
Receiving data is plenty fast (I can max out my internet connection
easily, and with iperf3 I get ~900Mbit/s).
Transmitting data from the device is unfortunately very slow, traffic
sometimes even stalls completely.

I have attached the iperf results and the output of
/sys/kernel/debug/stmmaceth/eth0/descriptors_status.
Below you can find the ifconfig, netstat and stmmac dma_cap info
(*after* I ran all tests).

The "involved parties" are:
- Meson GXBB specific network configuration registers (I have have
double-checked them with the reference drivers: everything seems fine
here)
- stmmac: it seems that nobody else has reported these kind of issues
so far, however I'd still like to hear where I should enable some
debugging bits to rule out any stmmac bug
- RTL8211F PHY driver: unfortunately there are no public datasheets
available so this is hard to debug. but I'm guessing that TX delay
could cause similar issues, so this may be the cause as well.


Thanks for any input in advance!
Regards,
Martin


[root@alarm ~]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.235  netmask 255.255.255.0  broadcast 192.168.1.255
        ether e2:aa:53:fc:f5:c5  txqueuelen 1000  (Ethernet)
        RX packets 1967602  bytes 2968750265 (2.7 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 101875  bytes 8548285 (8.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 18

[root@alarm ~]# netstat -i
Kernel Interface table
Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0      1500  1967801      0      0 0        101934      0      0      0 BMRU

[root@alarm ~]# cat /sys/kernel/debug/stmmaceth/eth0/dma_cap
==============================
        DMA HW features
==============================
        10/100 Mbps Y
        1000 Mbps Y
        Half duple Y
        Hash Filter: Y
        Multiple MAC address registers: Y
        PCS (TBI/SGMII/RTBI PHY interfatces): N
        SMA (MDIO) Interface: Y
        PMT Remote wake up: Y
        PMT Magic Frame: Y
        RMON module: Y
        IEEE 1588-2002 Time Stamp: N
        IEEE 1588-2008 Advanced Time Stamp:N
        802.3az - Energy-Efficient Ethernet (EEE) Y
        AV features: N
        Checksum Offload in TX: Y
        IP Checksum Offload (type1) in RX: N
        IP Checksum Offload (type2) in RX: Y
        RXFIFO > 2048bytes: Y
        Number of Additional RX channel: 0
        Number of Additional TX channel: 0
        Enhanced descriptors: N

[-- Attachment #2: iperf-results.txt --]
[-- Type: text/plain, Size: 5073 bytes --]

UDP test (iperf3):
[root@alarm ~]# iperf3 --client 192.168.1.100 -u
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.235 port 38931 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  16  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  12526562.925 ms  113/159 (71%)  
[  4] Sent 159 datagrams

iperf Done.
[root@alarm ~]# iperf3 --client 192.168.1.100 -u -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.235 port 45128 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-1.00   sec   136 KBytes  1.11 Mbits/sec  81407591.898 ms  0/17 (0%)  
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  28987137.507 ms  0/16 (0%)  
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  10321569.793 ms  0/16 (0%)  
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  3675244.000 ms  0/16 (0%)  
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  1308659.322 ms  0/16 (0%)  
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  465979.740 ms  0/16 (0%)  
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  165923.341 ms  0/16 (0%)  
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  59081.019 ms  0/16 (0%)  
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  21037.233 ms  0/16 (0%)  
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  7490.825 ms  0/16 (0%)  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.26 MBytes  1.06 Mbits/sec  7490.825 ms  0/161 (0%)  
[  4] Sent 161 datagrams

iperf Done.
[root@alarm ~]# 


TCP test (iperf3):
[root@alarm ~]# iperf3 --client 192.168.1.100      
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.235 port 47508 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   115 KBytes   937 Kbits/sec    8   1.41 KBytes       
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   115 KBytes  93.8 Kbits/sec   11             sender
[  4]   0.00-10.00  sec  7.07 KBytes  5.79 Kbits/sec                  receiver

iperf Done.
[root@alarm ~]# iperf3 --client 192.168.1.100 -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.235 port 47512 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  84.9 MBytes   712 Mbits/sec                  
[  4]   1.00-2.00   sec   106 MBytes   891 Mbits/sec                  
[  4]   2.00-3.00   sec   107 MBytes   894 Mbits/sec                  
[  4]   3.00-4.00   sec  93.5 MBytes   784 Mbits/sec                  
[  4]   4.00-5.00   sec  98.6 MBytes   827 Mbits/sec                  
[  4]   5.00-6.00   sec  80.5 MBytes   675 Mbits/sec                  
[  4]   6.00-7.00   sec   107 MBytes   894 Mbits/sec                  
[  4]   7.00-8.00   sec   106 MBytes   886 Mbits/sec                  
[  4]   8.00-9.00   sec   107 MBytes   894 Mbits/sec                  
[  4]   9.00-10.00  sec   106 MBytes   893 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   997 MBytes   836 Mbits/sec  1569             sender
[  4]   0.00-10.00  sec   996 MBytes   835 Mbits/sec                  receiver

iperf Done.
[root@alarm ~]#

[-- Attachment #3: stmmac-descsriptor_status.txt --]
[-- Type: text/plain, Size: 48977 bytes --]

RX descriptor ring:
0 [0x23b2000]: 0x85e60320 0x803fffff 0x7378e702 0x0

1 [0x23b2000]: 0x85e60320 0x803fffff 0x7378de82 0x0

2 [0x23b2000]: 0x85e60320 0x803fffff 0x7378d602 0x0

3 [0x23b2000]: 0x85e60320 0x803fffff 0x7378cd82 0x0

4 [0x23b2000]: 0x85e60320 0x803fffff 0x7378c502 0x0

5 [0x23b2000]: 0x85e60320 0x803fffff 0x7378bc82 0x0

6 [0x23b2000]: 0x85e60320 0x803fffff 0x7378b402 0x0

7 [0x23b2000]: 0x85e60320 0x803fffff 0x7378ab82 0x0

8 [0x23b2000]: 0x85e60320 0x803fffff 0x7378a302 0x0

9 [0x23b2000]: 0x85e60320 0x803fffff 0x73789a82 0x0

10 [0x23b2000]: 0x85e60320 0x803fffff 0x73789202 0x0

11 [0x23b2000]: 0x85e60320 0x803fffff 0x73788982 0x0

12 [0x23b2000]: 0x85e60320 0x803fffff 0x73788102 0x0

13 [0x23b2000]: 0x85e60320 0x803fffff 0x721df802 0x0

14 [0x23b2000]: 0x85e60320 0x803fffff 0x721def82 0x0

15 [0x23b2000]: 0x85e60320 0x803fffff 0x721de702 0x0

16 [0x23b2000]: 0x85e60320 0x803fffff 0x721dde82 0x0

17 [0x23b2000]: 0x85e60320 0x803fffff 0x721dd602 0x0

18 [0x23b2000]: 0x85e60320 0x803fffff 0x721dcd82 0x0

19 [0x23b2000]: 0x85e60320 0x803fffff 0x721dc502 0x0

20 [0x23b2000]: 0x85e60320 0x803fffff 0x721dbc82 0x0

21 [0x23b2000]: 0x85e60320 0x803fffff 0x721db402 0x0

22 [0x23b2000]: 0x85e60320 0x803fffff 0x721dab82 0x0

23 [0x23b2000]: 0x85e60320 0x803fffff 0x721da302 0x0

24 [0x23b2000]: 0x85e60320 0x803fffff 0x721d9a82 0x0

25 [0x23b2000]: 0x85e60320 0x803fffff 0x721d9202 0x0

26 [0x23b2000]: 0x85e60320 0x803fffff 0x721d8982 0x0

27 [0x23b2000]: 0x85e60320 0x803fffff 0x721d8102 0x0

28 [0x23b2000]: 0x85e60320 0x803fffff 0x735b7802 0x0

29 [0x23b2000]: 0x85e60320 0x803fffff 0x735b6f82 0x0

30 [0x23b2000]: 0x85e60320 0x803fffff 0x735b6702 0x0

31 [0x23b2000]: 0x85e60320 0x803fffff 0x735b5e82 0x0

32 [0x23b2000]: 0x85e60320 0x803fffff 0x735b5602 0x0

33 [0x23b2000]: 0x85e60320 0x803fffff 0x735b4d82 0x0

34 [0x23b2000]: 0x85e60320 0x803fffff 0x735b4502 0x0

35 [0x23b2000]: 0x85e60320 0x803fffff 0x735b3c82 0x0

36 [0x23b2000]: 0x85e60320 0x803fffff 0x735b3402 0x0

37 [0x23b2000]: 0x85e60320 0x803fffff 0x735b2b82 0x0

38 [0x23b2000]: 0x85e60320 0x803fffff 0x735b2302 0x0

39 [0x23b2000]: 0x85e60320 0x803fffff 0x735b1a82 0x0

40 [0x23b2000]: 0x85e60320 0x803fffff 0x735b1202 0x0

41 [0x23b2000]: 0x85e60320 0x803fffff 0x735b0982 0x0

42 [0x23b2000]: 0x85e60320 0x803fffff 0x735b0102 0x0

43 [0x23b2000]: 0x85e60320 0x803fffff 0x737e7802 0x0

44 [0x23b2000]: 0x85e60320 0x803fffff 0x737e6f82 0x0

45 [0x23b2000]: 0x85e60320 0x803fffff 0x737e6702 0x0

46 [0x23b2000]: 0x85e60320 0x803fffff 0x737e5e82 0x0

47 [0x23b2000]: 0x85e60320 0x803fffff 0x737e5602 0x0

48 [0x23b2000]: 0x85e60320 0x803fffff 0x737e4d82 0x0

49 [0x23b2000]: 0x85e60320 0x803fffff 0x737e4502 0x0

50 [0x23b2000]: 0x85e60320 0x803fffff 0x737e3c82 0x0

51 [0x23b2000]: 0x85e60320 0x803fffff 0x737e3402 0x0

52 [0x23b2000]: 0x85e60320 0x803fffff 0x737e2b82 0x0

53 [0x23b2000]: 0x85e60320 0x803fffff 0x737e2302 0x0

54 [0x23b2000]: 0x85e60320 0x803fffff 0x737e1a82 0x0

55 [0x23b2000]: 0x85e60320 0x803fffff 0x737e1202 0x0

56 [0x23b2000]: 0x85e60320 0x803fffff 0x737e0982 0x0

57 [0x23b2000]: 0x85e60320 0x803fffff 0x737e0102 0x0

58 [0x23b2000]: 0x85e60320 0x803fffff 0x7086f802 0x0

59 [0x23b2000]: 0x85e60320 0x803fffff 0x7086ef82 0x0

60 [0x23b2000]: 0x85e60320 0x803fffff 0x7086e702 0x0

61 [0x23b2000]: 0x85e60320 0x803fffff 0x7086de82 0x0

62 [0x23b2000]: 0x85e60320 0x803fffff 0x7086d602 0x0

63 [0x23b2000]: 0x85e60320 0x803fffff 0x7086cd82 0x0

64 [0x23b2000]: 0x85e60320 0x803fffff 0x7086c502 0x0

65 [0x23b2000]: 0x85e60320 0x803fffff 0x7086bc82 0x0

66 [0x23b2000]: 0x85e60320 0x803fffff 0x7086b402 0x0

67 [0x23b2000]: 0x85e60320 0x803fffff 0x7086ab82 0x0

68 [0x23b2000]: 0x85e60320 0x803fffff 0x7086a302 0x0

69 [0x23b2000]: 0x85e60320 0x803fffff 0x70869a82 0x0

70 [0x23b2000]: 0x85e60320 0x803fffff 0x70869202 0x0

71 [0x23b2000]: 0x85e60320 0x803fffff 0x70868982 0x0

72 [0x23b2000]: 0x85e60320 0x803fffff 0x70868102 0x0

73 [0x23b2000]: 0x85e60320 0x803fffff 0x7200f802 0x0

74 [0x23b2000]: 0x85e60320 0x803fffff 0x7200ef82 0x0

75 [0x23b2000]: 0x85e60320 0x803fffff 0x7200e702 0x0

76 [0x23b2000]: 0x85e60320 0x803fffff 0x7200de82 0x0

77 [0x23b2000]: 0x85e60320 0x803fffff 0x7200d602 0x0

78 [0x23b2000]: 0x85e60320 0x803fffff 0x7200cd82 0x0

79 [0x23b2000]: 0x85e60320 0x803fffff 0x7200c502 0x0

80 [0x23b2000]: 0x85e60320 0x803fffff 0x7200bc82 0x0

81 [0x23b2000]: 0x85e60320 0x803fffff 0x7200b402 0x0

82 [0x23b2000]: 0x85e60320 0x803fffff 0x7200ab82 0x0

83 [0x23b2000]: 0x85e60320 0x803fffff 0x7200a302 0x0

84 [0x23b2000]: 0x85e60320 0x803fffff 0x72009a82 0x0

85 [0x23b2000]: 0x85e60320 0x803fffff 0x72009202 0x0

86 [0x23b2000]: 0x85e60320 0x803fffff 0x72008982 0x0

87 [0x23b2000]: 0x85e60320 0x803fffff 0x72008102 0x0

88 [0x23b2000]: 0x85e60320 0x803fffff 0x721e7802 0x0

89 [0x23b2000]: 0x85e60320 0x803fffff 0x721e6f82 0x0

90 [0x23b2000]: 0x85e60320 0x803fffff 0x721e6702 0x0

91 [0x23b2000]: 0x85e60320 0x803fffff 0x721e5e82 0x0

92 [0x23b2000]: 0x85e60320 0x803fffff 0x721e5602 0x0

93 [0x23b2000]: 0x85e60320 0x803fffff 0x721e4d82 0x0

94 [0x23b2000]: 0x85e60320 0x803fffff 0x721e4502 0x0

95 [0x23b2000]: 0x85e60320 0x803fffff 0x721e3c82 0x0

96 [0x23b2000]: 0x85e60320 0x803fffff 0x721e3402 0x0

97 [0x23b2000]: 0x85e60320 0x803fffff 0x721e2b82 0x0

98 [0x23b2000]: 0x85e60320 0x803fffff 0x721e2302 0x0

99 [0x23b2000]: 0x85e60320 0x803fffff 0x721e1a82 0x0

100 [0x23b2000]: 0x85e60320 0x803fffff 0x721e1202 0x0

101 [0x23b2000]: 0x85e60320 0x803fffff 0x721e0982 0x0

102 [0x23b2000]: 0x85e60320 0x803fffff 0x721e0102 0x0

103 [0x23b2000]: 0x85e60320 0x803fffff 0x73687802 0x0

104 [0x23b2000]: 0x85e60320 0x803fffff 0x73686f82 0x0

105 [0x23b2000]: 0x85e60320 0x803fffff 0x73686702 0x0

106 [0x23b2000]: 0x85e60320 0x803fffff 0x73685e82 0x0

107 [0x23b2000]: 0x85e60320 0x803fffff 0x73685602 0x0

108 [0x23b2000]: 0x85e60320 0x803fffff 0x73684d82 0x0

109 [0x23b2000]: 0x85e60320 0x803fffff 0x73684502 0x0

110 [0x23b2000]: 0x85e60320 0x803fffff 0x73683c82 0x0

111 [0x23b2000]: 0x85e60320 0x803fffff 0x73683402 0x0

112 [0x23b2000]: 0x85e60320 0x803fffff 0x73682b82 0x0

113 [0x23b2000]: 0x85e60320 0x803fffff 0x73682302 0x0

114 [0x23b2000]: 0x85e60320 0x803fffff 0x73681a82 0x0

115 [0x23b2000]: 0x85e60320 0x803fffff 0x73681202 0x0

116 [0x23b2000]: 0x85e60320 0x803fffff 0x73680982 0x0

117 [0x23b2000]: 0x85e60320 0x803fffff 0x73680102 0x0

118 [0x23b2000]: 0x85e60320 0x803fffff 0x7205f802 0x0

119 [0x23b2000]: 0x85e60320 0x803fffff 0x7205ef82 0x0

120 [0x23b2000]: 0x85e60320 0x803fffff 0x7205e702 0x0

121 [0x23b2000]: 0x85e60320 0x803fffff 0x7205de82 0x0

122 [0x23b2000]: 0x85e60320 0x803fffff 0x7205d602 0x0

123 [0x23b2000]: 0x85e60320 0x803fffff 0x7205cd82 0x0

124 [0x23b2000]: 0x85e60320 0x803fffff 0x7205c502 0x0

125 [0x23b2000]: 0x85e60320 0x803fffff 0x7205bc82 0x0

126 [0x23b2000]: 0x85e60320 0x803fffff 0x7205b402 0x0

127 [0x23b2000]: 0x85e60320 0x803fffff 0x7205ab82 0x0

128 [0x23b2000]: 0x85e60320 0x803fffff 0x7205a302 0x0

129 [0x23b2000]: 0x85e60320 0x803fffff 0x72059a82 0x0

130 [0x23b2000]: 0x85e60320 0x803fffff 0x72059202 0x0

131 [0x23b2000]: 0x85e60320 0x803fffff 0x72058982 0x0

132 [0x23b2000]: 0x85e60320 0x803fffff 0x72058102 0x0

133 [0x23b2000]: 0x85e60320 0x803fffff 0x72167802 0x0

134 [0x23b2000]: 0x85e60320 0x803fffff 0x72166f82 0x0

135 [0x23b2000]: 0x85e60320 0x803fffff 0x72166702 0x0

136 [0x23b2000]: 0x85e60320 0x803fffff 0x72165e82 0x0

137 [0x23b2000]: 0x85e60320 0x803fffff 0x72165602 0x0

138 [0x23b2000]: 0x85e60320 0x803fffff 0x72164d82 0x0

139 [0x23b2000]: 0x85e60320 0x803fffff 0x72164502 0x0

140 [0x23b2000]: 0x85e60320 0x803fffff 0x72163c82 0x0

141 [0x23b2000]: 0x85e60320 0x803fffff 0x72163402 0x0

142 [0x23b2000]: 0x85e60320 0x803fffff 0x72162b82 0x0

143 [0x23b2000]: 0x85e60320 0x803fffff 0x72162302 0x0

144 [0x23b2000]: 0x85e60320 0x803fffff 0x72161a82 0x0

145 [0x23b2000]: 0x85e60320 0x803fffff 0x72161202 0x0

146 [0x23b2000]: 0x85e60320 0x803fffff 0x72160982 0x0

147 [0x23b2000]: 0x85e60320 0x803fffff 0x72160102 0x0

148 [0x23b2000]: 0x85e60320 0x803fffff 0x737ef802 0x0

149 [0x23b2000]: 0x85e60320 0x803fffff 0x737eef82 0x0

150 [0x23b2000]: 0x85e60320 0x803fffff 0x737ee702 0x0

151 [0x23b2000]: 0x85e60320 0x803fffff 0x737ede82 0x0

152 [0x23b2000]: 0x85e60320 0x803fffff 0x737ed602 0x0

153 [0x23b2000]: 0x85e60320 0x803fffff 0x737ecd82 0x0

154 [0x23b2000]: 0x85e60320 0x803fffff 0x737ec502 0x0

155 [0x23b2000]: 0x85e60320 0x803fffff 0x737ebc82 0x0

156 [0x23b2000]: 0x85e60320 0x803fffff 0x737eb402 0x0

157 [0x23b2000]: 0x85e60320 0x803fffff 0x737eab82 0x0

158 [0x23b2000]: 0x85e60320 0x803fffff 0x737ea302 0x0

159 [0x23b2000]: 0x85e60320 0x803fffff 0x737e9a82 0x0

160 [0x23b2000]: 0x85e60320 0x803fffff 0x737e9202 0x0

161 [0x23b2000]: 0x85e60320 0x803fffff 0x737e8982 0x0

162 [0x23b2000]: 0x85e60320 0x803fffff 0x737e8102 0x0

163 [0x23b2000]: 0x85e60320 0x803fffff 0x735bf802 0x0

164 [0x23b2000]: 0x85e60320 0x803fffff 0x735bef82 0x0

165 [0x23b2000]: 0x85e60320 0x803fffff 0x735be702 0x0

166 [0x23b2000]: 0x85e60320 0x803fffff 0x735bde82 0x0

167 [0x23b2000]: 0x85e60320 0x803fffff 0x735bd602 0x0

168 [0x23b2000]: 0x85e60320 0x803fffff 0x735bcd82 0x0

169 [0x23b2000]: 0x85e60320 0x803fffff 0x735bc502 0x0

170 [0x23b2000]: 0x85e60320 0x803fffff 0x735bbc82 0x0

171 [0x23b2000]: 0x85e60320 0x803fffff 0x735bb402 0x0

172 [0x23b2000]: 0x85e60320 0x803fffff 0x735bab82 0x0

173 [0x23b2000]: 0x85e60320 0x803fffff 0x735ba302 0x0

174 [0x23b2000]: 0x85e60320 0x803fffff 0x735b9a82 0x0

175 [0x23b2000]: 0x85e60320 0x803fffff 0x735b9202 0x0

176 [0x23b2000]: 0x85e60320 0x803fffff 0x735b8982 0x0

177 [0x23b2000]: 0x85e60320 0x803fffff 0x735b8102 0x0

178 [0x23b2000]: 0x85e60320 0x803fffff 0x73697802 0x0

179 [0x23b2000]: 0x85e60320 0x803fffff 0x73696f82 0x0

180 [0x23b2000]: 0x85e60320 0x803fffff 0x73696702 0x0

181 [0x23b2000]: 0x85e60320 0x803fffff 0x73695e82 0x0

182 [0x23b2000]: 0x85e60320 0x803fffff 0x73695602 0x0

183 [0x23b2000]: 0x85e60320 0x803fffff 0x73694d82 0x0

184 [0x23b2000]: 0x85e60320 0x803fffff 0x73694502 0x0

185 [0x23b2000]: 0x85e60320 0x803fffff 0x73693c82 0x0

186 [0x23b2000]: 0x85e60320 0x803fffff 0x73693402 0x0

187 [0x23b2000]: 0x85e60320 0x803fffff 0x73692b82 0x0

188 [0x23b2000]: 0x85e60320 0x803fffff 0x73692302 0x0

189 [0x23b2000]: 0x85e60320 0x803fffff 0x73691a82 0x0

190 [0x23b2000]: 0x85e60320 0x803fffff 0x73691202 0x0

191 [0x23b2000]: 0x85e60320 0x803fffff 0x73690982 0x0

192 [0x23b2000]: 0x85e60320 0x803fffff 0x73690102 0x0

193 [0x23b2000]: 0x85e60320 0x803fffff 0x7081f802 0x0

194 [0x23b2000]: 0x85e60320 0x803fffff 0x7081ef82 0x0

195 [0x23b2000]: 0x85e60320 0x803fffff 0x7081e702 0x0

196 [0x23b2000]: 0x85e60320 0x803fffff 0x7081de82 0x0

197 [0x23b2000]: 0x85e60320 0x803fffff 0x7081d602 0x0

198 [0x23b2000]: 0x85e60320 0x803fffff 0x7081cd82 0x0

199 [0x23b2000]: 0x85e60320 0x803fffff 0x7081c502 0x0

200 [0x23b2000]: 0x85e60320 0x803fffff 0x7081bc82 0x0

201 [0x23b2000]: 0x85e60320 0x803fffff 0x7081b402 0x0

202 [0x23b2000]: 0x85e60320 0x803fffff 0x7081ab82 0x0

203 [0x23b2000]: 0x85e60320 0x803fffff 0x7081a302 0x0

204 [0x23b2000]: 0x85e60320 0x803fffff 0x70819a82 0x0

205 [0x23b2000]: 0x85e60320 0x803fffff 0x70819202 0x0

206 [0x23b2000]: 0x85e60320 0x803fffff 0x70818982 0x0

207 [0x23b2000]: 0x85e60320 0x803fffff 0x70818102 0x0

208 [0x23b2000]: 0x85e60320 0x803fffff 0x73607802 0x0

209 [0x23b2000]: 0x85e60320 0x803fffff 0x73606f82 0x0

210 [0x23b2000]: 0x85e60320 0x803fffff 0x73606702 0x0

211 [0x23b2000]: 0x85e60320 0x803fffff 0x73605e82 0x0

212 [0x23b2000]: 0x85e60320 0x803fffff 0x73605602 0x0

213 [0x23b2000]: 0x85e60320 0x803fffff 0x73604d82 0x0

214 [0x23b2000]: 0x85e60320 0x803fffff 0x73604502 0x0

215 [0x23b2000]: 0x85e60320 0x803fffff 0x73603c82 0x0

216 [0x23b2000]: 0x85e60320 0x803fffff 0x73603402 0x0

217 [0x23b2000]: 0x85e60320 0x803fffff 0x73602b82 0x0

218 [0x23b2000]: 0x85e60320 0x803fffff 0x73602302 0x0

219 [0x23b2000]: 0x85e60320 0x803fffff 0x73601a82 0x0

220 [0x23b2000]: 0x85e60320 0x803fffff 0x73601202 0x0

221 [0x23b2000]: 0x85e60320 0x803fffff 0x73600982 0x0

222 [0x23b2000]: 0x85e60320 0x803fffff 0x73600102 0x0

223 [0x23b2000]: 0x85e60320 0x803fffff 0x735f7802 0x0

224 [0x23b2000]: 0x85e60320 0x803fffff 0x735f6f82 0x0

225 [0x23b2000]: 0x85e60320 0x803fffff 0x735f6702 0x0

226 [0x23b2000]: 0x85e60320 0x803fffff 0x735f5e82 0x0

227 [0x23b2000]: 0x85e60320 0x803fffff 0x735f5602 0x0

228 [0x23b2000]: 0x85e60320 0x803fffff 0x735f4d82 0x0

229 [0x23b2000]: 0x85e60320 0x803fffff 0x735f4502 0x0

230 [0x23b2000]: 0x85e60320 0x803fffff 0x735f3c82 0x0

231 [0x23b2000]: 0x85e60320 0x803fffff 0x735f3402 0x0

232 [0x23b2000]: 0x85e60320 0x803fffff 0x735f2b82 0x0

233 [0x23b2000]: 0x85e60320 0x803fffff 0x735f2302 0x0

234 [0x23b2000]: 0x85e60320 0x803fffff 0x735f1a82 0x0

235 [0x23b2000]: 0x85e60320 0x803fffff 0x735f1202 0x0

236 [0x23b2000]: 0x85e60320 0x803fffff 0x735f0982 0x0

237 [0x23b2000]: 0x85e60320 0x803fffff 0x735f0102 0x0

238 [0x23b2000]: 0x85e60320 0x803fffff 0x7369f802 0x0

239 [0x23b2000]: 0x85e60320 0x803fffff 0x7369ef82 0x0

240 [0x23b2000]: 0x85e60320 0x803fffff 0x7369e702 0x0

241 [0x23b2000]: 0x85e60320 0x803fffff 0x7369de82 0x0

242 [0x23b2000]: 0x85e60320 0x803fffff 0x7369d602 0x0

243 [0x23b2000]: 0x85e60320 0x803fffff 0x7369cd82 0x0

244 [0x23b2000]: 0x85e60320 0x803fffff 0x7369c502 0x0

245 [0x23b2000]: 0x85e60320 0x803fffff 0x7369bc82 0x0

246 [0x23b2000]: 0x85e60320 0x803fffff 0x7369b402 0x0

247 [0x23b2000]: 0x85e60320 0x803fffff 0x7369ab82 0x0

248 [0x23b2000]: 0x85e60320 0x803fffff 0x7369a302 0x0

249 [0x23b2000]: 0x85e60320 0x803fffff 0x73699a82 0x0

250 [0x23b2000]: 0x85e60320 0x803fffff 0x73699202 0x0

251 [0x23b2000]: 0x85e60320 0x803fffff 0x73698982 0x0

252 [0x23b2000]: 0x85e60320 0x803fffff 0x73698102 0x0

253 [0x23b2000]: 0x85e60320 0x803fffff 0x7360f802 0x0

254 [0x23b2000]: 0x85e60320 0x803fffff 0x7360ef82 0x0

255 [0x23b2000]: 0x85e60320 0x803fffff 0x7360e702 0x0

256 [0x23b2000]: 0x85e60320 0x803fffff 0x7360de82 0x0

257 [0x23b2000]: 0x85e60320 0x803fffff 0x7360d602 0x0

258 [0x23b2000]: 0x85e60320 0x803fffff 0x7360cd82 0x0

259 [0x23b2000]: 0x85e60320 0x803fffff 0x7360c502 0x0

260 [0x23b2000]: 0x85e60320 0x803fffff 0x7360bc82 0x0

261 [0x23b2000]: 0x85e60320 0x803fffff 0x7360b402 0x0

262 [0x23b2000]: 0x85e60320 0x803fffff 0x7360ab82 0x0

263 [0x23b2000]: 0x85e60320 0x803fffff 0x7360a302 0x0

264 [0x23b2000]: 0x85e60320 0x803fffff 0x73609a82 0x0

265 [0x23b2000]: 0x85e60320 0x803fffff 0x73609202 0x0

266 [0x23b2000]: 0x85e60320 0x803fffff 0x73608982 0x0

267 [0x23b2000]: 0x85e60320 0x803fffff 0x73608102 0x0

268 [0x23b2000]: 0x85e60320 0x803fffff 0x735ff802 0x0

269 [0x23b2000]: 0x85e60320 0x803fffff 0x735fef82 0x0

270 [0x23b2000]: 0x85e60320 0x803fffff 0x735fe702 0x0

271 [0x23b2000]: 0x85e60320 0x803fffff 0x735fde82 0x0

272 [0x23b2000]: 0x85e60320 0x803fffff 0x735fd602 0x0

273 [0x23b2000]: 0x85e60320 0x803fffff 0x735fcd82 0x0

274 [0x23b2000]: 0x85e60320 0x803fffff 0x735fc502 0x0

275 [0x23b2000]: 0x85e60320 0x803fffff 0x735fbc82 0x0

276 [0x23b2000]: 0x85e60320 0x803fffff 0x735fb402 0x0

277 [0x23b2000]: 0x85e60320 0x803fffff 0x735fab82 0x0

278 [0x23b2000]: 0x85e60320 0x803fffff 0x735fa302 0x0

279 [0x23b2000]: 0x85e60320 0x803fffff 0x735f9a82 0x0

280 [0x23b2000]: 0x85e60320 0x803fffff 0x735f9202 0x0

281 [0x23b2000]: 0x85e60320 0x803fffff 0x735f8982 0x0

282 [0x23b2000]: 0x85e60320 0x803fffff 0x735f8102 0x0

283 [0x23b2000]: 0x85e60320 0x803fffff 0x737f7802 0x0

284 [0x23b2000]: 0x85e60320 0x803fffff 0x737f6f82 0x0

285 [0x23b2000]: 0x85e60320 0x803fffff 0x737f6702 0x0

286 [0x23b2000]: 0x85e60320 0x803fffff 0x737f5e82 0x0

287 [0x23b2000]: 0x85e60320 0x803fffff 0x737f5602 0x0

288 [0x23b2000]: 0x85e60320 0x803fffff 0x737f4d82 0x0

289 [0x23b2000]: 0x85e60320 0x803fffff 0x737f4502 0x0

290 [0x23b2000]: 0x85e60320 0x803fffff 0x737f3c82 0x0

291 [0x23b2000]: 0x85e60320 0x803fffff 0x737f3402 0x0

292 [0x23b2000]: 0x85e60320 0x803fffff 0x737f2b82 0x0

293 [0x23b2000]: 0x85e60320 0x803fffff 0x737f2302 0x0

294 [0x23b2000]: 0x85e60320 0x803fffff 0x737f1a82 0x0

295 [0x23b2000]: 0x85e60320 0x803fffff 0x737f1202 0x0

296 [0x23b2000]: 0x85e60320 0x803fffff 0x737f0982 0x0

297 [0x23b2000]: 0x85e60320 0x803fffff 0x737f0102 0x0

298 [0x23b2000]: 0x85e60320 0x803fffff 0x735e7802 0x0

299 [0x23b2000]: 0x85e60320 0x803fffff 0x735e6f82 0x0

300 [0x23b2000]: 0x85e60320 0x803fffff 0x735e6702 0x0

301 [0x23b2000]: 0x85e60320 0x803fffff 0x735e5e82 0x0

302 [0x23b2000]: 0x85e60320 0x803fffff 0x735e5602 0x0

303 [0x23b2000]: 0x85e60320 0x803fffff 0x735e4d82 0x0

304 [0x23b2000]: 0x85e60320 0x803fffff 0x735e4502 0x0

305 [0x23b2000]: 0x85e60320 0x803fffff 0x735e3c82 0x0

306 [0x23b2000]: 0x85e60320 0x803fffff 0x735e3402 0x0

307 [0x23b2000]: 0x85e60320 0x803fffff 0x735e2b82 0x0

308 [0x23b2000]: 0x85e60320 0x803fffff 0x735e2302 0x0

309 [0x23b2000]: 0x85e60320 0x803fffff 0x735e1a82 0x0

310 [0x23b2000]: 0x85e60320 0x803fffff 0x735e1202 0x0

311 [0x23b2000]: 0x85e60320 0x803fffff 0x735e0982 0x0

312 [0x23b2000]: 0x85e60320 0x803fffff 0x735e0102 0x0

313 [0x23b2000]: 0x85e60320 0x803fffff 0x735ef802 0x0

314 [0x23b2000]: 0x85e60320 0x803fffff 0x735eef82 0x0

315 [0x23b2000]: 0x85e60320 0x803fffff 0x735ee702 0x0

316 [0x23b2000]: 0x85e60320 0x803fffff 0x735ede82 0x0

317 [0x23b2000]: 0x85e60320 0x803fffff 0x735ed602 0x0

318 [0x23b2000]: 0x85e60320 0x803fffff 0x735ecd82 0x0

319 [0x23b2000]: 0x85e60320 0x803fffff 0x735ec502 0x0

320 [0x23b2000]: 0x85e60320 0x803fffff 0x735ebc82 0x0

321 [0x23b2000]: 0x85e60320 0x803fffff 0x735eb402 0x0

322 [0x23b2000]: 0x85e60320 0x803fffff 0x735eab82 0x0

323 [0x23b2000]: 0x85e60320 0x803fffff 0x735ea302 0x0

324 [0x23b2000]: 0x85e60320 0x803fffff 0x735e9a82 0x0

325 [0x23b2000]: 0x85e60320 0x803fffff 0x735e9202 0x0

326 [0x23b2000]: 0x85e60320 0x803fffff 0x735e8982 0x0

327 [0x23b2000]: 0x85e60320 0x803fffff 0x735e8102 0x0

328 [0x23b2000]: 0x85e60320 0x803fffff 0x708d7802 0x0

329 [0x23b2000]: 0x85e60320 0x803fffff 0x708d6f82 0x0

330 [0x23b2000]: 0x85e60320 0x803fffff 0x708d6702 0x0

331 [0x23b2000]: 0x85e60320 0x803fffff 0x708d5e82 0x0

332 [0x23b2000]: 0x85e60320 0x803fffff 0x708d5602 0x0

333 [0x23b2000]: 0x85e60320 0x803fffff 0x708d4d82 0x0

334 [0x23b2000]: 0x85e60320 0x803fffff 0x708d4502 0x0

335 [0x23b2000]: 0x85e60320 0x803fffff 0x708d3c82 0x0

336 [0x23b2000]: 0x85e60320 0x803fffff 0x708d3402 0x0

337 [0x23b2000]: 0x85e60320 0x803fffff 0x708d2b82 0x0

338 [0x23b2000]: 0x85e60320 0x803fffff 0x708d2302 0x0

339 [0x23b2000]: 0x85e60320 0x803fffff 0x708d1a82 0x0

340 [0x23b2000]: 0x85e60320 0x803fffff 0x708d1202 0x0

341 [0x23b2000]: 0x85e60320 0x803fffff 0x708d0982 0x0

342 [0x23b2000]: 0x85e60320 0x803fffff 0x708d0102 0x0

343 [0x23b2000]: 0x85e60320 0x803fffff 0x72127802 0x0

344 [0x23b2000]: 0x85e60320 0x803fffff 0x72126f82 0x0

345 [0x23b2000]: 0x85e60320 0x803fffff 0x72126702 0x0

346 [0x23b2000]: 0x85e60320 0x803fffff 0x72125e82 0x0

347 [0x23b2000]: 0x85e60320 0x803fffff 0x72125602 0x0

348 [0x23b2000]: 0x85e60320 0x803fffff 0x72124d82 0x0

349 [0x23b2000]: 0x85e60320 0x803fffff 0x72124502 0x0

350 [0x23b2000]: 0x85e60320 0x803fffff 0x72123c82 0x0

351 [0x23b2000]: 0x85e60320 0x803fffff 0x72123402 0x0

352 [0x23b2000]: 0x85e60320 0x803fffff 0x72122b82 0x0

353 [0x23b2000]: 0x85e60320 0x803fffff 0x72122302 0x0

354 [0x23b2000]: 0x85e60320 0x803fffff 0x72121a82 0x0

355 [0x23b2000]: 0x85e60320 0x803fffff 0x72121202 0x0

356 [0x23b2000]: 0x85e60320 0x803fffff 0x72120982 0x0

357 [0x23b2000]: 0x85e60320 0x803fffff 0x72120102 0x0

358 [0x23b2000]: 0x85e60320 0x803fffff 0x720c7802 0x0

359 [0x23b2000]: 0x85e60320 0x803fffff 0x720c6f82 0x0

360 [0x23b2000]: 0x85e60320 0x803fffff 0x720c6702 0x0

361 [0x23b2000]: 0x85e60320 0x803fffff 0x720c5e82 0x0

362 [0x23b2000]: 0x85e60320 0x803fffff 0x720c5602 0x0

363 [0x23b2000]: 0x85e60320 0x803fffff 0x720c4d82 0x0

364 [0x23b2000]: 0x85e60320 0x803fffff 0x720c4502 0x0

365 [0x23b2000]: 0x85e60320 0x803fffff 0x720c3c82 0x0

366 [0x23b2000]: 0x85e60320 0x803fffff 0x720c3402 0x0

367 [0x23b2000]: 0x85e60320 0x803fffff 0x720c2b82 0x0

368 [0x23b2000]: 0x85e60320 0x803fffff 0x720c2302 0x0

369 [0x23b2000]: 0x85e60320 0x803fffff 0x720c1a82 0x0

370 [0x23b2000]: 0x85e60320 0x803fffff 0x720c1202 0x0

371 [0x23b2000]: 0x85e60320 0x803fffff 0x720c0982 0x0

372 [0x23b2000]: 0x85e60320 0x803fffff 0x720c0102 0x0

373 [0x23b2000]: 0x85e60320 0x803fffff 0x7379f802 0x0

374 [0x23b2000]: 0x85e60320 0x803fffff 0x7379ef82 0x0

375 [0x23b2000]: 0x85e60320 0x803fffff 0x7379e702 0x0

376 [0x23b2000]: 0x85e60320 0x803fffff 0x7379de82 0x0

377 [0x23b2000]: 0x85e60320 0x803fffff 0x7379d602 0x0

378 [0x23b2000]: 0x85e60320 0x803fffff 0x7379cd82 0x0

379 [0x23b2000]: 0x85e60320 0x803fffff 0x7379c502 0x0

380 [0x23b2000]: 0x85e60320 0x803fffff 0x7379bc82 0x0

381 [0x23b2000]: 0x85e60320 0x803fffff 0x7379b402 0x0

382 [0x23b2000]: 0x85e60320 0x803fffff 0x7379ab82 0x0

383 [0x23b2000]: 0x85e60320 0x803fffff 0x7379a302 0x0

384 [0x23b2000]: 0x85e60320 0x803fffff 0x73799a82 0x0

385 [0x23b2000]: 0x85e60320 0x803fffff 0x73799202 0x0

386 [0x23b2000]: 0x85e60320 0x803fffff 0x73798982 0x0

387 [0x23b2000]: 0x85e60320 0x803fffff 0x73798102 0x0

388 [0x23b2000]: 0x85e60320 0x803fffff 0x72107802 0x0

389 [0x23b2000]: 0x85e60320 0x803fffff 0x72106f82 0x0

390 [0x23b2000]: 0x85e60320 0x803fffff 0x72106702 0x0

391 [0x23b2000]: 0x85e60320 0x803fffff 0x72105e82 0x0

392 [0x23b2000]: 0x85e60320 0x803fffff 0x72105602 0x0

393 [0x23b2000]: 0x85e60320 0x803fffff 0x72104d82 0x0

394 [0x23b2000]: 0x85e60320 0x803fffff 0x72104502 0x0

395 [0x23b2000]: 0x85e60320 0x803fffff 0x72103c82 0x0

396 [0x23b2000]: 0x85e60320 0x803fffff 0x72103402 0x0

397 [0x23b2000]: 0x85e60320 0x803fffff 0x72102b82 0x0

398 [0x23b2000]: 0x85e60320 0x803fffff 0x72102302 0x0

399 [0x23b2000]: 0x85e60320 0x803fffff 0x72101a82 0x0

400 [0x23b2000]: 0x85e60320 0x803fffff 0x72101202 0x0

401 [0x23b2000]: 0x85e60320 0x803fffff 0x72100982 0x0

402 [0x23b2000]: 0x85e60320 0x803fffff 0x72100102 0x0

403 [0x23b2000]: 0x85e60320 0x803fffff 0x73557802 0x0

404 [0x23b2000]: 0x85e60320 0x803fffff 0x73556f82 0x0

405 [0x23b2000]: 0x85e60320 0x803fffff 0x73556702 0x0

406 [0x23b2000]: 0x85e60320 0x803fffff 0x73555e82 0x0

407 [0x23b2000]: 0x85e60320 0x803fffff 0x73555602 0x0

408 [0x23b2000]: 0x85e60320 0x803fffff 0x73554d82 0x0

409 [0x23b2000]: 0x85e60320 0x803fffff 0x73554502 0x0

410 [0x23b2000]: 0x85e60320 0x803fffff 0x73553c82 0x0

411 [0x23b2000]: 0x85e60320 0x803fffff 0x73553402 0x0

412 [0x23b2000]: 0x85e60320 0x803fffff 0x73552b82 0x0

413 [0x23b2000]: 0x85e60320 0x803fffff 0x73552302 0x0

414 [0x23b2000]: 0x85e60320 0x803fffff 0x73551a82 0x0

415 [0x23b2000]: 0x85e60320 0x803fffff 0x73551202 0x0

416 [0x23b2000]: 0x85e60320 0x803fffff 0x73550982 0x0

417 [0x23b2000]: 0x85e60320 0x803fffff 0x73550102 0x0

418 [0x23b2000]: 0x85e60320 0x803fffff 0x73797802 0x0

419 [0x23b2000]: 0x85e60320 0x803fffff 0x73796f82 0x0

420 [0x23b2000]: 0x85e60320 0x803fffff 0x73796702 0x0

421 [0x23b2000]: 0x85e60320 0x803fffff 0x73795e82 0x0

422 [0x23b2000]: 0x85e60320 0x803fffff 0x73795602 0x0

423 [0x23b2000]: 0x85e60320 0x803fffff 0x73794d82 0x0

424 [0x23b2000]: 0x85e60320 0x803fffff 0x73794502 0x0

425 [0x23b2000]: 0x85e60320 0x803fffff 0x73793c82 0x0

426 [0x23b2000]: 0x85e60320 0x803fffff 0x73793402 0x0

427 [0x23b2000]: 0x85e60320 0x803fffff 0x73792b82 0x0

428 [0x23b2000]: 0x85e60320 0x803fffff 0x73792302 0x0

429 [0x23b2000]: 0x85e60320 0x803fffff 0x73791a82 0x0

430 [0x23b2000]: 0x85e60320 0x803fffff 0x73791202 0x0

431 [0x23b2000]: 0x85e60320 0x803fffff 0x73790982 0x0

432 [0x23b2000]: 0x85e60320 0x803fffff 0x73790102 0x0

433 [0x23b2000]: 0x85e60320 0x803fffff 0x73517802 0x0

434 [0x23b2000]: 0x85e60320 0x803fffff 0x73516f82 0x0

435 [0x23b2000]: 0x85e60320 0x803fffff 0x73516702 0x0

436 [0x23b2000]: 0x85e60320 0x803fffff 0x73515e82 0x0

437 [0x23b2000]: 0x85e60320 0x803fffff 0x73515602 0x0

438 [0x23b2000]: 0x85e60320 0x803fffff 0x73514d82 0x0

439 [0x23b2000]: 0x85e60320 0x803fffff 0x73514502 0x0

440 [0x23b2000]: 0x85e60320 0x803fffff 0x73513c82 0x0

441 [0x23b2000]: 0x85e60320 0x803fffff 0x73513402 0x0

442 [0x23b2000]: 0x85e60320 0x803fffff 0x73512b82 0x0

443 [0x23b2000]: 0x85e60320 0x803fffff 0x73512302 0x0

444 [0x23b2000]: 0x85e60320 0x803fffff 0x73511a82 0x0

445 [0x23b2000]: 0x85e60320 0x803fffff 0x73511202 0x0

446 [0x23b2000]: 0x85e60320 0x803fffff 0x73510982 0x0

447 [0x23b2000]: 0x85e60320 0x803fffff 0x73510102 0x0

448 [0x23b2000]: 0x85e60320 0x803fffff 0x708df802 0x0

449 [0x23b2000]: 0x85e60320 0x803fffff 0x708def82 0x0

450 [0x23b2000]: 0x85e60320 0x803fffff 0x708de702 0x0

451 [0x23b2000]: 0x85e60320 0x803fffff 0x708dde82 0x0

452 [0x23b2000]: 0x85e60320 0x803fffff 0x708dd602 0x0

453 [0x23b2000]: 0x85e60320 0x803fffff 0x708dcd82 0x0

454 [0x23b2000]: 0x85e60320 0x803fffff 0x708dc502 0x0

455 [0x23b2000]: 0x85e60320 0x803fffff 0x708dbc82 0x0

456 [0x23b2000]: 0x85e60320 0x803fffff 0x708db402 0x0

457 [0x23b2000]: 0x85e60320 0x803fffff 0x708dab82 0x0

458 [0x23b2000]: 0x85e60320 0x803fffff 0x708da302 0x0

459 [0x23b2000]: 0x85e60320 0x803fffff 0x708d9a82 0x0

460 [0x23b2000]: 0x85e60320 0x803fffff 0x708d9202 0x0

461 [0x23b2000]: 0x85e60320 0x803fffff 0x708d8982 0x0

462 [0x23b2000]: 0x85e60320 0x803fffff 0x708d8102 0x0

463 [0x23b2000]: 0x85e60320 0x803fffff 0x720e7802 0x0

464 [0x23b2000]: 0x85e60320 0x803fffff 0x720e6f82 0x0

465 [0x23b2000]: 0x85e60320 0x803fffff 0x720e6702 0x0

466 [0x23b2000]: 0x85e60320 0x803fffff 0x720e5e82 0x0

467 [0x23b2000]: 0x85e60320 0x803fffff 0x720e5602 0x0

468 [0x23b2000]: 0x85e60320 0x803fffff 0x720e4d82 0x0

469 [0x23b2000]: 0x85e60320 0x803fffff 0x720e4502 0x0

470 [0x23b2000]: 0x85e60320 0x803fffff 0x720e3c82 0x0

471 [0x23b2000]: 0x85e60320 0x803fffff 0x720e3402 0x0

472 [0x23b2000]: 0x85e60320 0x803fffff 0x720e2b82 0x0

473 [0x23b2000]: 0x85e60320 0x803fffff 0x720e2302 0x0

474 [0x23b2000]: 0x85e60320 0x803fffff 0x720e1a82 0x0

475 [0x23b2000]: 0x85e60320 0x803fffff 0x720e1202 0x0

476 [0x23b2000]: 0x85e60320 0x803fffff 0x720e0982 0x0

477 [0x23b2000]: 0x85e60320 0x803fffff 0x720e0102 0x0

478 [0x23b2000]: 0x85e60320 0x803fffff 0x7375f802 0x0

479 [0x23b2000]: 0x85e60320 0x803fffff 0x7375ef82 0x0

480 [0x23b2000]: 0x85e60320 0x803fffff 0x7375e702 0x0

481 [0x23b2000]: 0x85e60320 0x803fffff 0x7375de82 0x0

482 [0x23b2000]: 0x85e60320 0x803fffff 0x7375d602 0x0

483 [0x23b2000]: 0x85e60320 0x803fffff 0x7375cd82 0x0

484 [0x23b2000]: 0x85e60320 0x803fffff 0x7375c502 0x0

485 [0x23b2000]: 0x85e60320 0x803fffff 0x7375bc82 0x0

486 [0x23b2000]: 0x85e60320 0x803fffff 0x7375b402 0x0

487 [0x23b2000]: 0x83cd0320 0x803fffff 0x7375ab82 0x0

488 [0x23b2000]: 0x80460320 0x803fffff 0x70863402 0x0

489 [0x23b2000]: 0x80400381 0x803fffff 0x70862b82 0x0

490 [0x23b2000]: 0x80400381 0x803fffff 0x70862302 0x0

491 [0x23b2000]: 0x80400381 0x803fffff 0x70861a82 0x0

492 [0x23b2000]: 0x80400381 0x803fffff 0x70861202 0x0

493 [0x23b2000]: 0x80400381 0x803fffff 0x70860982 0x0

494 [0x23b2000]: 0x80400381 0x803fffff 0x70860102 0x0

495 [0x23b2000]: 0x80400381 0x803fffff 0x73617802 0x0

496 [0x23b2000]: 0x80400381 0x803fffff 0x73616f82 0x0

497 [0x23b2000]: 0x80400381 0x803fffff 0x73616702 0x0

498 [0x23b2000]: 0x80400301 0x803fffff 0x73615e82 0x0

499 [0x23b2000]: 0x80400381 0x803fffff 0x73615602 0x0

500 [0x23b2000]: 0x80400381 0x803fffff 0x73614d82 0x0

501 [0x23b2000]: 0x80400381 0x803fffff 0x73614502 0x0

502 [0x23b2000]: 0x80400381 0x803fffff 0x73613c82 0x0

503 [0x23b2000]: 0x80400381 0x803fffff 0x73613402 0x0

504 [0x23b2000]: 0x80400381 0x803fffff 0x73612b82 0x0

505 [0x23b2000]: 0x80400381 0x803fffff 0x73612302 0x0

506 [0x23b2000]: 0x80400381 0x803fffff 0x73611a82 0x0

507 [0x23b2000]: 0x80400381 0x803fffff 0x73611202 0x0

508 [0x23b2000]: 0x80400381 0x803fffff 0x73610982 0x0

509 [0x23b2000]: 0x80400381 0x803fffff 0x73610102 0x0

510 [0x23b2000]: 0x80400381 0x803fffff 0x7378f802 0x0

511 [0x23b2000]: 0x85e60320 0x823fffff 0x7378ef82 0x0

TX descriptor ring:
0 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

1 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

2 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

3 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

4 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

5 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

6 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

7 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

8 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

9 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

10 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

11 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

12 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

13 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

14 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

15 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

16 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

17 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

18 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

19 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

20 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

21 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

22 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

23 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

24 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

25 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

26 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

27 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

28 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

29 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

30 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

31 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

32 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

33 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

34 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

35 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

36 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

37 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

38 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

39 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

40 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

41 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

42 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

43 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

44 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

45 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

46 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

47 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

48 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

49 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

50 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

51 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

52 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

53 [0x23b5000]: 0x0 0x0 0x7390789e 0x0

54 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

55 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

56 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

57 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

58 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

59 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

60 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

61 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

62 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

63 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

64 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

65 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

66 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

67 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

68 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

69 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

70 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

71 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

72 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

73 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

74 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

75 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

76 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

77 [0x23b5000]: 0x0 0x0 0x72016202 0x0

78 [0x23b5000]: 0x0 0x0 0x72016202 0x0

79 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

80 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

81 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

82 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

83 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

84 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

85 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

86 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

87 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

88 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

89 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

90 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

91 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

92 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

93 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

94 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

95 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

96 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

97 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

98 [0x23b5000]: 0x0 0x0 0x73906c9e 0x0

99 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

100 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

101 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

102 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

103 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

104 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

105 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

106 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

107 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

108 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

109 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

110 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

111 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

112 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

113 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

114 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

115 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

116 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

117 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

118 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

119 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

120 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

121 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

122 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

123 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

124 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

125 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

126 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

127 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

128 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

129 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

130 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

131 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

132 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

133 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

134 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

135 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

136 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

137 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

138 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

139 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

140 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

141 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

142 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

143 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

144 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

145 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

146 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

147 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

148 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

149 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

150 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

151 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

152 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

153 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

154 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

155 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

156 [0x23b5000]: 0x0 0x0 0x7390709e 0x0

157 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

158 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

159 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

160 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

161 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

162 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

163 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

164 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

165 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

166 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

167 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

168 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

169 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

170 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

171 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

172 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

173 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

174 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

175 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

176 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

177 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

178 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

179 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

180 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

181 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

182 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

183 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

184 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

185 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

186 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

187 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

188 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

189 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

190 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

191 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

192 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

193 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

194 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

195 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

196 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

197 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

198 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

199 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

200 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

201 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

202 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

203 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

204 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

205 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

206 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

207 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

208 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

209 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

210 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

211 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

212 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

213 [0x23b5000]: 0x0 0x0 0x7390789e 0x0

214 [0x23b5000]: 0x0 0x0 0x7390709e 0x0

215 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

216 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

217 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

218 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

219 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

220 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

221 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

222 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

223 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

224 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

225 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

226 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

227 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

228 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

229 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

230 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

231 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

232 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

233 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

234 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

235 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

236 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

237 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

238 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

239 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

240 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

241 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

242 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

243 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

244 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

245 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

246 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

247 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

248 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

249 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

250 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

251 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

252 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

253 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

254 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

255 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

256 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

257 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

258 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

259 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

260 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

261 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

262 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

263 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

264 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

265 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

266 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

267 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

268 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

269 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

270 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

271 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

272 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

273 [0x23b5000]: 0x0 0x0 0x73906c9e 0x0

274 [0x23b5000]: 0x0 0x0 0x73905c9e 0x0

275 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

276 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

277 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

278 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

279 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

280 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

281 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

282 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

283 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

284 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

285 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

286 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

287 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

288 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

289 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

290 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

291 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

292 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

293 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

294 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

295 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

296 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

297 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

298 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

299 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

300 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

301 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

302 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

303 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

304 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

305 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

306 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

307 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

308 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

309 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

310 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

311 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

312 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

313 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

314 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

315 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

316 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

317 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

318 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

319 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

320 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

321 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

322 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

323 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

324 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

325 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

326 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

327 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

328 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

329 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

330 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

331 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

332 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

333 [0x23b5000]: 0x0 0x0 0x73907c9e 0x0

334 [0x23b5000]: 0x0 0x0 0x73905c9e 0x0

335 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

336 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

337 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

338 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

339 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

340 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

341 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

342 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

343 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

344 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

345 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

346 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

347 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

348 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

349 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

350 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

351 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

352 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

353 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

354 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

355 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

356 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

357 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

358 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

359 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

360 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

361 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

362 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

363 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

364 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

365 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

366 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

367 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

368 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

369 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

370 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

371 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

372 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

373 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

374 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

375 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

376 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

377 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

378 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

379 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

380 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

381 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

382 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

383 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

384 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

385 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

386 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

387 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

388 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

389 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

390 [0x23b5000]: 0x0 0x0 0x73906c9e 0x0

391 [0x23b5000]: 0x0 0x0 0x7390709e 0x0

392 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

393 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

394 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

395 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

396 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

397 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

398 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

399 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

400 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

401 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

402 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

403 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

404 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

405 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

406 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

407 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

408 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

409 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

410 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

411 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

412 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

413 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

414 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

415 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

416 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

417 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

418 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

419 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

420 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

421 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

422 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

423 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

424 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

425 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

426 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

427 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

428 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

429 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

430 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

431 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

432 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

433 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

434 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

435 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

436 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

437 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

438 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

439 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

440 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

441 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

442 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

443 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

444 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

445 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

446 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

447 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

448 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

449 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

450 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

451 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

452 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

453 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

454 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

455 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

456 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

457 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

458 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

459 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

460 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

461 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

462 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

463 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

464 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

465 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

466 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

467 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

468 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

469 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

470 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

471 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

472 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

473 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

474 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

475 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

476 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

477 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

478 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

479 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

480 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

481 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

482 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

483 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

484 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

485 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

486 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

487 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

488 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

489 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

490 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

491 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

492 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

493 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

494 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

495 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

496 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

497 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

498 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

499 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

500 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

501 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

502 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

503 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

504 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

505 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

506 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

507 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

508 [0x23b5000]: 0x0 0x0 0x7390789e 0x0

509 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

510 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

511 [0x23b5000]: 0x0 0x2000000 0x721acc9e 0x0

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-11 20:39 ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-09-11 20:39 UTC (permalink / raw)
  To: linus-amlogic

Hello,

I have a device with a Meson GXBB SoC with an stmmac IP block.
Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
Similar issues were reported in #linux-amlogic by a user with an
Odroid C2 board (= similar hardware).

The symptoms are:
Receiving data is plenty fast (I can max out my internet connection
easily, and with iperf3 I get ~900Mbit/s).
Transmitting data from the device is unfortunately very slow, traffic
sometimes even stalls completely.

I have attached the iperf results and the output of
/sys/kernel/debug/stmmaceth/eth0/descriptors_status.
Below you can find the ifconfig, netstat and stmmac dma_cap info
(*after* I ran all tests).

The "involved parties" are:
- Meson GXBB specific network configuration registers (I have have
double-checked them with the reference drivers: everything seems fine
here)
- stmmac: it seems that nobody else has reported these kind of issues
so far, however I'd still like to hear where I should enable some
debugging bits to rule out any stmmac bug
- RTL8211F PHY driver: unfortunately there are no public datasheets
available so this is hard to debug. but I'm guessing that TX delay
could cause similar issues, so this may be the cause as well.


Thanks for any input in advance!
Regards,
Martin


[root at alarm ~]# ifconfig eth0
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.1.235  netmask 255.255.255.0  broadcast 192.168.1.255
        ether e2:aa:53:fc:f5:c5  txqueuelen 1000  (Ethernet)
        RX packets 1967602  bytes 2968750265 (2.7 GiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 101875  bytes 8548285 (8.1 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 18

[root at alarm ~]# netstat -i
Kernel Interface table
Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0      1500  1967801      0      0 0        101934      0      0      0 BMRU

[root at alarm ~]# cat /sys/kernel/debug/stmmaceth/eth0/dma_cap
==============================
        DMA HW features
==============================
        10/100 Mbps Y
        1000 Mbps Y
        Half duple Y
        Hash Filter: Y
        Multiple MAC address registers: Y
        PCS (TBI/SGMII/RTBI PHY interfatces): N
        SMA (MDIO) Interface: Y
        PMT Remote wake up: Y
        PMT Magic Frame: Y
        RMON module: Y
        IEEE 1588-2002 Time Stamp: N
        IEEE 1588-2008 Advanced Time Stamp:N
        802.3az - Energy-Efficient Ethernet (EEE) Y
        AV features: N
        Checksum Offload in TX: Y
        IP Checksum Offload (type1) in RX: N
        IP Checksum Offload (type2) in RX: Y
        RXFIFO > 2048bytes: Y
        Number of Additional RX channel: 0
        Number of Additional TX channel: 0
        Enhanced descriptors: N
-------------- next part --------------
UDP test (iperf3):
[root at alarm ~]# iperf3 --client 192.168.1.100 -u
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.235 port 38931 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  16  
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  16  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.25 MBytes  1.05 Mbits/sec  12526562.925 ms  113/159 (71%)  
[  4] Sent 159 datagrams

iperf Done.
[root at alarm ~]# iperf3 --client 192.168.1.100 -u -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.235 port 45128 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-1.00   sec   136 KBytes  1.11 Mbits/sec  81407591.898 ms  0/17 (0%)  
[  4]   1.00-2.00   sec   128 KBytes  1.05 Mbits/sec  28987137.507 ms  0/16 (0%)  
[  4]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec  10321569.793 ms  0/16 (0%)  
[  4]   3.00-4.00   sec   128 KBytes  1.05 Mbits/sec  3675244.000 ms  0/16 (0%)  
[  4]   4.00-5.00   sec   128 KBytes  1.05 Mbits/sec  1308659.322 ms  0/16 (0%)  
[  4]   5.00-6.00   sec   128 KBytes  1.05 Mbits/sec  465979.740 ms  0/16 (0%)  
[  4]   6.00-7.00   sec   128 KBytes  1.05 Mbits/sec  165923.341 ms  0/16 (0%)  
[  4]   7.00-8.00   sec   128 KBytes  1.05 Mbits/sec  59081.019 ms  0/16 (0%)  
[  4]   8.00-9.00   sec   128 KBytes  1.05 Mbits/sec  21037.233 ms  0/16 (0%)  
[  4]   9.00-10.00  sec   128 KBytes  1.05 Mbits/sec  7490.825 ms  0/16 (0%)  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-10.00  sec  1.26 MBytes  1.06 Mbits/sec  7490.825 ms  0/161 (0%)  
[  4] Sent 161 datagrams

iperf Done.
[root at alarm ~]# 


TCP test (iperf3):
[root at alarm ~]# iperf3 --client 192.168.1.100      
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.235 port 47508 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   115 KBytes   937 Kbits/sec    8   1.41 KBytes       
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   115 KBytes  93.8 Kbits/sec   11             sender
[  4]   0.00-10.00  sec  7.07 KBytes  5.79 Kbits/sec                  receiver

iperf Done.
[root at alarm ~]# iperf3 --client 192.168.1.100 -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.235 port 47512 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  84.9 MBytes   712 Mbits/sec                  
[  4]   1.00-2.00   sec   106 MBytes   891 Mbits/sec                  
[  4]   2.00-3.00   sec   107 MBytes   894 Mbits/sec                  
[  4]   3.00-4.00   sec  93.5 MBytes   784 Mbits/sec                  
[  4]   4.00-5.00   sec  98.6 MBytes   827 Mbits/sec                  
[  4]   5.00-6.00   sec  80.5 MBytes   675 Mbits/sec                  
[  4]   6.00-7.00   sec   107 MBytes   894 Mbits/sec                  
[  4]   7.00-8.00   sec   106 MBytes   886 Mbits/sec                  
[  4]   8.00-9.00   sec   107 MBytes   894 Mbits/sec                  
[  4]   9.00-10.00  sec   106 MBytes   893 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   997 MBytes   836 Mbits/sec  1569             sender
[  4]   0.00-10.00  sec   996 MBytes   835 Mbits/sec                  receiver

iperf Done.
[root at alarm ~]#
-------------- next part --------------
RX descriptor ring:
0 [0x23b2000]: 0x85e60320 0x803fffff 0x7378e702 0x0

1 [0x23b2000]: 0x85e60320 0x803fffff 0x7378de82 0x0

2 [0x23b2000]: 0x85e60320 0x803fffff 0x7378d602 0x0

3 [0x23b2000]: 0x85e60320 0x803fffff 0x7378cd82 0x0

4 [0x23b2000]: 0x85e60320 0x803fffff 0x7378c502 0x0

5 [0x23b2000]: 0x85e60320 0x803fffff 0x7378bc82 0x0

6 [0x23b2000]: 0x85e60320 0x803fffff 0x7378b402 0x0

7 [0x23b2000]: 0x85e60320 0x803fffff 0x7378ab82 0x0

8 [0x23b2000]: 0x85e60320 0x803fffff 0x7378a302 0x0

9 [0x23b2000]: 0x85e60320 0x803fffff 0x73789a82 0x0

10 [0x23b2000]: 0x85e60320 0x803fffff 0x73789202 0x0

11 [0x23b2000]: 0x85e60320 0x803fffff 0x73788982 0x0

12 [0x23b2000]: 0x85e60320 0x803fffff 0x73788102 0x0

13 [0x23b2000]: 0x85e60320 0x803fffff 0x721df802 0x0

14 [0x23b2000]: 0x85e60320 0x803fffff 0x721def82 0x0

15 [0x23b2000]: 0x85e60320 0x803fffff 0x721de702 0x0

16 [0x23b2000]: 0x85e60320 0x803fffff 0x721dde82 0x0

17 [0x23b2000]: 0x85e60320 0x803fffff 0x721dd602 0x0

18 [0x23b2000]: 0x85e60320 0x803fffff 0x721dcd82 0x0

19 [0x23b2000]: 0x85e60320 0x803fffff 0x721dc502 0x0

20 [0x23b2000]: 0x85e60320 0x803fffff 0x721dbc82 0x0

21 [0x23b2000]: 0x85e60320 0x803fffff 0x721db402 0x0

22 [0x23b2000]: 0x85e60320 0x803fffff 0x721dab82 0x0

23 [0x23b2000]: 0x85e60320 0x803fffff 0x721da302 0x0

24 [0x23b2000]: 0x85e60320 0x803fffff 0x721d9a82 0x0

25 [0x23b2000]: 0x85e60320 0x803fffff 0x721d9202 0x0

26 [0x23b2000]: 0x85e60320 0x803fffff 0x721d8982 0x0

27 [0x23b2000]: 0x85e60320 0x803fffff 0x721d8102 0x0

28 [0x23b2000]: 0x85e60320 0x803fffff 0x735b7802 0x0

29 [0x23b2000]: 0x85e60320 0x803fffff 0x735b6f82 0x0

30 [0x23b2000]: 0x85e60320 0x803fffff 0x735b6702 0x0

31 [0x23b2000]: 0x85e60320 0x803fffff 0x735b5e82 0x0

32 [0x23b2000]: 0x85e60320 0x803fffff 0x735b5602 0x0

33 [0x23b2000]: 0x85e60320 0x803fffff 0x735b4d82 0x0

34 [0x23b2000]: 0x85e60320 0x803fffff 0x735b4502 0x0

35 [0x23b2000]: 0x85e60320 0x803fffff 0x735b3c82 0x0

36 [0x23b2000]: 0x85e60320 0x803fffff 0x735b3402 0x0

37 [0x23b2000]: 0x85e60320 0x803fffff 0x735b2b82 0x0

38 [0x23b2000]: 0x85e60320 0x803fffff 0x735b2302 0x0

39 [0x23b2000]: 0x85e60320 0x803fffff 0x735b1a82 0x0

40 [0x23b2000]: 0x85e60320 0x803fffff 0x735b1202 0x0

41 [0x23b2000]: 0x85e60320 0x803fffff 0x735b0982 0x0

42 [0x23b2000]: 0x85e60320 0x803fffff 0x735b0102 0x0

43 [0x23b2000]: 0x85e60320 0x803fffff 0x737e7802 0x0

44 [0x23b2000]: 0x85e60320 0x803fffff 0x737e6f82 0x0

45 [0x23b2000]: 0x85e60320 0x803fffff 0x737e6702 0x0

46 [0x23b2000]: 0x85e60320 0x803fffff 0x737e5e82 0x0

47 [0x23b2000]: 0x85e60320 0x803fffff 0x737e5602 0x0

48 [0x23b2000]: 0x85e60320 0x803fffff 0x737e4d82 0x0

49 [0x23b2000]: 0x85e60320 0x803fffff 0x737e4502 0x0

50 [0x23b2000]: 0x85e60320 0x803fffff 0x737e3c82 0x0

51 [0x23b2000]: 0x85e60320 0x803fffff 0x737e3402 0x0

52 [0x23b2000]: 0x85e60320 0x803fffff 0x737e2b82 0x0

53 [0x23b2000]: 0x85e60320 0x803fffff 0x737e2302 0x0

54 [0x23b2000]: 0x85e60320 0x803fffff 0x737e1a82 0x0

55 [0x23b2000]: 0x85e60320 0x803fffff 0x737e1202 0x0

56 [0x23b2000]: 0x85e60320 0x803fffff 0x737e0982 0x0

57 [0x23b2000]: 0x85e60320 0x803fffff 0x737e0102 0x0

58 [0x23b2000]: 0x85e60320 0x803fffff 0x7086f802 0x0

59 [0x23b2000]: 0x85e60320 0x803fffff 0x7086ef82 0x0

60 [0x23b2000]: 0x85e60320 0x803fffff 0x7086e702 0x0

61 [0x23b2000]: 0x85e60320 0x803fffff 0x7086de82 0x0

62 [0x23b2000]: 0x85e60320 0x803fffff 0x7086d602 0x0

63 [0x23b2000]: 0x85e60320 0x803fffff 0x7086cd82 0x0

64 [0x23b2000]: 0x85e60320 0x803fffff 0x7086c502 0x0

65 [0x23b2000]: 0x85e60320 0x803fffff 0x7086bc82 0x0

66 [0x23b2000]: 0x85e60320 0x803fffff 0x7086b402 0x0

67 [0x23b2000]: 0x85e60320 0x803fffff 0x7086ab82 0x0

68 [0x23b2000]: 0x85e60320 0x803fffff 0x7086a302 0x0

69 [0x23b2000]: 0x85e60320 0x803fffff 0x70869a82 0x0

70 [0x23b2000]: 0x85e60320 0x803fffff 0x70869202 0x0

71 [0x23b2000]: 0x85e60320 0x803fffff 0x70868982 0x0

72 [0x23b2000]: 0x85e60320 0x803fffff 0x70868102 0x0

73 [0x23b2000]: 0x85e60320 0x803fffff 0x7200f802 0x0

74 [0x23b2000]: 0x85e60320 0x803fffff 0x7200ef82 0x0

75 [0x23b2000]: 0x85e60320 0x803fffff 0x7200e702 0x0

76 [0x23b2000]: 0x85e60320 0x803fffff 0x7200de82 0x0

77 [0x23b2000]: 0x85e60320 0x803fffff 0x7200d602 0x0

78 [0x23b2000]: 0x85e60320 0x803fffff 0x7200cd82 0x0

79 [0x23b2000]: 0x85e60320 0x803fffff 0x7200c502 0x0

80 [0x23b2000]: 0x85e60320 0x803fffff 0x7200bc82 0x0

81 [0x23b2000]: 0x85e60320 0x803fffff 0x7200b402 0x0

82 [0x23b2000]: 0x85e60320 0x803fffff 0x7200ab82 0x0

83 [0x23b2000]: 0x85e60320 0x803fffff 0x7200a302 0x0

84 [0x23b2000]: 0x85e60320 0x803fffff 0x72009a82 0x0

85 [0x23b2000]: 0x85e60320 0x803fffff 0x72009202 0x0

86 [0x23b2000]: 0x85e60320 0x803fffff 0x72008982 0x0

87 [0x23b2000]: 0x85e60320 0x803fffff 0x72008102 0x0

88 [0x23b2000]: 0x85e60320 0x803fffff 0x721e7802 0x0

89 [0x23b2000]: 0x85e60320 0x803fffff 0x721e6f82 0x0

90 [0x23b2000]: 0x85e60320 0x803fffff 0x721e6702 0x0

91 [0x23b2000]: 0x85e60320 0x803fffff 0x721e5e82 0x0

92 [0x23b2000]: 0x85e60320 0x803fffff 0x721e5602 0x0

93 [0x23b2000]: 0x85e60320 0x803fffff 0x721e4d82 0x0

94 [0x23b2000]: 0x85e60320 0x803fffff 0x721e4502 0x0

95 [0x23b2000]: 0x85e60320 0x803fffff 0x721e3c82 0x0

96 [0x23b2000]: 0x85e60320 0x803fffff 0x721e3402 0x0

97 [0x23b2000]: 0x85e60320 0x803fffff 0x721e2b82 0x0

98 [0x23b2000]: 0x85e60320 0x803fffff 0x721e2302 0x0

99 [0x23b2000]: 0x85e60320 0x803fffff 0x721e1a82 0x0

100 [0x23b2000]: 0x85e60320 0x803fffff 0x721e1202 0x0

101 [0x23b2000]: 0x85e60320 0x803fffff 0x721e0982 0x0

102 [0x23b2000]: 0x85e60320 0x803fffff 0x721e0102 0x0

103 [0x23b2000]: 0x85e60320 0x803fffff 0x73687802 0x0

104 [0x23b2000]: 0x85e60320 0x803fffff 0x73686f82 0x0

105 [0x23b2000]: 0x85e60320 0x803fffff 0x73686702 0x0

106 [0x23b2000]: 0x85e60320 0x803fffff 0x73685e82 0x0

107 [0x23b2000]: 0x85e60320 0x803fffff 0x73685602 0x0

108 [0x23b2000]: 0x85e60320 0x803fffff 0x73684d82 0x0

109 [0x23b2000]: 0x85e60320 0x803fffff 0x73684502 0x0

110 [0x23b2000]: 0x85e60320 0x803fffff 0x73683c82 0x0

111 [0x23b2000]: 0x85e60320 0x803fffff 0x73683402 0x0

112 [0x23b2000]: 0x85e60320 0x803fffff 0x73682b82 0x0

113 [0x23b2000]: 0x85e60320 0x803fffff 0x73682302 0x0

114 [0x23b2000]: 0x85e60320 0x803fffff 0x73681a82 0x0

115 [0x23b2000]: 0x85e60320 0x803fffff 0x73681202 0x0

116 [0x23b2000]: 0x85e60320 0x803fffff 0x73680982 0x0

117 [0x23b2000]: 0x85e60320 0x803fffff 0x73680102 0x0

118 [0x23b2000]: 0x85e60320 0x803fffff 0x7205f802 0x0

119 [0x23b2000]: 0x85e60320 0x803fffff 0x7205ef82 0x0

120 [0x23b2000]: 0x85e60320 0x803fffff 0x7205e702 0x0

121 [0x23b2000]: 0x85e60320 0x803fffff 0x7205de82 0x0

122 [0x23b2000]: 0x85e60320 0x803fffff 0x7205d602 0x0

123 [0x23b2000]: 0x85e60320 0x803fffff 0x7205cd82 0x0

124 [0x23b2000]: 0x85e60320 0x803fffff 0x7205c502 0x0

125 [0x23b2000]: 0x85e60320 0x803fffff 0x7205bc82 0x0

126 [0x23b2000]: 0x85e60320 0x803fffff 0x7205b402 0x0

127 [0x23b2000]: 0x85e60320 0x803fffff 0x7205ab82 0x0

128 [0x23b2000]: 0x85e60320 0x803fffff 0x7205a302 0x0

129 [0x23b2000]: 0x85e60320 0x803fffff 0x72059a82 0x0

130 [0x23b2000]: 0x85e60320 0x803fffff 0x72059202 0x0

131 [0x23b2000]: 0x85e60320 0x803fffff 0x72058982 0x0

132 [0x23b2000]: 0x85e60320 0x803fffff 0x72058102 0x0

133 [0x23b2000]: 0x85e60320 0x803fffff 0x72167802 0x0

134 [0x23b2000]: 0x85e60320 0x803fffff 0x72166f82 0x0

135 [0x23b2000]: 0x85e60320 0x803fffff 0x72166702 0x0

136 [0x23b2000]: 0x85e60320 0x803fffff 0x72165e82 0x0

137 [0x23b2000]: 0x85e60320 0x803fffff 0x72165602 0x0

138 [0x23b2000]: 0x85e60320 0x803fffff 0x72164d82 0x0

139 [0x23b2000]: 0x85e60320 0x803fffff 0x72164502 0x0

140 [0x23b2000]: 0x85e60320 0x803fffff 0x72163c82 0x0

141 [0x23b2000]: 0x85e60320 0x803fffff 0x72163402 0x0

142 [0x23b2000]: 0x85e60320 0x803fffff 0x72162b82 0x0

143 [0x23b2000]: 0x85e60320 0x803fffff 0x72162302 0x0

144 [0x23b2000]: 0x85e60320 0x803fffff 0x72161a82 0x0

145 [0x23b2000]: 0x85e60320 0x803fffff 0x72161202 0x0

146 [0x23b2000]: 0x85e60320 0x803fffff 0x72160982 0x0

147 [0x23b2000]: 0x85e60320 0x803fffff 0x72160102 0x0

148 [0x23b2000]: 0x85e60320 0x803fffff 0x737ef802 0x0

149 [0x23b2000]: 0x85e60320 0x803fffff 0x737eef82 0x0

150 [0x23b2000]: 0x85e60320 0x803fffff 0x737ee702 0x0

151 [0x23b2000]: 0x85e60320 0x803fffff 0x737ede82 0x0

152 [0x23b2000]: 0x85e60320 0x803fffff 0x737ed602 0x0

153 [0x23b2000]: 0x85e60320 0x803fffff 0x737ecd82 0x0

154 [0x23b2000]: 0x85e60320 0x803fffff 0x737ec502 0x0

155 [0x23b2000]: 0x85e60320 0x803fffff 0x737ebc82 0x0

156 [0x23b2000]: 0x85e60320 0x803fffff 0x737eb402 0x0

157 [0x23b2000]: 0x85e60320 0x803fffff 0x737eab82 0x0

158 [0x23b2000]: 0x85e60320 0x803fffff 0x737ea302 0x0

159 [0x23b2000]: 0x85e60320 0x803fffff 0x737e9a82 0x0

160 [0x23b2000]: 0x85e60320 0x803fffff 0x737e9202 0x0

161 [0x23b2000]: 0x85e60320 0x803fffff 0x737e8982 0x0

162 [0x23b2000]: 0x85e60320 0x803fffff 0x737e8102 0x0

163 [0x23b2000]: 0x85e60320 0x803fffff 0x735bf802 0x0

164 [0x23b2000]: 0x85e60320 0x803fffff 0x735bef82 0x0

165 [0x23b2000]: 0x85e60320 0x803fffff 0x735be702 0x0

166 [0x23b2000]: 0x85e60320 0x803fffff 0x735bde82 0x0

167 [0x23b2000]: 0x85e60320 0x803fffff 0x735bd602 0x0

168 [0x23b2000]: 0x85e60320 0x803fffff 0x735bcd82 0x0

169 [0x23b2000]: 0x85e60320 0x803fffff 0x735bc502 0x0

170 [0x23b2000]: 0x85e60320 0x803fffff 0x735bbc82 0x0

171 [0x23b2000]: 0x85e60320 0x803fffff 0x735bb402 0x0

172 [0x23b2000]: 0x85e60320 0x803fffff 0x735bab82 0x0

173 [0x23b2000]: 0x85e60320 0x803fffff 0x735ba302 0x0

174 [0x23b2000]: 0x85e60320 0x803fffff 0x735b9a82 0x0

175 [0x23b2000]: 0x85e60320 0x803fffff 0x735b9202 0x0

176 [0x23b2000]: 0x85e60320 0x803fffff 0x735b8982 0x0

177 [0x23b2000]: 0x85e60320 0x803fffff 0x735b8102 0x0

178 [0x23b2000]: 0x85e60320 0x803fffff 0x73697802 0x0

179 [0x23b2000]: 0x85e60320 0x803fffff 0x73696f82 0x0

180 [0x23b2000]: 0x85e60320 0x803fffff 0x73696702 0x0

181 [0x23b2000]: 0x85e60320 0x803fffff 0x73695e82 0x0

182 [0x23b2000]: 0x85e60320 0x803fffff 0x73695602 0x0

183 [0x23b2000]: 0x85e60320 0x803fffff 0x73694d82 0x0

184 [0x23b2000]: 0x85e60320 0x803fffff 0x73694502 0x0

185 [0x23b2000]: 0x85e60320 0x803fffff 0x73693c82 0x0

186 [0x23b2000]: 0x85e60320 0x803fffff 0x73693402 0x0

187 [0x23b2000]: 0x85e60320 0x803fffff 0x73692b82 0x0

188 [0x23b2000]: 0x85e60320 0x803fffff 0x73692302 0x0

189 [0x23b2000]: 0x85e60320 0x803fffff 0x73691a82 0x0

190 [0x23b2000]: 0x85e60320 0x803fffff 0x73691202 0x0

191 [0x23b2000]: 0x85e60320 0x803fffff 0x73690982 0x0

192 [0x23b2000]: 0x85e60320 0x803fffff 0x73690102 0x0

193 [0x23b2000]: 0x85e60320 0x803fffff 0x7081f802 0x0

194 [0x23b2000]: 0x85e60320 0x803fffff 0x7081ef82 0x0

195 [0x23b2000]: 0x85e60320 0x803fffff 0x7081e702 0x0

196 [0x23b2000]: 0x85e60320 0x803fffff 0x7081de82 0x0

197 [0x23b2000]: 0x85e60320 0x803fffff 0x7081d602 0x0

198 [0x23b2000]: 0x85e60320 0x803fffff 0x7081cd82 0x0

199 [0x23b2000]: 0x85e60320 0x803fffff 0x7081c502 0x0

200 [0x23b2000]: 0x85e60320 0x803fffff 0x7081bc82 0x0

201 [0x23b2000]: 0x85e60320 0x803fffff 0x7081b402 0x0

202 [0x23b2000]: 0x85e60320 0x803fffff 0x7081ab82 0x0

203 [0x23b2000]: 0x85e60320 0x803fffff 0x7081a302 0x0

204 [0x23b2000]: 0x85e60320 0x803fffff 0x70819a82 0x0

205 [0x23b2000]: 0x85e60320 0x803fffff 0x70819202 0x0

206 [0x23b2000]: 0x85e60320 0x803fffff 0x70818982 0x0

207 [0x23b2000]: 0x85e60320 0x803fffff 0x70818102 0x0

208 [0x23b2000]: 0x85e60320 0x803fffff 0x73607802 0x0

209 [0x23b2000]: 0x85e60320 0x803fffff 0x73606f82 0x0

210 [0x23b2000]: 0x85e60320 0x803fffff 0x73606702 0x0

211 [0x23b2000]: 0x85e60320 0x803fffff 0x73605e82 0x0

212 [0x23b2000]: 0x85e60320 0x803fffff 0x73605602 0x0

213 [0x23b2000]: 0x85e60320 0x803fffff 0x73604d82 0x0

214 [0x23b2000]: 0x85e60320 0x803fffff 0x73604502 0x0

215 [0x23b2000]: 0x85e60320 0x803fffff 0x73603c82 0x0

216 [0x23b2000]: 0x85e60320 0x803fffff 0x73603402 0x0

217 [0x23b2000]: 0x85e60320 0x803fffff 0x73602b82 0x0

218 [0x23b2000]: 0x85e60320 0x803fffff 0x73602302 0x0

219 [0x23b2000]: 0x85e60320 0x803fffff 0x73601a82 0x0

220 [0x23b2000]: 0x85e60320 0x803fffff 0x73601202 0x0

221 [0x23b2000]: 0x85e60320 0x803fffff 0x73600982 0x0

222 [0x23b2000]: 0x85e60320 0x803fffff 0x73600102 0x0

223 [0x23b2000]: 0x85e60320 0x803fffff 0x735f7802 0x0

224 [0x23b2000]: 0x85e60320 0x803fffff 0x735f6f82 0x0

225 [0x23b2000]: 0x85e60320 0x803fffff 0x735f6702 0x0

226 [0x23b2000]: 0x85e60320 0x803fffff 0x735f5e82 0x0

227 [0x23b2000]: 0x85e60320 0x803fffff 0x735f5602 0x0

228 [0x23b2000]: 0x85e60320 0x803fffff 0x735f4d82 0x0

229 [0x23b2000]: 0x85e60320 0x803fffff 0x735f4502 0x0

230 [0x23b2000]: 0x85e60320 0x803fffff 0x735f3c82 0x0

231 [0x23b2000]: 0x85e60320 0x803fffff 0x735f3402 0x0

232 [0x23b2000]: 0x85e60320 0x803fffff 0x735f2b82 0x0

233 [0x23b2000]: 0x85e60320 0x803fffff 0x735f2302 0x0

234 [0x23b2000]: 0x85e60320 0x803fffff 0x735f1a82 0x0

235 [0x23b2000]: 0x85e60320 0x803fffff 0x735f1202 0x0

236 [0x23b2000]: 0x85e60320 0x803fffff 0x735f0982 0x0

237 [0x23b2000]: 0x85e60320 0x803fffff 0x735f0102 0x0

238 [0x23b2000]: 0x85e60320 0x803fffff 0x7369f802 0x0

239 [0x23b2000]: 0x85e60320 0x803fffff 0x7369ef82 0x0

240 [0x23b2000]: 0x85e60320 0x803fffff 0x7369e702 0x0

241 [0x23b2000]: 0x85e60320 0x803fffff 0x7369de82 0x0

242 [0x23b2000]: 0x85e60320 0x803fffff 0x7369d602 0x0

243 [0x23b2000]: 0x85e60320 0x803fffff 0x7369cd82 0x0

244 [0x23b2000]: 0x85e60320 0x803fffff 0x7369c502 0x0

245 [0x23b2000]: 0x85e60320 0x803fffff 0x7369bc82 0x0

246 [0x23b2000]: 0x85e60320 0x803fffff 0x7369b402 0x0

247 [0x23b2000]: 0x85e60320 0x803fffff 0x7369ab82 0x0

248 [0x23b2000]: 0x85e60320 0x803fffff 0x7369a302 0x0

249 [0x23b2000]: 0x85e60320 0x803fffff 0x73699a82 0x0

250 [0x23b2000]: 0x85e60320 0x803fffff 0x73699202 0x0

251 [0x23b2000]: 0x85e60320 0x803fffff 0x73698982 0x0

252 [0x23b2000]: 0x85e60320 0x803fffff 0x73698102 0x0

253 [0x23b2000]: 0x85e60320 0x803fffff 0x7360f802 0x0

254 [0x23b2000]: 0x85e60320 0x803fffff 0x7360ef82 0x0

255 [0x23b2000]: 0x85e60320 0x803fffff 0x7360e702 0x0

256 [0x23b2000]: 0x85e60320 0x803fffff 0x7360de82 0x0

257 [0x23b2000]: 0x85e60320 0x803fffff 0x7360d602 0x0

258 [0x23b2000]: 0x85e60320 0x803fffff 0x7360cd82 0x0

259 [0x23b2000]: 0x85e60320 0x803fffff 0x7360c502 0x0

260 [0x23b2000]: 0x85e60320 0x803fffff 0x7360bc82 0x0

261 [0x23b2000]: 0x85e60320 0x803fffff 0x7360b402 0x0

262 [0x23b2000]: 0x85e60320 0x803fffff 0x7360ab82 0x0

263 [0x23b2000]: 0x85e60320 0x803fffff 0x7360a302 0x0

264 [0x23b2000]: 0x85e60320 0x803fffff 0x73609a82 0x0

265 [0x23b2000]: 0x85e60320 0x803fffff 0x73609202 0x0

266 [0x23b2000]: 0x85e60320 0x803fffff 0x73608982 0x0

267 [0x23b2000]: 0x85e60320 0x803fffff 0x73608102 0x0

268 [0x23b2000]: 0x85e60320 0x803fffff 0x735ff802 0x0

269 [0x23b2000]: 0x85e60320 0x803fffff 0x735fef82 0x0

270 [0x23b2000]: 0x85e60320 0x803fffff 0x735fe702 0x0

271 [0x23b2000]: 0x85e60320 0x803fffff 0x735fde82 0x0

272 [0x23b2000]: 0x85e60320 0x803fffff 0x735fd602 0x0

273 [0x23b2000]: 0x85e60320 0x803fffff 0x735fcd82 0x0

274 [0x23b2000]: 0x85e60320 0x803fffff 0x735fc502 0x0

275 [0x23b2000]: 0x85e60320 0x803fffff 0x735fbc82 0x0

276 [0x23b2000]: 0x85e60320 0x803fffff 0x735fb402 0x0

277 [0x23b2000]: 0x85e60320 0x803fffff 0x735fab82 0x0

278 [0x23b2000]: 0x85e60320 0x803fffff 0x735fa302 0x0

279 [0x23b2000]: 0x85e60320 0x803fffff 0x735f9a82 0x0

280 [0x23b2000]: 0x85e60320 0x803fffff 0x735f9202 0x0

281 [0x23b2000]: 0x85e60320 0x803fffff 0x735f8982 0x0

282 [0x23b2000]: 0x85e60320 0x803fffff 0x735f8102 0x0

283 [0x23b2000]: 0x85e60320 0x803fffff 0x737f7802 0x0

284 [0x23b2000]: 0x85e60320 0x803fffff 0x737f6f82 0x0

285 [0x23b2000]: 0x85e60320 0x803fffff 0x737f6702 0x0

286 [0x23b2000]: 0x85e60320 0x803fffff 0x737f5e82 0x0

287 [0x23b2000]: 0x85e60320 0x803fffff 0x737f5602 0x0

288 [0x23b2000]: 0x85e60320 0x803fffff 0x737f4d82 0x0

289 [0x23b2000]: 0x85e60320 0x803fffff 0x737f4502 0x0

290 [0x23b2000]: 0x85e60320 0x803fffff 0x737f3c82 0x0

291 [0x23b2000]: 0x85e60320 0x803fffff 0x737f3402 0x0

292 [0x23b2000]: 0x85e60320 0x803fffff 0x737f2b82 0x0

293 [0x23b2000]: 0x85e60320 0x803fffff 0x737f2302 0x0

294 [0x23b2000]: 0x85e60320 0x803fffff 0x737f1a82 0x0

295 [0x23b2000]: 0x85e60320 0x803fffff 0x737f1202 0x0

296 [0x23b2000]: 0x85e60320 0x803fffff 0x737f0982 0x0

297 [0x23b2000]: 0x85e60320 0x803fffff 0x737f0102 0x0

298 [0x23b2000]: 0x85e60320 0x803fffff 0x735e7802 0x0

299 [0x23b2000]: 0x85e60320 0x803fffff 0x735e6f82 0x0

300 [0x23b2000]: 0x85e60320 0x803fffff 0x735e6702 0x0

301 [0x23b2000]: 0x85e60320 0x803fffff 0x735e5e82 0x0

302 [0x23b2000]: 0x85e60320 0x803fffff 0x735e5602 0x0

303 [0x23b2000]: 0x85e60320 0x803fffff 0x735e4d82 0x0

304 [0x23b2000]: 0x85e60320 0x803fffff 0x735e4502 0x0

305 [0x23b2000]: 0x85e60320 0x803fffff 0x735e3c82 0x0

306 [0x23b2000]: 0x85e60320 0x803fffff 0x735e3402 0x0

307 [0x23b2000]: 0x85e60320 0x803fffff 0x735e2b82 0x0

308 [0x23b2000]: 0x85e60320 0x803fffff 0x735e2302 0x0

309 [0x23b2000]: 0x85e60320 0x803fffff 0x735e1a82 0x0

310 [0x23b2000]: 0x85e60320 0x803fffff 0x735e1202 0x0

311 [0x23b2000]: 0x85e60320 0x803fffff 0x735e0982 0x0

312 [0x23b2000]: 0x85e60320 0x803fffff 0x735e0102 0x0

313 [0x23b2000]: 0x85e60320 0x803fffff 0x735ef802 0x0

314 [0x23b2000]: 0x85e60320 0x803fffff 0x735eef82 0x0

315 [0x23b2000]: 0x85e60320 0x803fffff 0x735ee702 0x0

316 [0x23b2000]: 0x85e60320 0x803fffff 0x735ede82 0x0

317 [0x23b2000]: 0x85e60320 0x803fffff 0x735ed602 0x0

318 [0x23b2000]: 0x85e60320 0x803fffff 0x735ecd82 0x0

319 [0x23b2000]: 0x85e60320 0x803fffff 0x735ec502 0x0

320 [0x23b2000]: 0x85e60320 0x803fffff 0x735ebc82 0x0

321 [0x23b2000]: 0x85e60320 0x803fffff 0x735eb402 0x0

322 [0x23b2000]: 0x85e60320 0x803fffff 0x735eab82 0x0

323 [0x23b2000]: 0x85e60320 0x803fffff 0x735ea302 0x0

324 [0x23b2000]: 0x85e60320 0x803fffff 0x735e9a82 0x0

325 [0x23b2000]: 0x85e60320 0x803fffff 0x735e9202 0x0

326 [0x23b2000]: 0x85e60320 0x803fffff 0x735e8982 0x0

327 [0x23b2000]: 0x85e60320 0x803fffff 0x735e8102 0x0

328 [0x23b2000]: 0x85e60320 0x803fffff 0x708d7802 0x0

329 [0x23b2000]: 0x85e60320 0x803fffff 0x708d6f82 0x0

330 [0x23b2000]: 0x85e60320 0x803fffff 0x708d6702 0x0

331 [0x23b2000]: 0x85e60320 0x803fffff 0x708d5e82 0x0

332 [0x23b2000]: 0x85e60320 0x803fffff 0x708d5602 0x0

333 [0x23b2000]: 0x85e60320 0x803fffff 0x708d4d82 0x0

334 [0x23b2000]: 0x85e60320 0x803fffff 0x708d4502 0x0

335 [0x23b2000]: 0x85e60320 0x803fffff 0x708d3c82 0x0

336 [0x23b2000]: 0x85e60320 0x803fffff 0x708d3402 0x0

337 [0x23b2000]: 0x85e60320 0x803fffff 0x708d2b82 0x0

338 [0x23b2000]: 0x85e60320 0x803fffff 0x708d2302 0x0

339 [0x23b2000]: 0x85e60320 0x803fffff 0x708d1a82 0x0

340 [0x23b2000]: 0x85e60320 0x803fffff 0x708d1202 0x0

341 [0x23b2000]: 0x85e60320 0x803fffff 0x708d0982 0x0

342 [0x23b2000]: 0x85e60320 0x803fffff 0x708d0102 0x0

343 [0x23b2000]: 0x85e60320 0x803fffff 0x72127802 0x0

344 [0x23b2000]: 0x85e60320 0x803fffff 0x72126f82 0x0

345 [0x23b2000]: 0x85e60320 0x803fffff 0x72126702 0x0

346 [0x23b2000]: 0x85e60320 0x803fffff 0x72125e82 0x0

347 [0x23b2000]: 0x85e60320 0x803fffff 0x72125602 0x0

348 [0x23b2000]: 0x85e60320 0x803fffff 0x72124d82 0x0

349 [0x23b2000]: 0x85e60320 0x803fffff 0x72124502 0x0

350 [0x23b2000]: 0x85e60320 0x803fffff 0x72123c82 0x0

351 [0x23b2000]: 0x85e60320 0x803fffff 0x72123402 0x0

352 [0x23b2000]: 0x85e60320 0x803fffff 0x72122b82 0x0

353 [0x23b2000]: 0x85e60320 0x803fffff 0x72122302 0x0

354 [0x23b2000]: 0x85e60320 0x803fffff 0x72121a82 0x0

355 [0x23b2000]: 0x85e60320 0x803fffff 0x72121202 0x0

356 [0x23b2000]: 0x85e60320 0x803fffff 0x72120982 0x0

357 [0x23b2000]: 0x85e60320 0x803fffff 0x72120102 0x0

358 [0x23b2000]: 0x85e60320 0x803fffff 0x720c7802 0x0

359 [0x23b2000]: 0x85e60320 0x803fffff 0x720c6f82 0x0

360 [0x23b2000]: 0x85e60320 0x803fffff 0x720c6702 0x0

361 [0x23b2000]: 0x85e60320 0x803fffff 0x720c5e82 0x0

362 [0x23b2000]: 0x85e60320 0x803fffff 0x720c5602 0x0

363 [0x23b2000]: 0x85e60320 0x803fffff 0x720c4d82 0x0

364 [0x23b2000]: 0x85e60320 0x803fffff 0x720c4502 0x0

365 [0x23b2000]: 0x85e60320 0x803fffff 0x720c3c82 0x0

366 [0x23b2000]: 0x85e60320 0x803fffff 0x720c3402 0x0

367 [0x23b2000]: 0x85e60320 0x803fffff 0x720c2b82 0x0

368 [0x23b2000]: 0x85e60320 0x803fffff 0x720c2302 0x0

369 [0x23b2000]: 0x85e60320 0x803fffff 0x720c1a82 0x0

370 [0x23b2000]: 0x85e60320 0x803fffff 0x720c1202 0x0

371 [0x23b2000]: 0x85e60320 0x803fffff 0x720c0982 0x0

372 [0x23b2000]: 0x85e60320 0x803fffff 0x720c0102 0x0

373 [0x23b2000]: 0x85e60320 0x803fffff 0x7379f802 0x0

374 [0x23b2000]: 0x85e60320 0x803fffff 0x7379ef82 0x0

375 [0x23b2000]: 0x85e60320 0x803fffff 0x7379e702 0x0

376 [0x23b2000]: 0x85e60320 0x803fffff 0x7379de82 0x0

377 [0x23b2000]: 0x85e60320 0x803fffff 0x7379d602 0x0

378 [0x23b2000]: 0x85e60320 0x803fffff 0x7379cd82 0x0

379 [0x23b2000]: 0x85e60320 0x803fffff 0x7379c502 0x0

380 [0x23b2000]: 0x85e60320 0x803fffff 0x7379bc82 0x0

381 [0x23b2000]: 0x85e60320 0x803fffff 0x7379b402 0x0

382 [0x23b2000]: 0x85e60320 0x803fffff 0x7379ab82 0x0

383 [0x23b2000]: 0x85e60320 0x803fffff 0x7379a302 0x0

384 [0x23b2000]: 0x85e60320 0x803fffff 0x73799a82 0x0

385 [0x23b2000]: 0x85e60320 0x803fffff 0x73799202 0x0

386 [0x23b2000]: 0x85e60320 0x803fffff 0x73798982 0x0

387 [0x23b2000]: 0x85e60320 0x803fffff 0x73798102 0x0

388 [0x23b2000]: 0x85e60320 0x803fffff 0x72107802 0x0

389 [0x23b2000]: 0x85e60320 0x803fffff 0x72106f82 0x0

390 [0x23b2000]: 0x85e60320 0x803fffff 0x72106702 0x0

391 [0x23b2000]: 0x85e60320 0x803fffff 0x72105e82 0x0

392 [0x23b2000]: 0x85e60320 0x803fffff 0x72105602 0x0

393 [0x23b2000]: 0x85e60320 0x803fffff 0x72104d82 0x0

394 [0x23b2000]: 0x85e60320 0x803fffff 0x72104502 0x0

395 [0x23b2000]: 0x85e60320 0x803fffff 0x72103c82 0x0

396 [0x23b2000]: 0x85e60320 0x803fffff 0x72103402 0x0

397 [0x23b2000]: 0x85e60320 0x803fffff 0x72102b82 0x0

398 [0x23b2000]: 0x85e60320 0x803fffff 0x72102302 0x0

399 [0x23b2000]: 0x85e60320 0x803fffff 0x72101a82 0x0

400 [0x23b2000]: 0x85e60320 0x803fffff 0x72101202 0x0

401 [0x23b2000]: 0x85e60320 0x803fffff 0x72100982 0x0

402 [0x23b2000]: 0x85e60320 0x803fffff 0x72100102 0x0

403 [0x23b2000]: 0x85e60320 0x803fffff 0x73557802 0x0

404 [0x23b2000]: 0x85e60320 0x803fffff 0x73556f82 0x0

405 [0x23b2000]: 0x85e60320 0x803fffff 0x73556702 0x0

406 [0x23b2000]: 0x85e60320 0x803fffff 0x73555e82 0x0

407 [0x23b2000]: 0x85e60320 0x803fffff 0x73555602 0x0

408 [0x23b2000]: 0x85e60320 0x803fffff 0x73554d82 0x0

409 [0x23b2000]: 0x85e60320 0x803fffff 0x73554502 0x0

410 [0x23b2000]: 0x85e60320 0x803fffff 0x73553c82 0x0

411 [0x23b2000]: 0x85e60320 0x803fffff 0x73553402 0x0

412 [0x23b2000]: 0x85e60320 0x803fffff 0x73552b82 0x0

413 [0x23b2000]: 0x85e60320 0x803fffff 0x73552302 0x0

414 [0x23b2000]: 0x85e60320 0x803fffff 0x73551a82 0x0

415 [0x23b2000]: 0x85e60320 0x803fffff 0x73551202 0x0

416 [0x23b2000]: 0x85e60320 0x803fffff 0x73550982 0x0

417 [0x23b2000]: 0x85e60320 0x803fffff 0x73550102 0x0

418 [0x23b2000]: 0x85e60320 0x803fffff 0x73797802 0x0

419 [0x23b2000]: 0x85e60320 0x803fffff 0x73796f82 0x0

420 [0x23b2000]: 0x85e60320 0x803fffff 0x73796702 0x0

421 [0x23b2000]: 0x85e60320 0x803fffff 0x73795e82 0x0

422 [0x23b2000]: 0x85e60320 0x803fffff 0x73795602 0x0

423 [0x23b2000]: 0x85e60320 0x803fffff 0x73794d82 0x0

424 [0x23b2000]: 0x85e60320 0x803fffff 0x73794502 0x0

425 [0x23b2000]: 0x85e60320 0x803fffff 0x73793c82 0x0

426 [0x23b2000]: 0x85e60320 0x803fffff 0x73793402 0x0

427 [0x23b2000]: 0x85e60320 0x803fffff 0x73792b82 0x0

428 [0x23b2000]: 0x85e60320 0x803fffff 0x73792302 0x0

429 [0x23b2000]: 0x85e60320 0x803fffff 0x73791a82 0x0

430 [0x23b2000]: 0x85e60320 0x803fffff 0x73791202 0x0

431 [0x23b2000]: 0x85e60320 0x803fffff 0x73790982 0x0

432 [0x23b2000]: 0x85e60320 0x803fffff 0x73790102 0x0

433 [0x23b2000]: 0x85e60320 0x803fffff 0x73517802 0x0

434 [0x23b2000]: 0x85e60320 0x803fffff 0x73516f82 0x0

435 [0x23b2000]: 0x85e60320 0x803fffff 0x73516702 0x0

436 [0x23b2000]: 0x85e60320 0x803fffff 0x73515e82 0x0

437 [0x23b2000]: 0x85e60320 0x803fffff 0x73515602 0x0

438 [0x23b2000]: 0x85e60320 0x803fffff 0x73514d82 0x0

439 [0x23b2000]: 0x85e60320 0x803fffff 0x73514502 0x0

440 [0x23b2000]: 0x85e60320 0x803fffff 0x73513c82 0x0

441 [0x23b2000]: 0x85e60320 0x803fffff 0x73513402 0x0

442 [0x23b2000]: 0x85e60320 0x803fffff 0x73512b82 0x0

443 [0x23b2000]: 0x85e60320 0x803fffff 0x73512302 0x0

444 [0x23b2000]: 0x85e60320 0x803fffff 0x73511a82 0x0

445 [0x23b2000]: 0x85e60320 0x803fffff 0x73511202 0x0

446 [0x23b2000]: 0x85e60320 0x803fffff 0x73510982 0x0

447 [0x23b2000]: 0x85e60320 0x803fffff 0x73510102 0x0

448 [0x23b2000]: 0x85e60320 0x803fffff 0x708df802 0x0

449 [0x23b2000]: 0x85e60320 0x803fffff 0x708def82 0x0

450 [0x23b2000]: 0x85e60320 0x803fffff 0x708de702 0x0

451 [0x23b2000]: 0x85e60320 0x803fffff 0x708dde82 0x0

452 [0x23b2000]: 0x85e60320 0x803fffff 0x708dd602 0x0

453 [0x23b2000]: 0x85e60320 0x803fffff 0x708dcd82 0x0

454 [0x23b2000]: 0x85e60320 0x803fffff 0x708dc502 0x0

455 [0x23b2000]: 0x85e60320 0x803fffff 0x708dbc82 0x0

456 [0x23b2000]: 0x85e60320 0x803fffff 0x708db402 0x0

457 [0x23b2000]: 0x85e60320 0x803fffff 0x708dab82 0x0

458 [0x23b2000]: 0x85e60320 0x803fffff 0x708da302 0x0

459 [0x23b2000]: 0x85e60320 0x803fffff 0x708d9a82 0x0

460 [0x23b2000]: 0x85e60320 0x803fffff 0x708d9202 0x0

461 [0x23b2000]: 0x85e60320 0x803fffff 0x708d8982 0x0

462 [0x23b2000]: 0x85e60320 0x803fffff 0x708d8102 0x0

463 [0x23b2000]: 0x85e60320 0x803fffff 0x720e7802 0x0

464 [0x23b2000]: 0x85e60320 0x803fffff 0x720e6f82 0x0

465 [0x23b2000]: 0x85e60320 0x803fffff 0x720e6702 0x0

466 [0x23b2000]: 0x85e60320 0x803fffff 0x720e5e82 0x0

467 [0x23b2000]: 0x85e60320 0x803fffff 0x720e5602 0x0

468 [0x23b2000]: 0x85e60320 0x803fffff 0x720e4d82 0x0

469 [0x23b2000]: 0x85e60320 0x803fffff 0x720e4502 0x0

470 [0x23b2000]: 0x85e60320 0x803fffff 0x720e3c82 0x0

471 [0x23b2000]: 0x85e60320 0x803fffff 0x720e3402 0x0

472 [0x23b2000]: 0x85e60320 0x803fffff 0x720e2b82 0x0

473 [0x23b2000]: 0x85e60320 0x803fffff 0x720e2302 0x0

474 [0x23b2000]: 0x85e60320 0x803fffff 0x720e1a82 0x0

475 [0x23b2000]: 0x85e60320 0x803fffff 0x720e1202 0x0

476 [0x23b2000]: 0x85e60320 0x803fffff 0x720e0982 0x0

477 [0x23b2000]: 0x85e60320 0x803fffff 0x720e0102 0x0

478 [0x23b2000]: 0x85e60320 0x803fffff 0x7375f802 0x0

479 [0x23b2000]: 0x85e60320 0x803fffff 0x7375ef82 0x0

480 [0x23b2000]: 0x85e60320 0x803fffff 0x7375e702 0x0

481 [0x23b2000]: 0x85e60320 0x803fffff 0x7375de82 0x0

482 [0x23b2000]: 0x85e60320 0x803fffff 0x7375d602 0x0

483 [0x23b2000]: 0x85e60320 0x803fffff 0x7375cd82 0x0

484 [0x23b2000]: 0x85e60320 0x803fffff 0x7375c502 0x0

485 [0x23b2000]: 0x85e60320 0x803fffff 0x7375bc82 0x0

486 [0x23b2000]: 0x85e60320 0x803fffff 0x7375b402 0x0

487 [0x23b2000]: 0x83cd0320 0x803fffff 0x7375ab82 0x0

488 [0x23b2000]: 0x80460320 0x803fffff 0x70863402 0x0

489 [0x23b2000]: 0x80400381 0x803fffff 0x70862b82 0x0

490 [0x23b2000]: 0x80400381 0x803fffff 0x70862302 0x0

491 [0x23b2000]: 0x80400381 0x803fffff 0x70861a82 0x0

492 [0x23b2000]: 0x80400381 0x803fffff 0x70861202 0x0

493 [0x23b2000]: 0x80400381 0x803fffff 0x70860982 0x0

494 [0x23b2000]: 0x80400381 0x803fffff 0x70860102 0x0

495 [0x23b2000]: 0x80400381 0x803fffff 0x73617802 0x0

496 [0x23b2000]: 0x80400381 0x803fffff 0x73616f82 0x0

497 [0x23b2000]: 0x80400381 0x803fffff 0x73616702 0x0

498 [0x23b2000]: 0x80400301 0x803fffff 0x73615e82 0x0

499 [0x23b2000]: 0x80400381 0x803fffff 0x73615602 0x0

500 [0x23b2000]: 0x80400381 0x803fffff 0x73614d82 0x0

501 [0x23b2000]: 0x80400381 0x803fffff 0x73614502 0x0

502 [0x23b2000]: 0x80400381 0x803fffff 0x73613c82 0x0

503 [0x23b2000]: 0x80400381 0x803fffff 0x73613402 0x0

504 [0x23b2000]: 0x80400381 0x803fffff 0x73612b82 0x0

505 [0x23b2000]: 0x80400381 0x803fffff 0x73612302 0x0

506 [0x23b2000]: 0x80400381 0x803fffff 0x73611a82 0x0

507 [0x23b2000]: 0x80400381 0x803fffff 0x73611202 0x0

508 [0x23b2000]: 0x80400381 0x803fffff 0x73610982 0x0

509 [0x23b2000]: 0x80400381 0x803fffff 0x73610102 0x0

510 [0x23b2000]: 0x80400381 0x803fffff 0x7378f802 0x0

511 [0x23b2000]: 0x85e60320 0x823fffff 0x7378ef82 0x0

TX descriptor ring:
0 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

1 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

2 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

3 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

4 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

5 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

6 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

7 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

8 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

9 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

10 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

11 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

12 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

13 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

14 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

15 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

16 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

17 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

18 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

19 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

20 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

21 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

22 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

23 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

24 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

25 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

26 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

27 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

28 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

29 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

30 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

31 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

32 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

33 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

34 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

35 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

36 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

37 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

38 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

39 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

40 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

41 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

42 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

43 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

44 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

45 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

46 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

47 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

48 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

49 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

50 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

51 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

52 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

53 [0x23b5000]: 0x0 0x0 0x7390789e 0x0

54 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

55 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

56 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

57 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

58 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

59 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

60 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

61 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

62 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

63 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

64 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

65 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

66 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

67 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

68 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

69 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

70 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

71 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

72 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

73 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

74 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

75 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

76 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

77 [0x23b5000]: 0x0 0x0 0x72016202 0x0

78 [0x23b5000]: 0x0 0x0 0x72016202 0x0

79 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

80 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

81 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

82 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

83 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

84 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

85 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

86 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

87 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

88 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

89 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

90 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

91 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

92 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

93 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

94 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

95 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

96 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

97 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

98 [0x23b5000]: 0x0 0x0 0x73906c9e 0x0

99 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

100 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

101 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

102 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

103 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

104 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

105 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

106 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

107 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

108 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

109 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

110 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

111 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

112 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

113 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

114 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

115 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

116 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

117 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

118 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

119 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

120 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

121 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

122 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

123 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

124 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

125 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

126 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

127 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

128 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

129 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

130 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

131 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

132 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

133 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

134 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

135 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

136 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

137 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

138 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

139 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

140 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

141 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

142 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

143 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

144 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

145 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

146 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

147 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

148 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

149 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

150 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

151 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

152 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

153 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

154 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

155 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

156 [0x23b5000]: 0x0 0x0 0x7390709e 0x0

157 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

158 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

159 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

160 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

161 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

162 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

163 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

164 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

165 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

166 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

167 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

168 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

169 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

170 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

171 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

172 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

173 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

174 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

175 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

176 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

177 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

178 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

179 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

180 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

181 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

182 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

183 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

184 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

185 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

186 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

187 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

188 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

189 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

190 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

191 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

192 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

193 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

194 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

195 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

196 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

197 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

198 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

199 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

200 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

201 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

202 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

203 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

204 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

205 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

206 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

207 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

208 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

209 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

210 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

211 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

212 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

213 [0x23b5000]: 0x0 0x0 0x7390789e 0x0

214 [0x23b5000]: 0x0 0x0 0x7390709e 0x0

215 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

216 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

217 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

218 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

219 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

220 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

221 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

222 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

223 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

224 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

225 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

226 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

227 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

228 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

229 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

230 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

231 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

232 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

233 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

234 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

235 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

236 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

237 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

238 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

239 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

240 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

241 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

242 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

243 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

244 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

245 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

246 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

247 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

248 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

249 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

250 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

251 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

252 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

253 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

254 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

255 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

256 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

257 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

258 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

259 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

260 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

261 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

262 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

263 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

264 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

265 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

266 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

267 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

268 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

269 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

270 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

271 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

272 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

273 [0x23b5000]: 0x0 0x0 0x73906c9e 0x0

274 [0x23b5000]: 0x0 0x0 0x73905c9e 0x0

275 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

276 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

277 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

278 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

279 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

280 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

281 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

282 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

283 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

284 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

285 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

286 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

287 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

288 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

289 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

290 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

291 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

292 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

293 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

294 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

295 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

296 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

297 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

298 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

299 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

300 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

301 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

302 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

303 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

304 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

305 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

306 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

307 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

308 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

309 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

310 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

311 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

312 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

313 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

314 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

315 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

316 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

317 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

318 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

319 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

320 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

321 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

322 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

323 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

324 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

325 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

326 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

327 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

328 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

329 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

330 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

331 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

332 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

333 [0x23b5000]: 0x0 0x0 0x73907c9e 0x0

334 [0x23b5000]: 0x0 0x0 0x73905c9e 0x0

335 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

336 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

337 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

338 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

339 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

340 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

341 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

342 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

343 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

344 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

345 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

346 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

347 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

348 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

349 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

350 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

351 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

352 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

353 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

354 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

355 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

356 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

357 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

358 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

359 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

360 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

361 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

362 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

363 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

364 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

365 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

366 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

367 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

368 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

369 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

370 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

371 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

372 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

373 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

374 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

375 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

376 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

377 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

378 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

379 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

380 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

381 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

382 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

383 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

384 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

385 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

386 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

387 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

388 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

389 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

390 [0x23b5000]: 0x0 0x0 0x73906c9e 0x0

391 [0x23b5000]: 0x0 0x0 0x7390709e 0x0

392 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

393 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

394 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

395 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

396 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

397 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

398 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

399 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

400 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

401 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

402 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

403 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

404 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

405 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

406 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

407 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

408 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

409 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

410 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

411 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

412 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

413 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

414 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

415 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

416 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

417 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

418 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

419 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

420 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

421 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

422 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

423 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

424 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

425 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

426 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

427 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

428 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

429 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

430 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

431 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

432 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

433 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

434 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

435 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

436 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

437 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

438 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

439 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

440 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

441 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

442 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

443 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

444 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

445 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

446 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

447 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

448 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

449 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

450 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

451 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

452 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

453 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

454 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

455 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

456 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

457 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

458 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

459 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

460 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

461 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

462 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

463 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

464 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

465 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

466 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

467 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

468 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

469 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

470 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

471 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

472 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

473 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

474 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

475 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

476 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

477 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

478 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

479 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

480 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

481 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

482 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

483 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

484 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

485 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

486 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

487 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

488 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

489 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

490 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

491 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

492 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

493 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

494 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

495 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

496 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

497 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

498 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

499 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

500 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

501 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

502 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

503 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

504 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

505 [0x23b5000]: 0x0 0x0 0x721ad49e 0x0

506 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

507 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

508 [0x23b5000]: 0x0 0x0 0x7390789e 0x0

509 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

510 [0x23b5000]: 0x0 0x0 0x721acc9e 0x0

511 [0x23b5000]: 0x0 0x2000000 0x721acc9e 0x0

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-11 20:39 ` Martin Blumenstingl
@ 2016-09-12 16:37   ` Alexandre Torgue
  -1 siblings, 0 replies; 56+ messages in thread
From: Alexandre Torgue @ 2016-09-12 16:37 UTC (permalink / raw)
  To: Martin Blumenstingl, netdev, linux-amlogic
  Cc: Giuseppe Cavallaro, Johnson Leung

Hi Martin,


On 09/11/2016 10:39 PM, Martin Blumenstingl wrote:
> Hello,
>
> I have a device with a Meson GXBB SoC with an stmmac IP block.

Which Synopsys IP version do you use ?

> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
> Similar issues were reported in #linux-amlogic by a user with an
> Odroid C2 board (= similar hardware).
>
> The symptoms are:
> Receiving data is plenty fast (I can max out my internet connection
> easily, and with iperf3 I get ~900Mbit/s).
> Transmitting data from the device is unfortunately very slow, traffic
> sometimes even stalls completely.
>
> I have attached the iperf results and the output of
> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
> Below you can find the ifconfig, netstat and stmmac dma_cap info
> (*after* I ran all tests).
>
> The "involved parties" are:
> - Meson GXBB specific network configuration registers (I have have
> double-checked them with the reference drivers: everything seems fine
> here)
> - stmmac: it seems that nobody else has reported these kind of issues
> so far, however I'd still like to hear where I should enable some
> debugging bits to rule out any stmmac bug

On my side, I just tested on the same "kind" of system:
-SYNOPSYS GMAC 3.7
-RTL8211EG as PHY

With I perf, I reach:
	-RX: 932 Mbps
	-TX: 820Mbps

Can you check ethtool -S eth0 (most precisely "MMC"counter and errors) ?
Which kernel version do you use ?

Regards

Alex




> - RTL8211F PHY driver: unfortunately there are no public datasheets
> available so this is hard to debug. but I'm guessing that TX delay
> could cause similar issues, so this may be the cause as well.
>
>
> Thanks for any input in advance!
> Regards,
> Martin
>
>
> [root@alarm ~]# ifconfig eth0
> eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 192.168.1.235  netmask 255.255.255.0  broadcast 192.168.1.255
>         ether e2:aa:53:fc:f5:c5  txqueuelen 1000  (Ethernet)
>         RX packets 1967602  bytes 2968750265 (2.7 GiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 101875  bytes 8548285 (8.1 MiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>         device interrupt 18
>
> [root@alarm ~]# netstat -i
> Kernel Interface table
> Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
> eth0      1500  1967801      0      0 0        101934      0      0      0 BMRU
>
> [root@alarm ~]# cat /sys/kernel/debug/stmmaceth/eth0/dma_cap
> ==============================
>         DMA HW features
> ==============================
>         10/100 Mbps Y
>         1000 Mbps Y
>         Half duple Y
>         Hash Filter: Y
>         Multiple MAC address registers: Y
>         PCS (TBI/SGMII/RTBI PHY interfatces): N
>         SMA (MDIO) Interface: Y
>         PMT Remote wake up: Y
>         PMT Magic Frame: Y
>         RMON module: Y
>         IEEE 1588-2002 Time Stamp: N
>         IEEE 1588-2008 Advanced Time Stamp:N
>         802.3az - Energy-Efficient Ethernet (EEE) Y
>         AV features: N
>         Checksum Offload in TX: Y
>         IP Checksum Offload (type1) in RX: N
>         IP Checksum Offload (type2) in RX: Y
>         RXFIFO > 2048bytes: Y
>         Number of Additional RX channel: 0
>         Number of Additional TX channel: 0
>         Enhanced descriptors: N
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-12 16:37   ` Alexandre Torgue
  0 siblings, 0 replies; 56+ messages in thread
From: Alexandre Torgue @ 2016-09-12 16:37 UTC (permalink / raw)
  To: linus-amlogic

Hi Martin,


On 09/11/2016 10:39 PM, Martin Blumenstingl wrote:
> Hello,
>
> I have a device with a Meson GXBB SoC with an stmmac IP block.

Which Synopsys IP version do you use ?

> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
> Similar issues were reported in #linux-amlogic by a user with an
> Odroid C2 board (= similar hardware).
>
> The symptoms are:
> Receiving data is plenty fast (I can max out my internet connection
> easily, and with iperf3 I get ~900Mbit/s).
> Transmitting data from the device is unfortunately very slow, traffic
> sometimes even stalls completely.
>
> I have attached the iperf results and the output of
> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
> Below you can find the ifconfig, netstat and stmmac dma_cap info
> (*after* I ran all tests).
>
> The "involved parties" are:
> - Meson GXBB specific network configuration registers (I have have
> double-checked them with the reference drivers: everything seems fine
> here)
> - stmmac: it seems that nobody else has reported these kind of issues
> so far, however I'd still like to hear where I should enable some
> debugging bits to rule out any stmmac bug

On my side, I just tested on the same "kind" of system:
-SYNOPSYS GMAC 3.7
-RTL8211EG as PHY

With I perf, I reach:
	-RX: 932 Mbps
	-TX: 820Mbps

Can you check ethtool -S eth0 (most precisely "MMC"counter and errors) ?
Which kernel version do you use ?

Regards

Alex




> - RTL8211F PHY driver: unfortunately there are no public datasheets
> available so this is hard to debug. but I'm guessing that TX delay
> could cause similar issues, so this may be the cause as well.
>
>
> Thanks for any input in advance!
> Regards,
> Martin
>
>
> [root at alarm ~]# ifconfig eth0
> eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 192.168.1.235  netmask 255.255.255.0  broadcast 192.168.1.255
>         ether e2:aa:53:fc:f5:c5  txqueuelen 1000  (Ethernet)
>         RX packets 1967602  bytes 2968750265 (2.7 GiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 101875  bytes 8548285 (8.1 MiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>         device interrupt 18
>
> [root at alarm ~]# netstat -i
> Kernel Interface table
> Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
> eth0      1500  1967801      0      0 0        101934      0      0      0 BMRU
>
> [root at alarm ~]# cat /sys/kernel/debug/stmmaceth/eth0/dma_cap
> ==============================
>         DMA HW features
> ==============================
>         10/100 Mbps Y
>         1000 Mbps Y
>         Half duple Y
>         Hash Filter: Y
>         Multiple MAC address registers: Y
>         PCS (TBI/SGMII/RTBI PHY interfatces): N
>         SMA (MDIO) Interface: Y
>         PMT Remote wake up: Y
>         PMT Magic Frame: Y
>         RMON module: Y
>         IEEE 1588-2002 Time Stamp: N
>         IEEE 1588-2008 Advanced Time Stamp:N
>         802.3az - Energy-Efficient Ethernet (EEE) Y
>         AV features: N
>         Checksum Offload in TX: Y
>         IP Checksum Offload (type1) in RX: N
>         IP Checksum Offload (type2) in RX: Y
>         RXFIFO > 2048bytes: Y
>         Number of Additional RX channel: 0
>         Number of Additional TX channel: 0
>         Enhanced descriptors: N
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-12 16:37   ` Alexandre Torgue
@ 2016-09-12 21:26     ` Martin Blumenstingl
  -1 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-09-12 21:26 UTC (permalink / raw)
  To: Alexandre Torgue; +Cc: netdev, linux-amlogic, Giuseppe Cavallaro, Johnson Leung

Hi Alexandre,

On Mon, Sep 12, 2016 at 6:37 PM, Alexandre Torgue
<alexandre.torgue@st.com> wrote:
> Which Synopsys IP version do you use ?
found this in a dmesg log:
[    1.504784] stmmac - user ID: 0x11, Synopsys ID: 0x37
[    1.509785]  Ring mode enabled
[    1.512796]  DMA HW capability register supported
[    1.517286]  Normal descriptors
[    1.520565]  RX Checksum Offload Engine supported
[    1.525219]  COE Type 2
[    1.527638]  TX Checksum insertion supported
[    1.531862]  Wake-Up On Lan supported
[    1.535483]  Enable RX Mitigation via HW Watchdog Timer
[    1.543851] libphy: stmmac: probed
[    1.544025] eth0: PHY ID 001cc916 at 0 IRQ POLL (stmmac-0:00) active
[    1.550321] eth0: PHY ID 001cc916 at 7 IRQ POLL (stmmac-0:07)

>> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
>> Similar issues were reported in #linux-amlogic by a user with an
>> Odroid C2 board (= similar hardware).
>>
>> The symptoms are:
>> Receiving data is plenty fast (I can max out my internet connection
>> easily, and with iperf3 I get ~900Mbit/s).
>> Transmitting data from the device is unfortunately very slow, traffic
>> sometimes even stalls completely.
>>
>> I have attached the iperf results and the output of
>> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
>> Below you can find the ifconfig, netstat and stmmac dma_cap info
>> (*after* I ran all tests).
>>
>> The "involved parties" are:
>> - Meson GXBB specific network configuration registers (I have have
>> double-checked them with the reference drivers: everything seems fine
>> here)
>> - stmmac: it seems that nobody else has reported these kind of issues
>> so far, however I'd still like to hear where I should enable some
>> debugging bits to rule out any stmmac bug
>
>
> On my side, I just tested on the same "kind" of system:
> -SYNOPSYS GMAC 3.7
> -RTL8211EG as PHY
>
> With I perf, I reach:
>         -RX: 932 Mbps
>         -TX: 820Mbps
>
> Can you check ethtool -S eth0 (most precisely "MMC"counter and errors) ?
> Which kernel version do you use ?
I am using a 4.8.0-rc4 kernel, based on Kevin's "integration" branch: [0]
Unfortunately I don't have access to my device in the next few days,
but I'll keep you updated once I have the ethtool output.


Thanks for your time
Regards,
Martin


[0] https://git.kernel.org/cgit/linux/kernel/git/khilman/linux-amlogic.git/log/?h=v4.8/integ

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-12 21:26     ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-09-12 21:26 UTC (permalink / raw)
  To: linus-amlogic

Hi Alexandre,

On Mon, Sep 12, 2016 at 6:37 PM, Alexandre Torgue
<alexandre.torgue@st.com> wrote:
> Which Synopsys IP version do you use ?
found this in a dmesg log:
[    1.504784] stmmac - user ID: 0x11, Synopsys ID: 0x37
[    1.509785]  Ring mode enabled
[    1.512796]  DMA HW capability register supported
[    1.517286]  Normal descriptors
[    1.520565]  RX Checksum Offload Engine supported
[    1.525219]  COE Type 2
[    1.527638]  TX Checksum insertion supported
[    1.531862]  Wake-Up On Lan supported
[    1.535483]  Enable RX Mitigation via HW Watchdog Timer
[    1.543851] libphy: stmmac: probed
[    1.544025] eth0: PHY ID 001cc916 at 0 IRQ POLL (stmmac-0:00) active
[    1.550321] eth0: PHY ID 001cc916 at 7 IRQ POLL (stmmac-0:07)

>> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
>> Similar issues were reported in #linux-amlogic by a user with an
>> Odroid C2 board (= similar hardware).
>>
>> The symptoms are:
>> Receiving data is plenty fast (I can max out my internet connection
>> easily, and with iperf3 I get ~900Mbit/s).
>> Transmitting data from the device is unfortunately very slow, traffic
>> sometimes even stalls completely.
>>
>> I have attached the iperf results and the output of
>> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
>> Below you can find the ifconfig, netstat and stmmac dma_cap info
>> (*after* I ran all tests).
>>
>> The "involved parties" are:
>> - Meson GXBB specific network configuration registers (I have have
>> double-checked them with the reference drivers: everything seems fine
>> here)
>> - stmmac: it seems that nobody else has reported these kind of issues
>> so far, however I'd still like to hear where I should enable some
>> debugging bits to rule out any stmmac bug
>
>
> On my side, I just tested on the same "kind" of system:
> -SYNOPSYS GMAC 3.7
> -RTL8211EG as PHY
>
> With I perf, I reach:
>         -RX: 932 Mbps
>         -TX: 820Mbps
>
> Can you check ethtool -S eth0 (most precisely "MMC"counter and errors) ?
> Which kernel version do you use ?
I am using a 4.8.0-rc4 kernel, based on Kevin's "integration" branch: [0]
Unfortunately I don't have access to my device in the next few days,
but I'll keep you updated once I have the ethtool output.


Thanks for your time
Regards,
Martin


[0] https://git.kernel.org/cgit/linux/kernel/git/khilman/linux-amlogic.git/log/?h=v4.8/integ

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
       [not found] ` <216F2694-1C1D-44DA-AC15-57ED15C24BBE@bluematt.me>
@ 2016-09-12 21:29     ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-09-12 21:29 UTC (permalink / raw)
  To: Matt Corallo; +Cc: netdev, linux-amlogic, Giuseppe Cavallaro, Alexandre Torgue

Hi Matt,

On Sun, Sep 11, 2016 at 10:57 PM, Matt Corallo <linux@bluematt.me> wrote:
> The general advice is to force it into 100M mode then it seems to work fine.
> There is some issue with the driver seemingly not fully supporting 1G (at
> least on Odroid C2) which needs to be worked out.
thanks for the hint - by "forcing 100M mode" you mean "ethtool –s eth0
speed 100 duplex full"?
Do you know more about this issue (or do you just know the
workaround)? It'd be interesting to know which "driver" does not
support Gbit speeds (as there are three drivers involved: realtek PHY
driver, stmmac and Meson stmmac glue driver


Regards,
Martin

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-12 21:29     ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-09-12 21:29 UTC (permalink / raw)
  To: linus-amlogic

Hi Matt,

On Sun, Sep 11, 2016 at 10:57 PM, Matt Corallo <linux@bluematt.me> wrote:
> The general advice is to force it into 100M mode then it seems to work fine.
> There is some issue with the driver seemingly not fully supporting 1G (at
> least on Odroid C2) which needs to be worked out.
thanks for the hint - by "forcing 100M mode" you mean "ethtool ?s eth0
speed 100 duplex full"?
Do you know more about this issue (or do you just know the
workaround)? It'd be interesting to know which "driver" does not
support Gbit speeds (as there are three drivers involved: realtek PHY
driver, stmmac and Meson stmmac glue driver


Regards,
Martin

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-11 20:39 ` Martin Blumenstingl
@ 2016-09-14 15:30   ` Giuseppe CAVALLARO
  -1 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-09-14 15:30 UTC (permalink / raw)
  To: Martin Blumenstingl, netdev, linux-amlogic
  Cc: Alexandre Torgue, Johnson Leung

Hello Martin

On 9/11/2016 10:39 PM, Martin Blumenstingl wrote:
> Hello,
>
> I have a device with a Meson GXBB SoC with an stmmac IP block.
> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
> Similar issues were reported in #linux-amlogic by a user with an
> Odroid C2 board (= similar hardware).
>
> The symptoms are:
> Receiving data is plenty fast (I can max out my internet connection
> easily, and with iperf3 I get ~900Mbit/s).
> Transmitting data from the device is unfortunately very slow, traffic
> sometimes even stalls completely.
>
> I have attached the iperf results and the output of
> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
> Below you can find the ifconfig, netstat and stmmac dma_cap info
> (*after* I ran all tests).
>
> The "involved parties" are:
> - Meson GXBB specific network configuration registers (I have have
> double-checked them with the reference drivers: everything seems fine
> here)
> - stmmac: it seems that nobody else has reported these kind of issues
> so far, however I'd still like to hear where I should enable some
> debugging bits to rule out any stmmac bug

hmm I can also think that some configuration could impact!

For example, you could try disabling the scatter-gather or tx-cum
via ethtool and seeing if there is some benefit; so we could image
some problem on your HW or SYNP MAC integration for checksumming
on tx side.

Also you could check the AXI tuning and PBL value. To be honest
(thinking about your problem) I can actually suspect some related
problem on bus setup. So I suggest you to play with these value
(better if you ask for having values from HW validation on your side).
Otherwise the stmmac uses a default that cannot be good for your
platform. For example, sometime I have seen that PBL is better if
reduced to 8 instead of 32 and w/o 4xPBL...

> - RTL8211F PHY driver: unfortunately there are no public datasheets
> available so this is hard to debug. but I'm guessing that TX delay
> could cause similar issues, so this may be the cause as well.

as rule of thumb, I can only suggest you to see the RXDLY and TXDLY
and if you have (or need!) the resistor on PCB to have the 2ns of
extra delay. This can impact on RGMII case (1G).
Indeed, if this is true, I should expect some problem also when ping.

> Thanks for any input in advance!

welcome,

as Alex asked, pls provide us the output from ethtool -S eth0

Regards
Peppe

> Regards,
> Martin
>
>
> [root@alarm ~]# ifconfig eth0
> eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 192.168.1.235  netmask 255.255.255.0  broadcast 192.168.1.255
>         ether e2:aa:53:fc:f5:c5  txqueuelen 1000  (Ethernet)
>         RX packets 1967602  bytes 2968750265 (2.7 GiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 101875  bytes 8548285 (8.1 MiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>         device interrupt 18
>
> [root@alarm ~]# netstat -i
> Kernel Interface table
> Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
> eth0      1500  1967801      0      0 0        101934      0      0      0 BMRU
>
> [root@alarm ~]# cat /sys/kernel/debug/stmmaceth/eth0/dma_cap
> ==============================
>         DMA HW features
> ==============================
>         10/100 Mbps Y
>         1000 Mbps Y
>         Half duple Y
>         Hash Filter: Y
>         Multiple MAC address registers: Y
>         PCS (TBI/SGMII/RTBI PHY interfatces): N
>         SMA (MDIO) Interface: Y
>         PMT Remote wake up: Y
>         PMT Magic Frame: Y
>         RMON module: Y
>         IEEE 1588-2002 Time Stamp: N
>         IEEE 1588-2008 Advanced Time Stamp:N
>         802.3az - Energy-Efficient Ethernet (EEE) Y
>         AV features: N
>         Checksum Offload in TX: Y
>         IP Checksum Offload (type1) in RX: N
>         IP Checksum Offload (type2) in RX: Y
>         RXFIFO > 2048bytes: Y
>         Number of Additional RX channel: 0
>         Number of Additional TX channel: 0
>         Enhanced descriptors: N
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-14 15:30   ` Giuseppe CAVALLARO
  0 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-09-14 15:30 UTC (permalink / raw)
  To: linus-amlogic

Hello Martin

On 9/11/2016 10:39 PM, Martin Blumenstingl wrote:
> Hello,
>
> I have a device with a Meson GXBB SoC with an stmmac IP block.
> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII PHY.
> Similar issues were reported in #linux-amlogic by a user with an
> Odroid C2 board (= similar hardware).
>
> The symptoms are:
> Receiving data is plenty fast (I can max out my internet connection
> easily, and with iperf3 I get ~900Mbit/s).
> Transmitting data from the device is unfortunately very slow, traffic
> sometimes even stalls completely.
>
> I have attached the iperf results and the output of
> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
> Below you can find the ifconfig, netstat and stmmac dma_cap info
> (*after* I ran all tests).
>
> The "involved parties" are:
> - Meson GXBB specific network configuration registers (I have have
> double-checked them with the reference drivers: everything seems fine
> here)
> - stmmac: it seems that nobody else has reported these kind of issues
> so far, however I'd still like to hear where I should enable some
> debugging bits to rule out any stmmac bug

hmm I can also think that some configuration could impact!

For example, you could try disabling the scatter-gather or tx-cum
via ethtool and seeing if there is some benefit; so we could image
some problem on your HW or SYNP MAC integration for checksumming
on tx side.

Also you could check the AXI tuning and PBL value. To be honest
(thinking about your problem) I can actually suspect some related
problem on bus setup. So I suggest you to play with these value
(better if you ask for having values from HW validation on your side).
Otherwise the stmmac uses a default that cannot be good for your
platform. For example, sometime I have seen that PBL is better if
reduced to 8 instead of 32 and w/o 4xPBL...

> - RTL8211F PHY driver: unfortunately there are no public datasheets
> available so this is hard to debug. but I'm guessing that TX delay
> could cause similar issues, so this may be the cause as well.

as rule of thumb, I can only suggest you to see the RXDLY and TXDLY
and if you have (or need!) the resistor on PCB to have the 2ns of
extra delay. This can impact on RGMII case (1G).
Indeed, if this is true, I should expect some problem also when ping.

> Thanks for any input in advance!

welcome,

as Alex asked, pls provide us the output from ethtool -S eth0

Regards
Peppe

> Regards,
> Martin
>
>
> [root at alarm ~]# ifconfig eth0
> eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         inet 192.168.1.235  netmask 255.255.255.0  broadcast 192.168.1.255
>         ether e2:aa:53:fc:f5:c5  txqueuelen 1000  (Ethernet)
>         RX packets 1967602  bytes 2968750265 (2.7 GiB)
>         RX errors 0  dropped 0  overruns 0  frame 0
>         TX packets 101875  bytes 8548285 (8.1 MiB)
>         TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>         device interrupt 18
>
> [root at alarm ~]# netstat -i
> Kernel Interface table
> Iface      MTU    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
> eth0      1500  1967801      0      0 0        101934      0      0      0 BMRU
>
> [root at alarm ~]# cat /sys/kernel/debug/stmmaceth/eth0/dma_cap
> ==============================
>         DMA HW features
> ==============================
>         10/100 Mbps Y
>         1000 Mbps Y
>         Half duple Y
>         Hash Filter: Y
>         Multiple MAC address registers: Y
>         PCS (TBI/SGMII/RTBI PHY interfatces): N
>         SMA (MDIO) Interface: Y
>         PMT Remote wake up: Y
>         PMT Magic Frame: Y
>         RMON module: Y
>         IEEE 1588-2002 Time Stamp: N
>         IEEE 1588-2008 Advanced Time Stamp:N
>         802.3az - Energy-Efficient Ethernet (EEE) Y
>         AV features: N
>         Checksum Offload in TX: Y
>         IP Checksum Offload (type1) in RX: N
>         IP Checksum Offload (type2) in RX: Y
>         RXFIFO > 2048bytes: Y
>         Number of Additional RX channel: 0
>         Number of Additional TX channel: 0
>         Enhanced descriptors: N
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-12 21:26     ` Martin Blumenstingl
@ 2016-09-17 21:23       ` André Roth
  -1 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-09-17 21:23 UTC (permalink / raw)
  To: Martin Blumenstingl
  Cc: Alexandre Torgue, Johnson Leung, netdev, Giuseppe Cavallaro,
	linux-amlogic

[-- Attachment #1: Type: text/plain, Size: 3257 bytes --]


Hi all,

I have an odroid c2 board which shows this issue. No data is
transmitted or received after a moment of intense tx traffic. Copying a
1GB file per scp from the board triggers it repeatedly. 

The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.

When switching the network to 100Mb/s the copying does
not seam to trigger the issue.

I've attached the ethtool statistics before and after the problem.

Thanks for your help, 

 André



> Hi Alexandre,
> 
> On Mon, Sep 12, 2016 at 6:37 PM, Alexandre Torgue
> <alexandre.torgue@st.com> wrote:
> > Which Synopsys IP version do you use ?  
> found this in a dmesg log:
> [    1.504784] stmmac - user ID: 0x11, Synopsys ID: 0x37
> [    1.509785]  Ring mode enabled
> [    1.512796]  DMA HW capability register supported
> [    1.517286]  Normal descriptors
> [    1.520565]  RX Checksum Offload Engine supported
> [    1.525219]  COE Type 2
> [    1.527638]  TX Checksum insertion supported
> [    1.531862]  Wake-Up On Lan supported
> [    1.535483]  Enable RX Mitigation via HW Watchdog Timer
> [    1.543851] libphy: stmmac: probed
> [    1.544025] eth0: PHY ID 001cc916 at 0 IRQ POLL (stmmac-0:00)
> active [    1.550321] eth0: PHY ID 001cc916 at 7 IRQ POLL
> (stmmac-0:07)
> 
> >> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII
> >> PHY. Similar issues were reported in #linux-amlogic by a user with
> >> an Odroid C2 board (= similar hardware).
> >>
> >> The symptoms are:
> >> Receiving data is plenty fast (I can max out my internet connection
> >> easily, and with iperf3 I get ~900Mbit/s).
> >> Transmitting data from the device is unfortunately very slow,
> >> traffic sometimes even stalls completely.
> >>
> >> I have attached the iperf results and the output of
> >> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
> >> Below you can find the ifconfig, netstat and stmmac dma_cap info
> >> (*after* I ran all tests).
> >>
> >> The "involved parties" are:
> >> - Meson GXBB specific network configuration registers (I have have
> >> double-checked them with the reference drivers: everything seems
> >> fine here)
> >> - stmmac: it seems that nobody else has reported these kind of
> >> issues so far, however I'd still like to hear where I should
> >> enable some debugging bits to rule out any stmmac bug  
> >
> >
> > On my side, I just tested on the same "kind" of system:
> > -SYNOPSYS GMAC 3.7
> > -RTL8211EG as PHY
> >
> > With I perf, I reach:
> >         -RX: 932 Mbps
> >         -TX: 820Mbps
> >
> > Can you check ethtool -S eth0 (most precisely "MMC"counter and
> > errors) ? Which kernel version do you use ?  
> I am using a 4.8.0-rc4 kernel, based on Kevin's "integration" branch:
> [0] Unfortunately I don't have access to my device in the next few
> days, but I'll keep you updated once I have the ethtool output.
> 
> 
> Thanks for your time
> Regards,
> Martin
> 
> 
> [0]
> https://git.kernel.org/cgit/linux/kernel/git/khilman/linux-amlogic.git/log/?h=v4.8/integ
> 
> _______________________________________________
> linux-amlogic mailing list
> linux-amlogic@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-amlogic
> 


[-- Attachment #2: ethstats.after --]
[-- Type: application/octet-stream, Size: 5155 bytes --]

NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 31723
     mmc_rx_octetcount_gb: 2360579
     mmc_rx_octetcount_g: 2360579
     mmc_rx_broadcastframe_g: 0
     mmc_rx_multicastframe_g: 0
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 4
     mmc_rx_65_to_127_octets_gb: 31700
     mmc_rx_128_to_255_octets_gb: 11
     mmc_rx_256_to_511_octets_gb: 2
     mmc_rx_512_to_1023_octets_gb: 5
     mmc_rx_1024_to_max_octets_gb: 1
     mmc_rx_unicast_g: 31723
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 2147385342
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 31719
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 0
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 1789381
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 0
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 27
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 31692
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 3856
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 1151145
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 2931
     threshold: 1
     tx_pkt_n: 307436
     rx_pkt_n: 31706
     normal_irq_n: 24352
     rx_normal_irq_n: 15239
     napi_poll: 24346
     tx_normal_irq_n: 9161
     tx_clean: 24463
     tx_set_ic_bit: 9366
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 12723
     irq_tx_path_exit_lpi_mode_n: 12722
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0

[-- Attachment #3: ethstats.before --]
[-- Type: application/octet-stream, Size: 5093 bytes --]

NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 30
     mmc_rx_octetcount_gb: 5049
     mmc_rx_octetcount_g: 5049
     mmc_rx_broadcastframe_g: 0
     mmc_rx_multicastframe_g: 0
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 4
     mmc_rx_65_to_127_octets_gb: 13
     mmc_rx_128_to_255_octets_gb: 8
     mmc_rx_256_to_511_octets_gb: 1
     mmc_rx_512_to_1023_octets_gb: 4
     mmc_rx_1024_to_max_octets_gb: 0
     mmc_rx_unicast_g: 30
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 1073692671
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 26
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 0
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 4325
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 0
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 26
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 0
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 3805
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 0
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 0
     threshold: 1
     tx_pkt_n: 162
     rx_pkt_n: 30
     normal_irq_n: 31
     rx_normal_irq_n: 29
     napi_poll: 31
     tx_normal_irq_n: 2
     tx_clean: 138
     tx_set_ic_bit: 2
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 124
     irq_tx_path_exit_lpi_mode_n: 123
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-17 21:23       ` André Roth
  0 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-09-17 21:23 UTC (permalink / raw)
  To: linus-amlogic


Hi all,

I have an odroid c2 board which shows this issue. No data is
transmitted or received after a moment of intense tx traffic. Copying a
1GB file per scp from the board triggers it repeatedly. 

The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.

When switching the network to 100Mb/s the copying does
not seam to trigger the issue.

I've attached the ethtool statistics before and after the problem.

Thanks for your help, 

 Andr?



> Hi Alexandre,
> 
> On Mon, Sep 12, 2016 at 6:37 PM, Alexandre Torgue
> <alexandre.torgue@st.com> wrote:
> > Which Synopsys IP version do you use ?  
> found this in a dmesg log:
> [    1.504784] stmmac - user ID: 0x11, Synopsys ID: 0x37
> [    1.509785]  Ring mode enabled
> [    1.512796]  DMA HW capability register supported
> [    1.517286]  Normal descriptors
> [    1.520565]  RX Checksum Offload Engine supported
> [    1.525219]  COE Type 2
> [    1.527638]  TX Checksum insertion supported
> [    1.531862]  Wake-Up On Lan supported
> [    1.535483]  Enable RX Mitigation via HW Watchdog Timer
> [    1.543851] libphy: stmmac: probed
> [    1.544025] eth0: PHY ID 001cc916 at 0 IRQ POLL (stmmac-0:00)
> active [    1.550321] eth0: PHY ID 001cc916 at 7 IRQ POLL
> (stmmac-0:07)
> 
> >> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII
> >> PHY. Similar issues were reported in #linux-amlogic by a user with
> >> an Odroid C2 board (= similar hardware).
> >>
> >> The symptoms are:
> >> Receiving data is plenty fast (I can max out my internet connection
> >> easily, and with iperf3 I get ~900Mbit/s).
> >> Transmitting data from the device is unfortunately very slow,
> >> traffic sometimes even stalls completely.
> >>
> >> I have attached the iperf results and the output of
> >> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
> >> Below you can find the ifconfig, netstat and stmmac dma_cap info
> >> (*after* I ran all tests).
> >>
> >> The "involved parties" are:
> >> - Meson GXBB specific network configuration registers (I have have
> >> double-checked them with the reference drivers: everything seems
> >> fine here)
> >> - stmmac: it seems that nobody else has reported these kind of
> >> issues so far, however I'd still like to hear where I should
> >> enable some debugging bits to rule out any stmmac bug  
> >
> >
> > On my side, I just tested on the same "kind" of system:
> > -SYNOPSYS GMAC 3.7
> > -RTL8211EG as PHY
> >
> > With I perf, I reach:
> >         -RX: 932 Mbps
> >         -TX: 820Mbps
> >
> > Can you check ethtool -S eth0 (most precisely "MMC"counter and
> > errors) ? Which kernel version do you use ?  
> I am using a 4.8.0-rc4 kernel, based on Kevin's "integration" branch:
> [0] Unfortunately I don't have access to my device in the next few
> days, but I'll keep you updated once I have the ethtool output.
> 
> 
> Thanks for your time
> Regards,
> Martin
> 
> 
> [0]
> https://git.kernel.org/cgit/linux/kernel/git/khilman/linux-amlogic.git/log/?h=v4.8/integ
> 
> _______________________________________________
> linux-amlogic mailing list
> linux-amlogic at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-amlogic
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ethstats.after
Type: application/octet-stream
Size: 5155 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-amlogic/attachments/20160917/710c5535/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ethstats.before
Type: application/octet-stream
Size: 5093 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-amlogic/attachments/20160917/710c5535/attachment-0001.obj>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-14 15:30   ` Giuseppe CAVALLARO
@ 2016-09-18 20:42     ` André Roth
  -1 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-09-18 20:42 UTC (permalink / raw)
  To: Giuseppe CAVALLARO
  Cc: Martin Blumenstingl, netdev, linux-amlogic, Alexandre Torgue


Hello,

> For example, you could try disabling the scatter-gather or tx-cum
> via ethtool and seeing if there is some benefit; so we could image
> some problem on your HW or SYNP MAC integration for checksumming
> on tx side.

disabling the following: 
  ethtool -K eth0 sg off    
or:
  ethtool -K eth0 tx off    
does not prevent the network communication going down..

> Also you could check the AXI tuning and PBL value. To be honest
> (thinking about your problem) I can actually suspect some related
> problem on bus setup. So I suggest you to play with these value
> (better if you ask for having values from HW validation on your side).
> Otherwise the stmmac uses a default that cannot be good for your
> platform. For example, sometime I have seen that PBL is better if
> reduced to 8 instead of 32 and w/o 4xPBL...

how can I set those values ?

thanks for your time,

 andre

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-18 20:42     ` André Roth
  0 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-09-18 20:42 UTC (permalink / raw)
  To: linus-amlogic


Hello,

> For example, you could try disabling the scatter-gather or tx-cum
> via ethtool and seeing if there is some benefit; so we could image
> some problem on your HW or SYNP MAC integration for checksumming
> on tx side.

disabling the following: 
  ethtool -K eth0 sg off    
or:
  ethtool -K eth0 tx off    
does not prevent the network communication going down..

> Also you could check the AXI tuning and PBL value. To be honest
> (thinking about your problem) I can actually suspect some related
> problem on bus setup. So I suggest you to play with these value
> (better if you ask for having values from HW validation on your side).
> Otherwise the stmmac uses a default that cannot be good for your
> platform. For example, sometime I have seen that PBL is better if
> reduced to 8 instead of 32 and w/o 4xPBL...

how can I set those values ?

thanks for your time,

 andre

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-18 20:42     ` André Roth
@ 2016-09-19  5:38       ` Giuseppe CAVALLARO
  -1 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-09-19  5:38 UTC (permalink / raw)
  To: André Roth
  Cc: Martin Blumenstingl, netdev, linux-amlogic, Alexandre Torgue

Hello André

On 9/18/2016 10:42 PM, André Roth wrote:
>
> Hello,
>
>> For example, you could try disabling the scatter-gather or tx-cum
>> via ethtool and seeing if there is some benefit; so we could image
>> some problem on your HW or SYNP MAC integration for checksumming
>> on tx side.
>
> disabling the following:
>   ethtool -K eth0 sg off
> or:
>   ethtool -K eth0 tx off
> does not prevent the network communication going down..
>
>> Also you could check the AXI tuning and PBL value. To be honest
>> (thinking about your problem) I can actually suspect some related
>> problem on bus setup. So I suggest you to play with these value
>> (better if you ask for having values from HW validation on your side).
>> Otherwise the stmmac uses a default that cannot be good for your
>> platform. For example, sometime I have seen that PBL is better if
>> reduced to 8 instead of 32 and w/o 4xPBL...
>
> how can I set those values ?
>
> thanks for your time,

welcome

take a look at Documentation/devicetree/bindings/net/stmmac.txt

peppe

>
>  andre
>
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-19  5:38       ` Giuseppe CAVALLARO
  0 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-09-19  5:38 UTC (permalink / raw)
  To: linus-amlogic

Hello Andr?

On 9/18/2016 10:42 PM, Andr? Roth wrote:
>
> Hello,
>
>> For example, you could try disabling the scatter-gather or tx-cum
>> via ethtool and seeing if there is some benefit; so we could image
>> some problem on your HW or SYNP MAC integration for checksumming
>> on tx side.
>
> disabling the following:
>   ethtool -K eth0 sg off
> or:
>   ethtool -K eth0 tx off
> does not prevent the network communication going down..
>
>> Also you could check the AXI tuning and PBL value. To be honest
>> (thinking about your problem) I can actually suspect some related
>> problem on bus setup. So I suggest you to play with these value
>> (better if you ask for having values from HW validation on your side).
>> Otherwise the stmmac uses a default that cannot be good for your
>> platform. For example, sometime I have seen that PBL is better if
>> reduced to 8 instead of 32 and w/o 4xPBL...
>
> how can I set those values ?
>
> thanks for your time,

welcome

take a look at Documentation/devicetree/bindings/net/stmmac.txt

peppe

>
>  andre
>
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-17 21:23       ` André Roth
@ 2016-09-26  6:17         ` Giuseppe CAVALLARO
  -1 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-09-26  6:17 UTC (permalink / raw)
  To: André Roth, Martin Blumenstingl
  Cc: Alexandre Torgue, Johnson Leung, netdev, linux-amlogic

Hello André

On 9/17/2016 11:23 PM, André Roth wrote:
>
> Hi all,
>
> I have an odroid c2 board which shows this issue. No data is
> transmitted or received after a moment of intense tx traffic. Copying a
> 1GB file per scp from the board triggers it repeatedly.
>
> The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.
>
> When switching the network to 100Mb/s the copying does
> not seam to trigger the issue.
>
> I've attached the ethtool statistics before and after the problem.

at first glance, it enters in EEE mode often in the ethtool.after.
On some platforms we met problems and it was necessary to disable the
feature. Maybe, you can start looking at if this is true on yours.
We will see to provide a clean subset of patches to switch-on/off it.

Peppe

>
> Thanks for your help,
>
>  André
>
>
>
>> Hi Alexandre,
>>
>> On Mon, Sep 12, 2016 at 6:37 PM, Alexandre Torgue
>> <alexandre.torgue@st.com> wrote:
>>> Which Synopsys IP version do you use ?
>> found this in a dmesg log:
>> [    1.504784] stmmac - user ID: 0x11, Synopsys ID: 0x37
>> [    1.509785]  Ring mode enabled
>> [    1.512796]  DMA HW capability register supported
>> [    1.517286]  Normal descriptors
>> [    1.520565]  RX Checksum Offload Engine supported
>> [    1.525219]  COE Type 2
>> [    1.527638]  TX Checksum insertion supported
>> [    1.531862]  Wake-Up On Lan supported
>> [    1.535483]  Enable RX Mitigation via HW Watchdog Timer
>> [    1.543851] libphy: stmmac: probed
>> [    1.544025] eth0: PHY ID 001cc916 at 0 IRQ POLL (stmmac-0:00)
>> active [    1.550321] eth0: PHY ID 001cc916 at 7 IRQ POLL
>> (stmmac-0:07)
>>
>>>> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII
>>>> PHY. Similar issues were reported in #linux-amlogic by a user with
>>>> an Odroid C2 board (= similar hardware).
>>>>
>>>> The symptoms are:
>>>> Receiving data is plenty fast (I can max out my internet connection
>>>> easily, and with iperf3 I get ~900Mbit/s).
>>>> Transmitting data from the device is unfortunately very slow,
>>>> traffic sometimes even stalls completely.
>>>>
>>>> I have attached the iperf results and the output of
>>>> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
>>>> Below you can find the ifconfig, netstat and stmmac dma_cap info
>>>> (*after* I ran all tests).
>>>>
>>>> The "involved parties" are:
>>>> - Meson GXBB specific network configuration registers (I have have
>>>> double-checked them with the reference drivers: everything seems
>>>> fine here)
>>>> - stmmac: it seems that nobody else has reported these kind of
>>>> issues so far, however I'd still like to hear where I should
>>>> enable some debugging bits to rule out any stmmac bug
>>>
>>>
>>> On my side, I just tested on the same "kind" of system:
>>> -SYNOPSYS GMAC 3.7
>>> -RTL8211EG as PHY
>>>
>>> With I perf, I reach:
>>>         -RX: 932 Mbps
>>>         -TX: 820Mbps
>>>
>>> Can you check ethtool -S eth0 (most precisely "MMC"counter and
>>> errors) ? Which kernel version do you use ?
>> I am using a 4.8.0-rc4 kernel, based on Kevin's "integration" branch:
>> [0] Unfortunately I don't have access to my device in the next few
>> days, but I'll keep you updated once I have the ethtool output.
>>
>>
>> Thanks for your time
>> Regards,
>> Martin
>>
>>
>> [0]
>> https://git.kernel.org/cgit/linux/kernel/git/khilman/linux-amlogic.git/log/?h=v4.8/integ
>>
>> _______________________________________________
>> linux-amlogic mailing list
>> linux-amlogic@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-amlogic
>>
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-09-26  6:17         ` Giuseppe CAVALLARO
  0 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-09-26  6:17 UTC (permalink / raw)
  To: linus-amlogic

Hello Andr?

On 9/17/2016 11:23 PM, Andr? Roth wrote:
>
> Hi all,
>
> I have an odroid c2 board which shows this issue. No data is
> transmitted or received after a moment of intense tx traffic. Copying a
> 1GB file per scp from the board triggers it repeatedly.
>
> The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.
>
> When switching the network to 100Mb/s the copying does
> not seam to trigger the issue.
>
> I've attached the ethtool statistics before and after the problem.

at first glance, it enters in EEE mode often in the ethtool.after.
On some platforms we met problems and it was necessary to disable the
feature. Maybe, you can start looking at if this is true on yours.
We will see to provide a clean subset of patches to switch-on/off it.

Peppe

>
> Thanks for your help,
>
>  Andr?
>
>
>
>> Hi Alexandre,
>>
>> On Mon, Sep 12, 2016 at 6:37 PM, Alexandre Torgue
>> <alexandre.torgue@st.com> wrote:
>>> Which Synopsys IP version do you use ?
>> found this in a dmesg log:
>> [    1.504784] stmmac - user ID: 0x11, Synopsys ID: 0x37
>> [    1.509785]  Ring mode enabled
>> [    1.512796]  DMA HW capability register supported
>> [    1.517286]  Normal descriptors
>> [    1.520565]  RX Checksum Offload Engine supported
>> [    1.525219]  COE Type 2
>> [    1.527638]  TX Checksum insertion supported
>> [    1.531862]  Wake-Up On Lan supported
>> [    1.535483]  Enable RX Mitigation via HW Watchdog Timer
>> [    1.543851] libphy: stmmac: probed
>> [    1.544025] eth0: PHY ID 001cc916 at 0 IRQ POLL (stmmac-0:00)
>> active [    1.550321] eth0: PHY ID 001cc916 at 7 IRQ POLL
>> (stmmac-0:07)
>>
>>>> Gbit ethernet on my device is provided by a Realtek RTL8211F RGMII
>>>> PHY. Similar issues were reported in #linux-amlogic by a user with
>>>> an Odroid C2 board (= similar hardware).
>>>>
>>>> The symptoms are:
>>>> Receiving data is plenty fast (I can max out my internet connection
>>>> easily, and with iperf3 I get ~900Mbit/s).
>>>> Transmitting data from the device is unfortunately very slow,
>>>> traffic sometimes even stalls completely.
>>>>
>>>> I have attached the iperf results and the output of
>>>> /sys/kernel/debug/stmmaceth/eth0/descriptors_status.
>>>> Below you can find the ifconfig, netstat and stmmac dma_cap info
>>>> (*after* I ran all tests).
>>>>
>>>> The "involved parties" are:
>>>> - Meson GXBB specific network configuration registers (I have have
>>>> double-checked them with the reference drivers: everything seems
>>>> fine here)
>>>> - stmmac: it seems that nobody else has reported these kind of
>>>> issues so far, however I'd still like to hear where I should
>>>> enable some debugging bits to rule out any stmmac bug
>>>
>>>
>>> On my side, I just tested on the same "kind" of system:
>>> -SYNOPSYS GMAC 3.7
>>> -RTL8211EG as PHY
>>>
>>> With I perf, I reach:
>>>         -RX: 932 Mbps
>>>         -TX: 820Mbps
>>>
>>> Can you check ethtool -S eth0 (most precisely "MMC"counter and
>>> errors) ? Which kernel version do you use ?
>> I am using a 4.8.0-rc4 kernel, based on Kevin's "integration" branch:
>> [0] Unfortunately I don't have access to my device in the next few
>> days, but I'll keep you updated once I have the ethtool output.
>>
>>
>> Thanks for your time
>> Regards,
>> Martin
>>
>>
>> [0]
>> https://git.kernel.org/cgit/linux/kernel/git/khilman/linux-amlogic.git/log/?h=v4.8/integ
>>
>> _______________________________________________
>> linux-amlogic mailing list
>> linux-amlogic at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-amlogic
>>
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-26  6:17         ` Giuseppe CAVALLARO
@ 2016-10-01 15:58           ` Martin Blumenstingl
  -1 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 15:58 UTC (permalink / raw)
  To: Giuseppe CAVALLARO
  Cc: André Roth, Alexandre Torgue, Johnson Leung, netdev, linux-amlogic

[-- Attachment #1: Type: text/plain, Size: 1386 bytes --]

Hello Peppe,

On Mon, Sep 26, 2016 at 8:17 AM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> Hello André
>
> On 9/17/2016 11:23 PM, André Roth wrote:
>>
>>
>> Hi all,
>>
>> I have an odroid c2 board which shows this issue. No data is
>> transmitted or received after a moment of intense tx traffic. Copying a
>> 1GB file per scp from the board triggers it repeatedly.
>>
>> The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.
>>
>> When switching the network to 100Mb/s the copying does
>> not seam to trigger the issue.
>>
>> I've attached the ethtool statistics before and after the problem.
>
>
> at first glance, it enters in EEE mode often in the ethtool.after.
> On some platforms we met problems and it was necessary to disable the
> feature. Maybe, you can start looking at if this is true on yours.
> We will see to provide a clean subset of patches to switch-on/off it.
I did some hacking in the stmmac driver to disable the LPI stuff (see
the attachment)

Unfortunately this did not fix the problem.

I did not issue any ethtool commands not shown in the logs.
Also I did not have time to change the AXI tuning / PBL value yet - so
those are also untouched.

I will keep testing, but unfortunately my device is starting to fall
apart (I sometimes have DDR initialization issues and u-boot fails to
come up, oh dear...).

[-- Attachment #2: ethstats-with-stmmac-LPI-disabled.txt --]
[-- Type: text/plain, Size: 17966 bytes --]

# ethtool -S eth0   
NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 94
     mmc_rx_octetcount_gb: 89097
     mmc_rx_octetcount_g: 89097
     mmc_rx_broadcastframe_g: 29
     mmc_rx_multicastframe_g: 1
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 29
     mmc_rx_65_to_127_octets_gb: 4
     mmc_rx_128_to_255_octets_gb: 0
     mmc_rx_256_to_511_octets_gb: 4
     mmc_rx_512_to_1023_octets_gb: 1
     mmc_rx_1024_to_max_octets_gb: 56
     mmc_rx_unicast_g: 64
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 1073692671
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 65
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 1
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 86071
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 46
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 4
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 61
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 997
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 83774
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 0
     threshold: 1
     tx_pkt_n: 30
     rx_pkt_n: 94
     normal_irq_n: 36
     rx_normal_irq_n: 36
     napi_poll: 36
     tx_normal_irq_n: 0
     tx_clean: 46
     tx_set_ic_bit: 0
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 0
     irq_tx_path_exit_lpi_mode_n: 0
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0
# iperf3 --client 192.168.1.100             
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.248 port 36472 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   100 KBytes   822 Kbits/sec   10   1.41 KBytes       
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   100 KBytes  82.2 Kbits/sec   13             sender
[  4]   0.00-10.00  sec  2.83 KBytes  2.32 Kbits/sec                  receiver

iperf Done.
# ethtool -S eth0              
NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 120
     mmc_rx_octetcount_gb: 91176
     mmc_rx_octetcount_g: 91176
     mmc_rx_broadcastframe_g: 32
     mmc_rx_multicastframe_g: 1
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 33
     mmc_rx_65_to_127_octets_gb: 25
     mmc_rx_128_to_255_octets_gb: 0
     mmc_rx_256_to_511_octets_gb: 5
     mmc_rx_512_to_1023_octets_gb: 1
     mmc_rx_1024_to_max_octets_gb: 56
     mmc_rx_unicast_g: 87
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 2147385342
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 87
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 1
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 87498
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 46
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 4
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 83
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 997
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 84761
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 0
     threshold: 1
     tx_pkt_n: 80
     rx_pkt_n: 120
     normal_irq_n: 56
     rx_normal_irq_n: 55
     napi_poll: 56
     tx_normal_irq_n: 1
     tx_clean: 79
     tx_set_ic_bit: 1
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 0
     irq_tx_path_exit_lpi_mode_n: 0
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0
# iperf3 --client 192.168.1.100 -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.248 port 36476 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   111 MBytes   929 Mbits/sec                  
[  4]   1.00-2.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   2.00-3.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   3.00-4.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   4.00-5.00   sec   111 MBytes   935 Mbits/sec                  
[  4]   5.00-6.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   6.00-7.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   7.00-8.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   8.00-9.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   9.00-10.00  sec   111 MBytes   934 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.09 GBytes   936 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.09 GBytes   934 Mbits/sec                  receiver

iperf Done.
# ethtool -S eth0                 
NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 807320
     mmc_rx_octetcount_gb: 1225389106
     mmc_rx_octetcount_g: 1225389106
     mmc_rx_broadcastframe_g: 38
     mmc_rx_multicastframe_g: 1
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 39
     mmc_rx_65_to_127_octets_gb: 40
     mmc_rx_128_to_255_octets_gb: 0
     mmc_rx_256_to_511_octets_gb: 6
     mmc_rx_512_to_1023_octets_gb: 1
     mmc_rx_1024_to_max_octets_gb: 807234
     mmc_rx_unicast_g: 807281
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 3221078013
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 807281
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 1
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 1210855552
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 46
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 4
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 807277
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 997
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 1194708935
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 456
     threshold: 1
     tx_pkt_n: 29240
     rx_pkt_n: 807320
     normal_irq_n: 25764
     rx_normal_irq_n: 25307
     napi_poll: 25764
     tx_normal_irq_n: 457
     tx_clean: 25790
     tx_set_ic_bit: 457
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 0
     irq_tx_path_exit_lpi_mode_n: 0
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0
# 

[-- Attachment #3: stmmac-hack-disable-LPI.patch --]
[-- Type: text/x-patch, Size: 2072 bytes --]

diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c
index cbefe9e..b474a64 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_core.c
@@ -345,6 +345,7 @@ static int dwmac1000_irq_status(struct mac_device_info *hw,
 
 static void dwmac1000_set_eee_mode(struct mac_device_info *hw)
 {
+#if 0
 	void __iomem *ioaddr = hw->pcsr;
 	u32 value;
 
@@ -355,6 +356,7 @@ static void dwmac1000_set_eee_mode(struct mac_device_info *hw)
 	value = readl(ioaddr + LPI_CTRL_STATUS);
 	value |= LPI_CTRL_STATUS_LPIEN | LPI_CTRL_STATUS_LPITXA;
 	writel(value, ioaddr + LPI_CTRL_STATUS);
+#endif
 }
 
 static void dwmac1000_reset_eee_mode(struct mac_device_info *hw)
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c
index 99074695..b7d3034 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c
@@ -42,7 +42,7 @@ static void dwmac1000_dma_axi(void __iomem *ioaddr, struct stmmac_axi *axi)
 		value |= DMA_AXI_EN_LPI;
 	if (axi->axi_xit_frm)
 		value |= DMA_AXI_LPI_XIT_FRM;
-
+value &= DMA_AXI_EN_LPI;
 	value |= (axi->axi_wr_osr_lmt & DMA_AXI_WR_OSR_LMT_MASK) <<
 		 DMA_AXI_WR_OSR_LMT_SHIFT;
 
diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index 4c8c60a..f477ec0 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -312,6 +312,7 @@ bool stmmac_eee_init(struct stmmac_priv *priv)
 			spin_unlock_irqrestore(&priv->lock, flags);
 			goto out;
 		}
+#if 0
 		/* Activate the EEE and start timers */
 		spin_lock_irqsave(&priv->lock, flags);
 		if (!priv->eee_active) {
@@ -333,6 +334,7 @@ bool stmmac_eee_init(struct stmmac_priv *priv)
 		spin_unlock_irqrestore(&priv->lock, flags);
 
 		pr_debug("stmmac: Energy-Efficient Ethernet initialized\n");
+#endif
 	}
 out:
 	return ret;

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-10-01 15:58           ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 15:58 UTC (permalink / raw)
  To: linus-amlogic

Hello Peppe,

On Mon, Sep 26, 2016 at 8:17 AM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> Hello Andr?
>
> On 9/17/2016 11:23 PM, Andr? Roth wrote:
>>
>>
>> Hi all,
>>
>> I have an odroid c2 board which shows this issue. No data is
>> transmitted or received after a moment of intense tx traffic. Copying a
>> 1GB file per scp from the board triggers it repeatedly.
>>
>> The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.
>>
>> When switching the network to 100Mb/s the copying does
>> not seam to trigger the issue.
>>
>> I've attached the ethtool statistics before and after the problem.
>
>
> at first glance, it enters in EEE mode often in the ethtool.after.
> On some platforms we met problems and it was necessary to disable the
> feature. Maybe, you can start looking at if this is true on yours.
> We will see to provide a clean subset of patches to switch-on/off it.
I did some hacking in the stmmac driver to disable the LPI stuff (see
the attachment)

Unfortunately this did not fix the problem.

I did not issue any ethtool commands not shown in the logs.
Also I did not have time to change the AXI tuning / PBL value yet - so
those are also untouched.

I will keep testing, but unfortunately my device is starting to fall
apart (I sometimes have DDR initialization issues and u-boot fails to
come up, oh dear...).
-------------- next part --------------
# ethtool -S eth0   
NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 94
     mmc_rx_octetcount_gb: 89097
     mmc_rx_octetcount_g: 89097
     mmc_rx_broadcastframe_g: 29
     mmc_rx_multicastframe_g: 1
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 29
     mmc_rx_65_to_127_octets_gb: 4
     mmc_rx_128_to_255_octets_gb: 0
     mmc_rx_256_to_511_octets_gb: 4
     mmc_rx_512_to_1023_octets_gb: 1
     mmc_rx_1024_to_max_octets_gb: 56
     mmc_rx_unicast_g: 64
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 1073692671
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 65
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 1
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 86071
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 46
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 4
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 61
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 997
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 83774
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 0
     threshold: 1
     tx_pkt_n: 30
     rx_pkt_n: 94
     normal_irq_n: 36
     rx_normal_irq_n: 36
     napi_poll: 36
     tx_normal_irq_n: 0
     tx_clean: 46
     tx_set_ic_bit: 0
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 0
     irq_tx_path_exit_lpi_mode_n: 0
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0
# iperf3 --client 192.168.1.100             
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.248 port 36472 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   100 KBytes   822 Kbits/sec   10   1.41 KBytes       
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   100 KBytes  82.2 Kbits/sec   13             sender
[  4]   0.00-10.00  sec  2.83 KBytes  2.32 Kbits/sec                  receiver

iperf Done.
# ethtool -S eth0              
NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 120
     mmc_rx_octetcount_gb: 91176
     mmc_rx_octetcount_g: 91176
     mmc_rx_broadcastframe_g: 32
     mmc_rx_multicastframe_g: 1
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 33
     mmc_rx_65_to_127_octets_gb: 25
     mmc_rx_128_to_255_octets_gb: 0
     mmc_rx_256_to_511_octets_gb: 5
     mmc_rx_512_to_1023_octets_gb: 1
     mmc_rx_1024_to_max_octets_gb: 56
     mmc_rx_unicast_g: 87
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 2147385342
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 87
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 1
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 87498
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 46
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 4
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 83
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 997
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 84761
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 0
     threshold: 1
     tx_pkt_n: 80
     rx_pkt_n: 120
     normal_irq_n: 56
     rx_normal_irq_n: 55
     napi_poll: 56
     tx_normal_irq_n: 1
     tx_clean: 79
     tx_set_ic_bit: 1
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 0
     irq_tx_path_exit_lpi_mode_n: 0
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0
# iperf3 --client 192.168.1.100 -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.248 port 36476 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   111 MBytes   929 Mbits/sec                  
[  4]   1.00-2.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   2.00-3.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   3.00-4.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   4.00-5.00   sec   111 MBytes   935 Mbits/sec                  
[  4]   5.00-6.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   6.00-7.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   7.00-8.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   8.00-9.00   sec   111 MBytes   934 Mbits/sec                  
[  4]   9.00-10.00  sec   111 MBytes   934 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.09 GBytes   936 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.09 GBytes   934 Mbits/sec                  receiver

iperf Done.
# ethtool -S eth0                 
NIC statistics:
     mmc_tx_octetcount_gb: 0
     mmc_tx_framecount_gb: 0
     mmc_tx_broadcastframe_g: 0
     mmc_tx_multicastframe_g: 0
     mmc_tx_64_octets_gb: 0
     mmc_tx_65_to_127_octets_gb: 0
     mmc_tx_128_to_255_octets_gb: 0
     mmc_tx_256_to_511_octets_gb: 0
     mmc_tx_512_to_1023_octets_gb: 0
     mmc_tx_1024_to_max_octets_gb: 0
     mmc_tx_unicast_gb: 0
     mmc_tx_multicast_gb: 0
     mmc_tx_broadcast_gb: 0
     mmc_tx_underflow_error: 0
     mmc_tx_singlecol_g: 0
     mmc_tx_multicol_g: 0
     mmc_tx_deferred: 0
     mmc_tx_latecol: 0
     mmc_tx_exesscol: 0
     mmc_tx_carrier_error: 0
     mmc_tx_octetcount_g: 0
     mmc_tx_framecount_g: 0
     mmc_tx_excessdef: 0
     mmc_tx_pause_frame: 0
     mmc_tx_vlan_frame_g: 0
     mmc_rx_framecount_gb: 807320
     mmc_rx_octetcount_gb: 1225389106
     mmc_rx_octetcount_g: 1225389106
     mmc_rx_broadcastframe_g: 38
     mmc_rx_multicastframe_g: 1
     mmc_rx_crc_error: 0
     mmc_rx_align_error: 0
     mmc_rx_run_error: 0
     mmc_rx_jabber_error: 0
     mmc_rx_undersize_g: 0
     mmc_rx_oversize_g: 0
     mmc_rx_64_octets_gb: 39
     mmc_rx_65_to_127_octets_gb: 40
     mmc_rx_128_to_255_octets_gb: 0
     mmc_rx_256_to_511_octets_gb: 6
     mmc_rx_512_to_1023_octets_gb: 1
     mmc_rx_1024_to_max_octets_gb: 807234
     mmc_rx_unicast_g: 807281
     mmc_rx_length_error: 0
     mmc_rx_autofrangetype: 0
     mmc_rx_pause_frames: 0
     mmc_rx_fifo_overflow: 0
     mmc_rx_vlan_frames_gb: 0
     mmc_rx_watchdog_error: 0
     mmc_rx_ipc_intr_mask: 3221078013
     mmc_rx_ipc_intr: 0
     mmc_rx_ipv4_gd: 807281
     mmc_rx_ipv4_hderr: 0
     mmc_rx_ipv4_nopay: 1
     mmc_rx_ipv4_frag: 0
     mmc_rx_ipv4_udsbl: 0
     mmc_rx_ipv4_gd_octets: 1210855552
     mmc_rx_ipv4_hderr_octets: 0
     mmc_rx_ipv4_nopay_octets: 46
     mmc_rx_ipv4_frag_octets: 0
     mmc_rx_ipv4_udsbl_octets: 0
     mmc_rx_ipv6_gd_octets: 0
     mmc_rx_ipv6_hderr_octets: 0
     mmc_rx_ipv6_nopay_octets: 0
     mmc_rx_ipv6_gd: 0
     mmc_rx_ipv6_hderr: 0
     mmc_rx_ipv6_nopay: 0
     mmc_rx_udp_gd: 4
     mmc_rx_udp_err: 0
     mmc_rx_tcp_gd: 807277
     mmc_rx_tcp_err: 0
     mmc_rx_icmp_gd: 0
     mmc_rx_icmp_err: 0
     mmc_rx_udp_gd_octets: 997
     mmc_rx_udp_err_octets: 0
     mmc_rx_tcp_gd_octets: 1194708935
     mmc_rx_tcp_err_octets: 0
     mmc_rx_icmp_gd_octets: 0
     mmc_rx_icmp_err_octets: 0
     tx_underflow: 0
     tx_carrier: 0
     tx_losscarrier: 0
     vlan_tag: 0
     tx_deferred: 0
     tx_vlan: 0
     tx_jabber: 0
     tx_frame_flushed: 0
     tx_payload_error: 0
     tx_ip_header_error: 0
     rx_desc: 0
     sa_filter_fail: 0
     overflow_error: 0
     ipc_csum_error: 0
     rx_collision: 0
     rx_crc: 0
     dribbling_bit: 0
     rx_length: 0
     rx_mii: 0
     rx_multicast: 0
     rx_gmac_overflow: 0
     rx_watchdog: 0
     da_rx_filter_fail: 0
     sa_rx_filter_fail: 0
     rx_missed_cntr: 0
     rx_overflow_cntr: 0
     rx_vlan: 0
     tx_undeflow_irq: 0
     tx_process_stopped_irq: 0
     tx_jabber_irq: 0
     rx_overflow_irq: 0
     rx_buf_unav_irq: 0
     rx_process_stopped_irq: 0
     rx_watchdog_irq: 0
     tx_early_irq: 0
     fatal_bus_error_irq: 0
     rx_early_irq: 456
     threshold: 1
     tx_pkt_n: 29240
     rx_pkt_n: 807320
     normal_irq_n: 25764
     rx_normal_irq_n: 25307
     napi_poll: 25764
     tx_normal_irq_n: 457
     tx_clean: 25790
     tx_set_ic_bit: 457
     irq_receive_pmt_irq_n: 0
     mmc_tx_irq_n: 0
     mmc_rx_irq_n: 0
     mmc_rx_csum_offload_irq_n: 0
     irq_tx_path_in_lpi_mode_n: 0
     irq_tx_path_exit_lpi_mode_n: 0
     irq_rx_path_in_lpi_mode_n: 0
     irq_rx_path_exit_lpi_mode_n: 0
     phy_eee_wakeup_error_n: 0
     ip_hdr_err: 0
     ip_payload_err: 0
     ip_csum_bypassed: 0
     ipv4_pkt_rcvd: 0
     ipv6_pkt_rcvd: 0
     rx_msg_type_ext_no_ptp: 0
     rx_msg_type_sync: 0
     rx_msg_type_follow_up: 0
     rx_msg_type_delay_req: 0
     rx_msg_type_delay_resp: 0
     rx_msg_type_pdelay_req: 0
     rx_msg_type_pdelay_resp: 0
     rx_msg_type_pdelay_follow_up: 0
     ptp_frame_type: 0
     ptp_ver: 0
     timestamp_dropped: 0
     av_pkt_rcvd: 0
     av_tagged_pkt_rcvd: 0
     vlan_tag_priority_val: 0
     l3_filter_match: 0
     l4_filter_match: 0
     l3_l4_filter_no_match: 0
     irq_pcs_ane_n: 0
     irq_pcs_link_n: 0
     irq_rgmii_n: 1
     mtl_tx_status_fifo_full: 0
     mtl_tx_fifo_not_empty: 0
     mmtl_fifo_ctrl: 0
     mtl_tx_fifo_read_ctrl_write: 0
     mtl_tx_fifo_read_ctrl_wait: 0
     mtl_tx_fifo_read_ctrl_read: 0
     mtl_tx_fifo_read_ctrl_idle: 0
     mac_tx_in_pause: 0
     mac_tx_frame_ctrl_xfer: 0
     mac_tx_frame_ctrl_idle: 0
     mac_tx_frame_ctrl_wait: 0
     mac_tx_frame_ctrl_pause: 0
     mac_gmii_tx_proto_engine: 0
     mtl_rx_fifo_fill_level_full: 0
     mtl_rx_fifo_fill_above_thresh: 0
     mtl_rx_fifo_fill_below_thresh: 0
     mtl_rx_fifo_fill_level_empty: 0
     mtl_rx_fifo_read_ctrl_flush: 0
     mtl_rx_fifo_read_ctrl_read_data: 0
     mtl_rx_fifo_read_ctrl_status: 0
     mtl_rx_fifo_read_ctrl_idle: 0
     mtl_rx_fifo_ctrl_active: 0
     mac_rx_frame_ctrl_fifo: 0
     mac_gmii_rx_proto_engine: 0
     tx_tso_frames: 0
     tx_tso_nfrags: 0
# 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stmmac-hack-disable-LPI.patch
Type: text/x-patch
Size: 2072 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-amlogic/attachments/20161001/f7914e10/attachment.bin>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-14 15:30   ` Giuseppe CAVALLARO
@ 2016-10-01 20:15     ` Martin Blumenstingl
  -1 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 20:15 UTC (permalink / raw)
  To: Giuseppe CAVALLARO; +Cc: netdev, linux-amlogic, Alexandre Torgue, Johnson Leung

On Wed, Sep 14, 2016 at 5:30 PM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> Also you could check the AXI tuning and PBL value. To be honest
> (thinking about your problem) I can actually suspect some related
> problem on bus setup. So I suggest you to play with these value
> (better if you ask for having values from HW validation on your side).
> Otherwise the stmmac uses a default that cannot be good for your
> platform. For example, sometime I have seen that PBL is better if
> reduced to 8 instead of 32 and w/o 4xPBL...
unfortunately the original kernel/firmware shipped with my device
seems to be also affected by that TX bug, so comparing with the
"stock" values does not work for me.
Additionally I should mention that this is an end-consumer device, so
I cannot simply ask the HW guys for the correct values.

I dumped the following registers (which I thought might be related to
this issue) on the original kernel (0xc9410000 is the location of the
stmmac IP block):
for REG in 0x00000018 0x00000030 0x0000003c 0x00001000 0x00001018
0x00001028; do printf "%x 1\n" $((REG + 0xc9410000)) >
/sys/kernel/debug/aml_reg/dump; cat /sys/kernel/debug/aml_reg/dump;
done

on my device this results in:
[0xc9410018] = 0x2000000
[0xc9410030] = 0x0
[0xc941003c] = 0x0
[0xc9411000] = 0x1100802
[0xc9411018] = 0x2202006
[0xc9411028] = 0x0

maybe someone else could try the command from above on his device
(running the original Amlogic kernel).
please also state if ethernet is working properly on the original
kernel (and preferably which device/board this is).


Regards,
Martin

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-10-01 20:15     ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 20:15 UTC (permalink / raw)
  To: linus-amlogic

On Wed, Sep 14, 2016 at 5:30 PM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> Also you could check the AXI tuning and PBL value. To be honest
> (thinking about your problem) I can actually suspect some related
> problem on bus setup. So I suggest you to play with these value
> (better if you ask for having values from HW validation on your side).
> Otherwise the stmmac uses a default that cannot be good for your
> platform. For example, sometime I have seen that PBL is better if
> reduced to 8 instead of 32 and w/o 4xPBL...
unfortunately the original kernel/firmware shipped with my device
seems to be also affected by that TX bug, so comparing with the
"stock" values does not work for me.
Additionally I should mention that this is an end-consumer device, so
I cannot simply ask the HW guys for the correct values.

I dumped the following registers (which I thought might be related to
this issue) on the original kernel (0xc9410000 is the location of the
stmmac IP block):
for REG in 0x00000018 0x00000030 0x0000003c 0x00001000 0x00001018
0x00001028; do printf "%x 1\n" $((REG + 0xc9410000)) >
/sys/kernel/debug/aml_reg/dump; cat /sys/kernel/debug/aml_reg/dump;
done

on my device this results in:
[0xc9410018] = 0x2000000
[0xc9410030] = 0x0
[0xc941003c] = 0x0
[0xc9411000] = 0x1100802
[0xc9411018] = 0x2202006
[0xc9411028] = 0x0

maybe someone else could try the command from above on his device
(running the original Amlogic kernel).
please also state if ethernet is working properly on the original
kernel (and preferably which device/board this is).


Regards,
Martin

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-09-14 15:30   ` Giuseppe CAVALLARO
@ 2016-10-01 20:26     ` Martin Blumenstingl
  -1 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 20:26 UTC (permalink / raw)
  To: Giuseppe CAVALLARO; +Cc: netdev, linux-amlogic, Alexandre Torgue

On Wed, Sep 14, 2016 at 5:30 PM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> as rule of thumb, I can only suggest you to see the RXDLY and TXDLY
> and if you have (or need!) the resistor on PCB to have the 2ns of
> extra delay. This can impact on RGMII case (1G).
> Indeed, if this is true, I should expect some problem also when ping.
I did a simple ping-test:

device to computer:
50 packets transmitted, 50 received, 0% packet loss, time 50170ms
rtt min/avg/max/mdev = 0.457/0.659/0.779/0.083 ms

computer to device:
50 packets transmitted, 50 received, 0% packet loss, time 49001ms
rtt min/avg/max/mdev = 0.483/0.629/1.356/0.114 ms

both running simultaneously, device to computer:
51 packets transmitted, 50 received, 1% packet loss, time 51182ms
rtt min/avg/max/mdev = 0.459/0.648/0.729/0.082 ms

both running simultaneously, computer to device:
50 packets transmitted, 38 received, 24% packet loss, time 48997ms
rtt min/avg/max/mdev = 0.482/0.594/0.640/0.053 ms

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-10-01 20:26     ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 20:26 UTC (permalink / raw)
  To: linus-amlogic

On Wed, Sep 14, 2016 at 5:30 PM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> as rule of thumb, I can only suggest you to see the RXDLY and TXDLY
> and if you have (or need!) the resistor on PCB to have the 2ns of
> extra delay. This can impact on RGMII case (1G).
> Indeed, if this is true, I should expect some problem also when ping.
I did a simple ping-test:

device to computer:
50 packets transmitted, 50 received, 0% packet loss, time 50170ms
rtt min/avg/max/mdev = 0.457/0.659/0.779/0.083 ms

computer to device:
50 packets transmitted, 50 received, 0% packet loss, time 49001ms
rtt min/avg/max/mdev = 0.483/0.629/1.356/0.114 ms

both running simultaneously, device to computer:
51 packets transmitted, 50 received, 1% packet loss, time 51182ms
rtt min/avg/max/mdev = 0.459/0.648/0.729/0.082 ms

both running simultaneously, computer to device:
50 packets transmitted, 38 received, 24% packet loss, time 48997ms
rtt min/avg/max/mdev = 0.482/0.594/0.640/0.053 ms

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-10-01 20:15     ` Martin Blumenstingl
@ 2016-10-01 20:40       ` Martin Blumenstingl
  -1 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 20:40 UTC (permalink / raw)
  To: Giuseppe CAVALLARO; +Cc: netdev, linux-amlogic, Alexandre Torgue, Johnson Leung

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]

On Sat, Oct 1, 2016 at 10:15 PM, Martin Blumenstingl
<martin.blumenstingl@googlemail.com> wrote:
> On Wed, Sep 14, 2016 at 5:30 PM, Giuseppe CAVALLARO
> <peppe.cavallaro@st.com> wrote:
>> Also you could check the AXI tuning and PBL value. To be honest
>> (thinking about your problem) I can actually suspect some related
>> problem on bus setup. So I suggest you to play with these value
>> (better if you ask for having values from HW validation on your side).
>> Otherwise the stmmac uses a default that cannot be good for your
>> platform. For example, sometime I have seen that PBL is better if
>> reduced to 8 instead of 32 and w/o 4xPBL...
> unfortunately the original kernel/firmware shipped with my device
> seems to be also affected by that TX bug, so comparing with the
> "stock" values does not work for me.
I tried the following changes:
- snps,pbl = <4>; in .dts
- snps,pbl = <2>; in .dts
- snps,pbl = <8>; in .dts and disabled DMA_BUS_MODE_MAXPBL (= 4xPBL) in stmmac

None of these seem to have any impact on the TX problem

[-- Attachment #2: meson-stmmac-pbl.patch --]
[-- Type: text/x-patch, Size: 1012 bytes --]

diff --git a/arch/arm64/boot/dts/amlogic/meson-gxbb.dtsi b/arch/arm64/boot/dts/amlogic/meson-gxbb.dtsi
index 2e89ec4..7deab7a 100644
--- a/arch/arm64/boot/dts/amlogic/meson-gxbb.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-gxbb.dtsi
@@ -698,6 +698,7 @@
 				 <&clkc CLKID_MPLL2>;
 			clock-names = "stmmaceth", "clkin0", "clkin1";
 			phy-mode = "rgmii";
+			snps,pbl = <8>;
 			status = "disabled";
 		};
 	};
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c
index 99074695..fffb407 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c
@@ -97,7 +97,7 @@ static void dwmac1000_dma_init(void __iomem *ioaddr, int pbl, int fb, int mb,
 	 * so only the bits: 13-8 are programmed with the PBL passed from the
 	 * platform.
 	 */
-	value |= DMA_BUS_MODE_MAXPBL;
+	value &= ~DMA_BUS_MODE_MAXPBL;
 	value &= ~DMA_BUS_MODE_PBL_MASK;
 	value |= (pbl << DMA_BUS_MODE_PBL_SHIFT);
 

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-10-01 20:40       ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-10-01 20:40 UTC (permalink / raw)
  To: linus-amlogic

On Sat, Oct 1, 2016 at 10:15 PM, Martin Blumenstingl
<martin.blumenstingl@googlemail.com> wrote:
> On Wed, Sep 14, 2016 at 5:30 PM, Giuseppe CAVALLARO
> <peppe.cavallaro@st.com> wrote:
>> Also you could check the AXI tuning and PBL value. To be honest
>> (thinking about your problem) I can actually suspect some related
>> problem on bus setup. So I suggest you to play with these value
>> (better if you ask for having values from HW validation on your side).
>> Otherwise the stmmac uses a default that cannot be good for your
>> platform. For example, sometime I have seen that PBL is better if
>> reduced to 8 instead of 32 and w/o 4xPBL...
> unfortunately the original kernel/firmware shipped with my device
> seems to be also affected by that TX bug, so comparing with the
> "stock" values does not work for me.
I tried the following changes:
- snps,pbl = <4>; in .dts
- snps,pbl = <2>; in .dts
- snps,pbl = <8>; in .dts and disabled DMA_BUS_MODE_MAXPBL (= 4xPBL) in stmmac

None of these seem to have any impact on the TX problem
-------------- next part --------------
A non-text attachment was scrubbed...
Name: meson-stmmac-pbl.patch
Type: text/x-patch
Size: 1012 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-amlogic/attachments/20161001/8e44374c/attachment.bin>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-10-01 20:15     ` Martin Blumenstingl
@ 2016-10-31 10:25       ` André Roth
  -1 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-10-31 10:25 UTC (permalink / raw)
  To: Martin Blumenstingl
  Cc: Giuseppe CAVALLARO, Johnson Leung, netdev, Alexandre Torgue,
	linux-amlogic


Hi all,
 
> on my device this results in:
> [0xc9410018] = 0x2000000
> [0xc9410030] = 0x0
> [0xc941003c] = 0x0
> [0xc9411000] = 0x1100802
> [0xc9411018] = 0x2202006
> [0xc9411028] = 0x0
> 
> maybe someone else could try the command from above on his device
> (running the original Amlogic kernel).

those registers have the same value on an original image from
hardkernel: 

Linux odroid64 3.14.65-65 #1 SMP PREEMPT Sat May 28
02:50:51 BRT 2016 aarch64 aarch64 aarch64 GNU/Linux

> please also state if ethernet is working properly on the original
> kernel (and preferably which device/board this is).

yes, the ethernet works flawless in 100 and 1000 Mbit/s on the 3.14
kernel.

I can now confirm that both 100 and 1000 Mbit/s do not work properly
on the 4.8/integ branch. the network connection is interrupted after
some outbound traffic. it can be recovered by running a ifdown/ifup
which restarts dhclient, which I think is able to somehow reset the
interface. 

Anything I can help to debug the issue further ?

Regards,

 André
 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-10-31 10:25       ` André Roth
  0 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-10-31 10:25 UTC (permalink / raw)
  To: linus-amlogic


Hi all,
 
> on my device this results in:
> [0xc9410018] = 0x2000000
> [0xc9410030] = 0x0
> [0xc941003c] = 0x0
> [0xc9411000] = 0x1100802
> [0xc9411018] = 0x2202006
> [0xc9411028] = 0x0
> 
> maybe someone else could try the command from above on his device
> (running the original Amlogic kernel).

those registers have the same value on an original image from
hardkernel: 

Linux odroid64 3.14.65-65 #1 SMP PREEMPT Sat May 28
02:50:51 BRT 2016 aarch64 aarch64 aarch64 GNU/Linux

> please also state if ethernet is working properly on the original
> kernel (and preferably which device/board this is).

yes, the ethernet works flawless in 100 and 1000 Mbit/s on the 3.14
kernel.

I can now confirm that both 100 and 1000 Mbit/s do not work properly
on the 4.8/integ branch. the network connection is interrupted after
some outbound traffic. it can be recovered by running a ifdown/ifup
which restarts dhclient, which I think is able to somehow reset the
interface. 

Anything I can help to debug the issue further ?

Regards,

 Andr?
 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-10-01 15:58           ` Martin Blumenstingl
@ 2016-11-03 16:36             ` Jerome Brunet
  -1 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-03 16:36 UTC (permalink / raw)
  To: Martin Blumenstingl, Giuseppe CAVALLARO
  Cc: Johnson Leung, netdev, André Roth, Alexandre Torgue, linux-amlogic

[-- Attachment #1: Type: text/plain, Size: 5446 bytes --]

On Sat, 2016-10-01 at 17:58 +0200, Martin Blumenstingl wrote:
> Hello Peppe,
> 
> On Mon, Sep 26, 2016 at 8:17 AM, Giuseppe CAVALLARO
> <peppe.cavallaro@st.com> wrote:
> > 
> > Hello André
> > 
> > On 9/17/2016 11:23 PM, André Roth wrote:
> > > 
> > > 
> > > 
> > > Hi all,
> > > 
> > > I have an odroid c2 board which shows this issue. No data is
> > > transmitted or received after a moment of intense tx traffic.
> > > Copying a
> > > 1GB file per scp from the board triggers it repeatedly.
> > > 
> > > The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.
> > > 
> > > When switching the network to 100Mb/s the copying does
> > > not seam to trigger the issue.
> > > 
> > > I've attached the ethtool statistics before and after the
> > > problem.
> > 
> > 
> > at first glance, it enters in EEE mode often in the ethtool.after.
> > On some platforms we met problems and it was necessary to disable
> > the
> > feature. Maybe, you can start looking at if this is true on yours.
> > We will see to provide a clean subset of patches to switch-on/off
> > it.
> I did some hacking in the stmmac driver to disable the LPI stuff (see
> the attachment)
> 
> Unfortunately this did not fix the problem.
> 
> I did not issue any ethtool commands not shown in the logs.
> Also I did not have time to change the AXI tuning / PBL value yet -
> so
> those are also untouched.
> 
> I will keep testing, but unfortunately my device is starting to fall
> apart (I sometimes have DDR initialization issues and u-boot fails to
> come up, oh dear...).

Hi all,

I did several tests on this issue with amlogic's S905 SoC (Synopsys MAC
- user ID: 0x11, Synopsys ID: 0x37.) 

With the OdroidC2 (PHY Realtek RTL8211F), EEE is on by default.
Just before launching iperf3, here are the ethtool stats regarding LPI:
     irq_tx_path_in_lpi_mode_n: 6
     irq_tx_path_exit_lpi_mode_n: 5
     irq_rx_path_in_lpi_mode_n: 76
     irq_rx_path_exit_lpi_mode_n: 75
     phy_eee_wakeup_error_n: 0

Sending data with iperf usually works for little while (between 0 and
10s)

# iperf3 -c 192.168.1.170 -p12345
Connecting to host 192.168.1.170, port 12345
local 192.168.1.30 port 54450 connected to 192.168.1.170 port 12345
Interval           Transfer     Bandwidth       Retr  Cwnd
0.00-1.00   sec   112 MBytes   938 Mbits/sec    0    409 KBytes       
1.00-2.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes       
2.00-3.00   sec   112 MBytes   939 Mbits/sec    0    426 KBytes       
3.00-4.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes       
4.00-5.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes       
5.00-6.00   sec   112 MBytes   939 Mbits/sec    0    426 KBytes       
6.00-7.00   sec  9.26 MBytes  77.6 Mbits/sec    2   1.41 KBytes       
7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes       
^C10.00-13.58  sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
Interval           Transfer     Bandwidth       Retr
0.00-13.58  sec   681 MBytes   421 Mbits/sec    4             sender
0.00-13.58  sec  0.00 Bytes  0.00 bits/sec                  receiver
iperf3: interrupt - the client has terminated

iperf3 does not exit ant the link seems completely broken. We cannot
send or receive until the interface is brought down then up again.

Here are the LPI related stats after the test:
     irq_tx_path_in_lpi_mode_n: 48
     irq_tx_path_exit_lpi_mode_n: 48
     irq_rx_path_in_lpi_mode_n: 325
     irq_rx_path_exit_lpi_mode_n: 325
     phy_eee_wakeup_error_n: 0

Like Martin, I tried playing around with eee in stmmac, but I could not
improve the situation. Then I tried disabling EEE advertisement on the
PHY (patch attached). With this patch, iperf3 runs nicely for me.

This is what the folks of FreeBSD have done for the Same MAC/PHY
combination [0]

On the P200 Board (PHY Micrel KSZ9031), EEE is off by default. There is
no problem on this board right now. I tried to force the activation of
EEE on this board and ended up in the same situation as the OdroidC2
(link broken). The stats were a bit different though:
     irq_tx_path_in_lpi_mode_n: 28
     irq_tx_path_exit_lpi_mode_n: 28
     irq_rx_path_in_lpi_mode_n: 408
     irq_rx_path_exit_lpi_mode_n: 408
     phy_eee_wakeup_error_n: 5440

To everybody having similar issue with their OdroidC2, could you try
the attached patch and let us know if it changes anything for you ?

Peppe, Alexandre,
What is your view on this ? I'm not sure that removing EEE
advertisement is the right way to address the problem ?
Could it be an issue stmmac ?
If there is any other information / test which would help understand
the issue, please let me know.

Cheers

Jerome


[0] : https://github.com/freebsd/freebsd-base-graphics/commit/1f49e276c
3801545dc0a337792a5f07e6ad39c84
 

> _______________________________________________
> linux-amlogic mailing list
> linux-amlogic@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-amlogic

[-- Attachment #2: realtek8211f-disable-eee-1000.patch --]
[-- Type: text/x-patch, Size: 2912 bytes --]

diff --git a/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts b/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts
index a45d1013c225..3cbeec63a439 100644
--- a/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts
+++ b/arch/arm64/boot/dts/amlogic/meson-gxbb-odroidc2.dts
@@ -127,3 +127,18 @@
 &usb1 {
 	status = "okay";
 };
+
+&ethmac {
+	phy-handle = <&eth_phy0>;
+
+	mdio {
+		compatible = "snps,dwmac-mdio";
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		eth_phy0: ethernet-phy@0 {
+			reg = <0>;
+			realtek,disable-eee-1000t;
+		};
+	};
+};
diff --git a/drivers/net/phy/realtek.c b/drivers/net/phy/realtek.c
index aadd6e9f54ad..30e20ba10f45 100644
--- a/drivers/net/phy/realtek.c
+++ b/drivers/net/phy/realtek.c
@@ -15,6 +15,12 @@
  */
 #include <linux/phy.h>
 #include <linux/module.h>
+#include <linux/of.h>
+
+struct rtl8211f_phy_priv {
+	bool eee_1000_disable;
+	bool eee_100_disable;
+};
 
 #define RTL821x_PHYSR		0x11
 #define RTL821x_PHYSR_DUPLEX	0x2000
@@ -93,6 +99,25 @@ static int rtl8211f_config_intr(struct phy_device *phydev)
 	return err;
 }
 
+static void rtl8211f_force_eee(struct phy_device *phydev)
+{
+	struct rtl8211f_phy_priv *priv = phydev->priv;
+	u16 val;
+
+	if (priv->eee_1000_disable || priv->eee_100_disable) {
+		val = phy_read_mmd_indirect(phydev, MDIO_AN_EEE_ADV,
+					    MDIO_MMD_AN);
+
+		if (priv->eee_1000_disable)
+			val &= ~MDIO_AN_EEE_ADV_1000T;
+		if (priv->eee_100_disable)
+			val &= ~MDIO_AN_EEE_ADV_100TX;
+
+		phy_write_mmd_indirect(phydev, MDIO_AN_EEE_ADV,
+				       MDIO_MMD_AN, val);
+	}
+}
+
 static int rtl8211f_config_init(struct phy_device *phydev)
 {
 	int ret;
@@ -102,6 +127,8 @@ static int rtl8211f_config_init(struct phy_device *phydev)
 	if (ret < 0)
 		return ret;
 
+	rtl8211f_force_eee(phydev);
+
 	if (phydev->interface == PHY_INTERFACE_MODE_RGMII) {
 		/* enable TXDLY */
 		phy_write(phydev, RTL8211F_PAGE_SELECT, 0xd08);
@@ -115,6 +142,26 @@ static int rtl8211f_config_init(struct phy_device *phydev)
 	return 0;
 }
 
+static int rtl8211f_phy_probe(struct phy_device *phydev)
+{
+	struct device *dev = &phydev->mdio.dev;
+	struct device_node *of_node = dev->of_node;
+	struct rtl8211f_phy_priv *priv;
+
+	priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+	if (!priv)
+		return -ENOMEM;
+
+	if (of_property_read_bool(of_node, "realtek,disable-eee-1000t"))
+		priv->eee_1000_disable= true;
+	if (of_property_read_bool(of_node, "realtek,disable-eee-100t"))
+		priv->eee_100_disable= true;
+
+	phydev->priv = priv;
+
+	return 0;
+}
+
 static struct phy_driver realtek_drvs[] = {
 	{
 		.phy_id         = 0x00008201,
@@ -164,6 +211,7 @@ static struct phy_driver realtek_drvs[] = {
 		.phy_id_mask	= 0x001fffff,
 		.features	= PHY_GBIT_FEATURES,
 		.flags		= PHY_HAS_INTERRUPT,
+		.probe		= &rtl8211f_phy_probe,
 		.config_aneg	= &genphy_config_aneg,
 		.config_init	= &rtl8211f_config_init,
 		.read_status	= &genphy_read_status,

^ permalink raw reply related	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-03 16:36             ` Jerome Brunet
  0 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-03 16:36 UTC (permalink / raw)
  To: linus-amlogic

On Sat, 2016-10-01 at 17:58 +0200, Martin Blumenstingl wrote:
> Hello Peppe,
> 
> On Mon, Sep 26, 2016 at 8:17 AM, Giuseppe CAVALLARO
> <peppe.cavallaro@st.com> wrote:
> > 
> > Hello Andr?
> > 
> > On 9/17/2016 11:23 PM, Andr? Roth wrote:
> > > 
> > > 
> > > 
> > > Hi all,
> > > 
> > > I have an odroid c2 board which shows this issue. No data is
> > > transmitted or received after a moment of intense tx traffic.
> > > Copying a
> > > 1GB file per scp from the board triggers it repeatedly.
> > > 
> > > The board has a stmmac - user ID: 0x11, Synopsys ID: 0x37.
> > > 
> > > When switching the network to 100Mb/s the copying does
> > > not seam to trigger the issue.
> > > 
> > > I've attached the ethtool statistics before and after the
> > > problem.
> > 
> > 
> > at first glance, it enters in EEE mode often in the ethtool.after.
> > On some platforms we met problems and it was necessary to disable
> > the
> > feature. Maybe, you can start looking at if this is true on yours.
> > We will see to provide a clean subset of patches to switch-on/off
> > it.
> I did some hacking in the stmmac driver to disable the LPI stuff (see
> the attachment)
> 
> Unfortunately this did not fix the problem.
> 
> I did not issue any ethtool commands not shown in the logs.
> Also I did not have time to change the AXI tuning / PBL value yet -
> so
> those are also untouched.
> 
> I will keep testing, but unfortunately my device is starting to fall
> apart (I sometimes have DDR initialization issues and u-boot fails to
> come up, oh dear...).

Hi all,

I did several tests on this issue with amlogic's S905 SoC (Synopsys MAC
- user ID: 0x11, Synopsys ID: 0x37.)?

With the OdroidC2 (PHY Realtek RTL8211F), EEE is on by default.
Just before launching iperf3, here are the ethtool stats regarding LPI:
? ? ?irq_tx_path_in_lpi_mode_n: 6
?????irq_tx_path_exit_lpi_mode_n: 5
?????irq_rx_path_in_lpi_mode_n: 76
?????irq_rx_path_exit_lpi_mode_n: 75
?????phy_eee_wakeup_error_n: 0

Sending data with iperf usually works for little while (between 0 and
10s)

# iperf3 -c 192.168.1.170 -p12345
Connecting to host 192.168.1.170, port 12345
local 192.168.1.30 port 54450 connected to 192.168.1.170 port 12345
Interval???????????Transfer?????Bandwidth???????Retr??Cwnd
0.00-1.00???sec???112 MBytes???938 Mbits/sec????0????409 KBytes???????
1.00-2.00???sec???112 MBytes???940 Mbits/sec????0????426 KBytes ? ? ??
2.00-3.00???sec???112 MBytes???939 Mbits/sec????0????426 KBytes???????
3.00-4.00???sec???112 MBytes???940 Mbits/sec????0????426 KBytes???????
4.00-5.00???sec???112 MBytes???940 Mbits/sec????0????426 KBytes???????
5.00-6.00???sec???112 MBytes???939 Mbits/sec????0????426 KBytes???????
6.00-7.00???sec??9.26 MBytes??77.6 Mbits/sec????2???1.41 KBytes???????
7.00-8.00???sec??0.00 Bytes??0.00 bits/sec????1???1.41 KBytes???????
8.00-9.00???sec??0.00 Bytes??0.00 bits/sec????0???1.41 KBytes???????
^C10.00-13.58??sec??0.00 Bytes??0.00 bits/sec????1???1.41 KBytes???????
- - - - - - - - - - - - - - - - - - - - - - - - -
Interval???????????Transfer?????Bandwidth???????Retr
0.00-13.58??sec???681 MBytes???421 Mbits/sec????4?????????????sender
0.00-13.58??sec??0.00 Bytes??0.00 bits/sec??????????????????receiver
iperf3: interrupt - the client has terminated

iperf3 does not exit ant the link seems completely broken. We cannot
send or receive until the interface is brought down then up again.

Here are the LPI related stats after the test:
? ? ?irq_tx_path_in_lpi_mode_n: 48
?????irq_tx_path_exit_lpi_mode_n: 48
?????irq_rx_path_in_lpi_mode_n: 325
?????irq_rx_path_exit_lpi_mode_n: 325
?????phy_eee_wakeup_error_n: 0

Like Martin, I tried playing around with eee in stmmac, but I could not
improve the situation. Then I tried disabling EEE advertisement on the
PHY (patch attached). With this patch, iperf3 runs nicely for me.

This is what the folks of FreeBSD have done for the Same MAC/PHY
combination [0]

On the P200 Board (PHY Micrel KSZ9031), EEE is off by default. There is
no problem on this board right now. I tried to force the activation of
EEE on this board and ended up in the same situation as the OdroidC2
(link broken). The stats were a bit different though:
? ? ?irq_tx_path_in_lpi_mode_n: 28
?????irq_tx_path_exit_lpi_mode_n: 28
?????irq_rx_path_in_lpi_mode_n: 408
?????irq_rx_path_exit_lpi_mode_n: 408
?????phy_eee_wakeup_error_n: 5440

To everybody having similar issue with their OdroidC2, could you try
the attached patch and let us know if it changes anything for you ?

Peppe, Alexandre,
What is your view on this ? I'm not sure that removing EEE
advertisement is the right way to address the problem ?
Could it be an issue stmmac ?
If there is any other information / test which would help understand
the issue, please let me know.

Cheers

Jerome


[0] :?https://github.com/freebsd/freebsd-base-graphics/commit/1f49e276c
3801545dc0a337792a5f07e6ad39c84
?

> _______________________________________________
> linux-amlogic mailing list
> linux-amlogic at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-amlogic
-------------- next part --------------
A non-text attachment was scrubbed...
Name: realtek8211f-disable-eee-1000.patch
Type: text/x-patch
Size: 2912 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/linux-amlogic/attachments/20161103/27c2f20a/attachment.bin>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-10-31 10:25       ` André Roth
@ 2016-11-03 16:57         ` Jerome Brunet
  -1 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-03 16:57 UTC (permalink / raw)
  To: André Roth, Martin Blumenstingl
  Cc: Johnson Leung, Giuseppe CAVALLARO, linux-amlogic,
	Alexandre Torgue, netdev

On Mon, 2016-10-31 at 11:25 +0100, André Roth wrote:
> Hi all,
>  
> > 
> > on my device this results in:
> > [0xc9410018] = 0x2000000
> > [0xc9410030] = 0x0
> > [0xc941003c] = 0x0
> > [0xc9411000] = 0x1100802
> > [0xc9411018] = 0x2202006
> > [0xc9411028] = 0x0
> > 
> > maybe someone else could try the command from above on his device
> > (running the original Amlogic kernel).
> 
> those registers have the same value on an original image from
> hardkernel: 
> 
> Linux odroid64 3.14.65-65 #1 SMP PREEMPT Sat May 28
> 02:50:51 BRT 2016 aarch64 aarch64 aarch64 GNU/Linux
> 
> > 
> > please also state if ethernet is working properly on the original
> > kernel (and preferably which device/board this is).
> 
> yes, the ethernet works flawless in 100 and 1000 Mbit/s on the 3.14
> kernel.

Andre, the 3.14 kernel you are talking, is it this one ? : 
https://github.com/hardkernel/linux/tree/odroidc2-3.14.y

Because in drivers/net/phy/realtek.c, they disable EEE, but
also 1000Base-T Full Duplex advertisement ?

+	/* disable 1000m adv*/
+	val = phy_read(phydev, 0x9);
+	phy_write(phydev, 0x9, val&(~(1<<9)));

If this is the kernel you are running, you should not be able to have
ethernet at 1000MB/s ? Or is it in half duplex mode ?

> 
> I can now confirm that both 100 and 1000 Mbit/s do not work properly
> on the 4.8/integ branch. the network connection is interrupted after
> some outbound traffic. it can be recovered by running a ifdown/ifup
> which restarts dhclient, which I think is able to somehow reset the
> interface. 
> 
> Anything I can help to debug the issue further ?
> 
> Regards,
> 
>  André
>  
> 
> 
> _______________________________________________
> linux-amlogic mailing list
> linux-amlogic@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-amlogic

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-03 16:57         ` Jerome Brunet
  0 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-03 16:57 UTC (permalink / raw)
  To: linus-amlogic

On Mon, 2016-10-31 at 11:25 +0100, Andr? Roth wrote:
> Hi all,
> ?
> > 
> > on my device this results in:
> > [0xc9410018] = 0x2000000
> > [0xc9410030] = 0x0
> > [0xc941003c] = 0x0
> > [0xc9411000] = 0x1100802
> > [0xc9411018] = 0x2202006
> > [0xc9411028] = 0x0
> > 
> > maybe someone else could try the command from above on his device
> > (running the original Amlogic kernel).
> 
> those registers have the same value on an original image from
> hardkernel:?
> 
> Linux odroid64 3.14.65-65 #1 SMP PREEMPT Sat May 28
> 02:50:51 BRT 2016 aarch64 aarch64 aarch64 GNU/Linux
> 
> > 
> > please also state if ethernet is working properly on the original
> > kernel (and preferably which device/board this is).
> 
> yes, the ethernet works flawless in 100 and 1000 Mbit/s on the 3.14
> kernel.

Andre, the 3.14 kernel you are talking, is it this one ? :?
https://github.com/hardkernel/linux/tree/odroidc2-3.14.y

Because in drivers/net/phy/realtek.c, they disable EEE, but
also?1000Base-T Full Duplex advertisement ?

+	/* disable 1000m adv*/
+	val = phy_read(phydev, 0x9);
+	phy_write(phydev, 0x9, val&(~(1<<9)));

If this is the kernel you are running, you should not be able to have
ethernet@1000MB/s ? Or is it in half duplex mode ?

> 
> I can now confirm that both 100 and 1000 Mbit/s do not work properly
> on the 4.8/integ branch. the network connection is interrupted after
> some outbound traffic. it can be recovered by running a ifdown/ifup
> which restarts dhclient, which I think is able to somehow reset the
> interface.?
> 
> Anything I can help to debug the issue further ?
> 
> Regards,
> 
> ?Andr?
> ?
> 
> 
> _______________________________________________
> linux-amlogic mailing list
> linux-amlogic at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-amlogic

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-03 16:36             ` Jerome Brunet
@ 2016-11-05 12:20               ` Martin Blumenstingl
  -1 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-11-05 12:20 UTC (permalink / raw)
  To: Jerome Brunet
  Cc: Giuseppe CAVALLARO, Johnson Leung, netdev, André Roth,
	Alexandre Torgue, linux-amlogic, ganbold, yongari

On Thu, Nov 3, 2016 at 5:36 PM, Jerome Brunet <jbrunet@baylibre.com> wrote:
> Hi all,
>
> I did several tests on this issue with amlogic's S905 SoC (Synopsys MAC
> - user ID: 0x11, Synopsys ID: 0x37.)
>
> With the OdroidC2 (PHY Realtek RTL8211F), EEE is on by default.
> Just before launching iperf3, here are the ethtool stats regarding LPI:
>      irq_tx_path_in_lpi_mode_n: 6
>      irq_tx_path_exit_lpi_mode_n: 5
>      irq_rx_path_in_lpi_mode_n: 76
>      irq_rx_path_exit_lpi_mode_n: 75
>      phy_eee_wakeup_error_n: 0
>
> Sending data with iperf usually works for little while (between 0 and
> 10s)
>
> # iperf3 -c 192.168.1.170 -p12345
> Connecting to host 192.168.1.170, port 12345
> local 192.168.1.30 port 54450 connected to 192.168.1.170 port 12345
> Interval           Transfer     Bandwidth       Retr  Cwnd
> 0.00-1.00   sec   112 MBytes   938 Mbits/sec    0    409 KBytes
> 1.00-2.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes
> 2.00-3.00   sec   112 MBytes   939 Mbits/sec    0    426 KBytes
> 3.00-4.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes
> 4.00-5.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes
> 5.00-6.00   sec   112 MBytes   939 Mbits/sec    0    426 KBytes
> 6.00-7.00   sec  9.26 MBytes  77.6 Mbits/sec    2   1.41 KBytes
> 7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
> 8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> ^C10.00-13.58  sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> Interval           Transfer     Bandwidth       Retr
> 0.00-13.58  sec   681 MBytes   421 Mbits/sec    4             sender
> 0.00-13.58  sec  0.00 Bytes  0.00 bits/sec                  receiver
> iperf3: interrupt - the client has terminated
>
> iperf3 does not exit ant the link seems completely broken. We cannot
> send or receive until the interface is brought down then up again.
>
> Here are the LPI related stats after the test:
>      irq_tx_path_in_lpi_mode_n: 48
>      irq_tx_path_exit_lpi_mode_n: 48
>      irq_rx_path_in_lpi_mode_n: 325
>      irq_rx_path_exit_lpi_mode_n: 325
>      phy_eee_wakeup_error_n: 0
>
> Like Martin, I tried playing around with eee in stmmac, but I could not
> improve the situation. Then I tried disabling EEE advertisement on the
> PHY (patch attached). With this patch, iperf3 runs nicely for me.
>
> This is what the folks of FreeBSD have done for the Same MAC/PHY
> combination [0]
>
> On the P200 Board (PHY Micrel KSZ9031), EEE is off by default. There is
> no problem on this board right now. I tried to force the activation of
> EEE on this board and ended up in the same situation as the OdroidC2
> (link broken). The stats were a bit different though:
>      irq_tx_path_in_lpi_mode_n: 28
>      irq_tx_path_exit_lpi_mode_n: 28
>      irq_rx_path_in_lpi_mode_n: 408
>      irq_rx_path_exit_lpi_mode_n: 408
>      phy_eee_wakeup_error_n: 5440
>
> To everybody having similar issue with their OdroidC2, could you try
> the attached patch and let us know if it changes anything for you ?
I have tried this on my Vega S95 Meta clone - the patch can be found
here if anyone cares: [0].
Unfortunately this does not improve the situation for me:
# iperf3 --client 192.168.1.100
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.198 port 50720 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   178 KBytes  1.46 Mbits/sec   13   1.41 KBytes
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  4]   3.00-4.00   sec  5.66 KBytes  46.3 Kbits/sec    4   1.41 KBytes
[  4]   4.00-5.00   sec  63.6 KBytes   521 Kbits/sec    2   1.41 KBytes
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    4   1.41 KBytes
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    2   1.41 KBytes
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   247 KBytes   203 Kbits/sec   29             sender
[  4]   0.00-10.00  sec  90.5 KBytes  74.1 Kbits/sec                  receiver

iperf Done.
# iperf3 --client 192.168.1.100 -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.198 port 50724 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   111 MBytes   930 Mbits/sec
[  4]   1.00-2.00   sec   111 MBytes   935 Mbits/sec
[  4]   2.00-3.00   sec   111 MBytes   934 Mbits/sec
[  4]   3.00-4.00   sec   111 MBytes   934 Mbits/sec
[  4]   4.00-5.00   sec   111 MBytes   934 Mbits/sec
[  4]   5.00-6.00   sec   111 MBytes   935 Mbits/sec
[  4]   6.00-7.00   sec   111 MBytes   935 Mbits/sec
[  4]   7.00-8.00   sec   111 MBytes   934 Mbits/sec
[  4]   8.00-9.00   sec   111 MBytes   934 Mbits/sec
[  4]   9.00-10.00  sec   111 MBytes   934 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.09 GBytes   936 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.09 GBytes   934 Mbits/sec                  receiver

iperf Done.
#

However, if I remove the realtek,disable-eee-* properties and use
max-speed = <100>; instead ethernet is working fine (but limited to
100Mbit/s obviously).

> Peppe, Alexandre,
> What is your view on this ? I'm not sure that removing EEE
> advertisement is the right way to address the problem ?
> Could it be an issue stmmac ?
> If there is any other information / test which would help understand
> the issue, please let me know.
I CC'ed the two FreeBSD developers to who added the corresponding
FreeBSD code. Maybe they could also explain why that change was
needed.
Peppe's and Alexandre's feedback will hopefully also lead to
improvements in the FreeBSD driver.


[0] https://paste.kde.org/p9lwilneh

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-05 12:20               ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-11-05 12:20 UTC (permalink / raw)
  To: linus-amlogic

On Thu, Nov 3, 2016 at 5:36 PM, Jerome Brunet <jbrunet@baylibre.com> wrote:
> Hi all,
>
> I did several tests on this issue with amlogic's S905 SoC (Synopsys MAC
> - user ID: 0x11, Synopsys ID: 0x37.)
>
> With the OdroidC2 (PHY Realtek RTL8211F), EEE is on by default.
> Just before launching iperf3, here are the ethtool stats regarding LPI:
>      irq_tx_path_in_lpi_mode_n: 6
>      irq_tx_path_exit_lpi_mode_n: 5
>      irq_rx_path_in_lpi_mode_n: 76
>      irq_rx_path_exit_lpi_mode_n: 75
>      phy_eee_wakeup_error_n: 0
>
> Sending data with iperf usually works for little while (between 0 and
> 10s)
>
> # iperf3 -c 192.168.1.170 -p12345
> Connecting to host 192.168.1.170, port 12345
> local 192.168.1.30 port 54450 connected to 192.168.1.170 port 12345
> Interval           Transfer     Bandwidth       Retr  Cwnd
> 0.00-1.00   sec   112 MBytes   938 Mbits/sec    0    409 KBytes
> 1.00-2.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes
> 2.00-3.00   sec   112 MBytes   939 Mbits/sec    0    426 KBytes
> 3.00-4.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes
> 4.00-5.00   sec   112 MBytes   940 Mbits/sec    0    426 KBytes
> 5.00-6.00   sec   112 MBytes   939 Mbits/sec    0    426 KBytes
> 6.00-7.00   sec  9.26 MBytes  77.6 Mbits/sec    2   1.41 KBytes
> 7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
> 8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
> ^C10.00-13.58  sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
> - - - - - - - - - - - - - - - - - - - - - - - - -
> Interval           Transfer     Bandwidth       Retr
> 0.00-13.58  sec   681 MBytes   421 Mbits/sec    4             sender
> 0.00-13.58  sec  0.00 Bytes  0.00 bits/sec                  receiver
> iperf3: interrupt - the client has terminated
>
> iperf3 does not exit ant the link seems completely broken. We cannot
> send or receive until the interface is brought down then up again.
>
> Here are the LPI related stats after the test:
>      irq_tx_path_in_lpi_mode_n: 48
>      irq_tx_path_exit_lpi_mode_n: 48
>      irq_rx_path_in_lpi_mode_n: 325
>      irq_rx_path_exit_lpi_mode_n: 325
>      phy_eee_wakeup_error_n: 0
>
> Like Martin, I tried playing around with eee in stmmac, but I could not
> improve the situation. Then I tried disabling EEE advertisement on the
> PHY (patch attached). With this patch, iperf3 runs nicely for me.
>
> This is what the folks of FreeBSD have done for the Same MAC/PHY
> combination [0]
>
> On the P200 Board (PHY Micrel KSZ9031), EEE is off by default. There is
> no problem on this board right now. I tried to force the activation of
> EEE on this board and ended up in the same situation as the OdroidC2
> (link broken). The stats were a bit different though:
>      irq_tx_path_in_lpi_mode_n: 28
>      irq_tx_path_exit_lpi_mode_n: 28
>      irq_rx_path_in_lpi_mode_n: 408
>      irq_rx_path_exit_lpi_mode_n: 408
>      phy_eee_wakeup_error_n: 5440
>
> To everybody having similar issue with their OdroidC2, could you try
> the attached patch and let us know if it changes anything for you ?
I have tried this on my Vega S95 Meta clone - the patch can be found
here if anyone cares: [0].
Unfortunately this does not improve the situation for me:
# iperf3 --client 192.168.1.100
Connecting to host 192.168.1.100, port 5201
[  4] local 192.168.1.198 port 50720 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   178 KBytes  1.46 Mbits/sec   13   1.41 KBytes
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  4]   3.00-4.00   sec  5.66 KBytes  46.3 Kbits/sec    4   1.41 KBytes
[  4]   4.00-5.00   sec  63.6 KBytes   521 Kbits/sec    2   1.41 KBytes
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    4   1.41 KBytes
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    2   1.41 KBytes
[  4]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.41 KBytes
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    1   1.41 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   247 KBytes   203 Kbits/sec   29             sender
[  4]   0.00-10.00  sec  90.5 KBytes  74.1 Kbits/sec                  receiver

iperf Done.
# iperf3 --client 192.168.1.100 -R
Connecting to host 192.168.1.100, port 5201
Reverse mode, remote host 192.168.1.100 is sending
[  4] local 192.168.1.198 port 50724 connected to 192.168.1.100 port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec   111 MBytes   930 Mbits/sec
[  4]   1.00-2.00   sec   111 MBytes   935 Mbits/sec
[  4]   2.00-3.00   sec   111 MBytes   934 Mbits/sec
[  4]   3.00-4.00   sec   111 MBytes   934 Mbits/sec
[  4]   4.00-5.00   sec   111 MBytes   934 Mbits/sec
[  4]   5.00-6.00   sec   111 MBytes   935 Mbits/sec
[  4]   6.00-7.00   sec   111 MBytes   935 Mbits/sec
[  4]   7.00-8.00   sec   111 MBytes   934 Mbits/sec
[  4]   8.00-9.00   sec   111 MBytes   934 Mbits/sec
[  4]   9.00-10.00  sec   111 MBytes   934 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  1.09 GBytes   936 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  1.09 GBytes   934 Mbits/sec                  receiver

iperf Done.
#

However, if I remove the realtek,disable-eee-* properties and use
max-speed = <100>; instead ethernet is working fine (but limited to
100Mbit/s obviously).

> Peppe, Alexandre,
> What is your view on this ? I'm not sure that removing EEE
> advertisement is the right way to address the problem ?
> Could it be an issue stmmac ?
> If there is any other information / test which would help understand
> the issue, please let me know.
I CC'ed the two FreeBSD developers to who added the corresponding
FreeBSD code. Maybe they could also explain why that change was
needed.
Peppe's and Alexandre's feedback will hopefully also lead to
improvements in the FreeBSD driver.


[0] https://paste.kde.org/p9lwilneh

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-03 16:57         ` Jerome Brunet
@ 2016-11-07 10:59           ` Giuseppe CAVALLARO
  -1 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-11-07 10:59 UTC (permalink / raw)
  To: Jerome Brunet, André Roth, Martin Blumenstingl, Alexandre Torgue
  Cc: Johnson Leung, linux-amlogic, netdev

Hi Jerome

We met, on some platform, problems that, in a glance,
look similar to yours and those were related to EEE and
LPI interrupts.
We had a subset of patches to disable EEE but, indeed,
if you disable from PHY, the STMMAC will never act
the protocol. We can provide you these patches... I will
ask Alex to decide how to proceed.
There is a issue on lpi timer and I have another patch to
share but I am not sure this can help.

In sum, if EEE is a suspected problem we can share and review
some patches.
In the meantime, I will read again the thread just to see if
there is something I am missing.

Regards
Peppe

On 11/3/2016 5:57 PM, Jerome Brunet wrote:
> On Mon, 2016-10-31 at 11:25 +0100, André Roth wrote:
>> Hi all,
>>
>>>
>>> on my device this results in:
>>> [0xc9410018] = 0x2000000
>>> [0xc9410030] = 0x0
>>> [0xc941003c] = 0x0
>>> [0xc9411000] = 0x1100802
>>> [0xc9411018] = 0x2202006
>>> [0xc9411028] = 0x0
>>>
>>> maybe someone else could try the command from above on his device
>>> (running the original Amlogic kernel).
>>
>> those registers have the same value on an original image from
>> hardkernel:
>>
>> Linux odroid64 3.14.65-65 #1 SMP PREEMPT Sat May 28
>> 02:50:51 BRT 2016 aarch64 aarch64 aarch64 GNU/Linux
>>
>>>
>>> please also state if ethernet is working properly on the original
>>> kernel (and preferably which device/board this is).
>>
>> yes, the ethernet works flawless in 100 and 1000 Mbit/s on the 3.14
>> kernel.
>
> Andre, the 3.14 kernel you are talking, is it this one ? :
> https://github.com/hardkernel/linux/tree/odroidc2-3.14.y
>
> Because in drivers/net/phy/realtek.c, they disable EEE, but
> also 1000Base-T Full Duplex advertisement ?
>
> +	/* disable 1000m adv*/
> +	val = phy_read(phydev, 0x9);
> +	phy_write(phydev, 0x9, val&(~(1<<9)));
>
> If this is the kernel you are running, you should not be able to have
> ethernet at 1000MB/s ? Or is it in half duplex mode ?
>
>>
>> I can now confirm that both 100 and 1000 Mbit/s do not work properly
>> on the 4.8/integ branch. the network connection is interrupted after
>> some outbound traffic. it can be recovered by running a ifdown/ifup
>> which restarts dhclient, which I think is able to somehow reset the
>> interface.
>>
>> Anything I can help to debug the issue further ?
>>
>> Regards,
>>
>>  André
>>
>>
>>
>> _______________________________________________
>> linux-amlogic mailing list
>> linux-amlogic@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-amlogic
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-07 10:59           ` Giuseppe CAVALLARO
  0 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-11-07 10:59 UTC (permalink / raw)
  To: linus-amlogic

Hi Jerome

We met, on some platform, problems that, in a glance,
look similar to yours and those were related to EEE and
LPI interrupts.
We had a subset of patches to disable EEE but, indeed,
if you disable from PHY, the STMMAC will never act
the protocol. We can provide you these patches... I will
ask Alex to decide how to proceed.
There is a issue on lpi timer and I have another patch to
share but I am not sure this can help.

In sum, if EEE is a suspected problem we can share and review
some patches.
In the meantime, I will read again the thread just to see if
there is something I am missing.

Regards
Peppe

On 11/3/2016 5:57 PM, Jerome Brunet wrote:
> On Mon, 2016-10-31 at 11:25 +0100, Andr? Roth wrote:
>> Hi all,
>>
>>>
>>> on my device this results in:
>>> [0xc9410018] = 0x2000000
>>> [0xc9410030] = 0x0
>>> [0xc941003c] = 0x0
>>> [0xc9411000] = 0x1100802
>>> [0xc9411018] = 0x2202006
>>> [0xc9411028] = 0x0
>>>
>>> maybe someone else could try the command from above on his device
>>> (running the original Amlogic kernel).
>>
>> those registers have the same value on an original image from
>> hardkernel:
>>
>> Linux odroid64 3.14.65-65 #1 SMP PREEMPT Sat May 28
>> 02:50:51 BRT 2016 aarch64 aarch64 aarch64 GNU/Linux
>>
>>>
>>> please also state if ethernet is working properly on the original
>>> kernel (and preferably which device/board this is).
>>
>> yes, the ethernet works flawless in 100 and 1000 Mbit/s on the 3.14
>> kernel.
>
> Andre, the 3.14 kernel you are talking, is it this one ? :
> https://github.com/hardkernel/linux/tree/odroidc2-3.14.y
>
> Because in drivers/net/phy/realtek.c, they disable EEE, but
> also 1000Base-T Full Duplex advertisement ?
>
> +	/* disable 1000m adv*/
> +	val = phy_read(phydev, 0x9);
> +	phy_write(phydev, 0x9, val&(~(1<<9)));
>
> If this is the kernel you are running, you should not be able to have
> ethernet at 1000MB/s ? Or is it in half duplex mode ?
>
>>
>> I can now confirm that both 100 and 1000 Mbit/s do not work properly
>> on the 4.8/integ branch. the network connection is interrupted after
>> some outbound traffic. it can be recovered by running a ifdown/ifup
>> which restarts dhclient, which I think is able to somehow reset the
>> interface.
>>
>> Anything I can help to debug the issue further ?
>>
>> Regards,
>>
>>  Andr?
>>
>>
>>
>> _______________________________________________
>> linux-amlogic mailing list
>> linux-amlogic at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-amlogic
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-07 10:59           ` Giuseppe CAVALLARO
@ 2016-11-07 17:37             ` Martin Blumenstingl
  -1 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-11-07 17:37 UTC (permalink / raw)
  To: Giuseppe CAVALLARO
  Cc: Jerome Brunet, André Roth, Alexandre Torgue, Johnson Leung,
	linux-amlogic, netdev, afaerber

Hi Peppe,

On Mon, Nov 7, 2016 at 11:59 AM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> In the meantime, I will read again the thread just to see if
> there is something I am missing.
if you are re-reading this thread: please note that there are two
devices in discussion here!
Both are using the Amlogic S905 (GXBB) SoC and both are experiencing
the same issue (Gbit TX issues, RX with Gbit speeds and RX/TX with
100Mbit speed are NOT affected):
- Odroid-C2 (used by Jerome and André Roth)
- Tronsmart Vega S95 Meta (my device)

The (Gbit TX) problem seems to be gone on the Odroid-C2 with Jerome's
patch which disables EEE in drivers/net/phy/realtek.c (at least in his
tests, I don't have that device so I can't verify).
The same problem still appears on my Tronsmart Vega S95 Meta even with
the patched PHY driver.

Unfortunately I don't have a second device to rule out that my
Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
errors from time to time in u-boot). Maybe Andreas Faerber can test
ethernet with and without Jerome's patch on one of his Tronsmart
devices.


Regards,
Martin

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-07 17:37             ` Martin Blumenstingl
  0 siblings, 0 replies; 56+ messages in thread
From: Martin Blumenstingl @ 2016-11-07 17:37 UTC (permalink / raw)
  To: linus-amlogic

Hi Peppe,

On Mon, Nov 7, 2016 at 11:59 AM, Giuseppe CAVALLARO
<peppe.cavallaro@st.com> wrote:
> In the meantime, I will read again the thread just to see if
> there is something I am missing.
if you are re-reading this thread: please note that there are two
devices in discussion here!
Both are using the Amlogic S905 (GXBB) SoC and both are experiencing
the same issue (Gbit TX issues, RX with Gbit speeds and RX/TX with
100Mbit speed are NOT affected):
- Odroid-C2 (used by Jerome and Andr? Roth)
- Tronsmart Vega S95 Meta (my device)

The (Gbit TX) problem seems to be gone on the Odroid-C2 with Jerome's
patch which disables EEE in drivers/net/phy/realtek.c (at least in his
tests, I don't have that device so I can't verify).
The same problem still appears on my Tronsmart Vega S95 Meta even with
the patched PHY driver.

Unfortunately I don't have a second device to rule out that my
Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
errors from time to time in u-boot). Maybe Andreas Faerber can test
ethernet with and without Jerome's patch on one of his Tronsmart
devices.


Regards,
Martin

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-03 16:57         ` Jerome Brunet
@ 2016-11-13 19:13           ` André Roth
  -1 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-11-13 19:13 UTC (permalink / raw)
  To: Jerome Brunet
  Cc: Martin Blumenstingl, Johnson Leung, Giuseppe CAVALLARO,
	linux-amlogic, Alexandre Torgue, netdev


> Andre, the 3.14 kernel you are talking, is it this one ? : 
> https://github.com/hardkernel/linux/tree/odroidc2-3.14.y

yes
 
> Because in drivers/net/phy/realtek.c, they disable EEE, but
> also 1000Base-T Full Duplex advertisement ?
> 
> +	/* disable 1000m adv*/
> +	val = phy_read(phydev, 0x9);
> +	phy_write(phydev, 0x9, val&(~(1<<9)));
> 
> If this is the kernel you are running, you should not be able to have
> ethernet at 1000MB/s ? Or is it in half duplex mode ?

ethtool shows 1000Mb/s Full-Duplex and the bandwith is around 300Mb/s
(as measured by scp). kernel version: 3.14.65-73

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-13 19:13           ` André Roth
  0 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-11-13 19:13 UTC (permalink / raw)
  To: linus-amlogic


> Andre, the 3.14 kernel you are talking, is it this one ? :?
> https://github.com/hardkernel/linux/tree/odroidc2-3.14.y

yes
 
> Because in drivers/net/phy/realtek.c, they disable EEE, but
> also?1000Base-T Full Duplex advertisement ?
> 
> +	/* disable 1000m adv*/
> +	val = phy_read(phydev, 0x9);
> +	phy_write(phydev, 0x9, val&(~(1<<9)));
> 
> If this is the kernel you are running, you should not be able to have
> ethernet at 1000MB/s ? Or is it in half duplex mode ?

ethtool shows 1000Mb/s Full-Duplex and the bandwith is around 300Mb/s
(as measured by scp). kernel version: 3.14.65-73

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-03 16:36             ` Jerome Brunet
@ 2016-11-13 19:20               ` André Roth
  -1 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-11-13 19:20 UTC (permalink / raw)
  To: Jerome Brunet
  Cc: Martin Blumenstingl, Giuseppe CAVALLARO, Johnson Leung, netdev,
	Alexandre Torgue, linux-amlogic


Hi all,

> To everybody having similar issue with their OdroidC2, could you try
> the attached patch and let us know if it changes anything for you ?

I can confirm that this patch removes the problem, I have now constant
~300Mb/s in both directions without any issue ! :)

I used the v4.10/integ branch, which shows the problem without applying
this patch. 

Thanks for your work,

 André 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-13 19:20               ` André Roth
  0 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-11-13 19:20 UTC (permalink / raw)
  To: linus-amlogic


Hi all,

> To everybody having similar issue with their OdroidC2, could you try
> the attached patch and let us know if it changes anything for you ?

I can confirm that this patch removes the problem, I have now constant
~300Mb/s in both directions without any issue ! :)

I used the v4.10/integ branch, which shows the problem without applying
this patch. 

Thanks for your work,

 Andr? 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-07 17:37             ` Martin Blumenstingl
@ 2016-11-14  7:47               ` Giuseppe CAVALLARO
  -1 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-11-14  7:47 UTC (permalink / raw)
  To: Martin Blumenstingl
  Cc: Jerome Brunet, André Roth, Alexandre Torgue, Johnson Leung,
	linux-amlogic, netdev, afaerber

Hello Martin

On 11/7/2016 6:37 PM, Martin Blumenstingl wrote:
> Hi Peppe,
>
> On Mon, Nov 7, 2016 at 11:59 AM, Giuseppe CAVALLARO
> <peppe.cavallaro@st.com> wrote:
>> In the meantime, I will read again the thread just to see if
>> there is something I am missing.
> if you are re-reading this thread: please note that there are two
> devices in discussion here!

many thx for the sum :-)

> Both are using the Amlogic S905 (GXBB) SoC and both are experiencing
> the same issue (Gbit TX issues, RX with Gbit speeds and RX/TX with
> 100Mbit speed are NOT affected):
> - Odroid-C2 (used by Jerome and André Roth)
> - Tronsmart Vega S95 Meta (my device)
>
> The (Gbit TX) problem seems to be gone on the Odroid-C2 with Jerome's
> patch which disables EEE in drivers/net/phy/realtek.c (at least in his
> tests, I don't have that device so I can't verify).
> The same problem still appears on my Tronsmart Vega S95 Meta even with
> the patched PHY driver.

just an doubt, maybe useful, in the past, on GiGa setup I saw similar
problems and it was due to retiming so maybe 2ns could be necessary
(or better granularity via PAD logic if available).

Regards
Peppe

> Unfortunately I don't have a second device to rule out that my
> Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> errors from time to time in u-boot). Maybe Andreas Faerber can test
> ethernet with and without Jerome's patch on one of his Tronsmart
> devices.
>
>
> Regards,
> Martin
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-14  7:47               ` Giuseppe CAVALLARO
  0 siblings, 0 replies; 56+ messages in thread
From: Giuseppe CAVALLARO @ 2016-11-14  7:47 UTC (permalink / raw)
  To: linus-amlogic

Hello Martin

On 11/7/2016 6:37 PM, Martin Blumenstingl wrote:
> Hi Peppe,
>
> On Mon, Nov 7, 2016 at 11:59 AM, Giuseppe CAVALLARO
> <peppe.cavallaro@st.com> wrote:
>> In the meantime, I will read again the thread just to see if
>> there is something I am missing.
> if you are re-reading this thread: please note that there are two
> devices in discussion here!

many thx for the sum :-)

> Both are using the Amlogic S905 (GXBB) SoC and both are experiencing
> the same issue (Gbit TX issues, RX with Gbit speeds and RX/TX with
> 100Mbit speed are NOT affected):
> - Odroid-C2 (used by Jerome and Andr? Roth)
> - Tronsmart Vega S95 Meta (my device)
>
> The (Gbit TX) problem seems to be gone on the Odroid-C2 with Jerome's
> patch which disables EEE in drivers/net/phy/realtek.c (at least in his
> tests, I don't have that device so I can't verify).
> The same problem still appears on my Tronsmart Vega S95 Meta even with
> the patched PHY driver.

just an doubt, maybe useful, in the past, on GiGa setup I saw similar
problems and it was due to retiming so maybe 2ns could be necessary
(or better granularity via PAD logic if available).

Regards
Peppe

> Unfortunately I don't have a second device to rule out that my
> Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> errors from time to time in u-boot). Maybe Andreas Faerber can test
> ethernet with and without Jerome's patch on one of his Tronsmart
> devices.
>
>
> Regards,
> Martin
>

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-13 19:13           ` André Roth
@ 2016-11-14 10:49             ` Jerome Brunet
  -1 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-14 10:49 UTC (permalink / raw)
  To: André Roth
  Cc: Martin Blumenstingl, Johnson Leung, Giuseppe CAVALLARO,
	linux-amlogic, Alexandre Torgue, netdev

On Sun, 2016-11-13 at 20:13 +0100, André Roth wrote:
> > 
> > Andre, the 3.14 kernel you are talking, is it this one ? : 
> > https://github.com/hardkernel/linux/tree/odroidc2-3.14.y
> 
> yes
>  
> > 
> > Because in drivers/net/phy/realtek.c, they disable EEE, but
> > also 1000Base-T Full Duplex advertisement ?
> > 
> > +	/* disable 1000m adv*/
> > +	val = phy_read(phydev, 0x9);
> > +	phy_write(phydev, 0x9, val&(~(1<<9)));
> > 
> > If this is the kernel you are running, you should not be able to
> > have
> > ethernet at 1000MB/s ? Or is it in half duplex mode ?
> 
> ethtool shows 1000Mb/s Full-Duplex and the bandwith is around 300Mb/s
> (as measured by scp). kernel version: 3.14.65-73


Andre,

I checked again the kernel at https://github.com/hardkernel/linux/tree/
odroidc2-3.14.y. The version you mention (3.14.65-73) seems to be:
sha1: c75d5f4d1516cdd86d90a9d1c565bb0ed9251036
tag: jenkins-deb s905 kernel-73

In this particular version, both realtek drivers:
- drivers/net/phy/realtek.c
- drivers/amlogic/ethernet/phy/am_realtek.c

have the hack to disable 1000M advertisement. I don't understand how it
possible for you to have 1000Base-T Full Duplex with this, maybe I'm
missing something here ?

If you did compile the kernel yourself, could you check the 2 file
mentioned above ? Just to be sure there was no patch applied at the
last minute, which would not show up in the git history of hardkernel ?

Thx

Jerome

> 
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-14 10:49             ` Jerome Brunet
  0 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-14 10:49 UTC (permalink / raw)
  To: linus-amlogic

On Sun, 2016-11-13 at 20:13 +0100, Andr? Roth wrote:
> > 
> > Andre, the 3.14 kernel you are talking, is it this one ? :?
> > https://github.com/hardkernel/linux/tree/odroidc2-3.14.y
> 
> yes
> ?
> > 
> > Because in drivers/net/phy/realtek.c, they disable EEE, but
> > also?1000Base-T Full Duplex advertisement ?
> > 
> > +	/* disable 1000m adv*/
> > +	val = phy_read(phydev, 0x9);
> > +	phy_write(phydev, 0x9, val&(~(1<<9)));
> > 
> > If this is the kernel you are running, you should not be able to
> > have
> > ethernet at 1000MB/s ? Or is it in half duplex mode ?
> 
> ethtool shows 1000Mb/s Full-Duplex and the bandwith is around 300Mb/s
> (as measured by scp). kernel version: 3.14.65-73


Andre,

I checked again the kernel at?https://github.com/hardkernel/linux/tree/
odroidc2-3.14.y. The version you mention (3.14.65-73) seems to be:
sha1:?c75d5f4d1516cdd86d90a9d1c565bb0ed9251036
tag: jenkins-deb s905 kernel-73

In this particular version, both realtek drivers:
- drivers/net/phy/realtek.c
- drivers/amlogic/ethernet/phy/am_realtek.c

have the hack to disable 1000M advertisement. I don't understand how it
possible for you to have 1000Base-T Full Duplex with this, maybe I'm
missing something here ?

If you did compile the kernel yourself, could you check the 2 file
mentioned above ? Just to be sure there was no patch applied at the
last minute, which would not show up in the git history of hardkernel ?

Thx

Jerome

> 
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-14  7:47               ` Giuseppe CAVALLARO
@ 2016-11-14 11:02                 ` Jerome Brunet
  -1 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-14 11:02 UTC (permalink / raw)
  To: Giuseppe CAVALLARO, Martin Blumenstingl
  Cc: André Roth, Alexandre Torgue, Johnson Leung, linux-amlogic,
	netdev, afaerber

On Mon, 2016-11-14 at 08:47 +0100, Giuseppe CAVALLARO wrote:
> Hello Martin
> 
> On 11/7/2016 6:37 PM, Martin Blumenstingl wrote:
> > 
> > Hi Peppe,
> > 
> > On Mon, Nov 7, 2016 at 11:59 AM, Giuseppe CAVALLARO
> > <peppe.cavallaro@st.com> wrote:
> > > 
> > > In the meantime, I will read again the thread just to see if
> > > there is something I am missing.
> > if you are re-reading this thread: please note that there are two
> > devices in discussion here!
> 
> many thx for the sum :-)
> 
> > 
> > Both are using the Amlogic S905 (GXBB) SoC and both are
> > experiencing
> > the same issue (Gbit TX issues, RX with Gbit speeds and RX/TX with
> > 100Mbit speed are NOT affected):
> > - Odroid-C2 (used by Jerome and André Roth)
> > - Tronsmart Vega S95 Meta (my device)
> > 
> > The (Gbit TX) problem seems to be gone on the Odroid-C2 with
> > Jerome's
> > patch which disables EEE in drivers/net/phy/realtek.c (at least in
> > his
> > tests, I don't have that device so I can't verify).
> > The same problem still appears on my Tronsmart Vega S95 Meta even
> > with
> > the patched PHY driver.
> 
> just an doubt, maybe useful, in the past, on GiGa setup I saw similar
> problems and it was due to retiming so maybe 2ns could be necessary
> (or better granularity via PAD logic if available).
> 
> Regards
> Peppe

Peppe, Martin,

With Andre's feedback, I think we can confirm that disabling EEE solve
the problem for the OdroidC2 design.

We do have the same results as Martin on MXQ-Pro based designs. For
these particular boards, disabling EEE does not seems to enough to get
a stable Tx path in 1000Base-T.

I will submit the patch for the Odroidc2 later today.
For the Vega, you should probably check the Tx delay as Peppe suggests.
To do these tests, It would probably be better to disable EEE as well.
Do you want me to include this change for the vega in the patch ?

Cheers
Jerome 

> 
> > 
> > Unfortunately I don't have a second device to rule out that my
> > Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> > errors from time to time in u-boot). Maybe Andreas Faerber can test
> > ethernet with and without Jerome's patch on one of his Tronsmart
> > devices.
> > 
> > 
> > Regards,
> > Martin
> > 
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-14 11:02                 ` Jerome Brunet
  0 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-14 11:02 UTC (permalink / raw)
  To: linus-amlogic

On Mon, 2016-11-14 at 08:47 +0100, Giuseppe CAVALLARO wrote:
> Hello Martin
> 
> On 11/7/2016 6:37 PM, Martin Blumenstingl wrote:
> > 
> > Hi Peppe,
> > 
> > On Mon, Nov 7, 2016 at 11:59 AM, Giuseppe CAVALLARO
> > <peppe.cavallaro@st.com> wrote:
> > > 
> > > In the meantime, I will read again the thread just to see if
> > > there is something I am missing.
> > if you are re-reading this thread: please note that there are two
> > devices in discussion here!
> 
> many thx for the sum :-)
> 
> > 
> > Both are using the Amlogic S905 (GXBB) SoC and both are
> > experiencing
> > the same issue (Gbit TX issues, RX with Gbit speeds and RX/TX with
> > 100Mbit speed are NOT affected):
> > - Odroid-C2 (used by Jerome and Andr? Roth)
> > - Tronsmart Vega S95 Meta (my device)
> > 
> > The (Gbit TX) problem seems to be gone on the Odroid-C2 with
> > Jerome's
> > patch which disables EEE in drivers/net/phy/realtek.c (at least in
> > his
> > tests, I don't have that device so I can't verify).
> > The same problem still appears on my Tronsmart Vega S95 Meta even
> > with
> > the patched PHY driver.
> 
> just an doubt, maybe useful, in the past, on GiGa setup I saw similar
> problems and it was due to retiming so maybe 2ns could be necessary
> (or better granularity via PAD logic if available).
> 
> Regards
> Peppe

Peppe, Martin,

With Andre's feedback, I think we can confirm that disabling EEE solve
the problem for the OdroidC2 design.

We do have the same results as Martin on MXQ-Pro based designs. For
these particular boards, disabling EEE does not seems to enough to get
a stable Tx path in 1000Base-T.

I will submit the patch for the Odroidc2 later today.
For the Vega, you should probably check the Tx delay as Peppe suggests.
To do these tests, It would probably be better to disable EEE as well.
Do you want me to include this change for the vega in the patch ?

Cheers
Jerome?

> 
> > 
> > Unfortunately I don't have a second device to rule out that my
> > Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> > errors from time to time in u-boot). Maybe Andreas Faerber can test
> > ethernet with and without Jerome's patch on one of his Tronsmart
> > devices.
> > 
> > 
> > Regards,
> > Martin
> > 
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-07 17:37             ` Martin Blumenstingl
@ 2016-11-14 15:00               ` Andreas Färber
  -1 siblings, 0 replies; 56+ messages in thread
From: Andreas Färber @ 2016-11-14 15:00 UTC (permalink / raw)
  To: Martin Blumenstingl
  Cc: Giuseppe CAVALLARO, Alexandre Torgue, netdev, André Roth,
	Johnson Leung, linux-amlogic, Jerome Brunet

Hi,

Am 07.11.2016 um 18:37 schrieb Martin Blumenstingl:
> The same problem still appears on my Tronsmart Vega S95 Meta even with
> the patched PHY driver.
> 
> Unfortunately I don't have a second device to rule out that my
> Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> errors from time to time in u-boot). Maybe Andreas Faerber can test
> ethernet with and without Jerome's patch on one of his Tronsmart
> devices.

So far I only ran into the stall issue on my Odroid-C2, not on my Vega
S95 Telos, both on v4.10/integ branch with a rebased MMC patch of yours
on top.

I'm assuming you mean realtek8211f-disable-eee-1000.patch from Nov 3.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-14 15:00               ` Andreas Färber
  0 siblings, 0 replies; 56+ messages in thread
From: Andreas Färber @ 2016-11-14 15:00 UTC (permalink / raw)
  To: linus-amlogic

Hi,

Am 07.11.2016 um 18:37 schrieb Martin Blumenstingl:
> The same problem still appears on my Tronsmart Vega S95 Meta even with
> the patched PHY driver.
> 
> Unfortunately I don't have a second device to rule out that my
> Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> errors from time to time in u-boot). Maybe Andreas Faerber can test
> ethernet with and without Jerome's patch on one of his Tronsmart
> devices.

So far I only ran into the stall issue on my Odroid-C2, not on my Vega
S95 Telos, both on v4.10/integ branch with a rebased MMC patch of yours
on top.

I'm assuming you mean realtek8211f-disable-eee-1000.patch from Nov 3.

Regards,
Andreas

-- 
SUSE Linux GmbH, Maxfeldstr. 5, 90409 N?rnberg, Germany
GF: Felix Imend?rffer, Jane Smithard, Graham Norton
HRB 21284 (AG N?rnberg)

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-14 15:00               ` Andreas Färber
@ 2016-11-15 11:27                 ` Jerome Brunet
  -1 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-15 11:27 UTC (permalink / raw)
  To: Andreas Färber, Martin Blumenstingl
  Cc: Giuseppe CAVALLARO, Alexandre Torgue, netdev, André Roth,
	Johnson Leung, linux-amlogic

On Mon, 2016-11-14 at 16:00 +0100, Andreas Färber wrote:
> Hi,
> 
> Am 07.11.2016 um 18:37 schrieb Martin Blumenstingl:
> > 
> > The same problem still appears on my Tronsmart Vega S95 Meta even
> > with
> > the patched PHY driver.
> > 
> > Unfortunately I don't have a second device to rule out that my
> > Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> > errors from time to time in u-boot). Maybe Andreas Faerber can test
> > ethernet with and without Jerome's patch on one of his Tronsmart
> > devices.
> 
> So far I only ran into the stall issue on my Odroid-C2, not on my
> Vega
> S95 Telos, both on v4.10/integ branch with a rebased MMC patch of
> yours
> on top.

The status of vega/mxq boards seems a bit confused.
This is the status I have so far:
These designs are not affected by the EEE issue. The patch mentioned
has no effect on these boards.
Some boards run fine at GBit speed (like yours) , some have terrible Tx
throughput (like mine and Martin's one). However, unlike on the
odroidc2, the link is not broken after the test, it is not necessary to
ifdown/ifup the interface to do another test.

I'm wondering if the vega/mxq designs all use the rtl8211f phy ?

> 
> I'm assuming you mean realtek8211f-disable-eee-1000.patch from Nov 3.
> 

Indeed. I won't submit anything for the vega since it has no effect.

> Regards,
> Andreas
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-15 11:27                 ` Jerome Brunet
  0 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-15 11:27 UTC (permalink / raw)
  To: linus-amlogic

On Mon, 2016-11-14 at 16:00 +0100, Andreas F?rber wrote:
> Hi,
> 
> Am 07.11.2016 um 18:37 schrieb Martin Blumenstingl:
> > 
> > The same problem still appears on my Tronsmart Vega S95 Meta even
> > with
> > the patched PHY driver.
> > 
> > Unfortunately I don't have a second device to rule out that my
> > Tronsmart Vega S95 Meta could be broken (not unlikely, I get DDR
> > errors from time to time in u-boot). Maybe Andreas Faerber can test
> > ethernet with and without Jerome's patch on one of his Tronsmart
> > devices.
> 
> So far I only ran into the stall issue on my Odroid-C2, not on my
> Vega
> S95 Telos, both on v4.10/integ branch with a rebased MMC patch of
> yours
> on top.

The status of vega/mxq boards seems a bit confused.
This is the status I have so far:
These designs are not affected by the EEE issue. The patch mentioned
has no effect on these boards.
Some boards run fine at GBit speed (like yours) , some have terrible Tx
throughput (like mine and Martin's one). However, unlike on the
odroidc2, the link is not broken after the test, it is not necessary to
ifdown/ifup the interface to do another test.

I'm wondering if the vega/mxq designs all use the rtl8211f phy ?

> 
> I'm assuming you mean realtek8211f-disable-eee-1000.patch from Nov 3.
> 

Indeed. I won't submit anything for the vega since it has no effect.

> Regards,
> Andreas
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-14 10:49             ` Jerome Brunet
@ 2016-11-17 18:44               ` André Roth
  -1 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-11-17 18:44 UTC (permalink / raw)
  To: Jerome Brunet
  Cc: Martin Blumenstingl, Johnson Leung, Giuseppe CAVALLARO,
	linux-amlogic, Alexandre Torgue, netdev


Hi all,

> I checked again the kernel
> at https://github.com/hardkernel/linux/tree/ odroidc2-3.14.y. The
> version you mention (3.14.65-73) seems to be:
> sha1: c75d5f4d1516cdd86d90a9d1c565bb0ed9251036 tag: jenkins-deb s905
> kernel-73

I downloaded the prebuilt image from hardkernel, I did not build the
kernel myself. but hardkernel has an earlier release of the same kernel
version, which works fine too. I assume they would have committed the
change in the newer version..
 
> In this particular version, both realtek drivers:
> - drivers/net/phy/realtek.c
> - drivers/amlogic/ethernet/phy/am_realtek.c
> 
> have the hack to disable 1000M advertisement. I don't understand how
> it possible for you to have 1000Base-T Full Duplex with this, maybe
> I'm missing something here ?

that's what I don't understand as well...

the patched kernel shows the following:

$ uname -a
Linux T-06 4.9.0-rc4+ #21 SMP PREEMPT Sun Nov 13 12:07:19 UTC 2016

$ sudo ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                             100baseT/Half
100baseT/Full 1000baseT/Half 1000baseT/Full 
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: ug
        Wake-on: d
        Current message level: 0x0000003f (63)
                               drv probe link timer ifdown ifup
        Link detected: yes

$ sudo ethtool --show-eee eth0
EEE Settings for eth0:
        EEE status: disabled
        Tx LPI: disabled
        Supported EEE link modes:  100baseT/Full 
                                   1000baseT/Full 
        Advertised EEE link modes:  100baseT/Full 
        Link partner advertised EEE link modes:  100baseT/Full 
                                                 1000baseT/Full 

can it be that "EEE link modes" and the "normal" link modes are two
different things ? 

> If you did compile the kernel yourself, could you check the 2 file
> mentioned above ? Just to be sure there was no patch applied at the
> last minute, which would not show up in the git history of
> hardkernel ?

I cannot check this easily at the moment..

Regards,

 André

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-17 18:44               ` André Roth
  0 siblings, 0 replies; 56+ messages in thread
From: André Roth @ 2016-11-17 18:44 UTC (permalink / raw)
  To: linus-amlogic


Hi all,

> I checked again the kernel
> at?https://github.com/hardkernel/linux/tree/ odroidc2-3.14.y. The
> version you mention (3.14.65-73) seems to be:
> sha1:?c75d5f4d1516cdd86d90a9d1c565bb0ed9251036 tag: jenkins-deb s905
> kernel-73

I downloaded the prebuilt image from hardkernel, I did not build the
kernel myself. but hardkernel has an earlier release of the same kernel
version, which works fine too. I assume they would have committed the
change in the newer version..
 
> In this particular version, both realtek drivers:
> - drivers/net/phy/realtek.c
> - drivers/amlogic/ethernet/phy/am_realtek.c
> 
> have the hack to disable 1000M advertisement. I don't understand how
> it possible for you to have 1000Base-T Full Duplex with this, maybe
> I'm missing something here ?

that's what I don't understand as well...

the patched kernel shows the following:

$ uname -a
Linux T-06 4.9.0-rc4+ #21 SMP PREEMPT Sun Nov 13 12:07:19 UTC 2016

$ sudo ethtool eth0
Settings for eth0:
        Supported ports: [ TP MII ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supported pause frame use: No
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Advertised pause frame use: No
        Advertised auto-negotiation: Yes
        Link partner advertised link modes:  10baseT/Half 10baseT/Full 
                                             100baseT/Half
100baseT/Full 1000baseT/Half 1000baseT/Full 
        Link partner advertised pause frame use: Symmetric Receive-only
        Link partner advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 0
        Transceiver: external
        Auto-negotiation: on
        Supports Wake-on: ug
        Wake-on: d
        Current message level: 0x0000003f (63)
                               drv probe link timer ifdown ifup
        Link detected: yes

$ sudo ethtool --show-eee eth0
EEE Settings for eth0:
        EEE status: disabled
        Tx LPI: disabled
        Supported EEE link modes:  100baseT/Full 
                                   1000baseT/Full 
        Advertised EEE link modes:  100baseT/Full 
        Link partner advertised EEE link modes:  100baseT/Full 
                                                 1000baseT/Full 

can it be that "EEE link modes" and the "normal" link modes are two
different things ? 

> If you did compile the kernel yourself, could you check the 2 file
> mentioned above ? Just to be sure there was no patch applied at the
> last minute, which would not show up in the git history of
> hardkernel ?

I cannot check this easily at the moment..

Regards,

 Andr?

^ permalink raw reply	[flat|nested] 56+ messages in thread

* Re: stmmac/RTL8211F/Meson GXBB: TX throughput problems
  2016-11-17 18:44               ` André Roth
@ 2016-11-17 21:47                 ` Jerome Brunet
  -1 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-17 21:47 UTC (permalink / raw)
  To: André Roth
  Cc: Martin Blumenstingl, Johnson Leung, Giuseppe CAVALLARO,
	linux-amlogic, Alexandre Torgue, netdev

On Thu, 2016-11-17 at 19:44 +0100, André Roth wrote:
> Hi all,
> 
> > 
> > I checked again the kernel
> > at https://github.com/hardkernel/linux/tree/ odroidc2-3.14.y. The
> > version you mention (3.14.65-73) seems to be:
> > sha1: c75d5f4d1516cdd86d90a9d1c565bb0ed9251036 tag: jenkins-deb
> > s905
> > kernel-73
> 
> I downloaded the prebuilt image from hardkernel, I did not build the
> kernel myself. but hardkernel has an earlier release of the same
> kernel
> version, which works fine too. I assume they would have committed the
> change in the newer version..
>  
> > 
> > In this particular version, both realtek drivers:
> > - drivers/net/phy/realtek.c
> > - drivers/amlogic/ethernet/phy/am_realtek.c
> > 
> > have the hack to disable 1000M advertisement. I don't understand
> > how
> > it possible for you to have 1000Base-T Full Duplex with this, maybe
> > I'm missing something here ?
> 
> that's what I don't understand as well...
> 
> the patched kernel shows the following:
> 
> $ uname -a
> Linux T-06 4.9.0-rc4+ #21 SMP PREEMPT Sun Nov 13 12:07:19 UTC 2016
> 
> $ sudo ethtool eth0
> Settings for eth0:
>         Supported ports: [ TP MII ]
>         Supported link modes:   10baseT/Half 10baseT/Full 
>                                 100baseT/Half 100baseT/Full 
>                                 1000baseT/Full 
>         Supported pause frame use: No
>         Supports auto-negotiation: Yes
>         Advertised link modes:  10baseT/Half 10baseT/Full 
>                                 100baseT/Half 100baseT/Full 
>                                 1000baseT/Full 
>         Advertised pause frame use: No
>         Advertised auto-negotiation: Yes
>         Link partner advertised link modes:  10baseT/Half
> 10baseT/Full 
>                                              100baseT/Half
> 100baseT/Full 1000baseT/Half 1000baseT/Full 
>         Link partner advertised pause frame use: Symmetric Receive-
> only
>         Link partner advertised auto-negotiation: Yes
>         Speed: 1000Mb/s
>         Duplex: Full
>         Port: MII
>         PHYAD: 0
>         Transceiver: external
>         Auto-negotiation: on
>         Supports Wake-on: ug
>         Wake-on: d
>         Current message level: 0x0000003f (63)
>                                drv probe link timer ifdown ifup
>         Link detected: yes
> 
> $ sudo ethtool --show-eee eth0
> EEE Settings for eth0:
>         EEE status: disabled
>         Tx LPI: disabled
>         Supported EEE link modes:  100baseT/Full 
>                                    1000baseT/Full 
>         Advertised EEE link modes:  100baseT/Full 
>         Link partner advertised EEE link modes:  100baseT/Full 
>                                                  1000baseT/Full 
> 
> can it be that "EEE link modes" and the "normal" link modes are two
> different things ? 

Exactly, They are.
Hardkernel code disable both.

With hardkernel's kernel, you should not have 1000baseT/Full in
"Advertised link modes" and you would have nothing reported in
"Advertised EEE link modes"

> 
> > 
> > If you did compile the kernel yourself, could you check the 2 file
> > mentioned above ? Just to be sure there was no patch applied at the
> > last minute, which would not show up in the git history of
> > hardkernel ?
> 
> I cannot check this easily at the moment..
> 
> Regards,
> 
>  André
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

* stmmac/RTL8211F/Meson GXBB: TX throughput problems
@ 2016-11-17 21:47                 ` Jerome Brunet
  0 siblings, 0 replies; 56+ messages in thread
From: Jerome Brunet @ 2016-11-17 21:47 UTC (permalink / raw)
  To: linus-amlogic

On Thu, 2016-11-17 at 19:44 +0100, Andr? Roth wrote:
> Hi all,
> 
> > 
> > I checked again the kernel
> > at?https://github.com/hardkernel/linux/tree/ odroidc2-3.14.y. The
> > version you mention (3.14.65-73) seems to be:
> > sha1:?c75d5f4d1516cdd86d90a9d1c565bb0ed9251036 tag: jenkins-deb
> > s905
> > kernel-73
> 
> I downloaded the prebuilt image from hardkernel, I did not build the
> kernel myself. but hardkernel has an earlier release of the same
> kernel
> version, which works fine too. I assume they would have committed the
> change in the newer version..
> ?
> > 
> > In this particular version, both realtek drivers:
> > - drivers/net/phy/realtek.c
> > - drivers/amlogic/ethernet/phy/am_realtek.c
> > 
> > have the hack to disable 1000M advertisement. I don't understand
> > how
> > it possible for you to have 1000Base-T Full Duplex with this, maybe
> > I'm missing something here ?
> 
> that's what I don't understand as well...
> 
> the patched kernel shows the following:
> 
> $ uname -a
> Linux T-06 4.9.0-rc4+ #21 SMP PREEMPT Sun Nov 13 12:07:19 UTC 2016
> 
> $ sudo ethtool eth0
> Settings for eth0:
> ????????Supported ports: [ TP MII ]
> ????????Supported link modes:???10baseT/Half 10baseT/Full?
> ????????????????????????????????100baseT/Half 100baseT/Full?
> ????????????????????????????????1000baseT/Full?
> ????????Supported pause frame use: No
> ????????Supports auto-negotiation: Yes
> ????????Advertised link modes:??10baseT/Half 10baseT/Full?
> ????????????????????????????????100baseT/Half 100baseT/Full?
> ????????????????????????????????1000baseT/Full?
> ????????Advertised pause frame use: No
> ????????Advertised auto-negotiation: Yes
> ????????Link partner advertised link modes:??10baseT/Half
> 10baseT/Full?
> ?????????????????????????????????????????????100baseT/Half
> 100baseT/Full 1000baseT/Half 1000baseT/Full?
> ????????Link partner advertised pause frame use: Symmetric Receive-
> only
> ????????Link partner advertised auto-negotiation: Yes
> ????????Speed: 1000Mb/s
> ????????Duplex: Full
> ????????Port: MII
> ????????PHYAD: 0
> ????????Transceiver: external
> ????????Auto-negotiation: on
> ????????Supports Wake-on: ug
> ????????Wake-on: d
> ????????Current message level: 0x0000003f (63)
> ???????????????????????????????drv probe link timer ifdown ifup
> ????????Link detected: yes
> 
> $ sudo ethtool --show-eee eth0
> EEE Settings for eth0:
> ????????EEE status: disabled
> ????????Tx LPI: disabled
> ????????Supported EEE link modes:??100baseT/Full?
> ???????????????????????????????????1000baseT/Full?
> ????????Advertised EEE link modes:??100baseT/Full?
> ????????Link partner advertised EEE link modes:??100baseT/Full?
> ?????????????????????????????????????????????????1000baseT/Full?
> 
> can it be that "EEE link modes" and the "normal" link modes are two
> different things ??

Exactly, They are.
Hardkernel code disable both.

With hardkernel's kernel, you should not have?1000baseT/Full in
"Advertised link modes" and you would have nothing reported in
"Advertised EEE link modes"

> 
> > 
> > If you did compile the kernel yourself, could you check the 2 file
> > mentioned above ? Just to be sure there was no patch applied at the
> > last minute, which would not show up in the git history of
> > hardkernel ?
> 
> I cannot check this easily at the moment..
> 
> Regards,
> 
> ?Andr?
> 

^ permalink raw reply	[flat|nested] 56+ messages in thread

end of thread, other threads:[~2016-11-17 21:47 UTC | newest]

Thread overview: 56+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-11 20:39 stmmac/RTL8211F/Meson GXBB: TX throughput problems Martin Blumenstingl
2016-09-11 20:39 ` Martin Blumenstingl
2016-09-12 16:37 ` Alexandre Torgue
2016-09-12 16:37   ` Alexandre Torgue
2016-09-12 21:26   ` Martin Blumenstingl
2016-09-12 21:26     ` Martin Blumenstingl
2016-09-17 21:23     ` André Roth
2016-09-17 21:23       ` André Roth
2016-09-26  6:17       ` Giuseppe CAVALLARO
2016-09-26  6:17         ` Giuseppe CAVALLARO
2016-10-01 15:58         ` Martin Blumenstingl
2016-10-01 15:58           ` Martin Blumenstingl
2016-11-03 16:36           ` Jerome Brunet
2016-11-03 16:36             ` Jerome Brunet
2016-11-05 12:20             ` Martin Blumenstingl
2016-11-05 12:20               ` Martin Blumenstingl
2016-11-13 19:20             ` André Roth
2016-11-13 19:20               ` André Roth
     [not found] ` <216F2694-1C1D-44DA-AC15-57ED15C24BBE@bluematt.me>
2016-09-12 21:29   ` Martin Blumenstingl
2016-09-12 21:29     ` Martin Blumenstingl
2016-09-14 15:30 ` Giuseppe CAVALLARO
2016-09-14 15:30   ` Giuseppe CAVALLARO
2016-09-18 20:42   ` André Roth
2016-09-18 20:42     ` André Roth
2016-09-19  5:38     ` Giuseppe CAVALLARO
2016-09-19  5:38       ` Giuseppe CAVALLARO
2016-10-01 20:15   ` Martin Blumenstingl
2016-10-01 20:15     ` Martin Blumenstingl
2016-10-01 20:40     ` Martin Blumenstingl
2016-10-01 20:40       ` Martin Blumenstingl
2016-10-31 10:25     ` André Roth
2016-10-31 10:25       ` André Roth
2016-11-03 16:57       ` Jerome Brunet
2016-11-03 16:57         ` Jerome Brunet
2016-11-07 10:59         ` Giuseppe CAVALLARO
2016-11-07 10:59           ` Giuseppe CAVALLARO
2016-11-07 17:37           ` Martin Blumenstingl
2016-11-07 17:37             ` Martin Blumenstingl
2016-11-14  7:47             ` Giuseppe CAVALLARO
2016-11-14  7:47               ` Giuseppe CAVALLARO
2016-11-14 11:02               ` Jerome Brunet
2016-11-14 11:02                 ` Jerome Brunet
2016-11-14 15:00             ` Andreas Färber
2016-11-14 15:00               ` Andreas Färber
2016-11-15 11:27               ` Jerome Brunet
2016-11-15 11:27                 ` Jerome Brunet
2016-11-13 19:13         ` André Roth
2016-11-13 19:13           ` André Roth
2016-11-14 10:49           ` Jerome Brunet
2016-11-14 10:49             ` Jerome Brunet
2016-11-17 18:44             ` André Roth
2016-11-17 18:44               ` André Roth
2016-11-17 21:47               ` Jerome Brunet
2016-11-17 21:47                 ` Jerome Brunet
2016-10-01 20:26   ` Martin Blumenstingl
2016-10-01 20:26     ` Martin Blumenstingl

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.