* [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix @ 2019-12-26 19:01 Martin Blumenstingl 2019-12-26 19:01 ` [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs Martin Blumenstingl 2020-01-09 18:21 ` [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix patchwork-bot+linux-amlogic 0 siblings, 2 replies; 5+ messages in thread From: Martin Blumenstingl @ 2019-12-26 19:01 UTC (permalink / raw) To: andrew, f.fainelli, davem, netdev, linux-amlogic Cc: Martin Blumenstingl, linux-kernel, linux-arm-kernel The Ethernet TX performance has been historically bad on Meson8b and Meson8m2 SoCs because high packet loss was seen. Today I (presumably) found out why this is: the input clock (which feeds the RGMII TX clock) has to be at least 4 times 125MHz. With the fixed "divide by 2" in the clock tree this means that m250_div needs to be at least 2. With this patch and a 2ns TX delay generated by either the MAC *or* the PHY this results in improved Ethernet TX performance and no packet loss anymore: # iperf3 -c 192.168.1.100 Connecting to host 192.168.1.100, port 5201 [ 5] local 192.168.1.163 port 42636 connected to 192.168.1.100 port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 105 MBytes 878 Mbits/sec 0 609 KBytes [ 5] 1.00-2.00 sec 106 MBytes 885 Mbits/sec 0 683 KBytes [ 5] 2.00-3.09 sec 73.7 MBytes 570 Mbits/sec 0 683 KBytes [ 5] 3.09-4.00 sec 81.9 MBytes 754 Mbits/sec 0 795 KBytes [ 5] 4.00-5.00 sec 104 MBytes 869 Mbits/sec 0 877 KBytes [ 5] 5.00-6.00 sec 105 MBytes 878 Mbits/sec 0 877 KBytes [ 5] 6.00-7.00 sec 68.0 MBytes 571 Mbits/sec 0 877 KBytes [ 5] 7.00-8.00 sec 80.7 MBytes 676 Mbits/sec 0 877 KBytes [ 5] 8.00-9.01 sec 102 MBytes 853 Mbits/sec 0 877 KBytes [ 5] 9.01-10.00 sec 101 MBytes 859 Mbits/sec 0 877 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 927 MBytes 778 Mbits/sec 0 sender [ 5] 0.00-10.01 sec 927 MBytes 777 Mbits/sec receiver The .dts of these boards are still using an incorrect TX delay of 4ns. This will be fixed in follow-up patches when it's clear whether the MAC really generates an RX delay and how this can be configured. Changes since v1 [0]: - update cover-letter title - dropped the .dts patches after discussion with Andrew in [1] - slightly reworded the patch description of patch #1 to indicate that the goal is to fix the TX delay generated by the MAC, without suggesting that it's recommended to let the MAC actually generate it. [0] https://patchwork.kernel.org/cover/11309887/ [1] https://patchwork.kernel.org/patch/11309891/ Martin Blumenstingl (1): net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs .../net/ethernet/stmicro/stmmac/dwmac-meson8b.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) -- 2.24.1 _______________________________________________ linux-amlogic mailing list linux-amlogic@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-amlogic ^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs 2019-12-26 19:01 [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix Martin Blumenstingl @ 2019-12-26 19:01 ` Martin Blumenstingl 2019-12-26 21:36 ` Andrew Lunn 2019-12-28 0:37 ` David Miller 2020-01-09 18:21 ` [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix patchwork-bot+linux-amlogic 1 sibling, 2 replies; 5+ messages in thread From: Martin Blumenstingl @ 2019-12-26 19:01 UTC (permalink / raw) To: andrew, f.fainelli, davem, netdev, linux-amlogic Cc: Martin Blumenstingl, linux-kernel, linux-arm-kernel GXBB and newer SoCs use the fixed FCLK_DIV2 (1GHz) clock as input for the m250_sel clock. Meson8b and Meson8m2 use MPLL2 instead, whose rate can be adjusted at runtime. So far we have been running MPLL2 with ~250MHz (and the internal m250_div with value 1), which worked enough that we could transfer data with an TX delay of 4ns. Unfortunately there is high packet loss with an RGMII PHY when transferring data (receiving data works fine though). Odroid-C1's u-boot is running with a TX delay of only 2ns as well as the internal m250_div set to 2 - no lost (TX) packets can be observed with that setting in u-boot. Manual testing has shown that the TX packet loss goes away when using the following settings in Linux (the vendor kernel uses the same settings): - MPLL2 clock set to ~500MHz - m250_div set to 2 - TX delay set to 2ns on the MAC side Update the m250_div divider settings to only accept dividers greater or equal 2 to fix the TX delay generated by the MAC. iperf3 results before the change: [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 182 MBytes 153 Mbits/sec 514 sender [ 5] 0.00-10.00 sec 182 MBytes 152 Mbits/sec receiver iperf3 results after the change (including an updated TX delay of 2ns): [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-10.00 sec 927 MBytes 778 Mbits/sec 0 sender [ 5] 0.00-10.01 sec 927 MBytes 777 Mbits/sec receiver Fixes: 4f6a71b84e1afd ("net: stmmac: dwmac-meson8b: fix internal RGMII clock configuration") Signed-off-by: Martin Blumenstingl <martin.blumenstingl@googlemail.com> --- .../net/ethernet/stmicro/stmmac/dwmac-meson8b.c | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c index bd6c01004913..0e2fa14f1423 100644 --- a/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c +++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c @@ -112,6 +112,14 @@ static int meson8b_init_rgmii_tx_clk(struct meson8b_dwmac *dwmac) struct device *dev = dwmac->dev; const char *parent_name, *mux_parent_names[MUX_CLK_NUM_PARENTS]; struct meson8b_dwmac_clk_configs *clk_configs; + static const struct clk_div_table div_table[] = { + { .div = 2, .val = 2, }, + { .div = 3, .val = 3, }, + { .div = 4, .val = 4, }, + { .div = 5, .val = 5, }, + { .div = 6, .val = 6, }, + { .div = 7, .val = 7, }, + }; clk_configs = devm_kzalloc(dev, sizeof(*clk_configs), GFP_KERNEL); if (!clk_configs) @@ -146,9 +154,9 @@ static int meson8b_init_rgmii_tx_clk(struct meson8b_dwmac *dwmac) clk_configs->m250_div.reg = dwmac->regs + PRG_ETH0; clk_configs->m250_div.shift = PRG_ETH0_CLK_M250_DIV_SHIFT; clk_configs->m250_div.width = PRG_ETH0_CLK_M250_DIV_WIDTH; - clk_configs->m250_div.flags = CLK_DIVIDER_ONE_BASED | - CLK_DIVIDER_ALLOW_ZERO | - CLK_DIVIDER_ROUND_CLOSEST; + clk_configs->m250_div.table = div_table; + clk_configs->m250_div.flags = CLK_DIVIDER_ALLOW_ZERO | + CLK_DIVIDER_ROUND_CLOSEST; clk = meson8b_dwmac_register_clk(dwmac, "m250_div", &parent_name, 1, &clk_divider_ops, &clk_configs->m250_div.hw); -- 2.24.1 _______________________________________________ linux-amlogic mailing list linux-amlogic@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-amlogic ^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs 2019-12-26 19:01 ` [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs Martin Blumenstingl @ 2019-12-26 21:36 ` Andrew Lunn 2019-12-28 0:37 ` David Miller 1 sibling, 0 replies; 5+ messages in thread From: Andrew Lunn @ 2019-12-26 21:36 UTC (permalink / raw) To: Martin Blumenstingl Cc: f.fainelli, netdev, linux-kernel, linux-amlogic, davem, linux-arm-kernel On Thu, Dec 26, 2019 at 08:01:01PM +0100, Martin Blumenstingl wrote: > GXBB and newer SoCs use the fixed FCLK_DIV2 (1GHz) clock as input for > the m250_sel clock. Meson8b and Meson8m2 use MPLL2 instead, whose rate > can be adjusted at runtime. > > So far we have been running MPLL2 with ~250MHz (and the internal > m250_div with value 1), which worked enough that we could transfer data > with an TX delay of 4ns. Unfortunately there is high packet loss with > an RGMII PHY when transferring data (receiving data works fine though). > Odroid-C1's u-boot is running with a TX delay of only 2ns as well as > the internal m250_div set to 2 - no lost (TX) packets can be observed > with that setting in u-boot. > > Manual testing has shown that the TX packet loss goes away when using > the following settings in Linux (the vendor kernel uses the same > settings): > - MPLL2 clock set to ~500MHz > - m250_div set to 2 > - TX delay set to 2ns on the MAC side > > Update the m250_div divider settings to only accept dividers greater or > equal 2 to fix the TX delay generated by the MAC. > > iperf3 results before the change: > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 182 MBytes 153 Mbits/sec 514 sender > [ 5] 0.00-10.00 sec 182 MBytes 152 Mbits/sec receiver > > iperf3 results after the change (including an updated TX delay of 2ns): > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-10.00 sec 927 MBytes 778 Mbits/sec 0 sender > [ 5] 0.00-10.01 sec 927 MBytes 777 Mbits/sec receiver > > Fixes: 4f6a71b84e1afd ("net: stmmac: dwmac-meson8b: fix internal RGMII clock configuration") > Signed-off-by: Martin Blumenstingl <martin.blumenstingl@googlemail.com> Reviewed-by: Andrew Lunn <andrew@lunn.ch> Andrew _______________________________________________ linux-amlogic mailing list linux-amlogic@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-amlogic ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs 2019-12-26 19:01 ` [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs Martin Blumenstingl 2019-12-26 21:36 ` Andrew Lunn @ 2019-12-28 0:37 ` David Miller 1 sibling, 0 replies; 5+ messages in thread From: David Miller @ 2019-12-28 0:37 UTC (permalink / raw) To: martin.blumenstingl Cc: andrew, f.fainelli, netdev, linux-kernel, linux-amlogic, linux-arm-kernel From: Martin Blumenstingl <martin.blumenstingl@googlemail.com> Date: Thu, 26 Dec 2019 20:01:01 +0100 > GXBB and newer SoCs use the fixed FCLK_DIV2 (1GHz) clock as input for > the m250_sel clock. Meson8b and Meson8m2 use MPLL2 instead, whose rate > can be adjusted at runtime. > > So far we have been running MPLL2 with ~250MHz (and the internal > m250_div with value 1), which worked enough that we could transfer data > with an TX delay of 4ns. Unfortunately there is high packet loss with > an RGMII PHY when transferring data (receiving data works fine though). > Odroid-C1's u-boot is running with a TX delay of only 2ns as well as > the internal m250_div set to 2 - no lost (TX) packets can be observed > with that setting in u-boot. > > Manual testing has shown that the TX packet loss goes away when using > the following settings in Linux (the vendor kernel uses the same > settings): > - MPLL2 clock set to ~500MHz > - m250_div set to 2 > - TX delay set to 2ns on the MAC side > > Update the m250_div divider settings to only accept dividers greater or > equal 2 to fix the TX delay generated by the MAC. > > iperf3 results before the change: > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 182 MBytes 153 Mbits/sec 514 sender > [ 5] 0.00-10.00 sec 182 MBytes 152 Mbits/sec receiver > > iperf3 results after the change (including an updated TX delay of 2ns): > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-10.00 sec 927 MBytes 778 Mbits/sec 0 sender > [ 5] 0.00-10.01 sec 927 MBytes 777 Mbits/sec receiver > > Fixes: 4f6a71b84e1afd ("net: stmmac: dwmac-meson8b: fix internal RGMII clock configuration") > Signed-off-by: Martin Blumenstingl <martin.blumenstingl@googlemail.com> Applied and queued up for -stable, thanks. _______________________________________________ linux-amlogic mailing list linux-amlogic@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-amlogic ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix 2019-12-26 19:01 [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix Martin Blumenstingl 2019-12-26 19:01 ` [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs Martin Blumenstingl @ 2020-01-09 18:21 ` patchwork-bot+linux-amlogic 1 sibling, 0 replies; 5+ messages in thread From: patchwork-bot+linux-amlogic @ 2020-01-09 18:21 UTC (permalink / raw) To: Martin Blumenstingl; +Cc: linux-amlogic, khilman Hello: This patch was applied to khilman/linux-amlogic.git (refs/heads/for-next). On Thu, 26 Dec 2019 20:01:00 +0100 you wrote: > The Ethernet TX performance has been historically bad on Meson8b and > Meson8m2 SoCs because high packet loss was seen. Today I (presumably) > found out why this is: the input clock (which feeds the RGMII TX clock) > has to be at least 4 times 125MHz. With the fixed "divide by 2" in the > clock tree this means that m250_div needs to be at least 2. > > With this patch and a 2ns TX delay generated by either the MAC *or* the > PHY this results in improved Ethernet TX performance and no packet loss > anymore: > # iperf3 -c 192.168.1.100 > Connecting to host 192.168.1.100, port 5201 > [ 5] local 192.168.1.163 port 42636 connected to 192.168.1.100 port 5201 > [ ID] Interval Transfer Bitrate Retr Cwnd > [ 5] 0.00-1.00 sec 105 MBytes 878 Mbits/sec 0 609 KBytes > [ 5] 1.00-2.00 sec 106 MBytes 885 Mbits/sec 0 683 KBytes > [ 5] 2.00-3.09 sec 73.7 MBytes 570 Mbits/sec 0 683 KBytes > [ 5] 3.09-4.00 sec 81.9 MBytes 754 Mbits/sec 0 795 KBytes > [ 5] 4.00-5.00 sec 104 MBytes 869 Mbits/sec 0 877 KBytes > [ 5] 5.00-6.00 sec 105 MBytes 878 Mbits/sec 0 877 KBytes > [ 5] 6.00-7.00 sec 68.0 MBytes 571 Mbits/sec 0 877 KBytes > [ 5] 7.00-8.00 sec 80.7 MBytes 676 Mbits/sec 0 877 KBytes > [ 5] 8.00-9.01 sec 102 MBytes 853 Mbits/sec 0 877 KBytes > [ 5] 9.01-10.00 sec 101 MBytes 859 Mbits/sec 0 877 KBytes > - - - - - - - - - - - - - - - - - - - - - - - - - > [ ID] Interval Transfer Bitrate Retr > [ 5] 0.00-10.00 sec 927 MBytes 778 Mbits/sec 0 sender > [ 5] 0.00-10.01 sec 927 MBytes 777 Mbits/sec receiver > > [...] Here is a summary with links: - [v2,1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs https://git.kernel.org/khilman/linux-amlogic/c/bd6f48546b9cb7a785344fc78058c420923d7ed8 You are awesome, thank you! -- Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/pwbot _______________________________________________ linux-amlogic mailing list linux-amlogic@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-amlogic ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2020-01-09 18:21 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-12-26 19:01 [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix Martin Blumenstingl 2019-12-26 19:01 ` [PATCH v2 1/1] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on Meson8b/8m2 SoCs Martin Blumenstingl 2019-12-26 21:36 ` Andrew Lunn 2019-12-28 0:37 ` David Miller 2020-01-09 18:21 ` [PATCH v2 0/1] dwmac-meson8b: Ethernet RGMII TX delay fix patchwork-bot+linux-amlogic
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).