Re: Synopsys Ethernet QoS

From: Niklas Cassel <niklas.cassel@axis.com>
To: Pavel Machek <pavel@ucw.cz>
Cc: Giuseppe CAVALLARO <peppe.cavallaro@st.com>,
	Joao Pinto <Joao.Pinto@synopsys.com>,
	Florian Fainelli <f.fainelli@gmail.com>,
	Andy Shevchenko <andy.shevchenko@gmail.com>,
	David Miller <davem@davemloft.net>, <larper@axis.com>,
	<rabinv@axis.com>, netdev <netdev@vger.kernel.org>,
	<CARLOS.PALMINHA@synopsys.com>, <Jie.Deng1@synopsys.com>,
	Stephen Warren <swarren@nvidia.com>
Subject: Re: Synopsys Ethernet QoS
Date: Mon, 19 Dec 2016 18:58:05 +0100	[thread overview]
Message-ID: <06d47014-5533-e761-2674-17f5688fb13c@axis.com> (raw)
In-Reply-To: <20161214125735.GA19542@amd>

On 12/14/2016 01:57 PM, Pavel Machek wrote:
> Hi!
>
>> So if there is a long time before handling interrupts,
>> I guess that it makes sense that one stream could
>> get an advantage in the net scheduler.
>>
>> If I find the time, and if no one beats me to it, I will try to replace
>> the normal timers with HR timers + a smaller default timeout.
>>
> Can you try something like this? Highres timers will be needed, too,
> but this fixes the logic problem.
>
> You'll need to apply it twice as code is copy&pasted.
>
> Best regards,

Hello Pavel

If I first apply your other fix "stmmac: fix memory barriers",
then I can apply this fix without getting a netdev watchdog timeout on tx queue0,
and everything appears to work as it should.

Regarding to fairness, I can't really see a difference, with or without your patch.
I've noticed that for TX, the streams actually stabilize after 5 seconds or so,
but with the default test length of 10 seconds, it's easy to get confused
by the test result summary. So I guess from a fairness point of view,
TX is not really a problem.

For RX fairness, it is very much a real issue. The streams never stabilize.
One key difference is that RX coalescing is implemented by using the
RX watchdog.
Here is an iperf3 run that went for 30 seconds:

[  4]   0.00-30.00  sec  1.54 GBytes   440 Mbits/sec    0             sender
[  4]   0.00-30.00  sec  1.54 GBytes   440 Mbits/sec                  receiver
[  6]   0.00-30.00  sec   600 MBytes   168 Mbits/sec    0             sender
[  6]   0.00-30.00  sec   599 MBytes   168 Mbits/sec                  receiver
[  8]   0.00-30.00  sec  1.17 GBytes   334 Mbits/sec    0             sender
[  8]   0.00-30.00  sec  1.17 GBytes   334 Mbits/sec                  receiver
[SUM]   0.00-30.00  sec  3.29 GBytes   942 Mbits/sec    0             sender
[SUM]   0.00-30.00  sec  3.29 GBytes   942 Mbits/sec                  receiver

> 									Pavel
>
> +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
>
>  	 */
>  	priv->tx_count_frames += nfrags + 1;
>  	if (likely(priv->tx_coal_frames > priv->tx_count_frames)) {
> -		mod_timer(&priv->txtimer,
> -			  STMMAC_COAL_TIMER(priv->tx_coal_timer));
> +		if (priv->tx_count_frames == nfrags + 1)
> +			mod_timer(&priv->txtimer,
> +				  STMMAC_COAL_TIMER(priv->tx_coal_timer));
>  	} else {
>  		priv->tx_count_frames = 0;
>  		priv->hw->desc->set_tx_ic(desc);
>
>