[RFC,PATCH 1/2] netdevice: Add netif_tx_lock_q

[RFC,PATCH 1/2] netdevice: Add netif_tx_lock_q

[David Wu]

It is similar to netif_tx_lock, but only locks one queue,
so it seems to be suitable for multiple queues.

If the current queue status is frozen, it will requeue skb,
which will not block the current thread, it improve performance
when tested.

Signed-off-by: David Wu <david.wu@rock-chips.com>
---
 include/linux/netdevice.h | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index b6fedd54cd8e..30054a94210e 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -4085,6 +4085,16 @@ static inline void netif_tx_lock(struct net_device *dev)
 	}
 }
 
+static inline void netif_tx_lock_q(struct netdev_queue *txq)
+{
+	int cpu;
+
+	cpu = smp_processor_id();
+	__netif_tx_lock(txq, cpu);
+	set_bit(__QUEUE_STATE_FROZEN, &txq->state);
+	__netif_tx_unlock(txq);
+}
+
 static inline void netif_tx_lock_bh(struct net_device *dev)
 {
 	local_bh_disable();
@@ -4108,6 +4118,12 @@ static inline void netif_tx_unlock(struct net_device *dev)
 	spin_unlock(&dev->tx_global_lock);
 }
 
+static inline void netif_tx_unlock_q(struct netdev_queue *txq)
+{
+	clear_bit(__QUEUE_STATE_FROZEN, &txq->state);
+	netif_schedule_queue(txq);
+}
+
 static inline void netif_tx_unlock_bh(struct net_device *dev)
 {
 	netif_tx_unlock(dev);
-- 
2.19.1

[RFC,PATCH 2/2] net: stmmac: Change the tx clean lock

[David Wu]

At tx clean, use a frozen queue instead of blocking
the current queue, could still queue skb, which improve
performance.

Signed-off-by: David Wu <david.wu@rock-chips.com>
---
 drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
index cb7a5bad4cfe..946058bcc9ed 100644
--- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
+++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c
@@ -1897,7 +1897,7 @@ static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue)
 	unsigned int bytes_compl = 0, pkts_compl = 0;
 	unsigned int entry, count = 0;
 
-	__netif_tx_lock_bh(netdev_get_tx_queue(priv->dev, queue));
+	netif_tx_lock_q(netdev_get_tx_queue(priv->dev, queue));
 
 	priv->xstats.tx_clean++;
 
@@ -1994,7 +1994,7 @@ static int stmmac_tx_clean(struct stmmac_priv *priv, int budget, u32 queue)
 	if (tx_q->dirty_tx != tx_q->cur_tx)
 		mod_timer(&tx_q->txtimer, STMMAC_COAL_TIMER(priv->tx_coal_timer));
 
-	__netif_tx_unlock_bh(netdev_get_tx_queue(priv->dev, queue));
+	netif_tx_unlock_q(netdev_get_tx_queue(priv->dev, queue));
 
 	return count;
 }
-- 
2.19.1

RE: [RFC,PATCH 2/2] net: stmmac: Change the tx clean lock

[Jose Abreu]

From: David Wu <david.wu@rock-chips.com>
Date: Mar/24/2020, 09:38:28 (UTC+00:00)

> At tx clean, use a frozen queue instead of blocking
> the current queue, could still queue skb, which improve
> performance.

Please provide performance improvement numbers.

---
Thanks,
Jose Miguel Abreu

Re: [RFC,PATCH 2/2] net: stmmac: Change the tx clean lock

[David Wu]

Hi Jose,

 From the test I did, there will be some improvement, an improvement of 
tens Mbits/sec.

Before patch:
# iperf -c 192.168.1.102 -i 1 -t 10 -w 300K -u -b 1000M
------------------------------------------------------------
Client connecting to 192.168.1.102, UDP port 5001
Sending 1470 byte datagrams, IPG target: 11.76 us (kalman adjust)
UDP buffer size:  600 KByte (WARNING: requested  300 KByte)
------------------------------------------------------------
[  3] local 192.168.1.103 port 45018 connected with 192.168.1.102 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   103 MBytes   862 Mbits/sec
[  3]  1.0- 2.0 sec   104 MBytes   868 Mbits/sec
[  3]  2.0- 3.0 sec   104 MBytes   869 Mbits/sec
[  3]  3.0- 4.0 sec   104 MBytes   869 Mbits/sec
[  3]  4.0- 5.0 sec   104 MBytes   870 Mbits/sec
[  3]  5.0- 6.0 sec   104 MBytes   869 Mbits/sec
[  3]  6.0- 7.0 sec   104 MBytes   869 Mbits/sec
[  3]  7.0- 8.0 sec   104 MBytes   870 Mbits/sec
[  3]  8.0- 9.0 sec   104 MBytes   871 Mbits/sec
[  3]  0.0-10.0 sec  1.01 GBytes   869 Mbits/sec
[  3] Sent 738834 datagrams


After patch:
# iperf -c 192.168.1.102 -i 1 -t 10 -w 300K -u -b 1000M
------------------------------------------------------------
Client connecting to 192.168.1.102, UDP port 5001
Sending 1470 byte datagrams, IPG target: 11.76 us (kalman adjust)
UDP buffer size:  600 KByte (WARNING: requested  300 KByte)
------------------------------------------------------------
[  3] local 192.168.1.103 port 35654 connected with 192.168.1.102 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0- 1.0 sec   114 MBytes   953 Mbits/sec
[  3]  1.0- 2.0 sec   114 MBytes   956 Mbits/sec
[  3]  2.0- 3.0 sec   115 MBytes   962 Mbits/sec
[  3]  3.0- 4.0 sec   114 MBytes   955 Mbits/sec
[  3]  4.0- 5.0 sec   114 MBytes   957 Mbits/sec
[  3]  5.0- 6.0 sec   114 MBytes   957 Mbits/sec
[  3]  6.0- 7.0 sec   114 MBytes   955 Mbits/sec
[  3]  7.0- 8.0 sec   114 MBytes   956 Mbits/sec
[  3]  8.0- 9.0 sec   114 MBytes   955 Mbits/sec
[  3]  0.0- 9.9 sec  1.10 GBytes   957 Mbits/sec
[  3] Sent 804442 datagrams


在 2020/4/13 下午10:31, Jose Abreu 写道:
> From: David Wu <david.wu@rock-chips.com>
> Date: Mar/24/2020, 09:38:28 (UTC+00:00)
> 
>> At tx clean, use a frozen queue instead of blocking
>> the current queue, could still queue skb, which improve
>> performance.
> 
> Please provide performance improvement numbers.
> 
> ---
> Thanks,
> Jose Miguel Abreu
> 
> 
> 
>