[PATCH] net: sort queues in xps maps

* [PATCH] net: sort queues in xps maps
@ 2022-07-13  2:24 Yonglong Li
  2022-07-14  2:07 ` Jakub Kicinski
  0 siblings, 1 reply; 5+ messages in thread
From: Yonglong Li @ 2022-07-13  2:24 UTC (permalink / raw)
  To: netdev; +Cc: davem, edumazet, kuba, pabeni, alexanderduyck, liyonglong

in the following case that set xps of each tx-queue with same cpu mask,
packets in the same tcp stream may be hash to different tx queue. Because
the order of queues in each xps map is not the same.

first set each tx-queue with different cpu mask
echo 0 > /sys/class/net/eth0/queues/tx-0
echo 1 > /sys/class/net/eth0/queues/tx-1
echo 2 > /sys/class/net/eth0/queues/tx-2
echo 4 > /sys/class/net/eth0/queues/tx-3
and then set each tx-queue with same cpu mask
echo f > /sys/class/net/eth0/queues/tx-0
echo f > /sys/class/net/eth0/queues/tx-1
echo f > /sys/class/net/eth0/queues/tx-2
echo f > /sys/class/net/eth0/queues/tx-3

at this point the order of each map queues is differnet, It will cause
packets in the same stream be hashed to diffetent tx queue:
attr_map[0].queues = [0,1,2,3]
attr_map[1].queues = [1,0,2,3]
attr_map[2].queues = [2,0,1,3]
attr_map[3].queues = [3,0,1,2]

It is more reasonable that pacekts in the same stream be hashed to the same
tx queue when all tx queue bind with the same CPUs.

Fixes: 537c00de1c9b ("net: Add functions netif_reset_xps_queue and netif_set_xps_queue")
Signed-off-by: Yonglong Li <liyonglong@chinatelecom.cn>
---
 net/core/dev.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/net/core/dev.c b/net/core/dev.c
index 978ed06..aab26b4 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -150,6 +150,7 @@
 #include <linux/pm_runtime.h>
 #include <linux/prandom.h>
 #include <linux/once_lite.h>
+#include <linux/sort.h>
 
 #include "dev.h"
 #include "net-sysfs.h"
@@ -199,6 +200,11 @@ static int call_netdevice_notifiers_extack(unsigned long val,
 
 static DECLARE_RWSEM(devnet_rename_sem);
 
+static int cmp_u16(const void *a, const void *b)
+{
+	return *(u16 *)a - *(u16 *)b;
+}
+
 static inline void dev_base_seq_inc(struct net *net)
 {
 	while (++net->dev_base_seq == 0)
@@ -2654,6 +2660,13 @@ int __netif_set_xps_queue(struct net_device *dev, const unsigned long *mask,
 					  skip_tc);
 	}
 
+	for (j = -1; j = netif_attrmask_next_and(j, online_mask, mask, nr_ids),
+	     j < nr_ids;) {
+		tci = j * num_tc + tc;
+		map = xmap_dereference(new_dev_maps->attr_map[tci]);
+		sort(map->queues, map->len, sizeof(u16), cmp_u16, NULL);
+	}
+
 	rcu_assign_pointer(dev->xps_maps[type], new_dev_maps);
 
 	/* Cleanup old maps */
-- 
1.8.3.1


^ permalink raw reply related	[flat|nested] 5+ messages in thread