* [PATCH net-next] net: bonding: Use per-cpu rr_tx_counter
@ 2021-06-15 8:54 Jussi Maki
2021-06-15 18:30 ` patchwork-bot+netdevbpf
0 siblings, 1 reply; 2+ messages in thread
From: Jussi Maki @ 2021-06-15 8:54 UTC (permalink / raw)
To: netdev; +Cc: j.vosburgh, Jussi Maki
The round-robin rr_tx_counter was shared across CPUs leading to
significant cache thrashing at high packet rates. This patch switches
the round-robin packet counter to use a per-cpu variable to decide
the destination slave.
On a test with 2x100Gbit ICE nic with pktgen_sample_04_many_flows.sh
(-s 64 -t 32) the tx rate was 19.6Mpps before and 22.3Mpps after
this patch.
"perf top -e cache_misses" before:
12.31% [bonding] [k] bond_xmit_roundrobin_slave_get
10.59% [sch_fq_codel] [k] fq_codel_dequeue
9.34% [kernel] [k] skb_release_data
after:
15.42% [sch_fq_codel] [k] fq_codel_dequeue
10.06% [kernel] [k] __memset
9.12% [kernel] [k] skb_release_data
Signed-off-by: Jussi Maki <joamaki@gmail.com>
---
drivers/net/bonding/bond_main.c | 18 +++++++++++++++---
include/net/bonding.h | 2 +-
2 files changed, 16 insertions(+), 4 deletions(-)
diff --git a/drivers/net/bonding/bond_main.c b/drivers/net/bonding/bond_main.c
index eb79a9f05914..1d9137e77dfc 100644
--- a/drivers/net/bonding/bond_main.c
+++ b/drivers/net/bonding/bond_main.c
@@ -4202,16 +4202,16 @@ static u32 bond_rr_gen_slave_id(struct bonding *bond)
slave_id = prandom_u32();
break;
case 1:
- slave_id = bond->rr_tx_counter;
+ slave_id = this_cpu_inc_return(*bond->rr_tx_counter);
break;
default:
reciprocal_packets_per_slave =
bond->params.reciprocal_packets_per_slave;
- slave_id = reciprocal_divide(bond->rr_tx_counter,
+ slave_id = this_cpu_inc_return(*bond->rr_tx_counter);
+ slave_id = reciprocal_divide(slave_id,
reciprocal_packets_per_slave);
break;
}
- bond->rr_tx_counter++;
return slave_id;
}
@@ -4852,6 +4852,9 @@ static void bond_destructor(struct net_device *bond_dev)
if (bond->wq)
destroy_workqueue(bond->wq);
+
+ if (bond->rr_tx_counter)
+ free_percpu(bond->rr_tx_counter);
}
void bond_setup(struct net_device *bond_dev)
@@ -5350,6 +5353,15 @@ static int bond_init(struct net_device *bond_dev)
if (!bond->wq)
return -ENOMEM;
+ if (BOND_MODE(bond) == BOND_MODE_ROUNDROBIN) {
+ bond->rr_tx_counter = alloc_percpu(u32);
+ if (!bond->rr_tx_counter) {
+ destroy_workqueue(bond->wq);
+ bond->wq = NULL;
+ return -ENOMEM;
+ }
+ }
+
spin_lock_init(&bond->stats_lock);
netdev_lockdep_set_classes(bond_dev);
diff --git a/include/net/bonding.h b/include/net/bonding.h
index 019e998d944a..15335732e166 100644
--- a/include/net/bonding.h
+++ b/include/net/bonding.h
@@ -232,7 +232,7 @@ struct bonding {
char proc_file_name[IFNAMSIZ];
#endif /* CONFIG_PROC_FS */
struct list_head bond_list;
- u32 rr_tx_counter;
+ u32 __percpu *rr_tx_counter;
struct ad_bond_info ad_info;
struct alb_bond_info alb_info;
struct bond_params params;
--
2.30.2
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [PATCH net-next] net: bonding: Use per-cpu rr_tx_counter
2021-06-15 8:54 [PATCH net-next] net: bonding: Use per-cpu rr_tx_counter Jussi Maki
@ 2021-06-15 18:30 ` patchwork-bot+netdevbpf
0 siblings, 0 replies; 2+ messages in thread
From: patchwork-bot+netdevbpf @ 2021-06-15 18:30 UTC (permalink / raw)
To: Jussi Maki; +Cc: netdev, j.vosburgh
Hello:
This patch was applied to netdev/net-next.git (refs/heads/master):
On Tue, 15 Jun 2021 08:54:15 +0000 you wrote:
> The round-robin rr_tx_counter was shared across CPUs leading to
> significant cache thrashing at high packet rates. This patch switches
> the round-robin packet counter to use a per-cpu variable to decide
> the destination slave.
>
> On a test with 2x100Gbit ICE nic with pktgen_sample_04_many_flows.sh
> (-s 64 -t 32) the tx rate was 19.6Mpps before and 22.3Mpps after
> this patch.
>
> [...]
Here is the summary with links:
- [net-next] net: bonding: Use per-cpu rr_tx_counter
https://git.kernel.org/netdev/net-next/c/848ca9182a7d
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-06-15 18:30 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-15 8:54 [PATCH net-next] net: bonding: Use per-cpu rr_tx_counter Jussi Maki
2021-06-15 18:30 ` patchwork-bot+netdevbpf
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).