All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v3 net-next 0/3] tcp: add rx/tx cache to reduce lock contention
@ 2019-03-22 15:56 Eric Dumazet
  2019-03-22 15:56 ` [PATCH v3 net-next 1/3] net: convert rps_needed and rfs_needed to new static branch api Eric Dumazet
                   ` (4 more replies)
  0 siblings, 5 replies; 11+ messages in thread
From: Eric Dumazet @ 2019-03-22 15:56 UTC (permalink / raw)
  To: David S . Miller; +Cc: netdev, Eric Dumazet, Eric Dumazet

On hosts with many cpus we can observe a very serious contention
on spinlocks used in mm slab layer.

The following can happen quite often :

1) TX path
  sendmsg() allocates one (fclone) skb on CPU A, sends a clone.
  ACK is received on CPU B, and consumes the skb that was in the retransmit
  queue.

2) RX path
  network driver allocates skb on CPU C
  recvmsg() happens on CPU D, freeing the skb after it has been delivered
  to user space.

In both cases, we are hitting the asymetric alloc/free pattern
for which slab has to drain alien caches. At 8 Mpps per second,
this represents 16 Mpps alloc/free per second and has a huge penalty.

In an interesting experiment, I tried to use a single kmem_cache for all the skbs
(in skb_init() : skbuff_fclone_cache = skbuff_head_cache =
                  kmem_cache_create("skbuff_fclone_cache", sizeof(struct sk_buff_fclones),);
qnd most of the contention disappeared, since cpus could better use
their local slab per-cpu cache.

But we can do actually better, in the following patches.

TX : at ACK time, no longer free the skb but put it back in a tcp socket cache,
     so that next sendmsg() can reuse it immediately.

RX : at recvmsg() time, do not free the skb but put it in a tcp socket cache
   so that it can be freed by the cpu feeding the incoming packets in BH.

This increased the performance of small RPC benchmark by about 10 % on a host
with 112 hyperthreads.

v2 : - Solved a race condition : sk_stream_alloc_skb() to make sure the prior
       clone has been freed.
     - Really test rps_needed in sk_eat_skb() as claimed.
     - Fixed rps_needed use in drivers/net/tun.c

v3: Added a #ifdef CONFIG_RPS, to avoid compile error (kbuild robot)

Eric Dumazet (3):
  net: convert rps_needed and rfs_needed to new static branch api
  tcp: add one skb cache for tx
  tcp: add one skb cache for rx

 drivers/net/tun.c          |  2 +-
 include/linux/netdevice.h  |  4 +--
 include/net/sock.h         | 17 +++++++++++-
 net/core/dev.c             | 10 +++----
 net/core/net-sysfs.c       |  4 +--
 net/core/sysctl_net_core.c |  8 +++---
 net/ipv4/af_inet.c         |  4 +++
 net/ipv4/tcp.c             | 54 +++++++++++++++++++-------------------
 net/ipv4/tcp_ipv4.c        | 11 ++++++--
 net/ipv6/tcp_ipv6.c        | 12 ++++++---
 10 files changed, 79 insertions(+), 47 deletions(-)

-- 
2.21.0.392.gf8f6787159e-goog


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2019-04-12 14:43 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-22 15:56 [PATCH v3 net-next 0/3] tcp: add rx/tx cache to reduce lock contention Eric Dumazet
2019-03-22 15:56 ` [PATCH v3 net-next 1/3] net: convert rps_needed and rfs_needed to new static branch api Eric Dumazet
2019-03-22 15:56 ` [PATCH v3 net-next 2/3] tcp: add one skb cache for tx Eric Dumazet
2019-03-22 15:56 ` [PATCH v3 net-next 3/3] tcp: add one skb cache for rx Eric Dumazet
2019-04-03  1:17   ` Jakub Kicinski
2019-04-03  8:15     ` Eric Dumazet
2019-04-12 14:43   ` [tcp] 01b4c2aab8: lmbench3.TCP.socket.bandwidth.10MB.MB/sec -20.2% regression kernel test robot
2019-04-12 14:43     ` kernel test robot
2019-03-22 16:37 ` [PATCH v3 net-next 0/3] tcp: add rx/tx cache to reduce lock contention Tariq Toukan
2019-03-22 16:55   ` Eric Dumazet
2019-03-24  1:58 ` David Miller

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.