From: Eric Dumazet <edumazet@google.com>
To: "David S . Miller" <davem@davemloft.net>
Cc: netdev <netdev@vger.kernel.org>,
Eliezer Tamir <eliezer.tamir@linux.intel.com>,
Eli Cohen <eli@mellanox.com>,
Eugenia Emantayev <eugenia@mellanox.com>,
Ariel Elior <ariel.elior@qlogic.com>,
Willem de Bruijn <willemb@google.com>,
Rida Assaf <rida@google.com>, Eric Dumazet <edumazet@google.com>,
Eric Dumazet <eric.dumazet@gmail.com>
Subject: [PATCH v2 net-next 04/14] net: allow BH servicing in sk_busy_loop()
Date: Wed, 18 Nov 2015 06:30:53 -0800 [thread overview]
Message-ID: <1447857063-618-5-git-send-email-edumazet@google.com> (raw)
In-Reply-To: <1447857063-618-1-git-send-email-edumazet@google.com>
Instead of blocking BH in whole sk_busy_loop(), block them
only around ->ndo_busy_poll() calls.
This has many benefits.
1) allow tunneled traffic to use busy poll as well as native traffic.
Tunnels handlers usually call netif_rx() and depend on net_rx_action()
being run (from sofirq handler)
2) allow RFS/RPS being used (sending IPI to other cpus if needed)
3) use the 'lets burn cpu cycles' budget to do useful work
(like TX completions, timers, RCU callbacks...)
4) reduce BH latencies, making busy poll a better citizen.
Tested:
Tested with SIT tunnel
lpaa5:~# echo 0 >/proc/sys/net/core/busy_read
lpaa5:~# ./netperf -H 2002:af6:786::1 -t TCP_RR
MIGRATED TCP REQUEST/RESPONSE TEST from ::0 (::) port 0 AF_INET6 to 2002:af6:786::1 () port 0 AF_INET6 : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec
16384 87380 1 1 10.00 37373.93
16384 87380
Now enable busy poll on both hosts
lpaa5:~# echo 70 >/proc/sys/net/core/busy_read
lpaa6:~# echo 70 >/proc/sys/net/core/busy_read
lpaa5:~# ./netperf -H 2002:af6:786::1 -t TCP_RR
MIGRATED TCP REQUEST/RESPONSE TEST from ::0 (::) port 0 AF_INET6 to 2002:af6:786::1 () port 0 AF_INET6 : first burst 0
Local /Remote
Socket Size Request Resp. Elapsed Trans.
Send Recv Size Size Time Rate
bytes Bytes bytes bytes secs. per sec
16384 87380 1 1 10.00 58314.77
16384 87380
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
net/core/dev.c | 18 +++++++-----------
1 file changed, 7 insertions(+), 11 deletions(-)
diff --git a/net/core/dev.c b/net/core/dev.c
index 74a816b299df..2002eec2617d 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4684,11 +4684,7 @@ bool sk_busy_loop(struct sock *sk, int nonblock)
struct napi_struct *napi;
int rc = false;
- /*
- * rcu read lock for napi hash
- * bh so we don't race with net_rx_action
- */
- rcu_read_lock_bh();
+ rcu_read_lock();
napi = napi_by_id(sk->sk_napi_id);
if (!napi)
@@ -4699,23 +4695,23 @@ bool sk_busy_loop(struct sock *sk, int nonblock)
goto out;
do {
+ local_bh_disable();
rc = ops->ndo_busy_poll(napi);
+ if (rc > 0)
+ NET_ADD_STATS_BH(sock_net(sk),
+ LINUX_MIB_BUSYPOLLRXPACKETS, rc);
+ local_bh_enable();
if (rc == LL_FLUSH_FAILED)
break; /* permanent failure */
- if (rc > 0)
- /* local bh are disabled so it is ok to use _BH */
- NET_ADD_STATS_BH(sock_net(sk),
- LINUX_MIB_BUSYPOLLRXPACKETS, rc);
cpu_relax();
-
} while (!nonblock && skb_queue_empty(&sk->sk_receive_queue) &&
!need_resched() && !busy_loop_timeout(end_time));
rc = !skb_queue_empty(&sk->sk_receive_queue);
out:
- rcu_read_unlock_bh();
+ rcu_read_unlock();
return rc;
}
EXPORT_SYMBOL(sk_busy_loop);
--
2.6.0.rc2.230.g3dd15c0
next prev parent reply other threads:[~2015-11-18 14:31 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-11-18 14:30 [PATCH v2 net-next 00/14] net: extend busy polling support Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 01/14] net: better skb->sender_cpu and skb->napi_id cohabitation Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 02/14] mlx4: mlx4_en_low_latency_recv() called with BH disabled Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 03/14] net: un-inline sk_busy_loop() Eric Dumazet
2015-11-18 14:30 ` Eric Dumazet [this message]
2015-11-18 14:30 ` [PATCH v2 net-next 05/14] net: network drivers no longer need to implement ndo_busy_poll() Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 06/14] mlx5: add busy polling support Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 07/14] mlx5: support napi_complete_done() Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 08/14] bnx2x: remove bnx2x_low_latency_recv() support Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 09/14] mlx4: remove mlx4_en_low_latency_recv() Eric Dumazet
2015-11-18 14:30 ` [PATCH v2 net-next 10/14] net: move skb_mark_napi_id() into core networking stack Eric Dumazet
2015-11-18 14:31 ` [PATCH v2 net-next 11/14] net: add netif_tx_napi_add() Eric Dumazet
2015-11-18 14:31 ` [PATCH v2 net-next 12/14] net: move napi_hash[] into read mostly section Eric Dumazet
2015-11-18 14:31 ` [PATCH v2 net-next 13/14] net: napi_hash_del() returns a boolean status Eric Dumazet
2015-11-18 14:31 ` [PATCH v2 net-next 14/14] net: provide generic busy polling to all NAPI drivers Eric Dumazet
2015-11-18 21:18 ` [PATCH v2 net-next 00/14] net: extend busy polling support David Miller
2015-11-19 6:09 ` Or Gerlitz
2015-11-19 16:53 ` Eric Dumazet
2015-11-19 17:05 ` David Miller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1447857063-618-5-git-send-email-edumazet@google.com \
--to=edumazet@google.com \
--cc=ariel.elior@qlogic.com \
--cc=davem@davemloft.net \
--cc=eli@mellanox.com \
--cc=eliezer.tamir@linux.intel.com \
--cc=eric.dumazet@gmail.com \
--cc=eugenia@mellanox.com \
--cc=netdev@vger.kernel.org \
--cc=rida@google.com \
--cc=willemb@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.