From: Davidlohr Bueso <dave@stgolabs.net>
To: akpm@linux-foundation.org
Cc: jbaron@akamai.com, viro@zeniv.linux.org.uk, dave@stgolabs.net,
linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
Davidlohr Bueso <dbueso@suse.de>
Subject: [PATCH 3/6] fs/epoll: drop ovflist branch prediction
Date: Wed, 7 Nov 2018 21:10:03 -0800 [thread overview]
Message-ID: <20181108051006.18751-4-dave@stgolabs.net> (raw)
In-Reply-To: <20181108051006.18751-1-dave@stgolabs.net>
The ep->ovflist is a secondary ready-list to temporarily store
events that might occur when doing sproc without holding the
ep->wq.lock. This accounts for every time we check for ready
events and also send events back to userspace; both callbacks,
particularly the later because of copy_to_user, can account
for a non-trivial time.
As such, the unlikely() check to see if the pointer is being
used, seems both misleading and sub-optimal. In fact, we go
to an awful lot of trouble to sync both lists, and populating
the ovflist is far from an uncommon scenario.
For example, profiling a concurrent epoll_wait(2) benchmark,
with CONFIG_PROFILE_ANNOTATED_BRANCHES shows that for a two threads
a 33% incorrect rate was seen; and when incrementally increasing
the number of epoll instances (which is used, for example for
multiple queuing load balancing models), up to a 90% incorrect
rate was seen.
Similarly, by deleting the prediction, 3% throughput boost was
seen across incremental threads.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
fs/eventpoll.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 101d46b81f64..347da3f4f5d3 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -1153,7 +1153,7 @@ static int ep_poll_callback(wait_queue_entry_t *wait, unsigned mode, int sync, v
* semantics). All the events that happen during that period of time are
* chained in ep->ovflist and requeued later on.
*/
- if (unlikely(ep->ovflist != EP_UNACTIVE_PTR)) {
+ if (ep->ovflist != EP_UNACTIVE_PTR) {
if (epi->next == EP_UNACTIVE_PTR) {
epi->next = ep->ovflist;
ep->ovflist = epi;
--
2.16.4
next prev parent reply other threads:[~2018-11-08 5:10 UTC|newest]
Thread overview: 8+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-08 5:10 [PATCH -next 0/6] epoll: some miscellaneous optimizations Davidlohr Bueso
2018-11-08 5:10 ` [PATCH 1/6] fs/epoll: remove max_nests argument from ep_call_nested() Davidlohr Bueso
2018-11-08 5:10 ` [PATCH 2/6] fs/epoll: simplify ep_send_events_proc() ready-list loop Davidlohr Bueso
2018-11-08 5:10 ` Davidlohr Bueso [this message]
2018-11-08 5:10 ` [PATCH 4/6] fs/epoll: robustify ep->mtx held checks Davidlohr Bueso
2018-11-08 5:10 ` [PATCH 5/6] fs/epoll: reduce the scope of wq lock in epoll_wait() Davidlohr Bueso
2018-11-09 15:52 ` Davidlohr Bueso
2018-11-08 5:10 ` [PATCH 6/6] fs/epoll: avoid barrier after an epoll_wait(2) timeout Davidlohr Bueso
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181108051006.18751-4-dave@stgolabs.net \
--to=dave@stgolabs.net \
--cc=akpm@linux-foundation.org \
--cc=dbueso@suse.de \
--cc=jbaron@akamai.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).