linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Roman Penyaev <rpenyaev@suse.de>
To: unlisted-recipients:; (no To-header on input)
Cc: Roman Penyaev <rpenyaev@suse.de>,
	Andrew Morton <akpm@linux-foundation.org>,
	Davidlohr Bueso <dbueso@suse.de>, Jason Baron <jbaron@akamai.com>,
	Al Viro <viro@zeniv.linux.org.uk>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Linus Torvalds <torvalds@linux-foundation.org>,
	Andrea Parri <andrea.parri@amarulasolutions.com>,
	linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [RFC PATCH v2 07/13] epoll: call ep_add_event_to_uring() from ep_poll_callback()
Date: Mon, 21 Jan 2019 21:14:50 +0100	[thread overview]
Message-ID: <20190121201456.28338-8-rpenyaev@suse.de> (raw)
In-Reply-To: <20190121201456.28338-1-rpenyaev@suse.de>

Each ep_poll_callback() is called when fd calls wakeup() on epfd.
So account new event in user ring.

The tricky part here is EPOLLONESHOT.  Since we are lockless we
have to be deal with ep_poll_callbacks() called in parallel, thus
use cmpxchg to clear public event bits and filter out concurrent
call from another cpu.

Signed-off-by: Roman Penyaev <rpenyaev@suse.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Davidlohr Bueso <dbueso@suse.de>
Cc: Jason Baron <jbaron@akamai.com>
Cc: Al Viro <viro@zeniv.linux.org.uk>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andrea Parri <andrea.parri@amarulasolutions.com>
Cc: linux-fsdevel@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
---
 fs/eventpoll.c | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/fs/eventpoll.c b/fs/eventpoll.c
index 26d837252ba4..1d0039b334b8 100644
--- a/fs/eventpoll.c
+++ b/fs/eventpoll.c
@@ -1406,6 +1406,29 @@ struct file *get_epoll_tfile_raw_ptr(struct file *file, int tfd,
 }
 #endif /* CONFIG_CHECKPOINT_RESTORE */
 
+/**
+ * Atomically clear public event bits and return %true if the old value has
+ * public event bits set.
+ */
+static inline bool ep_clear_public_event_bits(struct epitem *epi)
+{
+	__poll_t old, flags;
+
+	/*
+	 * Here we race with ourselves and with ep_modify(), which can
+	 * change the event bits.  In order not to override events updated
+	 * by ep_modify() we have to do cmpxchg.
+	 */
+
+	old = epi->event.events;
+	do {
+		flags = old;
+	} while ((old = cmpxchg(&epi->event.events, flags,
+				flags & EP_PRIVATE_BITS)) != flags);
+
+	return flags & ~EP_PRIVATE_BITS;
+}
+
 /**
  * Adds a new entry to the tail of the list in a lockless way, i.e.
  * multiple CPUs are allowed to call this function concurrently.
@@ -1525,6 +1548,20 @@ static int ep_poll_callback(struct epitem *epi, __poll_t pollflags)
 	if (pollflags && !(pollflags & epi->event.events))
 		goto out_unlock;
 
+	if (ep_polled_by_user(ep)) {
+		/*
+		 * For polled descriptor from user we have to disable events on
+		 * callback path in case of one-shot.
+		 */
+		if ((epi->event.events & EPOLLONESHOT) &&
+		    !ep_clear_public_event_bits(epi))
+			/* Race is lost, another callback has cleared events */
+			goto out_unlock;
+
+		ep_add_event_to_uring(epi, pollflags);
+		goto wakeup;
+	}
+
 	/*
 	 * If we are transferring events to userspace, we can hold no locks
 	 * (because we're accessing user memory, and because of linux f_op->poll()
@@ -1544,6 +1581,7 @@ static int ep_poll_callback(struct epitem *epi, __poll_t pollflags)
 		ep_pm_stay_awake_rcu(epi);
 	}
 
+wakeup:
 	/*
 	 * Wake up ( if active ) both the eventpoll wait list and the ->poll()
 	 * wait list.
-- 
2.19.1


  parent reply	other threads:[~2019-01-21 20:16 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-01-21 20:14 [RFC PATCH v2 00/13] epoll: support pollable epoll from userspace Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 01/13] epoll: move private helpers from a header to the source Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 02/13] epoll: introduce user structures for polling from userspace Roman Penyaev
2019-01-21 21:34   ` Linus Torvalds
2019-01-22 11:46     ` Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 03/13] epoll: allocate user header and user events ring " Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 04/13] epoll: some sanity flags checks for epoll syscalls " Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 05/13] epoll: offload polling to a work in case of epfd polled " Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 06/13] epoll: introduce helpers for adding/removing events to uring Roman Penyaev
2019-01-21 20:14 ` Roman Penyaev [this message]
2019-01-21 20:14 ` [RFC PATCH v2 08/13] epoll: support polling from userspace for ep_insert() Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 09/13] epoll: support polling from userspace for ep_remove() Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 10/13] epoll: support polling from userspace for ep_modify() Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 11/13] epoll: support polling from userspace for ep_poll() Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 12/13] epoll: support mapping for epfd when polled from userspace Roman Penyaev
2019-01-21 20:14 ` [RFC PATCH v2 13/13] epoll: implement epoll_create2() syscall Roman Penyaev

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190121201456.28338-8-rpenyaev@suse.de \
    --to=rpenyaev@suse.de \
    --cc=akpm@linux-foundation.org \
    --cc=andrea.parri@amarulasolutions.com \
    --cc=dbueso@suse.de \
    --cc=jbaron@akamai.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=torvalds@linux-foundation.org \
    --cc=viro@zeniv.linux.org.uk \
    --subject='Re: [RFC PATCH v2 07/13] epoll: call ep_add_event_to_uring() from ep_poll_callback()' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).