From: Pavel Begunkov <asml.silence@gmail.com>
To: io-uring@vger.kernel.org
Cc: Jens Axboe <axboe@kernel.dk>, Andres Freund <andres@anarazel.de>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>,
Peter Zijlstra <peterz@infradead.org>,
Darren Hart <dvhart@infradead.org>,
Davidlohr Bueso <dave@stgolabs.net>,
linux-kernel@vger.kernel.org
Subject: [RFC 1/4] futex: add op wake for a single key
Date: Tue, 1 Jun 2021 15:58:26 +0100 [thread overview]
Message-ID: <2fded39d933cb51f0992fd51416ddef5f0e81493.1622558659.git.asml.silence@gmail.com> (raw)
In-Reply-To: <cover.1622558659.git.asml.silence@gmail.com>
Add a new futex wake function futex_wake_op_single(), which works
similar to futex_wake_op() but only for a single futex address as it
takes too many arguments. Also export it and other functions that will
be used by io_uring.
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
include/linux/futex.h | 15 ++++++++++
kernel/futex.c | 64 +++++++++++++++++++++++++++++++++++++++++--
2 files changed, 77 insertions(+), 2 deletions(-)
diff --git a/include/linux/futex.h b/include/linux/futex.h
index b70df27d7e85..04d500ae5983 100644
--- a/include/linux/futex.h
+++ b/include/linux/futex.h
@@ -77,6 +77,10 @@ void futex_exec_release(struct task_struct *tsk);
long do_futex(u32 __user *uaddr, int op, u32 val, ktime_t *timeout,
u32 __user *uaddr2, u32 val2, u32 val3);
+int futex_wake_op_single(u32 __user *uaddr, int nr_wake, unsigned int op,
+ bool shared, bool try);
+int futex_wait(u32 __user *uaddr, unsigned int flags, u32 val,
+ ktime_t *abs_time, u32 bitset);
#else
static inline void futex_init_task(struct task_struct *tsk) { }
static inline void futex_exit_recursive(struct task_struct *tsk) { }
@@ -88,6 +92,17 @@ static inline long do_futex(u32 __user *uaddr, int op, u32 val,
{
return -EINVAL;
}
+static inline int futex_wake_op_single(u32 __user *uaddr, int nr_wake,
+ unsigned int op, bool shared, bool try)
+{
+ return -EINVAL;
+}
+static inline int futex_wait(u32 __user *uaddr, unsigned int flags, u32 val,
+ ktime_t *abs_time, u32 bitset)
+{
+ return -EINVAL;
+}
+
#endif
#endif
diff --git a/kernel/futex.c b/kernel/futex.c
index 4938a00bc785..75dc600062a4 100644
--- a/kernel/futex.c
+++ b/kernel/futex.c
@@ -1681,6 +1681,66 @@ static int futex_atomic_op_inuser(unsigned int encoded_op, u32 __user *uaddr)
}
}
+int futex_wake_op_single(u32 __user *uaddr, int nr_wake, unsigned int op,
+ bool shared, bool try)
+{
+ union futex_key key;
+ struct futex_hash_bucket *hb;
+ struct futex_q *this, *next;
+ int ret, op_ret;
+ DEFINE_WAKE_Q(wake_q);
+
+retry:
+ ret = get_futex_key(uaddr, shared, &key, FUTEX_WRITE);
+ if (unlikely(ret != 0))
+ return ret;
+ hb = hash_futex(&key);
+retry_private:
+ spin_lock(&hb->lock);
+ op_ret = futex_atomic_op_inuser(op, uaddr);
+ if (unlikely(op_ret < 0)) {
+ spin_unlock(&hb->lock);
+
+ if (!IS_ENABLED(CONFIG_MMU) ||
+ unlikely(op_ret != -EFAULT && op_ret != -EAGAIN)) {
+ /*
+ * we don't get EFAULT from MMU faults if we don't have
+ * an MMU, but we might get them from range checking
+ */
+ ret = op_ret;
+ return ret;
+ }
+ if (try)
+ return -EAGAIN;
+
+ if (op_ret == -EFAULT) {
+ ret = fault_in_user_writeable(uaddr);
+ if (ret)
+ return ret;
+ }
+ cond_resched();
+ if (shared)
+ goto retry;
+ goto retry_private;
+ }
+ if (op_ret) {
+ plist_for_each_entry_safe(this, next, &hb->chain, list) {
+ if (match_futex(&this->key, &key)) {
+ if (this->pi_state || this->rt_waiter) {
+ ret = -EINVAL;
+ break;
+ }
+ mark_wake_futex(&wake_q, this);
+ if (++ret >= nr_wake)
+ break;
+ }
+ }
+ }
+ spin_unlock(&hb->lock);
+ wake_up_q(&wake_q);
+ return ret;
+}
+
/*
* Wake up all waiters hashed on the physical page that is mapped
* to this virtual address:
@@ -2680,8 +2740,8 @@ static int futex_wait_setup(u32 __user *uaddr, u32 val, unsigned int flags,
return ret;
}
-static int futex_wait(u32 __user *uaddr, unsigned int flags, u32 val,
- ktime_t *abs_time, u32 bitset)
+int futex_wait(u32 __user *uaddr, unsigned int flags, u32 val,
+ ktime_t *abs_time, u32 bitset)
{
struct hrtimer_sleeper timeout, *to;
struct restart_block *restart;
--
2.31.1
next prev parent reply other threads:[~2021-06-01 14:59 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-01 14:58 [RFC 0/4] futex request support Pavel Begunkov
2021-06-01 14:58 ` Pavel Begunkov [this message]
2021-06-01 14:58 ` [RFC 2/4] io_uring: frame out futex op Pavel Begunkov
2021-06-01 14:58 ` [RFC 3/4] io_uring: support futex wake requests Pavel Begunkov
2021-06-01 14:58 ` [RFC 4/4] io_uring: implement futex wait Pavel Begunkov
2021-06-01 15:45 ` Jens Axboe
2021-06-01 15:58 ` Pavel Begunkov
2021-06-01 16:01 ` Jens Axboe
2021-06-01 16:29 ` Pavel Begunkov
2021-06-01 21:53 ` Thomas Gleixner
2021-06-03 10:31 ` Pavel Begunkov
2021-06-04 9:19 ` Thomas Gleixner
2021-06-04 11:58 ` Pavel Begunkov
2021-06-05 2:09 ` Thomas Gleixner
2021-06-07 12:14 ` Pavel Begunkov
2021-06-03 19:03 ` Andres Freund
2021-06-03 21:10 ` Peter Zijlstra
2021-06-03 21:21 ` Andres Freund
2021-06-05 0:43 ` Thomas Gleixner
2021-06-07 11:31 ` Pavel Begunkov
2021-06-07 11:48 ` Peter Zijlstra
2021-06-03 18:59 ` [RFC 0/4] futex request support Andres Freund
2021-06-04 15:26 ` Pavel Begunkov
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2fded39d933cb51f0992fd51416ddef5f0e81493.1622558659.git.asml.silence@gmail.com \
--to=asml.silence@gmail.com \
--cc=andres@anarazel.de \
--cc=axboe@kernel.dk \
--cc=dave@stgolabs.net \
--cc=dvhart@infradead.org \
--cc=io-uring@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).