All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v4 00/15] Add futex2 syscalls
@ 2021-06-03 19:59 André Almeida
  2021-06-03 19:59 ` [PATCH v4 01/15] futex2: Implement wait and wake functions André Almeida
                   ` (16 more replies)
  0 siblings, 17 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Hi,

This patch series introduces the futex2 syscalls.

* What happened to the current futex()?

For some years now, developers have been trying to add new features to
futex, but maintainers have been reluctant to accept then, given the
multiplexed interface full of legacy features and tricky to do big
changes. Some problems that people tried to address with patchsets are:
NUMA-awareness[0], smaller sized futexes[1], wait on multiple futexes[2].
NUMA, for instance, just doesn't fit the current API in a reasonable
way. Considering that, it's not possible to merge new features into the
current futex.

 ** The NUMA problem

 At the current implementation, all futex kernel side infrastructure is
 stored on a single node. Given that, all futex() calls issued by
 processors that aren't located on that node will have a memory access
 penalty when doing it.

 ** The 32bit sized futex problem

 Futexes are used to implement atomic operations in userspace.
 Supporting 8, 16, 32 and 64 bit sized futexes allows user libraries to
 implement all those sizes in a performant way. Thanks Boost devs for
 feedback: https://lists.boost.org/Archives/boost/2021/05/251508.php

 Embedded systems or anything with memory constrains could benefit of
 using smaller sizes for the futex userspace integer.

 ** The wait on multiple problem

 The use case lies in the Wine implementation of the Windows NT interface
 WaitMultipleObjects. This Windows API function allows a thread to sleep
 waiting on the first of a set of event sources (mutexes, timers, signal,
 console input, etc) to signal.  Considering this is a primitive
 synchronization operation for Windows applications, being able to quickly
 signal events on the producer side, and quickly go to sleep on the
 consumer side is essential for good performance of those running over Wine.

[0] https://lore.kernel.org/lkml/20160505204230.932454245@linutronix.de/
[1] https://lore.kernel.org/lkml/20191221155659.3159-2-malteskarupke@web.de/
[2] https://lore.kernel.org/lkml/20200213214525.183689-1-andrealmeid@collabora.com/

* The solution

As proposed by Peter Zijlstra and Florian Weimer[3], a new interface
is required to solve this, which must be designed with those features in
mind. futex2() is that interface. As opposed to the current multiplexed
interface, the new one should have one syscall per operation. This will
allow the maintainability of the API if it gets extended, and will help
users with type checking of arguments.

In particular, the new interface is extended to support the ability to
wait on any of a list of futexes at a time, which could be seen as a
vectored extension of the FUTEX_WAIT semantics.

[3] https://lore.kernel.org/lkml/20200303120050.GC2596@hirez.programming.kicks-ass.net/

* The interface

The new interface can be seen in details in the following patches, but
this is a high level summary of what the interface can do:

 - Supports wake/wait semantics, as in futex()
 - Supports requeue operations, similarly as FUTEX_CMP_REQUEUE, but with
   individual flags for each address
 - Supports waiting for a vector of futexes, using a new syscall named
   futex_waitv()
 - Supports variable sized futexes (8bits, 16bits, 32bits and 64bits)
 - Supports NUMA-awareness operations, where the user can specify on
   which memory node would like to operate

* Implementation

The internal implementation follows a similar design to the original futex.
Given that we want to replicate the same external behavior of current
futex, this should be somewhat expected. For some functions, like the
init and the code to get a shared key, I literally copied code and
comments from kernel/futex.c. I decided to do so instead of exposing the
original function as a public function since in that way we can freely
modify our implementation if required, without any impact on old futex.
Also, the comments precisely describes the details and corner cases of
the implementation.

Each patch contains a brief description of implementation, but patch 7
"docs: locking: futex2: Add documentation" adds a more complete document
about it.

* The patchset

This patchset can be also found at my git tree:

https://gitlab.collabora.com/tonyk/linux/-/tree/futex2-dev

  - Patch 1: Implements wait/wake, and the basics foundations of futex2

  - Patches 2-5: Implement the remaining features (shared, waitv,
    requeue, sizes).

  - Patch 6:  Adds the x86_x32 ABI handling. I kept it in a separated
    patch since I'm not sure if x86_x32 is still a thing, or if it should
    return -ENOSYS.

  - Patch 7: Add a documentation file which details the interface and
    the internal implementation.

  - Patches 8-14: Selftests for all operations along with perf
    support for futex2.

  - Patch 15: While working on porting glibc for futex2, I found out
    that there's a futex_wake() call at the user thread exit path, if
    that thread was created with clone(..., CLONE_CHILD_SETTID, ...). In
    order to make pthreads work with futex2, it was required to add
    this patch. Note that this is more a proof-of-concept of what we
    will need to do in future, rather than part of the interface and
    shouldn't be merged as it is.

* Testing:

This patchset provides selftests for each operation and their flags.
Along with that, the following work was done:

 ** Stability

 To stress the interface in "real world scenarios":

 - glibc[4]: nptl's low level locking was modified to use futex2 API
   (except for robust and PI things). All relevant nptl/ tests passed.

 - Wine[5]: Proton/Wine was modified in order to use futex2() for the
   emulation of Windows NT sync mechanisms based on futex, called "fsync".
   Triple-A games with huge CPU's loads and tons of parallel jobs worked
   as expected when compared with the previous FUTEX_WAIT_MULTIPLE
   implementation at futex(). Some games issue 42k futex2() calls
   per second.

 - Full GNU/Linux distro: I installed the modified glibc in my host
   machine, so all pthread's programs would use futex2(). After tweaking
   systemd[6] to allow futex2() calls at seccomp, everything worked as
   expected (web browsers do some syscall sandboxing and need some
   configuration as well).

 - perf: The perf benchmarks tests can also be used to stress the
   interface, and they can be found in this patchset.

 ** Performance

 - For comparing futex() and futex2() performance, I used the artificial
   benchmarks implemented at perf (wake, wake-parallel, hash and
   requeue). The setup was 200 runs for each test and using 8, 80, 800,
   8000 for the number of threads, Note that for this test, I'm not using
   patch 14 ("kernel: Enable waitpid() for futex2") , for reasons explained
   at "The patchset" section.

 - For the first three ones, I measured an average of 4% gain in
   performance. This is not a big step, but it shows that the new
   interface is at least comparable in performance with the current one.

 - For requeue, I measured an average of 21% decrease in performance
   compared to the original futex implementation. This is expected given
   the new design with individual flags. The performance trade-offs are
   explained at patch 4 ("futex2: Implement requeue operation").

[4] https://gitlab.collabora.com/tonyk/glibc/-/tree/futex2
[5] https://gitlab.collabora.com/tonyk/wine/-/tree/proton_5.13
[6] https://gitlab.collabora.com/tonyk/systemd

* FAQ

 ** "Where's the code for NUMA?"

 NUMA will be implemented in future versions of this patch, and like the
 size feature, it will require work with users of futex to get feedback
 about it.

 ** "Where's the PI/robust stuff?"

 As said by Peter Zijlstra at [3], all those new features are related to
 the "simple" futex interface, that doesn't use PI or robust. Do we want
 to have this complexity at futex2() and if so, should it be part of
 this patchset or can it be future work?

Thanks,
	André

* Changelog

Changes from v3:
- Implemented variable sized futexes
v3: https://lore.kernel.org/lkml/20210427231248.220501-1-andrealmeid@collabora.com/

Changes from v2:
- API now supports 64bit futexes, in addition to 8, 16 and 32.
- This API change will break the glibc[4] and Proton[5] ports for now.
- Refactored futex2_wait and futex2_waitv selftests
v2: https://lore.kernel.org/lkml/20210304004219.134051-1-andrealmeid@collabora.com/

Changes from v1:
- Unified futex_set_timer_and_wait and __futex_wait code
- Dropped _carefull from linked list function calls
- Fixed typos on docs patch
- uAPI flags are now added as features are introduced, instead of all flags
  in patch 1
- Removed struct futex_single_waiter in favor of an anon struct
v1: https://lore.kernel.org/lkml/20210215152404.250281-1-andrealmeid@collabora.com/

André Almeida (15):
  futex2: Implement wait and wake functions
  futex2: Add support for shared futexes
  futex2: Implement vectorized wait
  futex2: Implement requeue operation
  futex2: Implement support for different futex sizes
  futex2: Add compatibility entry point for x86_x32 ABI
  docs: locking: futex2: Add documentation
  selftests: futex2: Add wake/wait test
  selftests: futex2: Add timeout test
  selftests: futex2: Add wouldblock test
  selftests: futex2: Add waitv test
  selftests: futex2: Add requeue test
  selftests: futex2: Add futex sizes test
  perf bench: Add futex2 benchmark tests
  kernel: Enable waitpid() for futex2

 Documentation/locking/futex2.rst              |  198 +++
 Documentation/locking/index.rst               |    1 +
 MAINTAINERS                                   |    2 +-
 arch/arm/tools/syscall.tbl                    |    4 +
 arch/arm64/include/asm/unistd.h               |    2 +-
 arch/arm64/include/asm/unistd32.h             |    8 +
 arch/x86/entry/syscalls/syscall_32.tbl        |    4 +
 arch/x86/entry/syscalls/syscall_64.tbl        |    4 +
 fs/inode.c                                    |    1 +
 include/linux/compat.h                        |   26 +
 include/linux/fs.h                            |    1 +
 include/linux/syscalls.h                      |   17 +
 include/uapi/asm-generic/unistd.h             |   14 +-
 include/uapi/linux/futex.h                    |   34 +
 init/Kconfig                                  |    7 +
 kernel/Makefile                               |    1 +
 kernel/fork.c                                 |    2 +
 kernel/futex2.c                               | 1289 +++++++++++++++++
 kernel/sys_ni.c                               |    9 +
 tools/arch/x86/include/asm/unistd_64.h        |   12 +
 tools/include/uapi/asm-generic/unistd.h       |   11 +-
 .../arch/x86/entry/syscalls/syscall_64.tbl    |    4 +
 tools/perf/bench/bench.h                      |    4 +
 tools/perf/bench/futex-hash.c                 |   24 +-
 tools/perf/bench/futex-requeue.c              |   57 +-
 tools/perf/bench/futex-wake-parallel.c        |   41 +-
 tools/perf/bench/futex-wake.c                 |   37 +-
 tools/perf/bench/futex.h                      |   47 +
 tools/perf/builtin-bench.c                    |   18 +-
 .../selftests/futex/functional/.gitignore     |    4 +
 .../selftests/futex/functional/Makefile       |    7 +-
 .../futex/functional/futex2_requeue.c         |  164 +++
 .../selftests/futex/functional/futex2_sizes.c |  146 ++
 .../selftests/futex/functional/futex2_wait.c  |  195 +++
 .../selftests/futex/functional/futex2_waitv.c |  154 ++
 .../futex/functional/futex_wait_timeout.c     |   58 +-
 .../futex/functional/futex_wait_wouldblock.c  |   33 +-
 .../testing/selftests/futex/functional/run.sh |    6 +
 .../selftests/futex/include/futex2test.h      |  113 ++
 39 files changed, 2707 insertions(+), 52 deletions(-)
 create mode 100644 Documentation/locking/futex2.rst
 create mode 100644 kernel/futex2.c
 create mode 100644 tools/testing/selftests/futex/functional/futex2_requeue.c
 create mode 100644 tools/testing/selftests/futex/functional/futex2_sizes.c
 create mode 100644 tools/testing/selftests/futex/functional/futex2_wait.c
 create mode 100644 tools/testing/selftests/futex/functional/futex2_waitv.c
 create mode 100644 tools/testing/selftests/futex/include/futex2test.h

-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 01/15] futex2: Implement wait and wake functions
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 02/15] futex2: Add support for shared futexes André Almeida
                   ` (15 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Create a new set of futex syscalls known as futex2. This new interface
is aimed to implement a more maintainable code, while removing obsolete
features and expanding it with new functionalities.

Implements wait and wake semantics for futexes, along with the base
infrastructure for future operations. The whole wait path is designed to
be used by N waiters, thus making easier to implement vectorized wait.

* Syscalls implemented by this patch:

- futex_wait(void *uaddr, unsigned int val, unsigned int flags,
	     struct timespec *timo)

   The user thread is put to sleep, waiting for a futex_wake() at uaddr,
   if the value at *uaddr is the same as val (otherwise, the syscall
   returns immediately with -EAGAIN). timo is an optional timeout value
   for the operation.

   Return 0 on success, error code otherwise.

 - futex_wake(void *uaddr, unsigned long nr_wake, unsigned int flags)

   Wake `nr_wake` threads waiting at uaddr.

   Return the number of woken threads on success, error code otherwise.

** The `flag` argument

 The flag is used to specify the size of the futex word
 (FUTEX_[8, 16, 32, 64]). It's mandatory to define one.

 By default, the timeout uses a monotonic clock, but can be used as a
 realtime one by using the FUTEX_REALTIME_CLOCK flag.

 By default, futexes are of the private type, that means that this user
 address will be accessed by threads that shares the same memory region.
 This allows for some internal optimizations, so they are faster.
 However, if the address needs to be shared with different processes
 (like using `mmap()` or `shm()`), they need to be defined as shared and
 the flag FUTEX_SHARED_FLAG is used to set that.

 By default, the operation has no NUMA-awareness, meaning that the user
 can't choose the memory node where the kernel side futex data will be
 stored. The user can choose the node where it wants to operate by
 setting the FUTEX_NUMA_FLAG and using the following structure (where X
 can be 8, 16, or 32, 64):

  struct futexX_numa {
          __uX value;
          __sX hint;
  };

 This structure should be passed at the `void *uaddr` of futex
 functions. The address of the structure will be used to be waited/waken
 on, and the `value` will be compared to `val` as usual. The `hint`
 member is used to defined which node the futex will use. When waiting,
 the futex will be registered on a kernel-side table stored on that
 node; when waking, the futex will be searched for on that given table.
 That means that there's no redundancy between tables, and the wrong
 `hint` value will led to undesired behavior.  Userspace is responsible
 for dealing with node migrations issues that may occur. `hint` can
 range from [0, MAX_NUMA_NODES], for specifying a node, or -1, to use
 the same node the current process is using.

 When not using FUTEX_NUMA_FLAG on a NUMA system, the futex will be
 stored on a global table on some node, defined at compilation time.

** The `timo` argument

As per the Y2038 work done in the kernel, new interfaces shouldn't add
timeout options known to be buggy. Given that, `timo` should be a 64bit
timeout at all platforms, using an absolute timeout value.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 MAINTAINERS                                   |   2 +-
 arch/arm/tools/syscall.tbl                    |   2 +
 arch/arm64/include/asm/unistd.h               |   2 +-
 arch/arm64/include/asm/unistd32.h             |   4 +
 arch/x86/entry/syscalls/syscall_32.tbl        |   2 +
 arch/x86/entry/syscalls/syscall_64.tbl        |   2 +
 include/linux/compat.h                        |   5 +
 include/linux/syscalls.h                      |   6 +
 include/uapi/asm-generic/unistd.h             |   8 +-
 include/uapi/linux/futex.h                    |   5 +
 init/Kconfig                                  |   7 +
 kernel/Makefile                               |   1 +
 kernel/futex2.c                               | 619 ++++++++++++++++++
 kernel/sys_ni.c                               |   5 +
 tools/include/uapi/asm-generic/unistd.h       |   8 +-
 .../arch/x86/entry/syscalls/syscall_64.tbl    |   2 +
 16 files changed, 676 insertions(+), 4 deletions(-)
 create mode 100644 kernel/futex2.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 673cadd5107a..b4b81b9a6e37 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -7521,7 +7521,7 @@ F:	Documentation/locking/*futex*
 F:	include/asm-generic/futex.h
 F:	include/linux/futex.h
 F:	include/uapi/linux/futex.h
-F:	kernel/futex.c
+F:	kernel/futex*
 F:	tools/perf/bench/futex*
 F:	tools/testing/selftests/futex/
 
diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl
index 28e03b5fec00..b60a8bdab623 100644
--- a/arch/arm/tools/syscall.tbl
+++ b/arch/arm/tools/syscall.tbl
@@ -460,3 +460,5 @@
 444	common	landlock_create_ruleset		sys_landlock_create_ruleset
 445	common	landlock_add_rule		sys_landlock_add_rule
 446	common	landlock_restrict_self		sys_landlock_restrict_self
+447	common	futex_wait			sys_futex_wait
+448	common	futex_wake			sys_futex_wake
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index 727bfc3be99b..3cb206aea3db 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -38,7 +38,7 @@
 #define __ARM_NR_compat_set_tls		(__ARM_NR_COMPAT_BASE + 5)
 #define __ARM_NR_COMPAT_END		(__ARM_NR_COMPAT_BASE + 0x800)
 
-#define __NR_compat_syscalls		447
+#define __NR_compat_syscalls		449
 #endif
 
 #define __ARCH_WANT_SYS_CLONE
diff --git a/arch/arm64/include/asm/unistd32.h b/arch/arm64/include/asm/unistd32.h
index 5dab69d2c22b..bca8835d7184 100644
--- a/arch/arm64/include/asm/unistd32.h
+++ b/arch/arm64/include/asm/unistd32.h
@@ -900,6 +900,10 @@ __SYSCALL(__NR_landlock_create_ruleset, sys_landlock_create_ruleset)
 __SYSCALL(__NR_landlock_add_rule, sys_landlock_add_rule)
 #define __NR_landlock_restrict_self 446
 __SYSCALL(__NR_landlock_restrict_self, sys_landlock_restrict_self)
+#define __NR_futex_wait 447
+__SYSCALL(__NR_futex_wait, compat_sys_futex_wait)
+#define __NR_futex_wake 448
+__SYSCALL(__NR_futex_wake, sys_futex_wake)
 
 /*
  * Please add new compat syscalls above this comment and update
diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl
index 4bbc267fb36b..e3b827a9c094 100644
--- a/arch/x86/entry/syscalls/syscall_32.tbl
+++ b/arch/x86/entry/syscalls/syscall_32.tbl
@@ -451,3 +451,5 @@
 444	i386	landlock_create_ruleset	sys_landlock_create_ruleset
 445	i386	landlock_add_rule	sys_landlock_add_rule
 446	i386	landlock_restrict_self	sys_landlock_restrict_self
+447	i386	futex_wait		sys_futex_wait			compat_sys_futex_wait
+448	i386	futex_wake		sys_futex_wake
diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl
index ce18119ea0d0..63b447255df2 100644
--- a/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/arch/x86/entry/syscalls/syscall_64.tbl
@@ -368,6 +368,8 @@
 444	common	landlock_create_ruleset	sys_landlock_create_ruleset
 445	common	landlock_add_rule	sys_landlock_add_rule
 446	common	landlock_restrict_self	sys_landlock_restrict_self
+447	common	futex_wait		sys_futex_wait
+448	common	futex_wake		sys_futex_wake
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
diff --git a/include/linux/compat.h b/include/linux/compat.h
index 8855b1b702b2..fe45135f3554 100644
--- a/include/linux/compat.h
+++ b/include/linux/compat.h
@@ -692,6 +692,11 @@ asmlinkage long
 compat_sys_get_robust_list(int pid, compat_uptr_t __user *head_ptr,
 			   compat_size_t __user *len_ptr);
 
+/* kernel/futex2.c */
+asmlinkage long
+compat_sys_futex_wait(void __user *uaddr, compat_u64 val, unsigned int flags,
+		      struct __kernel_timespec __user *timo);
+
 /* kernel/itimer.c */
 asmlinkage long compat_sys_getitimer(int which,
 				     struct old_itimerval32 __user *it);
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index 050511e8f1f8..b9c2874410d0 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -623,6 +623,12 @@ asmlinkage long sys_get_robust_list(int pid,
 asmlinkage long sys_set_robust_list(struct robust_list_head __user *head,
 				    size_t len);
 
+/* kernel/futex2.c */
+asmlinkage long sys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
+			       struct __kernel_timespec __user *timo);
+asmlinkage long sys_futex_wake(void __user *uaddr, unsigned int nr_wake,
+			       unsigned int flags);
+
 /* kernel/hrtimer.c */
 asmlinkage long sys_nanosleep(struct __kernel_timespec __user *rqtp,
 			      struct __kernel_timespec __user *rmtp);
diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
index 6de5a7fc066b..50bfed52575f 100644
--- a/include/uapi/asm-generic/unistd.h
+++ b/include/uapi/asm-generic/unistd.h
@@ -873,8 +873,14 @@ __SYSCALL(__NR_landlock_add_rule, sys_landlock_add_rule)
 #define __NR_landlock_restrict_self 446
 __SYSCALL(__NR_landlock_restrict_self, sys_landlock_restrict_self)
 
+#define __NR_futex_wait 447
+__SC_COMP(__NR_futex_wait, sys_futex_wait, compat_sys_futex_wait)
+
+#define __NR_futex_wake 448
+__SYSCALL(__NR_futex_wake, sys_futex_wake)
+
 #undef __NR_syscalls
-#define __NR_syscalls 447
+#define __NR_syscalls 449
 
 /*
  * 32 bit systems traditionally used different
diff --git a/include/uapi/linux/futex.h b/include/uapi/linux/futex.h
index a89eb0accd5e..8d30f4b6d094 100644
--- a/include/uapi/linux/futex.h
+++ b/include/uapi/linux/futex.h
@@ -41,6 +41,11 @@
 #define FUTEX_CMP_REQUEUE_PI_PRIVATE	(FUTEX_CMP_REQUEUE_PI | \
 					 FUTEX_PRIVATE_FLAG)
 
+/* Size argument to futex2 syscall */
+#define FUTEX_32	2
+
+#define FUTEX_SIZE_MASK	0x3
+
 /*
  * Support for robust futexes: the kernel cleans up held futexes at
  * thread exit time.
diff --git a/init/Kconfig b/init/Kconfig
index a61c92066c2e..d87629ec7e48 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -1555,6 +1555,13 @@ config FUTEX
 	  support for "fast userspace mutexes".  The resulting kernel may not
 	  run glibc-based applications correctly.
 
+config FUTEX2
+	bool "Enable futex2 support" if EXPERT
+	depends on FUTEX
+	default y
+	help
+	  Support for futex2 interface.
+
 config FUTEX_PI
 	bool
 	depends on FUTEX && RT_MUTEXES
diff --git a/kernel/Makefile b/kernel/Makefile
index 4df609be42d0..1eaf2af50283 100644
--- a/kernel/Makefile
+++ b/kernel/Makefile
@@ -60,6 +60,7 @@ obj-$(CONFIG_PROFILING) += profile.o
 obj-$(CONFIG_STACKTRACE) += stacktrace.o
 obj-y += time/
 obj-$(CONFIG_FUTEX) += futex.o
+obj-$(CONFIG_FUTEX2) += futex2.o
 obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o
 obj-$(CONFIG_SMP) += smp.o
 ifneq ($(CONFIG_SMP),y)
diff --git a/kernel/futex2.c b/kernel/futex2.c
new file mode 100644
index 000000000000..3ac92b54d925
--- /dev/null
+++ b/kernel/futex2.c
@@ -0,0 +1,619 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * futex2 system call interface by André Almeida <andrealmeid@collabora.com>
+ *
+ * Copyright 2021 Collabora Ltd.
+ *
+ * Based on original futex implementation by:
+ *  (C) 2002 Rusty Russell, IBM
+ *  (C) 2003, 2006 Ingo Molnar, Red Hat Inc.
+ *  (C) 2003, 2004 Jamie Lokier
+ *  (C) 2006 Thomas Gleixner, Timesys Corp.
+ *  (C) 2007 Eric Dumazet
+ *  (C) 2009 Darren Hart, IBM
+ */
+
+#include <linux/freezer.h>
+#include <linux/jhash.h>
+#include <linux/memblock.h>
+#include <linux/sched/wake_q.h>
+#include <linux/spinlock.h>
+#include <linux/syscalls.h>
+#include <uapi/linux/futex.h>
+
+/**
+ * struct futex_key - Components to build unique key for a futex
+ * @pointer: Pointer to current->mm
+ * @index: Start address of the page containing futex
+ * @offset: Address offset of uaddr in a page
+ */
+struct futex_key {
+	u64 pointer;
+	unsigned long index;
+	unsigned long offset;
+};
+
+/**
+ * struct futex_waiter - List entry for a waiter
+ * @uaddr:        Virtual address of userspace futex
+ * @key:          Information that uniquely identify a futex
+ * @list:	  List node struct
+ * @val:	  Expected value for this waiter
+ * @flags:        Flags
+ * @bucket:       Pointer to the bucket for this waiter
+ * @index:        Index of waiter in futexv list
+ */
+struct futex_waiter {
+	void __user *uaddr;
+	struct futex_key key;
+	struct list_head list;
+	u64 val;
+	unsigned int flags;
+	struct futex_bucket *bucket;
+	unsigned int index;
+};
+
+/**
+ * struct futex_waiter_head - List of futexes to be waited
+ * @task:    Task to be awaken
+ * @hint:    Was someone on this list awakened?
+ * @objects: List of futexes
+ */
+struct futex_waiter_head {
+	struct task_struct *task;
+	bool hint;
+	struct futex_waiter objects[0];
+};
+
+/**
+ * struct futex_bucket - A bucket of futex's hash table
+ * @waiters: Number of waiters in the bucket
+ * @lock:    Bucket lock
+ * @list:    List of waiters on this bucket
+ */
+struct futex_bucket {
+	atomic_t waiters;
+	spinlock_t lock;
+	struct list_head list;
+};
+
+/* Mask for futex2 flag operations */
+#define FUTEX2_MASK (FUTEX_SIZE_MASK | FUTEX_CLOCK_REALTIME)
+
+static struct futex_bucket *futex_table;
+static unsigned int futex2_hashsize;
+
+/*
+ * Reflects a new waiter being added to the waitqueue.
+ */
+static inline void bucket_inc_waiters(struct futex_bucket *bucket)
+{
+#ifdef CONFIG_SMP
+	atomic_inc(&bucket->waiters);
+	/*
+	 * Issue a barrier after adding so futex_wake() will see that the
+	 * value had increased
+	 */
+	smp_mb__after_atomic();
+#endif
+}
+
+/*
+ * Reflects a waiter being removed from the waitqueue by wakeup
+ * paths.
+ */
+static inline void bucket_dec_waiters(struct futex_bucket *bucket)
+{
+#ifdef CONFIG_SMP
+	atomic_dec(&bucket->waiters);
+#endif
+}
+
+/*
+ * Get the number of waiters in a bucket
+ */
+static inline int bucket_get_waiters(struct futex_bucket *bucket)
+{
+#ifdef CONFIG_SMP
+	/*
+	 * Issue a barrier before reading so we get an updated value from
+	 * futex_wait()
+	 */
+	smp_mb();
+	return atomic_read(&bucket->waiters);
+#else
+	return 1;
+#endif
+}
+
+/**
+ * futex_get_bucket - Check if the user address is valid, prepare internal
+ *                    data and calculate the hash
+ * @uaddr:   futex user address
+ * @key:     data that uniquely identifies a futex
+ *
+ * Return: address of bucket on success, error code otherwise
+ */
+static struct futex_bucket *futex_get_bucket(void __user *uaddr,
+					     struct futex_key *key)
+{
+	uintptr_t address = (uintptr_t)uaddr;
+	u32 hash_key;
+
+	/* Checking if uaddr is valid and accessible */
+	if (unlikely(!IS_ALIGNED(address, sizeof(u32))))
+		return ERR_PTR(-EINVAL);
+	if (unlikely(!access_ok(uaddr, sizeof(u32))))
+		return ERR_PTR(-EFAULT);
+
+	key->offset = address % PAGE_SIZE;
+	address -= key->offset;
+	key->pointer = (u64)address;
+	key->index = (unsigned long)current->mm;
+
+	/* Generate hash key for this futex using uaddr and current->mm */
+	hash_key = jhash2((u32 *)key, sizeof(*key) / sizeof(u32), 0);
+
+	/* Since HASH_SIZE is 2^n, subtracting 1 makes a perfect bit mask */
+	return &futex_table[hash_key & (futex2_hashsize - 1)];
+}
+
+/**
+ * futex_get_user - Get the userspace value on this address
+ * @uval:  variable to store the value
+ * @uaddr: userspace address
+ *
+ * Check the comment at futex_enqueue() for more information.
+ */
+static int futex_get_user(u32 *uval, u32 __user *uaddr)
+{
+	int ret;
+
+	pagefault_disable();
+	ret = __get_user(*uval, uaddr);
+	pagefault_enable();
+
+	return ret;
+}
+
+/**
+ * futex_setup_time - Prepare the timeout mechanism and start it.
+ * @timo:    Timeout value from userspace
+ * @timeout: Pointer to hrtimer handler
+ * @flags: Flags from userspace, to decide which clockid to use
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int futex_setup_time(struct __kernel_timespec __user *timo,
+			    struct hrtimer_sleeper *timeout,
+			    unsigned int flags)
+{
+	ktime_t time;
+	struct timespec64 ts;
+	clockid_t clockid = (flags & FUTEX_CLOCK_REALTIME) ?
+			    CLOCK_REALTIME : CLOCK_MONOTONIC;
+
+	if (get_timespec64(&ts, timo))
+		return -EFAULT;
+
+	if (!timespec64_valid(&ts))
+		return -EINVAL;
+
+	time = timespec64_to_ktime(ts);
+
+	hrtimer_init_sleeper(timeout, clockid, HRTIMER_MODE_ABS);
+
+	hrtimer_set_expires(&timeout->timer, time);
+
+	hrtimer_sleeper_start_expires(timeout, HRTIMER_MODE_ABS);
+
+	return 0;
+}
+
+/**
+ * futex_dequeue_multiple - Remove multiple futexes from hash table
+ * @futexv: list of waiters
+ * @nr:     number of futexes to be removed
+ *
+ * This function is used if (a) something went wrong while enqueuing, and we
+ * need to undo our work (then nr <= nr_futexes) or (b) we woke up, and thus
+ * need to remove every waiter, check if some was indeed woken and return.
+ * Before removing a waiter, we check if it's on the list, since we have no
+ * clue who have been waken.
+ *
+ * Return:
+ *  * -1  - If no futex was woken during the removal
+ *  * 0>= - At least one futex was found woken, index of the last one
+ */
+static int futex_dequeue_multiple(struct futex_waiter_head *futexv, unsigned int nr)
+{
+	int i, ret = -1;
+
+	for (i = 0; i < nr; i++) {
+		spin_lock(&futexv->objects[i].bucket->lock);
+		if (!list_empty(&futexv->objects[i].list)) {
+			list_del_init(&futexv->objects[i].list);
+			bucket_dec_waiters(futexv->objects[i].bucket);
+		} else {
+			ret = i;
+		}
+		spin_unlock(&futexv->objects[i].bucket->lock);
+	}
+
+	return ret;
+}
+
+/**
+ * futex_enqueue - Check the value and enqueue a futex on a wait list
+ *
+ * @futexv:     List of futexes
+ * @nr_futexes: Number of futexes in the list
+ * @awakened:	If a futex was awakened during enqueueing, store the index here
+ *
+ * Get the value from the userspace address and compares with the expected one.
+ *
+ * Getting the value from user futex address:
+ *
+ * Since we are in a hurry, we use a spin lock and we can't sleep.
+ * Try to get the value with page fault disabled (when enable, we might
+ * sleep).
+ *
+ * If we fail, we aren't sure if the address is invalid or is just a
+ * page fault. Then, release the lock (so we can sleep) and try to get
+ * the value with page fault enabled. In order to trigger a page fault
+ * handling, we just call __get_user() again. If we sleep with enqueued
+ * futexes, we might miss a wake, so dequeue everything before sleeping.
+ *
+ * If get_user succeeds, this mean that the address is valid and we do
+ * the work again. Since we just handled the page fault, the page is
+ * likely pinned in memory and we should be luckier this time and be
+ * able to get the value. If we fail anyway, we will try again.
+ *
+ * If even with page faults enabled we get and error, this means that
+ * the address is not valid and we return from the syscall.
+ *
+ * If we got an unexpected value or need to treat a page fault and realized that
+ * a futex was awakened, we can priority this and return success.
+ *
+ * In success, enqueue the futex in the correct bucket
+ *
+ * Return:
+ * * 1  - We were awake in the process and nothing is enqueued
+ * * 0  - Everything is enqueued and we are ready to sleep
+ * * 0< - Something went wrong, nothing is enqueued, return error code
+ */
+static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futexes,
+			 int *awakened)
+{
+	int i, ret;
+	u32 uval, val;
+	u32 __user *uaddr;
+	struct futex_bucket *bucket;
+
+retry:
+	set_current_state(TASK_INTERRUPTIBLE);
+
+	for (i = 0; i < nr_futexes; i++) {
+		uaddr = (u32 __user *)futexv->objects[i].uaddr;
+		val = (u32)futexv->objects[i].val;
+
+		bucket = futexv->objects[i].bucket;
+
+		bucket_inc_waiters(bucket);
+		spin_lock(&bucket->lock);
+
+		ret = futex_get_user(&uval, uaddr);
+
+		if (unlikely(ret)) {
+			spin_unlock(&bucket->lock);
+
+			bucket_dec_waiters(bucket);
+			__set_current_state(TASK_RUNNING);
+			*awakened = futex_dequeue_multiple(futexv, i);
+
+			if (*awakened >= 0)
+				return 1;
+
+			if (__get_user(uval, uaddr))
+				return -EFAULT;
+
+			goto retry;
+		}
+
+		if (uval != val) {
+			spin_unlock(&bucket->lock);
+
+			bucket_dec_waiters(bucket);
+			__set_current_state(TASK_RUNNING);
+			*awakened = futex_dequeue_multiple(futexv, i);
+
+			if (*awakened >= 0)
+				return 1;
+
+			return -EAGAIN;
+		}
+
+		list_add_tail(&futexv->objects[i].list, &bucket->list);
+		spin_unlock(&bucket->lock);
+	}
+
+	return 0;
+}
+
+/**
+ * __futex_waitv - Enqueue the list of futexes and wait to be woken
+ * @futexv: List of futexes to wait
+ * @nr_futexes: Length of futexv
+ * @timo:	Timeout
+ * @flags:	Timeout flags
+ *
+ * Return:
+ * * 0 >= - Hint of which futex woke us
+ * * 0 <  - Error code
+ */
+static int __futex_waitv(struct futex_waiter_head *futexv, unsigned int nr_futexes,
+			 struct __kernel_timespec __user *timo,
+			 unsigned int flags)
+{
+	int ret;
+	struct hrtimer_sleeper timeout;
+
+	if (timo) {
+		ret = futex_setup_time(timo, &timeout, flags);
+		if (ret)
+			return ret;
+	}
+
+	while (1) {
+		int awakened = -1;
+
+		ret = futex_enqueue(futexv, nr_futexes, &awakened);
+
+		if (ret) {
+			if (awakened >= 0)
+				ret = awakened;
+			break;
+		}
+
+		/* Before sleeping, check if someone was woken */
+		if (!futexv->hint && (!timo || timeout.task))
+			freezable_schedule();
+
+		__set_current_state(TASK_RUNNING);
+
+		/*
+		 * One of those things triggered this wake:
+		 *
+		 * * We have been removed from the bucket. futex_wake() woke
+		 *   us. We just need to dequeue and return 0 to userspace.
+		 *
+		 * However, if no futex was dequeued by a futex_wake():
+		 *
+		 * * If the there's a timeout and it has expired,
+		 *   return -ETIMEDOUT.
+		 *
+		 * * If there is a signal pending, something wants to kill our
+		 *   thread, return -ERESTARTSYS.
+		 *
+		 * * If there's no signal pending, it was a spurious wake
+		 *   (scheduler gave us a chance to do some work, even if we
+		 *   don't want to). We need to remove ourselves from the
+		 *   bucket and add again, to prevent losing wakeups in the
+		 *   meantime.
+		 */
+
+		ret = futex_dequeue_multiple(futexv, nr_futexes);
+
+		/* Normal wake */
+		if (ret >= 0)
+			break;
+
+		if (timo && !timeout.task) {
+			ret = -ETIMEDOUT;
+			break;
+		}
+
+		if (signal_pending(current)) {
+			ret = -ERESTARTSYS;
+			break;
+		}
+
+		/* Spurious wake, do everything again */
+	}
+
+	if (timo)
+		hrtimer_cancel(&timeout.timer);
+
+	return ret;
+}
+
+static long ksys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
+			    struct __kernel_timespec __user *timo)
+{
+	unsigned int size = flags & FUTEX_SIZE_MASK;
+	struct futex_waiter *waiter;
+	struct futex_waiter_head *futexv;
+
+	/* Wrapper for a futexv_waiter_head with one element */
+	struct {
+		struct futex_waiter_head futexv;
+		struct futex_waiter waiter;
+	} __packed wait_single;
+
+	if (flags & ~FUTEX2_MASK)
+		return -EINVAL;
+
+	if (size != FUTEX_32)
+		return -EINVAL;
+
+	futexv = &wait_single.futexv;
+	futexv->task = current;
+	futexv->hint = false;
+
+	waiter = &wait_single.waiter;
+	waiter->index = 0;
+	waiter->val = val;
+	waiter->uaddr = uaddr;
+	memset(&wait_single.waiter.key, 0, sizeof(struct futex_key));
+
+	INIT_LIST_HEAD(&waiter->list);
+
+	/* Get an unlocked hash bucket */
+	waiter->bucket = futex_get_bucket(uaddr, &waiter->key);
+	if (IS_ERR(waiter->bucket))
+		return PTR_ERR(waiter->bucket);
+
+	return __futex_waitv(futexv, 1, timo, flags);
+}
+
+#ifdef CONFIG_COMPAT
+COMPAT_SYSCALL_DEFINE4(compat_futex_wait, void __user *, uaddr, compat_u64, val,
+		       unsigned int, flags,
+		       struct __kernel_timespec __user *, timo)
+{
+	return ksys_futex_wait(uaddr, val, flags, timo);
+}
+#endif
+
+/**
+ * sys_futex_wait - Wait on a futex address if (*uaddr) == val
+ * @uaddr: User address of futex
+ * @val:   Expected value of futex
+ * @flags: Specify the size of futex and the clockid
+ * @timo:  Optional absolute timeout.
+ *
+ * The user thread is put to sleep, waiting for a futex_wake() at uaddr, if the
+ * value at *uaddr is the same as val (otherwise, the syscall returns
+ * immediately with -EAGAIN).
+ *
+ * Returns 0 on success, error code otherwise.
+ */
+SYSCALL_DEFINE4(futex_wait, void __user *, uaddr, u64, val, unsigned int, flags,
+		struct __kernel_timespec __user *, timo)
+{
+
+	return ksys_futex_wait(uaddr, val, flags, timo);
+}
+
+/**
+ * futex_get_parent - For a given futex in a futexv list, get a pointer to the futexv
+ * @waiter: Address of futex in the list
+ * @index: Index of futex in the list
+ *
+ * Return: A pointer to its futexv struct
+ */
+static inline struct futex_waiter_head *futex_get_parent(uintptr_t waiter,
+							 unsigned int index)
+{
+	uintptr_t parent = waiter - sizeof(struct futex_waiter_head)
+			   - (uintptr_t)(index * sizeof(struct futex_waiter));
+
+	return (struct futex_waiter_head *)parent;
+}
+
+/**
+ * futex_mark_wake - Find the task to be wake and add it in wake queue
+ * @waiter: Waiter to be wake
+ * @bucket: Bucket to be decremented
+ * @wake_q: Wake queue to insert the task
+ */
+static void futex_mark_wake(struct futex_waiter *waiter,
+			    struct futex_bucket *bucket,
+			    struct wake_q_head *wake_q)
+{
+	struct task_struct *task;
+	struct futex_waiter_head *parent = futex_get_parent((uintptr_t)waiter,
+							    waiter->index);
+
+	lockdep_assert_held(&bucket->lock);
+	parent->hint = true;
+	task = parent->task;
+	get_task_struct(task);
+	list_del_init(&waiter->list);
+	wake_q_add_safe(wake_q, task);
+	bucket_dec_waiters(bucket);
+}
+
+static inline bool futex_match(struct futex_key key1, struct futex_key key2)
+{
+	return (key1.index == key2.index &&
+		key1.pointer == key2.pointer &&
+		key1.offset == key2.offset);
+}
+
+/**
+ * sys_futex_wake - Wake a number of futexes waiting on an address
+ * @uaddr:   Address of futex to be woken up
+ * @nr_wake: Number of futexes waiting in uaddr to be woken up
+ * @flags:   Flags for size and shared
+ *
+ * Wake `nr_wake` threads waiting at uaddr.
+ *
+ * Returns the number of woken threads on success, error code otherwise.
+ */
+SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
+		unsigned int, flags)
+{
+	unsigned int size = flags & FUTEX_SIZE_MASK;
+	struct futex_waiter waiter, *aux, *tmp;
+	struct futex_bucket *bucket;
+	DEFINE_WAKE_Q(wake_q);
+	int ret = 0;
+
+	if (flags & ~FUTEX2_MASK)
+		return -EINVAL;
+
+	if (size != FUTEX_32)
+		return -EINVAL;
+
+	bucket = futex_get_bucket(uaddr, &waiter.key);
+	if (IS_ERR(bucket))
+		return PTR_ERR(bucket);
+
+	if (!bucket_get_waiters(bucket) || !nr_wake)
+		return 0;
+
+	spin_lock(&bucket->lock);
+	list_for_each_entry_safe(aux, tmp, &bucket->list, list) {
+		if (futex_match(waiter.key, aux->key)) {
+			futex_mark_wake(aux, bucket, &wake_q);
+			if (++ret >= nr_wake)
+				break;
+		}
+	}
+	spin_unlock(&bucket->lock);
+
+	wake_up_q(&wake_q);
+
+	return ret;
+}
+
+static int __init futex2_init(void)
+{
+	int i;
+	unsigned int futex_shift;
+
+#if CONFIG_BASE_SMALL
+	futex2_hashsize = 16;
+#else
+	futex2_hashsize = roundup_pow_of_two(256 * num_possible_cpus());
+#endif
+
+	futex_table = alloc_large_system_hash("futex2", sizeof(struct futex_bucket),
+					      futex2_hashsize, 0,
+					      futex2_hashsize < 256 ? HASH_SMALL : 0,
+					      &futex_shift, NULL,
+					      futex2_hashsize, futex2_hashsize);
+	futex2_hashsize = 1UL << futex_shift;
+
+	BUG_ON(!is_power_of_2(futex2_hashsize));
+
+	for (i = 0; i < futex2_hashsize; i++) {
+		INIT_LIST_HEAD(&futex_table[i].list);
+		spin_lock_init(&futex_table[i].lock);
+		atomic_set(&futex_table[i].waiters, 0);
+	}
+
+	return 0;
+}
+core_initcall(futex2_init);
diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
index 0ea8128468c3..dbe397eaea46 100644
--- a/kernel/sys_ni.c
+++ b/kernel/sys_ni.c
@@ -151,6 +151,11 @@ COND_SYSCALL_COMPAT(set_robust_list);
 COND_SYSCALL(get_robust_list);
 COND_SYSCALL_COMPAT(get_robust_list);
 
+/* kernel/futex2.c */
+COND_SYSCALL(futex_wait);
+COND_SYSCALL_COMPAT(futex_wait);
+COND_SYSCALL(futex_wake);
+
 /* kernel/hrtimer.c */
 
 /* kernel/itimer.c */
diff --git a/tools/include/uapi/asm-generic/unistd.h b/tools/include/uapi/asm-generic/unistd.h
index 6de5a7fc066b..b48fd1899c3d 100644
--- a/tools/include/uapi/asm-generic/unistd.h
+++ b/tools/include/uapi/asm-generic/unistd.h
@@ -873,8 +873,14 @@ __SYSCALL(__NR_landlock_add_rule, sys_landlock_add_rule)
 #define __NR_landlock_restrict_self 446
 __SYSCALL(__NR_landlock_restrict_self, sys_landlock_restrict_self)
 
+#define __NR_futex_wait 447
+__SYSCALL(__NR_futex_wait, sys_futex_wait)
+
+#define __NR_futex_wake 448
+__SYSCALL(__NR_futex_wake, sys_futex_wake)
+
 #undef __NR_syscalls
-#define __NR_syscalls 447
+#define __NR_syscalls 449
 
 /*
  * 32 bit systems traditionally used different
diff --git a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
index ce18119ea0d0..8eb17cc08a69 100644
--- a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
@@ -368,6 +368,8 @@
 444	common	landlock_create_ruleset	sys_landlock_create_ruleset
 445	common	landlock_add_rule	sys_landlock_add_rule
 446	common	landlock_restrict_self	sys_landlock_restrict_self
+447	common  futex_wait              sys_futex_wait
+448	common  futex_wake              sys_futex_wake
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 02/15] futex2: Add support for shared futexes
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
  2021-06-03 19:59 ` [PATCH v4 01/15] futex2: Implement wait and wake functions André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 03/15] futex2: Implement vectorized wait André Almeida
                   ` (14 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Add support for shared futexes for cross-process resources. This design
relies on the same approach done in old futex to create an unique id for
file-backed shared memory, by using a counter at struct inode.

There are two types of futexes: private and shared ones. The private are
futexes meant to be used by threads that shares the same memory space,
are easier to be uniquely identified an thus can have some performance
optimization. The elements for identifying one are: the start address of
the page where the address is, the address offset within the page and
the current->mm pointer.

Now, for uniquely identifying shared futex:

- If the page containing the user address is an anonymous page, we can
  just use the same data used for private futexes (the start address of
  the page, the address offset within the page and the current->mm
  pointer) that will be enough for uniquely identifying such futex. We
  also set one bit at the key to differentiate if a private futex is
  used on the same address (mixing shared and private calls are not
  allowed).

- If the page is file-backed, current->mm maybe isn't the same one for
  every user of this futex, so we need to use other data: the
  page->index, an UUID for the struct inode and the offset within the
  page.

Note that members of futex_key doesn't have any particular meaning after
they are part of the struct - they are just bytes to identify a futex.
Given that, we don't need to use a particular name or type that matches
the original data, we only need to care about the bitsize of each
component and make both private and shared data fit in the same memory
space.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 fs/inode.c                 |   1 +
 include/linux/fs.h         |   1 +
 include/uapi/linux/futex.h |   2 +
 kernel/futex2.c            | 221 +++++++++++++++++++++++++++++++++++--
 4 files changed, 218 insertions(+), 7 deletions(-)

diff --git a/fs/inode.c b/fs/inode.c
index c93500d84264..73e82a304d10 100644
--- a/fs/inode.c
+++ b/fs/inode.c
@@ -138,6 +138,7 @@ int inode_init_always(struct super_block *sb, struct inode *inode)
 	inode->i_blkbits = sb->s_blocksize_bits;
 	inode->i_flags = 0;
 	atomic64_set(&inode->i_sequence, 0);
+	atomic64_set(&inode->i_sequence2, 0);
 	atomic_set(&inode->i_count, 1);
 	inode->i_op = &empty_iops;
 	inode->i_fop = &no_open_fops;
diff --git a/include/linux/fs.h b/include/linux/fs.h
index c3c88fdb9b2a..5dd112c04357 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -682,6 +682,7 @@ struct inode {
 	};
 	atomic64_t		i_version;
 	atomic64_t		i_sequence; /* see futex */
+	atomic64_t		i_sequence2; /* see futex2 */
 	atomic_t		i_count;
 	atomic_t		i_dio_count;
 	atomic_t		i_writecount;
diff --git a/include/uapi/linux/futex.h b/include/uapi/linux/futex.h
index 8d30f4b6d094..70ea66fffb1c 100644
--- a/include/uapi/linux/futex.h
+++ b/include/uapi/linux/futex.h
@@ -46,6 +46,8 @@
 
 #define FUTEX_SIZE_MASK	0x3
 
+#define FUTEX_SHARED_FLAG 8
+
 /*
  * Support for robust futexes: the kernel cleans up held futexes at
  * thread exit time.
diff --git a/kernel/futex2.c b/kernel/futex2.c
index 3ac92b54d925..f112c8c48f91 100644
--- a/kernel/futex2.c
+++ b/kernel/futex2.c
@@ -14,8 +14,10 @@
  */
 
 #include <linux/freezer.h>
+#include <linux/hugetlb.h>
 #include <linux/jhash.h>
 #include <linux/memblock.h>
+#include <linux/pagemap.h>
 #include <linux/sched/wake_q.h>
 #include <linux/spinlock.h>
 #include <linux/syscalls.h>
@@ -23,8 +25,8 @@
 
 /**
  * struct futex_key - Components to build unique key for a futex
- * @pointer: Pointer to current->mm
- * @index: Start address of the page containing futex
+ * @pointer: Pointer to current->mm or inode's UUID for file backed futexes
+ * @index: Start address of the page containing futex or index of the page
  * @offset: Address offset of uaddr in a page
  */
 struct futex_key {
@@ -78,7 +80,12 @@ struct futex_bucket {
 };
 
 /* Mask for futex2 flag operations */
-#define FUTEX2_MASK (FUTEX_SIZE_MASK | FUTEX_CLOCK_REALTIME)
+#define FUTEX2_MASK (FUTEX_SIZE_MASK | FUTEX_CLOCK_REALTIME | FUTEX_SHARED_FLAG)
+
+#define is_object_shared ((futexv->objects[i].flags & FUTEX_SHARED_FLAG) ? true : false)
+
+#define FUT_OFF_INODE    1 /* We set bit 0 if key has a reference on inode */
+#define FUT_OFF_MMSHARED 2 /* We set bit 1 if key has a reference on mm */
 
 static struct futex_bucket *futex_table;
 static unsigned int futex2_hashsize;
@@ -126,16 +133,198 @@ static inline int bucket_get_waiters(struct futex_bucket *bucket)
 #endif
 }
 
+/**
+ * futex_get_inode_uuid - Gets an UUID for an inode
+ * @inode: inode to get UUID
+ *
+ * Generate a machine wide unique identifier for this inode.
+ *
+ * This relies on u64 not wrapping in the life-time of the machine; which with
+ * 1ns resolution means almost 585 years.
+ *
+ * This further relies on the fact that a well formed program will not unmap
+ * the file while it has a (shared) futex waiting on it. This mapping will have
+ * a file reference which pins the mount and inode.
+ *
+ * If for some reason an inode gets evicted and read back in again, it will get
+ * a new sequence number and will _NOT_ match, even though it is the exact same
+ * file.
+ *
+ * It is important that match_futex() will never have a false-positive, esp.
+ * for PI futexes that can mess up the state. The above argues that false-negatives
+ * are only possible for malformed programs.
+ *
+ * Returns: UUID for the given inode
+ */
+static u64 futex_get_inode_uuid(struct inode *inode)
+{
+	static atomic64_t i_seq;
+	u64 old;
+
+	/* Does the inode already have a sequence number? */
+	old = atomic64_read(&inode->i_sequence2);
+
+	if (likely(old))
+		return old;
+
+	for (;;) {
+		u64 new = atomic64_add_return(1, &i_seq);
+
+		if (WARN_ON_ONCE(!new))
+			continue;
+
+		old = atomic64_cmpxchg_relaxed(&inode->i_sequence2, 0, new);
+		if (old)
+			return old;
+		return new;
+	}
+}
+
+/**
+ * futex_get_shared_key - Get a key for a shared futex
+ * @address: Futex memory address
+ * @mm:      Current process mm_struct pointer
+ * @key:     Key struct to be filled
+ *
+ * Returns: 0 on success, error code otherwise
+ */
+static int futex_get_shared_key(uintptr_t address, struct mm_struct *mm,
+				struct futex_key *key)
+{
+	int ret;
+	struct page *page, *tail;
+	struct address_space *mapping;
+
+again:
+	ret = get_user_pages_fast(address, 1, 0, &page);
+	if (ret < 0)
+		return ret;
+
+	/*
+	 * The treatment of mapping from this point on is critical. The page
+	 * lock protects many things but in this context the page lock
+	 * stabilizes mapping, prevents inode freeing in the shared
+	 * file-backed region case and guards against movement to swap cache.
+	 *
+	 * Strictly speaking the page lock is not needed in all cases being
+	 * considered here and page lock forces unnecessarily serialization
+	 * From this point on, mapping will be re-verified if necessary and
+	 * page lock will be acquired only if it is unavoidable
+	 *
+	 * Mapping checks require the head page for any compound page so the
+	 * head page and mapping is looked up now. For anonymous pages, it
+	 * does not matter if the page splits in the future as the key is
+	 * based on the address. For filesystem-backed pages, the tail is
+	 * required as the index of the page determines the key. For
+	 * base pages, there is no tail page and tail == page.
+	 */
+	tail = page;
+	page = compound_head(page);
+	mapping = READ_ONCE(page->mapping);
+
+	/*
+	 * If page->mapping is NULL, then it cannot be a PageAnon
+	 * page; but it might be the ZERO_PAGE or in the gate area or
+	 * in a special mapping (all cases which we are happy to fail);
+	 * or it may have been a good file page when get_user_pages_fast
+	 * found it, but truncated or holepunched or subjected to
+	 * invalidate_complete_page2 before we got the page lock (also
+	 * cases which we are happy to fail).  And we hold a reference,
+	 * so refcount care in invalidate_complete_page's remove_mapping
+	 * prevents drop_caches from setting mapping to NULL beneath us.
+	 *
+	 * The case we do have to guard against is when memory pressure made
+	 * shmem_writepage move it from filecache to swapcache beneath us:
+	 * an unlikely race, but we do need to retry for page->mapping.
+	 */
+	if (unlikely(!mapping)) {
+		int shmem_swizzled;
+
+		/*
+		 * Page lock is required to identify which special case above
+		 * applies. If this is really a shmem page then the page lock
+		 * will prevent unexpected transitions.
+		 */
+		lock_page(page);
+		shmem_swizzled = PageSwapCache(page) || page->mapping;
+		unlock_page(page);
+		put_page(page);
+
+		if (shmem_swizzled)
+			goto again;
+
+		return -EFAULT;
+	}
+
+	/*
+	 * If the futex key is stored on an anonymous page, then the associated
+	 * object is the mm which is implicitly pinned by the calling process.
+	 *
+	 * NOTE: When userspace waits on a MAP_SHARED mapping, even if
+	 * it's a read-only handle, it's expected that futexes attach to
+	 * the object not the particular process.
+	 */
+	if (PageAnon(page)) {
+		key->offset |= FUT_OFF_MMSHARED;
+	} else {
+		struct inode *inode;
+
+		/*
+		 * The associated futex object in this case is the inode and
+		 * the page->mapping must be traversed. Ordinarily this should
+		 * be stabilised under page lock but it's not strictly
+		 * necessary in this case as we just want to pin the inode, not
+		 * update the radix tree or anything like that.
+		 *
+		 * The RCU read lock is taken as the inode is finally freed
+		 * under RCU. If the mapping still matches expectations then the
+		 * mapping->host can be safely accessed as being a valid inode.
+		 */
+		rcu_read_lock();
+
+		if (READ_ONCE(page->mapping) != mapping) {
+			rcu_read_unlock();
+			put_page(page);
+
+			goto again;
+		}
+
+		inode = READ_ONCE(mapping->host);
+		if (!inode) {
+			rcu_read_unlock();
+			put_page(page);
+
+			goto again;
+		}
+
+		key->pointer = futex_get_inode_uuid(inode);
+		key->index = (unsigned long)basepage_index(tail);
+		key->offset |= FUT_OFF_INODE;
+
+		rcu_read_unlock();
+	}
+
+	put_page(page);
+
+	return 0;
+}
+
 /**
  * futex_get_bucket - Check if the user address is valid, prepare internal
  *                    data and calculate the hash
  * @uaddr:   futex user address
  * @key:     data that uniquely identifies a futex
+ * @shared:  is this a shared futex?
+ *
+ * For private futexes, each uaddr will be unique for a given mm_struct, and it
+ * won't be freed for the life time of the process. For shared futexes, check
+ * futex_get_shared_key().
  *
  * Return: address of bucket on success, error code otherwise
  */
 static struct futex_bucket *futex_get_bucket(void __user *uaddr,
-					     struct futex_key *key)
+					     struct futex_key *key,
+					     bool shared)
 {
 	uintptr_t address = (uintptr_t)uaddr;
 	u32 hash_key;
@@ -151,6 +340,9 @@ static struct futex_bucket *futex_get_bucket(void __user *uaddr,
 	key->pointer = (u64)address;
 	key->index = (unsigned long)current->mm;
 
+	if (shared)
+		futex_get_shared_key(address, current->mm, key);
+
 	/* Generate hash key for this futex using uaddr and current->mm */
 	hash_key = jhash2((u32 *)key, sizeof(*key) / sizeof(u32), 0);
 
@@ -288,6 +480,7 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
 	int i, ret;
 	u32 uval, val;
 	u32 __user *uaddr;
+	bool retry = false;
 	struct futex_bucket *bucket;
 
 retry:
@@ -297,6 +490,18 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
 		uaddr = (u32 __user *)futexv->objects[i].uaddr;
 		val = (u32)futexv->objects[i].val;
 
+		if (is_object_shared && retry) {
+			struct futex_bucket *tmp =
+				futex_get_bucket((void __user *)uaddr,
+						 &futexv->objects[i].key, true);
+			if (IS_ERR(tmp)) {
+				__set_current_state(TASK_RUNNING);
+				futex_dequeue_multiple(futexv, i);
+				return PTR_ERR(tmp);
+			}
+			futexv->objects[i].bucket = tmp;
+		}
+
 		bucket = futexv->objects[i].bucket;
 
 		bucket_inc_waiters(bucket);
@@ -317,6 +522,7 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
 			if (__get_user(uval, uaddr))
 				return -EFAULT;
 
+			retry = true;
 			goto retry;
 		}
 
@@ -430,6 +636,7 @@ static int __futex_waitv(struct futex_waiter_head *futexv, unsigned int nr_futex
 static long ksys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
 			    struct __kernel_timespec __user *timo)
 {
+	bool shared = (flags & FUTEX_SHARED_FLAG) ? true : false;
 	unsigned int size = flags & FUTEX_SIZE_MASK;
 	struct futex_waiter *waiter;
 	struct futex_waiter_head *futexv;
@@ -459,7 +666,7 @@ static long ksys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
 	INIT_LIST_HEAD(&waiter->list);
 
 	/* Get an unlocked hash bucket */
-	waiter->bucket = futex_get_bucket(uaddr, &waiter->key);
+	waiter->bucket = futex_get_bucket(uaddr, &waiter->key, shared);
 	if (IS_ERR(waiter->bucket))
 		return PTR_ERR(waiter->bucket);
 
@@ -491,7 +698,6 @@ COMPAT_SYSCALL_DEFINE4(compat_futex_wait, void __user *, uaddr, compat_u64, val,
 SYSCALL_DEFINE4(futex_wait, void __user *, uaddr, u64, val, unsigned int, flags,
 		struct __kernel_timespec __user *, timo)
 {
-
 	return ksys_futex_wait(uaddr, val, flags, timo);
 }
 
@@ -554,6 +760,7 @@ static inline bool futex_match(struct futex_key key1, struct futex_key key2)
 SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
 		unsigned int, flags)
 {
+	bool shared = (flags & FUTEX_SHARED_FLAG) ? true : false;
 	unsigned int size = flags & FUTEX_SIZE_MASK;
 	struct futex_waiter waiter, *aux, *tmp;
 	struct futex_bucket *bucket;
@@ -566,7 +773,7 @@ SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
 	if (size != FUTEX_32)
 		return -EINVAL;
 
-	bucket = futex_get_bucket(uaddr, &waiter.key);
+	bucket = futex_get_bucket(uaddr, &waiter.key, shared);
 	if (IS_ERR(bucket))
 		return PTR_ERR(bucket);
 
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 03/15] futex2: Implement vectorized wait
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
  2021-06-03 19:59 ` [PATCH v4 01/15] futex2: Implement wait and wake functions André Almeida
  2021-06-03 19:59 ` [PATCH v4 02/15] futex2: Add support for shared futexes André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 04/15] futex2: Implement requeue operation André Almeida
                   ` (13 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Add support to wait on multiple futexes. This is the interface
implemented by this syscall:

futex_waitv(struct futex_waitv *waiters, unsigned int nr_futexes,
	    unsigned int flags, struct timespec *timo)

struct futex_waitv {
	__u64 val;
	void *uaddr;
	unsigned int flags;
};

Given an array of struct futex_waitv, wait on each uaddr. The thread
wakes if a futex_wake() is performed at any uaddr. The syscall returns
immediately if any waiter has *uaddr != val. *timo is an optional
timeout value for the operation. The flags argument of the syscall
should be used solely for specifying the timeout as realtime, if needed.
Flags for shared futexes, sizes, etc. should be used on the individual
flags of each waiter.

Returns the array index of one of the awakened futexes. There’s no given
information of how many were awakened, or any particular attribute of it
(if it’s the first awakened, if it is of the smaller index...).

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 arch/arm/tools/syscall.tbl                    |   1 +
 arch/arm64/include/asm/unistd.h               |   2 +-
 arch/arm64/include/asm/unistd32.h             |   2 +
 arch/x86/entry/syscalls/syscall_32.tbl        |   1 +
 arch/x86/entry/syscalls/syscall_64.tbl        |   1 +
 include/linux/compat.h                        |   9 +
 include/linux/syscalls.h                      |   4 +
 include/uapi/asm-generic/unistd.h             |   5 +-
 include/uapi/linux/futex.h                    |  14 ++
 kernel/futex2.c                               | 177 ++++++++++++++++++
 kernel/sys_ni.c                               |   2 +
 tools/include/uapi/asm-generic/unistd.h       |   5 +-
 .../arch/x86/entry/syscalls/syscall_64.tbl    |   1 +
 13 files changed, 221 insertions(+), 3 deletions(-)

diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl
index b60a8bdab623..6e476c34bd00 100644
--- a/arch/arm/tools/syscall.tbl
+++ b/arch/arm/tools/syscall.tbl
@@ -462,3 +462,4 @@
 446	common	landlock_restrict_self		sys_landlock_restrict_self
 447	common	futex_wait			sys_futex_wait
 448	common	futex_wake			sys_futex_wake
+449	common  futex_waitv                     sys_futex_waitv
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index 3cb206aea3db..6bdb5f5db438 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -38,7 +38,7 @@
 #define __ARM_NR_compat_set_tls		(__ARM_NR_COMPAT_BASE + 5)
 #define __ARM_NR_COMPAT_END		(__ARM_NR_COMPAT_BASE + 0x800)
 
-#define __NR_compat_syscalls		449
+#define __NR_compat_syscalls		450
 #endif
 
 #define __ARCH_WANT_SYS_CLONE
diff --git a/arch/arm64/include/asm/unistd32.h b/arch/arm64/include/asm/unistd32.h
index bca8835d7184..729083a76472 100644
--- a/arch/arm64/include/asm/unistd32.h
+++ b/arch/arm64/include/asm/unistd32.h
@@ -904,6 +904,8 @@ __SYSCALL(__NR_landlock_restrict_self, sys_landlock_restrict_self)
 __SYSCALL(__NR_futex_wait, compat_sys_futex_wait)
 #define __NR_futex_wake 448
 __SYSCALL(__NR_futex_wake, sys_futex_wake)
+#define __NR_futex_waitv 449
+__SYSCALL(__NR_futex_waitv, compat_sys_futex_waitv)
 
 /*
  * Please add new compat syscalls above this comment and update
diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl
index e3b827a9c094..5573437c1914 100644
--- a/arch/x86/entry/syscalls/syscall_32.tbl
+++ b/arch/x86/entry/syscalls/syscall_32.tbl
@@ -453,3 +453,4 @@
 446	i386	landlock_restrict_self	sys_landlock_restrict_self
 447	i386	futex_wait		sys_futex_wait			compat_sys_futex_wait
 448	i386	futex_wake		sys_futex_wake
+449	i386	futex_waitv		sys_futex_waitv			compat_sys_futex_waitv
diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl
index 63b447255df2..bad4aca3e9ba 100644
--- a/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/arch/x86/entry/syscalls/syscall_64.tbl
@@ -370,6 +370,7 @@
 446	common	landlock_restrict_self	sys_landlock_restrict_self
 447	common	futex_wait		sys_futex_wait
 448	common	futex_wake		sys_futex_wake
+449	common	futex_waitv		sys_futex_waitv
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
diff --git a/include/linux/compat.h b/include/linux/compat.h
index fe45135f3554..78e3c8d9689c 100644
--- a/include/linux/compat.h
+++ b/include/linux/compat.h
@@ -368,6 +368,12 @@ struct compat_robust_list_head {
 	compat_uptr_t			list_op_pending;
 };
 
+struct compat_futex_waitv {
+	compat_u64 val;
+	compat_uptr_t uaddr;
+	compat_uint_t flags;
+};
+
 #ifdef CONFIG_COMPAT_OLD_SIGACTION
 struct compat_old_sigaction {
 	compat_uptr_t			sa_handler;
@@ -696,6 +702,9 @@ compat_sys_get_robust_list(int pid, compat_uptr_t __user *head_ptr,
 asmlinkage long
 compat_sys_futex_wait(void __user *uaddr, compat_u64 val, unsigned int flags,
 		      struct __kernel_timespec __user *timo);
+asmlinkage long compat_sys_futex_waitv(struct compat_futex_waitv *waiters,
+				       compat_uint_t nr_futexes, compat_uint_t flags,
+				       struct __kernel_timespec __user *timo);
 
 /* kernel/itimer.c */
 asmlinkage long compat_sys_getitimer(int which,
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index b9c2874410d0..a24193d8b180 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -71,6 +71,7 @@ struct open_how;
 struct mount_attr;
 struct landlock_ruleset_attr;
 enum landlock_rule_type;
+struct futex_waitv;
 
 #include <linux/types.h>
 #include <linux/aio_abi.h>
@@ -628,6 +629,9 @@ asmlinkage long sys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
 			       struct __kernel_timespec __user *timo);
 asmlinkage long sys_futex_wake(void __user *uaddr, unsigned int nr_wake,
 			       unsigned int flags);
+asmlinkage long sys_futex_waitv(struct futex_waitv __user *waiters,
+				unsigned int nr_futexes, unsigned int flags,
+				struct __kernel_timespec __user *timo);
 
 /* kernel/hrtimer.c */
 asmlinkage long sys_nanosleep(struct __kernel_timespec __user *rqtp,
diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
index 50bfed52575f..debe684f648f 100644
--- a/include/uapi/asm-generic/unistd.h
+++ b/include/uapi/asm-generic/unistd.h
@@ -879,8 +879,11 @@ __SC_COMP(__NR_futex_wait, sys_futex_wait, compat_sys_futex_wait)
 #define __NR_futex_wake 448
 __SYSCALL(__NR_futex_wake, sys_futex_wake)
 
+#define __NR_futex_waitv 449
+__SC_COMP(__NR_futex_waitv, sys_futex_waitv, compat_sys_futex_waitv)
+
 #undef __NR_syscalls
-#define __NR_syscalls 449
+#define __NR_syscalls 450
 
 /*
  * 32 bit systems traditionally used different
diff --git a/include/uapi/linux/futex.h b/include/uapi/linux/futex.h
index 70ea66fffb1c..ca019b682b2e 100644
--- a/include/uapi/linux/futex.h
+++ b/include/uapi/linux/futex.h
@@ -48,6 +48,20 @@
 
 #define FUTEX_SHARED_FLAG 8
 
+#define FUTEX_WAITV_MAX 128
+
+/**
+ * struct futex_waitv - A waiter for vectorized wait
+ * @val:   Expected value at uaddr
+ * @uaddr: User address to wait on
+ * @flags: Flags for this waiter
+ */
+struct futex_waitv {
+	__u64 val;
+	void __user *uaddr;
+	unsigned int flags;
+};
+
 /*
  * Support for robust futexes: the kernel cleans up held futexes at
  * thread exit time.
diff --git a/kernel/futex2.c b/kernel/futex2.c
index f112c8c48f91..9c957c6bf699 100644
--- a/kernel/futex2.c
+++ b/kernel/futex2.c
@@ -82,6 +82,12 @@ struct futex_bucket {
 /* Mask for futex2 flag operations */
 #define FUTEX2_MASK (FUTEX_SIZE_MASK | FUTEX_CLOCK_REALTIME | FUTEX_SHARED_FLAG)
 
+/* Mask for sys_futex_waitv flag */
+#define FUTEXV_MASK (FUTEX_CLOCK_REALTIME)
+
+/* Mask for each futex in futex_waitv list */
+#define FUTEXV_WAITER_MASK (FUTEX_SIZE_MASK | FUTEX_SHARED_FLAG)
+
 #define is_object_shared ((futexv->objects[i].flags & FUTEX_SHARED_FLAG) ? true : false)
 
 #define FUT_OFF_INODE    1 /* We set bit 0 if key has a reference on inode */
@@ -701,6 +707,177 @@ SYSCALL_DEFINE4(futex_wait, void __user *, uaddr, u64, val, unsigned int, flags,
 	return ksys_futex_wait(uaddr, val, flags, timo);
 }
 
+#ifdef CONFIG_COMPAT
+/**
+ * compat_futex_parse_waitv - Parse a waitv array from userspace
+ * @futexv:	Kernel side list of waiters to be filled
+ * @uwaitv:     Userspace list to be parsed
+ * @nr_futexes: Length of futexv
+ *
+ * Return: Error code on failure, pointer to a prepared futexv otherwise
+ */
+static int compat_futex_parse_waitv(struct futex_waiter_head *futexv,
+				    struct compat_futex_waitv __user *uwaitv,
+				    unsigned int nr_futexes)
+{
+	struct compat_futex_waitv waitv;
+	struct futex_bucket *bucket;
+	unsigned int i;
+
+	for (i = 0; i < nr_futexes; i++) {
+		if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
+			return -EFAULT;
+
+		if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
+		    (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
+			return -EINVAL;
+
+		futexv->objects[i].key.pointer = 0;
+		futexv->objects[i].flags  = waitv.flags;
+		futexv->objects[i].uaddr  = compat_ptr(waitv.uaddr);
+		futexv->objects[i].val    = waitv.val;
+		futexv->objects[i].index  = i;
+
+		bucket = futex_get_bucket(compat_ptr(waitv.uaddr),
+					  &futexv->objects[i].key,
+					  is_object_shared);
+
+		if (IS_ERR(bucket))
+			return PTR_ERR(bucket);
+
+		futexv->objects[i].bucket = bucket;
+
+		INIT_LIST_HEAD(&futexv->objects[i].list);
+	}
+
+	return 0;
+}
+
+COMPAT_SYSCALL_DEFINE4(futex_waitv, struct compat_futex_waitv __user *, waiters,
+		       unsigned int, nr_futexes, unsigned int, flags,
+		       struct __kernel_timespec __user *, timo)
+{
+	struct futex_waiter_head *futexv;
+	int ret;
+
+	if (flags & ~FUTEXV_MASK)
+		return -EINVAL;
+
+	if (!nr_futexes || nr_futexes > FUTEX_WAITV_MAX || !waiters)
+		return -EINVAL;
+
+	futexv = kmalloc((sizeof(struct futex_waiter) * nr_futexes) +
+			 sizeof(*futexv), GFP_KERNEL);
+	if (!futexv)
+		return -ENOMEM;
+
+	futexv->hint = false;
+	futexv->task = current;
+
+	ret = compat_futex_parse_waitv(futexv, waiters, nr_futexes);
+
+	if (!ret)
+		ret = __futex_waitv(futexv, nr_futexes, timo, flags);
+
+	kfree(futexv);
+
+	return ret;
+}
+#endif
+
+/**
+ * futex_parse_waitv - Parse a waitv array from userspace
+ * @futexv:	Kernel side list of waiters to be filled
+ * @uwaitv:     Userspace list to be parsed
+ * @nr_futexes: Length of futexv
+ *
+ * Return: Error code on failure, pointer to a prepared futexv otherwise
+ */
+static int futex_parse_waitv(struct futex_waiter_head *futexv,
+			     struct futex_waitv __user *uwaitv,
+			     unsigned int nr_futexes)
+{
+	struct futex_bucket *bucket;
+	struct futex_waitv waitv;
+	unsigned int i;
+
+	for (i = 0; i < nr_futexes; i++) {
+		if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
+			return -EFAULT;
+
+		if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
+		    (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
+			return -EINVAL;
+
+		futexv->objects[i].key.pointer = 0;
+		futexv->objects[i].flags  = waitv.flags;
+		futexv->objects[i].uaddr  = waitv.uaddr;
+		futexv->objects[i].val    = waitv.val;
+		futexv->objects[i].index  = i;
+
+		bucket = futex_get_bucket(waitv.uaddr, &futexv->objects[i].key,
+					  is_object_shared);
+
+		if (IS_ERR(bucket))
+			return PTR_ERR(bucket);
+
+		futexv->objects[i].bucket = bucket;
+
+		INIT_LIST_HEAD(&futexv->objects[i].list);
+	}
+
+	return 0;
+}
+
+/**
+ * sys_futex_waitv - Wait on a list of futexes
+ * @waiters:    List of futexes to wait on
+ * @nr_futexes: Length of futexv
+ * @flags:      Flag for timeout (monotonic/realtime)
+ * @timo:	Optional absolute timeout.
+ *
+ * Given an array of `struct futex_waitv`, wait on each uaddr. The thread wakes
+ * if a futex_wake() is performed at any uaddr. The syscall returns immediately
+ * if any waiter has *uaddr != val. *timo is an optional timeout value for the
+ * operation. Each waiter has individual flags. The `flags` argument for the
+ * syscall should be used solely for specifying the timeout as realtime, if
+ * needed. Flags for shared futexes, sizes, etc. should be used on the
+ * individual flags of each waiter.
+ *
+ * Returns the array index of one of the awaken futexes. There's no given
+ * information of how many were awakened, or any particular attribute of it (if
+ * it's the first awakened, if it is of the smaller index...).
+ */
+SYSCALL_DEFINE4(futex_waitv, struct futex_waitv __user *, waiters,
+		unsigned int, nr_futexes, unsigned int, flags,
+		struct __kernel_timespec __user *, timo)
+{
+	struct futex_waiter_head *futexv;
+	int ret;
+
+	if (flags & ~FUTEXV_MASK)
+		return -EINVAL;
+
+	if (!nr_futexes || nr_futexes > FUTEX_WAITV_MAX || !waiters)
+		return -EINVAL;
+
+	futexv = kmalloc((sizeof(struct futex_waiter) * nr_futexes) +
+			 sizeof(*futexv), GFP_KERNEL);
+	if (!futexv)
+		return -ENOMEM;
+
+	futexv->hint = false;
+	futexv->task = current;
+
+	ret = futex_parse_waitv(futexv, waiters, nr_futexes);
+	if (!ret)
+		ret = __futex_waitv(futexv, nr_futexes, timo, flags);
+
+	kfree(futexv);
+
+	return ret;
+}
+
 /**
  * futex_get_parent - For a given futex in a futexv list, get a pointer to the futexv
  * @waiter: Address of futex in the list
diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
index dbe397eaea46..93807bb7be51 100644
--- a/kernel/sys_ni.c
+++ b/kernel/sys_ni.c
@@ -155,6 +155,8 @@ COND_SYSCALL_COMPAT(get_robust_list);
 COND_SYSCALL(futex_wait);
 COND_SYSCALL_COMPAT(futex_wait);
 COND_SYSCALL(futex_wake);
+COND_SYSCALL(futex_waitv);
+COND_SYSCALL_COMPAT(futex_waitv);
 
 /* kernel/hrtimer.c */
 
diff --git a/tools/include/uapi/asm-generic/unistd.h b/tools/include/uapi/asm-generic/unistd.h
index b48fd1899c3d..9a94140ef376 100644
--- a/tools/include/uapi/asm-generic/unistd.h
+++ b/tools/include/uapi/asm-generic/unistd.h
@@ -879,8 +879,11 @@ __SYSCALL(__NR_futex_wait, sys_futex_wait)
 #define __NR_futex_wake 448
 __SYSCALL(__NR_futex_wake, sys_futex_wake)
 
+#define __NR_futex_waitv 449
+__SC_COMP(__NR_futex_waitv, sys_futex_waitv, compat_sys_futex_waitv)
+
 #undef __NR_syscalls
-#define __NR_syscalls 449
+#define __NR_syscalls 450
 
 /*
  * 32 bit systems traditionally used different
diff --git a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
index 8eb17cc08a69..a5336eeffe45 100644
--- a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
@@ -370,6 +370,7 @@
 446	common	landlock_restrict_self	sys_landlock_restrict_self
 447	common  futex_wait              sys_futex_wait
 448	common  futex_wake              sys_futex_wake
+449	common  futex_waitv             sys_futex_waitv
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 04/15] futex2: Implement requeue operation
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (2 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 03/15] futex2: Implement vectorized wait André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 05/15] futex2: Implement support for different futex sizes André Almeida
                   ` (12 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Implement requeue interface similarly to FUTEX_CMP_REQUEUE operation.
This is the syscall implemented by this patch:

futex_requeue(struct futex_requeue *uaddr1, struct futex_requeue *uaddr2,
	      unsigned int nr_wake, unsigned int nr_requeue,
	      u64 cmpval, unsigned int flags)

struct futex_requeue {
	void *uaddr;
	unsigned int flags;
};

If (uaddr1->uaddr == cmpval), wake at uaddr1->uaddr a nr_wake number of
waiters and then, remove a number of nr_requeue waiters at uaddr1->uaddr
and add them to uaddr2->uaddr list. Each uaddr has its own set of flags,
that must be defined at struct futex_requeue (such as size, shared, NUMA).
The flags argument of the syscall is there just for the sake of
extensibility, and right now it needs to be zero.

Return the number of the woken futexes + the number of requeued ones on
success, error code otherwise.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---

The original FUTEX_CMP_REQUEUE interfaces is such as follows:

futex(*uaddr1, FUTEX_CMP_REQUEUE, nr_wake, nr_requeue, *uaddr2, cmpval);

Given that when this interface was created they was only one type of
futex (as opposed to futex2, where there is shared, sizes, and NUMA),
there was no way to specify individual flags for uaddr1 and 2. When
FUTEX_PRIVATE was implemented, a new opcode was created as well
(FUTEX_CMP_REQUEUE_PRIVATE), but they apply both futexes, so they
should be of the same type regarding private/shared. This imposes a
limitation on the use cases of the operation, and to overcome that at futex2,
`struct futex_requeue` was created, so one can set individual flags for
each futex. This flexibility is a trade-off with performance, given that
now we need to perform two extra copy_from_user(). One alternative would
be to use the upper half of flags bits to the first one, and the bottom
half for the second futex, but this would also impose limitations, given
that we would limit by half the flags possibilities. If equal futexes
are common enough, the following extension could be added to overcome
the current performance:

- A flag FUTEX_REQUEUE_EQUAL is added to futex2() flags;
- If futex_requeue() see this flag, that means that both futexes uses
  the same set of attributes.
- Then, the function parses the flags as of futex_wait/wake().
- *uaddr1 and *uaddr2 are used as void* (instead of struct
  futex_requeue) just like wait/wake().

In that way, we could avoid the copy_from_user().
---
 arch/arm/tools/syscall.tbl                    |   1 +
 arch/arm64/include/asm/unistd.h               |   2 +-
 arch/arm64/include/asm/unistd32.h             |   2 +
 arch/x86/entry/syscalls/syscall_32.tbl        |   1 +
 arch/x86/entry/syscalls/syscall_64.tbl        |   1 +
 include/linux/compat.h                        |  12 +
 include/linux/syscalls.h                      |   5 +
 include/uapi/asm-generic/unistd.h             |   5 +-
 include/uapi/linux/futex.h                    |  10 +
 kernel/futex2.c                               | 208 ++++++++++++++++++
 kernel/sys_ni.c                               |   2 +
 .../arch/x86/entry/syscalls/syscall_64.tbl    |   1 +
 12 files changed, 248 insertions(+), 2 deletions(-)

diff --git a/arch/arm/tools/syscall.tbl b/arch/arm/tools/syscall.tbl
index 6e476c34bd00..25f175ada125 100644
--- a/arch/arm/tools/syscall.tbl
+++ b/arch/arm/tools/syscall.tbl
@@ -463,3 +463,4 @@
 447	common	futex_wait			sys_futex_wait
 448	common	futex_wake			sys_futex_wake
 449	common  futex_waitv                     sys_futex_waitv
+450	common	futex_requeue			sys_futex_requeue
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index 6bdb5f5db438..4e65da3445c7 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -38,7 +38,7 @@
 #define __ARM_NR_compat_set_tls		(__ARM_NR_COMPAT_BASE + 5)
 #define __ARM_NR_COMPAT_END		(__ARM_NR_COMPAT_BASE + 0x800)
 
-#define __NR_compat_syscalls		450
+#define __NR_compat_syscalls		451
 #endif
 
 #define __ARCH_WANT_SYS_CLONE
diff --git a/arch/arm64/include/asm/unistd32.h b/arch/arm64/include/asm/unistd32.h
index 729083a76472..3c16e0d742ac 100644
--- a/arch/arm64/include/asm/unistd32.h
+++ b/arch/arm64/include/asm/unistd32.h
@@ -906,6 +906,8 @@ __SYSCALL(__NR_futex_wait, compat_sys_futex_wait)
 __SYSCALL(__NR_futex_wake, sys_futex_wake)
 #define __NR_futex_waitv 449
 __SYSCALL(__NR_futex_waitv, compat_sys_futex_waitv)
+#define __NR_futex_requeue 450
+__SYSCALL(__NR_futex_requeue, compat_sys_futex_requeue)
 
 /*
  * Please add new compat syscalls above this comment and update
diff --git a/arch/x86/entry/syscalls/syscall_32.tbl b/arch/x86/entry/syscalls/syscall_32.tbl
index 5573437c1914..f02c3da76945 100644
--- a/arch/x86/entry/syscalls/syscall_32.tbl
+++ b/arch/x86/entry/syscalls/syscall_32.tbl
@@ -454,3 +454,4 @@
 447	i386	futex_wait		sys_futex_wait			compat_sys_futex_wait
 448	i386	futex_wake		sys_futex_wake
 449	i386	futex_waitv		sys_futex_waitv			compat_sys_futex_waitv
+450	i386	futex_requeue		sys_futex_requeue		compat_sys_futex_requeue
diff --git a/arch/x86/entry/syscalls/syscall_64.tbl b/arch/x86/entry/syscalls/syscall_64.tbl
index bad4aca3e9ba..a1a39ed156e8 100644
--- a/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/arch/x86/entry/syscalls/syscall_64.tbl
@@ -371,6 +371,7 @@
 447	common	futex_wait		sys_futex_wait
 448	common	futex_wake		sys_futex_wake
 449	common	futex_waitv		sys_futex_waitv
+450	common	futex_requeue		sys_futex_requeue
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
diff --git a/include/linux/compat.h b/include/linux/compat.h
index 78e3c8d9689c..1425ef149dda 100644
--- a/include/linux/compat.h
+++ b/include/linux/compat.h
@@ -374,6 +374,11 @@ struct compat_futex_waitv {
 	compat_uint_t flags;
 };
 
+struct compat_futex_requeue {
+	compat_uptr_t uaddr;
+	compat_uint_t flags;
+};
+
 #ifdef CONFIG_COMPAT_OLD_SIGACTION
 struct compat_old_sigaction {
 	compat_uptr_t			sa_handler;
@@ -706,6 +711,13 @@ asmlinkage long compat_sys_futex_waitv(struct compat_futex_waitv *waiters,
 				       compat_uint_t nr_futexes, compat_uint_t flags,
 				       struct __kernel_timespec __user *timo);
 
+asmlinkage long compat_sys_futex_requeue(struct compat_futex_requeue *uaddr1,
+					 struct compat_futex_requeue *uaddr2,
+					 compat_uint_t nr_wake,
+					 compat_uint_t nr_requeue,
+					 compat_u64 cmpval,
+					 compat_uint_t flags);
+
 /* kernel/itimer.c */
 asmlinkage long compat_sys_getitimer(int which,
 				     struct old_itimerval32 __user *it);
diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index a24193d8b180..c108df6b3b82 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -72,6 +72,7 @@ struct mount_attr;
 struct landlock_ruleset_attr;
 enum landlock_rule_type;
 struct futex_waitv;
+struct futex_requeue;
 
 #include <linux/types.h>
 #include <linux/aio_abi.h>
@@ -632,6 +633,10 @@ asmlinkage long sys_futex_wake(void __user *uaddr, unsigned int nr_wake,
 asmlinkage long sys_futex_waitv(struct futex_waitv __user *waiters,
 				unsigned int nr_futexes, unsigned int flags,
 				struct __kernel_timespec __user *timo);
+asmlinkage long sys_futex_requeue(struct futex_requeue __user *uaddr1,
+				  struct futex_requeue __user *uaddr2,
+				  unsigned int nr_wake, unsigned int nr_requeue,
+				  u64 cmpval, unsigned int flags);
 
 /* kernel/hrtimer.c */
 asmlinkage long sys_nanosleep(struct __kernel_timespec __user *rqtp,
diff --git a/include/uapi/asm-generic/unistd.h b/include/uapi/asm-generic/unistd.h
index debe684f648f..5455c0be7798 100644
--- a/include/uapi/asm-generic/unistd.h
+++ b/include/uapi/asm-generic/unistd.h
@@ -882,8 +882,11 @@ __SYSCALL(__NR_futex_wake, sys_futex_wake)
 #define __NR_futex_waitv 449
 __SC_COMP(__NR_futex_waitv, sys_futex_waitv, compat_sys_futex_waitv)
 
+#define __NR_futex_requeue 450
+__SC_COMP(__NR_futex_requeue, sys_futex_requeue, compat_sys_futex_requeue)
+
 #undef __NR_syscalls
-#define __NR_syscalls 450
+#define __NR_syscalls 451
 
 /*
  * 32 bit systems traditionally used different
diff --git a/include/uapi/linux/futex.h b/include/uapi/linux/futex.h
index ca019b682b2e..06ea9bdfa69e 100644
--- a/include/uapi/linux/futex.h
+++ b/include/uapi/linux/futex.h
@@ -62,6 +62,16 @@ struct futex_waitv {
 	unsigned int flags;
 };
 
+/**
+ * struct futex_requeue - Define an address and its flags for requeue operation
+ * @uaddr: User address of one of the requeue arguments
+ * @flags: Flags for this address
+ */
+struct futex_requeue {
+	void __user *uaddr;
+	unsigned int flags;
+};
+
 /*
  * Support for robust futexes: the kernel cleans up held futexes at
  * thread exit time.
diff --git a/kernel/futex2.c b/kernel/futex2.c
index 9c957c6bf699..012d7f7fc17a 100644
--- a/kernel/futex2.c
+++ b/kernel/futex2.c
@@ -972,6 +972,214 @@ SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
 	return ret;
 }
 
+static void futex_double_unlock(struct futex_bucket *b1, struct futex_bucket *b2)
+{
+	spin_unlock(&b1->lock);
+	if (b1 != b2)
+		spin_unlock(&b2->lock);
+}
+
+static inline int __futex_requeue(struct futex_requeue rq1,
+				  struct futex_requeue rq2, unsigned int nr_wake,
+				  unsigned int nr_requeue, u64 cmpval)
+{
+	struct futex_waiter w1, w2, *aux, *tmp;
+	bool retry = false;
+	struct futex_bucket *b1, *b2;
+	DEFINE_WAKE_Q(wake_q);
+	u32 uval;
+	int ret;
+	bool shared1 = (rq1.flags  & FUTEX_SHARED_FLAG) ? true : false;
+	bool shared2 = (rq2.flags  & FUTEX_SHARED_FLAG) ? true : false;
+
+	b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
+	if (IS_ERR(b1))
+		return PTR_ERR(b1);
+
+	b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
+	if (IS_ERR(b2))
+		return PTR_ERR(b2);
+
+retry:
+	if (shared1 && retry) {
+		b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
+		if (IS_ERR(b1))
+			return PTR_ERR(b1);
+	}
+
+	if (shared2 && retry) {
+		b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
+		if (IS_ERR(b2))
+			return PTR_ERR(b2);
+	}
+
+	bucket_inc_waiters(b2);
+	/*
+	 * To ensure the locks are taken in the same order for all threads (and
+	 * thus avoiding deadlocks), take the "smaller" one first
+	 */
+	if (b1 <= b2) {
+		spin_lock(&b1->lock);
+		if (b1 < b2)
+			spin_lock_nested(&b2->lock, SINGLE_DEPTH_NESTING);
+	} else {
+		spin_lock(&b2->lock);
+		spin_lock_nested(&b1->lock, SINGLE_DEPTH_NESTING);
+	}
+
+	ret = futex_get_user(&uval, rq1.uaddr);
+
+	if (unlikely(ret)) {
+		futex_double_unlock(b1, b2);
+		if (__get_user(uval, (u32 __user *)rq1.uaddr))
+			return -EFAULT;
+
+		bucket_dec_waiters(b2);
+		retry = true;
+		goto retry;
+	}
+
+	if (uval != cmpval) {
+		futex_double_unlock(b1, b2);
+
+		bucket_dec_waiters(b2);
+		return -EAGAIN;
+	}
+
+	list_for_each_entry_safe(aux, tmp, &b1->list, list) {
+		if (futex_match(w1.key, aux->key)) {
+			if (ret < nr_wake) {
+				futex_mark_wake(aux, b1, &wake_q);
+				ret++;
+				continue;
+			}
+
+			if (ret >= nr_wake + nr_requeue)
+				break;
+
+			aux->key.pointer = w2.key.pointer;
+			aux->key.index = w2.key.index;
+			aux->key.offset = w2.key.offset;
+
+			if (b1 != b2) {
+				list_del_init(&aux->list);
+				bucket_dec_waiters(b1);
+
+				list_add_tail(&aux->list, &b2->list);
+				bucket_inc_waiters(b2);
+			}
+			ret++;
+		}
+	}
+
+	futex_double_unlock(b1, b2);
+	wake_up_q(&wake_q);
+	bucket_dec_waiters(b2);
+
+	return ret;
+}
+
+#ifdef CONFIG_COMPAT
+static int compat_futex_parse_requeue(struct futex_requeue *rq,
+				      struct compat_futex_requeue __user *uaddr)
+{
+	struct compat_futex_requeue tmp;
+
+	if (copy_from_user(&tmp, uaddr, sizeof(tmp)))
+		return -EFAULT;
+
+	if (tmp.flags & ~FUTEXV_WAITER_MASK ||
+	    (tmp.flags & FUTEX_SIZE_MASK) != FUTEX_32)
+		return -EINVAL;
+
+	rq->uaddr = compat_ptr(tmp.uaddr);
+	rq->flags = tmp.flags;
+
+	return 0;
+}
+
+COMPAT_SYSCALL_DEFINE6(futex_requeue, struct compat_futex_requeue __user *, uaddr1,
+		       struct compat_futex_requeue __user *, uaddr2,
+		       unsigned int, nr_wake, unsigned int, nr_requeue,
+		       compat_u64, cmpval, unsigned int, flags)
+{
+	struct futex_requeue rq1, rq2;
+	int ret;
+
+	if (flags)
+		return -EINVAL;
+
+	ret = compat_futex_parse_requeue(&rq1, uaddr1);
+	if (ret)
+		return ret;
+
+	ret = compat_futex_parse_requeue(&rq2, uaddr2);
+	if (ret)
+		return ret;
+
+	return __futex_requeue(rq1, rq2, nr_wake, nr_requeue, cmpval);
+}
+#endif
+
+/**
+ * futex_parse_requeue - Copy a user struct futex_requeue and check it's flags
+ * @rq:    Kernel struct
+ * @uaddr: Address of user struct
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static int futex_parse_requeue(struct futex_requeue *rq,
+			       struct futex_requeue __user *uaddr)
+{
+	if (copy_from_user(rq, uaddr, sizeof(*rq)))
+		return -EFAULT;
+
+	if (rq->flags & ~FUTEXV_WAITER_MASK ||
+	    (rq->flags & FUTEX_SIZE_MASK) != FUTEX_32)
+		return -EINVAL;
+
+	return 0;
+}
+
+/**
+ * sys_futex_requeue - Wake futexes at uaddr1 and requeue from uaddr1 to uaddr2
+ * @uaddr1:	Address of futexes to be waken/dequeued
+ * @uaddr2:	Address for the futexes to be enqueued
+ * @nr_wake:	Number of futexes waiting in uaddr1 to be woken up
+ * @nr_requeue: Number of futexes to be requeued from uaddr1 to uaddr2
+ * @cmpval:	Expected value at uaddr1
+ * @flags:	Reserved flags arg for requeue operation expansion. Must be 0.
+ *
+ * If (uaddr1->uaddr == cmpval), wake at uaddr1->uaddr a nr_wake number of
+ * waiters and then, remove a number of nr_requeue waiters at uaddr1->uaddr
+ * and add then to uaddr2->uaddr list. Each uaddr has its own set of flags,
+ * that must be defined at struct futex_requeue (such as size, shared, NUMA).
+ *
+ * Return the number of the woken futexes + the number of requeued ones on
+ * success, error code otherwise.
+ */
+SYSCALL_DEFINE6(futex_requeue, struct futex_requeue __user *, uaddr1,
+		struct futex_requeue __user *, uaddr2,
+		unsigned int, nr_wake, unsigned int, nr_requeue,
+		u64, cmpval, unsigned int, flags)
+{
+	struct futex_requeue rq1, rq2;
+	int ret;
+
+	if (flags)
+		return -EINVAL;
+
+	ret = futex_parse_requeue(&rq1, uaddr1);
+	if (ret)
+		return ret;
+
+	ret = futex_parse_requeue(&rq2, uaddr2);
+	if (ret)
+		return ret;
+
+	return __futex_requeue(rq1, rq2, nr_wake, nr_requeue, cmpval);
+}
+
 static int __init futex2_init(void)
 {
 	int i;
diff --git a/kernel/sys_ni.c b/kernel/sys_ni.c
index 93807bb7be51..20a425b79fca 100644
--- a/kernel/sys_ni.c
+++ b/kernel/sys_ni.c
@@ -157,6 +157,8 @@ COND_SYSCALL_COMPAT(futex_wait);
 COND_SYSCALL(futex_wake);
 COND_SYSCALL(futex_waitv);
 COND_SYSCALL_COMPAT(futex_waitv);
+COND_SYSCALL(futex_requeue);
+COND_SYSCALL_COMPAT(futex_requeue);
 
 /* kernel/hrtimer.c */
 
diff --git a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
index a5336eeffe45..dd371799843c 100644
--- a/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
+++ b/tools/perf/arch/x86/entry/syscalls/syscall_64.tbl
@@ -371,6 +371,7 @@
 447	common  futex_wait              sys_futex_wait
 448	common  futex_wake              sys_futex_wake
 449	common  futex_waitv             sys_futex_waitv
+450	common  futex_requeue		sys_futex_requeue
 
 #
 # Due to a historical design error, certain syscalls are numbered differently
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 05/15] futex2: Implement support for different futex sizes
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (3 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 04/15] futex2: Implement requeue operation André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-04  0:23   ` kernel test robot
  2021-06-06 19:12   ` Davidlohr Bueso
  2021-06-03 19:59 ` [PATCH v4 06/15] futex2: Add compatibility entry point for x86_x32 ABI André Almeida
                   ` (11 subsequent siblings)
  16 siblings, 2 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Implement support for 8, 16 and 64 bit futexes, along with the existing
32 bit support. Userspace should use flags to specify in the syscall
the size of the *uaddr they are operating on.

Variable sized futexes are useful for implementing atomic primitives in
userspace in an efficient manner. 64bit sized futexes are also
particularly useful when userspace stores information to be used in an
atomic fashion on the futex value, given more room for flexibility.

Overlapping futexes are not allowed, so userspace can't wait and wake on
the same memory address if the are using different sizes.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 include/uapi/linux/futex.h |   3 +
 kernel/futex2.c            | 124 ++++++++++++++++++++++++-------------
 2 files changed, 84 insertions(+), 43 deletions(-)

diff --git a/include/uapi/linux/futex.h b/include/uapi/linux/futex.h
index 06ea9bdfa69e..5786270b0c75 100644
--- a/include/uapi/linux/futex.h
+++ b/include/uapi/linux/futex.h
@@ -42,7 +42,10 @@
 					 FUTEX_PRIVATE_FLAG)
 
 /* Size argument to futex2 syscall */
+#define FUTEX_8		0
+#define FUTEX_16	1
 #define FUTEX_32	2
+#define FUTEX_64	3
 
 #define FUTEX_SIZE_MASK	0x3
 
diff --git a/kernel/futex2.c b/kernel/futex2.c
index 012d7f7fc17a..1e97e5f2e793 100644
--- a/kernel/futex2.c
+++ b/kernel/futex2.c
@@ -89,9 +89,11 @@ struct futex_bucket {
 #define FUTEXV_WAITER_MASK (FUTEX_SIZE_MASK | FUTEX_SHARED_FLAG)
 
 #define is_object_shared ((futexv->objects[i].flags & FUTEX_SHARED_FLAG) ? true : false)
+#define object_size (futexv->objects[i].flags & FUTEX_SIZE_MASK)
 
-#define FUT_OFF_INODE    1 /* We set bit 0 if key has a reference on inode */
-#define FUT_OFF_MMSHARED 2 /* We set bit 1 if key has a reference on mm */
+#define FUT_OFF_INODE    PAGE_SIZE
+#define FUT_OFF_MMSHARED (PAGE_SIZE << 1)
+#define FUT_OFF_SIZE     1
 
 static struct futex_bucket *futex_table;
 static unsigned int futex2_hashsize;
@@ -321,6 +323,7 @@ static int futex_get_shared_key(uintptr_t address, struct mm_struct *mm,
  * @uaddr:   futex user address
  * @key:     data that uniquely identifies a futex
  * @shared:  is this a shared futex?
+ * @flags:   flags for the size
  *
  * For private futexes, each uaddr will be unique for a given mm_struct, and it
  * won't be freed for the life time of the process. For shared futexes, check
@@ -330,21 +333,41 @@ static int futex_get_shared_key(uintptr_t address, struct mm_struct *mm,
  */
 static struct futex_bucket *futex_get_bucket(void __user *uaddr,
 					     struct futex_key *key,
-					     bool shared)
+					     bool shared, unsigned int flags)
 {
 	uintptr_t address = (uintptr_t)uaddr;
 	u32 hash_key;
 
+	size_t size;
+
+	switch (flags) {
+	case FUTEX_8:
+		size = sizeof(u8);
+		break;
+	case FUTEX_16:
+		size = sizeof(u16);
+		break;
+	case FUTEX_32:
+		size = sizeof(u32);
+		break;
+	case FUTEX_64:
+		size = sizeof(u64);
+		break;
+	default:
+		return ERR_PTR(-EINVAL);
+	}
+
 	/* Checking if uaddr is valid and accessible */
-	if (unlikely(!IS_ALIGNED(address, sizeof(u32))))
+	if (unlikely(!IS_ALIGNED(address, size)))
 		return ERR_PTR(-EINVAL);
-	if (unlikely(!access_ok(uaddr, sizeof(u32))))
+	if (unlikely(!access_ok(uaddr, size)))
 		return ERR_PTR(-EFAULT);
 
 	key->offset = address % PAGE_SIZE;
 	address -= key->offset;
 	key->pointer = (u64)address;
 	key->index = (unsigned long)current->mm;
+	key->offset |= FUT_OFF_SIZE << (size - sizeof(u8));
 
 	if (shared)
 		futex_get_shared_key(address, current->mm, key);
@@ -358,18 +381,39 @@ static struct futex_bucket *futex_get_bucket(void __user *uaddr,
 
 /**
  * futex_get_user - Get the userspace value on this address
- * @uval:  variable to store the value
- * @uaddr: userspace address
+ * @uval:	variable to store the value
+ * @uaddr:	userspace address
+ * @pagefault:	true if pagefault should be disabled
+ * @flags:	flags for the size
  *
  * Check the comment at futex_enqueue() for more information.
  */
-static int futex_get_user(u32 *uval, u32 __user *uaddr)
+static int futex_get_user(u64 *uval, void __user *uaddr, unsigned int flags, bool pagefault)
 {
 	int ret;
 
-	pagefault_disable();
-	ret = __get_user(*uval, uaddr);
-	pagefault_enable();
+	if (pagefault)
+		pagefault_disable();
+
+	switch (flags) {
+	case FUTEX_8:
+		ret = __get_user(*uval, (u8 __user *)uaddr);
+		break;
+	case FUTEX_16:
+		ret = __get_user(*uval, (u16 __user *)uaddr);
+		break;
+	case FUTEX_32:
+		ret = __get_user(*uval, (u32 __user *)uaddr);
+		break;
+	case FUTEX_64:
+		ret = __get_user(*uval, (u64 __user *)uaddr);
+		break;
+	default:
+		BUG();
+	}
+
+	if (pagefault)
+		pagefault_enable();
 
 	return ret;
 }
@@ -484,8 +528,8 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
 			 int *awakened)
 {
 	int i, ret;
-	u32 uval, val;
-	u32 __user *uaddr;
+	u64 uval, val;
+	void __user *uaddr;
 	bool retry = false;
 	struct futex_bucket *bucket;
 
@@ -493,13 +537,14 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
 	set_current_state(TASK_INTERRUPTIBLE);
 
 	for (i = 0; i < nr_futexes; i++) {
-		uaddr = (u32 __user *)futexv->objects[i].uaddr;
-		val = (u32)futexv->objects[i].val;
+		uaddr = futexv->objects[i].uaddr;
+		val = (u64)futexv->objects[i].val;
 
 		if (is_object_shared && retry) {
 			struct futex_bucket *tmp =
 				futex_get_bucket((void __user *)uaddr,
-						 &futexv->objects[i].key, true);
+						 &futexv->objects[i].key, true,
+						 object_size);
 			if (IS_ERR(tmp)) {
 				__set_current_state(TASK_RUNNING);
 				futex_dequeue_multiple(futexv, i);
@@ -513,7 +558,7 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
 		bucket_inc_waiters(bucket);
 		spin_lock(&bucket->lock);
 
-		ret = futex_get_user(&uval, uaddr);
+		ret = futex_get_user(&uval, uaddr, object_size, true);
 
 		if (unlikely(ret)) {
 			spin_unlock(&bucket->lock);
@@ -525,7 +570,7 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
 			if (*awakened >= 0)
 				return 1;
 
-			if (__get_user(uval, uaddr))
+			if (futex_get_user(&uval, uaddr, object_size, false))
 				return -EFAULT;
 
 			retry = true;
@@ -656,9 +701,6 @@ static long ksys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
 	if (flags & ~FUTEX2_MASK)
 		return -EINVAL;
 
-	if (size != FUTEX_32)
-		return -EINVAL;
-
 	futexv = &wait_single.futexv;
 	futexv->task = current;
 	futexv->hint = false;
@@ -667,12 +709,13 @@ static long ksys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
 	waiter->index = 0;
 	waiter->val = val;
 	waiter->uaddr = uaddr;
+	waiter->flags = flags;
 	memset(&wait_single.waiter.key, 0, sizeof(struct futex_key));
 
 	INIT_LIST_HEAD(&waiter->list);
 
 	/* Get an unlocked hash bucket */
-	waiter->bucket = futex_get_bucket(uaddr, &waiter->key, shared);
+	waiter->bucket = futex_get_bucket(uaddr, &waiter->key, shared, size);
 	if (IS_ERR(waiter->bucket))
 		return PTR_ERR(waiter->bucket);
 
@@ -728,8 +771,7 @@ static int compat_futex_parse_waitv(struct futex_waiter_head *futexv,
 		if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
 			return -EFAULT;
 
-		if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
-		    (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
+		if (waitv.flags & ~FUTEXV_WAITER_MASK)
 			return -EINVAL;
 
 		futexv->objects[i].key.pointer = 0;
@@ -740,7 +782,7 @@ static int compat_futex_parse_waitv(struct futex_waiter_head *futexv,
 
 		bucket = futex_get_bucket(compat_ptr(waitv.uaddr),
 					  &futexv->objects[i].key,
-					  is_object_shared);
+					  is_object_shared, object_size);
 
 		if (IS_ERR(bucket))
 			return PTR_ERR(bucket);
@@ -805,8 +847,7 @@ static int futex_parse_waitv(struct futex_waiter_head *futexv,
 		if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
 			return -EFAULT;
 
-		if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
-		    (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
+		if (waitv.flags & ~FUTEXV_WAITER_MASK)
 			return -EINVAL;
 
 		futexv->objects[i].key.pointer = 0;
@@ -816,7 +857,7 @@ static int futex_parse_waitv(struct futex_waiter_head *futexv,
 		futexv->objects[i].index  = i;
 
 		bucket = futex_get_bucket(waitv.uaddr, &futexv->objects[i].key,
-					  is_object_shared);
+					  is_object_shared, object_size);
 
 		if (IS_ERR(bucket))
 			return PTR_ERR(bucket);
@@ -947,10 +988,7 @@ SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
 	if (flags & ~FUTEX2_MASK)
 		return -EINVAL;
 
-	if (size != FUTEX_32)
-		return -EINVAL;
-
-	bucket = futex_get_bucket(uaddr, &waiter.key, shared);
+	bucket = futex_get_bucket(uaddr, &waiter.key, shared, size);
 	if (IS_ERR(bucket))
 		return PTR_ERR(bucket);
 
@@ -987,28 +1025,30 @@ static inline int __futex_requeue(struct futex_requeue rq1,
 	bool retry = false;
 	struct futex_bucket *b1, *b2;
 	DEFINE_WAKE_Q(wake_q);
-	u32 uval;
+	u64 uval;
 	int ret;
 	bool shared1 = (rq1.flags  & FUTEX_SHARED_FLAG) ? true : false;
 	bool shared2 = (rq2.flags  & FUTEX_SHARED_FLAG) ? true : false;
+	unsigned int size1 = (rq1.flags  & FUTEX_SIZE_MASK);
+	unsigned int size2 = (rq2.flags  & FUTEX_SIZE_MASK);
 
-	b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
+	b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1, size1);
 	if (IS_ERR(b1))
 		return PTR_ERR(b1);
 
-	b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
+	b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2, size2);
 	if (IS_ERR(b2))
 		return PTR_ERR(b2);
 
 retry:
 	if (shared1 && retry) {
-		b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
+		b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1, size1);
 		if (IS_ERR(b1))
 			return PTR_ERR(b1);
 	}
 
 	if (shared2 && retry) {
-		b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
+		b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2, size2);
 		if (IS_ERR(b2))
 			return PTR_ERR(b2);
 	}
@@ -1027,11 +1067,11 @@ static inline int __futex_requeue(struct futex_requeue rq1,
 		spin_lock_nested(&b1->lock, SINGLE_DEPTH_NESTING);
 	}
 
-	ret = futex_get_user(&uval, rq1.uaddr);
+	ret = futex_get_user(&uval, rq1.uaddr, size1, true);
 
 	if (unlikely(ret)) {
 		futex_double_unlock(b1, b2);
-		if (__get_user(uval, (u32 __user *)rq1.uaddr))
+		if (futex_get_user(&uval, rq1.uaddr, size1, false))
 			return -EFAULT;
 
 		bucket_dec_waiters(b2);
@@ -1088,8 +1128,7 @@ static int compat_futex_parse_requeue(struct futex_requeue *rq,
 	if (copy_from_user(&tmp, uaddr, sizeof(tmp)))
 		return -EFAULT;
 
-	if (tmp.flags & ~FUTEXV_WAITER_MASK ||
-	    (tmp.flags & FUTEX_SIZE_MASK) != FUTEX_32)
+	if (tmp.flags & ~FUTEXV_WAITER_MASK)
 		return -EINVAL;
 
 	rq->uaddr = compat_ptr(tmp.uaddr);
@@ -1134,8 +1173,7 @@ static int futex_parse_requeue(struct futex_requeue *rq,
 	if (copy_from_user(rq, uaddr, sizeof(*rq)))
 		return -EFAULT;
 
-	if (rq->flags & ~FUTEXV_WAITER_MASK ||
-	    (rq->flags & FUTEX_SIZE_MASK) != FUTEX_32)
+	if (rq->flags & ~FUTEXV_WAITER_MASK)
 		return -EINVAL;
 
 	return 0;
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 06/15] futex2: Add compatibility entry point for x86_x32 ABI
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (4 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 05/15] futex2: Implement support for different futex sizes André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 07/15] docs: locking: futex2: Add documentation André Almeida
                   ` (10 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

New syscalls should use the same entry point for x86_64 and x86_x32
paths. Add a wrapper for x32 calls to use parse functions that assumes
32bit pointers.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 kernel/futex2.c | 36 +++++++++++++++++++++++++++++++++++-
 1 file changed, 35 insertions(+), 1 deletion(-)

diff --git a/kernel/futex2.c b/kernel/futex2.c
index 1e97e5f2e793..5fd0b3d73b53 100644
--- a/kernel/futex2.c
+++ b/kernel/futex2.c
@@ -23,6 +23,10 @@
 #include <linux/syscalls.h>
 #include <uapi/linux/futex.h>
 
+#ifdef CONFIG_X86_64
+#include <linux/compat.h>
+#endif
+
 /**
  * struct futex_key - Components to build unique key for a futex
  * @pointer: Pointer to current->mm or inode's UUID for file backed futexes
@@ -910,7 +914,16 @@ SYSCALL_DEFINE4(futex_waitv, struct futex_waitv __user *, waiters,
 	futexv->hint = false;
 	futexv->task = current;
 
-	ret = futex_parse_waitv(futexv, waiters, nr_futexes);
+#ifdef CONFIG_X86_X32_ABI
+	if (in_x32_syscall()) {
+		ret = compat_futex_parse_waitv(futexv, (struct compat_futex_waitv *)waiters,
+					       nr_futexes);
+	} else
+#endif
+	{
+		ret = futex_parse_waitv(futexv, waiters, nr_futexes);
+	}
+
 	if (!ret)
 		ret = __futex_waitv(futexv, nr_futexes, timo, flags);
 
@@ -1215,6 +1228,27 @@ SYSCALL_DEFINE6(futex_requeue, struct futex_requeue __user *, uaddr1,
 	if (ret)
 		return ret;
 
+#ifdef CONFIG_X86_X32_ABI
+	if (in_x32_syscall()) {
+		ret = compat_futex_parse_requeue(&rq1, (struct compat_futex_requeue *)uaddr1);
+		if (ret)
+			return ret;
+
+		ret = compat_futex_parse_requeue(&rq2, (struct compat_futex_requeue *)uaddr2);
+		if (ret)
+			return ret;
+	} else
+#endif
+	{
+		ret = futex_parse_requeue(&rq1, uaddr1);
+		if (ret)
+			return ret;
+
+		ret = futex_parse_requeue(&rq2, uaddr2);
+		if (ret)
+			return ret;
+	}
+
 	return __futex_requeue(rq1, rq2, nr_wake, nr_requeue, cmpval);
 }
 
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 07/15] docs: locking: futex2: Add documentation
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (5 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 06/15] futex2: Add compatibility entry point for x86_x32 ABI André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-06 19:23   ` Davidlohr Bueso
  2021-06-03 19:59 ` [PATCH v4 08/15] selftests: futex2: Add wake/wait test André Almeida
                   ` (9 subsequent siblings)
  16 siblings, 1 reply; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Add a new documentation file specifying both userspace API and internal
implementation details of futex2 syscalls.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 Documentation/locking/futex2.rst | 198 +++++++++++++++++++++++++++++++
 Documentation/locking/index.rst  |   1 +
 2 files changed, 199 insertions(+)
 create mode 100644 Documentation/locking/futex2.rst

diff --git a/Documentation/locking/futex2.rst b/Documentation/locking/futex2.rst
new file mode 100644
index 000000000000..2f74d7c97a55
--- /dev/null
+++ b/Documentation/locking/futex2.rst
@@ -0,0 +1,198 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======
+futex2
+======
+
+:Author: André Almeida <andrealmeid@collabora.com>
+
+futex, or fast user mutex, is a set of syscalls to allow userspace to create
+performant synchronization mechanisms, such as mutexes, semaphores and
+conditional variables in userspace. C standard libraries, like glibc, uses it
+as a means to implement more high level interfaces like pthreads.
+
+The interface
+=============
+
+uAPI functions
+--------------
+
+.. kernel-doc:: kernel/futex2.c
+   :identifiers: sys_futex_wait sys_futex_wake sys_futex_waitv sys_futex_requeue
+
+uAPI structures
+---------------
+
+.. kernel-doc:: include/uapi/linux/futex.h
+
+The ``flag`` argument
+---------------------
+
+The flag is used to specify the size of the futex word
+(FUTEX_[8, 16, 32, 64]). It's mandatory to define one, since there's no
+default size.
+
+By default, the timeout uses a monotonic clock, but can be used as a realtime
+one by using the FUTEX_REALTIME_CLOCK flag.
+
+By default, futexes are of the private type, that means that this user address
+will be accessed by threads that share the same memory region. This allows for
+some internal optimizations, so they are faster. However, if the address needs
+to be shared with different processes (like using ``mmap()`` or ``shm()``), they
+need to be defined as shared and the flag FUTEX_SHARED_FLAG is used to set that.
+
+By default, the operation has no NUMA-awareness, meaning that the user can't
+choose the memory node where the kernel side futex data will be stored. The
+user can choose the node where it wants to operate by setting the
+FUTEX_NUMA_FLAG and using the following structure (where X can be 8, 16, 32 or
+64)::
+
+ struct futexX_numa {
+         __uX value;
+         __sX hint;
+ };
+
+This structure should be passed at the ``void *uaddr`` of futex functions. The
+address of the structure will be used to be waited on/waken on, and the
+``value`` will be compared to ``val`` as usual. The ``hint`` member is used to
+define which node the futex will use. When waiting, the futex will be
+registered on a kernel-side table stored on that node; when waking, the futex
+will be searched for on that given table. That means that there's no redundancy
+between tables, and the wrong ``hint`` value will lead to undesired behavior.
+Userspace is responsible for dealing with node migrations issues that may
+occur. ``hint`` can range from [0, MAX_NUMA_NODES), for specifying a node, or
+-1, to use the same node the current process is using.
+
+When not using FUTEX_NUMA_FLAG on a NUMA system, the futex will be stored on a
+global table on allocated on the first node.
+
+The ``timo`` argument
+---------------------
+
+As per the Y2038 work done in the kernel, new interfaces shouldn't add timeout
+options known to be buggy. Given that, ``timo`` should be a 64-bit timeout at
+all platforms, using an absolute timeout value.
+
+Implementation
+==============
+
+The internal implementation follows a similar design to the original futex.
+Given that we want to replicate the same external behavior of current futex,
+this should be somewhat expected.
+
+Waiting
+-------
+
+For the wait operations, they are all treated as if you want to wait on N
+futexes, so the path for futex_wait and futex_waitv is the basically the same.
+For both syscalls, the first step is to prepare an internal list for the list
+of futexes to wait for (using struct futexv_head). For futex_wait() calls, this
+list will have a single object.
+
+We have a hash table, where waiters register themselves before sleeping. Then
+the wake function checks this table looking for waiters at uaddr.  The hash
+bucket to be used is determined by a struct futex_key, that stores information
+to uniquely identify an address from a given process. Given the huge address
+space, there'll be hash collisions, so we store information to be later used on
+collision treatment.
+
+First, for every futex we want to wait on, we check if (``*uaddr == val``).
+This check is done holding the bucket lock, so we are correctly serialized with
+any futex_wake() calls. If any waiter fails the check above, we dequeue all
+futexes. The check (``*uaddr == val``) can fail for two reasons:
+
+- The values are different, and we return -EAGAIN. However, if while
+  dequeueing we found that some futexes were awakened, we prioritize this
+  and return success.
+
+- When trying to access the user address, we do so with page faults
+  disabled because we are holding a bucket's spin lock (and can't sleep
+  while holding a spin lock). If there's an error, it might be a page
+  fault, or an invalid address. We release the lock, dequeue everyone
+  (because it's illegal to sleep while there are futexes enqueued, we
+  could lose wakeups) and try again with page fault enabled. If we
+  succeed, this means that the address is valid, but we need to do
+  all the work again. For serialization reasons, we need to have the
+  spin lock when getting the user value. Additionally, for shared
+  futexes, we also need to recalculate the hash, since the underlying
+  mapping mechanisms could have changed when dealing with page fault.
+  If, even with page fault enabled, we can't access the address, it
+  means it's an invalid user address, and we return -EFAULT. For this
+  case, we prioritize the error, even if some futexes were awaken.
+
+If the check is OK, they are enqueued on a linked list in our bucket, and
+proceed to the next one. If all waiters succeed, we put the thread to sleep
+until a futex_wake() call, timeout expires or we get a signal. After waking up,
+we dequeue everyone, and check if some futex was awakened. This dequeue is done
+by iteratively walking at each element of struct futex_head list.
+
+All enqueuing/dequeuing operations requires to hold the bucket lock, to avoid
+racing while modifying the list.
+
+Waking
+------
+
+We get the bucket that's storing the waiters at uaddr, and wake the required
+number of waiters, checking for hash collision.
+
+There's an optimization that makes futex_wake() not take the bucket lock if
+there's no one to be woken on that bucket. It checks an atomic counter that each
+bucket has, if it says 0, then the syscall exits. In order for this to work, the
+waiter thread increases it before taking the lock, so the wake thread will
+correctly see that there's someone waiting and will continue the path to take
+the bucket lock. To get the correct serialization, the waiter issues a memory
+barrier after increasing the bucket counter and the waker issues a memory
+barrier before checking it.
+
+Requeuing
+---------
+
+The requeue path first checks for each struct futex_requeue and their flags.
+Then, it will compare the expected value with the one at uaddr1::uaddr.
+Following the same serialization explained at Waking_, we increase the atomic
+counter for the bucket of uaddr2 before taking the lock. We need to have both
+buckets locks at same time so we don't race with other futex operation. To
+ensure the locks are taken in the same order for all threads (and thus avoiding
+deadlocks), every requeue operation takes the "smaller" bucket first, when
+comparing both addresses.
+
+If the compare with user value succeeds, we proceed by waking ``nr_wake``
+futexes, and then requeuing ``nr_requeue`` from bucket of uaddr1 to the uaddr2.
+This consists in a simple list deletion/addition and replacing the old futex key
+with the new one.
+
+Futex keys
+----------
+
+There are two types of futexes: private and shared ones. The private are futexes
+meant to be used by threads that share the same memory space, are easier to be
+uniquely identified and thus can have some performance optimization. The
+elements for identifying one are: the start address of the page where the
+address is, the address offset within the page and the current->mm pointer.
+
+Now, for uniquely identifying a shared futex:
+
+- If the page containing the user address is an anonymous page, we can
+  just use the same data used for private futexes (the start address of
+  the page, the address offset within the page and the current->mm
+  pointer); that will be enough for uniquely identifying such futex. We
+  also set one bit at the key to differentiate if a private futex is
+  used on the same address (mixing shared and private calls does not
+  work).
+
+- If the page is file-backed, current->mm maybe isn't the same one for
+  every user of this futex, so we need to use other data: the
+  page->index, a UUID for the struct inode and the offset within the
+  page.
+
+Note that members of futex_key don't have any particular meaning after they
+are part of the struct - they are just bytes to identify a futex.  Given that,
+we don't need to use a particular name or type that matches the original data,
+we only need to care about the bitsize of each component and make both private
+and shared fit in the same memory space.
+
+Source code documentation
+=========================
+
+.. kernel-doc:: kernel/futex2.c
+   :no-identifiers: sys_futex_wait sys_futex_wake sys_futex_waitv sys_futex_requeue
diff --git a/Documentation/locking/index.rst b/Documentation/locking/index.rst
index 7003bd5aeff4..9bf03c7fa1ec 100644
--- a/Documentation/locking/index.rst
+++ b/Documentation/locking/index.rst
@@ -24,6 +24,7 @@ locking
     percpu-rw-semaphore
     robust-futexes
     robust-futex-ABI
+    futex2
 
 .. only::  subproject and html
 
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 08/15] selftests: futex2: Add wake/wait test
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (6 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 07/15] docs: locking: futex2: Add documentation André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 09/15] selftests: futex2: Add timeout test André Almeida
                   ` (8 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Add a simple file to test wake/wait mechanism using futex2 interface.
Test three scenarios: using a common local int variable as private
futex, a shm futex as shared futex and a file-backed shared memory as a
shared futex. This should test all branches of futex_get_key().

Create helper files so more tests can evaluate futex2. While 32bit ABIs
from glibc aren't yet able to use 64 bit sized time variables, add a
temporary workaround that implements the required types and calls the
appropriated syscalls, since futex2 doesn't supports 32 bit sized time.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 .../selftests/futex/functional/.gitignore     |   1 +
 .../selftests/futex/functional/Makefile       |   4 +-
 .../selftests/futex/functional/futex2_wait.c  | 195 ++++++++++++++++++
 .../testing/selftests/futex/functional/run.sh |   3 +
 .../selftests/futex/include/futex2test.h      |  79 +++++++
 5 files changed, 281 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/futex/functional/futex2_wait.c
 create mode 100644 tools/testing/selftests/futex/include/futex2test.h

diff --git a/tools/testing/selftests/futex/functional/.gitignore b/tools/testing/selftests/futex/functional/.gitignore
index 0efcd494daab..d61f1df94360 100644
--- a/tools/testing/selftests/futex/functional/.gitignore
+++ b/tools/testing/selftests/futex/functional/.gitignore
@@ -6,3 +6,4 @@ futex_wait_private_mapped_file
 futex_wait_timeout
 futex_wait_uninitialized_heap
 futex_wait_wouldblock
+futex2_wait
diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile
index 23207829ec75..7142a94a7ac3 100644
--- a/tools/testing/selftests/futex/functional/Makefile
+++ b/tools/testing/selftests/futex/functional/Makefile
@@ -5,6 +5,7 @@ LDLIBS := -lpthread -lrt
 
 HEADERS := \
 	../include/futextest.h \
+	../include/futex2test.h \
 	../include/atomic.h \
 	../include/logging.h
 TEST_GEN_FILES := \
@@ -14,7 +15,8 @@ TEST_GEN_FILES := \
 	futex_requeue_pi_signal_restart \
 	futex_requeue_pi_mismatched_ops \
 	futex_wait_uninitialized_heap \
-	futex_wait_private_mapped_file
+	futex_wait_private_mapped_file \
+	futex2_wait
 
 TEST_PROGS := run.sh
 
diff --git a/tools/testing/selftests/futex/functional/futex2_wait.c b/tools/testing/selftests/futex/functional/futex2_wait.c
new file mode 100644
index 000000000000..25ac6d0898f5
--- /dev/null
+++ b/tools/testing/selftests/futex/functional/futex2_wait.c
@@ -0,0 +1,195 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/******************************************************************************
+ *
+ *   Copyright Collabora Ltd., 2021
+ *
+ * DESCRIPTION
+ *	Test wait/wake mechanism of futex2, using 32bit sized futexes.
+ *
+ * AUTHOR
+ *	André Almeida <andrealmeid@collabora.com>
+ *
+ * HISTORY
+ *      2021-Feb-5: Initial version by André <andrealmeid@collabora.com>
+ *
+ *****************************************************************************/
+
+#include <errno.h>
+#include <error.h>
+#include <getopt.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <pthread.h>
+#include <sys/shm.h>
+#include <sys/mman.h>
+#include <fcntl.h>
+#include <string.h>
+#include "futex2test.h"
+#include "logging.h"
+
+#define TEST_NAME "futex2-wait"
+#define timeout_ns  30000000
+#define WAKE_WAIT_US 10000
+#define SHM_PATH "futex2_shm_file"
+
+void *futex;
+
+void usage(char *prog)
+{
+	printf("Usage: %s\n", prog);
+	printf("  -c	Use color\n");
+	printf("  -h	Display this help message\n");
+	printf("  -v L	Verbosity level: %d=QUIET %d=CRITICAL %d=INFO\n",
+	       VQUIET, VCRITICAL, VINFO);
+}
+
+static void *waiterfn(void *arg)
+{
+	struct timespec64 to64;
+	unsigned int flags = 0;
+
+	if (arg)
+		flags = *((unsigned int *) arg);
+
+	/* setting absolute timeout for futex2 */
+	if (gettime64(CLOCK_MONOTONIC, &to64))
+		error("gettime64 failed\n", errno);
+
+	to64.tv_nsec += timeout_ns;
+
+	if (to64.tv_nsec >= 1000000000) {
+		to64.tv_sec++;
+		to64.tv_nsec -= 1000000000;
+	}
+
+	if (futex2_wait(futex, 0, FUTEX_32 | flags, &to64))
+		printf("waiter failed errno %d\n", errno);
+
+	return NULL;
+}
+
+int main(int argc, char *argv[])
+{
+	unsigned int flags = FUTEX_SHARED_FLAG;
+	int res, ret = RET_PASS, fd, c, shm_id;
+	u_int32_t f_private = 0, *shared_data;
+	pthread_t waiter;
+	void *shm;
+
+	futex = &f_private;
+
+	while ((c = getopt(argc, argv, "cht:v:")) != -1) {
+		switch (c) {
+		case 'c':
+			log_color(1);
+			break;
+		case 'h':
+			usage(basename(argv[0]));
+			exit(0);
+		case 'v':
+			log_verbosity(atoi(optarg));
+			break;
+		default:
+			usage(basename(argv[0]));
+			exit(1);
+		}
+	}
+
+	ksft_print_header();
+	ksft_set_plan(3);
+	ksft_print_msg("%s: Test FUTEX2_WAIT\n", basename(argv[0]));
+
+	/* Testing a private futex */
+	info("Calling private futex2_wait on futex: %p\n", futex);
+	if (pthread_create(&waiter, NULL, waiterfn, NULL))
+		error("pthread_create failed\n", errno);
+
+	usleep(WAKE_WAIT_US);
+
+	info("Calling private futex2_wake on futex: %p\n", futex);
+	res = futex2_wake(futex, 1, FUTEX_32);
+	if (res != 1) {
+		ksft_test_result_fail("futex2_wake private returned: %d %s\n",
+				      errno, strerror(errno));
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_wake private\n");
+	}
+
+	/* Testing an anon page shared memory */
+	shm_id = shmget(IPC_PRIVATE, 4096, IPC_CREAT | 0666);
+	if (shm_id < 0) {
+		perror("shmget");
+		exit(1);
+	}
+
+	shared_data = shmat(shm_id, NULL, 0);
+
+	*shared_data = 0;
+	futex = shared_data;
+
+	info("Calling (page anon) shared futex2_wait on futex: %p\n", futex);
+	if (pthread_create(&waiter, NULL, waiterfn, &flags))
+		error("pthread_create failed\n", errno);
+
+	usleep(WAKE_WAIT_US);
+
+	info("Calling (page anon) shared futex2_wake on futex: %p\n", futex);
+	res = futex2_wake(futex, 1, FUTEX_32 | FUTEX_SHARED_FLAG);
+	if (res != 1) {
+		ksft_test_result_fail("futex2_wake shared (page anon) returned: %d %s\n",
+				      errno, strerror(errno));
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_wake shared (page anon)\n");
+	}
+
+
+	/* Testing a file backed shared memory */
+	fd = open(SHM_PATH, O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);
+	if (fd < 0) {
+		perror("open");
+		exit(1);
+	}
+
+	if (ftruncate(fd, sizeof(f_private))) {
+		perror("ftruncate");
+		exit(1);
+	}
+
+	shm = mmap(NULL, sizeof(f_private), PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
+	if (shm == MAP_FAILED) {
+		perror("mmap");
+		exit(1);
+	}
+
+	memcpy(shm, &f_private, sizeof(f_private));
+
+	futex = shm;
+
+	info("Calling shared (file backed) futex2_wait on futex: %p\n", futex);
+	if (pthread_create(&waiter, NULL, waiterfn, &flags))
+		error("pthread_create failed\n", errno);
+
+	usleep(WAKE_WAIT_US);
+
+	info("Calling shared (file backed) futex2_wake on futex: %p\n", futex);
+	res = futex2_wake(shm, 1, FUTEX_32 | FUTEX_SHARED_FLAG);
+	if (res != 1) {
+		ksft_test_result_fail("futex2_wake shared (file backed) returned: %d %s\n",
+				      errno, strerror(errno));
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_wake shared (file backed)\n");
+	}
+
+	/* Freeing resources */
+	shmdt(shared_data);
+	munmap(shm, sizeof(f_private));
+	remove(SHM_PATH);
+
+	ksft_print_cnts();
+	return ret;
+}
diff --git a/tools/testing/selftests/futex/functional/run.sh b/tools/testing/selftests/futex/functional/run.sh
index 1acb6ace1680..3730159c865a 100755
--- a/tools/testing/selftests/futex/functional/run.sh
+++ b/tools/testing/selftests/futex/functional/run.sh
@@ -73,3 +73,6 @@ echo
 echo
 ./futex_wait_uninitialized_heap $COLOR
 ./futex_wait_private_mapped_file $COLOR
+
+echo
+./futex2_wait $COLOR
diff --git a/tools/testing/selftests/futex/include/futex2test.h b/tools/testing/selftests/futex/include/futex2test.h
new file mode 100644
index 000000000000..e724d56b917e
--- /dev/null
+++ b/tools/testing/selftests/futex/include/futex2test.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/******************************************************************************
+ *
+ *   Copyright Collabora Ltd., 2021
+ *
+ * DESCRIPTION
+ *	Futex2 library addons for old futex library
+ *
+ * AUTHOR
+ *	André Almeida <andrealmeid@collabora.com>
+ *
+ * HISTORY
+ *      2021-Feb-5: Initial version by André <andrealmeid@collabora.com>
+ *
+ *****************************************************************************/
+#include "futextest.h"
+#include <stdio.h>
+
+#define NSEC_PER_SEC	1000000000L
+
+#ifndef FUTEX_8
+# define FUTEX_8	0
+#endif
+#ifndef FUTEX_16
+# define FUTEX_16	1
+#endif
+#ifndef FUTEX_32
+# define FUTEX_32	2
+#endif
+
+/*
+ * - Y2038 section for 32-bit applications -
+ *
+ * Remove this when glibc is ready for y2038. Then, always compile with
+ * `-DTIME_BITS=64` or `-D__USE_TIME_BITS64`. glibc will provide both
+ * timespec64 and clock_gettime64 so we won't need to define here.
+ */
+#if defined(__i386__) || __TIMESIZE == 32
+# define NR_gettime __NR_clock_gettime64
+#else
+# define NR_gettime __NR_clock_gettime
+#endif
+
+struct timespec64 {
+	long long tv_sec;	/* seconds */
+	long long tv_nsec;	/* nanoseconds */
+};
+
+int gettime64(clock_t clockid, struct timespec64 *tv)
+{
+	return syscall(NR_gettime, clockid, tv);
+}
+/*
+ * - End of Y2038 section -
+ */
+
+/**
+ * futex2_wait - If (*uaddr == val), wait at uaddr until timo
+ * @uaddr: User address to wait on
+ * @val:   Expected value at uaddr, return if is not equal
+ * @flags: Operation flags
+ * @timo:  Optional timeout for operation
+ */
+static inline int futex2_wait(volatile void *uaddr, unsigned long val,
+			      unsigned long flags, struct timespec64 *timo)
+{
+	return syscall(__NR_futex_wait, uaddr, val, flags, timo);
+}
+
+/**
+ * futex2_wake - Wake a number of waiters at uaddr
+ * @uaddr: Address to wake
+ * @nr:    Number of waiters to wake
+ * @flags: Operation flags
+ */
+static inline int futex2_wake(volatile void *uaddr, unsigned int nr, unsigned long flags)
+{
+	return syscall(__NR_futex_wake, uaddr, nr, flags);
+}
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 09/15] selftests: futex2: Add timeout test
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (7 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 08/15] selftests: futex2: Add wake/wait test André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 10/15] selftests: futex2: Add wouldblock test André Almeida
                   ` (7 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Adapt existing futex wait timeout file to test the same mechanism for
futex2. futex2 accepts only absolute 64bit timers, but supports both
monotonic and realtime clocks.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 .../futex/functional/futex_wait_timeout.c     | 58 ++++++++++++++++---
 1 file changed, 49 insertions(+), 9 deletions(-)

diff --git a/tools/testing/selftests/futex/functional/futex_wait_timeout.c b/tools/testing/selftests/futex/functional/futex_wait_timeout.c
index ee55e6d389a3..b4dffe9e3b44 100644
--- a/tools/testing/selftests/futex/functional/futex_wait_timeout.c
+++ b/tools/testing/selftests/futex/functional/futex_wait_timeout.c
@@ -11,6 +11,7 @@
  *
  * HISTORY
  *      2009-Nov-6: Initial version by Darren Hart <dvhart@linux.intel.com>
+ *      2021-Feb-5: Add futex2 test by André <andrealmeid@collabora.com>
  *
  *****************************************************************************/
 
@@ -20,7 +21,7 @@
 #include <stdlib.h>
 #include <string.h>
 #include <time.h>
-#include "futextest.h"
+#include "futex2test.h"
 #include "logging.h"
 
 #define TEST_NAME "futex-wait-timeout"
@@ -40,7 +41,8 @@ void usage(char *prog)
 int main(int argc, char *argv[])
 {
 	futex_t f1 = FUTEX_INITIALIZER;
-	struct timespec to;
+	struct timespec to = {.tv_sec = 0, .tv_nsec = timeout_ns};
+	struct timespec64 to64;
 	int res, ret = RET_PASS;
 	int c;
 
@@ -65,22 +67,60 @@ int main(int argc, char *argv[])
 	}
 
 	ksft_print_header();
-	ksft_set_plan(1);
+	ksft_set_plan(3);
 	ksft_print_msg("%s: Block on a futex and wait for timeout\n",
 	       basename(argv[0]));
 	ksft_print_msg("\tArguments: timeout=%ldns\n", timeout_ns);
 
-	/* initialize timeout */
-	to.tv_sec = 0;
-	to.tv_nsec = timeout_ns;
-
 	info("Calling futex_wait on f1: %u @ %p\n", f1, &f1);
 	res = futex_wait(&f1, f1, &to, FUTEX_PRIVATE_FLAG);
 	if (!res || errno != ETIMEDOUT) {
-		fail("futex_wait returned %d\n", ret < 0 ? errno : ret);
+		ksft_test_result_fail("futex_wait returned %d\n", ret < 0 ? errno : ret);
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex_wait timeout succeeds\n");
+	}
+
+	/* setting absolute monotonic timeout for futex2 */
+	if (gettime64(CLOCK_MONOTONIC, &to64))
+		error("gettime64 failed\n", errno);
+
+	to64.tv_nsec += timeout_ns;
+
+	if (to64.tv_nsec >= 1000000000) {
+		to64.tv_sec++;
+		to64.tv_nsec -= 1000000000;
+	}
+
+	info("Calling futex2_wait on f1: %u @ %p\n", f1, &f1);
+	res = futex2_wait(&f1, f1, FUTEX_32, &to64);
+	if (!res || errno != ETIMEDOUT) {
+		ksft_test_result_fail("futex2_wait monotonic returned %d\n", ret < 0 ? errno : ret);
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_wait monotonic timeout succeeds\n");
+	}
+
+	/* setting absolute realtime timeout for futex2 */
+	if (gettime64(CLOCK_REALTIME, &to64))
+		error("gettime64 failed\n", errno);
+
+	to64.tv_nsec += timeout_ns;
+
+	if (to64.tv_nsec >= 1000000000) {
+		to64.tv_sec++;
+		to64.tv_nsec -= 1000000000;
+	}
+
+	info("Calling futex2_wait on f1: %u @ %p\n", f1, &f1);
+	res = futex2_wait(&f1, f1, FUTEX_32 | FUTEX_CLOCK_REALTIME, &to64);
+	if (!res || errno != ETIMEDOUT) {
+		ksft_test_result_fail("futex2_wait realtime returned %d\n", ret < 0 ? errno : ret);
 		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_wait realtime timeout succeeds\n");
 	}
 
-	print_result(TEST_NAME, ret);
+	ksft_print_cnts();
 	return ret;
 }
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 10/15] selftests: futex2: Add wouldblock test
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (8 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 09/15] selftests: futex2: Add timeout test André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 11/15] selftests: futex2: Add waitv test André Almeida
                   ` (6 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Adapt existing futex wait wouldblock file to test the same mechanism for
futex2.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 .../futex/functional/futex_wait_wouldblock.c  | 33 ++++++++++++++++---
 1 file changed, 29 insertions(+), 4 deletions(-)

diff --git a/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c b/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c
index 0ae390ff8164..510a98320248 100644
--- a/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c
+++ b/tools/testing/selftests/futex/functional/futex_wait_wouldblock.c
@@ -12,6 +12,7 @@
  *
  * HISTORY
  *      2009-Nov-14: Initial version by Gowrishankar <gowrishankar.m@in.ibm.com>
+ *      2021-Feb-5: Add futex2 test by André <andrealmeid@collabora.com>
  *
  *****************************************************************************/
 
@@ -21,7 +22,7 @@
 #include <stdlib.h>
 #include <string.h>
 #include <time.h>
-#include "futextest.h"
+#include "futex2test.h"
 #include "logging.h"
 
 #define TEST_NAME "futex-wait-wouldblock"
@@ -39,6 +40,7 @@ void usage(char *prog)
 int main(int argc, char *argv[])
 {
 	struct timespec to = {.tv_sec = 0, .tv_nsec = timeout_ns};
+	struct timespec64 to64;
 	futex_t f1 = FUTEX_INITIALIZER;
 	int res, ret = RET_PASS;
 	int c;
@@ -61,18 +63,41 @@ int main(int argc, char *argv[])
 	}
 
 	ksft_print_header();
-	ksft_set_plan(1);
+	ksft_set_plan(2);
 	ksft_print_msg("%s: Test the unexpected futex value in FUTEX_WAIT\n",
 	       basename(argv[0]));
 
 	info("Calling futex_wait on f1: %u @ %p with val=%u\n", f1, &f1, f1+1);
 	res = futex_wait(&f1, f1+1, &to, FUTEX_PRIVATE_FLAG);
 	if (!res || errno != EWOULDBLOCK) {
-		fail("futex_wait returned: %d %s\n",
+		ksft_test_result_fail("futex_wait returned: %d %s\n",
 		     res ? errno : res, res ? strerror(errno) : "");
 		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex_wait wouldblock\n");
 	}
 
-	print_result(TEST_NAME, ret);
+	/* setting absolute timeout for futex2 */
+	if (gettime64(CLOCK_MONOTONIC, &to64))
+		error("gettime64 failed\n", errno);
+
+	to64.tv_nsec += timeout_ns;
+
+	if (to64.tv_nsec >= 1000000000) {
+		to64.tv_sec++;
+		to64.tv_nsec -= 1000000000;
+	}
+
+	info("Calling futex2_wait on f1: %u @ %p with val=%u\n", f1, &f1, f1+1);
+	res = futex2_wait(&f1, f1+1, FUTEX_32, &to64);
+	if (!res || errno != EWOULDBLOCK) {
+		ksft_test_result_fail("futex2_wait returned: %d %s\n",
+		     res ? errno : res, res ? strerror(errno) : "");
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_wait wouldblock\n");
+	}
+
+	ksft_print_cnts();
 	return ret;
 }
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 11/15] selftests: futex2: Add waitv test
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (9 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 10/15] selftests: futex2: Add wouldblock test André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 12/15] selftests: futex2: Add requeue test André Almeida
                   ` (5 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Create a new file to test the waitv mechanism. Test both private and
shared futexes. Wake the last futex in the array, and check if the
return value from futex_waitv() is the right index.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 .../selftests/futex/functional/.gitignore     |   1 +
 .../selftests/futex/functional/Makefile       |   3 +-
 .../selftests/futex/functional/futex2_waitv.c | 154 ++++++++++++++++++
 .../testing/selftests/futex/functional/run.sh |   3 +
 .../selftests/futex/include/futex2test.h      |  17 ++
 5 files changed, 177 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/futex/functional/futex2_waitv.c

diff --git a/tools/testing/selftests/futex/functional/.gitignore b/tools/testing/selftests/futex/functional/.gitignore
index d61f1df94360..d0b8f637b786 100644
--- a/tools/testing/selftests/futex/functional/.gitignore
+++ b/tools/testing/selftests/futex/functional/.gitignore
@@ -7,3 +7,4 @@ futex_wait_timeout
 futex_wait_uninitialized_heap
 futex_wait_wouldblock
 futex2_wait
+futex2_waitv
diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile
index 7142a94a7ac3..b857b9450507 100644
--- a/tools/testing/selftests/futex/functional/Makefile
+++ b/tools/testing/selftests/futex/functional/Makefile
@@ -16,7 +16,8 @@ TEST_GEN_FILES := \
 	futex_requeue_pi_mismatched_ops \
 	futex_wait_uninitialized_heap \
 	futex_wait_private_mapped_file \
-	futex2_wait
+	futex2_wait \
+	futex2_waitv
 
 TEST_PROGS := run.sh
 
diff --git a/tools/testing/selftests/futex/functional/futex2_waitv.c b/tools/testing/selftests/futex/functional/futex2_waitv.c
new file mode 100644
index 000000000000..f15822bbcf8e
--- /dev/null
+++ b/tools/testing/selftests/futex/functional/futex2_waitv.c
@@ -0,0 +1,154 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/******************************************************************************
+ *
+ *   Copyright Collabora Ltd., 2021
+ *
+ * DESCRIPTION
+ *	Test waitv/wake mechanism of futex2, using 32bit sized futexes.
+ *
+ * AUTHOR
+ *	André Almeida <andrealmeid@collabora.com>
+ *
+ * HISTORY
+ *      2021-Feb-5: Initial version by André <andrealmeid@collabora.com>
+ *
+ *****************************************************************************/
+
+#include <errno.h>
+#include <error.h>
+#include <getopt.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <pthread.h>
+#include <sys/shm.h>
+#include "futex2test.h"
+#include "logging.h"
+
+#define TEST_NAME "futex2-wait"
+#define WAKE_WAIT_US 10000
+#define NR_FUTEXES 30
+struct futex_waitv waitv[NR_FUTEXES];
+u_int32_t futexes[NR_FUTEXES] = {0};
+
+void usage(char *prog)
+{
+	printf("Usage: %s\n", prog);
+	printf("  -c	Use color\n");
+	printf("  -h	Display this help message\n");
+	printf("  -v L	Verbosity level: %d=QUIET %d=CRITICAL %d=INFO\n",
+	       VQUIET, VCRITICAL, VINFO);
+}
+
+void *waiterfn(void *arg)
+{
+	struct timespec64 to64;
+	int res;
+
+	/* setting absolute timeout for futex2 */
+	if (gettime64(CLOCK_MONOTONIC, &to64))
+		error("gettime64 failed\n", errno);
+
+	to64.tv_sec++;
+
+	res = futex2_waitv(waitv, NR_FUTEXES, 0, &to64);
+	if (res < 0) {
+		ksft_test_result_fail("futex2_waitv returned: %d %s\n",
+				      errno, strerror(errno));
+	} else if (res != NR_FUTEXES - 1) {
+		ksft_test_result_fail("futex2_waitv returned: %d, expecting %d\n",
+				      res, NR_FUTEXES - 1);
+	}
+
+	return NULL;
+}
+
+int main(int argc, char *argv[])
+{
+	pthread_t waiter;
+	int res, ret = RET_PASS;
+	int c, i;
+
+	while ((c = getopt(argc, argv, "cht:v:")) != -1) {
+		switch (c) {
+		case 'c':
+			log_color(1);
+			break;
+		case 'h':
+			usage(basename(argv[0]));
+			exit(0);
+		case 'v':
+			log_verbosity(atoi(optarg));
+			break;
+		default:
+			usage(basename(argv[0]));
+			exit(1);
+		}
+	}
+
+	ksft_print_header();
+	ksft_set_plan(2);
+	ksft_print_msg("%s: Test FUTEX2_WAITV\n",
+		       basename(argv[0]));
+
+	for (i = 0; i < NR_FUTEXES; i++) {
+		waitv[i].uaddr = &futexes[i];
+		waitv[i].flags = FUTEX_32;
+		waitv[i].val = 0;
+	}
+
+	/* Private waitv */
+	if (pthread_create(&waiter, NULL, waiterfn, NULL))
+		error("pthread_create failed\n", errno);
+
+	usleep(WAKE_WAIT_US);
+
+	res = futex2_wake(waitv[NR_FUTEXES - 1].uaddr, 1, FUTEX_32);
+	if (res != 1) {
+		ksft_test_result_fail("futex2_wake private returned: %d %s\n",
+				      res ? errno : res,
+				      res ? strerror(errno) : "");
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_wake private\n");
+	}
+
+	/* Shared waitv */
+	for (i = 0; i < NR_FUTEXES; i++) {
+		int shm_id = shmget(IPC_PRIVATE, 4096, IPC_CREAT | 0666);
+
+		if (shm_id < 0) {
+			perror("shmget");
+			exit(1);
+		}
+
+		unsigned int *shared_data = shmat(shm_id, NULL, 0);
+
+		*shared_data = 0;
+		waitv[i].uaddr = shared_data;
+		waitv[i].flags = FUTEX_32 | FUTEX_SHARED_FLAG;
+		waitv[i].val = 0;
+	}
+
+	if (pthread_create(&waiter, NULL, waiterfn, NULL))
+		error("pthread_create failed\n", errno);
+
+	usleep(WAKE_WAIT_US);
+
+	res = futex2_wake(waitv[NR_FUTEXES - 1].uaddr, 1, FUTEX_32 | FUTEX_SHARED_FLAG);
+	if (res != 1) {
+		ksft_test_result_fail("futex2_waitv shared returned: %d %s\n",
+				      res ? errno : res,
+				      res ? strerror(errno) : "");
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_waitv shared\n");
+	}
+
+	for (i = 0; i < NR_FUTEXES; i++)
+		shmdt(waitv[i].uaddr);
+
+	ksft_print_cnts();
+	return ret;
+}
diff --git a/tools/testing/selftests/futex/functional/run.sh b/tools/testing/selftests/futex/functional/run.sh
index 3730159c865a..18b3883d7236 100755
--- a/tools/testing/selftests/futex/functional/run.sh
+++ b/tools/testing/selftests/futex/functional/run.sh
@@ -76,3 +76,6 @@ echo
 
 echo
 ./futex2_wait $COLOR
+
+echo
+./futex2_waitv $COLOR
diff --git a/tools/testing/selftests/futex/include/futex2test.h b/tools/testing/selftests/futex/include/futex2test.h
index e724d56b917e..0ed3b20935be 100644
--- a/tools/testing/selftests/futex/include/futex2test.h
+++ b/tools/testing/selftests/futex/include/futex2test.h
@@ -28,6 +28,10 @@
 # define FUTEX_32	2
 #endif
 
+#ifndef FUTEX_SHARED_FLAG
+#define FUTEX_SHARED_FLAG 8
+#endif
+
 /*
  * - Y2038 section for 32-bit applications -
  *
@@ -77,3 +81,16 @@ static inline int futex2_wake(volatile void *uaddr, unsigned int nr, unsigned lo
 {
 	return syscall(__NR_futex_wake, uaddr, nr, flags);
 }
+
+/**
+ * futex2_waitv - Wait at multiple futexes, wake on any
+ * @waiters:    Array of waiters
+ * @nr_waiters: Length of waiters array
+ * @flags: Operation flags
+ * @timo:  Optional timeout for operation
+ */
+static inline int futex2_waitv(volatile struct futex_waitv *waiters, unsigned long nr_waiters,
+			      unsigned long flags, struct timespec64 *timo)
+{
+	return syscall(__NR_futex_waitv, waiters, nr_waiters, flags, timo);
+}
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 12/15] selftests: futex2: Add requeue test
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (10 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 11/15] selftests: futex2: Add waitv test André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 13/15] selftests: futex2: Add futex sizes test André Almeida
                   ` (4 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Add testing for futex_requeue(). The first test just requeue from one
waiter to another one, and wake it. The second performs both wake and
requeue, and we check return values to see if the operation
woke/requeued the expected number of waiters.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 .../selftests/futex/functional/.gitignore     |   1 +
 .../selftests/futex/functional/Makefile       |   3 +-
 .../futex/functional/futex2_requeue.c         | 164 ++++++++++++++++++
 .../selftests/futex/include/futex2test.h      |  16 ++
 4 files changed, 183 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/futex/functional/futex2_requeue.c

diff --git a/tools/testing/selftests/futex/functional/.gitignore b/tools/testing/selftests/futex/functional/.gitignore
index d0b8f637b786..af7557e821da 100644
--- a/tools/testing/selftests/futex/functional/.gitignore
+++ b/tools/testing/selftests/futex/functional/.gitignore
@@ -8,3 +8,4 @@ futex_wait_uninitialized_heap
 futex_wait_wouldblock
 futex2_wait
 futex2_waitv
+futex2_requeue
diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile
index b857b9450507..ec0e713f0e42 100644
--- a/tools/testing/selftests/futex/functional/Makefile
+++ b/tools/testing/selftests/futex/functional/Makefile
@@ -17,7 +17,8 @@ TEST_GEN_FILES := \
 	futex_wait_uninitialized_heap \
 	futex_wait_private_mapped_file \
 	futex2_wait \
-	futex2_waitv
+	futex2_waitv \
+	futex2_requeue
 
 TEST_PROGS := run.sh
 
diff --git a/tools/testing/selftests/futex/functional/futex2_requeue.c b/tools/testing/selftests/futex/functional/futex2_requeue.c
new file mode 100644
index 000000000000..5b3d0775af99
--- /dev/null
+++ b/tools/testing/selftests/futex/functional/futex2_requeue.c
@@ -0,0 +1,164 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/******************************************************************************
+ *
+ *   Copyright Collabora Ltd., 2021
+ *
+ * DESCRIPTION
+ *	Test requeue mechanism of futex2, using 32bit sized futexes.
+ *
+ * AUTHOR
+ *	André Almeida <andrealmeid@collabora.com>
+ *
+ * HISTORY
+ *      2021-Feb-5: Initial version by André <andrealmeid@collabora.com>
+ *
+ *****************************************************************************/
+
+#include <errno.h>
+#include <error.h>
+#include <getopt.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <pthread.h>
+#include <sys/shm.h>
+#include <limits.h>
+#include "futex2test.h"
+#include "logging.h"
+
+#define TEST_NAME "futex2-wait"
+#define timeout_ns  30000000
+#define WAKE_WAIT_US 10000
+volatile futex_t *f1;
+
+void usage(char *prog)
+{
+	printf("Usage: %s\n", prog);
+	printf("  -c	Use color\n");
+	printf("  -h	Display this help message\n");
+	printf("  -v L	Verbosity level: %d=QUIET %d=CRITICAL %d=INFO\n",
+	       VQUIET, VCRITICAL, VINFO);
+}
+
+void *waiterfn(void *arg)
+{
+	struct timespec64 to64;
+
+	/* setting absolute timeout for futex2 */
+	if (gettime64(CLOCK_MONOTONIC, &to64))
+		error("gettime64 failed\n", errno);
+
+	to64.tv_nsec += timeout_ns;
+
+	if (to64.tv_nsec >= 1000000000) {
+		to64.tv_sec++;
+		to64.tv_nsec -= 1000000000;
+	}
+
+	if (futex2_wait(f1, *f1, FUTEX_32, &to64))
+		printf("waiter failed errno %d\n", errno);
+
+	return NULL;
+}
+
+int main(int argc, char *argv[])
+{
+	pthread_t waiter[10];
+	int res, ret = RET_PASS;
+	int c, i;
+	volatile futex_t _f1 = 0;
+	volatile futex_t f2 = 0;
+	struct futex_requeue r1, r2;
+
+	f1 = &_f1;
+
+	r1.flags = FUTEX_32;
+	r2.flags = FUTEX_32;
+
+	r1.uaddr = f1;
+	r2.uaddr = &f2;
+
+	while ((c = getopt(argc, argv, "cht:v:")) != -1) {
+		switch (c) {
+		case 'c':
+			log_color(1);
+			break;
+		case 'h':
+			usage(basename(argv[0]));
+			exit(0);
+		case 'v':
+			log_verbosity(atoi(optarg));
+			break;
+		default:
+			usage(basename(argv[0]));
+			exit(1);
+		}
+	}
+
+	ksft_print_header();
+	ksft_set_plan(2);
+	ksft_print_msg("%s: Test FUTEX2_REQUEUE\n",
+		       basename(argv[0]));
+
+	/*
+	 * Requeue a waiter from f1 to f2, and wake f2.
+	 */
+	if (pthread_create(&waiter[0], NULL, waiterfn, NULL))
+		error("pthread_create failed\n", errno);
+
+	usleep(WAKE_WAIT_US);
+
+	res = futex2_requeue(&r1, &r2, 0, 1, 0, 0);
+	if (res != 1) {
+		ksft_test_result_fail("futex2_requeue private returned: %d %s\n",
+				      res ? errno : res,
+				      res ? strerror(errno) : "");
+		ret = RET_FAIL;
+	}
+
+
+	info("Calling private futex2_wake on f2: %u @ %p with val=%u\n", f2, &f2, f2);
+	res = futex2_wake(&f2, 1, FUTEX_32);
+	if (res != 1) {
+		ksft_test_result_fail("futex2_requeue private returned: %d %s\n",
+				      res ? errno : res,
+				      res ? strerror(errno) : "");
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_requeue simple\n");
+	}
+
+
+	/*
+	 * Create 10 waiters at f1. At futex_requeue, wake 3 and requeue 7.
+	 * At futex_wake, wake INT_MAX (should be exaclty 7).
+	 */
+	for (i = 0; i < 10; i++) {
+		if (pthread_create(&waiter[i], NULL, waiterfn, NULL))
+			error("pthread_create failed\n", errno);
+	}
+
+	usleep(WAKE_WAIT_US);
+
+	res = futex2_requeue(&r1, &r2, 3, 7, 0, 0);
+	if (res != 10) {
+		ksft_test_result_fail("futex2_requeue private returned: %d %s\n",
+				      res ? errno : res,
+				      res ? strerror(errno) : "");
+		ret = RET_FAIL;
+	}
+
+	res = futex2_wake(&f2, INT_MAX, FUTEX_32);
+	if (res != 7) {
+		ksft_test_result_fail("futex2_requeue private returned: %d %s\n",
+				      res ? errno : res,
+				      res ? strerror(errno) : "");
+		ret = RET_FAIL;
+	} else {
+		ksft_test_result_pass("futex2_requeue\n");
+	}
+
+	ksft_print_cnts();
+	return ret;
+}
diff --git a/tools/testing/selftests/futex/include/futex2test.h b/tools/testing/selftests/futex/include/futex2test.h
index 0ed3b20935be..b9879f1e0523 100644
--- a/tools/testing/selftests/futex/include/futex2test.h
+++ b/tools/testing/selftests/futex/include/futex2test.h
@@ -94,3 +94,19 @@ static inline int futex2_waitv(volatile struct futex_waitv *waiters, unsigned lo
 {
 	return syscall(__NR_futex_waitv, waiters, nr_waiters, flags, timo);
 }
+
+/**
+ * futex2_requeue - Wake futexes at uaddr1 and requeue from uaddr1 to uaddr2
+ * @uaddr1:     Original address to wake and requeue from
+ * @uaddr2:     Address to requeue to
+ * @nr_wake:    Number of futexes to wake at uaddr1 before requeuing
+ * @nr_requeue: Number of futexes to requeue from uaddr1 to uaddr2
+ * @cmpval:     If (uaddr1->uaddr != cmpval), return immediatally
+ * @flgas:      Operation flags
+ */
+static inline int futex2_requeue(struct futex_requeue *uaddr1, struct futex_requeue *uaddr2,
+				 unsigned int nr_wake, unsigned int nr_requeue,
+				 unsigned int cmpval, unsigned long flags)
+{
+	return syscall(__NR_futex_requeue, uaddr1, uaddr2, nr_wake, nr_requeue, cmpval, flags);
+}
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 13/15] selftests: futex2: Add futex sizes test
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (11 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 12/15] selftests: futex2: Add requeue test André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 14/15] perf bench: Add futex2 benchmark tests André Almeida
                   ` (3 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Add a selftest for the variable size futex2 API. This initial test
just validates the basic (and correct) case, where both uses the same
size and the value is in the correct range.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 .../selftests/futex/functional/.gitignore     |   1 +
 .../selftests/futex/functional/Makefile       |   3 +-
 .../selftests/futex/functional/futex2_sizes.c | 146 ++++++++++++++++++
 .../selftests/futex/include/futex2test.h      |   3 +-
 4 files changed, 151 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/futex/functional/futex2_sizes.c

diff --git a/tools/testing/selftests/futex/functional/.gitignore b/tools/testing/selftests/futex/functional/.gitignore
index af7557e821da..9e5d9c5a5510 100644
--- a/tools/testing/selftests/futex/functional/.gitignore
+++ b/tools/testing/selftests/futex/functional/.gitignore
@@ -9,3 +9,4 @@ futex_wait_wouldblock
 futex2_wait
 futex2_waitv
 futex2_requeue
+futex2_sizes
diff --git a/tools/testing/selftests/futex/functional/Makefile b/tools/testing/selftests/futex/functional/Makefile
index ec0e713f0e42..9b4fb89eeb14 100644
--- a/tools/testing/selftests/futex/functional/Makefile
+++ b/tools/testing/selftests/futex/functional/Makefile
@@ -18,7 +18,8 @@ TEST_GEN_FILES := \
 	futex_wait_private_mapped_file \
 	futex2_wait \
 	futex2_waitv \
-	futex2_requeue
+	futex2_requeue \
+	futex2_sizes
 
 TEST_PROGS := run.sh
 
diff --git a/tools/testing/selftests/futex/functional/futex2_sizes.c b/tools/testing/selftests/futex/functional/futex2_sizes.c
new file mode 100644
index 000000000000..ee5fa48bff91
--- /dev/null
+++ b/tools/testing/selftests/futex/functional/futex2_sizes.c
@@ -0,0 +1,146 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/******************************************************************************
+ *
+ *   Copyright Collabora Ltd., 2021
+ *
+ * DESCRIPTION
+ *	Test wait/wake mechanism of futex2, using 32bit sized futexes.
+ *
+ * AUTHOR
+ *	André Almeida <andrealmeid@collabora.com>
+ *
+ * HISTORY
+ *      2021-Feb-5: Initial version by André <andrealmeid@collabora.com>
+ *
+ *****************************************************************************/
+
+#include <errno.h>
+#include <error.h>
+#include <getopt.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#include <pthread.h>
+#include <string.h>
+#include "futex2test.h"
+#include "logging.h"
+
+#define TEST_NAME "futex2-sizes"
+
+#define futex8  uint8_t
+#define futex16 uint16_t
+#define futex32 uint32_t
+#define futex64 uint64_t
+
+// edge case values, to test sizes
+#define VALUE16 257        // 2^8  + 1
+#define VALUE32 65537      // 2^16 + 1
+#define VALUE64 4294967297 // 2^32 + 1
+
+#define WAKE_WAIT_US 100000
+
+void *futex;
+
+void usage(char *prog)
+{
+	printf("Usage: %s\n", prog);
+	printf("  -c	Use color\n");
+	printf("  -h	Display this help message\n");
+	printf("  -v L	Verbosity level: %d=QUIET %d=CRITICAL %d=INFO\n",
+	       VQUIET, VCRITICAL, VINFO);
+}
+
+struct futex {
+	unsigned long flags;
+	unsigned long val;
+};
+
+void *waiterfn(void *arg)
+{
+	int ret;
+	unsigned int *flags = (unsigned int *) arg;
+
+	ret = futex2_wait(futex, 0, *flags, NULL);
+	if (ret == ERROR)
+		error("waiter failed %d errno %d\n", ret, errno);
+
+	return NULL;
+}
+
+/*
+ * create a thread to wait, then wake it
+ */
+void test_single_waiter(unsigned int flags, int *ret)
+{
+	pthread_t waiter;
+	int res;
+
+	pthread_create(&waiter, NULL, waiterfn, &flags);
+
+	usleep(WAKE_WAIT_US);
+
+	info("Calling futex2_wake at addr %p flags %u\n", futex, flags);
+	res = futex2_wake(futex, 1, flags);
+	if (res == 1) {
+		ksft_test_result_pass("futex2_sizes\n");
+	} else {
+		ksft_test_result_fail("futex2_sizes returned: %d %s\n",
+				      errno, strerror(errno));
+		*ret = RET_FAIL;
+	}
+}
+
+int main(int argc, char *argv[])
+{
+	int res, ret = RET_PASS, fd, c, shm_id;
+	u_int32_t f_private = 0;
+	pthread_t waiter;
+
+	futex8  f8 = 0;
+	futex16 f16 = 0;
+	futex32 f32 = 0;
+	futex64 f64 = 0;
+	unsigned int flags = 0;
+
+	while ((c = getopt(argc, argv, "cht:v:")) != -1) {
+		switch (c) {
+		case 'c':
+			log_color(1);
+			break;
+		case 'h':
+			usage(basename(argv[0]));
+			exit(0);
+		case 'v':
+			log_verbosity(atoi(optarg));
+			break;
+		default:
+			usage(basename(argv[0]));
+			exit(1);
+		}
+	}
+
+	ksft_print_header();
+	ksft_set_plan(4);
+	ksft_print_msg("%s: Test FUTEX2_SIZES\n", basename(argv[0]));
+
+	info("Calling futex2_wait futex: %p\n", futex);
+	futex = &f8;
+	flags = FUTEX_8;
+	test_single_waiter(flags, &ret);
+
+	futex = &f16;
+	flags = FUTEX_16;
+	test_single_waiter(flags, &ret);
+
+	futex = &f32;
+	flags = FUTEX_32;
+	test_single_waiter(flags, &ret);
+
+	futex = &f64;
+	flags = FUTEX_64;
+	test_single_waiter(flags, &ret);
+
+	ksft_print_cnts();
+	return ret;
+}
diff --git a/tools/testing/selftests/futex/include/futex2test.h b/tools/testing/selftests/futex/include/futex2test.h
index b9879f1e0523..af11fd191112 100644
--- a/tools/testing/selftests/futex/include/futex2test.h
+++ b/tools/testing/selftests/futex/include/futex2test.h
@@ -15,6 +15,7 @@
  *****************************************************************************/
 #include "futextest.h"
 #include <stdio.h>
+#include <stdint.h>
 
 #define NSEC_PER_SEC	1000000000L
 
@@ -65,7 +66,7 @@ int gettime64(clock_t clockid, struct timespec64 *tv)
  * @flags: Operation flags
  * @timo:  Optional timeout for operation
  */
-static inline int futex2_wait(volatile void *uaddr, unsigned long val,
+static inline int futex2_wait(volatile void *uaddr, uint64_t val,
 			      unsigned long flags, struct timespec64 *timo)
 {
 	return syscall(__NR_futex_wait, uaddr, val, flags, timo);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 14/15] perf bench: Add futex2 benchmark tests
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (12 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 13/15] selftests: futex2: Add futex sizes test André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-03 19:59 ` [PATCH v4 15/15] kernel: Enable waitpid() for futex2 André Almeida
                   ` (2 subsequent siblings)
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

Add support at the existing futex benchmarking code base to enable
futex2 calls. `perf bench` tests can be used not only as a way to
measure the performance of implementation, but also as stress testing
for the kernel infrastructure.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---
 tools/arch/x86/include/asm/unistd_64.h | 12 ++++++
 tools/perf/bench/bench.h               |  4 ++
 tools/perf/bench/futex-hash.c          | 24 +++++++++--
 tools/perf/bench/futex-requeue.c       | 57 ++++++++++++++++++++------
 tools/perf/bench/futex-wake-parallel.c | 41 +++++++++++++++---
 tools/perf/bench/futex-wake.c          | 37 +++++++++++++----
 tools/perf/bench/futex.h               | 47 +++++++++++++++++++++
 tools/perf/builtin-bench.c             | 18 ++++++--
 8 files changed, 206 insertions(+), 34 deletions(-)

diff --git a/tools/arch/x86/include/asm/unistd_64.h b/tools/arch/x86/include/asm/unistd_64.h
index 4205ed4158bf..191d43d84f04 100644
--- a/tools/arch/x86/include/asm/unistd_64.h
+++ b/tools/arch/x86/include/asm/unistd_64.h
@@ -17,3 +17,15 @@
 #ifndef __NR_setns
 #define __NR_setns 308
 #endif
+
+#ifndef __NR_futex_wait
+# define __NR_futex_wait 447
+#endif
+
+#ifndef __NR_futex_wake
+# define __NR_futex_wake 448
+#endif
+
+#ifndef __NR_futex_requeue
+# define __NR_futex_requeue 450
+#endif
diff --git a/tools/perf/bench/bench.h b/tools/perf/bench/bench.h
index eac36afab2b3..12346844b354 100644
--- a/tools/perf/bench/bench.h
+++ b/tools/perf/bench/bench.h
@@ -38,9 +38,13 @@ int bench_mem_memcpy(int argc, const char **argv);
 int bench_mem_memset(int argc, const char **argv);
 int bench_mem_find_bit(int argc, const char **argv);
 int bench_futex_hash(int argc, const char **argv);
+int bench_futex2_hash(int argc, const char **argv);
 int bench_futex_wake(int argc, const char **argv);
+int bench_futex2_wake(int argc, const char **argv);
 int bench_futex_wake_parallel(int argc, const char **argv);
+int bench_futex2_wake_parallel(int argc, const char **argv);
 int bench_futex_requeue(int argc, const char **argv);
+int bench_futex2_requeue(int argc, const char **argv);
 /* pi futexes */
 int bench_futex_lock_pi(int argc, const char **argv);
 int bench_epoll_wait(int argc, const char **argv);
diff --git a/tools/perf/bench/futex-hash.c b/tools/perf/bench/futex-hash.c
index b65373ce5c4f..1068749af40c 100644
--- a/tools/perf/bench/futex-hash.c
+++ b/tools/perf/bench/futex-hash.c
@@ -33,7 +33,7 @@ static unsigned int nthreads = 0;
 static unsigned int nsecs    = 10;
 /* amount of futexes per thread */
 static unsigned int nfutexes = 1024;
-static bool fshared = false, done = false, silent = false;
+static bool fshared = false, done = false, silent = false, futex2 = false;
 static int futex_flag = 0;
 
 struct timeval bench__start, bench__end, bench__runtime;
@@ -85,7 +85,10 @@ static void *workerfn(void *arg)
 			 * such as internal waitqueue handling, thus enlarging
 			 * the critical region protected by hb->lock.
 			 */
-			ret = futex_wait(&w->futex[i], 1234, NULL, futex_flag);
+			if (!futex2)
+				ret = futex_wait(&w->futex[i], 1234, NULL, futex_flag);
+			else
+				ret = futex2_wait(&w->futex[i], 1234, futex_flag, NULL);
 			if (!silent &&
 			    (!ret || errno != EAGAIN || errno != EWOULDBLOCK))
 				warn("Non-expected futex return call");
@@ -116,7 +119,7 @@ static void print_summary(void)
 	       (int)bench__runtime.tv_sec);
 }
 
-int bench_futex_hash(int argc, const char **argv)
+static int __bench_futex_hash(int argc, const char **argv)
 {
 	int ret = 0;
 	cpu_set_t cpuset;
@@ -148,7 +151,9 @@ int bench_futex_hash(int argc, const char **argv)
 	if (!worker)
 		goto errmem;
 
-	if (!fshared)
+	if (futex2)
+		futex_flag = FUTEX_32 | (fshared * FUTEX_SHARED_FLAG);
+	else if (!fshared)
 		futex_flag = FUTEX_PRIVATE_FLAG;
 
 	printf("Run summary [PID %d]: %d threads, each operating on %d [%s] futexes for %d secs.\n\n",
@@ -228,3 +233,14 @@ int bench_futex_hash(int argc, const char **argv)
 errmem:
 	err(EXIT_FAILURE, "calloc");
 }
+
+int bench_futex_hash(int argc, const char **argv)
+{
+	return __bench_futex_hash(argc, argv);
+}
+
+int bench_futex2_hash(int argc, const char **argv)
+{
+	futex2 = true;
+	return __bench_futex_hash(argc, argv);
+}
diff --git a/tools/perf/bench/futex-requeue.c b/tools/perf/bench/futex-requeue.c
index 5fa23295ee5f..6cdd649b54f4 100644
--- a/tools/perf/bench/futex-requeue.c
+++ b/tools/perf/bench/futex-requeue.c
@@ -2,8 +2,8 @@
 /*
  * Copyright (C) 2013  Davidlohr Bueso <davidlohr@hp.com>
  *
- * futex-requeue: Block a bunch of threads on futex1 and requeue them
- *                on futex2, N at a time.
+ * futex-requeue: Block a bunch of threads on addr1 and requeue them
+ *                on addr2, N at a time.
  *
  * This program is particularly useful to measure the latency of nthread
  * requeues without waking up any tasks -- thus mimicking a regular futex_wait.
@@ -28,7 +28,10 @@
 #include <stdlib.h>
 #include <sys/time.h>
 
-static u_int32_t futex1 = 0, futex2 = 0;
+static u_int32_t addr1 = 0, addr2 = 0;
+
+static struct futex_requeue rq1 = { .uaddr = &addr1, .flags = FUTEX_32 };
+static struct futex_requeue rq2 = { .uaddr = &addr2, .flags = FUTEX_32 };
 
 /*
  * How many tasks to requeue at a time.
@@ -37,7 +40,7 @@ static u_int32_t futex1 = 0, futex2 = 0;
 static unsigned int nrequeue = 1;
 
 static pthread_t *worker;
-static bool done = false, silent = false, fshared = false;
+static bool done = false, silent = false, fshared = false, futex2 = false;
 static pthread_mutex_t thread_lock;
 static pthread_cond_t thread_parent, thread_worker;
 static struct stats requeuetime_stats, requeued_stats;
@@ -79,7 +82,11 @@ static void *workerfn(void *arg __maybe_unused)
 	pthread_cond_wait(&thread_worker, &thread_lock);
 	pthread_mutex_unlock(&thread_lock);
 
-	futex_wait(&futex1, 0, NULL, futex_flag);
+	if (!futex2)
+		futex_wait(&addr1, 0, NULL, futex_flag);
+	else
+		futex2_wait(&addr1, 0, futex_flag, NULL);
+
 	return NULL;
 }
 
@@ -111,7 +118,7 @@ static void toggle_done(int sig __maybe_unused,
 	done = true;
 }
 
-int bench_futex_requeue(int argc, const char **argv)
+static int __bench_futex_requeue(int argc, const char **argv)
 {
 	int ret = 0;
 	unsigned int i, j;
@@ -139,15 +146,20 @@ int bench_futex_requeue(int argc, const char **argv)
 	if (!worker)
 		err(EXIT_FAILURE, "calloc");
 
-	if (!fshared)
+	if (futex2) {
+		futex_flag = FUTEX_32 | (fshared * FUTEX_SHARED_FLAG);
+		rq1.flags |= FUTEX_SHARED_FLAG * fshared;
+		rq2.flags |= FUTEX_SHARED_FLAG * fshared;
+	} else if (!fshared) {
 		futex_flag = FUTEX_PRIVATE_FLAG;
+	}
 
 	if (nrequeue > nthreads)
 		nrequeue = nthreads;
 
 	printf("Run summary [PID %d]: Requeuing %d threads (from [%s] %p to %p), "
 	       "%d at a time.\n\n",  getpid(), nthreads,
-	       fshared ? "shared":"private", &futex1, &futex2, nrequeue);
+	       fshared ? "shared":"private", &addr1, &addr2, nrequeue);
 
 	init_stats(&requeued_stats);
 	init_stats(&requeuetime_stats);
@@ -176,11 +188,15 @@ int bench_futex_requeue(int argc, const char **argv)
 		gettimeofday(&start, NULL);
 		while (nrequeued < nthreads) {
 			/*
-			 * Do not wakeup any tasks blocked on futex1, allowing
+			 * Do not wakeup any tasks blocked on addr1, allowing
 			 * us to really measure futex_wait functionality.
 			 */
-			nrequeued += futex_cmp_requeue(&futex1, 0, &futex2, 0,
-						       nrequeue, futex_flag);
+			if (!futex2)
+				nrequeued += futex_cmp_requeue(&addr1, 0, &addr2,
+							0, nrequeue, futex_flag);
+			else
+				nrequeued += futex2_requeue(&rq1, &rq2,
+							0, nrequeue, 0, 0);
 		}
 
 		gettimeofday(&end, NULL);
@@ -194,8 +210,12 @@ int bench_futex_requeue(int argc, const char **argv)
 			       j + 1, nrequeued, nthreads, runtime.tv_usec / (double)USEC_PER_MSEC);
 		}
 
-		/* everybody should be blocked on futex2, wake'em up */
-		nrequeued = futex_wake(&futex2, nrequeued, futex_flag);
+		/* everybody should be blocked on addr2, wake'em up */
+		if (!futex2)
+			nrequeued = futex_wake(&addr2, nrequeued, futex_flag);
+		else
+			nrequeued = futex2_wake(&addr2, nrequeued, futex_flag);
+
 		if (nthreads != nrequeued)
 			warnx("couldn't wakeup all tasks (%d/%d)", nrequeued, nthreads);
 
@@ -220,3 +240,14 @@ int bench_futex_requeue(int argc, const char **argv)
 	usage_with_options(bench_futex_requeue_usage, options);
 	exit(EXIT_FAILURE);
 }
+
+int bench_futex_requeue(int argc, const char **argv)
+{
+       return __bench_futex_requeue(argc, argv);
+}
+
+int bench_futex2_requeue(int argc, const char **argv)
+{
+       futex2 = true;
+       return __bench_futex_requeue(argc, argv);
+}
diff --git a/tools/perf/bench/futex-wake-parallel.c b/tools/perf/bench/futex-wake-parallel.c
index 6e6f5247e1fe..cac90fc0bfb3 100644
--- a/tools/perf/bench/futex-wake-parallel.c
+++ b/tools/perf/bench/futex-wake-parallel.c
@@ -17,6 +17,12 @@ int bench_futex_wake_parallel(int argc __maybe_unused, const char **argv __maybe
 	pr_err("%s: pthread_barrier_t unavailable, disabling this test...\n", __func__);
 	return 0;
 }
+
+int bench_futex2_wake_parallel(int argc __maybe_unused, const char **argv __maybe_unused)
+{
+	pr_err("%s: pthread_barrier_t unavailable, disabling this test...\n", __func__);
+	return 0;
+}
 #else /* HAVE_PTHREAD_BARRIER */
 /* For the CLR_() macros */
 #include <string.h>
@@ -47,7 +53,7 @@ static unsigned int nwakes = 1;
 static u_int32_t futex = 0;
 
 static pthread_t *blocked_worker;
-static bool done = false, silent = false, fshared = false;
+static bool done = false, silent = false, fshared = false, futex2 = false;
 static unsigned int nblocked_threads = 0, nwaking_threads = 0;
 static pthread_mutex_t thread_lock;
 static pthread_cond_t thread_parent, thread_worker;
@@ -78,7 +84,11 @@ static void *waking_workerfn(void *arg)
 
 	gettimeofday(&start, NULL);
 
-	waker->nwoken = futex_wake(&futex, nwakes, futex_flag);
+	if (!futex2)
+		waker->nwoken = futex_wake(&futex, nwakes, futex_flag);
+	else
+		waker->nwoken = futex2_wake(&futex, nwakes, futex_flag);
+
 	if (waker->nwoken != nwakes)
 		warnx("couldn't wakeup all tasks (%d/%d)",
 		      waker->nwoken, nwakes);
@@ -129,8 +139,13 @@ static void *blocked_workerfn(void *arg __maybe_unused)
 	pthread_mutex_unlock(&thread_lock);
 
 	while (1) { /* handle spurious wakeups */
-		if (futex_wait(&futex, 0, NULL, futex_flag) != EINTR)
-			break;
+		if (!futex2) {
+			if (futex_wait(&futex, 0, NULL, futex_flag) != EINTR)
+				break;
+		} else {
+			if (futex2_wait(&futex, 0, futex_flag, NULL) != EINTR)
+				break;
+		}
 	}
 
 	pthread_exit(NULL);
@@ -217,7 +232,7 @@ static void toggle_done(int sig __maybe_unused,
 	done = true;
 }
 
-int bench_futex_wake_parallel(int argc, const char **argv)
+static int __bench_futex_wake_parallel(int argc, const char **argv)
 {
 	int ret = 0;
 	unsigned int i, j;
@@ -261,7 +276,9 @@ int bench_futex_wake_parallel(int argc, const char **argv)
 	if (!blocked_worker)
 		err(EXIT_FAILURE, "calloc");
 
-	if (!fshared)
+	if (futex2)
+		futex_flag = FUTEX_32 | (fshared * FUTEX_SHARED_FLAG);
+	else if (!fshared)
 		futex_flag = FUTEX_PRIVATE_FLAG;
 
 	printf("Run summary [PID %d]: blocking on %d threads (at [%s] "
@@ -321,4 +338,16 @@ int bench_futex_wake_parallel(int argc, const char **argv)
 	free(blocked_worker);
 	return ret;
 }
+
+int bench_futex_wake_parallel(int argc, const char **argv)
+{
+	return __bench_futex_wake_parallel(argc, argv);
+}
+
+int bench_futex2_wake_parallel(int argc, const char **argv)
+{
+	futex2 = true;
+	return __bench_futex_wake_parallel(argc, argv);
+}
+
 #endif /* HAVE_PTHREAD_BARRIER */
diff --git a/tools/perf/bench/futex-wake.c b/tools/perf/bench/futex-wake.c
index 6d217868f53c..546d2818eed8 100644
--- a/tools/perf/bench/futex-wake.c
+++ b/tools/perf/bench/futex-wake.c
@@ -38,7 +38,7 @@ static u_int32_t futex1 = 0;
 static unsigned int nwakes = 1;
 
 pthread_t *worker;
-static bool done = false, silent = false, fshared = false;
+static bool done = false, silent = false, fshared = false, futex2 = false;
 static pthread_mutex_t thread_lock;
 static pthread_cond_t thread_parent, thread_worker;
 static struct stats waketime_stats, wakeup_stats;
@@ -68,8 +68,13 @@ static void *workerfn(void *arg __maybe_unused)
 	pthread_mutex_unlock(&thread_lock);
 
 	while (1) {
-		if (futex_wait(&futex1, 0, NULL, futex_flag) != EINTR)
-			break;
+		if (!futex2) {
+			if (futex_wait(&futex1, 0, NULL, futex_flag) != EINTR)
+				break;
+		} else {
+			if (futex2_wait(&futex1, 0, futex_flag, NULL) != EINTR)
+				break;
+		}
 	}
 
 	pthread_exit(NULL);
@@ -117,7 +122,7 @@ static void toggle_done(int sig __maybe_unused,
 	done = true;
 }
 
-int bench_futex_wake(int argc, const char **argv)
+static int __bench_futex_wake(int argc, const char **argv)
 {
 	int ret = 0;
 	unsigned int i, j;
@@ -147,7 +152,9 @@ int bench_futex_wake(int argc, const char **argv)
 	if (!worker)
 		err(EXIT_FAILURE, "calloc");
 
-	if (!fshared)
+	if (futex2)
+		futex_flag = FUTEX_32 | (fshared * FUTEX_SHARED_FLAG);
+	else if (!fshared)
 		futex_flag = FUTEX_PRIVATE_FLAG;
 
 	printf("Run summary [PID %d]: blocking on %d threads (at [%s] futex %p), "
@@ -179,9 +186,14 @@ int bench_futex_wake(int argc, const char **argv)
 
 		/* Ok, all threads are patiently blocked, start waking folks up */
 		gettimeofday(&start, NULL);
-		while (nwoken != nthreads)
-			nwoken += futex_wake(&futex1, nwakes, futex_flag);
+		while (nwoken != nthreads) {
+			if (!futex2)
+				nwoken += futex_wake(&futex1, nwakes, futex_flag);
+			else
+				nwoken += futex2_wake(&futex1, nwakes, futex_flag);
+		}
 		gettimeofday(&end, NULL);
+
 		timersub(&end, &start, &runtime);
 
 		update_stats(&wakeup_stats, nwoken);
@@ -211,3 +223,14 @@ int bench_futex_wake(int argc, const char **argv)
 	free(worker);
 	return ret;
 }
+
+int bench_futex_wake(int argc, const char **argv)
+{
+	return __bench_futex_wake(argc, argv);
+}
+
+int bench_futex2_wake(int argc, const char **argv)
+{
+	futex2 = true;
+	return __bench_futex_wake(argc, argv);
+}
diff --git a/tools/perf/bench/futex.h b/tools/perf/bench/futex.h
index 31b53cc7d5bc..6b2213cf3f64 100644
--- a/tools/perf/bench/futex.h
+++ b/tools/perf/bench/futex.h
@@ -86,4 +86,51 @@ futex_cmp_requeue(u_int32_t *uaddr, u_int32_t val, u_int32_t *uaddr2, int nr_wak
 	return futex(uaddr, FUTEX_CMP_REQUEUE, nr_wake, nr_requeue, uaddr2,
 		 val, opflags);
 }
+
+/**
+ * futex2_wait - Wait at uaddr if *uaddr == val, until timo.
+ * @uaddr: User address to wait for
+ * @val:   Expected value at uaddr
+ * @flags: Operation options
+ * @timo:  Optional timeout
+ *
+ * Return: 0 on success, error code otherwise
+ */
+static inline int futex2_wait(volatile void *uaddr, unsigned long val,
+			      unsigned long flags, struct timespec *timo)
+{
+	return syscall(__NR_futex_wait, uaddr, val, flags, timo);
+}
+
+/**
+ * futex2_wake - Wake a number of waiters waiting at uaddr
+ * @uaddr: Address to wake
+ * @nr:    Number of waiters to wake
+ * @flags: Operation options
+ *
+ * Return: number of waked futexes
+ */
+static inline int futex2_wake(volatile void *uaddr, unsigned int nr, unsigned long flags)
+{
+	return syscall(__NR_futex_wake, uaddr, nr, flags);
+}
+
+/**
+ * futex2_requeue - Requeue waiters from an address to another one
+ * @uaddr1:     Address where waiters are currently waiting on
+ * @uaddr2:     New address to wait
+ * @nr_wake:    Number of waiters at uaddr1 to be wake
+ * @nr_requeue: After waking nr_wake, number of waiters to be requeued
+ * @cmpval:     Expected value at uaddr1
+ * @flags: Operation options
+ *
+ * Return: waked futexes + requeued futexes at uaddr1
+ */
+static inline int futex2_requeue(volatile struct futex_requeue *uaddr1,
+				 volatile struct futex_requeue *uaddr2,
+				 unsigned int nr_wake, unsigned int nr_requeue,
+				 unsigned int cmpval, unsigned long flags)
+{
+	return syscall(__NR_futex_requeue, uaddr1, uaddr2, nr_wake, nr_requeue, cmpval, flags);
+}
 #endif /* _FUTEX_H */
diff --git a/tools/perf/builtin-bench.c b/tools/perf/builtin-bench.c
index 62a7b7420a44..e41a95ad2db6 100644
--- a/tools/perf/builtin-bench.c
+++ b/tools/perf/builtin-bench.c
@@ -12,10 +12,11 @@
  *
  *  sched ... scheduler and IPC performance
  *  syscall ... System call performance
- *  mem   ... memory access performance
- *  numa  ... NUMA scheduling and MM performance
- *  futex ... Futex performance
- *  epoll ... Event poll performance
+ *  mem    ... memory access performance
+ *  numa   ... NUMA scheduling and MM performance
+ *  futex  ... Futex performance
+ *  futex2 ... Futex2 performance
+ *  epoll  ... Event poll performance
  */
 #include <subcmd/parse-options.h>
 #include "builtin.h"
@@ -75,6 +76,14 @@ static struct bench futex_benchmarks[] = {
 	{ NULL,		NULL,						NULL			}
 };
 
+static struct bench futex2_benchmarks[] = {
+	{ "hash",	   "Benchmark for futex2 hash table",            bench_futex2_hash	},
+	{ "wake",	   "Benchmark for futex2 wake calls",            bench_futex2_wake	},
+	{ "wake-parallel", "Benchmark for parallel futex2 wake calls",   bench_futex2_wake_parallel },
+	{ "requeue",	   "Benchmark for futex2 requeue calls",         bench_futex2_requeue	},
+	{ NULL,		NULL,						NULL			}
+};
+
 #ifdef HAVE_EVENTFD_SUPPORT
 static struct bench epoll_benchmarks[] = {
 	{ "wait",	"Benchmark epoll concurrent epoll_waits",       bench_epoll_wait	},
@@ -105,6 +114,7 @@ static struct collection collections[] = {
 	{ "numa",	"NUMA scheduling and MM benchmarks",		numa_benchmarks		},
 #endif
 	{"futex",       "Futex stressing benchmarks",                   futex_benchmarks        },
+	{"futex2",      "Futex2 stressing benchmarks",                  futex2_benchmarks        },
 #ifdef HAVE_EVENTFD_SUPPORT
 	{"epoll",       "Epoll stressing benchmarks",                   epoll_benchmarks        },
 #endif
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH v4 15/15] kernel: Enable waitpid() for futex2
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (13 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 14/15] perf bench: Add futex2 benchmark tests André Almeida
@ 2021-06-03 19:59 ` André Almeida
  2021-06-04  4:51 ` [PATCH v4 00/15] Add futex2 syscalls Zebediah Figura
  2021-06-04 11:36 ` Nicholas Piggin
  16 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-03 19:59 UTC (permalink / raw)
  To: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, z.figura12, joel, malteskarupke,
	linux-api, fweimer, libc-alpha, linux-kselftest, shuah, acme,
	corbet, Peter Oskolkov, Andrey Semashev, Davidlohr Bueso,
	André Almeida

To make pthreads works as expected if they are using futex2, wake
clear_child_tid with futex2 as well. This is make applications that uses
waitpid() (and clone(CLONE_CHILD_SETTID)) wake while waiting for the
child to terminate. Given that apps should not mix futex() and futex2(),
any correct app will trigger a harmless noop wakeup on the interface
that it isn't using.

Signed-off-by: André Almeida <andrealmeid@collabora.com>
---

This commit is here for the intend to show what we need to do in order
to get a full NPTL working on top of futex2. It should be merged after
we talk to glibc folks on the details around the futex_wait() side. For
instance, we could use this as an opportunity to use private futexes or
8bit sized futexes, but both sides need to use the exactly same flags.
---
 include/linux/syscalls.h |  2 ++
 kernel/fork.c            |  2 ++
 kernel/futex2.c          | 30 ++++++++++++++++++------------
 3 files changed, 22 insertions(+), 12 deletions(-)

diff --git a/include/linux/syscalls.h b/include/linux/syscalls.h
index c108df6b3b82..94e0356ceeaa 100644
--- a/include/linux/syscalls.h
+++ b/include/linux/syscalls.h
@@ -1324,6 +1324,8 @@ int ksys_ipc(unsigned int call, int first, unsigned long second,
 	unsigned long third, void __user * ptr, long fifth);
 int compat_ksys_ipc(u32 call, int first, int second,
 	u32 third, u32 ptr, u32 fifth);
+long ksys_futex_wake(void __user *uaddr, unsigned long nr_wake,
+		     unsigned int flags);
 
 /*
  * The following kernel syscall equivalents are just wrappers to fs-internal
diff --git a/kernel/fork.c b/kernel/fork.c
index dc06afd725cb..344430d882b1 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -1322,6 +1322,8 @@ static void mm_release(struct task_struct *tsk, struct mm_struct *mm)
 			put_user(0, tsk->clear_child_tid);
 			do_futex(tsk->clear_child_tid, FUTEX_WAKE,
 					1, NULL, NULL, 0, 0);
+			ksys_futex_wake(tsk->clear_child_tid, 1,
+					FUTEX_32 | FUTEX_SHARED_FLAG);
 		}
 		tsk->clear_child_tid = NULL;
 	}
diff --git a/kernel/futex2.c b/kernel/futex2.c
index 5fd0b3d73b53..bf2de369b78a 100644
--- a/kernel/futex2.c
+++ b/kernel/futex2.c
@@ -978,18 +978,8 @@ static inline bool futex_match(struct futex_key key1, struct futex_key key2)
 		key1.offset == key2.offset);
 }
 
-/**
- * sys_futex_wake - Wake a number of futexes waiting on an address
- * @uaddr:   Address of futex to be woken up
- * @nr_wake: Number of futexes waiting in uaddr to be woken up
- * @flags:   Flags for size and shared
- *
- * Wake `nr_wake` threads waiting at uaddr.
- *
- * Returns the number of woken threads on success, error code otherwise.
- */
-SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
-		unsigned int, flags)
+long ksys_futex_wake(void __user *uaddr, unsigned long nr_wake,
+		     unsigned int flags)
 {
 	bool shared = (flags & FUTEX_SHARED_FLAG) ? true : false;
 	unsigned int size = flags & FUTEX_SIZE_MASK;
@@ -1023,6 +1013,22 @@ SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
 	return ret;
 }
 
+/**
+ * sys_futex_wake - Wake a number of futexes waiting on an address
+ * @uaddr:   Address of futex to be woken up
+ * @nr_wake: Number of futexes waiting in uaddr to be woken up
+ * @flags:   Flags for size and shared
+ *
+ * Wake `nr_wake` threads waiting at uaddr.
+ *
+ * Returns the number of woken threads on success, error code otherwise.
+ */
+SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
+		unsigned int, flags)
+{
+	return ksys_futex_wake(uaddr, nr_wake, flags);
+}
+
 static void futex_double_unlock(struct futex_bucket *b1, struct futex_bucket *b2)
 {
 	spin_unlock(&b1->lock);
-- 
2.31.1


^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 05/15] futex2: Implement support for different futex sizes
  2021-06-03 19:59 ` [PATCH v4 05/15] futex2: Implement support for different futex sizes André Almeida
@ 2021-06-04  0:23   ` kernel test robot
  2021-06-06 19:12   ` Davidlohr Bueso
  1 sibling, 0 replies; 53+ messages in thread
From: kernel test robot @ 2021-06-04  0:23 UTC (permalink / raw)
  To: André Almeida, Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
	Darren Hart, linux-kernel, Steven Rostedt,
	Sebastian Andrzej Siewior
  Cc: kbuild-all, kernel, krisman, pgriffais

[-- Attachment #1: Type: text/plain, Size: 1710 bytes --]

Hi "André,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on arm64/for-next/core]
[also build test ERROR on linus/master v5.13-rc4]
[cannot apply to tip/locking/core tip/x86/asm next-20210603]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Andr-Almeida/Add-futex2-syscalls/20210604-040253
base:   https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-next/core
config: arm-zeus_defconfig (attached as .config)
compiler: arm-linux-gnueabi-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/0d97dd96e50eedfb960644c316285cf5fa121d1e
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Andr-Almeida/Add-futex2-syscalls/20210604-040253
        git checkout 0d97dd96e50eedfb960644c316285cf5fa121d1e
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   arm-linux-gnueabi-ld: kernel/futex2.o: in function `futex_get_user':
>> futex2.c:(.text+0xec): undefined reference to `__get_user_bad'

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 22926 bytes --]

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (14 preceding siblings ...)
  2021-06-03 19:59 ` [PATCH v4 15/15] kernel: Enable waitpid() for futex2 André Almeida
@ 2021-06-04  4:51 ` Zebediah Figura
  2021-06-04 17:04   ` André Almeida
  2021-06-04 11:36 ` Nicholas Piggin
  16 siblings, 1 reply; 53+ messages in thread
From: Zebediah Figura @ 2021-06-04  4:51 UTC (permalink / raw)
  To: André Almeida, Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
	Darren Hart, linux-kernel, Steven Rostedt,
	Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, joel, malteskarupke, linux-api,
	fweimer, libc-alpha, linux-kselftest, shuah, acme, corbet,
	Peter Oskolkov, Andrey Semashev, Davidlohr Bueso

On 6/3/21 2:59 PM, André Almeida wrote:
>   ** The wait on multiple problem
> 
>   The use case lies in the Wine implementation of the Windows NT interface
>   WaitMultipleObjects. This Windows API function allows a thread to sleep
>   waiting on the first of a set of event sources (mutexes, timers, signal,
>   console input, etc) to signal.  Considering this is a primitive
>   synchronization operation for Windows applications, being able to quickly
>   signal events on the producer side, and quickly go to sleep on the
>   consumer side is essential for good performance of those running over Wine.
> 

I know this is part of the cover letter, but I really do want to clarify 
that this isn't really accurate. The use case that this is referring to 
is not "the Wine implementation of WaitForMultipleObjects", it is an 
out-of-tree implementation of WaitForMultipleObjects that provides 
improved performance compared to the in-tree implementation.

This is especially salient because:

(1) this out-of-tree implementation is only in a small handful of cases 
any more performant than a different out-of-tree implementation which 
uses eventfd and poll() instead;

(2) these implementations will remain out-of-tree due to compatibility 
and robustness problems;

(3) I believe there is potential for an upstreamable implementation 
which does not rely on futex or futex2.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
                   ` (15 preceding siblings ...)
  2021-06-04  4:51 ` [PATCH v4 00/15] Add futex2 syscalls Zebediah Figura
@ 2021-06-04 11:36 ` Nicholas Piggin
  2021-06-04 20:01   ` André Almeida
  16 siblings, 1 reply; 53+ messages in thread
From: Nicholas Piggin @ 2021-06-04 11:36 UTC (permalink / raw)
  To: André Almeida, Sebastian Andrzej Siewior, Darren Hart,
	linux-kernel, Ingo Molnar, Peter Zijlstra, Steven Rostedt,
	Thomas Gleixner
  Cc: acme, Andrey Semashev, corbet, Davidlohr Bueso, fweimer, joel,
	kernel, krisman, libc-alpha, linux-api, linux-kselftest,
	malteskarupke, pgriffais, Peter Oskolkov, shuah, z.figura12

Excerpts from André Almeida's message of June 4, 2021 5:59 am:
> Hi,
> 
> This patch series introduces the futex2 syscalls.
> 
> * What happened to the current futex()?
> 
> For some years now, developers have been trying to add new features to
> futex, but maintainers have been reluctant to accept then, given the
> multiplexed interface full of legacy features and tricky to do big
> changes. Some problems that people tried to address with patchsets are:
> NUMA-awareness[0], smaller sized futexes[1], wait on multiple futexes[2].
> NUMA, for instance, just doesn't fit the current API in a reasonable
> way. Considering that, it's not possible to merge new features into the
> current futex.
> 
>  ** The NUMA problem
> 
>  At the current implementation, all futex kernel side infrastructure is
>  stored on a single node. Given that, all futex() calls issued by
>  processors that aren't located on that node will have a memory access
>  penalty when doing it.
> 
>  ** The 32bit sized futex problem
> 
>  Futexes are used to implement atomic operations in userspace.
>  Supporting 8, 16, 32 and 64 bit sized futexes allows user libraries to
>  implement all those sizes in a performant way. Thanks Boost devs for
>  feedback: https://lists.boost.org/Archives/boost/2021/05/251508.php
> 
>  Embedded systems or anything with memory constrains could benefit of
>  using smaller sizes for the futex userspace integer.
> 
>  ** The wait on multiple problem
> 
>  The use case lies in the Wine implementation of the Windows NT interface
>  WaitMultipleObjects. This Windows API function allows a thread to sleep
>  waiting on the first of a set of event sources (mutexes, timers, signal,
>  console input, etc) to signal.  Considering this is a primitive
>  synchronization operation for Windows applications, being able to quickly
>  signal events on the producer side, and quickly go to sleep on the
>  consumer side is essential for good performance of those running over Wine.
> 
> [0] https://lore.kernel.org/lkml/20160505204230.932454245@linutronix.de/
> [1] https://lore.kernel.org/lkml/20191221155659.3159-2-malteskarupke@web.de/
> [2] https://lore.kernel.org/lkml/20200213214525.183689-1-andrealmeid@collabora.com/
> 
> * The solution
> 
> As proposed by Peter Zijlstra and Florian Weimer[3], a new interface
> is required to solve this, which must be designed with those features in
> mind. futex2() is that interface. As opposed to the current multiplexed
> interface, the new one should have one syscall per operation. This will
> allow the maintainability of the API if it gets extended, and will help
> users with type checking of arguments.
> 
> In particular, the new interface is extended to support the ability to
> wait on any of a list of futexes at a time, which could be seen as a
> vectored extension of the FUTEX_WAIT semantics.
> 
> [3] https://lore.kernel.org/lkml/20200303120050.GC2596@hirez.programming.kicks-ass.net/
> 
> * The interface
> 
> The new interface can be seen in details in the following patches, but
> this is a high level summary of what the interface can do:
> 
>  - Supports wake/wait semantics, as in futex()
>  - Supports requeue operations, similarly as FUTEX_CMP_REQUEUE, but with
>    individual flags for each address
>  - Supports waiting for a vector of futexes, using a new syscall named
>    futex_waitv()
>  - Supports variable sized futexes (8bits, 16bits, 32bits and 64bits)
>  - Supports NUMA-awareness operations, where the user can specify on
>    which memory node would like to operate

A few comments.

- Do you need a syscall for wait and for waitv, or can wait just be a
degenerate case of waitv (which could use the stack variable)?  I guess
it does save the user access unlock.

- Did you consider a wakev interface? An example is a read-write mutex 
which has read-blocking futexes split (e.g., per-node) for scalability 
then the writer may unlock and wake all readers with one call. We 
actually have some scalability challenges of this nature with certain 
large database programs.

- Great to see 64-bit support in, it is helpful to do some interesting 
things with locks without hacks (e.g., putting an address in the lock 
word).

- Are we really keen on squashing node ID into flags in this day and age?
I guess okay but seems like it would be nice to allow a bit more space
in general for the operations. I don't want to turn it into a whole big
multiplexing nightmare again with lots of such flags, or propose
complexity with no code behind it, but I think we need a bit of leeway
for unforeseen locking innovations to be added carefully. The pthread
locking today is still fairly primitive really, I don't think we know
what will work best for the next 10 years.

One scalability issue we are starting to hit and will only get worse is 
futex queue spinlock contention. Perhaps this is better addressed in 
userspace but the kernel could play a part so I like to leave some doors
open. One example is that the wait (or wake) side may like to depend not
just on the memory value, but on the success of a cmpxchg to avoid 
unqueueing and queueing spuriously, which increases lock contention but
also ends up putting the poor task on the back of the list -- yes RT
priorities can help the realtime case, but non-RT cases can get bad
outlier latencies if lock stealing is allowed (which can be very good
for performance).

- The private global futex hash table sucks for various reasons, and
having 128 waiters per thread makes it orders of magnitude easier for
userspace to DoS stuff with hash collisions. NUMA doesn't fix that, the
per process hashing that Thomas suggested does fix the DoS but the
non-deterministic hash collisions still seem to be a problem for real
time response, and at the other end of the scale some apps (certain 
databases, etc) can have ten thousand futex waiters at once so birthday
paradox can also lead to guaranteed (low level) variable beahviour 
within a single process.

I know the kernel in general is not very robust against this kind of 
DoS/nondeterminism, but it's a bit sad to introduce new APIs with the 
problem still there. Yes we could address it later, but I think it's 
better done first because the solution might influence what the best 
syscall API is.

For example the idea of using the address as the handle for the wait 
queue _and_ the value is very convenient but hashing is annoying for
all the above reasons and the shared wait queue case is pretty clunky. 
It's also constraining in some corner cases to have the wait queue 
associated with the address 1:1. For example a type of NUMA mutex might 
want to have per-node waitqueues associated with a lock word, and wake
local node waiters 99% of the time. Not trivial to do with futexes and
seems to at least require bouncing of more cache lines, possibly more
atomics, etc.

Could anything else be done?

I'll be burned at the stake for suggesting it but it would be great if 
we could use file descriptors. At least for the shared futex, maybe 
private could use a per-process futex allocator. It solves all of the
above, although I'm sure has many of its own problem. It may not play
so nicely with the pthread mutex API because of the whole static
initialiser problem, but the first futex proposal did use fds. But it's
an example of an alternate API.

And.. thanks for the great work, apologies if I missed some discussion
related to the above comments, I did some reading but I know there has
been a lot of discussions going on.

Thanks,
Nick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-04  4:51 ` [PATCH v4 00/15] Add futex2 syscalls Zebediah Figura
@ 2021-06-04 17:04   ` André Almeida
  0 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-04 17:04 UTC (permalink / raw)
  To: Zebediah Figura, Thomas Gleixner, Ingo Molnar, Peter Zijlstra,
	Darren Hart, linux-kernel, Steven Rostedt,
	Sebastian Andrzej Siewior
  Cc: kernel, krisman, pgriffais, joel, malteskarupke, linux-api,
	fweimer, libc-alpha, linux-kselftest, shuah, acme, corbet,
	Peter Oskolkov, Andrey Semashev, Davidlohr Bueso

Às 01:51 de 04/06/21, Zebediah Figura escreveu:
> On 6/3/21 2:59 PM, André Almeida wrote:
>>   ** The wait on multiple problem
>>
>>   The use case lies in the Wine implementation of the Windows NT
>> interface
>>   WaitMultipleObjects. This Windows API function allows a thread to sleep
>>   waiting on the first of a set of event sources (mutexes, timers,
>> signal,
>>   console input, etc) to signal.  Considering this is a primitive
>>   synchronization operation for Windows applications, being able to
>> quickly
>>   signal events on the producer side, and quickly go to sleep on the
>>   consumer side is essential for good performance of those running
>> over Wine.
>>
> 
> I know this is part of the cover letter, but I really do want to clarify
> that this isn't really accurate. The use case that this is referring to
> is not "the Wine implementation of WaitForMultipleObjects", it is an
> out-of-tree implementation of WaitForMultipleObjects that provides
> improved performance compared to the in-tree implementation.
> 
> This is especially salient because:
> 
> (1) this out-of-tree implementation is only in a small handful of cases
> any more performant than a different out-of-tree implementation which
> uses eventfd and poll() instead;
> 
> (2) these implementations will remain out-of-tree due to compatibility
> and robustness problems;
> 
> (3) I believe there is potential for an upstreamable implementation
> which does not rely on futex or futex2.

I'll let it more clear next time that this applies to Proton's Wine, and
not Wine.

Along with that, wait on multiple will be useful for other workloads,
such as the ones that uses Boost's mass locking algorithms and native
game engines for instance.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-04 11:36 ` Nicholas Piggin
@ 2021-06-04 20:01   ` André Almeida
  2021-06-05  1:09     ` Nicholas Piggin
  0 siblings, 1 reply; 53+ messages in thread
From: André Almeida @ 2021-06-04 20:01 UTC (permalink / raw)
  To: Nicholas Piggin
  Cc: acme, Andrey Semashev, corbet, Darren Hart, Peter Zijlstra,
	linux-kernel, Thomas Gleixner, Ingo Molnar, Davidlohr Bueso,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kselftest, malteskarupke, pgriffais, Peter Oskolkov, shuah,
	z.figura12, Steven Rostedt, Sebastian Andrzej Siewior

Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
> Excerpts from André Almeida's message of June 4, 2021 5:59 am:
>> Hi,
>>
>> This patch series introduces the futex2 syscalls.
>>
>> * What happened to the current futex()?
>>
>> For some years now, developers have been trying to add new features to
>> futex, but maintainers have been reluctant to accept then, given the
>> multiplexed interface full of legacy features and tricky to do big
>> changes. Some problems that people tried to address with patchsets are:
>> NUMA-awareness[0], smaller sized futexes[1], wait on multiple futexes[2].
>> NUMA, for instance, just doesn't fit the current API in a reasonable
>> way. Considering that, it's not possible to merge new features into the
>> current futex.
>>
>>  ** The NUMA problem
>>
>>  At the current implementation, all futex kernel side infrastructure is
>>  stored on a single node. Given that, all futex() calls issued by
>>  processors that aren't located on that node will have a memory access
>>  penalty when doing it.
>>
>>  ** The 32bit sized futex problem
>>
>>  Futexes are used to implement atomic operations in userspace.
>>  Supporting 8, 16, 32 and 64 bit sized futexes allows user libraries to
>>  implement all those sizes in a performant way. Thanks Boost devs for
>>  feedback: https://lists.boost.org/Archives/boost/2021/05/251508.php
>>
>>  Embedded systems or anything with memory constrains could benefit of
>>  using smaller sizes for the futex userspace integer.
>>
>>  ** The wait on multiple problem
>>
>>  The use case lies in the Wine implementation of the Windows NT interface
>>  WaitMultipleObjects. This Windows API function allows a thread to sleep
>>  waiting on the first of a set of event sources (mutexes, timers, signal,
>>  console input, etc) to signal.  Considering this is a primitive
>>  synchronization operation for Windows applications, being able to quickly
>>  signal events on the producer side, and quickly go to sleep on the
>>  consumer side is essential for good performance of those running over Wine.
>>
>> [0] https://lore.kernel.org/lkml/20160505204230.932454245@linutronix.de/
>> [1] https://lore.kernel.org/lkml/20191221155659.3159-2-malteskarupke@web.de/
>> [2] https://lore.kernel.org/lkml/20200213214525.183689-1-andrealmeid@collabora.com/
>>
>> * The solution
>>
>> As proposed by Peter Zijlstra and Florian Weimer[3], a new interface
>> is required to solve this, which must be designed with those features in
>> mind. futex2() is that interface. As opposed to the current multiplexed
>> interface, the new one should have one syscall per operation. This will
>> allow the maintainability of the API if it gets extended, and will help
>> users with type checking of arguments.
>>
>> In particular, the new interface is extended to support the ability to
>> wait on any of a list of futexes at a time, which could be seen as a
>> vectored extension of the FUTEX_WAIT semantics.
>>
>> [3] https://lore.kernel.org/lkml/20200303120050.GC2596@hirez.programming.kicks-ass.net/
>>
>> * The interface
>>
>> The new interface can be seen in details in the following patches, but
>> this is a high level summary of what the interface can do:
>>
>>  - Supports wake/wait semantics, as in futex()
>>  - Supports requeue operations, similarly as FUTEX_CMP_REQUEUE, but with
>>    individual flags for each address
>>  - Supports waiting for a vector of futexes, using a new syscall named
>>    futex_waitv()
>>  - Supports variable sized futexes (8bits, 16bits, 32bits and 64bits)
>>  - Supports NUMA-awareness operations, where the user can specify on
>>    which memory node would like to operate
> 
> A few comments.
> 
> - Do you need a syscall for wait and for waitv, or can wait just be a
> degenerate case of waitv (which could use the stack variable)?  I guess
> it does save the user access unlock.
> 

Yes. waitv with one element has a overhead compared to wait, given the
extra user_copy(). Given that waiting on a single futex is the common
case, I think it's worth to have it.

> - Did you consider a wakev interface? An example is a read-write mutex 
> which has read-blocking futexes split (e.g., per-node) for scalability 
> then the writer may unlock and wake all readers with one call. We 
> actually have some scalability challenges of this nature with certain 
> large database programs.
> 

Not really, I haven't heard any use case for that until now. It should
be easy to implement it, though, and I think you have an interesting use
case here. Could you point me some of those database programs?

> - Great to see 64-bit support in, it is helpful to do some interesting 
> things with locks without hacks (e.g., putting an address in the lock 
> word).
> 
> - Are we really keen on squashing node ID into flags in this day and age?
> I guess okay but seems like it would be nice to allow a bit more space
> in general for the operations. I don't want to turn it into a whole big
> multiplexing nightmare again with lots of such flags, or propose
> complexity with no code behind it, but I think we need a bit of leeway
> for unforeseen locking innovations to be added carefully. The pthread
> locking today is still fairly primitive really, I don't think we know
> what will work best for the next 10 years.

In the interface that I'd proposed, the node ID isn't part of the flags.
You have a flag FUTEX_FLAG_NUMA, and when that is used, you pass in
`void *uaddr` a pointer to a `struct futex_numa { int value, int hint
}`, where hint should be the node ID you would like to work on, and
value is just the userspace futex. This is documented in more details in
patch 7 "docs: locking: futex2: Add documentation".

If you have any feedback about how this NUMA interface looks like, I
would like to hear.

Also, did something in my writing indicated that the node ID would be
part of the flags? I'll improve this it if so.

> 
> One scalability issue we are starting to hit and will only get worse is 
> futex queue spinlock contention. Perhaps this is better addressed in 
> userspace but the kernel could play a part so I like to leave some doors
> open. One example is that the wait (or wake) side may like to depend not
> just on the memory value, but on the success of a cmpxchg to avoid 
> unqueueing and queueing spuriously, which increases lock contention but
> also ends up putting the poor task on the back of the list -- yes RT
> priorities can help the realtime case, but non-RT cases can get bad
> outlier latencies if lock stealing is allowed (which can be very good
> for performance).
> 

Sorry, I'm not sure what do you mean here. Are you proposing to have a
cmpxchg in kernel side, so the lock would be taken by the kernel, and
not by the userspace like it's now?

> - The private global futex hash table sucks for various reasons, and
> having 128 waiters per thread makes it orders of magnitude easier for
> userspace to DoS stuff with hash collisions. NUMA doesn't fix that, the
> per process hashing that Thomas suggested does fix the DoS but the
> non-deterministic hash collisions still seem to be a problem for real
> time response, and at the other end of the scale some apps (certain 
> databases, etc) can have ten thousand futex waiters at once so birthday
> paradox can also lead to guaranteed (low level) variable beahviour 
> within a single process.
> 
> I know the kernel in general is not very robust against this kind of 
> DoS/nondeterminism, but it's a bit sad to introduce new APIs with the 
> problem still there. Yes we could address it later, but I think it's 
> better done first because the solution might influence what the best 
> syscall API is.
> 
> For example the idea of using the address as the handle for the wait 
> queue _and_ the value is very convenient but hashing is annoying for
> all the above reasons and the shared wait queue case is pretty clunky. 
> It's also constraining in some corner cases to have the wait queue 
> associated with the address 1:1. For example a type of NUMA mutex might 
> want to have per-node waitqueues associated with a lock word, and wake
> local node waiters 99% of the time. Not trivial to do with futexes and
> seems to at least require bouncing of more cache lines, possibly more
> atomics, etc.
> 
> Could anything else be done?

I wasn't aware that userspace doing DoS is something to be concerned
from the kernel point of view. Is this scenario considering a malicious
actor? If so, there are plenty of resources to be denied, so not sure
how futex could be protected of this. Or is this just a program that
uses tons of futexes?

> 
> I'll be burned at the stake for suggesting it but it would be great if 
> we could use file descriptors. At least for the shared futex, maybe 
> private could use a per-process futex allocator. It solves all of the
> above, although I'm sure has many of its own problem. It may not play
> so nicely with the pthread mutex API because of the whole static
> initialiser problem, but the first futex proposal did use fds. But it's
> an example of an alternate API.
> 

FDs and futex doesn't play well, because for futex_wait() you need to
tell the kernel the expected value in the futex address to avoid
sleeping in a free lock. FD operations (poll, select) don't have this
`value` argument, so they could sleep forever, but I'm not sure if you
had taken this in consideration.

> And.. thanks for the great work, apologies if I missed some discussion
> related to the above comments, I did some reading but I know there has
> been a lot of discussions going on.
> 

:) and thank you for the valuable feedback and detailed ideas!

> Thanks,
> Nick
> 

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-04 20:01   ` André Almeida
@ 2021-06-05  1:09     ` Nicholas Piggin
  2021-06-05  8:56       ` Andrey Semashev
  2021-06-07 15:40       ` André Almeida
  0 siblings, 2 replies; 53+ messages in thread
From: Nicholas Piggin @ 2021-06-05  1:09 UTC (permalink / raw)
  To: André Almeida
  Cc: acme, Andrey Semashev, Sebastian Andrzej Siewior, corbet,
	Davidlohr Bueso, Darren Hart, fweimer, joel, kernel, krisman,
	libc-alpha, linux-api, linux-kernel, linux-kselftest,
	malteskarupke, Ingo Molnar, Peter Zijlstra, pgriffais,
	Peter Oskolkov, Steven Rostedt, shuah, Thomas Gleixner,
	z.figura12

Excerpts from André Almeida's message of June 5, 2021 6:01 am:
> Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
>> Excerpts from André Almeida's message of June 4, 2021 5:59 am:
>>> Hi,
>>>
>>> This patch series introduces the futex2 syscalls.
>>>
>>> * What happened to the current futex()?
>>>
>>> For some years now, developers have been trying to add new features to
>>> futex, but maintainers have been reluctant to accept then, given the
>>> multiplexed interface full of legacy features and tricky to do big
>>> changes. Some problems that people tried to address with patchsets are:
>>> NUMA-awareness[0], smaller sized futexes[1], wait on multiple futexes[2].
>>> NUMA, for instance, just doesn't fit the current API in a reasonable
>>> way. Considering that, it's not possible to merge new features into the
>>> current futex.
>>>
>>>  ** The NUMA problem
>>>
>>>  At the current implementation, all futex kernel side infrastructure is
>>>  stored on a single node. Given that, all futex() calls issued by
>>>  processors that aren't located on that node will have a memory access
>>>  penalty when doing it.
>>>
>>>  ** The 32bit sized futex problem
>>>
>>>  Futexes are used to implement atomic operations in userspace.
>>>  Supporting 8, 16, 32 and 64 bit sized futexes allows user libraries to
>>>  implement all those sizes in a performant way. Thanks Boost devs for
>>>  feedback: https://lists.boost.org/Archives/boost/2021/05/251508.php
>>>
>>>  Embedded systems or anything with memory constrains could benefit of
>>>  using smaller sizes for the futex userspace integer.
>>>
>>>  ** The wait on multiple problem
>>>
>>>  The use case lies in the Wine implementation of the Windows NT interface
>>>  WaitMultipleObjects. This Windows API function allows a thread to sleep
>>>  waiting on the first of a set of event sources (mutexes, timers, signal,
>>>  console input, etc) to signal.  Considering this is a primitive
>>>  synchronization operation for Windows applications, being able to quickly
>>>  signal events on the producer side, and quickly go to sleep on the
>>>  consumer side is essential for good performance of those running over Wine.
>>>
>>> [0] https://lore.kernel.org/lkml/20160505204230.932454245@linutronix.de/
>>> [1] https://lore.kernel.org/lkml/20191221155659.3159-2-malteskarupke@web.de/
>>> [2] https://lore.kernel.org/lkml/20200213214525.183689-1-andrealmeid@collabora.com/
>>>
>>> * The solution
>>>
>>> As proposed by Peter Zijlstra and Florian Weimer[3], a new interface
>>> is required to solve this, which must be designed with those features in
>>> mind. futex2() is that interface. As opposed to the current multiplexed
>>> interface, the new one should have one syscall per operation. This will
>>> allow the maintainability of the API if it gets extended, and will help
>>> users with type checking of arguments.
>>>
>>> In particular, the new interface is extended to support the ability to
>>> wait on any of a list of futexes at a time, which could be seen as a
>>> vectored extension of the FUTEX_WAIT semantics.
>>>
>>> [3] https://lore.kernel.org/lkml/20200303120050.GC2596@hirez.programming.kicks-ass.net/
>>>
>>> * The interface
>>>
>>> The new interface can be seen in details in the following patches, but
>>> this is a high level summary of what the interface can do:
>>>
>>>  - Supports wake/wait semantics, as in futex()
>>>  - Supports requeue operations, similarly as FUTEX_CMP_REQUEUE, but with
>>>    individual flags for each address
>>>  - Supports waiting for a vector of futexes, using a new syscall named
>>>    futex_waitv()
>>>  - Supports variable sized futexes (8bits, 16bits, 32bits and 64bits)
>>>  - Supports NUMA-awareness operations, where the user can specify on
>>>    which memory node would like to operate
>> 
>> A few comments.
>> 
>> - Do you need a syscall for wait and for waitv, or can wait just be a
>> degenerate case of waitv (which could use the stack variable)?  I guess
>> it does save the user access unlock.
>> 
> 
> Yes. waitv with one element has a overhead compared to wait, given the
> extra user_copy(). Given that waiting on a single futex is the common
> case, I think it's worth to have it.

Okay.

>> - Did you consider a wakev interface? An example is a read-write mutex 
>> which has read-blocking futexes split (e.g., per-node) for scalability 
>> then the writer may unlock and wake all readers with one call. We 
>> actually have some scalability challenges of this nature with certain 
>> large database programs.
>> 
> 
> Not really, I haven't heard any use case for that until now. It should
> be easy to implement it, though, and I think you have an interesting use
> case here. Could you point me some of those database programs?

Not source code unfortunately. I know that's not a very good answer, but 
they are far ahead of what most open source apps are doing scalability 
wise today, and they end up rolling their own complex locking. Hopefully
the example I give is simple enough to understand.

> 
>> - Great to see 64-bit support in, it is helpful to do some interesting 
>> things with locks without hacks (e.g., putting an address in the lock 
>> word).
>> 
>> - Are we really keen on squashing node ID into flags in this day and age?
>> I guess okay but seems like it would be nice to allow a bit more space
>> in general for the operations. I don't want to turn it into a whole big
>> multiplexing nightmare again with lots of such flags, or propose
>> complexity with no code behind it, but I think we need a bit of leeway
>> for unforeseen locking innovations to be added carefully. The pthread
>> locking today is still fairly primitive really, I don't think we know
>> what will work best for the next 10 years.
> 
> In the interface that I'd proposed, the node ID isn't part of the flags.
> You have a flag FUTEX_FLAG_NUMA, and when that is used, you pass in
> `void *uaddr` a pointer to a `struct futex_numa { int value, int hint
> }`, where hint should be the node ID you would like to work on, and
> value is just the userspace futex. This is documented in more details in
> patch 7 "docs: locking: futex2: Add documentation".
> 
> If you have any feedback about how this NUMA interface looks like, I
> would like to hear.
> 
> Also, did something in my writing indicated that the node ID would be
> part of the flags? I'll improve this it if so.

Oh I did miss this, thank you. No it wasn't your writing, I think it was 
me trying to read through a lot of messages and got confused with some
earlier conversations.

I'll look a bit more at the NUMA interface.

> 
>> 
>> One scalability issue we are starting to hit and will only get worse is 
>> futex queue spinlock contention. Perhaps this is better addressed in 
>> userspace but the kernel could play a part so I like to leave some doors
>> open. One example is that the wait (or wake) side may like to depend not
>> just on the memory value, but on the success of a cmpxchg to avoid 
>> unqueueing and queueing spuriously, which increases lock contention but
>> also ends up putting the poor task on the back of the list -- yes RT
>> priorities can help the realtime case, but non-RT cases can get bad
>> outlier latencies if lock stealing is allowed (which can be very good
>> for performance).
>> 
> 
> Sorry, I'm not sure what do you mean here. Are you proposing to have a
> cmpxchg in kernel side, so the lock would be taken by the kernel, and
> not by the userspace like it's now?

Yes. Only in slow paths, of course, to reduce starvation / erratic
latencies and spurious wait queue manipulations.

Actually one other scalability thing while I remember it:

futex_wait currently requires that the lock word is tested under the 
queue spin lock (to avoid consuming a wakeup). The problem with this is 
that the lock word can be a very hot cache line if you have a lot of
concurrency, so accessing it under the queue lock can increase queue
lock hold time.

I would prefer if the new API was relaxed to avoid this restriction
(e.g., any wait call may consume a wakeup so it's up to userspace to
avoid that if it is a problem).

>> - The private global futex hash table sucks for various reasons, and
>> having 128 waiters per thread makes it orders of magnitude easier for
>> userspace to DoS stuff with hash collisions. NUMA doesn't fix that, the
>> per process hashing that Thomas suggested does fix the DoS but the
>> non-deterministic hash collisions still seem to be a problem for real
>> time response, and at the other end of the scale some apps (certain 
>> databases, etc) can have ten thousand futex waiters at once so birthday
>> paradox can also lead to guaranteed (low level) variable beahviour 
>> within a single process.
>> 
>> I know the kernel in general is not very robust against this kind of 
>> DoS/nondeterminism, but it's a bit sad to introduce new APIs with the 
>> problem still there. Yes we could address it later, but I think it's 
>> better done first because the solution might influence what the best 
>> syscall API is.
>> 
>> For example the idea of using the address as the handle for the wait 
>> queue _and_ the value is very convenient but hashing is annoying for
>> all the above reasons and the shared wait queue case is pretty clunky. 
>> It's also constraining in some corner cases to have the wait queue 
>> associated with the address 1:1. For example a type of NUMA mutex might 
>> want to have per-node waitqueues associated with a lock word, and wake
>> local node waiters 99% of the time. Not trivial to do with futexes and
>> seems to at least require bouncing of more cache lines, possibly more
>> atomics, etc.
>> 
>> Could anything else be done?
> 
> I wasn't aware that userspace doing DoS is something to be concerned
> from the kernel point of view. Is this scenario considering a malicious
> actor? If so, there are plenty of resources to be denied, so not sure
> how futex could be protected of this. Or is this just a program that
> uses tons of futexes?

Both really. AFAIKS one of the efforts that prompted the futex 
modernisation work was the RT latency issues from Thomas in 2016 when 
the per process table was proposed.


>> I'll be burned at the stake for suggesting it but it would be great if 
>> we could use file descriptors. At least for the shared futex, maybe 
>> private could use a per-process futex allocator. It solves all of the
>> above, although I'm sure has many of its own problem. It may not play
>> so nicely with the pthread mutex API because of the whole static
>> initialiser problem, but the first futex proposal did use fds. But it's
>> an example of an alternate API.
>> 
> 
> FDs and futex doesn't play well, because for futex_wait() you need to
> tell the kernel the expected value in the futex address to avoid
> sleeping in a free lock. FD operations (poll, select) don't have this
> `value` argument, so they could sleep forever, but I'm not sure if you
> had taken this in consideration.

I had. The futex wait API would take a fd additional. The only 
difference is the waitqueue that is used when a sleep or wake is 
required is derived from the fd, not from an address.

I think the bigger sticking points would be if it's too heavyweight an 
object to use (which could be somewhat mitigated with a simpler ida
allocator although that's difficult to do with shared), and whether libc
could sanely use them due to the static initialiser problem of pthread
mutexes.

Thanks,
Nick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-05  1:09     ` Nicholas Piggin
@ 2021-06-05  8:56       ` Andrey Semashev
  2021-06-06 11:57         ` Nicholas Piggin
  2021-06-07 15:40       ` André Almeida
  1 sibling, 1 reply; 53+ messages in thread
From: Andrey Semashev @ 2021-06-05  8:56 UTC (permalink / raw)
  To: Nicholas Piggin, André Almeida
  Cc: acme, Sebastian Andrzej Siewior, corbet, Davidlohr Bueso,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, Peter Zijlstra, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

On 6/5/21 4:09 AM, Nicholas Piggin wrote:
> Excerpts from André Almeida's message of June 5, 2021 6:01 am:
>> Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
> 
>>> I'll be burned at the stake for suggesting it but it would be great if
>>> we could use file descriptors. At least for the shared futex, maybe
>>> private could use a per-process futex allocator. It solves all of the
>>> above, although I'm sure has many of its own problem. It may not play
>>> so nicely with the pthread mutex API because of the whole static
>>> initialiser problem, but the first futex proposal did use fds. But it's
>>> an example of an alternate API.
>>>
>>
>> FDs and futex doesn't play well, because for futex_wait() you need to
>> tell the kernel the expected value in the futex address to avoid
>> sleeping in a free lock. FD operations (poll, select) don't have this
>> `value` argument, so they could sleep forever, but I'm not sure if you
>> had taken this in consideration.
> 
> I had. The futex wait API would take a fd additional. The only
> difference is the waitqueue that is used when a sleep or wake is
> required is derived from the fd, not from an address.
> 
> I think the bigger sticking points would be if it's too heavyweight an
> object to use (which could be somewhat mitigated with a simpler ida
> allocator although that's difficult to do with shared), and whether libc
> could sanely use them due to the static initialiser problem of pthread
> mutexes.

The static initialization feature is not the only benefit of the current 
futex design, and probably not the most important one. You can work 
around the static initialization in userspace, e.g. by initializing fd 
to an invalid value and creating a valid fd upon the first use. Although 
that would still incur a performance penalty and add a new source of 
failure.

What is more important is that waiting on fd always requires a kernel 
call. This will be terrible for performance of uncontended locks, which 
is the majority of time.

Another important point is that a futex that is not being waited on 
consumes zero kernel resources while fd is a limited resource even when 
not used. You can have millions futexes in userspace and you are 
guaranteed not to exhaust any limit as long as you have memory. That is 
an important feature, and the current userspace is relying on it by 
assuming that creating mutexes and condition variables is cheap.

Having futex fd would be useful in some cases to be able to integrate 
futexes with IO. I did have use cases where I would have liked to have 
FUTEX_FD in the past. These cases arise when you already have a thread 
that operates on fds and you want to avoid having a separate thread that 
blocks on futexes in a similar fashion. But, IMO, that should be an 
optional opt-in feature. By far, not every futex needs to have an fd. 
For just waiting on multiple futexes, the native support that futex2 
provides is superior.

PS: I'm not asking FUTEX_FD to be implemented as part of futex2 API. 
futex2 would be great even without it.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-05  8:56       ` Andrey Semashev
@ 2021-06-06 11:57         ` Nicholas Piggin
  2021-06-06 13:15           ` Andrey Semashev
  0 siblings, 1 reply; 53+ messages in thread
From: Nicholas Piggin @ 2021-06-06 11:57 UTC (permalink / raw)
  To: André Almeida, Andrey Semashev
  Cc: acme, Sebastian Andrzej Siewior, corbet, Davidlohr Bueso,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, Peter Zijlstra, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

Excerpts from Andrey Semashev's message of June 5, 2021 6:56 pm:
> On 6/5/21 4:09 AM, Nicholas Piggin wrote:
>> Excerpts from André Almeida's message of June 5, 2021 6:01 am:
>>> Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
>> 
>>>> I'll be burned at the stake for suggesting it but it would be great if
>>>> we could use file descriptors. At least for the shared futex, maybe
>>>> private could use a per-process futex allocator. It solves all of the
>>>> above, although I'm sure has many of its own problem. It may not play
>>>> so nicely with the pthread mutex API because of the whole static
>>>> initialiser problem, but the first futex proposal did use fds. But it's
>>>> an example of an alternate API.
>>>>
>>>
>>> FDs and futex doesn't play well, because for futex_wait() you need to
>>> tell the kernel the expected value in the futex address to avoid
>>> sleeping in a free lock. FD operations (poll, select) don't have this
>>> `value` argument, so they could sleep forever, but I'm not sure if you
>>> had taken this in consideration.
>> 
>> I had. The futex wait API would take a fd additional. The only
>> difference is the waitqueue that is used when a sleep or wake is
>> required is derived from the fd, not from an address.
>> 
>> I think the bigger sticking points would be if it's too heavyweight an
>> object to use (which could be somewhat mitigated with a simpler ida
>> allocator although that's difficult to do with shared), and whether libc
>> could sanely use them due to the static initialiser problem of pthread
>> mutexes.
> 
> The static initialization feature is not the only benefit of the current 
> futex design, and probably not the most important one. You can work 
> around the static initialization in userspace, e.g. by initializing fd 
> to an invalid value and creating a valid fd upon the first use. Although 
> that would still incur a performance penalty and add a new source of 
> failure.

Sounds like a serious problem, but maybe it isn't. On the other hand,
maybe we don't have to support pthread mutexes as they are anyway 
because futex already does that fairly well.

> What is more important is that waiting on fd always requires a kernel 
> call. This will be terrible for performance of uncontended locks, which 
> is the majority of time.

No. As I said just before, it would be the same except the waitqueue is 
derived from fd rather than address.

> 
> Another important point is that a futex that is not being waited on 
> consumes zero kernel resources while fd is a limited resource even when 
> not used. You can have millions futexes in userspace and you are 
> guaranteed not to exhaust any limit as long as you have memory. That is 
> an important feature, and the current userspace is relying on it by 
> assuming that creating mutexes and condition variables is cheap.

Is it an important feture? Would 1 byte of kernel memory per uncontended 
futex be okay? 10? 100?

I do see it's very nice the current design that requires no 
initialization for uncontended, I'm just asking questions to get an idea 
of what constraints we're working with. We have a pretty good API 
already which can support unlimited uncontended futexes, so I'm 
wondering do we really need another very very similar API that doesn't
fix the really difficult problems of the existing one?

Thanks,
Nick

> Having futex fd would be useful in some cases to be able to integrate 
> futexes with IO. I did have use cases where I would have liked to have 
> FUTEX_FD in the past. These cases arise when you already have a thread 
> that operates on fds and you want to avoid having a separate thread that 
> blocks on futexes in a similar fashion. But, IMO, that should be an 
> optional opt-in feature. By far, not every futex needs to have an fd. 
> For just waiting on multiple futexes, the native support that futex2 
> provides is superior.
> 
> PS: I'm not asking FUTEX_FD to be implemented as part of futex2 API. 
> futex2 would be great even without it.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-06 11:57         ` Nicholas Piggin
@ 2021-06-06 13:15           ` Andrey Semashev
  2021-06-08  1:25             ` Nicholas Piggin
  0 siblings, 1 reply; 53+ messages in thread
From: Andrey Semashev @ 2021-06-06 13:15 UTC (permalink / raw)
  To: Nicholas Piggin, André Almeida
  Cc: acme, Sebastian Andrzej Siewior, corbet, Davidlohr Bueso,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, Peter Zijlstra, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

On 6/6/21 2:57 PM, Nicholas Piggin wrote:
> Excerpts from Andrey Semashev's message of June 5, 2021 6:56 pm:
>> On 6/5/21 4:09 AM, Nicholas Piggin wrote:
>>> Excerpts from André Almeida's message of June 5, 2021 6:01 am:
>>>> Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
>>>
>>>>> I'll be burned at the stake for suggesting it but it would be great if
>>>>> we could use file descriptors. At least for the shared futex, maybe
>>>>> private could use a per-process futex allocator. It solves all of the
>>>>> above, although I'm sure has many of its own problem. It may not play
>>>>> so nicely with the pthread mutex API because of the whole static
>>>>> initialiser problem, but the first futex proposal did use fds. But it's
>>>>> an example of an alternate API.
>>>>>
>>>>
>>>> FDs and futex doesn't play well, because for futex_wait() you need to
>>>> tell the kernel the expected value in the futex address to avoid
>>>> sleeping in a free lock. FD operations (poll, select) don't have this
>>>> `value` argument, so they could sleep forever, but I'm not sure if you
>>>> had taken this in consideration.
>>>
>>> I had. The futex wait API would take a fd additional. The only
>>> difference is the waitqueue that is used when a sleep or wake is
>>> required is derived from the fd, not from an address.
>>>
>>> I think the bigger sticking points would be if it's too heavyweight an
>>> object to use (which could be somewhat mitigated with a simpler ida
>>> allocator although that's difficult to do with shared), and whether libc
>>> could sanely use them due to the static initialiser problem of pthread
>>> mutexes.
>>
>> The static initialization feature is not the only benefit of the current
>> futex design, and probably not the most important one. You can work
>> around the static initialization in userspace, e.g. by initializing fd
>> to an invalid value and creating a valid fd upon the first use. Although
>> that would still incur a performance penalty and add a new source of
>> failure.
> 
> Sounds like a serious problem, but maybe it isn't. On the other hand,
> maybe we don't have to support pthread mutexes as they are anyway
> because futex already does that fairly well.
> 
>> What is more important is that waiting on fd always requires a kernel
>> call. This will be terrible for performance of uncontended locks, which
>> is the majority of time.
> 
> No. As I said just before, it would be the same except the waitqueue is
> derived from fd rather than address.

Sorry, in that case I'm not sure I understand how that would work. You 
do need to allocate a fd, do you?

>> Another important point is that a futex that is not being waited on
>> consumes zero kernel resources while fd is a limited resource even when
>> not used. You can have millions futexes in userspace and you are
>> guaranteed not to exhaust any limit as long as you have memory. That is
>> an important feature, and the current userspace is relying on it by
>> assuming that creating mutexes and condition variables is cheap.
> 
> Is it an important feture? Would 1 byte of kernel memory per uncontended
> futex be okay? 10? 100?
> 
> I do see it's very nice the current design that requires no
> initialization for uncontended, I'm just asking questions to get an idea
> of what constraints we're working with. We have a pretty good API
> already which can support unlimited uncontended futexes, so I'm
> wondering do we really need another very very similar API that doesn't
> fix the really difficult problems of the existing one?

It does provide the very much needed features that are missing in the 
current futex. Namely, more futex sizes and wait for multiple. So the 
argument of "why have two similar APIs" is not quite fair. It would be, 
if there was feature parity with futex.

I believe, the low cost of a futex is an important feature, and was one 
of the reasons for its original design and introduction. Otherwise we 
would be using eventfds in mutexes.

One other feature that I didn't mention earlier and which follows from 
its "address in memory" design is the ability to use futexes in 
process-shared memory. This is important for process-shared pthread 
components, too, but has its own value even without this, if you use 
futexes directly. With fds, you can't place the fd in a shared memory 
since every process needs to have its own fd referring to the same 
kernel object, and passing fds cannot be done without a UNIX socket. 
This is incompatible with pthreads API design and would require 
non-trivial design changes to the applications using futexes directly.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 05/15] futex2: Implement support for different futex sizes
  2021-06-03 19:59 ` [PATCH v4 05/15] futex2: Implement support for different futex sizes André Almeida
  2021-06-04  0:23   ` kernel test robot
@ 2021-06-06 19:12   ` Davidlohr Bueso
  2021-06-06 23:01     ` Andrey Semashev
  1 sibling, 1 reply; 53+ messages in thread
From: Davidlohr Bueso @ 2021-06-06 19:12 UTC (permalink / raw)
  To: Andr� Almeida
  Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior, kernel,
	krisman, pgriffais, z.figura12, joel, malteskarupke, linux-api,
	fweimer, libc-alpha, linux-kselftest, shuah, acme, corbet,
	Peter Oskolkov, Andrey Semashev, torvalds

On Thu, 03 Jun 2021, Andr� Almeida wrote:

>Implement support for 8, 16 and 64 bit futexes, along with the existing
>32 bit support. Userspace should use flags to specify in the syscall
>the size of the *uaddr they are operating on.
>
>Variable sized futexes are useful for implementing atomic primitives in
>userspace in an efficient manner. 64bit sized futexes are also
>particularly useful when userspace stores information to be used in an
>atomic fashion on the futex value, given more room for flexibility.

Note that at least in the past, Linus has been vehemently against 64-bit
futexes.

Basically this additional data, like for implementing read/write locks,
does not need to be in the futex atomic wait/wake parts. You can instead
split the userspace lock into two adjacent 32-bit words and do 64-bit
atomic ops on it.

Of course, this is a new interface altogether, so this time it might
be fair game.

Thanks,
Davidlohr

>
>Overlapping futexes are not allowed, so userspace can't wait and wake on
>the same memory address if the are using different sizes.
>
>Signed-off-by: André Almeida <andrealmeid@collabora.com>
>---
> include/uapi/linux/futex.h |   3 +
> kernel/futex2.c            | 124 ++++++++++++++++++++++++-------------
> 2 files changed, 84 insertions(+), 43 deletions(-)
>
>diff --git a/include/uapi/linux/futex.h b/include/uapi/linux/futex.h
>index 06ea9bdfa69e..5786270b0c75 100644
>--- a/include/uapi/linux/futex.h
>+++ b/include/uapi/linux/futex.h
>@@ -42,7 +42,10 @@
>					 FUTEX_PRIVATE_FLAG)
>
> /* Size argument to futex2 syscall */
>+#define FUTEX_8		0
>+#define FUTEX_16	1
> #define FUTEX_32	2
>+#define FUTEX_64	3
>
> #define FUTEX_SIZE_MASK	0x3
>
>diff --git a/kernel/futex2.c b/kernel/futex2.c
>index 012d7f7fc17a..1e97e5f2e793 100644
>--- a/kernel/futex2.c
>+++ b/kernel/futex2.c
>@@ -89,9 +89,11 @@ struct futex_bucket {
> #define FUTEXV_WAITER_MASK (FUTEX_SIZE_MASK | FUTEX_SHARED_FLAG)
>
> #define is_object_shared ((futexv->objects[i].flags & FUTEX_SHARED_FLAG) ? true : false)
>+#define object_size (futexv->objects[i].flags & FUTEX_SIZE_MASK)
>
>-#define FUT_OFF_INODE    1 /* We set bit 0 if key has a reference on inode */
>-#define FUT_OFF_MMSHARED 2 /* We set bit 1 if key has a reference on mm */
>+#define FUT_OFF_INODE    PAGE_SIZE
>+#define FUT_OFF_MMSHARED (PAGE_SIZE << 1)
>+#define FUT_OFF_SIZE     1
>
> static struct futex_bucket *futex_table;
> static unsigned int futex2_hashsize;
>@@ -321,6 +323,7 @@ static int futex_get_shared_key(uintptr_t address, struct mm_struct *mm,
>  * @uaddr:   futex user address
>  * @key:     data that uniquely identifies a futex
>  * @shared:  is this a shared futex?
>+ * @flags:   flags for the size
>  *
>  * For private futexes, each uaddr will be unique for a given mm_struct, and it
>  * won't be freed for the life time of the process. For shared futexes, check
>@@ -330,21 +333,41 @@ static int futex_get_shared_key(uintptr_t address, struct mm_struct *mm,
>  */
> static struct futex_bucket *futex_get_bucket(void __user *uaddr,
>					     struct futex_key *key,
>-					     bool shared)
>+					     bool shared, unsigned int flags)
> {
>	uintptr_t address = (uintptr_t)uaddr;
>	u32 hash_key;
>
>+	size_t size;
>+
>+	switch (flags) {
>+	case FUTEX_8:
>+		size = sizeof(u8);
>+		break;
>+	case FUTEX_16:
>+		size = sizeof(u16);
>+		break;
>+	case FUTEX_32:
>+		size = sizeof(u32);
>+		break;
>+	case FUTEX_64:
>+		size = sizeof(u64);
>+		break;
>+	default:
>+		return ERR_PTR(-EINVAL);
>+	}
>+
>	/* Checking if uaddr is valid and accessible */
>-	if (unlikely(!IS_ALIGNED(address, sizeof(u32))))
>+	if (unlikely(!IS_ALIGNED(address, size)))
>		return ERR_PTR(-EINVAL);
>-	if (unlikely(!access_ok(uaddr, sizeof(u32))))
>+	if (unlikely(!access_ok(uaddr, size)))
>		return ERR_PTR(-EFAULT);
>
>	key->offset = address % PAGE_SIZE;
>	address -= key->offset;
>	key->pointer = (u64)address;
>	key->index = (unsigned long)current->mm;
>+	key->offset |= FUT_OFF_SIZE << (size - sizeof(u8));
>
>	if (shared)
>		futex_get_shared_key(address, current->mm, key);
>@@ -358,18 +381,39 @@ static struct futex_bucket *futex_get_bucket(void __user *uaddr,
>
> /**
>  * futex_get_user - Get the userspace value on this address
>- * @uval:  variable to store the value
>- * @uaddr: userspace address
>+ * @uval:	variable to store the value
>+ * @uaddr:	userspace address
>+ * @pagefault:	true if pagefault should be disabled
>+ * @flags:	flags for the size
>  *
>  * Check the comment at futex_enqueue() for more information.
>  */
>-static int futex_get_user(u32 *uval, u32 __user *uaddr)
>+static int futex_get_user(u64 *uval, void __user *uaddr, unsigned int flags, bool pagefault)
> {
>	int ret;
>
>-	pagefault_disable();
>-	ret = __get_user(*uval, uaddr);
>-	pagefault_enable();
>+	if (pagefault)
>+		pagefault_disable();
>+
>+	switch (flags) {
>+	case FUTEX_8:
>+		ret = __get_user(*uval, (u8 __user *)uaddr);
>+		break;
>+	case FUTEX_16:
>+		ret = __get_user(*uval, (u16 __user *)uaddr);
>+		break;
>+	case FUTEX_32:
>+		ret = __get_user(*uval, (u32 __user *)uaddr);
>+		break;
>+	case FUTEX_64:
>+		ret = __get_user(*uval, (u64 __user *)uaddr);
>+		break;
>+	default:
>+		BUG();
>+	}
>+
>+	if (pagefault)
>+		pagefault_enable();
>
>	return ret;
> }
>@@ -484,8 +528,8 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
>			 int *awakened)
> {
>	int i, ret;
>-	u32 uval, val;
>-	u32 __user *uaddr;
>+	u64 uval, val;
>+	void __user *uaddr;
>	bool retry = false;
>	struct futex_bucket *bucket;
>
>@@ -493,13 +537,14 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
>	set_current_state(TASK_INTERRUPTIBLE);
>
>	for (i = 0; i < nr_futexes; i++) {
>-		uaddr = (u32 __user *)futexv->objects[i].uaddr;
>-		val = (u32)futexv->objects[i].val;
>+		uaddr = futexv->objects[i].uaddr;
>+		val = (u64)futexv->objects[i].val;
>
>		if (is_object_shared && retry) {
>			struct futex_bucket *tmp =
>				futex_get_bucket((void __user *)uaddr,
>-						 &futexv->objects[i].key, true);
>+						 &futexv->objects[i].key, true,
>+						 object_size);
>			if (IS_ERR(tmp)) {
>				__set_current_state(TASK_RUNNING);
>				futex_dequeue_multiple(futexv, i);
>@@ -513,7 +558,7 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
>		bucket_inc_waiters(bucket);
>		spin_lock(&bucket->lock);
>
>-		ret = futex_get_user(&uval, uaddr);
>+		ret = futex_get_user(&uval, uaddr, object_size, true);
>
>		if (unlikely(ret)) {
>			spin_unlock(&bucket->lock);
>@@ -525,7 +570,7 @@ static int futex_enqueue(struct futex_waiter_head *futexv, unsigned int nr_futex
>			if (*awakened >= 0)
>				return 1;
>
>-			if (__get_user(uval, uaddr))
>+			if (futex_get_user(&uval, uaddr, object_size, false))
>				return -EFAULT;
>
>			retry = true;
>@@ -656,9 +701,6 @@ static long ksys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
>	if (flags & ~FUTEX2_MASK)
>		return -EINVAL;
>
>-	if (size != FUTEX_32)
>-		return -EINVAL;
>-
>	futexv = &wait_single.futexv;
>	futexv->task = current;
>	futexv->hint = false;
>@@ -667,12 +709,13 @@ static long ksys_futex_wait(void __user *uaddr, u64 val, unsigned int flags,
>	waiter->index = 0;
>	waiter->val = val;
>	waiter->uaddr = uaddr;
>+	waiter->flags = flags;
>	memset(&wait_single.waiter.key, 0, sizeof(struct futex_key));
>
>	INIT_LIST_HEAD(&waiter->list);
>
>	/* Get an unlocked hash bucket */
>-	waiter->bucket = futex_get_bucket(uaddr, &waiter->key, shared);
>+	waiter->bucket = futex_get_bucket(uaddr, &waiter->key, shared, size);
>	if (IS_ERR(waiter->bucket))
>		return PTR_ERR(waiter->bucket);
>
>@@ -728,8 +771,7 @@ static int compat_futex_parse_waitv(struct futex_waiter_head *futexv,
>		if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
>			return -EFAULT;
>
>-		if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
>-		    (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
>+		if (waitv.flags & ~FUTEXV_WAITER_MASK)
>			return -EINVAL;
>
>		futexv->objects[i].key.pointer = 0;
>@@ -740,7 +782,7 @@ static int compat_futex_parse_waitv(struct futex_waiter_head *futexv,
>
>		bucket = futex_get_bucket(compat_ptr(waitv.uaddr),
>					  &futexv->objects[i].key,
>-					  is_object_shared);
>+					  is_object_shared, object_size);
>
>		if (IS_ERR(bucket))
>			return PTR_ERR(bucket);
>@@ -805,8 +847,7 @@ static int futex_parse_waitv(struct futex_waiter_head *futexv,
>		if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
>			return -EFAULT;
>
>-		if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
>-		    (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
>+		if (waitv.flags & ~FUTEXV_WAITER_MASK)
>			return -EINVAL;
>
>		futexv->objects[i].key.pointer = 0;
>@@ -816,7 +857,7 @@ static int futex_parse_waitv(struct futex_waiter_head *futexv,
>		futexv->objects[i].index  = i;
>
>		bucket = futex_get_bucket(waitv.uaddr, &futexv->objects[i].key,
>-					  is_object_shared);
>+					  is_object_shared, object_size);
>
>		if (IS_ERR(bucket))
>			return PTR_ERR(bucket);
>@@ -947,10 +988,7 @@ SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, unsigned int, nr_wake,
>	if (flags & ~FUTEX2_MASK)
>		return -EINVAL;
>
>-	if (size != FUTEX_32)
>-		return -EINVAL;
>-
>-	bucket = futex_get_bucket(uaddr, &waiter.key, shared);
>+	bucket = futex_get_bucket(uaddr, &waiter.key, shared, size);
>	if (IS_ERR(bucket))
>		return PTR_ERR(bucket);
>
>@@ -987,28 +1025,30 @@ static inline int __futex_requeue(struct futex_requeue rq1,
>	bool retry = false;
>	struct futex_bucket *b1, *b2;
>	DEFINE_WAKE_Q(wake_q);
>-	u32 uval;
>+	u64 uval;
>	int ret;
>	bool shared1 = (rq1.flags  & FUTEX_SHARED_FLAG) ? true : false;
>	bool shared2 = (rq2.flags  & FUTEX_SHARED_FLAG) ? true : false;
>+	unsigned int size1 = (rq1.flags  & FUTEX_SIZE_MASK);
>+	unsigned int size2 = (rq2.flags  & FUTEX_SIZE_MASK);
>
>-	b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
>+	b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1, size1);
>	if (IS_ERR(b1))
>		return PTR_ERR(b1);
>
>-	b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
>+	b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2, size2);
>	if (IS_ERR(b2))
>		return PTR_ERR(b2);
>
> retry:
>	if (shared1 && retry) {
>-		b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
>+		b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1, size1);
>		if (IS_ERR(b1))
>			return PTR_ERR(b1);
>	}
>
>	if (shared2 && retry) {
>-		b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
>+		b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2, size2);
>		if (IS_ERR(b2))
>			return PTR_ERR(b2);
>	}
>@@ -1027,11 +1067,11 @@ static inline int __futex_requeue(struct futex_requeue rq1,
>		spin_lock_nested(&b1->lock, SINGLE_DEPTH_NESTING);
>	}
>
>-	ret = futex_get_user(&uval, rq1.uaddr);
>+	ret = futex_get_user(&uval, rq1.uaddr, size1, true);
>
>	if (unlikely(ret)) {
>		futex_double_unlock(b1, b2);
>-		if (__get_user(uval, (u32 __user *)rq1.uaddr))
>+		if (futex_get_user(&uval, rq1.uaddr, size1, false))
>			return -EFAULT;
>
>		bucket_dec_waiters(b2);
>@@ -1088,8 +1128,7 @@ static int compat_futex_parse_requeue(struct futex_requeue *rq,
>	if (copy_from_user(&tmp, uaddr, sizeof(tmp)))
>		return -EFAULT;
>
>-	if (tmp.flags & ~FUTEXV_WAITER_MASK ||
>-	    (tmp.flags & FUTEX_SIZE_MASK) != FUTEX_32)
>+	if (tmp.flags & ~FUTEXV_WAITER_MASK)
>		return -EINVAL;
>
>	rq->uaddr = compat_ptr(tmp.uaddr);
>@@ -1134,8 +1173,7 @@ static int futex_parse_requeue(struct futex_requeue *rq,
>	if (copy_from_user(rq, uaddr, sizeof(*rq)))
>		return -EFAULT;
>
>-	if (rq->flags & ~FUTEXV_WAITER_MASK ||
>-	    (rq->flags & FUTEX_SIZE_MASK) != FUTEX_32)
>+	if (rq->flags & ~FUTEXV_WAITER_MASK)
>		return -EINVAL;
>
>	return 0;
>--
>2.31.1
>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 07/15] docs: locking: futex2: Add documentation
  2021-06-03 19:59 ` [PATCH v4 07/15] docs: locking: futex2: Add documentation André Almeida
@ 2021-06-06 19:23   ` Davidlohr Bueso
  0 siblings, 0 replies; 53+ messages in thread
From: Davidlohr Bueso @ 2021-06-06 19:23 UTC (permalink / raw)
  Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior, kernel,
	krisman, pgriffais, z.figura12, joel, malteskarupke, linux-api,
	fweimer, libc-alpha, linux-kselftest, shuah, acme, corbet,
	Peter Oskolkov, Andrey Semashev, mtk.manpages

On Thu, 03 Jun 2021, Andr� Almeida wrote:

>Add a new documentation file specifying both userspace API and internal
>implementation details of futex2 syscalls.

I think equally important would be to provide a manpage for each new
syscall you are introducing, and keep mkt in the loop as in the past he
extensively documented and improved futex manpages, and overall has a
lot of experience with dealing with kernel interfaces.

Thanks,
Davidlohr

>
>Signed-off-by: André Almeida <andrealmeid@collabora.com>
>---
> Documentation/locking/futex2.rst | 198 +++++++++++++++++++++++++++++++
> Documentation/locking/index.rst  |   1 +
> 2 files changed, 199 insertions(+)
> create mode 100644 Documentation/locking/futex2.rst
>
>diff --git a/Documentation/locking/futex2.rst b/Documentation/locking/futex2.rst
>new file mode 100644
>index 000000000000..2f74d7c97a55
>--- /dev/null
>+++ b/Documentation/locking/futex2.rst
>@@ -0,0 +1,198 @@
>+.. SPDX-License-Identifier: GPL-2.0
>+
>+======
>+futex2
>+======
>+
>+:Author: André Almeida <andrealmeid@collabora.com>
>+
>+futex, or fast user mutex, is a set of syscalls to allow userspace to create
>+performant synchronization mechanisms, such as mutexes, semaphores and
>+conditional variables in userspace. C standard libraries, like glibc, uses it
>+as a means to implement more high level interfaces like pthreads.
>+
>+The interface
>+=============
>+
>+uAPI functions
>+--------------
>+
>+.. kernel-doc:: kernel/futex2.c
>+   :identifiers: sys_futex_wait sys_futex_wake sys_futex_waitv sys_futex_requeue
>+
>+uAPI structures
>+---------------
>+
>+.. kernel-doc:: include/uapi/linux/futex.h
>+
>+The ``flag`` argument
>+---------------------
>+
>+The flag is used to specify the size of the futex word
>+(FUTEX_[8, 16, 32, 64]). It's mandatory to define one, since there's no
>+default size.
>+
>+By default, the timeout uses a monotonic clock, but can be used as a realtime
>+one by using the FUTEX_REALTIME_CLOCK flag.
>+
>+By default, futexes are of the private type, that means that this user address
>+will be accessed by threads that share the same memory region. This allows for
>+some internal optimizations, so they are faster. However, if the address needs
>+to be shared with different processes (like using ``mmap()`` or ``shm()``), they
>+need to be defined as shared and the flag FUTEX_SHARED_FLAG is used to set that.
>+
>+By default, the operation has no NUMA-awareness, meaning that the user can't
>+choose the memory node where the kernel side futex data will be stored. The
>+user can choose the node where it wants to operate by setting the
>+FUTEX_NUMA_FLAG and using the following structure (where X can be 8, 16, 32 or
>+64)::
>+
>+ struct futexX_numa {
>+         __uX value;
>+         __sX hint;
>+ };
>+
>+This structure should be passed at the ``void *uaddr`` of futex functions. The
>+address of the structure will be used to be waited on/waken on, and the
>+``value`` will be compared to ``val`` as usual. The ``hint`` member is used to
>+define which node the futex will use. When waiting, the futex will be
>+registered on a kernel-side table stored on that node; when waking, the futex
>+will be searched for on that given table. That means that there's no redundancy
>+between tables, and the wrong ``hint`` value will lead to undesired behavior.
>+Userspace is responsible for dealing with node migrations issues that may
>+occur. ``hint`` can range from [0, MAX_NUMA_NODES), for specifying a node, or
>+-1, to use the same node the current process is using.
>+
>+When not using FUTEX_NUMA_FLAG on a NUMA system, the futex will be stored on a
>+global table on allocated on the first node.
>+
>+The ``timo`` argument
>+---------------------
>+
>+As per the Y2038 work done in the kernel, new interfaces shouldn't add timeout
>+options known to be buggy. Given that, ``timo`` should be a 64-bit timeout at
>+all platforms, using an absolute timeout value.
>+
>+Implementation
>+==============
>+
>+The internal implementation follows a similar design to the original futex.
>+Given that we want to replicate the same external behavior of current futex,
>+this should be somewhat expected.
>+
>+Waiting
>+-------
>+
>+For the wait operations, they are all treated as if you want to wait on N
>+futexes, so the path for futex_wait and futex_waitv is the basically the same.
>+For both syscalls, the first step is to prepare an internal list for the list
>+of futexes to wait for (using struct futexv_head). For futex_wait() calls, this
>+list will have a single object.
>+
>+We have a hash table, where waiters register themselves before sleeping. Then
>+the wake function checks this table looking for waiters at uaddr.  The hash
>+bucket to be used is determined by a struct futex_key, that stores information
>+to uniquely identify an address from a given process. Given the huge address
>+space, there'll be hash collisions, so we store information to be later used on
>+collision treatment.
>+
>+First, for every futex we want to wait on, we check if (``*uaddr == val``).
>+This check is done holding the bucket lock, so we are correctly serialized with
>+any futex_wake() calls. If any waiter fails the check above, we dequeue all
>+futexes. The check (``*uaddr == val``) can fail for two reasons:
>+
>+- The values are different, and we return -EAGAIN. However, if while
>+  dequeueing we found that some futexes were awakened, we prioritize this
>+  and return success.
>+
>+- When trying to access the user address, we do so with page faults
>+  disabled because we are holding a bucket's spin lock (and can't sleep
>+  while holding a spin lock). If there's an error, it might be a page
>+  fault, or an invalid address. We release the lock, dequeue everyone
>+  (because it's illegal to sleep while there are futexes enqueued, we
>+  could lose wakeups) and try again with page fault enabled. If we
>+  succeed, this means that the address is valid, but we need to do
>+  all the work again. For serialization reasons, we need to have the
>+  spin lock when getting the user value. Additionally, for shared
>+  futexes, we also need to recalculate the hash, since the underlying
>+  mapping mechanisms could have changed when dealing with page fault.
>+  If, even with page fault enabled, we can't access the address, it
>+  means it's an invalid user address, and we return -EFAULT. For this
>+  case, we prioritize the error, even if some futexes were awaken.
>+
>+If the check is OK, they are enqueued on a linked list in our bucket, and
>+proceed to the next one. If all waiters succeed, we put the thread to sleep
>+until a futex_wake() call, timeout expires or we get a signal. After waking up,
>+we dequeue everyone, and check if some futex was awakened. This dequeue is done
>+by iteratively walking at each element of struct futex_head list.
>+
>+All enqueuing/dequeuing operations requires to hold the bucket lock, to avoid
>+racing while modifying the list.
>+
>+Waking
>+------
>+
>+We get the bucket that's storing the waiters at uaddr, and wake the required
>+number of waiters, checking for hash collision.
>+
>+There's an optimization that makes futex_wake() not take the bucket lock if
>+there's no one to be woken on that bucket. It checks an atomic counter that each
>+bucket has, if it says 0, then the syscall exits. In order for this to work, the
>+waiter thread increases it before taking the lock, so the wake thread will
>+correctly see that there's someone waiting and will continue the path to take
>+the bucket lock. To get the correct serialization, the waiter issues a memory
>+barrier after increasing the bucket counter and the waker issues a memory
>+barrier before checking it.
>+
>+Requeuing
>+---------
>+
>+The requeue path first checks for each struct futex_requeue and their flags.
>+Then, it will compare the expected value with the one at uaddr1::uaddr.
>+Following the same serialization explained at Waking_, we increase the atomic
>+counter for the bucket of uaddr2 before taking the lock. We need to have both
>+buckets locks at same time so we don't race with other futex operation. To
>+ensure the locks are taken in the same order for all threads (and thus avoiding
>+deadlocks), every requeue operation takes the "smaller" bucket first, when
>+comparing both addresses.
>+
>+If the compare with user value succeeds, we proceed by waking ``nr_wake``
>+futexes, and then requeuing ``nr_requeue`` from bucket of uaddr1 to the uaddr2.
>+This consists in a simple list deletion/addition and replacing the old futex key
>+with the new one.
>+
>+Futex keys
>+----------
>+
>+There are two types of futexes: private and shared ones. The private are futexes
>+meant to be used by threads that share the same memory space, are easier to be
>+uniquely identified and thus can have some performance optimization. The
>+elements for identifying one are: the start address of the page where the
>+address is, the address offset within the page and the current->mm pointer.
>+
>+Now, for uniquely identifying a shared futex:
>+
>+- If the page containing the user address is an anonymous page, we can
>+  just use the same data used for private futexes (the start address of
>+  the page, the address offset within the page and the current->mm
>+  pointer); that will be enough for uniquely identifying such futex. We
>+  also set one bit at the key to differentiate if a private futex is
>+  used on the same address (mixing shared and private calls does not
>+  work).
>+
>+- If the page is file-backed, current->mm maybe isn't the same one for
>+  every user of this futex, so we need to use other data: the
>+  page->index, a UUID for the struct inode and the offset within the
>+  page.
>+
>+Note that members of futex_key don't have any particular meaning after they
>+are part of the struct - they are just bytes to identify a futex.  Given that,
>+we don't need to use a particular name or type that matches the original data,
>+we only need to care about the bitsize of each component and make both private
>+and shared fit in the same memory space.
>+
>+Source code documentation
>+=========================
>+
>+.. kernel-doc:: kernel/futex2.c
>+   :no-identifiers: sys_futex_wait sys_futex_wake sys_futex_waitv sys_futex_requeue
>diff --git a/Documentation/locking/index.rst b/Documentation/locking/index.rst
>index 7003bd5aeff4..9bf03c7fa1ec 100644
>--- a/Documentation/locking/index.rst
>+++ b/Documentation/locking/index.rst
>@@ -24,6 +24,7 @@ locking
>     percpu-rw-semaphore
>     robust-futexes
>     robust-futex-ABI
>+    futex2
>
> .. only::  subproject and html
>
>--
>2.31.1
>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 05/15] futex2: Implement support for different futex sizes
  2021-06-06 19:12   ` Davidlohr Bueso
@ 2021-06-06 23:01     ` Andrey Semashev
  0 siblings, 0 replies; 53+ messages in thread
From: Andrey Semashev @ 2021-06-06 23:01 UTC (permalink / raw)
  To: Davidlohr Bueso, Andr� Almeida
  Cc: Thomas Gleixner, Ingo Molnar, Peter Zijlstra, Darren Hart,
	linux-kernel, Steven Rostedt, Sebastian Andrzej Siewior, kernel,
	krisman, pgriffais, z.figura12, joel, malteskarupke, linux-api,
	fweimer, libc-alpha, linux-kselftest, shuah, acme, corbet,
	Peter Oskolkov, torvalds

On 6/6/21 10:12 PM, Davidlohr Bueso wrote:
> On Thu, 03 Jun 2021, Andr� Almeida wrote:
> 
>> Implement support for 8, 16 and 64 bit futexes, along with the existing
>> 32 bit support. Userspace should use flags to specify in the syscall
>> the size of the *uaddr they are operating on.
>>
>> Variable sized futexes are useful for implementing atomic primitives in
>> userspace in an efficient manner. 64bit sized futexes are also
>> particularly useful when userspace stores information to be used in an
>> atomic fashion on the futex value, given more room for flexibility.
> 
> Note that at least in the past, Linus has been vehemently against 64-bit
> futexes.
> 
> Basically this additional data, like for implementing read/write locks,
> does not need to be in the futex atomic wait/wake parts. You can instead
> split the userspace lock into two adjacent 32-bit words and do 64-bit
> atomic ops on it.

If the kernel performs a 32-bit atomic operation on the futex and the 
userspace operates on it using 64-bit atomics, I think you are losing 
the atomicity guarantee. For example, Intel SDM Volume 3 Section 8.1.2.2 
reads:

   Software should access semaphores (shared memory used for signalling
   between multiple processors) using identical addresses and operand
   lengths. For example, if one processor accesses a semaphore using a
   word access, other processors should not access the semaphore using a
   byte access.

I wouldn't be surprised if other architectures had a similar requirement.

> Of course, this is a new interface altogether, so this time it might
> be fair game.
> 
> Thanks,
> Davidlohr
> 
>>
>> Overlapping futexes are not allowed, so userspace can't wait and wake on
>> the same memory address if the are using different sizes.
>>
>> Signed-off-by: André Almeida <andrealmeid@collabora.com>
>> ---
>> include/uapi/linux/futex.h |   3 +
>> kernel/futex2.c            | 124 ++++++++++++++++++++++++-------------
>> 2 files changed, 84 insertions(+), 43 deletions(-)
>>
>> diff --git a/include/uapi/linux/futex.h b/include/uapi/linux/futex.h
>> index 06ea9bdfa69e..5786270b0c75 100644
>> --- a/include/uapi/linux/futex.h
>> +++ b/include/uapi/linux/futex.h
>> @@ -42,7 +42,10 @@
>>                      FUTEX_PRIVATE_FLAG)
>>
>> /* Size argument to futex2 syscall */
>> +#define FUTEX_8        0
>> +#define FUTEX_16    1
>> #define FUTEX_32    2
>> +#define FUTEX_64    3
>>
>> #define FUTEX_SIZE_MASK    0x3
>>
>> diff --git a/kernel/futex2.c b/kernel/futex2.c
>> index 012d7f7fc17a..1e97e5f2e793 100644
>> --- a/kernel/futex2.c
>> +++ b/kernel/futex2.c
>> @@ -89,9 +89,11 @@ struct futex_bucket {
>> #define FUTEXV_WAITER_MASK (FUTEX_SIZE_MASK | FUTEX_SHARED_FLAG)
>>
>> #define is_object_shared ((futexv->objects[i].flags & 
>> FUTEX_SHARED_FLAG) ? true : false)
>> +#define object_size (futexv->objects[i].flags & FUTEX_SIZE_MASK)
>>
>> -#define FUT_OFF_INODE    1 /* We set bit 0 if key has a reference on 
>> inode */
>> -#define FUT_OFF_MMSHARED 2 /* We set bit 1 if key has a reference on 
>> mm */
>> +#define FUT_OFF_INODE    PAGE_SIZE
>> +#define FUT_OFF_MMSHARED (PAGE_SIZE << 1)
>> +#define FUT_OFF_SIZE     1
>>
>> static struct futex_bucket *futex_table;
>> static unsigned int futex2_hashsize;
>> @@ -321,6 +323,7 @@ static int futex_get_shared_key(uintptr_t address, 
>> struct mm_struct *mm,
>>  * @uaddr:   futex user address
>>  * @key:     data that uniquely identifies a futex
>>  * @shared:  is this a shared futex?
>> + * @flags:   flags for the size
>>  *
>>  * For private futexes, each uaddr will be unique for a given 
>> mm_struct, and it
>>  * won't be freed for the life time of the process. For shared 
>> futexes, check
>> @@ -330,21 +333,41 @@ static int futex_get_shared_key(uintptr_t 
>> address, struct mm_struct *mm,
>>  */
>> static struct futex_bucket *futex_get_bucket(void __user *uaddr,
>>                          struct futex_key *key,
>> -                         bool shared)
>> +                         bool shared, unsigned int flags)
>> {
>>     uintptr_t address = (uintptr_t)uaddr;
>>     u32 hash_key;
>>
>> +    size_t size;
>> +
>> +    switch (flags) {
>> +    case FUTEX_8:
>> +        size = sizeof(u8);
>> +        break;
>> +    case FUTEX_16:
>> +        size = sizeof(u16);
>> +        break;
>> +    case FUTEX_32:
>> +        size = sizeof(u32);
>> +        break;
>> +    case FUTEX_64:
>> +        size = sizeof(u64);
>> +        break;
>> +    default:
>> +        return ERR_PTR(-EINVAL);
>> +    }
>> +
>>     /* Checking if uaddr is valid and accessible */
>> -    if (unlikely(!IS_ALIGNED(address, sizeof(u32))))
>> +    if (unlikely(!IS_ALIGNED(address, size)))
>>         return ERR_PTR(-EINVAL);
>> -    if (unlikely(!access_ok(uaddr, sizeof(u32))))
>> +    if (unlikely(!access_ok(uaddr, size)))
>>         return ERR_PTR(-EFAULT);
>>
>>     key->offset = address % PAGE_SIZE;
>>     address -= key->offset;
>>     key->pointer = (u64)address;
>>     key->index = (unsigned long)current->mm;
>> +    key->offset |= FUT_OFF_SIZE << (size - sizeof(u8));
>>
>>     if (shared)
>>         futex_get_shared_key(address, current->mm, key);
>> @@ -358,18 +381,39 @@ static struct futex_bucket 
>> *futex_get_bucket(void __user *uaddr,
>>
>> /**
>>  * futex_get_user - Get the userspace value on this address
>> - * @uval:  variable to store the value
>> - * @uaddr: userspace address
>> + * @uval:    variable to store the value
>> + * @uaddr:    userspace address
>> + * @pagefault:    true if pagefault should be disabled
>> + * @flags:    flags for the size
>>  *
>>  * Check the comment at futex_enqueue() for more information.
>>  */
>> -static int futex_get_user(u32 *uval, u32 __user *uaddr)
>> +static int futex_get_user(u64 *uval, void __user *uaddr, unsigned int 
>> flags, bool pagefault)
>> {
>>     int ret;
>>
>> -    pagefault_disable();
>> -    ret = __get_user(*uval, uaddr);
>> -    pagefault_enable();
>> +    if (pagefault)
>> +        pagefault_disable();
>> +
>> +    switch (flags) {
>> +    case FUTEX_8:
>> +        ret = __get_user(*uval, (u8 __user *)uaddr);
>> +        break;
>> +    case FUTEX_16:
>> +        ret = __get_user(*uval, (u16 __user *)uaddr);
>> +        break;
>> +    case FUTEX_32:
>> +        ret = __get_user(*uval, (u32 __user *)uaddr);
>> +        break;
>> +    case FUTEX_64:
>> +        ret = __get_user(*uval, (u64 __user *)uaddr);
>> +        break;
>> +    default:
>> +        BUG();
>> +    }
>> +
>> +    if (pagefault)
>> +        pagefault_enable();
>>
>>     return ret;
>> }
>> @@ -484,8 +528,8 @@ static int futex_enqueue(struct futex_waiter_head 
>> *futexv, unsigned int nr_futex
>>              int *awakened)
>> {
>>     int i, ret;
>> -    u32 uval, val;
>> -    u32 __user *uaddr;
>> +    u64 uval, val;
>> +    void __user *uaddr;
>>     bool retry = false;
>>     struct futex_bucket *bucket;
>>
>> @@ -493,13 +537,14 @@ static int futex_enqueue(struct 
>> futex_waiter_head *futexv, unsigned int nr_futex
>>     set_current_state(TASK_INTERRUPTIBLE);
>>
>>     for (i = 0; i < nr_futexes; i++) {
>> -        uaddr = (u32 __user *)futexv->objects[i].uaddr;
>> -        val = (u32)futexv->objects[i].val;
>> +        uaddr = futexv->objects[i].uaddr;
>> +        val = (u64)futexv->objects[i].val;
>>
>>         if (is_object_shared && retry) {
>>             struct futex_bucket *tmp =
>>                 futex_get_bucket((void __user *)uaddr,
>> -                         &futexv->objects[i].key, true);
>> +                         &futexv->objects[i].key, true,
>> +                         object_size);
>>             if (IS_ERR(tmp)) {
>>                 __set_current_state(TASK_RUNNING);
>>                 futex_dequeue_multiple(futexv, i);
>> @@ -513,7 +558,7 @@ static int futex_enqueue(struct futex_waiter_head 
>> *futexv, unsigned int nr_futex
>>         bucket_inc_waiters(bucket);
>>         spin_lock(&bucket->lock);
>>
>> -        ret = futex_get_user(&uval, uaddr);
>> +        ret = futex_get_user(&uval, uaddr, object_size, true);
>>
>>         if (unlikely(ret)) {
>>             spin_unlock(&bucket->lock);
>> @@ -525,7 +570,7 @@ static int futex_enqueue(struct futex_waiter_head 
>> *futexv, unsigned int nr_futex
>>             if (*awakened >= 0)
>>                 return 1;
>>
>> -            if (__get_user(uval, uaddr))
>> +            if (futex_get_user(&uval, uaddr, object_size, false))
>>                 return -EFAULT;
>>
>>             retry = true;
>> @@ -656,9 +701,6 @@ static long ksys_futex_wait(void __user *uaddr, 
>> u64 val, unsigned int flags,
>>     if (flags & ~FUTEX2_MASK)
>>         return -EINVAL;
>>
>> -    if (size != FUTEX_32)
>> -        return -EINVAL;
>> -
>>     futexv = &wait_single.futexv;
>>     futexv->task = current;
>>     futexv->hint = false;
>> @@ -667,12 +709,13 @@ static long ksys_futex_wait(void __user *uaddr, 
>> u64 val, unsigned int flags,
>>     waiter->index = 0;
>>     waiter->val = val;
>>     waiter->uaddr = uaddr;
>> +    waiter->flags = flags;
>>     memset(&wait_single.waiter.key, 0, sizeof(struct futex_key));
>>
>>     INIT_LIST_HEAD(&waiter->list);
>>
>>     /* Get an unlocked hash bucket */
>> -    waiter->bucket = futex_get_bucket(uaddr, &waiter->key, shared);
>> +    waiter->bucket = futex_get_bucket(uaddr, &waiter->key, shared, 
>> size);
>>     if (IS_ERR(waiter->bucket))
>>         return PTR_ERR(waiter->bucket);
>>
>> @@ -728,8 +771,7 @@ static int compat_futex_parse_waitv(struct 
>> futex_waiter_head *futexv,
>>         if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
>>             return -EFAULT;
>>
>> -        if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
>> -            (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
>> +        if (waitv.flags & ~FUTEXV_WAITER_MASK)
>>             return -EINVAL;
>>
>>         futexv->objects[i].key.pointer = 0;
>> @@ -740,7 +782,7 @@ static int compat_futex_parse_waitv(struct 
>> futex_waiter_head *futexv,
>>
>>         bucket = futex_get_bucket(compat_ptr(waitv.uaddr),
>>                       &futexv->objects[i].key,
>> -                      is_object_shared);
>> +                      is_object_shared, object_size);
>>
>>         if (IS_ERR(bucket))
>>             return PTR_ERR(bucket);
>> @@ -805,8 +847,7 @@ static int futex_parse_waitv(struct 
>> futex_waiter_head *futexv,
>>         if (copy_from_user(&waitv, &uwaitv[i], sizeof(waitv)))
>>             return -EFAULT;
>>
>> -        if ((waitv.flags & ~FUTEXV_WAITER_MASK) ||
>> -            (waitv.flags & FUTEX_SIZE_MASK) != FUTEX_32)
>> +        if (waitv.flags & ~FUTEXV_WAITER_MASK)
>>             return -EINVAL;
>>
>>         futexv->objects[i].key.pointer = 0;
>> @@ -816,7 +857,7 @@ static int futex_parse_waitv(struct 
>> futex_waiter_head *futexv,
>>         futexv->objects[i].index  = i;
>>
>>         bucket = futex_get_bucket(waitv.uaddr, &futexv->objects[i].key,
>> -                      is_object_shared);
>> +                      is_object_shared, object_size);
>>
>>         if (IS_ERR(bucket))
>>             return PTR_ERR(bucket);
>> @@ -947,10 +988,7 @@ SYSCALL_DEFINE3(futex_wake, void __user *, uaddr, 
>> unsigned int, nr_wake,
>>     if (flags & ~FUTEX2_MASK)
>>         return -EINVAL;
>>
>> -    if (size != FUTEX_32)
>> -        return -EINVAL;
>> -
>> -    bucket = futex_get_bucket(uaddr, &waiter.key, shared);
>> +    bucket = futex_get_bucket(uaddr, &waiter.key, shared, size);
>>     if (IS_ERR(bucket))
>>         return PTR_ERR(bucket);
>>
>> @@ -987,28 +1025,30 @@ static inline int __futex_requeue(struct 
>> futex_requeue rq1,
>>     bool retry = false;
>>     struct futex_bucket *b1, *b2;
>>     DEFINE_WAKE_Q(wake_q);
>> -    u32 uval;
>> +    u64 uval;
>>     int ret;
>>     bool shared1 = (rq1.flags  & FUTEX_SHARED_FLAG) ? true : false;
>>     bool shared2 = (rq2.flags  & FUTEX_SHARED_FLAG) ? true : false;
>> +    unsigned int size1 = (rq1.flags  & FUTEX_SIZE_MASK);
>> +    unsigned int size2 = (rq2.flags  & FUTEX_SIZE_MASK);
>>
>> -    b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
>> +    b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1, size1);
>>     if (IS_ERR(b1))
>>         return PTR_ERR(b1);
>>
>> -    b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
>> +    b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2, size2);
>>     if (IS_ERR(b2))
>>         return PTR_ERR(b2);
>>
>> retry:
>>     if (shared1 && retry) {
>> -        b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1);
>> +        b1 = futex_get_bucket(rq1.uaddr, &w1.key, shared1, size1);
>>         if (IS_ERR(b1))
>>             return PTR_ERR(b1);
>>     }
>>
>>     if (shared2 && retry) {
>> -        b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2);
>> +        b2 = futex_get_bucket(rq2.uaddr, &w2.key, shared2, size2);
>>         if (IS_ERR(b2))
>>             return PTR_ERR(b2);
>>     }
>> @@ -1027,11 +1067,11 @@ static inline int __futex_requeue(struct 
>> futex_requeue rq1,
>>         spin_lock_nested(&b1->lock, SINGLE_DEPTH_NESTING);
>>     }
>>
>> -    ret = futex_get_user(&uval, rq1.uaddr);
>> +    ret = futex_get_user(&uval, rq1.uaddr, size1, true);
>>
>>     if (unlikely(ret)) {
>>         futex_double_unlock(b1, b2);
>> -        if (__get_user(uval, (u32 __user *)rq1.uaddr))
>> +        if (futex_get_user(&uval, rq1.uaddr, size1, false))
>>             return -EFAULT;
>>
>>         bucket_dec_waiters(b2);
>> @@ -1088,8 +1128,7 @@ static int compat_futex_parse_requeue(struct 
>> futex_requeue *rq,
>>     if (copy_from_user(&tmp, uaddr, sizeof(tmp)))
>>         return -EFAULT;
>>
>> -    if (tmp.flags & ~FUTEXV_WAITER_MASK ||
>> -        (tmp.flags & FUTEX_SIZE_MASK) != FUTEX_32)
>> +    if (tmp.flags & ~FUTEXV_WAITER_MASK)
>>         return -EINVAL;
>>
>>     rq->uaddr = compat_ptr(tmp.uaddr);
>> @@ -1134,8 +1173,7 @@ static int futex_parse_requeue(struct 
>> futex_requeue *rq,
>>     if (copy_from_user(rq, uaddr, sizeof(*rq)))
>>         return -EFAULT;
>>
>> -    if (rq->flags & ~FUTEXV_WAITER_MASK ||
>> -        (rq->flags & FUTEX_SIZE_MASK) != FUTEX_32)
>> +    if (rq->flags & ~FUTEXV_WAITER_MASK)
>>         return -EINVAL;
>>
>>     return 0;
>> -- 
>> 2.31.1
>>


^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-05  1:09     ` Nicholas Piggin
  2021-06-05  8:56       ` Andrey Semashev
@ 2021-06-07 15:40       ` André Almeida
  2021-06-08  1:31         ` Nicholas Piggin
                           ` (2 more replies)
  1 sibling, 3 replies; 53+ messages in thread
From: André Almeida @ 2021-06-07 15:40 UTC (permalink / raw)
  To: Nicholas Piggin
  Cc: acme, Andrey Semashev, Sebastian Andrzej Siewior, corbet,
	Davidlohr Bueso, Darren Hart, fweimer, joel, kernel, krisman,
	libc-alpha, linux-api, linux-kernel, linux-kselftest,
	malteskarupke, Ingo Molnar, Peter Zijlstra, pgriffais,
	Peter Oskolkov, Steven Rostedt, shuah, Thomas Gleixner,
	z.figura12

Às 22:09 de 04/06/21, Nicholas Piggin escreveu:
> Excerpts from André Almeida's message of June 5, 2021 6:01 am:
>> Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
>>> Excerpts from André Almeida's message of June 4, 2021 5:59 am:
>>> - Did you consider a wakev interface? An example is a read-write mutex 
>>> which has read-blocking futexes split (e.g., per-node) for scalability 
>>> then the writer may unlock and wake all readers with one call. We 
>>> actually have some scalability challenges of this nature with certain 
>>> large database programs.
>>>
>>
>> Not really, I haven't heard any use case for that until now. It should
>> be easy to implement it, though, and I think you have an interesting use
>> case here. Could you point me some of those database programs?
> 
> Not source code unfortunately. I know that's not a very good answer, but 
> they are far ahead of what most open source apps are doing scalability 
> wise today, and they end up rolling their own complex locking. Hopefully
> the example I give is simple enough to understand.
> 

I see, that's makes things a bit harder. I understood the use case and
the wakev can be implemented without affecting the rest of API, so I
think I'll get back to it later, for now.

>>>
>>> - Are we really keen on squashing node ID into flags in this day and age?
>>> I guess okay but seems like it would be nice to allow a bit more space
>>> in general for the operations. I don't want to turn it into a whole big
>>> multiplexing nightmare again with lots of such flags, or propose
>>> complexity with no code behind it, but I think we need a bit of leeway
>>> for unforeseen locking innovations to be added carefully. The pthread
>>> locking today is still fairly primitive really, I don't think we know
>>> what will work best for the next 10 years.
>>
>> In the interface that I'd proposed, the node ID isn't part of the flags.
>> You have a flag FUTEX_FLAG_NUMA, and when that is used, you pass in
>> `void *uaddr` a pointer to a `struct futex_numa { int value, int hint
>> }`, where hint should be the node ID you would like to work on, and
>> value is just the userspace futex. This is documented in more details in
>> patch 7 "docs: locking: futex2: Add documentation".
>>
>> If you have any feedback about how this NUMA interface looks like, I
>> would like to hear.
>>
>> Also, did something in my writing indicated that the node ID would be
>> part of the flags? I'll improve this it if so.
> 
> Oh I did miss this, thank you. No it wasn't your writing, I think it was 
> me trying to read through a lot of messages and got confused with some
> earlier conversations.
> 
> I'll look a bit more at the NUMA interface.
> 

Thanks!

>>
>>>
>>> One scalability issue we are starting to hit and will only get worse is 
>>> futex queue spinlock contention. Perhaps this is better addressed in 
>>> userspace but the kernel could play a part so I like to leave some doors
>>> open. One example is that the wait (or wake) side may like to depend not
>>> just on the memory value, but on the success of a cmpxchg to avoid 
>>> unqueueing and queueing spuriously, which increases lock contention but
>>> also ends up putting the poor task on the back of the list -- yes RT
>>> priorities can help the realtime case, but non-RT cases can get bad
>>> outlier latencies if lock stealing is allowed (which can be very good
>>> for performance).
>>>
>>
>> Sorry, I'm not sure what do you mean here. Are you proposing to have a
>> cmpxchg in kernel side, so the lock would be taken by the kernel, and
>> not by the userspace like it's now?
> 
> Yes. Only in slow paths, of course, to reduce starvation / erratic
> latencies and spurious wait queue manipulations.

Right, so if we need to go into the kernel to do the cmpxchg, we can't
take a free lock without a syscall, and this goes against the futex
semantics, the "strength" of this interface is to not require context
switch in uncontended cases.

Is not a bad thing itself to go into the kernel to get a lock, other
operating systems do that and if the kernel has more knowledge about who
has the lock, it can even make some smart decisions. But this is not
futex, this probably belongs to another interface (that's probably
slower in the common case than futex).

> 
> Actually one other scalability thing while I remember it:
> 
> futex_wait currently requires that the lock word is tested under the 
> queue spin lock (to avoid consuming a wakeup). The problem with this is 
> that the lock word can be a very hot cache line if you have a lot of
> concurrency, so accessing it under the queue lock can increase queue
> lock hold time.
> 
> I would prefer if the new API was relaxed to avoid this restriction
> (e.g., any wait call may consume a wakeup so it's up to userspace to
> avoid that if it is a problem).

Maybe I'm wrong, but AFAIK the goal of checking the lock word inside the
spin lock is to avoid sleeping forever (in other words, wrongly assuming
that the lock is taken and missing a wakeup call), not to avoid
consuming wakeups. Or at least this is my interpretation of this long
comment in futex.c:

https://elixir.bootlin.com/linux/v5.12.9/source/kernel/futex.c#L51

So removing this requirement of checking the futex word with the lock
taken could led to undesirable behavior.

> 
>>> - The private global futex hash table sucks for various reasons, and
>>> having 128 waiters per thread makes it orders of magnitude easier for
>>> userspace to DoS stuff with hash collisions. NUMA doesn't fix that, the
>>> per process hashing that Thomas suggested does fix the DoS but the
>>> non-deterministic hash collisions still seem to be a problem for real
>>> time response, and at the other end of the scale some apps (certain 
>>> databases, etc) can have ten thousand futex waiters at once so birthday
>>> paradox can also lead to guaranteed (low level) variable beahviour 
>>> within a single process.
>>>
>>> I know the kernel in general is not very robust against this kind of 
>>> DoS/nondeterminism, but it's a bit sad to introduce new APIs with the 
>>> problem still there. Yes we could address it later, but I think it's 
>>> better done first because the solution might influence what the best 
>>> syscall API is.
>>>
>>> For example the idea of using the address as the handle for the wait 
>>> queue _and_ the value is very convenient but hashing is annoying for
>>> all the above reasons and the shared wait queue case is pretty clunky. 
>>> It's also constraining in some corner cases to have the wait queue 
>>> associated with the address 1:1. For example a type of NUMA mutex might 
>>> want to have per-node waitqueues associated with a lock word, and wake
>>> local node waiters 99% of the time. Not trivial to do with futexes and
>>> seems to at least require bouncing of more cache lines, possibly more
>>> atomics, etc.
>>>
>>> Could anything else be done?
>>
>> I wasn't aware that userspace doing DoS is something to be concerned
>> from the kernel point of view. Is this scenario considering a malicious
>> actor? If so, there are plenty of resources to be denied, so not sure
>> how futex could be protected of this. Or is this just a program that
>> uses tons of futexes?
> 
> Both really. AFAIKS one of the efforts that prompted the futex 
> modernisation work was the RT latency issues from Thomas in 2016 when 
> the per process table was proposed.
> 

When I first read Thomas proposal for per table process, I thought that
the main goal there was to solve NUMA locality issues, not RT latency,
but I think you are right. However, re-reading the thread at [0], it
seems that the RT problems where not completely solved in that
interface, maybe the people involved with that patchset can help to shed
some light on it.

Otherwise, this same proposal could be integrated in futex2, given that
we would only need to provide to userland some extra flags and add some
`if`s around the hash table code (in a very similar way the NUMA code
will be implemented in futex2).

[0] https://lore.kernel.org/lkml/20160505204230.932454245@linutronix.de/

> 
>>> I'll be burned at the stake for suggesting it but it would be great if 
>>> we could use file descriptors. At least for the shared futex, maybe 
>>> private could use a per-process futex allocator. It solves all of the
>>> above, although I'm sure has many of its own problem. It may not play
>>> so nicely with the pthread mutex API because of the whole static
>>> initialiser problem, but the first futex proposal did use fds. But it's
>>> an example of an alternate API.
>>>
>>
>> FDs and futex doesn't play well, because for futex_wait() you need to
>> tell the kernel the expected value in the futex address to avoid
>> sleeping in a free lock. FD operations (poll, select) don't have this
>> `value` argument, so they could sleep forever, but I'm not sure if you
>> had taken this in consideration.
> 
> I had. The futex wait API would take a fd additional. The only 
> difference is the waitqueue that is used when a sleep or wake is 
> required is derived from the fd, not from an address.
> 
> I think the bigger sticking points would be if it's too heavyweight an 
> object to use (which could be somewhat mitigated with a simpler ida
> allocator although that's difficult to do with shared), and whether libc
> could sanely use them due to the static initialiser problem of pthread
> mutexes.
> 
> Thanks,
> Nick
> 

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-06 13:15           ` Andrey Semashev
@ 2021-06-08  1:25             ` Nicholas Piggin
  2021-06-08 11:03               ` Andrey Semashev
  0 siblings, 1 reply; 53+ messages in thread
From: Nicholas Piggin @ 2021-06-08  1:25 UTC (permalink / raw)
  To: André Almeida, Andrey Semashev
  Cc: acme, Sebastian Andrzej Siewior, corbet, Davidlohr Bueso,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, Peter Zijlstra, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

Excerpts from Andrey Semashev's message of June 6, 2021 11:15 pm:
> On 6/6/21 2:57 PM, Nicholas Piggin wrote:
>> Excerpts from Andrey Semashev's message of June 5, 2021 6:56 pm:
>>> On 6/5/21 4:09 AM, Nicholas Piggin wrote:
>>>> Excerpts from André Almeida's message of June 5, 2021 6:01 am:
>>>>> Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
>>>>
>>>>>> I'll be burned at the stake for suggesting it but it would be great if
>>>>>> we could use file descriptors. At least for the shared futex, maybe
>>>>>> private could use a per-process futex allocator. It solves all of the
>>>>>> above, although I'm sure has many of its own problem. It may not play
>>>>>> so nicely with the pthread mutex API because of the whole static
>>>>>> initialiser problem, but the first futex proposal did use fds. But it's
>>>>>> an example of an alternate API.
>>>>>>
>>>>>
>>>>> FDs and futex doesn't play well, because for futex_wait() you need to
>>>>> tell the kernel the expected value in the futex address to avoid
>>>>> sleeping in a free lock. FD operations (poll, select) don't have this
>>>>> `value` argument, so they could sleep forever, but I'm not sure if you
>>>>> had taken this in consideration.
>>>>
>>>> I had. The futex wait API would take a fd additional. The only
>>>> difference is the waitqueue that is used when a sleep or wake is
>>>> required is derived from the fd, not from an address.
>>>>
>>>> I think the bigger sticking points would be if it's too heavyweight an
>>>> object to use (which could be somewhat mitigated with a simpler ida
>>>> allocator although that's difficult to do with shared), and whether libc
>>>> could sanely use them due to the static initialiser problem of pthread
>>>> mutexes.
>>>
>>> The static initialization feature is not the only benefit of the current
>>> futex design, and probably not the most important one. You can work
>>> around the static initialization in userspace, e.g. by initializing fd
>>> to an invalid value and creating a valid fd upon the first use. Although
>>> that would still incur a performance penalty and add a new source of
>>> failure.
>> 
>> Sounds like a serious problem, but maybe it isn't. On the other hand,
>> maybe we don't have to support pthread mutexes as they are anyway
>> because futex already does that fairly well.
>> 
>>> What is more important is that waiting on fd always requires a kernel
>>> call. This will be terrible for performance of uncontended locks, which
>>> is the majority of time.
>> 
>> No. As I said just before, it would be the same except the waitqueue is
>> derived from fd rather than address.
> 
> Sorry, in that case I'm not sure I understand how that would work. You 
> do need to allocate a fd, do you?

Yes. As I said, imagine a futex_wait API that also takes a fd. The
wait queue is derived from that fd rather than the hash table.

>>> Another important point is that a futex that is not being waited on
>>> consumes zero kernel resources while fd is a limited resource even when
>>> not used. You can have millions futexes in userspace and you are
>>> guaranteed not to exhaust any limit as long as you have memory. That is
>>> an important feature, and the current userspace is relying on it by
>>> assuming that creating mutexes and condition variables is cheap.
>> 
>> Is it an important feture? Would 1 byte of kernel memory per uncontended
>> futex be okay? 10? 100?
>> 
>> I do see it's very nice the current design that requires no
>> initialization for uncontended, I'm just asking questions to get an idea
>> of what constraints we're working with. We have a pretty good API
>> already which can support unlimited uncontended futexes, so I'm
>> wondering do we really need another very very similar API that doesn't
>> fix the really difficult problems of the existing one?
> 
> It does provide the very much needed features that are missing in the 
> current futex. Namely, more futex sizes and wait for multiple. So the 
> argument of "why have two similar APIs" is not quite fair. It would be, 
> if there was feature parity with futex.

It does provide some extra features sure, with some straightforward 
extension of the existing API. The really interesting or tricky part of
the API is left unchanged though.

My line of thinking is that while we're changing the API anyway, we 
should see if it can be changed to help those other problems too.

> I believe, the low cost of a futex is an important feature, and was one 
> of the reasons for its original design and introduction.

It is of course. The first futex proposal did use fds, interestingly.
I didn't look back further into the libc side of that thing, but maybe
I should.

> Otherwise we 
> would be using eventfds in mutexes.

I don't think so, not even if eventfd came before the futex syscall.

> 
> One other feature that I didn't mention earlier and which follows from 
> its "address in memory" design is the ability to use futexes in 
> process-shared memory. This is important for process-shared pthread 
> components, too, but has its own value even without this, if you use 
> futexes directly. With fds, you can't place the fd in a shared memory 
> since every process needs to have its own fd referring to the same 
> kernel object, and passing fds cannot be done without a UNIX socket. 
> This is incompatible with pthreads API design and would require 
> non-trivial design changes to the applications using futexes directly.
> 

That may be true. file is a natural object to share such a resource, but 
the means to share the fd is not so easy. OTOH you could also use a 
syscall to open the same file and get a new fd.

Are shared pthread mutexes using existing pthread APIs that are today
implemented okay with futex1 system call a good reason to constrain 
futex2 I wonder? Or do we have an opportunity to make a bigger change
to the API so it suffers less from non deterministic latency (for
example)?

I don't want to limit it to just files vs addresses, fds was an example 
of something that could solve some of the problems.

Thanks,
Nick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-07 15:40       ` André Almeida
@ 2021-06-08  1:31         ` Nicholas Piggin
  2021-06-08  2:33         ` Davidlohr Bueso
  2021-06-08 12:26         ` Sebastian Andrzej Siewior
  2 siblings, 0 replies; 53+ messages in thread
From: Nicholas Piggin @ 2021-06-08  1:31 UTC (permalink / raw)
  To: André Almeida
  Cc: acme, Andrey Semashev, Sebastian Andrzej Siewior, corbet,
	Davidlohr Bueso, Darren Hart, fweimer, joel, kernel, krisman,
	libc-alpha, linux-api, linux-kernel, linux-kselftest,
	malteskarupke, Ingo Molnar, Peter Zijlstra, pgriffais,
	Peter Oskolkov, Steven Rostedt, shuah, Thomas Gleixner,
	z.figura12

Excerpts from André Almeida's message of June 8, 2021 1:40 am:
> Às 22:09 de 04/06/21, Nicholas Piggin escreveu:
>> Excerpts from André Almeida's message of June 5, 2021 6:01 am:
>>> Às 08:36 de 04/06/21, Nicholas Piggin escreveu:
>>>> Excerpts from André Almeida's message of June 4, 2021 5:59 am:
>>>> - Did you consider a wakev interface? An example is a read-write mutex 
>>>> which has read-blocking futexes split (e.g., per-node) for scalability 
>>>> then the writer may unlock and wake all readers with one call. We 
>>>> actually have some scalability challenges of this nature with certain 
>>>> large database programs.
>>>>
>>>
>>> Not really, I haven't heard any use case for that until now. It should
>>> be easy to implement it, though, and I think you have an interesting use
>>> case here. Could you point me some of those database programs?
>> 
>> Not source code unfortunately. I know that's not a very good answer, but 
>> they are far ahead of what most open source apps are doing scalability 
>> wise today, and they end up rolling their own complex locking. Hopefully
>> the example I give is simple enough to understand.
>> 
> 
> I see, that's makes things a bit harder. I understood the use case and
> the wakev can be implemented without affecting the rest of API, so I
> think I'll get back to it later, for now.

Yeah that's fine.

>>>> - Are we really keen on squashing node ID into flags in this day and age?
>>>> I guess okay but seems like it would be nice to allow a bit more space
>>>> in general for the operations. I don't want to turn it into a whole big
>>>> multiplexing nightmare again with lots of such flags, or propose
>>>> complexity with no code behind it, but I think we need a bit of leeway
>>>> for unforeseen locking innovations to be added carefully. The pthread
>>>> locking today is still fairly primitive really, I don't think we know
>>>> what will work best for the next 10 years.
>>>
>>> In the interface that I'd proposed, the node ID isn't part of the flags.
>>> You have a flag FUTEX_FLAG_NUMA, and when that is used, you pass in
>>> `void *uaddr` a pointer to a `struct futex_numa { int value, int hint
>>> }`, where hint should be the node ID you would like to work on, and
>>> value is just the userspace futex. This is documented in more details in
>>> patch 7 "docs: locking: futex2: Add documentation".
>>>
>>> If you have any feedback about how this NUMA interface looks like, I
>>> would like to hear.
>>>
>>> Also, did something in my writing indicated that the node ID would be
>>> part of the flags? I'll improve this it if so.
>> 
>> Oh I did miss this, thank you. No it wasn't your writing, I think it was 
>> me trying to read through a lot of messages and got confused with some
>> earlier conversations.
>> 
>> I'll look a bit more at the NUMA interface.
>> 
> 
> Thanks!
> 
>>>
>>>>
>>>> One scalability issue we are starting to hit and will only get worse is 
>>>> futex queue spinlock contention. Perhaps this is better addressed in 
>>>> userspace but the kernel could play a part so I like to leave some doors
>>>> open. One example is that the wait (or wake) side may like to depend not
>>>> just on the memory value, but on the success of a cmpxchg to avoid 
>>>> unqueueing and queueing spuriously, which increases lock contention but
>>>> also ends up putting the poor task on the back of the list -- yes RT
>>>> priorities can help the realtime case, but non-RT cases can get bad
>>>> outlier latencies if lock stealing is allowed (which can be very good
>>>> for performance).
>>>>
>>>
>>> Sorry, I'm not sure what do you mean here. Are you proposing to have a
>>> cmpxchg in kernel side, so the lock would be taken by the kernel, and
>>> not by the userspace like it's now?
>> 
>> Yes. Only in slow paths, of course, to reduce starvation / erratic
>> latencies and spurious wait queue manipulations.
> 
> Right, so if we need to go into the kernel to do the cmpxchg, we can't
> take a free lock without a syscall,

Yes you can.

> and this goes against the futex
> semantics, the "strength" of this interface is to not require context
> switch in uncontended cases.
> 
> Is not a bad thing itself to go into the kernel to get a lock, other
> operating systems do that and if the kernel has more knowledge about who
> has the lock, it can even make some smart decisions. But this is not
> futex, this probably belongs to another interface (that's probably
> slower in the common case than futex).
> 
>> 
>> Actually one other scalability thing while I remember it:
>> 
>> futex_wait currently requires that the lock word is tested under the 
>> queue spin lock (to avoid consuming a wakeup). The problem with this is 
>> that the lock word can be a very hot cache line if you have a lot of
>> concurrency, so accessing it under the queue lock can increase queue
>> lock hold time.
>> 
>> I would prefer if the new API was relaxed to avoid this restriction
>> (e.g., any wait call may consume a wakeup so it's up to userspace to
>> avoid that if it is a problem).
> 
> Maybe I'm wrong, but AFAIK the goal of checking the lock word inside the
> spin lock is to avoid sleeping forever (in other words, wrongly assuming
> that the lock is taken and missing a wakeup call), not to avoid
> consuming wakeups. Or at least this is my interpretation of this long
> comment in futex.c:
> 
> https://elixir.bootlin.com/linux/v5.12.9/source/kernel/futex.c#L51
> 
> So removing this requirement of checking the futex word with the lock
> taken could led to undesirable behavior.

No, there are two requirements. Obviously you need to avoid the missed
wakeup at minimum. You don't need to check under the lock unless you
want to avoid consuming an extra wakeup though (it can possibly be done
in more complex ways like you detect if you took a wakeup and if so then
look at the queue and see if you can pass it on, or have some extra flag
to signal you are ready for wake up, but those all seem more complex and
fragile and possibly have weird corner cases, better to just set out 
that userspace should deal with it).

> 
>> 
>>>> - The private global futex hash table sucks for various reasons, and
>>>> having 128 waiters per thread makes it orders of magnitude easier for
>>>> userspace to DoS stuff with hash collisions. NUMA doesn't fix that, the
>>>> per process hashing that Thomas suggested does fix the DoS but the
>>>> non-deterministic hash collisions still seem to be a problem for real
>>>> time response, and at the other end of the scale some apps (certain 
>>>> databases, etc) can have ten thousand futex waiters at once so birthday
>>>> paradox can also lead to guaranteed (low level) variable beahviour 
>>>> within a single process.
>>>>
>>>> I know the kernel in general is not very robust against this kind of 
>>>> DoS/nondeterminism, but it's a bit sad to introduce new APIs with the 
>>>> problem still there. Yes we could address it later, but I think it's 
>>>> better done first because the solution might influence what the best 
>>>> syscall API is.
>>>>
>>>> For example the idea of using the address as the handle for the wait 
>>>> queue _and_ the value is very convenient but hashing is annoying for
>>>> all the above reasons and the shared wait queue case is pretty clunky. 
>>>> It's also constraining in some corner cases to have the wait queue 
>>>> associated with the address 1:1. For example a type of NUMA mutex might 
>>>> want to have per-node waitqueues associated with a lock word, and wake
>>>> local node waiters 99% of the time. Not trivial to do with futexes and
>>>> seems to at least require bouncing of more cache lines, possibly more
>>>> atomics, etc.
>>>>
>>>> Could anything else be done?
>>>
>>> I wasn't aware that userspace doing DoS is something to be concerned
>>> from the kernel point of view. Is this scenario considering a malicious
>>> actor? If so, there are plenty of resources to be denied, so not sure
>>> how futex could be protected of this. Or is this just a program that
>>> uses tons of futexes?
>> 
>> Both really. AFAIKS one of the efforts that prompted the futex 
>> modernisation work was the RT latency issues from Thomas in 2016 when 
>> the per process table was proposed.
>> 
> 
> When I first read Thomas proposal for per table process, I thought that
> the main goal there was to solve NUMA locality issues, not RT latency,
> but I think you are right. However, re-reading the thread at [0], it
> seems that the RT problems where not completely solved in that
> interface, maybe the people involved with that patchset can help to shed
> some light on it.
> 
> Otherwise, this same proposal could be integrated in futex2, given that
> we would only need to provide to userland some extra flags and add some
> `if`s around the hash table code (in a very similar way the NUMA code
> will be implemented in futex2).
> 
> [0] https://lore.kernel.org/lkml/20160505204230.932454245@linutronix.de/

Right. The accidental collisions within a process (or 
accidental/deliberate collisions with shared futex) problems remain.

Thanks,
Nick

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-07 15:40       ` André Almeida
  2021-06-08  1:31         ` Nicholas Piggin
@ 2021-06-08  2:33         ` Davidlohr Bueso
  2021-06-08  4:45           ` Nicholas Piggin
  2021-06-08 12:26         ` Sebastian Andrzej Siewior
  2 siblings, 1 reply; 53+ messages in thread
From: Davidlohr Bueso @ 2021-06-08  2:33 UTC (permalink / raw)
  To: Andr� Almeida
  Cc: Nicholas Piggin, acme, Andrey Semashev,
	Sebastian Andrzej Siewior, corbet, Darren Hart, fweimer, joel,
	kernel, krisman, libc-alpha, linux-api, linux-kernel,
	linux-kselftest, malteskarupke, Ingo Molnar, Peter Zijlstra,
	pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On Mon, 07 Jun 2021, Andr� Almeida wrote:

>Às 22:09 de 04/06/21, Nicholas Piggin escreveu:
>> Actually one other scalability thing while I remember it:
>>
>> futex_wait currently requires that the lock word is tested under the
>> queue spin lock (to avoid consuming a wakeup). The problem with this is
>> that the lock word can be a very hot cache line if you have a lot of
>> concurrency, so accessing it under the queue lock can increase queue
>> lock hold time.
>>
>> I would prefer if the new API was relaxed to avoid this restriction
>> (e.g., any wait call may consume a wakeup so it's up to userspace to
>> avoid that if it is a problem).
>
>Maybe I'm wrong, but AFAIK the goal of checking the lock word inside the
>spin lock is to avoid sleeping forever (in other words, wrongly assuming
>that the lock is taken and missing a wakeup call), not to avoid
>consuming wakeups. Or at least this is my interpretation of this long
>comment in futex.c:
>
>https://elixir.bootlin.com/linux/v5.12.9/source/kernel/futex.c#L51

I think what Nick is referring to is that futex_wait() could return 0
instead of EAGAIN upon a uval != val condition if the check is done
without the hb lock. The value could have changed between when userspace
did the condition check and called into futex(2) to block in the slowpath.

But such spurious scenarios should be pretty rare, and while I agree that
the cacheline can be hot, I'm not sure how much of a performance issue this
really is(?), compared to other issues, certainly not to govern futex2
design. Changing such semantics would be a _huge_ difference between futex1
and futex2.

At least compared, for example, to the hb collisions serializing independent
futexes, affecting both performance and determinism. And I agree that a new
interface should address this problem - albeit most of the workloads I have
seen in production use but a handful of futexes and larger thread counts.
One thing that crossed my mind (but have not actually sat down to look at)
would be to use rlhastables for the dynamic resizing, but of course that would
probably add a decent amount of overhead to the simple hashing we currently have.

Thanks,
Davidlohr

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08  2:33         ` Davidlohr Bueso
@ 2021-06-08  4:45           ` Nicholas Piggin
  0 siblings, 0 replies; 53+ messages in thread
From: Nicholas Piggin @ 2021-06-08  4:45 UTC (permalink / raw)
  To: Andr� Almeida, Davidlohr Bueso
  Cc: acme, Andrey Semashev, Sebastian Andrzej Siewior, corbet,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, Peter Zijlstra, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

Excerpts from Davidlohr Bueso's message of June 8, 2021 12:33 pm:
> On Mon, 07 Jun 2021, Andr� Almeida wrote:
> 
>>Às 22:09 de 04/06/21, Nicholas Piggin escreveu:
>>> Actually one other scalability thing while I remember it:
>>>
>>> futex_wait currently requires that the lock word is tested under the
>>> queue spin lock (to avoid consuming a wakeup). The problem with this is
>>> that the lock word can be a very hot cache line if you have a lot of
>>> concurrency, so accessing it under the queue lock can increase queue
>>> lock hold time.
>>>
>>> I would prefer if the new API was relaxed to avoid this restriction
>>> (e.g., any wait call may consume a wakeup so it's up to userspace to
>>> avoid that if it is a problem).
>>
>>Maybe I'm wrong, but AFAIK the goal of checking the lock word inside the
>>spin lock is to avoid sleeping forever (in other words, wrongly assuming
>>that the lock is taken and missing a wakeup call), not to avoid
>>consuming wakeups. Or at least this is my interpretation of this long
>>comment in futex.c:
>>
>>https://elixir.bootlin.com/linux/v5.12.9/source/kernel/futex.c#L51
> 
> I think what Nick is referring to is that futex_wait() could return 0
> instead of EAGAIN upon a uval != val condition if the check is done
> without the hb lock. The value could have changed between when userspace
> did the condition check and called into futex(2) to block in the slowpath.

I just mean the check could be done after queueing ourselves on the wait 
queue (and unlocking the waitqueue lock, not checking while holding the
lock). That is the standard pattern used everywhere else by the kernel:

  prepare_to_wait() /* -> lock; add_wait_queue; unlock; */
  check_again();
  schedule();

It can still return EAGAIN if there is a reasonable use for it, but I'd
be wary about user code that cares about this -- it's racy you could 
arrive right before the value changes or right after it changes, so any
user code checking this I would be suspicious of (I'm willing to see a
use case that really cares).

> 
> But such spurious scenarios should be pretty rare, and while I agree that
> the cacheline can be hot, I'm not sure how much of a performance issue this
> really is(?),

It's not a spurious anything. The problem is the contention on the
lock word cacheline means it can take a relatively long time just to perform
that one load instruction. Mandating that it must be done while holding the
lock translates to increased lock hold times.

This matters particularly in situations that have lock stealing, 
optimistic spinning, reader-writer locks or more exotic kind of things 
that allow some common types of critical section to go through while others
are blocking. And partiuclarly when such things hash collide on other
futexes that share the same hash lock.

> compared to other issues, certainly not to govern futex2
> design. Changing such semantics would be a _huge_ difference between futex1
> and futex2.

futex1 behaviour should not govern futex2 design. That's the only nice 
thing you get with an API change, so we should take full advantage of 
it. I'm not saying make changes for no reason, but I gave a reason, so 
that should be countered with a better reason to not change.

Thanks,
Nick

> 
> At least compared, for example, to the hb collisions serializing independent
> futexes, affecting both performance and determinism. And I agree that a new
> interface should address this problem - albeit most of the workloads I have
> seen in production use but a handful of futexes and larger thread counts.
> One thing that crossed my mind (but have not actually sat down to look at)
> would be to use rlhastables for the dynamic resizing, but of course that would
> probably add a decent amount of overhead to the simple hashing we currently have.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08  1:25             ` Nicholas Piggin
@ 2021-06-08 11:03               ` Andrey Semashev
  2021-06-08 11:13                 ` Greg KH
  0 siblings, 1 reply; 53+ messages in thread
From: Andrey Semashev @ 2021-06-08 11:03 UTC (permalink / raw)
  To: Nicholas Piggin, André Almeida
  Cc: acme, Sebastian Andrzej Siewior, corbet, Davidlohr Bueso,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, Peter Zijlstra, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

On 6/8/21 4:25 AM, Nicholas Piggin wrote:
> 
> Are shared pthread mutexes using existing pthread APIs that are today
> implemented okay with futex1 system call a good reason to constrain
> futex2 I wonder? Or do we have an opportunity to make a bigger change
> to the API so it suffers less from non deterministic latency (for
> example)?

If futex2 is not able to cover futex1 use cases then it cannot be viewed 
as a replacement. In the long term this means futex1 cannot be 
deprecated and has to be maintained. My impression was that futex1 was 
basically unmaintainable(*) and futex2 was an evolution of futex1 so 
that users of futex1 could migrate relatively easily and futex1 
eventually removed. Maybe my impression was wrong, but I would like to 
see futex2 as a replacement and extension of futex1, so the latter can 
be deprecated at some point.

In any case, creating a new API should consider requirements of its 
potential users. If futex2 is intended to eventually replace futex1 then 
all current futex1 users are potential users of futex2. If not, then the 
futex2 submission should list its intended users, at least in general 
terms, and their requirements that led to the proposed API design.

(*) I use "unmaintainable" in a broad sense here. It exists and works in 
newer kernel versions and may receive code changes that are necessary to 
keep it working, but maintainers refuse any extensions or modifications 
of the code, mostly because of its complexity.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 11:03               ` Andrey Semashev
@ 2021-06-08 11:13                 ` Greg KH
  2021-06-08 11:44                   ` Peter Zijlstra
                                     ` (2 more replies)
  0 siblings, 3 replies; 53+ messages in thread
From: Greg KH @ 2021-06-08 11:13 UTC (permalink / raw)
  To: Andrey Semashev
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On Tue, Jun 08, 2021 at 02:03:50PM +0300, Andrey Semashev wrote:
> On 6/8/21 4:25 AM, Nicholas Piggin wrote:
> > 
> > Are shared pthread mutexes using existing pthread APIs that are today
> > implemented okay with futex1 system call a good reason to constrain
> > futex2 I wonder? Or do we have an opportunity to make a bigger change
> > to the API so it suffers less from non deterministic latency (for
> > example)?
> 
> If futex2 is not able to cover futex1 use cases then it cannot be viewed as
> a replacement. In the long term this means futex1 cannot be deprecated and
> has to be maintained. My impression was that futex1 was basically
> unmaintainable(*) and futex2 was an evolution of futex1 so that users of
> futex1 could migrate relatively easily and futex1 eventually removed. Maybe
> my impression was wrong, but I would like to see futex2 as a replacement and
> extension of futex1, so the latter can be deprecated at some point.

You can never delete a kernel system call, so even if you "deprecate"
it, it still needs to be supported for forever.

Best of all would be if internally your "futex2" code would replace the
"futex1" code so that there is no two different code bases.  That would
be the only sane way forward, having 2 code bases to work with is just
insane.

> (*) I use "unmaintainable" in a broad sense here. It exists and works in
> newer kernel versions and may receive code changes that are necessary to
> keep it working, but maintainers refuse any extensions or modifications of
> the code, mostly because of its complexity.

Adding additional complexity for no good reason is not a good idea,
especially if you are asking others to maintain and support that
complexity.  Would you want to have to do that work?

So what's keeping the futex2 code from doing all that futex1 does so
that the futex1 code can be deleted internally?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 11:13                 ` Greg KH
@ 2021-06-08 11:44                   ` Peter Zijlstra
  2021-06-08 14:31                     ` Davidlohr Bueso
  2021-06-08 12:06                   ` Andrey Semashev
  2021-06-08 14:14                   ` André Almeida
  2 siblings, 1 reply; 53+ messages in thread
From: Peter Zijlstra @ 2021-06-08 11:44 UTC (permalink / raw)
  To: Greg KH
  Cc: Andrey Semashev, Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On Tue, Jun 08, 2021 at 01:13:30PM +0200, Greg KH wrote:
> On Tue, Jun 08, 2021 at 02:03:50PM +0300, Andrey Semashev wrote:
> So what's keeping the futex2 code from doing all that futex1 does so
> that the futex1 code can be deleted internally?

I'd much rather see it the other way around. Much of what futex2 does
can already be done with the futex1 code-base. And then we can add
features on top.

I've been moaning about this for the past many versions, even older
versions actually implemented some of the new features in futex1,
showing it can be done.

We just wanted a saner futex interface because the existing futex
syscall is a monster and making it even worse seemed like a bad idea.

So *again*: please add the new syscalls on top of futex1 and then
extend.  Do not re-implement.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 11:13                 ` Greg KH
  2021-06-08 11:44                   ` Peter Zijlstra
@ 2021-06-08 12:06                   ` Andrey Semashev
  2021-06-08 12:33                     ` Greg KH
  2021-06-08 12:35                     ` Greg KH
  2021-06-08 14:14                   ` André Almeida
  2 siblings, 2 replies; 53+ messages in thread
From: Andrey Semashev @ 2021-06-08 12:06 UTC (permalink / raw)
  To: Greg KH
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On 6/8/21 2:13 PM, Greg KH wrote:
> On Tue, Jun 08, 2021 at 02:03:50PM +0300, Andrey Semashev wrote:
>> On 6/8/21 4:25 AM, Nicholas Piggin wrote:
>>>
>>> Are shared pthread mutexes using existing pthread APIs that are today
>>> implemented okay with futex1 system call a good reason to constrain
>>> futex2 I wonder? Or do we have an opportunity to make a bigger change
>>> to the API so it suffers less from non deterministic latency (for
>>> example)?
>>
>> If futex2 is not able to cover futex1 use cases then it cannot be viewed as
>> a replacement. In the long term this means futex1 cannot be deprecated and
>> has to be maintained. My impression was that futex1 was basically
>> unmaintainable(*) and futex2 was an evolution of futex1 so that users of
>> futex1 could migrate relatively easily and futex1 eventually removed. Maybe
>> my impression was wrong, but I would like to see futex2 as a replacement and
>> extension of futex1, so the latter can be deprecated at some point.
> 
> You can never delete a kernel system call, so even if you "deprecate"
> it, it still needs to be supported for forever.

If I'm not mistaken, some syscalls were dropped from kernel in the past, 
after it was established they are no longer used. So it is not 
impossible, though might be more difficult specifically with futex.

> Best of all would be if internally your "futex2" code would replace the
> "futex1" code so that there is no two different code bases.  That would
> be the only sane way forward, having 2 code bases to work with is just
> insane.

Yes, implementing futex1 in terms of futex2 internally is a possible way 
forward. Though I'm not sure it is reasonable to require that to be done 
in the initial futex2 submission. This requires all of the futex1 
functionality to implemented in futex2 from the start, which I think is 
too much to ask. Even with some futex1 features missing, futex2 would be 
already very much useful to users, and it is easier to implement the 
missing bits incrementally over time.

Also, one other point I'd like to make is that not all futex1 features 
might need to be reimplemented if futex2 provides a better alternative. 
For example, as a user, I would like to see a different approach to 
robust futexes that does not mandate a single user (libc) and allows to 
use robust futexes directly.

>> (*) I use "unmaintainable" in a broad sense here. It exists and works in
>> newer kernel versions and may receive code changes that are necessary to
>> keep it working, but maintainers refuse any extensions or modifications of
>> the code, mostly because of its complexity.
> 
> Adding additional complexity for no good reason is not a good idea,
> especially if you are asking others to maintain and support that
> complexity.  Would you want to have to do that work?
> 
> So what's keeping the futex2 code from doing all that futex1 does so
> that the futex1 code can be deleted internally?

I think, André will answer this, but my guess is, as stated above, this 
is a lot of work and time while the intermediate version is already useful.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-07 15:40       ` André Almeida
  2021-06-08  1:31         ` Nicholas Piggin
  2021-06-08  2:33         ` Davidlohr Bueso
@ 2021-06-08 12:26         ` Sebastian Andrzej Siewior
  2021-06-08 14:23           ` Peter Zijlstra
  2 siblings, 1 reply; 53+ messages in thread
From: Sebastian Andrzej Siewior @ 2021-06-08 12:26 UTC (permalink / raw)
  To: André Almeida
  Cc: Nicholas Piggin, acme, Andrey Semashev, corbet, Davidlohr Bueso,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, Peter Zijlstra, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

On 2021-06-07 12:40:54 [-0300], André Almeida wrote:
> 
> When I first read Thomas proposal for per table process, I thought that
> the main goal there was to solve NUMA locality issues, not RT latency,
> but I think you are right. However, re-reading the thread at [0], it
> seems that the RT problems where not completely solved in that
> interface, maybe the people involved with that patchset can help to shed
> some light on it.
> 
> Otherwise, this same proposal could be integrated in futex2, given that
> we would only need to provide to userland some extra flags and add some
> `if`s around the hash table code (in a very similar way the NUMA code
> will be implemented in futex2).

There are slides at [0] describing some attempts and the kernel tree [1]
from that time.

The process-table solves the problem to some degree that two random
process don't collide on the same hash bucket. But as Peter Zijlstra
pointed out back then two threads from the same task could collide on
the same hash bucket (and with ASLR not always). So the collision is
there but limited and this was not perfect.

All the attempts with API extensions didn't go well because glibc did
not want to change a bit. This starts with a mutex that has a static
initializer which has to work (I don't remember why the first
pthread_mutex_lock() could not fail with -ENOMEM but there was
something) and ends with glibc's struct mutex which is full and has no
room for additional data storage.

The additional data in user's struct mutex + init would have the benefit
that instead uaddr (which is hashed for the in-kernel lookup) a cookie
could be used for the hash-less lookup (and NUMA pointer where memory
should be stored).

So. We couldn't change a thing back then so nothing did happen. We
didn't want to create a new interface and a library implementing it plus
all the functionality around it (like pthread_cond, phtread_barrier, …).
Not to mention that if glibc continues to use the "old" locking
internally then the application is still affected by the hash-collision
locking (or the NUMA problem) should it block on the lock.

[0] https://linutronix.de/PDF/2016_futex_Siewior.pdf
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bigeasy/linux-futex.git/

Sebastian

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 12:06                   ` Andrey Semashev
@ 2021-06-08 12:33                     ` Greg KH
  2021-06-08 12:35                     ` Greg KH
  1 sibling, 0 replies; 53+ messages in thread
From: Greg KH @ 2021-06-08 12:33 UTC (permalink / raw)
  To: Andrey Semashev
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On Tue, Jun 08, 2021 at 03:06:48PM +0300, Andrey Semashev wrote:
> On 6/8/21 2:13 PM, Greg KH wrote:
> > On Tue, Jun 08, 2021 at 02:03:50PM +0300, Andrey Semashev wrote:
> > > On 6/8/21 4:25 AM, Nicholas Piggin wrote:
> > > > 
> > > > Are shared pthread mutexes using existing pthread APIs that are today
> > > > implemented okay with futex1 system call a good reason to constrain
> > > > futex2 I wonder? Or do we have an opportunity to make a bigger change
> > > > to the API so it suffers less from non deterministic latency (for
> > > > example)?
> > > 
> > > If futex2 is not able to cover futex1 use cases then it cannot be viewed as
> > > a replacement. In the long term this means futex1 cannot be deprecated and
> > > has to be maintained. My impression was that futex1 was basically
> > > unmaintainable(*) and futex2 was an evolution of futex1 so that users of
> > > futex1 could migrate relatively easily and futex1 eventually removed. Maybe
> > > my impression was wrong, but I would like to see futex2 as a replacement and
> > > extension of futex1, so the latter can be deprecated at some point.
> > 
> > You can never delete a kernel system call, so even if you "deprecate"
> > it, it still needs to be supported for forever.
> 
> If I'm not mistaken, some syscalls were dropped from kernel in the past,
> after it was established they are no longer used. So it is not impossible,
> though might be more difficult specifically with futex.

Those syscalls were all "compatible with other obsolete operating
system" syscalls from what I remember.  You can still run binaries
built in 1995 just fine on your system today (I have a few around here
somewhere...)

Thinking that you can drop futex() in the next 10+ years is very wishful
thinking given just how slow userspace applications are ever updated,
sorry.

greg k-h

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 12:06                   ` Andrey Semashev
  2021-06-08 12:33                     ` Greg KH
@ 2021-06-08 12:35                     ` Greg KH
  2021-06-08 13:18                       ` Andrey Semashev
  1 sibling, 1 reply; 53+ messages in thread
From: Greg KH @ 2021-06-08 12:35 UTC (permalink / raw)
  To: Andrey Semashev
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On Tue, Jun 08, 2021 at 03:06:48PM +0300, Andrey Semashev wrote:
> On 6/8/21 2:13 PM, Greg KH wrote:
> > On Tue, Jun 08, 2021 at 02:03:50PM +0300, Andrey Semashev wrote:
> > > On 6/8/21 4:25 AM, Nicholas Piggin wrote:
> > > > 
> > > > Are shared pthread mutexes using existing pthread APIs that are today
> > > > implemented okay with futex1 system call a good reason to constrain
> > > > futex2 I wonder? Or do we have an opportunity to make a bigger change
> > > > to the API so it suffers less from non deterministic latency (for
> > > > example)?
> > > 
> > > If futex2 is not able to cover futex1 use cases then it cannot be viewed as
> > > a replacement. In the long term this means futex1 cannot be deprecated and
> > > has to be maintained. My impression was that futex1 was basically
> > > unmaintainable(*) and futex2 was an evolution of futex1 so that users of
> > > futex1 could migrate relatively easily and futex1 eventually removed. Maybe
> > > my impression was wrong, but I would like to see futex2 as a replacement and
> > > extension of futex1, so the latter can be deprecated at some point.
> > 
> > You can never delete a kernel system call, so even if you "deprecate"
> > it, it still needs to be supported for forever.
> 
> If I'm not mistaken, some syscalls were dropped from kernel in the past,
> after it was established they are no longer used. So it is not impossible,
> though might be more difficult specifically with futex.
> 
> > Best of all would be if internally your "futex2" code would replace the
> > "futex1" code so that there is no two different code bases.  That would
> > be the only sane way forward, having 2 code bases to work with is just
> > insane.
> 
> Yes, implementing futex1 in terms of futex2 internally is a possible way
> forward. Though I'm not sure it is reasonable to require that to be done in
> the initial futex2 submission. This requires all of the futex1 functionality
> to implemented in futex2 from the start, which I think is too much to ask.
> Even with some futex1 features missing, futex2 would be already very much
> useful to users, and it is easier to implement the missing bits
> incrementally over time.

Then do it the other way around, as Peter points out.

> > So what's keeping the futex2 code from doing all that futex1 does so
> > that the futex1 code can be deleted internally?
> 
> I think, André will answer this, but my guess is, as stated above, this is a
> lot of work and time while the intermediate version is already useful.

useful to who?  I still do not understand what users will be needing
this.  All I can tell is a single userspace program wants to use it, and
that is a fork from the real project it was based on and that the
maintainers have no plan to merge it back.

So who does need/want this?

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 12:35                     ` Greg KH
@ 2021-06-08 13:18                       ` Andrey Semashev
  2021-06-08 13:27                         ` Greg KH
  2021-06-08 17:06                         ` Zebediah Figura
  0 siblings, 2 replies; 53+ messages in thread
From: Andrey Semashev @ 2021-06-08 13:18 UTC (permalink / raw)
  To: Greg KH
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On 6/8/21 3:35 PM, Greg KH wrote:
> On Tue, Jun 08, 2021 at 03:06:48PM +0300, Andrey Semashev wrote:
>> On 6/8/21 2:13 PM, Greg KH wrote:
> 
>>> So what's keeping the futex2 code from doing all that futex1 does so
>>> that the futex1 code can be deleted internally?
>>
>> I think, André will answer this, but my guess is, as stated above, this is a
>> lot of work and time while the intermediate version is already useful.
> 
> useful to who?  I still do not understand what users will be needing
> this.  All I can tell is a single userspace program wants to use it, and
> that is a fork from the real project it was based on and that the
> maintainers have no plan to merge it back.
> 
> So who does need/want this?

I mentioned C++ std::atomic and Boost.Atomic before. Those need variable 
sized futexes.

The project you mention is probably Wine and its derivatives. Those need 
variable sized futexes and "wait for multiple" operation. I'm not sure 
about the "no plan to merge it back" part, I probably missed it in an 
earlier discussion. There are multiple different patches and versions 
out there, and I don't know which one it refers to. But WaitOnAddress 
and WaitForMultipleObjects APIs are very important and I would assume 
Wine wants to emulate those with best efficiency.

I have a media processing engine application that needs 64-bit futexes 
would benefit from a "wait for multiple" function. Its source code is 
not open currently, so I'm not sure if you can count it as a valid user.

There is a generic std::lock algorithm in C++ and an equivalent in 
Boost.Thread that is supposed to lock multiple lockables (a mutex-like 
object). Those could benefit from the "wait for multiple" function in 
some cases, e.g. when the objects are actually futex-based mutexes, and 
the function can access the internal futex. I'm not saying this will 
definitely be implemented, it just looks like a possible optimization to me.

I think someone mentioned databases earlier in the discussion, but I 
don't know the details. I hope someone will be able to expand.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 13:18                       ` Andrey Semashev
@ 2021-06-08 13:27                         ` Greg KH
  2021-06-08 13:41                           ` Andrey Semashev
  2021-06-08 17:06                         ` Zebediah Figura
  1 sibling, 1 reply; 53+ messages in thread
From: Greg KH @ 2021-06-08 13:27 UTC (permalink / raw)
  To: Andrey Semashev
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On Tue, Jun 08, 2021 at 04:18:42PM +0300, Andrey Semashev wrote:
> On 6/8/21 3:35 PM, Greg KH wrote:
> > On Tue, Jun 08, 2021 at 03:06:48PM +0300, Andrey Semashev wrote:
> > > On 6/8/21 2:13 PM, Greg KH wrote:
> > 
> > > > So what's keeping the futex2 code from doing all that futex1 does so
> > > > that the futex1 code can be deleted internally?
> > > 
> > > I think, André will answer this, but my guess is, as stated above, this is a
> > > lot of work and time while the intermediate version is already useful.
> > 
> > useful to who?  I still do not understand what users will be needing
> > this.  All I can tell is a single userspace program wants to use it, and
> > that is a fork from the real project it was based on and that the
> > maintainers have no plan to merge it back.
> > 
> > So who does need/want this?
> 
> I mentioned C++ std::atomic and Boost.Atomic before. Those need variable
> sized futexes.

And has anyone converted them to use this new api to see if it works
well or not?

As was pointed out to me numerous times when I tried to propose
readfile(), you need a real user that can show and prove it is needed
before we can take new syscalls, especially complex beasts like this
one.

thanks,

greg k-h

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 13:27                         ` Greg KH
@ 2021-06-08 13:41                           ` Andrey Semashev
  0 siblings, 0 replies; 53+ messages in thread
From: Andrey Semashev @ 2021-06-08 13:41 UTC (permalink / raw)
  To: Greg KH
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On 6/8/21 4:27 PM, Greg KH wrote:
> On Tue, Jun 08, 2021 at 04:18:42PM +0300, Andrey Semashev wrote:
>> On 6/8/21 3:35 PM, Greg KH wrote:
>>> On Tue, Jun 08, 2021 at 03:06:48PM +0300, Andrey Semashev wrote:
>>>> On 6/8/21 2:13 PM, Greg KH wrote:
>>>
>>>>> So what's keeping the futex2 code from doing all that futex1 does so
>>>>> that the futex1 code can be deleted internally?
>>>>
>>>> I think, André will answer this, but my guess is, as stated above, this is a
>>>> lot of work and time while the intermediate version is already useful.
>>>
>>> useful to who?  I still do not understand what users will be needing
>>> this.  All I can tell is a single userspace program wants to use it, and
>>> that is a fork from the real project it was based on and that the
>>> maintainers have no plan to merge it back.
>>>
>>> So who does need/want this?
>>
>> I mentioned C++ std::atomic and Boost.Atomic before. Those need variable
>> sized futexes.
> 
> And has anyone converted them to use this new api to see if it works
> well or not?
> 
> As was pointed out to me numerous times when I tried to propose
> readfile(), you need a real user that can show and prove it is needed
> before we can take new syscalls, especially complex beasts like this
> one.

André has mentioned that he tested the patch set with patched Wine and 
glibc.

I didn't patch Boost.Atomic or std::atomic, but it doesn't look to be 
problematic. The only difference it would make there is to enable 
futex2-based implementation for multiple atomic sizes and set up flags 
to indicate the futex size, instead of only enabling futex-based 
implementation for 32-bit atomics.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 11:13                 ` Greg KH
  2021-06-08 11:44                   ` Peter Zijlstra
  2021-06-08 12:06                   ` Andrey Semashev
@ 2021-06-08 14:14                   ` André Almeida
  2 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-08 14:14 UTC (permalink / raw)
  To: Greg KH, Peter Zijlstra
  Cc: Nicholas Piggin, acme, Sebastian Andrzej Siewior, corbet,
	Andrey Semashev, Davidlohr Bueso, Darren Hart, fweimer, joel,
	kernel, krisman, libc-alpha, linux-api, linux-kernel,
	linux-kselftest, malteskarupke, Ingo Molnar, pgriffais,
	Peter Oskolkov, Steven Rostedt, shuah, Thomas Gleixner,
	z.figura12

Hi Greg,

Às 08:13 de 08/06/21, Greg KH escreveu:
> On Tue, Jun 08, 2021 at 02:03:50PM +0300, Andrey Semashev wrote:
>> On 6/8/21 4:25 AM, Nicholas Piggin wrote:
>>>
>>> Are shared pthread mutexes using existing pthread APIs that are today
>>> implemented okay with futex1 system call a good reason to constrain
>>> futex2 I wonder? Or do we have an opportunity to make a bigger change
>>> to the API so it suffers less from non deterministic latency (for
>>> example)?
>>
>> If futex2 is not able to cover futex1 use cases then it cannot be viewed as
>> a replacement. In the long term this means futex1 cannot be deprecated and
>> has to be maintained. My impression was that futex1 was basically
>> unmaintainable(*) and futex2 was an evolution of futex1 so that users of
>> futex1 could migrate relatively easily and futex1 eventually removed. Maybe
>> my impression was wrong, but I would like to see futex2 as a replacement and
>> extension of futex1, so the latter can be deprecated at some point.
> 
> You can never delete a kernel system call, so even if you "deprecate"
> it, it still needs to be supported for forever.
> 
> Best of all would be if internally your "futex2" code would replace the
> "futex1" code so that there is no two different code bases.  That would
> be the only sane way forward, having 2 code bases to work with is just
> insane.
> 
>> (*) I use "unmaintainable" in a broad sense here. It exists and works in
>> newer kernel versions and may receive code changes that are necessary to
>> keep it working, but maintainers refuse any extensions or modifications of
>> the code, mostly because of its complexity.
> 
> Adding additional complexity for no good reason is not a good idea,
> especially if you are asking others to maintain and support that
> complexity.  Would you want to have to do that work?
> 
> So what's keeping the futex2 code from doing all that futex1 does so
> that the futex1 code can be deleted internally?
> 

My very first submission of futex2[0] was just an overlay on top of
futex.c, I didn't get much feedback at that time, but I think this is
what you and Peter are thinking of?

After that, last year at Plumbers' RT MC, I presented a talk called
"futex2: A New Interface" and my conclusion after the discussion on this
talk + responses I got from my FUTEX_WAIT_MULTIPLE patchset[1] was that
this work couldn't be done at futex.c, given how fragile things are
there. futex.c would be "feature freeze" and no new major changes would
happen there.

This is the context where this new futex2 code base comes from. So,
which one is it? Happy to go either way but I'm getting conflicting
messages here.

Thanks,
	André

[0]
https://lore.kernel.org/lkml/20200612185122.327860-2-andrealmeid@collabora.com/

[1]
https://lore.kernel.org/lkml/20200213214525.183689-1-andrealmeid@collabora.com/

> thanks,
> 
> greg k-h
> 

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 12:26         ` Sebastian Andrzej Siewior
@ 2021-06-08 14:23           ` Peter Zijlstra
  2021-06-08 14:57             ` Sebastian Andrzej Siewior
                               ` (3 more replies)
  0 siblings, 4 replies; 53+ messages in thread
From: Peter Zijlstra @ 2021-06-08 14:23 UTC (permalink / raw)
  To: Sebastian Andrzej Siewior
  Cc: André Almeida, Nicholas Piggin, acme, Andrey Semashev,
	corbet, Davidlohr Bueso, Darren Hart, fweimer, joel, kernel,
	krisman, libc-alpha, linux-api, linux-kernel, linux-kselftest,
	malteskarupke, Ingo Molnar, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

On Tue, Jun 08, 2021 at 02:26:22PM +0200, Sebastian Andrzej Siewior wrote:
> On 2021-06-07 12:40:54 [-0300], André Almeida wrote:
> > 
> > When I first read Thomas proposal for per table process, I thought that
> > the main goal there was to solve NUMA locality issues, not RT latency,
> > but I think you are right. However, re-reading the thread at [0], it
> > seems that the RT problems where not completely solved in that
> > interface, maybe the people involved with that patchset can help to shed
> > some light on it.
> > 
> > Otherwise, this same proposal could be integrated in futex2, given that
> > we would only need to provide to userland some extra flags and add some
> > `if`s around the hash table code (in a very similar way the NUMA code
> > will be implemented in futex2).
> 
> There are slides at [0] describing some attempts and the kernel tree [1]
> from that time.
> 
> The process-table solves the problem to some degree that two random
> process don't collide on the same hash bucket. But as Peter Zijlstra
> pointed out back then two threads from the same task could collide on
> the same hash bucket (and with ASLR not always). So the collision is
> there but limited and this was not perfect.
> 
> All the attempts with API extensions didn't go well because glibc did
> not want to change a bit. This starts with a mutex that has a static
> initializer which has to work (I don't remember why the first
> pthread_mutex_lock() could not fail with -ENOMEM but there was
> something) and ends with glibc's struct mutex which is full and has no
> room for additional data storage.
> 
> The additional data in user's struct mutex + init would have the benefit
> that instead uaddr (which is hashed for the in-kernel lookup) a cookie
> could be used for the hash-less lookup (and NUMA pointer where memory
> should be stored).
> 
> So. We couldn't change a thing back then so nothing did happen. We
> didn't want to create a new interface and a library implementing it plus
> all the functionality around it (like pthread_cond, phtread_barrier, …).
> Not to mention that if glibc continues to use the "old" locking
> internally then the application is still affected by the hash-collision
> locking (or the NUMA problem) should it block on the lock.

There's more futex users than glibc, and some of them are really hurting
because of the NUMA issue. Oracle used to (I've no idea what they do or
do not do these days) use sysvsem because the futex hash table was a
massive bottleneck for them.

And as Nick said, other vendors are having the same problems.

And if you don't extend the futex to store the nid you put the waiter in
(see all the problems above) you will have to do wakeups on all nodes,
which is both slower than it is today, and scales possibly even worse.

The whole numa-aware qspinlock saga is in part because of futex.


That said; if we're going to do the whole futex-vector thing, we really
do need a new interface, because the futex multiplex monster is about to
crumble (see the fun wrt timeouts for example).

And if we're going to do a new interface, we ought to make one that can
solve all these problems. Now, ideally glibc will bring forth some
opinions, but if they don't want to play, we'll go back to the good old
days of non-standard locking libraries.. we're halfway there already due
to glibc not wanting to break with POSIX were we know POSIX was just
dead wrong broken.

See: https://github.com/dvhart/librtpi



^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 11:44                   ` Peter Zijlstra
@ 2021-06-08 14:31                     ` Davidlohr Bueso
  0 siblings, 0 replies; 53+ messages in thread
From: Davidlohr Bueso @ 2021-06-08 14:31 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Greg KH, Andrey Semashev, Nicholas Piggin,
	Andr� Almeida, acme, Sebastian Andrzej Siewior,
	corbet, Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12

On Tue, 08 Jun 2021, Peter Zijlstra wrote:

>On Tue, Jun 08, 2021 at 01:13:30PM +0200, Greg KH wrote:
>> On Tue, Jun 08, 2021 at 02:03:50PM +0300, Andrey Semashev wrote:
>> So what's keeping the futex2 code from doing all that futex1 does so
>> that the futex1 code can be deleted internally?
>
>I'd much rather see it the other way around. Much of what futex2 does
>can already be done with the futex1 code-base. And then we can add
>features on top.

Yeah furthermore, how can futex2 be thought of replacing futex1 in the
future when the PI aspects haven't even been mentioned?

Thanks,
Davidlohr

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 14:23           ` Peter Zijlstra
@ 2021-06-08 14:57             ` Sebastian Andrzej Siewior
  2021-06-08 15:04             ` André Almeida
                               ` (2 subsequent siblings)
  3 siblings, 0 replies; 53+ messages in thread
From: Sebastian Andrzej Siewior @ 2021-06-08 14:57 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: André Almeida, Nicholas Piggin, acme, Andrey Semashev,
	corbet, Davidlohr Bueso, Darren Hart, fweimer, joel, kernel,
	krisman, libc-alpha, linux-api, linux-kernel, linux-kselftest,
	malteskarupke, Ingo Molnar, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

On 2021-06-08 16:23:45 [+0200], Peter Zijlstra wrote:
> There's more futex users than glibc, and some of them are really hurting
> because of the NUMA issue. Oracle used to (I've no idea what they do or
> do not do these days) use sysvsem because the futex hash table was a
> massive bottleneck for them.
> 
> And as Nick said, other vendors are having the same problems.

I just wanted to do a brief summary of last events. The implementation
tglx did with the cookie resulting in a quick lookup did not have any
downsides except that the user-API had to change glibc couldn't. So if
we are back to square one why not start with that.

> And if you don't extend the futex to store the nid you put the waiter in
> (see all the problems above) you will have to do wakeups on all nodes,
> which is both slower than it is today, and scales possibly even worse.
> 
> The whole numa-aware qspinlock saga is in part because of futex.

sure.

> That said; if we're going to do the whole futex-vector thing, we really
> do need a new interface, because the futex multiplex monster is about to
> crumble (see the fun wrt timeouts for example).

This might have been a series of unfortunate events leading to this. The
sad part is that glibc has a comment that the kernel does not support
this and nobody bother to change it (until recently).

> And if we're going to do a new interface, we ought to make one that can
> solve all these problems. Now, ideally glibc will bring forth some
> opinions, but if they don't want to play, we'll go back to the good old
> days of non-standard locking libraries.. we're halfway there already due
> to glibc not wanting to break with POSIX were we know POSIX was just
> dead wrong broken.
> 
> See: https://github.com/dvhart/librtpi

I'm aware of that, I hacked on it, too :) This was the unfortunate
result of a ~8y old bug which was not fixed instead and part of the code
was rewritten and a bit-spinlock was added in user-land. You may
remember the discussion regarding spins in userland…
That said, REQUEUE_PI is no longer used by glibc.

Sebastian

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 14:23           ` Peter Zijlstra
  2021-06-08 14:57             ` Sebastian Andrzej Siewior
@ 2021-06-08 15:04             ` André Almeida
  2021-06-08 18:08             ` Adhemerval Zanella
  2021-06-09 16:26             ` David Laight
  3 siblings, 0 replies; 53+ messages in thread
From: André Almeida @ 2021-06-08 15:04 UTC (permalink / raw)
  To: Peter Zijlstra
  Cc: Nicholas Piggin, acme, Andrey Semashev, corbet, Davidlohr Bueso,
	Darren Hart, fweimer, joel, kernel, krisman, libc-alpha,
	linux-api, linux-kernel, linux-kselftest, malteskarupke,
	Ingo Molnar, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner, z.figura12, Sebastian Andrzej Siewior

Às 11:23 de 08/06/21, Peter Zijlstra escreveu:
> On Tue, Jun 08, 2021 at 02:26:22PM +0200, Sebastian Andrzej Siewior wrote:
>> On 2021-06-07 12:40:54 [-0300], André Almeida wrote:
>>>
>>> When I first read Thomas proposal for per table process, I thought that
>>> the main goal there was to solve NUMA locality issues, not RT latency,
>>> but I think you are right. However, re-reading the thread at [0], it
>>> seems that the RT problems where not completely solved in that
>>> interface, maybe the people involved with that patchset can help to shed
>>> some light on it.
>>>
>>> Otherwise, this same proposal could be integrated in futex2, given that
>>> we would only need to provide to userland some extra flags and add some
>>> `if`s around the hash table code (in a very similar way the NUMA code
>>> will be implemented in futex2).
>>
>> There are slides at [0] describing some attempts and the kernel tree [1]
>> from that time.
>>
>> The process-table solves the problem to some degree that two random
>> process don't collide on the same hash bucket. But as Peter Zijlstra
>> pointed out back then two threads from the same task could collide on
>> the same hash bucket (and with ASLR not always). So the collision is
>> there but limited and this was not perfect.
>>
>> All the attempts with API extensions didn't go well because glibc did
>> not want to change a bit. This starts with a mutex that has a static
>> initializer which has to work (I don't remember why the first
>> pthread_mutex_lock() could not fail with -ENOMEM but there was
>> something) and ends with glibc's struct mutex which is full and has no
>> room for additional data storage.
>>
>> The additional data in user's struct mutex + init would have the benefit
>> that instead uaddr (which is hashed for the in-kernel lookup) a cookie
>> could be used for the hash-less lookup (and NUMA pointer where memory
>> should be stored).
>>
>> So. We couldn't change a thing back then so nothing did happen. We
>> didn't want to create a new interface and a library implementing it plus
>> all the functionality around it (like pthread_cond, phtread_barrier, …).
>> Not to mention that if glibc continues to use the "old" locking
>> internally then the application is still affected by the hash-collision
>> locking (or the NUMA problem) should it block on the lock.
> 
> There's more futex users than glibc, and some of them are really hurting
> because of the NUMA issue. Oracle used to (I've no idea what they do or
> do not do these days) use sysvsem because the futex hash table was a
> massive bottleneck for them.
> 
> And as Nick said, other vendors are having the same problems.

Since we're talking about NUMA, which userspace communities would be
able to provide feedback about the futex2() NUMA-aware feature, to check
if this interface would help solving those issues?

> 
> And if you don't extend the futex to store the nid you put the waiter in
> (see all the problems above) you will have to do wakeups on all nodes,
> which is both slower than it is today, and scales possibly even worse.
> 
> The whole numa-aware qspinlock saga is in part because of futex.
> 
> 
> That said; if we're going to do the whole futex-vector thing, we really
> do need a new interface, because the futex multiplex monster is about to
> crumble (see the fun wrt timeouts for example).
> 
> And if we're going to do a new interface, we ought to make one that can
> solve all these problems. Now, ideally glibc will bring forth some
> opinions, but if they don't want to play, we'll go back to the good old
> days of non-standard locking libraries.. we're halfway there already due
> to glibc not wanting to break with POSIX were we know POSIX was just
> dead wrong broken.
> 
> See: https://github.com/dvhart/librtpi
> 
> 

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 13:18                       ` Andrey Semashev
  2021-06-08 13:27                         ` Greg KH
@ 2021-06-08 17:06                         ` Zebediah Figura
  1 sibling, 0 replies; 53+ messages in thread
From: Zebediah Figura @ 2021-06-08 17:06 UTC (permalink / raw)
  To: Andrey Semashev, Greg KH
  Cc: Nicholas Piggin, André Almeida, acme,
	Sebastian Andrzej Siewior, corbet, Davidlohr Bueso, Darren Hart,
	fweimer, joel, kernel, krisman, libc-alpha, linux-api,
	linux-kernel, linux-kselftest, malteskarupke, Ingo Molnar,
	Peter Zijlstra, pgriffais, Peter Oskolkov, Steven Rostedt, shuah,
	Thomas Gleixner

On 6/8/21 8:18 AM, Andrey Semashev wrote:
> On 6/8/21 3:35 PM, Greg KH wrote:
>> On Tue, Jun 08, 2021 at 03:06:48PM +0300, Andrey Semashev wrote:
>>> On 6/8/21 2:13 PM, Greg KH wrote:
>>
>>>> So what's keeping the futex2 code from doing all that futex1 does so
>>>> that the futex1 code can be deleted internally?
>>>
>>> I think, André will answer this, but my guess is, as stated above, 
>>> this is a
>>> lot of work and time while the intermediate version is already useful.
>>
>> useful to who?  I still do not understand what users will be needing
>> this.  All I can tell is a single userspace program wants to use it, and
>> that is a fork from the real project it was based on and that the
>> maintainers have no plan to merge it back.
>>
>> So who does need/want this?
> 
> I mentioned C++ std::atomic and Boost.Atomic before. Those need variable 
> sized futexes.
> 
> The project you mention is probably Wine and its derivatives. Those need 
> variable sized futexes and "wait for multiple" operation. I'm not sure 
> about the "no plan to merge it back" part, I probably missed it in an 
> earlier discussion. There are multiple different patches and versions 
> out there, and I don't know which one it refers to. But WaitOnAddress 
> and WaitForMultipleObjects APIs are very important and I would assume 
> Wine wants to emulate those with best efficiency.

See [0]. The short version is that we can't use futexes the way that 
out-of-tree patch set does, due to compatibility and robustness 
problems. I wrote said patch set and I'm currently working on a 
different solution for upstreaming.

We also can't exactly use futexes to back WaitOnAddress() directly. We 
actually do currently, but for various complex reasons that needs to 
change, and none of the proposals for futex2 help the new implementation.

ἔρρωσο,
Zebediah

[0] 
https://lore.kernel.org/lkml/dab34fd2-b494-8686-bcd7-68beeba4f386@gmail.com/

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 14:23           ` Peter Zijlstra
  2021-06-08 14:57             ` Sebastian Andrzej Siewior
  2021-06-08 15:04             ` André Almeida
@ 2021-06-08 18:08             ` Adhemerval Zanella
  2021-06-08 18:19               ` Florian Weimer
  2021-06-09 16:26             ` David Laight
  3 siblings, 1 reply; 53+ messages in thread
From: Adhemerval Zanella @ 2021-06-08 18:08 UTC (permalink / raw)
  To: Peter Zijlstra, Sebastian Andrzej Siewior
  Cc: linux-kselftest, joel, Andrey Semashev, kernel, shuah,
	Davidlohr Bueso, Peter Oskolkov, corbet, krisman, malteskarupke,
	Ingo Molnar, Darren Hart, acme, Steven Rostedt,
	André Almeida, Thomas Gleixner, fweimer, libc-alpha,
	linux-api, z.figura12, Nicholas Piggin, linux-kernel, pgriffais


> All the attempts with API extensions didn't go well because glibc did
> not want to change a bit. This starts with a mutex that has a static
> initializer which has to work (I don't remember why the first
> pthread_mutex_lock() could not fail with -ENOMEM but there was
> something) and ends with glibc's struct mutex which is full and has no
> room for additional data storage.

Yes, we have binary compatibility constraints that prevents us to simply
broken old binaries.  This is quite true for static initialization,
which imposes even harder constraints, different than the pthread_mutex_t
size where we can workaround with symbols versioning. But even then we hear
from users that out pthread_mutex_t is still way larger, specially for
fine grained locking so I am not sure if we do want to extend it.

> That said; if we're going to do the whole futex-vector thing, we really
> do need a new interface, because the futex multiplex monster is about to
> crumble (see the fun wrt timeouts for example).
> 
> And if we're going to do a new interface, we ought to make one that can
> solve all these problems. Now, ideally glibc will bring forth some
> opinions, but if they don't want to play, we'll go back to the good old
> days of non-standard locking libraries.. we're halfway there already due
> to glibc not wanting to break with POSIX were we know POSIX was just
> dead wrong broken.
> 
> See: https://github.com/dvhart/librtpi

You are right, we don't really want to break POSIX requirements in this
regard because users constantly come with scenarios where they do expect
our implementation to be conformant [1].  And even now, there are case we 
don't get it fully right [2] and it is really hard to fix such issues.

If I recall correctly from a recent plumber couple of years ago about 
the librtpi, the presents stated their implement do not follow POSIX
standard by design.  It suits then for their required work, but it is 
not what we really aim for glibc.  We *might* try to provide as an 
extension, but even then I am not if it would be fully possible due 
API constraints.

So, regarding the futex2 we might try to support it eventually; but if
this newer interface is not a really a superset of futex1 we won't 
put much effort. Supporting newer syscall requires an extra effort from
glibc, we need to keep fallback for older ones in case the kernel is
too old and it also imposes runtime costs.

Also currently we don't have a specific usage.  The proposed patch to
add the 'pthread_mutex_lock_any' and 'pthreada_timedlock_any' [3] 
also did not gave much detail in realword usages or how it can be
leveraged.

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=13165
[2] https://sourceware.org/bugzilla/show_bug.cgi?id=25847
[3] https://sourceware.org/pipermail/libc-alpha/2019-July/105422.html

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 18:08             ` Adhemerval Zanella
@ 2021-06-08 18:19               ` Florian Weimer
  2021-06-08 18:22                 ` Adhemerval Zanella
  0 siblings, 1 reply; 53+ messages in thread
From: Florian Weimer @ 2021-06-08 18:19 UTC (permalink / raw)
  To: Adhemerval Zanella
  Cc: Peter Zijlstra, Sebastian Andrzej Siewior, linux-kselftest, joel,
	Andrey Semashev, kernel, shuah, Davidlohr Bueso, Peter Oskolkov,
	corbet, krisman, malteskarupke, Ingo Molnar, Darren Hart, acme,
	Steven Rostedt, André Almeida, Thomas Gleixner, libc-alpha,
	linux-api, z.figura12, Nicholas Piggin, linux-kernel, pgriffais

* Adhemerval Zanella:

> Also currently we don't have a specific usage.  The proposed patch to
> add the 'pthread_mutex_lock_any' and 'pthreada_timedlock_any' [3] 
> also did not gave much detail in realword usages or how it can be
> leveraged.

The current rwlock implementation in glibc uses a torn 32-bit futex read
which is part of an atomically updated 64-bit word.  That's just really,
really ugly, and I suspect we could make that go away with futex2.

Thanks,
Florian


^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 18:19               ` Florian Weimer
@ 2021-06-08 18:22                 ` Adhemerval Zanella
  0 siblings, 0 replies; 53+ messages in thread
From: Adhemerval Zanella @ 2021-06-08 18:22 UTC (permalink / raw)
  To: Florian Weimer
  Cc: Peter Zijlstra, Sebastian Andrzej Siewior, linux-kselftest, joel,
	Andrey Semashev, kernel, shuah, Davidlohr Bueso, Peter Oskolkov,
	corbet, krisman, malteskarupke, Ingo Molnar, Darren Hart, acme,
	Steven Rostedt, André Almeida, Thomas Gleixner, libc-alpha,
	linux-api, z.figura12, Nicholas Piggin, linux-kernel, pgriffais



On 08/06/2021 15:19, Florian Weimer wrote:
> * Adhemerval Zanella:
> 
>> Also currently we don't have a specific usage.  The proposed patch to
>> add the 'pthread_mutex_lock_any' and 'pthreada_timedlock_any' [3] 
>> also did not gave much detail in realword usages or how it can be
>> leveraged.
> 
> The current rwlock implementation in glibc uses a torn 32-bit futex read
> which is part of an atomically updated 64-bit word.  That's just really,
> really ugly, and I suspect we could make that go away with futex2.

You are right, I had in the mind the multiple wait proposed by this
patch and by the glib RFC one.  Not only rwlock, but the posix
semaphore might also be simplified on 32 bits I think.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* RE: [PATCH v4 00/15] Add futex2 syscalls
  2021-06-08 14:23           ` Peter Zijlstra
                               ` (2 preceding siblings ...)
  2021-06-08 18:08             ` Adhemerval Zanella
@ 2021-06-09 16:26             ` David Laight
  3 siblings, 0 replies; 53+ messages in thread
From: David Laight @ 2021-06-09 16:26 UTC (permalink / raw)
  To: 'Peter Zijlstra', Sebastian Andrzej Siewior
  Cc: André Almeida, Nicholas Piggin, acme, Andrey Semashev,
	corbet, Davidlohr Bueso, Darren Hart, fweimer, joel, kernel,
	krisman, libc-alpha, linux-api, linux-kernel, linux-kselftest,
	malteskarupke, Ingo Molnar, pgriffais, Peter Oskolkov,
	Steven Rostedt, shuah, Thomas Gleixner, z.figura12

From: Peter Zijlstra
> Sent: 08 June 2021 15:24
...
> And if we're going to do a new interface, we ought to make one that can
> solve all these problems. Now, ideally glibc will bring forth some
> opinions, but if they don't want to play, we'll go back to the good old
> days of non-standard locking libraries.. we're halfway there already due
> to glibc not wanting to break with POSIX were we know POSIX was just
> dead wrong broken.

I had a problem with pthread_broadcast().
I've got multiple threads all bound to different cpu and
I really do want to wake them all up at once.
Now, I know they'll spin on the mutex for a bit - but that
is soon released (the 'adaptive' spin is probably long enough).

But what actually happens (probably because of the way glibc
is constrained by the futext system call) is:
1) The first thread is woken.
2) It's cpu comes out of sleep.
3) Once running it wakes the second thread.
4) It's cpu comes out of sleep.
...
So you get a very slow ripple of the threads starting.

Worse still, if the thread can't be scheduled (eg because
its cpu is running all the network stack softint code)
then none of the later threads start running.

I've mitigated it by using a separate cv for each thread
and looping to wake them all - but it shouldn't really
be necessary.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)

^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2021-06-09 16:26 UTC | newest]

Thread overview: 53+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-03 19:59 [PATCH v4 00/15] Add futex2 syscalls André Almeida
2021-06-03 19:59 ` [PATCH v4 01/15] futex2: Implement wait and wake functions André Almeida
2021-06-03 19:59 ` [PATCH v4 02/15] futex2: Add support for shared futexes André Almeida
2021-06-03 19:59 ` [PATCH v4 03/15] futex2: Implement vectorized wait André Almeida
2021-06-03 19:59 ` [PATCH v4 04/15] futex2: Implement requeue operation André Almeida
2021-06-03 19:59 ` [PATCH v4 05/15] futex2: Implement support for different futex sizes André Almeida
2021-06-04  0:23   ` kernel test robot
2021-06-06 19:12   ` Davidlohr Bueso
2021-06-06 23:01     ` Andrey Semashev
2021-06-03 19:59 ` [PATCH v4 06/15] futex2: Add compatibility entry point for x86_x32 ABI André Almeida
2021-06-03 19:59 ` [PATCH v4 07/15] docs: locking: futex2: Add documentation André Almeida
2021-06-06 19:23   ` Davidlohr Bueso
2021-06-03 19:59 ` [PATCH v4 08/15] selftests: futex2: Add wake/wait test André Almeida
2021-06-03 19:59 ` [PATCH v4 09/15] selftests: futex2: Add timeout test André Almeida
2021-06-03 19:59 ` [PATCH v4 10/15] selftests: futex2: Add wouldblock test André Almeida
2021-06-03 19:59 ` [PATCH v4 11/15] selftests: futex2: Add waitv test André Almeida
2021-06-03 19:59 ` [PATCH v4 12/15] selftests: futex2: Add requeue test André Almeida
2021-06-03 19:59 ` [PATCH v4 13/15] selftests: futex2: Add futex sizes test André Almeida
2021-06-03 19:59 ` [PATCH v4 14/15] perf bench: Add futex2 benchmark tests André Almeida
2021-06-03 19:59 ` [PATCH v4 15/15] kernel: Enable waitpid() for futex2 André Almeida
2021-06-04  4:51 ` [PATCH v4 00/15] Add futex2 syscalls Zebediah Figura
2021-06-04 17:04   ` André Almeida
2021-06-04 11:36 ` Nicholas Piggin
2021-06-04 20:01   ` André Almeida
2021-06-05  1:09     ` Nicholas Piggin
2021-06-05  8:56       ` Andrey Semashev
2021-06-06 11:57         ` Nicholas Piggin
2021-06-06 13:15           ` Andrey Semashev
2021-06-08  1:25             ` Nicholas Piggin
2021-06-08 11:03               ` Andrey Semashev
2021-06-08 11:13                 ` Greg KH
2021-06-08 11:44                   ` Peter Zijlstra
2021-06-08 14:31                     ` Davidlohr Bueso
2021-06-08 12:06                   ` Andrey Semashev
2021-06-08 12:33                     ` Greg KH
2021-06-08 12:35                     ` Greg KH
2021-06-08 13:18                       ` Andrey Semashev
2021-06-08 13:27                         ` Greg KH
2021-06-08 13:41                           ` Andrey Semashev
2021-06-08 17:06                         ` Zebediah Figura
2021-06-08 14:14                   ` André Almeida
2021-06-07 15:40       ` André Almeida
2021-06-08  1:31         ` Nicholas Piggin
2021-06-08  2:33         ` Davidlohr Bueso
2021-06-08  4:45           ` Nicholas Piggin
2021-06-08 12:26         ` Sebastian Andrzej Siewior
2021-06-08 14:23           ` Peter Zijlstra
2021-06-08 14:57             ` Sebastian Andrzej Siewior
2021-06-08 15:04             ` André Almeida
2021-06-08 18:08             ` Adhemerval Zanella
2021-06-08 18:19               ` Florian Weimer
2021-06-08 18:22                 ` Adhemerval Zanella
2021-06-09 16:26             ` David Laight

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.