From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: linux-kernel@vger.kernel.org
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
stable@vger.kernel.org, stable@kernel.org,
Will Deacon <will.deacon@arm.com>
Subject: [PATCH 4.14 42/42] arm64: futex: Bound number of LDXR/STXR loops in FUTEX_WAKE_OP
Date: Thu, 9 May 2019 20:42:31 +0200 [thread overview]
Message-ID: <20190509181300.512525411@linuxfoundation.org> (raw)
In-Reply-To: <20190509181252.616018683@linuxfoundation.org>
From: Will Deacon <will.deacon@arm.com>
commit 03110a5cb2161690ae5ac04994d47ed0cd6cef75 upstream.
Our futex implementation makes use of LDXR/STXR loops to perform atomic
updates to user memory from atomic context. This can lead to latency
problems if we end up spinning around the LL/SC sequence at the expense
of doing something useful.
Rework our futex atomic operations so that we return -EAGAIN if we fail
to update the futex word after 128 attempts. The core futex code will
reschedule if necessary and we'll try again later.
Cc: <stable@kernel.org>
Fixes: 6170a97460db ("arm64: Atomic operations")
Signed-off-by: Will Deacon <will.deacon@arm.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
---
arch/arm64/include/asm/futex.h | 55 +++++++++++++++++++++++++----------------
1 file changed, 34 insertions(+), 21 deletions(-)
--- a/arch/arm64/include/asm/futex.h
+++ b/arch/arm64/include/asm/futex.h
@@ -23,26 +23,34 @@
#include <asm/errno.h>
+#define FUTEX_MAX_LOOPS 128 /* What's the largest number you can think of? */
+
#define __futex_atomic_op(insn, ret, oldval, uaddr, tmp, oparg) \
do { \
+ unsigned int loops = FUTEX_MAX_LOOPS; \
+ \
uaccess_enable(); \
asm volatile( \
" prfm pstl1strm, %2\n" \
"1: ldxr %w1, %2\n" \
insn "\n" \
"2: stlxr %w0, %w3, %2\n" \
-" cbnz %w0, 1b\n" \
-" dmb ish\n" \
+" cbz %w0, 3f\n" \
+" sub %w4, %w4, %w0\n" \
+" cbnz %w4, 1b\n" \
+" mov %w0, %w7\n" \
"3:\n" \
+" dmb ish\n" \
" .pushsection .fixup,\"ax\"\n" \
" .align 2\n" \
-"4: mov %w0, %w5\n" \
+"4: mov %w0, %w6\n" \
" b 3b\n" \
" .popsection\n" \
_ASM_EXTABLE(1b, 4b) \
_ASM_EXTABLE(2b, 4b) \
- : "=&r" (ret), "=&r" (oldval), "+Q" (*uaddr), "=&r" (tmp) \
- : "r" (oparg), "Ir" (-EFAULT) \
+ : "=&r" (ret), "=&r" (oldval), "+Q" (*uaddr), "=&r" (tmp), \
+ "+r" (loops) \
+ : "r" (oparg), "Ir" (-EFAULT), "Ir" (-EAGAIN) \
: "memory"); \
uaccess_disable(); \
} while (0)
@@ -57,23 +65,23 @@ arch_futex_atomic_op_inuser(int op, int
switch (op) {
case FUTEX_OP_SET:
- __futex_atomic_op("mov %w3, %w4",
+ __futex_atomic_op("mov %w3, %w5",
ret, oldval, uaddr, tmp, oparg);
break;
case FUTEX_OP_ADD:
- __futex_atomic_op("add %w3, %w1, %w4",
+ __futex_atomic_op("add %w3, %w1, %w5",
ret, oldval, uaddr, tmp, oparg);
break;
case FUTEX_OP_OR:
- __futex_atomic_op("orr %w3, %w1, %w4",
+ __futex_atomic_op("orr %w3, %w1, %w5",
ret, oldval, uaddr, tmp, oparg);
break;
case FUTEX_OP_ANDN:
- __futex_atomic_op("and %w3, %w1, %w4",
+ __futex_atomic_op("and %w3, %w1, %w5",
ret, oldval, uaddr, tmp, ~oparg);
break;
case FUTEX_OP_XOR:
- __futex_atomic_op("eor %w3, %w1, %w4",
+ __futex_atomic_op("eor %w3, %w1, %w5",
ret, oldval, uaddr, tmp, oparg);
break;
default:
@@ -93,6 +101,7 @@ futex_atomic_cmpxchg_inatomic(u32 *uval,
u32 oldval, u32 newval)
{
int ret = 0;
+ unsigned int loops = FUTEX_MAX_LOOPS;
u32 val, tmp;
u32 __user *uaddr;
@@ -104,20 +113,24 @@ futex_atomic_cmpxchg_inatomic(u32 *uval,
asm volatile("// futex_atomic_cmpxchg_inatomic\n"
" prfm pstl1strm, %2\n"
"1: ldxr %w1, %2\n"
-" sub %w3, %w1, %w4\n"
-" cbnz %w3, 3f\n"
-"2: stlxr %w3, %w5, %2\n"
-" cbnz %w3, 1b\n"
-" dmb ish\n"
+" sub %w3, %w1, %w5\n"
+" cbnz %w3, 4f\n"
+"2: stlxr %w3, %w6, %2\n"
+" cbz %w3, 3f\n"
+" sub %w4, %w4, %w3\n"
+" cbnz %w4, 1b\n"
+" mov %w0, %w8\n"
"3:\n"
+" dmb ish\n"
+"4:\n"
" .pushsection .fixup,\"ax\"\n"
-"4: mov %w0, %w6\n"
-" b 3b\n"
+"5: mov %w0, %w7\n"
+" b 4b\n"
" .popsection\n"
- _ASM_EXTABLE(1b, 4b)
- _ASM_EXTABLE(2b, 4b)
- : "+r" (ret), "=&r" (val), "+Q" (*uaddr), "=&r" (tmp)
- : "r" (oldval), "r" (newval), "Ir" (-EFAULT)
+ _ASM_EXTABLE(1b, 5b)
+ _ASM_EXTABLE(2b, 5b)
+ : "+r" (ret), "=&r" (val), "+Q" (*uaddr), "=&r" (tmp), "+r" (loops)
+ : "r" (oldval), "r" (newval), "Ir" (-EFAULT), "Ir" (-EAGAIN)
: "memory");
uaccess_disable();
next prev parent reply other threads:[~2019-05-09 18:46 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-05-09 18:41 [PATCH 4.14 00/42] 4.14.118-stable review Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 01/42] scsi: libsas: fix a race condition when smp task timeout Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 02/42] Drivers: hv: vmbus: Remove the undesired put_cpu_ptr() in hv_synic_cleanup() Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 03/42] ubsan: Fix nasty -Wbuiltin-declaration-mismatch GCC-9 warnings Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 04/42] staging: greybus: power_supply: fix prop-descriptor request size Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 05/42] ASoC: hdmi-codec: fix S/PDIF DAI Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 06/42] ASoC:soc-pcm:fix a codec fixup issue in TDM case Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 07/42] ASoC: nau8824: fix the issue of the widget with prefix name Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 08/42] ASoC: nau8810: fix the issue of widget with prefixed name Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 09/42] ASoC: samsung: odroid: Fix clock configuration for 44100 sample rate Greg Kroah-Hartman
2019-05-09 18:41 ` [PATCH 4.14 10/42] ASoC: wm_adsp: Add locking to wm_adsp2_bus_error Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 11/42] ASoC: cs4270: Set auto-increment bit for register writes Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 12/42] IB/hfi1: Eliminate opcode tests on mr deref Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 13/42] MIPS: KGDB: fix kgdb support for SMP platforms Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 14/42] ASoC: tlv320aic32x4: Fix Common Pins Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 15/42] drm/mediatek: Fix an error code in mtk_hdmi_dt_parse_pdata() Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 16/42] perf/x86/intel: Fix handling of wakeup_events for multi-entry PEBS Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 17/42] perf/x86/intel: Initialize TFA MSR Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 18/42] linux/kernel.h: Use parentheses around argument in u64_to_user_ptr() Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 19/42] xtensa: fix initialization of pt_regs::syscall in start_thread Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 20/42] ASoC: rockchip: pdm: fix regmap_ops hang issue Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 21/42] slab: fix a crash by reading /proc/slab_allocators Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 22/42] virtio_pci: fix a NULL pointer reference in vp_del_vqs Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 23/42] RDMA/vmw_pvrdma: Fix memory leak on pvrdma_pci_remove Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 24/42] scsi: csiostor: fix missing data copy in csio_scsi_err_handler() Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 25/42] drm/mediatek: fix possible object reference leak Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 26/42] ASoC: Intel: kbl: fix wrong number of channels Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 27/42] virtio-blk: limit number of hw queues by nr_cpu_ids Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 28/42] platform/x86: pmc_atom: Drop __initconst on dmi table Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 29/42] iommu/amd: Set exclusion range correctly Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 30/42] genirq: Prevent use-after-free and work list corruption Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 31/42] usb: dwc3: Fix default lpm_nyet_threshold value Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 32/42] USB: serial: f81232: fix interrupt worker not stop Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 33/42] USB: cdc-acm: fix unthrottle races Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 34/42] usb-storage: Set virt_boundary_mask to avoid SG overflows Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 35/42] intel_th: pci: Add Comet Lake support Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 36/42] scsi: qla2xxx: Fix incorrect region-size setting in optrom SYSFS routines Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 37/42] Bluetooth: hidp: fix buffer overflow Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 38/42] Bluetooth: Align minimum encryption key size for LE and BR/EDR connections Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 39/42] UAS: fix alignment of scatter/gather segments Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 40/42] ASoC: Intel: avoid Oops if DMA setup fails Greg Kroah-Hartman
2019-05-09 18:42 ` [PATCH 4.14 41/42] locking/futex: Allow low-level atomic operations to return -EAGAIN Greg Kroah-Hartman
2019-05-09 18:42 ` Greg Kroah-Hartman [this message]
2019-05-09 23:27 ` [PATCH 4.14 00/42] 4.14.118-stable review kernelci.org bot
2019-05-10 8:45 ` Naresh Kamboju
2019-05-10 10:16 ` Jon Hunter
2019-05-10 13:35 ` Guenter Roeck
2019-05-10 21:26 ` shuah
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190509181300.512525411@linuxfoundation.org \
--to=gregkh@linuxfoundation.org \
--cc=linux-kernel@vger.kernel.org \
--cc=stable@kernel.org \
--cc=stable@vger.kernel.org \
--cc=will.deacon@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).