Linux-ARM-Kernel Archive on lore.kernel.org
 help / color / Atom feed
From: Will Deacon <will@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Mark Rutland <mark.rutland@arm.com>,
	"Michael S. Tsirkin" <mst@redhat.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Catalin Marinas <catalin.marinas@arm.com>,
	Jason Wang <jasowang@redhat.com>,
	virtualization@lists.linux-foundation.org,
	Will Deacon <will@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
	Alan Stern <stern@rowland.harvard.edu>,
	Sami Tolvanen <samitolvanen@google.com>,
	Matt Turner <mattst88@gmail.com>,
	kernel-team@android.com, Marco Elver <elver@google.com>,
	Kees Cook <keescook@chromium.org>,
	"Paul E. McKenney" <paulmck@kernel.org>,
	Boqun Feng <boqun.feng@gmail.com>,
	Josh Triplett <josh@joshtriplett.org>,
	Ivan Kokshaysky <ink@jurassic.park.msu.ru>,
	linux-arm-kernel@lists.infradead.org,
	Richard Henderson <rth@twiddle.net>,
	Nick Desaulniers <ndesaulniers@google.com>,
	linux-alpha@vger.kernel.org
Subject: [PATCH 18/18] arm64: lto: Strengthen READ_ONCE() to acquire when CLANG_LTO=y
Date: Tue, 30 Jun 2020 18:37:34 +0100
Message-ID: <20200630173734.14057-19-will@kernel.org> (raw)
In-Reply-To: <20200630173734.14057-1-will@kernel.org>

When building with LTO, there is an increased risk of the compiler
converting an address dependency headed by a READ_ONCE() invocation
into a control dependency and consequently allowing for harmful
reordering by the CPU.

Ensure that such transformations are harmless by overriding the generic
READ_ONCE() definition with one that provides acquire semantics when
building with LTO.

Signed-off-by: Will Deacon <will@kernel.org>
---
 arch/arm64/include/asm/rwonce.h   | 63 +++++++++++++++++++++++++++++++
 arch/arm64/kernel/vdso/Makefile   |  2 +-
 arch/arm64/kernel/vdso32/Makefile |  2 +-
 3 files changed, 65 insertions(+), 2 deletions(-)
 create mode 100644 arch/arm64/include/asm/rwonce.h

diff --git a/arch/arm64/include/asm/rwonce.h b/arch/arm64/include/asm/rwonce.h
new file mode 100644
index 000000000000..515e360b01a1
--- /dev/null
+++ b/arch/arm64/include/asm/rwonce.h
@@ -0,0 +1,63 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 Google LLC.
+ */
+#ifndef __ASM_RWONCE_H
+#define __ASM_RWONCE_H
+
+#ifdef CONFIG_CLANG_LTO
+
+#include <linux/compiler_types.h>
+#include <asm/alternative-macros.h>
+
+#ifndef BUILD_VDSO
+
+#ifdef CONFIG_AS_HAS_LDAPR
+#define __LOAD_RCPC(sfx, regs...)					\
+	ALTERNATIVE(							\
+		"ldar"	#sfx "\t" #regs,				\
+		".arch_extension rcpc\n"				\
+		"ldapr"	#sfx "\t" #regs,				\
+	ARM64_HAS_LDAPR)
+#else
+#define __LOAD_RCPC(sfx, regs...)	"ldar" #sfx "\t" #regs
+#endif /* CONFIG_AS_HAS_LDAPR */
+
+#define __READ_ONCE(x)							\
+({									\
+	int atomic = 1;							\
+	union { __unqual_scalar_typeof(x) __val; char __c[1]; } __u;	\
+	typeof(&(x)) __x = &(x);					\
+	switch (sizeof(x)) {						\
+	case 1:								\
+		asm volatile(__LOAD_RCPC(b, %w0, %1)			\
+			: "=r" (*(__u8 *)__u.__c)			\
+			: "Q" (*__x) : "memory");			\
+		break;							\
+	case 2:								\
+		asm volatile(__LOAD_RCPC(h, %w0, %1)			\
+			: "=r" (*(__u16 *)__u.__c)			\
+			: "Q" (*__x) : "memory");			\
+		break;							\
+	case 4:								\
+		asm volatile(__LOAD_RCPC(, %w0, %1)			\
+			: "=r" (*(__u32 *)__u.__c)			\
+			: "Q" (*__x) : "memory");			\
+		break;							\
+	case 8:								\
+		asm volatile(__LOAD_RCPC(, %0, %1)			\
+			: "=r" (*(__u64 *)__u.__c)			\
+			: "Q" (*__x) : "memory");			\
+		break;							\
+	default:							\
+		atomic = 0;						\
+	}								\
+	atomic ? (typeof(x))__u.__val : (*(volatile typeof(x) *)__x);	\
+})
+
+#endif	/* !BUILD_VDSO */
+#endif	/* CONFIG_CLANG_LTO */
+
+#include <asm-generic/rwonce.h>
+
+#endif	/* __ASM_RWONCE_H */
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 45d5cfe46429..60df97f2e7de 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -28,7 +28,7 @@ ldflags-y := -shared -nostdlib -soname=linux-vdso.so.1 --hash-style=sysv	\
 	     $(btildflags-y) -T
 
 ccflags-y := -fno-common -fno-builtin -fno-stack-protector -ffixed-x18
-ccflags-y += -DDISABLE_BRANCH_PROFILING
+ccflags-y += -DDISABLE_BRANCH_PROFILING -DBUILD_VDSO
 
 CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) $(GCC_PLUGINS_CFLAGS)
 KBUILD_CFLAGS			+= $(DISABLE_LTO)
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
index d88148bef6b0..4fdf3754a058 100644
--- a/arch/arm64/kernel/vdso32/Makefile
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -43,7 +43,7 @@ cc32-as-instr = $(call try-run,\
 # As a result we set our own flags here.
 
 # KBUILD_CPPFLAGS and NOSTDINC_FLAGS from top-level Makefile
-VDSO_CPPFLAGS := -D__KERNEL__ -nostdinc -isystem $(shell $(CC_COMPAT) -print-file-name=include)
+VDSO_CPPFLAGS := -DBUILD_VDSO -D__KERNEL__ -nostdinc -isystem $(shell $(CC_COMPAT) -print-file-name=include)
 VDSO_CPPFLAGS += $(LINUXINCLUDE)
 
 # Common C and assembly flags
-- 
2.27.0.212.ge8ba1cc988-goog


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  parent reply index

Thread overview: 58+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-30 17:37 [PATCH 00/18] Allow architectures to override __READ_ONCE() Will Deacon
2020-06-30 17:37 ` [PATCH 01/18] tools: bpf: Use local copy of headers including uapi/linux/filter.h Will Deacon
2020-07-01 16:38   ` Alexei Starovoitov
2020-06-30 17:37 ` [PATCH 02/18] compiler.h: Split {READ, WRITE}_ONCE definitions out into rwonce.h Will Deacon
2020-06-30 19:11   ` Arnd Bergmann
2020-07-01 10:16     ` [PATCH 02/18] compiler.h: Split {READ,WRITE}_ONCE " Will Deacon
2020-07-01 11:33       ` [PATCH 02/18] compiler.h: Split {READ, WRITE}_ONCE " Arnd Bergmann
2020-06-30 17:37 ` [PATCH 03/18] asm/rwonce: Allow __READ_ONCE to be overridden by the architecture Will Deacon
2020-06-30 17:37 ` [PATCH 04/18] alpha: Override READ_ONCE() with barriered implementation Will Deacon
2020-07-02  9:32   ` Mark Rutland
2020-07-02  9:48     ` Will Deacon
2020-07-02 10:08       ` Arnd Bergmann
2020-07-02 11:18         ` Will Deacon
2020-07-02 11:39           ` Arnd Bergmann
2020-07-02 14:43   ` Joel Fernandes
2020-07-02 14:55     ` Will Deacon
2020-07-02 15:07       ` Joel Fernandes
2020-06-30 17:37 ` [PATCH 05/18] asm/rwonce: Remove smp_read_barrier_depends() invocation Will Deacon
2020-06-30 17:37 ` [PATCH 06/18] vhost: Remove redundant use of read_barrier_depends() barrier Will Deacon
2020-06-30 17:37 ` [PATCH 07/18] alpha: Replace smp_read_barrier_depends() usage with smp_[r]mb() Will Deacon
2020-06-30 17:37 ` [PATCH 08/18] locking/barriers: Remove definitions for [smp_]read_barrier_depends() Will Deacon
2020-06-30 17:37 ` [PATCH 09/18] Documentation/barriers: Remove references to [smp_]read_barrier_depends() Will Deacon
2020-06-30 17:37 ` [PATCH 10/18] Documentation/barriers/kokr: " Will Deacon
2020-06-30 17:37 ` [PATCH 11/18] tools/memory-model: Remove smp_read_barrier_depends() from informal doc Will Deacon
2020-06-30 17:37 ` [PATCH 12/18] include/linux: Remove smp_read_barrier_depends() from comments Will Deacon
2020-06-30 17:37 ` [PATCH 13/18] checkpatch: Remove checks relating to [smp_]read_barrier_depends() Will Deacon
2020-06-30 17:37 ` [PATCH 14/18] arm64: Reduce the number of header files pulled into vmlinux.lds.S Will Deacon
2020-06-30 17:37 ` [PATCH 15/18] arm64: alternatives: Split up alternative.h Will Deacon
2020-06-30 17:37 ` [PATCH 16/18] arm64: cpufeatures: Add capability for LDAPR instruction Will Deacon
2020-06-30 17:37 ` [PATCH 17/18] arm64: alternatives: Remove READ_ONCE() usage during patch operation Will Deacon
2020-06-30 17:37 ` Will Deacon [this message]
2020-06-30 19:25   ` [PATCH 18/18] arm64: lto: Strengthen READ_ONCE() to acquire when CLANG_LTO=y Arnd Bergmann
2020-07-01 10:19     ` Will Deacon
2020-07-01 10:59       ` Arnd Bergmann
2020-06-30 19:47   ` Marco Elver
2020-06-30 20:20     ` Peter Zijlstra
2020-06-30 22:57     ` Sami Tolvanen
2020-07-01 10:25       ` Will Deacon
2020-07-01 10:24     ` Will Deacon
2020-07-01 17:07   ` Dave P Martin
2020-07-02  7:23     ` Will Deacon
2020-07-06 16:00       ` Dave Martin
2020-07-06 16:34         ` Paul E. McKenney
2020-07-06 17:05           ` Dave Martin
2020-07-06 17:36             ` Paul E. McKenney
2020-07-07 10:29               ` Dave Martin
2020-07-07 22:51                 ` Paul E. McKenney
2020-07-07 23:01                   ` Nick Desaulniers
2020-07-08  7:15                     ` Marco Elver
2020-07-08  9:16                     ` Peter Zijlstra
2020-07-08 18:20                       ` Paul E. McKenney
2020-07-06 18:35         ` Will Deacon
2020-07-06 19:23           ` Marco Elver
2020-07-06 19:42             ` Paul E. McKenney
2020-07-06 16:08   ` Dave Martin
2020-07-06 18:35     ` Will Deacon
2020-07-07 10:10       ` Dave Martin
2020-07-01  7:38 ` [PATCH 00/18] Allow architectures to override __READ_ONCE() Josh Triplett

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200630173734.14057-19-will@kernel.org \
    --to=will@kernel.org \
    --cc=arnd@arndb.de \
    --cc=boqun.feng@gmail.com \
    --cc=catalin.marinas@arm.com \
    --cc=elver@google.com \
    --cc=ink@jurassic.park.msu.ru \
    --cc=jasowang@redhat.com \
    --cc=josh@joshtriplett.org \
    --cc=keescook@chromium.org \
    --cc=kernel-team@android.com \
    --cc=linux-alpha@vger.kernel.org \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mark.rutland@arm.com \
    --cc=mattst88@gmail.com \
    --cc=mst@redhat.com \
    --cc=ndesaulniers@google.com \
    --cc=paulmck@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rth@twiddle.net \
    --cc=samitolvanen@google.com \
    --cc=stern@rowland.harvard.edu \
    --cc=virtualization@lists.linux-foundation.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-ARM-Kernel Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/0 linux-arm-kernel/git/0.git
	git clone --mirror https://lore.kernel.org/linux-arm-kernel/1 linux-arm-kernel/git/1.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-arm-kernel linux-arm-kernel/ https://lore.kernel.org/linux-arm-kernel \
		linux-arm-kernel@lists.infradead.org
	public-inbox-index linux-arm-kernel

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.infradead.lists.linux-arm-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git