* [PATCH v3 00/24] Unify vDSOs across more architectures
@ 2018-12-13 16:57 Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
` (24 more replies)
0 siblings, 25 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.
Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.
The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in
"asm/vdso/".
* The generic implementation includes the arch specific one and lives
in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place
with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm and mips.
The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].
Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].
To simplify the testing, a copy of the patchset on top of a recent linux tree
can be found at [3] and [4].
[1] https://github.com/nlynch-mentor/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v3
[4] git://linux-arm.org/linux-vf.git vdso-mips/v3
Changes:
--------
v3:
- Ported the latest fixes and optimizations done on the x86
architecture to the generic library.
- Addressed review comments.
- Improved the documentation of the interfaces.
- Changed the HAVE_ARCH_TIMER config option to a more generic
HAVE_HW_COUNTER.
v2:
- Added -ffixed-x18 to arm64
- Repleced occurrences of timeval and timespec
- Modified datapage.h to be compliant with y2038 on all the architectures
- Removed __u_vdso type
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Peter Collingbourne (1):
arm64: Build vDSO with -ffixed-x18
Vincenzo Frascino (23):
kernel: Standardize vdso_datapage
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Introduce vdso_direct
clock: csrc-4k: Add support for vdso_direct
clock: gic-timer: Add support for vdso_direct
mips: Add support for generic vDSO
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 85 +++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 100 ++--
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 +---------
arch/arm64/Kconfig | 25 +
arch/arm64/Makefile | 21 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 +-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 122 +++++
arch/arm64/include/asm/vdso/gettimeofday.h | 88 ++++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 41 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 +--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 498 +++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 +
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 +++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 81 +++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++
arch/mips/Kconfig | 3 +
arch/mips/include/asm/clocksource.h | 1 +
arch/mips/include/asm/vdso.h | 60 +--
arch/mips/include/asm/vdso/gettimeofday.h | 168 ++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/csrc-r4k.c | 4 +-
arch/mips/kernel/vdso.c | 103 +++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +
drivers/clocksource/mips-gic-timer.c | 5 +-
include/vdso/datapage.h | 61 +++
include/vdso/helpers.h | 58 ++
include/vdso/types.h | 39 ++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 41 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 192 +++++++
57 files changed, 2253 insertions(+), 1303 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 include/vdso/datapage.h
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
--
2.19.2
^ permalink raw reply [flat|nested] 58+ messages in thread
* [PATCH v3 00/24] Unify vDSOs across more architectures
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 01/24] kernel: Standardize vdso_datapage Vincenzo Frascino
` (23 subsequent siblings)
24 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.
Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.
The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in
"asm/vdso/".
* The generic implementation includes the arch specific one and lives
in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place
with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm and mips.
The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].
Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].
To simplify the testing, a copy of the patchset on top of a recent linux tree
can be found at [3] and [4].
[1] https://github.com/nlynch-mentor/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v3
[4] git://linux-arm.org/linux-vf.git vdso-mips/v3
Changes:
--------
v3:
- Ported the latest fixes and optimizations done on the x86
architecture to the generic library.
- Addressed review comments.
- Improved the documentation of the interfaces.
- Changed the HAVE_ARCH_TIMER config option to a more generic
HAVE_HW_COUNTER.
v2:
- Added -ffixed-x18 to arm64
- Repleced occurrences of timeval and timespec
- Modified datapage.h to be compliant with y2038 on all the architectures
- Removed __u_vdso type
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Peter Collingbourne (1):
arm64: Build vDSO with -ffixed-x18
Vincenzo Frascino (23):
kernel: Standardize vdso_datapage
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Introduce vdso_direct
clock: csrc-4k: Add support for vdso_direct
clock: gic-timer: Add support for vdso_direct
mips: Add support for generic vDSO
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 85 +++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 100 ++--
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 +---------
arch/arm64/Kconfig | 25 +
arch/arm64/Makefile | 21 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 +-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 122 +++++
arch/arm64/include/asm/vdso/gettimeofday.h | 88 ++++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 41 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 +--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 498 +++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 +
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 +++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 81 +++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++
arch/mips/Kconfig | 3 +
arch/mips/include/asm/clocksource.h | 1 +
arch/mips/include/asm/vdso.h | 60 +--
arch/mips/include/asm/vdso/gettimeofday.h | 168 ++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/csrc-r4k.c | 4 +-
arch/mips/kernel/vdso.c | 103 +++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +
drivers/clocksource/mips-gic-timer.c | 5 +-
include/vdso/datapage.h | 61 +++
include/vdso/helpers.h | 58 ++
include/vdso/types.h | 39 ++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 41 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 192 +++++++
57 files changed, 2253 insertions(+), 1303 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 include/vdso/datapage.h
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
--
2.19.2
^ permalink raw reply [flat|nested] 58+ messages in thread
* [PATCH v3 01/24] kernel: Standardize vdso_datapage
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 02/24] kernel: Define gettimeofday vdso common code Vincenzo Frascino
` (22 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch generalized the data structures in vgtod.h
from x86 private includes to general includes (include/vdso).
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 60 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 60 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..e88678f918e5
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/bits.h>
+#include <linux/types.h>
+#include <linux/time.h>
+
+#define VDSO_BASES (CLOCK_TAI + 1)
+#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
+ BIT(CLOCK_MONOTONIC) | \
+ BIT(CLOCK_MONOTONIC_RAW) | \
+ BIT(CLOCK_BOOTTIME) | \
+ BIT(CLOCK_TAI))
+#define VDSO_COARSE (BIT(CLOCK_REALTIME_COARSE) | \
+ BIT(CLOCK_MONOTONIC_COARSE))
+
+/*
+ * There is one vdso_timestamp object in vvar for each vDSO-accelerated
+ * clock_id. For high-resolution clocks, this encodes the time
+ * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
+ * the actual time.
+ *
+ * To be noticed that nsec is left-shifted by vdso_data.shift.
+ */
+struct vdso_timestamp {
+ u64 sec;
+ u64 nsec;
+};
+
+/*
+ * vdso_data will be accessed by 32 and 64 bit code at the same time
+ * so we should be careful before modifying this structure.
+ */
+struct vdso_data {
+ u32 seq; /* Timebase sequence counter */
+
+ s32 clock_mode;
+ u64 cycle_last; /* Timebase at clocksource init */
+ u64 mask; /* Clocksource mask (mono = raw) */
+ u32 mult; /* Clocksource multiplier */
+ u32 shift; /* Clocksource shift (mono = raw) */
+
+ struct vdso_timestamp basetime[VDSO_BASES];
+
+ s32 tz_minuteswest; /* Timezone definitions */
+ s32 tz_dsttime;
+ u32 use_syscall;
+ u32 __reserved;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 01/24] kernel: Standardize vdso_datapage
2018-12-13 16:57 ` [PATCH v3 01/24] kernel: Standardize vdso_datapage Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch generalized the data structures in vgtod.h
from x86 private includes to general includes (include/vdso).
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 60 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 60 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..e88678f918e5
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,60 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/bits.h>
+#include <linux/types.h>
+#include <linux/time.h>
+
+#define VDSO_BASES (CLOCK_TAI + 1)
+#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
+ BIT(CLOCK_MONOTONIC) | \
+ BIT(CLOCK_MONOTONIC_RAW) | \
+ BIT(CLOCK_BOOTTIME) | \
+ BIT(CLOCK_TAI))
+#define VDSO_COARSE (BIT(CLOCK_REALTIME_COARSE) | \
+ BIT(CLOCK_MONOTONIC_COARSE))
+
+/*
+ * There is one vdso_timestamp object in vvar for each vDSO-accelerated
+ * clock_id. For high-resolution clocks, this encodes the time
+ * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
+ * the actual time.
+ *
+ * To be noticed that nsec is left-shifted by vdso_data.shift.
+ */
+struct vdso_timestamp {
+ u64 sec;
+ u64 nsec;
+};
+
+/*
+ * vdso_data will be accessed by 32 and 64 bit code at the same time
+ * so we should be careful before modifying this structure.
+ */
+struct vdso_data {
+ u32 seq; /* Timebase sequence counter */
+
+ s32 clock_mode;
+ u64 cycle_last; /* Timebase at clocksource init */
+ u64 mask; /* Clocksource mask (mono = raw) */
+ u32 mult; /* Clocksource multiplier */
+ u32 shift; /* Clocksource shift (mono = raw) */
+
+ struct vdso_timestamp basetime[VDSO_BASES];
+
+ s32 tz_minuteswest; /* Timezone definitions */
+ s32 tz_dsttime;
+ u32 use_syscall;
+ u32 __reserved;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 02/24] kernel: Define gettimeofday vdso common code
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 01/24] kernel: Standardize vdso_datapage Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-14 18:15 ` Mark Salyzyn
2018-12-13 16:57 ` [PATCH v3 03/24] arm64: Build vDSO with -ffixed-x18 Vincenzo Frascino
` (21 subsequent siblings)
24 siblings, 2 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
This patch tries to unify the gettimeofday vdso implementation
introducing lib/vdso. The code contained in this library can
ideally be reused by all the architectures avoiding, where
possible, code duplication.
In doing so, tries to maintain the performances using inlining
as much as possible and consequently reduces the surface for
ROP type of attacks.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
include/vdso/helpers.h | 58 +++++++++++++
include/vdso/types.h | 39 +++++++++
lib/Kconfig | 5 ++
lib/vdso/Kconfig | 41 +++++++++
lib/vdso/Makefile | 22 +++++
lib/vdso/gettimeofday.c | 188 ++++++++++++++++++++++++++++++++++++++++
7 files changed, 354 insertions(+)
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index e88678f918e5..bdc9648720eb 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,6 +9,7 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
+#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
new file mode 100644
index 000000000000..66ec7f6b459d
--- /dev/null
+++ b/include/vdso/helpers.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_HELPERS_H
+#define __VDSO_HELPERS_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <vdso/datapage.h>
+
+/*
+ * To improve performances, in this file, __always_inline it is used
+ * for the functions called multiple times.
+ */
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->seq);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ smp_rmb();
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->seq);
+ return seq != start;
+}
+
+static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
+{
+ ++vd->seq;
+ smp_wmb();
+}
+
+static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
+{
+ smp_wmb();
+ ++vd->seq;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_HELPERS_H */
diff --git a/include/vdso/types.h b/include/vdso/types.h
new file mode 100644
index 000000000000..f456a0a6a2e1
--- /dev/null
+++ b/include/vdso/types.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_TYPES_H
+#define __VDSO_TYPES_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+#include <linux/time.h>
+
+/*
+ * The definitions below are required to overcome the limitations
+ * of time_t on 32 bit architectures, which overflows in 2038.
+ * The new code should use the replacements based on time64_t and
+ * timespec64.
+ *
+ * The abstraction below will be updated once the migration to
+ * time64_t is complete.
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#ifdef ENABLE_COMPAT_VDSO
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#define __vdso_timespec __kernel_timespec
+#define __vdso_timeval __kernel_old_timeval
+#endif /* CONFIG_COMPAT_VDSO */
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_TYPES_H */
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..38a048e514a2 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -557,6 +557,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..55c453b95dcd
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config HAVE_HW_COUNTER
+ bool
+ depends on GENERIC_GETTIMEOFDAY
+ help
+ Select this configuration option if the architecture has an arch
+ timer.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..30010d25bf5b
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,188 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2017 Cavium, Inc.
+ * Copyright (C) 2015 Mentor Graphics Corporation
+ *
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/uaccess.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
+
+/*
+ * The generic vDSO implementation requires that gettimeofday.h
+ * provides:
+ * - __arch_get_vdso_data(): to get the vdso datapage.
+ * - __arch_get_hw_counter(): to get the hw counter based on the
+ * clock_mode.
+ * - __arch_get_realtime_res(): to get the correct realtime res.
+ * - __arch_get_coarse_res(): to get the correct coarse res.
+ * - gettimeofday_fallback(): fallback for gettimeofday.
+ * - clock_gettime_fallback(): fallback for clock_gettime.
+ * - clock_getres_fallback(): fallback for clock_getres.
+ */
+#include <asm/vdso/gettimeofday.h>
+
+#ifdef CONFIG_HAVE_HW_COUNTER
+static notrace int do_hres(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u64 cycles, last, sec, ns;
+ u32 seq;
+
+ if (vd->use_syscall)
+ return -1;
+
+ do {
+ seq = vdso_read_begin(vd);
+ cycles = __arch_get_hw_counter(vd->clock_mode) & vd->mask;
+ ns = vdso_ts->nsec;
+ last = vd->cycle_last;
+ if (unlikely((s64)cycles < 0))
+ return clock_gettime_fallback(clk, ts);
+ if (cycles > last)
+ ns += (cycles - last) * vd->mult;
+ ns >>= vd->shift;
+ sec = vdso_ts->sec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static notrace int do_hres(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ return -1;
+}
+#endif
+
+static notrace void do_coarse(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u32 seq;
+
+ do {
+ seq = vdso_read_begin(vd);
+ ts->tv_sec = vdso_ts->sec;
+ ts->tv_nsec = vdso_ts->nsec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (likely(msk & VDSO_HRES)) {
+ return do_hres(vd, clock, ts);
+ } else if (msk & VDSO_COARSE) {
+ do_coarse(vd, clock, ts);
+ return 0;
+ }
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct __vdso_timespec ts;
+
+ if (do_hres(vd, CLOCK_REALTIME, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / 1000;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+#ifdef VDSO_HAS_TIME
+static notrace time_t __cvdso_time(time_t *time)
+{
+ u32 seq;
+ time_t t;
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ t = vdso_ts->sec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ if (unlikely(time != NULL))
+ *time = t;
+
+ return t;
+}
+#endif /* VDSO_HAS_TIME */
+
+static notrace int __cvdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ u64 ns;
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (msk & VDSO_HRES)
+ ns = __arch_get_realtime_res(clock);
+ else if (msk & VDSO_COARSE)
+ ns = __arch_get_coarse_res(clock);
+ else
+ goto fallback;
+
+ if (res) {
+ res->tv_sec = 0;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+
+fallback:
+ return clock_getres_fallback(clock, res);
+}
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 02/24] kernel: Define gettimeofday vdso common code
2018-12-13 16:57 ` [PATCH v3 02/24] kernel: Define gettimeofday vdso common code Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-14 18:15 ` Mark Salyzyn
1 sibling, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
This patch tries to unify the gettimeofday vdso implementation
introducing lib/vdso. The code contained in this library can
ideally be reused by all the architectures avoiding, where
possible, code duplication.
In doing so, tries to maintain the performances using inlining
as much as possible and consequently reduces the surface for
ROP type of attacks.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
include/vdso/helpers.h | 58 +++++++++++++
include/vdso/types.h | 39 +++++++++
lib/Kconfig | 5 ++
lib/vdso/Kconfig | 41 +++++++++
lib/vdso/Makefile | 22 +++++
lib/vdso/gettimeofday.c | 188 ++++++++++++++++++++++++++++++++++++++++
7 files changed, 354 insertions(+)
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index e88678f918e5..bdc9648720eb 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,6 +9,7 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
+#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
new file mode 100644
index 000000000000..66ec7f6b459d
--- /dev/null
+++ b/include/vdso/helpers.h
@@ -0,0 +1,58 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_HELPERS_H
+#define __VDSO_HELPERS_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <vdso/datapage.h>
+
+/*
+ * To improve performances, in this file, __always_inline it is used
+ * for the functions called multiple times.
+ */
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->seq);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ smp_rmb();
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->seq);
+ return seq != start;
+}
+
+static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
+{
+ ++vd->seq;
+ smp_wmb();
+}
+
+static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
+{
+ smp_wmb();
+ ++vd->seq;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_HELPERS_H */
diff --git a/include/vdso/types.h b/include/vdso/types.h
new file mode 100644
index 000000000000..f456a0a6a2e1
--- /dev/null
+++ b/include/vdso/types.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_TYPES_H
+#define __VDSO_TYPES_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+#include <linux/time.h>
+
+/*
+ * The definitions below are required to overcome the limitations
+ * of time_t on 32 bit architectures, which overflows in 2038.
+ * The new code should use the replacements based on time64_t and
+ * timespec64.
+ *
+ * The abstraction below will be updated once the migration to
+ * time64_t is complete.
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#ifdef ENABLE_COMPAT_VDSO
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#define __vdso_timespec __kernel_timespec
+#define __vdso_timeval __kernel_old_timeval
+#endif /* CONFIG_COMPAT_VDSO */
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_TYPES_H */
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..38a048e514a2 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -557,6 +557,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..55c453b95dcd
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config HAVE_HW_COUNTER
+ bool
+ depends on GENERIC_GETTIMEOFDAY
+ help
+ Select this configuration option if the architecture has an arch
+ timer.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..30010d25bf5b
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,188 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2017 Cavium, Inc.
+ * Copyright (C) 2015 Mentor Graphics Corporation
+ *
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/uaccess.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
+
+/*
+ * The generic vDSO implementation requires that gettimeofday.h
+ * provides:
+ * - __arch_get_vdso_data(): to get the vdso datapage.
+ * - __arch_get_hw_counter(): to get the hw counter based on the
+ * clock_mode.
+ * - __arch_get_realtime_res(): to get the correct realtime res.
+ * - __arch_get_coarse_res(): to get the correct coarse res.
+ * - gettimeofday_fallback(): fallback for gettimeofday.
+ * - clock_gettime_fallback(): fallback for clock_gettime.
+ * - clock_getres_fallback(): fallback for clock_getres.
+ */
+#include <asm/vdso/gettimeofday.h>
+
+#ifdef CONFIG_HAVE_HW_COUNTER
+static notrace int do_hres(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u64 cycles, last, sec, ns;
+ u32 seq;
+
+ if (vd->use_syscall)
+ return -1;
+
+ do {
+ seq = vdso_read_begin(vd);
+ cycles = __arch_get_hw_counter(vd->clock_mode) & vd->mask;
+ ns = vdso_ts->nsec;
+ last = vd->cycle_last;
+ if (unlikely((s64)cycles < 0))
+ return clock_gettime_fallback(clk, ts);
+ if (cycles > last)
+ ns += (cycles - last) * vd->mult;
+ ns >>= vd->shift;
+ sec = vdso_ts->sec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static notrace int do_hres(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ return -1;
+}
+#endif
+
+static notrace void do_coarse(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u32 seq;
+
+ do {
+ seq = vdso_read_begin(vd);
+ ts->tv_sec = vdso_ts->sec;
+ ts->tv_nsec = vdso_ts->nsec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (likely(msk & VDSO_HRES)) {
+ return do_hres(vd, clock, ts);
+ } else if (msk & VDSO_COARSE) {
+ do_coarse(vd, clock, ts);
+ return 0;
+ }
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct __vdso_timespec ts;
+
+ if (do_hres(vd, CLOCK_REALTIME, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / 1000;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+#ifdef VDSO_HAS_TIME
+static notrace time_t __cvdso_time(time_t *time)
+{
+ u32 seq;
+ time_t t;
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ t = vdso_ts->sec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ if (unlikely(time != NULL))
+ *time = t;
+
+ return t;
+}
+#endif /* VDSO_HAS_TIME */
+
+static notrace int __cvdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ u64 ns;
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (msk & VDSO_HRES)
+ ns = __arch_get_realtime_res(clock);
+ else if (msk & VDSO_COARSE)
+ ns = __arch_get_coarse_res(clock);
+ else
+ goto fallback;
+
+ if (res) {
+ res->tv_sec = 0;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+
+fallback:
+ return clock_getres_fallback(clock, res);
+}
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 03/24] arm64: Build vDSO with -ffixed-x18
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (2 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 02/24] kernel: Define gettimeofday vdso common code Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 04/24] arm64: Substitute gettimeofday with C implementation Vincenzo Frascino
` (20 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne, Mark Salyzyn
From: Peter Collingbourne <pcc@google.com>
The vDSO needs to be build with x18 reserved in order to accommodate
userspace platform ABIs built on top of Linux that use the register
to carry inter-procedural state, as provided for by the AAPCS.
An example of such a platform ABI is the one that will be used by an
upcoming version of Android.
Although this change is currently a no-op due to the fact that the vDSO
is currently implemented in pure assembly on arm64, it is necessary
in order to prepare for another change [1] that will add C code to
the vDSO.
[1] https://patchwork.kernel.org/patch/10044501/
Signed-off-by: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Mark Salyzyn <salyzyn@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/kernel/vdso/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..8cd4707f832a 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -12,7 +12,7 @@ obj-vdso := gettimeofday.o note.o sigreturn.o
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 03/24] arm64: Build vDSO with -ffixed-x18
2018-12-13 16:57 ` [PATCH v3 03/24] arm64: Build vDSO with -ffixed-x18 Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Mark Salyzyn
From: Peter Collingbourne <pcc@google.com>
The vDSO needs to be build with x18 reserved in order to accommodate
userspace platform ABIs built on top of Linux that use the register
to carry inter-procedural state, as provided for by the AAPCS.
An example of such a platform ABI is the one that will be used by an
upcoming version of Android.
Although this change is currently a no-op due to the fact that the vDSO
is currently implemented in pure assembly on arm64, it is necessary
in order to prepare for another change [1] that will add C code to
the vDSO.
[1] https://patchwork.kernel.org/patch/10044501/
Signed-off-by: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Mark Salyzyn <salyzyn@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/kernel/vdso/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..8cd4707f832a 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -12,7 +12,7 @@ obj-vdso := gettimeofday.o note.o sigreturn.o
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 04/24] arm64: Substitute gettimeofday with C implementation
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (3 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 03/24] arm64: Build vDSO with -ffixed-x18 Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 05/24] arm64: compat: Alloc separate pages for vectors and sigpage Vincenzo Frascino
` (19 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
This patch re-implements the gettimeofday vdso in C in order to use
lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME
and CLOCK_TAI.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 3 +
arch/arm64/include/asm/vdso/gettimeofday.h | 88 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 35 ++-
arch/arm64/kernel/vdso.c | 94 ++++--
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 ++
8 files changed, 230 insertions(+), 421 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ea2ab0330e3a..985f76109531 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -69,6 +69,7 @@ config ARM64
select ARCH_HAS_UBSAN_SANITIZE_ALL
select ARM_AMBA
select ARM_ARCH_TIMER
+ select HAVE_HW_COUNTER
select ARM_GIC
select AUDIT_ARCH_COMPAT_GENERIC
select ARM_GIC_V2M if PCI
@@ -101,6 +102,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
@@ -153,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..8a0173b51697
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct __vdso_timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 323aeb5f2fe6..a716ff952a57 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -105,19 +105,26 @@ int main(void)
DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
- DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
- DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
- DEFINE(VDSO_RAW_TIME_NSEC, offsetof(struct vdso_data, raw_time_nsec));
- DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
- DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
- DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
- DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
- DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
- DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
- DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
- DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
- DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
- DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_SEQ, offsetof(struct vdso_data, seq));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
+ DEFINE(VDSO_CYCLE_LAST, offsetof(struct vdso_data, cycle_last));
+ DEFINE(VDSO_MASK, offsetof(struct vdso_data, mask));
+ DEFINE(VDSO_MULT, offsetof(struct vdso_data, mult));
+ DEFINE(VDSO_SHIFT, offsetof(struct vdso_data, shift));
+ DEFINE(VDSO_REALTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].sec));
+ DEFINE(VDSO_REALTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].nsec));
+ DEFINE(VDSO_MONO_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].sec));
+ DEFINE(VDSO_MONO_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].nsec));
+ DEFINE(VDSO_MONO_RAW_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].sec));
+ DEFINE(VDSO_MONO_RAW_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].nsec));
+ DEFINE(VDSO_BOOTTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].sec));
+ DEFINE(VDSO_BOOTTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].nsec));
+ DEFINE(VDSO_TAI_SEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].sec));
+ DEFINE(VDSO_TAI_NSEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].nsec));
+ DEFINE(VDSO_RT_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].sec));
+ DEFINE(VDSO_RT_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].nsec));
+ DEFINE(VDSO_MONO_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].sec));
+ DEFINE(VDSO_MONO_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].nsec));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..c91039b94f28 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,12 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,38 +216,85 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
void update_vsyscall(struct timekeeper *tk)
{
+ struct vdso_timestamp *vdso_ts;
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
-
- ++vdso_data->tb_seq_count;
- smp_wmb();
-
- vdso_data->use_syscall = use_syscall;
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift;
- vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ vdso_data->use_syscall = use_syscall;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
if (!use_syscall) {
- /* tkr_mono.cycle_last == tkr_raw.cycle_last */
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->raw_time_sec = tk->raw_sec;
- vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mono_mult = tk->tkr_mono.mult;
- vdso_data->cs_raw_mult = tk->tkr_raw.mult;
- /* tkr_mono.shift == tkr_raw.shift */
- vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->mask = VDSO_PRECISION_MASK;
+ vdso_data->mult = tk->tkr_mono.mult;
+ vdso_data->shift = tk->tkr_mono.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- smp_wmb();
- ++vdso_data->tb_seq_count;
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 8cd4707f832a..c904b2a9b255 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,31 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS := -Bsymbolic
+
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +48,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +65,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
- cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
+ cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) -Wl,-n -Wl,-T $^ -o $@
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..160e68196339
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 04/24] arm64: Substitute gettimeofday with C implementation
2018-12-13 16:57 ` [PATCH v3 04/24] arm64: Substitute gettimeofday with C implementation Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
This patch re-implements the gettimeofday vdso in C in order to use
lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME
and CLOCK_TAI.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 3 +
arch/arm64/include/asm/vdso/gettimeofday.h | 88 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 35 ++-
arch/arm64/kernel/vdso.c | 94 ++++--
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 ++
8 files changed, 230 insertions(+), 421 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ea2ab0330e3a..985f76109531 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -69,6 +69,7 @@ config ARM64
select ARCH_HAS_UBSAN_SANITIZE_ALL
select ARM_AMBA
select ARM_ARCH_TIMER
+ select HAVE_HW_COUNTER
select ARM_GIC
select AUDIT_ARCH_COMPAT_GENERIC
select ARM_GIC_V2M if PCI
@@ -101,6 +102,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
@@ -153,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..8a0173b51697
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,88 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct __vdso_timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 323aeb5f2fe6..a716ff952a57 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -105,19 +105,26 @@ int main(void)
DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
- DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
- DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
- DEFINE(VDSO_RAW_TIME_NSEC, offsetof(struct vdso_data, raw_time_nsec));
- DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
- DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
- DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
- DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
- DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
- DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
- DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
- DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
- DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
- DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_SEQ, offsetof(struct vdso_data, seq));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
+ DEFINE(VDSO_CYCLE_LAST, offsetof(struct vdso_data, cycle_last));
+ DEFINE(VDSO_MASK, offsetof(struct vdso_data, mask));
+ DEFINE(VDSO_MULT, offsetof(struct vdso_data, mult));
+ DEFINE(VDSO_SHIFT, offsetof(struct vdso_data, shift));
+ DEFINE(VDSO_REALTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].sec));
+ DEFINE(VDSO_REALTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].nsec));
+ DEFINE(VDSO_MONO_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].sec));
+ DEFINE(VDSO_MONO_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].nsec));
+ DEFINE(VDSO_MONO_RAW_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].sec));
+ DEFINE(VDSO_MONO_RAW_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].nsec));
+ DEFINE(VDSO_BOOTTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].sec));
+ DEFINE(VDSO_BOOTTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].nsec));
+ DEFINE(VDSO_TAI_SEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].sec));
+ DEFINE(VDSO_TAI_NSEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].nsec));
+ DEFINE(VDSO_RT_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].sec));
+ DEFINE(VDSO_RT_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].nsec));
+ DEFINE(VDSO_MONO_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].sec));
+ DEFINE(VDSO_MONO_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].nsec));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..c91039b94f28 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,12 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,38 +216,85 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
void update_vsyscall(struct timekeeper *tk)
{
+ struct vdso_timestamp *vdso_ts;
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
-
- ++vdso_data->tb_seq_count;
- smp_wmb();
-
- vdso_data->use_syscall = use_syscall;
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift;
- vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ vdso_data->use_syscall = use_syscall;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
if (!use_syscall) {
- /* tkr_mono.cycle_last == tkr_raw.cycle_last */
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->raw_time_sec = tk->raw_sec;
- vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mono_mult = tk->tkr_mono.mult;
- vdso_data->cs_raw_mult = tk->tkr_raw.mult;
- /* tkr_mono.shift == tkr_raw.shift */
- vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->mask = VDSO_PRECISION_MASK;
+ vdso_data->mult = tk->tkr_mono.mult;
+ vdso_data->shift = tk->tkr_mono.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- smp_wmb();
- ++vdso_data->tb_seq_count;
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 8cd4707f832a..c904b2a9b255 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,31 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS := -Bsymbolic
+
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +48,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +65,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
- cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
+ cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) -Wl,-n -Wl,-T $^ -o $@
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..160e68196339
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 05/24] arm64: compat: Alloc separate pages for vectors and sigpage
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (4 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 04/24] arm64: Substitute gettimeofday with C implementation Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 06/24] arm64: compat: Split kuser32 Vincenzo Frascino
` (18 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
This patch addresses the problem creating separate pages for vectors and
sigpage in a similar fashion to what happens today on arm.
This patch changes as well the meaning of mm->context.vdso for AArch32
compat since it now points to sigpage and not to vectors anymore in
order to make simpler the implementation of the signal handling (the
address of sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 433b9554c6a1..f2b6b486deb7 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -210,10 +210,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 6b0d4dff5012..6d4070cb813c 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -67,9 +67,9 @@
#define STACK_TOP_MAX TASK_SIZE_64
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 24b09003f821..265a6fab331d 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index c91039b94f28..5df7aeef324a 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -54,61 +56,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 05/24] arm64: compat: Alloc separate pages for vectors and sigpage
2018-12-13 16:57 ` [PATCH v3 05/24] arm64: compat: Alloc separate pages for vectors and sigpage Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
This patch addresses the problem creating separate pages for vectors and
sigpage in a similar fashion to what happens today on arm.
This patch changes as well the meaning of mm->context.vdso for AArch32
compat since it now points to sigpage and not to vectors anymore in
order to make simpler the implementation of the signal handling (the
address of sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 433b9554c6a1..f2b6b486deb7 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -210,10 +210,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 6b0d4dff5012..6d4070cb813c 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -67,9 +67,9 @@
#define STACK_TOP_MAX TASK_SIZE_64
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 24b09003f821..265a6fab331d 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index c91039b94f28..5df7aeef324a 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -54,61 +56,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 06/24] arm64: compat: Split kuser32
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (5 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 05/24] arm64: compat: Alloc separate pages for vectors and sigpage Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 07/24] arm64: compat: Refactor aarch32_alloc_vdso_pages() Vincenzo Frascino
` (17 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
This patch splits the current version of kuser32 in kuser32 (for kuser
helpers) and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 4c8b13bede80..6bd4f619732a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 06/24] arm64: compat: Split kuser32
2018-12-13 16:57 ` [PATCH v3 06/24] arm64: compat: Split kuser32 Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
This patch splits the current version of kuser32 in kuser32 (for kuser
helpers) and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 4c8b13bede80..6bd4f619732a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 07/24] arm64: compat: Refactor aarch32_alloc_vdso_pages()
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (6 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 06/24] arm64: compat: Split kuser32 Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 08/24] arm64: compat: Add KUSER_HELPERS config option Vincenzo Frascino
` (16 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
This patch divides the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 5df7aeef324a..d438e65299a2 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,40 +75,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 07/24] arm64: compat: Refactor aarch32_alloc_vdso_pages()
2018-12-13 16:57 ` [PATCH v3 07/24] arm64: compat: Refactor aarch32_alloc_vdso_pages() Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
This patch divides the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 5df7aeef324a..d438e65299a2 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,40 +75,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 08/24] arm64: compat: Add KUSER_HELPERS config option
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (7 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 07/24] arm64: compat: Refactor aarch32_alloc_vdso_pages() Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 09/24] arm64: compat: Add missing syscall numbers Vincenzo Frascino
` (15 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
This patch adds a KUSER_HELPERS config option that can be used to
disable the mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 985f76109531..aafa18e2e7c4 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1358,6 +1358,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 6bd4f619732a..2fcba6b87829 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index d438e65299a2..7ee3d21350ec 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,6 +75,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -96,6 +97,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -127,6 +134,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -139,6 +147,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 08/24] arm64: compat: Add KUSER_HELPERS config option
2018-12-13 16:57 ` [PATCH v3 08/24] arm64: compat: Add KUSER_HELPERS config option Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
This patch adds a KUSER_HELPERS config option that can be used to
disable the mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 985f76109531..aafa18e2e7c4 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1358,6 +1358,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 6bd4f619732a..2fcba6b87829 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index d438e65299a2..7ee3d21350ec 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,6 +75,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -96,6 +97,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -127,6 +134,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -139,6 +147,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 09/24] arm64: compat: Add missing syscall numbers
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (8 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 08/24] arm64: compat: Add KUSER_HELPERS config option Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 10/24] arm64: compat: Expose signal related structures Vincenzo Frascino
` (14 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
This patch adds the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index b13ca091f833..a85b30b766a1 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 09/24] arm64: compat: Add missing syscall numbers
2018-12-13 16:57 ` [PATCH v3 09/24] arm64: compat: Add missing syscall numbers Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
This patch adds the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index b13ca091f833..a85b30b766a1 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 10/24] arm64: compat: Expose signal related structures
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (9 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 09/24] arm64: compat: Add missing syscall numbers Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 11/24] arm64: compat: Generate asm offsets for signals Vincenzo Frascino
` (13 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
This patch exposes the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 265a6fab331d..16c490a83131 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 10/24] arm64: compat: Expose signal related structures
2018-12-13 16:57 ` [PATCH v3 10/24] arm64: compat: Expose signal related structures Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
This patch exposes the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 265a6fab331d..16c490a83131 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 11/24] arm64: compat: Generate asm offsets for signals
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (10 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 10/24] arm64: compat: Expose signal related structures Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 12/24] lib: vdso: Add compat support Vincenzo Frascino
` (12 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch updates asm-offsets for arm64 to generate the correct offsets
for compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index a716ff952a57..e762e656e176 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -81,6 +82,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 11/24] arm64: compat: Generate asm offsets for signals
2018-12-13 16:57 ` [PATCH v3 11/24] arm64: compat: Generate asm offsets for signals Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch updates asm-offsets for arm64 to generate the correct offsets
for compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index a716ff952a57..e762e656e176 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -81,6 +82,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 12/24] lib: vdso: Add compat support
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (11 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 11/24] arm64: compat: Generate asm offsets for signals Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 13/24] arm64: compat: Add vDSO Vincenzo Frascino
` (11 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
This patch adds support to the generic code for compat fallback
functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 30010d25bf5b..e21bd58139c8 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -28,7 +28,11 @@
* - clock_gettime_fallback(): fallback for clock_gettime.
* - clock_getres_fallback(): fallback for clock_getres.
*/
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
#ifdef CONFIG_HAVE_HW_COUNTER
static notrace int do_hres(const struct vdso_data *vd,
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 12/24] lib: vdso: Add compat support
2018-12-13 16:57 ` [PATCH v3 12/24] lib: vdso: Add compat support Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
This patch adds support to the generic code for compat fallback
functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 30010d25bf5b..e21bd58139c8 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -28,7 +28,11 @@
* - clock_gettime_fallback(): fallback for clock_gettime.
* - clock_getres_fallback(): fallback for clock_getres.
*/
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
#ifdef CONFIG_HAVE_HW_COUNTER
static notrace int do_hres(const struct vdso_data *vd,
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 13/24] arm64: compat: Add vDSO
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (12 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 12/24] lib: vdso: Add compat support Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 14/24] arm64: Refactor vDSO code Vincenzo Frascino
` (10 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch provides the arm64 compat (AArch32) vDSO in kernel/vdso32 in
a similar way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 122 ++++++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 81 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++++
9 files changed, 562 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..62e3a90107b9
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+/*
+ * We use the hidden visibility to prevent the compiler from generating a GOT
+ * relocation. Not only is going through a GOT useless (the entry couldn't and
+ * musn't be overridden by another library), it does not even work: the linker
+ * cannot generate an absolute address to the data page.
+ *
+ * With the hidden visibility, the compiler simply generates a PC-relative
+ * relocation (R_ARM_REL32), and this is what we need.
+ */
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ /* The checks below are required for ABI consistency with arm */
+ if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
+ return -EINVAL;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ /*
+ * This simply puts &_vdso_data into ret. The reason why we don't use
+ * `ret = &_vdso_data` is that the compiler tends to optimise this in a
+ * very suboptimal way: instead of keeping &_vdso_data in a register,
+ * it goes through a relocation almost every time _vdso_data must be
+ * accessed (even in subfunctions). This is both time and space
+ * consuming: each relocation uses a word in the code section, and it
+ * has to be loaded at runtime.
+ *
+ * This trick hides the assignment from the compiler. Since it cannot
+ * track where the pointer comes from, it will only use one relocation
+ * where __arch_get_vdso_data() is called, and then keep the result in
+ * a register.
+ */
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..4fd3093ea78c
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..b470a8ed35a7
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 13/24] arm64: compat: Add vDSO
2018-12-13 16:57 ` [PATCH v3 13/24] arm64: compat: Add vDSO Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch provides the arm64 compat (AArch32) vDSO in kernel/vdso32 in
a similar way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 122 ++++++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 81 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++++
9 files changed, 562 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..62e3a90107b9
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,122 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+/*
+ * We use the hidden visibility to prevent the compiler from generating a GOT
+ * relocation. Not only is going through a GOT useless (the entry couldn't and
+ * musn't be overridden by another library), it does not even work: the linker
+ * cannot generate an absolute address to the data page.
+ *
+ * With the hidden visibility, the compiler simply generates a PC-relative
+ * relocation (R_ARM_REL32), and this is what we need.
+ */
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ /* The checks below are required for ABI consistency with arm */
+ if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
+ return -EINVAL;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ /*
+ * This simply puts &_vdso_data into ret. The reason why we don't use
+ * `ret = &_vdso_data` is that the compiler tends to optimise this in a
+ * very suboptimal way: instead of keeping &_vdso_data in a register,
+ * it goes through a relocation almost every time _vdso_data must be
+ * accessed (even in subfunctions). This is both time and space
+ * consuming: each relocation uses a word in the code section, and it
+ * has to be loaded at runtime.
+ *
+ * This trick hides the assignment from the compiler. Since it cannot
+ * track where the pointer comes from, it will only use one relocation
+ * where __arch_get_vdso_data() is called, and then keep the result in
+ * a register.
+ */
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..4fd3093ea78c
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..b470a8ed35a7
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 14/24] arm64: Refactor vDSO code
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (13 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 13/24] arm64: compat: Add vDSO Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 15/24] arm64: compat: vDSO setup for compat layer Vincenzo Frascino
` (9 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
This patch refactors vdso.c to simplify the implementation of arm64 vDSO
compat (which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 7ee3d21350ec..7d1ad20fdb7b 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,7 +41,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -52,6 +75,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 14/24] arm64: Refactor vDSO code
2018-12-13 16:57 ` [PATCH v3 14/24] arm64: Refactor vDSO code Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
This patch refactors vdso.c to simplify the implementation of arm64 vDSO
compat (which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 7ee3d21350ec..7d1ad20fdb7b 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,7 +41,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -52,6 +75,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 15/24] arm64: compat: vDSO setup for compat layer
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (14 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 14/24] arm64: Refactor vDSO code Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 16/24] arm64: elf: vDSO code page discovery Vincenzo Frascino
` (8 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
This patch adds the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 7d1ad20fdb7b..e3f78875b0d4 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,10 +41,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -64,6 +70,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -179,23 +192,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -227,6 +262,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -249,11 +293,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -278,6 +327,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -301,6 +351,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -314,7 +365,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 15/24] arm64: compat: vDSO setup for compat layer
2018-12-13 16:57 ` [PATCH v3 15/24] arm64: compat: vDSO setup for compat layer Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
This patch adds the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 7d1ad20fdb7b..e3f78875b0d4 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,10 +41,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -64,6 +70,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -179,23 +192,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -227,6 +262,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -249,11 +293,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -278,6 +327,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -301,6 +351,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -314,7 +365,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 16/24] arm64: elf: vDSO code page discovery
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (15 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 15/24] arm64: compat: vDSO setup for compat layer Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 17/24] arm64: compat: Get sigreturn trampolines from vDSO Vincenzo Frascino
` (7 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
This patch adds the necessary code to the elf arm64 module to make this
possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index f2b6b486deb7..3fb46968da6a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -209,7 +209,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 16/24] arm64: elf: vDSO code page discovery
2018-12-13 16:57 ` [PATCH v3 16/24] arm64: elf: vDSO code page discovery Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
This patch adds the necessary code to the elf arm64 module to make this
possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index f2b6b486deb7..3fb46968da6a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -209,7 +209,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 17/24] arm64: compat: Get sigreturn trampolines from vDSO
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (16 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 16/24] arm64: elf: vDSO code page discovery Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 18/24] arm64: Add vDSO compat support Vincenzo Frascino
` (6 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
This patch adds the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 16c490a83131..be471a81084e 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 17/24] arm64: compat: Get sigreturn trampolines from vDSO
2018-12-13 16:57 ` [PATCH v3 17/24] arm64: compat: Get sigreturn trampolines from vDSO Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
This patch adds the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 16c490a83131..be471a81084e 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 18/24] arm64: Add vDSO compat support
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (17 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 17/24] arm64: compat: Get sigreturn trampolines from vDSO Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-17 15:26 ` Mark Salyzyn
2018-12-13 16:57 ` [PATCH v3 19/24] arm64: Enable compat vDSO support Vincenzo Frascino
` (5 subsequent siblings)
24 siblings, 2 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch adds vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 21 +++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index aafa18e2e7c4..ed9c36f0ccb9 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,6 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6cb9fc7e9382..814d2b0ac017 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -144,6 +158,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 2fcba6b87829..29234a70d168 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+arm64-obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
@@ -60,6 +63,7 @@ arm64-obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o
arm64-obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-y += $(arm64-obj-y) vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
obj-m += $(arm64-obj-m)
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 18/24] arm64: Add vDSO compat support
2018-12-13 16:57 ` [PATCH v3 18/24] arm64: Add vDSO compat support Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-17 15:26 ` Mark Salyzyn
1 sibling, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch adds vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 21 +++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index aafa18e2e7c4..ed9c36f0ccb9 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,6 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6cb9fc7e9382..814d2b0ac017 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -144,6 +158,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 2fcba6b87829..29234a70d168 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+arm64-obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
@@ -60,6 +63,7 @@ arm64-obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o
arm64-obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-y += $(arm64-obj-y) vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
obj-m += $(arm64-obj-m)
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 19/24] arm64: Enable compat vDSO support
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (18 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 18/24] arm64: Add vDSO compat support Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 20/24] arm: Add support for generic vDSO Vincenzo Frascino
` (4 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch enables compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c9a57d11330b..cfd5d67c6407 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -709,6 +709,7 @@ CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_PRINTK_TIME=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_FS=y
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 19/24] arm64: Enable compat vDSO support
2018-12-13 16:57 ` [PATCH v3 19/24] arm64: Enable compat vDSO support Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch enables compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c9a57d11330b..cfd5d67c6407 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -709,6 +709,7 @@ CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_PRINTK_TIME=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_FS=y
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 20/24] arm: Add support for generic vDSO
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (19 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 19/24] arm64: Enable compat vDSO support Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 21/24] mips: Introduce vdso_direct Vincenzo Frascino
` (3 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 85 ++++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 100 ++++++---
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 ++---------------------
8 files changed, 201 insertions(+), 309 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 91be74d8df65..fdf5dc492669 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -47,6 +47,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
@@ -96,6 +98,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
@@ -1353,6 +1356,7 @@ config HAVE_ARM_ARCH_TIMER
bool "Architected timer support"
depends on CPU_V7
select ARM_ARCH_TIMER
+ select HAVE_HW_COUNTER
select GENERIC_CLOCKEVENTS
help
This option enables support for the ARM architected timer
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..d36cb229fa45
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(int clock_mode)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..bc12bee9ed3d 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -34,6 +34,7 @@
#include <asm/vdso.h>
#include <asm/vdso_datapage.h>
#include <clocksource/arm_arch_timer.h>
+#include <vdso/helpers.h>
#define MAX_SYMNAME 64
@@ -274,18 +275,6 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
mm->context.vdso = addr;
}
-static void vdso_write_begin(struct vdso_data *vdata)
-{
- ++vdso_data->seq_count;
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
-}
-
-static void vdso_write_end(struct vdso_data *vdata)
-{
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
-}
-
static bool tk_is_cntvct(const struct timekeeper *tk)
{
if (!IS_ENABLED(CONFIG_ARM_ARCH_TIMER))
@@ -307,17 +296,19 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
*/
void update_vsyscall(struct timekeeper *tk)
{
- struct timespec64 *wtm = &tk->wall_to_monotonic;
+ struct vdso_timestamp *vdso_ts;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
+ u64 nsec;
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +319,69 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift);
- vdso_data->wtm_clock_sec = wtm->tv_sec;
- vdso_data->wtm_clock_nsec = wtm->tv_nsec;
-
- if (vdso_data->tk_is_cntvct) {
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
- vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ vdso_data->use_syscall = !__tk_is_cntvct;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (__tk_is_cntvct) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->mask = tk->tkr_mono.mask;
+ vdso_data->mult = tk->tkr_mono.mult;
+ vdso_data->shift = tk->tkr_mono.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1d81e8c3acf6 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..b470a8ed35a7 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
+#include <linux/types.h>
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
+ return __cvdso_clock_gettime(clock, ts);
}
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
-
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
-
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 20/24] arm: Add support for generic vDSO
2018-12-13 16:57 ` [PATCH v3 20/24] arm: Add support for generic vDSO Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 85 ++++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 100 ++++++---
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 ++---------------------
8 files changed, 201 insertions(+), 309 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 91be74d8df65..fdf5dc492669 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -47,6 +47,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
@@ -96,6 +98,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
@@ -1353,6 +1356,7 @@ config HAVE_ARM_ARCH_TIMER
bool "Architected timer support"
depends on CPU_V7
select ARM_ARCH_TIMER
+ select HAVE_HW_COUNTER
select GENERIC_CLOCKEVENTS
help
This option enables support for the ARM architected timer
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..d36cb229fa45
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(int clock_mode)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..bc12bee9ed3d 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -34,6 +34,7 @@
#include <asm/vdso.h>
#include <asm/vdso_datapage.h>
#include <clocksource/arm_arch_timer.h>
+#include <vdso/helpers.h>
#define MAX_SYMNAME 64
@@ -274,18 +275,6 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
mm->context.vdso = addr;
}
-static void vdso_write_begin(struct vdso_data *vdata)
-{
- ++vdso_data->seq_count;
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
-}
-
-static void vdso_write_end(struct vdso_data *vdata)
-{
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
-}
-
static bool tk_is_cntvct(const struct timekeeper *tk)
{
if (!IS_ENABLED(CONFIG_ARM_ARCH_TIMER))
@@ -307,17 +296,19 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
*/
void update_vsyscall(struct timekeeper *tk)
{
- struct timespec64 *wtm = &tk->wall_to_monotonic;
+ struct vdso_timestamp *vdso_ts;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
+ u64 nsec;
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +319,69 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift);
- vdso_data->wtm_clock_sec = wtm->tv_sec;
- vdso_data->wtm_clock_nsec = wtm->tv_nsec;
-
- if (vdso_data->tk_is_cntvct) {
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
- vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ vdso_data->use_syscall = !__tk_is_cntvct;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (__tk_is_cntvct) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->mask = tk->tkr_mono.mask;
+ vdso_data->mult = tk->tkr_mono.mult;
+ vdso_data->shift = tk->tkr_mono.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1d81e8c3acf6 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..b470a8ed35a7 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
+#include <linux/types.h>
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
+ return __cvdso_clock_gettime(clock, ts);
}
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
-
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
-
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 21/24] mips: Introduce vdso_direct
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (20 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 20/24] arm: Add support for generic vDSO Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 22/24] clock: csrc-4k: Add support for vdso_direct Vincenzo Frascino
` (2 subsequent siblings)
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
vdso_direct is an architectural clock configuration parameter that
exposes the usage of not of the vDSO.
If vdso_direct is false, the time related syscalls will be used instead
of the vDSO library.
This patch introduces the parameter in the mips architecture to prepare
the ground for the support of the generic vDSO library.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/include/asm/clocksource.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/mips/include/asm/clocksource.h b/arch/mips/include/asm/clocksource.h
index 3deb1d0c1a94..eab11ee4871a 100644
--- a/arch/mips/include/asm/clocksource.h
+++ b/arch/mips/include/asm/clocksource.h
@@ -23,6 +23,7 @@
* @vdso_clock_mode: Method the VDSO should use to access the clocksource.
*/
struct arch_clocksource_data {
+ bool vdso_direct;
u8 vdso_clock_mode;
};
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 21/24] mips: Introduce vdso_direct
2018-12-13 16:57 ` [PATCH v3 21/24] mips: Introduce vdso_direct Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
vdso_direct is an architectural clock configuration parameter that
exposes the usage of not of the vDSO.
If vdso_direct is false, the time related syscalls will be used instead
of the vDSO library.
This patch introduces the parameter in the mips architecture to prepare
the ground for the support of the generic vDSO library.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/include/asm/clocksource.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/mips/include/asm/clocksource.h b/arch/mips/include/asm/clocksource.h
index 3deb1d0c1a94..eab11ee4871a 100644
--- a/arch/mips/include/asm/clocksource.h
+++ b/arch/mips/include/asm/clocksource.h
@@ -23,6 +23,7 @@
* @vdso_clock_mode: Method the VDSO should use to access the clocksource.
*/
struct arch_clocksource_data {
+ bool vdso_direct;
u8 vdso_clock_mode;
};
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 22/24] clock: csrc-4k: Add support for vdso_direct
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (21 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 21/24] mips: Introduce vdso_direct Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 23/24] clock: gic-timer: " Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 24/24] mips: Add support for generic vDSO Vincenzo Frascino
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on mips
architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/kernel/csrc-r4k.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/mips/kernel/csrc-r4k.c b/arch/mips/kernel/csrc-r4k.c
index eed099f35bf1..97e2a8aafdd6 100644
--- a/arch/mips/kernel/csrc-r4k.c
+++ b/arch/mips/kernel/csrc-r4k.c
@@ -77,8 +77,10 @@ int __init init_r4k_clocksource(void)
* R2 onwards makes the count accessible to user mode so it can be used
* by the VDSO (HWREna is configured by configure_hwrena()).
*/
- if (cpu_has_mips_r2_r6 && rdhwr_count_usable())
+ if (cpu_has_mips_r2_r6 && rdhwr_count_usable()) {
+ clocksource_mips.archdata.vdso_direct = true;
clocksource_mips.archdata.vdso_clock_mode = VDSO_CLOCK_R4K;
+ }
clocksource_register_hz(&clocksource_mips, mips_hpt_frequency);
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 22/24] clock: csrc-4k: Add support for vdso_direct
2018-12-13 16:57 ` [PATCH v3 22/24] clock: csrc-4k: Add support for vdso_direct Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on mips
architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/kernel/csrc-r4k.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/mips/kernel/csrc-r4k.c b/arch/mips/kernel/csrc-r4k.c
index eed099f35bf1..97e2a8aafdd6 100644
--- a/arch/mips/kernel/csrc-r4k.c
+++ b/arch/mips/kernel/csrc-r4k.c
@@ -77,8 +77,10 @@ int __init init_r4k_clocksource(void)
* R2 onwards makes the count accessible to user mode so it can be used
* by the VDSO (HWREna is configured by configure_hwrena()).
*/
- if (cpu_has_mips_r2_r6 && rdhwr_count_usable())
+ if (cpu_has_mips_r2_r6 && rdhwr_count_usable()) {
+ clocksource_mips.archdata.vdso_direct = true;
clocksource_mips.archdata.vdso_clock_mode = VDSO_CLOCK_R4K;
+ }
clocksource_register_hz(&clocksource_mips, mips_hpt_frequency);
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 23/24] clock: gic-timer: Add support for vdso_direct
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (22 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 22/24] clock: csrc-4k: Add support for vdso_direct Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 24/24] mips: Add support for generic vDSO Vincenzo Frascino
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on
mips architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
drivers/clocksource/mips-gic-timer.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c
index 54f8a331b53a..e7e0789084cd 100644
--- a/drivers/clocksource/mips-gic-timer.c
+++ b/drivers/clocksource/mips-gic-timer.c
@@ -158,7 +158,10 @@ static struct clocksource gic_clocksource = {
.name = "GIC",
.read = gic_hpt_read,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
- .archdata = { .vdso_clock_mode = VDSO_CLOCK_GIC },
+ .archdata = {
+ .vdso_direct = true,
+ .vdso_clock_mode = VDSO_CLOCK_GIC
+ },
};
static int __init __gic_clocksource_init(void)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 23/24] clock: gic-timer: Add support for vdso_direct
2018-12-13 16:57 ` [PATCH v3 23/24] clock: gic-timer: " Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on
mips architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
drivers/clocksource/mips-gic-timer.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c
index 54f8a331b53a..e7e0789084cd 100644
--- a/drivers/clocksource/mips-gic-timer.c
+++ b/drivers/clocksource/mips-gic-timer.c
@@ -158,7 +158,10 @@ static struct clocksource gic_clocksource = {
.name = "GIC",
.read = gic_hpt_read,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
- .archdata = { .vdso_clock_mode = VDSO_CLOCK_GIC },
+ .archdata = {
+ .vdso_direct = true,
+ .vdso_clock_mode = VDSO_CLOCK_GIC
+ },
};
static int __init __gic_clocksource_init(void)
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 24/24] mips: Add support for generic vDSO
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
` (23 preceding siblings ...)
2018-12-13 16:57 ` [PATCH v3 23/24] clock: gic-timer: " Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
24 siblings, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
The mips vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 3 +
arch/mips/include/asm/vdso.h | 60 +----
arch/mips/include/asm/vdso/gettimeofday.h | 168 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 103 ++++++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +++
11 files changed, 313 insertions(+), 356 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8272ea4c7264..a665b774a988 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -32,6 +32,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -70,6 +71,8 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
+ select HAVE_HW_COUNTER
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES && 64BIT
select MODULES_USE_ELF_REL if MODULES
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..0b1d0b4578a6 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,8 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
-{
- ++data->seq_count;
-
- /* Ensure sequence update is written before other data page values. */
- smp_wmb();
-}
-
-static inline void vdso_data_write_end(union mips_vdso_data *data)
-{
- /* Ensure data values are written before updating sequence again. */
- smp_wmb();
- ++data->seq_count;
-}
-
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e3ceabd2befc
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,168 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct __vdso_timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 48a9c6b90e07..8b1c9ad0989a 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -24,9 +24,11 @@
#include <asm/mips-cps.h>
#include <asm/page.h>
#include <asm/vdso.h>
+#include <vdso/helpers.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +74,90 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ struct vdso_timestamp *vdso_ts;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+#ifdef DISABLE_MIPS_VDSO
+ vdso_data->use_syscall = true;
+#else
+ vdso_data->use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
+#endif /* DISABLE_MIPS_VDSO */
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->mask = tk->tkr_mono.mask;
+ vdso_data->mult = tk->tkr_mono.mult;
+ vdso_data->shift = tk->tkr_mono.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- vdso_data_write_end(&vdso_data);
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +230,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +256,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 58a0315ad743..e38c9a6571a1 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -14,15 +20,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -49,6 +63,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
#
@@ -96,6 +112,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -134,6 +151,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -174,6 +192,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..90b2c2785e32 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..8da361d919f0
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* [PATCH v3 24/24] mips: Add support for generic vDSO
2018-12-13 16:57 ` [PATCH v3 24/24] mips: Add support for generic vDSO Vincenzo Frascino
@ 2018-12-13 16:57 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2018-12-13 16:57 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
The mips vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 3 +
arch/mips/include/asm/vdso.h | 60 +----
arch/mips/include/asm/vdso/gettimeofday.h | 168 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 103 ++++++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +++
11 files changed, 313 insertions(+), 356 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8272ea4c7264..a665b774a988 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -32,6 +32,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -70,6 +71,8 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
+ select HAVE_HW_COUNTER
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES && 64BIT
select MODULES_USE_ELF_REL if MODULES
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..0b1d0b4578a6 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,8 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
-{
- ++data->seq_count;
-
- /* Ensure sequence update is written before other data page values. */
- smp_wmb();
-}
-
-static inline void vdso_data_write_end(union mips_vdso_data *data)
-{
- /* Ensure data values are written before updating sequence again. */
- smp_wmb();
- ++data->seq_count;
-}
-
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e3ceabd2befc
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,168 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#define __arch_get_realtime_res(x) (MONOTONIC_RES_NSEC)
+#define __arch_get_coarse_res(x) (LOW_RES_NSEC)
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct __vdso_timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 48a9c6b90e07..8b1c9ad0989a 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -24,9 +24,11 @@
#include <asm/mips-cps.h>
#include <asm/page.h>
#include <asm/vdso.h>
+#include <vdso/helpers.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +74,90 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ struct vdso_timestamp *vdso_ts;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+#ifdef DISABLE_MIPS_VDSO
+ vdso_data->use_syscall = true;
+#else
+ vdso_data->use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
+#endif /* DISABLE_MIPS_VDSO */
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->mask = tk->tkr_mono.mask;
+ vdso_data->mult = tk->tkr_mono.mult;
+ vdso_data->shift = tk->tkr_mono.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- vdso_data_write_end(&vdso_data);
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +230,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +256,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 58a0315ad743..e38c9a6571a1 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -14,15 +20,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -49,6 +63,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
#
@@ -96,6 +112,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -134,6 +151,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -174,6 +192,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..90b2c2785e32 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..8da361d919f0
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.2
^ permalink raw reply related [flat|nested] 58+ messages in thread
* Re: [PATCH v3 02/24] kernel: Define gettimeofday vdso common code
2018-12-13 16:57 ` [PATCH v3 02/24] kernel: Define gettimeofday vdso common code Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
@ 2018-12-14 18:15 ` Mark Salyzyn
2018-12-14 18:15 ` Mark Salyzyn
2019-01-10 10:36 ` Vincenzo Frascino
1 sibling, 2 replies; 58+ messages in thread
From: Mark Salyzyn @ 2018-12-14 18:15 UTC (permalink / raw)
To: Vincenzo Frascino, linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Thomas Gleixner,
Peter Collingbourne
On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> . . .
> +}
> +
> +#ifdef VDSO_HAS_TIME
> +static notrace time_t __cvdso_time(time_t *time)
> +{
> + u32 seq;
> + time_t t;
> + const struct vdso_data *vd = __arch_get_vdso_data();
> + struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
const struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
-- Mark
^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: [PATCH v3 02/24] kernel: Define gettimeofday vdso common code
2018-12-14 18:15 ` Mark Salyzyn
@ 2018-12-14 18:15 ` Mark Salyzyn
2019-01-10 10:36 ` Vincenzo Frascino
1 sibling, 0 replies; 58+ messages in thread
From: Mark Salyzyn @ 2018-12-14 18:15 UTC (permalink / raw)
To: Vincenzo Frascino, linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Peter Collingbourne
On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> . . .
> +}
> +
> +#ifdef VDSO_HAS_TIME
> +static notrace time_t __cvdso_time(time_t *time)
> +{
> + u32 seq;
> + time_t t;
> + const struct vdso_data *vd = __arch_get_vdso_data();
> + struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
const struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
-- Mark
^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: [PATCH v3 18/24] arm64: Add vDSO compat support
2018-12-13 16:57 ` [PATCH v3 18/24] arm64: Add vDSO compat support Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
@ 2018-12-17 15:26 ` Mark Salyzyn
2018-12-17 15:26 ` Mark Salyzyn
2019-01-10 10:39 ` Vincenzo Frascino
1 sibling, 2 replies; 58+ messages in thread
From: Mark Salyzyn @ 2018-12-17 15:26 UTC (permalink / raw)
To: Vincenzo Frascino, linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Thomas Gleixner,
Peter Collingbourne
On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
> This patch adds vDSO compat support to the arm64 building system.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/Kconfig | 1 +
> arch/arm64/Makefile | 21 +++++++++++++++++++--
> arch/arm64/kernel/Makefile | 6 +++++-
> 3 files changed, 25 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index aafa18e2e7c4..ed9c36f0ccb9 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -103,6 +103,7 @@ config ARM64
> select GENERIC_STRNLEN_USER
> select GENERIC_TIME_VSYSCALL
> select GENERIC_GETTIMEOFDAY
> + select GENERIC_COMPAT_VDSO
> select HANDLE_DOMAIN_IRQ
> select HARDIRQS_SW_RESEND
> select HAVE_ACPI_APEI if (ACPI && EFI)
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 6cb9fc7e9382..814d2b0ac017 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
> endif
> endif
>
> -KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
> +ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
> + CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
> +
> + ifeq ($(CROSS_COMPILE_ARM32),)
> + $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
> + else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
> + $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
> + else
> + export CROSS_COMPILE_ARM32
> + export CONFIG_COMPAT_VDSO := y
> + compat_vdso := -DCONFIG_COMPAT_VDSO=1
> + endif
> +endif
> +
> +KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
> KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
> -KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
> +KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
>
> KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
> KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
Does not build with clang
Do not hard-code gcc, use $(cc-name)
-- Mark
^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: [PATCH v3 18/24] arm64: Add vDSO compat support
2018-12-17 15:26 ` Mark Salyzyn
@ 2018-12-17 15:26 ` Mark Salyzyn
2019-01-10 10:39 ` Vincenzo Frascino
1 sibling, 0 replies; 58+ messages in thread
From: Mark Salyzyn @ 2018-12-17 15:26 UTC (permalink / raw)
To: Vincenzo Frascino, linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Peter Collingbourne
On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
> This patch adds vDSO compat support to the arm64 building system.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/Kconfig | 1 +
> arch/arm64/Makefile | 21 +++++++++++++++++++--
> arch/arm64/kernel/Makefile | 6 +++++-
> 3 files changed, 25 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index aafa18e2e7c4..ed9c36f0ccb9 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -103,6 +103,7 @@ config ARM64
> select GENERIC_STRNLEN_USER
> select GENERIC_TIME_VSYSCALL
> select GENERIC_GETTIMEOFDAY
> + select GENERIC_COMPAT_VDSO
> select HANDLE_DOMAIN_IRQ
> select HARDIRQS_SW_RESEND
> select HAVE_ACPI_APEI if (ACPI && EFI)
> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
> index 6cb9fc7e9382..814d2b0ac017 100644
> --- a/arch/arm64/Makefile
> +++ b/arch/arm64/Makefile
> @@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
> endif
> endif
>
> -KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
> +ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
> + CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
> +
> + ifeq ($(CROSS_COMPILE_ARM32),)
> + $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
> + else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
> + $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
> + else
> + export CROSS_COMPILE_ARM32
> + export CONFIG_COMPAT_VDSO := y
> + compat_vdso := -DCONFIG_COMPAT_VDSO=1
> + endif
> +endif
> +
> +KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
> KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
> -KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
> +KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
>
> KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
> KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
Does not build with clang
Do not hard-code gcc, use $(cc-name)
-- Mark
^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: [PATCH v3 02/24] kernel: Define gettimeofday vdso common code
2018-12-14 18:15 ` Mark Salyzyn
2018-12-14 18:15 ` Mark Salyzyn
@ 2019-01-10 10:36 ` Vincenzo Frascino
2019-01-10 10:36 ` Vincenzo Frascino
1 sibling, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2019-01-10 10:36 UTC (permalink / raw)
To: Mark Salyzyn, linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Thomas Gleixner,
Peter Collingbourne
On 14/12/2018 18:15, Mark Salyzyn wrote:
> On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
>> In the last few years we assisted to an explosion of vdso
>> implementations that mostly share similar code.
>>
>> . . .
>> +}
>> +
>> +#ifdef VDSO_HAS_TIME
>> +static notrace time_t __cvdso_time(time_t *time)
>> +{
>> + u32 seq;
>> + time_t t;
>> + const struct vdso_data *vd = __arch_get_vdso_data();
>> + struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
> const struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
>
Thanks for this, I will fix in v4.
> -- Mark
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: [PATCH v3 02/24] kernel: Define gettimeofday vdso common code
2019-01-10 10:36 ` Vincenzo Frascino
@ 2019-01-10 10:36 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2019-01-10 10:36 UTC (permalink / raw)
To: Mark Salyzyn, linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Peter Collingbourne
On 14/12/2018 18:15, Mark Salyzyn wrote:
> On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
>> In the last few years we assisted to an explosion of vdso
>> implementations that mostly share similar code.
>>
>> . . .
>> +}
>> +
>> +#ifdef VDSO_HAS_TIME
>> +static notrace time_t __cvdso_time(time_t *time)
>> +{
>> + u32 seq;
>> + time_t t;
>> + const struct vdso_data *vd = __arch_get_vdso_data();
>> + struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
> const struct vdso_timestamp *vdso_ts = &vd->basetime[CLOCK_REALTIME];
>
Thanks for this, I will fix in v4.
> -- Mark
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: [PATCH v3 18/24] arm64: Add vDSO compat support
2018-12-17 15:26 ` Mark Salyzyn
2018-12-17 15:26 ` Mark Salyzyn
@ 2019-01-10 10:39 ` Vincenzo Frascino
2019-01-10 10:39 ` Vincenzo Frascino
1 sibling, 1 reply; 58+ messages in thread
From: Vincenzo Frascino @ 2019-01-10 10:39 UTC (permalink / raw)
To: Mark Salyzyn, linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Thomas Gleixner,
Peter Collingbourne
On 17/12/2018 15:26, Mark Salyzyn wrote:
> On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
>> This patch adds vDSO compat support to the arm64 building system.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>> ---
>> arch/arm64/Kconfig | 1 +
>> arch/arm64/Makefile | 21 +++++++++++++++++++--
>> arch/arm64/kernel/Makefile | 6 +++++-
>> 3 files changed, 25 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index aafa18e2e7c4..ed9c36f0ccb9 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -103,6 +103,7 @@ config ARM64
>> select GENERIC_STRNLEN_USER
>> select GENERIC_TIME_VSYSCALL
>> select GENERIC_GETTIMEOFDAY
>> + select GENERIC_COMPAT_VDSO
>> select HANDLE_DOMAIN_IRQ
>> select HARDIRQS_SW_RESEND
>> select HAVE_ACPI_APEI if (ACPI && EFI)
>> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
>> index 6cb9fc7e9382..814d2b0ac017 100644
>> --- a/arch/arm64/Makefile
>> +++ b/arch/arm64/Makefile
>> @@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
>> endif
>> endif
>>
>> -KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
>> +ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
>> + CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
>> +
>> + ifeq ($(CROSS_COMPILE_ARM32),)
>> + $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
>> + else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
>> + $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
>> + else
>> + export CROSS_COMPILE_ARM32
>> + export CONFIG_COMPAT_VDSO := y
>> + compat_vdso := -DCONFIG_COMPAT_VDSO=1
>> + endif
>> +endif
>> +
>> +KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
>> KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
>> -KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
>> +KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
>>
>> KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
>> KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
> Does not build with clang
>
> Do not hard-code gcc, use $(cc-name)
>
$(cc-name) does not seem a supported function: Documentation/kbuild/makefiles.txt
The way of enabling clang seems via CONFIG_CC_IS_CLANG.
> -- Mark
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 58+ messages in thread
* Re: [PATCH v3 18/24] arm64: Add vDSO compat support
2019-01-10 10:39 ` Vincenzo Frascino
@ 2019-01-10 10:39 ` Vincenzo Frascino
0 siblings, 0 replies; 58+ messages in thread
From: Vincenzo Frascino @ 2019-01-10 10:39 UTC (permalink / raw)
To: Mark Salyzyn, linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Peter Collingbourne
On 17/12/2018 15:26, Mark Salyzyn wrote:
> On 12/13/2018 08:57 AM, Vincenzo Frascino wrote:
>> This patch adds vDSO compat support to the arm64 building system.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>> ---
>> arch/arm64/Kconfig | 1 +
>> arch/arm64/Makefile | 21 +++++++++++++++++++--
>> arch/arm64/kernel/Makefile | 6 +++++-
>> 3 files changed, 25 insertions(+), 3 deletions(-)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index aafa18e2e7c4..ed9c36f0ccb9 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -103,6 +103,7 @@ config ARM64
>> select GENERIC_STRNLEN_USER
>> select GENERIC_TIME_VSYSCALL
>> select GENERIC_GETTIMEOFDAY
>> + select GENERIC_COMPAT_VDSO
>> select HANDLE_DOMAIN_IRQ
>> select HARDIRQS_SW_RESEND
>> select HAVE_ACPI_APEI if (ACPI && EFI)
>> diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
>> index 6cb9fc7e9382..814d2b0ac017 100644
>> --- a/arch/arm64/Makefile
>> +++ b/arch/arm64/Makefile
>> @@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
>> endif
>> endif
>>
>> -KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
>> +ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
>> + CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
>> +
>> + ifeq ($(CROSS_COMPILE_ARM32),)
>> + $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
>> + else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
>> + $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
>> + else
>> + export CROSS_COMPILE_ARM32
>> + export CONFIG_COMPAT_VDSO := y
>> + compat_vdso := -DCONFIG_COMPAT_VDSO=1
>> + endif
>> +endif
>> +
>> +KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
>> KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
>> -KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
>> +KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
>>
>> KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
>> KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
> Does not build with clang
>
> Do not hard-code gcc, use $(cc-name)
>
$(cc-name) does not seem a supported function: Documentation/kbuild/makefiles.txt
The way of enabling clang seems via CONFIG_CC_IS_CLANG.
> -- Mark
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 58+ messages in thread
end of thread, other threads:[~2019-01-10 10:39 UTC | newest]
Thread overview: 58+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-12-13 16:57 [PATCH v3 00/24] Unify vDSOs across more architectures Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 01/24] kernel: Standardize vdso_datapage Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 02/24] kernel: Define gettimeofday vdso common code Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-14 18:15 ` Mark Salyzyn
2018-12-14 18:15 ` Mark Salyzyn
2019-01-10 10:36 ` Vincenzo Frascino
2019-01-10 10:36 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 03/24] arm64: Build vDSO with -ffixed-x18 Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 04/24] arm64: Substitute gettimeofday with C implementation Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 05/24] arm64: compat: Alloc separate pages for vectors and sigpage Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 06/24] arm64: compat: Split kuser32 Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 07/24] arm64: compat: Refactor aarch32_alloc_vdso_pages() Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 08/24] arm64: compat: Add KUSER_HELPERS config option Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 09/24] arm64: compat: Add missing syscall numbers Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 10/24] arm64: compat: Expose signal related structures Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 11/24] arm64: compat: Generate asm offsets for signals Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 12/24] lib: vdso: Add compat support Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 13/24] arm64: compat: Add vDSO Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 14/24] arm64: Refactor vDSO code Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 15/24] arm64: compat: vDSO setup for compat layer Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 16/24] arm64: elf: vDSO code page discovery Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 17/24] arm64: compat: Get sigreturn trampolines from vDSO Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 18/24] arm64: Add vDSO compat support Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-17 15:26 ` Mark Salyzyn
2018-12-17 15:26 ` Mark Salyzyn
2019-01-10 10:39 ` Vincenzo Frascino
2019-01-10 10:39 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 19/24] arm64: Enable compat vDSO support Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 20/24] arm: Add support for generic vDSO Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 21/24] mips: Introduce vdso_direct Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 22/24] clock: csrc-4k: Add support for vdso_direct Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 23/24] clock: gic-timer: " Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
2018-12-13 16:57 ` [PATCH v3 24/24] mips: Add support for generic vDSO Vincenzo Frascino
2018-12-13 16:57 ` Vincenzo Frascino
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).