* [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.
Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.
The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in
"asm/vdso/".
* The generic implementation includes the arch specific one and lives
in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place
with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm and mips.
The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].
Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].
To simplify the testing, a copy of the patchset on top of a recent linux tree
can be found at [3].
[1] https://github.com/nlynch-mentor/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v1
[4] git://linux-arm.org/linux-vf.git vdso-mips/v1
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Vincenzo Frascino (27):
kernel: Standardize vdso_datapage
kernel: Add Monotonic boot time support
kernel: Add International Atomic Time support
kernel: Add masks support for Raw and NTP time
kernel: Add clock_mode support
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Introduce vdso_direct
clock: csrc-4k: Add support for vdso_direct
clock: gic-timer: Add support for vdso_direct
mips: Add support for generic vDSO
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 79 ++++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 28 +-
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 2 +
arch/arm/vdso/vgettimeofday.c | 263 +----------
arch/arm64/Kconfig | 25 ++
arch/arm64/Makefile | 21 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 ++-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 89 ++++
arch/arm64/include/asm/vdso/gettimeofday.h | 82 ++++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 13 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 ++--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 413 ++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 24 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 --------------
arch/arm64/kernel/vdso/vdso.lds.S | 1 +
arch/arm64/kernel/vdso/vgettimeofday.c | 30 ++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 82 ++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 43 ++
arch/mips/Kconfig | 3 +
arch/mips/include/asm/clocksource.h | 1 +
arch/mips/include/asm/vdso.h | 53 +--
arch/mips/include/asm/vdso/gettimeofday.h | 161 +++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/csrc-r4k.c | 4 +-
arch/mips/kernel/vdso.c | 56 ++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 2 +
arch/mips/vdso/vgettimeofday.c | 30 ++
drivers/clocksource/mips-gic-timer.c | 5 +-
include/vdso/datapage.h | 65 +++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 41 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 418 ++++++++++++++++++
56 files changed, 2167 insertions(+), 1233 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 include/vdso/datapage.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
--
2.19.1
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.
Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.
The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in
"asm/vdso/".
* The generic implementation includes the arch specific one and lives
in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place
with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm and mips.
The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].
Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].
To simplify the testing, a copy of the patchset on top of a recent linux tree
can be found at [3].
[1] https://github.com/nlynch-mentor/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v1
[4] git://linux-arm.org/linux-vf.git vdso-mips/v1
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Vincenzo Frascino (27):
kernel: Standardize vdso_datapage
kernel: Add Monotonic boot time support
kernel: Add International Atomic Time support
kernel: Add masks support for Raw and NTP time
kernel: Add clock_mode support
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Introduce vdso_direct
clock: csrc-4k: Add support for vdso_direct
clock: gic-timer: Add support for vdso_direct
mips: Add support for generic vDSO
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 79 ++++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 28 +-
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 2 +
arch/arm/vdso/vgettimeofday.c | 263 +----------
arch/arm64/Kconfig | 25 ++
arch/arm64/Makefile | 21 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 ++-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 89 ++++
arch/arm64/include/asm/vdso/gettimeofday.h | 82 ++++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 13 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 ++--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 413 ++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 24 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 --------------
arch/arm64/kernel/vdso/vdso.lds.S | 1 +
arch/arm64/kernel/vdso/vgettimeofday.c | 30 ++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 82 ++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 43 ++
arch/mips/Kconfig | 3 +
arch/mips/include/asm/clocksource.h | 1 +
arch/mips/include/asm/vdso.h | 53 +--
arch/mips/include/asm/vdso/gettimeofday.h | 161 +++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/csrc-r4k.c | 4 +-
arch/mips/kernel/vdso.c | 56 ++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 2 +
arch/mips/vdso/vgettimeofday.c | 30 ++
drivers/clocksource/mips-gic-timer.c | 5 +-
include/vdso/datapage.h | 65 +++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 41 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 418 ++++++++++++++++++
56 files changed, 2167 insertions(+), 1233 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 include/vdso/datapage.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
--
2.19.1
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 01/27] kernel: Standardize vdso_datapage
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch moves vdso_datapage.h from arm64 private
includes to general includes (include/vdso) and to avoid possible conflicts
replaces the header guard.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 48 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..a02ca170d56e
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+struct vdso_data {
+ __u64 cs_cycle_last; /* Timebase at clocksource init */
+ __u64 raw_time_sec; /* Raw time */
+ __u64 raw_time_nsec;
+ __u64 xtime_clock_sec; /* Kernel time */
+ __u64 xtime_clock_nsec;
+ __u64 xtime_coarse_sec; /* Coarse time */
+ __u64 xtime_coarse_nsec;
+ __u64 wtm_clock_sec; /* Wall to monotonic time */
+ __u64 wtm_clock_nsec;
+ __u32 tb_seq_count; /* Timebase sequence counter */
+ __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u32 cs_shift; /* Clocksource shift (mono = raw) */
+ __u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u32 tz_minuteswest; /* Whacky timezone stuff */
+ __u32 tz_dsttime;
+ __u32 use_syscall;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 01/27] kernel: Standardize vdso_datapage
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch moves vdso_datapage.h from arm64 private
includes to general includes (include/vdso) and to avoid possible conflicts
replaces the header guard.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 48 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..a02ca170d56e
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+struct vdso_data {
+ __u64 cs_cycle_last; /* Timebase at clocksource init */
+ __u64 raw_time_sec; /* Raw time */
+ __u64 raw_time_nsec;
+ __u64 xtime_clock_sec; /* Kernel time */
+ __u64 xtime_clock_nsec;
+ __u64 xtime_coarse_sec; /* Coarse time */
+ __u64 xtime_coarse_nsec;
+ __u64 wtm_clock_sec; /* Wall to monotonic time */
+ __u64 wtm_clock_nsec;
+ __u32 tb_seq_count; /* Timebase sequence counter */
+ __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u32 cs_shift; /* Clocksource shift (mono = raw) */
+ __u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u32 tz_minuteswest; /* Whacky timezone stuff */
+ __u32 tz_dsttime;
+ __u32 use_syscall;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 01/27] kernel: Standardize vdso_datapage
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch moves vdso_datapage.h from arm64 private
includes to general includes (include/vdso) and to avoid possible conflicts
replaces the header guard.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 48 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..a02ca170d56e
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,48 @@
+/*
+ * Copyright (C) 2012 ARM Limited
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
+ */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+
+struct vdso_data {
+ __u64 cs_cycle_last; /* Timebase at clocksource init */
+ __u64 raw_time_sec; /* Raw time */
+ __u64 raw_time_nsec;
+ __u64 xtime_clock_sec; /* Kernel time */
+ __u64 xtime_clock_nsec;
+ __u64 xtime_coarse_sec; /* Coarse time */
+ __u64 xtime_coarse_nsec;
+ __u64 wtm_clock_sec; /* Wall to monotonic time */
+ __u64 wtm_clock_nsec;
+ __u32 tb_seq_count; /* Timebase sequence counter */
+ __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u32 cs_shift; /* Clocksource shift (mono = raw) */
+ __u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u32 tz_minuteswest; /* Whacky timezone stuff */
+ __u32 tz_dsttime;
+ __u32 use_syscall;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 02/27] kernel: Add Monotonic boot time support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Currently, datapage.h has no support for CLOCK_BOOTTIME.
CLOCK_BOOTTIME is identical to CLOCK_MONOTONIC, but includes
any time spent in suspend.
This patch adds monotonic boot time support (btm_nsec) to datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index a02ca170d56e..52189c21d422 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -32,6 +32,7 @@ struct vdso_data {
__u64 xtime_coarse_nsec;
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
+ __u64 btm_nsec; /* Monotonic to boot time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 02/27] kernel: Add Monotonic boot time support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
Currently, datapage.h has no support for CLOCK_BOOTTIME.
CLOCK_BOOTTIME is identical to CLOCK_MONOTONIC, but includes
any time spent in suspend.
This patch adds monotonic boot time support (btm_nsec) to datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index a02ca170d56e..52189c21d422 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -32,6 +32,7 @@ struct vdso_data {
__u64 xtime_coarse_nsec;
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
+ __u64 btm_nsec; /* Monotonic to boot time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 02/27] kernel: Add Monotonic boot time support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Currently, datapage.h has no support for CLOCK_BOOTTIME.
CLOCK_BOOTTIME is identical to CLOCK_MONOTONIC, but includes
any time spent in suspend.
This patch adds monotonic boot time support (btm_nsec) to datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index a02ca170d56e..52189c21d422 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -32,6 +32,7 @@ struct vdso_data {
__u64 xtime_coarse_nsec;
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
+ __u64 btm_nsec; /* Monotonic to boot time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 03/27] kernel: Add International Atomic Time support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Currently, datapage.h has no support for CLOCK_TAI.
CLOCK_TAI is similar to CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
This patch adds CLOCK_TAI (tai_sec) support to datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index 52189c21d422..d50a108afaed 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -33,6 +33,7 @@ struct vdso_data {
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
__u64 btm_nsec; /* Monotonic to boot time */
+ __u64 tai_sec; /* International Atomic Time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 03/27] kernel: Add International Atomic Time support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
Currently, datapage.h has no support for CLOCK_TAI.
CLOCK_TAI is similar to CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
This patch adds CLOCK_TAI (tai_sec) support to datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index 52189c21d422..d50a108afaed 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -33,6 +33,7 @@ struct vdso_data {
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
__u64 btm_nsec; /* Monotonic to boot time */
+ __u64 tai_sec; /* International Atomic Time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 03/27] kernel: Add International Atomic Time support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Currently, datapage.h has no support for CLOCK_TAI.
CLOCK_TAI is similar to CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
This patch adds CLOCK_TAI (tai_sec) support to datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index 52189c21d422..d50a108afaed 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -33,6 +33,7 @@ struct vdso_data {
__u64 wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
__u64 btm_nsec; /* Monotonic to boot time */
+ __u64 tai_sec; /* International Atomic Time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 04/27] kernel: Add masks support for Raw and NTP time
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Depending on the clocksource selected there can be supported a
different resolution hence we require a mask to be applied to the
time calculation.
This patch exposes the mask in the datapage for Raw and NTP adjusted
clock.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index d50a108afaed..fb6ce4c3f2d0 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -36,8 +36,10 @@ struct vdso_data {
__u64 tai_sec; /* International Atomic Time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u64 cs_mono_mask; /* NTP-adjusted clocksource mask */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
__u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u64 cs_raw_mask; /* Raw clocksource mask */
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 04/27] kernel: Add masks support for Raw and NTP time
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
Depending on the clocksource selected there can be supported a
different resolution hence we require a mask to be applied to the
time calculation.
This patch exposes the mask in the datapage for Raw and NTP adjusted
clock.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index d50a108afaed..fb6ce4c3f2d0 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -36,8 +36,10 @@ struct vdso_data {
__u64 tai_sec; /* International Atomic Time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u64 cs_mono_mask; /* NTP-adjusted clocksource mask */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
__u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u64 cs_raw_mask; /* Raw clocksource mask */
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 04/27] kernel: Add masks support for Raw and NTP time
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Depending on the clocksource selected there can be supported a
different resolution hence we require a mask to be applied to the
time calculation.
This patch exposes the mask in the datapage for Raw and NTP adjusted
clock.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 2 ++
1 file changed, 2 insertions(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index d50a108afaed..fb6ce4c3f2d0 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -36,8 +36,10 @@ struct vdso_data {
__u64 tai_sec; /* International Atomic Time */
__u32 tb_seq_count; /* Timebase sequence counter */
__u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u64 cs_mono_mask; /* NTP-adjusted clocksource mask */
__u32 cs_shift; /* Clocksource shift (mono = raw) */
__u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u64 cs_raw_mask; /* Raw clocksource mask */
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 05/27] kernel: Add clock_mode support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Some architectures store the clock mode into the vDSO datapage,
consequently the generic datapage needs to be aligned to expose this
information.
This patch adds support for clock_mode in datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index fb6ce4c3f2d0..9a934158f1b8 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -43,6 +43,7 @@ struct vdso_data {
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
+ __u32 clock_mode;
};
#endif /* !__ASSEMBLY__ */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 05/27] kernel: Add clock_mode support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
Some architectures store the clock mode into the vDSO datapage,
consequently the generic datapage needs to be aligned to expose this
information.
This patch adds support for clock_mode in datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index fb6ce4c3f2d0..9a934158f1b8 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -43,6 +43,7 @@ struct vdso_data {
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
+ __u32 clock_mode;
};
#endif /* !__ASSEMBLY__ */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 05/27] kernel: Add clock_mode support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Some architectures store the clock mode into the vDSO datapage,
consequently the generic datapage needs to be aligned to expose this
information.
This patch adds support for clock_mode in datapage.h.
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index fb6ce4c3f2d0..9a934158f1b8 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -43,6 +43,7 @@ struct vdso_data {
__u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
+ __u32 clock_mode;
};
#endif /* !__ASSEMBLY__ */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
This patch tries to unify the gettimeofday vdso implementation
introducing lib/vdso. The code contained in this library can
ideally be reused by all the architectures avoiding, where
possible, code duplication.
In doing so, tries to maintain the performances using inlining as much
as possible and consequently reduces the surface for ROP type of attacks.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 40 ++--
lib/Kconfig | 5 +
lib/vdso/Kconfig | 41 ++++
lib/vdso/Makefile | 22 +++
lib/vdso/gettimeofday.c | 414 ++++++++++++++++++++++++++++++++++++++++
5 files changed, 508 insertions(+), 14 deletions(-)
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index 9a934158f1b8..c9d2062b7a66 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -22,25 +22,37 @@
#include <linux/types.h>
+/*
+ * To avoid performance issues __u_vdso (unsigned vdso type) depends
+ * on the architecture:
+ * 32 bit only __u_vdso is defined as __u32
+ * 64 bit plus compat __u_vdso is defined as __u64
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+typedef __u32 __u_vdso;
+#else
+typedef __u64 __u_vdso;
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
+ __u64 cs_cycle_last; /* Timebase at clocksource init */
+ __u_vdso raw_time_sec; /* Raw time */
__u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
+ __u_vdso xtime_clock_sec; /* Kernel time */
__u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
+ __u_vdso xtime_coarse_sec; /* Coarse time */
__u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
+ __u_vdso wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
- __u64 btm_nsec; /* Monotonic to boot time */
- __u64 tai_sec; /* International Atomic Time */
- __u32 tb_seq_count; /* Timebase sequence counter */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u64 cs_mono_mask; /* NTP-adjusted clocksource mask */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u64 cs_raw_mask; /* Raw clocksource mask */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
+ __u64 btm_nsec; /* Monotonic to boot time */
+ __u_vdso tai_sec; /* International Atomic Time */
+ __u32 tb_seq_count; /* Timebase sequence counter */
+ __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u_vdso cs_mono_mask; /* NTP-adjusted clocksource mask */
+ __u32 cs_shift; /* Clocksource shift (mono = raw) */
+ __u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u_vdso cs_raw_mask; /* Raw clocksource mask */
+ __u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
__u32 clock_mode;
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..38a048e514a2 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -557,6 +557,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..e005cf5d379b
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config HAVE_ARCH_TIMER
+ bool
+ depends on GENERIC_GETTIMEOFDAY
+ help
+ Select this configuration option if the architecture has an arch
+ timer.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..ec903180d3da
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,414 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2017 Cavium, Inc.
+ * Copyright (C) 2015 Mentor Graphics Corporation
+ *
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/uaccess.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+
+#include <asm/vdso/gettimeofday.h>
+
+/* To improve performances, in this file, __always_inline it is used
+ * for the functions called multiple times.
+ */
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->tb_seq_count);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ /* smp_rmb() pairs with the second smp_wmb() in update_vsyscall */
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ /* smp_rmb() pairs with the first smp_wmb() in update_vsyscall */
+ smp_rmb();
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->tb_seq_count);
+ return seq != start;
+}
+
+/*
+ * Returns the clock delta, in nanoseconds left-shifted by the clock
+ * shift.
+ */
+static __always_inline notrace u64 get_clock_shifted_nsec(u64 cycle_last,
+ u64 mult,
+ u64 mask)
+{
+ u64 res;
+
+ /* Read the virtual counter. */
+ res = clock_get_virtual_counter();
+
+ if (res > cycle_last)
+ res = res - cycle_last;
+ /*
+ * VDSO Precision Mask: represents the
+ * precision bits we can guaranty.
+ */
+ res &= mask;
+ return res * mult;
+}
+
+#ifdef CONFIG_HAVE_ARCH_TIMER
+static __always_inline notrace int __do_realtime_or_tai(
+ const struct vdso_data *vd,
+ struct timespec *ts,
+ bool is_tai)
+{
+ u32 seq, cs_mono_mult, cs_shift;
+ u64 ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return -1;
+repeat:
+ seq = vdso_read_begin(vd);
+ cycle_last = vd->cs_cycle_last;
+ cs_mono_mult = vd->cs_mono_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ if (is_tai)
+ sec = vd->tai_sec;
+ else
+ sec = vd->xtime_clock_sec;
+ ns = vd->xtime_clock_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_mono_mult, cs_mono_mask);
+ ns >>= cs_shift;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static __always_inline notrace int __do_realtime_or_tai(
+ const struct vdso_data *vd,
+ struct timespec *ts,
+ bool is_tai)
+{
+ return -1;
+}
+#endif
+
+/*
+ * Handles CLOCK_REALTIME - A representation of the "wall-clock" time.
+ * Can be both stepped and slewed by time adjustment code. It can move
+ * forward and backward.
+ */
+static __always_inline notrace int do_realtime(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_realtime_or_tai(vd, ts, false);
+}
+
+/*
+ * Handles CLOCK_TAI - Like CLOCK_REALTIME, but uses the International
+ * Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
+ * second updates.
+ */
+static notrace int do_tai(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_realtime_or_tai(vd, ts, true);
+}
+
+#ifdef CONFIG_HAVE_ARCH_TIMER
+static __always_inline notrace int __do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts,
+ bool boottime)
+{
+ u32 seq, cs_mono_mult, cs_shift;
+ u64 ns, wtm_ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return 1;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ cycle_last = vd->cs_cycle_last;
+ cs_mono_mult = vd->cs_mono_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ sec = vd->xtime_clock_sec;
+ ns = vd->xtime_clock_nsec;
+ sec += vd->wtm_clock_sec;
+
+ if (boottime)
+ wtm_ns = vd->wtm_clock_nsec + vd->btm_nsec;
+ else
+ ns += vd->wtm_clock_nsec << cs_shift;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_mono_mult, cs_mono_mask);
+ ns >>= cs_shift;
+
+ if (boottime)
+ ns += wtm_ns;
+
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static __always_inline notrace int __do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts,
+ bool boottime)
+{
+ return -1;
+}
+#endif
+
+/*
+ * Handles CLOCK_MONOTONIC - A representation of the interval from an
+ * arbitrary given time. Can be slewed but not stepped by time adjustment
+ * code. It can move forward but not backward.
+ */
+static notrace int do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_monotonic(vd, ts, false);
+}
+
+/*
+ * Handles CLOCK_MONOTONIC_RAW - This is a version of CLOCK_MONOTONIC that can
+ * be neither slewed nor stepped by time adjustment code. It cannot move
+ * forward or backward.
+ */
+static notrace int do_monotonic_raw(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq, cs_raw_mult, cs_shift;
+ u64 ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return -1;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ cycle_last = vd->cs_cycle_last;
+ cs_raw_mult = vd->cs_raw_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ sec = vd->raw_time_sec;
+ ns = vd->raw_time_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_raw_mult, cs_mono_mask);
+ ns >>= cs_shift;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+
+/*
+ * Handles CLOCK_REALTIME_COARSE - This is a version of CLOCK_REALTIME
+ * at a lower resolution.
+ */
+static notrace void do_realtime_coarse(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq;
+ u64 ns;
+ __u_vdso sec;
+
+repeat:
+ seq = vdso_read_begin(vd);
+ sec = vd->xtime_coarse_sec;
+ ns = vd->xtime_coarse_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ts->tv_sec = sec;
+ ts->tv_nsec = ns;
+}
+
+/*
+ * Handles CLOCK_MONOTONIC_COARSE - This is a version of CLOCK_MONOTONIC
+ * at a lower resolution.
+ */
+static notrace void do_monotonic_coarse(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq;
+ u64 ns, wtm_ns;
+ __u_vdso sec, wtm_sec;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ sec = vd->xtime_coarse_sec;
+ ns = vd->xtime_coarse_nsec;
+ wtm_sec = vd->wtm_clock_sec;
+ wtm_ns = vd->wtm_clock_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ sec += wtm_sec;
+ ns += wtm_ns;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+}
+
+/*
+ * Handles CLOCK_BOOTTIME - This is a version of CLOCK_MONOTONIC that keeps
+ * into account the time spent in suspend mode.
+ * Available on on 2.6.39+ kernel version.
+ */
+static notrace int do_boottime(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_monotonic(vd, ts, true);
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ switch (clock) {
+ case CLOCK_REALTIME:
+ if (do_realtime(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_TAI:
+ if (do_tai(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_MONOTONIC:
+ if (do_monotonic(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_MONOTONIC_RAW:
+ if (do_monotonic_raw(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_BOOTTIME:
+ if (do_boottime(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_REALTIME_COARSE:
+ do_realtime_coarse(vd, ts);
+ break;
+ case CLOCK_MONOTONIC_COARSE:
+ do_monotonic_coarse(vd, ts);
+ break;
+ default:
+ goto fallback;
+ }
+
+ return 0;
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct timespec ts;
+
+ if (do_realtime(vd, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / 1000;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+static notrace time_t __cvdso_time(time_t *time)
+{
+ u32 seq;
+ time_t t;
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ t = vd->xtime_coarse_sec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ if (unlikely(time != NULL))
+ *time = t;
+
+ return t;
+}
+
+static notrace int __cvdso_clock_getres(clockid_t clock_id,
+ struct timespec *res)
+{
+ u64 ns;
+
+ if (clock_id == CLOCK_REALTIME ||
+ clock_id == CLOCK_TAI ||
+ clock_id == CLOCK_BOOTTIME ||
+ clock_id == CLOCK_MONOTONIC ||
+ clock_id == CLOCK_MONOTONIC_RAW)
+ ns = MONOTONIC_RES_NSEC;
+ else if (clock_id == CLOCK_REALTIME_COARSE ||
+ clock_id == CLOCK_MONOTONIC_COARSE)
+ ns = LOW_RES_NSEC;
+ else
+ return clock_getres_fallback(clock_id, res);
+
+ if (res) {
+ res->tv_sec = 0;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+}
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
This patch tries to unify the gettimeofday vdso implementation
introducing lib/vdso. The code contained in this library can
ideally be reused by all the architectures avoiding, where
possible, code duplication.
In doing so, tries to maintain the performances using inlining as much
as possible and consequently reduces the surface for ROP type of attacks.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 40 ++--
lib/Kconfig | 5 +
lib/vdso/Kconfig | 41 ++++
lib/vdso/Makefile | 22 +++
lib/vdso/gettimeofday.c | 414 ++++++++++++++++++++++++++++++++++++++++
5 files changed, 508 insertions(+), 14 deletions(-)
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index 9a934158f1b8..c9d2062b7a66 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -22,25 +22,37 @@
#include <linux/types.h>
+/*
+ * To avoid performance issues __u_vdso (unsigned vdso type) depends
+ * on the architecture:
+ * 32 bit only __u_vdso is defined as __u32
+ * 64 bit plus compat __u_vdso is defined as __u64
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+typedef __u32 __u_vdso;
+#else
+typedef __u64 __u_vdso;
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
+ __u64 cs_cycle_last; /* Timebase at clocksource init */
+ __u_vdso raw_time_sec; /* Raw time */
__u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
+ __u_vdso xtime_clock_sec; /* Kernel time */
__u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
+ __u_vdso xtime_coarse_sec; /* Coarse time */
__u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
+ __u_vdso wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
- __u64 btm_nsec; /* Monotonic to boot time */
- __u64 tai_sec; /* International Atomic Time */
- __u32 tb_seq_count; /* Timebase sequence counter */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u64 cs_mono_mask; /* NTP-adjusted clocksource mask */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u64 cs_raw_mask; /* Raw clocksource mask */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
+ __u64 btm_nsec; /* Monotonic to boot time */
+ __u_vdso tai_sec; /* International Atomic Time */
+ __u32 tb_seq_count; /* Timebase sequence counter */
+ __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u_vdso cs_mono_mask; /* NTP-adjusted clocksource mask */
+ __u32 cs_shift; /* Clocksource shift (mono = raw) */
+ __u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u_vdso cs_raw_mask; /* Raw clocksource mask */
+ __u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
__u32 clock_mode;
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..38a048e514a2 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -557,6 +557,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..e005cf5d379b
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config HAVE_ARCH_TIMER
+ bool
+ depends on GENERIC_GETTIMEOFDAY
+ help
+ Select this configuration option if the architecture has an arch
+ timer.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..ec903180d3da
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,414 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2017 Cavium, Inc.
+ * Copyright (C) 2015 Mentor Graphics Corporation
+ *
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/uaccess.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+
+#include <asm/vdso/gettimeofday.h>
+
+/* To improve performances, in this file, __always_inline it is used
+ * for the functions called multiple times.
+ */
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->tb_seq_count);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ /* smp_rmb() pairs with the second smp_wmb() in update_vsyscall */
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ /* smp_rmb() pairs with the first smp_wmb() in update_vsyscall */
+ smp_rmb();
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->tb_seq_count);
+ return seq != start;
+}
+
+/*
+ * Returns the clock delta, in nanoseconds left-shifted by the clock
+ * shift.
+ */
+static __always_inline notrace u64 get_clock_shifted_nsec(u64 cycle_last,
+ u64 mult,
+ u64 mask)
+{
+ u64 res;
+
+ /* Read the virtual counter. */
+ res = clock_get_virtual_counter();
+
+ if (res > cycle_last)
+ res = res - cycle_last;
+ /*
+ * VDSO Precision Mask: represents the
+ * precision bits we can guaranty.
+ */
+ res &= mask;
+ return res * mult;
+}
+
+#ifdef CONFIG_HAVE_ARCH_TIMER
+static __always_inline notrace int __do_realtime_or_tai(
+ const struct vdso_data *vd,
+ struct timespec *ts,
+ bool is_tai)
+{
+ u32 seq, cs_mono_mult, cs_shift;
+ u64 ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return -1;
+repeat:
+ seq = vdso_read_begin(vd);
+ cycle_last = vd->cs_cycle_last;
+ cs_mono_mult = vd->cs_mono_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ if (is_tai)
+ sec = vd->tai_sec;
+ else
+ sec = vd->xtime_clock_sec;
+ ns = vd->xtime_clock_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_mono_mult, cs_mono_mask);
+ ns >>= cs_shift;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static __always_inline notrace int __do_realtime_or_tai(
+ const struct vdso_data *vd,
+ struct timespec *ts,
+ bool is_tai)
+{
+ return -1;
+}
+#endif
+
+/*
+ * Handles CLOCK_REALTIME - A representation of the "wall-clock" time.
+ * Can be both stepped and slewed by time adjustment code. It can move
+ * forward and backward.
+ */
+static __always_inline notrace int do_realtime(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_realtime_or_tai(vd, ts, false);
+}
+
+/*
+ * Handles CLOCK_TAI - Like CLOCK_REALTIME, but uses the International
+ * Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
+ * second updates.
+ */
+static notrace int do_tai(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_realtime_or_tai(vd, ts, true);
+}
+
+#ifdef CONFIG_HAVE_ARCH_TIMER
+static __always_inline notrace int __do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts,
+ bool boottime)
+{
+ u32 seq, cs_mono_mult, cs_shift;
+ u64 ns, wtm_ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return 1;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ cycle_last = vd->cs_cycle_last;
+ cs_mono_mult = vd->cs_mono_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ sec = vd->xtime_clock_sec;
+ ns = vd->xtime_clock_nsec;
+ sec += vd->wtm_clock_sec;
+
+ if (boottime)
+ wtm_ns = vd->wtm_clock_nsec + vd->btm_nsec;
+ else
+ ns += vd->wtm_clock_nsec << cs_shift;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_mono_mult, cs_mono_mask);
+ ns >>= cs_shift;
+
+ if (boottime)
+ ns += wtm_ns;
+
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static __always_inline notrace int __do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts,
+ bool boottime)
+{
+ return -1;
+}
+#endif
+
+/*
+ * Handles CLOCK_MONOTONIC - A representation of the interval from an
+ * arbitrary given time. Can be slewed but not stepped by time adjustment
+ * code. It can move forward but not backward.
+ */
+static notrace int do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_monotonic(vd, ts, false);
+}
+
+/*
+ * Handles CLOCK_MONOTONIC_RAW - This is a version of CLOCK_MONOTONIC that can
+ * be neither slewed nor stepped by time adjustment code. It cannot move
+ * forward or backward.
+ */
+static notrace int do_monotonic_raw(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq, cs_raw_mult, cs_shift;
+ u64 ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return -1;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ cycle_last = vd->cs_cycle_last;
+ cs_raw_mult = vd->cs_raw_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ sec = vd->raw_time_sec;
+ ns = vd->raw_time_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_raw_mult, cs_mono_mask);
+ ns >>= cs_shift;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+
+/*
+ * Handles CLOCK_REALTIME_COARSE - This is a version of CLOCK_REALTIME
+ * at a lower resolution.
+ */
+static notrace void do_realtime_coarse(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq;
+ u64 ns;
+ __u_vdso sec;
+
+repeat:
+ seq = vdso_read_begin(vd);
+ sec = vd->xtime_coarse_sec;
+ ns = vd->xtime_coarse_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ts->tv_sec = sec;
+ ts->tv_nsec = ns;
+}
+
+/*
+ * Handles CLOCK_MONOTONIC_COARSE - This is a version of CLOCK_MONOTONIC
+ * at a lower resolution.
+ */
+static notrace void do_monotonic_coarse(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq;
+ u64 ns, wtm_ns;
+ __u_vdso sec, wtm_sec;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ sec = vd->xtime_coarse_sec;
+ ns = vd->xtime_coarse_nsec;
+ wtm_sec = vd->wtm_clock_sec;
+ wtm_ns = vd->wtm_clock_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ sec += wtm_sec;
+ ns += wtm_ns;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+}
+
+/*
+ * Handles CLOCK_BOOTTIME - This is a version of CLOCK_MONOTONIC that keeps
+ * into account the time spent in suspend mode.
+ * Available on on 2.6.39+ kernel version.
+ */
+static notrace int do_boottime(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_monotonic(vd, ts, true);
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ switch (clock) {
+ case CLOCK_REALTIME:
+ if (do_realtime(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_TAI:
+ if (do_tai(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_MONOTONIC:
+ if (do_monotonic(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_MONOTONIC_RAW:
+ if (do_monotonic_raw(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_BOOTTIME:
+ if (do_boottime(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_REALTIME_COARSE:
+ do_realtime_coarse(vd, ts);
+ break;
+ case CLOCK_MONOTONIC_COARSE:
+ do_monotonic_coarse(vd, ts);
+ break;
+ default:
+ goto fallback;
+ }
+
+ return 0;
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct timespec ts;
+
+ if (do_realtime(vd, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / 1000;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+static notrace time_t __cvdso_time(time_t *time)
+{
+ u32 seq;
+ time_t t;
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ t = vd->xtime_coarse_sec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ if (unlikely(time != NULL))
+ *time = t;
+
+ return t;
+}
+
+static notrace int __cvdso_clock_getres(clockid_t clock_id,
+ struct timespec *res)
+{
+ u64 ns;
+
+ if (clock_id == CLOCK_REALTIME ||
+ clock_id == CLOCK_TAI ||
+ clock_id == CLOCK_BOOTTIME ||
+ clock_id == CLOCK_MONOTONIC ||
+ clock_id == CLOCK_MONOTONIC_RAW)
+ ns = MONOTONIC_RES_NSEC;
+ else if (clock_id == CLOCK_REALTIME_COARSE ||
+ clock_id == CLOCK_MONOTONIC_COARSE)
+ ns = LOW_RES_NSEC;
+ else
+ return clock_getres_fallback(clock_id, res);
+
+ if (res) {
+ res->tv_sec = 0;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+}
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
This patch tries to unify the gettimeofday vdso implementation
introducing lib/vdso. The code contained in this library can
ideally be reused by all the architectures avoiding, where
possible, code duplication.
In doing so, tries to maintain the performances using inlining as much
as possible and consequently reduces the surface for ROP type of attacks.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 40 ++--
lib/Kconfig | 5 +
lib/vdso/Kconfig | 41 ++++
lib/vdso/Makefile | 22 +++
lib/vdso/gettimeofday.c | 414 ++++++++++++++++++++++++++++++++++++++++
5 files changed, 508 insertions(+), 14 deletions(-)
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index 9a934158f1b8..c9d2062b7a66 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -22,25 +22,37 @@
#include <linux/types.h>
+/*
+ * To avoid performance issues __u_vdso (unsigned vdso type) depends
+ * on the architecture:
+ * 32 bit only __u_vdso is defined as __u32
+ * 64 bit plus compat __u_vdso is defined as __u64
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+typedef __u32 __u_vdso;
+#else
+typedef __u64 __u_vdso;
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
+ __u64 cs_cycle_last; /* Timebase at clocksource init */
+ __u_vdso raw_time_sec; /* Raw time */
__u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
+ __u_vdso xtime_clock_sec; /* Kernel time */
__u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
+ __u_vdso xtime_coarse_sec; /* Coarse time */
__u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
+ __u_vdso wtm_clock_sec; /* Wall to monotonic time */
__u64 wtm_clock_nsec;
- __u64 btm_nsec; /* Monotonic to boot time */
- __u64 tai_sec; /* International Atomic Time */
- __u32 tb_seq_count; /* Timebase sequence counter */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u64 cs_mono_mask; /* NTP-adjusted clocksource mask */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u64 cs_raw_mask; /* Raw clocksource mask */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
+ __u64 btm_nsec; /* Monotonic to boot time */
+ __u_vdso tai_sec; /* International Atomic Time */
+ __u32 tb_seq_count; /* Timebase sequence counter */
+ __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
+ __u_vdso cs_mono_mask; /* NTP-adjusted clocksource mask */
+ __u32 cs_shift; /* Clocksource shift (mono = raw) */
+ __u32 cs_raw_mult; /* Raw clocksource multiplier */
+ __u_vdso cs_raw_mask; /* Raw clocksource mask */
+ __u32 tz_minuteswest; /* Whacky timezone stuff */
__u32 tz_dsttime;
__u32 use_syscall;
__u32 clock_mode;
diff --git a/lib/Kconfig b/lib/Kconfig
index a9965f4af4dd..38a048e514a2 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -557,6 +557,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..e005cf5d379b
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config HAVE_ARCH_TIMER
+ bool
+ depends on GENERIC_GETTIMEOFDAY
+ help
+ Select this configuration option if the architecture has an arch
+ timer.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..ec903180d3da
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,414 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2017 Cavium, Inc.
+ * Copyright (C) 2015 Mentor Graphics Corporation
+ *
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/uaccess.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+
+#include <asm/vdso/gettimeofday.h>
+
+/* To improve performances, in this file, __always_inline it is used
+ * for the functions called multiple times.
+ */
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->tb_seq_count);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ /* smp_rmb() pairs with the second smp_wmb() in update_vsyscall */
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ /* smp_rmb() pairs with the first smp_wmb() in update_vsyscall */
+ smp_rmb();
+ /* Trying to access concurrent shared memory */
+ seq = READ_ONCE(vd->tb_seq_count);
+ return seq != start;
+}
+
+/*
+ * Returns the clock delta, in nanoseconds left-shifted by the clock
+ * shift.
+ */
+static __always_inline notrace u64 get_clock_shifted_nsec(u64 cycle_last,
+ u64 mult,
+ u64 mask)
+{
+ u64 res;
+
+ /* Read the virtual counter. */
+ res = clock_get_virtual_counter();
+
+ if (res > cycle_last)
+ res = res - cycle_last;
+ /*
+ * VDSO Precision Mask: represents the
+ * precision bits we can guaranty.
+ */
+ res &= mask;
+ return res * mult;
+}
+
+#ifdef CONFIG_HAVE_ARCH_TIMER
+static __always_inline notrace int __do_realtime_or_tai(
+ const struct vdso_data *vd,
+ struct timespec *ts,
+ bool is_tai)
+{
+ u32 seq, cs_mono_mult, cs_shift;
+ u64 ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return -1;
+repeat:
+ seq = vdso_read_begin(vd);
+ cycle_last = vd->cs_cycle_last;
+ cs_mono_mult = vd->cs_mono_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ if (is_tai)
+ sec = vd->tai_sec;
+ else
+ sec = vd->xtime_clock_sec;
+ ns = vd->xtime_clock_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_mono_mult, cs_mono_mask);
+ ns >>= cs_shift;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static __always_inline notrace int __do_realtime_or_tai(
+ const struct vdso_data *vd,
+ struct timespec *ts,
+ bool is_tai)
+{
+ return -1;
+}
+#endif
+
+/*
+ * Handles CLOCK_REALTIME - A representation of the "wall-clock" time.
+ * Can be both stepped and slewed by time adjustment code. It can move
+ * forward and backward.
+ */
+static __always_inline notrace int do_realtime(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_realtime_or_tai(vd, ts, false);
+}
+
+/*
+ * Handles CLOCK_TAI - Like CLOCK_REALTIME, but uses the International
+ * Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
+ * second updates.
+ */
+static notrace int do_tai(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_realtime_or_tai(vd, ts, true);
+}
+
+#ifdef CONFIG_HAVE_ARCH_TIMER
+static __always_inline notrace int __do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts,
+ bool boottime)
+{
+ u32 seq, cs_mono_mult, cs_shift;
+ u64 ns, wtm_ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return 1;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ cycle_last = vd->cs_cycle_last;
+ cs_mono_mult = vd->cs_mono_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ sec = vd->xtime_clock_sec;
+ ns = vd->xtime_clock_nsec;
+ sec += vd->wtm_clock_sec;
+
+ if (boottime)
+ wtm_ns = vd->wtm_clock_nsec + vd->btm_nsec;
+ else
+ ns += vd->wtm_clock_nsec << cs_shift;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_mono_mult, cs_mono_mask);
+ ns >>= cs_shift;
+
+ if (boottime)
+ ns += wtm_ns;
+
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+#else
+static __always_inline notrace int __do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts,
+ bool boottime)
+{
+ return -1;
+}
+#endif
+
+/*
+ * Handles CLOCK_MONOTONIC - A representation of the interval from an
+ * arbitrary given time. Can be slewed but not stepped by time adjustment
+ * code. It can move forward but not backward.
+ */
+static notrace int do_monotonic(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_monotonic(vd, ts, false);
+}
+
+/*
+ * Handles CLOCK_MONOTONIC_RAW - This is a version of CLOCK_MONOTONIC that can
+ * be neither slewed nor stepped by time adjustment code. It cannot move
+ * forward or backward.
+ */
+static notrace int do_monotonic_raw(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq, cs_raw_mult, cs_shift;
+ u64 ns;
+ __u_vdso sec;
+ u64 cycle_last, cs_mono_mask;
+
+ if (vd->use_syscall)
+ return -1;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ cycle_last = vd->cs_cycle_last;
+ cs_raw_mult = vd->cs_raw_mult;
+ cs_shift = vd->cs_shift;
+ cs_mono_mask = vd->cs_mono_mask;
+
+ sec = vd->raw_time_sec;
+ ns = vd->raw_time_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ns += get_clock_shifted_nsec(cycle_last, cs_raw_mult, cs_mono_mask);
+ ns >>= cs_shift;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+
+/*
+ * Handles CLOCK_REALTIME_COARSE - This is a version of CLOCK_REALTIME
+ * at a lower resolution.
+ */
+static notrace void do_realtime_coarse(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq;
+ u64 ns;
+ __u_vdso sec;
+
+repeat:
+ seq = vdso_read_begin(vd);
+ sec = vd->xtime_coarse_sec;
+ ns = vd->xtime_coarse_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ ts->tv_sec = sec;
+ ts->tv_nsec = ns;
+}
+
+/*
+ * Handles CLOCK_MONOTONIC_COARSE - This is a version of CLOCK_MONOTONIC
+ * at a lower resolution.
+ */
+static notrace void do_monotonic_coarse(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ u32 seq;
+ u64 ns, wtm_ns;
+ __u_vdso sec, wtm_sec;
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ sec = vd->xtime_coarse_sec;
+ ns = vd->xtime_coarse_nsec;
+ wtm_sec = vd->wtm_clock_sec;
+ wtm_ns = vd->wtm_clock_nsec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ sec += wtm_sec;
+ ns += wtm_ns;
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+}
+
+/*
+ * Handles CLOCK_BOOTTIME - This is a version of CLOCK_MONOTONIC that keeps
+ * into account the time spent in suspend mode.
+ * Available on on 2.6.39+ kernel version.
+ */
+static notrace int do_boottime(const struct vdso_data *vd,
+ struct timespec *ts)
+{
+ return __do_monotonic(vd, ts, true);
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ switch (clock) {
+ case CLOCK_REALTIME:
+ if (do_realtime(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_TAI:
+ if (do_tai(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_MONOTONIC:
+ if (do_monotonic(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_MONOTONIC_RAW:
+ if (do_monotonic_raw(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_BOOTTIME:
+ if (do_boottime(vd, ts))
+ goto fallback;
+ break;
+ case CLOCK_REALTIME_COARSE:
+ do_realtime_coarse(vd, ts);
+ break;
+ case CLOCK_MONOTONIC_COARSE:
+ do_monotonic_coarse(vd, ts);
+ break;
+ default:
+ goto fallback;
+ }
+
+ return 0;
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct timespec ts;
+
+ if (do_realtime(vd, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / 1000;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+static notrace time_t __cvdso_time(time_t *time)
+{
+ u32 seq;
+ time_t t;
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+repeat:
+ seq = vdso_read_begin(vd);
+
+ t = vd->xtime_coarse_sec;
+
+ if (unlikely(vdso_read_retry(vd, seq)))
+ goto repeat;
+
+ if (unlikely(time != NULL))
+ *time = t;
+
+ return t;
+}
+
+static notrace int __cvdso_clock_getres(clockid_t clock_id,
+ struct timespec *res)
+{
+ u64 ns;
+
+ if (clock_id == CLOCK_REALTIME ||
+ clock_id == CLOCK_TAI ||
+ clock_id == CLOCK_BOOTTIME ||
+ clock_id == CLOCK_MONOTONIC ||
+ clock_id == CLOCK_MONOTONIC_RAW)
+ ns = MONOTONIC_RES_NSEC;
+ else if (clock_id == CLOCK_REALTIME_COARSE ||
+ clock_id == CLOCK_MONOTONIC_COARSE)
+ ns = LOW_RES_NSEC;
+ else
+ return clock_getres_fallback(clock_id, res);
+
+ if (res) {
+ res->tv_sec = 0;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+}
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-09 16:05 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:05 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, salyzyn, Paul Burton,
Thomas Gleixner, pcc, Linux ARM
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +/*
> + * To avoid performance issues __u_vdso (unsigned vdso type) depends
> + * on the architecture:
> + * 32 bit only __u_vdso is defined as __u32
> + * 64 bit plus compat __u_vdso is defined as __u64
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +typedef __u32 __u_vdso;
> +#else
> +typedef __u64 __u_vdso;
> +#endif /* CONFIG_GENERIC_VDSO_32 */
> +
> struct vdso_data {
> - __u64 cs_cycle_last; /* Timebase at clocksource init */
> - __u64 raw_time_sec; /* Raw time */
> + __u64 cs_cycle_last; /* Timebase at clocksource init */
> + __u_vdso raw_time_sec; /* Raw time */
> __u64 raw_time_nsec;
> - __u64 xtime_clock_sec; /* Kernel time */
> + __u_vdso xtime_clock_sec; /* Kernel time */
> __u64 xtime_clock_nsec;
> - __u64 xtime_coarse_sec; /* Coarse time */
> + __u_vdso xtime_coarse_sec; /* Coarse time */
> __u64 xtime_coarse_nsec;
> - __u64 wtm_clock_sec; /* Wall to monotonic time */
> + __u_vdso wtm_clock_sec; /* Wall to monotonic time */
It looks like you change the layout in a way that makes it incompatible between
32-bit and 64-bit user space. Doesn't that break the compat-vdso code
for 32-bit?
It also seems like we need the seconds to be 64-bit wide, at least
for anything that counts in CLOCK_REALTIME or a related format,
otherwise we have to change it back for y2038 support.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-09 16:05 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:05 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +/*
> + * To avoid performance issues __u_vdso (unsigned vdso type) depends
> + * on the architecture:
> + * 32 bit only __u_vdso is defined as __u32
> + * 64 bit plus compat __u_vdso is defined as __u64
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +typedef __u32 __u_vdso;
> +#else
> +typedef __u64 __u_vdso;
> +#endif /* CONFIG_GENERIC_VDSO_32 */
> +
> struct vdso_data {
> - __u64 cs_cycle_last; /* Timebase at clocksource init */
> - __u64 raw_time_sec; /* Raw time */
> + __u64 cs_cycle_last; /* Timebase at clocksource init */
> + __u_vdso raw_time_sec; /* Raw time */
> __u64 raw_time_nsec;
> - __u64 xtime_clock_sec; /* Kernel time */
> + __u_vdso xtime_clock_sec; /* Kernel time */
> __u64 xtime_clock_nsec;
> - __u64 xtime_coarse_sec; /* Coarse time */
> + __u_vdso xtime_coarse_sec; /* Coarse time */
> __u64 xtime_coarse_nsec;
> - __u64 wtm_clock_sec; /* Wall to monotonic time */
> + __u_vdso wtm_clock_sec; /* Wall to monotonic time */
It looks like you change the layout in a way that makes it incompatible between
32-bit and 64-bit user space. Doesn't that break the compat-vdso code
for 32-bit?
It also seems like we need the seconds to be 64-bit wide, at least
for anything that counts in CLOCK_REALTIME or a related format,
otherwise we have to change it back for y2038 support.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-09 16:05 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:05 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, salyzyn, pcc
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +/*
> + * To avoid performance issues __u_vdso (unsigned vdso type) depends
> + * on the architecture:
> + * 32 bit only __u_vdso is defined as __u32
> + * 64 bit plus compat __u_vdso is defined as __u64
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +typedef __u32 __u_vdso;
> +#else
> +typedef __u64 __u_vdso;
> +#endif /* CONFIG_GENERIC_VDSO_32 */
> +
> struct vdso_data {
> - __u64 cs_cycle_last; /* Timebase at clocksource init */
> - __u64 raw_time_sec; /* Raw time */
> + __u64 cs_cycle_last; /* Timebase at clocksource init */
> + __u_vdso raw_time_sec; /* Raw time */
> __u64 raw_time_nsec;
> - __u64 xtime_clock_sec; /* Kernel time */
> + __u_vdso xtime_clock_sec; /* Kernel time */
> __u64 xtime_clock_nsec;
> - __u64 xtime_coarse_sec; /* Coarse time */
> + __u_vdso xtime_coarse_sec; /* Coarse time */
> __u64 xtime_coarse_nsec;
> - __u64 wtm_clock_sec; /* Wall to monotonic time */
> + __u_vdso wtm_clock_sec; /* Wall to monotonic time */
It looks like you change the layout in a way that makes it incompatible between
32-bit and 64-bit user space. Doesn't that break the compat-vdso code
for 32-bit?
It also seems like we need the seconds to be 64-bit wide, at least
for anything that counts in CLOCK_REALTIME or a related format,
otherwise we have to change it back for y2038 support.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-13 10:21 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:21 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, salyzyn, Paul Burton,
Thomas Gleixner, pcc, Linux ARM
Hi Arnd,
thank you for reviewing my patches.
On 09/11/2018 16:05, Arnd Bergmann wrote:
> On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>> +/*
>> + * To avoid performance issues __u_vdso (unsigned vdso type) depends
>> + * on the architecture:
>> + * 32 bit only __u_vdso is defined as __u32
>> + * 64 bit plus compat __u_vdso is defined as __u64
>> + */
>> +#ifdef CONFIG_GENERIC_VDSO_32
>> +typedef __u32 __u_vdso;
>> +#else
>> +typedef __u64 __u_vdso;
>> +#endif /* CONFIG_GENERIC_VDSO_32 */
>> +
>> struct vdso_data {
>> - __u64 cs_cycle_last; /* Timebase at clocksource init */
>> - __u64 raw_time_sec; /* Raw time */
>> + __u64 cs_cycle_last; /* Timebase at clocksource init */
>> + __u_vdso raw_time_sec; /* Raw time */
>> __u64 raw_time_nsec;
>> - __u64 xtime_clock_sec; /* Kernel time */
>> + __u_vdso xtime_clock_sec; /* Kernel time */
>> __u64 xtime_clock_nsec;
>> - __u64 xtime_coarse_sec; /* Coarse time */
>> + __u_vdso xtime_coarse_sec; /* Coarse time */
>> __u64 xtime_coarse_nsec;
>> - __u64 wtm_clock_sec; /* Wall to monotonic time */
>> + __u_vdso wtm_clock_sec; /* Wall to monotonic time */
>
> It looks like you change the layout in a way that makes it incompatible between
> 32-bit and 64-bit user space. Doesn't that break the compat-vdso code
> for 32-bit?
>
No, it does not since vdso_data structure is not part of the ABI and the userspace can access the vdso library only via function symbols. There was a proposal some time ago to introduce ABI data structures in the vdso data page but it was rejected [1] and [2].
CONFIG_GENERIC_VDSO_32 is meant to be used in 32 bit only platforms and the rationale behind it is to preserve performances were possible.
[1] https://lkml.org/lkml/2015/9/2/460
[2] https://lkml.org/lkml/2015/9/2/534
> It also seems like we need the seconds to be 64-bit wide, at least
> for anything that counts in CLOCK_REALTIME or a related format,
> otherwise we have to change it back for y2038 support.
>
I agree with you, we should address the y2038 support here, even if there are some architectures that currently use u32 for CLOCK_REALTIME sec.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-13 10:21 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:21 UTC (permalink / raw)
To: linux-arm-kernel
Hi Arnd,
thank you for reviewing my patches.
On 09/11/2018 16:05, Arnd Bergmann wrote:
> On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>> +/*
>> + * To avoid performance issues __u_vdso (unsigned vdso type) depends
>> + * on the architecture:
>> + * 32 bit only __u_vdso is defined as __u32
>> + * 64 bit plus compat __u_vdso is defined as __u64
>> + */
>> +#ifdef CONFIG_GENERIC_VDSO_32
>> +typedef __u32 __u_vdso;
>> +#else
>> +typedef __u64 __u_vdso;
>> +#endif /* CONFIG_GENERIC_VDSO_32 */
>> +
>> struct vdso_data {
>> - __u64 cs_cycle_last; /* Timebase at clocksource init */
>> - __u64 raw_time_sec; /* Raw time */
>> + __u64 cs_cycle_last; /* Timebase at clocksource init */
>> + __u_vdso raw_time_sec; /* Raw time */
>> __u64 raw_time_nsec;
>> - __u64 xtime_clock_sec; /* Kernel time */
>> + __u_vdso xtime_clock_sec; /* Kernel time */
>> __u64 xtime_clock_nsec;
>> - __u64 xtime_coarse_sec; /* Coarse time */
>> + __u_vdso xtime_coarse_sec; /* Coarse time */
>> __u64 xtime_coarse_nsec;
>> - __u64 wtm_clock_sec; /* Wall to monotonic time */
>> + __u_vdso wtm_clock_sec; /* Wall to monotonic time */
>
> It looks like you change the layout in a way that makes it incompatible between
> 32-bit and 64-bit user space. Doesn't that break the compat-vdso code
> for 32-bit?
>
No, it does not since vdso_data structure is not part of the ABI and the userspace can access the vdso library only via function symbols. There was a proposal some time ago to introduce ABI data structures in the vdso data page but it was rejected [1] and [2].
CONFIG_GENERIC_VDSO_32 is meant to be used in 32 bit only platforms and the rationale behind it is to preserve performances were possible.
[1] https://lkml.org/lkml/2015/9/2/460
[2] https://lkml.org/lkml/2015/9/2/534
> It also seems like we need the seconds to be 64-bit wide, at least
> for anything that counts in CLOCK_REALTIME or a related format,
> otherwise we have to change it back for y2038 support.
>
I agree with you, we should address the y2038 support here, even if there are some architectures that currently use u32 for CLOCK_REALTIME sec.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-13 10:21 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:21 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, salyzyn, pcc
Hi Arnd,
thank you for reviewing my patches.
On 09/11/2018 16:05, Arnd Bergmann wrote:
> On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>> +/*
>> + * To avoid performance issues __u_vdso (unsigned vdso type) depends
>> + * on the architecture:
>> + * 32 bit only __u_vdso is defined as __u32
>> + * 64 bit plus compat __u_vdso is defined as __u64
>> + */
>> +#ifdef CONFIG_GENERIC_VDSO_32
>> +typedef __u32 __u_vdso;
>> +#else
>> +typedef __u64 __u_vdso;
>> +#endif /* CONFIG_GENERIC_VDSO_32 */
>> +
>> struct vdso_data {
>> - __u64 cs_cycle_last; /* Timebase at clocksource init */
>> - __u64 raw_time_sec; /* Raw time */
>> + __u64 cs_cycle_last; /* Timebase at clocksource init */
>> + __u_vdso raw_time_sec; /* Raw time */
>> __u64 raw_time_nsec;
>> - __u64 xtime_clock_sec; /* Kernel time */
>> + __u_vdso xtime_clock_sec; /* Kernel time */
>> __u64 xtime_clock_nsec;
>> - __u64 xtime_coarse_sec; /* Coarse time */
>> + __u_vdso xtime_coarse_sec; /* Coarse time */
>> __u64 xtime_coarse_nsec;
>> - __u64 wtm_clock_sec; /* Wall to monotonic time */
>> + __u_vdso wtm_clock_sec; /* Wall to monotonic time */
>
> It looks like you change the layout in a way that makes it incompatible between
> 32-bit and 64-bit user space. Doesn't that break the compat-vdso code
> for 32-bit?
>
No, it does not since vdso_data structure is not part of the ABI and the userspace can access the vdso library only via function symbols. There was a proposal some time ago to introduce ABI data structures in the vdso data page but it was rejected [1] and [2].
CONFIG_GENERIC_VDSO_32 is meant to be used in 32 bit only platforms and the rationale behind it is to preserve performances were possible.
[1] https://lkml.org/lkml/2015/9/2/460
[2] https://lkml.org/lkml/2015/9/2/534
> It also seems like we need the seconds to be 64-bit wide, at least
> for anything that counts in CLOCK_REALTIME or a related format,
> otherwise we have to change it back for y2038 support.
>
I agree with you, we should address the y2038 support here, even if there are some architectures that currently use u32 for CLOCK_REALTIME sec.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-10 16:18 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-10 16:18 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> This patch tries to unify the gettimeofday vdso implementation
> introducing lib/vdso. The code contained in this library can
> ideally be reused by all the architectures avoiding, where
> possible, code duplication.
>
> In doing so, tries to maintain the performances using inlining as much
> as possible and consequently reduces the surface for ROP type of attacks.
>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
I just noticed that this new version misses some of the cleanups
that Thomas recently did in the x86 version, e.g. f3e839384164
("x86/vdso: Replace the clockid switch case").
I haven't looked at all the details, but I'd suggest using the latest
x86 code as a base.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-10 16:18 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-10 16:18 UTC (permalink / raw)
To: linux-arm-kernel
On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> This patch tries to unify the gettimeofday vdso implementation
> introducing lib/vdso. The code contained in this library can
> ideally be reused by all the architectures avoiding, where
> possible, code duplication.
>
> In doing so, tries to maintain the performances using inlining as much
> as possible and consequently reduces the surface for ROP type of attacks.
>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
I just noticed that this new version misses some of the cleanups
that Thomas recently did in the x86 version, e.g. f3e839384164
("x86/vdso: Replace the clockid switch case").
I haven't looked at all the details, but I'd suggest using the latest
x86 code as a base.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-10 16:18 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-10 16:18 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne
On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> This patch tries to unify the gettimeofday vdso implementation
> introducing lib/vdso. The code contained in this library can
> ideally be reused by all the architectures avoiding, where
> possible, code duplication.
>
> In doing so, tries to maintain the performances using inlining as much
> as possible and consequently reduces the surface for ROP type of attacks.
>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
I just noticed that this new version misses some of the cleanups
that Thomas recently did in the x86 version, e.g. f3e839384164
("x86/vdso: Replace the clockid switch case").
I haven't looked at all the details, but I'd suggest using the latest
x86 code as a base.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-13 10:24 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:24 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On 10/11/2018 16:18, Arnd Bergmann wrote:
> On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
>> In the last few years we assisted to an explosion of vdso
>> implementations that mostly share similar code.
>>
>> This patch tries to unify the gettimeofday vdso implementation
>> introducing lib/vdso. The code contained in this library can
>> ideally be reused by all the architectures avoiding, where
>> possible, code duplication.
>>
>> In doing so, tries to maintain the performances using inlining as much
>> as possible and consequently reduces the surface for ROP type of attacks.
>>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> I just noticed that this new version misses some of the cleanups
> that Thomas recently did in the x86 version, e.g. f3e839384164
> ("x86/vdso: Replace the clockid switch case").
>
> I haven't looked at all the details, but I'd suggest using the latest
> x86 code as a base.
>
Ok, I will have a look at Thomas' work. Thanks for pointing this out.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-13 10:24 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:24 UTC (permalink / raw)
To: linux-arm-kernel
On 10/11/2018 16:18, Arnd Bergmann wrote:
> On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
>> In the last few years we assisted to an explosion of vdso
>> implementations that mostly share similar code.
>>
>> This patch tries to unify the gettimeofday vdso implementation
>> introducing lib/vdso. The code contained in this library can
>> ideally be reused by all the architectures avoiding, where
>> possible, code duplication.
>>
>> In doing so, tries to maintain the performances using inlining as much
>> as possible and consequently reduces the surface for ROP type of attacks.
>>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> I just noticed that this new version misses some of the cleanups
> that Thomas recently did in the x86 version, e.g. f3e839384164
> ("x86/vdso: Replace the clockid switch case").
>
> I haven't looked at all the details, but I'd suggest using the latest
> x86 code as a base.
>
Ok, I will have a look at Thomas' work. Thanks for pointing this out.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 06/27] kernel: Define gettimeofday vdso common code
@ 2018-11-13 10:24 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:24 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne
On 10/11/2018 16:18, Arnd Bergmann wrote:
> On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
>> In the last few years we assisted to an explosion of vdso
>> implementations that mostly share similar code.
>>
>> This patch tries to unify the gettimeofday vdso implementation
>> introducing lib/vdso. The code contained in this library can
>> ideally be reused by all the architectures avoiding, where
>> possible, code duplication.
>>
>> In doing so, tries to maintain the performances using inlining as much
>> as possible and consequently reduces the surface for ROP type of attacks.
>>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> I just noticed that this new version misses some of the cleanups
> that Thomas recently did in the x86 version, e.g. f3e839384164
> ("x86/vdso: Replace the clockid switch case").
>
> I haven't looked at all the details, but I'd suggest using the latest
> x86 code as a base.
>
Ok, I will have a look at Thomas' work. Thanks for pointing this out.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
This patch re-implements the gettimeofday vdso in C in order to use
lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME,
CLOCK_TAI and __kernel_time.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 3 +
arch/arm64/include/asm/vdso/gettimeofday.h | 82 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 7 +-
arch/arm64/kernel/vdso.c | 9 +-
arch/arm64/kernel/vdso/Makefile | 24 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vdso.lds.S | 1 +
arch/arm64/kernel/vdso/vgettimeofday.c | 30 ++
9 files changed, 146 insertions(+), 385 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 787d7850e064..71ca1995a088 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -69,6 +69,7 @@ config ARM64
select ARCH_HAS_UBSAN_SANITIZE_ALL
select ARM_AMBA
select ARM_ARCH_TIMER
+ select HAVE_ARCH_TIMER
select ARM_GIC
select AUDIT_ARCH_COMPAT_GENERIC
select ARM_GIC_V2M if PCI
@@ -101,6 +102,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
@@ -153,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b41a3af66e56
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 323aeb5f2fe6..06cfd2363a86 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -114,13 +114,18 @@ int main(void)
DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
+ DEFINE(VDSO_BTM_NSEC, offsetof(struct vdso_data, btm_nsec));
+ DEFINE(VDSO_CLK_TAI, offsetof(struct vdso_data, tai_sec));
DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
+ DEFINE(VDSO_CS_MONO_MASK, offsetof(struct vdso_data, cs_mono_mask));
DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_CS_RAW_MASK, offsetof(struct vdso_data, cs_raw_mask));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..47834fe0bbe3 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,11 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,6 +215,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
@@ -243,6 +245,11 @@ void update_vsyscall(struct timekeeper *tk)
vdso_data->cs_raw_mult = tk->tkr_raw.mult;
/* tkr_mono.shift == tkr_raw.shift */
vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = VDSO_PRECISION_MASK;
+ vdso_data->cs_raw_mask = VDSO_PRECISION_MASK;
}
smp_wmb();
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..4e9fbd8dd95f 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,29 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +46,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +63,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
index beca249bc2f3..9de0ffc369c5 100644
--- a/arch/arm64/kernel/vdso/vdso.lds.S
+++ b/arch/arm64/kernel/vdso/vdso.lds.S
@@ -88,6 +88,7 @@ VERSION
__kernel_gettimeofday;
__kernel_clock_gettime;
__kernel_clock_getres;
+ __kernel_time;
local: *;
};
}
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..288c85a6e308
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __kernel_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
This patch re-implements the gettimeofday vdso in C in order to use
lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME,
CLOCK_TAI and __kernel_time.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 3 +
arch/arm64/include/asm/vdso/gettimeofday.h | 82 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 7 +-
arch/arm64/kernel/vdso.c | 9 +-
arch/arm64/kernel/vdso/Makefile | 24 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vdso.lds.S | 1 +
arch/arm64/kernel/vdso/vgettimeofday.c | 30 ++
9 files changed, 146 insertions(+), 385 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 787d7850e064..71ca1995a088 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -69,6 +69,7 @@ config ARM64
select ARCH_HAS_UBSAN_SANITIZE_ALL
select ARM_AMBA
select ARM_ARCH_TIMER
+ select HAVE_ARCH_TIMER
select ARM_GIC
select AUDIT_ARCH_COMPAT_GENERIC
select ARM_GIC_V2M if PCI
@@ -101,6 +102,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
@@ -153,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b41a3af66e56
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 323aeb5f2fe6..06cfd2363a86 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -114,13 +114,18 @@ int main(void)
DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
+ DEFINE(VDSO_BTM_NSEC, offsetof(struct vdso_data, btm_nsec));
+ DEFINE(VDSO_CLK_TAI, offsetof(struct vdso_data, tai_sec));
DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
+ DEFINE(VDSO_CS_MONO_MASK, offsetof(struct vdso_data, cs_mono_mask));
DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_CS_RAW_MASK, offsetof(struct vdso_data, cs_raw_mask));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..47834fe0bbe3 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,11 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,6 +215,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
@@ -243,6 +245,11 @@ void update_vsyscall(struct timekeeper *tk)
vdso_data->cs_raw_mult = tk->tkr_raw.mult;
/* tkr_mono.shift == tkr_raw.shift */
vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = VDSO_PRECISION_MASK;
+ vdso_data->cs_raw_mask = VDSO_PRECISION_MASK;
}
smp_wmb();
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..4e9fbd8dd95f 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,29 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +46,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +63,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
index beca249bc2f3..9de0ffc369c5 100644
--- a/arch/arm64/kernel/vdso/vdso.lds.S
+++ b/arch/arm64/kernel/vdso/vdso.lds.S
@@ -88,6 +88,7 @@ VERSION
__kernel_gettimeofday;
__kernel_clock_gettime;
__kernel_clock_getres;
+ __kernel_time;
local: *;
};
}
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..288c85a6e308
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __kernel_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
This patch re-implements the gettimeofday vdso in C in order to use
lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME,
CLOCK_TAI and __kernel_time.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 3 +
arch/arm64/include/asm/vdso/gettimeofday.h | 82 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 7 +-
arch/arm64/kernel/vdso.c | 9 +-
arch/arm64/kernel/vdso/Makefile | 24 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vdso.lds.S | 1 +
arch/arm64/kernel/vdso/vgettimeofday.c | 30 ++
9 files changed, 146 insertions(+), 385 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 787d7850e064..71ca1995a088 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -69,6 +69,7 @@ config ARM64
select ARCH_HAS_UBSAN_SANITIZE_ALL
select ARM_AMBA
select ARM_ARCH_TIMER
+ select HAVE_ARCH_TIMER
select ARM_GIC
select AUDIT_ARCH_COMPAT_GENERIC
select ARM_GIC_V2M if PCI
@@ -101,6 +102,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
@@ -153,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b41a3af66e56
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 323aeb5f2fe6..06cfd2363a86 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -114,13 +114,18 @@ int main(void)
DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
+ DEFINE(VDSO_BTM_NSEC, offsetof(struct vdso_data, btm_nsec));
+ DEFINE(VDSO_CLK_TAI, offsetof(struct vdso_data, tai_sec));
DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
+ DEFINE(VDSO_CS_MONO_MASK, offsetof(struct vdso_data, cs_mono_mask));
DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_CS_RAW_MASK, offsetof(struct vdso_data, cs_raw_mask));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..47834fe0bbe3 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,11 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,6 +215,8 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
@@ -243,6 +245,11 @@ void update_vsyscall(struct timekeeper *tk)
vdso_data->cs_raw_mult = tk->tkr_raw.mult;
/* tkr_mono.shift == tkr_raw.shift */
vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = VDSO_PRECISION_MASK;
+ vdso_data->cs_raw_mask = VDSO_PRECISION_MASK;
}
smp_wmb();
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..4e9fbd8dd95f 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,29 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +46,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +63,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
index beca249bc2f3..9de0ffc369c5 100644
--- a/arch/arm64/kernel/vdso/vdso.lds.S
+++ b/arch/arm64/kernel/vdso/vdso.lds.S
@@ -88,6 +88,7 @@ VERSION
__kernel_gettimeofday;
__kernel_clock_gettime;
__kernel_clock_getres;
+ __kernel_time;
local: *;
};
}
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..288c85a6e308
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __kernel_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 16:13 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:13 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, salyzyn, Paul Burton,
Thomas Gleixner, pcc, Linux ARM
> +
> +extern struct vdso_data _vdso_data;
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
I'm trying to get rid of the last users of 'struct timeval' in the kernel so
we can remove the definition (it will clash with future glibc implementations
in the uapi headers). Could you change this to use __kernel_old_timeval
instead?
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
And this should be __kernel_timespec respectively on the kernel-user interface,
at least for 64-bit architectures.
On 32-bit, the corresponding type is 'struct old_timespec32'.
> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> index beca249bc2f3..9de0ffc369c5 100644
> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> @@ -88,6 +88,7 @@ VERSION
> __kernel_gettimeofday;
> __kernel_clock_gettime;
> __kernel_clock_getres;
> + __kernel_time;
> local: *;
> };
> }
I would prefer to not add any deprecated interfaces in the VDSO. If we
have the 64-bit version of clock_gettime, we don't need the 32-bit version
of it, and we don't need gettimeofday() or time() either. The C library
can easily implement those by calling into clock_gettime.
> +notrace int __kernel_clock_gettime(clockid_t clock, struct timespec *ts)
> +{
> + return __cvdso_clock_gettime(clock, ts);
> +}
> +
> +notrace int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz)
> +{
> + return __cvdso_gettimeofday(tv, tz);
> +}
> +
> +notrace time_t __kernel_time(time_t *time)
> +{
> + return __cvdso_time(time);
> +}
> +
> +notrace int __kernel_clock_getres(clockid_t clock_id, struct timespec *res)
> +{
> + return __cvdso_clock_getres(clock_id, res);
> +}
> +
These should use __kernel_old_timeval and __kernel_timespec again.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 16:13 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:13 UTC (permalink / raw)
To: linux-arm-kernel
> +
> +extern struct vdso_data _vdso_data;
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
I'm trying to get rid of the last users of 'struct timeval' in the kernel so
we can remove the definition (it will clash with future glibc implementations
in the uapi headers). Could you change this to use __kernel_old_timeval
instead?
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
And this should be __kernel_timespec respectively on the kernel-user interface,
at least for 64-bit architectures.
On 32-bit, the corresponding type is 'struct old_timespec32'.
> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> index beca249bc2f3..9de0ffc369c5 100644
> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> @@ -88,6 +88,7 @@ VERSION
> __kernel_gettimeofday;
> __kernel_clock_gettime;
> __kernel_clock_getres;
> + __kernel_time;
> local: *;
> };
> }
I would prefer to not add any deprecated interfaces in the VDSO. If we
have the 64-bit version of clock_gettime, we don't need the 32-bit version
of it, and we don't need gettimeofday() or time() either. The C library
can easily implement those by calling into clock_gettime.
> +notrace int __kernel_clock_gettime(clockid_t clock, struct timespec *ts)
> +{
> + return __cvdso_clock_gettime(clock, ts);
> +}
> +
> +notrace int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz)
> +{
> + return __cvdso_gettimeofday(tv, tz);
> +}
> +
> +notrace time_t __kernel_time(time_t *time)
> +{
> + return __cvdso_time(time);
> +}
> +
> +notrace int __kernel_clock_getres(clockid_t clock_id, struct timespec *res)
> +{
> + return __cvdso_clock_getres(clock_id, res);
> +}
> +
These should use __kernel_old_timeval and __kernel_timespec again.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 16:13 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:13 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, salyzyn, pcc
> +
> +extern struct vdso_data _vdso_data;
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
I'm trying to get rid of the last users of 'struct timeval' in the kernel so
we can remove the definition (it will clash with future glibc implementations
in the uapi headers). Could you change this to use __kernel_old_timeval
instead?
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
And this should be __kernel_timespec respectively on the kernel-user interface,
at least for 64-bit architectures.
On 32-bit, the corresponding type is 'struct old_timespec32'.
> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> index beca249bc2f3..9de0ffc369c5 100644
> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> @@ -88,6 +88,7 @@ VERSION
> __kernel_gettimeofday;
> __kernel_clock_gettime;
> __kernel_clock_getres;
> + __kernel_time;
> local: *;
> };
> }
I would prefer to not add any deprecated interfaces in the VDSO. If we
have the 64-bit version of clock_gettime, we don't need the 32-bit version
of it, and we don't need gettimeofday() or time() either. The C library
can easily implement those by calling into clock_gettime.
> +notrace int __kernel_clock_gettime(clockid_t clock, struct timespec *ts)
> +{
> + return __cvdso_clock_gettime(clock, ts);
> +}
> +
> +notrace int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz)
> +{
> + return __cvdso_gettimeofday(tv, tz);
> +}
> +
> +notrace time_t __kernel_time(time_t *time)
> +{
> + return __cvdso_time(time);
> +}
> +
> +notrace int __kernel_clock_getres(clockid_t clock_id, struct timespec *res)
> +{
> + return __cvdso_clock_getres(clock_id, res);
> +}
> +
These should use __kernel_old_timeval and __kernel_timespec again.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 17:23 ` Mark Salyzyn
0 siblings, 0 replies; 144+ messages in thread
From: Mark Salyzyn @ 2018-11-09 17:23 UTC (permalink / raw)
To: Arnd Bergmann, vincenzo.frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Thomas Gleixner, pcc, Linux ARM
On 11/09/2018 08:13 AM, Arnd Bergmann wrote:
>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>> index beca249bc2f3..9de0ffc369c5 100644
>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>> @@ -88,6 +88,7 @@ VERSION
>> __kernel_gettimeofday;
>> __kernel_clock_gettime;
>> __kernel_clock_getres;
>> + __kernel_time;
>> local: *;
>> };
>> }
> I would prefer to not add any deprecated interfaces in the VDSO. If we
> have the 64-bit version of clock_gettime, we don't need the 32-bit version
> of it, and we don't need gettimeofday() or time() either. The C library
> can easily implement those by calling into clock_gettime.
Alas time() calls are simple and in the order of get_res, the cost of
which is a five fold
improvement over the alternate higher resolution calls (clock_gettime or
up to 100 fold
on some architectures over gettimeofday). We could measure a small
improvement in
cpu utilization (and thus battery life by extension) attributed to arm64
Android by
calling __kernel_time over __kernel_clock_gettime with coarse
resolution. A game of
inches admittedly, but super KISS to add, and it is not as-if
__kernel_time existence
will cause an issue with deprecation of a system call entry point.
-- Mark
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 17:23 ` Mark Salyzyn
0 siblings, 0 replies; 144+ messages in thread
From: Mark Salyzyn @ 2018-11-09 17:23 UTC (permalink / raw)
To: linux-arm-kernel
On 11/09/2018 08:13 AM, Arnd Bergmann wrote:
>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>> index beca249bc2f3..9de0ffc369c5 100644
>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>> @@ -88,6 +88,7 @@ VERSION
>> __kernel_gettimeofday;
>> __kernel_clock_gettime;
>> __kernel_clock_getres;
>> + __kernel_time;
>> local: *;
>> };
>> }
> I would prefer to not add any deprecated interfaces in the VDSO. If we
> have the 64-bit version of clock_gettime, we don't need the 32-bit version
> of it, and we don't need gettimeofday() or time() either. The C library
> can easily implement those by calling into clock_gettime.
Alas time() calls are simple and in the order of get_res, the cost of
which is a five fold
improvement over the alternate higher resolution calls (clock_gettime or
up to 100 fold
on some architectures over gettimeofday). We could measure a small
improvement in
cpu utilization (and thus battery life by extension) attributed to arm64
Android by
calling __kernel_time over __kernel_clock_gettime with coarse
resolution. A game of
inches admittedly, but super KISS to add, and it is not as-if
__kernel_time existence
will cause an issue with deprecation of a system call entry point.
-- Mark
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 17:23 ` Mark Salyzyn
0 siblings, 0 replies; 144+ messages in thread
From: Mark Salyzyn @ 2018-11-09 17:23 UTC (permalink / raw)
To: Arnd Bergmann, vincenzo.frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, pcc
On 11/09/2018 08:13 AM, Arnd Bergmann wrote:
>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>> index beca249bc2f3..9de0ffc369c5 100644
>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>> @@ -88,6 +88,7 @@ VERSION
>> __kernel_gettimeofday;
>> __kernel_clock_gettime;
>> __kernel_clock_getres;
>> + __kernel_time;
>> local: *;
>> };
>> }
> I would prefer to not add any deprecated interfaces in the VDSO. If we
> have the 64-bit version of clock_gettime, we don't need the 32-bit version
> of it, and we don't need gettimeofday() or time() either. The C library
> can easily implement those by calling into clock_gettime.
Alas time() calls are simple and in the order of get_res, the cost of
which is a five fold
improvement over the alternate higher resolution calls (clock_gettime or
up to 100 fold
on some architectures over gettimeofday). We could measure a small
improvement in
cpu utilization (and thus battery life by extension) attributed to arm64
Android by
calling __kernel_time over __kernel_clock_gettime with coarse
resolution. A game of
inches admittedly, but super KISS to add, and it is not as-if
__kernel_time existence
will cause an issue with deprecation of a system call entry point.
-- Mark
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 21:13 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 21:13 UTC (permalink / raw)
To: salyzyn
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Thomas Gleixner, vincenzo.frascino, Peter Collingbourne,
Linux ARM
On Fri, Nov 9, 2018 at 6:23 PM Mark Salyzyn <salyzyn@android.com> wrote:
> On 11/09/2018 08:13 AM, Arnd Bergmann wrote:
> >> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> >> index beca249bc2f3..9de0ffc369c5 100644
> >> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> >> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> >> @@ -88,6 +88,7 @@ VERSION
> >> __kernel_gettimeofday;
> >> __kernel_clock_gettime;
> >> __kernel_clock_getres;
> >> + __kernel_time;
> >> local: *;
> >> };
> >> }
> > I would prefer to not add any deprecated interfaces in the VDSO. If we
> > have the 64-bit version of clock_gettime, we don't need the 32-bit version
> > of it, and we don't need gettimeofday() or time() either. The C library
> > can easily implement those by calling into clock_gettime.
>
> Alas time() calls are simple and in the order of get_res, the cost of
> which is a five fold improvement over the alternate higher resolution calls
> (clock_gettime or up to 100 fold on some architectures over gettimeofday).
>
> We could measure a small improvement in cpu utilization (and thus
> battery life by extension) attributed to arm64 Android by calling
> __kernel_time over __kernel_clock_gettime with coarse resolution.
>
> A game of inches admittedly, but super KISS to add, and it is not as-if
> __kernel_time existence will cause an issue with deprecation of a system
> call entry point.
But wouldn't it be much better to fix the application in that case?
I can't see any reason why a user would call time() so often that
it shows up in CPU profiles, when the result only changes every
few billion CPU cycles.
Moreover, most architectures cannot return a time64_t from a
system call today, so we'd have to change the system call
entry points on all of those first to implement the slowpath
for a new time64() system call.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 21:13 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 21:13 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 9, 2018 at 6:23 PM Mark Salyzyn <salyzyn@android.com> wrote:
> On 11/09/2018 08:13 AM, Arnd Bergmann wrote:
> >> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> >> index beca249bc2f3..9de0ffc369c5 100644
> >> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> >> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> >> @@ -88,6 +88,7 @@ VERSION
> >> __kernel_gettimeofday;
> >> __kernel_clock_gettime;
> >> __kernel_clock_getres;
> >> + __kernel_time;
> >> local: *;
> >> };
> >> }
> > I would prefer to not add any deprecated interfaces in the VDSO. If we
> > have the 64-bit version of clock_gettime, we don't need the 32-bit version
> > of it, and we don't need gettimeofday() or time() either. The C library
> > can easily implement those by calling into clock_gettime.
>
> Alas time() calls are simple and in the order of get_res, the cost of
> which is a five fold improvement over the alternate higher resolution calls
> (clock_gettime or up to 100 fold on some architectures over gettimeofday).
>
> We could measure a small improvement in cpu utilization (and thus
> battery life by extension) attributed to arm64 Android by calling
> __kernel_time over __kernel_clock_gettime with coarse resolution.
>
> A game of inches admittedly, but super KISS to add, and it is not as-if
> __kernel_time existence will cause an issue with deprecation of a system
> call entry point.
But wouldn't it be much better to fix the application in that case?
I can't see any reason why a user would call time() so often that
it shows up in CPU profiles, when the result only changes every
few billion CPU cycles.
Moreover, most architectures cannot return a time64_t from a
system call today, so we'd have to change the system call
entry points on all of those first to implement the slowpath
for a new time64() system call.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 21:13 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 21:13 UTC (permalink / raw)
To: salyzyn
Cc: vincenzo.frascino, linux-arch, Linux ARM, Catalin Marinas,
Will Deacon, Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Peter Collingbourne
On Fri, Nov 9, 2018 at 6:23 PM Mark Salyzyn <salyzyn@android.com> wrote:
> On 11/09/2018 08:13 AM, Arnd Bergmann wrote:
> >> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> >> index beca249bc2f3..9de0ffc369c5 100644
> >> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> >> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> >> @@ -88,6 +88,7 @@ VERSION
> >> __kernel_gettimeofday;
> >> __kernel_clock_gettime;
> >> __kernel_clock_getres;
> >> + __kernel_time;
> >> local: *;
> >> };
> >> }
> > I would prefer to not add any deprecated interfaces in the VDSO. If we
> > have the 64-bit version of clock_gettime, we don't need the 32-bit version
> > of it, and we don't need gettimeofday() or time() either. The C library
> > can easily implement those by calling into clock_gettime.
>
> Alas time() calls are simple and in the order of get_res, the cost of
> which is a five fold improvement over the alternate higher resolution calls
> (clock_gettime or up to 100 fold on some architectures over gettimeofday).
>
> We could measure a small improvement in cpu utilization (and thus
> battery life by extension) attributed to arm64 Android by calling
> __kernel_time over __kernel_clock_gettime with coarse resolution.
>
> A game of inches admittedly, but super KISS to add, and it is not as-if
> __kernel_time existence will cause an issue with deprecation of a system
> call entry point.
But wouldn't it be much better to fix the application in that case?
I can't see any reason why a user would call time() so often that
it shows up in CPU profiles, when the result only changes every
few billion CPU cycles.
Moreover, most architectures cannot return a time64_t from a
system call today, so we'd have to change the system call
entry points on all of those first to implement the slowpath
for a new time64() system call.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-13 10:58 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:58 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, salyzyn, Paul Burton,
Thomas Gleixner, pcc, Linux ARM
On 09/11/2018 16:13, Arnd Bergmann wrote:
>> +
>> +extern struct vdso_data _vdso_data;
>> +
>> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
>> + struct timezone *_tz)
>
> I'm trying to get rid of the last users of 'struct timeval' in the kernel so
> we can remove the definition (it will clash with future glibc implementations
> in the uapi headers). Could you change this to use __kernel_old_timeval
> instead?
>
Ok, will update all the data types in v2.
...
>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>> index beca249bc2f3..9de0ffc369c5 100644
>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>> @@ -88,6 +88,7 @@ VERSION
>> __kernel_gettimeofday;
>> __kernel_clock_gettime;
>> __kernel_clock_getres;
>> + __kernel_time;
>> local: *;
>> };
>> }
>
> I would prefer to not add any deprecated interfaces in the VDSO. If we
> have the 64-bit version of clock_gettime, we don't need the 32-bit version
> of it, and we don't need gettimeofday() or time() either. The C library
> can easily implement those by calling into clock_gettime.
>
I like the idea, this would make the vdso lib code more simple and more maintainable.
In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
I am not familiar with the development plans of the various C libraries, but looking at bionic libc currently seems using all the vdso exposed functions [1].
[1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
...
>
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-13 10:58 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:58 UTC (permalink / raw)
To: linux-arm-kernel
On 09/11/2018 16:13, Arnd Bergmann wrote:
>> +
>> +extern struct vdso_data _vdso_data;
>> +
>> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
>> + struct timezone *_tz)
>
> I'm trying to get rid of the last users of 'struct timeval' in the kernel so
> we can remove the definition (it will clash with future glibc implementations
> in the uapi headers). Could you change this to use __kernel_old_timeval
> instead?
>
Ok, will update all the data types in v2.
...
>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>> index beca249bc2f3..9de0ffc369c5 100644
>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>> @@ -88,6 +88,7 @@ VERSION
>> __kernel_gettimeofday;
>> __kernel_clock_gettime;
>> __kernel_clock_getres;
>> + __kernel_time;
>> local: *;
>> };
>> }
>
> I would prefer to not add any deprecated interfaces in the VDSO. If we
> have the 64-bit version of clock_gettime, we don't need the 32-bit version
> of it, and we don't need gettimeofday() or time() either. The C library
> can easily implement those by calling into clock_gettime.
>
I like the idea, this would make the vdso lib code more simple and more maintainable.
In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
I am not familiar with the development plans of the various C libraries, but looking at bionic libc currently seems using all the vdso exposed functions [1].
[1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
...
>
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-13 10:58 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:58 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, salyzyn, pcc
On 09/11/2018 16:13, Arnd Bergmann wrote:
>> +
>> +extern struct vdso_data _vdso_data;
>> +
>> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
>> + struct timezone *_tz)
>
> I'm trying to get rid of the last users of 'struct timeval' in the kernel so
> we can remove the definition (it will clash with future glibc implementations
> in the uapi headers). Could you change this to use __kernel_old_timeval
> instead?
>
Ok, will update all the data types in v2.
...
>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>> index beca249bc2f3..9de0ffc369c5 100644
>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>> @@ -88,6 +88,7 @@ VERSION
>> __kernel_gettimeofday;
>> __kernel_clock_gettime;
>> __kernel_clock_getres;
>> + __kernel_time;
>> local: *;
>> };
>> }
>
> I would prefer to not add any deprecated interfaces in the VDSO. If we
> have the 64-bit version of clock_gettime, we don't need the 32-bit version
> of it, and we don't need gettimeofday() or time() either. The C library
> can easily implement those by calling into clock_gettime.
>
I like the idea, this would make the vdso lib code more simple and more maintainable.
In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
I am not familiar with the development plans of the various C libraries, but looking at bionic libc currently seems using all the vdso exposed functions [1].
[1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
...
>
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-14 17:47 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-14 17:47 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Mark Salyzyn,
Paul Burton, Thomas Gleixner, Peter Collingbourne, Linux ARM
On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> On 09/11/2018 16:13, Arnd Bergmann wrote:
> >> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> >> index beca249bc2f3..9de0ffc369c5 100644
> >> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> >> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> >> @@ -88,6 +88,7 @@ VERSION
> >> __kernel_gettimeofday;
> >> __kernel_clock_gettime;
> >> __kernel_clock_getres;
> >> + __kernel_time;
> >> local: *;
> >> };
> >> }
> >
> > I would prefer to not add any deprecated interfaces in the VDSO. If we
> > have the 64-bit version of clock_gettime, we don't need the 32-bit version
> > of it, and we don't need gettimeofday() or time() either. The C library
> > can easily implement those by calling into clock_gettime.
> >
>
> I like the idea, this would make the vdso lib code more simple and more maintainable.
>
> In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
>
> Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
Just to clarify: we can never remove interfaces that an older version of the
C library was using. What I'm asking is that we don't introduce any of the
unnecessary ones for architectures that don't already have them.
> I am not familiar with the development plans of the various C libraries, but looking at
> bionic libc currently seems using all the vdso exposed functions [1].
>
> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
It looks like this implementation checks for each one of them to be present
first and then uses a fallback implementation if it does not exist. This would
clearly let us remove the handlers we don't want to support, but there
are two possible downsides:
- some other libc might be lacking that fallback path
- the fallback might be much slower, e.g. time() should fallback to
the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
best performance, rather than the time() syscall or CLOCK_REALTIME.
So I'd argue that if an architecture already has a time() vdso implementation,
we probably want to keep that entry point but make it point to the best
generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-14 17:47 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-14 17:47 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> On 09/11/2018 16:13, Arnd Bergmann wrote:
> >> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> >> index beca249bc2f3..9de0ffc369c5 100644
> >> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> >> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> >> @@ -88,6 +88,7 @@ VERSION
> >> __kernel_gettimeofday;
> >> __kernel_clock_gettime;
> >> __kernel_clock_getres;
> >> + __kernel_time;
> >> local: *;
> >> };
> >> }
> >
> > I would prefer to not add any deprecated interfaces in the VDSO. If we
> > have the 64-bit version of clock_gettime, we don't need the 32-bit version
> > of it, and we don't need gettimeofday() or time() either. The C library
> > can easily implement those by calling into clock_gettime.
> >
>
> I like the idea, this would make the vdso lib code more simple and more maintainable.
>
> In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
>
> Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
Just to clarify: we can never remove interfaces that an older version of the
C library was using. What I'm asking is that we don't introduce any of the
unnecessary ones for architectures that don't already have them.
> I am not familiar with the development plans of the various C libraries, but looking at
> bionic libc currently seems using all the vdso exposed functions [1].
>
> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
It looks like this implementation checks for each one of them to be present
first and then uses a fallback implementation if it does not exist. This would
clearly let us remove the handlers we don't want to support, but there
are two possible downsides:
- some other libc might be lacking that fallback path
- the fallback might be much slower, e.g. time() should fallback to
the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
best performance, rather than the time() syscall or CLOCK_REALTIME.
So I'd argue that if an architecture already has a time() vdso implementation,
we probably want to keep that entry point but make it point to the best
generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-14 17:47 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-14 17:47 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne
On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> On 09/11/2018 16:13, Arnd Bergmann wrote:
> >> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
> >> index beca249bc2f3..9de0ffc369c5 100644
> >> --- a/arch/arm64/kernel/vdso/vdso.lds.S
> >> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
> >> @@ -88,6 +88,7 @@ VERSION
> >> __kernel_gettimeofday;
> >> __kernel_clock_gettime;
> >> __kernel_clock_getres;
> >> + __kernel_time;
> >> local: *;
> >> };
> >> }
> >
> > I would prefer to not add any deprecated interfaces in the VDSO. If we
> > have the 64-bit version of clock_gettime, we don't need the 32-bit version
> > of it, and we don't need gettimeofday() or time() either. The C library
> > can easily implement those by calling into clock_gettime.
> >
>
> I like the idea, this would make the vdso lib code more simple and more maintainable.
>
> In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
>
> Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
Just to clarify: we can never remove interfaces that an older version of the
C library was using. What I'm asking is that we don't introduce any of the
unnecessary ones for architectures that don't already have them.
> I am not familiar with the development plans of the various C libraries, but looking at
> bionic libc currently seems using all the vdso exposed functions [1].
>
> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
It looks like this implementation checks for each one of them to be present
first and then uses a fallback implementation if it does not exist. This would
clearly let us remove the handlers we don't want to support, but there
are two possible downsides:
- some other libc might be lacking that fallback path
- the fallback might be much slower, e.g. time() should fallback to
the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
best performance, rather than the time() syscall or CLOCK_REALTIME.
So I'd argue that if an architecture already has a time() vdso implementation,
we probably want to keep that entry point but make it point to the best
generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-23 10:49 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-23 10:49 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Mark Salyzyn,
Paul Burton, Thomas Gleixner, Peter Collingbourne, Linux ARM
On 11/14/18 5:47 PM, Arnd Bergmann wrote:
> On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> On 09/11/2018 16:13, Arnd Bergmann wrote:
>
>>>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>>>> index beca249bc2f3..9de0ffc369c5 100644
>>>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>>>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>>>> @@ -88,6 +88,7 @@ VERSION
>>>> __kernel_gettimeofday;
>>>> __kernel_clock_gettime;
>>>> __kernel_clock_getres;
>>>> + __kernel_time;
>>>> local: *;
>>>> };
>>>> }
>>>
>>> I would prefer to not add any deprecated interfaces in the VDSO. If we
>>> have the 64-bit version of clock_gettime, we don't need the 32-bit version
>>> of it, and we don't need gettimeofday() or time() either. The C library
>>> can easily implement those by calling into clock_gettime.
>>>
>>
>> I like the idea, this would make the vdso lib code more simple and more maintainable.
>>
>> In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
>>
>> Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
>
> Just to clarify: we can never remove interfaces that an older version of the
> C library was using. What I'm asking is that we don't introduce any of the
> unnecessary ones for architectures that don't already have them.
>
I agree, I realize now I should have worded my answer differently. With
"deprecate and remove the unused code" was referring to redirect gettimeofday
through clock_gettime, not to remove completely gettimeofday interface since
this would cause userspace breakage for old C libraries as you are pointing out
as well.
>> I am not familiar with the development plans of the various C libraries, but looking at
>> bionic libc currently seems using all the vdso exposed functions [1].
>>
>> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
>
> It looks like this implementation checks for each one of them to be present
> first and then uses a fallback implementation if it does not exist. This would
> clearly let us remove the handlers we don't want to support, but there
> are two possible downsides:
>> - some other libc might be lacking that fallback path
> - the fallback might be much slower, e.g. time() should fallback to
> the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
> best performance, rather than the time() syscall or CLOCK_REALTIME.
>
> So I'd argue that if an architecture already has a time() vdso implementation,
> we probably want to keep that entry point but make it point to the best
> generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
What I was trying to point out here is that if the symbol is present in the vdso
library the C library implementation tends to prefer to use it.
Said that, I agree with what you are saying and in this first iteration I will
make sure that we do not add new symbols for the architectures that did not
support them previously, but I will keep the library the most generic possible,
I would prefer though to introduce the redirection for gettimeofday() and time()
to the best performing clock_gettime() at a later stage (and with a separate
patchset) because it would involve some cleanup in the vdso.c of the
architectures that would use the library (i.e. update_vsyscall_tz() not required
anymore) and because based on what I measured with a quick test the performance
difference seems small with the current implementation of the library.
>
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-23 10:49 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-23 10:49 UTC (permalink / raw)
To: linux-arm-kernel
On 11/14/18 5:47 PM, Arnd Bergmann wrote:
> On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> On 09/11/2018 16:13, Arnd Bergmann wrote:
>
>>>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>>>> index beca249bc2f3..9de0ffc369c5 100644
>>>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>>>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>>>> @@ -88,6 +88,7 @@ VERSION
>>>> __kernel_gettimeofday;
>>>> __kernel_clock_gettime;
>>>> __kernel_clock_getres;
>>>> + __kernel_time;
>>>> local: *;
>>>> };
>>>> }
>>>
>>> I would prefer to not add any deprecated interfaces in the VDSO. If we
>>> have the 64-bit version of clock_gettime, we don't need the 32-bit version
>>> of it, and we don't need gettimeofday() or time() either. The C library
>>> can easily implement those by calling into clock_gettime.
>>>
>>
>> I like the idea, this would make the vdso lib code more simple and more maintainable.
>>
>> In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
>>
>> Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
>
> Just to clarify: we can never remove interfaces that an older version of the
> C library was using. What I'm asking is that we don't introduce any of the
> unnecessary ones for architectures that don't already have them.
>
I agree, I realize now I should have worded my answer differently. With
"deprecate and remove the unused code" was referring to redirect gettimeofday
through clock_gettime, not to remove completely gettimeofday interface since
this would cause userspace breakage for old C libraries as you are pointing out
as well.
>> I am not familiar with the development plans of the various C libraries, but looking at
>> bionic libc currently seems using all the vdso exposed functions [1].
>>
>> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
>
> It looks like this implementation checks for each one of them to be present
> first and then uses a fallback implementation if it does not exist. This would
> clearly let us remove the handlers we don't want to support, but there
> are two possible downsides:
>> - some other libc might be lacking that fallback path
> - the fallback might be much slower, e.g. time() should fallback to
> the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
> best performance, rather than the time() syscall or CLOCK_REALTIME.
>
> So I'd argue that if an architecture already has a time() vdso implementation,
> we probably want to keep that entry point but make it point to the best
> generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
What I was trying to point out here is that if the symbol is present in the vdso
library the C library implementation tends to prefer to use it.
Said that, I agree with what you are saying and in this first iteration I will
make sure that we do not add new symbols for the architectures that did not
support them previously, but I will keep the library the most generic possible,
I would prefer though to introduce the redirection for gettimeofday() and time()
to the best performing clock_gettime() at a later stage (and with a separate
patchset) because it would involve some cleanup in the vdso.c of the
architectures that would use the library (i.e. update_vsyscall_tz() not required
anymore) and because based on what I measured with a quick test the performance
difference seems small with the current implementation of the library.
>
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-23 10:49 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-23 10:49 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne
On 11/14/18 5:47 PM, Arnd Bergmann wrote:
> On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> On 09/11/2018 16:13, Arnd Bergmann wrote:
>
>>>> diff --git a/arch/arm64/kernel/vdso/vdso.lds.S b/arch/arm64/kernel/vdso/vdso.lds.S
>>>> index beca249bc2f3..9de0ffc369c5 100644
>>>> --- a/arch/arm64/kernel/vdso/vdso.lds.S
>>>> +++ b/arch/arm64/kernel/vdso/vdso.lds.S
>>>> @@ -88,6 +88,7 @@ VERSION
>>>> __kernel_gettimeofday;
>>>> __kernel_clock_gettime;
>>>> __kernel_clock_getres;
>>>> + __kernel_time;
>>>> local: *;
>>>> };
>>>> }
>>>
>>> I would prefer to not add any deprecated interfaces in the VDSO. If we
>>> have the 64-bit version of clock_gettime, we don't need the 32-bit version
>>> of it, and we don't need gettimeofday() or time() either. The C library
>>> can easily implement those by calling into clock_gettime.
>>>
>>
>> I like the idea, this would make the vdso lib code more simple and more maintainable.
>>
>> In this patchset I tried to cover the widest possible scenario making things configurable: each architecture can select and enable exactly what it needs from the vdso common code.
>>
>> Based on what you are proposing, once the C library will implement things in this way, it will be easy to deprecate and remove the unused code.
>
> Just to clarify: we can never remove interfaces that an older version of the
> C library was using. What I'm asking is that we don't introduce any of the
> unnecessary ones for architectures that don't already have them.
>
I agree, I realize now I should have worded my answer differently. With
"deprecate and remove the unused code" was referring to redirect gettimeofday
through clock_gettime, not to remove completely gettimeofday interface since
this would cause userspace breakage for old C libraries as you are pointing out
as well.
>> I am not familiar with the development plans of the various C libraries, but looking at
>> bionic libc currently seems using all the vdso exposed functions [1].
>>
>> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
>
> It looks like this implementation checks for each one of them to be present
> first and then uses a fallback implementation if it does not exist. This would
> clearly let us remove the handlers we don't want to support, but there
> are two possible downsides:
>> - some other libc might be lacking that fallback path
> - the fallback might be much slower, e.g. time() should fallback to
> the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
> best performance, rather than the time() syscall or CLOCK_REALTIME.
>
> So I'd argue that if an architecture already has a time() vdso implementation,
> we probably want to keep that entry point but make it point to the best
> generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
What I was trying to point out here is that if the symbol is present in the vdso
library the C library implementation tends to prefer to use it.
Said that, I agree with what you are saying and in this first iteration I will
make sure that we do not add new symbols for the architectures that did not
support them previously, but I will keep the library the most generic possible,
I would prefer though to introduce the redirection for gettimeofday() and time()
to the best performing clock_gettime() at a later stage (and with a separate
patchset) because it would involve some cleanup in the vdso.c of the
architectures that would use the library (i.e. update_vsyscall_tz() not required
anymore) and because based on what I measured with a quick test the performance
difference seems small with the current implementation of the library.
>
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-23 11:14 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-23 11:14 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Mark Salyzyn,
Paul Burton, Thomas Gleixner, Peter Collingbourne, Linux ARM
On Fri, Nov 23, 2018 at 11:48 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> On 11/14/18 5:47 PM, Arnd Bergmann wrote:
> > On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >>
> >> On 09/11/2018 16:13, Arnd Bergmann wrote:
> >> I am not familiar with the development plans of the various C libraries, but looking at
> >> bionic libc currently seems using all the vdso exposed functions [1].
> >>
> >> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
> >
> > It looks like this implementation checks for each one of them to be present
> > first and then uses a fallback implementation if it does not exist. This would
> > clearly let us remove the handlers we don't want to support, but there
> > are two possible downsides:
> >> - some other libc might be lacking that fallback path
> > - the fallback might be much slower, e.g. time() should fallback to
> > the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
> > best performance, rather than the time() syscall or CLOCK_REALTIME.
> >
> > So I'd argue that if an architecture already has a time() vdso implementation,
> > we probably want to keep that entry point but make it point to the best
> > generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
>
> What I was trying to point out here is that if the symbol is present in the vdso
> library the C library implementation tends to prefer to use it.
>
> Said that, I agree with what you are saying and in this first iteration I will
> make sure that we do not add new symbols for the architectures that did not
> support them previously, but I will keep the library the most generic possible,
>
> I would prefer though to introduce the redirection for gettimeofday() and time()
> to the best performing clock_gettime() at a later stage (and with a separate
> patchset) because it would involve some cleanup in the vdso.c of the
> architectures that would use the library (i.e. update_vsyscall_tz() not required
> anymore) and because based on what I measured with a quick test the performance
> difference seems small with the current implementation of the library.
Ok, makes sense.
Thanks,
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-23 11:14 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-23 11:14 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 23, 2018 at 11:48 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> On 11/14/18 5:47 PM, Arnd Bergmann wrote:
> > On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >>
> >> On 09/11/2018 16:13, Arnd Bergmann wrote:
> >> I am not familiar with the development plans of the various C libraries, but looking at
> >> bionic libc currently seems using all the vdso exposed functions [1].
> >>
> >> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
> >
> > It looks like this implementation checks for each one of them to be present
> > first and then uses a fallback implementation if it does not exist. This would
> > clearly let us remove the handlers we don't want to support, but there
> > are two possible downsides:
> >> - some other libc might be lacking that fallback path
> > - the fallback might be much slower, e.g. time() should fallback to
> > the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
> > best performance, rather than the time() syscall or CLOCK_REALTIME.
> >
> > So I'd argue that if an architecture already has a time() vdso implementation,
> > we probably want to keep that entry point but make it point to the best
> > generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
>
> What I was trying to point out here is that if the symbol is present in the vdso
> library the C library implementation tends to prefer to use it.
>
> Said that, I agree with what you are saying and in this first iteration I will
> make sure that we do not add new symbols for the architectures that did not
> support them previously, but I will keep the library the most generic possible,
>
> I would prefer though to introduce the redirection for gettimeofday() and time()
> to the best performing clock_gettime() at a later stage (and with a separate
> patchset) because it would involve some cleanup in the vdso.c of the
> architectures that would use the library (i.e. update_vsyscall_tz() not required
> anymore) and because based on what I measured with a quick test the performance
> difference seems small with the current implementation of the library.
Ok, makes sense.
Thanks,
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-23 11:14 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-23 11:14 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne
On Fri, Nov 23, 2018 at 11:48 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> On 11/14/18 5:47 PM, Arnd Bergmann wrote:
> > On Tue, Nov 13, 2018 at 2:58 AM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >>
> >> On 09/11/2018 16:13, Arnd Bergmann wrote:
> >> I am not familiar with the development plans of the various C libraries, but looking at
> >> bionic libc currently seems using all the vdso exposed functions [1].
> >>
> >> [1] https://github.com/aosp-mirror/platform_bionic/blob/master/libc/bionic/vdso.cpp
> >
> > It looks like this implementation checks for each one of them to be present
> > first and then uses a fallback implementation if it does not exist. This would
> > clearly let us remove the handlers we don't want to support, but there
> > are two possible downsides:
> >> - some other libc might be lacking that fallback path
> > - the fallback might be much slower, e.g. time() should fallback to
> > the vdso version of clock_gettime(CLOCK_REALTIME_COARSE) for
> > best performance, rather than the time() syscall or CLOCK_REALTIME.
> >
> > So I'd argue that if an architecture already has a time() vdso implementation,
> > we probably want to keep that entry point but make it point to the best
> > generic implementation (i.e. clock_gettime CLOCK_REALTIME_COARSE).
>
> What I was trying to point out here is that if the symbol is present in the vdso
> library the C library implementation tends to prefer to use it.
>
> Said that, I agree with what you are saying and in this first iteration I will
> make sure that we do not add new symbols for the architectures that did not
> support them previously, but I will keep the library the most generic possible,
>
> I would prefer though to introduce the redirection for gettimeofday() and time()
> to the best performing clock_gettime() at a later stage (and with a separate
> patchset) because it would involve some cleanup in the vdso.c of the
> architectures that would use the library (i.e. update_vsyscall_tz() not required
> anymore) and because based on what I measured with a quick test the performance
> difference seems small with the current implementation of the library.
Ok, makes sense.
Thanks,
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 18:09 ` Peter Collingbourne
0 siblings, 0 replies; 144+ messages in thread
From: Peter Collingbourne @ 2018-11-09 18:09 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, arnd, catalin.marinas, daniel.lezcano, will.deacon,
linux, ralf, salyzyn, paul.burton, tglx, linux-arm-kernel
On Fri, Nov 9, 2018 at 4:38 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> -ccflags-y := -shared -fno-common -fno-builtin
> +ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
This either needs to include -ffixed-x18 or
https://patchwork.kernel.org/patch/10671685/ needs to appear earlier
in this series.
Peter
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 18:09 ` Peter Collingbourne
0 siblings, 0 replies; 144+ messages in thread
From: Peter Collingbourne @ 2018-11-09 18:09 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 9, 2018 at 4:38 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> -ccflags-y := -shared -fno-common -fno-builtin
> +ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
This either needs to include -ffixed-x18 or
https://patchwork.kernel.org/patch/10671685/ needs to appear earlier
in this series.
Peter
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-09 18:09 ` Peter Collingbourne
0 siblings, 0 replies; 144+ messages in thread
From: Peter Collingbourne @ 2018-11-09 18:09 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, linux-arm-kernel, catalin.marinas, will.deacon, arnd,
linux, ralf, paul.burton, daniel.lezcano, tglx, salyzyn
On Fri, Nov 9, 2018 at 4:38 AM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> -ccflags-y := -shared -fno-common -fno-builtin
> +ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
This either needs to include -ffixed-x18 or
https://patchwork.kernel.org/patch/10671685/ needs to appear earlier
in this series.
Peter
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-13 10:36 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:36 UTC (permalink / raw)
To: Peter Collingbourne
Cc: linux-arch, arnd, catalin.marinas, daniel.lezcano, will.deacon,
linux, ralf, salyzyn, paul.burton, tglx, linux-arm-kernel
Hi Peter,
thank you for your review.
On 09/11/2018 18:09, Peter Collingbourne wrote:
> On Fri, Nov 9, 2018 at 4:38 AM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>> -ccflags-y := -shared -fno-common -fno-builtin
>> +ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
>
> This either needs to include -ffixed-x18 or
> https://patchwork.kernel.org/patch/10671685/ needs to appear earlier
> in this series.
>
I did not add -ffixed-x18 to my flags because I was planning to cherry-pick your patch. It will be in v2.
> Peter
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-13 10:36 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:36 UTC (permalink / raw)
To: linux-arm-kernel
Hi Peter,
thank you for your review.
On 09/11/2018 18:09, Peter Collingbourne wrote:
> On Fri, Nov 9, 2018 at 4:38 AM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>> -ccflags-y := -shared -fno-common -fno-builtin
>> +ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
>
> This either needs to include -ffixed-x18 or
> https://patchwork.kernel.org/patch/10671685/ needs to appear earlier
> in this series.
>
I did not add -ffixed-x18 to my flags because I was planning to cherry-pick your patch. It will be in v2.
> Peter
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-13 10:36 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-13 10:36 UTC (permalink / raw)
To: Peter Collingbourne
Cc: linux-arch, linux-arm-kernel, catalin.marinas, will.deacon, arnd,
linux, ralf, paul.burton, daniel.lezcano, tglx, salyzyn
Hi Peter,
thank you for your review.
On 09/11/2018 18:09, Peter Collingbourne wrote:
> On Fri, Nov 9, 2018 at 4:38 AM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>> -ccflags-y := -shared -fno-common -fno-builtin
>> +ccflags-y := -shared -fno-common -fno-builtin -fno-stack-protector
>
> This either needs to include -ffixed-x18 or
> https://patchwork.kernel.org/patch/10671685/ needs to appear earlier
> in this series.
>
I did not add -ffixed-x18 to my flags because I was planning to cherry-pick your patch. It will be in v2.
> Peter
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-10 0:29 ` David Miller
0 siblings, 0 replies; 144+ messages in thread
From: David Miller @ 2018-11-10 0:29 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, arnd, catalin.marinas, daniel.lezcano, will.deacon,
linux, ralf, salyzyn, paul.burton, tglx, pcc, linux-arm-kernel
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Date: Fri, 9 Nov 2018 12:37:10 +0000
> To take advantage of the commonly defined vdso interface for
> gettimeofday the architectural code requires an adaptation.
>
> This patch re-implements the gettimeofday vdso in C in order to use
> lib/vdso.
>
> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
> CLOCK_TAI and __kernel_time.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
What is the performance penalty for going to C from ASM like this?
I like the vDSO consolidation, but actually I think it makes a lot of
sense to implement these things in assembler with some C implementation
as guidance or for architectures that haven't optimized into an ASM
version yet.
I'd also like to see if we can somehow make PowerPC's optimizations in
this area usable in more places.
It defines a reciprocal in order to use a multiply instead of a divide
after capturing the time values.
I say all of this because I want to do something similar on sparc, and
having a generic implementation of all of the math would make that easier
for me and other architecture maintainers who might want to do the same.
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-10 0:29 ` David Miller
0 siblings, 0 replies; 144+ messages in thread
From: David Miller @ 2018-11-10 0:29 UTC (permalink / raw)
To: linux-arm-kernel
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Date: Fri, 9 Nov 2018 12:37:10 +0000
> To take advantage of the commonly defined vdso interface for
> gettimeofday the architectural code requires an adaptation.
>
> This patch re-implements the gettimeofday vdso in C in order to use
> lib/vdso.
>
> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
> CLOCK_TAI and __kernel_time.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
What is the performance penalty for going to C from ASM like this?
I like the vDSO consolidation, but actually I think it makes a lot of
sense to implement these things in assembler with some C implementation
as guidance or for architectures that haven't optimized into an ASM
version yet.
I'd also like to see if we can somehow make PowerPC's optimizations in
this area usable in more places.
It defines a reciprocal in order to use a multiply instead of a divide
after capturing the time values.
I say all of this because I want to do something similar on sparc, and
having a generic implementation of all of the math would make that easier
for me and other architecture maintainers who might want to do the same.
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-10 0:29 ` David Miller
0 siblings, 0 replies; 144+ messages in thread
From: David Miller @ 2018-11-10 0:29 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, linux-arm-kernel, catalin.marinas, will.deacon, arnd,
linux, ralf, paul.burton, daniel.lezcano, tglx, salyzyn, pcc
From: Vincenzo Frascino <vincenzo.frascino@arm.com>
Date: Fri, 9 Nov 2018 12:37:10 +0000
> To take advantage of the commonly defined vdso interface for
> gettimeofday the architectural code requires an adaptation.
>
> This patch re-implements the gettimeofday vdso in C in order to use
> lib/vdso.
>
> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
> CLOCK_TAI and __kernel_time.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
What is the performance penalty for going to C from ASM like this?
I like the vDSO consolidation, but actually I think it makes a lot of
sense to implement these things in assembler with some C implementation
as guidance or for architectures that haven't optimized into an ASM
version yet.
I'd also like to see if we can somehow make PowerPC's optimizations in
this area usable in more places.
It defines a reciprocal in order to use a multiply instead of a divide
after capturing the time values.
I say all of this because I want to do something similar on sparc, and
having a generic implementation of all of the math would make that easier
for me and other architecture maintainers who might want to do the same.
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-12 18:21 ` Mark Salyzyn
0 siblings, 0 replies; 144+ messages in thread
From: Mark Salyzyn @ 2018-11-12 18:21 UTC (permalink / raw)
To: David Miller, vincenzo.frascino
Cc: linux-arch, arnd, catalin.marinas, daniel.lezcano, will.deacon,
linux, ralf, paul.burton, tglx, pcc, linux-arm-kernel
On 11/09/2018 04:29 PM, David Miller wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Date: Fri, 9 Nov 2018 12:37:10 +0000
>
>> To take advantage of the commonly defined vdso interface for
>> gettimeofday the architectural code requires an adaptation.
>>
>> This patch re-implements the gettimeofday vdso in C in order to use
>> lib/vdso.
>>
>> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
>> CLOCK_TAI and __kernel_time.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> What is the performance penalty for going to C from ASM like this?
>
> I like the vDSO consolidation, but actually I think it makes a lot of
> sense to implement these things in assembler with some C implementation
> as guidance or for architectures that haven't optimized into an ASM
> version yet.
>
> I'd also like to see if we can somehow make PowerPC's optimizations in
> this area usable in more places.
>
> It defines a reciprocal in order to use a multiply instead of a divide
> after capturing the time values.
>
> I say all of this because I want to do something similar on sparc, and
> having a generic implementation of all of the math would make that easier
> for me and other architecture maintainers who might want to do the same.
I saw an _improvement_ in performance for the C implementation (for
arm64), the optimizer did a better job with the specific processor
architecture and actually killed a few bugs in the assembler
implementation. The improvement was in the order of a nano-second or two
in micro-benchmarks.
I used the bionic benchmarks to measure the performance.
-- Mark
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-12 18:21 ` Mark Salyzyn
0 siblings, 0 replies; 144+ messages in thread
From: Mark Salyzyn @ 2018-11-12 18:21 UTC (permalink / raw)
To: linux-arm-kernel
On 11/09/2018 04:29 PM, David Miller wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Date: Fri, 9 Nov 2018 12:37:10 +0000
>
>> To take advantage of the commonly defined vdso interface for
>> gettimeofday the architectural code requires an adaptation.
>>
>> This patch re-implements the gettimeofday vdso in C in order to use
>> lib/vdso.
>>
>> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
>> CLOCK_TAI and __kernel_time.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> What is the performance penalty for going to C from ASM like this?
>
> I like the vDSO consolidation, but actually I think it makes a lot of
> sense to implement these things in assembler with some C implementation
> as guidance or for architectures that haven't optimized into an ASM
> version yet.
>
> I'd also like to see if we can somehow make PowerPC's optimizations in
> this area usable in more places.
>
> It defines a reciprocal in order to use a multiply instead of a divide
> after capturing the time values.
>
> I say all of this because I want to do something similar on sparc, and
> having a generic implementation of all of the math would make that easier
> for me and other architecture maintainers who might want to do the same.
I saw an _improvement_ in performance for the C implementation (for
arm64), the optimizer did a better job with the specific processor
architecture and actually killed a few bugs in the assembler
implementation. The improvement was in the order of a nano-second or two
in micro-benchmarks.
I used the bionic benchmarks to measure the performance.
-- Mark
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-12 18:21 ` Mark Salyzyn
0 siblings, 0 replies; 144+ messages in thread
From: Mark Salyzyn @ 2018-11-12 18:21 UTC (permalink / raw)
To: David Miller, vincenzo.frascino
Cc: linux-arch, linux-arm-kernel, catalin.marinas, will.deacon, arnd,
linux, ralf, paul.burton, daniel.lezcano, tglx, pcc
On 11/09/2018 04:29 PM, David Miller wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Date: Fri, 9 Nov 2018 12:37:10 +0000
>
>> To take advantage of the commonly defined vdso interface for
>> gettimeofday the architectural code requires an adaptation.
>>
>> This patch re-implements the gettimeofday vdso in C in order to use
>> lib/vdso.
>>
>> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
>> CLOCK_TAI and __kernel_time.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> What is the performance penalty for going to C from ASM like this?
>
> I like the vDSO consolidation, but actually I think it makes a lot of
> sense to implement these things in assembler with some C implementation
> as guidance or for architectures that haven't optimized into an ASM
> version yet.
>
> I'd also like to see if we can somehow make PowerPC's optimizations in
> this area usable in more places.
>
> It defines a reciprocal in order to use a multiply instead of a divide
> after capturing the time values.
>
> I say all of this because I want to do something similar on sparc, and
> having a generic implementation of all of the math would make that easier
> for me and other architecture maintainers who might want to do the same.
I saw an _improvement_ in performance for the C implementation (for
arm64), the optimizer did a better job with the specific processor
architecture and actually killed a few bugs in the assembler
implementation. The improvement was in the order of a nano-second or two
in micro-benchmarks.
I used the bionic benchmarks to measure the performance.
-- Mark
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-14 10:22 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-14 10:22 UTC (permalink / raw)
To: David Miller
Cc: linux-arch, arnd, catalin.marinas, daniel.lezcano, will.deacon,
linux, ralf, salyzyn, paul.burton, tglx, pcc, linux-arm-kernel
Hi David,
thank you for reviewing my code.
On 10/11/2018 00:29, David Miller wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Date: Fri, 9 Nov 2018 12:37:10 +0000
>
>> To take advantage of the commonly defined vdso interface for
>> gettimeofday the architectural code requires an adaptation.
>>
>> This patch re-implements the gettimeofday vdso in C in order to use
>> lib/vdso.
>>
>> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
>> CLOCK_TAI and __kernel_time.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> What is the performance penalty for going to C from ASM like this?
>
> I like the vDSO consolidation, but actually I think it makes a lot of
> sense to implement these things in assembler with some C implementation
> as guidance or for architectures that haven't optimized into an ASM
> version yet.
>
> I'd also like to see if we can somehow make PowerPC's optimizations in
> this area usable in more places.
>
> It defines a reciprocal in order to use a multiply instead of a divide
> after capturing the time values.
> > I say all of this because I want to do something similar on sparc, and
> having a generic implementation of all of the math would make that easier
> for me and other architecture maintainers who might want to do the same.
>
I did run some benchmarks on arm64 with both vdsotest-bench (I added a
script to the vdsotest implementation that can be used to replicate the
results) and an ad-hoc test that stresses the vdso library and the
differences in terms of performance penalty are very little (~1%) to
none in some cases.
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-14 10:22 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-14 10:22 UTC (permalink / raw)
To: linux-arm-kernel
Hi David,
thank you for reviewing my code.
On 10/11/2018 00:29, David Miller wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Date: Fri, 9 Nov 2018 12:37:10 +0000
>
>> To take advantage of the commonly defined vdso interface for
>> gettimeofday the architectural code requires an adaptation.
>>
>> This patch re-implements the gettimeofday vdso in C in order to use
>> lib/vdso.
>>
>> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
>> CLOCK_TAI and __kernel_time.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> What is the performance penalty for going to C from ASM like this?
>
> I like the vDSO consolidation, but actually I think it makes a lot of
> sense to implement these things in assembler with some C implementation
> as guidance or for architectures that haven't optimized into an ASM
> version yet.
>
> I'd also like to see if we can somehow make PowerPC's optimizations in
> this area usable in more places.
>
> It defines a reciprocal in order to use a multiply instead of a divide
> after capturing the time values.
> > I say all of this because I want to do something similar on sparc, and
> having a generic implementation of all of the math would make that easier
> for me and other architecture maintainers who might want to do the same.
>
I did run some benchmarks on arm64 with both vdsotest-bench (I added a
script to the vdsotest implementation that can be used to replicate the
results) and an ad-hoc test that stresses the vdso library and the
differences in terms of performance penalty are very little (~1%) to
none in some cases.
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 07/27] arm64: Substitute gettimeofday with C implementation
@ 2018-11-14 10:22 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-14 10:22 UTC (permalink / raw)
To: David Miller
Cc: linux-arch, linux-arm-kernel, catalin.marinas, will.deacon, arnd,
linux, ralf, paul.burton, daniel.lezcano, tglx, salyzyn, pcc
Hi David,
thank you for reviewing my code.
On 10/11/2018 00:29, David Miller wrote:
> From: Vincenzo Frascino <vincenzo.frascino@arm.com>
> Date: Fri, 9 Nov 2018 12:37:10 +0000
>
>> To take advantage of the commonly defined vdso interface for
>> gettimeofday the architectural code requires an adaptation.
>>
>> This patch re-implements the gettimeofday vdso in C in order to use
>> lib/vdso.
>>
>> With the new implementation arm64 gains support for CLOCK_BOOTTIME,
>> CLOCK_TAI and __kernel_time.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> What is the performance penalty for going to C from ASM like this?
>
> I like the vDSO consolidation, but actually I think it makes a lot of
> sense to implement these things in assembler with some C implementation
> as guidance or for architectures that haven't optimized into an ASM
> version yet.
>
> I'd also like to see if we can somehow make PowerPC's optimizations in
> this area usable in more places.
>
> It defines a reciprocal in order to use a multiply instead of a divide
> after capturing the time values.
> > I say all of this because I want to do something similar on sparc, and
> having a generic implementation of all of the math would make that easier
> for me and other architecture maintainers who might want to do the same.
>
I did run some benchmarks on arm64 with both vdsotest-bench (I added a
script to the vdsotest implementation that can be used to replicate the
results) and an ad-hoc test that stresses the vdso library and the
differences in terms of performance penalty are very little (~1%) to
none in some cases.
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 08/27] arm64: compat: Alloc separate pages for vectors and sigpage
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
This patch addresses the problem creating separate pages for vectors and
sigpage in a similar fashion to what happens today on arm.
This patch changes as well the meaning of mm->context.vdso for AArch32
compat since it now points to sigpage and not to vectors anymore in
order to make simpler the implementation of the signal handling (the
address of sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 433b9554c6a1..f2b6b486deb7 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -210,10 +210,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 3e2091708b8e..8b1fb27a897e 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -59,9 +59,9 @@
#define STACK_TOP_MAX TASK_SIZE_64
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 24b09003f821..265a6fab331d 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 47834fe0bbe3..88b674079d07 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -53,61 +55,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 08/27] arm64: compat: Alloc separate pages for vectors and sigpage
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
This patch addresses the problem creating separate pages for vectors and
sigpage in a similar fashion to what happens today on arm.
This patch changes as well the meaning of mm->context.vdso for AArch32
compat since it now points to sigpage and not to vectors anymore in
order to make simpler the implementation of the signal handling (the
address of sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 433b9554c6a1..f2b6b486deb7 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -210,10 +210,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 3e2091708b8e..8b1fb27a897e 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -59,9 +59,9 @@
#define STACK_TOP_MAX TASK_SIZE_64
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 24b09003f821..265a6fab331d 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 47834fe0bbe3..88b674079d07 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -53,61 +55,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 08/27] arm64: compat: Alloc separate pages for vectors and sigpage
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
This patch addresses the problem creating separate pages for vectors and
sigpage in a similar fashion to what happens today on arm.
This patch changes as well the meaning of mm->context.vdso for AArch32
compat since it now points to sigpage and not to vectors anymore in
order to make simpler the implementation of the signal handling (the
address of sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 433b9554c6a1..f2b6b486deb7 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -210,10 +210,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 3e2091708b8e..8b1fb27a897e 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -59,9 +59,9 @@
#define STACK_TOP_MAX TASK_SIZE_64
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 24b09003f821..265a6fab331d 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 47834fe0bbe3..88b674079d07 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -53,61 +55,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 09/27] arm64: compat: Split kuser32
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
This patch splits the current version of kuser32 in kuser32 (for kuser
helpers) and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 4c8b13bede80..6bd4f619732a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 09/27] arm64: compat: Split kuser32
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
This patch splits the current version of kuser32 in kuser32 (for kuser
helpers) and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 4c8b13bede80..6bd4f619732a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 09/27] arm64: compat: Split kuser32
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
This patch splits the current version of kuser32 in kuser32 (for kuser
helpers) and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 4c8b13bede80..6bd4f619732a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 10/27] arm64: compat: Refactor aarch32_alloc_vdso_pages()
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
This patch divides the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 88b674079d07..cee7205eefc5 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -74,40 +74,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 10/27] arm64: compat: Refactor aarch32_alloc_vdso_pages()
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
This patch divides the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 88b674079d07..cee7205eefc5 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -74,40 +74,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 10/27] arm64: compat: Refactor aarch32_alloc_vdso_pages()
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
This patch divides the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 88b674079d07..cee7205eefc5 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -74,40 +74,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 11/27] arm64: compat: Add KUSER_HELPERS config option
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
This patch adds a KUSER_HELPERS config option that can be used to
disable the mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 71ca1995a088..2c7d447401b5 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1333,6 +1333,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 6bd4f619732a..2fcba6b87829 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index cee7205eefc5..8903087cca4c 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -74,6 +74,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -95,6 +96,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -126,6 +133,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -138,6 +146,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 11/27] arm64: compat: Add KUSER_HELPERS config option
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
This patch adds a KUSER_HELPERS config option that can be used to
disable the mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 71ca1995a088..2c7d447401b5 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1333,6 +1333,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 6bd4f619732a..2fcba6b87829 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index cee7205eefc5..8903087cca4c 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -74,6 +74,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -95,6 +96,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -126,6 +133,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -138,6 +146,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 11/27] arm64: compat: Add KUSER_HELPERS config option
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
This patch adds a KUSER_HELPERS config option that can be used to
disable the mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 71ca1995a088..2c7d447401b5 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1333,6 +1333,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 6bd4f619732a..2fcba6b87829 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
arm64-obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index cee7205eefc5..8903087cca4c 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -74,6 +74,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -95,6 +96,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -126,6 +133,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -138,6 +146,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 12/27] arm64: compat: Add missing syscall numbers
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
This patch adds the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index b13ca091f833..a85b30b766a1 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 12/27] arm64: compat: Add missing syscall numbers
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
This patch adds the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index b13ca091f833..a85b30b766a1 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 12/27] arm64: compat: Add missing syscall numbers
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
This patch adds the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index b13ca091f833..a85b30b766a1 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 13/27] arm64: compat: Expose signal related structures
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
This patch exposes the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 265a6fab331d..16c490a83131 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 13/27] arm64: compat: Expose signal related structures
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
This patch exposes the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 265a6fab331d..16c490a83131 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 13/27] arm64: compat: Expose signal related structures
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
This patch exposes the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 265a6fab331d..16c490a83131 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 14/27] arm64: compat: Generate asm offsets for signals
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch updates asm-offsets for arm64 to generate the correct offsets
for compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 06cfd2363a86..4f8317c23908 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -81,6 +82,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 14/27] arm64: compat: Generate asm offsets for signals
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
This patch updates asm-offsets for arm64 to generate the correct offsets
for compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 06cfd2363a86..4f8317c23908 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -81,6 +82,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 14/27] arm64: compat: Generate asm offsets for signals
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch updates asm-offsets for arm64 to generate the correct offsets
for compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 06cfd2363a86..4f8317c23908 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -81,6 +82,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 15/27] lib: vdso: Add compat support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
This patch adds support to the generic code for compat fallback
functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index ec903180d3da..572ac162bdac 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -15,7 +15,11 @@
#include <linux/hrtimer.h>
#include <vdso/datapage.h>
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
/* To improve performances, in this file, __always_inline it is used
* for the functions called multiple times.
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 15/27] lib: vdso: Add compat support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
This patch adds support to the generic code for compat fallback
functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index ec903180d3da..572ac162bdac 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -15,7 +15,11 @@
#include <linux/hrtimer.h>
#include <vdso/datapage.h>
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
/* To improve performances, in this file, __always_inline it is used
* for the functions called multiple times.
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 15/27] lib: vdso: Add compat support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
This patch adds support to the generic code for compat fallback
functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index ec903180d3da..572ac162bdac 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -15,7 +15,11 @@
#include <linux/hrtimer.h>
#include <vdso/datapage.h>
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
/* To improve performances, in this file, __always_inline it is used
* for the functions called multiple times.
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 16/27] arm64: compat: Add vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch provides the arm64 compat (AArch32) vDSO in kernel/vdso32 in
a similar way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of __vdso_time
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 89 +++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 82 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 43 ++++
9 files changed, 532 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..2c86fae08642
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..a4673200f3c9
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..9c01b4e50e65
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __vdso_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 16/27] arm64: compat: Add vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
This patch provides the arm64 compat (AArch32) vDSO in kernel/vdso32 in
a similar way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of __vdso_time
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 89 +++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 82 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 43 ++++
9 files changed, 532 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..2c86fae08642
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..a4673200f3c9
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..9c01b4e50e65
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __vdso_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 16/27] arm64: compat: Add vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch provides the arm64 compat (AArch32) vDSO in kernel/vdso32 in
a similar way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of __vdso_time
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 89 +++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 82 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 43 ++++
9 files changed, 532 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..2c86fae08642
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,89 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..a4673200f3c9
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..9c01b4e50e65
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,43 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __vdso_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* Re: [PATCH 16/27] arm64: compat: Add vDSO
@ 2018-11-09 16:23 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:23 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, salyzyn, Paul Burton,
Thomas Gleixner, pcc, Linux ARM
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
Corresponding to what I said about the native syscalls, this should
eventually use 'old_timeval32' here.
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
and 'old_timespec32' here.
We could do those changes as a follow-up, but since you change that code
already, we could just as well do it now.
Then there is the question about the replacement. I don't see a need for
64-bit versions of time, gettimeofday or clock_getres, but we definitely want
a new clock_gettime. This probably has to wait until we have assigned a
syscall number (for the fallback implementation), just mentioning it now
so we keep it in mind.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 16/27] arm64: compat: Add vDSO
@ 2018-11-09 16:23 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:23 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
Corresponding to what I said about the native syscalls, this should
eventually use 'old_timeval32' here.
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
and 'old_timespec32' here.
We could do those changes as a follow-up, but since you change that code
already, we could just as well do it now.
Then there is the question about the replacement. I don't see a need for
64-bit versions of time, gettimeofday or clock_getres, but we definitely want
a new clock_gettime. This probably has to wait until we have assigned a
syscall number (for the fallback implementation), just mentioning it now
so we keep it in mind.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 16/27] arm64: compat: Add vDSO
@ 2018-11-09 16:23 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:23 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, salyzyn, pcc
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
Corresponding to what I said about the native syscalls, this should
eventually use 'old_timeval32' here.
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
and 'old_timespec32' here.
We could do those changes as a follow-up, but since you change that code
already, we could just as well do it now.
Then there is the question about the replacement. I don't see a need for
64-bit versions of time, gettimeofday or clock_getres, but we definitely want
a new clock_gettime. This probably has to wait until we have assigned a
syscall number (for the fallback implementation), just mentioning it now
so we keep it in mind.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 17/27] arm64: Refactor vDSO code
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
This patch refactors vdso.c to simplify the implementation of arm64 vDSO
compat (which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 8903087cca4c..f8020db654db 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -40,7 +40,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -51,6 +74,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -61,7 +184,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -201,15 +324,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -224,39 +339,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -264,43 +350,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 17/27] arm64: Refactor vDSO code
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
This patch refactors vdso.c to simplify the implementation of arm64 vDSO
compat (which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 8903087cca4c..f8020db654db 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -40,7 +40,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -51,6 +74,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -61,7 +184,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -201,15 +324,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -224,39 +339,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -264,43 +350,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 17/27] arm64: Refactor vDSO code
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
This patch refactors vdso.c to simplify the implementation of arm64 vDSO
compat (which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 8903087cca4c..f8020db654db 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -40,7 +40,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -51,6 +74,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -61,7 +184,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -201,15 +324,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -224,39 +339,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -264,43 +350,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 18/27] arm64: compat: vDSO setup for compat layer
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
This patch adds the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index f8020db654db..c36290b46e84 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -40,10 +40,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -63,6 +69,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -178,23 +191,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -226,6 +261,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -248,11 +292,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -277,6 +326,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -300,6 +350,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -313,7 +364,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 18/27] arm64: compat: vDSO setup for compat layer
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
This patch adds the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index f8020db654db..c36290b46e84 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -40,10 +40,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -63,6 +69,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -178,23 +191,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -226,6 +261,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -248,11 +292,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -277,6 +326,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -300,6 +350,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -313,7 +364,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 18/27] arm64: compat: vDSO setup for compat layer
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
This patch adds the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index f8020db654db..c36290b46e84 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -40,10 +40,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -63,6 +69,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -178,23 +191,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -226,6 +261,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -248,11 +292,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -277,6 +326,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -300,6 +350,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -313,7 +364,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 19/27] arm64: elf: vDSO code page discovery
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
This patch adds the necessary code to the elf arm64 module to make this
possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index f2b6b486deb7..3fb46968da6a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -209,7 +209,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 19/27] arm64: elf: vDSO code page discovery
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
This patch adds the necessary code to the elf arm64 module to make this
possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index f2b6b486deb7..3fb46968da6a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -209,7 +209,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 19/27] arm64: elf: vDSO code page discovery
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
This patch adds the necessary code to the elf arm64 module to make this
possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index f2b6b486deb7..3fb46968da6a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -209,7 +209,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 20/27] arm64: compat: Get sigreturn trampolines from vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
This patch adds the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 16c490a83131..be471a81084e 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 20/27] arm64: compat: Get sigreturn trampolines from vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
This patch adds the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 16c490a83131..be471a81084e 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 20/27] arm64: compat: Get sigreturn trampolines from vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
This patch adds the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 16c490a83131..be471a81084e 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 21/27] arm64: Add vDSO compat support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch adds vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 21 +++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 2c7d447401b5..79829ddf7cc9 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,6 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6cb9fc7e9382..814d2b0ac017 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -144,6 +158,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 2fcba6b87829..29234a70d168 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+arm64-obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
@@ -60,6 +63,7 @@ arm64-obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o
arm64-obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-y += $(arm64-obj-y) vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
obj-m += $(arm64-obj-m)
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 21/27] arm64: Add vDSO compat support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
This patch adds vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 21 +++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 2c7d447401b5..79829ddf7cc9 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,6 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6cb9fc7e9382..814d2b0ac017 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -144,6 +158,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 2fcba6b87829..29234a70d168 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+arm64-obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
@@ -60,6 +63,7 @@ arm64-obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o
arm64-obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-y += $(arm64-obj-y) vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
obj-m += $(arm64-obj-m)
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 21/27] arm64: Add vDSO compat support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch adds vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 21 +++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 25 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 2c7d447401b5..79829ddf7cc9 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,6 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ACPI_APEI if (ACPI && EFI)
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index 6cb9fc7e9382..814d2b0ac017 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,23 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -144,6 +158,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 2fcba6b87829..29234a70d168 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
arm64-obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+arm64-obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
arm64-obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
arm64-obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
arm64-obj-$(CONFIG_MODULES) += arm64ksyms.o module.o
@@ -60,6 +63,7 @@ arm64-obj-$(CONFIG_ARM_SDE_INTERFACE) += sdei.o
arm64-obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-y += $(arm64-obj-y) vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
obj-m += $(arm64-obj-m)
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 22/27] arm64: Enable compat vDSO support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch enables compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c9a57d11330b..cfd5d67c6407 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -709,6 +709,7 @@ CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_PRINTK_TIME=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_FS=y
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 22/27] arm64: Enable compat vDSO support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
This patch enables compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c9a57d11330b..cfd5d67c6407 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -709,6 +709,7 @@ CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_PRINTK_TIME=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_FS=y
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 22/27] arm64: Enable compat vDSO support
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch enables compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c9a57d11330b..cfd5d67c6407 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -709,6 +709,7 @@ CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ISO8859_1=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_PRINTK_TIME=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_FS=y
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 23/27] arm: Add support for generic vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of __vdso_time
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 79 +++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 28 ++-
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 2 +
arch/arm/vdso/vgettimeofday.c | 263 ++---------------------
8 files changed, 143 insertions(+), 286 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 91be74d8df65..ebcbdcf02283 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -47,6 +47,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
@@ -96,6 +98,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
@@ -1353,6 +1356,7 @@ config HAVE_ARM_ARCH_TIMER
bool "Architected timer support"
depends on CPU_V7
select ARM_ARCH_TIMER
+ select HAVE_ARCH_TIMER
select GENERIC_CLOCKEVENTS
help
This option enables support for the ARM architected timer
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e66b0d28ca9d
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..b26fb6a1d5c7 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -276,14 +276,14 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
static void vdso_write_begin(struct vdso_data *vdata)
{
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
}
static void vdso_write_end(struct vdso_data *vdata)
{
smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
}
static bool tk_is_cntvct(const struct timekeeper *tk)
@@ -307,10 +307,10 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
@@ -318,6 +318,7 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
void update_vsyscall(struct timekeeper *tk)
{
struct timespec64 *wtm = &tk->wall_to_monotonic;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +329,27 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
+ vdso_data->use_syscall = !__tk_is_cntvct;
vdso_data->xtime_coarse_sec = tk->xtime_sec;
vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
tk->tkr_mono.shift);
vdso_data->wtm_clock_sec = wtm->tv_sec;
vdso_data->wtm_clock_nsec = wtm->tv_nsec;
- if (vdso_data->tk_is_cntvct) {
+ if (__tk_is_cntvct) {
vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->raw_time_sec = tk->raw_sec;
+ vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
+ vdso_data->cs_mono_mult = tk->tkr_mono.mult;
+ vdso_data->cs_raw_mult = tk->tkr_raw.mult;
vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = tk->tkr_mono.mask;
+ vdso_data->cs_raw_mask = tk->tkr_raw.mask;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1eb577091d1f 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,8 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..9c01b4e50e65 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
-
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
-{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
-}
-
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
-{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
-}
-
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
+#include <linux/types.h>
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
{
- return -1;
+ return __cvdso_clock_gettime(clock, ts);
}
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
+notrace time_t __vdso_time(time_t *time)
{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
+ return __cvdso_time(time);
}
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 23/27] arm: Add support for generic vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of __vdso_time
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 79 +++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 28 ++-
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 2 +
arch/arm/vdso/vgettimeofday.c | 263 ++---------------------
8 files changed, 143 insertions(+), 286 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 91be74d8df65..ebcbdcf02283 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -47,6 +47,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
@@ -96,6 +98,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
@@ -1353,6 +1356,7 @@ config HAVE_ARM_ARCH_TIMER
bool "Architected timer support"
depends on CPU_V7
select ARM_ARCH_TIMER
+ select HAVE_ARCH_TIMER
select GENERIC_CLOCKEVENTS
help
This option enables support for the ARM architected timer
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e66b0d28ca9d
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..b26fb6a1d5c7 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -276,14 +276,14 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
static void vdso_write_begin(struct vdso_data *vdata)
{
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
}
static void vdso_write_end(struct vdso_data *vdata)
{
smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
}
static bool tk_is_cntvct(const struct timekeeper *tk)
@@ -307,10 +307,10 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
@@ -318,6 +318,7 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
void update_vsyscall(struct timekeeper *tk)
{
struct timespec64 *wtm = &tk->wall_to_monotonic;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +329,27 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
+ vdso_data->use_syscall = !__tk_is_cntvct;
vdso_data->xtime_coarse_sec = tk->xtime_sec;
vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
tk->tkr_mono.shift);
vdso_data->wtm_clock_sec = wtm->tv_sec;
vdso_data->wtm_clock_nsec = wtm->tv_nsec;
- if (vdso_data->tk_is_cntvct) {
+ if (__tk_is_cntvct) {
vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->raw_time_sec = tk->raw_sec;
+ vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
+ vdso_data->cs_mono_mult = tk->tkr_mono.mult;
+ vdso_data->cs_raw_mult = tk->tkr_raw.mult;
vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = tk->tkr_mono.mask;
+ vdso_data->cs_raw_mask = tk->tkr_raw.mask;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1eb577091d1f 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,8 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..9c01b4e50e65 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
-
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
-{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
-}
-
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
-{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
-}
-
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
+#include <linux/types.h>
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
{
- return -1;
+ return __cvdso_clock_gettime(clock, ts);
}
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
+notrace time_t __vdso_time(time_t *time)
{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
+ return __cvdso_time(time);
}
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 23/27] arm: Add support for generic vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of __vdso_time
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 4 +
arch/arm/include/asm/vdso/gettimeofday.h | 79 +++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 28 ++-
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 2 +
arch/arm/vdso/vgettimeofday.c | 263 ++---------------------
8 files changed, 143 insertions(+), 286 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 91be74d8df65..ebcbdcf02283 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -47,6 +47,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if (AEABI && !OABI_COMPAT)
@@ -96,6 +98,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
@@ -1353,6 +1356,7 @@ config HAVE_ARM_ARCH_TIMER
bool "Architected timer support"
depends on CPU_V7
select ARM_ARCH_TIMER
+ select HAVE_ARCH_TIMER
select GENERIC_CLOCKEVENTS
help
This option enables support for the ARM architected timer
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e66b0d28ca9d
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..b26fb6a1d5c7 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -276,14 +276,14 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
static void vdso_write_begin(struct vdso_data *vdata)
{
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
}
static void vdso_write_end(struct vdso_data *vdata)
{
smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
+ ++vdso_data->tb_seq_count;
}
static bool tk_is_cntvct(const struct timekeeper *tk)
@@ -307,10 +307,10 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
@@ -318,6 +318,7 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
void update_vsyscall(struct timekeeper *tk)
{
struct timespec64 *wtm = &tk->wall_to_monotonic;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +329,27 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
+ vdso_data->use_syscall = !__tk_is_cntvct;
vdso_data->xtime_coarse_sec = tk->xtime_sec;
vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
tk->tkr_mono.shift);
vdso_data->wtm_clock_sec = wtm->tv_sec;
vdso_data->wtm_clock_nsec = wtm->tv_nsec;
- if (vdso_data->tk_is_cntvct) {
+ if (__tk_is_cntvct) {
vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->raw_time_sec = tk->raw_sec;
+ vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
+ vdso_data->cs_mono_mult = tk->tkr_mono.mult;
+ vdso_data->cs_raw_mult = tk->tkr_raw.mult;
vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = tk->tkr_mono.mask;
+ vdso_data->cs_raw_mask = tk->tkr_raw.mask;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1eb577091d1f 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,8 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..9c01b4e50e65 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,31 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
-
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
-{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
-}
-
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
-{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
-}
-
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
+#include <linux/types.h>
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
{
- return -1;
+ return __cvdso_clock_gettime(clock, ts);
}
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
+notrace time_t __vdso_time(time_t *time)
{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
+ return __cvdso_time(time);
}
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* Re: [PATCH 23/27] arm: Add support for generic vDSO
@ 2018-11-09 16:27 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:27 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, salyzyn, Paul Burton,
Thomas Gleixner, pcc, Linux ARM
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
> +{
>
> +
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
Same as for the arm64 compat version, these could use old_timeval32
and old_timespec32 right away.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 23/27] arm: Add support for generic vDSO
@ 2018-11-09 16:27 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:27 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
> +{
>
> +
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
Same as for the arm64 compat version, these could use old_timeval32
and old_timespec32 right away.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 23/27] arm: Add support for generic vDSO
@ 2018-11-09 16:27 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-09 16:27 UTC (permalink / raw)
To: vincenzo.frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King - ARM Linux, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, salyzyn, pcc
On Fri, Nov 9, 2018 at 1:38 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +
> +static __always_inline notrace int gettimeofday_fallback(struct timeval *_tv,
> + struct timezone *_tz)
> +{
>
> +
> +static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
> + struct timespec *_ts)
Same as for the arm64 compat version, these could use old_timeval32
and old_timespec32 right away.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 24/27] mips: Introduce vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
vdso_direct is an architectural clock configuration parameter that
exposes the usage of not of the vDSO.
If vdso_direct is false, the time related syscalls will be used instead
of the vDSO library.
This patch introduces the parameter in the mips architecture to prepare
the ground for the support of the generic vDSO library.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/include/asm/clocksource.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/mips/include/asm/clocksource.h b/arch/mips/include/asm/clocksource.h
index 3deb1d0c1a94..eab11ee4871a 100644
--- a/arch/mips/include/asm/clocksource.h
+++ b/arch/mips/include/asm/clocksource.h
@@ -23,6 +23,7 @@
* @vdso_clock_mode: Method the VDSO should use to access the clocksource.
*/
struct arch_clocksource_data {
+ bool vdso_direct;
u8 vdso_clock_mode;
};
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 24/27] mips: Introduce vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
vdso_direct is an architectural clock configuration parameter that
exposes the usage of not of the vDSO.
If vdso_direct is false, the time related syscalls will be used instead
of the vDSO library.
This patch introduces the parameter in the mips architecture to prepare
the ground for the support of the generic vDSO library.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/include/asm/clocksource.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/mips/include/asm/clocksource.h b/arch/mips/include/asm/clocksource.h
index 3deb1d0c1a94..eab11ee4871a 100644
--- a/arch/mips/include/asm/clocksource.h
+++ b/arch/mips/include/asm/clocksource.h
@@ -23,6 +23,7 @@
* @vdso_clock_mode: Method the VDSO should use to access the clocksource.
*/
struct arch_clocksource_data {
+ bool vdso_direct;
u8 vdso_clock_mode;
};
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 24/27] mips: Introduce vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
vdso_direct is an architectural clock configuration parameter that
exposes the usage of not of the vDSO.
If vdso_direct is false, the time related syscalls will be used instead
of the vDSO library.
This patch introduces the parameter in the mips architecture to prepare
the ground for the support of the generic vDSO library.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/include/asm/clocksource.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/mips/include/asm/clocksource.h b/arch/mips/include/asm/clocksource.h
index 3deb1d0c1a94..eab11ee4871a 100644
--- a/arch/mips/include/asm/clocksource.h
+++ b/arch/mips/include/asm/clocksource.h
@@ -23,6 +23,7 @@
* @vdso_clock_mode: Method the VDSO should use to access the clocksource.
*/
struct arch_clocksource_data {
+ bool vdso_direct;
u8 vdso_clock_mode;
};
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 25/27] clock: csrc-4k: Add support for vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on mips
architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/kernel/csrc-r4k.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/mips/kernel/csrc-r4k.c b/arch/mips/kernel/csrc-r4k.c
index eed099f35bf1..97e2a8aafdd6 100644
--- a/arch/mips/kernel/csrc-r4k.c
+++ b/arch/mips/kernel/csrc-r4k.c
@@ -77,8 +77,10 @@ int __init init_r4k_clocksource(void)
* R2 onwards makes the count accessible to user mode so it can be used
* by the VDSO (HWREna is configured by configure_hwrena()).
*/
- if (cpu_has_mips_r2_r6 && rdhwr_count_usable())
+ if (cpu_has_mips_r2_r6 && rdhwr_count_usable()) {
+ clocksource_mips.archdata.vdso_direct = true;
clocksource_mips.archdata.vdso_clock_mode = VDSO_CLOCK_R4K;
+ }
clocksource_register_hz(&clocksource_mips, mips_hpt_frequency);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 25/27] clock: csrc-4k: Add support for vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
This patch is a consequence of the introduction of vdso_direct on mips
architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/kernel/csrc-r4k.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/mips/kernel/csrc-r4k.c b/arch/mips/kernel/csrc-r4k.c
index eed099f35bf1..97e2a8aafdd6 100644
--- a/arch/mips/kernel/csrc-r4k.c
+++ b/arch/mips/kernel/csrc-r4k.c
@@ -77,8 +77,10 @@ int __init init_r4k_clocksource(void)
* R2 onwards makes the count accessible to user mode so it can be used
* by the VDSO (HWREna is configured by configure_hwrena()).
*/
- if (cpu_has_mips_r2_r6 && rdhwr_count_usable())
+ if (cpu_has_mips_r2_r6 && rdhwr_count_usable()) {
+ clocksource_mips.archdata.vdso_direct = true;
clocksource_mips.archdata.vdso_clock_mode = VDSO_CLOCK_R4K;
+ }
clocksource_register_hz(&clocksource_mips, mips_hpt_frequency);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 25/27] clock: csrc-4k: Add support for vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on mips
architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/kernel/csrc-r4k.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/mips/kernel/csrc-r4k.c b/arch/mips/kernel/csrc-r4k.c
index eed099f35bf1..97e2a8aafdd6 100644
--- a/arch/mips/kernel/csrc-r4k.c
+++ b/arch/mips/kernel/csrc-r4k.c
@@ -77,8 +77,10 @@ int __init init_r4k_clocksource(void)
* R2 onwards makes the count accessible to user mode so it can be used
* by the VDSO (HWREna is configured by configure_hwrena()).
*/
- if (cpu_has_mips_r2_r6 && rdhwr_count_usable())
+ if (cpu_has_mips_r2_r6 && rdhwr_count_usable()) {
+ clocksource_mips.archdata.vdso_direct = true;
clocksource_mips.archdata.vdso_clock_mode = VDSO_CLOCK_R4K;
+ }
clocksource_register_hz(&clocksource_mips, mips_hpt_frequency);
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 26/27] clock: gic-timer: Add support for vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on
mips architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
drivers/clocksource/mips-gic-timer.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c
index 54f8a331b53a..e7e0789084cd 100644
--- a/drivers/clocksource/mips-gic-timer.c
+++ b/drivers/clocksource/mips-gic-timer.c
@@ -158,7 +158,10 @@ static struct clocksource gic_clocksource = {
.name = "GIC",
.read = gic_hpt_read,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
- .archdata = { .vdso_clock_mode = VDSO_CLOCK_GIC },
+ .archdata = {
+ .vdso_direct = true,
+ .vdso_clock_mode = VDSO_CLOCK_GIC
+ },
};
static int __init __gic_clocksource_init(void)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 26/27] clock: gic-timer: Add support for vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
This patch is a consequence of the introduction of vdso_direct on
mips architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
drivers/clocksource/mips-gic-timer.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c
index 54f8a331b53a..e7e0789084cd 100644
--- a/drivers/clocksource/mips-gic-timer.c
+++ b/drivers/clocksource/mips-gic-timer.c
@@ -158,7 +158,10 @@ static struct clocksource gic_clocksource = {
.name = "GIC",
.read = gic_hpt_read,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
- .archdata = { .vdso_clock_mode = VDSO_CLOCK_GIC },
+ .archdata = {
+ .vdso_direct = true,
+ .vdso_clock_mode = VDSO_CLOCK_GIC
+ },
};
static int __init __gic_clocksource_init(void)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 26/27] clock: gic-timer: Add support for vdso_direct
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
This patch is a consequence of the introduction of vdso_direct on
mips architectures.
If the clock source is selected, vdso_direct is set to true.
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
drivers/clocksource/mips-gic-timer.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/clocksource/mips-gic-timer.c b/drivers/clocksource/mips-gic-timer.c
index 54f8a331b53a..e7e0789084cd 100644
--- a/drivers/clocksource/mips-gic-timer.c
+++ b/drivers/clocksource/mips-gic-timer.c
@@ -158,7 +158,10 @@ static struct clocksource gic_clocksource = {
.name = "GIC",
.read = gic_hpt_read,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
- .archdata = { .vdso_clock_mode = VDSO_CLOCK_GIC },
+ .archdata = {
+ .vdso_direct = true,
+ .vdso_clock_mode = VDSO_CLOCK_GIC
+ },
};
static int __init __gic_clocksource_init(void)
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* [PATCH 27/27] mips: Add support for generic vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Arnd Bergmann, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne
The mips vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of __vdso_time
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 3 +
arch/mips/include/asm/vdso.h | 53 +----
arch/mips/include/asm/vdso/gettimeofday.h | 161 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 56 +++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 2 +
arch/mips/vdso/vgettimeofday.c | 30 +++
11 files changed, 266 insertions(+), 345 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8272ea4c7264..1bb7ac0c5b2d 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -32,6 +32,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -70,6 +71,8 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
+ select HAVE_ARCH_TIMER
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES && 64BIT
select MODULES_USE_ELF_REL if MODULES
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..4ae362be14b9 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,23 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
+static inline void vdso_data_write_begin(struct vdso_data *data)
{
- ++data->seq_count;
+ ++data->tb_seq_count;
/* Ensure sequence update is written before other data page values. */
smp_wmb();
}
-static inline void vdso_data_write_end(union mips_vdso_data *data)
+static inline void vdso_data_write_end(struct vdso_data *data)
{
/* Ensure data values are written before updating sequence again. */
smp_wmb();
- ++data->seq_count;
+ ++data->tb_seq_count;
}
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b159d205d742
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,161 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (data->clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 48a9c6b90e07..5f280f62dc25 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -26,7 +26,8 @@
#include <asm/vdso.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +73,44 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ vdso_data_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+#ifdef DISABLE_MIPS_VDSO
+ vdso_data->use_syscall = true;
+#else
+ vdso_data->use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
+#endif /* DISABLE_MIPS_VDSO */
+ vdso_data->xtime_coarse_sec = tk->xtime_sec;
+ vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
+ tk->tkr_mono.shift;
+ vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
+ vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->raw_time_sec = tk->raw_sec;
+ vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
+ vdso_data->xtime_clock_sec = tk->xtime_sec;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
+ vdso_data->cs_mono_mult = tk->tkr_mono.mult;
+ vdso_data->cs_raw_mult = tk->tkr_raw.mult;
+ vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = tk->tkr_mono.mask;
+ vdso_data->cs_raw_mask = tk->tkr_mono.mask;
}
- vdso_data_write_end(&vdso_data);
+ vdso_data_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +183,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +209,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 58a0315ad743..e38c9a6571a1 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -14,15 +20,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -49,6 +63,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
#
@@ -96,6 +112,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -134,6 +151,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -174,6 +192,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..1177fecfb083 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,8 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..3501d68f3bb1
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __vdso_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 27/27] mips: Add support for generic vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arm-kernel
The mips vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of __vdso_time
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 3 +
arch/mips/include/asm/vdso.h | 53 +----
arch/mips/include/asm/vdso/gettimeofday.h | 161 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 56 +++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 2 +
arch/mips/vdso/vgettimeofday.c | 30 +++
11 files changed, 266 insertions(+), 345 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8272ea4c7264..1bb7ac0c5b2d 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -32,6 +32,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -70,6 +71,8 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
+ select HAVE_ARCH_TIMER
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES && 64BIT
select MODULES_USE_ELF_REL if MODULES
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..4ae362be14b9 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,23 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
+static inline void vdso_data_write_begin(struct vdso_data *data)
{
- ++data->seq_count;
+ ++data->tb_seq_count;
/* Ensure sequence update is written before other data page values. */
smp_wmb();
}
-static inline void vdso_data_write_end(union mips_vdso_data *data)
+static inline void vdso_data_write_end(struct vdso_data *data)
{
/* Ensure data values are written before updating sequence again. */
smp_wmb();
- ++data->seq_count;
+ ++data->tb_seq_count;
}
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b159d205d742
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,161 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (data->clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 48a9c6b90e07..5f280f62dc25 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -26,7 +26,8 @@
#include <asm/vdso.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +73,44 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ vdso_data_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+#ifdef DISABLE_MIPS_VDSO
+ vdso_data->use_syscall = true;
+#else
+ vdso_data->use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
+#endif /* DISABLE_MIPS_VDSO */
+ vdso_data->xtime_coarse_sec = tk->xtime_sec;
+ vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
+ tk->tkr_mono.shift;
+ vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
+ vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->raw_time_sec = tk->raw_sec;
+ vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
+ vdso_data->xtime_clock_sec = tk->xtime_sec;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
+ vdso_data->cs_mono_mult = tk->tkr_mono.mult;
+ vdso_data->cs_raw_mult = tk->tkr_raw.mult;
+ vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = tk->tkr_mono.mask;
+ vdso_data->cs_raw_mask = tk->tkr_mono.mask;
}
- vdso_data_write_end(&vdso_data);
+ vdso_data_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +183,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +209,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 58a0315ad743..e38c9a6571a1 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -14,15 +20,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -49,6 +63,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
#
@@ -96,6 +112,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -134,6 +151,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -174,6 +192,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..1177fecfb083 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,8 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..3501d68f3bb1
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __vdso_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread* [PATCH 27/27] mips: Add support for generic vDSO
@ 2018-11-09 12:37 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-09 12:37 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne
The mips vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
This patch introduces the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of __vdso_time
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 3 +
arch/mips/include/asm/vdso.h | 53 +----
arch/mips/include/asm/vdso/gettimeofday.h | 161 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 56 +++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 2 +
arch/mips/vdso/vgettimeofday.c | 30 +++
11 files changed, 266 insertions(+), 345 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index 8272ea4c7264..1bb7ac0c5b2d 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -32,6 +32,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -70,6 +71,8 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
+ select HAVE_ARCH_TIMER
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_RELA if MODULES && 64BIT
select MODULES_USE_ELF_REL if MODULES
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..4ae362be14b9 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,23 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
+static inline void vdso_data_write_begin(struct vdso_data *data)
{
- ++data->seq_count;
+ ++data->tb_seq_count;
/* Ensure sequence update is written before other data page values. */
smp_wmb();
}
-static inline void vdso_data_write_end(union mips_vdso_data *data)
+static inline void vdso_data_write_end(struct vdso_data *data)
{
/* Ensure data values are written before updating sequence again. */
smp_wmb();
- ++data->seq_count;
+ ++data->tb_seq_count;
}
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b159d205d742
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,161 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(struct timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(clockid_t _clkid,
+ struct timespec *_ts)
+{
+ register struct timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 clock_get_virtual_counter(void)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (data->clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 48a9c6b90e07..5f280f62dc25 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -26,7 +26,8 @@
#include <asm/vdso.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +73,44 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ vdso_data_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+#ifdef DISABLE_MIPS_VDSO
+ vdso_data->use_syscall = true;
+#else
+ vdso_data->use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
+#endif /* DISABLE_MIPS_VDSO */
+ vdso_data->xtime_coarse_sec = tk->xtime_sec;
+ vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
+ tk->tkr_mono.shift;
+ vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
+ vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->raw_time_sec = tk->raw_sec;
+ vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
+ vdso_data->xtime_clock_sec = tk->xtime_sec;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
+ vdso_data->cs_mono_mult = tk->tkr_mono.mult;
+ vdso_data->cs_raw_mult = tk->tkr_raw.mult;
+ vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->btm_nsec = ktime_to_ns(tk->offs_boot);
+ vdso_data->tai_sec = tk->xtime_sec +
+ tk->tai_offset;
+ vdso_data->cs_mono_mask = tk->tkr_mono.mask;
+ vdso_data->cs_raw_mask = tk->tkr_mono.mask;
}
- vdso_data_write_end(&vdso_data);
+ vdso_data_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +183,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +209,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 58a0315ad743..e38c9a6571a1 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -14,15 +20,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -49,6 +63,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
#
@@ -96,6 +112,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -134,6 +151,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -174,6 +192,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..1177fecfb083 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,8 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
+ __vdso_time;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..3501d68f3bb1
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,30 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace time_t __vdso_time(time_t *time)
+{
+ return __cvdso_time(time);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id, struct timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.19.1
^ permalink raw reply related [flat|nested] 144+ messages in thread
* Re: [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-10 16:33 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-10 16:33 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> vDSO (virtual dynamic shared object) is a mechanism that the Linux
> kernel provides as an alternative to system calls to reduce where
> possible the costs in terms of cycles.
> This is possible because certain syscalls like gettimeofday() do
> not write any data and return one or more values that are stored
> in the kernel, which makes relatively safe calling them directly
> as a library function.
>
> Even if the mechanism is pretty much standard, every architecture
> in the last few years ended up implementing their own vDSO library
> in the architectural code.
>
> The purpose of this patch-set is to identify the commonalities in
> between the architectures and try to consolidate the common code
> paths, starting with gettimeofday().
I'm very happy to see the generalization overall, as this is going
to make it much easier to add the 64-bit time_t on the 32-bit
vdso implementations, as well as other extensions we may want
in the future.
> This implementation contains the following design choices:
> * Every architecture defines the arch specific code in an header in
> "asm/vdso/".
> * The generic implementation includes the arch specific one and lives
> in "lib/vdso".
Makes sense.
> * The arch specific code for gettimeofday lives in
> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
> * This approach allows to consolidate the common code in a single place
> with the benefit of avoiding code duplication.
I would prefer to have that named clock_gettime.c and only implement
that one system call there. We can easily have the gettimeofday(),
time() and clock_getres() in the common code where that is needed
for backwards compatibility. As I already mentioned in my comment
for one of the patches, we won't have time() or gettimeofday() system
calls on new architectures, and 32-bit architectures won't have them
with 64-bit time_t.
I suppose we can have the time() and gettimeofday() calls
implemented in the generic vdso for everyone, but then
only hook those up on architectures that already had
them. The implementation of course is trivial based on
clock_gettime() with CLOCK_REALTIME or CLOCK_REALTIME_COARSE
respectively.
> This implementation contains the portings to the common library for: arm64,
> compat mode for arm64, arm and mips.
>
> The mips porting has been tested on qemu for mips32el. A configuration to
> repeat the tests can be found at [4].
Ah, I missed the fact that you didn't do this for x86, so I assumed that
this work was triggered by Thomas' recent cleanup of the x86 clock_gettime
vdso code. I think it would be best to get x86 to use the same code and
get the recent improvements ported to the others as well that way.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-10 16:33 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-10 16:33 UTC (permalink / raw)
To: linux-arm-kernel
On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> vDSO (virtual dynamic shared object) is a mechanism that the Linux
> kernel provides as an alternative to system calls to reduce where
> possible the costs in terms of cycles.
> This is possible because certain syscalls like gettimeofday() do
> not write any data and return one or more values that are stored
> in the kernel, which makes relatively safe calling them directly
> as a library function.
>
> Even if the mechanism is pretty much standard, every architecture
> in the last few years ended up implementing their own vDSO library
> in the architectural code.
>
> The purpose of this patch-set is to identify the commonalities in
> between the architectures and try to consolidate the common code
> paths, starting with gettimeofday().
I'm very happy to see the generalization overall, as this is going
to make it much easier to add the 64-bit time_t on the 32-bit
vdso implementations, as well as other extensions we may want
in the future.
> This implementation contains the following design choices:
> * Every architecture defines the arch specific code in an header in
> "asm/vdso/".
> * The generic implementation includes the arch specific one and lives
> in "lib/vdso".
Makes sense.
> * The arch specific code for gettimeofday lives in
> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
> * This approach allows to consolidate the common code in a single place
> with the benefit of avoiding code duplication.
I would prefer to have that named clock_gettime.c and only implement
that one system call there. We can easily have the gettimeofday(),
time() and clock_getres() in the common code where that is needed
for backwards compatibility. As I already mentioned in my comment
for one of the patches, we won't have time() or gettimeofday() system
calls on new architectures, and 32-bit architectures won't have them
with 64-bit time_t.
I suppose we can have the time() and gettimeofday() calls
implemented in the generic vdso for everyone, but then
only hook those up on architectures that already had
them. The implementation of course is trivial based on
clock_gettime() with CLOCK_REALTIME or CLOCK_REALTIME_COARSE
respectively.
> This implementation contains the portings to the common library for: arm64,
> compat mode for arm64, arm and mips.
>
> The mips porting has been tested on qemu for mips32el. A configuration to
> repeat the tests can be found at [4].
Ah, I missed the fact that you didn't do this for x86, so I assumed that
this work was triggered by Thomas' recent cleanup of the x86 clock_gettime
vdso code. I think it would be best to get x86 to use the same code and
get the recent improvements ported to the others as well that way.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-10 16:33 ` Arnd Bergmann
0 siblings, 0 replies; 144+ messages in thread
From: Arnd Bergmann @ 2018-11-10 16:33 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne
On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> vDSO (virtual dynamic shared object) is a mechanism that the Linux
> kernel provides as an alternative to system calls to reduce where
> possible the costs in terms of cycles.
> This is possible because certain syscalls like gettimeofday() do
> not write any data and return one or more values that are stored
> in the kernel, which makes relatively safe calling them directly
> as a library function.
>
> Even if the mechanism is pretty much standard, every architecture
> in the last few years ended up implementing their own vDSO library
> in the architectural code.
>
> The purpose of this patch-set is to identify the commonalities in
> between the architectures and try to consolidate the common code
> paths, starting with gettimeofday().
I'm very happy to see the generalization overall, as this is going
to make it much easier to add the 64-bit time_t on the 32-bit
vdso implementations, as well as other extensions we may want
in the future.
> This implementation contains the following design choices:
> * Every architecture defines the arch specific code in an header in
> "asm/vdso/".
> * The generic implementation includes the arch specific one and lives
> in "lib/vdso".
Makes sense.
> * The arch specific code for gettimeofday lives in
> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
> * This approach allows to consolidate the common code in a single place
> with the benefit of avoiding code duplication.
I would prefer to have that named clock_gettime.c and only implement
that one system call there. We can easily have the gettimeofday(),
time() and clock_getres() in the common code where that is needed
for backwards compatibility. As I already mentioned in my comment
for one of the patches, we won't have time() or gettimeofday() system
calls on new architectures, and 32-bit architectures won't have them
with 64-bit time_t.
I suppose we can have the time() and gettimeofday() calls
implemented in the generic vdso for everyone, but then
only hook those up on architectures that already had
them. The implementation of course is trivial based on
clock_gettime() with CLOCK_REALTIME or CLOCK_REALTIME_COARSE
respectively.
> This implementation contains the portings to the common library for: arm64,
> compat mode for arm64, arm and mips.
>
> The mips porting has been tested on qemu for mips32el. A configuration to
> repeat the tests can be found at [4].
Ah, I missed the fact that you didn't do this for x86, so I assumed that
this work was triggered by Thomas' recent cleanup of the x86 clock_gettime
vdso code. I think it would be best to get x86 to use the same code and
get the recent improvements ported to the others as well that way.
Arnd
^ permalink raw reply [flat|nested] 144+ messages in thread* Re: [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-23 11:05 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-23 11:05 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Catalin Marinas, Daniel Lezcano, Will Deacon,
Russell King, Ralf Baechle, Mark Salyzyn, Paul Burton,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On 11/10/18 4:33 PM, Arnd Bergmann wrote:
> On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
>> vDSO (virtual dynamic shared object) is a mechanism that the Linux
>> kernel provides as an alternative to system calls to reduce where
>> possible the costs in terms of cycles.
>> This is possible because certain syscalls like gettimeofday() do
>> not write any data and return one or more values that are stored
>> in the kernel, which makes relatively safe calling them directly
>> as a library function.
>>
>> Even if the mechanism is pretty much standard, every architecture
>> in the last few years ended up implementing their own vDSO library
>> in the architectural code.
>>
>> The purpose of this patch-set is to identify the commonalities in
>> between the architectures and try to consolidate the common code
>> paths, starting with gettimeofday().
>
> I'm very happy to see the generalization overall, as this is going
> to make it much easier to add the 64-bit time_t on the 32-bit
> vdso implementations, as well as other extensions we may want
> in the future.
>
Thanks,
>> This implementation contains the following design choices:
>> * Every architecture defines the arch specific code in an header in
>> "asm/vdso/".
>> * The generic implementation includes the arch specific one and lives
>> in "lib/vdso".
>
> Makes sense.
>
>> * The arch specific code for gettimeofday lives in
>> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
>> * This approach allows to consolidate the common code in a single place
>> with the benefit of avoiding code duplication.
>
> I would prefer to have that named clock_gettime.c and only implement
> that one system call there. We can easily have the gettimeofday(),
> time() and clock_getres() in the common code where that is needed
> for backwards compatibility. As I already mentioned in my comment
> for one of the patches, we won't have time() or gettimeofday() system
> calls on new architectures, and 32-bit architectures won't have them
> with 64-bit time_t.
>
The reason why I used gettimeofday.c as a filename is because it is the naming
convention that I could see more often in the kernel and I feel that, if we have
even the implementations of gettimeofday(), time() and clock_getres() in the
same file is probably the most representative.
> I suppose we can have the time() and gettimeofday() calls
> implemented in the generic vdso for everyone, but then
> only hook those up on architectures that already had
> them. The implementation of course is trivial based on
> clock_gettime() with CLOCK_REALTIME or CLOCK_REALTIME_COARSE
> respectively.
>
I agree, and in the long term would be my preferred approach as well, but I
would prefer to keep this "optimization" in a separate patchset for what I
mentioned in my previous email.
>> This implementation contains the portings to the common library for: arm64,
>> compat mode for arm64, arm and mips.
>>
>> The mips porting has been tested on qemu for mips32el. A configuration to
>> repeat the tests can be found at [4].
>
> Ah, I missed the fact that you didn't do this for x86, so I assumed that
> this work was triggered by Thomas' recent cleanup of the x86 clock_gettime
> vdso code. I think it would be best to get x86 to use the same code and
> get the recent improvements ported to the others as well that way.
>
Yes, I agree it would be great to have have the other architectures to join the
effort because I believe that the whole ecosystem would benefit from it.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-23 11:05 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-23 11:05 UTC (permalink / raw)
To: linux-arm-kernel
On 11/10/18 4:33 PM, Arnd Bergmann wrote:
> On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
>> vDSO (virtual dynamic shared object) is a mechanism that the Linux
>> kernel provides as an alternative to system calls to reduce where
>> possible the costs in terms of cycles.
>> This is possible because certain syscalls like gettimeofday() do
>> not write any data and return one or more values that are stored
>> in the kernel, which makes relatively safe calling them directly
>> as a library function.
>>
>> Even if the mechanism is pretty much standard, every architecture
>> in the last few years ended up implementing their own vDSO library
>> in the architectural code.
>>
>> The purpose of this patch-set is to identify the commonalities in
>> between the architectures and try to consolidate the common code
>> paths, starting with gettimeofday().
>
> I'm very happy to see the generalization overall, as this is going
> to make it much easier to add the 64-bit time_t on the 32-bit
> vdso implementations, as well as other extensions we may want
> in the future.
>
Thanks,
>> This implementation contains the following design choices:
>> * Every architecture defines the arch specific code in an header in
>> "asm/vdso/".
>> * The generic implementation includes the arch specific one and lives
>> in "lib/vdso".
>
> Makes sense.
>
>> * The arch specific code for gettimeofday lives in
>> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
>> * This approach allows to consolidate the common code in a single place
>> with the benefit of avoiding code duplication.
>
> I would prefer to have that named clock_gettime.c and only implement
> that one system call there. We can easily have the gettimeofday(),
> time() and clock_getres() in the common code where that is needed
> for backwards compatibility. As I already mentioned in my comment
> for one of the patches, we won't have time() or gettimeofday() system
> calls on new architectures, and 32-bit architectures won't have them
> with 64-bit time_t.
>
The reason why I used gettimeofday.c as a filename is because it is the naming
convention that I could see more often in the kernel and I feel that, if we have
even the implementations of gettimeofday(), time() and clock_getres() in the
same file is probably the most representative.
> I suppose we can have the time() and gettimeofday() calls
> implemented in the generic vdso for everyone, but then
> only hook those up on architectures that already had
> them. The implementation of course is trivial based on
> clock_gettime() with CLOCK_REALTIME or CLOCK_REALTIME_COARSE
> respectively.
>
I agree, and in the long term would be my preferred approach as well, but I
would prefer to keep this "optimization" in a separate patchset for what I
mentioned in my previous email.
>> This implementation contains the portings to the common library for: arm64,
>> compat mode for arm64, arm and mips.
>>
>> The mips porting has been tested on qemu for mips32el. A configuration to
>> repeat the tests can be found at [4].
>
> Ah, I missed the fact that you didn't do this for x86, so I assumed that
> this work was triggered by Thomas' recent cleanup of the x86 clock_gettime
> vdso code. I think it would be best to get x86 to use the same code and
> get the recent improvements ported to the others as well that way.
>
Yes, I agree it would be great to have have the other architectures to join the
effort because I believe that the whole ecosystem would benefit from it.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread
* Re: [PATCH 00/27] Unify vDSOs across more architectures
@ 2018-11-23 11:05 ` Vincenzo Frascino
0 siblings, 0 replies; 144+ messages in thread
From: Vincenzo Frascino @ 2018-11-23 11:05 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne
On 11/10/18 4:33 PM, Arnd Bergmann wrote:
> On 11/9/18, Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
>> vDSO (virtual dynamic shared object) is a mechanism that the Linux
>> kernel provides as an alternative to system calls to reduce where
>> possible the costs in terms of cycles.
>> This is possible because certain syscalls like gettimeofday() do
>> not write any data and return one or more values that are stored
>> in the kernel, which makes relatively safe calling them directly
>> as a library function.
>>
>> Even if the mechanism is pretty much standard, every architecture
>> in the last few years ended up implementing their own vDSO library
>> in the architectural code.
>>
>> The purpose of this patch-set is to identify the commonalities in
>> between the architectures and try to consolidate the common code
>> paths, starting with gettimeofday().
>
> I'm very happy to see the generalization overall, as this is going
> to make it much easier to add the 64-bit time_t on the 32-bit
> vdso implementations, as well as other extensions we may want
> in the future.
>
Thanks,
>> This implementation contains the following design choices:
>> * Every architecture defines the arch specific code in an header in
>> "asm/vdso/".
>> * The generic implementation includes the arch specific one and lives
>> in "lib/vdso".
>
> Makes sense.
>
>> * The arch specific code for gettimeofday lives in
>> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
>> * This approach allows to consolidate the common code in a single place
>> with the benefit of avoiding code duplication.
>
> I would prefer to have that named clock_gettime.c and only implement
> that one system call there. We can easily have the gettimeofday(),
> time() and clock_getres() in the common code where that is needed
> for backwards compatibility. As I already mentioned in my comment
> for one of the patches, we won't have time() or gettimeofday() system
> calls on new architectures, and 32-bit architectures won't have them
> with 64-bit time_t.
>
The reason why I used gettimeofday.c as a filename is because it is the naming
convention that I could see more often in the kernel and I feel that, if we have
even the implementations of gettimeofday(), time() and clock_getres() in the
same file is probably the most representative.
> I suppose we can have the time() and gettimeofday() calls
> implemented in the generic vdso for everyone, but then
> only hook those up on architectures that already had
> them. The implementation of course is trivial based on
> clock_gettime() with CLOCK_REALTIME or CLOCK_REALTIME_COARSE
> respectively.
>
I agree, and in the long term would be my preferred approach as well, but I
would prefer to keep this "optimization" in a separate patchset for what I
mentioned in my previous email.
>> This implementation contains the portings to the common library for: arm64,
>> compat mode for arm64, arm and mips.
>>
>> The mips porting has been tested on qemu for mips32el. A configuration to
>> repeat the tests can be found at [4].
>
> Ah, I missed the fact that you didn't do this for x86, so I assumed that
> this work was triggered by Thomas' recent cleanup of the x86 clock_gettime
> vdso code. I think it would be best to get x86 to use the same code and
> get the recent improvements ported to the others as well that way.
>
Yes, I agree it would be great to have have the other architectures to join the
effort because I believe that the whole ecosystem would benefit from it.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 144+ messages in thread