* [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.
Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.
The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in
"asm/vdso/".
* The generic implementation includes the arch specific one and lives
in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place
with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm, mips, x86_64, compat mode for x86_64 and i386.
The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].
The x86_64 porting has been tested on an Intel Xeon 5120T based machine
running Ubuntu 18.04 and using the Ubuntu provided defconfig.
The i386 porting has been tested on qemu using the i386_defconfig
configuration.
Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].
A new test has been added to the linux kselftest in order to validate the
newly added library.
To simplify the testing, a copy of the patchset on top of a recent linux
tree can be found at [3] and [4].
[1] https://github.com/nathanlynch/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v5
[4] git://linux-arm.org/linux-vf.git vdso-mips/v5
Changes:
--------
v5:
- Rebased on 5.0-rc7.
- Added x86_64, compat mode for x86_64 and i386 portings.
- Extended vDSO kselftest.
- Addressed review comments.
v4:
- Rebased on 5.0-rc2.
- Addressed review comments.
- Disabled compat vdso on arm64 when the kernel is compiled with
clang.
v3:
- Ported the latest fixes and optimizations done on the x86
architecture to the generic library.
- Addressed review comments.
- Improved the documentation of the interfaces.
- Changed the HAVE_ARCH_TIMER config option to a more generic
HAVE_HW_COUNTER.
v2:
- Added -ffixed-x18 to arm64
- Repleced occurrences of timeval and timespec
- Modified datapage.h to be compliant with y2038 on all the architectures
- Removed __u_vdso type
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Dmitry Safonov <0x7f454c46@gmail.com>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Peter Collingbourne (1):
arm64: Build vDSO with -ffixed-x18
Vincenzo Frascino (22):
kernel: Standardize vdso_datapage
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Add support for generic vDSO
x86: Add support for generic vDSO
kselftest: Extend vDSO selftest
arch/arm/Kconfig | 3 +
arch/arm/include/asm/vdso/gettimeofday.h | 82 +++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 108 ++--
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 +--------
arch/arm64/Kconfig | 24 +
arch/arm64/Makefile | 23 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 +-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 119 ++++
arch/arm64/include/asm/vdso/gettimeofday.h | 85 +++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 45 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 +--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 506 +++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 +
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 +++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 81 +++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +--
arch/mips/include/asm/vdso/gettimeofday.h | 165 ++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +
arch/x86/Kconfig | 3 +
arch/x86/entry/vdso/Makefile | 9 +
arch/x86/entry/vdso/vclock_gettime.c | 239 +--------
arch/x86/entry/vdso/vdso.lds.S | 2 +
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
arch/x86/entry/vdso/vdsox32.lds.S | 1 +
arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 ++--
arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++
arch/x86/include/asm/vgtod.h | 73 +--
arch/x86/include/asm/vvar.h | 2 +-
include/vdso/datapage.h | 75 +++
include/vdso/helpers.h | 52 ++
include/vdso/types.h | 39 ++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 37 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 179 +++++++
tools/testing/selftests/vDSO/Makefile | 2 +
tools/testing/selftests/vDSO/vdso_full_test.c | 246 +++++++++
66 files changed, 2812 insertions(+), 1627 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
create mode 100644 include/vdso/datapage.h
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c
--
2.20.1
^ permalink raw reply [flat|nested] 178+ messages in thread
* [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.
Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.
The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in
"asm/vdso/".
* The generic implementation includes the arch specific one and lives
in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place
with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm, mips, x86_64, compat mode for x86_64 and i386.
The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].
The x86_64 porting has been tested on an Intel Xeon 5120T based machine
running Ubuntu 18.04 and using the Ubuntu provided defconfig.
The i386 porting has been tested on qemu using the i386_defconfig
configuration.
Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].
A new test has been added to the linux kselftest in order to validate the
newly added library.
To simplify the testing, a copy of the patchset on top of a recent linux
tree can be found at [3] and [4].
[1] https://github.com/nathanlynch/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v5
[4] git://linux-arm.org/linux-vf.git vdso-mips/v5
Changes:
--------
v5:
- Rebased on 5.0-rc7.
- Added x86_64, compat mode for x86_64 and i386 portings.
- Extended vDSO kselftest.
- Addressed review comments.
v4:
- Rebased on 5.0-rc2.
- Addressed review comments.
- Disabled compat vdso on arm64 when the kernel is compiled with
clang.
v3:
- Ported the latest fixes and optimizations done on the x86
architecture to the generic library.
- Addressed review comments.
- Improved the documentation of the interfaces.
- Changed the HAVE_ARCH_TIMER config option to a more generic
HAVE_HW_COUNTER.
v2:
- Added -ffixed-x18 to arm64
- Repleced occurrences of timeval and timespec
- Modified datapage.h to be compliant with y2038 on all the architectures
- Removed __u_vdso type
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Dmitry Safonov <0x7f454c46@gmail.com>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Peter Collingbourne (1):
arm64: Build vDSO with -ffixed-x18
Vincenzo Frascino (22):
kernel: Standardize vdso_datapage
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Add support for generic vDSO
x86: Add support for generic vDSO
kselftest: Extend vDSO selftest
arch/arm/Kconfig | 3 +
arch/arm/include/asm/vdso/gettimeofday.h | 82 +++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 108 ++--
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 +--------
arch/arm64/Kconfig | 24 +
arch/arm64/Makefile | 23 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 +-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 119 ++++
arch/arm64/include/asm/vdso/gettimeofday.h | 85 +++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 45 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 +--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 506 +++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 +
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 +++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 81 +++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +--
arch/mips/include/asm/vdso/gettimeofday.h | 165 ++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +
arch/x86/Kconfig | 3 +
arch/x86/entry/vdso/Makefile | 9 +
arch/x86/entry/vdso/vclock_gettime.c | 239 +--------
arch/x86/entry/vdso/vdso.lds.S | 2 +
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
arch/x86/entry/vdso/vdsox32.lds.S | 1 +
arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 ++--
arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++
arch/x86/include/asm/vgtod.h | 73 +--
arch/x86/include/asm/vvar.h | 2 +-
include/vdso/datapage.h | 75 +++
include/vdso/helpers.h | 52 ++
include/vdso/types.h | 39 ++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 37 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 179 +++++++
tools/testing/selftests/vDSO/Makefile | 2 +
tools/testing/selftests/vDSO/vdso_full_test.c | 246 +++++++++
66 files changed, 2812 insertions(+), 1627 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
create mode 100644 include/vdso/datapage.h
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c
--
2.20.1
^ permalink raw reply [flat|nested] 178+ messages in thread
* [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
vDSO (virtual dynamic shared object) is a mechanism that the Linux
kernel provides as an alternative to system calls to reduce where
possible the costs in terms of cycles.
This is possible because certain syscalls like gettimeofday() do
not write any data and return one or more values that are stored
in the kernel, which makes relatively safe calling them directly
as a library function.
Even if the mechanism is pretty much standard, every architecture
in the last few years ended up implementing their own vDSO library
in the architectural code.
The purpose of this patch-set is to identify the commonalities in
between the architectures and try to consolidate the common code
paths, starting with gettimeofday().
This implementation contains the following design choices:
* Every architecture defines the arch specific code in an header in
"asm/vdso/".
* The generic implementation includes the arch specific one and lives
in "lib/vdso".
* The arch specific code for gettimeofday lives in
"<arch path>/vdso/gettimeofday.c" and includes the generic code only.
* This approach allows to consolidate the common code in a single place
with the benefit of avoiding code duplication.
This implementation contains the portings to the common library for: arm64,
compat mode for arm64, arm, mips, x86_64, compat mode for x86_64 and i386.
The mips porting has been tested on qemu for mips32el. A configuration to
repeat the tests can be found at [4].
The x86_64 porting has been tested on an Intel Xeon 5120T based machine
running Ubuntu 18.04 and using the Ubuntu provided defconfig.
The i386 porting has been tested on qemu using the i386_defconfig
configuration.
Last but not least from this porting arm64, compat arm64, arm and mips gain
the support for:
* CLOCK_BOOTTIME that can be useful in certain scenarios since it keeps
track of the time during sleep as well.
* CLOCK_TAI that is like CLOCK_REALTIME, but uses the International
Atomic Time (TAI) reference instead of UTC to avoid jumping on leap
second updates.
for both clock_gettime and clock_getres.
The porting has been validated using the vdsotest test-suite [1] extended
to cover all the clock ids [2].
A new test has been added to the linux kselftest in order to validate the
newly added library.
To simplify the testing, a copy of the patchset on top of a recent linux
tree can be found at [3] and [4].
[1] https://github.com/nathanlynch/vdsotest
[2] https://github.com/fvincenzo/vdsotest
[3] git://linux-arm.org/linux-vf.git vdso/v5
[4] git://linux-arm.org/linux-vf.git vdso-mips/v5
Changes:
--------
v5:
- Rebased on 5.0-rc7.
- Added x86_64, compat mode for x86_64 and i386 portings.
- Extended vDSO kselftest.
- Addressed review comments.
v4:
- Rebased on 5.0-rc2.
- Addressed review comments.
- Disabled compat vdso on arm64 when the kernel is compiled with
clang.
v3:
- Ported the latest fixes and optimizations done on the x86
architecture to the generic library.
- Addressed review comments.
- Improved the documentation of the interfaces.
- Changed the HAVE_ARCH_TIMER config option to a more generic
HAVE_HW_COUNTER.
v2:
- Added -ffixed-x18 to arm64
- Repleced occurrences of timeval and timespec
- Modified datapage.h to be compliant with y2038 on all the architectures
- Removed __u_vdso type
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Cc: Russell King <linux@armlinux.org.uk>
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Cc: Daniel Lezcano <daniel.lezcano@linaro.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Mark Salyzyn <salyzyn@android.com>
Cc: Peter Collingbourne <pcc@google.com>
Cc: Shuah Khan <shuah@kernel.org>
Cc: Dmitry Safonov <0x7f454c46@gmail.com>
Cc: Rasmus Villemoes <linux@rasmusvillemoes.dk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Peter Collingbourne (1):
arm64: Build vDSO with -ffixed-x18
Vincenzo Frascino (22):
kernel: Standardize vdso_datapage
kernel: Define gettimeofday vdso common code
arm64: Substitute gettimeofday with C implementation
arm64: compat: Alloc separate pages for vectors and sigpage
arm64: compat: Split kuser32
arm64: compat: Refactor aarch32_alloc_vdso_pages()
arm64: compat: Add KUSER_HELPERS config option
arm64: compat: Add missing syscall numbers
arm64: compat: Expose signal related structures
arm64: compat: Generate asm offsets for signals
lib: vdso: Add compat support
arm64: compat: Add vDSO
arm64: Refactor vDSO code
arm64: compat: vDSO setup for compat layer
arm64: elf: vDSO code page discovery
arm64: compat: Get sigreturn trampolines from vDSO
arm64: Add vDSO compat support
arm64: Enable compat vDSO support
arm: Add support for generic vDSO
mips: Add support for generic vDSO
x86: Add support for generic vDSO
kselftest: Extend vDSO selftest
arch/arm/Kconfig | 3 +
arch/arm/include/asm/vdso/gettimeofday.h | 82 +++
arch/arm/include/asm/vdso_datapage.h | 25 +-
arch/arm/kernel/vdso.c | 108 ++--
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 +
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 +--------
arch/arm64/Kconfig | 24 +
arch/arm64/Makefile | 23 +-
arch/arm64/configs/defconfig | 1 +
arch/arm64/include/asm/elf.h | 20 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 58 +-
arch/arm64/include/asm/unistd.h | 3 +
arch/arm64/include/asm/vdso.h | 3 +
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++
.../include/asm/vdso/compat_gettimeofday.h | 119 ++++
arch/arm64/include/asm/vdso/gettimeofday.h | 85 +++
arch/arm64/include/asm/vdso_datapage.h | 47 --
arch/arm64/kernel/Makefile | 7 +-
arch/arm64/kernel/asm-offsets.c | 45 +-
arch/arm64/kernel/kuser32.S | 65 +--
arch/arm64/kernel/signal32.c | 76 +--
arch/arm64/kernel/sigreturn32.S | 46 ++
arch/arm64/kernel/vdso.c | 506 +++++++++++++-----
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 +
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 +++++++
arch/arm64/kernel/vdso32/note.c | 15 +
arch/arm64/kernel/vdso32/sigreturn.S | 62 +++
arch/arm64/kernel/vdso32/vdso.S | 19 +
arch/arm64/kernel/vdso32/vdso.lds.S | 81 +++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +--
arch/mips/include/asm/vdso/gettimeofday.h | 165 ++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++-
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +
arch/x86/Kconfig | 3 +
arch/x86/entry/vdso/Makefile | 9 +
arch/x86/entry/vdso/vclock_gettime.c | 239 +--------
arch/x86/entry/vdso/vdso.lds.S | 2 +
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
arch/x86/entry/vdso/vdsox32.lds.S | 1 +
arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 ++--
arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++
arch/x86/include/asm/vgtod.h | 73 +--
arch/x86/include/asm/vvar.h | 2 +-
include/vdso/datapage.h | 75 +++
include/vdso/helpers.h | 52 ++
include/vdso/types.h | 39 ++
lib/Kconfig | 5 +
lib/vdso/Kconfig | 37 ++
lib/vdso/Makefile | 22 +
lib/vdso/gettimeofday.c | 179 +++++++
tools/testing/selftests/vDSO/Makefile | 2 +
tools/testing/selftests/vDSO/vdso_full_test.c | 246 +++++++++
66 files changed, 2812 insertions(+), 1627 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
create mode 100644 arch/arm64/kernel/sigreturn32.S
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
create mode 100644 include/vdso/datapage.h
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch generalized the data structures in vgtod.h
from x86 private includes to general includes (include/vdso).
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 74 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 74 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..da346ad02b03
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/bits.h>
+#include <linux/types.h>
+#include <linux/time.h>
+
+#define VDSO_BASES (CLOCK_TAI + 1)
+#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
+ BIT(CLOCK_MONOTONIC) | \
+ BIT(CLOCK_MONOTONIC_RAW) | \
+ BIT(CLOCK_BOOTTIME) | \
+ BIT(CLOCK_TAI))
+#define VDSO_COARSE (BIT(CLOCK_REALTIME_COARSE) | \
+ BIT(CLOCK_MONOTONIC_COARSE))
+
+/*
+ * There is one vdso_timestamp object in vvar for each vDSO-accelerated
+ * clock_id. For high-resolution clocks, this encodes the time
+ * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
+ * the actual time.
+ *
+ * To be noticed that for highres clocks nsec is left-shifted by
+ * vdso_data.cs[x].shift.
+ */
+struct vdso_timestamp {
+ u64 sec;
+ u64 nsec;
+};
+
+#define CLOCKSOURCE_RAW 0
+#define CLOCKSOURCE_MONO 1
+#define CLOCKSOURCE_BASES (CLOCKSOURCE_MONO + 1)
+
+/*
+ * There is one vdso_clocksource object in vvar for each vDSO clocksource
+ * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
+ * and optimal in terms of access pattern.
+ *
+ * Note that mask and shift are the same for mono and raw.
+ */
+struct vdso_clocksource {
+ u64 mask; /* Clocksource mask */
+ u32 mult; /* Clocksource multiplier */
+ u32 shift; /* Clocksource shift */
+};
+
+/*
+ * vdso_data will be accessed by 32 and 64 bit code at the same time
+ * so we should be careful before modifying this structure.
+ */
+struct vdso_data {
+ u32 seq; /* Timebase sequence counter */
+
+ s32 clock_mode;
+ u64 cycle_last; /* Timebase at clocksource init */
+
+ struct vdso_clocksource cs[CLOCKSOURCE_BASES];
+ struct vdso_timestamp basetime[VDSO_BASES];
+
+ s32 tz_minuteswest; /* Timezone definitions */
+ s32 tz_dsttime;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch generalized the data structures in vgtod.h
from x86 private includes to general includes (include/vdso).
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 74 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 74 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..da346ad02b03
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/bits.h>
+#include <linux/types.h>
+#include <linux/time.h>
+
+#define VDSO_BASES (CLOCK_TAI + 1)
+#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
+ BIT(CLOCK_MONOTONIC) | \
+ BIT(CLOCK_MONOTONIC_RAW) | \
+ BIT(CLOCK_BOOTTIME) | \
+ BIT(CLOCK_TAI))
+#define VDSO_COARSE (BIT(CLOCK_REALTIME_COARSE) | \
+ BIT(CLOCK_MONOTONIC_COARSE))
+
+/*
+ * There is one vdso_timestamp object in vvar for each vDSO-accelerated
+ * clock_id. For high-resolution clocks, this encodes the time
+ * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
+ * the actual time.
+ *
+ * To be noticed that for highres clocks nsec is left-shifted by
+ * vdso_data.cs[x].shift.
+ */
+struct vdso_timestamp {
+ u64 sec;
+ u64 nsec;
+};
+
+#define CLOCKSOURCE_RAW 0
+#define CLOCKSOURCE_MONO 1
+#define CLOCKSOURCE_BASES (CLOCKSOURCE_MONO + 1)
+
+/*
+ * There is one vdso_clocksource object in vvar for each vDSO clocksource
+ * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
+ * and optimal in terms of access pattern.
+ *
+ * Note that mask and shift are the same for mono and raw.
+ */
+struct vdso_clocksource {
+ u64 mask; /* Clocksource mask */
+ u32 mult; /* Clocksource multiplier */
+ u32 shift; /* Clocksource shift */
+};
+
+/*
+ * vdso_data will be accessed by 32 and 64 bit code at the same time
+ * so we should be careful before modifying this structure.
+ */
+struct vdso_data {
+ u32 seq; /* Timebase sequence counter */
+
+ s32 clock_mode;
+ u64 cycle_last; /* Timebase at clocksource init */
+
+ struct vdso_clocksource cs[CLOCKSOURCE_BASES];
+ struct vdso_timestamp basetime[VDSO_BASES];
+
+ s32 tz_minuteswest; /* Timezone definitions */
+ s32 tz_dsttime;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
In an effort to unify the common code for managing the vdso library in
between all the architectures that support it, this patch tries to
provide a common format for the vdso datapage.
As a result of this, this patch generalized the data structures in vgtod.h
from x86 private includes to general includes (include/vdso).
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 74 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 74 insertions(+)
create mode 100644 include/vdso/datapage.h
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
new file mode 100644
index 000000000000..da346ad02b03
--- /dev/null
+++ b/include/vdso/datapage.h
@@ -0,0 +1,74 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_DATAPAGE_H
+#define __VDSO_DATAPAGE_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/bits.h>
+#include <linux/types.h>
+#include <linux/time.h>
+
+#define VDSO_BASES (CLOCK_TAI + 1)
+#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
+ BIT(CLOCK_MONOTONIC) | \
+ BIT(CLOCK_MONOTONIC_RAW) | \
+ BIT(CLOCK_BOOTTIME) | \
+ BIT(CLOCK_TAI))
+#define VDSO_COARSE (BIT(CLOCK_REALTIME_COARSE) | \
+ BIT(CLOCK_MONOTONIC_COARSE))
+
+/*
+ * There is one vdso_timestamp object in vvar for each vDSO-accelerated
+ * clock_id. For high-resolution clocks, this encodes the time
+ * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
+ * the actual time.
+ *
+ * To be noticed that for highres clocks nsec is left-shifted by
+ * vdso_data.cs[x].shift.
+ */
+struct vdso_timestamp {
+ u64 sec;
+ u64 nsec;
+};
+
+#define CLOCKSOURCE_RAW 0
+#define CLOCKSOURCE_MONO 1
+#define CLOCKSOURCE_BASES (CLOCKSOURCE_MONO + 1)
+
+/*
+ * There is one vdso_clocksource object in vvar for each vDSO clocksource
+ * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
+ * and optimal in terms of access pattern.
+ *
+ * Note that mask and shift are the same for mono and raw.
+ */
+struct vdso_clocksource {
+ u64 mask; /* Clocksource mask */
+ u32 mult; /* Clocksource multiplier */
+ u32 shift; /* Clocksource shift */
+};
+
+/*
+ * vdso_data will be accessed by 32 and 64 bit code at the same time
+ * so we should be careful before modifying this structure.
+ */
+struct vdso_data {
+ u32 seq; /* Timebase sequence counter */
+
+ s32 clock_mode;
+ u64 cycle_last; /* Timebase at clocksource init */
+
+ struct vdso_clocksource cs[CLOCKSOURCE_BASES];
+ struct vdso_timestamp basetime[VDSO_BASES];
+
+ s32 tz_minuteswest; /* Timezone definitions */
+ s32 tz_dsttime;
+};
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_DATAPAGE_H */
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
Try to unify the gettimeofday vdso implementation introducing
lib/vdso. The code contained in this library can ideally be
reused by all the architectures avoiding, where possible, code
duplication.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
include/vdso/helpers.h | 52 ++++++++++++
include/vdso/types.h | 39 +++++++++
lib/Kconfig | 5 ++
lib/vdso/Kconfig | 37 +++++++++
lib/vdso/Makefile | 22 +++++
lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++
7 files changed, 331 insertions(+)
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index da346ad02b03..ff332fcba73c 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,6 +9,7 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
+#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
new file mode 100644
index 000000000000..511dea979f6b
--- /dev/null
+++ b/include/vdso/helpers.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_HELPERS_H
+#define __VDSO_HELPERS_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <vdso/datapage.h>
+
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ seq = READ_ONCE(vd->seq);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ smp_rmb();
+ seq = READ_ONCE(vd->seq);
+ return seq != start;
+}
+
+static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
+{
+ ++vd->seq;
+ smp_wmb();
+}
+
+static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
+{
+ smp_wmb();
+ ++vd->seq;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_HELPERS_H */
diff --git a/include/vdso/types.h b/include/vdso/types.h
new file mode 100644
index 000000000000..f456a0a6a2e1
--- /dev/null
+++ b/include/vdso/types.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_TYPES_H
+#define __VDSO_TYPES_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+#include <linux/time.h>
+
+/*
+ * The definitions below are required to overcome the limitations
+ * of time_t on 32 bit architectures, which overflows in 2038.
+ * The new code should use the replacements based on time64_t and
+ * timespec64.
+ *
+ * The abstraction below will be updated once the migration to
+ * time64_t is complete.
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#ifdef ENABLE_COMPAT_VDSO
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#define __vdso_timespec __kernel_timespec
+#define __vdso_timeval __kernel_old_timeval
+#endif /* CONFIG_COMPAT_VDSO */
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_TYPES_H */
diff --git a/lib/Kconfig b/lib/Kconfig
index a9e56539bd11..dff3e3c782da 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -565,6 +565,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..34d91f952d70
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ default ""
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+ If a 64 bit compiler (i.e. x86_64) can compile the VDSO for
+ 32 bit, it does not need to define this parameter.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..39f92f7d3218
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
+
+/*
+ * The generic vDSO implementation requires that gettimeofday.h
+ * provides:
+ * - __arch_get_vdso_data(): to get the vdso datapage.
+ * - __arch_get_hw_counter(): to get the hw counter based on the
+ * clock_mode.
+ * - gettimeofday_fallback(): fallback for gettimeofday.
+ * - clock_gettime_fallback(): fallback for clock_gettime.
+ * - clock_getres_fallback(): fallback for clock_getres.
+ */
+#include <asm/vdso/gettimeofday.h>
+
+static notrace int do_hres(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u64 cycles, last, sec, ns;
+ u32 seq, cs_index = CLOCKSOURCE_MONO;
+
+ if (clk == CLOCK_MONOTONIC_RAW)
+ cs_index = CLOCKSOURCE_RAW;
+
+ do {
+ seq = vdso_read_begin(vd);
+ cycles = __arch_get_hw_counter(vd->clock_mode) &
+ vd->cs[cs_index].mask;
+ ns = vdso_ts->nsec;
+ last = vd->cycle_last;
+ if (unlikely((s64)cycles < 0))
+ return clock_gettime_fallback(clk, ts);
+ if (cycles > last)
+ ns += (cycles - last) * vd->cs[cs_index].mult;
+ ns >>= vd->cs[cs_index].shift;
+ sec = vdso_ts->sec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+
+ /*
+ * Do this outside the loop: a race inside the loop could result
+ * in __iter_div_u64_rem() being extremely slow.
+ */
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+
+static notrace void do_coarse(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u32 seq;
+
+ do {
+ seq = vdso_read_begin(vd);
+ ts->tv_sec = vdso_ts->sec;
+ ts->tv_nsec = vdso_ts->nsec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (likely(msk & VDSO_HRES)) {
+ return do_hres(vd, clock, ts);
+ } else if (msk & VDSO_COARSE) {
+ do_coarse(vd, clock, ts);
+ return 0;
+ }
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct __vdso_timespec ts;
+
+ if (do_hres(vd, CLOCK_REALTIME, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+#ifdef VDSO_HAS_TIME
+static notrace time_t __cvdso_time(time_t *time)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
+
+ if (time)
+ *time = t;
+
+ return t;
+}
+#endif /* VDSO_HAS_TIME */
+
+static notrace int __cvdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ u64 sec, ns;
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (msk & VDSO_HRES) {
+ /*
+ * Preserves the behaviour of posix_get_hrtimer_res().
+ */
+ sec = 0;
+ ns = MONOTONIC_RES_NSEC;
+ } else if (msk & VDSO_COARSE) {
+ /*
+ * Preserves the behaviour of posix_get_coarse_res().
+ */
+ ns = LOW_RES_NSEC;
+ sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ } else {
+ goto fallback;
+ }
+
+ if (res) {
+ res->tv_sec = sec;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+
+fallback:
+ return clock_getres_fallback(clock, res);
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
Try to unify the gettimeofday vdso implementation introducing
lib/vdso. The code contained in this library can ideally be
reused by all the architectures avoiding, where possible, code
duplication.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
include/vdso/helpers.h | 52 ++++++++++++
include/vdso/types.h | 39 +++++++++
lib/Kconfig | 5 ++
lib/vdso/Kconfig | 37 +++++++++
lib/vdso/Makefile | 22 +++++
lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++
7 files changed, 331 insertions(+)
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index da346ad02b03..ff332fcba73c 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,6 +9,7 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
+#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
new file mode 100644
index 000000000000..511dea979f6b
--- /dev/null
+++ b/include/vdso/helpers.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_HELPERS_H
+#define __VDSO_HELPERS_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <vdso/datapage.h>
+
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ seq = READ_ONCE(vd->seq);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ smp_rmb();
+ seq = READ_ONCE(vd->seq);
+ return seq != start;
+}
+
+static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
+{
+ ++vd->seq;
+ smp_wmb();
+}
+
+static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
+{
+ smp_wmb();
+ ++vd->seq;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_HELPERS_H */
diff --git a/include/vdso/types.h b/include/vdso/types.h
new file mode 100644
index 000000000000..f456a0a6a2e1
--- /dev/null
+++ b/include/vdso/types.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_TYPES_H
+#define __VDSO_TYPES_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+#include <linux/time.h>
+
+/*
+ * The definitions below are required to overcome the limitations
+ * of time_t on 32 bit architectures, which overflows in 2038.
+ * The new code should use the replacements based on time64_t and
+ * timespec64.
+ *
+ * The abstraction below will be updated once the migration to
+ * time64_t is complete.
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#ifdef ENABLE_COMPAT_VDSO
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#define __vdso_timespec __kernel_timespec
+#define __vdso_timeval __kernel_old_timeval
+#endif /* CONFIG_COMPAT_VDSO */
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_TYPES_H */
diff --git a/lib/Kconfig b/lib/Kconfig
index a9e56539bd11..dff3e3c782da 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -565,6 +565,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..34d91f952d70
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ default ""
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+ If a 64 bit compiler (i.e. x86_64) can compile the VDSO for
+ 32 bit, it does not need to define this parameter.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..39f92f7d3218
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
+
+/*
+ * The generic vDSO implementation requires that gettimeofday.h
+ * provides:
+ * - __arch_get_vdso_data(): to get the vdso datapage.
+ * - __arch_get_hw_counter(): to get the hw counter based on the
+ * clock_mode.
+ * - gettimeofday_fallback(): fallback for gettimeofday.
+ * - clock_gettime_fallback(): fallback for clock_gettime.
+ * - clock_getres_fallback(): fallback for clock_getres.
+ */
+#include <asm/vdso/gettimeofday.h>
+
+static notrace int do_hres(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u64 cycles, last, sec, ns;
+ u32 seq, cs_index = CLOCKSOURCE_MONO;
+
+ if (clk == CLOCK_MONOTONIC_RAW)
+ cs_index = CLOCKSOURCE_RAW;
+
+ do {
+ seq = vdso_read_begin(vd);
+ cycles = __arch_get_hw_counter(vd->clock_mode) &
+ vd->cs[cs_index].mask;
+ ns = vdso_ts->nsec;
+ last = vd->cycle_last;
+ if (unlikely((s64)cycles < 0))
+ return clock_gettime_fallback(clk, ts);
+ if (cycles > last)
+ ns += (cycles - last) * vd->cs[cs_index].mult;
+ ns >>= vd->cs[cs_index].shift;
+ sec = vdso_ts->sec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+
+ /*
+ * Do this outside the loop: a race inside the loop could result
+ * in __iter_div_u64_rem() being extremely slow.
+ */
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+
+static notrace void do_coarse(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u32 seq;
+
+ do {
+ seq = vdso_read_begin(vd);
+ ts->tv_sec = vdso_ts->sec;
+ ts->tv_nsec = vdso_ts->nsec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (likely(msk & VDSO_HRES)) {
+ return do_hres(vd, clock, ts);
+ } else if (msk & VDSO_COARSE) {
+ do_coarse(vd, clock, ts);
+ return 0;
+ }
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct __vdso_timespec ts;
+
+ if (do_hres(vd, CLOCK_REALTIME, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+#ifdef VDSO_HAS_TIME
+static notrace time_t __cvdso_time(time_t *time)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
+
+ if (time)
+ *time = t;
+
+ return t;
+}
+#endif /* VDSO_HAS_TIME */
+
+static notrace int __cvdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ u64 sec, ns;
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (msk & VDSO_HRES) {
+ /*
+ * Preserves the behaviour of posix_get_hrtimer_res().
+ */
+ sec = 0;
+ ns = MONOTONIC_RES_NSEC;
+ } else if (msk & VDSO_COARSE) {
+ /*
+ * Preserves the behaviour of posix_get_coarse_res().
+ */
+ ns = LOW_RES_NSEC;
+ sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ } else {
+ goto fallback;
+ }
+
+ if (res) {
+ res->tv_sec = sec;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+
+fallback:
+ return clock_getres_fallback(clock, res);
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
In the last few years we assisted to an explosion of vdso
implementations that mostly share similar code.
Try to unify the gettimeofday vdso implementation introducing
lib/vdso. The code contained in this library can ideally be
reused by all the architectures avoiding, where possible, code
duplication.
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
include/vdso/datapage.h | 1 +
include/vdso/helpers.h | 52 ++++++++++++
include/vdso/types.h | 39 +++++++++
lib/Kconfig | 5 ++
lib/vdso/Kconfig | 37 +++++++++
lib/vdso/Makefile | 22 +++++
lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++
7 files changed, 331 insertions(+)
create mode 100644 include/vdso/helpers.h
create mode 100644 include/vdso/types.h
create mode 100644 lib/vdso/Kconfig
create mode 100644 lib/vdso/Makefile
create mode 100644 lib/vdso/gettimeofday.c
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index da346ad02b03..ff332fcba73c 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,6 +9,7 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
+#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
new file mode 100644
index 000000000000..511dea979f6b
--- /dev/null
+++ b/include/vdso/helpers.h
@@ -0,0 +1,52 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_HELPERS_H
+#define __VDSO_HELPERS_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <vdso/datapage.h>
+
+static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
+{
+ u32 seq;
+
+repeat:
+ seq = READ_ONCE(vd->seq);
+ if (seq & 1) {
+ cpu_relax();
+ goto repeat;
+ }
+
+ smp_rmb();
+ return seq;
+}
+
+static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
+ u32 start)
+{
+ u32 seq;
+
+ smp_rmb();
+ seq = READ_ONCE(vd->seq);
+ return seq != start;
+}
+
+static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
+{
+ ++vd->seq;
+ smp_wmb();
+}
+
+static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
+{
+ smp_wmb();
+ ++vd->seq;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_HELPERS_H */
diff --git a/include/vdso/types.h b/include/vdso/types.h
new file mode 100644
index 000000000000..f456a0a6a2e1
--- /dev/null
+++ b/include/vdso/types.h
@@ -0,0 +1,39 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __VDSO_TYPES_H
+#define __VDSO_TYPES_H
+
+#ifdef __KERNEL__
+
+#ifndef __ASSEMBLY__
+
+#include <linux/types.h>
+#include <linux/time.h>
+
+/*
+ * The definitions below are required to overcome the limitations
+ * of time_t on 32 bit architectures, which overflows in 2038.
+ * The new code should use the replacements based on time64_t and
+ * timespec64.
+ *
+ * The abstraction below will be updated once the migration to
+ * time64_t is complete.
+ */
+#ifdef CONFIG_GENERIC_VDSO_32
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#ifdef ENABLE_COMPAT_VDSO
+#define __vdso_timespec old_timespec32
+#define __vdso_timeval old_timeval32
+#else
+#define __vdso_timespec __kernel_timespec
+#define __vdso_timeval __kernel_old_timeval
+#endif /* CONFIG_COMPAT_VDSO */
+#endif /* CONFIG_GENERIC_VDSO_32 */
+
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __KERNEL__ */
+
+#endif /* __VDSO_TYPES_H */
diff --git a/lib/Kconfig b/lib/Kconfig
index a9e56539bd11..dff3e3c782da 100644
--- a/lib/Kconfig
+++ b/lib/Kconfig
@@ -565,6 +565,11 @@ config OID_REGISTRY
config UCS2_STRING
tristate
+#
+# generic vdso
+#
+source "lib/vdso/Kconfig"
+
source "lib/fonts/Kconfig"
config SG_SPLIT
diff --git a/lib/vdso/Kconfig b/lib/vdso/Kconfig
new file mode 100644
index 000000000000..34d91f952d70
--- /dev/null
+++ b/lib/vdso/Kconfig
@@ -0,0 +1,37 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config HAVE_GENERIC_VDSO
+ bool
+ default n
+
+if HAVE_GENERIC_VDSO
+
+config GENERIC_GETTIMEOFDAY
+ bool
+ help
+ This is a generic implementation of gettimeofday vdso.
+ Each architecture that enables this feature has to
+ provide the fallback implementation.
+
+config GENERIC_VDSO_32
+ bool
+ depends on GENERIC_GETTIMEOFDAY && !64BIT
+ help
+ This config option helps to avoid possible performance issues
+ in 32 bit only architectures.
+
+config GENERIC_COMPAT_VDSO
+ bool
+ help
+ This config option enables the compat VDSO layer.
+
+config CROSS_COMPILE_COMPAT_VDSO
+ string "32 bit Toolchain prefix for compat vDSO"
+ default ""
+ depends on GENERIC_COMPAT_VDSO
+ help
+ Defines the cross-compiler prefix for compiling compat vDSO.
+ If a 64 bit compiler (i.e. x86_64) can compile the VDSO for
+ 32 bit, it does not need to define this parameter.
+
+endif
diff --git a/lib/vdso/Makefile b/lib/vdso/Makefile
new file mode 100644
index 000000000000..c415a685d61b
--- /dev/null
+++ b/lib/vdso/Makefile
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: GPL-2.0
+
+GENERIC_VDSO_MK_PATH := $(abspath $(lastword $(MAKEFILE_LIST)))
+GENERIC_VDSO_DIR := $(dir $(GENERIC_VDSO_MK_PATH))
+
+c-gettimeofday-$(CONFIG_GENERIC_GETTIMEOFDAY) := $(addprefix $(GENERIC_VDSO_DIR), gettimeofday.c)
+
+# This cmd checks that the vdso library does not contain absolute relocation
+# It has to be called after the linking of the vdso library and requires it
+# as a parameter.
+#
+# $(ARCH_REL_TYPE_ABS) is defined in the arch specific makefile and corresponds
+# to the absolute relocation types printed by "objdump -R" and accepted by the
+# dynamic linker.
+ifndef ARCH_REL_TYPE_ABS
+$(error ARCH_REL_TYPE_ABS is not set)
+endif
+
+quiet_cmd_vdso_check = VDSOCHK $@
+ cmd_vdso_check = if $(OBJDUMP) -R $@ | egrep -h "$(ARCH_REL_TYPE_ABS)"; \
+ then (echo >&2 "$@: dynamic relocations are not supported"; \
+ rm -f $@; /bin/false); fi
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
new file mode 100644
index 000000000000..39f92f7d3218
--- /dev/null
+++ b/lib/vdso/gettimeofday.c
@@ -0,0 +1,175 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Generic userspace implementations of gettimeofday() and similar.
+ */
+#include <linux/compiler.h>
+#include <linux/math64.h>
+#include <linux/time.h>
+#include <linux/kernel.h>
+#include <linux/hrtimer.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
+
+/*
+ * The generic vDSO implementation requires that gettimeofday.h
+ * provides:
+ * - __arch_get_vdso_data(): to get the vdso datapage.
+ * - __arch_get_hw_counter(): to get the hw counter based on the
+ * clock_mode.
+ * - gettimeofday_fallback(): fallback for gettimeofday.
+ * - clock_gettime_fallback(): fallback for clock_gettime.
+ * - clock_getres_fallback(): fallback for clock_getres.
+ */
+#include <asm/vdso/gettimeofday.h>
+
+static notrace int do_hres(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u64 cycles, last, sec, ns;
+ u32 seq, cs_index = CLOCKSOURCE_MONO;
+
+ if (clk == CLOCK_MONOTONIC_RAW)
+ cs_index = CLOCKSOURCE_RAW;
+
+ do {
+ seq = vdso_read_begin(vd);
+ cycles = __arch_get_hw_counter(vd->clock_mode) &
+ vd->cs[cs_index].mask;
+ ns = vdso_ts->nsec;
+ last = vd->cycle_last;
+ if (unlikely((s64)cycles < 0))
+ return clock_gettime_fallback(clk, ts);
+ if (cycles > last)
+ ns += (cycles - last) * vd->cs[cs_index].mult;
+ ns >>= vd->cs[cs_index].shift;
+ sec = vdso_ts->sec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+
+ /*
+ * Do this outside the loop: a race inside the loop could result
+ * in __iter_div_u64_rem() being extremely slow.
+ */
+ ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ ts->tv_nsec = ns;
+
+ return 0;
+}
+
+static notrace void do_coarse(const struct vdso_data *vd,
+ clockid_t clk,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
+ u32 seq;
+
+ do {
+ seq = vdso_read_begin(vd);
+ ts->tv_sec = vdso_ts->sec;
+ ts->tv_nsec = vdso_ts->nsec;
+ } while (unlikely(vdso_read_retry(vd, seq)));
+}
+
+static notrace int __cvdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (likely(msk & VDSO_HRES)) {
+ return do_hres(vd, clock, ts);
+ } else if (msk & VDSO_COARSE) {
+ do_coarse(vd, clock, ts);
+ return 0;
+ }
+fallback:
+ return clock_gettime_fallback(clock, ts);
+}
+
+static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+
+ if (likely(tv != NULL)) {
+ struct __vdso_timespec ts;
+
+ if (do_hres(vd, CLOCK_REALTIME, &ts))
+ return gettimeofday_fallback(tv, tz);
+
+ tv->tv_sec = ts.tv_sec;
+ tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
+ }
+
+ if (unlikely(tz != NULL)) {
+ tz->tz_minuteswest = vd->tz_minuteswest;
+ tz->tz_dsttime = vd->tz_dsttime;
+ }
+
+ return 0;
+}
+
+#ifdef VDSO_HAS_TIME
+static notrace time_t __cvdso_time(time_t *time)
+{
+ const struct vdso_data *vd = __arch_get_vdso_data();
+ time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
+
+ if (time)
+ *time = t;
+
+ return t;
+}
+#endif /* VDSO_HAS_TIME */
+
+static notrace int __cvdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ u64 sec, ns;
+ u32 msk;
+
+ /* Check for negative values or invalid clocks */
+ if (unlikely((u32) clock >= MAX_CLOCKS))
+ goto fallback;
+
+ /*
+ * Convert the clockid to a bitmask and use it to check which
+ * clocks are handled in the VDSO directly.
+ */
+ msk = 1U << clock;
+ if (msk & VDSO_HRES) {
+ /*
+ * Preserves the behaviour of posix_get_hrtimer_res().
+ */
+ sec = 0;
+ ns = MONOTONIC_RES_NSEC;
+ } else if (msk & VDSO_COARSE) {
+ /*
+ * Preserves the behaviour of posix_get_coarse_res().
+ */
+ ns = LOW_RES_NSEC;
+ sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
+ } else {
+ goto fallback;
+ }
+
+ if (res) {
+ res->tv_sec = sec;
+ res->tv_nsec = ns;
+ }
+
+ return 0;
+
+fallback:
+ return clock_getres_fallback(clock, res);
+}
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 03/23] arm64: Build vDSO with -ffixed-x18
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne, Mark Salyzyn
From: Peter Collingbourne <pcc@google.com>
The vDSO needs to be build with x18 reserved in order to accommodate
userspace platform ABIs built on top of Linux that use the register
to carry inter-procedural state, as provided for by the AAPCS.
An example of such a platform ABI is the one that will be used by an
upcoming version of Android.
Although this change is currently a no-op due to the fact that the vDSO
is currently implemented in pure assembly on arm64, it is necessary
in order to prepare for another change [1] that will add C code to
the vDSO.
[1] https://patchwork.kernel.org/patch/10044501/
Signed-off-by: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Mark Salyzyn <salyzyn@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/kernel/vdso/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..8cd4707f832a 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -12,7 +12,7 @@ obj-vdso := gettimeofday.o note.o sigreturn.o
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 03/23] arm64: Build vDSO with -ffixed-x18
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes, Mark Salyzyn
From: Peter Collingbourne <pcc@google.com>
The vDSO needs to be build with x18 reserved in order to accommodate
userspace platform ABIs built on top of Linux that use the register
to carry inter-procedural state, as provided for by the AAPCS.
An example of such a platform ABI is the one that will be used by an
upcoming version of Android.
Although this change is currently a no-op due to the fact that the vDSO
is currently implemented in pure assembly on arm64, it is necessary
in order to prepare for another change [1] that will add C code to
the vDSO.
[1] https://patchwork.kernel.org/patch/10044501/
Signed-off-by: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Mark Salyzyn <salyzyn@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/kernel/vdso/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..8cd4707f832a 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -12,7 +12,7 @@ obj-vdso := gettimeofday.o note.o sigreturn.o
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 03/23] arm64: Build vDSO with -ffixed-x18
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne, Mark Salyzyn
From: Peter Collingbourne <pcc@google.com>
The vDSO needs to be build with x18 reserved in order to accommodate
userspace platform ABIs built on top of Linux that use the register
to carry inter-procedural state, as provided for by the AAPCS.
An example of such a platform ABI is the one that will be used by an
upcoming version of Android.
Although this change is currently a no-op due to the fact that the vDSO
is currently implemented in pure assembly on arm64, it is necessary
in order to prepare for another change [1] that will add C code to
the vDSO.
[1] https://patchwork.kernel.org/patch/10044501/
Signed-off-by: Peter Collingbourne <pcc@google.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
Cc: Mark Salyzyn <salyzyn@google.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: linux-arm-kernel@lists.infradead.org
---
arch/arm64/kernel/vdso/Makefile | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index b215c712d897..8cd4707f832a 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -12,7 +12,7 @@ obj-vdso := gettimeofday.o note.o sigreturn.o
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
Re-implement the gettimeofday vdso in C in order to use lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME
and CLOCK_TAI.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 2 +
arch/arm64/include/asm/vdso/gettimeofday.h | 85 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 39 ++-
arch/arm64/kernel/vdso.c | 102 +++++--
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 ++
8 files changed, 237 insertions(+), 422 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index a4168d366127..d898da2e20f5 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -100,6 +100,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
@@ -154,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..0aff6362533f
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct __vdso_timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 65b8afc84466..4adb52190a90 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -108,22 +108,31 @@ int main(void)
DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
- DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
- DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
- DEFINE(VDSO_RAW_TIME_NSEC, offsetof(struct vdso_data, raw_time_nsec));
- DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
- DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
- DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
- DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
- DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
- DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
- DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
- DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
- DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
- DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_SEQ, offsetof(struct vdso_data, seq));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
+ DEFINE(VDSO_CYCLE_LAST, offsetof(struct vdso_data, cycle_last));
+ DEFINE(VDSO_MONO_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mask));
+ DEFINE(VDSO_MONO_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mult));
+ DEFINE(VDSO_MONO_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].shift));
+ DEFINE(VDSO_RAW_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mask));
+ DEFINE(VDSO_RAW_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mult));
+ DEFINE(VDSO_RAW_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].shift));
+ DEFINE(VDSO_REALTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].sec));
+ DEFINE(VDSO_REALTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].nsec));
+ DEFINE(VDSO_MONO_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].sec));
+ DEFINE(VDSO_MONO_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].nsec));
+ DEFINE(VDSO_MONO_RAW_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].sec));
+ DEFINE(VDSO_MONO_RAW_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].nsec));
+ DEFINE(VDSO_BOOTTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].sec));
+ DEFINE(VDSO_BOOTTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].nsec));
+ DEFINE(VDSO_TAI_SEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].sec));
+ DEFINE(VDSO_TAI_NSEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].nsec));
+ DEFINE(VDSO_RT_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].sec));
+ DEFINE(VDSO_RT_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].nsec));
+ DEFINE(VDSO_MONO_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].sec));
+ DEFINE(VDSO_MONO_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].nsec));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
- DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..8af3625fd9c5 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,12 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,38 +216,93 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
void update_vsyscall(struct timekeeper *tk)
{
+ struct vdso_timestamp *vdso_ts;
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
-
- ++vdso_data->tb_seq_count;
- smp_wmb();
-
- vdso_data->use_syscall = use_syscall;
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift;
- vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
if (!use_syscall) {
- /* tkr_mono.cycle_last == tkr_raw.cycle_last */
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->raw_time_sec = tk->raw_sec;
- vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mono_mult = tk->tkr_mono.mult;
- vdso_data->cs_raw_mult = tk->tkr_raw.mult;
- /* tkr_mono.shift == tkr_raw.shift */
- vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = VDSO_PRECISION_MASK;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = VDSO_PRECISION_MASK;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- smp_wmb();
- ++vdso_data->tb_seq_count;
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 8cd4707f832a..c904b2a9b255 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,31 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS := -Bsymbolic
+
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +48,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +65,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
- cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
+ cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) -Wl,-n -Wl,-T $^ -o $@
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..160e68196339
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
Re-implement the gettimeofday vdso in C in order to use lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME
and CLOCK_TAI.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 2 +
arch/arm64/include/asm/vdso/gettimeofday.h | 85 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 39 ++-
arch/arm64/kernel/vdso.c | 102 +++++--
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 ++
8 files changed, 237 insertions(+), 422 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index a4168d366127..d898da2e20f5 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -100,6 +100,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
@@ -154,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..0aff6362533f
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct __vdso_timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 65b8afc84466..4adb52190a90 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -108,22 +108,31 @@ int main(void)
DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
- DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
- DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
- DEFINE(VDSO_RAW_TIME_NSEC, offsetof(struct vdso_data, raw_time_nsec));
- DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
- DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
- DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
- DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
- DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
- DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
- DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
- DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
- DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
- DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_SEQ, offsetof(struct vdso_data, seq));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
+ DEFINE(VDSO_CYCLE_LAST, offsetof(struct vdso_data, cycle_last));
+ DEFINE(VDSO_MONO_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mask));
+ DEFINE(VDSO_MONO_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mult));
+ DEFINE(VDSO_MONO_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].shift));
+ DEFINE(VDSO_RAW_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mask));
+ DEFINE(VDSO_RAW_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mult));
+ DEFINE(VDSO_RAW_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].shift));
+ DEFINE(VDSO_REALTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].sec));
+ DEFINE(VDSO_REALTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].nsec));
+ DEFINE(VDSO_MONO_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].sec));
+ DEFINE(VDSO_MONO_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].nsec));
+ DEFINE(VDSO_MONO_RAW_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].sec));
+ DEFINE(VDSO_MONO_RAW_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].nsec));
+ DEFINE(VDSO_BOOTTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].sec));
+ DEFINE(VDSO_BOOTTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].nsec));
+ DEFINE(VDSO_TAI_SEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].sec));
+ DEFINE(VDSO_TAI_NSEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].nsec));
+ DEFINE(VDSO_RT_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].sec));
+ DEFINE(VDSO_RT_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].nsec));
+ DEFINE(VDSO_MONO_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].sec));
+ DEFINE(VDSO_MONO_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].nsec));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
- DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..8af3625fd9c5 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,12 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,38 +216,93 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
void update_vsyscall(struct timekeeper *tk)
{
+ struct vdso_timestamp *vdso_ts;
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
-
- ++vdso_data->tb_seq_count;
- smp_wmb();
-
- vdso_data->use_syscall = use_syscall;
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift;
- vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
if (!use_syscall) {
- /* tkr_mono.cycle_last == tkr_raw.cycle_last */
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->raw_time_sec = tk->raw_sec;
- vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mono_mult = tk->tkr_mono.mult;
- vdso_data->cs_raw_mult = tk->tkr_raw.mult;
- /* tkr_mono.shift == tkr_raw.shift */
- vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = VDSO_PRECISION_MASK;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = VDSO_PRECISION_MASK;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- smp_wmb();
- ++vdso_data->tb_seq_count;
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 8cd4707f832a..c904b2a9b255 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,31 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS := -Bsymbolic
+
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +48,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +65,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
- cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
+ cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) -Wl,-n -Wl,-T $^ -o $@
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..160e68196339
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
To take advantage of the commonly defined vdso interface for
gettimeofday the architectural code requires an adaptation.
Re-implement the gettimeofday vdso in C in order to use lib/vdso.
With the new implementation arm64 gains support for CLOCK_BOOTTIME
and CLOCK_TAI.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 2 +
arch/arm64/include/asm/vdso/gettimeofday.h | 85 ++++++
arch/arm64/include/asm/vdso_datapage.h | 47 ---
arch/arm64/kernel/asm-offsets.c | 39 ++-
arch/arm64/kernel/vdso.c | 102 +++++--
arch/arm64/kernel/vdso/Makefile | 28 +-
arch/arm64/kernel/vdso/gettimeofday.S | 328 ---------------------
arch/arm64/kernel/vdso/vgettimeofday.c | 28 ++
8 files changed, 237 insertions(+), 422 deletions(-)
create mode 100644 arch/arm64/include/asm/vdso/gettimeofday.h
delete mode 100644 arch/arm64/include/asm/vdso_datapage.h
delete mode 100644 arch/arm64/kernel/vdso/gettimeofday.S
create mode 100644 arch/arm64/kernel/vdso/vgettimeofday.c
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index a4168d366127..d898da2e20f5 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -100,6 +100,7 @@ config ARM64
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
@@ -154,6 +155,7 @@ config ARM64
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_KPROBES
select HAVE_KRETPROBES
+ select HAVE_GENERIC_VDSO
select IOMMU_DMA if IOMMU_SUPPORT
select IRQ_DOMAIN
select IRQ_FORCED_THREADING
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..0aff6362533f
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,85 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data _vdso_data;
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("x1") = _tz;
+ register struct __vdso_timeval *tv asm("x0") = _tv;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_gettimeofday;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_gettime;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("x1") = _ts;
+ register clockid_t clkid asm("x0") = _clkid;
+ register long ret asm ("x0");
+ register long nr asm("x8") = __NR_clock_getres;
+
+ asm volatile(
+ " svc #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ asm volatile("mrs %0, cntvct_el0" : "=r" (res) :: "memory");
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return &_vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/include/asm/vdso_datapage.h b/arch/arm64/include/asm/vdso_datapage.h
deleted file mode 100644
index 2b9a63771eda..000000000000
--- a/arch/arm64/include/asm/vdso_datapage.h
+++ /dev/null
@@ -1,47 +0,0 @@
-/*
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- */
-#ifndef __ASM_VDSO_DATAPAGE_H
-#define __ASM_VDSO_DATAPAGE_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-struct vdso_data {
- __u64 cs_cycle_last; /* Timebase at clocksource init */
- __u64 raw_time_sec; /* Raw time */
- __u64 raw_time_nsec;
- __u64 xtime_clock_sec; /* Kernel time */
- __u64 xtime_clock_nsec;
- __u64 xtime_coarse_sec; /* Coarse time */
- __u64 xtime_coarse_nsec;
- __u64 wtm_clock_sec; /* Wall to monotonic time */
- __u64 wtm_clock_nsec;
- __u32 tb_seq_count; /* Timebase sequence counter */
- /* cs_* members must be adjacent and in this order (ldp accesses) */
- __u32 cs_mono_mult; /* NTP-adjusted clocksource multiplier */
- __u32 cs_shift; /* Clocksource shift (mono = raw) */
- __u32 cs_raw_mult; /* Raw clocksource multiplier */
- __u32 tz_minuteswest; /* Whacky timezone stuff */
- __u32 tz_dsttime;
- __u32 use_syscall;
-};
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __ASM_VDSO_DATAPAGE_H */
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 65b8afc84466..4adb52190a90 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -25,13 +25,13 @@
#include <linux/kvm_host.h>
#include <linux/preempt.h>
#include <linux/suspend.h>
+#include <vdso/datapage.h>
#include <asm/cpufeature.h>
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
-#include <asm/vdso_datapage.h>
#include <linux/kbuild.h>
#include <linux/arm-smccc.h>
@@ -108,22 +108,31 @@ int main(void)
DEFINE(CLOCK_COARSE_RES, LOW_RES_NSEC);
DEFINE(NSEC_PER_SEC, NSEC_PER_SEC);
BLANK();
- DEFINE(VDSO_CS_CYCLE_LAST, offsetof(struct vdso_data, cs_cycle_last));
- DEFINE(VDSO_RAW_TIME_SEC, offsetof(struct vdso_data, raw_time_sec));
- DEFINE(VDSO_RAW_TIME_NSEC, offsetof(struct vdso_data, raw_time_nsec));
- DEFINE(VDSO_XTIME_CLK_SEC, offsetof(struct vdso_data, xtime_clock_sec));
- DEFINE(VDSO_XTIME_CLK_NSEC, offsetof(struct vdso_data, xtime_clock_nsec));
- DEFINE(VDSO_XTIME_CRS_SEC, offsetof(struct vdso_data, xtime_coarse_sec));
- DEFINE(VDSO_XTIME_CRS_NSEC, offsetof(struct vdso_data, xtime_coarse_nsec));
- DEFINE(VDSO_WTM_CLK_SEC, offsetof(struct vdso_data, wtm_clock_sec));
- DEFINE(VDSO_WTM_CLK_NSEC, offsetof(struct vdso_data, wtm_clock_nsec));
- DEFINE(VDSO_TB_SEQ_COUNT, offsetof(struct vdso_data, tb_seq_count));
- DEFINE(VDSO_CS_MONO_MULT, offsetof(struct vdso_data, cs_mono_mult));
- DEFINE(VDSO_CS_RAW_MULT, offsetof(struct vdso_data, cs_raw_mult));
- DEFINE(VDSO_CS_SHIFT, offsetof(struct vdso_data, cs_shift));
+ DEFINE(VDSO_SEQ, offsetof(struct vdso_data, seq));
+ DEFINE(VDSO_CLK_MODE, offsetof(struct vdso_data, clock_mode));
+ DEFINE(VDSO_CYCLE_LAST, offsetof(struct vdso_data, cycle_last));
+ DEFINE(VDSO_MONO_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mask));
+ DEFINE(VDSO_MONO_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].mult));
+ DEFINE(VDSO_MONO_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_MONO].shift));
+ DEFINE(VDSO_RAW_MASK, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mask));
+ DEFINE(VDSO_RAW_MULT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].mult));
+ DEFINE(VDSO_RAW_SHIFT, offsetof(struct vdso_data, cs[CLOCKSOURCE_RAW].shift));
+ DEFINE(VDSO_REALTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].sec));
+ DEFINE(VDSO_REALTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME].nsec));
+ DEFINE(VDSO_MONO_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].sec));
+ DEFINE(VDSO_MONO_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC].nsec));
+ DEFINE(VDSO_MONO_RAW_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].sec));
+ DEFINE(VDSO_MONO_RAW_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_RAW].nsec));
+ DEFINE(VDSO_BOOTTIME_SEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].sec));
+ DEFINE(VDSO_BOOTTIME_NSEC, offsetof(struct vdso_data, basetime[CLOCK_BOOTTIME].nsec));
+ DEFINE(VDSO_TAI_SEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].sec));
+ DEFINE(VDSO_TAI_NSEC, offsetof(struct vdso_data, basetime[CLOCK_TAI].nsec));
+ DEFINE(VDSO_RT_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].sec));
+ DEFINE(VDSO_RT_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_REALTIME_COARSE].nsec));
+ DEFINE(VDSO_MONO_COARSE_SEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].sec));
+ DEFINE(VDSO_MONO_COARSE_NSEC, offsetof(struct vdso_data, basetime[CLOCK_MONOTONIC_COARSE].nsec));
DEFINE(VDSO_TZ_MINWEST, offsetof(struct vdso_data, tz_minuteswest));
DEFINE(VDSO_TZ_DSTTIME, offsetof(struct vdso_data, tz_dsttime));
- DEFINE(VDSO_USE_SYSCALL, offsetof(struct vdso_data, use_syscall));
BLANK();
DEFINE(TVAL_TV_SEC, offsetof(struct timeval, tv_sec));
DEFINE(TVAL_TV_USEC, offsetof(struct timeval, tv_usec));
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 2d419006ad43..8af3625fd9c5 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -31,11 +31,12 @@
#include <linux/slab.h>
#include <linux/timekeeper_internal.h>
#include <linux/vmalloc.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <asm/cacheflush.h>
#include <asm/signal32.h>
#include <asm/vdso.h>
-#include <asm/vdso_datapage.h>
extern char vdso_start[], vdso_end[];
static unsigned long vdso_pages __ro_after_init;
@@ -215,38 +216,93 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
return PTR_ERR(ret);
}
+#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
+
/*
* Update the vDSO data page to keep in sync with kernel timekeeping.
*/
void update_vsyscall(struct timekeeper *tk)
{
+ struct vdso_timestamp *vdso_ts;
u32 use_syscall = !tk->tkr_mono.clock->archdata.vdso_direct;
-
- ++vdso_data->tb_seq_count;
- smp_wmb();
-
- vdso_data->use_syscall = use_syscall;
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift;
- vdso_data->wtm_clock_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
if (!use_syscall) {
- /* tkr_mono.cycle_last == tkr_raw.cycle_last */
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->raw_time_sec = tk->raw_sec;
- vdso_data->raw_time_nsec = tk->tkr_raw.xtime_nsec;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mono_mult = tk->tkr_mono.mult;
- vdso_data->cs_raw_mult = tk->tkr_raw.mult;
- /* tkr_mono.shift == tkr_raw.shift */
- vdso_data->cs_shift = tk->tkr_mono.shift;
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = VDSO_PRECISION_MASK;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = VDSO_PRECISION_MASK;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- smp_wmb();
- ++vdso_data->tb_seq_count;
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
index 8cd4707f832a..c904b2a9b255 100644
--- a/arch/arm64/kernel/vdso/Makefile
+++ b/arch/arm64/kernel/vdso/Makefile
@@ -6,16 +6,31 @@
# Heavily based on the vDSO Makefiles for other archs.
#
-obj-vdso := gettimeofday.o note.o sigreturn.o
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso := vgettimeofday.o note.o sigreturn.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg
obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
-ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
+ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
+ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS := -Bsymbolic
+
+CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny
+else
+CFLAGS_vgettimeofday.o = -O2 -mcmodel=tiny -include $(c-gettimeofday-y)
+endif
+
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -33,6 +48,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file, .lds has to be first
$(obj)/vdso.so.dbg: $(src)/vdso.lds $(obj-vdso)
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
# Strip rule for the .so file
$(obj)/%.so: OBJCOPYFLAGS := -S
@@ -49,15 +65,9 @@ endef
include/generated/vdso-offsets.h: $(obj)/vdso.so.dbg FORCE
$(call if_changed,vdsosym)
-# Assembly rules for the .S files
-$(obj-vdso): %.o: %.S FORCE
- $(call if_changed_dep,vdsoas)
-
# Actual build commands
quiet_cmd_vdsold = VDSOL $@
- cmd_vdsold = $(CC) $(c_flags) -Wl,-n -Wl,-T $^ -o $@
-quiet_cmd_vdsoas = VDSOA $@
- cmd_vdsoas = $(CC) $(a_flags) -c -o $@ $<
+ cmd_vdsold = $(CC) $(c_flags) $(VDSO_LDFLAGS) -Wl,-n -Wl,-T $^ -o $@
# Install commands for the unstripped file
quiet_cmd_vdso_install = INSTALL $@
diff --git a/arch/arm64/kernel/vdso/gettimeofday.S b/arch/arm64/kernel/vdso/gettimeofday.S
deleted file mode 100644
index c39872a7b03c..000000000000
--- a/arch/arm64/kernel/vdso/gettimeofday.S
+++ /dev/null
@@ -1,328 +0,0 @@
-/*
- * Userspace implementations of gettimeofday() and friends.
- *
- * Copyright (C) 2012 ARM Limited
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- * Author: Will Deacon <will.deacon@arm.com>
- */
-
-#include <linux/linkage.h>
-#include <asm/asm-offsets.h>
-#include <asm/unistd.h>
-
-#define NSEC_PER_SEC_LO16 0xca00
-#define NSEC_PER_SEC_HI16 0x3b9a
-
-vdso_data .req x6
-seqcnt .req w7
-w_tmp .req w8
-x_tmp .req x8
-
-/*
- * Conventions for macro arguments:
- * - An argument is write-only if its name starts with "res".
- * - All other arguments are read-only, unless otherwise specified.
- */
-
- .macro seqcnt_acquire
-9999: ldr seqcnt, [vdso_data, #VDSO_TB_SEQ_COUNT]
- tbnz seqcnt, #0, 9999b
- dmb ishld
- .endm
-
- .macro seqcnt_check fail
- dmb ishld
- ldr w_tmp, [vdso_data, #VDSO_TB_SEQ_COUNT]
- cmp w_tmp, seqcnt
- b.ne \fail
- .endm
-
- .macro syscall_check fail
- ldr w_tmp, [vdso_data, #VDSO_USE_SYSCALL]
- cbnz w_tmp, \fail
- .endm
-
- .macro get_nsec_per_sec res
- mov \res, #NSEC_PER_SEC_LO16
- movk \res, #NSEC_PER_SEC_HI16, lsl #16
- .endm
-
- /*
- * Returns the clock delta, in nanoseconds left-shifted by the clock
- * shift.
- */
- .macro get_clock_shifted_nsec res, cycle_last, mult
- /* Read the virtual counter. */
- isb
- mrs x_tmp, cntvct_el0
- /* Calculate cycle delta and convert to ns. */
- sub \res, x_tmp, \cycle_last
- /* We can only guarantee 56 bits of precision. */
- movn x_tmp, #0xff00, lsl #48
- and \res, x_tmp, \res
- mul \res, \res, \mult
- .endm
-
- /*
- * Returns in res_{sec,nsec} the REALTIME timespec, based on the
- * "wall time" (xtime) and the clock_mono delta.
- */
- .macro get_ts_realtime res_sec, res_nsec, \
- clock_nsec, xtime_sec, xtime_nsec, nsec_to_sec
- add \res_nsec, \clock_nsec, \xtime_nsec
- udiv x_tmp, \res_nsec, \nsec_to_sec
- add \res_sec, \xtime_sec, x_tmp
- msub \res_nsec, x_tmp, \nsec_to_sec, \res_nsec
- .endm
-
- /*
- * Returns in res_{sec,nsec} the timespec based on the clock_raw delta,
- * used for CLOCK_MONOTONIC_RAW.
- */
- .macro get_ts_clock_raw res_sec, res_nsec, clock_nsec, nsec_to_sec
- udiv \res_sec, \clock_nsec, \nsec_to_sec
- msub \res_nsec, \res_sec, \nsec_to_sec, \clock_nsec
- .endm
-
- /* sec and nsec are modified in place. */
- .macro add_ts sec, nsec, ts_sec, ts_nsec, nsec_to_sec
- /* Add timespec. */
- add \sec, \sec, \ts_sec
- add \nsec, \nsec, \ts_nsec
-
- /* Normalise the new timespec. */
- cmp \nsec, \nsec_to_sec
- b.lt 9999f
- sub \nsec, \nsec, \nsec_to_sec
- add \sec, \sec, #1
-9999:
- cmp \nsec, #0
- b.ge 9998f
- add \nsec, \nsec, \nsec_to_sec
- sub \sec, \sec, #1
-9998:
- .endm
-
- .macro clock_gettime_return, shift=0
- .if \shift == 1
- lsr x11, x11, x12
- .endif
- stp x10, x11, [x1, #TSPEC_TV_SEC]
- mov x0, xzr
- ret
- .endm
-
- .macro jump_slot jumptable, index, label
- .if (. - \jumptable) != 4 * (\index)
- .error "Jump slot index mismatch"
- .endif
- b \label
- .endm
-
- .text
-
-/* int __kernel_gettimeofday(struct timeval *tv, struct timezone *tz); */
-ENTRY(__kernel_gettimeofday)
- .cfi_startproc
- adr vdso_data, _vdso_data
- /* If tv is NULL, skip to the timezone code. */
- cbz x0, 2f
-
- /* Compute the time of day. */
-1: seqcnt_acquire
- syscall_check fail=4f
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=1b
-
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- /* Convert ns to us. */
- mov x13, #1000
- lsl x13, x13, x12
- udiv x11, x11, x13
- stp x10, x11, [x0, #TVAL_TV_SEC]
-2:
- /* If tz is NULL, return 0. */
- cbz x1, 3f
- ldp w4, w5, [vdso_data, #VDSO_TZ_MINWEST]
- stp w4, w5, [x1, #TZ_MINWEST]
-3:
- mov x0, xzr
- ret
-4:
- /* Syscall fallback. */
- mov x8, #__NR_gettimeofday
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_gettimeofday)
-
-#define JUMPSLOT_MAX CLOCK_MONOTONIC_COARSE
-
-/* int __kernel_clock_gettime(clockid_t clock_id, struct timespec *tp); */
-ENTRY(__kernel_clock_gettime)
- .cfi_startproc
- cmp w0, #JUMPSLOT_MAX
- b.hi syscall
- adr vdso_data, _vdso_data
- adr x_tmp, jumptable
- add x_tmp, x_tmp, w0, uxtw #2
- br x_tmp
-
- ALIGN
-jumptable:
- jump_slot jumptable, CLOCK_REALTIME, realtime
- jump_slot jumptable, CLOCK_MONOTONIC, monotonic
- b syscall
- b syscall
- jump_slot jumptable, CLOCK_MONOTONIC_RAW, monotonic_raw
- jump_slot jumptable, CLOCK_REALTIME_COARSE, realtime_coarse
- jump_slot jumptable, CLOCK_MONOTONIC_COARSE, monotonic_coarse
-
- .if (. - jumptable) != 4 * (JUMPSLOT_MAX + 1)
- .error "Wrong jumptable size"
- .endif
-
- ALIGN
-realtime:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- seqcnt_check fail=realtime
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_mono_mult, w12 = cs_shift */
- ldp w11, w12, [vdso_data, #VDSO_CS_MONO_MULT]
- ldp x13, x14, [vdso_data, #VDSO_XTIME_CLK_SEC]
- ldp x3, x4, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic
-
- /* All computations are done with left-shifted nsecs. */
- lsl x4, x4, x12
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_realtime res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, xtime_sec=x13, xtime_nsec=x14, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x3, ts_nsec=x4, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-monotonic_raw:
- seqcnt_acquire
- syscall_check fail=syscall
- ldr x10, [vdso_data, #VDSO_CS_CYCLE_LAST]
- /* w11 = cs_raw_mult, w12 = cs_shift */
- ldp w12, w11, [vdso_data, #VDSO_CS_SHIFT]
- ldp x13, x14, [vdso_data, #VDSO_RAW_TIME_SEC]
- seqcnt_check fail=monotonic_raw
-
- /* All computations are done with left-shifted nsecs. */
- get_nsec_per_sec res=x9
- lsl x9, x9, x12
-
- get_clock_shifted_nsec res=x15, cycle_last=x10, mult=x11
- get_ts_clock_raw res_sec=x10, res_nsec=x11, \
- clock_nsec=x15, nsec_to_sec=x9
-
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return, shift=1
-
- ALIGN
-realtime_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- seqcnt_check fail=realtime_coarse
- clock_gettime_return
-
- ALIGN
-monotonic_coarse:
- seqcnt_acquire
- ldp x10, x11, [vdso_data, #VDSO_XTIME_CRS_SEC]
- ldp x13, x14, [vdso_data, #VDSO_WTM_CLK_SEC]
- seqcnt_check fail=monotonic_coarse
-
- /* Computations are done in (non-shifted) nsecs. */
- get_nsec_per_sec res=x9
- add_ts sec=x10, nsec=x11, ts_sec=x13, ts_nsec=x14, nsec_to_sec=x9
- clock_gettime_return
-
- ALIGN
-syscall: /* Syscall fallback. */
- mov x8, #__NR_clock_gettime
- svc #0
- ret
- .cfi_endproc
-ENDPROC(__kernel_clock_gettime)
-
-/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
-ENTRY(__kernel_clock_getres)
- .cfi_startproc
- cmp w0, #CLOCK_REALTIME
- ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
- ccmp w0, #CLOCK_MONOTONIC_RAW, #0x4, ne
- b.ne 1f
-
- ldr x2, 5f
- b 2f
-1:
- cmp w0, #CLOCK_REALTIME_COARSE
- ccmp w0, #CLOCK_MONOTONIC_COARSE, #0x4, ne
- b.ne 4f
- ldr x2, 6f
-2:
- cbz x1, 3f
- stp xzr, x2, [x1]
-
-3: /* res == NULL. */
- mov w0, wzr
- ret
-
-4: /* Syscall fallback. */
- mov x8, #__NR_clock_getres
- svc #0
- ret
-5:
- .quad CLOCK_REALTIME_RES
-6:
- .quad CLOCK_COARSE_RES
- .cfi_endproc
-ENDPROC(__kernel_clock_getres)
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..160e68196339
--- /dev/null
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __kernel_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __kernel_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 05/23] arm64: compat: Alloc separate pages for vectors and sigpage
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
Address the problem creating separate pages for vectors and sigpage in
a similar fashion to what happens today on arm.
Change as well the meaning of mm->context.vdso for AArch32 compat since
it now points to sigpage and not to vectors anymore in order to make
simpler the implementation of the signal handling (the address of
sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 6adc1a90e7e6..355d120b78cb 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -214,10 +214,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index f1a7ab18faf3..5e8fb35fad33 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -78,9 +78,9 @@
#endif /* CONFIG_ARM64_FORCE_52BIT */
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index cb7800acd19f..3846a1b710b5 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 8af3625fd9c5..41161d5f99f7 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -54,61 +56,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 05/23] arm64: compat: Alloc separate pages for vectors and sigpage
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
Address the problem creating separate pages for vectors and sigpage in
a similar fashion to what happens today on arm.
Change as well the meaning of mm->context.vdso for AArch32 compat since
it now points to sigpage and not to vectors anymore in order to make
simpler the implementation of the signal handling (the address of
sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 6adc1a90e7e6..355d120b78cb 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -214,10 +214,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index f1a7ab18faf3..5e8fb35fad33 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -78,9 +78,9 @@
#endif /* CONFIG_ARM64_FORCE_52BIT */
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index cb7800acd19f..3846a1b710b5 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 8af3625fd9c5..41161d5f99f7 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -54,61 +56,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 05/23] arm64: compat: Alloc separate pages for vectors and sigpage
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
In the current implementation AArch32 installs a special page called
"[vectors]" that contains sigreturn trampolines and kuser helpers,
and this is done at fixed address specified by the kuser helpers ABI.
Having sigreturn trampolines and kuser helpers in the same page, makes
difficult to maintain compatibility with arm because it makes not
possible to disable kuser helpers.
Address the problem creating separate pages for vectors and sigpage in
a similar fashion to what happens today on arm.
Change as well the meaning of mm->context.vdso for AArch32 compat since
it now points to sigpage and not to vectors anymore in order to make
simpler the implementation of the signal handling (the address of
sigpage is randomized).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 6 +-
arch/arm64/include/asm/processor.h | 4 +-
arch/arm64/include/asm/signal32.h | 2 -
arch/arm64/kernel/signal32.c | 5 +-
arch/arm64/kernel/vdso.c | 112 ++++++++++++++++++++++-------
5 files changed, 93 insertions(+), 36 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 6adc1a90e7e6..355d120b78cb 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -214,10 +214,10 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
set_thread_flag(TIF_32BIT); \
})
#define COMPAT_ARCH_DLINFO
-extern int aarch32_setup_vectors_page(struct linux_binprm *bprm,
- int uses_interp);
+extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
+ int uses_interp);
#define compat_arch_setup_additional_pages \
- aarch32_setup_vectors_page
+ aarch32_setup_additional_pages
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index f1a7ab18faf3..5e8fb35fad33 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -78,9 +78,9 @@
#endif /* CONFIG_ARM64_FORCE_52BIT */
#ifdef CONFIG_COMPAT
-#define AARCH32_VECTORS_BASE 0xffff0000
+#define AARCH32_KUSER_BASE 0xffff0000
#define STACK_TOP (test_thread_flag(TIF_32BIT) ? \
- AARCH32_VECTORS_BASE : STACK_TOP_MAX)
+ AARCH32_KUSER_BASE : STACK_TOP_MAX)
#else
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 81abea0b7650..58e288aaf0ba 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,8 +20,6 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
-#define AARCH32_KERN_SIGRET_CODE_OFFSET 0x500
-
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index cb7800acd19f..3846a1b710b5 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -379,6 +379,7 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
+ void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -399,12 +400,12 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
} else {
/* Set up sigreturn pointer */
unsigned int idx = thumb << 1;
+ sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
- retcode = AARCH32_VECTORS_BASE +
- AARCH32_KERN_SIGRET_CODE_OFFSET +
+ retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
}
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 8af3625fd9c5..41161d5f99f7 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -1,5 +1,7 @@
/*
- * VDSO implementation for AArch64 and vector page setup for AArch32.
+ * VDSO implementation for AArch64 and for AArch32:
+ * AArch64: vDSO implementation contains pages setup and data page update.
+ * AArch32: vDSO implementation contains sigreturn and kuser pages setup.
*
* Copyright (C) 2012 ARM Limited
*
@@ -54,61 +56,117 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
/*
* Create and map the vectors page for AArch32 tasks.
*/
-static struct page *vectors_page[1] __ro_after_init;
+/*
+ * aarch32_vdso_pages:
+ * 0 - kuser helpers
+ * 1 - sigreturn code
+ */
+static struct page *aarch32_vdso_pages[2] __ro_after_init;
+static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+ {
+ /* Must be named [vectors] for compatibility with arm. */
+ .name = "[vectors]",
+ .pages = &aarch32_vdso_pages[0],
+ },
+ {
+ /* Must be named [sigpage] for compatibility with arm. */
+ .name = "[sigpage]",
+ .pages = &aarch32_vdso_pages[1],
+ },
+};
-static int __init alloc_vectors_page(void)
+static int __init aarch32_alloc_vdso_pages(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vpage;
+ unsigned long vdso_pages[2];
- vpage = get_zeroed_page(GFP_ATOMIC);
+ vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[0])
+ return -ENOMEM;
- if (!vpage)
+ vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_pages[1])
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)vpage + 0x1000 - kuser_sz, __kuser_helper_start,
- kuser_sz);
+ memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ __kuser_helper_start,
+ kuser_sz);
/* sigreturn code */
- memcpy((void *)vpage + AARCH32_KERN_SIGRET_CODE_OFFSET,
- __aarch32_sigret_code_start, sigret_sz);
+ memcpy((void *)vdso_pages[1],
+ __aarch32_sigret_code_start,
+ sigret_sz);
- flush_icache_range(vpage, vpage + PAGE_SIZE);
- vectors_page[0] = virt_to_page(vpage);
+ flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
+ flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
return 0;
}
-arch_initcall(alloc_vectors_page);
+arch_initcall(aarch32_alloc_vdso_pages);
-int aarch32_setup_vectors_page(struct linux_binprm *bprm, int uses_interp)
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
- struct mm_struct *mm = current->mm;
- unsigned long addr = AARCH32_VECTORS_BASE;
- static const struct vm_special_mapping spec = {
- .name = "[vectors]",
- .pages = vectors_page,
+ void *ret;
+
+ /* The kuser helpers must be mapped at the ABI-defined high address */
+ ret = _install_special_mapping(mm, AARCH32_KUSER_BASE, PAGE_SIZE,
+ VM_READ | VM_EXEC |
+ VM_MAYREAD | VM_MAYEXEC,
+ &aarch32_vdso_spec[0]);
+
+ return PTR_ERR_OR_ZERO(ret);
+}
- };
+static int aarch32_sigreturn_setup(struct mm_struct *mm)
+{
+ unsigned long addr;
void *ret;
- if (down_write_killable(&mm->mmap_sem))
- return -EINTR;
- current->mm->context.vdso = (void *)addr;
+ addr = get_unmapped_area(NULL, 0, PAGE_SIZE, 0, 0);
+ if (IS_ERR_VALUE(addr)) {
+ ret = ERR_PTR(addr);
+ goto out;
+ }
- /* Map vectors page at the high address. */
ret = _install_special_mapping(mm, addr, PAGE_SIZE,
- VM_READ|VM_EXEC|VM_MAYREAD|VM_MAYEXEC,
- &spec);
+ VM_READ | VM_EXEC | VM_MAYREAD |
+ VM_MAYWRITE | VM_MAYEXEC,
+ &aarch32_vdso_spec[1]);
+ if (IS_ERR(ret))
+ goto out;
- up_write(&mm->mmap_sem);
+ mm->context.vdso = (void *)addr;
+out:
return PTR_ERR_OR_ZERO(ret);
}
+
+int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
+{
+ struct mm_struct *mm = current->mm;
+ int ret;
+
+ if (down_write_killable(&mm->mmap_sem))
+ return -EINTR;
+
+ ret = aarch32_kuser_helpers_setup(mm);
+ if (ret)
+ goto out;
+
+ ret = aarch32_sigreturn_setup(mm);
+
+out:
+ up_write(&mm->mmap_sem);
+ return ret;
+}
#endif /* CONFIG_COMPAT */
static int vdso_mremap(const struct vm_special_mapping *sm,
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 06/23] arm64: compat: Split kuser32
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
Split the current version of kuser32 in kuser32 (for kuser helpers)
and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index cd434d0719c1..50f76b88a967 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 06/23] arm64: compat: Split kuser32
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
Split the current version of kuser32 in kuser32 (for kuser helpers)
and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index cd434d0719c1..50f76b88a967 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 06/23] arm64: compat: Split kuser32
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
To make it possible to disable kuser helpers in aarch32 we need to
divide the kuser and the sigreturn functionalities.
Split the current version of kuser32 in kuser32 (for kuser helpers)
and sigreturn32 (for sigreturn helpers).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/Makefile | 2 +-
arch/arm64/kernel/kuser32.S | 58 ++-------------------------------
arch/arm64/kernel/sigreturn32.S | 46 ++++++++++++++++++++++++++
3 files changed, 49 insertions(+), 57 deletions(-)
create mode 100644 arch/arm64/kernel/sigreturn32.S
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index cd434d0719c1..50f76b88a967 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,7 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
- sys_compat.o
+ sigreturn32.o sys_compat.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index 997e6b27ff6a..f19e2b015097 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -1,24 +1,9 @@
/*
- * Low-level user helpers placed in the vectors page for AArch32.
+ * AArch32 user helpers.
* Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
*
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
- * Copyright (C) 2012 ARM Ltd.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
- *
- *
- * AArch32 user helpers.
+ * Copyright (C) 2012-2018 ARM Ltd.
*
* Each segment is 32-byte aligned and will be moved to the top of the high
* vector page. New segments (if ever needed) must be added in front of
@@ -77,42 +62,3 @@ __kuser_helper_version: // 0xffff0ffc
.word ((__kuser_helper_end - __kuser_helper_start) >> 5)
.globl __kuser_helper_end
__kuser_helper_end:
-
-/*
- * AArch32 sigreturn code
- *
- * For ARM syscalls, the syscall number has to be loaded into r7.
- * We do not support an OABI userspace.
- *
- * For Thumb syscalls, we also pass the syscall number via r7. We therefore
- * need two 16-bit instructions.
- */
- .globl __aarch32_sigret_code_start
-__aarch32_sigret_code_start:
-
- /*
- * ARM Code
- */
- .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
- .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
-
- /*
- * ARM code
- */
- .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
-
- /*
- * Thumb code
- */
- .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
- .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
-
- .globl __aarch32_sigret_code_end
-__aarch32_sigret_code_end:
diff --git a/arch/arm64/kernel/sigreturn32.S b/arch/arm64/kernel/sigreturn32.S
new file mode 100644
index 000000000000..475d30d471ac
--- /dev/null
+++ b/arch/arm64/kernel/sigreturn32.S
@@ -0,0 +1,46 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * AArch32 sigreturn code.
+ * Based on the kuser helpers in arch/arm/kernel/entry-armv.S.
+ *
+ * Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
+ * Copyright (C) 2012-2018 ARM Ltd.
+ *
+ * For ARM syscalls, the syscall number has to be loaded into r7.
+ * We do not support an OABI userspace.
+ *
+ * For Thumb syscalls, we also pass the syscall number via r7. We therefore
+ * need two 16-bit instructions.
+ */
+
+#include <asm/unistd.h>
+
+ .globl __aarch32_sigret_code_start
+__aarch32_sigret_code_start:
+
+ /*
+ * ARM Code
+ */
+ .byte __NR_compat_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_sigreturn, 0x27 // svc #__NR_compat_sigreturn
+ .byte __NR_compat_sigreturn, 0xdf // mov r7, #__NR_compat_sigreturn
+
+ /*
+ * ARM code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x70, 0xa0, 0xe3 // mov r7, #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0x00, 0x00, 0xef // svc #__NR_compat_rt_sigreturn
+
+ /*
+ * Thumb code
+ */
+ .byte __NR_compat_rt_sigreturn, 0x27 // svc #__NR_compat_rt_sigreturn
+ .byte __NR_compat_rt_sigreturn, 0xdf // mov r7, #__NR_compat_rt_sigreturn
+
+ .globl __aarch32_sigret_code_end
+__aarch32_sigret_code_end:
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 07/23] arm64: compat: Refactor aarch32_alloc_vdso_pages()
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
Divide the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 41161d5f99f7..4c9476516e2f 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,40 +75,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 07/23] arm64: compat: Refactor aarch32_alloc_vdso_pages()
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
Divide the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 41161d5f99f7..4c9476516e2f 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,40 +75,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 07/23] arm64: compat: Refactor aarch32_alloc_vdso_pages()
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
aarch32_alloc_vdso_pages() needs to the refactored to make it
easier to disable kuser helpers.
Divide the function in aarch32_alloc_kuser_vdso_page() and
aarch32_alloc_sigreturn_vdso_page().
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 49 ++++++++++++++++++++++++++--------------
1 file changed, 32 insertions(+), 17 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 41161d5f99f7..4c9476516e2f 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,40 +75,55 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
-static int __init aarch32_alloc_vdso_pages(void)
+static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
- extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
-
int kuser_sz = __kuser_helper_end - __kuser_helper_start;
- int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
- unsigned long vdso_pages[2];
-
- vdso_pages[0] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[0])
- return -ENOMEM;
+ unsigned long vdso_page;
- vdso_pages[1] = get_zeroed_page(GFP_ATOMIC);
- if (!vdso_pages[1])
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
return -ENOMEM;
/* kuser helpers */
- memcpy((void *)(vdso_pages[0] + 0x1000 - kuser_sz),
+ memcpy((void *)(vdso_page + 0x1000 - kuser_sz),
__kuser_helper_start,
kuser_sz);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
+
+ aarch32_vdso_pages[0] = virt_to_page(vdso_page);
+
+ return 0;
+}
+
+static int aarch32_alloc_sigreturn_vdso_page(void)
+{
+ extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
+ int sigret_sz = __aarch32_sigret_code_end - __aarch32_sigret_code_start;
+ unsigned long vdso_page;
+
+ vdso_page = get_zeroed_page(GFP_ATOMIC);
+ if (!vdso_page)
+ return -ENOMEM;
+
/* sigreturn code */
- memcpy((void *)vdso_pages[1],
+ memcpy((void *)vdso_page,
__aarch32_sigret_code_start,
sigret_sz);
- flush_icache_range(vdso_pages[0], vdso_pages[0] + PAGE_SIZE);
- flush_icache_range(vdso_pages[1], vdso_pages[1] + PAGE_SIZE);
+ flush_icache_range(vdso_page, vdso_page + PAGE_SIZE);
- aarch32_vdso_pages[0] = virt_to_page(vdso_pages[0]);
- aarch32_vdso_pages[1] = virt_to_page(vdso_pages[1]);
+ aarch32_vdso_pages[1] = virt_to_page(vdso_page);
return 0;
+
+}
+
+static int __init aarch32_alloc_vdso_pages(void)
+{
+ return aarch32_alloc_kuser_vdso_page() &
+ aarch32_alloc_sigreturn_vdso_page();
}
arch_initcall(aarch32_alloc_vdso_pages);
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
Add a KUSER_HELPERS config option that can be used to disable the
mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index d898da2e20f5..ed3290494f1c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1465,6 +1465,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 50f76b88a967..c7bd0794855a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 4c9476516e2f..523e56658b84 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,6 +75,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -96,6 +97,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -127,6 +134,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -139,6 +147,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
Add a KUSER_HELPERS config option that can be used to disable the
mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index d898da2e20f5..ed3290494f1c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1465,6 +1465,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 50f76b88a967..c7bd0794855a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 4c9476516e2f..523e56658b84 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,6 +75,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -96,6 +97,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -127,6 +134,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -139,6 +147,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
When kuser helpers are enabled the kernel maps the relative code at
a fixed address (0xffff0000). Making configurable the option to disable
them means that the kernel can remove this mapping and any access to
this memory area results in a sigfault.
Add a KUSER_HELPERS config option that can be used to disable the
mapping when it is turned off.
This option can be turned off if and only if the applications are
designed specifically for the platform and they do not make use of the
kuser helpers code.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 21 +++++++++++++++++++++
arch/arm64/kernel/Makefile | 3 ++-
arch/arm64/kernel/kuser32.S | 7 +++----
arch/arm64/kernel/vdso.c | 15 +++++++++++++++
4 files changed, 41 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index d898da2e20f5..ed3290494f1c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1465,6 +1465,27 @@ config COMPAT
If you want to execute 32-bit userspace applications, say Y.
+config KUSER_HELPERS
+ bool "Enable kuser helpers page for compatibility with 32 bit applications."
+ depends on COMPAT
+ default y
+ help
+ Enables kuser helpers to be mapped in a special purpose page at a fixed
+ address to maintain independence from the type of CPU present in the SoC.
+ This feature is provided for compatibility reasons in fact allows 32 bit
+ applications compliant with ARMv4 up to ARMv8 to run without any
+ modification.
+
+ Warning: Being always mapped at a fixed address makes it easier to create
+ exploits based on ROP type of attacks.
+
+ As a consequence of this, this feature is made configurable but be aware that
+ it can be turned off if and only if the binaries and the libraries running on
+ a specific platform are designed to do not make use of these helpers, otherwise
+ should be left on.
+
+ See Documentation/arm/kernel_user_helpers.txt for details.
+
config SYSVIPC_COMPAT
def_bool y
depends on COMPAT && SYSVIPC
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index 50f76b88a967..c7bd0794855a 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -27,8 +27,9 @@ OBJCOPYFLAGS := --prefix-symbols=__efistub_
$(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
+obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
sigreturn32.o sys_compat.o
+obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_ARM64_MODULE_PLTS) += module-plts.o
diff --git a/arch/arm64/kernel/kuser32.S b/arch/arm64/kernel/kuser32.S
index f19e2b015097..7d38633bf33f 100644
--- a/arch/arm64/kernel/kuser32.S
+++ b/arch/arm64/kernel/kuser32.S
@@ -5,10 +5,9 @@
* Copyright (C) 2005-2011 Nicolas Pitre <nico@fluxnic.net>
* Copyright (C) 2012-2018 ARM Ltd.
*
- * Each segment is 32-byte aligned and will be moved to the top of the high
- * vector page. New segments (if ever needed) must be added in front of
- * existing ones. This mechanism should be used only for things that are
- * really small and justified, and not be abused freely.
+ * The kuser helpers below are mapped at a fixed address by
+ * aarch32_setup_additional_pages() ad are provided for compatibility
+ * reasons with 32 bit (aarch32) applications that need them.
*
* See Documentation/arm/kernel_user_helpers.txt for formal definitions.
*/
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 4c9476516e2f..523e56658b84 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -75,6 +75,7 @@ static const struct vm_special_mapping aarch32_vdso_spec[2] = {
},
};
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_alloc_kuser_vdso_page(void)
{
extern char __kuser_helper_start[], __kuser_helper_end[];
@@ -96,6 +97,12 @@ static int aarch32_alloc_kuser_vdso_page(void)
return 0;
}
+#else
+static int aarch32_alloc_kuser_vdso_page(void)
+{
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPER */
static int aarch32_alloc_sigreturn_vdso_page(void)
{
@@ -127,6 +134,7 @@ static int __init aarch32_alloc_vdso_pages(void)
}
arch_initcall(aarch32_alloc_vdso_pages);
+#ifdef CONFIG_KUSER_HELPERS
static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
{
void *ret;
@@ -139,6 +147,13 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
return PTR_ERR_OR_ZERO(ret);
}
+#else
+static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
+{
+ /* kuser helpers not enabled */
+ return 0;
+}
+#endif /* CONFIG_KUSER_HELPERS */
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 09/23] arm64: compat: Add missing syscall numbers
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
Add the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index a7b1fc58ffdf..2a65fe82cec6 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 09/23] arm64: compat: Add missing syscall numbers
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
Add the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index a7b1fc58ffdf..2a65fe82cec6 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 09/23] arm64: compat: Add missing syscall numbers
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
vDSO requires gettimeofday and clock_gettime syscalls to implement the
fallback mechanism.
Add the missing syscall numbers to unistd.h for arm64.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Arnd Bergmann <arnd@arndb.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/unistd.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/arch/arm64/include/asm/unistd.h b/arch/arm64/include/asm/unistd.h
index a7b1fc58ffdf..2a65fe82cec6 100644
--- a/arch/arm64/include/asm/unistd.h
+++ b/arch/arm64/include/asm/unistd.h
@@ -33,8 +33,11 @@
#define __NR_compat_exit 1
#define __NR_compat_read 3
#define __NR_compat_write 4
+#define __NR_compat_gettimeofday 78
#define __NR_compat_sigreturn 119
#define __NR_compat_rt_sigreturn 173
+#define __NR_compat_clock_getres 247
+#define __NR_compat_clock_gettime 263
/*
* The following SVCs are ARM private.
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 10/23] arm64: compat: Expose signal related structures
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
Expose the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 3846a1b710b5..d916fd7f6409 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 10/23] arm64: compat: Expose signal related structures
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
Expose the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 3846a1b710b5..d916fd7f6409 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 10/23] arm64: compat: Expose signal related structures
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The compat signal data structures are required as part of the compat
vDSO implementation in order to provide the unwinding information for
the sigreturn trampolines.
Expose the mentioned data structures as part of signal32.h.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 46 +++++++++++++++++++++++++++++++
arch/arm64/kernel/signal32.c | 46 -------------------------------
2 files changed, 46 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 58e288aaf0ba..1f05268f4c6d 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -20,6 +20,52 @@
#ifdef CONFIG_COMPAT
#include <linux/compat.h>
+struct compat_sigcontext {
+ /* We always set these two fields to 0 */
+ compat_ulong_t trap_no;
+ compat_ulong_t error_code;
+
+ compat_ulong_t oldmask;
+ compat_ulong_t arm_r0;
+ compat_ulong_t arm_r1;
+ compat_ulong_t arm_r2;
+ compat_ulong_t arm_r3;
+ compat_ulong_t arm_r4;
+ compat_ulong_t arm_r5;
+ compat_ulong_t arm_r6;
+ compat_ulong_t arm_r7;
+ compat_ulong_t arm_r8;
+ compat_ulong_t arm_r9;
+ compat_ulong_t arm_r10;
+ compat_ulong_t arm_fp;
+ compat_ulong_t arm_ip;
+ compat_ulong_t arm_sp;
+ compat_ulong_t arm_lr;
+ compat_ulong_t arm_pc;
+ compat_ulong_t arm_cpsr;
+ compat_ulong_t fault_address;
+};
+
+struct compat_ucontext {
+ compat_ulong_t uc_flags;
+ compat_uptr_t uc_link;
+ compat_stack_t uc_stack;
+ struct compat_sigcontext uc_mcontext;
+ compat_sigset_t uc_sigmask;
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
+};
+
+struct compat_sigframe {
+ struct compat_ucontext uc;
+ compat_ulong_t retcode[2];
+};
+
+struct compat_rt_sigframe {
+ struct compat_siginfo info;
+ struct compat_sigframe sig;
+};
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 3846a1b710b5..d916fd7f6409 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -30,42 +30,6 @@
#include <linux/uaccess.h>
#include <asm/unistd.h>
-struct compat_sigcontext {
- /* We always set these two fields to 0 */
- compat_ulong_t trap_no;
- compat_ulong_t error_code;
-
- compat_ulong_t oldmask;
- compat_ulong_t arm_r0;
- compat_ulong_t arm_r1;
- compat_ulong_t arm_r2;
- compat_ulong_t arm_r3;
- compat_ulong_t arm_r4;
- compat_ulong_t arm_r5;
- compat_ulong_t arm_r6;
- compat_ulong_t arm_r7;
- compat_ulong_t arm_r8;
- compat_ulong_t arm_r9;
- compat_ulong_t arm_r10;
- compat_ulong_t arm_fp;
- compat_ulong_t arm_ip;
- compat_ulong_t arm_sp;
- compat_ulong_t arm_lr;
- compat_ulong_t arm_pc;
- compat_ulong_t arm_cpsr;
- compat_ulong_t fault_address;
-};
-
-struct compat_ucontext {
- compat_ulong_t uc_flags;
- compat_uptr_t uc_link;
- compat_stack_t uc_stack;
- struct compat_sigcontext uc_mcontext;
- compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof (compat_sigset_t) / sizeof (int))];
- compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
-};
-
struct compat_vfp_sigframe {
compat_ulong_t magic;
compat_ulong_t size;
@@ -92,16 +56,6 @@ struct compat_aux_sigframe {
unsigned long end_magic;
} __attribute__((__aligned__(8)));
-struct compat_sigframe {
- struct compat_ucontext uc;
- compat_ulong_t retcode[2];
-};
-
-struct compat_rt_sigframe {
- struct compat_siginfo info;
- struct compat_sigframe sig;
-};
-
#define _BLOCKABLE (~(sigmask(SIGKILL) | sigmask(SIGSTOP)))
static inline int put_sigset_t(compat_sigset_t __user *uset, sigset_t *set)
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Update asm-offsets for arm64 to generate the correct offsets for
compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 4adb52190a90..afd28ccda174 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -84,6 +85,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Update asm-offsets for arm64 to generate the correct offsets for
compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 4adb52190a90..afd28ccda174 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -84,6 +85,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Update asm-offsets for arm64 to generate the correct offsets for
compat signals.
They will be useful for the implementation of the compat sigreturn
trampolines in vDSO context.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/signal32.h | 14 +++++++++++++-
arch/arm64/kernel/asm-offsets.c | 6 ++++++
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
index 1f05268f4c6d..a6fd7be1b22a 100644
--- a/arch/arm64/include/asm/signal32.h
+++ b/arch/arm64/include/asm/signal32.h
@@ -52,7 +52,7 @@ struct compat_ucontext {
compat_stack_t uc_stack;
struct compat_sigcontext uc_mcontext;
compat_sigset_t uc_sigmask;
- int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
+ int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
compat_ulong_t uc_regspace[128] __attribute__((__aligned__(8)));
};
@@ -66,6 +66,18 @@ struct compat_rt_sigframe {
struct compat_sigframe sig;
};
+/* Macros for asm-offsets.c */
+#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
+#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
+ offsetof(struct compat_rt_sigframe, sig) + \
+ offsetof(struct compat_sigframe, uc) + \
+ offsetof(struct compat_ucontext, uc_mcontext) + \
+ offsetof(struct compat_sigcontext, arm_r0))
+
int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct pt_regs *regs);
int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
index 4adb52190a90..afd28ccda174 100644
--- a/arch/arm64/kernel/asm-offsets.c
+++ b/arch/arm64/kernel/asm-offsets.c
@@ -30,6 +30,7 @@
#include <asm/fixmap.h>
#include <asm/thread_info.h>
#include <asm/memory.h>
+#include <asm/signal32.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
#include <linux/kbuild.h>
@@ -84,6 +85,11 @@ int main(void)
DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
BLANK();
+#ifdef CONFIG_COMPAT
+ DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
+ DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
+ BLANK();
+#endif
DEFINE(MM_CONTEXT_ID, offsetof(struct mm_struct, context.id.counter));
BLANK();
DEFINE(VMA_VM_MM, offsetof(struct vm_area_struct, vm_mm));
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 12/23] lib: vdso: Add compat support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
Add support to the generic code for compat fallback functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 39f92f7d3218..466d0c68fa80 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -20,7 +20,11 @@
* - clock_gettime_fallback(): fallback for clock_gettime.
* - clock_getres_fallback(): fallback for clock_getres.
*/
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
static notrace int do_hres(const struct vdso_data *vd,
clockid_t clk,
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 12/23] lib: vdso: Add compat support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
Add support to the generic code for compat fallback functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 39f92f7d3218..466d0c68fa80 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -20,7 +20,11 @@
* - clock_gettime_fallback(): fallback for clock_gettime.
* - clock_getres_fallback(): fallback for clock_getres.
*/
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
static notrace int do_hres(const struct vdso_data *vd,
clockid_t clk,
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 12/23] lib: vdso: Add compat support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Some 64 bit architectures have support for 32 bit applications that
require a separate version of the vDSOs.
Add support to the generic code for compat fallback functions.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
lib/vdso/gettimeofday.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 39f92f7d3218..466d0c68fa80 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -20,7 +20,11 @@
* - clock_gettime_fallback(): fallback for clock_gettime.
* - clock_getres_fallback(): fallback for clock_getres.
*/
+#ifdef ENABLE_COMPAT_VDSO
+#include <asm/vdso/compat_gettimeofday.h>
+#else
#include <asm/vdso/gettimeofday.h>
+#endif /* ENABLE_COMPAT_VDSO */
static notrace int do_hres(const struct vdso_data *vd,
clockid_t clk,
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 13/23] arm64: compat: Add vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Provide the arm64 compat (AArch32) vDSO in kernel/vdso32 in a similar
way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 119 +++++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 81 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++++
9 files changed, 559 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..888c70c24f71
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+/*
+ * We use the hidden visibility to prevent the compiler from generating a GOT
+ * relocation. Not only is going through a GOT useless (the entry couldn't and
+ * musn't be overridden by another library), it does not even work: the linker
+ * cannot generate an absolute address to the data page.
+ *
+ * With the hidden visibility, the compiler simply generates a PC-relative
+ * relocation (R_ARM_REL32), and this is what we need.
+ */
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ /* The checks below are required for ABI consistency with arm */
+ if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
+ return -EINVAL;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ /*
+ * This simply puts &_vdso_data into ret. The reason why we don't use
+ * `ret = &_vdso_data` is that the compiler tends to optimise this in a
+ * very suboptimal way: instead of keeping &_vdso_data in a register,
+ * it goes through a relocation almost every time _vdso_data must be
+ * accessed (even in subfunctions). This is both time and space
+ * consuming: each relocation uses a word in the code section, and it
+ * has to be loaded at runtime.
+ *
+ * This trick hides the assignment from the compiler. Since it cannot
+ * track where the pointer comes from, it will only use one relocation
+ * where __arch_get_vdso_data() is called, and then keep the result in
+ * a register.
+ */
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..4fd3093ea78c
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..b470a8ed35a7
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 13/23] arm64: compat: Add vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Provide the arm64 compat (AArch32) vDSO in kernel/vdso32 in a similar
way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 119 +++++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 81 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++++
9 files changed, 559 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..888c70c24f71
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+/*
+ * We use the hidden visibility to prevent the compiler from generating a GOT
+ * relocation. Not only is going through a GOT useless (the entry couldn't and
+ * musn't be overridden by another library), it does not even work: the linker
+ * cannot generate an absolute address to the data page.
+ *
+ * With the hidden visibility, the compiler simply generates a PC-relative
+ * relocation (R_ARM_REL32), and this is what we need.
+ */
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ /* The checks below are required for ABI consistency with arm */
+ if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
+ return -EINVAL;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ /*
+ * This simply puts &_vdso_data into ret. The reason why we don't use
+ * `ret = &_vdso_data` is that the compiler tends to optimise this in a
+ * very suboptimal way: instead of keeping &_vdso_data in a register,
+ * it goes through a relocation almost every time _vdso_data must be
+ * accessed (even in subfunctions). This is both time and space
+ * consuming: each relocation uses a word in the code section, and it
+ * has to be loaded at runtime.
+ *
+ * This trick hides the assignment from the compiler. Since it cannot
+ * track where the pointer comes from, it will only use one relocation
+ * where __arch_get_vdso_data() is called, and then keep the result in
+ * a register.
+ */
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..4fd3093ea78c
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..b470a8ed35a7
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 13/23] arm64: compat: Add vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Provide the arm64 compat (AArch32) vDSO in kernel/vdso32 in a similar
way to what happens in kernel/vdso.
The compat vDSO leverages on an adaptation of the arm architecture code
with few changes:
- Use of lib/vdso for gettimeofday
- Implementation of syscall based fallback
- Introduction of clock_getres for the compat library
- Implementation of trampolines
- Implementation of elf note
To build the compat vDSO a 32 bit compiler is required and needs to be
specified via CONFIG_CROSS_COMPILE_COMPAT_VDSO.
The implementation of the configuration option will be contained in a
future patch.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso/compat_barrier.h | 36 ++++
.../include/asm/vdso/compat_gettimeofday.h | 119 +++++++++++
arch/arm64/kernel/vdso32/.gitignore | 2 +
arch/arm64/kernel/vdso32/Makefile | 184 ++++++++++++++++++
arch/arm64/kernel/vdso32/note.c | 15 ++
arch/arm64/kernel/vdso32/sigreturn.S | 62 ++++++
arch/arm64/kernel/vdso32/vdso.S | 19 ++
arch/arm64/kernel/vdso32/vdso.lds.S | 81 ++++++++
arch/arm64/kernel/vdso32/vgettimeofday.c | 41 ++++
9 files changed, 559 insertions(+)
create mode 100644 arch/arm64/include/asm/vdso/compat_barrier.h
create mode 100644 arch/arm64/include/asm/vdso/compat_gettimeofday.h
create mode 100644 arch/arm64/kernel/vdso32/.gitignore
create mode 100644 arch/arm64/kernel/vdso32/Makefile
create mode 100644 arch/arm64/kernel/vdso32/note.c
create mode 100644 arch/arm64/kernel/vdso32/sigreturn.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.S
create mode 100644 arch/arm64/kernel/vdso32/vdso.lds.S
create mode 100644 arch/arm64/kernel/vdso32/vgettimeofday.c
diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
new file mode 100644
index 000000000000..b149761d9661
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_barrier.h
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __COMPAT_BARRIER_H
+#define __COMPAT_BARRIER_H
+
+#include <asm/barrier.h>
+
+#if __LINUX_ARM_ARCH__ >= 8
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() dmb(ishld)
+#define aarch32_smp_wmb() dmb(ishst)
+#else
+#define aarch32_smp_mb() dmb(ish)
+#define aarch32_smp_rmb() aarch32_smp_mb()
+#define aarch32_smp_wmb() dmb(ishst)
+#endif
+
+/*
+ * Warning: This code is meant to be used with
+ * ENABLE_COMPAT_VDSO only.
+ */
+#ifndef ENABLE_COMPAT_VDSO
+#error This header is meant to be used with ENABLE_COMPAT_VDSO only
+#endif
+
+#undef smp_mb
+#undef smp_rmb
+#undef smp_wmb
+
+#define smp_mb() aarch32_smp_mb()
+#define smp_rmb() aarch32_smp_rmb()
+#define smp_wmb() aarch32_smp_wmb()
+
+#endif /* __COMPAT_BARRIER_H */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
new file mode 100644
index 000000000000..888c70c24f71
--- /dev/null
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+#include <asm/vdso/compat_barrier.h>
+
+/*
+ * We use the hidden visibility to prevent the compiler from generating a GOT
+ * relocation. Not only is going through a GOT useless (the entry couldn't and
+ * musn't be overridden by another library), it does not even work: the linker
+ * cannot generate an absolute address to the data page.
+ *
+ * With the hidden visibility, the compiler simply generates a PC-relative
+ * relocation (R_ARM_REL32), and this is what we need.
+ */
+extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_compat_clock_getres;
+
+ /* The checks below are required for ABI consistency with arm */
+ if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
+ return -EINVAL;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ u64 res;
+
+ isb();
+ asm volatile("mrrc p15, 1, %Q0, %R0, c14" : "=r" (res));
+
+ return res;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ const struct vdso_data *ret;
+
+ /*
+ * This simply puts &_vdso_data into ret. The reason why we don't use
+ * `ret = &_vdso_data` is that the compiler tends to optimise this in a
+ * very suboptimal way: instead of keeping &_vdso_data in a register,
+ * it goes through a relocation almost every time _vdso_data must be
+ * accessed (even in subfunctions). This is both time and space
+ * consuming: each relocation uses a word in the code section, and it
+ * has to be loaded at runtime.
+ *
+ * This trick hides the assignment from the compiler. Since it cannot
+ * track where the pointer comes from, it will only use one relocation
+ * where __arch_get_vdso_data() is called, and then keep the result in
+ * a register.
+ */
+ asm volatile("mov %0, %1" : "=r"(ret) : "r"(&_vdso_data));
+
+ return ret;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
new file mode 100644
index 000000000000..4fea950fa5ed
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/.gitignore
@@ -0,0 +1,2 @@
+vdso.lds
+vdso.so.raw
diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
new file mode 100644
index 000000000000..ba57d2ae620f
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/Makefile
@@ -0,0 +1,184 @@
+# SPDX-License-Identifier: GPL-2.0
+#
+# Makefile for vdso32
+#
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
+CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
+
+# Same as cc-*option, but using CC_ARM32 instead of CC
+cc32-option = $(call try-run,\
+ $(CC_ARM32) $(1) -c -x c /dev/null -o "$$TMP",$(1),$(2))
+cc32-disable-warning = $(call try-run,\
+ $(CC_ARM32) -W$(strip $(1)) -c -x c /dev/null -o "$$TMP",-Wno-$(strip $(1)))
+cc32-ldoption = $(call try-run,\
+ $(CC_ARM32) $(1) -nostdlib -x c /dev/null -o "$$TMP",$(1),$(2))
+
+# We cannot use the global flags to compile the vDSO files, the main reason
+# being that the 32-bit compiler may be older than the main (64-bit) compiler
+# and therefore may not understand flags set using $(cc-option ...). Besides,
+# arch-specific options should be taken from the arm Makefile instead of the
+# arm64 one.
+# As a result we set our own flags here.
+
+# From top-level Makefile
+# NOSTDINC_FLAGS
+VDSO_CPPFLAGS := -nostdinc -isystem $(shell $(CC_ARM32) -print-file-name=include)
+VDSO_CPPFLAGS += $(LINUXINCLUDE)
+VDSO_CPPFLAGS += $(KBUILD_CPPFLAGS)
+
+# Common C and assembly flags
+# From top-level Makefile
+VDSO_CAFLAGS := $(VDSO_CPPFLAGS)
+VDSO_CAFLAGS += $(call cc32-option,-fno-PIE)
+ifdef CONFIG_DEBUG_INFO
+VDSO_CAFLAGS += -g
+endif
+ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC_ARM32)), y)
+VDSO_CAFLAGS += -DCC_HAVE_ASM_GOTO
+endif
+
+# From arm Makefile
+VDSO_CAFLAGS += $(call cc32-option,-fno-dwarf2-cfi-asm)
+VDSO_CAFLAGS += -mabi=aapcs-linux -mfloat-abi=soft
+ifeq ($(CONFIG_CPU_BIG_ENDIAN), y)
+VDSO_CAFLAGS += -mbig-endian
+else
+VDSO_CAFLAGS += -mlittle-endian
+endif
+
+# From arm vDSO Makefile
+VDSO_CAFLAGS += -fPIC -fno-builtin -fno-stack-protector
+VDSO_CAFLAGS += -DDISABLE_BRANCH_PROFILING
+
+# Try to compile for ARMv8. If the compiler is too old and doesn't support it,
+# fall back to v7. There is no easy way to check for what architecture the code
+# is being compiled, so define a macro specifying that (see arch/arm/Makefile).
+VDSO_CAFLAGS += $(call cc32-option,-march=armv8-a -D__LINUX_ARM_ARCH__=8,\
+ -march=armv7-a -D__LINUX_ARM_ARCH__=7)
+
+VDSO_CFLAGS := $(VDSO_CAFLAGS)
+VDSO_CFLAGS += -DENABLE_COMPAT_VDSO=1
+# KBUILD_CFLAGS from top-level Makefile
+VDSO_CFLAGS += -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs \
+ -fno-strict-aliasing -fno-common \
+ -Werror-implicit-function-declaration \
+ -Wno-format-security \
+ -std=gnu89
+VDSO_CFLAGS += -O2
+# Some useful compiler-dependent flags from top-level Makefile
+VDSO_CFLAGS += $(call cc32-option,-Wdeclaration-after-statement,)
+VDSO_CFLAGS += $(call cc32-option,-Wno-pointer-sign)
+VDSO_CFLAGS += $(call cc32-option,-fno-strict-overflow)
+VDSO_CFLAGS += $(call cc32-option,-Werror=strict-prototypes)
+VDSO_CFLAGS += $(call cc32-option,-Werror=date-time)
+VDSO_CFLAGS += $(call cc32-option,-Werror=incompatible-pointer-types)
+
+# The 32-bit compiler does not provide 128-bit integers, which are used in
+# some headers that are indirectly included from the vDSO code.
+# This hack makes the compiler happy and should trigger a warning/error if
+# variables of such type are referenced.
+VDSO_CFLAGS += -D__uint128_t='void*'
+# Silence some warnings coming from headers that operate on long's
+# (on GCC 4.8 or older, there is unfortunately no way to silence this warning)
+VDSO_CFLAGS += $(call cc32-disable-warning,shift-count-overflow)
+VDSO_CFLAGS += -Wno-int-to-pointer-cast
+
+VDSO_AFLAGS := $(VDSO_CAFLAGS)
+VDSO_AFLAGS += -D__ASSEMBLY__
+
+VDSO_LDFLAGS := $(VDSO_CPPFLAGS)
+# From arm vDSO Makefile
+VDSO_LDFLAGS += -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-soname=linux-vdso.so.1
+VDSO_LDFLAGS += -Wl,-z,max-page-size=4096 -Wl,-z,common-page-size=4096
+VDSO_LDFLAGS += -nostdlib -shared -mfloat-abi=soft
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--hash-style=sysv)
+VDSO_LDFLAGS += $(call cc32-ldoption,-Wl$(comma)--build-id)
+VDSO_LDFLAGS += $(call cc32-ldoption,-fuse-ld=bfd)
+
+
+# Borrow vdsomunge.c from the arm vDSO
+# We have to use a relative path because scripts/Makefile.host prefixes
+# $(hostprogs-y) with $(obj)
+munge := ../../../arm/vdso/vdsomunge
+hostprogs-y := $(munge)
+
+c-obj-vdso := note.o
+c-obj-vdso-gettimeofday := vgettimeofday.o
+asm-obj-vdso := sigreturn.o
+
+ifneq ($(c-gettimeofday-y),)
+VDSO_CFLAGS_gettimeofday_o += -include $(c-gettimeofday-y)
+endif
+
+# Build rules
+targets := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso) vdso.so vdso.so.dbg vdso.so.raw
+c-obj-vdso := $(addprefix $(obj)/, $(c-obj-vdso))
+c-obj-vdso-gettimeofday := $(addprefix $(obj)/, $(c-obj-vdso-gettimeofday))
+asm-obj-vdso := $(addprefix $(obj)/, $(asm-obj-vdso))
+obj-vdso := $(c-obj-vdso) $(c-obj-vdso-gettimeofday) $(asm-obj-vdso)
+
+obj-y += vdso.o
+extra-y += vdso.lds
+CPPFLAGS_vdso.lds += -P -C -U$(ARCH)
+
+# Force dependency (vdso.s includes vdso.so through incbin)
+$(obj)/vdso.o: $(obj)/vdso.so
+
+include/generated/vdso32-offsets.h: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,vdsosym)
+
+# Strip rule for vdso.so
+$(obj)/vdso.so: OBJCOPYFLAGS := -S
+$(obj)/vdso.so: $(obj)/vdso.so.dbg FORCE
+ $(call if_changed,objcopy)
+
+$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/$(munge) FORCE
+ $(call if_changed,vdsomunge)
+
+# Link rule for the .so file, .lds has to be first
+$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
+ $(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
+
+# Compilation rules for the vDSO sources
+$(c-obj-vdso): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc)
+$(c-obj-vdso-gettimeofday): %.o: %.c FORCE
+ $(call if_changed_dep,vdsocc_gettimeofday)
+$(asm-obj-vdso): %.o: %.S FORCE
+ $(call if_changed_dep,vdsoas)
+
+# Actual build commands
+quiet_cmd_vdsold = VDSOL $@
+ cmd_vdsold = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_LDFLAGS) \
+ -Wl,-T $(filter %.lds,$^) $(filter %.o,$^) -o $@
+quiet_cmd_vdsocc = VDSOC $@
+ cmd_vdsocc = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) -c -o $@ $<
+quiet_cmd_vdsocc_gettimeofday = VDSOC_GTD $@
+ cmd_vdsocc_gettimeofday = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_CFLAGS) $(VDSO_CFLAGS_gettimeofday_o) -c -o $@ $<
+quiet_cmd_vdsoas = VDSOA $@
+ cmd_vdsoas = $(CC_ARM32) -Wp,-MD,$(depfile) $(VDSO_AFLAGS) -c -o $@ $<
+
+quiet_cmd_vdsomunge = MUNGE $@
+ cmd_vdsomunge = $(obj)/$(munge) $< $@
+
+# Generate vDSO offsets using helper script (borrowed from the 64-bit vDSO)
+gen-vdsosym := $(srctree)/$(src)/../vdso/gen_vdso_offsets.sh
+quiet_cmd_vdsosym = VDSOSYM $@
+# The AArch64 nm should be able to read an AArch32 binary
+ cmd_vdsosym = $(NM) $< | $(gen-vdsosym) | LC_ALL=C sort > $@
+
+# Install commands for the unstripped file
+quiet_cmd_vdso_install = INSTALL $@
+ cmd_vdso_install = cp $(obj)/$@.dbg $(MODLIB)/vdso/vdso32.so
+
+vdso.so: $(obj)/vdso.so.dbg
+ @mkdir -p $(MODLIB)/vdso
+ $(call cmd,vdso_install)
+
+vdso_install: vdso.so
diff --git a/arch/arm64/kernel/vdso32/note.c b/arch/arm64/kernel/vdso32/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm64/kernel/vdso32/sigreturn.S b/arch/arm64/kernel/vdso32/sigreturn.S
new file mode 100644
index 000000000000..1a81277c2d09
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/sigreturn.S
@@ -0,0 +1,62 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * This file provides both A32 and T32 versions, in accordance with the
+ * arm sigreturn code.
+ *
+ * Copyright (C) 2018 ARM Limited
+ */
+
+#include <linux/linkage.h>
+#include <asm/asm-offsets.h>
+#include <asm/unistd.h>
+
+#define ARM_ENTRY(name) \
+ ENTRY(name)
+
+#define ARM_ENDPROC(name) \
+ .type name, %function; \
+ END(name)
+
+ .text
+
+ .arm
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_arm)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_arm)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_arm)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_arm)
+
+ .thumb
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_sigreturn_thumb)
+ mov r7, #__NR_compat_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_sigreturn_thumb)
+
+ .fnstart
+ .save {r0-r15}
+ .pad #COMPAT_RT_SIGFRAME_REGS_OFFSET
+ nop
+ARM_ENTRY(__kernel_rt_sigreturn_thumb)
+ mov r7, #__NR_compat_rt_sigreturn
+ svc #0
+ .fnend
+ARM_ENDPROC(__kernel_rt_sigreturn_thumb)
diff --git a/arch/arm64/kernel/vdso32/vdso.S b/arch/arm64/kernel/vdso32/vdso.S
new file mode 100644
index 000000000000..e72ac7bc4c04
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.S
@@ -0,0 +1,19 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2012 ARM Limited
+ */
+
+#include <linux/init.h>
+#include <linux/linkage.h>
+#include <linux/const.h>
+#include <asm/page.h>
+
+ .globl vdso32_start, vdso32_end
+ .section .rodata
+ .balign PAGE_SIZE
+vdso32_start:
+ .incbin "arch/arm64/kernel/vdso32/vdso.so"
+ .balign PAGE_SIZE
+vdso32_end:
+
+ .previous
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
new file mode 100644
index 000000000000..4fd3093ea78c
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -0,0 +1,81 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Adapted from arm64 version.
+ *
+ * GNU linker script for the VDSO library.
+ * Heavily based on the vDSO linker scripts for other archs.
+ *
+ * Copyright (C) 2012-2018 ARM Limited
+ */
+
+#include <linux/const.h>
+#include <asm/page.h>
+#include <asm/vdso.h>
+
+OUTPUT_FORMAT("elf32-littlearm", "elf32-bigarm", "elf32-littlearm")
+OUTPUT_ARCH(arm)
+
+SECTIONS
+{
+ PROVIDE_HIDDEN(_vdso_data = . - PAGE_SIZE);
+ . = VDSO_LBASE + SIZEOF_HEADERS;
+
+ .hash : { *(.hash) } :text
+ .gnu.hash : { *(.gnu.hash) }
+ .dynsym : { *(.dynsym) }
+ .dynstr : { *(.dynstr) }
+ .gnu.version : { *(.gnu.version) }
+ .gnu.version_d : { *(.gnu.version_d) }
+ .gnu.version_r : { *(.gnu.version_r) }
+
+ .note : { *(.note.*) } :text :note
+
+ .dynamic : { *(.dynamic) } :text :dynamic
+
+ .rodata : { *(.rodata*) } :text
+
+ .text : { *(.text*) } :text =0xe7f001f2
+
+ .got : { *(.got) }
+ .rel.plt : { *(.rel.plt) }
+
+ /DISCARD/ : {
+ *(.note.GNU-stack)
+ *(.data .data.* .gnu.linkonce.d.* .sdata*)
+ *(.bss .sbss .dynbss .dynsbss)
+ }
+}
+
+/*
+ * We must supply the ELF program headers explicitly to get just one
+ * PT_LOAD segment, and set the flags explicitly to make segments read-only.
+ */
+PHDRS
+{
+ text PT_LOAD FLAGS(5) FILEHDR PHDRS; /* PF_R|PF_X */
+ dynamic PT_DYNAMIC FLAGS(4); /* PF_R */
+ note PT_NOTE FLAGS(4); /* PF_R */
+}
+
+VERSION
+{
+ LINUX_2.6 {
+ global:
+ __vdso_clock_gettime;
+ __vdso_gettimeofday;
+ __vdso_clock_getres;
+ __kernel_sigreturn_arm;
+ __kernel_sigreturn_thumb;
+ __kernel_rt_sigreturn_arm;
+ __kernel_rt_sigreturn_thumb;
+ local: *;
+ };
+}
+
+/*
+ * Make the sigreturn code visible to the kernel.
+ */
+VDSO_compat_sigreturn_arm = __kernel_sigreturn_arm;
+VDSO_compat_sigreturn_thumb = __kernel_sigreturn_thumb;
+VDSO_compat_rt_sigreturn_arm = __kernel_rt_sigreturn_arm;
+VDSO_compat_rt_sigreturn_thumb = __kernel_rt_sigreturn_thumb;
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
new file mode 100644
index 000000000000..b470a8ed35a7
--- /dev/null
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -0,0 +1,41 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
+/* Avoid unresolved references emitted by GCC */
+
+void __aeabi_unwind_cpp_pr0(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr1(void)
+{
+}
+
+void __aeabi_unwind_cpp_pr2(void)
+{
+}
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 14/23] arm64: Refactor vDSO code
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
Refactor vdso.c to simplify the implementation of arm64 vDSO compat
(which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 523e56658b84..c217245768ea 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,7 +41,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -52,6 +75,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 14/23] arm64: Refactor vDSO code
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
Refactor vdso.c to simplify the implementation of arm64 vDSO compat
(which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 523e56658b84..c217245768ea 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,7 +41,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -52,6 +75,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 14/23] arm64: Refactor vDSO code
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Most of the code for initializing the vDSOs in arm64 and compat will
be in common, hence a refactor of the current code is required to avoid
duplication and simplify maintainability.
Refactor vdso.c to simplify the implementation of arm64 vDSO compat
(which will be pushed with a future patch).
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
1 file changed, 135 insertions(+), 73 deletions(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index 523e56658b84..c217245768ea 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,7 +41,30 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
-static unsigned long vdso_pages __ro_after_init;
+
+/* vdso_lookup arch_index */
+enum arch_vdso_type {
+ ARM64_VDSO = 0,
+};
+
+struct __vdso_lookup_t {
+ const char *name;
+ const char *vdso_code_start;
+ const char *vdso_code_end;
+ unsigned long vdso_pages;
+ /* Data Mapping */
+ struct vm_special_mapping *dm;
+ /* Code Mapping */
+ struct vm_special_mapping *cm;
+};
+
+static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
+ {
+ .name = "vdso",
+ .vdso_code_start = vdso_start,
+ .vdso_code_end = vdso_end,
+ },
+};
/*
* The vDSO data page.
@@ -52,6 +75,106 @@ static union {
} vdso_data_store __page_aligned_data;
struct vdso_data *vdso_data = &vdso_data_store.data;
+static int __vdso_remap(enum arch_vdso_type arch_index,
+ const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
+ unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start;
+
+ if (vdso_size != new_size)
+ return -EINVAL;
+
+ current->mm->context.vdso = (void *)new_vma->vm_start;
+
+ return 0;
+}
+
+static int __vdso_init(enum arch_vdso_type arch_index)
+{
+ int i;
+ struct page **vdso_pagelist;
+ unsigned long pfn;
+
+ if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
+ pr_err("vDSO is not a valid ELF object!\n");
+ return -EINVAL;
+ }
+
+ vdso_lookup[arch_index].vdso_pages = (
+ vdso_lookup[arch_index].vdso_code_end -
+ vdso_lookup[arch_index].vdso_code_start) >>
+ PAGE_SHIFT;
+ pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
+ vdso_lookup[arch_index].name,
+ vdso_lookup[arch_index].vdso_pages + 1,
+ vdso_lookup[arch_index].vdso_pages,
+ vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
+
+ /* Allocate the vDSO pagelist, plus a page for the data. */
+ vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
+ sizeof(struct page *),
+ GFP_KERNEL);
+ if (vdso_pagelist == NULL)
+ return -ENOMEM;
+
+ /* Grab the vDSO data page. */
+ vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
+
+
+ /* Grab the vDSO code pages. */
+ pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
+
+ for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
+ vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
+
+ vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
+ vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
+
+ return 0;
+}
+
+static int __setup_additional_pages(enum arch_vdso_type arch_index,
+ struct mm_struct *mm,
+ struct linux_binprm *bprm,
+ int uses_interp)
+{
+ unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
+ void *ret;
+
+ vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
+ /* Be sure to map the data page */
+ vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+
+ vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
+ if (IS_ERR_VALUE(vdso_base)) {
+ ret = ERR_PTR(vdso_base);
+ goto up_fail;
+ }
+
+ ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
+ VM_READ|VM_MAYREAD,
+ vdso_lookup[arch_index].dm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ vdso_base += PAGE_SIZE;
+ mm->context.vdso = (void *)vdso_base;
+ ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
+ VM_READ|VM_EXEC|
+ VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
+ vdso_lookup[arch_index].cm);
+ if (IS_ERR(ret))
+ goto up_fail;
+
+ return 0;
+
+up_fail:
+ mm->context.vdso = NULL;
+ return PTR_ERR(ret);
+}
+
#ifdef CONFIG_COMPAT
/*
* Create and map the vectors page for AArch32 tasks.
@@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
* 1 - sigreturn code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static const struct vm_special_mapping aarch32_vdso_spec[2] = {
+static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
@@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
static int vdso_mremap(const struct vm_special_mapping *sm,
struct vm_area_struct *new_vma)
{
- unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
- unsigned long vdso_size = vdso_end - vdso_start;
-
- if (vdso_size != new_size)
- return -EINVAL;
-
- current->mm->context.vdso = (void *)new_vma->vm_start;
-
- return 0;
+ return __vdso_remap(ARM64_VDSO, sm, new_vma);
}
static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
@@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
static int __init vdso_init(void)
{
- int i;
- struct page **vdso_pagelist;
- unsigned long pfn;
+ vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
+ vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
- if (memcmp(vdso_start, "\177ELF", 4)) {
- pr_err("vDSO is not a valid ELF object!\n");
- return -EINVAL;
- }
-
- vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
- pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
- vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
-
- /* Allocate the vDSO pagelist, plus a page for the data. */
- vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
- GFP_KERNEL);
- if (vdso_pagelist == NULL)
- return -ENOMEM;
-
- /* Grab the vDSO data page. */
- vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
-
-
- /* Grab the vDSO code pages. */
- pfn = sym_to_pfn(vdso_start);
-
- for (i = 0; i < vdso_pages; i++)
- vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
-
- vdso_spec[0].pages = &vdso_pagelist[0];
- vdso_spec[1].pages = &vdso_pagelist[1];
-
- return 0;
+ return __vdso_init(ARM64_VDSO);
}
arch_initcall(vdso_init);
@@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp)
{
struct mm_struct *mm = current->mm;
- unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
- void *ret;
-
- vdso_text_len = vdso_pages << PAGE_SHIFT;
- /* Be sure to map the data page */
- vdso_mapping_len = vdso_text_len + PAGE_SIZE;
+ int ret;
if (down_write_killable(&mm->mmap_sem))
return -EINTR;
- vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
- if (IS_ERR_VALUE(vdso_base)) {
- ret = ERR_PTR(vdso_base);
- goto up_fail;
- }
- ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
- VM_READ|VM_MAYREAD,
- &vdso_spec[0]);
- if (IS_ERR(ret))
- goto up_fail;
-
- vdso_base += PAGE_SIZE;
- mm->context.vdso = (void *)vdso_base;
- ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
- VM_READ|VM_EXEC|
- VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
- &vdso_spec[1]);
- if (IS_ERR(ret))
- goto up_fail;
+ ret = __setup_additional_pages(ARM64_VDSO,
+ mm,
+ bprm,
+ uses_interp);
up_write(&mm->mmap_sem);
- return 0;
-up_fail:
- mm->context.vdso = NULL;
- up_write(&mm->mmap_sem);
- return PTR_ERR(ret);
+ return ret;
}
#define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 15/23] arm64: compat: vDSO setup for compat layer
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
Add the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index c217245768ea..55343f8f3593 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,10 +41,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -64,6 +70,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -179,23 +192,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -227,6 +262,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -249,11 +293,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -278,6 +327,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -301,6 +351,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -314,7 +365,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 15/23] arm64: compat: vDSO setup for compat layer
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
Add the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index c217245768ea..55343f8f3593 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,10 +41,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -64,6 +70,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -179,23 +192,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -227,6 +262,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -249,11 +293,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -278,6 +327,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -301,6 +351,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -314,7 +365,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 15/23] arm64: compat: vDSO setup for compat layer
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
If CONFIG_GENERIC_COMPAT_VDSO is enabled, compat vDSO are installed in a
compat (32 bit) process instead of sigpage.
Add the necessary code to setup the vDSO required pages.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/kernel/vdso.c | 60 +++++++++++++++++++++++++++++++++++++++-
1 file changed, 59 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
index c217245768ea..55343f8f3593 100644
--- a/arch/arm64/kernel/vdso.c
+++ b/arch/arm64/kernel/vdso.c
@@ -41,10 +41,16 @@
#include <asm/vdso.h>
extern char vdso_start[], vdso_end[];
+#ifdef CONFIG_COMPAT_VDSO
+extern char vdso32_start[], vdso32_end[];
+#endif /* CONFIG_COMPAT_VDSO */
/* vdso_lookup arch_index */
enum arch_vdso_type {
ARM64_VDSO = 0,
+#ifdef CONFIG_COMPAT_VDSO
+ ARM64_VDSO32 = 1,
+#endif /* CONFIG_COMPAT_VDSO */
};
struct __vdso_lookup_t {
@@ -64,6 +70,13 @@ static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
.vdso_code_start = vdso_start,
.vdso_code_end = vdso_end,
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "vdso32",
+ .vdso_code_start = vdso32_start,
+ .vdso_code_end = vdso32_end,
+ },
+#endif /* CONFIG_COMPAT_VDSO */
};
/*
@@ -179,23 +192,45 @@ static int __setup_additional_pages(enum arch_vdso_type arch_index,
/*
* Create and map the vectors page for AArch32 tasks.
*/
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_mremap(const struct vm_special_mapping *sm,
+ struct vm_area_struct *new_vma)
+{
+ return __vdso_remap(ARM64_VDSO32, sm, new_vma);
+}
+#endif /* CONFIG_COMPAT_VDSO */
+
/*
* aarch32_vdso_pages:
* 0 - kuser helpers
* 1 - sigreturn code
+ * or (CONFIG_COMPAT_VDSO):
+ * 0 - kuser helpers
+ * 1 - vdso data
+ * 2 - vdso code
*/
static struct page *aarch32_vdso_pages[2] __ro_after_init;
-static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
+static struct vm_special_mapping aarch32_vdso_spec[3] __ro_after_init = {
{
/* Must be named [vectors] for compatibility with arm. */
.name = "[vectors]",
.pages = &aarch32_vdso_pages[0],
},
+#ifdef CONFIG_COMPAT_VDSO
+ {
+ .name = "[vvar]",
+ },
+ {
+ .name = "[vdso]",
+ .mremap = aarch32_vdso_mremap,
+ },
+#else
{
/* Must be named [sigpage] for compatibility with arm. */
.name = "[sigpage]",
.pages = &aarch32_vdso_pages[1],
},
+#endif /* CONFIG_COMPAT_VDSO */
};
#ifdef CONFIG_KUSER_HELPERS
@@ -227,6 +262,15 @@ static int aarch32_alloc_kuser_vdso_page(void)
}
#endif /* CONFIG_KUSER_HELPER */
+#ifdef CONFIG_COMPAT_VDSO
+static int aarch32_vdso_init(void)
+{
+ vdso_lookup[ARM64_VDSO32].dm = &aarch32_vdso_spec[1];
+ vdso_lookup[ARM64_VDSO32].cm = &aarch32_vdso_spec[2];
+
+ return __vdso_init(ARM64_VDSO32);
+}
+#else
static int aarch32_alloc_sigreturn_vdso_page(void)
{
extern char __aarch32_sigret_code_start[], __aarch32_sigret_code_end[];
@@ -249,11 +293,16 @@ static int aarch32_alloc_sigreturn_vdso_page(void)
return 0;
}
+#endif /* CONFIG_COMPAT_VDSO */
static int __init aarch32_alloc_vdso_pages(void)
{
return aarch32_alloc_kuser_vdso_page() &
+#ifdef CONFIG_COMPAT_VDSO
+ aarch32_vdso_init();
+#else
aarch32_alloc_sigreturn_vdso_page();
+#endif /* CONFIG_COMPAT_VDSO */
}
arch_initcall(aarch32_alloc_vdso_pages);
@@ -278,6 +327,7 @@ static int aarch32_kuser_helpers_setup(struct mm_struct *mm)
}
#endif /* CONFIG_KUSER_HELPERS */
+#ifndef CONFIG_COMPAT_VDSO
static int aarch32_sigreturn_setup(struct mm_struct *mm)
{
unsigned long addr;
@@ -301,6 +351,7 @@ static int aarch32_sigreturn_setup(struct mm_struct *mm)
out:
return PTR_ERR_OR_ZERO(ret);
}
+#endif /* !CONFIG_COMPAT_VDSO */
int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
{
@@ -314,7 +365,14 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
if (ret)
goto out;
+#ifdef CONFIG_COMPAT_VDSO
+ ret = __setup_additional_pages(ARM64_VDSO32,
+ mm,
+ bprm,
+ uses_interp);
+#else
ret = aarch32_sigreturn_setup(mm);
+#endif /* CONFIG_COMPAT_VDSO */
out:
up_write(&mm->mmap_sem);
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 16/23] arm64: elf: vDSO code page discovery
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
Add the necessary code to the elf arm64 module to make this possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 355d120b78cb..34cabaf78011 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -213,7 +213,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 16/23] arm64: elf: vDSO code page discovery
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
Add the necessary code to the elf arm64 module to make this possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 355d120b78cb..34cabaf78011 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -213,7 +213,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 16/23] arm64: elf: vDSO code page discovery
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Like in normal vDSOs, when compat vDSOs are enabled the auxiliary
vector symbol AT_SYSINFO_EHDR needs to point at the address of the
vDSO code, to allow the dynamic linker to find it.
Add the necessary code to the elf arm64 module to make this possible.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/elf.h | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 355d120b78cb..34cabaf78011 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -213,7 +213,21 @@ typedef compat_elf_greg_t compat_elf_gregset_t[COMPAT_ELF_NGREG];
({ \
set_thread_flag(TIF_32BIT); \
})
+#ifdef CONFIG_GENERIC_COMPAT_VDSO
+#define COMPAT_ARCH_DLINFO \
+do { \
+ /* \
+ * Note that we use Elf64_Off instead of elf_addr_t because \
+ * elf_addr_t in compat is defined as Elf32_Addr and casting \
+ * current->mm->context.vdso to it triggers a cast warning of \
+ * cast from pointer to integer of different size. \
+ */ \
+ NEW_AUX_ENT(AT_SYSINFO_EHDR, \
+ (Elf64_Off)current->mm->context.vdso); \
+} while (0)
+#else
#define COMPAT_ARCH_DLINFO
+#endif
extern int aarch32_setup_additional_pages(struct linux_binprm *bprm,
int uses_interp);
#define compat_arch_setup_additional_pages \
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 17/23] arm64: compat: Get sigreturn trampolines from vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
Add the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index d916fd7f6409..b8b53213cde6 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 17/23] arm64: compat: Get sigreturn trampolines from vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
Add the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index d916fd7f6409..b8b53213cde6 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 17/23] arm64: compat: Get sigreturn trampolines from vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
When the compat vDSO is enabled, the sigreturn trampolines are not
anymore available through [sigpage] but through [vdso].
Add the relevant code the enable the feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/include/asm/vdso.h | 3 +++
arch/arm64/kernel/signal32.c | 29 +++++++++++++++++++++++++++--
2 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/include/asm/vdso.h b/arch/arm64/include/asm/vdso.h
index 839ce0031bd5..9b197e5ea759 100644
--- a/arch/arm64/include/asm/vdso.h
+++ b/arch/arm64/include/asm/vdso.h
@@ -28,6 +28,9 @@
#ifndef __ASSEMBLY__
#include <generated/vdso-offsets.h>
+#ifdef CONFIG_COMPAT_VDSO
+#include <generated/vdso32-offsets.h>
+#endif
#define VDSO_SYMBOL(base, name) \
({ \
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index d916fd7f6409..b8b53213cde6 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -29,6 +29,7 @@
#include <asm/traps.h>
#include <linux/uaccess.h>
#include <asm/unistd.h>
+#include <asm/vdso.h>
struct compat_vfp_sigframe {
compat_ulong_t magic;
@@ -333,7 +334,6 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
compat_ulong_t retcode;
compat_ulong_t spsr = regs->pstate & ~(PSR_f | PSR_AA32_E_BIT);
int thumb;
- void *sigreturn_base;
/* Check if the handler is written for ARM or Thumb */
thumb = handler & 1;
@@ -353,14 +353,39 @@ static void compat_setup_return(struct pt_regs *regs, struct k_sigaction *ka,
retcode = ptr_to_compat(ka->sa.sa_restorer);
} else {
/* Set up sigreturn pointer */
+#ifdef CONFIG_COMPAT_VDSO
+ void *vdso_base = current->mm->context.vdso;
+ void *vdso_trampoline;
+
+ if (ka->sa.sa_flags & SA_SIGINFO) {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_rt_sigreturn_arm);
+ }
+ } else {
+ if (thumb) {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_thumb);
+ } else {
+ vdso_trampoline = VDSO_SYMBOL(vdso_base,
+ compat_sigreturn_arm);
+ }
+ }
+
+ retcode = ptr_to_compat(vdso_trampoline) + thumb;
+#else
unsigned int idx = thumb << 1;
- sigreturn_base = current->mm->context.vdso;
+ void *sigreturn_base = current->mm->context.vdso;
if (ka->sa.sa_flags & SA_SIGINFO)
idx += 3;
retcode = ptr_to_compat(sigreturn_base) +
(idx << 2) + thumb;
+#endif
}
regs->regs[0] = usig;
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 18/23] arm64: Add vDSO compat support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Add vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 23 +++++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ed3290494f1c..67bf39985d8c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -101,6 +101,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index b025304bde46..111cb0f7b73b 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,25 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CONFIG_CC_IS_CLANG), y)
+ $(warning CROSS_COMPILE_ARM32 is clang, the compat vDSO will not be built)
+ else ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -163,6 +179,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index c7bd0794855a..78a55fc8e0c7 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
@@ -62,6 +65,7 @@ obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o
obj-y += vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 18/23] arm64: Add vDSO compat support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Add vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 23 +++++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ed3290494f1c..67bf39985d8c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -101,6 +101,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index b025304bde46..111cb0f7b73b 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,25 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CONFIG_CC_IS_CLANG), y)
+ $(warning CROSS_COMPILE_ARM32 is clang, the compat vDSO will not be built)
+ else ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -163,6 +179,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index c7bd0794855a..78a55fc8e0c7 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
@@ -62,6 +65,7 @@ obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o
obj-y += vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 18/23] arm64: Add vDSO compat support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Add vDSO compat support to the arm64 building system.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/Kconfig | 1 +
arch/arm64/Makefile | 23 +++++++++++++++++++++--
arch/arm64/kernel/Makefile | 6 +++++-
3 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ed3290494f1c..67bf39985d8c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -101,6 +101,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
+ select GENERIC_COMPAT_VDSO
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
diff --git a/arch/arm64/Makefile b/arch/arm64/Makefile
index b025304bde46..111cb0f7b73b 100644
--- a/arch/arm64/Makefile
+++ b/arch/arm64/Makefile
@@ -49,9 +49,25 @@ $(warning Detected assembler with broken .inst; disassembly will be unreliable)
endif
endif
-KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst)
+ifeq ($(CONFIG_GENERIC_COMPAT_VDSO), y)
+ CROSS_COMPILE_ARM32 ?= $(CONFIG_CROSS_COMPILE_COMPAT_VDSO:"%"=%)
+
+ ifeq ($(CONFIG_CC_IS_CLANG), y)
+ $(warning CROSS_COMPILE_ARM32 is clang, the compat vDSO will not be built)
+ else ifeq ($(CROSS_COMPILE_ARM32),)
+ $(warning CROSS_COMPILE_ARM32 not defined or empty, the compat vDSO will not be built)
+ else ifeq ($(shell which $(CROSS_COMPILE_ARM32)gcc 2> /dev/null),)
+ $(error $(CROSS_COMPILE_ARM32)gcc not found, check CROSS_COMPILE_ARM32)
+ else
+ export CROSS_COMPILE_ARM32
+ export CONFIG_COMPAT_VDSO := y
+ compat_vdso := -DCONFIG_COMPAT_VDSO=1
+ endif
+endif
+
+KBUILD_CFLAGS += -mgeneral-regs-only $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += -fno-asynchronous-unwind-tables
-KBUILD_AFLAGS += $(lseinstr) $(brokengasinst)
+KBUILD_AFLAGS += $(lseinstr) $(brokengasinst) $(compat_vdso)
KBUILD_CFLAGS += $(call cc-option,-mabi=lp64)
KBUILD_AFLAGS += $(call cc-option,-mabi=lp64)
@@ -163,6 +179,9 @@ ifeq ($(KBUILD_EXTMOD),)
prepare: vdso_prepare
vdso_prepare: prepare0
$(Q)$(MAKE) $(build)=arch/arm64/kernel/vdso include/generated/vdso-offsets.h
+ $(if $(CONFIG_COMPAT_VDSO),$(Q)$(MAKE) \
+ $(build)=arch/arm64/kernel/vdso32 \
+ include/generated/vdso32-offsets.h)
endif
define archhelp
diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile
index c7bd0794855a..78a55fc8e0c7 100644
--- a/arch/arm64/kernel/Makefile
+++ b/arch/arm64/kernel/Makefile
@@ -28,7 +28,10 @@ $(obj)/%.stub.o: $(obj)/%.o FORCE
$(call if_changed,objcopy)
obj-$(CONFIG_COMPAT) += sys32.o signal32.o \
- sigreturn32.o sys_compat.o
+ sys_compat.o
+ifneq ($(CONFIG_COMPAT_VDSO), y)
+obj-$(CONFIG_COMPAT) += sigreturn32.o
+endif
obj-$(CONFIG_KUSER_HELPERS) += kuser32.o
obj-$(CONFIG_FUNCTION_TRACER) += ftrace.o entry-ftrace.o
obj-$(CONFIG_MODULES) += module.o
@@ -62,6 +65,7 @@ obj-$(CONFIG_ARM64_SSBD) += ssbd.o
obj-$(CONFIG_ARM64_PTR_AUTH) += pointer_auth.o
obj-y += vdso/ probes/
+obj-$(CONFIG_COMPAT_VDSO) += vdso32/
head-y := head.o
extra-y += $(head-y) vmlinux.lds
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Enable compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c8432e24207e..3adebe5f3573 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
CONFIG_ACPI_APEI_EINJ=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_ARM64_CRYPTO=y
CONFIG_CRYPTO_SHA1_ARM64_CE=y
CONFIG_CRYPTO_SHA2_ARM64_CE=y
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Enable compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c8432e24207e..3adebe5f3573 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
CONFIG_ACPI_APEI_EINJ=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_ARM64_CRYPTO=y
CONFIG_CRYPTO_SHA1_ARM64_CE=y
CONFIG_CRYPTO_SHA2_ARM64_CE=y
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
Enable compat vDSO support in the arm64 defconfig.
Note that the default compiler is set to arm-linux-gnueabihf-.
For this to take effect and for the compat vDSO to compile correctly, it
is required that this compiler is in the PATH.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm64/configs/defconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index c8432e24207e..3adebe5f3573 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
CONFIG_ACPI_APEI_EINJ=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
+CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
CONFIG_ARM64_CRYPTO=y
CONFIG_CRYPTO_SHA1_ARM64_CE=y
CONFIG_CRYPTO_SHA2_ARM64_CE=y
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 20/23] arm: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
Introduce the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 3 +
arch/arm/include/asm/vdso/gettimeofday.h | 82 +++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 108 ++++++---
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 ++---------------------
8 files changed, 205 insertions(+), 309 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 26524b75970a..d373c6e8aec0 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -48,6 +48,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if AEABI && !OABI_COMPAT
@@ -97,6 +99,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e7aeda0b255f
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(int clock_mode)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..68930c90c504 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -34,6 +34,7 @@
#include <asm/vdso.h>
#include <asm/vdso_datapage.h>
#include <clocksource/arm_arch_timer.h>
+#include <vdso/helpers.h>
#define MAX_SYMNAME 64
@@ -274,18 +275,6 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
mm->context.vdso = addr;
}
-static void vdso_write_begin(struct vdso_data *vdata)
-{
- ++vdso_data->seq_count;
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
-}
-
-static void vdso_write_end(struct vdso_data *vdata)
-{
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
-}
-
static bool tk_is_cntvct(const struct timekeeper *tk)
{
if (!IS_ENABLED(CONFIG_ARM_ARCH_TIMER))
@@ -307,17 +296,19 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
*/
void update_vsyscall(struct timekeeper *tk)
{
- struct timespec64 *wtm = &tk->wall_to_monotonic;
+ struct vdso_timestamp *vdso_ts;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
+ u64 nsec;
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +319,77 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift);
- vdso_data->wtm_clock_sec = wtm->tv_sec;
- vdso_data->wtm_clock_nsec = wtm->tv_nsec;
-
- if (vdso_data->tk_is_cntvct) {
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
- vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (__tk_is_cntvct) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1d81e8c3acf6 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..b470a8ed35a7 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
+#include <linux/types.h>
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
+ return __cvdso_clock_gettime(clock, ts);
}
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
-
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
-
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 20/23] arm: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
Introduce the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 3 +
arch/arm/include/asm/vdso/gettimeofday.h | 82 +++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 108 ++++++---
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 ++---------------------
8 files changed, 205 insertions(+), 309 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 26524b75970a..d373c6e8aec0 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -48,6 +48,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if AEABI && !OABI_COMPAT
@@ -97,6 +99,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e7aeda0b255f
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(int clock_mode)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..68930c90c504 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -34,6 +34,7 @@
#include <asm/vdso.h>
#include <asm/vdso_datapage.h>
#include <clocksource/arm_arch_timer.h>
+#include <vdso/helpers.h>
#define MAX_SYMNAME 64
@@ -274,18 +275,6 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
mm->context.vdso = addr;
}
-static void vdso_write_begin(struct vdso_data *vdata)
-{
- ++vdso_data->seq_count;
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
-}
-
-static void vdso_write_end(struct vdso_data *vdata)
-{
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
-}
-
static bool tk_is_cntvct(const struct timekeeper *tk)
{
if (!IS_ENABLED(CONFIG_ARM_ARCH_TIMER))
@@ -307,17 +296,19 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
*/
void update_vsyscall(struct timekeeper *tk)
{
- struct timespec64 *wtm = &tk->wall_to_monotonic;
+ struct vdso_timestamp *vdso_ts;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
+ u64 nsec;
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +319,77 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift);
- vdso_data->wtm_clock_sec = wtm->tv_sec;
- vdso_data->wtm_clock_nsec = wtm->tv_nsec;
-
- if (vdso_data->tk_is_cntvct) {
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
- vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (__tk_is_cntvct) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1d81e8c3acf6 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..b470a8ed35a7 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
+#include <linux/types.h>
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
+ return __cvdso_clock_gettime(clock, ts);
}
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
-
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
-
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 20/23] arm: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The arm vDSO library requires some adaptations to use to take advantage
of the newly introduced generic vDSO library.
Introduce the following changes:
- Modification vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
- Implementation of elf note
Cc: Russell King <linux@armlinux.org.uk>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/arm/Kconfig | 3 +
arch/arm/include/asm/vdso/gettimeofday.h | 82 +++++++
arch/arm/include/asm/vdso_datapage.h | 25 +--
arch/arm/kernel/vdso.c | 108 ++++++---
arch/arm/vdso/Makefile | 13 +-
arch/arm/vdso/note.c | 15 ++
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 267 ++---------------------
8 files changed, 205 insertions(+), 309 deletions(-)
create mode 100644 arch/arm/include/asm/vdso/gettimeofday.h
create mode 100644 arch/arm/vdso/note.c
diff --git a/arch/arm/Kconfig b/arch/arm/Kconfig
index 26524b75970a..d373c6e8aec0 100644
--- a/arch/arm/Kconfig
+++ b/arch/arm/Kconfig
@@ -48,6 +48,8 @@ config ARM
select GENERIC_SMP_IDLE_THREAD
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
+ select GENERIC_GETTIMEOFDAY
+ select GENERIC_VDSO_32
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_ARCH_AUDITSYSCALL if AEABI && !OABI_COMPAT
@@ -97,6 +99,7 @@ config ARM
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UID16
select HAVE_VIRT_CPU_ACCOUNTING_GEN
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select MODULES_USE_ELF_REL
select NEED_DMA_MAP_STATE
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..e7aeda0b255f
--- /dev/null
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,82 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2018 ARM Limited
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <asm/arch_timer.h>
+#include <asm/unistd.h>
+#include <uapi/linux/time.h>
+
+extern struct vdso_data *__get_datapage(void);
+
+static __always_inline notrace int gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("r1") = _tz;
+ register struct __vdso_timeval *tv asm("r0") = _tv;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_gettimeofday;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_gettime;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("r1") = _ts;
+ register clockid_t clkid asm("r0") = _clkid;
+ register long ret asm ("r0");
+ register long nr asm("r7") = __NR_clock_getres;
+
+ asm volatile(
+ " swi #0\n"
+ : "=r" (ret)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "memory");
+
+ return ret;
+}
+
+static __always_inline notrace u64 __arch_get_hw_counter(int clock_mode)
+{
+ return arch_counter_get_cntvct();
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return __get_datapage();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/arm/include/asm/vdso_datapage.h b/arch/arm/include/asm/vdso_datapage.h
index 9be259442fca..b31df2467827 100644
--- a/arch/arm/include/asm/vdso_datapage.h
+++ b/arch/arm/include/asm/vdso_datapage.h
@@ -22,32 +22,9 @@
#ifndef __ASSEMBLY__
+#include <vdso/datapage.h>
#include <asm/page.h>
-/* Try to be cache-friendly on systems that don't implement the
- * generic timer: fit the unconditionally updated fields in the first
- * 32 bytes.
- */
-struct vdso_data {
- u32 seq_count; /* sequence count - odd during updates */
- u16 tk_is_cntvct; /* fall back to syscall if false */
- u16 cs_shift; /* clocksource shift */
- u32 xtime_coarse_sec; /* coarse time */
- u32 xtime_coarse_nsec;
-
- u32 wtm_clock_sec; /* wall to monotonic offset */
- u32 wtm_clock_nsec;
- u32 xtime_clock_sec; /* CLOCK_REALTIME - seconds */
- u32 cs_mult; /* clocksource multiplier */
-
- u64 cs_cycle_last; /* last cycle value */
- u64 cs_mask; /* clocksource mask */
-
- u64 xtime_clock_snsec; /* CLOCK_REALTIME sub-ns base */
- u32 tz_minuteswest; /* timezone info for gettimeofday(2) */
- u32 tz_dsttime;
-};
-
union vdso_data_store {
struct vdso_data data;
u8 page[PAGE_SIZE];
diff --git a/arch/arm/kernel/vdso.c b/arch/arm/kernel/vdso.c
index f4dd7f9663c1..68930c90c504 100644
--- a/arch/arm/kernel/vdso.c
+++ b/arch/arm/kernel/vdso.c
@@ -34,6 +34,7 @@
#include <asm/vdso.h>
#include <asm/vdso_datapage.h>
#include <clocksource/arm_arch_timer.h>
+#include <vdso/helpers.h>
#define MAX_SYMNAME 64
@@ -274,18 +275,6 @@ void arm_install_vdso(struct mm_struct *mm, unsigned long addr)
mm->context.vdso = addr;
}
-static void vdso_write_begin(struct vdso_data *vdata)
-{
- ++vdso_data->seq_count;
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_retry */
-}
-
-static void vdso_write_end(struct vdso_data *vdata)
-{
- smp_wmb(); /* Pairs with smp_rmb in vdso_read_begin */
- ++vdso_data->seq_count;
-}
-
static bool tk_is_cntvct(const struct timekeeper *tk)
{
if (!IS_ENABLED(CONFIG_ARM_ARCH_TIMER))
@@ -307,17 +296,19 @@ static bool tk_is_cntvct(const struct timekeeper *tk)
* counter again, making it even, indicating to userspace that the
* update is finished.
*
- * Userspace is expected to sample seq_count before reading any other
- * fields from the data page. If seq_count is odd, userspace is
+ * Userspace is expected to sample tb_seq_count before reading any other
+ * fields from the data page. If tb_seq_count is odd, userspace is
* expected to wait until it becomes even. After copying data from
- * the page, userspace must sample seq_count again; if it has changed
+ * the page, userspace must sample tb_seq_count again; if it has changed
* from its previous value, userspace must retry the whole sequence.
*
* Calls to update_vsyscall are serialized by the timekeeping core.
*/
void update_vsyscall(struct timekeeper *tk)
{
- struct timespec64 *wtm = &tk->wall_to_monotonic;
+ struct vdso_timestamp *vdso_ts;
+ u32 __tk_is_cntvct = tk_is_cntvct(tk);
+ u64 nsec;
if (!cntvct_ok) {
/* The entry points have been zeroed, so there is no
@@ -328,20 +319,77 @@ void update_vsyscall(struct timekeeper *tk)
vdso_write_begin(vdso_data);
- vdso_data->tk_is_cntvct = tk_is_cntvct(tk);
- vdso_data->xtime_coarse_sec = tk->xtime_sec;
- vdso_data->xtime_coarse_nsec = (u32)(tk->tkr_mono.xtime_nsec >>
- tk->tkr_mono.shift);
- vdso_data->wtm_clock_sec = wtm->tv_sec;
- vdso_data->wtm_clock_nsec = wtm->tv_nsec;
-
- if (vdso_data->tk_is_cntvct) {
- vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_snsec = tk->tkr_mono.xtime_nsec;
- vdso_data->cs_mult = tk->tkr_mono.mult;
- vdso_data->cs_shift = tk->tkr_mono.shift;
- vdso_data->cs_mask = tk->tkr_mono.mask;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (__tk_is_cntvct) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
vdso_write_end(vdso_data);
diff --git a/arch/arm/vdso/Makefile b/arch/arm/vdso/Makefile
index f4efff9d3afb..05f8c6699d48 100644
--- a/arch/arm/vdso/Makefile
+++ b/arch/arm/vdso/Makefile
@@ -1,7 +1,13 @@
# SPDX-License-Identifier: GPL-2.0
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
+include $(srctree)/lib/vdso/Makefile
+
hostprogs-y := vdsomunge
-obj-vdso := vgettimeofday.o datapage.o
+obj-vdso := vgettimeofday.o datapage.o note.o
# Build rules
targets := $(obj-vdso) vdso.so vdso.so.dbg vdso.so.raw vdso.lds
@@ -25,7 +31,11 @@ CFLAGS_REMOVE_vdso.o = -pg
# Force -O2 to avoid libgcc dependencies
CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
+ifeq ($(c-gettimeofday-y),)
CFLAGS_vgettimeofday.o = -O2
+else
+CFLAGS_vgettimeofday.o = -O2 -include $(c-gettimeofday-y)
+endif
# Disable gcov profiling for VDSO code
GCOV_PROFILE := n
@@ -39,6 +49,7 @@ $(obj)/vdso.o : $(obj)/vdso.so
# Link rule for the .so file
$(obj)/vdso.so.raw: $(src)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso.so.dbg: $(obj)/vdso.so.raw $(obj)/vdsomunge FORCE
$(call if_changed,vdsomunge)
diff --git a/arch/arm/vdso/note.c b/arch/arm/vdso/note.c
new file mode 100644
index 000000000000..eff5bf9efb8b
--- /dev/null
+++ b/arch/arm/vdso/note.c
@@ -0,0 +1,15 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2012-2018 ARM Limited
+ *
+ * This supplies .note.* sections to go into the PT_NOTE inside the vDSO text.
+ * Here we can supply some information useful to userland.
+ */
+
+#include <linux/uts.h>
+#include <linux/version.h>
+#include <linux/elfnote.h>
+#include <linux/build-salt.h>
+
+ELFNOTE32("Linux", 0, LINUX_VERSION_CODE);
+BUILD_SALT;
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 89ca89f12d23..1d81e8c3acf6 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -82,6 +82,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index a9dd619c6c29..b470a8ed35a7 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -1,270 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2015 Mentor Graphics Corporation.
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
*
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public License
- * as published by the Free Software Foundation; version 2 of the
- * License.
+ * Copyright (C) 2018 ARM Limited
*
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
-
-#include <linux/compiler.h>
-#include <linux/hrtimer.h>
#include <linux/time.h>
-#include <asm/arch_timer.h>
-#include <asm/barrier.h>
-#include <asm/bug.h>
-#include <asm/page.h>
-#include <asm/unistd.h>
-#include <asm/vdso_datapage.h>
-
-#ifndef CONFIG_AEABI
-#error This code depends on AEABI system call conventions
-#endif
+#include <linux/types.h>
-extern struct vdso_data *__get_datapage(void);
-
-static notrace u32 __vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
{
- u32 seq;
-repeat:
- seq = READ_ONCE(vdata->seq_count);
- if (seq & 1) {
- cpu_relax();
- goto repeat;
- }
- return seq;
+ return __cvdso_clock_gettime(clock, ts);
}
-static notrace u32 vdso_read_begin(const struct vdso_data *vdata)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- u32 seq;
-
- seq = __vdso_read_begin(vdata);
-
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_end */
- return seq;
-}
-
-static notrace int vdso_read_retry(const struct vdso_data *vdata, u32 start)
-{
- smp_rmb(); /* Pairs with smp_wmb in vdso_write_begin */
- return vdata->seq_count != start;
-}
-
-static notrace long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("r1") = _ts;
- register clockid_t clkid asm("r0") = _clkid;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-static notrace int do_realtime_coarse(struct timespec *ts,
- struct vdso_data *vdata)
-{
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-static notrace int do_monotonic_coarse(struct timespec *ts,
- struct vdso_data *vdata)
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
{
- struct timespec tomono;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- ts->tv_sec = vdata->xtime_coarse_sec;
- ts->tv_nsec = vdata->xtime_coarse_nsec;
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- timespec_add_ns(ts, tomono.tv_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_ARM_ARCH_TIMER
-
-static notrace u64 get_ns(struct vdso_data *vdata)
-{
- u64 cycle_delta;
- u64 cycle_now;
- u64 nsec;
-
- cycle_now = arch_counter_get_cntvct();
-
- cycle_delta = (cycle_now - vdata->cs_cycle_last) & vdata->cs_mask;
-
- nsec = (cycle_delta * vdata->cs_mult) + vdata->xtime_clock_snsec;
- nsec >>= vdata->cs_shift;
-
- return nsec;
-}
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs);
-
- return 0;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- struct timespec tomono;
- u64 nsecs;
- u32 seq;
-
- do {
- seq = vdso_read_begin(vdata);
-
- if (!vdata->tk_is_cntvct)
- return -1;
-
- ts->tv_sec = vdata->xtime_clock_sec;
- nsecs = get_ns(vdata);
-
- tomono.tv_sec = vdata->wtm_clock_sec;
- tomono.tv_nsec = vdata->wtm_clock_nsec;
-
- } while (vdso_read_retry(vdata, seq));
-
- ts->tv_sec += tomono.tv_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, nsecs + tomono.tv_nsec);
-
- return 0;
-}
-
-#else /* CONFIG_ARM_ARCH_TIMER */
-
-static notrace int do_realtime(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-static notrace int do_monotonic(struct timespec *ts, struct vdso_data *vdata)
-{
- return -1;
-}
-
-#endif /* CONFIG_ARM_ARCH_TIMER */
-
-notrace int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- struct vdso_data *vdata;
- int ret = -1;
-
- vdata = __get_datapage();
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, vdata);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, vdata);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, vdata);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, vdata);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
-
-static notrace long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("r1") = _tz;
- register struct timeval *tv asm("r0") = _tv;
- register long ret asm ("r0");
- register long nr asm("r7") = __NR_gettimeofday;
-
- asm volatile(
- " swi #0\n"
- : "=r" (ret)
- : "r" (tv), "r" (tz), "r" (nr)
- : "memory");
-
- return ret;
-}
-
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- struct timespec ts;
- struct vdso_data *vdata;
- int ret;
-
- vdata = __get_datapage();
-
- ret = do_realtime(&ts, vdata);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
- if (tz) {
- tz->tz_minuteswest = vdata->tz_minuteswest;
- tz->tz_dsttime = vdata->tz_dsttime;
- }
-
- return ret;
+ return __cvdso_clock_getres(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 21/23] mips: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The mips vDSO library requires some adaptations to take advantage of the
newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +----
arch/mips/include/asm/vdso/gettimeofday.h | 165 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++++++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +++
11 files changed, 313 insertions(+), 356 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index a84c24d894aa..900fad3eb505 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -34,6 +34,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -74,6 +75,7 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select ISA if EISA
select MODULES_USE_ELF_RELA if MODULES && 64BIT
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..0b1d0b4578a6 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,8 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
-{
- ++data->seq_count;
-
- /* Ensure sequence update is written before other data page values. */
- smp_wmb();
-}
-
-static inline void vdso_data_write_end(union mips_vdso_data *data)
-{
- /* Ensure data values are written before updating sequence again. */
- smp_wmb();
- ++data->seq_count;
-}
-
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b275392f6b1b
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,165 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct __vdso_timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 9df3ebdc7b0f..30cbcb6fea59 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -24,9 +24,11 @@
#include <asm/mips-cps.h>
#include <asm/page.h>
#include <asm/vdso.h>
+#include <vdso/helpers.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +74,94 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ struct vdso_timestamp *vdso_ts;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- vdso_data_write_end(&vdso_data);
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +234,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +260,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 0ede4deb8181..70280c5f4e41 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -15,15 +21,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -50,6 +64,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
UBSAN_SANITIZE := n
@@ -98,6 +114,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -136,6 +153,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -176,6 +194,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..90b2c2785e32 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..8da361d919f0
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 21/23] mips: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
The mips vDSO library requires some adaptations to take advantage of the
newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +----
arch/mips/include/asm/vdso/gettimeofday.h | 165 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++++++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +++
11 files changed, 313 insertions(+), 356 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index a84c24d894aa..900fad3eb505 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -34,6 +34,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -74,6 +75,7 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select ISA if EISA
select MODULES_USE_ELF_RELA if MODULES && 64BIT
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..0b1d0b4578a6 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,8 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
-{
- ++data->seq_count;
-
- /* Ensure sequence update is written before other data page values. */
- smp_wmb();
-}
-
-static inline void vdso_data_write_end(union mips_vdso_data *data)
-{
- /* Ensure data values are written before updating sequence again. */
- smp_wmb();
- ++data->seq_count;
-}
-
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b275392f6b1b
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,165 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct __vdso_timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 9df3ebdc7b0f..30cbcb6fea59 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -24,9 +24,11 @@
#include <asm/mips-cps.h>
#include <asm/page.h>
#include <asm/vdso.h>
+#include <vdso/helpers.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +74,94 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ struct vdso_timestamp *vdso_ts;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- vdso_data_write_end(&vdso_data);
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +234,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +260,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 0ede4deb8181..70280c5f4e41 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -15,15 +21,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -50,6 +64,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
UBSAN_SANITIZE := n
@@ -98,6 +114,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -136,6 +153,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -176,6 +194,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..90b2c2785e32 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..8da361d919f0
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 21/23] mips: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The mips vDSO library requires some adaptations to take advantage of the
newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Ralf Baechle <ralf@linux-mips.org>
Cc: Paul Burton <paul.burton@mips.com>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/mips/Kconfig | 2 +
arch/mips/include/asm/vdso.h | 60 +----
arch/mips/include/asm/vdso/gettimeofday.h | 165 +++++++++++++
arch/mips/{ => include/asm}/vdso/vdso.h | 6 +-
arch/mips/kernel/vdso.c | 107 +++++++--
arch/mips/vdso/Makefile | 23 +-
arch/mips/vdso/elf.S | 2 +-
arch/mips/vdso/gettimeofday.c | 273 ----------------------
arch/mips/vdso/sigreturn.S | 2 +-
arch/mips/vdso/vdso.lds.S | 1 +
arch/mips/vdso/vgettimeofday.c | 28 +++
11 files changed, 313 insertions(+), 356 deletions(-)
create mode 100644 arch/mips/include/asm/vdso/gettimeofday.h
rename arch/mips/{ => include/asm}/vdso/vdso.h (90%)
delete mode 100644 arch/mips/vdso/gettimeofday.c
create mode 100644 arch/mips/vdso/vgettimeofday.c
diff --git a/arch/mips/Kconfig b/arch/mips/Kconfig
index a84c24d894aa..900fad3eb505 100644
--- a/arch/mips/Kconfig
+++ b/arch/mips/Kconfig
@@ -34,6 +34,7 @@ config MIPS
select GENERIC_SCHED_CLOCK if !CAVIUM_OCTEON_SOC
select GENERIC_SMP_IDLE_THREAD
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HANDLE_DOMAIN_IRQ
select HAVE_ARCH_COMPILER_H
select HAVE_ARCH_JUMP_LABEL
@@ -74,6 +75,7 @@ config MIPS
select HAVE_STACKPROTECTOR
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_VIRT_CPU_ACCOUNTING_GEN if 64BIT || !SMP
+ select HAVE_GENERIC_VDSO
select IRQ_FORCED_THREADING
select ISA if EISA
select MODULES_USE_ELF_RELA if MODULES && 64BIT
diff --git a/arch/mips/include/asm/vdso.h b/arch/mips/include/asm/vdso.h
index 91bf0c2c265c..0b1d0b4578a6 100644
--- a/arch/mips/include/asm/vdso.h
+++ b/arch/mips/include/asm/vdso.h
@@ -12,6 +12,7 @@
#define __ASM_VDSO_H
#include <linux/mm_types.h>
+#include <vdso/datapage.h>
#include <asm/barrier.h>
@@ -54,12 +55,12 @@ extern struct mips_vdso_image vdso_image_n32;
#endif
/**
- * union mips_vdso_data - Data provided by the kernel for the VDSO.
+ * struct vdso_data - Data provided by the kernel for the VDSO.
* @xtime_sec: Current real time (seconds part).
* @xtime_nsec: Current real time (nanoseconds part, shifted).
* @wall_to_mono_sec: Wall-to-monotonic offset (seconds part).
* @wall_to_mono_nsec: Wall-to-monotonic offset (nanoseconds part).
- * @seq_count: Counter to synchronise updates (odd = updating).
+ * @tb_seq_count: Counter to synchronise updates (odd = updating).
* @cs_shift: Clocksource shift value.
* @clock_mode: Clocksource to use for time functions.
* @cs_mult: Clocksource multiplier value.
@@ -76,61 +77,8 @@ extern struct mips_vdso_image vdso_image_n32;
* for both 64- and 32-bit (for 32-bit userland on 64-bit kernel).
*/
union mips_vdso_data {
- struct {
- u64 xtime_sec;
- u64 xtime_nsec;
- u64 wall_to_mono_sec;
- u64 wall_to_mono_nsec;
- u32 seq_count;
- u32 cs_shift;
- u8 clock_mode;
- u32 cs_mult;
- u64 cs_cycle_last;
- u64 cs_mask;
- s32 tz_minuteswest;
- s32 tz_dsttime;
- };
-
+ struct vdso_data data;
u8 page[PAGE_SIZE];
};
-static inline u32 vdso_data_read_begin(const union mips_vdso_data *data)
-{
- u32 seq;
-
- while (true) {
- seq = READ_ONCE(data->seq_count);
- if (likely(!(seq & 1))) {
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return seq;
- }
-
- cpu_relax();
- }
-}
-
-static inline bool vdso_data_read_retry(const union mips_vdso_data *data,
- u32 start_seq)
-{
- /* Paired with smp_wmb() in vdso_data_write_*(). */
- smp_rmb();
- return unlikely(data->seq_count != start_seq);
-}
-
-static inline void vdso_data_write_begin(union mips_vdso_data *data)
-{
- ++data->seq_count;
-
- /* Ensure sequence update is written before other data page values. */
- smp_wmb();
-}
-
-static inline void vdso_data_write_end(union mips_vdso_data *data)
-{
- /* Ensure data values are written before updating sequence again. */
- smp_wmb();
- ++data->seq_count;
-}
-
#endif /* __ASM_VDSO_H */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..b275392f6b1b
--- /dev/null
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,165 @@
+/*
+ * Copyright (C) 2018 ARM Limited
+ * Copyright (C) 2015 Imagination Technologies
+ * Author: Alex Smith <alex.smith@imgtec.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/compiler.h>
+#include <linux/time.h>
+
+#include <asm/vdso/vdso.h>
+#include <asm/clocksource.h>
+#include <asm/io.h>
+#include <asm/unistd.h>
+#include <asm/vdso.h>
+
+#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ register struct timezone *tz asm("a1") = _tz;
+ register struct __vdso_timeval *tv asm("a0") = _tv;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_gettimeofday;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (tv), "r" (tz), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#else
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ return -1;
+}
+
+#endif
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_gettime;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+static __always_inline notrace int clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ register struct __vdso_timespec *ts asm("a1") = _ts;
+ register clockid_t clkid asm("a0") = _clkid;
+ register long ret asm("v0");
+ register long nr asm("v0") = __NR_clock_getres;
+ register long error asm("a3");
+
+ asm volatile(
+ " syscall\n"
+ : "=r" (ret), "=r" (error)
+ : "r" (clkid), "r" (ts), "r" (nr)
+ : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
+ "$14", "$15", "$24", "$25", "hi", "lo", "memory");
+
+ return error ? -ret : ret;
+}
+
+#ifdef CONFIG_CSRC_R4K
+
+static __always_inline u64 read_r4k_count(void)
+{
+ unsigned int count;
+
+ __asm__ __volatile__(
+ " .set push\n"
+ " .set mips32r2\n"
+ " rdhwr %0, $2\n"
+ " .set pop\n"
+ : "=r" (count));
+
+ return count;
+}
+
+#endif
+
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+
+static __always_inline u64 read_gic_count(const struct vdso_data *data)
+{
+ void __iomem *gic = get_gic(data);
+ u32 hi, hi2, lo;
+
+ do {
+ hi = __raw_readl(gic + sizeof(lo));
+ lo = __raw_readl(gic);
+ hi2 = __raw_readl(gic + sizeof(lo));
+ } while (hi2 != hi);
+
+ return (((u64)hi) << 32) + lo;
+}
+
+#endif
+
+static __always_inline notrace u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ const struct vdso_data *data = get_vdso_data();
+ u64 cycle_now;
+
+ switch (clock_mode) {
+#ifdef CONFIG_CSRC_R4K
+ case VDSO_CLOCK_R4K:
+ cycle_now = read_r4k_count();
+ break;
+#endif
+#ifdef CONFIG_CLKSRC_MIPS_GIC
+ case VDSO_CLOCK_GIC:
+ cycle_now = read_gic_count(data);
+ break;
+#endif
+ default:
+ cycle_now = 0;
+ break;
+ }
+
+ return cycle_now;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return get_vdso_data();
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/mips/vdso/vdso.h b/arch/mips/include/asm/vdso/vdso.h
similarity index 90%
rename from arch/mips/vdso/vdso.h
rename to arch/mips/include/asm/vdso/vdso.h
index cfb1be441dec..048d12fbb925 100644
--- a/arch/mips/vdso/vdso.h
+++ b/arch/mips/include/asm/vdso/vdso.h
@@ -72,14 +72,14 @@ static inline unsigned long get_vdso_base(void)
return addr;
}
-static inline const union mips_vdso_data *get_vdso_data(void)
+static inline const struct vdso_data *get_vdso_data(void)
{
- return (const union mips_vdso_data *)(get_vdso_base() - PAGE_SIZE);
+ return (const struct vdso_data *)(get_vdso_base() - PAGE_SIZE);
}
#ifdef CONFIG_CLKSRC_MIPS_GIC
-static inline void __iomem *get_gic(const union mips_vdso_data *data)
+static inline void __iomem *get_gic(const struct vdso_data *data)
{
return (void __iomem *)data - PAGE_SIZE;
}
diff --git a/arch/mips/kernel/vdso.c b/arch/mips/kernel/vdso.c
index 9df3ebdc7b0f..30cbcb6fea59 100644
--- a/arch/mips/kernel/vdso.c
+++ b/arch/mips/kernel/vdso.c
@@ -24,9 +24,11 @@
#include <asm/mips-cps.h>
#include <asm/page.h>
#include <asm/vdso.h>
+#include <vdso/helpers.h>
/* Kernel-provided data used by the VDSO. */
-static union mips_vdso_data vdso_data __page_aligned_data;
+static union mips_vdso_data mips_vdso_data __page_aligned_data;
+static struct vdso_data *vdso_data = &mips_vdso_data.data;
/*
* Mapping for the VDSO data/GIC pages. The real pages are mapped manually, as
@@ -72,29 +74,94 @@ subsys_initcall(init_vdso);
void update_vsyscall(struct timekeeper *tk)
{
- vdso_data_write_begin(&vdso_data);
-
- vdso_data.xtime_sec = tk->xtime_sec;
- vdso_data.xtime_nsec = tk->tkr_mono.xtime_nsec;
- vdso_data.wall_to_mono_sec = tk->wall_to_monotonic.tv_sec;
- vdso_data.wall_to_mono_nsec = tk->wall_to_monotonic.tv_nsec;
- vdso_data.cs_shift = tk->tkr_mono.shift;
-
- vdso_data.clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.cs_mult = tk->tkr_mono.mult;
- vdso_data.cs_cycle_last = tk->tkr_mono.cycle_last;
- vdso_data.cs_mask = tk->tkr_mono.mask;
+ struct vdso_timestamp *vdso_ts;
+ u64 nsec;
+
+ vdso_write_begin(vdso_data);
+
+ vdso_data->clock_mode = tk->tkr_mono.clock->archdata.vdso_clock_mode;
+
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
+ while (nsec >= NSEC_PER_SEC) {
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->clock_mode = 0;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdso_data->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdso_data->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdso_data->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdso_data->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdso_data->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdso_data->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdso_data->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdso_data->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
}
- vdso_data_write_end(&vdso_data);
+ vdso_write_end(vdso_data);
}
void update_vsyscall_tz(void)
{
- if (vdso_data.clock_mode != VDSO_CLOCK_NONE) {
- vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vdso_data.tz_dsttime = sys_tz.tz_dsttime;
+ if (vdso_data->clock_mode != VDSO_CLOCK_NONE) {
+ vdso_data->tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data->tz_dsttime = sys_tz.tz_dsttime;
}
}
@@ -167,7 +234,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
*/
if (cpu_has_dc_aliases) {
base = __ALIGN_MASK(base, shm_align_mask);
- base += ((unsigned long)&vdso_data - gic_size) & shm_align_mask;
+ base += ((unsigned long)vdso_data - gic_size) & shm_align_mask;
}
data_addr = base + gic_size;
@@ -193,7 +260,7 @@ int arch_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
/* Map data page. */
ret = remap_pfn_range(vma, data_addr,
- virt_to_phys(&vdso_data) >> PAGE_SHIFT,
+ virt_to_phys(vdso_data) >> PAGE_SHIFT,
PAGE_SIZE, PAGE_READONLY);
if (ret)
goto out;
diff --git a/arch/mips/vdso/Makefile b/arch/mips/vdso/Makefile
index 0ede4deb8181..70280c5f4e41 100644
--- a/arch/mips/vdso/Makefile
+++ b/arch/mips/vdso/Makefile
@@ -1,6 +1,12 @@
# SPDX-License-Identifier: GPL-2.0
# Objects to go into the VDSO.
-obj-vdso-y := elf.o gettimeofday.o sigreturn.o
+
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_MIPS_JUMP_SLOT|R_MIPS_GLOB_DAT
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso-y := elf.o vgettimeofday.o sigreturn.o
# Common compiler flags between ABIs.
ccflags-vdso := \
@@ -15,15 +21,23 @@ ifdef CONFIG_CC_IS_CLANG
ccflags-vdso += $(filter --target=%,$(KBUILD_CFLAGS))
endif
+#
+# The -fno-jump-tables flag only prevents the compiler from generating
+# jump tables but does not prevent the compiler from emitting absolute
+# offsets.
cflags-vdso := $(ccflags-vdso) \
$(filter -W%,$(filter-out -Wa$(comma)%,$(KBUILD_CFLAGS))) \
-O2 -g -fPIC -fno-strict-aliasing -fno-common -fno-builtin -G 0 \
- -DDISABLE_BRANCH_PROFILING \
+ -fno-stack-protector -fno-jump-tables -DDISABLE_BRANCH_PROFILING \
$(call cc-option, -fno-asynchronous-unwind-tables) \
$(call cc-option, -fno-stack-protector)
aflags-vdso := $(ccflags-vdso) \
-D__ASSEMBLY__ -Wa,-gdwarf-2
+ifneq ($(c-gettimeofday-y),)
+CFLAGS_vgettimeofday.o = -include $(c-gettimeofday-y)
+endif
+
#
# For the pre-R6 code in arch/mips/vdso/vdso.h for locating
# the base address of VDSO, the linker will emit a R_MIPS_PC32
@@ -50,6 +64,8 @@ VDSO_LDFLAGS := \
$(call cc-ldoption, -Wl$(comma)--hash-style=sysv) \
$(call cc-ldoption, -Wl$(comma)--build-id)
+CFLAGS_REMOVE_vdso.o = -pg
+
GCOV_PROFILE := n
UBSAN_SANITIZE := n
@@ -98,6 +114,7 @@ $(obj)/vdso.lds: KBUILD_CPPFLAGS := $(ccflags-vdso) $(native-abi)
$(obj)/vdso.so.dbg.raw: $(obj)/vdso.lds $(obj-vdso) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-image.c: $(obj)/vdso.so.dbg.raw $(obj)/vdso.so.raw \
$(obj)/genvdso FORCE
@@ -136,6 +153,7 @@ $(obj)/vdso-o32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-o32.so.dbg.raw: $(obj)/vdso-o32.lds $(obj-vdso-o32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-o32-image.c: VDSO_NAME := o32
$(obj)/vdso-o32-image.c: $(obj)/vdso-o32.so.dbg.raw $(obj)/vdso-o32.so.raw \
@@ -176,6 +194,7 @@ $(obj)/vdso-n32.lds: $(src)/vdso.lds.S FORCE
$(obj)/vdso-n32.so.dbg.raw: $(obj)/vdso-n32.lds $(obj-vdso-n32) FORCE
$(call if_changed,vdsold)
+ $(call if_changed,vdso_check)
$(obj)/vdso-n32-image.c: VDSO_NAME := n32
$(obj)/vdso-n32-image.c: $(obj)/vdso-n32.so.dbg.raw $(obj)/vdso-n32.so.raw \
diff --git a/arch/mips/vdso/elf.S b/arch/mips/vdso/elf.S
index 428a1917afc6..c0c85d126094 100644
--- a/arch/mips/vdso/elf.S
+++ b/arch/mips/vdso/elf.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <asm/isa-rev.h>
diff --git a/arch/mips/vdso/gettimeofday.c b/arch/mips/vdso/gettimeofday.c
deleted file mode 100644
index e22b422f282c..000000000000
--- a/arch/mips/vdso/gettimeofday.c
+++ /dev/null
@@ -1,273 +0,0 @@
-/*
- * Copyright (C) 2015 Imagination Technologies
- * Author: Alex Smith <alex.smith@imgtec.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include "vdso.h"
-
-#include <linux/compiler.h>
-#include <linux/time.h>
-
-#include <asm/clocksource.h>
-#include <asm/io.h>
-#include <asm/unistd.h>
-#include <asm/vdso.h>
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-static __always_inline long gettimeofday_fallback(struct timeval *_tv,
- struct timezone *_tz)
-{
- register struct timezone *tz asm("a1") = _tz;
- register struct timeval *tv asm("a0") = _tv;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_gettimeofday;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (tv), "r" (tz), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-#endif
-
-static __always_inline long clock_gettime_fallback(clockid_t _clkid,
- struct timespec *_ts)
-{
- register struct timespec *ts asm("a1") = _ts;
- register clockid_t clkid asm("a0") = _clkid;
- register long ret asm("v0");
- register long nr asm("v0") = __NR_clock_gettime;
- register long error asm("a3");
-
- asm volatile(
- " syscall\n"
- : "=r" (ret), "=r" (error)
- : "r" (clkid), "r" (ts), "r" (nr)
- : "$1", "$3", "$8", "$9", "$10", "$11", "$12", "$13",
- "$14", "$15", "$24", "$25", "hi", "lo", "memory");
-
- return error ? -ret : ret;
-}
-
-static __always_inline int do_realtime_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
- } while (vdso_data_read_retry(data, start_seq));
-
- return 0;
-}
-
-static __always_inline int do_monotonic_coarse(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- ts->tv_sec = data->xtime_sec;
- ts->tv_nsec = data->xtime_nsec >> data->cs_shift;
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- timespec_add_ns(ts, to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_CSRC_R4K
-
-static __always_inline u64 read_r4k_count(void)
-{
- unsigned int count;
-
- __asm__ __volatile__(
- " .set push\n"
- " .set mips32r2\n"
- " rdhwr %0, $2\n"
- " .set pop\n"
- : "=r" (count));
-
- return count;
-}
-
-#endif
-
-#ifdef CONFIG_CLKSRC_MIPS_GIC
-
-static __always_inline u64 read_gic_count(const union mips_vdso_data *data)
-{
- void __iomem *gic = get_gic(data);
- u32 hi, hi2, lo;
-
- do {
- hi = __raw_readl(gic + sizeof(lo));
- lo = __raw_readl(gic);
- hi2 = __raw_readl(gic + sizeof(lo));
- } while (hi2 != hi);
-
- return (((u64)hi) << 32) + lo;
-}
-
-#endif
-
-static __always_inline u64 get_ns(const union mips_vdso_data *data)
-{
- u64 cycle_now, delta, nsec;
-
- switch (data->clock_mode) {
-#ifdef CONFIG_CSRC_R4K
- case VDSO_CLOCK_R4K:
- cycle_now = read_r4k_count();
- break;
-#endif
-#ifdef CONFIG_CLKSRC_MIPS_GIC
- case VDSO_CLOCK_GIC:
- cycle_now = read_gic_count(data);
- break;
-#endif
- default:
- return 0;
- }
-
- delta = (cycle_now - data->cs_cycle_last) & data->cs_mask;
-
- nsec = (delta * data->cs_mult) + data->xtime_nsec;
- nsec >>= data->cs_shift;
-
- return nsec;
-}
-
-static __always_inline int do_realtime(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns);
-
- return 0;
-}
-
-static __always_inline int do_monotonic(struct timespec *ts,
- const union mips_vdso_data *data)
-{
- u32 start_seq;
- u64 ns;
- u64 to_mono_sec;
- u64 to_mono_nsec;
-
- do {
- start_seq = vdso_data_read_begin(data);
-
- if (data->clock_mode == VDSO_CLOCK_NONE)
- return -ENOSYS;
-
- ts->tv_sec = data->xtime_sec;
- ns = get_ns(data);
-
- to_mono_sec = data->wall_to_mono_sec;
- to_mono_nsec = data->wall_to_mono_nsec;
- } while (vdso_data_read_retry(data, start_seq));
-
- ts->tv_sec += to_mono_sec;
- ts->tv_nsec = 0;
- timespec_add_ns(ts, ns + to_mono_nsec);
-
- return 0;
-}
-
-#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
-
-/*
- * This is behind the ifdef so that we don't provide the symbol when there's no
- * possibility of there being a usable clocksource, because there's nothing we
- * can do without it. When libc fails the symbol lookup it should fall back on
- * the standard syscall path.
- */
-int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
-{
- const union mips_vdso_data *data = get_vdso_data();
- struct timespec ts;
- int ret;
-
- ret = do_realtime(&ts, data);
- if (ret)
- return gettimeofday_fallback(tv, tz);
-
- if (tv) {
- tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / 1000;
- }
-
- if (tz) {
- tz->tz_minuteswest = data->tz_minuteswest;
- tz->tz_dsttime = data->tz_dsttime;
- }
-
- return 0;
-}
-
-#endif /* CONFIG_MIPS_CLOCK_VSYSCALL */
-
-int __vdso_clock_gettime(clockid_t clkid, struct timespec *ts)
-{
- const union mips_vdso_data *data = get_vdso_data();
- int ret = -1;
-
- switch (clkid) {
- case CLOCK_REALTIME_COARSE:
- ret = do_realtime_coarse(ts, data);
- break;
- case CLOCK_MONOTONIC_COARSE:
- ret = do_monotonic_coarse(ts, data);
- break;
- case CLOCK_REALTIME:
- ret = do_realtime(ts, data);
- break;
- case CLOCK_MONOTONIC:
- ret = do_monotonic(ts, data);
- break;
- default:
- break;
- }
-
- if (ret)
- ret = clock_gettime_fallback(clkid, ts);
-
- return ret;
-}
diff --git a/arch/mips/vdso/sigreturn.S b/arch/mips/vdso/sigreturn.S
index 30c6219912ac..c2b05956e4cb 100644
--- a/arch/mips/vdso/sigreturn.S
+++ b/arch/mips/vdso/sigreturn.S
@@ -8,7 +8,7 @@
* option) any later version.
*/
-#include "vdso.h"
+#include <asm/vdso/vdso.h>
#include <uapi/asm/unistd.h>
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 8df7dd53e8e0..90b2c2785e32 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -99,6 +99,7 @@ VERSION
global:
__vdso_clock_gettime;
__vdso_gettimeofday;
+ __vdso_clock_getres;
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
new file mode 100644
index 000000000000..8da361d919f0
--- /dev/null
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -0,0 +1,28 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * ARM64 compat userspace implementations of gettimeofday() and similar.
+ *
+ * Copyright (C) 2018 ARM Limited
+ *
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __vdso_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock_id, res);
+}
+
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The x86 vDSO library requires some adaptations to take advantage of the
newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/x86/Kconfig | 3 +
arch/x86/entry/vdso/Makefile | 9 +
arch/x86/entry/vdso/vclock_gettime.c | 239 +++--------------------
arch/x86/entry/vdso/vdso.lds.S | 2 +
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
arch/x86/entry/vdso/vdsox32.lds.S | 1 +
arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 +++++++----
arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++++++++++++++
arch/x86/include/asm/vgtod.h | 73 +------
arch/x86/include/asm/vvar.h | 2 +-
10 files changed, 318 insertions(+), 325 deletions(-)
create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 68261430fe6e..9422cc122567 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -18,6 +18,7 @@ config X86_32
select HAVE_GENERIC_DMA_COHERENT
select MODULES_USE_ELF_REL
select OLD_SIGACTION
+ select GENERIC_VDSO_32
config X86_64
def_bool y
@@ -111,6 +112,7 @@ config X86
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HARDLOCKUP_CHECK_TIMESTAMP if X86_64
select HAVE_ACPI_APEI if ACPI
select HAVE_ACPI_APEI_NMI if ACPI
@@ -194,6 +196,7 @@ config X86
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
+ select HAVE_GENERIC_VDSO
select HOTPLUG_SMT if SMP
select IRQ_FORCED_THREADING
select NEED_SG_DMA_LENGTH
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 5bfe2243a08f..5dc3f251ca13 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -3,6 +3,12 @@
# Building vDSO images for x86.
#
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_X86_64_JUMP_SLOT|R_X86_64_GLOB_DAT|R_X86_64_RELATIVE|
+ARCH_REL_TYPE_ABS += R_386_GLOB_DAT|R_386_JMP_SLOT|R_386_RELATIVE
+include $(srctree)/lib/vdso/Makefile
+
KBUILD_CFLAGS += $(DISABLE_LTO)
KASAN_SANITIZE := n
UBSAN_SANITIZE := n
@@ -51,6 +57,7 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 --no-undefined \
$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi
hostprogs-y += vdso2c
@@ -121,6 +128,7 @@ $(obj)/%.so: $(obj)/%.so.dbg
$(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
CPPFLAGS_vdso32.lds = $(CPPFLAGS_vdso.lds)
VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1
@@ -160,6 +168,7 @@ $(obj)/vdso32.so.dbg: FORCE \
$(obj)/vdso32/system_call.o \
$(obj)/vdso32/sigreturn.o
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
#
# The DSO images are built using a special linker script.
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index 007b3fe9d727..6ff8cabf4c3e 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -1,240 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2006 Andi Kleen, SUSE Labs.
- * Subject to the GNU Public License, v.2
- *
* Fast user context implementation of clock_gettime, gettimeofday, and time.
*
+ * Copyright 2019 ARM Limited
+ * Copyright 2006 Andi Kleen, SUSE Labs.
* 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
* sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
- *
- * The code should have no internal unresolved relocations.
- * Check with readelf after changing.
*/
-
-#include <uapi/linux/time.h>
-#include <asm/vgtod.h>
-#include <asm/vvar.h>
-#include <asm/unistd.h>
-#include <asm/msr.h>
-#include <asm/pvclock.h>
-#include <asm/mshyperv.h>
-#include <linux/math64.h>
#include <linux/time.h>
-#include <linux/kernel.h>
+#include <linux/types.h>
-#define gtod (&VVAR(vsyscall_gtod_data))
+#include "../../../../lib/vdso/gettimeofday.c"
-extern int __vdso_clock_gettime(clockid_t clock, struct timespec *ts);
-extern int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz);
+extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
+extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
extern time_t __vdso_time(time_t *t);
+extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
-#ifdef CONFIG_PARAVIRT_CLOCK
-extern u8 pvclock_page
- __attribute__((visibility("hidden")));
-#endif
-
-#ifdef CONFIG_HYPERV_TSCPAGE
-extern u8 hvclock_page
- __attribute__((visibility("hidden")));
-#endif
-
-#ifndef BUILD_VDSO32
-
-notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
-{
- long ret;
- asm ("syscall" : "=a" (ret), "=m" (*ts) :
- "0" (__NR_clock_gettime), "D" (clock), "S" (ts) :
- "rcx", "r11");
- return ret;
-}
-
-#else
-
-notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
-{
- long ret;
-
- asm (
- "mov %%ebx, %%edx \n"
- "mov %[clock], %%ebx \n"
- "call __kernel_vsyscall \n"
- "mov %%edx, %%ebx \n"
- : "=a" (ret), "=m" (*ts)
- : "0" (__NR_clock_gettime), [clock] "g" (clock), "c" (ts)
- : "edx");
- return ret;
-}
-
-#endif
-
-#ifdef CONFIG_PARAVIRT_CLOCK
-static notrace const struct pvclock_vsyscall_time_info *get_pvti0(void)
-{
- return (const struct pvclock_vsyscall_time_info *)&pvclock_page;
-}
-
-static notrace u64 vread_pvclock(void)
-{
- const struct pvclock_vcpu_time_info *pvti = &get_pvti0()->pvti;
- u32 version;
- u64 ret;
-
- /*
- * Note: The kernel and hypervisor must guarantee that cpu ID
- * number maps 1:1 to per-CPU pvclock time info.
- *
- * Because the hypervisor is entirely unaware of guest userspace
- * preemption, it cannot guarantee that per-CPU pvclock time
- * info is updated if the underlying CPU changes or that that
- * version is increased whenever underlying CPU changes.
- *
- * On KVM, we are guaranteed that pvti updates for any vCPU are
- * atomic as seen by *all* vCPUs. This is an even stronger
- * guarantee than we get with a normal seqlock.
- *
- * On Xen, we don't appear to have that guarantee, but Xen still
- * supplies a valid seqlock using the version field.
- *
- * We only do pvclock vdso timing at all if
- * PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
- * mean that all vCPUs have matching pvti and that the TSC is
- * synced, so we can just look at vCPU 0's pvti.
- */
-
- do {
- version = pvclock_read_begin(pvti);
-
- if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
- return U64_MAX;
-
- ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
- } while (pvclock_read_retry(pvti, version));
-
- return ret;
-}
-#endif
-#ifdef CONFIG_HYPERV_TSCPAGE
-static notrace u64 vread_hvclock(void)
-{
- const struct ms_hyperv_tsc_page *tsc_pg =
- (const struct ms_hyperv_tsc_page *)&hvclock_page;
-
- return hv_read_tsc_page(tsc_pg);
-}
-#endif
-
-notrace static inline u64 vgetcyc(int mode)
-{
- if (mode == VCLOCK_TSC)
- return (u64)rdtsc_ordered();
-#ifdef CONFIG_PARAVIRT_CLOCK
- else if (mode == VCLOCK_PVCLOCK)
- return vread_pvclock();
-#endif
-#ifdef CONFIG_HYPERV_TSCPAGE
- else if (mode == VCLOCK_HVCLOCK)
- return vread_hvclock();
-#endif
- return U64_MAX;
-}
-
-notrace static int do_hres(clockid_t clk, struct timespec *ts)
+notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
{
- struct vgtod_ts *base = >od->basetime[clk];
- u64 cycles, last, sec, ns;
- unsigned int seq;
-
- do {
- seq = gtod_read_begin(gtod);
- cycles = vgetcyc(gtod->vclock_mode);
- ns = base->nsec;
- last = gtod->cycle_last;
- if (unlikely((s64)cycles < 0))
- return vdso_fallback_gettime(clk, ts);
- if (cycles > last)
- ns += (cycles - last) * gtod->mult;
- ns >>= gtod->shift;
- sec = base->sec;
- } while (unlikely(gtod_read_retry(gtod, seq)));
-
- /*
- * Do this outside the loop: a race inside the loop could result
- * in __iter_div_u64_rem() being extremely slow.
- */
- ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
- ts->tv_nsec = ns;
-
- return 0;
-}
-
-notrace static void do_coarse(clockid_t clk, struct timespec *ts)
-{
- struct vgtod_ts *base = >od->basetime[clk];
- unsigned int seq;
-
- do {
- seq = gtod_read_begin(gtod);
- ts->tv_sec = base->sec;
- ts->tv_nsec = base->nsec;
- } while (unlikely(gtod_read_retry(gtod, seq)));
+ return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
-{
- unsigned int msk;
-
- /* Sort out negative (CPU/FD) and invalid clocks */
- if (unlikely((unsigned int) clock >= MAX_CLOCKS))
- return vdso_fallback_gettime(clock, ts);
-
- /*
- * Convert the clockid to a bitmask and use it to check which
- * clocks are handled in the VDSO directly.
- */
- msk = 1U << clock;
- if (likely(msk & VGTOD_HRES)) {
- return do_hres(clock, ts);
- } else if (msk & VGTOD_COARSE) {
- do_coarse(clock, ts);
- return 0;
- }
- return vdso_fallback_gettime(clock, ts);
-}
-
-int clock_gettime(clockid_t, struct timespec *)
+int clock_gettime(clockid_t, struct __vdso_timespec *)
__attribute__((weak, alias("__vdso_clock_gettime")));
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- if (likely(tv != NULL)) {
- struct timespec *ts = (struct timespec *) tv;
-
- do_hres(CLOCK_REALTIME, ts);
- tv->tv_usec /= 1000;
- }
- if (unlikely(tz != NULL)) {
- tz->tz_minuteswest = gtod->tz_minuteswest;
- tz->tz_dsttime = gtod->tz_dsttime;
- }
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-int gettimeofday(struct timeval *, struct timezone *)
+int gettimeofday(struct __vdso_timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
-/*
- * This will break when the xtime seconds get inaccurate, but that is
- * unlikely
- */
notrace time_t __vdso_time(time_t *t)
{
- /* This is atomic on x86 so we don't need any locks. */
- time_t result = READ_ONCE(gtod->basetime[CLOCK_REALTIME].sec);
-
- if (t)
- *t = result;
- return result;
+ return __cvdso_time(t);
}
time_t time(time_t *t)
__attribute__((weak, alias("__vdso_time")));
+
+notrace int __vdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock, res);
+}
+int clock_getres(clockid_t, struct __vdso_timespec *)
+ __attribute__((weak, alias("__vdso_clock_getres")));
diff --git a/arch/x86/entry/vdso/vdso.lds.S b/arch/x86/entry/vdso/vdso.lds.S
index d3a2dce4cfa9..36b644e16272 100644
--- a/arch/x86/entry/vdso/vdso.lds.S
+++ b/arch/x86/entry/vdso/vdso.lds.S
@@ -25,6 +25,8 @@ VERSION {
__vdso_getcpu;
time;
__vdso_time;
+ clock_getres;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/x86/entry/vdso/vdso32/vdso32.lds.S b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
index 422764a81d32..991b26cc855b 100644
--- a/arch/x86/entry/vdso/vdso32/vdso32.lds.S
+++ b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
@@ -26,6 +26,7 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_time;
+ __vdso_clock_getres;
};
LINUX_2.5 {
diff --git a/arch/x86/entry/vdso/vdsox32.lds.S b/arch/x86/entry/vdso/vdsox32.lds.S
index 05cd1c5c4a15..16a8050a4fb6 100644
--- a/arch/x86/entry/vdso/vdsox32.lds.S
+++ b/arch/x86/entry/vdso/vdsox32.lds.S
@@ -21,6 +21,7 @@ VERSION {
__vdso_gettimeofday;
__vdso_getcpu;
__vdso_time;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/x86/entry/vsyscall/vsyscall_gtod.c b/arch/x86/entry/vsyscall/vsyscall_gtod.c
index cfcdba082feb..f7b0978f3919 100644
--- a/arch/x86/entry/vsyscall/vsyscall_gtod.c
+++ b/arch/x86/entry/vsyscall/vsyscall_gtod.c
@@ -19,65 +19,97 @@
int vclocks_used __read_mostly;
-DEFINE_VVAR(struct vsyscall_gtod_data, vsyscall_gtod_data);
+DEFINE_VVAR(struct vdso_data, vdso_data);
void update_vsyscall_tz(void)
{
- vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
+ vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data.tz_dsttime = sys_tz.tz_dsttime;
}
void update_vsyscall(struct timekeeper *tk)
{
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
- struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
- struct vgtod_ts *base;
+ struct vdso_data *vdata = &vdso_data;
+ struct vdso_timestamp *vdso_ts;
u64 nsec;
/* Mark the new vclock used. */
BUILD_BUG_ON(VCLOCK_MAX >= 32);
WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
- gtod_write_begin(vdata);
+ vdso_write_begin(vdata);
/* copy vsyscall data */
- vdata->vclock_mode = vclock_mode;
+ vdata->clock_mode = vclock_mode;
vdata->cycle_last = tk->tkr_mono.cycle_last;
- vdata->mask = tk->tkr_mono.mask;
- vdata->mult = tk->tkr_mono.mult;
- vdata->shift = tk->tkr_mono.shift;
-
- base = &vdata->basetime[CLOCK_REALTIME];
- base->sec = tk->xtime_sec;
- base->nsec = tk->tkr_mono.xtime_nsec;
-
- base = &vdata->basetime[CLOCK_TAI];
- base->sec = tk->xtime_sec + (s64)tk->tai_offset;
- base->nsec = tk->tkr_mono.xtime_nsec;
-
- base = &vdata->basetime[CLOCK_MONOTONIC];
- base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec;
- nsec += ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift);
+ vdata->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdata->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdata->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdata->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdata->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdata->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdata->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
- nsec -= ((u64)NSEC_PER_SEC) << tk->tkr_mono.shift;
- base->sec++;
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
}
- base->nsec = nsec;
-
- base = &vdata->basetime[CLOCK_REALTIME_COARSE];
- base->sec = tk->xtime_sec;
- base->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
-
- base = &vdata->basetime[CLOCK_MONOTONIC_COARSE];
- base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
- nsec += tk->wall_to_monotonic.tv_nsec;
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdata->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdata->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdata->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
while (nsec >= NSEC_PER_SEC) {
- nsec -= NSEC_PER_SEC;
- base->sec++;
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
}
- base->nsec = nsec;
+ vdso_ts->nsec = nsec;
- gtod_write_end(vdata);
+ vdso_write_end(vdata);
}
diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..1a86fdc1e400
--- /dev/null
+++ b/arch/x86/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,203 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Fast user context implementation of clock_gettime, gettimeofday, and time.
+ *
+ * Copyright (C) 2019 ARM Limited.
+ * Copyright 2006 Andi Kleen, SUSE Labs.
+ * 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
+ * sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <uapi/linux/time.h>
+#include <asm/vgtod.h>
+#include <asm/vvar.h>
+#include <asm/unistd.h>
+#include <asm/msr.h>
+#include <asm/pvclock.h>
+#include <asm/mshyperv.h>
+
+#define _vdso_data (&VVAR(vdso_data))
+
+#define VDSO_HAS_TIME 1
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+extern u8 pvclock_page
+ __attribute__((visibility("hidden")));
+#endif
+
+#ifdef CONFIG_HYPERV_TSCPAGE
+extern u8 hvclock_page
+ __attribute__((visibility("hidden")));
+#endif
+
+#ifndef BUILD_VDSO32
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+ asm ("syscall" : "=a" (ret), "=m" (*_ts) :
+ "0" (__NR_clock_gettime), "D" (_clkid), "S" (_ts) :
+ "rcx", "r11");
+ return ret;
+}
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ long ret;
+ asm("syscall" : "=a" (ret) :
+ "0" (__NR_gettimeofday), "D" (_tv), "S" (_tz) : "memory");
+ return ret;
+}
+
+static __always_inline notrace long clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+ asm ("syscall" : "=a" (ret), "=m" (*_ts) :
+ "0" (__NR_clock_getres), "D" (_clkid), "S" (_ts) :
+ "rcx", "r11");
+ return ret;
+}
+
+#else
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+
+ asm (
+ "mov %%ebx, %%edx \n"
+ "mov %[clock], %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret), "=m" (*_ts)
+ : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
+ : "edx");
+ return ret;
+}
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ long ret;
+ asm(
+ "mov %%ebx, %%edx \n"
+ "mov %2, %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret)
+ : "0" (__NR_gettimeofday), "g" (_tv), "c" (_tz)
+ : "memory", "edx");
+ return ret;
+}
+
+static __always_inline notrace long clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+
+ asm (
+ "mov %%ebx, %%edx \n"
+ "mov %[clock], %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret), "=m" (*_ts)
+ : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
+ : "edx");
+ return ret;
+}
+
+#endif
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+static notrace const struct pvclock_vsyscall_time_info *get_pvti0(void)
+{
+ return (const struct pvclock_vsyscall_time_info *)&pvclock_page;
+}
+
+static notrace u64 vread_pvclock(void)
+{
+ const struct pvclock_vcpu_time_info *pvti = &get_pvti0()->pvti;
+ u32 version;
+ u64 ret;
+
+ /*
+ * Note: The kernel and hypervisor must guarantee that cpu ID
+ * number maps 1:1 to per-CPU pvclock time info.
+ *
+ * Because the hypervisor is entirely unaware of guest userspace
+ * preemption, it cannot guarantee that per-CPU pvclock time
+ * info is updated if the underlying CPU changes or that that
+ * version is increased whenever underlying CPU changes.
+ *
+ * On KVM, we are guaranteed that pvti updates for any vCPU are
+ * atomic as seen by *all* vCPUs. This is an even stronger
+ * guarantee than we get with a normal seqlock.
+ *
+ * On Xen, we don't appear to have that guarantee, but Xen still
+ * supplies a valid seqlock using the version field.
+ *
+ * We only do pvclock vdso timing at all if
+ * PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
+ * mean that all vCPUs have matching pvti and that the TSC is
+ * synced, so we can just look at vCPU 0's pvti.
+ */
+
+ do {
+ version = pvclock_read_begin(pvti);
+
+ if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
+ return U64_MAX;
+
+ ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
+ } while (pvclock_read_retry(pvti, version));
+
+ return ret;
+}
+#endif
+#ifdef CONFIG_HYPERV_TSCPAGE
+static notrace u64 vread_hvclock(void)
+{
+ const struct ms_hyperv_tsc_page *tsc_pg =
+ (const struct ms_hyperv_tsc_page *)&hvclock_page;
+
+ return hv_read_tsc_page(tsc_pg);
+}
+#endif
+
+notrace static inline u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ if (clock_mode == VCLOCK_TSC)
+ return (u64)rdtsc_ordered();
+#ifdef CONFIG_PARAVIRT_CLOCK
+ else if (clock_mode == VCLOCK_PVCLOCK)
+ return vread_pvclock();
+#endif
+#ifdef CONFIG_HYPERV_TSCPAGE
+ else if (clock_mode == VCLOCK_HVCLOCK)
+ return vread_hvclock();
+#endif
+ return U64_MAX;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return _vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/x86/include/asm/vgtod.h b/arch/x86/include/asm/vgtod.h
index 913a133f8e6f..0b4e343c62be 100644
--- a/arch/x86/include/asm/vgtod.h
+++ b/arch/x86/include/asm/vgtod.h
@@ -4,6 +4,8 @@
#include <linux/compiler.h>
#include <linux/clocksource.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <uapi/linux/time.h>
@@ -13,43 +15,7 @@ typedef u64 gtod_long_t;
typedef unsigned long gtod_long_t;
#endif
-/*
- * There is one of these objects in the vvar page for each
- * vDSO-accelerated clockid. For high-resolution clocks, this encodes
- * the time corresponding to vsyscall_gtod_data.cycle_last. For coarse
- * clocks, this encodes the actual time.
- *
- * To confuse the reader, for high-resolution clocks, nsec is left-shifted
- * by vsyscall_gtod_data.shift.
- */
-struct vgtod_ts {
- u64 sec;
- u64 nsec;
-};
-
-#define VGTOD_BASES (CLOCK_TAI + 1)
-#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))
-#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE))
-
-/*
- * vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
- * so be carefull by modifying this structure.
- */
-struct vsyscall_gtod_data {
- unsigned int seq;
-
- int vclock_mode;
- u64 cycle_last;
- u64 mask;
- u32 mult;
- u32 shift;
-
- struct vgtod_ts basetime[VGTOD_BASES];
-
- int tz_minuteswest;
- int tz_dsttime;
-};
-extern struct vsyscall_gtod_data vsyscall_gtod_data;
+extern struct vdso_data vdso_data;
extern int vclocks_used;
static inline bool vclock_was_used(int vclock)
@@ -57,37 +23,4 @@ static inline bool vclock_was_used(int vclock)
return READ_ONCE(vclocks_used) & (1 << vclock);
}
-static inline unsigned int gtod_read_begin(const struct vsyscall_gtod_data *s)
-{
- unsigned int ret;
-
-repeat:
- ret = READ_ONCE(s->seq);
- if (unlikely(ret & 1)) {
- cpu_relax();
- goto repeat;
- }
- smp_rmb();
- return ret;
-}
-
-static inline int gtod_read_retry(const struct vsyscall_gtod_data *s,
- unsigned int start)
-{
- smp_rmb();
- return unlikely(s->seq != start);
-}
-
-static inline void gtod_write_begin(struct vsyscall_gtod_data *s)
-{
- ++s->seq;
- smp_wmb();
-}
-
-static inline void gtod_write_end(struct vsyscall_gtod_data *s)
-{
- smp_wmb();
- ++s->seq;
-}
-
#endif /* _ASM_X86_VGTOD_H */
diff --git a/arch/x86/include/asm/vvar.h b/arch/x86/include/asm/vvar.h
index 3f32dfc2ab73..e5598cb4d3a6 100644
--- a/arch/x86/include/asm/vvar.h
+++ b/arch/x86/include/asm/vvar.h
@@ -44,7 +44,7 @@ extern char __vvar_page;
/* DECLARE_VVAR(offset, type, name) */
-DECLARE_VVAR(128, struct vsyscall_gtod_data, vsyscall_gtod_data)
+DECLARE_VVAR(128, struct vdso_data, vdso_data)
#undef DECLARE_VVAR
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
The x86 vDSO library requires some adaptations to take advantage of the
newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/x86/Kconfig | 3 +
arch/x86/entry/vdso/Makefile | 9 +
arch/x86/entry/vdso/vclock_gettime.c | 239 +++--------------------
arch/x86/entry/vdso/vdso.lds.S | 2 +
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
arch/x86/entry/vdso/vdsox32.lds.S | 1 +
arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 +++++++----
arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++++++++++++++
arch/x86/include/asm/vgtod.h | 73 +------
arch/x86/include/asm/vvar.h | 2 +-
10 files changed, 318 insertions(+), 325 deletions(-)
create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 68261430fe6e..9422cc122567 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -18,6 +18,7 @@ config X86_32
select HAVE_GENERIC_DMA_COHERENT
select MODULES_USE_ELF_REL
select OLD_SIGACTION
+ select GENERIC_VDSO_32
config X86_64
def_bool y
@@ -111,6 +112,7 @@ config X86
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HARDLOCKUP_CHECK_TIMESTAMP if X86_64
select HAVE_ACPI_APEI if ACPI
select HAVE_ACPI_APEI_NMI if ACPI
@@ -194,6 +196,7 @@ config X86
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
+ select HAVE_GENERIC_VDSO
select HOTPLUG_SMT if SMP
select IRQ_FORCED_THREADING
select NEED_SG_DMA_LENGTH
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 5bfe2243a08f..5dc3f251ca13 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -3,6 +3,12 @@
# Building vDSO images for x86.
#
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_X86_64_JUMP_SLOT|R_X86_64_GLOB_DAT|R_X86_64_RELATIVE|
+ARCH_REL_TYPE_ABS += R_386_GLOB_DAT|R_386_JMP_SLOT|R_386_RELATIVE
+include $(srctree)/lib/vdso/Makefile
+
KBUILD_CFLAGS += $(DISABLE_LTO)
KASAN_SANITIZE := n
UBSAN_SANITIZE := n
@@ -51,6 +57,7 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 --no-undefined \
$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi
hostprogs-y += vdso2c
@@ -121,6 +128,7 @@ $(obj)/%.so: $(obj)/%.so.dbg
$(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
CPPFLAGS_vdso32.lds = $(CPPFLAGS_vdso.lds)
VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1
@@ -160,6 +168,7 @@ $(obj)/vdso32.so.dbg: FORCE \
$(obj)/vdso32/system_call.o \
$(obj)/vdso32/sigreturn.o
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
#
# The DSO images are built using a special linker script.
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index 007b3fe9d727..6ff8cabf4c3e 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -1,240 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2006 Andi Kleen, SUSE Labs.
- * Subject to the GNU Public License, v.2
- *
* Fast user context implementation of clock_gettime, gettimeofday, and time.
*
+ * Copyright 2019 ARM Limited
+ * Copyright 2006 Andi Kleen, SUSE Labs.
* 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
* sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
- *
- * The code should have no internal unresolved relocations.
- * Check with readelf after changing.
*/
-
-#include <uapi/linux/time.h>
-#include <asm/vgtod.h>
-#include <asm/vvar.h>
-#include <asm/unistd.h>
-#include <asm/msr.h>
-#include <asm/pvclock.h>
-#include <asm/mshyperv.h>
-#include <linux/math64.h>
#include <linux/time.h>
-#include <linux/kernel.h>
+#include <linux/types.h>
-#define gtod (&VVAR(vsyscall_gtod_data))
+#include "../../../../lib/vdso/gettimeofday.c"
-extern int __vdso_clock_gettime(clockid_t clock, struct timespec *ts);
-extern int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz);
+extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
+extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
extern time_t __vdso_time(time_t *t);
+extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
-#ifdef CONFIG_PARAVIRT_CLOCK
-extern u8 pvclock_page
- __attribute__((visibility("hidden")));
-#endif
-
-#ifdef CONFIG_HYPERV_TSCPAGE
-extern u8 hvclock_page
- __attribute__((visibility("hidden")));
-#endif
-
-#ifndef BUILD_VDSO32
-
-notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
-{
- long ret;
- asm ("syscall" : "=a" (ret), "=m" (*ts) :
- "0" (__NR_clock_gettime), "D" (clock), "S" (ts) :
- "rcx", "r11");
- return ret;
-}
-
-#else
-
-notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
-{
- long ret;
-
- asm (
- "mov %%ebx, %%edx \n"
- "mov %[clock], %%ebx \n"
- "call __kernel_vsyscall \n"
- "mov %%edx, %%ebx \n"
- : "=a" (ret), "=m" (*ts)
- : "0" (__NR_clock_gettime), [clock] "g" (clock), "c" (ts)
- : "edx");
- return ret;
-}
-
-#endif
-
-#ifdef CONFIG_PARAVIRT_CLOCK
-static notrace const struct pvclock_vsyscall_time_info *get_pvti0(void)
-{
- return (const struct pvclock_vsyscall_time_info *)&pvclock_page;
-}
-
-static notrace u64 vread_pvclock(void)
-{
- const struct pvclock_vcpu_time_info *pvti = &get_pvti0()->pvti;
- u32 version;
- u64 ret;
-
- /*
- * Note: The kernel and hypervisor must guarantee that cpu ID
- * number maps 1:1 to per-CPU pvclock time info.
- *
- * Because the hypervisor is entirely unaware of guest userspace
- * preemption, it cannot guarantee that per-CPU pvclock time
- * info is updated if the underlying CPU changes or that that
- * version is increased whenever underlying CPU changes.
- *
- * On KVM, we are guaranteed that pvti updates for any vCPU are
- * atomic as seen by *all* vCPUs. This is an even stronger
- * guarantee than we get with a normal seqlock.
- *
- * On Xen, we don't appear to have that guarantee, but Xen still
- * supplies a valid seqlock using the version field.
- *
- * We only do pvclock vdso timing at all if
- * PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
- * mean that all vCPUs have matching pvti and that the TSC is
- * synced, so we can just look at vCPU 0's pvti.
- */
-
- do {
- version = pvclock_read_begin(pvti);
-
- if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
- return U64_MAX;
-
- ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
- } while (pvclock_read_retry(pvti, version));
-
- return ret;
-}
-#endif
-#ifdef CONFIG_HYPERV_TSCPAGE
-static notrace u64 vread_hvclock(void)
-{
- const struct ms_hyperv_tsc_page *tsc_pg =
- (const struct ms_hyperv_tsc_page *)&hvclock_page;
-
- return hv_read_tsc_page(tsc_pg);
-}
-#endif
-
-notrace static inline u64 vgetcyc(int mode)
-{
- if (mode == VCLOCK_TSC)
- return (u64)rdtsc_ordered();
-#ifdef CONFIG_PARAVIRT_CLOCK
- else if (mode == VCLOCK_PVCLOCK)
- return vread_pvclock();
-#endif
-#ifdef CONFIG_HYPERV_TSCPAGE
- else if (mode == VCLOCK_HVCLOCK)
- return vread_hvclock();
-#endif
- return U64_MAX;
-}
-
-notrace static int do_hres(clockid_t clk, struct timespec *ts)
+notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
{
- struct vgtod_ts *base = >od->basetime[clk];
- u64 cycles, last, sec, ns;
- unsigned int seq;
-
- do {
- seq = gtod_read_begin(gtod);
- cycles = vgetcyc(gtod->vclock_mode);
- ns = base->nsec;
- last = gtod->cycle_last;
- if (unlikely((s64)cycles < 0))
- return vdso_fallback_gettime(clk, ts);
- if (cycles > last)
- ns += (cycles - last) * gtod->mult;
- ns >>= gtod->shift;
- sec = base->sec;
- } while (unlikely(gtod_read_retry(gtod, seq)));
-
- /*
- * Do this outside the loop: a race inside the loop could result
- * in __iter_div_u64_rem() being extremely slow.
- */
- ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
- ts->tv_nsec = ns;
-
- return 0;
-}
-
-notrace static void do_coarse(clockid_t clk, struct timespec *ts)
-{
- struct vgtod_ts *base = >od->basetime[clk];
- unsigned int seq;
-
- do {
- seq = gtod_read_begin(gtod);
- ts->tv_sec = base->sec;
- ts->tv_nsec = base->nsec;
- } while (unlikely(gtod_read_retry(gtod, seq)));
+ return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
-{
- unsigned int msk;
-
- /* Sort out negative (CPU/FD) and invalid clocks */
- if (unlikely((unsigned int) clock >= MAX_CLOCKS))
- return vdso_fallback_gettime(clock, ts);
-
- /*
- * Convert the clockid to a bitmask and use it to check which
- * clocks are handled in the VDSO directly.
- */
- msk = 1U << clock;
- if (likely(msk & VGTOD_HRES)) {
- return do_hres(clock, ts);
- } else if (msk & VGTOD_COARSE) {
- do_coarse(clock, ts);
- return 0;
- }
- return vdso_fallback_gettime(clock, ts);
-}
-
-int clock_gettime(clockid_t, struct timespec *)
+int clock_gettime(clockid_t, struct __vdso_timespec *)
__attribute__((weak, alias("__vdso_clock_gettime")));
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- if (likely(tv != NULL)) {
- struct timespec *ts = (struct timespec *) tv;
-
- do_hres(CLOCK_REALTIME, ts);
- tv->tv_usec /= 1000;
- }
- if (unlikely(tz != NULL)) {
- tz->tz_minuteswest = gtod->tz_minuteswest;
- tz->tz_dsttime = gtod->tz_dsttime;
- }
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-int gettimeofday(struct timeval *, struct timezone *)
+int gettimeofday(struct __vdso_timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
-/*
- * This will break when the xtime seconds get inaccurate, but that is
- * unlikely
- */
notrace time_t __vdso_time(time_t *t)
{
- /* This is atomic on x86 so we don't need any locks. */
- time_t result = READ_ONCE(gtod->basetime[CLOCK_REALTIME].sec);
-
- if (t)
- *t = result;
- return result;
+ return __cvdso_time(t);
}
time_t time(time_t *t)
__attribute__((weak, alias("__vdso_time")));
+
+notrace int __vdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock, res);
+}
+int clock_getres(clockid_t, struct __vdso_timespec *)
+ __attribute__((weak, alias("__vdso_clock_getres")));
diff --git a/arch/x86/entry/vdso/vdso.lds.S b/arch/x86/entry/vdso/vdso.lds.S
index d3a2dce4cfa9..36b644e16272 100644
--- a/arch/x86/entry/vdso/vdso.lds.S
+++ b/arch/x86/entry/vdso/vdso.lds.S
@@ -25,6 +25,8 @@ VERSION {
__vdso_getcpu;
time;
__vdso_time;
+ clock_getres;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/x86/entry/vdso/vdso32/vdso32.lds.S b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
index 422764a81d32..991b26cc855b 100644
--- a/arch/x86/entry/vdso/vdso32/vdso32.lds.S
+++ b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
@@ -26,6 +26,7 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_time;
+ __vdso_clock_getres;
};
LINUX_2.5 {
diff --git a/arch/x86/entry/vdso/vdsox32.lds.S b/arch/x86/entry/vdso/vdsox32.lds.S
index 05cd1c5c4a15..16a8050a4fb6 100644
--- a/arch/x86/entry/vdso/vdsox32.lds.S
+++ b/arch/x86/entry/vdso/vdsox32.lds.S
@@ -21,6 +21,7 @@ VERSION {
__vdso_gettimeofday;
__vdso_getcpu;
__vdso_time;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/x86/entry/vsyscall/vsyscall_gtod.c b/arch/x86/entry/vsyscall/vsyscall_gtod.c
index cfcdba082feb..f7b0978f3919 100644
--- a/arch/x86/entry/vsyscall/vsyscall_gtod.c
+++ b/arch/x86/entry/vsyscall/vsyscall_gtod.c
@@ -19,65 +19,97 @@
int vclocks_used __read_mostly;
-DEFINE_VVAR(struct vsyscall_gtod_data, vsyscall_gtod_data);
+DEFINE_VVAR(struct vdso_data, vdso_data);
void update_vsyscall_tz(void)
{
- vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
+ vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data.tz_dsttime = sys_tz.tz_dsttime;
}
void update_vsyscall(struct timekeeper *tk)
{
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
- struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
- struct vgtod_ts *base;
+ struct vdso_data *vdata = &vdso_data;
+ struct vdso_timestamp *vdso_ts;
u64 nsec;
/* Mark the new vclock used. */
BUILD_BUG_ON(VCLOCK_MAX >= 32);
WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
- gtod_write_begin(vdata);
+ vdso_write_begin(vdata);
/* copy vsyscall data */
- vdata->vclock_mode = vclock_mode;
+ vdata->clock_mode = vclock_mode;
vdata->cycle_last = tk->tkr_mono.cycle_last;
- vdata->mask = tk->tkr_mono.mask;
- vdata->mult = tk->tkr_mono.mult;
- vdata->shift = tk->tkr_mono.shift;
-
- base = &vdata->basetime[CLOCK_REALTIME];
- base->sec = tk->xtime_sec;
- base->nsec = tk->tkr_mono.xtime_nsec;
-
- base = &vdata->basetime[CLOCK_TAI];
- base->sec = tk->xtime_sec + (s64)tk->tai_offset;
- base->nsec = tk->tkr_mono.xtime_nsec;
-
- base = &vdata->basetime[CLOCK_MONOTONIC];
- base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec;
- nsec += ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift);
+ vdata->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdata->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdata->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdata->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdata->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdata->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdata->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
- nsec -= ((u64)NSEC_PER_SEC) << tk->tkr_mono.shift;
- base->sec++;
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
}
- base->nsec = nsec;
-
- base = &vdata->basetime[CLOCK_REALTIME_COARSE];
- base->sec = tk->xtime_sec;
- base->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
-
- base = &vdata->basetime[CLOCK_MONOTONIC_COARSE];
- base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
- nsec += tk->wall_to_monotonic.tv_nsec;
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdata->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdata->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdata->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
while (nsec >= NSEC_PER_SEC) {
- nsec -= NSEC_PER_SEC;
- base->sec++;
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
}
- base->nsec = nsec;
+ vdso_ts->nsec = nsec;
- gtod_write_end(vdata);
+ vdso_write_end(vdata);
}
diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..1a86fdc1e400
--- /dev/null
+++ b/arch/x86/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,203 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Fast user context implementation of clock_gettime, gettimeofday, and time.
+ *
+ * Copyright (C) 2019 ARM Limited.
+ * Copyright 2006 Andi Kleen, SUSE Labs.
+ * 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
+ * sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <uapi/linux/time.h>
+#include <asm/vgtod.h>
+#include <asm/vvar.h>
+#include <asm/unistd.h>
+#include <asm/msr.h>
+#include <asm/pvclock.h>
+#include <asm/mshyperv.h>
+
+#define _vdso_data (&VVAR(vdso_data))
+
+#define VDSO_HAS_TIME 1
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+extern u8 pvclock_page
+ __attribute__((visibility("hidden")));
+#endif
+
+#ifdef CONFIG_HYPERV_TSCPAGE
+extern u8 hvclock_page
+ __attribute__((visibility("hidden")));
+#endif
+
+#ifndef BUILD_VDSO32
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+ asm ("syscall" : "=a" (ret), "=m" (*_ts) :
+ "0" (__NR_clock_gettime), "D" (_clkid), "S" (_ts) :
+ "rcx", "r11");
+ return ret;
+}
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ long ret;
+ asm("syscall" : "=a" (ret) :
+ "0" (__NR_gettimeofday), "D" (_tv), "S" (_tz) : "memory");
+ return ret;
+}
+
+static __always_inline notrace long clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+ asm ("syscall" : "=a" (ret), "=m" (*_ts) :
+ "0" (__NR_clock_getres), "D" (_clkid), "S" (_ts) :
+ "rcx", "r11");
+ return ret;
+}
+
+#else
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+
+ asm (
+ "mov %%ebx, %%edx \n"
+ "mov %[clock], %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret), "=m" (*_ts)
+ : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
+ : "edx");
+ return ret;
+}
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ long ret;
+ asm(
+ "mov %%ebx, %%edx \n"
+ "mov %2, %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret)
+ : "0" (__NR_gettimeofday), "g" (_tv), "c" (_tz)
+ : "memory", "edx");
+ return ret;
+}
+
+static __always_inline notrace long clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+
+ asm (
+ "mov %%ebx, %%edx \n"
+ "mov %[clock], %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret), "=m" (*_ts)
+ : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
+ : "edx");
+ return ret;
+}
+
+#endif
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+static notrace const struct pvclock_vsyscall_time_info *get_pvti0(void)
+{
+ return (const struct pvclock_vsyscall_time_info *)&pvclock_page;
+}
+
+static notrace u64 vread_pvclock(void)
+{
+ const struct pvclock_vcpu_time_info *pvti = &get_pvti0()->pvti;
+ u32 version;
+ u64 ret;
+
+ /*
+ * Note: The kernel and hypervisor must guarantee that cpu ID
+ * number maps 1:1 to per-CPU pvclock time info.
+ *
+ * Because the hypervisor is entirely unaware of guest userspace
+ * preemption, it cannot guarantee that per-CPU pvclock time
+ * info is updated if the underlying CPU changes or that that
+ * version is increased whenever underlying CPU changes.
+ *
+ * On KVM, we are guaranteed that pvti updates for any vCPU are
+ * atomic as seen by *all* vCPUs. This is an even stronger
+ * guarantee than we get with a normal seqlock.
+ *
+ * On Xen, we don't appear to have that guarantee, but Xen still
+ * supplies a valid seqlock using the version field.
+ *
+ * We only do pvclock vdso timing at all if
+ * PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
+ * mean that all vCPUs have matching pvti and that the TSC is
+ * synced, so we can just look at vCPU 0's pvti.
+ */
+
+ do {
+ version = pvclock_read_begin(pvti);
+
+ if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
+ return U64_MAX;
+
+ ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
+ } while (pvclock_read_retry(pvti, version));
+
+ return ret;
+}
+#endif
+#ifdef CONFIG_HYPERV_TSCPAGE
+static notrace u64 vread_hvclock(void)
+{
+ const struct ms_hyperv_tsc_page *tsc_pg =
+ (const struct ms_hyperv_tsc_page *)&hvclock_page;
+
+ return hv_read_tsc_page(tsc_pg);
+}
+#endif
+
+notrace static inline u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ if (clock_mode == VCLOCK_TSC)
+ return (u64)rdtsc_ordered();
+#ifdef CONFIG_PARAVIRT_CLOCK
+ else if (clock_mode == VCLOCK_PVCLOCK)
+ return vread_pvclock();
+#endif
+#ifdef CONFIG_HYPERV_TSCPAGE
+ else if (clock_mode == VCLOCK_HVCLOCK)
+ return vread_hvclock();
+#endif
+ return U64_MAX;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return _vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/x86/include/asm/vgtod.h b/arch/x86/include/asm/vgtod.h
index 913a133f8e6f..0b4e343c62be 100644
--- a/arch/x86/include/asm/vgtod.h
+++ b/arch/x86/include/asm/vgtod.h
@@ -4,6 +4,8 @@
#include <linux/compiler.h>
#include <linux/clocksource.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <uapi/linux/time.h>
@@ -13,43 +15,7 @@ typedef u64 gtod_long_t;
typedef unsigned long gtod_long_t;
#endif
-/*
- * There is one of these objects in the vvar page for each
- * vDSO-accelerated clockid. For high-resolution clocks, this encodes
- * the time corresponding to vsyscall_gtod_data.cycle_last. For coarse
- * clocks, this encodes the actual time.
- *
- * To confuse the reader, for high-resolution clocks, nsec is left-shifted
- * by vsyscall_gtod_data.shift.
- */
-struct vgtod_ts {
- u64 sec;
- u64 nsec;
-};
-
-#define VGTOD_BASES (CLOCK_TAI + 1)
-#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))
-#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE))
-
-/*
- * vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
- * so be carefull by modifying this structure.
- */
-struct vsyscall_gtod_data {
- unsigned int seq;
-
- int vclock_mode;
- u64 cycle_last;
- u64 mask;
- u32 mult;
- u32 shift;
-
- struct vgtod_ts basetime[VGTOD_BASES];
-
- int tz_minuteswest;
- int tz_dsttime;
-};
-extern struct vsyscall_gtod_data vsyscall_gtod_data;
+extern struct vdso_data vdso_data;
extern int vclocks_used;
static inline bool vclock_was_used(int vclock)
@@ -57,37 +23,4 @@ static inline bool vclock_was_used(int vclock)
return READ_ONCE(vclocks_used) & (1 << vclock);
}
-static inline unsigned int gtod_read_begin(const struct vsyscall_gtod_data *s)
-{
- unsigned int ret;
-
-repeat:
- ret = READ_ONCE(s->seq);
- if (unlikely(ret & 1)) {
- cpu_relax();
- goto repeat;
- }
- smp_rmb();
- return ret;
-}
-
-static inline int gtod_read_retry(const struct vsyscall_gtod_data *s,
- unsigned int start)
-{
- smp_rmb();
- return unlikely(s->seq != start);
-}
-
-static inline void gtod_write_begin(struct vsyscall_gtod_data *s)
-{
- ++s->seq;
- smp_wmb();
-}
-
-static inline void gtod_write_end(struct vsyscall_gtod_data *s)
-{
- smp_wmb();
- ++s->seq;
-}
-
#endif /* _ASM_X86_VGTOD_H */
diff --git a/arch/x86/include/asm/vvar.h b/arch/x86/include/asm/vvar.h
index 3f32dfc2ab73..e5598cb4d3a6 100644
--- a/arch/x86/include/asm/vvar.h
+++ b/arch/x86/include/asm/vvar.h
@@ -44,7 +44,7 @@ extern char __vvar_page;
/* DECLARE_VVAR(offset, type, name) */
-DECLARE_VVAR(128, struct vsyscall_gtod_data, vsyscall_gtod_data)
+DECLARE_VVAR(128, struct vdso_data, vdso_data)
#undef DECLARE_VVAR
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The x86 vDSO library requires some adaptations to take advantage of the
newly introduced generic vDSO library.
Introduce the following changes:
- Modification of vdso.c to be compliant with the common vdso datapage
- Use of lib/vdso for gettimeofday
Cc: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
arch/x86/Kconfig | 3 +
arch/x86/entry/vdso/Makefile | 9 +
arch/x86/entry/vdso/vclock_gettime.c | 239 +++--------------------
arch/x86/entry/vdso/vdso.lds.S | 2 +
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
arch/x86/entry/vdso/vdsox32.lds.S | 1 +
arch/x86/entry/vsyscall/vsyscall_gtod.c | 110 +++++++----
arch/x86/include/asm/vdso/gettimeofday.h | 203 +++++++++++++++++++
arch/x86/include/asm/vgtod.h | 73 +------
arch/x86/include/asm/vvar.h | 2 +-
10 files changed, 318 insertions(+), 325 deletions(-)
create mode 100644 arch/x86/include/asm/vdso/gettimeofday.h
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 68261430fe6e..9422cc122567 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -18,6 +18,7 @@ config X86_32
select HAVE_GENERIC_DMA_COHERENT
select MODULES_USE_ELF_REL
select OLD_SIGACTION
+ select GENERIC_VDSO_32
config X86_64
def_bool y
@@ -111,6 +112,7 @@ config X86
select GENERIC_STRNCPY_FROM_USER
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
+ select GENERIC_GETTIMEOFDAY
select HARDLOCKUP_CHECK_TIMESTAMP if X86_64
select HAVE_ACPI_APEI if ACPI
select HAVE_ACPI_APEI_NMI if ACPI
@@ -194,6 +196,7 @@ config X86
select HAVE_SYSCALL_TRACEPOINTS
select HAVE_UNSTABLE_SCHED_CLOCK
select HAVE_USER_RETURN_NOTIFIER
+ select HAVE_GENERIC_VDSO
select HOTPLUG_SMT if SMP
select IRQ_FORCED_THREADING
select NEED_SG_DMA_LENGTH
diff --git a/arch/x86/entry/vdso/Makefile b/arch/x86/entry/vdso/Makefile
index 5bfe2243a08f..5dc3f251ca13 100644
--- a/arch/x86/entry/vdso/Makefile
+++ b/arch/x86/entry/vdso/Makefile
@@ -3,6 +3,12 @@
# Building vDSO images for x86.
#
+# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
+# the inclusion of generic Makefile.
+ARCH_REL_TYPE_ABS := R_X86_64_JUMP_SLOT|R_X86_64_GLOB_DAT|R_X86_64_RELATIVE|
+ARCH_REL_TYPE_ABS += R_386_GLOB_DAT|R_386_JMP_SLOT|R_386_RELATIVE
+include $(srctree)/lib/vdso/Makefile
+
KBUILD_CFLAGS += $(DISABLE_LTO)
KASAN_SANITIZE := n
UBSAN_SANITIZE := n
@@ -51,6 +57,7 @@ VDSO_LDFLAGS_vdso.lds = -m elf_x86_64 -soname linux-vdso.so.1 --no-undefined \
$(obj)/vdso64.so.dbg: $(obj)/vdso.lds $(vobjs) FORCE
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi -I$(srctree)/arch/$(SUBARCH)/include/uapi
hostprogs-y += vdso2c
@@ -121,6 +128,7 @@ $(obj)/%.so: $(obj)/%.so.dbg
$(obj)/vdsox32.so.dbg: $(obj)/vdsox32.lds $(vobjx32s) FORCE
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
CPPFLAGS_vdso32.lds = $(CPPFLAGS_vdso.lds)
VDSO_LDFLAGS_vdso32.lds = -m elf_i386 -soname linux-gate.so.1
@@ -160,6 +168,7 @@ $(obj)/vdso32.so.dbg: FORCE \
$(obj)/vdso32/system_call.o \
$(obj)/vdso32/sigreturn.o
$(call if_changed,vdso)
+ $(call if_changed,vdso_check)
#
# The DSO images are built using a special linker script.
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index 007b3fe9d727..6ff8cabf4c3e 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -1,240 +1,49 @@
+// SPDX-License-Identifier: GPL-2.0
/*
- * Copyright 2006 Andi Kleen, SUSE Labs.
- * Subject to the GNU Public License, v.2
- *
* Fast user context implementation of clock_gettime, gettimeofday, and time.
*
+ * Copyright 2019 ARM Limited
+ * Copyright 2006 Andi Kleen, SUSE Labs.
* 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
* sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
- *
- * The code should have no internal unresolved relocations.
- * Check with readelf after changing.
*/
-
-#include <uapi/linux/time.h>
-#include <asm/vgtod.h>
-#include <asm/vvar.h>
-#include <asm/unistd.h>
-#include <asm/msr.h>
-#include <asm/pvclock.h>
-#include <asm/mshyperv.h>
-#include <linux/math64.h>
#include <linux/time.h>
-#include <linux/kernel.h>
+#include <linux/types.h>
-#define gtod (&VVAR(vsyscall_gtod_data))
+#include "../../../../lib/vdso/gettimeofday.c"
-extern int __vdso_clock_gettime(clockid_t clock, struct timespec *ts);
-extern int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz);
+extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
+extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
extern time_t __vdso_time(time_t *t);
+extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
-#ifdef CONFIG_PARAVIRT_CLOCK
-extern u8 pvclock_page
- __attribute__((visibility("hidden")));
-#endif
-
-#ifdef CONFIG_HYPERV_TSCPAGE
-extern u8 hvclock_page
- __attribute__((visibility("hidden")));
-#endif
-
-#ifndef BUILD_VDSO32
-
-notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
-{
- long ret;
- asm ("syscall" : "=a" (ret), "=m" (*ts) :
- "0" (__NR_clock_gettime), "D" (clock), "S" (ts) :
- "rcx", "r11");
- return ret;
-}
-
-#else
-
-notrace static long vdso_fallback_gettime(long clock, struct timespec *ts)
-{
- long ret;
-
- asm (
- "mov %%ebx, %%edx \n"
- "mov %[clock], %%ebx \n"
- "call __kernel_vsyscall \n"
- "mov %%edx, %%ebx \n"
- : "=a" (ret), "=m" (*ts)
- : "0" (__NR_clock_gettime), [clock] "g" (clock), "c" (ts)
- : "edx");
- return ret;
-}
-
-#endif
-
-#ifdef CONFIG_PARAVIRT_CLOCK
-static notrace const struct pvclock_vsyscall_time_info *get_pvti0(void)
-{
- return (const struct pvclock_vsyscall_time_info *)&pvclock_page;
-}
-
-static notrace u64 vread_pvclock(void)
-{
- const struct pvclock_vcpu_time_info *pvti = &get_pvti0()->pvti;
- u32 version;
- u64 ret;
-
- /*
- * Note: The kernel and hypervisor must guarantee that cpu ID
- * number maps 1:1 to per-CPU pvclock time info.
- *
- * Because the hypervisor is entirely unaware of guest userspace
- * preemption, it cannot guarantee that per-CPU pvclock time
- * info is updated if the underlying CPU changes or that that
- * version is increased whenever underlying CPU changes.
- *
- * On KVM, we are guaranteed that pvti updates for any vCPU are
- * atomic as seen by *all* vCPUs. This is an even stronger
- * guarantee than we get with a normal seqlock.
- *
- * On Xen, we don't appear to have that guarantee, but Xen still
- * supplies a valid seqlock using the version field.
- *
- * We only do pvclock vdso timing at all if
- * PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
- * mean that all vCPUs have matching pvti and that the TSC is
- * synced, so we can just look at vCPU 0's pvti.
- */
-
- do {
- version = pvclock_read_begin(pvti);
-
- if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
- return U64_MAX;
-
- ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
- } while (pvclock_read_retry(pvti, version));
-
- return ret;
-}
-#endif
-#ifdef CONFIG_HYPERV_TSCPAGE
-static notrace u64 vread_hvclock(void)
-{
- const struct ms_hyperv_tsc_page *tsc_pg =
- (const struct ms_hyperv_tsc_page *)&hvclock_page;
-
- return hv_read_tsc_page(tsc_pg);
-}
-#endif
-
-notrace static inline u64 vgetcyc(int mode)
-{
- if (mode == VCLOCK_TSC)
- return (u64)rdtsc_ordered();
-#ifdef CONFIG_PARAVIRT_CLOCK
- else if (mode == VCLOCK_PVCLOCK)
- return vread_pvclock();
-#endif
-#ifdef CONFIG_HYPERV_TSCPAGE
- else if (mode == VCLOCK_HVCLOCK)
- return vread_hvclock();
-#endif
- return U64_MAX;
-}
-
-notrace static int do_hres(clockid_t clk, struct timespec *ts)
+notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
{
- struct vgtod_ts *base = >od->basetime[clk];
- u64 cycles, last, sec, ns;
- unsigned int seq;
-
- do {
- seq = gtod_read_begin(gtod);
- cycles = vgetcyc(gtod->vclock_mode);
- ns = base->nsec;
- last = gtod->cycle_last;
- if (unlikely((s64)cycles < 0))
- return vdso_fallback_gettime(clk, ts);
- if (cycles > last)
- ns += (cycles - last) * gtod->mult;
- ns >>= gtod->shift;
- sec = base->sec;
- } while (unlikely(gtod_read_retry(gtod, seq)));
-
- /*
- * Do this outside the loop: a race inside the loop could result
- * in __iter_div_u64_rem() being extremely slow.
- */
- ts->tv_sec = sec + __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
- ts->tv_nsec = ns;
-
- return 0;
-}
-
-notrace static void do_coarse(clockid_t clk, struct timespec *ts)
-{
- struct vgtod_ts *base = >od->basetime[clk];
- unsigned int seq;
-
- do {
- seq = gtod_read_begin(gtod);
- ts->tv_sec = base->sec;
- ts->tv_nsec = base->nsec;
- } while (unlikely(gtod_read_retry(gtod, seq)));
+ return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_clock_gettime(clockid_t clock, struct timespec *ts)
-{
- unsigned int msk;
-
- /* Sort out negative (CPU/FD) and invalid clocks */
- if (unlikely((unsigned int) clock >= MAX_CLOCKS))
- return vdso_fallback_gettime(clock, ts);
-
- /*
- * Convert the clockid to a bitmask and use it to check which
- * clocks are handled in the VDSO directly.
- */
- msk = 1U << clock;
- if (likely(msk & VGTOD_HRES)) {
- return do_hres(clock, ts);
- } else if (msk & VGTOD_COARSE) {
- do_coarse(clock, ts);
- return 0;
- }
- return vdso_fallback_gettime(clock, ts);
-}
-
-int clock_gettime(clockid_t, struct timespec *)
+int clock_gettime(clockid_t, struct __vdso_timespec *)
__attribute__((weak, alias("__vdso_clock_gettime")));
-notrace int __vdso_gettimeofday(struct timeval *tv, struct timezone *tz)
+notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+ struct timezone *tz)
{
- if (likely(tv != NULL)) {
- struct timespec *ts = (struct timespec *) tv;
-
- do_hres(CLOCK_REALTIME, ts);
- tv->tv_usec /= 1000;
- }
- if (unlikely(tz != NULL)) {
- tz->tz_minuteswest = gtod->tz_minuteswest;
- tz->tz_dsttime = gtod->tz_dsttime;
- }
-
- return 0;
+ return __cvdso_gettimeofday(tv, tz);
}
-int gettimeofday(struct timeval *, struct timezone *)
+int gettimeofday(struct __vdso_timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
-/*
- * This will break when the xtime seconds get inaccurate, but that is
- * unlikely
- */
notrace time_t __vdso_time(time_t *t)
{
- /* This is atomic on x86 so we don't need any locks. */
- time_t result = READ_ONCE(gtod->basetime[CLOCK_REALTIME].sec);
-
- if (t)
- *t = result;
- return result;
+ return __cvdso_time(t);
}
time_t time(time_t *t)
__attribute__((weak, alias("__vdso_time")));
+
+notrace int __vdso_clock_getres(clockid_t clock,
+ struct __vdso_timespec *res)
+{
+ return __cvdso_clock_getres(clock, res);
+}
+int clock_getres(clockid_t, struct __vdso_timespec *)
+ __attribute__((weak, alias("__vdso_clock_getres")));
diff --git a/arch/x86/entry/vdso/vdso.lds.S b/arch/x86/entry/vdso/vdso.lds.S
index d3a2dce4cfa9..36b644e16272 100644
--- a/arch/x86/entry/vdso/vdso.lds.S
+++ b/arch/x86/entry/vdso/vdso.lds.S
@@ -25,6 +25,8 @@ VERSION {
__vdso_getcpu;
time;
__vdso_time;
+ clock_getres;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/x86/entry/vdso/vdso32/vdso32.lds.S b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
index 422764a81d32..991b26cc855b 100644
--- a/arch/x86/entry/vdso/vdso32/vdso32.lds.S
+++ b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
@@ -26,6 +26,7 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_time;
+ __vdso_clock_getres;
};
LINUX_2.5 {
diff --git a/arch/x86/entry/vdso/vdsox32.lds.S b/arch/x86/entry/vdso/vdsox32.lds.S
index 05cd1c5c4a15..16a8050a4fb6 100644
--- a/arch/x86/entry/vdso/vdsox32.lds.S
+++ b/arch/x86/entry/vdso/vdsox32.lds.S
@@ -21,6 +21,7 @@ VERSION {
__vdso_gettimeofday;
__vdso_getcpu;
__vdso_time;
+ __vdso_clock_getres;
local: *;
};
}
diff --git a/arch/x86/entry/vsyscall/vsyscall_gtod.c b/arch/x86/entry/vsyscall/vsyscall_gtod.c
index cfcdba082feb..f7b0978f3919 100644
--- a/arch/x86/entry/vsyscall/vsyscall_gtod.c
+++ b/arch/x86/entry/vsyscall/vsyscall_gtod.c
@@ -19,65 +19,97 @@
int vclocks_used __read_mostly;
-DEFINE_VVAR(struct vsyscall_gtod_data, vsyscall_gtod_data);
+DEFINE_VVAR(struct vdso_data, vdso_data);
void update_vsyscall_tz(void)
{
- vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
- vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
+ vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
+ vdso_data.tz_dsttime = sys_tz.tz_dsttime;
}
void update_vsyscall(struct timekeeper *tk)
{
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
- struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
- struct vgtod_ts *base;
+ struct vdso_data *vdata = &vdso_data;
+ struct vdso_timestamp *vdso_ts;
u64 nsec;
/* Mark the new vclock used. */
BUILD_BUG_ON(VCLOCK_MAX >= 32);
WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
- gtod_write_begin(vdata);
+ vdso_write_begin(vdata);
/* copy vsyscall data */
- vdata->vclock_mode = vclock_mode;
+ vdata->clock_mode = vclock_mode;
vdata->cycle_last = tk->tkr_mono.cycle_last;
- vdata->mask = tk->tkr_mono.mask;
- vdata->mult = tk->tkr_mono.mult;
- vdata->shift = tk->tkr_mono.shift;
-
- base = &vdata->basetime[CLOCK_REALTIME];
- base->sec = tk->xtime_sec;
- base->nsec = tk->tkr_mono.xtime_nsec;
-
- base = &vdata->basetime[CLOCK_TAI];
- base->sec = tk->xtime_sec + (s64)tk->tai_offset;
- base->nsec = tk->tkr_mono.xtime_nsec;
-
- base = &vdata->basetime[CLOCK_MONOTONIC];
- base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec;
- nsec += ((u64)tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift);
+ vdata->cs[CLOCKSOURCE_MONO].mask
+ = tk->tkr_mono.mask;
+ vdata->cs[CLOCKSOURCE_MONO].mult
+ = tk->tkr_mono.mult;
+ vdata->cs[CLOCKSOURCE_MONO].shift
+ = tk->tkr_mono.shift;
+ vdata->cs[CLOCKSOURCE_RAW].mask
+ = tk->tkr_raw.mask;
+ vdata->cs[CLOCKSOURCE_RAW].mult
+ = tk->tkr_raw.mult;
+ vdata->cs[CLOCKSOURCE_RAW].shift
+ = tk->tkr_raw.shift;
+ /* CLOCK_REALTIME */
+ vdso_ts = &vdata->basetime[CLOCK_REALTIME];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_MONOTONIC */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)tk->wall_to_monotonic.tv_nsec <<
+ tk->tkr_mono.shift);
while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
- nsec -= ((u64)NSEC_PER_SEC) << tk->tkr_mono.shift;
- base->sec++;
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
}
- base->nsec = nsec;
-
- base = &vdata->basetime[CLOCK_REALTIME_COARSE];
- base->sec = tk->xtime_sec;
- base->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
-
- base = &vdata->basetime[CLOCK_MONOTONIC_COARSE];
- base->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
- nsec += tk->wall_to_monotonic.tv_nsec;
+ vdso_ts->nsec = nsec;
+ /* CLOCK_MONOTONIC_RAW */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC_RAW];
+ vdso_ts->sec = tk->raw_sec;
+ vdso_ts->nsec = tk->tkr_raw.xtime_nsec;
+ /* CLOCK_BOOTTIME */
+ vdso_ts = &vdata->basetime[CLOCK_BOOTTIME];
+ vdso_ts->sec = tk->xtime_sec +
+ tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec;
+ nsec = nsec +
+ ((u64)(tk->wall_to_monotonic.tv_nsec +
+ ktime_to_ns(tk->offs_boot)) <<
+ tk->tkr_mono.shift);
+ while (nsec >= (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
+ nsec = nsec -
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift);
+ vdso_ts->sec++;
+ }
+ vdso_ts->nsec = nsec;
+ /* CLOCK_TAI */
+ vdso_ts = &vdata->basetime[CLOCK_TAI];
+ vdso_ts->sec = tk->xtime_sec + (s64)tk->tai_offset;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec;
+ /* CLOCK_REALTIME_COARSE */
+ vdso_ts = &vdata->basetime[CLOCK_REALTIME_COARSE];
+ vdso_ts->sec = tk->xtime_sec;
+ vdso_ts->nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ /* CLOCK_MONOTONIC_COARSE */
+ vdso_ts = &vdata->basetime[CLOCK_MONOTONIC_COARSE];
+ vdso_ts->sec = tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
+ nsec = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
+ nsec = nsec + tk->wall_to_monotonic.tv_nsec;
while (nsec >= NSEC_PER_SEC) {
- nsec -= NSEC_PER_SEC;
- base->sec++;
+ nsec = nsec - NSEC_PER_SEC;
+ vdso_ts->sec++;
}
- base->nsec = nsec;
+ vdso_ts->nsec = nsec;
- gtod_write_end(vdata);
+ vdso_write_end(vdata);
}
diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
new file mode 100644
index 000000000000..1a86fdc1e400
--- /dev/null
+++ b/arch/x86/include/asm/vdso/gettimeofday.h
@@ -0,0 +1,203 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Fast user context implementation of clock_gettime, gettimeofday, and time.
+ *
+ * Copyright (C) 2019 ARM Limited.
+ * Copyright 2006 Andi Kleen, SUSE Labs.
+ * 32 Bit compat layer by Stefani Seibold <stefani@seibold.net>
+ * sponsored by Rohde & Schwarz GmbH & Co. KG Munich/Germany
+ */
+#ifndef __ASM_VDSO_GETTIMEOFDAY_H
+#define __ASM_VDSO_GETTIMEOFDAY_H
+
+#ifndef __ASSEMBLY__
+
+#include <uapi/linux/time.h>
+#include <asm/vgtod.h>
+#include <asm/vvar.h>
+#include <asm/unistd.h>
+#include <asm/msr.h>
+#include <asm/pvclock.h>
+#include <asm/mshyperv.h>
+
+#define _vdso_data (&VVAR(vdso_data))
+
+#define VDSO_HAS_TIME 1
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+extern u8 pvclock_page
+ __attribute__((visibility("hidden")));
+#endif
+
+#ifdef CONFIG_HYPERV_TSCPAGE
+extern u8 hvclock_page
+ __attribute__((visibility("hidden")));
+#endif
+
+#ifndef BUILD_VDSO32
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+ asm ("syscall" : "=a" (ret), "=m" (*_ts) :
+ "0" (__NR_clock_gettime), "D" (_clkid), "S" (_ts) :
+ "rcx", "r11");
+ return ret;
+}
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ long ret;
+ asm("syscall" : "=a" (ret) :
+ "0" (__NR_gettimeofday), "D" (_tv), "S" (_tz) : "memory");
+ return ret;
+}
+
+static __always_inline notrace long clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+ asm ("syscall" : "=a" (ret), "=m" (*_ts) :
+ "0" (__NR_clock_getres), "D" (_clkid), "S" (_ts) :
+ "rcx", "r11");
+ return ret;
+}
+
+#else
+
+static __always_inline notrace long clock_gettime_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+
+ asm (
+ "mov %%ebx, %%edx \n"
+ "mov %[clock], %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret), "=m" (*_ts)
+ : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
+ : "edx");
+ return ret;
+}
+
+static __always_inline notrace long gettimeofday_fallback(
+ struct __vdso_timeval *_tv,
+ struct timezone *_tz)
+{
+ long ret;
+ asm(
+ "mov %%ebx, %%edx \n"
+ "mov %2, %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret)
+ : "0" (__NR_gettimeofday), "g" (_tv), "c" (_tz)
+ : "memory", "edx");
+ return ret;
+}
+
+static __always_inline notrace long clock_getres_fallback(
+ clockid_t _clkid,
+ struct __vdso_timespec *_ts)
+{
+ long ret;
+
+ asm (
+ "mov %%ebx, %%edx \n"
+ "mov %[clock], %%ebx \n"
+ "call __kernel_vsyscall \n"
+ "mov %%edx, %%ebx \n"
+ : "=a" (ret), "=m" (*_ts)
+ : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
+ : "edx");
+ return ret;
+}
+
+#endif
+
+#ifdef CONFIG_PARAVIRT_CLOCK
+static notrace const struct pvclock_vsyscall_time_info *get_pvti0(void)
+{
+ return (const struct pvclock_vsyscall_time_info *)&pvclock_page;
+}
+
+static notrace u64 vread_pvclock(void)
+{
+ const struct pvclock_vcpu_time_info *pvti = &get_pvti0()->pvti;
+ u32 version;
+ u64 ret;
+
+ /*
+ * Note: The kernel and hypervisor must guarantee that cpu ID
+ * number maps 1:1 to per-CPU pvclock time info.
+ *
+ * Because the hypervisor is entirely unaware of guest userspace
+ * preemption, it cannot guarantee that per-CPU pvclock time
+ * info is updated if the underlying CPU changes or that that
+ * version is increased whenever underlying CPU changes.
+ *
+ * On KVM, we are guaranteed that pvti updates for any vCPU are
+ * atomic as seen by *all* vCPUs. This is an even stronger
+ * guarantee than we get with a normal seqlock.
+ *
+ * On Xen, we don't appear to have that guarantee, but Xen still
+ * supplies a valid seqlock using the version field.
+ *
+ * We only do pvclock vdso timing at all if
+ * PVCLOCK_TSC_STABLE_BIT is set, and we interpret that bit to
+ * mean that all vCPUs have matching pvti and that the TSC is
+ * synced, so we can just look at vCPU 0's pvti.
+ */
+
+ do {
+ version = pvclock_read_begin(pvti);
+
+ if (unlikely(!(pvti->flags & PVCLOCK_TSC_STABLE_BIT)))
+ return U64_MAX;
+
+ ret = __pvclock_read_cycles(pvti, rdtsc_ordered());
+ } while (pvclock_read_retry(pvti, version));
+
+ return ret;
+}
+#endif
+#ifdef CONFIG_HYPERV_TSCPAGE
+static notrace u64 vread_hvclock(void)
+{
+ const struct ms_hyperv_tsc_page *tsc_pg =
+ (const struct ms_hyperv_tsc_page *)&hvclock_page;
+
+ return hv_read_tsc_page(tsc_pg);
+}
+#endif
+
+notrace static inline u64 __arch_get_hw_counter(s32 clock_mode)
+{
+ if (clock_mode == VCLOCK_TSC)
+ return (u64)rdtsc_ordered();
+#ifdef CONFIG_PARAVIRT_CLOCK
+ else if (clock_mode == VCLOCK_PVCLOCK)
+ return vread_pvclock();
+#endif
+#ifdef CONFIG_HYPERV_TSCPAGE
+ else if (clock_mode == VCLOCK_HVCLOCK)
+ return vread_hvclock();
+#endif
+ return U64_MAX;
+}
+
+static __always_inline notrace const struct vdso_data *__arch_get_vdso_data(void)
+{
+ return _vdso_data;
+}
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_GETTIMEOFDAY_H */
diff --git a/arch/x86/include/asm/vgtod.h b/arch/x86/include/asm/vgtod.h
index 913a133f8e6f..0b4e343c62be 100644
--- a/arch/x86/include/asm/vgtod.h
+++ b/arch/x86/include/asm/vgtod.h
@@ -4,6 +4,8 @@
#include <linux/compiler.h>
#include <linux/clocksource.h>
+#include <vdso/datapage.h>
+#include <vdso/helpers.h>
#include <uapi/linux/time.h>
@@ -13,43 +15,7 @@ typedef u64 gtod_long_t;
typedef unsigned long gtod_long_t;
#endif
-/*
- * There is one of these objects in the vvar page for each
- * vDSO-accelerated clockid. For high-resolution clocks, this encodes
- * the time corresponding to vsyscall_gtod_data.cycle_last. For coarse
- * clocks, this encodes the actual time.
- *
- * To confuse the reader, for high-resolution clocks, nsec is left-shifted
- * by vsyscall_gtod_data.shift.
- */
-struct vgtod_ts {
- u64 sec;
- u64 nsec;
-};
-
-#define VGTOD_BASES (CLOCK_TAI + 1)
-#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))
-#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE))
-
-/*
- * vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
- * so be carefull by modifying this structure.
- */
-struct vsyscall_gtod_data {
- unsigned int seq;
-
- int vclock_mode;
- u64 cycle_last;
- u64 mask;
- u32 mult;
- u32 shift;
-
- struct vgtod_ts basetime[VGTOD_BASES];
-
- int tz_minuteswest;
- int tz_dsttime;
-};
-extern struct vsyscall_gtod_data vsyscall_gtod_data;
+extern struct vdso_data vdso_data;
extern int vclocks_used;
static inline bool vclock_was_used(int vclock)
@@ -57,37 +23,4 @@ static inline bool vclock_was_used(int vclock)
return READ_ONCE(vclocks_used) & (1 << vclock);
}
-static inline unsigned int gtod_read_begin(const struct vsyscall_gtod_data *s)
-{
- unsigned int ret;
-
-repeat:
- ret = READ_ONCE(s->seq);
- if (unlikely(ret & 1)) {
- cpu_relax();
- goto repeat;
- }
- smp_rmb();
- return ret;
-}
-
-static inline int gtod_read_retry(const struct vsyscall_gtod_data *s,
- unsigned int start)
-{
- smp_rmb();
- return unlikely(s->seq != start);
-}
-
-static inline void gtod_write_begin(struct vsyscall_gtod_data *s)
-{
- ++s->seq;
- smp_wmb();
-}
-
-static inline void gtod_write_end(struct vsyscall_gtod_data *s)
-{
- smp_wmb();
- ++s->seq;
-}
-
#endif /* _ASM_X86_VGTOD_H */
diff --git a/arch/x86/include/asm/vvar.h b/arch/x86/include/asm/vvar.h
index 3f32dfc2ab73..e5598cb4d3a6 100644
--- a/arch/x86/include/asm/vvar.h
+++ b/arch/x86/include/asm/vvar.h
@@ -44,7 +44,7 @@ extern char __vvar_page;
/* DECLARE_VVAR(offset, type, name) */
-DECLARE_VVAR(128, struct vsyscall_gtod_data, vsyscall_gtod_data)
+DECLARE_VVAR(128, struct vdso_data, vdso_data)
#undef DECLARE_VVAR
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 23/23] kselftest: Extend vDSO selftest
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The current version of the multiarch vDSO selftest verifies only
gettimeofday.
Extend the vDSO selftest to the other library functions:
- time
- clock_getres
- clock_gettime
The extension has been used to verify the unified vdso library on the
supported architectures.
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
tools/testing/selftests/vDSO/Makefile | 2 +
tools/testing/selftests/vDSO/vdso_full_test.c | 246 ++++++++++++++++++
2 files changed, 248 insertions(+)
create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c
diff --git a/tools/testing/selftests/vDSO/Makefile b/tools/testing/selftests/vDSO/Makefile
index 9e03d61f52fd..68e9b4a1cdcf 100644
--- a/tools/testing/selftests/vDSO/Makefile
+++ b/tools/testing/selftests/vDSO/Makefile
@@ -5,6 +5,7 @@ uname_M := $(shell uname -m 2>/dev/null || echo not)
ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/x86/ -e s/x86_64/x86/)
TEST_GEN_PROGS := $(OUTPUT)/vdso_test
+TEST_GEN_PROGS += $(OUTPUT)/vdso_full_test
ifeq ($(ARCH),x86)
TEST_GEN_PROGS += $(OUTPUT)/vdso_standalone_test_x86
endif
@@ -18,6 +19,7 @@ endif
all: $(TEST_GEN_PROGS)
$(OUTPUT)/vdso_test: parse_vdso.c vdso_test.c
+$(OUTPUT)/vdso_full_test: parse_vdso.c vdso_full_test.c
$(OUTPUT)/vdso_standalone_test_x86: vdso_standalone_test_x86.c parse_vdso.c
$(CC) $(CFLAGS) $(CFLAGS_vdso_standalone_test_x86) \
vdso_standalone_test_x86.c parse_vdso.c \
diff --git a/tools/testing/selftests/vDSO/vdso_full_test.c b/tools/testing/selftests/vDSO/vdso_full_test.c
new file mode 100644
index 000000000000..d46776658aa2
--- /dev/null
+++ b/tools/testing/selftests/vDSO/vdso_full_test.c
@@ -0,0 +1,246 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * vdso_full_test.c: Sample code to test all the timers.
+ * Copyright (c) 2019 Arm Ltd.
+ *
+ * Compile with:
+ * gcc -std=gnu99 vdso_full_test.c parse_vdso.c
+ *
+ * Tested on ARM, ARM64, MIPS32 and x86 (32-bit and 64-bit).
+ * Might work on other architectures.
+ */
+
+#include <stdint.h>
+#include <elf.h>
+#include <stdio.h>
+#include <time.h>
+#include <sys/auxv.h>
+#include <sys/time.h>
+#define _GNU_SOURCE
+#include <unistd.h>
+#include <sys/syscall.h>
+
+#include "../kselftest.h"
+
+extern void *vdso_sym(const char *version, const char *name);
+extern void vdso_init_from_sysinfo_ehdr(uintptr_t base);
+extern void vdso_init_from_auxv(void *auxv);
+
+/*
+ * ARM64's vDSO exports its vDSO implementation with different names and
+ * a different version from other architectures, so we need to handle it
+ * as a special case.
+ */
+#if defined(__aarch64__)
+const char *version = "LINUX_2.6.39";
+const char *name[4] = {
+ "__kernel_gettimeofday",
+ "__kernel_clock_gettime",
+ "__kernel_time",
+ "__kernel_clock_getres",
+};
+#else
+/* Tested on x86, arm, mips */
+const char *version = "LINUX_2.6";
+const char *name[4] = {
+ "__vdso_gettimeofday",
+ "__vdso_clock_gettime",
+ "__vdso_time",
+ "__vdso_clock_getres",
+};
+#endif
+
+typedef long (*vdso_gettimeofday_t)(struct timeval *tv, struct timezone *tz);
+typedef long (*vdso_clock_gettime_t)(clockid_t clk_id, struct timespec *ts);
+typedef long (*vdso_clock_getres_t)(clockid_t clk_id, struct timespec *ts);
+typedef time_t (*vdso_time_t)(time_t *t);
+
+static int vdso_test_gettimeofday(void)
+{
+ /* Find gettimeofday. */
+ vdso_gettimeofday_t vdso_gettimeofday =
+ (vdso_gettimeofday_t)vdso_sym(version, name[0]);
+
+ if (!vdso_gettimeofday) {
+ printf("Could not find %s\n", name[0]);
+ return KSFT_SKIP;
+ }
+
+ struct timeval tv;
+ long ret = vdso_gettimeofday(&tv, 0);
+
+ if (ret == 0) {
+ printf("The time is %lld.%06lld\n",
+ (long long)tv.tv_sec, (long long)tv.tv_usec);
+ } else {
+ printf("%s failed\n", name[0]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_clock_gettime(clockid_t clk_id)
+{
+ /* Find clock_gettime. */
+ vdso_clock_gettime_t vdso_clock_gettime =
+ (vdso_clock_gettime_t)vdso_sym(version, name[1]);
+
+ if (!vdso_clock_gettime) {
+ printf("Could not find %s\n", name[1]);
+ return KSFT_SKIP;
+ }
+
+ struct timespec ts;
+ long ret = vdso_clock_gettime(clk_id, &ts);
+
+ if (ret == 0) {
+ printf("The time is %lld.%06lld\n",
+ (long long)ts.tv_sec, (long long)ts.tv_nsec);
+ } else {
+ printf("%s failed\n", name[1]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_time(void)
+{
+ /* Find time. */
+ vdso_time_t vdso_time =
+ (vdso_time_t)vdso_sym(version, name[2]);
+
+ if (!vdso_time) {
+ printf("Could not find %s\n", name[2]);
+ return KSFT_SKIP;
+ }
+
+ long ret = vdso_time(NULL);
+
+ if (ret > 0) {
+ printf("The time in hours since January 1, 1970 is %lld\n",
+ (long long)(ret / 3600));
+ } else {
+ printf("%s failed\n", name[2]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_clock_getres(clockid_t clk_id)
+{
+ /* Find clock_getres. */
+ vdso_clock_getres_t vdso_clock_getres =
+ (vdso_clock_getres_t)vdso_sym(version, name[3]);
+
+ if (!vdso_clock_getres) {
+ printf("Could not find %s\n", name[3]);
+ return KSFT_SKIP;
+ }
+
+ struct timespec ts, sys_ts;
+ long ret = vdso_clock_getres(clk_id, &ts);
+
+ if (ret == 0) {
+ printf("The resolution is %lld %lld\n",
+ (long long)ts.tv_sec, (long long)ts.tv_nsec);
+ } else {
+ printf("%s failed\n", name[3]);
+ return KSFT_FAIL;
+ }
+
+ ret = syscall(SYS_clock_getres, clk_id, &sys_ts);
+
+ if ((sys_ts.tv_sec != ts.tv_sec) || (sys_ts.tv_nsec != ts.tv_nsec)) {
+ printf("%s failed\n", name[3]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+const char *vdso_clock_name[12] = {
+ "CLOCK_REALTIME",
+ "CLOCK_MONOTONIC",
+ "CLOCK_PROCESS_CPUTIME_ID",
+ "CLOCK_THREAD_CPUTIME_ID",
+ "CLOCK_MONOTONIC_RAW",
+ "CLOCK_REALTIME_COARSE",
+ "CLOCK_MONOTONIC_COARSE",
+ "CLOCK_BOOTTIME",
+ "CLOCK_REALTIME_ALARM",
+ "CLOCK_BOOTTIME_ALARM",
+ "CLOCK_SGI_CYCLE",
+ "CLOCK_TAI",
+};
+
+/*
+ * Macro to call vdso_test_clock_gettime and vdso_test_clock_getres
+ * with different values for clock_id.
+ */
+#define vdso_test_clock(clock_id) \
+do { \
+ printf("clock_id: %s\n", vdso_clock_name[clock_id]); \
+ vdso_test_clock_gettime(clock_id); \
+ vdso_test_clock_getres(clock_id); \
+} while (0)
+
+int main(int argc, char **argv)
+{
+ unsigned long sysinfo_ehdr = getauxval(AT_SYSINFO_EHDR);
+
+ if (!sysinfo_ehdr) {
+ printf("AT_SYSINFO_EHDR is not present!\n");
+ return KSFT_SKIP;
+ }
+
+ vdso_init_from_sysinfo_ehdr(getauxval(AT_SYSINFO_EHDR));
+
+ vdso_test_gettimeofday();
+
+#if _POSIX_TIMERS > 0
+
+#ifdef CLOCK_REALTIME
+ vdso_test_clock(CLOCK_REALTIME);
+#endif
+
+#ifdef CLOCK_REALTIME_HR
+ vdso_test_clock(CLOCK_REALTIME_HR);
+#endif
+
+#ifdef CLOCK_BOOTTIME
+ vdso_test_clock(CLOCK_BOOTTIME);
+#endif
+
+#ifdef CLOCK_TAI
+ vdso_test_clock(CLOCK_TAI);
+#endif
+
+#ifdef CLOCK_REALTIME_COARSE
+ vdso_test_clock(CLOCK_REALTIME_COARSE);
+#endif
+
+#ifdef CLOCK_REALTIME_HR
+ vdso_test_clock(CLOCK_REALTIME_HR);
+#endif
+
+#ifdef CLOCK_MONOTONIC
+ vdso_test_clock(CLOCK_MONOTONIC);
+#endif
+
+#ifdef CLOCK_MONOTONIC_RAW
+ vdso_test_clock(CLOCK_MONOTONIC_RAW);
+#endif
+
+#ifdef CLOCK_MONOTONIC_COARSE
+ vdso_test_clock(CLOCK_MONOTONIC_COARSE);
+#endif
+
+#endif
+
+ vdso_test_time();
+
+ return 0;
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 23/23] kselftest: Extend vDSO selftest
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Arnd Bergmann, Russell King,
Ralf Baechle, Paul Burton, Daniel Lezcano, Thomas Gleixner,
Mark Salyzyn, Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
The current version of the multiarch vDSO selftest verifies only
gettimeofday.
Extend the vDSO selftest to the other library functions:
- time
- clock_getres
- clock_gettime
The extension has been used to verify the unified vdso library on the
supported architectures.
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
tools/testing/selftests/vDSO/Makefile | 2 +
tools/testing/selftests/vDSO/vdso_full_test.c | 246 ++++++++++++++++++
2 files changed, 248 insertions(+)
create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c
diff --git a/tools/testing/selftests/vDSO/Makefile b/tools/testing/selftests/vDSO/Makefile
index 9e03d61f52fd..68e9b4a1cdcf 100644
--- a/tools/testing/selftests/vDSO/Makefile
+++ b/tools/testing/selftests/vDSO/Makefile
@@ -5,6 +5,7 @@ uname_M := $(shell uname -m 2>/dev/null || echo not)
ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/x86/ -e s/x86_64/x86/)
TEST_GEN_PROGS := $(OUTPUT)/vdso_test
+TEST_GEN_PROGS += $(OUTPUT)/vdso_full_test
ifeq ($(ARCH),x86)
TEST_GEN_PROGS += $(OUTPUT)/vdso_standalone_test_x86
endif
@@ -18,6 +19,7 @@ endif
all: $(TEST_GEN_PROGS)
$(OUTPUT)/vdso_test: parse_vdso.c vdso_test.c
+$(OUTPUT)/vdso_full_test: parse_vdso.c vdso_full_test.c
$(OUTPUT)/vdso_standalone_test_x86: vdso_standalone_test_x86.c parse_vdso.c
$(CC) $(CFLAGS) $(CFLAGS_vdso_standalone_test_x86) \
vdso_standalone_test_x86.c parse_vdso.c \
diff --git a/tools/testing/selftests/vDSO/vdso_full_test.c b/tools/testing/selftests/vDSO/vdso_full_test.c
new file mode 100644
index 000000000000..d46776658aa2
--- /dev/null
+++ b/tools/testing/selftests/vDSO/vdso_full_test.c
@@ -0,0 +1,246 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * vdso_full_test.c: Sample code to test all the timers.
+ * Copyright (c) 2019 Arm Ltd.
+ *
+ * Compile with:
+ * gcc -std=gnu99 vdso_full_test.c parse_vdso.c
+ *
+ * Tested on ARM, ARM64, MIPS32 and x86 (32-bit and 64-bit).
+ * Might work on other architectures.
+ */
+
+#include <stdint.h>
+#include <elf.h>
+#include <stdio.h>
+#include <time.h>
+#include <sys/auxv.h>
+#include <sys/time.h>
+#define _GNU_SOURCE
+#include <unistd.h>
+#include <sys/syscall.h>
+
+#include "../kselftest.h"
+
+extern void *vdso_sym(const char *version, const char *name);
+extern void vdso_init_from_sysinfo_ehdr(uintptr_t base);
+extern void vdso_init_from_auxv(void *auxv);
+
+/*
+ * ARM64's vDSO exports its vDSO implementation with different names and
+ * a different version from other architectures, so we need to handle it
+ * as a special case.
+ */
+#if defined(__aarch64__)
+const char *version = "LINUX_2.6.39";
+const char *name[4] = {
+ "__kernel_gettimeofday",
+ "__kernel_clock_gettime",
+ "__kernel_time",
+ "__kernel_clock_getres",
+};
+#else
+/* Tested on x86, arm, mips */
+const char *version = "LINUX_2.6";
+const char *name[4] = {
+ "__vdso_gettimeofday",
+ "__vdso_clock_gettime",
+ "__vdso_time",
+ "__vdso_clock_getres",
+};
+#endif
+
+typedef long (*vdso_gettimeofday_t)(struct timeval *tv, struct timezone *tz);
+typedef long (*vdso_clock_gettime_t)(clockid_t clk_id, struct timespec *ts);
+typedef long (*vdso_clock_getres_t)(clockid_t clk_id, struct timespec *ts);
+typedef time_t (*vdso_time_t)(time_t *t);
+
+static int vdso_test_gettimeofday(void)
+{
+ /* Find gettimeofday. */
+ vdso_gettimeofday_t vdso_gettimeofday =
+ (vdso_gettimeofday_t)vdso_sym(version, name[0]);
+
+ if (!vdso_gettimeofday) {
+ printf("Could not find %s\n", name[0]);
+ return KSFT_SKIP;
+ }
+
+ struct timeval tv;
+ long ret = vdso_gettimeofday(&tv, 0);
+
+ if (ret == 0) {
+ printf("The time is %lld.%06lld\n",
+ (long long)tv.tv_sec, (long long)tv.tv_usec);
+ } else {
+ printf("%s failed\n", name[0]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_clock_gettime(clockid_t clk_id)
+{
+ /* Find clock_gettime. */
+ vdso_clock_gettime_t vdso_clock_gettime =
+ (vdso_clock_gettime_t)vdso_sym(version, name[1]);
+
+ if (!vdso_clock_gettime) {
+ printf("Could not find %s\n", name[1]);
+ return KSFT_SKIP;
+ }
+
+ struct timespec ts;
+ long ret = vdso_clock_gettime(clk_id, &ts);
+
+ if (ret == 0) {
+ printf("The time is %lld.%06lld\n",
+ (long long)ts.tv_sec, (long long)ts.tv_nsec);
+ } else {
+ printf("%s failed\n", name[1]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_time(void)
+{
+ /* Find time. */
+ vdso_time_t vdso_time =
+ (vdso_time_t)vdso_sym(version, name[2]);
+
+ if (!vdso_time) {
+ printf("Could not find %s\n", name[2]);
+ return KSFT_SKIP;
+ }
+
+ long ret = vdso_time(NULL);
+
+ if (ret > 0) {
+ printf("The time in hours since January 1, 1970 is %lld\n",
+ (long long)(ret / 3600));
+ } else {
+ printf("%s failed\n", name[2]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_clock_getres(clockid_t clk_id)
+{
+ /* Find clock_getres. */
+ vdso_clock_getres_t vdso_clock_getres =
+ (vdso_clock_getres_t)vdso_sym(version, name[3]);
+
+ if (!vdso_clock_getres) {
+ printf("Could not find %s\n", name[3]);
+ return KSFT_SKIP;
+ }
+
+ struct timespec ts, sys_ts;
+ long ret = vdso_clock_getres(clk_id, &ts);
+
+ if (ret == 0) {
+ printf("The resolution is %lld %lld\n",
+ (long long)ts.tv_sec, (long long)ts.tv_nsec);
+ } else {
+ printf("%s failed\n", name[3]);
+ return KSFT_FAIL;
+ }
+
+ ret = syscall(SYS_clock_getres, clk_id, &sys_ts);
+
+ if ((sys_ts.tv_sec != ts.tv_sec) || (sys_ts.tv_nsec != ts.tv_nsec)) {
+ printf("%s failed\n", name[3]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+const char *vdso_clock_name[12] = {
+ "CLOCK_REALTIME",
+ "CLOCK_MONOTONIC",
+ "CLOCK_PROCESS_CPUTIME_ID",
+ "CLOCK_THREAD_CPUTIME_ID",
+ "CLOCK_MONOTONIC_RAW",
+ "CLOCK_REALTIME_COARSE",
+ "CLOCK_MONOTONIC_COARSE",
+ "CLOCK_BOOTTIME",
+ "CLOCK_REALTIME_ALARM",
+ "CLOCK_BOOTTIME_ALARM",
+ "CLOCK_SGI_CYCLE",
+ "CLOCK_TAI",
+};
+
+/*
+ * Macro to call vdso_test_clock_gettime and vdso_test_clock_getres
+ * with different values for clock_id.
+ */
+#define vdso_test_clock(clock_id) \
+do { \
+ printf("clock_id: %s\n", vdso_clock_name[clock_id]); \
+ vdso_test_clock_gettime(clock_id); \
+ vdso_test_clock_getres(clock_id); \
+} while (0)
+
+int main(int argc, char **argv)
+{
+ unsigned long sysinfo_ehdr = getauxval(AT_SYSINFO_EHDR);
+
+ if (!sysinfo_ehdr) {
+ printf("AT_SYSINFO_EHDR is not present!\n");
+ return KSFT_SKIP;
+ }
+
+ vdso_init_from_sysinfo_ehdr(getauxval(AT_SYSINFO_EHDR));
+
+ vdso_test_gettimeofday();
+
+#if _POSIX_TIMERS > 0
+
+#ifdef CLOCK_REALTIME
+ vdso_test_clock(CLOCK_REALTIME);
+#endif
+
+#ifdef CLOCK_REALTIME_HR
+ vdso_test_clock(CLOCK_REALTIME_HR);
+#endif
+
+#ifdef CLOCK_BOOTTIME
+ vdso_test_clock(CLOCK_BOOTTIME);
+#endif
+
+#ifdef CLOCK_TAI
+ vdso_test_clock(CLOCK_TAI);
+#endif
+
+#ifdef CLOCK_REALTIME_COARSE
+ vdso_test_clock(CLOCK_REALTIME_COARSE);
+#endif
+
+#ifdef CLOCK_REALTIME_HR
+ vdso_test_clock(CLOCK_REALTIME_HR);
+#endif
+
+#ifdef CLOCK_MONOTONIC
+ vdso_test_clock(CLOCK_MONOTONIC);
+#endif
+
+#ifdef CLOCK_MONOTONIC_RAW
+ vdso_test_clock(CLOCK_MONOTONIC_RAW);
+#endif
+
+#ifdef CLOCK_MONOTONIC_COARSE
+ vdso_test_clock(CLOCK_MONOTONIC_COARSE);
+#endif
+
+#endif
+
+ vdso_test_time();
+
+ return 0;
+}
--
2.20.1
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH v5 23/23] kselftest: Extend vDSO selftest
@ 2019-02-22 12:24 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-22 12:24 UTC (permalink / raw)
To: linux-arch, linux-arm-kernel
Cc: Shuah Khan, Arnd Bergmann, Catalin Marinas, Daniel Lezcano,
Will Deacon, Russell King, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Peter Collingbourne
The current version of the multiarch vDSO selftest verifies only
gettimeofday.
Extend the vDSO selftest to the other library functions:
- time
- clock_getres
- clock_gettime
The extension has been used to verify the unified vdso library on the
supported architectures.
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
---
tools/testing/selftests/vDSO/Makefile | 2 +
tools/testing/selftests/vDSO/vdso_full_test.c | 246 ++++++++++++++++++
2 files changed, 248 insertions(+)
create mode 100644 tools/testing/selftests/vDSO/vdso_full_test.c
diff --git a/tools/testing/selftests/vDSO/Makefile b/tools/testing/selftests/vDSO/Makefile
index 9e03d61f52fd..68e9b4a1cdcf 100644
--- a/tools/testing/selftests/vDSO/Makefile
+++ b/tools/testing/selftests/vDSO/Makefile
@@ -5,6 +5,7 @@ uname_M := $(shell uname -m 2>/dev/null || echo not)
ARCH ?= $(shell echo $(uname_M) | sed -e s/i.86/x86/ -e s/x86_64/x86/)
TEST_GEN_PROGS := $(OUTPUT)/vdso_test
+TEST_GEN_PROGS += $(OUTPUT)/vdso_full_test
ifeq ($(ARCH),x86)
TEST_GEN_PROGS += $(OUTPUT)/vdso_standalone_test_x86
endif
@@ -18,6 +19,7 @@ endif
all: $(TEST_GEN_PROGS)
$(OUTPUT)/vdso_test: parse_vdso.c vdso_test.c
+$(OUTPUT)/vdso_full_test: parse_vdso.c vdso_full_test.c
$(OUTPUT)/vdso_standalone_test_x86: vdso_standalone_test_x86.c parse_vdso.c
$(CC) $(CFLAGS) $(CFLAGS_vdso_standalone_test_x86) \
vdso_standalone_test_x86.c parse_vdso.c \
diff --git a/tools/testing/selftests/vDSO/vdso_full_test.c b/tools/testing/selftests/vDSO/vdso_full_test.c
new file mode 100644
index 000000000000..d46776658aa2
--- /dev/null
+++ b/tools/testing/selftests/vDSO/vdso_full_test.c
@@ -0,0 +1,246 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * vdso_full_test.c: Sample code to test all the timers.
+ * Copyright (c) 2019 Arm Ltd.
+ *
+ * Compile with:
+ * gcc -std=gnu99 vdso_full_test.c parse_vdso.c
+ *
+ * Tested on ARM, ARM64, MIPS32 and x86 (32-bit and 64-bit).
+ * Might work on other architectures.
+ */
+
+#include <stdint.h>
+#include <elf.h>
+#include <stdio.h>
+#include <time.h>
+#include <sys/auxv.h>
+#include <sys/time.h>
+#define _GNU_SOURCE
+#include <unistd.h>
+#include <sys/syscall.h>
+
+#include "../kselftest.h"
+
+extern void *vdso_sym(const char *version, const char *name);
+extern void vdso_init_from_sysinfo_ehdr(uintptr_t base);
+extern void vdso_init_from_auxv(void *auxv);
+
+/*
+ * ARM64's vDSO exports its vDSO implementation with different names and
+ * a different version from other architectures, so we need to handle it
+ * as a special case.
+ */
+#if defined(__aarch64__)
+const char *version = "LINUX_2.6.39";
+const char *name[4] = {
+ "__kernel_gettimeofday",
+ "__kernel_clock_gettime",
+ "__kernel_time",
+ "__kernel_clock_getres",
+};
+#else
+/* Tested on x86, arm, mips */
+const char *version = "LINUX_2.6";
+const char *name[4] = {
+ "__vdso_gettimeofday",
+ "__vdso_clock_gettime",
+ "__vdso_time",
+ "__vdso_clock_getres",
+};
+#endif
+
+typedef long (*vdso_gettimeofday_t)(struct timeval *tv, struct timezone *tz);
+typedef long (*vdso_clock_gettime_t)(clockid_t clk_id, struct timespec *ts);
+typedef long (*vdso_clock_getres_t)(clockid_t clk_id, struct timespec *ts);
+typedef time_t (*vdso_time_t)(time_t *t);
+
+static int vdso_test_gettimeofday(void)
+{
+ /* Find gettimeofday. */
+ vdso_gettimeofday_t vdso_gettimeofday =
+ (vdso_gettimeofday_t)vdso_sym(version, name[0]);
+
+ if (!vdso_gettimeofday) {
+ printf("Could not find %s\n", name[0]);
+ return KSFT_SKIP;
+ }
+
+ struct timeval tv;
+ long ret = vdso_gettimeofday(&tv, 0);
+
+ if (ret == 0) {
+ printf("The time is %lld.%06lld\n",
+ (long long)tv.tv_sec, (long long)tv.tv_usec);
+ } else {
+ printf("%s failed\n", name[0]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_clock_gettime(clockid_t clk_id)
+{
+ /* Find clock_gettime. */
+ vdso_clock_gettime_t vdso_clock_gettime =
+ (vdso_clock_gettime_t)vdso_sym(version, name[1]);
+
+ if (!vdso_clock_gettime) {
+ printf("Could not find %s\n", name[1]);
+ return KSFT_SKIP;
+ }
+
+ struct timespec ts;
+ long ret = vdso_clock_gettime(clk_id, &ts);
+
+ if (ret == 0) {
+ printf("The time is %lld.%06lld\n",
+ (long long)ts.tv_sec, (long long)ts.tv_nsec);
+ } else {
+ printf("%s failed\n", name[1]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_time(void)
+{
+ /* Find time. */
+ vdso_time_t vdso_time =
+ (vdso_time_t)vdso_sym(version, name[2]);
+
+ if (!vdso_time) {
+ printf("Could not find %s\n", name[2]);
+ return KSFT_SKIP;
+ }
+
+ long ret = vdso_time(NULL);
+
+ if (ret > 0) {
+ printf("The time in hours since January 1, 1970 is %lld\n",
+ (long long)(ret / 3600));
+ } else {
+ printf("%s failed\n", name[2]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+static int vdso_test_clock_getres(clockid_t clk_id)
+{
+ /* Find clock_getres. */
+ vdso_clock_getres_t vdso_clock_getres =
+ (vdso_clock_getres_t)vdso_sym(version, name[3]);
+
+ if (!vdso_clock_getres) {
+ printf("Could not find %s\n", name[3]);
+ return KSFT_SKIP;
+ }
+
+ struct timespec ts, sys_ts;
+ long ret = vdso_clock_getres(clk_id, &ts);
+
+ if (ret == 0) {
+ printf("The resolution is %lld %lld\n",
+ (long long)ts.tv_sec, (long long)ts.tv_nsec);
+ } else {
+ printf("%s failed\n", name[3]);
+ return KSFT_FAIL;
+ }
+
+ ret = syscall(SYS_clock_getres, clk_id, &sys_ts);
+
+ if ((sys_ts.tv_sec != ts.tv_sec) || (sys_ts.tv_nsec != ts.tv_nsec)) {
+ printf("%s failed\n", name[3]);
+ return KSFT_FAIL;
+ }
+
+ return 0;
+}
+
+const char *vdso_clock_name[12] = {
+ "CLOCK_REALTIME",
+ "CLOCK_MONOTONIC",
+ "CLOCK_PROCESS_CPUTIME_ID",
+ "CLOCK_THREAD_CPUTIME_ID",
+ "CLOCK_MONOTONIC_RAW",
+ "CLOCK_REALTIME_COARSE",
+ "CLOCK_MONOTONIC_COARSE",
+ "CLOCK_BOOTTIME",
+ "CLOCK_REALTIME_ALARM",
+ "CLOCK_BOOTTIME_ALARM",
+ "CLOCK_SGI_CYCLE",
+ "CLOCK_TAI",
+};
+
+/*
+ * Macro to call vdso_test_clock_gettime and vdso_test_clock_getres
+ * with different values for clock_id.
+ */
+#define vdso_test_clock(clock_id) \
+do { \
+ printf("clock_id: %s\n", vdso_clock_name[clock_id]); \
+ vdso_test_clock_gettime(clock_id); \
+ vdso_test_clock_getres(clock_id); \
+} while (0)
+
+int main(int argc, char **argv)
+{
+ unsigned long sysinfo_ehdr = getauxval(AT_SYSINFO_EHDR);
+
+ if (!sysinfo_ehdr) {
+ printf("AT_SYSINFO_EHDR is not present!\n");
+ return KSFT_SKIP;
+ }
+
+ vdso_init_from_sysinfo_ehdr(getauxval(AT_SYSINFO_EHDR));
+
+ vdso_test_gettimeofday();
+
+#if _POSIX_TIMERS > 0
+
+#ifdef CLOCK_REALTIME
+ vdso_test_clock(CLOCK_REALTIME);
+#endif
+
+#ifdef CLOCK_REALTIME_HR
+ vdso_test_clock(CLOCK_REALTIME_HR);
+#endif
+
+#ifdef CLOCK_BOOTTIME
+ vdso_test_clock(CLOCK_BOOTTIME);
+#endif
+
+#ifdef CLOCK_TAI
+ vdso_test_clock(CLOCK_TAI);
+#endif
+
+#ifdef CLOCK_REALTIME_COARSE
+ vdso_test_clock(CLOCK_REALTIME_COARSE);
+#endif
+
+#ifdef CLOCK_REALTIME_HR
+ vdso_test_clock(CLOCK_REALTIME_HR);
+#endif
+
+#ifdef CLOCK_MONOTONIC
+ vdso_test_clock(CLOCK_MONOTONIC);
+#endif
+
+#ifdef CLOCK_MONOTONIC_RAW
+ vdso_test_clock(CLOCK_MONOTONIC_RAW);
+#endif
+
+#ifdef CLOCK_MONOTONIC_COARSE
+ vdso_test_clock(CLOCK_MONOTONIC_COARSE);
+#endif
+
+#endif
+
+ vdso_test_time();
+
+ return 0;
+}
--
2.20.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-22 12:58 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 12:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Hi Vincenzo,
On Fri, Feb 22, 2019 at 12:24:08PM +0000, Vincenzo Frascino wrote:
> +#ifdef __KERNEL__
I believe that __KERNEL__ guards haven't been necessary (in non-UAPI
headers) since the UAPI split, and can be removed.
[...]
> +#include <linux/bits.h>
> +#include <linux/types.h>
> +#include <linux/time.h>
Nit: could we please keep these in alphabetical order?
> +/*
> + * There is one vdso_timestamp object in vvar for each vDSO-accelerated
> + * clock_id. For high-resolution clocks, this encodes the time
> + * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
> + * the actual time.
> + *
> + * To be noticed that for highres clocks nsec is left-shifted by
> + * vdso_data.cs[x].shift.
Nit: this would read better as:
Note that for highres clocks nsec is left-shifted by
vdso_data.cs[x].shift.
[...]
> +/*
> + * vdso_data will be accessed by 32 and 64 bit code at the same time
> + * so we should be careful before modifying this structure.
> + */
Perhaps:
/*
* This structure will be accessed by native and compat vDSOs, so we
* need to ensure the layout is identical for native and compat code.
*/
... which I think is what you're trrying to say?
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-22 12:58 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 12:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Hi Vincenzo,
On Fri, Feb 22, 2019 at 12:24:08PM +0000, Vincenzo Frascino wrote:
> +#ifdef __KERNEL__
I believe that __KERNEL__ guards haven't been necessary (in non-UAPI
headers) since the UAPI split, and can be removed.
[...]
> +#include <linux/bits.h>
> +#include <linux/types.h>
> +#include <linux/time.h>
Nit: could we please keep these in alphabetical order?
> +/*
> + * There is one vdso_timestamp object in vvar for each vDSO-accelerated
> + * clock_id. For high-resolution clocks, this encodes the time
> + * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
> + * the actual time.
> + *
> + * To be noticed that for highres clocks nsec is left-shifted by
> + * vdso_data.cs[x].shift.
Nit: this would read better as:
Note that for highres clocks nsec is left-shifted by
vdso_data.cs[x].shift.
[...]
> +/*
> + * vdso_data will be accessed by 32 and 64 bit code at the same time
> + * so we should be careful before modifying this structure.
> + */
Perhaps:
/*
* This structure will be accessed by native and compat vDSOs, so we
* need to ensure the layout is identical for native and compat code.
*/
... which I think is what you're trrying to say?
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-22 12:58 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 12:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Hi Vincenzo,
On Fri, Feb 22, 2019 at 12:24:08PM +0000, Vincenzo Frascino wrote:
> +#ifdef __KERNEL__
I believe that __KERNEL__ guards haven't been necessary (in non-UAPI
headers) since the UAPI split, and can be removed.
[...]
> +#include <linux/bits.h>
> +#include <linux/types.h>
> +#include <linux/time.h>
Nit: could we please keep these in alphabetical order?
> +/*
> + * There is one vdso_timestamp object in vvar for each vDSO-accelerated
> + * clock_id. For high-resolution clocks, this encodes the time
> + * corresponding to vdso_data.cycle_last. For coarse clocks this encodes
> + * the actual time.
> + *
> + * To be noticed that for highres clocks nsec is left-shifted by
> + * vdso_data.cs[x].shift.
Nit: this would read better as:
Note that for highres clocks nsec is left-shifted by
vdso_data.cs[x].shift.
[...]
> +/*
> + * vdso_data will be accessed by 32 and 64 bit code at the same time
> + * so we should be careful before modifying this structure.
> + */
Perhaps:
/*
* This structure will be accessed by native and compat vDSOs, so we
* need to ensure the layout is identical for native and compat code.
*/
... which I think is what you're trrying to say?
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 13:34 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 13:34 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:09PM +0000, Vincenzo Frascino wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> Try to unify the gettimeofday vdso implementation introducing
> lib/vdso. The code contained in this library can ideally be
> reused by all the architectures avoiding, where possible, code
> duplication.
>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> include/vdso/datapage.h | 1 +
> include/vdso/helpers.h | 52 ++++++++++++
> include/vdso/types.h | 39 +++++++++
> lib/Kconfig | 5 ++
> lib/vdso/Kconfig | 37 +++++++++
> lib/vdso/Makefile | 22 +++++
> lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++
> 7 files changed, 331 insertions(+)
> create mode 100644 include/vdso/helpers.h
> create mode 100644 include/vdso/types.h
> create mode 100644 lib/vdso/Kconfig
> create mode 100644 lib/vdso/Makefile
> create mode 100644 lib/vdso/gettimeofday.c
>
> diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
> index da346ad02b03..ff332fcba73c 100644
> --- a/include/vdso/datapage.h
> +++ b/include/vdso/datapage.h
> @@ -9,6 +9,7 @@
> #include <linux/bits.h>
> #include <linux/types.h>
> #include <linux/time.h>
> +#include <vdso/types.h>
>
> #define VDSO_BASES (CLOCK_TAI + 1)
> #define VDSO_HRES (BIT(CLOCK_REALTIME) | \
> diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
> new file mode 100644
> index 000000000000..511dea979f6b
> --- /dev/null
> +++ b/include/vdso/helpers.h
> @@ -0,0 +1,52 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __VDSO_HELPERS_H
> +#define __VDSO_HELPERS_H
> +
> +#ifdef __KERNEL__
Nit: __KERNEL__ guards can go.
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <vdso/datapage.h>
> +
> +static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
Rather than explicitly annotating all functions with notrace, can't we
disable instrumentation for all C files used for the vDSO using compiler
flags?
That would be more robust, and make the code less noisy to read.
> +{
> + u32 seq;
> +
> +repeat:
> + seq = READ_ONCE(vd->seq);
> + if (seq & 1) {
> + cpu_relax();
> + goto repeat;
> + }
> +
> + smp_rmb();
> + return seq;
> +}
You could simplify the repeat loop as:
while ((seq = READ_ONCE(vd->seq)) & 1)
cpu_relax();
> +
> +static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
> + u32 start)
> +{
> + u32 seq;
> +
> + smp_rmb();
> + seq = READ_ONCE(vd->seq);
> + return seq != start;
> +}
> +
> +static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
> +{
> + ++vd->seq;
> + smp_wmb();
> +}
> +
> +static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
> +{
> + smp_wmb();
> + ++vd->seq;
> +}
I realise this is what existing vDSO update code does, but I do think
that these should be using WRITE_ONCE() to perform the update, to ensure
that the write is not torn.
e.g. these should be:
static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
{
WRITE_ONCE(vd->seq, vd->seq + 1);
smp_wmb();
}
static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
{
smp_wmb();
WRITE_ONCE(vd->seq, vd->seq + 1);
}
Otherwise, the compiler can validly tear updates to vd->seq, and there's
the possibility that a reader sees an inconsistent value for vd->seq,
and consequently uses inconsistent data read from the vdso data page.
[...]
> +#include <linux/types.h>
> +#include <linux/time.h>
Nit: please order headers alphabetically>
> +
> +/*
> + * The definitions below are required to overcome the limitations
> + * of time_t on 32 bit architectures, which overflows in 2038.
> + * The new code should use the replacements based on time64_t and
> + * timespec64.
> + *
> + * The abstraction below will be updated once the migration to
> + * time64_t is complete.
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#ifdef ENABLE_COMPAT_VDSO
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#define __vdso_timespec __kernel_timespec
> +#define __vdso_timeval __kernel_old_timeval
> +#endif /* CONFIG_COMPAT_VDSO */
> +#endif /* CONFIG_GENERIC_VDSO_32 */
I'm not sure what the comment is trying to say.
For a 64-bit kernel, does this affec the native vDSO, or just the compat
vDSO?
[...]
> +config HAVE_GENERIC_VDSO
> + bool
> + default n
IIRC, 'n' is the implicit default.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 13:34 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 13:34 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 12:24:09PM +0000, Vincenzo Frascino wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> Try to unify the gettimeofday vdso implementation introducing
> lib/vdso. The code contained in this library can ideally be
> reused by all the architectures avoiding, where possible, code
> duplication.
>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> include/vdso/datapage.h | 1 +
> include/vdso/helpers.h | 52 ++++++++++++
> include/vdso/types.h | 39 +++++++++
> lib/Kconfig | 5 ++
> lib/vdso/Kconfig | 37 +++++++++
> lib/vdso/Makefile | 22 +++++
> lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++
> 7 files changed, 331 insertions(+)
> create mode 100644 include/vdso/helpers.h
> create mode 100644 include/vdso/types.h
> create mode 100644 lib/vdso/Kconfig
> create mode 100644 lib/vdso/Makefile
> create mode 100644 lib/vdso/gettimeofday.c
>
> diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
> index da346ad02b03..ff332fcba73c 100644
> --- a/include/vdso/datapage.h
> +++ b/include/vdso/datapage.h
> @@ -9,6 +9,7 @@
> #include <linux/bits.h>
> #include <linux/types.h>
> #include <linux/time.h>
> +#include <vdso/types.h>
>
> #define VDSO_BASES (CLOCK_TAI + 1)
> #define VDSO_HRES (BIT(CLOCK_REALTIME) | \
> diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
> new file mode 100644
> index 000000000000..511dea979f6b
> --- /dev/null
> +++ b/include/vdso/helpers.h
> @@ -0,0 +1,52 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __VDSO_HELPERS_H
> +#define __VDSO_HELPERS_H
> +
> +#ifdef __KERNEL__
Nit: __KERNEL__ guards can go.
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <vdso/datapage.h>
> +
> +static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
Rather than explicitly annotating all functions with notrace, can't we
disable instrumentation for all C files used for the vDSO using compiler
flags?
That would be more robust, and make the code less noisy to read.
> +{
> + u32 seq;
> +
> +repeat:
> + seq = READ_ONCE(vd->seq);
> + if (seq & 1) {
> + cpu_relax();
> + goto repeat;
> + }
> +
> + smp_rmb();
> + return seq;
> +}
You could simplify the repeat loop as:
while ((seq = READ_ONCE(vd->seq)) & 1)
cpu_relax();
> +
> +static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
> + u32 start)
> +{
> + u32 seq;
> +
> + smp_rmb();
> + seq = READ_ONCE(vd->seq);
> + return seq != start;
> +}
> +
> +static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
> +{
> + ++vd->seq;
> + smp_wmb();
> +}
> +
> +static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
> +{
> + smp_wmb();
> + ++vd->seq;
> +}
I realise this is what existing vDSO update code does, but I do think
that these should be using WRITE_ONCE() to perform the update, to ensure
that the write is not torn.
e.g. these should be:
static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
{
WRITE_ONCE(vd->seq, vd->seq + 1);
smp_wmb();
}
static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
{
smp_wmb();
WRITE_ONCE(vd->seq, vd->seq + 1);
}
Otherwise, the compiler can validly tear updates to vd->seq, and there's
the possibility that a reader sees an inconsistent value for vd->seq,
and consequently uses inconsistent data read from the vdso data page.
[...]
> +#include <linux/types.h>
> +#include <linux/time.h>
Nit: please order headers alphabetically>
> +
> +/*
> + * The definitions below are required to overcome the limitations
> + * of time_t on 32 bit architectures, which overflows in 2038.
> + * The new code should use the replacements based on time64_t and
> + * timespec64.
> + *
> + * The abstraction below will be updated once the migration to
> + * time64_t is complete.
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#ifdef ENABLE_COMPAT_VDSO
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#define __vdso_timespec __kernel_timespec
> +#define __vdso_timeval __kernel_old_timeval
> +#endif /* CONFIG_COMPAT_VDSO */
> +#endif /* CONFIG_GENERIC_VDSO_32 */
I'm not sure what the comment is trying to say.
For a 64-bit kernel, does this affec the native vDSO, or just the compat
vDSO?
[...]
> +config HAVE_GENERIC_VDSO
> + bool
> + default n
IIRC, 'n' is the implicit default.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 13:34 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 13:34 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:09PM +0000, Vincenzo Frascino wrote:
> In the last few years we assisted to an explosion of vdso
> implementations that mostly share similar code.
>
> Try to unify the gettimeofday vdso implementation introducing
> lib/vdso. The code contained in this library can ideally be
> reused by all the architectures avoiding, where possible, code
> duplication.
>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> include/vdso/datapage.h | 1 +
> include/vdso/helpers.h | 52 ++++++++++++
> include/vdso/types.h | 39 +++++++++
> lib/Kconfig | 5 ++
> lib/vdso/Kconfig | 37 +++++++++
> lib/vdso/Makefile | 22 +++++
> lib/vdso/gettimeofday.c | 175 ++++++++++++++++++++++++++++++++++++++++
> 7 files changed, 331 insertions(+)
> create mode 100644 include/vdso/helpers.h
> create mode 100644 include/vdso/types.h
> create mode 100644 lib/vdso/Kconfig
> create mode 100644 lib/vdso/Makefile
> create mode 100644 lib/vdso/gettimeofday.c
>
> diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
> index da346ad02b03..ff332fcba73c 100644
> --- a/include/vdso/datapage.h
> +++ b/include/vdso/datapage.h
> @@ -9,6 +9,7 @@
> #include <linux/bits.h>
> #include <linux/types.h>
> #include <linux/time.h>
> +#include <vdso/types.h>
>
> #define VDSO_BASES (CLOCK_TAI + 1)
> #define VDSO_HRES (BIT(CLOCK_REALTIME) | \
> diff --git a/include/vdso/helpers.h b/include/vdso/helpers.h
> new file mode 100644
> index 000000000000..511dea979f6b
> --- /dev/null
> +++ b/include/vdso/helpers.h
> @@ -0,0 +1,52 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef __VDSO_HELPERS_H
> +#define __VDSO_HELPERS_H
> +
> +#ifdef __KERNEL__
Nit: __KERNEL__ guards can go.
> +
> +#ifndef __ASSEMBLY__
> +
> +#include <vdso/datapage.h>
> +
> +static __always_inline notrace u32 vdso_read_begin(const struct vdso_data *vd)
Rather than explicitly annotating all functions with notrace, can't we
disable instrumentation for all C files used for the vDSO using compiler
flags?
That would be more robust, and make the code less noisy to read.
> +{
> + u32 seq;
> +
> +repeat:
> + seq = READ_ONCE(vd->seq);
> + if (seq & 1) {
> + cpu_relax();
> + goto repeat;
> + }
> +
> + smp_rmb();
> + return seq;
> +}
You could simplify the repeat loop as:
while ((seq = READ_ONCE(vd->seq)) & 1)
cpu_relax();
> +
> +static __always_inline notrace u32 vdso_read_retry(const struct vdso_data *vd,
> + u32 start)
> +{
> + u32 seq;
> +
> + smp_rmb();
> + seq = READ_ONCE(vd->seq);
> + return seq != start;
> +}
> +
> +static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
> +{
> + ++vd->seq;
> + smp_wmb();
> +}
> +
> +static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
> +{
> + smp_wmb();
> + ++vd->seq;
> +}
I realise this is what existing vDSO update code does, but I do think
that these should be using WRITE_ONCE() to perform the update, to ensure
that the write is not torn.
e.g. these should be:
static __always_inline notrace void vdso_write_begin(struct vdso_data *vd)
{
WRITE_ONCE(vd->seq, vd->seq + 1);
smp_wmb();
}
static __always_inline notrace void vdso_write_end(struct vdso_data *vd)
{
smp_wmb();
WRITE_ONCE(vd->seq, vd->seq + 1);
}
Otherwise, the compiler can validly tear updates to vd->seq, and there's
the possibility that a reader sees an inconsistent value for vd->seq,
and consequently uses inconsistent data read from the vdso data page.
[...]
> +#include <linux/types.h>
> +#include <linux/time.h>
Nit: please order headers alphabetically>
> +
> +/*
> + * The definitions below are required to overcome the limitations
> + * of time_t on 32 bit architectures, which overflows in 2038.
> + * The new code should use the replacements based on time64_t and
> + * timespec64.
> + *
> + * The abstraction below will be updated once the migration to
> + * time64_t is complete.
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#ifdef ENABLE_COMPAT_VDSO
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#define __vdso_timespec __kernel_timespec
> +#define __vdso_timeval __kernel_old_timeval
> +#endif /* CONFIG_COMPAT_VDSO */
> +#endif /* CONFIG_GENERIC_VDSO_32 */
I'm not sure what the comment is trying to say.
For a 64-bit kernel, does this affec the native vDSO, or just the compat
vDSO?
[...]
> +config HAVE_GENERIC_VDSO
> + bool
> + default n
IIRC, 'n' is the implicit default.
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation
@ 2019-02-22 13:42 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 13:42 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:11PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
> index 8cd4707f832a..c904b2a9b255 100644
> --- a/arch/arm64/kernel/vdso/Makefile
> +++ b/arch/arm64/kernel/vdso/Makefile
> @@ -6,16 +6,31 @@
> # Heavily based on the vDSO Makefiles for other archs.
> #
>
> -obj-vdso := gettimeofday.o note.o sigreturn.o
> +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
> +# the inclusion of generic Makefile.
> +ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
> +include $(srctree)/lib/vdso/Makefile
> +
> +obj-vdso := vgettimeofday.o note.o sigreturn.o
>
> # Build rules
> targets := $(obj-vdso) vdso.so vdso.so.dbg
> obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
>
> -ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
> +ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
> +ccflags-y += -DDISABLE_BRANCH_PROFILING
> ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
> $(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
>
> +VDSO_LDFLAGS := -Bsymbolic
> +
> +CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
Please use CC_FLAGS_FTRACE rather than '-pg'
What said, wouldn't it be better to come up with the VDSO cflags
entirely separately from what the main kernel uses?
That way we're unlikely to have any surprises in future with
instrumentation etc.
I strongly suspect you need to disable more things here, e.g. UBSAN,
KASAN, KCOV. Have you tried compiling a kernel with those features
enabled?
VDSO files probably also need to be marked as OBJECT_FILES_NON_STANDARD.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation
@ 2019-02-22 13:42 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 13:42 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 12:24:11PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
> index 8cd4707f832a..c904b2a9b255 100644
> --- a/arch/arm64/kernel/vdso/Makefile
> +++ b/arch/arm64/kernel/vdso/Makefile
> @@ -6,16 +6,31 @@
> # Heavily based on the vDSO Makefiles for other archs.
> #
>
> -obj-vdso := gettimeofday.o note.o sigreturn.o
> +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
> +# the inclusion of generic Makefile.
> +ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
> +include $(srctree)/lib/vdso/Makefile
> +
> +obj-vdso := vgettimeofday.o note.o sigreturn.o
>
> # Build rules
> targets := $(obj-vdso) vdso.so vdso.so.dbg
> obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
>
> -ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
> +ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
> +ccflags-y += -DDISABLE_BRANCH_PROFILING
> ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
> $(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
>
> +VDSO_LDFLAGS := -Bsymbolic
> +
> +CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
Please use CC_FLAGS_FTRACE rather than '-pg'
What said, wouldn't it be better to come up with the VDSO cflags
entirely separately from what the main kernel uses?
That way we're unlikely to have any surprises in future with
instrumentation etc.
I strongly suspect you need to disable more things here, e.g. UBSAN,
KASAN, KCOV. Have you tried compiling a kernel with those features
enabled?
VDSO files probably also need to be marked as OBJECT_FILES_NON_STANDARD.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation
@ 2019-02-22 13:42 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 13:42 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:11PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/kernel/vdso/Makefile b/arch/arm64/kernel/vdso/Makefile
> index 8cd4707f832a..c904b2a9b255 100644
> --- a/arch/arm64/kernel/vdso/Makefile
> +++ b/arch/arm64/kernel/vdso/Makefile
> @@ -6,16 +6,31 @@
> # Heavily based on the vDSO Makefiles for other archs.
> #
>
> -obj-vdso := gettimeofday.o note.o sigreturn.o
> +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
> +# the inclusion of generic Makefile.
> +ARCH_REL_TYPE_ABS := R_AARCH64_JUMP_SLOT|R_AARCH64_GLOB_DAT|R_AARCH64_ABS64
> +include $(srctree)/lib/vdso/Makefile
> +
> +obj-vdso := vgettimeofday.o note.o sigreturn.o
>
> # Build rules
> targets := $(obj-vdso) vdso.so vdso.so.dbg
> obj-vdso := $(addprefix $(obj)/, $(obj-vdso))
>
> -ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18
> +ccflags-y := -shared -fno-common -fno-builtin -ffixed-x18 -fno-stack-protector
> +ccflags-y += -DDISABLE_BRANCH_PROFILING
> ccflags-y += -nostdlib -Wl,-soname=linux-vdso.so.1 \
> $(call cc-ldoption, -Wl$(comma)--hash-style=sysv)
>
> +VDSO_LDFLAGS := -Bsymbolic
> +
> +CFLAGS_REMOVE_vgettimeofday.o = -pg -Os
Please use CC_FLAGS_FTRACE rather than '-pg'
What said, wouldn't it be better to come up with the VDSO cflags
entirely separately from what the main kernel uses?
That way we're unlikely to have any surprises in future with
instrumentation etc.
I strongly suspect you need to disable more things here, e.g. UBSAN,
KASAN, KCOV. Have you tried compiling a kernel with those features
enabled?
VDSO files probably also need to be marked as OBJECT_FILES_NON_STANDARD.
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 13:49 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-22 13:49 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +/*
> + * The definitions below are required to overcome the limitations
> + * of time_t on 32 bit architectures, which overflows in 2038.
> + * The new code should use the replacements based on time64_t and
> + * timespec64.
> + *
> + * The abstraction below will be updated once the migration to
> + * time64_t is complete.
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#ifdef ENABLE_COMPAT_VDSO
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#define __vdso_timespec __kernel_timespec
> +#define __vdso_timeval __kernel_old_timeval
> +#endif /* CONFIG_COMPAT_VDSO */
> +#endif /* CONFIG_GENERIC_VDSO_32 */
I don't think we need __vdso_timeval at all, just use
__kernel_old_timeval everywhere. There is no generic
64-bit timeval type in the kernel, and there won't be because
gettimeofday() is deprecated.
For __vdso_timespec, I see how you ended up with this
redefinition, and it makes the current version of your patches
easier, but I fear it will in turn make it harder to add the
__kernel_old_timeval based variant.
What I'd prefer to see here is to always use the fixed length
types everywhere in the code, such as
static notrace int __cvdso_clock_gettime64(clockid_t clock,
struct __kernel_timespec *ts)
{
... /* normal clock_gettime using 64-bit times */
}
static notrace int __cvdso_clock_gettime32(clockid_t clock,
struct old_timespec32 *ts32)
{
struct __kernel_timespec ts;
int ret = __cvdso_clock_gettime64(clock, &ts);
ts32->tv_sec = ts->tv_sec;
ts32->tv_nsec = ts->tv_nsec;
return ret;
}
and then have two different versions for 32-bit and 64-bit
architectures, calling one or the other function of the
generic code.
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 13:49 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-22 13:49 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +/*
> + * The definitions below are required to overcome the limitations
> + * of time_t on 32 bit architectures, which overflows in 2038.
> + * The new code should use the replacements based on time64_t and
> + * timespec64.
> + *
> + * The abstraction below will be updated once the migration to
> + * time64_t is complete.
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#ifdef ENABLE_COMPAT_VDSO
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#define __vdso_timespec __kernel_timespec
> +#define __vdso_timeval __kernel_old_timeval
> +#endif /* CONFIG_COMPAT_VDSO */
> +#endif /* CONFIG_GENERIC_VDSO_32 */
I don't think we need __vdso_timeval at all, just use
__kernel_old_timeval everywhere. There is no generic
64-bit timeval type in the kernel, and there won't be because
gettimeofday() is deprecated.
For __vdso_timespec, I see how you ended up with this
redefinition, and it makes the current version of your patches
easier, but I fear it will in turn make it harder to add the
__kernel_old_timeval based variant.
What I'd prefer to see here is to always use the fixed length
types everywhere in the code, such as
static notrace int __cvdso_clock_gettime64(clockid_t clock,
struct __kernel_timespec *ts)
{
... /* normal clock_gettime using 64-bit times */
}
static notrace int __cvdso_clock_gettime32(clockid_t clock,
struct old_timespec32 *ts32)
{
struct __kernel_timespec ts;
int ret = __cvdso_clock_gettime64(clock, &ts);
ts32->tv_sec = ts->tv_sec;
ts32->tv_nsec = ts->tv_nsec;
return ret;
}
and then have two different versions for 32-bit and 64-bit
architectures, calling one or the other function of the
generic code.
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 13:49 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-22 13:49 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> +/*
> + * The definitions below are required to overcome the limitations
> + * of time_t on 32 bit architectures, which overflows in 2038.
> + * The new code should use the replacements based on time64_t and
> + * timespec64.
> + *
> + * The abstraction below will be updated once the migration to
> + * time64_t is complete.
> + */
> +#ifdef CONFIG_GENERIC_VDSO_32
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#ifdef ENABLE_COMPAT_VDSO
> +#define __vdso_timespec old_timespec32
> +#define __vdso_timeval old_timeval32
> +#else
> +#define __vdso_timespec __kernel_timespec
> +#define __vdso_timeval __kernel_old_timeval
> +#endif /* CONFIG_COMPAT_VDSO */
> +#endif /* CONFIG_GENERIC_VDSO_32 */
I don't think we need __vdso_timeval at all, just use
__kernel_old_timeval everywhere. There is no generic
64-bit timeval type in the kernel, and there won't be because
gettimeofday() is deprecated.
For __vdso_timespec, I see how you ended up with this
redefinition, and it makes the current version of your patches
easier, but I fear it will in turn make it harder to add the
__kernel_old_timeval based variant.
What I'd prefer to see here is to always use the fixed length
types everywhere in the code, such as
static notrace int __cvdso_clock_gettime64(clockid_t clock,
struct __kernel_timespec *ts)
{
... /* normal clock_gettime using 64-bit times */
}
static notrace int __cvdso_clock_gettime32(clockid_t clock,
struct old_timespec32 *ts32)
{
struct __kernel_timespec ts;
int ret = __cvdso_clock_gettime64(clock, &ts);
ts32->tv_sec = ts->tv_sec;
ts32->tv_nsec = ts->tv_nsec;
return ret;
}
and then have two different versions for 32-bit and 64-bit
architectures, calling one or the other function of the
generic code.
Arnd
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 14:04 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:04 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:15PM +0000, Vincenzo Frascino wrote:
> When kuser helpers are enabled the kernel maps the relative code at
> a fixed address (0xffff0000). Making configurable the option to disable
> them means that the kernel can remove this mapping and any access to
> this memory area results in a sigfault.
>
> Add a KUSER_HELPERS config option that can be used to disable the
> mapping when it is turned off.
>
> This option can be turned off if and only if the applications are
> designed specifically for the platform and they do not make use of the
> kuser helpers code.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO
changes.
I think the KUSER bits should be split into a preparatory series, so that they
can be merged soon, rather than tying them artificially to the rest of the vDSO
bits (which I expect will require significantly more review).
> ---
> arch/arm64/Kconfig | 21 +++++++++++++++++++++
> arch/arm64/kernel/Makefile | 3 ++-
> arch/arm64/kernel/kuser32.S | 7 +++----
> arch/arm64/kernel/vdso.c | 15 +++++++++++++++
> 4 files changed, 41 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index d898da2e20f5..ed3290494f1c 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1465,6 +1465,27 @@ config COMPAT
>
> If you want to execute 32-bit userspace applications, say Y.
>
> +config KUSER_HELPERS
> + bool "Enable kuser helpers page for compatibility with 32 bit applications."
> + depends on COMPAT
> + default y
> + help
> + Enables kuser helpers to be mapped in a special purpose page at a fixed
> + address to maintain independence from the type of CPU present in the SoC.
> + This feature is provided for compatibility reasons in fact allows 32 bit
> + applications compliant with ARMv4 up to ARMv8 to run without any
> + modification.
> +
> + Warning: Being always mapped at a fixed address makes it easier to create
> + exploits based on ROP type of attacks.
> +
> + As a consequence of this, this feature is made configurable but be aware that
> + it can be turned off if and only if the binaries and the libraries running on
> + a specific platform are designed to do not make use of these helpers, otherwise
> + should be left on.
> +
> + See Documentation/arm/kernel_user_helpers.txt for details.
This is quite divergent from the text for the 32-bit arm KUSER_HELPERS.
Could we please reuse that existing text as far as possible, with minimal
changes?
e.g.
config KUSER_HELPERS
bool "Enable kuser helpers for compat tasks"
depends on COMPAT
default y
help
Warning: disabling this option may break user programs.
Provide kuser helpers to compat tasks. The kernel provides
helper code to userspace in read only form at a fixed location
to allow userspace to be independent of the CPU type fitted to
the system. This permits binaries to be run on ARMv4 through
to ARMv8 without modification.
See Documentation/arm/kernel_user_helpers.txt for details.
However, the fixed address nature of these helpers can be used
by ROP (return orientated programming) authors when creating
exploits.
If all of the binaries and libraries which run on your platform
are built specifically for your platform, and make no use of
these helpers, then you can turn this option off to hinder
such exploits. However, in that case, if a binary or library
relying on those helpers is run, it will not function correctly.
Say N here only if you are absolutely certain that you do not
need these helpers; otherwise, the safe option is to say Y.
I believe that Will wanted to default the kuser helpers off, as applications
using them need the instruction emulation which is also disabled by default.
I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
the ARMv4 reference is correct.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 14:04 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:04 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 12:24:15PM +0000, Vincenzo Frascino wrote:
> When kuser helpers are enabled the kernel maps the relative code at
> a fixed address (0xffff0000). Making configurable the option to disable
> them means that the kernel can remove this mapping and any access to
> this memory area results in a sigfault.
>
> Add a KUSER_HELPERS config option that can be used to disable the
> mapping when it is turned off.
>
> This option can be turned off if and only if the applications are
> designed specifically for the platform and they do not make use of the
> kuser helpers code.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO
changes.
I think the KUSER bits should be split into a preparatory series, so that they
can be merged soon, rather than tying them artificially to the rest of the vDSO
bits (which I expect will require significantly more review).
> ---
> arch/arm64/Kconfig | 21 +++++++++++++++++++++
> arch/arm64/kernel/Makefile | 3 ++-
> arch/arm64/kernel/kuser32.S | 7 +++----
> arch/arm64/kernel/vdso.c | 15 +++++++++++++++
> 4 files changed, 41 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index d898da2e20f5..ed3290494f1c 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1465,6 +1465,27 @@ config COMPAT
>
> If you want to execute 32-bit userspace applications, say Y.
>
> +config KUSER_HELPERS
> + bool "Enable kuser helpers page for compatibility with 32 bit applications."
> + depends on COMPAT
> + default y
> + help
> + Enables kuser helpers to be mapped in a special purpose page at a fixed
> + address to maintain independence from the type of CPU present in the SoC.
> + This feature is provided for compatibility reasons in fact allows 32 bit
> + applications compliant with ARMv4 up to ARMv8 to run without any
> + modification.
> +
> + Warning: Being always mapped at a fixed address makes it easier to create
> + exploits based on ROP type of attacks.
> +
> + As a consequence of this, this feature is made configurable but be aware that
> + it can be turned off if and only if the binaries and the libraries running on
> + a specific platform are designed to do not make use of these helpers, otherwise
> + should be left on.
> +
> + See Documentation/arm/kernel_user_helpers.txt for details.
This is quite divergent from the text for the 32-bit arm KUSER_HELPERS.
Could we please reuse that existing text as far as possible, with minimal
changes?
e.g.
config KUSER_HELPERS
bool "Enable kuser helpers for compat tasks"
depends on COMPAT
default y
help
Warning: disabling this option may break user programs.
Provide kuser helpers to compat tasks. The kernel provides
helper code to userspace in read only form at a fixed location
to allow userspace to be independent of the CPU type fitted to
the system. This permits binaries to be run on ARMv4 through
to ARMv8 without modification.
See Documentation/arm/kernel_user_helpers.txt for details.
However, the fixed address nature of these helpers can be used
by ROP (return orientated programming) authors when creating
exploits.
If all of the binaries and libraries which run on your platform
are built specifically for your platform, and make no use of
these helpers, then you can turn this option off to hinder
such exploits. However, in that case, if a binary or library
relying on those helpers is run, it will not function correctly.
Say N here only if you are absolutely certain that you do not
need these helpers; otherwise, the safe option is to say Y.
I believe that Will wanted to default the kuser helpers off, as applications
using them need the instruction emulation which is also disabled by default.
I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
the ARMv4 reference is correct.
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 14:04 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:04 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:15PM +0000, Vincenzo Frascino wrote:
> When kuser helpers are enabled the kernel maps the relative code at
> a fixed address (0xffff0000). Making configurable the option to disable
> them means that the kernel can remove this mapping and any access to
> this memory area results in a sigfault.
>
> Add a KUSER_HELPERS config option that can be used to disable the
> mapping when it is turned off.
>
> This option can be turned off if and only if the applications are
> designed specifically for the platform and they do not make use of the
> kuser helpers code.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO
changes.
I think the KUSER bits should be split into a preparatory series, so that they
can be merged soon, rather than tying them artificially to the rest of the vDSO
bits (which I expect will require significantly more review).
> ---
> arch/arm64/Kconfig | 21 +++++++++++++++++++++
> arch/arm64/kernel/Makefile | 3 ++-
> arch/arm64/kernel/kuser32.S | 7 +++----
> arch/arm64/kernel/vdso.c | 15 +++++++++++++++
> 4 files changed, 41 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index d898da2e20f5..ed3290494f1c 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -1465,6 +1465,27 @@ config COMPAT
>
> If you want to execute 32-bit userspace applications, say Y.
>
> +config KUSER_HELPERS
> + bool "Enable kuser helpers page for compatibility with 32 bit applications."
> + depends on COMPAT
> + default y
> + help
> + Enables kuser helpers to be mapped in a special purpose page at a fixed
> + address to maintain independence from the type of CPU present in the SoC.
> + This feature is provided for compatibility reasons in fact allows 32 bit
> + applications compliant with ARMv4 up to ARMv8 to run without any
> + modification.
> +
> + Warning: Being always mapped at a fixed address makes it easier to create
> + exploits based on ROP type of attacks.
> +
> + As a consequence of this, this feature is made configurable but be aware that
> + it can be turned off if and only if the binaries and the libraries running on
> + a specific platform are designed to do not make use of these helpers, otherwise
> + should be left on.
> +
> + See Documentation/arm/kernel_user_helpers.txt for details.
This is quite divergent from the text for the 32-bit arm KUSER_HELPERS.
Could we please reuse that existing text as far as possible, with minimal
changes?
e.g.
config KUSER_HELPERS
bool "Enable kuser helpers for compat tasks"
depends on COMPAT
default y
help
Warning: disabling this option may break user programs.
Provide kuser helpers to compat tasks. The kernel provides
helper code to userspace in read only form at a fixed location
to allow userspace to be independent of the CPU type fitted to
the system. This permits binaries to be run on ARMv4 through
to ARMv8 without modification.
See Documentation/arm/kernel_user_helpers.txt for details.
However, the fixed address nature of these helpers can be used
by ROP (return orientated programming) authors when creating
exploits.
If all of the binaries and libraries which run on your platform
are built specifically for your platform, and make no use of
these helpers, then you can turn this option off to hinder
such exploits. However, in that case, if a binary or library
relying on those helpers is run, it will not function correctly.
Say N here only if you are absolutely certain that you do not
need these helpers; otherwise, the safe option is to say Y.
I believe that Will wanted to default the kuser helpers off, as applications
using them need the instruction emulation which is also disabled by default.
I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
the ARMv4 reference is correct.
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 14:09 ` Russell King - ARM Linux admin
0 siblings, 0 replies; 178+ messages in thread
From: Russell King - ARM Linux admin @ 2019-02-22 14:09 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Vincenzo Frascino, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 02:04:27PM +0000, Mark Rutland wrote:
> I believe that Will wanted to default the kuser helpers off, as applications
> using them need the instruction emulation which is also disabled by default.
>
> I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
> the ARMv4 reference is correct.
Do you expect ARMv4 - even ARMv4 OABI binaries to fail on ARM64?
(I've run my OABI ARMv4 distro under KVM on Macchiatobin when I've
needed to rebuild a RPM without problems - using its own gcc. No
observable issues.)
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 14:09 ` Russell King - ARM Linux admin
0 siblings, 0 replies; 178+ messages in thread
From: Russell King - ARM Linux admin @ 2019-02-22 14:09 UTC (permalink / raw)
To: Mark Rutland
Cc: Vincenzo Frascino, linux-arch, linux-arm-kernel, Catalin Marinas,
Will Deacon, Arnd Bergmann, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 02:04:27PM +0000, Mark Rutland wrote:
> I believe that Will wanted to default the kuser helpers off, as applications
> using them need the instruction emulation which is also disabled by default.
>
> I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
> the ARMv4 reference is correct.
Do you expect ARMv4 - even ARMv4 OABI binaries to fail on ARM64?
(I've run my OABI ARMv4 distro under KVM on Macchiatobin when I've
needed to rebuild a RPM without problems - using its own gcc. No
observable issues.)
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-22 14:09 ` Russell King - ARM Linux admin
0 siblings, 0 replies; 178+ messages in thread
From: Russell King - ARM Linux admin @ 2019-02-22 14:09 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Ralf Baechle, Mark Salyzyn,
Paul Burton, Dmitry Safonov, Rasmus Villemoes, Thomas Gleixner,
Vincenzo Frascino, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 02:04:27PM +0000, Mark Rutland wrote:
> I believe that Will wanted to default the kuser helpers off, as applications
> using them need the instruction emulation which is also disabled by default.
>
> I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
> the ARMv4 reference is correct.
Do you expect ARMv4 - even ARMv4 OABI binaries to fail on ARM64?
(I've run my OABI ARMv4 distro under KVM on Macchiatobin when I've
needed to rebuild a RPM without problems - using its own gcc. No
observable issues.)
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals
@ 2019-02-22 14:22 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:22 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:18PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
> index 1f05268f4c6d..a6fd7be1b22a 100644
> --- a/arch/arm64/include/asm/signal32.h
> +++ b/arch/arm64/include/asm/signal32.h
> @@ -52,7 +52,7 @@ struct compat_ucontext {
> compat_stack_t uc_stack;
> struct compat_sigcontext uc_mcontext;
> compat_sigset_t uc_sigmask;
> - int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
> + int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
Spurious whitespace change?
> +/* Macros for asm-offsets.c */
> +#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
> + offsetof(struct compat_sigframe, uc) + \
> + offsetof(struct compat_ucontext, uc_mcontext) + \
> + offsetof(struct compat_sigcontext, arm_r0))
> +
> +#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
> + offsetof(struct compat_rt_sigframe, sig) + \
> + offsetof(struct compat_sigframe, uc) + \
> + offsetof(struct compat_ucontext, uc_mcontext) + \
> + offsetof(struct compat_sigcontext, arm_r0))
> +
> int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
> struct pt_regs *regs);
> int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 4adb52190a90..afd28ccda174 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -30,6 +30,7 @@
> #include <asm/fixmap.h>
> #include <asm/thread_info.h>
> #include <asm/memory.h>
> +#include <asm/signal32.h>
> #include <asm/smp_plat.h>
> #include <asm/suspend.h>
> #include <linux/kbuild.h>
> @@ -84,6 +85,11 @@ int main(void)
> DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
> DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
> BLANK();
> +#ifdef CONFIG_COMPAT
> + DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
> + DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
Can't we do:
DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_sigframe, uc.us_mcontext.arm_r0)
DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_rt_sigframe, sig.uc.us_mcontext.arm_r0)
... and avoid the OFFSET_OF_* macros entirely?
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals
@ 2019-02-22 14:22 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:22 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 12:24:18PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
> index 1f05268f4c6d..a6fd7be1b22a 100644
> --- a/arch/arm64/include/asm/signal32.h
> +++ b/arch/arm64/include/asm/signal32.h
> @@ -52,7 +52,7 @@ struct compat_ucontext {
> compat_stack_t uc_stack;
> struct compat_sigcontext uc_mcontext;
> compat_sigset_t uc_sigmask;
> - int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
> + int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
Spurious whitespace change?
> +/* Macros for asm-offsets.c */
> +#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
> + offsetof(struct compat_sigframe, uc) + \
> + offsetof(struct compat_ucontext, uc_mcontext) + \
> + offsetof(struct compat_sigcontext, arm_r0))
> +
> +#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
> + offsetof(struct compat_rt_sigframe, sig) + \
> + offsetof(struct compat_sigframe, uc) + \
> + offsetof(struct compat_ucontext, uc_mcontext) + \
> + offsetof(struct compat_sigcontext, arm_r0))
> +
> int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
> struct pt_regs *regs);
> int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 4adb52190a90..afd28ccda174 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -30,6 +30,7 @@
> #include <asm/fixmap.h>
> #include <asm/thread_info.h>
> #include <asm/memory.h>
> +#include <asm/signal32.h>
> #include <asm/smp_plat.h>
> #include <asm/suspend.h>
> #include <linux/kbuild.h>
> @@ -84,6 +85,11 @@ int main(void)
> DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
> DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
> BLANK();
> +#ifdef CONFIG_COMPAT
> + DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
> + DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
Can't we do:
DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_sigframe, uc.us_mcontext.arm_r0)
DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_rt_sigframe, sig.uc.us_mcontext.arm_r0)
... and avoid the OFFSET_OF_* macros entirely?
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals
@ 2019-02-22 14:22 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:22 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:18PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/include/asm/signal32.h b/arch/arm64/include/asm/signal32.h
> index 1f05268f4c6d..a6fd7be1b22a 100644
> --- a/arch/arm64/include/asm/signal32.h
> +++ b/arch/arm64/include/asm/signal32.h
> @@ -52,7 +52,7 @@ struct compat_ucontext {
> compat_stack_t uc_stack;
> struct compat_sigcontext uc_mcontext;
> compat_sigset_t uc_sigmask;
> - int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
> + int __unused[32 - (sizeof(compat_sigset_t) / sizeof(int))];
Spurious whitespace change?
> +/* Macros for asm-offsets.c */
> +#define OFFSET_OF_COMPAT_SIGFRAME_REGS ( \
> + offsetof(struct compat_sigframe, uc) + \
> + offsetof(struct compat_ucontext, uc_mcontext) + \
> + offsetof(struct compat_sigcontext, arm_r0))
> +
> +#define OFFSET_OF_COMPAT_RT_SIGFRAME_REGS ( \
> + offsetof(struct compat_rt_sigframe, sig) + \
> + offsetof(struct compat_sigframe, uc) + \
> + offsetof(struct compat_ucontext, uc_mcontext) + \
> + offsetof(struct compat_sigcontext, arm_r0))
> +
> int compat_setup_frame(int usig, struct ksignal *ksig, sigset_t *set,
> struct pt_regs *regs);
> int compat_setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
> diff --git a/arch/arm64/kernel/asm-offsets.c b/arch/arm64/kernel/asm-offsets.c
> index 4adb52190a90..afd28ccda174 100644
> --- a/arch/arm64/kernel/asm-offsets.c
> +++ b/arch/arm64/kernel/asm-offsets.c
> @@ -30,6 +30,7 @@
> #include <asm/fixmap.h>
> #include <asm/thread_info.h>
> #include <asm/memory.h>
> +#include <asm/signal32.h>
> #include <asm/smp_plat.h>
> #include <asm/suspend.h>
> #include <linux/kbuild.h>
> @@ -84,6 +85,11 @@ int main(void)
> DEFINE(S_STACKFRAME, offsetof(struct pt_regs, stackframe));
> DEFINE(S_FRAME_SIZE, sizeof(struct pt_regs));
> BLANK();
> +#ifdef CONFIG_COMPAT
> + DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_SIGFRAME_REGS);
> + DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, OFFSET_OF_COMPAT_RT_SIGFRAME_REGS);
Can't we do:
DEFINE(COMPAT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_sigframe, uc.us_mcontext.arm_r0)
DEFINE(COMPAT_RT_SIGFRAME_REGS_OFFSET, offsetof(struct compat_rt_sigframe, sig.uc.us_mcontext.arm_r0)
... and avoid the OFFSET_OF_* macros entirely?
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 13/23] arm64: compat: Add vDSO
@ 2019-02-22 14:32 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:32 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:20PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
> new file mode 100644
> index 000000000000..b149761d9661
> --- /dev/null
> +++ b/arch/arm64/include/asm/vdso/compat_barrier.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2018 ARM Limited
> + */
> +#ifndef __COMPAT_BARRIER_H
> +#define __COMPAT_BARRIER_H
> +
> +#include <asm/barrier.h>
I think it would be safer to not include this header, and explicitly
define the AArch32 barriers here in isolation.
> +
> +#if __LINUX_ARM_ARCH__ >= 8
> +#define aarch32_smp_mb() dmb(ish)
> +#define aarch32_smp_rmb() dmb(ishld)
> +#define aarch32_smp_wmb() dmb(ishst)
> +#else
> +#define aarch32_smp_mb() dmb(ish)
> +#define aarch32_smp_rmb() aarch32_smp_mb()
> +#define aarch32_smp_wmb() dmb(ishst)
> +#endif
Can't we just code an AArch32-specific version of dmb() here? AFAICT
that's the only reason we include the native barrier.h.
> +/*
> + * Warning: This code is meant to be used with
> + * ENABLE_COMPAT_VDSO only.
> + */
> +#ifndef ENABLE_COMPAT_VDSO
> +#error This header is meant to be used with ENABLE_COMPAT_VDSO only
> +#endif
> +
> +#undef smp_mb
> +#undef smp_rmb
> +#undef smp_wmb
> +
> +#define smp_mb() aarch32_smp_mb()
> +#define smp_rmb() aarch32_smp_rmb()
> +#define smp_wmb() aarch32_smp_wmb()
This undeffing looks very fragile to me.
[...]
> diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
> new file mode 100644
> index 000000000000..4fea950fa5ed
> --- /dev/null
> +++ b/arch/arm64/kernel/vdso32/.gitignore
> @@ -0,0 +1,2 @@
> +vdso.lds
> +vdso.so.raw
> diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
> new file mode 100644
> index 000000000000..ba57d2ae620f
> --- /dev/null
> +++ b/arch/arm64/kernel/vdso32/Makefile
> @@ -0,0 +1,184 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for vdso32
> +#
> +
> +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
> +# the inclusion of generic Makefile.
> +ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
> +include $(srctree)/lib/vdso/Makefile
> +
> +CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
To match HOSTCC, it might make sense to call this COMPATCC
... and likewise, CROSS_COMPILE_COMPAT
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 13/23] arm64: compat: Add vDSO
@ 2019-02-22 14:32 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:32 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 12:24:20PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
> new file mode 100644
> index 000000000000..b149761d9661
> --- /dev/null
> +++ b/arch/arm64/include/asm/vdso/compat_barrier.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2018 ARM Limited
> + */
> +#ifndef __COMPAT_BARRIER_H
> +#define __COMPAT_BARRIER_H
> +
> +#include <asm/barrier.h>
I think it would be safer to not include this header, and explicitly
define the AArch32 barriers here in isolation.
> +
> +#if __LINUX_ARM_ARCH__ >= 8
> +#define aarch32_smp_mb() dmb(ish)
> +#define aarch32_smp_rmb() dmb(ishld)
> +#define aarch32_smp_wmb() dmb(ishst)
> +#else
> +#define aarch32_smp_mb() dmb(ish)
> +#define aarch32_smp_rmb() aarch32_smp_mb()
> +#define aarch32_smp_wmb() dmb(ishst)
> +#endif
Can't we just code an AArch32-specific version of dmb() here? AFAICT
that's the only reason we include the native barrier.h.
> +/*
> + * Warning: This code is meant to be used with
> + * ENABLE_COMPAT_VDSO only.
> + */
> +#ifndef ENABLE_COMPAT_VDSO
> +#error This header is meant to be used with ENABLE_COMPAT_VDSO only
> +#endif
> +
> +#undef smp_mb
> +#undef smp_rmb
> +#undef smp_wmb
> +
> +#define smp_mb() aarch32_smp_mb()
> +#define smp_rmb() aarch32_smp_rmb()
> +#define smp_wmb() aarch32_smp_wmb()
This undeffing looks very fragile to me.
[...]
> diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
> new file mode 100644
> index 000000000000..4fea950fa5ed
> --- /dev/null
> +++ b/arch/arm64/kernel/vdso32/.gitignore
> @@ -0,0 +1,2 @@
> +vdso.lds
> +vdso.so.raw
> diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
> new file mode 100644
> index 000000000000..ba57d2ae620f
> --- /dev/null
> +++ b/arch/arm64/kernel/vdso32/Makefile
> @@ -0,0 +1,184 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for vdso32
> +#
> +
> +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
> +# the inclusion of generic Makefile.
> +ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
> +include $(srctree)/lib/vdso/Makefile
> +
> +CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
To match HOSTCC, it might make sense to call this COMPATCC
... and likewise, CROSS_COMPILE_COMPAT
Thanks,
Mark.
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 13/23] arm64: compat: Add vDSO
@ 2019-02-22 14:32 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:32 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:20PM +0000, Vincenzo Frascino wrote:
> diff --git a/arch/arm64/include/asm/vdso/compat_barrier.h b/arch/arm64/include/asm/vdso/compat_barrier.h
> new file mode 100644
> index 000000000000..b149761d9661
> --- /dev/null
> +++ b/arch/arm64/include/asm/vdso/compat_barrier.h
> @@ -0,0 +1,36 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (C) 2018 ARM Limited
> + */
> +#ifndef __COMPAT_BARRIER_H
> +#define __COMPAT_BARRIER_H
> +
> +#include <asm/barrier.h>
I think it would be safer to not include this header, and explicitly
define the AArch32 barriers here in isolation.
> +
> +#if __LINUX_ARM_ARCH__ >= 8
> +#define aarch32_smp_mb() dmb(ish)
> +#define aarch32_smp_rmb() dmb(ishld)
> +#define aarch32_smp_wmb() dmb(ishst)
> +#else
> +#define aarch32_smp_mb() dmb(ish)
> +#define aarch32_smp_rmb() aarch32_smp_mb()
> +#define aarch32_smp_wmb() dmb(ishst)
> +#endif
Can't we just code an AArch32-specific version of dmb() here? AFAICT
that's the only reason we include the native barrier.h.
> +/*
> + * Warning: This code is meant to be used with
> + * ENABLE_COMPAT_VDSO only.
> + */
> +#ifndef ENABLE_COMPAT_VDSO
> +#error This header is meant to be used with ENABLE_COMPAT_VDSO only
> +#endif
> +
> +#undef smp_mb
> +#undef smp_rmb
> +#undef smp_wmb
> +
> +#define smp_mb() aarch32_smp_mb()
> +#define smp_rmb() aarch32_smp_rmb()
> +#define smp_wmb() aarch32_smp_wmb()
This undeffing looks very fragile to me.
[...]
> diff --git a/arch/arm64/kernel/vdso32/.gitignore b/arch/arm64/kernel/vdso32/.gitignore
> new file mode 100644
> index 000000000000..4fea950fa5ed
> --- /dev/null
> +++ b/arch/arm64/kernel/vdso32/.gitignore
> @@ -0,0 +1,2 @@
> +vdso.lds
> +vdso.so.raw
> diff --git a/arch/arm64/kernel/vdso32/Makefile b/arch/arm64/kernel/vdso32/Makefile
> new file mode 100644
> index 000000000000..ba57d2ae620f
> --- /dev/null
> +++ b/arch/arm64/kernel/vdso32/Makefile
> @@ -0,0 +1,184 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for vdso32
> +#
> +
> +# Absolute relocation type $(ARCH_REL_TYPE_ABS) needs to be defined before
> +# the inclusion of generic Makefile.
> +ARCH_REL_TYPE_ABS := R_ARM_JUMP_SLOT|R_ARM_GLOB_DAT|R_ARM_ABS32
> +include $(srctree)/lib/vdso/Makefile
> +
> +CC_ARM32 := $(CROSS_COMPILE_ARM32)gcc
To match HOSTCC, it might make sense to call this COMPATCC
... and likewise, CROSS_COMPILE_COMPAT
Thanks,
Mark.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 14:36 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-22 14:36 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Fri, Feb 22, 2019 at 2:49 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> For __vdso_timespec, I see how you ended up with this
> redefinition, and it makes the current version of your patches
> easier, but I fear it will in turn make it harder to add the
> __kernel_old_timeval based variant.
I meant __kernel_timespec here, not __kernel_old_timeval.
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 14:36 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-22 14:36 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, Feb 22, 2019 at 2:49 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> For __vdso_timespec, I see how you ended up with this
> redefinition, and it makes the current version of your patches
> easier, but I fear it will in turn make it harder to add the
> __kernel_old_timeval based variant.
I meant __kernel_timespec here, not __kernel_old_timeval.
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-22 14:36 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-22 14:36 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Fri, Feb 22, 2019 at 2:49 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino <vincenzo.frascino@arm.com> wrote:
> For __vdso_timespec, I see how you ended up with this
> redefinition, and it makes the current version of your patches
> easier, but I fear it will in turn make it harder to add the
> __kernel_old_timeval based variant.
I meant __kernel_timespec here, not __kernel_old_timeval.
Arnd
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 14/23] arm64: Refactor vDSO code
@ 2019-02-22 14:37 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:37 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:21PM +0000, Vincenzo Frascino wrote:
> Most of the code for initializing the vDSOs in arm64 and compat will
> be in common, hence a refactor of the current code is required to avoid
> duplication and simplify maintainability.
>
> Refactor vdso.c to simplify the implementation of arm64 vDSO compat
> (which will be pushed with a future patch).
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
> 1 file changed, 135 insertions(+), 73 deletions(-)
>
> diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
> index 523e56658b84..c217245768ea 100644
> --- a/arch/arm64/kernel/vdso.c
> +++ b/arch/arm64/kernel/vdso.c
> @@ -41,7 +41,30 @@
> #include <asm/vdso.h>
>
> extern char vdso_start[], vdso_end[];
> -static unsigned long vdso_pages __ro_after_init;
> +
> +/* vdso_lookup arch_index */
> +enum arch_vdso_type {
> + ARM64_VDSO = 0,
> +};
> +
> +struct __vdso_lookup_t {
If you want to give this a _t suffix, please use a typedef so that you
don't need to also say 'struct' to use it.
I think this would be better named as struct vdso_instance, or struct
vdso_abi.
Thanks,
Mark.
> + const char *name;
> + const char *vdso_code_start;
> + const char *vdso_code_end;
> + unsigned long vdso_pages;
> + /* Data Mapping */
> + struct vm_special_mapping *dm;
> + /* Code Mapping */
> + struct vm_special_mapping *cm;
> +};
> +
> +static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
> + {
> + .name = "vdso",
> + .vdso_code_start = vdso_start,
> + .vdso_code_end = vdso_end,
> + },
> +};
>
> /*
> * The vDSO data page.
> @@ -52,6 +75,106 @@ static union {
> } vdso_data_store __page_aligned_data;
> struct vdso_data *vdso_data = &vdso_data_store.data;
>
> +static int __vdso_remap(enum arch_vdso_type arch_index,
> + const struct vm_special_mapping *sm,
> + struct vm_area_struct *new_vma)
> +{
> + unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
> + unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
> + vdso_lookup[arch_index].vdso_code_start;
> +
> + if (vdso_size != new_size)
> + return -EINVAL;
> +
> + current->mm->context.vdso = (void *)new_vma->vm_start;
> +
> + return 0;
> +}
> +
> +static int __vdso_init(enum arch_vdso_type arch_index)
> +{
> + int i;
> + struct page **vdso_pagelist;
> + unsigned long pfn;
> +
> + if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
> + pr_err("vDSO is not a valid ELF object!\n");
> + return -EINVAL;
> + }
> +
> + vdso_lookup[arch_index].vdso_pages = (
> + vdso_lookup[arch_index].vdso_code_end -
> + vdso_lookup[arch_index].vdso_code_start) >>
> + PAGE_SHIFT;
> + pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
> + vdso_lookup[arch_index].name,
> + vdso_lookup[arch_index].vdso_pages + 1,
> + vdso_lookup[arch_index].vdso_pages,
> + vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
> +
> + /* Allocate the vDSO pagelist, plus a page for the data. */
> + vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
> + sizeof(struct page *),
> + GFP_KERNEL);
> + if (vdso_pagelist == NULL)
> + return -ENOMEM;
> +
> + /* Grab the vDSO data page. */
> + vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
> +
> +
> + /* Grab the vDSO code pages. */
> + pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
> +
> + for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
> + vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
> +
> + vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
> + vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
> +
> + return 0;
> +}
> +
> +static int __setup_additional_pages(enum arch_vdso_type arch_index,
> + struct mm_struct *mm,
> + struct linux_binprm *bprm,
> + int uses_interp)
> +{
> + unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
> + void *ret;
> +
> + vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
> + /* Be sure to map the data page */
> + vdso_mapping_len = vdso_text_len + PAGE_SIZE;
> +
> + vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
> + if (IS_ERR_VALUE(vdso_base)) {
> + ret = ERR_PTR(vdso_base);
> + goto up_fail;
> + }
> +
> + ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
> + VM_READ|VM_MAYREAD,
> + vdso_lookup[arch_index].dm);
> + if (IS_ERR(ret))
> + goto up_fail;
> +
> + vdso_base += PAGE_SIZE;
> + mm->context.vdso = (void *)vdso_base;
> + ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
> + VM_READ|VM_EXEC|
> + VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
> + vdso_lookup[arch_index].cm);
> + if (IS_ERR(ret))
> + goto up_fail;
> +
> + return 0;
> +
> +up_fail:
> + mm->context.vdso = NULL;
> + return PTR_ERR(ret);
> +}
> +
> #ifdef CONFIG_COMPAT
> /*
> * Create and map the vectors page for AArch32 tasks.
> @@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
> * 1 - sigreturn code
> */
> static struct page *aarch32_vdso_pages[2] __ro_after_init;
> -static const struct vm_special_mapping aarch32_vdso_spec[2] = {
> +static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
> {
> /* Must be named [vectors] for compatibility with arm. */
> .name = "[vectors]",
> @@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
> static int vdso_mremap(const struct vm_special_mapping *sm,
> struct vm_area_struct *new_vma)
> {
> - unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
> - unsigned long vdso_size = vdso_end - vdso_start;
> -
> - if (vdso_size != new_size)
> - return -EINVAL;
> -
> - current->mm->context.vdso = (void *)new_vma->vm_start;
> -
> - return 0;
> + return __vdso_remap(ARM64_VDSO, sm, new_vma);
> }
>
> static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
> @@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
>
> static int __init vdso_init(void)
> {
> - int i;
> - struct page **vdso_pagelist;
> - unsigned long pfn;
> + vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
> + vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
>
> - if (memcmp(vdso_start, "\177ELF", 4)) {
> - pr_err("vDSO is not a valid ELF object!\n");
> - return -EINVAL;
> - }
> -
> - vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
> - pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
> - vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
> -
> - /* Allocate the vDSO pagelist, plus a page for the data. */
> - vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
> - GFP_KERNEL);
> - if (vdso_pagelist == NULL)
> - return -ENOMEM;
> -
> - /* Grab the vDSO data page. */
> - vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
> -
> -
> - /* Grab the vDSO code pages. */
> - pfn = sym_to_pfn(vdso_start);
> -
> - for (i = 0; i < vdso_pages; i++)
> - vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
> -
> - vdso_spec[0].pages = &vdso_pagelist[0];
> - vdso_spec[1].pages = &vdso_pagelist[1];
> -
> - return 0;
> + return __vdso_init(ARM64_VDSO);
> }
> arch_initcall(vdso_init);
>
> @@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
> int uses_interp)
> {
> struct mm_struct *mm = current->mm;
> - unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
> - void *ret;
> -
> - vdso_text_len = vdso_pages << PAGE_SHIFT;
> - /* Be sure to map the data page */
> - vdso_mapping_len = vdso_text_len + PAGE_SIZE;
> + int ret;
>
> if (down_write_killable(&mm->mmap_sem))
> return -EINTR;
> - vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
> - if (IS_ERR_VALUE(vdso_base)) {
> - ret = ERR_PTR(vdso_base);
> - goto up_fail;
> - }
> - ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
> - VM_READ|VM_MAYREAD,
> - &vdso_spec[0]);
> - if (IS_ERR(ret))
> - goto up_fail;
> -
> - vdso_base += PAGE_SIZE;
> - mm->context.vdso = (void *)vdso_base;
> - ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
> - VM_READ|VM_EXEC|
> - VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
> - &vdso_spec[1]);
> - if (IS_ERR(ret))
> - goto up_fail;
>
> + ret = __setup_additional_pages(ARM64_VDSO,
> + mm,
> + bprm,
> + uses_interp);
>
> up_write(&mm->mmap_sem);
> - return 0;
>
> -up_fail:
> - mm->context.vdso = NULL;
> - up_write(&mm->mmap_sem);
> - return PTR_ERR(ret);
> + return ret;
> }
>
> #define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 14/23] arm64: Refactor vDSO code
@ 2019-02-22 14:37 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:37 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 12:24:21PM +0000, Vincenzo Frascino wrote:
> Most of the code for initializing the vDSOs in arm64 and compat will
> be in common, hence a refactor of the current code is required to avoid
> duplication and simplify maintainability.
>
> Refactor vdso.c to simplify the implementation of arm64 vDSO compat
> (which will be pushed with a future patch).
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
> 1 file changed, 135 insertions(+), 73 deletions(-)
>
> diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
> index 523e56658b84..c217245768ea 100644
> --- a/arch/arm64/kernel/vdso.c
> +++ b/arch/arm64/kernel/vdso.c
> @@ -41,7 +41,30 @@
> #include <asm/vdso.h>
>
> extern char vdso_start[], vdso_end[];
> -static unsigned long vdso_pages __ro_after_init;
> +
> +/* vdso_lookup arch_index */
> +enum arch_vdso_type {
> + ARM64_VDSO = 0,
> +};
> +
> +struct __vdso_lookup_t {
If you want to give this a _t suffix, please use a typedef so that you
don't need to also say 'struct' to use it.
I think this would be better named as struct vdso_instance, or struct
vdso_abi.
Thanks,
Mark.
> + const char *name;
> + const char *vdso_code_start;
> + const char *vdso_code_end;
> + unsigned long vdso_pages;
> + /* Data Mapping */
> + struct vm_special_mapping *dm;
> + /* Code Mapping */
> + struct vm_special_mapping *cm;
> +};
> +
> +static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
> + {
> + .name = "vdso",
> + .vdso_code_start = vdso_start,
> + .vdso_code_end = vdso_end,
> + },
> +};
>
> /*
> * The vDSO data page.
> @@ -52,6 +75,106 @@ static union {
> } vdso_data_store __page_aligned_data;
> struct vdso_data *vdso_data = &vdso_data_store.data;
>
> +static int __vdso_remap(enum arch_vdso_type arch_index,
> + const struct vm_special_mapping *sm,
> + struct vm_area_struct *new_vma)
> +{
> + unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
> + unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
> + vdso_lookup[arch_index].vdso_code_start;
> +
> + if (vdso_size != new_size)
> + return -EINVAL;
> +
> + current->mm->context.vdso = (void *)new_vma->vm_start;
> +
> + return 0;
> +}
> +
> +static int __vdso_init(enum arch_vdso_type arch_index)
> +{
> + int i;
> + struct page **vdso_pagelist;
> + unsigned long pfn;
> +
> + if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
> + pr_err("vDSO is not a valid ELF object!\n");
> + return -EINVAL;
> + }
> +
> + vdso_lookup[arch_index].vdso_pages = (
> + vdso_lookup[arch_index].vdso_code_end -
> + vdso_lookup[arch_index].vdso_code_start) >>
> + PAGE_SHIFT;
> + pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
> + vdso_lookup[arch_index].name,
> + vdso_lookup[arch_index].vdso_pages + 1,
> + vdso_lookup[arch_index].vdso_pages,
> + vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
> +
> + /* Allocate the vDSO pagelist, plus a page for the data. */
> + vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
> + sizeof(struct page *),
> + GFP_KERNEL);
> + if (vdso_pagelist == NULL)
> + return -ENOMEM;
> +
> + /* Grab the vDSO data page. */
> + vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
> +
> +
> + /* Grab the vDSO code pages. */
> + pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
> +
> + for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
> + vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
> +
> + vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
> + vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
> +
> + return 0;
> +}
> +
> +static int __setup_additional_pages(enum arch_vdso_type arch_index,
> + struct mm_struct *mm,
> + struct linux_binprm *bprm,
> + int uses_interp)
> +{
> + unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
> + void *ret;
> +
> + vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
> + /* Be sure to map the data page */
> + vdso_mapping_len = vdso_text_len + PAGE_SIZE;
> +
> + vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
> + if (IS_ERR_VALUE(vdso_base)) {
> + ret = ERR_PTR(vdso_base);
> + goto up_fail;
> + }
> +
> + ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
> + VM_READ|VM_MAYREAD,
> + vdso_lookup[arch_index].dm);
> + if (IS_ERR(ret))
> + goto up_fail;
> +
> + vdso_base += PAGE_SIZE;
> + mm->context.vdso = (void *)vdso_base;
> + ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
> + VM_READ|VM_EXEC|
> + VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
> + vdso_lookup[arch_index].cm);
> + if (IS_ERR(ret))
> + goto up_fail;
> +
> + return 0;
> +
> +up_fail:
> + mm->context.vdso = NULL;
> + return PTR_ERR(ret);
> +}
> +
> #ifdef CONFIG_COMPAT
> /*
> * Create and map the vectors page for AArch32 tasks.
> @@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
> * 1 - sigreturn code
> */
> static struct page *aarch32_vdso_pages[2] __ro_after_init;
> -static const struct vm_special_mapping aarch32_vdso_spec[2] = {
> +static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
> {
> /* Must be named [vectors] for compatibility with arm. */
> .name = "[vectors]",
> @@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
> static int vdso_mremap(const struct vm_special_mapping *sm,
> struct vm_area_struct *new_vma)
> {
> - unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
> - unsigned long vdso_size = vdso_end - vdso_start;
> -
> - if (vdso_size != new_size)
> - return -EINVAL;
> -
> - current->mm->context.vdso = (void *)new_vma->vm_start;
> -
> - return 0;
> + return __vdso_remap(ARM64_VDSO, sm, new_vma);
> }
>
> static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
> @@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
>
> static int __init vdso_init(void)
> {
> - int i;
> - struct page **vdso_pagelist;
> - unsigned long pfn;
> + vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
> + vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
>
> - if (memcmp(vdso_start, "\177ELF", 4)) {
> - pr_err("vDSO is not a valid ELF object!\n");
> - return -EINVAL;
> - }
> -
> - vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
> - pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
> - vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
> -
> - /* Allocate the vDSO pagelist, plus a page for the data. */
> - vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
> - GFP_KERNEL);
> - if (vdso_pagelist == NULL)
> - return -ENOMEM;
> -
> - /* Grab the vDSO data page. */
> - vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
> -
> -
> - /* Grab the vDSO code pages. */
> - pfn = sym_to_pfn(vdso_start);
> -
> - for (i = 0; i < vdso_pages; i++)
> - vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
> -
> - vdso_spec[0].pages = &vdso_pagelist[0];
> - vdso_spec[1].pages = &vdso_pagelist[1];
> -
> - return 0;
> + return __vdso_init(ARM64_VDSO);
> }
> arch_initcall(vdso_init);
>
> @@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
> int uses_interp)
> {
> struct mm_struct *mm = current->mm;
> - unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
> - void *ret;
> -
> - vdso_text_len = vdso_pages << PAGE_SHIFT;
> - /* Be sure to map the data page */
> - vdso_mapping_len = vdso_text_len + PAGE_SIZE;
> + int ret;
>
> if (down_write_killable(&mm->mmap_sem))
> return -EINTR;
> - vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
> - if (IS_ERR_VALUE(vdso_base)) {
> - ret = ERR_PTR(vdso_base);
> - goto up_fail;
> - }
> - ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
> - VM_READ|VM_MAYREAD,
> - &vdso_spec[0]);
> - if (IS_ERR(ret))
> - goto up_fail;
> -
> - vdso_base += PAGE_SIZE;
> - mm->context.vdso = (void *)vdso_base;
> - ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
> - VM_READ|VM_EXEC|
> - VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
> - &vdso_spec[1]);
> - if (IS_ERR(ret))
> - goto up_fail;
>
> + ret = __setup_additional_pages(ARM64_VDSO,
> + mm,
> + bprm,
> + uses_interp);
>
> up_write(&mm->mmap_sem);
> - return 0;
>
> -up_fail:
> - mm->context.vdso = NULL;
> - up_write(&mm->mmap_sem);
> - return PTR_ERR(ret);
> + return ret;
> }
>
> #define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 14/23] arm64: Refactor vDSO code
@ 2019-02-22 14:37 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:37 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:21PM +0000, Vincenzo Frascino wrote:
> Most of the code for initializing the vDSOs in arm64 and compat will
> be in common, hence a refactor of the current code is required to avoid
> duplication and simplify maintainability.
>
> Refactor vdso.c to simplify the implementation of arm64 vDSO compat
> (which will be pushed with a future patch).
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/kernel/vdso.c | 208 +++++++++++++++++++++++++--------------
> 1 file changed, 135 insertions(+), 73 deletions(-)
>
> diff --git a/arch/arm64/kernel/vdso.c b/arch/arm64/kernel/vdso.c
> index 523e56658b84..c217245768ea 100644
> --- a/arch/arm64/kernel/vdso.c
> +++ b/arch/arm64/kernel/vdso.c
> @@ -41,7 +41,30 @@
> #include <asm/vdso.h>
>
> extern char vdso_start[], vdso_end[];
> -static unsigned long vdso_pages __ro_after_init;
> +
> +/* vdso_lookup arch_index */
> +enum arch_vdso_type {
> + ARM64_VDSO = 0,
> +};
> +
> +struct __vdso_lookup_t {
If you want to give this a _t suffix, please use a typedef so that you
don't need to also say 'struct' to use it.
I think this would be better named as struct vdso_instance, or struct
vdso_abi.
Thanks,
Mark.
> + const char *name;
> + const char *vdso_code_start;
> + const char *vdso_code_end;
> + unsigned long vdso_pages;
> + /* Data Mapping */
> + struct vm_special_mapping *dm;
> + /* Code Mapping */
> + struct vm_special_mapping *cm;
> +};
> +
> +static struct __vdso_lookup_t vdso_lookup[2] __ro_after_init = {
> + {
> + .name = "vdso",
> + .vdso_code_start = vdso_start,
> + .vdso_code_end = vdso_end,
> + },
> +};
>
> /*
> * The vDSO data page.
> @@ -52,6 +75,106 @@ static union {
> } vdso_data_store __page_aligned_data;
> struct vdso_data *vdso_data = &vdso_data_store.data;
>
> +static int __vdso_remap(enum arch_vdso_type arch_index,
> + const struct vm_special_mapping *sm,
> + struct vm_area_struct *new_vma)
> +{
> + unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
> + unsigned long vdso_size = vdso_lookup[arch_index].vdso_code_end -
> + vdso_lookup[arch_index].vdso_code_start;
> +
> + if (vdso_size != new_size)
> + return -EINVAL;
> +
> + current->mm->context.vdso = (void *)new_vma->vm_start;
> +
> + return 0;
> +}
> +
> +static int __vdso_init(enum arch_vdso_type arch_index)
> +{
> + int i;
> + struct page **vdso_pagelist;
> + unsigned long pfn;
> +
> + if (memcmp(vdso_lookup[arch_index].vdso_code_start, "\177ELF", 4)) {
> + pr_err("vDSO is not a valid ELF object!\n");
> + return -EINVAL;
> + }
> +
> + vdso_lookup[arch_index].vdso_pages = (
> + vdso_lookup[arch_index].vdso_code_end -
> + vdso_lookup[arch_index].vdso_code_start) >>
> + PAGE_SHIFT;
> + pr_info("%s: %ld pages (%ld code @ %p, %ld data @ %p)\n",
> + vdso_lookup[arch_index].name,
> + vdso_lookup[arch_index].vdso_pages + 1,
> + vdso_lookup[arch_index].vdso_pages,
> + vdso_lookup[arch_index].vdso_code_start, 1L, vdso_data);
> +
> + /* Allocate the vDSO pagelist, plus a page for the data. */
> + vdso_pagelist = kcalloc(vdso_lookup[arch_index].vdso_pages + 1,
> + sizeof(struct page *),
> + GFP_KERNEL);
> + if (vdso_pagelist == NULL)
> + return -ENOMEM;
> +
> + /* Grab the vDSO data page. */
> + vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
> +
> +
> + /* Grab the vDSO code pages. */
> + pfn = sym_to_pfn(vdso_lookup[arch_index].vdso_code_start);
> +
> + for (i = 0; i < vdso_lookup[arch_index].vdso_pages; i++)
> + vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
> +
> + vdso_lookup[arch_index].dm->pages = &vdso_pagelist[0];
> + vdso_lookup[arch_index].cm->pages = &vdso_pagelist[1];
> +
> + return 0;
> +}
> +
> +static int __setup_additional_pages(enum arch_vdso_type arch_index,
> + struct mm_struct *mm,
> + struct linux_binprm *bprm,
> + int uses_interp)
> +{
> + unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
> + void *ret;
> +
> + vdso_text_len = vdso_lookup[arch_index].vdso_pages << PAGE_SHIFT;
> + /* Be sure to map the data page */
> + vdso_mapping_len = vdso_text_len + PAGE_SIZE;
> +
> + vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
> + if (IS_ERR_VALUE(vdso_base)) {
> + ret = ERR_PTR(vdso_base);
> + goto up_fail;
> + }
> +
> + ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
> + VM_READ|VM_MAYREAD,
> + vdso_lookup[arch_index].dm);
> + if (IS_ERR(ret))
> + goto up_fail;
> +
> + vdso_base += PAGE_SIZE;
> + mm->context.vdso = (void *)vdso_base;
> + ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
> + VM_READ|VM_EXEC|
> + VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
> + vdso_lookup[arch_index].cm);
> + if (IS_ERR(ret))
> + goto up_fail;
> +
> + return 0;
> +
> +up_fail:
> + mm->context.vdso = NULL;
> + return PTR_ERR(ret);
> +}
> +
> #ifdef CONFIG_COMPAT
> /*
> * Create and map the vectors page for AArch32 tasks.
> @@ -62,7 +185,7 @@ struct vdso_data *vdso_data = &vdso_data_store.data;
> * 1 - sigreturn code
> */
> static struct page *aarch32_vdso_pages[2] __ro_after_init;
> -static const struct vm_special_mapping aarch32_vdso_spec[2] = {
> +static struct vm_special_mapping aarch32_vdso_spec[2] __ro_after_init = {
> {
> /* Must be named [vectors] for compatibility with arm. */
> .name = "[vectors]",
> @@ -202,15 +325,7 @@ int aarch32_setup_additional_pages(struct linux_binprm *bprm, int uses_interp)
> static int vdso_mremap(const struct vm_special_mapping *sm,
> struct vm_area_struct *new_vma)
> {
> - unsigned long new_size = new_vma->vm_end - new_vma->vm_start;
> - unsigned long vdso_size = vdso_end - vdso_start;
> -
> - if (vdso_size != new_size)
> - return -EINVAL;
> -
> - current->mm->context.vdso = (void *)new_vma->vm_start;
> -
> - return 0;
> + return __vdso_remap(ARM64_VDSO, sm, new_vma);
> }
>
> static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
> @@ -225,39 +340,10 @@ static struct vm_special_mapping vdso_spec[2] __ro_after_init = {
>
> static int __init vdso_init(void)
> {
> - int i;
> - struct page **vdso_pagelist;
> - unsigned long pfn;
> + vdso_lookup[ARM64_VDSO].dm = &vdso_spec[0];
> + vdso_lookup[ARM64_VDSO].cm = &vdso_spec[1];
>
> - if (memcmp(vdso_start, "\177ELF", 4)) {
> - pr_err("vDSO is not a valid ELF object!\n");
> - return -EINVAL;
> - }
> -
> - vdso_pages = (vdso_end - vdso_start) >> PAGE_SHIFT;
> - pr_info("vdso: %ld pages (%ld code @ %p, %ld data @ %p)\n",
> - vdso_pages + 1, vdso_pages, vdso_start, 1L, vdso_data);
> -
> - /* Allocate the vDSO pagelist, plus a page for the data. */
> - vdso_pagelist = kcalloc(vdso_pages + 1, sizeof(struct page *),
> - GFP_KERNEL);
> - if (vdso_pagelist == NULL)
> - return -ENOMEM;
> -
> - /* Grab the vDSO data page. */
> - vdso_pagelist[0] = phys_to_page(__pa_symbol(vdso_data));
> -
> -
> - /* Grab the vDSO code pages. */
> - pfn = sym_to_pfn(vdso_start);
> -
> - for (i = 0; i < vdso_pages; i++)
> - vdso_pagelist[i + 1] = pfn_to_page(pfn + i);
> -
> - vdso_spec[0].pages = &vdso_pagelist[0];
> - vdso_spec[1].pages = &vdso_pagelist[1];
> -
> - return 0;
> + return __vdso_init(ARM64_VDSO);
> }
> arch_initcall(vdso_init);
>
> @@ -265,43 +351,19 @@ int arch_setup_additional_pages(struct linux_binprm *bprm,
> int uses_interp)
> {
> struct mm_struct *mm = current->mm;
> - unsigned long vdso_base, vdso_text_len, vdso_mapping_len;
> - void *ret;
> -
> - vdso_text_len = vdso_pages << PAGE_SHIFT;
> - /* Be sure to map the data page */
> - vdso_mapping_len = vdso_text_len + PAGE_SIZE;
> + int ret;
>
> if (down_write_killable(&mm->mmap_sem))
> return -EINTR;
> - vdso_base = get_unmapped_area(NULL, 0, vdso_mapping_len, 0, 0);
> - if (IS_ERR_VALUE(vdso_base)) {
> - ret = ERR_PTR(vdso_base);
> - goto up_fail;
> - }
> - ret = _install_special_mapping(mm, vdso_base, PAGE_SIZE,
> - VM_READ|VM_MAYREAD,
> - &vdso_spec[0]);
> - if (IS_ERR(ret))
> - goto up_fail;
> -
> - vdso_base += PAGE_SIZE;
> - mm->context.vdso = (void *)vdso_base;
> - ret = _install_special_mapping(mm, vdso_base, vdso_text_len,
> - VM_READ|VM_EXEC|
> - VM_MAYREAD|VM_MAYWRITE|VM_MAYEXEC,
> - &vdso_spec[1]);
> - if (IS_ERR(ret))
> - goto up_fail;
>
> + ret = __setup_additional_pages(ARM64_VDSO,
> + mm,
> + bprm,
> + uses_interp);
>
> up_write(&mm->mmap_sem);
> - return 0;
>
> -up_fail:
> - mm->context.vdso = NULL;
> - up_write(&mm->mmap_sem);
> - return PTR_ERR(ret);
> + return ret;
> }
>
> #define VDSO_PRECISION_MASK ~(0xFF00ULL<<48)
> --
> 2.20.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 14:39 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:39 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:26PM +0000, Vincenzo Frascino wrote:
> Enable compat vDSO support in the arm64 defconfig.
>
> Note that the default compiler is set to arm-linux-gnueabihf-.
> For this to take effect and for the compat vDSO to compile correctly, it
> is required that this compiler is in the PATH.
I do not have such a compiler in my path by default.
I hope that we handle that gracefully, rather than causing a build
failure.
Can't we detect the presence of the compat compiler at config time, as
happens for the native compiler?
Thanks,
Mark.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/configs/defconfig | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> index c8432e24207e..3adebe5f3573 100644
> --- a/arch/arm64/configs/defconfig
> +++ b/arch/arm64/configs/defconfig
> @@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
> CONFIG_ACPI_APEI_EINJ=y
> CONFIG_VIRTUALIZATION=y
> CONFIG_KVM=y
> +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
> CONFIG_ARM64_CRYPTO=y
> CONFIG_CRYPTO_SHA1_ARM64_CE=y
> CONFIG_CRYPTO_SHA2_ARM64_CE=y
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 14:39 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:39 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 12:24:26PM +0000, Vincenzo Frascino wrote:
> Enable compat vDSO support in the arm64 defconfig.
>
> Note that the default compiler is set to arm-linux-gnueabihf-.
> For this to take effect and for the compat vDSO to compile correctly, it
> is required that this compiler is in the PATH.
I do not have such a compiler in my path by default.
I hope that we handle that gracefully, rather than causing a build
failure.
Can't we detect the presence of the compat compiler at config time, as
happens for the native compiler?
Thanks,
Mark.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/configs/defconfig | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> index c8432e24207e..3adebe5f3573 100644
> --- a/arch/arm64/configs/defconfig
> +++ b/arch/arm64/configs/defconfig
> @@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
> CONFIG_ACPI_APEI_EINJ=y
> CONFIG_VIRTUALIZATION=y
> CONFIG_KVM=y
> +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
> CONFIG_ARM64_CRYPTO=y
> CONFIG_CRYPTO_SHA1_ARM64_CE=y
> CONFIG_CRYPTO_SHA2_ARM64_CE=y
> --
> 2.20.1
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 14:39 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:39 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 12:24:26PM +0000, Vincenzo Frascino wrote:
> Enable compat vDSO support in the arm64 defconfig.
>
> Note that the default compiler is set to arm-linux-gnueabihf-.
> For this to take effect and for the compat vDSO to compile correctly, it
> is required that this compiler is in the PATH.
I do not have such a compiler in my path by default.
I hope that we handle that gracefully, rather than causing a build
failure.
Can't we detect the presence of the compat compiler at config time, as
happens for the native compiler?
Thanks,
Mark.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> ---
> arch/arm64/configs/defconfig | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> index c8432e24207e..3adebe5f3573 100644
> --- a/arch/arm64/configs/defconfig
> +++ b/arch/arm64/configs/defconfig
> @@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
> CONFIG_ACPI_APEI_EINJ=y
> CONFIG_VIRTUALIZATION=y
> CONFIG_KVM=y
> +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
> CONFIG_ARM64_CRYPTO=y
> CONFIG_CRYPTO_SHA1_ARM64_CE=y
> CONFIG_CRYPTO_SHA2_ARM64_CE=y
> --
> 2.20.1
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 14:41 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:41 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 02:39:58PM +0000, Mark Rutland wrote:
> On Fri, Feb 22, 2019 at 12:24:26PM +0000, Vincenzo Frascino wrote:
> > Enable compat vDSO support in the arm64 defconfig.
> >
> > Note that the default compiler is set to arm-linux-gnueabihf-.
> > For this to take effect and for the compat vDSO to compile correctly, it
> > is required that this compiler is in the PATH.
>
> I do not have such a compiler in my path by default.
>
> I hope that we handle that gracefully, rather than causing a build
> failure.
>
> Can't we detect the presence of the compat compiler at config time, as
> happens for the native compiler?
>
> Thanks,
> Mark.
>
> >
> > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > Cc: Will Deacon <will.deacon@arm.com>
> > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > ---
> > arch/arm64/configs/defconfig | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> > index c8432e24207e..3adebe5f3573 100644
> > --- a/arch/arm64/configs/defconfig
> > +++ b/arch/arm64/configs/defconfig
> > @@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
> > CONFIG_ACPI_APEI_EINJ=y
> > CONFIG_VIRTUALIZATION=y
> > CONFIG_KVM=y
> > +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
Also, the kernel.org crosstool compilers are arm-linux-gnueabi-.
Could we please avoid having a default, in the same way that we don't
have a default for CROSS_COMPILE?
Otherwise, it's always gongi to be wrong for someone, and we'll never be
able to change it anyway.
Thanks,
Mark.
> > CONFIG_ARM64_CRYPTO=y
> > CONFIG_CRYPTO_SHA1_ARM64_CE=y
> > CONFIG_CRYPTO_SHA2_ARM64_CE=y
> > --
> > 2.20.1
> >
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 14:41 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:41 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
On Fri, Feb 22, 2019 at 02:39:58PM +0000, Mark Rutland wrote:
> On Fri, Feb 22, 2019 at 12:24:26PM +0000, Vincenzo Frascino wrote:
> > Enable compat vDSO support in the arm64 defconfig.
> >
> > Note that the default compiler is set to arm-linux-gnueabihf-.
> > For this to take effect and for the compat vDSO to compile correctly, it
> > is required that this compiler is in the PATH.
>
> I do not have such a compiler in my path by default.
>
> I hope that we handle that gracefully, rather than causing a build
> failure.
>
> Can't we detect the presence of the compat compiler at config time, as
> happens for the native compiler?
>
> Thanks,
> Mark.
>
> >
> > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > Cc: Will Deacon <will.deacon@arm.com>
> > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > ---
> > arch/arm64/configs/defconfig | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> > index c8432e24207e..3adebe5f3573 100644
> > --- a/arch/arm64/configs/defconfig
> > +++ b/arch/arm64/configs/defconfig
> > @@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
> > CONFIG_ACPI_APEI_EINJ=y
> > CONFIG_VIRTUALIZATION=y
> > CONFIG_KVM=y
> > +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
Also, the kernel.org crosstool compilers are arm-linux-gnueabi-.
Could we please avoid having a default, in the same way that we don't
have a default for CROSS_COMPILE?
Otherwise, it's always gongi to be wrong for someone, and we'll never be
able to change it anyway.
Thanks,
Mark.
> > CONFIG_ARM64_CRYPTO=y
> > CONFIG_CRYPTO_SHA1_ARM64_CE=y
> > CONFIG_CRYPTO_SHA2_ARM64_CE=y
> > --
> > 2.20.1
> >
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 19/23] arm64: Enable compat vDSO support
@ 2019-02-22 14:41 ` Mark Rutland
0 siblings, 0 replies; 178+ messages in thread
From: Mark Rutland @ 2019-02-22 14:41 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
On Fri, Feb 22, 2019 at 02:39:58PM +0000, Mark Rutland wrote:
> On Fri, Feb 22, 2019 at 12:24:26PM +0000, Vincenzo Frascino wrote:
> > Enable compat vDSO support in the arm64 defconfig.
> >
> > Note that the default compiler is set to arm-linux-gnueabihf-.
> > For this to take effect and for the compat vDSO to compile correctly, it
> > is required that this compiler is in the PATH.
>
> I do not have such a compiler in my path by default.
>
> I hope that we handle that gracefully, rather than causing a build
> failure.
>
> Can't we detect the presence of the compat compiler at config time, as
> happens for the native compiler?
>
> Thanks,
> Mark.
>
> >
> > Cc: Catalin Marinas <catalin.marinas@arm.com>
> > Cc: Will Deacon <will.deacon@arm.com>
> > Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
> > ---
> > arch/arm64/configs/defconfig | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
> > index c8432e24207e..3adebe5f3573 100644
> > --- a/arch/arm64/configs/defconfig
> > +++ b/arch/arm64/configs/defconfig
> > @@ -120,6 +120,7 @@ CONFIG_ACPI_APEI_MEMORY_FAILURE=y
> > CONFIG_ACPI_APEI_EINJ=y
> > CONFIG_VIRTUALIZATION=y
> > CONFIG_KVM=y
> > +CONFIG_CROSS_COMPILE_COMPAT_VDSO="arm-linux-gnueabihf-"
Also, the kernel.org crosstool compilers are arm-linux-gnueabi-.
Could we please avoid having a default, in the same way that we don't
have a default for CROSS_COMPILE?
Otherwise, it's always gongi to be wrong for someone, and we'll never be
able to change it anyway.
Thanks,
Mark.
> > CONFIG_ARM64_CRYPTO=y
> > CONFIG_CRYPTO_SHA1_ARM64_CE=y
> > CONFIG_CRYPTO_SHA2_ARM64_CE=y
> > --
> > 2.20.1
> >
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-23 10:34 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 10:34 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +static notrace int __cvdso_clock_getres(clockid_t clock,
> + struct __vdso_timespec *res)
> +{
> + u64 sec, ns;
> + u32 msk;
> +
> + /* Check for negative values or invalid clocks */
> + if (unlikely((u32) clock >= MAX_CLOCKS))
> + goto fallback;
> +
> + /*
> + * Convert the clockid to a bitmask and use it to check which
> + * clocks are handled in the VDSO directly.
> + */
> + msk = 1U << clock;
> + if (msk & VDSO_HRES) {
> + /*
> + * Preserves the behaviour of posix_get_hrtimer_res().
> + */
So much for the theory.
> + sec = 0;
> + ns = MONOTONIC_RES_NSEC;
posix_get_hrtimer_res() does:
sec = 0;
ns = hrtimer_resolution;
and hrtimer_resolution depends on the enablement of high resolution timers
either compile or run time.
So you need to have a copy of hrtimer_resolution in the vdso data and use
that.
> + } else if (msk & VDSO_COARSE) {
> + /*
> + * Preserves the behaviour of posix_get_coarse_res().
> + */
> + ns = LOW_RES_NSEC;
> + sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
Do we allow CONFIG_HZ = 1?
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-23 10:34 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 10:34 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +static notrace int __cvdso_clock_getres(clockid_t clock,
> + struct __vdso_timespec *res)
> +{
> + u64 sec, ns;
> + u32 msk;
> +
> + /* Check for negative values or invalid clocks */
> + if (unlikely((u32) clock >= MAX_CLOCKS))
> + goto fallback;
> +
> + /*
> + * Convert the clockid to a bitmask and use it to check which
> + * clocks are handled in the VDSO directly.
> + */
> + msk = 1U << clock;
> + if (msk & VDSO_HRES) {
> + /*
> + * Preserves the behaviour of posix_get_hrtimer_res().
> + */
So much for the theory.
> + sec = 0;
> + ns = MONOTONIC_RES_NSEC;
posix_get_hrtimer_res() does:
sec = 0;
ns = hrtimer_resolution;
and hrtimer_resolution depends on the enablement of high resolution timers
either compile or run time.
So you need to have a copy of hrtimer_resolution in the vdso data and use
that.
> + } else if (msk & VDSO_COARSE) {
> + /*
> + * Preserves the behaviour of posix_get_coarse_res().
> + */
> + ns = LOW_RES_NSEC;
> + sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
Do we allow CONFIG_HZ = 1?
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-23 10:34 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 10:34 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +static notrace int __cvdso_clock_getres(clockid_t clock,
> + struct __vdso_timespec *res)
> +{
> + u64 sec, ns;
> + u32 msk;
> +
> + /* Check for negative values or invalid clocks */
> + if (unlikely((u32) clock >= MAX_CLOCKS))
> + goto fallback;
> +
> + /*
> + * Convert the clockid to a bitmask and use it to check which
> + * clocks are handled in the VDSO directly.
> + */
> + msk = 1U << clock;
> + if (msk & VDSO_HRES) {
> + /*
> + * Preserves the behaviour of posix_get_hrtimer_res().
> + */
So much for the theory.
> + sec = 0;
> + ns = MONOTONIC_RES_NSEC;
posix_get_hrtimer_res() does:
sec = 0;
ns = hrtimer_resolution;
and hrtimer_resolution depends on the enablement of high resolution timers
either compile or run time.
So you need to have a copy of hrtimer_resolution in the vdso data and use
that.
> + } else if (msk & VDSO_COARSE) {
> + /*
> + * Preserves the behaviour of posix_get_coarse_res().
> + */
> + ns = LOW_RES_NSEC;
> + sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
Do we allow CONFIG_HZ = 1?
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-23 10:45 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 10:45 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> void update_vsyscall_tz(void)
> {
> - vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
> - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
> + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
> + vdso_data.tz_dsttime = sys_tz.tz_dsttime;
> }
>
> void update_vsyscall(struct timekeeper *tk)
> {
> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
> - struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
> - struct vgtod_ts *base;
> + struct vdso_data *vdata = &vdso_data;
> + struct vdso_timestamp *vdso_ts;
> u64 nsec;
>
> /* Mark the new vclock used. */
> BUILD_BUG_ON(VCLOCK_MAX >= 32);
> WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
>
> - gtod_write_begin(vdata);
> + vdso_write_begin(vdata);
This begs the question why this update code is still architecture
specific. The vdso data is now generic, so the whole update can be generic
as well. The only x86 specific thing is this perhaps:
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
vdata->clock_mode = vclock_mode;
But either than can be made generic and mandatory or easily resolved with a
trivial arch_update_vsyscall() inline.
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-23 10:45 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 10:45 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> void update_vsyscall_tz(void)
> {
> - vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
> - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
> + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
> + vdso_data.tz_dsttime = sys_tz.tz_dsttime;
> }
>
> void update_vsyscall(struct timekeeper *tk)
> {
> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
> - struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
> - struct vgtod_ts *base;
> + struct vdso_data *vdata = &vdso_data;
> + struct vdso_timestamp *vdso_ts;
> u64 nsec;
>
> /* Mark the new vclock used. */
> BUILD_BUG_ON(VCLOCK_MAX >= 32);
> WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
>
> - gtod_write_begin(vdata);
> + vdso_write_begin(vdata);
This begs the question why this update code is still architecture
specific. The vdso data is now generic, so the whole update can be generic
as well. The only x86 specific thing is this perhaps:
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
vdata->clock_mode = vclock_mode;
But either than can be made generic and mandatory or easily resolved with a
trivial arch_update_vsyscall() inline.
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-23 10:45 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 10:45 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> void update_vsyscall_tz(void)
> {
> - vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
> - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
> + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
> + vdso_data.tz_dsttime = sys_tz.tz_dsttime;
> }
>
> void update_vsyscall(struct timekeeper *tk)
> {
> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
> - struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
> - struct vgtod_ts *base;
> + struct vdso_data *vdata = &vdso_data;
> + struct vdso_timestamp *vdso_ts;
> u64 nsec;
>
> /* Mark the new vclock used. */
> BUILD_BUG_ON(VCLOCK_MAX >= 32);
> WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
>
> - gtod_write_begin(vdata);
> + vdso_write_begin(vdata);
This begs the question why this update code is still architecture
specific. The vdso data is now generic, so the whole update can be generic
as well. The only x86 specific thing is this perhaps:
int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
vdata->clock_mode = vclock_mode;
But either than can be made generic and mandatory or easily resolved with a
trivial arch_update_vsyscall() inline.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-23 16:51 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 16:51 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +/*
> + * There is one vdso_clocksource object in vvar for each vDSO clocksource
> + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
> + * and optimal in terms of access pattern.
> + *
> + * Note that mask and shift are the same for mono and raw.
> + */
> +struct vdso_clocksource {
> + u64 mask; /* Clocksource mask */
> + u32 mult; /* Clocksource multiplier */
> + u32 shift; /* Clocksource shift */
Can you please get rid of the tail comments and use proper kerneldoc
format?
> +/*
> + * vdso_data will be accessed by 32 and 64 bit code at the same time
> + * so we should be careful before modifying this structure.
> + */
> +struct vdso_data {
> + u32 seq; /* Timebase sequence counter */
> +
> + s32 clock_mode;
> + u64 cycle_last; /* Timebase at clocksource init */
> +
> + struct vdso_clocksource cs[CLOCKSOURCE_BASES];
Why would you need different clocksource parameters? That really bloats the
data structure and makes the cache access pattern worse. Also the vdso
update needs to copy the same data over and over for no value.
The only clock ID which needs a different mult/shift would be
MONOTONIC_RAW, but if we expose that through the VDSO then we really can be
smarter than this. See incomplete and uncompilable patch below for
reference. You get the idea.
> + struct vdso_timestamp basetime[VDSO_BASES];
> +
> + s32 tz_minuteswest; /* Timezone definitions */
> + s32 tz_dsttime;
Also please keep the tabular alignment of the members
u32 foo;
struct bar bar;
that's way better to read than
u32 foo;
struct bar bar;
Thanks
tglx
8<-----------------
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -139,9 +139,10 @@ notrace static inline u64 vgetcyc(int mo
return U64_MAX;
}
-notrace static int do_hres(clockid_t clk, struct timespec *ts)
+notrace static int do_hres(clockid_t clk, struct timespec *ts,
+ struct vsyscall_gtod_data *vdata)
{
- struct vgtod_ts *base = >od->basetime[clk];
+ struct vgtod_ts *base = &vdata->basetime[clk];
u64 cycles, last, sec, ns;
unsigned int seq;
@@ -168,9 +169,10 @@ notrace static int do_hres(clockid_t clk
return 0;
}
-notrace static void do_coarse(clockid_t clk, struct timespec *ts)
+notrace static void do_coarse(clockid_t clk, struct timespec *ts,
+ struct vsyscall_gtod_data *vdata)
{
- struct vgtod_ts *base = >od->basetime[clk];
+ struct vgtod_ts *base = &vdata->basetime[clk];
unsigned int seq;
do {
@@ -194,10 +196,12 @@ notrace int __vdso_clock_gettime(clockid
*/
msk = 1U << clock;
if (likely(msk & VGTOD_HRES)) {
- return do_hres(clock, ts);
+ return do_hres(clock, ts, >od[0]);
} else if (msk & VGTOD_COARSE) {
- do_coarse(clock, ts);
+ do_coarse(clock, ts, >od[0]);
return 0;
+ } else if (msk & VGTOD_RAW) {
+ return do_hres(clock, ts, >od[1]);
}
return vdso_fallback_gettime(clock, ts);
}
@@ -210,7 +214,7 @@ notrace int __vdso_gettimeofday(struct t
if (likely(tv != NULL)) {
struct timespec *ts = (struct timespec *) tv;
- do_hres(CLOCK_REALTIME, ts);
+ do_hres(CLOCK_REALTIME, ts, >od[0]);
tv->tv_usec /= 1000;
}
if (unlikely(tz != NULL)) {
--- a/arch/x86/include/asm/vgtod.h
+++ b/arch/x86/include/asm/vgtod.h
@@ -30,6 +30,7 @@ struct vgtod_ts {
#define VGTOD_BASES (CLOCK_TAI + 1)
#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))
#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE))
+#define VGTOD_RAW BIT(CLOCK_MOTONONIC_RAW)
/*
* vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
@@ -49,7 +50,8 @@ struct vsyscall_gtod_data {
int tz_minuteswest;
int tz_dsttime;
};
-extern struct vsyscall_gtod_data vsyscall_gtod_data;
+
+extern struct vsyscall_gtod_data vsyscall_gtod_data[2];
extern int vclocks_used;
static inline bool vclock_was_used(int vclock)
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-23 16:51 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 16:51 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +/*
> + * There is one vdso_clocksource object in vvar for each vDSO clocksource
> + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
> + * and optimal in terms of access pattern.
> + *
> + * Note that mask and shift are the same for mono and raw.
> + */
> +struct vdso_clocksource {
> + u64 mask; /* Clocksource mask */
> + u32 mult; /* Clocksource multiplier */
> + u32 shift; /* Clocksource shift */
Can you please get rid of the tail comments and use proper kerneldoc
format?
> +/*
> + * vdso_data will be accessed by 32 and 64 bit code at the same time
> + * so we should be careful before modifying this structure.
> + */
> +struct vdso_data {
> + u32 seq; /* Timebase sequence counter */
> +
> + s32 clock_mode;
> + u64 cycle_last; /* Timebase at clocksource init */
> +
> + struct vdso_clocksource cs[CLOCKSOURCE_BASES];
Why would you need different clocksource parameters? That really bloats the
data structure and makes the cache access pattern worse. Also the vdso
update needs to copy the same data over and over for no value.
The only clock ID which needs a different mult/shift would be
MONOTONIC_RAW, but if we expose that through the VDSO then we really can be
smarter than this. See incomplete and uncompilable patch below for
reference. You get the idea.
> + struct vdso_timestamp basetime[VDSO_BASES];
> +
> + s32 tz_minuteswest; /* Timezone definitions */
> + s32 tz_dsttime;
Also please keep the tabular alignment of the members
u32 foo;
struct bar bar;
that's way better to read than
u32 foo;
struct bar bar;
Thanks
tglx
8<-----------------
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -139,9 +139,10 @@ notrace static inline u64 vgetcyc(int mo
return U64_MAX;
}
-notrace static int do_hres(clockid_t clk, struct timespec *ts)
+notrace static int do_hres(clockid_t clk, struct timespec *ts,
+ struct vsyscall_gtod_data *vdata)
{
- struct vgtod_ts *base = >od->basetime[clk];
+ struct vgtod_ts *base = &vdata->basetime[clk];
u64 cycles, last, sec, ns;
unsigned int seq;
@@ -168,9 +169,10 @@ notrace static int do_hres(clockid_t clk
return 0;
}
-notrace static void do_coarse(clockid_t clk, struct timespec *ts)
+notrace static void do_coarse(clockid_t clk, struct timespec *ts,
+ struct vsyscall_gtod_data *vdata)
{
- struct vgtod_ts *base = >od->basetime[clk];
+ struct vgtod_ts *base = &vdata->basetime[clk];
unsigned int seq;
do {
@@ -194,10 +196,12 @@ notrace int __vdso_clock_gettime(clockid
*/
msk = 1U << clock;
if (likely(msk & VGTOD_HRES)) {
- return do_hres(clock, ts);
+ return do_hres(clock, ts, >od[0]);
} else if (msk & VGTOD_COARSE) {
- do_coarse(clock, ts);
+ do_coarse(clock, ts, >od[0]);
return 0;
+ } else if (msk & VGTOD_RAW) {
+ return do_hres(clock, ts, >od[1]);
}
return vdso_fallback_gettime(clock, ts);
}
@@ -210,7 +214,7 @@ notrace int __vdso_gettimeofday(struct t
if (likely(tv != NULL)) {
struct timespec *ts = (struct timespec *) tv;
- do_hres(CLOCK_REALTIME, ts);
+ do_hres(CLOCK_REALTIME, ts, >od[0]);
tv->tv_usec /= 1000;
}
if (unlikely(tz != NULL)) {
--- a/arch/x86/include/asm/vgtod.h
+++ b/arch/x86/include/asm/vgtod.h
@@ -30,6 +30,7 @@ struct vgtod_ts {
#define VGTOD_BASES (CLOCK_TAI + 1)
#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))
#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE))
+#define VGTOD_RAW BIT(CLOCK_MOTONONIC_RAW)
/*
* vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
@@ -49,7 +50,8 @@ struct vsyscall_gtod_data {
int tz_minuteswest;
int tz_dsttime;
};
-extern struct vsyscall_gtod_data vsyscall_gtod_data;
+
+extern struct vsyscall_gtod_data vsyscall_gtod_data[2];
extern int vclocks_used;
static inline bool vclock_was_used(int vclock)
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-23 16:51 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 16:51 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +/*
> + * There is one vdso_clocksource object in vvar for each vDSO clocksource
> + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
> + * and optimal in terms of access pattern.
> + *
> + * Note that mask and shift are the same for mono and raw.
> + */
> +struct vdso_clocksource {
> + u64 mask; /* Clocksource mask */
> + u32 mult; /* Clocksource multiplier */
> + u32 shift; /* Clocksource shift */
Can you please get rid of the tail comments and use proper kerneldoc
format?
> +/*
> + * vdso_data will be accessed by 32 and 64 bit code at the same time
> + * so we should be careful before modifying this structure.
> + */
> +struct vdso_data {
> + u32 seq; /* Timebase sequence counter */
> +
> + s32 clock_mode;
> + u64 cycle_last; /* Timebase at clocksource init */
> +
> + struct vdso_clocksource cs[CLOCKSOURCE_BASES];
Why would you need different clocksource parameters? That really bloats the
data structure and makes the cache access pattern worse. Also the vdso
update needs to copy the same data over and over for no value.
The only clock ID which needs a different mult/shift would be
MONOTONIC_RAW, but if we expose that through the VDSO then we really can be
smarter than this. See incomplete and uncompilable patch below for
reference. You get the idea.
> + struct vdso_timestamp basetime[VDSO_BASES];
> +
> + s32 tz_minuteswest; /* Timezone definitions */
> + s32 tz_dsttime;
Also please keep the tabular alignment of the members
u32 foo;
struct bar bar;
that's way better to read than
u32 foo;
struct bar bar;
Thanks
tglx
8<-----------------
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -139,9 +139,10 @@ notrace static inline u64 vgetcyc(int mo
return U64_MAX;
}
-notrace static int do_hres(clockid_t clk, struct timespec *ts)
+notrace static int do_hres(clockid_t clk, struct timespec *ts,
+ struct vsyscall_gtod_data *vdata)
{
- struct vgtod_ts *base = >od->basetime[clk];
+ struct vgtod_ts *base = &vdata->basetime[clk];
u64 cycles, last, sec, ns;
unsigned int seq;
@@ -168,9 +169,10 @@ notrace static int do_hres(clockid_t clk
return 0;
}
-notrace static void do_coarse(clockid_t clk, struct timespec *ts)
+notrace static void do_coarse(clockid_t clk, struct timespec *ts,
+ struct vsyscall_gtod_data *vdata)
{
- struct vgtod_ts *base = >od->basetime[clk];
+ struct vgtod_ts *base = &vdata->basetime[clk];
unsigned int seq;
do {
@@ -194,10 +196,12 @@ notrace int __vdso_clock_gettime(clockid
*/
msk = 1U << clock;
if (likely(msk & VGTOD_HRES)) {
- return do_hres(clock, ts);
+ return do_hres(clock, ts, >od[0]);
} else if (msk & VGTOD_COARSE) {
- do_coarse(clock, ts);
+ do_coarse(clock, ts, >od[0]);
return 0;
+ } else if (msk & VGTOD_RAW) {
+ return do_hres(clock, ts, >od[1]);
}
return vdso_fallback_gettime(clock, ts);
}
@@ -210,7 +214,7 @@ notrace int __vdso_gettimeofday(struct t
if (likely(tv != NULL)) {
struct timespec *ts = (struct timespec *) tv;
- do_hres(CLOCK_REALTIME, ts);
+ do_hres(CLOCK_REALTIME, ts, >od[0]);
tv->tv_usec /= 1000;
}
if (unlikely(tz != NULL)) {
--- a/arch/x86/include/asm/vgtod.h
+++ b/arch/x86/include/asm/vgtod.h
@@ -30,6 +30,7 @@ struct vgtod_ts {
#define VGTOD_BASES (CLOCK_TAI + 1)
#define VGTOD_HRES (BIT(CLOCK_REALTIME) | BIT(CLOCK_MONOTONIC) | BIT(CLOCK_TAI))
#define VGTOD_COARSE (BIT(CLOCK_REALTIME_COARSE) | BIT(CLOCK_MONOTONIC_COARSE))
+#define VGTOD_RAW BIT(CLOCK_MOTONONIC_RAW)
/*
* vsyscall_gtod_data will be accessed by 32 and 64 bit code at the same time
@@ -49,7 +50,8 @@ struct vsyscall_gtod_data {
int tz_minuteswest;
int tz_dsttime;
};
-extern struct vsyscall_gtod_data vsyscall_gtod_data;
+
+extern struct vsyscall_gtod_data vsyscall_gtod_data[2];
extern int vclocks_used;
static inline bool vclock_was_used(int vclock)
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-23 17:31 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 17:31 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +static notrace int do_hres(const struct vdso_data *vd,
> + clockid_t clk,
> + struct __vdso_timespec *ts)
> +{
> + const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> + u64 cycles, last, sec, ns;
> + u32 seq, cs_index = CLOCKSOURCE_MONO;
> +
> + if (clk == CLOCK_MONOTONIC_RAW)
> + cs_index = CLOCKSOURCE_RAW;
Uuurgh. So you create an array with 16 members and then use two. This code
is really optimized and now you add not only the pointless array, you also
need the extra index plus another conditional. Not to talk about the cache
impact which makes things even worse. In the x86 implementation we have:
u32 seq; + 0
int mode; + 4
u64 mask; + 8
u32 mult; + 16
u32 shift; + 20
struct vgtod_ts basetimer[VGTOD_BASES]; + 24
Each basetime array member occupies 16 bytes. So
CLOCK_REALTIME + 24
CLOCK_MONOTONIC + 40
..
cacheline boundary
..
CLOCK_REALTIME_COARSE + 104
CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary
CLOCK_BOOTTIME + 136
CLOCK_REALTIME_ALARM + 152
CLOCK_BOOTTIME_ALARM + 168
So the most used clocks REALTIME/MONO are in the first cacheline.
So with your scheme the thing becomes
u32 seq; + 0
int mode; + 4
struct cs cs[16] + 8
struct vgtod_ts basetimer[VGTOD_BASES]; + 264
and
CLOCK_REALTIME + 264
CLOCK_MONOTONIC + 280
IOW, the most important clocks touch TWO cachelines now which are not even
adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
We're surely not going to sacrify the performance which we carefully tuned
in that code just to support MONO_RAW. The solution I showed you in the
other reply does not have these problems at all.
It's easy enough to benchmark these implementations and without trying I'm
pretty sure that you can see the performance drop nicely. Please do so next
time and provide the numbers in the changelogs.
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-23 17:31 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 17:31 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +static notrace int do_hres(const struct vdso_data *vd,
> + clockid_t clk,
> + struct __vdso_timespec *ts)
> +{
> + const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> + u64 cycles, last, sec, ns;
> + u32 seq, cs_index = CLOCKSOURCE_MONO;
> +
> + if (clk == CLOCK_MONOTONIC_RAW)
> + cs_index = CLOCKSOURCE_RAW;
Uuurgh. So you create an array with 16 members and then use two. This code
is really optimized and now you add not only the pointless array, you also
need the extra index plus another conditional. Not to talk about the cache
impact which makes things even worse. In the x86 implementation we have:
u32 seq; + 0
int mode; + 4
u64 mask; + 8
u32 mult; + 16
u32 shift; + 20
struct vgtod_ts basetimer[VGTOD_BASES]; + 24
Each basetime array member occupies 16 bytes. So
CLOCK_REALTIME + 24
CLOCK_MONOTONIC + 40
..
cacheline boundary
..
CLOCK_REALTIME_COARSE + 104
CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary
CLOCK_BOOTTIME + 136
CLOCK_REALTIME_ALARM + 152
CLOCK_BOOTTIME_ALARM + 168
So the most used clocks REALTIME/MONO are in the first cacheline.
So with your scheme the thing becomes
u32 seq; + 0
int mode; + 4
struct cs cs[16] + 8
struct vgtod_ts basetimer[VGTOD_BASES]; + 264
and
CLOCK_REALTIME + 264
CLOCK_MONOTONIC + 280
IOW, the most important clocks touch TWO cachelines now which are not even
adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
We're surely not going to sacrify the performance which we carefully tuned
in that code just to support MONO_RAW. The solution I showed you in the
other reply does not have these problems at all.
It's easy enough to benchmark these implementations and without trying I'm
pretty sure that you can see the performance drop nicely. Please do so next
time and provide the numbers in the changelogs.
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-23 17:31 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 17:31 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> +static notrace int do_hres(const struct vdso_data *vd,
> + clockid_t clk,
> + struct __vdso_timespec *ts)
> +{
> + const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> + u64 cycles, last, sec, ns;
> + u32 seq, cs_index = CLOCKSOURCE_MONO;
> +
> + if (clk == CLOCK_MONOTONIC_RAW)
> + cs_index = CLOCKSOURCE_RAW;
Uuurgh. So you create an array with 16 members and then use two. This code
is really optimized and now you add not only the pointless array, you also
need the extra index plus another conditional. Not to talk about the cache
impact which makes things even worse. In the x86 implementation we have:
u32 seq; + 0
int mode; + 4
u64 mask; + 8
u32 mult; + 16
u32 shift; + 20
struct vgtod_ts basetimer[VGTOD_BASES]; + 24
Each basetime array member occupies 16 bytes. So
CLOCK_REALTIME + 24
CLOCK_MONOTONIC + 40
..
cacheline boundary
..
CLOCK_REALTIME_COARSE + 104
CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary
CLOCK_BOOTTIME + 136
CLOCK_REALTIME_ALARM + 152
CLOCK_BOOTTIME_ALARM + 168
So the most used clocks REALTIME/MONO are in the first cacheline.
So with your scheme the thing becomes
u32 seq; + 0
int mode; + 4
struct cs cs[16] + 8
struct vgtod_ts basetimer[VGTOD_BASES]; + 264
and
CLOCK_REALTIME + 264
CLOCK_MONOTONIC + 280
IOW, the most important clocks touch TWO cachelines now which are not even
adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
We're surely not going to sacrify the performance which we carefully tuned
in that code just to support MONO_RAW. The solution I showed you in the
other reply does not have these problems at all.
It's easy enough to benchmark these implementations and without trying I'm
pretty sure that you can see the performance drop nicely. Please do so next
time and provide the numbers in the changelogs.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-23 17:39 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 17:39 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> This implementation contains the following design choices:
> * Every architecture defines the arch specific code in an header in
> "asm/vdso/".
> * The generic implementation includes the arch specific one and lives
> in "lib/vdso".
> * The arch specific code for gettimeofday lives in
> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
> * This approach allows to consolidate the common code in a single place
> with the benefit of avoiding code duplication.
This is moving into the right direction. Good job!
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-23 17:39 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 17:39 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> This implementation contains the following design choices:
> * Every architecture defines the arch specific code in an header in
> "asm/vdso/".
> * The generic implementation includes the arch specific one and lives
> in "lib/vdso".
> * The arch specific code for gettimeofday lives in
> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
> * This approach allows to consolidate the common code in a single place
> with the benefit of avoiding code duplication.
This is moving into the right direction. Good job!
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-23 17:39 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-23 17:39 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
> This implementation contains the following design choices:
> * Every architecture defines the arch specific code in an header in
> "asm/vdso/".
> * The generic implementation includes the arch specific one and lives
> in "lib/vdso".
> * The arch specific code for gettimeofday lives in
> "<arch path>/vdso/gettimeofday.c" and includes the generic code only.
> * This approach allows to consolidate the common code in a single place
> with the benefit of avoiding code duplication.
This is moving into the right direction. Good job!
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-25 14:09 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-25 14:09 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
Hi Thomas,
thank you for your review.
On 23/02/2019 10:34, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +static notrace int __cvdso_clock_getres(clockid_t clock,
>> + struct __vdso_timespec *res)
>> +{
>> + u64 sec, ns;
>> + u32 msk;
>> +
>> + /* Check for negative values or invalid clocks */
>> + if (unlikely((u32) clock >= MAX_CLOCKS))
>> + goto fallback;
>> +
>> + /*
>> + * Convert the clockid to a bitmask and use it to check which
>> + * clocks are handled in the VDSO directly.
>> + */
>> + msk = 1U << clock;
>> + if (msk & VDSO_HRES) {
>> + /*
>> + * Preserves the behaviour of posix_get_hrtimer_res().
>> + */
>
> So much for the theory.
>
>> + sec = 0;
>> + ns = MONOTONIC_RES_NSEC;
>
> posix_get_hrtimer_res() does:
>
> sec = 0;
> ns = hrtimer_resolution;
>
> and hrtimer_resolution depends on the enablement of high resolution timers
> either compile or run time.
>
> So you need to have a copy of hrtimer_resolution in the vdso data and use
> that.
>
I agree, MONOTONIC_RES_NSEC can be HIGH_RES_NSEC or LOW_RES_NSEC depending on
the HIGH_RES_TIMERS configuration option, but does not cover the run time
switch. I will add a copy of hrtimer_resolution in the vdso data in the next
iteration of the patches.
I had a look at the other implementations as well, and it seems that all the
architectures that currently implement getres() make the same wrong assumption I
made. I am going to provide a separate patch set that targets this.
>> + } else if (msk & VDSO_COARSE) {
>> + /*
>> + * Preserves the behaviour of posix_get_coarse_res().
>> + */
>> + ns = LOW_RES_NSEC;
>> + sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
>
> Do we allow CONFIG_HZ = 1?
>
I had a closer look at it today and seems that jiffies.h supports CONFIG_HZ=12
as a minimum case. Hence I think it should be safe to remove __iter_div_u64_rem
and set sec=0 in this case.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-25 14:09 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-25 14:09 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
Hi Thomas,
thank you for your review.
On 23/02/2019 10:34, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +static notrace int __cvdso_clock_getres(clockid_t clock,
>> + struct __vdso_timespec *res)
>> +{
>> + u64 sec, ns;
>> + u32 msk;
>> +
>> + /* Check for negative values or invalid clocks */
>> + if (unlikely((u32) clock >= MAX_CLOCKS))
>> + goto fallback;
>> +
>> + /*
>> + * Convert the clockid to a bitmask and use it to check which
>> + * clocks are handled in the VDSO directly.
>> + */
>> + msk = 1U << clock;
>> + if (msk & VDSO_HRES) {
>> + /*
>> + * Preserves the behaviour of posix_get_hrtimer_res().
>> + */
>
> So much for the theory.
>
>> + sec = 0;
>> + ns = MONOTONIC_RES_NSEC;
>
> posix_get_hrtimer_res() does:
>
> sec = 0;
> ns = hrtimer_resolution;
>
> and hrtimer_resolution depends on the enablement of high resolution timers
> either compile or run time.
>
> So you need to have a copy of hrtimer_resolution in the vdso data and use
> that.
>
I agree, MONOTONIC_RES_NSEC can be HIGH_RES_NSEC or LOW_RES_NSEC depending on
the HIGH_RES_TIMERS configuration option, but does not cover the run time
switch. I will add a copy of hrtimer_resolution in the vdso data in the next
iteration of the patches.
I had a look at the other implementations as well, and it seems that all the
architectures that currently implement getres() make the same wrong assumption I
made. I am going to provide a separate patch set that targets this.
>> + } else if (msk & VDSO_COARSE) {
>> + /*
>> + * Preserves the behaviour of posix_get_coarse_res().
>> + */
>> + ns = LOW_RES_NSEC;
>> + sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
>
> Do we allow CONFIG_HZ = 1?
>
I had a closer look at it today and seems that jiffies.h supports CONFIG_HZ=12
as a minimum case. Hence I think it should be safe to remove __iter_div_u64_rem
and set sec=0 in this case.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-25 14:09 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-25 14:09 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
Hi Thomas,
thank you for your review.
On 23/02/2019 10:34, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +static notrace int __cvdso_clock_getres(clockid_t clock,
>> + struct __vdso_timespec *res)
>> +{
>> + u64 sec, ns;
>> + u32 msk;
>> +
>> + /* Check for negative values or invalid clocks */
>> + if (unlikely((u32) clock >= MAX_CLOCKS))
>> + goto fallback;
>> +
>> + /*
>> + * Convert the clockid to a bitmask and use it to check which
>> + * clocks are handled in the VDSO directly.
>> + */
>> + msk = 1U << clock;
>> + if (msk & VDSO_HRES) {
>> + /*
>> + * Preserves the behaviour of posix_get_hrtimer_res().
>> + */
>
> So much for the theory.
>
>> + sec = 0;
>> + ns = MONOTONIC_RES_NSEC;
>
> posix_get_hrtimer_res() does:
>
> sec = 0;
> ns = hrtimer_resolution;
>
> and hrtimer_resolution depends on the enablement of high resolution timers
> either compile or run time.
>
> So you need to have a copy of hrtimer_resolution in the vdso data and use
> that.
>
I agree, MONOTONIC_RES_NSEC can be HIGH_RES_NSEC or LOW_RES_NSEC depending on
the HIGH_RES_TIMERS configuration option, but does not cover the run time
switch. I will add a copy of hrtimer_resolution in the vdso data in the next
iteration of the patches.
I had a look at the other implementations as well, and it seems that all the
architectures that currently implement getres() make the same wrong assumption I
made. I am going to provide a separate patch set that targets this.
>> + } else if (msk & VDSO_COARSE) {
>> + /*
>> + * Preserves the behaviour of posix_get_coarse_res().
>> + */
>> + ns = LOW_RES_NSEC;
>> + sec = __iter_div_u64_rem(ns, NSEC_PER_SEC, &ns);
>
> Do we allow CONFIG_HZ = 1?
>
I had a closer look at it today and seems that jiffies.h supports CONFIG_HZ=12
as a minimum case. Hence I think it should be safe to remove __iter_div_u64_rem
and set sec=0 in this case.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-26 12:10 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-26 12:10 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Hi Mark,
thank you for your review.
On 22/02/2019 14:04, Mark Rutland wrote:
> On Fri, Feb 22, 2019 at 12:24:15PM +0000, Vincenzo Frascino wrote:
>> When kuser helpers are enabled the kernel maps the relative code at
>> a fixed address (0xffff0000). Making configurable the option to disable
>> them means that the kernel can remove this mapping and any access to
>> this memory area results in a sigfault.
>>
>> Add a KUSER_HELPERS config option that can be used to disable the
>> mapping when it is turned off.
>>
>> This option can be turned off if and only if the applications are
>> designed specifically for the platform and they do not make use of the
>> kuser helpers code.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO
> changes.
>
> I think the KUSER bits should be split into a preparatory series, so that they
> can be merged soon, rather than tying them artificially to the rest of the vDSO
> bits (which I expect will require significantly more review).
>
This is a good point. I will extract them and post as a separate patch set.
In doing so I will update the config description below.
--
Regards,
Vincenzo
>> ---
>> arch/arm64/Kconfig | 21 +++++++++++++++++++++
>> arch/arm64/kernel/Makefile | 3 ++-
>> arch/arm64/kernel/kuser32.S | 7 +++----
>> arch/arm64/kernel/vdso.c | 15 +++++++++++++++
>> 4 files changed, 41 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index d898da2e20f5..ed3290494f1c 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -1465,6 +1465,27 @@ config COMPAT
>>
>> If you want to execute 32-bit userspace applications, say Y.
>>
>> +config KUSER_HELPERS
>> + bool "Enable kuser helpers page for compatibility with 32 bit applications."
>> + depends on COMPAT
>> + default y
>> + help
>> + Enables kuser helpers to be mapped in a special purpose page at a fixed
>> + address to maintain independence from the type of CPU present in the SoC.
>> + This feature is provided for compatibility reasons in fact allows 32 bit
>> + applications compliant with ARMv4 up to ARMv8 to run without any
>> + modification.
>> +
>> + Warning: Being always mapped at a fixed address makes it easier to create
>> + exploits based on ROP type of attacks.
>> +
>> + As a consequence of this, this feature is made configurable but be aware that
>> + it can be turned off if and only if the binaries and the libraries running on
>> + a specific platform are designed to do not make use of these helpers, otherwise
>> + should be left on.
>> +
>> + See Documentation/arm/kernel_user_helpers.txt for details.
>
> This is quite divergent from the text for the 32-bit arm KUSER_HELPERS.
>
> Could we please reuse that existing text as far as possible, with minimal
> changes?
>
> e.g.
>
> config KUSER_HELPERS
> bool "Enable kuser helpers for compat tasks"
> depends on COMPAT
> default y
> help
> Warning: disabling this option may break user programs.
>
> Provide kuser helpers to compat tasks. The kernel provides
> helper code to userspace in read only form at a fixed location
> to allow userspace to be independent of the CPU type fitted to
> the system. This permits binaries to be run on ARMv4 through
> to ARMv8 without modification.
>
> See Documentation/arm/kernel_user_helpers.txt for details.
>
> However, the fixed address nature of these helpers can be used
> by ROP (return orientated programming) authors when creating
> exploits.
>
> If all of the binaries and libraries which run on your platform
> are built specifically for your platform, and make no use of
> these helpers, then you can turn this option off to hinder
> such exploits. However, in that case, if a binary or library
> relying on those helpers is run, it will not function correctly.
>
> Say N here only if you are absolutely certain that you do not
> need these helpers; otherwise, the safe option is to say Y.
>
> I believe that Will wanted to default the kuser helpers off, as applications
> using them need the instruction emulation which is also disabled by default.
>
> I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
> the ARMv4 reference is correct.
>
> Thanks,
> Mark.
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-26 12:10 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-26 12:10 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Hi Mark,
thank you for your review.
On 22/02/2019 14:04, Mark Rutland wrote:
> On Fri, Feb 22, 2019 at 12:24:15PM +0000, Vincenzo Frascino wrote:
>> When kuser helpers are enabled the kernel maps the relative code at
>> a fixed address (0xffff0000). Making configurable the option to disable
>> them means that the kernel can remove this mapping and any access to
>> this memory area results in a sigfault.
>>
>> Add a KUSER_HELPERS config option that can be used to disable the
>> mapping when it is turned off.
>>
>> This option can be turned off if and only if the applications are
>> designed specifically for the platform and they do not make use of the
>> kuser helpers code.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO
> changes.
>
> I think the KUSER bits should be split into a preparatory series, so that they
> can be merged soon, rather than tying them artificially to the rest of the vDSO
> bits (which I expect will require significantly more review).
>
This is a good point. I will extract them and post as a separate patch set.
In doing so I will update the config description below.
--
Regards,
Vincenzo
>> ---
>> arch/arm64/Kconfig | 21 +++++++++++++++++++++
>> arch/arm64/kernel/Makefile | 3 ++-
>> arch/arm64/kernel/kuser32.S | 7 +++----
>> arch/arm64/kernel/vdso.c | 15 +++++++++++++++
>> 4 files changed, 41 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index d898da2e20f5..ed3290494f1c 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -1465,6 +1465,27 @@ config COMPAT
>>
>> If you want to execute 32-bit userspace applications, say Y.
>>
>> +config KUSER_HELPERS
>> + bool "Enable kuser helpers page for compatibility with 32 bit applications."
>> + depends on COMPAT
>> + default y
>> + help
>> + Enables kuser helpers to be mapped in a special purpose page at a fixed
>> + address to maintain independence from the type of CPU present in the SoC.
>> + This feature is provided for compatibility reasons in fact allows 32 bit
>> + applications compliant with ARMv4 up to ARMv8 to run without any
>> + modification.
>> +
>> + Warning: Being always mapped at a fixed address makes it easier to create
>> + exploits based on ROP type of attacks.
>> +
>> + As a consequence of this, this feature is made configurable but be aware that
>> + it can be turned off if and only if the binaries and the libraries running on
>> + a specific platform are designed to do not make use of these helpers, otherwise
>> + should be left on.
>> +
>> + See Documentation/arm/kernel_user_helpers.txt for details.
>
> This is quite divergent from the text for the 32-bit arm KUSER_HELPERS.
>
> Could we please reuse that existing text as far as possible, with minimal
> changes?
>
> e.g.
>
> config KUSER_HELPERS
> bool "Enable kuser helpers for compat tasks"
> depends on COMPAT
> default y
> help
> Warning: disabling this option may break user programs.
>
> Provide kuser helpers to compat tasks. The kernel provides
> helper code to userspace in read only form at a fixed location
> to allow userspace to be independent of the CPU type fitted to
> the system. This permits binaries to be run on ARMv4 through
> to ARMv8 without modification.
>
> See Documentation/arm/kernel_user_helpers.txt for details.
>
> However, the fixed address nature of these helpers can be used
> by ROP (return orientated programming) authors when creating
> exploits.
>
> If all of the binaries and libraries which run on your platform
> are built specifically for your platform, and make no use of
> these helpers, then you can turn this option off to hinder
> such exploits. However, in that case, if a binary or library
> relying on those helpers is run, it will not function correctly.
>
> Say N here only if you are absolutely certain that you do not
> need these helpers; otherwise, the safe option is to say Y.
>
> I believe that Will wanted to default the kuser helpers off, as applications
> using them need the instruction emulation which is also disabled by default.
>
> I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
> the ARMv4 reference is correct.
>
> Thanks,
> Mark.
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option
@ 2019-02-26 12:10 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-26 12:10 UTC (permalink / raw)
To: Mark Rutland
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Hi Mark,
thank you for your review.
On 22/02/2019 14:04, Mark Rutland wrote:
> On Fri, Feb 22, 2019 at 12:24:15PM +0000, Vincenzo Frascino wrote:
>> When kuser helpers are enabled the kernel maps the relative code at
>> a fixed address (0xffff0000). Making configurable the option to disable
>> them means that the kernel can remove this mapping and any access to
>> this memory area results in a sigfault.
>>
>> Add a KUSER_HELPERS config option that can be used to disable the
>> mapping when it is turned off.
>>
>> This option can be turned off if and only if the applications are
>> designed specifically for the platform and they do not make use of the
>> kuser helpers code.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: Will Deacon <will.deacon@arm.com>
>> Signed-off-by: Vincenzo Frascino <vincenzo.frascino@arm.com>
>
> IIUC, the KUSER bits aren't striclty dependent on the rest of the vDSO
> changes.
>
> I think the KUSER bits should be split into a preparatory series, so that they
> can be merged soon, rather than tying them artificially to the rest of the vDSO
> bits (which I expect will require significantly more review).
>
This is a good point. I will extract them and post as a separate patch set.
In doing so I will update the config description below.
--
Regards,
Vincenzo
>> ---
>> arch/arm64/Kconfig | 21 +++++++++++++++++++++
>> arch/arm64/kernel/Makefile | 3 ++-
>> arch/arm64/kernel/kuser32.S | 7 +++----
>> arch/arm64/kernel/vdso.c | 15 +++++++++++++++
>> 4 files changed, 41 insertions(+), 5 deletions(-)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index d898da2e20f5..ed3290494f1c 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -1465,6 +1465,27 @@ config COMPAT
>>
>> If you want to execute 32-bit userspace applications, say Y.
>>
>> +config KUSER_HELPERS
>> + bool "Enable kuser helpers page for compatibility with 32 bit applications."
>> + depends on COMPAT
>> + default y
>> + help
>> + Enables kuser helpers to be mapped in a special purpose page at a fixed
>> + address to maintain independence from the type of CPU present in the SoC.
>> + This feature is provided for compatibility reasons in fact allows 32 bit
>> + applications compliant with ARMv4 up to ARMv8 to run without any
>> + modification.
>> +
>> + Warning: Being always mapped at a fixed address makes it easier to create
>> + exploits based on ROP type of attacks.
>> +
>> + As a consequence of this, this feature is made configurable but be aware that
>> + it can be turned off if and only if the binaries and the libraries running on
>> + a specific platform are designed to do not make use of these helpers, otherwise
>> + should be left on.
>> +
>> + See Documentation/arm/kernel_user_helpers.txt for details.
>
> This is quite divergent from the text for the 32-bit arm KUSER_HELPERS.
>
> Could we please reuse that existing text as far as possible, with minimal
> changes?
>
> e.g.
>
> config KUSER_HELPERS
> bool "Enable kuser helpers for compat tasks"
> depends on COMPAT
> default y
> help
> Warning: disabling this option may break user programs.
>
> Provide kuser helpers to compat tasks. The kernel provides
> helper code to userspace in read only form at a fixed location
> to allow userspace to be independent of the CPU type fitted to
> the system. This permits binaries to be run on ARMv4 through
> to ARMv8 without modification.
>
> See Documentation/arm/kernel_user_helpers.txt for details.
>
> However, the fixed address nature of these helpers can be used
> by ROP (return orientated programming) authors when creating
> exploits.
>
> If all of the binaries and libraries which run on your platform
> are built specifically for your platform, and make no use of
> these helpers, then you can turn this option off to hinder
> such exploits. However, in that case, if a binary or library
> relying on those helpers is run, it will not function correctly.
>
> Say N here only if you are absolutely certain that you do not
> need these helpers; otherwise, the safe option is to say Y.
>
> I believe that Will wanted to default the kuser helpers off, as applications
> using them need the instruction emulation which is also disabled by default.
>
> I'm also not sure which binaries we expect to work on arm64, so I'm not sure if
> the ARMv4 reference is correct.
>
> Thanks,
> Mark.
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 13:47 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 13:47 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
Hi Thomas,
On 23/02/2019 17:31, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +static notrace int do_hres(const struct vdso_data *vd,
>> + clockid_t clk,
>> + struct __vdso_timespec *ts)
>> +{
>> + const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
>> + u64 cycles, last, sec, ns;
>> + u32 seq, cs_index = CLOCKSOURCE_MONO;
>> +
>> + if (clk == CLOCK_MONOTONIC_RAW)
>> + cs_index = CLOCKSOURCE_RAW;
>
> Uuurgh. So you create an array with 16 members and then use two. This code
> is really optimized and now you add not only the pointless array, you also
> need the extra index plus another conditional. Not to talk about the cache
> impact which makes things even worse. In the x86 implementation we have:
>
> u32 seq; + 0
> int mode; + 4
> u64 mask; + 8
> u32 mult; + 16
> u32 shift; + 20
> struct vgtod_ts basetimer[VGTOD_BASES]; + 24
>
> Each basetime array member occupies 16 bytes. So
>
> CLOCK_REALTIME + 24
> CLOCK_MONOTONIC + 40
> ..
> cacheline boundary
> ..
> CLOCK_REALTIME_COARSE + 104
> CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary
> CLOCK_BOOTTIME + 136
> CLOCK_REALTIME_ALARM + 152
> CLOCK_BOOTTIME_ALARM + 168
>
> So the most used clocks REALTIME/MONO are in the first cacheline.
>
> So with your scheme the thing becomes
>
> u32 seq; + 0
> int mode; + 4
> struct cs cs[16] + 8
> struct vgtod_ts basetimer[VGTOD_BASES]; + 264
>
> and
>
> CLOCK_REALTIME + 264
> CLOCK_MONOTONIC + 280
>
The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
the situation with my scheme should be the following:
u32 seq: + 0
s32 clock_mode; + 4
u64 cycle_last; + 8
struct vdso_cs cs[2]; + 16
struct vdso_ts basetime[VDSO_BASES]; + 48
which I agree makes still things a bit worse.
Assuming L1_CACHE_SHIFT == 6:
CLOCK_REALTIME + 48
...
cache boundary
...
CLOCK_MONOTONIC + 64
CLOCK_PROCESS_CPUTIME_ID + 80
CLOCK_THREAD_CPUTIME_ID + 96
CLOCK_MONOTONIC_RAW + 112
...
cache boundary
...
CLOCK_REALTIME_COARSE + 128
CLOCK_MONOTONIC_COARSE + 144
CLOCK_BOOTTIME + 160
CLOCK_REALTIME_ALARM + 172
CLOCK_BOOTTIME_ALARM + 188
...
> IOW, the most important clocks touch TWO cachelines now which are not even
> adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
>
> We're surely not going to sacrify the performance which we carefully tuned
> in that code just to support MONO_RAW. The solution I showed you in the
> other reply does not have these problems at all.
>
> It's easy enough to benchmark these implementations and without trying I'm
> pretty sure that you can see the performance drop nicely. Please do so next
> time and provide the numbers in the changelogs.
>
I did run some benchmarks this morning to quantify the performance impact and
seems that using vdsotest[1] the difference in between a stock linux kernel
5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
5120T), is below 1%. Please find the results below, I will add them as well to
the next changelog.
[1] https://github.com/nathanlynch/vdsotest
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
8<-----------------
Unified vDSO:
=============
clock-gettime-monotonic: syscall: 351 nsec/call
clock-gettime-monotonic: libc: 37 nsec/call
clock-gettime-monotonic: vdso: 31 nsec/call
clock-getres-monotonic: syscall: 271 nsec/call
clock-getres-monotonic: libc: 269 nsec/call
clock-getres-monotonic: vdso: 9 nsec/call
clock-gettime-monotonic-coarse: syscall: 280 nsec/call
clock-gettime-monotonic-coarse: libc: 22 nsec/call
clock-gettime-monotonic-coarse: vdso: 11 nsec/call
clock-getres-monotonic-coarse: syscall: 274 nsec/call
clock-getres-monotonic-coarse: libc: 276 nsec/call
clock-getres-monotonic-coarse: vdso: 10 nsec/call
clock-gettime-monotonic-raw: syscall: 337 nsec/call
clock-gettime-monotonic-raw: libc: 38 nsec/call
clock-gettime-monotonic-raw: vdso: 32 nsec/call
clock-getres-monotonic-raw: syscall: 284 nsec/call
clock-getres-monotonic-raw: libc: 271 nsec/call
clock-getres-monotonic-raw: vdso: 9 nsec/call
clock-gettime-tai: syscall: 332 nsec/call
clock-gettime-tai: libc: 37 nsec/call
clock-gettime-tai: vdso: 31 nsec/call
clock-getres-tai: syscall: 273 nsec/call
clock-getres-tai: libc: 281 nsec/call
clock-getres-tai: vdso: 10 nsec/call
clock-gettime-boottime: syscall: 338 nsec/call
clock-gettime-boottime: libc: 37 nsec/call
clock-gettime-boottime: vdso: 32 nsec/call
clock-getres-boottime: syscall: 283 nsec/call
clock-getres-boottime: libc: 278 nsec/call
clock-getres-boottime: vdso: 9 nsec/call
clock-gettime-realtime: syscall: 338 nsec/call
clock-gettime-realtime: libc: 39 nsec/call
clock-gettime-realtime: vdso: 32 nsec/call
clock-getres-realtime: syscall: 281 nsec/call
clock-getres-realtime: libc: 277 nsec/call
clock-getres-realtime: vdso: 10 nsec/call
clock-gettime-realtime-coarse: syscall: 286 nsec/call
clock-gettime-realtime-coarse: libc: 21 nsec/call
clock-gettime-realtime-coarse: vdso: 12 nsec/call
clock-getres-realtime-coarse: syscall: 285 nsec/call
clock-getres-realtime-coarse: libc: 283 nsec/call
clock-getres-realtime-coarse: vdso: 11 nsec/call
getcpu: syscall: 234 nsec/call
getcpu: libc: 31 nsec/call
getcpu: vdso: 20 nsec/call
gettimeofday: syscall: 293 nsec/call
gettimeofday: libc: 32 nsec/call
gettimeofday: vdso: 31 nsec/call
Stock Kernel:
=============
clock-gettime-monotonic: syscall: 349 nsec/call
clock-gettime-monotonic: libc: 37 nsec/call
clock-gettime-monotonic: vdso: 28 nsec/call
clock-getres-monotonic: syscall: 296 nsec/call
clock-getres-monotonic: libc: 295 nsec/call
clock-getres-monotonic: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-coarse: syscall: 296 nsec/call
clock-gettime-monotonic-coarse: libc: 21 nsec/call
clock-gettime-monotonic-coarse: vdso: 11 nsec/call
clock-getres-monotonic-coarse: syscall: 287 nsec/call
clock-getres-monotonic-coarse: libc: 288 nsec/call
clock-getres-monotonic-coarse: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-raw: syscall: 353 nsec/call
clock-gettime-monotonic-raw: libc: 360 nsec/call
clock-gettime-monotonic-raw: vdso: 352 nsec/call
clock-getres-monotonic-raw: syscall: 282 nsec/call
clock-getres-monotonic-raw: libc: 286 nsec/call
clock-getres-monotonic-raw: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-tai: syscall: 351 nsec/call
clock-gettime-tai: libc: 364 nsec/call
clock-gettime-tai: vdso: 365 nsec/call
clock-getres-tai: syscall: 287 nsec/call
clock-getres-tai: libc: 287 nsec/call
clock-getres-tai: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-boottime: syscall: 347 nsec/call
clock-gettime-boottime: libc: 364 nsec/call
clock-gettime-boottime: vdso: 355 nsec/call
clock-getres-boottime: syscall: 287 nsec/call
clock-getres-boottime: libc: 287 nsec/call
clock-getres-boottime: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime: syscall: 346 nsec/call
clock-gettime-realtime: libc: 36 nsec/call
clock-gettime-realtime: vdso: 29 nsec/call
clock-getres-realtime: syscall: 285 nsec/call
clock-getres-realtime: libc: 287 nsec/call
clock-getres-realtime: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime-coarse: syscall: 296 nsec/call
clock-gettime-realtime-coarse: libc: 20 nsec/call
clock-gettime-realtime-coarse: vdso: 11 nsec/call
clock-getres-realtime-coarse: syscall: 301 nsec/call
clock-getres-realtime-coarse: libc: 297 nsec/call
clock-getres-realtime-coarse: vdso: not tested
Note: vDSO version of clock_getres not found
getcpu: syscall: 255 nsec/call
getcpu: libc: 32 nsec/call
getcpu: vdso: 21 nsec/call
gettimeofday: syscall: 339 nsec/call
gettimeofday: libc: 31 nsec/call
gettimeofday: vdso: 30 nsec/call
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 13:47 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 13:47 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
Hi Thomas,
On 23/02/2019 17:31, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +static notrace int do_hres(const struct vdso_data *vd,
>> + clockid_t clk,
>> + struct __vdso_timespec *ts)
>> +{
>> + const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
>> + u64 cycles, last, sec, ns;
>> + u32 seq, cs_index = CLOCKSOURCE_MONO;
>> +
>> + if (clk == CLOCK_MONOTONIC_RAW)
>> + cs_index = CLOCKSOURCE_RAW;
>
> Uuurgh. So you create an array with 16 members and then use two. This code
> is really optimized and now you add not only the pointless array, you also
> need the extra index plus another conditional. Not to talk about the cache
> impact which makes things even worse. In the x86 implementation we have:
>
> u32 seq; + 0
> int mode; + 4
> u64 mask; + 8
> u32 mult; + 16
> u32 shift; + 20
> struct vgtod_ts basetimer[VGTOD_BASES]; + 24
>
> Each basetime array member occupies 16 bytes. So
>
> CLOCK_REALTIME + 24
> CLOCK_MONOTONIC + 40
> ..
> cacheline boundary
> ..
> CLOCK_REALTIME_COARSE + 104
> CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary
> CLOCK_BOOTTIME + 136
> CLOCK_REALTIME_ALARM + 152
> CLOCK_BOOTTIME_ALARM + 168
>
> So the most used clocks REALTIME/MONO are in the first cacheline.
>
> So with your scheme the thing becomes
>
> u32 seq; + 0
> int mode; + 4
> struct cs cs[16] + 8
> struct vgtod_ts basetimer[VGTOD_BASES]; + 264
>
> and
>
> CLOCK_REALTIME + 264
> CLOCK_MONOTONIC + 280
>
The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
the situation with my scheme should be the following:
u32 seq: + 0
s32 clock_mode; + 4
u64 cycle_last; + 8
struct vdso_cs cs[2]; + 16
struct vdso_ts basetime[VDSO_BASES]; + 48
which I agree makes still things a bit worse.
Assuming L1_CACHE_SHIFT == 6:
CLOCK_REALTIME + 48
...
cache boundary
...
CLOCK_MONOTONIC + 64
CLOCK_PROCESS_CPUTIME_ID + 80
CLOCK_THREAD_CPUTIME_ID + 96
CLOCK_MONOTONIC_RAW + 112
...
cache boundary
...
CLOCK_REALTIME_COARSE + 128
CLOCK_MONOTONIC_COARSE + 144
CLOCK_BOOTTIME + 160
CLOCK_REALTIME_ALARM + 172
CLOCK_BOOTTIME_ALARM + 188
...
> IOW, the most important clocks touch TWO cachelines now which are not even
> adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
>
> We're surely not going to sacrify the performance which we carefully tuned
> in that code just to support MONO_RAW. The solution I showed you in the
> other reply does not have these problems at all.
>
> It's easy enough to benchmark these implementations and without trying I'm
> pretty sure that you can see the performance drop nicely. Please do so next
> time and provide the numbers in the changelogs.
>
I did run some benchmarks this morning to quantify the performance impact and
seems that using vdsotest[1] the difference in between a stock linux kernel
5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
5120T), is below 1%. Please find the results below, I will add them as well to
the next changelog.
[1] https://github.com/nathanlynch/vdsotest
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
8<-----------------
Unified vDSO:
=============
clock-gettime-monotonic: syscall: 351 nsec/call
clock-gettime-monotonic: libc: 37 nsec/call
clock-gettime-monotonic: vdso: 31 nsec/call
clock-getres-monotonic: syscall: 271 nsec/call
clock-getres-monotonic: libc: 269 nsec/call
clock-getres-monotonic: vdso: 9 nsec/call
clock-gettime-monotonic-coarse: syscall: 280 nsec/call
clock-gettime-monotonic-coarse: libc: 22 nsec/call
clock-gettime-monotonic-coarse: vdso: 11 nsec/call
clock-getres-monotonic-coarse: syscall: 274 nsec/call
clock-getres-monotonic-coarse: libc: 276 nsec/call
clock-getres-monotonic-coarse: vdso: 10 nsec/call
clock-gettime-monotonic-raw: syscall: 337 nsec/call
clock-gettime-monotonic-raw: libc: 38 nsec/call
clock-gettime-monotonic-raw: vdso: 32 nsec/call
clock-getres-monotonic-raw: syscall: 284 nsec/call
clock-getres-monotonic-raw: libc: 271 nsec/call
clock-getres-monotonic-raw: vdso: 9 nsec/call
clock-gettime-tai: syscall: 332 nsec/call
clock-gettime-tai: libc: 37 nsec/call
clock-gettime-tai: vdso: 31 nsec/call
clock-getres-tai: syscall: 273 nsec/call
clock-getres-tai: libc: 281 nsec/call
clock-getres-tai: vdso: 10 nsec/call
clock-gettime-boottime: syscall: 338 nsec/call
clock-gettime-boottime: libc: 37 nsec/call
clock-gettime-boottime: vdso: 32 nsec/call
clock-getres-boottime: syscall: 283 nsec/call
clock-getres-boottime: libc: 278 nsec/call
clock-getres-boottime: vdso: 9 nsec/call
clock-gettime-realtime: syscall: 338 nsec/call
clock-gettime-realtime: libc: 39 nsec/call
clock-gettime-realtime: vdso: 32 nsec/call
clock-getres-realtime: syscall: 281 nsec/call
clock-getres-realtime: libc: 277 nsec/call
clock-getres-realtime: vdso: 10 nsec/call
clock-gettime-realtime-coarse: syscall: 286 nsec/call
clock-gettime-realtime-coarse: libc: 21 nsec/call
clock-gettime-realtime-coarse: vdso: 12 nsec/call
clock-getres-realtime-coarse: syscall: 285 nsec/call
clock-getres-realtime-coarse: libc: 283 nsec/call
clock-getres-realtime-coarse: vdso: 11 nsec/call
getcpu: syscall: 234 nsec/call
getcpu: libc: 31 nsec/call
getcpu: vdso: 20 nsec/call
gettimeofday: syscall: 293 nsec/call
gettimeofday: libc: 32 nsec/call
gettimeofday: vdso: 31 nsec/call
Stock Kernel:
=============
clock-gettime-monotonic: syscall: 349 nsec/call
clock-gettime-monotonic: libc: 37 nsec/call
clock-gettime-monotonic: vdso: 28 nsec/call
clock-getres-monotonic: syscall: 296 nsec/call
clock-getres-monotonic: libc: 295 nsec/call
clock-getres-monotonic: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-coarse: syscall: 296 nsec/call
clock-gettime-monotonic-coarse: libc: 21 nsec/call
clock-gettime-monotonic-coarse: vdso: 11 nsec/call
clock-getres-monotonic-coarse: syscall: 287 nsec/call
clock-getres-monotonic-coarse: libc: 288 nsec/call
clock-getres-monotonic-coarse: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-raw: syscall: 353 nsec/call
clock-gettime-monotonic-raw: libc: 360 nsec/call
clock-gettime-monotonic-raw: vdso: 352 nsec/call
clock-getres-monotonic-raw: syscall: 282 nsec/call
clock-getres-monotonic-raw: libc: 286 nsec/call
clock-getres-monotonic-raw: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-tai: syscall: 351 nsec/call
clock-gettime-tai: libc: 364 nsec/call
clock-gettime-tai: vdso: 365 nsec/call
clock-getres-tai: syscall: 287 nsec/call
clock-getres-tai: libc: 287 nsec/call
clock-getres-tai: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-boottime: syscall: 347 nsec/call
clock-gettime-boottime: libc: 364 nsec/call
clock-gettime-boottime: vdso: 355 nsec/call
clock-getres-boottime: syscall: 287 nsec/call
clock-getres-boottime: libc: 287 nsec/call
clock-getres-boottime: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime: syscall: 346 nsec/call
clock-gettime-realtime: libc: 36 nsec/call
clock-gettime-realtime: vdso: 29 nsec/call
clock-getres-realtime: syscall: 285 nsec/call
clock-getres-realtime: libc: 287 nsec/call
clock-getres-realtime: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime-coarse: syscall: 296 nsec/call
clock-gettime-realtime-coarse: libc: 20 nsec/call
clock-gettime-realtime-coarse: vdso: 11 nsec/call
clock-getres-realtime-coarse: syscall: 301 nsec/call
clock-getres-realtime-coarse: libc: 297 nsec/call
clock-getres-realtime-coarse: vdso: not tested
Note: vDSO version of clock_getres not found
getcpu: syscall: 255 nsec/call
getcpu: libc: 32 nsec/call
getcpu: vdso: 21 nsec/call
gettimeofday: syscall: 339 nsec/call
gettimeofday: libc: 31 nsec/call
gettimeofday: vdso: 30 nsec/call
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 13:47 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 13:47 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
Hi Thomas,
On 23/02/2019 17:31, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +static notrace int do_hres(const struct vdso_data *vd,
>> + clockid_t clk,
>> + struct __vdso_timespec *ts)
>> +{
>> + const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
>> + u64 cycles, last, sec, ns;
>> + u32 seq, cs_index = CLOCKSOURCE_MONO;
>> +
>> + if (clk == CLOCK_MONOTONIC_RAW)
>> + cs_index = CLOCKSOURCE_RAW;
>
> Uuurgh. So you create an array with 16 members and then use two. This code
> is really optimized and now you add not only the pointless array, you also
> need the extra index plus another conditional. Not to talk about the cache
> impact which makes things even worse. In the x86 implementation we have:
>
> u32 seq; + 0
> int mode; + 4
> u64 mask; + 8
> u32 mult; + 16
> u32 shift; + 20
> struct vgtod_ts basetimer[VGTOD_BASES]; + 24
>
> Each basetime array member occupies 16 bytes. So
>
> CLOCK_REALTIME + 24
> CLOCK_MONOTONIC + 40
> ..
> cacheline boundary
> ..
> CLOCK_REALTIME_COARSE + 104
> CLOCK_MONOTONIC_COARSE + 120 <- cacheline boundary
> CLOCK_BOOTTIME + 136
> CLOCK_REALTIME_ALARM + 152
> CLOCK_BOOTTIME_ALARM + 168
>
> So the most used clocks REALTIME/MONO are in the first cacheline.
>
> So with your scheme the thing becomes
>
> u32 seq; + 0
> int mode; + 4
> struct cs cs[16] + 8
> struct vgtod_ts basetimer[VGTOD_BASES]; + 264
>
> and
>
> CLOCK_REALTIME + 264
> CLOCK_MONOTONIC + 280
>
The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
the situation with my scheme should be the following:
u32 seq: + 0
s32 clock_mode; + 4
u64 cycle_last; + 8
struct vdso_cs cs[2]; + 16
struct vdso_ts basetime[VDSO_BASES]; + 48
which I agree makes still things a bit worse.
Assuming L1_CACHE_SHIFT == 6:
CLOCK_REALTIME + 48
...
cache boundary
...
CLOCK_MONOTONIC + 64
CLOCK_PROCESS_CPUTIME_ID + 80
CLOCK_THREAD_CPUTIME_ID + 96
CLOCK_MONOTONIC_RAW + 112
...
cache boundary
...
CLOCK_REALTIME_COARSE + 128
CLOCK_MONOTONIC_COARSE + 144
CLOCK_BOOTTIME + 160
CLOCK_REALTIME_ALARM + 172
CLOCK_BOOTTIME_ALARM + 188
...
> IOW, the most important clocks touch TWO cachelines now which are not even
> adjacent. No, they are 256 bytes apart, which really sucks for prefetching.
>
> We're surely not going to sacrify the performance which we carefully tuned
> in that code just to support MONO_RAW. The solution I showed you in the
> other reply does not have these problems at all.
>
> It's easy enough to benchmark these implementations and without trying I'm
> pretty sure that you can see the performance drop nicely. Please do so next
> time and provide the numbers in the changelogs.
>
I did run some benchmarks this morning to quantify the performance impact and
seems that using vdsotest[1] the difference in between a stock linux kernel
5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
5120T), is below 1%. Please find the results below, I will add them as well to
the next changelog.
[1] https://github.com/nathanlynch/vdsotest
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
8<-----------------
Unified vDSO:
=============
clock-gettime-monotonic: syscall: 351 nsec/call
clock-gettime-monotonic: libc: 37 nsec/call
clock-gettime-monotonic: vdso: 31 nsec/call
clock-getres-monotonic: syscall: 271 nsec/call
clock-getres-monotonic: libc: 269 nsec/call
clock-getres-monotonic: vdso: 9 nsec/call
clock-gettime-monotonic-coarse: syscall: 280 nsec/call
clock-gettime-monotonic-coarse: libc: 22 nsec/call
clock-gettime-monotonic-coarse: vdso: 11 nsec/call
clock-getres-monotonic-coarse: syscall: 274 nsec/call
clock-getres-monotonic-coarse: libc: 276 nsec/call
clock-getres-monotonic-coarse: vdso: 10 nsec/call
clock-gettime-monotonic-raw: syscall: 337 nsec/call
clock-gettime-monotonic-raw: libc: 38 nsec/call
clock-gettime-monotonic-raw: vdso: 32 nsec/call
clock-getres-monotonic-raw: syscall: 284 nsec/call
clock-getres-monotonic-raw: libc: 271 nsec/call
clock-getres-monotonic-raw: vdso: 9 nsec/call
clock-gettime-tai: syscall: 332 nsec/call
clock-gettime-tai: libc: 37 nsec/call
clock-gettime-tai: vdso: 31 nsec/call
clock-getres-tai: syscall: 273 nsec/call
clock-getres-tai: libc: 281 nsec/call
clock-getres-tai: vdso: 10 nsec/call
clock-gettime-boottime: syscall: 338 nsec/call
clock-gettime-boottime: libc: 37 nsec/call
clock-gettime-boottime: vdso: 32 nsec/call
clock-getres-boottime: syscall: 283 nsec/call
clock-getres-boottime: libc: 278 nsec/call
clock-getres-boottime: vdso: 9 nsec/call
clock-gettime-realtime: syscall: 338 nsec/call
clock-gettime-realtime: libc: 39 nsec/call
clock-gettime-realtime: vdso: 32 nsec/call
clock-getres-realtime: syscall: 281 nsec/call
clock-getres-realtime: libc: 277 nsec/call
clock-getres-realtime: vdso: 10 nsec/call
clock-gettime-realtime-coarse: syscall: 286 nsec/call
clock-gettime-realtime-coarse: libc: 21 nsec/call
clock-gettime-realtime-coarse: vdso: 12 nsec/call
clock-getres-realtime-coarse: syscall: 285 nsec/call
clock-getres-realtime-coarse: libc: 283 nsec/call
clock-getres-realtime-coarse: vdso: 11 nsec/call
getcpu: syscall: 234 nsec/call
getcpu: libc: 31 nsec/call
getcpu: vdso: 20 nsec/call
gettimeofday: syscall: 293 nsec/call
gettimeofday: libc: 32 nsec/call
gettimeofday: vdso: 31 nsec/call
Stock Kernel:
=============
clock-gettime-monotonic: syscall: 349 nsec/call
clock-gettime-monotonic: libc: 37 nsec/call
clock-gettime-monotonic: vdso: 28 nsec/call
clock-getres-monotonic: syscall: 296 nsec/call
clock-getres-monotonic: libc: 295 nsec/call
clock-getres-monotonic: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-coarse: syscall: 296 nsec/call
clock-gettime-monotonic-coarse: libc: 21 nsec/call
clock-gettime-monotonic-coarse: vdso: 11 nsec/call
clock-getres-monotonic-coarse: syscall: 287 nsec/call
clock-getres-monotonic-coarse: libc: 288 nsec/call
clock-getres-monotonic-coarse: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-monotonic-raw: syscall: 353 nsec/call
clock-gettime-monotonic-raw: libc: 360 nsec/call
clock-gettime-monotonic-raw: vdso: 352 nsec/call
clock-getres-monotonic-raw: syscall: 282 nsec/call
clock-getres-monotonic-raw: libc: 286 nsec/call
clock-getres-monotonic-raw: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-tai: syscall: 351 nsec/call
clock-gettime-tai: libc: 364 nsec/call
clock-gettime-tai: vdso: 365 nsec/call
clock-getres-tai: syscall: 287 nsec/call
clock-getres-tai: libc: 287 nsec/call
clock-getres-tai: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-boottime: syscall: 347 nsec/call
clock-gettime-boottime: libc: 364 nsec/call
clock-gettime-boottime: vdso: 355 nsec/call
clock-getres-boottime: syscall: 287 nsec/call
clock-getres-boottime: libc: 287 nsec/call
clock-getres-boottime: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime: syscall: 346 nsec/call
clock-gettime-realtime: libc: 36 nsec/call
clock-gettime-realtime: vdso: 29 nsec/call
clock-getres-realtime: syscall: 285 nsec/call
clock-getres-realtime: libc: 287 nsec/call
clock-getres-realtime: vdso: not tested
Note: vDSO version of clock_getres not found
clock-gettime-realtime-coarse: syscall: 296 nsec/call
clock-gettime-realtime-coarse: libc: 20 nsec/call
clock-gettime-realtime-coarse: vdso: 11 nsec/call
clock-getres-realtime-coarse: syscall: 301 nsec/call
clock-getres-realtime-coarse: libc: 297 nsec/call
clock-getres-realtime-coarse: vdso: not tested
Note: vDSO version of clock_getres not found
getcpu: syscall: 255 nsec/call
getcpu: libc: 32 nsec/call
getcpu: vdso: 21 nsec/call
gettimeofday: syscall: 339 nsec/call
gettimeofday: libc: 31 nsec/call
gettimeofday: vdso: 30 nsec/call
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-27 14:23 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 14:23 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On 23/02/2019 16:51, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +/*
>> + * There is one vdso_clocksource object in vvar for each vDSO clocksource
>> + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
>> + * and optimal in terms of access pattern.
>> + *
>> + * Note that mask and shift are the same for mono and raw.
>> + */
>> +struct vdso_clocksource {
>> + u64 mask; /* Clocksource mask */
>> + u32 mult; /* Clocksource multiplier */
>> + u32 shift; /* Clocksource shift */
>
> Can you please get rid of the tail comments and use proper kerneldoc
> format?
>
I will fix it in v6, thanks.
>> +/*
>> + * vdso_data will be accessed by 32 and 64 bit code at the same time
>> + * so we should be careful before modifying this structure.
>> + */
>> +struct vdso_data {
>> + u32 seq; /* Timebase sequence counter */
>> +
>> + s32 clock_mode;
>> + u64 cycle_last; /* Timebase at clocksource init */
>> +
>> + struct vdso_clocksource cs[CLOCKSOURCE_BASES];
>
> Why would you need different clocksource parameters? That really bloats the
> data structure and makes the cache access pattern worse. Also the vdso
> update needs to copy the same data over and over for no value.
>
> The only clock ID which needs a different mult/shift would be
> MONOTONIC_RAW, but if we expose that through the VDSO then we really can be
> smarter than this. See incomplete and uncompilable patch below for
> reference. You get the idea.
>
Ok, thank you for providing the reference code. I will update my patches in v6
accordingly.
...
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-27 14:23 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 14:23 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On 23/02/2019 16:51, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +/*
>> + * There is one vdso_clocksource object in vvar for each vDSO clocksource
>> + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
>> + * and optimal in terms of access pattern.
>> + *
>> + * Note that mask and shift are the same for mono and raw.
>> + */
>> +struct vdso_clocksource {
>> + u64 mask; /* Clocksource mask */
>> + u32 mult; /* Clocksource multiplier */
>> + u32 shift; /* Clocksource shift */
>
> Can you please get rid of the tail comments and use proper kerneldoc
> format?
>
I will fix it in v6, thanks.
>> +/*
>> + * vdso_data will be accessed by 32 and 64 bit code at the same time
>> + * so we should be careful before modifying this structure.
>> + */
>> +struct vdso_data {
>> + u32 seq; /* Timebase sequence counter */
>> +
>> + s32 clock_mode;
>> + u64 cycle_last; /* Timebase at clocksource init */
>> +
>> + struct vdso_clocksource cs[CLOCKSOURCE_BASES];
>
> Why would you need different clocksource parameters? That really bloats the
> data structure and makes the cache access pattern worse. Also the vdso
> update needs to copy the same data over and over for no value.
>
> The only clock ID which needs a different mult/shift would be
> MONOTONIC_RAW, but if we expose that through the VDSO then we really can be
> smarter than this. See incomplete and uncompilable patch below for
> reference. You get the idea.
>
Ok, thank you for providing the reference code. I will update my patches in v6
accordingly.
...
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 01/23] kernel: Standardize vdso_datapage
@ 2019-02-27 14:23 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 14:23 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On 23/02/2019 16:51, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> +/*
>> + * There is one vdso_clocksource object in vvar for each vDSO clocksource
>> + * (mono, raw). This struct is designed to keep vdso_data "cache-line friendly"
>> + * and optimal in terms of access pattern.
>> + *
>> + * Note that mask and shift are the same for mono and raw.
>> + */
>> +struct vdso_clocksource {
>> + u64 mask; /* Clocksource mask */
>> + u32 mult; /* Clocksource multiplier */
>> + u32 shift; /* Clocksource shift */
>
> Can you please get rid of the tail comments and use proper kerneldoc
> format?
>
I will fix it in v6, thanks.
>> +/*
>> + * vdso_data will be accessed by 32 and 64 bit code at the same time
>> + * so we should be careful before modifying this structure.
>> + */
>> +struct vdso_data {
>> + u32 seq; /* Timebase sequence counter */
>> +
>> + s32 clock_mode;
>> + u64 cycle_last; /* Timebase at clocksource init */
>> +
>> + struct vdso_clocksource cs[CLOCKSOURCE_BASES];
>
> Why would you need different clocksource parameters? That really bloats the
> data structure and makes the cache access pattern worse. Also the vdso
> update needs to copy the same data over and over for no value.
>
> The only clock ID which needs a different mult/shift would be
> MONOTONIC_RAW, but if we expose that through the VDSO then we really can be
> smarter than this. See incomplete and uncompilable patch below for
> reference. You get the idea.
>
Ok, thank you for providing the reference code. I will update my patches in v6
accordingly.
...
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 14:52 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 14:52 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
Hi Arnd,
thank you for your review.
On 22/02/2019 13:49, Arnd Bergmann wrote:
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>
>> +/*
>> + * The definitions below are required to overcome the limitations
>> + * of time_t on 32 bit architectures, which overflows in 2038.
>> + * The new code should use the replacements based on time64_t and
>> + * timespec64.
>> + *
>> + * The abstraction below will be updated once the migration to
>> + * time64_t is complete.
>> + */
>> +#ifdef CONFIG_GENERIC_VDSO_32
>> +#define __vdso_timespec old_timespec32
>> +#define __vdso_timeval old_timeval32
>> +#else
>> +#ifdef ENABLE_COMPAT_VDSO
>> +#define __vdso_timespec old_timespec32
>> +#define __vdso_timeval old_timeval32
>> +#else
>> +#define __vdso_timespec __kernel_timespec
>> +#define __vdso_timeval __kernel_old_timeval
>> +#endif /* CONFIG_COMPAT_VDSO */
>> +#endif /* CONFIG_GENERIC_VDSO_32 */
>
> I don't think we need __vdso_timeval at all, just use
> __kernel_old_timeval everywhere. There is no generic
> 64-bit timeval type in the kernel, and there won't be because
> gettimeofday() is deprecated.
>
Ok, I will update my implementation accordingly in v6.
> For __vdso_timespec, I see how you ended up with this
> redefinition, and it makes the current version of your patches
> easier, but I fear it will in turn make it harder to add the
> __kernel_old_timeval based variant.
>
What is __kernel_old_timespec (based on you next email)? Why do you think it
will make harder to add the new variants?
--
Regards,
Vincenzo
> What I'd prefer to see here is to always use the fixed length
> types everywhere in the code, such as
>
> static notrace int __cvdso_clock_gettime64(clockid_t clock,
> struct __kernel_timespec *ts)
> {
> ... /* normal clock_gettime using 64-bit times */
> }
>
> static notrace int __cvdso_clock_gettime32(clockid_t clock,
> struct old_timespec32 *ts32)
> {
> struct __kernel_timespec ts;
> int ret = __cvdso_clock_gettime64(clock, &ts);
>
> ts32->tv_sec = ts->tv_sec;
> ts32->tv_nsec = ts->tv_nsec;
>
> return ret;
> }
>
> and then have two different versions for 32-bit and 64-bit
> architectures, calling one or the other function of the
> generic code.
>
> Arnd
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 14:52 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 14:52 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
Hi Arnd,
thank you for your review.
On 22/02/2019 13:49, Arnd Bergmann wrote:
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>
>> +/*
>> + * The definitions below are required to overcome the limitations
>> + * of time_t on 32 bit architectures, which overflows in 2038.
>> + * The new code should use the replacements based on time64_t and
>> + * timespec64.
>> + *
>> + * The abstraction below will be updated once the migration to
>> + * time64_t is complete.
>> + */
>> +#ifdef CONFIG_GENERIC_VDSO_32
>> +#define __vdso_timespec old_timespec32
>> +#define __vdso_timeval old_timeval32
>> +#else
>> +#ifdef ENABLE_COMPAT_VDSO
>> +#define __vdso_timespec old_timespec32
>> +#define __vdso_timeval old_timeval32
>> +#else
>> +#define __vdso_timespec __kernel_timespec
>> +#define __vdso_timeval __kernel_old_timeval
>> +#endif /* CONFIG_COMPAT_VDSO */
>> +#endif /* CONFIG_GENERIC_VDSO_32 */
>
> I don't think we need __vdso_timeval at all, just use
> __kernel_old_timeval everywhere. There is no generic
> 64-bit timeval type in the kernel, and there won't be because
> gettimeofday() is deprecated.
>
Ok, I will update my implementation accordingly in v6.
> For __vdso_timespec, I see how you ended up with this
> redefinition, and it makes the current version of your patches
> easier, but I fear it will in turn make it harder to add the
> __kernel_old_timeval based variant.
>
What is __kernel_old_timespec (based on you next email)? Why do you think it
will make harder to add the new variants?
--
Regards,
Vincenzo
> What I'd prefer to see here is to always use the fixed length
> types everywhere in the code, such as
>
> static notrace int __cvdso_clock_gettime64(clockid_t clock,
> struct __kernel_timespec *ts)
> {
> ... /* normal clock_gettime using 64-bit times */
> }
>
> static notrace int __cvdso_clock_gettime32(clockid_t clock,
> struct old_timespec32 *ts32)
> {
> struct __kernel_timespec ts;
> int ret = __cvdso_clock_gettime64(clock, &ts);
>
> ts32->tv_sec = ts->tv_sec;
> ts32->tv_nsec = ts->tv_nsec;
>
> return ret;
> }
>
> and then have two different versions for 32-bit and 64-bit
> architectures, calling one or the other function of the
> generic code.
>
> Arnd
>
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 14:52 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 14:52 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
Hi Arnd,
thank you for your review.
On 22/02/2019 13:49, Arnd Bergmann wrote:
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>
>> +/*
>> + * The definitions below are required to overcome the limitations
>> + * of time_t on 32 bit architectures, which overflows in 2038.
>> + * The new code should use the replacements based on time64_t and
>> + * timespec64.
>> + *
>> + * The abstraction below will be updated once the migration to
>> + * time64_t is complete.
>> + */
>> +#ifdef CONFIG_GENERIC_VDSO_32
>> +#define __vdso_timespec old_timespec32
>> +#define __vdso_timeval old_timeval32
>> +#else
>> +#ifdef ENABLE_COMPAT_VDSO
>> +#define __vdso_timespec old_timespec32
>> +#define __vdso_timeval old_timeval32
>> +#else
>> +#define __vdso_timespec __kernel_timespec
>> +#define __vdso_timeval __kernel_old_timeval
>> +#endif /* CONFIG_COMPAT_VDSO */
>> +#endif /* CONFIG_GENERIC_VDSO_32 */
>
> I don't think we need __vdso_timeval at all, just use
> __kernel_old_timeval everywhere. There is no generic
> 64-bit timeval type in the kernel, and there won't be because
> gettimeofday() is deprecated.
>
Ok, I will update my implementation accordingly in v6.
> For __vdso_timespec, I see how you ended up with this
> redefinition, and it makes the current version of your patches
> easier, but I fear it will in turn make it harder to add the
> __kernel_old_timeval based variant.
>
What is __kernel_old_timespec (based on you next email)? Why do you think it
will make harder to add the new variants?
--
Regards,
Vincenzo
> What I'd prefer to see here is to always use the fixed length
> types everywhere in the code, such as
>
> static notrace int __cvdso_clock_gettime64(clockid_t clock,
> struct __kernel_timespec *ts)
> {
> ... /* normal clock_gettime using 64-bit times */
> }
>
> static notrace int __cvdso_clock_gettime32(clockid_t clock,
> struct old_timespec32 *ts32)
> {
> struct __kernel_timespec ts;
> int ret = __cvdso_clock_gettime64(clock, &ts);
>
> ts32->tv_sec = ts->tv_sec;
> ts32->tv_nsec = ts->tv_nsec;
>
> return ret;
> }
>
> and then have two different versions for 32-bit and 64-bit
> architectures, calling one or the other function of the
> generic code.
>
> Arnd
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 15:49 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-27 15:49 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
Vincenzo,
On Wed, 27 Feb 2019, Vincenzo Frascino wrote:
>
> The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
> the situation with my scheme should be the following:
Oops. I misread the patch, but still...
> u32 seq: + 0
> s32 clock_mode; + 4
> u64 cycle_last; + 8
> struct vdso_cs cs[2]; + 16
> struct vdso_ts basetime[VDSO_BASES]; + 48
>
> which I agree makes still things a bit worse.
> > It's easy enough to benchmark these implementations and without trying I'm
> > pretty sure that you can see the performance drop nicely. Please do so next
> > time and provide the numbers in the changelogs.
> >
>
> I did run some benchmarks this morning to quantify the performance impact and
> seems that using vdsotest[1] the difference in between a stock linux kernel
> 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
> 5120T), is below 1%. Please find the results below, I will add them as well to
> the next changelog.
I have some doubts about 1%.
NEW STOCK
clock-gettime-monotonic: vdso: 31 28 ~ 10% slower
clock-gettime-realtime: vdso: 32 29 ~ 10% slower
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 15:49 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-27 15:49 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
Vincenzo,
On Wed, 27 Feb 2019, Vincenzo Frascino wrote:
>
> The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
> the situation with my scheme should be the following:
Oops. I misread the patch, but still...
> u32 seq: + 0
> s32 clock_mode; + 4
> u64 cycle_last; + 8
> struct vdso_cs cs[2]; + 16
> struct vdso_ts basetime[VDSO_BASES]; + 48
>
> which I agree makes still things a bit worse.
> > It's easy enough to benchmark these implementations and without trying I'm
> > pretty sure that you can see the performance drop nicely. Please do so next
> > time and provide the numbers in the changelogs.
> >
>
> I did run some benchmarks this morning to quantify the performance impact and
> seems that using vdsotest[1] the difference in between a stock linux kernel
> 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
> 5120T), is below 1%. Please find the results below, I will add them as well to
> the next changelog.
I have some doubts about 1%.
NEW STOCK
clock-gettime-monotonic: vdso: 31 28 ~ 10% slower
clock-gettime-realtime: vdso: 32 29 ~ 10% slower
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 15:49 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-27 15:49 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
Vincenzo,
On Wed, 27 Feb 2019, Vincenzo Frascino wrote:
>
> The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
> the situation with my scheme should be the following:
Oops. I misread the patch, but still...
> u32 seq: + 0
> s32 clock_mode; + 4
> u64 cycle_last; + 8
> struct vdso_cs cs[2]; + 16
> struct vdso_ts basetime[VDSO_BASES]; + 48
>
> which I agree makes still things a bit worse.
> > It's easy enough to benchmark these implementations and without trying I'm
> > pretty sure that you can see the performance drop nicely. Please do so next
> > time and provide the numbers in the changelogs.
> >
>
> I did run some benchmarks this morning to quantify the performance impact and
> seems that using vdsotest[1] the difference in between a stock linux kernel
> 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
> 5120T), is below 1%. Please find the results below, I will add them as well to
> the next changelog.
I have some doubts about 1%.
NEW STOCK
clock-gettime-monotonic: vdso: 31 28 ~ 10% slower
clock-gettime-realtime: vdso: 32 29 ~ 10% slower
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 16:06 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 16:06 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On 27/02/2019 15:49, Thomas Gleixner wrote:
> Vincenzo,
>
> On Wed, 27 Feb 2019, Vincenzo Frascino wrote:
>>
>> The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
>> the situation with my scheme should be the following:
>
> Oops. I misread the patch, but still...
>
>> u32 seq: + 0
>> s32 clock_mode; + 4
>> u64 cycle_last; + 8
>> struct vdso_cs cs[2]; + 16
>> struct vdso_ts basetime[VDSO_BASES]; + 48
>>
>> which I agree makes still things a bit worse.
>
>>> It's easy enough to benchmark these implementations and without trying I'm
>>> pretty sure that you can see the performance drop nicely. Please do so next
>>> time and provide the numbers in the changelogs.
>>>
>>
>> I did run some benchmarks this morning to quantify the performance impact and
>> seems that using vdsotest[1] the difference in between a stock linux kernel
>> 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
>> 5120T), is below 1%. Please find the results below, I will add them as well to
>> the next changelog.
>
> I have some doubts about 1%.
> NEW STOCK
> clock-gettime-monotonic: vdso: 31 28 ~ 10% slower
> clock-gettime-realtime: vdso: 32 29 ~ 10% slower
>
Sorry, there was an error in my script that I used to extract percentages. I
agree! Luckily I shared the numbers. Thanks Thomas.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 16:06 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 16:06 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On 27/02/2019 15:49, Thomas Gleixner wrote:
> Vincenzo,
>
> On Wed, 27 Feb 2019, Vincenzo Frascino wrote:
>>
>> The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
>> the situation with my scheme should be the following:
>
> Oops. I misread the patch, but still...
>
>> u32 seq: + 0
>> s32 clock_mode; + 4
>> u64 cycle_last; + 8
>> struct vdso_cs cs[2]; + 16
>> struct vdso_ts basetime[VDSO_BASES]; + 48
>>
>> which I agree makes still things a bit worse.
>
>>> It's easy enough to benchmark these implementations and without trying I'm
>>> pretty sure that you can see the performance drop nicely. Please do so next
>>> time and provide the numbers in the changelogs.
>>>
>>
>> I did run some benchmarks this morning to quantify the performance impact and
>> seems that using vdsotest[1] the difference in between a stock linux kernel
>> 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
>> 5120T), is below 1%. Please find the results below, I will add them as well to
>> the next changelog.
>
> I have some doubts about 1%.
> NEW STOCK
> clock-gettime-monotonic: vdso: 31 28 ~ 10% slower
> clock-gettime-realtime: vdso: 32 29 ~ 10% slower
>
Sorry, there was an error in my script that I used to extract percentages. I
agree! Luckily I shared the numbers. Thanks Thomas.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-27 16:06 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 16:06 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On 27/02/2019 15:49, Thomas Gleixner wrote:
> Vincenzo,
>
> On Wed, 27 Feb 2019, Vincenzo Frascino wrote:
>>
>> The clocksource array has two elements (CLOCKSOURCE_RAW, CLOCKSOURCE_MONO) and
>> the situation with my scheme should be the following:
>
> Oops. I misread the patch, but still...
>
>> u32 seq: + 0
>> s32 clock_mode; + 4
>> u64 cycle_last; + 8
>> struct vdso_cs cs[2]; + 16
>> struct vdso_ts basetime[VDSO_BASES]; + 48
>>
>> which I agree makes still things a bit worse.
>
>>> It's easy enough to benchmark these implementations and without trying I'm
>>> pretty sure that you can see the performance drop nicely. Please do so next
>>> time and provide the numbers in the changelogs.
>>>
>>
>> I did run some benchmarks this morning to quantify the performance impact and
>> seems that using vdsotest[1] the difference in between a stock linux kernel
>> 5.0.0-rc7 and one that has unified vDSO, running on my x86 machine (Xeon Gold
>> 5120T), is below 1%. Please find the results below, I will add them as well to
>> the next changelog.
>
> I have some doubts about 1%.
> NEW STOCK
> clock-gettime-monotonic: vdso: 31 28 ~ 10% slower
> clock-gettime-realtime: vdso: 32 29 ~ 10% slower
>
Sorry, there was an error in my script that I used to extract percentages. I
agree! Luckily I shared the numbers. Thanks Thomas.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-27 16:18 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 16:18 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On 23/02/2019 10:45, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> void update_vsyscall_tz(void)
>> {
>> - vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
>> - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
>> + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
>> + vdso_data.tz_dsttime = sys_tz.tz_dsttime;
>> }
>>
>> void update_vsyscall(struct timekeeper *tk)
>> {
>> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
>> - struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
>> - struct vgtod_ts *base;
>> + struct vdso_data *vdata = &vdso_data;
>> + struct vdso_timestamp *vdso_ts;
>> u64 nsec;
>>
>> /* Mark the new vclock used. */
>> BUILD_BUG_ON(VCLOCK_MAX >= 32);
>> WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
>>
>> - gtod_write_begin(vdata);
>> + vdso_write_begin(vdata);
>
> This begs the question why this update code is still architecture
> specific. The vdso data is now generic, so the whole update can be generic
> as well. The only x86 specific thing is this perhaps:
>
> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
>
> vdata->clock_mode = vclock_mode;
>
> But either than can be made generic and mandatory or easily resolved with a
> trivial arch_update_vsyscall() inline.
>
I agree, I will make it generic in the next patch series.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-27 16:18 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 16:18 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On 23/02/2019 10:45, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> void update_vsyscall_tz(void)
>> {
>> - vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
>> - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
>> + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
>> + vdso_data.tz_dsttime = sys_tz.tz_dsttime;
>> }
>>
>> void update_vsyscall(struct timekeeper *tk)
>> {
>> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
>> - struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
>> - struct vgtod_ts *base;
>> + struct vdso_data *vdata = &vdso_data;
>> + struct vdso_timestamp *vdso_ts;
>> u64 nsec;
>>
>> /* Mark the new vclock used. */
>> BUILD_BUG_ON(VCLOCK_MAX >= 32);
>> WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
>>
>> - gtod_write_begin(vdata);
>> + vdso_write_begin(vdata);
>
> This begs the question why this update code is still architecture
> specific. The vdso data is now generic, so the whole update can be generic
> as well. The only x86 specific thing is this perhaps:
>
> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
>
> vdata->clock_mode = vclock_mode;
>
> But either than can be made generic and mandatory or easily resolved with a
> trivial arch_update_vsyscall() inline.
>
I agree, I will make it generic in the next patch series.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 22/23] x86: Add support for generic vDSO
@ 2019-02-27 16:18 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-27 16:18 UTC (permalink / raw)
To: Thomas Gleixner
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Peter Collingbourne, linux-arm-kernel
On 23/02/2019 10:45, Thomas Gleixner wrote:
> On Fri, 22 Feb 2019, Vincenzo Frascino wrote:
>> void update_vsyscall_tz(void)
>> {
>> - vsyscall_gtod_data.tz_minuteswest = sys_tz.tz_minuteswest;
>> - vsyscall_gtod_data.tz_dsttime = sys_tz.tz_dsttime;
>> + vdso_data.tz_minuteswest = sys_tz.tz_minuteswest;
>> + vdso_data.tz_dsttime = sys_tz.tz_dsttime;
>> }
>>
>> void update_vsyscall(struct timekeeper *tk)
>> {
>> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
>> - struct vsyscall_gtod_data *vdata = &vsyscall_gtod_data;
>> - struct vgtod_ts *base;
>> + struct vdso_data *vdata = &vdso_data;
>> + struct vdso_timestamp *vdso_ts;
>> u64 nsec;
>>
>> /* Mark the new vclock used. */
>> BUILD_BUG_ON(VCLOCK_MAX >= 32);
>> WRITE_ONCE(vclocks_used, READ_ONCE(vclocks_used) | (1 << vclock_mode));
>>
>> - gtod_write_begin(vdata);
>> + vdso_write_begin(vdata);
>
> This begs the question why this update code is still architecture
> specific. The vdso data is now generic, so the whole update can be generic
> as well. The only x86 specific thing is this perhaps:
>
> int vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
>
> vdata->clock_mode = vclock_mode;
>
> But either than can be made generic and mandatory or easily resolved with a
> trivial arch_update_vsyscall() inline.
>
I agree, I will make it generic in the next patch series.
> Thanks,
>
> tglx
>
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-28 9:29 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 9:29 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Wed, Feb 27, 2019 at 3:52 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> On 22/02/2019 13:49, Arnd Bergmann wrote:
> > On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >
> >> +/*
> >> + * The definitions below are required to overcome the limitations
> >> + * of time_t on 32 bit architectures, which overflows in 2038.
> >> + * The new code should use the replacements based on time64_t and
> >> + * timespec64.
> >> + *
> >> + * The abstraction below will be updated once the migration to
> >> + * time64_t is complete.
> >> + */
> >> +#ifdef CONFIG_GENERIC_VDSO_32
> >> +#define __vdso_timespec old_timespec32
> >> +#define __vdso_timeval old_timeval32
> >> +#else
> >> +#ifdef ENABLE_COMPAT_VDSO
> >> +#define __vdso_timespec old_timespec32
> >> +#define __vdso_timeval old_timeval32
> >> +#else
> >> +#define __vdso_timespec __kernel_timespec
> >> +#define __vdso_timeval __kernel_old_timeval
> >> +#endif /* CONFIG_COMPAT_VDSO */
> >> +#endif /* CONFIG_GENERIC_VDSO_32 */
> > For __vdso_timespec, I see how you ended up with this
> > redefinition, and it makes the current version of your patches
> > easier, but I fear it will in turn make it harder to add the
> > __kernel_old_timeval based variant.
> >
>
> What is __kernel_old_timespec (based on you next email)? Why do you think it
> will make harder to add the new variants?
I mean you should ideally use the types that you have listed above
directly, without the abstraction.
In the long run I want one set of functions using __kernel_timespec that
implements the 64-bit time interfaces on both 32-bit and 64-bit kernels,
including the compat vdso, and another set using old_timespec32 for
just the native 32-bit and compat version.
This would match what we do in the normal system calls (in linux-5.1+),
where we always have the regular implementation use 64-bit types
only, and have an optional _time32 version that is used for existing
32-bit user space, on both native 32 bit kernels and compat syscalls
on 64-bit.
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-28 9:29 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 9:29 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Wed, Feb 27, 2019 at 3:52 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> On 22/02/2019 13:49, Arnd Bergmann wrote:
> > On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >
> >> +/*
> >> + * The definitions below are required to overcome the limitations
> >> + * of time_t on 32 bit architectures, which overflows in 2038.
> >> + * The new code should use the replacements based on time64_t and
> >> + * timespec64.
> >> + *
> >> + * The abstraction below will be updated once the migration to
> >> + * time64_t is complete.
> >> + */
> >> +#ifdef CONFIG_GENERIC_VDSO_32
> >> +#define __vdso_timespec old_timespec32
> >> +#define __vdso_timeval old_timeval32
> >> +#else
> >> +#ifdef ENABLE_COMPAT_VDSO
> >> +#define __vdso_timespec old_timespec32
> >> +#define __vdso_timeval old_timeval32
> >> +#else
> >> +#define __vdso_timespec __kernel_timespec
> >> +#define __vdso_timeval __kernel_old_timeval
> >> +#endif /* CONFIG_COMPAT_VDSO */
> >> +#endif /* CONFIG_GENERIC_VDSO_32 */
> > For __vdso_timespec, I see how you ended up with this
> > redefinition, and it makes the current version of your patches
> > easier, but I fear it will in turn make it harder to add the
> > __kernel_old_timeval based variant.
> >
>
> What is __kernel_old_timespec (based on you next email)? Why do you think it
> will make harder to add the new variants?
I mean you should ideally use the types that you have listed above
directly, without the abstraction.
In the long run I want one set of functions using __kernel_timespec that
implements the 64-bit time interfaces on both 32-bit and 64-bit kernels,
including the compat vdso, and another set using old_timespec32 for
just the native 32-bit and compat version.
This would match what we do in the normal system calls (in linux-5.1+),
where we always have the regular implementation use 64-bit types
only, and have an optional _time32 version that is used for existing
32-bit user space, on both native 32 bit kernels and compat syscalls
on 64-bit.
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 02/23] kernel: Define gettimeofday vdso common code
@ 2019-02-28 9:29 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 9:29 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Wed, Feb 27, 2019 at 3:52 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
> On 22/02/2019 13:49, Arnd Bergmann wrote:
> > On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >
> >> +/*
> >> + * The definitions below are required to overcome the limitations
> >> + * of time_t on 32 bit architectures, which overflows in 2038.
> >> + * The new code should use the replacements based on time64_t and
> >> + * timespec64.
> >> + *
> >> + * The abstraction below will be updated once the migration to
> >> + * time64_t is complete.
> >> + */
> >> +#ifdef CONFIG_GENERIC_VDSO_32
> >> +#define __vdso_timespec old_timespec32
> >> +#define __vdso_timeval old_timeval32
> >> +#else
> >> +#ifdef ENABLE_COMPAT_VDSO
> >> +#define __vdso_timespec old_timespec32
> >> +#define __vdso_timeval old_timeval32
> >> +#else
> >> +#define __vdso_timespec __kernel_timespec
> >> +#define __vdso_timeval __kernel_old_timeval
> >> +#endif /* CONFIG_COMPAT_VDSO */
> >> +#endif /* CONFIG_GENERIC_VDSO_32 */
> > For __vdso_timespec, I see how you ended up with this
> > redefinition, and it makes the current version of your patches
> > easier, but I fear it will in turn make it harder to add the
> > __kernel_old_timeval based variant.
> >
>
> What is __kernel_old_timespec (based on you next email)? Why do you think it
> will make harder to add the new variants?
I mean you should ideally use the types that you have listed above
directly, without the abstraction.
In the long run I want one set of functions using __kernel_timespec that
implements the 64-bit time interfaces on both 32-bit and 64-bit kernels,
including the compat vdso, and another set using old_timespec32 for
just the native 32-bit and compat version.
This would match what we do in the normal system calls (in linux-5.1+),
where we always have the regular implementation use 64-bit types
only, and have an optional _time32 version that is used for existing
32-bit user space, on both native 32 bit kernels and compat syscalls
on 64-bit.
Arnd
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 11:40 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:40 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> vDSO (virtual dynamic shared object) is a mechanism that the Linux
> kernel provides as an alternative to system calls to reduce where
> possible the costs in terms of cycles.
> This is possible because certain syscalls like gettimeofday() do
> not write any data and return one or more values that are stored
> in the kernel, which makes relatively safe calling them directly
> as a library function.
I tried applying the series on top of v5.0-rc7, and got a set of
build warnings with arm64 defconfig:
In file included from /git/arm-soc/arch/arm64/include/asm/thread_info.h:30:0,
from /git/arm-soc/include/linux/thread_info.h:38,
from /git/arm-soc/arch/arm64/include/asm/preempt.h:5,
from /git/arm-soc/include/linux/preempt.h:78,
from /git/arm-soc/include/linux/spinlock.h:51,
from /git/arm-soc/include/linux/seqlock.h:36,
from /git/arm-soc/include/linux/time.h:6,
from /git/arm-soc/lib/vdso/gettimeofday.c:7,
from <command-line>:0:
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
shift count >= width of type [enabled by default]
return kimage_vaddr - KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
return __virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
In file included from /git/arm-soc/arch/arm64/include/asm/pgtable-hwdef.h:19:0,
from /git/arm-soc/arch/arm64/include/asm/processor.h:46,
from /git/arm-soc/arch/arm64/include/asm/elf.h:129,
from /git/arm-soc/include/linux/elf.h:5,
from /git/arm-soc/include/linux/elfnote.h:62,
from /git/arm-soc/arch/arm64/kernel/vdso32/note.c:11:
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
shift count >= width of type [enabled by default]
return kimage_vaddr - KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
return __virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
I think this is all harmless, but we need to limit the set of header files
that can be included indirectly when compiling the vdso in arm32 mode.
Arnd
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 11:40 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:40 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> vDSO (virtual dynamic shared object) is a mechanism that the Linux
> kernel provides as an alternative to system calls to reduce where
> possible the costs in terms of cycles.
> This is possible because certain syscalls like gettimeofday() do
> not write any data and return one or more values that are stored
> in the kernel, which makes relatively safe calling them directly
> as a library function.
I tried applying the series on top of v5.0-rc7, and got a set of
build warnings with arm64 defconfig:
In file included from /git/arm-soc/arch/arm64/include/asm/thread_info.h:30:0,
from /git/arm-soc/include/linux/thread_info.h:38,
from /git/arm-soc/arch/arm64/include/asm/preempt.h:5,
from /git/arm-soc/include/linux/preempt.h:78,
from /git/arm-soc/include/linux/spinlock.h:51,
from /git/arm-soc/include/linux/seqlock.h:36,
from /git/arm-soc/include/linux/time.h:6,
from /git/arm-soc/lib/vdso/gettimeofday.c:7,
from <command-line>:0:
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
shift count >= width of type [enabled by default]
return kimage_vaddr - KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
return __virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
In file included from /git/arm-soc/arch/arm64/include/asm/pgtable-hwdef.h:19:0,
from /git/arm-soc/arch/arm64/include/asm/processor.h:46,
from /git/arm-soc/arch/arm64/include/asm/elf.h:129,
from /git/arm-soc/include/linux/elf.h:5,
from /git/arm-soc/include/linux/elfnote.h:62,
from /git/arm-soc/arch/arm64/kernel/vdso32/note.c:11:
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
/git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
shift count >= width of type [enabled by default]
return kimage_vaddr - KIMAGE_VADDR;
^
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
return __virt_to_phys((unsigned long)(x));
^
/git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
shift count >= width of type [enabled by default]
/git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
/git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
shift count >= width of type [enabled by default]
return (void *)(__phys_to_virt(x));
^
I think this is all harmless, but we need to limit the set of header files
that can be included indirectly when compiling the vdso in arm32 mode.
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 11:58 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Hi Vincento,
I've implemented now what I tried to explain earlier. This moves the
implementation of clock_gettime() and clock_getres() over to 64-bit
__kernel_timespec, with a conversion in the end for architectures that
actually use the 32-bit calls.
The vdso/types.h file is now obsolete, but we do need to duplicate
a few functions on mips32 and x86-32. It may be possible to fold those
into the generic lib/vdso/gettimeofday.c file as another step.
If there are no objections, please fold these changes into your
series.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
arch/arm/include/asm/vdso/gettimeofday.h | 16 +++---
arch/arm/vdso/vgettimeofday.c | 10 ++--
.../include/asm/vdso/compat_gettimeofday.h | 16 +++---
arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++--
arch/arm64/kernel/vdso/vgettimeofday.c | 6 +-
arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++--
arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++---
arch/mips/vdso/vgettimeofday.c | 28 ++++++++-
arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------
arch/x86/include/asm/vdso/gettimeofday.h | 16 +++---
include/vdso/datapage.h | 1 -
include/vdso/types.h | 39 -------------
lib/vdso/gettimeofday.c | 48 +++++++++++-----
13 files changed, 164 insertions(+), 121 deletions(-)
delete mode 100644 include/vdso/types.h
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
index e7aeda0b255f..112df50920b0 100644
--- a/arch/arm/include/asm/vdso/gettimeofday.h
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -14,11 +14,11 @@
extern struct vdso_data *__get_datapage(void);
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
- register struct __vdso_timeval *tv asm("r0") = _tv;
+ register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_gettimeofday;
@@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
+ register long nr asm("r7") = __NR_clock_gettime64;
asm volatile(
" swi #0\n"
@@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_getres;
+ register long nr asm("r7") = __NR_clock_getres_time64;
asm volatile(
" swi #0\n"
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index b470a8ed35a7..b4607a5ecf31 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -9,21 +9,21 @@
#include <linux/types.h>
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
{
- return __cvdso_clock_gettime(clock, ts);
+ return __cvdso_clock_gettime32(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct old_timespec32 *res)
{
- return __cvdso_clock_getres(clock_id, res);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
index 888c70c24f71..b41d8f01a20c 100644
--- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -24,11 +24,11 @@
extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
- register struct __vdso_timeval *tv asm("r0") = _tv;
+ register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_gettimeofday;
@@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_compat_clock_gettime;
+ register long nr asm("r7") = __NR_compat_clock_gettime64;
asm volatile(
" swi #0\n"
@@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_compat_clock_getres;
+ register long nr asm("r7") = __NR_compat_clock_getres_time64;
/* The checks below are required for ABI consistency with arm */
if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
index 0aff6362533f..126fc6f4f7cf 100644
--- a/arch/arm64/include/asm/vdso/gettimeofday.h
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -13,11 +13,11 @@
extern struct vdso_data _vdso_data;
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("x1") = _tz;
- register struct __vdso_timeval *tv asm("x0") = _tv;
+ register struct __kernel_old_timeval *tv asm("x0") = _tv;
register long ret asm ("x0");
register long nr asm("x8") = __NR_gettimeofday;
@@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("x1") = _ts;
+ register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_gettime;
@@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("x1") = _ts;
+ register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_getres;
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
index 160e68196339..bbc83b6e2b1a 100644
--- a/arch/arm64/kernel/vdso/vgettimeofday.c
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -9,19 +9,19 @@
#include <linux/types.h>
notrace int __kernel_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __kernel_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
index b470a8ed35a7..6762f800a8ea 100644
--- a/arch/arm64/kernel/vdso32/vgettimeofday.c
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -9,21 +9,27 @@
#include <linux/types.h>
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+notrace int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct old_timespec32 *res)
{
- return __cvdso_clock_getres(clock_id, res);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
index b275392f6b1b..ed6dafda5fc2 100644
--- a/arch/mips/include/asm/vdso/gettimeofday.h
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -25,11 +25,11 @@
#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("a1") = _tz;
- register struct __vdso_timeval *tv asm("a0") = _tv;
+ register struct __kernel_old_timeval *tv asm("a0") = _tv;
register long ret asm("v0");
register long nr asm("v0") = __NR_gettimeofday;
register long error asm("a3");
@@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(
#else
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
return -1;
@@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("a1") = _ts;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
register clockid_t clkid asm("a0") = _clkid;
register long ret asm("v0");
+#if _MIPS_SIM == _MIPS_SIM_ABI64
register long nr asm("v0") = __NR_clock_gettime;
+#else
+ register long nr asm("v0") = __NR_clock_gettime64
+#endif
register long error asm("a3");
asm volatile(
@@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("a1") = _ts;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
register clockid_t clkid asm("a0") = _clkid;
register long ret asm("v0");
+#if _MIPS_SIM == _MIPS_SIM_ABI64
register long nr asm("v0") = __NR_clock_getres;
+#else
+ register long nr asm("v0") = __NR_clock_getres64
+#endif
register long error asm("a3");
asm volatile(
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
index 8da361d919f0..7a3320faa29a 100644
--- a/arch/mips/vdso/vgettimeofday.c
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -8,21 +8,43 @@
#include <linux/time.h>
#include <linux/types.h>
+#if _MIPS_SIM != _MIPS_SIM_ABI64
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday_time32(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct old_timespec32 *res)
+{
+ return __cvdso_clock_getres_time32(clock_id, res);
+}
+
+#else
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
+#endif
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index 6ff8cabf4c3e..c9512c114ac1 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -12,25 +12,14 @@
#include "../../../../lib/vdso/gettimeofday.c"
-extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
-extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
+extern int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
extern time_t __vdso_time(time_t *t);
-extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
-
-notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
-{
- return __cvdso_clock_gettime(clock, ts);
-}
-
-int clock_gettime(clockid_t, struct __vdso_timespec *)
- __attribute__((weak, alias("__vdso_clock_gettime")));
-
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
-int gettimeofday(struct __vdso_timeval *, struct timezone *)
+int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
notrace time_t __vdso_time(time_t *t)
@@ -40,10 +29,46 @@ notrace time_t __vdso_time(time_t *t)
time_t time(time_t *t)
__attribute__((weak, alias("__vdso_time")));
+
+#ifdef __x86_64__
+/* both 64-bit and x32 use these */
+extern int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
+extern int __vdso_clock_getres(clockid_t clock, struct __kernel_timespec *res);
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+int clock_gettime(clockid_t, struct __kernel_timespec *)
+ __attribute__((weak, alias("__vdso_clock_gettime")));
+
notrace int __vdso_clock_getres(clockid_t clock,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock, res);
}
-int clock_getres(clockid_t, struct __vdso_timespec *)
+int clock_getres(clockid_t, struct __kernel_timespec *)
+ __attribute__((weak, alias("__vdso_clock_getres")));
+
+#else
+/* i386 only */
+extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
+extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+int clock_gettime(clockid_t, struct old_timespec32 *)
+ __attribute__((weak, alias("__vdso_clock_gettime")));
+
+notrace int __vdso_clock_getres(clockid_t clock,
+ struct old_timespec32 *res)
+{
+ return __cvdso_clock_getres_time32(clock, res);
+}
+int clock_getres(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_getres")));
+#endif
diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
index 1a86fdc1e400..510a0ebd5748 100644
--- a/arch/x86/include/asm/vdso/gettimeofday.h
+++ b/arch/x86/include/asm/vdso/gettimeofday.h
@@ -38,7 +38,7 @@ extern u8 hvclock_page
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback(
}
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
@@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
@@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback(
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
- : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
+ : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
@@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
@@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback(
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
- : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
+ : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index ff332fcba73c..da346ad02b03 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,7 +9,6 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
-#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/types.h b/include/vdso/types.h
deleted file mode 100644
index f456a0a6a2e1..000000000000
--- a/include/vdso/types.h
+++ /dev/null
@@ -1,39 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef __VDSO_TYPES_H
-#define __VDSO_TYPES_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-#include <linux/types.h>
-#include <linux/time.h>
-
-/*
- * The definitions below are required to overcome the limitations
- * of time_t on 32 bit architectures, which overflows in 2038.
- * The new code should use the replacements based on time64_t and
- * timespec64.
- *
- * The abstraction below will be updated once the migration to
- * time64_t is complete.
- */
-#ifdef CONFIG_GENERIC_VDSO_32
-#define __vdso_timespec old_timespec32
-#define __vdso_timeval old_timeval32
-#else
-#ifdef ENABLE_COMPAT_VDSO
-#define __vdso_timespec old_timespec32
-#define __vdso_timeval old_timeval32
-#else
-#define __vdso_timespec __kernel_timespec
-#define __vdso_timeval __kernel_old_timeval
-#endif /* CONFIG_COMPAT_VDSO */
-#endif /* CONFIG_GENERIC_VDSO_32 */
-
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __VDSO_TYPES_H */
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 466d0c68fa80..f10e3703abaa 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -28,7 +28,7 @@
static notrace int do_hres(const struct vdso_data *vd,
clockid_t clk,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
u64 cycles, last, sec, ns;
@@ -63,7 +63,7 @@ static notrace int do_hres(const struct vdso_data *vd,
static notrace void do_coarse(const struct vdso_data *vd,
clockid_t clk,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
u32 seq;
@@ -75,8 +75,8 @@ static notrace void do_coarse(const struct vdso_data *vd,
} while (unlikely(vdso_read_retry(vd, seq)));
}
-static notrace int __cvdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+static notrace __maybe_unused int
+__cvdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
{
const struct vdso_data *vd = __arch_get_vdso_data();
u32 msk;
@@ -100,19 +100,31 @@ static notrace int __cvdso_clock_gettime(clockid_t clock,
return clock_gettime_fallback(clock, ts);
}
-static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
- struct timezone *tz)
+static notrace __maybe_unused int
+__cvdso_clock_gettime32(clockid_t clock, struct old_timespec32 *res)
+{
+ struct __kernel_timespec ts;
+ int ret = __cvdso_clock_gettime(clock, &ts);
+
+ res->tv_sec = ts.tv_sec;
+ res->tv_nsec = ts.tv_nsec;
+
+ return ret;
+}
+
+static notrace __maybe_unused int
+__cvdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
{
const struct vdso_data *vd = __arch_get_vdso_data();
if (likely(tv != NULL)) {
- struct __vdso_timespec ts;
+ struct __kernel_timespec ts;
if (do_hres(vd, CLOCK_REALTIME, &ts))
return gettimeofday_fallback(tv, tz);
tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
+ tv->tv_usec = (u32)ts.tv_nsec / NSEC_PER_USEC;
}
if (unlikely(tz != NULL)) {
@@ -123,8 +135,7 @@ static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
return 0;
}
-#ifdef VDSO_HAS_TIME
-static notrace time_t __cvdso_time(time_t *time)
+static notrace __maybe_unused time_t __cvdso_time(time_t *time)
{
const struct vdso_data *vd = __arch_get_vdso_data();
time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
@@ -134,10 +145,9 @@ static notrace time_t __cvdso_time(time_t *time)
return t;
}
-#endif /* VDSO_HAS_TIME */
-static notrace int __cvdso_clock_getres(clockid_t clock,
- struct __vdso_timespec *res)
+static notrace __maybe_unused
+int __cvdso_clock_getres(clockid_t clock, struct __kernel_timespec *res)
{
u64 sec, ns;
u32 msk;
@@ -177,3 +187,15 @@ static notrace int __cvdso_clock_getres(clockid_t clock,
fallback:
return clock_getres_fallback(clock, res);
}
+
+static notrace __maybe_unused int
+__cvdso_clock_getres_time32(clockid_t clock, struct old_timespec32 *res)
+{
+ struct __kernel_timespec ts;
+ int ret = __cvdso_clock_getres(clock, &ts);
+
+ res->tv_sec = ts.tv_sec;
+ res->tv_nsec = ts.tv_nsec;
+
+ return ret;
+}
--
2.20.0
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 11:58 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Hi Vincento,
I've implemented now what I tried to explain earlier. This moves the
implementation of clock_gettime() and clock_getres() over to 64-bit
__kernel_timespec, with a conversion in the end for architectures that
actually use the 32-bit calls.
The vdso/types.h file is now obsolete, but we do need to duplicate
a few functions on mips32 and x86-32. It may be possible to fold those
into the generic lib/vdso/gettimeofday.c file as another step.
If there are no objections, please fold these changes into your
series.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
arch/arm/include/asm/vdso/gettimeofday.h | 16 +++---
arch/arm/vdso/vgettimeofday.c | 10 ++--
.../include/asm/vdso/compat_gettimeofday.h | 16 +++---
arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++--
arch/arm64/kernel/vdso/vgettimeofday.c | 6 +-
arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++--
arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++---
arch/mips/vdso/vgettimeofday.c | 28 ++++++++-
arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------
arch/x86/include/asm/vdso/gettimeofday.h | 16 +++---
include/vdso/datapage.h | 1 -
include/vdso/types.h | 39 -------------
lib/vdso/gettimeofday.c | 48 +++++++++++-----
13 files changed, 164 insertions(+), 121 deletions(-)
delete mode 100644 include/vdso/types.h
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
index e7aeda0b255f..112df50920b0 100644
--- a/arch/arm/include/asm/vdso/gettimeofday.h
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -14,11 +14,11 @@
extern struct vdso_data *__get_datapage(void);
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
- register struct __vdso_timeval *tv asm("r0") = _tv;
+ register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_gettimeofday;
@@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
+ register long nr asm("r7") = __NR_clock_gettime64;
asm volatile(
" swi #0\n"
@@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_getres;
+ register long nr asm("r7") = __NR_clock_getres_time64;
asm volatile(
" swi #0\n"
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index b470a8ed35a7..b4607a5ecf31 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -9,21 +9,21 @@
#include <linux/types.h>
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
{
- return __cvdso_clock_gettime(clock, ts);
+ return __cvdso_clock_gettime32(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct old_timespec32 *res)
{
- return __cvdso_clock_getres(clock_id, res);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
index 888c70c24f71..b41d8f01a20c 100644
--- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -24,11 +24,11 @@
extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
- register struct __vdso_timeval *tv asm("r0") = _tv;
+ register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_gettimeofday;
@@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_compat_clock_gettime;
+ register long nr asm("r7") = __NR_compat_clock_gettime64;
asm volatile(
" swi #0\n"
@@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_compat_clock_getres;
+ register long nr asm("r7") = __NR_compat_clock_getres_time64;
/* The checks below are required for ABI consistency with arm */
if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
index 0aff6362533f..126fc6f4f7cf 100644
--- a/arch/arm64/include/asm/vdso/gettimeofday.h
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -13,11 +13,11 @@
extern struct vdso_data _vdso_data;
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("x1") = _tz;
- register struct __vdso_timeval *tv asm("x0") = _tv;
+ register struct __kernel_old_timeval *tv asm("x0") = _tv;
register long ret asm ("x0");
register long nr asm("x8") = __NR_gettimeofday;
@@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("x1") = _ts;
+ register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_gettime;
@@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("x1") = _ts;
+ register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_getres;
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
index 160e68196339..bbc83b6e2b1a 100644
--- a/arch/arm64/kernel/vdso/vgettimeofday.c
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -9,19 +9,19 @@
#include <linux/types.h>
notrace int __kernel_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __kernel_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
index b470a8ed35a7..6762f800a8ea 100644
--- a/arch/arm64/kernel/vdso32/vgettimeofday.c
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -9,21 +9,27 @@
#include <linux/types.h>
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+notrace int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct old_timespec32 *res)
{
- return __cvdso_clock_getres(clock_id, res);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
index b275392f6b1b..ed6dafda5fc2 100644
--- a/arch/mips/include/asm/vdso/gettimeofday.h
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -25,11 +25,11 @@
#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("a1") = _tz;
- register struct __vdso_timeval *tv asm("a0") = _tv;
+ register struct __kernel_old_timeval *tv asm("a0") = _tv;
register long ret asm("v0");
register long nr asm("v0") = __NR_gettimeofday;
register long error asm("a3");
@@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(
#else
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
return -1;
@@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("a1") = _ts;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
register clockid_t clkid asm("a0") = _clkid;
register long ret asm("v0");
+#if _MIPS_SIM == _MIPS_SIM_ABI64
register long nr asm("v0") = __NR_clock_gettime;
+#else
+ register long nr asm("v0") = __NR_clock_gettime64
+#endif
register long error asm("a3");
asm volatile(
@@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("a1") = _ts;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
register clockid_t clkid asm("a0") = _clkid;
register long ret asm("v0");
+#if _MIPS_SIM == _MIPS_SIM_ABI64
register long nr asm("v0") = __NR_clock_getres;
+#else
+ register long nr asm("v0") = __NR_clock_getres64
+#endif
register long error asm("a3");
asm volatile(
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
index 8da361d919f0..7a3320faa29a 100644
--- a/arch/mips/vdso/vgettimeofday.c
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -8,21 +8,43 @@
#include <linux/time.h>
#include <linux/types.h>
+#if _MIPS_SIM != _MIPS_SIM_ABI64
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday_time32(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct old_timespec32 *res)
+{
+ return __cvdso_clock_getres_time32(clock_id, res);
+}
+
+#else
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
+#endif
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index 6ff8cabf4c3e..c9512c114ac1 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -12,25 +12,14 @@
#include "../../../../lib/vdso/gettimeofday.c"
-extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
-extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
+extern int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
extern time_t __vdso_time(time_t *t);
-extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
-
-notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
-{
- return __cvdso_clock_gettime(clock, ts);
-}
-
-int clock_gettime(clockid_t, struct __vdso_timespec *)
- __attribute__((weak, alias("__vdso_clock_gettime")));
-
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
-int gettimeofday(struct __vdso_timeval *, struct timezone *)
+int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
notrace time_t __vdso_time(time_t *t)
@@ -40,10 +29,46 @@ notrace time_t __vdso_time(time_t *t)
time_t time(time_t *t)
__attribute__((weak, alias("__vdso_time")));
+
+#ifdef __x86_64__
+/* both 64-bit and x32 use these */
+extern int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
+extern int __vdso_clock_getres(clockid_t clock, struct __kernel_timespec *res);
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+int clock_gettime(clockid_t, struct __kernel_timespec *)
+ __attribute__((weak, alias("__vdso_clock_gettime")));
+
notrace int __vdso_clock_getres(clockid_t clock,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock, res);
}
-int clock_getres(clockid_t, struct __vdso_timespec *)
+int clock_getres(clockid_t, struct __kernel_timespec *)
+ __attribute__((weak, alias("__vdso_clock_getres")));
+
+#else
+/* i386 only */
+extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
+extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+int clock_gettime(clockid_t, struct old_timespec32 *)
+ __attribute__((weak, alias("__vdso_clock_gettime")));
+
+notrace int __vdso_clock_getres(clockid_t clock,
+ struct old_timespec32 *res)
+{
+ return __cvdso_clock_getres_time32(clock, res);
+}
+int clock_getres(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_getres")));
+#endif
diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
index 1a86fdc1e400..510a0ebd5748 100644
--- a/arch/x86/include/asm/vdso/gettimeofday.h
+++ b/arch/x86/include/asm/vdso/gettimeofday.h
@@ -38,7 +38,7 @@ extern u8 hvclock_page
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback(
}
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
@@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
@@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback(
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
- : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
+ : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
@@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
@@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback(
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
- : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
+ : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index ff332fcba73c..da346ad02b03 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,7 +9,6 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
-#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/types.h b/include/vdso/types.h
deleted file mode 100644
index f456a0a6a2e1..000000000000
--- a/include/vdso/types.h
+++ /dev/null
@@ -1,39 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef __VDSO_TYPES_H
-#define __VDSO_TYPES_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-#include <linux/types.h>
-#include <linux/time.h>
-
-/*
- * The definitions below are required to overcome the limitations
- * of time_t on 32 bit architectures, which overflows in 2038.
- * The new code should use the replacements based on time64_t and
- * timespec64.
- *
- * The abstraction below will be updated once the migration to
- * time64_t is complete.
- */
-#ifdef CONFIG_GENERIC_VDSO_32
-#define __vdso_timespec old_timespec32
-#define __vdso_timeval old_timeval32
-#else
-#ifdef ENABLE_COMPAT_VDSO
-#define __vdso_timespec old_timespec32
-#define __vdso_timeval old_timeval32
-#else
-#define __vdso_timespec __kernel_timespec
-#define __vdso_timeval __kernel_old_timeval
-#endif /* CONFIG_COMPAT_VDSO */
-#endif /* CONFIG_GENERIC_VDSO_32 */
-
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __VDSO_TYPES_H */
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 466d0c68fa80..f10e3703abaa 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -28,7 +28,7 @@
static notrace int do_hres(const struct vdso_data *vd,
clockid_t clk,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
u64 cycles, last, sec, ns;
@@ -63,7 +63,7 @@ static notrace int do_hres(const struct vdso_data *vd,
static notrace void do_coarse(const struct vdso_data *vd,
clockid_t clk,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
u32 seq;
@@ -75,8 +75,8 @@ static notrace void do_coarse(const struct vdso_data *vd,
} while (unlikely(vdso_read_retry(vd, seq)));
}
-static notrace int __cvdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+static notrace __maybe_unused int
+__cvdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
{
const struct vdso_data *vd = __arch_get_vdso_data();
u32 msk;
@@ -100,19 +100,31 @@ static notrace int __cvdso_clock_gettime(clockid_t clock,
return clock_gettime_fallback(clock, ts);
}
-static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
- struct timezone *tz)
+static notrace __maybe_unused int
+__cvdso_clock_gettime32(clockid_t clock, struct old_timespec32 *res)
+{
+ struct __kernel_timespec ts;
+ int ret = __cvdso_clock_gettime(clock, &ts);
+
+ res->tv_sec = ts.tv_sec;
+ res->tv_nsec = ts.tv_nsec;
+
+ return ret;
+}
+
+static notrace __maybe_unused int
+__cvdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
{
const struct vdso_data *vd = __arch_get_vdso_data();
if (likely(tv != NULL)) {
- struct __vdso_timespec ts;
+ struct __kernel_timespec ts;
if (do_hres(vd, CLOCK_REALTIME, &ts))
return gettimeofday_fallback(tv, tz);
tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
+ tv->tv_usec = (u32)ts.tv_nsec / NSEC_PER_USEC;
}
if (unlikely(tz != NULL)) {
@@ -123,8 +135,7 @@ static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
return 0;
}
-#ifdef VDSO_HAS_TIME
-static notrace time_t __cvdso_time(time_t *time)
+static notrace __maybe_unused time_t __cvdso_time(time_t *time)
{
const struct vdso_data *vd = __arch_get_vdso_data();
time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
@@ -134,10 +145,9 @@ static notrace time_t __cvdso_time(time_t *time)
return t;
}
-#endif /* VDSO_HAS_TIME */
-static notrace int __cvdso_clock_getres(clockid_t clock,
- struct __vdso_timespec *res)
+static notrace __maybe_unused
+int __cvdso_clock_getres(clockid_t clock, struct __kernel_timespec *res)
{
u64 sec, ns;
u32 msk;
@@ -177,3 +187,15 @@ static notrace int __cvdso_clock_getres(clockid_t clock,
fallback:
return clock_getres_fallback(clock, res);
}
+
+static notrace __maybe_unused int
+__cvdso_clock_getres_time32(clockid_t clock, struct old_timespec32 *res)
+{
+ struct __kernel_timespec ts;
+ int ret = __cvdso_clock_getres(clock, &ts);
+
+ res->tv_sec = ts.tv_sec;
+ res->tv_nsec = ts.tv_nsec;
+
+ return ret;
+}
--
2.20.0
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 11:58 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Hi Vincento,
I've implemented now what I tried to explain earlier. This moves the
implementation of clock_gettime() and clock_getres() over to 64-bit
__kernel_timespec, with a conversion in the end for architectures that
actually use the 32-bit calls.
The vdso/types.h file is now obsolete, but we do need to duplicate
a few functions on mips32 and x86-32. It may be possible to fold those
into the generic lib/vdso/gettimeofday.c file as another step.
If there are no objections, please fold these changes into your
series.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
arch/arm/include/asm/vdso/gettimeofday.h | 16 +++---
arch/arm/vdso/vgettimeofday.c | 10 ++--
.../include/asm/vdso/compat_gettimeofday.h | 16 +++---
arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++--
arch/arm64/kernel/vdso/vgettimeofday.c | 6 +-
arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++--
arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++---
arch/mips/vdso/vgettimeofday.c | 28 ++++++++-
arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------
arch/x86/include/asm/vdso/gettimeofday.h | 16 +++---
include/vdso/datapage.h | 1 -
include/vdso/types.h | 39 -------------
lib/vdso/gettimeofday.c | 48 +++++++++++-----
13 files changed, 164 insertions(+), 121 deletions(-)
delete mode 100644 include/vdso/types.h
diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
index e7aeda0b255f..112df50920b0 100644
--- a/arch/arm/include/asm/vdso/gettimeofday.h
+++ b/arch/arm/include/asm/vdso/gettimeofday.h
@@ -14,11 +14,11 @@
extern struct vdso_data *__get_datapage(void);
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
- register struct __vdso_timeval *tv asm("r0") = _tv;
+ register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_gettimeofday;
@@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_gettime;
+ register long nr asm("r7") = __NR_clock_gettime64;
asm volatile(
" swi #0\n"
@@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_clock_getres;
+ register long nr asm("r7") = __NR_clock_getres_time64;
asm volatile(
" swi #0\n"
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index b470a8ed35a7..b4607a5ecf31 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -9,21 +9,21 @@
#include <linux/types.h>
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
{
- return __cvdso_clock_gettime(clock, ts);
+ return __cvdso_clock_gettime32(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct old_timespec32 *res)
{
- return __cvdso_clock_getres(clock_id, res);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
index 888c70c24f71..b41d8f01a20c 100644
--- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h
+++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
@@ -24,11 +24,11 @@
extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("r1") = _tz;
- register struct __vdso_timeval *tv asm("r0") = _tv;
+ register struct __kernel_old_timeval *tv asm("r0") = _tv;
register long ret asm ("r0");
register long nr asm("r7") = __NR_compat_gettimeofday;
@@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_compat_clock_gettime;
+ register long nr asm("r7") = __NR_compat_clock_gettime64;
asm volatile(
" swi #0\n"
@@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("r1") = _ts;
+ register struct __kernel_timespec *ts asm("r1") = _ts;
register clockid_t clkid asm("r0") = _clkid;
register long ret asm ("r0");
- register long nr asm("r7") = __NR_compat_clock_getres;
+ register long nr asm("r7") = __NR_compat_clock_getres_time64;
/* The checks below are required for ABI consistency with arm */
if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
index 0aff6362533f..126fc6f4f7cf 100644
--- a/arch/arm64/include/asm/vdso/gettimeofday.h
+++ b/arch/arm64/include/asm/vdso/gettimeofday.h
@@ -13,11 +13,11 @@
extern struct vdso_data _vdso_data;
static __always_inline notrace int gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("x1") = _tz;
- register struct __vdso_timeval *tv asm("x0") = _tv;
+ register struct __kernel_old_timeval *tv asm("x0") = _tv;
register long ret asm ("x0");
register long nr asm("x8") = __NR_gettimeofday;
@@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("x1") = _ts;
+ register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_gettime;
@@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("x1") = _ts;
+ register struct __kernel_timespec *ts asm("x1") = _ts;
register clockid_t clkid asm("x0") = _clkid;
register long ret asm ("x0");
register long nr asm("x8") = __NR_clock_getres;
diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
index 160e68196339..bbc83b6e2b1a 100644
--- a/arch/arm64/kernel/vdso/vgettimeofday.c
+++ b/arch/arm64/kernel/vdso/vgettimeofday.c
@@ -9,19 +9,19 @@
#include <linux/types.h>
notrace int __kernel_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
+notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __kernel_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
index b470a8ed35a7..6762f800a8ea 100644
--- a/arch/arm64/kernel/vdso32/vgettimeofday.c
+++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
@@ -9,21 +9,27 @@
#include <linux/types.h>
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+notrace int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct old_timespec32 *res)
{
- return __cvdso_clock_getres(clock_id, res);
+ return __cvdso_clock_getres_time32(clock_id, res);
}
/* Avoid unresolved references emitted by GCC */
diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
index b275392f6b1b..ed6dafda5fc2 100644
--- a/arch/mips/include/asm/vdso/gettimeofday.h
+++ b/arch/mips/include/asm/vdso/gettimeofday.h
@@ -25,11 +25,11 @@
#ifdef CONFIG_MIPS_CLOCK_VSYSCALL
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
register struct timezone *tz asm("a1") = _tz;
- register struct __vdso_timeval *tv asm("a0") = _tv;
+ register struct __kernel_old_timeval *tv asm("a0") = _tv;
register long ret asm("v0");
register long nr asm("v0") = __NR_gettimeofday;
register long error asm("a3");
@@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(
#else
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
return -1;
@@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("a1") = _ts;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
register clockid_t clkid asm("a0") = _clkid;
register long ret asm("v0");
+#if _MIPS_SIM == _MIPS_SIM_ABI64
register long nr asm("v0") = __NR_clock_gettime;
+#else
+ register long nr asm("v0") = __NR_clock_gettime64
+#endif
register long error asm("a3");
asm volatile(
@@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback(
static __always_inline notrace int clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
- register struct __vdso_timespec *ts asm("a1") = _ts;
+ register struct __kernel_timespec *ts asm("a1") = _ts;
register clockid_t clkid asm("a0") = _clkid;
register long ret asm("v0");
+#if _MIPS_SIM == _MIPS_SIM_ABI64
register long nr asm("v0") = __NR_clock_getres;
+#else
+ register long nr asm("v0") = __NR_clock_getres64
+#endif
register long error asm("a3");
asm volatile(
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
index 8da361d919f0..7a3320faa29a 100644
--- a/arch/mips/vdso/vgettimeofday.c
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -8,21 +8,43 @@
#include <linux/time.h>
#include <linux/types.h>
+#if _MIPS_SIM != _MIPS_SIM_ABI64
notrace int __vdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+ struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
+ struct timezone *tz)
+{
+ return __cvdso_gettimeofday_time32(tv, tz);
+}
+
+notrace int __vdso_clock_getres(clockid_t clock_id,
+ struct old_timespec32 *res)
+{
+ return __cvdso_clock_getres_time32(clock_id, res);
+}
+
+#else
+
+notrace int __vdso_clock_gettime(clockid_t clock,
+ struct __kernel_timespec *ts)
{
return __cvdso_clock_gettime(clock, ts);
}
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
notrace int __vdso_clock_getres(clockid_t clock_id,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock_id, res);
}
+#endif
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index 6ff8cabf4c3e..c9512c114ac1 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -12,25 +12,14 @@
#include "../../../../lib/vdso/gettimeofday.c"
-extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
-extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
+extern int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
extern time_t __vdso_time(time_t *t);
-extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
-
-notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
-{
- return __cvdso_clock_gettime(clock, ts);
-}
-
-int clock_gettime(clockid_t, struct __vdso_timespec *)
- __attribute__((weak, alias("__vdso_clock_gettime")));
-
-notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
+notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
return __cvdso_gettimeofday(tv, tz);
}
-int gettimeofday(struct __vdso_timeval *, struct timezone *)
+int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
__attribute__((weak, alias("__vdso_gettimeofday")));
notrace time_t __vdso_time(time_t *t)
@@ -40,10 +29,46 @@ notrace time_t __vdso_time(time_t *t)
time_t time(time_t *t)
__attribute__((weak, alias("__vdso_time")));
+
+#ifdef __x86_64__
+/* both 64-bit and x32 use these */
+extern int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
+extern int __vdso_clock_getres(clockid_t clock, struct __kernel_timespec *res);
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
+int clock_gettime(clockid_t, struct __kernel_timespec *)
+ __attribute__((weak, alias("__vdso_clock_gettime")));
+
notrace int __vdso_clock_getres(clockid_t clock,
- struct __vdso_timespec *res)
+ struct __kernel_timespec *res)
{
return __cvdso_clock_getres(clock, res);
}
-int clock_getres(clockid_t, struct __vdso_timespec *)
+int clock_getres(clockid_t, struct __kernel_timespec *)
+ __attribute__((weak, alias("__vdso_clock_getres")));
+
+#else
+/* i386 only */
+extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
+extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
+
+notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
+{
+ return __cvdso_clock_gettime32(clock, ts);
+}
+
+int clock_gettime(clockid_t, struct old_timespec32 *)
+ __attribute__((weak, alias("__vdso_clock_gettime")));
+
+notrace int __vdso_clock_getres(clockid_t clock,
+ struct old_timespec32 *res)
+{
+ return __cvdso_clock_getres_time32(clock, res);
+}
+int clock_getres(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_getres")));
+#endif
diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
index 1a86fdc1e400..510a0ebd5748 100644
--- a/arch/x86/include/asm/vdso/gettimeofday.h
+++ b/arch/x86/include/asm/vdso/gettimeofday.h
@@ -38,7 +38,7 @@ extern u8 hvclock_page
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback(
}
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
@@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
asm ("syscall" : "=a" (ret), "=m" (*_ts) :
@@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback(
static __always_inline notrace long clock_gettime_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
@@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback(
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
- : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
+ : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
static __always_inline notrace long gettimeofday_fallback(
- struct __vdso_timeval *_tv,
+ struct __kernel_old_timeval *_tv,
struct timezone *_tz)
{
long ret;
@@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback(
static __always_inline notrace long clock_getres_fallback(
clockid_t _clkid,
- struct __vdso_timespec *_ts)
+ struct __kernel_timespec *_ts)
{
long ret;
@@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback(
"call __kernel_vsyscall \n"
"mov %%edx, %%ebx \n"
: "=a" (ret), "=m" (*_ts)
- : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
+ : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts)
: "edx");
return ret;
}
diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
index ff332fcba73c..da346ad02b03 100644
--- a/include/vdso/datapage.h
+++ b/include/vdso/datapage.h
@@ -9,7 +9,6 @@
#include <linux/bits.h>
#include <linux/types.h>
#include <linux/time.h>
-#include <vdso/types.h>
#define VDSO_BASES (CLOCK_TAI + 1)
#define VDSO_HRES (BIT(CLOCK_REALTIME) | \
diff --git a/include/vdso/types.h b/include/vdso/types.h
deleted file mode 100644
index f456a0a6a2e1..000000000000
--- a/include/vdso/types.h
+++ /dev/null
@@ -1,39 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0 */
-#ifndef __VDSO_TYPES_H
-#define __VDSO_TYPES_H
-
-#ifdef __KERNEL__
-
-#ifndef __ASSEMBLY__
-
-#include <linux/types.h>
-#include <linux/time.h>
-
-/*
- * The definitions below are required to overcome the limitations
- * of time_t on 32 bit architectures, which overflows in 2038.
- * The new code should use the replacements based on time64_t and
- * timespec64.
- *
- * The abstraction below will be updated once the migration to
- * time64_t is complete.
- */
-#ifdef CONFIG_GENERIC_VDSO_32
-#define __vdso_timespec old_timespec32
-#define __vdso_timeval old_timeval32
-#else
-#ifdef ENABLE_COMPAT_VDSO
-#define __vdso_timespec old_timespec32
-#define __vdso_timeval old_timeval32
-#else
-#define __vdso_timespec __kernel_timespec
-#define __vdso_timeval __kernel_old_timeval
-#endif /* CONFIG_COMPAT_VDSO */
-#endif /* CONFIG_GENERIC_VDSO_32 */
-
-
-#endif /* !__ASSEMBLY__ */
-
-#endif /* __KERNEL__ */
-
-#endif /* __VDSO_TYPES_H */
diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
index 466d0c68fa80..f10e3703abaa 100644
--- a/lib/vdso/gettimeofday.c
+++ b/lib/vdso/gettimeofday.c
@@ -28,7 +28,7 @@
static notrace int do_hres(const struct vdso_data *vd,
clockid_t clk,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
u64 cycles, last, sec, ns;
@@ -63,7 +63,7 @@ static notrace int do_hres(const struct vdso_data *vd,
static notrace void do_coarse(const struct vdso_data *vd,
clockid_t clk,
- struct __vdso_timespec *ts)
+ struct __kernel_timespec *ts)
{
const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
u32 seq;
@@ -75,8 +75,8 @@ static notrace void do_coarse(const struct vdso_data *vd,
} while (unlikely(vdso_read_retry(vd, seq)));
}
-static notrace int __cvdso_clock_gettime(clockid_t clock,
- struct __vdso_timespec *ts)
+static notrace __maybe_unused int
+__cvdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
{
const struct vdso_data *vd = __arch_get_vdso_data();
u32 msk;
@@ -100,19 +100,31 @@ static notrace int __cvdso_clock_gettime(clockid_t clock,
return clock_gettime_fallback(clock, ts);
}
-static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
- struct timezone *tz)
+static notrace __maybe_unused int
+__cvdso_clock_gettime32(clockid_t clock, struct old_timespec32 *res)
+{
+ struct __kernel_timespec ts;
+ int ret = __cvdso_clock_gettime(clock, &ts);
+
+ res->tv_sec = ts.tv_sec;
+ res->tv_nsec = ts.tv_nsec;
+
+ return ret;
+}
+
+static notrace __maybe_unused int
+__cvdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
{
const struct vdso_data *vd = __arch_get_vdso_data();
if (likely(tv != NULL)) {
- struct __vdso_timespec ts;
+ struct __kernel_timespec ts;
if (do_hres(vd, CLOCK_REALTIME, &ts))
return gettimeofday_fallback(tv, tz);
tv->tv_sec = ts.tv_sec;
- tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
+ tv->tv_usec = (u32)ts.tv_nsec / NSEC_PER_USEC;
}
if (unlikely(tz != NULL)) {
@@ -123,8 +135,7 @@ static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
return 0;
}
-#ifdef VDSO_HAS_TIME
-static notrace time_t __cvdso_time(time_t *time)
+static notrace __maybe_unused time_t __cvdso_time(time_t *time)
{
const struct vdso_data *vd = __arch_get_vdso_data();
time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
@@ -134,10 +145,9 @@ static notrace time_t __cvdso_time(time_t *time)
return t;
}
-#endif /* VDSO_HAS_TIME */
-static notrace int __cvdso_clock_getres(clockid_t clock,
- struct __vdso_timespec *res)
+static notrace __maybe_unused
+int __cvdso_clock_getres(clockid_t clock, struct __kernel_timespec *res)
{
u64 sec, ns;
u32 msk;
@@ -177,3 +187,15 @@ static notrace int __cvdso_clock_getres(clockid_t clock,
fallback:
return clock_getres_fallback(clock, res);
}
+
+static notrace __maybe_unused int
+__cvdso_clock_getres_time32(clockid_t clock, struct old_timespec32 *res)
+{
+ struct __kernel_timespec ts;
+ int ret = __cvdso_clock_getres(clock, &ts);
+
+ res->tv_sec = ts.tv_sec;
+ res->tv_nsec = ts.tv_nsec;
+
+ return ret;
+}
--
2.20.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH 2/2] vdso: add clock_gettime64
@ 2019-02-28 11:58 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Adding a clock_gettime64() vdso call on all32-bit machines is now trivial.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 6 ++++++
arch/arm64/kernel/vdso32/vdso.lds.S | 1 +
arch/mips/vdso/vdso.lds.S | 3 +++
arch/mips/vdso/vgettimeofday.c | 6 ++++++
arch/x86/entry/vdso/vclock_gettime.c | 6 ++++++
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
7 files changed, 24 insertions(+)
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 1d81e8c3acf6..05581140fd12 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -83,6 +83,7 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+ __vdso_clock_gettime64;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index b4607a5ecf31..6762f800a8ea 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -14,6 +14,12 @@ notrace int __vdso_clock_gettime(clockid_t clock,
return __cvdso_clock_gettime32(clock, ts);
}
+notrace int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
index 4fd3093ea78c..a3944927eaeb 100644
--- a/arch/arm64/kernel/vdso32/vdso.lds.S
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -68,6 +68,7 @@ VERSION
__kernel_sigreturn_thumb;
__kernel_rt_sigreturn_arm;
__kernel_rt_sigreturn_thumb;
+ __vdso_clock_gettime64;
local: *;
};
}
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 90b2c2785e32..659fe0c3750a 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -100,6 +100,9 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+#if _MIPS_SIM != _MIPS_SIM_ABI64
+ __vdso_clock_gettime64;
+#endif
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
index 7a3320faa29a..0b4fea088f89 100644
--- a/arch/mips/vdso/vgettimeofday.c
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -27,6 +27,12 @@ notrace int __vdso_clock_getres(clockid_t clock_id,
return __cvdso_clock_getres_time32(clock_id, res);
}
+notrace int __vdso_clock_gettime_time64(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
#else
notrace int __vdso_clock_gettime(clockid_t clock,
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index c9512c114ac1..0a7d4bdeadc9 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -54,6 +54,7 @@ int clock_getres(clockid_t, struct __kernel_timespec *)
#else
/* i386 only */
extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
+extern int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts);
extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
@@ -64,6 +65,11 @@ notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
int clock_gettime(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_gettime")));
+notrace int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
notrace int __vdso_clock_getres(clockid_t clock,
struct old_timespec32 *res)
{
diff --git a/arch/x86/entry/vdso/vdso32/vdso32.lds.S b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
index 991b26cc855b..c7720995ab1a 100644
--- a/arch/x86/entry/vdso/vdso32/vdso32.lds.S
+++ b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
@@ -27,6 +27,7 @@ VERSION
__vdso_gettimeofday;
__vdso_time;
__vdso_clock_getres;
+ __vdso_clock_gettime64;
};
LINUX_2.5 {
--
2.20.0
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH 2/2] vdso: add clock_gettime64
@ 2019-02-28 11:58 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Arnd Bergmann, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Thomas Gleixner, Mark Salyzyn,
Peter Collingbourne, Shuah Khan, Dmitry Safonov,
Rasmus Villemoes
Adding a clock_gettime64() vdso call on all32-bit machines is now trivial.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 6 ++++++
arch/arm64/kernel/vdso32/vdso.lds.S | 1 +
arch/mips/vdso/vdso.lds.S | 3 +++
arch/mips/vdso/vgettimeofday.c | 6 ++++++
arch/x86/entry/vdso/vclock_gettime.c | 6 ++++++
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
7 files changed, 24 insertions(+)
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 1d81e8c3acf6..05581140fd12 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -83,6 +83,7 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+ __vdso_clock_gettime64;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index b4607a5ecf31..6762f800a8ea 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -14,6 +14,12 @@ notrace int __vdso_clock_gettime(clockid_t clock,
return __cvdso_clock_gettime32(clock, ts);
}
+notrace int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
index 4fd3093ea78c..a3944927eaeb 100644
--- a/arch/arm64/kernel/vdso32/vdso.lds.S
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -68,6 +68,7 @@ VERSION
__kernel_sigreturn_thumb;
__kernel_rt_sigreturn_arm;
__kernel_rt_sigreturn_thumb;
+ __vdso_clock_gettime64;
local: *;
};
}
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 90b2c2785e32..659fe0c3750a 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -100,6 +100,9 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+#if _MIPS_SIM != _MIPS_SIM_ABI64
+ __vdso_clock_gettime64;
+#endif
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
index 7a3320faa29a..0b4fea088f89 100644
--- a/arch/mips/vdso/vgettimeofday.c
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -27,6 +27,12 @@ notrace int __vdso_clock_getres(clockid_t clock_id,
return __cvdso_clock_getres_time32(clock_id, res);
}
+notrace int __vdso_clock_gettime_time64(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
#else
notrace int __vdso_clock_gettime(clockid_t clock,
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index c9512c114ac1..0a7d4bdeadc9 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -54,6 +54,7 @@ int clock_getres(clockid_t, struct __kernel_timespec *)
#else
/* i386 only */
extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
+extern int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts);
extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
@@ -64,6 +65,11 @@ notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
int clock_gettime(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_gettime")));
+notrace int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
notrace int __vdso_clock_getres(clockid_t clock,
struct old_timespec32 *res)
{
diff --git a/arch/x86/entry/vdso/vdso32/vdso32.lds.S b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
index 991b26cc855b..c7720995ab1a 100644
--- a/arch/x86/entry/vdso/vdso32/vdso32.lds.S
+++ b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
@@ -27,6 +27,7 @@ VERSION
__vdso_gettimeofday;
__vdso_time;
__vdso_clock_getres;
+ __vdso_clock_gettime64;
};
LINUX_2.5 {
--
2.20.0
^ permalink raw reply related [flat|nested] 178+ messages in thread
* [PATCH 2/2] vdso: add clock_gettime64
@ 2019-02-28 11:58 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 11:58 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Arnd Bergmann, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Rasmus Villemoes,
Thomas Gleixner, Peter Collingbourne, linux-arm-kernel
Adding a clock_gettime64() vdso call on all32-bit machines is now trivial.
Signed-off-by: Arnd Bergmann <arnd@arndb.de>
---
arch/arm/vdso/vdso.lds.S | 1 +
arch/arm/vdso/vgettimeofday.c | 6 ++++++
arch/arm64/kernel/vdso32/vdso.lds.S | 1 +
arch/mips/vdso/vdso.lds.S | 3 +++
arch/mips/vdso/vgettimeofday.c | 6 ++++++
arch/x86/entry/vdso/vclock_gettime.c | 6 ++++++
arch/x86/entry/vdso/vdso32/vdso32.lds.S | 1 +
7 files changed, 24 insertions(+)
diff --git a/arch/arm/vdso/vdso.lds.S b/arch/arm/vdso/vdso.lds.S
index 1d81e8c3acf6..05581140fd12 100644
--- a/arch/arm/vdso/vdso.lds.S
+++ b/arch/arm/vdso/vdso.lds.S
@@ -83,6 +83,7 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+ __vdso_clock_gettime64;
local: *;
};
}
diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
index b4607a5ecf31..6762f800a8ea 100644
--- a/arch/arm/vdso/vgettimeofday.c
+++ b/arch/arm/vdso/vgettimeofday.c
@@ -14,6 +14,12 @@ notrace int __vdso_clock_gettime(clockid_t clock,
return __cvdso_clock_gettime32(clock, ts);
}
+notrace int __vdso_clock_gettime64(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
struct timezone *tz)
{
diff --git a/arch/arm64/kernel/vdso32/vdso.lds.S b/arch/arm64/kernel/vdso32/vdso.lds.S
index 4fd3093ea78c..a3944927eaeb 100644
--- a/arch/arm64/kernel/vdso32/vdso.lds.S
+++ b/arch/arm64/kernel/vdso32/vdso.lds.S
@@ -68,6 +68,7 @@ VERSION
__kernel_sigreturn_thumb;
__kernel_rt_sigreturn_arm;
__kernel_rt_sigreturn_thumb;
+ __vdso_clock_gettime64;
local: *;
};
}
diff --git a/arch/mips/vdso/vdso.lds.S b/arch/mips/vdso/vdso.lds.S
index 90b2c2785e32..659fe0c3750a 100644
--- a/arch/mips/vdso/vdso.lds.S
+++ b/arch/mips/vdso/vdso.lds.S
@@ -100,6 +100,9 @@ VERSION
__vdso_clock_gettime;
__vdso_gettimeofday;
__vdso_clock_getres;
+#if _MIPS_SIM != _MIPS_SIM_ABI64
+ __vdso_clock_gettime64;
+#endif
#endif
local: *;
};
diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
index 7a3320faa29a..0b4fea088f89 100644
--- a/arch/mips/vdso/vgettimeofday.c
+++ b/arch/mips/vdso/vgettimeofday.c
@@ -27,6 +27,12 @@ notrace int __vdso_clock_getres(clockid_t clock_id,
return __cvdso_clock_getres_time32(clock_id, res);
}
+notrace int __vdso_clock_gettime_time64(clockid_t clock,
+ struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
#else
notrace int __vdso_clock_gettime(clockid_t clock,
diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
index c9512c114ac1..0a7d4bdeadc9 100644
--- a/arch/x86/entry/vdso/vclock_gettime.c
+++ b/arch/x86/entry/vdso/vclock_gettime.c
@@ -54,6 +54,7 @@ int clock_getres(clockid_t, struct __kernel_timespec *)
#else
/* i386 only */
extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
+extern int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts);
extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
@@ -64,6 +65,11 @@ notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
int clock_gettime(clockid_t, struct old_timespec32 *)
__attribute__((weak, alias("__vdso_clock_gettime")));
+notrace int __vdso_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts)
+{
+ return __cvdso_clock_gettime(clock, ts);
+}
+
notrace int __vdso_clock_getres(clockid_t clock,
struct old_timespec32 *res)
{
diff --git a/arch/x86/entry/vdso/vdso32/vdso32.lds.S b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
index 991b26cc855b..c7720995ab1a 100644
--- a/arch/x86/entry/vdso/vdso32/vdso32.lds.S
+++ b/arch/x86/entry/vdso/vdso32/vdso32.lds.S
@@ -27,6 +27,7 @@ VERSION
__vdso_gettimeofday;
__vdso_time;
__vdso_clock_getres;
+ __vdso_clock_gettime64;
};
LINUX_2.5 {
--
2.20.0
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:09 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 12:09 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
Hi Arnd,
thank you for testing my code.
On 28/02/2019 11:40, Arnd Bergmann wrote:
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> vDSO (virtual dynamic shared object) is a mechanism that the Linux
>> kernel provides as an alternative to system calls to reduce where
>> possible the costs in terms of cycles.
>> This is possible because certain syscalls like gettimeofday() do
>> not write any data and return one or more values that are stored
>> in the kernel, which makes relatively safe calling them directly
>> as a library function.
>
> I tried applying the series on top of v5.0-rc7, and got a set of
> build warnings with arm64 defconfig:
>
> In file included from /git/arm-soc/arch/arm64/include/asm/thread_info.h:30:0,
> from /git/arm-soc/include/linux/thread_info.h:38,
> from /git/arm-soc/arch/arm64/include/asm/preempt.h:5,
> from /git/arm-soc/include/linux/preempt.h:78,
> from /git/arm-soc/include/linux/spinlock.h:51,
> from /git/arm-soc/include/linux/seqlock.h:36,
> from /git/arm-soc/include/linux/time.h:6,
> from /git/arm-soc/lib/vdso/gettimeofday.c:7,
> from <command-line>:0:
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
> shift count >= width of type [enabled by default]
> return kimage_vaddr - KIMAGE_VADDR;
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> return __virt_to_phys((unsigned long)(x));
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
> shift count >= width of type [enabled by default]
> return (void *)(__phys_to_virt(x));
> ^
> In file included from /git/arm-soc/arch/arm64/include/asm/pgtable-hwdef.h:19:0,
> from /git/arm-soc/arch/arm64/include/asm/processor.h:46,
> from /git/arm-soc/arch/arm64/include/asm/elf.h:129,
> from /git/arm-soc/include/linux/elf.h:5,
> from /git/arm-soc/include/linux/elfnote.h:62,
> from /git/arm-soc/arch/arm64/kernel/vdso32/note.c:11:
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
> shift count >= width of type [enabled by default]
> return kimage_vaddr - KIMAGE_VADDR;
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> return __virt_to_phys((unsigned long)(x));
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
> shift count >= width of type [enabled by default]
> return (void *)(__phys_to_virt(x));
> ^
>
> I think this is all harmless, but we need to limit the set of header files
> that can be included indirectly when compiling the vdso in arm32 mode.
>
I agree with you on the principle, but I am unable to replicate the warnings.
Did you make any customization to the default defconfig? What is the top
commit-id you have on your tree?
> Arnd
>
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:09 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 12:09 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
Hi Arnd,
thank you for testing my code.
On 28/02/2019 11:40, Arnd Bergmann wrote:
> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> vDSO (virtual dynamic shared object) is a mechanism that the Linux
>> kernel provides as an alternative to system calls to reduce where
>> possible the costs in terms of cycles.
>> This is possible because certain syscalls like gettimeofday() do
>> not write any data and return one or more values that are stored
>> in the kernel, which makes relatively safe calling them directly
>> as a library function.
>
> I tried applying the series on top of v5.0-rc7, and got a set of
> build warnings with arm64 defconfig:
>
> In file included from /git/arm-soc/arch/arm64/include/asm/thread_info.h:30:0,
> from /git/arm-soc/include/linux/thread_info.h:38,
> from /git/arm-soc/arch/arm64/include/asm/preempt.h:5,
> from /git/arm-soc/include/linux/preempt.h:78,
> from /git/arm-soc/include/linux/spinlock.h:51,
> from /git/arm-soc/include/linux/seqlock.h:36,
> from /git/arm-soc/include/linux/time.h:6,
> from /git/arm-soc/lib/vdso/gettimeofday.c:7,
> from <command-line>:0:
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
> shift count >= width of type [enabled by default]
> return kimage_vaddr - KIMAGE_VADDR;
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> return __virt_to_phys((unsigned long)(x));
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
> shift count >= width of type [enabled by default]
> return (void *)(__phys_to_virt(x));
> ^
> In file included from /git/arm-soc/arch/arm64/include/asm/pgtable-hwdef.h:19:0,
> from /git/arm-soc/arch/arm64/include/asm/processor.h:46,
> from /git/arm-soc/arch/arm64/include/asm/elf.h:129,
> from /git/arm-soc/include/linux/elf.h:5,
> from /git/arm-soc/include/linux/elfnote.h:62,
> from /git/arm-soc/arch/arm64/kernel/vdso32/note.c:11:
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘kaslr_offset’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:191:2: warning: left
> shift count >= width of type [enabled by default]
> return kimage_vaddr - KIMAGE_VADDR;
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘virt_to_phys’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> return __virt_to_phys((unsigned long)(x));
> ^
> /git/arm-soc/arch/arm64/include/asm/memory.h:283:2: warning: left
> shift count >= width of type [enabled by default]
> /git/arm-soc/arch/arm64/include/asm/memory.h: In function ‘phys_to_virt’:
> /git/arm-soc/arch/arm64/include/asm/memory.h:289:2: warning: left
> shift count >= width of type [enabled by default]
> return (void *)(__phys_to_virt(x));
> ^
>
> I think this is all harmless, but we need to limit the set of header files
> that can be included indirectly when compiling the vdso in arm32 mode.
>
I agree with you on the principle, but I am unable to replicate the warnings.
Did you make any customization to the default defconfig? What is the top
commit-id you have on your tree?
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:38 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 12:38 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> thank you for testing my code.
>
> On 28/02/2019 11:40, Arnd Bergmann wrote:
> > On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> > ^
> >
> > I think this is all harmless, but we need to limit the set of header files
> > that can be included indirectly when compiling the vdso in arm32 mode.
> >
>
> I agree with you on the principle, but I am unable to replicate the warnings.
> Did you make any customization to the default defconfig? What is the top
> commit-id you have on your tree?
I observed this on top of v5.0-rc7 earlier, and have since rebased to
today's linux-next, where it still happens with the regular 'make defconfig',
on all compiler versions I have (4.8 through 8.1).
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:38 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 12:38 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> thank you for testing my code.
>
> On 28/02/2019 11:40, Arnd Bergmann wrote:
> > On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> > ^
> >
> > I think this is all harmless, but we need to limit the set of header files
> > that can be included indirectly when compiling the vdso in arm32 mode.
> >
>
> I agree with you on the principle, but I am unable to replicate the warnings.
> Did you make any customization to the default defconfig? What is the top
> commit-id you have on your tree?
I observed this on top of v5.0-rc7 earlier, and have since rebased to
today's linux-next, where it still happens with the regular 'make defconfig',
on all compiler versions I have (4.8 through 8.1).
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:38 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 12:38 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> thank you for testing my code.
>
> On 28/02/2019 11:40, Arnd Bergmann wrote:
> > On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> > ^
> >
> > I think this is all harmless, but we need to limit the set of header files
> > that can be included indirectly when compiling the vdso in arm32 mode.
> >
>
> I agree with you on the principle, but I am unable to replicate the warnings.
> Did you make any customization to the default defconfig? What is the top
> commit-id you have on your tree?
I observed this on top of v5.0-rc7 earlier, and have since rebased to
today's linux-next, where it still happens with the regular 'make defconfig',
on all compiler versions I have (4.8 through 8.1).
Arnd
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:42 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 12:42 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On 28/02/2019 12:38, Arnd Bergmann wrote:
> On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> thank you for testing my code.
>>
>> On 28/02/2019 11:40, Arnd Bergmann wrote:
>>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
>>> <vincenzo.frascino@arm.com> wrote:
>>> ^
>>>
>>> I think this is all harmless, but we need to limit the set of header files
>>> that can be included indirectly when compiling the vdso in arm32 mode.
>>>
>>
>> I agree with you on the principle, but I am unable to replicate the warnings.
>> Did you make any customization to the default defconfig? What is the top
>> commit-id you have on your tree?
>
> I observed this on top of v5.0-rc7 earlier, and have since rebased to
> today's linux-next, where it still happens with the regular 'make defconfig',
> on all compiler versions I have (4.8 through 8.1).
>
Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:42 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 12:42 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On 28/02/2019 12:38, Arnd Bergmann wrote:
> On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> thank you for testing my code.
>>
>> On 28/02/2019 11:40, Arnd Bergmann wrote:
>>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
>>> <vincenzo.frascino@arm.com> wrote:
>>> ^
>>>
>>> I think this is all harmless, but we need to limit the set of header files
>>> that can be included indirectly when compiling the vdso in arm32 mode.
>>>
>>
>> I agree with you on the principle, but I am unable to replicate the warnings.
>> Did you make any customization to the default defconfig? What is the top
>> commit-id you have on your tree?
>
> I observed this on top of v5.0-rc7 earlier, and have since rebased to
> today's linux-next, where it still happens with the regular 'make defconfig',
> on all compiler versions I have (4.8 through 8.1).
>
Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 12:42 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 12:42 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On 28/02/2019 12:38, Arnd Bergmann wrote:
> On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> thank you for testing my code.
>>
>> On 28/02/2019 11:40, Arnd Bergmann wrote:
>>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
>>> <vincenzo.frascino@arm.com> wrote:
>>> ^
>>>
>>> I think this is all harmless, but we need to limit the set of header files
>>> that can be included indirectly when compiling the vdso in arm32 mode.
>>>
>>
>> I agree with you on the principle, but I am unable to replicate the warnings.
>> Did you make any customization to the default defconfig? What is the top
>> commit-id you have on your tree?
>
> I observed this on top of v5.0-rc7 earlier, and have since rebased to
> today's linux-next, where it still happens with the regular 'make defconfig',
> on all compiler versions I have (4.8 through 8.1).
>
Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
> Arnd
>
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 13:42 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-28 13:42 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Vincenzo Frascino,
Peter Collingbourne, linux-arm-kernel
On Thu, 28 Feb 2019, Arnd Bergmann wrote:
> Hi Vincento,
>
> I've implemented now what I tried to explain earlier. This moves the
> implementation of clock_gettime() and clock_getres() over to 64-bit
> __kernel_timespec, with a conversion in the end for architectures that
> actually use the 32-bit calls.
>
> The vdso/types.h file is now obsolete, but we do need to duplicate
> a few functions on mips32 and x86-32. It may be possible to fold those
> into the generic lib/vdso/gettimeofday.c file as another step.
>
> If there are no objections, please fold these changes into your
> series.
Please write the new generic code in that way and apply the architecture
changes when converting them over.
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 13:42 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-28 13:42 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Vincenzo Frascino, linux-arch, linux-arm-kernel, Catalin Marinas,
Will Deacon, Russell King, Ralf Baechle, Paul Burton,
Daniel Lezcano, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Thu, 28 Feb 2019, Arnd Bergmann wrote:
> Hi Vincento,
>
> I've implemented now what I tried to explain earlier. This moves the
> implementation of clock_gettime() and clock_getres() over to 64-bit
> __kernel_timespec, with a conversion in the end for architectures that
> actually use the 32-bit calls.
>
> The vdso/types.h file is now obsolete, but we do need to duplicate
> a few functions on mips32 and x86-32. It may be possible to fold those
> into the generic lib/vdso/gettimeofday.c file as another step.
>
> If there are no objections, please fold these changes into your
> series.
Please write the new generic code in that way and apply the architecture
changes when converting them over.
Thanks,
tglx
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 13:42 ` Thomas Gleixner
0 siblings, 0 replies; 178+ messages in thread
From: Thomas Gleixner @ 2019-02-28 13:42 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Vincenzo Frascino,
Peter Collingbourne, linux-arm-kernel
On Thu, 28 Feb 2019, Arnd Bergmann wrote:
> Hi Vincento,
>
> I've implemented now what I tried to explain earlier. This moves the
> implementation of clock_gettime() and clock_getres() over to 64-bit
> __kernel_timespec, with a conversion in the end for architectures that
> actually use the 32-bit calls.
>
> The vdso/types.h file is now obsolete, but we do need to duplicate
> a few functions on mips32 and x86-32. It may be possible to fold those
> into the generic lib/vdso/gettimeofday.c file as another step.
>
> If there are no objections, please fold these changes into your
> series.
Please write the new generic code in that way and apply the architecture
changes when converting them over.
Thanks,
tglx
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 13:45 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 13:45 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, linux-arm-kernel
Hi Arnd,
thank you for providing the changes.
On 28/02/2019 11:58, Arnd Bergmann wrote:
> Hi Vincento,
>
> I've implemented now what I tried to explain earlier. This moves the
> implementation of clock_gettime() and clock_getres() over to 64-bit
> __kernel_timespec, with a conversion in the end for architectures that
> actually use the 32-bit calls.
>
> The vdso/types.h file is now obsolete, but we do need to duplicate
> a few functions on mips32 and x86-32. It may be possible to fold those
> into the generic lib/vdso/gettimeofday.c file as another step.
>
> If there are no objections, please fold these changes into your
> series.
>
No objections on my side, I will fold the changes in my set.
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> arch/arm/include/asm/vdso/gettimeofday.h | 16 +++---
> arch/arm/vdso/vgettimeofday.c | 10 ++--
> .../include/asm/vdso/compat_gettimeofday.h | 16 +++---
> arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++--
> arch/arm64/kernel/vdso/vgettimeofday.c | 6 +-
> arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++--
> arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++---
> arch/mips/vdso/vgettimeofday.c | 28 ++++++++-
> arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------
> arch/x86/include/asm/vdso/gettimeofday.h | 16 +++---
> include/vdso/datapage.h | 1 -
> include/vdso/types.h | 39 -------------
> lib/vdso/gettimeofday.c | 48 +++++++++++-----
> 13 files changed, 164 insertions(+), 121 deletions(-)
> delete mode 100644 include/vdso/types.h
>
> diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
> index e7aeda0b255f..112df50920b0 100644
> --- a/arch/arm/include/asm/vdso/gettimeofday.h
> +++ b/arch/arm/include/asm/vdso/gettimeofday.h
> @@ -14,11 +14,11 @@
> extern struct vdso_data *__get_datapage(void);
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("r1") = _tz;
> - register struct __vdso_timeval *tv asm("r0") = _tv;
> + register struct __kernel_old_timeval *tv asm("r0") = _tv;
> register long ret asm ("r0");
> register long nr asm("r7") = __NR_gettimeofday;
>
> @@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_clock_gettime;
> + register long nr asm("r7") = __NR_clock_gettime64;
>
> asm volatile(
> " swi #0\n"
> @@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_clock_getres;
> + register long nr asm("r7") = __NR_clock_getres_time64;
>
> asm volatile(
> " swi #0\n"
> diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
> index b470a8ed35a7..b4607a5ecf31 100644
> --- a/arch/arm/vdso/vgettimeofday.c
> +++ b/arch/arm/vdso/vgettimeofday.c
> @@ -9,21 +9,21 @@
> #include <linux/types.h>
>
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> {
> - return __cvdso_clock_gettime(clock, ts);
> + return __cvdso_clock_gettime32(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct old_timespec32 *res)
> {
> - return __cvdso_clock_getres(clock_id, res);
> + return __cvdso_clock_getres_time32(clock_id, res);
> }
>
> /* Avoid unresolved references emitted by GCC */
> diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> index 888c70c24f71..b41d8f01a20c 100644
> --- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> +++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> @@ -24,11 +24,11 @@
> extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("r1") = _tz;
> - register struct __vdso_timeval *tv asm("r0") = _tv;
> + register struct __kernel_old_timeval *tv asm("r0") = _tv;
> register long ret asm ("r0");
> register long nr asm("r7") = __NR_compat_gettimeofday;
>
> @@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_compat_clock_gettime;
> + register long nr asm("r7") = __NR_compat_clock_gettime64;
>
> asm volatile(
> " swi #0\n"
> @@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_compat_clock_getres;
> + register long nr asm("r7") = __NR_compat_clock_getres_time64;
>
> /* The checks below are required for ABI consistency with arm */
> if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
> diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
> index 0aff6362533f..126fc6f4f7cf 100644
> --- a/arch/arm64/include/asm/vdso/gettimeofday.h
> +++ b/arch/arm64/include/asm/vdso/gettimeofday.h
> @@ -13,11 +13,11 @@
> extern struct vdso_data _vdso_data;
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("x1") = _tz;
> - register struct __vdso_timeval *tv asm("x0") = _tv;
> + register struct __kernel_old_timeval *tv asm("x0") = _tv;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_gettimeofday;
>
> @@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("x1") = _ts;
> + register struct __kernel_timespec *ts asm("x1") = _ts;
> register clockid_t clkid asm("x0") = _clkid;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_clock_gettime;
> @@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("x1") = _ts;
> + register struct __kernel_timespec *ts asm("x1") = _ts;
> register clockid_t clkid asm("x0") = _clkid;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_clock_getres;
> diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
> index 160e68196339..bbc83b6e2b1a 100644
> --- a/arch/arm64/kernel/vdso/vgettimeofday.c
> +++ b/arch/arm64/kernel/vdso/vgettimeofday.c
> @@ -9,19 +9,19 @@
> #include <linux/types.h>
>
> notrace int __kernel_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __kernel_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock_id, res);
> }
> diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
> index b470a8ed35a7..6762f800a8ea 100644
> --- a/arch/arm64/kernel/vdso32/vgettimeofday.c
> +++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
> @@ -9,21 +9,27 @@
> #include <linux/types.h>
>
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +notrace int __vdso_clock_gettime64(clockid_t clock,
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct old_timespec32 *res)
> {
> - return __cvdso_clock_getres(clock_id, res);
> + return __cvdso_clock_getres_time32(clock_id, res);
> }
>
> /* Avoid unresolved references emitted by GCC */
> diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
> index b275392f6b1b..ed6dafda5fc2 100644
> --- a/arch/mips/include/asm/vdso/gettimeofday.h
> +++ b/arch/mips/include/asm/vdso/gettimeofday.h
> @@ -25,11 +25,11 @@
> #ifdef CONFIG_MIPS_CLOCK_VSYSCALL
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("a1") = _tz;
> - register struct __vdso_timeval *tv asm("a0") = _tv;
> + register struct __kernel_old_timeval *tv asm("a0") = _tv;
> register long ret asm("v0");
> register long nr asm("v0") = __NR_gettimeofday;
> register long error asm("a3");
> @@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(
> #else
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> return -1;
> @@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("a1") = _ts;
> + register struct __kernel_timespec *ts asm("a1") = _ts;
> register clockid_t clkid asm("a0") = _clkid;
> register long ret asm("v0");
> +#if _MIPS_SIM == _MIPS_SIM_ABI64
> register long nr asm("v0") = __NR_clock_gettime;
> +#else
> + register long nr asm("v0") = __NR_clock_gettime64
> +#endif
> register long error asm("a3");
>
> asm volatile(
> @@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("a1") = _ts;
> + register struct __kernel_timespec *ts asm("a1") = _ts;
> register clockid_t clkid asm("a0") = _clkid;
> register long ret asm("v0");
> +#if _MIPS_SIM == _MIPS_SIM_ABI64
> register long nr asm("v0") = __NR_clock_getres;
> +#else
> + register long nr asm("v0") = __NR_clock_getres64
> +#endif
> register long error asm("a3");
>
> asm volatile(
> diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
> index 8da361d919f0..7a3320faa29a 100644
> --- a/arch/mips/vdso/vgettimeofday.c
> +++ b/arch/mips/vdso/vgettimeofday.c
> @@ -8,21 +8,43 @@
> #include <linux/time.h>
> #include <linux/types.h>
>
> +#if _MIPS_SIM != _MIPS_SIM_ABI64
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> + struct timezone *tz)
> +{
> + return __cvdso_gettimeofday_time32(tv, tz);
> +}
> +
> +notrace int __vdso_clock_getres(clockid_t clock_id,
> + struct old_timespec32 *res)
> +{
> + return __cvdso_clock_getres_time32(clock_id, res);
> +}
> +
> +#else
> +
> +notrace int __vdso_clock_gettime(clockid_t clock,
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock_id, res);
> }
>
> +#endif
> diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
> index 6ff8cabf4c3e..c9512c114ac1 100644
> --- a/arch/x86/entry/vdso/vclock_gettime.c
> +++ b/arch/x86/entry/vdso/vclock_gettime.c
> @@ -12,25 +12,14 @@
>
> #include "../../../../lib/vdso/gettimeofday.c"
>
> -extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
> -extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
> +extern int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
> extern time_t __vdso_time(time_t *t);
> -extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
> -
> -notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
> -{
> - return __cvdso_clock_gettime(clock, ts);
> -}
> -
> -int clock_gettime(clockid_t, struct __vdso_timespec *)
> - __attribute__((weak, alias("__vdso_clock_gettime")));
> -
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
> -int gettimeofday(struct __vdso_timeval *, struct timezone *)
> +int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
> __attribute__((weak, alias("__vdso_gettimeofday")));
>
> notrace time_t __vdso_time(time_t *t)
> @@ -40,10 +29,46 @@ notrace time_t __vdso_time(time_t *t)
> time_t time(time_t *t)
> __attribute__((weak, alias("__vdso_time")));
>
> +
> +#ifdef __x86_64__
> +/* both 64-bit and x32 use these */
> +extern int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
> +extern int __vdso_clock_getres(clockid_t clock, struct __kernel_timespec *res);
> +
> +notrace int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
> +{
> + return __cvdso_clock_gettime(clock, ts);
> +}
> +
> +int clock_gettime(clockid_t, struct __kernel_timespec *)
> + __attribute__((weak, alias("__vdso_clock_gettime")));
> +
> notrace int __vdso_clock_getres(clockid_t clock,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock, res);
> }
> -int clock_getres(clockid_t, struct __vdso_timespec *)
> +int clock_getres(clockid_t, struct __kernel_timespec *)
> + __attribute__((weak, alias("__vdso_clock_getres")));
> +
> +#else
> +/* i386 only */
> +extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
> +extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
> +
> +notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +int clock_gettime(clockid_t, struct old_timespec32 *)
> + __attribute__((weak, alias("__vdso_clock_gettime")));
> +
> +notrace int __vdso_clock_getres(clockid_t clock,
> + struct old_timespec32 *res)
> +{
> + return __cvdso_clock_getres_time32(clock, res);
> +}
> +int clock_getres(clockid_t, struct old_timespec32 *)
> __attribute__((weak, alias("__vdso_clock_getres")));
> +#endif
> diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
> index 1a86fdc1e400..510a0ebd5748 100644
> --- a/arch/x86/include/asm/vdso/gettimeofday.h
> +++ b/arch/x86/include/asm/vdso/gettimeofday.h
> @@ -38,7 +38,7 @@ extern u8 hvclock_page
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
> asm ("syscall" : "=a" (ret), "=m" (*_ts) :
> @@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback(
> }
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> long ret;
> @@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
> asm ("syscall" : "=a" (ret), "=m" (*_ts) :
> @@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
>
> @@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback(
> "call __kernel_vsyscall \n"
> "mov %%edx, %%ebx \n"
> : "=a" (ret), "=m" (*_ts)
> - : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
> + : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts)
> : "edx");
> return ret;
> }
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> long ret;
> @@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
>
> @@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback(
> "call __kernel_vsyscall \n"
> "mov %%edx, %%ebx \n"
> : "=a" (ret), "=m" (*_ts)
> - : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
> + : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts)
> : "edx");
> return ret;
> }
> diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
> index ff332fcba73c..da346ad02b03 100644
> --- a/include/vdso/datapage.h
> +++ b/include/vdso/datapage.h
> @@ -9,7 +9,6 @@
> #include <linux/bits.h>
> #include <linux/types.h>
> #include <linux/time.h>
> -#include <vdso/types.h>
>
> #define VDSO_BASES (CLOCK_TAI + 1)
> #define VDSO_HRES (BIT(CLOCK_REALTIME) | \
> diff --git a/include/vdso/types.h b/include/vdso/types.h
> deleted file mode 100644
> index f456a0a6a2e1..000000000000
> --- a/include/vdso/types.h
> +++ /dev/null
> @@ -1,39 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef __VDSO_TYPES_H
> -#define __VDSO_TYPES_H
> -
> -#ifdef __KERNEL__
> -
> -#ifndef __ASSEMBLY__
> -
> -#include <linux/types.h>
> -#include <linux/time.h>
> -
> -/*
> - * The definitions below are required to overcome the limitations
> - * of time_t on 32 bit architectures, which overflows in 2038.
> - * The new code should use the replacements based on time64_t and
> - * timespec64.
> - *
> - * The abstraction below will be updated once the migration to
> - * time64_t is complete.
> - */
> -#ifdef CONFIG_GENERIC_VDSO_32
> -#define __vdso_timespec old_timespec32
> -#define __vdso_timeval old_timeval32
> -#else
> -#ifdef ENABLE_COMPAT_VDSO
> -#define __vdso_timespec old_timespec32
> -#define __vdso_timeval old_timeval32
> -#else
> -#define __vdso_timespec __kernel_timespec
> -#define __vdso_timeval __kernel_old_timeval
> -#endif /* CONFIG_COMPAT_VDSO */
> -#endif /* CONFIG_GENERIC_VDSO_32 */
> -
> -
> -#endif /* !__ASSEMBLY__ */
> -
> -#endif /* __KERNEL__ */
> -
> -#endif /* __VDSO_TYPES_H */
> diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
> index 466d0c68fa80..f10e3703abaa 100644
> --- a/lib/vdso/gettimeofday.c
> +++ b/lib/vdso/gettimeofday.c
> @@ -28,7 +28,7 @@
>
> static notrace int do_hres(const struct vdso_data *vd,
> clockid_t clk,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> u64 cycles, last, sec, ns;
> @@ -63,7 +63,7 @@ static notrace int do_hres(const struct vdso_data *vd,
>
> static notrace void do_coarse(const struct vdso_data *vd,
> clockid_t clk,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> u32 seq;
> @@ -75,8 +75,8 @@ static notrace void do_coarse(const struct vdso_data *vd,
> } while (unlikely(vdso_read_retry(vd, seq)));
> }
>
> -static notrace int __cvdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> +static notrace __maybe_unused int
> +__cvdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
> u32 msk;
> @@ -100,19 +100,31 @@ static notrace int __cvdso_clock_gettime(clockid_t clock,
> return clock_gettime_fallback(clock, ts);
> }
>
> -static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
> - struct timezone *tz)
> +static notrace __maybe_unused int
> +__cvdso_clock_gettime32(clockid_t clock, struct old_timespec32 *res)
> +{
> + struct __kernel_timespec ts;
> + int ret = __cvdso_clock_gettime(clock, &ts);
> +
> + res->tv_sec = ts.tv_sec;
> + res->tv_nsec = ts.tv_nsec;
> +
> + return ret;
> +}
> +
> +static notrace __maybe_unused int
> +__cvdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
>
> if (likely(tv != NULL)) {
> - struct __vdso_timespec ts;
> + struct __kernel_timespec ts;
>
> if (do_hres(vd, CLOCK_REALTIME, &ts))
> return gettimeofday_fallback(tv, tz);
>
> tv->tv_sec = ts.tv_sec;
> - tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
> + tv->tv_usec = (u32)ts.tv_nsec / NSEC_PER_USEC;
> }
>
> if (unlikely(tz != NULL)) {
> @@ -123,8 +135,7 @@ static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
> return 0;
> }
>
> -#ifdef VDSO_HAS_TIME
> -static notrace time_t __cvdso_time(time_t *time)
> +static notrace __maybe_unused time_t __cvdso_time(time_t *time)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
> time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
> @@ -134,10 +145,9 @@ static notrace time_t __cvdso_time(time_t *time)
>
> return t;
> }
> -#endif /* VDSO_HAS_TIME */
>
> -static notrace int __cvdso_clock_getres(clockid_t clock,
> - struct __vdso_timespec *res)
> +static notrace __maybe_unused
> +int __cvdso_clock_getres(clockid_t clock, struct __kernel_timespec *res)
> {
> u64 sec, ns;
> u32 msk;
> @@ -177,3 +187,15 @@ static notrace int __cvdso_clock_getres(clockid_t clock,
> fallback:
> return clock_getres_fallback(clock, res);
> }
> +
> +static notrace __maybe_unused int
> +__cvdso_clock_getres_time32(clockid_t clock, struct old_timespec32 *res)
> +{
> + struct __kernel_timespec ts;
> + int ret = __cvdso_clock_getres(clock, &ts);
> +
> + res->tv_sec = ts.tv_sec;
> + res->tv_nsec = ts.tv_nsec;
> +
> + return ret;
> +}
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 13:45 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 13:45 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, linux-arm-kernel, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
Hi Arnd,
thank you for providing the changes.
On 28/02/2019 11:58, Arnd Bergmann wrote:
> Hi Vincento,
>
> I've implemented now what I tried to explain earlier. This moves the
> implementation of clock_gettime() and clock_getres() over to 64-bit
> __kernel_timespec, with a conversion in the end for architectures that
> actually use the 32-bit calls.
>
> The vdso/types.h file is now obsolete, but we do need to duplicate
> a few functions on mips32 and x86-32. It may be possible to fold those
> into the generic lib/vdso/gettimeofday.c file as another step.
>
> If there are no objections, please fold these changes into your
> series.
>
No objections on my side, I will fold the changes in my set.
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> arch/arm/include/asm/vdso/gettimeofday.h | 16 +++---
> arch/arm/vdso/vgettimeofday.c | 10 ++--
> .../include/asm/vdso/compat_gettimeofday.h | 16 +++---
> arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++--
> arch/arm64/kernel/vdso/vgettimeofday.c | 6 +-
> arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++--
> arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++---
> arch/mips/vdso/vgettimeofday.c | 28 ++++++++-
> arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------
> arch/x86/include/asm/vdso/gettimeofday.h | 16 +++---
> include/vdso/datapage.h | 1 -
> include/vdso/types.h | 39 -------------
> lib/vdso/gettimeofday.c | 48 +++++++++++-----
> 13 files changed, 164 insertions(+), 121 deletions(-)
> delete mode 100644 include/vdso/types.h
>
> diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
> index e7aeda0b255f..112df50920b0 100644
> --- a/arch/arm/include/asm/vdso/gettimeofday.h
> +++ b/arch/arm/include/asm/vdso/gettimeofday.h
> @@ -14,11 +14,11 @@
> extern struct vdso_data *__get_datapage(void);
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("r1") = _tz;
> - register struct __vdso_timeval *tv asm("r0") = _tv;
> + register struct __kernel_old_timeval *tv asm("r0") = _tv;
> register long ret asm ("r0");
> register long nr asm("r7") = __NR_gettimeofday;
>
> @@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_clock_gettime;
> + register long nr asm("r7") = __NR_clock_gettime64;
>
> asm volatile(
> " swi #0\n"
> @@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_clock_getres;
> + register long nr asm("r7") = __NR_clock_getres_time64;
>
> asm volatile(
> " swi #0\n"
> diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
> index b470a8ed35a7..b4607a5ecf31 100644
> --- a/arch/arm/vdso/vgettimeofday.c
> +++ b/arch/arm/vdso/vgettimeofday.c
> @@ -9,21 +9,21 @@
> #include <linux/types.h>
>
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> {
> - return __cvdso_clock_gettime(clock, ts);
> + return __cvdso_clock_gettime32(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct old_timespec32 *res)
> {
> - return __cvdso_clock_getres(clock_id, res);
> + return __cvdso_clock_getres_time32(clock_id, res);
> }
>
> /* Avoid unresolved references emitted by GCC */
> diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> index 888c70c24f71..b41d8f01a20c 100644
> --- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> +++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> @@ -24,11 +24,11 @@
> extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("r1") = _tz;
> - register struct __vdso_timeval *tv asm("r0") = _tv;
> + register struct __kernel_old_timeval *tv asm("r0") = _tv;
> register long ret asm ("r0");
> register long nr asm("r7") = __NR_compat_gettimeofday;
>
> @@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_compat_clock_gettime;
> + register long nr asm("r7") = __NR_compat_clock_gettime64;
>
> asm volatile(
> " swi #0\n"
> @@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_compat_clock_getres;
> + register long nr asm("r7") = __NR_compat_clock_getres_time64;
>
> /* The checks below are required for ABI consistency with arm */
> if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
> diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
> index 0aff6362533f..126fc6f4f7cf 100644
> --- a/arch/arm64/include/asm/vdso/gettimeofday.h
> +++ b/arch/arm64/include/asm/vdso/gettimeofday.h
> @@ -13,11 +13,11 @@
> extern struct vdso_data _vdso_data;
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("x1") = _tz;
> - register struct __vdso_timeval *tv asm("x0") = _tv;
> + register struct __kernel_old_timeval *tv asm("x0") = _tv;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_gettimeofday;
>
> @@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("x1") = _ts;
> + register struct __kernel_timespec *ts asm("x1") = _ts;
> register clockid_t clkid asm("x0") = _clkid;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_clock_gettime;
> @@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("x1") = _ts;
> + register struct __kernel_timespec *ts asm("x1") = _ts;
> register clockid_t clkid asm("x0") = _clkid;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_clock_getres;
> diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
> index 160e68196339..bbc83b6e2b1a 100644
> --- a/arch/arm64/kernel/vdso/vgettimeofday.c
> +++ b/arch/arm64/kernel/vdso/vgettimeofday.c
> @@ -9,19 +9,19 @@
> #include <linux/types.h>
>
> notrace int __kernel_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __kernel_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock_id, res);
> }
> diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
> index b470a8ed35a7..6762f800a8ea 100644
> --- a/arch/arm64/kernel/vdso32/vgettimeofday.c
> +++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
> @@ -9,21 +9,27 @@
> #include <linux/types.h>
>
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +notrace int __vdso_clock_gettime64(clockid_t clock,
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct old_timespec32 *res)
> {
> - return __cvdso_clock_getres(clock_id, res);
> + return __cvdso_clock_getres_time32(clock_id, res);
> }
>
> /* Avoid unresolved references emitted by GCC */
> diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
> index b275392f6b1b..ed6dafda5fc2 100644
> --- a/arch/mips/include/asm/vdso/gettimeofday.h
> +++ b/arch/mips/include/asm/vdso/gettimeofday.h
> @@ -25,11 +25,11 @@
> #ifdef CONFIG_MIPS_CLOCK_VSYSCALL
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("a1") = _tz;
> - register struct __vdso_timeval *tv asm("a0") = _tv;
> + register struct __kernel_old_timeval *tv asm("a0") = _tv;
> register long ret asm("v0");
> register long nr asm("v0") = __NR_gettimeofday;
> register long error asm("a3");
> @@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(
> #else
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> return -1;
> @@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("a1") = _ts;
> + register struct __kernel_timespec *ts asm("a1") = _ts;
> register clockid_t clkid asm("a0") = _clkid;
> register long ret asm("v0");
> +#if _MIPS_SIM == _MIPS_SIM_ABI64
> register long nr asm("v0") = __NR_clock_gettime;
> +#else
> + register long nr asm("v0") = __NR_clock_gettime64
> +#endif
> register long error asm("a3");
>
> asm volatile(
> @@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("a1") = _ts;
> + register struct __kernel_timespec *ts asm("a1") = _ts;
> register clockid_t clkid asm("a0") = _clkid;
> register long ret asm("v0");
> +#if _MIPS_SIM == _MIPS_SIM_ABI64
> register long nr asm("v0") = __NR_clock_getres;
> +#else
> + register long nr asm("v0") = __NR_clock_getres64
> +#endif
> register long error asm("a3");
>
> asm volatile(
> diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
> index 8da361d919f0..7a3320faa29a 100644
> --- a/arch/mips/vdso/vgettimeofday.c
> +++ b/arch/mips/vdso/vgettimeofday.c
> @@ -8,21 +8,43 @@
> #include <linux/time.h>
> #include <linux/types.h>
>
> +#if _MIPS_SIM != _MIPS_SIM_ABI64
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> + struct timezone *tz)
> +{
> + return __cvdso_gettimeofday_time32(tv, tz);
> +}
> +
> +notrace int __vdso_clock_getres(clockid_t clock_id,
> + struct old_timespec32 *res)
> +{
> + return __cvdso_clock_getres_time32(clock_id, res);
> +}
> +
> +#else
> +
> +notrace int __vdso_clock_gettime(clockid_t clock,
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock_id, res);
> }
>
> +#endif
> diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
> index 6ff8cabf4c3e..c9512c114ac1 100644
> --- a/arch/x86/entry/vdso/vclock_gettime.c
> +++ b/arch/x86/entry/vdso/vclock_gettime.c
> @@ -12,25 +12,14 @@
>
> #include "../../../../lib/vdso/gettimeofday.c"
>
> -extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
> -extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
> +extern int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
> extern time_t __vdso_time(time_t *t);
> -extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
> -
> -notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
> -{
> - return __cvdso_clock_gettime(clock, ts);
> -}
> -
> -int clock_gettime(clockid_t, struct __vdso_timespec *)
> - __attribute__((weak, alias("__vdso_clock_gettime")));
> -
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
> -int gettimeofday(struct __vdso_timeval *, struct timezone *)
> +int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
> __attribute__((weak, alias("__vdso_gettimeofday")));
>
> notrace time_t __vdso_time(time_t *t)
> @@ -40,10 +29,46 @@ notrace time_t __vdso_time(time_t *t)
> time_t time(time_t *t)
> __attribute__((weak, alias("__vdso_time")));
>
> +
> +#ifdef __x86_64__
> +/* both 64-bit and x32 use these */
> +extern int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
> +extern int __vdso_clock_getres(clockid_t clock, struct __kernel_timespec *res);
> +
> +notrace int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
> +{
> + return __cvdso_clock_gettime(clock, ts);
> +}
> +
> +int clock_gettime(clockid_t, struct __kernel_timespec *)
> + __attribute__((weak, alias("__vdso_clock_gettime")));
> +
> notrace int __vdso_clock_getres(clockid_t clock,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock, res);
> }
> -int clock_getres(clockid_t, struct __vdso_timespec *)
> +int clock_getres(clockid_t, struct __kernel_timespec *)
> + __attribute__((weak, alias("__vdso_clock_getres")));
> +
> +#else
> +/* i386 only */
> +extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
> +extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
> +
> +notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +int clock_gettime(clockid_t, struct old_timespec32 *)
> + __attribute__((weak, alias("__vdso_clock_gettime")));
> +
> +notrace int __vdso_clock_getres(clockid_t clock,
> + struct old_timespec32 *res)
> +{
> + return __cvdso_clock_getres_time32(clock, res);
> +}
> +int clock_getres(clockid_t, struct old_timespec32 *)
> __attribute__((weak, alias("__vdso_clock_getres")));
> +#endif
> diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
> index 1a86fdc1e400..510a0ebd5748 100644
> --- a/arch/x86/include/asm/vdso/gettimeofday.h
> +++ b/arch/x86/include/asm/vdso/gettimeofday.h
> @@ -38,7 +38,7 @@ extern u8 hvclock_page
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
> asm ("syscall" : "=a" (ret), "=m" (*_ts) :
> @@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback(
> }
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> long ret;
> @@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
> asm ("syscall" : "=a" (ret), "=m" (*_ts) :
> @@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
>
> @@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback(
> "call __kernel_vsyscall \n"
> "mov %%edx, %%ebx \n"
> : "=a" (ret), "=m" (*_ts)
> - : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
> + : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts)
> : "edx");
> return ret;
> }
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> long ret;
> @@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
>
> @@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback(
> "call __kernel_vsyscall \n"
> "mov %%edx, %%ebx \n"
> : "=a" (ret), "=m" (*_ts)
> - : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
> + : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts)
> : "edx");
> return ret;
> }
> diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
> index ff332fcba73c..da346ad02b03 100644
> --- a/include/vdso/datapage.h
> +++ b/include/vdso/datapage.h
> @@ -9,7 +9,6 @@
> #include <linux/bits.h>
> #include <linux/types.h>
> #include <linux/time.h>
> -#include <vdso/types.h>
>
> #define VDSO_BASES (CLOCK_TAI + 1)
> #define VDSO_HRES (BIT(CLOCK_REALTIME) | \
> diff --git a/include/vdso/types.h b/include/vdso/types.h
> deleted file mode 100644
> index f456a0a6a2e1..000000000000
> --- a/include/vdso/types.h
> +++ /dev/null
> @@ -1,39 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef __VDSO_TYPES_H
> -#define __VDSO_TYPES_H
> -
> -#ifdef __KERNEL__
> -
> -#ifndef __ASSEMBLY__
> -
> -#include <linux/types.h>
> -#include <linux/time.h>
> -
> -/*
> - * The definitions below are required to overcome the limitations
> - * of time_t on 32 bit architectures, which overflows in 2038.
> - * The new code should use the replacements based on time64_t and
> - * timespec64.
> - *
> - * The abstraction below will be updated once the migration to
> - * time64_t is complete.
> - */
> -#ifdef CONFIG_GENERIC_VDSO_32
> -#define __vdso_timespec old_timespec32
> -#define __vdso_timeval old_timeval32
> -#else
> -#ifdef ENABLE_COMPAT_VDSO
> -#define __vdso_timespec old_timespec32
> -#define __vdso_timeval old_timeval32
> -#else
> -#define __vdso_timespec __kernel_timespec
> -#define __vdso_timeval __kernel_old_timeval
> -#endif /* CONFIG_COMPAT_VDSO */
> -#endif /* CONFIG_GENERIC_VDSO_32 */
> -
> -
> -#endif /* !__ASSEMBLY__ */
> -
> -#endif /* __KERNEL__ */
> -
> -#endif /* __VDSO_TYPES_H */
> diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
> index 466d0c68fa80..f10e3703abaa 100644
> --- a/lib/vdso/gettimeofday.c
> +++ b/lib/vdso/gettimeofday.c
> @@ -28,7 +28,7 @@
>
> static notrace int do_hres(const struct vdso_data *vd,
> clockid_t clk,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> u64 cycles, last, sec, ns;
> @@ -63,7 +63,7 @@ static notrace int do_hres(const struct vdso_data *vd,
>
> static notrace void do_coarse(const struct vdso_data *vd,
> clockid_t clk,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> u32 seq;
> @@ -75,8 +75,8 @@ static notrace void do_coarse(const struct vdso_data *vd,
> } while (unlikely(vdso_read_retry(vd, seq)));
> }
>
> -static notrace int __cvdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> +static notrace __maybe_unused int
> +__cvdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
> u32 msk;
> @@ -100,19 +100,31 @@ static notrace int __cvdso_clock_gettime(clockid_t clock,
> return clock_gettime_fallback(clock, ts);
> }
>
> -static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
> - struct timezone *tz)
> +static notrace __maybe_unused int
> +__cvdso_clock_gettime32(clockid_t clock, struct old_timespec32 *res)
> +{
> + struct __kernel_timespec ts;
> + int ret = __cvdso_clock_gettime(clock, &ts);
> +
> + res->tv_sec = ts.tv_sec;
> + res->tv_nsec = ts.tv_nsec;
> +
> + return ret;
> +}
> +
> +static notrace __maybe_unused int
> +__cvdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
>
> if (likely(tv != NULL)) {
> - struct __vdso_timespec ts;
> + struct __kernel_timespec ts;
>
> if (do_hres(vd, CLOCK_REALTIME, &ts))
> return gettimeofday_fallback(tv, tz);
>
> tv->tv_sec = ts.tv_sec;
> - tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
> + tv->tv_usec = (u32)ts.tv_nsec / NSEC_PER_USEC;
> }
>
> if (unlikely(tz != NULL)) {
> @@ -123,8 +135,7 @@ static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
> return 0;
> }
>
> -#ifdef VDSO_HAS_TIME
> -static notrace time_t __cvdso_time(time_t *time)
> +static notrace __maybe_unused time_t __cvdso_time(time_t *time)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
> time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
> @@ -134,10 +145,9 @@ static notrace time_t __cvdso_time(time_t *time)
>
> return t;
> }
> -#endif /* VDSO_HAS_TIME */
>
> -static notrace int __cvdso_clock_getres(clockid_t clock,
> - struct __vdso_timespec *res)
> +static notrace __maybe_unused
> +int __cvdso_clock_getres(clockid_t clock, struct __kernel_timespec *res)
> {
> u64 sec, ns;
> u32 msk;
> @@ -177,3 +187,15 @@ static notrace int __cvdso_clock_getres(clockid_t clock,
> fallback:
> return clock_getres_fallback(clock, res);
> }
> +
> +static notrace __maybe_unused int
> +__cvdso_clock_getres_time32(clockid_t clock, struct old_timespec32 *res)
> +{
> + struct __kernel_timespec ts;
> + int ret = __cvdso_clock_getres(clock, &ts);
> +
> + res->tv_sec = ts.tv_sec;
> + res->tv_nsec = ts.tv_nsec;
> +
> + return ret;
> +}
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH 1/2] vdso: use fixed-size time types
@ 2019-02-28 13:45 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 13:45 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, linux-arm-kernel
Hi Arnd,
thank you for providing the changes.
On 28/02/2019 11:58, Arnd Bergmann wrote:
> Hi Vincento,
>
> I've implemented now what I tried to explain earlier. This moves the
> implementation of clock_gettime() and clock_getres() over to 64-bit
> __kernel_timespec, with a conversion in the end for architectures that
> actually use the 32-bit calls.
>
> The vdso/types.h file is now obsolete, but we do need to duplicate
> a few functions on mips32 and x86-32. It may be possible to fold those
> into the generic lib/vdso/gettimeofday.c file as another step.
>
> If there are no objections, please fold these changes into your
> series.
>
No objections on my side, I will fold the changes in my set.
> Signed-off-by: Arnd Bergmann <arnd@arndb.de>
> ---
> arch/arm/include/asm/vdso/gettimeofday.h | 16 +++---
> arch/arm/vdso/vgettimeofday.c | 10 ++--
> .../include/asm/vdso/compat_gettimeofday.h | 16 +++---
> arch/arm64/include/asm/vdso/gettimeofday.h | 12 ++--
> arch/arm64/kernel/vdso/vgettimeofday.c | 6 +-
> arch/arm64/kernel/vdso32/vgettimeofday.c | 14 +++--
> arch/mips/include/asm/vdso/gettimeofday.h | 22 ++++---
> arch/mips/vdso/vgettimeofday.c | 28 ++++++++-
> arch/x86/entry/vdso/vclock_gettime.c | 57 +++++++++++++------
> arch/x86/include/asm/vdso/gettimeofday.h | 16 +++---
> include/vdso/datapage.h | 1 -
> include/vdso/types.h | 39 -------------
> lib/vdso/gettimeofday.c | 48 +++++++++++-----
> 13 files changed, 164 insertions(+), 121 deletions(-)
> delete mode 100644 include/vdso/types.h
>
> diff --git a/arch/arm/include/asm/vdso/gettimeofday.h b/arch/arm/include/asm/vdso/gettimeofday.h
> index e7aeda0b255f..112df50920b0 100644
> --- a/arch/arm/include/asm/vdso/gettimeofday.h
> +++ b/arch/arm/include/asm/vdso/gettimeofday.h
> @@ -14,11 +14,11 @@
> extern struct vdso_data *__get_datapage(void);
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("r1") = _tz;
> - register struct __vdso_timeval *tv asm("r0") = _tv;
> + register struct __kernel_old_timeval *tv asm("r0") = _tv;
> register long ret asm ("r0");
> register long nr asm("r7") = __NR_gettimeofday;
>
> @@ -33,12 +33,12 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_clock_gettime;
> + register long nr asm("r7") = __NR_clock_gettime64;
>
> asm volatile(
> " swi #0\n"
> @@ -51,12 +51,12 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_clock_getres;
> + register long nr asm("r7") = __NR_clock_getres_time64;
>
> asm volatile(
> " swi #0\n"
> diff --git a/arch/arm/vdso/vgettimeofday.c b/arch/arm/vdso/vgettimeofday.c
> index b470a8ed35a7..b4607a5ecf31 100644
> --- a/arch/arm/vdso/vgettimeofday.c
> +++ b/arch/arm/vdso/vgettimeofday.c
> @@ -9,21 +9,21 @@
> #include <linux/types.h>
>
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> {
> - return __cvdso_clock_gettime(clock, ts);
> + return __cvdso_clock_gettime32(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct old_timespec32 *res)
> {
> - return __cvdso_clock_getres(clock_id, res);
> + return __cvdso_clock_getres_time32(clock_id, res);
> }
>
> /* Avoid unresolved references emitted by GCC */
> diff --git a/arch/arm64/include/asm/vdso/compat_gettimeofday.h b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> index 888c70c24f71..b41d8f01a20c 100644
> --- a/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> +++ b/arch/arm64/include/asm/vdso/compat_gettimeofday.h
> @@ -24,11 +24,11 @@
> extern struct vdso_data _vdso_data __attribute__((visibility("hidden")));
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("r1") = _tz;
> - register struct __vdso_timeval *tv asm("r0") = _tv;
> + register struct __kernel_old_timeval *tv asm("r0") = _tv;
> register long ret asm ("r0");
> register long nr asm("r7") = __NR_compat_gettimeofday;
>
> @@ -43,12 +43,12 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_compat_clock_gettime;
> + register long nr asm("r7") = __NR_compat_clock_gettime64;
>
> asm volatile(
> " swi #0\n"
> @@ -61,12 +61,12 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("r1") = _ts;
> + register struct __kernel_timespec *ts asm("r1") = _ts;
> register clockid_t clkid asm("r0") = _clkid;
> register long ret asm ("r0");
> - register long nr asm("r7") = __NR_compat_clock_getres;
> + register long nr asm("r7") = __NR_compat_clock_getres_time64;
>
> /* The checks below are required for ABI consistency with arm */
> if ((_clkid >= MAX_CLOCKS) || (_ts == NULL))
> diff --git a/arch/arm64/include/asm/vdso/gettimeofday.h b/arch/arm64/include/asm/vdso/gettimeofday.h
> index 0aff6362533f..126fc6f4f7cf 100644
> --- a/arch/arm64/include/asm/vdso/gettimeofday.h
> +++ b/arch/arm64/include/asm/vdso/gettimeofday.h
> @@ -13,11 +13,11 @@
> extern struct vdso_data _vdso_data;
>
> static __always_inline notrace int gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("x1") = _tz;
> - register struct __vdso_timeval *tv asm("x0") = _tv;
> + register struct __kernel_old_timeval *tv asm("x0") = _tv;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_gettimeofday;
>
> @@ -32,9 +32,9 @@ static __always_inline notrace int gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("x1") = _ts;
> + register struct __kernel_timespec *ts asm("x1") = _ts;
> register clockid_t clkid asm("x0") = _clkid;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_clock_gettime;
> @@ -50,9 +50,9 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("x1") = _ts;
> + register struct __kernel_timespec *ts asm("x1") = _ts;
> register clockid_t clkid asm("x0") = _clkid;
> register long ret asm ("x0");
> register long nr asm("x8") = __NR_clock_getres;
> diff --git a/arch/arm64/kernel/vdso/vgettimeofday.c b/arch/arm64/kernel/vdso/vgettimeofday.c
> index 160e68196339..bbc83b6e2b1a 100644
> --- a/arch/arm64/kernel/vdso/vgettimeofday.c
> +++ b/arch/arm64/kernel/vdso/vgettimeofday.c
> @@ -9,19 +9,19 @@
> #include <linux/types.h>
>
> notrace int __kernel_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __kernel_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __kernel_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __kernel_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock_id, res);
> }
> diff --git a/arch/arm64/kernel/vdso32/vgettimeofday.c b/arch/arm64/kernel/vdso32/vgettimeofday.c
> index b470a8ed35a7..6762f800a8ea 100644
> --- a/arch/arm64/kernel/vdso32/vgettimeofday.c
> +++ b/arch/arm64/kernel/vdso32/vgettimeofday.c
> @@ -9,21 +9,27 @@
> #include <linux/types.h>
>
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +notrace int __vdso_clock_gettime64(clockid_t clock,
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct old_timespec32 *res)
> {
> - return __cvdso_clock_getres(clock_id, res);
> + return __cvdso_clock_getres_time32(clock_id, res);
> }
>
> /* Avoid unresolved references emitted by GCC */
> diff --git a/arch/mips/include/asm/vdso/gettimeofday.h b/arch/mips/include/asm/vdso/gettimeofday.h
> index b275392f6b1b..ed6dafda5fc2 100644
> --- a/arch/mips/include/asm/vdso/gettimeofday.h
> +++ b/arch/mips/include/asm/vdso/gettimeofday.h
> @@ -25,11 +25,11 @@
> #ifdef CONFIG_MIPS_CLOCK_VSYSCALL
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> register struct timezone *tz asm("a1") = _tz;
> - register struct __vdso_timeval *tv asm("a0") = _tv;
> + register struct __kernel_old_timeval *tv asm("a0") = _tv;
> register long ret asm("v0");
> register long nr asm("v0") = __NR_gettimeofday;
> register long error asm("a3");
> @@ -47,7 +47,7 @@ static __always_inline notrace long gettimeofday_fallback(
> #else
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> return -1;
> @@ -57,12 +57,16 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("a1") = _ts;
> + register struct __kernel_timespec *ts asm("a1") = _ts;
> register clockid_t clkid asm("a0") = _clkid;
> register long ret asm("v0");
> +#if _MIPS_SIM == _MIPS_SIM_ABI64
> register long nr asm("v0") = __NR_clock_gettime;
> +#else
> + register long nr asm("v0") = __NR_clock_gettime64
> +#endif
> register long error asm("a3");
>
> asm volatile(
> @@ -77,12 +81,16 @@ static __always_inline notrace long clock_gettime_fallback(
>
> static __always_inline notrace int clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> - register struct __vdso_timespec *ts asm("a1") = _ts;
> + register struct __kernel_timespec *ts asm("a1") = _ts;
> register clockid_t clkid asm("a0") = _clkid;
> register long ret asm("v0");
> +#if _MIPS_SIM == _MIPS_SIM_ABI64
> register long nr asm("v0") = __NR_clock_getres;
> +#else
> + register long nr asm("v0") = __NR_clock_getres64
> +#endif
> register long error asm("a3");
>
> asm volatile(
> diff --git a/arch/mips/vdso/vgettimeofday.c b/arch/mips/vdso/vgettimeofday.c
> index 8da361d919f0..7a3320faa29a 100644
> --- a/arch/mips/vdso/vgettimeofday.c
> +++ b/arch/mips/vdso/vgettimeofday.c
> @@ -8,21 +8,43 @@
> #include <linux/time.h>
> #include <linux/types.h>
>
> +#if _MIPS_SIM != _MIPS_SIM_ABI64
> notrace int __vdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> + struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> + struct timezone *tz)
> +{
> + return __cvdso_gettimeofday_time32(tv, tz);
> +}
> +
> +notrace int __vdso_clock_getres(clockid_t clock_id,
> + struct old_timespec32 *res)
> +{
> + return __cvdso_clock_getres_time32(clock_id, res);
> +}
> +
> +#else
> +
> +notrace int __vdso_clock_gettime(clockid_t clock,
> + struct __kernel_timespec *ts)
> {
> return __cvdso_clock_gettime(clock, ts);
> }
>
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
>
> notrace int __vdso_clock_getres(clockid_t clock_id,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock_id, res);
> }
>
> +#endif
> diff --git a/arch/x86/entry/vdso/vclock_gettime.c b/arch/x86/entry/vdso/vclock_gettime.c
> index 6ff8cabf4c3e..c9512c114ac1 100644
> --- a/arch/x86/entry/vdso/vclock_gettime.c
> +++ b/arch/x86/entry/vdso/vclock_gettime.c
> @@ -12,25 +12,14 @@
>
> #include "../../../../lib/vdso/gettimeofday.c"
>
> -extern int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts);
> -extern int __vdso_gettimeofday(struct __vdso_timeval *tv, struct timezone *tz);
> +extern int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz);
> extern time_t __vdso_time(time_t *t);
> -extern int __vdso_clock_getres(clockid_t clock, struct __vdso_timespec *res);
> -
> -notrace int __vdso_clock_gettime(clockid_t clock, struct __vdso_timespec *ts)
> -{
> - return __cvdso_clock_gettime(clock, ts);
> -}
> -
> -int clock_gettime(clockid_t, struct __vdso_timespec *)
> - __attribute__((weak, alias("__vdso_clock_gettime")));
> -
> -notrace int __vdso_gettimeofday(struct __vdso_timeval *tv,
> +notrace int __vdso_gettimeofday(struct __kernel_old_timeval *tv,
> struct timezone *tz)
> {
> return __cvdso_gettimeofday(tv, tz);
> }
> -int gettimeofday(struct __vdso_timeval *, struct timezone *)
> +int gettimeofday(struct __kernel_old_timeval *, struct timezone *)
> __attribute__((weak, alias("__vdso_gettimeofday")));
>
> notrace time_t __vdso_time(time_t *t)
> @@ -40,10 +29,46 @@ notrace time_t __vdso_time(time_t *t)
> time_t time(time_t *t)
> __attribute__((weak, alias("__vdso_time")));
>
> +
> +#ifdef __x86_64__
> +/* both 64-bit and x32 use these */
> +extern int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts);
> +extern int __vdso_clock_getres(clockid_t clock, struct __kernel_timespec *res);
> +
> +notrace int __vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
> +{
> + return __cvdso_clock_gettime(clock, ts);
> +}
> +
> +int clock_gettime(clockid_t, struct __kernel_timespec *)
> + __attribute__((weak, alias("__vdso_clock_gettime")));
> +
> notrace int __vdso_clock_getres(clockid_t clock,
> - struct __vdso_timespec *res)
> + struct __kernel_timespec *res)
> {
> return __cvdso_clock_getres(clock, res);
> }
> -int clock_getres(clockid_t, struct __vdso_timespec *)
> +int clock_getres(clockid_t, struct __kernel_timespec *)
> + __attribute__((weak, alias("__vdso_clock_getres")));
> +
> +#else
> +/* i386 only */
> +extern int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts);
> +extern int __vdso_clock_getres(clockid_t clock, struct old_timespec32 *res);
> +
> +notrace int __vdso_clock_gettime(clockid_t clock, struct old_timespec32 *ts)
> +{
> + return __cvdso_clock_gettime32(clock, ts);
> +}
> +
> +int clock_gettime(clockid_t, struct old_timespec32 *)
> + __attribute__((weak, alias("__vdso_clock_gettime")));
> +
> +notrace int __vdso_clock_getres(clockid_t clock,
> + struct old_timespec32 *res)
> +{
> + return __cvdso_clock_getres_time32(clock, res);
> +}
> +int clock_getres(clockid_t, struct old_timespec32 *)
> __attribute__((weak, alias("__vdso_clock_getres")));
> +#endif
> diff --git a/arch/x86/include/asm/vdso/gettimeofday.h b/arch/x86/include/asm/vdso/gettimeofday.h
> index 1a86fdc1e400..510a0ebd5748 100644
> --- a/arch/x86/include/asm/vdso/gettimeofday.h
> +++ b/arch/x86/include/asm/vdso/gettimeofday.h
> @@ -38,7 +38,7 @@ extern u8 hvclock_page
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
> asm ("syscall" : "=a" (ret), "=m" (*_ts) :
> @@ -48,7 +48,7 @@ static __always_inline notrace long clock_gettime_fallback(
> }
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> long ret;
> @@ -59,7 +59,7 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
> asm ("syscall" : "=a" (ret), "=m" (*_ts) :
> @@ -72,7 +72,7 @@ static __always_inline notrace long clock_getres_fallback(
>
> static __always_inline notrace long clock_gettime_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
>
> @@ -82,13 +82,13 @@ static __always_inline notrace long clock_gettime_fallback(
> "call __kernel_vsyscall \n"
> "mov %%edx, %%ebx \n"
> : "=a" (ret), "=m" (*_ts)
> - : "0" (__NR_clock_gettime), [clock] "g" (_clkid), "c" (_ts)
> + : "0" (__NR_clock_gettime64), [clock] "g" (_clkid), "c" (_ts)
> : "edx");
> return ret;
> }
>
> static __always_inline notrace long gettimeofday_fallback(
> - struct __vdso_timeval *_tv,
> + struct __kernel_old_timeval *_tv,
> struct timezone *_tz)
> {
> long ret;
> @@ -105,7 +105,7 @@ static __always_inline notrace long gettimeofday_fallback(
>
> static __always_inline notrace long clock_getres_fallback(
> clockid_t _clkid,
> - struct __vdso_timespec *_ts)
> + struct __kernel_timespec *_ts)
> {
> long ret;
>
> @@ -115,7 +115,7 @@ static __always_inline notrace long clock_getres_fallback(
> "call __kernel_vsyscall \n"
> "mov %%edx, %%ebx \n"
> : "=a" (ret), "=m" (*_ts)
> - : "0" (__NR_clock_getres), [clock] "g" (_clkid), "c" (_ts)
> + : "0" (__NR_clock_getres_time64), [clock] "g" (_clkid), "c" (_ts)
> : "edx");
> return ret;
> }
> diff --git a/include/vdso/datapage.h b/include/vdso/datapage.h
> index ff332fcba73c..da346ad02b03 100644
> --- a/include/vdso/datapage.h
> +++ b/include/vdso/datapage.h
> @@ -9,7 +9,6 @@
> #include <linux/bits.h>
> #include <linux/types.h>
> #include <linux/time.h>
> -#include <vdso/types.h>
>
> #define VDSO_BASES (CLOCK_TAI + 1)
> #define VDSO_HRES (BIT(CLOCK_REALTIME) | \
> diff --git a/include/vdso/types.h b/include/vdso/types.h
> deleted file mode 100644
> index f456a0a6a2e1..000000000000
> --- a/include/vdso/types.h
> +++ /dev/null
> @@ -1,39 +0,0 @@
> -/* SPDX-License-Identifier: GPL-2.0 */
> -#ifndef __VDSO_TYPES_H
> -#define __VDSO_TYPES_H
> -
> -#ifdef __KERNEL__
> -
> -#ifndef __ASSEMBLY__
> -
> -#include <linux/types.h>
> -#include <linux/time.h>
> -
> -/*
> - * The definitions below are required to overcome the limitations
> - * of time_t on 32 bit architectures, which overflows in 2038.
> - * The new code should use the replacements based on time64_t and
> - * timespec64.
> - *
> - * The abstraction below will be updated once the migration to
> - * time64_t is complete.
> - */
> -#ifdef CONFIG_GENERIC_VDSO_32
> -#define __vdso_timespec old_timespec32
> -#define __vdso_timeval old_timeval32
> -#else
> -#ifdef ENABLE_COMPAT_VDSO
> -#define __vdso_timespec old_timespec32
> -#define __vdso_timeval old_timeval32
> -#else
> -#define __vdso_timespec __kernel_timespec
> -#define __vdso_timeval __kernel_old_timeval
> -#endif /* CONFIG_COMPAT_VDSO */
> -#endif /* CONFIG_GENERIC_VDSO_32 */
> -
> -
> -#endif /* !__ASSEMBLY__ */
> -
> -#endif /* __KERNEL__ */
> -
> -#endif /* __VDSO_TYPES_H */
> diff --git a/lib/vdso/gettimeofday.c b/lib/vdso/gettimeofday.c
> index 466d0c68fa80..f10e3703abaa 100644
> --- a/lib/vdso/gettimeofday.c
> +++ b/lib/vdso/gettimeofday.c
> @@ -28,7 +28,7 @@
>
> static notrace int do_hres(const struct vdso_data *vd,
> clockid_t clk,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> u64 cycles, last, sec, ns;
> @@ -63,7 +63,7 @@ static notrace int do_hres(const struct vdso_data *vd,
>
> static notrace void do_coarse(const struct vdso_data *vd,
> clockid_t clk,
> - struct __vdso_timespec *ts)
> + struct __kernel_timespec *ts)
> {
> const struct vdso_timestamp *vdso_ts = &vd->basetime[clk];
> u32 seq;
> @@ -75,8 +75,8 @@ static notrace void do_coarse(const struct vdso_data *vd,
> } while (unlikely(vdso_read_retry(vd, seq)));
> }
>
> -static notrace int __cvdso_clock_gettime(clockid_t clock,
> - struct __vdso_timespec *ts)
> +static notrace __maybe_unused int
> +__cvdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
> u32 msk;
> @@ -100,19 +100,31 @@ static notrace int __cvdso_clock_gettime(clockid_t clock,
> return clock_gettime_fallback(clock, ts);
> }
>
> -static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
> - struct timezone *tz)
> +static notrace __maybe_unused int
> +__cvdso_clock_gettime32(clockid_t clock, struct old_timespec32 *res)
> +{
> + struct __kernel_timespec ts;
> + int ret = __cvdso_clock_gettime(clock, &ts);
> +
> + res->tv_sec = ts.tv_sec;
> + res->tv_nsec = ts.tv_nsec;
> +
> + return ret;
> +}
> +
> +static notrace __maybe_unused int
> +__cvdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
>
> if (likely(tv != NULL)) {
> - struct __vdso_timespec ts;
> + struct __kernel_timespec ts;
>
> if (do_hres(vd, CLOCK_REALTIME, &ts))
> return gettimeofday_fallback(tv, tz);
>
> tv->tv_sec = ts.tv_sec;
> - tv->tv_usec = ts.tv_nsec / NSEC_PER_USEC;
> + tv->tv_usec = (u32)ts.tv_nsec / NSEC_PER_USEC;
> }
>
> if (unlikely(tz != NULL)) {
> @@ -123,8 +135,7 @@ static notrace int __cvdso_gettimeofday(struct __vdso_timeval *tv,
> return 0;
> }
>
> -#ifdef VDSO_HAS_TIME
> -static notrace time_t __cvdso_time(time_t *time)
> +static notrace __maybe_unused time_t __cvdso_time(time_t *time)
> {
> const struct vdso_data *vd = __arch_get_vdso_data();
> time_t t = READ_ONCE(vd->basetime[CLOCK_REALTIME].sec);
> @@ -134,10 +145,9 @@ static notrace time_t __cvdso_time(time_t *time)
>
> return t;
> }
> -#endif /* VDSO_HAS_TIME */
>
> -static notrace int __cvdso_clock_getres(clockid_t clock,
> - struct __vdso_timespec *res)
> +static notrace __maybe_unused
> +int __cvdso_clock_getres(clockid_t clock, struct __kernel_timespec *res)
> {
> u64 sec, ns;
> u32 msk;
> @@ -177,3 +187,15 @@ static notrace int __cvdso_clock_getres(clockid_t clock,
> fallback:
> return clock_getres_fallback(clock, res);
> }
> +
> +static notrace __maybe_unused int
> +__cvdso_clock_getres_time32(clockid_t clock, struct old_timespec32 *res)
> +{
> + struct __kernel_timespec ts;
> + int ret = __cvdso_clock_getres(clock, &ts);
> +
> + res->tv_sec = ts.tv_sec;
> + res->tv_nsec = ts.tv_nsec;
> +
> + return ret;
> +}
>
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 13:54 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 13:54 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Thu, Feb 28, 2019 at 1:42 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> On 28/02/2019 12:38, Arnd Bergmann wrote:
> > On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >>
> >> thank you for testing my code.
> >>
> >> On 28/02/2019 11:40, Arnd Bergmann wrote:
> >>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> >>> <vincenzo.frascino@arm.com> wrote:
> >>> ^
> >>>
> >>> I think this is all harmless, but we need to limit the set of header files
> >>> that can be included indirectly when compiling the vdso in arm32 mode.
> >>>
> >>
> >> I agree with you on the principle, but I am unable to replicate the warnings.
> >> Did you make any customization to the default defconfig? What is the top
> >> commit-id you have on your tree?
> >
> > I observed this on top of v5.0-rc7 earlier, and have since rebased to
> > today's linux-next, where it still happens with the regular 'make defconfig',
> > on all compiler versions I have (4.8 through 8.1).
> >
>
> Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
I just found a related issue: when CONFIG_CPU_BIG_ENDIAN is
set, everything breaks in the compat vdso because of the included
arm64 headers in arm32 code. That should be easier to reproduce.
For the moment, I'm using this patch:
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,7 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
- select GENERIC_COMPAT_VDSO
+ select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 13:54 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 13:54 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
On Thu, Feb 28, 2019 at 1:42 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> On 28/02/2019 12:38, Arnd Bergmann wrote:
> > On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >>
> >> thank you for testing my code.
> >>
> >> On 28/02/2019 11:40, Arnd Bergmann wrote:
> >>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> >>> <vincenzo.frascino@arm.com> wrote:
> >>> ^
> >>>
> >>> I think this is all harmless, but we need to limit the set of header files
> >>> that can be included indirectly when compiling the vdso in arm32 mode.
> >>>
> >>
> >> I agree with you on the principle, but I am unable to replicate the warnings.
> >> Did you make any customization to the default defconfig? What is the top
> >> commit-id you have on your tree?
> >
> > I observed this on top of v5.0-rc7 earlier, and have since rebased to
> > today's linux-next, where it still happens with the regular 'make defconfig',
> > on all compiler versions I have (4.8 through 8.1).
> >
>
> Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
I just found a related issue: when CONFIG_CPU_BIG_ENDIAN is
set, everything breaks in the compat vdso because of the included
arm64 headers in arm32 code. That should be easier to reproduce.
For the moment, I'm using this patch:
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,7 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
- select GENERIC_COMPAT_VDSO
+ select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
Arnd
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 13:54 ` Arnd Bergmann
0 siblings, 0 replies; 178+ messages in thread
From: Arnd Bergmann @ 2019-02-28 13:54 UTC (permalink / raw)
To: Vincenzo Frascino
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
On Thu, Feb 28, 2019 at 1:42 PM Vincenzo Frascino
<vincenzo.frascino@arm.com> wrote:
>
> On 28/02/2019 12:38, Arnd Bergmann wrote:
> > On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
> > <vincenzo.frascino@arm.com> wrote:
> >>
> >> thank you for testing my code.
> >>
> >> On 28/02/2019 11:40, Arnd Bergmann wrote:
> >>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
> >>> <vincenzo.frascino@arm.com> wrote:
> >>> ^
> >>>
> >>> I think this is all harmless, but we need to limit the set of header files
> >>> that can be included indirectly when compiling the vdso in arm32 mode.
> >>>
> >>
> >> I agree with you on the principle, but I am unable to replicate the warnings.
> >> Did you make any customization to the default defconfig? What is the top
> >> commit-id you have on your tree?
> >
> > I observed this on top of v5.0-rc7 earlier, and have since rebased to
> > today's linux-next, where it still happens with the regular 'make defconfig',
> > on all compiler versions I have (4.8 through 8.1).
> >
>
> Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
I just found a related issue: when CONFIG_CPU_BIG_ENDIAN is
set, everything breaks in the compat vdso because of the included
arm64 headers in arm32 code. That should be easier to reproduce.
For the moment, I'm using this patch:
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -103,7 +103,7 @@ config ARM64
select GENERIC_STRNLEN_USER
select GENERIC_TIME_VSYSCALL
select GENERIC_GETTIMEOFDAY
- select GENERIC_COMPAT_VDSO
+ select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN
select HANDLE_DOMAIN_IRQ
select HARDIRQS_SW_RESEND
select HAVE_PCI
Arnd
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 15:51 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 15:51 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
Hi Arnd,
On 28/02/2019 13:54, Arnd Bergmann wrote:
> On Thu, Feb 28, 2019 at 1:42 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> On 28/02/2019 12:38, Arnd Bergmann wrote:
>>> On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
>>> <vincenzo.frascino@arm.com> wrote:
>>>>
>>>> thank you for testing my code.
>>>>
>>>> On 28/02/2019 11:40, Arnd Bergmann wrote:
>>>>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
>>>>> <vincenzo.frascino@arm.com> wrote:
>>>>> ^
>>>>>
>>>>> I think this is all harmless, but we need to limit the set of header files
>>>>> that can be included indirectly when compiling the vdso in arm32 mode.
>>>>>
>>>>
>>>> I agree with you on the principle, but I am unable to replicate the warnings.
>>>> Did you make any customization to the default defconfig? What is the top
>>>> commit-id you have on your tree?
>>>
>>> I observed this on top of v5.0-rc7 earlier, and have since rebased to
>>> today's linux-next, where it still happens with the regular 'make defconfig',
>>> on all compiler versions I have (4.8 through 8.1).
>>>
>>
>> Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
>
> I just found a related issue: when CONFIG_CPU_BIG_ENDIAN is
> set, everything breaks in the compat vdso because of the included
> arm64 headers in arm32 code. That should be easier to reproduce.
> For the moment, I'm using this patch:
>
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -103,7 +103,7 @@ config ARM64
> select GENERIC_STRNLEN_USER
> select GENERIC_TIME_VSYSCALL
> select GENERIC_GETTIMEOFDAY
> - select GENERIC_COMPAT_VDSO
> + select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN
> select HANDLE_DOMAIN_IRQ
> select HARDIRQS_SW_RESEND
> select HAVE_PCI
>
>
Thanks for this, but I can't still reproduce the issue.
I rebased my patches on:
commit 42fd8df9d1d9c83046226e771a4f5f3d05a706ff (tag: next-20190228,
linux-next/master)
and enabled CONFIG_CPU_BIG_ENDIAN.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 15:51 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 15:51 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Linux ARM, Catalin Marinas, Will Deacon,
Russell King, Ralf Baechle, Paul Burton, Daniel Lezcano,
Thomas Gleixner, Mark Salyzyn, Peter Collingbourne, Shuah Khan,
Dmitry Safonov, Rasmus Villemoes
Hi Arnd,
On 28/02/2019 13:54, Arnd Bergmann wrote:
> On Thu, Feb 28, 2019 at 1:42 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> On 28/02/2019 12:38, Arnd Bergmann wrote:
>>> On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
>>> <vincenzo.frascino@arm.com> wrote:
>>>>
>>>> thank you for testing my code.
>>>>
>>>> On 28/02/2019 11:40, Arnd Bergmann wrote:
>>>>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
>>>>> <vincenzo.frascino@arm.com> wrote:
>>>>> ^
>>>>>
>>>>> I think this is all harmless, but we need to limit the set of header files
>>>>> that can be included indirectly when compiling the vdso in arm32 mode.
>>>>>
>>>>
>>>> I agree with you on the principle, but I am unable to replicate the warnings.
>>>> Did you make any customization to the default defconfig? What is the top
>>>> commit-id you have on your tree?
>>>
>>> I observed this on top of v5.0-rc7 earlier, and have since rebased to
>>> today's linux-next, where it still happens with the regular 'make defconfig',
>>> on all compiler versions I have (4.8 through 8.1).
>>>
>>
>> Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
>
> I just found a related issue: when CONFIG_CPU_BIG_ENDIAN is
> set, everything breaks in the compat vdso because of the included
> arm64 headers in arm32 code. That should be easier to reproduce.
> For the moment, I'm using this patch:
>
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -103,7 +103,7 @@ config ARM64
> select GENERIC_STRNLEN_USER
> select GENERIC_TIME_VSYSCALL
> select GENERIC_GETTIMEOFDAY
> - select GENERIC_COMPAT_VDSO
> + select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN
> select HANDLE_DOMAIN_IRQ
> select HARDIRQS_SW_RESEND
> select HAVE_PCI
>
>
Thanks for this, but I can't still reproduce the issue.
I rebased my patches on:
commit 42fd8df9d1d9c83046226e771a4f5f3d05a706ff (tag: next-20190228,
linux-next/master)
and enabled CONFIG_CPU_BIG_ENDIAN.
> Arnd
>
--
Regards,
Vincenzo
^ permalink raw reply [flat|nested] 178+ messages in thread
* Re: [PATCH v5 00/23] Unify vDSOs across more architectures
@ 2019-02-28 15:51 ` Vincenzo Frascino
0 siblings, 0 replies; 178+ messages in thread
From: Vincenzo Frascino @ 2019-02-28 15:51 UTC (permalink / raw)
To: Arnd Bergmann
Cc: linux-arch, Shuah Khan, Rasmus Villemoes, Catalin Marinas,
Daniel Lezcano, Will Deacon, Russell King, Ralf Baechle,
Mark Salyzyn, Paul Burton, Dmitry Safonov, Thomas Gleixner,
Peter Collingbourne, Linux ARM
Hi Arnd,
On 28/02/2019 13:54, Arnd Bergmann wrote:
> On Thu, Feb 28, 2019 at 1:42 PM Vincenzo Frascino
> <vincenzo.frascino@arm.com> wrote:
>>
>> On 28/02/2019 12:38, Arnd Bergmann wrote:
>>> On Thu, Feb 28, 2019 at 1:09 PM Vincenzo Frascino
>>> <vincenzo.frascino@arm.com> wrote:
>>>>
>>>> thank you for testing my code.
>>>>
>>>> On 28/02/2019 11:40, Arnd Bergmann wrote:
>>>>> On Fri, Feb 22, 2019 at 1:25 PM Vincenzo Frascino
>>>>> <vincenzo.frascino@arm.com> wrote:
>>>>> ^
>>>>>
>>>>> I think this is all harmless, but we need to limit the set of header files
>>>>> that can be included indirectly when compiling the vdso in arm32 mode.
>>>>>
>>>>
>>>> I agree with you on the principle, but I am unable to replicate the warnings.
>>>> Did you make any customization to the default defconfig? What is the top
>>>> commit-id you have on your tree?
>>>
>>> I observed this on top of v5.0-rc7 earlier, and have since rebased to
>>> today's linux-next, where it still happens with the regular 'make defconfig',
>>> on all compiler versions I have (4.8 through 8.1).
>>>
>>
>> Thanks, I will try to rebase my patches on linux-next and see if I can reproduce it.
>
> I just found a related issue: when CONFIG_CPU_BIG_ENDIAN is
> set, everything breaks in the compat vdso because of the included
> arm64 headers in arm32 code. That should be easier to reproduce.
> For the moment, I'm using this patch:
>
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -103,7 +103,7 @@ config ARM64
> select GENERIC_STRNLEN_USER
> select GENERIC_TIME_VSYSCALL
> select GENERIC_GETTIMEOFDAY
> - select GENERIC_COMPAT_VDSO
> + select GENERIC_COMPAT_VDSO if !CPU_BIG_ENDIAN
> select HANDLE_DOMAIN_IRQ
> select HARDIRQS_SW_RESEND
> select HAVE_PCI
>
>
Thanks for this, but I can't still reproduce the issue.
I rebased my patches on:
commit 42fd8df9d1d9c83046226e771a4f5f3d05a706ff (tag: next-20190228,
linux-next/master)
and enabled CONFIG_CPU_BIG_ENDIAN.
> Arnd
>
--
Regards,
Vincenzo
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 178+ messages in thread
end of thread, other threads:[~2019-02-28 15:51 UTC | newest]
Thread overview: 178+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-02-22 12:24 [PATCH v5 00/23] Unify vDSOs across more architectures Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 01/23] kernel: Standardize vdso_datapage Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:58 ` Mark Rutland
2019-02-22 12:58 ` Mark Rutland
2019-02-22 12:58 ` Mark Rutland
2019-02-23 16:51 ` Thomas Gleixner
2019-02-23 16:51 ` Thomas Gleixner
2019-02-23 16:51 ` Thomas Gleixner
2019-02-27 14:23 ` Vincenzo Frascino
2019-02-27 14:23 ` Vincenzo Frascino
2019-02-27 14:23 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 02/23] kernel: Define gettimeofday vdso common code Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 13:34 ` Mark Rutland
2019-02-22 13:34 ` Mark Rutland
2019-02-22 13:34 ` Mark Rutland
2019-02-22 13:49 ` Arnd Bergmann
2019-02-22 13:49 ` Arnd Bergmann
2019-02-22 13:49 ` Arnd Bergmann
2019-02-22 14:36 ` Arnd Bergmann
2019-02-22 14:36 ` Arnd Bergmann
2019-02-22 14:36 ` Arnd Bergmann
2019-02-27 14:52 ` Vincenzo Frascino
2019-02-27 14:52 ` Vincenzo Frascino
2019-02-27 14:52 ` Vincenzo Frascino
2019-02-28 9:29 ` Arnd Bergmann
2019-02-28 9:29 ` Arnd Bergmann
2019-02-28 9:29 ` Arnd Bergmann
2019-02-28 11:58 ` [PATCH 1/2] vdso: use fixed-size time types Arnd Bergmann
2019-02-28 11:58 ` Arnd Bergmann
2019-02-28 11:58 ` Arnd Bergmann
2019-02-28 11:58 ` [PATCH 2/2] vdso: add clock_gettime64 Arnd Bergmann
2019-02-28 11:58 ` Arnd Bergmann
2019-02-28 11:58 ` Arnd Bergmann
2019-02-28 13:42 ` [PATCH 1/2] vdso: use fixed-size time types Thomas Gleixner
2019-02-28 13:42 ` Thomas Gleixner
2019-02-28 13:42 ` Thomas Gleixner
2019-02-28 13:45 ` Vincenzo Frascino
2019-02-28 13:45 ` Vincenzo Frascino
2019-02-28 13:45 ` Vincenzo Frascino
2019-02-23 10:34 ` [PATCH v5 02/23] kernel: Define gettimeofday vdso common code Thomas Gleixner
2019-02-23 10:34 ` Thomas Gleixner
2019-02-23 10:34 ` Thomas Gleixner
2019-02-25 14:09 ` Vincenzo Frascino
2019-02-25 14:09 ` Vincenzo Frascino
2019-02-25 14:09 ` Vincenzo Frascino
2019-02-23 17:31 ` Thomas Gleixner
2019-02-23 17:31 ` Thomas Gleixner
2019-02-23 17:31 ` Thomas Gleixner
2019-02-27 13:47 ` Vincenzo Frascino
2019-02-27 13:47 ` Vincenzo Frascino
2019-02-27 13:47 ` Vincenzo Frascino
2019-02-27 15:49 ` Thomas Gleixner
2019-02-27 15:49 ` Thomas Gleixner
2019-02-27 15:49 ` Thomas Gleixner
2019-02-27 16:06 ` Vincenzo Frascino
2019-02-27 16:06 ` Vincenzo Frascino
2019-02-27 16:06 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 03/23] arm64: Build vDSO with -ffixed-x18 Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 04/23] arm64: Substitute gettimeofday with C implementation Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 13:42 ` Mark Rutland
2019-02-22 13:42 ` Mark Rutland
2019-02-22 13:42 ` Mark Rutland
2019-02-22 12:24 ` [PATCH v5 05/23] arm64: compat: Alloc separate pages for vectors and sigpage Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 06/23] arm64: compat: Split kuser32 Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 07/23] arm64: compat: Refactor aarch32_alloc_vdso_pages() Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 08/23] arm64: compat: Add KUSER_HELPERS config option Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 14:04 ` Mark Rutland
2019-02-22 14:04 ` Mark Rutland
2019-02-22 14:04 ` Mark Rutland
2019-02-22 14:09 ` Russell King - ARM Linux admin
2019-02-22 14:09 ` Russell King - ARM Linux admin
2019-02-22 14:09 ` Russell King - ARM Linux admin
2019-02-26 12:10 ` Vincenzo Frascino
2019-02-26 12:10 ` Vincenzo Frascino
2019-02-26 12:10 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 09/23] arm64: compat: Add missing syscall numbers Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 10/23] arm64: compat: Expose signal related structures Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 11/23] arm64: compat: Generate asm offsets for signals Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 14:22 ` Mark Rutland
2019-02-22 14:22 ` Mark Rutland
2019-02-22 14:22 ` Mark Rutland
2019-02-22 12:24 ` [PATCH v5 12/23] lib: vdso: Add compat support Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 13/23] arm64: compat: Add vDSO Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 14:32 ` Mark Rutland
2019-02-22 14:32 ` Mark Rutland
2019-02-22 14:32 ` Mark Rutland
2019-02-22 12:24 ` [PATCH v5 14/23] arm64: Refactor vDSO code Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 14:37 ` Mark Rutland
2019-02-22 14:37 ` Mark Rutland
2019-02-22 14:37 ` Mark Rutland
2019-02-22 12:24 ` [PATCH v5 15/23] arm64: compat: vDSO setup for compat layer Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 16/23] arm64: elf: vDSO code page discovery Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 17/23] arm64: compat: Get sigreturn trampolines from vDSO Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 18/23] arm64: Add vDSO compat support Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 19/23] arm64: Enable compat vDSO support Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 14:39 ` Mark Rutland
2019-02-22 14:39 ` Mark Rutland
2019-02-22 14:39 ` Mark Rutland
2019-02-22 14:41 ` Mark Rutland
2019-02-22 14:41 ` Mark Rutland
2019-02-22 14:41 ` Mark Rutland
2019-02-22 12:24 ` [PATCH v5 20/23] arm: Add support for generic vDSO Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 21/23] mips: " Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 22/23] x86: " Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-23 10:45 ` Thomas Gleixner
2019-02-23 10:45 ` Thomas Gleixner
2019-02-23 10:45 ` Thomas Gleixner
2019-02-27 16:18 ` Vincenzo Frascino
2019-02-27 16:18 ` Vincenzo Frascino
2019-02-27 16:18 ` Vincenzo Frascino
2019-02-22 12:24 ` [PATCH v5 23/23] kselftest: Extend vDSO selftest Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-22 12:24 ` Vincenzo Frascino
2019-02-23 17:39 ` [PATCH v5 00/23] Unify vDSOs across more architectures Thomas Gleixner
2019-02-23 17:39 ` Thomas Gleixner
2019-02-23 17:39 ` Thomas Gleixner
2019-02-28 11:40 ` Arnd Bergmann
2019-02-28 11:40 ` Arnd Bergmann
2019-02-28 12:09 ` Vincenzo Frascino
2019-02-28 12:09 ` Vincenzo Frascino
2019-02-28 12:38 ` Arnd Bergmann
2019-02-28 12:38 ` Arnd Bergmann
2019-02-28 12:38 ` Arnd Bergmann
2019-02-28 12:42 ` Vincenzo Frascino
2019-02-28 12:42 ` Vincenzo Frascino
2019-02-28 12:42 ` Vincenzo Frascino
2019-02-28 13:54 ` Arnd Bergmann
2019-02-28 13:54 ` Arnd Bergmann
2019-02-28 13:54 ` Arnd Bergmann
2019-02-28 15:51 ` Vincenzo Frascino
2019-02-28 15:51 ` Vincenzo Frascino
2019-02-28 15:51 ` Vincenzo Frascino
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.