* [PATCH v2 00/28] ARM Scalable Vector Extension (SVE)
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
This series implements Linux kernel support for the ARM Scalable Vector
Extension (SVE). [1] It supersedes the previous v1: see [3] for link.
See the individual patches for details of changes.
The patches apply on v4.13-rc7 + linux-arm64/for-next/core.
For convenience, a git tree is available. [4]
To reduce spam, some people may not been copied on the entire series.
For those who did not receive the whole series, it can be found in the
linux-arm-kernel archive. [2]
*Note* The final two patches (27-28) of the series are still RFC --
before committing to this ABI it would be good to get feedback on
whether the approach makes sense and whether it suitable for other
architectures. These two patches are not required by the rest of the
series and can be revised or merged later.
Support for use of SVE by KVM guests is not currently included.
Instead, such use will be trapped and reflected to the guest as
undefined instruction execution. SVE is hidden from the view of the
CPU feature registers visible to guests, so that guests will not
expect it to work.
This series has been build- and boot-tested on Juno r0 and the ARM FVP
Base model with SVE plugin. Because there is no hardware with SVE
support yet, testing of the SVE functionality has only been performed on
the model.
Regression testing of v1 using LTP showed no regressions on the kernel
tests.
Regression testing of v2 is under way.
Series summary:
* Patches 1-5 contain some individual bits of preparatory spadework,
which are indirectly related to SVE.
Dave Martin (5):
regset: Add support for dynamically sized regsets
arm64: KVM: Hide unsupported AArch64 CPU features from guests
arm64: efi: Add missing Kconfig dependency on KERNEL_MODE_NEON
arm64: Port deprecated instruction emulation to new sysctl interface
arm64: fpsimd: Simplify uses of {set,clear}_ti_thread_flag()
Non-trivial changes among these are:
* Patch 1: updates the regset core code to handle regsets whose size
is not fixed at compile time. This avoids bloating coredumps even
though the maximum theoretical SVE regset size is large.
* Patch 2: extends KVM to modify the ARM architectural ID registers
seen by guests, by trapping and emulating certain registers. For
SVE this is a temporary measure, but it may be useful for other
architecture extensions. This patch may also be built on in the
future, since the only registers currently emulated are those
required for hiding SVE.
* Patches 6-10 add SVE-specific system register and structure layout
definitions, and the low-level boot code and accessors needed for
making use of SVE.
Dave Martin (5):
arm64/sve: System register and exception syndrome definitions
arm64/sve: Low-level SVE architectural state manipulation functions
arm64/sve: Kconfig update and conditional compilation support
arm64/sve: Signal frame and context structure definition
arm64/sve: Low-level CPU setup
* Patches 11-13 implement the core context management facilities to
provide each user task with its own SVE register context, signal
handling facilities, and sane programmer's model interoperation
between SVE and FPSIMD.
Dave Martin (3):
arm64/sve: Core task context handling
arm64/sve: Support vector length resetting for new processes
arm64/sve: Signal handling support
* Patches 14 and 16 provide backend logic for detecting and making use
of the different SVE vector lengths supported by the hardware.
* Patch 15 moves around code in cpufeatures.c to fit.
Dave Martin (3):
arm64/sve: Backend logic for setting the vector length
arm64: cpufeature: Move sys_caps_initialised declarations
arm64/sve: Probe SVE capabilities and usable vector lengths
* Patches 17-18 update the kernel-mode NEON / EFI FPSIMD frameworks to
interoperate correctly with SVE.
Dave Martin (2):
arm64/sve: Preserve SVE registers around kernel-mode NEON use
arm64/sve: Preserve SVE registers around EFI runtime service calls
* Patches 19-21 implement the userspace frontend for managing SVE,
comprising ptrace, some new arch-specific prctl() calls, and a new
sysctl for init-time setup.
Dave Martin (3):
arm64/sve: ptrace and ELF coredump support
arm64/sve: Add prctl controls for userspace vector length management
arm64/sve: Add sysctl to set the default vector length for new
processes
* Patches 22-24 provide stub KVM extensions for using KVM only on the
host, while denying guest access. (A future series will extend this
with full support for SVE in guests.)
Dave Martin (3):
arm64/sve: KVM: Prevent guests from using SVE
arm64/sve: KVM: Treat guest SVE use as undefined instruction
execution
arm64/sve: KVM: Hide SVE from CPU features exposed to guests
And finally:
* Patch 25 disengages the safety catch, enabling the kernel SVE runtime
support and allowing userspace to use SVE.
Dave Martin (1):
arm64/sve: Detect SVE and activate runtime support
* Patch 26 adds some basic documentation.
Dave Martin (1):
arm64/sve: Add documentation
* Patches 27-28 (which may be considered RFC) propose a mechanism to
report the maximum runtime signal frame size to userspace.
Dave Martin (2):
arm64: signal: Report signal frame size to userspace via auxv
arm64/sve: signal: Include SVE when computing AT_MINSIGSTKSZ
References:
[1] ARM Scalable Vector Extension
https://community.arm.com/groups/processors/blog/2016/08/22/technology-update-the-scalable-vector-extension-sve-for-the-armv8-a-architecture
[2] linux-arm-kernel August 2017 Archives by thread
http://lists.infradead.org/pipermail/linux-arm-kernel/2017-August/thread.html
[3] [PATCH 00/27] ARM Scalable Vector Extension (SVE)
http://lists.infradead.org/pipermail/linux-arm-kernel/2017-August/524691.html
[4] http://linux-arm.org/git?p=linux-dm.git;a=shortlog;h=refs/heads/sve/v2
git://linux-arm.org/linux-dm.git sve/v2
Full series and diffstat:
Dave Martin (28):
regset: Add support for dynamically sized regsets
arm64: KVM: Hide unsupported AArch64 CPU features from guests
arm64: efi: Add missing Kconfig dependency on KERNEL_MODE_NEON
arm64: Port deprecated instruction emulation to new sysctl interface
arm64: fpsimd: Simplify uses of {set,clear}_ti_thread_flag()
arm64/sve: System register and exception syndrome definitions
arm64/sve: Low-level SVE architectural state manipulation functions
arm64/sve: Kconfig update and conditional compilation support
arm64/sve: Signal frame and context structure definition
arm64/sve: Low-level CPU setup
arm64/sve: Core task context handling
arm64/sve: Support vector length resetting for new processes
arm64/sve: Signal handling support
arm64/sve: Backend logic for setting the vector length
arm64: cpufeature: Move sys_caps_initialised declarations
arm64/sve: Probe SVE capabilities and usable vector lengths
arm64/sve: Preserve SVE registers around kernel-mode NEON use
arm64/sve: Preserve SVE registers around EFI runtime service calls
arm64/sve: ptrace and ELF coredump support
arm64/sve: Add prctl controls for userspace vector length management
arm64/sve: Add sysctl to set the default vector length for new
processes
arm64/sve: KVM: Prevent guests from using SVE
arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
arm64/sve: KVM: Hide SVE from CPU features exposed to guests
arm64/sve: Detect SVE and activate runtime support
arm64/sve: Add documentation
arm64: signal: Report signal frame size to userspace via auxv
arm64/sve: signal: Include SVE when computing AT_MINSIGSTKSZ
Documentation/arm64/cpu-feature-registers.txt | 6 +-
Documentation/arm64/sve.txt | 477 +++++++++++++++++
arch/arm/include/asm/kvm_host.h | 3 +
arch/arm64/Kconfig | 12 +
arch/arm64/include/asm/cpu.h | 4 +
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/cpufeature.h | 35 ++
arch/arm64/include/asm/elf.h | 5 +
arch/arm64/include/asm/esr.h | 3 +-
arch/arm64/include/asm/fpsimd.h | 71 ++-
arch/arm64/include/asm/fpsimdmacros.h | 148 ++++++
arch/arm64/include/asm/kvm_arm.h | 5 +-
arch/arm64/include/asm/kvm_host.h | 11 +
arch/arm64/include/asm/processor.h | 10 +
arch/arm64/include/asm/sysreg.h | 24 +
arch/arm64/include/asm/thread_info.h | 2 +
arch/arm64/include/asm/traps.h | 2 +
arch/arm64/include/uapi/asm/auxvec.h | 3 +-
arch/arm64/include/uapi/asm/hwcap.h | 1 +
arch/arm64/include/uapi/asm/ptrace.h | 135 +++++
arch/arm64/include/uapi/asm/sigcontext.h | 120 ++++-
arch/arm64/kernel/armv8_deprecated.c | 15 +-
arch/arm64/kernel/cpufeature.c | 96 +++-
arch/arm64/kernel/cpuinfo.c | 7 +
arch/arm64/kernel/entry-fpsimd.S | 17 +
arch/arm64/kernel/entry.S | 14 +-
arch/arm64/kernel/fpsimd.c | 729 +++++++++++++++++++++++++-
arch/arm64/kernel/head.S | 13 +-
arch/arm64/kernel/process.c | 4 +
arch/arm64/kernel/ptrace.c | 270 +++++++++-
arch/arm64/kernel/signal.c | 222 +++++++-
arch/arm64/kernel/signal32.c | 2 +-
arch/arm64/kernel/traps.c | 5 +-
arch/arm64/kvm/handle_exit.c | 8 +
arch/arm64/kvm/hyp/switch.c | 12 +-
arch/arm64/kvm/sys_regs.c | 292 +++++++++--
arch/arm64/mm/proc.S | 14 +-
fs/binfmt_elf.c | 6 +-
include/linux/regset.h | 67 ++-
include/uapi/linux/elf.h | 1 +
include/uapi/linux/prctl.h | 9 +
kernel/sys.c | 12 +
virt/kvm/arm/arm.c | 3 +
43 files changed, 2753 insertions(+), 145 deletions(-)
create mode 100644 Documentation/arm64/sve.txt
--
2.1.4
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 00/28] ARM Scalable Vector Extension (SVE)
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This series implements Linux kernel support for the ARM Scalable Vector
Extension (SVE). [1] It supersedes the previous v1: see [3] for link.
See the individual patches for details of changes.
The patches apply on v4.13-rc7 + linux-arm64/for-next/core.
For convenience, a git tree is available. [4]
To reduce spam, some people may not been copied on the entire series.
For those who did not receive the whole series, it can be found in the
linux-arm-kernel archive. [2]
*Note* The final two patches (27-28) of the series are still RFC --
before committing to this ABI it would be good to get feedback on
whether the approach makes sense and whether it suitable for other
architectures. These two patches are not required by the rest of the
series and can be revised or merged later.
Support for use of SVE by KVM guests is not currently included.
Instead, such use will be trapped and reflected to the guest as
undefined instruction execution. SVE is hidden from the view of the
CPU feature registers visible to guests, so that guests will not
expect it to work.
This series has been build- and boot-tested on Juno r0 and the ARM FVP
Base model with SVE plugin. Because there is no hardware with SVE
support yet, testing of the SVE functionality has only been performed on
the model.
Regression testing of v1 using LTP showed no regressions on the kernel
tests.
Regression testing of v2 is under way.
Series summary:
* Patches 1-5 contain some individual bits of preparatory spadework,
which are indirectly related to SVE.
Dave Martin (5):
regset: Add support for dynamically sized regsets
arm64: KVM: Hide unsupported AArch64 CPU features from guests
arm64: efi: Add missing Kconfig dependency on KERNEL_MODE_NEON
arm64: Port deprecated instruction emulation to new sysctl interface
arm64: fpsimd: Simplify uses of {set,clear}_ti_thread_flag()
Non-trivial changes among these are:
* Patch 1: updates the regset core code to handle regsets whose size
is not fixed at compile time. This avoids bloating coredumps even
though the maximum theoretical SVE regset size is large.
* Patch 2: extends KVM to modify the ARM architectural ID registers
seen by guests, by trapping and emulating certain registers. For
SVE this is a temporary measure, but it may be useful for other
architecture extensions. This patch may also be built on in the
future, since the only registers currently emulated are those
required for hiding SVE.
* Patches 6-10 add SVE-specific system register and structure layout
definitions, and the low-level boot code and accessors needed for
making use of SVE.
Dave Martin (5):
arm64/sve: System register and exception syndrome definitions
arm64/sve: Low-level SVE architectural state manipulation functions
arm64/sve: Kconfig update and conditional compilation support
arm64/sve: Signal frame and context structure definition
arm64/sve: Low-level CPU setup
* Patches 11-13 implement the core context management facilities to
provide each user task with its own SVE register context, signal
handling facilities, and sane programmer's model interoperation
between SVE and FPSIMD.
Dave Martin (3):
arm64/sve: Core task context handling
arm64/sve: Support vector length resetting for new processes
arm64/sve: Signal handling support
* Patches 14 and 16 provide backend logic for detecting and making use
of the different SVE vector lengths supported by the hardware.
* Patch 15 moves around code in cpufeatures.c to fit.
Dave Martin (3):
arm64/sve: Backend logic for setting the vector length
arm64: cpufeature: Move sys_caps_initialised declarations
arm64/sve: Probe SVE capabilities and usable vector lengths
* Patches 17-18 update the kernel-mode NEON / EFI FPSIMD frameworks to
interoperate correctly with SVE.
Dave Martin (2):
arm64/sve: Preserve SVE registers around kernel-mode NEON use
arm64/sve: Preserve SVE registers around EFI runtime service calls
* Patches 19-21 implement the userspace frontend for managing SVE,
comprising ptrace, some new arch-specific prctl() calls, and a new
sysctl for init-time setup.
Dave Martin (3):
arm64/sve: ptrace and ELF coredump support
arm64/sve: Add prctl controls for userspace vector length management
arm64/sve: Add sysctl to set the default vector length for new
processes
* Patches 22-24 provide stub KVM extensions for using KVM only on the
host, while denying guest access. (A future series will extend this
with full support for SVE in guests.)
Dave Martin (3):
arm64/sve: KVM: Prevent guests from using SVE
arm64/sve: KVM: Treat guest SVE use as undefined instruction
execution
arm64/sve: KVM: Hide SVE from CPU features exposed to guests
And finally:
* Patch 25 disengages the safety catch, enabling the kernel SVE runtime
support and allowing userspace to use SVE.
Dave Martin (1):
arm64/sve: Detect SVE and activate runtime support
* Patch 26 adds some basic documentation.
Dave Martin (1):
arm64/sve: Add documentation
* Patches 27-28 (which may be considered RFC) propose a mechanism to
report the maximum runtime signal frame size to userspace.
Dave Martin (2):
arm64: signal: Report signal frame size to userspace via auxv
arm64/sve: signal: Include SVE when computing AT_MINSIGSTKSZ
References:
[1] ARM Scalable Vector Extension
https://community.arm.com/groups/processors/blog/2016/08/22/technology-update-the-scalable-vector-extension-sve-for-the-armv8-a-architecture
[2] linux-arm-kernel August 2017 Archives by thread
http://lists.infradead.org/pipermail/linux-arm-kernel/2017-August/thread.html
[3] [PATCH 00/27] ARM Scalable Vector Extension (SVE)
http://lists.infradead.org/pipermail/linux-arm-kernel/2017-August/524691.html
[4] http://linux-arm.org/git?p=linux-dm.git;a=shortlog;h=refs/heads/sve/v2
git://linux-arm.org/linux-dm.git sve/v2
Full series and diffstat:
Dave Martin (28):
regset: Add support for dynamically sized regsets
arm64: KVM: Hide unsupported AArch64 CPU features from guests
arm64: efi: Add missing Kconfig dependency on KERNEL_MODE_NEON
arm64: Port deprecated instruction emulation to new sysctl interface
arm64: fpsimd: Simplify uses of {set,clear}_ti_thread_flag()
arm64/sve: System register and exception syndrome definitions
arm64/sve: Low-level SVE architectural state manipulation functions
arm64/sve: Kconfig update and conditional compilation support
arm64/sve: Signal frame and context structure definition
arm64/sve: Low-level CPU setup
arm64/sve: Core task context handling
arm64/sve: Support vector length resetting for new processes
arm64/sve: Signal handling support
arm64/sve: Backend logic for setting the vector length
arm64: cpufeature: Move sys_caps_initialised declarations
arm64/sve: Probe SVE capabilities and usable vector lengths
arm64/sve: Preserve SVE registers around kernel-mode NEON use
arm64/sve: Preserve SVE registers around EFI runtime service calls
arm64/sve: ptrace and ELF coredump support
arm64/sve: Add prctl controls for userspace vector length management
arm64/sve: Add sysctl to set the default vector length for new
processes
arm64/sve: KVM: Prevent guests from using SVE
arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
arm64/sve: KVM: Hide SVE from CPU features exposed to guests
arm64/sve: Detect SVE and activate runtime support
arm64/sve: Add documentation
arm64: signal: Report signal frame size to userspace via auxv
arm64/sve: signal: Include SVE when computing AT_MINSIGSTKSZ
Documentation/arm64/cpu-feature-registers.txt | 6 +-
Documentation/arm64/sve.txt | 477 +++++++++++++++++
arch/arm/include/asm/kvm_host.h | 3 +
arch/arm64/Kconfig | 12 +
arch/arm64/include/asm/cpu.h | 4 +
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/cpufeature.h | 35 ++
arch/arm64/include/asm/elf.h | 5 +
arch/arm64/include/asm/esr.h | 3 +-
arch/arm64/include/asm/fpsimd.h | 71 ++-
arch/arm64/include/asm/fpsimdmacros.h | 148 ++++++
arch/arm64/include/asm/kvm_arm.h | 5 +-
arch/arm64/include/asm/kvm_host.h | 11 +
arch/arm64/include/asm/processor.h | 10 +
arch/arm64/include/asm/sysreg.h | 24 +
arch/arm64/include/asm/thread_info.h | 2 +
arch/arm64/include/asm/traps.h | 2 +
arch/arm64/include/uapi/asm/auxvec.h | 3 +-
arch/arm64/include/uapi/asm/hwcap.h | 1 +
arch/arm64/include/uapi/asm/ptrace.h | 135 +++++
arch/arm64/include/uapi/asm/sigcontext.h | 120 ++++-
arch/arm64/kernel/armv8_deprecated.c | 15 +-
arch/arm64/kernel/cpufeature.c | 96 +++-
arch/arm64/kernel/cpuinfo.c | 7 +
arch/arm64/kernel/entry-fpsimd.S | 17 +
arch/arm64/kernel/entry.S | 14 +-
arch/arm64/kernel/fpsimd.c | 729 +++++++++++++++++++++++++-
arch/arm64/kernel/head.S | 13 +-
arch/arm64/kernel/process.c | 4 +
arch/arm64/kernel/ptrace.c | 270 +++++++++-
arch/arm64/kernel/signal.c | 222 +++++++-
arch/arm64/kernel/signal32.c | 2 +-
arch/arm64/kernel/traps.c | 5 +-
arch/arm64/kvm/handle_exit.c | 8 +
arch/arm64/kvm/hyp/switch.c | 12 +-
arch/arm64/kvm/sys_regs.c | 292 +++++++++--
arch/arm64/mm/proc.S | 14 +-
fs/binfmt_elf.c | 6 +-
include/linux/regset.h | 67 ++-
include/uapi/linux/elf.h | 1 +
include/uapi/linux/prctl.h | 9 +
kernel/sys.c | 12 +
virt/kvm/arm/arm.c | 3 +
43 files changed, 2753 insertions(+), 145 deletions(-)
create mode 100644 Documentation/arm64/sve.txt
--
2.1.4
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 01/28] regset: Add support for dynamically sized regsets
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Oleg Nesterov, Alexander Viro
Currently the regset API doesn't allow for the possibility that
regsets (or at least, the amount of meaningful data in a regset)
may change in size.
In particular, this results in useless padding being added to
coredumps in a regset's current size is smaller than its
theoretical maximum size.
This patch adds a get_size() function to struct user_regset.
Individual regset implementations can implement this function to
return the current size of the regset data. A regset_size()
function is added to provide callers with an abstract interface for
determining the size of a regset without needing to know whether
the regset is dynamically sized or not.
The only affected user of this interface is the ELF coredump code:
This patch ports ELF coredump to dump regsets with their actual
size in the coredump. This has no effect except for new regsets
that are dynamically sized and provide a get_size() implementation.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
fs/binfmt_elf.c | 6 ++---
include/linux/regset.h | 67 ++++++++++++++++++++++++++++++++++++++++++++------
2 files changed, 63 insertions(+), 10 deletions(-)
diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
index 6466153..94ca285 100644
--- a/fs/binfmt_elf.c
+++ b/fs/binfmt_elf.c
@@ -1696,7 +1696,7 @@ static int fill_thread_core_info(struct elf_thread_core_info *t,
long signr, size_t *total)
{
unsigned int i;
- unsigned int regset_size = view->regsets[0].n * view->regsets[0].size;
+ unsigned int size = regset_size(t->task, &view->regsets[0]);
/*
* NT_PRSTATUS is the one special case, because the regset data
@@ -1705,7 +1705,7 @@ static int fill_thread_core_info(struct elf_thread_core_info *t,
* We assume that regset 0 is NT_PRSTATUS.
*/
fill_prstatus(&t->prstatus, t->task, signr);
- (void) view->regsets[0].get(t->task, &view->regsets[0], 0, regset_size,
+ (void) view->regsets[0].get(t->task, &view->regsets[0], 0, size,
&t->prstatus.pr_reg, NULL);
fill_note(&t->notes[0], "CORE", NT_PRSTATUS,
@@ -1725,7 +1725,7 @@ static int fill_thread_core_info(struct elf_thread_core_info *t,
if (regset->core_note_type && regset->get &&
(!regset->active || regset->active(t->task, regset))) {
int ret;
- size_t size = regset->n * regset->size;
+ size_t size = regset_size(t->task, regset);
void *data = kmalloc(size, GFP_KERNEL);
if (unlikely(!data))
return 0;
diff --git a/include/linux/regset.h b/include/linux/regset.h
index 8e0c9fe..494ceda 100644
--- a/include/linux/regset.h
+++ b/include/linux/regset.h
@@ -107,6 +107,28 @@ typedef int user_regset_writeback_fn(struct task_struct *target,
int immediate);
/**
+ * user_regset_get_size_fn - type of @get_size function in &struct user_regset
+ * @target: thread being examined
+ * @regset: regset being examined
+ *
+ * This call is optional; usually the pointer is %NULL.
+ *
+ * When provided, this function must return the current size of regset
+ * data, as observed by the @get function in &struct user_regset. The
+ * value returned must be a multiple of @size. The returned size is
+ * required to be valid only until the next time (if any) @regset is
+ * modified for @target.
+ *
+ * This function is intended for dynamically sized regsets. A regset
+ * that is statically sized does not need to implement it.
+ *
+ * This function should not be called directly: instead, callers should
+ * call regset_size() to determine the current size of a regset.
+ */
+typedef unsigned int user_regset_get_size_fn(struct task_struct *target,
+ const struct user_regset *regset);
+
+/**
* struct user_regset - accessible thread CPU state
* @n: Number of slots (registers).
* @size: Size in bytes of a slot (register).
@@ -117,19 +139,33 @@ typedef int user_regset_writeback_fn(struct task_struct *target,
* @set: Function to store values.
* @active: Function to report if regset is active, or %NULL.
* @writeback: Function to write data back to user memory, or %NULL.
+ * @get_size: Function to return the regset's size, or %NULL.
*
* This data structure describes a machine resource we call a register set.
* This is part of the state of an individual thread, not necessarily
* actual CPU registers per se. A register set consists of a number of
* similar slots, given by @n. Each slot is @size bytes, and aligned to
- * @align bytes (which is at least @size).
+ * @align bytes (which is at least @size). For dynamically-sized
+ * regsets, @n must contain the maximum possible number of slots for the
+ * regset, and @get_size must point to a function that returns the
+ * current regset size.
*
- * These functions must be called only on the current thread or on a
- * thread that is in %TASK_STOPPED or %TASK_TRACED state, that we are
- * guaranteed will not be woken up and return to user mode, and that we
- * have called wait_task_inactive() on. (The target thread always might
- * wake up for SIGKILL while these functions are working, in which case
- * that thread's user_regset state might be scrambled.)
+ * Callers that need to know only the current size of the regset and do
+ * not care about its internal structure should call regset_size()
+ * instead of inspecting @n or calling @get_size.
+ *
+ * For backward compatibility, the @get and @set methods must pad to, or
+ * accept, @n * @size bytes, even if the current regset size is smaller.
+ * The precise semantics of these operations depend on the regset being
+ * accessed.
+ *
+ * The functions to which &struct user_regset members point must be
+ * called only on the current thread or on a thread that is in
+ * %TASK_STOPPED or %TASK_TRACED state, that we are guaranteed will not
+ * be woken up and return to user mode, and that we have called
+ * wait_task_inactive() on. (The target thread always might wake up for
+ * SIGKILL while these functions are working, in which case that
+ * thread's user_regset state might be scrambled.)
*
* The @pos argument must be aligned according to @align; the @count
* argument must be a multiple of @size. These functions are not
@@ -156,6 +192,7 @@ struct user_regset {
user_regset_set_fn *set;
user_regset_active_fn *active;
user_regset_writeback_fn *writeback;
+ user_regset_get_size_fn *get_size;
unsigned int n;
unsigned int size;
unsigned int align;
@@ -371,5 +408,21 @@ static inline int copy_regset_from_user(struct task_struct *target,
return regset->set(target, regset, offset, size, NULL, data);
}
+/**
+ * regset_size - determine the current size of a regset
+ * @target: thread to be examined
+ * @regset: regset to be examined
+ *
+ * Note that the returned size is valid only until the next time
+ * (if any) @regset is modified for @target.
+ */
+static inline unsigned int regset_size(struct task_struct *target,
+ const struct user_regset *regset)
+{
+ if (!regset->get_size)
+ return regset->n * regset->size;
+ else
+ return regset->get_size(target, regset);
+}
#endif /* <linux/regset.h> */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 01/28] regset: Add support for dynamically sized regsets
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Currently the regset API doesn't allow for the possibility that
regsets (or at least, the amount of meaningful data in a regset)
may change in size.
In particular, this results in useless padding being added to
coredumps in a regset's current size is smaller than its
theoretical maximum size.
This patch adds a get_size() function to struct user_regset.
Individual regset implementations can implement this function to
return the current size of the regset data. A regset_size()
function is added to provide callers with an abstract interface for
determining the size of a regset without needing to know whether
the regset is dynamically sized or not.
The only affected user of this interface is the ELF coredump code:
This patch ports ELF coredump to dump regsets with their actual
size in the coredump. This has no effect except for new regsets
that are dynamically sized and provide a get_size() implementation.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
---
fs/binfmt_elf.c | 6 ++---
include/linux/regset.h | 67 ++++++++++++++++++++++++++++++++++++++++++++------
2 files changed, 63 insertions(+), 10 deletions(-)
diff --git a/fs/binfmt_elf.c b/fs/binfmt_elf.c
index 6466153..94ca285 100644
--- a/fs/binfmt_elf.c
+++ b/fs/binfmt_elf.c
@@ -1696,7 +1696,7 @@ static int fill_thread_core_info(struct elf_thread_core_info *t,
long signr, size_t *total)
{
unsigned int i;
- unsigned int regset_size = view->regsets[0].n * view->regsets[0].size;
+ unsigned int size = regset_size(t->task, &view->regsets[0]);
/*
* NT_PRSTATUS is the one special case, because the regset data
@@ -1705,7 +1705,7 @@ static int fill_thread_core_info(struct elf_thread_core_info *t,
* We assume that regset 0 is NT_PRSTATUS.
*/
fill_prstatus(&t->prstatus, t->task, signr);
- (void) view->regsets[0].get(t->task, &view->regsets[0], 0, regset_size,
+ (void) view->regsets[0].get(t->task, &view->regsets[0], 0, size,
&t->prstatus.pr_reg, NULL);
fill_note(&t->notes[0], "CORE", NT_PRSTATUS,
@@ -1725,7 +1725,7 @@ static int fill_thread_core_info(struct elf_thread_core_info *t,
if (regset->core_note_type && regset->get &&
(!regset->active || regset->active(t->task, regset))) {
int ret;
- size_t size = regset->n * regset->size;
+ size_t size = regset_size(t->task, regset);
void *data = kmalloc(size, GFP_KERNEL);
if (unlikely(!data))
return 0;
diff --git a/include/linux/regset.h b/include/linux/regset.h
index 8e0c9fe..494ceda 100644
--- a/include/linux/regset.h
+++ b/include/linux/regset.h
@@ -107,6 +107,28 @@ typedef int user_regset_writeback_fn(struct task_struct *target,
int immediate);
/**
+ * user_regset_get_size_fn - type of @get_size function in &struct user_regset
+ * @target: thread being examined
+ * @regset: regset being examined
+ *
+ * This call is optional; usually the pointer is %NULL.
+ *
+ * When provided, this function must return the current size of regset
+ * data, as observed by the @get function in &struct user_regset. The
+ * value returned must be a multiple of @size. The returned size is
+ * required to be valid only until the next time (if any) @regset is
+ * modified for @target.
+ *
+ * This function is intended for dynamically sized regsets. A regset
+ * that is statically sized does not need to implement it.
+ *
+ * This function should not be called directly: instead, callers should
+ * call regset_size() to determine the current size of a regset.
+ */
+typedef unsigned int user_regset_get_size_fn(struct task_struct *target,
+ const struct user_regset *regset);
+
+/**
* struct user_regset - accessible thread CPU state
* @n: Number of slots (registers).
* @size: Size in bytes of a slot (register).
@@ -117,19 +139,33 @@ typedef int user_regset_writeback_fn(struct task_struct *target,
* @set: Function to store values.
* @active: Function to report if regset is active, or %NULL.
* @writeback: Function to write data back to user memory, or %NULL.
+ * @get_size: Function to return the regset's size, or %NULL.
*
* This data structure describes a machine resource we call a register set.
* This is part of the state of an individual thread, not necessarily
* actual CPU registers per se. A register set consists of a number of
* similar slots, given by @n. Each slot is @size bytes, and aligned to
- * @align bytes (which is at least @size).
+ * @align bytes (which is at least @size). For dynamically-sized
+ * regsets, @n must contain the maximum possible number of slots for the
+ * regset, and @get_size must point to a function that returns the
+ * current regset size.
*
- * These functions must be called only on the current thread or on a
- * thread that is in %TASK_STOPPED or %TASK_TRACED state, that we are
- * guaranteed will not be woken up and return to user mode, and that we
- * have called wait_task_inactive() on. (The target thread always might
- * wake up for SIGKILL while these functions are working, in which case
- * that thread's user_regset state might be scrambled.)
+ * Callers that need to know only the current size of the regset and do
+ * not care about its internal structure should call regset_size()
+ * instead of inspecting @n or calling @get_size.
+ *
+ * For backward compatibility, the @get and @set methods must pad to, or
+ * accept, @n * @size bytes, even if the current regset size is smaller.
+ * The precise semantics of these operations depend on the regset being
+ * accessed.
+ *
+ * The functions to which &struct user_regset members point must be
+ * called only on the current thread or on a thread that is in
+ * %TASK_STOPPED or %TASK_TRACED state, that we are guaranteed will not
+ * be woken up and return to user mode, and that we have called
+ * wait_task_inactive() on. (The target thread always might wake up for
+ * SIGKILL while these functions are working, in which case that
+ * thread's user_regset state might be scrambled.)
*
* The @pos argument must be aligned according to @align; the @count
* argument must be a multiple of @size. These functions are not
@@ -156,6 +192,7 @@ struct user_regset {
user_regset_set_fn *set;
user_regset_active_fn *active;
user_regset_writeback_fn *writeback;
+ user_regset_get_size_fn *get_size;
unsigned int n;
unsigned int size;
unsigned int align;
@@ -371,5 +408,21 @@ static inline int copy_regset_from_user(struct task_struct *target,
return regset->set(target, regset, offset, size, NULL, data);
}
+/**
+ * regset_size - determine the current size of a regset
+ * @target: thread to be examined
+ * @regset: regset to be examined
+ *
+ * Note that the returned size is valid only until the next time
+ * (if any) @regset is modified for @target.
+ */
+static inline unsigned int regset_size(struct task_struct *target,
+ const struct user_regset *regset)
+{
+ if (!regset->get_size)
+ return regset->n * regset->size;
+ else
+ return regset->get_size(target, regset);
+}
#endif /* <linux/regset.h> */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Currently, a guest kernel sees the true CPU feature registers
(ID_*_EL1) when it reads them using MRS instructions. This means
that the guest will observe features that are present in the
hardware but the host doesn't understand or doesn't provide support
for. A guest may legimitately try to use such a feature as per the
architecture, but use of the feature may trap instead of working
normally, triggering undef injection into the guest.
This is not a problem for the host, but the guest may go wrong when
running on newer hardware than the host knows about.
This patch hides from guest VMs any AArch64-specific CPU features
that the host doesn't support, by exposing to the guest the
sanitised versions of the registers computed by the cpufeatures
framework, instead of the true hardware registers. To achieve
this, HCR_EL2.TID3 is now set for AArch64 guests, and emulation
code is added to KVM to report the sanitised versions of the
affected registers in response to MRS and register reads from
userspace.
The affected registers are removed from invariant_sys_regs[] (since
the invariant_sys_regs handling is no longer quite correct for
them) and added to sys_reg_desgs[], with appropriate access(),
get_user() and set_user() methods. No runtime vcpu storage is
allocated for the registers: instead, they are read on demand from
the cpufeatures framework. This may need modification in the
future if there is a need for userspace to customise the features
visible to the guest.
Attempts by userspace to write the registers are handled similarly
to the current invariant_sys_regs handling: writes are permitted,
but only if they don't attempt to change the value. This is
sufficient to support VM snapshot/restore from userspace.
Because of the additional registers, restoring a VM on an older
kernel may not work unless userspace knows how to handle the extra
VM registers exposed to the KVM user ABI by this patch.
Under the principle of least damage, this patch makes no attempt to
handle any of the other registers currently in
invariant_sys_regs[], or to emulate registers for AArch32: however,
these could be handled in a similar way in future, as necessary.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
Changes since v1
----------------
Requested by Marc Zyngier:
* Get rid of ternary operator use in walk_sys_regs().
* Call write_to_read_only() if an attempt to write an ID reg is
trapped, rather than reinventing.
Probably we won't get there anyway: the architecture says that this
should undef at EL1 instead.
* Make ID register sysreg table less cryptic and spread the entries one
per line.
Also, make the architecturally unallocated and allocated but hidden
cases more clearly distinct. These require the same behaviour but for
different reasons, so it's better to identify them as separate.
Other:
* Delete BUG_ON()s that are skipped by construction:
These check that the result of sys_reg_to_index() is a 64-bit
register, which is always true because sys_reg_to_index()
explicitly sets this.
* Remove duplicate const in __access_id_reg args [sparse]
---
arch/arm64/include/asm/sysreg.h | 3 +
arch/arm64/kvm/hyp/switch.c | 6 +
arch/arm64/kvm/sys_regs.c | 282 +++++++++++++++++++++++++++++++++-------
3 files changed, 246 insertions(+), 45 deletions(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index f707fed..480ecd6 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -149,6 +149,9 @@
#define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
#define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
+#define SYS_ID_AA64AFR0_EL1 sys_reg(3, 0, 0, 5, 4)
+#define SYS_ID_AA64AFR1_EL1 sys_reg(3, 0, 0, 5, 5)
+
#define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)
#define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 945e79c..35a90b8 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -81,11 +81,17 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
* it will cause an exception.
*/
val = vcpu->arch.hcr_el2;
+
if (!(val & HCR_RW) && system_supports_fpsimd()) {
write_sysreg(1 << 30, fpexc32_el2);
isb();
}
+
+ if (val & HCR_RW) /* for AArch64 only: */
+ val |= HCR_TID3; /* TID3: trap feature register accesses */
+
write_sysreg(val, hcr_el2);
+
/* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
write_sysreg(1 << 15, hstr_el2);
/*
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 2e070d3..b1f7552 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -892,6 +892,137 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
return true;
}
+/* Read a sanitised cpufeature ID register by sys_reg_desc */
+static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+{
+ u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
+ (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
+
+ return raz ? 0 : read_sanitised_ftr_reg(id);
+}
+
+/* cpufeature ID register access trap handlers */
+
+static bool __access_id_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r,
+ bool raz)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = read_id_reg(r, raz);
+ return true;
+}
+
+static bool access_id_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return __access_id_reg(vcpu, p, r, false);
+}
+
+static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return __access_id_reg(vcpu, p, r, true);
+}
+
+static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
+static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
+static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
+
+/*
+ * cpufeature ID register user accessors
+ *
+ * For now, these registers are immutable for userspace, so no values
+ * are stored, and for set_id_reg() we don't allow the effective value
+ * to be changed.
+ */
+static int __get_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
+ bool raz)
+{
+ const u64 id = sys_reg_to_index(rd);
+ const u64 val = read_id_reg(rd, raz);
+
+ return reg_to_user(uaddr, &val, id);
+}
+
+static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
+ bool raz)
+{
+ const u64 id = sys_reg_to_index(rd);
+ int err;
+ u64 val;
+
+ err = reg_from_user(&val, uaddr, id);
+ if (err)
+ return err;
+
+ /* This is what we mean by invariant: you can't change it. */
+ if (val != read_id_reg(rd, raz))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __get_id_reg(rd, uaddr, false);
+}
+
+static int set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __set_id_reg(rd, uaddr, false);
+}
+
+static int get_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __get_id_reg(rd, uaddr, true);
+}
+
+static int set_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __set_id_reg(rd, uaddr, true);
+}
+
+/* sys_reg_desc initialiser for known cpufeature ID registers */
+#define ID_SANITISED(name) { \
+ SYS_DESC(SYS_##name), \
+ .access = access_id_reg, \
+ .get_user = get_id_reg, \
+ .set_user = set_id_reg, \
+}
+
+/*
+ * sys_reg_desc initialiser for architecturally unallocated cpufeature ID
+ * register with encoding Op0=3, Op1=0, CRn=0, CRm=crm, Op2=op2
+ * (1 <= crm < 8, 0 <= Op2 < 8).
+ */
+#define ID_UNALLOCATED(crm, op2) { \
+ Op0(3), Op1(0), CRn(0), CRm(crm), Op2(op2), \
+ .access = access_raz_id_reg, \
+ .get_user = get_raz_id_reg, \
+ .set_user = set_raz_id_reg, \
+}
+
+/*
+ * sys_reg_desc initialiser for known ID registers that we hide from guests.
+ * For now, these are exposed just like unallocated ID regs: they appear
+ * RAZ for the guest.
+ */
+#define ID_HIDDEN(name) { \
+ SYS_DESC(SYS_##name), \
+ .access = access_raz_id_reg, \
+ .get_user = get_raz_id_reg, \
+ .set_user = set_raz_id_reg, \
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -944,6 +1075,84 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_DBGVCR32_EL2), NULL, reset_val, DBGVCR32_EL2, 0 },
{ SYS_DESC(SYS_MPIDR_EL1), NULL, reset_mpidr, MPIDR_EL1 },
+
+ /*
+ * ID regs: all ID_SANITISED() entries here must have corresponding
+ * entries in arm64_ftr_regs[].
+ */
+
+ /* AArch64 mappings of the AArch32 ID registers */
+ /* CRm=1 */
+ ID_SANITISED(ID_PFR0_EL1),
+ ID_SANITISED(ID_PFR1_EL1),
+ ID_SANITISED(ID_DFR0_EL1),
+ ID_HIDDEN(ID_AFR0_EL1),
+ ID_SANITISED(ID_MMFR0_EL1),
+ ID_SANITISED(ID_MMFR1_EL1),
+ ID_SANITISED(ID_MMFR2_EL1),
+ ID_SANITISED(ID_MMFR3_EL1),
+
+ /* CRm=2 */
+ ID_SANITISED(ID_ISAR0_EL1),
+ ID_SANITISED(ID_ISAR1_EL1),
+ ID_SANITISED(ID_ISAR2_EL1),
+ ID_SANITISED(ID_ISAR3_EL1),
+ ID_SANITISED(ID_ISAR4_EL1),
+ ID_SANITISED(ID_ISAR5_EL1),
+ ID_SANITISED(ID_MMFR4_EL1),
+ ID_UNALLOCATED(2,7),
+
+ /* CRm=3 */
+ ID_SANITISED(MVFR0_EL1),
+ ID_SANITISED(MVFR1_EL1),
+ ID_SANITISED(MVFR2_EL1),
+ ID_UNALLOCATED(3,3),
+ ID_UNALLOCATED(3,4),
+ ID_UNALLOCATED(3,5),
+ ID_UNALLOCATED(3,6),
+ ID_UNALLOCATED(3,7),
+
+ /* AArch64 ID registers */
+ /* CRm=4 */
+ ID_SANITISED(ID_AA64PFR0_EL1),
+ ID_SANITISED(ID_AA64PFR1_EL1),
+ ID_UNALLOCATED(4,2),
+ ID_UNALLOCATED(4,3),
+ ID_UNALLOCATED(4,4),
+ ID_UNALLOCATED(4,5),
+ ID_UNALLOCATED(4,6),
+ ID_UNALLOCATED(4,7),
+
+ /* CRm=5 */
+ ID_SANITISED(ID_AA64DFR0_EL1),
+ ID_SANITISED(ID_AA64DFR1_EL1),
+ ID_UNALLOCATED(5,2),
+ ID_UNALLOCATED(5,3),
+ ID_HIDDEN(ID_AA64AFR0_EL1),
+ ID_HIDDEN(ID_AA64AFR1_EL1),
+ ID_UNALLOCATED(5,6),
+ ID_UNALLOCATED(5,7),
+
+ /* CRm=6 */
+ ID_SANITISED(ID_AA64ISAR0_EL1),
+ ID_SANITISED(ID_AA64ISAR1_EL1),
+ ID_UNALLOCATED(6,2),
+ ID_UNALLOCATED(6,3),
+ ID_UNALLOCATED(6,4),
+ ID_UNALLOCATED(6,5),
+ ID_UNALLOCATED(6,6),
+ ID_UNALLOCATED(6,7),
+
+ /* CRm=7 */
+ ID_SANITISED(ID_AA64MMFR0_EL1),
+ ID_SANITISED(ID_AA64MMFR1_EL1),
+ ID_SANITISED(ID_AA64MMFR2_EL1),
+ ID_UNALLOCATED(7,3),
+ ID_UNALLOCATED(7,4),
+ ID_UNALLOCATED(7,5),
+ ID_UNALLOCATED(7,6),
+ ID_UNALLOCATED(7,7),
+
{ SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
@@ -1790,8 +1999,8 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
if (!r)
r = find_reg(¶ms, sys_reg_descs, ARRAY_SIZE(sys_reg_descs));
- /* Not saved in the sys_reg array? */
- if (r && !r->reg)
+ /* Not saved in the sys_reg array and not otherwise accessible? */
+ if (r && !(r->reg || r->get_user))
r = NULL;
return r;
@@ -1815,20 +2024,6 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
FUNCTION_INVARIANT(midr_el1)
FUNCTION_INVARIANT(ctr_el0)
FUNCTION_INVARIANT(revidr_el1)
-FUNCTION_INVARIANT(id_pfr0_el1)
-FUNCTION_INVARIANT(id_pfr1_el1)
-FUNCTION_INVARIANT(id_dfr0_el1)
-FUNCTION_INVARIANT(id_afr0_el1)
-FUNCTION_INVARIANT(id_mmfr0_el1)
-FUNCTION_INVARIANT(id_mmfr1_el1)
-FUNCTION_INVARIANT(id_mmfr2_el1)
-FUNCTION_INVARIANT(id_mmfr3_el1)
-FUNCTION_INVARIANT(id_isar0_el1)
-FUNCTION_INVARIANT(id_isar1_el1)
-FUNCTION_INVARIANT(id_isar2_el1)
-FUNCTION_INVARIANT(id_isar3_el1)
-FUNCTION_INVARIANT(id_isar4_el1)
-FUNCTION_INVARIANT(id_isar5_el1)
FUNCTION_INVARIANT(clidr_el1)
FUNCTION_INVARIANT(aidr_el1)
@@ -1836,20 +2031,6 @@ FUNCTION_INVARIANT(aidr_el1)
static struct sys_reg_desc invariant_sys_regs[] = {
{ SYS_DESC(SYS_MIDR_EL1), NULL, get_midr_el1 },
{ SYS_DESC(SYS_REVIDR_EL1), NULL, get_revidr_el1 },
- { SYS_DESC(SYS_ID_PFR0_EL1), NULL, get_id_pfr0_el1 },
- { SYS_DESC(SYS_ID_PFR1_EL1), NULL, get_id_pfr1_el1 },
- { SYS_DESC(SYS_ID_DFR0_EL1), NULL, get_id_dfr0_el1 },
- { SYS_DESC(SYS_ID_AFR0_EL1), NULL, get_id_afr0_el1 },
- { SYS_DESC(SYS_ID_MMFR0_EL1), NULL, get_id_mmfr0_el1 },
- { SYS_DESC(SYS_ID_MMFR1_EL1), NULL, get_id_mmfr1_el1 },
- { SYS_DESC(SYS_ID_MMFR2_EL1), NULL, get_id_mmfr2_el1 },
- { SYS_DESC(SYS_ID_MMFR3_EL1), NULL, get_id_mmfr3_el1 },
- { SYS_DESC(SYS_ID_ISAR0_EL1), NULL, get_id_isar0_el1 },
- { SYS_DESC(SYS_ID_ISAR1_EL1), NULL, get_id_isar1_el1 },
- { SYS_DESC(SYS_ID_ISAR2_EL1), NULL, get_id_isar2_el1 },
- { SYS_DESC(SYS_ID_ISAR3_EL1), NULL, get_id_isar3_el1 },
- { SYS_DESC(SYS_ID_ISAR4_EL1), NULL, get_id_isar4_el1 },
- { SYS_DESC(SYS_ID_ISAR5_EL1), NULL, get_id_isar5_el1 },
{ SYS_DESC(SYS_CLIDR_EL1), NULL, get_clidr_el1 },
{ SYS_DESC(SYS_AIDR_EL1), NULL, get_aidr_el1 },
{ SYS_DESC(SYS_CTR_EL0), NULL, get_ctr_el0 },
@@ -2079,12 +2260,31 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
return true;
}
+static int walk_one_sys_reg(const struct sys_reg_desc *rd,
+ u64 __user **uind,
+ unsigned int *total)
+{
+ /*
+ * Ignore registers we trap but don't save,
+ * and for which no custom user accessor is provided.
+ */
+ if (!(rd->reg || rd->get_user))
+ return 0;
+
+ if (!copy_reg_to_user(rd, uind))
+ return -EFAULT;
+
+ (*total)++;
+ return 0;
+}
+
/* Assumed ordered tables, see kvm_sys_reg_table_init. */
static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
{
const struct sys_reg_desc *i1, *i2, *end1, *end2;
unsigned int total = 0;
size_t num;
+ int err;
/* We check for duplicates here, to allow arch-specific overrides. */
i1 = get_target_table(vcpu->arch.target, true, &num);
@@ -2098,21 +2298,13 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
while (i1 || i2) {
int cmp = cmp_sys_reg(i1, i2);
/* target-specific overrides generic entry. */
- if (cmp <= 0) {
- /* Ignore registers we trap but don't save. */
- if (i1->reg) {
- if (!copy_reg_to_user(i1, &uind))
- return -EFAULT;
- total++;
- }
- } else {
- /* Ignore registers we trap but don't save. */
- if (i2->reg) {
- if (!copy_reg_to_user(i2, &uind))
- return -EFAULT;
- total++;
- }
- }
+ if (cmp <= 0)
+ err = walk_one_sys_reg(i1, &uind, &total);
+ else
+ err = walk_one_sys_reg(i2, &uind, &total);
+
+ if (err)
+ return err;
if (cmp <= 0 && ++i1 == end1)
i1 = NULL;
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Currently, a guest kernel sees the true CPU feature registers
(ID_*_EL1) when it reads them using MRS instructions. This means
that the guest will observe features that are present in the
hardware but the host doesn't understand or doesn't provide support
for. A guest may legimitately try to use such a feature as per the
architecture, but use of the feature may trap instead of working
normally, triggering undef injection into the guest.
This is not a problem for the host, but the guest may go wrong when
running on newer hardware than the host knows about.
This patch hides from guest VMs any AArch64-specific CPU features
that the host doesn't support, by exposing to the guest the
sanitised versions of the registers computed by the cpufeatures
framework, instead of the true hardware registers. To achieve
this, HCR_EL2.TID3 is now set for AArch64 guests, and emulation
code is added to KVM to report the sanitised versions of the
affected registers in response to MRS and register reads from
userspace.
The affected registers are removed from invariant_sys_regs[] (since
the invariant_sys_regs handling is no longer quite correct for
them) and added to sys_reg_desgs[], with appropriate access(),
get_user() and set_user() methods. No runtime vcpu storage is
allocated for the registers: instead, they are read on demand from
the cpufeatures framework. This may need modification in the
future if there is a need for userspace to customise the features
visible to the guest.
Attempts by userspace to write the registers are handled similarly
to the current invariant_sys_regs handling: writes are permitted,
but only if they don't attempt to change the value. This is
sufficient to support VM snapshot/restore from userspace.
Because of the additional registers, restoring a VM on an older
kernel may not work unless userspace knows how to handle the extra
VM registers exposed to the KVM user ABI by this patch.
Under the principle of least damage, this patch makes no attempt to
handle any of the other registers currently in
invariant_sys_regs[], or to emulate registers for AArch32: however,
these could be handled in a similar way in future, as necessary.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
Changes since v1
----------------
Requested by Marc Zyngier:
* Get rid of ternary operator use in walk_sys_regs().
* Call write_to_read_only() if an attempt to write an ID reg is
trapped, rather than reinventing.
Probably we won't get there anyway: the architecture says that this
should undef at EL1 instead.
* Make ID register sysreg table less cryptic and spread the entries one
per line.
Also, make the architecturally unallocated and allocated but hidden
cases more clearly distinct. These require the same behaviour but for
different reasons, so it's better to identify them as separate.
Other:
* Delete BUG_ON()s that are skipped by construction:
These check that the result of sys_reg_to_index() is a 64-bit
register, which is always true because sys_reg_to_index()
explicitly sets this.
* Remove duplicate const in __access_id_reg args [sparse]
---
arch/arm64/include/asm/sysreg.h | 3 +
arch/arm64/kvm/hyp/switch.c | 6 +
arch/arm64/kvm/sys_regs.c | 282 +++++++++++++++++++++++++++++++++-------
3 files changed, 246 insertions(+), 45 deletions(-)
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index f707fed..480ecd6 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -149,6 +149,9 @@
#define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
#define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
+#define SYS_ID_AA64AFR0_EL1 sys_reg(3, 0, 0, 5, 4)
+#define SYS_ID_AA64AFR1_EL1 sys_reg(3, 0, 0, 5, 5)
+
#define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)
#define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 945e79c..35a90b8 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -81,11 +81,17 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
* it will cause an exception.
*/
val = vcpu->arch.hcr_el2;
+
if (!(val & HCR_RW) && system_supports_fpsimd()) {
write_sysreg(1 << 30, fpexc32_el2);
isb();
}
+
+ if (val & HCR_RW) /* for AArch64 only: */
+ val |= HCR_TID3; /* TID3: trap feature register accesses */
+
write_sysreg(val, hcr_el2);
+
/* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
write_sysreg(1 << 15, hstr_el2);
/*
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index 2e070d3..b1f7552 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -892,6 +892,137 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
return true;
}
+/* Read a sanitised cpufeature ID register by sys_reg_desc */
+static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
+{
+ u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
+ (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
+
+ return raz ? 0 : read_sanitised_ftr_reg(id);
+}
+
+/* cpufeature ID register access trap handlers */
+
+static bool __access_id_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r,
+ bool raz)
+{
+ if (p->is_write)
+ return write_to_read_only(vcpu, p, r);
+
+ p->regval = read_id_reg(r, raz);
+ return true;
+}
+
+static bool access_id_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return __access_id_reg(vcpu, p, r, false);
+}
+
+static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
+ struct sys_reg_params *p,
+ const struct sys_reg_desc *r)
+{
+ return __access_id_reg(vcpu, p, r, true);
+}
+
+static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
+static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
+static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
+
+/*
+ * cpufeature ID register user accessors
+ *
+ * For now, these registers are immutable for userspace, so no values
+ * are stored, and for set_id_reg() we don't allow the effective value
+ * to be changed.
+ */
+static int __get_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
+ bool raz)
+{
+ const u64 id = sys_reg_to_index(rd);
+ const u64 val = read_id_reg(rd, raz);
+
+ return reg_to_user(uaddr, &val, id);
+}
+
+static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
+ bool raz)
+{
+ const u64 id = sys_reg_to_index(rd);
+ int err;
+ u64 val;
+
+ err = reg_from_user(&val, uaddr, id);
+ if (err)
+ return err;
+
+ /* This is what we mean by invariant: you can't change it. */
+ if (val != read_id_reg(rd, raz))
+ return -EINVAL;
+
+ return 0;
+}
+
+static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __get_id_reg(rd, uaddr, false);
+}
+
+static int set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __set_id_reg(rd, uaddr, false);
+}
+
+static int get_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __get_id_reg(rd, uaddr, true);
+}
+
+static int set_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
+ const struct kvm_one_reg *reg, void __user *uaddr)
+{
+ return __set_id_reg(rd, uaddr, true);
+}
+
+/* sys_reg_desc initialiser for known cpufeature ID registers */
+#define ID_SANITISED(name) { \
+ SYS_DESC(SYS_##name), \
+ .access = access_id_reg, \
+ .get_user = get_id_reg, \
+ .set_user = set_id_reg, \
+}
+
+/*
+ * sys_reg_desc initialiser for architecturally unallocated cpufeature ID
+ * register with encoding Op0=3, Op1=0, CRn=0, CRm=crm, Op2=op2
+ * (1 <= crm < 8, 0 <= Op2 < 8).
+ */
+#define ID_UNALLOCATED(crm, op2) { \
+ Op0(3), Op1(0), CRn(0), CRm(crm), Op2(op2), \
+ .access = access_raz_id_reg, \
+ .get_user = get_raz_id_reg, \
+ .set_user = set_raz_id_reg, \
+}
+
+/*
+ * sys_reg_desc initialiser for known ID registers that we hide from guests.
+ * For now, these are exposed just like unallocated ID regs: they appear
+ * RAZ for the guest.
+ */
+#define ID_HIDDEN(name) { \
+ SYS_DESC(SYS_##name), \
+ .access = access_raz_id_reg, \
+ .get_user = get_raz_id_reg, \
+ .set_user = set_raz_id_reg, \
+}
+
/*
* Architected system registers.
* Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
@@ -944,6 +1075,84 @@ static const struct sys_reg_desc sys_reg_descs[] = {
{ SYS_DESC(SYS_DBGVCR32_EL2), NULL, reset_val, DBGVCR32_EL2, 0 },
{ SYS_DESC(SYS_MPIDR_EL1), NULL, reset_mpidr, MPIDR_EL1 },
+
+ /*
+ * ID regs: all ID_SANITISED() entries here must have corresponding
+ * entries in arm64_ftr_regs[].
+ */
+
+ /* AArch64 mappings of the AArch32 ID registers */
+ /* CRm=1 */
+ ID_SANITISED(ID_PFR0_EL1),
+ ID_SANITISED(ID_PFR1_EL1),
+ ID_SANITISED(ID_DFR0_EL1),
+ ID_HIDDEN(ID_AFR0_EL1),
+ ID_SANITISED(ID_MMFR0_EL1),
+ ID_SANITISED(ID_MMFR1_EL1),
+ ID_SANITISED(ID_MMFR2_EL1),
+ ID_SANITISED(ID_MMFR3_EL1),
+
+ /* CRm=2 */
+ ID_SANITISED(ID_ISAR0_EL1),
+ ID_SANITISED(ID_ISAR1_EL1),
+ ID_SANITISED(ID_ISAR2_EL1),
+ ID_SANITISED(ID_ISAR3_EL1),
+ ID_SANITISED(ID_ISAR4_EL1),
+ ID_SANITISED(ID_ISAR5_EL1),
+ ID_SANITISED(ID_MMFR4_EL1),
+ ID_UNALLOCATED(2,7),
+
+ /* CRm=3 */
+ ID_SANITISED(MVFR0_EL1),
+ ID_SANITISED(MVFR1_EL1),
+ ID_SANITISED(MVFR2_EL1),
+ ID_UNALLOCATED(3,3),
+ ID_UNALLOCATED(3,4),
+ ID_UNALLOCATED(3,5),
+ ID_UNALLOCATED(3,6),
+ ID_UNALLOCATED(3,7),
+
+ /* AArch64 ID registers */
+ /* CRm=4 */
+ ID_SANITISED(ID_AA64PFR0_EL1),
+ ID_SANITISED(ID_AA64PFR1_EL1),
+ ID_UNALLOCATED(4,2),
+ ID_UNALLOCATED(4,3),
+ ID_UNALLOCATED(4,4),
+ ID_UNALLOCATED(4,5),
+ ID_UNALLOCATED(4,6),
+ ID_UNALLOCATED(4,7),
+
+ /* CRm=5 */
+ ID_SANITISED(ID_AA64DFR0_EL1),
+ ID_SANITISED(ID_AA64DFR1_EL1),
+ ID_UNALLOCATED(5,2),
+ ID_UNALLOCATED(5,3),
+ ID_HIDDEN(ID_AA64AFR0_EL1),
+ ID_HIDDEN(ID_AA64AFR1_EL1),
+ ID_UNALLOCATED(5,6),
+ ID_UNALLOCATED(5,7),
+
+ /* CRm=6 */
+ ID_SANITISED(ID_AA64ISAR0_EL1),
+ ID_SANITISED(ID_AA64ISAR1_EL1),
+ ID_UNALLOCATED(6,2),
+ ID_UNALLOCATED(6,3),
+ ID_UNALLOCATED(6,4),
+ ID_UNALLOCATED(6,5),
+ ID_UNALLOCATED(6,6),
+ ID_UNALLOCATED(6,7),
+
+ /* CRm=7 */
+ ID_SANITISED(ID_AA64MMFR0_EL1),
+ ID_SANITISED(ID_AA64MMFR1_EL1),
+ ID_SANITISED(ID_AA64MMFR2_EL1),
+ ID_UNALLOCATED(7,3),
+ ID_UNALLOCATED(7,4),
+ ID_UNALLOCATED(7,5),
+ ID_UNALLOCATED(7,6),
+ ID_UNALLOCATED(7,7),
+
{ SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
{ SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
{ SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
@@ -1790,8 +1999,8 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
if (!r)
r = find_reg(¶ms, sys_reg_descs, ARRAY_SIZE(sys_reg_descs));
- /* Not saved in the sys_reg array? */
- if (r && !r->reg)
+ /* Not saved in the sys_reg array and not otherwise accessible? */
+ if (r && !(r->reg || r->get_user))
r = NULL;
return r;
@@ -1815,20 +2024,6 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
FUNCTION_INVARIANT(midr_el1)
FUNCTION_INVARIANT(ctr_el0)
FUNCTION_INVARIANT(revidr_el1)
-FUNCTION_INVARIANT(id_pfr0_el1)
-FUNCTION_INVARIANT(id_pfr1_el1)
-FUNCTION_INVARIANT(id_dfr0_el1)
-FUNCTION_INVARIANT(id_afr0_el1)
-FUNCTION_INVARIANT(id_mmfr0_el1)
-FUNCTION_INVARIANT(id_mmfr1_el1)
-FUNCTION_INVARIANT(id_mmfr2_el1)
-FUNCTION_INVARIANT(id_mmfr3_el1)
-FUNCTION_INVARIANT(id_isar0_el1)
-FUNCTION_INVARIANT(id_isar1_el1)
-FUNCTION_INVARIANT(id_isar2_el1)
-FUNCTION_INVARIANT(id_isar3_el1)
-FUNCTION_INVARIANT(id_isar4_el1)
-FUNCTION_INVARIANT(id_isar5_el1)
FUNCTION_INVARIANT(clidr_el1)
FUNCTION_INVARIANT(aidr_el1)
@@ -1836,20 +2031,6 @@ FUNCTION_INVARIANT(aidr_el1)
static struct sys_reg_desc invariant_sys_regs[] = {
{ SYS_DESC(SYS_MIDR_EL1), NULL, get_midr_el1 },
{ SYS_DESC(SYS_REVIDR_EL1), NULL, get_revidr_el1 },
- { SYS_DESC(SYS_ID_PFR0_EL1), NULL, get_id_pfr0_el1 },
- { SYS_DESC(SYS_ID_PFR1_EL1), NULL, get_id_pfr1_el1 },
- { SYS_DESC(SYS_ID_DFR0_EL1), NULL, get_id_dfr0_el1 },
- { SYS_DESC(SYS_ID_AFR0_EL1), NULL, get_id_afr0_el1 },
- { SYS_DESC(SYS_ID_MMFR0_EL1), NULL, get_id_mmfr0_el1 },
- { SYS_DESC(SYS_ID_MMFR1_EL1), NULL, get_id_mmfr1_el1 },
- { SYS_DESC(SYS_ID_MMFR2_EL1), NULL, get_id_mmfr2_el1 },
- { SYS_DESC(SYS_ID_MMFR3_EL1), NULL, get_id_mmfr3_el1 },
- { SYS_DESC(SYS_ID_ISAR0_EL1), NULL, get_id_isar0_el1 },
- { SYS_DESC(SYS_ID_ISAR1_EL1), NULL, get_id_isar1_el1 },
- { SYS_DESC(SYS_ID_ISAR2_EL1), NULL, get_id_isar2_el1 },
- { SYS_DESC(SYS_ID_ISAR3_EL1), NULL, get_id_isar3_el1 },
- { SYS_DESC(SYS_ID_ISAR4_EL1), NULL, get_id_isar4_el1 },
- { SYS_DESC(SYS_ID_ISAR5_EL1), NULL, get_id_isar5_el1 },
{ SYS_DESC(SYS_CLIDR_EL1), NULL, get_clidr_el1 },
{ SYS_DESC(SYS_AIDR_EL1), NULL, get_aidr_el1 },
{ SYS_DESC(SYS_CTR_EL0), NULL, get_ctr_el0 },
@@ -2079,12 +2260,31 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
return true;
}
+static int walk_one_sys_reg(const struct sys_reg_desc *rd,
+ u64 __user **uind,
+ unsigned int *total)
+{
+ /*
+ * Ignore registers we trap but don't save,
+ * and for which no custom user accessor is provided.
+ */
+ if (!(rd->reg || rd->get_user))
+ return 0;
+
+ if (!copy_reg_to_user(rd, uind))
+ return -EFAULT;
+
+ (*total)++;
+ return 0;
+}
+
/* Assumed ordered tables, see kvm_sys_reg_table_init. */
static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
{
const struct sys_reg_desc *i1, *i2, *end1, *end2;
unsigned int total = 0;
size_t num;
+ int err;
/* We check for duplicates here, to allow arch-specific overrides. */
i1 = get_target_table(vcpu->arch.target, true, &num);
@@ -2098,21 +2298,13 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
while (i1 || i2) {
int cmp = cmp_sys_reg(i1, i2);
/* target-specific overrides generic entry. */
- if (cmp <= 0) {
- /* Ignore registers we trap but don't save. */
- if (i1->reg) {
- if (!copy_reg_to_user(i1, &uind))
- return -EFAULT;
- total++;
- }
- } else {
- /* Ignore registers we trap but don't save. */
- if (i2->reg) {
- if (!copy_reg_to_user(i2, &uind))
- return -EFAULT;
- total++;
- }
- }
+ if (cmp <= 0)
+ err = walk_one_sys_reg(i1, &uind, &total);
+ else
+ err = walk_one_sys_reg(i2, &uind, &total);
+
+ if (err)
+ return err;
if (cmp <= 0 && ++i1 == end1)
i1 = NULL;
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 03/28] arm64: efi: Add missing Kconfig dependency on KERNEL_MODE_NEON
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
The EFI runtime services ABI permits calls to EFI to clobber
certain FPSIMD/NEON registers, as per the AArch64 procedure call
standard.
Saving/restoring the clobbered registers around such calls needs
KERNEL_MODE_NEON, but the dependency is missing from Kconfig.
This patch adds the missing dependency.
This will aid bisection of the patches implementing support for the
ARM Scalable Vector Extension (SVE).
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 0df64a6..ca711ac 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1063,6 +1063,7 @@ config EFI_STUB
config EFI
bool "UEFI runtime support"
depends on OF && !CPU_BIG_ENDIAN
+ depends on KERNEL_MODE_NEON
select LIBFDT
select UCS2_STRING
select EFI_PARAMS_FROM_FDT
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 03/28] arm64: efi: Add missing Kconfig dependency on KERNEL_MODE_NEON
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
The EFI runtime services ABI permits calls to EFI to clobber
certain FPSIMD/NEON registers, as per the AArch64 procedure call
standard.
Saving/restoring the clobbered registers around such calls needs
KERNEL_MODE_NEON, but the dependency is missing from Kconfig.
This patch adds the missing dependency.
This will aid bisection of the patches implementing support for the
ARM Scalable Vector Extension (SVE).
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
---
arch/arm64/Kconfig | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 0df64a6..ca711ac 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1063,6 +1063,7 @@ config EFI_STUB
config EFI
bool "UEFI runtime support"
depends on OF && !CPU_BIG_ENDIAN
+ depends on KERNEL_MODE_NEON
select LIBFDT
select UCS2_STRING
select EFI_PARAMS_FROM_FDT
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 04/28] arm64: Port deprecated instruction emulation to new sysctl interface
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Currently, armv8_deprected.c takes charge of the "abi" sysctl
directory, which makes life difficult for other code that wants to
register sysctls in the same directory.
There is a "new" [1] sysctl registration interface that removes the
need to define ctl_tables for parent directories explicitly, which
is ideal here.
This patch ports register_insn_emulation_sysctl() over to the
register_sysctl() interface and removes the redundant ctl_table for
"abi".
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
[1] fea478d4101a (sysctl: Add register_sysctl for normal sysctl
users)
The commit message notes an intent to port users of the
pre-existing interfaces over to register_sysctl(), though the
number of users of the new interface currently appears negligible.
---
arch/arm64/kernel/armv8_deprecated.c | 15 +++------------
1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c
index f0e6d71..e15eb2d 100644
--- a/arch/arm64/kernel/armv8_deprecated.c
+++ b/arch/arm64/kernel/armv8_deprecated.c
@@ -228,15 +228,7 @@ static int emulation_proc_handler(struct ctl_table *table, int write,
return ret;
}
-static struct ctl_table ctl_abi[] = {
- {
- .procname = "abi",
- .mode = 0555,
- },
- { }
-};
-
-static void __init register_insn_emulation_sysctl(struct ctl_table *table)
+static void __init register_insn_emulation_sysctl(void)
{
unsigned long flags;
int i = 0;
@@ -262,8 +254,7 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table)
}
raw_spin_unlock_irqrestore(&insn_emulation_lock, flags);
- table->child = insns_sysctl;
- register_sysctl_table(table);
+ register_sysctl("abi", insns_sysctl);
}
/*
@@ -644,7 +635,7 @@ static int __init armv8_deprecated_init(void)
cpuhp_setup_state_nocalls(CPUHP_AP_ARM64_ISNDEP_STARTING,
"arm64/isndep:starting",
run_all_insn_set_hw_mode, NULL);
- register_insn_emulation_sysctl(ctl_abi);
+ register_insn_emulation_sysctl();
return 0;
}
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 04/28] arm64: Port deprecated instruction emulation to new sysctl interface
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Currently, armv8_deprected.c takes charge of the "abi" sysctl
directory, which makes life difficult for other code that wants to
register sysctls in the same directory.
There is a "new" [1] sysctl registration interface that removes the
need to define ctl_tables for parent directories explicitly, which
is ideal here.
This patch ports register_insn_emulation_sysctl() over to the
register_sysctl() interface and removes the redundant ctl_table for
"abi".
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
[1] fea478d4101a (sysctl: Add register_sysctl for normal sysctl
users)
The commit message notes an intent to port users of the
pre-existing interfaces over to register_sysctl(), though the
number of users of the new interface currently appears negligible.
---
arch/arm64/kernel/armv8_deprecated.c | 15 +++------------
1 file changed, 3 insertions(+), 12 deletions(-)
diff --git a/arch/arm64/kernel/armv8_deprecated.c b/arch/arm64/kernel/armv8_deprecated.c
index f0e6d71..e15eb2d 100644
--- a/arch/arm64/kernel/armv8_deprecated.c
+++ b/arch/arm64/kernel/armv8_deprecated.c
@@ -228,15 +228,7 @@ static int emulation_proc_handler(struct ctl_table *table, int write,
return ret;
}
-static struct ctl_table ctl_abi[] = {
- {
- .procname = "abi",
- .mode = 0555,
- },
- { }
-};
-
-static void __init register_insn_emulation_sysctl(struct ctl_table *table)
+static void __init register_insn_emulation_sysctl(void)
{
unsigned long flags;
int i = 0;
@@ -262,8 +254,7 @@ static void __init register_insn_emulation_sysctl(struct ctl_table *table)
}
raw_spin_unlock_irqrestore(&insn_emulation_lock, flags);
- table->child = insns_sysctl;
- register_sysctl_table(table);
+ register_sysctl("abi", insns_sysctl);
}
/*
@@ -644,7 +635,7 @@ static int __init armv8_deprecated_init(void)
cpuhp_setup_state_nocalls(CPUHP_AP_ARM64_ISNDEP_STARTING,
"arm64/isndep:starting",
run_all_insn_set_hw_mode, NULL);
- register_insn_emulation_sysctl(ctl_abi);
+ register_insn_emulation_sysctl();
return 0;
}
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 05/28] arm64: fpsimd: Simplify uses of {set,clear}_ti_thread_flag()
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
The existing FPSIMD context switch code contains a couple of
instances of {set,clear}_ti_thread(task_thread_info(task)). Since
there are thread flag manipulators that operate directly on
task_struct, this verbosity isn't strictly needed.
For consistency, this patch simplifies the affected calls. This
should have no impact on behaviour.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/fpsimd.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 3a68cf3..9d762dd 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -159,11 +159,9 @@ void fpsimd_thread_switch(struct task_struct *next)
if (__this_cpu_read(fpsimd_last_state) == st
&& st->cpu == smp_processor_id())
- clear_ti_thread_flag(task_thread_info(next),
- TIF_FOREIGN_FPSTATE);
+ clear_tsk_thread_flag(next, TIF_FOREIGN_FPSTATE);
else
- set_ti_thread_flag(task_thread_info(next),
- TIF_FOREIGN_FPSTATE);
+ set_tsk_thread_flag(next, TIF_FOREIGN_FPSTATE);
}
}
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 05/28] arm64: fpsimd: Simplify uses of {set, clear}_ti_thread_flag()
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
The existing FPSIMD context switch code contains a couple of
instances of {set,clear}_ti_thread(task_thread_info(task)). Since
there are thread flag manipulators that operate directly on
task_struct, this verbosity isn't strictly needed.
For consistency, this patch simplifies the affected calls. This
should have no impact on behaviour.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
Acked-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/fpsimd.c | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 3a68cf3..9d762dd 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -159,11 +159,9 @@ void fpsimd_thread_switch(struct task_struct *next)
if (__this_cpu_read(fpsimd_last_state) == st
&& st->cpu == smp_processor_id())
- clear_ti_thread_flag(task_thread_info(next),
- TIF_FOREIGN_FPSTATE);
+ clear_tsk_thread_flag(next, TIF_FOREIGN_FPSTATE);
else
- set_ti_thread_flag(task_thread_info(next),
- TIF_FOREIGN_FPSTATE);
+ set_tsk_thread_flag(next, TIF_FOREIGN_FPSTATE);
}
}
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 06/28] arm64/sve: System register and exception syndrome definitions
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
The SVE architecture adds some system registers, ID register fields
and a dedicated ESR exception class.
This patch adds the appropriate definitions that will be needed by
the kernel.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Bennée:
* Add comments to clarify CPACR_EL1_ZEN_ELxEN bit meanings.
* Add comment clarifying the status of the LEN field expansion bits.
---
arch/arm64/include/asm/esr.h | 3 ++-
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/sysreg.h | 21 +++++++++++++++++++++
arch/arm64/kernel/traps.c | 1 +
4 files changed, 25 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index 66ed8b6..014d7d8 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -43,7 +43,8 @@
#define ESR_ELx_EC_HVC64 (0x16)
#define ESR_ELx_EC_SMC64 (0x17)
#define ESR_ELx_EC_SYS64 (0x18)
-/* Unallocated EC: 0x19 - 0x1E */
+#define ESR_ELx_EC_SVE (0x19)
+/* Unallocated EC: 0x1A - 0x1E */
#define ESR_ELx_EC_IMP_DEF (0x1f)
#define ESR_ELx_EC_IABT_LOW (0x20)
#define ESR_ELx_EC_IABT_CUR (0x21)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 61d694c..dbf0537 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -185,6 +185,7 @@
#define CPTR_EL2_TCPAC (1 << 31)
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
+#define CPTR_EL2_TZ (1 << 8)
#define CPTR_EL2_DEFAULT 0x000033ff
/* Hyp Debug Configuration Register bits */
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 480ecd6..36fe2ae 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -145,6 +145,7 @@
#define SYS_ID_AA64PFR0_EL1 sys_reg(3, 0, 0, 4, 0)
#define SYS_ID_AA64PFR1_EL1 sys_reg(3, 0, 0, 4, 1)
+#define SYS_ID_AA64ZFR0_EL1 sys_reg(3, 0, 0, 4, 4)
#define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
#define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
@@ -163,6 +164,8 @@
#define SYS_ACTLR_EL1 sys_reg(3, 0, 1, 0, 1)
#define SYS_CPACR_EL1 sys_reg(3, 0, 1, 0, 2)
+#define SYS_ZCR_EL1 sys_reg(3, 0, 1, 2, 0)
+
#define SYS_TTBR0_EL1 sys_reg(3, 0, 2, 0, 0)
#define SYS_TTBR1_EL1 sys_reg(3, 0, 2, 0, 1)
#define SYS_TCR_EL1 sys_reg(3, 0, 2, 0, 2)
@@ -253,6 +256,8 @@
#define SYS_PMCCFILTR_EL0 sys_reg (3, 3, 14, 15, 7)
+#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
+
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
#define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
#define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
@@ -335,6 +340,7 @@
#define ID_AA64ISAR1_DPB_SHIFT 0
/* id_aa64pfr0 */
+#define ID_AA64PFR0_SVE_SHIFT 32
#define ID_AA64PFR0_GIC_SHIFT 24
#define ID_AA64PFR0_ASIMD_SHIFT 20
#define ID_AA64PFR0_FP_SHIFT 16
@@ -343,6 +349,7 @@
#define ID_AA64PFR0_EL1_SHIFT 4
#define ID_AA64PFR0_EL0_SHIFT 0
+#define ID_AA64PFR0_SVE 0x1
#define ID_AA64PFR0_FP_NI 0xf
#define ID_AA64PFR0_FP_SUPPORTED 0x0
#define ID_AA64PFR0_ASIMD_NI 0xf
@@ -444,6 +451,20 @@
#endif
+/*
+ * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which
+ * are reserved by the SVE architecture for future expansion of the LEN
+ * field, with compatible semantics.
+ */
+#define ZCR_ELx_LEN_SHIFT 0
+#define ZCR_ELx_LEN_SIZE 9
+#define ZCR_ELx_LEN_MASK 0x1ff
+
+#define CPACR_EL1_ZEN_EL1EN (1 << 16) /* enable EL1 access */
+#define CPACR_EL1_ZEN_EL0EN (1 << 17) /* enable EL0 access, if EL1EN set */
+#define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
+
+
/* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */
#define SYS_MPIDR_SAFE_VAL (1UL << 31)
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 5ea4b85..f202932 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -603,6 +603,7 @@ static const char *esr_class_str[] = {
[ESR_ELx_EC_HVC64] = "HVC (AArch64)",
[ESR_ELx_EC_SMC64] = "SMC (AArch64)",
[ESR_ELx_EC_SYS64] = "MSR/MRS (AArch64)",
+ [ESR_ELx_EC_SVE] = "SVE",
[ESR_ELx_EC_IMP_DEF] = "EL3 IMP DEF",
[ESR_ELx_EC_IABT_LOW] = "IABT (lower EL)",
[ESR_ELx_EC_IABT_CUR] = "IABT (current EL)",
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 06/28] arm64/sve: System register and exception syndrome definitions
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
The SVE architecture adds some system registers, ID register fields
and a dedicated ESR exception class.
This patch adds the appropriate definitions that will be needed by
the kernel.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Benn?e:
* Add comments to clarify CPACR_EL1_ZEN_ELxEN bit meanings.
* Add comment clarifying the status of the LEN field expansion bits.
---
arch/arm64/include/asm/esr.h | 3 ++-
arch/arm64/include/asm/kvm_arm.h | 1 +
arch/arm64/include/asm/sysreg.h | 21 +++++++++++++++++++++
arch/arm64/kernel/traps.c | 1 +
4 files changed, 25 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
index 66ed8b6..014d7d8 100644
--- a/arch/arm64/include/asm/esr.h
+++ b/arch/arm64/include/asm/esr.h
@@ -43,7 +43,8 @@
#define ESR_ELx_EC_HVC64 (0x16)
#define ESR_ELx_EC_SMC64 (0x17)
#define ESR_ELx_EC_SYS64 (0x18)
-/* Unallocated EC: 0x19 - 0x1E */
+#define ESR_ELx_EC_SVE (0x19)
+/* Unallocated EC: 0x1A - 0x1E */
#define ESR_ELx_EC_IMP_DEF (0x1f)
#define ESR_ELx_EC_IABT_LOW (0x20)
#define ESR_ELx_EC_IABT_CUR (0x21)
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index 61d694c..dbf0537 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -185,6 +185,7 @@
#define CPTR_EL2_TCPAC (1 << 31)
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
+#define CPTR_EL2_TZ (1 << 8)
#define CPTR_EL2_DEFAULT 0x000033ff
/* Hyp Debug Configuration Register bits */
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 480ecd6..36fe2ae 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -145,6 +145,7 @@
#define SYS_ID_AA64PFR0_EL1 sys_reg(3, 0, 0, 4, 0)
#define SYS_ID_AA64PFR1_EL1 sys_reg(3, 0, 0, 4, 1)
+#define SYS_ID_AA64ZFR0_EL1 sys_reg(3, 0, 0, 4, 4)
#define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
#define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
@@ -163,6 +164,8 @@
#define SYS_ACTLR_EL1 sys_reg(3, 0, 1, 0, 1)
#define SYS_CPACR_EL1 sys_reg(3, 0, 1, 0, 2)
+#define SYS_ZCR_EL1 sys_reg(3, 0, 1, 2, 0)
+
#define SYS_TTBR0_EL1 sys_reg(3, 0, 2, 0, 0)
#define SYS_TTBR1_EL1 sys_reg(3, 0, 2, 0, 1)
#define SYS_TCR_EL1 sys_reg(3, 0, 2, 0, 2)
@@ -253,6 +256,8 @@
#define SYS_PMCCFILTR_EL0 sys_reg (3, 3, 14, 15, 7)
+#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
+
#define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
#define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
#define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
@@ -335,6 +340,7 @@
#define ID_AA64ISAR1_DPB_SHIFT 0
/* id_aa64pfr0 */
+#define ID_AA64PFR0_SVE_SHIFT 32
#define ID_AA64PFR0_GIC_SHIFT 24
#define ID_AA64PFR0_ASIMD_SHIFT 20
#define ID_AA64PFR0_FP_SHIFT 16
@@ -343,6 +349,7 @@
#define ID_AA64PFR0_EL1_SHIFT 4
#define ID_AA64PFR0_EL0_SHIFT 0
+#define ID_AA64PFR0_SVE 0x1
#define ID_AA64PFR0_FP_NI 0xf
#define ID_AA64PFR0_FP_SUPPORTED 0x0
#define ID_AA64PFR0_ASIMD_NI 0xf
@@ -444,6 +451,20 @@
#endif
+/*
+ * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which
+ * are reserved by the SVE architecture for future expansion of the LEN
+ * field, with compatible semantics.
+ */
+#define ZCR_ELx_LEN_SHIFT 0
+#define ZCR_ELx_LEN_SIZE 9
+#define ZCR_ELx_LEN_MASK 0x1ff
+
+#define CPACR_EL1_ZEN_EL1EN (1 << 16) /* enable EL1 access */
+#define CPACR_EL1_ZEN_EL0EN (1 << 17) /* enable EL0 access, if EL1EN set */
+#define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
+
+
/* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */
#define SYS_MPIDR_SAFE_VAL (1UL << 31)
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 5ea4b85..f202932 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -603,6 +603,7 @@ static const char *esr_class_str[] = {
[ESR_ELx_EC_HVC64] = "HVC (AArch64)",
[ESR_ELx_EC_SMC64] = "SMC (AArch64)",
[ESR_ELx_EC_SYS64] = "MSR/MRS (AArch64)",
+ [ESR_ELx_EC_SVE] = "SVE",
[ESR_ELx_EC_IMP_DEF] = "EL3 IMP DEF",
[ESR_ELx_EC_IABT_LOW] = "IABT (lower EL)",
[ESR_ELx_EC_IABT_CUR] = "IABT (current EL)",
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 07/28] arm64/sve: Low-level SVE architectural state manipulation functions
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Manipulating the SVE architectural state, including the vector and
predicate registers, first-fault register and the vector length,
requires the use of dedicated instructions added by SVE.
This patch adds suitable assembly functions for saving and
restoring the SVE registers and querying the vector length.
Setting of the vector length is done as part of register restore.
Since people building kernels may not all get an SVE-enabled
toolchain for a while, this patch uses macros that generate
explicit opcodes in place of assembler mnemonics.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Bennée:
* Annotate instruction generation macros with the canonical
architectural syntax so that people can cross-reference more easily
against the architectural documentation.
---
arch/arm64/include/asm/fpsimd.h | 5 ++
arch/arm64/include/asm/fpsimdmacros.h | 148 ++++++++++++++++++++++++++++++++++
arch/arm64/kernel/entry-fpsimd.S | 17 ++++
3 files changed, 170 insertions(+)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 410c481..026a7c7 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -67,6 +67,11 @@ extern void fpsimd_update_current_state(struct fpsimd_state *state);
extern void fpsimd_flush_task_state(struct task_struct *target);
+extern void sve_save_state(void *state, u32 *pfpsr);
+extern void sve_load_state(void const *state, u32 const *pfpsr,
+ unsigned long vq_minus_1);
+extern unsigned int sve_get_vl(void);
+
/* For use by EFI runtime services calls only */
extern void __efi_fpsimd_begin(void);
extern void __efi_fpsimd_end(void);
diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h
index 0f5fdd3..e050d76 100644
--- a/arch/arm64/include/asm/fpsimdmacros.h
+++ b/arch/arm64/include/asm/fpsimdmacros.h
@@ -75,3 +75,151 @@
ldr w\tmpnr, [\state, #16 * 2 + 4]
fpsimd_restore_fpcr x\tmpnr, \state
.endm
+
+/* Sanity-check macros to help avoid encoding garbage instructions */
+
+.macro _check_general_reg nr
+ .if (\nr) < 0 || (\nr) > 30
+ .error "Bad register number \nr."
+ .endif
+.endm
+
+.macro _sve_check_zreg znr
+ .if (\znr) < 0 || (\znr) > 31
+ .error "Bad Scalable Vector Extension vector register number \znr."
+ .endif
+.endm
+
+.macro _sve_check_preg pnr
+ .if (\pnr) < 0 || (\pnr) > 15
+ .error "Bad Scalable Vector Extension predicate register number \pnr."
+ .endif
+.endm
+
+.macro _check_num n, min, max
+ .if (\n) < (\min) || (\n) > (\max)
+ .error "Number \n out of range [\min,\max]"
+ .endif
+.endm
+
+/* SVE instruction encodings for non-SVE-capable assemblers */
+
+/* STR (vector): STR Z\nz, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_str_v nz, nxbase, offset=0
+ _sve_check_zreg \nz
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0xe5804000 \
+ | (\nz) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* LDR (vector): LDR Z\nz, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_ldr_v nz, nxbase, offset=0
+ _sve_check_zreg \nz
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0x85804000 \
+ | (\nz) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* STR (predicate): STR P\np, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_str_p np, nxbase, offset=0
+ _sve_check_preg \np
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0xe5800000 \
+ | (\np) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* LDR (predicate): LDR P\np, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_ldr_p np, nxbase, offset=0
+ _sve_check_preg \np
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0x85800000 \
+ | (\np) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* RDVL X\nx, #\imm */
+.macro _sve_rdvl nx, imm
+ _check_general_reg \nx
+ _check_num (\imm), -0x20, 0x1f
+ .inst 0x04bf5000 \
+ | (\nx) \
+ | (((\imm) & 0x3f) << 5)
+.endm
+
+/* RDFFR (unpredicated): RDFFR P\np.B */
+.macro _sve_rdffr np
+ _sve_check_preg \np
+ .inst 0x2519f000 \
+ | (\np)
+.endm
+
+/* WRFFR P\np.B */
+.macro _sve_wrffr np
+ _sve_check_preg \np
+ .inst 0x25289000 \
+ | ((\np) << 5)
+.endm
+
+.macro __for from:req, to:req
+ .if (\from) == (\to)
+ _for__body \from
+ .else
+ __for \from, (\from) + ((\to) - (\from)) / 2
+ __for (\from) + ((\to) - (\from)) / 2 + 1, \to
+ .endif
+.endm
+
+.macro _for var:req, from:req, to:req, insn:vararg
+ .macro _for__body \var:req
+ \insn
+ .endm
+
+ __for \from, \to
+
+ .purgem _for__body
+.endm
+
+.macro sve_save nxbase, xpfpsr, nxtmp
+ _for n, 0, 31, _sve_str_v \n, \nxbase, \n - 34
+ _for n, 0, 15, _sve_str_p \n, \nxbase, \n - 16
+ _sve_rdffr 0
+ _sve_str_p 0, \nxbase
+ _sve_ldr_p 0, \nxbase, -16
+
+ mrs x\nxtmp, fpsr
+ str w\nxtmp, [\xpfpsr]
+ mrs x\nxtmp, fpcr
+ str w\nxtmp, [\xpfpsr, #4]
+.endm
+
+.macro sve_load nxbase, xpfpsr, xvqminus1, nxtmp
+ mrs_s x\nxtmp, SYS_ZCR_EL1
+ bic x\nxtmp, x\nxtmp, ZCR_ELx_LEN_MASK
+ orr x\nxtmp, x\nxtmp, \xvqminus1
+ msr_s SYS_ZCR_EL1, x\nxtmp // self-synchronising
+
+ _for n, 0, 31, _sve_ldr_v \n, \nxbase, \n - 34
+ _sve_ldr_p 0, \nxbase
+ _sve_wrffr 0
+ _for n, 0, 15, _sve_ldr_p \n, \nxbase, \n - 16
+
+ ldr w\nxtmp, [\xpfpsr]
+ msr fpsr, x\nxtmp
+ ldr w\nxtmp, [\xpfpsr, #4]
+ msr fpcr, x\nxtmp
+.endm
diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
index 6a27cd6..73f17bf 100644
--- a/arch/arm64/kernel/entry-fpsimd.S
+++ b/arch/arm64/kernel/entry-fpsimd.S
@@ -41,3 +41,20 @@ ENTRY(fpsimd_load_state)
fpsimd_restore x0, 8
ret
ENDPROC(fpsimd_load_state)
+
+#ifdef CONFIG_ARM64_SVE
+ENTRY(sve_save_state)
+ sve_save 0, x1, 2
+ ret
+ENDPROC(sve_save_state)
+
+ENTRY(sve_load_state)
+ sve_load 0, x1, x2, 3
+ ret
+ENDPROC(sve_load_state)
+
+ENTRY(sve_get_vl)
+ _sve_rdvl 0, 1
+ ret
+ENDPROC(sve_get_vl)
+#endif /* CONFIG_ARM64_SVE */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 07/28] arm64/sve: Low-level SVE architectural state manipulation functions
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Manipulating the SVE architectural state, including the vector and
predicate registers, first-fault register and the vector length,
requires the use of dedicated instructions added by SVE.
This patch adds suitable assembly functions for saving and
restoring the SVE registers and querying the vector length.
Setting of the vector length is done as part of register restore.
Since people building kernels may not all get an SVE-enabled
toolchain for a while, this patch uses macros that generate
explicit opcodes in place of assembler mnemonics.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Benn?e:
* Annotate instruction generation macros with the canonical
architectural syntax so that people can cross-reference more easily
against the architectural documentation.
---
arch/arm64/include/asm/fpsimd.h | 5 ++
arch/arm64/include/asm/fpsimdmacros.h | 148 ++++++++++++++++++++++++++++++++++
arch/arm64/kernel/entry-fpsimd.S | 17 ++++
3 files changed, 170 insertions(+)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 410c481..026a7c7 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -67,6 +67,11 @@ extern void fpsimd_update_current_state(struct fpsimd_state *state);
extern void fpsimd_flush_task_state(struct task_struct *target);
+extern void sve_save_state(void *state, u32 *pfpsr);
+extern void sve_load_state(void const *state, u32 const *pfpsr,
+ unsigned long vq_minus_1);
+extern unsigned int sve_get_vl(void);
+
/* For use by EFI runtime services calls only */
extern void __efi_fpsimd_begin(void);
extern void __efi_fpsimd_end(void);
diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h
index 0f5fdd3..e050d76 100644
--- a/arch/arm64/include/asm/fpsimdmacros.h
+++ b/arch/arm64/include/asm/fpsimdmacros.h
@@ -75,3 +75,151 @@
ldr w\tmpnr, [\state, #16 * 2 + 4]
fpsimd_restore_fpcr x\tmpnr, \state
.endm
+
+/* Sanity-check macros to help avoid encoding garbage instructions */
+
+.macro _check_general_reg nr
+ .if (\nr) < 0 || (\nr) > 30
+ .error "Bad register number \nr."
+ .endif
+.endm
+
+.macro _sve_check_zreg znr
+ .if (\znr) < 0 || (\znr) > 31
+ .error "Bad Scalable Vector Extension vector register number \znr."
+ .endif
+.endm
+
+.macro _sve_check_preg pnr
+ .if (\pnr) < 0 || (\pnr) > 15
+ .error "Bad Scalable Vector Extension predicate register number \pnr."
+ .endif
+.endm
+
+.macro _check_num n, min, max
+ .if (\n) < (\min) || (\n) > (\max)
+ .error "Number \n out of range [\min,\max]"
+ .endif
+.endm
+
+/* SVE instruction encodings for non-SVE-capable assemblers */
+
+/* STR (vector): STR Z\nz, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_str_v nz, nxbase, offset=0
+ _sve_check_zreg \nz
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0xe5804000 \
+ | (\nz) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* LDR (vector): LDR Z\nz, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_ldr_v nz, nxbase, offset=0
+ _sve_check_zreg \nz
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0x85804000 \
+ | (\nz) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* STR (predicate): STR P\np, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_str_p np, nxbase, offset=0
+ _sve_check_preg \np
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0xe5800000 \
+ | (\np) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* LDR (predicate): LDR P\np, [X\nxbase, #\offset, MUL VL] */
+.macro _sve_ldr_p np, nxbase, offset=0
+ _sve_check_preg \np
+ _check_general_reg \nxbase
+ _check_num (\offset), -0x100, 0xff
+ .inst 0x85800000 \
+ | (\np) \
+ | ((\nxbase) << 5) \
+ | (((\offset) & 7) << 10) \
+ | (((\offset) & 0x1f8) << 13)
+.endm
+
+/* RDVL X\nx, #\imm */
+.macro _sve_rdvl nx, imm
+ _check_general_reg \nx
+ _check_num (\imm), -0x20, 0x1f
+ .inst 0x04bf5000 \
+ | (\nx) \
+ | (((\imm) & 0x3f) << 5)
+.endm
+
+/* RDFFR (unpredicated): RDFFR P\np.B */
+.macro _sve_rdffr np
+ _sve_check_preg \np
+ .inst 0x2519f000 \
+ | (\np)
+.endm
+
+/* WRFFR P\np.B */
+.macro _sve_wrffr np
+ _sve_check_preg \np
+ .inst 0x25289000 \
+ | ((\np) << 5)
+.endm
+
+.macro __for from:req, to:req
+ .if (\from) == (\to)
+ _for__body \from
+ .else
+ __for \from, (\from) + ((\to) - (\from)) / 2
+ __for (\from) + ((\to) - (\from)) / 2 + 1, \to
+ .endif
+.endm
+
+.macro _for var:req, from:req, to:req, insn:vararg
+ .macro _for__body \var:req
+ \insn
+ .endm
+
+ __for \from, \to
+
+ .purgem _for__body
+.endm
+
+.macro sve_save nxbase, xpfpsr, nxtmp
+ _for n, 0, 31, _sve_str_v \n, \nxbase, \n - 34
+ _for n, 0, 15, _sve_str_p \n, \nxbase, \n - 16
+ _sve_rdffr 0
+ _sve_str_p 0, \nxbase
+ _sve_ldr_p 0, \nxbase, -16
+
+ mrs x\nxtmp, fpsr
+ str w\nxtmp, [\xpfpsr]
+ mrs x\nxtmp, fpcr
+ str w\nxtmp, [\xpfpsr, #4]
+.endm
+
+.macro sve_load nxbase, xpfpsr, xvqminus1, nxtmp
+ mrs_s x\nxtmp, SYS_ZCR_EL1
+ bic x\nxtmp, x\nxtmp, ZCR_ELx_LEN_MASK
+ orr x\nxtmp, x\nxtmp, \xvqminus1
+ msr_s SYS_ZCR_EL1, x\nxtmp // self-synchronising
+
+ _for n, 0, 31, _sve_ldr_v \n, \nxbase, \n - 34
+ _sve_ldr_p 0, \nxbase
+ _sve_wrffr 0
+ _for n, 0, 15, _sve_ldr_p \n, \nxbase, \n - 16
+
+ ldr w\nxtmp, [\xpfpsr]
+ msr fpsr, x\nxtmp
+ ldr w\nxtmp, [\xpfpsr, #4]
+ msr fpcr, x\nxtmp
+.endm
diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
index 6a27cd6..73f17bf 100644
--- a/arch/arm64/kernel/entry-fpsimd.S
+++ b/arch/arm64/kernel/entry-fpsimd.S
@@ -41,3 +41,20 @@ ENTRY(fpsimd_load_state)
fpsimd_restore x0, 8
ret
ENDPROC(fpsimd_load_state)
+
+#ifdef CONFIG_ARM64_SVE
+ENTRY(sve_save_state)
+ sve_save 0, x1, 2
+ ret
+ENDPROC(sve_save_state)
+
+ENTRY(sve_load_state)
+ sve_load 0, x1, x2, 3
+ ret
+ENDPROC(sve_load_state)
+
+ENTRY(sve_get_vl)
+ _sve_rdvl 0, 1
+ ret
+ENDPROC(sve_get_vl)
+#endif /* CONFIG_ARM64_SVE */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 08/28] arm64/sve: Kconfig update and conditional compilation support
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
This patch adds CONFIG_ARM64_SVE to control building of SVE support
into the kernel, and adds a stub predicate system_supports_sve() to
control conditional compilation and runtime SVE support.
system_supports_sve() just returns false for now: it will be
replaced with a non-trivial implementation in a later patch, once
SVE support is complete enough to be enabled safely.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/Kconfig | 11 +++++++++++
arch/arm64/include/asm/cpufeature.h | 5 +++++
2 files changed, 16 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ca711ac..9b3a50e 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -975,6 +975,17 @@ config ARM64_PMEM
endmenu
+config ARM64_SVE
+ bool "ARM Scalable Vector Extension support"
+ default y
+ help
+ The Scalable Vector Extension (SVE) is an extension to the AArch64
+ execution state which complements and extends the SIMD functionality
+ of the base architecture to support much larger vectors and to enable
+ additional vectorisation opportunities.
+
+ To enable use of this extension on CPUs that implement it, say Y.
+
config ARM64_MODULE_CMODEL_LARGE
bool
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 428ee1f..4ea3441 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -262,6 +262,11 @@ static inline bool system_uses_ttbr0_pan(void)
!cpus_have_const_cap(ARM64_HAS_PAN);
}
+static inline bool system_supports_sve(void)
+{
+ return false;
+}
+
#endif /* __ASSEMBLY__ */
#endif
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 08/28] arm64/sve: Kconfig update and conditional compilation support
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch adds CONFIG_ARM64_SVE to control building of SVE support
into the kernel, and adds a stub predicate system_supports_sve() to
control conditional compilation and runtime SVE support.
system_supports_sve() just returns false for now: it will be
replaced with a non-trivial implementation in a later patch, once
SVE support is complete enough to be enabled safely.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
---
arch/arm64/Kconfig | 11 +++++++++++
arch/arm64/include/asm/cpufeature.h | 5 +++++
2 files changed, 16 insertions(+)
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index ca711ac..9b3a50e 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -975,6 +975,17 @@ config ARM64_PMEM
endmenu
+config ARM64_SVE
+ bool "ARM Scalable Vector Extension support"
+ default y
+ help
+ The Scalable Vector Extension (SVE) is an extension to the AArch64
+ execution state which complements and extends the SIMD functionality
+ of the base architecture to support much larger vectors and to enable
+ additional vectorisation opportunities.
+
+ To enable use of this extension on CPUs that implement it, say Y.
+
config ARM64_MODULE_CMODEL_LARGE
bool
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 428ee1f..4ea3441 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -262,6 +262,11 @@ static inline bool system_uses_ttbr0_pan(void)
!cpus_have_const_cap(ARM64_HAS_PAN);
}
+static inline bool system_supports_sve(void)
+{
+ return false;
+}
+
#endif /* __ASSEMBLY__ */
#endif
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 09/28] arm64/sve: Signal frame and context structure definition
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
gdb, Alan Hayward, Yao Qi
This patch defines the representation that will be used for the SVE
register state in the signal frame, and implements support for
saving and restoring the SVE registers around signals.
The same layout will also be used for the in-kernel task state.
Due to the variability of the SVE vector length, it is not possible
to define a fixed C struct to describe all the registers. Instead,
Macros are defined in sigcontext.h to facilitate access to the
parts of the structure.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Bennée:
* Add SVE_VQ_BYTES #define to make it clear when the code is
intentionally referring to the size in bytes of a quadword, and migrate
away from magic 16/0x10 where used with this meaning.
---
arch/arm64/include/uapi/asm/sigcontext.h | 117 ++++++++++++++++++++++++++++++-
1 file changed, 116 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h
index f0a76b9..c78cf8e 100644
--- a/arch/arm64/include/uapi/asm/sigcontext.h
+++ b/arch/arm64/include/uapi/asm/sigcontext.h
@@ -16,6 +16,8 @@
#ifndef _UAPI__ASM_SIGCONTEXT_H
#define _UAPI__ASM_SIGCONTEXT_H
+#ifndef __ASSEMBLY__
+
#include <linux/types.h>
/*
@@ -41,10 +43,11 @@ struct sigcontext {
*
* 0x210 fpsimd_context
* 0x10 esr_context
+ * 0x8a0 sve_context (vl <= 64) (optional)
* 0x20 extra_context (optional)
* 0x10 terminator (null _aarch64_ctx)
*
- * 0xdb0 (reserved for future allocation)
+ * 0x510 (reserved for future allocation)
*
* New records that can exceed this space need to be opt-in for userspace, so
* that an expanded signal frame is not generated unexpectedly. The mechanism
@@ -116,4 +119,116 @@ struct extra_context {
__u32 __reserved[3];
};
+#define SVE_MAGIC 0x53564501
+
+struct sve_context {
+ struct _aarch64_ctx head;
+ __u16 vl;
+ __u16 __reserved[3];
+};
+
+#endif /* !__ASSEMBLY__ */
+
+/*
+ * The SVE architecture leaves space for future expansion of the
+ * vector length beyond its initial architectural limit of 2048 bits
+ * (16 quadwords).
+ */
+#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
+
+#define SVE_VQ_MIN 1
+#define SVE_VQ_MAX 0x200
+
+#define SVE_VL_MIN (SVE_VQ_MIN * SVE_VQ_BYTES)
+#define SVE_VL_MAX (SVE_VQ_MAX * SVE_VQ_BYTES)
+
+#define SVE_NUM_ZREGS 32
+#define SVE_NUM_PREGS 16
+
+#define sve_vl_valid(vl) \
+ ((vl) % SVE_VQ_BYTES == 0 && (vl) >= SVE_VL_MIN && (vl) <= SVE_VL_MAX)
+#define sve_vq_from_vl(vl) ((vl) / SVE_VQ_BYTES)
+#define sve_vl_from_vq(vq) ((vq) * SVE_VQ_BYTES)
+
+/*
+ * If the SVE registers are currently live for the thread at signal delivery,
+ * sve_context.head.size >=
+ * SVE_SIG_CONTEXT_SIZE(sve_vq_from_vl(sve_context.vl))
+ * and the register data may be accessed using the SVE_SIG_*() macros.
+ *
+ * If sve_context.head.size <
+ * SVE_SIG_CONTEXT_SIZE(sve_vq_from_vl(sve_context.vl)),
+ * the SVE registers were not live for the thread and no register data
+ * is included: in this case, the SVE_SIG_*() macros should not be
+ * used except for this check.
+ *
+ * The same convention applies when returning from a signal: a caller
+ * will need to remove or resize the sve_context block if it wants to
+ * make the SVE registers live when they were previously non-live or
+ * vice-versa. This may require the the caller to allocate fresh
+ * memory and/or move other context blocks in the signal frame.
+ *
+ * Changing the vector length during signal return is not permitted:
+ * sve_context.vl must equal the thread's current vector length when
+ * doing a sigreturn.
+ *
+ *
+ * Note: for all these macros, the "vq" argument denotes the SVE
+ * vector length in quadwords (i.e., units of 128 bits).
+ *
+ * The correct way to obtain vq is to use sve_vq_from_vl(vl). The
+ * result is valid if and only if sve_vl_valid(vl) is true. This is
+ * guaranteed for a struct sve_context written by the kernel.
+ *
+ *
+ * Additional macros describe the contents and layout of the payload.
+ * For each, SVE_SIG_x_OFFSET(args) is the start offset relative to
+ * the start of struct sve_context, and SVE_SIG_x_SIZE(args) is the
+ * size in bytes:
+ *
+ * x type description
+ * - ---- -----------
+ * REGS the entire SVE context
+ *
+ * ZREGS __uint128_t[SVE_NUM_ZREGS][vq] all Z-registers
+ * ZREG __uint128_t[vq] individual Z-register Zn
+ *
+ * PREGS uint16_t[SVE_NUM_PREGS][vq] all P-registers
+ * PREG uint16_t[vq] individual P-register Pn
+ *
+ * FFR uint16_t[vq] first-fault status register
+ *
+ * Additional data might be appended in the future.
+ */
+
+#define SVE_SIG_ZREG_SIZE(vq) ((__u32)(vq) * SVE_VQ_BYTES)
+#define SVE_SIG_PREG_SIZE(vq) ((__u32)(vq) * (SVE_VQ_BYTES / 8))
+#define SVE_SIG_FFR_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
+
+#define SVE_SIG_REGS_OFFSET \
+ ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+#define SVE_SIG_ZREGS_OFFSET SVE_SIG_REGS_OFFSET
+#define SVE_SIG_ZREG_OFFSET(vq, n) \
+ (SVE_SIG_ZREGS_OFFSET + SVE_SIG_ZREG_SIZE(vq) * (n))
+#define SVE_SIG_ZREGS_SIZE(vq) \
+ (SVE_SIG_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_SIG_ZREGS_OFFSET)
+
+#define SVE_SIG_PREGS_OFFSET(vq) \
+ (SVE_SIG_ZREGS_OFFSET + SVE_SIG_ZREGS_SIZE(vq))
+#define SVE_SIG_PREG_OFFSET(vq, n) \
+ (SVE_SIG_PREGS_OFFSET(vq) + SVE_SIG_PREG_SIZE(vq) * (n))
+#define SVE_SIG_PREGS_SIZE(vq) \
+ (SVE_SIG_PREG_OFFSET(vq, SVE_NUM_PREGS) - SVE_SIG_PREGS_OFFSET(vq))
+
+#define SVE_SIG_FFR_OFFSET(vq) \
+ (SVE_SIG_PREGS_OFFSET(vq) + SVE_SIG_PREGS_SIZE(vq))
+
+#define SVE_SIG_REGS_SIZE(vq) \
+ (SVE_SIG_FFR_OFFSET(vq) + SVE_SIG_FFR_SIZE(vq) - SVE_SIG_REGS_OFFSET)
+
+#define SVE_SIG_CONTEXT_SIZE(vq) (SVE_SIG_REGS_OFFSET + SVE_SIG_REGS_SIZE(vq))
+
+
#endif /* _UAPI__ASM_SIGCONTEXT_H */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 09/28] arm64/sve: Signal frame and context structure definition
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch defines the representation that will be used for the SVE
register state in the signal frame, and implements support for
saving and restoring the SVE registers around signals.
The same layout will also be used for the in-kernel task state.
Due to the variability of the SVE vector length, it is not possible
to define a fixed C struct to describe all the registers. Instead,
Macros are defined in sigcontext.h to facilitate access to the
parts of the structure.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Benn?e:
* Add SVE_VQ_BYTES #define to make it clear when the code is
intentionally referring to the size in bytes of a quadword, and migrate
away from magic 16/0x10 where used with this meaning.
---
arch/arm64/include/uapi/asm/sigcontext.h | 117 ++++++++++++++++++++++++++++++-
1 file changed, 116 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h
index f0a76b9..c78cf8e 100644
--- a/arch/arm64/include/uapi/asm/sigcontext.h
+++ b/arch/arm64/include/uapi/asm/sigcontext.h
@@ -16,6 +16,8 @@
#ifndef _UAPI__ASM_SIGCONTEXT_H
#define _UAPI__ASM_SIGCONTEXT_H
+#ifndef __ASSEMBLY__
+
#include <linux/types.h>
/*
@@ -41,10 +43,11 @@ struct sigcontext {
*
* 0x210 fpsimd_context
* 0x10 esr_context
+ * 0x8a0 sve_context (vl <= 64) (optional)
* 0x20 extra_context (optional)
* 0x10 terminator (null _aarch64_ctx)
*
- * 0xdb0 (reserved for future allocation)
+ * 0x510 (reserved for future allocation)
*
* New records that can exceed this space need to be opt-in for userspace, so
* that an expanded signal frame is not generated unexpectedly. The mechanism
@@ -116,4 +119,116 @@ struct extra_context {
__u32 __reserved[3];
};
+#define SVE_MAGIC 0x53564501
+
+struct sve_context {
+ struct _aarch64_ctx head;
+ __u16 vl;
+ __u16 __reserved[3];
+};
+
+#endif /* !__ASSEMBLY__ */
+
+/*
+ * The SVE architecture leaves space for future expansion of the
+ * vector length beyond its initial architectural limit of 2048 bits
+ * (16 quadwords).
+ */
+#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
+
+#define SVE_VQ_MIN 1
+#define SVE_VQ_MAX 0x200
+
+#define SVE_VL_MIN (SVE_VQ_MIN * SVE_VQ_BYTES)
+#define SVE_VL_MAX (SVE_VQ_MAX * SVE_VQ_BYTES)
+
+#define SVE_NUM_ZREGS 32
+#define SVE_NUM_PREGS 16
+
+#define sve_vl_valid(vl) \
+ ((vl) % SVE_VQ_BYTES == 0 && (vl) >= SVE_VL_MIN && (vl) <= SVE_VL_MAX)
+#define sve_vq_from_vl(vl) ((vl) / SVE_VQ_BYTES)
+#define sve_vl_from_vq(vq) ((vq) * SVE_VQ_BYTES)
+
+/*
+ * If the SVE registers are currently live for the thread@signal delivery,
+ * sve_context.head.size >=
+ * SVE_SIG_CONTEXT_SIZE(sve_vq_from_vl(sve_context.vl))
+ * and the register data may be accessed using the SVE_SIG_*() macros.
+ *
+ * If sve_context.head.size <
+ * SVE_SIG_CONTEXT_SIZE(sve_vq_from_vl(sve_context.vl)),
+ * the SVE registers were not live for the thread and no register data
+ * is included: in this case, the SVE_SIG_*() macros should not be
+ * used except for this check.
+ *
+ * The same convention applies when returning from a signal: a caller
+ * will need to remove or resize the sve_context block if it wants to
+ * make the SVE registers live when they were previously non-live or
+ * vice-versa. This may require the the caller to allocate fresh
+ * memory and/or move other context blocks in the signal frame.
+ *
+ * Changing the vector length during signal return is not permitted:
+ * sve_context.vl must equal the thread's current vector length when
+ * doing a sigreturn.
+ *
+ *
+ * Note: for all these macros, the "vq" argument denotes the SVE
+ * vector length in quadwords (i.e., units of 128 bits).
+ *
+ * The correct way to obtain vq is to use sve_vq_from_vl(vl). The
+ * result is valid if and only if sve_vl_valid(vl) is true. This is
+ * guaranteed for a struct sve_context written by the kernel.
+ *
+ *
+ * Additional macros describe the contents and layout of the payload.
+ * For each, SVE_SIG_x_OFFSET(args) is the start offset relative to
+ * the start of struct sve_context, and SVE_SIG_x_SIZE(args) is the
+ * size in bytes:
+ *
+ * x type description
+ * - ---- -----------
+ * REGS the entire SVE context
+ *
+ * ZREGS __uint128_t[SVE_NUM_ZREGS][vq] all Z-registers
+ * ZREG __uint128_t[vq] individual Z-register Zn
+ *
+ * PREGS uint16_t[SVE_NUM_PREGS][vq] all P-registers
+ * PREG uint16_t[vq] individual P-register Pn
+ *
+ * FFR uint16_t[vq] first-fault status register
+ *
+ * Additional data might be appended in the future.
+ */
+
+#define SVE_SIG_ZREG_SIZE(vq) ((__u32)(vq) * SVE_VQ_BYTES)
+#define SVE_SIG_PREG_SIZE(vq) ((__u32)(vq) * (SVE_VQ_BYTES / 8))
+#define SVE_SIG_FFR_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
+
+#define SVE_SIG_REGS_OFFSET \
+ ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+#define SVE_SIG_ZREGS_OFFSET SVE_SIG_REGS_OFFSET
+#define SVE_SIG_ZREG_OFFSET(vq, n) \
+ (SVE_SIG_ZREGS_OFFSET + SVE_SIG_ZREG_SIZE(vq) * (n))
+#define SVE_SIG_ZREGS_SIZE(vq) \
+ (SVE_SIG_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_SIG_ZREGS_OFFSET)
+
+#define SVE_SIG_PREGS_OFFSET(vq) \
+ (SVE_SIG_ZREGS_OFFSET + SVE_SIG_ZREGS_SIZE(vq))
+#define SVE_SIG_PREG_OFFSET(vq, n) \
+ (SVE_SIG_PREGS_OFFSET(vq) + SVE_SIG_PREG_SIZE(vq) * (n))
+#define SVE_SIG_PREGS_SIZE(vq) \
+ (SVE_SIG_PREG_OFFSET(vq, SVE_NUM_PREGS) - SVE_SIG_PREGS_OFFSET(vq))
+
+#define SVE_SIG_FFR_OFFSET(vq) \
+ (SVE_SIG_PREGS_OFFSET(vq) + SVE_SIG_PREGS_SIZE(vq))
+
+#define SVE_SIG_REGS_SIZE(vq) \
+ (SVE_SIG_FFR_OFFSET(vq) + SVE_SIG_FFR_SIZE(vq) - SVE_SIG_REGS_OFFSET)
+
+#define SVE_SIG_CONTEXT_SIZE(vq) (SVE_SIG_REGS_OFFSET + SVE_SIG_REGS_SIZE(vq))
+
+
#endif /* _UAPI__ASM_SIGCONTEXT_H */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 10/28] arm64/sve: Low-level CPU setup
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
To enable the kernel to use SVE, all SVE traps from EL1 must be
disabled. To take maximum advantage of the hardware, the full
available vector length also needs to be enabled for EL1 by
programming ZCR_EL2.LEN. (The kernel will program ZCR_EL1.LEN as
required, but this cannot override the limit set by ZCR_EL2.)
In advance of full SVE support being implemented for userspace, it
also necessary to ensure that SVE traps from EL0 are enabled.
This patch makes the appropriate changes to the primary and
secondary CPU initialisation code.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
---
arch/arm64/kernel/head.S | 13 ++++++++++++-
arch/arm64/mm/proc.S | 14 ++++++++++++--
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 7434ec0..f411f71 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -516,8 +516,19 @@ CPU_LE( movk x0, #0x30d0, lsl #16 ) // Clear EE and E0E on LE systems
mov x0, #0x33ff
msr cptr_el2, x0 // Disable copro. traps to EL2
+ /* SVE register access */
+ mrs x1, id_aa64pfr0_el1
+ ubfx x1, x1, #ID_AA64PFR0_SVE_SHIFT, #4
+ cbz x1, 7f
+
+ bic x0, x0, #CPTR_EL2_TZ // Also disable SVE traps
+ msr cptr_el2, x0 // Disable copro. traps to EL2
+ isb
+ mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector
+ msr_s SYS_ZCR_EL2, x1 // length for EL1.
+
/* Hypervisor stub */
- adr_l x0, __hyp_stub_vectors
+7: adr_l x0, __hyp_stub_vectors
msr vbar_el2, x0
/* spsr */
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 877d42f..dd22ef2 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -27,6 +27,7 @@
#include <asm/pgtable-hwdef.h>
#include <asm/cpufeature.h>
#include <asm/alternative.h>
+#include <asm/sysreg.h>
#ifdef CONFIG_ARM64_64K_PAGES
#define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
@@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
tlbi vmalle1 // Invalidate local TLB
dsb nsh
- mov x0, #3 << 20
- msr cpacr_el1, x0 // Enable FP/ASIMD
+ mov x0, #3 << 20 // FEN
+
+ /* SVE */
+ mrs x5, id_aa64pfr0_el1
+ ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
+ cbz x5, 1f
+
+ bic x0, x0, #CPACR_EL1_ZEN
+ orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
+1: msr cpacr_el1, x0 // Enable FP/ASIMD
+
mov x0, #1 << 12 // Reset mdscr_el1 and disable
msr mdscr_el1, x0 // access to the DCC from EL0
isb // Unmask debug exceptions now,
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
To enable the kernel to use SVE, all SVE traps from EL1 must be
disabled. To take maximum advantage of the hardware, the full
available vector length also needs to be enabled for EL1 by
programming ZCR_EL2.LEN. (The kernel will program ZCR_EL1.LEN as
required, but this cannot override the limit set by ZCR_EL2.)
In advance of full SVE support being implemented for userspace, it
also necessary to ensure that SVE traps from EL0 are enabled.
This patch makes the appropriate changes to the primary and
secondary CPU initialisation code.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
---
arch/arm64/kernel/head.S | 13 ++++++++++++-
arch/arm64/mm/proc.S | 14 ++++++++++++--
2 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 7434ec0..f411f71 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -516,8 +516,19 @@ CPU_LE( movk x0, #0x30d0, lsl #16 ) // Clear EE and E0E on LE systems
mov x0, #0x33ff
msr cptr_el2, x0 // Disable copro. traps to EL2
+ /* SVE register access */
+ mrs x1, id_aa64pfr0_el1
+ ubfx x1, x1, #ID_AA64PFR0_SVE_SHIFT, #4
+ cbz x1, 7f
+
+ bic x0, x0, #CPTR_EL2_TZ // Also disable SVE traps
+ msr cptr_el2, x0 // Disable copro. traps to EL2
+ isb
+ mov x1, #ZCR_ELx_LEN_MASK // SVE: Enable full vector
+ msr_s SYS_ZCR_EL2, x1 // length for EL1.
+
/* Hypervisor stub */
- adr_l x0, __hyp_stub_vectors
+7: adr_l x0, __hyp_stub_vectors
msr vbar_el2, x0
/* spsr */
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 877d42f..dd22ef2 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -27,6 +27,7 @@
#include <asm/pgtable-hwdef.h>
#include <asm/cpufeature.h>
#include <asm/alternative.h>
+#include <asm/sysreg.h>
#ifdef CONFIG_ARM64_64K_PAGES
#define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
@@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
tlbi vmalle1 // Invalidate local TLB
dsb nsh
- mov x0, #3 << 20
- msr cpacr_el1, x0 // Enable FP/ASIMD
+ mov x0, #3 << 20 // FEN
+
+ /* SVE */
+ mrs x5, id_aa64pfr0_el1
+ ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
+ cbz x5, 1f
+
+ bic x0, x0, #CPACR_EL1_ZEN
+ orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
+1: msr cpacr_el1, x0 // Enable FP/ASIMD
+
mov x0, #1 << 12 // Reset mdscr_el1 and disable
msr mdscr_el1, x0 // access to the DCC from EL0
isb // Unmask debug exceptions now,
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
This patch adds the core support for switching and managing the SVE
architectural state of user tasks.
Calls to the existing FPSIMD low-level save/restore functions are
factored out as new functions task_fpsimd_{save,load}(), since SVE
now dynamically may or may not need to be handled at these points
depending on the kernel configuration, hardware features discovered
at boot, and the runtime state of the task. To make these
decisions as fast as possible, const cpucaps are used where
feasible, via the system_supports_sve() helper.
The SVE registers are only tracked for threads that have explicitly
used SVE, indicated by the new thread flag TIF_SVE. Otherwise, the
FPSIMD view of the architectural state is stored in
thread.fpsimd_state as usual.
When in use, the SVE registers are not stored directly in
thread_struct due to their potentially large and variable size.
Because the task_struct slab allocator must be configured very
early during kernel boot, it is also tricky to configure it
correctly to match the maximum vector length provided by the
hardware, since this depends on examining secondary CPUs as well as
the primary. Instead, a pointer sve_state in thread_struct points
to a dynamically allocated buffer containing the SVE register data,
and code is added to allocate, duplicate and free this buffer at
appropriate times.
TIF_SVE is set when taking an SVE access trap from userspace, if
suitable hardware support has been detected. This enables SVE for
the thread: a subsequent return to userspace will disable the trap
accordingly. If such a trap is taken without sufficient hardware
support, SIGILL is sent to the thread instead as if an undefined
instruction had been executed: this may happen if userspace tries
to use SVE in a system where not all CPUs support it for example.
The kernel may clear TIF_SVE and disable SVE for the thread
whenever an explicit syscall is made by userspace, though this is
considered an optimisation opportunity rather than a deterministic
guarantee: the kernel may not do this on every syscall, but it is
permitted to do so. For backwards compatibility reasons and
conformance with the spirit of the base AArch64 procedure call
standard, the subset of the SVE register state that aliases the
FPSIMD registers is still preserved across a syscall even if this
happens.
TIF_SVE is also cleared, and SVE disabled, on exec: this is an
obvious slow path and a hint that we are running a new binary that
may not use SVE.
Code is added to sync data between thread.fpsimd_state and
thread.sve_state whenever enabling/disabling SVE, in a manner
consistent with the SVE architectural programmer's model.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Ard Biesheuvel:
* Fix unbalanced ifelse bracing to match kernel coding style.
* Improve abstraction of change_cpacr to be a bit less clunky.
* Rearrange task_fpsimd_{load,save}() and friends to eliminate
forward declarations.
Changes related to Ard Biesheuvel's comments:
* Undo suprious replacement of assignment by memset in
arch_dup_task_struct().
Requested by Alex Bennée:
* Add missing include of <linux/compat.h>.
* Make thread_struct.sve_vl an unsigned int: that's the type used
virtually everywhere else, and the memory saving is too minor to be
interesting.
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code. The fpsimd_dup_sve() case is changed
to a WARN_ON, for user threads only, and only if actually clearing
TIF_SVE.
Other:
* [bugfix] Unconditionally drop child task's reference to the parent's
SVE state on fork. Otherwise we can end up with it aliased, which is
bad.
---
arch/arm64/include/asm/fpsimd.h | 19 +++
arch/arm64/include/asm/processor.h | 2 +
arch/arm64/include/asm/thread_info.h | 1 +
arch/arm64/include/asm/traps.h | 2 +
arch/arm64/kernel/entry.S | 14 ++-
arch/arm64/kernel/fpsimd.c | 227 ++++++++++++++++++++++++++++++++++-
arch/arm64/kernel/process.c | 4 +
arch/arm64/kernel/traps.c | 4 +-
8 files changed, 265 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 026a7c7..72090a1 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -20,6 +20,8 @@
#ifndef __ASSEMBLY__
+#include <linux/stddef.h>
+
/*
* FP/SIMD storage area has:
* - FPSR and FPCR
@@ -72,6 +74,23 @@ extern void sve_load_state(void const *state, u32 const *pfpsr,
unsigned long vq_minus_1);
extern unsigned int sve_get_vl(void);
+#ifdef CONFIG_ARM64_SVE
+
+extern size_t sve_state_size(struct task_struct const *task);
+
+extern void sve_alloc(struct task_struct *task);
+extern void fpsimd_release_thread(struct task_struct *task);
+extern void fpsimd_dup_sve(struct task_struct *dst,
+ struct task_struct const *src);
+
+#else /* ! CONFIG_ARM64_SVE */
+
+static void __maybe_unused sve_alloc(struct task_struct *task) { }
+static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
+static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
+ struct task_struct const *src) { }
+#endif /* ! CONFIG_ARM64_SVE */
+
/* For use by EFI runtime services calls only */
extern void __efi_fpsimd_begin(void);
extern void __efi_fpsimd_end(void);
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 29adab8..4831d28 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -85,6 +85,8 @@ struct thread_struct {
unsigned long tp2_value;
#endif
struct fpsimd_state fpsimd_state;
+ void *sve_state; /* SVE registers, if any */
+ unsigned int sve_vl; /* SVE vector length */
unsigned long fault_address; /* fault info */
unsigned long fault_code; /* ESR_EL1 value */
struct debug_info debug; /* debugging */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 2eca178..f0880fc 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -91,6 +91,7 @@ void arch_setup_new_exec(void);
#define TIF_RESTORE_SIGMASK 20
#define TIF_SINGLESTEP 21
#define TIF_32BIT 22 /* 32bit process */
+#define TIF_SVE 23 /* Scalable Vector Extension in use */
#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
diff --git a/arch/arm64/include/asm/traps.h b/arch/arm64/include/asm/traps.h
index 4136168..282163f 100644
--- a/arch/arm64/include/asm/traps.h
+++ b/arch/arm64/include/asm/traps.h
@@ -34,6 +34,8 @@ struct undef_hook {
void register_undef_hook(struct undef_hook *hook);
void unregister_undef_hook(struct undef_hook *hook);
+void force_signal_inject(int signal, int code, struct pt_regs *regs,
+ unsigned long address);
void arm64_notify_segfault(struct pt_regs *regs, unsigned long addr);
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index e1c59d4..e57020f 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -607,6 +607,8 @@ el0_sync:
b.eq el0_ia
cmp x24, #ESR_ELx_EC_FP_ASIMD // FP/ASIMD access
b.eq el0_fpsimd_acc
+ cmp x24, #ESR_ELx_EC_SVE // SVE access
+ b.eq el0_sve_acc
cmp x24, #ESR_ELx_EC_FP_EXC64 // FP/ASIMD exception
b.eq el0_fpsimd_exc
cmp x24, #ESR_ELx_EC_SYS64 // configurable trap
@@ -705,9 +707,19 @@ el0_fpsimd_acc:
mov x1, sp
bl do_fpsimd_acc
b ret_to_user
+el0_sve_acc:
+ /*
+ * Scalable Vector Extension access
+ */
+ enable_dbg
+ ct_user_exit
+ mov x0, x25
+ mov x1, sp
+ bl do_sve_acc
+ b ret_to_user
el0_fpsimd_exc:
/*
- * Floating Point or Advanced SIMD exception
+ * Floating Point, Advanced SIMD or SVE exception
*/
enable_dbg
ct_user_exit
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 9d762dd..9b1ebd7 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -18,18 +18,25 @@
*/
#include <linux/bottom_half.h>
+#include <linux/bug.h>
+#include <linux/compat.h>
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/percpu.h>
#include <linux/preempt.h>
+#include <linux/ptrace.h>
#include <linux/sched/signal.h>
#include <linux/signal.h>
+#include <linux/slab.h>
#include <asm/fpsimd.h>
#include <asm/cputype.h>
#include <asm/simd.h>
+#include <asm/sigcontext.h>
+#include <asm/sysreg.h>
+#include <asm/traps.h>
#define FPEXC_IOF (1 << 0)
#define FPEXC_DZF (1 << 1)
@@ -99,6 +106,190 @@
*/
static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
+static void sve_free(struct task_struct *task)
+{
+ kfree(task->thread.sve_state);
+ task->thread.sve_state = NULL;
+}
+
+/* Offset of FFR in the SVE register dump */
+static size_t sve_ffr_offset(int vl)
+{
+ return SVE_SIG_FFR_OFFSET(sve_vq_from_vl(vl)) - SVE_SIG_REGS_OFFSET;
+}
+
+static void *sve_pffr(struct task_struct *task)
+{
+ return (char *)task->thread.sve_state +
+ sve_ffr_offset(task->thread.sve_vl);
+}
+
+static void change_cpacr(u64 val, u64 mask)
+{
+ u64 cpacr = read_sysreg(CPACR_EL1);
+ u64 new = (cpacr & ~mask) | val;
+
+ if (new != cpacr)
+ write_sysreg(new, CPACR_EL1);
+}
+
+static void sve_user_disable(void)
+{
+ change_cpacr(0, CPACR_EL1_ZEN_EL0EN);
+}
+
+static void sve_user_enable(void)
+{
+ change_cpacr(CPACR_EL1_ZEN_EL0EN, CPACR_EL1_ZEN_EL0EN);
+}
+
+static void task_fpsimd_load(void)
+{
+ if (system_supports_sve() && test_thread_flag(TIF_SVE))
+ sve_load_state(sve_pffr(current),
+ ¤t->thread.fpsimd_state.fpsr,
+ sve_vq_from_vl(current->thread.sve_vl) - 1);
+ else
+ fpsimd_load_state(¤t->thread.fpsimd_state);
+
+ if (system_supports_sve()) {
+ /* Toggle SVE trapping for userspace if needed */
+ if (test_thread_flag(TIF_SVE))
+ sve_user_enable();
+ else
+ sve_user_disable();
+
+ /* Serialised by exception return to user */
+ }
+}
+
+static void task_fpsimd_save(void)
+{
+ if (system_supports_sve() &&
+ in_syscall(current_pt_regs()) &&
+ test_thread_flag(TIF_SVE)) {
+ clear_thread_flag(TIF_SVE);
+
+ /* Trap if the task tries to use SVE again: */
+ sve_user_disable();
+ }
+
+ if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) {
+ if (system_supports_sve() && test_thread_flag(TIF_SVE))
+ sve_save_state(sve_pffr(current),
+ ¤t->thread.fpsimd_state.fpsr);
+ else
+ fpsimd_save_state(¤t->thread.fpsimd_state);
+ }
+}
+
+#define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
+ (SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
+
+static void fpsimd_to_sve(struct task_struct *task)
+{
+ unsigned int vq;
+ void *sst = task->thread.sve_state;
+ struct fpsimd_state const *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!system_supports_sve())
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+ for (i = 0; i < 32; ++i)
+ memcpy(ZREG(sst, vq, i), &fst->vregs[i],
+ sizeof(fst->vregs[i]));
+}
+
+#ifdef CONFIG_ARM64_SVE
+
+static void sve_to_fpsimd(struct task_struct *task)
+{
+ unsigned int vq;
+ void const *sst = task->thread.sve_state;
+ struct fpsimd_state *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!system_supports_sve())
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+ for (i = 0; i < 32; ++i)
+ memcpy(&fst->vregs[i], ZREG(sst, vq, i),
+ sizeof(fst->vregs[i]));
+}
+
+size_t sve_state_size(struct task_struct const *task)
+{
+ return SVE_SIG_REGS_SIZE(sve_vq_from_vl(task->thread.sve_vl));
+}
+
+void sve_alloc(struct task_struct *task)
+{
+ if (task->thread.sve_state) {
+ memset(task->thread.sve_state, 0, sve_state_size(current));
+ return;
+ }
+
+ /* This is a small allocation (maximum ~8KB) and Should Not Fail. */
+ task->thread.sve_state =
+ kzalloc(sve_state_size(task), GFP_KERNEL);
+
+ /*
+ * If future SVE revisions can have larger vectors though,
+ * this may cease to be true:
+ */
+ BUG_ON(!task->thread.sve_state);
+}
+
+/*
+ * Handle SVE state across fork():
+ *
+ * dst and src must not end up with aliases of the same sve_state.
+ * Because a task cannot fork except in a syscall, we can discard SVE
+ * state for dst here, so long as we take care to retain the FPSIMD
+ * subset of the state if SVE is in use. Reallocation of the SVE state
+ * will be deferred until dst tries to use SVE.
+ */
+void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
+{
+ if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
+ WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
+ sve_to_fpsimd(dst);
+ }
+
+ dst->thread.sve_state = NULL;
+}
+
+void fpsimd_release_thread(struct task_struct *dead_task)
+{
+ sve_free(dead_task);
+}
+
+#endif /* CONFIG_ARM64_SVE */
+
+/*
+ * Trapped SVE access
+ */
+void do_sve_acc(unsigned int esr, struct pt_regs *regs)
+{
+ /* Even if we chose not to use SVE, the hardware could still trap: */
+ if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
+ force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
+ return;
+ }
+
+ task_fpsimd_save();
+
+ sve_alloc(current);
+ fpsimd_to_sve(current);
+ if (test_and_set_thread_flag(TIF_SVE))
+ WARN_ON(1); /* SVE access shouldn't have trapped */
+
+ task_fpsimd_load();
+}
+
/*
* Trapped FP/ASIMD access.
*/
@@ -144,8 +335,8 @@ void fpsimd_thread_switch(struct task_struct *next)
* the registers is in fact the most recent userland FPSIMD state of
* 'current'.
*/
- if (current->mm && !test_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ if (current->mm)
+ task_fpsimd_save();
if (next->mm) {
/*
@@ -167,6 +358,8 @@ void fpsimd_thread_switch(struct task_struct *next)
void fpsimd_flush_thread(void)
{
+ int vl;
+
if (!system_supports_fpsimd())
return;
@@ -174,6 +367,30 @@ void fpsimd_flush_thread(void)
memset(¤t->thread.fpsimd_state, 0, sizeof(struct fpsimd_state));
fpsimd_flush_task_state(current);
+
+ if (system_supports_sve()) {
+ clear_thread_flag(TIF_SVE);
+ sve_free(current);
+
+ /*
+ * Reset the task vector length as required.
+ * This is where we ensure that all user tasks have a valid
+ * vector length configured: no kernel task can become a user
+ * task without an exec and hence a call to this function.
+ * If a bug causes this to go wrong, we make some noise and
+ * try to fudge thread.sve_vl to a safe value here.
+ */
+ vl = current->thread.sve_vl;
+
+ if (vl == 0)
+ vl = SVE_VL_MIN;
+
+ if (WARN_ON(!sve_vl_valid(vl)))
+ vl = SVE_VL_MIN;
+
+ current->thread.sve_vl = vl;
+ }
+
set_thread_flag(TIF_FOREIGN_FPSTATE);
local_bh_enable();
@@ -211,7 +428,7 @@ void fpsimd_restore_current_state(void)
if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
struct fpsimd_state *st = ¤t->thread.fpsimd_state;
- fpsimd_load_state(st);
+ task_fpsimd_load();
__this_cpu_write(fpsimd_last_state, st);
st->cpu = smp_processor_id();
}
@@ -376,8 +593,8 @@ static int fpsimd_cpu_pm_notifier(struct notifier_block *self,
{
switch (cmd) {
case CPU_PM_ENTER:
- if (current->mm && !test_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ if (current->mm)
+ task_fpsimd_save();
this_cpu_write(fpsimd_last_state, NULL);
break;
case CPU_PM_EXIT:
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index e6bf19c..eb1cae7 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -239,6 +239,7 @@ void flush_thread(void)
void release_thread(struct task_struct *dead_task)
{
+ fpsimd_release_thread(dead_task);
}
int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
@@ -246,6 +247,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
if (current->mm)
fpsimd_preserve_current_state();
*dst = *src;
+
+ fpsimd_dup_sve(dst, src);
+
return 0;
}
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index f202932..7f3b5c6 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -358,8 +358,8 @@ static int call_undef_hook(struct pt_regs *regs)
return fn ? fn(regs, instr) : 1;
}
-static void force_signal_inject(int signal, int code, struct pt_regs *regs,
- unsigned long address)
+void force_signal_inject(int signal, int code, struct pt_regs *regs,
+ unsigned long address)
{
siginfo_t info;
void __user *pc = (void __user *)instruction_pointer(regs);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch adds the core support for switching and managing the SVE
architectural state of user tasks.
Calls to the existing FPSIMD low-level save/restore functions are
factored out as new functions task_fpsimd_{save,load}(), since SVE
now dynamically may or may not need to be handled at these points
depending on the kernel configuration, hardware features discovered
at boot, and the runtime state of the task. To make these
decisions as fast as possible, const cpucaps are used where
feasible, via the system_supports_sve() helper.
The SVE registers are only tracked for threads that have explicitly
used SVE, indicated by the new thread flag TIF_SVE. Otherwise, the
FPSIMD view of the architectural state is stored in
thread.fpsimd_state as usual.
When in use, the SVE registers are not stored directly in
thread_struct due to their potentially large and variable size.
Because the task_struct slab allocator must be configured very
early during kernel boot, it is also tricky to configure it
correctly to match the maximum vector length provided by the
hardware, since this depends on examining secondary CPUs as well as
the primary. Instead, a pointer sve_state in thread_struct points
to a dynamically allocated buffer containing the SVE register data,
and code is added to allocate, duplicate and free this buffer at
appropriate times.
TIF_SVE is set when taking an SVE access trap from userspace, if
suitable hardware support has been detected. This enables SVE for
the thread: a subsequent return to userspace will disable the trap
accordingly. If such a trap is taken without sufficient hardware
support, SIGILL is sent to the thread instead as if an undefined
instruction had been executed: this may happen if userspace tries
to use SVE in a system where not all CPUs support it for example.
The kernel may clear TIF_SVE and disable SVE for the thread
whenever an explicit syscall is made by userspace, though this is
considered an optimisation opportunity rather than a deterministic
guarantee: the kernel may not do this on every syscall, but it is
permitted to do so. For backwards compatibility reasons and
conformance with the spirit of the base AArch64 procedure call
standard, the subset of the SVE register state that aliases the
FPSIMD registers is still preserved across a syscall even if this
happens.
TIF_SVE is also cleared, and SVE disabled, on exec: this is an
obvious slow path and a hint that we are running a new binary that
may not use SVE.
Code is added to sync data between thread.fpsimd_state and
thread.sve_state whenever enabling/disabling SVE, in a manner
consistent with the SVE architectural programmer's model.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Ard Biesheuvel:
* Fix unbalanced ifelse bracing to match kernel coding style.
* Improve abstraction of change_cpacr to be a bit less clunky.
* Rearrange task_fpsimd_{load,save}() and friends to eliminate
forward declarations.
Changes related to Ard Biesheuvel's comments:
* Undo suprious replacement of assignment by memset in
arch_dup_task_struct().
Requested by Alex Benn?e:
* Add missing include of <linux/compat.h>.
* Make thread_struct.sve_vl an unsigned int: that's the type used
virtually everywhere else, and the memory saving is too minor to be
interesting.
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code. The fpsimd_dup_sve() case is changed
to a WARN_ON, for user threads only, and only if actually clearing
TIF_SVE.
Other:
* [bugfix] Unconditionally drop child task's reference to the parent's
SVE state on fork. Otherwise we can end up with it aliased, which is
bad.
---
arch/arm64/include/asm/fpsimd.h | 19 +++
arch/arm64/include/asm/processor.h | 2 +
arch/arm64/include/asm/thread_info.h | 1 +
arch/arm64/include/asm/traps.h | 2 +
arch/arm64/kernel/entry.S | 14 ++-
arch/arm64/kernel/fpsimd.c | 227 ++++++++++++++++++++++++++++++++++-
arch/arm64/kernel/process.c | 4 +
arch/arm64/kernel/traps.c | 4 +-
8 files changed, 265 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 026a7c7..72090a1 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -20,6 +20,8 @@
#ifndef __ASSEMBLY__
+#include <linux/stddef.h>
+
/*
* FP/SIMD storage area has:
* - FPSR and FPCR
@@ -72,6 +74,23 @@ extern void sve_load_state(void const *state, u32 const *pfpsr,
unsigned long vq_minus_1);
extern unsigned int sve_get_vl(void);
+#ifdef CONFIG_ARM64_SVE
+
+extern size_t sve_state_size(struct task_struct const *task);
+
+extern void sve_alloc(struct task_struct *task);
+extern void fpsimd_release_thread(struct task_struct *task);
+extern void fpsimd_dup_sve(struct task_struct *dst,
+ struct task_struct const *src);
+
+#else /* ! CONFIG_ARM64_SVE */
+
+static void __maybe_unused sve_alloc(struct task_struct *task) { }
+static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
+static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
+ struct task_struct const *src) { }
+#endif /* ! CONFIG_ARM64_SVE */
+
/* For use by EFI runtime services calls only */
extern void __efi_fpsimd_begin(void);
extern void __efi_fpsimd_end(void);
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 29adab8..4831d28 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -85,6 +85,8 @@ struct thread_struct {
unsigned long tp2_value;
#endif
struct fpsimd_state fpsimd_state;
+ void *sve_state; /* SVE registers, if any */
+ unsigned int sve_vl; /* SVE vector length */
unsigned long fault_address; /* fault info */
unsigned long fault_code; /* ESR_EL1 value */
struct debug_info debug; /* debugging */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index 2eca178..f0880fc 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -91,6 +91,7 @@ void arch_setup_new_exec(void);
#define TIF_RESTORE_SIGMASK 20
#define TIF_SINGLESTEP 21
#define TIF_32BIT 22 /* 32bit process */
+#define TIF_SVE 23 /* Scalable Vector Extension in use */
#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
diff --git a/arch/arm64/include/asm/traps.h b/arch/arm64/include/asm/traps.h
index 4136168..282163f 100644
--- a/arch/arm64/include/asm/traps.h
+++ b/arch/arm64/include/asm/traps.h
@@ -34,6 +34,8 @@ struct undef_hook {
void register_undef_hook(struct undef_hook *hook);
void unregister_undef_hook(struct undef_hook *hook);
+void force_signal_inject(int signal, int code, struct pt_regs *regs,
+ unsigned long address);
void arm64_notify_segfault(struct pt_regs *regs, unsigned long addr);
diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S
index e1c59d4..e57020f 100644
--- a/arch/arm64/kernel/entry.S
+++ b/arch/arm64/kernel/entry.S
@@ -607,6 +607,8 @@ el0_sync:
b.eq el0_ia
cmp x24, #ESR_ELx_EC_FP_ASIMD // FP/ASIMD access
b.eq el0_fpsimd_acc
+ cmp x24, #ESR_ELx_EC_SVE // SVE access
+ b.eq el0_sve_acc
cmp x24, #ESR_ELx_EC_FP_EXC64 // FP/ASIMD exception
b.eq el0_fpsimd_exc
cmp x24, #ESR_ELx_EC_SYS64 // configurable trap
@@ -705,9 +707,19 @@ el0_fpsimd_acc:
mov x1, sp
bl do_fpsimd_acc
b ret_to_user
+el0_sve_acc:
+ /*
+ * Scalable Vector Extension access
+ */
+ enable_dbg
+ ct_user_exit
+ mov x0, x25
+ mov x1, sp
+ bl do_sve_acc
+ b ret_to_user
el0_fpsimd_exc:
/*
- * Floating Point or Advanced SIMD exception
+ * Floating Point, Advanced SIMD or SVE exception
*/
enable_dbg
ct_user_exit
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 9d762dd..9b1ebd7 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -18,18 +18,25 @@
*/
#include <linux/bottom_half.h>
+#include <linux/bug.h>
+#include <linux/compat.h>
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/percpu.h>
#include <linux/preempt.h>
+#include <linux/ptrace.h>
#include <linux/sched/signal.h>
#include <linux/signal.h>
+#include <linux/slab.h>
#include <asm/fpsimd.h>
#include <asm/cputype.h>
#include <asm/simd.h>
+#include <asm/sigcontext.h>
+#include <asm/sysreg.h>
+#include <asm/traps.h>
#define FPEXC_IOF (1 << 0)
#define FPEXC_DZF (1 << 1)
@@ -99,6 +106,190 @@
*/
static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
+static void sve_free(struct task_struct *task)
+{
+ kfree(task->thread.sve_state);
+ task->thread.sve_state = NULL;
+}
+
+/* Offset of FFR in the SVE register dump */
+static size_t sve_ffr_offset(int vl)
+{
+ return SVE_SIG_FFR_OFFSET(sve_vq_from_vl(vl)) - SVE_SIG_REGS_OFFSET;
+}
+
+static void *sve_pffr(struct task_struct *task)
+{
+ return (char *)task->thread.sve_state +
+ sve_ffr_offset(task->thread.sve_vl);
+}
+
+static void change_cpacr(u64 val, u64 mask)
+{
+ u64 cpacr = read_sysreg(CPACR_EL1);
+ u64 new = (cpacr & ~mask) | val;
+
+ if (new != cpacr)
+ write_sysreg(new, CPACR_EL1);
+}
+
+static void sve_user_disable(void)
+{
+ change_cpacr(0, CPACR_EL1_ZEN_EL0EN);
+}
+
+static void sve_user_enable(void)
+{
+ change_cpacr(CPACR_EL1_ZEN_EL0EN, CPACR_EL1_ZEN_EL0EN);
+}
+
+static void task_fpsimd_load(void)
+{
+ if (system_supports_sve() && test_thread_flag(TIF_SVE))
+ sve_load_state(sve_pffr(current),
+ ¤t->thread.fpsimd_state.fpsr,
+ sve_vq_from_vl(current->thread.sve_vl) - 1);
+ else
+ fpsimd_load_state(¤t->thread.fpsimd_state);
+
+ if (system_supports_sve()) {
+ /* Toggle SVE trapping for userspace if needed */
+ if (test_thread_flag(TIF_SVE))
+ sve_user_enable();
+ else
+ sve_user_disable();
+
+ /* Serialised by exception return to user */
+ }
+}
+
+static void task_fpsimd_save(void)
+{
+ if (system_supports_sve() &&
+ in_syscall(current_pt_regs()) &&
+ test_thread_flag(TIF_SVE)) {
+ clear_thread_flag(TIF_SVE);
+
+ /* Trap if the task tries to use SVE again: */
+ sve_user_disable();
+ }
+
+ if (!test_thread_flag(TIF_FOREIGN_FPSTATE)) {
+ if (system_supports_sve() && test_thread_flag(TIF_SVE))
+ sve_save_state(sve_pffr(current),
+ ¤t->thread.fpsimd_state.fpsr);
+ else
+ fpsimd_save_state(¤t->thread.fpsimd_state);
+ }
+}
+
+#define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
+ (SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
+
+static void fpsimd_to_sve(struct task_struct *task)
+{
+ unsigned int vq;
+ void *sst = task->thread.sve_state;
+ struct fpsimd_state const *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!system_supports_sve())
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+ for (i = 0; i < 32; ++i)
+ memcpy(ZREG(sst, vq, i), &fst->vregs[i],
+ sizeof(fst->vregs[i]));
+}
+
+#ifdef CONFIG_ARM64_SVE
+
+static void sve_to_fpsimd(struct task_struct *task)
+{
+ unsigned int vq;
+ void const *sst = task->thread.sve_state;
+ struct fpsimd_state *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!system_supports_sve())
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+ for (i = 0; i < 32; ++i)
+ memcpy(&fst->vregs[i], ZREG(sst, vq, i),
+ sizeof(fst->vregs[i]));
+}
+
+size_t sve_state_size(struct task_struct const *task)
+{
+ return SVE_SIG_REGS_SIZE(sve_vq_from_vl(task->thread.sve_vl));
+}
+
+void sve_alloc(struct task_struct *task)
+{
+ if (task->thread.sve_state) {
+ memset(task->thread.sve_state, 0, sve_state_size(current));
+ return;
+ }
+
+ /* This is a small allocation (maximum ~8KB) and Should Not Fail. */
+ task->thread.sve_state =
+ kzalloc(sve_state_size(task), GFP_KERNEL);
+
+ /*
+ * If future SVE revisions can have larger vectors though,
+ * this may cease to be true:
+ */
+ BUG_ON(!task->thread.sve_state);
+}
+
+/*
+ * Handle SVE state across fork():
+ *
+ * dst and src must not end up with aliases of the same sve_state.
+ * Because a task cannot fork except in a syscall, we can discard SVE
+ * state for dst here, so long as we take care to retain the FPSIMD
+ * subset of the state if SVE is in use. Reallocation of the SVE state
+ * will be deferred until dst tries to use SVE.
+ */
+void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
+{
+ if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
+ WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
+ sve_to_fpsimd(dst);
+ }
+
+ dst->thread.sve_state = NULL;
+}
+
+void fpsimd_release_thread(struct task_struct *dead_task)
+{
+ sve_free(dead_task);
+}
+
+#endif /* CONFIG_ARM64_SVE */
+
+/*
+ * Trapped SVE access
+ */
+void do_sve_acc(unsigned int esr, struct pt_regs *regs)
+{
+ /* Even if we chose not to use SVE, the hardware could still trap: */
+ if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
+ force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
+ return;
+ }
+
+ task_fpsimd_save();
+
+ sve_alloc(current);
+ fpsimd_to_sve(current);
+ if (test_and_set_thread_flag(TIF_SVE))
+ WARN_ON(1); /* SVE access shouldn't have trapped */
+
+ task_fpsimd_load();
+}
+
/*
* Trapped FP/ASIMD access.
*/
@@ -144,8 +335,8 @@ void fpsimd_thread_switch(struct task_struct *next)
* the registers is in fact the most recent userland FPSIMD state of
* 'current'.
*/
- if (current->mm && !test_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ if (current->mm)
+ task_fpsimd_save();
if (next->mm) {
/*
@@ -167,6 +358,8 @@ void fpsimd_thread_switch(struct task_struct *next)
void fpsimd_flush_thread(void)
{
+ int vl;
+
if (!system_supports_fpsimd())
return;
@@ -174,6 +367,30 @@ void fpsimd_flush_thread(void)
memset(¤t->thread.fpsimd_state, 0, sizeof(struct fpsimd_state));
fpsimd_flush_task_state(current);
+
+ if (system_supports_sve()) {
+ clear_thread_flag(TIF_SVE);
+ sve_free(current);
+
+ /*
+ * Reset the task vector length as required.
+ * This is where we ensure that all user tasks have a valid
+ * vector length configured: no kernel task can become a user
+ * task without an exec and hence a call to this function.
+ * If a bug causes this to go wrong, we make some noise and
+ * try to fudge thread.sve_vl to a safe value here.
+ */
+ vl = current->thread.sve_vl;
+
+ if (vl == 0)
+ vl = SVE_VL_MIN;
+
+ if (WARN_ON(!sve_vl_valid(vl)))
+ vl = SVE_VL_MIN;
+
+ current->thread.sve_vl = vl;
+ }
+
set_thread_flag(TIF_FOREIGN_FPSTATE);
local_bh_enable();
@@ -211,7 +428,7 @@ void fpsimd_restore_current_state(void)
if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
struct fpsimd_state *st = ¤t->thread.fpsimd_state;
- fpsimd_load_state(st);
+ task_fpsimd_load();
__this_cpu_write(fpsimd_last_state, st);
st->cpu = smp_processor_id();
}
@@ -376,8 +593,8 @@ static int fpsimd_cpu_pm_notifier(struct notifier_block *self,
{
switch (cmd) {
case CPU_PM_ENTER:
- if (current->mm && !test_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ if (current->mm)
+ task_fpsimd_save();
this_cpu_write(fpsimd_last_state, NULL);
break;
case CPU_PM_EXIT:
diff --git a/arch/arm64/kernel/process.c b/arch/arm64/kernel/process.c
index e6bf19c..eb1cae7 100644
--- a/arch/arm64/kernel/process.c
+++ b/arch/arm64/kernel/process.c
@@ -239,6 +239,7 @@ void flush_thread(void)
void release_thread(struct task_struct *dead_task)
{
+ fpsimd_release_thread(dead_task);
}
int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
@@ -246,6 +247,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
if (current->mm)
fpsimd_preserve_current_state();
*dst = *src;
+
+ fpsimd_dup_sve(dst, src);
+
return 0;
}
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index f202932..7f3b5c6 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -358,8 +358,8 @@ static int call_undef_hook(struct pt_regs *regs)
return fn ? fn(regs, instr) : 1;
}
-static void force_signal_inject(int signal, int code, struct pt_regs *regs,
- unsigned long address)
+void force_signal_inject(int signal, int code, struct pt_regs *regs,
+ unsigned long address)
{
siginfo_t info;
void __user *pc = (void __user *)instruction_pointer(regs);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 12/28] arm64/sve: Support vector length resetting for new processes
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
It's desirable to be able to reset the vector length to some sane
default for new processes, since the new binary and its libraries
processes may or may not be SVE-aware.
This patch tracks the desired post-exec vector length (if any) in a
new thread member sve_vl_onexec, and adds a new thread flag
TIF_SVE_VL_INHERIT to control whether to inherit or reset the
vector length. Currently these are inactive. Subsequent patches
will provide the capability to configure them.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Bennée:
* Make sve_vl_onexec an unsigned int: that's the type used virtually
everywhere else, and the memory saving is too minor to be interesting.
---
arch/arm64/include/asm/processor.h | 1 +
arch/arm64/include/asm/thread_info.h | 1 +
arch/arm64/kernel/fpsimd.c | 16 ++++++++++++----
3 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 4831d28..3faceac 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -87,6 +87,7 @@ struct thread_struct {
struct fpsimd_state fpsimd_state;
void *sve_state; /* SVE registers, if any */
unsigned int sve_vl; /* SVE vector length */
+ unsigned int sve_vl_onexec; /* SVE vl after next exec */
unsigned long fault_address; /* fault info */
unsigned long fault_code; /* ESR_EL1 value */
struct debug_info debug; /* debugging */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index f0880fc..d3568ab 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -92,6 +92,7 @@ void arch_setup_new_exec(void);
#define TIF_SINGLESTEP 21
#define TIF_32BIT 22 /* 32bit process */
#define TIF_SVE 23 /* Scalable Vector Extension in use */
+#define TIF_SVE_VL_INHERIT 24 /* Inherit sve_vl_onexec across exec */
#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 9b1ebd7..e20b44d 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -106,6 +106,9 @@
*/
static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
+/* Default VL for tasks that don't set it explicitly: */
+static int sve_default_vl = SVE_VL_MIN;
+
static void sve_free(struct task_struct *task)
{
kfree(task->thread.sve_state);
@@ -380,15 +383,20 @@ void fpsimd_flush_thread(void)
* If a bug causes this to go wrong, we make some noise and
* try to fudge thread.sve_vl to a safe value here.
*/
- vl = current->thread.sve_vl;
-
- if (vl == 0)
- vl = SVE_VL_MIN;
+ vl = current->thread.sve_vl_onexec ?
+ current->thread.sve_vl_onexec : sve_default_vl;
if (WARN_ON(!sve_vl_valid(vl)))
vl = SVE_VL_MIN;
current->thread.sve_vl = vl;
+
+ /*
+ * If the task is not set to inherit, ensure that the vector
+ * length will be reset by a subsequent exec:
+ */
+ if (!test_thread_flag(TIF_SVE_VL_INHERIT))
+ current->thread.sve_vl_onexec = 0;
}
set_thread_flag(TIF_FOREIGN_FPSTATE);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 12/28] arm64/sve: Support vector length resetting for new processes
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
It's desirable to be able to reset the vector length to some sane
default for new processes, since the new binary and its libraries
processes may or may not be SVE-aware.
This patch tracks the desired post-exec vector length (if any) in a
new thread member sve_vl_onexec, and adds a new thread flag
TIF_SVE_VL_INHERIT to control whether to inherit or reset the
vector length. Currently these are inactive. Subsequent patches
will provide the capability to configure them.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Benn?e:
* Make sve_vl_onexec an unsigned int: that's the type used virtually
everywhere else, and the memory saving is too minor to be interesting.
---
arch/arm64/include/asm/processor.h | 1 +
arch/arm64/include/asm/thread_info.h | 1 +
arch/arm64/kernel/fpsimd.c | 16 ++++++++++++----
3 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 4831d28..3faceac 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -87,6 +87,7 @@ struct thread_struct {
struct fpsimd_state fpsimd_state;
void *sve_state; /* SVE registers, if any */
unsigned int sve_vl; /* SVE vector length */
+ unsigned int sve_vl_onexec; /* SVE vl after next exec */
unsigned long fault_address; /* fault info */
unsigned long fault_code; /* ESR_EL1 value */
struct debug_info debug; /* debugging */
diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
index f0880fc..d3568ab 100644
--- a/arch/arm64/include/asm/thread_info.h
+++ b/arch/arm64/include/asm/thread_info.h
@@ -92,6 +92,7 @@ void arch_setup_new_exec(void);
#define TIF_SINGLESTEP 21
#define TIF_32BIT 22 /* 32bit process */
#define TIF_SVE 23 /* Scalable Vector Extension in use */
+#define TIF_SVE_VL_INHERIT 24 /* Inherit sve_vl_onexec across exec */
#define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
#define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 9b1ebd7..e20b44d 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -106,6 +106,9 @@
*/
static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
+/* Default VL for tasks that don't set it explicitly: */
+static int sve_default_vl = SVE_VL_MIN;
+
static void sve_free(struct task_struct *task)
{
kfree(task->thread.sve_state);
@@ -380,15 +383,20 @@ void fpsimd_flush_thread(void)
* If a bug causes this to go wrong, we make some noise and
* try to fudge thread.sve_vl to a safe value here.
*/
- vl = current->thread.sve_vl;
-
- if (vl == 0)
- vl = SVE_VL_MIN;
+ vl = current->thread.sve_vl_onexec ?
+ current->thread.sve_vl_onexec : sve_default_vl;
if (WARN_ON(!sve_vl_valid(vl)))
vl = SVE_VL_MIN;
current->thread.sve_vl = vl;
+
+ /*
+ * If the task is not set to inherit, ensure that the vector
+ * length will be reset by a subsequent exec:
+ */
+ if (!test_thread_flag(TIF_SVE_VL_INHERIT))
+ current->thread.sve_vl_onexec = 0;
}
set_thread_flag(TIF_FOREIGN_FPSTATE);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 13/28] arm64/sve: Signal handling support
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
This patch implements support for saving and restoring the SVE
registers around signals.
A fixed-size header struct sve_context is always included in the
signal frame encoding the thread's vector length at the time of
signal delivery, optionally followed by a variable-layout structure
encoding the SVE registers.
Because of the need to preserve backwards compatibility, the FPSIMD
view of the SVE registers is always dumped as a struct
fpsimd_context in the usual way, in addition to any sve_context.
The SVE vector registers are dumped in full, including bits 127:0
of each register which alias the corresponding FPSIMD vector
registers in the hardware. To avoid any ambiguity about which
alias to restore during sigreturn, the kernel always restores bits
127:0 of each SVE vector register from the fpsimd_context in the
signal frame (which must be present): userspace needs to take this
into account if it wants to modify the SVE vector register contents
on return from a signal.
FPSR and FPCR, which are used by both FPSIMD and SVE, are not
included in sve_context because they are always present in
fpsimd_context anyway.
For signal delivery, a new helper
fpsimd_signal_preserve_current_state() is added to update _both_
the FPSIMD and SVE views in the task struct, to make it easier to
populate this information into the signal frame. Because of the
redundancy between the two views of the state, only one is updated
otherwise. In order to avoid racing with a pending discard of the
SVE state, this flush is hoisted before the sigframe layout phase,
so that the layout and population phases see a consistent view of
the thread.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Ard Biesheuvel:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
Requested by Alex Bennée:
* Add comments to explain the barrier()s when writing thread.sve_state
in restore_sve_fpsimd_context().
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other:
* Delete spurious barrier() from restore_sve_fpsimd_context().
Once TIF_FOREIGN_FPSTATE is set, context switch won't write to
thread_struct any more, so it doesn't matter what TIF_SVE is until the
next ret_to_user or ptrace interaction, neither of which can race
with this function.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/fpsimd.h | 1 +
arch/arm64/kernel/fpsimd.c | 23 ++++--
arch/arm64/kernel/signal.c | 173 ++++++++++++++++++++++++++++++++++++++--
arch/arm64/kernel/signal32.c | 2 +-
4 files changed, 183 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 72090a1..7efd04e 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -63,6 +63,7 @@ extern void fpsimd_load_state(struct fpsimd_state *state);
extern void fpsimd_thread_switch(struct task_struct *next);
extern void fpsimd_flush_thread(void);
+extern void fpsimd_signal_preserve_current_state(void);
extern void fpsimd_preserve_current_state(void);
extern void fpsimd_restore_current_state(void);
extern void fpsimd_update_current_state(struct fpsimd_state *state);
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index e20b44d..f82cde8 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -205,8 +205,6 @@ static void fpsimd_to_sve(struct task_struct *task)
sizeof(fst->vregs[i]));
}
-#ifdef CONFIG_ARM64_SVE
-
static void sve_to_fpsimd(struct task_struct *task)
{
unsigned int vq;
@@ -223,6 +221,8 @@ static void sve_to_fpsimd(struct task_struct *task)
sizeof(fst->vregs[i]));
}
+#ifdef CONFIG_ARM64_SVE
+
size_t sve_state_size(struct task_struct const *task)
{
return SVE_SIG_REGS_SIZE(sve_vq_from_vl(task->thread.sve_vl));
@@ -414,13 +414,17 @@ void fpsimd_preserve_current_state(void)
return;
local_bh_disable();
-
- if (!test_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
-
+ task_fpsimd_save();
local_bh_enable();
}
+void fpsimd_signal_preserve_current_state(void)
+{
+ fpsimd_preserve_current_state();
+ if (system_supports_sve() && test_thread_flag(TIF_SVE))
+ sve_to_fpsimd(current);
+}
+
/*
* Load the userland FPSIMD state of 'current' from memory, but only if the
* FPSIMD state already held in the registers is /not/ the most recent FPSIMD
@@ -456,7 +460,12 @@ void fpsimd_update_current_state(struct fpsimd_state *state)
local_bh_disable();
- fpsimd_load_state(state);
+ if (system_supports_sve() && test_thread_flag(TIF_SVE)) {
+ current->thread.fpsimd_state = *state;
+ fpsimd_to_sve(current);
+ }
+ task_fpsimd_load();
+
if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
struct fpsimd_state *st = ¤t->thread.fpsimd_state;
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 4991e87..c3f94cf 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -62,6 +62,7 @@ struct rt_sigframe_user_layout {
unsigned long fpsimd_offset;
unsigned long esr_offset;
+ unsigned long sve_offset;
unsigned long extra_offset;
unsigned long end_offset;
};
@@ -178,9 +179,6 @@ static int preserve_fpsimd_context(struct fpsimd_context __user *ctx)
struct fpsimd_state *fpsimd = ¤t->thread.fpsimd_state;
int err;
- /* dump the hardware registers to the fpsimd_state structure */
- fpsimd_preserve_current_state();
-
/* copy the FP and status/control registers */
err = __copy_to_user(ctx->vregs, fpsimd->vregs, sizeof(fpsimd->vregs));
__put_user_error(fpsimd->fpsr, &ctx->fpsr, err);
@@ -213,6 +211,8 @@ static int restore_fpsimd_context(struct fpsimd_context __user *ctx)
__get_user_error(fpsimd.fpsr, &ctx->fpsr, err);
__get_user_error(fpsimd.fpcr, &ctx->fpcr, err);
+ clear_thread_flag(TIF_SVE);
+
/* load the hardware registers from the fpsimd_state structure */
if (!err)
fpsimd_update_current_state(&fpsimd);
@@ -220,10 +220,118 @@ static int restore_fpsimd_context(struct fpsimd_context __user *ctx)
return err ? -EFAULT : 0;
}
+
struct user_ctxs {
struct fpsimd_context __user *fpsimd;
+ struct sve_context __user *sve;
};
+#ifdef CONFIG_ARM64_SVE
+
+static int preserve_sve_context(struct sve_context __user *ctx)
+{
+ int err = 0;
+ u16 reserved[ARRAY_SIZE(ctx->__reserved)];
+ unsigned int vl = current->thread.sve_vl;
+ unsigned int vq = 0;
+
+ if (test_thread_flag(TIF_SVE))
+ vq = sve_vq_from_vl(vl);
+
+ memset(reserved, 0, sizeof(reserved));
+
+ __put_user_error(SVE_MAGIC, &ctx->head.magic, err);
+ __put_user_error(round_up(SVE_SIG_CONTEXT_SIZE(vq), 16),
+ &ctx->head.size, err);
+ __put_user_error(vl, &ctx->vl, err);
+ BUILD_BUG_ON(sizeof(ctx->__reserved) != sizeof(reserved));
+ err |= copy_to_user(&ctx->__reserved, reserved, sizeof(reserved));
+
+ if (vq) {
+ /*
+ * This assumes that the SVE state has already been saved to
+ * the task struct by calling preserve_fpsimd_context().
+ */
+ err |= copy_to_user((char __user *)ctx + SVE_SIG_REGS_OFFSET,
+ current->thread.sve_state,
+ SVE_SIG_REGS_SIZE(vq));
+ }
+
+ return err ? -EFAULT : 0;
+}
+
+static int restore_sve_fpsimd_context(struct user_ctxs *user)
+{
+ int err;
+ unsigned int vq;
+ struct fpsimd_state fpsimd;
+ struct sve_context sve;
+
+ if (__copy_from_user(&sve, user->sve, sizeof(sve)))
+ return -EFAULT;
+
+ if (sve.vl != current->thread.sve_vl)
+ return -EINVAL;
+
+ if (sve.head.size <= sizeof(*user->sve)) {
+ clear_thread_flag(TIF_SVE);
+ goto fpsimd_only;
+ }
+
+ vq = sve_vq_from_vl(sve.vl);
+
+ if (sve.head.size < SVE_SIG_CONTEXT_SIZE(vq))
+ return -EINVAL;
+
+ /*
+ * Careful: we are about __copy_from_user() directly into
+ * thread.sve_state with preemption enabled, so protection is
+ * needed to prevent a racing context switch from writing stale
+ * registers back over the new data.
+ */
+
+ fpsimd_flush_task_state(current);
+ barrier();
+ /* From now, fpsimd_thread_switch() won't clear TIF_FOREIGN_FPSTATE */
+
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
+ barrier();
+ /* From now, fpsimd_thread_switch() won't touch thread.sve_state */
+
+ sve_alloc(current);
+ err = __copy_from_user(current->thread.sve_state,
+ (char __user const *)user->sve +
+ SVE_SIG_REGS_OFFSET,
+ SVE_SIG_REGS_SIZE(vq));
+ if (err)
+ return err;
+
+ set_thread_flag(TIF_SVE);
+
+fpsimd_only:
+ /* copy the FP and status/control registers */
+ /* restore_sigframe() already checked that user->fpsimd != NULL. */
+ err = __copy_from_user(fpsimd.vregs, user->fpsimd->vregs,
+ sizeof(fpsimd.vregs));
+ __get_user_error(fpsimd.fpsr, &user->fpsimd->fpsr, err);
+ __get_user_error(fpsimd.fpcr, &user->fpsimd->fpcr, err);
+
+ /* load the hardware registers from the fpsimd_state structure */
+ if (!err)
+ fpsimd_update_current_state(&fpsimd);
+
+ return err;
+}
+
+#else /* ! CONFIG_ARM64_SVE */
+
+/* Turn any non-optimised out attempts to use these into a link error: */
+extern int preserve_sve_context(void __user *ctx);
+extern int restore_sve_fpsimd_context(struct user_ctxs *user);
+
+#endif /* ! CONFIG_ARM64_SVE */
+
+
static int parse_user_sigframe(struct user_ctxs *user,
struct rt_sigframe __user *sf)
{
@@ -236,6 +344,7 @@ static int parse_user_sigframe(struct user_ctxs *user,
char const __user *const sfp = (char const __user *)sf;
user->fpsimd = NULL;
+ user->sve = NULL;
if (!IS_ALIGNED((unsigned long)base, 16))
goto invalid;
@@ -286,6 +395,19 @@ static int parse_user_sigframe(struct user_ctxs *user,
/* ignore */
break;
+ case SVE_MAGIC:
+ if (!system_supports_sve())
+ goto invalid;
+
+ if (user->sve)
+ goto invalid;
+
+ if (size < sizeof(*user->sve))
+ goto invalid;
+
+ user->sve = (struct sve_context __user *)head;
+ break;
+
case EXTRA_MAGIC:
if (have_extra_context)
goto invalid;
@@ -358,9 +480,6 @@ static int parse_user_sigframe(struct user_ctxs *user,
}
done:
- if (!user->fpsimd)
- goto invalid;
-
return 0;
invalid:
@@ -394,8 +513,19 @@ static int restore_sigframe(struct pt_regs *regs,
if (err == 0)
err = parse_user_sigframe(&user, sf);
- if (err == 0)
- err = restore_fpsimd_context(user.fpsimd);
+ if (err == 0) {
+ if (!user.fpsimd)
+ return -EINVAL;
+
+ if (user.sve) {
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ err = restore_sve_fpsimd_context(&user);
+ } else {
+ err = restore_fpsimd_context(user.fpsimd);
+ }
+ }
return err;
}
@@ -454,6 +584,18 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return err;
}
+ if (system_supports_sve()) {
+ unsigned int vq = 0;
+
+ if (test_thread_flag(TIF_SVE))
+ vq = sve_vq_from_vl(current->thread.sve_vl);
+
+ err = sigframe_alloc(user, &user->sve_offset,
+ SVE_SIG_CONTEXT_SIZE(vq));
+ if (err)
+ return err;
+ }
+
return sigframe_alloc_end(user);
}
@@ -495,6 +637,13 @@ static int setup_sigframe(struct rt_sigframe_user_layout *user,
__put_user_error(current->thread.fault_code, &esr_ctx->esr, err);
}
+ /* Scalable Vector Extension state, if present */
+ if (system_supports_sve() && err == 0 && user->sve_offset) {
+ struct sve_context __user *sve_ctx =
+ apply_user_offset(user, user->sve_offset);
+ err |= preserve_sve_context(sve_ctx);
+ }
+
if (err == 0 && user->extra_offset) {
char __user *sfp = (char __user *)user->sigframe;
char __user *userp =
@@ -594,6 +743,14 @@ static int setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct rt_sigframe __user *frame;
int err = 0;
+ /*
+ * Ensure FPSIMD/SVE state in task_struct is up-to-date.
+ * This is needed here in order to complete any pending SVE discard:
+ * otherwise, discard may occur between deciding on the sigframe
+ * layout and dumping the register data.
+ */
+ fpsimd_signal_preserve_current_state();
+
if (get_sigframe(&user, ksig, regs))
return 1;
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 4e5a664..202337d 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -244,7 +244,7 @@ static int compat_preserve_vfp_context(struct compat_vfp_sigframe __user *frame)
* Note that this also saves V16-31, which aren't visible
* in AArch32.
*/
- fpsimd_preserve_current_state();
+ fpsimd_signal_preserve_current_state();
/* Place structure header on the stack */
__put_user_error(magic, &frame->magic, err);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 13/28] arm64/sve: Signal handling support
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch implements support for saving and restoring the SVE
registers around signals.
A fixed-size header struct sve_context is always included in the
signal frame encoding the thread's vector length at the time of
signal delivery, optionally followed by a variable-layout structure
encoding the SVE registers.
Because of the need to preserve backwards compatibility, the FPSIMD
view of the SVE registers is always dumped as a struct
fpsimd_context in the usual way, in addition to any sve_context.
The SVE vector registers are dumped in full, including bits 127:0
of each register which alias the corresponding FPSIMD vector
registers in the hardware. To avoid any ambiguity about which
alias to restore during sigreturn, the kernel always restores bits
127:0 of each SVE vector register from the fpsimd_context in the
signal frame (which must be present): userspace needs to take this
into account if it wants to modify the SVE vector register contents
on return from a signal.
FPSR and FPCR, which are used by both FPSIMD and SVE, are not
included in sve_context because they are always present in
fpsimd_context anyway.
For signal delivery, a new helper
fpsimd_signal_preserve_current_state() is added to update _both_
the FPSIMD and SVE views in the task struct, to make it easier to
populate this information into the signal frame. Because of the
redundancy between the two views of the state, only one is updated
otherwise. In order to avoid racing with a pending discard of the
SVE state, this flush is hoisted before the sigframe layout phase,
so that the layout and population phases see a consistent view of
the thread.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Ard Biesheuvel:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
Requested by Alex Benn?e:
* Add comments to explain the barrier()s when writing thread.sve_state
in restore_sve_fpsimd_context().
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other:
* Delete spurious barrier() from restore_sve_fpsimd_context().
Once TIF_FOREIGN_FPSTATE is set, context switch won't write to
thread_struct any more, so it doesn't matter what TIF_SVE is until the
next ret_to_user or ptrace interaction, neither of which can race
with this function.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/fpsimd.h | 1 +
arch/arm64/kernel/fpsimd.c | 23 ++++--
arch/arm64/kernel/signal.c | 173 ++++++++++++++++++++++++++++++++++++++--
arch/arm64/kernel/signal32.c | 2 +-
4 files changed, 183 insertions(+), 16 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 72090a1..7efd04e 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -63,6 +63,7 @@ extern void fpsimd_load_state(struct fpsimd_state *state);
extern void fpsimd_thread_switch(struct task_struct *next);
extern void fpsimd_flush_thread(void);
+extern void fpsimd_signal_preserve_current_state(void);
extern void fpsimd_preserve_current_state(void);
extern void fpsimd_restore_current_state(void);
extern void fpsimd_update_current_state(struct fpsimd_state *state);
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index e20b44d..f82cde8 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -205,8 +205,6 @@ static void fpsimd_to_sve(struct task_struct *task)
sizeof(fst->vregs[i]));
}
-#ifdef CONFIG_ARM64_SVE
-
static void sve_to_fpsimd(struct task_struct *task)
{
unsigned int vq;
@@ -223,6 +221,8 @@ static void sve_to_fpsimd(struct task_struct *task)
sizeof(fst->vregs[i]));
}
+#ifdef CONFIG_ARM64_SVE
+
size_t sve_state_size(struct task_struct const *task)
{
return SVE_SIG_REGS_SIZE(sve_vq_from_vl(task->thread.sve_vl));
@@ -414,13 +414,17 @@ void fpsimd_preserve_current_state(void)
return;
local_bh_disable();
-
- if (!test_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
-
+ task_fpsimd_save();
local_bh_enable();
}
+void fpsimd_signal_preserve_current_state(void)
+{
+ fpsimd_preserve_current_state();
+ if (system_supports_sve() && test_thread_flag(TIF_SVE))
+ sve_to_fpsimd(current);
+}
+
/*
* Load the userland FPSIMD state of 'current' from memory, but only if the
* FPSIMD state already held in the registers is /not/ the most recent FPSIMD
@@ -456,7 +460,12 @@ void fpsimd_update_current_state(struct fpsimd_state *state)
local_bh_disable();
- fpsimd_load_state(state);
+ if (system_supports_sve() && test_thread_flag(TIF_SVE)) {
+ current->thread.fpsimd_state = *state;
+ fpsimd_to_sve(current);
+ }
+ task_fpsimd_load();
+
if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
struct fpsimd_state *st = ¤t->thread.fpsimd_state;
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 4991e87..c3f94cf 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -62,6 +62,7 @@ struct rt_sigframe_user_layout {
unsigned long fpsimd_offset;
unsigned long esr_offset;
+ unsigned long sve_offset;
unsigned long extra_offset;
unsigned long end_offset;
};
@@ -178,9 +179,6 @@ static int preserve_fpsimd_context(struct fpsimd_context __user *ctx)
struct fpsimd_state *fpsimd = ¤t->thread.fpsimd_state;
int err;
- /* dump the hardware registers to the fpsimd_state structure */
- fpsimd_preserve_current_state();
-
/* copy the FP and status/control registers */
err = __copy_to_user(ctx->vregs, fpsimd->vregs, sizeof(fpsimd->vregs));
__put_user_error(fpsimd->fpsr, &ctx->fpsr, err);
@@ -213,6 +211,8 @@ static int restore_fpsimd_context(struct fpsimd_context __user *ctx)
__get_user_error(fpsimd.fpsr, &ctx->fpsr, err);
__get_user_error(fpsimd.fpcr, &ctx->fpcr, err);
+ clear_thread_flag(TIF_SVE);
+
/* load the hardware registers from the fpsimd_state structure */
if (!err)
fpsimd_update_current_state(&fpsimd);
@@ -220,10 +220,118 @@ static int restore_fpsimd_context(struct fpsimd_context __user *ctx)
return err ? -EFAULT : 0;
}
+
struct user_ctxs {
struct fpsimd_context __user *fpsimd;
+ struct sve_context __user *sve;
};
+#ifdef CONFIG_ARM64_SVE
+
+static int preserve_sve_context(struct sve_context __user *ctx)
+{
+ int err = 0;
+ u16 reserved[ARRAY_SIZE(ctx->__reserved)];
+ unsigned int vl = current->thread.sve_vl;
+ unsigned int vq = 0;
+
+ if (test_thread_flag(TIF_SVE))
+ vq = sve_vq_from_vl(vl);
+
+ memset(reserved, 0, sizeof(reserved));
+
+ __put_user_error(SVE_MAGIC, &ctx->head.magic, err);
+ __put_user_error(round_up(SVE_SIG_CONTEXT_SIZE(vq), 16),
+ &ctx->head.size, err);
+ __put_user_error(vl, &ctx->vl, err);
+ BUILD_BUG_ON(sizeof(ctx->__reserved) != sizeof(reserved));
+ err |= copy_to_user(&ctx->__reserved, reserved, sizeof(reserved));
+
+ if (vq) {
+ /*
+ * This assumes that the SVE state has already been saved to
+ * the task struct by calling preserve_fpsimd_context().
+ */
+ err |= copy_to_user((char __user *)ctx + SVE_SIG_REGS_OFFSET,
+ current->thread.sve_state,
+ SVE_SIG_REGS_SIZE(vq));
+ }
+
+ return err ? -EFAULT : 0;
+}
+
+static int restore_sve_fpsimd_context(struct user_ctxs *user)
+{
+ int err;
+ unsigned int vq;
+ struct fpsimd_state fpsimd;
+ struct sve_context sve;
+
+ if (__copy_from_user(&sve, user->sve, sizeof(sve)))
+ return -EFAULT;
+
+ if (sve.vl != current->thread.sve_vl)
+ return -EINVAL;
+
+ if (sve.head.size <= sizeof(*user->sve)) {
+ clear_thread_flag(TIF_SVE);
+ goto fpsimd_only;
+ }
+
+ vq = sve_vq_from_vl(sve.vl);
+
+ if (sve.head.size < SVE_SIG_CONTEXT_SIZE(vq))
+ return -EINVAL;
+
+ /*
+ * Careful: we are about __copy_from_user() directly into
+ * thread.sve_state with preemption enabled, so protection is
+ * needed to prevent a racing context switch from writing stale
+ * registers back over the new data.
+ */
+
+ fpsimd_flush_task_state(current);
+ barrier();
+ /* From now, fpsimd_thread_switch() won't clear TIF_FOREIGN_FPSTATE */
+
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
+ barrier();
+ /* From now, fpsimd_thread_switch() won't touch thread.sve_state */
+
+ sve_alloc(current);
+ err = __copy_from_user(current->thread.sve_state,
+ (char __user const *)user->sve +
+ SVE_SIG_REGS_OFFSET,
+ SVE_SIG_REGS_SIZE(vq));
+ if (err)
+ return err;
+
+ set_thread_flag(TIF_SVE);
+
+fpsimd_only:
+ /* copy the FP and status/control registers */
+ /* restore_sigframe() already checked that user->fpsimd != NULL. */
+ err = __copy_from_user(fpsimd.vregs, user->fpsimd->vregs,
+ sizeof(fpsimd.vregs));
+ __get_user_error(fpsimd.fpsr, &user->fpsimd->fpsr, err);
+ __get_user_error(fpsimd.fpcr, &user->fpsimd->fpcr, err);
+
+ /* load the hardware registers from the fpsimd_state structure */
+ if (!err)
+ fpsimd_update_current_state(&fpsimd);
+
+ return err;
+}
+
+#else /* ! CONFIG_ARM64_SVE */
+
+/* Turn any non-optimised out attempts to use these into a link error: */
+extern int preserve_sve_context(void __user *ctx);
+extern int restore_sve_fpsimd_context(struct user_ctxs *user);
+
+#endif /* ! CONFIG_ARM64_SVE */
+
+
static int parse_user_sigframe(struct user_ctxs *user,
struct rt_sigframe __user *sf)
{
@@ -236,6 +344,7 @@ static int parse_user_sigframe(struct user_ctxs *user,
char const __user *const sfp = (char const __user *)sf;
user->fpsimd = NULL;
+ user->sve = NULL;
if (!IS_ALIGNED((unsigned long)base, 16))
goto invalid;
@@ -286,6 +395,19 @@ static int parse_user_sigframe(struct user_ctxs *user,
/* ignore */
break;
+ case SVE_MAGIC:
+ if (!system_supports_sve())
+ goto invalid;
+
+ if (user->sve)
+ goto invalid;
+
+ if (size < sizeof(*user->sve))
+ goto invalid;
+
+ user->sve = (struct sve_context __user *)head;
+ break;
+
case EXTRA_MAGIC:
if (have_extra_context)
goto invalid;
@@ -358,9 +480,6 @@ static int parse_user_sigframe(struct user_ctxs *user,
}
done:
- if (!user->fpsimd)
- goto invalid;
-
return 0;
invalid:
@@ -394,8 +513,19 @@ static int restore_sigframe(struct pt_regs *regs,
if (err == 0)
err = parse_user_sigframe(&user, sf);
- if (err == 0)
- err = restore_fpsimd_context(user.fpsimd);
+ if (err == 0) {
+ if (!user.fpsimd)
+ return -EINVAL;
+
+ if (user.sve) {
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ err = restore_sve_fpsimd_context(&user);
+ } else {
+ err = restore_fpsimd_context(user.fpsimd);
+ }
+ }
return err;
}
@@ -454,6 +584,18 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return err;
}
+ if (system_supports_sve()) {
+ unsigned int vq = 0;
+
+ if (test_thread_flag(TIF_SVE))
+ vq = sve_vq_from_vl(current->thread.sve_vl);
+
+ err = sigframe_alloc(user, &user->sve_offset,
+ SVE_SIG_CONTEXT_SIZE(vq));
+ if (err)
+ return err;
+ }
+
return sigframe_alloc_end(user);
}
@@ -495,6 +637,13 @@ static int setup_sigframe(struct rt_sigframe_user_layout *user,
__put_user_error(current->thread.fault_code, &esr_ctx->esr, err);
}
+ /* Scalable Vector Extension state, if present */
+ if (system_supports_sve() && err == 0 && user->sve_offset) {
+ struct sve_context __user *sve_ctx =
+ apply_user_offset(user, user->sve_offset);
+ err |= preserve_sve_context(sve_ctx);
+ }
+
if (err == 0 && user->extra_offset) {
char __user *sfp = (char __user *)user->sigframe;
char __user *userp =
@@ -594,6 +743,14 @@ static int setup_rt_frame(int usig, struct ksignal *ksig, sigset_t *set,
struct rt_sigframe __user *frame;
int err = 0;
+ /*
+ * Ensure FPSIMD/SVE state in task_struct is up-to-date.
+ * This is needed here in order to complete any pending SVE discard:
+ * otherwise, discard may occur between deciding on the sigframe
+ * layout and dumping the register data.
+ */
+ fpsimd_signal_preserve_current_state();
+
if (get_sigframe(&user, ksig, regs))
return 1;
diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c
index 4e5a664..202337d 100644
--- a/arch/arm64/kernel/signal32.c
+++ b/arch/arm64/kernel/signal32.c
@@ -244,7 +244,7 @@ static int compat_preserve_vfp_context(struct compat_vfp_sigframe __user *frame)
* Note that this also saves V16-31, which aren't visible
* in AArch32.
*/
- fpsimd_preserve_current_state();
+ fpsimd_signal_preserve_current_state();
/* Place structure header on the stack */
__put_user_error(magic, &frame->magic, err);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
gdb, Alan Hayward, Yao Qi
This patch implements the core logic for changing a task's vector
length on request from userspace. This will be used by the ptrace
and prctl frontends that are implemented in later patches.
The SVE architecture permits, but does not require, implementations
to support vector lengths that are not a power of two. To handle
this, logic is added to check a requested vector length against a
possibly sparse bitmap of available vector lengths at runtime, so
that the best supported value can be chosen.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Bennée:
* Comment the definition of SVE_VL_ARCH_MAX.
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other changes related Alex Bennée's comments:
* sve_max_vl is definitely not supposed to be changed after boot.
Make it official by marking it __ro_after_init.
* Migrate away from magic number for SVE_VQ_BYTES.
---
arch/arm64/include/asm/fpsimd.h | 8 +++
arch/arm64/kernel/fpsimd.c | 128 +++++++++++++++++++++++++++++++++++++++-
include/uapi/linux/prctl.h | 5 ++
3 files changed, 140 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 7efd04e..32c8e19 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -20,6 +20,7 @@
#ifndef __ASSEMBLY__
+#include <linux/cache.h>
#include <linux/stddef.h>
/*
@@ -70,11 +71,16 @@ extern void fpsimd_update_current_state(struct fpsimd_state *state);
extern void fpsimd_flush_task_state(struct task_struct *target);
+/* Maximum VL that SVE VL-agnostic software can transparently support */
+#define SVE_VL_ARCH_MAX 0x100
+
extern void sve_save_state(void *state, u32 *pfpsr);
extern void sve_load_state(void const *state, u32 const *pfpsr,
unsigned long vq_minus_1);
extern unsigned int sve_get_vl(void);
+extern int __ro_after_init sve_max_vl;
+
#ifdef CONFIG_ARM64_SVE
extern size_t sve_state_size(struct task_struct const *task);
@@ -83,6 +89,8 @@ extern void sve_alloc(struct task_struct *task);
extern void fpsimd_release_thread(struct task_struct *task);
extern void fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src);
+extern int sve_set_vector_length(struct task_struct *task,
+ unsigned long vl, unsigned long flags);
#else /* ! CONFIG_ARM64_SVE */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index f82cde8..713476e 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -17,8 +17,10 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/bitmap.h>
#include <linux/bottom_half.h>
#include <linux/bug.h>
+#include <linux/cache.h>
#include <linux/compat.h>
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
@@ -26,6 +28,7 @@
#include <linux/init.h>
#include <linux/percpu.h>
#include <linux/preempt.h>
+#include <linux/prctl.h>
#include <linux/ptrace.h>
#include <linux/sched/signal.h>
#include <linux/signal.h>
@@ -109,6 +112,20 @@ static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
/* Default VL for tasks that don't set it explicitly: */
static int sve_default_vl = SVE_VL_MIN;
+#ifdef CONFIG_ARM64_SVE
+
+/* Maximum supported vector length across all CPUs (initially poisoned) */
+int __ro_after_init sve_max_vl = -1;
+/* Set of available vector lengths, as vq_to_bit(vq): */
+static DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+
+#else /* ! CONFIG_ARM64_SVE */
+
+/* Dummy declaration for code that will be optimised out: */
+extern DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+
+#endif /* ! CONFIG_ARM64_SVE */
+
static void sve_free(struct task_struct *task)
{
kfree(task->thread.sve_state);
@@ -186,6 +203,44 @@ static void task_fpsimd_save(void)
}
}
+static unsigned int vq_to_bit(unsigned int vq)
+{
+ return SVE_VQ_MAX - vq;
+}
+
+static unsigned int bit_to_vq(unsigned int bit)
+{
+ if (WARN_ON(bit >= SVE_VQ_MAX))
+ bit = SVE_VQ_MAX - 1;
+
+ return SVE_VQ_MAX - bit;
+}
+
+/*
+ * All vector length selection from userspace comes through here.
+ * We're on a slow path, so some sanity-checks are included.
+ * If things go wrong there's a bug somewhere, but try to fall back to a
+ * safe choice.
+ */
+static unsigned int find_supported_vector_length(unsigned int vl)
+{
+ int bit;
+ int max_vl = sve_max_vl;
+
+ if (WARN_ON(!sve_vl_valid(vl)))
+ vl = SVE_VL_MIN;
+
+ if (WARN_ON(!sve_vl_valid(max_vl)))
+ max_vl = SVE_VL_MIN;
+
+ if (vl > max_vl)
+ vl = max_vl;
+
+ bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
+ vq_to_bit(sve_vq_from_vl(vl)));
+ return sve_vl_from_vq(bit_to_vq(bit));
+}
+
#define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
(SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
@@ -265,6 +320,73 @@ void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
dst->thread.sve_state = NULL;
}
+int sve_set_vector_length(struct task_struct *task,
+ unsigned long vl, unsigned long flags)
+{
+ WARN_ON(task == current && preemptible());
+
+ if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
+ PR_SVE_SET_VL_ONEXEC))
+ return -EINVAL;
+
+ if (!sve_vl_valid(vl))
+ return -EINVAL;
+
+ /*
+ * Clamp to the maximum vector length that VL-agnostic SVE code can
+ * work with. A flag may be assigned in the future to allow setting
+ * of larger vector lengths without confusing older software.
+ */
+ if (vl > SVE_VL_ARCH_MAX)
+ vl = SVE_VL_ARCH_MAX;
+
+ vl = find_supported_vector_length(vl);
+
+ if (flags & (PR_SVE_VL_INHERIT |
+ PR_SVE_SET_VL_ONEXEC))
+ task->thread.sve_vl_onexec = vl;
+ else
+ /* Reset VL to system default on next exec: */
+ task->thread.sve_vl_onexec = 0;
+
+ /* Only actually set the VL if not deferred: */
+ if (flags & PR_SVE_SET_VL_ONEXEC)
+ goto out;
+
+ /*
+ * To ensure the FPSIMD bits of the SVE vector registers are preserved,
+ * write any live register state back to task_struct, and convert to a
+ * non-SVE thread.
+ */
+ if (vl != task->thread.sve_vl) {
+ if (task == current) {
+ task_fpsimd_save();
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
+ }
+
+ if (test_and_clear_tsk_thread_flag(task, TIF_SVE))
+ sve_to_fpsimd(task);
+
+ /*
+ * Force reallocation of task SVE state to the correct size
+ * on next use:
+ */
+ sve_free(task);
+ }
+
+ task->thread.sve_vl = vl;
+
+ fpsimd_flush_task_state(task);
+
+out:
+ if (flags & PR_SVE_VL_INHERIT)
+ set_thread_flag(TIF_SVE_VL_INHERIT);
+ else
+ clear_thread_flag(TIF_SVE_VL_INHERIT);
+
+ return 0;
+}
+
void fpsimd_release_thread(struct task_struct *dead_task)
{
sve_free(dead_task);
@@ -361,7 +483,7 @@ void fpsimd_thread_switch(struct task_struct *next)
void fpsimd_flush_thread(void)
{
- int vl;
+ int vl, supported_vl;
if (!system_supports_fpsimd())
return;
@@ -389,6 +511,10 @@ void fpsimd_flush_thread(void)
if (WARN_ON(!sve_vl_valid(vl)))
vl = SVE_VL_MIN;
+ supported_vl = find_supported_vector_length(vl);
+ if (WARN_ON(supported_vl != vl))
+ vl = supported_vl;
+
current->thread.sve_vl = vl;
/*
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index a8d0759..1b64901 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -197,4 +197,9 @@ struct prctl_mm_map {
# define PR_CAP_AMBIENT_LOWER 3
# define PR_CAP_AMBIENT_CLEAR_ALL 4
+/* arm64 Scalable Vector Extension controls */
+# define PR_SVE_SET_VL_ONEXEC (1 << 18) /* defer effect until exec */
+# define PR_SVE_VL_LEN_MASK 0xffff
+# define PR_SVE_VL_INHERIT (1 << 17) /* inherit across exec */
+
#endif /* _LINUX_PRCTL_H */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch implements the core logic for changing a task's vector
length on request from userspace. This will be used by the ptrace
and prctl frontends that are implemented in later patches.
The SVE architecture permits, but does not require, implementations
to support vector lengths that are not a power of two. To handle
this, logic is added to check a requested vector length against a
possibly sparse bitmap of available vector lengths at runtime, so
that the best supported value can be chosen.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Requested by Alex Benn?e:
* Comment the definition of SVE_VL_ARCH_MAX.
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other changes related Alex Benn?e's comments:
* sve_max_vl is definitely not supposed to be changed after boot.
Make it official by marking it __ro_after_init.
* Migrate away from magic number for SVE_VQ_BYTES.
---
arch/arm64/include/asm/fpsimd.h | 8 +++
arch/arm64/kernel/fpsimd.c | 128 +++++++++++++++++++++++++++++++++++++++-
include/uapi/linux/prctl.h | 5 ++
3 files changed, 140 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 7efd04e..32c8e19 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -20,6 +20,7 @@
#ifndef __ASSEMBLY__
+#include <linux/cache.h>
#include <linux/stddef.h>
/*
@@ -70,11 +71,16 @@ extern void fpsimd_update_current_state(struct fpsimd_state *state);
extern void fpsimd_flush_task_state(struct task_struct *target);
+/* Maximum VL that SVE VL-agnostic software can transparently support */
+#define SVE_VL_ARCH_MAX 0x100
+
extern void sve_save_state(void *state, u32 *pfpsr);
extern void sve_load_state(void const *state, u32 const *pfpsr,
unsigned long vq_minus_1);
extern unsigned int sve_get_vl(void);
+extern int __ro_after_init sve_max_vl;
+
#ifdef CONFIG_ARM64_SVE
extern size_t sve_state_size(struct task_struct const *task);
@@ -83,6 +89,8 @@ extern void sve_alloc(struct task_struct *task);
extern void fpsimd_release_thread(struct task_struct *task);
extern void fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src);
+extern int sve_set_vector_length(struct task_struct *task,
+ unsigned long vl, unsigned long flags);
#else /* ! CONFIG_ARM64_SVE */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index f82cde8..713476e 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -17,8 +17,10 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
+#include <linux/bitmap.h>
#include <linux/bottom_half.h>
#include <linux/bug.h>
+#include <linux/cache.h>
#include <linux/compat.h>
#include <linux/cpu.h>
#include <linux/cpu_pm.h>
@@ -26,6 +28,7 @@
#include <linux/init.h>
#include <linux/percpu.h>
#include <linux/preempt.h>
+#include <linux/prctl.h>
#include <linux/ptrace.h>
#include <linux/sched/signal.h>
#include <linux/signal.h>
@@ -109,6 +112,20 @@ static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
/* Default VL for tasks that don't set it explicitly: */
static int sve_default_vl = SVE_VL_MIN;
+#ifdef CONFIG_ARM64_SVE
+
+/* Maximum supported vector length across all CPUs (initially poisoned) */
+int __ro_after_init sve_max_vl = -1;
+/* Set of available vector lengths, as vq_to_bit(vq): */
+static DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+
+#else /* ! CONFIG_ARM64_SVE */
+
+/* Dummy declaration for code that will be optimised out: */
+extern DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+
+#endif /* ! CONFIG_ARM64_SVE */
+
static void sve_free(struct task_struct *task)
{
kfree(task->thread.sve_state);
@@ -186,6 +203,44 @@ static void task_fpsimd_save(void)
}
}
+static unsigned int vq_to_bit(unsigned int vq)
+{
+ return SVE_VQ_MAX - vq;
+}
+
+static unsigned int bit_to_vq(unsigned int bit)
+{
+ if (WARN_ON(bit >= SVE_VQ_MAX))
+ bit = SVE_VQ_MAX - 1;
+
+ return SVE_VQ_MAX - bit;
+}
+
+/*
+ * All vector length selection from userspace comes through here.
+ * We're on a slow path, so some sanity-checks are included.
+ * If things go wrong there's a bug somewhere, but try to fall back to a
+ * safe choice.
+ */
+static unsigned int find_supported_vector_length(unsigned int vl)
+{
+ int bit;
+ int max_vl = sve_max_vl;
+
+ if (WARN_ON(!sve_vl_valid(vl)))
+ vl = SVE_VL_MIN;
+
+ if (WARN_ON(!sve_vl_valid(max_vl)))
+ max_vl = SVE_VL_MIN;
+
+ if (vl > max_vl)
+ vl = max_vl;
+
+ bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
+ vq_to_bit(sve_vq_from_vl(vl)));
+ return sve_vl_from_vq(bit_to_vq(bit));
+}
+
#define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
(SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
@@ -265,6 +320,73 @@ void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
dst->thread.sve_state = NULL;
}
+int sve_set_vector_length(struct task_struct *task,
+ unsigned long vl, unsigned long flags)
+{
+ WARN_ON(task == current && preemptible());
+
+ if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
+ PR_SVE_SET_VL_ONEXEC))
+ return -EINVAL;
+
+ if (!sve_vl_valid(vl))
+ return -EINVAL;
+
+ /*
+ * Clamp to the maximum vector length that VL-agnostic SVE code can
+ * work with. A flag may be assigned in the future to allow setting
+ * of larger vector lengths without confusing older software.
+ */
+ if (vl > SVE_VL_ARCH_MAX)
+ vl = SVE_VL_ARCH_MAX;
+
+ vl = find_supported_vector_length(vl);
+
+ if (flags & (PR_SVE_VL_INHERIT |
+ PR_SVE_SET_VL_ONEXEC))
+ task->thread.sve_vl_onexec = vl;
+ else
+ /* Reset VL to system default on next exec: */
+ task->thread.sve_vl_onexec = 0;
+
+ /* Only actually set the VL if not deferred: */
+ if (flags & PR_SVE_SET_VL_ONEXEC)
+ goto out;
+
+ /*
+ * To ensure the FPSIMD bits of the SVE vector registers are preserved,
+ * write any live register state back to task_struct, and convert to a
+ * non-SVE thread.
+ */
+ if (vl != task->thread.sve_vl) {
+ if (task == current) {
+ task_fpsimd_save();
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
+ }
+
+ if (test_and_clear_tsk_thread_flag(task, TIF_SVE))
+ sve_to_fpsimd(task);
+
+ /*
+ * Force reallocation of task SVE state to the correct size
+ * on next use:
+ */
+ sve_free(task);
+ }
+
+ task->thread.sve_vl = vl;
+
+ fpsimd_flush_task_state(task);
+
+out:
+ if (flags & PR_SVE_VL_INHERIT)
+ set_thread_flag(TIF_SVE_VL_INHERIT);
+ else
+ clear_thread_flag(TIF_SVE_VL_INHERIT);
+
+ return 0;
+}
+
void fpsimd_release_thread(struct task_struct *dead_task)
{
sve_free(dead_task);
@@ -361,7 +483,7 @@ void fpsimd_thread_switch(struct task_struct *next)
void fpsimd_flush_thread(void)
{
- int vl;
+ int vl, supported_vl;
if (!system_supports_fpsimd())
return;
@@ -389,6 +511,10 @@ void fpsimd_flush_thread(void)
if (WARN_ON(!sve_vl_valid(vl)))
vl = SVE_VL_MIN;
+ supported_vl = find_supported_vector_length(vl);
+ if (WARN_ON(supported_vl != vl))
+ vl = supported_vl;
+
current->thread.sve_vl = vl;
/*
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index a8d0759..1b64901 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -197,4 +197,9 @@ struct prctl_mm_map {
# define PR_CAP_AMBIENT_LOWER 3
# define PR_CAP_AMBIENT_CLEAR_ALL 4
+/* arm64 Scalable Vector Extension controls */
+# define PR_SVE_SET_VL_ONEXEC (1 << 18) /* defer effect until exec */
+# define PR_SVE_VL_LEN_MASK 0xffff
+# define PR_SVE_VL_INHERIT (1 << 17) /* inherit across exec */
+
#endif /* _LINUX_PRCTL_H */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Suzuki K Poulose
update_cpu_features() currently cannot tell whether it is being
called during early or late secondary boot. This doesn't
desperately matter for anything it currently does.
However, SVE will need to know here whether the set of available
vector lengths is fixed of still to be determined when booting a
CPU so that it can be updated appropriately.
This patch simply moves the sys_caps_initialised stuff to the top
of the file so that it can be more widely. There doesn't seem to
be a more obvious place to put it.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index cd52d36..43ba8df 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -51,6 +51,21 @@ unsigned int compat_elf_hwcap2 __read_mostly;
DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
EXPORT_SYMBOL(cpu_hwcaps);
+/*
+ * Flag to indicate if we have computed the system wide
+ * capabilities based on the boot time active CPUs. This
+ * will be used to determine if a new booting CPU should
+ * go through the verification process to make sure that it
+ * supports the system capabilities, without using a hotplug
+ * notifier.
+ */
+static bool sys_caps_initialised;
+
+static inline void set_sys_caps_initialised(void)
+{
+ sys_caps_initialised = true;
+}
+
static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
{
/* file-wide pr_fmt adds "CPU features: " prefix */
@@ -1041,21 +1056,6 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
}
/*
- * Flag to indicate if we have computed the system wide
- * capabilities based on the boot time active CPUs. This
- * will be used to determine if a new booting CPU should
- * go through the verification process to make sure that it
- * supports the system capabilities, without using a hotplug
- * notifier.
- */
-static bool sys_caps_initialised;
-
-static inline void set_sys_caps_initialised(void)
-{
- sys_caps_initialised = true;
-}
-
-/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
*/
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
update_cpu_features() currently cannot tell whether it is being
called during early or late secondary boot. This doesn't
desperately matter for anything it currently does.
However, SVE will need to know here whether the set of available
vector lengths is fixed of still to be determined when booting a
CPU so that it can be updated appropriately.
This patch simply moves the sys_caps_initialised stuff to the top
of the file so that it can be more widely. There doesn't seem to
be a more obvious place to put it.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++---------------
1 file changed, 15 insertions(+), 15 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index cd52d36..43ba8df 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -51,6 +51,21 @@ unsigned int compat_elf_hwcap2 __read_mostly;
DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
EXPORT_SYMBOL(cpu_hwcaps);
+/*
+ * Flag to indicate if we have computed the system wide
+ * capabilities based on the boot time active CPUs. This
+ * will be used to determine if a new booting CPU should
+ * go through the verification process to make sure that it
+ * supports the system capabilities, without using a hotplug
+ * notifier.
+ */
+static bool sys_caps_initialised;
+
+static inline void set_sys_caps_initialised(void)
+{
+ sys_caps_initialised = true;
+}
+
static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
{
/* file-wide pr_fmt adds "CPU features: " prefix */
@@ -1041,21 +1056,6 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
}
/*
- * Flag to indicate if we have computed the system wide
- * capabilities based on the boot time active CPUs. This
- * will be used to determine if a new booting CPU should
- * go through the verification process to make sure that it
- * supports the system capabilities, without using a hotplug
- * notifier.
- */
-static bool sys_caps_initialised;
-
-static inline void set_sys_caps_initialised(void)
-{
- sys_caps_initialised = true;
-}
-
-/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
*/
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Suzuki K Poulose
This patch uses the cpufeatures framework to determine common SVE
capabilities and vector lengths, and configures the runtime SVE
support code appropriately.
ZCR_ELx is not really a feature register, but it is convenient to
use it as a template for recording the maximum vector length
supported by a CPU, using the LEN field. This field is similar to
a feature field in that it is a contiguous bitfield for which we
want to determine the minimum system-wide value. This patch adds
ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
custom code to populate it. Finding the minimum supported value of
the LEN field is left to the cpufeatures framework in the usual
way.
The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
so for now we just require it to be zero.
Note that much of this code is dormant and SVE still won't be used
yet, since system_supports_sve() remains hardwired to false.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
---
Changes since v1
----------------
Requested by Alex Bennée:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other changes related to Alex Bennée's comments:
* Migrate away from magic numbers for converting VL to VQ.
Requested by Suzuki Poulose:
* Make sve_vq_map __ro_after_init.
Other changes related to Suzuki Poulose's comments:
* Rely on cpufeatures for not attempting to update the vq map after boot.
---
arch/arm64/include/asm/cpu.h | 4 ++
arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
arch/arm64/include/asm/fpsimd.h | 10 ++++
arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
arch/arm64/kernel/cpuinfo.c | 6 ++
arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
6 files changed, 202 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
index 889226b..8839227 100644
--- a/arch/arm64/include/asm/cpu.h
+++ b/arch/arm64/include/asm/cpu.h
@@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
u64 reg_id_aa64mmfr2;
u64 reg_id_aa64pfr0;
u64 reg_id_aa64pfr1;
+ u64 reg_id_aa64zfr0;
u32 reg_id_dfr0;
u32 reg_id_isar0;
@@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
u32 reg_mvfr0;
u32 reg_mvfr1;
u32 reg_mvfr2;
+
+ /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
+ u64 reg_zcr;
};
DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 4ea3441..d98e7ba 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -10,7 +10,9 @@
#define __ASM_CPUFEATURE_H
#include <asm/cpucaps.h>
+#include <asm/fpsimd.h>
#include <asm/hwcap.h>
+#include <asm/sigcontext.h>
#include <asm/sysreg.h>
/*
@@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
return val == ID_AA64PFR0_EL0_32BIT_64BIT;
}
+static inline bool id_aa64pfr0_sve(u64 pfr0)
+{
+ u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
+
+ return val > 0;
+}
+
void __init setup_cpu_features(void);
void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
@@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
return false;
}
+/*
+ * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
+ * vector length.
+ * Use only if SVE is present. This function clobbers the SVE vector length.
+ */
+static u64 __maybe_unused read_zcr_features(void)
+{
+ u64 zcr;
+ unsigned int vq_max;
+
+ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
+
+ zcr = read_sysreg_s(SYS_ZCR_EL1);
+ zcr &= ~(u64)ZCR_ELx_LEN_MASK;
+ vq_max = sve_vq_from_vl(sve_get_vl());
+ zcr |= vq_max - 1;
+
+ return zcr;
+}
+
#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 32c8e19..6c22624 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -92,12 +92,22 @@ extern void fpsimd_dup_sve(struct task_struct *dst,
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
+extern void __init sve_init_vq_map(void);
+extern void sve_update_vq_map(void);
+extern int sve_verify_vq_map(void);
+extern void __init sve_setup(void);
+
#else /* ! CONFIG_ARM64_SVE */
static void __maybe_unused sve_alloc(struct task_struct *task) { }
static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src) { }
+static void __maybe_unused sve_init_vq_map(void) { }
+static void __maybe_unused sve_update_vq_map(void) { }
+static int __maybe_unused sve_verify_vq_map(void) { return 0; }
+static void __maybe_unused sve_setup(void) { }
+
#endif /* ! CONFIG_ARM64_SVE */
/* For use by EFI runtime services calls only */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 43ba8df..c30bb6b 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -27,6 +27,7 @@
#include <asm/cpu.h>
#include <asm/cpufeature.h>
#include <asm/cpu_ops.h>
+#include <asm/fpsimd.h>
#include <asm/mmu_context.h>
#include <asm/processor.h>
#include <asm/sysreg.h>
@@ -283,6 +284,12 @@ static const struct arm64_ftr_bits ftr_id_dfr0[] = {
ARM64_FTR_END,
};
+static const struct arm64_ftr_bits ftr_zcr[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
+ ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
+ ARM64_FTR_END,
+};
+
/*
* Common ftr bits for a 32bit register with all hidden, strict
* attributes, with 4bit feature fields and a default safe value of
@@ -349,6 +356,7 @@ static const struct __ftr_reg_entry {
/* Op1 = 0, CRn = 0, CRm = 4 */
ARM64_FTR_REG(SYS_ID_AA64PFR0_EL1, ftr_id_aa64pfr0),
ARM64_FTR_REG(SYS_ID_AA64PFR1_EL1, ftr_raz),
+ ARM64_FTR_REG(SYS_ID_AA64ZFR0_EL1, ftr_raz),
/* Op1 = 0, CRn = 0, CRm = 5 */
ARM64_FTR_REG(SYS_ID_AA64DFR0_EL1, ftr_id_aa64dfr0),
@@ -363,6 +371,9 @@ static const struct __ftr_reg_entry {
ARM64_FTR_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1),
ARM64_FTR_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2),
+ /* Op1 = 0, CRn = 1, CRm = 2 */
+ ARM64_FTR_REG(SYS_ZCR_EL1, ftr_zcr),
+
/* Op1 = 3, CRn = 0, CRm = 0 */
{ SYS_CTR_EL0, &arm64_ftr_reg_ctrel0 },
ARM64_FTR_REG(SYS_DCZID_EL0, ftr_dczid),
@@ -500,6 +511,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_ID_AA64MMFR2_EL1, info->reg_id_aa64mmfr2);
init_cpu_ftr_reg(SYS_ID_AA64PFR0_EL1, info->reg_id_aa64pfr0);
init_cpu_ftr_reg(SYS_ID_AA64PFR1_EL1, info->reg_id_aa64pfr1);
+ init_cpu_ftr_reg(SYS_ID_AA64ZFR0_EL1, info->reg_id_aa64zfr0);
if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0);
@@ -520,6 +532,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_MVFR2_EL1, info->reg_mvfr2);
}
+ if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
+ init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
+ sve_init_vq_map();
+ }
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -623,6 +639,9 @@ void update_cpu_features(int cpu,
taint |= check_update_ftr_reg(SYS_ID_AA64PFR1_EL1, cpu,
info->reg_id_aa64pfr1, boot->reg_id_aa64pfr1);
+ taint |= check_update_ftr_reg(SYS_ID_AA64ZFR0_EL1, cpu,
+ info->reg_id_aa64zfr0, boot->reg_id_aa64zfr0);
+
/*
* If we have AArch32, we care about 32-bit features for compat.
* If the system doesn't support AArch32, don't update them.
@@ -670,6 +689,14 @@ void update_cpu_features(int cpu,
info->reg_mvfr2, boot->reg_mvfr2);
}
+ if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
+ taint |= check_update_ftr_reg(SYS_ZCR_EL1, cpu,
+ info->reg_zcr, boot->reg_zcr);
+
+ if (!sys_caps_initialised)
+ sve_update_vq_map();
+ }
+
/*
* Mismatched CPU features are a recipe for disaster. Don't even
* pretend to support them.
@@ -1097,6 +1124,23 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps)
}
}
+static void verify_sve_features(void)
+{
+ u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
+ u64 zcr = read_zcr_features();
+
+ unsigned int safe_len = safe_zcr & ZCR_ELx_LEN_MASK;
+ unsigned int len = zcr & ZCR_ELx_LEN_MASK;
+
+ if (len < safe_len || sve_verify_vq_map()) {
+ pr_crit("CPU%d: SVE: required vector length(s) missing\n",
+ smp_processor_id());
+ cpu_die_early();
+ }
+
+ /* Add checks on other ZCR bits here if necessary */
+}
+
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
@@ -1110,8 +1154,12 @@ static void verify_local_cpu_capabilities(void)
verify_local_cpu_errata_workarounds();
verify_local_cpu_features(arm64_features);
verify_local_elf_hwcaps(arm64_elf_hwcaps);
+
if (system_supports_32bit_el0())
verify_local_elf_hwcaps(compat_elf_hwcaps);
+
+ if (system_supports_sve())
+ verify_sve_features();
}
void check_local_cpu_capabilities(void)
@@ -1189,6 +1237,8 @@ void __init setup_cpu_features(void)
if (system_supports_32bit_el0())
setup_elf_hwcaps(compat_elf_hwcaps);
+ sve_setup();
+
/* Advertise that we have computed the system capabilities */
set_sys_caps_initialised();
diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
index 3118859..be260e8 100644
--- a/arch/arm64/kernel/cpuinfo.c
+++ b/arch/arm64/kernel/cpuinfo.c
@@ -19,6 +19,7 @@
#include <asm/cpu.h>
#include <asm/cputype.h>
#include <asm/cpufeature.h>
+#include <asm/fpsimd.h>
#include <linux/bitops.h>
#include <linux/bug.h>
@@ -326,6 +327,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
info->reg_id_aa64mmfr2 = read_cpuid(ID_AA64MMFR2_EL1);
info->reg_id_aa64pfr0 = read_cpuid(ID_AA64PFR0_EL1);
info->reg_id_aa64pfr1 = read_cpuid(ID_AA64PFR1_EL1);
+ info->reg_id_aa64zfr0 = read_cpuid(ID_AA64ZFR0_EL1);
/* Update the 32bit ID registers only if AArch32 is implemented */
if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
@@ -348,6 +350,10 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
info->reg_mvfr2 = read_cpuid(MVFR2_EL1);
}
+ if (IS_ENABLED(CONFIG_ARM64_SVE) &&
+ id_aa64pfr0_sve(info->reg_id_aa64pfr0))
+ info->reg_zcr = read_zcr_features();
+
cpuinfo_detect_icache_policy(info);
}
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 713476e..cea05a7 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -110,19 +110,19 @@
static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
/* Default VL for tasks that don't set it explicitly: */
-static int sve_default_vl = SVE_VL_MIN;
+static int sve_default_vl = -1;
#ifdef CONFIG_ARM64_SVE
/* Maximum supported vector length across all CPUs (initially poisoned) */
int __ro_after_init sve_max_vl = -1;
/* Set of available vector lengths, as vq_to_bit(vq): */
-static DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
#else /* ! CONFIG_ARM64_SVE */
/* Dummy declaration for code that will be optimised out: */
-extern DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
#endif /* ! CONFIG_ARM64_SVE */
@@ -387,6 +387,103 @@ int sve_set_vector_length(struct task_struct *task,
return 0;
}
+static unsigned long *sve_alloc_vq_map(void)
+{
+ return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
+ GFP_KERNEL);
+}
+
+static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
+{
+ unsigned int vq, vl;
+ unsigned long zcr;
+
+ zcr = ZCR_ELx_LEN_MASK;
+ zcr = read_sysreg_s(SYS_ZCR_EL1) & ~zcr;
+
+ for (vq = SVE_VQ_MAX; vq >= SVE_VQ_MIN; --vq) {
+ write_sysreg_s(zcr | (vq - 1), SYS_ZCR_EL1); /* self-syncing */
+ vl = sve_get_vl();
+ vq = sve_vq_from_vl(vl); /* skip intervening lengths */
+ set_bit(vq_to_bit(vq), map);
+ }
+}
+
+void __init sve_init_vq_map(void)
+{
+ sve_probe_vqs(sve_vq_map);
+}
+
+/*
+ * If we haven't committed to the set of supported VQs yet, filter out
+ * those not supported by the current CPU.
+ */
+void sve_update_vq_map(void)
+{
+ unsigned long *map;
+
+ map = sve_alloc_vq_map();
+ sve_probe_vqs(map);
+ bitmap_and(sve_vq_map, sve_vq_map, map, SVE_VQ_MAX);
+ kfree(map);
+}
+
+/* Check whether the current CPU supports all VQs in the committed set */
+int sve_verify_vq_map(void)
+{
+ int ret = 0;
+ unsigned long *map = sve_alloc_vq_map();
+
+ sve_probe_vqs(map);
+ bitmap_andnot(map, sve_vq_map, map, SVE_VQ_MAX);
+ if (!bitmap_empty(map, SVE_VQ_MAX)) {
+ pr_warn("SVE: cpu%d: Required vector length(s) missing\n",
+ smp_processor_id());
+ ret = -EINVAL;
+ }
+
+ kfree(map);
+
+ return ret;
+}
+
+void __init sve_setup(void)
+{
+ u64 zcr;
+
+ if (!system_supports_sve())
+ return;
+
+ /*
+ * The SVE architecture mandates support for 128-bit vectors,
+ * so sve_vq_map must have at least SVE_VQ_MIN set.
+ * If something went wrong, at least try to patch it up:
+ */
+ if (WARN_ON(!test_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map)))
+ set_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map);
+
+ zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
+ sve_max_vl = sve_vl_from_vq((zcr & ZCR_ELx_LEN_MASK) + 1);
+
+ /*
+ * Sanity-check that the max VL we determined through CPU features
+ * corresponds properly to sve_vq_map. If not, do our best:
+ */
+ if (WARN_ON(sve_max_vl != find_supported_vector_length(sve_max_vl)))
+ sve_max_vl = find_supported_vector_length(sve_max_vl);
+
+ /*
+ * For the default VL, pick the maximum supported value <= 64.
+ * VL == 64 is guaranteed not to grow the signal frame.
+ */
+ sve_default_vl = find_supported_vector_length(64);
+
+ pr_info("SVE: maximum available vector length %u bytes per vector\n",
+ sve_max_vl);
+ pr_info("SVE: default vector length %u bytes per vector\n",
+ sve_default_vl);
+}
+
void fpsimd_release_thread(struct task_struct *dead_task)
{
sve_free(dead_task);
@@ -502,6 +599,9 @@ void fpsimd_flush_thread(void)
* This is where we ensure that all user tasks have a valid
* vector length configured: no kernel task can become a user
* task without an exec and hence a call to this function.
+ * By the time the first call to this function is made, all
+ * early hardware probing is complete, so sve_default_vl
+ * should be valid.
* If a bug causes this to go wrong, we make some noise and
* try to fudge thread.sve_vl to a safe value here.
*/
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch uses the cpufeatures framework to determine common SVE
capabilities and vector lengths, and configures the runtime SVE
support code appropriately.
ZCR_ELx is not really a feature register, but it is convenient to
use it as a template for recording the maximum vector length
supported by a CPU, using the LEN field. This field is similar to
a feature field in that it is a contiguous bitfield for which we
want to determine the minimum system-wide value. This patch adds
ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
custom code to populate it. Finding the minimum supported value of
the LEN field is left to the cpufeatures framework in the usual
way.
The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
so for now we just require it to be zero.
Note that much of this code is dormant and SVE still won't be used
yet, since system_supports_sve() remains hardwired to false.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
---
Changes since v1
----------------
Requested by Alex Benn?e:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other changes related to Alex Benn?e's comments:
* Migrate away from magic numbers for converting VL to VQ.
Requested by Suzuki Poulose:
* Make sve_vq_map __ro_after_init.
Other changes related to Suzuki Poulose's comments:
* Rely on cpufeatures for not attempting to update the vq map after boot.
---
arch/arm64/include/asm/cpu.h | 4 ++
arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
arch/arm64/include/asm/fpsimd.h | 10 ++++
arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
arch/arm64/kernel/cpuinfo.c | 6 ++
arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
6 files changed, 202 insertions(+), 3 deletions(-)
diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
index 889226b..8839227 100644
--- a/arch/arm64/include/asm/cpu.h
+++ b/arch/arm64/include/asm/cpu.h
@@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
u64 reg_id_aa64mmfr2;
u64 reg_id_aa64pfr0;
u64 reg_id_aa64pfr1;
+ u64 reg_id_aa64zfr0;
u32 reg_id_dfr0;
u32 reg_id_isar0;
@@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
u32 reg_mvfr0;
u32 reg_mvfr1;
u32 reg_mvfr2;
+
+ /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
+ u64 reg_zcr;
};
DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 4ea3441..d98e7ba 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -10,7 +10,9 @@
#define __ASM_CPUFEATURE_H
#include <asm/cpucaps.h>
+#include <asm/fpsimd.h>
#include <asm/hwcap.h>
+#include <asm/sigcontext.h>
#include <asm/sysreg.h>
/*
@@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
return val == ID_AA64PFR0_EL0_32BIT_64BIT;
}
+static inline bool id_aa64pfr0_sve(u64 pfr0)
+{
+ u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
+
+ return val > 0;
+}
+
void __init setup_cpu_features(void);
void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
@@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
return false;
}
+/*
+ * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
+ * vector length.
+ * Use only if SVE is present. This function clobbers the SVE vector length.
+ */
+static u64 __maybe_unused read_zcr_features(void)
+{
+ u64 zcr;
+ unsigned int vq_max;
+
+ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
+
+ zcr = read_sysreg_s(SYS_ZCR_EL1);
+ zcr &= ~(u64)ZCR_ELx_LEN_MASK;
+ vq_max = sve_vq_from_vl(sve_get_vl());
+ zcr |= vq_max - 1;
+
+ return zcr;
+}
+
#endif /* __ASSEMBLY__ */
#endif
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 32c8e19..6c22624 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -92,12 +92,22 @@ extern void fpsimd_dup_sve(struct task_struct *dst,
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
+extern void __init sve_init_vq_map(void);
+extern void sve_update_vq_map(void);
+extern int sve_verify_vq_map(void);
+extern void __init sve_setup(void);
+
#else /* ! CONFIG_ARM64_SVE */
static void __maybe_unused sve_alloc(struct task_struct *task) { }
static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src) { }
+static void __maybe_unused sve_init_vq_map(void) { }
+static void __maybe_unused sve_update_vq_map(void) { }
+static int __maybe_unused sve_verify_vq_map(void) { return 0; }
+static void __maybe_unused sve_setup(void) { }
+
#endif /* ! CONFIG_ARM64_SVE */
/* For use by EFI runtime services calls only */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 43ba8df..c30bb6b 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -27,6 +27,7 @@
#include <asm/cpu.h>
#include <asm/cpufeature.h>
#include <asm/cpu_ops.h>
+#include <asm/fpsimd.h>
#include <asm/mmu_context.h>
#include <asm/processor.h>
#include <asm/sysreg.h>
@@ -283,6 +284,12 @@ static const struct arm64_ftr_bits ftr_id_dfr0[] = {
ARM64_FTR_END,
};
+static const struct arm64_ftr_bits ftr_zcr[] = {
+ ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
+ ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
+ ARM64_FTR_END,
+};
+
/*
* Common ftr bits for a 32bit register with all hidden, strict
* attributes, with 4bit feature fields and a default safe value of
@@ -349,6 +356,7 @@ static const struct __ftr_reg_entry {
/* Op1 = 0, CRn = 0, CRm = 4 */
ARM64_FTR_REG(SYS_ID_AA64PFR0_EL1, ftr_id_aa64pfr0),
ARM64_FTR_REG(SYS_ID_AA64PFR1_EL1, ftr_raz),
+ ARM64_FTR_REG(SYS_ID_AA64ZFR0_EL1, ftr_raz),
/* Op1 = 0, CRn = 0, CRm = 5 */
ARM64_FTR_REG(SYS_ID_AA64DFR0_EL1, ftr_id_aa64dfr0),
@@ -363,6 +371,9 @@ static const struct __ftr_reg_entry {
ARM64_FTR_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1),
ARM64_FTR_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2),
+ /* Op1 = 0, CRn = 1, CRm = 2 */
+ ARM64_FTR_REG(SYS_ZCR_EL1, ftr_zcr),
+
/* Op1 = 3, CRn = 0, CRm = 0 */
{ SYS_CTR_EL0, &arm64_ftr_reg_ctrel0 },
ARM64_FTR_REG(SYS_DCZID_EL0, ftr_dczid),
@@ -500,6 +511,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_ID_AA64MMFR2_EL1, info->reg_id_aa64mmfr2);
init_cpu_ftr_reg(SYS_ID_AA64PFR0_EL1, info->reg_id_aa64pfr0);
init_cpu_ftr_reg(SYS_ID_AA64PFR1_EL1, info->reg_id_aa64pfr1);
+ init_cpu_ftr_reg(SYS_ID_AA64ZFR0_EL1, info->reg_id_aa64zfr0);
if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0);
@@ -520,6 +532,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_MVFR2_EL1, info->reg_mvfr2);
}
+ if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
+ init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
+ sve_init_vq_map();
+ }
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -623,6 +639,9 @@ void update_cpu_features(int cpu,
taint |= check_update_ftr_reg(SYS_ID_AA64PFR1_EL1, cpu,
info->reg_id_aa64pfr1, boot->reg_id_aa64pfr1);
+ taint |= check_update_ftr_reg(SYS_ID_AA64ZFR0_EL1, cpu,
+ info->reg_id_aa64zfr0, boot->reg_id_aa64zfr0);
+
/*
* If we have AArch32, we care about 32-bit features for compat.
* If the system doesn't support AArch32, don't update them.
@@ -670,6 +689,14 @@ void update_cpu_features(int cpu,
info->reg_mvfr2, boot->reg_mvfr2);
}
+ if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
+ taint |= check_update_ftr_reg(SYS_ZCR_EL1, cpu,
+ info->reg_zcr, boot->reg_zcr);
+
+ if (!sys_caps_initialised)
+ sve_update_vq_map();
+ }
+
/*
* Mismatched CPU features are a recipe for disaster. Don't even
* pretend to support them.
@@ -1097,6 +1124,23 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps)
}
}
+static void verify_sve_features(void)
+{
+ u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
+ u64 zcr = read_zcr_features();
+
+ unsigned int safe_len = safe_zcr & ZCR_ELx_LEN_MASK;
+ unsigned int len = zcr & ZCR_ELx_LEN_MASK;
+
+ if (len < safe_len || sve_verify_vq_map()) {
+ pr_crit("CPU%d: SVE: required vector length(s) missing\n",
+ smp_processor_id());
+ cpu_die_early();
+ }
+
+ /* Add checks on other ZCR bits here if necessary */
+}
+
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs@the boot time.
@@ -1110,8 +1154,12 @@ static void verify_local_cpu_capabilities(void)
verify_local_cpu_errata_workarounds();
verify_local_cpu_features(arm64_features);
verify_local_elf_hwcaps(arm64_elf_hwcaps);
+
if (system_supports_32bit_el0())
verify_local_elf_hwcaps(compat_elf_hwcaps);
+
+ if (system_supports_sve())
+ verify_sve_features();
}
void check_local_cpu_capabilities(void)
@@ -1189,6 +1237,8 @@ void __init setup_cpu_features(void)
if (system_supports_32bit_el0())
setup_elf_hwcaps(compat_elf_hwcaps);
+ sve_setup();
+
/* Advertise that we have computed the system capabilities */
set_sys_caps_initialised();
diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
index 3118859..be260e8 100644
--- a/arch/arm64/kernel/cpuinfo.c
+++ b/arch/arm64/kernel/cpuinfo.c
@@ -19,6 +19,7 @@
#include <asm/cpu.h>
#include <asm/cputype.h>
#include <asm/cpufeature.h>
+#include <asm/fpsimd.h>
#include <linux/bitops.h>
#include <linux/bug.h>
@@ -326,6 +327,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
info->reg_id_aa64mmfr2 = read_cpuid(ID_AA64MMFR2_EL1);
info->reg_id_aa64pfr0 = read_cpuid(ID_AA64PFR0_EL1);
info->reg_id_aa64pfr1 = read_cpuid(ID_AA64PFR1_EL1);
+ info->reg_id_aa64zfr0 = read_cpuid(ID_AA64ZFR0_EL1);
/* Update the 32bit ID registers only if AArch32 is implemented */
if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
@@ -348,6 +350,10 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
info->reg_mvfr2 = read_cpuid(MVFR2_EL1);
}
+ if (IS_ENABLED(CONFIG_ARM64_SVE) &&
+ id_aa64pfr0_sve(info->reg_id_aa64pfr0))
+ info->reg_zcr = read_zcr_features();
+
cpuinfo_detect_icache_policy(info);
}
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 713476e..cea05a7 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -110,19 +110,19 @@
static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
/* Default VL for tasks that don't set it explicitly: */
-static int sve_default_vl = SVE_VL_MIN;
+static int sve_default_vl = -1;
#ifdef CONFIG_ARM64_SVE
/* Maximum supported vector length across all CPUs (initially poisoned) */
int __ro_after_init sve_max_vl = -1;
/* Set of available vector lengths, as vq_to_bit(vq): */
-static DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
#else /* ! CONFIG_ARM64_SVE */
/* Dummy declaration for code that will be optimised out: */
-extern DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
#endif /* ! CONFIG_ARM64_SVE */
@@ -387,6 +387,103 @@ int sve_set_vector_length(struct task_struct *task,
return 0;
}
+static unsigned long *sve_alloc_vq_map(void)
+{
+ return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
+ GFP_KERNEL);
+}
+
+static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
+{
+ unsigned int vq, vl;
+ unsigned long zcr;
+
+ zcr = ZCR_ELx_LEN_MASK;
+ zcr = read_sysreg_s(SYS_ZCR_EL1) & ~zcr;
+
+ for (vq = SVE_VQ_MAX; vq >= SVE_VQ_MIN; --vq) {
+ write_sysreg_s(zcr | (vq - 1), SYS_ZCR_EL1); /* self-syncing */
+ vl = sve_get_vl();
+ vq = sve_vq_from_vl(vl); /* skip intervening lengths */
+ set_bit(vq_to_bit(vq), map);
+ }
+}
+
+void __init sve_init_vq_map(void)
+{
+ sve_probe_vqs(sve_vq_map);
+}
+
+/*
+ * If we haven't committed to the set of supported VQs yet, filter out
+ * those not supported by the current CPU.
+ */
+void sve_update_vq_map(void)
+{
+ unsigned long *map;
+
+ map = sve_alloc_vq_map();
+ sve_probe_vqs(map);
+ bitmap_and(sve_vq_map, sve_vq_map, map, SVE_VQ_MAX);
+ kfree(map);
+}
+
+/* Check whether the current CPU supports all VQs in the committed set */
+int sve_verify_vq_map(void)
+{
+ int ret = 0;
+ unsigned long *map = sve_alloc_vq_map();
+
+ sve_probe_vqs(map);
+ bitmap_andnot(map, sve_vq_map, map, SVE_VQ_MAX);
+ if (!bitmap_empty(map, SVE_VQ_MAX)) {
+ pr_warn("SVE: cpu%d: Required vector length(s) missing\n",
+ smp_processor_id());
+ ret = -EINVAL;
+ }
+
+ kfree(map);
+
+ return ret;
+}
+
+void __init sve_setup(void)
+{
+ u64 zcr;
+
+ if (!system_supports_sve())
+ return;
+
+ /*
+ * The SVE architecture mandates support for 128-bit vectors,
+ * so sve_vq_map must have at least SVE_VQ_MIN set.
+ * If something went wrong, at least try to patch it up:
+ */
+ if (WARN_ON(!test_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map)))
+ set_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map);
+
+ zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
+ sve_max_vl = sve_vl_from_vq((zcr & ZCR_ELx_LEN_MASK) + 1);
+
+ /*
+ * Sanity-check that the max VL we determined through CPU features
+ * corresponds properly to sve_vq_map. If not, do our best:
+ */
+ if (WARN_ON(sve_max_vl != find_supported_vector_length(sve_max_vl)))
+ sve_max_vl = find_supported_vector_length(sve_max_vl);
+
+ /*
+ * For the default VL, pick the maximum supported value <= 64.
+ * VL == 64 is guaranteed not to grow the signal frame.
+ */
+ sve_default_vl = find_supported_vector_length(64);
+
+ pr_info("SVE: maximum available vector length %u bytes per vector\n",
+ sve_max_vl);
+ pr_info("SVE: default vector length %u bytes per vector\n",
+ sve_default_vl);
+}
+
void fpsimd_release_thread(struct task_struct *dead_task)
{
sve_free(dead_task);
@@ -502,6 +599,9 @@ void fpsimd_flush_thread(void)
* This is where we ensure that all user tasks have a valid
* vector length configured: no kernel task can become a user
* task without an exec and hence a call to this function.
+ * By the time the first call to this function is made, all
+ * early hardware probing is complete, so sve_default_vl
+ * should be valid.
* If a bug causes this to go wrong, we make some noise and
* try to fudge thread.sve_vl to a safe value here.
*/
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 17/28] arm64/sve: Preserve SVE registers around kernel-mode NEON use
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Kernel-mode NEON will corrupt the SVE vector registers, due to the
way they alias the FPSIMD vector registers in the hardware.
This patch ensures that any live SVE register content for the task
is saved by kernel_neon_begin(). The data will be restored in the
usual way on return to userspace.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/fpsimd.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index cea05a7..dd89acf 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -744,8 +744,10 @@ void kernel_neon_begin(void)
__this_cpu_write(kernel_neon_busy, true);
/* Save unsaved task fpsimd state, if any: */
- if (current->mm && !test_and_set_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ if (current->mm) {
+ task_fpsimd_save();
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
+ }
/* Invalidate any task state remaining in the fpsimd regs: */
__this_cpu_write(fpsimd_last_state, NULL);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 17/28] arm64/sve: Preserve SVE registers around kernel-mode NEON use
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Kernel-mode NEON will corrupt the SVE vector registers, due to the
way they alias the FPSIMD vector registers in the hardware.
This patch ensures that any live SVE register content for the task
is saved by kernel_neon_begin(). The data will be restored in the
usual way on return to userspace.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
arch/arm64/kernel/fpsimd.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index cea05a7..dd89acf 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -744,8 +744,10 @@ void kernel_neon_begin(void)
__this_cpu_write(kernel_neon_busy, true);
/* Save unsaved task fpsimd state, if any: */
- if (current->mm && !test_and_set_thread_flag(TIF_FOREIGN_FPSTATE))
- fpsimd_save_state(¤t->thread.fpsimd_state);
+ if (current->mm) {
+ task_fpsimd_save();
+ set_thread_flag(TIF_FOREIGN_FPSTATE);
+ }
/* Invalidate any task state remaining in the fpsimd regs: */
__this_cpu_write(fpsimd_last_state, NULL);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Richard Sandiford, kvmarm
The EFI runtime services ABI allows EFI to make free use of the
FPSIMD registers during EFI runtime service calls, subject to the
callee-save requirements of the AArch64 procedure call standard.
However, the SVE architecture allows upper bits of the SVE vector
registers to be zeroed as a side-effect of FPSIMD V-register
writes. This means that the SVE vector registers must be saved in
their entirety in order to avoid data loss: non-SVE-aware EFI
implementations cannot restore them correctly.
The non-IRQ case is already handled gracefully by
kernel_neon_begin(). For the IRQ case, this patch allocates a
suitable per-CPU stash buffer for the full SVE register state and
uses it to preserve the affected registers around EFI calls. It is
currently unclear how the EFI runtime services ABI will be
clarified with respect to SVE, so it safest to assume that the
predicate registers and FFR must be saved and restored too.
No attempt is made to restore the restore the vector length after
a call, for now. It is deemed rather insane for EFI to change it,
and contemporary EFI implementations certainly won't.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Changes since v1
----------------
Requested by Ard Biesheuvel:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
* Make efi_sve_state_used static.
Changes related to Alex Bennée's comments:
* Migrate away from magic numbers for SVE_VQ_BYTES.
Other:
* Rename sve_kernel_mode_neon_setup() to sve_efi_setup().
The EFI FPSIMD code is something semi-independent from kernel-mode NEON
now, so the "neon" in the names no longer really makes sense.
* Make the EFI FPSIMD setup code dependent on CONFIG_EFI (it's not
supposed to work with CONFIG_EFI=n anyway).
---
arch/arm64/kernel/fpsimd.c | 60 +++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 54 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index dd89acf..fff9fcf 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -118,11 +118,13 @@ static int sve_default_vl = -1;
int __ro_after_init sve_max_vl = -1;
/* Set of available vector lengths, as vq_to_bit(vq): */
static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+static void __percpu *efi_sve_state;
#else /* ! CONFIG_ARM64_SVE */
/* Dummy declaration for code that will be optimised out: */
extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+extern void __percpu *efi_sve_state;
#endif /* ! CONFIG_ARM64_SVE */
@@ -447,6 +449,23 @@ int sve_verify_vq_map(void)
return ret;
}
+static void __init sve_efi_setup(void)
+{
+ if (!IS_ENABLED(CONFIG_EFI))
+ return;
+
+ /*
+ * alloc_percpu() warns and prints a backtrace if this goes wrong.
+ * This is evidence of a crippled system and we are returning void,
+ * so no attempt is made to handle this situation here.
+ */
+ BUG_ON(!sve_vl_valid(sve_max_vl));
+ efi_sve_state = __alloc_percpu(
+ SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
+ if (!efi_sve_state)
+ panic("Cannot allocate percpu memory for EFI SVE save/restore");
+}
+
void __init sve_setup(void)
{
u64 zcr;
@@ -482,6 +501,8 @@ void __init sve_setup(void)
sve_max_vl);
pr_info("SVE: default vector length %u bytes per vector\n",
sve_default_vl);
+
+ sve_efi_setup();
}
void fpsimd_release_thread(struct task_struct *dead_task)
@@ -783,6 +804,7 @@ EXPORT_SYMBOL(kernel_neon_end);
static DEFINE_PER_CPU(struct fpsimd_state, efi_fpsimd_state);
static DEFINE_PER_CPU(bool, efi_fpsimd_state_used);
+static DEFINE_PER_CPU(bool, efi_sve_state_used);
/*
* EFI runtime services support functions
@@ -808,10 +830,24 @@ void __efi_fpsimd_begin(void)
WARN_ON(preemptible());
- if (may_use_simd())
+ if (may_use_simd()) {
kernel_neon_begin();
- else {
- fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
+ } else {
+ /*
+ * If !efi_sve_state, SVE can't be in use yet and doesn't need
+ * preserving:
+ */
+ if (system_supports_sve() && likely(efi_sve_state)) {
+ char *sve_state = this_cpu_ptr(efi_sve_state);
+
+ __this_cpu_write(efi_sve_state_used, true);
+
+ sve_save_state(sve_state + sve_ffr_offset(sve_max_vl),
+ &this_cpu_ptr(&efi_fpsimd_state)->fpsr);
+ } else {
+ fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
+ }
+
__this_cpu_write(efi_fpsimd_state_used, true);
}
}
@@ -824,10 +860,22 @@ void __efi_fpsimd_end(void)
if (!system_supports_fpsimd())
return;
- if (__this_cpu_xchg(efi_fpsimd_state_used, false))
- fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
- else
+ if (!__this_cpu_xchg(efi_fpsimd_state_used, false)) {
kernel_neon_end();
+ } else {
+ if (system_supports_sve() &&
+ likely(__this_cpu_read(efi_sve_state_used))) {
+ char const *sve_state = this_cpu_ptr(efi_sve_state);
+
+ sve_load_state(sve_state + sve_ffr_offset(sve_max_vl),
+ &this_cpu_ptr(&efi_fpsimd_state)->fpsr,
+ sve_vq_from_vl(sve_get_vl()) - 1);
+
+ __this_cpu_write(efi_sve_state_used, false);
+ } else {
+ fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
+ }
+ }
}
#endif /* CONFIG_KERNEL_MODE_NEON */
--
2.1.4
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
The EFI runtime services ABI allows EFI to make free use of the
FPSIMD registers during EFI runtime service calls, subject to the
callee-save requirements of the AArch64 procedure call standard.
However, the SVE architecture allows upper bits of the SVE vector
registers to be zeroed as a side-effect of FPSIMD V-register
writes. This means that the SVE vector registers must be saved in
their entirety in order to avoid data loss: non-SVE-aware EFI
implementations cannot restore them correctly.
The non-IRQ case is already handled gracefully by
kernel_neon_begin(). For the IRQ case, this patch allocates a
suitable per-CPU stash buffer for the full SVE register state and
uses it to preserve the affected registers around EFI calls. It is
currently unclear how the EFI runtime services ABI will be
clarified with respect to SVE, so it safest to assume that the
predicate registers and FFR must be saved and restored too.
No attempt is made to restore the restore the vector length after
a call, for now. It is deemed rather insane for EFI to change it,
and contemporary EFI implementations certainly won't.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Changes since v1
----------------
Requested by Ard Biesheuvel:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
* Make efi_sve_state_used static.
Changes related to Alex Bennée's comments:
* Migrate away from magic numbers for SVE_VQ_BYTES.
Other:
* Rename sve_kernel_mode_neon_setup() to sve_efi_setup().
The EFI FPSIMD code is something semi-independent from kernel-mode NEON
now, so the "neon" in the names no longer really makes sense.
* Make the EFI FPSIMD setup code dependent on CONFIG_EFI (it's not
supposed to work with CONFIG_EFI=n anyway).
---
arch/arm64/kernel/fpsimd.c | 60 +++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 54 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index dd89acf..fff9fcf 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -118,11 +118,13 @@ static int sve_default_vl = -1;
int __ro_after_init sve_max_vl = -1;
/* Set of available vector lengths, as vq_to_bit(vq): */
static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+static void __percpu *efi_sve_state;
#else /* ! CONFIG_ARM64_SVE */
/* Dummy declaration for code that will be optimised out: */
extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+extern void __percpu *efi_sve_state;
#endif /* ! CONFIG_ARM64_SVE */
@@ -447,6 +449,23 @@ int sve_verify_vq_map(void)
return ret;
}
+static void __init sve_efi_setup(void)
+{
+ if (!IS_ENABLED(CONFIG_EFI))
+ return;
+
+ /*
+ * alloc_percpu() warns and prints a backtrace if this goes wrong.
+ * This is evidence of a crippled system and we are returning void,
+ * so no attempt is made to handle this situation here.
+ */
+ BUG_ON(!sve_vl_valid(sve_max_vl));
+ efi_sve_state = __alloc_percpu(
+ SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
+ if (!efi_sve_state)
+ panic("Cannot allocate percpu memory for EFI SVE save/restore");
+}
+
void __init sve_setup(void)
{
u64 zcr;
@@ -482,6 +501,8 @@ void __init sve_setup(void)
sve_max_vl);
pr_info("SVE: default vector length %u bytes per vector\n",
sve_default_vl);
+
+ sve_efi_setup();
}
void fpsimd_release_thread(struct task_struct *dead_task)
@@ -783,6 +804,7 @@ EXPORT_SYMBOL(kernel_neon_end);
static DEFINE_PER_CPU(struct fpsimd_state, efi_fpsimd_state);
static DEFINE_PER_CPU(bool, efi_fpsimd_state_used);
+static DEFINE_PER_CPU(bool, efi_sve_state_used);
/*
* EFI runtime services support functions
@@ -808,10 +830,24 @@ void __efi_fpsimd_begin(void)
WARN_ON(preemptible());
- if (may_use_simd())
+ if (may_use_simd()) {
kernel_neon_begin();
- else {
- fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
+ } else {
+ /*
+ * If !efi_sve_state, SVE can't be in use yet and doesn't need
+ * preserving:
+ */
+ if (system_supports_sve() && likely(efi_sve_state)) {
+ char *sve_state = this_cpu_ptr(efi_sve_state);
+
+ __this_cpu_write(efi_sve_state_used, true);
+
+ sve_save_state(sve_state + sve_ffr_offset(sve_max_vl),
+ &this_cpu_ptr(&efi_fpsimd_state)->fpsr);
+ } else {
+ fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
+ }
+
__this_cpu_write(efi_fpsimd_state_used, true);
}
}
@@ -824,10 +860,22 @@ void __efi_fpsimd_end(void)
if (!system_supports_fpsimd())
return;
- if (__this_cpu_xchg(efi_fpsimd_state_used, false))
- fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
- else
+ if (!__this_cpu_xchg(efi_fpsimd_state_used, false)) {
kernel_neon_end();
+ } else {
+ if (system_supports_sve() &&
+ likely(__this_cpu_read(efi_sve_state_used))) {
+ char const *sve_state = this_cpu_ptr(efi_sve_state);
+
+ sve_load_state(sve_state + sve_ffr_offset(sve_max_vl),
+ &this_cpu_ptr(&efi_fpsimd_state)->fpsr,
+ sve_vq_from_vl(sve_get_vl()) - 1);
+
+ __this_cpu_write(efi_sve_state_used, false);
+ } else {
+ fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
+ }
+ }
}
#endif /* CONFIG_KERNEL_MODE_NEON */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
The EFI runtime services ABI allows EFI to make free use of the
FPSIMD registers during EFI runtime service calls, subject to the
callee-save requirements of the AArch64 procedure call standard.
However, the SVE architecture allows upper bits of the SVE vector
registers to be zeroed as a side-effect of FPSIMD V-register
writes. This means that the SVE vector registers must be saved in
their entirety in order to avoid data loss: non-SVE-aware EFI
implementations cannot restore them correctly.
The non-IRQ case is already handled gracefully by
kernel_neon_begin(). For the IRQ case, this patch allocates a
suitable per-CPU stash buffer for the full SVE register state and
uses it to preserve the affected registers around EFI calls. It is
currently unclear how the EFI runtime services ABI will be
clarified with respect to SVE, so it safest to assume that the
predicate registers and FFR must be saved and restored too.
No attempt is made to restore the restore the vector length after
a call, for now. It is deemed rather insane for EFI to change it,
and contemporary EFI implementations certainly won't.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Changes since v1
----------------
Requested by Ard Biesheuvel:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
* Make efi_sve_state_used static.
Changes related to Alex Benn?e's comments:
* Migrate away from magic numbers for SVE_VQ_BYTES.
Other:
* Rename sve_kernel_mode_neon_setup() to sve_efi_setup().
The EFI FPSIMD code is something semi-independent from kernel-mode NEON
now, so the "neon" in the names no longer really makes sense.
* Make the EFI FPSIMD setup code dependent on CONFIG_EFI (it's not
supposed to work with CONFIG_EFI=n anyway).
---
arch/arm64/kernel/fpsimd.c | 60 +++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 54 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index dd89acf..fff9fcf 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -118,11 +118,13 @@ static int sve_default_vl = -1;
int __ro_after_init sve_max_vl = -1;
/* Set of available vector lengths, as vq_to_bit(vq): */
static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+static void __percpu *efi_sve_state;
#else /* ! CONFIG_ARM64_SVE */
/* Dummy declaration for code that will be optimised out: */
extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
+extern void __percpu *efi_sve_state;
#endif /* ! CONFIG_ARM64_SVE */
@@ -447,6 +449,23 @@ int sve_verify_vq_map(void)
return ret;
}
+static void __init sve_efi_setup(void)
+{
+ if (!IS_ENABLED(CONFIG_EFI))
+ return;
+
+ /*
+ * alloc_percpu() warns and prints a backtrace if this goes wrong.
+ * This is evidence of a crippled system and we are returning void,
+ * so no attempt is made to handle this situation here.
+ */
+ BUG_ON(!sve_vl_valid(sve_max_vl));
+ efi_sve_state = __alloc_percpu(
+ SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
+ if (!efi_sve_state)
+ panic("Cannot allocate percpu memory for EFI SVE save/restore");
+}
+
void __init sve_setup(void)
{
u64 zcr;
@@ -482,6 +501,8 @@ void __init sve_setup(void)
sve_max_vl);
pr_info("SVE: default vector length %u bytes per vector\n",
sve_default_vl);
+
+ sve_efi_setup();
}
void fpsimd_release_thread(struct task_struct *dead_task)
@@ -783,6 +804,7 @@ EXPORT_SYMBOL(kernel_neon_end);
static DEFINE_PER_CPU(struct fpsimd_state, efi_fpsimd_state);
static DEFINE_PER_CPU(bool, efi_fpsimd_state_used);
+static DEFINE_PER_CPU(bool, efi_sve_state_used);
/*
* EFI runtime services support functions
@@ -808,10 +830,24 @@ void __efi_fpsimd_begin(void)
WARN_ON(preemptible());
- if (may_use_simd())
+ if (may_use_simd()) {
kernel_neon_begin();
- else {
- fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
+ } else {
+ /*
+ * If !efi_sve_state, SVE can't be in use yet and doesn't need
+ * preserving:
+ */
+ if (system_supports_sve() && likely(efi_sve_state)) {
+ char *sve_state = this_cpu_ptr(efi_sve_state);
+
+ __this_cpu_write(efi_sve_state_used, true);
+
+ sve_save_state(sve_state + sve_ffr_offset(sve_max_vl),
+ &this_cpu_ptr(&efi_fpsimd_state)->fpsr);
+ } else {
+ fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
+ }
+
__this_cpu_write(efi_fpsimd_state_used, true);
}
}
@@ -824,10 +860,22 @@ void __efi_fpsimd_end(void)
if (!system_supports_fpsimd())
return;
- if (__this_cpu_xchg(efi_fpsimd_state_used, false))
- fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
- else
+ if (!__this_cpu_xchg(efi_fpsimd_state_used, false)) {
kernel_neon_end();
+ } else {
+ if (system_supports_sve() &&
+ likely(__this_cpu_read(efi_sve_state_used))) {
+ char const *sve_state = this_cpu_ptr(efi_sve_state);
+
+ sve_load_state(sve_state + sve_ffr_offset(sve_max_vl),
+ &this_cpu_ptr(&efi_fpsimd_state)->fpsr,
+ sve_vq_from_vl(sve_get_vl()) - 1);
+
+ __this_cpu_write(efi_sve_state_used, false);
+ } else {
+ fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
+ }
+ }
}
#endif /* CONFIG_KERNEL_MODE_NEON */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
gdb, Alan Hayward, Yao Qi, Oleg Nesterov, Alexander Viro
This patch defines and implements a new regset NT_ARM_SVE, which
describes a thread's SVE register state. This allows a debugger to
manipulate the SVE state, as well as being included in ELF
coredumps for post-mortem debugging.
Because the regset size and layout are dependent on the thread's
current vector length, it is not possible to define a C struct to
describe the regset contents as is done for existing regsets.
Instead, and for the same reasons, NT_ARM_SVE is based on the
freeform variable-layout approach used for the SVE signal frame.
Additionally, to reduce debug overhead when debugging threads that
might or might not have live SVE register state, NT_ARM_SVE may be
presented in one of two different formats: the old struct
user_fpsimd_state format is embedded for describing the state of a
thread with no live SVE state, whereas a new variable-layout
structure is embedded for describing live SVE state. This avoids a
debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
allows existing userspace code to handle the non-SVE case without
too much modification.
For this to work, NT_ARM_SVE is defined with a fixed-format header
of type struct user_sve_header, which the recipient can use to
figure out the content, size and layout of the reset of the regset.
Accessor macros are defined to allow the vector-length-dependent
parts of the regset to be manipulated.
Signed-off-by: Alan Hayward <alan.hayward@arm.com>
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Other changes related to Alex Bennée's comments:
* Migrate to SVE_VQ_BYTES instead of magic numbers.
Requested by Alex Bennée:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other:
* [ABI fix] Bail out with -EIO if attempting to set the
SVE regs for an unsupported VL, instead of misparsing the regset data.
* Replace some in-kernel open-coded arithmetic with ALIGN()/
DIV_ROUND_UP().
---
arch/arm64/include/asm/fpsimd.h | 13 +-
arch/arm64/include/uapi/asm/ptrace.h | 135 ++++++++++++++++++
arch/arm64/kernel/fpsimd.c | 40 +++++-
arch/arm64/kernel/ptrace.c | 270 +++++++++++++++++++++++++++++++++--
include/uapi/linux/elf.h | 1 +
5 files changed, 449 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 6c22624..2723cca 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -38,13 +38,16 @@ struct fpsimd_state {
__uint128_t vregs[32];
u32 fpsr;
u32 fpcr;
+ /*
+ * For ptrace compatibility, pad to next 128-bit
+ * boundary here if extending this struct.
+ */
};
};
/* the id of the last cpu to have restored this state */
unsigned int cpu;
};
-
#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
/* Masks for extracting the FPSR and FPCR from the FPSCR */
#define VFP_FPSCR_STAT_MASK 0xf800009f
@@ -89,6 +92,10 @@ extern void sve_alloc(struct task_struct *task);
extern void fpsimd_release_thread(struct task_struct *task);
extern void fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src);
+extern void fpsimd_sync_to_sve(struct task_struct *task);
+extern void sve_sync_to_fpsimd(struct task_struct *task);
+extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
+
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
@@ -103,6 +110,10 @@ static void __maybe_unused sve_alloc(struct task_struct *task) { }
static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src) { }
+static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
+static void __maybe_unused sve_sync_from_fpsimd_zeropad(
+ struct task_struct *task) { }
+
static void __maybe_unused sve_init_vq_map(void) { }
static void __maybe_unused sve_update_vq_map(void) { }
static int __maybe_unused sve_verify_vq_map(void) { return 0; }
diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
index d1ff83d..1915ab0 100644
--- a/arch/arm64/include/uapi/asm/ptrace.h
+++ b/arch/arm64/include/uapi/asm/ptrace.h
@@ -22,6 +22,7 @@
#include <linux/types.h>
#include <asm/hwcap.h>
+#include <asm/sigcontext.h>
/*
@@ -63,6 +64,8 @@
#ifndef __ASSEMBLY__
+#include <linux/prctl.h>
+
/*
* User structures for general purpose, floating point and debug registers.
*/
@@ -90,6 +93,138 @@ struct user_hwdebug_state {
} dbg_regs[16];
};
+/* SVE/FP/SIMD state (NT_ARM_SVE) */
+
+struct user_sve_header {
+ __u32 size; /* total meaningful regset content in bytes */
+ __u32 max_size; /* maxmium possible size for this thread */
+ __u16 vl; /* current vector length */
+ __u16 max_vl; /* maximum possible vector length */
+ __u16 flags;
+ __u16 __reserved;
+};
+
+/* Definitions for user_sve_header.flags: */
+#define SVE_PT_REGS_MASK (1 << 0)
+
+/* Flags: must be kept in sync with prctl interface in <linux/ptrace.h> */
+#define SVE_PT_REGS_FPSIMD 0
+#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
+
+#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
+#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
+
+
+/*
+ * The remainder of the SVE state follows struct user_sve_header. The
+ * total size of the SVE state (including header) depends on the
+ * metadata in the header: SVE_PT_SIZE(vq, flags) gives the total size
+ * of the state in bytes, including the header.
+ *
+ * Refer to <asm/sigcontext.h> for details of how to pass the correct
+ * "vq" argument to these macros.
+ */
+
+/* Offset from the start of struct user_sve_header to the register data */
+#define SVE_PT_REGS_OFFSET \
+ ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+/*
+ * The register data content and layout depends on the value of the
+ * flags field.
+ */
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD case:
+ *
+ * The payload starts at offset SVE_PT_FPSIMD_OFFSET, and is of type
+ * struct user_fpsimd_state. Additional data might be appended in the
+ * future: use SVE_PT_FPSIMD_SIZE(vq, flags) to compute the total size.
+ * SVE_PT_FPSIMD_SIZE(vq, flags) will never be less than
+ * sizeof(struct user_fpsimd_state).
+ */
+
+#define SVE_PT_FPSIMD_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_FPSIMD_SIZE(vq, flags) (sizeof(struct user_fpsimd_state))
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE case:
+ *
+ * The payload starts at offset SVE_PT_SVE_OFFSET, and is of size
+ * SVE_PT_SVE_SIZE(vq, flags).
+ *
+ * Additional macros describe the contents and layout of the payload.
+ * For each, SVE_PT_SVE_x_OFFSET(args) is the start offset relative to
+ * the start of struct user_sve_header, and SVE_PT_SVE_x_SIZE(args) is
+ * the size in bytes:
+ *
+ * x type description
+ * - ---- -----------
+ * ZREGS \
+ * ZREG |
+ * PREGS | refer to <asm/sigcontext.h>
+ * PREG |
+ * FFR /
+ *
+ * FPSR uint32_t FPSR
+ * FPCR uint32_t FPCR
+ *
+ * Additional data might be appended in the future.
+ */
+
+#define SVE_PT_SVE_ZREG_SIZE(vq) SVE_SIG_ZREG_SIZE(vq)
+#define SVE_PT_SVE_PREG_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
+#define SVE_PT_SVE_FFR_SIZE(vq) SVE_SIG_FFR_SIZE(vq)
+#define SVE_PT_SVE_FPSR_SIZE sizeof(__u32)
+#define SVE_PT_SVE_FPCR_SIZE sizeof(__u32)
+
+#define __SVE_SIG_TO_PT(offset) \
+ ((offset) - SVE_SIG_REGS_OFFSET + SVE_PT_REGS_OFFSET)
+
+#define SVE_PT_SVE_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_SVE_ZREGS_OFFSET \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREGS_OFFSET)
+#define SVE_PT_SVE_ZREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREG_OFFSET(vq, n))
+#define SVE_PT_SVE_ZREGS_SIZE(vq) \
+ (SVE_PT_SVE_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_PT_SVE_ZREGS_OFFSET)
+
+#define SVE_PT_SVE_PREGS_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREGS_OFFSET(vq))
+#define SVE_PT_SVE_PREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREG_OFFSET(vq, n))
+#define SVE_PT_SVE_PREGS_SIZE(vq) \
+ (SVE_PT_SVE_PREG_OFFSET(vq, SVE_NUM_PREGS) - \
+ SVE_PT_SVE_PREGS_OFFSET(vq))
+
+#define SVE_PT_SVE_FFR_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_FFR_OFFSET(vq))
+
+#define SVE_PT_SVE_FPSR_OFFSET(vq) \
+ ((SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq) + \
+ (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+#define SVE_PT_SVE_FPCR_OFFSET(vq) \
+ (SVE_PT_SVE_FPSR_OFFSET(vq) + SVE_PT_SVE_FPSR_SIZE)
+
+/*
+ * Any future extension appended after FPCR must be aligned to the next
+ * 128-bit boundary.
+ */
+
+#define SVE_PT_SVE_SIZE(vq, flags) \
+ ((SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE \
+ - SVE_PT_SVE_OFFSET + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+#define SVE_PT_SIZE(vq, flags) \
+ (((flags) & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE ? \
+ SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, flags) \
+ : SVE_PT_FPSIMD_OFFSET + SVE_PT_FPSIMD_SIZE(vq, flags))
+
#endif /* __ASSEMBLY__ */
#endif /* _UAPI__ASM_PTRACE_H */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index fff9fcf..361c019 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -303,6 +303,37 @@ void sve_alloc(struct task_struct *task)
BUG_ON(!task->thread.sve_state);
}
+void fpsimd_sync_to_sve(struct task_struct *task)
+{
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ fpsimd_to_sve(task);
+}
+
+void sve_sync_to_fpsimd(struct task_struct *task)
+{
+ if (test_tsk_thread_flag(task, TIF_SVE))
+ sve_to_fpsimd(task);
+}
+
+void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
+{
+ unsigned int vq;
+ void *sst = task->thread.sve_state;
+ struct fpsimd_state const *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+
+ memset(sst, 0, SVE_SIG_REGS_SIZE(vq));
+
+ for (i = 0; i < 32; ++i)
+ memcpy(ZREG(sst, vq, i), &fst->vregs[i],
+ sizeof(fst->vregs[i]));
+}
+
/*
* Handle SVE state across fork():
*
@@ -459,10 +490,17 @@ static void __init sve_efi_setup(void)
* This is evidence of a crippled system and we are returning void,
* so no attempt is made to handle this situation here.
*/
- BUG_ON(!sve_vl_valid(sve_max_vl));
+ if (!sve_vl_valid(sve_max_vl))
+ goto fail;
+
efi_sve_state = __alloc_percpu(
SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
if (!efi_sve_state)
+ goto fail;
+
+ return;
+
+fail:
panic("Cannot allocate percpu memory for EFI SVE save/restore");
}
diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
index 9cbb612..5ef4735b 100644
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -32,6 +32,7 @@
#include <linux/security.h>
#include <linux/init.h>
#include <linux/signal.h>
+#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/perf_event.h>
#include <linux/hw_breakpoint.h>
@@ -40,6 +41,7 @@
#include <linux/elf.h>
#include <asm/compat.h>
+#include <asm/cpufeature.h>
#include <asm/debug-monitors.h>
#include <asm/pgtable.h>
#include <asm/stacktrace.h>
@@ -618,33 +620,66 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
/*
* TODO: update fp accessors for lazy context switching (sync/flush hwstate)
*/
-static int fpr_get(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- void *kbuf, void __user *ubuf)
+static int __fpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf, unsigned int start_pos)
{
struct user_fpsimd_state *uregs;
+
+ sve_sync_to_fpsimd(target);
+
uregs = &target->thread.fpsimd_state.user_fpsimd;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs,
+ start_pos, start_pos + sizeof(*uregs));
+}
+
+static int fpr_get(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
if (target == current)
fpsimd_preserve_current_state();
- return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf, 0);
}
-static int fpr_set(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- const void *kbuf, const void __user *ubuf)
+static int __fpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf,
+ unsigned int start_pos)
{
int ret;
struct user_fpsimd_state newstate =
target->thread.fpsimd_state.user_fpsimd;
- ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
+ sve_sync_to_fpsimd(target);
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate,
+ start_pos, start_pos + sizeof(newstate));
if (ret)
return ret;
target->thread.fpsimd_state.user_fpsimd = newstate;
+
+ return ret;
+}
+
+static int fpr_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, 0);
+ if (ret)
+ return ret;
+
+ sve_sync_from_fpsimd_zeropad(target);
fpsimd_flush_task_state(target);
+
return ret;
}
@@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
return ret;
}
+#ifdef CONFIG_ARM64_SVE
+
+static void sve_init_header_from_task(struct user_sve_header *header,
+ struct task_struct *target)
+{
+ unsigned int vq;
+
+ memset(header, 0, sizeof(*header));
+
+ header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
+ SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
+ if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
+ header->flags |= SVE_PT_VL_INHERIT;
+
+ header->vl = target->thread.sve_vl;
+ vq = sve_vq_from_vl(header->vl);
+
+ if (WARN_ON(!sve_vl_valid(sve_max_vl)))
+ header->max_vl = header->vl;
+
+ header->size = SVE_PT_SIZE(vq, header->flags);
+ header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
+ SVE_PT_REGS_SVE);
+}
+
+static unsigned int sve_size_from_header(struct user_sve_header const *header)
+{
+ return ALIGN(header->size, SVE_VQ_BYTES);
+}
+
+static unsigned int sve_get_size(struct task_struct *target,
+ const struct user_regset *regset)
+{
+ struct user_sve_header header;
+
+ if (!system_supports_sve())
+ return 0;
+
+ sve_init_header_from_task(&header, target);
+ return sve_size_from_header(&header);
+}
+
+static int sve_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ sve_init_header_from_task(&header, target);
+ vq = sve_vq_from_vl(header.vl);
+
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ return ret;
+
+ if (target == current)
+ fpsimd_preserve_current_state();
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD)
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+
+ /* Otherwise: full SVE case */
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ return ret;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = sve_size_from_header(&header);
+ return user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+}
+
+static int sve_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ if (count < sizeof(header))
+ return -EINVAL;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ goto out;
+
+ /*
+ * Apart from PT_SVE_REGS_MASK, all PT_SVE_* flags are consumed by
+ * sve_set_vector_length(), which will also validate them for us:
+ */
+ ret = sve_set_vector_length(target, header.vl,
+ header.flags & ~SVE_PT_REGS_MASK);
+ if (ret)
+ goto out;
+
+ /* Actual VL set may be less than the user asked for: */
+ vq = sve_vq_from_vl(target->thread.sve_vl);
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD) {
+ sve_sync_to_fpsimd(target);
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+ clear_tsk_thread_flag(target, TIF_SVE);
+ goto out;
+ }
+
+ /* Otherwise: full SVE case */
+
+ /*
+ * If setting a different VL from the requested VL and there is
+ * register data, the data layout will be wrong: don't even
+ * try to set the registers in this case.
+ */
+ if (count && vq != sve_vq_from_vl(header.vl)) {
+ ret = -EIO;
+ goto out;
+ }
+
+ sve_alloc(target);
+ fpsimd_sync_to_sve(target);
+ set_tsk_thread_flag(target, TIF_SVE);
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ goto out;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ goto out;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+
+out:
+ fpsimd_flush_task_state(target);
+ return ret;
+}
+
+#endif /* CONFIG_ARM64_SVE */
+
enum aarch64_regset {
REGSET_GPR,
REGSET_FPR,
@@ -711,6 +950,9 @@ enum aarch64_regset {
REGSET_HW_WATCH,
#endif
REGSET_SYSTEM_CALL,
+#ifdef CONFIG_ARM64_SVE
+ REGSET_SVE,
+#endif
};
static const struct user_regset aarch64_regsets[] = {
@@ -768,6 +1010,18 @@ static const struct user_regset aarch64_regsets[] = {
.get = system_call_get,
.set = system_call_set,
},
+#ifdef CONFIG_ARM64_SVE
+ [REGSET_SVE] = { /* Scalable Vector Extension */
+ .core_note_type = NT_ARM_SVE,
+ .n = DIV_ROUND_UP(SVE_PT_SIZE(SVE_VQ_MAX, SVE_PT_REGS_SVE),
+ SVE_VQ_BYTES),
+ .size = SVE_VQ_BYTES,
+ .align = SVE_VQ_BYTES,
+ .get = sve_get,
+ .set = sve_set,
+ .get_size = sve_get_size,
+ },
+#endif
};
static const struct user_regset_view user_aarch64_view = {
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index b5280db..735b8f4 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -416,6 +416,7 @@ typedef struct elf64_shdr {
#define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
#define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
#define NT_ARM_SYSTEM_CALL 0x404 /* ARM system call number */
+#define NT_ARM_SVE 0x405 /* ARM Scalable Vector Extension registers */
#define NT_METAG_CBUF 0x500 /* Metag catch buffer registers */
#define NT_METAG_RPIPE 0x501 /* Metag read pipeline state */
#define NT_METAG_TLS 0x502 /* Metag TLS pointer */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch defines and implements a new regset NT_ARM_SVE, which
describes a thread's SVE register state. This allows a debugger to
manipulate the SVE state, as well as being included in ELF
coredumps for post-mortem debugging.
Because the regset size and layout are dependent on the thread's
current vector length, it is not possible to define a C struct to
describe the regset contents as is done for existing regsets.
Instead, and for the same reasons, NT_ARM_SVE is based on the
freeform variable-layout approach used for the SVE signal frame.
Additionally, to reduce debug overhead when debugging threads that
might or might not have live SVE register state, NT_ARM_SVE may be
presented in one of two different formats: the old struct
user_fpsimd_state format is embedded for describing the state of a
thread with no live SVE state, whereas a new variable-layout
structure is embedded for describing live SVE state. This avoids a
debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
allows existing userspace code to handle the non-SVE case without
too much modification.
For this to work, NT_ARM_SVE is defined with a fixed-format header
of type struct user_sve_header, which the recipient can use to
figure out the content, size and layout of the reset of the regset.
Accessor macros are defined to allow the vector-length-dependent
parts of the regset to be manipulated.
Signed-off-by: Alan Hayward <alan.hayward@arm.com>
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Other changes related to Alex Benn?e's comments:
* Migrate to SVE_VQ_BYTES instead of magic numbers.
Requested by Alex Benn?e:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other:
* [ABI fix] Bail out with -EIO if attempting to set the
SVE regs for an unsupported VL, instead of misparsing the regset data.
* Replace some in-kernel open-coded arithmetic with ALIGN()/
DIV_ROUND_UP().
---
arch/arm64/include/asm/fpsimd.h | 13 +-
arch/arm64/include/uapi/asm/ptrace.h | 135 ++++++++++++++++++
arch/arm64/kernel/fpsimd.c | 40 +++++-
arch/arm64/kernel/ptrace.c | 270 +++++++++++++++++++++++++++++++++--
include/uapi/linux/elf.h | 1 +
5 files changed, 449 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 6c22624..2723cca 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -38,13 +38,16 @@ struct fpsimd_state {
__uint128_t vregs[32];
u32 fpsr;
u32 fpcr;
+ /*
+ * For ptrace compatibility, pad to next 128-bit
+ * boundary here if extending this struct.
+ */
};
};
/* the id of the last cpu to have restored this state */
unsigned int cpu;
};
-
#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
/* Masks for extracting the FPSR and FPCR from the FPSCR */
#define VFP_FPSCR_STAT_MASK 0xf800009f
@@ -89,6 +92,10 @@ extern void sve_alloc(struct task_struct *task);
extern void fpsimd_release_thread(struct task_struct *task);
extern void fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src);
+extern void fpsimd_sync_to_sve(struct task_struct *task);
+extern void sve_sync_to_fpsimd(struct task_struct *task);
+extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
+
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
@@ -103,6 +110,10 @@ static void __maybe_unused sve_alloc(struct task_struct *task) { }
static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src) { }
+static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
+static void __maybe_unused sve_sync_from_fpsimd_zeropad(
+ struct task_struct *task) { }
+
static void __maybe_unused sve_init_vq_map(void) { }
static void __maybe_unused sve_update_vq_map(void) { }
static int __maybe_unused sve_verify_vq_map(void) { return 0; }
diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
index d1ff83d..1915ab0 100644
--- a/arch/arm64/include/uapi/asm/ptrace.h
+++ b/arch/arm64/include/uapi/asm/ptrace.h
@@ -22,6 +22,7 @@
#include <linux/types.h>
#include <asm/hwcap.h>
+#include <asm/sigcontext.h>
/*
@@ -63,6 +64,8 @@
#ifndef __ASSEMBLY__
+#include <linux/prctl.h>
+
/*
* User structures for general purpose, floating point and debug registers.
*/
@@ -90,6 +93,138 @@ struct user_hwdebug_state {
} dbg_regs[16];
};
+/* SVE/FP/SIMD state (NT_ARM_SVE) */
+
+struct user_sve_header {
+ __u32 size; /* total meaningful regset content in bytes */
+ __u32 max_size; /* maxmium possible size for this thread */
+ __u16 vl; /* current vector length */
+ __u16 max_vl; /* maximum possible vector length */
+ __u16 flags;
+ __u16 __reserved;
+};
+
+/* Definitions for user_sve_header.flags: */
+#define SVE_PT_REGS_MASK (1 << 0)
+
+/* Flags: must be kept in sync with prctl interface in <linux/ptrace.h> */
+#define SVE_PT_REGS_FPSIMD 0
+#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
+
+#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
+#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
+
+
+/*
+ * The remainder of the SVE state follows struct user_sve_header. The
+ * total size of the SVE state (including header) depends on the
+ * metadata in the header: SVE_PT_SIZE(vq, flags) gives the total size
+ * of the state in bytes, including the header.
+ *
+ * Refer to <asm/sigcontext.h> for details of how to pass the correct
+ * "vq" argument to these macros.
+ */
+
+/* Offset from the start of struct user_sve_header to the register data */
+#define SVE_PT_REGS_OFFSET \
+ ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+/*
+ * The register data content and layout depends on the value of the
+ * flags field.
+ */
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD case:
+ *
+ * The payload starts at offset SVE_PT_FPSIMD_OFFSET, and is of type
+ * struct user_fpsimd_state. Additional data might be appended in the
+ * future: use SVE_PT_FPSIMD_SIZE(vq, flags) to compute the total size.
+ * SVE_PT_FPSIMD_SIZE(vq, flags) will never be less than
+ * sizeof(struct user_fpsimd_state).
+ */
+
+#define SVE_PT_FPSIMD_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_FPSIMD_SIZE(vq, flags) (sizeof(struct user_fpsimd_state))
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE case:
+ *
+ * The payload starts at offset SVE_PT_SVE_OFFSET, and is of size
+ * SVE_PT_SVE_SIZE(vq, flags).
+ *
+ * Additional macros describe the contents and layout of the payload.
+ * For each, SVE_PT_SVE_x_OFFSET(args) is the start offset relative to
+ * the start of struct user_sve_header, and SVE_PT_SVE_x_SIZE(args) is
+ * the size in bytes:
+ *
+ * x type description
+ * - ---- -----------
+ * ZREGS \
+ * ZREG |
+ * PREGS | refer to <asm/sigcontext.h>
+ * PREG |
+ * FFR /
+ *
+ * FPSR uint32_t FPSR
+ * FPCR uint32_t FPCR
+ *
+ * Additional data might be appended in the future.
+ */
+
+#define SVE_PT_SVE_ZREG_SIZE(vq) SVE_SIG_ZREG_SIZE(vq)
+#define SVE_PT_SVE_PREG_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
+#define SVE_PT_SVE_FFR_SIZE(vq) SVE_SIG_FFR_SIZE(vq)
+#define SVE_PT_SVE_FPSR_SIZE sizeof(__u32)
+#define SVE_PT_SVE_FPCR_SIZE sizeof(__u32)
+
+#define __SVE_SIG_TO_PT(offset) \
+ ((offset) - SVE_SIG_REGS_OFFSET + SVE_PT_REGS_OFFSET)
+
+#define SVE_PT_SVE_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_SVE_ZREGS_OFFSET \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREGS_OFFSET)
+#define SVE_PT_SVE_ZREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREG_OFFSET(vq, n))
+#define SVE_PT_SVE_ZREGS_SIZE(vq) \
+ (SVE_PT_SVE_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_PT_SVE_ZREGS_OFFSET)
+
+#define SVE_PT_SVE_PREGS_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREGS_OFFSET(vq))
+#define SVE_PT_SVE_PREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREG_OFFSET(vq, n))
+#define SVE_PT_SVE_PREGS_SIZE(vq) \
+ (SVE_PT_SVE_PREG_OFFSET(vq, SVE_NUM_PREGS) - \
+ SVE_PT_SVE_PREGS_OFFSET(vq))
+
+#define SVE_PT_SVE_FFR_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_FFR_OFFSET(vq))
+
+#define SVE_PT_SVE_FPSR_OFFSET(vq) \
+ ((SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq) + \
+ (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+#define SVE_PT_SVE_FPCR_OFFSET(vq) \
+ (SVE_PT_SVE_FPSR_OFFSET(vq) + SVE_PT_SVE_FPSR_SIZE)
+
+/*
+ * Any future extension appended after FPCR must be aligned to the next
+ * 128-bit boundary.
+ */
+
+#define SVE_PT_SVE_SIZE(vq, flags) \
+ ((SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE \
+ - SVE_PT_SVE_OFFSET + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+#define SVE_PT_SIZE(vq, flags) \
+ (((flags) & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE ? \
+ SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, flags) \
+ : SVE_PT_FPSIMD_OFFSET + SVE_PT_FPSIMD_SIZE(vq, flags))
+
#endif /* __ASSEMBLY__ */
#endif /* _UAPI__ASM_PTRACE_H */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index fff9fcf..361c019 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -303,6 +303,37 @@ void sve_alloc(struct task_struct *task)
BUG_ON(!task->thread.sve_state);
}
+void fpsimd_sync_to_sve(struct task_struct *task)
+{
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ fpsimd_to_sve(task);
+}
+
+void sve_sync_to_fpsimd(struct task_struct *task)
+{
+ if (test_tsk_thread_flag(task, TIF_SVE))
+ sve_to_fpsimd(task);
+}
+
+void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
+{
+ unsigned int vq;
+ void *sst = task->thread.sve_state;
+ struct fpsimd_state const *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+
+ memset(sst, 0, SVE_SIG_REGS_SIZE(vq));
+
+ for (i = 0; i < 32; ++i)
+ memcpy(ZREG(sst, vq, i), &fst->vregs[i],
+ sizeof(fst->vregs[i]));
+}
+
/*
* Handle SVE state across fork():
*
@@ -459,10 +490,17 @@ static void __init sve_efi_setup(void)
* This is evidence of a crippled system and we are returning void,
* so no attempt is made to handle this situation here.
*/
- BUG_ON(!sve_vl_valid(sve_max_vl));
+ if (!sve_vl_valid(sve_max_vl))
+ goto fail;
+
efi_sve_state = __alloc_percpu(
SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
if (!efi_sve_state)
+ goto fail;
+
+ return;
+
+fail:
panic("Cannot allocate percpu memory for EFI SVE save/restore");
}
diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
index 9cbb612..5ef4735b 100644
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -32,6 +32,7 @@
#include <linux/security.h>
#include <linux/init.h>
#include <linux/signal.h>
+#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/perf_event.h>
#include <linux/hw_breakpoint.h>
@@ -40,6 +41,7 @@
#include <linux/elf.h>
#include <asm/compat.h>
+#include <asm/cpufeature.h>
#include <asm/debug-monitors.h>
#include <asm/pgtable.h>
#include <asm/stacktrace.h>
@@ -618,33 +620,66 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
/*
* TODO: update fp accessors for lazy context switching (sync/flush hwstate)
*/
-static int fpr_get(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- void *kbuf, void __user *ubuf)
+static int __fpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf, unsigned int start_pos)
{
struct user_fpsimd_state *uregs;
+
+ sve_sync_to_fpsimd(target);
+
uregs = &target->thread.fpsimd_state.user_fpsimd;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs,
+ start_pos, start_pos + sizeof(*uregs));
+}
+
+static int fpr_get(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
if (target == current)
fpsimd_preserve_current_state();
- return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf, 0);
}
-static int fpr_set(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- const void *kbuf, const void __user *ubuf)
+static int __fpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf,
+ unsigned int start_pos)
{
int ret;
struct user_fpsimd_state newstate =
target->thread.fpsimd_state.user_fpsimd;
- ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
+ sve_sync_to_fpsimd(target);
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate,
+ start_pos, start_pos + sizeof(newstate));
if (ret)
return ret;
target->thread.fpsimd_state.user_fpsimd = newstate;
+
+ return ret;
+}
+
+static int fpr_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, 0);
+ if (ret)
+ return ret;
+
+ sve_sync_from_fpsimd_zeropad(target);
fpsimd_flush_task_state(target);
+
return ret;
}
@@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
return ret;
}
+#ifdef CONFIG_ARM64_SVE
+
+static void sve_init_header_from_task(struct user_sve_header *header,
+ struct task_struct *target)
+{
+ unsigned int vq;
+
+ memset(header, 0, sizeof(*header));
+
+ header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
+ SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
+ if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
+ header->flags |= SVE_PT_VL_INHERIT;
+
+ header->vl = target->thread.sve_vl;
+ vq = sve_vq_from_vl(header->vl);
+
+ if (WARN_ON(!sve_vl_valid(sve_max_vl)))
+ header->max_vl = header->vl;
+
+ header->size = SVE_PT_SIZE(vq, header->flags);
+ header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
+ SVE_PT_REGS_SVE);
+}
+
+static unsigned int sve_size_from_header(struct user_sve_header const *header)
+{
+ return ALIGN(header->size, SVE_VQ_BYTES);
+}
+
+static unsigned int sve_get_size(struct task_struct *target,
+ const struct user_regset *regset)
+{
+ struct user_sve_header header;
+
+ if (!system_supports_sve())
+ return 0;
+
+ sve_init_header_from_task(&header, target);
+ return sve_size_from_header(&header);
+}
+
+static int sve_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ sve_init_header_from_task(&header, target);
+ vq = sve_vq_from_vl(header.vl);
+
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ return ret;
+
+ if (target == current)
+ fpsimd_preserve_current_state();
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD)
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+
+ /* Otherwise: full SVE case */
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ return ret;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = sve_size_from_header(&header);
+ return user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+}
+
+static int sve_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ if (count < sizeof(header))
+ return -EINVAL;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ goto out;
+
+ /*
+ * Apart from PT_SVE_REGS_MASK, all PT_SVE_* flags are consumed by
+ * sve_set_vector_length(), which will also validate them for us:
+ */
+ ret = sve_set_vector_length(target, header.vl,
+ header.flags & ~SVE_PT_REGS_MASK);
+ if (ret)
+ goto out;
+
+ /* Actual VL set may be less than the user asked for: */
+ vq = sve_vq_from_vl(target->thread.sve_vl);
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD) {
+ sve_sync_to_fpsimd(target);
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+ clear_tsk_thread_flag(target, TIF_SVE);
+ goto out;
+ }
+
+ /* Otherwise: full SVE case */
+
+ /*
+ * If setting a different VL from the requested VL and there is
+ * register data, the data layout will be wrong: don't even
+ * try to set the registers in this case.
+ */
+ if (count && vq != sve_vq_from_vl(header.vl)) {
+ ret = -EIO;
+ goto out;
+ }
+
+ sve_alloc(target);
+ fpsimd_sync_to_sve(target);
+ set_tsk_thread_flag(target, TIF_SVE);
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ goto out;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ goto out;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+
+out:
+ fpsimd_flush_task_state(target);
+ return ret;
+}
+
+#endif /* CONFIG_ARM64_SVE */
+
enum aarch64_regset {
REGSET_GPR,
REGSET_FPR,
@@ -711,6 +950,9 @@ enum aarch64_regset {
REGSET_HW_WATCH,
#endif
REGSET_SYSTEM_CALL,
+#ifdef CONFIG_ARM64_SVE
+ REGSET_SVE,
+#endif
};
static const struct user_regset aarch64_regsets[] = {
@@ -768,6 +1010,18 @@ static const struct user_regset aarch64_regsets[] = {
.get = system_call_get,
.set = system_call_set,
},
+#ifdef CONFIG_ARM64_SVE
+ [REGSET_SVE] = { /* Scalable Vector Extension */
+ .core_note_type = NT_ARM_SVE,
+ .n = DIV_ROUND_UP(SVE_PT_SIZE(SVE_VQ_MAX, SVE_PT_REGS_SVE),
+ SVE_VQ_BYTES),
+ .size = SVE_VQ_BYTES,
+ .align = SVE_VQ_BYTES,
+ .get = sve_get,
+ .set = sve_set,
+ .get_size = sve_get_size,
+ },
+#endif
};
static const struct user_regset_view user_aarch64_view = {
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index b5280db..735b8f4 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -416,6 +416,7 @@ typedef struct elf64_shdr {
#define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
#define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
#define NT_ARM_SYSTEM_CALL 0x404 /* ARM system call number */
+#define NT_ARM_SVE 0x405 /* ARM Scalable Vector Extension registers */
#define NT_METAG_CBUF 0x500 /* Metag catch buffer registers */
#define NT_METAG_RPIPE 0x501 /* Metag read pipeline state */
#define NT_METAG_TLS 0x502 /* Metag TLS pointer */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 20/28] arm64/sve: Add prctl controls for userspace vector length management
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Andrew Morton
This patch adds two arm64-specific prctls, to permit userspace to
control its vector length:
* PR_SVE_SET_VL: set the thread's SVE vector length and vector
length inheritance mode.
* PR_SVE_GET_VL: get the same information.
Although these calls shadow instruction set features in the SVE
architecture, these prctls provide additional control: the vector
length inheritance mode is Linux-specific and nothing to do with
the architecture, and the architecture does not permit EL0 to set
its own vector length directly. Both can be used in portable tools
without requiring the use of SVE instructions.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/fpsimd.h | 14 ++++++++++++
arch/arm64/include/asm/processor.h | 4 ++++
arch/arm64/kernel/fpsimd.c | 46 ++++++++++++++++++++++++++++++++++++++
include/uapi/linux/prctl.h | 4 ++++
kernel/sys.c | 12 ++++++++++
5 files changed, 80 insertions(+)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 2723cca..d084968 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -17,6 +17,7 @@
#define __ASM_FP_H
#include <asm/ptrace.h>
+#include <asm/errno.h>
#ifndef __ASSEMBLY__
@@ -99,6 +100,9 @@ extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
+extern int sve_set_current_vl(unsigned long arg);
+extern int sve_get_current_vl(void);
+
extern void __init sve_init_vq_map(void);
extern void sve_update_vq_map(void);
extern int sve_verify_vq_map(void);
@@ -114,6 +118,16 @@ static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
static void __maybe_unused sve_sync_from_fpsimd_zeropad(
struct task_struct *task) { }
+static int __maybe_unused sve_set_current_vl(unsigned long arg)
+{
+ return -EINVAL;
+}
+
+static int __maybe_unused sve_get_current_vl(void)
+{
+ return -EINVAL;
+}
+
static void __maybe_unused sve_init_vq_map(void) { }
static void __maybe_unused sve_update_vq_map(void) { }
static int __maybe_unused sve_verify_vq_map(void) { return 0; }
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 3faceac..df66452 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -197,4 +197,8 @@ static inline void spin_lock_prefetch(const void *ptr)
int cpu_enable_pan(void *__unused);
int cpu_enable_cache_maint_trap(void *__unused);
+/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
+#define SVE_SET_VL(arg) sve_set_current_vl(arg)
+#define SVE_GET_VL() sve_get_current_vl()
+
#endif /* __ASM_PROCESSOR_H */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 361c019..42e8331 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -27,6 +27,7 @@
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/percpu.h>
+#include <linux/prctl.h>
#include <linux/preempt.h>
#include <linux/prctl.h>
#include <linux/ptrace.h>
@@ -420,6 +421,51 @@ int sve_set_vector_length(struct task_struct *task,
return 0;
}
+/*
+ * Encode the current vector length and flags for return.
+ * This is only required for prctl(): ptrace has separate fields
+ */
+static int sve_prctl_status(void)
+{
+ int ret = current->thread.sve_vl;
+
+ if (test_thread_flag(TIF_SVE_VL_INHERIT))
+ ret |= PR_SVE_VL_INHERIT;
+
+ return ret;
+}
+
+/* PR_SVE_SET_VL */
+int sve_set_current_vl(unsigned long arg)
+{
+ unsigned long vl, flags;
+ int ret;
+
+ vl = arg & PR_SVE_VL_LEN_MASK;
+ flags = arg & ~vl;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ preempt_disable();
+ ret = sve_set_vector_length(current, vl, flags);
+ preempt_enable();
+
+ if (ret)
+ return ret;
+
+ return sve_prctl_status();
+}
+
+/* PR_SVE_GET_VL */
+int sve_get_current_vl(void)
+{
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ return sve_prctl_status();
+}
+
static unsigned long *sve_alloc_vq_map(void)
{
return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index 1b64901..1ef9370 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -198,7 +198,11 @@ struct prctl_mm_map {
# define PR_CAP_AMBIENT_CLEAR_ALL 4
/* arm64 Scalable Vector Extension controls */
+/* Flag values must be kept in sync with ptrace NT_ARM_SVE interface */
+#define PR_SVE_SET_VL 48 /* set task vector length */
# define PR_SVE_SET_VL_ONEXEC (1 << 18) /* defer effect until exec */
+#define PR_SVE_GET_VL 49 /* get task vector length */
+/* Bits common to PR_SVE_SET_VL and PR_SVE_GET_VL */
# define PR_SVE_VL_LEN_MASK 0xffff
# define PR_SVE_VL_INHERIT (1 << 17) /* inherit across exec */
diff --git a/kernel/sys.c b/kernel/sys.c
index 2855ee7..f8215a6 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -110,6 +110,12 @@
#ifndef SET_FP_MODE
# define SET_FP_MODE(a,b) (-EINVAL)
#endif
+#ifndef SVE_SET_VL
+# define SVE_SET_VL(a) (-EINVAL)
+#endif
+#ifndef SVE_GET_VL
+# define SVE_GET_VL() (-EINVAL)
+#endif
/*
* this is where the system-wide overflow UID and GID are defined, for
@@ -2389,6 +2395,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
case PR_GET_FP_MODE:
error = GET_FP_MODE(me);
break;
+ case PR_SVE_SET_VL:
+ error = SVE_SET_VL(arg2);
+ break;
+ case PR_SVE_GET_VL:
+ error = SVE_GET_VL();
+ break;
default:
error = -EINVAL;
break;
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 20/28] arm64/sve: Add prctl controls for userspace vector length management
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch adds two arm64-specific prctls, to permit userspace to
control its vector length:
* PR_SVE_SET_VL: set the thread's SVE vector length and vector
length inheritance mode.
* PR_SVE_GET_VL: get the same information.
Although these calls shadow instruction set features in the SVE
architecture, these prctls provide additional control: the vector
length inheritance mode is Linux-specific and nothing to do with
the architecture, and the architecture does not permit EL0 to set
its own vector length directly. Both can be used in portable tools
without requiring the use of SVE instructions.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/include/asm/fpsimd.h | 14 ++++++++++++
arch/arm64/include/asm/processor.h | 4 ++++
arch/arm64/kernel/fpsimd.c | 46 ++++++++++++++++++++++++++++++++++++++
include/uapi/linux/prctl.h | 4 ++++
kernel/sys.c | 12 ++++++++++
5 files changed, 80 insertions(+)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 2723cca..d084968 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -17,6 +17,7 @@
#define __ASM_FP_H
#include <asm/ptrace.h>
+#include <asm/errno.h>
#ifndef __ASSEMBLY__
@@ -99,6 +100,9 @@ extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
+extern int sve_set_current_vl(unsigned long arg);
+extern int sve_get_current_vl(void);
+
extern void __init sve_init_vq_map(void);
extern void sve_update_vq_map(void);
extern int sve_verify_vq_map(void);
@@ -114,6 +118,16 @@ static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
static void __maybe_unused sve_sync_from_fpsimd_zeropad(
struct task_struct *task) { }
+static int __maybe_unused sve_set_current_vl(unsigned long arg)
+{
+ return -EINVAL;
+}
+
+static int __maybe_unused sve_get_current_vl(void)
+{
+ return -EINVAL;
+}
+
static void __maybe_unused sve_init_vq_map(void) { }
static void __maybe_unused sve_update_vq_map(void) { }
static int __maybe_unused sve_verify_vq_map(void) { return 0; }
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 3faceac..df66452 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -197,4 +197,8 @@ static inline void spin_lock_prefetch(const void *ptr)
int cpu_enable_pan(void *__unused);
int cpu_enable_cache_maint_trap(void *__unused);
+/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
+#define SVE_SET_VL(arg) sve_set_current_vl(arg)
+#define SVE_GET_VL() sve_get_current_vl()
+
#endif /* __ASM_PROCESSOR_H */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 361c019..42e8331 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -27,6 +27,7 @@
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/percpu.h>
+#include <linux/prctl.h>
#include <linux/preempt.h>
#include <linux/prctl.h>
#include <linux/ptrace.h>
@@ -420,6 +421,51 @@ int sve_set_vector_length(struct task_struct *task,
return 0;
}
+/*
+ * Encode the current vector length and flags for return.
+ * This is only required for prctl(): ptrace has separate fields
+ */
+static int sve_prctl_status(void)
+{
+ int ret = current->thread.sve_vl;
+
+ if (test_thread_flag(TIF_SVE_VL_INHERIT))
+ ret |= PR_SVE_VL_INHERIT;
+
+ return ret;
+}
+
+/* PR_SVE_SET_VL */
+int sve_set_current_vl(unsigned long arg)
+{
+ unsigned long vl, flags;
+ int ret;
+
+ vl = arg & PR_SVE_VL_LEN_MASK;
+ flags = arg & ~vl;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ preempt_disable();
+ ret = sve_set_vector_length(current, vl, flags);
+ preempt_enable();
+
+ if (ret)
+ return ret;
+
+ return sve_prctl_status();
+}
+
+/* PR_SVE_GET_VL */
+int sve_get_current_vl(void)
+{
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ return sve_prctl_status();
+}
+
static unsigned long *sve_alloc_vq_map(void)
{
return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
index 1b64901..1ef9370 100644
--- a/include/uapi/linux/prctl.h
+++ b/include/uapi/linux/prctl.h
@@ -198,7 +198,11 @@ struct prctl_mm_map {
# define PR_CAP_AMBIENT_CLEAR_ALL 4
/* arm64 Scalable Vector Extension controls */
+/* Flag values must be kept in sync with ptrace NT_ARM_SVE interface */
+#define PR_SVE_SET_VL 48 /* set task vector length */
# define PR_SVE_SET_VL_ONEXEC (1 << 18) /* defer effect until exec */
+#define PR_SVE_GET_VL 49 /* get task vector length */
+/* Bits common to PR_SVE_SET_VL and PR_SVE_GET_VL */
# define PR_SVE_VL_LEN_MASK 0xffff
# define PR_SVE_VL_INHERIT (1 << 17) /* inherit across exec */
diff --git a/kernel/sys.c b/kernel/sys.c
index 2855ee7..f8215a6 100644
--- a/kernel/sys.c
+++ b/kernel/sys.c
@@ -110,6 +110,12 @@
#ifndef SET_FP_MODE
# define SET_FP_MODE(a,b) (-EINVAL)
#endif
+#ifndef SVE_SET_VL
+# define SVE_SET_VL(a) (-EINVAL)
+#endif
+#ifndef SVE_GET_VL
+# define SVE_GET_VL() (-EINVAL)
+#endif
/*
* this is where the system-wide overflow UID and GID are defined, for
@@ -2389,6 +2395,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
case PR_GET_FP_MODE:
error = GET_FP_MODE(me);
break;
+ case PR_SVE_SET_VL:
+ error = SVE_SET_VL(arg2);
+ break;
+ case PR_SVE_GET_VL:
+ error = SVE_GET_VL();
+ break;
default:
error = -EINVAL;
break;
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 21/28] arm64/sve: Add sysctl to set the default vector length for new processes
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Because of the effect of SVE on the size of the signal frame, the
default vector length used for new processes involves a tradeoff
between performance of SVE-enabled software on the one hand, and
reliability of non-SVE-aware software on the other hand.
For this reason, the best choice depends on the repertoire of
userspace software in use and is thus best left up to distro
maintainers, sysadmins and developers.
If CONFIG_SYSCTL is enabled, this patch exposes the default vector
length in /proc/sys/abi/sve_default_vector_length, where boot
scripts or the adventurous can poke it.
In common with other arm64 ABI sysctls, this control is currently
global: setting it requires CAP_SYS_ADMIN in the root user
namespace, but the value set is effective for subsequent execs in
all namespaces. The control only affects _new_ processes, however:
changing it does not affect the vector length of any existing
process.
The intended usage model is that if userspace is known to be fully
SVE-tolerant (or a developer is curious to find out) then init
scripts can crank this up during startup.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Changes requested by Alex Bennée:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
---
arch/arm64/kernel/fpsimd.c | 62 +++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 61 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 42e8331..b430ee0 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -34,6 +34,7 @@
#include <linux/sched/signal.h>
#include <linux/signal.h>
#include <linux/slab.h>
+#include <linux/sysctl.h>
#include <asm/fpsimd.h>
#include <asm/cputype.h>
@@ -244,6 +245,65 @@ static unsigned int find_supported_vector_length(unsigned int vl)
return sve_vl_from_vq(bit_to_vq(bit));
}
+#ifdef CONFIG_SYSCTL
+
+static int sve_proc_do_default_vl(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp,
+ loff_t *ppos)
+{
+ int ret;
+ int vl = sve_default_vl;
+ struct ctl_table tmp_table = {
+ .data = &vl,
+ .maxlen = sizeof(vl),
+ };
+
+ ret = proc_dointvec(&tmp_table, write, buffer, lenp, ppos);
+ if (ret || !write)
+ return ret;
+
+ /* Writing -1 has the special meaning "set to max": */
+ if (vl == -1) {
+ /* Fail safe if sve_max_vl wasn't initialised */
+ if (WARN_ON(!sve_vl_valid(sve_max_vl)))
+ vl = SVE_VL_MIN;
+ else
+ vl = sve_max_vl;
+
+ goto chosen;
+ }
+
+ if (!sve_vl_valid(vl))
+ return -EINVAL;
+
+ vl = find_supported_vector_length(vl);
+chosen:
+ sve_default_vl = vl;
+ return 0;
+}
+
+static struct ctl_table sve_default_vl_table[] = {
+ {
+ .procname = "sve_default_vector_length",
+ .mode = 0644,
+ .proc_handler = sve_proc_do_default_vl,
+ },
+ { }
+};
+
+static int __init sve_sysctl_init(void)
+{
+ if (system_supports_sve())
+ if (!register_sysctl("abi", sve_default_vl_table))
+ return -EINVAL;
+
+ return 0;
+}
+
+#else /* ! CONFIG_SYSCTL */
+static int __init sve_sysctl_init(void) { return 0; }
+#endif /* ! CONFIG_SYSCTL */
+
#define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
(SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
@@ -1030,6 +1090,6 @@ static int __init fpsimd_init(void)
if (!(elf_hwcap & HWCAP_ASIMD))
pr_notice("Advanced SIMD is not implemented\n");
- return 0;
+ return sve_sysctl_init();
}
late_initcall(fpsimd_init);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 21/28] arm64/sve: Add sysctl to set the default vector length for new processes
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Because of the effect of SVE on the size of the signal frame, the
default vector length used for new processes involves a tradeoff
between performance of SVE-enabled software on the one hand, and
reliability of non-SVE-aware software on the other hand.
For this reason, the best choice depends on the repertoire of
userspace software in use and is thus best left up to distro
maintainers, sysadmins and developers.
If CONFIG_SYSCTL is enabled, this patch exposes the default vector
length in /proc/sys/abi/sve_default_vector_length, where boot
scripts or the adventurous can poke it.
In common with other arm64 ABI sysctls, this control is currently
global: setting it requires CAP_SYS_ADMIN in the root user
namespace, but the value set is effective for subsequent execs in
all namespaces. The control only affects _new_ processes, however:
changing it does not affect the vector length of any existing
process.
The intended usage model is that if userspace is known to be fully
SVE-tolerant (or a developer is curious to find out) then init
scripts can crank this up during startup.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Changes requested by Alex Benn?e:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
---
arch/arm64/kernel/fpsimd.c | 62 +++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 61 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 42e8331..b430ee0 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -34,6 +34,7 @@
#include <linux/sched/signal.h>
#include <linux/signal.h>
#include <linux/slab.h>
+#include <linux/sysctl.h>
#include <asm/fpsimd.h>
#include <asm/cputype.h>
@@ -244,6 +245,65 @@ static unsigned int find_supported_vector_length(unsigned int vl)
return sve_vl_from_vq(bit_to_vq(bit));
}
+#ifdef CONFIG_SYSCTL
+
+static int sve_proc_do_default_vl(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp,
+ loff_t *ppos)
+{
+ int ret;
+ int vl = sve_default_vl;
+ struct ctl_table tmp_table = {
+ .data = &vl,
+ .maxlen = sizeof(vl),
+ };
+
+ ret = proc_dointvec(&tmp_table, write, buffer, lenp, ppos);
+ if (ret || !write)
+ return ret;
+
+ /* Writing -1 has the special meaning "set to max": */
+ if (vl == -1) {
+ /* Fail safe if sve_max_vl wasn't initialised */
+ if (WARN_ON(!sve_vl_valid(sve_max_vl)))
+ vl = SVE_VL_MIN;
+ else
+ vl = sve_max_vl;
+
+ goto chosen;
+ }
+
+ if (!sve_vl_valid(vl))
+ return -EINVAL;
+
+ vl = find_supported_vector_length(vl);
+chosen:
+ sve_default_vl = vl;
+ return 0;
+}
+
+static struct ctl_table sve_default_vl_table[] = {
+ {
+ .procname = "sve_default_vector_length",
+ .mode = 0644,
+ .proc_handler = sve_proc_do_default_vl,
+ },
+ { }
+};
+
+static int __init sve_sysctl_init(void)
+{
+ if (system_supports_sve())
+ if (!register_sysctl("abi", sve_default_vl_table))
+ return -EINVAL;
+
+ return 0;
+}
+
+#else /* ! CONFIG_SYSCTL */
+static int __init sve_sysctl_init(void) { return 0; }
+#endif /* ! CONFIG_SYSCTL */
+
#define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
(SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
@@ -1030,6 +1090,6 @@ static int __init fpsimd_init(void)
if (!(elf_hwcap & HWCAP_ASIMD))
pr_notice("Advanced SIMD is not implemented\n");
- return 0;
+ return sve_sysctl_init();
}
late_initcall(fpsimd_init);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 22/28] arm64/sve: KVM: Prevent guests from using SVE
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Until KVM has full SVE support, guests must not be allowed to
execute SVE instructions.
This patch enables the necessary traps, and also ensures that the
traps are disabled again on exit from the guest so that the host
can still use SVE if it wants to.
This patch introduces another instance of
__this_cpu_write(fpsimd_last_state, NULL), so this flush operation
is abstracted out as a separate helper fpsimd_flush_cpu_state().
Other instances are ported appropriately.
As a side effect of this refactoring, a this_cpu_write() in
fpsimd_cpu_pm_notifier() is changed to __this_cpu_write(). This
should be fine, since cpu_pm_enter() is supposed to be called only
with interrupts disabled.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Changes since v1
----------------
Requested by Marc Zyngier:
* Avoid the verbose arithmetic for CPTR_EL2_DEFAULT, and just
describe it in terms of the set of bits known to be RES1 in
CPTR_EL2.
Other:
* Fixup to drop task SVE state cached in the CPU registers across
guest entry/exit.
Without this, we may enter an EL0 process with wrong data in the
extended SVE bits and/or wrong trap configuration.
This is not a problem for the FPSIMD part of the state because KVM
explicitly restores the host FPSIMD state on guest exit; but this
restore is sufficient to corrupt the extra SVE bits even if nothing
else does.
* The fpsimd_flush_cpu_state() function, which was supposed to abstract
the underlying flush operation, wasn't used. [sparse]
This patch is now ported to use it. Other users of the same idiom are
ported too (which was the original intention).
fpsimd_flush_cpu_state() is marked inline, since all users are
ifdef'd and the function may be unused. Plus, it's trivially
suitable for inlining.
---
arch/arm/include/asm/kvm_host.h | 3 +++
arch/arm64/include/asm/fpsimd.h | 1 +
arch/arm64/include/asm/kvm_arm.h | 4 +++-
arch/arm64/include/asm/kvm_host.h | 11 +++++++++++
arch/arm64/kernel/fpsimd.c | 31 +++++++++++++++++++++++++++++--
arch/arm64/kvm/hyp/switch.c | 6 +++---
virt/kvm/arm/arm.c | 3 +++
7 files changed, 53 insertions(+), 6 deletions(-)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 127e2dd..fa4a442 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -299,4 +299,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
struct kvm_device_attr *attr);
+/* All host FP/SIMD state is restored on guest exit, so nothing to save: */
+static inline void kvm_fpsimd_flush_cpu_state(void) {}
+
#endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index d084968..5605fc1 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -74,6 +74,7 @@ extern void fpsimd_restore_current_state(void);
extern void fpsimd_update_current_state(struct fpsimd_state *state);
extern void fpsimd_flush_task_state(struct task_struct *target);
+extern void sve_flush_cpu_state(void);
/* Maximum VL that SVE VL-agnostic software can transparently support */
#define SVE_VL_ARCH_MAX 0x100
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index dbf0537..7f069ff 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -186,7 +186,8 @@
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
#define CPTR_EL2_TZ (1 << 8)
-#define CPTR_EL2_DEFAULT 0x000033ff
+#define CPTR_EL2_RES1 0x000032ff /* known RES1 bits in CPTR_EL2 */
+#define CPTR_EL2_DEFAULT CPTR_EL2_RES1
/* Hyp Debug Configuration Register bits */
#define MDCR_EL2_TPMS (1 << 14)
@@ -237,5 +238,6 @@
#define CPACR_EL1_FPEN (3 << 20)
#define CPACR_EL1_TTA (1 << 28)
+#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
#endif /* __ARM64_KVM_ARM_H__ */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d686300..05d8373 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -25,6 +25,7 @@
#include <linux/types.h>
#include <linux/kvm_types.h>
#include <asm/cpufeature.h>
+#include <asm/fpsimd.h>
#include <asm/kvm.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_mmio.h>
@@ -390,4 +391,14 @@ static inline void __cpu_init_stage2(void)
"PARange is %d bits, unsupported configuration!", parange);
}
+/*
+ * All host FP/SIMD state is restored on guest exit, so nothing needs
+ * doing here except in the SVE case:
+*/
+static inline void kvm_fpsimd_flush_cpu_state(void)
+{
+ if (system_supports_sve())
+ sve_flush_cpu_state();
+}
+
#endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index b430ee0..7837ced 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -875,6 +875,33 @@ void fpsimd_flush_task_state(struct task_struct *t)
t->thread.fpsimd_state.cpu = NR_CPUS;
}
+static inline void fpsimd_flush_cpu_state(void)
+{
+ __this_cpu_write(fpsimd_last_state, NULL);
+}
+
+/*
+ * Invalidate any task SVE state currently held in this CPU's regs.
+ *
+ * This is used to prevent the kernel from trying to reuse SVE register data
+ * that is detroyed by KVM guest enter/exit. This function should go away when
+ * KVM SVE support is implemented. Don't use it for anything else.
+ */
+#ifdef CONFIG_ARM64_SVE
+void sve_flush_cpu_state(void)
+{
+ struct fpsimd_state *const fpstate = __this_cpu_read(fpsimd_last_state);
+ struct task_struct *tsk;
+
+ if (!fpstate)
+ return;
+
+ tsk = container_of(fpstate, struct task_struct, thread.fpsimd_state);
+ if (test_tsk_thread_flag(tsk, TIF_SVE))
+ fpsimd_flush_cpu_state();
+}
+#endif /* CONFIG_ARM64_SVE */
+
#ifdef CONFIG_KERNEL_MODE_NEON
DEFINE_PER_CPU(bool, kernel_neon_busy);
@@ -915,7 +942,7 @@ void kernel_neon_begin(void)
}
/* Invalidate any task state remaining in the fpsimd regs: */
- __this_cpu_write(fpsimd_last_state, NULL);
+ fpsimd_flush_cpu_state();
preempt_disable();
@@ -1032,7 +1059,7 @@ static int fpsimd_cpu_pm_notifier(struct notifier_block *self,
case CPU_PM_ENTER:
if (current->mm)
task_fpsimd_save();
- this_cpu_write(fpsimd_last_state, NULL);
+ fpsimd_flush_cpu_state();
break;
case CPU_PM_EXIT:
if (current->mm)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 35a90b8..951f3eb 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -48,7 +48,7 @@ static void __hyp_text __activate_traps_vhe(void)
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
- val &= ~CPACR_EL1_FPEN;
+ val &= ~(CPACR_EL1_FPEN | CPACR_EL1_ZEN);
write_sysreg(val, cpacr_el1);
write_sysreg(__kvm_hyp_vector, vbar_el1);
@@ -59,7 +59,7 @@ static void __hyp_text __activate_traps_nvhe(void)
u64 val;
val = CPTR_EL2_DEFAULT;
- val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
+ val |= CPTR_EL2_TTA | CPTR_EL2_TFP | CPTR_EL2_TZ;
write_sysreg(val, cptr_el2);
}
@@ -117,7 +117,7 @@ static void __hyp_text __deactivate_traps_vhe(void)
write_sysreg(mdcr_el2, mdcr_el2);
write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
- write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
+ write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1);
write_sysreg(vectors, vbar_el1);
}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index a39a1e1..af9f5da 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -647,6 +647,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
*/
preempt_disable();
+ /* Flush FP/SIMD state that can't survive guest entry/exit */
+ kvm_fpsimd_flush_cpu_state();
+
kvm_pmu_flush_hwstate(vcpu);
kvm_timer_flush_hwstate(vcpu);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 22/28] arm64/sve: KVM: Prevent guests from using SVE
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Until KVM has full SVE support, guests must not be allowed to
execute SVE instructions.
This patch enables the necessary traps, and also ensures that the
traps are disabled again on exit from the guest so that the host
can still use SVE if it wants to.
This patch introduces another instance of
__this_cpu_write(fpsimd_last_state, NULL), so this flush operation
is abstracted out as a separate helper fpsimd_flush_cpu_state().
Other instances are ported appropriately.
As a side effect of this refactoring, a this_cpu_write() in
fpsimd_cpu_pm_notifier() is changed to __this_cpu_write(). This
should be fine, since cpu_pm_enter() is supposed to be called only
with interrupts disabled.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
---
Changes since v1
----------------
Requested by Marc Zyngier:
* Avoid the verbose arithmetic for CPTR_EL2_DEFAULT, and just
describe it in terms of the set of bits known to be RES1 in
CPTR_EL2.
Other:
* Fixup to drop task SVE state cached in the CPU registers across
guest entry/exit.
Without this, we may enter an EL0 process with wrong data in the
extended SVE bits and/or wrong trap configuration.
This is not a problem for the FPSIMD part of the state because KVM
explicitly restores the host FPSIMD state on guest exit; but this
restore is sufficient to corrupt the extra SVE bits even if nothing
else does.
* The fpsimd_flush_cpu_state() function, which was supposed to abstract
the underlying flush operation, wasn't used. [sparse]
This patch is now ported to use it. Other users of the same idiom are
ported too (which was the original intention).
fpsimd_flush_cpu_state() is marked inline, since all users are
ifdef'd and the function may be unused. Plus, it's trivially
suitable for inlining.
---
arch/arm/include/asm/kvm_host.h | 3 +++
arch/arm64/include/asm/fpsimd.h | 1 +
arch/arm64/include/asm/kvm_arm.h | 4 +++-
arch/arm64/include/asm/kvm_host.h | 11 +++++++++++
arch/arm64/kernel/fpsimd.c | 31 +++++++++++++++++++++++++++++--
arch/arm64/kvm/hyp/switch.c | 6 +++---
virt/kvm/arm/arm.c | 3 +++
7 files changed, 53 insertions(+), 6 deletions(-)
diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
index 127e2dd..fa4a442 100644
--- a/arch/arm/include/asm/kvm_host.h
+++ b/arch/arm/include/asm/kvm_host.h
@@ -299,4 +299,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
struct kvm_device_attr *attr);
+/* All host FP/SIMD state is restored on guest exit, so nothing to save: */
+static inline void kvm_fpsimd_flush_cpu_state(void) {}
+
#endif /* __ARM_KVM_HOST_H__ */
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index d084968..5605fc1 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -74,6 +74,7 @@ extern void fpsimd_restore_current_state(void);
extern void fpsimd_update_current_state(struct fpsimd_state *state);
extern void fpsimd_flush_task_state(struct task_struct *target);
+extern void sve_flush_cpu_state(void);
/* Maximum VL that SVE VL-agnostic software can transparently support */
#define SVE_VL_ARCH_MAX 0x100
diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
index dbf0537..7f069ff 100644
--- a/arch/arm64/include/asm/kvm_arm.h
+++ b/arch/arm64/include/asm/kvm_arm.h
@@ -186,7 +186,8 @@
#define CPTR_EL2_TTA (1 << 20)
#define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
#define CPTR_EL2_TZ (1 << 8)
-#define CPTR_EL2_DEFAULT 0x000033ff
+#define CPTR_EL2_RES1 0x000032ff /* known RES1 bits in CPTR_EL2 */
+#define CPTR_EL2_DEFAULT CPTR_EL2_RES1
/* Hyp Debug Configuration Register bits */
#define MDCR_EL2_TPMS (1 << 14)
@@ -237,5 +238,6 @@
#define CPACR_EL1_FPEN (3 << 20)
#define CPACR_EL1_TTA (1 << 28)
+#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
#endif /* __ARM64_KVM_ARM_H__ */
diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
index d686300..05d8373 100644
--- a/arch/arm64/include/asm/kvm_host.h
+++ b/arch/arm64/include/asm/kvm_host.h
@@ -25,6 +25,7 @@
#include <linux/types.h>
#include <linux/kvm_types.h>
#include <asm/cpufeature.h>
+#include <asm/fpsimd.h>
#include <asm/kvm.h>
#include <asm/kvm_asm.h>
#include <asm/kvm_mmio.h>
@@ -390,4 +391,14 @@ static inline void __cpu_init_stage2(void)
"PARange is %d bits, unsupported configuration!", parange);
}
+/*
+ * All host FP/SIMD state is restored on guest exit, so nothing needs
+ * doing here except in the SVE case:
+*/
+static inline void kvm_fpsimd_flush_cpu_state(void)
+{
+ if (system_supports_sve())
+ sve_flush_cpu_state();
+}
+
#endif /* __ARM64_KVM_HOST_H__ */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index b430ee0..7837ced 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -875,6 +875,33 @@ void fpsimd_flush_task_state(struct task_struct *t)
t->thread.fpsimd_state.cpu = NR_CPUS;
}
+static inline void fpsimd_flush_cpu_state(void)
+{
+ __this_cpu_write(fpsimd_last_state, NULL);
+}
+
+/*
+ * Invalidate any task SVE state currently held in this CPU's regs.
+ *
+ * This is used to prevent the kernel from trying to reuse SVE register data
+ * that is detroyed by KVM guest enter/exit. This function should go away when
+ * KVM SVE support is implemented. Don't use it for anything else.
+ */
+#ifdef CONFIG_ARM64_SVE
+void sve_flush_cpu_state(void)
+{
+ struct fpsimd_state *const fpstate = __this_cpu_read(fpsimd_last_state);
+ struct task_struct *tsk;
+
+ if (!fpstate)
+ return;
+
+ tsk = container_of(fpstate, struct task_struct, thread.fpsimd_state);
+ if (test_tsk_thread_flag(tsk, TIF_SVE))
+ fpsimd_flush_cpu_state();
+}
+#endif /* CONFIG_ARM64_SVE */
+
#ifdef CONFIG_KERNEL_MODE_NEON
DEFINE_PER_CPU(bool, kernel_neon_busy);
@@ -915,7 +942,7 @@ void kernel_neon_begin(void)
}
/* Invalidate any task state remaining in the fpsimd regs: */
- __this_cpu_write(fpsimd_last_state, NULL);
+ fpsimd_flush_cpu_state();
preempt_disable();
@@ -1032,7 +1059,7 @@ static int fpsimd_cpu_pm_notifier(struct notifier_block *self,
case CPU_PM_ENTER:
if (current->mm)
task_fpsimd_save();
- this_cpu_write(fpsimd_last_state, NULL);
+ fpsimd_flush_cpu_state();
break;
case CPU_PM_EXIT:
if (current->mm)
diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
index 35a90b8..951f3eb 100644
--- a/arch/arm64/kvm/hyp/switch.c
+++ b/arch/arm64/kvm/hyp/switch.c
@@ -48,7 +48,7 @@ static void __hyp_text __activate_traps_vhe(void)
val = read_sysreg(cpacr_el1);
val |= CPACR_EL1_TTA;
- val &= ~CPACR_EL1_FPEN;
+ val &= ~(CPACR_EL1_FPEN | CPACR_EL1_ZEN);
write_sysreg(val, cpacr_el1);
write_sysreg(__kvm_hyp_vector, vbar_el1);
@@ -59,7 +59,7 @@ static void __hyp_text __activate_traps_nvhe(void)
u64 val;
val = CPTR_EL2_DEFAULT;
- val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
+ val |= CPTR_EL2_TTA | CPTR_EL2_TFP | CPTR_EL2_TZ;
write_sysreg(val, cptr_el2);
}
@@ -117,7 +117,7 @@ static void __hyp_text __deactivate_traps_vhe(void)
write_sysreg(mdcr_el2, mdcr_el2);
write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
- write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
+ write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1);
write_sysreg(vectors, vbar_el1);
}
diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
index a39a1e1..af9f5da 100644
--- a/virt/kvm/arm/arm.c
+++ b/virt/kvm/arm/arm.c
@@ -647,6 +647,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
*/
preempt_disable();
+ /* Flush FP/SIMD state that can't survive guest entry/exit */
+ kvm_fpsimd_flush_cpu_state();
+
kvm_pmu_flush_hwstate(vcpu);
kvm_timer_flush_hwstate(vcpu);
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
When trapping forbidden attempts by a guest to use SVE, we want the
guest to see a trap consistent with SVE not being implemented.
This patch injects an undefined instruction exception into the
guest in response to such an exception.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/kvm/handle_exit.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 17d8a16..e3e42d0 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
return 1;
}
+static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
+{
+ /* Until SVE is supported for guests: */
+ kvm_inject_undefined(vcpu);
+ return 1;
+}
+
static exit_handle_fn arm_exit_handlers[] = {
[0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
[ESR_ELx_EC_WFx] = kvm_handle_wfx,
@@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_HVC64] = handle_hvc,
[ESR_ELx_EC_SMC64] = handle_smc,
[ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
+ [ESR_ELx_EC_SVE] = handle_sve,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
When trapping forbidden attempts by a guest to use SVE, we want the
guest to see a trap consistent with SVE not being implemented.
This patch injects an undefined instruction exception into the
guest in response to such an exception.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
arch/arm64/kvm/handle_exit.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
index 17d8a16..e3e42d0 100644
--- a/arch/arm64/kvm/handle_exit.c
+++ b/arch/arm64/kvm/handle_exit.c
@@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
return 1;
}
+static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
+{
+ /* Until SVE is supported for guests: */
+ kvm_inject_undefined(vcpu);
+ return 1;
+}
+
static exit_handle_fn arm_exit_handlers[] = {
[0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
[ESR_ELx_EC_WFx] = kvm_handle_wfx,
@@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
[ESR_ELx_EC_HVC64] = handle_hvc,
[ESR_ELx_EC_SMC64] = handle_smc,
[ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
+ [ESR_ELx_EC_SVE] = handle_sve,
[ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
[ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 24/28] arm64/sve: KVM: Hide SVE from CPU features exposed to guests
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
KVM guests cannot currently use SVE, because SVE is always
configured to trap to EL2.
However, a guest that sees SVE reported as present in
ID_AA64PFR0_EL1 may legitimately expect that SVE works and try to
use it. Instead of working, the guest will receive an injected
undef exception, which may cause the guest to oops or go into a
spin.
To avoid misleading the guest into believing that SVE will work,
this patch masks out the SVE field from ID_AA64PFR0_EL1 when a
guest attempts to read this register. No support is explicitly
added for ID_AA64ZFR0_EL1 either, so that is still emulated as
reading as zero, which is consistent with SVE not being
implemented.
This is a temporary measure, and will be removed in a later series
when full KVM support for SVE is implemented.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
Changes since v1
----------------
Requested by Marc Zyngier:
* Use pr_err() instead inventing "kvm_info_once" ad-hoc.
---
arch/arm64/kvm/sys_regs.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index b1f7552..a0ee9b0 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -23,6 +23,7 @@
#include <linux/bsearch.h>
#include <linux/kvm_host.h>
#include <linux/mm.h>
+#include <linux/printk.h>
#include <linux/uaccess.h>
#include <asm/cacheflush.h>
@@ -897,8 +898,17 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
{
u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
(u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
+ u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
- return raz ? 0 : read_sanitised_ftr_reg(id);
+ if (id == SYS_ID_AA64PFR0_EL1) {
+ if (val & (0xfUL << ID_AA64PFR0_SVE_SHIFT))
+ pr_err_once("kvm [%i]: SVE unsupported for guests, suppressing\n",
+ task_pid_nr(current));
+
+ val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
+ }
+
+ return val;
}
/* cpufeature ID register access trap handlers */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 24/28] arm64/sve: KVM: Hide SVE from CPU features exposed to guests
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
KVM guests cannot currently use SVE, because SVE is always
configured to trap to EL2.
However, a guest that sees SVE reported as present in
ID_AA64PFR0_EL1 may legitimately expect that SVE works and try to
use it. Instead of working, the guest will receive an injected
undef exception, which may cause the guest to oops or go into a
spin.
To avoid misleading the guest into believing that SVE will work,
this patch masks out the SVE field from ID_AA64PFR0_EL1 when a
guest attempts to read this register. No support is explicitly
added for ID_AA64ZFR0_EL1 either, so that is still emulated as
reading as zero, which is consistent with SVE not being
implemented.
This is a temporary measure, and will be removed in a later series
when full KVM support for SVE is implemented.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
---
Changes since v1
----------------
Requested by Marc Zyngier:
* Use pr_err() instead inventing "kvm_info_once" ad-hoc.
---
arch/arm64/kvm/sys_regs.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
index b1f7552..a0ee9b0 100644
--- a/arch/arm64/kvm/sys_regs.c
+++ b/arch/arm64/kvm/sys_regs.c
@@ -23,6 +23,7 @@
#include <linux/bsearch.h>
#include <linux/kvm_host.h>
#include <linux/mm.h>
+#include <linux/printk.h>
#include <linux/uaccess.h>
#include <asm/cacheflush.h>
@@ -897,8 +898,17 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
{
u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
(u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
+ u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
- return raz ? 0 : read_sanitised_ftr_reg(id);
+ if (id == SYS_ID_AA64PFR0_EL1) {
+ if (val & (0xfUL << ID_AA64PFR0_SVE_SHIFT))
+ pr_err_once("kvm [%i]: SVE unsupported for guests, suppressing\n",
+ task_pid_nr(current));
+
+ val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
+ }
+
+ return val;
}
/* cpufeature ID register access trap handlers */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 25/28] arm64/sve: Detect SVE and activate runtime support
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
This patch enables detection of hardware SVE support via the
cpufeatures framework, and reports its presence to the kernel and
userspace via the new ARM64_SVE cpucap and HWCAP_SVE hwcap
respectively.
Userspace can also detect SVE using ID_AA64PFR0_EL1, using the
cpufeatures MRS emulation.
When running on hardware that supports SVE, this enables runtime
kernel support for SVE, and allows user tasks to execute SVE
instructions and make of the of the SVE-specific user/kernel
interface extensions implemented by this series.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since v1
----------------
Requested by Suzuki Poulose:
* Update CPUID documentation to document exposure of SVE field.
* Applied deferred Reviewed-by dependent on the above change.
---
Documentation/arm64/cpu-feature-registers.txt | 6 +++++-
arch/arm64/include/asm/cpucaps.h | 3 ++-
arch/arm64/include/asm/cpufeature.h | 3 ++-
arch/arm64/include/uapi/asm/hwcap.h | 1 +
arch/arm64/kernel/cpufeature.c | 16 ++++++++++++++++
arch/arm64/kernel/cpuinfo.c | 1 +
6 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/Documentation/arm64/cpu-feature-registers.txt b/Documentation/arm64/cpu-feature-registers.txt
index dad411d..d65504c 100644
--- a/Documentation/arm64/cpu-feature-registers.txt
+++ b/Documentation/arm64/cpu-feature-registers.txt
@@ -132,7 +132,11 @@ infrastructure:
x--------------------------------------------------x
| Name | bits | visible |
|--------------------------------------------------|
- | RES0 | [63-28] | n |
+ | RES0 | [63-36] | n |
+ |--------------------------------------------------|
+ | SVE | [35-32] | y |
+ |--------------------------------------------------|
+ | RES0 | [31-28] | n |
|--------------------------------------------------|
| GIC | [27-24] | n |
|--------------------------------------------------|
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 8da6216..2ff7c5e 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -40,7 +40,8 @@
#define ARM64_WORKAROUND_858921 19
#define ARM64_WORKAROUND_CAVIUM_30115 20
#define ARM64_HAS_DCPOP 21
+#define ARM64_SVE 22
-#define ARM64_NCAPS 22
+#define ARM64_NCAPS 23
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index d98e7ba..bc987eb 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -273,7 +273,8 @@ static inline bool system_uses_ttbr0_pan(void)
static inline bool system_supports_sve(void)
{
- return false;
+ return IS_ENABLED(CONFIG_ARM64_SVE) &&
+ cpus_have_const_cap(ARM64_SVE);
}
/*
diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h
index 4b9344c..c6e1e45 100644
--- a/arch/arm64/include/uapi/asm/hwcap.h
+++ b/arch/arm64/include/uapi/asm/hwcap.h
@@ -36,5 +36,6 @@
#define HWCAP_FCMA (1 << 14)
#define HWCAP_LRCPC (1 << 15)
#define HWCAP_DCPOP (1 << 16)
+#define HWCAP_SVE (1 << 17)
#endif /* _UAPI__ASM_HWCAP_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c30bb6b..3e74d93 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -141,6 +141,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
};
static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64PFR0_GIC_SHIFT, 4, 0),
S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI),
@@ -942,6 +943,18 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.min_field_value = 1,
},
#endif
+#ifdef CONFIG_ARM64_SVE
+ {
+ .desc = "Scalable Vector Extension",
+ .capability = ARM64_SVE,
+ .def_scope = SCOPE_SYSTEM,
+ .sys_reg = SYS_ID_AA64PFR0_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64PFR0_SVE_SHIFT,
+ .min_field_value = ID_AA64PFR0_SVE,
+ .matches = has_cpuid_feature,
+ },
+#endif /* CONFIG_ARM64_SVE */
{},
};
@@ -974,6 +987,9 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = {
HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_JSCVT_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, HWCAP_JSCVT),
HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_FCMA_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, HWCAP_FCMA),
HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_LRCPC_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, HWCAP_LRCPC),
+#ifdef CONFIG_ARM64_SVE
+ HWCAP_CAP(SYS_ID_AA64PFR0_EL1, ID_AA64PFR0_SVE_SHIFT, FTR_UNSIGNED, ID_AA64PFR0_SVE, CAP_HWCAP, HWCAP_SVE),
+#endif
{},
};
diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
index be260e8..9559dde 100644
--- a/arch/arm64/kernel/cpuinfo.c
+++ b/arch/arm64/kernel/cpuinfo.c
@@ -70,6 +70,7 @@ static const char *const hwcap_str[] = {
"fcma",
"lrcpc",
"dcpop",
+ "sve",
NULL
};
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 25/28] arm64/sve: Detect SVE and activate runtime support
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch enables detection of hardware SVE support via the
cpufeatures framework, and reports its presence to the kernel and
userspace via the new ARM64_SVE cpucap and HWCAP_SVE hwcap
respectively.
Userspace can also detect SVE using ID_AA64PFR0_EL1, using the
cpufeatures MRS emulation.
When running on hardware that supports SVE, this enables runtime
kernel support for SVE, and allows user tasks to execute SVE
instructions and make of the of the SVE-specific user/kernel
interface extensions implemented by this series.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Changes since v1
----------------
Requested by Suzuki Poulose:
* Update CPUID documentation to document exposure of SVE field.
* Applied deferred Reviewed-by dependent on the above change.
---
Documentation/arm64/cpu-feature-registers.txt | 6 +++++-
arch/arm64/include/asm/cpucaps.h | 3 ++-
arch/arm64/include/asm/cpufeature.h | 3 ++-
arch/arm64/include/uapi/asm/hwcap.h | 1 +
arch/arm64/kernel/cpufeature.c | 16 ++++++++++++++++
arch/arm64/kernel/cpuinfo.c | 1 +
6 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/Documentation/arm64/cpu-feature-registers.txt b/Documentation/arm64/cpu-feature-registers.txt
index dad411d..d65504c 100644
--- a/Documentation/arm64/cpu-feature-registers.txt
+++ b/Documentation/arm64/cpu-feature-registers.txt
@@ -132,7 +132,11 @@ infrastructure:
x--------------------------------------------------x
| Name | bits | visible |
|--------------------------------------------------|
- | RES0 | [63-28] | n |
+ | RES0 | [63-36] | n |
+ |--------------------------------------------------|
+ | SVE | [35-32] | y |
+ |--------------------------------------------------|
+ | RES0 | [31-28] | n |
|--------------------------------------------------|
| GIC | [27-24] | n |
|--------------------------------------------------|
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index 8da6216..2ff7c5e 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -40,7 +40,8 @@
#define ARM64_WORKAROUND_858921 19
#define ARM64_WORKAROUND_CAVIUM_30115 20
#define ARM64_HAS_DCPOP 21
+#define ARM64_SVE 22
-#define ARM64_NCAPS 22
+#define ARM64_NCAPS 23
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index d98e7ba..bc987eb 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -273,7 +273,8 @@ static inline bool system_uses_ttbr0_pan(void)
static inline bool system_supports_sve(void)
{
- return false;
+ return IS_ENABLED(CONFIG_ARM64_SVE) &&
+ cpus_have_const_cap(ARM64_SVE);
}
/*
diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h
index 4b9344c..c6e1e45 100644
--- a/arch/arm64/include/uapi/asm/hwcap.h
+++ b/arch/arm64/include/uapi/asm/hwcap.h
@@ -36,5 +36,6 @@
#define HWCAP_FCMA (1 << 14)
#define HWCAP_LRCPC (1 << 15)
#define HWCAP_DCPOP (1 << 16)
+#define HWCAP_SVE (1 << 17)
#endif /* _UAPI__ASM_HWCAP_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index c30bb6b..3e74d93 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -141,6 +141,7 @@ static const struct arm64_ftr_bits ftr_id_aa64isar1[] = {
};
static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = {
+ ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_SVE_SHIFT, 4, 0),
ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_EXACT, ID_AA64PFR0_GIC_SHIFT, 4, 0),
S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI),
S_ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI),
@@ -942,6 +943,18 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.min_field_value = 1,
},
#endif
+#ifdef CONFIG_ARM64_SVE
+ {
+ .desc = "Scalable Vector Extension",
+ .capability = ARM64_SVE,
+ .def_scope = SCOPE_SYSTEM,
+ .sys_reg = SYS_ID_AA64PFR0_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64PFR0_SVE_SHIFT,
+ .min_field_value = ID_AA64PFR0_SVE,
+ .matches = has_cpuid_feature,
+ },
+#endif /* CONFIG_ARM64_SVE */
{},
};
@@ -974,6 +987,9 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = {
HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_JSCVT_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, HWCAP_JSCVT),
HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_FCMA_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, HWCAP_FCMA),
HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_LRCPC_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, HWCAP_LRCPC),
+#ifdef CONFIG_ARM64_SVE
+ HWCAP_CAP(SYS_ID_AA64PFR0_EL1, ID_AA64PFR0_SVE_SHIFT, FTR_UNSIGNED, ID_AA64PFR0_SVE, CAP_HWCAP, HWCAP_SVE),
+#endif
{},
};
diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
index be260e8..9559dde 100644
--- a/arch/arm64/kernel/cpuinfo.c
+++ b/arch/arm64/kernel/cpuinfo.c
@@ -70,6 +70,7 @@ static const char *const hwcap_str[] = {
"fcma",
"lrcpc",
"dcpop",
+ "sve",
NULL
};
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Mark Rutland
This patch adds basic documentation of the user/kernel interface
provided by the for SVE.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
---
Changes since v1
----------------
Requested by Alex Bennée:
* Add cross-reference from sigcontext.h to the description of VL/VQ.
Other:
* [ABI fix] Bail out with -EIO if attempting to set the SVE regs for an
unsupported VL, rather than misparsing the regset data.
* Document detection of SVE via CPUID MRS.
* Remove comment about coredump notes being padded for NT_ARM_SVE.
This is no longer the case with variable-size regsets supported
in the core code.
* Fix a missing bullet char
---
Documentation/arm64/sve.txt | 477 +++++++++++++++++++++++++++++++
arch/arm64/include/uapi/asm/sigcontext.h | 3 +
2 files changed, 480 insertions(+)
create mode 100644 Documentation/arm64/sve.txt
diff --git a/Documentation/arm64/sve.txt b/Documentation/arm64/sve.txt
new file mode 100644
index 0000000..3c0f97f
--- /dev/null
+++ b/Documentation/arm64/sve.txt
@@ -0,0 +1,477 @@
+ Scalable Vector Extension support for AArch64 Linux
+ ===================================================
+
+Author: Dave Martin <Dave.Martin@arm.com>
+Date: 4 August 2017
+
+This document outlines briefly the interface provided to userspace by Linux in
+order to support use of the ARM Scalable Vector Extension (SVE).
+
+This is an outline of the most important features and issues only and not
+intended to be exhaustive.
+
+This document does not aim to describe the SVE architecture or programmer's
+model. To aid understanding, a minimal description of relevant programmer's
+model features for SVE is included in Appendix A.
+
+
+1. General
+-----------
+
+* SVE registers Z0..Z31, P0..P15 and FFR and the current vector length VL, are
+ tracked per-thread.
+
+* The presence of SVE is reported to userspace via HWCAP_SVE in the aux vector
+ AT_HWCAP entry. Presence of this flag implies the presence of the SVE
+ instructions and registers, and the Linux-specific system interfaces
+ described in this document. SVE is reported in /proc/cpuinfo as "sve".
+
+* Support for the execution of SVE instructions in userspace can also be
+ detected by reading the CPU ID register ID_AA64PFR0_EL1 using an MRS
+ instruction, and checking that the value of the SVE field is nonzero. [3]
+
+ It does not guarantee the presence of the system interfaces described in the
+ following sections: software that needs to verify that those interfaces are
+ present must check for HWCAP_SVE instead.
+
+* Debuggers should restrict themselves to interacting with the target via the
+ NT_ARM_SVE regset. The recommended way of detecting support for this regset
+ is to connect to a target process first and then attempt a
+ ptrace(PTRACE_GETREGSET, pid, NT_ARM_SVE, &iov).
+
+
+2. Vector length terminology
+-----------------------------
+
+The size of an SVE vector (Z) register is referred to as the "vector length".
+
+To avoid confusion about the units used to express vector length, the kernel
+adopts the following conventions:
+
+* Vector length (VL) = size of a Z-register in bytes
+
+* Vector quadwords (VQ) = size of a Z-register in units of 128 bits
+
+(So, VL = 16 * VQ.)
+
+The VQ convention is used where the underlying granularity is important, such
+as in data structure definitions. In most other situations, the VL convention
+is used. This is consistent with the meaning of the "VL" pseudo-register in
+the SVE instruction set architecture.
+
+
+3. System call behaviour
+-------------------------
+
+* On syscall, V0..V31 are preserved (as without SVE). Thus, bits [127:0] of
+ Z0..Z31 are preserved. All other bits of Z0..Z31, and all of P0..P15 and FFR
+ become unspecified on return from a syscall.
+
+* The SVE registers are not used to pass arguments to or receive results from
+ any syscall.
+
+* In practice the affected registers/bits will be preserved or will be replaced
+ with zeros on return from a syscall, but userspace should not make
+ assumptions about this. The kernel behaviour may vary on a case-by-case
+ basis.
+
+
+4. Signal handling
+-------------------
+
+* A new signal frame record sve_context encodes the SVE registers on signal
+ delivery. [1]
+
+* This record is supplementary to fpsimd_context. The FPSR and FPCR registers
+ are only present in fpsimd_context. For convenience, the content of V0..V31
+ is duplicated between sve_context and fpsimd_context.
+
+* The signal frame record for SVE always contains basic metadata, in particular
+ the thread's vector length (in sve_context.vl).
+
+* The SVE registers may or may not be included in the record, depending on
+ whether the registers are live for the thread. The registers are present if
+ and only if:
+ sve_context.head.size >= SVE_SIG_CONTEXT_SIZE(sve_vq_from_vl(sve_context.vl)).
+
+* If the registers are present, the remainder of the record has a vl-dependent
+ size and layout. Macros SIG_SVE_* are defined [1] to facilitate access to
+ the members.
+
+* If the SVE context is too big to fit in sigcontext.__reserved[], then extra
+ space is allocated on the stack, an extra_context record is written in
+ __reserved[] referencing this space. sve_context is then written in the
+ extra space. Refer to [1] for further details about this mechanism.
+
+
+5. Signal return
+-----------------
+
+When returning from a signal handler:
+
+* If there is no sve_context record in the signal frame, or if the record is
+ present but contains no register data as desribed in the previous section,
+ then the SVE registers/bits become non-live and take unspecified values.
+
+* If sve_context is present in the signal frame and contains full register
+ data, the SVE registers become live and are populated with the specified
+ data. However, for backward compatibility reasons, bits [127:0] of Z0..Z31
+ are always restored from the corresponding members of fpsimd_context.vregs[]
+ and not from sve_context. The remaining bits are restored from sve_context.
+
+* Inclusion of fpsimd_context in the signal frame remains mandatory,
+ irrespective of whether sve_context is present or not.
+
+* The vector length cannot be changed via signal return. If sve_context.vl in
+ the signal frame does not match the current vector length, the signal return
+ attempt is treated as illegal, resulting in a forced SIGSEGV.
+
+
+6. prctl extensions
+--------------------
+
+Some new prctl() calls are added to allow programs to manage the SVE vector
+length:
+
+prctl(PR_SVE_SET_VL, unsigned long arg)
+
+ Sets the vector length of the calling thread and related flags, where
+ arg == vl | flags.
+
+ vl is the desired vector length, where sve_vl_valid(vl) must be true.
+
+ flags:
+
+ PR_SVE_SET_VL_INHERIT
+
+ Inherit the current vector length across execve(). Otherwise, the
+ vector length is reset to the system default at execve(). (See
+ Section 9.)
+
+ PR_SVE_SET_VL_ONEXEC
+
+ Defer the requested vector length change until the next execve().
+ This allows launching of a new program with a different vector
+ length, while avoiding runtime side effects in the caller.
+
+ This also overrides the effect of PR_SVE_SET_VL_INHERIT for the
+ first execve().
+
+ Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector
+ length change is cancelled.
+
+ Return value: a nonnegative on success, or a negative value on error:
+ EINVAL: SVE not supported, invalid vector length requested, or
+ invalid flags.
+
+ On success, the calling thread's vector length is changed to the largest
+ value supported by the system that is less than or equal to vl.
+ If vl == SVE_VL_MAX, the calling thread's vector length is changed to the
+ largest value supported by the system.
+
+ The returned value describes the resulting configuration, encoded as for
+ PR_SVE_GET_VL.
+
+ Changing the vector length causes all of P0..P15, FFR and all bits of
+ Z0..V31 except for Z0 bits [127:0] .. Z31 bits [127:0] to become
+ unspecified. Calling PR_SVE_SET_VL with vl equal to the thread's current
+ vector length does not constitute a change to the vector length for this
+ purpose.
+
+
+prctl(PR_SVE_GET_VL)
+
+ Gets the vector length of the calling thread.
+
+ The following flag may be OR-ed into the result:
+
+ PR_SVE_SET_VL_INHERIT
+
+ Vector length will be inherited across execve().
+
+ There is no way to determine whether there is an outstanding deferred
+ vector length change (which would only normally be the case between a
+ fork() or vfork() and the corresponding execve() in typical use).
+
+ To extract the vector length from the result, and it with
+ PR_SVE_VL_LEN_MASK.
+
+ Return value: a nonnegative value on success, or a negative value on error:
+ EINVAL: SVE not supported.
+
+
+7. ptrace extensions
+---------------------
+
+* A new regset NT_ARM_SVE is defined for use with PTRACE_GETREGSET and
+ PTRACE_SETREGSET.
+
+ Refer to [2] for definitions.
+
+The regset data starts with struct user_sve_header, containing:
+
+ size
+
+ Size of the complete regset, in bytes.
+ This depends on vl and possibly on other things in the future.
+
+ If a call to PTRACE_GETREGSET requests less data than the value of
+ size, the caller can allocate a larger buffer and retry in order to
+ read the complete regset.
+
+ max_size
+
+ Maximum size in bytes that the regset can grow to for the target
+ thread. The regset won't grow bigger than this even if the target
+ thread changes its vector length etc.
+
+ vl
+
+ Target thread's current vector length, in bytes.
+
+ max_vl
+
+ Maximum possible vector length for the target thread.
+
+ flags
+
+ either
+
+ SVE_PT_REGS_FPSIMD
+
+ SVE registers are not live (GETREGSET) or are to be made
+ non-live (SETREGSET).
+
+ The payload is of type struct user_fpsimd_state, with the same
+ meaning as for NT_PRFPREG, starting at offset
+ SVE_PT_FPSIMD_OFFSET from the start of user_sve_header.
+
+ Extra data might be appended in the future: the size of the
+ payload should be obtained using SVE_PT_FPSIMD_SIZE(vq, flags).
+
+ vq should be obtained using sve_vq_from_vl(vl).
+
+ or
+
+ SVE_PT_REGS_SVE
+
+ SVE registers are live (GETREGSET) or are to be made live
+ (SETREGSET).
+
+ The payload contains the SVE register data, starting at offset
+ SVE_PT_SVE_OFFSET from the start of user_sve_header, and with
+ size SVE_PT_SVE_SIZE(vq, flags);
+
+ ... OR-ed with zero or more of the following flags, which have the same
+ meaning and behaviour as the corresponding PR_SET_VL_* flags:
+
+ SVE_PT_VL_INHERIT
+
+ SVE_PT_VL_ONEXEC (SETREGSET only).
+
+* The effects of changing the vector length and/or flags are equivalent to
+ those documented for PR_SVE_SET_VL.
+
+* In the SVE_PT_REGS_SVE case, the size and layout of the payload depends on
+ the header fields. The SVE_PT_SVE_*() macros are provided to facilitate
+ access to the members.
+
+* In either case, for SETREGSET it is permissible to omit the payload, in which
+ case only the vector length and flags are changed (along with any
+ consequences of those changes).
+
+* For SETREGSET, if an SVE_PT_REGS_SVE payload is present and the
+ requested VL is not supported, the effect will be the same as if the
+ payload were omitted, except that an EIO error is reported. No
+ attempt is made to translate the payload data to the correct layout
+ for the vector length actually set. The thread's FPSIMD state is
+ preserved, but the remaining bits of the SVE registers become
+ unspecified. It is up to the caller to translate the payload layout
+ for the actual VL and retry.
+
+* The effect of writing a partial, incomplete payload is unspecified.
+
+
+8. ELF coredump extensions
+---------------------------
+
+* A NT_ARM_SVE note will be added to each coredump for each thread of the
+ dumped process. The contents will be equivalent to the data that would have
+ been read if a PTRACE_GETREGSET of NT_ARM_SVE were executed for each thread
+ when the coredump was generated.
+
+
+9. System runtime configuration
+--------------------------------
+
+* To mitigate the ABI impact of expansion of the signal frame, a policy
+ mechanism is provided for administrators, distro maintainers and developers
+ to set the default vector length for userspace processes:
+
+/proc/cpu/sve_default_vector_length
+
+ Writing the text representation of an integer to this file sets the system
+ default vector length to the specified value, unless the value is greater
+ than the maximum vector length supported by the system in which case the
+ default vector length is set to that maximum.
+
+ The result can be determined by reopening the file and reading its
+ contents.
+
+ At boot, the default vector length is initially set to 64 or the maximum
+ supported vector length, whichever is smaller. This determines the initial
+ vector length of the init process (PID 1).
+
+ Reading this file returns the current system default vector length.
+
+* At every execve() call, the new vector length of the new process is set to
+ the system default vector length, unless
+
+ * PR_SVE_SET_VL_INHERIT (or equivalently SVE_PT_VL_INHERIT) is set for the
+ calling thread, or
+
+ * a deferred vector length change is pending, established via the
+ PR_SVE_SET_VL_ONEXEC flag (or SVE_PT_VL_ONEXEC).
+
+* Modifying the system default vector length does not affect the vector length
+ of any existing process or thread that does not make an execve() call.
+
+
+Appendix A. SVE programmer's model (informative)
+=================================================
+
+This section provides a minimal description of the additions made by SVE to the
+ARMv8-A programmer's model that are relevant to this document.
+
+Note: This section is for information only and not intended to be complete or
+to replace any architectural specification.
+
+A.1. Registers
+---------------
+
+In A64 state, SVE adds the following:
+
+* 32 8VL-bit vector registers Z0..Z31
+ For each Zn, Zn bits [127:0] alias the ARMv8-A vector register Vn.
+
+ A register write using a Vn register name zeros all bits of the corresponding
+ Zn except for bits [127:0].
+
+* 16 VL-bit predicate registers P0..P15
+
+* 1 VL-bit special-purpose predicate register FFR (the "first-fault register")
+
+* a VL "pseudo-register" that determines the size of each vector register
+
+ The SVE instruction set architecture provides no way to write VL directly.
+ Instead, it can be modified only by EL1 and above, by writing appropriate
+ system registers.
+
+* The value of VL can be configured at runtime by EL1 and above:
+ 16 <= VL <= VLmax, where VL must be a multiple of 16.
+
+* The maximum vector length is determined by the hardware:
+ 16 <= VLmax <= 256.
+
+ (The SVE architecture specifies 256, but permits future architecture
+ revisions to raise this limit.)
+
+* FPSR and FPCR are retained from ARMv8-A, and interact with SVE floating-point
+ operations in a similar way to the way in which they interact with ARMv8
+ floating-point operations.
+
+ 8VL-1 128 0 bit index
+ +---- //// -----------------+
+ Z0 | : V0 |
+ : :
+ Z7 | : V7 |
+ Z8 | : * V8 |
+ : : :
+ Z15 | : *V15 |
+ Z16 | : V16 |
+ : :
+ Z31 | : V31 |
+ +---- //// -----------------+
+ 31 0
+ VL-1 0 +-------+
+ +---- //// --+ FPSR | |
+ P0 | | +-------+
+ : | | *FPCR | |
+ P15 | | +-------+
+ +---- //// --+
+ FFR | | +-----+
+ +---- //// --+ VL | |
+ +-----+
+
+(*) callee-save:
+ This only applies to bits [63:0] of Z-/V-registers.
+ FPCR contains callee-save and caller-save bits. See [4] for details.
+
+
+A.2. Procedure call standard
+-----------------------------
+
+The ARMv8-A base procedure call standard is extended as follows with respect to
+the additional SVE register state:
+
+* All SVE register bits that are not shared with FP/SIMD are caller-save.
+
+* Z8 bits [63:0] .. Z15 bits [63:0] are callee-save.
+
+ This follows from the way these bits are mapped to V8..V15, which are caller-
+ save in the base procedure call standard.
+
+
+Appendix B. ARMv8-A FP/SIMD programmer's model
+===============================================
+
+Note: This section is for information only and not intended to be complete or
+to replace any architectural specification.
+
+Refer to [4] for for more information.
+
+ARMv8-A defines the following floating-point / SIMD register state:
+
+* 32 128-bit vector registers V0..V31
+* 2 32-bit status/control registers FPSR, FPCR
+
+ 127 0 bit index
+ +---------------+
+ V0 | |
+ : : :
+ V7 | |
+ * V8 | |
+ : : : :
+ *V15 | |
+ V16 | |
+ : : :
+ V31 | |
+ +---------------+
+
+ 31 0
+ +-------+
+ FPSR | |
+ +-------+
+ *FPCR | |
+ +-------+
+
+(*) callee-save:
+ This only applies to bits [63:0] of V-registers.
+ FPCR contains a mixture of callee-save and caller-save bits.
+
+
+References
+==========
+
+[1] arch/arm64/include/uapi/asm/sigcontext.h
+ AArch64 Linux signal ABI definitions
+
+[2] arch/arm64/include/uapi/asm/ptrace.h
+ AArch64 Linux ptrace ABI definitions
+
+[3] linux/Documentation/arm64/cpu-feature-registers.txt
+
+[4] ARM IHI0055C
+ http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055c/IHI0055C_beta_aapcs64.pdf
+ http://infocenter.arm.com/help/topic/com.arm.doc.subset.swdev.abi/index.html
+ Procedure Call Standard for the ARM 64-bit Architecture (AArch64)
diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h
index c78cf8e..dc9f5f4 100644
--- a/arch/arm64/include/uapi/asm/sigcontext.h
+++ b/arch/arm64/include/uapi/asm/sigcontext.h
@@ -133,6 +133,9 @@ struct sve_context {
* The SVE architecture leaves space for future expansion of the
* vector length beyond its initial architectural limit of 2048 bits
* (16 quadwords).
+ *
+ * See linux/Documentation/arm64/sve.txt for a description of the VL/VQ
+ * terminology.
*/
#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
This patch adds basic documentation of the user/kernel interface
provided by the for SVE.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
Cc: Mark Rutland <mark.rutland@arm.com>
---
Changes since v1
----------------
Requested by Alex Benn?e:
* Add cross-reference from sigcontext.h to the description of VL/VQ.
Other:
* [ABI fix] Bail out with -EIO if attempting to set the SVE regs for an
unsupported VL, rather than misparsing the regset data.
* Document detection of SVE via CPUID MRS.
* Remove comment about coredump notes being padded for NT_ARM_SVE.
This is no longer the case with variable-size regsets supported
in the core code.
* Fix a missing bullet char
---
Documentation/arm64/sve.txt | 477 +++++++++++++++++++++++++++++++
arch/arm64/include/uapi/asm/sigcontext.h | 3 +
2 files changed, 480 insertions(+)
create mode 100644 Documentation/arm64/sve.txt
diff --git a/Documentation/arm64/sve.txt b/Documentation/arm64/sve.txt
new file mode 100644
index 0000000..3c0f97f
--- /dev/null
+++ b/Documentation/arm64/sve.txt
@@ -0,0 +1,477 @@
+ Scalable Vector Extension support for AArch64 Linux
+ ===================================================
+
+Author: Dave Martin <Dave.Martin@arm.com>
+Date: 4 August 2017
+
+This document outlines briefly the interface provided to userspace by Linux in
+order to support use of the ARM Scalable Vector Extension (SVE).
+
+This is an outline of the most important features and issues only and not
+intended to be exhaustive.
+
+This document does not aim to describe the SVE architecture or programmer's
+model. To aid understanding, a minimal description of relevant programmer's
+model features for SVE is included in Appendix A.
+
+
+1. General
+-----------
+
+* SVE registers Z0..Z31, P0..P15 and FFR and the current vector length VL, are
+ tracked per-thread.
+
+* The presence of SVE is reported to userspace via HWCAP_SVE in the aux vector
+ AT_HWCAP entry. Presence of this flag implies the presence of the SVE
+ instructions and registers, and the Linux-specific system interfaces
+ described in this document. SVE is reported in /proc/cpuinfo as "sve".
+
+* Support for the execution of SVE instructions in userspace can also be
+ detected by reading the CPU ID register ID_AA64PFR0_EL1 using an MRS
+ instruction, and checking that the value of the SVE field is nonzero. [3]
+
+ It does not guarantee the presence of the system interfaces described in the
+ following sections: software that needs to verify that those interfaces are
+ present must check for HWCAP_SVE instead.
+
+* Debuggers should restrict themselves to interacting with the target via the
+ NT_ARM_SVE regset. The recommended way of detecting support for this regset
+ is to connect to a target process first and then attempt a
+ ptrace(PTRACE_GETREGSET, pid, NT_ARM_SVE, &iov).
+
+
+2. Vector length terminology
+-----------------------------
+
+The size of an SVE vector (Z) register is referred to as the "vector length".
+
+To avoid confusion about the units used to express vector length, the kernel
+adopts the following conventions:
+
+* Vector length (VL) = size of a Z-register in bytes
+
+* Vector quadwords (VQ) = size of a Z-register in units of 128 bits
+
+(So, VL = 16 * VQ.)
+
+The VQ convention is used where the underlying granularity is important, such
+as in data structure definitions. In most other situations, the VL convention
+is used. This is consistent with the meaning of the "VL" pseudo-register in
+the SVE instruction set architecture.
+
+
+3. System call behaviour
+-------------------------
+
+* On syscall, V0..V31 are preserved (as without SVE). Thus, bits [127:0] of
+ Z0..Z31 are preserved. All other bits of Z0..Z31, and all of P0..P15 and FFR
+ become unspecified on return from a syscall.
+
+* The SVE registers are not used to pass arguments to or receive results from
+ any syscall.
+
+* In practice the affected registers/bits will be preserved or will be replaced
+ with zeros on return from a syscall, but userspace should not make
+ assumptions about this. The kernel behaviour may vary on a case-by-case
+ basis.
+
+
+4. Signal handling
+-------------------
+
+* A new signal frame record sve_context encodes the SVE registers on signal
+ delivery. [1]
+
+* This record is supplementary to fpsimd_context. The FPSR and FPCR registers
+ are only present in fpsimd_context. For convenience, the content of V0..V31
+ is duplicated between sve_context and fpsimd_context.
+
+* The signal frame record for SVE always contains basic metadata, in particular
+ the thread's vector length (in sve_context.vl).
+
+* The SVE registers may or may not be included in the record, depending on
+ whether the registers are live for the thread. The registers are present if
+ and only if:
+ sve_context.head.size >= SVE_SIG_CONTEXT_SIZE(sve_vq_from_vl(sve_context.vl)).
+
+* If the registers are present, the remainder of the record has a vl-dependent
+ size and layout. Macros SIG_SVE_* are defined [1] to facilitate access to
+ the members.
+
+* If the SVE context is too big to fit in sigcontext.__reserved[], then extra
+ space is allocated on the stack, an extra_context record is written in
+ __reserved[] referencing this space. sve_context is then written in the
+ extra space. Refer to [1] for further details about this mechanism.
+
+
+5. Signal return
+-----------------
+
+When returning from a signal handler:
+
+* If there is no sve_context record in the signal frame, or if the record is
+ present but contains no register data as desribed in the previous section,
+ then the SVE registers/bits become non-live and take unspecified values.
+
+* If sve_context is present in the signal frame and contains full register
+ data, the SVE registers become live and are populated with the specified
+ data. However, for backward compatibility reasons, bits [127:0] of Z0..Z31
+ are always restored from the corresponding members of fpsimd_context.vregs[]
+ and not from sve_context. The remaining bits are restored from sve_context.
+
+* Inclusion of fpsimd_context in the signal frame remains mandatory,
+ irrespective of whether sve_context is present or not.
+
+* The vector length cannot be changed via signal return. If sve_context.vl in
+ the signal frame does not match the current vector length, the signal return
+ attempt is treated as illegal, resulting in a forced SIGSEGV.
+
+
+6. prctl extensions
+--------------------
+
+Some new prctl() calls are added to allow programs to manage the SVE vector
+length:
+
+prctl(PR_SVE_SET_VL, unsigned long arg)
+
+ Sets the vector length of the calling thread and related flags, where
+ arg == vl | flags.
+
+ vl is the desired vector length, where sve_vl_valid(vl) must be true.
+
+ flags:
+
+ PR_SVE_SET_VL_INHERIT
+
+ Inherit the current vector length across execve(). Otherwise, the
+ vector length is reset to the system default at execve(). (See
+ Section 9.)
+
+ PR_SVE_SET_VL_ONEXEC
+
+ Defer the requested vector length change until the next execve().
+ This allows launching of a new program with a different vector
+ length, while avoiding runtime side effects in the caller.
+
+ This also overrides the effect of PR_SVE_SET_VL_INHERIT for the
+ first execve().
+
+ Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector
+ length change is cancelled.
+
+ Return value: a nonnegative on success, or a negative value on error:
+ EINVAL: SVE not supported, invalid vector length requested, or
+ invalid flags.
+
+ On success, the calling thread's vector length is changed to the largest
+ value supported by the system that is less than or equal to vl.
+ If vl == SVE_VL_MAX, the calling thread's vector length is changed to the
+ largest value supported by the system.
+
+ The returned value describes the resulting configuration, encoded as for
+ PR_SVE_GET_VL.
+
+ Changing the vector length causes all of P0..P15, FFR and all bits of
+ Z0..V31 except for Z0 bits [127:0] .. Z31 bits [127:0] to become
+ unspecified. Calling PR_SVE_SET_VL with vl equal to the thread's current
+ vector length does not constitute a change to the vector length for this
+ purpose.
+
+
+prctl(PR_SVE_GET_VL)
+
+ Gets the vector length of the calling thread.
+
+ The following flag may be OR-ed into the result:
+
+ PR_SVE_SET_VL_INHERIT
+
+ Vector length will be inherited across execve().
+
+ There is no way to determine whether there is an outstanding deferred
+ vector length change (which would only normally be the case between a
+ fork() or vfork() and the corresponding execve() in typical use).
+
+ To extract the vector length from the result, and it with
+ PR_SVE_VL_LEN_MASK.
+
+ Return value: a nonnegative value on success, or a negative value on error:
+ EINVAL: SVE not supported.
+
+
+7. ptrace extensions
+---------------------
+
+* A new regset NT_ARM_SVE is defined for use with PTRACE_GETREGSET and
+ PTRACE_SETREGSET.
+
+ Refer to [2] for definitions.
+
+The regset data starts with struct user_sve_header, containing:
+
+ size
+
+ Size of the complete regset, in bytes.
+ This depends on vl and possibly on other things in the future.
+
+ If a call to PTRACE_GETREGSET requests less data than the value of
+ size, the caller can allocate a larger buffer and retry in order to
+ read the complete regset.
+
+ max_size
+
+ Maximum size in bytes that the regset can grow to for the target
+ thread. The regset won't grow bigger than this even if the target
+ thread changes its vector length etc.
+
+ vl
+
+ Target thread's current vector length, in bytes.
+
+ max_vl
+
+ Maximum possible vector length for the target thread.
+
+ flags
+
+ either
+
+ SVE_PT_REGS_FPSIMD
+
+ SVE registers are not live (GETREGSET) or are to be made
+ non-live (SETREGSET).
+
+ The payload is of type struct user_fpsimd_state, with the same
+ meaning as for NT_PRFPREG, starting at offset
+ SVE_PT_FPSIMD_OFFSET from the start of user_sve_header.
+
+ Extra data might be appended in the future: the size of the
+ payload should be obtained using SVE_PT_FPSIMD_SIZE(vq, flags).
+
+ vq should be obtained using sve_vq_from_vl(vl).
+
+ or
+
+ SVE_PT_REGS_SVE
+
+ SVE registers are live (GETREGSET) or are to be made live
+ (SETREGSET).
+
+ The payload contains the SVE register data, starting at offset
+ SVE_PT_SVE_OFFSET from the start of user_sve_header, and with
+ size SVE_PT_SVE_SIZE(vq, flags);
+
+ ... OR-ed with zero or more of the following flags, which have the same
+ meaning and behaviour as the corresponding PR_SET_VL_* flags:
+
+ SVE_PT_VL_INHERIT
+
+ SVE_PT_VL_ONEXEC (SETREGSET only).
+
+* The effects of changing the vector length and/or flags are equivalent to
+ those documented for PR_SVE_SET_VL.
+
+* In the SVE_PT_REGS_SVE case, the size and layout of the payload depends on
+ the header fields. The SVE_PT_SVE_*() macros are provided to facilitate
+ access to the members.
+
+* In either case, for SETREGSET it is permissible to omit the payload, in which
+ case only the vector length and flags are changed (along with any
+ consequences of those changes).
+
+* For SETREGSET, if an SVE_PT_REGS_SVE payload is present and the
+ requested VL is not supported, the effect will be the same as if the
+ payload were omitted, except that an EIO error is reported. No
+ attempt is made to translate the payload data to the correct layout
+ for the vector length actually set. The thread's FPSIMD state is
+ preserved, but the remaining bits of the SVE registers become
+ unspecified. It is up to the caller to translate the payload layout
+ for the actual VL and retry.
+
+* The effect of writing a partial, incomplete payload is unspecified.
+
+
+8. ELF coredump extensions
+---------------------------
+
+* A NT_ARM_SVE note will be added to each coredump for each thread of the
+ dumped process. The contents will be equivalent to the data that would have
+ been read if a PTRACE_GETREGSET of NT_ARM_SVE were executed for each thread
+ when the coredump was generated.
+
+
+9. System runtime configuration
+--------------------------------
+
+* To mitigate the ABI impact of expansion of the signal frame, a policy
+ mechanism is provided for administrators, distro maintainers and developers
+ to set the default vector length for userspace processes:
+
+/proc/cpu/sve_default_vector_length
+
+ Writing the text representation of an integer to this file sets the system
+ default vector length to the specified value, unless the value is greater
+ than the maximum vector length supported by the system in which case the
+ default vector length is set to that maximum.
+
+ The result can be determined by reopening the file and reading its
+ contents.
+
+ At boot, the default vector length is initially set to 64 or the maximum
+ supported vector length, whichever is smaller. This determines the initial
+ vector length of the init process (PID 1).
+
+ Reading this file returns the current system default vector length.
+
+* At every execve() call, the new vector length of the new process is set to
+ the system default vector length, unless
+
+ * PR_SVE_SET_VL_INHERIT (or equivalently SVE_PT_VL_INHERIT) is set for the
+ calling thread, or
+
+ * a deferred vector length change is pending, established via the
+ PR_SVE_SET_VL_ONEXEC flag (or SVE_PT_VL_ONEXEC).
+
+* Modifying the system default vector length does not affect the vector length
+ of any existing process or thread that does not make an execve() call.
+
+
+Appendix A. SVE programmer's model (informative)
+=================================================
+
+This section provides a minimal description of the additions made by SVE to the
+ARMv8-A programmer's model that are relevant to this document.
+
+Note: This section is for information only and not intended to be complete or
+to replace any architectural specification.
+
+A.1. Registers
+---------------
+
+In A64 state, SVE adds the following:
+
+* 32 8VL-bit vector registers Z0..Z31
+ For each Zn, Zn bits [127:0] alias the ARMv8-A vector register Vn.
+
+ A register write using a Vn register name zeros all bits of the corresponding
+ Zn except for bits [127:0].
+
+* 16 VL-bit predicate registers P0..P15
+
+* 1 VL-bit special-purpose predicate register FFR (the "first-fault register")
+
+* a VL "pseudo-register" that determines the size of each vector register
+
+ The SVE instruction set architecture provides no way to write VL directly.
+ Instead, it can be modified only by EL1 and above, by writing appropriate
+ system registers.
+
+* The value of VL can be configured at runtime by EL1 and above:
+ 16 <= VL <= VLmax, where VL must be a multiple of 16.
+
+* The maximum vector length is determined by the hardware:
+ 16 <= VLmax <= 256.
+
+ (The SVE architecture specifies 256, but permits future architecture
+ revisions to raise this limit.)
+
+* FPSR and FPCR are retained from ARMv8-A, and interact with SVE floating-point
+ operations in a similar way to the way in which they interact with ARMv8
+ floating-point operations.
+
+ 8VL-1 128 0 bit index
+ +---- //// -----------------+
+ Z0 | : V0 |
+ : :
+ Z7 | : V7 |
+ Z8 | : * V8 |
+ : : :
+ Z15 | : *V15 |
+ Z16 | : V16 |
+ : :
+ Z31 | : V31 |
+ +---- //// -----------------+
+ 31 0
+ VL-1 0 +-------+
+ +---- //// --+ FPSR | |
+ P0 | | +-------+
+ : | | *FPCR | |
+ P15 | | +-------+
+ +---- //// --+
+ FFR | | +-----+
+ +---- //// --+ VL | |
+ +-----+
+
+(*) callee-save:
+ This only applies to bits [63:0] of Z-/V-registers.
+ FPCR contains callee-save and caller-save bits. See [4] for details.
+
+
+A.2. Procedure call standard
+-----------------------------
+
+The ARMv8-A base procedure call standard is extended as follows with respect to
+the additional SVE register state:
+
+* All SVE register bits that are not shared with FP/SIMD are caller-save.
+
+* Z8 bits [63:0] .. Z15 bits [63:0] are callee-save.
+
+ This follows from the way these bits are mapped to V8..V15, which are caller-
+ save in the base procedure call standard.
+
+
+Appendix B. ARMv8-A FP/SIMD programmer's model
+===============================================
+
+Note: This section is for information only and not intended to be complete or
+to replace any architectural specification.
+
+Refer to [4] for for more information.
+
+ARMv8-A defines the following floating-point / SIMD register state:
+
+* 32 128-bit vector registers V0..V31
+* 2 32-bit status/control registers FPSR, FPCR
+
+ 127 0 bit index
+ +---------------+
+ V0 | |
+ : : :
+ V7 | |
+ * V8 | |
+ : : : :
+ *V15 | |
+ V16 | |
+ : : :
+ V31 | |
+ +---------------+
+
+ 31 0
+ +-------+
+ FPSR | |
+ +-------+
+ *FPCR | |
+ +-------+
+
+(*) callee-save:
+ This only applies to bits [63:0] of V-registers.
+ FPCR contains a mixture of callee-save and caller-save bits.
+
+
+References
+==========
+
+[1] arch/arm64/include/uapi/asm/sigcontext.h
+ AArch64 Linux signal ABI definitions
+
+[2] arch/arm64/include/uapi/asm/ptrace.h
+ AArch64 Linux ptrace ABI definitions
+
+[3] linux/Documentation/arm64/cpu-feature-registers.txt
+
+[4] ARM IHI0055C
+ http://infocenter.arm.com/help/topic/com.arm.doc.ihi0055c/IHI0055C_beta_aapcs64.pdf
+ http://infocenter.arm.com/help/topic/com.arm.doc.subset.swdev.abi/index.html
+ Procedure Call Standard for the ARM 64-bit Architecture (AArch64)
diff --git a/arch/arm64/include/uapi/asm/sigcontext.h b/arch/arm64/include/uapi/asm/sigcontext.h
index c78cf8e..dc9f5f4 100644
--- a/arch/arm64/include/uapi/asm/sigcontext.h
+++ b/arch/arm64/include/uapi/asm/sigcontext.h
@@ -133,6 +133,9 @@ struct sve_context {
* The SVE architecture leaves space for future expansion of the
* vector length beyond its initial architectural limit of 2048 bits
* (16 quadwords).
+ *
+ * See linux/Documentation/arm64/sve.txt for a description of the VL/VQ
+ * terminology.
*/
#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [RFC PATCH v2 27/28] arm64: signal: Report signal frame size to userspace via auxv
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Michael Ellerman,
Richard Sandiford, kvmarm
Stateful CPU architecture extensions may require the signal frame
to grow to a size that exceeds the arch's MINSIGSTKSZ #define.
However, changing this #define is an ABI break.
To allow userspace the option of determining the signal frame size
in a more forwards-compatible way, this patch adds a new auxv entry
tagged with AT_MINSIGSTKSZ, which provides the maximum signal frame
size that the process can observe during its lifetime.
If AT_MINSIGSTKSZ is absent from the aux vector, the caller can
assume that the MINSIGSTKSZ #define is sufficient. This allows for
a consistent interface with older kernels that do not provide
AT_MINSIGSTKSZ.
The idea is that libc could expose this via sysconf() or some
similar mechanism.
There is deliberately no AT_SIGSTKSZ. The kernel knows nothing
about userspace's own stack overheads and should not pretend to
know.
For arm64:
The primary motivation for this interface is the Scalable Vector
Extension, which can require at least 4KB or so of extra space
in the signal frame for the largest hardware implementations.
To determine the correct value, a "Christmas tree" mode (via the
add_all argument) is added to setup_sigframe_layout(), to simulate
addition of all possible records to the signal frame at maximum
possible size.
If this procedure goes wrong somehow, resulting in a stupidly large
frame layout and hence failure of sigframe_alloc() to allocate a
record to the frame, then this is indicative of a kernel bug: the
kernel's internal SIGFRAME_MAXSZ is supposed to sanity-check
against generting frames that we consider _impossibly_ large. In
this case, SIGSTKSZ is returned as a "reasonable guess that is at
least bigger than MINSIGSTKSZ" and we WARN().
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since v1
----------------
Related to Ard Biesheuvel's comments:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
---
arch/arm64/include/asm/elf.h | 5 +++++
arch/arm64/include/asm/processor.h | 3 +++
arch/arm64/include/uapi/asm/auxvec.h | 3 ++-
arch/arm64/kernel/signal.c | 39 +++++++++++++++++++++++++++++++-----
4 files changed, 44 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 33be513..8a2708a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -24,6 +24,10 @@
#include <asm/ptrace.h>
#include <asm/user.h>
+#ifndef __ASSEMBLY__
+#include <asm/processor.h> /* for get_minsigstksz(), used by ARCH_DLINFO */
+#endif
+
/*
* AArch64 static relocation types.
*/
@@ -148,6 +152,7 @@ typedef struct user_fpsimd_state elf_fpregset_t;
do { \
NEW_AUX_ENT(AT_SYSINFO_EHDR, \
(elf_addr_t)current->mm->context.vdso); \
+ NEW_AUX_ENT(AT_MINSIGSTKSZ, get_minsigstksz()); \
} while (0)
#define ARCH_HAS_SETUP_ADDITIONAL_PAGES
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index df66452..18af4bd 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -197,6 +197,9 @@ static inline void spin_lock_prefetch(const void *ptr)
int cpu_enable_pan(void *__unused);
int cpu_enable_cache_maint_trap(void *__unused);
+/* User signal frame size discovery: */
+int get_minsigstksz(void);
+
/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
#define SVE_SET_VL(arg) sve_set_current_vl(arg)
#define SVE_GET_VL() sve_get_current_vl()
diff --git a/arch/arm64/include/uapi/asm/auxvec.h b/arch/arm64/include/uapi/asm/auxvec.h
index 4cf0c17..1d45b28 100644
--- a/arch/arm64/include/uapi/asm/auxvec.h
+++ b/arch/arm64/include/uapi/asm/auxvec.h
@@ -18,7 +18,8 @@
/* vDSO location */
#define AT_SYSINFO_EHDR 33
+#define AT_MINSIGSTKSZ 34 /* stack needed for signal delivery */
-#define AT_VECTOR_SIZE_ARCH 1 /* entries in ARCH_DLINFO */
+#define AT_VECTOR_SIZE_ARCH 2 /* entries in ARCH_DLINFO */
#endif
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index c3f94cf..7a3d6d2 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -566,8 +566,15 @@ asmlinkage long sys_rt_sigreturn(struct pt_regs *regs)
return 0;
}
-/* Determine the layout of optional records in the signal frame */
-static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
+/*
+ * Determine the layout of optional records in the signal frame
+ *
+ * add_all: if true, lays out the biggest possible signal frame for
+ * this task; otherwise, generates a layout for the current state
+ * of the task.
+ */
+static int setup_sigframe_layout(struct rt_sigframe_user_layout *user,
+ bool add_all)
{
int err;
@@ -577,7 +584,7 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return err;
/* fault information, if valid */
- if (current->thread.fault_code) {
+ if (add_all || current->thread.fault_code) {
err = sigframe_alloc(user, &user->esr_offset,
sizeof(struct esr_context));
if (err)
@@ -599,7 +606,6 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return sigframe_alloc_end(user);
}
-
static int setup_sigframe(struct rt_sigframe_user_layout *user,
struct pt_regs *regs, sigset_t *set)
{
@@ -697,7 +703,7 @@ static int get_sigframe(struct rt_sigframe_user_layout *user,
int err;
init_user_layout(user);
- err = setup_sigframe_layout(user);
+ err = setup_sigframe_layout(user, false);
if (err)
return err;
@@ -931,3 +937,26 @@ asmlinkage void do_notify_resume(struct pt_regs *regs,
thread_flags = READ_ONCE(current_thread_info()->flags);
} while (thread_flags & _TIF_WORK_MASK);
}
+
+/*
+ * Determine the stack space required for guaranteed signal devliery.
+ * This function is used to populate AT_MINSIGSTKSZ at process startup.
+ */
+int get_minsigstksz(void)
+{
+ struct rt_sigframe_user_layout user;
+ int err;
+
+ init_user_layout(&user);
+ err = setup_sigframe_layout(&user, true);
+
+ if (err) {
+ WARN_ON(1);
+
+ return SIGSTKSZ;
+ } else {
+ return sigframe_size(&user) +
+ round_up(sizeof(struct frame_record), 16) +
+ 16; /* max alignment padding */
+ }
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [RFC PATCH v2 27/28] arm64: signal: Report signal frame size to userspace via auxv
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Michael Ellerman
Stateful CPU architecture extensions may require the signal frame
to grow to a size that exceeds the arch's MINSIGSTKSZ #define.
However, changing this #define is an ABI break.
To allow userspace the option of determining the signal frame size
in a more forwards-compatible way, this patch adds a new auxv entry
tagged with AT_MINSIGSTKSZ, which provides the maximum signal frame
size that the process can observe during its lifetime.
If AT_MINSIGSTKSZ is absent from the aux vector, the caller can
assume that the MINSIGSTKSZ #define is sufficient. This allows for
a consistent interface with older kernels that do not provide
AT_MINSIGSTKSZ.
The idea is that libc could expose this via sysconf() or some
similar mechanism.
There is deliberately no AT_SIGSTKSZ. The kernel knows nothing
about userspace's own stack overheads and should not pretend to
know.
For arm64:
The primary motivation for this interface is the Scalable Vector
Extension, which can require at least 4KB or so of extra space
in the signal frame for the largest hardware implementations.
To determine the correct value, a "Christmas tree" mode (via the
add_all argument) is added to setup_sigframe_layout(), to simulate
addition of all possible records to the signal frame at maximum
possible size.
If this procedure goes wrong somehow, resulting in a stupidly large
frame layout and hence failure of sigframe_alloc() to allocate a
record to the frame, then this is indicative of a kernel bug: the
kernel's internal SIGFRAME_MAXSZ is supposed to sanity-check
against generting frames that we consider _impossibly_ large. In
this case, SIGSTKSZ is returned as a "reasonable guess that is at
least bigger than MINSIGSTKSZ" and we WARN().
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since v1
----------------
Related to Ard Biesheuvel's comments:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
---
arch/arm64/include/asm/elf.h | 5 +++++
arch/arm64/include/asm/processor.h | 3 +++
arch/arm64/include/uapi/asm/auxvec.h | 3 ++-
arch/arm64/kernel/signal.c | 39 +++++++++++++++++++++++++++++++-----
4 files changed, 44 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 33be513..8a2708a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -24,6 +24,10 @@
#include <asm/ptrace.h>
#include <asm/user.h>
+#ifndef __ASSEMBLY__
+#include <asm/processor.h> /* for get_minsigstksz(), used by ARCH_DLINFO */
+#endif
+
/*
* AArch64 static relocation types.
*/
@@ -148,6 +152,7 @@ typedef struct user_fpsimd_state elf_fpregset_t;
do { \
NEW_AUX_ENT(AT_SYSINFO_EHDR, \
(elf_addr_t)current->mm->context.vdso); \
+ NEW_AUX_ENT(AT_MINSIGSTKSZ, get_minsigstksz()); \
} while (0)
#define ARCH_HAS_SETUP_ADDITIONAL_PAGES
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index df66452..18af4bd 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -197,6 +197,9 @@ static inline void spin_lock_prefetch(const void *ptr)
int cpu_enable_pan(void *__unused);
int cpu_enable_cache_maint_trap(void *__unused);
+/* User signal frame size discovery: */
+int get_minsigstksz(void);
+
/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
#define SVE_SET_VL(arg) sve_set_current_vl(arg)
#define SVE_GET_VL() sve_get_current_vl()
diff --git a/arch/arm64/include/uapi/asm/auxvec.h b/arch/arm64/include/uapi/asm/auxvec.h
index 4cf0c17..1d45b28 100644
--- a/arch/arm64/include/uapi/asm/auxvec.h
+++ b/arch/arm64/include/uapi/asm/auxvec.h
@@ -18,7 +18,8 @@
/* vDSO location */
#define AT_SYSINFO_EHDR 33
+#define AT_MINSIGSTKSZ 34 /* stack needed for signal delivery */
-#define AT_VECTOR_SIZE_ARCH 1 /* entries in ARCH_DLINFO */
+#define AT_VECTOR_SIZE_ARCH 2 /* entries in ARCH_DLINFO */
#endif
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index c3f94cf..7a3d6d2 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -566,8 +566,15 @@ asmlinkage long sys_rt_sigreturn(struct pt_regs *regs)
return 0;
}
-/* Determine the layout of optional records in the signal frame */
-static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
+/*
+ * Determine the layout of optional records in the signal frame
+ *
+ * add_all: if true, lays out the biggest possible signal frame for
+ * this task; otherwise, generates a layout for the current state
+ * of the task.
+ */
+static int setup_sigframe_layout(struct rt_sigframe_user_layout *user,
+ bool add_all)
{
int err;
@@ -577,7 +584,7 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return err;
/* fault information, if valid */
- if (current->thread.fault_code) {
+ if (add_all || current->thread.fault_code) {
err = sigframe_alloc(user, &user->esr_offset,
sizeof(struct esr_context));
if (err)
@@ -599,7 +606,6 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return sigframe_alloc_end(user);
}
-
static int setup_sigframe(struct rt_sigframe_user_layout *user,
struct pt_regs *regs, sigset_t *set)
{
@@ -697,7 +703,7 @@ static int get_sigframe(struct rt_sigframe_user_layout *user,
int err;
init_user_layout(user);
- err = setup_sigframe_layout(user);
+ err = setup_sigframe_layout(user, false);
if (err)
return err;
@@ -931,3 +937,26 @@ asmlinkage void do_notify_resume(struct pt_regs *regs,
thread_flags = READ_ONCE(current_thread_info()->flags);
} while (thread_flags & _TIF_WORK_MASK);
}
+
+/*
+ * Determine the stack space required for guaranteed signal devliery.
+ * This function is used to populate AT_MINSIGSTKSZ at process startup.
+ */
+int get_minsigstksz(void)
+{
+ struct rt_sigframe_user_layout user;
+ int err;
+
+ init_user_layout(&user);
+ err = setup_sigframe_layout(&user, true);
+
+ if (err) {
+ WARN_ON(1);
+
+ return SIGSTKSZ;
+ } else {
+ return sigframe_size(&user) +
+ round_up(sizeof(struct frame_record), 16) +
+ 16; /* max alignment padding */
+ }
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [RFC PATCH v2 27/28] arm64: signal: Report signal frame size to userspace via auxv
@ 2017-08-31 17:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:00 UTC (permalink / raw)
To: linux-arm-kernel
Stateful CPU architecture extensions may require the signal frame
to grow to a size that exceeds the arch's MINSIGSTKSZ #define.
However, changing this #define is an ABI break.
To allow userspace the option of determining the signal frame size
in a more forwards-compatible way, this patch adds a new auxv entry
tagged with AT_MINSIGSTKSZ, which provides the maximum signal frame
size that the process can observe during its lifetime.
If AT_MINSIGSTKSZ is absent from the aux vector, the caller can
assume that the MINSIGSTKSZ #define is sufficient. This allows for
a consistent interface with older kernels that do not provide
AT_MINSIGSTKSZ.
The idea is that libc could expose this via sysconf() or some
similar mechanism.
There is deliberately no AT_SIGSTKSZ. The kernel knows nothing
about userspace's own stack overheads and should not pretend to
know.
For arm64:
The primary motivation for this interface is the Scalable Vector
Extension, which can require at least 4KB or so of extra space
in the signal frame for the largest hardware implementations.
To determine the correct value, a "Christmas tree" mode (via the
add_all argument) is added to setup_sigframe_layout(), to simulate
addition of all possible records to the signal frame at maximum
possible size.
If this procedure goes wrong somehow, resulting in a stupidly large
frame layout and hence failure of sigframe_alloc() to allocate a
record to the frame, then this is indicative of a kernel bug: the
kernel's internal SIGFRAME_MAXSZ is supposed to sanity-check
against generting frames that we consider _impossibly_ large. In
this case, SIGSTKSZ is returned as a "reasonable guess that is at
least bigger than MINSIGSTKSZ" and we WARN().
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
---
Changes since v1
----------------
Related to Ard Biesheuvel's comments:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
---
arch/arm64/include/asm/elf.h | 5 +++++
arch/arm64/include/asm/processor.h | 3 +++
arch/arm64/include/uapi/asm/auxvec.h | 3 ++-
arch/arm64/kernel/signal.c | 39 +++++++++++++++++++++++++++++++-----
4 files changed, 44 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/include/asm/elf.h b/arch/arm64/include/asm/elf.h
index 33be513..8a2708a 100644
--- a/arch/arm64/include/asm/elf.h
+++ b/arch/arm64/include/asm/elf.h
@@ -24,6 +24,10 @@
#include <asm/ptrace.h>
#include <asm/user.h>
+#ifndef __ASSEMBLY__
+#include <asm/processor.h> /* for get_minsigstksz(), used by ARCH_DLINFO */
+#endif
+
/*
* AArch64 static relocation types.
*/
@@ -148,6 +152,7 @@ typedef struct user_fpsimd_state elf_fpregset_t;
do { \
NEW_AUX_ENT(AT_SYSINFO_EHDR, \
(elf_addr_t)current->mm->context.vdso); \
+ NEW_AUX_ENT(AT_MINSIGSTKSZ, get_minsigstksz()); \
} while (0)
#define ARCH_HAS_SETUP_ADDITIONAL_PAGES
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index df66452..18af4bd 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -197,6 +197,9 @@ static inline void spin_lock_prefetch(const void *ptr)
int cpu_enable_pan(void *__unused);
int cpu_enable_cache_maint_trap(void *__unused);
+/* User signal frame size discovery: */
+int get_minsigstksz(void);
+
/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
#define SVE_SET_VL(arg) sve_set_current_vl(arg)
#define SVE_GET_VL() sve_get_current_vl()
diff --git a/arch/arm64/include/uapi/asm/auxvec.h b/arch/arm64/include/uapi/asm/auxvec.h
index 4cf0c17..1d45b28 100644
--- a/arch/arm64/include/uapi/asm/auxvec.h
+++ b/arch/arm64/include/uapi/asm/auxvec.h
@@ -18,7 +18,8 @@
/* vDSO location */
#define AT_SYSINFO_EHDR 33
+#define AT_MINSIGSTKSZ 34 /* stack needed for signal delivery */
-#define AT_VECTOR_SIZE_ARCH 1 /* entries in ARCH_DLINFO */
+#define AT_VECTOR_SIZE_ARCH 2 /* entries in ARCH_DLINFO */
#endif
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index c3f94cf..7a3d6d2 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -566,8 +566,15 @@ asmlinkage long sys_rt_sigreturn(struct pt_regs *regs)
return 0;
}
-/* Determine the layout of optional records in the signal frame */
-static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
+/*
+ * Determine the layout of optional records in the signal frame
+ *
+ * add_all: if true, lays out the biggest possible signal frame for
+ * this task; otherwise, generates a layout for the current state
+ * of the task.
+ */
+static int setup_sigframe_layout(struct rt_sigframe_user_layout *user,
+ bool add_all)
{
int err;
@@ -577,7 +584,7 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return err;
/* fault information, if valid */
- if (current->thread.fault_code) {
+ if (add_all || current->thread.fault_code) {
err = sigframe_alloc(user, &user->esr_offset,
sizeof(struct esr_context));
if (err)
@@ -599,7 +606,6 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user)
return sigframe_alloc_end(user);
}
-
static int setup_sigframe(struct rt_sigframe_user_layout *user,
struct pt_regs *regs, sigset_t *set)
{
@@ -697,7 +703,7 @@ static int get_sigframe(struct rt_sigframe_user_layout *user,
int err;
init_user_layout(user);
- err = setup_sigframe_layout(user);
+ err = setup_sigframe_layout(user, false);
if (err)
return err;
@@ -931,3 +937,26 @@ asmlinkage void do_notify_resume(struct pt_regs *regs,
thread_flags = READ_ONCE(current_thread_info()->flags);
} while (thread_flags & _TIF_WORK_MASK);
}
+
+/*
+ * Determine the stack space required for guaranteed signal devliery.
+ * This function is used to populate AT_MINSIGSTKSZ at process startup.
+ */
+int get_minsigstksz(void)
+{
+ struct rt_sigframe_user_layout user;
+ int err;
+
+ init_user_layout(&user);
+ err = setup_sigframe_layout(&user, true);
+
+ if (err) {
+ WARN_ON(1);
+
+ return SIGSTKSZ;
+ } else {
+ return sigframe_size(&user) +
+ round_up(sizeof(struct frame_record), 16) +
+ 16; /* max alignment padding */
+ }
+}
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [RFC PATCH v2 28/28] arm64/sve: signal: Include SVE when computing AT_MINSIGSTKSZ
2017-08-31 17:00 ` Dave Martin
@ 2017-08-31 17:01 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:01 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Michael Ellerman
The SVE context block in the signal frame needs to be considered
too when computing the maximum possible signal frame size.
Because the size of this block depends on the vector length, this
patch computes the size based not on the thread's current vector
length but instead on the maximum possible vector length: this
determines the maximum size of SVE context block that can be
observed in any signal frame for the lifetime of the process.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Changes related to Ard Biesheuvel's comments:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
Requested by Alex Bennée:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
---
arch/arm64/kernel/signal.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 7a3d6d2..c5ae575 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -594,8 +594,18 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user,
if (system_supports_sve()) {
unsigned int vq = 0;
- if (test_thread_flag(TIF_SVE))
- vq = sve_vq_from_vl(current->thread.sve_vl);
+ if (add_all || test_thread_flag(TIF_SVE)) {
+ int vl = sve_max_vl;
+
+ if (!add_all)
+ vl = current->thread.sve_vl;
+
+ /* Fail safe if something wasn't initialised */
+ if (WARN_ON(!sve_vl_valid(vl)))
+ vl = SVE_VL_MIN;
+
+ vq = sve_vq_from_vl(vl);
+ }
err = sigframe_alloc(user, &user->sve_offset,
SVE_SIG_CONTEXT_SIZE(vq));
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [RFC PATCH v2 28/28] arm64/sve: signal: Include SVE when computing AT_MINSIGSTKSZ
@ 2017-08-31 17:01 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-08-31 17:01 UTC (permalink / raw)
To: linux-arm-kernel
The SVE context block in the signal frame needs to be considered
too when computing the maximum possible signal frame size.
Because the size of this block depends on the vector length, this
patch computes the size based not on the thread's current vector
length but instead on the maximum possible vector length: this
determines the maximum size of SVE context block that can be
observed in any signal frame for the lifetime of the process.
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Changes related to Ard Biesheuvel's comments:
* Fix unbalanced ifelse bracing to conform to the kernel coding style.
Requested by Alex Benn?e:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
---
arch/arm64/kernel/signal.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c
index 7a3d6d2..c5ae575 100644
--- a/arch/arm64/kernel/signal.c
+++ b/arch/arm64/kernel/signal.c
@@ -594,8 +594,18 @@ static int setup_sigframe_layout(struct rt_sigframe_user_layout *user,
if (system_supports_sve()) {
unsigned int vq = 0;
- if (test_thread_flag(TIF_SVE))
- vq = sve_vq_from_vl(current->thread.sve_vl);
+ if (add_all || test_thread_flag(TIF_SVE)) {
+ int vl = sve_max_vl;
+
+ if (!add_all)
+ vl = current->thread.sve_vl;
+
+ /* Fail safe if something wasn't initialised */
+ if (WARN_ON(!sve_vl_valid(vl)))
+ vl = SVE_VL_MIN;
+
+ vq = sve_vq_from_vl(vl);
+ }
err = sigframe_alloc(user, &user->sve_offset,
SVE_SIG_CONTEXT_SIZE(vq));
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* RE: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
2017-08-31 17:00 ` Dave Martin
@ 2017-09-06 16:21 ` Okamoto, Takayuki
-1 siblings, 0 replies; 224+ messages in thread
From: Okamoto, Takayuki @ 2017-09-06 16:21 UTC (permalink / raw)
To: 'Dave Martin', linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
gdb, Alan Hayward, Yao Qi, Oleg Nesterov, Alexander Viro
Hi Dave,
I am an engineer of the postK computer from Fujitsu.
When I tried to read "max_vl" by ptrace with this patch on our local SVE
simulator, it was read as zero.
I think the cause of this incident is that "max_vl" is set as "header->vl"
only on warning case in sve_init_header_from_task().
"max_vl" should be set up also on normal case, like the following patch.
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -755,6 +755,8 @@ static void sve_init_header_from_task(struct user_sve_header *header,
if (WARN_ON(!sve_vl_valid(sve_max_vl)))
header->max_vl = header->vl;
+ else
+ header->max_vl = sve_max_vl;
header->size = SVE_PT_SIZE(vq, header->flags);
header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
Best regards,
Takayuki Okamoto
-----Original Message-----
From: gdb-owner@sourceware.org [mailto:gdb-owner@sourceware.org] On Behalf Of Dave Martin
Sent: Friday, September 1, 2017 2:01 AM
To: linux-arm-kernel@lists.infradead.org
Cc: Catalin Marinas <catalin.marinas@arm.com>; Will Deacon <will.deacon@arm.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Alex Bennée <alex.bennee@linaro.org>; Szabolcs Nagy <szabolcs.nagy@arm.com>; Richard Sandiford <richard.sandiford@arm.com>; kvmarm@lists.cs.columbia.edu; libc-alpha@sourceware.org; linux-arch@vger.kernel.org; gdb@sourceware.org; Alan Hayward <alan.hayward@arm.com>; Yao Qi <Yao.Qi@arm.com>; Oleg Nesterov <oleg@redhat.com>; Alexander Viro <viro@zeniv.linux.org.uk>
Subject: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
This patch defines and implements a new regset NT_ARM_SVE, which
describes a thread's SVE register state. This allows a debugger to
manipulate the SVE state, as well as being included in ELF
coredumps for post-mortem debugging.
Because the regset size and layout are dependent on the thread's
current vector length, it is not possible to define a C struct to
describe the regset contents as is done for existing regsets.
Instead, and for the same reasons, NT_ARM_SVE is based on the
freeform variable-layout approach used for the SVE signal frame.
Additionally, to reduce debug overhead when debugging threads that
might or might not have live SVE register state, NT_ARM_SVE may be
presented in one of two different formats: the old struct
user_fpsimd_state format is embedded for describing the state of a
thread with no live SVE state, whereas a new variable-layout
structure is embedded for describing live SVE state. This avoids a
debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
allows existing userspace code to handle the non-SVE case without
too much modification.
For this to work, NT_ARM_SVE is defined with a fixed-format header
of type struct user_sve_header, which the recipient can use to
figure out the content, size and layout of the reset of the regset.
Accessor macros are defined to allow the vector-length-dependent
parts of the regset to be manipulated.
Signed-off-by: Alan Hayward <alan.hayward@arm.com>
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Bennée <alex.bennee@linaro.org>
---
Changes since v1
----------------
Other changes related to Alex Bennée's comments:
* Migrate to SVE_VQ_BYTES instead of magic numbers.
Requested by Alex Bennée:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other:
* [ABI fix] Bail out with -EIO if attempting to set the
SVE regs for an unsupported VL, instead of misparsing the regset data.
* Replace some in-kernel open-coded arithmetic with ALIGN()/
DIV_ROUND_UP().
---
arch/arm64/include/asm/fpsimd.h | 13 +-
arch/arm64/include/uapi/asm/ptrace.h | 135 ++++++++++++++++++
arch/arm64/kernel/fpsimd.c | 40 +++++-
arch/arm64/kernel/ptrace.c | 270 +++++++++++++++++++++++++++++++++--
include/uapi/linux/elf.h | 1 +
5 files changed, 449 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 6c22624..2723cca 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -38,13 +38,16 @@ struct fpsimd_state {
__uint128_t vregs[32];
u32 fpsr;
u32 fpcr;
+ /*
+ * For ptrace compatibility, pad to next 128-bit
+ * boundary here if extending this struct.
+ */
};
};
/* the id of the last cpu to have restored this state */
unsigned int cpu;
};
-
#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
/* Masks for extracting the FPSR and FPCR from the FPSCR */
#define VFP_FPSCR_STAT_MASK 0xf800009f
@@ -89,6 +92,10 @@ extern void sve_alloc(struct task_struct *task);
extern void fpsimd_release_thread(struct task_struct *task);
extern void fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src);
+extern void fpsimd_sync_to_sve(struct task_struct *task);
+extern void sve_sync_to_fpsimd(struct task_struct *task);
+extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
+
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
@@ -103,6 +110,10 @@ static void __maybe_unused sve_alloc(struct task_struct *task) { }
static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src) { }
+static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
+static void __maybe_unused sve_sync_from_fpsimd_zeropad(
+ struct task_struct *task) { }
+
static void __maybe_unused sve_init_vq_map(void) { }
static void __maybe_unused sve_update_vq_map(void) { }
static int __maybe_unused sve_verify_vq_map(void) { return 0; }
diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
index d1ff83d..1915ab0 100644
--- a/arch/arm64/include/uapi/asm/ptrace.h
+++ b/arch/arm64/include/uapi/asm/ptrace.h
@@ -22,6 +22,7 @@
#include <linux/types.h>
#include <asm/hwcap.h>
+#include <asm/sigcontext.h>
/*
@@ -63,6 +64,8 @@
#ifndef __ASSEMBLY__
+#include <linux/prctl.h>
+
/*
* User structures for general purpose, floating point and debug registers.
*/
@@ -90,6 +93,138 @@ struct user_hwdebug_state {
} dbg_regs[16];
};
+/* SVE/FP/SIMD state (NT_ARM_SVE) */
+
+struct user_sve_header {
+ __u32 size; /* total meaningful regset content in bytes */
+ __u32 max_size; /* maxmium possible size for this thread */
+ __u16 vl; /* current vector length */
+ __u16 max_vl; /* maximum possible vector length */
+ __u16 flags;
+ __u16 __reserved;
+};
+
+/* Definitions for user_sve_header.flags: */
+#define SVE_PT_REGS_MASK (1 << 0)
+
+/* Flags: must be kept in sync with prctl interface in <linux/ptrace.h> */
+#define SVE_PT_REGS_FPSIMD 0
+#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
+
+#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
+#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
+
+
+/*
+ * The remainder of the SVE state follows struct user_sve_header. The
+ * total size of the SVE state (including header) depends on the
+ * metadata in the header: SVE_PT_SIZE(vq, flags) gives the total size
+ * of the state in bytes, including the header.
+ *
+ * Refer to <asm/sigcontext.h> for details of how to pass the correct
+ * "vq" argument to these macros.
+ */
+
+/* Offset from the start of struct user_sve_header to the register data */
+#define SVE_PT_REGS_OFFSET \
+ ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+/*
+ * The register data content and layout depends on the value of the
+ * flags field.
+ */
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD case:
+ *
+ * The payload starts at offset SVE_PT_FPSIMD_OFFSET, and is of type
+ * struct user_fpsimd_state. Additional data might be appended in the
+ * future: use SVE_PT_FPSIMD_SIZE(vq, flags) to compute the total size.
+ * SVE_PT_FPSIMD_SIZE(vq, flags) will never be less than
+ * sizeof(struct user_fpsimd_state).
+ */
+
+#define SVE_PT_FPSIMD_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_FPSIMD_SIZE(vq, flags) (sizeof(struct user_fpsimd_state))
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE case:
+ *
+ * The payload starts at offset SVE_PT_SVE_OFFSET, and is of size
+ * SVE_PT_SVE_SIZE(vq, flags).
+ *
+ * Additional macros describe the contents and layout of the payload.
+ * For each, SVE_PT_SVE_x_OFFSET(args) is the start offset relative to
+ * the start of struct user_sve_header, and SVE_PT_SVE_x_SIZE(args) is
+ * the size in bytes:
+ *
+ * x type description
+ * - ---- -----------
+ * ZREGS \
+ * ZREG |
+ * PREGS | refer to <asm/sigcontext.h>
+ * PREG |
+ * FFR /
+ *
+ * FPSR uint32_t FPSR
+ * FPCR uint32_t FPCR
+ *
+ * Additional data might be appended in the future.
+ */
+
+#define SVE_PT_SVE_ZREG_SIZE(vq) SVE_SIG_ZREG_SIZE(vq)
+#define SVE_PT_SVE_PREG_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
+#define SVE_PT_SVE_FFR_SIZE(vq) SVE_SIG_FFR_SIZE(vq)
+#define SVE_PT_SVE_FPSR_SIZE sizeof(__u32)
+#define SVE_PT_SVE_FPCR_SIZE sizeof(__u32)
+
+#define __SVE_SIG_TO_PT(offset) \
+ ((offset) - SVE_SIG_REGS_OFFSET + SVE_PT_REGS_OFFSET)
+
+#define SVE_PT_SVE_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_SVE_ZREGS_OFFSET \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREGS_OFFSET)
+#define SVE_PT_SVE_ZREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREG_OFFSET(vq, n))
+#define SVE_PT_SVE_ZREGS_SIZE(vq) \
+ (SVE_PT_SVE_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_PT_SVE_ZREGS_OFFSET)
+
+#define SVE_PT_SVE_PREGS_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREGS_OFFSET(vq))
+#define SVE_PT_SVE_PREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREG_OFFSET(vq, n))
+#define SVE_PT_SVE_PREGS_SIZE(vq) \
+ (SVE_PT_SVE_PREG_OFFSET(vq, SVE_NUM_PREGS) - \
+ SVE_PT_SVE_PREGS_OFFSET(vq))
+
+#define SVE_PT_SVE_FFR_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_FFR_OFFSET(vq))
+
+#define SVE_PT_SVE_FPSR_OFFSET(vq) \
+ ((SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq) + \
+ (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+#define SVE_PT_SVE_FPCR_OFFSET(vq) \
+ (SVE_PT_SVE_FPSR_OFFSET(vq) + SVE_PT_SVE_FPSR_SIZE)
+
+/*
+ * Any future extension appended after FPCR must be aligned to the next
+ * 128-bit boundary.
+ */
+
+#define SVE_PT_SVE_SIZE(vq, flags) \
+ ((SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE \
+ - SVE_PT_SVE_OFFSET + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+#define SVE_PT_SIZE(vq, flags) \
+ (((flags) & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE ? \
+ SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, flags) \
+ : SVE_PT_FPSIMD_OFFSET + SVE_PT_FPSIMD_SIZE(vq, flags))
+
#endif /* __ASSEMBLY__ */
#endif /* _UAPI__ASM_PTRACE_H */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index fff9fcf..361c019 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -303,6 +303,37 @@ void sve_alloc(struct task_struct *task)
BUG_ON(!task->thread.sve_state);
}
+void fpsimd_sync_to_sve(struct task_struct *task)
+{
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ fpsimd_to_sve(task);
+}
+
+void sve_sync_to_fpsimd(struct task_struct *task)
+{
+ if (test_tsk_thread_flag(task, TIF_SVE))
+ sve_to_fpsimd(task);
+}
+
+void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
+{
+ unsigned int vq;
+ void *sst = task->thread.sve_state;
+ struct fpsimd_state const *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+
+ memset(sst, 0, SVE_SIG_REGS_SIZE(vq));
+
+ for (i = 0; i < 32; ++i)
+ memcpy(ZREG(sst, vq, i), &fst->vregs[i],
+ sizeof(fst->vregs[i]));
+}
+
/*
* Handle SVE state across fork():
*
@@ -459,10 +490,17 @@ static void __init sve_efi_setup(void)
* This is evidence of a crippled system and we are returning void,
* so no attempt is made to handle this situation here.
*/
- BUG_ON(!sve_vl_valid(sve_max_vl));
+ if (!sve_vl_valid(sve_max_vl))
+ goto fail;
+
efi_sve_state = __alloc_percpu(
SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
if (!efi_sve_state)
+ goto fail;
+
+ return;
+
+fail:
panic("Cannot allocate percpu memory for EFI SVE save/restore");
}
diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
index 9cbb612..5ef4735b 100644
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -32,6 +32,7 @@
#include <linux/security.h>
#include <linux/init.h>
#include <linux/signal.h>
+#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/perf_event.h>
#include <linux/hw_breakpoint.h>
@@ -40,6 +41,7 @@
#include <linux/elf.h>
#include <asm/compat.h>
+#include <asm/cpufeature.h>
#include <asm/debug-monitors.h>
#include <asm/pgtable.h>
#include <asm/stacktrace.h>
@@ -618,33 +620,66 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
/*
* TODO: update fp accessors for lazy context switching (sync/flush hwstate)
*/
-static int fpr_get(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- void *kbuf, void __user *ubuf)
+static int __fpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf, unsigned int start_pos)
{
struct user_fpsimd_state *uregs;
+
+ sve_sync_to_fpsimd(target);
+
uregs = &target->thread.fpsimd_state.user_fpsimd;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs,
+ start_pos, start_pos + sizeof(*uregs));
+}
+
+static int fpr_get(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
if (target == current)
fpsimd_preserve_current_state();
- return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf, 0);
}
-static int fpr_set(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- const void *kbuf, const void __user *ubuf)
+static int __fpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf,
+ unsigned int start_pos)
{
int ret;
struct user_fpsimd_state newstate =
target->thread.fpsimd_state.user_fpsimd;
- ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
+ sve_sync_to_fpsimd(target);
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate,
+ start_pos, start_pos + sizeof(newstate));
if (ret)
return ret;
target->thread.fpsimd_state.user_fpsimd = newstate;
+
+ return ret;
+}
+
+static int fpr_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, 0);
+ if (ret)
+ return ret;
+
+ sve_sync_from_fpsimd_zeropad(target);
fpsimd_flush_task_state(target);
+
return ret;
}
@@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
return ret;
}
+#ifdef CONFIG_ARM64_SVE
+
+static void sve_init_header_from_task(struct user_sve_header *header,
+ struct task_struct *target)
+{
+ unsigned int vq;
+
+ memset(header, 0, sizeof(*header));
+
+ header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
+ SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
+ if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
+ header->flags |= SVE_PT_VL_INHERIT;
+
+ header->vl = target->thread.sve_vl;
+ vq = sve_vq_from_vl(header->vl);
+
+ if (WARN_ON(!sve_vl_valid(sve_max_vl)))
+ header->max_vl = header->vl;
+
+ header->size = SVE_PT_SIZE(vq, header->flags);
+ header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
+ SVE_PT_REGS_SVE);
+}
+
+static unsigned int sve_size_from_header(struct user_sve_header const *header)
+{
+ return ALIGN(header->size, SVE_VQ_BYTES);
+}
+
+static unsigned int sve_get_size(struct task_struct *target,
+ const struct user_regset *regset)
+{
+ struct user_sve_header header;
+
+ if (!system_supports_sve())
+ return 0;
+
+ sve_init_header_from_task(&header, target);
+ return sve_size_from_header(&header);
+}
+
+static int sve_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ sve_init_header_from_task(&header, target);
+ vq = sve_vq_from_vl(header.vl);
+
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ return ret;
+
+ if (target == current)
+ fpsimd_preserve_current_state();
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD)
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+
+ /* Otherwise: full SVE case */
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ return ret;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = sve_size_from_header(&header);
+ return user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+}
+
+static int sve_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ if (count < sizeof(header))
+ return -EINVAL;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ goto out;
+
+ /*
+ * Apart from PT_SVE_REGS_MASK, all PT_SVE_* flags are consumed by
+ * sve_set_vector_length(), which will also validate them for us:
+ */
+ ret = sve_set_vector_length(target, header.vl,
+ header.flags & ~SVE_PT_REGS_MASK);
+ if (ret)
+ goto out;
+
+ /* Actual VL set may be less than the user asked for: */
+ vq = sve_vq_from_vl(target->thread.sve_vl);
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD) {
+ sve_sync_to_fpsimd(target);
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+ clear_tsk_thread_flag(target, TIF_SVE);
+ goto out;
+ }
+
+ /* Otherwise: full SVE case */
+
+ /*
+ * If setting a different VL from the requested VL and there is
+ * register data, the data layout will be wrong: don't even
+ * try to set the registers in this case.
+ */
+ if (count && vq != sve_vq_from_vl(header.vl)) {
+ ret = -EIO;
+ goto out;
+ }
+
+ sve_alloc(target);
+ fpsimd_sync_to_sve(target);
+ set_tsk_thread_flag(target, TIF_SVE);
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ goto out;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ goto out;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+
+out:
+ fpsimd_flush_task_state(target);
+ return ret;
+}
+
+#endif /* CONFIG_ARM64_SVE */
+
enum aarch64_regset {
REGSET_GPR,
REGSET_FPR,
@@ -711,6 +950,9 @@ enum aarch64_regset {
REGSET_HW_WATCH,
#endif
REGSET_SYSTEM_CALL,
+#ifdef CONFIG_ARM64_SVE
+ REGSET_SVE,
+#endif
};
static const struct user_regset aarch64_regsets[] = {
@@ -768,6 +1010,18 @@ static const struct user_regset aarch64_regsets[] = {
.get = system_call_get,
.set = system_call_set,
},
+#ifdef CONFIG_ARM64_SVE
+ [REGSET_SVE] = { /* Scalable Vector Extension */
+ .core_note_type = NT_ARM_SVE,
+ .n = DIV_ROUND_UP(SVE_PT_SIZE(SVE_VQ_MAX, SVE_PT_REGS_SVE),
+ SVE_VQ_BYTES),
+ .size = SVE_VQ_BYTES,
+ .align = SVE_VQ_BYTES,
+ .get = sve_get,
+ .set = sve_set,
+ .get_size = sve_get_size,
+ },
+#endif
};
static const struct user_regset_view user_aarch64_view = {
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index b5280db..735b8f4 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -416,6 +416,7 @@ typedef struct elf64_shdr {
#define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
#define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
#define NT_ARM_SYSTEM_CALL 0x404 /* ARM system call number */
+#define NT_ARM_SVE 0x405 /* ARM Scalable Vector Extension registers */
#define NT_METAG_CBUF 0x500 /* Metag catch buffer registers */
#define NT_METAG_RPIPE 0x501 /* Metag read pipeline state */
#define NT_METAG_TLS 0x502 /* Metag TLS pointer */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-06 16:21 ` Okamoto, Takayuki
0 siblings, 0 replies; 224+ messages in thread
From: Okamoto, Takayuki @ 2017-09-06 16:21 UTC (permalink / raw)
To: linux-arm-kernel
Hi Dave,
I am an engineer of the postK computer from Fujitsu.
When I tried to read "max_vl" by ptrace with this patch on our local SVE
simulator, it was read as zero.
I think the cause of this incident is that "max_vl" is set as "header->vl"
only on warning case in sve_init_header_from_task().
"max_vl" should be set up also on normal case, like the following patch.
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -755,6 +755,8 @@ static void sve_init_header_from_task(struct user_sve_header *header,
if (WARN_ON(!sve_vl_valid(sve_max_vl)))
header->max_vl = header->vl;
+ else
+ header->max_vl = sve_max_vl;
header->size = SVE_PT_SIZE(vq, header->flags);
header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
Best regards,
Takayuki Okamoto
-----Original Message-----
From: gdb-owner@sourceware.org [mailto:gdb-owner at sourceware.org] On Behalf Of Dave Martin
Sent: Friday, September 1, 2017 2:01 AM
To: linux-arm-kernel at lists.infradead.org
Cc: Catalin Marinas <catalin.marinas@arm.com>; Will Deacon <will.deacon@arm.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Alex Benn?e <alex.bennee@linaro.org>; Szabolcs Nagy <szabolcs.nagy@arm.com>; Richard Sandiford <richard.sandiford@arm.com>; kvmarm at lists.cs.columbia.edu; libc-alpha at sourceware.org; linux-arch at vger.kernel.org; gdb at sourceware.org; Alan Hayward <alan.hayward@arm.com>; Yao Qi <Yao.Qi@arm.com>; Oleg Nesterov <oleg@redhat.com>; Alexander Viro <viro@zeniv.linux.org.uk>
Subject: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
This patch defines and implements a new regset NT_ARM_SVE, which
describes a thread's SVE register state. This allows a debugger to
manipulate the SVE state, as well as being included in ELF
coredumps for post-mortem debugging.
Because the regset size and layout are dependent on the thread's
current vector length, it is not possible to define a C struct to
describe the regset contents as is done for existing regsets.
Instead, and for the same reasons, NT_ARM_SVE is based on the
freeform variable-layout approach used for the SVE signal frame.
Additionally, to reduce debug overhead when debugging threads that
might or might not have live SVE register state, NT_ARM_SVE may be
presented in one of two different formats: the old struct
user_fpsimd_state format is embedded for describing the state of a
thread with no live SVE state, whereas a new variable-layout
structure is embedded for describing live SVE state. This avoids a
debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
allows existing userspace code to handle the non-SVE case without
too much modification.
For this to work, NT_ARM_SVE is defined with a fixed-format header
of type struct user_sve_header, which the recipient can use to
figure out the content, size and layout of the reset of the regset.
Accessor macros are defined to allow the vector-length-dependent
parts of the regset to be manipulated.
Signed-off-by: Alan Hayward <alan.hayward@arm.com>
Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Cc: Alex Benn?e <alex.bennee@linaro.org>
---
Changes since v1
----------------
Other changes related to Alex Benn?e's comments:
* Migrate to SVE_VQ_BYTES instead of magic numbers.
Requested by Alex Benn?e:
* Thin out BUG_ON()s:
Redundant BUG_ON()s and ones that just check invariants are removed.
Important sanity-checks are migrated to WARN_ON()s, with some
minimal best-effort patch-up code.
Other:
* [ABI fix] Bail out with -EIO if attempting to set the
SVE regs for an unsupported VL, instead of misparsing the regset data.
* Replace some in-kernel open-coded arithmetic with ALIGN()/
DIV_ROUND_UP().
---
arch/arm64/include/asm/fpsimd.h | 13 +-
arch/arm64/include/uapi/asm/ptrace.h | 135 ++++++++++++++++++
arch/arm64/kernel/fpsimd.c | 40 +++++-
arch/arm64/kernel/ptrace.c | 270 +++++++++++++++++++++++++++++++++--
include/uapi/linux/elf.h | 1 +
5 files changed, 449 insertions(+), 10 deletions(-)
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 6c22624..2723cca 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -38,13 +38,16 @@ struct fpsimd_state {
__uint128_t vregs[32];
u32 fpsr;
u32 fpcr;
+ /*
+ * For ptrace compatibility, pad to next 128-bit
+ * boundary here if extending this struct.
+ */
};
};
/* the id of the last cpu to have restored this state */
unsigned int cpu;
};
-
#if defined(__KERNEL__) && defined(CONFIG_COMPAT)
/* Masks for extracting the FPSR and FPCR from the FPSCR */
#define VFP_FPSCR_STAT_MASK 0xf800009f
@@ -89,6 +92,10 @@ extern void sve_alloc(struct task_struct *task);
extern void fpsimd_release_thread(struct task_struct *task);
extern void fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src);
+extern void fpsimd_sync_to_sve(struct task_struct *task);
+extern void sve_sync_to_fpsimd(struct task_struct *task);
+extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
+
extern int sve_set_vector_length(struct task_struct *task,
unsigned long vl, unsigned long flags);
@@ -103,6 +110,10 @@ static void __maybe_unused sve_alloc(struct task_struct *task) { }
static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
struct task_struct const *src) { }
+static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
+static void __maybe_unused sve_sync_from_fpsimd_zeropad(
+ struct task_struct *task) { }
+
static void __maybe_unused sve_init_vq_map(void) { }
static void __maybe_unused sve_update_vq_map(void) { }
static int __maybe_unused sve_verify_vq_map(void) { return 0; }
diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
index d1ff83d..1915ab0 100644
--- a/arch/arm64/include/uapi/asm/ptrace.h
+++ b/arch/arm64/include/uapi/asm/ptrace.h
@@ -22,6 +22,7 @@
#include <linux/types.h>
#include <asm/hwcap.h>
+#include <asm/sigcontext.h>
/*
@@ -63,6 +64,8 @@
#ifndef __ASSEMBLY__
+#include <linux/prctl.h>
+
/*
* User structures for general purpose, floating point and debug registers.
*/
@@ -90,6 +93,138 @@ struct user_hwdebug_state {
} dbg_regs[16];
};
+/* SVE/FP/SIMD state (NT_ARM_SVE) */
+
+struct user_sve_header {
+ __u32 size; /* total meaningful regset content in bytes */
+ __u32 max_size; /* maxmium possible size for this thread */
+ __u16 vl; /* current vector length */
+ __u16 max_vl; /* maximum possible vector length */
+ __u16 flags;
+ __u16 __reserved;
+};
+
+/* Definitions for user_sve_header.flags: */
+#define SVE_PT_REGS_MASK (1 << 0)
+
+/* Flags: must be kept in sync with prctl interface in <linux/ptrace.h> */
+#define SVE_PT_REGS_FPSIMD 0
+#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
+
+#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
+#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
+
+
+/*
+ * The remainder of the SVE state follows struct user_sve_header. The
+ * total size of the SVE state (including header) depends on the
+ * metadata in the header: SVE_PT_SIZE(vq, flags) gives the total size
+ * of the state in bytes, including the header.
+ *
+ * Refer to <asm/sigcontext.h> for details of how to pass the correct
+ * "vq" argument to these macros.
+ */
+
+/* Offset from the start of struct user_sve_header to the register data */
+#define SVE_PT_REGS_OFFSET \
+ ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+/*
+ * The register data content and layout depends on the value of the
+ * flags field.
+ */
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD case:
+ *
+ * The payload starts at offset SVE_PT_FPSIMD_OFFSET, and is of type
+ * struct user_fpsimd_state. Additional data might be appended in the
+ * future: use SVE_PT_FPSIMD_SIZE(vq, flags) to compute the total size.
+ * SVE_PT_FPSIMD_SIZE(vq, flags) will never be less than
+ * sizeof(struct user_fpsimd_state).
+ */
+
+#define SVE_PT_FPSIMD_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_FPSIMD_SIZE(vq, flags) (sizeof(struct user_fpsimd_state))
+
+/*
+ * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE case:
+ *
+ * The payload starts at offset SVE_PT_SVE_OFFSET, and is of size
+ * SVE_PT_SVE_SIZE(vq, flags).
+ *
+ * Additional macros describe the contents and layout of the payload.
+ * For each, SVE_PT_SVE_x_OFFSET(args) is the start offset relative to
+ * the start of struct user_sve_header, and SVE_PT_SVE_x_SIZE(args) is
+ * the size in bytes:
+ *
+ * x type description
+ * - ---- -----------
+ * ZREGS \
+ * ZREG |
+ * PREGS | refer to <asm/sigcontext.h>
+ * PREG |
+ * FFR /
+ *
+ * FPSR uint32_t FPSR
+ * FPCR uint32_t FPCR
+ *
+ * Additional data might be appended in the future.
+ */
+
+#define SVE_PT_SVE_ZREG_SIZE(vq) SVE_SIG_ZREG_SIZE(vq)
+#define SVE_PT_SVE_PREG_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
+#define SVE_PT_SVE_FFR_SIZE(vq) SVE_SIG_FFR_SIZE(vq)
+#define SVE_PT_SVE_FPSR_SIZE sizeof(__u32)
+#define SVE_PT_SVE_FPCR_SIZE sizeof(__u32)
+
+#define __SVE_SIG_TO_PT(offset) \
+ ((offset) - SVE_SIG_REGS_OFFSET + SVE_PT_REGS_OFFSET)
+
+#define SVE_PT_SVE_OFFSET SVE_PT_REGS_OFFSET
+
+#define SVE_PT_SVE_ZREGS_OFFSET \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREGS_OFFSET)
+#define SVE_PT_SVE_ZREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_ZREG_OFFSET(vq, n))
+#define SVE_PT_SVE_ZREGS_SIZE(vq) \
+ (SVE_PT_SVE_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_PT_SVE_ZREGS_OFFSET)
+
+#define SVE_PT_SVE_PREGS_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREGS_OFFSET(vq))
+#define SVE_PT_SVE_PREG_OFFSET(vq, n) \
+ __SVE_SIG_TO_PT(SVE_SIG_PREG_OFFSET(vq, n))
+#define SVE_PT_SVE_PREGS_SIZE(vq) \
+ (SVE_PT_SVE_PREG_OFFSET(vq, SVE_NUM_PREGS) - \
+ SVE_PT_SVE_PREGS_OFFSET(vq))
+
+#define SVE_PT_SVE_FFR_OFFSET(vq) \
+ __SVE_SIG_TO_PT(SVE_SIG_FFR_OFFSET(vq))
+
+#define SVE_PT_SVE_FPSR_OFFSET(vq) \
+ ((SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq) + \
+ (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+#define SVE_PT_SVE_FPCR_OFFSET(vq) \
+ (SVE_PT_SVE_FPSR_OFFSET(vq) + SVE_PT_SVE_FPSR_SIZE)
+
+/*
+ * Any future extension appended after FPCR must be aligned to the next
+ * 128-bit boundary.
+ */
+
+#define SVE_PT_SVE_SIZE(vq, flags) \
+ ((SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE \
+ - SVE_PT_SVE_OFFSET + (SVE_VQ_BYTES - 1)) \
+ / SVE_VQ_BYTES * SVE_VQ_BYTES)
+
+#define SVE_PT_SIZE(vq, flags) \
+ (((flags) & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE ? \
+ SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, flags) \
+ : SVE_PT_FPSIMD_OFFSET + SVE_PT_FPSIMD_SIZE(vq, flags))
+
#endif /* __ASSEMBLY__ */
#endif /* _UAPI__ASM_PTRACE_H */
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index fff9fcf..361c019 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -303,6 +303,37 @@ void sve_alloc(struct task_struct *task)
BUG_ON(!task->thread.sve_state);
}
+void fpsimd_sync_to_sve(struct task_struct *task)
+{
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ fpsimd_to_sve(task);
+}
+
+void sve_sync_to_fpsimd(struct task_struct *task)
+{
+ if (test_tsk_thread_flag(task, TIF_SVE))
+ sve_to_fpsimd(task);
+}
+
+void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
+{
+ unsigned int vq;
+ void *sst = task->thread.sve_state;
+ struct fpsimd_state const *fst = &task->thread.fpsimd_state;
+ unsigned int i;
+
+ if (!test_tsk_thread_flag(task, TIF_SVE))
+ return;
+
+ vq = sve_vq_from_vl(task->thread.sve_vl);
+
+ memset(sst, 0, SVE_SIG_REGS_SIZE(vq));
+
+ for (i = 0; i < 32; ++i)
+ memcpy(ZREG(sst, vq, i), &fst->vregs[i],
+ sizeof(fst->vregs[i]));
+}
+
/*
* Handle SVE state across fork():
*
@@ -459,10 +490,17 @@ static void __init sve_efi_setup(void)
* This is evidence of a crippled system and we are returning void,
* so no attempt is made to handle this situation here.
*/
- BUG_ON(!sve_vl_valid(sve_max_vl));
+ if (!sve_vl_valid(sve_max_vl))
+ goto fail;
+
efi_sve_state = __alloc_percpu(
SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
if (!efi_sve_state)
+ goto fail;
+
+ return;
+
+fail:
panic("Cannot allocate percpu memory for EFI SVE save/restore");
}
diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
index 9cbb612..5ef4735b 100644
--- a/arch/arm64/kernel/ptrace.c
+++ b/arch/arm64/kernel/ptrace.c
@@ -32,6 +32,7 @@
#include <linux/security.h>
#include <linux/init.h>
#include <linux/signal.h>
+#include <linux/string.h>
#include <linux/uaccess.h>
#include <linux/perf_event.h>
#include <linux/hw_breakpoint.h>
@@ -40,6 +41,7 @@
#include <linux/elf.h>
#include <asm/compat.h>
+#include <asm/cpufeature.h>
#include <asm/debug-monitors.h>
#include <asm/pgtable.h>
#include <asm/stacktrace.h>
@@ -618,33 +620,66 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
/*
* TODO: update fp accessors for lazy context switching (sync/flush hwstate)
*/
-static int fpr_get(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- void *kbuf, void __user *ubuf)
+static int __fpr_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf, unsigned int start_pos)
{
struct user_fpsimd_state *uregs;
+
+ sve_sync_to_fpsimd(target);
+
uregs = &target->thread.fpsimd_state.user_fpsimd;
+ return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs,
+ start_pos, start_pos + sizeof(*uregs));
+}
+
+static int fpr_get(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
if (target == current)
fpsimd_preserve_current_state();
- return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf, 0);
}
-static int fpr_set(struct task_struct *target, const struct user_regset *regset,
- unsigned int pos, unsigned int count,
- const void *kbuf, const void __user *ubuf)
+static int __fpr_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf,
+ unsigned int start_pos)
{
int ret;
struct user_fpsimd_state newstate =
target->thread.fpsimd_state.user_fpsimd;
- ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
+ sve_sync_to_fpsimd(target);
+
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate,
+ start_pos, start_pos + sizeof(newstate));
if (ret)
return ret;
target->thread.fpsimd_state.user_fpsimd = newstate;
+
+ return ret;
+}
+
+static int fpr_set(struct task_struct *target, const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, 0);
+ if (ret)
+ return ret;
+
+ sve_sync_from_fpsimd_zeropad(target);
fpsimd_flush_task_state(target);
+
return ret;
}
@@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
return ret;
}
+#ifdef CONFIG_ARM64_SVE
+
+static void sve_init_header_from_task(struct user_sve_header *header,
+ struct task_struct *target)
+{
+ unsigned int vq;
+
+ memset(header, 0, sizeof(*header));
+
+ header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
+ SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
+ if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
+ header->flags |= SVE_PT_VL_INHERIT;
+
+ header->vl = target->thread.sve_vl;
+ vq = sve_vq_from_vl(header->vl);
+
+ if (WARN_ON(!sve_vl_valid(sve_max_vl)))
+ header->max_vl = header->vl;
+
+ header->size = SVE_PT_SIZE(vq, header->flags);
+ header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
+ SVE_PT_REGS_SVE);
+}
+
+static unsigned int sve_size_from_header(struct user_sve_header const *header)
+{
+ return ALIGN(header->size, SVE_VQ_BYTES);
+}
+
+static unsigned int sve_get_size(struct task_struct *target,
+ const struct user_regset *regset)
+{
+ struct user_sve_header header;
+
+ if (!system_supports_sve())
+ return 0;
+
+ sve_init_header_from_task(&header, target);
+ return sve_size_from_header(&header);
+}
+
+static int sve_get(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ void *kbuf, void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ sve_init_header_from_task(&header, target);
+ vq = sve_vq_from_vl(header.vl);
+
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ return ret;
+
+ if (target == current)
+ fpsimd_preserve_current_state();
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD)
+ return __fpr_get(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+
+ /* Otherwise: full SVE case */
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ return ret;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+ if (ret)
+ return ret;
+
+ start = end;
+ end = sve_size_from_header(&header);
+ return user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
+ start, end);
+}
+
+static int sve_set(struct task_struct *target,
+ const struct user_regset *regset,
+ unsigned int pos, unsigned int count,
+ const void *kbuf, const void __user *ubuf)
+{
+ int ret;
+ struct user_sve_header header;
+ unsigned int vq;
+ unsigned long start, end;
+
+ if (!system_supports_sve())
+ return -EINVAL;
+
+ /* Header */
+ if (count < sizeof(header))
+ return -EINVAL;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &header,
+ 0, sizeof(header));
+ if (ret)
+ goto out;
+
+ /*
+ * Apart from PT_SVE_REGS_MASK, all PT_SVE_* flags are consumed by
+ * sve_set_vector_length(), which will also validate them for us:
+ */
+ ret = sve_set_vector_length(target, header.vl,
+ header.flags & ~SVE_PT_REGS_MASK);
+ if (ret)
+ goto out;
+
+ /* Actual VL set may be less than the user asked for: */
+ vq = sve_vq_from_vl(target->thread.sve_vl);
+
+ /* Registers: FPSIMD-only case */
+
+ BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
+ if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD) {
+ sve_sync_to_fpsimd(target);
+
+ ret = __fpr_set(target, regset, pos, count, kbuf, ubuf,
+ SVE_PT_FPSIMD_OFFSET);
+ clear_tsk_thread_flag(target, TIF_SVE);
+ goto out;
+ }
+
+ /* Otherwise: full SVE case */
+
+ /*
+ * If setting a different VL from the requested VL and there is
+ * register data, the data layout will be wrong: don't even
+ * try to set the registers in this case.
+ */
+ if (count && vq != sve_vq_from_vl(header.vl)) {
+ ret = -EIO;
+ goto out;
+ }
+
+ sve_alloc(target);
+ fpsimd_sync_to_sve(target);
+ set_tsk_thread_flag(target, TIF_SVE);
+
+ BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
+ start = SVE_PT_SVE_OFFSET;
+ end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ target->thread.sve_state,
+ start, end);
+ if (ret)
+ goto out;
+
+ start = end;
+ end = SVE_PT_SVE_FPSR_OFFSET(vq);
+ ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
+ start, end);
+ if (ret)
+ goto out;
+
+ /*
+ * Copy fpsr, and fpcr which must follow contiguously in
+ * struct fpsimd_state:
+ */
+ start = end;
+ end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
+ &target->thread.fpsimd_state.fpsr,
+ start, end);
+
+out:
+ fpsimd_flush_task_state(target);
+ return ret;
+}
+
+#endif /* CONFIG_ARM64_SVE */
+
enum aarch64_regset {
REGSET_GPR,
REGSET_FPR,
@@ -711,6 +950,9 @@ enum aarch64_regset {
REGSET_HW_WATCH,
#endif
REGSET_SYSTEM_CALL,
+#ifdef CONFIG_ARM64_SVE
+ REGSET_SVE,
+#endif
};
static const struct user_regset aarch64_regsets[] = {
@@ -768,6 +1010,18 @@ static const struct user_regset aarch64_regsets[] = {
.get = system_call_get,
.set = system_call_set,
},
+#ifdef CONFIG_ARM64_SVE
+ [REGSET_SVE] = { /* Scalable Vector Extension */
+ .core_note_type = NT_ARM_SVE,
+ .n = DIV_ROUND_UP(SVE_PT_SIZE(SVE_VQ_MAX, SVE_PT_REGS_SVE),
+ SVE_VQ_BYTES),
+ .size = SVE_VQ_BYTES,
+ .align = SVE_VQ_BYTES,
+ .get = sve_get,
+ .set = sve_set,
+ .get_size = sve_get_size,
+ },
+#endif
};
static const struct user_regset_view user_aarch64_view = {
diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
index b5280db..735b8f4 100644
--- a/include/uapi/linux/elf.h
+++ b/include/uapi/linux/elf.h
@@ -416,6 +416,7 @@ typedef struct elf64_shdr {
#define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
#define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
#define NT_ARM_SYSTEM_CALL 0x404 /* ARM system call number */
+#define NT_ARM_SVE 0x405 /* ARM Scalable Vector Extension registers */
#define NT_METAG_CBUF 0x500 /* Metag catch buffer registers */
#define NT_METAG_RPIPE 0x501 /* Metag read pipeline state */
#define NT_METAG_TLS 0x502 /* Metag TLS pointer */
--
2.1.4
^ permalink raw reply related [flat|nested] 224+ messages in thread
* Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
2017-09-06 16:21 ` Okamoto, Takayuki
@ 2017-09-06 18:16 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-06 18:16 UTC (permalink / raw)
To: Okamoto, Takayuki
Cc: linux-arm-kernel, linux-arch, libc-alpha, gdb, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Yao Qi, Alan Hayward,
Will Deacon, Oleg Nesterov, Alexander Viro, Richard Sandiford,
Alex Bennée, kvmarm
On Wed, Sep 06, 2017 at 04:21:50PM +0000, Okamoto, Takayuki wrote:
> Hi Dave,
>
> I am an engineer of the postK computer from Fujitsu.
>
> When I tried to read "max_vl" by ptrace with this patch on our local SVE
> simulator, it was read as zero.
> I think the cause of this incident is that "max_vl" is set as "header->vl"
> only on warning case in sve_init_header_from_task().
> "max_vl" should be set up also on normal case, like the following patch.
>
>
> --- a/arch/arm64/kernel/ptrace.c
> +++ b/arch/arm64/kernel/ptrace.c
> @@ -755,6 +755,8 @@ static void sve_init_header_from_task(struct user_sve_header *header,
>
> if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> header->max_vl = header->vl;
> + else
> + header->max_vl = sve_max_vl;
>
> header->size = SVE_PT_SIZE(vq, header->flags);
> header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
Hi, thanks for reporting this.
It looks like a refactoring mistake I made while removing BUG_ON()s,
which I missed in my testing.
Your fix looks correct and seems to work. For stylistic reasons, I may
write it like this instead, but the effect should be the same:
header->max_vl = sve_max_vl;
if (WARN_ON(!sve_vl_valid(sve_max_vl))
header->max_vl = header->vl;
Cheers
---Dave
>
>
> Best regards,
> Takayuki Okamoto
>
> -----Original Message-----
> From: gdb-owner@sourceware.org [mailto:gdb-owner@sourceware.org] On Behalf Of Dave Martin
> Sent: Friday, September 1, 2017 2:01 AM
> To: linux-arm-kernel@lists.infradead.org
> Cc: Catalin Marinas <catalin.marinas@arm.com>; Will Deacon <will.deacon@arm.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Alex Bennée <alex.bennee@linaro.org>; Szabolcs Nagy <szabolcs.nagy@arm.com>; Richard Sandiford <richard.sandiford@arm.com>; kvmarm@lists.cs.columbia.edu; libc-alpha@sourceware.org; linux-arch@vger.kernel.org; gdb@sourceware.org; Alan Hayward <alan.hayward@arm.com>; Yao Qi <Yao.Qi@arm.com>; Oleg Nesterov <oleg@redhat.com>; Alexander Viro <viro@zeniv.linux.org.uk>
> Subject: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
>
[...]
> @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
> return ret;
> }
>
> +#ifdef CONFIG_ARM64_SVE
> +
> +static void sve_init_header_from_task(struct user_sve_header *header,
> + struct task_struct *target)
> +{
> + unsigned int vq;
> +
> + memset(header, 0, sizeof(*header));
> +
> + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> + header->flags |= SVE_PT_VL_INHERIT;
> +
> + header->vl = target->thread.sve_vl;
> + vq = sve_vq_from_vl(header->vl);
> +
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + header->max_vl = header->vl;
> +
> + header->size = SVE_PT_SIZE(vq, header->flags);
> + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> + SVE_PT_REGS_SVE);
> +}
[...]
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-06 18:16 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-06 18:16 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 06, 2017 at 04:21:50PM +0000, Okamoto, Takayuki wrote:
> Hi Dave,
>
> I am an engineer of the postK computer from Fujitsu.
>
> When I tried to read "max_vl" by ptrace with this patch on our local SVE
> simulator, it was read as zero.
> I think the cause of this incident is that "max_vl" is set as "header->vl"
> only on warning case in sve_init_header_from_task().
> "max_vl" should be set up also on normal case, like the following patch.
>
>
> --- a/arch/arm64/kernel/ptrace.c
> +++ b/arch/arm64/kernel/ptrace.c
> @@ -755,6 +755,8 @@ static void sve_init_header_from_task(struct user_sve_header *header,
>
> if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> header->max_vl = header->vl;
> + else
> + header->max_vl = sve_max_vl;
>
> header->size = SVE_PT_SIZE(vq, header->flags);
> header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
Hi, thanks for reporting this.
It looks like a refactoring mistake I made while removing BUG_ON()s,
which I missed in my testing.
Your fix looks correct and seems to work. For stylistic reasons, I may
write it like this instead, but the effect should be the same:
header->max_vl = sve_max_vl;
if (WARN_ON(!sve_vl_valid(sve_max_vl))
header->max_vl = header->vl;
Cheers
---Dave
>
>
> Best regards,
> Takayuki Okamoto
>
> -----Original Message-----
> From: gdb-owner at sourceware.org [mailto:gdb-owner at sourceware.org] On Behalf Of Dave Martin
> Sent: Friday, September 1, 2017 2:01 AM
> To: linux-arm-kernel at lists.infradead.org
> Cc: Catalin Marinas <catalin.marinas@arm.com>; Will Deacon <will.deacon@arm.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Alex Benn?e <alex.bennee@linaro.org>; Szabolcs Nagy <szabolcs.nagy@arm.com>; Richard Sandiford <richard.sandiford@arm.com>; kvmarm at lists.cs.columbia.edu; libc-alpha at sourceware.org; linux-arch at vger.kernel.org; gdb at sourceware.org; Alan Hayward <alan.hayward@arm.com>; Yao Qi <Yao.Qi@arm.com>; Oleg Nesterov <oleg@redhat.com>; Alexander Viro <viro@zeniv.linux.org.uk>
> Subject: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
>
[...]
> @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
> return ret;
> }
>
> +#ifdef CONFIG_ARM64_SVE
> +
> +static void sve_init_header_from_task(struct user_sve_header *header,
> + struct task_struct *target)
> +{
> + unsigned int vq;
> +
> + memset(header, 0, sizeof(*header));
> +
> + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> + header->flags |= SVE_PT_VL_INHERIT;
> +
> + header->vl = target->thread.sve_vl;
> + vq = sve_vq_from_vl(header->vl);
> +
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + header->max_vl = header->vl;
> +
> + header->size = SVE_PT_SIZE(vq, header->flags);
> + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> + SVE_PT_REGS_SVE);
> +}
[...]
^ permalink raw reply [flat|nested] 224+ messages in thread
* RE: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-07 5:11 ` Okamoto, Takayuki
0 siblings, 0 replies; 224+ messages in thread
From: Okamoto, Takayuki @ 2017-09-07 5:11 UTC (permalink / raw)
To: 'Dave Martin'
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy, gdb,
Yao Qi, Will Deacon, Oleg Nesterov, Richard Sandiford,
Alexander Viro, Alan Hayward, Catalin Marinas, kvmarm,
linux-arm-kernel
Hi Dave,
Thank you for your reply.
> Your fix looks correct and seems to work. For stylistic reasons, I may
> write it like this instead, but the effect should be the same:
>
> header->max_vl = sve_max_vl;
> if (WARN_ON(!sve_vl_valid(sve_max_vl))
> header->max_vl = header->vl;
It is better than my fix.
Please, apply it at next version.
Best regards,
Takayuki Okamoto
> -----Original Message-----
> From: linux-arm-kernel
> [mailto:linux-arm-kernel-bounces@lists.infradead.org] On Behalf Of Dave
> Martin
> Sent: Thursday, September 7, 2017 3:17 AM
> To: Okamoto, Takayuki <tokamoto@jp.fujitsu.com>
> Cc: linux-arch@vger.kernel.org; libc-alpha@sourceware.org; Ard
> Biesheuvel <ard.biesheuvel@linaro.org>; Szabolcs Nagy
> <szabolcs.nagy@arm.com>; gdb@sourceware.org; Yao Qi <Yao.Qi@arm.com>;
> Alan Hayward <alan.hayward@arm.com>; Will Deacon <will.deacon@arm.com>;
> Oleg Nesterov <oleg@redhat.com>; Richard Sandiford
> <richard.sandiford@arm.com>; Alexander Viro <viro@zeniv.linux.org.uk>;
> Catalin Marinas <catalin.marinas@arm.com>; Alex Bennée
> <alex.bennee@linaro.org>; kvmarm@lists.cs.columbia.edu;
> linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
>
> On Wed, Sep 06, 2017 at 04:21:50PM +0000, Okamoto, Takayuki wrote:
> > Hi Dave,
> >
> > I am an engineer of the postK computer from Fujitsu.
> >
> > When I tried to read "max_vl" by ptrace with this patch on our local SVE
> > simulator, it was read as zero.
> > I think the cause of this incident is that "max_vl" is set as "header->vl"
> > only on warning case in sve_init_header_from_task().
> > "max_vl" should be set up also on normal case, like the following patch.
> >
> >
> > --- a/arch/arm64/kernel/ptrace.c
> > +++ b/arch/arm64/kernel/ptrace.c
> > @@ -755,6 +755,8 @@ static void sve_init_header_from_task(struct
> user_sve_header *header,
> >
> > if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> > header->max_vl = header->vl;
> > + else
> > + header->max_vl = sve_max_vl;
> >
> > header->size = SVE_PT_SIZE(vq, header->flags);
> > header->max_size =
> SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
>
> Hi, thanks for reporting this.
>
> It looks like a refactoring mistake I made while removing BUG_ON()s,
> which I missed in my testing.
>
> Your fix looks correct and seems to work. For stylistic reasons, I may
> write it like this instead, but the effect should be the same:
>
> header->max_vl = sve_max_vl;
> if (WARN_ON(!sve_vl_valid(sve_max_vl))
> header->max_vl = header->vl;
>
> Cheers
> ---Dave
>
> >
> >
> > Best regards,
> > Takayuki Okamoto
> >
> > -----Original Message-----
> > From: gdb-owner@sourceware.org [mailto:gdb-owner@sourceware.org] On
> Behalf Of Dave Martin
> > Sent: Friday, September 1, 2017 2:01 AM
> > To: linux-arm-kernel@lists.infradead.org
> > Cc: Catalin Marinas <catalin.marinas@arm.com>; Will Deacon
> <will.deacon@arm.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Alex
> Bennée <alex.bennee@linaro.org>; Szabolcs Nagy <szabolcs.nagy@arm.com>;
> Richard Sandiford <richard.sandiford@arm.com>;
> kvmarm@lists.cs.columbia.edu; libc-alpha@sourceware.org;
> linux-arch@vger.kernel.org; gdb@sourceware.org; Alan Hayward
> <alan.hayward@arm.com>; Yao Qi <Yao.Qi@arm.com>; Oleg Nesterov
> <oleg@redhat.com>; Alexander Viro <viro@zeniv.linux.org.uk>
> > Subject: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
> >
>
> [...]
>
> > @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct
> *target,
> > return ret;
> > }
> >
> > +#ifdef CONFIG_ARM64_SVE
> > +
> > +static void sve_init_header_from_task(struct user_sve_header *header,
> > + struct task_struct *target)
> > +{
> > + unsigned int vq;
> > +
> > + memset(header, 0, sizeof(*header));
> > +
> > + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> > + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> > + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> > + header->flags |= SVE_PT_VL_INHERIT;
> > +
> > + header->vl = target->thread.sve_vl;
> > + vq = sve_vq_from_vl(header->vl);
> > +
> > + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> > + header->max_vl = header->vl;
> > +
> > + header->size = SVE_PT_SIZE(vq, header->flags);
> > + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> > + SVE_PT_REGS_SVE);
> > +}
>
> [...]
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 224+ messages in thread
* RE: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-07 5:11 ` Okamoto, Takayuki
0 siblings, 0 replies; 224+ messages in thread
From: Okamoto, Takayuki @ 2017-09-07 5:11 UTC (permalink / raw)
To: 'Dave Martin'
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy, gdb,
Yao Qi, Alan Hayward, Will Deacon, Oleg Nesterov,
Richard Sandiford, Alexander Viro, Catalin Marinas,
Alex Bennée, kvmarm, linux-arm-kernel
Hi Dave,
Thank you for your reply.
> Your fix looks correct and seems to work. For stylistic reasons, I may
> write it like this instead, but the effect should be the same:
>
> header->max_vl = sve_max_vl;
> if (WARN_ON(!sve_vl_valid(sve_max_vl))
> header->max_vl = header->vl;
It is better than my fix.
Please, apply it at next version.
Best regards,
Takayuki Okamoto
> -----Original Message-----
> From: linux-arm-kernel
> [mailto:linux-arm-kernel-bounces@lists.infradead.org] On Behalf Of Dave
> Martin
> Sent: Thursday, September 7, 2017 3:17 AM
> To: Okamoto, Takayuki <tokamoto@jp.fujitsu.com>
> Cc: linux-arch@vger.kernel.org; libc-alpha@sourceware.org; Ard
> Biesheuvel <ard.biesheuvel@linaro.org>; Szabolcs Nagy
> <szabolcs.nagy@arm.com>; gdb@sourceware.org; Yao Qi <Yao.Qi@arm.com>;
> Alan Hayward <alan.hayward@arm.com>; Will Deacon <will.deacon@arm.com>;
> Oleg Nesterov <oleg@redhat.com>; Richard Sandiford
> <richard.sandiford@arm.com>; Alexander Viro <viro@zeniv.linux.org.uk>;
> Catalin Marinas <catalin.marinas@arm.com>; Alex Bennée
> <alex.bennee@linaro.org>; kvmarm@lists.cs.columbia.edu;
> linux-arm-kernel@lists.infradead.org
> Subject: Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
>
> On Wed, Sep 06, 2017 at 04:21:50PM +0000, Okamoto, Takayuki wrote:
> > Hi Dave,
> >
> > I am an engineer of the postK computer from Fujitsu.
> >
> > When I tried to read "max_vl" by ptrace with this patch on our local SVE
> > simulator, it was read as zero.
> > I think the cause of this incident is that "max_vl" is set as "header->vl"
> > only on warning case in sve_init_header_from_task().
> > "max_vl" should be set up also on normal case, like the following patch.
> >
> >
> > --- a/arch/arm64/kernel/ptrace.c
> > +++ b/arch/arm64/kernel/ptrace.c
> > @@ -755,6 +755,8 @@ static void sve_init_header_from_task(struct
> user_sve_header *header,
> >
> > if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> > header->max_vl = header->vl;
> > + else
> > + header->max_vl = sve_max_vl;
> >
> > header->size = SVE_PT_SIZE(vq, header->flags);
> > header->max_size =
> SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
>
> Hi, thanks for reporting this.
>
> It looks like a refactoring mistake I made while removing BUG_ON()s,
> which I missed in my testing.
>
> Your fix looks correct and seems to work. For stylistic reasons, I may
> write it like this instead, but the effect should be the same:
>
> header->max_vl = sve_max_vl;
> if (WARN_ON(!sve_vl_valid(sve_max_vl))
> header->max_vl = header->vl;
>
> Cheers
> ---Dave
>
> >
> >
> > Best regards,
> > Takayuki Okamoto
> >
> > -----Original Message-----
> > From: gdb-owner@sourceware.org [mailto:gdb-owner@sourceware.org] On
> Behalf Of Dave Martin
> > Sent: Friday, September 1, 2017 2:01 AM
> > To: linux-arm-kernel@lists.infradead.org
> > Cc: Catalin Marinas <catalin.marinas@arm.com>; Will Deacon
> <will.deacon@arm.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Alex
> Bennée <alex.bennee@linaro.org>; Szabolcs Nagy <szabolcs.nagy@arm.com>;
> Richard Sandiford <richard.sandiford@arm.com>;
> kvmarm@lists.cs.columbia.edu; libc-alpha@sourceware.org;
> linux-arch@vger.kernel.org; gdb@sourceware.org; Alan Hayward
> <alan.hayward@arm.com>; Yao Qi <Yao.Qi@arm.com>; Oleg Nesterov
> <oleg@redhat.com>; Alexander Viro <viro@zeniv.linux.org.uk>
> > Subject: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
> >
>
> [...]
>
> > @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct
> *target,
> > return ret;
> > }
> >
> > +#ifdef CONFIG_ARM64_SVE
> > +
> > +static void sve_init_header_from_task(struct user_sve_header *header,
> > + struct task_struct *target)
> > +{
> > + unsigned int vq;
> > +
> > + memset(header, 0, sizeof(*header));
> > +
> > + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> > + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> > + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> > + header->flags |= SVE_PT_VL_INHERIT;
> > +
> > + header->vl = target->thread.sve_vl;
> > + vq = sve_vq_from_vl(header->vl);
> > +
> > + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> > + header->max_vl = header->vl;
> > +
> > + header->size = SVE_PT_SIZE(vq, header->flags);
> > + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> > + SVE_PT_REGS_SVE);
> > +}
>
> [...]
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-07 5:11 ` Okamoto, Takayuki
0 siblings, 0 replies; 224+ messages in thread
From: Okamoto, Takayuki @ 2017-09-07 5:11 UTC (permalink / raw)
To: linux-arm-kernel
Hi Dave,
Thank you for your reply.
> Your fix looks correct and seems to work. For stylistic reasons, I may
> write it like this instead, but the effect should be the same:
>
> header->max_vl = sve_max_vl;
> if (WARN_ON(!sve_vl_valid(sve_max_vl))
> header->max_vl = header->vl;
It is better than my fix.
Please, apply it at next version.
Best regards,
Takayuki Okamoto
> -----Original Message-----
> From: linux-arm-kernel
> [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Dave
> Martin
> Sent: Thursday, September 7, 2017 3:17 AM
> To: Okamoto, Takayuki <tokamoto@jp.fujitsu.com>
> Cc: linux-arch at vger.kernel.org; libc-alpha at sourceware.org; Ard
> Biesheuvel <ard.biesheuvel@linaro.org>; Szabolcs Nagy
> <szabolcs.nagy@arm.com>; gdb at sourceware.org; Yao Qi <Yao.Qi@arm.com>;
> Alan Hayward <alan.hayward@arm.com>; Will Deacon <will.deacon@arm.com>;
> Oleg Nesterov <oleg@redhat.com>; Richard Sandiford
> <richard.sandiford@arm.com>; Alexander Viro <viro@zeniv.linux.org.uk>;
> Catalin Marinas <catalin.marinas@arm.com>; Alex Benn?e
> <alex.bennee@linaro.org>; kvmarm at lists.cs.columbia.edu;
> linux-arm-kernel at lists.infradead.org
> Subject: Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
>
> On Wed, Sep 06, 2017 at 04:21:50PM +0000, Okamoto, Takayuki wrote:
> > Hi Dave,
> >
> > I am an engineer of the postK computer from Fujitsu.
> >
> > When I tried to read "max_vl" by ptrace with this patch on our local SVE
> > simulator, it was read as zero.
> > I think the cause of this incident is that "max_vl" is set as "header->vl"
> > only on warning case in sve_init_header_from_task().
> > "max_vl" should be set up also on normal case, like the following patch.
> >
> >
> > --- a/arch/arm64/kernel/ptrace.c
> > +++ b/arch/arm64/kernel/ptrace.c
> > @@ -755,6 +755,8 @@ static void sve_init_header_from_task(struct
> user_sve_header *header,
> >
> > if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> > header->max_vl = header->vl;
> > + else
> > + header->max_vl = sve_max_vl;
> >
> > header->size = SVE_PT_SIZE(vq, header->flags);
> > header->max_size =
> SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
>
> Hi, thanks for reporting this.
>
> It looks like a refactoring mistake I made while removing BUG_ON()s,
> which I missed in my testing.
>
> Your fix looks correct and seems to work. For stylistic reasons, I may
> write it like this instead, but the effect should be the same:
>
> header->max_vl = sve_max_vl;
> if (WARN_ON(!sve_vl_valid(sve_max_vl))
> header->max_vl = header->vl;
>
> Cheers
> ---Dave
>
> >
> >
> > Best regards,
> > Takayuki Okamoto
> >
> > -----Original Message-----
> > From: gdb-owner at sourceware.org [mailto:gdb-owner at sourceware.org] On
> Behalf Of Dave Martin
> > Sent: Friday, September 1, 2017 2:01 AM
> > To: linux-arm-kernel at lists.infradead.org
> > Cc: Catalin Marinas <catalin.marinas@arm.com>; Will Deacon
> <will.deacon@arm.com>; Ard Biesheuvel <ard.biesheuvel@linaro.org>; Alex
> Benn?e <alex.bennee@linaro.org>; Szabolcs Nagy <szabolcs.nagy@arm.com>;
> Richard Sandiford <richard.sandiford@arm.com>;
> kvmarm at lists.cs.columbia.edu; libc-alpha at sourceware.org;
> linux-arch at vger.kernel.org; gdb at sourceware.org; Alan Hayward
> <alan.hayward@arm.com>; Yao Qi <Yao.Qi@arm.com>; Oleg Nesterov
> <oleg@redhat.com>; Alexander Viro <viro@zeniv.linux.org.uk>
> > Subject: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
> >
>
> [...]
>
> > @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct
> *target,
> > return ret;
> > }
> >
> > +#ifdef CONFIG_ARM64_SVE
> > +
> > +static void sve_init_header_from_task(struct user_sve_header *header,
> > + struct task_struct *target)
> > +{
> > + unsigned int vq;
> > +
> > + memset(header, 0, sizeof(*header));
> > +
> > + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> > + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> > + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> > + header->flags |= SVE_PT_VL_INHERIT;
> > +
> > + header->vl = target->thread.sve_vl;
> > + vq = sve_vq_from_vl(header->vl);
> > +
> > + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> > + header->max_vl = header->vl;
> > +
> > + header->size = SVE_PT_SIZE(vq, header->flags);
> > + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> > + SVE_PT_REGS_SVE);
> > +}
>
> [...]
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
2017-09-07 5:11 ` Okamoto, Takayuki
@ 2017-09-08 13:11 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-08 13:11 UTC (permalink / raw)
To: Okamoto, Takayuki
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy, gdb,
Yao Qi, Will Deacon, Oleg Nesterov, Alex Bennée,
Richard Sandiford, Alexander Viro, Alan Hayward, Catalin Marinas,
kvmarm, linux-arm-kernel
On Thu, Sep 07, 2017 at 05:11:45AM +0000, Okamoto, Takayuki wrote:
> Hi Dave,
>
> Thank you for your reply.
>
> > Your fix looks correct and seems to work. For stylistic reasons, I may
> > write it like this instead, but the effect should be the same:
> >
> > header->max_vl = sve_max_vl;
> > if (WARN_ON(!sve_vl_valid(sve_max_vl))
> > header->max_vl = header->vl;
>
> It is better than my fix.
> Please, apply it at next version.
I've rebased to v4.13 and pushed a branch to track fixes against v2,
here:
* http://linux-arm.org/git?p=linux-dm.git;a=shortlog;h=refs/heads/sve/v2%2Bfixes
* git://linux-arm.org/linux-dm.git sve/v2+fixes
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-08 13:11 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-08 13:11 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Sep 07, 2017 at 05:11:45AM +0000, Okamoto, Takayuki wrote:
> Hi Dave,
>
> Thank you for your reply.
>
> > Your fix looks correct and seems to work. For stylistic reasons, I may
> > write it like this instead, but the effect should be the same:
> >
> > header->max_vl = sve_max_vl;
> > if (WARN_ON(!sve_vl_valid(sve_max_vl))
> > header->max_vl = header->vl;
>
> It is better than my fix.
> Please, apply it at next version.
I've rebased to v4.13 and pushed a branch to track fixes against v2,
here:
* http://linux-arm.org/git?p=linux-dm.git;a=shortlog;h=refs/heads/sve/v2%2Bfixes
* git://linux-arm.org/linux-dm.git sve/v2+fixes
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 10/28] arm64/sve: Low-level CPU setup
2017-08-31 17:00 ` Dave Martin
@ 2017-09-13 13:32 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 13:32 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, linux-arch, libc-alpha, Ard Biesheuvel,
Szabolcs Nagy, Will Deacon, Richard Sandiford, Alex Bennée,
kvmarm
On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index 877d42f..dd22ef2 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -27,6 +27,7 @@
> #include <asm/pgtable-hwdef.h>
> #include <asm/cpufeature.h>
> #include <asm/alternative.h>
> +#include <asm/sysreg.h>
>
> #ifdef CONFIG_ARM64_64K_PAGES
> #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> tlbi vmalle1 // Invalidate local TLB
> dsb nsh
>
> - mov x0, #3 << 20
> - msr cpacr_el1, x0 // Enable FP/ASIMD
> + mov x0, #3 << 20 // FEN
> +
> + /* SVE */
> + mrs x5, id_aa64pfr0_el1
> + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> + cbz x5, 1f
> +
> + bic x0, x0, #CPACR_EL1_ZEN
> + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> +1: msr cpacr_el1, x0 // Enable FP/ASIMD
For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
I tried to do the same with FPSIMD but hit an issue with EFI run-time
services (I may be wrong though).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-09-13 13:32 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 13:32 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> index 877d42f..dd22ef2 100644
> --- a/arch/arm64/mm/proc.S
> +++ b/arch/arm64/mm/proc.S
> @@ -27,6 +27,7 @@
> #include <asm/pgtable-hwdef.h>
> #include <asm/cpufeature.h>
> #include <asm/alternative.h>
> +#include <asm/sysreg.h>
>
> #ifdef CONFIG_ARM64_64K_PAGES
> #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> tlbi vmalle1 // Invalidate local TLB
> dsb nsh
>
> - mov x0, #3 << 20
> - msr cpacr_el1, x0 // Enable FP/ASIMD
> + mov x0, #3 << 20 // FEN
> +
> + /* SVE */
> + mrs x5, id_aa64pfr0_el1
> + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> + cbz x5, 1f
> +
> + bic x0, x0, #CPACR_EL1_ZEN
> + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> +1: msr cpacr_el1, x0 // Enable FP/ASIMD
For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
I tried to do the same with FPSIMD but hit an issue with EFI run-time
services (I may be wrong though).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 09/28] arm64/sve: Signal frame and context structure definition
2017-08-31 17:00 ` Dave Martin
@ 2017-09-13 13:36 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 13:36 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, linux-arch, libc-alpha, gdb, Ard Biesheuvel,
Szabolcs Nagy, Yao Qi, Alan Hayward, Will Deacon,
Richard Sandiford, Alex Bennée, kvmarm
On Thu, Aug 31, 2017 at 06:00:41PM +0100, Dave P Martin wrote:
> +/*
> + * The SVE architecture leaves space for future expansion of the
> + * vector length beyond its initial architectural limit of 2048 bits
> + * (16 quadwords).
> + */
> +#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
> +
> +#define SVE_VQ_MIN 1
> +#define SVE_VQ_MAX 0x200
Just a nitpick (up to you): could you use 16 and 512 here instead of
hex? I usually associate hex numbers with some bit fields.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 09/28] arm64/sve: Signal frame and context structure definition
@ 2017-09-13 13:36 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 13:36 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Aug 31, 2017 at 06:00:41PM +0100, Dave P Martin wrote:
> +/*
> + * The SVE architecture leaves space for future expansion of the
> + * vector length beyond its initial architectural limit of 2048 bits
> + * (16 quadwords).
> + */
> +#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
> +
> +#define SVE_VQ_MIN 1
> +#define SVE_VQ_MAX 0x200
Just a nitpick (up to you): could you use 16 and 512 here instead of
hex? I usually associate hex numbers with some bit fields.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-08-31 17:00 ` Dave Martin
@ 2017-09-13 14:33 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 14:33 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, linux-arch, libc-alpha, Ard Biesheuvel,
Szabolcs Nagy, Will Deacon, Richard Sandiford, Alex Bennée,
kvmarm
On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> +/*
> + * Handle SVE state across fork():
> + *
> + * dst and src must not end up with aliases of the same sve_state.
> + * Because a task cannot fork except in a syscall, we can discard SVE
> + * state for dst here, so long as we take care to retain the FPSIMD
> + * subset of the state if SVE is in use. Reallocation of the SVE state
> + * will be deferred until dst tries to use SVE.
> + */
> +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> +{
> + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> + sve_to_fpsimd(dst);
> + }
> +
> + dst->thread.sve_state = NULL;
> +}
I first thought the thread flags are not visible in dst yet since
dup_task_struct() calls arch_dup_task_struct() before
setup_thread_stack(). However, at the end of the last year we enabled
CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
on this.
Anyway, IIUC we don't need sve_to_fpsimd() here. The
arch_dup_task_struct() already called fpsimd_preserve_current_state()
for src, so the FPSIMD state (which we care about) is transferred during
the *dst = *src assignment. So you'd only need the last statement,
possibly with a different function name like fpsimd_erase_sve (and maybe
make the function static inline in the header).
[...]
> int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> @@ -246,6 +247,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> if (current->mm)
> fpsimd_preserve_current_state();
> *dst = *src;
> +
> + fpsimd_dup_sve(dst, src);
> +
> return 0;
> }
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-13 14:33 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 14:33 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> +/*
> + * Handle SVE state across fork():
> + *
> + * dst and src must not end up with aliases of the same sve_state.
> + * Because a task cannot fork except in a syscall, we can discard SVE
> + * state for dst here, so long as we take care to retain the FPSIMD
> + * subset of the state if SVE is in use. Reallocation of the SVE state
> + * will be deferred until dst tries to use SVE.
> + */
> +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> +{
> + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> + sve_to_fpsimd(dst);
> + }
> +
> + dst->thread.sve_state = NULL;
> +}
I first thought the thread flags are not visible in dst yet since
dup_task_struct() calls arch_dup_task_struct() before
setup_thread_stack(). However, at the end of the last year we enabled
CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
on this.
Anyway, IIUC we don't need sve_to_fpsimd() here. The
arch_dup_task_struct() already called fpsimd_preserve_current_state()
for src, so the FPSIMD state (which we care about) is transferred during
the *dst = *src assignment. So you'd only need the last statement,
possibly with a different function name like fpsimd_erase_sve (and maybe
make the function static inline in the header).
[...]
> int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> @@ -246,6 +247,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> if (current->mm)
> fpsimd_preserve_current_state();
> *dst = *src;
> +
> + fpsimd_dup_sve(dst, src);
> +
> return 0;
> }
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests
@ 2017-09-13 14:37 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 14:37 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> Currently, a guest kernel sees the true CPU feature registers
> (ID_*_EL1) when it reads them using MRS instructions. This means
> that the guest will observe features that are present in the
> hardware but the host doesn't understand or doesn't provide support
> for. A guest may legimitately try to use such a feature as per the
> architecture, but use of the feature may trap instead of working
> normally, triggering undef injection into the guest.
>
> This is not a problem for the host, but the guest may go wrong when
> running on newer hardware than the host knows about.
>
> This patch hides from guest VMs any AArch64-specific CPU features
> that the host doesn't support, by exposing to the guest the
> sanitised versions of the registers computed by the cpufeatures
> framework, instead of the true hardware registers. To achieve
> this, HCR_EL2.TID3 is now set for AArch64 guests, and emulation
> code is added to KVM to report the sanitised versions of the
> affected registers in response to MRS and register reads from
> userspace.
>
> The affected registers are removed from invariant_sys_regs[] (since
> the invariant_sys_regs handling is no longer quite correct for
> them) and added to sys_reg_desgs[], with appropriate access(),
> get_user() and set_user() methods. No runtime vcpu storage is
> allocated for the registers: instead, they are read on demand from
> the cpufeatures framework. This may need modification in the
> future if there is a need for userspace to customise the features
> visible to the guest.
>
> Attempts by userspace to write the registers are handled similarly
> to the current invariant_sys_regs handling: writes are permitted,
> but only if they don't attempt to change the value. This is
> sufficient to support VM snapshot/restore from userspace.
>
> Because of the additional registers, restoring a VM on an older
> kernel may not work unless userspace knows how to handle the extra
> VM registers exposed to the KVM user ABI by this patch.
>
> Under the principle of least damage, this patch makes no attempt to
> handle any of the other registers currently in
> invariant_sys_regs[], or to emulate registers for AArch32: however,
> these could be handled in a similar way in future, as necessary.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Get rid of ternary operator use in walk_sys_regs().
>
> * Call write_to_read_only() if an attempt to write an ID reg is
> trapped, rather than reinventing.
> Probably we won't get there anyway: the architecture says that this
> should undef at EL1 instead.
>
> * Make ID register sysreg table less cryptic and spread the entries one
> per line.
> Also, make the architecturally unallocated and allocated but hidden
> cases more clearly distinct. These require the same behaviour but for
> different reasons, so it's better to identify them as separate.
>
> Other:
>
> * Delete BUG_ON()s that are skipped by construction:
> These check that the result of sys_reg_to_index() is a 64-bit
> register, which is always true because sys_reg_to_index()
> explicitly sets this.
>
> * Remove duplicate const in __access_id_reg args [sparse]
> ---
> arch/arm64/include/asm/sysreg.h | 3 +
> arch/arm64/kvm/hyp/switch.c | 6 +
> arch/arm64/kvm/sys_regs.c | 282 +++++++++++++++++++++++++++++++++-------
> 3 files changed, 246 insertions(+), 45 deletions(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index f707fed..480ecd6 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -149,6 +149,9 @@
> #define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
> #define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
>
> +#define SYS_ID_AA64AFR0_EL1 sys_reg(3, 0, 0, 5, 4)
> +#define SYS_ID_AA64AFR1_EL1 sys_reg(3, 0, 0, 5, 5)
> +
> #define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)
> #define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 945e79c..35a90b8 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -81,11 +81,17 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> * it will cause an exception.
> */
> val = vcpu->arch.hcr_el2;
> +
> if (!(val & HCR_RW) && system_supports_fpsimd()) {
> write_sysreg(1 << 30, fpexc32_el2);
> isb();
> }
> +
> + if (val & HCR_RW) /* for AArch64 only: */
> + val |= HCR_TID3; /* TID3: trap feature register accesses */
> +
I wondered as this is the hyp switch can we make use of testing val &
HCR_RW for both this and above. But it seems minimal in the generated
code so probably not.
> write_sysreg(val, hcr_el2);
> +
> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> write_sysreg(1 << 15, hstr_el2);
> /*
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 2e070d3..b1f7552 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -892,6 +892,137 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
> return true;
> }
>
> +/* Read a sanitised cpufeature ID register by sys_reg_desc */
> +static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
> +{
> + u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
> + (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> +
> + return raz ? 0 : read_sanitised_ftr_reg(id);
> +}
> +
> +/* cpufeature ID register access trap handlers */
> +
> +static bool __access_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r,
> + bool raz)
> +{
> + if (p->is_write)
> + return write_to_read_only(vcpu, p, r);
> +
> + p->regval = read_id_reg(r, raz);
> + return true;
> +}
> +
> +static bool access_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + return __access_id_reg(vcpu, p, r, false);
> +}
> +
> +static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + return __access_id_reg(vcpu, p, r, true);
> +}
> +
> +static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
> +static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
> +static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
> +
> +/*
> + * cpufeature ID register user accessors
> + *
> + * For now, these registers are immutable for userspace, so no values
> + * are stored, and for set_id_reg() we don't allow the effective value
> + * to be changed.
> + */
> +static int __get_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
> + bool raz)
> +{
> + const u64 id = sys_reg_to_index(rd);
> + const u64 val = read_id_reg(rd, raz);
> +
> + return reg_to_user(uaddr, &val, id);
> +}
> +
> +static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
> + bool raz)
> +{
> + const u64 id = sys_reg_to_index(rd);
> + int err;
> + u64 val;
> +
> + err = reg_from_user(&val, uaddr, id);
> + if (err)
> + return err;
> +
> + /* This is what we mean by invariant: you can't change it. */
> + if (val != read_id_reg(rd, raz))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __get_id_reg(rd, uaddr, false);
> +}
> +
> +static int set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __set_id_reg(rd, uaddr, false);
> +}
> +
> +static int get_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __get_id_reg(rd, uaddr, true);
> +}
> +
> +static int set_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __set_id_reg(rd, uaddr, true);
> +}
> +
> +/* sys_reg_desc initialiser for known cpufeature ID registers */
> +#define ID_SANITISED(name) { \
> + SYS_DESC(SYS_##name), \
> + .access = access_id_reg, \
> + .get_user = get_id_reg, \
> + .set_user = set_id_reg, \
> +}
> +
> +/*
> + * sys_reg_desc initialiser for architecturally unallocated cpufeature ID
> + * register with encoding Op0=3, Op1=0, CRn=0, CRm=crm, Op2=op2
> + * (1 <= crm < 8, 0 <= Op2 < 8).
> + */
> +#define ID_UNALLOCATED(crm, op2) { \
> + Op0(3), Op1(0), CRn(0), CRm(crm), Op2(op2), \
> + .access = access_raz_id_reg, \
> + .get_user = get_raz_id_reg, \
> + .set_user = set_raz_id_reg, \
> +}
> +
> +/*
> + * sys_reg_desc initialiser for known ID registers that we hide from guests.
> + * For now, these are exposed just like unallocated ID regs: they appear
> + * RAZ for the guest.
> + */
> +#define ID_HIDDEN(name) { \
> + SYS_DESC(SYS_##name), \
> + .access = access_raz_id_reg, \
> + .get_user = get_raz_id_reg, \
> + .set_user = set_raz_id_reg, \
> +}
> +
> /*
> * Architected system registers.
> * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
> @@ -944,6 +1075,84 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> { SYS_DESC(SYS_DBGVCR32_EL2), NULL, reset_val, DBGVCR32_EL2, 0 },
>
> { SYS_DESC(SYS_MPIDR_EL1), NULL, reset_mpidr, MPIDR_EL1 },
> +
> + /*
> + * ID regs: all ID_SANITISED() entries here must have corresponding
> + * entries in arm64_ftr_regs[].
> + */
arm64_ftr_regs isn't updated in this commit. Does this break bisection?
> +
> + /* AArch64 mappings of the AArch32 ID registers */
> + /* CRm=1 */
> + ID_SANITISED(ID_PFR0_EL1),
> + ID_SANITISED(ID_PFR1_EL1),
> + ID_SANITISED(ID_DFR0_EL1),
> + ID_HIDDEN(ID_AFR0_EL1),
> + ID_SANITISED(ID_MMFR0_EL1),
> + ID_SANITISED(ID_MMFR1_EL1),
> + ID_SANITISED(ID_MMFR2_EL1),
> + ID_SANITISED(ID_MMFR3_EL1),
> +
> + /* CRm=2 */
> + ID_SANITISED(ID_ISAR0_EL1),
> + ID_SANITISED(ID_ISAR1_EL1),
> + ID_SANITISED(ID_ISAR2_EL1),
> + ID_SANITISED(ID_ISAR3_EL1),
> + ID_SANITISED(ID_ISAR4_EL1),
> + ID_SANITISED(ID_ISAR5_EL1),
> + ID_SANITISED(ID_MMFR4_EL1),
> + ID_UNALLOCATED(2,7),
> +
> + /* CRm=3 */
> + ID_SANITISED(MVFR0_EL1),
> + ID_SANITISED(MVFR1_EL1),
> + ID_SANITISED(MVFR2_EL1),
> + ID_UNALLOCATED(3,3),
> + ID_UNALLOCATED(3,4),
> + ID_UNALLOCATED(3,5),
> + ID_UNALLOCATED(3,6),
> + ID_UNALLOCATED(3,7),
> +
> + /* AArch64 ID registers */
> + /* CRm=4 */
> + ID_SANITISED(ID_AA64PFR0_EL1),
> + ID_SANITISED(ID_AA64PFR1_EL1),
> + ID_UNALLOCATED(4,2),
> + ID_UNALLOCATED(4,3),
> + ID_UNALLOCATED(4,4),
> + ID_UNALLOCATED(4,5),
> + ID_UNALLOCATED(4,6),
> + ID_UNALLOCATED(4,7),
> +
> + /* CRm=5 */
> + ID_SANITISED(ID_AA64DFR0_EL1),
> + ID_SANITISED(ID_AA64DFR1_EL1),
> + ID_UNALLOCATED(5,2),
> + ID_UNALLOCATED(5,3),
> + ID_HIDDEN(ID_AA64AFR0_EL1),
> + ID_HIDDEN(ID_AA64AFR1_EL1),
> + ID_UNALLOCATED(5,6),
> + ID_UNALLOCATED(5,7),
> +
> + /* CRm=6 */
> + ID_SANITISED(ID_AA64ISAR0_EL1),
> + ID_SANITISED(ID_AA64ISAR1_EL1),
> + ID_UNALLOCATED(6,2),
> + ID_UNALLOCATED(6,3),
> + ID_UNALLOCATED(6,4),
> + ID_UNALLOCATED(6,5),
> + ID_UNALLOCATED(6,6),
> + ID_UNALLOCATED(6,7),
> +
> + /* CRm=7 */
> + ID_SANITISED(ID_AA64MMFR0_EL1),
> + ID_SANITISED(ID_AA64MMFR1_EL1),
> + ID_SANITISED(ID_AA64MMFR2_EL1),
> + ID_UNALLOCATED(7,3),
> + ID_UNALLOCATED(7,4),
> + ID_UNALLOCATED(7,5),
> + ID_UNALLOCATED(7,6),
> + ID_UNALLOCATED(7,7),
> +
I think it might be worthwhile adding a test to kvm-unit-tests to walk
all the ID registers to check this.
> { SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
> { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
> { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
> @@ -1790,8 +1999,8 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
> if (!r)
> r = find_reg(¶ms, sys_reg_descs, ARRAY_SIZE(sys_reg_descs));
>
> - /* Not saved in the sys_reg array? */
> - if (r && !r->reg)
> + /* Not saved in the sys_reg array and not otherwise accessible? */
> + if (r && !(r->reg || r->get_user))
> r = NULL;
>
> return r;
> @@ -1815,20 +2024,6 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
> FUNCTION_INVARIANT(midr_el1)
> FUNCTION_INVARIANT(ctr_el0)
> FUNCTION_INVARIANT(revidr_el1)
> -FUNCTION_INVARIANT(id_pfr0_el1)
> -FUNCTION_INVARIANT(id_pfr1_el1)
> -FUNCTION_INVARIANT(id_dfr0_el1)
> -FUNCTION_INVARIANT(id_afr0_el1)
> -FUNCTION_INVARIANT(id_mmfr0_el1)
> -FUNCTION_INVARIANT(id_mmfr1_el1)
> -FUNCTION_INVARIANT(id_mmfr2_el1)
> -FUNCTION_INVARIANT(id_mmfr3_el1)
> -FUNCTION_INVARIANT(id_isar0_el1)
> -FUNCTION_INVARIANT(id_isar1_el1)
> -FUNCTION_INVARIANT(id_isar2_el1)
> -FUNCTION_INVARIANT(id_isar3_el1)
> -FUNCTION_INVARIANT(id_isar4_el1)
> -FUNCTION_INVARIANT(id_isar5_el1)
> FUNCTION_INVARIANT(clidr_el1)
> FUNCTION_INVARIANT(aidr_el1)
>
> @@ -1836,20 +2031,6 @@ FUNCTION_INVARIANT(aidr_el1)
> static struct sys_reg_desc invariant_sys_regs[] = {
> { SYS_DESC(SYS_MIDR_EL1), NULL, get_midr_el1 },
> { SYS_DESC(SYS_REVIDR_EL1), NULL, get_revidr_el1 },
> - { SYS_DESC(SYS_ID_PFR0_EL1), NULL, get_id_pfr0_el1 },
> - { SYS_DESC(SYS_ID_PFR1_EL1), NULL, get_id_pfr1_el1 },
> - { SYS_DESC(SYS_ID_DFR0_EL1), NULL, get_id_dfr0_el1 },
> - { SYS_DESC(SYS_ID_AFR0_EL1), NULL, get_id_afr0_el1 },
> - { SYS_DESC(SYS_ID_MMFR0_EL1), NULL, get_id_mmfr0_el1 },
> - { SYS_DESC(SYS_ID_MMFR1_EL1), NULL, get_id_mmfr1_el1 },
> - { SYS_DESC(SYS_ID_MMFR2_EL1), NULL, get_id_mmfr2_el1 },
> - { SYS_DESC(SYS_ID_MMFR3_EL1), NULL, get_id_mmfr3_el1 },
> - { SYS_DESC(SYS_ID_ISAR0_EL1), NULL, get_id_isar0_el1 },
> - { SYS_DESC(SYS_ID_ISAR1_EL1), NULL, get_id_isar1_el1 },
> - { SYS_DESC(SYS_ID_ISAR2_EL1), NULL, get_id_isar2_el1 },
> - { SYS_DESC(SYS_ID_ISAR3_EL1), NULL, get_id_isar3_el1 },
> - { SYS_DESC(SYS_ID_ISAR4_EL1), NULL, get_id_isar4_el1 },
> - { SYS_DESC(SYS_ID_ISAR5_EL1), NULL, get_id_isar5_el1 },
> { SYS_DESC(SYS_CLIDR_EL1), NULL, get_clidr_el1 },
> { SYS_DESC(SYS_AIDR_EL1), NULL, get_aidr_el1 },
> { SYS_DESC(SYS_CTR_EL0), NULL, get_ctr_el0 },
> @@ -2079,12 +2260,31 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
> return true;
> }
>
> +static int walk_one_sys_reg(const struct sys_reg_desc *rd,
> + u64 __user **uind,
> + unsigned int *total)
> +{
> + /*
> + * Ignore registers we trap but don't save,
> + * and for which no custom user accessor is provided.
> + */
> + if (!(rd->reg || rd->get_user))
> + return 0;
> +
> + if (!copy_reg_to_user(rd, uind))
> + return -EFAULT;
> +
> + (*total)++;
> + return 0;
> +}
> +
> /* Assumed ordered tables, see kvm_sys_reg_table_init. */
> static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
> {
> const struct sys_reg_desc *i1, *i2, *end1, *end2;
> unsigned int total = 0;
> size_t num;
> + int err;
>
> /* We check for duplicates here, to allow arch-specific overrides. */
> i1 = get_target_table(vcpu->arch.target, true, &num);
> @@ -2098,21 +2298,13 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
> while (i1 || i2) {
> int cmp = cmp_sys_reg(i1, i2);
> /* target-specific overrides generic entry. */
> - if (cmp <= 0) {
> - /* Ignore registers we trap but don't save. */
> - if (i1->reg) {
> - if (!copy_reg_to_user(i1, &uind))
> - return -EFAULT;
> - total++;
> - }
> - } else {
> - /* Ignore registers we trap but don't save. */
> - if (i2->reg) {
> - if (!copy_reg_to_user(i2, &uind))
> - return -EFAULT;
> - total++;
> - }
> - }
> + if (cmp <= 0)
> + err = walk_one_sys_reg(i1, &uind, &total);
> + else
> + err = walk_one_sys_reg(i2, &uind, &total);
> +
> + if (err)
> + return err;
>
> if (cmp <= 0 && ++i1 == end1)
> i1 = NULL;
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests
@ 2017-09-13 14:37 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 14:37 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> Currently, a guest kernel sees the true CPU feature registers
> (ID_*_EL1) when it reads them using MRS instructions. This means
> that the guest will observe features that are present in the
> hardware but the host doesn't understand or doesn't provide support
> for. A guest may legimitately try to use such a feature as per the
> architecture, but use of the feature may trap instead of working
> normally, triggering undef injection into the guest.
>
> This is not a problem for the host, but the guest may go wrong when
> running on newer hardware than the host knows about.
>
> This patch hides from guest VMs any AArch64-specific CPU features
> that the host doesn't support, by exposing to the guest the
> sanitised versions of the registers computed by the cpufeatures
> framework, instead of the true hardware registers. To achieve
> this, HCR_EL2.TID3 is now set for AArch64 guests, and emulation
> code is added to KVM to report the sanitised versions of the
> affected registers in response to MRS and register reads from
> userspace.
>
> The affected registers are removed from invariant_sys_regs[] (since
> the invariant_sys_regs handling is no longer quite correct for
> them) and added to sys_reg_desgs[], with appropriate access(),
> get_user() and set_user() methods. No runtime vcpu storage is
> allocated for the registers: instead, they are read on demand from
> the cpufeatures framework. This may need modification in the
> future if there is a need for userspace to customise the features
> visible to the guest.
>
> Attempts by userspace to write the registers are handled similarly
> to the current invariant_sys_regs handling: writes are permitted,
> but only if they don't attempt to change the value. This is
> sufficient to support VM snapshot/restore from userspace.
>
> Because of the additional registers, restoring a VM on an older
> kernel may not work unless userspace knows how to handle the extra
> VM registers exposed to the KVM user ABI by this patch.
>
> Under the principle of least damage, this patch makes no attempt to
> handle any of the other registers currently in
> invariant_sys_regs[], or to emulate registers for AArch32: however,
> these could be handled in a similar way in future, as necessary.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Get rid of ternary operator use in walk_sys_regs().
>
> * Call write_to_read_only() if an attempt to write an ID reg is
> trapped, rather than reinventing.
> Probably we won't get there anyway: the architecture says that this
> should undef at EL1 instead.
>
> * Make ID register sysreg table less cryptic and spread the entries one
> per line.
> Also, make the architecturally unallocated and allocated but hidden
> cases more clearly distinct. These require the same behaviour but for
> different reasons, so it's better to identify them as separate.
>
> Other:
>
> * Delete BUG_ON()s that are skipped by construction:
> These check that the result of sys_reg_to_index() is a 64-bit
> register, which is always true because sys_reg_to_index()
> explicitly sets this.
>
> * Remove duplicate const in __access_id_reg args [sparse]
> ---
> arch/arm64/include/asm/sysreg.h | 3 +
> arch/arm64/kvm/hyp/switch.c | 6 +
> arch/arm64/kvm/sys_regs.c | 282 +++++++++++++++++++++++++++++++++-------
> 3 files changed, 246 insertions(+), 45 deletions(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index f707fed..480ecd6 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -149,6 +149,9 @@
> #define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
> #define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
>
> +#define SYS_ID_AA64AFR0_EL1 sys_reg(3, 0, 0, 5, 4)
> +#define SYS_ID_AA64AFR1_EL1 sys_reg(3, 0, 0, 5, 5)
> +
> #define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)
> #define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 945e79c..35a90b8 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -81,11 +81,17 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> * it will cause an exception.
> */
> val = vcpu->arch.hcr_el2;
> +
> if (!(val & HCR_RW) && system_supports_fpsimd()) {
> write_sysreg(1 << 30, fpexc32_el2);
> isb();
> }
> +
> + if (val & HCR_RW) /* for AArch64 only: */
> + val |= HCR_TID3; /* TID3: trap feature register accesses */
> +
I wondered as this is the hyp switch can we make use of testing val &
HCR_RW for both this and above. But it seems minimal in the generated
code so probably not.
> write_sysreg(val, hcr_el2);
> +
> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> write_sysreg(1 << 15, hstr_el2);
> /*
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 2e070d3..b1f7552 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -892,6 +892,137 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
> return true;
> }
>
> +/* Read a sanitised cpufeature ID register by sys_reg_desc */
> +static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
> +{
> + u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
> + (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> +
> + return raz ? 0 : read_sanitised_ftr_reg(id);
> +}
> +
> +/* cpufeature ID register access trap handlers */
> +
> +static bool __access_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r,
> + bool raz)
> +{
> + if (p->is_write)
> + return write_to_read_only(vcpu, p, r);
> +
> + p->regval = read_id_reg(r, raz);
> + return true;
> +}
> +
> +static bool access_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + return __access_id_reg(vcpu, p, r, false);
> +}
> +
> +static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + return __access_id_reg(vcpu, p, r, true);
> +}
> +
> +static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
> +static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
> +static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
> +
> +/*
> + * cpufeature ID register user accessors
> + *
> + * For now, these registers are immutable for userspace, so no values
> + * are stored, and for set_id_reg() we don't allow the effective value
> + * to be changed.
> + */
> +static int __get_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
> + bool raz)
> +{
> + const u64 id = sys_reg_to_index(rd);
> + const u64 val = read_id_reg(rd, raz);
> +
> + return reg_to_user(uaddr, &val, id);
> +}
> +
> +static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
> + bool raz)
> +{
> + const u64 id = sys_reg_to_index(rd);
> + int err;
> + u64 val;
> +
> + err = reg_from_user(&val, uaddr, id);
> + if (err)
> + return err;
> +
> + /* This is what we mean by invariant: you can't change it. */
> + if (val != read_id_reg(rd, raz))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __get_id_reg(rd, uaddr, false);
> +}
> +
> +static int set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __set_id_reg(rd, uaddr, false);
> +}
> +
> +static int get_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __get_id_reg(rd, uaddr, true);
> +}
> +
> +static int set_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __set_id_reg(rd, uaddr, true);
> +}
> +
> +/* sys_reg_desc initialiser for known cpufeature ID registers */
> +#define ID_SANITISED(name) { \
> + SYS_DESC(SYS_##name), \
> + .access = access_id_reg, \
> + .get_user = get_id_reg, \
> + .set_user = set_id_reg, \
> +}
> +
> +/*
> + * sys_reg_desc initialiser for architecturally unallocated cpufeature ID
> + * register with encoding Op0=3, Op1=0, CRn=0, CRm=crm, Op2=op2
> + * (1 <= crm < 8, 0 <= Op2 < 8).
> + */
> +#define ID_UNALLOCATED(crm, op2) { \
> + Op0(3), Op1(0), CRn(0), CRm(crm), Op2(op2), \
> + .access = access_raz_id_reg, \
> + .get_user = get_raz_id_reg, \
> + .set_user = set_raz_id_reg, \
> +}
> +
> +/*
> + * sys_reg_desc initialiser for known ID registers that we hide from guests.
> + * For now, these are exposed just like unallocated ID regs: they appear
> + * RAZ for the guest.
> + */
> +#define ID_HIDDEN(name) { \
> + SYS_DESC(SYS_##name), \
> + .access = access_raz_id_reg, \
> + .get_user = get_raz_id_reg, \
> + .set_user = set_raz_id_reg, \
> +}
> +
> /*
> * Architected system registers.
> * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
> @@ -944,6 +1075,84 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> { SYS_DESC(SYS_DBGVCR32_EL2), NULL, reset_val, DBGVCR32_EL2, 0 },
>
> { SYS_DESC(SYS_MPIDR_EL1), NULL, reset_mpidr, MPIDR_EL1 },
> +
> + /*
> + * ID regs: all ID_SANITISED() entries here must have corresponding
> + * entries in arm64_ftr_regs[].
> + */
arm64_ftr_regs isn't updated in this commit. Does this break bisection?
> +
> + /* AArch64 mappings of the AArch32 ID registers */
> + /* CRm=1 */
> + ID_SANITISED(ID_PFR0_EL1),
> + ID_SANITISED(ID_PFR1_EL1),
> + ID_SANITISED(ID_DFR0_EL1),
> + ID_HIDDEN(ID_AFR0_EL1),
> + ID_SANITISED(ID_MMFR0_EL1),
> + ID_SANITISED(ID_MMFR1_EL1),
> + ID_SANITISED(ID_MMFR2_EL1),
> + ID_SANITISED(ID_MMFR3_EL1),
> +
> + /* CRm=2 */
> + ID_SANITISED(ID_ISAR0_EL1),
> + ID_SANITISED(ID_ISAR1_EL1),
> + ID_SANITISED(ID_ISAR2_EL1),
> + ID_SANITISED(ID_ISAR3_EL1),
> + ID_SANITISED(ID_ISAR4_EL1),
> + ID_SANITISED(ID_ISAR5_EL1),
> + ID_SANITISED(ID_MMFR4_EL1),
> + ID_UNALLOCATED(2,7),
> +
> + /* CRm=3 */
> + ID_SANITISED(MVFR0_EL1),
> + ID_SANITISED(MVFR1_EL1),
> + ID_SANITISED(MVFR2_EL1),
> + ID_UNALLOCATED(3,3),
> + ID_UNALLOCATED(3,4),
> + ID_UNALLOCATED(3,5),
> + ID_UNALLOCATED(3,6),
> + ID_UNALLOCATED(3,7),
> +
> + /* AArch64 ID registers */
> + /* CRm=4 */
> + ID_SANITISED(ID_AA64PFR0_EL1),
> + ID_SANITISED(ID_AA64PFR1_EL1),
> + ID_UNALLOCATED(4,2),
> + ID_UNALLOCATED(4,3),
> + ID_UNALLOCATED(4,4),
> + ID_UNALLOCATED(4,5),
> + ID_UNALLOCATED(4,6),
> + ID_UNALLOCATED(4,7),
> +
> + /* CRm=5 */
> + ID_SANITISED(ID_AA64DFR0_EL1),
> + ID_SANITISED(ID_AA64DFR1_EL1),
> + ID_UNALLOCATED(5,2),
> + ID_UNALLOCATED(5,3),
> + ID_HIDDEN(ID_AA64AFR0_EL1),
> + ID_HIDDEN(ID_AA64AFR1_EL1),
> + ID_UNALLOCATED(5,6),
> + ID_UNALLOCATED(5,7),
> +
> + /* CRm=6 */
> + ID_SANITISED(ID_AA64ISAR0_EL1),
> + ID_SANITISED(ID_AA64ISAR1_EL1),
> + ID_UNALLOCATED(6,2),
> + ID_UNALLOCATED(6,3),
> + ID_UNALLOCATED(6,4),
> + ID_UNALLOCATED(6,5),
> + ID_UNALLOCATED(6,6),
> + ID_UNALLOCATED(6,7),
> +
> + /* CRm=7 */
> + ID_SANITISED(ID_AA64MMFR0_EL1),
> + ID_SANITISED(ID_AA64MMFR1_EL1),
> + ID_SANITISED(ID_AA64MMFR2_EL1),
> + ID_UNALLOCATED(7,3),
> + ID_UNALLOCATED(7,4),
> + ID_UNALLOCATED(7,5),
> + ID_UNALLOCATED(7,6),
> + ID_UNALLOCATED(7,7),
> +
I think it might be worthwhile adding a test to kvm-unit-tests to walk
all the ID registers to check this.
> { SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
> { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
> { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
> @@ -1790,8 +1999,8 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
> if (!r)
> r = find_reg(¶ms, sys_reg_descs, ARRAY_SIZE(sys_reg_descs));
>
> - /* Not saved in the sys_reg array? */
> - if (r && !r->reg)
> + /* Not saved in the sys_reg array and not otherwise accessible? */
> + if (r && !(r->reg || r->get_user))
> r = NULL;
>
> return r;
> @@ -1815,20 +2024,6 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
> FUNCTION_INVARIANT(midr_el1)
> FUNCTION_INVARIANT(ctr_el0)
> FUNCTION_INVARIANT(revidr_el1)
> -FUNCTION_INVARIANT(id_pfr0_el1)
> -FUNCTION_INVARIANT(id_pfr1_el1)
> -FUNCTION_INVARIANT(id_dfr0_el1)
> -FUNCTION_INVARIANT(id_afr0_el1)
> -FUNCTION_INVARIANT(id_mmfr0_el1)
> -FUNCTION_INVARIANT(id_mmfr1_el1)
> -FUNCTION_INVARIANT(id_mmfr2_el1)
> -FUNCTION_INVARIANT(id_mmfr3_el1)
> -FUNCTION_INVARIANT(id_isar0_el1)
> -FUNCTION_INVARIANT(id_isar1_el1)
> -FUNCTION_INVARIANT(id_isar2_el1)
> -FUNCTION_INVARIANT(id_isar3_el1)
> -FUNCTION_INVARIANT(id_isar4_el1)
> -FUNCTION_INVARIANT(id_isar5_el1)
> FUNCTION_INVARIANT(clidr_el1)
> FUNCTION_INVARIANT(aidr_el1)
>
> @@ -1836,20 +2031,6 @@ FUNCTION_INVARIANT(aidr_el1)
> static struct sys_reg_desc invariant_sys_regs[] = {
> { SYS_DESC(SYS_MIDR_EL1), NULL, get_midr_el1 },
> { SYS_DESC(SYS_REVIDR_EL1), NULL, get_revidr_el1 },
> - { SYS_DESC(SYS_ID_PFR0_EL1), NULL, get_id_pfr0_el1 },
> - { SYS_DESC(SYS_ID_PFR1_EL1), NULL, get_id_pfr1_el1 },
> - { SYS_DESC(SYS_ID_DFR0_EL1), NULL, get_id_dfr0_el1 },
> - { SYS_DESC(SYS_ID_AFR0_EL1), NULL, get_id_afr0_el1 },
> - { SYS_DESC(SYS_ID_MMFR0_EL1), NULL, get_id_mmfr0_el1 },
> - { SYS_DESC(SYS_ID_MMFR1_EL1), NULL, get_id_mmfr1_el1 },
> - { SYS_DESC(SYS_ID_MMFR2_EL1), NULL, get_id_mmfr2_el1 },
> - { SYS_DESC(SYS_ID_MMFR3_EL1), NULL, get_id_mmfr3_el1 },
> - { SYS_DESC(SYS_ID_ISAR0_EL1), NULL, get_id_isar0_el1 },
> - { SYS_DESC(SYS_ID_ISAR1_EL1), NULL, get_id_isar1_el1 },
> - { SYS_DESC(SYS_ID_ISAR2_EL1), NULL, get_id_isar2_el1 },
> - { SYS_DESC(SYS_ID_ISAR3_EL1), NULL, get_id_isar3_el1 },
> - { SYS_DESC(SYS_ID_ISAR4_EL1), NULL, get_id_isar4_el1 },
> - { SYS_DESC(SYS_ID_ISAR5_EL1), NULL, get_id_isar5_el1 },
> { SYS_DESC(SYS_CLIDR_EL1), NULL, get_clidr_el1 },
> { SYS_DESC(SYS_AIDR_EL1), NULL, get_aidr_el1 },
> { SYS_DESC(SYS_CTR_EL0), NULL, get_ctr_el0 },
> @@ -2079,12 +2260,31 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
> return true;
> }
>
> +static int walk_one_sys_reg(const struct sys_reg_desc *rd,
> + u64 __user **uind,
> + unsigned int *total)
> +{
> + /*
> + * Ignore registers we trap but don't save,
> + * and for which no custom user accessor is provided.
> + */
> + if (!(rd->reg || rd->get_user))
> + return 0;
> +
> + if (!copy_reg_to_user(rd, uind))
> + return -EFAULT;
> +
> + (*total)++;
> + return 0;
> +}
> +
> /* Assumed ordered tables, see kvm_sys_reg_table_init. */
> static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
> {
> const struct sys_reg_desc *i1, *i2, *end1, *end2;
> unsigned int total = 0;
> size_t num;
> + int err;
>
> /* We check for duplicates here, to allow arch-specific overrides. */
> i1 = get_target_table(vcpu->arch.target, true, &num);
> @@ -2098,21 +2298,13 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
> while (i1 || i2) {
> int cmp = cmp_sys_reg(i1, i2);
> /* target-specific overrides generic entry. */
> - if (cmp <= 0) {
> - /* Ignore registers we trap but don't save. */
> - if (i1->reg) {
> - if (!copy_reg_to_user(i1, &uind))
> - return -EFAULT;
> - total++;
> - }
> - } else {
> - /* Ignore registers we trap but don't save. */
> - if (i2->reg) {
> - if (!copy_reg_to_user(i2, &uind))
> - return -EFAULT;
> - total++;
> - }
> - }
> + if (cmp <= 0)
> + err = walk_one_sys_reg(i1, &uind, &total);
> + else
> + err = walk_one_sys_reg(i2, &uind, &total);
> +
> + if (err)
> + return err;
>
> if (cmp <= 0 && ++i1 == end1)
> i1 = NULL;
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests
@ 2017-09-13 14:37 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 14:37 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> Currently, a guest kernel sees the true CPU feature registers
> (ID_*_EL1) when it reads them using MRS instructions. This means
> that the guest will observe features that are present in the
> hardware but the host doesn't understand or doesn't provide support
> for. A guest may legimitately try to use such a feature as per the
> architecture, but use of the feature may trap instead of working
> normally, triggering undef injection into the guest.
>
> This is not a problem for the host, but the guest may go wrong when
> running on newer hardware than the host knows about.
>
> This patch hides from guest VMs any AArch64-specific CPU features
> that the host doesn't support, by exposing to the guest the
> sanitised versions of the registers computed by the cpufeatures
> framework, instead of the true hardware registers. To achieve
> this, HCR_EL2.TID3 is now set for AArch64 guests, and emulation
> code is added to KVM to report the sanitised versions of the
> affected registers in response to MRS and register reads from
> userspace.
>
> The affected registers are removed from invariant_sys_regs[] (since
> the invariant_sys_regs handling is no longer quite correct for
> them) and added to sys_reg_desgs[], with appropriate access(),
> get_user() and set_user() methods. No runtime vcpu storage is
> allocated for the registers: instead, they are read on demand from
> the cpufeatures framework. This may need modification in the
> future if there is a need for userspace to customise the features
> visible to the guest.
>
> Attempts by userspace to write the registers are handled similarly
> to the current invariant_sys_regs handling: writes are permitted,
> but only if they don't attempt to change the value. This is
> sufficient to support VM snapshot/restore from userspace.
>
> Because of the additional registers, restoring a VM on an older
> kernel may not work unless userspace knows how to handle the extra
> VM registers exposed to the KVM user ABI by this patch.
>
> Under the principle of least damage, this patch makes no attempt to
> handle any of the other registers currently in
> invariant_sys_regs[], or to emulate registers for AArch32: however,
> these could be handled in a similar way in future, as necessary.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Get rid of ternary operator use in walk_sys_regs().
>
> * Call write_to_read_only() if an attempt to write an ID reg is
> trapped, rather than reinventing.
> Probably we won't get there anyway: the architecture says that this
> should undef at EL1 instead.
>
> * Make ID register sysreg table less cryptic and spread the entries one
> per line.
> Also, make the architecturally unallocated and allocated but hidden
> cases more clearly distinct. These require the same behaviour but for
> different reasons, so it's better to identify them as separate.
>
> Other:
>
> * Delete BUG_ON()s that are skipped by construction:
> These check that the result of sys_reg_to_index() is a 64-bit
> register, which is always true because sys_reg_to_index()
> explicitly sets this.
>
> * Remove duplicate const in __access_id_reg args [sparse]
> ---
> arch/arm64/include/asm/sysreg.h | 3 +
> arch/arm64/kvm/hyp/switch.c | 6 +
> arch/arm64/kvm/sys_regs.c | 282 +++++++++++++++++++++++++++++++++-------
> 3 files changed, 246 insertions(+), 45 deletions(-)
>
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index f707fed..480ecd6 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -149,6 +149,9 @@
> #define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
> #define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
>
> +#define SYS_ID_AA64AFR0_EL1 sys_reg(3, 0, 0, 5, 4)
> +#define SYS_ID_AA64AFR1_EL1 sys_reg(3, 0, 0, 5, 5)
> +
> #define SYS_ID_AA64ISAR0_EL1 sys_reg(3, 0, 0, 6, 0)
> #define SYS_ID_AA64ISAR1_EL1 sys_reg(3, 0, 0, 6, 1)
>
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 945e79c..35a90b8 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -81,11 +81,17 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> * it will cause an exception.
> */
> val = vcpu->arch.hcr_el2;
> +
> if (!(val & HCR_RW) && system_supports_fpsimd()) {
> write_sysreg(1 << 30, fpexc32_el2);
> isb();
> }
> +
> + if (val & HCR_RW) /* for AArch64 only: */
> + val |= HCR_TID3; /* TID3: trap feature register accesses */
> +
I wondered as this is the hyp switch can we make use of testing val &
HCR_RW for both this and above. But it seems minimal in the generated
code so probably not.
> write_sysreg(val, hcr_el2);
> +
> /* Trap on AArch32 cp15 c15 accesses (EL1 or EL0) */
> write_sysreg(1 << 15, hstr_el2);
> /*
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index 2e070d3..b1f7552 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -892,6 +892,137 @@ static bool access_cntp_cval(struct kvm_vcpu *vcpu,
> return true;
> }
>
> +/* Read a sanitised cpufeature ID register by sys_reg_desc */
> +static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
> +{
> + u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
> + (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> +
> + return raz ? 0 : read_sanitised_ftr_reg(id);
> +}
> +
> +/* cpufeature ID register access trap handlers */
> +
> +static bool __access_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r,
> + bool raz)
> +{
> + if (p->is_write)
> + return write_to_read_only(vcpu, p, r);
> +
> + p->regval = read_id_reg(r, raz);
> + return true;
> +}
> +
> +static bool access_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + return __access_id_reg(vcpu, p, r, false);
> +}
> +
> +static bool access_raz_id_reg(struct kvm_vcpu *vcpu,
> + struct sys_reg_params *p,
> + const struct sys_reg_desc *r)
> +{
> + return __access_id_reg(vcpu, p, r, true);
> +}
> +
> +static int reg_from_user(u64 *val, const void __user *uaddr, u64 id);
> +static int reg_to_user(void __user *uaddr, const u64 *val, u64 id);
> +static u64 sys_reg_to_index(const struct sys_reg_desc *reg);
> +
> +/*
> + * cpufeature ID register user accessors
> + *
> + * For now, these registers are immutable for userspace, so no values
> + * are stored, and for set_id_reg() we don't allow the effective value
> + * to be changed.
> + */
> +static int __get_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
> + bool raz)
> +{
> + const u64 id = sys_reg_to_index(rd);
> + const u64 val = read_id_reg(rd, raz);
> +
> + return reg_to_user(uaddr, &val, id);
> +}
> +
> +static int __set_id_reg(const struct sys_reg_desc *rd, void __user *uaddr,
> + bool raz)
> +{
> + const u64 id = sys_reg_to_index(rd);
> + int err;
> + u64 val;
> +
> + err = reg_from_user(&val, uaddr, id);
> + if (err)
> + return err;
> +
> + /* This is what we mean by invariant: you can't change it. */
> + if (val != read_id_reg(rd, raz))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +static int get_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __get_id_reg(rd, uaddr, false);
> +}
> +
> +static int set_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __set_id_reg(rd, uaddr, false);
> +}
> +
> +static int get_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __get_id_reg(rd, uaddr, true);
> +}
> +
> +static int set_raz_id_reg(struct kvm_vcpu *vcpu, const struct sys_reg_desc *rd,
> + const struct kvm_one_reg *reg, void __user *uaddr)
> +{
> + return __set_id_reg(rd, uaddr, true);
> +}
> +
> +/* sys_reg_desc initialiser for known cpufeature ID registers */
> +#define ID_SANITISED(name) { \
> + SYS_DESC(SYS_##name), \
> + .access = access_id_reg, \
> + .get_user = get_id_reg, \
> + .set_user = set_id_reg, \
> +}
> +
> +/*
> + * sys_reg_desc initialiser for architecturally unallocated cpufeature ID
> + * register with encoding Op0=3, Op1=0, CRn=0, CRm=crm, Op2=op2
> + * (1 <= crm < 8, 0 <= Op2 < 8).
> + */
> +#define ID_UNALLOCATED(crm, op2) { \
> + Op0(3), Op1(0), CRn(0), CRm(crm), Op2(op2), \
> + .access = access_raz_id_reg, \
> + .get_user = get_raz_id_reg, \
> + .set_user = set_raz_id_reg, \
> +}
> +
> +/*
> + * sys_reg_desc initialiser for known ID registers that we hide from guests.
> + * For now, these are exposed just like unallocated ID regs: they appear
> + * RAZ for the guest.
> + */
> +#define ID_HIDDEN(name) { \
> + SYS_DESC(SYS_##name), \
> + .access = access_raz_id_reg, \
> + .get_user = get_raz_id_reg, \
> + .set_user = set_raz_id_reg, \
> +}
> +
> /*
> * Architected system registers.
> * Important: Must be sorted ascending by Op0, Op1, CRn, CRm, Op2
> @@ -944,6 +1075,84 @@ static const struct sys_reg_desc sys_reg_descs[] = {
> { SYS_DESC(SYS_DBGVCR32_EL2), NULL, reset_val, DBGVCR32_EL2, 0 },
>
> { SYS_DESC(SYS_MPIDR_EL1), NULL, reset_mpidr, MPIDR_EL1 },
> +
> + /*
> + * ID regs: all ID_SANITISED() entries here must have corresponding
> + * entries in arm64_ftr_regs[].
> + */
arm64_ftr_regs isn't updated in this commit. Does this break bisection?
> +
> + /* AArch64 mappings of the AArch32 ID registers */
> + /* CRm=1 */
> + ID_SANITISED(ID_PFR0_EL1),
> + ID_SANITISED(ID_PFR1_EL1),
> + ID_SANITISED(ID_DFR0_EL1),
> + ID_HIDDEN(ID_AFR0_EL1),
> + ID_SANITISED(ID_MMFR0_EL1),
> + ID_SANITISED(ID_MMFR1_EL1),
> + ID_SANITISED(ID_MMFR2_EL1),
> + ID_SANITISED(ID_MMFR3_EL1),
> +
> + /* CRm=2 */
> + ID_SANITISED(ID_ISAR0_EL1),
> + ID_SANITISED(ID_ISAR1_EL1),
> + ID_SANITISED(ID_ISAR2_EL1),
> + ID_SANITISED(ID_ISAR3_EL1),
> + ID_SANITISED(ID_ISAR4_EL1),
> + ID_SANITISED(ID_ISAR5_EL1),
> + ID_SANITISED(ID_MMFR4_EL1),
> + ID_UNALLOCATED(2,7),
> +
> + /* CRm=3 */
> + ID_SANITISED(MVFR0_EL1),
> + ID_SANITISED(MVFR1_EL1),
> + ID_SANITISED(MVFR2_EL1),
> + ID_UNALLOCATED(3,3),
> + ID_UNALLOCATED(3,4),
> + ID_UNALLOCATED(3,5),
> + ID_UNALLOCATED(3,6),
> + ID_UNALLOCATED(3,7),
> +
> + /* AArch64 ID registers */
> + /* CRm=4 */
> + ID_SANITISED(ID_AA64PFR0_EL1),
> + ID_SANITISED(ID_AA64PFR1_EL1),
> + ID_UNALLOCATED(4,2),
> + ID_UNALLOCATED(4,3),
> + ID_UNALLOCATED(4,4),
> + ID_UNALLOCATED(4,5),
> + ID_UNALLOCATED(4,6),
> + ID_UNALLOCATED(4,7),
> +
> + /* CRm=5 */
> + ID_SANITISED(ID_AA64DFR0_EL1),
> + ID_SANITISED(ID_AA64DFR1_EL1),
> + ID_UNALLOCATED(5,2),
> + ID_UNALLOCATED(5,3),
> + ID_HIDDEN(ID_AA64AFR0_EL1),
> + ID_HIDDEN(ID_AA64AFR1_EL1),
> + ID_UNALLOCATED(5,6),
> + ID_UNALLOCATED(5,7),
> +
> + /* CRm=6 */
> + ID_SANITISED(ID_AA64ISAR0_EL1),
> + ID_SANITISED(ID_AA64ISAR1_EL1),
> + ID_UNALLOCATED(6,2),
> + ID_UNALLOCATED(6,3),
> + ID_UNALLOCATED(6,4),
> + ID_UNALLOCATED(6,5),
> + ID_UNALLOCATED(6,6),
> + ID_UNALLOCATED(6,7),
> +
> + /* CRm=7 */
> + ID_SANITISED(ID_AA64MMFR0_EL1),
> + ID_SANITISED(ID_AA64MMFR1_EL1),
> + ID_SANITISED(ID_AA64MMFR2_EL1),
> + ID_UNALLOCATED(7,3),
> + ID_UNALLOCATED(7,4),
> + ID_UNALLOCATED(7,5),
> + ID_UNALLOCATED(7,6),
> + ID_UNALLOCATED(7,7),
> +
I think it might be worthwhile adding a test to kvm-unit-tests to walk
all the ID registers to check this.
> { SYS_DESC(SYS_SCTLR_EL1), access_vm_reg, reset_val, SCTLR_EL1, 0x00C50078 },
> { SYS_DESC(SYS_CPACR_EL1), NULL, reset_val, CPACR_EL1, 0 },
> { SYS_DESC(SYS_TTBR0_EL1), access_vm_reg, reset_unknown, TTBR0_EL1 },
> @@ -1790,8 +1999,8 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
> if (!r)
> r = find_reg(¶ms, sys_reg_descs, ARRAY_SIZE(sys_reg_descs));
>
> - /* Not saved in the sys_reg array? */
> - if (r && !r->reg)
> + /* Not saved in the sys_reg array and not otherwise accessible? */
> + if (r && !(r->reg || r->get_user))
> r = NULL;
>
> return r;
> @@ -1815,20 +2024,6 @@ static const struct sys_reg_desc *index_to_sys_reg_desc(struct kvm_vcpu *vcpu,
> FUNCTION_INVARIANT(midr_el1)
> FUNCTION_INVARIANT(ctr_el0)
> FUNCTION_INVARIANT(revidr_el1)
> -FUNCTION_INVARIANT(id_pfr0_el1)
> -FUNCTION_INVARIANT(id_pfr1_el1)
> -FUNCTION_INVARIANT(id_dfr0_el1)
> -FUNCTION_INVARIANT(id_afr0_el1)
> -FUNCTION_INVARIANT(id_mmfr0_el1)
> -FUNCTION_INVARIANT(id_mmfr1_el1)
> -FUNCTION_INVARIANT(id_mmfr2_el1)
> -FUNCTION_INVARIANT(id_mmfr3_el1)
> -FUNCTION_INVARIANT(id_isar0_el1)
> -FUNCTION_INVARIANT(id_isar1_el1)
> -FUNCTION_INVARIANT(id_isar2_el1)
> -FUNCTION_INVARIANT(id_isar3_el1)
> -FUNCTION_INVARIANT(id_isar4_el1)
> -FUNCTION_INVARIANT(id_isar5_el1)
> FUNCTION_INVARIANT(clidr_el1)
> FUNCTION_INVARIANT(aidr_el1)
>
> @@ -1836,20 +2031,6 @@ FUNCTION_INVARIANT(aidr_el1)
> static struct sys_reg_desc invariant_sys_regs[] = {
> { SYS_DESC(SYS_MIDR_EL1), NULL, get_midr_el1 },
> { SYS_DESC(SYS_REVIDR_EL1), NULL, get_revidr_el1 },
> - { SYS_DESC(SYS_ID_PFR0_EL1), NULL, get_id_pfr0_el1 },
> - { SYS_DESC(SYS_ID_PFR1_EL1), NULL, get_id_pfr1_el1 },
> - { SYS_DESC(SYS_ID_DFR0_EL1), NULL, get_id_dfr0_el1 },
> - { SYS_DESC(SYS_ID_AFR0_EL1), NULL, get_id_afr0_el1 },
> - { SYS_DESC(SYS_ID_MMFR0_EL1), NULL, get_id_mmfr0_el1 },
> - { SYS_DESC(SYS_ID_MMFR1_EL1), NULL, get_id_mmfr1_el1 },
> - { SYS_DESC(SYS_ID_MMFR2_EL1), NULL, get_id_mmfr2_el1 },
> - { SYS_DESC(SYS_ID_MMFR3_EL1), NULL, get_id_mmfr3_el1 },
> - { SYS_DESC(SYS_ID_ISAR0_EL1), NULL, get_id_isar0_el1 },
> - { SYS_DESC(SYS_ID_ISAR1_EL1), NULL, get_id_isar1_el1 },
> - { SYS_DESC(SYS_ID_ISAR2_EL1), NULL, get_id_isar2_el1 },
> - { SYS_DESC(SYS_ID_ISAR3_EL1), NULL, get_id_isar3_el1 },
> - { SYS_DESC(SYS_ID_ISAR4_EL1), NULL, get_id_isar4_el1 },
> - { SYS_DESC(SYS_ID_ISAR5_EL1), NULL, get_id_isar5_el1 },
> { SYS_DESC(SYS_CLIDR_EL1), NULL, get_clidr_el1 },
> { SYS_DESC(SYS_AIDR_EL1), NULL, get_aidr_el1 },
> { SYS_DESC(SYS_CTR_EL0), NULL, get_ctr_el0 },
> @@ -2079,12 +2260,31 @@ static bool copy_reg_to_user(const struct sys_reg_desc *reg, u64 __user **uind)
> return true;
> }
>
> +static int walk_one_sys_reg(const struct sys_reg_desc *rd,
> + u64 __user **uind,
> + unsigned int *total)
> +{
> + /*
> + * Ignore registers we trap but don't save,
> + * and for which no custom user accessor is provided.
> + */
> + if (!(rd->reg || rd->get_user))
> + return 0;
> +
> + if (!copy_reg_to_user(rd, uind))
> + return -EFAULT;
> +
> + (*total)++;
> + return 0;
> +}
> +
> /* Assumed ordered tables, see kvm_sys_reg_table_init. */
> static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
> {
> const struct sys_reg_desc *i1, *i2, *end1, *end2;
> unsigned int total = 0;
> size_t num;
> + int err;
>
> /* We check for duplicates here, to allow arch-specific overrides. */
> i1 = get_target_table(vcpu->arch.target, true, &num);
> @@ -2098,21 +2298,13 @@ static int walk_sys_regs(struct kvm_vcpu *vcpu, u64 __user *uind)
> while (i1 || i2) {
> int cmp = cmp_sys_reg(i1, i2);
> /* target-specific overrides generic entry. */
> - if (cmp <= 0) {
> - /* Ignore registers we trap but don't save. */
> - if (i1->reg) {
> - if (!copy_reg_to_user(i1, &uind))
> - return -EFAULT;
> - total++;
> - }
> - } else {
> - /* Ignore registers we trap but don't save. */
> - if (i2->reg) {
> - if (!copy_reg_to_user(i2, &uind))
> - return -EFAULT;
> - total++;
> - }
> - }
> + if (cmp <= 0)
> + err = walk_one_sys_reg(i1, &uind, &total);
> + else
> + err = walk_one_sys_reg(i2, &uind, &total);
> +
> + if (err)
> + return err;
>
> if (cmp <= 0 && ++i1 == end1)
> i1 = NULL;
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 06/28] arm64/sve: System register and exception syndrome definitions
@ 2017-09-13 14:48 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 14:48 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> The SVE architecture adds some system registers, ID register fields
> and a dedicated ESR exception class.
>
> This patch adds the appropriate definitions that will be needed by
> the kernel.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Add comments to clarify CPACR_EL1_ZEN_ELxEN bit meanings.
> * Add comment clarifying the status of the LEN field expansion bits.
> ---
> arch/arm64/include/asm/esr.h | 3 ++-
> arch/arm64/include/asm/kvm_arm.h | 1 +
> arch/arm64/include/asm/sysreg.h | 21 +++++++++++++++++++++
> arch/arm64/kernel/traps.c | 1 +
> 4 files changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
> index 66ed8b6..014d7d8 100644
> --- a/arch/arm64/include/asm/esr.h
> +++ b/arch/arm64/include/asm/esr.h
> @@ -43,7 +43,8 @@
> #define ESR_ELx_EC_HVC64 (0x16)
> #define ESR_ELx_EC_SMC64 (0x17)
> #define ESR_ELx_EC_SYS64 (0x18)
> -/* Unallocated EC: 0x19 - 0x1E */
> +#define ESR_ELx_EC_SVE (0x19)
> +/* Unallocated EC: 0x1A - 0x1E */
> #define ESR_ELx_EC_IMP_DEF (0x1f)
> #define ESR_ELx_EC_IABT_LOW (0x20)
> #define ESR_ELx_EC_IABT_CUR (0x21)
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 61d694c..dbf0537 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -185,6 +185,7 @@
> #define CPTR_EL2_TCPAC (1 << 31)
> #define CPTR_EL2_TTA (1 << 20)
> #define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
> +#define CPTR_EL2_TZ (1 << 8)
> #define CPTR_EL2_DEFAULT 0x000033ff
>
> /* Hyp Debug Configuration Register bits */
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 480ecd6..36fe2ae 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -145,6 +145,7 @@
>
> #define SYS_ID_AA64PFR0_EL1 sys_reg(3, 0, 0, 4, 0)
> #define SYS_ID_AA64PFR1_EL1 sys_reg(3, 0, 0, 4, 1)
> +#define SYS_ID_AA64ZFR0_EL1 sys_reg(3, 0, 0, 4, 4)
>
> #define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
> #define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
> @@ -163,6 +164,8 @@
> #define SYS_ACTLR_EL1 sys_reg(3, 0, 1, 0, 1)
> #define SYS_CPACR_EL1 sys_reg(3, 0, 1, 0, 2)
>
> +#define SYS_ZCR_EL1 sys_reg(3, 0, 1, 2, 0)
> +
> #define SYS_TTBR0_EL1 sys_reg(3, 0, 2, 0, 0)
> #define SYS_TTBR1_EL1 sys_reg(3, 0, 2, 0, 1)
> #define SYS_TCR_EL1 sys_reg(3, 0, 2, 0, 2)
> @@ -253,6 +256,8 @@
>
> #define SYS_PMCCFILTR_EL0 sys_reg (3, 3, 14, 15, 7)
>
> +#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
> +
> #define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
> #define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
> #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
> @@ -335,6 +340,7 @@
> #define ID_AA64ISAR1_DPB_SHIFT 0
>
> /* id_aa64pfr0 */
> +#define ID_AA64PFR0_SVE_SHIFT 32
> #define ID_AA64PFR0_GIC_SHIFT 24
> #define ID_AA64PFR0_ASIMD_SHIFT 20
> #define ID_AA64PFR0_FP_SHIFT 16
> @@ -343,6 +349,7 @@
> #define ID_AA64PFR0_EL1_SHIFT 4
> #define ID_AA64PFR0_EL0_SHIFT 0
>
> +#define ID_AA64PFR0_SVE 0x1
> #define ID_AA64PFR0_FP_NI 0xf
> #define ID_AA64PFR0_FP_SUPPORTED 0x0
> #define ID_AA64PFR0_ASIMD_NI 0xf
> @@ -444,6 +451,20 @@
> #endif
>
>
> +/*
> + * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which
> + * are reserved by the SVE architecture for future expansion of the LEN
> + * field, with compatible semantics.
> + */
> +#define ZCR_ELx_LEN_SHIFT 0
> +#define ZCR_ELx_LEN_SIZE 9
> +#define ZCR_ELx_LEN_MASK 0x1ff
> +
> +#define CPACR_EL1_ZEN_EL1EN (1 << 16) /* enable EL1 access */
> +#define CPACR_EL1_ZEN_EL0EN (1 << 17) /* enable EL0 access, if EL1EN set */
> +#define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
> +
> +
> /* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */
> #define SYS_MPIDR_SAFE_VAL (1UL << 31)
>
> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> index 5ea4b85..f202932 100644
> --- a/arch/arm64/kernel/traps.c
> +++ b/arch/arm64/kernel/traps.c
> @@ -603,6 +603,7 @@ static const char *esr_class_str[] = {
> [ESR_ELx_EC_HVC64] = "HVC (AArch64)",
> [ESR_ELx_EC_SMC64] = "SMC (AArch64)",
> [ESR_ELx_EC_SYS64] = "MSR/MRS (AArch64)",
> + [ESR_ELx_EC_SVE] = "SVE",
> [ESR_ELx_EC_IMP_DEF] = "EL3 IMP DEF",
> [ESR_ELx_EC_IABT_LOW] = "IABT (lower EL)",
> [ESR_ELx_EC_IABT_CUR] = "IABT (current EL)",
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 06/28] arm64/sve: System register and exception syndrome definitions
@ 2017-09-13 14:48 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 14:48 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> The SVE architecture adds some system registers, ID register fields
> and a dedicated ESR exception class.
>
> This patch adds the appropriate definitions that will be needed by
> the kernel.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Add comments to clarify CPACR_EL1_ZEN_ELxEN bit meanings.
> * Add comment clarifying the status of the LEN field expansion bits.
> ---
> arch/arm64/include/asm/esr.h | 3 ++-
> arch/arm64/include/asm/kvm_arm.h | 1 +
> arch/arm64/include/asm/sysreg.h | 21 +++++++++++++++++++++
> arch/arm64/kernel/traps.c | 1 +
> 4 files changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
> index 66ed8b6..014d7d8 100644
> --- a/arch/arm64/include/asm/esr.h
> +++ b/arch/arm64/include/asm/esr.h
> @@ -43,7 +43,8 @@
> #define ESR_ELx_EC_HVC64 (0x16)
> #define ESR_ELx_EC_SMC64 (0x17)
> #define ESR_ELx_EC_SYS64 (0x18)
> -/* Unallocated EC: 0x19 - 0x1E */
> +#define ESR_ELx_EC_SVE (0x19)
> +/* Unallocated EC: 0x1A - 0x1E */
> #define ESR_ELx_EC_IMP_DEF (0x1f)
> #define ESR_ELx_EC_IABT_LOW (0x20)
> #define ESR_ELx_EC_IABT_CUR (0x21)
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 61d694c..dbf0537 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -185,6 +185,7 @@
> #define CPTR_EL2_TCPAC (1 << 31)
> #define CPTR_EL2_TTA (1 << 20)
> #define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
> +#define CPTR_EL2_TZ (1 << 8)
> #define CPTR_EL2_DEFAULT 0x000033ff
>
> /* Hyp Debug Configuration Register bits */
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 480ecd6..36fe2ae 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -145,6 +145,7 @@
>
> #define SYS_ID_AA64PFR0_EL1 sys_reg(3, 0, 0, 4, 0)
> #define SYS_ID_AA64PFR1_EL1 sys_reg(3, 0, 0, 4, 1)
> +#define SYS_ID_AA64ZFR0_EL1 sys_reg(3, 0, 0, 4, 4)
>
> #define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
> #define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
> @@ -163,6 +164,8 @@
> #define SYS_ACTLR_EL1 sys_reg(3, 0, 1, 0, 1)
> #define SYS_CPACR_EL1 sys_reg(3, 0, 1, 0, 2)
>
> +#define SYS_ZCR_EL1 sys_reg(3, 0, 1, 2, 0)
> +
> #define SYS_TTBR0_EL1 sys_reg(3, 0, 2, 0, 0)
> #define SYS_TTBR1_EL1 sys_reg(3, 0, 2, 0, 1)
> #define SYS_TCR_EL1 sys_reg(3, 0, 2, 0, 2)
> @@ -253,6 +256,8 @@
>
> #define SYS_PMCCFILTR_EL0 sys_reg (3, 3, 14, 15, 7)
>
> +#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
> +
> #define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
> #define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
> #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
> @@ -335,6 +340,7 @@
> #define ID_AA64ISAR1_DPB_SHIFT 0
>
> /* id_aa64pfr0 */
> +#define ID_AA64PFR0_SVE_SHIFT 32
> #define ID_AA64PFR0_GIC_SHIFT 24
> #define ID_AA64PFR0_ASIMD_SHIFT 20
> #define ID_AA64PFR0_FP_SHIFT 16
> @@ -343,6 +349,7 @@
> #define ID_AA64PFR0_EL1_SHIFT 4
> #define ID_AA64PFR0_EL0_SHIFT 0
>
> +#define ID_AA64PFR0_SVE 0x1
> #define ID_AA64PFR0_FP_NI 0xf
> #define ID_AA64PFR0_FP_SUPPORTED 0x0
> #define ID_AA64PFR0_ASIMD_NI 0xf
> @@ -444,6 +451,20 @@
> #endif
>
>
> +/*
> + * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which
> + * are reserved by the SVE architecture for future expansion of the LEN
> + * field, with compatible semantics.
> + */
> +#define ZCR_ELx_LEN_SHIFT 0
> +#define ZCR_ELx_LEN_SIZE 9
> +#define ZCR_ELx_LEN_MASK 0x1ff
> +
> +#define CPACR_EL1_ZEN_EL1EN (1 << 16) /* enable EL1 access */
> +#define CPACR_EL1_ZEN_EL0EN (1 << 17) /* enable EL0 access, if EL1EN set */
> +#define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
> +
> +
> /* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */
> #define SYS_MPIDR_SAFE_VAL (1UL << 31)
>
> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> index 5ea4b85..f202932 100644
> --- a/arch/arm64/kernel/traps.c
> +++ b/arch/arm64/kernel/traps.c
> @@ -603,6 +603,7 @@ static const char *esr_class_str[] = {
> [ESR_ELx_EC_HVC64] = "HVC (AArch64)",
> [ESR_ELx_EC_SMC64] = "SMC (AArch64)",
> [ESR_ELx_EC_SYS64] = "MSR/MRS (AArch64)",
> + [ESR_ELx_EC_SVE] = "SVE",
> [ESR_ELx_EC_IMP_DEF] = "EL3 IMP DEF",
> [ESR_ELx_EC_IABT_LOW] = "IABT (lower EL)",
> [ESR_ELx_EC_IABT_CUR] = "IABT (current EL)",
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 06/28] arm64/sve: System register and exception syndrome definitions
@ 2017-09-13 14:48 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 14:48 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> The SVE architecture adds some system registers, ID register fields
> and a dedicated ESR exception class.
>
> This patch adds the appropriate definitions that will be needed by
> the kernel.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Benn?e:
>
> * Add comments to clarify CPACR_EL1_ZEN_ELxEN bit meanings.
> * Add comment clarifying the status of the LEN field expansion bits.
> ---
> arch/arm64/include/asm/esr.h | 3 ++-
> arch/arm64/include/asm/kvm_arm.h | 1 +
> arch/arm64/include/asm/sysreg.h | 21 +++++++++++++++++++++
> arch/arm64/kernel/traps.c | 1 +
> 4 files changed, 25 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h
> index 66ed8b6..014d7d8 100644
> --- a/arch/arm64/include/asm/esr.h
> +++ b/arch/arm64/include/asm/esr.h
> @@ -43,7 +43,8 @@
> #define ESR_ELx_EC_HVC64 (0x16)
> #define ESR_ELx_EC_SMC64 (0x17)
> #define ESR_ELx_EC_SYS64 (0x18)
> -/* Unallocated EC: 0x19 - 0x1E */
> +#define ESR_ELx_EC_SVE (0x19)
> +/* Unallocated EC: 0x1A - 0x1E */
> #define ESR_ELx_EC_IMP_DEF (0x1f)
> #define ESR_ELx_EC_IABT_LOW (0x20)
> #define ESR_ELx_EC_IABT_CUR (0x21)
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index 61d694c..dbf0537 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -185,6 +185,7 @@
> #define CPTR_EL2_TCPAC (1 << 31)
> #define CPTR_EL2_TTA (1 << 20)
> #define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
> +#define CPTR_EL2_TZ (1 << 8)
> #define CPTR_EL2_DEFAULT 0x000033ff
>
> /* Hyp Debug Configuration Register bits */
> diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
> index 480ecd6..36fe2ae 100644
> --- a/arch/arm64/include/asm/sysreg.h
> +++ b/arch/arm64/include/asm/sysreg.h
> @@ -145,6 +145,7 @@
>
> #define SYS_ID_AA64PFR0_EL1 sys_reg(3, 0, 0, 4, 0)
> #define SYS_ID_AA64PFR1_EL1 sys_reg(3, 0, 0, 4, 1)
> +#define SYS_ID_AA64ZFR0_EL1 sys_reg(3, 0, 0, 4, 4)
>
> #define SYS_ID_AA64DFR0_EL1 sys_reg(3, 0, 0, 5, 0)
> #define SYS_ID_AA64DFR1_EL1 sys_reg(3, 0, 0, 5, 1)
> @@ -163,6 +164,8 @@
> #define SYS_ACTLR_EL1 sys_reg(3, 0, 1, 0, 1)
> #define SYS_CPACR_EL1 sys_reg(3, 0, 1, 0, 2)
>
> +#define SYS_ZCR_EL1 sys_reg(3, 0, 1, 2, 0)
> +
> #define SYS_TTBR0_EL1 sys_reg(3, 0, 2, 0, 0)
> #define SYS_TTBR1_EL1 sys_reg(3, 0, 2, 0, 1)
> #define SYS_TCR_EL1 sys_reg(3, 0, 2, 0, 2)
> @@ -253,6 +256,8 @@
>
> #define SYS_PMCCFILTR_EL0 sys_reg (3, 3, 14, 15, 7)
>
> +#define SYS_ZCR_EL2 sys_reg(3, 4, 1, 2, 0)
> +
> #define SYS_DACR32_EL2 sys_reg(3, 4, 3, 0, 0)
> #define SYS_IFSR32_EL2 sys_reg(3, 4, 5, 0, 1)
> #define SYS_FPEXC32_EL2 sys_reg(3, 4, 5, 3, 0)
> @@ -335,6 +340,7 @@
> #define ID_AA64ISAR1_DPB_SHIFT 0
>
> /* id_aa64pfr0 */
> +#define ID_AA64PFR0_SVE_SHIFT 32
> #define ID_AA64PFR0_GIC_SHIFT 24
> #define ID_AA64PFR0_ASIMD_SHIFT 20
> #define ID_AA64PFR0_FP_SHIFT 16
> @@ -343,6 +349,7 @@
> #define ID_AA64PFR0_EL1_SHIFT 4
> #define ID_AA64PFR0_EL0_SHIFT 0
>
> +#define ID_AA64PFR0_SVE 0x1
> #define ID_AA64PFR0_FP_NI 0xf
> #define ID_AA64PFR0_FP_SUPPORTED 0x0
> #define ID_AA64PFR0_ASIMD_NI 0xf
> @@ -444,6 +451,20 @@
> #endif
>
>
> +/*
> + * The ZCR_ELx_LEN_* definitions intentionally include bits [8:4] which
> + * are reserved by the SVE architecture for future expansion of the LEN
> + * field, with compatible semantics.
> + */
> +#define ZCR_ELx_LEN_SHIFT 0
> +#define ZCR_ELx_LEN_SIZE 9
> +#define ZCR_ELx_LEN_MASK 0x1ff
> +
> +#define CPACR_EL1_ZEN_EL1EN (1 << 16) /* enable EL1 access */
> +#define CPACR_EL1_ZEN_EL0EN (1 << 17) /* enable EL0 access, if EL1EN set */
> +#define CPACR_EL1_ZEN (CPACR_EL1_ZEN_EL1EN | CPACR_EL1_ZEN_EL0EN)
> +
> +
> /* Safe value for MPIDR_EL1: Bit31:RES1, Bit30:U:0, Bit24:MT:0 */
> #define SYS_MPIDR_SAFE_VAL (1UL << 31)
>
> diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
> index 5ea4b85..f202932 100644
> --- a/arch/arm64/kernel/traps.c
> +++ b/arch/arm64/kernel/traps.c
> @@ -603,6 +603,7 @@ static const char *esr_class_str[] = {
> [ESR_ELx_EC_HVC64] = "HVC (AArch64)",
> [ESR_ELx_EC_SMC64] = "SMC (AArch64)",
> [ESR_ELx_EC_SYS64] = "MSR/MRS (AArch64)",
> + [ESR_ELx_EC_SVE] = "SVE",
> [ESR_ELx_EC_IMP_DEF] = "EL3 IMP DEF",
> [ESR_ELx_EC_IABT_LOW] = "IABT (lower EL)",
> [ESR_ELx_EC_IABT_CUR] = "IABT (current EL)",
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 07/28] arm64/sve: Low-level SVE architectural state manipulation functions
@ 2017-09-13 15:39 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 15:39 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> Manipulating the SVE architectural state, including the vector and
> predicate registers, first-fault register and the vector length,
> requires the use of dedicated instructions added by SVE.
>
> This patch adds suitable assembly functions for saving and
> restoring the SVE registers and querying the vector length.
> Setting of the vector length is done as part of register restore.
>
> Since people building kernels may not all get an SVE-enabled
> toolchain for a while, this patch uses macros that generate
> explicit opcodes in place of assembler mnemonics.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
It took me a while to find a way to properly dissemble the resulting
binaries, in the end needing to run a native objdump in Stretch. I'd
hopped my gdb-multiarch was bleeding edge enough but no ;-)
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Annotate instruction generation macros with the canonical
> architectural syntax so that people can cross-reference more easily
> against the architectural documentation.
> ---
> arch/arm64/include/asm/fpsimd.h | 5 ++
> arch/arm64/include/asm/fpsimdmacros.h | 148 ++++++++++++++++++++++++++++++++++
> arch/arm64/kernel/entry-fpsimd.S | 17 ++++
> 3 files changed, 170 insertions(+)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 410c481..026a7c7 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -67,6 +67,11 @@ extern void fpsimd_update_current_state(struct fpsimd_state *state);
>
> extern void fpsimd_flush_task_state(struct task_struct *target);
>
> +extern void sve_save_state(void *state, u32 *pfpsr);
> +extern void sve_load_state(void const *state, u32 const *pfpsr,
> + unsigned long vq_minus_1);
> +extern unsigned int sve_get_vl(void);
> +
> /* For use by EFI runtime services calls only */
> extern void __efi_fpsimd_begin(void);
> extern void __efi_fpsimd_end(void);
> diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h
> index 0f5fdd3..e050d76 100644
> --- a/arch/arm64/include/asm/fpsimdmacros.h
> +++ b/arch/arm64/include/asm/fpsimdmacros.h
> @@ -75,3 +75,151 @@
> ldr w\tmpnr, [\state, #16 * 2 + 4]
> fpsimd_restore_fpcr x\tmpnr, \state
> .endm
> +
> +/* Sanity-check macros to help avoid encoding garbage instructions */
> +
> +.macro _check_general_reg nr
> + .if (\nr) < 0 || (\nr) > 30
> + .error "Bad register number \nr."
> + .endif
> +.endm
> +
> +.macro _sve_check_zreg znr
> + .if (\znr) < 0 || (\znr) > 31
> + .error "Bad Scalable Vector Extension vector register number \znr."
> + .endif
> +.endm
> +
> +.macro _sve_check_preg pnr
> + .if (\pnr) < 0 || (\pnr) > 15
> + .error "Bad Scalable Vector Extension predicate register number \pnr."
> + .endif
> +.endm
> +
> +.macro _check_num n, min, max
> + .if (\n) < (\min) || (\n) > (\max)
> + .error "Number \n out of range [\min,\max]"
> + .endif
> +.endm
> +
> +/* SVE instruction encodings for non-SVE-capable assemblers */
> +
> +/* STR (vector): STR Z\nz, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_str_v nz, nxbase, offset=0
> + _sve_check_zreg \nz
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0xe5804000 \
> + | (\nz) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* LDR (vector): LDR Z\nz, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_ldr_v nz, nxbase, offset=0
> + _sve_check_zreg \nz
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0x85804000 \
> + | (\nz) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* STR (predicate): STR P\np, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_str_p np, nxbase, offset=0
> + _sve_check_preg \np
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0xe5800000 \
> + | (\np) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* LDR (predicate): LDR P\np, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_ldr_p np, nxbase, offset=0
> + _sve_check_preg \np
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0x85800000 \
> + | (\np) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* RDVL X\nx, #\imm */
> +.macro _sve_rdvl nx, imm
> + _check_general_reg \nx
> + _check_num (\imm), -0x20, 0x1f
> + .inst 0x04bf5000 \
> + | (\nx) \
> + | (((\imm) & 0x3f) << 5)
> +.endm
> +
> +/* RDFFR (unpredicated): RDFFR P\np.B */
> +.macro _sve_rdffr np
> + _sve_check_preg \np
> + .inst 0x2519f000 \
> + | (\np)
> +.endm
> +
> +/* WRFFR P\np.B */
> +.macro _sve_wrffr np
> + _sve_check_preg \np
> + .inst 0x25289000 \
> + | ((\np) << 5)
> +.endm
> +
> +.macro __for from:req, to:req
> + .if (\from) == (\to)
> + _for__body \from
> + .else
> + __for \from, (\from) + ((\to) - (\from)) / 2
> + __for (\from) + ((\to) - (\from)) / 2 + 1, \to
> + .endif
> +.endm
> +
> +.macro _for var:req, from:req, to:req, insn:vararg
> + .macro _for__body \var:req
> + \insn
> + .endm
> +
> + __for \from, \to
> +
> + .purgem _for__body
> +.endm
> +
> +.macro sve_save nxbase, xpfpsr, nxtmp
> + _for n, 0, 31, _sve_str_v \n, \nxbase, \n - 34
> + _for n, 0, 15, _sve_str_p \n, \nxbase, \n - 16
> + _sve_rdffr 0
> + _sve_str_p 0, \nxbase
> + _sve_ldr_p 0, \nxbase, -16
> +
> + mrs x\nxtmp, fpsr
> + str w\nxtmp, [\xpfpsr]
> + mrs x\nxtmp, fpcr
> + str w\nxtmp, [\xpfpsr, #4]
> +.endm
> +
> +.macro sve_load nxbase, xpfpsr, xvqminus1, nxtmp
> + mrs_s x\nxtmp, SYS_ZCR_EL1
> + bic x\nxtmp, x\nxtmp, ZCR_ELx_LEN_MASK
> + orr x\nxtmp, x\nxtmp, \xvqminus1
> + msr_s SYS_ZCR_EL1, x\nxtmp // self-synchronising
> +
> + _for n, 0, 31, _sve_ldr_v \n, \nxbase, \n - 34
> + _sve_ldr_p 0, \nxbase
> + _sve_wrffr 0
> + _for n, 0, 15, _sve_ldr_p \n, \nxbase, \n - 16
> +
> + ldr w\nxtmp, [\xpfpsr]
> + msr fpsr, x\nxtmp
> + ldr w\nxtmp, [\xpfpsr, #4]
> + msr fpcr, x\nxtmp
> +.endm
> diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
> index 6a27cd6..73f17bf 100644
> --- a/arch/arm64/kernel/entry-fpsimd.S
> +++ b/arch/arm64/kernel/entry-fpsimd.S
> @@ -41,3 +41,20 @@ ENTRY(fpsimd_load_state)
> fpsimd_restore x0, 8
> ret
> ENDPROC(fpsimd_load_state)
> +
> +#ifdef CONFIG_ARM64_SVE
> +ENTRY(sve_save_state)
> + sve_save 0, x1, 2
> + ret
> +ENDPROC(sve_save_state)
> +
> +ENTRY(sve_load_state)
> + sve_load 0, x1, x2, 3
> + ret
> +ENDPROC(sve_load_state)
> +
> +ENTRY(sve_get_vl)
> + _sve_rdvl 0, 1
> + ret
> +ENDPROC(sve_get_vl)
> +#endif /* CONFIG_ARM64_SVE */
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 07/28] arm64/sve: Low-level SVE architectural state manipulation functions
@ 2017-09-13 15:39 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 15:39 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> Manipulating the SVE architectural state, including the vector and
> predicate registers, first-fault register and the vector length,
> requires the use of dedicated instructions added by SVE.
>
> This patch adds suitable assembly functions for saving and
> restoring the SVE registers and querying the vector length.
> Setting of the vector length is done as part of register restore.
>
> Since people building kernels may not all get an SVE-enabled
> toolchain for a while, this patch uses macros that generate
> explicit opcodes in place of assembler mnemonics.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
It took me a while to find a way to properly dissemble the resulting
binaries, in the end needing to run a native objdump in Stretch. I'd
hopped my gdb-multiarch was bleeding edge enough but no ;-)
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Annotate instruction generation macros with the canonical
> architectural syntax so that people can cross-reference more easily
> against the architectural documentation.
> ---
> arch/arm64/include/asm/fpsimd.h | 5 ++
> arch/arm64/include/asm/fpsimdmacros.h | 148 ++++++++++++++++++++++++++++++++++
> arch/arm64/kernel/entry-fpsimd.S | 17 ++++
> 3 files changed, 170 insertions(+)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 410c481..026a7c7 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -67,6 +67,11 @@ extern void fpsimd_update_current_state(struct fpsimd_state *state);
>
> extern void fpsimd_flush_task_state(struct task_struct *target);
>
> +extern void sve_save_state(void *state, u32 *pfpsr);
> +extern void sve_load_state(void const *state, u32 const *pfpsr,
> + unsigned long vq_minus_1);
> +extern unsigned int sve_get_vl(void);
> +
> /* For use by EFI runtime services calls only */
> extern void __efi_fpsimd_begin(void);
> extern void __efi_fpsimd_end(void);
> diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h
> index 0f5fdd3..e050d76 100644
> --- a/arch/arm64/include/asm/fpsimdmacros.h
> +++ b/arch/arm64/include/asm/fpsimdmacros.h
> @@ -75,3 +75,151 @@
> ldr w\tmpnr, [\state, #16 * 2 + 4]
> fpsimd_restore_fpcr x\tmpnr, \state
> .endm
> +
> +/* Sanity-check macros to help avoid encoding garbage instructions */
> +
> +.macro _check_general_reg nr
> + .if (\nr) < 0 || (\nr) > 30
> + .error "Bad register number \nr."
> + .endif
> +.endm
> +
> +.macro _sve_check_zreg znr
> + .if (\znr) < 0 || (\znr) > 31
> + .error "Bad Scalable Vector Extension vector register number \znr."
> + .endif
> +.endm
> +
> +.macro _sve_check_preg pnr
> + .if (\pnr) < 0 || (\pnr) > 15
> + .error "Bad Scalable Vector Extension predicate register number \pnr."
> + .endif
> +.endm
> +
> +.macro _check_num n, min, max
> + .if (\n) < (\min) || (\n) > (\max)
> + .error "Number \n out of range [\min,\max]"
> + .endif
> +.endm
> +
> +/* SVE instruction encodings for non-SVE-capable assemblers */
> +
> +/* STR (vector): STR Z\nz, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_str_v nz, nxbase, offset=0
> + _sve_check_zreg \nz
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0xe5804000 \
> + | (\nz) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* LDR (vector): LDR Z\nz, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_ldr_v nz, nxbase, offset=0
> + _sve_check_zreg \nz
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0x85804000 \
> + | (\nz) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* STR (predicate): STR P\np, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_str_p np, nxbase, offset=0
> + _sve_check_preg \np
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0xe5800000 \
> + | (\np) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* LDR (predicate): LDR P\np, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_ldr_p np, nxbase, offset=0
> + _sve_check_preg \np
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0x85800000 \
> + | (\np) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* RDVL X\nx, #\imm */
> +.macro _sve_rdvl nx, imm
> + _check_general_reg \nx
> + _check_num (\imm), -0x20, 0x1f
> + .inst 0x04bf5000 \
> + | (\nx) \
> + | (((\imm) & 0x3f) << 5)
> +.endm
> +
> +/* RDFFR (unpredicated): RDFFR P\np.B */
> +.macro _sve_rdffr np
> + _sve_check_preg \np
> + .inst 0x2519f000 \
> + | (\np)
> +.endm
> +
> +/* WRFFR P\np.B */
> +.macro _sve_wrffr np
> + _sve_check_preg \np
> + .inst 0x25289000 \
> + | ((\np) << 5)
> +.endm
> +
> +.macro __for from:req, to:req
> + .if (\from) == (\to)
> + _for__body \from
> + .else
> + __for \from, (\from) + ((\to) - (\from)) / 2
> + __for (\from) + ((\to) - (\from)) / 2 + 1, \to
> + .endif
> +.endm
> +
> +.macro _for var:req, from:req, to:req, insn:vararg
> + .macro _for__body \var:req
> + \insn
> + .endm
> +
> + __for \from, \to
> +
> + .purgem _for__body
> +.endm
> +
> +.macro sve_save nxbase, xpfpsr, nxtmp
> + _for n, 0, 31, _sve_str_v \n, \nxbase, \n - 34
> + _for n, 0, 15, _sve_str_p \n, \nxbase, \n - 16
> + _sve_rdffr 0
> + _sve_str_p 0, \nxbase
> + _sve_ldr_p 0, \nxbase, -16
> +
> + mrs x\nxtmp, fpsr
> + str w\nxtmp, [\xpfpsr]
> + mrs x\nxtmp, fpcr
> + str w\nxtmp, [\xpfpsr, #4]
> +.endm
> +
> +.macro sve_load nxbase, xpfpsr, xvqminus1, nxtmp
> + mrs_s x\nxtmp, SYS_ZCR_EL1
> + bic x\nxtmp, x\nxtmp, ZCR_ELx_LEN_MASK
> + orr x\nxtmp, x\nxtmp, \xvqminus1
> + msr_s SYS_ZCR_EL1, x\nxtmp // self-synchronising
> +
> + _for n, 0, 31, _sve_ldr_v \n, \nxbase, \n - 34
> + _sve_ldr_p 0, \nxbase
> + _sve_wrffr 0
> + _for n, 0, 15, _sve_ldr_p \n, \nxbase, \n - 16
> +
> + ldr w\nxtmp, [\xpfpsr]
> + msr fpsr, x\nxtmp
> + ldr w\nxtmp, [\xpfpsr, #4]
> + msr fpcr, x\nxtmp
> +.endm
> diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
> index 6a27cd6..73f17bf 100644
> --- a/arch/arm64/kernel/entry-fpsimd.S
> +++ b/arch/arm64/kernel/entry-fpsimd.S
> @@ -41,3 +41,20 @@ ENTRY(fpsimd_load_state)
> fpsimd_restore x0, 8
> ret
> ENDPROC(fpsimd_load_state)
> +
> +#ifdef CONFIG_ARM64_SVE
> +ENTRY(sve_save_state)
> + sve_save 0, x1, 2
> + ret
> +ENDPROC(sve_save_state)
> +
> +ENTRY(sve_load_state)
> + sve_load 0, x1, x2, 3
> + ret
> +ENDPROC(sve_load_state)
> +
> +ENTRY(sve_get_vl)
> + _sve_rdvl 0, 1
> + ret
> +ENDPROC(sve_get_vl)
> +#endif /* CONFIG_ARM64_SVE */
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 07/28] arm64/sve: Low-level SVE architectural state manipulation functions
@ 2017-09-13 15:39 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-13 15:39 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> Manipulating the SVE architectural state, including the vector and
> predicate registers, first-fault register and the vector length,
> requires the use of dedicated instructions added by SVE.
>
> This patch adds suitable assembly functions for saving and
> restoring the SVE registers and querying the vector length.
> Setting of the vector length is done as part of register restore.
>
> Since people building kernels may not all get an SVE-enabled
> toolchain for a while, this patch uses macros that generate
> explicit opcodes in place of assembler mnemonics.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
It took me a while to find a way to properly dissemble the resulting
binaries, in the end needing to run a native objdump in Stretch. I'd
hopped my gdb-multiarch was bleeding edge enough but no ;-)
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Benn?e:
>
> * Annotate instruction generation macros with the canonical
> architectural syntax so that people can cross-reference more easily
> against the architectural documentation.
> ---
> arch/arm64/include/asm/fpsimd.h | 5 ++
> arch/arm64/include/asm/fpsimdmacros.h | 148 ++++++++++++++++++++++++++++++++++
> arch/arm64/kernel/entry-fpsimd.S | 17 ++++
> 3 files changed, 170 insertions(+)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 410c481..026a7c7 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -67,6 +67,11 @@ extern void fpsimd_update_current_state(struct fpsimd_state *state);
>
> extern void fpsimd_flush_task_state(struct task_struct *target);
>
> +extern void sve_save_state(void *state, u32 *pfpsr);
> +extern void sve_load_state(void const *state, u32 const *pfpsr,
> + unsigned long vq_minus_1);
> +extern unsigned int sve_get_vl(void);
> +
> /* For use by EFI runtime services calls only */
> extern void __efi_fpsimd_begin(void);
> extern void __efi_fpsimd_end(void);
> diff --git a/arch/arm64/include/asm/fpsimdmacros.h b/arch/arm64/include/asm/fpsimdmacros.h
> index 0f5fdd3..e050d76 100644
> --- a/arch/arm64/include/asm/fpsimdmacros.h
> +++ b/arch/arm64/include/asm/fpsimdmacros.h
> @@ -75,3 +75,151 @@
> ldr w\tmpnr, [\state, #16 * 2 + 4]
> fpsimd_restore_fpcr x\tmpnr, \state
> .endm
> +
> +/* Sanity-check macros to help avoid encoding garbage instructions */
> +
> +.macro _check_general_reg nr
> + .if (\nr) < 0 || (\nr) > 30
> + .error "Bad register number \nr."
> + .endif
> +.endm
> +
> +.macro _sve_check_zreg znr
> + .if (\znr) < 0 || (\znr) > 31
> + .error "Bad Scalable Vector Extension vector register number \znr."
> + .endif
> +.endm
> +
> +.macro _sve_check_preg pnr
> + .if (\pnr) < 0 || (\pnr) > 15
> + .error "Bad Scalable Vector Extension predicate register number \pnr."
> + .endif
> +.endm
> +
> +.macro _check_num n, min, max
> + .if (\n) < (\min) || (\n) > (\max)
> + .error "Number \n out of range [\min,\max]"
> + .endif
> +.endm
> +
> +/* SVE instruction encodings for non-SVE-capable assemblers */
> +
> +/* STR (vector): STR Z\nz, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_str_v nz, nxbase, offset=0
> + _sve_check_zreg \nz
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0xe5804000 \
> + | (\nz) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* LDR (vector): LDR Z\nz, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_ldr_v nz, nxbase, offset=0
> + _sve_check_zreg \nz
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0x85804000 \
> + | (\nz) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* STR (predicate): STR P\np, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_str_p np, nxbase, offset=0
> + _sve_check_preg \np
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0xe5800000 \
> + | (\np) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* LDR (predicate): LDR P\np, [X\nxbase, #\offset, MUL VL] */
> +.macro _sve_ldr_p np, nxbase, offset=0
> + _sve_check_preg \np
> + _check_general_reg \nxbase
> + _check_num (\offset), -0x100, 0xff
> + .inst 0x85800000 \
> + | (\np) \
> + | ((\nxbase) << 5) \
> + | (((\offset) & 7) << 10) \
> + | (((\offset) & 0x1f8) << 13)
> +.endm
> +
> +/* RDVL X\nx, #\imm */
> +.macro _sve_rdvl nx, imm
> + _check_general_reg \nx
> + _check_num (\imm), -0x20, 0x1f
> + .inst 0x04bf5000 \
> + | (\nx) \
> + | (((\imm) & 0x3f) << 5)
> +.endm
> +
> +/* RDFFR (unpredicated): RDFFR P\np.B */
> +.macro _sve_rdffr np
> + _sve_check_preg \np
> + .inst 0x2519f000 \
> + | (\np)
> +.endm
> +
> +/* WRFFR P\np.B */
> +.macro _sve_wrffr np
> + _sve_check_preg \np
> + .inst 0x25289000 \
> + | ((\np) << 5)
> +.endm
> +
> +.macro __for from:req, to:req
> + .if (\from) == (\to)
> + _for__body \from
> + .else
> + __for \from, (\from) + ((\to) - (\from)) / 2
> + __for (\from) + ((\to) - (\from)) / 2 + 1, \to
> + .endif
> +.endm
> +
> +.macro _for var:req, from:req, to:req, insn:vararg
> + .macro _for__body \var:req
> + \insn
> + .endm
> +
> + __for \from, \to
> +
> + .purgem _for__body
> +.endm
> +
> +.macro sve_save nxbase, xpfpsr, nxtmp
> + _for n, 0, 31, _sve_str_v \n, \nxbase, \n - 34
> + _for n, 0, 15, _sve_str_p \n, \nxbase, \n - 16
> + _sve_rdffr 0
> + _sve_str_p 0, \nxbase
> + _sve_ldr_p 0, \nxbase, -16
> +
> + mrs x\nxtmp, fpsr
> + str w\nxtmp, [\xpfpsr]
> + mrs x\nxtmp, fpcr
> + str w\nxtmp, [\xpfpsr, #4]
> +.endm
> +
> +.macro sve_load nxbase, xpfpsr, xvqminus1, nxtmp
> + mrs_s x\nxtmp, SYS_ZCR_EL1
> + bic x\nxtmp, x\nxtmp, ZCR_ELx_LEN_MASK
> + orr x\nxtmp, x\nxtmp, \xvqminus1
> + msr_s SYS_ZCR_EL1, x\nxtmp // self-synchronising
> +
> + _for n, 0, 31, _sve_ldr_v \n, \nxbase, \n - 34
> + _sve_ldr_p 0, \nxbase
> + _sve_wrffr 0
> + _for n, 0, 15, _sve_ldr_p \n, \nxbase, \n - 16
> +
> + ldr w\nxtmp, [\xpfpsr]
> + msr fpsr, x\nxtmp
> + ldr w\nxtmp, [\xpfpsr, #4]
> + msr fpcr, x\nxtmp
> +.endm
> diff --git a/arch/arm64/kernel/entry-fpsimd.S b/arch/arm64/kernel/entry-fpsimd.S
> index 6a27cd6..73f17bf 100644
> --- a/arch/arm64/kernel/entry-fpsimd.S
> +++ b/arch/arm64/kernel/entry-fpsimd.S
> @@ -41,3 +41,20 @@ ENTRY(fpsimd_load_state)
> fpsimd_restore x0, 8
> ret
> ENDPROC(fpsimd_load_state)
> +
> +#ifdef CONFIG_ARM64_SVE
> +ENTRY(sve_save_state)
> + sve_save 0, x1, 2
> + ret
> +ENDPROC(sve_save_state)
> +
> +ENTRY(sve_load_state)
> + sve_load 0, x1, x2, 3
> + ret
> +ENDPROC(sve_load_state)
> +
> +ENTRY(sve_get_vl)
> + _sve_rdvl 0, 1
> + ret
> +ENDPROC(sve_get_vl)
> +#endif /* CONFIG_ARM64_SVE */
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-08-31 17:00 ` Dave Martin
@ 2017-09-13 17:26 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 17:26 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> +el0_sve_acc:
> + /*
> + * Scalable Vector Extension access
> + */
> + enable_dbg
> + ct_user_exit
> + mov x0, x25
> + mov x1, sp
> + bl do_sve_acc
> + b ret_to_user
I think do_sve_acc() runs with interrupts disabled. We may have some
high latency for large SVE states.
> +/*
> + * Trapped SVE access
> + */
> +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> +{
> + /* Even if we chose not to use SVE, the hardware could still trap: */
> + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> + return;
> + }
> +
> + task_fpsimd_save();
> +
> + sve_alloc(current);
> + fpsimd_to_sve(current);
> + if (test_and_set_thread_flag(TIF_SVE))
> + WARN_ON(1); /* SVE access shouldn't have trapped */
> +
> + task_fpsimd_load();
> +}
When this function is entered, do we expect TIF_SVE to always be
cleared? It's worth adding a comment on the expected conditions. If
that's the case, task_fpsimd_save() would only save the FPSIMD state
which is fine. However, you subsequently transfer the FPSIMD state to
SVE, set TIF_SVE and restore the full SVE state. If we don't care about
the SVE state here, can we call task_fpsimd_load() *before* setting
TIF_SVE?
I may as well have confused myself with the state bouncing between
FPSIMD and SVE (more reasons to document the data flow better ;)).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-13 17:26 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 17:26 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> +el0_sve_acc:
> + /*
> + * Scalable Vector Extension access
> + */
> + enable_dbg
> + ct_user_exit
> + mov x0, x25
> + mov x1, sp
> + bl do_sve_acc
> + b ret_to_user
I think do_sve_acc() runs with interrupts disabled. We may have some
high latency for large SVE states.
> +/*
> + * Trapped SVE access
> + */
> +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> +{
> + /* Even if we chose not to use SVE, the hardware could still trap: */
> + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> + return;
> + }
> +
> + task_fpsimd_save();
> +
> + sve_alloc(current);
> + fpsimd_to_sve(current);
> + if (test_and_set_thread_flag(TIF_SVE))
> + WARN_ON(1); /* SVE access shouldn't have trapped */
> +
> + task_fpsimd_load();
> +}
When this function is entered, do we expect TIF_SVE to always be
cleared? It's worth adding a comment on the expected conditions. If
that's the case, task_fpsimd_save() would only save the FPSIMD state
which is fine. However, you subsequently transfer the FPSIMD state to
SVE, set TIF_SVE and restore the full SVE state. If we don't care about
the SVE state here, can we call task_fpsimd_load() *before* setting
TIF_SVE?
I may as well have confused myself with the state bouncing between
FPSIMD and SVE (more reasons to document the data flow better ;)).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-08-31 17:00 ` Dave Martin
@ 2017-09-13 17:29 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 17:29 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
gdb, Alan Hayward, Yao Qi
On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> This patch implements the core logic for changing a task's vector
> length on request from userspace. This will be used by the ptrace
> and prctl frontends that are implemented in later patches.
>
> The SVE architecture permits, but does not require, implementations
> to support vector lengths that are not a power of two. To handle
> this, logic is added to check a requested vector length against a
> possibly sparse bitmap of available vector lengths at runtime, so
> that the best supported value can be chosen.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
Can this be merged with patch 20? It seems to add the PR_ definitions
which get actually used later when the prctl interface is added.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-13 17:29 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 17:29 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> This patch implements the core logic for changing a task's vector
> length on request from userspace. This will be used by the ptrace
> and prctl frontends that are implemented in later patches.
>
> The SVE architecture permits, but does not require, implementations
> to support vector lengths that are not a power of two. To handle
> this, logic is added to check a requested vector length against a
> possibly sparse bitmap of available vector lengths at runtime, so
> that the best supported value can be chosen.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
Can this be merged with patch 20? It seems to add the PR_ definitions
which get actually used later when the prctl interface is added.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-09-13 17:29 ` Catalin Marinas
@ 2017-09-13 19:06 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 19:06 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Yao Qi, Alan Hayward, Will Deacon, gdb,
Alex Bennée, kvmarm, linux-arm-kernel
On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > This patch implements the core logic for changing a task's vector
> > length on request from userspace. This will be used by the ptrace
> > and prctl frontends that are implemented in later patches.
> >
> > The SVE architecture permits, but does not require, implementations
> > to support vector lengths that are not a power of two. To handle
> > this, logic is added to check a requested vector length against a
> > possibly sparse bitmap of available vector lengths at runtime, so
> > that the best supported value can be chosen.
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Bennée <alex.bennee@linaro.org>
>
> Can this be merged with patch 20? It seems to add the PR_ definitions
> which get actually used later when the prctl interface is added.
This patch is used both by patch 19 and by patch 20, which I preferred
not to merge with each other: ptrace and prctl are significantly
different things.
The prctl bit definitions are added here because they are the canonical
definitions used by both interfaces. The ptrace #defines are based on
them.
Does it make sense if I merge patch 20 into this one and apply patch 19
on top? This avoide the appearance of prctl #defines with no prctl
implementation.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-13 19:06 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 19:06 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > This patch implements the core logic for changing a task's vector
> > length on request from userspace. This will be used by the ptrace
> > and prctl frontends that are implemented in later patches.
> >
> > The SVE architecture permits, but does not require, implementations
> > to support vector lengths that are not a power of two. To handle
> > this, logic is added to check a requested vector length against a
> > possibly sparse bitmap of available vector lengths at runtime, so
> > that the best supported value can be chosen.
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Benn?e <alex.bennee@linaro.org>
>
> Can this be merged with patch 20? It seems to add the PR_ definitions
> which get actually used later when the prctl interface is added.
This patch is used both by patch 19 and by patch 20, which I preferred
not to merge with each other: ptrace and prctl are significantly
different things.
The prctl bit definitions are added here because they are the canonical
definitions used by both interfaces. The ptrace #defines are based on
them.
Does it make sense if I merge patch 20 into this one and apply patch 19
on top? This avoide the appearance of prctl #defines with no prctl
implementation.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-13 17:26 ` Catalin Marinas
@ 2017-09-13 19:17 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 19:17 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > +el0_sve_acc:
> > + /*
> > + * Scalable Vector Extension access
> > + */
> > + enable_dbg
> > + ct_user_exit
> > + mov x0, x25
> > + mov x1, sp
> > + bl do_sve_acc
> > + b ret_to_user
>
> I think do_sve_acc() runs with interrupts disabled. We may have some
> high latency for large SVE states.
Historically I wanted to play safe.
I meant to change this to enable_dbg_and_irq now, but it looks like I
forgot to do it.
I'll double-check that do_sve_acc() does nothing that makes it unsafe to
enable irqs, but it should be OK. It's certainly _intended_ to be
irq-safe.
> > +/*
> > + * Trapped SVE access
> > + */
> > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > +{
> > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > + return;
> > + }
> > +
> > + task_fpsimd_save();
> > +
> > + sve_alloc(current);
> > + fpsimd_to_sve(current);
> > + if (test_and_set_thread_flag(TIF_SVE))
> > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > +
> > + task_fpsimd_load();
> > +}
>
> When this function is entered, do we expect TIF_SVE to always be
> cleared? It's worth adding a comment on the expected conditions. If
Yes, and this is required for correctness, as you observe.
I had a BUG_ON() here which I removed, but it makes sense to add a
comment to capture the precondition here, and how it is satisfied.
> that's the case, task_fpsimd_save() would only save the FPSIMD state
> which is fine. However, you subsequently transfer the FPSIMD state to
> SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> the SVE state here, can we call task_fpsimd_load() *before* setting
> TIF_SVE?
There should be no way to reach this code with TIF_SVE set, unless
task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
broken -- either of which is a bug.
> I may as well have confused myself with the state bouncing between
> FPSIMD and SVE (more reasons to document the data flow better ;)).
Agreed, I think this does need more explanation... I'm currently
trying to figure out the best way to describe it.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-13 19:17 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 19:17 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > +el0_sve_acc:
> > + /*
> > + * Scalable Vector Extension access
> > + */
> > + enable_dbg
> > + ct_user_exit
> > + mov x0, x25
> > + mov x1, sp
> > + bl do_sve_acc
> > + b ret_to_user
>
> I think do_sve_acc() runs with interrupts disabled. We may have some
> high latency for large SVE states.
Historically I wanted to play safe.
I meant to change this to enable_dbg_and_irq now, but it looks like I
forgot to do it.
I'll double-check that do_sve_acc() does nothing that makes it unsafe to
enable irqs, but it should be OK. It's certainly _intended_ to be
irq-safe.
> > +/*
> > + * Trapped SVE access
> > + */
> > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > +{
> > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > + return;
> > + }
> > +
> > + task_fpsimd_save();
> > +
> > + sve_alloc(current);
> > + fpsimd_to_sve(current);
> > + if (test_and_set_thread_flag(TIF_SVE))
> > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > +
> > + task_fpsimd_load();
> > +}
>
> When this function is entered, do we expect TIF_SVE to always be
> cleared? It's worth adding a comment on the expected conditions. If
Yes, and this is required for correctness, as you observe.
I had a BUG_ON() here which I removed, but it makes sense to add a
comment to capture the precondition here, and how it is satisfied.
> that's the case, task_fpsimd_save() would only save the FPSIMD state
> which is fine. However, you subsequently transfer the FPSIMD state to
> SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> the SVE state here, can we call task_fpsimd_load() *before* setting
> TIF_SVE?
There should be no way to reach this code with TIF_SVE set, unless
task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
broken -- either of which is a bug.
> I may as well have confused myself with the state bouncing between
> FPSIMD and SVE (more reasons to document the data flow better ;)).
Agreed, I think this does need more explanation... I'm currently
trying to figure out the best way to describe it.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-09-13 19:21 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 19:21 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, kvmarm, linux-arm-kernel
On Wed, Sep 13, 2017 at 06:32:06AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> > diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> > index 877d42f..dd22ef2 100644
> > --- a/arch/arm64/mm/proc.S
> > +++ b/arch/arm64/mm/proc.S
> > @@ -27,6 +27,7 @@
> > #include <asm/pgtable-hwdef.h>
> > #include <asm/cpufeature.h>
> > #include <asm/alternative.h>
> > +#include <asm/sysreg.h>
> >
> > #ifdef CONFIG_ARM64_64K_PAGES
> > #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> > @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> > tlbi vmalle1 // Invalidate local TLB
> > dsb nsh
> >
> > - mov x0, #3 << 20
> > - msr cpacr_el1, x0 // Enable FP/ASIMD
> > + mov x0, #3 << 20 // FEN
> > +
> > + /* SVE */
> > + mrs x5, id_aa64pfr0_el1
> > + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> > + cbz x5, 1f
> > +
> > + bic x0, x0, #CPACR_EL1_ZEN
> > + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> > +1: msr cpacr_el1, x0 // Enable FP/ASIMD
>
> For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
> I tried to do the same with FPSIMD but hit an issue with EFI run-time
> services (I may be wrong though).
I'll take a look at this -- I believe it should be safe to disable this
trap for EL1 relatively late. This is needed before probing for
available vector lengths, but apart from that the kernel shouldn't touch
SVE until/unless some user task uses SVE.
This would change if we eventually enable kernel-mode SVE, but I wouldn't
expect that to get used in early boot before the cpufeatures code runs.
Ard may have a view on this.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-09-13 19:21 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 19:21 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 13, 2017 at 06:32:06AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> > diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> > index 877d42f..dd22ef2 100644
> > --- a/arch/arm64/mm/proc.S
> > +++ b/arch/arm64/mm/proc.S
> > @@ -27,6 +27,7 @@
> > #include <asm/pgtable-hwdef.h>
> > #include <asm/cpufeature.h>
> > #include <asm/alternative.h>
> > +#include <asm/sysreg.h>
> >
> > #ifdef CONFIG_ARM64_64K_PAGES
> > #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> > @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> > tlbi vmalle1 // Invalidate local TLB
> > dsb nsh
> >
> > - mov x0, #3 << 20
> > - msr cpacr_el1, x0 // Enable FP/ASIMD
> > + mov x0, #3 << 20 // FEN
> > +
> > + /* SVE */
> > + mrs x5, id_aa64pfr0_el1
> > + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> > + cbz x5, 1f
> > +
> > + bic x0, x0, #CPACR_EL1_ZEN
> > + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> > +1: msr cpacr_el1, x0 // Enable FP/ASIMD
>
> For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
> I tried to do the same with FPSIMD but hit an issue with EFI run-time
> services (I may be wrong though).
I'll take a look at this -- I believe it should be safe to disable this
trap for EL1 relatively late. This is needed before probing for
available vector lengths, but apart from that the kernel shouldn't touch
SVE until/unless some user task uses SVE.
This would change if we eventually enable kernel-mode SVE, but I wouldn't
expect that to get used in early boot before the cpufeatures code runs.
Ard may have a view on this.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-09-13 19:21 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 19:21 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 06:32:06AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> > diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> > index 877d42f..dd22ef2 100644
> > --- a/arch/arm64/mm/proc.S
> > +++ b/arch/arm64/mm/proc.S
> > @@ -27,6 +27,7 @@
> > #include <asm/pgtable-hwdef.h>
> > #include <asm/cpufeature.h>
> > #include <asm/alternative.h>
> > +#include <asm/sysreg.h>
> >
> > #ifdef CONFIG_ARM64_64K_PAGES
> > #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> > @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> > tlbi vmalle1 // Invalidate local TLB
> > dsb nsh
> >
> > - mov x0, #3 << 20
> > - msr cpacr_el1, x0 // Enable FP/ASIMD
> > + mov x0, #3 << 20 // FEN
> > +
> > + /* SVE */
> > + mrs x5, id_aa64pfr0_el1
> > + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> > + cbz x5, 1f
> > +
> > + bic x0, x0, #CPACR_EL1_ZEN
> > + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> > +1: msr cpacr_el1, x0 // Enable FP/ASIMD
>
> For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
> I tried to do the same with FPSIMD but hit an issue with EFI run-time
> services (I may be wrong though).
I'll take a look at this -- I believe it should be safe to disable this
trap for EL1 relatively late. This is needed before probing for
available vector lengths, but apart from that the kernel shouldn't touch
SVE until/unless some user task uses SVE.
This would change if we eventually enable kernel-mode SVE, but I wouldn't
expect that to get used in early boot before the cpufeatures code runs.
Ard may have a view on this.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 09/28] arm64/sve: Signal frame and context structure definition
2017-09-13 13:36 ` Catalin Marinas
@ 2017-09-13 21:33 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 21:33 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy, gdb,
Yao Qi, Will Deacon, Richard Sandiford, Alan Hayward,
Alex Bennée, kvmarm, linux-arm-kernel
On Wed, Sep 13, 2017 at 06:36:18AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:41PM +0100, Dave P Martin wrote:
> > +/*
> > + * The SVE architecture leaves space for future expansion of the
> > + * vector length beyond its initial architectural limit of 2048 bits
> > + * (16 quadwords).
> > + */
> > +#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
> > +
> > +#define SVE_VQ_MIN 1
> > +#define SVE_VQ_MAX 0x200
>
> Just a nitpick (up to you): could you use 16 and 512 here instead of
> hex? I usually associate hex numbers with some bit fields.
I have no strong opinion other than a desire to make these constants
typo-proof.
There's no particular reason why these shouldn't be in decimal, so I can
change them if you like, provided you promise to notice if I misspell
512 as 521...
(git grep 131027)
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 09/28] arm64/sve: Signal frame and context structure definition
@ 2017-09-13 21:33 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-13 21:33 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 06:36:18AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:41PM +0100, Dave P Martin wrote:
> > +/*
> > + * The SVE architecture leaves space for future expansion of the
> > + * vector length beyond its initial architectural limit of 2048 bits
> > + * (16 quadwords).
> > + */
> > +#define SVE_VQ_BYTES 0x10 /* number of bytes per quadword */
> > +
> > +#define SVE_VQ_MIN 1
> > +#define SVE_VQ_MAX 0x200
>
> Just a nitpick (up to you): could you use 16 and 512 here instead of
> hex? I usually associate hex numbers with some bit fields.
I have no strong opinion other than a desire to make these constants
typo-proof.
There's no particular reason why these shouldn't be in decimal, so I can
change them if you like, provided you promise to notice if I misspell
512 as 521...
(git grep 131027)
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-09-13 19:06 ` Dave Martin
@ 2017-09-13 22:11 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 22:11 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, gdb, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Yao Qi, Will Deacon, Alan Hayward,
Alex Bennée, kvmarm, linux-arm-kernel
On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > > This patch implements the core logic for changing a task's vector
> > > length on request from userspace. This will be used by the ptrace
> > > and prctl frontends that are implemented in later patches.
> > >
> > > The SVE architecture permits, but does not require, implementations
> > > to support vector lengths that are not a power of two. To handle
> > > this, logic is added to check a requested vector length against a
> > > possibly sparse bitmap of available vector lengths at runtime, so
> > > that the best supported value can be chosen.
> > >
> > > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > > Cc: Alex Bennée <alex.bennee@linaro.org>
> >
> > Can this be merged with patch 20? It seems to add the PR_ definitions
> > which get actually used later when the prctl interface is added.
>
> This patch is used both by patch 19 and by patch 20, which I preferred
> not to merge with each other: ptrace and prctl are significantly
> different things.
>
> The prctl bit definitions are added here because they are the canonical
> definitions used by both interfaces. The ptrace #defines are based on
> them.
>
> Does it make sense if I merge patch 20 into this one and apply patch 19
> on top? This avoide the appearance of prctl #defines with no prctl
> implementation.
That's fine, you can bring patch 20 forward. If there are other
non-trivial issues, feel free to ignore my comment.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-13 22:11 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 22:11 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > > This patch implements the core logic for changing a task's vector
> > > length on request from userspace. This will be used by the ptrace
> > > and prctl frontends that are implemented in later patches.
> > >
> > > The SVE architecture permits, but does not require, implementations
> > > to support vector lengths that are not a power of two. To handle
> > > this, logic is added to check a requested vector length against a
> > > possibly sparse bitmap of available vector lengths at runtime, so
> > > that the best supported value can be chosen.
> > >
> > > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > > Cc: Alex Benn?e <alex.bennee@linaro.org>
> >
> > Can this be merged with patch 20? It seems to add the PR_ definitions
> > which get actually used later when the prctl interface is added.
>
> This patch is used both by patch 19 and by patch 20, which I preferred
> not to merge with each other: ptrace and prctl are significantly
> different things.
>
> The prctl bit definitions are added here because they are the canonical
> definitions used by both interfaces. The ptrace #defines are based on
> them.
>
> Does it make sense if I merge patch 20 into this one and apply patch 19
> on top? This avoide the appearance of prctl #defines with no prctl
> implementation.
That's fine, you can bring patch 20 forward. If there are other
non-trivial issues, feel free to ignore my comment.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-13 19:17 ` Dave Martin
@ 2017-09-13 22:21 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 22:21 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 13, 2017 at 08:17:07PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > +/*
> > > + * Trapped SVE access
> > > + */
> > > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > > +{
> > > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > > + return;
> > > + }
> > > +
> > > + task_fpsimd_save();
> > > +
> > > + sve_alloc(current);
> > > + fpsimd_to_sve(current);
> > > + if (test_and_set_thread_flag(TIF_SVE))
> > > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > > +
> > > + task_fpsimd_load();
> > > +}
> >
> > When this function is entered, do we expect TIF_SVE to always be
> > cleared? It's worth adding a comment on the expected conditions. If
>
> Yes, and this is required for correctness, as you observe.
>
> I had a BUG_ON() here which I removed, but it makes sense to add a
> comment to capture the precondition here, and how it is satisfied.
>
> > that's the case, task_fpsimd_save() would only save the FPSIMD state
> > which is fine. However, you subsequently transfer the FPSIMD state to
> > SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> > the SVE state here, can we call task_fpsimd_load() *before* setting
> > TIF_SVE?
>
> There should be no way to reach this code with TIF_SVE set, unless
> task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
> broken -- either of which is a bug.
Thanks for confirming my assumptions. What I meant was rewriting the
above function as:
/* reset the SVE state (other than FPSIMD) */
task_fpsimd_save();
task_fpsimd_load();
sve_alloc(current);
set_thread_flag(TIF_SVE);
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-13 22:21 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-13 22:21 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 08:17:07PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > +/*
> > > + * Trapped SVE access
> > > + */
> > > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > > +{
> > > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > > + return;
> > > + }
> > > +
> > > + task_fpsimd_save();
> > > +
> > > + sve_alloc(current);
> > > + fpsimd_to_sve(current);
> > > + if (test_and_set_thread_flag(TIF_SVE))
> > > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > > +
> > > + task_fpsimd_load();
> > > +}
> >
> > When this function is entered, do we expect TIF_SVE to always be
> > cleared? It's worth adding a comment on the expected conditions. If
>
> Yes, and this is required for correctness, as you observe.
>
> I had a BUG_ON() here which I removed, but it makes sense to add a
> comment to capture the precondition here, and how it is satisfied.
>
> > that's the case, task_fpsimd_save() would only save the FPSIMD state
> > which is fine. However, you subsequently transfer the FPSIMD state to
> > SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> > the SVE state here, can we call task_fpsimd_load() *before* setting
> > TIF_SVE?
>
> There should be no way to reach this code with TIF_SVE set, unless
> task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
> broken -- either of which is a bug.
Thanks for confirming my assumptions. What I meant was rewriting the
above function as:
/* reset the SVE state (other than FPSIMD) */
task_fpsimd_save();
task_fpsimd_load();
sve_alloc(current);
set_thread_flag(TIF_SVE);
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 12/28] arm64/sve: Support vector length resetting for new processes
@ 2017-09-14 8:47 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 8:47 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> It's desirable to be able to reset the vector length to some sane
> default for new processes, since the new binary and its libraries
> processes may or may not be SVE-aware.
>
> This patch tracks the desired post-exec vector length (if any) in a
> new thread member sve_vl_onexec, and adds a new thread flag
> TIF_SVE_VL_INHERIT to control whether to inherit or reset the
> vector length. Currently these are inactive. Subsequent patches
> will provide the capability to configure them.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Make sve_vl_onexec an unsigned int: that's the type used virtually
> everywhere else, and the memory saving is too minor to be interesting.
> ---
> arch/arm64/include/asm/processor.h | 1 +
> arch/arm64/include/asm/thread_info.h | 1 +
> arch/arm64/kernel/fpsimd.c | 16 ++++++++++++----
> 3 files changed, 14 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
> index 4831d28..3faceac 100644
> --- a/arch/arm64/include/asm/processor.h
> +++ b/arch/arm64/include/asm/processor.h
> @@ -87,6 +87,7 @@ struct thread_struct {
> struct fpsimd_state fpsimd_state;
> void *sve_state; /* SVE registers, if any */
> unsigned int sve_vl; /* SVE vector length */
> + unsigned int sve_vl_onexec; /* SVE vl after next exec */
> unsigned long fault_address; /* fault info */
> unsigned long fault_code; /* ESR_EL1 value */
> struct debug_info debug; /* debugging */
> diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
> index f0880fc..d3568ab 100644
> --- a/arch/arm64/include/asm/thread_info.h
> +++ b/arch/arm64/include/asm/thread_info.h
> @@ -92,6 +92,7 @@ void arch_setup_new_exec(void);
> #define TIF_SINGLESTEP 21
> #define TIF_32BIT 22 /* 32bit process */
> #define TIF_SVE 23 /* Scalable Vector Extension in use */
> +#define TIF_SVE_VL_INHERIT 24 /* Inherit sve_vl_onexec across exec */
>
> #define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
> #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 9b1ebd7..e20b44d 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -106,6 +106,9 @@
> */
> static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
>
> +/* Default VL for tasks that don't set it explicitly: */
> +static int sve_default_vl = SVE_VL_MIN;
> +
> static void sve_free(struct task_struct *task)
> {
> kfree(task->thread.sve_state);
> @@ -380,15 +383,20 @@ void fpsimd_flush_thread(void)
> * If a bug causes this to go wrong, we make some noise and
> * try to fudge thread.sve_vl to a safe value here.
> */
> - vl = current->thread.sve_vl;
> -
> - if (vl == 0)
> - vl = SVE_VL_MIN;
> + vl = current->thread.sve_vl_onexec ?
> + current->thread.sve_vl_onexec : sve_default_vl;
>
> if (WARN_ON(!sve_vl_valid(vl)))
> vl = SVE_VL_MIN;
>
> current->thread.sve_vl = vl;
> +
> + /*
> + * If the task is not set to inherit, ensure that the vector
> + * length will be reset by a subsequent exec:
> + */
> + if (!test_thread_flag(TIF_SVE_VL_INHERIT))
> + current->thread.sve_vl_onexec = 0;
> }
>
> set_thread_flag(TIF_FOREIGN_FPSTATE);
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 12/28] arm64/sve: Support vector length resetting for new processes
@ 2017-09-14 8:47 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 8:47 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> It's desirable to be able to reset the vector length to some sane
> default for new processes, since the new binary and its libraries
> processes may or may not be SVE-aware.
>
> This patch tracks the desired post-exec vector length (if any) in a
> new thread member sve_vl_onexec, and adds a new thread flag
> TIF_SVE_VL_INHERIT to control whether to inherit or reset the
> vector length. Currently these are inactive. Subsequent patches
> will provide the capability to configure them.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Make sve_vl_onexec an unsigned int: that's the type used virtually
> everywhere else, and the memory saving is too minor to be interesting.
> ---
> arch/arm64/include/asm/processor.h | 1 +
> arch/arm64/include/asm/thread_info.h | 1 +
> arch/arm64/kernel/fpsimd.c | 16 ++++++++++++----
> 3 files changed, 14 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
> index 4831d28..3faceac 100644
> --- a/arch/arm64/include/asm/processor.h
> +++ b/arch/arm64/include/asm/processor.h
> @@ -87,6 +87,7 @@ struct thread_struct {
> struct fpsimd_state fpsimd_state;
> void *sve_state; /* SVE registers, if any */
> unsigned int sve_vl; /* SVE vector length */
> + unsigned int sve_vl_onexec; /* SVE vl after next exec */
> unsigned long fault_address; /* fault info */
> unsigned long fault_code; /* ESR_EL1 value */
> struct debug_info debug; /* debugging */
> diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
> index f0880fc..d3568ab 100644
> --- a/arch/arm64/include/asm/thread_info.h
> +++ b/arch/arm64/include/asm/thread_info.h
> @@ -92,6 +92,7 @@ void arch_setup_new_exec(void);
> #define TIF_SINGLESTEP 21
> #define TIF_32BIT 22 /* 32bit process */
> #define TIF_SVE 23 /* Scalable Vector Extension in use */
> +#define TIF_SVE_VL_INHERIT 24 /* Inherit sve_vl_onexec across exec */
>
> #define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
> #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 9b1ebd7..e20b44d 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -106,6 +106,9 @@
> */
> static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
>
> +/* Default VL for tasks that don't set it explicitly: */
> +static int sve_default_vl = SVE_VL_MIN;
> +
> static void sve_free(struct task_struct *task)
> {
> kfree(task->thread.sve_state);
> @@ -380,15 +383,20 @@ void fpsimd_flush_thread(void)
> * If a bug causes this to go wrong, we make some noise and
> * try to fudge thread.sve_vl to a safe value here.
> */
> - vl = current->thread.sve_vl;
> -
> - if (vl == 0)
> - vl = SVE_VL_MIN;
> + vl = current->thread.sve_vl_onexec ?
> + current->thread.sve_vl_onexec : sve_default_vl;
>
> if (WARN_ON(!sve_vl_valid(vl)))
> vl = SVE_VL_MIN;
>
> current->thread.sve_vl = vl;
> +
> + /*
> + * If the task is not set to inherit, ensure that the vector
> + * length will be reset by a subsequent exec:
> + */
> + if (!test_thread_flag(TIF_SVE_VL_INHERIT))
> + current->thread.sve_vl_onexec = 0;
> }
>
> set_thread_flag(TIF_FOREIGN_FPSTATE);
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 12/28] arm64/sve: Support vector length resetting for new processes
@ 2017-09-14 8:47 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 8:47 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> It's desirable to be able to reset the vector length to some sane
> default for new processes, since the new binary and its libraries
> processes may or may not be SVE-aware.
>
> This patch tracks the desired post-exec vector length (if any) in a
> new thread member sve_vl_onexec, and adds a new thread flag
> TIF_SVE_VL_INHERIT to control whether to inherit or reset the
> vector length. Currently these are inactive. Subsequent patches
> will provide the capability to configure them.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Benn?e:
>
> * Make sve_vl_onexec an unsigned int: that's the type used virtually
> everywhere else, and the memory saving is too minor to be interesting.
> ---
> arch/arm64/include/asm/processor.h | 1 +
> arch/arm64/include/asm/thread_info.h | 1 +
> arch/arm64/kernel/fpsimd.c | 16 ++++++++++++----
> 3 files changed, 14 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
> index 4831d28..3faceac 100644
> --- a/arch/arm64/include/asm/processor.h
> +++ b/arch/arm64/include/asm/processor.h
> @@ -87,6 +87,7 @@ struct thread_struct {
> struct fpsimd_state fpsimd_state;
> void *sve_state; /* SVE registers, if any */
> unsigned int sve_vl; /* SVE vector length */
> + unsigned int sve_vl_onexec; /* SVE vl after next exec */
> unsigned long fault_address; /* fault info */
> unsigned long fault_code; /* ESR_EL1 value */
> struct debug_info debug; /* debugging */
> diff --git a/arch/arm64/include/asm/thread_info.h b/arch/arm64/include/asm/thread_info.h
> index f0880fc..d3568ab 100644
> --- a/arch/arm64/include/asm/thread_info.h
> +++ b/arch/arm64/include/asm/thread_info.h
> @@ -92,6 +92,7 @@ void arch_setup_new_exec(void);
> #define TIF_SINGLESTEP 21
> #define TIF_32BIT 22 /* 32bit process */
> #define TIF_SVE 23 /* Scalable Vector Extension in use */
> +#define TIF_SVE_VL_INHERIT 24 /* Inherit sve_vl_onexec across exec */
>
> #define _TIF_SIGPENDING (1 << TIF_SIGPENDING)
> #define _TIF_NEED_RESCHED (1 << TIF_NEED_RESCHED)
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 9b1ebd7..e20b44d 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -106,6 +106,9 @@
> */
> static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
>
> +/* Default VL for tasks that don't set it explicitly: */
> +static int sve_default_vl = SVE_VL_MIN;
> +
> static void sve_free(struct task_struct *task)
> {
> kfree(task->thread.sve_state);
> @@ -380,15 +383,20 @@ void fpsimd_flush_thread(void)
> * If a bug causes this to go wrong, we make some noise and
> * try to fudge thread.sve_vl to a safe value here.
> */
> - vl = current->thread.sve_vl;
> -
> - if (vl == 0)
> - vl = SVE_VL_MIN;
> + vl = current->thread.sve_vl_onexec ?
> + current->thread.sve_vl_onexec : sve_default_vl;
>
> if (WARN_ON(!sve_vl_valid(vl)))
> vl = SVE_VL_MIN;
>
> current->thread.sve_vl = vl;
> +
> + /*
> + * If the task is not set to inherit, ensure that the vector
> + * length will be reset by a subsequent exec:
> + */
> + if (!test_thread_flag(TIF_SVE_VL_INHERIT))
> + current->thread.sve_vl_onexec = 0;
> }
>
> set_thread_flag(TIF_FOREIGN_FPSTATE);
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 13/28] arm64/sve: Signal handling support
@ 2017-09-14 9:30 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:30 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> This patch implements support for saving and restoring the SVE
> registers around signals.
>
> A fixed-size header struct sve_context is always included in the
> signal frame encoding the thread's vector length at the time of
> signal delivery, optionally followed by a variable-layout structure
> encoding the SVE registers.
>
> Because of the need to preserve backwards compatibility, the FPSIMD
> view of the SVE registers is always dumped as a struct
> fpsimd_context in the usual way, in addition to any sve_context.
>
> The SVE vector registers are dumped in full, including bits 127:0
> of each register which alias the corresponding FPSIMD vector
> registers in the hardware. To avoid any ambiguity about which
> alias to restore during sigreturn, the kernel always restores bits
> 127:0 of each SVE vector register from the fpsimd_context in the
> signal frame (which must be present): userspace needs to take this
> into account if it wants to modify the SVE vector register contents
> on return from a signal.
>
> FPSR and FPCR, which are used by both FPSIMD and SVE, are not
> included in sve_context because they are always present in
> fpsimd_context anyway.
>
> For signal delivery, a new helper
> fpsimd_signal_preserve_current_state() is added to update _both_
> the FPSIMD and SVE views in the task struct, to make it easier to
> populate this information into the signal frame. Because of the
> redundancy between the two views of the state, only one is updated
> otherwise. In order to avoid racing with a pending discard of the
> SVE state, this flush is hoisted before the sigframe layout phase,
> so that the layout and population phases see a consistent view of
> the thread.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Alex Bennée <alex.bennee@linaro.org>
>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 13/28] arm64/sve: Signal handling support
@ 2017-09-14 9:30 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:30 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> This patch implements support for saving and restoring the SVE
> registers around signals.
>
> A fixed-size header struct sve_context is always included in the
> signal frame encoding the thread's vector length at the time of
> signal delivery, optionally followed by a variable-layout structure
> encoding the SVE registers.
>
> Because of the need to preserve backwards compatibility, the FPSIMD
> view of the SVE registers is always dumped as a struct
> fpsimd_context in the usual way, in addition to any sve_context.
>
> The SVE vector registers are dumped in full, including bits 127:0
> of each register which alias the corresponding FPSIMD vector
> registers in the hardware. To avoid any ambiguity about which
> alias to restore during sigreturn, the kernel always restores bits
> 127:0 of each SVE vector register from the fpsimd_context in the
> signal frame (which must be present): userspace needs to take this
> into account if it wants to modify the SVE vector register contents
> on return from a signal.
>
> FPSR and FPCR, which are used by both FPSIMD and SVE, are not
> included in sve_context because they are always present in
> fpsimd_context anyway.
>
> For signal delivery, a new helper
> fpsimd_signal_preserve_current_state() is added to update _both_
> the FPSIMD and SVE views in the task struct, to make it easier to
> populate this information into the signal frame. Because of the
> redundancy between the two views of the state, only one is updated
> otherwise. In order to avoid racing with a pending discard of the
> SVE state, this flush is hoisted before the sigframe layout phase,
> so that the layout and population phases see a consistent view of
> the thread.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Alex Bennée <alex.bennee@linaro.org>
>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 13/28] arm64/sve: Signal handling support
@ 2017-09-14 9:30 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:30 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> This patch implements support for saving and restoring the SVE
> registers around signals.
>
> A fixed-size header struct sve_context is always included in the
> signal frame encoding the thread's vector length at the time of
> signal delivery, optionally followed by a variable-layout structure
> encoding the SVE registers.
>
> Because of the need to preserve backwards compatibility, the FPSIMD
> view of the SVE registers is always dumped as a struct
> fpsimd_context in the usual way, in addition to any sve_context.
>
> The SVE vector registers are dumped in full, including bits 127:0
> of each register which alias the corresponding FPSIMD vector
> registers in the hardware. To avoid any ambiguity about which
> alias to restore during sigreturn, the kernel always restores bits
> 127:0 of each SVE vector register from the fpsimd_context in the
> signal frame (which must be present): userspace needs to take this
> into account if it wants to modify the SVE vector register contents
> on return from a signal.
>
> FPSR and FPCR, which are used by both FPSIMD and SVE, are not
> included in sve_context because they are always present in
> fpsimd_context anyway.
>
> For signal delivery, a new helper
> fpsimd_signal_preserve_current_state() is added to update _both_
> the FPSIMD and SVE views in the task struct, to make it easier to
> populate this information into the signal frame. Because of the
> redundancy between the two views of the state, only one is updated
> otherwise. In order to avoid racing with a pending discard of the
> SVE state, this flush is hoisted before the sigframe layout phase,
> so that the layout and population phases see a consistent view of
> the thread.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations
@ 2017-09-14 9:33 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:33 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Suzuki K Poulose
Dave Martin <Dave.Martin@arm.com> writes:
> update_cpu_features() currently cannot tell whether it is being
> called during early or late secondary boot. This doesn't
> desperately matter for anything it currently does.
>
> However, SVE will need to know here whether the set of available
> vector lengths is fixed of still to be determined when booting a
> CPU so that it can be updated appropriately.
>
> This patch simply moves the sys_caps_initialised stuff to the top
> of the file so that it can be more widely. There doesn't seem to
> be a more obvious place to put it.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++---------------
> 1 file changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index cd52d36..43ba8df 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -51,6 +51,21 @@ unsigned int compat_elf_hwcap2 __read_mostly;
> DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
> EXPORT_SYMBOL(cpu_hwcaps);
>
> +/*
> + * Flag to indicate if we have computed the system wide
> + * capabilities based on the boot time active CPUs. This
> + * will be used to determine if a new booting CPU should
> + * go through the verification process to make sure that it
> + * supports the system capabilities, without using a hotplug
> + * notifier.
> + */
> +static bool sys_caps_initialised;
> +
> +static inline void set_sys_caps_initialised(void)
> +{
> + sys_caps_initialised = true;
> +}
> +
> static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
> {
> /* file-wide pr_fmt adds "CPU features: " prefix */
> @@ -1041,21 +1056,6 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
> }
>
> /*
> - * Flag to indicate if we have computed the system wide
> - * capabilities based on the boot time active CPUs. This
> - * will be used to determine if a new booting CPU should
> - * go through the verification process to make sure that it
> - * supports the system capabilities, without using a hotplug
> - * notifier.
> - */
> -static bool sys_caps_initialised;
> -
> -static inline void set_sys_caps_initialised(void)
> -{
> - sys_caps_initialised = true;
> -}
> -
> -/*
> * Check for CPU features that are used in early boot
> * based on the Boot CPU value.
> */
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations
@ 2017-09-14 9:33 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:33 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Suzuki K Poulose
Dave Martin <Dave.Martin@arm.com> writes:
> update_cpu_features() currently cannot tell whether it is being
> called during early or late secondary boot. This doesn't
> desperately matter for anything it currently does.
>
> However, SVE will need to know here whether the set of available
> vector lengths is fixed of still to be determined when booting a
> CPU so that it can be updated appropriately.
>
> This patch simply moves the sys_caps_initialised stuff to the top
> of the file so that it can be more widely. There doesn't seem to
> be a more obvious place to put it.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++---------------
> 1 file changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index cd52d36..43ba8df 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -51,6 +51,21 @@ unsigned int compat_elf_hwcap2 __read_mostly;
> DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
> EXPORT_SYMBOL(cpu_hwcaps);
>
> +/*
> + * Flag to indicate if we have computed the system wide
> + * capabilities based on the boot time active CPUs. This
> + * will be used to determine if a new booting CPU should
> + * go through the verification process to make sure that it
> + * supports the system capabilities, without using a hotplug
> + * notifier.
> + */
> +static bool sys_caps_initialised;
> +
> +static inline void set_sys_caps_initialised(void)
> +{
> + sys_caps_initialised = true;
> +}
> +
> static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
> {
> /* file-wide pr_fmt adds "CPU features: " prefix */
> @@ -1041,21 +1056,6 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
> }
>
> /*
> - * Flag to indicate if we have computed the system wide
> - * capabilities based on the boot time active CPUs. This
> - * will be used to determine if a new booting CPU should
> - * go through the verification process to make sure that it
> - * supports the system capabilities, without using a hotplug
> - * notifier.
> - */
> -static bool sys_caps_initialised;
> -
> -static inline void set_sys_caps_initialised(void)
> -{
> - sys_caps_initialised = true;
> -}
> -
> -/*
> * Check for CPU features that are used in early boot
> * based on the Boot CPU value.
> */
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations
@ 2017-09-14 9:33 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:33 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> update_cpu_features() currently cannot tell whether it is being
> called during early or late secondary boot. This doesn't
> desperately matter for anything it currently does.
>
> However, SVE will need to know here whether the set of available
> vector lengths is fixed of still to be determined when booting a
> CPU so that it can be updated appropriately.
>
> This patch simply moves the sys_caps_initialised stuff to the top
> of the file so that it can be more widely. There doesn't seem to
> be a more obvious place to put it.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
> ---
> arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++---------------
> 1 file changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index cd52d36..43ba8df 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -51,6 +51,21 @@ unsigned int compat_elf_hwcap2 __read_mostly;
> DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
> EXPORT_SYMBOL(cpu_hwcaps);
>
> +/*
> + * Flag to indicate if we have computed the system wide
> + * capabilities based on the boot time active CPUs. This
> + * will be used to determine if a new booting CPU should
> + * go through the verification process to make sure that it
> + * supports the system capabilities, without using a hotplug
> + * notifier.
> + */
> +static bool sys_caps_initialised;
> +
> +static inline void set_sys_caps_initialised(void)
> +{
> + sys_caps_initialised = true;
> +}
> +
> static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
> {
> /* file-wide pr_fmt adds "CPU features: " prefix */
> @@ -1041,21 +1056,6 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
> }
>
> /*
> - * Flag to indicate if we have computed the system wide
> - * capabilities based on the boot time active CPUs. This
> - * will be used to determine if a new booting CPU should
> - * go through the verification process to make sure that it
> - * supports the system capabilities, without using a hotplug
> - * notifier.
> - */
> -static bool sys_caps_initialised;
> -
> -static inline void set_sys_caps_initialised(void)
> -{
> - sys_caps_initialised = true;
> -}
> -
> -/*
> * Check for CPU features that are used in early boot
> * based on the Boot CPU value.
> */
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations
2017-08-31 17:00 ` Dave Martin
@ 2017-09-14 9:35 ` Suzuki K Poulose
-1 siblings, 0 replies; 224+ messages in thread
From: Suzuki K Poulose @ 2017-09-14 9:35 UTC (permalink / raw)
To: Dave Martin, linux-arm-kernel
Cc: Catalin Marinas, Will Deacon, Ard Biesheuvel, Alex Bennée,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
On 31/08/17 18:00, Dave Martin wrote:
> update_cpu_features() currently cannot tell whether it is being
> called during early or late secondary boot. This doesn't
> desperately matter for anything it currently does.
>
> However, SVE will need to know here whether the set of available
> vector lengths is fixed of still to be determined when booting a
> CPU so that it can be updated appropriately.
>
> This patch simply moves the sys_caps_initialised stuff to the top
> of the file so that it can be more widely. There doesn't seem to
> be a more obvious place to put it.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
> ---
> arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++---------------
> 1 file changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index cd52d36..43ba8df 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -51,6 +51,21 @@ unsigned int compat_elf_hwcap2 __read_mostly;
> DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
> EXPORT_SYMBOL(cpu_hwcaps);
>
> +/*
> + * Flag to indicate if we have computed the system wide
> + * capabilities based on the boot time active CPUs. This
> + * will be used to determine if a new booting CPU should
> + * go through the verification process to make sure that it
> + * supports the system capabilities, without using a hotplug
> + * notifier.
> + */
> +static bool sys_caps_initialised;
> +
> +static inline void set_sys_caps_initialised(void)
> +{
> + sys_caps_initialised = true;
> +}
> +
> static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
> {
> /* file-wide pr_fmt adds "CPU features: " prefix */
> @@ -1041,21 +1056,6 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
> }
>
> /*
> - * Flag to indicate if we have computed the system wide
> - * capabilities based on the boot time active CPUs. This
> - * will be used to determine if a new booting CPU should
> - * go through the verification process to make sure that it
> - * supports the system capabilities, without using a hotplug
> - * notifier.
> - */
> -static bool sys_caps_initialised;
> -
> -static inline void set_sys_caps_initialised(void)
> -{
> - sys_caps_initialised = true;
> -}
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations
@ 2017-09-14 9:35 ` Suzuki K Poulose
0 siblings, 0 replies; 224+ messages in thread
From: Suzuki K Poulose @ 2017-09-14 9:35 UTC (permalink / raw)
To: linux-arm-kernel
On 31/08/17 18:00, Dave Martin wrote:
> update_cpu_features() currently cannot tell whether it is being
> called during early or late secondary boot. This doesn't
> desperately matter for anything it currently does.
>
> However, SVE will need to know here whether the set of available
> vector lengths is fixed of still to be determined when booting a
> CPU so that it can be updated appropriately.
>
> This patch simply moves the sys_caps_initialised stuff to the top
> of the file so that it can be more widely. There doesn't seem to
> be a more obvious place to put it.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
> ---
> arch/arm64/kernel/cpufeature.c | 30 +++++++++++++++---------------
> 1 file changed, 15 insertions(+), 15 deletions(-)
>
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index cd52d36..43ba8df 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -51,6 +51,21 @@ unsigned int compat_elf_hwcap2 __read_mostly;
> DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
> EXPORT_SYMBOL(cpu_hwcaps);
>
> +/*
> + * Flag to indicate if we have computed the system wide
> + * capabilities based on the boot time active CPUs. This
> + * will be used to determine if a new booting CPU should
> + * go through the verification process to make sure that it
> + * supports the system capabilities, without using a hotplug
> + * notifier.
> + */
> +static bool sys_caps_initialised;
> +
> +static inline void set_sys_caps_initialised(void)
> +{
> + sys_caps_initialised = true;
> +}
> +
> static int dump_cpu_hwcaps(struct notifier_block *self, unsigned long v, void *p)
> {
> /* file-wide pr_fmt adds "CPU features: " prefix */
> @@ -1041,21 +1056,6 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
> }
>
> /*
> - * Flag to indicate if we have computed the system wide
> - * capabilities based on the boot time active CPUs. This
> - * will be used to determine if a new booting CPU should
> - * go through the verification process to make sure that it
> - * supports the system capabilities, without using a hotplug
> - * notifier.
> - */
> -static bool sys_caps_initialised;
> -
> -static inline void set_sys_caps_initialised(void)
> -{
> - sys_caps_initialised = true;
> -}
Reviewed-by: Suzuki K Poulose <suzuki.poulose@arm.com>
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-09-14 9:45 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:45 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Suzuki K Poulose
Dave Martin <Dave.Martin@arm.com> writes:
> This patch uses the cpufeatures framework to determine common SVE
> capabilities and vector lengths, and configures the runtime SVE
> support code appropriately.
>
> ZCR_ELx is not really a feature register, but it is convenient to
> use it as a template for recording the maximum vector length
> supported by a CPU, using the LEN field. This field is similar to
> a feature field in that it is a contiguous bitfield for which we
> want to determine the minimum system-wide value. This patch adds
> ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
> custom code to populate it. Finding the minimum supported value of
> the LEN field is left to the cpufeatures framework in the usual
> way.
>
> The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
> so for now we just require it to be zero.
>
> Note that much of this code is dormant and SVE still won't be used
> yet, since system_supports_sve() remains hardwired to false.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
>
> Other changes related to Alex Bennée's comments:
>
> * Migrate away from magic numbers for converting VL to VQ.
>
> Requested by Suzuki Poulose:
>
> * Make sve_vq_map __ro_after_init.
>
> Other changes related to Suzuki Poulose's comments:
>
> * Rely on cpufeatures for not attempting to update the vq map after boot.
> ---
> arch/arm64/include/asm/cpu.h | 4 ++
> arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
> arch/arm64/include/asm/fpsimd.h | 10 ++++
> arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
> arch/arm64/kernel/cpuinfo.c | 6 ++
> arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
> 6 files changed, 202 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
> index 889226b..8839227 100644
> --- a/arch/arm64/include/asm/cpu.h
> +++ b/arch/arm64/include/asm/cpu.h
> @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
> u64 reg_id_aa64mmfr2;
> u64 reg_id_aa64pfr0;
> u64 reg_id_aa64pfr1;
> + u64 reg_id_aa64zfr0;
>
> u32 reg_id_dfr0;
> u32 reg_id_isar0;
> @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
> u32 reg_mvfr0;
> u32 reg_mvfr1;
> u32 reg_mvfr2;
> +
> + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
> + u64 reg_zcr;
> };
>
> DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 4ea3441..d98e7ba 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -10,7 +10,9 @@
> #define __ASM_CPUFEATURE_H
>
> #include <asm/cpucaps.h>
> +#include <asm/fpsimd.h>
> #include <asm/hwcap.h>
> +#include <asm/sigcontext.h>
> #include <asm/sysreg.h>
>
> /*
> @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
> return val == ID_AA64PFR0_EL0_32BIT_64BIT;
> }
>
> +static inline bool id_aa64pfr0_sve(u64 pfr0)
> +{
> + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
> +
> + return val > 0;
> +}
> +
> void __init setup_cpu_features(void);
>
> void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
> return false;
> }
>
> +/*
> + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> + * vector length.
> + * Use only if SVE is present. This function clobbers the SVE vector length.
> + */
:nit whitespace formatting.
> +static u64 __maybe_unused read_zcr_features(void)
> +{
> + u64 zcr;
> + unsigned int vq_max;
> +
> + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
I'm confused, why are we writing something here? You mention clobbering
the SVE vector length but what was the point?
> +
> + zcr = read_sysreg_s(SYS_ZCR_EL1);
> + zcr &= ~(u64)ZCR_ELx_LEN_MASK;
> + vq_max = sve_vq_from_vl(sve_get_vl());
> + zcr |= vq_max - 1;
> +
> + return zcr;
> +}
> +
> #endif /* __ASSEMBLY__ */
>
> #endif
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 32c8e19..6c22624 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -92,12 +92,22 @@ extern void fpsimd_dup_sve(struct task_struct *dst,
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> +extern void __init sve_init_vq_map(void);
> +extern void sve_update_vq_map(void);
> +extern int sve_verify_vq_map(void);
> +extern void __init sve_setup(void);
> +
> #else /* ! CONFIG_ARM64_SVE */
>
> static void __maybe_unused sve_alloc(struct task_struct *task) { }
> static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
> static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src) { }
> +static void __maybe_unused sve_init_vq_map(void) { }
> +static void __maybe_unused sve_update_vq_map(void) { }
> +static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> +static void __maybe_unused sve_setup(void) { }
> +
> #endif /* ! CONFIG_ARM64_SVE */
>
> /* For use by EFI runtime services calls only */
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 43ba8df..c30bb6b 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -27,6 +27,7 @@
> #include <asm/cpu.h>
> #include <asm/cpufeature.h>
> #include <asm/cpu_ops.h>
> +#include <asm/fpsimd.h>
> #include <asm/mmu_context.h>
> #include <asm/processor.h>
> #include <asm/sysreg.h>
> @@ -283,6 +284,12 @@ static const struct arm64_ftr_bits ftr_id_dfr0[] = {
> ARM64_FTR_END,
> };
>
> +static const struct arm64_ftr_bits ftr_zcr[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
> + ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
> + ARM64_FTR_END,
> +};
> +
> /*
> * Common ftr bits for a 32bit register with all hidden, strict
> * attributes, with 4bit feature fields and a default safe value of
> @@ -349,6 +356,7 @@ static const struct __ftr_reg_entry {
> /* Op1 = 0, CRn = 0, CRm = 4 */
> ARM64_FTR_REG(SYS_ID_AA64PFR0_EL1, ftr_id_aa64pfr0),
> ARM64_FTR_REG(SYS_ID_AA64PFR1_EL1, ftr_raz),
> + ARM64_FTR_REG(SYS_ID_AA64ZFR0_EL1, ftr_raz),
>
> /* Op1 = 0, CRn = 0, CRm = 5 */
> ARM64_FTR_REG(SYS_ID_AA64DFR0_EL1, ftr_id_aa64dfr0),
> @@ -363,6 +371,9 @@ static const struct __ftr_reg_entry {
> ARM64_FTR_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1),
> ARM64_FTR_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2),
>
> + /* Op1 = 0, CRn = 1, CRm = 2 */
> + ARM64_FTR_REG(SYS_ZCR_EL1, ftr_zcr),
> +
> /* Op1 = 3, CRn = 0, CRm = 0 */
> { SYS_CTR_EL0, &arm64_ftr_reg_ctrel0 },
> ARM64_FTR_REG(SYS_DCZID_EL0, ftr_dczid),
> @@ -500,6 +511,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> init_cpu_ftr_reg(SYS_ID_AA64MMFR2_EL1, info->reg_id_aa64mmfr2);
> init_cpu_ftr_reg(SYS_ID_AA64PFR0_EL1, info->reg_id_aa64pfr0);
> init_cpu_ftr_reg(SYS_ID_AA64PFR1_EL1, info->reg_id_aa64pfr1);
> + init_cpu_ftr_reg(SYS_ID_AA64ZFR0_EL1, info->reg_id_aa64zfr0);
>
> if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
> init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0);
> @@ -520,6 +532,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> init_cpu_ftr_reg(SYS_MVFR2_EL1, info->reg_mvfr2);
> }
>
> + if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
> + init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
> + sve_init_vq_map();
> + }
> }
>
> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
> @@ -623,6 +639,9 @@ void update_cpu_features(int cpu,
> taint |= check_update_ftr_reg(SYS_ID_AA64PFR1_EL1, cpu,
> info->reg_id_aa64pfr1, boot->reg_id_aa64pfr1);
>
> + taint |= check_update_ftr_reg(SYS_ID_AA64ZFR0_EL1, cpu,
> + info->reg_id_aa64zfr0, boot->reg_id_aa64zfr0);
> +
> /*
> * If we have AArch32, we care about 32-bit features for compat.
> * If the system doesn't support AArch32, don't update them.
> @@ -670,6 +689,14 @@ void update_cpu_features(int cpu,
> info->reg_mvfr2, boot->reg_mvfr2);
> }
>
> + if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
> + taint |= check_update_ftr_reg(SYS_ZCR_EL1, cpu,
> + info->reg_zcr, boot->reg_zcr);
> +
> + if (!sys_caps_initialised)
> + sve_update_vq_map();
> + }
> +
> /*
> * Mismatched CPU features are a recipe for disaster. Don't even
> * pretend to support them.
> @@ -1097,6 +1124,23 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps)
> }
> }
>
> +static void verify_sve_features(void)
> +{
> + u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
> + u64 zcr = read_zcr_features();
> +
> + unsigned int safe_len = safe_zcr & ZCR_ELx_LEN_MASK;
> + unsigned int len = zcr & ZCR_ELx_LEN_MASK;
> +
> + if (len < safe_len || sve_verify_vq_map()) {
> + pr_crit("CPU%d: SVE: required vector length(s) missing\n",
> + smp_processor_id());
> + cpu_die_early();
> + }
> +
> + /* Add checks on other ZCR bits here if necessary */
> +}
> +
> /*
> * Run through the enabled system capabilities and enable() it on this CPU.
> * The capabilities were decided based on the available CPUs at the boot time.
> @@ -1110,8 +1154,12 @@ static void verify_local_cpu_capabilities(void)
> verify_local_cpu_errata_workarounds();
> verify_local_cpu_features(arm64_features);
> verify_local_elf_hwcaps(arm64_elf_hwcaps);
> +
> if (system_supports_32bit_el0())
> verify_local_elf_hwcaps(compat_elf_hwcaps);
> +
> + if (system_supports_sve())
> + verify_sve_features();
> }
>
> void check_local_cpu_capabilities(void)
> @@ -1189,6 +1237,8 @@ void __init setup_cpu_features(void)
> if (system_supports_32bit_el0())
> setup_elf_hwcaps(compat_elf_hwcaps);
>
> + sve_setup();
> +
> /* Advertise that we have computed the system capabilities */
> set_sys_caps_initialised();
>
> diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
> index 3118859..be260e8 100644
> --- a/arch/arm64/kernel/cpuinfo.c
> +++ b/arch/arm64/kernel/cpuinfo.c
> @@ -19,6 +19,7 @@
> #include <asm/cpu.h>
> #include <asm/cputype.h>
> #include <asm/cpufeature.h>
> +#include <asm/fpsimd.h>
>
> #include <linux/bitops.h>
> #include <linux/bug.h>
> @@ -326,6 +327,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
> info->reg_id_aa64mmfr2 = read_cpuid(ID_AA64MMFR2_EL1);
> info->reg_id_aa64pfr0 = read_cpuid(ID_AA64PFR0_EL1);
> info->reg_id_aa64pfr1 = read_cpuid(ID_AA64PFR1_EL1);
> + info->reg_id_aa64zfr0 = read_cpuid(ID_AA64ZFR0_EL1);
>
> /* Update the 32bit ID registers only if AArch32 is implemented */
> if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
> @@ -348,6 +350,10 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
> info->reg_mvfr2 = read_cpuid(MVFR2_EL1);
> }
>
> + if (IS_ENABLED(CONFIG_ARM64_SVE) &&
> + id_aa64pfr0_sve(info->reg_id_aa64pfr0))
> + info->reg_zcr = read_zcr_features();
> +
> cpuinfo_detect_icache_policy(info);
> }
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 713476e..cea05a7 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -110,19 +110,19 @@
> static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
>
> /* Default VL for tasks that don't set it explicitly: */
> -static int sve_default_vl = SVE_VL_MIN;
> +static int sve_default_vl = -1;
>
> #ifdef CONFIG_ARM64_SVE
>
> /* Maximum supported vector length across all CPUs (initially poisoned) */
> int __ro_after_init sve_max_vl = -1;
> /* Set of available vector lengths, as vq_to_bit(vq): */
> -static DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
>
> #else /* ! CONFIG_ARM64_SVE */
>
> /* Dummy declaration for code that will be optimised out: */
> -extern DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
>
> #endif /* ! CONFIG_ARM64_SVE */
>
> @@ -387,6 +387,103 @@ int sve_set_vector_length(struct task_struct *task,
> return 0;
> }
>
> +static unsigned long *sve_alloc_vq_map(void)
> +{
> + return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
> + GFP_KERNEL);
> +}
> +
> +static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
> +{
> + unsigned int vq, vl;
> + unsigned long zcr;
> +
> + zcr = ZCR_ELx_LEN_MASK;
> + zcr = read_sysreg_s(SYS_ZCR_EL1) & ~zcr;
> +
> + for (vq = SVE_VQ_MAX; vq >= SVE_VQ_MIN; --vq) {
> + write_sysreg_s(zcr | (vq - 1), SYS_ZCR_EL1); /* self-syncing */
> + vl = sve_get_vl();
> + vq = sve_vq_from_vl(vl); /* skip intervening lengths */
> + set_bit(vq_to_bit(vq), map);
> + }
> +}
> +
> +void __init sve_init_vq_map(void)
> +{
> + sve_probe_vqs(sve_vq_map);
> +}
> +
> +/*
> + * If we haven't committed to the set of supported VQs yet, filter out
> + * those not supported by the current CPU.
> + */
> +void sve_update_vq_map(void)
> +{
> + unsigned long *map;
> +
> + map = sve_alloc_vq_map();
> + sve_probe_vqs(map);
> + bitmap_and(sve_vq_map, sve_vq_map, map, SVE_VQ_MAX);
> + kfree(map);
> +}
> +
> +/* Check whether the current CPU supports all VQs in the committed set */
> +int sve_verify_vq_map(void)
> +{
> + int ret = 0;
> + unsigned long *map = sve_alloc_vq_map();
> +
> + sve_probe_vqs(map);
> + bitmap_andnot(map, sve_vq_map, map, SVE_VQ_MAX);
> + if (!bitmap_empty(map, SVE_VQ_MAX)) {
> + pr_warn("SVE: cpu%d: Required vector length(s) missing\n",
> + smp_processor_id());
> + ret = -EINVAL;
> + }
> +
> + kfree(map);
> +
> + return ret;
> +}
> +
> +void __init sve_setup(void)
> +{
> + u64 zcr;
> +
> + if (!system_supports_sve())
> + return;
> +
> + /*
> + * The SVE architecture mandates support for 128-bit vectors,
> + * so sve_vq_map must have at least SVE_VQ_MIN set.
> + * If something went wrong, at least try to patch it up:
> + */
> + if (WARN_ON(!test_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map)))
> + set_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map);
> +
> + zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
> + sve_max_vl = sve_vl_from_vq((zcr & ZCR_ELx_LEN_MASK) + 1);
> +
> + /*
> + * Sanity-check that the max VL we determined through CPU features
> + * corresponds properly to sve_vq_map. If not, do our best:
> + */
> + if (WARN_ON(sve_max_vl != find_supported_vector_length(sve_max_vl)))
> + sve_max_vl = find_supported_vector_length(sve_max_vl);
> +
> + /*
> + * For the default VL, pick the maximum supported value <= 64.
> + * VL == 64 is guaranteed not to grow the signal frame.
> + */
> + sve_default_vl = find_supported_vector_length(64);
> +
> + pr_info("SVE: maximum available vector length %u bytes per vector\n",
> + sve_max_vl);
> + pr_info("SVE: default vector length %u bytes per vector\n",
> + sve_default_vl);
> +}
> +
> void fpsimd_release_thread(struct task_struct *dead_task)
> {
> sve_free(dead_task);
> @@ -502,6 +599,9 @@ void fpsimd_flush_thread(void)
> * This is where we ensure that all user tasks have a valid
> * vector length configured: no kernel task can become a user
> * task without an exec and hence a call to this function.
> + * By the time the first call to this function is made, all
> + * early hardware probing is complete, so sve_default_vl
> + * should be valid.
> * If a bug causes this to go wrong, we make some noise and
> * try to fudge thread.sve_vl to a safe value here.
> */
Otherwise:
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-09-14 9:45 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:45 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Suzuki K Poulose
Dave Martin <Dave.Martin@arm.com> writes:
> This patch uses the cpufeatures framework to determine common SVE
> capabilities and vector lengths, and configures the runtime SVE
> support code appropriately.
>
> ZCR_ELx is not really a feature register, but it is convenient to
> use it as a template for recording the maximum vector length
> supported by a CPU, using the LEN field. This field is similar to
> a feature field in that it is a contiguous bitfield for which we
> want to determine the minimum system-wide value. This patch adds
> ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
> custom code to populate it. Finding the minimum supported value of
> the LEN field is left to the cpufeatures framework in the usual
> way.
>
> The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
> so for now we just require it to be zero.
>
> Note that much of this code is dormant and SVE still won't be used
> yet, since system_supports_sve() remains hardwired to false.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Bennée:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
>
> Other changes related to Alex Bennée's comments:
>
> * Migrate away from magic numbers for converting VL to VQ.
>
> Requested by Suzuki Poulose:
>
> * Make sve_vq_map __ro_after_init.
>
> Other changes related to Suzuki Poulose's comments:
>
> * Rely on cpufeatures for not attempting to update the vq map after boot.
> ---
> arch/arm64/include/asm/cpu.h | 4 ++
> arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
> arch/arm64/include/asm/fpsimd.h | 10 ++++
> arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
> arch/arm64/kernel/cpuinfo.c | 6 ++
> arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
> 6 files changed, 202 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
> index 889226b..8839227 100644
> --- a/arch/arm64/include/asm/cpu.h
> +++ b/arch/arm64/include/asm/cpu.h
> @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
> u64 reg_id_aa64mmfr2;
> u64 reg_id_aa64pfr0;
> u64 reg_id_aa64pfr1;
> + u64 reg_id_aa64zfr0;
>
> u32 reg_id_dfr0;
> u32 reg_id_isar0;
> @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
> u32 reg_mvfr0;
> u32 reg_mvfr1;
> u32 reg_mvfr2;
> +
> + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
> + u64 reg_zcr;
> };
>
> DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 4ea3441..d98e7ba 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -10,7 +10,9 @@
> #define __ASM_CPUFEATURE_H
>
> #include <asm/cpucaps.h>
> +#include <asm/fpsimd.h>
> #include <asm/hwcap.h>
> +#include <asm/sigcontext.h>
> #include <asm/sysreg.h>
>
> /*
> @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
> return val == ID_AA64PFR0_EL0_32BIT_64BIT;
> }
>
> +static inline bool id_aa64pfr0_sve(u64 pfr0)
> +{
> + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
> +
> + return val > 0;
> +}
> +
> void __init setup_cpu_features(void);
>
> void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
> return false;
> }
>
> +/*
> + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> + * vector length.
> + * Use only if SVE is present. This function clobbers the SVE vector length.
> + */
:nit whitespace formatting.
> +static u64 __maybe_unused read_zcr_features(void)
> +{
> + u64 zcr;
> + unsigned int vq_max;
> +
> + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
I'm confused, why are we writing something here? You mention clobbering
the SVE vector length but what was the point?
> +
> + zcr = read_sysreg_s(SYS_ZCR_EL1);
> + zcr &= ~(u64)ZCR_ELx_LEN_MASK;
> + vq_max = sve_vq_from_vl(sve_get_vl());
> + zcr |= vq_max - 1;
> +
> + return zcr;
> +}
> +
> #endif /* __ASSEMBLY__ */
>
> #endif
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 32c8e19..6c22624 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -92,12 +92,22 @@ extern void fpsimd_dup_sve(struct task_struct *dst,
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> +extern void __init sve_init_vq_map(void);
> +extern void sve_update_vq_map(void);
> +extern int sve_verify_vq_map(void);
> +extern void __init sve_setup(void);
> +
> #else /* ! CONFIG_ARM64_SVE */
>
> static void __maybe_unused sve_alloc(struct task_struct *task) { }
> static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
> static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src) { }
> +static void __maybe_unused sve_init_vq_map(void) { }
> +static void __maybe_unused sve_update_vq_map(void) { }
> +static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> +static void __maybe_unused sve_setup(void) { }
> +
> #endif /* ! CONFIG_ARM64_SVE */
>
> /* For use by EFI runtime services calls only */
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 43ba8df..c30bb6b 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -27,6 +27,7 @@
> #include <asm/cpu.h>
> #include <asm/cpufeature.h>
> #include <asm/cpu_ops.h>
> +#include <asm/fpsimd.h>
> #include <asm/mmu_context.h>
> #include <asm/processor.h>
> #include <asm/sysreg.h>
> @@ -283,6 +284,12 @@ static const struct arm64_ftr_bits ftr_id_dfr0[] = {
> ARM64_FTR_END,
> };
>
> +static const struct arm64_ftr_bits ftr_zcr[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
> + ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
> + ARM64_FTR_END,
> +};
> +
> /*
> * Common ftr bits for a 32bit register with all hidden, strict
> * attributes, with 4bit feature fields and a default safe value of
> @@ -349,6 +356,7 @@ static const struct __ftr_reg_entry {
> /* Op1 = 0, CRn = 0, CRm = 4 */
> ARM64_FTR_REG(SYS_ID_AA64PFR0_EL1, ftr_id_aa64pfr0),
> ARM64_FTR_REG(SYS_ID_AA64PFR1_EL1, ftr_raz),
> + ARM64_FTR_REG(SYS_ID_AA64ZFR0_EL1, ftr_raz),
>
> /* Op1 = 0, CRn = 0, CRm = 5 */
> ARM64_FTR_REG(SYS_ID_AA64DFR0_EL1, ftr_id_aa64dfr0),
> @@ -363,6 +371,9 @@ static const struct __ftr_reg_entry {
> ARM64_FTR_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1),
> ARM64_FTR_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2),
>
> + /* Op1 = 0, CRn = 1, CRm = 2 */
> + ARM64_FTR_REG(SYS_ZCR_EL1, ftr_zcr),
> +
> /* Op1 = 3, CRn = 0, CRm = 0 */
> { SYS_CTR_EL0, &arm64_ftr_reg_ctrel0 },
> ARM64_FTR_REG(SYS_DCZID_EL0, ftr_dczid),
> @@ -500,6 +511,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> init_cpu_ftr_reg(SYS_ID_AA64MMFR2_EL1, info->reg_id_aa64mmfr2);
> init_cpu_ftr_reg(SYS_ID_AA64PFR0_EL1, info->reg_id_aa64pfr0);
> init_cpu_ftr_reg(SYS_ID_AA64PFR1_EL1, info->reg_id_aa64pfr1);
> + init_cpu_ftr_reg(SYS_ID_AA64ZFR0_EL1, info->reg_id_aa64zfr0);
>
> if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
> init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0);
> @@ -520,6 +532,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> init_cpu_ftr_reg(SYS_MVFR2_EL1, info->reg_mvfr2);
> }
>
> + if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
> + init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
> + sve_init_vq_map();
> + }
> }
>
> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
> @@ -623,6 +639,9 @@ void update_cpu_features(int cpu,
> taint |= check_update_ftr_reg(SYS_ID_AA64PFR1_EL1, cpu,
> info->reg_id_aa64pfr1, boot->reg_id_aa64pfr1);
>
> + taint |= check_update_ftr_reg(SYS_ID_AA64ZFR0_EL1, cpu,
> + info->reg_id_aa64zfr0, boot->reg_id_aa64zfr0);
> +
> /*
> * If we have AArch32, we care about 32-bit features for compat.
> * If the system doesn't support AArch32, don't update them.
> @@ -670,6 +689,14 @@ void update_cpu_features(int cpu,
> info->reg_mvfr2, boot->reg_mvfr2);
> }
>
> + if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
> + taint |= check_update_ftr_reg(SYS_ZCR_EL1, cpu,
> + info->reg_zcr, boot->reg_zcr);
> +
> + if (!sys_caps_initialised)
> + sve_update_vq_map();
> + }
> +
> /*
> * Mismatched CPU features are a recipe for disaster. Don't even
> * pretend to support them.
> @@ -1097,6 +1124,23 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps)
> }
> }
>
> +static void verify_sve_features(void)
> +{
> + u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
> + u64 zcr = read_zcr_features();
> +
> + unsigned int safe_len = safe_zcr & ZCR_ELx_LEN_MASK;
> + unsigned int len = zcr & ZCR_ELx_LEN_MASK;
> +
> + if (len < safe_len || sve_verify_vq_map()) {
> + pr_crit("CPU%d: SVE: required vector length(s) missing\n",
> + smp_processor_id());
> + cpu_die_early();
> + }
> +
> + /* Add checks on other ZCR bits here if necessary */
> +}
> +
> /*
> * Run through the enabled system capabilities and enable() it on this CPU.
> * The capabilities were decided based on the available CPUs at the boot time.
> @@ -1110,8 +1154,12 @@ static void verify_local_cpu_capabilities(void)
> verify_local_cpu_errata_workarounds();
> verify_local_cpu_features(arm64_features);
> verify_local_elf_hwcaps(arm64_elf_hwcaps);
> +
> if (system_supports_32bit_el0())
> verify_local_elf_hwcaps(compat_elf_hwcaps);
> +
> + if (system_supports_sve())
> + verify_sve_features();
> }
>
> void check_local_cpu_capabilities(void)
> @@ -1189,6 +1237,8 @@ void __init setup_cpu_features(void)
> if (system_supports_32bit_el0())
> setup_elf_hwcaps(compat_elf_hwcaps);
>
> + sve_setup();
> +
> /* Advertise that we have computed the system capabilities */
> set_sys_caps_initialised();
>
> diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
> index 3118859..be260e8 100644
> --- a/arch/arm64/kernel/cpuinfo.c
> +++ b/arch/arm64/kernel/cpuinfo.c
> @@ -19,6 +19,7 @@
> #include <asm/cpu.h>
> #include <asm/cputype.h>
> #include <asm/cpufeature.h>
> +#include <asm/fpsimd.h>
>
> #include <linux/bitops.h>
> #include <linux/bug.h>
> @@ -326,6 +327,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
> info->reg_id_aa64mmfr2 = read_cpuid(ID_AA64MMFR2_EL1);
> info->reg_id_aa64pfr0 = read_cpuid(ID_AA64PFR0_EL1);
> info->reg_id_aa64pfr1 = read_cpuid(ID_AA64PFR1_EL1);
> + info->reg_id_aa64zfr0 = read_cpuid(ID_AA64ZFR0_EL1);
>
> /* Update the 32bit ID registers only if AArch32 is implemented */
> if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
> @@ -348,6 +350,10 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
> info->reg_mvfr2 = read_cpuid(MVFR2_EL1);
> }
>
> + if (IS_ENABLED(CONFIG_ARM64_SVE) &&
> + id_aa64pfr0_sve(info->reg_id_aa64pfr0))
> + info->reg_zcr = read_zcr_features();
> +
> cpuinfo_detect_icache_policy(info);
> }
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 713476e..cea05a7 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -110,19 +110,19 @@
> static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
>
> /* Default VL for tasks that don't set it explicitly: */
> -static int sve_default_vl = SVE_VL_MIN;
> +static int sve_default_vl = -1;
>
> #ifdef CONFIG_ARM64_SVE
>
> /* Maximum supported vector length across all CPUs (initially poisoned) */
> int __ro_after_init sve_max_vl = -1;
> /* Set of available vector lengths, as vq_to_bit(vq): */
> -static DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
>
> #else /* ! CONFIG_ARM64_SVE */
>
> /* Dummy declaration for code that will be optimised out: */
> -extern DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
>
> #endif /* ! CONFIG_ARM64_SVE */
>
> @@ -387,6 +387,103 @@ int sve_set_vector_length(struct task_struct *task,
> return 0;
> }
>
> +static unsigned long *sve_alloc_vq_map(void)
> +{
> + return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
> + GFP_KERNEL);
> +}
> +
> +static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
> +{
> + unsigned int vq, vl;
> + unsigned long zcr;
> +
> + zcr = ZCR_ELx_LEN_MASK;
> + zcr = read_sysreg_s(SYS_ZCR_EL1) & ~zcr;
> +
> + for (vq = SVE_VQ_MAX; vq >= SVE_VQ_MIN; --vq) {
> + write_sysreg_s(zcr | (vq - 1), SYS_ZCR_EL1); /* self-syncing */
> + vl = sve_get_vl();
> + vq = sve_vq_from_vl(vl); /* skip intervening lengths */
> + set_bit(vq_to_bit(vq), map);
> + }
> +}
> +
> +void __init sve_init_vq_map(void)
> +{
> + sve_probe_vqs(sve_vq_map);
> +}
> +
> +/*
> + * If we haven't committed to the set of supported VQs yet, filter out
> + * those not supported by the current CPU.
> + */
> +void sve_update_vq_map(void)
> +{
> + unsigned long *map;
> +
> + map = sve_alloc_vq_map();
> + sve_probe_vqs(map);
> + bitmap_and(sve_vq_map, sve_vq_map, map, SVE_VQ_MAX);
> + kfree(map);
> +}
> +
> +/* Check whether the current CPU supports all VQs in the committed set */
> +int sve_verify_vq_map(void)
> +{
> + int ret = 0;
> + unsigned long *map = sve_alloc_vq_map();
> +
> + sve_probe_vqs(map);
> + bitmap_andnot(map, sve_vq_map, map, SVE_VQ_MAX);
> + if (!bitmap_empty(map, SVE_VQ_MAX)) {
> + pr_warn("SVE: cpu%d: Required vector length(s) missing\n",
> + smp_processor_id());
> + ret = -EINVAL;
> + }
> +
> + kfree(map);
> +
> + return ret;
> +}
> +
> +void __init sve_setup(void)
> +{
> + u64 zcr;
> +
> + if (!system_supports_sve())
> + return;
> +
> + /*
> + * The SVE architecture mandates support for 128-bit vectors,
> + * so sve_vq_map must have at least SVE_VQ_MIN set.
> + * If something went wrong, at least try to patch it up:
> + */
> + if (WARN_ON(!test_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map)))
> + set_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map);
> +
> + zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
> + sve_max_vl = sve_vl_from_vq((zcr & ZCR_ELx_LEN_MASK) + 1);
> +
> + /*
> + * Sanity-check that the max VL we determined through CPU features
> + * corresponds properly to sve_vq_map. If not, do our best:
> + */
> + if (WARN_ON(sve_max_vl != find_supported_vector_length(sve_max_vl)))
> + sve_max_vl = find_supported_vector_length(sve_max_vl);
> +
> + /*
> + * For the default VL, pick the maximum supported value <= 64.
> + * VL == 64 is guaranteed not to grow the signal frame.
> + */
> + sve_default_vl = find_supported_vector_length(64);
> +
> + pr_info("SVE: maximum available vector length %u bytes per vector\n",
> + sve_max_vl);
> + pr_info("SVE: default vector length %u bytes per vector\n",
> + sve_default_vl);
> +}
> +
> void fpsimd_release_thread(struct task_struct *dead_task)
> {
> sve_free(dead_task);
> @@ -502,6 +599,9 @@ void fpsimd_flush_thread(void)
> * This is where we ensure that all user tasks have a valid
> * vector length configured: no kernel task can become a user
> * task without an exec and hence a call to this function.
> + * By the time the first call to this function is made, all
> + * early hardware probing is complete, so sve_default_vl
> + * should be valid.
> * If a bug causes this to go wrong, we make some noise and
> * try to fudge thread.sve_vl to a safe value here.
> */
Otherwise:
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-09-14 9:45 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 9:45 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> This patch uses the cpufeatures framework to determine common SVE
> capabilities and vector lengths, and configures the runtime SVE
> support code appropriately.
>
> ZCR_ELx is not really a feature register, but it is convenient to
> use it as a template for recording the maximum vector length
> supported by a CPU, using the LEN field. This field is similar to
> a feature field in that it is a contiguous bitfield for which we
> want to determine the minimum system-wide value. This patch adds
> ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
> custom code to populate it. Finding the minimum supported value of
> the LEN field is left to the cpufeatures framework in the usual
> way.
>
> The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
> so for now we just require it to be zero.
>
> Note that much of this code is dormant and SVE still won't be used
> yet, since system_supports_sve() remains hardwired to false.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
> Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Alex Benn?e:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
>
> Other changes related to Alex Benn?e's comments:
>
> * Migrate away from magic numbers for converting VL to VQ.
>
> Requested by Suzuki Poulose:
>
> * Make sve_vq_map __ro_after_init.
>
> Other changes related to Suzuki Poulose's comments:
>
> * Rely on cpufeatures for not attempting to update the vq map after boot.
> ---
> arch/arm64/include/asm/cpu.h | 4 ++
> arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
> arch/arm64/include/asm/fpsimd.h | 10 ++++
> arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
> arch/arm64/kernel/cpuinfo.c | 6 ++
> arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
> 6 files changed, 202 insertions(+), 3 deletions(-)
>
> diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
> index 889226b..8839227 100644
> --- a/arch/arm64/include/asm/cpu.h
> +++ b/arch/arm64/include/asm/cpu.h
> @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
> u64 reg_id_aa64mmfr2;
> u64 reg_id_aa64pfr0;
> u64 reg_id_aa64pfr1;
> + u64 reg_id_aa64zfr0;
>
> u32 reg_id_dfr0;
> u32 reg_id_isar0;
> @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
> u32 reg_mvfr0;
> u32 reg_mvfr1;
> u32 reg_mvfr2;
> +
> + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
> + u64 reg_zcr;
> };
>
> DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
> diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> index 4ea3441..d98e7ba 100644
> --- a/arch/arm64/include/asm/cpufeature.h
> +++ b/arch/arm64/include/asm/cpufeature.h
> @@ -10,7 +10,9 @@
> #define __ASM_CPUFEATURE_H
>
> #include <asm/cpucaps.h>
> +#include <asm/fpsimd.h>
> #include <asm/hwcap.h>
> +#include <asm/sigcontext.h>
> #include <asm/sysreg.h>
>
> /*
> @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
> return val == ID_AA64PFR0_EL0_32BIT_64BIT;
> }
>
> +static inline bool id_aa64pfr0_sve(u64 pfr0)
> +{
> + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
> +
> + return val > 0;
> +}
> +
> void __init setup_cpu_features(void);
>
> void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
> return false;
> }
>
> +/*
> + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> + * vector length.
> + * Use only if SVE is present. This function clobbers the SVE vector length.
> + */
:nit whitespace formatting.
> +static u64 __maybe_unused read_zcr_features(void)
> +{
> + u64 zcr;
> + unsigned int vq_max;
> +
> + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
I'm confused, why are we writing something here? You mention clobbering
the SVE vector length but what was the point?
> +
> + zcr = read_sysreg_s(SYS_ZCR_EL1);
> + zcr &= ~(u64)ZCR_ELx_LEN_MASK;
> + vq_max = sve_vq_from_vl(sve_get_vl());
> + zcr |= vq_max - 1;
> +
> + return zcr;
> +}
> +
> #endif /* __ASSEMBLY__ */
>
> #endif
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 32c8e19..6c22624 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -92,12 +92,22 @@ extern void fpsimd_dup_sve(struct task_struct *dst,
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> +extern void __init sve_init_vq_map(void);
> +extern void sve_update_vq_map(void);
> +extern int sve_verify_vq_map(void);
> +extern void __init sve_setup(void);
> +
> #else /* ! CONFIG_ARM64_SVE */
>
> static void __maybe_unused sve_alloc(struct task_struct *task) { }
> static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
> static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src) { }
> +static void __maybe_unused sve_init_vq_map(void) { }
> +static void __maybe_unused sve_update_vq_map(void) { }
> +static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> +static void __maybe_unused sve_setup(void) { }
> +
> #endif /* ! CONFIG_ARM64_SVE */
>
> /* For use by EFI runtime services calls only */
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 43ba8df..c30bb6b 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -27,6 +27,7 @@
> #include <asm/cpu.h>
> #include <asm/cpufeature.h>
> #include <asm/cpu_ops.h>
> +#include <asm/fpsimd.h>
> #include <asm/mmu_context.h>
> #include <asm/processor.h>
> #include <asm/sysreg.h>
> @@ -283,6 +284,12 @@ static const struct arm64_ftr_bits ftr_id_dfr0[] = {
> ARM64_FTR_END,
> };
>
> +static const struct arm64_ftr_bits ftr_zcr[] = {
> + ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE,
> + ZCR_ELx_LEN_SHIFT, ZCR_ELx_LEN_SIZE, 0), /* LEN */
> + ARM64_FTR_END,
> +};
> +
> /*
> * Common ftr bits for a 32bit register with all hidden, strict
> * attributes, with 4bit feature fields and a default safe value of
> @@ -349,6 +356,7 @@ static const struct __ftr_reg_entry {
> /* Op1 = 0, CRn = 0, CRm = 4 */
> ARM64_FTR_REG(SYS_ID_AA64PFR0_EL1, ftr_id_aa64pfr0),
> ARM64_FTR_REG(SYS_ID_AA64PFR1_EL1, ftr_raz),
> + ARM64_FTR_REG(SYS_ID_AA64ZFR0_EL1, ftr_raz),
>
> /* Op1 = 0, CRn = 0, CRm = 5 */
> ARM64_FTR_REG(SYS_ID_AA64DFR0_EL1, ftr_id_aa64dfr0),
> @@ -363,6 +371,9 @@ static const struct __ftr_reg_entry {
> ARM64_FTR_REG(SYS_ID_AA64MMFR1_EL1, ftr_id_aa64mmfr1),
> ARM64_FTR_REG(SYS_ID_AA64MMFR2_EL1, ftr_id_aa64mmfr2),
>
> + /* Op1 = 0, CRn = 1, CRm = 2 */
> + ARM64_FTR_REG(SYS_ZCR_EL1, ftr_zcr),
> +
> /* Op1 = 3, CRn = 0, CRm = 0 */
> { SYS_CTR_EL0, &arm64_ftr_reg_ctrel0 },
> ARM64_FTR_REG(SYS_DCZID_EL0, ftr_dczid),
> @@ -500,6 +511,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> init_cpu_ftr_reg(SYS_ID_AA64MMFR2_EL1, info->reg_id_aa64mmfr2);
> init_cpu_ftr_reg(SYS_ID_AA64PFR0_EL1, info->reg_id_aa64pfr0);
> init_cpu_ftr_reg(SYS_ID_AA64PFR1_EL1, info->reg_id_aa64pfr1);
> + init_cpu_ftr_reg(SYS_ID_AA64ZFR0_EL1, info->reg_id_aa64zfr0);
>
> if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
> init_cpu_ftr_reg(SYS_ID_DFR0_EL1, info->reg_id_dfr0);
> @@ -520,6 +532,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
> init_cpu_ftr_reg(SYS_MVFR2_EL1, info->reg_mvfr2);
> }
>
> + if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
> + init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
> + sve_init_vq_map();
> + }
> }
>
> static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
> @@ -623,6 +639,9 @@ void update_cpu_features(int cpu,
> taint |= check_update_ftr_reg(SYS_ID_AA64PFR1_EL1, cpu,
> info->reg_id_aa64pfr1, boot->reg_id_aa64pfr1);
>
> + taint |= check_update_ftr_reg(SYS_ID_AA64ZFR0_EL1, cpu,
> + info->reg_id_aa64zfr0, boot->reg_id_aa64zfr0);
> +
> /*
> * If we have AArch32, we care about 32-bit features for compat.
> * If the system doesn't support AArch32, don't update them.
> @@ -670,6 +689,14 @@ void update_cpu_features(int cpu,
> info->reg_mvfr2, boot->reg_mvfr2);
> }
>
> + if (id_aa64pfr0_sve(info->reg_id_aa64pfr0)) {
> + taint |= check_update_ftr_reg(SYS_ZCR_EL1, cpu,
> + info->reg_zcr, boot->reg_zcr);
> +
> + if (!sys_caps_initialised)
> + sve_update_vq_map();
> + }
> +
> /*
> * Mismatched CPU features are a recipe for disaster. Don't even
> * pretend to support them.
> @@ -1097,6 +1124,23 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps)
> }
> }
>
> +static void verify_sve_features(void)
> +{
> + u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
> + u64 zcr = read_zcr_features();
> +
> + unsigned int safe_len = safe_zcr & ZCR_ELx_LEN_MASK;
> + unsigned int len = zcr & ZCR_ELx_LEN_MASK;
> +
> + if (len < safe_len || sve_verify_vq_map()) {
> + pr_crit("CPU%d: SVE: required vector length(s) missing\n",
> + smp_processor_id());
> + cpu_die_early();
> + }
> +
> + /* Add checks on other ZCR bits here if necessary */
> +}
> +
> /*
> * Run through the enabled system capabilities and enable() it on this CPU.
> * The capabilities were decided based on the available CPUs at the boot time.
> @@ -1110,8 +1154,12 @@ static void verify_local_cpu_capabilities(void)
> verify_local_cpu_errata_workarounds();
> verify_local_cpu_features(arm64_features);
> verify_local_elf_hwcaps(arm64_elf_hwcaps);
> +
> if (system_supports_32bit_el0())
> verify_local_elf_hwcaps(compat_elf_hwcaps);
> +
> + if (system_supports_sve())
> + verify_sve_features();
> }
>
> void check_local_cpu_capabilities(void)
> @@ -1189,6 +1237,8 @@ void __init setup_cpu_features(void)
> if (system_supports_32bit_el0())
> setup_elf_hwcaps(compat_elf_hwcaps);
>
> + sve_setup();
> +
> /* Advertise that we have computed the system capabilities */
> set_sys_caps_initialised();
>
> diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c
> index 3118859..be260e8 100644
> --- a/arch/arm64/kernel/cpuinfo.c
> +++ b/arch/arm64/kernel/cpuinfo.c
> @@ -19,6 +19,7 @@
> #include <asm/cpu.h>
> #include <asm/cputype.h>
> #include <asm/cpufeature.h>
> +#include <asm/fpsimd.h>
>
> #include <linux/bitops.h>
> #include <linux/bug.h>
> @@ -326,6 +327,7 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
> info->reg_id_aa64mmfr2 = read_cpuid(ID_AA64MMFR2_EL1);
> info->reg_id_aa64pfr0 = read_cpuid(ID_AA64PFR0_EL1);
> info->reg_id_aa64pfr1 = read_cpuid(ID_AA64PFR1_EL1);
> + info->reg_id_aa64zfr0 = read_cpuid(ID_AA64ZFR0_EL1);
>
> /* Update the 32bit ID registers only if AArch32 is implemented */
> if (id_aa64pfr0_32bit_el0(info->reg_id_aa64pfr0)) {
> @@ -348,6 +350,10 @@ static void __cpuinfo_store_cpu(struct cpuinfo_arm64 *info)
> info->reg_mvfr2 = read_cpuid(MVFR2_EL1);
> }
>
> + if (IS_ENABLED(CONFIG_ARM64_SVE) &&
> + id_aa64pfr0_sve(info->reg_id_aa64pfr0))
> + info->reg_zcr = read_zcr_features();
> +
> cpuinfo_detect_icache_policy(info);
> }
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 713476e..cea05a7 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -110,19 +110,19 @@
> static DEFINE_PER_CPU(struct fpsimd_state *, fpsimd_last_state);
>
> /* Default VL for tasks that don't set it explicitly: */
> -static int sve_default_vl = SVE_VL_MIN;
> +static int sve_default_vl = -1;
>
> #ifdef CONFIG_ARM64_SVE
>
> /* Maximum supported vector length across all CPUs (initially poisoned) */
> int __ro_after_init sve_max_vl = -1;
> /* Set of available vector lengths, as vq_to_bit(vq): */
> -static DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
>
> #else /* ! CONFIG_ARM64_SVE */
>
> /* Dummy declaration for code that will be optimised out: */
> -extern DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
>
> #endif /* ! CONFIG_ARM64_SVE */
>
> @@ -387,6 +387,103 @@ int sve_set_vector_length(struct task_struct *task,
> return 0;
> }
>
> +static unsigned long *sve_alloc_vq_map(void)
> +{
> + return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
> + GFP_KERNEL);
> +}
> +
> +static void sve_probe_vqs(DECLARE_BITMAP(map, SVE_VQ_MAX))
> +{
> + unsigned int vq, vl;
> + unsigned long zcr;
> +
> + zcr = ZCR_ELx_LEN_MASK;
> + zcr = read_sysreg_s(SYS_ZCR_EL1) & ~zcr;
> +
> + for (vq = SVE_VQ_MAX; vq >= SVE_VQ_MIN; --vq) {
> + write_sysreg_s(zcr | (vq - 1), SYS_ZCR_EL1); /* self-syncing */
> + vl = sve_get_vl();
> + vq = sve_vq_from_vl(vl); /* skip intervening lengths */
> + set_bit(vq_to_bit(vq), map);
> + }
> +}
> +
> +void __init sve_init_vq_map(void)
> +{
> + sve_probe_vqs(sve_vq_map);
> +}
> +
> +/*
> + * If we haven't committed to the set of supported VQs yet, filter out
> + * those not supported by the current CPU.
> + */
> +void sve_update_vq_map(void)
> +{
> + unsigned long *map;
> +
> + map = sve_alloc_vq_map();
> + sve_probe_vqs(map);
> + bitmap_and(sve_vq_map, sve_vq_map, map, SVE_VQ_MAX);
> + kfree(map);
> +}
> +
> +/* Check whether the current CPU supports all VQs in the committed set */
> +int sve_verify_vq_map(void)
> +{
> + int ret = 0;
> + unsigned long *map = sve_alloc_vq_map();
> +
> + sve_probe_vqs(map);
> + bitmap_andnot(map, sve_vq_map, map, SVE_VQ_MAX);
> + if (!bitmap_empty(map, SVE_VQ_MAX)) {
> + pr_warn("SVE: cpu%d: Required vector length(s) missing\n",
> + smp_processor_id());
> + ret = -EINVAL;
> + }
> +
> + kfree(map);
> +
> + return ret;
> +}
> +
> +void __init sve_setup(void)
> +{
> + u64 zcr;
> +
> + if (!system_supports_sve())
> + return;
> +
> + /*
> + * The SVE architecture mandates support for 128-bit vectors,
> + * so sve_vq_map must have at least SVE_VQ_MIN set.
> + * If something went wrong, at least try to patch it up:
> + */
> + if (WARN_ON(!test_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map)))
> + set_bit(vq_to_bit(SVE_VQ_MIN), sve_vq_map);
> +
> + zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
> + sve_max_vl = sve_vl_from_vq((zcr & ZCR_ELx_LEN_MASK) + 1);
> +
> + /*
> + * Sanity-check that the max VL we determined through CPU features
> + * corresponds properly to sve_vq_map. If not, do our best:
> + */
> + if (WARN_ON(sve_max_vl != find_supported_vector_length(sve_max_vl)))
> + sve_max_vl = find_supported_vector_length(sve_max_vl);
> +
> + /*
> + * For the default VL, pick the maximum supported value <= 64.
> + * VL == 64 is guaranteed not to grow the signal frame.
> + */
> + sve_default_vl = find_supported_vector_length(64);
> +
> + pr_info("SVE: maximum available vector length %u bytes per vector\n",
> + sve_max_vl);
> + pr_info("SVE: default vector length %u bytes per vector\n",
> + sve_default_vl);
> +}
> +
> void fpsimd_release_thread(struct task_struct *dead_task)
> {
> sve_free(dead_task);
> @@ -502,6 +599,9 @@ void fpsimd_flush_thread(void)
> * This is where we ensure that all user tasks have a valid
> * vector length configured: no kernel task can become a user
> * task without an exec and hence a call to this function.
> + * By the time the first call to this function is made, all
> + * early hardware probing is complete, so sve_default_vl
> + * should be valid.
> * If a bug causes this to go wrong, we make some noise and
> * try to fudge thread.sve_vl to a safe value here.
> */
Otherwise:
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 17/28] arm64/sve: Preserve SVE registers around kernel-mode NEON use
@ 2017-09-14 10:52 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 10:52 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Richard Sandiford, kvmarm,
linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> Kernel-mode NEON will corrupt the SVE vector registers, due to the
> way they alias the FPSIMD vector registers in the hardware.
>
> This patch ensures that any live SVE register content for the task
> is saved by kernel_neon_begin(). The data will be restored in the
> usual way on return to userspace.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/kernel/fpsimd.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index cea05a7..dd89acf 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -744,8 +744,10 @@ void kernel_neon_begin(void)
> __this_cpu_write(kernel_neon_busy, true);
>
> /* Save unsaved task fpsimd state, if any: */
> - if (current->mm && !test_and_set_thread_flag(TIF_FOREIGN_FPSTATE))
> - fpsimd_save_state(¤t->thread.fpsimd_state);
> + if (current->mm) {
> + task_fpsimd_save();
> + set_thread_flag(TIF_FOREIGN_FPSTATE);
> + }
>
> /* Invalidate any task state remaining in the fpsimd regs: */
> __this_cpu_write(fpsimd_last_state, NULL);
--
Alex Bennée
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 17/28] arm64/sve: Preserve SVE registers around kernel-mode NEON use
@ 2017-09-14 10:52 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 10:52 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> Kernel-mode NEON will corrupt the SVE vector registers, due to the
> way they alias the FPSIMD vector registers in the hardware.
>
> This patch ensures that any live SVE register content for the task
> is saved by kernel_neon_begin(). The data will be restored in the
> usual way on return to userspace.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/kernel/fpsimd.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index cea05a7..dd89acf 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -744,8 +744,10 @@ void kernel_neon_begin(void)
> __this_cpu_write(kernel_neon_busy, true);
>
> /* Save unsaved task fpsimd state, if any: */
> - if (current->mm && !test_and_set_thread_flag(TIF_FOREIGN_FPSTATE))
> - fpsimd_save_state(¤t->thread.fpsimd_state);
> + if (current->mm) {
> + task_fpsimd_save();
> + set_thread_flag(TIF_FOREIGN_FPSTATE);
> + }
>
> /* Invalidate any task state remaining in the fpsimd regs: */
> __this_cpu_write(fpsimd_last_state, NULL);
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 17/28] arm64/sve: Preserve SVE registers around kernel-mode NEON use
@ 2017-09-14 10:52 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 10:52 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> Kernel-mode NEON will corrupt the SVE vector registers, due to the
> way they alias the FPSIMD vector registers in the hardware.
>
> This patch ensures that any live SVE register content for the task
> is saved by kernel_neon_begin(). The data will be restored in the
> usual way on return to userspace.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Reviewed-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
> ---
> arch/arm64/kernel/fpsimd.c | 6 ++++--
> 1 file changed, 4 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index cea05a7..dd89acf 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -744,8 +744,10 @@ void kernel_neon_begin(void)
> __this_cpu_write(kernel_neon_busy, true);
>
> /* Save unsaved task fpsimd state, if any: */
> - if (current->mm && !test_and_set_thread_flag(TIF_FOREIGN_FPSTATE))
> - fpsimd_save_state(¤t->thread.fpsimd_state);
> + if (current->mm) {
> + task_fpsimd_save();
> + set_thread_flag(TIF_FOREIGN_FPSTATE);
> + }
>
> /* Invalidate any task state remaining in the fpsimd regs: */
> __this_cpu_write(fpsimd_last_state, NULL);
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls
@ 2017-09-14 11:01 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 11:01 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> The EFI runtime services ABI allows EFI to make free use of the
> FPSIMD registers during EFI runtime service calls, subject to the
> callee-save requirements of the AArch64 procedure call standard.
>
> However, the SVE architecture allows upper bits of the SVE vector
> registers to be zeroed as a side-effect of FPSIMD V-register
> writes. This means that the SVE vector registers must be saved in
> their entirety in order to avoid data loss: non-SVE-aware EFI
> implementations cannot restore them correctly.
>
> The non-IRQ case is already handled gracefully by
> kernel_neon_begin(). For the IRQ case, this patch allocates a
> suitable per-CPU stash buffer for the full SVE register state and
> uses it to preserve the affected registers around EFI calls. It is
> currently unclear how the EFI runtime services ABI will be
> clarified with respect to SVE, so it safest to assume that the
> predicate registers and FFR must be saved and restored too.
>
> No attempt is made to restore the restore the vector length after
> a call, for now. It is deemed rather insane for EFI to change it,
> and contemporary EFI implementations certainly won't.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Ard Biesheuvel:
>
> * Fix unbalanced ifelse bracing to conform to the kernel coding style.
>
> * Make efi_sve_state_used static.
>
> Changes related to Alex Bennée's comments:
>
> * Migrate away from magic numbers for SVE_VQ_BYTES.
>
> Other:
>
> * Rename sve_kernel_mode_neon_setup() to sve_efi_setup().
> The EFI FPSIMD code is something semi-independent from kernel-mode NEON
> now, so the "neon" in the names no longer really makes sense.
>
> * Make the EFI FPSIMD setup code dependent on CONFIG_EFI (it's not
> supposed to work with CONFIG_EFI=n anyway).
> ---
> arch/arm64/kernel/fpsimd.c | 60 +++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 54 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index dd89acf..fff9fcf 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -118,11 +118,13 @@ static int sve_default_vl = -1;
> int __ro_after_init sve_max_vl = -1;
> /* Set of available vector lengths, as vq_to_bit(vq): */
> static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +static void __percpu *efi_sve_state;
>
> #else /* ! CONFIG_ARM64_SVE */
>
> /* Dummy declaration for code that will be optimised out: */
> extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +extern void __percpu *efi_sve_state;
>
> #endif /* ! CONFIG_ARM64_SVE */
>
> @@ -447,6 +449,23 @@ int sve_verify_vq_map(void)
> return ret;
> }
>
> +static void __init sve_efi_setup(void)
> +{
> + if (!IS_ENABLED(CONFIG_EFI))
> + return;
> +
> + /*
> + * alloc_percpu() warns and prints a backtrace if this goes wrong.
> + * This is evidence of a crippled system and we are returning void,
> + * so no attempt is made to handle this situation here.
> + */
> + BUG_ON(!sve_vl_valid(sve_max_vl));
> + efi_sve_state = __alloc_percpu(
> + SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
> + if (!efi_sve_state)
> + panic("Cannot allocate percpu memory for EFI SVE save/restore");
> +}
> +
> void __init sve_setup(void)
> {
> u64 zcr;
> @@ -482,6 +501,8 @@ void __init sve_setup(void)
> sve_max_vl);
> pr_info("SVE: default vector length %u bytes per vector\n",
> sve_default_vl);
> +
> + sve_efi_setup();
> }
>
> void fpsimd_release_thread(struct task_struct *dead_task)
> @@ -783,6 +804,7 @@ EXPORT_SYMBOL(kernel_neon_end);
>
> static DEFINE_PER_CPU(struct fpsimd_state, efi_fpsimd_state);
> static DEFINE_PER_CPU(bool, efi_fpsimd_state_used);
> +static DEFINE_PER_CPU(bool, efi_sve_state_used);
>
> /*
> * EFI runtime services support functions
> @@ -808,10 +830,24 @@ void __efi_fpsimd_begin(void)
>
> WARN_ON(preemptible());
>
> - if (may_use_simd())
> + if (may_use_simd()) {
> kernel_neon_begin();
> - else {
> - fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
> + } else {
> + /*
> + * If !efi_sve_state, SVE can't be in use yet and doesn't need
> + * preserving:
> + */
> + if (system_supports_sve() && likely(efi_sve_state)) {
> + char *sve_state = this_cpu_ptr(efi_sve_state);
> +
> + __this_cpu_write(efi_sve_state_used, true);
> +
> + sve_save_state(sve_state + sve_ffr_offset(sve_max_vl),
> + &this_cpu_ptr(&efi_fpsimd_state)->fpsr);
> + } else {
> + fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
> + }
> +
> __this_cpu_write(efi_fpsimd_state_used, true);
> }
> }
> @@ -824,10 +860,22 @@ void __efi_fpsimd_end(void)
> if (!system_supports_fpsimd())
> return;
>
> - if (__this_cpu_xchg(efi_fpsimd_state_used, false))
> - fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
> - else
> + if (!__this_cpu_xchg(efi_fpsimd_state_used, false)) {
> kernel_neon_end();
> + } else {
> + if (system_supports_sve() &&
> + likely(__this_cpu_read(efi_sve_state_used))) {
> + char const *sve_state = this_cpu_ptr(efi_sve_state);
> +
> + sve_load_state(sve_state + sve_ffr_offset(sve_max_vl),
> + &this_cpu_ptr(&efi_fpsimd_state)->fpsr,
> + sve_vq_from_vl(sve_get_vl()) - 1);
> +
> + __this_cpu_write(efi_sve_state_used, false);
> + } else {
> + fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
> + }
> + }
> }
>
> #endif /* CONFIG_KERNEL_MODE_NEON */
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls
@ 2017-09-14 11:01 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 11:01 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> The EFI runtime services ABI allows EFI to make free use of the
> FPSIMD registers during EFI runtime service calls, subject to the
> callee-save requirements of the AArch64 procedure call standard.
>
> However, the SVE architecture allows upper bits of the SVE vector
> registers to be zeroed as a side-effect of FPSIMD V-register
> writes. This means that the SVE vector registers must be saved in
> their entirety in order to avoid data loss: non-SVE-aware EFI
> implementations cannot restore them correctly.
>
> The non-IRQ case is already handled gracefully by
> kernel_neon_begin(). For the IRQ case, this patch allocates a
> suitable per-CPU stash buffer for the full SVE register state and
> uses it to preserve the affected registers around EFI calls. It is
> currently unclear how the EFI runtime services ABI will be
> clarified with respect to SVE, so it safest to assume that the
> predicate registers and FFR must be saved and restored too.
>
> No attempt is made to restore the restore the vector length after
> a call, for now. It is deemed rather insane for EFI to change it,
> and contemporary EFI implementations certainly won't.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Ard Biesheuvel:
>
> * Fix unbalanced ifelse bracing to conform to the kernel coding style.
>
> * Make efi_sve_state_used static.
>
> Changes related to Alex Bennée's comments:
>
> * Migrate away from magic numbers for SVE_VQ_BYTES.
>
> Other:
>
> * Rename sve_kernel_mode_neon_setup() to sve_efi_setup().
> The EFI FPSIMD code is something semi-independent from kernel-mode NEON
> now, so the "neon" in the names no longer really makes sense.
>
> * Make the EFI FPSIMD setup code dependent on CONFIG_EFI (it's not
> supposed to work with CONFIG_EFI=n anyway).
> ---
> arch/arm64/kernel/fpsimd.c | 60 +++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 54 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index dd89acf..fff9fcf 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -118,11 +118,13 @@ static int sve_default_vl = -1;
> int __ro_after_init sve_max_vl = -1;
> /* Set of available vector lengths, as vq_to_bit(vq): */
> static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +static void __percpu *efi_sve_state;
>
> #else /* ! CONFIG_ARM64_SVE */
>
> /* Dummy declaration for code that will be optimised out: */
> extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +extern void __percpu *efi_sve_state;
>
> #endif /* ! CONFIG_ARM64_SVE */
>
> @@ -447,6 +449,23 @@ int sve_verify_vq_map(void)
> return ret;
> }
>
> +static void __init sve_efi_setup(void)
> +{
> + if (!IS_ENABLED(CONFIG_EFI))
> + return;
> +
> + /*
> + * alloc_percpu() warns and prints a backtrace if this goes wrong.
> + * This is evidence of a crippled system and we are returning void,
> + * so no attempt is made to handle this situation here.
> + */
> + BUG_ON(!sve_vl_valid(sve_max_vl));
> + efi_sve_state = __alloc_percpu(
> + SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
> + if (!efi_sve_state)
> + panic("Cannot allocate percpu memory for EFI SVE save/restore");
> +}
> +
> void __init sve_setup(void)
> {
> u64 zcr;
> @@ -482,6 +501,8 @@ void __init sve_setup(void)
> sve_max_vl);
> pr_info("SVE: default vector length %u bytes per vector\n",
> sve_default_vl);
> +
> + sve_efi_setup();
> }
>
> void fpsimd_release_thread(struct task_struct *dead_task)
> @@ -783,6 +804,7 @@ EXPORT_SYMBOL(kernel_neon_end);
>
> static DEFINE_PER_CPU(struct fpsimd_state, efi_fpsimd_state);
> static DEFINE_PER_CPU(bool, efi_fpsimd_state_used);
> +static DEFINE_PER_CPU(bool, efi_sve_state_used);
>
> /*
> * EFI runtime services support functions
> @@ -808,10 +830,24 @@ void __efi_fpsimd_begin(void)
>
> WARN_ON(preemptible());
>
> - if (may_use_simd())
> + if (may_use_simd()) {
> kernel_neon_begin();
> - else {
> - fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
> + } else {
> + /*
> + * If !efi_sve_state, SVE can't be in use yet and doesn't need
> + * preserving:
> + */
> + if (system_supports_sve() && likely(efi_sve_state)) {
> + char *sve_state = this_cpu_ptr(efi_sve_state);
> +
> + __this_cpu_write(efi_sve_state_used, true);
> +
> + sve_save_state(sve_state + sve_ffr_offset(sve_max_vl),
> + &this_cpu_ptr(&efi_fpsimd_state)->fpsr);
> + } else {
> + fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
> + }
> +
> __this_cpu_write(efi_fpsimd_state_used, true);
> }
> }
> @@ -824,10 +860,22 @@ void __efi_fpsimd_end(void)
> if (!system_supports_fpsimd())
> return;
>
> - if (__this_cpu_xchg(efi_fpsimd_state_used, false))
> - fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
> - else
> + if (!__this_cpu_xchg(efi_fpsimd_state_used, false)) {
> kernel_neon_end();
> + } else {
> + if (system_supports_sve() &&
> + likely(__this_cpu_read(efi_sve_state_used))) {
> + char const *sve_state = this_cpu_ptr(efi_sve_state);
> +
> + sve_load_state(sve_state + sve_ffr_offset(sve_max_vl),
> + &this_cpu_ptr(&efi_fpsimd_state)->fpsr,
> + sve_vq_from_vl(sve_get_vl()) - 1);
> +
> + __this_cpu_write(efi_sve_state_used, false);
> + } else {
> + fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
> + }
> + }
> }
>
> #endif /* CONFIG_KERNEL_MODE_NEON */
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls
@ 2017-09-14 11:01 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 11:01 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> The EFI runtime services ABI allows EFI to make free use of the
> FPSIMD registers during EFI runtime service calls, subject to the
> callee-save requirements of the AArch64 procedure call standard.
>
> However, the SVE architecture allows upper bits of the SVE vector
> registers to be zeroed as a side-effect of FPSIMD V-register
> writes. This means that the SVE vector registers must be saved in
> their entirety in order to avoid data loss: non-SVE-aware EFI
> implementations cannot restore them correctly.
>
> The non-IRQ case is already handled gracefully by
> kernel_neon_begin(). For the IRQ case, this patch allocates a
> suitable per-CPU stash buffer for the full SVE register state and
> uses it to preserve the affected registers around EFI calls. It is
> currently unclear how the EFI runtime services ABI will be
> clarified with respect to SVE, so it safest to assume that the
> predicate registers and FFR must be saved and restored too.
>
> No attempt is made to restore the restore the vector length after
> a call, for now. It is deemed rather insane for EFI to change it,
> and contemporary EFI implementations certainly won't.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Ard Biesheuvel:
>
> * Fix unbalanced ifelse bracing to conform to the kernel coding style.
>
> * Make efi_sve_state_used static.
>
> Changes related to Alex Benn?e's comments:
>
> * Migrate away from magic numbers for SVE_VQ_BYTES.
>
> Other:
>
> * Rename sve_kernel_mode_neon_setup() to sve_efi_setup().
> The EFI FPSIMD code is something semi-independent from kernel-mode NEON
> now, so the "neon" in the names no longer really makes sense.
>
> * Make the EFI FPSIMD setup code dependent on CONFIG_EFI (it's not
> supposed to work with CONFIG_EFI=n anyway).
> ---
> arch/arm64/kernel/fpsimd.c | 60 +++++++++++++++++++++++++++++++++++++++++-----
> 1 file changed, 54 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index dd89acf..fff9fcf 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -118,11 +118,13 @@ static int sve_default_vl = -1;
> int __ro_after_init sve_max_vl = -1;
> /* Set of available vector lengths, as vq_to_bit(vq): */
> static __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +static void __percpu *efi_sve_state;
>
> #else /* ! CONFIG_ARM64_SVE */
>
> /* Dummy declaration for code that will be optimised out: */
> extern __ro_after_init DECLARE_BITMAP(sve_vq_map, SVE_VQ_MAX);
> +extern void __percpu *efi_sve_state;
>
> #endif /* ! CONFIG_ARM64_SVE */
>
> @@ -447,6 +449,23 @@ int sve_verify_vq_map(void)
> return ret;
> }
>
> +static void __init sve_efi_setup(void)
> +{
> + if (!IS_ENABLED(CONFIG_EFI))
> + return;
> +
> + /*
> + * alloc_percpu() warns and prints a backtrace if this goes wrong.
> + * This is evidence of a crippled system and we are returning void,
> + * so no attempt is made to handle this situation here.
> + */
> + BUG_ON(!sve_vl_valid(sve_max_vl));
> + efi_sve_state = __alloc_percpu(
> + SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
> + if (!efi_sve_state)
> + panic("Cannot allocate percpu memory for EFI SVE save/restore");
> +}
> +
> void __init sve_setup(void)
> {
> u64 zcr;
> @@ -482,6 +501,8 @@ void __init sve_setup(void)
> sve_max_vl);
> pr_info("SVE: default vector length %u bytes per vector\n",
> sve_default_vl);
> +
> + sve_efi_setup();
> }
>
> void fpsimd_release_thread(struct task_struct *dead_task)
> @@ -783,6 +804,7 @@ EXPORT_SYMBOL(kernel_neon_end);
>
> static DEFINE_PER_CPU(struct fpsimd_state, efi_fpsimd_state);
> static DEFINE_PER_CPU(bool, efi_fpsimd_state_used);
> +static DEFINE_PER_CPU(bool, efi_sve_state_used);
>
> /*
> * EFI runtime services support functions
> @@ -808,10 +830,24 @@ void __efi_fpsimd_begin(void)
>
> WARN_ON(preemptible());
>
> - if (may_use_simd())
> + if (may_use_simd()) {
> kernel_neon_begin();
> - else {
> - fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
> + } else {
> + /*
> + * If !efi_sve_state, SVE can't be in use yet and doesn't need
> + * preserving:
> + */
> + if (system_supports_sve() && likely(efi_sve_state)) {
> + char *sve_state = this_cpu_ptr(efi_sve_state);
> +
> + __this_cpu_write(efi_sve_state_used, true);
> +
> + sve_save_state(sve_state + sve_ffr_offset(sve_max_vl),
> + &this_cpu_ptr(&efi_fpsimd_state)->fpsr);
> + } else {
> + fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state));
> + }
> +
> __this_cpu_write(efi_fpsimd_state_used, true);
> }
> }
> @@ -824,10 +860,22 @@ void __efi_fpsimd_end(void)
> if (!system_supports_fpsimd())
> return;
>
> - if (__this_cpu_xchg(efi_fpsimd_state_used, false))
> - fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
> - else
> + if (!__this_cpu_xchg(efi_fpsimd_state_used, false)) {
> kernel_neon_end();
> + } else {
> + if (system_supports_sve() &&
> + likely(__this_cpu_read(efi_sve_state_used))) {
> + char const *sve_state = this_cpu_ptr(efi_sve_state);
> +
> + sve_load_state(sve_state + sve_ffr_offset(sve_max_vl),
> + &this_cpu_ptr(&efi_fpsimd_state)->fpsr,
> + sve_vq_from_vl(sve_get_vl()) - 1);
> +
> + __this_cpu_write(efi_sve_state_used, false);
> + } else {
> + fpsimd_load_state(this_cpu_ptr(&efi_fpsimd_state));
> + }
> + }
> }
>
> #endif /* CONFIG_KERNEL_MODE_NEON */
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-14 12:57 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 12:57 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
gdb, Alan Hayward, Yao Qi, Oleg Nesterov, Alexander Viro
Dave Martin <Dave.Martin@arm.com> writes:
> This patch defines and implements a new regset NT_ARM_SVE, which
> describes a thread's SVE register state. This allows a debugger to
> manipulate the SVE state, as well as being included in ELF
> coredumps for post-mortem debugging.
>
> Because the regset size and layout are dependent on the thread's
> current vector length, it is not possible to define a C struct to
> describe the regset contents as is done for existing regsets.
> Instead, and for the same reasons, NT_ARM_SVE is based on the
> freeform variable-layout approach used for the SVE signal frame.
>
> Additionally, to reduce debug overhead when debugging threads that
> might or might not have live SVE register state, NT_ARM_SVE may be
> presented in one of two different formats: the old struct
> user_fpsimd_state format is embedded for describing the state of a
> thread with no live SVE state, whereas a new variable-layout
> structure is embedded for describing live SVE state. This avoids a
> debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
> allows existing userspace code to handle the non-SVE case without
> too much modification.
>
> For this to work, NT_ARM_SVE is defined with a fixed-format header
> of type struct user_sve_header, which the recipient can use to
> figure out the content, size and layout of the reset of the regset.
> Accessor macros are defined to allow the vector-length-dependent
> parts of the regset to be manipulated.
>
> Signed-off-by: Alan Hayward <alan.hayward@arm.com>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Other changes related to Alex Bennée's comments:
>
> * Migrate to SVE_VQ_BYTES instead of magic numbers.
>
> Requested by Alex Bennée:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
>
> Other:
>
> * [ABI fix] Bail out with -EIO if attempting to set the
> SVE regs for an unsupported VL, instead of misparsing the regset data.
>
> * Replace some in-kernel open-coded arithmetic with ALIGN()/
> DIV_ROUND_UP().
> ---
> arch/arm64/include/asm/fpsimd.h | 13 +-
> arch/arm64/include/uapi/asm/ptrace.h | 135 ++++++++++++++++++
> arch/arm64/kernel/fpsimd.c | 40 +++++-
> arch/arm64/kernel/ptrace.c | 270 +++++++++++++++++++++++++++++++++--
> include/uapi/linux/elf.h | 1 +
> 5 files changed, 449 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 6c22624..2723cca 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -38,13 +38,16 @@ struct fpsimd_state {
> __uint128_t vregs[32];
> u32 fpsr;
> u32 fpcr;
> + /*
> + * For ptrace compatibility, pad to next 128-bit
> + * boundary here if extending this struct.
> + */
> };
> };
> /* the id of the last cpu to have restored this state */
> unsigned int cpu;
> };
>
> -
> #if defined(__KERNEL__) && defined(CONFIG_COMPAT)
> /* Masks for extracting the FPSR and FPCR from the FPSCR */
> #define VFP_FPSCR_STAT_MASK 0xf800009f
> @@ -89,6 +92,10 @@ extern void sve_alloc(struct task_struct *task);
> extern void fpsimd_release_thread(struct task_struct *task);
> extern void fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src);
> +extern void fpsimd_sync_to_sve(struct task_struct *task);
> +extern void sve_sync_to_fpsimd(struct task_struct *task);
> +extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
> +
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> @@ -103,6 +110,10 @@ static void __maybe_unused sve_alloc(struct task_struct *task) { }
> static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
> static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src) { }
> +static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
> +static void __maybe_unused sve_sync_from_fpsimd_zeropad(
> + struct task_struct *task) { }
> +
> static void __maybe_unused sve_init_vq_map(void) { }
> static void __maybe_unused sve_update_vq_map(void) { }
> static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
> index d1ff83d..1915ab0 100644
> --- a/arch/arm64/include/uapi/asm/ptrace.h
> +++ b/arch/arm64/include/uapi/asm/ptrace.h
> @@ -22,6 +22,7 @@
> #include <linux/types.h>
>
> #include <asm/hwcap.h>
> +#include <asm/sigcontext.h>
>
>
> /*
> @@ -63,6 +64,8 @@
>
> #ifndef __ASSEMBLY__
>
> +#include <linux/prctl.h>
> +
> /*
> * User structures for general purpose, floating point and debug registers.
> */
> @@ -90,6 +93,138 @@ struct user_hwdebug_state {
> } dbg_regs[16];
> };
>
> +/* SVE/FP/SIMD state (NT_ARM_SVE) */
> +
> +struct user_sve_header {
> + __u32 size; /* total meaningful regset content in bytes */
> + __u32 max_size; /* maxmium possible size for this thread */
> + __u16 vl; /* current vector length */
> + __u16 max_vl; /* maximum possible vector length */
> + __u16 flags;
> + __u16 __reserved;
> +};
> +
> +/* Definitions for user_sve_header.flags: */
> +#define SVE_PT_REGS_MASK (1 << 0)
> +
> +/* Flags: must be kept in sync with prctl interface in
> <linux/ptrace.h> */
Which flags? We base some on PR_foo flags but we seem to shift them
anyway so where is the requirement for them to match from?
> +#define SVE_PT_REGS_FPSIMD 0
> +#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
> +
> +#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
> +#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
> +
> +
> +/*
> + * The remainder of the SVE state follows struct user_sve_header. The
> + * total size of the SVE state (including header) depends on the
> + * metadata in the header: SVE_PT_SIZE(vq, flags) gives the total size
> + * of the state in bytes, including the header.
> + *
> + * Refer to <asm/sigcontext.h> for details of how to pass the correct
> + * "vq" argument to these macros.
> + */
> +
> +/* Offset from the start of struct user_sve_header to the register data */
> +#define SVE_PT_REGS_OFFSET \
> + ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +
> +/*
> + * The register data content and layout depends on the value of the
> + * flags field.
> + */
> +
> +/*
> + * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD case:
> + *
> + * The payload starts at offset SVE_PT_FPSIMD_OFFSET, and is of type
> + * struct user_fpsimd_state. Additional data might be appended in the
> + * future: use SVE_PT_FPSIMD_SIZE(vq, flags) to compute the total size.
> + * SVE_PT_FPSIMD_SIZE(vq, flags) will never be less than
> + * sizeof(struct user_fpsimd_state).
> + */
> +
> +#define SVE_PT_FPSIMD_OFFSET SVE_PT_REGS_OFFSET
> +
> +#define SVE_PT_FPSIMD_SIZE(vq, flags) (sizeof(struct user_fpsimd_state))
> +
> +/*
> + * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE case:
> + *
> + * The payload starts at offset SVE_PT_SVE_OFFSET, and is of size
> + * SVE_PT_SVE_SIZE(vq, flags).
> + *
> + * Additional macros describe the contents and layout of the payload.
> + * For each, SVE_PT_SVE_x_OFFSET(args) is the start offset relative to
> + * the start of struct user_sve_header, and SVE_PT_SVE_x_SIZE(args) is
> + * the size in bytes:
> + *
> + * x type description
> + * - ---- -----------
> + * ZREGS \
> + * ZREG |
> + * PREGS | refer to <asm/sigcontext.h>
> + * PREG |
> + * FFR /
> + *
> + * FPSR uint32_t FPSR
> + * FPCR uint32_t FPCR
> + *
> + * Additional data might be appended in the future.
> + */
> +
> +#define SVE_PT_SVE_ZREG_SIZE(vq) SVE_SIG_ZREG_SIZE(vq)
> +#define SVE_PT_SVE_PREG_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
> +#define SVE_PT_SVE_FFR_SIZE(vq) SVE_SIG_FFR_SIZE(vq)
> +#define SVE_PT_SVE_FPSR_SIZE sizeof(__u32)
> +#define SVE_PT_SVE_FPCR_SIZE sizeof(__u32)
> +
> +#define __SVE_SIG_TO_PT(offset) \
> + ((offset) - SVE_SIG_REGS_OFFSET + SVE_PT_REGS_OFFSET)
> +
> +#define SVE_PT_SVE_OFFSET SVE_PT_REGS_OFFSET
> +
> +#define SVE_PT_SVE_ZREGS_OFFSET \
> + __SVE_SIG_TO_PT(SVE_SIG_ZREGS_OFFSET)
> +#define SVE_PT_SVE_ZREG_OFFSET(vq, n) \
> + __SVE_SIG_TO_PT(SVE_SIG_ZREG_OFFSET(vq, n))
> +#define SVE_PT_SVE_ZREGS_SIZE(vq) \
> + (SVE_PT_SVE_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_PT_SVE_ZREGS_OFFSET)
> +
> +#define SVE_PT_SVE_PREGS_OFFSET(vq) \
> + __SVE_SIG_TO_PT(SVE_SIG_PREGS_OFFSET(vq))
> +#define SVE_PT_SVE_PREG_OFFSET(vq, n) \
> + __SVE_SIG_TO_PT(SVE_SIG_PREG_OFFSET(vq, n))
> +#define SVE_PT_SVE_PREGS_SIZE(vq) \
> + (SVE_PT_SVE_PREG_OFFSET(vq, SVE_NUM_PREGS) - \
> + SVE_PT_SVE_PREGS_OFFSET(vq))
> +
> +#define SVE_PT_SVE_FFR_OFFSET(vq) \
> + __SVE_SIG_TO_PT(SVE_SIG_FFR_OFFSET(vq))
> +
> +#define SVE_PT_SVE_FPSR_OFFSET(vq) \
> + ((SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq) + \
> + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +#define SVE_PT_SVE_FPCR_OFFSET(vq) \
> + (SVE_PT_SVE_FPSR_OFFSET(vq) + SVE_PT_SVE_FPSR_SIZE)
> +
> +/*
> + * Any future extension appended after FPCR must be aligned to the next
> + * 128-bit boundary.
> + */
> +
> +#define SVE_PT_SVE_SIZE(vq, flags) \
> + ((SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE \
> + - SVE_PT_SVE_OFFSET + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +
> +#define SVE_PT_SIZE(vq, flags) \
> + (((flags) & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE ? \
> + SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, flags) \
> + : SVE_PT_FPSIMD_OFFSET + SVE_PT_FPSIMD_SIZE(vq, flags))
> +
> #endif /* __ASSEMBLY__ */
>
> #endif /* _UAPI__ASM_PTRACE_H */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index fff9fcf..361c019 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -303,6 +303,37 @@ void sve_alloc(struct task_struct *task)
> BUG_ON(!task->thread.sve_state);
> }
>
> +void fpsimd_sync_to_sve(struct task_struct *task)
> +{
> + if (!test_tsk_thread_flag(task, TIF_SVE))
> + fpsimd_to_sve(task);
> +}
> +
> +void sve_sync_to_fpsimd(struct task_struct *task)
> +{
> + if (test_tsk_thread_flag(task, TIF_SVE))
> + sve_to_fpsimd(task);
> +}
> +
> +void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
> +{
> + unsigned int vq;
> + void *sst = task->thread.sve_state;
> + struct fpsimd_state const *fst = &task->thread.fpsimd_state;
> + unsigned int i;
> +
> + if (!test_tsk_thread_flag(task, TIF_SVE))
> + return;
> +
> + vq = sve_vq_from_vl(task->thread.sve_vl);
> +
> + memset(sst, 0, SVE_SIG_REGS_SIZE(vq));
> +
> + for (i = 0; i < 32; ++i)
> + memcpy(ZREG(sst, vq, i), &fst->vregs[i],
> + sizeof(fst->vregs[i]));
> +}
> +
> /*
> * Handle SVE state across fork():
> *
> @@ -459,10 +490,17 @@ static void __init sve_efi_setup(void)
> * This is evidence of a crippled system and we are returning void,
> * so no attempt is made to handle this situation here.
> */
> - BUG_ON(!sve_vl_valid(sve_max_vl));
> + if (!sve_vl_valid(sve_max_vl))
> + goto fail;
> +
> efi_sve_state = __alloc_percpu(
> SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
> if (!efi_sve_state)
> + goto fail;
> +
> + return;
> +
> +fail:
> panic("Cannot allocate percpu memory for EFI SVE save/restore");
> }
>
> diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
> index 9cbb612..5ef4735b 100644
> --- a/arch/arm64/kernel/ptrace.c
> +++ b/arch/arm64/kernel/ptrace.c
> @@ -32,6 +32,7 @@
> #include <linux/security.h>
> #include <linux/init.h>
> #include <linux/signal.h>
> +#include <linux/string.h>
> #include <linux/uaccess.h>
> #include <linux/perf_event.h>
> #include <linux/hw_breakpoint.h>
> @@ -40,6 +41,7 @@
> #include <linux/elf.h>
>
> #include <asm/compat.h>
> +#include <asm/cpufeature.h>
> #include <asm/debug-monitors.h>
> #include <asm/pgtable.h>
> #include <asm/stacktrace.h>
> @@ -618,33 +620,66 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
> /*
> * TODO: update fp accessors for lazy context switching (sync/flush hwstate)
> */
> -static int fpr_get(struct task_struct *target, const struct user_regset *regset,
> - unsigned int pos, unsigned int count,
> - void *kbuf, void __user *ubuf)
> +static int __fpr_get(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf, unsigned int start_pos)
> {
> struct user_fpsimd_state *uregs;
> +
> + sve_sync_to_fpsimd(target);
> +
> uregs = &target->thread.fpsimd_state.user_fpsimd;
>
> + return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs,
> + start_pos, start_pos + sizeof(*uregs));
> +}
> +
> +static int fpr_get(struct task_struct *target, const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf)
> +{
> if (target == current)
> fpsimd_preserve_current_state();
>
> - return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
> + return __fpr_get(target, regset, pos, count, kbuf, ubuf, 0);
> }
>
> -static int fpr_set(struct task_struct *target, const struct user_regset *regset,
> - unsigned int pos, unsigned int count,
> - const void *kbuf, const void __user *ubuf)
> +static int __fpr_set(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf,
> + unsigned int start_pos)
> {
> int ret;
> struct user_fpsimd_state newstate =
> target->thread.fpsimd_state.user_fpsimd;
>
> - ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
> + sve_sync_to_fpsimd(target);
> +
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate,
> + start_pos, start_pos + sizeof(newstate));
> if (ret)
> return ret;
>
> target->thread.fpsimd_state.user_fpsimd = newstate;
> +
> + return ret;
> +}
> +
> +static int fpr_set(struct task_struct *target, const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf)
> +{
> + int ret;
> +
> + ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, 0);
> + if (ret)
> + return ret;
> +
> + sve_sync_from_fpsimd_zeropad(target);
> fpsimd_flush_task_state(target);
> +
> return ret;
> }
>
> @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
> return ret;
> }
>
> +#ifdef CONFIG_ARM64_SVE
> +
> +static void sve_init_header_from_task(struct user_sve_header *header,
> + struct task_struct *target)
> +{
> + unsigned int vq;
> +
> + memset(header, 0, sizeof(*header));
> +
> + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> + header->flags |= SVE_PT_VL_INHERIT;
> +
> + header->vl = target->thread.sve_vl;
> + vq = sve_vq_from_vl(header->vl);
> +
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + header->max_vl = header->vl;
> +
> + header->size = SVE_PT_SIZE(vq, header->flags);
> + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> + SVE_PT_REGS_SVE);
> +}
> +
> +static unsigned int sve_size_from_header(struct user_sve_header const *header)
> +{
> + return ALIGN(header->size, SVE_VQ_BYTES);
> +}
> +
> +static unsigned int sve_get_size(struct task_struct *target,
> + const struct user_regset *regset)
> +{
> + struct user_sve_header header;
> +
> + if (!system_supports_sve())
> + return 0;
> +
> + sve_init_header_from_task(&header, target);
> + return sve_size_from_header(&header);
> +}
> +
> +static int sve_get(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf)
> +{
> + int ret;
> + struct user_sve_header header;
> + unsigned int vq;
> + unsigned long start, end;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + /* Header */
> + sve_init_header_from_task(&header, target);
> + vq = sve_vq_from_vl(header.vl);
> +
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &header,
> + 0, sizeof(header));
> + if (ret)
> + return ret;
> +
> + if (target == current)
> + fpsimd_preserve_current_state();
> +
> + /* Registers: FPSIMD-only case */
> +
> + BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
> + if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD)
> + return __fpr_get(target, regset, pos, count, kbuf, ubuf,
> + SVE_PT_FPSIMD_OFFSET);
> +
> + /* Otherwise: full SVE case */
> +
> + BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
> + start = SVE_PT_SVE_OFFSET;
> + end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
> + target->thread.sve_state,
> + start, end);
> + if (ret)
> + return ret;
> +
> + start = end;
> + end = SVE_PT_SVE_FPSR_OFFSET(vq);
> + ret = user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
> + start, end);
> + if (ret)
> + return ret;
> +
> + /*
> + * Copy fpsr, and fpcr which must follow contiguously in
> + * struct fpsimd_state:
> + */
> + start = end;
> + end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
> + &target->thread.fpsimd_state.fpsr,
> + start, end);
> + if (ret)
> + return ret;
> +
> + start = end;
> + end = sve_size_from_header(&header);
> + return user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
> + start, end);
> +}
> +
> +static int sve_set(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf)
> +{
> + int ret;
> + struct user_sve_header header;
> + unsigned int vq;
> + unsigned long start, end;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + /* Header */
> + if (count < sizeof(header))
> + return -EINVAL;
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &header,
> + 0, sizeof(header));
> + if (ret)
> + goto out;
> +
> + /*
> + * Apart from PT_SVE_REGS_MASK, all PT_SVE_* flags are consumed by
> + * sve_set_vector_length(), which will also validate them for us:
> + */
> + ret = sve_set_vector_length(target, header.vl,
> + header.flags & ~SVE_PT_REGS_MASK);
> + if (ret)
> + goto out;
> +
> + /* Actual VL set may be less than the user asked for: */
> + vq = sve_vq_from_vl(target->thread.sve_vl);
> +
> + /* Registers: FPSIMD-only case */
> +
> + BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
> + if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD) {
> + sve_sync_to_fpsimd(target);
> +
> + ret = __fpr_set(target, regset, pos, count, kbuf, ubuf,
> + SVE_PT_FPSIMD_OFFSET);
> + clear_tsk_thread_flag(target, TIF_SVE);
> + goto out;
> + }
> +
> + /* Otherwise: full SVE case */
> +
> + /*
> + * If setting a different VL from the requested VL and there is
> + * register data, the data layout will be wrong: don't even
> + * try to set the registers in this case.
> + */
> + if (count && vq != sve_vq_from_vl(header.vl)) {
> + ret = -EIO;
> + goto out;
> + }
> +
> + sve_alloc(target);
> + fpsimd_sync_to_sve(target);
> + set_tsk_thread_flag(target, TIF_SVE);
> +
> + BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
> + start = SVE_PT_SVE_OFFSET;
> + end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
> + target->thread.sve_state,
> + start, end);
> + if (ret)
> + goto out;
> +
> + start = end;
> + end = SVE_PT_SVE_FPSR_OFFSET(vq);
> + ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
> + start, end);
> + if (ret)
> + goto out;
> +
> + /*
> + * Copy fpsr, and fpcr which must follow contiguously in
> + * struct fpsimd_state:
> + */
> + start = end;
> + end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
> + &target->thread.fpsimd_state.fpsr,
> + start, end);
> +
> +out:
> + fpsimd_flush_task_state(target);
> + return ret;
> +}
> +
> +#endif /* CONFIG_ARM64_SVE */
> +
> enum aarch64_regset {
> REGSET_GPR,
> REGSET_FPR,
> @@ -711,6 +950,9 @@ enum aarch64_regset {
> REGSET_HW_WATCH,
> #endif
> REGSET_SYSTEM_CALL,
> +#ifdef CONFIG_ARM64_SVE
> + REGSET_SVE,
> +#endif
> };
>
> static const struct user_regset aarch64_regsets[] = {
> @@ -768,6 +1010,18 @@ static const struct user_regset aarch64_regsets[] = {
> .get = system_call_get,
> .set = system_call_set,
> },
> +#ifdef CONFIG_ARM64_SVE
> + [REGSET_SVE] = { /* Scalable Vector Extension */
> + .core_note_type = NT_ARM_SVE,
> + .n = DIV_ROUND_UP(SVE_PT_SIZE(SVE_VQ_MAX, SVE_PT_REGS_SVE),
> + SVE_VQ_BYTES),
> + .size = SVE_VQ_BYTES,
> + .align = SVE_VQ_BYTES,
> + .get = sve_get,
> + .set = sve_set,
> + .get_size = sve_get_size,
> + },
> +#endif
> };
>
> static const struct user_regset_view user_aarch64_view = {
> diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
> index b5280db..735b8f4 100644
> --- a/include/uapi/linux/elf.h
> +++ b/include/uapi/linux/elf.h
> @@ -416,6 +416,7 @@ typedef struct elf64_shdr {
> #define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
> #define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
> #define NT_ARM_SYSTEM_CALL 0x404 /* ARM system call number */
> +#define NT_ARM_SVE 0x405 /* ARM Scalable Vector Extension registers */
> #define NT_METAG_CBUF 0x500 /* Metag catch buffer registers */
> #define NT_METAG_RPIPE 0x501 /* Metag read pipeline state */
> #define NT_METAG_TLS 0x502 /* Metag TLS pointer */
Otherwise:
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-14 12:57 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 12:57 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
gdb, Alan Hayward, Yao Qi, Oleg Nesterov, Alexander Viro
Dave Martin <Dave.Martin@arm.com> writes:
> This patch defines and implements a new regset NT_ARM_SVE, which
> describes a thread's SVE register state. This allows a debugger to
> manipulate the SVE state, as well as being included in ELF
> coredumps for post-mortem debugging.
>
> Because the regset size and layout are dependent on the thread's
> current vector length, it is not possible to define a C struct to
> describe the regset contents as is done for existing regsets.
> Instead, and for the same reasons, NT_ARM_SVE is based on the
> freeform variable-layout approach used for the SVE signal frame.
>
> Additionally, to reduce debug overhead when debugging threads that
> might or might not have live SVE register state, NT_ARM_SVE may be
> presented in one of two different formats: the old struct
> user_fpsimd_state format is embedded for describing the state of a
> thread with no live SVE state, whereas a new variable-layout
> structure is embedded for describing live SVE state. This avoids a
> debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
> allows existing userspace code to handle the non-SVE case without
> too much modification.
>
> For this to work, NT_ARM_SVE is defined with a fixed-format header
> of type struct user_sve_header, which the recipient can use to
> figure out the content, size and layout of the reset of the regset.
> Accessor macros are defined to allow the vector-length-dependent
> parts of the regset to be manipulated.
>
> Signed-off-by: Alan Hayward <alan.hayward@arm.com>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Other changes related to Alex Bennée's comments:
>
> * Migrate to SVE_VQ_BYTES instead of magic numbers.
>
> Requested by Alex Bennée:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
>
> Other:
>
> * [ABI fix] Bail out with -EIO if attempting to set the
> SVE regs for an unsupported VL, instead of misparsing the regset data.
>
> * Replace some in-kernel open-coded arithmetic with ALIGN()/
> DIV_ROUND_UP().
> ---
> arch/arm64/include/asm/fpsimd.h | 13 +-
> arch/arm64/include/uapi/asm/ptrace.h | 135 ++++++++++++++++++
> arch/arm64/kernel/fpsimd.c | 40 +++++-
> arch/arm64/kernel/ptrace.c | 270 +++++++++++++++++++++++++++++++++--
> include/uapi/linux/elf.h | 1 +
> 5 files changed, 449 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 6c22624..2723cca 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -38,13 +38,16 @@ struct fpsimd_state {
> __uint128_t vregs[32];
> u32 fpsr;
> u32 fpcr;
> + /*
> + * For ptrace compatibility, pad to next 128-bit
> + * boundary here if extending this struct.
> + */
> };
> };
> /* the id of the last cpu to have restored this state */
> unsigned int cpu;
> };
>
> -
> #if defined(__KERNEL__) && defined(CONFIG_COMPAT)
> /* Masks for extracting the FPSR and FPCR from the FPSCR */
> #define VFP_FPSCR_STAT_MASK 0xf800009f
> @@ -89,6 +92,10 @@ extern void sve_alloc(struct task_struct *task);
> extern void fpsimd_release_thread(struct task_struct *task);
> extern void fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src);
> +extern void fpsimd_sync_to_sve(struct task_struct *task);
> +extern void sve_sync_to_fpsimd(struct task_struct *task);
> +extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
> +
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> @@ -103,6 +110,10 @@ static void __maybe_unused sve_alloc(struct task_struct *task) { }
> static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
> static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src) { }
> +static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
> +static void __maybe_unused sve_sync_from_fpsimd_zeropad(
> + struct task_struct *task) { }
> +
> static void __maybe_unused sve_init_vq_map(void) { }
> static void __maybe_unused sve_update_vq_map(void) { }
> static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
> index d1ff83d..1915ab0 100644
> --- a/arch/arm64/include/uapi/asm/ptrace.h
> +++ b/arch/arm64/include/uapi/asm/ptrace.h
> @@ -22,6 +22,7 @@
> #include <linux/types.h>
>
> #include <asm/hwcap.h>
> +#include <asm/sigcontext.h>
>
>
> /*
> @@ -63,6 +64,8 @@
>
> #ifndef __ASSEMBLY__
>
> +#include <linux/prctl.h>
> +
> /*
> * User structures for general purpose, floating point and debug registers.
> */
> @@ -90,6 +93,138 @@ struct user_hwdebug_state {
> } dbg_regs[16];
> };
>
> +/* SVE/FP/SIMD state (NT_ARM_SVE) */
> +
> +struct user_sve_header {
> + __u32 size; /* total meaningful regset content in bytes */
> + __u32 max_size; /* maxmium possible size for this thread */
> + __u16 vl; /* current vector length */
> + __u16 max_vl; /* maximum possible vector length */
> + __u16 flags;
> + __u16 __reserved;
> +};
> +
> +/* Definitions for user_sve_header.flags: */
> +#define SVE_PT_REGS_MASK (1 << 0)
> +
> +/* Flags: must be kept in sync with prctl interface in
> <linux/ptrace.h> */
Which flags? We base some on PR_foo flags but we seem to shift them
anyway so where is the requirement for them to match from?
> +#define SVE_PT_REGS_FPSIMD 0
> +#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
> +
> +#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
> +#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
> +
> +
> +/*
> + * The remainder of the SVE state follows struct user_sve_header. The
> + * total size of the SVE state (including header) depends on the
> + * metadata in the header: SVE_PT_SIZE(vq, flags) gives the total size
> + * of the state in bytes, including the header.
> + *
> + * Refer to <asm/sigcontext.h> for details of how to pass the correct
> + * "vq" argument to these macros.
> + */
> +
> +/* Offset from the start of struct user_sve_header to the register data */
> +#define SVE_PT_REGS_OFFSET \
> + ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +
> +/*
> + * The register data content and layout depends on the value of the
> + * flags field.
> + */
> +
> +/*
> + * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD case:
> + *
> + * The payload starts at offset SVE_PT_FPSIMD_OFFSET, and is of type
> + * struct user_fpsimd_state. Additional data might be appended in the
> + * future: use SVE_PT_FPSIMD_SIZE(vq, flags) to compute the total size.
> + * SVE_PT_FPSIMD_SIZE(vq, flags) will never be less than
> + * sizeof(struct user_fpsimd_state).
> + */
> +
> +#define SVE_PT_FPSIMD_OFFSET SVE_PT_REGS_OFFSET
> +
> +#define SVE_PT_FPSIMD_SIZE(vq, flags) (sizeof(struct user_fpsimd_state))
> +
> +/*
> + * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE case:
> + *
> + * The payload starts at offset SVE_PT_SVE_OFFSET, and is of size
> + * SVE_PT_SVE_SIZE(vq, flags).
> + *
> + * Additional macros describe the contents and layout of the payload.
> + * For each, SVE_PT_SVE_x_OFFSET(args) is the start offset relative to
> + * the start of struct user_sve_header, and SVE_PT_SVE_x_SIZE(args) is
> + * the size in bytes:
> + *
> + * x type description
> + * - ---- -----------
> + * ZREGS \
> + * ZREG |
> + * PREGS | refer to <asm/sigcontext.h>
> + * PREG |
> + * FFR /
> + *
> + * FPSR uint32_t FPSR
> + * FPCR uint32_t FPCR
> + *
> + * Additional data might be appended in the future.
> + */
> +
> +#define SVE_PT_SVE_ZREG_SIZE(vq) SVE_SIG_ZREG_SIZE(vq)
> +#define SVE_PT_SVE_PREG_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
> +#define SVE_PT_SVE_FFR_SIZE(vq) SVE_SIG_FFR_SIZE(vq)
> +#define SVE_PT_SVE_FPSR_SIZE sizeof(__u32)
> +#define SVE_PT_SVE_FPCR_SIZE sizeof(__u32)
> +
> +#define __SVE_SIG_TO_PT(offset) \
> + ((offset) - SVE_SIG_REGS_OFFSET + SVE_PT_REGS_OFFSET)
> +
> +#define SVE_PT_SVE_OFFSET SVE_PT_REGS_OFFSET
> +
> +#define SVE_PT_SVE_ZREGS_OFFSET \
> + __SVE_SIG_TO_PT(SVE_SIG_ZREGS_OFFSET)
> +#define SVE_PT_SVE_ZREG_OFFSET(vq, n) \
> + __SVE_SIG_TO_PT(SVE_SIG_ZREG_OFFSET(vq, n))
> +#define SVE_PT_SVE_ZREGS_SIZE(vq) \
> + (SVE_PT_SVE_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_PT_SVE_ZREGS_OFFSET)
> +
> +#define SVE_PT_SVE_PREGS_OFFSET(vq) \
> + __SVE_SIG_TO_PT(SVE_SIG_PREGS_OFFSET(vq))
> +#define SVE_PT_SVE_PREG_OFFSET(vq, n) \
> + __SVE_SIG_TO_PT(SVE_SIG_PREG_OFFSET(vq, n))
> +#define SVE_PT_SVE_PREGS_SIZE(vq) \
> + (SVE_PT_SVE_PREG_OFFSET(vq, SVE_NUM_PREGS) - \
> + SVE_PT_SVE_PREGS_OFFSET(vq))
> +
> +#define SVE_PT_SVE_FFR_OFFSET(vq) \
> + __SVE_SIG_TO_PT(SVE_SIG_FFR_OFFSET(vq))
> +
> +#define SVE_PT_SVE_FPSR_OFFSET(vq) \
> + ((SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq) + \
> + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +#define SVE_PT_SVE_FPCR_OFFSET(vq) \
> + (SVE_PT_SVE_FPSR_OFFSET(vq) + SVE_PT_SVE_FPSR_SIZE)
> +
> +/*
> + * Any future extension appended after FPCR must be aligned to the next
> + * 128-bit boundary.
> + */
> +
> +#define SVE_PT_SVE_SIZE(vq, flags) \
> + ((SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE \
> + - SVE_PT_SVE_OFFSET + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +
> +#define SVE_PT_SIZE(vq, flags) \
> + (((flags) & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE ? \
> + SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, flags) \
> + : SVE_PT_FPSIMD_OFFSET + SVE_PT_FPSIMD_SIZE(vq, flags))
> +
> #endif /* __ASSEMBLY__ */
>
> #endif /* _UAPI__ASM_PTRACE_H */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index fff9fcf..361c019 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -303,6 +303,37 @@ void sve_alloc(struct task_struct *task)
> BUG_ON(!task->thread.sve_state);
> }
>
> +void fpsimd_sync_to_sve(struct task_struct *task)
> +{
> + if (!test_tsk_thread_flag(task, TIF_SVE))
> + fpsimd_to_sve(task);
> +}
> +
> +void sve_sync_to_fpsimd(struct task_struct *task)
> +{
> + if (test_tsk_thread_flag(task, TIF_SVE))
> + sve_to_fpsimd(task);
> +}
> +
> +void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
> +{
> + unsigned int vq;
> + void *sst = task->thread.sve_state;
> + struct fpsimd_state const *fst = &task->thread.fpsimd_state;
> + unsigned int i;
> +
> + if (!test_tsk_thread_flag(task, TIF_SVE))
> + return;
> +
> + vq = sve_vq_from_vl(task->thread.sve_vl);
> +
> + memset(sst, 0, SVE_SIG_REGS_SIZE(vq));
> +
> + for (i = 0; i < 32; ++i)
> + memcpy(ZREG(sst, vq, i), &fst->vregs[i],
> + sizeof(fst->vregs[i]));
> +}
> +
> /*
> * Handle SVE state across fork():
> *
> @@ -459,10 +490,17 @@ static void __init sve_efi_setup(void)
> * This is evidence of a crippled system and we are returning void,
> * so no attempt is made to handle this situation here.
> */
> - BUG_ON(!sve_vl_valid(sve_max_vl));
> + if (!sve_vl_valid(sve_max_vl))
> + goto fail;
> +
> efi_sve_state = __alloc_percpu(
> SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
> if (!efi_sve_state)
> + goto fail;
> +
> + return;
> +
> +fail:
> panic("Cannot allocate percpu memory for EFI SVE save/restore");
> }
>
> diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
> index 9cbb612..5ef4735b 100644
> --- a/arch/arm64/kernel/ptrace.c
> +++ b/arch/arm64/kernel/ptrace.c
> @@ -32,6 +32,7 @@
> #include <linux/security.h>
> #include <linux/init.h>
> #include <linux/signal.h>
> +#include <linux/string.h>
> #include <linux/uaccess.h>
> #include <linux/perf_event.h>
> #include <linux/hw_breakpoint.h>
> @@ -40,6 +41,7 @@
> #include <linux/elf.h>
>
> #include <asm/compat.h>
> +#include <asm/cpufeature.h>
> #include <asm/debug-monitors.h>
> #include <asm/pgtable.h>
> #include <asm/stacktrace.h>
> @@ -618,33 +620,66 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
> /*
> * TODO: update fp accessors for lazy context switching (sync/flush hwstate)
> */
> -static int fpr_get(struct task_struct *target, const struct user_regset *regset,
> - unsigned int pos, unsigned int count,
> - void *kbuf, void __user *ubuf)
> +static int __fpr_get(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf, unsigned int start_pos)
> {
> struct user_fpsimd_state *uregs;
> +
> + sve_sync_to_fpsimd(target);
> +
> uregs = &target->thread.fpsimd_state.user_fpsimd;
>
> + return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs,
> + start_pos, start_pos + sizeof(*uregs));
> +}
> +
> +static int fpr_get(struct task_struct *target, const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf)
> +{
> if (target == current)
> fpsimd_preserve_current_state();
>
> - return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
> + return __fpr_get(target, regset, pos, count, kbuf, ubuf, 0);
> }
>
> -static int fpr_set(struct task_struct *target, const struct user_regset *regset,
> - unsigned int pos, unsigned int count,
> - const void *kbuf, const void __user *ubuf)
> +static int __fpr_set(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf,
> + unsigned int start_pos)
> {
> int ret;
> struct user_fpsimd_state newstate =
> target->thread.fpsimd_state.user_fpsimd;
>
> - ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
> + sve_sync_to_fpsimd(target);
> +
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate,
> + start_pos, start_pos + sizeof(newstate));
> if (ret)
> return ret;
>
> target->thread.fpsimd_state.user_fpsimd = newstate;
> +
> + return ret;
> +}
> +
> +static int fpr_set(struct task_struct *target, const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf)
> +{
> + int ret;
> +
> + ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, 0);
> + if (ret)
> + return ret;
> +
> + sve_sync_from_fpsimd_zeropad(target);
> fpsimd_flush_task_state(target);
> +
> return ret;
> }
>
> @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
> return ret;
> }
>
> +#ifdef CONFIG_ARM64_SVE
> +
> +static void sve_init_header_from_task(struct user_sve_header *header,
> + struct task_struct *target)
> +{
> + unsigned int vq;
> +
> + memset(header, 0, sizeof(*header));
> +
> + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> + header->flags |= SVE_PT_VL_INHERIT;
> +
> + header->vl = target->thread.sve_vl;
> + vq = sve_vq_from_vl(header->vl);
> +
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + header->max_vl = header->vl;
> +
> + header->size = SVE_PT_SIZE(vq, header->flags);
> + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> + SVE_PT_REGS_SVE);
> +}
> +
> +static unsigned int sve_size_from_header(struct user_sve_header const *header)
> +{
> + return ALIGN(header->size, SVE_VQ_BYTES);
> +}
> +
> +static unsigned int sve_get_size(struct task_struct *target,
> + const struct user_regset *regset)
> +{
> + struct user_sve_header header;
> +
> + if (!system_supports_sve())
> + return 0;
> +
> + sve_init_header_from_task(&header, target);
> + return sve_size_from_header(&header);
> +}
> +
> +static int sve_get(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf)
> +{
> + int ret;
> + struct user_sve_header header;
> + unsigned int vq;
> + unsigned long start, end;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + /* Header */
> + sve_init_header_from_task(&header, target);
> + vq = sve_vq_from_vl(header.vl);
> +
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &header,
> + 0, sizeof(header));
> + if (ret)
> + return ret;
> +
> + if (target == current)
> + fpsimd_preserve_current_state();
> +
> + /* Registers: FPSIMD-only case */
> +
> + BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
> + if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD)
> + return __fpr_get(target, regset, pos, count, kbuf, ubuf,
> + SVE_PT_FPSIMD_OFFSET);
> +
> + /* Otherwise: full SVE case */
> +
> + BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
> + start = SVE_PT_SVE_OFFSET;
> + end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
> + target->thread.sve_state,
> + start, end);
> + if (ret)
> + return ret;
> +
> + start = end;
> + end = SVE_PT_SVE_FPSR_OFFSET(vq);
> + ret = user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
> + start, end);
> + if (ret)
> + return ret;
> +
> + /*
> + * Copy fpsr, and fpcr which must follow contiguously in
> + * struct fpsimd_state:
> + */
> + start = end;
> + end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
> + &target->thread.fpsimd_state.fpsr,
> + start, end);
> + if (ret)
> + return ret;
> +
> + start = end;
> + end = sve_size_from_header(&header);
> + return user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
> + start, end);
> +}
> +
> +static int sve_set(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf)
> +{
> + int ret;
> + struct user_sve_header header;
> + unsigned int vq;
> + unsigned long start, end;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + /* Header */
> + if (count < sizeof(header))
> + return -EINVAL;
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &header,
> + 0, sizeof(header));
> + if (ret)
> + goto out;
> +
> + /*
> + * Apart from PT_SVE_REGS_MASK, all PT_SVE_* flags are consumed by
> + * sve_set_vector_length(), which will also validate them for us:
> + */
> + ret = sve_set_vector_length(target, header.vl,
> + header.flags & ~SVE_PT_REGS_MASK);
> + if (ret)
> + goto out;
> +
> + /* Actual VL set may be less than the user asked for: */
> + vq = sve_vq_from_vl(target->thread.sve_vl);
> +
> + /* Registers: FPSIMD-only case */
> +
> + BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
> + if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD) {
> + sve_sync_to_fpsimd(target);
> +
> + ret = __fpr_set(target, regset, pos, count, kbuf, ubuf,
> + SVE_PT_FPSIMD_OFFSET);
> + clear_tsk_thread_flag(target, TIF_SVE);
> + goto out;
> + }
> +
> + /* Otherwise: full SVE case */
> +
> + /*
> + * If setting a different VL from the requested VL and there is
> + * register data, the data layout will be wrong: don't even
> + * try to set the registers in this case.
> + */
> + if (count && vq != sve_vq_from_vl(header.vl)) {
> + ret = -EIO;
> + goto out;
> + }
> +
> + sve_alloc(target);
> + fpsimd_sync_to_sve(target);
> + set_tsk_thread_flag(target, TIF_SVE);
> +
> + BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
> + start = SVE_PT_SVE_OFFSET;
> + end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
> + target->thread.sve_state,
> + start, end);
> + if (ret)
> + goto out;
> +
> + start = end;
> + end = SVE_PT_SVE_FPSR_OFFSET(vq);
> + ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
> + start, end);
> + if (ret)
> + goto out;
> +
> + /*
> + * Copy fpsr, and fpcr which must follow contiguously in
> + * struct fpsimd_state:
> + */
> + start = end;
> + end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
> + &target->thread.fpsimd_state.fpsr,
> + start, end);
> +
> +out:
> + fpsimd_flush_task_state(target);
> + return ret;
> +}
> +
> +#endif /* CONFIG_ARM64_SVE */
> +
> enum aarch64_regset {
> REGSET_GPR,
> REGSET_FPR,
> @@ -711,6 +950,9 @@ enum aarch64_regset {
> REGSET_HW_WATCH,
> #endif
> REGSET_SYSTEM_CALL,
> +#ifdef CONFIG_ARM64_SVE
> + REGSET_SVE,
> +#endif
> };
>
> static const struct user_regset aarch64_regsets[] = {
> @@ -768,6 +1010,18 @@ static const struct user_regset aarch64_regsets[] = {
> .get = system_call_get,
> .set = system_call_set,
> },
> +#ifdef CONFIG_ARM64_SVE
> + [REGSET_SVE] = { /* Scalable Vector Extension */
> + .core_note_type = NT_ARM_SVE,
> + .n = DIV_ROUND_UP(SVE_PT_SIZE(SVE_VQ_MAX, SVE_PT_REGS_SVE),
> + SVE_VQ_BYTES),
> + .size = SVE_VQ_BYTES,
> + .align = SVE_VQ_BYTES,
> + .get = sve_get,
> + .set = sve_set,
> + .get_size = sve_get_size,
> + },
> +#endif
> };
>
> static const struct user_regset_view user_aarch64_view = {
> diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
> index b5280db..735b8f4 100644
> --- a/include/uapi/linux/elf.h
> +++ b/include/uapi/linux/elf.h
> @@ -416,6 +416,7 @@ typedef struct elf64_shdr {
> #define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
> #define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
> #define NT_ARM_SYSTEM_CALL 0x404 /* ARM system call number */
> +#define NT_ARM_SVE 0x405 /* ARM Scalable Vector Extension registers */
> #define NT_METAG_CBUF 0x500 /* Metag catch buffer registers */
> #define NT_METAG_RPIPE 0x501 /* Metag read pipeline state */
> #define NT_METAG_TLS 0x502 /* Metag TLS pointer */
Otherwise:
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-14 12:57 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 12:57 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> This patch defines and implements a new regset NT_ARM_SVE, which
> describes a thread's SVE register state. This allows a debugger to
> manipulate the SVE state, as well as being included in ELF
> coredumps for post-mortem debugging.
>
> Because the regset size and layout are dependent on the thread's
> current vector length, it is not possible to define a C struct to
> describe the regset contents as is done for existing regsets.
> Instead, and for the same reasons, NT_ARM_SVE is based on the
> freeform variable-layout approach used for the SVE signal frame.
>
> Additionally, to reduce debug overhead when debugging threads that
> might or might not have live SVE register state, NT_ARM_SVE may be
> presented in one of two different formats: the old struct
> user_fpsimd_state format is embedded for describing the state of a
> thread with no live SVE state, whereas a new variable-layout
> structure is embedded for describing live SVE state. This avoids a
> debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
> allows existing userspace code to handle the non-SVE case without
> too much modification.
>
> For this to work, NT_ARM_SVE is defined with a fixed-format header
> of type struct user_sve_header, which the recipient can use to
> figure out the content, size and layout of the reset of the regset.
> Accessor macros are defined to allow the vector-length-dependent
> parts of the regset to be manipulated.
>
> Signed-off-by: Alan Hayward <alan.hayward@arm.com>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Other changes related to Alex Benn?e's comments:
>
> * Migrate to SVE_VQ_BYTES instead of magic numbers.
>
> Requested by Alex Benn?e:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
>
> Other:
>
> * [ABI fix] Bail out with -EIO if attempting to set the
> SVE regs for an unsupported VL, instead of misparsing the regset data.
>
> * Replace some in-kernel open-coded arithmetic with ALIGN()/
> DIV_ROUND_UP().
> ---
> arch/arm64/include/asm/fpsimd.h | 13 +-
> arch/arm64/include/uapi/asm/ptrace.h | 135 ++++++++++++++++++
> arch/arm64/kernel/fpsimd.c | 40 +++++-
> arch/arm64/kernel/ptrace.c | 270 +++++++++++++++++++++++++++++++++--
> include/uapi/linux/elf.h | 1 +
> 5 files changed, 449 insertions(+), 10 deletions(-)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 6c22624..2723cca 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -38,13 +38,16 @@ struct fpsimd_state {
> __uint128_t vregs[32];
> u32 fpsr;
> u32 fpcr;
> + /*
> + * For ptrace compatibility, pad to next 128-bit
> + * boundary here if extending this struct.
> + */
> };
> };
> /* the id of the last cpu to have restored this state */
> unsigned int cpu;
> };
>
> -
> #if defined(__KERNEL__) && defined(CONFIG_COMPAT)
> /* Masks for extracting the FPSR and FPCR from the FPSCR */
> #define VFP_FPSCR_STAT_MASK 0xf800009f
> @@ -89,6 +92,10 @@ extern void sve_alloc(struct task_struct *task);
> extern void fpsimd_release_thread(struct task_struct *task);
> extern void fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src);
> +extern void fpsimd_sync_to_sve(struct task_struct *task);
> +extern void sve_sync_to_fpsimd(struct task_struct *task);
> +extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
> +
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> @@ -103,6 +110,10 @@ static void __maybe_unused sve_alloc(struct task_struct *task) { }
> static void __maybe_unused fpsimd_release_thread(struct task_struct *task) { }
> static void __maybe_unused fpsimd_dup_sve(struct task_struct *dst,
> struct task_struct const *src) { }
> +static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
> +static void __maybe_unused sve_sync_from_fpsimd_zeropad(
> + struct task_struct *task) { }
> +
> static void __maybe_unused sve_init_vq_map(void) { }
> static void __maybe_unused sve_update_vq_map(void) { }
> static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
> index d1ff83d..1915ab0 100644
> --- a/arch/arm64/include/uapi/asm/ptrace.h
> +++ b/arch/arm64/include/uapi/asm/ptrace.h
> @@ -22,6 +22,7 @@
> #include <linux/types.h>
>
> #include <asm/hwcap.h>
> +#include <asm/sigcontext.h>
>
>
> /*
> @@ -63,6 +64,8 @@
>
> #ifndef __ASSEMBLY__
>
> +#include <linux/prctl.h>
> +
> /*
> * User structures for general purpose, floating point and debug registers.
> */
> @@ -90,6 +93,138 @@ struct user_hwdebug_state {
> } dbg_regs[16];
> };
>
> +/* SVE/FP/SIMD state (NT_ARM_SVE) */
> +
> +struct user_sve_header {
> + __u32 size; /* total meaningful regset content in bytes */
> + __u32 max_size; /* maxmium possible size for this thread */
> + __u16 vl; /* current vector length */
> + __u16 max_vl; /* maximum possible vector length */
> + __u16 flags;
> + __u16 __reserved;
> +};
> +
> +/* Definitions for user_sve_header.flags: */
> +#define SVE_PT_REGS_MASK (1 << 0)
> +
> +/* Flags: must be kept in sync with prctl interface in
> <linux/ptrace.h> */
Which flags? We base some on PR_foo flags but we seem to shift them
anyway so where is the requirement for them to match from?
> +#define SVE_PT_REGS_FPSIMD 0
> +#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
> +
> +#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
> +#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
> +
> +
> +/*
> + * The remainder of the SVE state follows struct user_sve_header. The
> + * total size of the SVE state (including header) depends on the
> + * metadata in the header: SVE_PT_SIZE(vq, flags) gives the total size
> + * of the state in bytes, including the header.
> + *
> + * Refer to <asm/sigcontext.h> for details of how to pass the correct
> + * "vq" argument to these macros.
> + */
> +
> +/* Offset from the start of struct user_sve_header to the register data */
> +#define SVE_PT_REGS_OFFSET \
> + ((sizeof(struct sve_context) + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +
> +/*
> + * The register data content and layout depends on the value of the
> + * flags field.
> + */
> +
> +/*
> + * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD case:
> + *
> + * The payload starts at offset SVE_PT_FPSIMD_OFFSET, and is of type
> + * struct user_fpsimd_state. Additional data might be appended in the
> + * future: use SVE_PT_FPSIMD_SIZE(vq, flags) to compute the total size.
> + * SVE_PT_FPSIMD_SIZE(vq, flags) will never be less than
> + * sizeof(struct user_fpsimd_state).
> + */
> +
> +#define SVE_PT_FPSIMD_OFFSET SVE_PT_REGS_OFFSET
> +
> +#define SVE_PT_FPSIMD_SIZE(vq, flags) (sizeof(struct user_fpsimd_state))
> +
> +/*
> + * (flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE case:
> + *
> + * The payload starts at offset SVE_PT_SVE_OFFSET, and is of size
> + * SVE_PT_SVE_SIZE(vq, flags).
> + *
> + * Additional macros describe the contents and layout of the payload.
> + * For each, SVE_PT_SVE_x_OFFSET(args) is the start offset relative to
> + * the start of struct user_sve_header, and SVE_PT_SVE_x_SIZE(args) is
> + * the size in bytes:
> + *
> + * x type description
> + * - ---- -----------
> + * ZREGS \
> + * ZREG |
> + * PREGS | refer to <asm/sigcontext.h>
> + * PREG |
> + * FFR /
> + *
> + * FPSR uint32_t FPSR
> + * FPCR uint32_t FPCR
> + *
> + * Additional data might be appended in the future.
> + */
> +
> +#define SVE_PT_SVE_ZREG_SIZE(vq) SVE_SIG_ZREG_SIZE(vq)
> +#define SVE_PT_SVE_PREG_SIZE(vq) SVE_SIG_PREG_SIZE(vq)
> +#define SVE_PT_SVE_FFR_SIZE(vq) SVE_SIG_FFR_SIZE(vq)
> +#define SVE_PT_SVE_FPSR_SIZE sizeof(__u32)
> +#define SVE_PT_SVE_FPCR_SIZE sizeof(__u32)
> +
> +#define __SVE_SIG_TO_PT(offset) \
> + ((offset) - SVE_SIG_REGS_OFFSET + SVE_PT_REGS_OFFSET)
> +
> +#define SVE_PT_SVE_OFFSET SVE_PT_REGS_OFFSET
> +
> +#define SVE_PT_SVE_ZREGS_OFFSET \
> + __SVE_SIG_TO_PT(SVE_SIG_ZREGS_OFFSET)
> +#define SVE_PT_SVE_ZREG_OFFSET(vq, n) \
> + __SVE_SIG_TO_PT(SVE_SIG_ZREG_OFFSET(vq, n))
> +#define SVE_PT_SVE_ZREGS_SIZE(vq) \
> + (SVE_PT_SVE_ZREG_OFFSET(vq, SVE_NUM_ZREGS) - SVE_PT_SVE_ZREGS_OFFSET)
> +
> +#define SVE_PT_SVE_PREGS_OFFSET(vq) \
> + __SVE_SIG_TO_PT(SVE_SIG_PREGS_OFFSET(vq))
> +#define SVE_PT_SVE_PREG_OFFSET(vq, n) \
> + __SVE_SIG_TO_PT(SVE_SIG_PREG_OFFSET(vq, n))
> +#define SVE_PT_SVE_PREGS_SIZE(vq) \
> + (SVE_PT_SVE_PREG_OFFSET(vq, SVE_NUM_PREGS) - \
> + SVE_PT_SVE_PREGS_OFFSET(vq))
> +
> +#define SVE_PT_SVE_FFR_OFFSET(vq) \
> + __SVE_SIG_TO_PT(SVE_SIG_FFR_OFFSET(vq))
> +
> +#define SVE_PT_SVE_FPSR_OFFSET(vq) \
> + ((SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq) + \
> + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +#define SVE_PT_SVE_FPCR_OFFSET(vq) \
> + (SVE_PT_SVE_FPSR_OFFSET(vq) + SVE_PT_SVE_FPSR_SIZE)
> +
> +/*
> + * Any future extension appended after FPCR must be aligned to the next
> + * 128-bit boundary.
> + */
> +
> +#define SVE_PT_SVE_SIZE(vq, flags) \
> + ((SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE \
> + - SVE_PT_SVE_OFFSET + (SVE_VQ_BYTES - 1)) \
> + / SVE_VQ_BYTES * SVE_VQ_BYTES)
> +
> +#define SVE_PT_SIZE(vq, flags) \
> + (((flags) & SVE_PT_REGS_MASK) == SVE_PT_REGS_SVE ? \
> + SVE_PT_SVE_OFFSET + SVE_PT_SVE_SIZE(vq, flags) \
> + : SVE_PT_FPSIMD_OFFSET + SVE_PT_FPSIMD_SIZE(vq, flags))
> +
> #endif /* __ASSEMBLY__ */
>
> #endif /* _UAPI__ASM_PTRACE_H */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index fff9fcf..361c019 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -303,6 +303,37 @@ void sve_alloc(struct task_struct *task)
> BUG_ON(!task->thread.sve_state);
> }
>
> +void fpsimd_sync_to_sve(struct task_struct *task)
> +{
> + if (!test_tsk_thread_flag(task, TIF_SVE))
> + fpsimd_to_sve(task);
> +}
> +
> +void sve_sync_to_fpsimd(struct task_struct *task)
> +{
> + if (test_tsk_thread_flag(task, TIF_SVE))
> + sve_to_fpsimd(task);
> +}
> +
> +void sve_sync_from_fpsimd_zeropad(struct task_struct *task)
> +{
> + unsigned int vq;
> + void *sst = task->thread.sve_state;
> + struct fpsimd_state const *fst = &task->thread.fpsimd_state;
> + unsigned int i;
> +
> + if (!test_tsk_thread_flag(task, TIF_SVE))
> + return;
> +
> + vq = sve_vq_from_vl(task->thread.sve_vl);
> +
> + memset(sst, 0, SVE_SIG_REGS_SIZE(vq));
> +
> + for (i = 0; i < 32; ++i)
> + memcpy(ZREG(sst, vq, i), &fst->vregs[i],
> + sizeof(fst->vregs[i]));
> +}
> +
> /*
> * Handle SVE state across fork():
> *
> @@ -459,10 +490,17 @@ static void __init sve_efi_setup(void)
> * This is evidence of a crippled system and we are returning void,
> * so no attempt is made to handle this situation here.
> */
> - BUG_ON(!sve_vl_valid(sve_max_vl));
> + if (!sve_vl_valid(sve_max_vl))
> + goto fail;
> +
> efi_sve_state = __alloc_percpu(
> SVE_SIG_REGS_SIZE(sve_vq_from_vl(sve_max_vl)), SVE_VQ_BYTES);
> if (!efi_sve_state)
> + goto fail;
> +
> + return;
> +
> +fail:
> panic("Cannot allocate percpu memory for EFI SVE save/restore");
> }
>
> diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c
> index 9cbb612..5ef4735b 100644
> --- a/arch/arm64/kernel/ptrace.c
> +++ b/arch/arm64/kernel/ptrace.c
> @@ -32,6 +32,7 @@
> #include <linux/security.h>
> #include <linux/init.h>
> #include <linux/signal.h>
> +#include <linux/string.h>
> #include <linux/uaccess.h>
> #include <linux/perf_event.h>
> #include <linux/hw_breakpoint.h>
> @@ -40,6 +41,7 @@
> #include <linux/elf.h>
>
> #include <asm/compat.h>
> +#include <asm/cpufeature.h>
> #include <asm/debug-monitors.h>
> #include <asm/pgtable.h>
> #include <asm/stacktrace.h>
> @@ -618,33 +620,66 @@ static int gpr_set(struct task_struct *target, const struct user_regset *regset,
> /*
> * TODO: update fp accessors for lazy context switching (sync/flush hwstate)
> */
> -static int fpr_get(struct task_struct *target, const struct user_regset *regset,
> - unsigned int pos, unsigned int count,
> - void *kbuf, void __user *ubuf)
> +static int __fpr_get(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf, unsigned int start_pos)
> {
> struct user_fpsimd_state *uregs;
> +
> + sve_sync_to_fpsimd(target);
> +
> uregs = &target->thread.fpsimd_state.user_fpsimd;
>
> + return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs,
> + start_pos, start_pos + sizeof(*uregs));
> +}
> +
> +static int fpr_get(struct task_struct *target, const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf)
> +{
> if (target == current)
> fpsimd_preserve_current_state();
>
> - return user_regset_copyout(&pos, &count, &kbuf, &ubuf, uregs, 0, -1);
> + return __fpr_get(target, regset, pos, count, kbuf, ubuf, 0);
> }
>
> -static int fpr_set(struct task_struct *target, const struct user_regset *regset,
> - unsigned int pos, unsigned int count,
> - const void *kbuf, const void __user *ubuf)
> +static int __fpr_set(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf,
> + unsigned int start_pos)
> {
> int ret;
> struct user_fpsimd_state newstate =
> target->thread.fpsimd_state.user_fpsimd;
>
> - ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate, 0, -1);
> + sve_sync_to_fpsimd(target);
> +
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &newstate,
> + start_pos, start_pos + sizeof(newstate));
> if (ret)
> return ret;
>
> target->thread.fpsimd_state.user_fpsimd = newstate;
> +
> + return ret;
> +}
> +
> +static int fpr_set(struct task_struct *target, const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf)
> +{
> + int ret;
> +
> + ret = __fpr_set(target, regset, pos, count, kbuf, ubuf, 0);
> + if (ret)
> + return ret;
> +
> + sve_sync_from_fpsimd_zeropad(target);
> fpsimd_flush_task_state(target);
> +
> return ret;
> }
>
> @@ -702,6 +737,210 @@ static int system_call_set(struct task_struct *target,
> return ret;
> }
>
> +#ifdef CONFIG_ARM64_SVE
> +
> +static void sve_init_header_from_task(struct user_sve_header *header,
> + struct task_struct *target)
> +{
> + unsigned int vq;
> +
> + memset(header, 0, sizeof(*header));
> +
> + header->flags = test_tsk_thread_flag(target, TIF_SVE) ?
> + SVE_PT_REGS_SVE : SVE_PT_REGS_FPSIMD;
> + if (test_tsk_thread_flag(target, TIF_SVE_VL_INHERIT))
> + header->flags |= SVE_PT_VL_INHERIT;
> +
> + header->vl = target->thread.sve_vl;
> + vq = sve_vq_from_vl(header->vl);
> +
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + header->max_vl = header->vl;
> +
> + header->size = SVE_PT_SIZE(vq, header->flags);
> + header->max_size = SVE_PT_SIZE(sve_vq_from_vl(header->max_vl),
> + SVE_PT_REGS_SVE);
> +}
> +
> +static unsigned int sve_size_from_header(struct user_sve_header const *header)
> +{
> + return ALIGN(header->size, SVE_VQ_BYTES);
> +}
> +
> +static unsigned int sve_get_size(struct task_struct *target,
> + const struct user_regset *regset)
> +{
> + struct user_sve_header header;
> +
> + if (!system_supports_sve())
> + return 0;
> +
> + sve_init_header_from_task(&header, target);
> + return sve_size_from_header(&header);
> +}
> +
> +static int sve_get(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + void *kbuf, void __user *ubuf)
> +{
> + int ret;
> + struct user_sve_header header;
> + unsigned int vq;
> + unsigned long start, end;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + /* Header */
> + sve_init_header_from_task(&header, target);
> + vq = sve_vq_from_vl(header.vl);
> +
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, &header,
> + 0, sizeof(header));
> + if (ret)
> + return ret;
> +
> + if (target == current)
> + fpsimd_preserve_current_state();
> +
> + /* Registers: FPSIMD-only case */
> +
> + BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
> + if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD)
> + return __fpr_get(target, regset, pos, count, kbuf, ubuf,
> + SVE_PT_FPSIMD_OFFSET);
> +
> + /* Otherwise: full SVE case */
> +
> + BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
> + start = SVE_PT_SVE_OFFSET;
> + end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
> + target->thread.sve_state,
> + start, end);
> + if (ret)
> + return ret;
> +
> + start = end;
> + end = SVE_PT_SVE_FPSR_OFFSET(vq);
> + ret = user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
> + start, end);
> + if (ret)
> + return ret;
> +
> + /*
> + * Copy fpsr, and fpcr which must follow contiguously in
> + * struct fpsimd_state:
> + */
> + start = end;
> + end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
> + ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
> + &target->thread.fpsimd_state.fpsr,
> + start, end);
> + if (ret)
> + return ret;
> +
> + start = end;
> + end = sve_size_from_header(&header);
> + return user_regset_copyout_zero(&pos, &count, &kbuf, &ubuf,
> + start, end);
> +}
> +
> +static int sve_set(struct task_struct *target,
> + const struct user_regset *regset,
> + unsigned int pos, unsigned int count,
> + const void *kbuf, const void __user *ubuf)
> +{
> + int ret;
> + struct user_sve_header header;
> + unsigned int vq;
> + unsigned long start, end;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + /* Header */
> + if (count < sizeof(header))
> + return -EINVAL;
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &header,
> + 0, sizeof(header));
> + if (ret)
> + goto out;
> +
> + /*
> + * Apart from PT_SVE_REGS_MASK, all PT_SVE_* flags are consumed by
> + * sve_set_vector_length(), which will also validate them for us:
> + */
> + ret = sve_set_vector_length(target, header.vl,
> + header.flags & ~SVE_PT_REGS_MASK);
> + if (ret)
> + goto out;
> +
> + /* Actual VL set may be less than the user asked for: */
> + vq = sve_vq_from_vl(target->thread.sve_vl);
> +
> + /* Registers: FPSIMD-only case */
> +
> + BUILD_BUG_ON(SVE_PT_FPSIMD_OFFSET != sizeof(header));
> + if ((header.flags & SVE_PT_REGS_MASK) == SVE_PT_REGS_FPSIMD) {
> + sve_sync_to_fpsimd(target);
> +
> + ret = __fpr_set(target, regset, pos, count, kbuf, ubuf,
> + SVE_PT_FPSIMD_OFFSET);
> + clear_tsk_thread_flag(target, TIF_SVE);
> + goto out;
> + }
> +
> + /* Otherwise: full SVE case */
> +
> + /*
> + * If setting a different VL from the requested VL and there is
> + * register data, the data layout will be wrong: don't even
> + * try to set the registers in this case.
> + */
> + if (count && vq != sve_vq_from_vl(header.vl)) {
> + ret = -EIO;
> + goto out;
> + }
> +
> + sve_alloc(target);
> + fpsimd_sync_to_sve(target);
> + set_tsk_thread_flag(target, TIF_SVE);
> +
> + BUILD_BUG_ON(SVE_PT_SVE_OFFSET != sizeof(header));
> + start = SVE_PT_SVE_OFFSET;
> + end = SVE_PT_SVE_FFR_OFFSET(vq) + SVE_PT_SVE_FFR_SIZE(vq);
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
> + target->thread.sve_state,
> + start, end);
> + if (ret)
> + goto out;
> +
> + start = end;
> + end = SVE_PT_SVE_FPSR_OFFSET(vq);
> + ret = user_regset_copyin_ignore(&pos, &count, &kbuf, &ubuf,
> + start, end);
> + if (ret)
> + goto out;
> +
> + /*
> + * Copy fpsr, and fpcr which must follow contiguously in
> + * struct fpsimd_state:
> + */
> + start = end;
> + end = SVE_PT_SVE_FPCR_OFFSET(vq) + SVE_PT_SVE_FPCR_SIZE;
> + ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
> + &target->thread.fpsimd_state.fpsr,
> + start, end);
> +
> +out:
> + fpsimd_flush_task_state(target);
> + return ret;
> +}
> +
> +#endif /* CONFIG_ARM64_SVE */
> +
> enum aarch64_regset {
> REGSET_GPR,
> REGSET_FPR,
> @@ -711,6 +950,9 @@ enum aarch64_regset {
> REGSET_HW_WATCH,
> #endif
> REGSET_SYSTEM_CALL,
> +#ifdef CONFIG_ARM64_SVE
> + REGSET_SVE,
> +#endif
> };
>
> static const struct user_regset aarch64_regsets[] = {
> @@ -768,6 +1010,18 @@ static const struct user_regset aarch64_regsets[] = {
> .get = system_call_get,
> .set = system_call_set,
> },
> +#ifdef CONFIG_ARM64_SVE
> + [REGSET_SVE] = { /* Scalable Vector Extension */
> + .core_note_type = NT_ARM_SVE,
> + .n = DIV_ROUND_UP(SVE_PT_SIZE(SVE_VQ_MAX, SVE_PT_REGS_SVE),
> + SVE_VQ_BYTES),
> + .size = SVE_VQ_BYTES,
> + .align = SVE_VQ_BYTES,
> + .get = sve_get,
> + .set = sve_set,
> + .get_size = sve_get_size,
> + },
> +#endif
> };
>
> static const struct user_regset_view user_aarch64_view = {
> diff --git a/include/uapi/linux/elf.h b/include/uapi/linux/elf.h
> index b5280db..735b8f4 100644
> --- a/include/uapi/linux/elf.h
> +++ b/include/uapi/linux/elf.h
> @@ -416,6 +416,7 @@ typedef struct elf64_shdr {
> #define NT_ARM_HW_BREAK 0x402 /* ARM hardware breakpoint registers */
> #define NT_ARM_HW_WATCH 0x403 /* ARM hardware watchpoint registers */
> #define NT_ARM_SYSTEM_CALL 0x404 /* ARM system call number */
> +#define NT_ARM_SVE 0x405 /* ARM Scalable Vector Extension registers */
> #define NT_METAG_CBUF 0x500 /* Metag catch buffer registers */
> #define NT_METAG_RPIPE 0x501 /* Metag read pipeline state */
> #define NT_METAG_TLS 0x502 /* Metag TLS pointer */
Otherwise:
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 20/28] arm64/sve: Add prctl controls for userspace vector length management
@ 2017-09-14 13:02 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:02 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Andrew Morton
Dave Martin <Dave.Martin@arm.com> writes:
> This patch adds two arm64-specific prctls, to permit userspace to
> control its vector length:
>
> * PR_SVE_SET_VL: set the thread's SVE vector length and vector
> length inheritance mode.
>
> * PR_SVE_GET_VL: get the same information.
>
> Although these calls shadow instruction set features in the SVE
> architecture, these prctls provide additional control: the vector
> length inheritance mode is Linux-specific and nothing to do with
> the architecture, and the architecture does not permit EL0 to set
> its own vector length directly. Both can be used in portable tools
> without requiring the use of SVE instructions.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/include/asm/fpsimd.h | 14 ++++++++++++
> arch/arm64/include/asm/processor.h | 4 ++++
> arch/arm64/kernel/fpsimd.c | 46 ++++++++++++++++++++++++++++++++++++++
> include/uapi/linux/prctl.h | 4 ++++
> kernel/sys.c | 12 ++++++++++
> 5 files changed, 80 insertions(+)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 2723cca..d084968 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -17,6 +17,7 @@
> #define __ASM_FP_H
>
> #include <asm/ptrace.h>
> +#include <asm/errno.h>
>
> #ifndef __ASSEMBLY__
>
> @@ -99,6 +100,9 @@ extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> +extern int sve_set_current_vl(unsigned long arg);
> +extern int sve_get_current_vl(void);
> +
> extern void __init sve_init_vq_map(void);
> extern void sve_update_vq_map(void);
> extern int sve_verify_vq_map(void);
> @@ -114,6 +118,16 @@ static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
> static void __maybe_unused sve_sync_from_fpsimd_zeropad(
> struct task_struct *task) { }
>
> +static int __maybe_unused sve_set_current_vl(unsigned long arg)
> +{
> + return -EINVAL;
> +}
> +
> +static int __maybe_unused sve_get_current_vl(void)
> +{
> + return -EINVAL;
> +}
> +
> static void __maybe_unused sve_init_vq_map(void) { }
> static void __maybe_unused sve_update_vq_map(void) { }
> static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
> index 3faceac..df66452 100644
> --- a/arch/arm64/include/asm/processor.h
> +++ b/arch/arm64/include/asm/processor.h
> @@ -197,4 +197,8 @@ static inline void spin_lock_prefetch(const void *ptr)
> int cpu_enable_pan(void *__unused);
> int cpu_enable_cache_maint_trap(void *__unused);
>
> +/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
> +#define SVE_SET_VL(arg) sve_set_current_vl(arg)
> +#define SVE_GET_VL() sve_get_current_vl()
> +
> #endif /* __ASM_PROCESSOR_H */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 361c019..42e8331 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -27,6 +27,7 @@
> #include <linux/kernel.h>
> #include <linux/init.h>
> #include <linux/percpu.h>
> +#include <linux/prctl.h>
> #include <linux/preempt.h>
> #include <linux/prctl.h>
> #include <linux/ptrace.h>
> @@ -420,6 +421,51 @@ int sve_set_vector_length(struct task_struct *task,
> return 0;
> }
>
> +/*
> + * Encode the current vector length and flags for return.
> + * This is only required for prctl(): ptrace has separate fields
> + */
> +static int sve_prctl_status(void)
> +{
> + int ret = current->thread.sve_vl;
> +
> + if (test_thread_flag(TIF_SVE_VL_INHERIT))
> + ret |= PR_SVE_VL_INHERIT;
> +
> + return ret;
> +}
> +
> +/* PR_SVE_SET_VL */
> +int sve_set_current_vl(unsigned long arg)
> +{
> + unsigned long vl, flags;
> + int ret;
> +
> + vl = arg & PR_SVE_VL_LEN_MASK;
> + flags = arg & ~vl;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + preempt_disable();
> + ret = sve_set_vector_length(current, vl, flags);
> + preempt_enable();
> +
> + if (ret)
> + return ret;
> +
> + return sve_prctl_status();
> +}
> +
> +/* PR_SVE_GET_VL */
> +int sve_get_current_vl(void)
> +{
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + return sve_prctl_status();
> +}
> +
> static unsigned long *sve_alloc_vq_map(void)
> {
> return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
> diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
> index 1b64901..1ef9370 100644
> --- a/include/uapi/linux/prctl.h
> +++ b/include/uapi/linux/prctl.h
> @@ -198,7 +198,11 @@ struct prctl_mm_map {
> # define PR_CAP_AMBIENT_CLEAR_ALL 4
>
> /* arm64 Scalable Vector Extension controls */
> +/* Flag values must be kept in sync with ptrace NT_ARM_SVE interface */
> +#define PR_SVE_SET_VL 48 /* set task vector length */
> # define PR_SVE_SET_VL_ONEXEC (1 << 18) /* defer effect until exec */
> +#define PR_SVE_GET_VL 49 /* get task vector length */
> +/* Bits common to PR_SVE_SET_VL and PR_SVE_GET_VL */
> # define PR_SVE_VL_LEN_MASK 0xffff
> # define PR_SVE_VL_INHERIT (1 << 17) /* inherit across exec */
>
> diff --git a/kernel/sys.c b/kernel/sys.c
> index 2855ee7..f8215a6 100644
> --- a/kernel/sys.c
> +++ b/kernel/sys.c
> @@ -110,6 +110,12 @@
> #ifndef SET_FP_MODE
> # define SET_FP_MODE(a,b) (-EINVAL)
> #endif
> +#ifndef SVE_SET_VL
> +# define SVE_SET_VL(a) (-EINVAL)
> +#endif
> +#ifndef SVE_GET_VL
> +# define SVE_GET_VL() (-EINVAL)
> +#endif
>
> /*
> * this is where the system-wide overflow UID and GID are defined, for
> @@ -2389,6 +2395,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
> case PR_GET_FP_MODE:
> error = GET_FP_MODE(me);
> break;
> + case PR_SVE_SET_VL:
> + error = SVE_SET_VL(arg2);
> + break;
> + case PR_SVE_GET_VL:
> + error = SVE_GET_VL();
> + break;
> default:
> error = -EINVAL;
> break;
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 20/28] arm64/sve: Add prctl controls for userspace vector length management
@ 2017-09-14 13:02 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:02 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Andrew Morton
Dave Martin <Dave.Martin@arm.com> writes:
> This patch adds two arm64-specific prctls, to permit userspace to
> control its vector length:
>
> * PR_SVE_SET_VL: set the thread's SVE vector length and vector
> length inheritance mode.
>
> * PR_SVE_GET_VL: get the same information.
>
> Although these calls shadow instruction set features in the SVE
> architecture, these prctls provide additional control: the vector
> length inheritance mode is Linux-specific and nothing to do with
> the architecture, and the architecture does not permit EL0 to set
> its own vector length directly. Both can be used in portable tools
> without requiring the use of SVE instructions.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/include/asm/fpsimd.h | 14 ++++++++++++
> arch/arm64/include/asm/processor.h | 4 ++++
> arch/arm64/kernel/fpsimd.c | 46 ++++++++++++++++++++++++++++++++++++++
> include/uapi/linux/prctl.h | 4 ++++
> kernel/sys.c | 12 ++++++++++
> 5 files changed, 80 insertions(+)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 2723cca..d084968 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -17,6 +17,7 @@
> #define __ASM_FP_H
>
> #include <asm/ptrace.h>
> +#include <asm/errno.h>
>
> #ifndef __ASSEMBLY__
>
> @@ -99,6 +100,9 @@ extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> +extern int sve_set_current_vl(unsigned long arg);
> +extern int sve_get_current_vl(void);
> +
> extern void __init sve_init_vq_map(void);
> extern void sve_update_vq_map(void);
> extern int sve_verify_vq_map(void);
> @@ -114,6 +118,16 @@ static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
> static void __maybe_unused sve_sync_from_fpsimd_zeropad(
> struct task_struct *task) { }
>
> +static int __maybe_unused sve_set_current_vl(unsigned long arg)
> +{
> + return -EINVAL;
> +}
> +
> +static int __maybe_unused sve_get_current_vl(void)
> +{
> + return -EINVAL;
> +}
> +
> static void __maybe_unused sve_init_vq_map(void) { }
> static void __maybe_unused sve_update_vq_map(void) { }
> static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
> index 3faceac..df66452 100644
> --- a/arch/arm64/include/asm/processor.h
> +++ b/arch/arm64/include/asm/processor.h
> @@ -197,4 +197,8 @@ static inline void spin_lock_prefetch(const void *ptr)
> int cpu_enable_pan(void *__unused);
> int cpu_enable_cache_maint_trap(void *__unused);
>
> +/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
> +#define SVE_SET_VL(arg) sve_set_current_vl(arg)
> +#define SVE_GET_VL() sve_get_current_vl()
> +
> #endif /* __ASM_PROCESSOR_H */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 361c019..42e8331 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -27,6 +27,7 @@
> #include <linux/kernel.h>
> #include <linux/init.h>
> #include <linux/percpu.h>
> +#include <linux/prctl.h>
> #include <linux/preempt.h>
> #include <linux/prctl.h>
> #include <linux/ptrace.h>
> @@ -420,6 +421,51 @@ int sve_set_vector_length(struct task_struct *task,
> return 0;
> }
>
> +/*
> + * Encode the current vector length and flags for return.
> + * This is only required for prctl(): ptrace has separate fields
> + */
> +static int sve_prctl_status(void)
> +{
> + int ret = current->thread.sve_vl;
> +
> + if (test_thread_flag(TIF_SVE_VL_INHERIT))
> + ret |= PR_SVE_VL_INHERIT;
> +
> + return ret;
> +}
> +
> +/* PR_SVE_SET_VL */
> +int sve_set_current_vl(unsigned long arg)
> +{
> + unsigned long vl, flags;
> + int ret;
> +
> + vl = arg & PR_SVE_VL_LEN_MASK;
> + flags = arg & ~vl;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + preempt_disable();
> + ret = sve_set_vector_length(current, vl, flags);
> + preempt_enable();
> +
> + if (ret)
> + return ret;
> +
> + return sve_prctl_status();
> +}
> +
> +/* PR_SVE_GET_VL */
> +int sve_get_current_vl(void)
> +{
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + return sve_prctl_status();
> +}
> +
> static unsigned long *sve_alloc_vq_map(void)
> {
> return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
> diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
> index 1b64901..1ef9370 100644
> --- a/include/uapi/linux/prctl.h
> +++ b/include/uapi/linux/prctl.h
> @@ -198,7 +198,11 @@ struct prctl_mm_map {
> # define PR_CAP_AMBIENT_CLEAR_ALL 4
>
> /* arm64 Scalable Vector Extension controls */
> +/* Flag values must be kept in sync with ptrace NT_ARM_SVE interface */
> +#define PR_SVE_SET_VL 48 /* set task vector length */
> # define PR_SVE_SET_VL_ONEXEC (1 << 18) /* defer effect until exec */
> +#define PR_SVE_GET_VL 49 /* get task vector length */
> +/* Bits common to PR_SVE_SET_VL and PR_SVE_GET_VL */
> # define PR_SVE_VL_LEN_MASK 0xffff
> # define PR_SVE_VL_INHERIT (1 << 17) /* inherit across exec */
>
> diff --git a/kernel/sys.c b/kernel/sys.c
> index 2855ee7..f8215a6 100644
> --- a/kernel/sys.c
> +++ b/kernel/sys.c
> @@ -110,6 +110,12 @@
> #ifndef SET_FP_MODE
> # define SET_FP_MODE(a,b) (-EINVAL)
> #endif
> +#ifndef SVE_SET_VL
> +# define SVE_SET_VL(a) (-EINVAL)
> +#endif
> +#ifndef SVE_GET_VL
> +# define SVE_GET_VL() (-EINVAL)
> +#endif
>
> /*
> * this is where the system-wide overflow UID and GID are defined, for
> @@ -2389,6 +2395,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
> case PR_GET_FP_MODE:
> error = GET_FP_MODE(me);
> break;
> + case PR_SVE_SET_VL:
> + error = SVE_SET_VL(arg2);
> + break;
> + case PR_SVE_GET_VL:
> + error = SVE_GET_VL();
> + break;
> default:
> error = -EINVAL;
> break;
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 20/28] arm64/sve: Add prctl controls for userspace vector length management
@ 2017-09-14 13:02 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:02 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> This patch adds two arm64-specific prctls, to permit userspace to
> control its vector length:
>
> * PR_SVE_SET_VL: set the thread's SVE vector length and vector
> length inheritance mode.
>
> * PR_SVE_GET_VL: get the same information.
>
> Although these calls shadow instruction set features in the SVE
> architecture, these prctls provide additional control: the vector
> length inheritance mode is Linux-specific and nothing to do with
> the architecture, and the architecture does not permit EL0 to set
> its own vector length directly. Both can be used in portable tools
> without requiring the use of SVE instructions.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
> ---
> arch/arm64/include/asm/fpsimd.h | 14 ++++++++++++
> arch/arm64/include/asm/processor.h | 4 ++++
> arch/arm64/kernel/fpsimd.c | 46 ++++++++++++++++++++++++++++++++++++++
> include/uapi/linux/prctl.h | 4 ++++
> kernel/sys.c | 12 ++++++++++
> 5 files changed, 80 insertions(+)
>
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index 2723cca..d084968 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -17,6 +17,7 @@
> #define __ASM_FP_H
>
> #include <asm/ptrace.h>
> +#include <asm/errno.h>
>
> #ifndef __ASSEMBLY__
>
> @@ -99,6 +100,9 @@ extern void sve_sync_from_fpsimd_zeropad(struct task_struct *task);
> extern int sve_set_vector_length(struct task_struct *task,
> unsigned long vl, unsigned long flags);
>
> +extern int sve_set_current_vl(unsigned long arg);
> +extern int sve_get_current_vl(void);
> +
> extern void __init sve_init_vq_map(void);
> extern void sve_update_vq_map(void);
> extern int sve_verify_vq_map(void);
> @@ -114,6 +118,16 @@ static void __maybe_unused sve_sync_to_fpsimd(struct task_struct *task) { }
> static void __maybe_unused sve_sync_from_fpsimd_zeropad(
> struct task_struct *task) { }
>
> +static int __maybe_unused sve_set_current_vl(unsigned long arg)
> +{
> + return -EINVAL;
> +}
> +
> +static int __maybe_unused sve_get_current_vl(void)
> +{
> + return -EINVAL;
> +}
> +
> static void __maybe_unused sve_init_vq_map(void) { }
> static void __maybe_unused sve_update_vq_map(void) { }
> static int __maybe_unused sve_verify_vq_map(void) { return 0; }
> diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
> index 3faceac..df66452 100644
> --- a/arch/arm64/include/asm/processor.h
> +++ b/arch/arm64/include/asm/processor.h
> @@ -197,4 +197,8 @@ static inline void spin_lock_prefetch(const void *ptr)
> int cpu_enable_pan(void *__unused);
> int cpu_enable_cache_maint_trap(void *__unused);
>
> +/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
> +#define SVE_SET_VL(arg) sve_set_current_vl(arg)
> +#define SVE_GET_VL() sve_get_current_vl()
> +
> #endif /* __ASM_PROCESSOR_H */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 361c019..42e8331 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -27,6 +27,7 @@
> #include <linux/kernel.h>
> #include <linux/init.h>
> #include <linux/percpu.h>
> +#include <linux/prctl.h>
> #include <linux/preempt.h>
> #include <linux/prctl.h>
> #include <linux/ptrace.h>
> @@ -420,6 +421,51 @@ int sve_set_vector_length(struct task_struct *task,
> return 0;
> }
>
> +/*
> + * Encode the current vector length and flags for return.
> + * This is only required for prctl(): ptrace has separate fields
> + */
> +static int sve_prctl_status(void)
> +{
> + int ret = current->thread.sve_vl;
> +
> + if (test_thread_flag(TIF_SVE_VL_INHERIT))
> + ret |= PR_SVE_VL_INHERIT;
> +
> + return ret;
> +}
> +
> +/* PR_SVE_SET_VL */
> +int sve_set_current_vl(unsigned long arg)
> +{
> + unsigned long vl, flags;
> + int ret;
> +
> + vl = arg & PR_SVE_VL_LEN_MASK;
> + flags = arg & ~vl;
> +
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + preempt_disable();
> + ret = sve_set_vector_length(current, vl, flags);
> + preempt_enable();
> +
> + if (ret)
> + return ret;
> +
> + return sve_prctl_status();
> +}
> +
> +/* PR_SVE_GET_VL */
> +int sve_get_current_vl(void)
> +{
> + if (!system_supports_sve())
> + return -EINVAL;
> +
> + return sve_prctl_status();
> +}
> +
> static unsigned long *sve_alloc_vq_map(void)
> {
> return kzalloc(BITS_TO_LONGS(SVE_VQ_MAX) * sizeof(unsigned long),
> diff --git a/include/uapi/linux/prctl.h b/include/uapi/linux/prctl.h
> index 1b64901..1ef9370 100644
> --- a/include/uapi/linux/prctl.h
> +++ b/include/uapi/linux/prctl.h
> @@ -198,7 +198,11 @@ struct prctl_mm_map {
> # define PR_CAP_AMBIENT_CLEAR_ALL 4
>
> /* arm64 Scalable Vector Extension controls */
> +/* Flag values must be kept in sync with ptrace NT_ARM_SVE interface */
> +#define PR_SVE_SET_VL 48 /* set task vector length */
> # define PR_SVE_SET_VL_ONEXEC (1 << 18) /* defer effect until exec */
> +#define PR_SVE_GET_VL 49 /* get task vector length */
> +/* Bits common to PR_SVE_SET_VL and PR_SVE_GET_VL */
> # define PR_SVE_VL_LEN_MASK 0xffff
> # define PR_SVE_VL_INHERIT (1 << 17) /* inherit across exec */
>
> diff --git a/kernel/sys.c b/kernel/sys.c
> index 2855ee7..f8215a6 100644
> --- a/kernel/sys.c
> +++ b/kernel/sys.c
> @@ -110,6 +110,12 @@
> #ifndef SET_FP_MODE
> # define SET_FP_MODE(a,b) (-EINVAL)
> #endif
> +#ifndef SVE_SET_VL
> +# define SVE_SET_VL(a) (-EINVAL)
> +#endif
> +#ifndef SVE_GET_VL
> +# define SVE_GET_VL() (-EINVAL)
> +#endif
>
> /*
> * this is where the system-wide overflow UID and GID are defined, for
> @@ -2389,6 +2395,12 @@ SYSCALL_DEFINE5(prctl, int, option, unsigned long, arg2, unsigned long, arg3,
> case PR_GET_FP_MODE:
> error = GET_FP_MODE(me);
> break;
> + case PR_SVE_SET_VL:
> + error = SVE_SET_VL(arg2);
> + break;
> + case PR_SVE_GET_VL:
> + error = SVE_GET_VL();
> + break;
> default:
> error = -EINVAL;
> break;
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 21/28] arm64/sve: Add sysctl to set the default vector length for new processes
@ 2017-09-14 13:05 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:05 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> Because of the effect of SVE on the size of the signal frame, the
> default vector length used for new processes involves a tradeoff
> between performance of SVE-enabled software on the one hand, and
> reliability of non-SVE-aware software on the other hand.
>
> For this reason, the best choice depends on the repertoire of
> userspace software in use and is thus best left up to distro
> maintainers, sysadmins and developers.
>
> If CONFIG_SYSCTL is enabled, this patch exposes the default vector
> length in /proc/sys/abi/sve_default_vector_length, where boot
> scripts or the adventurous can poke it.
>
> In common with other arm64 ABI sysctls, this control is currently
> global: setting it requires CAP_SYS_ADMIN in the root user
> namespace, but the value set is effective for subsequent execs in
> all namespaces. The control only affects _new_ processes, however:
> changing it does not affect the vector length of any existing
> process.
>
> The intended usage model is that if userspace is known to be fully
> SVE-tolerant (or a developer is curious to find out) then init
> scripts can crank this up during startup.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Changes requested by Alex Bennée:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
> ---
> arch/arm64/kernel/fpsimd.c | 62 +++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 61 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 42e8331..b430ee0 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -34,6 +34,7 @@
> #include <linux/sched/signal.h>
> #include <linux/signal.h>
> #include <linux/slab.h>
> +#include <linux/sysctl.h>
>
> #include <asm/fpsimd.h>
> #include <asm/cputype.h>
> @@ -244,6 +245,65 @@ static unsigned int find_supported_vector_length(unsigned int vl)
> return sve_vl_from_vq(bit_to_vq(bit));
> }
>
> +#ifdef CONFIG_SYSCTL
> +
> +static int sve_proc_do_default_vl(struct ctl_table *table, int write,
> + void __user *buffer, size_t *lenp,
> + loff_t *ppos)
> +{
> + int ret;
> + int vl = sve_default_vl;
> + struct ctl_table tmp_table = {
> + .data = &vl,
> + .maxlen = sizeof(vl),
> + };
> +
> + ret = proc_dointvec(&tmp_table, write, buffer, lenp, ppos);
> + if (ret || !write)
> + return ret;
> +
> + /* Writing -1 has the special meaning "set to max": */
> + if (vl == -1) {
> + /* Fail safe if sve_max_vl wasn't initialised */
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + vl = SVE_VL_MIN;
> + else
> + vl = sve_max_vl;
> +
> + goto chosen;
> + }
> +
> + if (!sve_vl_valid(vl))
> + return -EINVAL;
> +
> + vl = find_supported_vector_length(vl);
> +chosen:
> + sve_default_vl = vl;
> + return 0;
> +}
> +
> +static struct ctl_table sve_default_vl_table[] = {
> + {
> + .procname = "sve_default_vector_length",
> + .mode = 0644,
> + .proc_handler = sve_proc_do_default_vl,
> + },
> + { }
> +};
> +
> +static int __init sve_sysctl_init(void)
> +{
> + if (system_supports_sve())
> + if (!register_sysctl("abi", sve_default_vl_table))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +#else /* ! CONFIG_SYSCTL */
> +static int __init sve_sysctl_init(void) { return 0; }
> +#endif /* ! CONFIG_SYSCTL */
> +
> #define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
> (SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
>
> @@ -1030,6 +1090,6 @@ static int __init fpsimd_init(void)
> if (!(elf_hwcap & HWCAP_ASIMD))
> pr_notice("Advanced SIMD is not implemented\n");
>
> - return 0;
> + return sve_sysctl_init();
> }
> late_initcall(fpsimd_init);
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 21/28] arm64/sve: Add sysctl to set the default vector length for new processes
@ 2017-09-14 13:05 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:05 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch
Dave Martin <Dave.Martin@arm.com> writes:
> Because of the effect of SVE on the size of the signal frame, the
> default vector length used for new processes involves a tradeoff
> between performance of SVE-enabled software on the one hand, and
> reliability of non-SVE-aware software on the other hand.
>
> For this reason, the best choice depends on the repertoire of
> userspace software in use and is thus best left up to distro
> maintainers, sysadmins and developers.
>
> If CONFIG_SYSCTL is enabled, this patch exposes the default vector
> length in /proc/sys/abi/sve_default_vector_length, where boot
> scripts or the adventurous can poke it.
>
> In common with other arm64 ABI sysctls, this control is currently
> global: setting it requires CAP_SYS_ADMIN in the root user
> namespace, but the value set is effective for subsequent execs in
> all namespaces. The control only affects _new_ processes, however:
> changing it does not affect the vector length of any existing
> process.
>
> The intended usage model is that if userspace is known to be fully
> SVE-tolerant (or a developer is curious to find out) then init
> scripts can crank this up during startup.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Changes requested by Alex Bennée:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
> ---
> arch/arm64/kernel/fpsimd.c | 62 +++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 61 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 42e8331..b430ee0 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -34,6 +34,7 @@
> #include <linux/sched/signal.h>
> #include <linux/signal.h>
> #include <linux/slab.h>
> +#include <linux/sysctl.h>
>
> #include <asm/fpsimd.h>
> #include <asm/cputype.h>
> @@ -244,6 +245,65 @@ static unsigned int find_supported_vector_length(unsigned int vl)
> return sve_vl_from_vq(bit_to_vq(bit));
> }
>
> +#ifdef CONFIG_SYSCTL
> +
> +static int sve_proc_do_default_vl(struct ctl_table *table, int write,
> + void __user *buffer, size_t *lenp,
> + loff_t *ppos)
> +{
> + int ret;
> + int vl = sve_default_vl;
> + struct ctl_table tmp_table = {
> + .data = &vl,
> + .maxlen = sizeof(vl),
> + };
> +
> + ret = proc_dointvec(&tmp_table, write, buffer, lenp, ppos);
> + if (ret || !write)
> + return ret;
> +
> + /* Writing -1 has the special meaning "set to max": */
> + if (vl == -1) {
> + /* Fail safe if sve_max_vl wasn't initialised */
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + vl = SVE_VL_MIN;
> + else
> + vl = sve_max_vl;
> +
> + goto chosen;
> + }
> +
> + if (!sve_vl_valid(vl))
> + return -EINVAL;
> +
> + vl = find_supported_vector_length(vl);
> +chosen:
> + sve_default_vl = vl;
> + return 0;
> +}
> +
> +static struct ctl_table sve_default_vl_table[] = {
> + {
> + .procname = "sve_default_vector_length",
> + .mode = 0644,
> + .proc_handler = sve_proc_do_default_vl,
> + },
> + { }
> +};
> +
> +static int __init sve_sysctl_init(void)
> +{
> + if (system_supports_sve())
> + if (!register_sysctl("abi", sve_default_vl_table))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +#else /* ! CONFIG_SYSCTL */
> +static int __init sve_sysctl_init(void) { return 0; }
> +#endif /* ! CONFIG_SYSCTL */
> +
> #define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
> (SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
>
> @@ -1030,6 +1090,6 @@ static int __init fpsimd_init(void)
> if (!(elf_hwcap & HWCAP_ASIMD))
> pr_notice("Advanced SIMD is not implemented\n");
>
> - return 0;
> + return sve_sysctl_init();
> }
> late_initcall(fpsimd_init);
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 21/28] arm64/sve: Add sysctl to set the default vector length for new processes
@ 2017-09-14 13:05 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:05 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> Because of the effect of SVE on the size of the signal frame, the
> default vector length used for new processes involves a tradeoff
> between performance of SVE-enabled software on the one hand, and
> reliability of non-SVE-aware software on the other hand.
>
> For this reason, the best choice depends on the repertoire of
> userspace software in use and is thus best left up to distro
> maintainers, sysadmins and developers.
>
> If CONFIG_SYSCTL is enabled, this patch exposes the default vector
> length in /proc/sys/abi/sve_default_vector_length, where boot
> scripts or the adventurous can poke it.
>
> In common with other arm64 ABI sysctls, this control is currently
> global: setting it requires CAP_SYS_ADMIN in the root user
> namespace, but the value set is effective for subsequent execs in
> all namespaces. The control only affects _new_ processes, however:
> changing it does not affect the vector length of any existing
> process.
>
> The intended usage model is that if userspace is known to be fully
> SVE-tolerant (or a developer is curious to find out) then init
> scripts can crank this up during startup.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Alex Benn?e <alex.bennee@linaro.org>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Changes requested by Alex Benn?e:
>
> * Thin out BUG_ON()s:
> Redundant BUG_ON()s and ones that just check invariants are removed.
> Important sanity-checks are migrated to WARN_ON()s, with some
> minimal best-effort patch-up code.
> ---
> arch/arm64/kernel/fpsimd.c | 62 +++++++++++++++++++++++++++++++++++++++++++++-
> 1 file changed, 61 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index 42e8331..b430ee0 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -34,6 +34,7 @@
> #include <linux/sched/signal.h>
> #include <linux/signal.h>
> #include <linux/slab.h>
> +#include <linux/sysctl.h>
>
> #include <asm/fpsimd.h>
> #include <asm/cputype.h>
> @@ -244,6 +245,65 @@ static unsigned int find_supported_vector_length(unsigned int vl)
> return sve_vl_from_vq(bit_to_vq(bit));
> }
>
> +#ifdef CONFIG_SYSCTL
> +
> +static int sve_proc_do_default_vl(struct ctl_table *table, int write,
> + void __user *buffer, size_t *lenp,
> + loff_t *ppos)
> +{
> + int ret;
> + int vl = sve_default_vl;
> + struct ctl_table tmp_table = {
> + .data = &vl,
> + .maxlen = sizeof(vl),
> + };
> +
> + ret = proc_dointvec(&tmp_table, write, buffer, lenp, ppos);
> + if (ret || !write)
> + return ret;
> +
> + /* Writing -1 has the special meaning "set to max": */
> + if (vl == -1) {
> + /* Fail safe if sve_max_vl wasn't initialised */
> + if (WARN_ON(!sve_vl_valid(sve_max_vl)))
> + vl = SVE_VL_MIN;
> + else
> + vl = sve_max_vl;
> +
> + goto chosen;
> + }
> +
> + if (!sve_vl_valid(vl))
> + return -EINVAL;
> +
> + vl = find_supported_vector_length(vl);
> +chosen:
> + sve_default_vl = vl;
> + return 0;
> +}
> +
> +static struct ctl_table sve_default_vl_table[] = {
> + {
> + .procname = "sve_default_vector_length",
> + .mode = 0644,
> + .proc_handler = sve_proc_do_default_vl,
> + },
> + { }
> +};
> +
> +static int __init sve_sysctl_init(void)
> +{
> + if (system_supports_sve())
> + if (!register_sysctl("abi", sve_default_vl_table))
> + return -EINVAL;
> +
> + return 0;
> +}
> +
> +#else /* ! CONFIG_SYSCTL */
> +static int __init sve_sysctl_init(void) { return 0; }
> +#endif /* ! CONFIG_SYSCTL */
> +
> #define ZREG(sve_state, vq, n) ((char *)(sve_state) + \
> (SVE_SIG_ZREG_OFFSET(vq, n) - SVE_SIG_REGS_OFFSET))
>
> @@ -1030,6 +1090,6 @@ static int __init fpsimd_init(void)
> if (!(elf_hwcap & HWCAP_ASIMD))
> pr_notice("Advanced SIMD is not implemented\n");
>
> - return 0;
> + return sve_sysctl_init();
> }
> late_initcall(fpsimd_init);
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 22/28] arm64/sve: KVM: Prevent guests from using SVE
@ 2017-09-14 13:28 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:28 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> Until KVM has full SVE support, guests must not be allowed to
> execute SVE instructions.
>
> This patch enables the necessary traps, and also ensures that the
> traps are disabled again on exit from the guest so that the host
> can still use SVE if it wants to.
>
> This patch introduces another instance of
> __this_cpu_write(fpsimd_last_state, NULL), so this flush operation
> is abstracted out as a separate helper fpsimd_flush_cpu_state().
> Other instances are ported appropriately.
>
> As a side effect of this refactoring, a this_cpu_write() in
> fpsimd_cpu_pm_notifier() is changed to __this_cpu_write(). This
> should be fine, since cpu_pm_enter() is supposed to be called only
> with interrupts disabled.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Avoid the verbose arithmetic for CPTR_EL2_DEFAULT, and just
> describe it in terms of the set of bits known to be RES1 in
> CPTR_EL2.
>
> Other:
>
> * Fixup to drop task SVE state cached in the CPU registers across
> guest entry/exit.
>
> Without this, we may enter an EL0 process with wrong data in the
> extended SVE bits and/or wrong trap configuration.
>
> This is not a problem for the FPSIMD part of the state because KVM
> explicitly restores the host FPSIMD state on guest exit; but this
> restore is sufficient to corrupt the extra SVE bits even if nothing
> else does.
>
> * The fpsimd_flush_cpu_state() function, which was supposed to abstract
> the underlying flush operation, wasn't used. [sparse]
>
> This patch is now ported to use it. Other users of the same idiom are
> ported too (which was the original intention).
>
> fpsimd_flush_cpu_state() is marked inline, since all users are
> ifdef'd and the function may be unused. Plus, it's trivially
> suitable for inlining.
> ---
> arch/arm/include/asm/kvm_host.h | 3 +++
> arch/arm64/include/asm/fpsimd.h | 1 +
> arch/arm64/include/asm/kvm_arm.h | 4 +++-
> arch/arm64/include/asm/kvm_host.h | 11 +++++++++++
> arch/arm64/kernel/fpsimd.c | 31 +++++++++++++++++++++++++++++--
> arch/arm64/kvm/hyp/switch.c | 6 +++---
> virt/kvm/arm/arm.c | 3 +++
> 7 files changed, 53 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 127e2dd..fa4a442 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -299,4 +299,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> struct kvm_device_attr *attr);
>
> +/* All host FP/SIMD state is restored on guest exit, so nothing to save: */
> +static inline void kvm_fpsimd_flush_cpu_state(void) {}
> +
> #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index d084968..5605fc1 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -74,6 +74,7 @@ extern void fpsimd_restore_current_state(void);
> extern void fpsimd_update_current_state(struct fpsimd_state *state);
>
> extern void fpsimd_flush_task_state(struct task_struct *target);
> +extern void sve_flush_cpu_state(void);
>
> /* Maximum VL that SVE VL-agnostic software can transparently support */
> #define SVE_VL_ARCH_MAX 0x100
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index dbf0537..7f069ff 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -186,7 +186,8 @@
> #define CPTR_EL2_TTA (1 << 20)
> #define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
> #define CPTR_EL2_TZ (1 << 8)
> -#define CPTR_EL2_DEFAULT 0x000033ff
> +#define CPTR_EL2_RES1 0x000032ff /* known RES1 bits in CPTR_EL2 */
> +#define CPTR_EL2_DEFAULT CPTR_EL2_RES1
>
> /* Hyp Debug Configuration Register bits */
> #define MDCR_EL2_TPMS (1 << 14)
> @@ -237,5 +238,6 @@
>
> #define CPACR_EL1_FPEN (3 << 20)
> #define CPACR_EL1_TTA (1 << 28)
> +#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
>
> #endif /* __ARM64_KVM_ARM_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index d686300..05d8373 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -25,6 +25,7 @@
> #include <linux/types.h>
> #include <linux/kvm_types.h>
> #include <asm/cpufeature.h>
> +#include <asm/fpsimd.h>
> #include <asm/kvm.h>
> #include <asm/kvm_asm.h>
> #include <asm/kvm_mmio.h>
> @@ -390,4 +391,14 @@ static inline void __cpu_init_stage2(void)
> "PARange is %d bits, unsupported configuration!", parange);
> }
>
> +/*
> + * All host FP/SIMD state is restored on guest exit, so nothing needs
> + * doing here except in the SVE case:
> +*/
> +static inline void kvm_fpsimd_flush_cpu_state(void)
> +{
> + if (system_supports_sve())
> + sve_flush_cpu_state();
> +}
> +
> #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index b430ee0..7837ced 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -875,6 +875,33 @@ void fpsimd_flush_task_state(struct task_struct *t)
> t->thread.fpsimd_state.cpu = NR_CPUS;
> }
>
> +static inline void fpsimd_flush_cpu_state(void)
> +{
> + __this_cpu_write(fpsimd_last_state, NULL);
> +}
> +
> +/*
> + * Invalidate any task SVE state currently held in this CPU's regs.
> + *
> + * This is used to prevent the kernel from trying to reuse SVE register data
> + * that is detroyed by KVM guest enter/exit. This function should go away when
> + * KVM SVE support is implemented. Don't use it for anything else.
> + */
> +#ifdef CONFIG_ARM64_SVE
> +void sve_flush_cpu_state(void)
> +{
> + struct fpsimd_state *const fpstate = __this_cpu_read(fpsimd_last_state);
> + struct task_struct *tsk;
> +
> + if (!fpstate)
> + return;
> +
> + tsk = container_of(fpstate, struct task_struct, thread.fpsimd_state);
> + if (test_tsk_thread_flag(tsk, TIF_SVE))
> + fpsimd_flush_cpu_state();
> +}
> +#endif /* CONFIG_ARM64_SVE */
> +
> #ifdef CONFIG_KERNEL_MODE_NEON
>
> DEFINE_PER_CPU(bool, kernel_neon_busy);
> @@ -915,7 +942,7 @@ void kernel_neon_begin(void)
> }
>
> /* Invalidate any task state remaining in the fpsimd regs: */
> - __this_cpu_write(fpsimd_last_state, NULL);
> + fpsimd_flush_cpu_state();
>
> preempt_disable();
>
> @@ -1032,7 +1059,7 @@ static int fpsimd_cpu_pm_notifier(struct notifier_block *self,
> case CPU_PM_ENTER:
> if (current->mm)
> task_fpsimd_save();
> - this_cpu_write(fpsimd_last_state, NULL);
> + fpsimd_flush_cpu_state();
> break;
> case CPU_PM_EXIT:
> if (current->mm)
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 35a90b8..951f3eb 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -48,7 +48,7 @@ static void __hyp_text __activate_traps_vhe(void)
>
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> - val &= ~CPACR_EL1_FPEN;
> + val &= ~(CPACR_EL1_FPEN | CPACR_EL1_ZEN);
> write_sysreg(val, cpacr_el1);
>
> write_sysreg(__kvm_hyp_vector, vbar_el1);
> @@ -59,7 +59,7 @@ static void __hyp_text __activate_traps_nvhe(void)
> u64 val;
>
> val = CPTR_EL2_DEFAULT;
> - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> + val |= CPTR_EL2_TTA | CPTR_EL2_TFP | CPTR_EL2_TZ;
> write_sysreg(val, cptr_el2);
> }
>
> @@ -117,7 +117,7 @@ static void __hyp_text __deactivate_traps_vhe(void)
>
> write_sysreg(mdcr_el2, mdcr_el2);
> write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
> - write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
> + write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1);
> write_sysreg(vectors, vbar_el1);
> }
>
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index a39a1e1..af9f5da 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -647,6 +647,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> */
> preempt_disable();
>
> + /* Flush FP/SIMD state that can't survive guest entry/exit */
> + kvm_fpsimd_flush_cpu_state();
> +
> kvm_pmu_flush_hwstate(vcpu);
>
> kvm_timer_flush_hwstate(vcpu);
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 22/28] arm64/sve: KVM: Prevent guests from using SVE
@ 2017-09-14 13:28 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:28 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> Until KVM has full SVE support, guests must not be allowed to
> execute SVE instructions.
>
> This patch enables the necessary traps, and also ensures that the
> traps are disabled again on exit from the guest so that the host
> can still use SVE if it wants to.
>
> This patch introduces another instance of
> __this_cpu_write(fpsimd_last_state, NULL), so this flush operation
> is abstracted out as a separate helper fpsimd_flush_cpu_state().
> Other instances are ported appropriately.
>
> As a side effect of this refactoring, a this_cpu_write() in
> fpsimd_cpu_pm_notifier() is changed to __this_cpu_write(). This
> should be fine, since cpu_pm_enter() is supposed to be called only
> with interrupts disabled.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Avoid the verbose arithmetic for CPTR_EL2_DEFAULT, and just
> describe it in terms of the set of bits known to be RES1 in
> CPTR_EL2.
>
> Other:
>
> * Fixup to drop task SVE state cached in the CPU registers across
> guest entry/exit.
>
> Without this, we may enter an EL0 process with wrong data in the
> extended SVE bits and/or wrong trap configuration.
>
> This is not a problem for the FPSIMD part of the state because KVM
> explicitly restores the host FPSIMD state on guest exit; but this
> restore is sufficient to corrupt the extra SVE bits even if nothing
> else does.
>
> * The fpsimd_flush_cpu_state() function, which was supposed to abstract
> the underlying flush operation, wasn't used. [sparse]
>
> This patch is now ported to use it. Other users of the same idiom are
> ported too (which was the original intention).
>
> fpsimd_flush_cpu_state() is marked inline, since all users are
> ifdef'd and the function may be unused. Plus, it's trivially
> suitable for inlining.
> ---
> arch/arm/include/asm/kvm_host.h | 3 +++
> arch/arm64/include/asm/fpsimd.h | 1 +
> arch/arm64/include/asm/kvm_arm.h | 4 +++-
> arch/arm64/include/asm/kvm_host.h | 11 +++++++++++
> arch/arm64/kernel/fpsimd.c | 31 +++++++++++++++++++++++++++++--
> arch/arm64/kvm/hyp/switch.c | 6 +++---
> virt/kvm/arm/arm.c | 3 +++
> 7 files changed, 53 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 127e2dd..fa4a442 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -299,4 +299,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> struct kvm_device_attr *attr);
>
> +/* All host FP/SIMD state is restored on guest exit, so nothing to save: */
> +static inline void kvm_fpsimd_flush_cpu_state(void) {}
> +
> #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index d084968..5605fc1 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -74,6 +74,7 @@ extern void fpsimd_restore_current_state(void);
> extern void fpsimd_update_current_state(struct fpsimd_state *state);
>
> extern void fpsimd_flush_task_state(struct task_struct *target);
> +extern void sve_flush_cpu_state(void);
>
> /* Maximum VL that SVE VL-agnostic software can transparently support */
> #define SVE_VL_ARCH_MAX 0x100
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index dbf0537..7f069ff 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -186,7 +186,8 @@
> #define CPTR_EL2_TTA (1 << 20)
> #define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
> #define CPTR_EL2_TZ (1 << 8)
> -#define CPTR_EL2_DEFAULT 0x000033ff
> +#define CPTR_EL2_RES1 0x000032ff /* known RES1 bits in CPTR_EL2 */
> +#define CPTR_EL2_DEFAULT CPTR_EL2_RES1
>
> /* Hyp Debug Configuration Register bits */
> #define MDCR_EL2_TPMS (1 << 14)
> @@ -237,5 +238,6 @@
>
> #define CPACR_EL1_FPEN (3 << 20)
> #define CPACR_EL1_TTA (1 << 28)
> +#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
>
> #endif /* __ARM64_KVM_ARM_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index d686300..05d8373 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -25,6 +25,7 @@
> #include <linux/types.h>
> #include <linux/kvm_types.h>
> #include <asm/cpufeature.h>
> +#include <asm/fpsimd.h>
> #include <asm/kvm.h>
> #include <asm/kvm_asm.h>
> #include <asm/kvm_mmio.h>
> @@ -390,4 +391,14 @@ static inline void __cpu_init_stage2(void)
> "PARange is %d bits, unsupported configuration!", parange);
> }
>
> +/*
> + * All host FP/SIMD state is restored on guest exit, so nothing needs
> + * doing here except in the SVE case:
> +*/
> +static inline void kvm_fpsimd_flush_cpu_state(void)
> +{
> + if (system_supports_sve())
> + sve_flush_cpu_state();
> +}
> +
> #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index b430ee0..7837ced 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -875,6 +875,33 @@ void fpsimd_flush_task_state(struct task_struct *t)
> t->thread.fpsimd_state.cpu = NR_CPUS;
> }
>
> +static inline void fpsimd_flush_cpu_state(void)
> +{
> + __this_cpu_write(fpsimd_last_state, NULL);
> +}
> +
> +/*
> + * Invalidate any task SVE state currently held in this CPU's regs.
> + *
> + * This is used to prevent the kernel from trying to reuse SVE register data
> + * that is detroyed by KVM guest enter/exit. This function should go away when
> + * KVM SVE support is implemented. Don't use it for anything else.
> + */
> +#ifdef CONFIG_ARM64_SVE
> +void sve_flush_cpu_state(void)
> +{
> + struct fpsimd_state *const fpstate = __this_cpu_read(fpsimd_last_state);
> + struct task_struct *tsk;
> +
> + if (!fpstate)
> + return;
> +
> + tsk = container_of(fpstate, struct task_struct, thread.fpsimd_state);
> + if (test_tsk_thread_flag(tsk, TIF_SVE))
> + fpsimd_flush_cpu_state();
> +}
> +#endif /* CONFIG_ARM64_SVE */
> +
> #ifdef CONFIG_KERNEL_MODE_NEON
>
> DEFINE_PER_CPU(bool, kernel_neon_busy);
> @@ -915,7 +942,7 @@ void kernel_neon_begin(void)
> }
>
> /* Invalidate any task state remaining in the fpsimd regs: */
> - __this_cpu_write(fpsimd_last_state, NULL);
> + fpsimd_flush_cpu_state();
>
> preempt_disable();
>
> @@ -1032,7 +1059,7 @@ static int fpsimd_cpu_pm_notifier(struct notifier_block *self,
> case CPU_PM_ENTER:
> if (current->mm)
> task_fpsimd_save();
> - this_cpu_write(fpsimd_last_state, NULL);
> + fpsimd_flush_cpu_state();
> break;
> case CPU_PM_EXIT:
> if (current->mm)
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 35a90b8..951f3eb 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -48,7 +48,7 @@ static void __hyp_text __activate_traps_vhe(void)
>
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> - val &= ~CPACR_EL1_FPEN;
> + val &= ~(CPACR_EL1_FPEN | CPACR_EL1_ZEN);
> write_sysreg(val, cpacr_el1);
>
> write_sysreg(__kvm_hyp_vector, vbar_el1);
> @@ -59,7 +59,7 @@ static void __hyp_text __activate_traps_nvhe(void)
> u64 val;
>
> val = CPTR_EL2_DEFAULT;
> - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> + val |= CPTR_EL2_TTA | CPTR_EL2_TFP | CPTR_EL2_TZ;
> write_sysreg(val, cptr_el2);
> }
>
> @@ -117,7 +117,7 @@ static void __hyp_text __deactivate_traps_vhe(void)
>
> write_sysreg(mdcr_el2, mdcr_el2);
> write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
> - write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
> + write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1);
> write_sysreg(vectors, vbar_el1);
> }
>
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index a39a1e1..af9f5da 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -647,6 +647,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> */
> preempt_disable();
>
> + /* Flush FP/SIMD state that can't survive guest entry/exit */
> + kvm_fpsimd_flush_cpu_state();
> +
> kvm_pmu_flush_hwstate(vcpu);
>
> kvm_timer_flush_hwstate(vcpu);
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 22/28] arm64/sve: KVM: Prevent guests from using SVE
@ 2017-09-14 13:28 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:28 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> Until KVM has full SVE support, guests must not be allowed to
> execute SVE instructions.
>
> This patch enables the necessary traps, and also ensures that the
> traps are disabled again on exit from the guest so that the host
> can still use SVE if it wants to.
>
> This patch introduces another instance of
> __this_cpu_write(fpsimd_last_state, NULL), so this flush operation
> is abstracted out as a separate helper fpsimd_flush_cpu_state().
> Other instances are ported appropriately.
>
> As a side effect of this refactoring, a this_cpu_write() in
> fpsimd_cpu_pm_notifier() is changed to __this_cpu_write(). This
> should be fine, since cpu_pm_enter() is supposed to be called only
> with interrupts disabled.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Avoid the verbose arithmetic for CPTR_EL2_DEFAULT, and just
> describe it in terms of the set of bits known to be RES1 in
> CPTR_EL2.
>
> Other:
>
> * Fixup to drop task SVE state cached in the CPU registers across
> guest entry/exit.
>
> Without this, we may enter an EL0 process with wrong data in the
> extended SVE bits and/or wrong trap configuration.
>
> This is not a problem for the FPSIMD part of the state because KVM
> explicitly restores the host FPSIMD state on guest exit; but this
> restore is sufficient to corrupt the extra SVE bits even if nothing
> else does.
>
> * The fpsimd_flush_cpu_state() function, which was supposed to abstract
> the underlying flush operation, wasn't used. [sparse]
>
> This patch is now ported to use it. Other users of the same idiom are
> ported too (which was the original intention).
>
> fpsimd_flush_cpu_state() is marked inline, since all users are
> ifdef'd and the function may be unused. Plus, it's trivially
> suitable for inlining.
> ---
> arch/arm/include/asm/kvm_host.h | 3 +++
> arch/arm64/include/asm/fpsimd.h | 1 +
> arch/arm64/include/asm/kvm_arm.h | 4 +++-
> arch/arm64/include/asm/kvm_host.h | 11 +++++++++++
> arch/arm64/kernel/fpsimd.c | 31 +++++++++++++++++++++++++++++--
> arch/arm64/kvm/hyp/switch.c | 6 +++---
> virt/kvm/arm/arm.c | 3 +++
> 7 files changed, 53 insertions(+), 6 deletions(-)
>
> diff --git a/arch/arm/include/asm/kvm_host.h b/arch/arm/include/asm/kvm_host.h
> index 127e2dd..fa4a442 100644
> --- a/arch/arm/include/asm/kvm_host.h
> +++ b/arch/arm/include/asm/kvm_host.h
> @@ -299,4 +299,7 @@ int kvm_arm_vcpu_arch_get_attr(struct kvm_vcpu *vcpu,
> int kvm_arm_vcpu_arch_has_attr(struct kvm_vcpu *vcpu,
> struct kvm_device_attr *attr);
>
> +/* All host FP/SIMD state is restored on guest exit, so nothing to save: */
> +static inline void kvm_fpsimd_flush_cpu_state(void) {}
> +
> #endif /* __ARM_KVM_HOST_H__ */
> diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
> index d084968..5605fc1 100644
> --- a/arch/arm64/include/asm/fpsimd.h
> +++ b/arch/arm64/include/asm/fpsimd.h
> @@ -74,6 +74,7 @@ extern void fpsimd_restore_current_state(void);
> extern void fpsimd_update_current_state(struct fpsimd_state *state);
>
> extern void fpsimd_flush_task_state(struct task_struct *target);
> +extern void sve_flush_cpu_state(void);
>
> /* Maximum VL that SVE VL-agnostic software can transparently support */
> #define SVE_VL_ARCH_MAX 0x100
> diff --git a/arch/arm64/include/asm/kvm_arm.h b/arch/arm64/include/asm/kvm_arm.h
> index dbf0537..7f069ff 100644
> --- a/arch/arm64/include/asm/kvm_arm.h
> +++ b/arch/arm64/include/asm/kvm_arm.h
> @@ -186,7 +186,8 @@
> #define CPTR_EL2_TTA (1 << 20)
> #define CPTR_EL2_TFP (1 << CPTR_EL2_TFP_SHIFT)
> #define CPTR_EL2_TZ (1 << 8)
> -#define CPTR_EL2_DEFAULT 0x000033ff
> +#define CPTR_EL2_RES1 0x000032ff /* known RES1 bits in CPTR_EL2 */
> +#define CPTR_EL2_DEFAULT CPTR_EL2_RES1
>
> /* Hyp Debug Configuration Register bits */
> #define MDCR_EL2_TPMS (1 << 14)
> @@ -237,5 +238,6 @@
>
> #define CPACR_EL1_FPEN (3 << 20)
> #define CPACR_EL1_TTA (1 << 28)
> +#define CPACR_EL1_DEFAULT (CPACR_EL1_FPEN | CPACR_EL1_ZEN_EL1EN)
>
> #endif /* __ARM64_KVM_ARM_H__ */
> diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h
> index d686300..05d8373 100644
> --- a/arch/arm64/include/asm/kvm_host.h
> +++ b/arch/arm64/include/asm/kvm_host.h
> @@ -25,6 +25,7 @@
> #include <linux/types.h>
> #include <linux/kvm_types.h>
> #include <asm/cpufeature.h>
> +#include <asm/fpsimd.h>
> #include <asm/kvm.h>
> #include <asm/kvm_asm.h>
> #include <asm/kvm_mmio.h>
> @@ -390,4 +391,14 @@ static inline void __cpu_init_stage2(void)
> "PARange is %d bits, unsupported configuration!", parange);
> }
>
> +/*
> + * All host FP/SIMD state is restored on guest exit, so nothing needs
> + * doing here except in the SVE case:
> +*/
> +static inline void kvm_fpsimd_flush_cpu_state(void)
> +{
> + if (system_supports_sve())
> + sve_flush_cpu_state();
> +}
> +
> #endif /* __ARM64_KVM_HOST_H__ */
> diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
> index b430ee0..7837ced 100644
> --- a/arch/arm64/kernel/fpsimd.c
> +++ b/arch/arm64/kernel/fpsimd.c
> @@ -875,6 +875,33 @@ void fpsimd_flush_task_state(struct task_struct *t)
> t->thread.fpsimd_state.cpu = NR_CPUS;
> }
>
> +static inline void fpsimd_flush_cpu_state(void)
> +{
> + __this_cpu_write(fpsimd_last_state, NULL);
> +}
> +
> +/*
> + * Invalidate any task SVE state currently held in this CPU's regs.
> + *
> + * This is used to prevent the kernel from trying to reuse SVE register data
> + * that is detroyed by KVM guest enter/exit. This function should go away when
> + * KVM SVE support is implemented. Don't use it for anything else.
> + */
> +#ifdef CONFIG_ARM64_SVE
> +void sve_flush_cpu_state(void)
> +{
> + struct fpsimd_state *const fpstate = __this_cpu_read(fpsimd_last_state);
> + struct task_struct *tsk;
> +
> + if (!fpstate)
> + return;
> +
> + tsk = container_of(fpstate, struct task_struct, thread.fpsimd_state);
> + if (test_tsk_thread_flag(tsk, TIF_SVE))
> + fpsimd_flush_cpu_state();
> +}
> +#endif /* CONFIG_ARM64_SVE */
> +
> #ifdef CONFIG_KERNEL_MODE_NEON
>
> DEFINE_PER_CPU(bool, kernel_neon_busy);
> @@ -915,7 +942,7 @@ void kernel_neon_begin(void)
> }
>
> /* Invalidate any task state remaining in the fpsimd regs: */
> - __this_cpu_write(fpsimd_last_state, NULL);
> + fpsimd_flush_cpu_state();
>
> preempt_disable();
>
> @@ -1032,7 +1059,7 @@ static int fpsimd_cpu_pm_notifier(struct notifier_block *self,
> case CPU_PM_ENTER:
> if (current->mm)
> task_fpsimd_save();
> - this_cpu_write(fpsimd_last_state, NULL);
> + fpsimd_flush_cpu_state();
> break;
> case CPU_PM_EXIT:
> if (current->mm)
> diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> index 35a90b8..951f3eb 100644
> --- a/arch/arm64/kvm/hyp/switch.c
> +++ b/arch/arm64/kvm/hyp/switch.c
> @@ -48,7 +48,7 @@ static void __hyp_text __activate_traps_vhe(void)
>
> val = read_sysreg(cpacr_el1);
> val |= CPACR_EL1_TTA;
> - val &= ~CPACR_EL1_FPEN;
> + val &= ~(CPACR_EL1_FPEN | CPACR_EL1_ZEN);
> write_sysreg(val, cpacr_el1);
>
> write_sysreg(__kvm_hyp_vector, vbar_el1);
> @@ -59,7 +59,7 @@ static void __hyp_text __activate_traps_nvhe(void)
> u64 val;
>
> val = CPTR_EL2_DEFAULT;
> - val |= CPTR_EL2_TTA | CPTR_EL2_TFP;
> + val |= CPTR_EL2_TTA | CPTR_EL2_TFP | CPTR_EL2_TZ;
> write_sysreg(val, cptr_el2);
> }
>
> @@ -117,7 +117,7 @@ static void __hyp_text __deactivate_traps_vhe(void)
>
> write_sysreg(mdcr_el2, mdcr_el2);
> write_sysreg(HCR_HOST_VHE_FLAGS, hcr_el2);
> - write_sysreg(CPACR_EL1_FPEN, cpacr_el1);
> + write_sysreg(CPACR_EL1_DEFAULT, cpacr_el1);
> write_sysreg(vectors, vbar_el1);
> }
>
> diff --git a/virt/kvm/arm/arm.c b/virt/kvm/arm/arm.c
> index a39a1e1..af9f5da 100644
> --- a/virt/kvm/arm/arm.c
> +++ b/virt/kvm/arm/arm.c
> @@ -647,6 +647,9 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
> */
> preempt_disable();
>
> + /* Flush FP/SIMD state that can't survive guest entry/exit */
> + kvm_fpsimd_flush_cpu_state();
> +
> kvm_pmu_flush_hwstate(vcpu);
>
> kvm_timer_flush_hwstate(vcpu);
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-14 13:30 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:30 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> When trapping forbidden attempts by a guest to use SVE, we want the
> guest to see a trap consistent with SVE not being implemented.
>
> This patch injects an undefined instruction exception into the
> guest in response to such an exception.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/kvm/handle_exit.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index 17d8a16..e3e42d0 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> return 1;
> }
>
> +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + /* Until SVE is supported for guests: */
> + kvm_inject_undefined(vcpu);
> + return 1;
> +}
> +
> static exit_handle_fn arm_exit_handlers[] = {
> [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> [ESR_ELx_EC_HVC64] = handle_hvc,
> [ESR_ELx_EC_SMC64] = handle_smc,
> [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> + [ESR_ELx_EC_SVE] = handle_sve,
> [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-14 13:30 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:30 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> When trapping forbidden attempts by a guest to use SVE, we want the
> guest to see a trap consistent with SVE not being implemented.
>
> This patch injects an undefined instruction exception into the
> guest in response to such an exception.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
> ---
> arch/arm64/kvm/handle_exit.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index 17d8a16..e3e42d0 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> return 1;
> }
>
> +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + /* Until SVE is supported for guests: */
> + kvm_inject_undefined(vcpu);
> + return 1;
> +}
> +
> static exit_handle_fn arm_exit_handlers[] = {
> [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> [ESR_ELx_EC_HVC64] = handle_hvc,
> [ESR_ELx_EC_SMC64] = handle_smc,
> [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> + [ESR_ELx_EC_SVE] = handle_sve,
> [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-14 13:30 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:30 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> When trapping forbidden attempts by a guest to use SVE, we want the
> guest to see a trap consistent with SVE not being implemented.
>
> This patch injects an undefined instruction exception into the
> guest in response to such an exception.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
> ---
> arch/arm64/kvm/handle_exit.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index 17d8a16..e3e42d0 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> return 1;
> }
>
> +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + /* Until SVE is supported for guests: */
> + kvm_inject_undefined(vcpu);
> + return 1;
> +}
> +
> static exit_handle_fn arm_exit_handlers[] = {
> [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> [ESR_ELx_EC_HVC64] = handle_hvc,
> [ESR_ELx_EC_SMC64] = handle_smc,
> [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> + [ESR_ELx_EC_SVE] = handle_sve,
> [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-14 13:31 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:31 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> When trapping forbidden attempts by a guest to use SVE, we want the
> guest to see a trap consistent with SVE not being implemented.
>
> This patch injects an undefined instruction exception into the
> guest in response to such an exception.
I do wonder if this should be merged with the previous trap enabling
patch though?
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/kvm/handle_exit.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index 17d8a16..e3e42d0 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> return 1;
> }
>
> +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + /* Until SVE is supported for guests: */
> + kvm_inject_undefined(vcpu);
> + return 1;
> +}
> +
> static exit_handle_fn arm_exit_handlers[] = {
> [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> [ESR_ELx_EC_HVC64] = handle_hvc,
> [ESR_ELx_EC_SMC64] = handle_smc,
> [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> + [ESR_ELx_EC_SVE] = handle_sve,
> [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-14 13:31 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:31 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> When trapping forbidden attempts by a guest to use SVE, we want the
> guest to see a trap consistent with SVE not being implemented.
>
> This patch injects an undefined instruction exception into the
> guest in response to such an exception.
I do wonder if this should be merged with the previous trap enabling
patch though?
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/kvm/handle_exit.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index 17d8a16..e3e42d0 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> return 1;
> }
>
> +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + /* Until SVE is supported for guests: */
> + kvm_inject_undefined(vcpu);
> + return 1;
> +}
> +
> static exit_handle_fn arm_exit_handlers[] = {
> [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> [ESR_ELx_EC_HVC64] = handle_hvc,
> [ESR_ELx_EC_SMC64] = handle_smc,
> [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> + [ESR_ELx_EC_SVE] = handle_sve,
> [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-14 13:31 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:31 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> When trapping forbidden attempts by a guest to use SVE, we want the
> guest to see a trap consistent with SVE not being implemented.
>
> This patch injects an undefined instruction exception into the
> guest in response to such an exception.
I do wonder if this should be merged with the previous trap enabling
patch though?
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> ---
> arch/arm64/kvm/handle_exit.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> index 17d8a16..e3e42d0 100644
> --- a/arch/arm64/kvm/handle_exit.c
> +++ b/arch/arm64/kvm/handle_exit.c
> @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> return 1;
> }
>
> +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> +{
> + /* Until SVE is supported for guests: */
> + kvm_inject_undefined(vcpu);
> + return 1;
> +}
> +
> static exit_handle_fn arm_exit_handlers[] = {
> [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> [ESR_ELx_EC_HVC64] = handle_hvc,
> [ESR_ELx_EC_SMC64] = handle_smc,
> [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> + [ESR_ELx_EC_SVE] = handle_sve,
> [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 24/28] arm64/sve: KVM: Hide SVE from CPU features exposed to guests
@ 2017-09-14 13:32 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:32 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> KVM guests cannot currently use SVE, because SVE is always
> configured to trap to EL2.
>
> However, a guest that sees SVE reported as present in
> ID_AA64PFR0_EL1 may legitimately expect that SVE works and try to
> use it. Instead of working, the guest will receive an injected
> undef exception, which may cause the guest to oops or go into a
> spin.
>
> To avoid misleading the guest into believing that SVE will work,
> this patch masks out the SVE field from ID_AA64PFR0_EL1 when a
> guest attempts to read this register. No support is explicitly
> added for ID_AA64ZFR0_EL1 either, so that is still emulated as
> reading as zero, which is consistent with SVE not being
> implemented.
>
> This is a temporary measure, and will be removed in a later series
> when full KVM support for SVE is implemented.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Use pr_err() instead inventing "kvm_info_once" ad-hoc.
> ---
> arch/arm64/kvm/sys_regs.c | 12 +++++++++++-
> 1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index b1f7552..a0ee9b0 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -23,6 +23,7 @@
> #include <linux/bsearch.h>
> #include <linux/kvm_host.h>
> #include <linux/mm.h>
> +#include <linux/printk.h>
> #include <linux/uaccess.h>
>
> #include <asm/cacheflush.h>
> @@ -897,8 +898,17 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
> {
> u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
> (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> + u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
>
> - return raz ? 0 : read_sanitised_ftr_reg(id);
> + if (id == SYS_ID_AA64PFR0_EL1) {
> + if (val & (0xfUL << ID_AA64PFR0_SVE_SHIFT))
> + pr_err_once("kvm [%i]: SVE unsupported for guests, suppressing\n",
> + task_pid_nr(current));
> +
> + val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> + }
> +
> + return val;
> }
>
> /* cpufeature ID register access trap handlers */
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 24/28] arm64/sve: KVM: Hide SVE from CPU features exposed to guests
@ 2017-09-14 13:32 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:32 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, kvmarm, libc-alpha, linux-arch,
Christoffer Dall, Marc Zyngier
Dave Martin <Dave.Martin@arm.com> writes:
> KVM guests cannot currently use SVE, because SVE is always
> configured to trap to EL2.
>
> However, a guest that sees SVE reported as present in
> ID_AA64PFR0_EL1 may legitimately expect that SVE works and try to
> use it. Instead of working, the guest will receive an injected
> undef exception, which may cause the guest to oops or go into a
> spin.
>
> To avoid misleading the guest into believing that SVE will work,
> this patch masks out the SVE field from ID_AA64PFR0_EL1 when a
> guest attempts to read this register. No support is explicitly
> added for ID_AA64ZFR0_EL1 either, so that is still emulated as
> reading as zero, which is consistent with SVE not being
> implemented.
>
> This is a temporary measure, and will be removed in a later series
> when full KVM support for SVE is implemented.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Use pr_err() instead inventing "kvm_info_once" ad-hoc.
> ---
> arch/arm64/kvm/sys_regs.c | 12 +++++++++++-
> 1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index b1f7552..a0ee9b0 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -23,6 +23,7 @@
> #include <linux/bsearch.h>
> #include <linux/kvm_host.h>
> #include <linux/mm.h>
> +#include <linux/printk.h>
> #include <linux/uaccess.h>
>
> #include <asm/cacheflush.h>
> @@ -897,8 +898,17 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
> {
> u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
> (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> + u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
>
> - return raz ? 0 : read_sanitised_ftr_reg(id);
> + if (id == SYS_ID_AA64PFR0_EL1) {
> + if (val & (0xfUL << ID_AA64PFR0_SVE_SHIFT))
> + pr_err_once("kvm [%i]: SVE unsupported for guests, suppressing\n",
> + task_pid_nr(current));
> +
> + val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> + }
> +
> + return val;
> }
>
> /* cpufeature ID register access trap handlers */
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 24/28] arm64/sve: KVM: Hide SVE from CPU features exposed to guests
@ 2017-09-14 13:32 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-14 13:32 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> KVM guests cannot currently use SVE, because SVE is always
> configured to trap to EL2.
>
> However, a guest that sees SVE reported as present in
> ID_AA64PFR0_EL1 may legitimately expect that SVE works and try to
> use it. Instead of working, the guest will receive an injected
> undef exception, which may cause the guest to oops or go into a
> spin.
>
> To avoid misleading the guest into believing that SVE will work,
> this patch masks out the SVE field from ID_AA64PFR0_EL1 when a
> guest attempts to read this register. No support is explicitly
> added for ID_AA64ZFR0_EL1 either, so that is still emulated as
> reading as zero, which is consistent with SVE not being
> implemented.
>
> This is a temporary measure, and will be removed in a later series
> when full KVM support for SVE is implemented.
>
> Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> ---
>
> Changes since v1
> ----------------
>
> Requested by Marc Zyngier:
>
> * Use pr_err() instead inventing "kvm_info_once" ad-hoc.
> ---
> arch/arm64/kvm/sys_regs.c | 12 +++++++++++-
> 1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/arch/arm64/kvm/sys_regs.c b/arch/arm64/kvm/sys_regs.c
> index b1f7552..a0ee9b0 100644
> --- a/arch/arm64/kvm/sys_regs.c
> +++ b/arch/arm64/kvm/sys_regs.c
> @@ -23,6 +23,7 @@
> #include <linux/bsearch.h>
> #include <linux/kvm_host.h>
> #include <linux/mm.h>
> +#include <linux/printk.h>
> #include <linux/uaccess.h>
>
> #include <asm/cacheflush.h>
> @@ -897,8 +898,17 @@ static u64 read_id_reg(struct sys_reg_desc const *r, bool raz)
> {
> u32 id = sys_reg((u32)r->Op0, (u32)r->Op1,
> (u32)r->CRn, (u32)r->CRm, (u32)r->Op2);
> + u64 val = raz ? 0 : read_sanitised_ftr_reg(id);
>
> - return raz ? 0 : read_sanitised_ftr_reg(id);
> + if (id == SYS_ID_AA64PFR0_EL1) {
> + if (val & (0xfUL << ID_AA64PFR0_SVE_SHIFT))
> + pr_err_once("kvm [%i]: SVE unsupported for guests, suppressing\n",
> + task_pid_nr(current));
> +
> + val &= ~(0xfUL << ID_AA64PFR0_SVE_SHIFT);
> + }
> +
> + return val;
> }
>
> /* cpufeature ID register access trap handlers */
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-13 22:21 ` Catalin Marinas
@ 2017-09-14 19:40 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-14 19:40 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 13, 2017 at 03:21:29PM -0700, Catalin Marinas wrote:
> On Wed, Sep 13, 2017 at 08:17:07PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > +/*
> > > > + * Trapped SVE access
> > > > + */
> > > > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > > > +{
> > > > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > > > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > > > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > > > + return;
> > > > + }
> > > > +
> > > > + task_fpsimd_save();
> > > > +
> > > > + sve_alloc(current);
> > > > + fpsimd_to_sve(current);
> > > > + if (test_and_set_thread_flag(TIF_SVE))
> > > > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > > > +
> > > > + task_fpsimd_load();
> > > > +}
> > >
> > > When this function is entered, do we expect TIF_SVE to always be
> > > cleared? It's worth adding a comment on the expected conditions. If
> >
> > Yes, and this is required for correctness, as you observe.
> >
> > I had a BUG_ON() here which I removed, but it makes sense to add a
> > comment to capture the precondition here, and how it is satisfied.
> >
> > > that's the case, task_fpsimd_save() would only save the FPSIMD state
> > > which is fine. However, you subsequently transfer the FPSIMD state to
> > > SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> > > the SVE state here, can we call task_fpsimd_load() *before* setting
> > > TIF_SVE?
> >
> > There should be no way to reach this code with TIF_SVE set, unless
> > task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
> > broken -- either of which is a bug.
>
> Thanks for confirming my assumptions. What I meant was rewriting the
> above function as:
>
> /* reset the SVE state (other than FPSIMD) */
> task_fpsimd_save();
> task_fpsimd_load();
I think this works, but can you explain your rationale?
I think the main effect of your suggestion is that it is cheaper, due
to eliminating some unnecessary load/store operations.
We could go one better, and do
mov v0.16b, v0.16b
mov v1.16b, v1.16b
// ...
mov v31.16b, v31.16b
which doesn't require any memory access.
But I still prefer to zero p0..p15, ffr for cleanliness, even though
the SVE programmer's model doesn't require this (unlike for the Z-reg
high bits where we do need to zero them in order not to violate the
programmer's model).
Currently sve_alloc()+task_fpsimd_load() ensures that all the non-FPSIMD
regs are zeroed too, in addition to the Z-reg high bits.
So we might want a special-purpose helper -- if so, we can do it all
with no memory access.
pfalse p0.b
// ..
pfalse p15.b
wrffr p0.b
This would allow the memset-zero an sve_alloc() to be removed, but I
would need to check what other code is relying on it.
I guess I hadn't done this because I viewed it as an optimisation.
Thoughts?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-14 19:40 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-14 19:40 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 03:21:29PM -0700, Catalin Marinas wrote:
> On Wed, Sep 13, 2017 at 08:17:07PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > +/*
> > > > + * Trapped SVE access
> > > > + */
> > > > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > > > +{
> > > > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > > > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > > > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > > > + return;
> > > > + }
> > > > +
> > > > + task_fpsimd_save();
> > > > +
> > > > + sve_alloc(current);
> > > > + fpsimd_to_sve(current);
> > > > + if (test_and_set_thread_flag(TIF_SVE))
> > > > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > > > +
> > > > + task_fpsimd_load();
> > > > +}
> > >
> > > When this function is entered, do we expect TIF_SVE to always be
> > > cleared? It's worth adding a comment on the expected conditions. If
> >
> > Yes, and this is required for correctness, as you observe.
> >
> > I had a BUG_ON() here which I removed, but it makes sense to add a
> > comment to capture the precondition here, and how it is satisfied.
> >
> > > that's the case, task_fpsimd_save() would only save the FPSIMD state
> > > which is fine. However, you subsequently transfer the FPSIMD state to
> > > SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> > > the SVE state here, can we call task_fpsimd_load() *before* setting
> > > TIF_SVE?
> >
> > There should be no way to reach this code with TIF_SVE set, unless
> > task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
> > broken -- either of which is a bug.
>
> Thanks for confirming my assumptions. What I meant was rewriting the
> above function as:
>
> /* reset the SVE state (other than FPSIMD) */
> task_fpsimd_save();
> task_fpsimd_load();
I think this works, but can you explain your rationale?
I think the main effect of your suggestion is that it is cheaper, due
to eliminating some unnecessary load/store operations.
We could go one better, and do
mov v0.16b, v0.16b
mov v1.16b, v1.16b
// ...
mov v31.16b, v31.16b
which doesn't require any memory access.
But I still prefer to zero p0..p15, ffr for cleanliness, even though
the SVE programmer's model doesn't require this (unlike for the Z-reg
high bits where we do need to zero them in order not to violate the
programmer's model).
Currently sve_alloc()+task_fpsimd_load() ensures that all the non-FPSIMD
regs are zeroed too, in addition to the Z-reg high bits.
So we might want a special-purpose helper -- if so, we can do it all
with no memory access.
pfalse p0.b
// ..
pfalse p15.b
wrffr p0.b
This would allow the memset-zero an sve_alloc() to be removed, but I
would need to check what other code is relying on it.
I guess I hadn't done this because I viewed it as an optimisation.
Thoughts?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-13 14:33 ` Catalin Marinas
@ 2017-09-14 19:55 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-14 19:55 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > +/*
> > + * Handle SVE state across fork():
> > + *
> > + * dst and src must not end up with aliases of the same sve_state.
> > + * Because a task cannot fork except in a syscall, we can discard SVE
> > + * state for dst here, so long as we take care to retain the FPSIMD
> > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > + * will be deferred until dst tries to use SVE.
> > + */
> > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > +{
> > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > + sve_to_fpsimd(dst);
> > + }
> > +
> > + dst->thread.sve_state = NULL;
> > +}
>
> I first thought the thread flags are not visible in dst yet since
> dup_task_struct() calls arch_dup_task_struct() before
> setup_thread_stack(). However, at the end of the last year we enabled
> CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> on this.
Hmmm, I see your point, but there are some sequencing issues here.
> Anyway, IIUC we don't need sve_to_fpsimd() here. The
> arch_dup_task_struct() already called fpsimd_preserve_current_state()
I consider SVE discard as an optional side effect of task_fpsimd_save(),
not something that is guaranteed to happen -- the decision about whether
to do so may become more intelligent later on. So, for src, we may
discard SVE (because syscall), but for dst we must NULL .sve_state (and
therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
dst->sve_state.
The latter requires operating on the thread_flags of dst. I'll need to
check whether there's another suitable hook for updating the thread flags
of dst, if we aren't confident that they will always have been
initialised by the time arch_dup_task_struct() is called.
Either way, there would be an intra-thread ordering requirement between
the task_struct and thread_info updates here, _if_ dst were schedulable:
dst:TIF_SVE must be cleared before dst->sve_state is NULLed.
dst is not schedulable until fork is done though, so maybe this doesn't
really matter...
> for src, so the FPSIMD state (which we care about) is transferred during
> the *dst = *src assignment. So you'd only need the last statement,
> possibly with a different function name like fpsimd_erase_sve (and maybe
> make the function static inline in the header).
Not quite: TIF_SVE must be cleared so that a context switch or
kernel_neon_begin() after dst is scheduled doesn't try to save state in
the (NULL) dst->sve_state.
>
> [...]
> > int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> > @@ -246,6 +247,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> > if (current->mm)
> > fpsimd_preserve_current_state();
> > *dst = *src;
> > +
> > + fpsimd_dup_sve(dst, src);
> > +
> > return 0;
> > }
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-14 19:55 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-14 19:55 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > +/*
> > + * Handle SVE state across fork():
> > + *
> > + * dst and src must not end up with aliases of the same sve_state.
> > + * Because a task cannot fork except in a syscall, we can discard SVE
> > + * state for dst here, so long as we take care to retain the FPSIMD
> > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > + * will be deferred until dst tries to use SVE.
> > + */
> > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > +{
> > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > + sve_to_fpsimd(dst);
> > + }
> > +
> > + dst->thread.sve_state = NULL;
> > +}
>
> I first thought the thread flags are not visible in dst yet since
> dup_task_struct() calls arch_dup_task_struct() before
> setup_thread_stack(). However, at the end of the last year we enabled
> CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> on this.
Hmmm, I see your point, but there are some sequencing issues here.
> Anyway, IIUC we don't need sve_to_fpsimd() here. The
> arch_dup_task_struct() already called fpsimd_preserve_current_state()
I consider SVE discard as an optional side effect of task_fpsimd_save(),
not something that is guaranteed to happen -- the decision about whether
to do so may become more intelligent later on. So, for src, we may
discard SVE (because syscall), but for dst we must NULL .sve_state (and
therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
dst->sve_state.
The latter requires operating on the thread_flags of dst. I'll need to
check whether there's another suitable hook for updating the thread flags
of dst, if we aren't confident that they will always have been
initialised by the time arch_dup_task_struct() is called.
Either way, there would be an intra-thread ordering requirement between
the task_struct and thread_info updates here, _if_ dst were schedulable:
dst:TIF_SVE must be cleared before dst->sve_state is NULLed.
dst is not schedulable until fork is done though, so maybe this doesn't
really matter...
> for src, so the FPSIMD state (which we care about) is transferred during
> the *dst = *src assignment. So you'd only need the last statement,
> possibly with a different function name like fpsimd_erase_sve (and maybe
> make the function static inline in the header).
Not quite: TIF_SVE must be cleared so that a context switch or
kernel_neon_begin() after dst is scheduled doesn't try to save state in
the (NULL) dst->sve_state.
>
> [...]
> > int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> > @@ -246,6 +247,9 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> > if (current->mm)
> > fpsimd_preserve_current_state();
> > *dst = *src;
> > +
> > + fpsimd_dup_sve(dst, src);
> > +
> > return 0;
> > }
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests
2017-09-13 14:37 ` Alex Bennée
@ 2017-09-15 0:04 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-15 0:04 UTC (permalink / raw)
To: Alex Bennée
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Marc Zyngier, Christoffer Dall,
Richard Sandiford, kvmarm, linux-arm-kernel
On Wed, Sep 13, 2017 at 03:37:42PM +0100, Alex Bennée wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
[...]
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 945e79c..35a90b8 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -81,11 +81,17 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > * it will cause an exception.
> > */
> > val = vcpu->arch.hcr_el2;
> > +
> > if (!(val & HCR_RW) && system_supports_fpsimd()) {
> > write_sysreg(1 << 30, fpexc32_el2);
> > isb();
> > }
> > +
> > + if (val & HCR_RW) /* for AArch64 only: */
> > + val |= HCR_TID3; /* TID3: trap feature register accesses */
> > +
>
> I wondered as this is the hyp switch can we make use of testing val &
> HCR_RW for both this and above. But it seems minimal in the generated
> code so probably not.
I figured that the code was cleaner ths way, since they're independent
bits of code that both happen to be applicable only to AArch64 guests.
[...]
> > +
> > + /*
> > + * ID regs: all ID_SANITISED() entries here must have corresponding
> > + * entries in arm64_ftr_regs[].
> > + */
>
> arm64_ftr_regs isn't updated in this commit. Does this break bisection?
This commit only adds ID_SANITISED() entries for regs that are already
present in arm64_ftr_regs[]. (If you spot any that are missing, give me
a shout...)
SVE only adds one new ID register, ID_AA64ZFR0_EL1 -- but SVE defines no
fields in there yet, so I just leave it ID_UNALLOCATED() which will
cause it to read as zero for the guest.
> > +
> > + /* AArch64 mappings of the AArch32 ID registers */
> > + /* CRm=1 */
> > + ID_SANITISED(ID_PFR0_EL1),
> > + ID_SANITISED(ID_PFR1_EL1),
[...]
> > + /* CRm=7 */
> > + ID_SANITISED(ID_AA64MMFR0_EL1),
> > + ID_SANITISED(ID_AA64MMFR1_EL1),
> > + ID_SANITISED(ID_AA64MMFR2_EL1),
> > + ID_UNALLOCATED(7,3),
> > + ID_UNALLOCATED(7,4),
> > + ID_UNALLOCATED(7,5),
> > + ID_UNALLOCATED(7,6),
> > + ID_UNALLOCATED(7,7),
> > +
>
> I think it might be worthwhile adding a test to kvm-unit-tests to walk
> all the ID registers to check this.
Sounds sensible, I'll take a look at that.
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests
@ 2017-09-15 0:04 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-15 0:04 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 03:37:42PM +0100, Alex Benn?e wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
[...]
> > diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c
> > index 945e79c..35a90b8 100644
> > --- a/arch/arm64/kvm/hyp/switch.c
> > +++ b/arch/arm64/kvm/hyp/switch.c
> > @@ -81,11 +81,17 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu)
> > * it will cause an exception.
> > */
> > val = vcpu->arch.hcr_el2;
> > +
> > if (!(val & HCR_RW) && system_supports_fpsimd()) {
> > write_sysreg(1 << 30, fpexc32_el2);
> > isb();
> > }
> > +
> > + if (val & HCR_RW) /* for AArch64 only: */
> > + val |= HCR_TID3; /* TID3: trap feature register accesses */
> > +
>
> I wondered as this is the hyp switch can we make use of testing val &
> HCR_RW for both this and above. But it seems minimal in the generated
> code so probably not.
I figured that the code was cleaner ths way, since they're independent
bits of code that both happen to be applicable only to AArch64 guests.
[...]
> > +
> > + /*
> > + * ID regs: all ID_SANITISED() entries here must have corresponding
> > + * entries in arm64_ftr_regs[].
> > + */
>
> arm64_ftr_regs isn't updated in this commit. Does this break bisection?
This commit only adds ID_SANITISED() entries for regs that are already
present in arm64_ftr_regs[]. (If you spot any that are missing, give me
a shout...)
SVE only adds one new ID register, ID_AA64ZFR0_EL1 -- but SVE defines no
fields in there yet, so I just leave it ID_UNALLOCATED() which will
cause it to read as zero for the guest.
> > +
> > + /* AArch64 mappings of the AArch32 ID registers */
> > + /* CRm=1 */
> > + ID_SANITISED(ID_PFR0_EL1),
> > + ID_SANITISED(ID_PFR1_EL1),
[...]
> > + /* CRm=7 */
> > + ID_SANITISED(ID_AA64MMFR0_EL1),
> > + ID_SANITISED(ID_AA64MMFR1_EL1),
> > + ID_SANITISED(ID_AA64MMFR2_EL1),
> > + ID_UNALLOCATED(7,3),
> > + ID_UNALLOCATED(7,4),
> > + ID_UNALLOCATED(7,5),
> > + ID_UNALLOCATED(7,6),
> > + ID_UNALLOCATED(7,7),
> > +
>
> I think it might be worthwhile adding a test to kvm-unit-tests to walk
> all the ID registers to check this.
Sounds sensible, I'll take a look at that.
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-14 19:40 ` Dave Martin
@ 2017-09-19 17:13 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-19 17:13 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Thu, Sep 14, 2017 at 08:40:41PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 03:21:29PM -0700, Catalin Marinas wrote:
> > On Wed, Sep 13, 2017 at 08:17:07PM +0100, Dave P Martin wrote:
> > > On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> > > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > > +/*
> > > > > + * Trapped SVE access
> > > > > + */
> > > > > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > > > > +{
> > > > > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > > > > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > > > > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + task_fpsimd_save();
> > > > > +
> > > > > + sve_alloc(current);
> > > > > + fpsimd_to_sve(current);
> > > > > + if (test_and_set_thread_flag(TIF_SVE))
> > > > > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > > > > +
> > > > > + task_fpsimd_load();
> > > > > +}
> > > >
> > > > When this function is entered, do we expect TIF_SVE to always be
> > > > cleared? It's worth adding a comment on the expected conditions. If
> > >
> > > Yes, and this is required for correctness, as you observe.
> > >
> > > I had a BUG_ON() here which I removed, but it makes sense to add a
> > > comment to capture the precondition here, and how it is satisfied.
> > >
> > > > that's the case, task_fpsimd_save() would only save the FPSIMD state
> > > > which is fine. However, you subsequently transfer the FPSIMD state to
> > > > SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> > > > the SVE state here, can we call task_fpsimd_load() *before* setting
> > > > TIF_SVE?
> > >
> > > There should be no way to reach this code with TIF_SVE set, unless
> > > task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
> > > broken -- either of which is a bug.
> >
> > Thanks for confirming my assumptions. What I meant was rewriting the
> > above function as:
> >
> > /* reset the SVE state (other than FPSIMD) */
> > task_fpsimd_save();
> > task_fpsimd_load();
>
> I think this works, but can you explain your rationale?
>
> I think the main effect of your suggestion is that it is cheaper, due
> to eliminating some unnecessary load/store operations.
My rationale was to avoid copying between the in-memory FPSIMD and SVE
state.
> We could go one better, and do
>
> mov v0.16b, v0.16b
> mov v1.16b, v1.16b
> // ...
> mov v31.16b, v31.16b
>
> which doesn't require any memory access.
Yes, that's even better.
> But I still prefer to zero p0..p15, ffr for cleanliness, even though
> the SVE programmer's model doesn't require this (unlike for the Z-reg
> high bits where we do need to zero them in order not to violate the
> programmer's model).
I missed the px, ffr aspect. Can you not have a clear_sve_state() (or a
better name) function to zero the predicate regs, ffr and the top bits
of the vectors?
> Currently sve_alloc()+task_fpsimd_load() ensures that all the non-FPSIMD
> regs are zeroed too, in addition to the Z-reg high bits.
Yes, just wondering if this can be implemented with less memory accesses
since the SVE state is irrelevant at this stage.
>
> So we might want a special-purpose helper -- if so, we can do it all
> with no memory access.
>
> pfalse p0.b
> // ..
> pfalse p15.b
> wrffr p0.b
>
> This would allow the memset-zero an sve_alloc() to be removed, but I
> would need to check what other code is relying on it.
>
> I guess I hadn't done this because I viewed it as an optimisation.
It looked like some low-hanging optimisation to slightly accelerate the
allocation of the SVE state on access, though I'm also worried I don't
fully understand all the corner cases (like what happens if we allow
interrupts during this function and get preempted).
Anyway, I'm fine to leave this as it is for now and try to optimise it
later with additional patches on top.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-19 17:13 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-19 17:13 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Sep 14, 2017 at 08:40:41PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 03:21:29PM -0700, Catalin Marinas wrote:
> > On Wed, Sep 13, 2017 at 08:17:07PM +0100, Dave P Martin wrote:
> > > On Wed, Sep 13, 2017 at 10:26:05AM -0700, Catalin Marinas wrote:
> > > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > > +/*
> > > > > + * Trapped SVE access
> > > > > + */
> > > > > +void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > > > > +{
> > > > > + /* Even if we chose not to use SVE, the hardware could still trap: */
> > > > > + if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > > > > + force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > > > > + return;
> > > > > + }
> > > > > +
> > > > > + task_fpsimd_save();
> > > > > +
> > > > > + sve_alloc(current);
> > > > > + fpsimd_to_sve(current);
> > > > > + if (test_and_set_thread_flag(TIF_SVE))
> > > > > + WARN_ON(1); /* SVE access shouldn't have trapped */
> > > > > +
> > > > > + task_fpsimd_load();
> > > > > +}
> > > >
> > > > When this function is entered, do we expect TIF_SVE to always be
> > > > cleared? It's worth adding a comment on the expected conditions. If
> > >
> > > Yes, and this is required for correctness, as you observe.
> > >
> > > I had a BUG_ON() here which I removed, but it makes sense to add a
> > > comment to capture the precondition here, and how it is satisfied.
> > >
> > > > that's the case, task_fpsimd_save() would only save the FPSIMD state
> > > > which is fine. However, you subsequently transfer the FPSIMD state to
> > > > SVE, set TIF_SVE and restore the full SVE state. If we don't care about
> > > > the SVE state here, can we call task_fpsimd_load() *before* setting
> > > > TIF_SVE?
> > >
> > > There should be no way to reach this code with TIF_SVE set, unless
> > > task_fpsimd_load() sets the CPACR trap bit wrongly, or the hardware is
> > > broken -- either of which is a bug.
> >
> > Thanks for confirming my assumptions. What I meant was rewriting the
> > above function as:
> >
> > /* reset the SVE state (other than FPSIMD) */
> > task_fpsimd_save();
> > task_fpsimd_load();
>
> I think this works, but can you explain your rationale?
>
> I think the main effect of your suggestion is that it is cheaper, due
> to eliminating some unnecessary load/store operations.
My rationale was to avoid copying between the in-memory FPSIMD and SVE
state.
> We could go one better, and do
>
> mov v0.16b, v0.16b
> mov v1.16b, v1.16b
> // ...
> mov v31.16b, v31.16b
>
> which doesn't require any memory access.
Yes, that's even better.
> But I still prefer to zero p0..p15, ffr for cleanliness, even though
> the SVE programmer's model doesn't require this (unlike for the Z-reg
> high bits where we do need to zero them in order not to violate the
> programmer's model).
I missed the px, ffr aspect. Can you not have a clear_sve_state() (or a
better name) function to zero the predicate regs, ffr and the top bits
of the vectors?
> Currently sve_alloc()+task_fpsimd_load() ensures that all the non-FPSIMD
> regs are zeroed too, in addition to the Z-reg high bits.
Yes, just wondering if this can be implemented with less memory accesses
since the SVE state is irrelevant at this stage.
>
> So we might want a special-purpose helper -- if so, we can do it all
> with no memory access.
>
> pfalse p0.b
> // ..
> pfalse p15.b
> wrffr p0.b
>
> This would allow the memset-zero an sve_alloc() to be removed, but I
> would need to check what other code is relying on it.
>
> I guess I hadn't done this because I viewed it as an optimisation.
It looked like some low-hanging optimisation to slightly accelerate the
allocation of the SVE state on access, though I'm also worried I don't
fully understand all the corner cases (like what happens if we allow
interrupts during this function and get preempted).
Anyway, I'm fine to leave this as it is for now and try to optimise it
later with additional patches on top.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-08-31 17:00 ` Dave Martin
@ 2017-09-20 10:57 ` Alan Hayward
-1 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-20 10:57 UTC (permalink / raw)
To: Dave P Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Alex Bennée, Szabolcs Nagy, Richard Sandiford, kvmarm,
libc-alpha, linux-arch, gdb, Yao Qi
> On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>
> +int sve_set_vector_length(struct task_struct *task,
> + unsigned long vl, unsigned long flags)
> +{
> +WARN_ON(task == current && preemptible());
> +
> +if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
> + PR_SVE_SET_VL_ONEXEC))
> +return -EINVAL;
> +
> +if (!sve_vl_valid(vl))
> +return -EINVAL;
> +
> +/*
> + * Clamp to the maximum vector length that VL-agnostic SVE code can
> + * work with. A flag may be assigned in the future to allow setting
> + * of larger vector lengths without confusing older software.
> + */
> +if (vl > SVE_VL_ARCH_MAX)
> +vl = SVE_VL_ARCH_MAX;
> +
> +vl = find_supported_vector_length(vl);
> +
Given, sve_set_vector_length is called when setting the vector length in
PTRACE_SETREGSET, it looks to me like if you set VL to a value that’s not
supported by the hardware, then it’s going to round down to the previous value.
Is that correct? I’m not sure if that’s explained in the docs?
What happens if you give a vl value lower than the min supported value in the
hardware?
> +/*
> + * All vector length selection from userspace comes through here.
> + * We're on a slow path, so some sanity-checks are included.
> + * If things go wrong there's a bug somewhere, but try to fall back to a
> + * safe choice.
> + */
> +static unsigned int find_supported_vector_length(unsigned int vl)
> +{
> +int bit;
> +int max_vl = sve_max_vl;
> +
> +if (WARN_ON(!sve_vl_valid(vl)))
> +vl = SVE_VL_MIN;
> +
> +if (WARN_ON(!sve_vl_valid(max_vl)))
> +max_vl = SVE_VL_MIN;
> +
> +if (vl > max_vl)
> +vl = max_vl;
> +
> +bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
> + vq_to_bit(sve_vq_from_vl(vl)));
> +return sve_vl_from_vq(bit_to_vq(bit));
> +}
> +
Thanks,
Alan.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-20 10:57 ` Alan Hayward
0 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-20 10:57 UTC (permalink / raw)
To: linux-arm-kernel
> On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>
> +int sve_set_vector_length(struct task_struct *task,
> + unsigned long vl, unsigned long flags)
> +{
> +WARN_ON(task == current && preemptible());
> +
> +if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
> + PR_SVE_SET_VL_ONEXEC))
> +return -EINVAL;
> +
> +if (!sve_vl_valid(vl))
> +return -EINVAL;
> +
> +/*
> + * Clamp to the maximum vector length that VL-agnostic SVE code can
> + * work with. A flag may be assigned in the future to allow setting
> + * of larger vector lengths without confusing older software.
> + */
> +if (vl > SVE_VL_ARCH_MAX)
> +vl = SVE_VL_ARCH_MAX;
> +
> +vl = find_supported_vector_length(vl);
> +
Given, sve_set_vector_length is called when setting the vector length in
PTRACE_SETREGSET, it looks to me like if you set VL to a value that?s not
supported by the hardware, then it?s going to round down to the previous value.
Is that correct? I?m not sure if that?s explained in the docs?
What happens if you give a vl value lower than the min supported value in the
hardware?
> +/*
> + * All vector length selection from userspace comes through here.
> + * We're on a slow path, so some sanity-checks are included.
> + * If things go wrong there's a bug somewhere, but try to fall back to a
> + * safe choice.
> + */
> +static unsigned int find_supported_vector_length(unsigned int vl)
> +{
> +int bit;
> +int max_vl = sve_max_vl;
> +
> +if (WARN_ON(!sve_vl_valid(vl)))
> +vl = SVE_VL_MIN;
> +
> +if (WARN_ON(!sve_vl_valid(max_vl)))
> +max_vl = SVE_VL_MIN;
> +
> +if (vl > max_vl)
> +vl = max_vl;
> +
> +bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
> + vq_to_bit(sve_vq_from_vl(vl)));
> +return sve_vl_from_vq(bit_to_vq(bit));
> +}
> +
Thanks,
Alan.
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-08-31 17:00 ` Dave Martin
@ 2017-09-20 10:59 ` Alan Hayward
-1 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-20 10:59 UTC (permalink / raw)
To: Dave P Martin
Cc: linux-arm-kernel, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Alex Bennée, Szabolcs Nagy, Richard Sandiford, kvmarm,
libc-alpha, linux-arch, gdb, Yao Qi, nd
(Resending without disclaimer)
> On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>
> +int sve_set_vector_length(struct task_struct *task,
> + unsigned long vl, unsigned long flags)
> +{
> + WARN_ON(task == current && preemptible());
> +
> + if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
> + PR_SVE_SET_VL_ONEXEC))
> + return -EINVAL;
> +
> + if (!sve_vl_valid(vl))
> + return -EINVAL;
> +
> + /*
> + * Clamp to the maximum vector length that VL-agnostic SVE code can
> + * work with. A flag may be assigned in the future to allow setting
> + * of larger vector lengths without confusing older software.
> + */
> + if (vl > SVE_VL_ARCH_MAX)
> + vl = SVE_VL_ARCH_MAX;
> +
> + vl = find_supported_vector_length(vl);
> +
Given, sve_set_vector_length is called when setting the vector length in
PTRACE_SETREGSET, it looks to me like if you set VL to a value that’s not
supported by the hardware, then it’s going to round down to the previous value.
Is that correct? I’m not sure if that’s explained in the docs?
What happens if you give a vl value lower than the min supported value in the
hardware?
> +/*
> + * All vector length selection from userspace comes through here.
> + * We're on a slow path, so some sanity-checks are included.
> + * If things go wrong there's a bug somewhere, but try to fall back to a
> + * safe choice.
> + */
> +static unsigned int find_supported_vector_length(unsigned int vl)
> +{
> + int bit;
> + int max_vl = sve_max_vl;
> +
> + if (WARN_ON(!sve_vl_valid(vl)))
> + vl = SVE_VL_MIN;
> +
> + if (WARN_ON(!sve_vl_valid(max_vl)))
> + max_vl = SVE_VL_MIN;
> +
> + if (vl > max_vl)
> + vl = max_vl;
> +
> + bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
> + vq_to_bit(sve_vq_from_vl(vl)));
> + return sve_vl_from_vq(bit_to_vq(bit));
> +}
> +
Thanks,
Alan.
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-20 10:59 ` Alan Hayward
0 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-20 10:59 UTC (permalink / raw)
To: linux-arm-kernel
(Resending without disclaimer)
> On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>
> +int sve_set_vector_length(struct task_struct *task,
> + unsigned long vl, unsigned long flags)
> +{
> + WARN_ON(task == current && preemptible());
> +
> + if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
> + PR_SVE_SET_VL_ONEXEC))
> + return -EINVAL;
> +
> + if (!sve_vl_valid(vl))
> + return -EINVAL;
> +
> + /*
> + * Clamp to the maximum vector length that VL-agnostic SVE code can
> + * work with. A flag may be assigned in the future to allow setting
> + * of larger vector lengths without confusing older software.
> + */
> + if (vl > SVE_VL_ARCH_MAX)
> + vl = SVE_VL_ARCH_MAX;
> +
> + vl = find_supported_vector_length(vl);
> +
Given, sve_set_vector_length is called when setting the vector length in
PTRACE_SETREGSET, it looks to me like if you set VL to a value that?s not
supported by the hardware, then it?s going to round down to the previous value.
Is that correct? I?m not sure if that?s explained in the docs?
What happens if you give a vl value lower than the min supported value in the
hardware?
> +/*
> + * All vector length selection from userspace comes through here.
> + * We're on a slow path, so some sanity-checks are included.
> + * If things go wrong there's a bug somewhere, but try to fall back to a
> + * safe choice.
> + */
> +static unsigned int find_supported_vector_length(unsigned int vl)
> +{
> + int bit;
> + int max_vl = sve_max_vl;
> +
> + if (WARN_ON(!sve_vl_valid(vl)))
> + vl = SVE_VL_MIN;
> +
> + if (WARN_ON(!sve_vl_valid(max_vl)))
> + max_vl = SVE_VL_MIN;
> +
> + if (vl > max_vl)
> + vl = max_vl;
> +
> + bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
> + vq_to_bit(sve_vq_from_vl(vl)));
> + return sve_vl_from_vq(bit_to_vq(bit));
> +}
> +
Thanks,
Alan.
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-09-20 10:59 ` Alan Hayward
@ 2017-09-20 11:09 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-20 11:09 UTC (permalink / raw)
To: Alan Hayward
Cc: Dave P Martin, linux-arch, libc-alpha, gdb, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Yao Qi, Will Deacon,
Richard Sandiford, nd, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 20, 2017 at 10:59:55AM +0000, Alan Hayward wrote:
> (Resending without disclaimer)
>
> > On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>
> >
> > +int sve_set_vector_length(struct task_struct *task,
> > + unsigned long vl, unsigned long flags)
> > +{
> > + WARN_ON(task == current && preemptible());
> > +
> > + if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
> > + PR_SVE_SET_VL_ONEXEC))
> > + return -EINVAL;
> > +
> > + if (!sve_vl_valid(vl))
> > + return -EINVAL;
> > +
> > + /*
> > + * Clamp to the maximum vector length that VL-agnostic SVE code can
> > + * work with. A flag may be assigned in the future to allow setting
> > + * of larger vector lengths without confusing older software.
> > + */
> > + if (vl > SVE_VL_ARCH_MAX)
> > + vl = SVE_VL_ARCH_MAX;
> > +
> > + vl = find_supported_vector_length(vl);
> > +
>
>
> Given, sve_set_vector_length is called when setting the vector length in
> PTRACE_SETREGSET, it looks to me like if you set VL to a value that’s not
> supported by the hardware, then it’s going to round down to the previous value.
> Is that correct? I’m not sure if that’s explained in the docs?
Does this cover it?
"On success, the calling thread's vector length is changed to the
largest value supported by the system that is less than or equal to vl."
(For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
> What happens if you give a vl value lower than the min supported value in the
> hardware?
This is impossible, unless vl < SVE_VL_MIN (which is rejected explicitly
by the !sve_vl_valid() check in sve_set_vector_length()).
The architecture required support for all power-of-two vector lengths
less than the maximum supported vector length, so by construction
SVE_VL_MIN is supported by all hardware.
To be defensive, if we fail to detect support for SVE_VL_MIN, I set the
corresponding bit in sve_vq_map and WARN. This is just to help ensure
find_supported_vector_length doesn't fall off the end of sve_vq_map.
Does that sounds correct? There may be a clearer way of achieving this.
Cheers
---Dave
>
>
> > +/*
> > + * All vector length selection from userspace comes through here.
> > + * We're on a slow path, so some sanity-checks are included.
> > + * If things go wrong there's a bug somewhere, but try to fall back to a
> > + * safe choice.
> > + */
> > +static unsigned int find_supported_vector_length(unsigned int vl)
> > +{
> > + int bit;
> > + int max_vl = sve_max_vl;
> > +
> > + if (WARN_ON(!sve_vl_valid(vl)))
> > + vl = SVE_VL_MIN;
> > +
> > + if (WARN_ON(!sve_vl_valid(max_vl)))
> > + max_vl = SVE_VL_MIN;
> > +
> > + if (vl > max_vl)
> > + vl = max_vl;
> > +
> > + bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
> > + vq_to_bit(sve_vq_from_vl(vl)));
> > + return sve_vl_from_vq(bit_to_vq(bit));
> > +}
> > +
>
>
> Thanks,
> Alan.
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-20 11:09 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-20 11:09 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 20, 2017 at 10:59:55AM +0000, Alan Hayward wrote:
> (Resending without disclaimer)
>
> > On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>
> >
> > +int sve_set_vector_length(struct task_struct *task,
> > + unsigned long vl, unsigned long flags)
> > +{
> > + WARN_ON(task == current && preemptible());
> > +
> > + if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
> > + PR_SVE_SET_VL_ONEXEC))
> > + return -EINVAL;
> > +
> > + if (!sve_vl_valid(vl))
> > + return -EINVAL;
> > +
> > + /*
> > + * Clamp to the maximum vector length that VL-agnostic SVE code can
> > + * work with. A flag may be assigned in the future to allow setting
> > + * of larger vector lengths without confusing older software.
> > + */
> > + if (vl > SVE_VL_ARCH_MAX)
> > + vl = SVE_VL_ARCH_MAX;
> > +
> > + vl = find_supported_vector_length(vl);
> > +
>
>
> Given, sve_set_vector_length is called when setting the vector length in
> PTRACE_SETREGSET, it looks to me like if you set VL to a value that?s not
> supported by the hardware, then it?s going to round down to the previous value.
> Is that correct? I?m not sure if that?s explained in the docs?
Does this cover it?
"On success, the calling thread's vector length is changed to the
largest value supported by the system that is less than or equal to vl."
(For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
> What happens if you give a vl value lower than the min supported value in the
> hardware?
This is impossible, unless vl < SVE_VL_MIN (which is rejected explicitly
by the !sve_vl_valid() check in sve_set_vector_length()).
The architecture required support for all power-of-two vector lengths
less than the maximum supported vector length, so by construction
SVE_VL_MIN is supported by all hardware.
To be defensive, if we fail to detect support for SVE_VL_MIN, I set the
corresponding bit in sve_vq_map and WARN. This is just to help ensure
find_supported_vector_length doesn't fall off the end of sve_vq_map.
Does that sounds correct? There may be a clearer way of achieving this.
Cheers
---Dave
>
>
> > +/*
> > + * All vector length selection from userspace comes through here.
> > + * We're on a slow path, so some sanity-checks are included.
> > + * If things go wrong there's a bug somewhere, but try to fall back to a
> > + * safe choice.
> > + */
> > +static unsigned int find_supported_vector_length(unsigned int vl)
> > +{
> > + int bit;
> > + int max_vl = sve_max_vl;
> > +
> > + if (WARN_ON(!sve_vl_valid(vl)))
> > + vl = SVE_VL_MIN;
> > +
> > + if (WARN_ON(!sve_vl_valid(max_vl)))
> > + max_vl = SVE_VL_MIN;
> > +
> > + if (vl > max_vl)
> > + vl = max_vl;
> > +
> > + bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
> > + vq_to_bit(sve_vq_from_vl(vl)));
> > + return sve_vl_from_vq(bit_to_vq(bit));
> > +}
> > +
>
>
> Thanks,
> Alan.
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-14 19:55 ` Dave Martin
@ 2017-09-20 13:58 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-20 13:58 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Thu, Sep 14, 2017 at 08:55:56PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > +/*
> > > + * Handle SVE state across fork():
> > > + *
> > > + * dst and src must not end up with aliases of the same sve_state.
> > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > + * will be deferred until dst tries to use SVE.
> > > + */
> > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > +{
> > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > + sve_to_fpsimd(dst);
> > > + }
> > > +
> > > + dst->thread.sve_state = NULL;
> > > +}
> >
> > I first thought the thread flags are not visible in dst yet since
> > dup_task_struct() calls arch_dup_task_struct() before
> > setup_thread_stack(). However, at the end of the last year we enabled
> > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > on this.
>
> Hmmm, I see your point, but there are some sequencing issues here.
>
> > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > arch_dup_task_struct() already called fpsimd_preserve_current_state()
>
> I consider SVE discard as an optional side effect of task_fpsimd_save(),
> not something that is guaranteed to happen -- the decision about whether
> to do so may become more intelligent later on. So, for src, we may
> discard SVE (because syscall), but for dst we must NULL .sve_state (and
> therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
> dst->sve_state.
My point was that the SVE state of src is already preserved at this
point and copied into dst. You don't need the sve_to_fpsimd(dst) again
which basically does the same copying of the src SVE saved state into
the FPSIMD one in dst. This has already been done in
arch_dup_task_struct() by the combination of
fpsimd_preserve_current_state() and *dst = *src (and, of course,
clearing TIF_SVE in dst).
I don't think the TIF_SVE clearing in src is just a side effect of
task_fpsimd_save() here but rather a requirement. When returning from
fork(), both src and dst would need to have the same state. However,
your fpsimd_dup_sve() implementation makes it very clear that the SVE
state is lost in dst. This is only allowed if we also lose it in src (as
a result of a syscall). So making dst->sve_state = NULL requires that
TIF_SVE is also cleared in both src and dst. Alternatively, you'd have
to allocate a new state here and copy the full src SVE state across to
dst, together with setting TIF_SVE (that's not necessary, however, since
we get here as a result of a syscall).
> > for src, so the FPSIMD state (which we care about) is transferred during
> > the *dst = *src assignment. So you'd only need the last statement,
> > possibly with a different function name like fpsimd_erase_sve (and maybe
> > make the function static inline in the header).
>
> Not quite: TIF_SVE must be cleared so that a context switch or
> kernel_neon_begin() after dst is scheduled doesn't try to save state in
> the (NULL) dst->sve_state.
Yes, TIF_SVE must also be cleared in dst when dst->sve_state = NULL (I
may have forgotten to mention this).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-09-20 13:58 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-09-20 13:58 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Sep 14, 2017 at 08:55:56PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > +/*
> > > + * Handle SVE state across fork():
> > > + *
> > > + * dst and src must not end up with aliases of the same sve_state.
> > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > + * will be deferred until dst tries to use SVE.
> > > + */
> > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > +{
> > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > + sve_to_fpsimd(dst);
> > > + }
> > > +
> > > + dst->thread.sve_state = NULL;
> > > +}
> >
> > I first thought the thread flags are not visible in dst yet since
> > dup_task_struct() calls arch_dup_task_struct() before
> > setup_thread_stack(). However, at the end of the last year we enabled
> > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > on this.
>
> Hmmm, I see your point, but there are some sequencing issues here.
>
> > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > arch_dup_task_struct() already called fpsimd_preserve_current_state()
>
> I consider SVE discard as an optional side effect of task_fpsimd_save(),
> not something that is guaranteed to happen -- the decision about whether
> to do so may become more intelligent later on. So, for src, we may
> discard SVE (because syscall), but for dst we must NULL .sve_state (and
> therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
> dst->sve_state.
My point was that the SVE state of src is already preserved at this
point and copied into dst. You don't need the sve_to_fpsimd(dst) again
which basically does the same copying of the src SVE saved state into
the FPSIMD one in dst. This has already been done in
arch_dup_task_struct() by the combination of
fpsimd_preserve_current_state() and *dst = *src (and, of course,
clearing TIF_SVE in dst).
I don't think the TIF_SVE clearing in src is just a side effect of
task_fpsimd_save() here but rather a requirement. When returning from
fork(), both src and dst would need to have the same state. However,
your fpsimd_dup_sve() implementation makes it very clear that the SVE
state is lost in dst. This is only allowed if we also lose it in src (as
a result of a syscall). So making dst->sve_state = NULL requires that
TIF_SVE is also cleared in both src and dst. Alternatively, you'd have
to allocate a new state here and copy the full src SVE state across to
dst, together with setting TIF_SVE (that's not necessary, however, since
we get here as a result of a syscall).
> > for src, so the FPSIMD state (which we care about) is transferred during
> > the *dst = *src assignment. So you'd only need the last statement,
> > possibly with a different function name like fpsimd_erase_sve (and maybe
> > make the function static inline in the header).
>
> Not quite: TIF_SVE must be cleared so that a context switch or
> kernel_neon_begin() after dst is scheduled doesn't try to save state in
> the (NULL) dst->sve_state.
Yes, TIF_SVE must also be cleared in dst when dst->sve_state = NULL (I
may have forgotten to mention this).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-09-20 11:09 ` Dave Martin
@ 2017-09-20 18:08 ` Alan Hayward
-1 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-20 18:08 UTC (permalink / raw)
To: Dave Martin
Cc: Dave P Martin, linux-arch, libc-alpha, gdb, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Yao Qi, Will Deacon,
Richard Sandiford, nd, Alex Bennée, kvmarm,
linux-arm-kernel
> On 20 Sep 2017, at 12:09, Dave Martin <dave.martin@foss.arm.com> wrote:
>
> On Wed, Sep 20, 2017 at 10:59:55AM +0000, Alan Hayward wrote:
>> (Resending without disclaimer)
>>
>>> On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>>
>>>
>>> +int sve_set_vector_length(struct task_struct *task,
>>> + unsigned long vl, unsigned long flags)
>>> +{
>>> + WARN_ON(task == current && preemptible());
>>> +
>>> + if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
>>> + PR_SVE_SET_VL_ONEXEC))
>>> + return -EINVAL;
>>> +
>>> + if (!sve_vl_valid(vl))
>>> + return -EINVAL;
>>> +
>>> + /*
>>> + * Clamp to the maximum vector length that VL-agnostic SVE code can
>>> + * work with. A flag may be assigned in the future to allow setting
>>> + * of larger vector lengths without confusing older software.
>>> + */
>>> + if (vl > SVE_VL_ARCH_MAX)
>>> + vl = SVE_VL_ARCH_MAX;
>>> +
>>> + vl = find_supported_vector_length(vl);
>>> +
>>
>>
>> Given, sve_set_vector_length is called when setting the vector length in
>> PTRACE_SETREGSET, it looks to me like if you set VL to a value that’s not
>> supported by the hardware, then it’s going to round down to the previous value.
>> Is that correct? I’m not sure if that’s explained in the docs?
>
> Does this cover it?
>
> "On success, the calling thread's vector length is changed to the
> largest value supported by the system that is less than or equal to vl."
>
> (For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
For ptrace is it worth mentioning user should do a GET after a SET to confirm
what VL value was actually set?
>
>> What happens if you give a vl value lower than the min supported value in the
>> hardware?
>
> This is impossible, unless vl < SVE_VL_MIN (which is rejected explicitly
> by the !sve_vl_valid() check in sve_set_vector_length()).
>
> The architecture required support for all power-of-two vector lengths
> less than the maximum supported vector length, so by construction
> SVE_VL_MIN is supported by all hardware.
Ok, I’m happy with that.
>
> To be defensive, if we fail to detect support for SVE_VL_MIN, I set the
> corresponding bit in sve_vq_map and WARN. This is just to help ensure
> find_supported_vector_length doesn't fall off the end of sve_vq_map.
>
>
> Does that sounds correct? There may be a clearer way of achieving this.
>
> Cheers
> ---Dave
>
>>
>>
>>> +/*
>>> + * All vector length selection from userspace comes through here.
>>> + * We're on a slow path, so some sanity-checks are included.
>>> + * If things go wrong there's a bug somewhere, but try to fall back to a
>>> + * safe choice.
>>> + */
>>> +static unsigned int find_supported_vector_length(unsigned int vl)
>>> +{
>>> + int bit;
>>> + int max_vl = sve_max_vl;
>>> +
>>> + if (WARN_ON(!sve_vl_valid(vl)))
>>> + vl = SVE_VL_MIN;
>>> +
>>> + if (WARN_ON(!sve_vl_valid(max_vl)))
>>> + max_vl = SVE_VL_MIN;
>>> +
>>> + if (vl > max_vl)
>>> + vl = max_vl;
>>> +
>>> + bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
>>> + vq_to_bit(sve_vq_from_vl(vl)));
>>> + return sve_vl_from_vq(bit_to_vq(bit));
>>> +}
>>> +
>>
>>
>> Thanks,
>> Alan.
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-20 18:08 ` Alan Hayward
0 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-20 18:08 UTC (permalink / raw)
To: linux-arm-kernel
> On 20 Sep 2017, at 12:09, Dave Martin <dave.martin@foss.arm.com> wrote:
>
> On Wed, Sep 20, 2017 at 10:59:55AM +0000, Alan Hayward wrote:
>> (Resending without disclaimer)
>>
>>> On 31 Aug 2017, at 18:00, Dave Martin <Dave.Martin@arm.com> wrote:
>>
>>>
>>> +int sve_set_vector_length(struct task_struct *task,
>>> + unsigned long vl, unsigned long flags)
>>> +{
>>> + WARN_ON(task == current && preemptible());
>>> +
>>> + if (flags & ~(unsigned long)(PR_SVE_VL_INHERIT |
>>> + PR_SVE_SET_VL_ONEXEC))
>>> + return -EINVAL;
>>> +
>>> + if (!sve_vl_valid(vl))
>>> + return -EINVAL;
>>> +
>>> + /*
>>> + * Clamp to the maximum vector length that VL-agnostic SVE code can
>>> + * work with. A flag may be assigned in the future to allow setting
>>> + * of larger vector lengths without confusing older software.
>>> + */
>>> + if (vl > SVE_VL_ARCH_MAX)
>>> + vl = SVE_VL_ARCH_MAX;
>>> +
>>> + vl = find_supported_vector_length(vl);
>>> +
>>
>>
>> Given, sve_set_vector_length is called when setting the vector length in
>> PTRACE_SETREGSET, it looks to me like if you set VL to a value that?s not
>> supported by the hardware, then it?s going to round down to the previous value.
>> Is that correct? I?m not sure if that?s explained in the docs?
>
> Does this cover it?
>
> "On success, the calling thread's vector length is changed to the
> largest value supported by the system that is less than or equal to vl."
>
> (For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
For ptrace is it worth mentioning user should do a GET after a SET to confirm
what VL value was actually set?
>
>> What happens if you give a vl value lower than the min supported value in the
>> hardware?
>
> This is impossible, unless vl < SVE_VL_MIN (which is rejected explicitly
> by the !sve_vl_valid() check in sve_set_vector_length()).
>
> The architecture required support for all power-of-two vector lengths
> less than the maximum supported vector length, so by construction
> SVE_VL_MIN is supported by all hardware.
Ok, I?m happy with that.
>
> To be defensive, if we fail to detect support for SVE_VL_MIN, I set the
> corresponding bit in sve_vq_map and WARN. This is just to help ensure
> find_supported_vector_length doesn't fall off the end of sve_vq_map.
>
>
> Does that sounds correct? There may be a clearer way of achieving this.
>
> Cheers
> ---Dave
>
>>
>>
>>> +/*
>>> + * All vector length selection from userspace comes through here.
>>> + * We're on a slow path, so some sanity-checks are included.
>>> + * If things go wrong there's a bug somewhere, but try to fall back to a
>>> + * safe choice.
>>> + */
>>> +static unsigned int find_supported_vector_length(unsigned int vl)
>>> +{
>>> + int bit;
>>> + int max_vl = sve_max_vl;
>>> +
>>> + if (WARN_ON(!sve_vl_valid(vl)))
>>> + vl = SVE_VL_MIN;
>>> +
>>> + if (WARN_ON(!sve_vl_valid(max_vl)))
>>> + max_vl = SVE_VL_MIN;
>>> +
>>> + if (vl > max_vl)
>>> + vl = max_vl;
>>> +
>>> + bit = find_next_bit(sve_vq_map, SVE_VQ_MAX,
>>> + vq_to_bit(sve_vq_from_vl(vl)));
>>> + return sve_vl_from_vq(bit_to_vq(bit));
>>> +}
>>> +
>>
>>
>> Thanks,
>> Alan.
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-09-20 18:08 ` Alan Hayward
@ 2017-09-21 11:19 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-21 11:19 UTC (permalink / raw)
To: Alan Hayward
Cc: Dave Martin, linux-arch, libc-alpha, gdb, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Yao Qi, Will Deacon,
Richard Sandiford, nd, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 20, 2017 at 06:08:21PM +0000, Alan Hayward wrote:
>
> > On 20 Sep 2017, at 12:09, Dave Martin <dave.martin@foss.arm.com> wrote:
[...]
> >> Given, sve_set_vector_length is called when setting the vector length in
> >> PTRACE_SETREGSET, it looks to me like if you set VL to a value that’s not
> >> supported by the hardware, then it’s going to round down to the previous value.
> >> Is that correct? I’m not sure if that’s explained in the docs?
> >
> > Does this cover it?
> >
> > "On success, the calling thread's vector length is changed to the
> > largest value supported by the system that is less than or equal to vl."
> >
> > (For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
>
> For ptrace is it worth mentioning user should do a GET after a SET to confirm
> what VL value was actually set?
This seems worth a clarification -- I'd thought this was already
mentioned, but it isn't.
How about:
The caller must make a further GETREGSET call if it needs to know what VL is
actually set by SETREGSET, unless is it known in advance that the requested
VL is supported.
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-21 11:19 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-21 11:19 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 20, 2017 at 06:08:21PM +0000, Alan Hayward wrote:
>
> > On 20 Sep 2017, at 12:09, Dave Martin <dave.martin@foss.arm.com> wrote:
[...]
> >> Given, sve_set_vector_length is called when setting the vector length in
> >> PTRACE_SETREGSET, it looks to me like if you set VL to a value that?s not
> >> supported by the hardware, then it?s going to round down to the previous value.
> >> Is that correct? I?m not sure if that?s explained in the docs?
> >
> > Does this cover it?
> >
> > "On success, the calling thread's vector length is changed to the
> > largest value supported by the system that is less than or equal to vl."
> >
> > (For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
>
> For ptrace is it worth mentioning user should do a GET after a SET to confirm
> what VL value was actually set?
This seems worth a clarification -- I'd thought this was already
mentioned, but it isn't.
How about:
The caller must make a further GETREGSET call if it needs to know what VL is
actually set by SETREGSET, unless is it known in advance that the requested
VL is supported.
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-09-21 11:19 ` Dave Martin
@ 2017-09-21 11:57 ` Alan Hayward
-1 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-21 11:57 UTC (permalink / raw)
To: Dave P Martin
Cc: Dave Martin, linux-arch, libc-alpha, gdb, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Yao Qi, Will Deacon,
Richard Sandiford, nd, Alex Bennée, kvmarm,
linux-arm-kernel
> On 21 Sep 2017, at 12:19, Dave Martin <Dave.Martin@arm.com> wrote:
>
> On Wed, Sep 20, 2017 at 06:08:21PM +0000, Alan Hayward wrote:
>>
>>> On 20 Sep 2017, at 12:09, Dave Martin <dave.martin@foss.arm.com> wrote:
>
> [...]
>
>>>> Given, sve_set_vector_length is called when setting the vector length in
>>>> PTRACE_SETREGSET, it looks to me like if you set VL to a value that’s not
>>>> supported by the hardware, then it’s going to round down to the previous value.
>>>> Is that correct? I’m not sure if that’s explained in the docs?
>>>
>>> Does this cover it?
>>>
>>> "On success, the calling thread's vector length is changed to the
>>> largest value supported by the system that is less than or equal to vl."
>>>
>>> (For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
>>
>> For ptrace is it worth mentioning user should do a GET after a SET to confirm
>> what VL value was actually set?
>
> This seems worth a clarification -- I'd thought this was already
> mentioned, but it isn't.
>
> How about:
>
> The caller must make a further GETREGSET call if it needs to know what VL is
> actually set by SETREGSET, unless is it known in advance that the requested
> VL is supported.
>
Looks good to me.
Alan.
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-09-21 11:57 ` Alan Hayward
0 siblings, 0 replies; 224+ messages in thread
From: Alan Hayward @ 2017-09-21 11:57 UTC (permalink / raw)
To: linux-arm-kernel
> On 21 Sep 2017, at 12:19, Dave Martin <Dave.Martin@arm.com> wrote:
>
> On Wed, Sep 20, 2017 at 06:08:21PM +0000, Alan Hayward wrote:
>>
>>> On 20 Sep 2017, at 12:09, Dave Martin <dave.martin@foss.arm.com> wrote:
>
> [...]
>
>>>> Given, sve_set_vector_length is called when setting the vector length in
>>>> PTRACE_SETREGSET, it looks to me like if you set VL to a value that?s not
>>>> supported by the hardware, then it?s going to round down to the previous value.
>>>> Is that correct? I?m not sure if that?s explained in the docs?
>>>
>>> Does this cover it?
>>>
>>> "On success, the calling thread's vector length is changed to the
>>> largest value supported by the system that is less than or equal to vl."
>>>
>>> (For ptrace, I just cross-reference the PR_SVE_SET_VL behaviour, above.)
>>
>> For ptrace is it worth mentioning user should do a GET after a SET to confirm
>> what VL value was actually set?
>
> This seems worth a clarification -- I'd thought this was already
> mentioned, but it isn't.
>
> How about:
>
> The caller must make a further GETREGSET call if it needs to know what VL is
> actually set by SETREGSET, unless is it known in advance that the requested
> VL is supported.
>
Looks good to me.
Alan.
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
2017-09-14 9:45 ` Alex Bennée
@ 2017-09-28 14:22 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-28 14:22 UTC (permalink / raw)
To: Alex Bennée
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Suzuki K Poulose, Will Deacon,
Richard Sandiford, kvmarm, linux-arm-kernel
On Thu, Sep 14, 2017 at 10:45:07AM +0100, Alex Bennée wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > This patch uses the cpufeatures framework to determine common SVE
> > capabilities and vector lengths, and configures the runtime SVE
> > support code appropriately.
> >
> > ZCR_ELx is not really a feature register, but it is convenient to
> > use it as a template for recording the maximum vector length
> > supported by a CPU, using the LEN field. This field is similar to
> > a feature field in that it is a contiguous bitfield for which we
> > want to determine the minimum system-wide value. This patch adds
> > ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
> > custom code to populate it. Finding the minimum supported value of
> > the LEN field is left to the cpufeatures framework in the usual
> > way.
> >
> > The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
> > so for now we just require it to be zero.
> >
> > Note that much of this code is dormant and SVE still won't be used
> > yet, since system_supports_sve() remains hardwired to false.
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Bennée <alex.bennee@linaro.org>
> > Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
> >
> > ---
> >
> > Changes since v1
> > ----------------
> >
> > Requested by Alex Bennée:
> >
> > * Thin out BUG_ON()s:
> > Redundant BUG_ON()s and ones that just check invariants are removed.
> > Important sanity-checks are migrated to WARN_ON()s, with some
> > minimal best-effort patch-up code.
> >
> > Other changes related to Alex Bennée's comments:
> >
> > * Migrate away from magic numbers for converting VL to VQ.
> >
> > Requested by Suzuki Poulose:
> >
> > * Make sve_vq_map __ro_after_init.
> >
> > Other changes related to Suzuki Poulose's comments:
> >
> > * Rely on cpufeatures for not attempting to update the vq map after boot.
> > ---
> > arch/arm64/include/asm/cpu.h | 4 ++
> > arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
> > arch/arm64/include/asm/fpsimd.h | 10 ++++
> > arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
> > arch/arm64/kernel/cpuinfo.c | 6 ++
> > arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
> > 6 files changed, 202 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
> > index 889226b..8839227 100644
> > --- a/arch/arm64/include/asm/cpu.h
> > +++ b/arch/arm64/include/asm/cpu.h
> > @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
> > u64 reg_id_aa64mmfr2;
> > u64 reg_id_aa64pfr0;
> > u64 reg_id_aa64pfr1;
> > + u64 reg_id_aa64zfr0;
> >
> > u32 reg_id_dfr0;
> > u32 reg_id_isar0;
> > @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
> > u32 reg_mvfr0;
> > u32 reg_mvfr1;
> > u32 reg_mvfr2;
> > +
> > + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
> > + u64 reg_zcr;
> > };
> >
> > DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
> > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > index 4ea3441..d98e7ba 100644
> > --- a/arch/arm64/include/asm/cpufeature.h
> > +++ b/arch/arm64/include/asm/cpufeature.h
> > @@ -10,7 +10,9 @@
> > #define __ASM_CPUFEATURE_H
> >
> > #include <asm/cpucaps.h>
> > +#include <asm/fpsimd.h>
> > #include <asm/hwcap.h>
> > +#include <asm/sigcontext.h>
> > #include <asm/sysreg.h>
> >
> > /*
> > @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
> > return val == ID_AA64PFR0_EL0_32BIT_64BIT;
> > }
> >
> > +static inline bool id_aa64pfr0_sve(u64 pfr0)
> > +{
> > + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
> > +
> > + return val > 0;
> > +}
> > +
> > void __init setup_cpu_features(void);
> >
> > void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> > @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
> > return false;
> > }
> >
> > +/*
> > + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> > + * vector length.
> > + * Use only if SVE is present. This function clobbers the SVE vector length.
> > + */
>
> :nit whitespace formatting.
I'll add some newlines now to make this cleaner.
/*
* Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
* vector length.
*
* Use only if SVE is present.
* This function clobbers the SVE vector length.
*/
OK?
>
> > +static u64 __maybe_unused read_zcr_features(void)
> > +{
> > + u64 zcr;
> > + unsigned int vq_max;
> > +
> > + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>
> I'm confused, why are we writing something here? You mention clobbering
> the SVE vector length but what was the point?
Hmm, this deserves a comment -- coming back to this code, I had to think
about it. Are the following extra comments sufficient explanation?
/*
* Set the maximum possible VL, and write zeroes to all other
* bits to see if they stick.
*/
write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
zcr = read_sysreg_s(SYS_ZCR_EL1);
zcr &= ~(u64)ZCR_ELx_LEN_MASK; /* flag up sticky 1s outside LEN field */
vq_max = sve_vq_from_vl(sve_get_vl());
zcr |= vq_max - 1; /* set LEN field to maximum effective value */
[...]
> Otherwise:
>
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
I'll wait on your responses to the above first.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-09-28 14:22 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-28 14:22 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Sep 14, 2017 at 10:45:07AM +0100, Alex Benn?e wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > This patch uses the cpufeatures framework to determine common SVE
> > capabilities and vector lengths, and configures the runtime SVE
> > support code appropriately.
> >
> > ZCR_ELx is not really a feature register, but it is convenient to
> > use it as a template for recording the maximum vector length
> > supported by a CPU, using the LEN field. This field is similar to
> > a feature field in that it is a contiguous bitfield for which we
> > want to determine the minimum system-wide value. This patch adds
> > ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
> > custom code to populate it. Finding the minimum supported value of
> > the LEN field is left to the cpufeatures framework in the usual
> > way.
> >
> > The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
> > so for now we just require it to be zero.
> >
> > Note that much of this code is dormant and SVE still won't be used
> > yet, since system_supports_sve() remains hardwired to false.
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Benn?e <alex.bennee@linaro.org>
> > Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
> >
> > ---
> >
> > Changes since v1
> > ----------------
> >
> > Requested by Alex Benn?e:
> >
> > * Thin out BUG_ON()s:
> > Redundant BUG_ON()s and ones that just check invariants are removed.
> > Important sanity-checks are migrated to WARN_ON()s, with some
> > minimal best-effort patch-up code.
> >
> > Other changes related to Alex Benn?e's comments:
> >
> > * Migrate away from magic numbers for converting VL to VQ.
> >
> > Requested by Suzuki Poulose:
> >
> > * Make sve_vq_map __ro_after_init.
> >
> > Other changes related to Suzuki Poulose's comments:
> >
> > * Rely on cpufeatures for not attempting to update the vq map after boot.
> > ---
> > arch/arm64/include/asm/cpu.h | 4 ++
> > arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
> > arch/arm64/include/asm/fpsimd.h | 10 ++++
> > arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
> > arch/arm64/kernel/cpuinfo.c | 6 ++
> > arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
> > 6 files changed, 202 insertions(+), 3 deletions(-)
> >
> > diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
> > index 889226b..8839227 100644
> > --- a/arch/arm64/include/asm/cpu.h
> > +++ b/arch/arm64/include/asm/cpu.h
> > @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
> > u64 reg_id_aa64mmfr2;
> > u64 reg_id_aa64pfr0;
> > u64 reg_id_aa64pfr1;
> > + u64 reg_id_aa64zfr0;
> >
> > u32 reg_id_dfr0;
> > u32 reg_id_isar0;
> > @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
> > u32 reg_mvfr0;
> > u32 reg_mvfr1;
> > u32 reg_mvfr2;
> > +
> > + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
> > + u64 reg_zcr;
> > };
> >
> > DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
> > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
> > index 4ea3441..d98e7ba 100644
> > --- a/arch/arm64/include/asm/cpufeature.h
> > +++ b/arch/arm64/include/asm/cpufeature.h
> > @@ -10,7 +10,9 @@
> > #define __ASM_CPUFEATURE_H
> >
> > #include <asm/cpucaps.h>
> > +#include <asm/fpsimd.h>
> > #include <asm/hwcap.h>
> > +#include <asm/sigcontext.h>
> > #include <asm/sysreg.h>
> >
> > /*
> > @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
> > return val == ID_AA64PFR0_EL0_32BIT_64BIT;
> > }
> >
> > +static inline bool id_aa64pfr0_sve(u64 pfr0)
> > +{
> > + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
> > +
> > + return val > 0;
> > +}
> > +
> > void __init setup_cpu_features(void);
> >
> > void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
> > @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
> > return false;
> > }
> >
> > +/*
> > + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> > + * vector length.
> > + * Use only if SVE is present. This function clobbers the SVE vector length.
> > + */
>
> :nit whitespace formatting.
I'll add some newlines now to make this cleaner.
/*
* Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
* vector length.
*
* Use only if SVE is present.
* This function clobbers the SVE vector length.
*/
OK?
>
> > +static u64 __maybe_unused read_zcr_features(void)
> > +{
> > + u64 zcr;
> > + unsigned int vq_max;
> > +
> > + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>
> I'm confused, why are we writing something here? You mention clobbering
> the SVE vector length but what was the point?
Hmm, this deserves a comment -- coming back to this code, I had to think
about it. Are the following extra comments sufficient explanation?
/*
* Set the maximum possible VL, and write zeroes to all other
* bits to see if they stick.
*/
write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
zcr = read_sysreg_s(SYS_ZCR_EL1);
zcr &= ~(u64)ZCR_ELx_LEN_MASK; /* flag up sticky 1s outside LEN field */
vq_max = sve_vq_from_vl(sve_get_vl());
zcr |= vq_max - 1; /* set LEN field to maximum effective value */
[...]
> Otherwise:
>
> Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
I'll wait on your responses to the above first.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
2017-09-14 12:57 ` Alex Bennée
@ 2017-09-28 14:57 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-28 14:57 UTC (permalink / raw)
To: Alex Bennée
Cc: linux-arch, libc-alpha, gdb, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Yao Qi, Alan Hayward, Will Deacon,
Oleg Nesterov, Alexander Viro, Richard Sandiford, kvmarm,
linux-arm-kernel
On Thu, Sep 14, 2017 at 01:57:08PM +0100, Alex Bennée wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > This patch defines and implements a new regset NT_ARM_SVE, which
> > describes a thread's SVE register state. This allows a debugger to
> > manipulate the SVE state, as well as being included in ELF
> > coredumps for post-mortem debugging.
> >
> > Because the regset size and layout are dependent on the thread's
> > current vector length, it is not possible to define a C struct to
> > describe the regset contents as is done for existing regsets.
> > Instead, and for the same reasons, NT_ARM_SVE is based on the
> > freeform variable-layout approach used for the SVE signal frame.
> >
> > Additionally, to reduce debug overhead when debugging threads that
> > might or might not have live SVE register state, NT_ARM_SVE may be
> > presented in one of two different formats: the old struct
> > user_fpsimd_state format is embedded for describing the state of a
> > thread with no live SVE state, whereas a new variable-layout
> > structure is embedded for describing live SVE state. This avoids a
> > debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
> > allows existing userspace code to handle the non-SVE case without
> > too much modification.
> >
> > For this to work, NT_ARM_SVE is defined with a fixed-format header
> > of type struct user_sve_header, which the recipient can use to
> > figure out the content, size and layout of the reset of the regset.
> > Accessor macros are defined to allow the vector-length-dependent
> > parts of the regset to be manipulated.
> >
> > Signed-off-by: Alan Hayward <alan.hayward@arm.com>
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Bennée <alex.bennee@linaro.org>
> >
> > ---
> >
> > Changes since v1
> > ----------------
> >
> > Other changes related to Alex Bennée's comments:
> >
> > * Migrate to SVE_VQ_BYTES instead of magic numbers.
> >
> > Requested by Alex Bennée:
> >
> > * Thin out BUG_ON()s:
> > Redundant BUG_ON()s and ones that just check invariants are removed.
> > Important sanity-checks are migrated to WARN_ON()s, with some
> > minimal best-effort patch-up code.
> >
> > Other:
> >
> > * [ABI fix] Bail out with -EIO if attempting to set the
> > SVE regs for an unsupported VL, instead of misparsing the regset data.
> >
> > * Replace some in-kernel open-coded arithmetic with ALIGN()/
> > DIV_ROUND_UP().
> > ---
[...]
> > diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
> > index d1ff83d..1915ab0 100644
> > --- a/arch/arm64/include/uapi/asm/ptrace.h
> > +++ b/arch/arm64/include/uapi/asm/ptrace.h
> > @@ -22,6 +22,7 @@
> > #include <linux/types.h>
> >
> > #include <asm/hwcap.h>
> > +#include <asm/sigcontext.h>
> >
> >
> > /*
> > @@ -63,6 +64,8 @@
> >
> > #ifndef __ASSEMBLY__
> >
> > +#include <linux/prctl.h>
> > +
> > /*
> > * User structures for general purpose, floating point and debug registers.
> > */
> > @@ -90,6 +93,138 @@ struct user_hwdebug_state {
> > } dbg_regs[16];
> > };
> >
> > +/* SVE/FP/SIMD state (NT_ARM_SVE) */
> > +
> > +struct user_sve_header {
> > + __u32 size; /* total meaningful regset content in bytes */
> > + __u32 max_size; /* maxmium possible size for this thread */
> > + __u16 vl; /* current vector length */
> > + __u16 max_vl; /* maximum possible vector length */
> > + __u16 flags;
> > + __u16 __reserved;
> > +};
> > +
> > +/* Definitions for user_sve_header.flags: */
> > +#define SVE_PT_REGS_MASK (1 << 0)
> > +
> > +/* Flags: must be kept in sync with prctl interface in
> > <linux/ptrace.h> */
>
> Which flags? We base some on PR_foo flags but we seem to shift them
All the prctl flags that have equivalents here, because they're part of
the internal API to sve_set_vector_length(). It didn't quite seem
appropriate to document that in a userspace header, but it's probably
better to say something here than not. I'll improve the comment.
> anyway so where is the requirement for them to match from?
There is a bug here though: sve_set() in ptrace.c is supposed to shift
the flags from header.flags (which is a u16) back into the
PR_SVE_SET_VL position (<< 16) for the flags argument of
sve_set_vector_length(). But this isn't done, so attempting to set (or
restore) those flags through ptrace may resulting EINVALs from
sve_set_vector_length().
I'll write a test for this case and implement a fix, something like...
-8<-
static int sve_set(struct task_struct *target,
[...]
ret = sve_set_vector_length(target, header.vl,
- header.flags & ~SVE_PT_REGS_MASK);
+ (header.flags & ~SVE_PT_REGS_MASK) << 16UL);
->8-
What do you think?
[...]
> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Again, I'll wait for your feedback first.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-28 14:57 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-28 14:57 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Sep 14, 2017 at 01:57:08PM +0100, Alex Benn?e wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > This patch defines and implements a new regset NT_ARM_SVE, which
> > describes a thread's SVE register state. This allows a debugger to
> > manipulate the SVE state, as well as being included in ELF
> > coredumps for post-mortem debugging.
> >
> > Because the regset size and layout are dependent on the thread's
> > current vector length, it is not possible to define a C struct to
> > describe the regset contents as is done for existing regsets.
> > Instead, and for the same reasons, NT_ARM_SVE is based on the
> > freeform variable-layout approach used for the SVE signal frame.
> >
> > Additionally, to reduce debug overhead when debugging threads that
> > might or might not have live SVE register state, NT_ARM_SVE may be
> > presented in one of two different formats: the old struct
> > user_fpsimd_state format is embedded for describing the state of a
> > thread with no live SVE state, whereas a new variable-layout
> > structure is embedded for describing live SVE state. This avoids a
> > debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
> > allows existing userspace code to handle the non-SVE case without
> > too much modification.
> >
> > For this to work, NT_ARM_SVE is defined with a fixed-format header
> > of type struct user_sve_header, which the recipient can use to
> > figure out the content, size and layout of the reset of the regset.
> > Accessor macros are defined to allow the vector-length-dependent
> > parts of the regset to be manipulated.
> >
> > Signed-off-by: Alan Hayward <alan.hayward@arm.com>
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Benn?e <alex.bennee@linaro.org>
> >
> > ---
> >
> > Changes since v1
> > ----------------
> >
> > Other changes related to Alex Benn?e's comments:
> >
> > * Migrate to SVE_VQ_BYTES instead of magic numbers.
> >
> > Requested by Alex Benn?e:
> >
> > * Thin out BUG_ON()s:
> > Redundant BUG_ON()s and ones that just check invariants are removed.
> > Important sanity-checks are migrated to WARN_ON()s, with some
> > minimal best-effort patch-up code.
> >
> > Other:
> >
> > * [ABI fix] Bail out with -EIO if attempting to set the
> > SVE regs for an unsupported VL, instead of misparsing the regset data.
> >
> > * Replace some in-kernel open-coded arithmetic with ALIGN()/
> > DIV_ROUND_UP().
> > ---
[...]
> > diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
> > index d1ff83d..1915ab0 100644
> > --- a/arch/arm64/include/uapi/asm/ptrace.h
> > +++ b/arch/arm64/include/uapi/asm/ptrace.h
> > @@ -22,6 +22,7 @@
> > #include <linux/types.h>
> >
> > #include <asm/hwcap.h>
> > +#include <asm/sigcontext.h>
> >
> >
> > /*
> > @@ -63,6 +64,8 @@
> >
> > #ifndef __ASSEMBLY__
> >
> > +#include <linux/prctl.h>
> > +
> > /*
> > * User structures for general purpose, floating point and debug registers.
> > */
> > @@ -90,6 +93,138 @@ struct user_hwdebug_state {
> > } dbg_regs[16];
> > };
> >
> > +/* SVE/FP/SIMD state (NT_ARM_SVE) */
> > +
> > +struct user_sve_header {
> > + __u32 size; /* total meaningful regset content in bytes */
> > + __u32 max_size; /* maxmium possible size for this thread */
> > + __u16 vl; /* current vector length */
> > + __u16 max_vl; /* maximum possible vector length */
> > + __u16 flags;
> > + __u16 __reserved;
> > +};
> > +
> > +/* Definitions for user_sve_header.flags: */
> > +#define SVE_PT_REGS_MASK (1 << 0)
> > +
> > +/* Flags: must be kept in sync with prctl interface in
> > <linux/ptrace.h> */
>
> Which flags? We base some on PR_foo flags but we seem to shift them
All the prctl flags that have equivalents here, because they're part of
the internal API to sve_set_vector_length(). It didn't quite seem
appropriate to document that in a userspace header, but it's probably
better to say something here than not. I'll improve the comment.
> anyway so where is the requirement for them to match from?
There is a bug here though: sve_set() in ptrace.c is supposed to shift
the flags from header.flags (which is a u16) back into the
PR_SVE_SET_VL position (<< 16) for the flags argument of
sve_set_vector_length(). But this isn't done, so attempting to set (or
restore) those flags through ptrace may resulting EINVALs from
sve_set_vector_length().
I'll write a test for this case and implement a fix, something like...
-8<-
static int sve_set(struct task_struct *target,
[...]
ret = sve_set_vector_length(target, header.vl,
- header.flags & ~SVE_PT_REGS_MASK);
+ (header.flags & ~SVE_PT_REGS_MASK) << 16UL);
->8-
What do you think?
[...]
> Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
Again, I'll wait for your feedback first.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-09-28 17:32 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-28 17:32 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Suzuki K Poulose, Will Deacon,
Richard Sandiford, kvmarm, linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Thu, Sep 14, 2017 at 10:45:07AM +0100, Alex Bennée wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > This patch uses the cpufeatures framework to determine common SVE
>> > capabilities and vector lengths, and configures the runtime SVE
>> > support code appropriately.
>> >
>> > ZCR_ELx is not really a feature register, but it is convenient to
>> > use it as a template for recording the maximum vector length
>> > supported by a CPU, using the LEN field. This field is similar to
>> > a feature field in that it is a contiguous bitfield for which we
>> > want to determine the minimum system-wide value. This patch adds
>> > ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
>> > custom code to populate it. Finding the minimum supported value of
>> > the LEN field is left to the cpufeatures framework in the usual
>> > way.
>> >
>> > The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
>> > so for now we just require it to be zero.
>> >
>> > Note that much of this code is dormant and SVE still won't be used
>> > yet, since system_supports_sve() remains hardwired to false.
>> >
>> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>> > Cc: Alex Bennée <alex.bennee@linaro.org>
>> > Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
>> >
>> > ---
>> >
>> > Changes since v1
>> > ----------------
>> >
>> > Requested by Alex Bennée:
>> >
>> > * Thin out BUG_ON()s:
>> > Redundant BUG_ON()s and ones that just check invariants are removed.
>> > Important sanity-checks are migrated to WARN_ON()s, with some
>> > minimal best-effort patch-up code.
>> >
>> > Other changes related to Alex Bennée's comments:
>> >
>> > * Migrate away from magic numbers for converting VL to VQ.
>> >
>> > Requested by Suzuki Poulose:
>> >
>> > * Make sve_vq_map __ro_after_init.
>> >
>> > Other changes related to Suzuki Poulose's comments:
>> >
>> > * Rely on cpufeatures for not attempting to update the vq map after boot.
>> > ---
>> > arch/arm64/include/asm/cpu.h | 4 ++
>> > arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
>> > arch/arm64/include/asm/fpsimd.h | 10 ++++
>> > arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
>> > arch/arm64/kernel/cpuinfo.c | 6 ++
>> > arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
>> > 6 files changed, 202 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
>> > index 889226b..8839227 100644
>> > --- a/arch/arm64/include/asm/cpu.h
>> > +++ b/arch/arm64/include/asm/cpu.h
>> > @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
>> > u64 reg_id_aa64mmfr2;
>> > u64 reg_id_aa64pfr0;
>> > u64 reg_id_aa64pfr1;
>> > + u64 reg_id_aa64zfr0;
>> >
>> > u32 reg_id_dfr0;
>> > u32 reg_id_isar0;
>> > @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
>> > u32 reg_mvfr0;
>> > u32 reg_mvfr1;
>> > u32 reg_mvfr2;
>> > +
>> > + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
>> > + u64 reg_zcr;
>> > };
>> >
>> > DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
>> > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>> > index 4ea3441..d98e7ba 100644
>> > --- a/arch/arm64/include/asm/cpufeature.h
>> > +++ b/arch/arm64/include/asm/cpufeature.h
>> > @@ -10,7 +10,9 @@
>> > #define __ASM_CPUFEATURE_H
>> >
>> > #include <asm/cpucaps.h>
>> > +#include <asm/fpsimd.h>
>> > #include <asm/hwcap.h>
>> > +#include <asm/sigcontext.h>
>> > #include <asm/sysreg.h>
>> >
>> > /*
>> > @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
>> > return val == ID_AA64PFR0_EL0_32BIT_64BIT;
>> > }
>> >
>> > +static inline bool id_aa64pfr0_sve(u64 pfr0)
>> > +{
>> > + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
>> > +
>> > + return val > 0;
>> > +}
>> > +
>> > void __init setup_cpu_features(void);
>> >
>> > void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
>> > @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
>> > return false;
>> > }
>> >
>> > +/*
>> > + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
>> > + * vector length.
>> > + * Use only if SVE is present. This function clobbers the SVE vector length.
>> > + */
>>
>> :nit whitespace formatting.
>
> I'll add some newlines now to make this cleaner.
>
> /*
> * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> * vector length.
> *
> * Use only if SVE is present.
> * This function clobbers the SVE vector length.
> */
>
> OK?
Yep.
>
>>
>> > +static u64 __maybe_unused read_zcr_features(void)
>> > +{
>> > + u64 zcr;
>> > + unsigned int vq_max;
>> > +
>> > + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>>
>> I'm confused, why are we writing something here? You mention clobbering
>> the SVE vector length but what was the point?
>
> Hmm, this deserves a comment -- coming back to this code, I had to think
> about it. Are the following extra comments sufficient explanation?
>
> /*
> * Set the maximum possible VL, and write zeroes to all other
> * bits to see if they stick.
> */
> write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>
> zcr = read_sysreg_s(SYS_ZCR_EL1);
> zcr &= ~(u64)ZCR_ELx_LEN_MASK; /* flag up sticky 1s outside LEN field */
> vq_max = sve_vq_from_vl(sve_get_vl());
> zcr |= vq_max - 1; /* set LEN field to maximum effective value */
>
>
> [...]
OK that makes more sense. Thanks.
>
>> Otherwise:
>>
>> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> I'll wait on your responses to the above first.
Still good ;-)
>
> Cheers
> ---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-09-28 17:32 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-28 17:32 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Suzuki K Poulose, Will Deacon,
Richard Sandiford, kvmarm, linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Thu, Sep 14, 2017 at 10:45:07AM +0100, Alex Bennée wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > This patch uses the cpufeatures framework to determine common SVE
>> > capabilities and vector lengths, and configures the runtime SVE
>> > support code appropriately.
>> >
>> > ZCR_ELx is not really a feature register, but it is convenient to
>> > use it as a template for recording the maximum vector length
>> > supported by a CPU, using the LEN field. This field is similar to
>> > a feature field in that it is a contiguous bitfield for which we
>> > want to determine the minimum system-wide value. This patch adds
>> > ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
>> > custom code to populate it. Finding the minimum supported value of
>> > the LEN field is left to the cpufeatures framework in the usual
>> > way.
>> >
>> > The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
>> > so for now we just require it to be zero.
>> >
>> > Note that much of this code is dormant and SVE still won't be used
>> > yet, since system_supports_sve() remains hardwired to false.
>> >
>> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>> > Cc: Alex Bennée <alex.bennee@linaro.org>
>> > Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
>> >
>> > ---
>> >
>> > Changes since v1
>> > ----------------
>> >
>> > Requested by Alex Bennée:
>> >
>> > * Thin out BUG_ON()s:
>> > Redundant BUG_ON()s and ones that just check invariants are removed.
>> > Important sanity-checks are migrated to WARN_ON()s, with some
>> > minimal best-effort patch-up code.
>> >
>> > Other changes related to Alex Bennée's comments:
>> >
>> > * Migrate away from magic numbers for converting VL to VQ.
>> >
>> > Requested by Suzuki Poulose:
>> >
>> > * Make sve_vq_map __ro_after_init.
>> >
>> > Other changes related to Suzuki Poulose's comments:
>> >
>> > * Rely on cpufeatures for not attempting to update the vq map after boot.
>> > ---
>> > arch/arm64/include/asm/cpu.h | 4 ++
>> > arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
>> > arch/arm64/include/asm/fpsimd.h | 10 ++++
>> > arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
>> > arch/arm64/kernel/cpuinfo.c | 6 ++
>> > arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
>> > 6 files changed, 202 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
>> > index 889226b..8839227 100644
>> > --- a/arch/arm64/include/asm/cpu.h
>> > +++ b/arch/arm64/include/asm/cpu.h
>> > @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
>> > u64 reg_id_aa64mmfr2;
>> > u64 reg_id_aa64pfr0;
>> > u64 reg_id_aa64pfr1;
>> > + u64 reg_id_aa64zfr0;
>> >
>> > u32 reg_id_dfr0;
>> > u32 reg_id_isar0;
>> > @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
>> > u32 reg_mvfr0;
>> > u32 reg_mvfr1;
>> > u32 reg_mvfr2;
>> > +
>> > + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
>> > + u64 reg_zcr;
>> > };
>> >
>> > DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
>> > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>> > index 4ea3441..d98e7ba 100644
>> > --- a/arch/arm64/include/asm/cpufeature.h
>> > +++ b/arch/arm64/include/asm/cpufeature.h
>> > @@ -10,7 +10,9 @@
>> > #define __ASM_CPUFEATURE_H
>> >
>> > #include <asm/cpucaps.h>
>> > +#include <asm/fpsimd.h>
>> > #include <asm/hwcap.h>
>> > +#include <asm/sigcontext.h>
>> > #include <asm/sysreg.h>
>> >
>> > /*
>> > @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
>> > return val == ID_AA64PFR0_EL0_32BIT_64BIT;
>> > }
>> >
>> > +static inline bool id_aa64pfr0_sve(u64 pfr0)
>> > +{
>> > + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
>> > +
>> > + return val > 0;
>> > +}
>> > +
>> > void __init setup_cpu_features(void);
>> >
>> > void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
>> > @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
>> > return false;
>> > }
>> >
>> > +/*
>> > + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
>> > + * vector length.
>> > + * Use only if SVE is present. This function clobbers the SVE vector length.
>> > + */
>>
>> :nit whitespace formatting.
>
> I'll add some newlines now to make this cleaner.
>
> /*
> * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> * vector length.
> *
> * Use only if SVE is present.
> * This function clobbers the SVE vector length.
> */
>
> OK?
Yep.
>
>>
>> > +static u64 __maybe_unused read_zcr_features(void)
>> > +{
>> > + u64 zcr;
>> > + unsigned int vq_max;
>> > +
>> > + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>>
>> I'm confused, why are we writing something here? You mention clobbering
>> the SVE vector length but what was the point?
>
> Hmm, this deserves a comment -- coming back to this code, I had to think
> about it. Are the following extra comments sufficient explanation?
>
> /*
> * Set the maximum possible VL, and write zeroes to all other
> * bits to see if they stick.
> */
> write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>
> zcr = read_sysreg_s(SYS_ZCR_EL1);
> zcr &= ~(u64)ZCR_ELx_LEN_MASK; /* flag up sticky 1s outside LEN field */
> vq_max = sve_vq_from_vl(sve_get_vl());
> zcr |= vq_max - 1; /* set LEN field to maximum effective value */
>
>
> [...]
OK that makes more sense. Thanks.
>
>> Otherwise:
>>
>> Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
>
> I'll wait on your responses to the above first.
Still good ;-)
>
> Cheers
> ---Dave
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths
@ 2017-09-28 17:32 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-28 17:32 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Thu, Sep 14, 2017 at 10:45:07AM +0100, Alex Benn?e wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > This patch uses the cpufeatures framework to determine common SVE
>> > capabilities and vector lengths, and configures the runtime SVE
>> > support code appropriately.
>> >
>> > ZCR_ELx is not really a feature register, but it is convenient to
>> > use it as a template for recording the maximum vector length
>> > supported by a CPU, using the LEN field. This field is similar to
>> > a feature field in that it is a contiguous bitfield for which we
>> > want to determine the minimum system-wide value. This patch adds
>> > ZCR as a pseudo-register in cpuinfo/cpufeatures, with appropriate
>> > custom code to populate it. Finding the minimum supported value of
>> > the LEN field is left to the cpufeatures framework in the usual
>> > way.
>> >
>> > The meaning of ID_AA64ZFR0_EL1 is not architecturally defined yet,
>> > so for now we just require it to be zero.
>> >
>> > Note that much of this code is dormant and SVE still won't be used
>> > yet, since system_supports_sve() remains hardwired to false.
>> >
>> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>> > Cc: Alex Benn?e <alex.bennee@linaro.org>
>> > Cc: Suzuki K Poulose <Suzuki.Poulose@arm.com>
>> >
>> > ---
>> >
>> > Changes since v1
>> > ----------------
>> >
>> > Requested by Alex Benn?e:
>> >
>> > * Thin out BUG_ON()s:
>> > Redundant BUG_ON()s and ones that just check invariants are removed.
>> > Important sanity-checks are migrated to WARN_ON()s, with some
>> > minimal best-effort patch-up code.
>> >
>> > Other changes related to Alex Benn?e's comments:
>> >
>> > * Migrate away from magic numbers for converting VL to VQ.
>> >
>> > Requested by Suzuki Poulose:
>> >
>> > * Make sve_vq_map __ro_after_init.
>> >
>> > Other changes related to Suzuki Poulose's comments:
>> >
>> > * Rely on cpufeatures for not attempting to update the vq map after boot.
>> > ---
>> > arch/arm64/include/asm/cpu.h | 4 ++
>> > arch/arm64/include/asm/cpufeature.h | 29 ++++++++++
>> > arch/arm64/include/asm/fpsimd.h | 10 ++++
>> > arch/arm64/kernel/cpufeature.c | 50 +++++++++++++++++
>> > arch/arm64/kernel/cpuinfo.c | 6 ++
>> > arch/arm64/kernel/fpsimd.c | 106 +++++++++++++++++++++++++++++++++++-
>> > 6 files changed, 202 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/arch/arm64/include/asm/cpu.h b/arch/arm64/include/asm/cpu.h
>> > index 889226b..8839227 100644
>> > --- a/arch/arm64/include/asm/cpu.h
>> > +++ b/arch/arm64/include/asm/cpu.h
>> > @@ -41,6 +41,7 @@ struct cpuinfo_arm64 {
>> > u64 reg_id_aa64mmfr2;
>> > u64 reg_id_aa64pfr0;
>> > u64 reg_id_aa64pfr1;
>> > + u64 reg_id_aa64zfr0;
>> >
>> > u32 reg_id_dfr0;
>> > u32 reg_id_isar0;
>> > @@ -59,6 +60,9 @@ struct cpuinfo_arm64 {
>> > u32 reg_mvfr0;
>> > u32 reg_mvfr1;
>> > u32 reg_mvfr2;
>> > +
>> > + /* pseudo-ZCR for recording maximum ZCR_EL1 LEN value: */
>> > + u64 reg_zcr;
>> > };
>> >
>> > DECLARE_PER_CPU(struct cpuinfo_arm64, cpu_data);
>> > diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
>> > index 4ea3441..d98e7ba 100644
>> > --- a/arch/arm64/include/asm/cpufeature.h
>> > +++ b/arch/arm64/include/asm/cpufeature.h
>> > @@ -10,7 +10,9 @@
>> > #define __ASM_CPUFEATURE_H
>> >
>> > #include <asm/cpucaps.h>
>> > +#include <asm/fpsimd.h>
>> > #include <asm/hwcap.h>
>> > +#include <asm/sigcontext.h>
>> > #include <asm/sysreg.h>
>> >
>> > /*
>> > @@ -223,6 +225,13 @@ static inline bool id_aa64pfr0_32bit_el0(u64 pfr0)
>> > return val == ID_AA64PFR0_EL0_32BIT_64BIT;
>> > }
>> >
>> > +static inline bool id_aa64pfr0_sve(u64 pfr0)
>> > +{
>> > + u32 val = cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_SVE_SHIFT);
>> > +
>> > + return val > 0;
>> > +}
>> > +
>> > void __init setup_cpu_features(void);
>> >
>> > void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
>> > @@ -267,6 +276,26 @@ static inline bool system_supports_sve(void)
>> > return false;
>> > }
>> >
>> > +/*
>> > + * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
>> > + * vector length.
>> > + * Use only if SVE is present. This function clobbers the SVE vector length.
>> > + */
>>
>> :nit whitespace formatting.
>
> I'll add some newlines now to make this cleaner.
>
> /*
> * Read the pseudo-ZCR used by cpufeatures to identify the supported SVE
> * vector length.
> *
> * Use only if SVE is present.
> * This function clobbers the SVE vector length.
> */
>
> OK?
Yep.
>
>>
>> > +static u64 __maybe_unused read_zcr_features(void)
>> > +{
>> > + u64 zcr;
>> > + unsigned int vq_max;
>> > +
>> > + write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>>
>> I'm confused, why are we writing something here? You mention clobbering
>> the SVE vector length but what was the point?
>
> Hmm, this deserves a comment -- coming back to this code, I had to think
> about it. Are the following extra comments sufficient explanation?
>
> /*
> * Set the maximum possible VL, and write zeroes to all other
> * bits to see if they stick.
> */
> write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL1);
>
> zcr = read_sysreg_s(SYS_ZCR_EL1);
> zcr &= ~(u64)ZCR_ELx_LEN_MASK; /* flag up sticky 1s outside LEN field */
> vq_max = sve_vq_from_vl(sve_get_vl());
> zcr |= vq_max - 1; /* set LEN field to maximum effective value */
>
>
> [...]
OK that makes more sense. Thanks.
>
>> Otherwise:
>>
>> Reviewed-by: Alex Benn?e <alex.bennee@linaro.org>
>
> I'll wait on your responses to the above first.
Still good ;-)
>
> Cheers
> ---Dave
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
2017-09-14 12:57 ` Alex Bennée
@ 2017-09-29 12:46 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-29 12:46 UTC (permalink / raw)
To: Alex Bennée
Cc: linux-arch, libc-alpha, gdb, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Yao Qi, Alan Hayward, Will Deacon,
Oleg Nesterov, Alexander Viro, Richard Sandiford, kvmarm,
linux-arm-kernel
On Thu, Sep 14, 2017 at 01:57:08PM +0100, Alex Bennée wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > This patch defines and implements a new regset NT_ARM_SVE, which
> > describes a thread's SVE register state. This allows a debugger to
> > manipulate the SVE state, as well as being included in ELF
> > coredumps for post-mortem debugging.
> >
> > Because the regset size and layout are dependent on the thread's
> > current vector length, it is not possible to define a C struct to
> > describe the regset contents as is done for existing regsets.
> > Instead, and for the same reasons, NT_ARM_SVE is based on the
> > freeform variable-layout approach used for the SVE signal frame.
> >
> > Additionally, to reduce debug overhead when debugging threads that
> > might or might not have live SVE register state, NT_ARM_SVE may be
> > presented in one of two different formats: the old struct
> > user_fpsimd_state format is embedded for describing the state of a
> > thread with no live SVE state, whereas a new variable-layout
> > structure is embedded for describing live SVE state. This avoids a
> > debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
> > allows existing userspace code to handle the non-SVE case without
> > too much modification.
> >
> > For this to work, NT_ARM_SVE is defined with a fixed-format header
> > of type struct user_sve_header, which the recipient can use to
> > figure out the content, size and layout of the reset of the regset.
> > Accessor macros are defined to allow the vector-length-dependent
> > parts of the regset to be manipulated.
> >
> > Signed-off-by: Alan Hayward <alan.hayward@arm.com>
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Bennée <alex.bennee@linaro.org>
> >
> > ---
> >
> > Changes since v1
> > ----------------
> >
> > Other changes related to Alex Bennée's comments:
> >
> > * Migrate to SVE_VQ_BYTES instead of magic numbers.
> >
> > Requested by Alex Bennée:
> >
> > * Thin out BUG_ON()s:
> > Redundant BUG_ON()s and ones that just check invariants are removed.
> > Important sanity-checks are migrated to WARN_ON()s, with some
> > minimal best-effort patch-up code.
> >
> > Other:
> >
> > * [ABI fix] Bail out with -EIO if attempting to set the
> > SVE regs for an unsupported VL, instead of misparsing the regset data.
> >
> > * Replace some in-kernel open-coded arithmetic with ALIGN()/
> > DIV_ROUND_UP().
> > ---
[...]
> > diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
[...]
> > +/* Definitions for user_sve_header.flags: */
> > +#define SVE_PT_REGS_MASK (1 << 0)
> > +
> > +/* Flags: must be kept in sync with prctl interface in
> > <linux/ptrace.h> */
>
> Which flags? We base some on PR_foo flags but we seem to shift them
> anyway so where is the requirement for them to match from?
I've rearranged this as:
-8<-
/* Definitions for user_sve_header.flags: */
#define SVE_PT_REGS_MASK (1 << 0)
#define SVE_PT_REGS_FPSIMD 0
#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
/*
* Common SVE_PT_* flags:
* These must be kept in sync with prctl interface in <linux/ptrace.h>
*/
#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
->8-
This avoids the suggestion that SVE_PT_REGS_{MASK,FPSIMD,SVE} are
supposed to have prctl counterparts.
I don't really want to write more, in case it is misinterpreted as
specification of behaviour.
This comment is really only meant as a reminder to maintainers that
they should go look at prctl.h too, before blindly making changes
here.
Any good? If you have a different suggestion, I'm all ears...
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support
@ 2017-09-29 12:46 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-29 12:46 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Sep 14, 2017 at 01:57:08PM +0100, Alex Benn?e wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > This patch defines and implements a new regset NT_ARM_SVE, which
> > describes a thread's SVE register state. This allows a debugger to
> > manipulate the SVE state, as well as being included in ELF
> > coredumps for post-mortem debugging.
> >
> > Because the regset size and layout are dependent on the thread's
> > current vector length, it is not possible to define a C struct to
> > describe the regset contents as is done for existing regsets.
> > Instead, and for the same reasons, NT_ARM_SVE is based on the
> > freeform variable-layout approach used for the SVE signal frame.
> >
> > Additionally, to reduce debug overhead when debugging threads that
> > might or might not have live SVE register state, NT_ARM_SVE may be
> > presented in one of two different formats: the old struct
> > user_fpsimd_state format is embedded for describing the state of a
> > thread with no live SVE state, whereas a new variable-layout
> > structure is embedded for describing live SVE state. This avoids a
> > debugger needing to poll NT_PRFPREG in addition to NT_ARM_SVE, and
> > allows existing userspace code to handle the non-SVE case without
> > too much modification.
> >
> > For this to work, NT_ARM_SVE is defined with a fixed-format header
> > of type struct user_sve_header, which the recipient can use to
> > figure out the content, size and layout of the reset of the regset.
> > Accessor macros are defined to allow the vector-length-dependent
> > parts of the regset to be manipulated.
> >
> > Signed-off-by: Alan Hayward <alan.hayward@arm.com>
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > Cc: Alex Benn?e <alex.bennee@linaro.org>
> >
> > ---
> >
> > Changes since v1
> > ----------------
> >
> > Other changes related to Alex Benn?e's comments:
> >
> > * Migrate to SVE_VQ_BYTES instead of magic numbers.
> >
> > Requested by Alex Benn?e:
> >
> > * Thin out BUG_ON()s:
> > Redundant BUG_ON()s and ones that just check invariants are removed.
> > Important sanity-checks are migrated to WARN_ON()s, with some
> > minimal best-effort patch-up code.
> >
> > Other:
> >
> > * [ABI fix] Bail out with -EIO if attempting to set the
> > SVE regs for an unsupported VL, instead of misparsing the regset data.
> >
> > * Replace some in-kernel open-coded arithmetic with ALIGN()/
> > DIV_ROUND_UP().
> > ---
[...]
> > diff --git a/arch/arm64/include/uapi/asm/ptrace.h b/arch/arm64/include/uapi/asm/ptrace.h
[...]
> > +/* Definitions for user_sve_header.flags: */
> > +#define SVE_PT_REGS_MASK (1 << 0)
> > +
> > +/* Flags: must be kept in sync with prctl interface in
> > <linux/ptrace.h> */
>
> Which flags? We base some on PR_foo flags but we seem to shift them
> anyway so where is the requirement for them to match from?
I've rearranged this as:
-8<-
/* Definitions for user_sve_header.flags: */
#define SVE_PT_REGS_MASK (1 << 0)
#define SVE_PT_REGS_FPSIMD 0
#define SVE_PT_REGS_SVE SVE_PT_REGS_MASK
/*
* Common SVE_PT_* flags:
* These must be kept in sync with prctl interface in <linux/ptrace.h>
*/
#define SVE_PT_VL_INHERIT (PR_SVE_VL_INHERIT >> 16)
#define SVE_PT_VL_ONEXEC (PR_SVE_SET_VL_ONEXEC >> 16)
->8-
This avoids the suggestion that SVE_PT_REGS_{MASK,FPSIMD,SVE} are
supposed to have prctl counterparts.
I don't really want to write more, in case it is misinterpreted as
specification of behaviour.
This comment is really only meant as a reminder to maintainers that
they should go look at prctl.h too, before blindly making changes
here.
Any good? If you have a different suggestion, I'm all ears...
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
2017-09-14 13:31 ` Alex Bennée
@ 2017-09-29 13:00 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-29 13:00 UTC (permalink / raw)
To: Alex Bennée
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Marc Zyngier, Christoffer Dall,
Richard Sandiford, kvmarm, linux-arm-kernel
On Thu, Sep 14, 2017 at 02:31:13PM +0100, Alex Bennée wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > When trapping forbidden attempts by a guest to use SVE, we want the
> > guest to see a trap consistent with SVE not being implemented.
> >
> > This patch injects an undefined instruction exception into the
> > guest in response to such an exception.
>
> I do wonder if this should be merged with the previous trap enabling
> patch though?
Yes, that would make sense now I look at it.
Can I keep your Reviewed-by on the combined patch?
Cheers
---Dave
>
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > ---
> > arch/arm64/kvm/handle_exit.c | 8 ++++++++
> > 1 file changed, 8 insertions(+)
> >
> > diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> > index 17d8a16..e3e42d0 100644
> > --- a/arch/arm64/kvm/handle_exit.c
> > +++ b/arch/arm64/kvm/handle_exit.c
> > @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > return 1;
> > }
> >
> > +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > +{
> > + /* Until SVE is supported for guests: */
> > + kvm_inject_undefined(vcpu);
> > + return 1;
> > +}
> > +
> > static exit_handle_fn arm_exit_handlers[] = {
> > [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> > [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> > @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> > [ESR_ELx_EC_HVC64] = handle_hvc,
> > [ESR_ELx_EC_SMC64] = handle_smc,
> > [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> > + [ESR_ELx_EC_SVE] = handle_sve,
> > [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> > [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> > [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
>
>
> --
> Alex Bennée
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-29 13:00 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-09-29 13:00 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Sep 14, 2017 at 02:31:13PM +0100, Alex Benn?e wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > When trapping forbidden attempts by a guest to use SVE, we want the
> > guest to see a trap consistent with SVE not being implemented.
> >
> > This patch injects an undefined instruction exception into the
> > guest in response to such an exception.
>
> I do wonder if this should be merged with the previous trap enabling
> patch though?
Yes, that would make sense now I look at it.
Can I keep your Reviewed-by on the combined patch?
Cheers
---Dave
>
> >
> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > ---
> > arch/arm64/kvm/handle_exit.c | 8 ++++++++
> > 1 file changed, 8 insertions(+)
> >
> > diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
> > index 17d8a16..e3e42d0 100644
> > --- a/arch/arm64/kvm/handle_exit.c
> > +++ b/arch/arm64/kvm/handle_exit.c
> > @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > return 1;
> > }
> >
> > +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
> > +{
> > + /* Until SVE is supported for guests: */
> > + kvm_inject_undefined(vcpu);
> > + return 1;
> > +}
> > +
> > static exit_handle_fn arm_exit_handlers[] = {
> > [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
> > [ESR_ELx_EC_WFx] = kvm_handle_wfx,
> > @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
> > [ESR_ELx_EC_HVC64] = handle_hvc,
> > [ESR_ELx_EC_SMC64] = handle_smc,
> > [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
> > + [ESR_ELx_EC_SVE] = handle_sve,
> > [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
> > [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
> > [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
>
>
> --
> Alex Benn?e
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel at lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-29 14:43 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-29 14:43 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Marc Zyngier, Christoffer Dall,
Richard Sandiford, kvmarm, linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Thu, Sep 14, 2017 at 02:31:13PM +0100, Alex Bennée wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > When trapping forbidden attempts by a guest to use SVE, we want the
>> > guest to see a trap consistent with SVE not being implemented.
>> >
>> > This patch injects an undefined instruction exception into the
>> > guest in response to such an exception.
>>
>> I do wonder if this should be merged with the previous trap enabling
>> patch though?
>
> Yes, that would make sense now I look at it.
>
> Can I keep your Reviewed-by on the combined patch?
Sure.
>
> Cheers
> ---Dave
>
>>
>> >
>> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>> > ---
>> > arch/arm64/kvm/handle_exit.c | 8 ++++++++
>> > 1 file changed, 8 insertions(+)
>> >
>> > diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
>> > index 17d8a16..e3e42d0 100644
>> > --- a/arch/arm64/kvm/handle_exit.c
>> > +++ b/arch/arm64/kvm/handle_exit.c
>> > @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
>> > return 1;
>> > }
>> >
>> > +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
>> > +{
>> > + /* Until SVE is supported for guests: */
>> > + kvm_inject_undefined(vcpu);
>> > + return 1;
>> > +}
>> > +
>> > static exit_handle_fn arm_exit_handlers[] = {
>> > [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
>> > [ESR_ELx_EC_WFx] = kvm_handle_wfx,
>> > @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
>> > [ESR_ELx_EC_HVC64] = handle_hvc,
>> > [ESR_ELx_EC_SMC64] = handle_smc,
>> > [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
>> > + [ESR_ELx_EC_SVE] = handle_sve,
>> > [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
>> > [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
>> > [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
>>
>>
>> --
>> Alex Bennée
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-29 14:43 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-29 14:43 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Marc Zyngier, Christoffer Dall,
Richard Sandiford, kvmarm, linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Thu, Sep 14, 2017 at 02:31:13PM +0100, Alex Bennée wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > When trapping forbidden attempts by a guest to use SVE, we want the
>> > guest to see a trap consistent with SVE not being implemented.
>> >
>> > This patch injects an undefined instruction exception into the
>> > guest in response to such an exception.
>>
>> I do wonder if this should be merged with the previous trap enabling
>> patch though?
>
> Yes, that would make sense now I look at it.
>
> Can I keep your Reviewed-by on the combined patch?
Sure.
>
> Cheers
> ---Dave
>
>>
>> >
>> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>> > ---
>> > arch/arm64/kvm/handle_exit.c | 8 ++++++++
>> > 1 file changed, 8 insertions(+)
>> >
>> > diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
>> > index 17d8a16..e3e42d0 100644
>> > --- a/arch/arm64/kvm/handle_exit.c
>> > +++ b/arch/arm64/kvm/handle_exit.c
>> > @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
>> > return 1;
>> > }
>> >
>> > +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
>> > +{
>> > + /* Until SVE is supported for guests: */
>> > + kvm_inject_undefined(vcpu);
>> > + return 1;
>> > +}
>> > +
>> > static exit_handle_fn arm_exit_handlers[] = {
>> > [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
>> > [ESR_ELx_EC_WFx] = kvm_handle_wfx,
>> > @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
>> > [ESR_ELx_EC_HVC64] = handle_hvc,
>> > [ESR_ELx_EC_SMC64] = handle_smc,
>> > [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
>> > + [ESR_ELx_EC_SVE] = handle_sve,
>> > [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
>> > [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
>> > [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
>>
>>
>> --
>> Alex Bennée
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution
@ 2017-09-29 14:43 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-09-29 14:43 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Thu, Sep 14, 2017 at 02:31:13PM +0100, Alex Benn?e wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > When trapping forbidden attempts by a guest to use SVE, we want the
>> > guest to see a trap consistent with SVE not being implemented.
>> >
>> > This patch injects an undefined instruction exception into the
>> > guest in response to such an exception.
>>
>> I do wonder if this should be merged with the previous trap enabling
>> patch though?
>
> Yes, that would make sense now I look at it.
>
> Can I keep your Reviewed-by on the combined patch?
Sure.
>
> Cheers
> ---Dave
>
>>
>> >
>> > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
>> > ---
>> > arch/arm64/kvm/handle_exit.c | 8 ++++++++
>> > 1 file changed, 8 insertions(+)
>> >
>> > diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c
>> > index 17d8a16..e3e42d0 100644
>> > --- a/arch/arm64/kvm/handle_exit.c
>> > +++ b/arch/arm64/kvm/handle_exit.c
>> > @@ -147,6 +147,13 @@ static int kvm_handle_unknown_ec(struct kvm_vcpu *vcpu, struct kvm_run *run)
>> > return 1;
>> > }
>> >
>> > +static int handle_sve(struct kvm_vcpu *vcpu, struct kvm_run *run)
>> > +{
>> > + /* Until SVE is supported for guests: */
>> > + kvm_inject_undefined(vcpu);
>> > + return 1;
>> > +}
>> > +
>> > static exit_handle_fn arm_exit_handlers[] = {
>> > [0 ... ESR_ELx_EC_MAX] = kvm_handle_unknown_ec,
>> > [ESR_ELx_EC_WFx] = kvm_handle_wfx,
>> > @@ -160,6 +167,7 @@ static exit_handle_fn arm_exit_handlers[] = {
>> > [ESR_ELx_EC_HVC64] = handle_hvc,
>> > [ESR_ELx_EC_SMC64] = handle_smc,
>> > [ESR_ELx_EC_SYS64] = kvm_handle_sys_reg,
>> > + [ESR_ELx_EC_SVE] = handle_sve,
>> > [ESR_ELx_EC_IABT_LOW] = kvm_handle_guest_abort,
>> > [ESR_ELx_EC_DABT_LOW] = kvm_handle_guest_abort,
>> > [ESR_ELx_EC_SOFTSTP_LOW]= kvm_handle_guest_debug,
>>
>>
>> --
>> Alex Benn?e
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel at lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-20 13:58 ` Catalin Marinas
@ 2017-10-03 11:11 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-03 11:11 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 20, 2017 at 02:58:56PM +0100, Catalin Marinas wrote:
> On Thu, Sep 14, 2017 at 08:55:56PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > +/*
> > > > + * Handle SVE state across fork():
> > > > + *
> > > > + * dst and src must not end up with aliases of the same sve_state.
> > > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > > + * will be deferred until dst tries to use SVE.
> > > > + */
> > > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > > +{
> > > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > > + sve_to_fpsimd(dst);
> > > > + }
> > > > +
> > > > + dst->thread.sve_state = NULL;
> > > > +}
> > >
> > > I first thought the thread flags are not visible in dst yet since
> > > dup_task_struct() calls arch_dup_task_struct() before
> > > setup_thread_stack(). However, at the end of the last year we enabled
> > > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > > on this.
> >
> > Hmmm, I see your point, but there are some sequencing issues here.
> >
> > > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> >
> > I consider SVE discard as an optional side effect of task_fpsimd_save(),
> > not something that is guaranteed to happen -- the decision about whether
> > to do so may become more intelligent later on. So, for src, we may
> > discard SVE (because syscall), but for dst we must NULL .sve_state (and
> > therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
> > dst->sve_state.
>
> My point was that the SVE state of src is already preserved at this
> point and copied into dst. You don't need the sve_to_fpsimd(dst) again
> which basically does the same copying of the src SVE saved state into
> the FPSIMD one in dst. This has already been done in
> arch_dup_task_struct() by the combination of
> fpsimd_preserve_current_state() and *dst = *src (and, of course,
> clearing TIF_SVE in dst).
>
> I don't think the TIF_SVE clearing in src is just a side effect of
> task_fpsimd_save() here but rather a requirement. When returning from
> fork(), both src and dst would need to have the same state. However,
> your fpsimd_dup_sve() implementation makes it very clear that the SVE
> state is lost in dst. This is only allowed if we also lose it in src (as
> a result of a syscall). So making dst->sve_state = NULL requires that
> TIF_SVE is also cleared in both src and dst. Alternatively, you'd have
> to allocate a new state here and copy the full src SVE state across to
> dst, together with setting TIF_SVE (that's not necessary, however, since
> we get here as a result of a syscall).
The currently intended ABI is that the SVE bits are unspecified after a
syscall, so it is legitimate (though perhaps surprising) for different
things to happen in dst and src.
This complicates things a lot though, just to avoid the next SVE usage
exception in src after the fork.
It should be simpler to do what you suggest and discard the SVE state of
src unconditionally before the copy: then we really are just cloning the
thread apart from the need to set dst->thread.sve_state to NULL.
fpsimd_preserve_current_state() does not necessarily write back to
current->thread.fpsmid_state though: at the moment, it does do this as a
side effect of task_fpsimd_save() because we happen to be in a syscall
(i.e., fork).
What we really want is unconditional discarding of the state. This
wasn't needed anywhere else yet, so there's no explicit helper for it.
But it makes sense to add one.
What about refactoring along these lines:
fpsimd.c:
/* Unconditionally discard the SVE state */
void task_sve_discard(struct task_struct *task)
{
if (!system_supports_sve())
return;
local_bh_disable();
if (test_and_clear_tsk_thread_flag(task, TIF_SVE))
sve_to_fpsimd(task);
local_bh_enable();
}
process.c:
int arch_dup_task_struct(sturct task_struct *dst, struct task_struct *src)
{
if (current->mm) {
fpsimd_preserve_current_state();
task_sve_discard(src);
}
*dst = *src;
dst->thread.sve_state = NULL;
}
This also avoids having to touch dst's thread flags, since now we
are just cloning the task except for assigning NULL to
dst->thread.sve_state.
> > > for src, so the FPSIMD state (which we care about) is transferred during
> > > the *dst = *src assignment. So you'd only need the last statement,
> > > possibly with a different function name like fpsimd_erase_sve (and maybe
> > > make the function static inline in the header).
> >
> > Not quite: TIF_SVE must be cleared so that a context switch or
> > kernel_neon_begin() after dst is scheduled doesn't try to save state in
> > the (NULL) dst->sve_state.
>
> Yes, TIF_SVE must also be cleared in dst when dst->sve_state = NULL (I
> may have forgotten to mention this).
With the above factoring, the constraint "TIF_SVE implies sve_state
valid" is then never false, because TIF_SVE is already cleared before
the NULL assignment.
What do you think?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-03 11:11 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-03 11:11 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 20, 2017 at 02:58:56PM +0100, Catalin Marinas wrote:
> On Thu, Sep 14, 2017 at 08:55:56PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > +/*
> > > > + * Handle SVE state across fork():
> > > > + *
> > > > + * dst and src must not end up with aliases of the same sve_state.
> > > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > > + * will be deferred until dst tries to use SVE.
> > > > + */
> > > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > > +{
> > > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > > + sve_to_fpsimd(dst);
> > > > + }
> > > > +
> > > > + dst->thread.sve_state = NULL;
> > > > +}
> > >
> > > I first thought the thread flags are not visible in dst yet since
> > > dup_task_struct() calls arch_dup_task_struct() before
> > > setup_thread_stack(). However, at the end of the last year we enabled
> > > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > > on this.
> >
> > Hmmm, I see your point, but there are some sequencing issues here.
> >
> > > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> >
> > I consider SVE discard as an optional side effect of task_fpsimd_save(),
> > not something that is guaranteed to happen -- the decision about whether
> > to do so may become more intelligent later on. So, for src, we may
> > discard SVE (because syscall), but for dst we must NULL .sve_state (and
> > therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
> > dst->sve_state.
>
> My point was that the SVE state of src is already preserved at this
> point and copied into dst. You don't need the sve_to_fpsimd(dst) again
> which basically does the same copying of the src SVE saved state into
> the FPSIMD one in dst. This has already been done in
> arch_dup_task_struct() by the combination of
> fpsimd_preserve_current_state() and *dst = *src (and, of course,
> clearing TIF_SVE in dst).
>
> I don't think the TIF_SVE clearing in src is just a side effect of
> task_fpsimd_save() here but rather a requirement. When returning from
> fork(), both src and dst would need to have the same state. However,
> your fpsimd_dup_sve() implementation makes it very clear that the SVE
> state is lost in dst. This is only allowed if we also lose it in src (as
> a result of a syscall). So making dst->sve_state = NULL requires that
> TIF_SVE is also cleared in both src and dst. Alternatively, you'd have
> to allocate a new state here and copy the full src SVE state across to
> dst, together with setting TIF_SVE (that's not necessary, however, since
> we get here as a result of a syscall).
The currently intended ABI is that the SVE bits are unspecified after a
syscall, so it is legitimate (though perhaps surprising) for different
things to happen in dst and src.
This complicates things a lot though, just to avoid the next SVE usage
exception in src after the fork.
It should be simpler to do what you suggest and discard the SVE state of
src unconditionally before the copy: then we really are just cloning the
thread apart from the need to set dst->thread.sve_state to NULL.
fpsimd_preserve_current_state() does not necessarily write back to
current->thread.fpsmid_state though: at the moment, it does do this as a
side effect of task_fpsimd_save() because we happen to be in a syscall
(i.e., fork).
What we really want is unconditional discarding of the state. This
wasn't needed anywhere else yet, so there's no explicit helper for it.
But it makes sense to add one.
What about refactoring along these lines:
fpsimd.c:
/* Unconditionally discard the SVE state */
void task_sve_discard(struct task_struct *task)
{
if (!system_supports_sve())
return;
local_bh_disable();
if (test_and_clear_tsk_thread_flag(task, TIF_SVE))
sve_to_fpsimd(task);
local_bh_enable();
}
process.c:
int arch_dup_task_struct(sturct task_struct *dst, struct task_struct *src)
{
if (current->mm) {
fpsimd_preserve_current_state();
task_sve_discard(src);
}
*dst = *src;
dst->thread.sve_state = NULL;
}
This also avoids having to touch dst's thread flags, since now we
are just cloning the task except for assigning NULL to
dst->thread.sve_state.
> > > for src, so the FPSIMD state (which we care about) is transferred during
> > > the *dst = *src assignment. So you'd only need the last statement,
> > > possibly with a different function name like fpsimd_erase_sve (and maybe
> > > make the function static inline in the header).
> >
> > Not quite: TIF_SVE must be cleared so that a context switch or
> > kernel_neon_begin() after dst is scheduled doesn't try to save state in
> > the (NULL) dst->sve_state.
>
> Yes, TIF_SVE must also be cleared in dst when dst->sve_state = NULL (I
> may have forgotten to mention this).
With the above factoring, the constraint "TIF_SVE implies sve_state
valid" is then never false, because TIF_SVE is already cleared before
the NULL assignment.
What do you think?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-09-13 14:33 ` Catalin Marinas
@ 2017-10-03 11:33 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-03 11:33 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > +/*
> > + * Handle SVE state across fork():
> > + *
> > + * dst and src must not end up with aliases of the same sve_state.
> > + * Because a task cannot fork except in a syscall, we can discard SVE
> > + * state for dst here, so long as we take care to retain the FPSIMD
> > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > + * will be deferred until dst tries to use SVE.
> > + */
> > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > +{
> > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > + sve_to_fpsimd(dst);
> > + }
> > +
> > + dst->thread.sve_state = NULL;
> > +}
>
> I first thought the thread flags are not visible in dst yet since
> dup_task_struct() calls arch_dup_task_struct() before
> setup_thread_stack(). However, at the end of the last year we enabled
> CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> on this.
>
> Anyway, IIUC we don't need sve_to_fpsimd() here. The
> arch_dup_task_struct() already called fpsimd_preserve_current_state()
> for src, so the FPSIMD state (which we care about) is transferred during
> the *dst = *src assignment. So you'd only need the last statement,
> possibly with a different function name like fpsimd_erase_sve (and maybe
> make the function static inline in the header).
Regarding the intended meanings of the thread flags, does this help?
--8<--
TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
unchanged:
* If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
registers of the CPU the task is running on contain the authoritative
FPSIMD/SVE state of the task. The backing memory may be stale.
* Otherwise (i.e., task not running, or task running and
TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
authoritative. If additionally per_cpu(fpsimd_last_state,
task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
task->fpsimd_state.cpu's registers are also up to date for task, but
not authorititive: the current FPSIMD/SVE state may be read from
them, but they must not be written.
The FPSIMD/SVE backing memory is selected by TIF_SVE:
* TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
stored in task->thread.sve_state, formatted appropriately for vector
length task->thread.sve_vl. task->thread.sve_state must point to a
valid buffer at least sve_state_size(task) bytes in size.
* TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
logically zero[*] but not stored anywhere; Pn, FFR are not stored and
have unspecified values from userspace's point of view.
task->thread.sve_state does not need to be non-null, valid or any
particular size: it must not be dereferenced.
In practice I don't exploit the "unspecifiedness" much. The Zn high
bits, Pn and FFR are all zeroed when setting TIF_SVE again:
sve_alloc() is the common path for this.
* FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
whether TIF_SVE is clear or set, since these are not vector length
dependent.
[*] theoretically unspecified, which is what I've written in sve.txt.
However, on any FPSIMD Vn write the upper bits of the corresponding Zn
must become logically zero in order to conform to the SVE programmer's
model. It's not feasible to track which Vn have been written
individually because that would involve trapping every FPSIMD insn until
all possible Vn have been written. So the kernel ensures that the Zn
high bits become zero.
Maybe we should just guarantee "zero-or-preserved" behaviour for
userspace. This may close down some future optimisation opportunities,
but maybe it's better to be simple.
Thoughts?
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-03 11:33 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-03 11:33 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > +/*
> > + * Handle SVE state across fork():
> > + *
> > + * dst and src must not end up with aliases of the same sve_state.
> > + * Because a task cannot fork except in a syscall, we can discard SVE
> > + * state for dst here, so long as we take care to retain the FPSIMD
> > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > + * will be deferred until dst tries to use SVE.
> > + */
> > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > +{
> > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > + sve_to_fpsimd(dst);
> > + }
> > +
> > + dst->thread.sve_state = NULL;
> > +}
>
> I first thought the thread flags are not visible in dst yet since
> dup_task_struct() calls arch_dup_task_struct() before
> setup_thread_stack(). However, at the end of the last year we enabled
> CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> on this.
>
> Anyway, IIUC we don't need sve_to_fpsimd() here. The
> arch_dup_task_struct() already called fpsimd_preserve_current_state()
> for src, so the FPSIMD state (which we care about) is transferred during
> the *dst = *src assignment. So you'd only need the last statement,
> possibly with a different function name like fpsimd_erase_sve (and maybe
> make the function static inline in the header).
Regarding the intended meanings of the thread flags, does this help?
--8<--
TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
unchanged:
* If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
registers of the CPU the task is running on contain the authoritative
FPSIMD/SVE state of the task. The backing memory may be stale.
* Otherwise (i.e., task not running, or task running and
TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
authoritative. If additionally per_cpu(fpsimd_last_state,
task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
task->fpsimd_state.cpu's registers are also up to date for task, but
not authorititive: the current FPSIMD/SVE state may be read from
them, but they must not be written.
The FPSIMD/SVE backing memory is selected by TIF_SVE:
* TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
stored in task->thread.sve_state, formatted appropriately for vector
length task->thread.sve_vl. task->thread.sve_state must point to a
valid buffer at least sve_state_size(task) bytes in size.
* TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
logically zero[*] but not stored anywhere; Pn, FFR are not stored and
have unspecified values from userspace's point of view.
task->thread.sve_state does not need to be non-null, valid or any
particular size: it must not be dereferenced.
In practice I don't exploit the "unspecifiedness" much. The Zn high
bits, Pn and FFR are all zeroed when setting TIF_SVE again:
sve_alloc() is the common path for this.
* FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
whether TIF_SVE is clear or set, since these are not vector length
dependent.
[*] theoretically unspecified, which is what I've written in sve.txt.
However, on any FPSIMD Vn write the upper bits of the corresponding Zn
must become logically zero in order to conform to the SVE programmer's
model. It's not feasible to track which Vn have been written
individually because that would involve trapping every FPSIMD insn until
all possible Vn have been written. So the kernel ensures that the Zn
high bits become zero.
Maybe we should just guarantee "zero-or-preserved" behaviour for
userspace. This may close down some future optimisation opportunities,
but maybe it's better to be simple.
Thoughts?
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-10-03 11:11 ` Dave Martin
@ 2017-10-04 17:29 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-04 17:29 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Tue, Oct 03, 2017 at 12:11:01PM +0100, Dave P Martin wrote:
> On Wed, Sep 20, 2017 at 02:58:56PM +0100, Catalin Marinas wrote:
> > On Thu, Sep 14, 2017 at 08:55:56PM +0100, Dave P Martin wrote:
> > > On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > > +/*
> > > > > + * Handle SVE state across fork():
> > > > > + *
> > > > > + * dst and src must not end up with aliases of the same sve_state.
> > > > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > > > + * will be deferred until dst tries to use SVE.
> > > > > + */
> > > > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > > > +{
> > > > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > > > + sve_to_fpsimd(dst);
> > > > > + }
> > > > > +
> > > > > + dst->thread.sve_state = NULL;
> > > > > +}
> > > >
> > > > I first thought the thread flags are not visible in dst yet since
> > > > dup_task_struct() calls arch_dup_task_struct() before
> > > > setup_thread_stack(). However, at the end of the last year we enabled
> > > > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > > > on this.
> > >
> > > Hmmm, I see your point, but there are some sequencing issues here.
> > >
> > > > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > > > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> > >
> > > I consider SVE discard as an optional side effect of task_fpsimd_save(),
> > > not something that is guaranteed to happen -- the decision about whether
> > > to do so may become more intelligent later on. So, for src, we may
> > > discard SVE (because syscall), but for dst we must NULL .sve_state (and
> > > therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
> > > dst->sve_state.
> >
> > My point was that the SVE state of src is already preserved at this
> > point and copied into dst. You don't need the sve_to_fpsimd(dst) again
> > which basically does the same copying of the src SVE saved state into
> > the FPSIMD one in dst. This has already been done in
> > arch_dup_task_struct() by the combination of
> > fpsimd_preserve_current_state() and *dst = *src (and, of course,
> > clearing TIF_SVE in dst).
> >
> > I don't think the TIF_SVE clearing in src is just a side effect of
> > task_fpsimd_save() here but rather a requirement. When returning from
> > fork(), both src and dst would need to have the same state. However,
> > your fpsimd_dup_sve() implementation makes it very clear that the SVE
> > state is lost in dst. This is only allowed if we also lose it in src (as
> > a result of a syscall). So making dst->sve_state = NULL requires that
> > TIF_SVE is also cleared in both src and dst. Alternatively, you'd have
> > to allocate a new state here and copy the full src SVE state across to
> > dst, together with setting TIF_SVE (that's not necessary, however, since
> > we get here as a result of a syscall).
>
> The currently intended ABI is that the SVE bits are unspecified after a
> syscall, so it is legitimate (though perhaps surprising) for different
> things to happen in dst and src.
>
> This complicates things a lot though, just to avoid the next SVE usage
> exception in src after the fork.
>
>
> It should be simpler to do what you suggest and discard the SVE state of
> src unconditionally before the copy: then we really are just cloning the
> thread apart from the need to set dst->thread.sve_state to NULL.
>
> fpsimd_preserve_current_state() does not necessarily write back to
> current->thread.fpsmid_state though: at the moment, it does do this as a
> side effect of task_fpsimd_save() because we happen to be in a syscall
> (i.e., fork).
>
> What we really want is unconditional discarding of the state. This
> wasn't needed anywhere else yet, so there's no explicit helper for it.
> But it makes sense to add one.
>
> What about refactoring along these lines:
>
>
> fpsimd.c:
> /* Unconditionally discard the SVE state */
> void task_sve_discard(struct task_struct *task)
> {
> if (!system_supports_sve())
> return;
>
> local_bh_disable();
> if (test_and_clear_tsk_thread_flag(task, TIF_SVE))
> sve_to_fpsimd(task);
> local_bh_enable();
> }
>
> process.c:
> int arch_dup_task_struct(sturct task_struct *dst, struct task_struct *src)
> {
> if (current->mm) {
> fpsimd_preserve_current_state();
> task_sve_discard(src);
> }
>
> *dst = *src;
>
> dst->thread.sve_state = NULL;
> }
>
>
> This also avoids having to touch dst's thread flags, since now we
> are just cloning the task except for assigning NULL to
> dst->thread.sve_state.
This looks fine to me, the execution of task_sve_discard() is nearly a
no-op with the current code.
We still have some local_bh_disable/enable() calls, though I don't think
it's worth optimising them now (e.g. having a
fpsimd_preserve_current_state_and_discard_sve() function with a
"sve_discard" argument to task_fpsimd_save() to force this).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-04 17:29 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-04 17:29 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Oct 03, 2017 at 12:11:01PM +0100, Dave P Martin wrote:
> On Wed, Sep 20, 2017 at 02:58:56PM +0100, Catalin Marinas wrote:
> > On Thu, Sep 14, 2017 at 08:55:56PM +0100, Dave P Martin wrote:
> > > On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > > +/*
> > > > > + * Handle SVE state across fork():
> > > > > + *
> > > > > + * dst and src must not end up with aliases of the same sve_state.
> > > > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > > > + * will be deferred until dst tries to use SVE.
> > > > > + */
> > > > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > > > +{
> > > > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > > > + sve_to_fpsimd(dst);
> > > > > + }
> > > > > +
> > > > > + dst->thread.sve_state = NULL;
> > > > > +}
> > > >
> > > > I first thought the thread flags are not visible in dst yet since
> > > > dup_task_struct() calls arch_dup_task_struct() before
> > > > setup_thread_stack(). However, at the end of the last year we enabled
> > > > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > > > on this.
> > >
> > > Hmmm, I see your point, but there are some sequencing issues here.
> > >
> > > > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > > > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> > >
> > > I consider SVE discard as an optional side effect of task_fpsimd_save(),
> > > not something that is guaranteed to happen -- the decision about whether
> > > to do so may become more intelligent later on. So, for src, we may
> > > discard SVE (because syscall), but for dst we must NULL .sve_state (and
> > > therefore clear TIF_SVE) simply to avoid aliasing of src->sve_state and
> > > dst->sve_state.
> >
> > My point was that the SVE state of src is already preserved at this
> > point and copied into dst. You don't need the sve_to_fpsimd(dst) again
> > which basically does the same copying of the src SVE saved state into
> > the FPSIMD one in dst. This has already been done in
> > arch_dup_task_struct() by the combination of
> > fpsimd_preserve_current_state() and *dst = *src (and, of course,
> > clearing TIF_SVE in dst).
> >
> > I don't think the TIF_SVE clearing in src is just a side effect of
> > task_fpsimd_save() here but rather a requirement. When returning from
> > fork(), both src and dst would need to have the same state. However,
> > your fpsimd_dup_sve() implementation makes it very clear that the SVE
> > state is lost in dst. This is only allowed if we also lose it in src (as
> > a result of a syscall). So making dst->sve_state = NULL requires that
> > TIF_SVE is also cleared in both src and dst. Alternatively, you'd have
> > to allocate a new state here and copy the full src SVE state across to
> > dst, together with setting TIF_SVE (that's not necessary, however, since
> > we get here as a result of a syscall).
>
> The currently intended ABI is that the SVE bits are unspecified after a
> syscall, so it is legitimate (though perhaps surprising) for different
> things to happen in dst and src.
>
> This complicates things a lot though, just to avoid the next SVE usage
> exception in src after the fork.
>
>
> It should be simpler to do what you suggest and discard the SVE state of
> src unconditionally before the copy: then we really are just cloning the
> thread apart from the need to set dst->thread.sve_state to NULL.
>
> fpsimd_preserve_current_state() does not necessarily write back to
> current->thread.fpsmid_state though: at the moment, it does do this as a
> side effect of task_fpsimd_save() because we happen to be in a syscall
> (i.e., fork).
>
> What we really want is unconditional discarding of the state. This
> wasn't needed anywhere else yet, so there's no explicit helper for it.
> But it makes sense to add one.
>
> What about refactoring along these lines:
>
>
> fpsimd.c:
> /* Unconditionally discard the SVE state */
> void task_sve_discard(struct task_struct *task)
> {
> if (!system_supports_sve())
> return;
>
> local_bh_disable();
> if (test_and_clear_tsk_thread_flag(task, TIF_SVE))
> sve_to_fpsimd(task);
> local_bh_enable();
> }
>
> process.c:
> int arch_dup_task_struct(sturct task_struct *dst, struct task_struct *src)
> {
> if (current->mm) {
> fpsimd_preserve_current_state();
> task_sve_discard(src);
> }
>
> *dst = *src;
>
> dst->thread.sve_state = NULL;
> }
>
>
> This also avoids having to touch dst's thread flags, since now we
> are just cloning the task except for assigning NULL to
> dst->thread.sve_state.
This looks fine to me, the execution of task_sve_discard() is nearly a
no-op with the current code.
We still have some local_bh_disable/enable() calls, though I don't think
it's worth optimising them now (e.g. having a
fpsimd_preserve_current_state_and_discard_sve() function with a
"sve_discard" argument to task_fpsimd_save() to force this).
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 10/28] arm64/sve: Low-level CPU setup
2017-09-13 19:21 ` Dave Martin
@ 2017-10-05 10:47 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 10:47 UTC (permalink / raw)
To: Catalin Marinas, Suzuki Poulose
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Wed, Sep 13, 2017 at 08:21:11PM +0100, Dave Martin wrote:
> On Wed, Sep 13, 2017 at 06:32:06AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> > > diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> > > index 877d42f..dd22ef2 100644
> > > --- a/arch/arm64/mm/proc.S
> > > +++ b/arch/arm64/mm/proc.S
> > > @@ -27,6 +27,7 @@
> > > #include <asm/pgtable-hwdef.h>
> > > #include <asm/cpufeature.h>
> > > #include <asm/alternative.h>
> > > +#include <asm/sysreg.h>
> > >
> > > #ifdef CONFIG_ARM64_64K_PAGES
> > > #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> > > @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> > > tlbi vmalle1 // Invalidate local TLB
> > > dsb nsh
> > >
> > > - mov x0, #3 << 20
> > > - msr cpacr_el1, x0 // Enable FP/ASIMD
> > > + mov x0, #3 << 20 // FEN
> > > +
> > > + /* SVE */
> > > + mrs x5, id_aa64pfr0_el1
> > > + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> > > + cbz x5, 1f
> > > +
> > > + bic x0, x0, #CPACR_EL1_ZEN
> > > + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> > > +1: msr cpacr_el1, x0 // Enable FP/ASIMD
> >
> > For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
> > I tried to do the same with FPSIMD but hit an issue with EFI run-time
> > services (I may be wrong though).
>
> I'll take a look at this -- I believe it should be safe to disable this
> trap for EL1 relatively late. This is needed before probing for
> available vector lengths, but apart from that the kernel shouldn't touch
> SVE until/unless some user task uses SVE.
>
> This would change if we eventually enable kernel-mode SVE, but I wouldn't
> expect that to get used in early boot before the cpufeatures code runs.
>
> Ard may have a view on this.
I've had a go at this, but there's a lot of splatter.
I can add a helper el1_enable_sve() say, and call it before probing
ZCR_EL1 in the cpufeatures code.
This makes enabling SVE a side-effect of the cpufeatures code, which
I'm not that comfortable with -- it feels like something that later
refactoring could easily break.
I can also add an explicit call to el1_enable_sve() in sve_setup(),
but this only works on the boot cpu.
For secondaries, I could add something in secondary_start_kernel(),
but this looks incongruous since there's no other call to do something
similar yet.
** Suzuki, do we have any other case where the trap for a CPU feature is
turned off by the cpufeatures code? If there's already a template for
this then I'm happy to follow it.
Otherwise, maybe it's simpler to keep it in __cpu_setup after all
since that's a common path that all CPUs pass through.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-10-05 10:47 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 10:47 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 08:21:11PM +0100, Dave Martin wrote:
> On Wed, Sep 13, 2017 at 06:32:06AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> > > diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> > > index 877d42f..dd22ef2 100644
> > > --- a/arch/arm64/mm/proc.S
> > > +++ b/arch/arm64/mm/proc.S
> > > @@ -27,6 +27,7 @@
> > > #include <asm/pgtable-hwdef.h>
> > > #include <asm/cpufeature.h>
> > > #include <asm/alternative.h>
> > > +#include <asm/sysreg.h>
> > >
> > > #ifdef CONFIG_ARM64_64K_PAGES
> > > #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> > > @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> > > tlbi vmalle1 // Invalidate local TLB
> > > dsb nsh
> > >
> > > - mov x0, #3 << 20
> > > - msr cpacr_el1, x0 // Enable FP/ASIMD
> > > + mov x0, #3 << 20 // FEN
> > > +
> > > + /* SVE */
> > > + mrs x5, id_aa64pfr0_el1
> > > + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> > > + cbz x5, 1f
> > > +
> > > + bic x0, x0, #CPACR_EL1_ZEN
> > > + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> > > +1: msr cpacr_el1, x0 // Enable FP/ASIMD
> >
> > For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
> > I tried to do the same with FPSIMD but hit an issue with EFI run-time
> > services (I may be wrong though).
>
> I'll take a look at this -- I believe it should be safe to disable this
> trap for EL1 relatively late. This is needed before probing for
> available vector lengths, but apart from that the kernel shouldn't touch
> SVE until/unless some user task uses SVE.
>
> This would change if we eventually enable kernel-mode SVE, but I wouldn't
> expect that to get used in early boot before the cpufeatures code runs.
>
> Ard may have a view on this.
I've had a go at this, but there's a lot of splatter.
I can add a helper el1_enable_sve() say, and call it before probing
ZCR_EL1 in the cpufeatures code.
This makes enabling SVE a side-effect of the cpufeatures code, which
I'm not that comfortable with -- it feels like something that later
refactoring could easily break.
I can also add an explicit call to el1_enable_sve() in sve_setup(),
but this only works on the boot cpu.
For secondaries, I could add something in secondary_start_kernel(),
but this looks incongruous since there's no other call to do something
similar yet.
** Suzuki, do we have any other case where the trap for a CPU feature is
turned off by the cpufeatures code? If there's already a template for
this then I'm happy to follow it.
Otherwise, maybe it's simpler to keep it in __cpu_setup after all
since that's a common path that all CPUs pass through.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 10/28] arm64/sve: Low-level CPU setup
2017-10-05 10:47 ` Dave Martin
@ 2017-10-05 11:04 ` Suzuki K Poulose
-1 siblings, 0 replies; 224+ messages in thread
From: Suzuki K Poulose @ 2017-10-05 11:04 UTC (permalink / raw)
To: Dave Martin, Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On 05/10/17 11:47, Dave Martin wrote:
> On Wed, Sep 13, 2017 at 08:21:11PM +0100, Dave Martin wrote:
>> On Wed, Sep 13, 2017 at 06:32:06AM -0700, Catalin Marinas wrote:
>>> On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
>>>> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
>>>> index 877d42f..dd22ef2 100644
>>>> --- a/arch/arm64/mm/proc.S
>>>> +++ b/arch/arm64/mm/proc.S
>>>> @@ -27,6 +27,7 @@
>>>> #include <asm/pgtable-hwdef.h>
>>>> #include <asm/cpufeature.h>
>>>> #include <asm/alternative.h>
>>>> +#include <asm/sysreg.h>
>>>>
>>>> #ifdef CONFIG_ARM64_64K_PAGES
>>>> #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
>>>> @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
>>>> tlbi vmalle1 // Invalidate local TLB
>>>> dsb nsh
>>>>
>>>> - mov x0, #3 << 20
>>>> - msr cpacr_el1, x0 // Enable FP/ASIMD
>>>> + mov x0, #3 << 20 // FEN
>>>> +
>>>> + /* SVE */
>>>> + mrs x5, id_aa64pfr0_el1
>>>> + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
>>>> + cbz x5, 1f
>>>> +
>>>> + bic x0, x0, #CPACR_EL1_ZEN
>>>> + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
>>>> +1: msr cpacr_el1, x0 // Enable FP/ASIMD
>>>
>>> For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
>>> I tried to do the same with FPSIMD but hit an issue with EFI run-time
>>> services (I may be wrong though).
>>
>> I'll take a look at this -- I believe it should be safe to disable this
>> trap for EL1 relatively late. This is needed before probing for
>> available vector lengths, but apart from that the kernel shouldn't touch
>> SVE until/unless some user task uses SVE.
>>
>> This would change if we eventually enable kernel-mode SVE, but I wouldn't
>> expect that to get used in early boot before the cpufeatures code runs.
>>
>> Ard may have a view on this.
>
> I've had a go at this, but there's a lot of splatter.
>
> I can add a helper el1_enable_sve() say, and call it before probing
> ZCR_EL1 in the cpufeatures code.
>
> This makes enabling SVE a side-effect of the cpufeatures code, which
> I'm not that comfortable with -- it feels like something that later
> refactoring could easily break.
>
> I can also add an explicit call to el1_enable_sve() in sve_setup(),
> but this only works on the boot cpu.
>
> For secondaries, I could add something in secondary_start_kernel(),
> but this looks incongruous since there's no other call to do something
> similar yet.
>
>
> ** Suzuki, do we have any other case where the trap for a CPU feature is
> turned off by the cpufeatures code? If there's already a template for
> this then I'm happy to follow it.
The closest thing you have is an "enable" callback for each "available"
capability, which gets invoked on all CPUs (boot time active CPUs and the
ones which are brought up later). This is used by things like, PAN to
do some CPU specific setups.
See :
enable_cpu_capabilities() // For all boot time active CPUs
and
verify_local_cpu_features() // For CPUs brought up later
Cheers
Suzuki
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-10-05 11:04 ` Suzuki K Poulose
0 siblings, 0 replies; 224+ messages in thread
From: Suzuki K Poulose @ 2017-10-05 11:04 UTC (permalink / raw)
To: linux-arm-kernel
On 05/10/17 11:47, Dave Martin wrote:
> On Wed, Sep 13, 2017 at 08:21:11PM +0100, Dave Martin wrote:
>> On Wed, Sep 13, 2017 at 06:32:06AM -0700, Catalin Marinas wrote:
>>> On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
>>>> diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
>>>> index 877d42f..dd22ef2 100644
>>>> --- a/arch/arm64/mm/proc.S
>>>> +++ b/arch/arm64/mm/proc.S
>>>> @@ -27,6 +27,7 @@
>>>> #include <asm/pgtable-hwdef.h>
>>>> #include <asm/cpufeature.h>
>>>> #include <asm/alternative.h>
>>>> +#include <asm/sysreg.h>
>>>>
>>>> #ifdef CONFIG_ARM64_64K_PAGES
>>>> #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
>>>> @@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
>>>> tlbi vmalle1 // Invalidate local TLB
>>>> dsb nsh
>>>>
>>>> - mov x0, #3 << 20
>>>> - msr cpacr_el1, x0 // Enable FP/ASIMD
>>>> + mov x0, #3 << 20 // FEN
>>>> +
>>>> + /* SVE */
>>>> + mrs x5, id_aa64pfr0_el1
>>>> + ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
>>>> + cbz x5, 1f
>>>> +
>>>> + bic x0, x0, #CPACR_EL1_ZEN
>>>> + orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
>>>> +1: msr cpacr_el1, x0 // Enable FP/ASIMD
>>>
>>> For EL1, I wonder whether we could move this later to cpufeature.c. IIRC
>>> I tried to do the same with FPSIMD but hit an issue with EFI run-time
>>> services (I may be wrong though).
>>
>> I'll take a look at this -- I believe it should be safe to disable this
>> trap for EL1 relatively late. This is needed before probing for
>> available vector lengths, but apart from that the kernel shouldn't touch
>> SVE until/unless some user task uses SVE.
>>
>> This would change if we eventually enable kernel-mode SVE, but I wouldn't
>> expect that to get used in early boot before the cpufeatures code runs.
>>
>> Ard may have a view on this.
>
> I've had a go at this, but there's a lot of splatter.
>
> I can add a helper el1_enable_sve() say, and call it before probing
> ZCR_EL1 in the cpufeatures code.
>
> This makes enabling SVE a side-effect of the cpufeatures code, which
> I'm not that comfortable with -- it feels like something that later
> refactoring could easily break.
>
> I can also add an explicit call to el1_enable_sve() in sve_setup(),
> but this only works on the boot cpu.
>
> For secondaries, I could add something in secondary_start_kernel(),
> but this looks incongruous since there's no other call to do something
> similar yet.
>
>
> ** Suzuki, do we have any other case where the trap for a CPU feature is
> turned off by the cpufeatures code? If there's already a template for
> this then I'm happy to follow it.
The closest thing you have is an "enable" callback for each "available"
capability, which gets invoked on all CPUs (boot time active CPUs and the
ones which are brought up later). This is used by things like, PAN to
do some CPU specific setups.
See :
enable_cpu_capabilities() // For all boot time active CPUs
and
verify_local_cpu_features() // For CPUs brought up later
Cheers
Suzuki
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 10/28] arm64/sve: Low-level CPU setup
2017-10-05 11:04 ` Suzuki K Poulose
@ 2017-10-05 11:22 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 11:22 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: Catalin Marinas, linux-arch, libc-alpha, Ard Biesheuvel,
Szabolcs Nagy, Richard Sandiford, Will Deacon, Alex Bennée,
kvmarm, linux-arm-kernel
On Thu, Oct 05, 2017 at 12:04:12PM +0100, Suzuki K Poulose wrote:
> On 05/10/17 11:47, Dave Martin wrote:
[...]
> >>>On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> >>>>diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> >>>>index 877d42f..dd22ef2 100644
> >>>>--- a/arch/arm64/mm/proc.S
> >>>>+++ b/arch/arm64/mm/proc.S
> >>>>@@ -27,6 +27,7 @@
> >>>> #include <asm/pgtable-hwdef.h>
> >>>> #include <asm/cpufeature.h>
> >>>> #include <asm/alternative.h>
> >>>>+#include <asm/sysreg.h>
> >>>> #ifdef CONFIG_ARM64_64K_PAGES
> >>>> #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> >>>>@@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> >>>> tlbi vmalle1 // Invalidate local TLB
> >>>> dsb nsh
> >>>>- mov x0, #3 << 20
> >>>>- msr cpacr_el1, x0 // Enable FP/ASIMD
> >>>>+ mov x0, #3 << 20 // FEN
> >>>>+
> >>>>+ /* SVE */
> >>>>+ mrs x5, id_aa64pfr0_el1
> >>>>+ ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> >>>>+ cbz x5, 1f
> >>>>+
> >>>>+ bic x0, x0, #CPACR_EL1_ZEN
> >>>>+ orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> >>>>+1: msr cpacr_el1, x0 // Enable FP/ASIMD
[..]
> >I can add a helper el1_enable_sve() say, and call it before probing
> >ZCR_EL1 in the cpufeatures code.
> >
> >This makes enabling SVE a side-effect of the cpufeatures code, which
> >I'm not that comfortable with -- it feels like something that later
> >refactoring could easily break.
> >
> >I can also add an explicit call to el1_enable_sve() in sve_setup(),
> >but this only works on the boot cpu.
> >
> >For secondaries, I could add something in secondary_start_kernel(),
> >but this looks incongruous since there's no other call to do something
> >similar yet.
> >
> >
> >** Suzuki, do we have any other case where the trap for a CPU feature is
> >turned off by the cpufeatures code? If there's already a template for
> >this then I'm happy to follow it.
>
> The closest thing you have is an "enable" callback for each "available"
> capability, which gets invoked on all CPUs (boot time active CPUs and the
> ones which are brought up later). This is used by things like, PAN to
> do some CPU specific setups.
>
> See :
> enable_cpu_capabilities() // For all boot time active CPUs
> and
> verify_local_cpu_features() // For CPUs brought up later
That may allow me to do something tidier, provided the enable method is
called early enough.
I'll take a look.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 10/28] arm64/sve: Low-level CPU setup
@ 2017-10-05 11:22 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 11:22 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 05, 2017 at 12:04:12PM +0100, Suzuki K Poulose wrote:
> On 05/10/17 11:47, Dave Martin wrote:
[...]
> >>>On Thu, Aug 31, 2017 at 06:00:42PM +0100, Dave P Martin wrote:
> >>>>diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
> >>>>index 877d42f..dd22ef2 100644
> >>>>--- a/arch/arm64/mm/proc.S
> >>>>+++ b/arch/arm64/mm/proc.S
> >>>>@@ -27,6 +27,7 @@
> >>>> #include <asm/pgtable-hwdef.h>
> >>>> #include <asm/cpufeature.h>
> >>>> #include <asm/alternative.h>
> >>>>+#include <asm/sysreg.h>
> >>>> #ifdef CONFIG_ARM64_64K_PAGES
> >>>> #define TCR_TG_FLAGS TCR_TG0_64K | TCR_TG1_64K
> >>>>@@ -186,8 +187,17 @@ ENTRY(__cpu_setup)
> >>>> tlbi vmalle1 // Invalidate local TLB
> >>>> dsb nsh
> >>>>- mov x0, #3 << 20
> >>>>- msr cpacr_el1, x0 // Enable FP/ASIMD
> >>>>+ mov x0, #3 << 20 // FEN
> >>>>+
> >>>>+ /* SVE */
> >>>>+ mrs x5, id_aa64pfr0_el1
> >>>>+ ubfx x5, x5, #ID_AA64PFR0_SVE_SHIFT, #4
> >>>>+ cbz x5, 1f
> >>>>+
> >>>>+ bic x0, x0, #CPACR_EL1_ZEN
> >>>>+ orr x0, x0, #CPACR_EL1_ZEN_EL1EN // SVE: trap for EL0, not EL1
> >>>>+1: msr cpacr_el1, x0 // Enable FP/ASIMD
[..]
> >I can add a helper el1_enable_sve() say, and call it before probing
> >ZCR_EL1 in the cpufeatures code.
> >
> >This makes enabling SVE a side-effect of the cpufeatures code, which
> >I'm not that comfortable with -- it feels like something that later
> >refactoring could easily break.
> >
> >I can also add an explicit call to el1_enable_sve() in sve_setup(),
> >but this only works on the boot cpu.
> >
> >For secondaries, I could add something in secondary_start_kernel(),
> >but this looks incongruous since there's no other call to do something
> >similar yet.
> >
> >
> >** Suzuki, do we have any other case where the trap for a CPU feature is
> >turned off by the cpufeatures code? If there's already a template for
> >this then I'm happy to follow it.
>
> The closest thing you have is an "enable" callback for each "available"
> capability, which gets invoked on all CPUs (boot time active CPUs and the
> ones which are brought up later). This is used by things like, PAN to
> do some CPU specific setups.
>
> See :
> enable_cpu_capabilities() // For all boot time active CPUs
> and
> verify_local_cpu_features() // For CPUs brought up later
That may allow me to do something tidier, provided the enable method is
called early enough.
I'll take a look.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-10-03 11:33 ` Dave Martin
@ 2017-10-05 11:28 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-05 11:28 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > +/*
> > > + * Handle SVE state across fork():
> > > + *
> > > + * dst and src must not end up with aliases of the same sve_state.
> > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > + * will be deferred until dst tries to use SVE.
> > > + */
> > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > +{
> > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > + sve_to_fpsimd(dst);
> > > + }
> > > +
> > > + dst->thread.sve_state = NULL;
> > > +}
> >
> > I first thought the thread flags are not visible in dst yet since
> > dup_task_struct() calls arch_dup_task_struct() before
> > setup_thread_stack(). However, at the end of the last year we enabled
> > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > on this.
> >
> > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> > for src, so the FPSIMD state (which we care about) is transferred during
> > the *dst = *src assignment. So you'd only need the last statement,
> > possibly with a different function name like fpsimd_erase_sve (and maybe
> > make the function static inline in the header).
>
> Regarding the intended meanings of the thread flags, does this help?
>
> --8<--
>
> TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> unchanged:
>
> * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> registers of the CPU the task is running on contain the authoritative
> FPSIMD/SVE state of the task. The backing memory may be stale.
>
> * Otherwise (i.e., task not running, or task running and
> TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> authoritative. If additionally per_cpu(fpsimd_last_state,
> task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> task->fpsimd_state.cpu's registers are also up to date for task, but
> not authorititive: the current FPSIMD/SVE state may be read from
> them, but they must not be written.
>
> The FPSIMD/SVE backing memory is selected by TIF_SVE:
>
> * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> stored in task->thread.sve_state, formatted appropriately for vector
> length task->thread.sve_vl. task->thread.sve_state must point to a
> valid buffer at least sve_state_size(task) bytes in size.
>
> * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> have unspecified values from userspace's point of view.
> task->thread.sve_state does not need to be non-null, valid or any
> particular size: it must not be dereferenced.
>
> In practice I don't exploit the "unspecifiedness" much. The Zn high
> bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> sve_alloc() is the common path for this.
>
> * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> whether TIF_SVE is clear or set, since these are not vector length
> dependent.
This looks fine. I think we need to make sure (with a warning) that
task_fpsimd_save() (and probably load) is always called in a
non-preemptible context.
> [*] theoretically unspecified, which is what I've written in sve.txt.
> However, on any FPSIMD Vn write the upper bits of the corresponding Zn
> must become logically zero in order to conform to the SVE programmer's
> model. It's not feasible to track which Vn have been written
> individually because that would involve trapping every FPSIMD insn until
> all possible Vn have been written. So the kernel ensures that the Zn
> high bits become zero.
>
> Maybe we should just guarantee "zero-or-preserved" behaviour for
> userspace. This may close down some future optimisation opportunities,
> but maybe it's better to be simple.
Does it work if we leave it as "unspecified" in the document but just do
zero-or-preserved in the kernel code?
Just wondering, as an optimisation for do_sve_acc() - instead of
sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
ensure the zeroing of the top SVE bits and we only need to allocate the
SVE state on the saving path. This means enabling SVE for user and
setting TIF_SVE without having the backing storage allocated.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-05 11:28 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-05 11:28 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > +/*
> > > + * Handle SVE state across fork():
> > > + *
> > > + * dst and src must not end up with aliases of the same sve_state.
> > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > + * will be deferred until dst tries to use SVE.
> > > + */
> > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > +{
> > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > + sve_to_fpsimd(dst);
> > > + }
> > > +
> > > + dst->thread.sve_state = NULL;
> > > +}
> >
> > I first thought the thread flags are not visible in dst yet since
> > dup_task_struct() calls arch_dup_task_struct() before
> > setup_thread_stack(). However, at the end of the last year we enabled
> > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > on this.
> >
> > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> > for src, so the FPSIMD state (which we care about) is transferred during
> > the *dst = *src assignment. So you'd only need the last statement,
> > possibly with a different function name like fpsimd_erase_sve (and maybe
> > make the function static inline in the header).
>
> Regarding the intended meanings of the thread flags, does this help?
>
> --8<--
>
> TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> unchanged:
>
> * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> registers of the CPU the task is running on contain the authoritative
> FPSIMD/SVE state of the task. The backing memory may be stale.
>
> * Otherwise (i.e., task not running, or task running and
> TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> authoritative. If additionally per_cpu(fpsimd_last_state,
> task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> task->fpsimd_state.cpu's registers are also up to date for task, but
> not authorititive: the current FPSIMD/SVE state may be read from
> them, but they must not be written.
>
> The FPSIMD/SVE backing memory is selected by TIF_SVE:
>
> * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> stored in task->thread.sve_state, formatted appropriately for vector
> length task->thread.sve_vl. task->thread.sve_state must point to a
> valid buffer at least sve_state_size(task) bytes in size.
>
> * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> have unspecified values from userspace's point of view.
> task->thread.sve_state does not need to be non-null, valid or any
> particular size: it must not be dereferenced.
>
> In practice I don't exploit the "unspecifiedness" much. The Zn high
> bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> sve_alloc() is the common path for this.
>
> * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> whether TIF_SVE is clear or set, since these are not vector length
> dependent.
This looks fine. I think we need to make sure (with a warning) that
task_fpsimd_save() (and probably load) is always called in a
non-preemptible context.
> [*] theoretically unspecified, which is what I've written in sve.txt.
> However, on any FPSIMD Vn write the upper bits of the corresponding Zn
> must become logically zero in order to conform to the SVE programmer's
> model. It's not feasible to track which Vn have been written
> individually because that would involve trapping every FPSIMD insn until
> all possible Vn have been written. So the kernel ensures that the Zn
> high bits become zero.
>
> Maybe we should just guarantee "zero-or-preserved" behaviour for
> userspace. This may close down some future optimisation opportunities,
> but maybe it's better to be simple.
Does it work if we leave it as "unspecified" in the document but just do
zero-or-preserved in the kernel code?
Just wondering, as an optimisation for do_sve_acc() - instead of
sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
ensure the zeroing of the top SVE bits and we only need to allocate the
SVE state on the saving path. This means enabling SVE for user and
setting TIF_SVE without having the backing storage allocated.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
2017-08-31 17:00 ` Dave Martin
@ 2017-10-05 16:39 ` Szabolcs Nagy
-1 siblings, 0 replies; 224+ messages in thread
From: Szabolcs Nagy @ 2017-10-05 16:39 UTC (permalink / raw)
To: Dave Martin, linux-arm-kernel
Cc: nd, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Alex Bennée, Richard Sandiford, kvmarm, libc-alpha,
linux-arch, Mark Rutland
On 31/08/17 18:00, Dave Martin wrote:
> +prctl(PR_SVE_SET_VL, unsigned long arg)
> +
> + Sets the vector length of the calling thread and related flags, where
> + arg == vl | flags.
> +
> + vl is the desired vector length, where sve_vl_valid(vl) must be true.
> +
> + flags:
> +
> + PR_SVE_SET_VL_INHERIT
> +
> + Inherit the current vector length across execve(). Otherwise, the
> + vector length is reset to the system default at execve(). (See
> + Section 9.)
> +
> + PR_SVE_SET_VL_ONEXEC
> +
> + Defer the requested vector length change until the next execve().
> + This allows launching of a new program with a different vector
> + length, while avoiding runtime side effects in the caller.
> +
> + This also overrides the effect of PR_SVE_SET_VL_INHERIT for the
> + first execve().
> +
> + Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector
> + length change is cancelled.
> +
based on later text it seems this works if exeve is
called in the same thread as prctl was called in.
this is a bit weird from user-space pov so it may
make sense to state it here explicitly.
> + Return value: a nonnegative on success, or a negative value on error:
> + EINVAL: SVE not supported, invalid vector length requested, or
> + invalid flags.
> +
> + On success, the calling thread's vector length is changed to the largest
> + value supported by the system that is less than or equal to vl.
> + If vl == SVE_VL_MAX, the calling thread's vector length is changed to the
> + largest value supported by the system.
> +
> + The returned value describes the resulting configuration, encoded as for
> + PR_SVE_GET_VL.
> +
> + Changing the vector length causes all of P0..P15, FFR and all bits of
> + Z0..V31 except for Z0 bits [127:0] .. Z31 bits [127:0] to become
> + unspecified. Calling PR_SVE_SET_VL with vl equal to the thread's current
> + vector length does not constitute a change to the vector length for this
> + purpose.
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-05 16:39 ` Szabolcs Nagy
0 siblings, 0 replies; 224+ messages in thread
From: Szabolcs Nagy @ 2017-10-05 16:39 UTC (permalink / raw)
To: linux-arm-kernel
On 31/08/17 18:00, Dave Martin wrote:
> +prctl(PR_SVE_SET_VL, unsigned long arg)
> +
> + Sets the vector length of the calling thread and related flags, where
> + arg == vl | flags.
> +
> + vl is the desired vector length, where sve_vl_valid(vl) must be true.
> +
> + flags:
> +
> + PR_SVE_SET_VL_INHERIT
> +
> + Inherit the current vector length across execve(). Otherwise, the
> + vector length is reset to the system default at execve(). (See
> + Section 9.)
> +
> + PR_SVE_SET_VL_ONEXEC
> +
> + Defer the requested vector length change until the next execve().
> + This allows launching of a new program with a different vector
> + length, while avoiding runtime side effects in the caller.
> +
> + This also overrides the effect of PR_SVE_SET_VL_INHERIT for the
> + first execve().
> +
> + Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector
> + length change is cancelled.
> +
based on later text it seems this works if exeve is
called in the same thread as prctl was called in.
this is a bit weird from user-space pov so it may
make sense to state it here explicitly.
> + Return value: a nonnegative on success, or a negative value on error:
> + EINVAL: SVE not supported, invalid vector length requested, or
> + invalid flags.
> +
> + On success, the calling thread's vector length is changed to the largest
> + value supported by the system that is less than or equal to vl.
> + If vl == SVE_VL_MAX, the calling thread's vector length is changed to the
> + largest value supported by the system.
> +
> + The returned value describes the resulting configuration, encoded as for
> + PR_SVE_GET_VL.
> +
> + Changing the vector length causes all of P0..P15, FFR and all bits of
> + Z0..V31 except for Z0 bits [127:0] .. Z31 bits [127:0] to become
> + unspecified. Calling PR_SVE_SET_VL with vl equal to the thread's current
> + vector length does not constitute a change to the vector length for this
> + purpose.
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-09-13 22:11 ` Catalin Marinas
@ 2017-10-05 16:42 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 16:42 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy, gdb,
Yao Qi, Alan Hayward, Will Deacon, Richard Sandiford,
Alex Bennée, kvmarm, linux-arm-kernel
On Wed, Sep 13, 2017 at 03:11:23PM -0700, Catalin Marinas wrote:
> On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > > > This patch implements the core logic for changing a task's vector
> > > > length on request from userspace. This will be used by the ptrace
> > > > and prctl frontends that are implemented in later patches.
> > > >
> > > > The SVE architecture permits, but does not require, implementations
> > > > to support vector lengths that are not a power of two. To handle
> > > > this, logic is added to check a requested vector length against a
> > > > possibly sparse bitmap of available vector lengths at runtime, so
> > > > that the best supported value can be chosen.
> > > >
> > > > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > > > Cc: Alex Bennée <alex.bennee@linaro.org>
> > >
> > > Can this be merged with patch 20? It seems to add the PR_ definitions
> > > which get actually used later when the prctl interface is added.
> >
> > This patch is used both by patch 19 and by patch 20, which I preferred
> > not to merge with each other: ptrace and prctl are significantly
> > different things.
> >
> > The prctl bit definitions are added here because they are the canonical
> > definitions used by both interfaces. The ptrace #defines are based on
> > them.
> >
> > Does it make sense if I merge patch 20 into this one and apply patch 19
> > on top? This avoide the appearance of prctl #defines with no prctl
> > implementation.
>
> That's fine, you can bring patch 20 forward. If there are other
> non-trivial issues, feel free to ignore my comment.
I've had a go at this, but I think it's going to be more trouble than
it's worth -- there are other interdependencies between the patches
which make them tricky to reorder.
I could add a note in the commit message for this patch explaining why
the prctl flag #defines are being added here. What do you think?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-10-05 16:42 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 16:42 UTC (permalink / raw)
To: linux-arm-kernel
On Wed, Sep 13, 2017 at 03:11:23PM -0700, Catalin Marinas wrote:
> On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > > > This patch implements the core logic for changing a task's vector
> > > > length on request from userspace. This will be used by the ptrace
> > > > and prctl frontends that are implemented in later patches.
> > > >
> > > > The SVE architecture permits, but does not require, implementations
> > > > to support vector lengths that are not a power of two. To handle
> > > > this, logic is added to check a requested vector length against a
> > > > possibly sparse bitmap of available vector lengths at runtime, so
> > > > that the best supported value can be chosen.
> > > >
> > > > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > > > Cc: Alex Benn?e <alex.bennee@linaro.org>
> > >
> > > Can this be merged with patch 20? It seems to add the PR_ definitions
> > > which get actually used later when the prctl interface is added.
> >
> > This patch is used both by patch 19 and by patch 20, which I preferred
> > not to merge with each other: ptrace and prctl are significantly
> > different things.
> >
> > The prctl bit definitions are added here because they are the canonical
> > definitions used by both interfaces. The ptrace #defines are based on
> > them.
> >
> > Does it make sense if I merge patch 20 into this one and apply patch 19
> > on top? This avoide the appearance of prctl #defines with no prctl
> > implementation.
>
> That's fine, you can bring patch 20 forward. If there are other
> non-trivial issues, feel free to ignore my comment.
I've had a go at this, but I think it's going to be more trouble than
it's worth -- there are other interdependencies between the patches
which make them tricky to reorder.
I could add a note in the commit message for this patch explaining why
the prctl flag #defines are being added here. What do you think?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-10-05 16:42 ` Dave Martin
@ 2017-10-05 16:53 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-05 16:53 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy, gdb,
Yao Qi, Will Deacon, Richard Sandiford, Alan Hayward,
Alex Bennée, kvmarm, linux-arm-kernel
On Thu, Oct 05, 2017 at 05:42:29PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 03:11:23PM -0700, Catalin Marinas wrote:
> > On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> > > On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> > > > On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > > > > This patch implements the core logic for changing a task's vector
> > > > > length on request from userspace. This will be used by the ptrace
> > > > > and prctl frontends that are implemented in later patches.
> > > > >
> > > > > The SVE architecture permits, but does not require, implementations
> > > > > to support vector lengths that are not a power of two. To handle
> > > > > this, logic is added to check a requested vector length against a
> > > > > possibly sparse bitmap of available vector lengths at runtime, so
> > > > > that the best supported value can be chosen.
> > > > >
> > > > > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > > > > Cc: Alex Bennée <alex.bennee@linaro.org>
> > > >
> > > > Can this be merged with patch 20? It seems to add the PR_ definitions
> > > > which get actually used later when the prctl interface is added.
> > >
> > > This patch is used both by patch 19 and by patch 20, which I preferred
> > > not to merge with each other: ptrace and prctl are significantly
> > > different things.
> > >
> > > The prctl bit definitions are added here because they are the canonical
> > > definitions used by both interfaces. The ptrace #defines are based on
> > > them.
> > >
> > > Does it make sense if I merge patch 20 into this one and apply patch 19
> > > on top? This avoide the appearance of prctl #defines with no prctl
> > > implementation.
> >
> > That's fine, you can bring patch 20 forward. If there are other
> > non-trivial issues, feel free to ignore my comment.
>
> I've had a go at this, but I think it's going to be more trouble than
> it's worth -- there are other interdependencies between the patches
> which make them tricky to reorder.
>
> I could add a note in the commit message for this patch explaining why
> the prctl flag #defines are being added here. What do you think?
As I said, it's up to you. A line in the commit message would do.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-10-05 16:53 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-05 16:53 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 05, 2017 at 05:42:29PM +0100, Dave P Martin wrote:
> On Wed, Sep 13, 2017 at 03:11:23PM -0700, Catalin Marinas wrote:
> > On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> > > On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
> > > > On Thu, Aug 31, 2017 at 06:00:46PM +0100, Dave P Martin wrote:
> > > > > This patch implements the core logic for changing a task's vector
> > > > > length on request from userspace. This will be used by the ptrace
> > > > > and prctl frontends that are implemented in later patches.
> > > > >
> > > > > The SVE architecture permits, but does not require, implementations
> > > > > to support vector lengths that are not a power of two. To handle
> > > > > this, logic is added to check a requested vector length against a
> > > > > possibly sparse bitmap of available vector lengths at runtime, so
> > > > > that the best supported value can be chosen.
> > > > >
> > > > > Signed-off-by: Dave Martin <Dave.Martin@arm.com>
> > > > > Cc: Alex Benn?e <alex.bennee@linaro.org>
> > > >
> > > > Can this be merged with patch 20? It seems to add the PR_ definitions
> > > > which get actually used later when the prctl interface is added.
> > >
> > > This patch is used both by patch 19 and by patch 20, which I preferred
> > > not to merge with each other: ptrace and prctl are significantly
> > > different things.
> > >
> > > The prctl bit definitions are added here because they are the canonical
> > > definitions used by both interfaces. The ptrace #defines are based on
> > > them.
> > >
> > > Does it make sense if I merge patch 20 into this one and apply patch 19
> > > on top? This avoide the appearance of prctl #defines with no prctl
> > > implementation.
> >
> > That's fine, you can bring patch 20 forward. If there are other
> > non-trivial issues, feel free to ignore my comment.
>
> I've had a go at this, but I think it's going to be more trouble than
> it's worth -- there are other interdependencies between the patches
> which make them tricky to reorder.
>
> I could add a note in the commit message for this patch explaining why
> the prctl flag #defines are being added here. What do you think?
As I said, it's up to you. A line in the commit message would do.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
2017-10-05 16:39 ` Szabolcs Nagy
@ 2017-10-05 17:02 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 17:02 UTC (permalink / raw)
To: Szabolcs Nagy
Cc: linux-arm-kernel, linux-arch, Mark Rutland, libc-alpha,
Ard Biesheuvel, Catalin Marinas, Will Deacon, Richard Sandiford
On Thu, Oct 05, 2017 at 05:39:24PM +0100, Szabolcs Nagy wrote:
> On 31/08/17 18:00, Dave Martin wrote:
[...]
> > + PR_SVE_SET_VL_ONEXEC
> > +
> > + Defer the requested vector length change until the next execve().
> > + This allows launching of a new program with a different vector
> > + length, while avoiding runtime side effects in the caller.
> > +
> > + This also overrides the effect of PR_SVE_SET_VL_INHERIT for the
> > + first execve().
> > +
> > + Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector
> > + length change is cancelled.
> > +
>
> based on later text it seems this works if exeve is
> called in the same thread as prctl was called in.
>
> this is a bit weird from user-space pov so it may
> make sense to state it here explicitly.
True. Looking at the prctl(2) man page it looks like other per-thread
properties are inherited across execve() in a similar way, but it's at
least worth a mention. PR_SET_SECCOMP seems to work like this, for
example.
So, the intention is that you do a prctl(...ONEXEC) in the run up to
execve(), rather than doing it at other random times. The primary
reason for ONEXEC is to avoid the side-effects of actually changing
the VL.
Looking at this though...
I wonder whether PR_SVE_SET_VL(... PR_SVE_SET_VL_ONEXEC) should return
the VL set for exec, rather than the current VL (which is unchanged by
definition in this case, thus uninteresting).
This would allow the ONEXEC flag to be used to probe for available VLs
without the other side-effects of changing VL, something like:
int old = prctl(PR_SVE_GET_VL);
int ret;
ret = prctl(PR_SVE_SET_VL, 144 | PR_SVE_SET_VL_ONEXEC);
if (ret == -1) {
perror("PR_SVE_SET_VL");
goto error;
}
if ((ret & PR_SVE_VL_LEN_MASK) == 144)
have_vl_144 = true;
if (prctl(PR_SVE_SET_VL, old | PR_SVE_SET_VL_ONEXEC) == -1) {
perror("PR_SVE_SET_VL");
goto error;
}
This does _not_ do the expected thing right now, since there's no
direct way to retrieve thread.sve_vl_onexec directly from the kernel
(and it didn't really seem justified to add one).
Thoughts?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-05 17:02 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 17:02 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 05, 2017 at 05:39:24PM +0100, Szabolcs Nagy wrote:
> On 31/08/17 18:00, Dave Martin wrote:
[...]
> > + PR_SVE_SET_VL_ONEXEC
> > +
> > + Defer the requested vector length change until the next execve().
> > + This allows launching of a new program with a different vector
> > + length, while avoiding runtime side effects in the caller.
> > +
> > + This also overrides the effect of PR_SVE_SET_VL_INHERIT for the
> > + first execve().
> > +
> > + Without PR_SVE_SET_VL_ONEXEC, any outstanding deferred vector
> > + length change is cancelled.
> > +
>
> based on later text it seems this works if exeve is
> called in the same thread as prctl was called in.
>
> this is a bit weird from user-space pov so it may
> make sense to state it here explicitly.
True. Looking at the prctl(2) man page it looks like other per-thread
properties are inherited across execve() in a similar way, but it's at
least worth a mention. PR_SET_SECCOMP seems to work like this, for
example.
So, the intention is that you do a prctl(...ONEXEC) in the run up to
execve(), rather than doing it at other random times. The primary
reason for ONEXEC is to avoid the side-effects of actually changing
the VL.
Looking at this though...
I wonder whether PR_SVE_SET_VL(... PR_SVE_SET_VL_ONEXEC) should return
the VL set for exec, rather than the current VL (which is unchanged by
definition in this case, thus uninteresting).
This would allow the ONEXEC flag to be used to probe for available VLs
without the other side-effects of changing VL, something like:
int old = prctl(PR_SVE_GET_VL);
int ret;
ret = prctl(PR_SVE_SET_VL, 144 | PR_SVE_SET_VL_ONEXEC);
if (ret == -1) {
perror("PR_SVE_SET_VL");
goto error;
}
if ((ret & PR_SVE_VL_LEN_MASK) == 144)
have_vl_144 = true;
if (prctl(PR_SVE_SET_VL, old | PR_SVE_SET_VL_ONEXEC) == -1) {
perror("PR_SVE_SET_VL");
goto error;
}
This does _not_ do the expected thing right now, since there's no
direct way to retrieve thread.sve_vl_onexec directly from the kernel
(and it didn't really seem justified to add one).
Thoughts?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
2017-10-05 16:53 ` Catalin Marinas
@ 2017-10-05 17:04 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 17:04 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy, gdb,
Yao Qi, Alan Hayward, Will Deacon, Richard Sandiford,
Alex Bennée, kvmarm, linux-arm-kernel
On Thu, Oct 05, 2017 at 05:53:34PM +0100, Catalin Marinas wrote:
> On Thu, Oct 05, 2017 at 05:42:29PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 03:11:23PM -0700, Catalin Marinas wrote:
> > > On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> > > > On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
[...]
> > > > > Can this be merged with patch 20? It seems to add the PR_ definitions
> > > > > which get actually used later when the prctl interface is added.
> > > >
> > > > This patch is used both by patch 19 and by patch 20, which I preferred
> > > > not to merge with each other: ptrace and prctl are significantly
> > > > different things.
> > > >
> > > > The prctl bit definitions are added here because they are the canonical
> > > > definitions used by both interfaces. The ptrace #defines are based on
> > > > them.
> > > >
> > > > Does it make sense if I merge patch 20 into this one and apply patch 19
> > > > on top? This avoide the appearance of prctl #defines with no prctl
> > > > implementation.
> > >
> > > That's fine, you can bring patch 20 forward. If there are other
> > > non-trivial issues, feel free to ignore my comment.
> >
> > I've had a go at this, but I think it's going to be more trouble than
> > it's worth -- there are other interdependencies between the patches
> > which make them tricky to reorder.
> >
> > I could add a note in the commit message for this patch explaining why
> > the prctl flag #defines are being added here. What do you think?
>
> As I said, it's up to you. A line in the commit message would do.
OK, I think I'll stick with this then.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length
@ 2017-10-05 17:04 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-05 17:04 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 05, 2017 at 05:53:34PM +0100, Catalin Marinas wrote:
> On Thu, Oct 05, 2017 at 05:42:29PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 03:11:23PM -0700, Catalin Marinas wrote:
> > > On Wed, Sep 13, 2017 at 08:06:12PM +0100, Dave P Martin wrote:
> > > > On Wed, Sep 13, 2017 at 10:29:11AM -0700, Catalin Marinas wrote:
[...]
> > > > > Can this be merged with patch 20? It seems to add the PR_ definitions
> > > > > which get actually used later when the prctl interface is added.
> > > >
> > > > This patch is used both by patch 19 and by patch 20, which I preferred
> > > > not to merge with each other: ptrace and prctl are significantly
> > > > different things.
> > > >
> > > > The prctl bit definitions are added here because they are the canonical
> > > > definitions used by both interfaces. The ptrace #defines are based on
> > > > them.
> > > >
> > > > Does it make sense if I merge patch 20 into this one and apply patch 19
> > > > on top? This avoide the appearance of prctl #defines with no prctl
> > > > implementation.
> > >
> > > That's fine, you can bring patch 20 forward. If there are other
> > > non-trivial issues, feel free to ignore my comment.
> >
> > I've had a go at this, but I think it's going to be more trouble than
> > it's worth -- there are other interdependencies between the patches
> > which make them tricky to reorder.
> >
> > I could add a note in the commit message for this patch explaining why
> > the prctl flag #defines are being added here. What do you think?
>
> As I said, it's up to you. A line in the commit message would do.
OK, I think I'll stick with this then.
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-10-05 11:28 ` Catalin Marinas
@ 2017-10-06 13:10 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-06 13:10 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > +/*
> > > > + * Handle SVE state across fork():
> > > > + *
> > > > + * dst and src must not end up with aliases of the same sve_state.
> > > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > > + * will be deferred until dst tries to use SVE.
> > > > + */
> > > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > > +{
> > > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > > + sve_to_fpsimd(dst);
> > > > + }
> > > > +
> > > > + dst->thread.sve_state = NULL;
> > > > +}
> > >
> > > I first thought the thread flags are not visible in dst yet since
> > > dup_task_struct() calls arch_dup_task_struct() before
> > > setup_thread_stack(). However, at the end of the last year we enabled
> > > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > > on this.
> > >
> > > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> > > for src, so the FPSIMD state (which we care about) is transferred during
> > > the *dst = *src assignment. So you'd only need the last statement,
> > > possibly with a different function name like fpsimd_erase_sve (and maybe
> > > make the function static inline in the header).
> >
> > Regarding the intended meanings of the thread flags, does this help?
> >
> > --8<--
> >
> > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > unchanged:
> >
> > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > registers of the CPU the task is running on contain the authoritative
> > FPSIMD/SVE state of the task. The backing memory may be stale.
> >
> > * Otherwise (i.e., task not running, or task running and
> > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > authoritative. If additionally per_cpu(fpsimd_last_state,
> > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > task->fpsimd_state.cpu's registers are also up to date for task, but
> > not authorititive: the current FPSIMD/SVE state may be read from
> > them, but they must not be written.
> >
> > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> >
> > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > stored in task->thread.sve_state, formatted appropriately for vector
> > length task->thread.sve_vl. task->thread.sve_state must point to a
> > valid buffer at least sve_state_size(task) bytes in size.
> >
> > * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> > logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> > have unspecified values from userspace's point of view.
> > task->thread.sve_state does not need to be non-null, valid or any
> > particular size: it must not be dereferenced.
> >
> > In practice I don't exploit the "unspecifiedness" much. The Zn high
> > bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> > sve_alloc() is the common path for this.
> >
> > * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> > whether TIF_SVE is clear or set, since these are not vector length
> > dependent.
>
> This looks fine. I think we need to make sure (with a warning) that
> task_fpsimd_save() (and probably load) is always called in a
> non-preemptible context.
>
> > [*] theoretically unspecified, which is what I've written in sve.txt.
> > However, on any FPSIMD Vn write the upper bits of the corresponding Zn
> > must become logically zero in order to conform to the SVE programmer's
> > model. It's not feasible to track which Vn have been written
> > individually because that would involve trapping every FPSIMD insn until
> > all possible Vn have been written. So the kernel ensures that the Zn
> > high bits become zero.
> >
> > Maybe we should just guarantee "zero-or-preserved" behaviour for
> > userspace. This may close down some future optimisation opportunities,
> > but maybe it's better to be simple.
>
> Does it work if we leave it as "unspecified" in the document but just do
> zero-or-preserved in the kernel code?
Sure, that's the behaviour today in effect.
I'll leave the documentation unchanged, then we can take advantage of
this flexibility later if is proves to be useful.
> Just wondering, as an optimisation for do_sve_acc() - instead of
> sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
> FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
> ensure the zeroing of the top SVE bits and we only need to allocate the
> SVE state on the saving path. This means enabling SVE for user and
> setting TIF_SVE without having the backing storage allocated.
Currently the set of places where the "TIF_SVE implies sve_state valid"
assumption is applied is not very constrained, so while your suggestion
is reasonable I'd rather not mess with it just now, if possible.
But we can do this (which is what my current fixup has):
el0_sve_acc:
enable_dbg_and_irq
// ...
bl do_sve_acc
b ret_to_user
void do_sve_acc(unsigned int esr, struct pt_regs *regs)
{
/* Even if we chose not to use SVE, the hardware could still trap: */
if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
return;
}
sve_alloc(current);
local_bh_disable();
if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
task_fpsimd_load(); /* flushes high Zn bits as a side-effect */
sve_flush_pregs();
} else {
sve_flush_all(); /* flush all the SVE bits in-place */
}
if (test_and_set_thread_flag(TIF_SVE))
WARN_ON(1); /* SVE access shouldn't have trapped */
local_bh_enable();
}
where sve_flush_all() zeroes all the high Zn bits via a series of
MOV Vn, Vn instructions, and also zeroes Pn and FFR. sve_fplush_pregs()
just does the latter.
In the common case the sve_alloc() does a memzero, but doesn't
allocate memory because more often than not, it will already have
been allocated.
On this path, the memzero is now pointless -- I'll need to double-check
what else may be impacted by its removal (probably only ptrace, and I'm
not sure if the zeroing is strictly required there).
There is still a bit of room for improvement, but it's still a step up
from v2.
What do you think?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-06 13:10 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-06 13:10 UTC (permalink / raw)
To: linux-arm-kernel
On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > On Wed, Sep 13, 2017 at 07:33:25AM -0700, Catalin Marinas wrote:
> > > On Thu, Aug 31, 2017 at 06:00:43PM +0100, Dave P Martin wrote:
> > > > +/*
> > > > + * Handle SVE state across fork():
> > > > + *
> > > > + * dst and src must not end up with aliases of the same sve_state.
> > > > + * Because a task cannot fork except in a syscall, we can discard SVE
> > > > + * state for dst here, so long as we take care to retain the FPSIMD
> > > > + * subset of the state if SVE is in use. Reallocation of the SVE state
> > > > + * will be deferred until dst tries to use SVE.
> > > > + */
> > > > +void fpsimd_dup_sve(struct task_struct *dst, struct task_struct const *src)
> > > > +{
> > > > + if (test_and_clear_tsk_thread_flag(dst, TIF_SVE)) {
> > > > + WARN_ON(dst->mm && !in_syscall(task_pt_regs(dst)));
> > > > + sve_to_fpsimd(dst);
> > > > + }
> > > > +
> > > > + dst->thread.sve_state = NULL;
> > > > +}
> > >
> > > I first thought the thread flags are not visible in dst yet since
> > > dup_task_struct() calls arch_dup_task_struct() before
> > > setup_thread_stack(). However, at the end of the last year we enabled
> > > CONFIG_THREAD_INFO_IN_TASK_STRUCT. But I don't particularly like relying
> > > on this.
> > >
> > > Anyway, IIUC we don't need sve_to_fpsimd() here. The
> > > arch_dup_task_struct() already called fpsimd_preserve_current_state()
> > > for src, so the FPSIMD state (which we care about) is transferred during
> > > the *dst = *src assignment. So you'd only need the last statement,
> > > possibly with a different function name like fpsimd_erase_sve (and maybe
> > > make the function static inline in the header).
> >
> > Regarding the intended meanings of the thread flags, does this help?
> >
> > --8<--
> >
> > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > unchanged:
> >
> > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > registers of the CPU the task is running on contain the authoritative
> > FPSIMD/SVE state of the task. The backing memory may be stale.
> >
> > * Otherwise (i.e., task not running, or task running and
> > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > authoritative. If additionally per_cpu(fpsimd_last_state,
> > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > task->fpsimd_state.cpu's registers are also up to date for task, but
> > not authorititive: the current FPSIMD/SVE state may be read from
> > them, but they must not be written.
> >
> > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> >
> > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > stored in task->thread.sve_state, formatted appropriately for vector
> > length task->thread.sve_vl. task->thread.sve_state must point to a
> > valid buffer at least sve_state_size(task) bytes in size.
> >
> > * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> > logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> > have unspecified values from userspace's point of view.
> > task->thread.sve_state does not need to be non-null, valid or any
> > particular size: it must not be dereferenced.
> >
> > In practice I don't exploit the "unspecifiedness" much. The Zn high
> > bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> > sve_alloc() is the common path for this.
> >
> > * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> > whether TIF_SVE is clear or set, since these are not vector length
> > dependent.
>
> This looks fine. I think we need to make sure (with a warning) that
> task_fpsimd_save() (and probably load) is always called in a
> non-preemptible context.
>
> > [*] theoretically unspecified, which is what I've written in sve.txt.
> > However, on any FPSIMD Vn write the upper bits of the corresponding Zn
> > must become logically zero in order to conform to the SVE programmer's
> > model. It's not feasible to track which Vn have been written
> > individually because that would involve trapping every FPSIMD insn until
> > all possible Vn have been written. So the kernel ensures that the Zn
> > high bits become zero.
> >
> > Maybe we should just guarantee "zero-or-preserved" behaviour for
> > userspace. This may close down some future optimisation opportunities,
> > but maybe it's better to be simple.
>
> Does it work if we leave it as "unspecified" in the document but just do
> zero-or-preserved in the kernel code?
Sure, that's the behaviour today in effect.
I'll leave the documentation unchanged, then we can take advantage of
this flexibility later if is proves to be useful.
> Just wondering, as an optimisation for do_sve_acc() - instead of
> sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
> FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
> ensure the zeroing of the top SVE bits and we only need to allocate the
> SVE state on the saving path. This means enabling SVE for user and
> setting TIF_SVE without having the backing storage allocated.
Currently the set of places where the "TIF_SVE implies sve_state valid"
assumption is applied is not very constrained, so while your suggestion
is reasonable I'd rather not mess with it just now, if possible.
But we can do this (which is what my current fixup has):
el0_sve_acc:
enable_dbg_and_irq
// ...
bl do_sve_acc
b ret_to_user
void do_sve_acc(unsigned int esr, struct pt_regs *regs)
{
/* Even if we chose not to use SVE, the hardware could still trap: */
if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
return;
}
sve_alloc(current);
local_bh_disable();
if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
task_fpsimd_load(); /* flushes high Zn bits as a side-effect */
sve_flush_pregs();
} else {
sve_flush_all(); /* flush all the SVE bits in-place */
}
if (test_and_set_thread_flag(TIF_SVE))
WARN_ON(1); /* SVE access shouldn't have trapped */
local_bh_enable();
}
where sve_flush_all() zeroes all the high Zn bits via a series of
MOV Vn, Vn instructions, and also zeroes Pn and FFR. sve_fplush_pregs()
just does the latter.
In the common case the sve_alloc() does a memzero, but doesn't
allocate memory because more often than not, it will already have
been allocated.
On this path, the memzero is now pointless -- I'll need to double-check
what else may be impacted by its removal (probably only ptrace, and I'm
not sure if the zeroing is strictly required there).
There is still a bit of room for improvement, but it's still a step up
from v2.
What do you think?
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-10-06 13:10 ` Dave Martin
@ 2017-10-06 13:36 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-06 13:36 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Fri, Oct 06, 2017 at 02:10:09PM +0100, Dave P Martin wrote:
> On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> > On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > > unchanged:
> > >
> > > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > > registers of the CPU the task is running on contain the authoritative
> > > FPSIMD/SVE state of the task. The backing memory may be stale.
> > >
> > > * Otherwise (i.e., task not running, or task running and
> > > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > > authoritative. If additionally per_cpu(fpsimd_last_state,
> > > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > > task->fpsimd_state.cpu's registers are also up to date for task, but
> > > not authorititive: the current FPSIMD/SVE state may be read from
> > > them, but they must not be written.
> > >
> > > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> > >
> > > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > > stored in task->thread.sve_state, formatted appropriately for vector
> > > length task->thread.sve_vl. task->thread.sve_state must point to a
> > > valid buffer at least sve_state_size(task) bytes in size.
"Zn [...] stored in task->thread.sve_state" - is this still true with
the changes you proposed? I guess even without these changes, you have
situations where the hardware regs are out of sync with sve_state (see
more below).
> > > * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> > > logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> > > have unspecified values from userspace's point of view.
> > > task->thread.sve_state does not need to be non-null, valid or any
> > > particular size: it must not be dereferenced.
> > >
> > > In practice I don't exploit the "unspecifiedness" much. The Zn high
> > > bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> > > sve_alloc() is the common path for this.
> > >
> > > * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> > > whether TIF_SVE is clear or set, since these are not vector length
> > > dependent.
[...]
> > Just wondering, as an optimisation for do_sve_acc() - instead of
> > sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
> > FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
> > ensure the zeroing of the top SVE bits and we only need to allocate the
> > SVE state on the saving path. This means enabling SVE for user and
> > setting TIF_SVE without having the backing storage allocated.
>
> Currently the set of places where the "TIF_SVE implies sve_state valid"
> assumption is applied is not very constrained, so while your suggestion
> is reasonable I'd rather not mess with it just now, if possible.
>
>
> But we can do this (which is what my current fixup has):
>
> el0_sve_acc:
> enable_dbg_and_irq
> // ...
> bl do_sve_acc
> b ret_to_user
>
> void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> {
> /* Even if we chose not to use SVE, the hardware could still trap: */
> if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> return;
> }
>
> sve_alloc(current);
>
> local_bh_disable();
> if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
> task_fpsimd_load(); /* flushes high Zn bits as a side-effect */
> sve_flush_pregs();
> } else {
> sve_flush_all(); /* flush all the SVE bits in-place */
> }
>
> if (test_and_set_thread_flag(TIF_SVE))
> WARN_ON(1); /* SVE access shouldn't have trapped */
> local_bh_enable();
> }
>
> where sve_flush_all() zeroes all the high Zn bits via a series of
> MOV Vn, Vn instructions, and also zeroes Pn and FFR. sve_fplush_pregs()
> just does the latter.
This looks fine to me but I added a comment above. IIUC, we can now have
TIF_SVE set while sve_state contains stale data. I don't see an issue
given that every time you enter the kernel from user space you have
TIF_SVE set and the sve_state storage out of sync. Maybe tweak the
TIF_SVE description above slightly.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-06 13:36 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-06 13:36 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Oct 06, 2017 at 02:10:09PM +0100, Dave P Martin wrote:
> On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> > On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > > unchanged:
> > >
> > > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > > registers of the CPU the task is running on contain the authoritative
> > > FPSIMD/SVE state of the task. The backing memory may be stale.
> > >
> > > * Otherwise (i.e., task not running, or task running and
> > > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > > authoritative. If additionally per_cpu(fpsimd_last_state,
> > > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > > task->fpsimd_state.cpu's registers are also up to date for task, but
> > > not authorititive: the current FPSIMD/SVE state may be read from
> > > them, but they must not be written.
> > >
> > > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> > >
> > > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > > stored in task->thread.sve_state, formatted appropriately for vector
> > > length task->thread.sve_vl. task->thread.sve_state must point to a
> > > valid buffer at least sve_state_size(task) bytes in size.
"Zn [...] stored in task->thread.sve_state" - is this still true with
the changes you proposed? I guess even without these changes, you have
situations where the hardware regs are out of sync with sve_state (see
more below).
> > > * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> > > logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> > > have unspecified values from userspace's point of view.
> > > task->thread.sve_state does not need to be non-null, valid or any
> > > particular size: it must not be dereferenced.
> > >
> > > In practice I don't exploit the "unspecifiedness" much. The Zn high
> > > bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> > > sve_alloc() is the common path for this.
> > >
> > > * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> > > whether TIF_SVE is clear or set, since these are not vector length
> > > dependent.
[...]
> > Just wondering, as an optimisation for do_sve_acc() - instead of
> > sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
> > FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
> > ensure the zeroing of the top SVE bits and we only need to allocate the
> > SVE state on the saving path. This means enabling SVE for user and
> > setting TIF_SVE without having the backing storage allocated.
>
> Currently the set of places where the "TIF_SVE implies sve_state valid"
> assumption is applied is not very constrained, so while your suggestion
> is reasonable I'd rather not mess with it just now, if possible.
>
>
> But we can do this (which is what my current fixup has):
>
> el0_sve_acc:
> enable_dbg_and_irq
> // ...
> bl do_sve_acc
> b ret_to_user
>
> void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> {
> /* Even if we chose not to use SVE, the hardware could still trap: */
> if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> return;
> }
>
> sve_alloc(current);
>
> local_bh_disable();
> if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
> task_fpsimd_load(); /* flushes high Zn bits as a side-effect */
> sve_flush_pregs();
> } else {
> sve_flush_all(); /* flush all the SVE bits in-place */
> }
>
> if (test_and_set_thread_flag(TIF_SVE))
> WARN_ON(1); /* SVE access shouldn't have trapped */
> local_bh_enable();
> }
>
> where sve_flush_all() zeroes all the high Zn bits via a series of
> MOV Vn, Vn instructions, and also zeroes Pn and FFR. sve_fplush_pregs()
> just does the latter.
This looks fine to me but I added a comment above. IIUC, we can now have
TIF_SVE set while sve_state contains stale data. I don't see an issue
given that every time you enter the kernel from user space you have
TIF_SVE set and the sve_state storage out of sync. Maybe tweak the
TIF_SVE description above slightly.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-10-06 13:36 ` Catalin Marinas
@ 2017-10-06 15:15 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-06 15:15 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Fri, Oct 06, 2017 at 02:36:40PM +0100, Catalin Marinas wrote:
> On Fri, Oct 06, 2017 at 02:10:09PM +0100, Dave P Martin wrote:
> > On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> > > On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > > > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > > > unchanged:
> > > >
> > > > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > > > registers of the CPU the task is running on contain the authoritative
> > > > FPSIMD/SVE state of the task. The backing memory may be stale.
> > > >
> > > > * Otherwise (i.e., task not running, or task running and
> > > > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > > > authoritative. If additionally per_cpu(fpsimd_last_state,
> > > > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > > > task->fpsimd_state.cpu's registers are also up to date for task, but
> > > > not authorititive: the current FPSIMD/SVE state may be read from
> > > > them, but they must not be written.
> > > >
> > > > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> > > >
> > > > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > > > stored in task->thread.sve_state, formatted appropriately for vector
> > > > length task->thread.sve_vl. task->thread.sve_state must point to a
> > > > valid buffer at least sve_state_size(task) bytes in size.
>
> "Zn [...] stored in task->thread.sve_state" - is this still true with
> the changes you proposed? I guess even without these changes, you have
> situations where the hardware regs are out of sync with sve_state (see
> more below).
I guess I need to tweak the wording here.
TIF_SVE says where the vector state should be loaded/stored from,
but does not say whether the data is up to date in memory, or when
it should be loaded/stored.
The latter is described by a cocktail of different things including
which bit of kernel code we are executing if any, whether the task
is running/stopped etc., TIF_FOREIGN_FPSTATE,
task->thread.fpsimd_state.cpu and per_cpu(fpsimd_last_state).
Does this make any better sense of my code below?
>
> > > > * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> > > > logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> > > > have unspecified values from userspace's point of view.
> > > > task->thread.sve_state does not need to be non-null, valid or any
> > > > particular size: it must not be dereferenced.
> > > >
> > > > In practice I don't exploit the "unspecifiedness" much. The Zn high
> > > > bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> > > > sve_alloc() is the common path for this.
> > > >
> > > > * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> > > > whether TIF_SVE is clear or set, since these are not vector length
> > > > dependent.
> [...]
> > > Just wondering, as an optimisation for do_sve_acc() - instead of
> > > sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
> > > FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
> > > ensure the zeroing of the top SVE bits and we only need to allocate the
> > > SVE state on the saving path. This means enabling SVE for user and
> > > setting TIF_SVE without having the backing storage allocated.
> >
> > Currently the set of places where the "TIF_SVE implies sve_state valid"
> > assumption is applied is not very constrained, so while your suggestion
> > is reasonable I'd rather not mess with it just now, if possible.
> >
> >
> > But we can do this (which is what my current fixup has):
> >
> > el0_sve_acc:
> > enable_dbg_and_irq
> > // ...
> > bl do_sve_acc
> > b ret_to_user
> >
> > void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > {
> > /* Even if we chose not to use SVE, the hardware could still trap: */
> > if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > return;
> > }
> >
> > sve_alloc(current);
> >
> > local_bh_disable();
> > if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
> > task_fpsimd_load(); /* flushes high Zn bits as a side-effect */
> > sve_flush_pregs();
> > } else {
> > sve_flush_all(); /* flush all the SVE bits in-place */
> > }
> >
> > if (test_and_set_thread_flag(TIF_SVE))
> > WARN_ON(1); /* SVE access shouldn't have trapped */
> > local_bh_enable();
> > }
> >
> > where sve_flush_all() zeroes all the high Zn bits via a series of
> > MOV Vn, Vn instructions, and also zeroes Pn and FFR. sve_fplush_pregs()
> > just does the latter.
>
> This looks fine to me but I added a comment above. IIUC, we can now have
> TIF_SVE set while sve_state contains stale data. I don't see an issue
> given that every time you enter the kernel from user space you have
> TIF_SVE set and the sve_state storage out of sync. Maybe tweak the
> TIF_SVE description above slightly.
>
See my comment above ... any better?
If so, I'll paste some of that explanatory text into fpsimd.c (in lieu
of a better place to put it).
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-06 15:15 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-06 15:15 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Oct 06, 2017 at 02:36:40PM +0100, Catalin Marinas wrote:
> On Fri, Oct 06, 2017 at 02:10:09PM +0100, Dave P Martin wrote:
> > On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> > > On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > > > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > > > unchanged:
> > > >
> > > > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > > > registers of the CPU the task is running on contain the authoritative
> > > > FPSIMD/SVE state of the task. The backing memory may be stale.
> > > >
> > > > * Otherwise (i.e., task not running, or task running and
> > > > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > > > authoritative. If additionally per_cpu(fpsimd_last_state,
> > > > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > > > task->fpsimd_state.cpu's registers are also up to date for task, but
> > > > not authorititive: the current FPSIMD/SVE state may be read from
> > > > them, but they must not be written.
> > > >
> > > > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> > > >
> > > > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > > > stored in task->thread.sve_state, formatted appropriately for vector
> > > > length task->thread.sve_vl. task->thread.sve_state must point to a
> > > > valid buffer at least sve_state_size(task) bytes in size.
>
> "Zn [...] stored in task->thread.sve_state" - is this still true with
> the changes you proposed? I guess even without these changes, you have
> situations where the hardware regs are out of sync with sve_state (see
> more below).
I guess I need to tweak the wording here.
TIF_SVE says where the vector state should be loaded/stored from,
but does not say whether the data is up to date in memory, or when
it should be loaded/stored.
The latter is described by a cocktail of different things including
which bit of kernel code we are executing if any, whether the task
is running/stopped etc., TIF_FOREIGN_FPSTATE,
task->thread.fpsimd_state.cpu and per_cpu(fpsimd_last_state).
Does this make any better sense of my code below?
>
> > > > * TIF_SVE clear: Vn are stored in task->fpsimd_state; Zn[max : 128] are
> > > > logically zero[*] but not stored anywhere; Pn, FFR are not stored and
> > > > have unspecified values from userspace's point of view.
> > > > task->thread.sve_state does not need to be non-null, valid or any
> > > > particular size: it must not be dereferenced.
> > > >
> > > > In practice I don't exploit the "unspecifiedness" much. The Zn high
> > > > bits, Pn and FFR are all zeroed when setting TIF_SVE again:
> > > > sve_alloc() is the common path for this.
> > > >
> > > > * FPSR and FPCR are always stored in task->fpsimd_state irrespctive of
> > > > whether TIF_SVE is clear or set, since these are not vector length
> > > > dependent.
> [...]
> > > Just wondering, as an optimisation for do_sve_acc() - instead of
> > > sve_alloc() and fpsimd_to_sve(), can we not force the loading of the
> > > FPSIMD regs on the return to user via TIF_FOREIGN_FPSTATE? This would
> > > ensure the zeroing of the top SVE bits and we only need to allocate the
> > > SVE state on the saving path. This means enabling SVE for user and
> > > setting TIF_SVE without having the backing storage allocated.
> >
> > Currently the set of places where the "TIF_SVE implies sve_state valid"
> > assumption is applied is not very constrained, so while your suggestion
> > is reasonable I'd rather not mess with it just now, if possible.
> >
> >
> > But we can do this (which is what my current fixup has):
> >
> > el0_sve_acc:
> > enable_dbg_and_irq
> > // ...
> > bl do_sve_acc
> > b ret_to_user
> >
> > void do_sve_acc(unsigned int esr, struct pt_regs *regs)
> > {
> > /* Even if we chose not to use SVE, the hardware could still trap: */
> > if (unlikely(!system_supports_sve()) || WARN_ON(is_compat_task())) {
> > force_signal_inject(SIGILL, ILL_ILLOPC, regs, 0);
> > return;
> > }
> >
> > sve_alloc(current);
> >
> > local_bh_disable();
> > if (test_and_clear_thread_flag(TIF_FOREIGN_FPSTATE)) {
> > task_fpsimd_load(); /* flushes high Zn bits as a side-effect */
> > sve_flush_pregs();
> > } else {
> > sve_flush_all(); /* flush all the SVE bits in-place */
> > }
> >
> > if (test_and_set_thread_flag(TIF_SVE))
> > WARN_ON(1); /* SVE access shouldn't have trapped */
> > local_bh_enable();
> > }
> >
> > where sve_flush_all() zeroes all the high Zn bits via a series of
> > MOV Vn, Vn instructions, and also zeroes Pn and FFR. sve_fplush_pregs()
> > just does the latter.
>
> This looks fine to me but I added a comment above. IIUC, we can now have
> TIF_SVE set while sve_state contains stale data. I don't see an issue
> given that every time you enter the kernel from user space you have
> TIF_SVE set and the sve_state storage out of sync. Maybe tweak the
> TIF_SVE description above slightly.
>
See my comment above ... any better?
If so, I'll paste some of that explanatory text into fpsimd.c (in lieu
of a better place to put it).
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 11/28] arm64/sve: Core task context handling
2017-10-06 15:15 ` Dave Martin
@ 2017-10-06 15:33 ` Catalin Marinas
-1 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-06 15:33 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Richard Sandiford, Will Deacon, Alex Bennée, kvmarm,
linux-arm-kernel
On Fri, Oct 06, 2017 at 04:15:28PM +0100, Dave P Martin wrote:
> On Fri, Oct 06, 2017 at 02:36:40PM +0100, Catalin Marinas wrote:
> > On Fri, Oct 06, 2017 at 02:10:09PM +0100, Dave P Martin wrote:
> > > On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> > > > On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > > > > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > > > > unchanged:
> > > > >
> > > > > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > > > > registers of the CPU the task is running on contain the authoritative
> > > > > FPSIMD/SVE state of the task. The backing memory may be stale.
> > > > >
> > > > > * Otherwise (i.e., task not running, or task running and
> > > > > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > > > > authoritative. If additionally per_cpu(fpsimd_last_state,
> > > > > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > > > > task->fpsimd_state.cpu's registers are also up to date for task, but
> > > > > not authorititive: the current FPSIMD/SVE state may be read from
> > > > > them, but they must not be written.
> > > > >
> > > > > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> > > > >
> > > > > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > > > > stored in task->thread.sve_state, formatted appropriately for vector
> > > > > length task->thread.sve_vl. task->thread.sve_state must point to a
> > > > > valid buffer at least sve_state_size(task) bytes in size.
> >
> > "Zn [...] stored in task->thread.sve_state" - is this still true with
> > the changes you proposed? I guess even without these changes, you have
> > situations where the hardware regs are out of sync with sve_state (see
> > more below).
>
> I guess I need to tweak the wording here.
>
> TIF_SVE says where the vector state should be loaded/stored from,
> but does not say whether the data is up to date in memory, or when
> it should be loaded/stored.
>
> The latter is described by a cocktail of different things including
> which bit of kernel code we are executing if any, whether the task
> is running/stopped etc., TIF_FOREIGN_FPSTATE,
> task->thread.fpsimd_state.cpu and per_cpu(fpsimd_last_state).
>
> Does this make any better sense of my code below?
Yes, it looks fine.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 11/28] arm64/sve: Core task context handling
@ 2017-10-06 15:33 ` Catalin Marinas
0 siblings, 0 replies; 224+ messages in thread
From: Catalin Marinas @ 2017-10-06 15:33 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Oct 06, 2017 at 04:15:28PM +0100, Dave P Martin wrote:
> On Fri, Oct 06, 2017 at 02:36:40PM +0100, Catalin Marinas wrote:
> > On Fri, Oct 06, 2017 at 02:10:09PM +0100, Dave P Martin wrote:
> > > On Thu, Oct 05, 2017 at 12:28:35PM +0100, Catalin Marinas wrote:
> > > > On Tue, Oct 03, 2017 at 12:33:03PM +0100, Dave P Martin wrote:
> > > > > TIF_FOREIGN_FPSTATE's meaning is expanded to cover SVE, but otherwise
> > > > > unchanged:
> > > > >
> > > > > * If a task is running and !TIF_FOREIGN_FPSTATE, then the the CPU
> > > > > registers of the CPU the task is running on contain the authoritative
> > > > > FPSIMD/SVE state of the task. The backing memory may be stale.
> > > > >
> > > > > * Otherwise (i.e., task not running, or task running and
> > > > > TIF_FOREIGN_FPSTATE), the task's FPSIMD/SVE backing memory is
> > > > > authoritative. If additionally per_cpu(fpsimd_last_state,
> > > > > task->fpsimd_state.cpu) == &task->fpsimd_state.cpu, then
> > > > > task->fpsimd_state.cpu's registers are also up to date for task, but
> > > > > not authorititive: the current FPSIMD/SVE state may be read from
> > > > > them, but they must not be written.
> > > > >
> > > > > The FPSIMD/SVE backing memory is selected by TIF_SVE:
> > > > >
> > > > > * TIF_SVE set: Zn (incorporating Vn in bits[127:0]), Pn and FFR are
> > > > > stored in task->thread.sve_state, formatted appropriately for vector
> > > > > length task->thread.sve_vl. task->thread.sve_state must point to a
> > > > > valid buffer at least sve_state_size(task) bytes in size.
> >
> > "Zn [...] stored in task->thread.sve_state" - is this still true with
> > the changes you proposed? I guess even without these changes, you have
> > situations where the hardware regs are out of sync with sve_state (see
> > more below).
>
> I guess I need to tweak the wording here.
>
> TIF_SVE says where the vector state should be loaded/stored from,
> but does not say whether the data is up to date in memory, or when
> it should be loaded/stored.
>
> The latter is described by a cocktail of different things including
> which bit of kernel code we are executing if any, whether the task
> is running/stopped etc., TIF_FOREIGN_FPSTATE,
> task->thread.fpsimd_state.cpu and per_cpu(fpsimd_last_state).
>
> Does this make any better sense of my code below?
Yes, it looks fine.
--
Catalin
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
2017-08-31 17:00 ` Dave Martin
@ 2017-10-06 15:43 ` Szabolcs Nagy
-1 siblings, 0 replies; 224+ messages in thread
From: Szabolcs Nagy @ 2017-10-06 15:43 UTC (permalink / raw)
To: Dave Martin, linux-arm-kernel
Cc: nd, Catalin Marinas, Will Deacon, Ard Biesheuvel,
Alex Bennée, Richard Sandiford, kvmarm, libc-alpha,
linux-arch, Mark Rutland
On 31/08/17 18:00, Dave Martin wrote:
> +9. System runtime configuration
> +--------------------------------
> +
> +* To mitigate the ABI impact of expansion of the signal frame, a policy
> + mechanism is provided for administrators, distro maintainers and developers
> + to set the default vector length for userspace processes:
> +
> +/proc/cpu/sve_default_vector_length
elsewhere in the patch series i see
/proc/sys/abi/sve_default_vector_length
is this supposed to be the same?
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-06 15:43 ` Szabolcs Nagy
0 siblings, 0 replies; 224+ messages in thread
From: Szabolcs Nagy @ 2017-10-06 15:43 UTC (permalink / raw)
To: linux-arm-kernel
On 31/08/17 18:00, Dave Martin wrote:
> +9. System runtime configuration
> +--------------------------------
> +
> +* To mitigate the ABI impact of expansion of the signal frame, a policy
> + mechanism is provided for administrators, distro maintainers and developers
> + to set the default vector length for userspace processes:
> +
> +/proc/cpu/sve_default_vector_length
elsewhere in the patch series i see
/proc/sys/abi/sve_default_vector_length
is this supposed to be the same?
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
2017-10-06 15:43 ` Szabolcs Nagy
@ 2017-10-06 17:37 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-06 17:37 UTC (permalink / raw)
To: Szabolcs Nagy
Cc: linux-arm-kernel, linux-arch, Mark Rutland, libc-alpha,
Ard Biesheuvel, Catalin Marinas, Will Deacon, Richard Sandiford
On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
> On 31/08/17 18:00, Dave Martin wrote:
> > +9. System runtime configuration
> > +--------------------------------
> > +
> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
> > + mechanism is provided for administrators, distro maintainers and developers
> > + to set the default vector length for userspace processes:
> > +
> > +/proc/cpu/sve_default_vector_length
>
>
> elsewhere in the patch series i see
>
> /proc/sys/abi/sve_default_vector_length
>
> is this supposed to be the same?
Good spot, thanks!
/proc/cpu/ was the old location: they should both say /proc/abi/.
I'll fix it.
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-06 17:37 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-06 17:37 UTC (permalink / raw)
To: linux-arm-kernel
On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
> On 31/08/17 18:00, Dave Martin wrote:
> > +9. System runtime configuration
> > +--------------------------------
> > +
> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
> > + mechanism is provided for administrators, distro maintainers and developers
> > + to set the default vector length for userspace processes:
> > +
> > +/proc/cpu/sve_default_vector_length
>
>
> elsewhere in the patch series i see
>
> /proc/sys/abi/sve_default_vector_length
>
> is this supposed to be the same?
Good spot, thanks!
/proc/cpu/ was the old location: they should both say /proc/abi/.
I'll fix it.
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 9:34 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-10-09 9:34 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, libc-alpha, Ard Biesheuvel, Szabolcs Nagy,
Catalin Marinas, Will Deacon, Richard Sandiford, nd, kvmarm,
linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
>> On 31/08/17 18:00, Dave Martin wrote:
>> > +9. System runtime configuration
>> > +--------------------------------
>> > +
>> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
>> > + mechanism is provided for administrators, distro maintainers and developers
>> > + to set the default vector length for userspace processes:
>> > +
>> > +/proc/cpu/sve_default_vector_length
>>
>>
>> elsewhere in the patch series i see
>>
>> /proc/sys/abi/sve_default_vector_length
>>
>> is this supposed to be the same?
>
> Good spot, thanks!
>
> /proc/cpu/ was the old location: they should both say /proc/abi/.
> I'll fix it.
Isn't /sys (or rather sysfs) the preferred location for modern control
knobs that mirror the kernels object model or is SVE a special case for
extending /proc?
>
> ---Dave
--
Alex Bennée
_______________________________________________
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 9:34 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-10-09 9:34 UTC (permalink / raw)
To: Dave Martin
Cc: Szabolcs Nagy, linux-arm-kernel, linux-arch, Mark Rutland,
libc-alpha, Ard Biesheuvel, Catalin Marinas, Will Deacon,
Richard Sandiford
Dave Martin <Dave.Martin@arm.com> writes:
> On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
>> On 31/08/17 18:00, Dave Martin wrote:
>> > +9. System runtime configuration
>> > +--------------------------------
>> > +
>> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
>> > + mechanism is provided for administrators, distro maintainers and developers
>> > + to set the default vector length for userspace processes:
>> > +
>> > +/proc/cpu/sve_default_vector_length
>>
>>
>> elsewhere in the patch series i see
>>
>> /proc/sys/abi/sve_default_vector_length
>>
>> is this supposed to be the same?
>
> Good spot, thanks!
>
> /proc/cpu/ was the old location: they should both say /proc/abi/.
> I'll fix it.
Isn't /sys (or rather sysfs) the preferred location for modern control
knobs that mirror the kernels object model or is SVE a special case for
extending /proc?
>
> ---Dave
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 9:34 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-10-09 9:34 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
>> On 31/08/17 18:00, Dave Martin wrote:
>> > +9. System runtime configuration
>> > +--------------------------------
>> > +
>> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
>> > + mechanism is provided for administrators, distro maintainers and developers
>> > + to set the default vector length for userspace processes:
>> > +
>> > +/proc/cpu/sve_default_vector_length
>>
>>
>> elsewhere in the patch series i see
>>
>> /proc/sys/abi/sve_default_vector_length
>>
>> is this supposed to be the same?
>
> Good spot, thanks!
>
> /proc/cpu/ was the old location: they should both say /proc/abi/.
> I'll fix it.
Isn't /sys (or rather sysfs) the preferred location for modern control
knobs that mirror the kernels object model or is SVE a special case for
extending /proc?
>
> ---Dave
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
2017-10-09 9:34 ` Alex Bennée
@ 2017-10-09 9:49 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-09 9:49 UTC (permalink / raw)
To: Alex Bennée
Cc: linux-arch, Mark Rutland, libc-alpha, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Will Deacon, Richard Sandiford
On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Bennée wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
> >> On 31/08/17 18:00, Dave Martin wrote:
> >> > +9. System runtime configuration
> >> > +--------------------------------
> >> > +
> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
> >> > + mechanism is provided for administrators, distro maintainers and developers
> >> > + to set the default vector length for userspace processes:
> >> > +
> >> > +/proc/cpu/sve_default_vector_length
> >>
> >>
> >> elsewhere in the patch series i see
> >>
> >> /proc/sys/abi/sve_default_vector_length
> >>
> >> is this supposed to be the same?
> >
> > Good spot, thanks!
> >
> > /proc/cpu/ was the old location: they should both say /proc/abi/.
> > I'll fix it.
>
> Isn't /sys (or rather sysfs) the preferred location for modern control
> knobs that mirror the kernels object model or is SVE a special case for
> extending /proc?
I couldn't figure out which kernel object this maps to. There's no
device, no driver. This isn't even per-cpu.
sysctl is already used for similar knobs to this one, so I followed that
precedent -- though if someone argues strongly enough it could be
changed.
Are there already examples of arch controls like this in sysfs? I
wasn't aware of any, but I didn't look all that hard...
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 9:49 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-09 9:49 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Benn?e wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
> >> On 31/08/17 18:00, Dave Martin wrote:
> >> > +9. System runtime configuration
> >> > +--------------------------------
> >> > +
> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
> >> > + mechanism is provided for administrators, distro maintainers and developers
> >> > + to set the default vector length for userspace processes:
> >> > +
> >> > +/proc/cpu/sve_default_vector_length
> >>
> >>
> >> elsewhere in the patch series i see
> >>
> >> /proc/sys/abi/sve_default_vector_length
> >>
> >> is this supposed to be the same?
> >
> > Good spot, thanks!
> >
> > /proc/cpu/ was the old location: they should both say /proc/abi/.
> > I'll fix it.
>
> Isn't /sys (or rather sysfs) the preferred location for modern control
> knobs that mirror the kernels object model or is SVE a special case for
> extending /proc?
I couldn't figure out which kernel object this maps to. There's no
device, no driver. This isn't even per-cpu.
sysctl is already used for similar knobs to this one, so I followed that
precedent -- though if someone argues strongly enough it could be
changed.
Are there already examples of arch controls like this in sysfs? I
wasn't aware of any, but I didn't look all that hard...
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 14:07 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-10-09 14:07 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, Mark Rutland, libc-alpha, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Will Deacon, Richard Sandiford
Dave Martin <Dave.Martin@arm.com> writes:
> On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Bennée wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
>> >> On 31/08/17 18:00, Dave Martin wrote:
>> >> > +9. System runtime configuration
>> >> > +--------------------------------
>> >> > +
>> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
>> >> > + mechanism is provided for administrators, distro maintainers and developers
>> >> > + to set the default vector length for userspace processes:
>> >> > +
>> >> > +/proc/cpu/sve_default_vector_length
>> >>
>> >>
>> >> elsewhere in the patch series i see
>> >>
>> >> /proc/sys/abi/sve_default_vector_length
>> >>
>> >> is this supposed to be the same?
>> >
>> > Good spot, thanks!
>> >
>> > /proc/cpu/ was the old location: they should both say /proc/abi/.
>> > I'll fix it.
>>
>> Isn't /sys (or rather sysfs) the preferred location for modern control
>> knobs that mirror the kernels object model or is SVE a special case for
>> extending /proc?
>
> I couldn't figure out which kernel object this maps to. There's no
> device, no driver. This isn't even per-cpu.
Hmm I can see:
/sys/devices/system/cpu
On both my x86 and arm64 systems - but I guess this is more ABIish than
CPU feature related.
> sysctl is already used for similar knobs to this one, so I followed that
> precedent -- though if someone argues strongly enough it could be
> changed.
>
> Are there already examples of arch controls like this in sysfs? I
> wasn't aware of any, but I didn't look all that hard...
Given the paucity of the /proc/sys/abi on both systems I guess this sort
of knob is rare enough that people haven't expressed a strong preference
for sysfs here. I have no objection to staying with /proc/sys/abi/.
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 14:07 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-10-09 14:07 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arch, Mark Rutland, libc-alpha, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Will Deacon, Richard Sandiford
Dave Martin <Dave.Martin@arm.com> writes:
> On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Bennée wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
>> >> On 31/08/17 18:00, Dave Martin wrote:
>> >> > +9. System runtime configuration
>> >> > +--------------------------------
>> >> > +
>> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
>> >> > + mechanism is provided for administrators, distro maintainers and developers
>> >> > + to set the default vector length for userspace processes:
>> >> > +
>> >> > +/proc/cpu/sve_default_vector_length
>> >>
>> >>
>> >> elsewhere in the patch series i see
>> >>
>> >> /proc/sys/abi/sve_default_vector_length
>> >>
>> >> is this supposed to be the same?
>> >
>> > Good spot, thanks!
>> >
>> > /proc/cpu/ was the old location: they should both say /proc/abi/.
>> > I'll fix it.
>>
>> Isn't /sys (or rather sysfs) the preferred location for modern control
>> knobs that mirror the kernels object model or is SVE a special case for
>> extending /proc?
>
> I couldn't figure out which kernel object this maps to. There's no
> device, no driver. This isn't even per-cpu.
Hmm I can see:
/sys/devices/system/cpu
On both my x86 and arm64 systems - but I guess this is more ABIish than
CPU feature related.
> sysctl is already used for similar knobs to this one, so I followed that
> precedent -- though if someone argues strongly enough it could be
> changed.
>
> Are there already examples of arch controls like this in sysfs? I
> wasn't aware of any, but I didn't look all that hard...
Given the paucity of the /proc/sys/abi on both systems I guess this sort
of knob is rare enough that people haven't expressed a strong preference
for sysfs here. I have no objection to staying with /proc/sys/abi/.
--
Alex Bennée
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 14:07 ` Alex Bennée
0 siblings, 0 replies; 224+ messages in thread
From: Alex Bennée @ 2017-10-09 14:07 UTC (permalink / raw)
To: linux-arm-kernel
Dave Martin <Dave.Martin@arm.com> writes:
> On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Benn?e wrote:
>>
>> Dave Martin <Dave.Martin@arm.com> writes:
>>
>> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
>> >> On 31/08/17 18:00, Dave Martin wrote:
>> >> > +9. System runtime configuration
>> >> > +--------------------------------
>> >> > +
>> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
>> >> > + mechanism is provided for administrators, distro maintainers and developers
>> >> > + to set the default vector length for userspace processes:
>> >> > +
>> >> > +/proc/cpu/sve_default_vector_length
>> >>
>> >>
>> >> elsewhere in the patch series i see
>> >>
>> >> /proc/sys/abi/sve_default_vector_length
>> >>
>> >> is this supposed to be the same?
>> >
>> > Good spot, thanks!
>> >
>> > /proc/cpu/ was the old location: they should both say /proc/abi/.
>> > I'll fix it.
>>
>> Isn't /sys (or rather sysfs) the preferred location for modern control
>> knobs that mirror the kernels object model or is SVE a special case for
>> extending /proc?
>
> I couldn't figure out which kernel object this maps to. There's no
> device, no driver. This isn't even per-cpu.
Hmm I can see:
/sys/devices/system/cpu
On both my x86 and arm64 systems - but I guess this is more ABIish than
CPU feature related.
> sysctl is already used for similar knobs to this one, so I followed that
> precedent -- though if someone argues strongly enough it could be
> changed.
>
> Are there already examples of arch controls like this in sysfs? I
> wasn't aware of any, but I didn't look all that hard...
Given the paucity of the /proc/sys/abi on both systems I guess this sort
of knob is rare enough that people haven't expressed a strong preference
for sysfs here. I have no objection to staying with /proc/sys/abi/.
--
Alex Benn?e
^ permalink raw reply [flat|nested] 224+ messages in thread
* Re: [PATCH v2 26/28] arm64/sve: Add documentation
2017-10-09 14:07 ` Alex Bennée
@ 2017-10-09 16:20 ` Dave Martin
-1 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-09 16:20 UTC (permalink / raw)
To: Alex Bennée
Cc: linux-arch, Mark Rutland, libc-alpha, Ard Biesheuvel,
Szabolcs Nagy, Catalin Marinas, Will Deacon, Richard Sandiford
On Mon, Oct 09, 2017 at 03:07:23PM +0100, Alex Bennée wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Bennée wrote:
> >>
> >> Dave Martin <Dave.Martin@arm.com> writes:
> >>
> >> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
> >> >> On 31/08/17 18:00, Dave Martin wrote:
> >> >> > +9. System runtime configuration
> >> >> > +--------------------------------
> >> >> > +
> >> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
> >> >> > + mechanism is provided for administrators, distro maintainers and developers
> >> >> > + to set the default vector length for userspace processes:
> >> >> > +
> >> >> > +/proc/cpu/sve_default_vector_length
> >> >>
> >> >>
> >> >> elsewhere in the patch series i see
> >> >>
> >> >> /proc/sys/abi/sve_default_vector_length
> >> >>
> >> >> is this supposed to be the same?
> >> >
> >> > Good spot, thanks!
> >> >
> >> > /proc/cpu/ was the old location: they should both say /proc/abi/.
> >> > I'll fix it.
> >>
> >> Isn't /sys (or rather sysfs) the preferred location for modern control
> >> knobs that mirror the kernels object model or is SVE a special case for
> >> extending /proc?
> >
> > I couldn't figure out which kernel object this maps to. There's no
> > device, no driver. This isn't even per-cpu.
>
> Hmm I can see:
>
> /sys/devices/system/cpu
>
> On both my x86 and arm64 systems - but I guess this is more ABIish than
> CPU feature related.
>
> > sysctl is already used for similar knobs to this one, so I followed that
> > precedent -- though if someone argues strongly enough it could be
> > changed.
> >
> > Are there already examples of arch controls like this in sysfs? I
> > wasn't aware of any, but I didn't look all that hard...
>
> Given the paucity of the /proc/sys/abi on both systems I guess this sort
> of knob is rare enough that people haven't expressed a strong preference
> for sysfs here. I have no objection to staying with /proc/sys/abi/.
That was my thinking: sysctls tend to control the kernel, especially
process behaviour, whereas /sys/ controls devices and subsystems.
That's not a concrete rule though and not written down, and doubtless a
major new set of sysctls would be shot down regardless of what they do.
Part of the problem with /proc is that people historically put things in
there that have random ad-hoc behaviour and semantics. The sysctl
framework at least imposes some sanity here.
There is also some support for specialising sysctls to user namespaces,
which makes some sense in that /proc/sys/abi/* should probably be per-
container -- though whether it's ever considered important enough to
actually be implemented is another question. I certainly don't attempt
to do it today.
I don't know how sysfs interacts with namespaces, but probably it can.
I guess I'll wait for someone to object loudly...
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
* [PATCH v2 26/28] arm64/sve: Add documentation
@ 2017-10-09 16:20 ` Dave Martin
0 siblings, 0 replies; 224+ messages in thread
From: Dave Martin @ 2017-10-09 16:20 UTC (permalink / raw)
To: linux-arm-kernel
On Mon, Oct 09, 2017 at 03:07:23PM +0100, Alex Benn?e wrote:
>
> Dave Martin <Dave.Martin@arm.com> writes:
>
> > On Mon, Oct 09, 2017 at 10:34:25AM +0100, Alex Benn?e wrote:
> >>
> >> Dave Martin <Dave.Martin@arm.com> writes:
> >>
> >> > On Fri, Oct 06, 2017 at 04:43:43PM +0100, Szabolcs Nagy wrote:
> >> >> On 31/08/17 18:00, Dave Martin wrote:
> >> >> > +9. System runtime configuration
> >> >> > +--------------------------------
> >> >> > +
> >> >> > +* To mitigate the ABI impact of expansion of the signal frame, a policy
> >> >> > + mechanism is provided for administrators, distro maintainers and developers
> >> >> > + to set the default vector length for userspace processes:
> >> >> > +
> >> >> > +/proc/cpu/sve_default_vector_length
> >> >>
> >> >>
> >> >> elsewhere in the patch series i see
> >> >>
> >> >> /proc/sys/abi/sve_default_vector_length
> >> >>
> >> >> is this supposed to be the same?
> >> >
> >> > Good spot, thanks!
> >> >
> >> > /proc/cpu/ was the old location: they should both say /proc/abi/.
> >> > I'll fix it.
> >>
> >> Isn't /sys (or rather sysfs) the preferred location for modern control
> >> knobs that mirror the kernels object model or is SVE a special case for
> >> extending /proc?
> >
> > I couldn't figure out which kernel object this maps to. There's no
> > device, no driver. This isn't even per-cpu.
>
> Hmm I can see:
>
> /sys/devices/system/cpu
>
> On both my x86 and arm64 systems - but I guess this is more ABIish than
> CPU feature related.
>
> > sysctl is already used for similar knobs to this one, so I followed that
> > precedent -- though if someone argues strongly enough it could be
> > changed.
> >
> > Are there already examples of arch controls like this in sysfs? I
> > wasn't aware of any, but I didn't look all that hard...
>
> Given the paucity of the /proc/sys/abi on both systems I guess this sort
> of knob is rare enough that people haven't expressed a strong preference
> for sysfs here. I have no objection to staying with /proc/sys/abi/.
That was my thinking: sysctls tend to control the kernel, especially
process behaviour, whereas /sys/ controls devices and subsystems.
That's not a concrete rule though and not written down, and doubtless a
major new set of sysctls would be shot down regardless of what they do.
Part of the problem with /proc is that people historically put things in
there that have random ad-hoc behaviour and semantics. The sysctl
framework at least imposes some sanity here.
There is also some support for specialising sysctls to user namespaces,
which makes some sense in that /proc/sys/abi/* should probably be per-
container -- though whether it's ever considered important enough to
actually be implemented is another question. I certainly don't attempt
to do it today.
I don't know how sysfs interacts with namespaces, but probably it can.
I guess I'll wait for someone to object loudly...
Cheers
---Dave
^ permalink raw reply [flat|nested] 224+ messages in thread
end of thread, other threads:[~2017-10-09 16:20 UTC | newest]
Thread overview: 224+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-08-31 17:00 [PATCH v2 00/28] ARM Scalable Vector Extension (SVE) Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 01/28] regset: Add support for dynamically sized regsets Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 02/28] arm64: KVM: Hide unsupported AArch64 CPU features from guests Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-13 14:37 ` Alex Bennée
2017-09-13 14:37 ` Alex Bennée
2017-09-13 14:37 ` Alex Bennée
2017-09-15 0:04 ` Dave Martin
2017-09-15 0:04 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 03/28] arm64: efi: Add missing Kconfig dependency on KERNEL_MODE_NEON Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 04/28] arm64: Port deprecated instruction emulation to new sysctl interface Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 05/28] arm64: fpsimd: Simplify uses of {set,clear}_ti_thread_flag() Dave Martin
2017-08-31 17:00 ` [PATCH v2 05/28] arm64: fpsimd: Simplify uses of {set, clear}_ti_thread_flag() Dave Martin
2017-08-31 17:00 ` [PATCH v2 06/28] arm64/sve: System register and exception syndrome definitions Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-13 14:48 ` Alex Bennée
2017-09-13 14:48 ` Alex Bennée
2017-09-13 14:48 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 07/28] arm64/sve: Low-level SVE architectural state manipulation functions Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-13 15:39 ` Alex Bennée
2017-09-13 15:39 ` Alex Bennée
2017-09-13 15:39 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 08/28] arm64/sve: Kconfig update and conditional compilation support Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 09/28] arm64/sve: Signal frame and context structure definition Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-13 13:36 ` Catalin Marinas
2017-09-13 13:36 ` Catalin Marinas
2017-09-13 21:33 ` Dave Martin
2017-09-13 21:33 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 10/28] arm64/sve: Low-level CPU setup Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-13 13:32 ` Catalin Marinas
2017-09-13 13:32 ` Catalin Marinas
2017-09-13 19:21 ` Dave Martin
2017-09-13 19:21 ` Dave Martin
2017-09-13 19:21 ` Dave Martin
2017-10-05 10:47 ` Dave Martin
2017-10-05 10:47 ` Dave Martin
2017-10-05 11:04 ` Suzuki K Poulose
2017-10-05 11:04 ` Suzuki K Poulose
2017-10-05 11:22 ` Dave Martin
2017-10-05 11:22 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 11/28] arm64/sve: Core task context handling Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-13 14:33 ` Catalin Marinas
2017-09-13 14:33 ` Catalin Marinas
2017-09-14 19:55 ` Dave Martin
2017-09-14 19:55 ` Dave Martin
2017-09-20 13:58 ` Catalin Marinas
2017-09-20 13:58 ` Catalin Marinas
2017-10-03 11:11 ` Dave Martin
2017-10-03 11:11 ` Dave Martin
2017-10-04 17:29 ` Catalin Marinas
2017-10-04 17:29 ` Catalin Marinas
2017-10-03 11:33 ` Dave Martin
2017-10-03 11:33 ` Dave Martin
2017-10-05 11:28 ` Catalin Marinas
2017-10-05 11:28 ` Catalin Marinas
2017-10-06 13:10 ` Dave Martin
2017-10-06 13:10 ` Dave Martin
2017-10-06 13:36 ` Catalin Marinas
2017-10-06 13:36 ` Catalin Marinas
2017-10-06 15:15 ` Dave Martin
2017-10-06 15:15 ` Dave Martin
2017-10-06 15:33 ` Catalin Marinas
2017-10-06 15:33 ` Catalin Marinas
2017-09-13 17:26 ` Catalin Marinas
2017-09-13 17:26 ` Catalin Marinas
2017-09-13 19:17 ` Dave Martin
2017-09-13 19:17 ` Dave Martin
2017-09-13 22:21 ` Catalin Marinas
2017-09-13 22:21 ` Catalin Marinas
2017-09-14 19:40 ` Dave Martin
2017-09-14 19:40 ` Dave Martin
2017-09-19 17:13 ` Catalin Marinas
2017-09-19 17:13 ` Catalin Marinas
2017-08-31 17:00 ` [PATCH v2 12/28] arm64/sve: Support vector length resetting for new processes Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 8:47 ` Alex Bennée
2017-09-14 8:47 ` Alex Bennée
2017-09-14 8:47 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 13/28] arm64/sve: Signal handling support Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 9:30 ` Alex Bennée
2017-09-14 9:30 ` Alex Bennée
2017-09-14 9:30 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 14/28] arm64/sve: Backend logic for setting the vector length Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-13 17:29 ` Catalin Marinas
2017-09-13 17:29 ` Catalin Marinas
2017-09-13 19:06 ` Dave Martin
2017-09-13 19:06 ` Dave Martin
2017-09-13 22:11 ` Catalin Marinas
2017-09-13 22:11 ` Catalin Marinas
2017-10-05 16:42 ` Dave Martin
2017-10-05 16:42 ` Dave Martin
2017-10-05 16:53 ` Catalin Marinas
2017-10-05 16:53 ` Catalin Marinas
2017-10-05 17:04 ` Dave Martin
2017-10-05 17:04 ` Dave Martin
2017-09-20 10:57 ` Alan Hayward
2017-09-20 10:57 ` Alan Hayward
2017-09-20 10:59 ` Alan Hayward
2017-09-20 10:59 ` Alan Hayward
2017-09-20 11:09 ` Dave Martin
2017-09-20 11:09 ` Dave Martin
2017-09-20 18:08 ` Alan Hayward
2017-09-20 18:08 ` Alan Hayward
2017-09-21 11:19 ` Dave Martin
2017-09-21 11:19 ` Dave Martin
2017-09-21 11:57 ` Alan Hayward
2017-09-21 11:57 ` Alan Hayward
2017-08-31 17:00 ` [PATCH v2 15/28] arm64: cpufeature: Move sys_caps_initialised declarations Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 9:33 ` Alex Bennée
2017-09-14 9:33 ` Alex Bennée
2017-09-14 9:33 ` Alex Bennée
2017-09-14 9:35 ` Suzuki K Poulose
2017-09-14 9:35 ` Suzuki K Poulose
2017-08-31 17:00 ` [PATCH v2 16/28] arm64/sve: Probe SVE capabilities and usable vector lengths Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 9:45 ` Alex Bennée
2017-09-14 9:45 ` Alex Bennée
2017-09-14 9:45 ` Alex Bennée
2017-09-28 14:22 ` Dave Martin
2017-09-28 14:22 ` Dave Martin
2017-09-28 17:32 ` Alex Bennée
2017-09-28 17:32 ` Alex Bennée
2017-09-28 17:32 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 17/28] arm64/sve: Preserve SVE registers around kernel-mode NEON use Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 10:52 ` Alex Bennée
2017-09-14 10:52 ` Alex Bennée
2017-09-14 10:52 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 18/28] arm64/sve: Preserve SVE registers around EFI runtime service calls Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 11:01 ` Alex Bennée
2017-09-14 11:01 ` Alex Bennée
2017-09-14 11:01 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 19/28] arm64/sve: ptrace and ELF coredump support Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-06 16:21 ` Okamoto, Takayuki
2017-09-06 16:21 ` Okamoto, Takayuki
2017-09-06 18:16 ` Dave Martin
2017-09-06 18:16 ` Dave Martin
2017-09-07 5:11 ` Okamoto, Takayuki
2017-09-07 5:11 ` Okamoto, Takayuki
2017-09-07 5:11 ` Okamoto, Takayuki
2017-09-08 13:11 ` Dave Martin
2017-09-08 13:11 ` Dave Martin
2017-09-14 12:57 ` Alex Bennée
2017-09-14 12:57 ` Alex Bennée
2017-09-14 12:57 ` Alex Bennée
2017-09-28 14:57 ` Dave Martin
2017-09-28 14:57 ` Dave Martin
2017-09-29 12:46 ` Dave Martin
2017-09-29 12:46 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 20/28] arm64/sve: Add prctl controls for userspace vector length management Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 13:02 ` Alex Bennée
2017-09-14 13:02 ` Alex Bennée
2017-09-14 13:02 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 21/28] arm64/sve: Add sysctl to set the default vector length for new processes Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 13:05 ` Alex Bennée
2017-09-14 13:05 ` Alex Bennée
2017-09-14 13:05 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 22/28] arm64/sve: KVM: Prevent guests from using SVE Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 13:28 ` Alex Bennée
2017-09-14 13:28 ` Alex Bennée
2017-09-14 13:28 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 23/28] arm64/sve: KVM: Treat guest SVE use as undefined instruction execution Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 13:30 ` Alex Bennée
2017-09-14 13:30 ` Alex Bennée
2017-09-14 13:30 ` Alex Bennée
2017-09-14 13:31 ` Alex Bennée
2017-09-14 13:31 ` Alex Bennée
2017-09-14 13:31 ` Alex Bennée
2017-09-29 13:00 ` Dave Martin
2017-09-29 13:00 ` Dave Martin
2017-09-29 14:43 ` Alex Bennée
2017-09-29 14:43 ` Alex Bennée
2017-09-29 14:43 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 24/28] arm64/sve: KVM: Hide SVE from CPU features exposed to guests Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-09-14 13:32 ` Alex Bennée
2017-09-14 13:32 ` Alex Bennée
2017-09-14 13:32 ` Alex Bennée
2017-08-31 17:00 ` [PATCH v2 25/28] arm64/sve: Detect SVE and activate runtime support Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` [PATCH v2 26/28] arm64/sve: Add documentation Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-10-05 16:39 ` Szabolcs Nagy
2017-10-05 16:39 ` Szabolcs Nagy
2017-10-05 17:02 ` Dave Martin
2017-10-05 17:02 ` Dave Martin
2017-10-06 15:43 ` Szabolcs Nagy
2017-10-06 15:43 ` Szabolcs Nagy
2017-10-06 17:37 ` Dave Martin
2017-10-06 17:37 ` Dave Martin
2017-10-09 9:34 ` Alex Bennée
2017-10-09 9:34 ` Alex Bennée
2017-10-09 9:34 ` Alex Bennée
2017-10-09 9:49 ` Dave Martin
2017-10-09 9:49 ` Dave Martin
2017-10-09 14:07 ` Alex Bennée
2017-10-09 14:07 ` Alex Bennée
2017-10-09 14:07 ` Alex Bennée
2017-10-09 16:20 ` Dave Martin
2017-10-09 16:20 ` Dave Martin
2017-08-31 17:00 ` [RFC PATCH v2 27/28] arm64: signal: Report signal frame size to userspace via auxv Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:00 ` Dave Martin
2017-08-31 17:01 ` [RFC PATCH v2 28/28] arm64/sve: signal: Include SVE when computing AT_MINSIGSTKSZ Dave Martin
2017-08-31 17:01 ` Dave Martin
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.