* [PATCH v4 00/22] arm64: Rework cpu capabilities handling
@ 2018-03-13 11:50 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:50 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
This series reworks the arm64 CPU capabilities handling (which
manages the system features and errata). The current infrastructure
doesn't allow fine control for handling different features or errata.
There is one rule for features and another rule for errata.
* Features are checked only once, after all the boot time CPUs are
activated. Any new CPU that is brought up is prevented from booting
if it misses a feature already established. If the new CPU has a
feature not enabled already, it is allowed to boot.
* Errata checks are performed on all the CPUs and any new CPU is
OK to miss the capability. However if a late CPU requires a work around,
then we fail the CPU.
This doesn't always apply to some features. e.g, KPTI is a security
feature which should be applied when at least one CPU needs it. So,
the tests should be performed on all the booting CPUs individually.
Also, if a CPU that needs this security feature is brought up later,
when the system has not enabled it, the CPU can boot making the system
insecure. Another exception is the hardware DBM for page tables. The
kernel can safely run with a mix of CPUs that have the feature turned
on and off. This again causes problem when a new CPU is brought up
which may not have the feature, which is killed.
Also there are other features like, GICV3 system register access,
which now need to be enabled very early based on the boot CPU to
allow the use of Priority handling to implement NMI.
This calls for finer level of control per capability and the series
implements the same by defining how to deal with a conflict of a
given capability on a CPU with that of the system level state. It
also consolidates the handling of features and errata into generic
helpers. The table of features and errata are left as they are to
allow easier look up for a given type.
The series also gets rid of duplicate entries for a single capability
by introducing a wrapper entry which takes care of managing a list
of entries with distinct matches/enable pair.
We also cleans up the MIDR range handling and cleans up some of the
errata checks where the entries were duplicated for checking different
CPU models. Finally it also implements a work around for Arm Cortex-A55
erratum 1024718 based on the new infrastructure.
Applies on arm64 for-next/core. The tree is available at :
git://linux-arm.org/linux-skp.git cpu-cap-rework/v4
Changes since V3
- Rebased to for-next/core (twice internally huh!)
- Address review comments from Dave
- Rename cap_list => match_list in Shared capability handling.
- Rename cpu_has_erratum_1024718() to cpu_has_broken_dbm() in
Patch 22, Erratum 1024718, for sharing the routine for different
CPUs which could be affected.
Changes since V2
- Mostly address review comments from Dave
- Drop "Change the type for BP hardening to accept late CPUs"
as we can't fake the capability, which is exposed to KVM guests.
- Split the "arm64: capabilities: Group handling of features and
errata" to smaller patches.
- Add tags from Dave.
- Use SCOPE_* in the Code instead of the ARM64_CPUCAP_SCOPE_* version.
- Hide description for DBM feature, as we turn the capability on always
irrespective of whether the CPUs have it or not.
Changes since V1
- Pickup almost all suggestions by Dave.
- Rename flags for handling conflicts
- Rename the "enable" call back to cpu_enable
- Update prototype for cpu_enable to void.
- Handle capabilities with multiple table entries, simplifying the
core logic.
- Add capabilities based on Boot CPU.
- Change the type for BP Hardening to accept late CPUs.
- More usersfor midr_range list.
- More commentary in the code.
- Flip type of Software prefetching capability to Weak from Strict.
Dave Martin (1):
arm64: capabilities: Update prototype for enable call back
Suzuki K Poulose (21):
arm64: capabilities: Move errata work around check on boot CPU
arm64: capabilities: Move errata processing code
arm64: capabilities: Prepare for fine grained capabilities
arm64: capabilities: Add flags to handle the conflicts on late CPU
arm64: capabilities: Unify the verification
arm64: capabilities: Filter the entries based on a given mask
arm64: capabilities: Prepare for grouping features and errata work
arounds
arm64: capabilities: Split the processing of errata work arounds
arm64: capabilities: Allow features based on local CPU scope
arm64: capabilities: Group handling of features and errata workarounds
arm64: capabilities: Introduce weak features based on local CPU
arm64: capabilities: Restrict KPTI detection to boot-time CPUs
arm64: capabilities: Add support for features enabled early
arm64: capabilities: Change scope of VHE to Boot CPU feature
arm64: capabilities: Clean up midr range helpers
arm64: Add helpers for checking CPU MIDR against a range
arm64: capabilities: Add support for checks based on a list of MIDRs
arm64: capabilities: Handle shared entries
arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35
arm64: Delay enabling hardware DBM feature
arm64: Add work around for Arm Cortex-A55 Erratum 1024718
Documentation/arm64/silicon-errata.txt | 1 +
arch/arm64/Kconfig | 14 ++
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/cpufeature.h | 258 ++++++++++++++++++++--
arch/arm64/include/asm/cputype.h | 43 ++++
arch/arm64/include/asm/fpsimd.h | 4 +-
arch/arm64/include/asm/processor.h | 7 +-
arch/arm64/include/asm/virt.h | 6 -
arch/arm64/kernel/cpu_errata.c | 294 +++++++++++++------------
arch/arm64/kernel/cpufeature.c | 381 +++++++++++++++++++++++++--------
arch/arm64/kernel/fpsimd.c | 5 +-
arch/arm64/kernel/smp.c | 44 ----
arch/arm64/kernel/traps.c | 4 +-
arch/arm64/mm/fault.c | 3 +-
arch/arm64/mm/proc.S | 13 +-
15 files changed, 769 insertions(+), 311 deletions(-)
--
2.14.3
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 00/22] arm64: Rework cpu capabilities handling
@ 2018-03-13 11:50 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:50 UTC (permalink / raw)
To: linux-arm-kernel
This series reworks the arm64 CPU capabilities handling (which
manages the system features and errata). The current infrastructure
doesn't allow fine control for handling different features or errata.
There is one rule for features and another rule for errata.
* Features are checked only once, after all the boot time CPUs are
activated. Any new CPU that is brought up is prevented from booting
if it misses a feature already established. If the new CPU has a
feature not enabled already, it is allowed to boot.
* Errata checks are performed on all the CPUs and any new CPU is
OK to miss the capability. However if a late CPU requires a work around,
then we fail the CPU.
This doesn't always apply to some features. e.g, KPTI is a security
feature which should be applied when at least one CPU needs it. So,
the tests should be performed on all the booting CPUs individually.
Also, if a CPU that needs this security feature is brought up later,
when the system has not enabled it, the CPU can boot making the system
insecure. Another exception is the hardware DBM for page tables. The
kernel can safely run with a mix of CPUs that have the feature turned
on and off. This again causes problem when a new CPU is brought up
which may not have the feature, which is killed.
Also there are other features like, GICV3 system register access,
which now need to be enabled very early based on the boot CPU to
allow the use of Priority handling to implement NMI.
This calls for finer level of control per capability and the series
implements the same by defining how to deal with a conflict of a
given capability on a CPU with that of the system level state. It
also consolidates the handling of features and errata into generic
helpers. The table of features and errata are left as they are to
allow easier look up for a given type.
The series also gets rid of duplicate entries for a single capability
by introducing a wrapper entry which takes care of managing a list
of entries with distinct matches/enable pair.
We also cleans up the MIDR range handling and cleans up some of the
errata checks where the entries were duplicated for checking different
CPU models. Finally it also implements a work around for Arm Cortex-A55
erratum 1024718 based on the new infrastructure.
Applies on arm64 for-next/core. The tree is available at :
git://linux-arm.org/linux-skp.git cpu-cap-rework/v4
Changes since V3
- Rebased to for-next/core (twice internally huh!)
- Address review comments from Dave
- Rename cap_list => match_list in Shared capability handling.
- Rename cpu_has_erratum_1024718() to cpu_has_broken_dbm() in
Patch 22, Erratum 1024718, for sharing the routine for different
CPUs which could be affected.
Changes since V2
- Mostly address review comments from Dave
- Drop "Change the type for BP hardening to accept late CPUs"
as we can't fake the capability, which is exposed to KVM guests.
- Split the "arm64: capabilities: Group handling of features and
errata" to smaller patches.
- Add tags from Dave.
- Use SCOPE_* in the Code instead of the ARM64_CPUCAP_SCOPE_* version.
- Hide description for DBM feature, as we turn the capability on always
irrespective of whether the CPUs have it or not.
Changes since V1
- Pickup almost all suggestions by Dave.
- Rename flags for handling conflicts
- Rename the "enable" call back to cpu_enable
- Update prototype for cpu_enable to void.
- Handle capabilities with multiple table entries, simplifying the
core logic.
- Add capabilities based on Boot CPU.
- Change the type for BP Hardening to accept late CPUs.
- More usersfor midr_range list.
- More commentary in the code.
- Flip type of Software prefetching capability to Weak from Strict.
Dave Martin (1):
arm64: capabilities: Update prototype for enable call back
Suzuki K Poulose (21):
arm64: capabilities: Move errata work around check on boot CPU
arm64: capabilities: Move errata processing code
arm64: capabilities: Prepare for fine grained capabilities
arm64: capabilities: Add flags to handle the conflicts on late CPU
arm64: capabilities: Unify the verification
arm64: capabilities: Filter the entries based on a given mask
arm64: capabilities: Prepare for grouping features and errata work
arounds
arm64: capabilities: Split the processing of errata work arounds
arm64: capabilities: Allow features based on local CPU scope
arm64: capabilities: Group handling of features and errata workarounds
arm64: capabilities: Introduce weak features based on local CPU
arm64: capabilities: Restrict KPTI detection to boot-time CPUs
arm64: capabilities: Add support for features enabled early
arm64: capabilities: Change scope of VHE to Boot CPU feature
arm64: capabilities: Clean up midr range helpers
arm64: Add helpers for checking CPU MIDR against a range
arm64: capabilities: Add support for checks based on a list of MIDRs
arm64: capabilities: Handle shared entries
arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35
arm64: Delay enabling hardware DBM feature
arm64: Add work around for Arm Cortex-A55 Erratum 1024718
Documentation/arm64/silicon-errata.txt | 1 +
arch/arm64/Kconfig | 14 ++
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/include/asm/cpufeature.h | 258 ++++++++++++++++++++--
arch/arm64/include/asm/cputype.h | 43 ++++
arch/arm64/include/asm/fpsimd.h | 4 +-
arch/arm64/include/asm/processor.h | 7 +-
arch/arm64/include/asm/virt.h | 6 -
arch/arm64/kernel/cpu_errata.c | 294 +++++++++++++------------
arch/arm64/kernel/cpufeature.c | 381 +++++++++++++++++++++++++--------
arch/arm64/kernel/fpsimd.c | 5 +-
arch/arm64/kernel/smp.c | 44 ----
arch/arm64/kernel/traps.c | 4 +-
arch/arm64/mm/fault.c | 3 +-
arch/arm64/mm/proc.S | 13 +-
15 files changed, 769 insertions(+), 311 deletions(-)
--
2.14.3
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 01/22] arm64: capabilities: Update prototype for enable call back
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:50 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:50 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Andre Przywara, James Morse,
Suzuki K Poulose
From: Dave Martin <dave.martin@arm.com>
We issue the enable() call back for all CPU hwcaps capabilities
available on the system, on all the CPUs. So far we have ignored
the argument passed to the call back, which had a prototype to
accept a "void *" for use with on_each_cpu() and later with
stop_machine(). However, with commit 0a0d111d40fd1
("arm64: cpufeature: Pass capability structure to ->enable callback"),
there are some users of the argument who wants the matching capability
struct pointer where there are multiple matching criteria for a single
capability. Clean up the declaration of the call back to make it clear.
1) Renamed to cpu_enable(), to imply taking necessary actions on the
called CPU for the entry.
2) Pass const pointer to the capability, to allow the call back to
check the entry. (e.,g to check if any action is needed on the CPU)
3) We don't care about the result of the call back, turning this to
a void.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Cc: James Morse <james.morse@arm.com>
Acked-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Julien Thierry <julien.thierry@arm.com>
Signed-off-by: Dave Martin <dave.martin@arm.com>
[
Rebased to for-next/core converting more users, rename call back,
drop call back results
]
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 ++++-
arch/arm64/include/asm/fpsimd.h | 4 ++-
arch/arm64/include/asm/processor.h | 7 ++---
arch/arm64/kernel/cpu_errata.c | 53 +++++++++++++++++--------------------
arch/arm64/kernel/cpufeature.c | 42 ++++++++++++++++-------------
arch/arm64/kernel/fpsimd.c | 5 ++--
arch/arm64/kernel/traps.c | 4 +--
arch/arm64/mm/fault.c | 3 +--
8 files changed, 67 insertions(+), 58 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index fbf0aab94d67..cba67cb24b22 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -100,7 +100,12 @@ struct arm64_cpu_capabilities {
u16 capability;
int def_scope; /* default scope */
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
- int (*enable)(void *); /* Called on all active CPUs */
+ /*
+ * Take the appropriate actions to enable this capability for this CPU.
+ * For each successfully booted CPU, this method is called for each
+ * globally detected capability.
+ */
+ void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
struct { /* To be used for erratum handling only */
u32 midr_model;
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 8857a0f0d0f7..7623762f7fa6 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -83,7 +83,9 @@ extern void sve_save_state(void *state, u32 *pfpsr);
extern void sve_load_state(void const *state, u32 const *pfpsr,
unsigned long vq_minus_1);
extern unsigned int sve_get_vl(void);
-extern int sve_kernel_enable(void *);
+
+struct arm64_cpu_capabilities;
+extern void sve_kernel_enable(const struct arm64_cpu_capabilities *__unused);
extern int __ro_after_init sve_max_vl;
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index fce604e3e599..4fc8867fde4d 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -37,6 +37,7 @@
#include <linux/string.h>
#include <asm/alternative.h>
+#include <asm/cpufeature.h>
#include <asm/fpsimd.h>
#include <asm/hw_breakpoint.h>
#include <asm/lse.h>
@@ -227,9 +228,9 @@ static inline void spin_lock_prefetch(const void *ptr)
#endif
-int cpu_enable_pan(void *__unused);
-int cpu_enable_cache_maint_trap(void *__unused);
-int cpu_clear_disr(void *__unused);
+void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused);
+void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused);
+void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused);
/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
#define SVE_SET_VL(arg) sve_set_current_vl(arg)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 186c0fc61dcd..74f058421dd8 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -65,11 +65,11 @@ has_mismatched_cache_line_size(const struct arm64_cpu_capabilities *entry,
(arm64_ftr_reg_ctrel0.sys_val & arm64_ftr_reg_ctrel0.strict_mask);
}
-static int cpu_enable_trap_ctr_access(void *__unused)
+static void
+cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *__unused)
{
/* Clear SCTLR_EL1.UCT */
config_sctlr_el1(SCTLR_EL1_UCT, 0);
- return 0;
}
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
@@ -173,25 +173,25 @@ static void call_hvc_arch_workaround_1(void)
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL);
}
-static int enable_smccc_arch_workaround_1(void *data)
+static void
+enable_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *entry)
{
- const struct arm64_cpu_capabilities *entry = data;
bp_hardening_cb_t cb;
void *smccc_start, *smccc_end;
struct arm_smccc_res res;
if (!entry->matches(entry, SCOPE_LOCAL_CPU))
- return 0;
+ return;
if (psci_ops.smccc_version == SMCCC_VERSION_1_0)
- return 0;
+ return;
switch (psci_ops.conduit) {
case PSCI_CONDUIT_HVC:
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
if (res.a0)
- return 0;
+ return;
cb = call_hvc_arch_workaround_1;
smccc_start = __smccc_workaround_1_hvc_start;
smccc_end = __smccc_workaround_1_hvc_end;
@@ -201,19 +201,19 @@ static int enable_smccc_arch_workaround_1(void *data)
arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
if (res.a0)
- return 0;
+ return;
cb = call_smc_arch_workaround_1;
smccc_start = __smccc_workaround_1_smc_start;
smccc_end = __smccc_workaround_1_smc_end;
break;
default:
- return 0;
+ return;
}
install_bp_hardening_cb(entry, cb, smccc_start, smccc_end);
- return 0;
+ return;
}
static void qcom_link_stack_sanitization(void)
@@ -228,15 +228,12 @@ static void qcom_link_stack_sanitization(void)
: "=&r" (tmp));
}
-static int qcom_enable_link_stack_sanitization(void *data)
+static void
+qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
{
- const struct arm64_cpu_capabilities *entry = data;
-
install_bp_hardening_cb(entry, qcom_link_stack_sanitization,
__qcom_hyp_sanitize_link_stack_start,
__qcom_hyp_sanitize_link_stack_end);
-
- return 0;
}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
@@ -266,7 +263,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "ARM errata 826319, 827319, 824069",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
- .enable = cpu_enable_cache_maint_trap,
+ .cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_819472
@@ -275,7 +272,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "ARM errata 819472",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
- .enable = cpu_enable_cache_maint_trap,
+ .cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_832075
@@ -365,7 +362,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
.def_scope = SCOPE_LOCAL_CPU,
- .enable = cpu_enable_trap_ctr_access,
+ .cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
{
@@ -404,27 +401,27 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
- .enable = qcom_enable_link_stack_sanitization,
+ .cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
@@ -433,7 +430,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
- .enable = qcom_enable_link_stack_sanitization,
+ .cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
@@ -442,12 +439,12 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
#endif
{
@@ -465,8 +462,8 @@ void verify_local_cpu_errata_workarounds(void)
for (; caps->matches; caps++) {
if (cpus_have_cap(caps->capability)) {
- if (caps->enable)
- caps->enable((void *)caps);
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
} else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
pr_crit("CPU%d: Requires work around for %s, not detected"
" at boot time\n",
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index bdab55223866..b8821234bc68 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -906,7 +906,8 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
ID_AA64PFR0_CSV3_SHIFT);
}
-static int kpti_install_ng_mappings(void *__unused)
+static void
+kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)
{
typedef void (kpti_remap_fn)(int, int, phys_addr_t);
extern kpti_remap_fn idmap_kpti_install_ng_mappings;
@@ -916,7 +917,7 @@ static int kpti_install_ng_mappings(void *__unused)
int cpu = smp_processor_id();
if (kpti_applied)
- return 0;
+ return;
remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
@@ -927,7 +928,7 @@ static int kpti_install_ng_mappings(void *__unused)
if (!cpu)
kpti_applied = true;
- return 0;
+ return;
}
static int __init parse_kpti(char *str)
@@ -944,7 +945,7 @@ static int __init parse_kpti(char *str)
__setup("kpti=", parse_kpti);
#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
-static int cpu_copy_el2regs(void *__unused)
+static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
{
/*
* Copy register values that aren't redirected by hardware.
@@ -956,8 +957,6 @@ static int cpu_copy_el2regs(void *__unused)
*/
if (!alternatives_applied)
write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
-
- return 0;
}
static const struct arm64_cpu_capabilities arm64_features[] = {
@@ -981,7 +980,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
.sign = FTR_UNSIGNED,
.min_field_value = 1,
- .enable = cpu_enable_pan,
+ .cpu_enable = cpu_enable_pan,
},
#endif /* CONFIG_ARM64_PAN */
#if defined(CONFIG_AS_LSE) && defined(CONFIG_ARM64_LSE_ATOMICS)
@@ -1029,7 +1028,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.capability = ARM64_HAS_VIRT_HOST_EXTN,
.def_scope = SCOPE_SYSTEM,
.matches = runs_at_el2,
- .enable = cpu_copy_el2regs,
+ .cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
@@ -1053,7 +1052,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
.def_scope = SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
- .enable = kpti_install_ng_mappings,
+ .cpu_enable = kpti_install_ng_mappings,
},
#endif
{
@@ -1084,7 +1083,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64PFR0_SVE_SHIFT,
.min_field_value = ID_AA64PFR0_SVE,
.matches = has_cpuid_feature,
- .enable = sve_kernel_enable,
+ .cpu_enable = sve_kernel_enable,
},
#endif /* CONFIG_ARM64_SVE */
#ifdef CONFIG_ARM64_RAS_EXTN
@@ -1097,7 +1096,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.sign = FTR_UNSIGNED,
.field_pos = ID_AA64PFR0_RAS_SHIFT,
.min_field_value = ID_AA64PFR0_RAS_V1,
- .enable = cpu_clear_disr,
+ .cpu_enable = cpu_clear_disr,
},
#endif /* CONFIG_ARM64_RAS_EXTN */
{
@@ -1253,6 +1252,14 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static int __enable_cpu_capability(void *arg)
+{
+ const struct arm64_cpu_capabilities *cap = arg;
+
+ cap->cpu_enable(cap);
+ return 0;
+}
+
/*
* Run through the enabled capabilities and enable() it on all active
* CPUs
@@ -1268,14 +1275,15 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
/* Ensure cpus_have_const_cap(num) works */
static_branch_enable(&cpu_hwcap_keys[num]);
- if (caps->enable) {
+ if (caps->cpu_enable) {
/*
* Use stop_machine() as it schedules the work allowing
* us to modify PSTATE, instead of on_each_cpu() which
* uses an IPI, giving us a PSTATE that disappears when
* we return.
*/
- stop_machine(caps->enable, (void *)caps, cpu_online_mask);
+ stop_machine(__enable_cpu_capability, (void *)caps,
+ cpu_online_mask);
}
}
}
@@ -1318,8 +1326,8 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list)
smp_processor_id(), caps->desc);
cpu_die_early();
}
- if (caps->enable)
- caps->enable((void *)caps);
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
}
}
@@ -1537,10 +1545,8 @@ static int __init enable_mrs_emulation(void)
core_initcall(enable_mrs_emulation);
-int cpu_clear_disr(void *__unused)
+void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused)
{
/* Firmware may have left a deferred SError in this register. */
write_sysreg_s(0, SYS_DISR_EL1);
-
- return 0;
}
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 65fc87645ec6..fc19e741b5c8 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -40,6 +40,7 @@
#include <linux/sysctl.h>
#include <asm/fpsimd.h>
+#include <asm/cpufeature.h>
#include <asm/cputype.h>
#include <asm/simd.h>
#include <asm/sigcontext.h>
@@ -756,12 +757,10 @@ static void __init sve_efi_setup(void)
* Enable SVE for EL1.
* Intended for use by the cpufeatures code during CPU boot.
*/
-int sve_kernel_enable(void *__always_unused p)
+void sve_kernel_enable(const struct arm64_cpu_capabilities *__always_unused p)
{
write_sysreg(read_sysreg(CPACR_EL1) | CPACR_EL1_ZEN_EL1EN, CPACR_EL1);
isb();
-
- return 0;
}
void __init sve_setup(void)
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 2b478565d774..ba964da31a25 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -38,6 +38,7 @@
#include <asm/atomic.h>
#include <asm/bug.h>
+#include <asm/cpufeature.h>
#include <asm/daifflags.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
@@ -407,10 +408,9 @@ asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc);
}
-int cpu_enable_cache_maint_trap(void *__unused)
+void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused)
{
config_sctlr_el1(SCTLR_EL1_UCI, 0);
- return 0;
}
#define __user_cache_maint(insn, address, res) \
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 551d044fb31f..4165485e8b6e 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -804,7 +804,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
NOKPROBE_SYMBOL(do_debug_exception);
#ifdef CONFIG_ARM64_PAN
-int cpu_enable_pan(void *__unused)
+void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused)
{
/*
* We modify PSTATE. This won't work from irq context as the PSTATE
@@ -814,6 +814,5 @@ int cpu_enable_pan(void *__unused)
config_sctlr_el1(SCTLR_EL1_SPAN, 0);
asm(SET_PSTATE_PAN(1));
- return 0;
}
#endif /* CONFIG_ARM64_PAN */
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 01/22] arm64: capabilities: Update prototype for enable call back
@ 2018-03-13 11:50 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:50 UTC (permalink / raw)
To: linux-arm-kernel
From: Dave Martin <dave.martin@arm.com>
We issue the enable() call back for all CPU hwcaps capabilities
available on the system, on all the CPUs. So far we have ignored
the argument passed to the call back, which had a prototype to
accept a "void *" for use with on_each_cpu() and later with
stop_machine(). However, with commit 0a0d111d40fd1
("arm64: cpufeature: Pass capability structure to ->enable callback"),
there are some users of the argument who wants the matching capability
struct pointer where there are multiple matching criteria for a single
capability. Clean up the declaration of the call back to make it clear.
1) Renamed to cpu_enable(), to imply taking necessary actions on the
called CPU for the entry.
2) Pass const pointer to the capability, to allow the call back to
check the entry. (e.,g to check if any action is needed on the CPU)
3) We don't care about the result of the call back, turning this to
a void.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Cc: James Morse <james.morse@arm.com>
Acked-by: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Julien Thierry <julien.thierry@arm.com>
Signed-off-by: Dave Martin <dave.martin@arm.com>
[
Rebased to for-next/core converting more users, rename call back,
drop call back results
]
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 ++++-
arch/arm64/include/asm/fpsimd.h | 4 ++-
arch/arm64/include/asm/processor.h | 7 ++---
arch/arm64/kernel/cpu_errata.c | 53 +++++++++++++++++--------------------
arch/arm64/kernel/cpufeature.c | 42 ++++++++++++++++-------------
arch/arm64/kernel/fpsimd.c | 5 ++--
arch/arm64/kernel/traps.c | 4 +--
arch/arm64/mm/fault.c | 3 +--
8 files changed, 67 insertions(+), 58 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index fbf0aab94d67..cba67cb24b22 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -100,7 +100,12 @@ struct arm64_cpu_capabilities {
u16 capability;
int def_scope; /* default scope */
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
- int (*enable)(void *); /* Called on all active CPUs */
+ /*
+ * Take the appropriate actions to enable this capability for this CPU.
+ * For each successfully booted CPU, this method is called for each
+ * globally detected capability.
+ */
+ void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
struct { /* To be used for erratum handling only */
u32 midr_model;
diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h
index 8857a0f0d0f7..7623762f7fa6 100644
--- a/arch/arm64/include/asm/fpsimd.h
+++ b/arch/arm64/include/asm/fpsimd.h
@@ -83,7 +83,9 @@ extern void sve_save_state(void *state, u32 *pfpsr);
extern void sve_load_state(void const *state, u32 const *pfpsr,
unsigned long vq_minus_1);
extern unsigned int sve_get_vl(void);
-extern int sve_kernel_enable(void *);
+
+struct arm64_cpu_capabilities;
+extern void sve_kernel_enable(const struct arm64_cpu_capabilities *__unused);
extern int __ro_after_init sve_max_vl;
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index fce604e3e599..4fc8867fde4d 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -37,6 +37,7 @@
#include <linux/string.h>
#include <asm/alternative.h>
+#include <asm/cpufeature.h>
#include <asm/fpsimd.h>
#include <asm/hw_breakpoint.h>
#include <asm/lse.h>
@@ -227,9 +228,9 @@ static inline void spin_lock_prefetch(const void *ptr)
#endif
-int cpu_enable_pan(void *__unused);
-int cpu_enable_cache_maint_trap(void *__unused);
-int cpu_clear_disr(void *__unused);
+void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused);
+void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused);
+void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused);
/* Userspace interface for PR_SVE_{SET,GET}_VL prctl()s: */
#define SVE_SET_VL(arg) sve_set_current_vl(arg)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 186c0fc61dcd..74f058421dd8 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -65,11 +65,11 @@ has_mismatched_cache_line_size(const struct arm64_cpu_capabilities *entry,
(arm64_ftr_reg_ctrel0.sys_val & arm64_ftr_reg_ctrel0.strict_mask);
}
-static int cpu_enable_trap_ctr_access(void *__unused)
+static void
+cpu_enable_trap_ctr_access(const struct arm64_cpu_capabilities *__unused)
{
/* Clear SCTLR_EL1.UCT */
config_sctlr_el1(SCTLR_EL1_UCT, 0);
- return 0;
}
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
@@ -173,25 +173,25 @@ static void call_hvc_arch_workaround_1(void)
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL);
}
-static int enable_smccc_arch_workaround_1(void *data)
+static void
+enable_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *entry)
{
- const struct arm64_cpu_capabilities *entry = data;
bp_hardening_cb_t cb;
void *smccc_start, *smccc_end;
struct arm_smccc_res res;
if (!entry->matches(entry, SCOPE_LOCAL_CPU))
- return 0;
+ return;
if (psci_ops.smccc_version == SMCCC_VERSION_1_0)
- return 0;
+ return;
switch (psci_ops.conduit) {
case PSCI_CONDUIT_HVC:
arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
if (res.a0)
- return 0;
+ return;
cb = call_hvc_arch_workaround_1;
smccc_start = __smccc_workaround_1_hvc_start;
smccc_end = __smccc_workaround_1_hvc_end;
@@ -201,19 +201,19 @@ static int enable_smccc_arch_workaround_1(void *data)
arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID,
ARM_SMCCC_ARCH_WORKAROUND_1, &res);
if (res.a0)
- return 0;
+ return;
cb = call_smc_arch_workaround_1;
smccc_start = __smccc_workaround_1_smc_start;
smccc_end = __smccc_workaround_1_smc_end;
break;
default:
- return 0;
+ return;
}
install_bp_hardening_cb(entry, cb, smccc_start, smccc_end);
- return 0;
+ return;
}
static void qcom_link_stack_sanitization(void)
@@ -228,15 +228,12 @@ static void qcom_link_stack_sanitization(void)
: "=&r" (tmp));
}
-static int qcom_enable_link_stack_sanitization(void *data)
+static void
+qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
{
- const struct arm64_cpu_capabilities *entry = data;
-
install_bp_hardening_cb(entry, qcom_link_stack_sanitization,
__qcom_hyp_sanitize_link_stack_start,
__qcom_hyp_sanitize_link_stack_end);
-
- return 0;
}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
@@ -266,7 +263,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "ARM errata 826319, 827319, 824069",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
- .enable = cpu_enable_cache_maint_trap,
+ .cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_819472
@@ -275,7 +272,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "ARM errata 819472",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
- .enable = cpu_enable_cache_maint_trap,
+ .cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_832075
@@ -365,7 +362,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
.def_scope = SCOPE_LOCAL_CPU,
- .enable = cpu_enable_trap_ctr_access,
+ .cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
{
@@ -404,27 +401,27 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
- .enable = qcom_enable_link_stack_sanitization,
+ .cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
@@ -433,7 +430,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
- .enable = qcom_enable_link_stack_sanitization,
+ .cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
@@ -442,12 +439,12 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
- .enable = enable_smccc_arch_workaround_1,
+ .cpu_enable = enable_smccc_arch_workaround_1,
},
#endif
{
@@ -465,8 +462,8 @@ void verify_local_cpu_errata_workarounds(void)
for (; caps->matches; caps++) {
if (cpus_have_cap(caps->capability)) {
- if (caps->enable)
- caps->enable((void *)caps);
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
} else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
pr_crit("CPU%d: Requires work around for %s, not detected"
" at boot time\n",
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index bdab55223866..b8821234bc68 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -906,7 +906,8 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
ID_AA64PFR0_CSV3_SHIFT);
}
-static int kpti_install_ng_mappings(void *__unused)
+static void
+kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)
{
typedef void (kpti_remap_fn)(int, int, phys_addr_t);
extern kpti_remap_fn idmap_kpti_install_ng_mappings;
@@ -916,7 +917,7 @@ static int kpti_install_ng_mappings(void *__unused)
int cpu = smp_processor_id();
if (kpti_applied)
- return 0;
+ return;
remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
@@ -927,7 +928,7 @@ static int kpti_install_ng_mappings(void *__unused)
if (!cpu)
kpti_applied = true;
- return 0;
+ return;
}
static int __init parse_kpti(char *str)
@@ -944,7 +945,7 @@ static int __init parse_kpti(char *str)
__setup("kpti=", parse_kpti);
#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
-static int cpu_copy_el2regs(void *__unused)
+static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
{
/*
* Copy register values that aren't redirected by hardware.
@@ -956,8 +957,6 @@ static int cpu_copy_el2regs(void *__unused)
*/
if (!alternatives_applied)
write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
-
- return 0;
}
static const struct arm64_cpu_capabilities arm64_features[] = {
@@ -981,7 +980,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
.sign = FTR_UNSIGNED,
.min_field_value = 1,
- .enable = cpu_enable_pan,
+ .cpu_enable = cpu_enable_pan,
},
#endif /* CONFIG_ARM64_PAN */
#if defined(CONFIG_AS_LSE) && defined(CONFIG_ARM64_LSE_ATOMICS)
@@ -1029,7 +1028,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.capability = ARM64_HAS_VIRT_HOST_EXTN,
.def_scope = SCOPE_SYSTEM,
.matches = runs_at_el2,
- .enable = cpu_copy_el2regs,
+ .cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
@@ -1053,7 +1052,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
.def_scope = SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
- .enable = kpti_install_ng_mappings,
+ .cpu_enable = kpti_install_ng_mappings,
},
#endif
{
@@ -1084,7 +1083,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.field_pos = ID_AA64PFR0_SVE_SHIFT,
.min_field_value = ID_AA64PFR0_SVE,
.matches = has_cpuid_feature,
- .enable = sve_kernel_enable,
+ .cpu_enable = sve_kernel_enable,
},
#endif /* CONFIG_ARM64_SVE */
#ifdef CONFIG_ARM64_RAS_EXTN
@@ -1097,7 +1096,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.sign = FTR_UNSIGNED,
.field_pos = ID_AA64PFR0_RAS_SHIFT,
.min_field_value = ID_AA64PFR0_RAS_V1,
- .enable = cpu_clear_disr,
+ .cpu_enable = cpu_clear_disr,
},
#endif /* CONFIG_ARM64_RAS_EXTN */
{
@@ -1253,6 +1252,14 @@ void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static int __enable_cpu_capability(void *arg)
+{
+ const struct arm64_cpu_capabilities *cap = arg;
+
+ cap->cpu_enable(cap);
+ return 0;
+}
+
/*
* Run through the enabled capabilities and enable() it on all active
* CPUs
@@ -1268,14 +1275,15 @@ void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
/* Ensure cpus_have_const_cap(num) works */
static_branch_enable(&cpu_hwcap_keys[num]);
- if (caps->enable) {
+ if (caps->cpu_enable) {
/*
* Use stop_machine() as it schedules the work allowing
* us to modify PSTATE, instead of on_each_cpu() which
* uses an IPI, giving us a PSTATE that disappears when
* we return.
*/
- stop_machine(caps->enable, (void *)caps, cpu_online_mask);
+ stop_machine(__enable_cpu_capability, (void *)caps,
+ cpu_online_mask);
}
}
}
@@ -1318,8 +1326,8 @@ verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list)
smp_processor_id(), caps->desc);
cpu_die_early();
}
- if (caps->enable)
- caps->enable((void *)caps);
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
}
}
@@ -1537,10 +1545,8 @@ static int __init enable_mrs_emulation(void)
core_initcall(enable_mrs_emulation);
-int cpu_clear_disr(void *__unused)
+void cpu_clear_disr(const struct arm64_cpu_capabilities *__unused)
{
/* Firmware may have left a deferred SError in this register. */
write_sysreg_s(0, SYS_DISR_EL1);
-
- return 0;
}
diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c
index 65fc87645ec6..fc19e741b5c8 100644
--- a/arch/arm64/kernel/fpsimd.c
+++ b/arch/arm64/kernel/fpsimd.c
@@ -40,6 +40,7 @@
#include <linux/sysctl.h>
#include <asm/fpsimd.h>
+#include <asm/cpufeature.h>
#include <asm/cputype.h>
#include <asm/simd.h>
#include <asm/sigcontext.h>
@@ -756,12 +757,10 @@ static void __init sve_efi_setup(void)
* Enable SVE for EL1.
* Intended for use by the cpufeatures code during CPU boot.
*/
-int sve_kernel_enable(void *__always_unused p)
+void sve_kernel_enable(const struct arm64_cpu_capabilities *__always_unused p)
{
write_sysreg(read_sysreg(CPACR_EL1) | CPACR_EL1_ZEN_EL1EN, CPACR_EL1);
isb();
-
- return 0;
}
void __init sve_setup(void)
diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c
index 2b478565d774..ba964da31a25 100644
--- a/arch/arm64/kernel/traps.c
+++ b/arch/arm64/kernel/traps.c
@@ -38,6 +38,7 @@
#include <asm/atomic.h>
#include <asm/bug.h>
+#include <asm/cpufeature.h>
#include <asm/daifflags.h>
#include <asm/debug-monitors.h>
#include <asm/esr.h>
@@ -407,10 +408,9 @@ asmlinkage void __exception do_undefinstr(struct pt_regs *regs)
force_signal_inject(SIGILL, ILL_ILLOPC, regs->pc);
}
-int cpu_enable_cache_maint_trap(void *__unused)
+void cpu_enable_cache_maint_trap(const struct arm64_cpu_capabilities *__unused)
{
config_sctlr_el1(SCTLR_EL1_UCI, 0);
- return 0;
}
#define __user_cache_maint(insn, address, res) \
diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c
index 551d044fb31f..4165485e8b6e 100644
--- a/arch/arm64/mm/fault.c
+++ b/arch/arm64/mm/fault.c
@@ -804,7 +804,7 @@ asmlinkage int __exception do_debug_exception(unsigned long addr,
NOKPROBE_SYMBOL(do_debug_exception);
#ifdef CONFIG_ARM64_PAN
-int cpu_enable_pan(void *__unused)
+void cpu_enable_pan(const struct arm64_cpu_capabilities *__unused)
{
/*
* We modify PSTATE. This won't work from irq context as the PSTATE
@@ -814,6 +814,5 @@ int cpu_enable_pan(void *__unused)
config_sctlr_el1(SCTLR_EL1_SPAN, 0);
asm(SET_PSTATE_PAN(1));
- return 0;
}
#endif /* CONFIG_ARM64_PAN */
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 02/22] arm64: capabilities: Move errata work around check on boot CPU
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
We trigger CPU errata work around check on the boot CPU from
smp_prepare_boot_cpu() to make sure that we run the checks only
after the CPU feature infrastructure is initialised. While this
is correct, we can also do this from init_cpu_features() which
initilises the infrastructure, and is called only on the
Boot CPU. This helps to consolidate the CPU capability handling
to cpufeature.c. No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++++
arch/arm64/kernel/smp.c | 6 ------
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index b8821234bc68..14c8348d488a 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -548,6 +548,12 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
sve_init_vq_map();
}
+
+ /*
+ * Run the errata work around checks on the boot CPU, once we have
+ * initialised the cpu feature infrastructure.
+ */
+ update_cpu_errata_workarounds();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 3b8ad7be9c33..5cef11450183 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -448,12 +448,6 @@ void __init smp_prepare_boot_cpu(void)
jump_label_init();
cpuinfo_store_boot_cpu();
save_boot_cpu_run_el();
- /*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure from
- * cpuinfo_store_boot_cpu() above.
- */
- update_cpu_errata_workarounds();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 02/22] arm64: capabilities: Move errata work around check on boot CPU
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
We trigger CPU errata work around check on the boot CPU from
smp_prepare_boot_cpu() to make sure that we run the checks only
after the CPU feature infrastructure is initialised. While this
is correct, we can also do this from init_cpu_features() which
initilises the infrastructure, and is called only on the
Boot CPU. This helps to consolidate the CPU capability handling
to cpufeature.c. No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++++
arch/arm64/kernel/smp.c | 6 ------
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index b8821234bc68..14c8348d488a 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -548,6 +548,12 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
init_cpu_ftr_reg(SYS_ZCR_EL1, info->reg_zcr);
sve_init_vq_map();
}
+
+ /*
+ * Run the errata work around checks on the boot CPU, once we have
+ * initialised the cpu feature infrastructure.
+ */
+ update_cpu_errata_workarounds();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 3b8ad7be9c33..5cef11450183 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -448,12 +448,6 @@ void __init smp_prepare_boot_cpu(void)
jump_label_init();
cpuinfo_store_boot_cpu();
save_boot_cpu_run_el();
- /*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure from
- * cpuinfo_store_boot_cpu() above.
- */
- update_cpu_errata_workarounds();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 03/22] arm64: capabilities: Move errata processing code
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose, Andre Przywara
We have errata work around processing code in cpu_errata.c,
which calls back into helpers defined in cpufeature.c. Now
that we are going to make the handling of capabilities
generic, by adding the information to each capability,
move the errata work around specific processing code.
No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 ------
arch/arm64/kernel/cpu_errata.c | 33 ----------------------------
arch/arm64/kernel/cpufeature.c | 43 ++++++++++++++++++++++++++++++++++---
3 files changed, 40 insertions(+), 43 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index cba67cb24b22..8efbda2858a8 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -245,15 +245,8 @@ static inline bool id_aa64pfr0_sve(u64 pfr0)
}
void __init setup_cpu_features(void);
-
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info);
-void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps);
void check_local_cpu_capabilities(void);
-void update_cpu_errata_workarounds(void);
-void __init enable_errata_workarounds(void);
-void verify_local_cpu_errata_workarounds(void);
u64 read_sanitised_ftr_reg(u32 id);
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 74f058421dd8..0de0ab6a874c 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -450,36 +450,3 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
}
};
-
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-void verify_local_cpu_errata_workarounds(void)
-{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
-}
-
-void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
-}
-
-void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata);
-}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 14c8348d488a..5fc500c13cfc 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -506,6 +506,9 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
reg->user_mask = user_mask;
}
+extern const struct arm64_cpu_capabilities arm64_errata[];
+static void update_cpu_errata_workarounds(void);
+
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
/* Before we start using the tables, make sure it is sorted */
@@ -1245,8 +1248,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info)
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ const char *info)
{
for (; caps->matches; caps++) {
if (!caps->matches(caps, caps->def_scope))
@@ -1270,7 +1273,8 @@ static int __enable_cpu_capability(void *arg)
* Run through the enabled capabilities and enable() it on all active
* CPUs
*/
-void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+static void __init
+enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
{
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
@@ -1354,6 +1358,39 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
+/*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU requires
+ * an errata not detected at boot, fail this CPU.
+ */
+static void verify_local_cpu_errata_workarounds(void)
+{
+ const struct arm64_cpu_capabilities *caps = arm64_errata;
+
+ for (; caps->matches; caps++) {
+ if (cpus_have_cap(caps->capability)) {
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
+ pr_crit("CPU%d: Requires work around for %s, not detected"
+ " at boot time\n",
+ smp_processor_id(),
+ caps->desc ? : "an erratum");
+ cpu_die_early();
+ }
+ }
+}
+
+static void update_cpu_errata_workarounds(void)
+{
+ update_cpu_capabilities(arm64_errata, "enabling workaround for");
+}
+
+static void __init enable_errata_workarounds(void)
+{
+ enable_cpu_capabilities(arm64_errata);
+}
+
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 03/22] arm64: capabilities: Move errata processing code
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
We have errata work around processing code in cpu_errata.c,
which calls back into helpers defined in cpufeature.c. Now
that we are going to make the handling of capabilities
generic, by adding the information to each capability,
move the errata work around specific processing code.
No functional changes.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 7 ------
arch/arm64/kernel/cpu_errata.c | 33 ----------------------------
arch/arm64/kernel/cpufeature.c | 43 ++++++++++++++++++++++++++++++++++---
3 files changed, 40 insertions(+), 43 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index cba67cb24b22..8efbda2858a8 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -245,15 +245,8 @@ static inline bool id_aa64pfr0_sve(u64 pfr0)
}
void __init setup_cpu_features(void);
-
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info);
-void enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps);
void check_local_cpu_capabilities(void);
-void update_cpu_errata_workarounds(void);
-void __init enable_errata_workarounds(void);
-void verify_local_cpu_errata_workarounds(void);
u64 read_sanitised_ftr_reg(u32 id);
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 74f058421dd8..0de0ab6a874c 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -450,36 +450,3 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
}
};
-
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-void verify_local_cpu_errata_workarounds(void)
-{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
-}
-
-void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
-}
-
-void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata);
-}
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 14c8348d488a..5fc500c13cfc 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -506,6 +506,9 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
reg->user_mask = user_mask;
}
+extern const struct arm64_cpu_capabilities arm64_errata[];
+static void update_cpu_errata_workarounds(void);
+
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
/* Before we start using the tables, make sure it is sorted */
@@ -1245,8 +1248,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info)
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ const char *info)
{
for (; caps->matches; caps++) {
if (!caps->matches(caps, caps->def_scope))
@@ -1270,7 +1273,8 @@ static int __enable_cpu_capability(void *arg)
* Run through the enabled capabilities and enable() it on all active
* CPUs
*/
-void __init enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+static void __init
+enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
{
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
@@ -1354,6 +1358,39 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
+/*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU requires
+ * an errata not detected at boot, fail this CPU.
+ */
+static void verify_local_cpu_errata_workarounds(void)
+{
+ const struct arm64_cpu_capabilities *caps = arm64_errata;
+
+ for (; caps->matches; caps++) {
+ if (cpus_have_cap(caps->capability)) {
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
+ pr_crit("CPU%d: Requires work around for %s, not detected"
+ " at boot time\n",
+ smp_processor_id(),
+ caps->desc ? : "an erratum");
+ cpu_die_early();
+ }
+ }
+}
+
+static void update_cpu_errata_workarounds(void)
+{
+ update_cpu_capabilities(arm64_errata, "enabling workaround for");
+}
+
+static void __init enable_errata_workarounds(void)
+{
+ enable_cpu_capabilities(arm64_errata);
+}
+
/*
* Run through the enabled system capabilities and enable() it on this CPU.
* The capabilities were decided based on the available CPUs at the boot time.
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 04/22] arm64: capabilities: Prepare for fine grained capabilities
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
We use arm64_cpu_capabilities to represent CPU ELF HWCAPs exposed
to the userspace and the CPU hwcaps used by the kernel, which
include cpu features and CPU errata work arounds. Capabilities
have some properties that decide how they should be treated :
1) Detection, i.e scope : A cap could be "detected" either :
- if it is present on at least one CPU (SCOPE_LOCAL_CPU)
Or
- if it is present on all the CPUs (SCOPE_SYSTEM)
2) When is it enabled ? - A cap is treated as "enabled" when the
system takes some action based on whether the capability is detected or
not. e.g, setting some control register, patching the kernel code.
Right now, we treat all caps are enabled at boot-time, after all
the CPUs are brought up by the kernel. But there are certain caps,
which are enabled early during the boot (e.g, VHE, GIC_CPUIF for NMI)
and kernel starts using them, even before the secondary CPUs are brought
up. We would need a way to describe this for each capability.
3) Conflict on a late CPU - When a CPU is brought up, it is checked
against the caps that are known to be enabled on the system (via
verify_local_cpu_capabilities()). Based on the state of the capability
on the CPU vs. that of System we could have the following combinations
of conflict.
x-----------------------------x
| Type | System | Late CPU |
------------------------------|
| a | y | n |
------------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds which requires some
work around, which cannot be delayed. And we ignore (b) for features.
Here, yet again, KPTI is an exception, where if a late CPU needs KPTI we
are too late to enable it (because we change the allocation of ASIDs
etc).
So this calls for a lot more fine grained behavior for each capability.
And if we define all the attributes to control their behavior properly,
we may be able to use a single table for the CPU hwcaps (which cover
errata and features, not the ELF HWCAPs). This is a prepartory step
to get there. More bits would be added for the properties listed above.
We are going to use a bit-mask to encode all the properties of a
capabilities. This patch encodes the "SCOPE" of the capability.
As such there is no change in how the capabilities are treated.
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 105 +++++++++++++++++++++++++++++++++---
arch/arm64/kernel/cpu_errata.c | 8 +--
arch/arm64/kernel/cpufeature.c | 42 +++++++--------
3 files changed, 124 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 8efbda2858a8..13fde0952c31 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -89,16 +89,104 @@ struct arm64_ftr_reg {
extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
-/* scope of capability check */
-enum {
- SCOPE_SYSTEM,
- SCOPE_LOCAL_CPU,
-};
+/*
+ * CPU capabilities:
+ *
+ * We use arm64_cpu_capabilities to represent system features, errata work
+ * arounds (both used internally by kernel and tracked in cpu_hwcaps) and
+ * ELF HWCAPs (which are exposed to user).
+ *
+ * To support systems with heterogeneous CPUs, we need to make sure that we
+ * detect the capabilities correctly on the system and take appropriate
+ * measures to ensure there are no incompatibilities.
+ *
+ * This comment tries to explain how we treat the capabilities.
+ * Each capability has the following list of attributes :
+ *
+ * 1) Scope of Detection : The system detects a given capability by
+ * performing some checks at runtime. This could be, e.g, checking the
+ * value of a field in CPU ID feature register or checking the cpu
+ * model. The capability provides a call back ( @matches() ) to
+ * perform the check. Scope defines how the checks should be performed.
+ * There are two cases:
+ *
+ * a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
+ * matches. This implies, we have to run the check on all the
+ * booting CPUs, until the system decides that state of the
+ * capability is finalised. (See section 2 below)
+ * Or
+ * b) SCOPE_SYSTEM: check all the CPUs and "detect" if all the CPUs
+ * matches. This implies, we run the check only once, when the
+ * system decides to finalise the state of the capability. If the
+ * capability relies on a field in one of the CPU ID feature
+ * registers, we use the sanitised value of the register from the
+ * CPU feature infrastructure to make the decision.
+ *
+ * The process of detection is usually denoted by "update" capability
+ * state in the code.
+ *
+ * 2) Finalise the state : The kernel should finalise the state of a
+ * capability at some point during its execution and take necessary
+ * actions if any. Usually, this is done, after all the boot-time
+ * enabled CPUs are brought up by the kernel, so that it can make
+ * better decision based on the available set of CPUs. However, there
+ * are some special cases, where the action is taken during the early
+ * boot by the primary boot CPU. (e.g, running the kernel at EL2 with
+ * Virtualisation Host Extensions). The kernel usually disallows any
+ * changes to the state of a capability once it finalises the capability
+ * and takes any action, as it may be impossible to execute the actions
+ * safely. A CPU brought up after a capability is "finalised" is
+ * referred to as "Late CPU" w.r.t the capability. e.g, all secondary
+ * CPUs are treated "late CPUs" for capabilities determined by the boot
+ * CPU.
+ *
+ * 3) Verification: When a CPU is brought online (e.g, by user or by the
+ * kernel), the kernel should make sure that it is safe to use the CPU,
+ * by verifying that the CPU is compliant with the state of the
+ * capabilities finalised already. This happens via :
+ *
+ * secondary_start_kernel()-> check_local_cpu_capabilities()
+ *
+ * As explained in (2) above, capabilities could be finalised at
+ * different points in the execution. Each CPU is verified against the
+ * "finalised" capabilities and if there is a conflict, the kernel takes
+ * an action, based on the severity (e.g, a CPU could be prevented from
+ * booting or cause a kernel panic). The CPU is allowed to "affect" the
+ * state of the capability, if it has not been finalised already.
+ *
+ * 4) Action: As mentioned in (2), the kernel can take an action for each
+ * detected capability, on all CPUs on the system. Appropriate actions
+ * include, turning on an architectural feature, modifying the control
+ * registers (e.g, SCTLR, TCR etc.) or patching the kernel via
+ * alternatives. The kernel patching is batched and performed at later
+ * point. The actions are always initiated only after the capability
+ * is finalised. This is usally denoted by "enabling" the capability.
+ * The actions are initiated as follows :
+ * a) Action is triggered on all online CPUs, after the capability is
+ * finalised, invoked within the stop_machine() context from
+ * enable_cpu_capabilitie().
+ *
+ * b) Any late CPU, brought up after (1), the action is triggered via:
+ *
+ * check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
+ *
+ */
+
+
+/* Decide how the capability is detected. On a local CPU vs System wide */
+#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
+#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+#define ARM64_CPUCAP_SCOPE_MASK \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | \
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+
+#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
+#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
- int def_scope; /* default scope */
+ u16 type;
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
/*
* Take the appropriate actions to enable this capability for this CPU.
@@ -127,6 +215,11 @@ struct arm64_cpu_capabilities {
};
};
+static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
+{
+ return cap->type & ARM64_CPUCAP_SCOPE_MASK;
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 0de0ab6a874c..2b5233157af1 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -238,14 +238,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -361,7 +361,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -375,7 +375,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5fc500c13cfc..1e51fb94125f 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -972,7 +972,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -983,7 +983,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -996,7 +996,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -1007,14 +1007,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1028,21 +1028,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1052,14 +1052,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1067,7 +1067,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1075,7 +1075,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1085,8 +1085,8 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.capability = ARM64_SVE,
- .def_scope = SCOPE_SYSTEM,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
.field_pos = ID_AA64PFR0_SVE_SHIFT,
@@ -1099,7 +1099,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1111,28 +1111,28 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to the PoU not required for I/D coherence",
.capability = ARM64_HAS_CACHE_IDC,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cache_idc,
},
{
.desc = "Instruction cache invalidation not required for I/D coherence",
.capability = ARM64_HAS_CACHE_DIC,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cache_dic,
},
{},
};
-#define HWCAP_CAP(reg, field, s, min_value, type, cap) \
+#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .def_scope = SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
.sign = s, \
.min_field_value = min_value, \
- .hwcap_type = type, \
+ .hwcap_type = cap_type, \
.hwcap = cap, \
}
@@ -1225,7 +1225,7 @@ static void __init setup_elf_hwcaps(const struct arm64_cpu_capabilities *hwcaps)
/* We support emulation of accesses to CPU ID feature registers */
elf_hwcap |= HWCAP_CPUID;
for (; hwcaps->matches; hwcaps++)
- if (hwcaps->matches(hwcaps, hwcaps->def_scope))
+ if (hwcaps->matches(hwcaps, cpucap_default_scope(hwcaps)))
cap_set_elf_hwcap(hwcaps);
}
@@ -1252,7 +1252,7 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info)
{
for (; caps->matches; caps++) {
- if (!caps->matches(caps, caps->def_scope))
+ if (!caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 04/22] arm64: capabilities: Prepare for fine grained capabilities
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
We use arm64_cpu_capabilities to represent CPU ELF HWCAPs exposed
to the userspace and the CPU hwcaps used by the kernel, which
include cpu features and CPU errata work arounds. Capabilities
have some properties that decide how they should be treated :
1) Detection, i.e scope : A cap could be "detected" either :
- if it is present on at least one CPU (SCOPE_LOCAL_CPU)
Or
- if it is present on all the CPUs (SCOPE_SYSTEM)
2) When is it enabled ? - A cap is treated as "enabled" when the
system takes some action based on whether the capability is detected or
not. e.g, setting some control register, patching the kernel code.
Right now, we treat all caps are enabled at boot-time, after all
the CPUs are brought up by the kernel. But there are certain caps,
which are enabled early during the boot (e.g, VHE, GIC_CPUIF for NMI)
and kernel starts using them, even before the secondary CPUs are brought
up. We would need a way to describe this for each capability.
3) Conflict on a late CPU - When a CPU is brought up, it is checked
against the caps that are known to be enabled on the system (via
verify_local_cpu_capabilities()). Based on the state of the capability
on the CPU vs. that of System we could have the following combinations
of conflict.
x-----------------------------x
| Type | System | Late CPU |
------------------------------|
| a | y | n |
------------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds which requires some
work around, which cannot be delayed. And we ignore (b) for features.
Here, yet again, KPTI is an exception, where if a late CPU needs KPTI we
are too late to enable it (because we change the allocation of ASIDs
etc).
So this calls for a lot more fine grained behavior for each capability.
And if we define all the attributes to control their behavior properly,
we may be able to use a single table for the CPU hwcaps (which cover
errata and features, not the ELF HWCAPs). This is a prepartory step
to get there. More bits would be added for the properties listed above.
We are going to use a bit-mask to encode all the properties of a
capabilities. This patch encodes the "SCOPE" of the capability.
As such there is no change in how the capabilities are treated.
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 105 +++++++++++++++++++++++++++++++++---
arch/arm64/kernel/cpu_errata.c | 8 +--
arch/arm64/kernel/cpufeature.c | 42 +++++++--------
3 files changed, 124 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 8efbda2858a8..13fde0952c31 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -89,16 +89,104 @@ struct arm64_ftr_reg {
extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
-/* scope of capability check */
-enum {
- SCOPE_SYSTEM,
- SCOPE_LOCAL_CPU,
-};
+/*
+ * CPU capabilities:
+ *
+ * We use arm64_cpu_capabilities to represent system features, errata work
+ * arounds (both used internally by kernel and tracked in cpu_hwcaps) and
+ * ELF HWCAPs (which are exposed to user).
+ *
+ * To support systems with heterogeneous CPUs, we need to make sure that we
+ * detect the capabilities correctly on the system and take appropriate
+ * measures to ensure there are no incompatibilities.
+ *
+ * This comment tries to explain how we treat the capabilities.
+ * Each capability has the following list of attributes :
+ *
+ * 1) Scope of Detection : The system detects a given capability by
+ * performing some checks at runtime. This could be, e.g, checking the
+ * value of a field in CPU ID feature register or checking the cpu
+ * model. The capability provides a call back ( @matches() ) to
+ * perform the check. Scope defines how the checks should be performed.
+ * There are two cases:
+ *
+ * a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
+ * matches. This implies, we have to run the check on all the
+ * booting CPUs, until the system decides that state of the
+ * capability is finalised. (See section 2 below)
+ * Or
+ * b) SCOPE_SYSTEM: check all the CPUs and "detect" if all the CPUs
+ * matches. This implies, we run the check only once, when the
+ * system decides to finalise the state of the capability. If the
+ * capability relies on a field in one of the CPU ID feature
+ * registers, we use the sanitised value of the register from the
+ * CPU feature infrastructure to make the decision.
+ *
+ * The process of detection is usually denoted by "update" capability
+ * state in the code.
+ *
+ * 2) Finalise the state : The kernel should finalise the state of a
+ * capability at some point during its execution and take necessary
+ * actions if any. Usually, this is done, after all the boot-time
+ * enabled CPUs are brought up by the kernel, so that it can make
+ * better decision based on the available set of CPUs. However, there
+ * are some special cases, where the action is taken during the early
+ * boot by the primary boot CPU. (e.g, running the kernel at EL2 with
+ * Virtualisation Host Extensions). The kernel usually disallows any
+ * changes to the state of a capability once it finalises the capability
+ * and takes any action, as it may be impossible to execute the actions
+ * safely. A CPU brought up after a capability is "finalised" is
+ * referred to as "Late CPU" w.r.t the capability. e.g, all secondary
+ * CPUs are treated "late CPUs" for capabilities determined by the boot
+ * CPU.
+ *
+ * 3) Verification: When a CPU is brought online (e.g, by user or by the
+ * kernel), the kernel should make sure that it is safe to use the CPU,
+ * by verifying that the CPU is compliant with the state of the
+ * capabilities finalised already. This happens via :
+ *
+ * secondary_start_kernel()-> check_local_cpu_capabilities()
+ *
+ * As explained in (2) above, capabilities could be finalised at
+ * different points in the execution. Each CPU is verified against the
+ * "finalised" capabilities and if there is a conflict, the kernel takes
+ * an action, based on the severity (e.g, a CPU could be prevented from
+ * booting or cause a kernel panic). The CPU is allowed to "affect" the
+ * state of the capability, if it has not been finalised already.
+ *
+ * 4) Action: As mentioned in (2), the kernel can take an action for each
+ * detected capability, on all CPUs on the system. Appropriate actions
+ * include, turning on an architectural feature, modifying the control
+ * registers (e.g, SCTLR, TCR etc.) or patching the kernel via
+ * alternatives. The kernel patching is batched and performed at later
+ * point. The actions are always initiated only after the capability
+ * is finalised. This is usally denoted by "enabling" the capability.
+ * The actions are initiated as follows :
+ * a) Action is triggered on all online CPUs, after the capability is
+ * finalised, invoked within the stop_machine() context from
+ * enable_cpu_capabilitie().
+ *
+ * b) Any late CPU, brought up after (1), the action is triggered via:
+ *
+ * check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
+ *
+ */
+
+
+/* Decide how the capability is detected. On a local CPU vs System wide */
+#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
+#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+#define ARM64_CPUCAP_SCOPE_MASK \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | \
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+
+#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
+#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
- int def_scope; /* default scope */
+ u16 type;
bool (*matches)(const struct arm64_cpu_capabilities *caps, int scope);
/*
* Take the appropriate actions to enable this capability for this CPU.
@@ -127,6 +215,11 @@ struct arm64_cpu_capabilities {
};
};
+static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
+{
+ return cap->type & ARM64_CPUCAP_SCOPE_MASK;
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 0de0ab6a874c..2b5233157af1 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -238,14 +238,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .def_scope = SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -361,7 +361,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -375,7 +375,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .def_scope = SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5fc500c13cfc..1e51fb94125f 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -972,7 +972,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -983,7 +983,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -996,7 +996,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -1007,14 +1007,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1028,21 +1028,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1052,14 +1052,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1067,7 +1067,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1075,7 +1075,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1085,8 +1085,8 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.capability = ARM64_SVE,
- .def_scope = SCOPE_SYSTEM,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
.field_pos = ID_AA64PFR0_SVE_SHIFT,
@@ -1099,7 +1099,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1111,28 +1111,28 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to the PoU not required for I/D coherence",
.capability = ARM64_HAS_CACHE_IDC,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cache_idc,
},
{
.desc = "Instruction cache invalidation not required for I/D coherence",
.capability = ARM64_HAS_CACHE_DIC,
- .def_scope = SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM,
.matches = has_cache_dic,
},
{},
};
-#define HWCAP_CAP(reg, field, s, min_value, type, cap) \
+#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .def_scope = SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
.sign = s, \
.min_field_value = min_value, \
- .hwcap_type = type, \
+ .hwcap_type = cap_type, \
.hwcap = cap, \
}
@@ -1225,7 +1225,7 @@ static void __init setup_elf_hwcaps(const struct arm64_cpu_capabilities *hwcaps)
/* We support emulation of accesses to CPU ID feature registers */
elf_hwcap |= HWCAP_CPUID;
for (; hwcaps->matches; hwcaps++)
- if (hwcaps->matches(hwcaps, hwcaps->def_scope))
+ if (hwcaps->matches(hwcaps, cpucap_default_scope(hwcaps)))
cap_set_elf_hwcap(hwcaps);
}
@@ -1252,7 +1252,7 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
const char *info)
{
for (; caps->matches; caps++) {
- if (!caps->matches(caps, caps->def_scope))
+ if (!caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 05/22] arm64: capabilities: Add flags to handle the conflicts on late CPU
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
When a CPU is brought up, it is checked against the caps that are
known to be enabled on the system (via verify_local_cpu_capabilities()).
Based on the state of the capability on the CPU vs. that of System we
could have the following combinations of conflict.
x-----------------------------x
| Type | System | Late CPU |
|-----------------------------|
| a | y | n |
|-----------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds that cannot be activated
after the kernel has finished booting.And we ignore (b) for features. Here,
yet again, KPTI is an exception, where if a late CPU needs KPTI we are too
late to enable it (because we change the allocation of ASIDs etc).
Add two different flags to indicate how the conflict should be handled.
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - CPUs may have the capability
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - CPUs may not have the cappability.
Now that we have the flags to describe the behavior of the errata and
the features, as we treat them, define types for ERRATUM and FEATURE.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 68 +++++++++++++++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 8 ++---
arch/arm64/kernel/cpufeature.c | 34 +++++++++----------
3 files changed, 89 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 13fde0952c31..b8d8e6012385 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -153,6 +153,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* an action, based on the severity (e.g, a CPU could be prevented from
* booting or cause a kernel panic). The CPU is allowed to "affect" the
* state of the capability, if it has not been finalised already.
+ * See section 5 for more details on conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each
* detected capability, on all CPUs on the system. Appropriate actions
@@ -170,6 +171,34 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*
* check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
*
+ * 5) Conflicts: Based on the state of the capability on a late CPU vs.
+ * the system state, we could have the following combinations :
+ *
+ * x-----------------------------x
+ * | Type | System | Late CPU |
+ * |-----------------------------|
+ * | a | y | n |
+ * |-----------------------------|
+ * | b | n | y |
+ * x-----------------------------x
+ *
+ * Two separate flag bits are defined to indicate whether each kind of
+ * conflict can be allowed:
+ * ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - Case(a) is allowed
+ * ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - Case(b) is allowed
+ *
+ * Case (a) is not permitted for a capability that the system requires
+ * all CPUs to have in order for the capability to be enabled. This is
+ * typical for capabilities that represent enhanced functionality.
+ *
+ * Case (b) is not permitted for a capability that must be enabled
+ * during boot if any CPU in the system requires it in order to run
+ * safely. This is typical for erratum work arounds that cannot be
+ * enabled after the corresponding capability is finalised.
+ *
+ * In some non-typical cases either both (a) and (b), or neither,
+ * should be permitted. This can be described by including neither
+ * or both flags in the capability's type field.
*/
@@ -183,6 +212,33 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+/*
+ * Is it permitted for a late CPU to have this capability when system
+ * hasn't already enabled it ?
+ */
+#define ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU ((u16)BIT(4))
+/* Is it safe for a late CPU to miss this capability when system has it */
+#define ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU ((u16)BIT(5))
+
+/*
+ * CPU errata workarounds that need to be enabled at boot time if one or
+ * more CPUs in the system requires it. When one of these capabilities
+ * has been enabled, it is safe to allow any CPU to boot that doesn't
+ * require the workaround. However, it is not safe if a "late" CPU
+ * requires a workaround and the system hasn't enabled it already.
+ */
+#define ARM64_CPUCAP_LOCAL_CPU_ERRATUM \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on system-wide value of a
+ * feature. It is safe for a late CPU to have this feature even though
+ * the system hasn't enabled it, although the featuer will not be used
+ * by Linux in this case. If the system has enabled this feature already,
+ * then every late CPU must have it.
+ */
+#define ARM64_CPUCAP_SYSTEM_FEATURE \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
@@ -220,6 +276,18 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
return cap->type & ARM64_CPUCAP_SCOPE_MASK;
}
+static inline bool
+cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU);
+}
+
+static inline bool
+cpucap_late_cpu_permitted(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU);
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 2b5233157af1..f5ab9545c5ea 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -238,14 +238,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -361,7 +361,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -375,7 +375,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 1e51fb94125f..5adacd39eb02 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -972,7 +972,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -983,7 +983,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -996,7 +996,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -1007,14 +1007,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1028,21 +1028,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1052,14 +1052,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1067,7 +1067,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1075,7 +1075,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1085,7 +1085,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.capability = ARM64_SVE,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1099,7 +1099,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1111,13 +1111,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to the PoU not required for I/D coherence",
.capability = ARM64_HAS_CACHE_IDC,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cache_idc,
},
{
.desc = "Instruction cache invalidation not required for I/D coherence",
.capability = ARM64_HAS_CACHE_DIC,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cache_dic,
},
{},
@@ -1126,7 +1126,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 05/22] arm64: capabilities: Add flags to handle the conflicts on late CPU
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
When a CPU is brought up, it is checked against the caps that are
known to be enabled on the system (via verify_local_cpu_capabilities()).
Based on the state of the capability on the CPU vs. that of System we
could have the following combinations of conflict.
x-----------------------------x
| Type | System | Late CPU |
|-----------------------------|
| a | y | n |
|-----------------------------|
| b | n | y |
x-----------------------------x
Case (a) is not permitted for caps which are system features, which the
system expects all the CPUs to have (e.g VHE). While (a) is ignored for
all errata work arounds. However, there could be exceptions to the plain
filtering approach. e.g, KPTI is an optional feature for a late CPU as
long as the system already enables it.
Case (b) is not permitted for errata work arounds that cannot be activated
after the kernel has finished booting.And we ignore (b) for features. Here,
yet again, KPTI is an exception, where if a late CPU needs KPTI we are too
late to enable it (because we change the allocation of ASIDs etc).
Add two different flags to indicate how the conflict should be handled.
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - CPUs may have the capability
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - CPUs may not have the cappability.
Now that we have the flags to describe the behavior of the errata and
the features, as we treat them, define types for ERRATUM and FEATURE.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 68 +++++++++++++++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 8 ++---
arch/arm64/kernel/cpufeature.c | 34 +++++++++----------
3 files changed, 89 insertions(+), 21 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 13fde0952c31..b8d8e6012385 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -153,6 +153,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* an action, based on the severity (e.g, a CPU could be prevented from
* booting or cause a kernel panic). The CPU is allowed to "affect" the
* state of the capability, if it has not been finalised already.
+ * See section 5 for more details on conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each
* detected capability, on all CPUs on the system. Appropriate actions
@@ -170,6 +171,34 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*
* check_local_cpu_capabilities() -> verify_local_cpu_capabilities()
*
+ * 5) Conflicts: Based on the state of the capability on a late CPU vs.
+ * the system state, we could have the following combinations :
+ *
+ * x-----------------------------x
+ * | Type | System | Late CPU |
+ * |-----------------------------|
+ * | a | y | n |
+ * |-----------------------------|
+ * | b | n | y |
+ * x-----------------------------x
+ *
+ * Two separate flag bits are defined to indicate whether each kind of
+ * conflict can be allowed:
+ * ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU - Case(a) is allowed
+ * ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU - Case(b) is allowed
+ *
+ * Case (a) is not permitted for a capability that the system requires
+ * all CPUs to have in order for the capability to be enabled. This is
+ * typical for capabilities that represent enhanced functionality.
+ *
+ * Case (b) is not permitted for a capability that must be enabled
+ * during boot if any CPU in the system requires it in order to run
+ * safely. This is typical for erratum work arounds that cannot be
+ * enabled after the corresponding capability is finalised.
+ *
+ * In some non-typical cases either both (a) and (b), or neither,
+ * should be permitted. This can be described by including neither
+ * or both flags in the capability's type field.
*/
@@ -183,6 +212,33 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+/*
+ * Is it permitted for a late CPU to have this capability when system
+ * hasn't already enabled it ?
+ */
+#define ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU ((u16)BIT(4))
+/* Is it safe for a late CPU to miss this capability when system has it */
+#define ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU ((u16)BIT(5))
+
+/*
+ * CPU errata workarounds that need to be enabled at boot time if one or
+ * more CPUs in the system requires it. When one of these capabilities
+ * has been enabled, it is safe to allow any CPU to boot that doesn't
+ * require the workaround. However, it is not safe if a "late" CPU
+ * requires a workaround and the system hasn't enabled it already.
+ */
+#define ARM64_CPUCAP_LOCAL_CPU_ERRATUM \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on system-wide value of a
+ * feature. It is safe for a late CPU to have this feature even though
+ * the system hasn't enabled it, although the featuer will not be used
+ * by Linux in this case. If the system has enabled this feature already,
+ * then every late CPU must have it.
+ */
+#define ARM64_CPUCAP_SYSTEM_FEATURE \
+ (ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
@@ -220,6 +276,18 @@ static inline int cpucap_default_scope(const struct arm64_cpu_capabilities *cap)
return cap->type & ARM64_CPUCAP_SCOPE_MASK;
}
+static inline bool
+cpucap_late_cpu_optional(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU);
+}
+
+static inline bool
+cpucap_late_cpu_permitted(const struct arm64_cpu_capabilities *cap)
+{
+ return !!(cap->type & ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU);
+}
+
extern DECLARE_BITMAP(cpu_hwcaps, ARM64_NCAPS);
extern struct static_key_false cpu_hwcap_keys[ARM64_NCAPS];
extern struct static_key_false arm64_const_caps_ready;
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 2b5233157af1..f5ab9545c5ea 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -238,14 +238,14 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = min, \
.midr_range_max = max
#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU, \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
.matches = is_affected_midr_range, \
.midr_model = model, \
.midr_range_min = 0, \
@@ -361,7 +361,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Mismatched cache line size",
.capability = ARM64_MISMATCHED_CACHE_LINE_SIZE,
.matches = has_mismatched_cache_line_size,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.cpu_enable = cpu_enable_trap_ctr_access,
},
#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003
@@ -375,7 +375,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- .type = ARM64_CPUCAP_SCOPE_LOCAL_CPU,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
.midr_model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 1e51fb94125f..5adacd39eb02 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -972,7 +972,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "GIC system register CPU interface",
.capability = ARM64_HAS_SYSREG_GIC_CPUIF,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_useable_gicv3_cpuif,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.field_pos = ID_AA64PFR0_GIC_SHIFT,
@@ -983,7 +983,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Privileged Access Never",
.capability = ARM64_HAS_PAN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR1_EL1,
.field_pos = ID_AA64MMFR1_PAN_SHIFT,
@@ -996,7 +996,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "LSE atomic instructions",
.capability = ARM64_HAS_LSE_ATOMICS,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR0_EL1,
.field_pos = ID_AA64ISAR0_ATOMICS_SHIFT,
@@ -1007,14 +1007,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
{
.desc = "User Access Override",
.capability = ARM64_HAS_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64MMFR2_EL1,
.field_pos = ID_AA64MMFR2_UAO_SHIFT,
@@ -1028,21 +1028,21 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_PAN
{
.capability = ARM64_ALT_PAN_NOT_UAO,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1052,14 +1052,14 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Reduced HYP mapping offset",
.capability = ARM64_HYP_OFFSET_LOW,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = hyp_offset_low,
},
#ifdef CONFIG_UNMAP_KERNEL_AT_EL0
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
@@ -1067,7 +1067,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
/* FP/SIMD is not implemented */
.capability = ARM64_HAS_NO_FPSIMD,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.min_field_value = 0,
.matches = has_no_fpsimd,
},
@@ -1075,7 +1075,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to Point of Persistence",
.capability = ARM64_HAS_DCPOP,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64ISAR1_EL1,
.field_pos = ID_AA64ISAR1_DPB_SHIFT,
@@ -1085,7 +1085,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#ifdef CONFIG_ARM64_SVE
{
.desc = "Scalable Vector Extension",
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.capability = ARM64_SVE,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1099,7 +1099,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "RAS Extension Support",
.capability = ARM64_HAS_RAS_EXTN,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cpuid_feature,
.sys_reg = SYS_ID_AA64PFR0_EL1,
.sign = FTR_UNSIGNED,
@@ -1111,13 +1111,13 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Data cache clean to the PoU not required for I/D coherence",
.capability = ARM64_HAS_CACHE_IDC,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cache_idc,
},
{
.desc = "Instruction cache invalidation not required for I/D coherence",
.capability = ARM64_HAS_CACHE_DIC,
- .type = ARM64_CPUCAP_SCOPE_SYSTEM,
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cache_dic,
},
{},
@@ -1126,7 +1126,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
#define HWCAP_CAP(reg, field, s, min_value, cap_type, cap) \
{ \
.desc = #cap, \
- .type = ARM64_CPUCAP_SCOPE_SYSTEM, \
+ .type = ARM64_CPUCAP_SYSTEM_FEATURE, \
.matches = has_cpuid_feature, \
.sys_reg = reg, \
.field_pos = field, \
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 06/22] arm64: capabilities: Unify the verification
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Now that each capability describes how to treat the conflicts
of CPU cap state vs System wide cap state, we can unify the
verification logic to a single place.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 91 +++++++++++++++++++++++++++---------------
1 file changed, 58 insertions(+), 33 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5adacd39eb02..d671db5012c6 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1298,6 +1298,58 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
}
}
+/*
+ * Run through the list of capabilities to check for conflicts.
+ * If the system has already detected a capability, take necessary
+ * action on this CPU.
+ *
+ * Returns "false" on conflicts.
+ */
+static bool
+__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+{
+ bool cpu_has_cap, system_has_cap;
+ const struct arm64_cpu_capabilities *caps;
+
+ for (caps = caps_list; caps->matches; caps++) {
+ cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ system_has_cap = cpus_have_cap(caps->capability);
+
+ if (system_has_cap) {
+ /*
+ * Check if the new CPU misses an advertised feature,
+ * which is not safe to miss.
+ */
+ if (!cpu_has_cap && !cpucap_late_cpu_optional(caps))
+ break;
+ /*
+ * We have to issue cpu_enable() irrespective of
+ * whether the CPU has it or not, as it is enabeld
+ * system wide. It is upto the call back to take
+ * appropriate action on this CPU.
+ */
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else {
+ /*
+ * Check if the CPU has this capability if it isn't
+ * safe to have when the system doesn't.
+ */
+ if (cpu_has_cap && !cpucap_late_cpu_permitted(caps))
+ break;
+ }
+ }
+
+ if (caps->matches) {
+ pr_crit("CPU%d: Detected conflict for capability %d (%s), System: %d, CPU: %d\n",
+ smp_processor_id(), caps->capability,
+ caps->desc, system_has_cap, cpu_has_cap);
+ return false;
+ }
+
+ return true;
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1320,25 +1372,10 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void
-verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list)
+static void verify_local_cpu_features(void)
{
- const struct arm64_cpu_capabilities *caps = caps_list;
- for (; caps->matches; caps++) {
- if (!cpus_have_cap(caps->capability))
- continue;
- /*
- * If the new CPU misses an advertised feature, we cannot proceed
- * further, park the cpu.
- */
- if (!__this_cpu_has_cap(caps_list, caps->capability)) {
- pr_crit("CPU%d: missing feature: %s\n",
- smp_processor_id(), caps->desc);
- cpu_die_early();
- }
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- }
+ if (!__verify_local_cpu_caps(arm64_features))
+ cpu_die_early();
}
static void verify_sve_features(void)
@@ -1365,20 +1402,8 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
+ if (!__verify_local_cpu_caps(arm64_errata))
+ cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
@@ -1402,7 +1427,7 @@ static void __init enable_errata_workarounds(void)
static void verify_local_cpu_capabilities(void)
{
verify_local_cpu_errata_workarounds();
- verify_local_cpu_features(arm64_features);
+ verify_local_cpu_features();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 06/22] arm64: capabilities: Unify the verification
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Now that each capability describes how to treat the conflicts
of CPU cap state vs System wide cap state, we can unify the
verification logic to a single place.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 91 +++++++++++++++++++++++++++---------------
1 file changed, 58 insertions(+), 33 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 5adacd39eb02..d671db5012c6 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1298,6 +1298,58 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
}
}
+/*
+ * Run through the list of capabilities to check for conflicts.
+ * If the system has already detected a capability, take necessary
+ * action on this CPU.
+ *
+ * Returns "false" on conflicts.
+ */
+static bool
+__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+{
+ bool cpu_has_cap, system_has_cap;
+ const struct arm64_cpu_capabilities *caps;
+
+ for (caps = caps_list; caps->matches; caps++) {
+ cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ system_has_cap = cpus_have_cap(caps->capability);
+
+ if (system_has_cap) {
+ /*
+ * Check if the new CPU misses an advertised feature,
+ * which is not safe to miss.
+ */
+ if (!cpu_has_cap && !cpucap_late_cpu_optional(caps))
+ break;
+ /*
+ * We have to issue cpu_enable() irrespective of
+ * whether the CPU has it or not, as it is enabeld
+ * system wide. It is upto the call back to take
+ * appropriate action on this CPU.
+ */
+ if (caps->cpu_enable)
+ caps->cpu_enable(caps);
+ } else {
+ /*
+ * Check if the CPU has this capability if it isn't
+ * safe to have when the system doesn't.
+ */
+ if (cpu_has_cap && !cpucap_late_cpu_permitted(caps))
+ break;
+ }
+ }
+
+ if (caps->matches) {
+ pr_crit("CPU%d: Detected conflict for capability %d (%s), System: %d, CPU: %d\n",
+ smp_processor_id(), caps->capability,
+ caps->desc, system_has_cap, cpu_has_cap);
+ return false;
+ }
+
+ return true;
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1320,25 +1372,10 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void
-verify_local_cpu_features(const struct arm64_cpu_capabilities *caps_list)
+static void verify_local_cpu_features(void)
{
- const struct arm64_cpu_capabilities *caps = caps_list;
- for (; caps->matches; caps++) {
- if (!cpus_have_cap(caps->capability))
- continue;
- /*
- * If the new CPU misses an advertised feature, we cannot proceed
- * further, park the cpu.
- */
- if (!__this_cpu_has_cap(caps_list, caps->capability)) {
- pr_crit("CPU%d: missing feature: %s\n",
- smp_processor_id(), caps->desc);
- cpu_die_early();
- }
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- }
+ if (!__verify_local_cpu_caps(arm64_features))
+ cpu_die_early();
}
static void verify_sve_features(void)
@@ -1365,20 +1402,8 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- const struct arm64_cpu_capabilities *caps = arm64_errata;
-
- for (; caps->matches; caps++) {
- if (cpus_have_cap(caps->capability)) {
- if (caps->cpu_enable)
- caps->cpu_enable(caps);
- } else if (caps->matches(caps, SCOPE_LOCAL_CPU)) {
- pr_crit("CPU%d: Requires work around for %s, not detected"
- " at boot time\n",
- smp_processor_id(),
- caps->desc ? : "an erratum");
- cpu_die_early();
- }
- }
+ if (!__verify_local_cpu_caps(arm64_errata))
+ cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
@@ -1402,7 +1427,7 @@ static void __init enable_errata_workarounds(void)
static void verify_local_cpu_capabilities(void)
{
verify_local_cpu_errata_workarounds();
- verify_local_cpu_features(arm64_features);
+ verify_local_cpu_features();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 07/22] arm64: capabilities: Filter the entries based on a given mask
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
While processing the list of capabilities, it is useful to
filter out some of the entries based on the given mask for the
scope of the capabilities to allow better control. This can be
used later for handling LOCAL vs SYSTEM wide capabilities and more.
All capabilities should have their scope set to either LOCAL_CPU or
SYSTEM. No functional/flow change.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/kernel/cpufeature.c | 33 ++++++++++++++++++++++-----------
2 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index b8d8e6012385..c187b926daf9 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -211,6 +211,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
* Is it permitted for a late CPU to have this capability when system
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d671db5012c6..e5229ad553e9 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1249,10 +1249,12 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
}
static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info)
+ u16 scope_mask, const char *info)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
- if (!caps->matches(caps, cpucap_default_scope(caps)))
+ if (!(caps->type & scope_mask) ||
+ !caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
@@ -1274,12 +1276,14 @@ static int __enable_cpu_capability(void *arg)
* CPUs
*/
static void __init
-enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
- if (!cpus_have_cap(num))
+ if (!(caps->type & scope_mask) || !cpus_have_cap(num))
continue;
/* Ensure cpus_have_const_cap(num) works */
@@ -1306,12 +1310,18 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
* Returns "false" on conflicts.
*/
static bool
-__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+ u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
const struct arm64_cpu_capabilities *caps;
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
+
for (caps = caps_list; caps->matches; caps++) {
+ if (!(caps->type & scope_mask))
+ continue;
+
cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
system_has_cap = cpus_have_cap(caps->capability);
@@ -1374,7 +1384,7 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
static void verify_local_cpu_features(void)
{
- if (!__verify_local_cpu_caps(arm64_features))
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
cpu_die_early();
}
@@ -1402,18 +1412,19 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- if (!__verify_local_cpu_caps(arm64_errata))
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void __init enable_errata_workarounds(void)
{
- enable_cpu_capabilities(arm64_errata);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
}
/*
@@ -1459,8 +1470,8 @@ void check_local_cpu_capabilities(void)
static void __init setup_feature_capabilities(void)
{
- update_cpu_capabilities(arm64_features, "detected:");
- enable_cpu_capabilities(arm64_features);
+ update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 07/22] arm64: capabilities: Filter the entries based on a given mask
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
While processing the list of capabilities, it is useful to
filter out some of the entries based on the given mask for the
scope of the capabilities to allow better control. This can be
used later for handling LOCAL vs SYSTEM wide capabilities and more.
All capabilities should have their scope set to either LOCAL_CPU or
SYSTEM. No functional/flow change.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/kernel/cpufeature.c | 33 ++++++++++++++++++++++-----------
2 files changed, 23 insertions(+), 11 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index b8d8e6012385..c187b926daf9 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -211,6 +211,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
* Is it permitted for a late CPU to have this capability when system
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index d671db5012c6..e5229ad553e9 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1249,10 +1249,12 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
}
static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- const char *info)
+ u16 scope_mask, const char *info)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
- if (!caps->matches(caps, cpucap_default_scope(caps)))
+ if (!(caps->type & scope_mask) ||
+ !caps->matches(caps, cpucap_default_scope(caps)))
continue;
if (!cpus_have_cap(caps->capability) && caps->desc)
@@ -1274,12 +1276,14 @@ static int __enable_cpu_capability(void *arg)
* CPUs
*/
static void __init
-enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
+enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask)
{
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
unsigned int num = caps->capability;
- if (!cpus_have_cap(num))
+ if (!(caps->type & scope_mask) || !cpus_have_cap(num))
continue;
/* Ensure cpus_have_const_cap(num) works */
@@ -1306,12 +1310,18 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps)
* Returns "false" on conflicts.
*/
static bool
-__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list)
+__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+ u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
const struct arm64_cpu_capabilities *caps;
+ scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
+
for (caps = caps_list; caps->matches; caps++) {
+ if (!(caps->type & scope_mask))
+ continue;
+
cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
system_has_cap = cpus_have_cap(caps->capability);
@@ -1374,7 +1384,7 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
static void verify_local_cpu_features(void)
{
- if (!__verify_local_cpu_caps(arm64_features))
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
cpu_die_early();
}
@@ -1402,18 +1412,19 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_errata_workarounds(void)
{
- if (!__verify_local_cpu_caps(arm64_errata))
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
cpu_die_early();
}
static void update_cpu_errata_workarounds(void)
{
- update_cpu_capabilities(arm64_errata, "enabling workaround for");
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void __init enable_errata_workarounds(void)
{
- enable_cpu_capabilities(arm64_errata);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
}
/*
@@ -1459,8 +1470,8 @@ void check_local_cpu_capabilities(void)
static void __init setup_feature_capabilities(void)
{
- update_cpu_capabilities(arm64_features, "detected:");
- enable_cpu_capabilities(arm64_features);
+ update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 08/22] arm64: capabilities: Prepare for grouping features and errata work arounds
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
We are about to group the handling of all capabilities (features
and errata workarounds). This patch open codes the wrapper routines
to make it easier to merge the handling.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 58 +++++++++++++-----------------------------
1 file changed, 18 insertions(+), 40 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index e5229ad553e9..4386675af189 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,7 +507,8 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_errata_workarounds(void);
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask, const char *info);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -556,7 +557,8 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1382,12 +1384,6 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void verify_local_cpu_features(void)
-{
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
- cpu_die_early();
-}
-
static void verify_sve_features(void)
{
u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
@@ -1405,27 +1401,6 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-static void verify_local_cpu_errata_workarounds(void)
-{
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
-}
-
-static void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
- "enabling workaround for");
-}
-
-static void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
-}
/*
* Run through the enabled system capabilities and enable() it on this CPU.
@@ -1437,8 +1412,15 @@ static void __init enable_errata_workarounds(void)
*/
static void verify_local_cpu_capabilities(void)
{
- verify_local_cpu_errata_workarounds();
- verify_local_cpu_features();
+ /*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU
+ * requires an errata not detected at boot, fail this CPU.
+ */
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
+ cpu_die_early();
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ cpu_die_early();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1463,17 +1445,12 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
else
verify_local_cpu_capabilities();
}
-static void __init setup_feature_capabilities(void)
-{
- update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
-}
-
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1495,8 +1472,9 @@ void __init setup_cpu_features(void)
u32 cwg;
/* Set the CPU feature capabilies */
- setup_feature_capabilities();
- enable_errata_workarounds();
+ update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 08/22] arm64: capabilities: Prepare for grouping features and errata work arounds
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
We are about to group the handling of all capabilities (features
and errata workarounds). This patch open codes the wrapper routines
to make it easier to merge the handling.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 58 +++++++++++++-----------------------------
1 file changed, 18 insertions(+), 40 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index e5229ad553e9..4386675af189 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,7 +507,8 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_errata_workarounds(void);
+static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask, const char *info);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -556,7 +557,8 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1382,12 +1384,6 @@ verify_local_elf_hwcaps(const struct arm64_cpu_capabilities *caps)
}
}
-static void verify_local_cpu_features(void)
-{
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
- cpu_die_early();
-}
-
static void verify_sve_features(void)
{
u64 safe_zcr = read_sanitised_ftr_reg(SYS_ZCR_EL1);
@@ -1405,27 +1401,6 @@ static void verify_sve_features(void)
/* Add checks on other ZCR bits here if necessary */
}
-/*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU requires
- * an errata not detected at boot, fail this CPU.
- */
-static void verify_local_cpu_errata_workarounds(void)
-{
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
-}
-
-static void update_cpu_errata_workarounds(void)
-{
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
- "enabling workaround for");
-}
-
-static void __init enable_errata_workarounds(void)
-{
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
-}
/*
* Run through the enabled system capabilities and enable() it on this CPU.
@@ -1437,8 +1412,15 @@ static void __init enable_errata_workarounds(void)
*/
static void verify_local_cpu_capabilities(void)
{
- verify_local_cpu_errata_workarounds();
- verify_local_cpu_features();
+ /*
+ * The CPU Errata work arounds are detected and applied at boot time
+ * and the related information is freed soon after. If the new CPU
+ * requires an errata not detected at boot, fail this CPU.
+ */
+ if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
+ cpu_die_early();
+ if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ cpu_die_early();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1463,17 +1445,12 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_errata_workarounds();
+ update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ "enabling workaround for");
else
verify_local_cpu_capabilities();
}
-static void __init setup_feature_capabilities(void)
-{
- update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
-}
-
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1495,8 +1472,9 @@ void __init setup_cpu_features(void)
u32 cwg;
/* Set the CPU feature capabilies */
- setup_feature_capabilities();
- enable_errata_workarounds();
+ update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ enable_cpu_capabilities(arm64_features, SCOPE_ALL);
+ enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 09/22] arm64: capabilities: Split the processing of errata work arounds
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Right now we run through the errata workarounds check on all boot
active CPUs, with SCOPE_ALL. This wouldn't help for detecting erratum
workarounds with a SYSTEM_SCOPE. There are none yet, but we plan to
introduce some: let us clean this up so that such workarounds can be
detected and enabled correctly.
So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
checks are run only once after all the boot time CPUs are active.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4386675af189..9f2152858c05 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -557,7 +557,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
}
@@ -1445,7 +1445,7 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
else
verify_local_cpu_capabilities();
@@ -1473,6 +1473,8 @@ void __init setup_cpu_features(void)
/* Set the CPU feature capabilies */
update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
+ "enabling workaround for");
enable_cpu_capabilities(arm64_features, SCOPE_ALL);
enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 09/22] arm64: capabilities: Split the processing of errata work arounds
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Right now we run through the errata workarounds check on all boot
active CPUs, with SCOPE_ALL. This wouldn't help for detecting erratum
workarounds with a SYSTEM_SCOPE. There are none yet, but we plan to
introduce some: let us clean this up so that such workarounds can be
detected and enabled correctly.
So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
checks are run only once after all the boot time CPUs are active.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4386675af189..9f2152858c05 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -557,7 +557,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around checks on the boot CPU, once we have
* initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
}
@@ -1445,7 +1445,7 @@ void check_local_cpu_capabilities(void)
* advertised capabilities.
*/
if (!sys_caps_initialised)
- update_cpu_capabilities(arm64_errata, SCOPE_ALL,
+ update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
else
verify_local_cpu_capabilities();
@@ -1473,6 +1473,8 @@ void __init setup_cpu_features(void)
/* Set the CPU feature capabilies */
update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
+ "enabling workaround for");
enable_cpu_capabilities(arm64_features, SCOPE_ALL);
enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
mark_const_caps_ready();
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 10/22] arm64: capabilities: Allow features based on local CPU scope
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
So far we have treated the feature capabilities as system wide
and this wouldn't help with features that could be detected locally
on one or more CPUs (e.g, KPTI, Software prefetch). This patch
splits the feature detection to two phases :
1) Local CPU features are checked on all boot time active CPUs.
2) System wide features are checked only once after all CPUs are
active.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9f2152858c05..0a286be446c3 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,6 +507,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
+static const struct arm64_cpu_capabilities arm64_features[];
static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
u16 scope_mask, const char *info);
@@ -554,11 +555,12 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure.
+ * Run the errata work around and local feature checks on the
+ * boot CPU, once we have initialised the cpu feature infrastructure.
*/
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU, "detected:");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1440,15 +1442,18 @@ void check_local_cpu_capabilities(void)
/*
* If we haven't finalised the system capabilities, this CPU gets
- * a chance to update the errata work arounds.
+ * a chance to update the errata work arounds and local features.
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised)
+ if (!sys_caps_initialised) {
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
- else
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
+ "detected:");
+ } else {
verify_local_cpu_capabilities();
+ }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1472,7 +1477,7 @@ void __init setup_cpu_features(void)
u32 cwg;
/* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ update_cpu_capabilities(arm64_features, SCOPE_SYSTEM, "detected:");
update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
"enabling workaround for");
enable_cpu_capabilities(arm64_features, SCOPE_ALL);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 10/22] arm64: capabilities: Allow features based on local CPU scope
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
So far we have treated the feature capabilities as system wide
and this wouldn't help with features that could be detected locally
on one or more CPUs (e.g, KPTI, Software prefetch). This patch
splits the feature detection to two phases :
1) Local CPU features are checked on all boot time active CPUs.
2) System wide features are checked only once after all CPUs are
active.
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 9f2152858c05..0a286be446c3 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,6 +507,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
+static const struct arm64_cpu_capabilities arm64_features[];
static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
u16 scope_mask, const char *info);
@@ -554,11 +555,12 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around checks on the boot CPU, once we have
- * initialised the cpu feature infrastructure.
+ * Run the errata work around and local feature checks on the
+ * boot CPU, once we have initialised the cpu feature infrastructure.
*/
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU, "detected:");
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1440,15 +1442,18 @@ void check_local_cpu_capabilities(void)
/*
* If we haven't finalised the system capabilities, this CPU gets
- * a chance to update the errata work arounds.
+ * a chance to update the errata work arounds and local features.
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised)
+ if (!sys_caps_initialised) {
update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
"enabling workaround for");
- else
+ update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
+ "detected:");
+ } else {
verify_local_cpu_capabilities();
+ }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1472,7 +1477,7 @@ void __init setup_cpu_features(void)
u32 cwg;
/* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_ALL, "detected:");
+ update_cpu_capabilities(arm64_features, SCOPE_SYSTEM, "detected:");
update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
"enabling workaround for");
enable_cpu_capabilities(arm64_features, SCOPE_ALL);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 11/22] arm64: capabilities: Group handling of features and errata workarounds
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Now that the features and errata workarounds have the same
rules and flow, group the handling of the tables.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 73 ++++++++++++++++++++++++------------------
1 file changed, 42 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0a286be446c3..fc6164bc9ef4 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,9 +507,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static const struct arm64_cpu_capabilities arm64_features[];
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask, const char *info);
+static void update_cpu_capabilities(u16 scope_mask);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -558,9 +556,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around and local feature checks on the
* boot CPU, once we have initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU, "detected:");
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1252,8 +1248,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask, const char *info)
+static void __update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask, const char *info)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
@@ -1267,6 +1263,13 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static void update_cpu_capabilities(u16 scope_mask)
+{
+ __update_cpu_capabilities(arm64_features, scope_mask, "detected:");
+ __update_cpu_capabilities(arm64_errata, scope_mask,
+ "enabling workaround for");
+}
+
static int __enable_cpu_capability(void *arg)
{
const struct arm64_cpu_capabilities *cap = arg;
@@ -1280,8 +1283,8 @@ static int __enable_cpu_capability(void *arg)
* CPUs
*/
static void __init
-enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask)
+__enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
@@ -1306,6 +1309,12 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static void __init enable_cpu_capabilities(u16 scope_mask)
+{
+ __enable_cpu_capabilities(arm64_features, scope_mask);
+ __enable_cpu_capabilities(arm64_errata, scope_mask);
+}
+
/*
* Run through the list of capabilities to check for conflicts.
* If the system has already detected a capability, take necessary
@@ -1364,6 +1373,12 @@ __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
return true;
}
+static bool verify_local_cpu_caps(u16 scope_mask)
+{
+ return __verify_local_cpu_caps(arm64_errata, scope_mask) &&
+ __verify_local_cpu_caps(arm64_features, scope_mask);
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1414,15 +1429,9 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- /*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU
- * requires an errata not detected at boot, fail this CPU.
- */
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ if (!verify_local_cpu_caps(SCOPE_ALL))
cpu_die_early();
+
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1446,14 +1455,10 @@ void check_local_cpu_capabilities(void)
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised) {
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
- "detected:");
- } else {
+ if (!sys_caps_initialised)
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ else
verify_local_cpu_capabilities();
- }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1472,16 +1477,22 @@ bool this_cpu_has_cap(unsigned int cap)
__this_cpu_has_cap(arm64_errata, cap));
}
+static void __init setup_system_capabilities(void)
+{
+ /*
+ * We have finalised the system-wide safe feature
+ * registers, finalise the capabilities that depend
+ * on it. Also enable all the available capabilities.
+ */
+ update_cpu_capabilities(SCOPE_SYSTEM);
+ enable_cpu_capabilities(SCOPE_ALL);
+}
+
void __init setup_cpu_features(void)
{
u32 cwg;
- /* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_SYSTEM, "detected:");
- update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
- "enabling workaround for");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
+ setup_system_capabilities();
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 11/22] arm64: capabilities: Group handling of features and errata workarounds
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Now that the features and errata workarounds have the same
rules and flow, group the handling of the tables.
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpufeature.c | 73 ++++++++++++++++++++++++------------------
1 file changed, 42 insertions(+), 31 deletions(-)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0a286be446c3..fc6164bc9ef4 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,9 +507,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static const struct arm64_cpu_capabilities arm64_features[];
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask, const char *info);
+static void update_cpu_capabilities(u16 scope_mask);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -558,9 +556,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
* Run the errata work around and local feature checks on the
* boot CPU, once we have initialised the cpu feature infrastructure.
*/
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU, "detected:");
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1252,8 +1248,8 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
}
-static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask, const char *info)
+static void __update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask, const char *info)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
@@ -1267,6 +1263,13 @@ static void update_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static void update_cpu_capabilities(u16 scope_mask)
+{
+ __update_cpu_capabilities(arm64_features, scope_mask, "detected:");
+ __update_cpu_capabilities(arm64_errata, scope_mask,
+ "enabling workaround for");
+}
+
static int __enable_cpu_capability(void *arg)
{
const struct arm64_cpu_capabilities *cap = arg;
@@ -1280,8 +1283,8 @@ static int __enable_cpu_capability(void *arg)
* CPUs
*/
static void __init
-enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
- u16 scope_mask)
+__enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
+ u16 scope_mask)
{
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
for (; caps->matches; caps++) {
@@ -1306,6 +1309,12 @@ enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
}
}
+static void __init enable_cpu_capabilities(u16 scope_mask)
+{
+ __enable_cpu_capabilities(arm64_features, scope_mask);
+ __enable_cpu_capabilities(arm64_errata, scope_mask);
+}
+
/*
* Run through the list of capabilities to check for conflicts.
* If the system has already detected a capability, take necessary
@@ -1364,6 +1373,12 @@ __verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
return true;
}
+static bool verify_local_cpu_caps(u16 scope_mask)
+{
+ return __verify_local_cpu_caps(arm64_errata, scope_mask) &&
+ __verify_local_cpu_caps(arm64_features, scope_mask);
+}
+
/*
* Check for CPU features that are used in early boot
* based on the Boot CPU value.
@@ -1414,15 +1429,9 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- /*
- * The CPU Errata work arounds are detected and applied at boot time
- * and the related information is freed soon after. If the new CPU
- * requires an errata not detected at boot, fail this CPU.
- */
- if (!__verify_local_cpu_caps(arm64_errata, SCOPE_ALL))
- cpu_die_early();
- if (!__verify_local_cpu_caps(arm64_features, SCOPE_ALL))
+ if (!verify_local_cpu_caps(SCOPE_ALL))
cpu_die_early();
+
verify_local_elf_hwcaps(arm64_elf_hwcaps);
if (system_supports_32bit_el0())
@@ -1446,14 +1455,10 @@ void check_local_cpu_capabilities(void)
* Otherwise, this CPU should verify that it has all the system
* advertised capabilities.
*/
- if (!sys_caps_initialised) {
- update_cpu_capabilities(arm64_errata, SCOPE_LOCAL_CPU,
- "enabling workaround for");
- update_cpu_capabilities(arm64_features, SCOPE_LOCAL_CPU,
- "detected:");
- } else {
+ if (!sys_caps_initialised)
+ update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ else
verify_local_cpu_capabilities();
- }
}
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
@@ -1472,16 +1477,22 @@ bool this_cpu_has_cap(unsigned int cap)
__this_cpu_has_cap(arm64_errata, cap));
}
+static void __init setup_system_capabilities(void)
+{
+ /*
+ * We have finalised the system-wide safe feature
+ * registers, finalise the capabilities that depend
+ * on it. Also enable all the available capabilities.
+ */
+ update_cpu_capabilities(SCOPE_SYSTEM);
+ enable_cpu_capabilities(SCOPE_ALL);
+}
+
void __init setup_cpu_features(void)
{
u32 cwg;
- /* Set the CPU feature capabilies */
- update_cpu_capabilities(arm64_features, SCOPE_SYSTEM, "detected:");
- update_cpu_capabilities(arm64_errata, SCOPE_SYSTEM,
- "enabling workaround for");
- enable_cpu_capabilities(arm64_features, SCOPE_ALL);
- enable_cpu_capabilities(arm64_errata, SCOPE_ALL);
+ setup_system_capabilities();
mark_const_caps_ready();
setup_elf_hwcaps(arm64_elf_hwcaps);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 12/22] arm64: capabilities: Introduce weak features based on local CPU
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Now that we have the flexibility of defining system features based
on individual CPUs, introduce CPU feature type that can be detected
on a local SCOPE and ignores the conflict on late CPUs. This is
applicable for ARM64_HAS_NO_HW_PREFETCH, where it is fine for
the system to have CPUs without hardware prefetch turning up
later. We only suffer a performance penalty, nothing fatal.
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 8 ++++++++
arch/arm64/kernel/cpufeature.c | 2 +-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index c187b926daf9..effb1c038221 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -239,6 +239,14 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
#define ARM64_CPUCAP_SYSTEM_FEATURE \
(ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on feature of one or more CPUs.
+ * All possible conflicts for a late CPU are ignored.
+ */
+#define ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
+ ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
struct arm64_cpu_capabilities {
const char *desc;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index fc6164bc9ef4..0955e02fa638 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1007,7 +1007,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 12/22] arm64: capabilities: Introduce weak features based on local CPU
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Now that we have the flexibility of defining system features based
on individual CPUs, introduce CPU feature type that can be detected
on a local SCOPE and ignores the conflict on late CPUs. This is
applicable for ARM64_HAS_NO_HW_PREFETCH, where it is fine for
the system to have CPUs without hardware prefetch turning up
later. We only suffer a performance penalty, nothing fatal.
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 8 ++++++++
arch/arm64/kernel/cpufeature.c | 2 +-
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index c187b926daf9..effb1c038221 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -239,6 +239,14 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
#define ARM64_CPUCAP_SYSTEM_FEATURE \
(ARM64_CPUCAP_SCOPE_SYSTEM | ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time based on feature of one or more CPUs.
+ * All possible conflicts for a late CPU are ignored.
+ */
+#define ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
+ ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
struct arm64_cpu_capabilities {
const char *desc;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index fc6164bc9ef4..0955e02fa638 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1007,7 +1007,7 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Software prefetching using PRFM",
.capability = ARM64_HAS_NO_HW_PREFETCH,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
.matches = has_no_hw_prefetch,
},
#ifdef CONFIG_ARM64_UAO
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 13/22] arm64: capabilities: Restrict KPTI detection to boot-time CPUs
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
KPTI is treated as a system wide feature and is only detected if all
the CPUs in the sysetm needs the defense, unless it is forced via kernel
command line. This leaves a system with a mix of CPUs with and without
the defense vulnerable. Also, if a late CPU needs KPTI but KPTI was not
activated at boot time, the CPU is currently allowed to boot, which is a
potential security vulnerability.
This patch ensures that the KPTI is turned on if at least one CPU detects
the capability (i.e, change scope to SCOPE_LOCAL_CPU). Also rejetcs a late
CPU, if it requires the defense, when the system hasn't enabled it,
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 9 +++++++++
arch/arm64/kernel/cpufeature.c | 16 +++++++++++-----
2 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index effb1c038221..4f050259dff3 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -248,6 +248,15 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time, on one or more CPUs. A late CPU
+ * is not allowed to have the capability when the system doesn't have it.
+ * It is Ok for a late CPU to miss the feature.
+ */
+#define ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0955e02fa638..0bbab4bd7644 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -877,10 +877,9 @@ static bool has_cache_dic(const struct arm64_cpu_capabilities *entry,
static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
- int __unused)
+ int scope)
{
char const *str = "command line option";
- u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
/*
* For reasons that aren't entirely clear, enabling KPTI on Cavium
@@ -911,8 +910,7 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
}
/* Defer to CPU feature registers */
- return !cpuid_feature_extract_unsigned_field(pfr0,
- ID_AA64PFR0_CSV3_SHIFT);
+ return !has_cpuid_feature(entry, scope);
}
static void
@@ -1059,7 +1057,15 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE,
+ /*
+ * The ID feature fields below are used to indicate that
+ * the CPU doesn't need KPTI. See unmap_kernel_at_el0 for
+ * more details.
+ */
+ .sys_reg = SYS_ID_AA64PFR0_EL1,
+ .field_pos = ID_AA64PFR0_CSV3_SHIFT,
+ .min_field_value = 1,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 13/22] arm64: capabilities: Restrict KPTI detection to boot-time CPUs
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
KPTI is treated as a system wide feature and is only detected if all
the CPUs in the sysetm needs the defense, unless it is forced via kernel
command line. This leaves a system with a mix of CPUs with and without
the defense vulnerable. Also, if a late CPU needs KPTI but KPTI was not
activated at boot time, the CPU is currently allowed to boot, which is a
potential security vulnerability.
This patch ensures that the KPTI is turned on if at least one CPU detects
the capability (i.e, change scope to SCOPE_LOCAL_CPU). Also rejetcs a late
CPU, if it requires the defense, when the system hasn't enabled it,
Cc: Will Deacon <will.deacon@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 9 +++++++++
arch/arm64/kernel/cpufeature.c | 16 +++++++++++-----
2 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index effb1c038221..4f050259dff3 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -248,6 +248,15 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU | \
ARM64_CPUCAP_PERMITTED_FOR_LATE_CPU)
+/*
+ * CPU feature detected at boot time, on one or more CPUs. A late CPU
+ * is not allowed to have the capability when the system doesn't have it.
+ * It is Ok for a late CPU to miss the feature.
+ */
+#define ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE \
+ (ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0955e02fa638..0bbab4bd7644 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -877,10 +877,9 @@ static bool has_cache_dic(const struct arm64_cpu_capabilities *entry,
static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
- int __unused)
+ int scope)
{
char const *str = "command line option";
- u64 pfr0 = read_sanitised_ftr_reg(SYS_ID_AA64PFR0_EL1);
/*
* For reasons that aren't entirely clear, enabling KPTI on Cavium
@@ -911,8 +910,7 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
}
/* Defer to CPU feature registers */
- return !cpuid_feature_extract_unsigned_field(pfr0,
- ID_AA64PFR0_CSV3_SHIFT);
+ return !has_cpuid_feature(entry, scope);
}
static void
@@ -1059,7 +1057,15 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
{
.desc = "Kernel page table isolation (KPTI)",
.capability = ARM64_UNMAP_KERNEL_AT_EL0,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_BOOT_RESTRICTED_CPU_LOCAL_FEATURE,
+ /*
+ * The ID feature fields below are used to indicate that
+ * the CPU doesn't need KPTI. See unmap_kernel_at_el0 for
+ * more details.
+ */
+ .sys_reg = SYS_ID_AA64PFR0_EL1,
+ .field_pos = ID_AA64PFR0_CSV3_SHIFT,
+ .min_field_value = 1,
.matches = unmap_kernel_at_el0,
.cpu_enable = kpti_install_ng_mappings,
},
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 14/22] arm64: capabilities: Add support for features enabled early
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose, Julien Thierry
The kernel detects and uses some of the features based on the boot
CPU and expects that all the following CPUs conform to it. e.g,
with VHE and the boot CPU running at EL2, the kernel decides to
keep the kernel running at EL2. If another CPU is brought up without
this capability, we use custom hooks (via check_early_cpu_features())
to handle it. To handle such capabilities add support for detecting
and enabling capabilities based on the boot CPU.
A bit is added to indicate if the capability should be detected
early on the boot CPU. The infrastructure then ensures that such
capabilities are probed and "enabled" early on in the boot CPU
and, enabled on the subsequent CPUs.
Cc: Julien Thierry <julien.thierry@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 48 +++++++++++++++++++++++++------
arch/arm64/kernel/cpufeature.c | 57 ++++++++++++++++++++++++++++---------
2 files changed, 83 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 4f050259dff3..2f5edefdff99 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -108,7 +108,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* value of a field in CPU ID feature register or checking the cpu
* model. The capability provides a call back ( @matches() ) to
* perform the check. Scope defines how the checks should be performed.
- * There are two cases:
+ * There are three cases:
*
* a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
* matches. This implies, we have to run the check on all the
@@ -121,6 +121,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* capability relies on a field in one of the CPU ID feature
* registers, we use the sanitised value of the register from the
* CPU feature infrastructure to make the decision.
+ * Or
+ * c) SCOPE_BOOT_CPU: Check only on the primary boot CPU to detect the
+ * feature. This category is for features that are "finalised"
+ * (or used) by the kernel very early even before the SMP cpus
+ * are brought up.
*
* The process of detection is usually denoted by "update" capability
* state in the code.
@@ -140,6 +145,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* CPUs are treated "late CPUs" for capabilities determined by the boot
* CPU.
*
+ * At the moment there are two passes of finalising the capabilities.
+ * a) Boot CPU scope capabilities - Finalised by primary boot CPU via
+ * setup_boot_cpu_capabilities().
+ * b) Everything except (a) - Run via setup_system_capabilities().
+ *
* 3) Verification: When a CPU is brought online (e.g, by user or by the
* kernel), the kernel should make sure that it is safe to use the CPU,
* by verifying that the CPU is compliant with the state of the
@@ -148,12 +158,21 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* secondary_start_kernel()-> check_local_cpu_capabilities()
*
* As explained in (2) above, capabilities could be finalised at
- * different points in the execution. Each CPU is verified against the
- * "finalised" capabilities and if there is a conflict, the kernel takes
- * an action, based on the severity (e.g, a CPU could be prevented from
- * booting or cause a kernel panic). The CPU is allowed to "affect" the
- * state of the capability, if it has not been finalised already.
- * See section 5 for more details on conflicts.
+ * different points in the execution. Each newly booted CPU is verified
+ * against the capabilities that have been finalised by the time it
+ * boots.
+ *
+ * a) SCOPE_BOOT_CPU : All CPUs are verified against the capability
+ * except for the primary boot CPU.
+ *
+ * b) SCOPE_LOCAL_CPU, SCOPE_SYSTEM: All CPUs hotplugged on by the
+ * user after the kernel boot are verified against the capability.
+ *
+ * If there is a conflict, the kernel takes an action, based on the
+ * severity (e.g, a CPU could be prevented from booting or cause a
+ * kernel panic). The CPU is allowed to "affect" the state of the
+ * capability, if it has not been finalised already. See section 5
+ * for more details on conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each
* detected capability, on all CPUs on the system. Appropriate actions
@@ -202,15 +221,26 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
-/* Decide how the capability is detected. On a local CPU vs System wide */
+/*
+ * Decide how the capability is detected.
+ * On any local CPU vs System wide vs the primary boot CPU
+ */
#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+/*
+ * The capabilitiy is detected on the Boot CPU and is used by kernel
+ * during early boot. i.e, the capability should be "detected" and
+ * "enabled" as early as possibly on all booting CPUs.
+ */
+#define ARM64_CPUCAP_SCOPE_BOOT_CPU ((u16)BIT(2))
#define ARM64_CPUCAP_SCOPE_MASK \
(ARM64_CPUCAP_SCOPE_SYSTEM | \
- ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_SCOPE_BOOT_CPU)
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_BOOT_CPU ARM64_CPUCAP_SCOPE_BOOT_CPU
#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0bbab4bd7644..e9e81aabfe91 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,7 +507,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_capabilities(u16 scope_mask);
+static void __init setup_boot_cpu_capabilities(void);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -553,10 +553,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around and local feature checks on the
- * boot CPU, once we have initialised the cpu feature infrastructure.
+ * Detect and enable early CPU capabilities based on the boot CPU,
+ * after we have initialised the CPU feature infrastructure.
*/
- update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ setup_boot_cpu_capabilities();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1304,13 +1304,24 @@ __enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
if (caps->cpu_enable) {
/*
- * Use stop_machine() as it schedules the work allowing
- * us to modify PSTATE, instead of on_each_cpu() which
- * uses an IPI, giving us a PSTATE that disappears when
- * we return.
+ * Capabilities with SCOPE_BOOT_CPU scope are finalised
+ * before any secondary CPU boots. Thus, each secondary
+ * will enable the capability as appropriate via
+ * check_local_cpu_capabilities(). The only exception is
+ * the boot CPU, for which the capability must be
+ * enabled here. This approach avoids costly
+ * stop_machine() calls for this case.
+ *
+ * Otherwise, use stop_machine() as it schedules the
+ * work allowing us to modify PSTATE, instead of
+ * on_each_cpu() which uses an IPI, giving us a PSTATE
+ * that disappears when we return.
*/
- stop_machine(__enable_cpu_capability, (void *)caps,
- cpu_online_mask);
+ if (scope_mask & SCOPE_BOOT_CPU)
+ caps->cpu_enable(caps);
+ else
+ stop_machine(__enable_cpu_capability,
+ (void *)caps, cpu_online_mask);
}
}
}
@@ -1393,6 +1404,12 @@ static void check_early_cpu_features(void)
{
verify_cpu_run_el();
verify_cpu_asid_bits();
+ /*
+ * Early features are used by the kernel already. If there
+ * is a conflict, we cannot proceed further.
+ */
+ if (!verify_local_cpu_caps(SCOPE_BOOT_CPU))
+ cpu_panic_kernel();
}
static void
@@ -1435,7 +1452,12 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- if (!verify_local_cpu_caps(SCOPE_ALL))
+ /*
+ * The capabilities with SCOPE_BOOT_CPU are checked from
+ * check_early_cpu_features(), as they need to be verified
+ * on all secondary CPUs.
+ */
+ if (!verify_local_cpu_caps(SCOPE_ALL & ~SCOPE_BOOT_CPU))
cpu_die_early();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
@@ -1467,6 +1489,14 @@ void check_local_cpu_capabilities(void)
verify_local_cpu_capabilities();
}
+static void __init setup_boot_cpu_capabilities(void)
+{
+ /* Detect capabilities with either SCOPE_BOOT_CPU or SCOPE_LOCAL_CPU */
+ update_cpu_capabilities(SCOPE_BOOT_CPU | SCOPE_LOCAL_CPU);
+ /* Enable the SCOPE_BOOT_CPU capabilities alone right away */
+ enable_cpu_capabilities(SCOPE_BOOT_CPU);
+}
+
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1488,10 +1518,11 @@ static void __init setup_system_capabilities(void)
/*
* We have finalised the system-wide safe feature
* registers, finalise the capabilities that depend
- * on it. Also enable all the available capabilities.
+ * on it. Also enable all the available capabilities,
+ * that are not enabled already.
*/
update_cpu_capabilities(SCOPE_SYSTEM);
- enable_cpu_capabilities(SCOPE_ALL);
+ enable_cpu_capabilities(SCOPE_ALL & ~SCOPE_BOOT_CPU);
}
void __init setup_cpu_features(void)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 14/22] arm64: capabilities: Add support for features enabled early
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
The kernel detects and uses some of the features based on the boot
CPU and expects that all the following CPUs conform to it. e.g,
with VHE and the boot CPU running at EL2, the kernel decides to
keep the kernel running at EL2. If another CPU is brought up without
this capability, we use custom hooks (via check_early_cpu_features())
to handle it. To handle such capabilities add support for detecting
and enabling capabilities based on the boot CPU.
A bit is added to indicate if the capability should be detected
early on the boot CPU. The infrastructure then ensures that such
capabilities are probed and "enabled" early on in the boot CPU
and, enabled on the subsequent CPUs.
Cc: Julien Thierry <julien.thierry@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Marc Zyngier <marc.zyngier@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 48 +++++++++++++++++++++++++------
arch/arm64/kernel/cpufeature.c | 57 ++++++++++++++++++++++++++++---------
2 files changed, 83 insertions(+), 22 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 4f050259dff3..2f5edefdff99 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -108,7 +108,7 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* value of a field in CPU ID feature register or checking the cpu
* model. The capability provides a call back ( @matches() ) to
* perform the check. Scope defines how the checks should be performed.
- * There are two cases:
+ * There are three cases:
*
* a) SCOPE_LOCAL_CPU: check all the CPUs and "detect" if at least one
* matches. This implies, we have to run the check on all the
@@ -121,6 +121,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* capability relies on a field in one of the CPU ID feature
* registers, we use the sanitised value of the register from the
* CPU feature infrastructure to make the decision.
+ * Or
+ * c) SCOPE_BOOT_CPU: Check only on the primary boot CPU to detect the
+ * feature. This category is for features that are "finalised"
+ * (or used) by the kernel very early even before the SMP cpus
+ * are brought up.
*
* The process of detection is usually denoted by "update" capability
* state in the code.
@@ -140,6 +145,11 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* CPUs are treated "late CPUs" for capabilities determined by the boot
* CPU.
*
+ * At the moment there are two passes of finalising the capabilities.
+ * a) Boot CPU scope capabilities - Finalised by primary boot CPU via
+ * setup_boot_cpu_capabilities().
+ * b) Everything except (a) - Run via setup_system_capabilities().
+ *
* 3) Verification: When a CPU is brought online (e.g, by user or by the
* kernel), the kernel should make sure that it is safe to use the CPU,
* by verifying that the CPU is compliant with the state of the
@@ -148,12 +158,21 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
* secondary_start_kernel()-> check_local_cpu_capabilities()
*
* As explained in (2) above, capabilities could be finalised at
- * different points in the execution. Each CPU is verified against the
- * "finalised" capabilities and if there is a conflict, the kernel takes
- * an action, based on the severity (e.g, a CPU could be prevented from
- * booting or cause a kernel panic). The CPU is allowed to "affect" the
- * state of the capability, if it has not been finalised already.
- * See section 5 for more details on conflicts.
+ * different points in the execution. Each newly booted CPU is verified
+ * against the capabilities that have been finalised by the time it
+ * boots.
+ *
+ * a) SCOPE_BOOT_CPU : All CPUs are verified against the capability
+ * except for the primary boot CPU.
+ *
+ * b) SCOPE_LOCAL_CPU, SCOPE_SYSTEM: All CPUs hotplugged on by the
+ * user after the kernel boot are verified against the capability.
+ *
+ * If there is a conflict, the kernel takes an action, based on the
+ * severity (e.g, a CPU could be prevented from booting or cause a
+ * kernel panic). The CPU is allowed to "affect" the state of the
+ * capability, if it has not been finalised already. See section 5
+ * for more details on conflicts.
*
* 4) Action: As mentioned in (2), the kernel can take an action for each
* detected capability, on all CPUs on the system. Appropriate actions
@@ -202,15 +221,26 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
*/
-/* Decide how the capability is detected. On a local CPU vs System wide */
+/*
+ * Decide how the capability is detected.
+ * On any local CPU vs System wide vs the primary boot CPU
+ */
#define ARM64_CPUCAP_SCOPE_LOCAL_CPU ((u16)BIT(0))
#define ARM64_CPUCAP_SCOPE_SYSTEM ((u16)BIT(1))
+/*
+ * The capabilitiy is detected on the Boot CPU and is used by kernel
+ * during early boot. i.e, the capability should be "detected" and
+ * "enabled" as early as possibly on all booting CPUs.
+ */
+#define ARM64_CPUCAP_SCOPE_BOOT_CPU ((u16)BIT(2))
#define ARM64_CPUCAP_SCOPE_MASK \
(ARM64_CPUCAP_SCOPE_SYSTEM | \
- ARM64_CPUCAP_SCOPE_LOCAL_CPU)
+ ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
+ ARM64_CPUCAP_SCOPE_BOOT_CPU)
#define SCOPE_SYSTEM ARM64_CPUCAP_SCOPE_SYSTEM
#define SCOPE_LOCAL_CPU ARM64_CPUCAP_SCOPE_LOCAL_CPU
+#define SCOPE_BOOT_CPU ARM64_CPUCAP_SCOPE_BOOT_CPU
#define SCOPE_ALL ARM64_CPUCAP_SCOPE_MASK
/*
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 0bbab4bd7644..e9e81aabfe91 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -507,7 +507,7 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
}
extern const struct arm64_cpu_capabilities arm64_errata[];
-static void update_cpu_capabilities(u16 scope_mask);
+static void __init setup_boot_cpu_capabilities(void);
void __init init_cpu_features(struct cpuinfo_arm64 *info)
{
@@ -553,10 +553,10 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)
}
/*
- * Run the errata work around and local feature checks on the
- * boot CPU, once we have initialised the cpu feature infrastructure.
+ * Detect and enable early CPU capabilities based on the boot CPU,
+ * after we have initialised the CPU feature infrastructure.
*/
- update_cpu_capabilities(SCOPE_LOCAL_CPU);
+ setup_boot_cpu_capabilities();
}
static void update_cpu_ftr_reg(struct arm64_ftr_reg *reg, u64 new)
@@ -1304,13 +1304,24 @@ __enable_cpu_capabilities(const struct arm64_cpu_capabilities *caps,
if (caps->cpu_enable) {
/*
- * Use stop_machine() as it schedules the work allowing
- * us to modify PSTATE, instead of on_each_cpu() which
- * uses an IPI, giving us a PSTATE that disappears when
- * we return.
+ * Capabilities with SCOPE_BOOT_CPU scope are finalised
+ * before any secondary CPU boots. Thus, each secondary
+ * will enable the capability as appropriate via
+ * check_local_cpu_capabilities(). The only exception is
+ * the boot CPU, for which the capability must be
+ * enabled here. This approach avoids costly
+ * stop_machine() calls for this case.
+ *
+ * Otherwise, use stop_machine() as it schedules the
+ * work allowing us to modify PSTATE, instead of
+ * on_each_cpu() which uses an IPI, giving us a PSTATE
+ * that disappears when we return.
*/
- stop_machine(__enable_cpu_capability, (void *)caps,
- cpu_online_mask);
+ if (scope_mask & SCOPE_BOOT_CPU)
+ caps->cpu_enable(caps);
+ else
+ stop_machine(__enable_cpu_capability,
+ (void *)caps, cpu_online_mask);
}
}
}
@@ -1393,6 +1404,12 @@ static void check_early_cpu_features(void)
{
verify_cpu_run_el();
verify_cpu_asid_bits();
+ /*
+ * Early features are used by the kernel already. If there
+ * is a conflict, we cannot proceed further.
+ */
+ if (!verify_local_cpu_caps(SCOPE_BOOT_CPU))
+ cpu_panic_kernel();
}
static void
@@ -1435,7 +1452,12 @@ static void verify_sve_features(void)
*/
static void verify_local_cpu_capabilities(void)
{
- if (!verify_local_cpu_caps(SCOPE_ALL))
+ /*
+ * The capabilities with SCOPE_BOOT_CPU are checked from
+ * check_early_cpu_features(), as they need to be verified
+ * on all secondary CPUs.
+ */
+ if (!verify_local_cpu_caps(SCOPE_ALL & ~SCOPE_BOOT_CPU))
cpu_die_early();
verify_local_elf_hwcaps(arm64_elf_hwcaps);
@@ -1467,6 +1489,14 @@ void check_local_cpu_capabilities(void)
verify_local_cpu_capabilities();
}
+static void __init setup_boot_cpu_capabilities(void)
+{
+ /* Detect capabilities with either SCOPE_BOOT_CPU or SCOPE_LOCAL_CPU */
+ update_cpu_capabilities(SCOPE_BOOT_CPU | SCOPE_LOCAL_CPU);
+ /* Enable the SCOPE_BOOT_CPU capabilities alone right away */
+ enable_cpu_capabilities(SCOPE_BOOT_CPU);
+}
+
DEFINE_STATIC_KEY_FALSE(arm64_const_caps_ready);
EXPORT_SYMBOL(arm64_const_caps_ready);
@@ -1488,10 +1518,11 @@ static void __init setup_system_capabilities(void)
/*
* We have finalised the system-wide safe feature
* registers, finalise the capabilities that depend
- * on it. Also enable all the available capabilities.
+ * on it. Also enable all the available capabilities,
+ * that are not enabled already.
*/
update_cpu_capabilities(SCOPE_SYSTEM);
- enable_cpu_capabilities(SCOPE_ALL);
+ enable_cpu_capabilities(SCOPE_ALL & ~SCOPE_BOOT_CPU);
}
void __init setup_cpu_features(void)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
We expect all CPUs to be running at the same EL inside the kernel
with or without VHE enabled and we have strict checks to ensure
that any mismatch triggers a kernel panic. If VHE is enabled,
we use the feature based on the boot CPU and all other CPUs
should follow. This makes it a perfect candidate for a cpability
based on the boot CPU, which should be matched by all the CPUs
(both when is ON and OFF). This saves us some not-so-pretty
hooks and special code, just for verifying the conflict.
The patch also makes the VHE capability entry depend on
CONFIG_ARM64_VHE.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 6 ++++++
arch/arm64/include/asm/virt.h | 6 ------
arch/arm64/kernel/cpufeature.c | 5 +++--
arch/arm64/kernel/smp.c | 38 -------------------------------------
4 files changed, 9 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 2f5edefdff99..6a1280493f57 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -287,6 +287,12 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
(ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * CPU feature used early in the boot based on the boot CPU. All secondary
+ * CPUs must match the state of the capability as detected by the boot CPU.
+ */
+#define ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE ARM64_CPUCAP_SCOPE_BOOT_CPU
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index c5f89442785c..9d1e24e030b3 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -102,12 +102,6 @@ static inline bool has_vhe(void)
return false;
}
-#ifdef CONFIG_ARM64_VHE
-extern void verify_cpu_run_el(void);
-#else
-static inline void verify_cpu_run_el(void) {}
-#endif
-
#endif /* __ASSEMBLY__ */
#endif /* ! __ASM__VIRT_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index e9e81aabfe91..006e95d46a4a 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1030,13 +1030,15 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
+#ifdef CONFIG_ARM64_VHE
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
+#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
@@ -1402,7 +1404,6 @@ static bool verify_local_cpu_caps(u16 scope_mask)
*/
static void check_early_cpu_features(void)
{
- verify_cpu_run_el();
verify_cpu_asid_bits();
/*
* Early features are used by the kernel already. If there
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 5cef11450183..f3e2e3aec0b0 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -85,43 +85,6 @@ enum ipi_msg_type {
IPI_WAKEUP
};
-#ifdef CONFIG_ARM64_VHE
-
-/* Whether the boot CPU is running in HYP mode or not*/
-static bool boot_cpu_hyp_mode;
-
-static inline void save_boot_cpu_run_el(void)
-{
- boot_cpu_hyp_mode = is_kernel_in_hyp_mode();
-}
-
-static inline bool is_boot_cpu_in_hyp_mode(void)
-{
- return boot_cpu_hyp_mode;
-}
-
-/*
- * Verify that a secondary CPU is running the kernel at the same
- * EL as that of the boot CPU.
- */
-void verify_cpu_run_el(void)
-{
- bool in_el2 = is_kernel_in_hyp_mode();
- bool boot_cpu_el2 = is_boot_cpu_in_hyp_mode();
-
- if (in_el2 ^ boot_cpu_el2) {
- pr_crit("CPU%d: mismatched Exception Level(EL%d) with boot CPU(EL%d)\n",
- smp_processor_id(),
- in_el2 ? 2 : 1,
- boot_cpu_el2 ? 2 : 1);
- cpu_panic_kernel();
- }
-}
-
-#else
-static inline void save_boot_cpu_run_el(void) {}
-#endif
-
#ifdef CONFIG_HOTPLUG_CPU
static int op_cpu_kill(unsigned int cpu);
#else
@@ -447,7 +410,6 @@ void __init smp_prepare_boot_cpu(void)
*/
jump_label_init();
cpuinfo_store_boot_cpu();
- save_boot_cpu_run_el();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
We expect all CPUs to be running at the same EL inside the kernel
with or without VHE enabled and we have strict checks to ensure
that any mismatch triggers a kernel panic. If VHE is enabled,
we use the feature based on the boot CPU and all other CPUs
should follow. This makes it a perfect candidate for a cpability
based on the boot CPU, which should be matched by all the CPUs
(both when is ON and OFF). This saves us some not-so-pretty
hooks and special code, just for verifying the conflict.
The patch also makes the VHE capability entry depend on
CONFIG_ARM64_VHE.
Cc: Marc Zyngier <marc.zyngier@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Cc: Will Deacon <will.deacon@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 6 ++++++
arch/arm64/include/asm/virt.h | 6 ------
arch/arm64/kernel/cpufeature.c | 5 +++--
arch/arm64/kernel/smp.c | 38 -------------------------------------
4 files changed, 9 insertions(+), 46 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 2f5edefdff99..6a1280493f57 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -287,6 +287,12 @@ extern struct arm64_ftr_reg arm64_ftr_reg_ctrel0;
(ARM64_CPUCAP_SCOPE_LOCAL_CPU | \
ARM64_CPUCAP_OPTIONAL_FOR_LATE_CPU)
+/*
+ * CPU feature used early in the boot based on the boot CPU. All secondary
+ * CPUs must match the state of the capability as detected by the boot CPU.
+ */
+#define ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE ARM64_CPUCAP_SCOPE_BOOT_CPU
+
struct arm64_cpu_capabilities {
const char *desc;
u16 capability;
diff --git a/arch/arm64/include/asm/virt.h b/arch/arm64/include/asm/virt.h
index c5f89442785c..9d1e24e030b3 100644
--- a/arch/arm64/include/asm/virt.h
+++ b/arch/arm64/include/asm/virt.h
@@ -102,12 +102,6 @@ static inline bool has_vhe(void)
return false;
}
-#ifdef CONFIG_ARM64_VHE
-extern void verify_cpu_run_el(void);
-#else
-static inline void verify_cpu_run_el(void) {}
-#endif
-
#endif /* __ASSEMBLY__ */
#endif /* ! __ASM__VIRT_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index e9e81aabfe91..006e95d46a4a 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1030,13 +1030,15 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = cpufeature_pan_not_uao,
},
#endif /* CONFIG_ARM64_PAN */
+#ifdef CONFIG_ARM64_VHE
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
- .type = ARM64_CPUCAP_SYSTEM_FEATURE,
+ .type = ARM64_CPUCAP_STRICT_BOOT_CPU_FEATURE,
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
+#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
@@ -1402,7 +1404,6 @@ static bool verify_local_cpu_caps(u16 scope_mask)
*/
static void check_early_cpu_features(void)
{
- verify_cpu_run_el();
verify_cpu_asid_bits();
/*
* Early features are used by the kernel already. If there
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index 5cef11450183..f3e2e3aec0b0 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -85,43 +85,6 @@ enum ipi_msg_type {
IPI_WAKEUP
};
-#ifdef CONFIG_ARM64_VHE
-
-/* Whether the boot CPU is running in HYP mode or not*/
-static bool boot_cpu_hyp_mode;
-
-static inline void save_boot_cpu_run_el(void)
-{
- boot_cpu_hyp_mode = is_kernel_in_hyp_mode();
-}
-
-static inline bool is_boot_cpu_in_hyp_mode(void)
-{
- return boot_cpu_hyp_mode;
-}
-
-/*
- * Verify that a secondary CPU is running the kernel at the same
- * EL as that of the boot CPU.
- */
-void verify_cpu_run_el(void)
-{
- bool in_el2 = is_kernel_in_hyp_mode();
- bool boot_cpu_el2 = is_boot_cpu_in_hyp_mode();
-
- if (in_el2 ^ boot_cpu_el2) {
- pr_crit("CPU%d: mismatched Exception Level(EL%d) with boot CPU(EL%d)\n",
- smp_processor_id(),
- in_el2 ? 2 : 1,
- boot_cpu_el2 ? 2 : 1);
- cpu_panic_kernel();
- }
-}
-
-#else
-static inline void save_boot_cpu_run_el(void) {}
-#endif
-
#ifdef CONFIG_HOTPLUG_CPU
static int op_cpu_kill(unsigned int cpu);
#else
@@ -447,7 +410,6 @@ void __init smp_prepare_boot_cpu(void)
*/
jump_label_init();
cpuinfo_store_boot_cpu();
- save_boot_cpu_run_el();
}
static u64 __init of_get_cpu_mpidr(struct device_node *dn)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 16/22] arm64: capabilities: Clean up midr range helpers
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
We are about to introduce generic MIDR range helpers. Clean
up the existing helpers in erratum handling, preparing them
to use generic version.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpu_errata.c | 106 +++++++++++++++++++++++------------------
1 file changed, 59 insertions(+), 47 deletions(-)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index f5ab9545c5ea..9ea14954972c 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -237,23 +237,38 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
-#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = min, \
- .midr_range_max = max
+#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
-#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = 0, \
+#define CAP_MIDR_ALL_VERSIONS(model) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
#define MIDR_FIXED(rev, revidr_mask) \
.fixed_revs = (struct arm64_midr_revidr[]){{ (rev), (revidr_mask) }, {}}
+#define ERRATA_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+
+/* Errata affecting a range of revisions of given model variant */
+#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
+ ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
+
+/* Errata affecting a single variant/revision of a model */
+#define ERRATA_MIDR_REV(model, var, rev) \
+ ERRATA_MIDR_RANGE(model, var, rev, var, rev)
+
+/* Errata affecting all variants/revisions of a given a model */
+#define ERRATA_MIDR_ALL_VERSIONS(model) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_ALL_VERSIONS(model)
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -262,7 +277,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[012] */
.desc = "ARM errata 826319, 827319, 824069",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 2),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -271,7 +286,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01] */
.desc = "ARM errata 819472",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 1),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -280,9 +295,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 832075",
.capability = ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_834220
@@ -290,9 +305,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 834220",
.capability = ARM64_WORKAROUND_834220,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_843419
@@ -300,7 +315,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01234] */
.desc = "ARM erratum 843419",
.capability = ARM64_WORKAROUND_843419,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x04),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4),
MIDR_FIXED(0x4, BIT(8)),
},
#endif
@@ -309,7 +324,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01234] */
.desc = "ARM erratum 845719",
.capability = ARM64_WORKAROUND_845719,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x04),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_23154
@@ -317,7 +332,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, pass 1.x */
.desc = "Cavium erratum 23154",
.capability = ARM64_WORKAROUND_CAVIUM_23154,
- MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX, 0, 0, 1),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_27456
@@ -325,15 +340,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.1 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 1)),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 1),
},
{
/* Cavium ThunderX, T81 pass 1.0 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_81XX, 0, 0),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_30115
@@ -341,20 +356,21 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX, 0x00,
- (1 << MIDR_VARIANT_SHIFT) | 2),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 2),
},
{
/* Cavium ThunderX, T81 pass 1.0 - 1.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX_81XX, 0, 0, 2),
},
{
/* Cavium ThunderX, T83 pass 1.0 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_83XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_83XX, 0, 0),
},
#endif
{
@@ -368,9 +384,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
@@ -384,9 +398,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1009",
.capability = ARM64_WORKAROUND_REPEAT_TLBI,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_858921
@@ -394,56 +406,56 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A73 all versions */
.desc = "ARM erratum 858921",
.capability = ARM64_WORKAROUND_858921,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
},
#endif
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
.cpu_enable = enable_smccc_arch_workaround_1,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 16/22] arm64: capabilities: Clean up midr range helpers
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
We are about to introduce generic MIDR range helpers. Clean
up the existing helpers in erratum handling, preparing them
to use generic version.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/kernel/cpu_errata.c | 106 +++++++++++++++++++++++------------------
1 file changed, 59 insertions(+), 47 deletions(-)
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index f5ab9545c5ea..9ea14954972c 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -237,23 +237,38 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
}
#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */
-#define MIDR_RANGE(model, min, max) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = min, \
- .midr_range_max = max
+#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
-#define MIDR_ALL_VERSIONS(model) \
- .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
- .matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = 0, \
+#define CAP_MIDR_ALL_VERSIONS(model) \
+ .matches = is_affected_midr_range, \
+ .midr_model = model, \
+ .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
.midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
#define MIDR_FIXED(rev, revidr_mask) \
.fixed_revs = (struct arm64_midr_revidr[]){{ (rev), (revidr_mask) }, {}}
+#define ERRATA_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+
+/* Errata affecting a range of revisions of given model variant */
+#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
+ ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
+
+/* Errata affecting a single variant/revision of a model */
+#define ERRATA_MIDR_REV(model, var, rev) \
+ ERRATA_MIDR_RANGE(model, var, rev, var, rev)
+
+/* Errata affecting all variants/revisions of a given a model */
+#define ERRATA_MIDR_ALL_VERSIONS(model) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_ALL_VERSIONS(model)
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -262,7 +277,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[012] */
.desc = "ARM errata 826319, 827319, 824069",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 2),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -271,7 +286,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01] */
.desc = "ARM errata 819472",
.capability = ARM64_WORKAROUND_CLEAN_CACHE,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 1),
.cpu_enable = cpu_enable_cache_maint_trap,
},
#endif
@@ -280,9 +295,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 832075",
.capability = ARM64_WORKAROUND_DEVICE_LOAD_ACQUIRE,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_834220
@@ -290,9 +305,9 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A57 r0p0 - r1p2 */
.desc = "ARM erratum 834220",
.capability = ARM64_WORKAROUND_834220,
- MIDR_RANGE(MIDR_CORTEX_A57,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 2)),
+ ERRATA_MIDR_RANGE(MIDR_CORTEX_A57,
+ 0, 0,
+ 1, 2),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_843419
@@ -300,7 +315,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01234] */
.desc = "ARM erratum 843419",
.capability = ARM64_WORKAROUND_843419,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x04),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4),
MIDR_FIXED(0x4, BIT(8)),
},
#endif
@@ -309,7 +324,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A53 r0p[01234] */
.desc = "ARM erratum 845719",
.capability = ARM64_WORKAROUND_845719,
- MIDR_RANGE(MIDR_CORTEX_A53, 0x00, 0x04),
+ ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A53, 0, 0, 4),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_23154
@@ -317,7 +332,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, pass 1.x */
.desc = "Cavium erratum 23154",
.capability = ARM64_WORKAROUND_CAVIUM_23154,
- MIDR_RANGE(MIDR_THUNDERX, 0x00, 0x01),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX, 0, 0, 1),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_27456
@@ -325,15 +340,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.1 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(1, 1)),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 1),
},
{
/* Cavium ThunderX, T81 pass 1.0 */
.desc = "Cavium erratum 27456",
.capability = ARM64_WORKAROUND_CAVIUM_27456,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_81XX, 0, 0),
},
#endif
#ifdef CONFIG_CAVIUM_ERRATUM_30115
@@ -341,20 +356,21 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cavium ThunderX, T88 pass 1.x - 2.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX, 0x00,
- (1 << MIDR_VARIANT_SHIFT) | 2),
+ ERRATA_MIDR_RANGE(MIDR_THUNDERX,
+ 0, 0,
+ 1, 2),
},
{
/* Cavium ThunderX, T81 pass 1.0 - 1.2 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_81XX, 0x00, 0x02),
+ ERRATA_MIDR_REV_RANGE(MIDR_THUNDERX_81XX, 0, 0, 2),
},
{
/* Cavium ThunderX, T83 pass 1.0 */
.desc = "Cavium erratum 30115",
.capability = ARM64_WORKAROUND_CAVIUM_30115,
- MIDR_RANGE(MIDR_THUNDERX_83XX, 0x00, 0x00),
+ ERRATA_MIDR_REV(MIDR_THUNDERX_83XX, 0, 0),
},
#endif
{
@@ -368,9 +384,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
{
.desc = "Qualcomm Technologies Kryo erratum 1003",
@@ -384,9 +398,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
{
.desc = "Qualcomm Technologies Falkor erratum 1009",
.capability = ARM64_WORKAROUND_REPEAT_TLBI,
- MIDR_RANGE(MIDR_QCOM_FALKOR_V1,
- MIDR_CPU_VAR_REV(0, 0),
- MIDR_CPU_VAR_REV(0, 0)),
+ ERRATA_MIDR_REV(MIDR_QCOM_FALKOR_V1, 0, 0),
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_858921
@@ -394,56 +406,56 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
/* Cortex-A73 all versions */
.desc = "ARM erratum 858921",
.capability = ARM64_WORKAROUND_858921,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
},
#endif
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
.cpu_enable = enable_smccc_arch_workaround_1,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 17/22] arm64: Add helpers for checking CPU MIDR against a range
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Add helpers for checking if the given CPU midr falls in a range
of variants/revisions for a given model.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 4 ++--
arch/arm64/include/asm/cputype.h | 30 ++++++++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 16 +++++-----------
3 files changed, 37 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 6a1280493f57..cd245871b578 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -10,6 +10,7 @@
#define __ASM_CPUFEATURE_H
#include <asm/cpucaps.h>
+#include <asm/cputype.h>
#include <asm/fpsimd.h>
#include <asm/hwcap.h>
#include <asm/sigcontext.h>
@@ -306,8 +307,7 @@ struct arm64_cpu_capabilities {
void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
struct { /* To be used for erratum handling only */
- u32 midr_model;
- u32 midr_range_min, midr_range_max;
+ struct midr_range midr_range;
const struct arm64_midr_revidr {
u32 midr_rv; /* revision/variant */
u32 revidr_mask;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 350c76a1d15b..bf6cfdab743f 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -117,6 +117,36 @@
#define read_cpuid(reg) read_sysreg_s(SYS_ ## reg)
+/*
+ * Represent a range of MIDR values for a given CPU model and a
+ * range of variant/revision values.
+ *
+ * @model - CPU model as defined by MIDR_CPU_MODEL
+ * @rv_min - Minimum value for the revision/variant as defined by
+ * MIDR_CPU_VAR_REV
+ * @rv_max - Maximum value for the variant/revision for the range.
+ */
+struct midr_range {
+ u32 model;
+ u32 rv_min;
+ u32 rv_max;
+};
+
+#define MIDR_RANGE(m, v_min, r_min, v_max, r_max) \
+ { \
+ .model = m, \
+ .rv_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .rv_max = MIDR_CPU_VAR_REV(v_max, r_max), \
+ }
+
+#define MIDR_ALL_VERSIONS(m) MIDR_RANGE(m, 0, 0, 0xf, 0xf)
+
+static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
+{
+ return MIDR_IS_CPU_MODEL_RANGE(midr, range->model,
+ range->rv_min, range->rv_max);
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 9ea14954972c..3c0bb6c4ed02 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -28,9 +28,7 @@ is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
u32 midr = read_cpuid_id(), revidr;
WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
- if (!MIDR_IS_CPU_MODEL_RANGE(midr, entry->midr_model,
- entry->midr_range_min,
- entry->midr_range_max))
+ if (!is_midr_in_range(midr, &entry->midr_range))
return false;
midr &= MIDR_REVISION_MASK | MIDR_VARIANT_MASK;
@@ -53,7 +51,7 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
model &= MIDR_IMPLEMENTOR_MASK | (0xf00 << MIDR_PARTNUM_SHIFT) |
MIDR_ARCHITECTURE_MASK;
- return model == entry->midr_model;
+ return model == entry->midr_range.model;
}
static bool
@@ -239,15 +237,11 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
- .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
+ .midr_range = MIDR_RANGE(model, v_min, r_min, v_max, r_max)
#define CAP_MIDR_ALL_VERSIONS(model) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
- .midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+ .midr_range = MIDR_ALL_VERSIONS(model)
#define MIDR_FIXED(rev, revidr_mask) \
.fixed_revs = (struct arm64_midr_revidr[]){{ (rev), (revidr_mask) }, {}}
@@ -390,7 +384,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
- .midr_model = MIDR_QCOM_KRYO,
+ .midr_range.model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 17/22] arm64: Add helpers for checking CPU MIDR against a range
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Add helpers for checking if the given CPU midr falls in a range
of variants/revisions for a given model.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 4 ++--
arch/arm64/include/asm/cputype.h | 30 ++++++++++++++++++++++++++++++
arch/arm64/kernel/cpu_errata.c | 16 +++++-----------
3 files changed, 37 insertions(+), 13 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 6a1280493f57..cd245871b578 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -10,6 +10,7 @@
#define __ASM_CPUFEATURE_H
#include <asm/cpucaps.h>
+#include <asm/cputype.h>
#include <asm/fpsimd.h>
#include <asm/hwcap.h>
#include <asm/sigcontext.h>
@@ -306,8 +307,7 @@ struct arm64_cpu_capabilities {
void (*cpu_enable)(const struct arm64_cpu_capabilities *cap);
union {
struct { /* To be used for erratum handling only */
- u32 midr_model;
- u32 midr_range_min, midr_range_max;
+ struct midr_range midr_range;
const struct arm64_midr_revidr {
u32 midr_rv; /* revision/variant */
u32 revidr_mask;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index 350c76a1d15b..bf6cfdab743f 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -117,6 +117,36 @@
#define read_cpuid(reg) read_sysreg_s(SYS_ ## reg)
+/*
+ * Represent a range of MIDR values for a given CPU model and a
+ * range of variant/revision values.
+ *
+ * @model - CPU model as defined by MIDR_CPU_MODEL
+ * @rv_min - Minimum value for the revision/variant as defined by
+ * MIDR_CPU_VAR_REV
+ * @rv_max - Maximum value for the variant/revision for the range.
+ */
+struct midr_range {
+ u32 model;
+ u32 rv_min;
+ u32 rv_max;
+};
+
+#define MIDR_RANGE(m, v_min, r_min, v_max, r_max) \
+ { \
+ .model = m, \
+ .rv_min = MIDR_CPU_VAR_REV(v_min, r_min), \
+ .rv_max = MIDR_CPU_VAR_REV(v_max, r_max), \
+ }
+
+#define MIDR_ALL_VERSIONS(m) MIDR_RANGE(m, 0, 0, 0xf, 0xf)
+
+static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
+{
+ return MIDR_IS_CPU_MODEL_RANGE(midr, range->model,
+ range->rv_min, range->rv_max);
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 9ea14954972c..3c0bb6c4ed02 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -28,9 +28,7 @@ is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
u32 midr = read_cpuid_id(), revidr;
WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
- if (!MIDR_IS_CPU_MODEL_RANGE(midr, entry->midr_model,
- entry->midr_range_min,
- entry->midr_range_max))
+ if (!is_midr_in_range(midr, &entry->midr_range))
return false;
midr &= MIDR_REVISION_MASK | MIDR_VARIANT_MASK;
@@ -53,7 +51,7 @@ is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
model &= MIDR_IMPLEMENTOR_MASK | (0xf00 << MIDR_PARTNUM_SHIFT) |
MIDR_ARCHITECTURE_MASK;
- return model == entry->midr_model;
+ return model == entry->midr_range.model;
}
static bool
@@ -239,15 +237,11 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
#define CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(v_min, r_min), \
- .midr_range_max = MIDR_CPU_VAR_REV(v_max, r_max)
+ .midr_range = MIDR_RANGE(model, v_min, r_min, v_max, r_max)
#define CAP_MIDR_ALL_VERSIONS(model) \
.matches = is_affected_midr_range, \
- .midr_model = model, \
- .midr_range_min = MIDR_CPU_VAR_REV(0, 0), \
- .midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK)
+ .midr_range = MIDR_ALL_VERSIONS(model)
#define MIDR_FIXED(rev, revidr_mask) \
.fixed_revs = (struct arm64_midr_revidr[]){{ (rev), (revidr_mask) }, {}}
@@ -390,7 +384,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
.desc = "Qualcomm Technologies Kryo erratum 1003",
.capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003,
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
- .midr_model = MIDR_QCOM_KRYO,
+ .midr_range.model = MIDR_QCOM_KRYO,
.matches = is_kryo_midr,
},
#endif
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 18/22] arm64: capabilities: Add support for checks based on a list of MIDRs
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Add helpers for detecting an errata on list of midr ranges
of affected CPUs, with the same work around.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/include/asm/cputype.h | 9 +++++
arch/arm64/kernel/cpu_errata.c | 81 ++++++++++++++++++++-----------------
arch/arm64/kernel/cpufeature.c | 10 +++--
4 files changed, 60 insertions(+), 41 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index cd245871b578..a16eb0731290 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -314,6 +314,7 @@ struct arm64_cpu_capabilities {
} * const fixed_revs;
};
+ const struct midr_range *midr_range_list;
struct { /* Feature register checking */
u32 sys_reg;
u8 field_pos;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index bf6cfdab743f..e86e65c187f8 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -147,6 +147,15 @@ static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
range->rv_min, range->rv_max);
}
+static inline bool
+is_midr_in_range_list(u32 midr, struct midr_range const *ranges)
+{
+ while (ranges->model)
+ if (is_midr_in_range(midr, ranges++))
+ return true;
+ return false;
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 3c0bb6c4ed02..9490b560d3fe 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -40,6 +40,14 @@ is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
return true;
}
+static bool __maybe_unused
+is_affected_midr_range_list(const struct arm64_cpu_capabilities *entry,
+ int scope)
+{
+ WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
+ return is_midr_in_range_list(read_cpuid_id(), entry->midr_range_list);
+}
+
static bool __maybe_unused
is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
{
@@ -250,6 +258,10 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+#define CAP_MIDR_RANGE_LIST(list) \
+ .matches = is_affected_midr_range_list, \
+ .midr_range_list = list
+
/* Errata affecting a range of revisions of given model variant */
#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
@@ -263,6 +275,35 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_ALL_VERSIONS(model)
+/* Errata affecting a list of midr ranges, with same work around */
+#define ERRATA_MIDR_RANGE_LIST(midr_list) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE_LIST(midr_list)
+
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+
+/*
+ * List of CPUs where we need to issue a psci call to
+ * harden the branch predictor.
+ */
+static const struct midr_range arm64_bp_harden_smccc_cpus[] = {
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ {},
+};
+
+static const struct midr_range qcom_bp_harden_cpus[] = {
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ {},
+};
+
+#endif
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -406,51 +447,17 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_smccc_cpus),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
- .cpu_enable = qcom_enable_link_stack_sanitization,
- },
- {
- .capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
- .cpu_enable = enable_smccc_arch_workaround_1,
+ ERRATA_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
},
#endif
{
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 006e95d46a4a..fdf6fa5b2f29 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -879,6 +879,11 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
int scope)
{
+ /* List of CPUs that are not vulnerable and don't need KPTI */
+ static const struct midr_range kpti_safe_list[] = {
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ };
char const *str = "command line option";
/*
@@ -903,11 +908,8 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
return true;
/* Don't force KPTI for CPUs that are not vulnerable */
- switch (read_cpuid_id() & MIDR_CPU_MODEL_MASK) {
- case MIDR_CAVIUM_THUNDERX2:
- case MIDR_BRCM_VULCAN:
+ if (is_midr_in_range_list(read_cpuid_id(), kpti_safe_list))
return false;
- }
/* Defer to CPU feature registers */
return !has_cpuid_feature(entry, scope);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 18/22] arm64: capabilities: Add support for checks based on a list of MIDRs
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Add helpers for detecting an errata on list of midr ranges
of affected CPUs, with the same work around.
Cc: Will Deacon <will.deacon@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>
Cc: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 1 +
arch/arm64/include/asm/cputype.h | 9 +++++
arch/arm64/kernel/cpu_errata.c | 81 ++++++++++++++++++++-----------------
arch/arm64/kernel/cpufeature.c | 10 +++--
4 files changed, 60 insertions(+), 41 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index cd245871b578..a16eb0731290 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -314,6 +314,7 @@ struct arm64_cpu_capabilities {
} * const fixed_revs;
};
+ const struct midr_range *midr_range_list;
struct { /* Feature register checking */
u32 sys_reg;
u8 field_pos;
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index bf6cfdab743f..e86e65c187f8 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -147,6 +147,15 @@ static inline bool is_midr_in_range(u32 midr, struct midr_range const *range)
range->rv_min, range->rv_max);
}
+static inline bool
+is_midr_in_range_list(u32 midr, struct midr_range const *ranges)
+{
+ while (ranges->model)
+ if (is_midr_in_range(midr, ranges++))
+ return true;
+ return false;
+}
+
/*
* The CPU ID never changes at run time, so we might as well tell the
* compiler that it's constant. Use this function to read the CPU ID
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 3c0bb6c4ed02..9490b560d3fe 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -40,6 +40,14 @@ is_affected_midr_range(const struct arm64_cpu_capabilities *entry, int scope)
return true;
}
+static bool __maybe_unused
+is_affected_midr_range_list(const struct arm64_cpu_capabilities *entry,
+ int scope)
+{
+ WARN_ON(scope != SCOPE_LOCAL_CPU || preemptible());
+ return is_midr_in_range_list(read_cpuid_id(), entry->midr_range_list);
+}
+
static bool __maybe_unused
is_kryo_midr(const struct arm64_cpu_capabilities *entry, int scope)
{
@@ -250,6 +258,10 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE(model, v_min, r_min, v_max, r_max)
+#define CAP_MIDR_RANGE_LIST(list) \
+ .matches = is_affected_midr_range_list, \
+ .midr_range_list = list
+
/* Errata affecting a range of revisions of given model variant */
#define ERRATA_MIDR_REV_RANGE(m, var, r_min, r_max) \
ERRATA_MIDR_RANGE(m, var, r_min, var, r_max)
@@ -263,6 +275,35 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_ALL_VERSIONS(model)
+/* Errata affecting a list of midr ranges, with same work around */
+#define ERRATA_MIDR_RANGE_LIST(midr_list) \
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
+ CAP_MIDR_RANGE_LIST(midr_list)
+
+#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
+
+/*
+ * List of CPUs where we need to issue a psci call to
+ * harden the branch predictor.
+ */
+static const struct midr_range arm64_bp_harden_smccc_cpus[] = {
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
+ MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ {},
+};
+
+static const struct midr_range qcom_bp_harden_cpus[] = {
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
+ {},
+};
+
+#endif
+
const struct arm64_cpu_capabilities arm64_errata[] = {
#if defined(CONFIG_ARM64_ERRATUM_826319) || \
defined(CONFIG_ARM64_ERRATUM_827319) || \
@@ -406,51 +447,17 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A57),
+ ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_smccc_cpus),
.cpu_enable = enable_smccc_arch_workaround_1,
},
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A72),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A73),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CORTEX_A75),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
+ ERRATA_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
.cpu_enable = qcom_enable_link_stack_sanitization,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR_V1),
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
- .cpu_enable = qcom_enable_link_stack_sanitization,
- },
- {
- .capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_QCOM_FALKOR),
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
- .cpu_enable = enable_smccc_arch_workaround_1,
+ ERRATA_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
},
#endif
{
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 006e95d46a4a..fdf6fa5b2f29 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -879,6 +879,11 @@ static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
int scope)
{
+ /* List of CPUs that are not vulnerable and don't need KPTI */
+ static const struct midr_range kpti_safe_list[] = {
+ MIDR_ALL_VERSIONS(MIDR_CAVIUM_THUNDERX2),
+ MIDR_ALL_VERSIONS(MIDR_BRCM_VULCAN),
+ };
char const *str = "command line option";
/*
@@ -903,11 +908,8 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
return true;
/* Don't force KPTI for CPUs that are not vulnerable */
- switch (read_cpuid_id() & MIDR_CPU_MODEL_MASK) {
- case MIDR_CAVIUM_THUNDERX2:
- case MIDR_BRCM_VULCAN:
+ if (is_midr_in_range_list(read_cpuid_id(), kpti_safe_list))
return false;
- }
/* Defer to CPU feature registers */
return !has_cpuid_feature(entry, scope);
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 19/22] arm64: capabilities: Handle shared entries
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Some capabilities have different criteria for detection and associated
actions based on the matching criteria, even though they all share the
same capability bit. So far we have used multiple entries with the same
capability bit to handle this. This is prone to errors, as the
cpu_enable is invoked for each entry, irrespective of whether the
detection rule applies to the CPU or not. And also this complicates
other helpers, e.g, __this_cpu_has_cap.
This patch adds a wrapper entry to cover all the possible variations
of a capability by maintaining list of matches + cpu_enable callbacks.
To avoid complicating the prototypes for the "matches()", we use
arm64_cpu_capabilities maintain the list and we ignore all the other
fields except the matches & cpu_enable.
This ensures :
1) The capabilitiy is set when at least one of the entry detects
2) Action is only taken for the entries that "matches".
This avoids explicit checks in the cpu_enable() take some action.
The only constraint here is that, all the entries should have the
same "type" (i.e, scope and conflict rules).
If a cpu_enable() method is associated with multiple matches for a
single capability, care should be taken that either the match criteria
are mutually exclusive, or that the method is robust against being
called multiple times.
This also reverts the changes introduced by commit 67948af41f2e6818ed
("arm64: capabilities: Handle duplicate entries for a capability").
Cc: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 12 ++++++++
arch/arm64/kernel/cpu_errata.c | 55 ++++++++++++++++++++++++++++++++-----
arch/arm64/kernel/cpufeature.c | 13 ++++-----
3 files changed, 66 insertions(+), 14 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index a16eb0731290..09b0f2a80c8f 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -323,6 +323,18 @@ struct arm64_cpu_capabilities {
bool sign;
unsigned long hwcap;
};
+ /*
+ * A list of "matches/cpu_enable" pair for the same
+ * "capability" of the same "type" as described by the parent.
+ * Only matches(), cpu_enable() and fields relevant to these
+ * methods are significant in the list. The cpu_enable is
+ * invoked only if the corresponding entry "matches()".
+ * However, if a cpu_enable() method is associated
+ * with multiple matches(), care should be taken that either
+ * the match criteria are mutually exclusive, or that the
+ * method is robust against being called multiple times.
+ */
+ const struct arm64_cpu_capabilities *match_list;
};
};
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 9490b560d3fe..6de823a1be10 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -280,6 +280,38 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE_LIST(midr_list)
+/*
+ * Generic helper for handling capabilties with multiple (match,enable) pairs
+ * of call backs, sharing the same capability bit.
+ * Iterate over each entry to see if at least one matches.
+ */
+static bool multi_entry_cap_matches(const struct arm64_cpu_capabilities *entry,
+ int scope)
+{
+ const struct arm64_cpu_capabilities *caps;
+
+ for (caps = entry->match_list; caps->matches; caps++)
+ if (caps->matches(caps, scope))
+ return true;
+
+ return false;
+}
+
+/*
+ * Take appropriate action for all matching entries in the shared capability
+ * entry.
+ */
+static void
+multi_entry_cap_cpu_enable(const struct arm64_cpu_capabilities *entry)
+{
+ const struct arm64_cpu_capabilities *caps;
+
+ for (caps = entry->match_list; caps->matches; caps++)
+ if (caps->matches(caps, SCOPE_LOCAL_CPU) &&
+ caps->cpu_enable)
+ caps->cpu_enable(caps);
+}
+
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
/*
@@ -302,6 +334,18 @@ static const struct midr_range qcom_bp_harden_cpus[] = {
{},
};
+static const struct arm64_cpu_capabilities arm64_bp_harden_list[] = {
+ {
+ CAP_MIDR_RANGE_LIST(arm64_bp_harden_smccc_cpus),
+ .cpu_enable = enable_smccc_arch_workaround_1,
+ },
+ {
+ CAP_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
+ .cpu_enable = qcom_enable_link_stack_sanitization,
+ },
+ {},
+};
+
#endif
const struct arm64_cpu_capabilities arm64_errata[] = {
@@ -447,13 +491,10 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_smccc_cpus),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
- .cpu_enable = qcom_enable_link_stack_sanitization,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
+ .matches = multi_entry_cap_matches,
+ .cpu_enable = multi_entry_cap_cpu_enable,
+ .match_list = arm64_bp_harden_list,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index fdf6fa5b2f29..53273e79f7ce 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1252,9 +1252,9 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
for (caps = cap_array; caps->matches; caps++)
- if (caps->capability == cap &&
- caps->matches(caps, SCOPE_LOCAL_CPU))
- return true;
+ if (caps->capability == cap)
+ return caps->matches(caps, SCOPE_LOCAL_CPU);
+
return false;
}
@@ -1344,19 +1344,18 @@ static void __init enable_cpu_capabilities(u16 scope_mask)
* Returns "false" on conflicts.
*/
static bool
-__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps,
u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
- const struct arm64_cpu_capabilities *caps;
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
- for (caps = caps_list; caps->matches; caps++) {
+ for (; caps->matches; caps++) {
if (!(caps->type & scope_mask))
continue;
- cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ cpu_has_cap = caps->matches(caps, SCOPE_LOCAL_CPU);
system_has_cap = cpus_have_cap(caps->capability);
if (system_has_cap) {
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 19/22] arm64: capabilities: Handle shared entries
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Some capabilities have different criteria for detection and associated
actions based on the matching criteria, even though they all share the
same capability bit. So far we have used multiple entries with the same
capability bit to handle this. This is prone to errors, as the
cpu_enable is invoked for each entry, irrespective of whether the
detection rule applies to the CPU or not. And also this complicates
other helpers, e.g, __this_cpu_has_cap.
This patch adds a wrapper entry to cover all the possible variations
of a capability by maintaining list of matches + cpu_enable callbacks.
To avoid complicating the prototypes for the "matches()", we use
arm64_cpu_capabilities maintain the list and we ignore all the other
fields except the matches & cpu_enable.
This ensures :
1) The capabilitiy is set when at least one of the entry detects
2) Action is only taken for the entries that "matches".
This avoids explicit checks in the cpu_enable() take some action.
The only constraint here is that, all the entries should have the
same "type" (i.e, scope and conflict rules).
If a cpu_enable() method is associated with multiple matches for a
single capability, care should be taken that either the match criteria
are mutually exclusive, or that the method is robust against being
called multiple times.
This also reverts the changes introduced by commit 67948af41f2e6818ed
("arm64: capabilities: Handle duplicate entries for a capability").
Cc: Robin Murphy <robin.murphy@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpufeature.h | 12 ++++++++
arch/arm64/kernel/cpu_errata.c | 55 ++++++++++++++++++++++++++++++++-----
arch/arm64/kernel/cpufeature.c | 13 ++++-----
3 files changed, 66 insertions(+), 14 deletions(-)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index a16eb0731290..09b0f2a80c8f 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -323,6 +323,18 @@ struct arm64_cpu_capabilities {
bool sign;
unsigned long hwcap;
};
+ /*
+ * A list of "matches/cpu_enable" pair for the same
+ * "capability" of the same "type" as described by the parent.
+ * Only matches(), cpu_enable() and fields relevant to these
+ * methods are significant in the list. The cpu_enable is
+ * invoked only if the corresponding entry "matches()".
+ * However, if a cpu_enable() method is associated
+ * with multiple matches(), care should be taken that either
+ * the match criteria are mutually exclusive, or that the
+ * method is robust against being called multiple times.
+ */
+ const struct arm64_cpu_capabilities *match_list;
};
};
diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c
index 9490b560d3fe..6de823a1be10 100644
--- a/arch/arm64/kernel/cpu_errata.c
+++ b/arch/arm64/kernel/cpu_errata.c
@@ -280,6 +280,38 @@ qcom_enable_link_stack_sanitization(const struct arm64_cpu_capabilities *entry)
.type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, \
CAP_MIDR_RANGE_LIST(midr_list)
+/*
+ * Generic helper for handling capabilties with multiple (match,enable) pairs
+ * of call backs, sharing the same capability bit.
+ * Iterate over each entry to see if at least one matches.
+ */
+static bool multi_entry_cap_matches(const struct arm64_cpu_capabilities *entry,
+ int scope)
+{
+ const struct arm64_cpu_capabilities *caps;
+
+ for (caps = entry->match_list; caps->matches; caps++)
+ if (caps->matches(caps, scope))
+ return true;
+
+ return false;
+}
+
+/*
+ * Take appropriate action for all matching entries in the shared capability
+ * entry.
+ */
+static void
+multi_entry_cap_cpu_enable(const struct arm64_cpu_capabilities *entry)
+{
+ const struct arm64_cpu_capabilities *caps;
+
+ for (caps = entry->match_list; caps->matches; caps++)
+ if (caps->matches(caps, SCOPE_LOCAL_CPU) &&
+ caps->cpu_enable)
+ caps->cpu_enable(caps);
+}
+
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
/*
@@ -302,6 +334,18 @@ static const struct midr_range qcom_bp_harden_cpus[] = {
{},
};
+static const struct arm64_cpu_capabilities arm64_bp_harden_list[] = {
+ {
+ CAP_MIDR_RANGE_LIST(arm64_bp_harden_smccc_cpus),
+ .cpu_enable = enable_smccc_arch_workaround_1,
+ },
+ {
+ CAP_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
+ .cpu_enable = qcom_enable_link_stack_sanitization,
+ },
+ {},
+};
+
#endif
const struct arm64_cpu_capabilities arm64_errata[] = {
@@ -447,13 +491,10 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR
{
.capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_RANGE_LIST(arm64_bp_harden_smccc_cpus),
- .cpu_enable = enable_smccc_arch_workaround_1,
- },
- {
- .capability = ARM64_HARDEN_BRANCH_PREDICTOR,
- ERRATA_MIDR_RANGE_LIST(qcom_bp_harden_cpus),
- .cpu_enable = qcom_enable_link_stack_sanitization,
+ .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM,
+ .matches = multi_entry_cap_matches,
+ .cpu_enable = multi_entry_cap_cpu_enable,
+ .match_list = arm64_bp_harden_list,
},
{
.capability = ARM64_HARDEN_BP_POST_GUEST_EXIT,
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index fdf6fa5b2f29..53273e79f7ce 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1252,9 +1252,9 @@ static bool __this_cpu_has_cap(const struct arm64_cpu_capabilities *cap_array,
return false;
for (caps = cap_array; caps->matches; caps++)
- if (caps->capability == cap &&
- caps->matches(caps, SCOPE_LOCAL_CPU))
- return true;
+ if (caps->capability == cap)
+ return caps->matches(caps, SCOPE_LOCAL_CPU);
+
return false;
}
@@ -1344,19 +1344,18 @@ static void __init enable_cpu_capabilities(u16 scope_mask)
* Returns "false" on conflicts.
*/
static bool
-__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps_list,
+__verify_local_cpu_caps(const struct arm64_cpu_capabilities *caps,
u16 scope_mask)
{
bool cpu_has_cap, system_has_cap;
- const struct arm64_cpu_capabilities *caps;
scope_mask &= ARM64_CPUCAP_SCOPE_MASK;
- for (caps = caps_list; caps->matches; caps++) {
+ for (; caps->matches; caps++) {
if (!(caps->type & scope_mask))
continue;
- cpu_has_cap = __this_cpu_has_cap(caps_list, caps->capability);
+ cpu_has_cap = caps->matches(caps, SCOPE_LOCAL_CPU);
system_has_cap = cpus_have_cap(caps->capability);
if (system_has_cap) {
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 20/22] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Update the MIDR encodings for the Cortex-A55 and Cortex-A35
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cputype.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index e86e65c187f8..30014a9f8f2b 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -83,6 +83,8 @@
#define ARM_CPU_PART_CORTEX_A53 0xD03
#define ARM_CPU_PART_CORTEX_A73 0xD09
#define ARM_CPU_PART_CORTEX_A75 0xD0A
+#define ARM_CPU_PART_CORTEX_A35 0xD04
+#define ARM_CPU_PART_CORTEX_A55 0xD05
#define APM_CPU_PART_POTENZA 0x000
@@ -102,6 +104,8 @@
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
+#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 20/22] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Update the MIDR encodings for the Cortex-A55 and Cortex-A35
Cc: Mark Rutland <mark.rutland@arm.com>
Reviewed-by: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cputype.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/arm64/include/asm/cputype.h b/arch/arm64/include/asm/cputype.h
index e86e65c187f8..30014a9f8f2b 100644
--- a/arch/arm64/include/asm/cputype.h
+++ b/arch/arm64/include/asm/cputype.h
@@ -83,6 +83,8 @@
#define ARM_CPU_PART_CORTEX_A53 0xD03
#define ARM_CPU_PART_CORTEX_A73 0xD09
#define ARM_CPU_PART_CORTEX_A75 0xD0A
+#define ARM_CPU_PART_CORTEX_A35 0xD04
+#define ARM_CPU_PART_CORTEX_A55 0xD05
#define APM_CPU_PART_POTENZA 0x000
@@ -102,6 +104,8 @@
#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72)
#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73)
#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75)
+#define MIDR_CORTEX_A35 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A35)
+#define MIDR_CORTEX_A55 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A55)
#define MIDR_THUNDERX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX)
#define MIDR_THUNDERX_81XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_81XX)
#define MIDR_THUNDERX_83XX MIDR_CPU_MODEL(ARM_CPU_IMP_CAVIUM, CAVIUM_CPU_PART_THUNDERX_83XX)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 21/22] arm64: Delay enabling hardware DBM feature
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
We enable hardware DBM bit in a capable CPU, very early in the
boot via __cpu_setup. This doesn't give us a flexibility of
optionally disable the feature, as the clearing the bit
is a bit costly as the TLB can cache the settings. Instead,
we delay enabling the feature until the CPU is brought up
into the kernel. We use the feature capability mechanism
to handle it.
The hardware DBM is a non-conflicting feature. i.e, the kernel
can safely run with a mix of CPUs with some using the feature
and the others don't. So, it is safe for a late CPU to have
this capability and enable it, even if the active CPUs don't.
To get this handled properly by the infrastructure, we
unconditionally set the capability and only enable it
on CPUs which really have the feature. Also, we print the
feature detection from the "matches" call back to make sure
we don't mislead the user when none of the CPUs could use the
feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/kernel/cpufeature.c | 71 ++++++++++++++++++++++++++++++++++++++++
arch/arm64/mm/proc.S | 13 ++++----
3 files changed, 79 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index ff9fb3aba17b..21bb624e0a7a 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -48,7 +48,8 @@
#define ARM64_WORKAROUND_843419 27
#define ARM64_HAS_CACHE_IDC 28
#define ARM64_HAS_CACHE_DIC 29
+#define ARM64_HW_DBM 30
-#define ARM64_NCAPS 30
+#define ARM64_NCAPS 31
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 53273e79f7ce..4e73c610a409 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -954,6 +954,57 @@ static int __init parse_kpti(char *str)
__setup("kpti=", parse_kpti);
#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
+#ifdef CONFIG_ARM64_HW_AFDBM
+static inline void __cpu_enable_hw_dbm(void)
+{
+ u64 tcr = read_sysreg(tcr_el1) | TCR_HD;
+
+ write_sysreg(tcr, tcr_el1);
+ isb();
+}
+
+static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
+{
+ return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
+}
+
+static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
+{
+ if (cpu_can_use_dbm(cap))
+ __cpu_enable_hw_dbm();
+}
+
+static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap,
+ int __unused)
+{
+ static bool detected = false;
+ /*
+ * DBM is a non-conflicting feature. i.e, the kernel can safely
+ * run a mix of CPUs with and without the feature. So, we
+ * unconditionally enable the capability to allow any late CPU
+ * to use the feature. We only enable the control bits on the
+ * CPU, if it actually supports.
+ *
+ * We have to make sure we print the "feature" detection only
+ * when at least one CPU actually uses it. So check if this CPU
+ * can actually use it and print the message exactly once.
+ *
+ * This is safe as all CPUs (including secondary CPUs - due to the
+ * LOCAL_CPU scope - and the hotplugged CPUs - via verification)
+ * goes through the "matches" check exactly once. Also if a CPU
+ * matches the criteria, it is guaranteed that the CPU will turn
+ * the DBM on, as the capability is unconditionally enabled.
+ */
+ if (!detected && cpu_can_use_dbm(cap)) {
+ detected = true;
+ pr_info("detected: Hardware dirty bit management\n");
+ }
+
+ return true;
+}
+
+#endif
+
static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
{
/*
@@ -1130,6 +1181,26 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cache_dic,
},
+#ifdef CONFIG_ARM64_HW_AFDBM
+ {
+ /*
+ * Since we turn this on always, we don't want the user to
+ * think that the feature is available when it may not be.
+ * So hide the description.
+ *
+ * .desc = "Hardware pagetable Dirty Bit Management",
+ *
+ */
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
+ .capability = ARM64_HW_DBM,
+ .sys_reg = SYS_ID_AA64MMFR1_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR1_HADBS_SHIFT,
+ .min_field_value = 2,
+ .matches = has_hw_dbm,
+ .cpu_enable = cpu_enable_hw_dbm,
+ },
+#endif
{},
};
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 8f074d64b760..5f9a73a4452c 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -448,16 +448,15 @@ ENTRY(__cpu_setup)
tcr_compute_pa_size x10, #TCR_IPS_SHIFT, x5, x6
#ifdef CONFIG_ARM64_HW_AFDBM
/*
- * Hardware update of the Access and Dirty bits.
+ * Enable hardware update of the Access Flags bit.
+ * Hardware dirty bit management is enabled later,
+ * via capabilities.
*/
mrs x9, ID_AA64MMFR1_EL1
and x9, x9, #0xf
- cbz x9, 2f
- cmp x9, #2
- b.lt 1f
- orr x10, x10, #TCR_HD // hardware Dirty flag update
-1: orr x10, x10, #TCR_HA // hardware Access flag update
-2:
+ cbz x9, 1f
+ orr x10, x10, #TCR_HA // hardware Access flag update
+1:
#endif /* CONFIG_ARM64_HW_AFDBM */
msr tcr_el1, x10
ret // return to head.S
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 21/22] arm64: Delay enabling hardware DBM feature
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
We enable hardware DBM bit in a capable CPU, very early in the
boot via __cpu_setup. This doesn't give us a flexibility of
optionally disable the feature, as the clearing the bit
is a bit costly as the TLB can cache the settings. Instead,
we delay enabling the feature until the CPU is brought up
into the kernel. We use the feature capability mechanism
to handle it.
The hardware DBM is a non-conflicting feature. i.e, the kernel
can safely run with a mix of CPUs with some using the feature
and the others don't. So, it is safe for a late CPU to have
this capability and enable it, even if the active CPUs don't.
To get this handled properly by the infrastructure, we
unconditionally set the capability and only enable it
on CPUs which really have the feature. Also, we print the
feature detection from the "matches" call back to make sure
we don't mislead the user when none of the CPUs could use the
feature.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
arch/arm64/include/asm/cpucaps.h | 3 +-
arch/arm64/kernel/cpufeature.c | 71 ++++++++++++++++++++++++++++++++++++++++
arch/arm64/mm/proc.S | 13 ++++----
3 files changed, 79 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index ff9fb3aba17b..21bb624e0a7a 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -48,7 +48,8 @@
#define ARM64_WORKAROUND_843419 27
#define ARM64_HAS_CACHE_IDC 28
#define ARM64_HAS_CACHE_DIC 29
+#define ARM64_HW_DBM 30
-#define ARM64_NCAPS 30
+#define ARM64_NCAPS 31
#endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 53273e79f7ce..4e73c610a409 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -954,6 +954,57 @@ static int __init parse_kpti(char *str)
__setup("kpti=", parse_kpti);
#endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */
+#ifdef CONFIG_ARM64_HW_AFDBM
+static inline void __cpu_enable_hw_dbm(void)
+{
+ u64 tcr = read_sysreg(tcr_el1) | TCR_HD;
+
+ write_sysreg(tcr, tcr_el1);
+ isb();
+}
+
+static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
+{
+ return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
+}
+
+static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
+{
+ if (cpu_can_use_dbm(cap))
+ __cpu_enable_hw_dbm();
+}
+
+static bool has_hw_dbm(const struct arm64_cpu_capabilities *cap,
+ int __unused)
+{
+ static bool detected = false;
+ /*
+ * DBM is a non-conflicting feature. i.e, the kernel can safely
+ * run a mix of CPUs with and without the feature. So, we
+ * unconditionally enable the capability to allow any late CPU
+ * to use the feature. We only enable the control bits on the
+ * CPU, if it actually supports.
+ *
+ * We have to make sure we print the "feature" detection only
+ * when at least one CPU actually uses it. So check if this CPU
+ * can actually use it and print the message exactly once.
+ *
+ * This is safe as all CPUs (including secondary CPUs - due to the
+ * LOCAL_CPU scope - and the hotplugged CPUs - via verification)
+ * goes through the "matches" check exactly once. Also if a CPU
+ * matches the criteria, it is guaranteed that the CPU will turn
+ * the DBM on, as the capability is unconditionally enabled.
+ */
+ if (!detected && cpu_can_use_dbm(cap)) {
+ detected = true;
+ pr_info("detected: Hardware dirty bit management\n");
+ }
+
+ return true;
+}
+
+#endif
+
static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
{
/*
@@ -1130,6 +1181,26 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.type = ARM64_CPUCAP_SYSTEM_FEATURE,
.matches = has_cache_dic,
},
+#ifdef CONFIG_ARM64_HW_AFDBM
+ {
+ /*
+ * Since we turn this on always, we don't want the user to
+ * think that the feature is available when it may not be.
+ * So hide the description.
+ *
+ * .desc = "Hardware pagetable Dirty Bit Management",
+ *
+ */
+ .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
+ .capability = ARM64_HW_DBM,
+ .sys_reg = SYS_ID_AA64MMFR1_EL1,
+ .sign = FTR_UNSIGNED,
+ .field_pos = ID_AA64MMFR1_HADBS_SHIFT,
+ .min_field_value = 2,
+ .matches = has_hw_dbm,
+ .cpu_enable = cpu_enable_hw_dbm,
+ },
+#endif
{},
};
diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S
index 8f074d64b760..5f9a73a4452c 100644
--- a/arch/arm64/mm/proc.S
+++ b/arch/arm64/mm/proc.S
@@ -448,16 +448,15 @@ ENTRY(__cpu_setup)
tcr_compute_pa_size x10, #TCR_IPS_SHIFT, x5, x6
#ifdef CONFIG_ARM64_HW_AFDBM
/*
- * Hardware update of the Access and Dirty bits.
+ * Enable hardware update of the Access Flags bit.
+ * Hardware dirty bit management is enabled later,
+ * via capabilities.
*/
mrs x9, ID_AA64MMFR1_EL1
and x9, x9, #0xf
- cbz x9, 2f
- cmp x9, #2
- b.lt 1f
- orr x10, x10, #TCR_HD // hardware Dirty flag update
-1: orr x10, x10, #TCR_HA // hardware Access flag update
-2:
+ cbz x9, 1f
+ orr x10, x10, #TCR_HA // hardware Access flag update
+1:
#endif /* CONFIG_ARM64_HW_AFDBM */
msr tcr_el1, x10
ret // return to head.S
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-13 11:51 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd, Suzuki K Poulose
Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
from an erratum 1024718, which causes incorrect updates when DBM/AP
bits in a page table entry is modified without a break-before-make
sequence. The work around is to skip enabling the hardware DBM feature
on the affected cores. The hardware Access Flag management features
is not affected. There are some other cores suffering from this
errata, which could be added to the midr_list to trigger the work
around.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: ckadabi@codeaurora.org
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Documentation/arm64/silicon-errata.txt | 1 +
arch/arm64/Kconfig | 14 ++++++++++++++
arch/arm64/kernel/cpufeature.c | 16 +++++++++++++++-
3 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index c1d520de6dfe..3b2f2dd82225 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -55,6 +55,7 @@ stable kernels.
| ARM | Cortex-A57 | #834220 | ARM64_ERRATUM_834220 |
| ARM | Cortex-A72 | #853709 | N/A |
| ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 |
+| ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 |
| ARM | MMU-500 | #841119,#826419 | N/A |
| | | | |
| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 0f350b80aa84..9331baf8135c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -465,6 +465,20 @@ config ARM64_ERRATUM_843419
If unsure, say Y.
+config ARM64_ERRATUM_1024718
+ bool "Cortex-A55: 1024718: Update of DBM/AP bits without break before make might result in incorrect update"
+ default y
+ help
+ This option adds work around for Arm Cortex-A55 Erratum 1024718.
+
+ Affected Cortex-A55 cores (r0p0, r0p1, r1p0) could cause incorrect
+ update of the hardware dirty bit when the DBM/AP bits are updated
+ without a break-before-make. The work around is to disable the usage
+ of hardware DBM locally on the affected cores. CPUs not affected by
+ erratum will continue to use the feature.
+
+ If unsure, say Y.
+
config CAVIUM_ERRATUM_22375
bool "Cavium erratum 22375, 24313"
default y
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4e73c610a409..0fc097e2b72e 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -963,9 +963,23 @@ static inline void __cpu_enable_hw_dbm(void)
isb();
}
+static bool cpu_has_broken_dbm(void)
+{
+ /* List of CPUs which have broken DBM support. */
+ static const struct midr_range cpus[] = {
+#ifdef CONFIG_ARM64_ERRATUM_1024718
+ MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
+#endif
+ {},
+ };
+
+ return is_midr_in_range_list(read_cpuid_id(), cpus);
+}
+
static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
{
- return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
+ return has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
+ !cpu_has_broken_dbm();
}
static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* [PATCH v4 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
@ 2018-03-13 11:51 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-13 11:51 UTC (permalink / raw)
To: linux-arm-kernel
Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
from an erratum 1024718, which causes incorrect updates when DBM/AP
bits in a page table entry is modified without a break-before-make
sequence. The work around is to skip enabling the hardware DBM feature
on the affected cores. The hardware Access Flag management features
is not affected. There are some other cores suffering from this
errata, which could be added to the midr_list to trigger the work
around.
Cc: Catalin Marinas <catalin.marinas@arm.com>
Cc: ckadabi at codeaurora.org
Cc: Dave Martin <dave.martin@arm.com>
Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
---
Documentation/arm64/silicon-errata.txt | 1 +
arch/arm64/Kconfig | 14 ++++++++++++++
arch/arm64/kernel/cpufeature.c | 16 +++++++++++++++-
3 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt
index c1d520de6dfe..3b2f2dd82225 100644
--- a/Documentation/arm64/silicon-errata.txt
+++ b/Documentation/arm64/silicon-errata.txt
@@ -55,6 +55,7 @@ stable kernels.
| ARM | Cortex-A57 | #834220 | ARM64_ERRATUM_834220 |
| ARM | Cortex-A72 | #853709 | N/A |
| ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 |
+| ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 |
| ARM | MMU-500 | #841119,#826419 | N/A |
| | | | |
| Cavium | ThunderX ITS | #22375, #24313 | CAVIUM_ERRATUM_22375 |
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 0f350b80aa84..9331baf8135c 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -465,6 +465,20 @@ config ARM64_ERRATUM_843419
If unsure, say Y.
+config ARM64_ERRATUM_1024718
+ bool "Cortex-A55: 1024718: Update of DBM/AP bits without break before make might result in incorrect update"
+ default y
+ help
+ This option adds work around for Arm Cortex-A55 Erratum 1024718.
+
+ Affected Cortex-A55 cores (r0p0, r0p1, r1p0) could cause incorrect
+ update of the hardware dirty bit when the DBM/AP bits are updated
+ without a break-before-make. The work around is to disable the usage
+ of hardware DBM locally on the affected cores. CPUs not affected by
+ erratum will continue to use the feature.
+
+ If unsure, say Y.
+
config CAVIUM_ERRATUM_22375
bool "Cavium erratum 22375, 24313"
default y
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4e73c610a409..0fc097e2b72e 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -963,9 +963,23 @@ static inline void __cpu_enable_hw_dbm(void)
isb();
}
+static bool cpu_has_broken_dbm(void)
+{
+ /* List of CPUs which have broken DBM support. */
+ static const struct midr_range cpus[] = {
+#ifdef CONFIG_ARM64_ERRATUM_1024718
+ MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
+#endif
+ {},
+ };
+
+ return is_midr_in_range_list(read_cpuid_id(), cpus);
+}
+
static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
{
- return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
+ return has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
+ !cpu_has_broken_dbm();
}
static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap)
--
2.14.3
^ permalink raw reply related [flat|nested] 62+ messages in thread
* Re: [PATCH v4 11/22] arm64: capabilities: Group handling of features and errata workarounds
2018-03-13 11:51 ` Suzuki K Poulose
@ 2018-03-21 15:07 ` Dave Martin
-1 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:07 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: linux-arm-kernel, mark.rutland, ckadabi, ard.biesheuvel,
marc.zyngier, catalin.marinas, will.deacon, linux-kernel, jnair,
robin.murphy, shankerd
On Tue, Mar 13, 2018 at 11:51:09AM +0000, Suzuki K Poulose wrote:
> Now that the features and errata workarounds have the same
> rules and flow, group the handling of the tables.
>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
It looks like nothing signifcant has changed since v3 [1], and I was
reasonably happy with this last time around [2], so
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
[...]
Cheers
---Dave
[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2018-February/558742.html
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2018-February/558763.html
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 11/22] arm64: capabilities: Group handling of features and errata workarounds
@ 2018-03-21 15:07 ` Dave Martin
0 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:07 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Mar 13, 2018 at 11:51:09AM +0000, Suzuki K Poulose wrote:
> Now that the features and errata workarounds have the same
> rules and flow, group the handling of the tables.
>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
It looks like nothing signifcant has changed since v3 [1], and I was
reasonably happy with this last time around [2], so
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
[...]
Cheers
---Dave
[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2018-February/558742.html
[2] http://lists.infradead.org/pipermail/linux-arm-kernel/2018-February/558763.html
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v4 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
2018-03-13 11:51 ` Suzuki K Poulose
@ 2018-03-21 15:18 ` Dave Martin
-1 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:18 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: linux-arm-kernel, mark.rutland, ckadabi, ard.biesheuvel,
marc.zyngier, catalin.marinas, will.deacon, linux-kernel, jnair,
robin.murphy, shankerd
On Tue, Mar 13, 2018 at 11:51:13AM +0000, Suzuki K Poulose wrote:
> We expect all CPUs to be running at the same EL inside the kernel
> with or without VHE enabled and we have strict checks to ensure
> that any mismatch triggers a kernel panic. If VHE is enabled,
> we use the feature based on the boot CPU and all other CPUs
> should follow. This makes it a perfect candidate for a cpability
Nit: capability (not worth fixing unless you respin for some other
reason though.)
> based on the boot CPU, which should be matched by all the CPUs
> (both when is ON and OFF). This saves us some not-so-pretty
> hooks and special code, just for verifying the conflict.
>
> The patch also makes the VHE capability entry depend on
> CONFIG_ARM64_VHE.
>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Dave Martin <dave.martin@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature
@ 2018-03-21 15:18 ` Dave Martin
0 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:18 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Mar 13, 2018 at 11:51:13AM +0000, Suzuki K Poulose wrote:
> We expect all CPUs to be running at the same EL inside the kernel
> with or without VHE enabled and we have strict checks to ensure
> that any mismatch triggers a kernel panic. If VHE is enabled,
> we use the feature based on the boot CPU and all other CPUs
> should follow. This makes it a perfect candidate for a cpability
Nit: capability (not worth fixing unless you respin for some other
reason though.)
> based on the boot CPU, which should be matched by all the CPUs
> (both when is ON and OFF). This saves us some not-so-pretty
> hooks and special code, just for verifying the conflict.
>
> The patch also makes the VHE capability entry depend on
> CONFIG_ARM64_VHE.
>
> Cc: Marc Zyngier <marc.zyngier@arm.com>
> Cc: Dave Martin <dave.martin@arm.com>
> Cc: Will Deacon <will.deacon@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
[...]
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v4 21/22] arm64: Delay enabling hardware DBM feature
2018-03-13 11:51 ` Suzuki K Poulose
@ 2018-03-21 15:22 ` Dave Martin
-1 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:22 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: linux-arm-kernel, mark.rutland, ckadabi, ard.biesheuvel,
marc.zyngier, catalin.marinas, will.deacon, linux-kernel, jnair,
robin.murphy, shankerd
On Tue, Mar 13, 2018 at 11:51:19AM +0000, Suzuki K Poulose wrote:
> We enable hardware DBM bit in a capable CPU, very early in the
> boot via __cpu_setup. This doesn't give us a flexibility of
> optionally disable the feature, as the clearing the bit
> is a bit costly as the TLB can cache the settings. Instead,
> we delay enabling the feature until the CPU is brought up
> into the kernel. We use the feature capability mechanism
> to handle it.
>
> The hardware DBM is a non-conflicting feature. i.e, the kernel
> can safely run with a mix of CPUs with some using the feature
> and the others don't. So, it is safe for a late CPU to have
> this capability and enable it, even if the active CPUs don't.
>
> To get this handled properly by the infrastructure, we
> unconditionally set the capability and only enable it
> on CPUs which really have the feature. Also, we print the
> feature detection from the "matches" call back to make sure
> we don't mislead the user when none of the CPUs could use the
> feature.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
[...]
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 21/22] arm64: Delay enabling hardware DBM feature
@ 2018-03-21 15:22 ` Dave Martin
0 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:22 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Mar 13, 2018 at 11:51:19AM +0000, Suzuki K Poulose wrote:
> We enable hardware DBM bit in a capable CPU, very early in the
> boot via __cpu_setup. This doesn't give us a flexibility of
> optionally disable the feature, as the clearing the bit
> is a bit costly as the TLB can cache the settings. Instead,
> we delay enabling the feature until the CPU is brought up
> into the kernel. We use the feature capability mechanism
> to handle it.
>
> The hardware DBM is a non-conflicting feature. i.e, the kernel
> can safely run with a mix of CPUs with some using the feature
> and the others don't. So, it is safe for a late CPU to have
> this capability and enable it, even if the active CPUs don't.
>
> To get this handled properly by the infrastructure, we
> unconditionally set the capability and only enable it
> on CPUs which really have the feature. Also, we print the
> feature detection from the "matches" call back to make sure
> we don't mislead the user when none of the CPUs could use the
> feature.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
[...]
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v4 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
2018-03-13 11:51 ` Suzuki K Poulose
@ 2018-03-21 15:31 ` Dave Martin
-1 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:31 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: linux-arm-kernel, mark.rutland, ckadabi, ard.biesheuvel,
marc.zyngier, catalin.marinas, will.deacon, linux-kernel, jnair,
robin.murphy, shankerd
On Tue, Mar 13, 2018 at 11:51:20AM +0000, Suzuki K Poulose wrote:
> Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
> from an erratum 1024718, which causes incorrect updates when DBM/AP
> bits in a page table entry is modified without a break-before-make
> sequence. The work around is to skip enabling the hardware DBM feature
> on the affected cores. The hardware Access Flag management features
> is not affected. There are some other cores suffering from this
> errata, which could be added to the midr_list to trigger the work
> around.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: ckadabi@codeaurora.org
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
[...]
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 4e73c610a409..0fc097e2b72e 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -963,9 +963,23 @@ static inline void __cpu_enable_hw_dbm(void)
> isb();
> }
>
> +static bool cpu_has_broken_dbm(void)
> +{
> + /* List of CPUs which have broken DBM support. */
> + static const struct midr_range cpus[] = {
> +#ifdef CONFIG_ARM64_ERRATUM_1024718
> + MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
> +#endif
> + {},
> + };
> +
> + return is_midr_in_range_list(read_cpuid_id(), cpus);
> +}
> +
> static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
> {
> - return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
> + return has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
> + !cpu_has_broken_dbm();
This refactoring seems sensbile to me. The mapping between erratum IDs
and features that don't work is never guaranteed to be 1:1, so it
makes sense for the check function to have a generic name and to paste
in the relevant erratum ID ranges depending on the kernel config.
[...]
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
@ 2018-03-21 15:31 ` Dave Martin
0 siblings, 0 replies; 62+ messages in thread
From: Dave Martin @ 2018-03-21 15:31 UTC (permalink / raw)
To: linux-arm-kernel
On Tue, Mar 13, 2018 at 11:51:20AM +0000, Suzuki K Poulose wrote:
> Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
> from an erratum 1024718, which causes incorrect updates when DBM/AP
> bits in a page table entry is modified without a break-before-make
> sequence. The work around is to skip enabling the hardware DBM feature
> on the affected cores. The hardware Access Flag management features
> is not affected. There are some other cores suffering from this
> errata, which could be added to the midr_list to trigger the work
> around.
>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: ckadabi at codeaurora.org
> Cc: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
[...]
> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
> index 4e73c610a409..0fc097e2b72e 100644
> --- a/arch/arm64/kernel/cpufeature.c
> +++ b/arch/arm64/kernel/cpufeature.c
> @@ -963,9 +963,23 @@ static inline void __cpu_enable_hw_dbm(void)
> isb();
> }
>
> +static bool cpu_has_broken_dbm(void)
> +{
> + /* List of CPUs which have broken DBM support. */
> + static const struct midr_range cpus[] = {
> +#ifdef CONFIG_ARM64_ERRATUM_1024718
> + MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
> +#endif
> + {},
> + };
> +
> + return is_midr_in_range_list(read_cpuid_id(), cpus);
> +}
> +
> static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
> {
> - return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
> + return has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
> + !cpu_has_broken_dbm();
This refactoring seems sensbile to me. The mapping between erratum IDs
and features that don't work is never guaranteed to be 1:1, so it
makes sense for the check function to have a generic name and to paste
in the relevant erratum ID ranges depending on the kernel config.
[...]
Reviewed-by: Dave Martin <Dave.Martin@arm.com>
Cheers
---Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v4 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
2018-03-21 15:31 ` Dave Martin
@ 2018-03-22 16:42 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-22 16:42 UTC (permalink / raw)
To: Dave Martin
Cc: linux-arm-kernel, mark.rutland, ckadabi, ard.biesheuvel,
marc.zyngier, catalin.marinas, will.deacon, linux-kernel, jnair,
robin.murphy, shankerd
On 21/03/18 15:31, Dave Martin wrote:
> On Tue, Mar 13, 2018 at 11:51:20AM +0000, Suzuki K Poulose wrote:
>> Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
>> from an erratum 1024718, which causes incorrect updates when DBM/AP
>> bits in a page table entry is modified without a break-before-make
>> sequence. The work around is to skip enabling the hardware DBM feature
>> on the affected cores. The hardware Access Flag management features
>> is not affected. There are some other cores suffering from this
>> errata, which could be added to the midr_list to trigger the work
>> around.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: ckadabi@codeaurora.org
>> Cc: Dave Martin <dave.martin@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> [...]
>
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index 4e73c610a409..0fc097e2b72e 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -963,9 +963,23 @@ static inline void __cpu_enable_hw_dbm(void)
>> isb();
>> }
>>
>> +static bool cpu_has_broken_dbm(void)
>> +{
>> + /* List of CPUs which have broken DBM support. */
>> + static const struct midr_range cpus[] = {
>> +#ifdef CONFIG_ARM64_ERRATUM_1024718
>> + MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
>> +#endif
>> + {},
>> + };
>> +
>> + return is_midr_in_range_list(read_cpuid_id(), cpus);
>> +}
>> +
>> static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
>> {
>> - return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
>> + return has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
>> + !cpu_has_broken_dbm();
>
> This refactoring seems sensbile to me. The mapping between erratum IDs
> and features that don't work is never guaranteed to be 1:1, so it
> makes sense for the check function to have a generic name and to paste
> in the relevant erratum ID ranges depending on the kernel config.
>
Thanks Dave. Apologies for the lost change log for the patch. It looked
like this (before it was lost in a rebase :-()
Changes since v3:
- Renamed cpu_has_erratum_1024718() to cpu_has_broken_dbm() to
allow for different CPU Errata configs to share a single routine
for black listing CPUs. Dropped Dave's Reviewed-by
> [...]
>
> Reviewed-by: Dave Martin <Dave.Martin@arm.com>
Cheers
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718
@ 2018-03-22 16:42 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-22 16:42 UTC (permalink / raw)
To: linux-arm-kernel
On 21/03/18 15:31, Dave Martin wrote:
> On Tue, Mar 13, 2018 at 11:51:20AM +0000, Suzuki K Poulose wrote:
>> Some variants of the Arm Cortex-55 cores (r0p0, r0p1, r1p0) suffer
>> from an erratum 1024718, which causes incorrect updates when DBM/AP
>> bits in a page table entry is modified without a break-before-make
>> sequence. The work around is to skip enabling the hardware DBM feature
>> on the affected cores. The hardware Access Flag management features
>> is not affected. There are some other cores suffering from this
>> errata, which could be added to the midr_list to trigger the work
>> around.
>>
>> Cc: Catalin Marinas <catalin.marinas@arm.com>
>> Cc: ckadabi at codeaurora.org
>> Cc: Dave Martin <dave.martin@arm.com>
>> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
>
> [...]
>
>> diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
>> index 4e73c610a409..0fc097e2b72e 100644
>> --- a/arch/arm64/kernel/cpufeature.c
>> +++ b/arch/arm64/kernel/cpufeature.c
>> @@ -963,9 +963,23 @@ static inline void __cpu_enable_hw_dbm(void)
>> isb();
>> }
>>
>> +static bool cpu_has_broken_dbm(void)
>> +{
>> + /* List of CPUs which have broken DBM support. */
>> + static const struct midr_range cpus[] = {
>> +#ifdef CONFIG_ARM64_ERRATUM_1024718
>> + MIDR_RANGE(MIDR_CORTEX_A55, 0, 0, 1, 0), // A55 r0p0 -r1p0
>> +#endif
>> + {},
>> + };
>> +
>> + return is_midr_in_range_list(read_cpuid_id(), cpus);
>> +}
>> +
>> static bool cpu_can_use_dbm(const struct arm64_cpu_capabilities *cap)
>> {
>> - return has_cpuid_feature(cap, SCOPE_LOCAL_CPU);
>> + return has_cpuid_feature(cap, SCOPE_LOCAL_CPU) &&
>> + !cpu_has_broken_dbm();
>
> This refactoring seems sensbile to me. The mapping between erratum IDs
> and features that don't work is never guaranteed to be 1:1, so it
> makes sense for the check function to have a generic name and to paste
> in the relevant erratum ID ranges depending on the kernel config.
>
Thanks Dave. Apologies for the lost change log for the patch. It looked
like this (before it was lost in a rebase :-()
Changes since v3:
- Renamed cpu_has_erratum_1024718() to cpu_has_broken_dbm() to
allow for different CPU Errata configs to share a single routine
for black listing CPUs. Dropped Dave's Reviewed-by
> [...]
>
> Reviewed-by: Dave Martin <Dave.Martin@arm.com>
Cheers
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v4 19/22] arm64: capabilities: Handle shared entries
2018-03-13 11:51 ` Suzuki K Poulose
@ 2018-03-22 16:43 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-22 16:43 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd
On 13/03/18 11:51, Suzuki K Poulose wrote:
> Some capabilities have different criteria for detection and associated
> actions based on the matching criteria, even though they all share the
> same capability bit. So far we have used multiple entries with the same
> capability bit to handle this. This is prone to errors, as the
> cpu_enable is invoked for each entry, irrespective of whether the
> detection rule applies to the CPU or not. And also this complicates
> other helpers, e.g, __this_cpu_has_cap.
>
> This patch adds a wrapper entry to cover all the possible variations
> of a capability by maintaining list of matches + cpu_enable callbacks.
> To avoid complicating the prototypes for the "matches()", we use
> arm64_cpu_capabilities maintain the list and we ignore all the other
> fields except the matches & cpu_enable.
>
> This ensures :
>
> 1) The capabilitiy is set when at least one of the entry detects
> 2) Action is only taken for the entries that "matches".
>
> This avoids explicit checks in the cpu_enable() take some action.
> The only constraint here is that, all the entries should have the
> same "type" (i.e, scope and conflict rules).
>
> If a cpu_enable() method is associated with multiple matches for a
> single capability, care should be taken that either the match criteria
> are mutually exclusive, or that the method is robust against being
> called multiple times.
>
> This also reverts the changes introduced by commit 67948af41f2e6818ed
> ("arm64: capabilities: Handle duplicate entries for a capability").
>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Reviewed-by: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Lost changelog:
Changes since v3
- Rename cap_list => match_list
- Address other comments from Dave
- Added Reviewed-by from Dave
Cheers
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 19/22] arm64: capabilities: Handle shared entries
@ 2018-03-22 16:43 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-22 16:43 UTC (permalink / raw)
To: linux-arm-kernel
On 13/03/18 11:51, Suzuki K Poulose wrote:
> Some capabilities have different criteria for detection and associated
> actions based on the matching criteria, even though they all share the
> same capability bit. So far we have used multiple entries with the same
> capability bit to handle this. This is prone to errors, as the
> cpu_enable is invoked for each entry, irrespective of whether the
> detection rule applies to the CPU or not. And also this complicates
> other helpers, e.g, __this_cpu_has_cap.
>
> This patch adds a wrapper entry to cover all the possible variations
> of a capability by maintaining list of matches + cpu_enable callbacks.
> To avoid complicating the prototypes for the "matches()", we use
> arm64_cpu_capabilities maintain the list and we ignore all the other
> fields except the matches & cpu_enable.
>
> This ensures :
>
> 1) The capabilitiy is set when at least one of the entry detects
> 2) Action is only taken for the entries that "matches".
>
> This avoids explicit checks in the cpu_enable() take some action.
> The only constraint here is that, all the entries should have the
> same "type" (i.e, scope and conflict rules).
>
> If a cpu_enable() method is associated with multiple matches for a
> single capability, care should be taken that either the match criteria
> are mutually exclusive, or that the method is robust against being
> called multiple times.
>
> This also reverts the changes introduced by commit 67948af41f2e6818ed
> ("arm64: capabilities: Handle duplicate entries for a capability").
>
> Cc: Robin Murphy <robin.murphy@arm.com>
> Reviewed-by: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Lost changelog:
Changes since v3
- Rename cap_list => match_list
- Address other comments from Dave
- Added Reviewed-by from Dave
Cheers
Suzuki
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v4 09/22] arm64: capabilities: Split the processing of errata work arounds
2018-03-13 11:51 ` Suzuki K Poulose
@ 2018-03-22 16:44 ` Suzuki K Poulose
-1 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-22 16:44 UTC (permalink / raw)
To: linux-arm-kernel
Cc: linux-kernel, will.deacon, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd
On 13/03/18 11:51, Suzuki K Poulose wrote:
> Right now we run through the errata workarounds check on all boot
> active CPUs, with SCOPE_ALL. This wouldn't help for detecting erratum
> workarounds with a SYSTEM_SCOPE. There are none yet, but we plan to
> introduce some: let us clean this up so that such workarounds can be
> detected and enabled correctly.
>
> So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
> checks are run only once after all the boot time CPUs are active.
>
> Reviewed-by: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Changes since v3
- Reworded commit description
- Added Reviewed-by tag from Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 09/22] arm64: capabilities: Split the processing of errata work arounds
@ 2018-03-22 16:44 ` Suzuki K Poulose
0 siblings, 0 replies; 62+ messages in thread
From: Suzuki K Poulose @ 2018-03-22 16:44 UTC (permalink / raw)
To: linux-arm-kernel
On 13/03/18 11:51, Suzuki K Poulose wrote:
> Right now we run through the errata workarounds check on all boot
> active CPUs, with SCOPE_ALL. This wouldn't help for detecting erratum
> workarounds with a SYSTEM_SCOPE. There are none yet, but we plan to
> introduce some: let us clean this up so that such workarounds can be
> detected and enabled correctly.
>
> So, we run the checks with SCOPE_LOCAL_CPU on all CPUs and SCOPE_SYSTEM
> checks are run only once after all the boot time CPUs are active.
>
> Reviewed-by: Dave Martin <dave.martin@arm.com>
> Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com>
Changes since v3
- Reworded commit description
- Added Reviewed-by tag from Dave
^ permalink raw reply [flat|nested] 62+ messages in thread
* Re: [PATCH v4 00/22] arm64: Rework cpu capabilities handling
2018-03-13 11:50 ` Suzuki K Poulose
@ 2018-03-26 11:53 ` Will Deacon
-1 siblings, 0 replies; 62+ messages in thread
From: Will Deacon @ 2018-03-26 11:53 UTC (permalink / raw)
To: Suzuki K Poulose
Cc: linux-arm-kernel, linux-kernel, dave.martin, catalin.marinas,
marc.zyngier, mark.rutland, ard.biesheuvel, jnair, ckadabi,
robin.murphy, shankerd
Hi Suzuki,
On Tue, Mar 13, 2018 at 11:50:58AM +0000, Suzuki K Poulose wrote:
> This series reworks the arm64 CPU capabilities handling (which
> manages the system features and errata). The current infrastructure
> doesn't allow fine control for handling different features or errata.
> There is one rule for features and another rule for errata.
Please can you repost this based on top of for-next/core and with Dave's
acks added?
Thanks,
Will
^ permalink raw reply [flat|nested] 62+ messages in thread
* [PATCH v4 00/22] arm64: Rework cpu capabilities handling
@ 2018-03-26 11:53 ` Will Deacon
0 siblings, 0 replies; 62+ messages in thread
From: Will Deacon @ 2018-03-26 11:53 UTC (permalink / raw)
To: linux-arm-kernel
Hi Suzuki,
On Tue, Mar 13, 2018 at 11:50:58AM +0000, Suzuki K Poulose wrote:
> This series reworks the arm64 CPU capabilities handling (which
> manages the system features and errata). The current infrastructure
> doesn't allow fine control for handling different features or errata.
> There is one rule for features and another rule for errata.
Please can you repost this based on top of for-next/core and with Dave's
acks added?
Thanks,
Will
^ permalink raw reply [flat|nested] 62+ messages in thread
end of thread, other threads:[~2018-03-26 11:53 UTC | newest]
Thread overview: 62+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-03-13 11:50 [PATCH v4 00/22] arm64: Rework cpu capabilities handling Suzuki K Poulose
2018-03-13 11:50 ` Suzuki K Poulose
2018-03-13 11:50 ` [PATCH v4 01/22] arm64: capabilities: Update prototype for enable call back Suzuki K Poulose
2018-03-13 11:50 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 02/22] arm64: capabilities: Move errata work around check on boot CPU Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 03/22] arm64: capabilities: Move errata processing code Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 04/22] arm64: capabilities: Prepare for fine grained capabilities Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 05/22] arm64: capabilities: Add flags to handle the conflicts on late CPU Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 06/22] arm64: capabilities: Unify the verification Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 07/22] arm64: capabilities: Filter the entries based on a given mask Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 08/22] arm64: capabilities: Prepare for grouping features and errata work arounds Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 09/22] arm64: capabilities: Split the processing of " Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-22 16:44 ` Suzuki K Poulose
2018-03-22 16:44 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 10/22] arm64: capabilities: Allow features based on local CPU scope Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 11/22] arm64: capabilities: Group handling of features and errata workarounds Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-21 15:07 ` Dave Martin
2018-03-21 15:07 ` Dave Martin
2018-03-13 11:51 ` [PATCH v4 12/22] arm64: capabilities: Introduce weak features based on local CPU Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 13/22] arm64: capabilities: Restrict KPTI detection to boot-time CPUs Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 14/22] arm64: capabilities: Add support for features enabled early Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 15/22] arm64: capabilities: Change scope of VHE to Boot CPU feature Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-21 15:18 ` Dave Martin
2018-03-21 15:18 ` Dave Martin
2018-03-13 11:51 ` [PATCH v4 16/22] arm64: capabilities: Clean up midr range helpers Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 17/22] arm64: Add helpers for checking CPU MIDR against a range Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 18/22] arm64: capabilities: Add support for checks based on a list of MIDRs Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 19/22] arm64: capabilities: Handle shared entries Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-22 16:43 ` Suzuki K Poulose
2018-03-22 16:43 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 20/22] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35 Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-13 11:51 ` [PATCH v4 21/22] arm64: Delay enabling hardware DBM feature Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-21 15:22 ` Dave Martin
2018-03-21 15:22 ` Dave Martin
2018-03-13 11:51 ` [PATCH v4 22/22] arm64: Add work around for Arm Cortex-A55 Erratum 1024718 Suzuki K Poulose
2018-03-13 11:51 ` Suzuki K Poulose
2018-03-21 15:31 ` Dave Martin
2018-03-21 15:31 ` Dave Martin
2018-03-22 16:42 ` Suzuki K Poulose
2018-03-22 16:42 ` Suzuki K Poulose
2018-03-26 11:53 ` [PATCH v4 00/22] arm64: Rework cpu capabilities handling Will Deacon
2018-03-26 11:53 ` Will Deacon
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.