From: Suzuki K Poulose <suzuki.poulose@arm.com> To: linux-arm-kernel@lists.infradead.org Cc: linux-kernel@vger.kernel.org, ard.biesheuvel@linaro.org, will.deacon@arm.com, mark.rutland@arm.com, marc.zyngier@arm.com, catalin.marinas@arm.com, ckadabi@codeaurora.org, dave.martin@arm.com, jnair@caviumnetworks.com, Suzuki K Poulose <suzuki.poulose@arm.com> Subject: [PATCH v2 19/20] arm64: Delay enabling hardware DBM feature Date: Wed, 31 Jan 2018 18:28:06 +0000 [thread overview] Message-ID: <20180131182807.32134-20-suzuki.poulose@arm.com> (raw) In-Reply-To: <20180131182807.32134-1-suzuki.poulose@arm.com> We enable hardware DBM bit in a capable CPU, very early in the boot via __cpu_setup. This doesn't give us a flexibility of optionally disable the feature, as the clearing the bit is a bit costly as the TLB can cache the settings. Instead, we delay enabling the feature until the CPU is brought up into the kernel. We use the feature capability mechanism to handle it. The hardware DBM is a non-conflicting feature. i.e, the kernel can safely run with a mix of CPUs with some using the feature and the others don't. So, it is safe for a late CPU to have this capability and enable it, even if the active CPUs don't. To get this handled properly by the infrastructure, we unconditionally set the capability and only enable it on CPUs which really have the feature. Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com> --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/kernel/cpufeature.c | 40 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/mm/proc.S | 9 +++------ 3 files changed, 45 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index bb263820de13..8df80cc828ac 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -45,7 +45,8 @@ #define ARM64_HARDEN_BRANCH_PREDICTOR 24 #define ARM64_HARDEN_BP_POST_GUEST_EXIT 25 #define ARM64_HAS_RAS_EXTN 26 +#define ARM64_HW_DBM 27 -#define ARM64_NCAPS 27 +#define ARM64_NCAPS 28 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 13e30c1b1e99..1f695a998eed 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -903,6 +903,33 @@ static int __init parse_kpti(char *str) __setup("kpti=", parse_kpti); #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ +#ifdef CONFIG_ARM64_HW_AFDBM +static bool has_hw_dbm(const struct arm64_cpu_capabilities *entry, int scope) +{ + /* + * DBM is a non-conflicting feature. i.e, the kernel can safely run + * a mix of CPUs with and without the feature. So, we unconditionally + * enable the capability to allow any late CPU to use the feature. + * We only enable the control bits on the CPU, if it actually supports. + */ + return true; +} + +static inline void __cpu_enable_hw_dbm(void) +{ + u64 tcr = read_sysreg(tcr_el1) | TCR_HD; + + write_sysreg(tcr, tcr_el1); + isb(); +} + +static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap) +{ + if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU)) + __cpu_enable_hw_dbm(); +} +#endif + static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused) { /* @@ -1059,6 +1086,19 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpu_enable = cpu_clear_disr, }, #endif /* CONFIG_ARM64_RAS_EXTN */ +#ifdef CONFIG_ARM64_HW_AFDBM + { + .desc = "Hardware pagetable Dirty Bit Management", + .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE, + .capability = ARM64_HW_DBM, + .sys_reg = SYS_ID_AA64MMFR1_EL1, + .sign = FTR_UNSIGNED, + .field_pos = ID_AA64MMFR1_HADBS_SHIFT, + .min_field_value = 2, + .matches = has_hw_dbm, + .cpu_enable = cpu_enable_hw_dbm, + }, +#endif {}, }; diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 9f177aac6390..68e08ad54944 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -253,12 +253,9 @@ ENTRY(__cpu_setup) */ mrs x9, ID_AA64MMFR1_EL1 and x9, x9, #0xf - cbz x9, 2f - cmp x9, #2 - b.lt 1f - orr x10, x10, #TCR_HD // hardware Dirty flag update -1: orr x10, x10, #TCR_HA // hardware Access flag update -2: + cbz x9, 1f + orr x10, x10, #TCR_HA // hardware Access flag update +1: #endif /* CONFIG_ARM64_HW_AFDBM */ msr tcr_el1, x10 ret // return to head.S -- 2.14.3
WARNING: multiple messages have this Message-ID (diff)
From: suzuki.poulose@arm.com (Suzuki K Poulose) To: linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 19/20] arm64: Delay enabling hardware DBM feature Date: Wed, 31 Jan 2018 18:28:06 +0000 [thread overview] Message-ID: <20180131182807.32134-20-suzuki.poulose@arm.com> (raw) In-Reply-To: <20180131182807.32134-1-suzuki.poulose@arm.com> We enable hardware DBM bit in a capable CPU, very early in the boot via __cpu_setup. This doesn't give us a flexibility of optionally disable the feature, as the clearing the bit is a bit costly as the TLB can cache the settings. Instead, we delay enabling the feature until the CPU is brought up into the kernel. We use the feature capability mechanism to handle it. The hardware DBM is a non-conflicting feature. i.e, the kernel can safely run with a mix of CPUs with some using the feature and the others don't. So, it is safe for a late CPU to have this capability and enable it, even if the active CPUs don't. To get this handled properly by the infrastructure, we unconditionally set the capability and only enable it on CPUs which really have the feature. Signed-off-by: Suzuki K Poulose <suzuki.poulose@arm.com> --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/kernel/cpufeature.c | 40 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/mm/proc.S | 9 +++------ 3 files changed, 45 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index bb263820de13..8df80cc828ac 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -45,7 +45,8 @@ #define ARM64_HARDEN_BRANCH_PREDICTOR 24 #define ARM64_HARDEN_BP_POST_GUEST_EXIT 25 #define ARM64_HAS_RAS_EXTN 26 +#define ARM64_HW_DBM 27 -#define ARM64_NCAPS 27 +#define ARM64_NCAPS 28 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 13e30c1b1e99..1f695a998eed 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -903,6 +903,33 @@ static int __init parse_kpti(char *str) __setup("kpti=", parse_kpti); #endif /* CONFIG_UNMAP_KERNEL_AT_EL0 */ +#ifdef CONFIG_ARM64_HW_AFDBM +static bool has_hw_dbm(const struct arm64_cpu_capabilities *entry, int scope) +{ + /* + * DBM is a non-conflicting feature. i.e, the kernel can safely run + * a mix of CPUs with and without the feature. So, we unconditionally + * enable the capability to allow any late CPU to use the feature. + * We only enable the control bits on the CPU, if it actually supports. + */ + return true; +} + +static inline void __cpu_enable_hw_dbm(void) +{ + u64 tcr = read_sysreg(tcr_el1) | TCR_HD; + + write_sysreg(tcr, tcr_el1); + isb(); +} + +static void cpu_enable_hw_dbm(struct arm64_cpu_capabilities const *cap) +{ + if (has_cpuid_feature(cap, SCOPE_LOCAL_CPU)) + __cpu_enable_hw_dbm(); +} +#endif + static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused) { /* @@ -1059,6 +1086,19 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .cpu_enable = cpu_clear_disr, }, #endif /* CONFIG_ARM64_RAS_EXTN */ +#ifdef CONFIG_ARM64_HW_AFDBM + { + .desc = "Hardware pagetable Dirty Bit Management", + .type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE, + .capability = ARM64_HW_DBM, + .sys_reg = SYS_ID_AA64MMFR1_EL1, + .sign = FTR_UNSIGNED, + .field_pos = ID_AA64MMFR1_HADBS_SHIFT, + .min_field_value = 2, + .matches = has_hw_dbm, + .cpu_enable = cpu_enable_hw_dbm, + }, +#endif {}, }; diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 9f177aac6390..68e08ad54944 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -253,12 +253,9 @@ ENTRY(__cpu_setup) */ mrs x9, ID_AA64MMFR1_EL1 and x9, x9, #0xf - cbz x9, 2f - cmp x9, #2 - b.lt 1f - orr x10, x10, #TCR_HD // hardware Dirty flag update -1: orr x10, x10, #TCR_HA // hardware Access flag update -2: + cbz x9, 1f + orr x10, x10, #TCR_HA // hardware Access flag update +1: #endif /* CONFIG_ARM64_HW_AFDBM */ msr tcr_el1, x10 ret // return to head.S -- 2.14.3
next prev parent reply other threads:[~2018-01-31 18:29 UTC|newest] Thread overview: 156+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-01-31 18:27 [PATCH v2 00/20] arm64: Rework cpu capabilities handling Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-01-31 18:27 ` [PATCH v2 01/20] arm64: capabilities: Update prototype for enable call back Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:37 ` Dave Martin 2018-02-07 10:37 ` Dave Martin 2018-02-07 11:23 ` Robin Murphy 2018-02-07 11:23 ` Robin Murphy 2018-01-31 18:27 ` [PATCH v2 02/20] arm64: capabilities: Move errata work around check on boot CPU Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:37 ` Dave Martin 2018-02-07 10:37 ` Dave Martin 2018-02-07 14:47 ` Suzuki K Poulose 2018-02-07 14:47 ` Suzuki K Poulose 2018-01-31 18:27 ` [PATCH v2 03/20] arm64: capabilities: Move errata processing code Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:37 ` Dave Martin 2018-02-07 10:37 ` Dave Martin 2018-01-31 18:27 ` [PATCH v2 04/20] arm64: capabilities: Prepare for fine grained capabilities Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:37 ` Dave Martin 2018-02-07 10:37 ` Dave Martin 2018-02-07 15:16 ` Suzuki K Poulose 2018-02-07 15:16 ` Suzuki K Poulose 2018-02-07 15:39 ` Dave Martin 2018-02-07 15:39 ` Dave Martin 2018-01-31 18:27 ` [PATCH v2 05/20] arm64: capabilities: Add flags to handle the conflicts on late CPU Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:38 ` Dave Martin 2018-02-07 10:38 ` Dave Martin 2018-02-07 11:31 ` Robin Murphy 2018-02-07 11:31 ` Robin Murphy 2018-02-07 16:53 ` Suzuki K Poulose 2018-02-07 16:53 ` Suzuki K Poulose 2018-01-31 18:27 ` [PATCH v2 06/20] arm64: capabilities: Unify the verification Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:38 ` Dave Martin 2018-02-07 10:38 ` Dave Martin 2018-02-07 16:56 ` Suzuki K Poulose 2018-02-07 16:56 ` Suzuki K Poulose 2018-01-31 18:27 ` [PATCH v2 07/20] arm64: capabilities: Filter the entries based on a given mask Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:38 ` Dave Martin 2018-02-07 10:38 ` Dave Martin 2018-02-07 17:01 ` Suzuki K Poulose 2018-02-07 17:01 ` Suzuki K Poulose 2018-01-31 18:27 ` [PATCH v2 08/20] arm64: capabilities: Group handling of features and errata Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:38 ` Dave Martin 2018-02-07 10:38 ` Dave Martin 2018-02-08 12:10 ` Suzuki K Poulose 2018-02-08 12:10 ` Suzuki K Poulose 2018-02-08 12:12 ` [PATCH 1/2] arm64: capabilities: Allow flexibility in scope Suzuki K Poulose 2018-02-08 12:12 ` Suzuki K Poulose 2018-02-08 12:12 ` [PATCH 2/2] arm64: capabilities: Group handling of features and errata workarounds Suzuki K Poulose 2018-02-08 12:12 ` Suzuki K Poulose 2018-02-08 16:10 ` [PATCH 1/2] arm64: capabilities: Allow flexibility in scope Dave Martin 2018-02-08 16:10 ` Dave Martin 2018-02-08 16:31 ` Suzuki K Poulose 2018-02-08 16:31 ` Suzuki K Poulose 2018-02-08 17:32 ` Dave Martin 2018-02-08 17:32 ` Dave Martin 2018-02-09 12:16 ` Suzuki K Poulose 2018-02-09 12:16 ` Suzuki K Poulose 2018-02-09 12:16 ` [PATCH 1/4] arm64: capabilities: Prepare for grouping features and errata work arounds Suzuki K Poulose 2018-02-09 12:16 ` Suzuki K Poulose 2018-02-09 12:16 ` [PATCH 2/4] arm64: capabilities: Split the processing of " Suzuki K Poulose 2018-02-09 12:16 ` Suzuki K Poulose 2018-02-09 12:16 ` [PATCH 3/4] arm64: capabilities: Allow features based on local CPU scope Suzuki K Poulose 2018-02-09 12:16 ` Suzuki K Poulose 2018-02-09 12:16 ` [PATCH 4/4] arm64: capabilities: Group handling of features and errata workarounds Suzuki K Poulose 2018-02-09 12:16 ` Suzuki K Poulose 2018-02-09 12:19 ` Suzuki K Poulose 2018-02-09 12:19 ` Suzuki K Poulose 2018-02-09 14:21 ` [PATCH 1/2] arm64: capabilities: Allow flexibility in scope Dave Martin 2018-02-09 14:21 ` Dave Martin 2018-01-31 18:27 ` [PATCH v2 09/20] arm64: capabilities: Introduce weak features based on local CPU Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:38 ` Dave Martin 2018-02-07 10:38 ` Dave Martin 2018-01-31 18:27 ` [PATCH v2 10/20] arm64: capabilities: Restrict KPTI detection to boot-time CPUs Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:38 ` Dave Martin 2018-02-07 10:38 ` Dave Martin 2018-02-07 18:15 ` Suzuki K Poulose 2018-02-07 18:15 ` Suzuki K Poulose 2018-02-08 11:05 ` Dave Martin 2018-02-08 11:05 ` Dave Martin 2018-01-31 18:27 ` [PATCH v2 11/20] arm64: capabilities: Add support for features enabled early Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:38 ` Dave Martin 2018-02-07 10:38 ` Dave Martin 2018-02-07 18:34 ` Suzuki K Poulose 2018-02-07 18:34 ` Suzuki K Poulose 2018-02-08 11:35 ` Dave Martin 2018-02-08 11:35 ` Dave Martin 2018-02-08 11:43 ` Suzuki K Poulose 2018-02-08 11:43 ` Suzuki K Poulose 2018-01-31 18:27 ` [PATCH v2 12/20] arm64: capabilities: Change scope of VHE to Boot CPU feature Suzuki K Poulose 2018-01-31 18:27 ` Suzuki K Poulose 2018-02-07 10:39 ` Dave Martin 2018-02-07 10:39 ` Dave Martin 2018-01-31 18:28 ` [PATCH v2 13/20] arm64: capabilities: Clean up midr range helpers Suzuki K Poulose 2018-01-31 18:28 ` Suzuki K Poulose 2018-02-07 10:39 ` Dave Martin 2018-02-07 10:39 ` Dave Martin 2018-01-31 18:28 ` [PATCH v2 14/20] arm64: Add helpers for checking CPU MIDR against a range Suzuki K Poulose 2018-01-31 18:28 ` Suzuki K Poulose 2018-02-07 10:39 ` Dave Martin 2018-02-07 10:39 ` Dave Martin 2018-01-31 18:28 ` [PATCH v2 15/20] arm64: capabilities: Add support for checks based on a list of MIDRs Suzuki K Poulose 2018-01-31 18:28 ` Suzuki K Poulose 2018-02-07 10:39 ` Dave Martin 2018-02-07 10:39 ` Dave Martin 2018-01-31 18:28 ` [PATCH v2 16/20] arm64: Handle shared capability entries Suzuki K Poulose 2018-01-31 18:28 ` Suzuki K Poulose 2018-02-07 10:39 ` Dave Martin 2018-02-07 10:39 ` Dave Martin 2018-02-08 10:53 ` Suzuki K Poulose 2018-02-08 10:53 ` Suzuki K Poulose 2018-02-08 12:01 ` Dave Martin 2018-02-08 12:01 ` Dave Martin 2018-02-08 12:32 ` Robin Murphy 2018-02-08 12:32 ` Robin Murphy 2018-02-09 10:05 ` Dave Martin 2018-02-09 10:05 ` Dave Martin 2018-02-08 12:04 ` Dave Martin 2018-02-08 12:04 ` Dave Martin 2018-02-08 12:05 ` Suzuki K Poulose 2018-02-08 12:05 ` Suzuki K Poulose 2018-01-31 18:28 ` [PATCH v2 17/20] arm64: bp hardening: Allow late CPUs to enable work around Suzuki K Poulose 2018-01-31 18:28 ` Suzuki K Poulose 2018-02-07 10:39 ` Dave Martin 2018-02-07 10:39 ` Dave Martin 2018-02-08 12:19 ` Suzuki K Poulose 2018-02-08 12:19 ` Suzuki K Poulose 2018-02-08 12:26 ` Marc Zyngier 2018-02-08 12:26 ` Marc Zyngier 2018-02-08 16:58 ` Suzuki K Poulose 2018-02-08 16:58 ` Suzuki K Poulose 2018-02-08 17:59 ` Suzuki K Poulose 2018-02-08 17:59 ` Suzuki K Poulose 2018-02-08 17:59 ` Suzuki K Poulose 2018-02-08 17:59 ` Suzuki K Poulose 2018-01-31 18:28 ` [PATCH v2 18/20] arm64: Add MIDR encoding for Arm Cortex-A55 and Cortex-A35 Suzuki K Poulose 2018-01-31 18:28 ` Suzuki K Poulose 2018-02-07 10:39 ` Dave Martin 2018-02-07 10:39 ` Dave Martin 2018-01-31 18:28 ` Suzuki K Poulose [this message] 2018-01-31 18:28 ` [PATCH v2 19/20] arm64: Delay enabling hardware DBM feature Suzuki K Poulose 2018-02-07 10:40 ` Dave Martin 2018-02-07 10:40 ` Dave Martin 2018-01-31 18:28 ` [PATCH v2 20/20] arm64: Add work around for Arm Cortex-A55 Erratum 1024718 Suzuki K Poulose 2018-01-31 18:28 ` Suzuki K Poulose 2018-02-07 10:40 ` Dave Martin 2018-02-07 10:40 ` Dave Martin
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20180131182807.32134-20-suzuki.poulose@arm.com \ --to=suzuki.poulose@arm.com \ --cc=ard.biesheuvel@linaro.org \ --cc=catalin.marinas@arm.com \ --cc=ckadabi@codeaurora.org \ --cc=dave.martin@arm.com \ --cc=jnair@caviumnetworks.com \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=linux-kernel@vger.kernel.org \ --cc=marc.zyngier@arm.com \ --cc=mark.rutland@arm.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.