[PATCH v4 0/4] E0PD support

[PATCH v4 0/4] E0PD support

[Mark Brown]

This series adds support for E0PD. We enable E0PD unconditionally where
present on systems where all the CPUs in the system support E0PD and
change to not enabling KPTI by default on systems where we have enabled
E0PD. It also converts the runtime checks for use of non-global mappings
into a variable.

v4: Use a variable to store our decision about using non-global mappings
    rather than rechecking constantly at runtime. I've added this as a
    separate patch mainly for bisection.
v3: Make E0PD a system wide feature.

Mark Brown (4):
  arm64: Add initial support for E0PD
  arm64: Factor out checks for KASLR in KPTI code into separate function
  arm64: Don't use KPTI where we have E0PD
  arm64: Use a variable to store non-global mappings decision

 arch/arm64/Kconfig                     | 15 ++++++
 arch/arm64/include/asm/cpucaps.h       |  3 +-
 arch/arm64/include/asm/mmu.h           | 48 ++---------------
 arch/arm64/include/asm/pgtable-hwdef.h |  2 +
 arch/arm64/include/asm/pgtable-prot.h  |  4 +-
 arch/arm64/include/asm/sysreg.h        |  1 +
 arch/arm64/kernel/cpufeature.c         | 71 ++++++++++++++++++++++++--
 7 files changed, 94 insertions(+), 50 deletions(-)

-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v4 1/4] arm64: Add initial support for E0PD

[Mark Brown]

Kernel Page Table Isolation (KPTI) is used to mitigate some speculation
based security issues by ensuring that the kernel is not mapped when
userspace is running but this approach is expensive and is incompatible
with SPE.  E0PD, introduced in the ARMv8.5 extensions, provides an
alternative to this which ensures that accesses from userspace to the
kernel's half of the memory map to always fault with constant time,
preventing timing attacks without requiring constant unmapping and
remapping or preventing legitimate accesses.

Currently this feature will only be enabled if all CPUs in the system
support E0PD, if some CPUs do not support the feature at boot time then
the feature will not be enabled and in the unlikely event that a late
CPU is the first CPU to lack the feature then we will reject that CPU.

This initial patch does not yet integrate with KPTI, this will be dealt
with in followup patches.  Ideally we could ensure that by default we
don't use KPTI on CPUs where E0PD is present.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/Kconfig                     | 15 ++++++++++++++
 arch/arm64/include/asm/cpucaps.h       |  3 ++-
 arch/arm64/include/asm/pgtable-hwdef.h |  2 ++
 arch/arm64/include/asm/sysreg.h        |  1 +
 arch/arm64/kernel/cpufeature.c         | 27 ++++++++++++++++++++++++++
 5 files changed, 47 insertions(+), 1 deletion(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 8f2544bdcbe6..2ecfe5d02d81 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1442,6 +1442,21 @@ config ARM64_PTR_AUTH
 
 endmenu
 
+menu "ARMv8.5 architectural features"
+
+config ARM64_E0PD
+	bool "Enable support for E0PD"
+	default y
+	help
+	   E0PD (part of the ARMv8.5 extensions) allows us to ensure
+	   that EL0 accesses made via TTBR1 always fault in constant time,
+	   providing benefits to KPTI with lower overhead and without
+	   disrupting legitimate access to kernel memory such as SPE.
+
+	   This option enables E0PD for TTBR1 where available.
+
+endmenu
+
 config ARM64_SVE
 	bool "ARM Scalable Vector Extension support"
 	default y
diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h
index ac1dbca3d0cd..c038fd92e36c 100644
--- a/arch/arm64/include/asm/cpucaps.h
+++ b/arch/arm64/include/asm/cpucaps.h
@@ -54,7 +54,8 @@
 #define ARM64_WORKAROUND_1463225		44
 #define ARM64_WORKAROUND_CAVIUM_TX2_219_TVM	45
 #define ARM64_WORKAROUND_CAVIUM_TX2_219_PRFM	46
+#define ARM64_HAS_E0PD				47
 
-#define ARM64_NCAPS				47
+#define ARM64_NCAPS				48
 
 #endif /* __ASM_CPUCAPS_H */
diff --git a/arch/arm64/include/asm/pgtable-hwdef.h b/arch/arm64/include/asm/pgtable-hwdef.h
index 3df60f97da1f..685842e52c3d 100644
--- a/arch/arm64/include/asm/pgtable-hwdef.h
+++ b/arch/arm64/include/asm/pgtable-hwdef.h
@@ -292,6 +292,8 @@
 #define TCR_HD			(UL(1) << 40)
 #define TCR_NFD0		(UL(1) << 53)
 #define TCR_NFD1		(UL(1) << 54)
+#define TCR_E0PD0		(UL(1) << 55)
+#define TCR_E0PD1		(UL(1) << 56)
 
 /*
  * TTBR.
diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h
index 6e919fafb43d..b085258cfe4e 100644
--- a/arch/arm64/include/asm/sysreg.h
+++ b/arch/arm64/include/asm/sysreg.h
@@ -655,6 +655,7 @@
 #define ID_AA64MMFR1_VMIDBITS_16	2
 
 /* id_aa64mmfr2 */
+#define ID_AA64MMFR2_E0PD_SHIFT		60
 #define ID_AA64MMFR2_FWB_SHIFT		40
 #define ID_AA64MMFR2_AT_SHIFT		32
 #define ID_AA64MMFR2_LVA_SHIFT		16
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 80f459ad0190..4e2009711c69 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -225,6 +225,7 @@ static const struct arm64_ftr_bits ftr_id_aa64mmfr1[] = {
 };
 
 static const struct arm64_ftr_bits ftr_id_aa64mmfr2[] = {
+	ARM64_FTR_BITS(FTR_HIDDEN, FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_E0PD_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_FWB_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_VISIBLE, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_AT_SHIFT, 4, 0),
 	ARM64_FTR_BITS(FTR_HIDDEN, FTR_STRICT, FTR_LOWER_SAFE, ID_AA64MMFR2_LVA_SHIFT, 4, 0),
@@ -1250,6 +1251,19 @@ static void cpu_enable_address_auth(struct arm64_cpu_capabilities const *cap)
 }
 #endif /* CONFIG_ARM64_PTR_AUTH */
 
+#ifdef CONFIG_ARM64_E0PD
+static void cpu_enable_e0pd(struct arm64_cpu_capabilities const *cap)
+{
+	/*
+	 * The cpu_enable() callback gets called even on CPUs that
+	 * don't detect the feature so we need to verify if we can
+	 * enable.
+	 */
+	if (this_cpu_has_cap(ARM64_HAS_E0PD))
+		sysreg_clear_set(tcr_el1, 0, TCR_E0PD1);
+}
+#endif /* CONFIG_ARM64_E0PD */
+
 #ifdef CONFIG_ARM64_PSEUDO_NMI
 static bool enable_pseudo_nmi;
 
@@ -1565,6 +1579,19 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
 		.sign = FTR_UNSIGNED,
 		.min_field_value = 1,
 	},
+#endif
+#ifdef CONFIG_ARM64_E0PD
+	{
+		.desc = "E0PD",
+		.capability = ARM64_HAS_E0PD,
+		.type = ARM64_CPUCAP_SYSTEM_FEATURE,
+		.sys_reg = SYS_ID_AA64MMFR2_EL1,
+		.sign = FTR_UNSIGNED,
+		.field_pos = ID_AA64MMFR2_E0PD_SHIFT,
+		.matches = has_cpuid_feature,
+		.min_field_value = 1,
+		.cpu_enable = cpu_enable_e0pd,
+	},
 #endif
 	{},
 };
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v4 2/4] arm64: Factor out checks for KASLR in KPTI code into separate function

[Mark Brown]

In preparation for integrating E0PD support with KASLR factor out the
checks for interaction between KASLR and KPTI done in boot context into
a new function kaslr_requires_kpti(), in the process clarifying the
distinction between what we do in boot context and what we do at
runtime.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/mmu.h   | 53 +++++++++++++++++++++++-----------
 arch/arm64/kernel/cpufeature.c |  2 +-
 2 files changed, 37 insertions(+), 18 deletions(-)

diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
index f217e3292919..55e285fff262 100644
--- a/arch/arm64/include/asm/mmu.h
+++ b/arch/arm64/include/asm/mmu.h
@@ -35,10 +35,37 @@ static inline bool arm64_kernel_unmapped_at_el0(void)
 	       cpus_have_const_cap(ARM64_UNMAP_KERNEL_AT_EL0);
 }
 
-static inline bool arm64_kernel_use_ng_mappings(void)
+static inline bool kaslr_requires_kpti(void)
 {
 	bool tx1_bug;
 
+	if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE))
+		return false;
+
+	/*
+	 * Systems affected by Cavium erratum 24756 are incompatible
+	 * with KPTI.
+	 */
+	if (!IS_ENABLED(CONFIG_CAVIUM_ERRATUM_27456)) {
+		tx1_bug = false;
+#ifndef MODULE
+	} else if (!static_branch_likely(&arm64_const_caps_ready)) {
+		extern const struct midr_range cavium_erratum_27456_cpus[];
+
+		tx1_bug = is_midr_in_range_list(read_cpuid_id(),
+						cavium_erratum_27456_cpus);
+#endif
+	} else {
+		tx1_bug = __cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_27456);
+	}
+	if (tx1_bug)
+		return false;
+
+	return kaslr_offset() > 0;
+}
+
+static inline bool arm64_kernel_use_ng_mappings(void)
+{
 	/* What's a kpti? Use global mappings if we don't know. */
 	if (!IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0))
 		return false;
@@ -52,29 +79,21 @@ static inline bool arm64_kernel_use_ng_mappings(void)
 	if (arm64_kernel_unmapped_at_el0())
 		return true;
 
-	if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE))
+	/*
+	 * Once we are far enough into boot for capabilities to be
+	 * ready we will have confirmed if we are using non-global
+	 * mappings so don't need to consider anything else here.
+	 */
+	if (static_branch_likely(&arm64_const_caps_ready))
 		return false;
 
 	/*
 	 * KASLR is enabled so we're going to be enabling kpti on non-broken
 	 * CPUs regardless of their susceptibility to Meltdown. Rather
 	 * than force everybody to go through the G -> nG dance later on,
-	 * just put down non-global mappings from the beginning.
+	 * just put down non-global mappings from the beginning
 	 */
-	if (!IS_ENABLED(CONFIG_CAVIUM_ERRATUM_27456)) {
-		tx1_bug = false;
-#ifndef MODULE
-	} else if (!static_branch_likely(&arm64_const_caps_ready)) {
-		extern const struct midr_range cavium_erratum_27456_cpus[];
-
-		tx1_bug = is_midr_in_range_list(read_cpuid_id(),
-						cavium_erratum_27456_cpus);
-#endif
-	} else {
-		tx1_bug = __cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_27456);
-	}
-
-	return !tx1_bug && kaslr_offset() > 0;
+	return kaslr_requires_kpti();
 }
 
 typedef void (*bp_hardening_cb_t)(void);
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 4e2009711c69..7a417034d7e8 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1008,7 +1008,7 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry,
 	}
 
 	/* Useful for KASLR robustness */
-	if (IS_ENABLED(CONFIG_RANDOMIZE_BASE) && kaslr_offset() > 0) {
+	if (kaslr_requires_kpti()) {
 		if (!__kpti_forced) {
 			str = "KASLR";
 			__kpti_forced = 1;
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v4 3/4] arm64: Don't use KPTI where we have E0PD

[Mark Brown]

Since E0PD is intended to fulfil the same role as KPTI we don't need to
use KPTI on CPUs where E0PD is available, we can rely on E0PD instead.
Change the check that forces KPTI on when KASLR is enabled to check for
E0PD before doing so, CPUs with E0PD are not expected to be affected by
meltdown so should not need to enable KPTI for other reasons.

Since E0PD is a system capability we will still enable KPTI if any of
the CPUs in the system lacks E0PD, this will rewrite any global mappings
that were established in systems where some but not all CPUs support
E0PD.  We may transiently have a mix of global and non-global mappings
while booting since we use the local CPU when deciding if KPTI will be
required prior to completing CPU enumeration but any global mappings
will be converted to non-global ones when KPTI is applied.

KPTI can still be forced on from the command line if required.

Signed-off-by: Mark Brown <broonie@kernel.org>
---
 arch/arm64/include/asm/mmu.h | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
index 55e285fff262..d61908bf4c9c 100644
--- a/arch/arm64/include/asm/mmu.h
+++ b/arch/arm64/include/asm/mmu.h
@@ -38,10 +38,21 @@ static inline bool arm64_kernel_unmapped_at_el0(void)
 static inline bool kaslr_requires_kpti(void)
 {
 	bool tx1_bug;
+	u64 ftr;
 
 	if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE))
 		return false;
 
+	/*
+	 * E0PD does a similar job to KPTI so can be used instead
+	 * where available.
+	 */
+	if (IS_ENABLED(CONFIG_ARM64_E0PD)) {
+		ftr = read_sysreg_s(SYS_ID_AA64MMFR2_EL1);
+		if ((ftr >> ID_AA64MMFR2_E0PD_SHIFT) & 0xf)
+			return false;
+	}
+
 	/*
 	 * Systems affected by Cavium erratum 24756 are incompatible
 	 * with KPTI.
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

[PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Mark Brown]

Refactor the code which checks to see if we need to use non-global
mappings to use a variable instead of checking with the CPU capabilities
each time, doing the initial check for KPTI early in boot before we
start allocating memory so we still avoid transitioning to non-global
mappings in common cases.

Since this variable always matches our decision about non-global
mappings this means we can also combine arm64_kernel_use_ng_mappings()
and arm64_unmap_kernel_at_el0() into a single function, the variable
simply stores the result and the decision code is elsewhere. We could
just have the users check the variable directly but having a function
makes it clear that these uses are read-only.

The result is that we simplify the code a bit and reduces the amount of
code executed at runtime.

Signed-off-by: Mark Brown <broonie@kernel.org>

vdfsv
---
 arch/arm64/include/asm/mmu.h          | 78 ++-------------------------
 arch/arm64/include/asm/pgtable-prot.h |  4 +-
 arch/arm64/kernel/cpufeature.c        | 42 ++++++++++++++-
 3 files changed, 46 insertions(+), 78 deletions(-)

diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h
index d61908bf4c9c..e4d862420bb4 100644
--- a/arch/arm64/include/asm/mmu.h
+++ b/arch/arm64/include/asm/mmu.h
@@ -29,82 +29,11 @@ typedef struct {
  */
 #define ASID(mm)	((mm)->context.id.counter & 0xffff)
 
-static inline bool arm64_kernel_unmapped_at_el0(void)
-{
-	return IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0) &&
-	       cpus_have_const_cap(ARM64_UNMAP_KERNEL_AT_EL0);
-}
+extern bool arm64_use_ng_mappings;
 
-static inline bool kaslr_requires_kpti(void)
-{
-	bool tx1_bug;
-	u64 ftr;
-
-	if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE))
-		return false;
-
-	/*
-	 * E0PD does a similar job to KPTI so can be used instead
-	 * where available.
-	 */
-	if (IS_ENABLED(CONFIG_ARM64_E0PD)) {
-		ftr = read_sysreg_s(SYS_ID_AA64MMFR2_EL1);
-		if ((ftr >> ID_AA64MMFR2_E0PD_SHIFT) & 0xf)
-			return false;
-	}
-
-	/*
-	 * Systems affected by Cavium erratum 24756 are incompatible
-	 * with KPTI.
-	 */
-	if (!IS_ENABLED(CONFIG_CAVIUM_ERRATUM_27456)) {
-		tx1_bug = false;
-#ifndef MODULE
-	} else if (!static_branch_likely(&arm64_const_caps_ready)) {
-		extern const struct midr_range cavium_erratum_27456_cpus[];
-
-		tx1_bug = is_midr_in_range_list(read_cpuid_id(),
-						cavium_erratum_27456_cpus);
-#endif
-	} else {
-		tx1_bug = __cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_27456);
-	}
-	if (tx1_bug)
-		return false;
-
-	return kaslr_offset() > 0;
-}
-
-static inline bool arm64_kernel_use_ng_mappings(void)
+static inline bool arm64_kernel_unmapped_at_el0(void)
 {
-	/* What's a kpti? Use global mappings if we don't know. */
-	if (!IS_ENABLED(CONFIG_UNMAP_KERNEL_AT_EL0))
-		return false;
-
-	/*
-	 * Note: this function is called before the CPU capabilities have
-	 * been configured, so our early mappings will be global. If we
-	 * later determine that kpti is required, then
-	 * kpti_install_ng_mappings() will make them non-global.
-	 */
-	if (arm64_kernel_unmapped_at_el0())
-		return true;
-
-	/*
-	 * Once we are far enough into boot for capabilities to be
-	 * ready we will have confirmed if we are using non-global
-	 * mappings so don't need to consider anything else here.
-	 */
-	if (static_branch_likely(&arm64_const_caps_ready))
-		return false;
-
-	/*
-	 * KASLR is enabled so we're going to be enabling kpti on non-broken
-	 * CPUs regardless of their susceptibility to Meltdown. Rather
-	 * than force everybody to go through the G -> nG dance later on,
-	 * just put down non-global mappings from the beginning
-	 */
-	return kaslr_requires_kpti();
+	return arm64_use_ng_mappings;
 }
 
 typedef void (*bp_hardening_cb_t)(void);
@@ -158,6 +87,7 @@ extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys,
 			       pgprot_t prot, bool page_mappings_only);
 extern void *fixmap_remap_fdt(phys_addr_t dt_phys, int *size, pgprot_t prot);
 extern void mark_linear_text_alias_ro(void);
+extern bool kaslr_requires_kpti(void);
 
 #define INIT_MM_CONTEXT(name)	\
 	.pgd = init_pg_dir,
diff --git a/arch/arm64/include/asm/pgtable-prot.h b/arch/arm64/include/asm/pgtable-prot.h
index 9a21b84536f2..eb1c6f83343d 100644
--- a/arch/arm64/include/asm/pgtable-prot.h
+++ b/arch/arm64/include/asm/pgtable-prot.h
@@ -26,8 +26,8 @@
 #define _PROT_DEFAULT		(PTE_TYPE_PAGE | PTE_AF | PTE_SHARED)
 #define _PROT_SECT_DEFAULT	(PMD_TYPE_SECT | PMD_SECT_AF | PMD_SECT_S)
 
-#define PTE_MAYBE_NG		(arm64_kernel_use_ng_mappings() ? PTE_NG : 0)
-#define PMD_MAYBE_NG		(arm64_kernel_use_ng_mappings() ? PMD_SECT_NG : 0)
+#define PTE_MAYBE_NG		(arm64_kernel_unmapped_at_el0() ? PTE_NG : 0)
+#define PMD_MAYBE_NG		(arm64_kernel_unmapped_at_el0() ? PMD_SECT_NG : 0)
 
 #define PROT_DEFAULT		(_PROT_DEFAULT | PTE_MAYBE_NG)
 #define PROT_SECT_DEFAULT	(_PROT_SECT_DEFAULT | PMD_MAYBE_NG)
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 7a417034d7e8..92058e7aaea9 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -47,6 +47,9 @@ static struct arm64_cpu_capabilities const __ro_after_init *cpu_hwcaps_ptrs[ARM6
 /* Need also bit for ARM64_CB_PATCH */
 DECLARE_BITMAP(boot_capabilities, ARM64_NPATCHABLE);
 
+bool arm64_use_ng_mappings = false;
+EXPORT_SYMBOL(arm64_use_ng_mappings);
+
 /*
  * Flag to indicate if we have computed the system wide
  * capabilities based on the boot time active CPUs. This
@@ -966,6 +969,39 @@ has_useable_cnp(const struct arm64_cpu_capabilities *entry, int scope)
 	return has_cpuid_feature(entry, scope);
 }
 
+bool kaslr_requires_kpti(void)
+{
+	bool tx1_bug;
+	u64 ftr;
+
+	if (!IS_ENABLED(CONFIG_RANDOMIZE_BASE))
+		return false;
+
+	/*
+	 * E0PD does a similar job to KPTI so can be used instead
+	 * where available.
+	 */
+	if (IS_ENABLED(CONFIG_ARM64_E0PD)) {
+		ftr = read_sysreg_s(SYS_ID_AA64MMFR2_EL1);
+		if ((ftr >> ID_AA64MMFR2_E0PD_SHIFT) & 0xf)
+			return false;
+	}
+
+	/*
+	 * Systems affected by Cavium erratum 24756 are incompatible
+	 * with KPTI.
+	 */
+	if (!IS_ENABLED(CONFIG_CAVIUM_ERRATUM_27456)) {
+		tx1_bug = false;
+	} else {
+		tx1_bug = __cpus_have_const_cap(ARM64_WORKAROUND_CAVIUM_27456);
+	}
+	if (tx1_bug)
+		return false;
+
+	return kaslr_offset() > 0;
+}
+
 static bool __meltdown_safe = true;
 static int __kpti_forced; /* 0: not forced, >0: forced on, <0: forced off */
 
@@ -1051,7 +1087,7 @@ kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)
 	 * it already or we have KASLR enabled and therefore have not
 	 * created any global mappings at all.
 	 */
-	if (kpti_applied || kaslr_offset() > 0)
+	if (kpti_applied || arm64_use_ng_mappings)
 		return;
 
 	remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
@@ -1060,8 +1096,10 @@ kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)
 	remap_fn(cpu, num_online_cpus(), __pa_symbol(swapper_pg_dir));
 	cpu_uninstall_idmap();
 
-	if (!cpu)
+	if (!cpu) {
 		kpti_applied = true;
+		arm64_use_ng_mappings = true;
+	}
 
 	return;
 }
-- 
2.20.1


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Catalin Marinas]

On Thu, Oct 24, 2019 at 10:42:07PM +0100, Mark Brown wrote:
> Refactor the code which checks to see if we need to use non-global
> mappings to use a variable instead of checking with the CPU capabilities
> each time, doing the initial check for KPTI early in boot before we
> start allocating memory so we still avoid transitioning to non-global
> mappings in common cases.
> 
> Since this variable always matches our decision about non-global
> mappings this means we can also combine arm64_kernel_use_ng_mappings()
> and arm64_unmap_kernel_at_el0() into a single function, the variable
> simply stores the result and the decision code is elsewhere. We could
> just have the users check the variable directly but having a function
> makes it clear that these uses are read-only.
> 
> The result is that we simplify the code a bit and reduces the amount of
> code executed at runtime.
> 
> Signed-off-by: Mark Brown <broonie@kernel.org>
> 
> vdfsv

Some random string here.

> @@ -158,6 +87,7 @@ extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys,
>  			       pgprot_t prot, bool page_mappings_only);
>  extern void *fixmap_remap_fdt(phys_addr_t dt_phys, int *size, pgprot_t prot);
>  extern void mark_linear_text_alias_ro(void);
> +extern bool kaslr_requires_kpti(void);

This doesn't seem to be used anywhere else, so we could remove it and
make it static.

> @@ -1051,7 +1087,7 @@ kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)
>  	 * it already or we have KASLR enabled and therefore have not
>  	 * created any global mappings at all.
>  	 */
> -	if (kpti_applied || kaslr_offset() > 0)
> +	if (kpti_applied || arm64_use_ng_mappings)
>  		return;

Can we get rid of kpti_applied? I think arm64_use_ng_mappings serves the
same purpose.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[Catalin Marinas]

On Thu, Oct 24, 2019 at 10:42:03PM +0100, Mark Brown wrote:
> This series adds support for E0PD. We enable E0PD unconditionally where
> present on systems where all the CPUs in the system support E0PD and
> change to not enabling KPTI by default on systems where we have enabled
> E0PD. It also converts the runtime checks for use of non-global mappings
> into a variable.
> 
> v4: Use a variable to store our decision about using non-global mappings
>     rather than rechecking constantly at runtime. I've added this as a
>     separate patch mainly for bisection.
> v3: Make E0PD a system wide feature.
> 
> Mark Brown (4):
>   arm64: Add initial support for E0PD
>   arm64: Factor out checks for KASLR in KPTI code into separate function
>   arm64: Don't use KPTI where we have E0PD
>   arm64: Use a variable to store non-global mappings decision

The patches look fine to me and I'm planning to queue them (with the
additional changes I mentioned on patch 4, I can do them locally).

I can still add acks/reviews in the next couple of days but I'd like to
push them into -next fairly soon to get some exposure.

Thanks.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1187 bytes --]

On Wed, Oct 30, 2019 at 12:13:08PM +0000, Catalin Marinas wrote:
> On Thu, Oct 24, 2019 at 10:42:07PM +0100, Mark Brown wrote:

> > vdfsv

> Some random string here.

Ah, must be rebase noise.

> > +extern bool kaslr_requires_kpti(void);

> This doesn't seem to be used anywhere else, so we could remove it and
> make it static.

Yes, that's bitrot - in an earlier version this was used in the early
init stuff but it isn't any more.

> > @@ -1051,7 +1087,7 @@ kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused)
> >  	 * it already or we have KASLR enabled and therefore have not
> >  	 * created any global mappings at all.
> >  	 */
> > -	if (kpti_applied || kaslr_offset() > 0)
> > +	if (kpti_applied || arm64_use_ng_mappings)
> >  		return;

> Can we get rid of kpti_applied? I think arm64_use_ng_mappings serves the
> same purpose.

Yes, I thought about that - we *can* currently remove it, I just left it
in as I felt at the time it might make it a bit clearer what was going
on so I'd see if anyone else had any thoughts.  I've dropped it.

BTW it'd be rather helpful to get patch 1 applied due to the constant
conflicts in cpufeature.h, it's safe by itself.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 292 bytes --]

On Wed, Oct 30, 2019 at 12:26:32PM +0000, Catalin Marinas wrote:

> The patches look fine to me and I'm planning to queue them (with the
> additional changes I mentioned on patch 4, I can do them locally).

Oh, thanks!  I'll skip reposting my updates for now then to cut down on
mail volume.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Catalin Marinas]

On Wed, Oct 30, 2019 at 12:41:53PM +0000, Mark Brown wrote:
> BTW it'd be rather helpful to get patch 1 applied due to the constant
> conflicts in cpufeature.h, it's safe by itself.

Since we moved to using topic for-next/* branches, usually based on
-rc3, I'd rather see patch series on top of the vanilla kernel. We can
handle the trivial conflicts.

Of course, if there is functional dependency, we can discuss which
branch to base it on but I'd avoid for-next/core as a base as that's
rather volatile.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 787 bytes --]

On Wed, Oct 30, 2019 at 02:17:32PM +0000, Catalin Marinas wrote:
> On Wed, Oct 30, 2019 at 12:41:53PM +0000, Mark Brown wrote:
> > BTW it'd be rather helpful to get patch 1 applied due to the constant
> > conflicts in cpufeature.h, it's safe by itself.

> Since we moved to using topic for-next/* branches, usually based on
> -rc3, I'd rather see patch series on top of the vanilla kernel. We can
> handle the trivial conflicts.

> Of course, if there is functional dependency, we can discuss which
> branch to base it on but I'd avoid for-next/core as a base as that's
> rather volatile.

For my working branch I usually track -next since I've been
burned repeatedly in the past by interactions with other work
that's going on, both code conflicts and unfortunate runtime
interactions.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Catalin Marinas]

On Thu, Oct 31, 2019 at 09:18:07AM +0100, Mark Brown wrote:
> On Wed, Oct 30, 2019 at 02:17:32PM +0000, Catalin Marinas wrote:
> > On Wed, Oct 30, 2019 at 12:41:53PM +0000, Mark Brown wrote:
> > > BTW it'd be rather helpful to get patch 1 applied due to the constant
> > > conflicts in cpufeature.h, it's safe by itself.
> 
> > Since we moved to using topic for-next/* branches, usually based on
> > -rc3, I'd rather see patch series on top of the vanilla kernel. We can
> > handle the trivial conflicts.
> 
> > Of course, if there is functional dependency, we can discuss which
> > branch to base it on but I'd avoid for-next/core as a base as that's
> > rather volatile.
> 
> For my working branch I usually track -next since I've been
> burned repeatedly in the past by interactions with other work
> that's going on, both code conflicts and unfortunate runtime
> interactions.

I guess it's up to you, I personally find tracking -next during
development a lot more distracting.

Anyway, if you send patches to Will or me for upstream, please rebase
(and test) on top of the vanilla kernel, otherwise we may not be able to
apply them. You can (should) do a merge with -next and flag any
conflicts or run-time issues.

-- 
Catalin

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 685 bytes --]

On Thu, Oct 31, 2019 at 10:18:19AM +0000, Catalin Marinas wrote:

> Anyway, if you send patches to Will or me for upstream, please rebase
> (and test) on top of the vanilla kernel, otherwise we may not be able to
> apply them. You can (should) do a merge with -next and flag any
> conflicts or run-time issues.

Right, that's mostly what I'm doing - I pull stuff out when I'm
preparing to send it, the -next tracking is just for ongoing
work while I'm developing things.  I had been using your tree
rather than mainline as the base because it's what most
maintainers want, they're expecting people to test with what
they've already got applied and send things they can apply
directly.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[Will Deacon]

[+John Garry]

Hi Mark,

On Thu, Oct 24, 2019 at 10:42:03PM +0100, Mark Brown wrote:
> This series adds support for E0PD. We enable E0PD unconditionally where
> present on systems where all the CPUs in the system support E0PD and
> change to not enabling KPTI by default on systems where we have enabled
> E0PD. It also converts the runtime checks for use of non-global mappings
> into a variable.

Although I welcome the simplification introduced by this patch set, I'm
worried that we might be removing a significant optimisation to KASLR
boot-time on machines with large memory. John reported issues with this
in the past, so I'm looping him in in case he has a chance to test this
(branch here [1]). Of course, I could just be missing something since
this code was really subtle to begin with.

On KASLR systems other than TX1, we should use nG mappings by default so
that we can avoid the time-consuming task of rewriting swapper during boot.
However, with your patches, I think we defer the decision to use nG mappings
to the cpufeature code which means we always end up rewriting swapper in
this scenario because of the global mappings installed early on.

Have I got that right?

Cheers,

Will

[1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/e0pd

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[John Garry]

On 31/10/2019 11:26, Will Deacon wrote:
> [+John Garry]
> 
> Hi Mark,
> 
> On Thu, Oct 24, 2019 at 10:42:03PM +0100, Mark Brown wrote:
>> This series adds support for E0PD. We enable E0PD unconditionally where
>> present on systems where all the CPUs in the system support E0PD and
>> change to not enabling KPTI by default on systems where we have enabled
>> E0PD. It also converts the runtime checks for use of non-global mappings
>> into a variable.
> 
> Although I welcome the simplification introduced by this patch set, I'm
> worried that we might be removing a significant optimisation to KASLR
> boot-time on machines with large memory. John reported issues with this
> in the past, so I'm looping him in in case he has a chance to test this
> (branch here [1]). Of course, I could just be missing something since
> this code was really subtle to begin with.

I booted our D05 and D06 boards with that branch and did not experience 
the pauses which we reported some time ago.

John

> 
> On KASLR systems other than TX1, we should use nG mappings by default so
> that we can avoid the time-consuming task of rewriting swapper during boot.
> However, with your patches, I think we defer the decision to use nG mappings
> to the cpufeature code which means we always end up rewriting swapper in
> this scenario because of the global mappings installed early on.
> 
> Have I got that right?
> 
> Cheers,
> 
> Will
> 
> [1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/e0pd
> .
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[Will Deacon]

On Thu, Oct 31, 2019 at 12:13:35PM +0000, John Garry wrote:
> On 31/10/2019 11:26, Will Deacon wrote:
> > On Thu, Oct 24, 2019 at 10:42:03PM +0100, Mark Brown wrote:
> > > This series adds support for E0PD. We enable E0PD unconditionally where
> > > present on systems where all the CPUs in the system support E0PD and
> > > change to not enabling KPTI by default on systems where we have enabled
> > > E0PD. It also converts the runtime checks for use of non-global mappings
> > > into a variable.
> > 
> > Although I welcome the simplification introduced by this patch set, I'm
> > worried that we might be removing a significant optimisation to KASLR
> > boot-time on machines with large memory. John reported issues with this
> > in the past, so I'm looping him in in case he has a chance to test this
> > (branch here [1]). Of course, I could just be missing something since
> > this code was really subtle to begin with.
> 
> I booted our D05 and D06 boards with that branch and did not experience the
> pauses which we reported some time ago.

Hmm. So I annotated the code locally and I see that we do rewrite swapper
in this case, so I'm surprised you're not seeing the performance issue
you ran into with this before. Have you got KASLR enabled and are you
passing a non-zero seed?

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 849 bytes --]

On Thu, Oct 31, 2019 at 12:42:18PM +0000, Will Deacon wrote:
> On Thu, Oct 31, 2019 at 12:13:35PM +0000, John Garry wrote:

> > I booted our D05 and D06 boards with that branch and did not experience the
> > pauses which we reported some time ago.

> Hmm. So I annotated the code locally and I see that we do rewrite swapper
> in this case, so I'm surprised you're not seeing the performance issue
> you ran into with this before. Have you got KASLR enabled and are you
> passing a non-zero seed?

Gah, sorry - I tested change which sets arm64_use_ng_mappings
early, should be sitting on my machine at home, in mm/init.c.
That will mean that there's only a slowdown in cases where we
have a mix of E0PD and non-E0PD CPUs with the boot CPU having
E0PD, hopefully no such systems have very large amounts of
memory.

I'll post it Monday at the latest.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[Will Deacon]

On Thu, Oct 31, 2019 at 02:00:18PM +0100, Mark Brown wrote:
> On Thu, Oct 31, 2019 at 12:42:18PM +0000, Will Deacon wrote:
> > On Thu, Oct 31, 2019 at 12:13:35PM +0000, John Garry wrote:
> 
> > > I booted our D05 and D06 boards with that branch and did not experience the
> > > pauses which we reported some time ago.
> 
> > Hmm. So I annotated the code locally and I see that we do rewrite swapper
> > in this case, so I'm surprised you're not seeing the performance issue
> > you ran into with this before. Have you got KASLR enabled and are you
> > passing a non-zero seed?
> 
> Gah, sorry - I tested change which sets arm64_use_ng_mappings
> early, should be sitting on my machine at home, in mm/init.c.
> That will mean that there's only a slowdown in cases where we
> have a mix of E0PD and non-E0PD CPUs with the boot CPU having
> E0PD, hopefully no such systems have very large amounts of
> memory.
> 
> I'll post it Monday at the latest.

Thanks, Mark.

Will

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[John Garry]

On 31/10/2019 12:42, Will Deacon wrote:
> On Thu, Oct 31, 2019 at 12:13:35PM +0000, John Garry wrote:
>> On 31/10/2019 11:26, Will Deacon wrote:
>>> On Thu, Oct 24, 2019 at 10:42:03PM +0100, Mark Brown wrote:
>>>> This series adds support for E0PD. We enable E0PD unconditionally where
>>>> present on systems where all the CPUs in the system support E0PD and
>>>> change to not enabling KPTI by default on systems where we have enabled
>>>> E0PD. It also converts the runtime checks for use of non-global mappings
>>>> into a variable.
>>>
>>> Although I welcome the simplification introduced by this patch set, I'm
>>> worried that we might be removing a significant optimisation to KASLR
>>> boot-time on machines with large memory. John reported issues with this
>>> in the past, so I'm looping him in in case he has a chance to test this
>>> (branch here [1]). Of course, I could just be missing something since
>>> this code was really subtle to begin with.
>>
>> I booted our D05 and D06 boards with that branch and did not experience the
>> pauses which we reported some time ago.
> 
> Hmm. So I annotated the code locally and I see that we do rewrite swapper
> in this case, so I'm surprised you're not seeing the performance issue
> you ran into with this before. Have you got KASLR enabled and are you
> passing a non-zero seed?

Just to confirm: yes and no (both boards pump out "EFI_RNG_PROTOCOL 
unavailable, no randomness supplied")

Thanks,
John

> 
> Will
> .
> 


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 0/4] E0PD support

[Ard Biesheuvel]

On Thu, 31 Oct 2019 at 15:14, John Garry <john.garry@huawei.com> wrote:
>
> On 31/10/2019 12:42, Will Deacon wrote:
> > On Thu, Oct 31, 2019 at 12:13:35PM +0000, John Garry wrote:
> >> On 31/10/2019 11:26, Will Deacon wrote:
> >>> On Thu, Oct 24, 2019 at 10:42:03PM +0100, Mark Brown wrote:
> >>>> This series adds support for E0PD. We enable E0PD unconditionally where
> >>>> present on systems where all the CPUs in the system support E0PD and
> >>>> change to not enabling KPTI by default on systems where we have enabled
> >>>> E0PD. It also converts the runtime checks for use of non-global mappings
> >>>> into a variable.
> >>>
> >>> Although I welcome the simplification introduced by this patch set, I'm
> >>> worried that we might be removing a significant optimisation to KASLR
> >>> boot-time on machines with large memory. John reported issues with this
> >>> in the past, so I'm looping him in in case he has a chance to test this
> >>> (branch here [1]). Of course, I could just be missing something since
> >>> this code was really subtle to begin with.
> >>
> >> I booted our D05 and D06 boards with that branch and did not experience the
> >> pauses which we reported some time ago.
> >
> > Hmm. So I annotated the code locally and I see that we do rewrite swapper
> > in this case, so I'm surprised you're not seeing the performance issue
> > you ran into with this before. Have you got KASLR enabled and are you
> > passing a non-zero seed?
>
> Just to confirm: yes and no (both boards pump out "EFI_RNG_PROTOCOL
> unavailable, no randomness supplied")
>

That is explained by the fact that we now use kaslr_offset() to decide
whether KASLR is enabled or not, while before, having the config
option set was sufficient to take the KASLR code path.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [PATCH v4 4/4] arm64: Use a variable to store non-global mappings decision

[Mark Brown]

[-- Attachment #1.1: Type: text/plain, Size: 1099 bytes --]

On Thu, Oct 31, 2019 at 10:18:19AM +0000, Catalin Marinas wrote:
> On Thu, Oct 31, 2019 at 09:18:07AM +0100, Mark Brown wrote:

> > For my working branch I usually track -next since I've been
> > burned repeatedly in the past by interactions with other work
> > that's going on, both code conflicts and unfortunate runtime
> > interactions.

> I guess it's up to you, I personally find tracking -next during
> development a lot more distracting.

> Anyway, if you send patches to Will or me for upstream, please rebase
> (and test) on top of the vanilla kernel, otherwise we may not be able to
> apply them. You can (should) do a merge with -next and flag any
> conflicts or run-time issues.

I should also point out here that while with my workflow I see the
issues rebasing on -next the same conflicts will also come up whenever
merging into -next for testing which happens about as often.  The core
problem is that cpucaps.h can't use an enum because the assembler
doesn't understand that so it's pretty much guaranteed to lead to
frequent conflicts, hence the desire to get such patches merged.

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel