* [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs
@ 2021-04-08 13:10 Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 1/3] arm64: cpufeature: Allow early filtering of feature override Marc Zyngier
` (3 more replies)
0 siblings, 4 replies; 10+ messages in thread
From: Marc Zyngier @ 2021-04-08 13:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Hector Martin, Arnd Bergmann, Mark Rutland, Will Deacon,
Catalin Marinas, kernel-team
This short series is a rewrite of [0] after some reviewing from
Will. It simplifies the esoteric "stay at EL2" path, and move the
feature override code where it actually belongs, allowing us to tell
the user that no, nVHE isn't a thing on these system.
The last patch rids us of CONFIG_ARM64_VHE, which has been the default
for longer than I can remember.
This allows the infamous M1 to boot (tested on a M1 Mini).
Note: the last patch will conflict with the KVM SVE series, so I'd
like to take that patch (or the series) via the KVM tree to avoid
extra moaning^Wfriction.
* From v2 [2]:
- Fixed typos and spacing issues
- Dropped the mVHE on VHE patch
- Added a patch removing the CONFIG_ARM64_VHE option altogether
- Collected Ack from Will
* From v1 [1]:
- added a comment describing the mapping various states
the override mask/val tuple can describe
- added a patch to prevent KVM from initialising, treating
the lack of CONFIG_ARM64_VHE as a mismatched boot
[0] https://lore.kernel.org/r/20210304213902.83903-2-marcan@marcan.st
[1] https://lore.kernel.org/r/20210325124721.941182-1-maz@kernel.org
[2] https://lore.kernel.org/r/20210330173947.999859-1-maz@kernel.org
Marc Zyngier (3):
arm64: cpufeature: Allow early filtering of feature override
arm64: Cope with CPUs stuck in VHE mode
arm64: Get rid of CONFIG_ARM64_VHE
.../admin-guide/kernel-parameters.txt | 3 +-
arch/arm64/Kconfig | 20 ----------
arch/arm64/include/asm/cpufeature.h | 17 ++++++++
arch/arm64/kernel/cpufeature.c | 10 +++--
arch/arm64/kernel/head.S | 39 +++++++++++++++++--
arch/arm64/kernel/hyp-stub.S | 10 ++---
arch/arm64/kernel/idreg-override.c | 26 ++++++++++++-
7 files changed, 89 insertions(+), 36 deletions(-)
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH v3 1/3] arm64: cpufeature: Allow early filtering of feature override
2021-04-08 13:10 [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Marc Zyngier
@ 2021-04-08 13:10 ` Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 2/3] arm64: Cope with CPUs stuck in VHE mode Marc Zyngier
` (2 subsequent siblings)
3 siblings, 0 replies; 10+ messages in thread
From: Marc Zyngier @ 2021-04-08 13:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Hector Martin, Arnd Bergmann, Mark Rutland, Will Deacon,
Catalin Marinas, kernel-team
Some CPUs are broken enough that some overrides need to be rejected
at the earliest opportunity. In some cases, that's right at cpu
feature override time.
Provide the necessary infrastructure to filter out overrides,
and to report such filtered out overrides to the core cpufeature code.
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/include/asm/cpufeature.h | 17 +++++++++++++++++
arch/arm64/kernel/cpufeature.c | 6 ++++++
arch/arm64/kernel/idreg-override.c | 13 +++++++++++++
3 files changed, 36 insertions(+)
diff --git a/arch/arm64/include/asm/cpufeature.h b/arch/arm64/include/asm/cpufeature.h
index 61177bac49fa..338840c00e8e 100644
--- a/arch/arm64/include/asm/cpufeature.h
+++ b/arch/arm64/include/asm/cpufeature.h
@@ -63,6 +63,23 @@ struct arm64_ftr_bits {
s64 safe_val; /* safe value for FTR_EXACT features */
};
+/*
+ * Describe the early feature override to the core override code:
+ *
+ * @val Values that are to be merged into the final
+ * sanitised value of the register. Only the bitfields
+ * set to 1 in @mask are valid
+ * @mask Mask of the features that are overridden by @val
+ *
+ * A @mask field set to full-1 indicates that the corresponding field
+ * in @val is a valid override.
+ *
+ * A @mask field set to full-0 with the corresponding @val field set
+ * to full-0 denotes that this field has no override
+ *
+ * A @mask field set to full-0 with the corresponding @val field set
+ * to full-1 denotes thath this field has an invalid override.
+ */
struct arm64_ftr_override {
u64 val;
u64 mask;
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 066030717a4c..6de15deaa912 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -809,6 +809,12 @@ static void __init init_cpu_ftr_reg(u32 sys_reg, u64 new)
reg->name,
ftrp->shift + ftrp->width - 1,
ftrp->shift, str, tmp);
+ } else if ((ftr_mask & reg->override->val) == ftr_mask) {
+ reg->override->val &= ~ftr_mask;
+ pr_warn("%s[%d:%d]: impossible override, ignored\n",
+ reg->name,
+ ftrp->shift + ftrp->width - 1,
+ ftrp->shift);
}
val = arm64_ftr_set_value(ftrp, val, ftr_new);
diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c
index 83f1c4b92095..be92fcd319a1 100644
--- a/arch/arm64/kernel/idreg-override.c
+++ b/arch/arm64/kernel/idreg-override.c
@@ -25,6 +25,7 @@ struct ftr_set_desc {
struct {
char name[FTR_DESC_FIELD_LEN];
u8 shift;
+ bool (*filter)(u64 val);
} fields[];
};
@@ -124,6 +125,18 @@ static void __init match_options(const char *cmdline)
if (find_field(cmdline, regs[i], f, &v))
continue;
+ /*
+ * If an override gets filtered out, advertise
+ * it by setting the value to 0xf, but
+ * clearing the mask... Yes, this is fragile.
+ */
+ if (regs[i]->fields[f].filter &&
+ !regs[i]->fields[f].filter(v)) {
+ regs[i]->override->val |= mask;
+ regs[i]->override->mask &= ~mask;
+ continue;
+ }
+
regs[i]->override->val &= ~mask;
regs[i]->override->val |= (v << shift) & mask;
regs[i]->override->mask |= mask;
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 2/3] arm64: Cope with CPUs stuck in VHE mode
2021-04-08 13:10 [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 1/3] arm64: cpufeature: Allow early filtering of feature override Marc Zyngier
@ 2021-04-08 13:10 ` Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE Marc Zyngier
2021-04-08 18:00 ` [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Catalin Marinas
3 siblings, 0 replies; 10+ messages in thread
From: Marc Zyngier @ 2021-04-08 13:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Hector Martin, Arnd Bergmann, Mark Rutland, Will Deacon,
Catalin Marinas, kernel-team
It seems that the CPUs part of the SoC known as Apple M1 have the
terrible habit of being stuck with HCR_EL2.E2H==1, in violation
of the architecture.
Try and work around this deplorable state of affairs by detecting
the stuck bit early and short-circuit the nVHE dance. Additional
filtering code ensures that attempts at switching to nVHE from
the command-line are also ignored.
It is still unknown whether there are many more such nuggets
to be found...
Reported-by: Hector Martin <marcan@marcan.st>
Acked-by: Will Deacon <will@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
arch/arm64/kernel/head.S | 39 +++++++++++++++++++++++++++---
arch/arm64/kernel/hyp-stub.S | 8 +++---
arch/arm64/kernel/idreg-override.c | 13 +++++++++-
3 files changed, 52 insertions(+), 8 deletions(-)
diff --git a/arch/arm64/kernel/head.S b/arch/arm64/kernel/head.S
index 840bda1869e9..96873dfa67fd 100644
--- a/arch/arm64/kernel/head.S
+++ b/arch/arm64/kernel/head.S
@@ -477,14 +477,13 @@ EXPORT_SYMBOL(kimage_vaddr)
* booted in EL1 or EL2 respectively.
*/
SYM_FUNC_START(init_kernel_el)
- mov_q x0, INIT_SCTLR_EL1_MMU_OFF
- msr sctlr_el1, x0
-
mrs x0, CurrentEL
cmp x0, #CurrentEL_EL2
b.eq init_el2
SYM_INNER_LABEL(init_el1, SYM_L_LOCAL)
+ mov_q x0, INIT_SCTLR_EL1_MMU_OFF
+ msr sctlr_el1, x0
isb
mov_q x0, INIT_PSTATE_EL1
msr spsr_el1, x0
@@ -504,9 +503,43 @@ SYM_INNER_LABEL(init_el2, SYM_L_LOCAL)
msr vbar_el2, x0
isb
+ /*
+ * Fruity CPUs seem to have HCR_EL2.E2H set to RES1,
+ * making it impossible to start in nVHE mode. Is that
+ * compliant with the architecture? Absolutely not!
+ */
+ mrs x0, hcr_el2
+ and x0, x0, #HCR_E2H
+ cbz x0, 1f
+
+ /* Switching to VHE requires a sane SCTLR_EL1 as a start */
+ mov_q x0, INIT_SCTLR_EL1_MMU_OFF
+ msr_s SYS_SCTLR_EL12, x0
+
+ /*
+ * Force an eret into a helper "function", and let it return
+ * to our original caller... This makes sure that we have
+ * initialised the basic PSTATE state.
+ */
+ mov x0, #INIT_PSTATE_EL2
+ msr spsr_el1, x0
+ adr x0, __cpu_stick_to_vhe
+ msr elr_el1, x0
+ eret
+
+1:
+ mov_q x0, INIT_SCTLR_EL1_MMU_OFF
+ msr sctlr_el1, x0
+
msr elr_el2, lr
mov w0, #BOOT_CPU_MODE_EL2
eret
+
+__cpu_stick_to_vhe:
+ mov x0, #HVC_VHE_RESTART
+ hvc #0
+ mov x0, #BOOT_CPU_MODE_EL2
+ ret
SYM_FUNC_END(init_kernel_el)
/*
diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
index 5eccbd62fec8..9c9b4d8fc395 100644
--- a/arch/arm64/kernel/hyp-stub.S
+++ b/arch/arm64/kernel/hyp-stub.S
@@ -27,12 +27,12 @@ SYM_CODE_START(__hyp_stub_vectors)
ventry el2_fiq_invalid // FIQ EL2t
ventry el2_error_invalid // Error EL2t
- ventry el2_sync_invalid // Synchronous EL2h
+ ventry elx_sync // Synchronous EL2h
ventry el2_irq_invalid // IRQ EL2h
ventry el2_fiq_invalid // FIQ EL2h
ventry el2_error_invalid // Error EL2h
- ventry el1_sync // Synchronous 64-bit EL1
+ ventry elx_sync // Synchronous 64-bit EL1
ventry el1_irq_invalid // IRQ 64-bit EL1
ventry el1_fiq_invalid // FIQ 64-bit EL1
ventry el1_error_invalid // Error 64-bit EL1
@@ -45,7 +45,7 @@ SYM_CODE_END(__hyp_stub_vectors)
.align 11
-SYM_CODE_START_LOCAL(el1_sync)
+SYM_CODE_START_LOCAL(elx_sync)
cmp x0, #HVC_SET_VECTORS
b.ne 1f
msr vbar_el2, x1
@@ -71,7 +71,7 @@ SYM_CODE_START_LOCAL(el1_sync)
9: mov x0, xzr
eret
-SYM_CODE_END(el1_sync)
+SYM_CODE_END(elx_sync)
// nVHE? No way! Give me the real thing!
SYM_CODE_START_LOCAL(mutate_to_vhe)
diff --git a/arch/arm64/kernel/idreg-override.c b/arch/arm64/kernel/idreg-override.c
index be92fcd319a1..e628c8ce1ffe 100644
--- a/arch/arm64/kernel/idreg-override.c
+++ b/arch/arm64/kernel/idreg-override.c
@@ -29,11 +29,22 @@ struct ftr_set_desc {
} fields[];
};
+static bool __init mmfr1_vh_filter(u64 val)
+{
+ /*
+ * If we ever reach this point while running VHE, we're
+ * guaranteed to be on one of these funky, VHE-stuck CPUs. If
+ * the user was trying to force nVHE on us, proceed with
+ * attitude adjustment.
+ */
+ return !(is_kernel_in_hyp_mode() && val == 0);
+}
+
static const struct ftr_set_desc mmfr1 __initconst = {
.name = "id_aa64mmfr1",
.override = &id_aa64mmfr1_override,
.fields = {
- { "vh", ID_AA64MMFR1_VHE_SHIFT },
+ { "vh", ID_AA64MMFR1_VHE_SHIFT, mmfr1_vh_filter },
{}
},
};
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE
2021-04-08 13:10 [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 1/3] arm64: cpufeature: Allow early filtering of feature override Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 2/3] arm64: Cope with CPUs stuck in VHE mode Marc Zyngier
@ 2021-04-08 13:10 ` Marc Zyngier
2021-04-08 16:59 ` Will Deacon
2021-04-08 21:47 ` Arnd Bergmann
2021-04-08 18:00 ` [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Catalin Marinas
3 siblings, 2 replies; 10+ messages in thread
From: Marc Zyngier @ 2021-04-08 13:10 UTC (permalink / raw)
To: linux-arm-kernel
Cc: Hector Martin, Arnd Bergmann, Mark Rutland, Will Deacon,
Catalin Marinas, kernel-team
CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago),
and has been enabled by default for almost all that time.
Given that newer systems that are VHE capable are finally becoming
available, and that some systems are even incapable of not running VHE,
drop the configuration altogether.
Anyone willing to stick to non-VHE on VHE hardware for obscure
reasons should use the 'kvm-arm.mode=nvhe' command-line option.
Suggested-by: Will Deacon <will@kernel.org>
Signed-off-by: Marc Zyngier <maz@kernel.org>
---
.../admin-guide/kernel-parameters.txt | 3 +--
arch/arm64/Kconfig | 20 -------------------
arch/arm64/kernel/cpufeature.c | 4 ----
arch/arm64/kernel/hyp-stub.S | 2 --
4 files changed, 1 insertion(+), 28 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 04545725f187..18f8bb3024f4 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -2279,8 +2279,7 @@
state is kept private from the host.
Not valid if the kernel is running in EL2.
- Defaults to VHE/nVHE based on hardware support and
- the value of CONFIG_ARM64_VHE.
+ Defaults to VHE/nVHE based on hardware support.
kvm-arm.vgic_v3_group0_trap=
[KVM,ARM] Trap guest accesses to GICv3 group-0
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 5656e7aacd69..e9fbb0b45793 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -1416,19 +1416,6 @@ config ARM64_USE_LSE_ATOMICS
built with binutils >= 2.25 in order for the new instructions
to be used.
-config ARM64_VHE
- bool "Enable support for Virtualization Host Extensions (VHE)"
- default y
- help
- Virtualization Host Extensions (VHE) allow the kernel to run
- directly at EL2 (instead of EL1) on processors that support
- it. This leads to better performance for KVM, as they reduce
- the cost of the world switch.
-
- Selecting this option allows the VHE feature to be detected
- at runtime, and does not affect processors that do not
- implement this feature.
-
endmenu
menu "ARMv8.2 architectural features"
@@ -1684,7 +1671,6 @@ endmenu
config ARM64_SVE
bool "ARM Scalable Vector Extension support"
default y
- depends on !KVM || ARM64_VHE
help
The Scalable Vector Extension (SVE) is an extension to the AArch64
execution state which complements and extends the SIMD functionality
@@ -1713,12 +1699,6 @@ config ARM64_SVE
booting the kernel. If unsure and you are not observing these
symptoms, you should assume that it is safe to say Y.
- CPUs that support SVE are architecturally required to support the
- Virtualization Host Extensions (VHE), so the kernel makes no
- provision for supporting SVE alongside KVM without VHE enabled.
- Thus, you will need to enable CONFIG_ARM64_VHE if you want to support
- KVM in the same kernel image.
-
config ARM64_MODULE_PLTS
bool "Use PLTs to allow module memory to spill over into vmalloc area"
depends on MODULES
diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c
index 6de15deaa912..fa6023ac3548 100644
--- a/arch/arm64/kernel/cpufeature.c
+++ b/arch/arm64/kernel/cpufeature.c
@@ -1623,7 +1623,6 @@ int get_cpu_with_amu_feat(void)
}
#endif
-#ifdef CONFIG_ARM64_VHE
static bool runs_at_el2(const struct arm64_cpu_capabilities *entry, int __unused)
{
return is_kernel_in_hyp_mode();
@@ -1642,7 +1641,6 @@ static void cpu_copy_el2regs(const struct arm64_cpu_capabilities *__unused)
if (!alternative_is_applied(ARM64_HAS_VIRT_HOST_EXTN))
write_sysreg(read_sysreg(tpidr_el1), tpidr_el2);
}
-#endif
static void cpu_has_fwb(const struct arm64_cpu_capabilities *__unused)
{
@@ -1845,7 +1843,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
.matches = has_no_hw_prefetch,
},
-#ifdef CONFIG_ARM64_VHE
{
.desc = "Virtualization Host Extensions",
.capability = ARM64_HAS_VIRT_HOST_EXTN,
@@ -1853,7 +1850,6 @@ static const struct arm64_cpu_capabilities arm64_features[] = {
.matches = runs_at_el2,
.cpu_enable = cpu_copy_el2regs,
},
-#endif /* CONFIG_ARM64_VHE */
{
.desc = "32-bit EL0 Support",
.capability = ARM64_HAS_32BIT_EL0,
diff --git a/arch/arm64/kernel/hyp-stub.S b/arch/arm64/kernel/hyp-stub.S
index 9c9b4d8fc395..74ad3db061d1 100644
--- a/arch/arm64/kernel/hyp-stub.S
+++ b/arch/arm64/kernel/hyp-stub.S
@@ -224,7 +224,6 @@ SYM_FUNC_END(__hyp_reset_vectors)
* Entry point to switch to VHE if deemed capable
*/
SYM_FUNC_START(switch_to_vhe)
-#ifdef CONFIG_ARM64_VHE
// Need to have booted at EL2
adr_l x1, __boot_cpu_mode
ldr w0, [x1]
@@ -240,6 +239,5 @@ SYM_FUNC_START(switch_to_vhe)
mov x0, #HVC_VHE_RESTART
hvc #0
1:
-#endif
ret
SYM_FUNC_END(switch_to_vhe)
--
2.29.2
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE
2021-04-08 13:10 ` [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE Marc Zyngier
@ 2021-04-08 16:59 ` Will Deacon
2021-04-08 17:24 ` Marc Zyngier
2021-04-08 21:47 ` Arnd Bergmann
1 sibling, 1 reply; 10+ messages in thread
From: Will Deacon @ 2021-04-08 16:59 UTC (permalink / raw)
To: Marc Zyngier
Cc: linux-arm-kernel, Hector Martin, Arnd Bergmann, Mark Rutland,
Catalin Marinas, kernel-team
On Thu, Apr 08, 2021 at 02:10:10PM +0100, Marc Zyngier wrote:
> CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago),
> and has been enabled by default for almost all that time.
>
> Given that newer systems that are VHE capable are finally becoming
> available, and that some systems are even incapable of not running VHE,
> drop the configuration altogether.
>
> Anyone willing to stick to non-VHE on VHE hardware for obscure
> reasons should use the 'kvm-arm.mode=nvhe' command-line option.
>
> Suggested-by: Will Deacon <will@kernel.org>
> Signed-off-by: Marc Zyngier <maz@kernel.org>
> ---
> .../admin-guide/kernel-parameters.txt | 3 +--
> arch/arm64/Kconfig | 20 -------------------
> arch/arm64/kernel/cpufeature.c | 4 ----
> arch/arm64/kernel/hyp-stub.S | 2 --
> 4 files changed, 1 insertion(+), 28 deletions(-)
Huh, I was really expecting to see ARM64_VHE crop up in some Makefiles, but
it doesn't seem to! So:
Acked-by: Will Deacon <will@kernel.org>
Will
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE
2021-04-08 16:59 ` Will Deacon
@ 2021-04-08 17:24 ` Marc Zyngier
0 siblings, 0 replies; 10+ messages in thread
From: Marc Zyngier @ 2021-04-08 17:24 UTC (permalink / raw)
To: Will Deacon
Cc: linux-arm-kernel, Hector Martin, Arnd Bergmann, Mark Rutland,
Catalin Marinas, kernel-team
On Thu, 08 Apr 2021 17:59:14 +0100,
Will Deacon <will@kernel.org> wrote:
>
> On Thu, Apr 08, 2021 at 02:10:10PM +0100, Marc Zyngier wrote:
> > CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago),
> > and has been enabled by default for almost all that time.
> >
> > Given that newer systems that are VHE capable are finally becoming
> > available, and that some systems are even incapable of not running VHE,
> > drop the configuration altogether.
> >
> > Anyone willing to stick to non-VHE on VHE hardware for obscure
> > reasons should use the 'kvm-arm.mode=nvhe' command-line option.
> >
> > Suggested-by: Will Deacon <will@kernel.org>
> > Signed-off-by: Marc Zyngier <maz@kernel.org>
> > ---
> > .../admin-guide/kernel-parameters.txt | 3 +--
> > arch/arm64/Kconfig | 20 -------------------
> > arch/arm64/kernel/cpufeature.c | 4 ----
> > arch/arm64/kernel/hyp-stub.S | 2 --
> > 4 files changed, 1 insertion(+), 28 deletions(-)
>
> Huh, I was really expecting to see ARM64_VHE crop up in some Makefiles, but
> it doesn't seem to! So:
It was a design decision from the very beginning that all the VHE code
would always be compiled in to avoid bitrot. It probably helped
weeding out some early bugs.
> Acked-by: Will Deacon <will@kernel.org>
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs
2021-04-08 13:10 [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Marc Zyngier
` (2 preceding siblings ...)
2021-04-08 13:10 ` [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE Marc Zyngier
@ 2021-04-08 18:00 ` Catalin Marinas
2021-04-08 18:08 ` Marc Zyngier
3 siblings, 1 reply; 10+ messages in thread
From: Catalin Marinas @ 2021-04-08 18:00 UTC (permalink / raw)
To: Marc Zyngier, linux-arm-kernel
Cc: Will Deacon, Mark Rutland, kernel-team, Arnd Bergmann, Hector Martin
On Thu, 8 Apr 2021 14:10:07 +0100, Marc Zyngier wrote:
> This short series is a rewrite of [0] after some reviewing from
> Will. It simplifies the esoteric "stay at EL2" path, and move the
> feature override code where it actually belongs, allowing us to tell
> the user that no, nVHE isn't a thing on these system.
>
> The last patch rids us of CONFIG_ARM64_VHE, which has been the default
> for longer than I can remember.
>
> [...]
Applied to arm64 (for-next/vhe-only), thanks!
[1/3] arm64: cpufeature: Allow early filtering of feature override
https://git.kernel.org/arm64/c/cac642c12a80
[2/3] arm64: Cope with CPUs stuck in VHE mode
https://git.kernel.org/arm64/c/31a32b49b80f
[3/3] arm64: Get rid of CONFIG_ARM64_VHE
https://git.kernel.org/arm64/c/2d726d0db6ac
--
Catalin
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs
2021-04-08 18:00 ` [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Catalin Marinas
@ 2021-04-08 18:08 ` Marc Zyngier
0 siblings, 0 replies; 10+ messages in thread
From: Marc Zyngier @ 2021-04-08 18:08 UTC (permalink / raw)
To: Catalin Marinas
Cc: linux-arm-kernel, Will Deacon, Mark Rutland, kernel-team,
Arnd Bergmann, Hector Martin
On 2021-04-08 19:00, Catalin Marinas wrote:
> On Thu, 8 Apr 2021 14:10:07 +0100, Marc Zyngier wrote:
>> This short series is a rewrite of [0] after some reviewing from
>> Will. It simplifies the esoteric "stay at EL2" path, and move the
>> feature override code where it actually belongs, allowing us to tell
>> the user that no, nVHE isn't a thing on these system.
>>
>> The last patch rids us of CONFIG_ARM64_VHE, which has been the default
>> for longer than I can remember.
>>
>> [...]
>
> Applied to arm64 (for-next/vhe-only), thanks!
>
> [1/3] arm64: cpufeature: Allow early filtering of feature override
> https://git.kernel.org/arm64/c/cac642c12a80
> [2/3] arm64: Cope with CPUs stuck in VHE mode
> https://git.kernel.org/arm64/c/31a32b49b80f
> [3/3] arm64: Get rid of CONFIG_ARM64_VHE
> https://git.kernel.org/arm64/c/2d726d0db6ac
Thanks Catalin.
In order to avoid conflicts with my SVE branch, I've taken this branch
into the kvmarm tree.
Please let me know if you decide to drop it for any reason, and I'll do
the same on my side.
Cheers,
M.
--
Jazz is not dead. It just smells funny...
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE
2021-04-08 13:10 ` [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE Marc Zyngier
2021-04-08 16:59 ` Will Deacon
@ 2021-04-08 21:47 ` Arnd Bergmann
2021-04-09 8:28 ` Marc Zyngier
1 sibling, 1 reply; 10+ messages in thread
From: Arnd Bergmann @ 2021-04-08 21:47 UTC (permalink / raw)
To: Marc Zyngier
Cc: Linux ARM, Hector Martin, Mark Rutland, Will Deacon,
Catalin Marinas, Android Kernel Team
On Thu, Apr 8, 2021 at 3:10 PM Marc Zyngier <maz@kernel.org> wrote:
>
> CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago),
> and has been enabled by default for almost all that time.
>
> Given that newer systems that are VHE capable are finally becoming
> available, and that some systems are even incapable of not running VHE,
> drop the configuration altogether.
>
> Anyone willing to stick to non-VHE on VHE hardware for obscure
> reasons should use the 'kvm-arm.mode=nvhe' command-line option.
Have you considered adding options to do the reverse logic for this
and other features, such as making support for the old non-VHE
optional at compile time?
I understand that so far the rule is (almost) always that an arm64 kernel
should run on any Armv8.0-A or higher system regardless of configuration,
but the now announced Armv9.0-A definition might be the chance to
introduce the concept of a minimum level the way we do on other
architectures (e.g. armv6/v6k/v7 or k8/pentium4/core2/atom/generic).
The way I can see this working would be to have a single user-visible
option that controls whether the kernel supports only Armv9.0-A/Armv8.5-A
and assumes all mandatory features of that are present, or it remains
as before and supports all implementations back to the first v8.
This would help eliminate the runtime detection for not just VHE but
also LSE, LPA, PAN, etc. Not sure how significant the cost of any of
those are in terms of runtime performance and/or code size, but it
would feel nice to be able to build a kernel that can actually rely
on sane hardware features even if it will take a few more years before
that hardware becomes common enough to actually get some distros
ship a kernel that requires v8.5/v9.0.
Arnd
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE
2021-04-08 21:47 ` Arnd Bergmann
@ 2021-04-09 8:28 ` Marc Zyngier
0 siblings, 0 replies; 10+ messages in thread
From: Marc Zyngier @ 2021-04-09 8:28 UTC (permalink / raw)
To: Arnd Bergmann
Cc: Linux ARM, Hector Martin, Mark Rutland, Will Deacon,
Catalin Marinas, Android Kernel Team
On Thu, 08 Apr 2021 22:47:38 +0100,
Arnd Bergmann <arnd@kernel.org> wrote:
>
> On Thu, Apr 8, 2021 at 3:10 PM Marc Zyngier <maz@kernel.org> wrote:
> >
> > CONFIG_ARM64_VHE was introduced with ARMv8.1 (some 7 years ago),
> > and has been enabled by default for almost all that time.
> >
> > Given that newer systems that are VHE capable are finally becoming
> > available, and that some systems are even incapable of not running VHE,
> > drop the configuration altogether.
> >
> > Anyone willing to stick to non-VHE on VHE hardware for obscure
> > reasons should use the 'kvm-arm.mode=nvhe' command-line option.
>
> Have you considered adding options to do the reverse logic for this
> and other features, such as making support for the old non-VHE
> optional at compile time?
This nVHE code is exclusively limited to KVM, because the whole point
of VHE is that the EL1 kernel can run at EL2 completely unchanged.
Furthermore, nVHE has properties that VHE cannot deliver, such as
memory isolation between host and guests.
So no, I wouldn't consider make it optional at compile time. Instead,
I would consider jettisoning the nVHE EL2 code after init to save a
handful of pages when running on a VHE-capable system in VHE mode.
>
> I understand that so far the rule is (almost) always that an arm64 kernel
> should run on any Armv8.0-A or higher system regardless of configuration,
> but the now announced Armv9.0-A definition might be the chance to
> introduce the concept of a minimum level the way we do on other
> architectures (e.g. armv6/v6k/v7 or k8/pentium4/core2/atom/generic).
>
> The way I can see this working would be to have a single user-visible
> option that controls whether the kernel supports only Armv9.0-A/Armv8.5-A
> and assumes all mandatory features of that are present, or it remains
> as before and supports all implementations back to the first v8.
The view that v8.5/v9 is a monolithic setup with a set of mandatory
option is unfortunately disconnected from reality. Reality is that an
implementer picks and chooses whatever feature set they want
irrespective of what the architecture says, and the architecture
revision is nothing but an index into the documentation.
Features that were mandatory often become optional, and/or are further
made unimplementable due to some other extension of the architecture.
We also have the extremely common case where a feature that is usable
on a host isn't virtualisable, meaning that the kernel gets exposed
different feature sets depending on whether it runs bare-metal or not,
which puts another nail on the "per-revision feature set" coffin.
The ARM architecture is effectively a mix-and-match system, where you
choose the feature set you want for a single vertically integrated
application, and not a nice linear progression of features that land
in bulk.
> This would help eliminate the runtime detection for not just VHE but
> also LSE, LPA, PAN, etc. Not sure how significant the cost of any of
> those are in terms of runtime performance and/or code size, but it
> would feel nice to be able to build a kernel that can actually rely
> on sane hardware features even if it will take a few more years before
> that hardware becomes common enough to actually get some distros
> ship a kernel that requires v8.5/v9.0.
We went there with 32bit because we were forced to: different ISAs,
different page table formats. I would rather keep a single kernel
until we get to that point of divergence *or* that we can demonstrate
such a significant performance/maintenance improvement that it'd be
silly not to do it.
The published architecture does not hint at such changes yet, nor has
performance overhead been reported due to "feature bloat". The current
feature discovery and runtime patching seems adequate and saves us
from the bitrot effect that multiple configurations inevitably would
trigger.
So at the moment, I am strongly in favor of keeping the current
"single kernel" approach, and even drop more of the existing
configuration symbols (though some have ugly toolchain constraints).
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2021-04-09 8:30 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-04-08 13:10 [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 1/3] arm64: cpufeature: Allow early filtering of feature override Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 2/3] arm64: Cope with CPUs stuck in VHE mode Marc Zyngier
2021-04-08 13:10 ` [PATCH v3 3/3] arm64: Get rid of CONFIG_ARM64_VHE Marc Zyngier
2021-04-08 16:59 ` Will Deacon
2021-04-08 17:24 ` Marc Zyngier
2021-04-08 21:47 ` Arnd Bergmann
2021-04-09 8:28 ` Marc Zyngier
2021-04-08 18:00 ` [PATCH v3 0/3] arm64: Dealing with VHE-only CPUs Catalin Marinas
2021-04-08 18:08 ` Marc Zyngier
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).