[PATCH v1] KVM: PPC: Book3S HV: Prevent POWER7/8 TLB flush flushing SLB

[PATCH v1] KVM: PPC: Book3S HV: Prevent POWER7/8 TLB flush flushing SLB

[Nicholas Piggin]

The POWER9 ERAT flush instruction is a SLBIA with IH=7, which is a
reserved value on POWER7/8. On POWER8 this invalidates the SLB entries
above index 0, similarly to SLBIA IH=0.

If the SLB entries are invalidated, and then the guest is bypassed, the
host SLB does not get re-loaded, so the bolted entries above 0 will be
lost. This can result in kernel stack access causing a SLB fault.

Kernel stack access causing a SLB fault was responsible for the infamous
mega bug (search "Fix SLB reload bug"). Although since commit
48e7b7695745 ("powerpc/64s/hash: Convert SLB miss handlers to C") that
starts using the kernel stack in the SLB miss handler, it might only
result in an infinite loop of SLB faults. In any case it's a bug.

Fix this by only executing the instruction on >= POWER9 where IH=7 is
defined not to invalidate the SLB. POWER7/8 don't require this ERAT
flush.

Fixes: 5008711259201 ("KVM: PPC: Book3S HV: Invalidate ERAT when flushing guest TLB entries")
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 arch/powerpc/kvm/book3s_hv_builtin.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
index fcf4760a3a0e..70b7a8f97153 100644
--- a/arch/powerpc/kvm/book3s_hv_builtin.c
+++ b/arch/powerpc/kvm/book3s_hv_builtin.c
@@ -695,6 +695,7 @@ static void flush_guest_tlb(struct kvm *kvm)
 				       "r" (0) : "memory");
 		}
 		asm volatile("ptesync": : :"memory");
+		// POWER9 congruence-class TLBIEL leaves ERAT. Flush it now.
 		asm volatile(PPC_RADIX_INVALIDATE_ERAT_GUEST : : :"memory");
 	} else {
 		for (set = 0; set < kvm->arch.tlb_sets; ++set) {
@@ -705,7 +706,9 @@ static void flush_guest_tlb(struct kvm *kvm)
 			rb += PPC_BIT(51);	/* increment set number */
 		}
 		asm volatile("ptesync": : :"memory");
-		asm volatile(PPC_ISA_3_0_INVALIDATE_ERAT : : :"memory");
+		// POWER9 congruence-class TLBIEL leaves ERAT. Flush it now.
+		if (cpu_has_feature(CPU_FTR_ARCH_300))
+			asm volatile(PPC_ISA_3_0_INVALIDATE_ERAT : : :"memory");
 	}
 }
 
-- 
2.23.0

Re: [PATCH v1] KVM: PPC: Book3S HV: Prevent POWER7/8 TLB flush flushing SLB

[Fabiano Rosas]

Nicholas Piggin <npiggin@gmail.com> writes:

> The POWER9 ERAT flush instruction is a SLBIA with IH=7, which is a
> reserved value on POWER7/8. On POWER8 this invalidates the SLB entries
> above index 0, similarly to SLBIA IH=0.
>
> If the SLB entries are invalidated, and then the guest is bypassed, the
> host SLB does not get re-loaded, so the bolted entries above 0 will be
> lost. This can result in kernel stack access causing a SLB fault.
>
> Kernel stack access causing a SLB fault was responsible for the infamous
> mega bug (search "Fix SLB reload bug"). Although since commit
> 48e7b7695745 ("powerpc/64s/hash: Convert SLB miss handlers to C") that
> starts using the kernel stack in the SLB miss handler, it might only
> result in an infinite loop of SLB faults. In any case it's a bug.
>
> Fix this by only executing the instruction on >= POWER9 where IH=7 is
> defined not to invalidate the SLB. POWER7/8 don't require this ERAT
> flush.
>
> Fixes: 5008711259201 ("KVM: PPC: Book3S HV: Invalidate ERAT when flushing guest TLB entries")
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>

Reviewed-by: Fabiano Rosas <farosas@linux.ibm.com>

> ---
>  arch/powerpc/kvm/book3s_hv_builtin.c | 5 ++++-
>  1 file changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/arch/powerpc/kvm/book3s_hv_builtin.c b/arch/powerpc/kvm/book3s_hv_builtin.c
> index fcf4760a3a0e..70b7a8f97153 100644
> --- a/arch/powerpc/kvm/book3s_hv_builtin.c
> +++ b/arch/powerpc/kvm/book3s_hv_builtin.c
> @@ -695,6 +695,7 @@ static void flush_guest_tlb(struct kvm *kvm)
>  				       "r" (0) : "memory");
>  		}
>  		asm volatile("ptesync": : :"memory");
> +		// POWER9 congruence-class TLBIEL leaves ERAT. Flush it now.
>  		asm volatile(PPC_RADIX_INVALIDATE_ERAT_GUEST : : :"memory");
>  	} else {
>  		for (set = 0; set < kvm->arch.tlb_sets; ++set) {
> @@ -705,7 +706,9 @@ static void flush_guest_tlb(struct kvm *kvm)
>  			rb += PPC_BIT(51);	/* increment set number */
>  		}
>  		asm volatile("ptesync": : :"memory");
> -		asm volatile(PPC_ISA_3_0_INVALIDATE_ERAT : : :"memory");
> +		// POWER9 congruence-class TLBIEL leaves ERAT. Flush it now.
> +		if (cpu_has_feature(CPU_FTR_ARCH_300))
> +			asm volatile(PPC_ISA_3_0_INVALIDATE_ERAT : : :"memory");
>  	}
>  }

Re: [PATCH v1] KVM: PPC: Book3S HV: Prevent POWER7/8 TLB flush flushing SLB

[Michael Ellerman]

On Fri, 19 Nov 2021 13:16:27 +1000, Nicholas Piggin wrote:
> The POWER9 ERAT flush instruction is a SLBIA with IH=7, which is a
> reserved value on POWER7/8. On POWER8 this invalidates the SLB entries
> above index 0, similarly to SLBIA IH=0.
> 
> If the SLB entries are invalidated, and then the guest is bypassed, the
> host SLB does not get re-loaded, so the bolted entries above 0 will be
> lost. This can result in kernel stack access causing a SLB fault.
> 
> [...]

Applied to powerpc/fixes.

[1/1] KVM: PPC: Book3S HV: Prevent POWER7/8 TLB flush flushing SLB
      https://git.kernel.org/powerpc/c/cf0b0e3712f7af90006f8317ff27278094c2c128

cheers