* [PATCH 0/2] ERAT invalidation fixes @ 2018-08-27 3:03 Nicholas Piggin 2018-08-27 3:03 ` [PATCH 1/2] powerpc/64s/hash: do not use PPC_INVALIDATE_ERAT on CPUs before POWER9 Nicholas Piggin 2018-08-27 3:03 ` [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation Nicholas Piggin 0 siblings, 2 replies; 7+ messages in thread From: Nicholas Piggin @ 2018-08-27 3:03 UTC (permalink / raw) To: linuxppc-dev; +Cc: Nicholas Piggin First one is a real bug-fix for POWER8 and earlier CPUs, second I'm not exactly sure how POWER9's ERAT behaves in this situation but it seems to match what we do in other places. Thanks, Nick Nicholas Piggin (2): powerpc/64s/hash: do not use PPC_INVALIDATE_ERAT on CPUs before POWER9 powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation arch/powerpc/kernel/mce_power.c | 7 +++++++ arch/powerpc/mm/hash_native_64.c | 4 ++-- arch/powerpc/mm/tlb-radix.c | 1 + 3 files changed, 10 insertions(+), 2 deletions(-) -- 2.18.0 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 1/2] powerpc/64s/hash: do not use PPC_INVALIDATE_ERAT on CPUs before POWER9 2018-08-27 3:03 [PATCH 0/2] ERAT invalidation fixes Nicholas Piggin @ 2018-08-27 3:03 ` Nicholas Piggin 2018-10-15 4:00 ` [1/2] " Michael Ellerman 2018-08-27 3:03 ` [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation Nicholas Piggin 1 sibling, 1 reply; 7+ messages in thread From: Nicholas Piggin @ 2018-08-27 3:03 UTC (permalink / raw) To: linuxppc-dev; +Cc: Nicholas Piggin PPC_INVALIDATE_ERAT is slbia IH=7 which is a new variant introduced with POWER9, and the result is undefined on earlier CPUs. Commits 7b9f71f974 ("powerpc/64s: POWER9 machine check handler") and d4748276ae ("powerpc/64s: Improve local TLB flush for boot and MCE on POWER9") caused POWER7/8 code to use this instruction. Remove it. An ERAT flush can be made by invalidatig the SLB, but before POWER9 that requires a flush and rebolt. Fixes: 7b9f71f974 ("powerpc/64s: POWER9 machine check handler") Fixes: d4748276ae ("powerpc/64s: Improve local TLB flush for boot and MCE on POWER9") Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- arch/powerpc/kernel/mce_power.c | 7 +++++++ arch/powerpc/mm/hash_native_64.c | 4 ++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/kernel/mce_power.c b/arch/powerpc/kernel/mce_power.c index 3497c8329c1d..3022d67f0c48 100644 --- a/arch/powerpc/kernel/mce_power.c +++ b/arch/powerpc/kernel/mce_power.c @@ -89,6 +89,13 @@ static void flush_and_reload_slb(void) static void flush_erat(void) { +#ifdef CONFIG_PPC_BOOK3S_64 + if (!early_cpu_has_feature(CPU_FTR_ARCH_300)) { + flush_and_reload_slb(); + return; + } +#endif + /* PPC_INVALIDATE_ERAT can only be used on ISA v3 and newer */ asm volatile(PPC_INVALIDATE_ERAT : : :"memory"); } diff --git a/arch/powerpc/mm/hash_native_64.c b/arch/powerpc/mm/hash_native_64.c index 729f02df8290..aaa28fd918fe 100644 --- a/arch/powerpc/mm/hash_native_64.c +++ b/arch/powerpc/mm/hash_native_64.c @@ -115,6 +115,8 @@ static void tlbiel_all_isa300(unsigned int num_sets, unsigned int is) tlbiel_hash_set_isa300(0, is, 0, 2, 1); asm volatile("ptesync": : :"memory"); + + asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory"); } void hash__tlbiel_all(unsigned int action) @@ -140,8 +142,6 @@ void hash__tlbiel_all(unsigned int action) tlbiel_all_isa206(POWER7_TLB_SETS, is); else WARN(1, "%s called on pre-POWER7 CPU\n", __func__); - - asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory"); } static inline unsigned long ___tlbie(unsigned long vpn, int psize, -- 2.18.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [1/2] powerpc/64s/hash: do not use PPC_INVALIDATE_ERAT on CPUs before POWER9 2018-08-27 3:03 ` [PATCH 1/2] powerpc/64s/hash: do not use PPC_INVALIDATE_ERAT on CPUs before POWER9 Nicholas Piggin @ 2018-10-15 4:00 ` Michael Ellerman 0 siblings, 0 replies; 7+ messages in thread From: Michael Ellerman @ 2018-10-15 4:00 UTC (permalink / raw) To: Nicholas Piggin, linuxppc-dev; +Cc: Nicholas Piggin On Mon, 2018-08-27 at 03:03:01 UTC, Nicholas Piggin wrote: > PPC_INVALIDATE_ERAT is slbia IH=7 which is a new variant introduced > with POWER9, and the result is undefined on earlier CPUs. > > Commits 7b9f71f974 ("powerpc/64s: POWER9 machine check handler") and > d4748276ae ("powerpc/64s: Improve local TLB flush for boot and MCE on > POWER9") caused POWER7/8 code to use this instruction. Remove it. An > ERAT flush can be made by invalidatig the SLB, but before POWER9 that > requires a flush and rebolt. > > Fixes: 7b9f71f974 ("powerpc/64s: POWER9 machine check handler") > Fixes: d4748276ae ("powerpc/64s: Improve local TLB flush for boot and > MCE on POWER9") > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> Series applied to powerpc next, thanks. https://git.kernel.org/powerpc/c/bc276ecba132caccb1fda5863a652c cheers ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation 2018-08-27 3:03 [PATCH 0/2] ERAT invalidation fixes Nicholas Piggin 2018-08-27 3:03 ` [PATCH 1/2] powerpc/64s/hash: do not use PPC_INVALIDATE_ERAT on CPUs before POWER9 Nicholas Piggin @ 2018-08-27 3:03 ` Nicholas Piggin 2018-08-27 8:16 ` Benjamin Herrenschmidt 1 sibling, 1 reply; 7+ messages in thread From: Nicholas Piggin @ 2018-08-27 3:03 UTC (permalink / raw) To: linuxppc-dev; +Cc: Nicholas Piggin Local radix TLB flush operations that operate on congruence classes have explicit ERAT flushes for POWER9. The process scoped LPID flush did not have a flush, so add it. Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- arch/powerpc/mm/tlb-radix.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c index fef3e1eb3a19..4e798f33c530 100644 --- a/arch/powerpc/mm/tlb-radix.c +++ b/arch/powerpc/mm/tlb-radix.c @@ -366,6 +366,7 @@ static inline void _tlbiel_lpid_guest(unsigned long lpid, unsigned long ric) __tlbiel_lpid_guest(lpid, set, RIC_FLUSH_TLB); asm volatile("ptesync": : :"memory"); + asm volatile(PPC_INVALIDATE_ERAT : : :"memory"); } -- 2.18.0 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation 2018-08-27 3:03 ` [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation Nicholas Piggin @ 2018-08-27 8:16 ` Benjamin Herrenschmidt 2018-08-27 9:11 ` Nicholas Piggin 2018-08-27 9:17 ` Nicholas Piggin 0 siblings, 2 replies; 7+ messages in thread From: Benjamin Herrenschmidt @ 2018-08-27 8:16 UTC (permalink / raw) To: Nicholas Piggin, linuxppc-dev; +Cc: Paul Mackerras On Mon, 2018-08-27 at 13:03 +1000, Nicholas Piggin wrote: > Local radix TLB flush operations that operate on congruence classes > have explicit ERAT flushes for POWER9. The process scoped LPID flush > did not have a flush, so add it. Paul, is that an actual bug ? I think the ERAT is flushed on LPID changes... > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > --- > arch/powerpc/mm/tlb-radix.c | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c > index fef3e1eb3a19..4e798f33c530 100644 > --- a/arch/powerpc/mm/tlb-radix.c > +++ b/arch/powerpc/mm/tlb-radix.c > @@ -366,6 +366,7 @@ static inline void _tlbiel_lpid_guest(unsigned long lpid, unsigned long ric) > __tlbiel_lpid_guest(lpid, set, RIC_FLUSH_TLB); > > asm volatile("ptesync": : :"memory"); > + asm volatile(PPC_INVALIDATE_ERAT : : :"memory"); > } > > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation 2018-08-27 8:16 ` Benjamin Herrenschmidt @ 2018-08-27 9:11 ` Nicholas Piggin 2018-08-27 9:17 ` Nicholas Piggin 1 sibling, 0 replies; 7+ messages in thread From: Nicholas Piggin @ 2018-08-27 9:11 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, Paul Mackerras On Mon, 27 Aug 2018 18:16:05 +1000 Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote: > On Mon, 2018-08-27 at 13:03 +1000, Nicholas Piggin wrote: > > Local radix TLB flush operations that operate on congruence classes > > have explicit ERAT flushes for POWER9. The process scoped LPID flush > > did not have a flush, so add it. =20 >=20 > Paul, is that an actual bug ? I think the ERAT is flushed on LPID > changes... We also have a PPC_INVALIDATE_ERAT in tlbiel_lpid. I'd like to add some comments for these things because I'm not entirely clear on them myself. The P9 UM says, "Additionally, mtpidr and mtlpidr instructions perform an implicit slbia with IH =3D x=E2=80=983=E2=80=99." Although slbia IH=3D3 does not appear to invalidate LPID!=3D0 && PID=3D=3D0 entries (guest OS mappigs). The UM also says that tlbiels will clear the involved ERATs, but this might not match hardware? Thanks, Nick >=20 > > Signed-off-by: Nicholas Piggin <npiggin@gmail.com> > > --- > > arch/powerpc/mm/tlb-radix.c | 1 + > > 1 file changed, 1 insertion(+) > >=20 > > diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c > > index fef3e1eb3a19..4e798f33c530 100644 > > --- a/arch/powerpc/mm/tlb-radix.c > > +++ b/arch/powerpc/mm/tlb-radix.c > > @@ -366,6 +366,7 @@ static inline void _tlbiel_lpid_guest(unsigned long= lpid, unsigned long ric) > > __tlbiel_lpid_guest(lpid, set, RIC_FLUSH_TLB); > > =20 > > asm volatile("ptesync": : :"memory"); > > + asm volatile(PPC_INVALIDATE_ERAT : : :"memory"); > > } > > =20 > > =20 >=20 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation 2018-08-27 8:16 ` Benjamin Herrenschmidt 2018-08-27 9:11 ` Nicholas Piggin @ 2018-08-27 9:17 ` Nicholas Piggin 1 sibling, 0 replies; 7+ messages in thread From: Nicholas Piggin @ 2018-08-27 9:17 UTC (permalink / raw) To: Benjamin Herrenschmidt; +Cc: linuxppc-dev, Paul Mackerras On Mon, 27 Aug 2018 18:16:05 +1000 Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote: > On Mon, 2018-08-27 at 13:03 +1000, Nicholas Piggin wrote: > > Local radix TLB flush operations that operate on congruence classes > > have explicit ERAT flushes for POWER9. The process scoped LPID flush > > did not have a flush, so add it. > > Paul, is that an actual bug ? I think the ERAT is flushed on LPID > changes... FWIW I'd like to add the following patch after these fixes. While I have your attention... [PATCH] powerpc/64s/radix: keep kernel ERAT over local process/guest invalidates Signed-off-by: Nicholas Piggin <npiggin@gmail.com> --- arch/powerpc/include/asm/ppc-opcode.h | 10 +++++++++- arch/powerpc/mm/tlb-radix.c | 6 +++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/include/asm/ppc-opcode.h b/arch/powerpc/include/asm/ppc-opcode.h index 665af14850e4..5fe617ab680a 100644 --- a/arch/powerpc/include/asm/ppc-opcode.h +++ b/arch/powerpc/include/asm/ppc-opcode.h @@ -568,7 +568,15 @@ #define PPC_SLBIA(IH) stringify_in_c(.long PPC_INST_SLBIA | \ ((IH & 0x7) << 21)) -#define PPC_INVALIDATE_ERAT PPC_SLBIA(7) + +/* + * These may only be used by ARCH_300. + * GUEST/USER invalidates should only be used by radix mode, on HPT they also + * invalidate SLBs so the SLBIA instruction should be used directly. + */ +#define PPC_INVALIDATE_ERAT PPC_SLBIA(7) +#define PPC_INVALIDATE_GUEST_ERAT PPC_SLBIA(6) +#define PPC_INVALIDATE_USER_ERAT PPC_SLBIA(3) #define VCMPEQUD_RC(vrt, vra, vrb) stringify_in_c(.long PPC_INST_VCMPEQUD | \ ___PPC_RT(vrt) | ___PPC_RA(vra) | \ diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c index 4e798f33c530..6887e4b2568b 100644 --- a/arch/powerpc/mm/tlb-radix.c +++ b/arch/powerpc/mm/tlb-radix.c @@ -262,7 +262,7 @@ static inline void _tlbiel_pid(unsigned long pid, unsigned long ric) __tlbiel_pid(pid, set, RIC_FLUSH_TLB); asm volatile("ptesync": : :"memory"); - asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory"); + asm volatile(PPC_INVALIDATE_USER_ERAT "; isync" : : :"memory"); } static inline void _tlbie_pid(unsigned long pid, unsigned long ric) @@ -314,7 +314,7 @@ static inline void _tlbiel_lpid(unsigned long lpid, unsigned long ric) __tlbiel_lpid(lpid, set, RIC_FLUSH_TLB); asm volatile("ptesync": : :"memory"); - asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory"); + asm volatile(PPC_INVALIDATE_GUEST_ERAT "; isync" : : :"memory"); } static inline void _tlbie_lpid(unsigned long lpid, unsigned long ric) @@ -366,7 +366,7 @@ static inline void _tlbiel_lpid_guest(unsigned long lpid, unsigned long ric) __tlbiel_lpid_guest(lpid, set, RIC_FLUSH_TLB); asm volatile("ptesync": : :"memory"); - asm volatile(PPC_INVALIDATE_ERAT : : :"memory"); + asm volatile(PPC_INVALIDATE_GUEST_ERAT : : :"memory"); } -- ^ permalink raw reply related [flat|nested] 7+ messages in thread
end of thread, other threads:[~2018-10-15 4:08 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-08-27 3:03 [PATCH 0/2] ERAT invalidation fixes Nicholas Piggin 2018-08-27 3:03 ` [PATCH 1/2] powerpc/64s/hash: do not use PPC_INVALIDATE_ERAT on CPUs before POWER9 Nicholas Piggin 2018-10-15 4:00 ` [1/2] " Michael Ellerman 2018-08-27 3:03 ` [PATCH 2/2] powerpc/64s/radix: Explicitly flush ERAT with local LPID invalidation Nicholas Piggin 2018-08-27 8:16 ` Benjamin Herrenschmidt 2018-08-27 9:11 ` Nicholas Piggin 2018-08-27 9:17 ` Nicholas Piggin
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.