[PATCH 1/2] powerpc/mm/radix: Don't do page walk cache flush when doing full mm flush

[PATCH 1/2] powerpc/mm/radix: Don't do page walk cache flush when doing full mm flush

[Aneesh Kumar K.V]

For fullmm tlb flush, we do a flush with RIC_FLUSH_ALL which will invalidate all
related caches (radix__tlb_flush()). Hence the pwc flush is not needed.

Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/mm/tlb-radix.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
index 83dc1ccc2fa1..f3e58bd60d1a 100644
--- a/arch/powerpc/mm/tlb-radix.c
+++ b/arch/powerpc/mm/tlb-radix.c
@@ -129,6 +129,12 @@ void radix__local_flush_tlb_pwc(struct mmu_gather *tlb, unsigned long addr)
 {
 	unsigned long pid;
 	struct mm_struct *mm = tlb->mm;
+	/*
+	 * If we are doing a full mm flush, we will do a tlb flush
+	 * with RIC_FLUSH_ALL later.
+	 */
+	if (tlb->fullmm)
+		return;
 
 	preempt_disable();
 
@@ -195,6 +201,12 @@ void radix__flush_tlb_pwc(struct mmu_gather *tlb, unsigned long addr)
 	unsigned long pid;
 	struct mm_struct *mm = tlb->mm;
 
+	/*
+	 * If we are doing a full mm flush, we will do a tlb flush
+	 * with RIC_FLUSH_ALL later.
+	 */
+	if (tlb->fullmm)
+		return;
 	preempt_disable();
 
 	pid = mm->context.id;
-- 
2.7.4

[PATCH 2/2] powerpc/mm/radix: Remove unnecessary ptesync

[Aneesh Kumar K.V]

For a tlbiel with pid, we need to issue tlbiel with set number encoded. We
don't need to do ptesync for each of those. Instead we need one for the entire
tlbiel pid operation.

Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
---
 arch/powerpc/mm/tlb-radix.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
index f3e58bd60d1a..b68b5219cf45 100644
--- a/arch/powerpc/mm/tlb-radix.c
+++ b/arch/powerpc/mm/tlb-radix.c
@@ -34,10 +34,8 @@ static inline void __tlbiel_pid(unsigned long pid, int set,
 	prs = 1; /* process scoped */
 	r = 1;   /* raidx format */
 
-	asm volatile("ptesync": : :"memory");
 	asm volatile(PPC_TLBIEL(%0, %4, %3, %2, %1)
 		     : : "r"(rb), "i"(r), "i"(prs), "i"(ric), "r"(rs) : "memory");
-	asm volatile("ptesync": : :"memory");
 }
 
 /*
@@ -47,9 +45,11 @@ static inline void _tlbiel_pid(unsigned long pid, unsigned long ric)
 {
 	int set;
 
+	asm volatile("ptesync": : :"memory");
 	for (set = 0; set < POWER9_TLB_SETS_RADIX ; set++) {
 		__tlbiel_pid(pid, set, ric);
 	}
+	asm volatile("ptesync": : :"memory");
 	asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory");
 }
 
-- 
2.7.4

Re: [PATCH 2/2] powerpc/mm/radix: Remove unnecessary ptesync

[Anton Blanchard]

On Sat,  1 Apr 2017 20:11:48 +0530
"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> wrote:

> For a tlbiel with pid, we need to issue tlbiel with set number
> encoded. We don't need to do ptesync for each of those. Instead we
> need one for the entire tlbiel pid operation.
> 
> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>

Thanks Aneesh.

Acked-by: Anton Blanchard <anton@samba.org>

Anton

> ---
>  arch/powerpc/mm/tlb-radix.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
> index f3e58bd60d1a..b68b5219cf45 100644
> --- a/arch/powerpc/mm/tlb-radix.c
> +++ b/arch/powerpc/mm/tlb-radix.c
> @@ -34,10 +34,8 @@ static inline void __tlbiel_pid(unsigned long pid,
> int set, prs = 1; /* process scoped */
>  	r = 1;   /* raidx format */
>  
> -	asm volatile("ptesync": : :"memory");
>  	asm volatile(PPC_TLBIEL(%0, %4, %3, %2, %1)
>  		     : : "r"(rb), "i"(r), "i"(prs), "i"(ric),
> "r"(rs) : "memory");
> -	asm volatile("ptesync": : :"memory");
>  }
>  
>  /*
> @@ -47,9 +45,11 @@ static inline void _tlbiel_pid(unsigned long pid,
> unsigned long ric) {
>  	int set;
>  
> +	asm volatile("ptesync": : :"memory");
>  	for (set = 0; set < POWER9_TLB_SETS_RADIX ; set++) {
>  		__tlbiel_pid(pid, set, ric);
>  	}
> +	asm volatile("ptesync": : :"memory");
>  	asm volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory");
>  }
>  

Re: [PATCH 1/2] powerpc/mm/radix: Don't do page walk cache flush when doing full mm flush

[Anton Blanchard]

On Sat,  1 Apr 2017 20:11:47 +0530
"Aneesh Kumar K.V" <aneesh.kumar@linux.vnet.ibm.com> wrote:

> For fullmm tlb flush, we do a flush with RIC_FLUSH_ALL which will
> invalidate all related caches (radix__tlb_flush()). Hence the pwc
> flush is not needed.

Thanks Aneesh. I see a 3x improvement in exec performance with these
2 patches.

Acked-by: Anton Blanchard <anton@samba.org>

Anton

> 
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> ---
>  arch/powerpc/mm/tlb-radix.c | 12 ++++++++++++
>  1 file changed, 12 insertions(+)
> 
> diff --git a/arch/powerpc/mm/tlb-radix.c b/arch/powerpc/mm/tlb-radix.c
> index 83dc1ccc2fa1..f3e58bd60d1a 100644
> --- a/arch/powerpc/mm/tlb-radix.c
> +++ b/arch/powerpc/mm/tlb-radix.c
> @@ -129,6 +129,12 @@ void radix__local_flush_tlb_pwc(struct
> mmu_gather *tlb, unsigned long addr) {
>  	unsigned long pid;
>  	struct mm_struct *mm = tlb->mm;
> +	/*
> +	 * If we are doing a full mm flush, we will do a tlb flush
> +	 * with RIC_FLUSH_ALL later.
> +	 */
> +	if (tlb->fullmm)
> +		return;
>  
>  	preempt_disable();
>  
> @@ -195,6 +201,12 @@ void radix__flush_tlb_pwc(struct mmu_gather
> *tlb, unsigned long addr) unsigned long pid;
>  	struct mm_struct *mm = tlb->mm;
>  
> +	/*
> +	 * If we are doing a full mm flush, we will do a tlb flush
> +	 * with RIC_FLUSH_ALL later.
> +	 */
> +	if (tlb->fullmm)
> +		return;
>  	preempt_disable();
>  
>  	pid = mm->context.id;

Re: [1/2] powerpc/mm/radix: Don't do page walk cache flush when doing full mm flush

[Michael Ellerman]

On Sat, 2017-04-01 at 14:41:47 UTC, "Aneesh Kumar K.V" wrote:
> For fullmm tlb flush, we do a flush with RIC_FLUSH_ALL which will invalidate all
> related caches (radix__tlb_flush()). Hence the pwc flush is not needed.
> 
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Acked-by: Anton Blanchard <anton@samba.org>

Applied to powerpc next, thanks.

https://git.kernel.org/powerpc/c/f6b0df55cad252fedd60aa2ba75a02

cheers

Re: [2/2] powerpc/mm/radix: Remove unnecessary ptesync

[Michael Ellerman]

On Sat, 2017-04-01 at 14:41:48 UTC, "Aneesh Kumar K.V" wrote:
> For a tlbiel with pid, we need to issue tlbiel with set number encoded. We
> don't need to do ptesync for each of those. Instead we need one for the entire
> tlbiel pid operation.
> 
> Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
> Signed-off-by: Aneesh Kumar K.V <aneesh.kumar@linux.vnet.ibm.com>
> Acked-by: Anton Blanchard <anton@samba.org>

Applied to powerpc next, thanks.

https://git.kernel.org/powerpc/c/f7327e0ba3805470cced2acfa053e7

cheers