[0/3] TLB flush counters

[0/3] TLB flush counters

[Atish Patra]

The RISC-V patch (2/3) is based on Gary's TLB flush patch series

https://patchwork.kernel.org/project/linux-riscv/list/?series=97315

The x86 kconfig fix patch(1/3) can be applied separately.

Atish Patra (3):
x86: Update DEBUG_TLBFLUSH options description.
RISC-V: Update tlb flush counters
RISC-V: Add DEBUG_TLBFLUSH option.

arch/riscv/Kconfig.debug          | 12 ++++++++++++
arch/riscv/include/asm/tlbflush.h |  5 +++++
arch/riscv/mm/tlbflush.c          | 12 ++++++++++++
arch/x86/Kconfig.debug            | 15 +++------------
4 files changed, 32 insertions(+), 12 deletions(-)

--
2.21.0

[1/3] x86: Update DEBUG_TLBFLUSH options description.

[Atish Patra]

CONFIG_DEBUG_TLBFLUSH was added in 'commit 3df3212f9722 ("x86/tlb: add
tlb_flushall_shift knob into debugfs")' to support tlb_flushall_shift
knob. The knob was removed in 'commit e9f4e0a9fe27 ("x86/mm: Rip out
complicated, out-of-date, buggy TLB flushing")'.  However, the debug
option was never removed from Kconfig. It was reused in commit
'9824cf9753ec ("mm: vmstats: tlb flush counters")' but the commit text was
never updated accordingly.

Update the Kconfig option description as per its current usage.

Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
 arch/x86/Kconfig.debug | 15 +++------------
 1 file changed, 3 insertions(+), 12 deletions(-)

diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
index 15d0fbe27872..c1a48d4ffebb 100644
--- a/arch/x86/Kconfig.debug
+++ b/arch/x86/Kconfig.debug
@@ -125,21 +125,12 @@ config DOUBLEFAULT
 	  hair.
 
 config DEBUG_TLBFLUSH
-	bool "Set upper limit of TLB entries to flush one-by-one"
+	bool "Save tlb flush statstics to vmstat"
 	depends on DEBUG_KERNEL
 	---help---
 
-	X86-only for now.
-
-	This option allows the user to tune the amount of TLB entries the
-	kernel flushes one-by-one instead of doing a full TLB flush. In
-	certain situations, the former is cheaper. This is controlled by the
-	tlb_flushall_shift knob under /sys/kernel/debug/x86. If you set it
-	to -1, the code flushes the whole TLB unconditionally. Otherwise,
-	for positive values of it, the kernel will use single TLB entry
-	invalidating instructions according to the following formula:
-
-	flush_entries <= active_tlb_entries / 2^tlb_flushall_shift
+	Add tlbflush statstics to vmstat. It is really helpful understand tlbflush
+	performance and behavior. It should be enabled only for debugging purpose.
 
 	If in doubt, say "N".
 
-- 
2.21.0

Re: [1/3] x86: Update DEBUG_TLBFLUSH options description.

[Christoph Hellwig]

Given that this option enables generic code (which you reuse for RISC-V
later in this series) please also move the config option to
mm/Kconfig, proabbly keyed off another ARCH_HAVE_DEBUG_TLBFLUSH
symbol that the architecture can select.

Re: [1/3] x86: Update DEBUG_TLBFLUSH options description.

[Atish Patra]

On 4/10/19 11:56 PM, Christoph Hellwig wrote:
> Given that this option enables generic code (which you reuse for RISC-V
> later in this series) please also move the config option to
> mm/Kconfig, proabbly keyed off another ARCH_HAVE_DEBUG_TLBFLUSH
> symbol that the architecture can select.
> 

Sure.

Regards,
Atish
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
> 

Re: [1/3] x86: Update DEBUG_TLBFLUSH options description.

[Borislav Petkov]

On Fri, Apr 12, 2019 at 01:14:54PM -0700, Atish Patra wrote:
> On 4/10/19 11:56 PM, Christoph Hellwig wrote:
> > Given that this option enables generic code (which you reuse for RISC-V
> > later in this series) please also move the config option to
> > mm/Kconfig, proabbly keyed off another ARCH_HAVE_DEBUG_TLBFLUSH
> > symbol that the architecture can select.
> > 
> 
> Sure.

And when you do that, instead of deleting the help text, make it
generic. Otherwise, there's no explanation anymore, how that option is
supposed to be used through tlb_flushall_shift.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.

Re: [1/3] x86: Update DEBUG_TLBFLUSH options description.

[Atish Patra]

On 4/14/19 7:29 AM, Borislav Petkov wrote:
> On Fri, Apr 12, 2019 at 01:14:54PM -0700, Atish Patra wrote:
>> On 4/10/19 11:56 PM, Christoph Hellwig wrote:
>>> Given that this option enables generic code (which you reuse for RISC-V
>>> later in this series) please also move the config option to
>>> mm/Kconfig, proabbly keyed off another ARCH_HAVE_DEBUG_TLBFLUSH
>>> symbol that the architecture can select.
>>>
>>
>> Sure.
> 
> And when you do that, instead of deleting the help text, make it
> generic. Otherwise, there's no explanation anymore, how that option is
> supposed to be used through tlb_flushall_shift.
> 
Not sure I am following you.
tlb_flushall_shift knob was removed by
commit e9f4e0a9fe27 ("x86/mm: Rip out complicated, out-of-date, buggy 
TLB flushing")


Regards,
Atish

Re: [1/3] x86: Update DEBUG_TLBFLUSH options description.

[Palmer Dabbelt]

On Wed, 10 Apr 2019 15:44:47 PDT (-0700), atish.patra@wdc.com wrote:
> CONFIG_DEBUG_TLBFLUSH was added in 'commit 3df3212f9722 ("x86/tlb: add
> tlb_flushall_shift knob into debugfs")' to support tlb_flushall_shift
> knob. The knob was removed in 'commit e9f4e0a9fe27 ("x86/mm: Rip out
> complicated, out-of-date, buggy TLB flushing")'.  However, the debug
> option was never removed from Kconfig. It was reused in commit
> '9824cf9753ec ("mm: vmstats: tlb flush counters")' but the commit text was
> never updated accordingly.
>
> Update the Kconfig option description as per its current usage.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
>  arch/x86/Kconfig.debug | 15 +++------------
>  1 file changed, 3 insertions(+), 12 deletions(-)
>
> diff --git a/arch/x86/Kconfig.debug b/arch/x86/Kconfig.debug
> index 15d0fbe27872..c1a48d4ffebb 100644
> --- a/arch/x86/Kconfig.debug
> +++ b/arch/x86/Kconfig.debug
> @@ -125,21 +125,12 @@ config DOUBLEFAULT
>  	  hair.
>
>  config DEBUG_TLBFLUSH
> -	bool "Set upper limit of TLB entries to flush one-by-one"
> +	bool "Save tlb flush statstics to vmstat"
>  	depends on DEBUG_KERNEL
>  	---help---
>
> -	X86-only for now.
> -
> -	This option allows the user to tune the amount of TLB entries the
> -	kernel flushes one-by-one instead of doing a full TLB flush. In
> -	certain situations, the former is cheaper. This is controlled by the
> -	tlb_flushall_shift knob under /sys/kernel/debug/x86. If you set it
> -	to -1, the code flushes the whole TLB unconditionally. Otherwise,
> -	for positive values of it, the kernel will use single TLB entry
> -	invalidating instructions according to the following formula:
> -
> -	flush_entries <= active_tlb_entries / 2^tlb_flushall_shift
> +	Add tlbflush statstics to vmstat. It is really helpful understand tlbflush
> +	performance and behavior. It should be enabled only for debugging purpose.
>
>  	If in doubt, say "N".

Reviewed-by: Palmer Dabbelt <palmer@sifivee.com>

I'm not going to take this via my tree, but I'll look into the next two.

[2/3] RISC-V: Update tlb flush counters

[Atish Patra]

The TLB flush counters under vmstat seems to be very helpful while
debugging TLB flush performance in RISC-V.

Update the counters in every TLB flush methods respectively.

Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
 arch/riscv/include/asm/tlbflush.h |  5 +++++
 arch/riscv/mm/tlbflush.c          | 12 ++++++++++++
 2 files changed, 17 insertions(+)

diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h
index 29a780ca232a..19779a083f52 100644
--- a/arch/riscv/include/asm/tlbflush.h
+++ b/arch/riscv/include/asm/tlbflush.h
@@ -9,6 +9,7 @@
 #define _ASM_RISCV_TLBFLUSH_H
 
 #include <linux/mm_types.h>
+#include <linux/vmstat.h>
 
 /*
  * Flush entire local TLB.  'sfence.vma' implicitly fences with the instruction
@@ -16,11 +17,13 @@
  */
 static inline void local_flush_tlb_all(void)
 {
+	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
 	__asm__ __volatile__ ("sfence.vma" : : : "memory");
 }
 
 static inline void local_flush_tlb_mm(struct mm_struct *mm)
 {
+	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
 	/* Flush ASID 0 so that global mappings are not affected */
 	__asm__ __volatile__ ("sfence.vma x0, %0" : : "r" (0) : "memory");
 }
@@ -28,6 +31,7 @@ static inline void local_flush_tlb_mm(struct mm_struct *mm)
 static inline void local_flush_tlb_page(struct vm_area_struct *vma,
 	unsigned long addr)
 {
+	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
 	__asm__ __volatile__ ("sfence.vma %0, %1"
 			      : : "r" (addr), "r" (0)
 			      : "memory");
@@ -35,6 +39,7 @@ static inline void local_flush_tlb_page(struct vm_area_struct *vma,
 
 static inline void local_flush_tlb_kernel_page(unsigned long addr)
 {
+	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
 	__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory");
 }
 
diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
index ceee76f14a0a..8072d7da32bb 100644
--- a/arch/riscv/mm/tlbflush.c
+++ b/arch/riscv/mm/tlbflush.c
@@ -4,6 +4,8 @@
  */
 
 #include <linux/mm.h>
+#include <linux/vmstat.h>
+#include <linux/cpumask.h>
 #include <asm/sbi.h>
 
 #define SFENCE_VMA_FLUSH_ALL ((unsigned long) -1)
@@ -110,6 +112,7 @@ static void ipi_remote_sfence_vma(void *info)
 	unsigned long size = data->size;
 	unsigned long i;
 
+	count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED);
 	if (size == SFENCE_VMA_FLUSH_ALL) {
 		local_flush_tlb_all();
 	}
@@ -129,6 +132,8 @@ static void ipi_remote_sfence_vma_asid(void *info)
 	unsigned long size = data->size;
 	unsigned long i;
 
+	count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED);
+	/* Flush entire MM context */
 	if (size == SFENCE_VMA_FLUSH_ALL) {
 		__asm__ __volatile__ ("sfence.vma x0, %0"
 				      : : "r" (asid)
@@ -158,6 +163,13 @@ static void remote_sfence_vma(unsigned long start, unsigned long size)
 static void remote_sfence_vma_asid(cpumask_t *mask, unsigned long start,
 				   unsigned long size, unsigned long asid)
 {
+	int cpuid = smp_processor_id();
+
+	if (cpumask_equal(mask, cpumask_of(cpuid)))
+		count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
+	else
+		count_vm_tlb_event(NR_TLB_REMOTE_FLUSH);
+
 	if (tlbi_ipi) {
 		struct tlbi info = {
 			.start = start,
-- 
2.21.0

Re: [2/3] RISC-V: Update tlb flush counters

[Palmer Dabbelt]

On Wed, 10 Apr 2019 15:44:48 PDT (-0700), atish.patra@wdc.com wrote:
> The TLB flush counters under vmstat seems to be very helpful while
> debugging TLB flush performance in RISC-V.
>
> Update the counters in every TLB flush methods respectively.
>
> Signed-off-by: Atish Patra <atish.patra@wdc.com>
> ---
>  arch/riscv/include/asm/tlbflush.h |  5 +++++
>  arch/riscv/mm/tlbflush.c          | 12 ++++++++++++
>  2 files changed, 17 insertions(+)
>
> diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h
> index 29a780ca232a..19779a083f52 100644
> --- a/arch/riscv/include/asm/tlbflush.h
> +++ b/arch/riscv/include/asm/tlbflush.h
> @@ -9,6 +9,7 @@
>  #define _ASM_RISCV_TLBFLUSH_H
>
>  #include <linux/mm_types.h>
> +#include <linux/vmstat.h>
>
>  /*
>   * Flush entire local TLB.  'sfence.vma' implicitly fences with the instruction
> @@ -16,11 +17,13 @@
>   */
>  static inline void local_flush_tlb_all(void)
>  {
> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
>  	__asm__ __volatile__ ("sfence.vma" : : : "memory");
>  }
>
>  static inline void local_flush_tlb_mm(struct mm_struct *mm)
>  {
> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
>  	/* Flush ASID 0 so that global mappings are not affected */
>  	__asm__ __volatile__ ("sfence.vma x0, %0" : : "r" (0) : "memory");
>  }
> @@ -28,6 +31,7 @@ static inline void local_flush_tlb_mm(struct mm_struct *mm)
>  static inline void local_flush_tlb_page(struct vm_area_struct *vma,
>  	unsigned long addr)
>  {
> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
>  	__asm__ __volatile__ ("sfence.vma %0, %1"
>  			      : : "r" (addr), "r" (0)
>  			      : "memory");
> @@ -35,6 +39,7 @@ static inline void local_flush_tlb_page(struct vm_area_struct *vma,
>
>  static inline void local_flush_tlb_kernel_page(unsigned long addr)
>  {
> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
>  	__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory");
>  }
>
> diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
> index ceee76f14a0a..8072d7da32bb 100644
> --- a/arch/riscv/mm/tlbflush.c
> +++ b/arch/riscv/mm/tlbflush.c
> @@ -4,6 +4,8 @@
>   */
>
>  #include <linux/mm.h>
> +#include <linux/vmstat.h>
> +#include <linux/cpumask.h>
>  #include <asm/sbi.h>
>
>  #define SFENCE_VMA_FLUSH_ALL ((unsigned long) -1)
> @@ -110,6 +112,7 @@ static void ipi_remote_sfence_vma(void *info)
>  	unsigned long size = data->size;
>  	unsigned long i;
>
> +	count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED);
>  	if (size == SFENCE_VMA_FLUSH_ALL) {
>  		local_flush_tlb_all();
>  	}
> @@ -129,6 +132,8 @@ static void ipi_remote_sfence_vma_asid(void *info)
>  	unsigned long size = data->size;
>  	unsigned long i;
>
> +	count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED);
> +	/* Flush entire MM context */
>  	if (size == SFENCE_VMA_FLUSH_ALL) {
>  		__asm__ __volatile__ ("sfence.vma x0, %0"
>  				      : : "r" (asid)
> @@ -158,6 +163,13 @@ static void remote_sfence_vma(unsigned long start, unsigned long size)
>  static void remote_sfence_vma_asid(cpumask_t *mask, unsigned long start,
>  				   unsigned long size, unsigned long asid)
>  {
> +	int cpuid = smp_processor_id();
> +
> +	if (cpumask_equal(mask, cpumask_of(cpuid)))
> +		count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
> +	else
> +		count_vm_tlb_event(NR_TLB_REMOTE_FLUSH);
> +
>  	if (tlbi_ipi) {
>  		struct tlbi info = {
>  			.start = start,

Looks good, but it's not applying on for-next (based on rc6).  Do you mind
re-spinning the patches?

Re: [2/3] RISC-V: Update tlb flush counters

[Atish Patra]

On 4/25/19 10:31 PM, Palmer Dabbelt wrote:
> On Wed, 10 Apr 2019 15:44:48 PDT (-0700), atish.patra@wdc.com wrote:
>> The TLB flush counters under vmstat seems to be very helpful while
>> debugging TLB flush performance in RISC-V.
>>
>> Update the counters in every TLB flush methods respectively.
>>
>> Signed-off-by: Atish Patra <atish.patra@wdc.com>
>> ---
>>   arch/riscv/include/asm/tlbflush.h |  5 +++++
>>   arch/riscv/mm/tlbflush.c          | 12 ++++++++++++
>>   2 files changed, 17 insertions(+)
>>
>> diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h
>> index 29a780ca232a..19779a083f52 100644
>> --- a/arch/riscv/include/asm/tlbflush.h
>> +++ b/arch/riscv/include/asm/tlbflush.h
>> @@ -9,6 +9,7 @@
>>   #define _ASM_RISCV_TLBFLUSH_H
>>
>>   #include <linux/mm_types.h>
>> +#include <linux/vmstat.h>
>>
>>   /*
>>    * Flush entire local TLB.  'sfence.vma' implicitly fences with the instruction
>> @@ -16,11 +17,13 @@
>>    */
>>   static inline void local_flush_tlb_all(void)
>>   {
>> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
>>   	__asm__ __volatile__ ("sfence.vma" : : : "memory");
>>   }
>>
>>   static inline void local_flush_tlb_mm(struct mm_struct *mm)
>>   {
>> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
>>   	/* Flush ASID 0 so that global mappings are not affected */
>>   	__asm__ __volatile__ ("sfence.vma x0, %0" : : "r" (0) : "memory");
>>   }
>> @@ -28,6 +31,7 @@ static inline void local_flush_tlb_mm(struct mm_struct *mm)
>>   static inline void local_flush_tlb_page(struct vm_area_struct *vma,
>>   	unsigned long addr)
>>   {
>> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
>>   	__asm__ __volatile__ ("sfence.vma %0, %1"
>>   			      : : "r" (addr), "r" (0)
>>   			      : "memory");
>> @@ -35,6 +39,7 @@ static inline void local_flush_tlb_page(struct vm_area_struct *vma,
>>
>>   static inline void local_flush_tlb_kernel_page(unsigned long addr)
>>   {
>> +	count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ONE);
>>   	__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory");
>>   }
>>
>> diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c
>> index ceee76f14a0a..8072d7da32bb 100644
>> --- a/arch/riscv/mm/tlbflush.c
>> +++ b/arch/riscv/mm/tlbflush.c
>> @@ -4,6 +4,8 @@
>>    */
>>
>>   #include <linux/mm.h>
>> +#include <linux/vmstat.h>
>> +#include <linux/cpumask.h>
>>   #include <asm/sbi.h>
>>
>>   #define SFENCE_VMA_FLUSH_ALL ((unsigned long) -1)
>> @@ -110,6 +112,7 @@ static void ipi_remote_sfence_vma(void *info)
>>   	unsigned long size = data->size;
>>   	unsigned long i;
>>
>> +	count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED);
>>   	if (size == SFENCE_VMA_FLUSH_ALL) {
>>   		local_flush_tlb_all();
>>   	}
>> @@ -129,6 +132,8 @@ static void ipi_remote_sfence_vma_asid(void *info)
>>   	unsigned long size = data->size;
>>   	unsigned long i;
>>
>> +	count_vm_tlb_event(NR_TLB_REMOTE_FLUSH_RECEIVED);
>> +	/* Flush entire MM context */
>>   	if (size == SFENCE_VMA_FLUSH_ALL) {
>>   		__asm__ __volatile__ ("sfence.vma x0, %0"
>>   				      : : "r" (asid)
>> @@ -158,6 +163,13 @@ static void remote_sfence_vma(unsigned long start, unsigned long size)
>>   static void remote_sfence_vma_asid(cpumask_t *mask, unsigned long start,
>>   				   unsigned long size, unsigned long asid)
>>   {
>> +	int cpuid = smp_processor_id();
>> +
>> +	if (cpumask_equal(mask, cpumask_of(cpuid)))
>> +		count_vm_tlb_event(NR_TLB_LOCAL_FLUSH_ALL);
>> +	else
>> +		count_vm_tlb_event(NR_TLB_REMOTE_FLUSH);
>> +
>>   	if (tlbi_ipi) {
>>   		struct tlbi info = {
>>   			.start = start,
> 
> Looks good, but it's not applying on for-next (based on rc6).  Do you mind
> re-spinning the patches?
> 
> 

This patch depends is based on Gary's TLB flush patch series

https://patchwork.kernel.org/project/linux-riscv/list/?series=97315

So it should only be merged on top of it.

Regards,
Atish
> _______________________________________________
> linux-riscv mailing list
> linux-riscv@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-riscv
> 

[3/3] RISC-V: Add DEBUG_TLBFLUSH option.

[Atish Patra]

The TLB flush counters under vmstat seems to be very helpful while
debugging TLB flush performance in RISC-V.

Add the Kconfig option only for debug kernels.

Signed-off-by: Atish Patra <atish.patra@wdc.com>
---
 arch/riscv/Kconfig.debug | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/riscv/Kconfig.debug b/arch/riscv/Kconfig.debug
index e69de29bb2d1..6a2d3762aeda 100644
--- a/arch/riscv/Kconfig.debug
+++ b/arch/riscv/Kconfig.debug
@@ -0,0 +1,12 @@
+# SPDX-License-Identifier: GPL-2.0
+
+config DEBUG_TLBFLUSH
+	bool "Save tlb flush statstics to vmstat"
+	depends on DEBUG_KERNEL
+	help
+
+	Add TLB flush statstics to vmstat. It is really helpful understand tlbflush
+	performance and behavior. It should be enabled only for debugging purpose.
+
+	If in doubt, say "N".
+
-- 
2.21.0