[RFC] mm/migration: Add trace events for THP migrations

[RFC] mm/migration: Add trace events for THP migrations

[Anshuman Khandual]

This adds two trace events for PMD based THP migration without split. These
events closely follow the implementation details like setting and removing
of PMD migration entries, which are essential operations for THP migration.

Cc: Steven Rostedt <rostedt@goodmis.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Zi Yan <ziy@nvidia.com>
Cc: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Cc: John Hubbard <jhubbard@nvidia.com>
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org
Signed-off-by: Anshuman Khandual <anshuman.khandual@arm.com>
---
This applies on v5.16-rc6

 include/trace/events/thp.h | 61 ++++++++++++++++++++++++++++++++++++++
 mm/huge_memory.c           |  5 ++++
 2 files changed, 66 insertions(+)

diff --git a/include/trace/events/thp.h b/include/trace/events/thp.h
index d7fbbe551841..be5aeb783af1 100644
--- a/include/trace/events/thp.h
+++ b/include/trace/events/thp.h
@@ -83,6 +83,67 @@ TRACE_EVENT(hugepage_splitting,
 		      __entry->addr, __entry->pte)
 );
 
+TRACE_EVENT(set_migration_pmd,
+
+	TP_PROTO(struct mm_struct *mm, unsigned long address,
+		 pmd_t *pmdp, struct page *page, unsigned long pmdval),
+
+	TP_ARGS(mm, address, pmdp, page, pmdval),
+
+	TP_STRUCT__entry(
+		__field(struct mm_struct *, mm)
+		__field(unsigned long, address)
+		__field(pmd_t *, pmdp)
+		__field(struct page *, page)
+		__field(unsigned long, pmdval)
+	),
+
+	TP_fast_assign(
+		__entry->mm = mm;
+		__entry->address = address;
+		__entry->pmdp = pmdp;
+		__entry->page = page;
+		__entry->pmdval = pmdval;
+	),
+
+	TP_printk("mm=%p, address=%lx, pmdp=%p, page=%p pmdval=%lx",
+		__entry->mm,
+		__entry->address,
+		__entry->pmdp,
+		__entry->page,
+		__entry->pmdval)
+);
+
+TRACE_EVENT(remove_migration_pmd,
+
+	TP_PROTO(struct mm_struct *mm, unsigned long address,
+		 pmd_t *pmdp, struct page *page, unsigned long pmdval),
+
+	TP_ARGS(mm, address, pmdp, page, pmdval),
+
+	TP_STRUCT__entry(
+		__field(struct mm_struct *, mm)
+		__field(unsigned long, address)
+		__field(pmd_t *, pmdp)
+		__field(struct page *, page)
+		__field(unsigned long, pmdval)
+	),
+
+	TP_fast_assign(
+		__entry->mm = mm;
+		__entry->address = address;
+		__entry->pmdp = pmdp;
+		__entry->page = page;
+		__entry->pmdval = pmdval;
+	),
+
+	TP_printk("mm=%p, address=%lx, pmdp=%p, page=%p pmdval=%lx",
+		__entry->mm,
+		__entry->address,
+		__entry->pmdp,
+		__entry->page,
+		__entry->pmdval)
+);
 #endif /* _TRACE_THP_H */
 
 /* This part must be outside protection */
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e5483347291c..611de486e095 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -39,6 +39,9 @@
 #include <asm/pgalloc.h>
 #include "internal.h"
 
+#define CREATE_TRACE_POINTS
+#include <trace/events/thp.h>
+
 /*
  * By default, transparent hugepage support is disabled in order to avoid
  * risking an increased memory footprint for applications that are not
@@ -3173,6 +3176,7 @@ void set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw,
 	set_pmd_at(mm, address, pvmw->pmd, pmdswp);
 	page_remove_rmap(page, true);
 	put_page(page);
+	trace_set_migration_pmd(mm, address, pvmw->pmd, page, pmd_val(pmdswp));
 }
 
 void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
@@ -3206,5 +3210,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new)
 	if ((vma->vm_flags & VM_LOCKED) && !PageDoubleMap(new))
 		mlock_vma_page(new);
 	update_mmu_cache_pmd(vma, address, pvmw->pmd);
+	trace_remove_migration_pmd(mm, address, pvmw->pmd, new, pmd_val(pmde));
 }
 #endif
-- 
2.20.1

Re: [RFC] mm/migration: Add trace events for THP migrations

[Matthew Wilcox]

On Fri, Dec 24, 2021 at 12:16:38PM +0530, Anshuman Khandual wrote:
> This adds two trace events for PMD based THP migration without split. These
> events closely follow the implementation details like setting and removing
> of PMD migration entries, which are essential operations for THP migration.

Why are you printing the address of a struct page?  What useful
information does this supply?  Same question for the struct mm.
And the pmdp, for that matter.

You haven't said _why_ you want these tracepoints.  So it's impossible
to suggest what you _should_ be doing, because what you _are_ doing
is obviously wrong.

> +	TP_printk("mm=%p, address=%lx, pmdp=%p, page=%p pmdval=%lx",
> +		__entry->mm,
> +		__entry->address,
> +		__entry->pmdp,
> +		__entry->page,
> +		__entry->pmdval)
> +);

Re: [RFC] mm/migration: Add trace events for THP migrations

[Anshuman Khandual]

Hello Matthew,

On 12/24/21 8:18 PM, Matthew Wilcox wrote:
> On Fri, Dec 24, 2021 at 12:16:38PM +0530, Anshuman Khandual wrote:
>> This adds two trace events for PMD based THP migration without split. These
>> events closely follow the implementation details like setting and removing
>> of PMD migration entries, which are essential operations for THP migration.
> 
> Why are you printing the address of a struct page?  What useful
> information does this supply?  Same question for the struct mm.
> And the pmdp, for that matter.

Just to make individual trace records comprehensive enough to capture
which (and where) the PMD entries went through migration entry state.
But is there any particular concern here capturing mm, page and pmdp ?

> 
> You haven't said _why_ you want these tracepoints.  So it's impossible
> to suggest what you _should_ be doing, because what you _are_ doing
> is obviously wrong.

Just for debug purpose. To see which (and where) PMD entries are being
migrated as is without a split, via PMD migration entries. Wondering if
you are suggesting just to capture addr, pmdval and just drop others ?

> 
>> +	TP_printk("mm=%p, address=%lx, pmdp=%p, page=%p pmdval=%lx",
>> +		__entry->mm,
>> +		__entry->address,
>> +		__entry->pmdp,
>> +		__entry->page,
>> +		__entry->pmdval)
>> +);
> 

- Anshuman