* [PATCH 0/2] Implement arm64 specific huge_ptep_get()
@ 2022-05-10 11:12 ` Baolin Wang
0 siblings, 0 replies; 12+ messages in thread
From: Baolin Wang @ 2022-05-10 11:12 UTC (permalink / raw)
To: catalin.marinas, will, mike.kravetz, akpm
Cc: songmuchun, willy, anshuman.khandual, christophe.leroy,
baolin.wang, linux-arm-kernel, linux-kernel, linux-mm
Hi,
As Mike pointed out [1], the huge_ptep_get() will only return one specific
pte value for the CONT-PTE or CONT-PMD size hugetlb on ARM64 system, which
will not take into account the subpages' dirty or young bits of a CONT-PTE/PMD
size hugetlb page. That will make us miss dirty or young flags of a CONT-PTE/PMD
size hugetlb page for those functions that want to check the dirty or
young flags of a hugetlb page. For example, the gather_hugetlb_stats() will
get inaccurate dirty hugetlb page statistics, and the DAMON for hugetlb monitoring
will also get inaccurate access statistics.
To fix this issue, this patch set introduces an ARM64 specific huge_ptep_get()
implementation, which will take into account any subpages' dirty or young bits.
[1] https://lore.kernel.org/linux-mm/85bd80b4-b4fd-0d3f-a2e5-149559f2f387@oracle.com/
Changes from RFC:
- Implement arm64 specific huge_ptep_get() instead of introducing a new interface.
- Add a new patch to convert huge_ptep_get() in hugetlbpage.c
Baolin Wang (2):
arm64/hugetlb: Use ptep_get() to get the pte value of a huge page
arm64/hugetlb: Implement arm64 specific huge_ptep_get()
arch/arm64/include/asm/hugetlb.h | 2 ++
arch/arm64/mm/hugetlbpage.c | 32 ++++++++++++++++++++++++++++----
2 files changed, 30 insertions(+), 4 deletions(-)
--
1.8.3.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 1/2] arm64/hugetlb: Use ptep_get() to get the pte value of a huge page
2022-05-10 11:12 ` Baolin Wang
@ 2022-05-10 11:12 ` Baolin Wang
-1 siblings, 0 replies; 12+ messages in thread
From: Baolin Wang @ 2022-05-10 11:12 UTC (permalink / raw)
To: catalin.marinas, will, mike.kravetz, akpm
Cc: songmuchun, willy, anshuman.khandual, christophe.leroy,
baolin.wang, linux-arm-kernel, linux-kernel, linux-mm
The original huge_ptep_get() on ARM64 is just a wrapper of ptep_get(),
which will not take into account any contig-PTEs dirty and access bits.
Meanwhile we will implement a new ARM64-specific huge_ptep_get()
interface in following patch, which will take into account any contig-PTEs
dirty and access bits and only be allowed to pass the head pte of
a contig-PTE/PMD size page.
Thus change to use ptep_get() to get the pte value of a huge page as
a preparation.
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
---
arch/arm64/mm/hugetlbpage.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index ca8e65c..be5e2f3 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -172,7 +172,7 @@ static pte_t get_clear_flush(struct mm_struct *mm,
unsigned long pgsize,
unsigned long ncontig)
{
- pte_t orig_pte = huge_ptep_get(ptep);
+ pte_t orig_pte = ptep_get(ptep);
bool valid = pte_valid(orig_pte);
unsigned long i, saddr = addr;
@@ -385,7 +385,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
{
int ncontig;
size_t pgsize;
- pte_t orig_pte = huge_ptep_get(ptep);
+ pte_t orig_pte = ptep_get(ptep);
if (!pte_cont(orig_pte))
return ptep_get_and_clear(mm, addr, ptep);
@@ -408,11 +408,11 @@ static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig)
{
int i;
- if (pte_write(pte) != pte_write(huge_ptep_get(ptep)))
+ if (pte_write(pte) != pte_write(ptep_get(ptep)))
return 1;
for (i = 0; i < ncontig; i++) {
- pte_t orig_pte = huge_ptep_get(ptep + i);
+ pte_t orig_pte = ptep_get(ptep + i);
if (pte_dirty(pte) != pte_dirty(orig_pte))
return 1;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH 1/2] arm64/hugetlb: Use ptep_get() to get the pte value of a huge page
@ 2022-05-10 11:12 ` Baolin Wang
0 siblings, 0 replies; 12+ messages in thread
From: Baolin Wang @ 2022-05-10 11:12 UTC (permalink / raw)
To: catalin.marinas, will, mike.kravetz, akpm
Cc: songmuchun, willy, anshuman.khandual, christophe.leroy,
baolin.wang, linux-arm-kernel, linux-kernel, linux-mm
The original huge_ptep_get() on ARM64 is just a wrapper of ptep_get(),
which will not take into account any contig-PTEs dirty and access bits.
Meanwhile we will implement a new ARM64-specific huge_ptep_get()
interface in following patch, which will take into account any contig-PTEs
dirty and access bits and only be allowed to pass the head pte of
a contig-PTE/PMD size page.
Thus change to use ptep_get() to get the pte value of a huge page as
a preparation.
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
---
arch/arm64/mm/hugetlbpage.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index ca8e65c..be5e2f3 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -172,7 +172,7 @@ static pte_t get_clear_flush(struct mm_struct *mm,
unsigned long pgsize,
unsigned long ncontig)
{
- pte_t orig_pte = huge_ptep_get(ptep);
+ pte_t orig_pte = ptep_get(ptep);
bool valid = pte_valid(orig_pte);
unsigned long i, saddr = addr;
@@ -385,7 +385,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
{
int ncontig;
size_t pgsize;
- pte_t orig_pte = huge_ptep_get(ptep);
+ pte_t orig_pte = ptep_get(ptep);
if (!pte_cont(orig_pte))
return ptep_get_and_clear(mm, addr, ptep);
@@ -408,11 +408,11 @@ static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig)
{
int i;
- if (pte_write(pte) != pte_write(huge_ptep_get(ptep)))
+ if (pte_write(pte) != pte_write(ptep_get(ptep)))
return 1;
for (i = 0; i < ncontig; i++) {
- pte_t orig_pte = huge_ptep_get(ptep + i);
+ pte_t orig_pte = ptep_get(ptep + i);
if (pte_dirty(pte) != pte_dirty(orig_pte))
return 1;
--
1.8.3.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH 1/2] arm64/hugetlb: Use ptep_get() to get the pte value of a huge page
2022-05-10 11:12 ` Baolin Wang
@ 2022-05-10 15:55 ` Muchun Song
-1 siblings, 0 replies; 12+ messages in thread
From: Muchun Song @ 2022-05-10 15:55 UTC (permalink / raw)
To: Baolin Wang
Cc: catalin.marinas, will, mike.kravetz, akpm, willy,
anshuman.khandual, christophe.leroy, linux-arm-kernel,
linux-kernel, linux-mm
On Tue, May 10, 2022 at 07:12:52PM +0800, Baolin Wang wrote:
> The original huge_ptep_get() on ARM64 is just a wrapper of ptep_get(),
> which will not take into account any contig-PTEs dirty and access bits.
> Meanwhile we will implement a new ARM64-specific huge_ptep_get()
> interface in following patch, which will take into account any contig-PTEs
> dirty and access bits and only be allowed to pass the head pte of
> a contig-PTE/PMD size page.
IIUC, the huge_ptep_get() you have implemented in patch 2 could
handle non-head pte. It'll return the original pte without potential
AD bit. I admit it is more efficeent to use ptep_get() directly,
but the judgement here should be updated.
With this update.
Reviewed-by: Muchun Song <songmuchun@bytedance.com>
Thanks.
> Thus change to use ptep_get() to get the pte value of a huge page as
> a preparation.
>
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
> arch/arm64/mm/hugetlbpage.c | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
> index ca8e65c..be5e2f3 100644
> --- a/arch/arm64/mm/hugetlbpage.c
> +++ b/arch/arm64/mm/hugetlbpage.c
> @@ -172,7 +172,7 @@ static pte_t get_clear_flush(struct mm_struct *mm,
> unsigned long pgsize,
> unsigned long ncontig)
> {
> - pte_t orig_pte = huge_ptep_get(ptep);
> + pte_t orig_pte = ptep_get(ptep);
> bool valid = pte_valid(orig_pte);
> unsigned long i, saddr = addr;
>
> @@ -385,7 +385,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
> {
> int ncontig;
> size_t pgsize;
> - pte_t orig_pte = huge_ptep_get(ptep);
> + pte_t orig_pte = ptep_get(ptep);
>
> if (!pte_cont(orig_pte))
> return ptep_get_and_clear(mm, addr, ptep);
> @@ -408,11 +408,11 @@ static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig)
> {
> int i;
>
> - if (pte_write(pte) != pte_write(huge_ptep_get(ptep)))
> + if (pte_write(pte) != pte_write(ptep_get(ptep)))
> return 1;
>
> for (i = 0; i < ncontig; i++) {
> - pte_t orig_pte = huge_ptep_get(ptep + i);
> + pte_t orig_pte = ptep_get(ptep + i);
>
> if (pte_dirty(pte) != pte_dirty(orig_pte))
> return 1;
> --
> 1.8.3.1
>
>
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 1/2] arm64/hugetlb: Use ptep_get() to get the pte value of a huge page
@ 2022-05-10 15:55 ` Muchun Song
0 siblings, 0 replies; 12+ messages in thread
From: Muchun Song @ 2022-05-10 15:55 UTC (permalink / raw)
To: Baolin Wang
Cc: catalin.marinas, will, mike.kravetz, akpm, willy,
anshuman.khandual, christophe.leroy, linux-arm-kernel,
linux-kernel, linux-mm
On Tue, May 10, 2022 at 07:12:52PM +0800, Baolin Wang wrote:
> The original huge_ptep_get() on ARM64 is just a wrapper of ptep_get(),
> which will not take into account any contig-PTEs dirty and access bits.
> Meanwhile we will implement a new ARM64-specific huge_ptep_get()
> interface in following patch, which will take into account any contig-PTEs
> dirty and access bits and only be allowed to pass the head pte of
> a contig-PTE/PMD size page.
IIUC, the huge_ptep_get() you have implemented in patch 2 could
handle non-head pte. It'll return the original pte without potential
AD bit. I admit it is more efficeent to use ptep_get() directly,
but the judgement here should be updated.
With this update.
Reviewed-by: Muchun Song <songmuchun@bytedance.com>
Thanks.
> Thus change to use ptep_get() to get the pte value of a huge page as
> a preparation.
>
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
> ---
> arch/arm64/mm/hugetlbpage.c | 8 ++++----
> 1 file changed, 4 insertions(+), 4 deletions(-)
>
> diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
> index ca8e65c..be5e2f3 100644
> --- a/arch/arm64/mm/hugetlbpage.c
> +++ b/arch/arm64/mm/hugetlbpage.c
> @@ -172,7 +172,7 @@ static pte_t get_clear_flush(struct mm_struct *mm,
> unsigned long pgsize,
> unsigned long ncontig)
> {
> - pte_t orig_pte = huge_ptep_get(ptep);
> + pte_t orig_pte = ptep_get(ptep);
> bool valid = pte_valid(orig_pte);
> unsigned long i, saddr = addr;
>
> @@ -385,7 +385,7 @@ pte_t huge_ptep_get_and_clear(struct mm_struct *mm,
> {
> int ncontig;
> size_t pgsize;
> - pte_t orig_pte = huge_ptep_get(ptep);
> + pte_t orig_pte = ptep_get(ptep);
>
> if (!pte_cont(orig_pte))
> return ptep_get_and_clear(mm, addr, ptep);
> @@ -408,11 +408,11 @@ static int __cont_access_flags_changed(pte_t *ptep, pte_t pte, int ncontig)
> {
> int i;
>
> - if (pte_write(pte) != pte_write(huge_ptep_get(ptep)))
> + if (pte_write(pte) != pte_write(ptep_get(ptep)))
> return 1;
>
> for (i = 0; i < ncontig; i++) {
> - pte_t orig_pte = huge_ptep_get(ptep + i);
> + pte_t orig_pte = ptep_get(ptep + i);
>
> if (pte_dirty(pte) != pte_dirty(orig_pte))
> return 1;
> --
> 1.8.3.1
>
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH 1/2] arm64/hugetlb: Use ptep_get() to get the pte value of a huge page
2022-05-10 15:55 ` Muchun Song
@ 2022-05-11 1:24 ` Baolin Wang
-1 siblings, 0 replies; 12+ messages in thread
From: Baolin Wang @ 2022-05-11 1:24 UTC (permalink / raw)
To: Muchun Song
Cc: catalin.marinas, will, mike.kravetz, akpm, willy,
anshuman.khandual, christophe.leroy, linux-arm-kernel,
linux-kernel, linux-mm
On 5/10/2022 11:55 PM, Muchun Song wrote:
> On Tue, May 10, 2022 at 07:12:52PM +0800, Baolin Wang wrote:
>> The original huge_ptep_get() on ARM64 is just a wrapper of ptep_get(),
>> which will not take into account any contig-PTEs dirty and access bits.
>> Meanwhile we will implement a new ARM64-specific huge_ptep_get()
>> interface in following patch, which will take into account any contig-PTEs
>> dirty and access bits and only be allowed to pass the head pte of
>> a contig-PTE/PMD size page.
>
> IIUC, the huge_ptep_get() you have implemented in patch 2 could
> handle non-head pte. It'll return the original pte without potential
> AD bit. I admit it is more efficeent to use ptep_get() directly,
> but the judgement here should be updated.
Ah, right. I missed the 'ncontig' will be 0 if a non-head pte passed.
Will update the commit message in next version. Thanks for reviewing.
>
> With this update.
>
> Reviewed-by: Muchun Song <songmuchun@bytedance.com>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 1/2] arm64/hugetlb: Use ptep_get() to get the pte value of a huge page
@ 2022-05-11 1:24 ` Baolin Wang
0 siblings, 0 replies; 12+ messages in thread
From: Baolin Wang @ 2022-05-11 1:24 UTC (permalink / raw)
To: Muchun Song
Cc: catalin.marinas, will, mike.kravetz, akpm, willy,
anshuman.khandual, christophe.leroy, linux-arm-kernel,
linux-kernel, linux-mm
On 5/10/2022 11:55 PM, Muchun Song wrote:
> On Tue, May 10, 2022 at 07:12:52PM +0800, Baolin Wang wrote:
>> The original huge_ptep_get() on ARM64 is just a wrapper of ptep_get(),
>> which will not take into account any contig-PTEs dirty and access bits.
>> Meanwhile we will implement a new ARM64-specific huge_ptep_get()
>> interface in following patch, which will take into account any contig-PTEs
>> dirty and access bits and only be allowed to pass the head pte of
>> a contig-PTE/PMD size page.
>
> IIUC, the huge_ptep_get() you have implemented in patch 2 could
> handle non-head pte. It'll return the original pte without potential
> AD bit. I admit it is more efficeent to use ptep_get() directly,
> but the judgement here should be updated.
Ah, right. I missed the 'ncontig' will be 0 if a non-head pte passed.
Will update the commit message in next version. Thanks for reviewing.
>
> With this update.
>
> Reviewed-by: Muchun Song <songmuchun@bytedance.com>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 2/2] arm64/hugetlb: Implement arm64 specific huge_ptep_get()
2022-05-10 11:12 ` Baolin Wang
@ 2022-05-10 11:12 ` Baolin Wang
-1 siblings, 0 replies; 12+ messages in thread
From: Baolin Wang @ 2022-05-10 11:12 UTC (permalink / raw)
To: catalin.marinas, will, mike.kravetz, akpm
Cc: songmuchun, willy, anshuman.khandual, christophe.leroy,
baolin.wang, linux-arm-kernel, linux-kernel, linux-mm
Now we use huge_ptep_get() to get the pte value of a hugetlb page,
however it will only return one specific pte value for the CONT-PTE
or CONT-PMD size hugetlb on ARM64 system, which can contain seravel
continuous pte or pmd entries with same page table attributes. And it
will not take into account the subpages' dirty or young bits of a
CONT-PTE/PMD size hugetlb page.
So the huge_ptep_get() is inconsistent with huge_ptep_get_and_clear(),
which already takes account the dirty or young bits for any subpages
in this CONT-PTE/PMD size hugetlb [1]. Meanwhile we can miss dirty or
young flags statistics for hugetlb pages with current huge_ptep_get(),
such as the gather_hugetlb_stats() function, and CONT-PTE/PMD hugetlb
monitoring with DAMON.
Thus define an ARM64 specific huge_ptep_get() implementation, that will
take into account any subpages' dirty or young bits for CONT-PTE/PMD size
hugetlb page, for those functions that want to check the dirty and young
flags of a hugetlb page.
[1] https://lore.kernel.org/linux-mm/85bd80b4-b4fd-0d3f-a2e5-149559f2f387@oracle.com/
Suggested-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
---
arch/arm64/include/asm/hugetlb.h | 2 ++
arch/arm64/mm/hugetlbpage.c | 24 ++++++++++++++++++++++++
2 files changed, 26 insertions(+)
diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h
index 616b2ca..1fd2846 100644
--- a/arch/arm64/include/asm/hugetlb.h
+++ b/arch/arm64/include/asm/hugetlb.h
@@ -44,6 +44,8 @@ extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
#define __HAVE_ARCH_HUGE_PTE_CLEAR
extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, unsigned long sz);
+#define __HAVE_ARCH_HUGE_PTEP_GET
+extern pte_t huge_ptep_get(pte_t *ptep);
extern void set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pte, unsigned long sz);
#define set_huge_swap_pte_at set_huge_swap_pte_at
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index be5e2f3..8fb0198 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -158,6 +158,30 @@ static inline int num_contig_ptes(unsigned long size, size_t *pgsize)
return contig_ptes;
}
+pte_t huge_ptep_get(pte_t *ptep)
+{
+ int ncontig, i;
+ size_t pgsize;
+ pte_t orig_pte = ptep_get(ptep);
+
+ if (!pte_present(orig_pte) || !pte_cont(orig_pte))
+ return orig_pte;
+
+ ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize);
+
+ for (i = 0; i < ncontig; i++, ptep++) {
+ pte_t pte = ptep_get(ptep);
+
+ if (pte_dirty(pte))
+ orig_pte = pte_mkdirty(orig_pte);
+
+ if (pte_young(pte))
+ orig_pte = pte_mkyoung(orig_pte);
+ }
+
+ return orig_pte;
+}
+
/*
* Changing some bits of contiguous entries requires us to follow a
* Break-Before-Make approach, breaking the whole contiguous set
--
1.8.3.1
^ permalink raw reply related [flat|nested] 12+ messages in thread* [PATCH 2/2] arm64/hugetlb: Implement arm64 specific huge_ptep_get()
@ 2022-05-10 11:12 ` Baolin Wang
0 siblings, 0 replies; 12+ messages in thread
From: Baolin Wang @ 2022-05-10 11:12 UTC (permalink / raw)
To: catalin.marinas, will, mike.kravetz, akpm
Cc: songmuchun, willy, anshuman.khandual, christophe.leroy,
baolin.wang, linux-arm-kernel, linux-kernel, linux-mm
Now we use huge_ptep_get() to get the pte value of a hugetlb page,
however it will only return one specific pte value for the CONT-PTE
or CONT-PMD size hugetlb on ARM64 system, which can contain seravel
continuous pte or pmd entries with same page table attributes. And it
will not take into account the subpages' dirty or young bits of a
CONT-PTE/PMD size hugetlb page.
So the huge_ptep_get() is inconsistent with huge_ptep_get_and_clear(),
which already takes account the dirty or young bits for any subpages
in this CONT-PTE/PMD size hugetlb [1]. Meanwhile we can miss dirty or
young flags statistics for hugetlb pages with current huge_ptep_get(),
such as the gather_hugetlb_stats() function, and CONT-PTE/PMD hugetlb
monitoring with DAMON.
Thus define an ARM64 specific huge_ptep_get() implementation, that will
take into account any subpages' dirty or young bits for CONT-PTE/PMD size
hugetlb page, for those functions that want to check the dirty and young
flags of a hugetlb page.
[1] https://lore.kernel.org/linux-mm/85bd80b4-b4fd-0d3f-a2e5-149559f2f387@oracle.com/
Suggested-by: Muchun Song <songmuchun@bytedance.com>
Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
---
arch/arm64/include/asm/hugetlb.h | 2 ++
arch/arm64/mm/hugetlbpage.c | 24 ++++++++++++++++++++++++
2 files changed, 26 insertions(+)
diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h
index 616b2ca..1fd2846 100644
--- a/arch/arm64/include/asm/hugetlb.h
+++ b/arch/arm64/include/asm/hugetlb.h
@@ -44,6 +44,8 @@ extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma,
#define __HAVE_ARCH_HUGE_PTE_CLEAR
extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, unsigned long sz);
+#define __HAVE_ARCH_HUGE_PTEP_GET
+extern pte_t huge_ptep_get(pte_t *ptep);
extern void set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr,
pte_t *ptep, pte_t pte, unsigned long sz);
#define set_huge_swap_pte_at set_huge_swap_pte_at
diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c
index be5e2f3..8fb0198 100644
--- a/arch/arm64/mm/hugetlbpage.c
+++ b/arch/arm64/mm/hugetlbpage.c
@@ -158,6 +158,30 @@ static inline int num_contig_ptes(unsigned long size, size_t *pgsize)
return contig_ptes;
}
+pte_t huge_ptep_get(pte_t *ptep)
+{
+ int ncontig, i;
+ size_t pgsize;
+ pte_t orig_pte = ptep_get(ptep);
+
+ if (!pte_present(orig_pte) || !pte_cont(orig_pte))
+ return orig_pte;
+
+ ncontig = num_contig_ptes(page_size(pte_page(orig_pte)), &pgsize);
+
+ for (i = 0; i < ncontig; i++, ptep++) {
+ pte_t pte = ptep_get(ptep);
+
+ if (pte_dirty(pte))
+ orig_pte = pte_mkdirty(orig_pte);
+
+ if (pte_young(pte))
+ orig_pte = pte_mkyoung(orig_pte);
+ }
+
+ return orig_pte;
+}
+
/*
* Changing some bits of contiguous entries requires us to follow a
* Break-Before-Make approach, breaking the whole contiguous set
--
1.8.3.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 12+ messages in thread* Re: [PATCH 2/2] arm64/hugetlb: Implement arm64 specific huge_ptep_get()
2022-05-10 11:12 ` Baolin Wang
@ 2022-05-10 15:59 ` Muchun Song
-1 siblings, 0 replies; 12+ messages in thread
From: Muchun Song @ 2022-05-10 15:59 UTC (permalink / raw)
To: Baolin Wang
Cc: catalin.marinas, will, mike.kravetz, akpm, willy,
anshuman.khandual, christophe.leroy, linux-arm-kernel,
linux-kernel, linux-mm
On Tue, May 10, 2022 at 07:12:53PM +0800, Baolin Wang wrote:
> Now we use huge_ptep_get() to get the pte value of a hugetlb page,
> however it will only return one specific pte value for the CONT-PTE
> or CONT-PMD size hugetlb on ARM64 system, which can contain seravel
> continuous pte or pmd entries with same page table attributes. And it
> will not take into account the subpages' dirty or young bits of a
> CONT-PTE/PMD size hugetlb page.
>
> So the huge_ptep_get() is inconsistent with huge_ptep_get_and_clear(),
> which already takes account the dirty or young bits for any subpages
> in this CONT-PTE/PMD size hugetlb [1]. Meanwhile we can miss dirty or
> young flags statistics for hugetlb pages with current huge_ptep_get(),
> such as the gather_hugetlb_stats() function, and CONT-PTE/PMD hugetlb
> monitoring with DAMON.
>
> Thus define an ARM64 specific huge_ptep_get() implementation, that will
> take into account any subpages' dirty or young bits for CONT-PTE/PMD size
> hugetlb page, for those functions that want to check the dirty and young
> flags of a hugetlb page.
>
> [1] https://lore.kernel.org/linux-mm/85bd80b4-b4fd-0d3f-a2e5-149559f2f387@oracle.com/
>
> Suggested-by: Muchun Song <songmuchun@bytedance.com>
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
LGTM. Thanks for your work.
Reviewed-by: Muchun Song <songmuchun@bytedance.com>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 2/2] arm64/hugetlb: Implement arm64 specific huge_ptep_get()
@ 2022-05-10 15:59 ` Muchun Song
0 siblings, 0 replies; 12+ messages in thread
From: Muchun Song @ 2022-05-10 15:59 UTC (permalink / raw)
To: Baolin Wang
Cc: catalin.marinas, will, mike.kravetz, akpm, willy,
anshuman.khandual, christophe.leroy, linux-arm-kernel,
linux-kernel, linux-mm
On Tue, May 10, 2022 at 07:12:53PM +0800, Baolin Wang wrote:
> Now we use huge_ptep_get() to get the pte value of a hugetlb page,
> however it will only return one specific pte value for the CONT-PTE
> or CONT-PMD size hugetlb on ARM64 system, which can contain seravel
> continuous pte or pmd entries with same page table attributes. And it
> will not take into account the subpages' dirty or young bits of a
> CONT-PTE/PMD size hugetlb page.
>
> So the huge_ptep_get() is inconsistent with huge_ptep_get_and_clear(),
> which already takes account the dirty or young bits for any subpages
> in this CONT-PTE/PMD size hugetlb [1]. Meanwhile we can miss dirty or
> young flags statistics for hugetlb pages with current huge_ptep_get(),
> such as the gather_hugetlb_stats() function, and CONT-PTE/PMD hugetlb
> monitoring with DAMON.
>
> Thus define an ARM64 specific huge_ptep_get() implementation, that will
> take into account any subpages' dirty or young bits for CONT-PTE/PMD size
> hugetlb page, for those functions that want to check the dirty and young
> flags of a hugetlb page.
>
> [1] https://lore.kernel.org/linux-mm/85bd80b4-b4fd-0d3f-a2e5-149559f2f387@oracle.com/
>
> Suggested-by: Muchun Song <songmuchun@bytedance.com>
> Signed-off-by: Baolin Wang <baolin.wang@linux.alibaba.com>
LGTM. Thanks for your work.
Reviewed-by: Muchun Song <songmuchun@bytedance.com>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 12+ messages in thread