From: SeongJae Park <sj@kernel.org>
To: gregkh@linuxfoundation.org
Cc: ryan.roberts@arm.com, akpm@linux-foundation.org, hch@lst.de,
kirill.shutemov@linux.intel.com, lstoakes@gmail.com,
rppt@kernel.org, sj@kernel.org, stable@vger.kernel.org,
urezki@gmail.com, willy@infradead.org, yuzhao@google.com,
ziy@nvidia.com, damon@lists.linux.dev, linux-mm@kvack.org,
linux-kernel@vger.kernel.org
Subject: [PATCH 5.15.y] mm/damon/ops-common: atomically test and clear young on ptes and pmds
Date: Mon, 17 Jul 2023 19:30:08 +0000 [thread overview]
Message-ID: <20230717193008.122040-1-sj@kernel.org> (raw)
In-Reply-To: <2023071613-reminder-relapse-b922@gregkh>
From: Ryan Roberts <ryan.roberts@arm.com>
commit c11d34fa139e4b0fb4249a30f37b178353533fa1 upstream.
It is racy to non-atomically read a pte, then clear the young bit, then
write it back as this could discard dirty information. Further, it is bad
practice to directly set a pte entry within a table. Instead clearing
young must go through the arch-provided helper,
ptep_test_and_clear_young() to ensure it is modified atomically and to
give the arch code visibility and allow it to check (and potentially
modify) the operation.
Link: https://lkml.kernel.org/r/20230602092949.545577-3-ryan.roberts@arm.com
Fixes: 3f49584b262c ("mm/damon: implement primitives for the virtual memory address spaces").
Signed-off-by: Ryan Roberts <ryan.roberts@arm.com>
Reviewed-by: Zi Yan <ziy@nvidia.com>
Reviewed-by: SeongJae Park <sj@kernel.org>
Reviewed-by: Mike Rapoport (IBM) <rppt@kernel.org>
Cc: Christoph Hellwig <hch@lst.de>
Cc: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
Cc: Lorenzo Stoakes <lstoakes@gmail.com>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Uladzislau Rezki (Sony) <urezki@gmail.com>
Cc: Yu Zhao <yuzhao@google.com>
Cc: <stable@vger.kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: SeongJae Park <sj@kernel.org>
---
This is a manual backport of the commit, which cannot cleanly
cherry-picked on 5.15.y[1], on 5.15.y, specifically 5.15.120.
[1] https://lore.kernel.org/stable/2023071613-reminder-relapse-b922@gregkh/
mm/damon/vaddr.c | 20 ++++++++------------
1 file changed, 8 insertions(+), 12 deletions(-)
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 1945196fd743..6ad96da15081 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -393,7 +393,7 @@ static struct page *damon_get_page(unsigned long pfn)
return page;
}
-static void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm,
+static void damon_ptep_mkold(pte_t *pte, struct vm_area_struct *vma,
unsigned long addr)
{
bool referenced = false;
@@ -402,13 +402,11 @@ static void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm,
if (!page)
return;
- if (pte_young(*pte)) {
+ if (ptep_test_and_clear_young(vma, addr, pte))
referenced = true;
- *pte = pte_mkold(*pte);
- }
#ifdef CONFIG_MMU_NOTIFIER
- if (mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE))
+ if (mmu_notifier_clear_young(vma->vm_mm, addr, addr + PAGE_SIZE))
referenced = true;
#endif /* CONFIG_MMU_NOTIFIER */
@@ -419,7 +417,7 @@ static void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm,
put_page(page);
}
-static void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm,
+static void damon_pmdp_mkold(pmd_t *pmd, struct vm_area_struct *vma,
unsigned long addr)
{
#ifdef CONFIG_TRANSPARENT_HUGEPAGE
@@ -429,13 +427,11 @@ static void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm,
if (!page)
return;
- if (pmd_young(*pmd)) {
+ if (pmdp_test_and_clear_young(vma, addr, pmd))
referenced = true;
- *pmd = pmd_mkold(*pmd);
- }
#ifdef CONFIG_MMU_NOTIFIER
- if (mmu_notifier_clear_young(mm, addr,
+ if (mmu_notifier_clear_young(vma->vm_mm, addr,
addr + ((1UL) << HPAGE_PMD_SHIFT)))
referenced = true;
#endif /* CONFIG_MMU_NOTIFIER */
@@ -462,7 +458,7 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,
}
if (pmd_huge(*pmd)) {
- damon_pmdp_mkold(pmd, walk->mm, addr);
+ damon_pmdp_mkold(pmd, walk->vma, addr);
spin_unlock(ptl);
return 0;
}
@@ -474,7 +470,7 @@ static int damon_mkold_pmd_entry(pmd_t *pmd, unsigned long addr,
pte = pte_offset_map_lock(walk->mm, pmd, addr, &ptl);
if (!pte_present(*pte))
goto out;
- damon_ptep_mkold(pte, walk->mm, addr);
+ damon_ptep_mkold(pte, walk->vma, addr);
out:
pte_unmap_unlock(pte, ptl);
return 0;
--
2.25.1
next parent reply other threads:[~2023-07-17 19:30 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <2023071613-reminder-relapse-b922@gregkh>
2023-07-17 19:30 ` SeongJae Park [this message]
2023-07-21 5:18 ` [PATCH 5.15.y] mm/damon/ops-common: atomically test and clear young on ptes and pmds Greg KH
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230717193008.122040-1-sj@kernel.org \
--to=sj@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=damon@lists.linux.dev \
--cc=gregkh@linuxfoundation.org \
--cc=hch@lst.de \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lstoakes@gmail.com \
--cc=rppt@kernel.org \
--cc=ryan.roberts@arm.com \
--cc=stable@vger.kernel.org \
--cc=urezki@gmail.com \
--cc=willy@infradead.org \
--cc=yuzhao@google.com \
--cc=ziy@nvidia.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).