From: Yu Zhao <yuzhao@google.com>
To: Andrew Morton <akpm@linux-foundation.org>,
Linus Torvalds <torvalds@linux-foundation.org>
Cc: Andi Kleen <ak@linux.intel.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
Hillf Danton <hdanton@sina.com>, Jens Axboe <axboe@kernel.dk>,
Jesse Barnes <jsbarnes@google.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Jonathan Corbet <corbet@lwn.net>,
Matthew Wilcox <willy@infradead.org>,
Mel Gorman <mgorman@suse.de>,
Michael Larabel <Michael@michaellarabel.com>,
Michal Hocko <mhocko@kernel.org>, Rik van Riel <riel@surriel.com>,
Vlastimil Babka <vbabka@suse.cz>, Will Deacon <will@kernel.org>,
Ying Huang <ying.huang@intel.com>,
linux-arm-kernel@lists.infradead.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
page-reclaim@google.com, x86@kernel.org,
Yu Zhao <yuzhao@google.com>,
Konstantin Kharlamov <Hi-Angel@yandex.ru>
Subject: [PATCH v6 2/9] mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG
Date: Tue, 4 Jan 2022 13:22:21 -0700 [thread overview]
Message-ID: <20220104202227.2903605-3-yuzhao@google.com> (raw)
In-Reply-To: <20220104202227.2903605-1-yuzhao@google.com>
Some architectures support the accessed bit in non-leaf PMD entries,
e.g., x86_64 sets the accessed bit in a non-leaf PMD entry when using
it as part of linear address translation [1]. Page table walkers that
clear the accessed bit may use this feature to reduce their search
space.
Although an inline function is preferable, this capability is added as
a configuration option for the consistency with the existing macros.
[1]: Intel 64 and IA-32 Architectures Software Developer's Manual
Volume 3 (June 2021), section 4.8
Signed-off-by: Yu Zhao <yuzhao@google.com>
Tested-by: Konstantin Kharlamov <Hi-Angel@yandex.ru>
---
arch/Kconfig | 9 +++++++++
arch/x86/Kconfig | 1 +
arch/x86/include/asm/pgtable.h | 3 ++-
arch/x86/mm/pgtable.c | 5 ++++-
include/linux/pgtable.h | 4 ++--
5 files changed, 18 insertions(+), 4 deletions(-)
diff --git a/arch/Kconfig b/arch/Kconfig
index d3c4ab249e9c..10f564340f79 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -1312,6 +1312,15 @@ config ARCH_HAS_PARANOID_L1D_FLUSH
config DYNAMIC_SIGFRAME
bool
+config ARCH_HAS_NONLEAF_PMD_YOUNG
+ bool
+ depends on PGTABLE_LEVELS > 2
+ help
+ Architectures that select this option are capable of setting the
+ accessed bit in non-leaf PMD entries when using them as part of linear
+ address translations. Page table walkers that clear the accessed bit
+ may use this feature to reduce their search space.
+
source "kernel/gcov/Kconfig"
source "scripts/gcc-plugins/Kconfig"
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 5c2ccb85f2ef..5a4843242f09 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -85,6 +85,7 @@ config X86
select ARCH_HAS_PMEM_API if X86_64
select ARCH_HAS_PTE_DEVMAP if X86_64
select ARCH_HAS_PTE_SPECIAL
+ select ARCH_HAS_NONLEAF_PMD_YOUNG if X86_64
select ARCH_HAS_UACCESS_FLUSHCACHE if X86_64
select ARCH_HAS_COPY_MC if X86_64
select ARCH_HAS_SET_MEMORY
diff --git a/arch/x86/include/asm/pgtable.h b/arch/x86/include/asm/pgtable.h
index c60b16f8b741..36205ec0acac 100644
--- a/arch/x86/include/asm/pgtable.h
+++ b/arch/x86/include/asm/pgtable.h
@@ -817,7 +817,8 @@ static inline unsigned long pmd_page_vaddr(pmd_t pmd)
static inline int pmd_bad(pmd_t pmd)
{
- return (pmd_flags(pmd) & ~_PAGE_USER) != _KERNPG_TABLE;
+ return (pmd_flags(pmd) & ~(_PAGE_USER | _PAGE_ACCESSED)) !=
+ (_KERNPG_TABLE & ~_PAGE_ACCESSED);
}
static inline unsigned long pages_to_mb(unsigned long npg)
diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c
index 3481b35cb4ec..a224193d84bf 100644
--- a/arch/x86/mm/pgtable.c
+++ b/arch/x86/mm/pgtable.c
@@ -550,7 +550,7 @@ int ptep_test_and_clear_young(struct vm_area_struct *vma,
return ret;
}
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG)
int pmdp_test_and_clear_young(struct vm_area_struct *vma,
unsigned long addr, pmd_t *pmdp)
{
@@ -562,6 +562,9 @@ int pmdp_test_and_clear_young(struct vm_area_struct *vma,
return ret;
}
+#endif
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
int pudp_test_and_clear_young(struct vm_area_struct *vma,
unsigned long addr, pud_t *pudp)
{
diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h
index 53bd6a26918f..b51f939a73f7 100644
--- a/include/linux/pgtable.h
+++ b/include/linux/pgtable.h
@@ -211,7 +211,7 @@ static inline int ptep_test_and_clear_young(struct vm_area_struct *vma,
#endif
#ifndef __HAVE_ARCH_PMDP_TEST_AND_CLEAR_YOUNG
-#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG)
static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma,
unsigned long address,
pmd_t *pmdp)
@@ -232,7 +232,7 @@ static inline int pmdp_test_and_clear_young(struct vm_area_struct *vma,
BUILD_BUG();
return 0;
}
-#endif /* CONFIG_TRANSPARENT_HUGEPAGE */
+#endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG */
#endif
#ifndef __HAVE_ARCH_PTEP_CLEAR_YOUNG_FLUSH
--
2.34.1.448.ga2b2bfdf31-goog
next prev parent reply other threads:[~2022-01-04 20:23 UTC|newest]
Thread overview: 111+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-04 20:22 [PATCH v6 0/9] Multigenerational LRU Framework Yu Zhao
2022-01-04 20:22 ` [PATCH v6 1/9] mm: x86, arm64: add arch_has_hw_pte_young() Yu Zhao
2022-01-05 10:45 ` Will Deacon
2022-01-05 20:47 ` Yu Zhao
2022-01-06 10:30 ` Will Deacon
2022-01-07 7:25 ` Yu Zhao
2022-01-11 14:19 ` Will Deacon
2022-01-11 22:27 ` Yu Zhao
2022-01-04 20:22 ` Yu Zhao [this message]
2022-01-04 21:24 ` [PATCH v6 2/9] mm: x86: add CONFIG_ARCH_HAS_NONLEAF_PMD_YOUNG Linus Torvalds
2022-01-04 20:22 ` [PATCH v6 3/9] mm/vmscan.c: refactor shrink_node() Yu Zhao
2022-01-04 20:22 ` [PATCH v6 4/9] mm: multigenerational lru: groundwork Yu Zhao
2022-01-04 21:34 ` Linus Torvalds
2022-01-11 8:16 ` Aneesh Kumar K.V
2022-01-12 2:16 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 5/9] mm: multigenerational lru: mm_struct list Yu Zhao
2022-01-07 9:06 ` Michal Hocko
2022-01-08 0:19 ` Yu Zhao
2022-01-10 15:21 ` Michal Hocko
2022-01-12 8:08 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 6/9] mm: multigenerational lru: aging Yu Zhao
2022-01-06 16:06 ` Michal Hocko
2022-01-06 21:27 ` Yu Zhao
2022-01-07 8:43 ` Michal Hocko
2022-01-07 21:12 ` Yu Zhao
2022-01-06 16:12 ` Michal Hocko
2022-01-06 21:41 ` Yu Zhao
2022-01-07 8:55 ` Michal Hocko
2022-01-07 9:00 ` Michal Hocko
2022-01-10 3:58 ` Yu Zhao
2022-01-10 14:37 ` Michal Hocko
2022-01-13 9:43 ` Yu Zhao
2022-01-13 12:02 ` Michal Hocko
2022-01-19 6:31 ` Yu Zhao
2022-01-19 9:44 ` Michal Hocko
2022-01-10 15:01 ` Michal Hocko
2022-01-10 16:01 ` Vlastimil Babka
2022-01-10 16:25 ` Michal Hocko
2022-01-11 23:16 ` Yu Zhao
2022-01-12 10:28 ` Michal Hocko
2022-01-13 9:25 ` Yu Zhao
2022-01-07 13:11 ` Michal Hocko
2022-01-07 23:36 ` Yu Zhao
2022-01-10 15:35 ` Michal Hocko
2022-01-11 1:18 ` Yu Zhao
2022-01-11 9:00 ` Michal Hocko
[not found] ` <1641900108.61dd684cb0e59@mail.inbox.lv>
2022-01-11 12:15 ` Michal Hocko
2022-01-13 17:00 ` Alexey Avramov
2022-01-11 14:22 ` Alexey Avramov
2022-01-07 14:44 ` Michal Hocko
2022-01-10 4:47 ` Yu Zhao
2022-01-10 10:54 ` Michal Hocko
2022-01-19 7:04 ` Yu Zhao
2022-01-19 9:42 ` Michal Hocko
2022-01-23 21:28 ` Yu Zhao
2022-01-24 14:01 ` Michal Hocko
2022-01-10 16:57 ` Michal Hocko
2022-01-12 1:01 ` Yu Zhao
2022-01-12 10:17 ` Michal Hocko
2022-01-12 23:43 ` Yu Zhao
2022-01-13 11:57 ` Michal Hocko
2022-01-23 21:40 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 7/9] mm: multigenerational lru: eviction Yu Zhao
2022-01-11 10:37 ` Aneesh Kumar K.V
2022-01-12 8:05 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 8/9] mm: multigenerational lru: user interface Yu Zhao
2022-01-10 10:27 ` Mike Rapoport
2022-01-12 8:35 ` Yu Zhao
2022-01-12 10:31 ` Michal Hocko
2022-01-12 15:45 ` Mike Rapoport
2022-01-13 9:47 ` Yu Zhao
2022-01-13 10:31 ` Aneesh Kumar K.V
2022-01-13 23:02 ` Yu Zhao
2022-01-14 5:20 ` Aneesh Kumar K.V
2022-01-14 6:50 ` Yu Zhao
2022-01-04 20:22 ` [PATCH v6 9/9] mm: multigenerational lru: Kconfig Yu Zhao
2022-01-04 21:39 ` Linus Torvalds
2022-01-04 20:22 ` [PATCH v6 0/9] Multigenerational LRU Framework Yu Zhao
2022-01-04 20:30 ` Yu Zhao
2022-01-04 21:43 ` Linus Torvalds
2022-01-05 21:12 ` Yu Zhao
2022-01-07 9:38 ` Michal Hocko
2022-01-07 18:45 ` Yu Zhao
2022-01-10 15:39 ` Michal Hocko
2022-01-10 22:04 ` Yu Zhao
2022-01-10 22:46 ` Jesse Barnes
2022-01-11 1:41 ` Linus Torvalds
2022-01-11 10:40 ` Michal Hocko
2022-01-11 8:41 ` Yu Zhao
2022-01-11 8:53 ` Holger Hoffstätte
2022-01-11 16:04 ` Shuang Zhai
2022-01-12 1:46 ` Suleiman Souhlal
2022-01-12 6:07 ` Sofia Trinh
2022-01-18 9:21 ` Yu Zhao
2022-01-18 9:36 ` Donald Carr
2022-01-19 20:19 ` Steven Barrett
2022-01-19 22:25 ` Brian Geffon
2022-01-05 2:44 ` Shuang Zhai
2022-01-05 8:55 ` SeongJae Park
2022-01-05 10:53 ` Yu Zhao
2022-01-05 11:12 ` Borislav Petkov
2022-01-05 11:25 ` SeongJae Park
2022-01-05 21:06 ` Yu Zhao
2022-01-10 14:49 ` Alexey Avramov
2022-01-11 10:24 ` Alexey Avramov
2022-01-12 20:56 ` Oleksandr Natalenko
2022-01-13 8:59 ` Yu Zhao
2022-01-23 5:43 ` Barry Song
2022-01-25 6:48 ` Yu Zhao
2022-01-28 8:54 ` Barry Song
2022-02-08 9:16 ` Yu Zhao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220104202227.2903605-3-yuzhao@google.com \
--to=yuzhao@google.com \
--cc=Hi-Angel@yandex.ru \
--cc=Michael@michaellarabel.com \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=axboe@kernel.dk \
--cc=catalin.marinas@arm.com \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=hannes@cmpxchg.org \
--cc=hdanton@sina.com \
--cc=jsbarnes@google.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@kernel.org \
--cc=page-reclaim@google.com \
--cc=riel@surriel.com \
--cc=torvalds@linux-foundation.org \
--cc=vbabka@suse.cz \
--cc=will@kernel.org \
--cc=willy@infradead.org \
--cc=x86@kernel.org \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).