All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH v2 0/3] mm/damon: Profiling enhancements for DAMON
@ 2024-03-18 13:28 Aravinda Prasad
  2024-03-18 13:28 ` [PATCH v2 1/3] mm/damon: mm infrastructure support Aravinda Prasad
                   ` (4 more replies)
  0 siblings, 5 replies; 15+ messages in thread
From: Aravinda Prasad @ 2024-03-18 13:28 UTC (permalink / raw)
  To: damon, linux-mm, sj, linux-kernel
  Cc: aravinda.prasad, s2322819, sandeep4.kumar, ying.huang,
	dave.hansen, dan.j.williams, sreenivas.subramoney,
	antti.kervinen, alexander.kanevskiy

DAMON randomly samples one or more pages in every region and tracks
accesses to them using the ACCESSED bit in PTE (or PMD for 2MB pages).
When the region size is large (e.g., several GBs), which is common
for large footprint applications, detecting whether the region is
accessed or not completely depends on whether the pages that are
actively accessed in the region are picked during random sampling.
If such pages are not picked for sampling, DAMON fails to identify
the region as accessed. However, increasing the sampling rate or
increasing the number of regions increases CPU overheads of kdamond.

This patch proposes profiling different levels of the application’s
page table tree to detect whether a region is accessed or not. This
patch set is based on the observation that, when the accessed bit for a
page is set, the accessed bits at the higher levels of the page table
tree (PMD/PUD/PGD) corresponding to the path of the page table walk
are also set. Hence, it is efficient to check the accessed bits at
the higher levels of the page table tree to detect whether a region
is accessed or not. For example, if the access bit for a PUD entry
is set, then one or more pages in the 1GB PUD subtree is accessed as
each PUD entry covers 1GB mapping. Hence, instead of sampling
thousands of 4K/2M pages to detect accesses in a large region, 
sampling at the higher level of page table tree is faster and efficient.

This patch set is based on 6.8-rc5 kernel (commit: f48159f8, mm-unstable
tree)

Changes since v1 [1]
====================

 - Added support for 5-level page table tree
 - Split the patch to mm infrastructure changes and DAMON enhancements
 - Code changes as per comments on v1
 - Added kerneldoc comments

[1] https://lkml.org/lkml/2023/12/15/272
 
Evaluation:

- MASIM benchmark with 1GB, 10GB, 100GB footprint with 10% hot data
  and 5TB with 10GB hot data.
- DAMON: 5ms sampling, 200ms aggregation interval. Rest all
  parameters set to default value.
- DAMON+PTP: Page table profiling applied to DAMON with the above
  parameters.

Profiling efficiency in detecting hot data:

Footprint	1GB	10GB	100GB	5TB
---------------------------------------------
DAMON		>90%	<50%	 ~0%	  0%
DAMON+PTP	>90%	>90%	>90%	>90%

CPU overheads (in billion cycles) for kdamond:

Footprint	1GB	10GB	100GB	5TB
---------------------------------------------
DAMON		1.15	19.53	3.52	9.55
DAMON+PTP	0.83	 3.20	1.27	2.55

A detailed explanation and evaluation can be found in the arXiv paper:
https://arxiv.org/pdf/2311.10275.pdf


Aravinda Prasad (3):
  mm/damon: mm infrastructure support
  mm/damon: profiling enhancement
  mm/damon: documentation updates

 Documentation/mm/damon/design.rst |  42 ++++++
 arch/x86/include/asm/pgtable.h    |  20 +++
 arch/x86/mm/pgtable.c             |  28 +++-
 include/linux/mmu_notifier.h      |  36 +++++
 include/linux/pgtable.h           |  79 ++++++++++
 mm/damon/vaddr.c                  | 233 ++++++++++++++++++++++++++++--
 6 files changed, 424 insertions(+), 14 deletions(-)

-- 
2.21.3


^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2024-03-25  7:50 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-03-18 13:28 [PATCH v2 0/3] mm/damon: Profiling enhancements for DAMON Aravinda Prasad
2024-03-18 13:28 ` [PATCH v2 1/3] mm/damon: mm infrastructure support Aravinda Prasad
2024-03-18 20:27   ` kernel test robot
2024-03-18 13:28 ` [PATCH v2 2/3] mm/damon: profiling enhancement Aravinda Prasad
2024-03-18 18:23   ` kernel test robot
2024-03-18 21:59   ` kernel test robot
2024-03-18 13:28 ` [PATCH v2 3/3] mm/damon: documentation updates Aravinda Prasad
2024-03-19  0:51 ` [PATCH v2 0/3] mm/damon: Profiling enhancements for DAMON Yu Zhao
2024-03-19  5:20 ` SeongJae Park
2024-03-19 10:56   ` Prasad, Aravinda
2024-03-20 12:31   ` Prasad, Aravinda
2024-03-21 23:10     ` SeongJae Park
2024-03-22 12:12       ` Prasad, Aravinda
2024-03-22 18:32         ` SeongJae Park
2024-03-25  7:50           ` Prasad, Aravinda

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.