archive mirror
 help / color / mirror / Atom feed
From: SeongJae Park <>
To: SeongJae Park <>
Cc: <>, SeongJae Park <>,
	<>, <>,
	<>, <>,
	<>, <>,
	<>, <>, <>,
	<>, <>,
	<>, <>, <>,
	<>, <>, <>,
	<>, <>,
	<>, <>,
	<>, <>,
	<>, <>
Subject: Re: [PATCH v5 00/14] Introduce Data Access MONitor (DAMON)
Date: Thu, 20 Feb 2020 09:17:10 +0100	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <> (raw)

On Mon, 17 Feb 2020 11:25:30 +0100 SeongJae Park <> wrote:

> From: SeongJae Park <>
> Introduction
> ============
> Memory management decisions can be improved if finer data access information is
> available.  However, because such finer information usually comes with higher
> overhead, most systems including Linux forgives the potential improvement and
> rely on only coarse information or some light-weight heuristics.  The
> pseudo-LRU and the aggressive THP promotions are such examples.
> A number of experimental data access pattern awared memory management
> optimizations (refer to 'Appendix A' for more details) say the sacrifices are
> huge.  However, none of those has successfully adopted to Linux kernel mainly
> due to the absence of a scalable and efficient data access monitoring
> mechanism.  Refer to 'Appendix B' to see the limitations of existing memory
> monitoring mechanisms.
> DAMON is a data access monitoring subsystem for the problem.  It is 1) accurate
> enough to be used for the DRAM level memory management (a straightforward
> DAMON-based optimization achieved up to 2.55x speedup), 2) light-weight enough
> to be applied online (compared to a straightforward access monitoring scheme,
> DAMON is up to 94.242.42x lighter) and 3) keeps predefined upper-bound overhead
> regardless of the size of target workloads (thus scalable).  Refer to 'Appendix
> C' if you interested in how it is possible.
> DAMON has mainly designed for the kernel's memory management mechanisms.
> However, because it is implemented as a standalone kernel module and provides
> several interfaces, it can be used by a wide range of users including kernel
> space programs, user space programs, programmers, and administrators.  DAMON
> is now supporting the monitoring only, but it will also provide simple and
> convenient data access pattern awared memory managements by itself.  Refer to
> 'Appendix D' for more detailed expected usages of DAMON.

So, with this patchset, you can (1) do data access pattern based memory
management optimizations and (2) analyze your program's dynamic data access
pattern.  For the analysis, this patchset also supports visualization of the
pattern in (1) heatmap form and distribution of the (2-1) dynamic working set
size and (2-2) monitoring overhead.

To help you better understand what DAMON can give you, I made web pages showing
the visualized dynamic data access pattern of various realistic workloads,
which I picked up from PARSEC3 and SPLASH-2X bechmark suites.  Note that
another big part of DAMON's usecases, access pattern based optimization is not
demonstrated in the pages, yet.

There are pages showing the heatmap format dynamic access pattern of each
workload for heap area[1], mmap()-ed area[2], and stack[3] area.  I splitted
the entire address space to the three area because the total address space is
too huge.

You can also show how the dynamic working set size of each workload is
distributed, and how it is changing chronologically[5].

The biggest characteristic of DAMON is its promise of the upperbound of
the monitoring overhead.  To show whether DAMON keeps the promise well, I
visualized the number of monitoring operations required for each 5
milliseconds, which is configured to not exceed 1000.  You can show the
distribution of the numbers[6] and how it changes chronologically[7].

As previously mentioned, these are only a fraction of the outputs of DAMON.  I
believe DAMON can be used for more wide and creative purposes.  Nonetheless, I
hope the pages help you understand what DAMON can give you in more intuitive


      parent reply	other threads:[~2020-02-20  8:17 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-17 10:25 [PATCH v5 00/14] Introduce Data Access MONitor (DAMON) SeongJae Park
2020-02-17 10:25 ` [PATCH v5 01/14] mm: " SeongJae Park
2020-02-17 10:25 ` [PATCH v5 02/14] mm/damon: Implement region based sampling SeongJae Park
2020-02-21 13:51   ` SeongJae Park
2020-02-17 10:25 ` [PATCH v5 03/14] mm/damon: Adaptively adjust regions SeongJae Park
2020-02-17 10:25 ` [PATCH v5 04/14] mm/damon: Apply dynamic memory mapping changes SeongJae Park
2020-02-17 10:25 ` [PATCH v5 05/14] mm/damon: Implement callbacks SeongJae Park
2020-02-17 10:25 ` [PATCH v5 06/14] mm/damon: Implement access pattern recording SeongJae Park
2020-02-17 10:25 ` [PATCH v5 07/14] mm/damon: Implement kernel space API SeongJae Park
2020-02-17 10:25 ` [PATCH v5 08/14] mm/damon: Add debugfs interface SeongJae Park
2020-02-17 10:25 ` [PATCH v5 09/14] mm/damon: Add a tracepoint for result writing SeongJae Park
2020-02-17 10:28 ` [PATCH v5 10/14] tools: Add a minimal user-space tool for DAMON SeongJae Park
2020-02-17 10:29 ` [PATCH v5 11/14] Documentation/admin-guide/mm: Add a document " SeongJae Park
2020-02-17 10:30 ` [PATCH v5 12/14] mm/damon: Add kunit tests SeongJae Park
2020-02-17 10:30 ` [PATCH v5 13/14] mm/damon: Add user selftests SeongJae Park
2020-02-17 10:31 ` [PATCH v5 14/14] MAINTAINERS: Update for DAMON SeongJae Park
2020-02-20  8:17 ` SeongJae Park [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).