Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
From: SeongJae Park <sjpark@amazon.com>
To: SeongJae Park <sjpark@amazon.com>
Cc: <Jonathan.Cameron@Huawei.com>, <aarcange@redhat.com>,
	<acme@kernel.org>, <alexander.shishkin@linux.intel.com>,
	<amit@kernel.org>, <benh@kernel.crashing.org>,
	<brendan.d.gregg@gmail.com>, <brendanhiggins@google.com>,
	<cai@lca.pw>, <colin.king@canonical.com>, <corbet@lwn.net>,
	<david@redhat.com>, <dwmw@amazon.com>, <fan.du@intel.com>,
	<foersleo@amazon.de>, <gthelen@google.com>, <irogers@google.com>,
	<jolsa@redhat.com>, <kirill@shutemov.name>,
	<mark.rutland@arm.com>, <mgorman@suse.de>, <minchan@kernel.org>,
	<mingo@redhat.com>, <namhyung@kernel.org>, <peterz@infradead.org>,
	<rdunlap@infradead.org>, <riel@surriel.com>,
	<rientjes@google.com>, <rostedt@goodmis.org>, <rppt@kernel.org>,
	<sblbir@amazon.com>, <shakeelb@google.com>, <shuah@kernel.org>,
	<sj38.park@gmail.com>, <snu@amazon.de>, <vbabka@suse.cz>,
	<vdavydov.dev@gmail.com>, <yang.shi@linux.alibaba.com>,
	<ying.huang@intel.com>, <zgf574564920@gmail.com>,
	<linux-damon@amazon.com>, <linux-mm@kvack.org>,
	<linux-doc@vger.kernel.org>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v20 00/15] Introduce Data Access MONitor (DAMON)
Date: Mon, 31 Aug 2020 13:22:35 +0200
Message-ID: <20200831112235.2675-1-sjpark@amazon.com> (raw)
In-Reply-To: <20200820072738.27602-1-sjpark@amazon.com>

On Thu, 20 Aug 2020 09:27:38 +0200 SeongJae Park <sjpark@amazon.com> wrote:

> On Mon, 17 Aug 2020 12:51:22 +0200 SeongJae Park <sjpark@amazon.com> wrote:
> 
> > From: SeongJae Park <sjpark@amazon.de>
> > 
> > Changes from Previous Version
> > =============================
> > 
> > - Place 'CREATE_TRACE_POINTS' after '#include' statements (Steven Rostedt)
> > - Support large record file (Alkaid)
> > - Place 'put_pid()' of virtual monitoring targets in 'cleanup' callback
> > - Avoid conflict between concurrent DAMON users
> > - Update evaluation result document
> > 
> > Introduction
> > ============
> > 
> > DAMON is a data access monitoring framework subsystem for the Linux kernel.
> > The core mechanisms of DAMON called 'region based sampling' and 'adaptive
> > regions adjustment' (refer to 'mechanisms.rst' in the 11th patch of this
> > patchset for the detail) make it
> > 
> >  - accurate (The monitored information is useful for DRAM level memory
> >    management. It might not appropriate for Cache-level accuracy, though.),
> >  - light-weight (The monitoring overhead is low enough to be applied online
> >    while making no impact on the performance of the target workloads.), and
> >  - scalable (the upper-bound of the instrumentation overhead is controllable
> >    regardless of the size of target workloads.).
> > 
> > Using this framework, therefore, the kernel's core memory management mechanisms
> > such as reclamation and THP can be optimized for better memory management.  The
> > experimental memory management optimization works that incurring high
> > instrumentation overhead will be able to have another try.  In user space,
> > meanwhile, users who have some special workloads will be able to write
> > personalized tools or applications for deeper understanding and specialized
> > optimizations of their systems.
> 
> DAMON will be presented in the next week LPC[1].  To be prepared for a screen
> sharing error (if I get no such error, I will do a live-demo), I recorded a
> simple demo video.  I would like to share it here to help your easier
> understanding of DAMON.
> 
>     https://youtu.be/l63eqbVBZRY
> 
> [1] https://linuxplumbersconf.org/event/7/contributions/659/

During the session, I introduced the list of future works and asked the
audiences to vote for the priority of the tasks:
https://youtu.be/jOBkKMA0uF0?t=13253

To summarize here, the tasks are (highest priority first):

1. Make current DAMON patchset series merged in the mainline (6 votes)
2. User space interface improvement (4 votes)
 - Multiple monitoring contexts
 - Charging of the monitoring threads' CPU usage
3. Support more address spaces (2 votes)
 - Cgroups, cached pages, specific file-backed pages, swap slots, ...
3. DAMON-based MM optimizations (2 votes)
 - Page reclaim, THP, compaction, NUMA balancing, ...
4. Optimize for special use-cases (1 vote)
 - Page granularity monitoring, accessed-or-not monitoring, ...

So, I'd like to focus on polishing current patchset so that it could be merged
in.  For that, I'd like to ask your more reviews.

While waiting for the reviews, I will start implementing other future features
that received many votes.  The support of multiple monitoring contexts for the
user space would be the first one.  Once the implementation is finished, I will
post it as separated RFC patchset (the user space interface will be compatible
with current one).

Any comment is welcome.


Thanks,
SeongJae Park


  reply index

Thread overview: 22+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-17 10:51 SeongJae Park
2020-08-17 10:51 ` [PATCH v20 01/15] mm: " SeongJae Park
2020-08-17 10:51 ` [PATCH v20 02/15] mm/damon: Implement region based sampling SeongJae Park
2020-08-17 10:51 ` [PATCH v20 03/15] mm/damon: Adaptively adjust regions SeongJae Park
2020-08-17 10:51 ` [PATCH v20 04/15] mm/damon: Track dynamic monitoring target regions update SeongJae Park
2020-08-17 10:51 ` [PATCH v20 05/15] mm/idle_page_tracking: Make PG_(idle|young) reusable SeongJae Park
2020-08-17 10:51 ` [PATCH v20 06/15] mm/damon: Implement callbacks for the virtual memory address spaces SeongJae Park
2020-08-17 10:51 ` [PATCH v20 07/15] mm/damon: Implement access pattern recording SeongJae Park
2020-08-17 10:51 ` [PATCH v20 08/15] mm/damon: Add a tracepoint SeongJae Park
2020-08-17 10:51 ` [PATCH v20 09/15] mm/damon: Implement a debugfs interface SeongJae Park
2020-08-17 10:51 ` [PATCH v20 10/15] damon/debugfs: Support pidfd target id SeongJae Park
2020-08-17 10:51 ` [PATCH v20 11/15] tools: Introduce a minimal user-space tool for DAMON SeongJae Park
2020-08-17 10:51 ` [PATCH v20 12/15] Documentation: Add documents " SeongJae Park
2020-08-17 10:51 ` [PATCH v20 13/15] mm/damon: Add kunit tests SeongJae Park
2020-08-17 10:51 ` [PATCH v20 14/15] mm/damon: Add user space selftests SeongJae Park
2020-08-17 10:51 ` [PATCH v20 15/15] MAINTAINERS: Update for DAMON SeongJae Park
2020-08-20  7:27 ` [PATCH v20 00/15] Introduce Data Access MONitor (DAMON) SeongJae Park
2020-08-31 11:22   ` SeongJae Park [this message]
2020-09-25 14:59     ` SeongJae Park
2020-09-09  8:27 ` SeongJae Park
2020-09-23 17:04 ` Shakeel Butt
2020-09-24  6:39   ` SeongJae Park

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200831112235.2675-1-sjpark@amazon.com \
    --to=sjpark@amazon.com \
    --cc=Jonathan.Cameron@Huawei.com \
    --cc=aarcange@redhat.com \
    --cc=acme@kernel.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=amit@kernel.org \
    --cc=benh@kernel.crashing.org \
    --cc=brendan.d.gregg@gmail.com \
    --cc=brendanhiggins@google.com \
    --cc=cai@lca.pw \
    --cc=colin.king@canonical.com \
    --cc=corbet@lwn.net \
    --cc=david@redhat.com \
    --cc=dwmw@amazon.com \
    --cc=fan.du@intel.com \
    --cc=foersleo@amazon.de \
    --cc=gthelen@google.com \
    --cc=irogers@google.com \
    --cc=jolsa@redhat.com \
    --cc=kirill@shutemov.name \
    --cc=linux-damon@amazon.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mark.rutland@arm.com \
    --cc=mgorman@suse.de \
    --cc=minchan@kernel.org \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rdunlap@infradead.org \
    --cc=riel@surriel.com \
    --cc=rientjes@google.com \
    --cc=rostedt@goodmis.org \
    --cc=rppt@kernel.org \
    --cc=sblbir@amazon.com \
    --cc=shakeelb@google.com \
    --cc=shuah@kernel.org \
    --cc=sj38.park@gmail.com \
    --cc=snu@amazon.de \
    --cc=vbabka@suse.cz \
    --cc=vdavydov.dev@gmail.com \
    --cc=yang.shi@linux.alibaba.com \
    --cc=ying.huang@intel.com \
    --cc=zgf574564920@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org
	public-inbox-index linux-mm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git