Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
From: Shakeel Butt <shakeelb@google.com>
To: SeongJae Park <sjpark@amazon.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	SeongJae Park <sjpark@amazon.de>,
	 Andrea Arcangeli <aarcange@redhat.com>,
	Yang Shi <yang.shi@linux.alibaba.com>,
	acme@kernel.org,  alexander.shishkin@linux.intel.com,
	amit@kernel.org,  brendan.d.gregg@gmail.com,
	Brendan Higgins <brendanhiggins@google.com>,
	 Qian Cai <cai@lca.pw>, Colin Ian King <colin.king@canonical.com>,
	Jonathan Corbet <corbet@lwn.net>,
	 dwmw@amazon.com, jolsa@redhat.com,
	 "Kirill A. Shutemov" <kirill@shutemov.name>,
	mark.rutland@arm.com, Mel Gorman <mgorman@suse.de>,
	 Minchan Kim <minchan@kernel.org>, Ingo Molnar <mingo@redhat.com>,
	namhyung@kernel.org,  peterz@infradead.org,
	Randy Dunlap <rdunlap@infradead.org>,
	 David Rientjes <rientjes@google.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	shuah@kernel.org,  sj38.park@gmail.com,
	Vlastimil Babka <vbabka@suse.cz>,
	 Vladimir Davydov <vdavydov.dev@gmail.com>,
	Linux MM <linux-mm@kvack.org>,
	linux-doc@vger.kernel.org,  LKML <linux-kernel@vger.kernel.org>
Subject: Re: Re: Re: [PATCH v6 00/14] Introduce Data Access MONitor (DAMON)
Date: Wed, 18 Mar 2020 12:52:48 -0700
Message-ID: <CALvZod7JoOKZRGb6nnmA4ymsZCXdHetS_CPFbFeC1Rqmx4yYHw@mail.gmail.com> (raw)
In-Reply-To: <20200312104345.10032-1-sjpark@amazon.com>

On Thu, Mar 12, 2020 at 3:44 AM SeongJae Park <sjpark@amazon.com> wrote:
>
> On Thu, 12 Mar 2020 11:07:59 +0100 SeongJae Park <sjpark@amazon.com> wrote:
>
> > On Tue, 10 Mar 2020 10:21:34 -0700 Shakeel Butt <shakeelb@google.com> wrote:
> >
> > > On Mon, Feb 24, 2020 at 4:31 AM SeongJae Park <sjpark@amazon.com> wrote:
> > > >
> > > > From: SeongJae Park <sjpark@amazon.de>
> > > >
> > > > Introduction
> > > > ============
> > > >
> [...]
> > >
> > > I do want to question the actual motivation of the design followed by this work.
> > >
> > > With the already present Page Idle Tracking feature in the kernel, I
> > > can envision that the region sampling and adaptive region adjustments
> > > can be done in the user space. Due to sampling, the additional
> > > overhead will be very small and configurable.
> > >
> > > Additionally the proposed mechanism has inherent assumption of the
> > > presence of spatial locality (for virtual memory) in the monitored
> > > processes which is very workload dependent.
> > >
> > > Given that the the same mechanism can be implemented in the user space
> > > within tolerable overhead and is workload dependent, why it should be
> > > done in the kernel? What exactly is the advantage of implementing this
> > > in kernel?
> >
> > First of all, DAMON is not for only user space processes, but also for kernel
> > space core mechanisms.  Many of the core mechanisms will be able to use DAMON
> > for access pattern based optimizations, with light overhead and reasonable
> > accuracy.

Which kernel space core mechanisms? I can see memory reclaim, do you
envision some other component as well.

Let's discuss how this can interact with memory reclaim and we can see
if there is any benefit to do this in kernel.

> >
> > Implementing DAMON in user space is of course possible, but it will be
> > inefficient.  Using it from kernel space would make no sense, and it would
> > incur unnecessarily frequent kernel-user context switches, which is very
> > expensive nowadays.
>
> Forgot mentioning about the spatial locality.  Yes, it is workload dependant,
> but still pervasive in many case.  Also, many core mechanisms in kernel such as
> read-ahead or LRU are already using some similar assumptions.
>

Not sure about the LRU but yes read-ahead in several places does
assume spatial locality. However most of those are configurable and
the userspace can enable/disable the read-ahead based on the workload.

>
> If it is so problematic, you could set the maximum number of regions to the
> number of pages in the system so that each region monitors each page.
>

How will this work in the process context? Number of regions equal to
the number of mapped pages?

Basically I am trying to envision the comparison of physical memory
based monitoring (using idle page tracking) vs pid+VA based
monitoring.

Anyways I am not against your proposal. I am trying to see how to make
it more general to be applicable to more use-cases and one such
use-case which I am interested in is monitoring all the user pages on
the system for proactive reclaim purpose.

Shakeel


  reply index

Thread overview: 51+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-02-24 12:30 SeongJae Park
2020-02-24 12:30 ` [PATCH v6 01/14] mm: " SeongJae Park
2020-03-10  8:54   ` Jonathan Cameron
2020-03-10 11:50     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 02/14] mm/damon: Implement region based sampling SeongJae Park
2020-03-10  8:57   ` Jonathan Cameron
2020-03-10 11:52     ` SeongJae Park
2020-03-10 15:55       ` Jonathan Cameron
2020-03-10 16:22         ` SeongJae Park
2020-03-10 17:39           ` Jonathan Cameron
2020-03-12  9:20             ` SeongJae Park
2020-03-13 17:29   ` Jonathan Cameron
2020-03-13 20:16     ` SeongJae Park
2020-03-17 11:32       ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 03/14] mm/damon: Adaptively adjust regions SeongJae Park
2020-03-10  8:57   ` Jonathan Cameron
2020-03-10 11:53     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 04/14] mm/damon: Apply dynamic memory mapping changes SeongJae Park
2020-03-10  9:00   ` Jonathan Cameron
2020-03-10 11:53     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 05/14] mm/damon: Implement callbacks SeongJae Park
2020-03-10  9:01   ` Jonathan Cameron
2020-03-10 11:55     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 06/14] mm/damon: Implement access pattern recording SeongJae Park
2020-03-10  9:01   ` Jonathan Cameron
2020-03-10 11:55     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 07/14] mm/damon: Implement kernel space API SeongJae Park
2020-03-10  9:01   ` Jonathan Cameron
2020-03-10 11:56     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 08/14] mm/damon: Add debugfs interface SeongJae Park
2020-03-10  9:02   ` Jonathan Cameron
2020-03-10 11:56     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 09/14] mm/damon: Add a tracepoint for result writing SeongJae Park
2020-03-10  9:03   ` Jonathan Cameron
2020-03-10 11:57     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 10/14] tools: Add a minimal user-space tool for DAMON SeongJae Park
2020-02-24 12:30 ` [PATCH v6 11/14] Documentation/admin-guide/mm: Add a document " SeongJae Park
2020-03-10  9:03   ` Jonathan Cameron
2020-03-10 11:57     ` SeongJae Park
2020-02-24 12:30 ` [PATCH v6 12/14] mm/damon: Add kunit tests SeongJae Park
2020-02-24 12:30 ` [PATCH v6 13/14] mm/damon: Add user selftests SeongJae Park
2020-02-24 12:30 ` [PATCH v6 14/14] MAINTAINERS: Update for DAMON SeongJae Park
2020-03-02 11:35 ` [PATCH v6 00/14] Introduce Data Access MONitor (DAMON) SeongJae Park
2020-03-09 10:23   ` SeongJae Park
2020-03-10 17:21 ` Shakeel Butt
2020-03-12 10:07   ` SeongJae Park
2020-03-12 10:43     ` SeongJae Park
2020-03-18 19:52       ` Shakeel Butt [this message]
2020-03-19  9:03         ` SeongJae Park
2020-03-23 17:29           ` Shakeel Butt
2020-03-24  8:34             ` SeongJae Park

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CALvZod7JoOKZRGb6nnmA4ymsZCXdHetS_CPFbFeC1Rqmx4yYHw@mail.gmail.com \
    --to=shakeelb@google.com \
    --cc=aarcange@redhat.com \
    --cc=acme@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=amit@kernel.org \
    --cc=brendan.d.gregg@gmail.com \
    --cc=brendanhiggins@google.com \
    --cc=cai@lca.pw \
    --cc=colin.king@canonical.com \
    --cc=corbet@lwn.net \
    --cc=dwmw@amazon.com \
    --cc=jolsa@redhat.com \
    --cc=kirill@shutemov.name \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mark.rutland@arm.com \
    --cc=mgorman@suse.de \
    --cc=minchan@kernel.org \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rdunlap@infradead.org \
    --cc=rientjes@google.com \
    --cc=rostedt@goodmis.org \
    --cc=shuah@kernel.org \
    --cc=sj38.park@gmail.com \
    --cc=sjpark@amazon.com \
    --cc=sjpark@amazon.de \
    --cc=vbabka@suse.cz \
    --cc=vdavydov.dev@gmail.com \
    --cc=yang.shi@linux.alibaba.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org
	public-inbox-index linux-mm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git