[LSF/MM/BPF TOPIC] DAMON Updates and Future Plans

[LSF/MM/BPF TOPIC] DAMON Updates and Future Plans

[SeongJae Park]

Hi all,


DAMON has merged into mainline as a data access monitoring tool that equips a
best-effort overhead-accuracy tradeoff mechanism, and then extended for data
access-aware system operations.  I'd like to briefly introduce current state of
DAMON and share/discuss about 2023 plans including below.

- Finer and easier-to-use DAMOS tuning
  - tuning aggressiveness based on user or kernel feed (e.g., QPS or PSI)
- Merging DAMON user space tool into the mainline
- Extending DAMON
  - Page-granularity monitoring
    - LRU-lists based page-granulariy monitoring
  - CPU-specific access monitoring
  - Read/Write-only access monitoring
- More DAMON-based Operation Schemes
  - Tiered memory management
  - THP memory footprint reduction
  - NUMA balancing

I hope to hear concerns/interests about the plans for prioritizing each work
items and get some suggestions of future works and collaboration with other
kernel subsystems/hackers.


Thanks,
SJ

RE: [LSF/MM/BPF TOPIC] DAMON Updates and Future Plans

[Pulavarty, Badari]

Hi SJ,

We discussed this earlier. 

- Are there plans to make DAMON monitor/manage cgroup instead of individual pids? 

Thanks,
Badari

-----Original Message-----
From: SeongJae Park <sj@kernel.org> 
Sent: Monday, February 13, 2023 4:33 PM
To: lsf-pc@lists.linux-foundation.org
Cc: SeongJae Park <sj@kernel.org>; damon@lists.linux.dev; linux-mm@kvack.org; linux-kernel@vger.kernel.org
Subject: [LSF/MM/BPF TOPIC] DAMON Updates and Future Plans

Hi all,


DAMON has merged into mainline as a data access monitoring tool that equips a best-effort overhead-accuracy tradeoff mechanism, and then extended for data access-aware system operations.  I'd like to briefly introduce current state of DAMON and share/discuss about 2023 plans including below.

- Finer and easier-to-use DAMOS tuning
  - tuning aggressiveness based on user or kernel feed (e.g., QPS or PSI)
- Merging DAMON user space tool into the mainline
- Extending DAMON
  - Page-granularity monitoring
    - LRU-lists based page-granulariy monitoring
  - CPU-specific access monitoring
  - Read/Write-only access monitoring
- More DAMON-based Operation Schemes
  - Tiered memory management
  - THP memory footprint reduction
  - NUMA balancing

I hope to hear concerns/interests about the plans for prioritizing each work items and get some suggestions of future works and collaboration with other kernel subsystems/hackers.


Thanks,
SJ

RE: [LSF/MM/BPF TOPIC] DAMON Updates and Future Plans

[SeongJae Park]

Hi Badari,

On Wed, 1 Mar 2023 00:24:54 +0000 "Pulavarty, Badari" <badari.pulavarty@intel.com> wrote:

> Hi SJ,
> 
> We discussed this earlier. 
> 
> - Are there plans to make DAMON monitor/manage cgroup instead of individual pids? 

Yes.

For moniroting cgroups, there are only rough ideas at the moment, and no much
progress has made so far.  I hope to discuss about real needs of this feature
(I didn't make much progress here mainly because there was no many people
pushing this) and the best way for implementing it during the session if
possible.

For monitoring-based memory management of cgroups (via DAMOS), however, DAMOS
provides cgroup-based management action target memory filtering[1].  Using the
feature, you can do cgroups-based access-aware memory management.  The patchset
has merged into the mainline a few days ago.

[1] https://lore.kernel.org/damon/20221205230830.144349-1-sj@kernel.org/


Thanks,
SJ

> 
> Thanks,
> Badari
> 
> -----Original Message-----
> From: SeongJae Park <sj@kernel.org> 
> Sent: Monday, February 13, 2023 4:33 PM
> To: lsf-pc@lists.linux-foundation.org
> Cc: SeongJae Park <sj@kernel.org>; damon@lists.linux.dev; linux-mm@kvack.org; linux-kernel@vger.kernel.org
> Subject: [LSF/MM/BPF TOPIC] DAMON Updates and Future Plans
> 
> Hi all,
> 
> 
> DAMON has merged into mainline as a data access monitoring tool that equips a best-effort overhead-accuracy tradeoff mechanism, and then extended for data access-aware system operations.  I'd like to briefly introduce current state of DAMON and share/discuss about 2023 plans including below.
> 
> - Finer and easier-to-use DAMOS tuning
>   - tuning aggressiveness based on user or kernel feed (e.g., QPS or PSI)
> - Merging DAMON user space tool into the mainline
> - Extending DAMON
>   - Page-granularity monitoring
>     - LRU-lists based page-granulariy monitoring
>   - CPU-specific access monitoring
>   - Read/Write-only access monitoring
> - More DAMON-based Operation Schemes
>   - Tiered memory management
>   - THP memory footprint reduction
>   - NUMA balancing
> 
> I hope to hear concerns/interests about the plans for prioritizing each work items and get some suggestions of future works and collaboration with other kernel subsystems/hackers.
> 
> 
> Thanks,
> SJ
> 

Re: [LSF/MM/BPF TOPIC] DAMON Updates and Future Plans

[SeongJae Park]

Hello,

On Tue, 14 Feb 2023 00:33:28 +0000 SeongJae Park <sj@kernel.org> wrote:

> Hi all,
> 
> 
> DAMON has merged into mainline as a data access monitoring tool that equips a
> best-effort overhead-accuracy tradeoff mechanism, and then extended for data
> access-aware system operations.  I'd like to briefly introduce current state of
> DAMON and share/discuss about 2023 plans including below.
> 
> - Finer and easier-to-use DAMOS tuning
>   - tuning aggressiveness based on user or kernel feed (e.g., QPS or PSI)
> - Merging DAMON user space tool into the mainline
> - Extending DAMON
>   - Page-granularity monitoring
>     - LRU-lists based page-granulariy monitoring
>   - CPU-specific access monitoring
>   - Read/Write-only access monitoring
> - More DAMON-based Operation Schemes
>   - Tiered memory management
>   - THP memory footprint reduction
>   - NUMA balancing
> 
> I hope to hear concerns/interests about the plans for prioritizing each work
> items and get some suggestions of future works and collaboration with other
> kernel subsystems/hackers.

We will start the discussion after about 1 hour and 20 minutes, so sharing the
slides here:
https://drive.google.com/file/d/10nsd4fq01On9p-oTmCS5KSW6qESmB6ER/view?usp=sharing

The link might be changed later.  I will update on this thread in the case.


Thanks,
SJ

> 
> 
> Thanks,
> SJ
> 

Re: [LSF/MM/BPF TOPIC] DAMON Updates and Future Plans

[SeongJae Park]

On Mon, 8 May 2023 21:40:19 +0000 SeongJae Park <sj@kernel.org> wrote:

> Hello,
> 
> On Tue, 14 Feb 2023 00:33:28 +0000 SeongJae Park <sj@kernel.org> wrote:
> 
> > Hi all,
> > 
> > 
> > DAMON has merged into mainline as a data access monitoring tool that equips a
> > best-effort overhead-accuracy tradeoff mechanism, and then extended for data
> > access-aware system operations.  I'd like to briefly introduce current state of
> > DAMON and share/discuss about 2023 plans including below.
> > 
> > - Finer and easier-to-use DAMOS tuning
> >   - tuning aggressiveness based on user or kernel feed (e.g., QPS or PSI)
> > - Merging DAMON user space tool into the mainline
> > - Extending DAMON
> >   - Page-granularity monitoring
> >     - LRU-lists based page-granulariy monitoring
> >   - CPU-specific access monitoring
> >   - Read/Write-only access monitoring
> > - More DAMON-based Operation Schemes
> >   - Tiered memory management
> >   - THP memory footprint reduction
> >   - NUMA balancing
> > 
> > I hope to hear concerns/interests about the plans for prioritizing each work
> > items and get some suggestions of future works and collaboration with other
> > kernel subsystems/hackers.
> 
> We will start the discussion after about 1 hour and 20 minutes, so sharing the
> slides here:
> https://drive.google.com/file/d/10nsd4fq01On9p-oTmCS5KSW6qESmB6ER/view?usp=sharing
> 
> The link might be changed later.  I will update on this thread in the case.

As mentioned above, I indeed moved it to Github[1].  Above Google Drive link
may be expired soon.

[1] https://github.com/damonitor/talks/blob/master/2023/lsfmmbpf/damon_lsfmmbpf_2023.pdf


Thanks,
SJ

> 
> 
> Thanks,
> SJ
> 
> > 
> > 
> > Thanks,
> > SJ
> >